2023-11-06T20:24:31.467 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-06T20:24:31.468 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362 --name vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi --owner scheduled_vshankar@teuthology --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} -- /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362/orig.config.yaml 2023-11-06T20:24:31.513 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-06T20:24:31.603 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362 branch: wip-vshankar-testing1-2023-11-02-1726-pacific description: fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: vshankar@redhat.com first_in_suite: false job_id: '7449362' kernel: client: branch: testing flavor: default kdb: true sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-vshankar-testing1-2023-11-02-1726-pacific ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 4e03306bd00b0abb2be37dbc132f82d281f8973d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4e03306bd00b0abb2be37dbc132f82d281f8973d selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing1-2023-11-02-1726-pacific sha1: 4e03306bd00b0abb2be37dbc132f82d281f8973d owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6875 sha1: 4e03306bd00b0abb2be37dbc132f82d281f8973d sleep_before_teardown: 0 subset: 378/1024 suite: fs suite_branch: wip-vshankar-testing1-2023-11-02-1726-pacific suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_4e03306bd00b0abb2be37dbc132f82d281f8973d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 4e03306bd00b0abb2be37dbc132f82d281f8973d targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBhYb7C6mh7TStE9qnElUtlktEmsqwCqHWi6AuvtIjAc5I+LEPtMn5m6Tc5YouBmPFlV8RWjsDjd/cj8RwnXvFw= smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUKXzZAO/PeanAIVEqBgMAF/dV5YdwYatA66k8kMbvkXopSNomPc8YE8Wu/L9oujrpYY+2GBQZ6hTHfS4+cXPQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-06_07:50:57 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3084822 2023-11-06T20:24:31.604 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_4e03306bd00b0abb2be37dbc132f82d281f8973d/qa; will attempt to use it 2023-11-06T20:24:31.604 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_4e03306bd00b0abb2be37dbc132f82d281f8973d/qa/tasks 2023-11-06T20:24:31.605 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-06T20:24:31.607 INFO:teuthology.task.internal:Checking packages... 2023-11-06T20:24:31.631 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '4e03306bd00b0abb2be37dbc132f82d281f8973d' 2023-11-06T20:24:31.631 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-06T20:24:31.632 INFO:teuthology.packaging:ref: None 2023-11-06T20:24:31.632 INFO:teuthology.packaging:tag: None 2023-11-06T20:24:31.632 INFO:teuthology.packaging:branch: wip-vshankar-testing1-2023-11-02-1726-pacific 2023-11-06T20:24:31.633 INFO:teuthology.packaging:sha1: 4e03306bd00b0abb2be37dbc132f82d281f8973d 2023-11-06T20:24:31.633 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing1-2023-11-02-1726-pacific 2023-11-06T20:24:31.838 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-132.g4e03306b 2023-11-06T20:24:31.838 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-06T20:24:31.846 INFO:teuthology.task.internal:no buildpackages task found 2023-11-06T20:24:31.846 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-06T20:24:31.860 INFO:teuthology.task.internal:Saving configuration 2023-11-06T20:24:31.876 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-06T20:24:31.888 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T20:24:31.915 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 20:19:39.592479', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBhYb7C6mh7TStE9qnElUtlktEmsqwCqHWi6AuvtIjAc5I+LEPtMn5m6Tc5YouBmPFlV8RWjsDjd/cj8RwnXvFw='} 2023-11-06T20:24:31.939 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 20:19:39.590569', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUKXzZAO/PeanAIVEqBgMAF/dV5YdwYatA66k8kMbvkXopSNomPc8YE8Wu/L9oujrpYY+2GBQZ6hTHfS4+cXPQ='} 2023-11-06T20:24:31.940 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-06T20:24:31.971 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-06T20:24:31.972 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-06T20:24:31.972 INFO:teuthology.run_tasks:Running task console_log... 2023-11-06T20:24:32.115 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f150d0e9a60>, signals=[15]) 2023-11-06T20:24:32.117 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-06T20:24:32.128 INFO:teuthology.task.internal:Opening connections... 2023-11-06T20:24:32.128 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:24:32.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T20:24:32.202 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T20:24:32.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T20:24:32.273 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-06T20:24:32.280 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2023-11-06T20:24:32.303 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2023-11-06T20:24:32.304 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2023-11-06T20:24:32.366 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2023-11-06T20:24:32.366 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8" 2023-11-06T20:24:32.367 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2023-11-06T20:24:32.367 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2023-11-06T20:24:32.367 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8" 2023-11-06T20:24:32.367 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8" 2023-11-06T20:24:32.367 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-06T20:24:32.368 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2023-11-06T20:24:32.368 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-06T20:24:32.368 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2023-11-06T20:24:32.368 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-06T20:24:32.368 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-06T20:24:32.369 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-06T20:24:32.369 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2023-11-06T20:24:32.393 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2023-11-06T20:24:32.417 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2023-11-06T20:24:32.418 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2023-11-06T20:24:32.479 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2023-11-06T20:24:32.479 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="8" 2023-11-06T20:24:32.479 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2023-11-06T20:24:32.479 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2023-11-06T20:24:32.479 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="8" 2023-11-06T20:24:32.480 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el8" 2023-11-06T20:24:32.480 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-06T20:24:32.480 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2023-11-06T20:24:32.480 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-06T20:24:32.480 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2023-11-06T20:24:32.481 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-06T20:24:32.481 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-06T20:24:32.481 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-06T20:24:32.481 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2023-11-06T20:24:32.501 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-06T20:24:32.514 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-06T20:24:32.525 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-06T20:24:32.526 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2023-11-06T20:24:32.529 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2023-11-06T20:24:32.549 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-06T20:24:32.558 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-06T20:24:32.558 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2023-11-06T20:24:32.591 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2023-11-06T20:24:32.620 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-06T20:24:32.673 INFO:teuthology.run_tasks:Running task kernel... 2023-11-06T20:24:32.691 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'} 2023-11-06T20:24:32.692 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-11-06T20:24:32.692 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-11-06T20:24:32.693 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-06T20:24:32.694 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-06T20:24:32.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T20:24:32.715 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-06T20:24:32.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T20:24:32.719 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-06T20:24:32.872 DEBUG:teuthology.task.kernel:sha1 for client.1 is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-06T20:24:32.873 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-11-06T20:24:32.873 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2023-11-06T20:24:32.896 INFO:teuthology.orchestra.run.smithi069.stdout:4.18.0-489.el8.x86_64 2023-11-06T20:24:32.896 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-06T20:24:32.897 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-06T20:24:32.898 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum upgrade -y linux-firmware 2023-11-06T20:24:33.059 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-06T20:24:33.060 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-11-06T20:24:33.060 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2023-11-06T20:24:33.082 INFO:teuthology.orchestra.run.smithi033.stdout:4.18.0-489.el8.x86_64 2023-11-06T20:24:33.083 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-06T20:24:33.083 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-06T20:24:33.083 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum upgrade -y linux-firmware 2023-11-06T20:24:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 8 - AppStream 6.0 MB/s | 34 MB 00:05 2023-11-06T20:24:49.124 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - AppStream 2.5 MB/s | 34 MB 00:13 2023-11-06T20:24:53.369 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 8 - BaseOS 8.4 MB/s | 52 MB 00:06 2023-11-06T20:25:00.907 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - BaseOS 10 MB/s | 52 MB 00:05 2023-11-06T20:25:01.002 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 8 - Extras 75 kB/s | 18 kB 00:00 2023-11-06T20:25:01.744 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 8 - Extras common packages 12 kB/s | 6.9 kB 00:00 2023-11-06T20:25:02.029 INFO:teuthology.orchestra.run.smithi069.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-11-06T20:25:04.342 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 7.4 MB/s | 16 MB 00:02 2023-11-06T20:25:08.122 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 456 kB/s | 24 kB 00:00 2023-11-06T20:25:11.913 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2023-11-06T20:25:11.914 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2023-11-06T20:25:11.914 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repo Size 2023-11-06T20:25:11.914 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2023-11-06T20:25:11.914 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2023-11-06T20:25:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: linux-firmware noarch 20230824-118.git0e048b06.el8 baseos 286 M 2023-11-06T20:25:11.915 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:25:11.915 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2023-11-06T20:25:11.915 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2023-11-06T20:25:11.915 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 1 Package 2023-11-06T20:25:11.916 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:25:11.916 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 286 M 2023-11-06T20:25:11.916 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2023-11-06T20:25:12.473 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - Extras 5.0 kB/s | 18 kB 00:03 2023-11-06T20:25:13.107 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - Extras common packages 16 kB/s | 6.9 kB 00:00 2023-11-06T20:25:13.408 INFO:teuthology.orchestra.run.smithi033.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-11-06T20:25:15.378 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 16 MB 00:01 2023-11-06T20:25:19.459 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 334 kB/s | 24 kB 00:00 2023-11-06T20:25:23.597 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2023-11-06T20:25:23.598 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2023-11-06T20:25:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repo Size 2023-11-06T20:25:23.598 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2023-11-06T20:25:23.599 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2023-11-06T20:25:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: linux-firmware noarch 20230824-118.git0e048b06.el8 baseos 286 M 2023-11-06T20:25:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:25:23.599 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2023-11-06T20:25:23.599 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2023-11-06T20:25:23.600 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 1 Package 2023-11-06T20:25:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:25:23.600 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 286 M 2023-11-06T20:25:23.601 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2023-11-06T20:25:23.613 INFO:teuthology.orchestra.run.smithi069.stdout:linux-firmware-20230824-118.git0e048b06.el8.noa 25 MB/s | 286 MB 00:11 2023-11-06T20:25:23.614 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2023-11-06T20:25:23.614 INFO:teuthology.orchestra.run.smithi069.stdout:Total 24 MB/s | 286 MB 00:11 2023-11-06T20:25:25.317 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2023-11-06T20:25:25.371 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2023-11-06T20:25:25.372 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2023-11-06T20:25:27.139 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2023-11-06T20:25:27.159 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2023-11-06T20:25:39.680 INFO:teuthology.orchestra.run.smithi033.stdout:linux-firmware-20230824-118.git0e048b06.el8.noa 18 MB/s | 286 MB 00:16 2023-11-06T20:25:39.680 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2023-11-06T20:25:39.680 INFO:teuthology.orchestra.run.smithi033.stdout:Total 18 MB/s | 286 MB 00:16 2023-11-06T20:25:41.469 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2023-11-06T20:25:41.547 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2023-11-06T20:25:41.547 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2023-11-06T20:25:43.422 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2023-11-06T20:25:43.450 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2023-11-06T20:25:45.961 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2023-11-06T20:25:46.311 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-11-06T20:25:46.531 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-06T20:25:46.767 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-06T20:25:46.767 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-11-06T20:25:47.528 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-06T20:25:47.529 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:25:47.529 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2023-11-06T20:25:47.529 INFO:teuthology.orchestra.run.smithi069.stdout: linux-firmware-20230824-118.git0e048b06.el8.noarch 2023-11-06T20:25:47.530 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:25:47.530 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2023-11-06T20:25:47.697 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.1... 2023-11-06T20:25:47.698 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-06T20:25:47.912 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/ 2023-11-06T20:25:47.912 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-06T20:25:47.915 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi069.front.sepia.ceph.com to complete... 2023-11-06T20:26:01.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06 20:26:01 URL:https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [383129572/383129572] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-06T20:26:01.218 INFO:teuthology.orchestra.run.smithi069.stderr:FINISHED --2023-11-06 20:26:01-- 2023-11-06T20:26:01.218 INFO:teuthology.orchestra.run.smithi069.stderr:Total wall clock time: 13s 2023-11-06T20:26:01.218 INFO:teuthology.orchestra.run.smithi069.stderr:Downloaded: 1 files, 365M in 13s (28.3 MB/s) 2023-11-06T20:26:01.220 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.1... 2023-11-06T20:26:01.220 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-06T20:26:02.881 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying... ######################################## 2023-11-06T20:26:02.925 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing... ######################################## 2023-11-06T20:26:03.034 INFO:teuthology.orchestra.run.smithi069.stdout:Updating / installing... 2023-11-06T20:26:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2023-11-06T20:26:04.099 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-11-06T20:26:04.487 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-06T20:26:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-06T20:26:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-11-06T20:26:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-06T20:26:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:26:05.709 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2023-11-06T20:26:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: linux-firmware-20230824-118.git0e048b06.el8.noarch 2023-11-06T20:26:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:26:05.710 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2023-11-06T20:26:05.874 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-11-06T20:26:05.874 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-06T20:26:06.100 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/ 2023-11-06T20:26:06.100 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-06T20:26:06.103 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi033.front.sepia.ceph.com to complete... 2023-11-06T20:26:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06 20:26:19 URL:https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [383129572/383129572] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-06T20:26:19.043 INFO:teuthology.orchestra.run.smithi033.stderr:FINISHED --2023-11-06 20:26:19-- 2023-11-06T20:26:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:Total wall clock time: 13s 2023-11-06T20:26:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:Downloaded: 1 files, 365M in 13s (28.9 MB/s) 2023-11-06T20:26:19.045 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-11-06T20:26:19.045 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-06T20:26:20.840 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying... ######################################## 2023-11-06T20:26:20.886 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing... ######################################## 2023-11-06T20:26:21.002 INFO:teuthology.orchestra.run.smithi033.stdout:Updating / installing... 2023-11-06T20:26:41.227 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-11-06T20:26:42.590 INFO:teuthology.orchestra.run.smithi069.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-06T20:27:02.021 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-11-06T20:27:03.472 INFO:teuthology.orchestra.run.smithi033.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-06T20:27:05.743 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-06T20:27:05.743 DEBUG:teuthology.orchestra.run.smithi069:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-06T20:27:05.803 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:05.803 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:05.804 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:05.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:05.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-11-06T20:27:05.804 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-11-06T20:27:05.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-11-06T20:27:05.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-11-06T20:27:05.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-06T20:27:05.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-06T20:27:05.805 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-06T20:27:05.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-06T20:27:05.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-06T20:27:05.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-06T20:27:05.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-06T20:27:05.806 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-06T20:27:05.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-06T20:27:05.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-06T20:27:05.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-06T20:27:05.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-06T20:27:05.807 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-06T20:27:05.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-06T20:27:05.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-06T20:27:05.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-06T20:27:05.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-06T20:27:05.808 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-06T20:27:05.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-06T20:27:05.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-06T20:27:05.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-11-06T20:27:05.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-11-06T20:27:05.809 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-11-06T20:27:05.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-06T20:27:05.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-11-06T20:27:05.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-06T20:27:05.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-06T20:27:05.810 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-11-06T20:27:05.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-11-06T20:27:05.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-11-06T20:27:05.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-11-06T20:27:05.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-11-06T20:27:05.811 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-11-06T20:27:05.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-11-06T20:27:05.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-11-06T20:27:05.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-06T20:27:05.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-11-06T20:27:05.812 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-11-06T20:27:05.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-11-06T20:27:05.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-11-06T20:27:05.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-11-06T20:27:05.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-11-06T20:27:05.813 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-11-06T20:27:05.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-11-06T20:27:05.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-11-06T20:27:05.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-11-06T20:27:05.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-11-06T20:27:05.814 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-11-06T20:27:05.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-11-06T20:27:05.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-11-06T20:27:05.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-11-06T20:27:05.815 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-11-06T20:27:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-11-06T20:27:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-11-06T20:27:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-11-06T20:27:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-11-06T20:27:05.816 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-11-06T20:27:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-11-06T20:27:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-11-06T20:27:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-11-06T20:27:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-11-06T20:27:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-11-06T20:27:05.817 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-11-06T20:27:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-11-06T20:27:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-11-06T20:27:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-11-06T20:27:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-11-06T20:27:05.818 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-11-06T20:27:05.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-11-06T20:27:05.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-11-06T20:27:05.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-11-06T20:27:05.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-11-06T20:27:05.819 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-11-06T20:27:05.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-11-06T20:27:05.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-11-06T20:27:05.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-11-06T20:27:05.820 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-11-06T20:27:05.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-11-06T20:27:05.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-11-06T20:27:05.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-11-06T20:27:05.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-06T20:27:05.821 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-11-06T20:27:05.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-11-06T20:27:05.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-11-06T20:27:05.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-11-06T20:27:05.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-11-06T20:27:05.822 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-11-06T20:27:05.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-11-06T20:27:05.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-11-06T20:27:05.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-11-06T20:27:05.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-11-06T20:27:05.823 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-11-06T20:27:05.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-11-06T20:27:05.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-11-06T20:27:05.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-11-06T20:27:05.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-11-06T20:27:05.824 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-11-06T20:27:05.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-11-06T20:27:05.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-11-06T20:27:05.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-11-06T20:27:05.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-11-06T20:27:05.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-11-06T20:27:05.825 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-11-06T20:27:05.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-11-06T20:27:05.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-11-06T20:27:05.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-06T20:27:05.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-06T20:27:05.826 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-11-06T20:27:05.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-11-06T20:27:05.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-11-06T20:27:05.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-11-06T20:27:05.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-11-06T20:27:05.827 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-11-06T20:27:05.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-11-06T20:27:05.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-11-06T20:27:05.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-11-06T20:27:05.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-11-06T20:27:05.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-11-06T20:27:05.828 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-11-06T20:27:05.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-11-06T20:27:05.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-06T20:27:05.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-06T20:27:05.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-11-06T20:27:05.829 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-11-06T20:27:05.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-11-06T20:27:05.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-11-06T20:27:05.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-11-06T20:27:05.830 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-11-06T20:27:05.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-11-06T20:27:05.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-11-06T20:27:05.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-11-06T20:27:05.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-11-06T20:27:05.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-11-06T20:27:05.831 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-11-06T20:27:05.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-11-06T20:27:05.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-11-06T20:27:05.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-11-06T20:27:05.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-11-06T20:27:05.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-11-06T20:27:05.832 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-11-06T20:27:05.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-11-06T20:27:05.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-11-06T20:27:05.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-11-06T20:27:05.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-11-06T20:27:05.833 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-11-06T20:27:05.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-11-06T20:27:05.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-11-06T20:27:05.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-11-06T20:27:05.834 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-11-06T20:27:05.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-06T20:27:05.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-06T20:27:05.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-11-06T20:27:05.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-11-06T20:27:05.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-11-06T20:27:05.835 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-11-06T20:27:05.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-11-06T20:27:05.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-11-06T20:27:05.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-11-06T20:27:05.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-11-06T20:27:05.836 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-11-06T20:27:05.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-11-06T20:27:05.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-11-06T20:27:05.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-06T20:27:05.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-11-06T20:27:05.837 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-11-06T20:27:05.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-11-06T20:27:05.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-11-06T20:27:05.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-11-06T20:27:05.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-11-06T20:27:05.838 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-11-06T20:27:05.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-06T20:27:05.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-11-06T20:27:05.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-11-06T20:27:05.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-11-06T20:27:05.839 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-11-06T20:27:05.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-11-06T20:27:05.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-11-06T20:27:05.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-06T20:27:05.840 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-06T20:27:05.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-06T20:27:05.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-11-06T20:27:05.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-06T20:27:05.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-11-06T20:27:05.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-06T20:27:05.841 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-06T20:27:05.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-06T20:27:05.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-06T20:27:05.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-06T20:27:05.842 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-06T20:27:05.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-11-06T20:27:05.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-11-06T20:27:05.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-11-06T20:27:05.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-11-06T20:27:05.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-11-06T20:27:05.843 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-11-06T20:27:05.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-11-06T20:27:05.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-06T20:27:05.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-06T20:27:05.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-06T20:27:05.844 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-06T20:27:05.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-06T20:27:05.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-11-06T20:27:05.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-11-06T20:27:05.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-11-06T20:27:05.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-06T20:27:05.845 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-06T20:27:05.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-06T20:27:05.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-06T20:27:05.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-06T20:27:05.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-06T20:27:05.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-11-06T20:27:05.846 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-11-06T20:27:05.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-06T20:27:05.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-11-06T20:27:05.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-11-06T20:27:05.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-11-06T20:27:05.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-11-06T20:27:05.847 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-06T20:27:05.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-06T20:27:05.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-06T20:27:05.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-11-06T20:27:05.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-11-06T20:27:05.848 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-11-06T20:27:05.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-06T20:27:05.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-11-06T20:27:05.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-11-06T20:27:05.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-11-06T20:27:05.849 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-06T20:27:05.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-11-06T20:27:05.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-11-06T20:27:05.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-06T20:27:05.850 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-11-06T20:27:05.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-06T20:27:05.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-11-06T20:27:05.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-11-06T20:27:05.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-11-06T20:27:05.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-11-06T20:27:05.851 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-11-06T20:27:05.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-11-06T20:27:05.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-11-06T20:27:05.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-11-06T20:27:05.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-11-06T20:27:05.852 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-11-06T20:27:05.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-11-06T20:27:05.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-11-06T20:27:05.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-11-06T20:27:05.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-11-06T20:27:05.853 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-11-06T20:27:05.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-11-06T20:27:05.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-06T20:27:05.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-11-06T20:27:05.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-11-06T20:27:05.854 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-06T20:27:05.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-11-06T20:27:05.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-11-06T20:27:05.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-11-06T20:27:05.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-11-06T20:27:05.855 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-11-06T20:27:05.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-11-06T20:27:05.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-11-06T20:27:05.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-11-06T20:27:05.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-11-06T20:27:05.856 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-11-06T20:27:05.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-11-06T20:27:05.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-06T20:27:05.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-06T20:27:05.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-11-06T20:27:05.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-11-06T20:27:05.857 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-11-06T20:27:05.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-11-06T20:27:05.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-11-06T20:27:05.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-11-06T20:27:05.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-11-06T20:27:05.858 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-11-06T20:27:05.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-06T20:27:05.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-06T20:27:05.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-11-06T20:27:05.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-11-06T20:27:05.859 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-11-06T20:27:05.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-11-06T20:27:05.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-11-06T20:27:05.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-11-06T20:27:05.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-11-06T20:27:05.860 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-11-06T20:27:05.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-11-06T20:27:05.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-06T20:27:05.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-11-06T20:27:05.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-06T20:27:05.861 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-06T20:27:05.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-06T20:27:05.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-06T20:27:05.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-06T20:27:05.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-06T20:27:05.862 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-06T20:27:05.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-06T20:27:05.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-06T20:27:05.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-11-06T20:27:05.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-11-06T20:27:05.863 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-11-06T20:27:05.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-06T20:27:05.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-11-06T20:27:05.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-11-06T20:27:05.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-11-06T20:27:05.864 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-11-06T20:27:05.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-11-06T20:27:05.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-06T20:27:05.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-11-06T20:27:05.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-11-06T20:27:05.865 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-11-06T20:27:05.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-11-06T20:27:05.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-11-06T20:27:05.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-11-06T20:27:05.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-11-06T20:27:05.866 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-11-06T20:27:05.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-11-06T20:27:05.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-11-06T20:27:05.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-11-06T20:27:05.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-11-06T20:27:05.867 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-11-06T20:27:05.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-11-06T20:27:05.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-11-06T20:27:05.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-11-06T20:27:05.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-11-06T20:27:05.868 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-11-06T20:27:05.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-11-06T20:27:05.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-06T20:27:05.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-11-06T20:27:05.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-11-06T20:27:05.869 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-11-06T20:27:05.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-11-06T20:27:05.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-11-06T20:27:05.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-11-06T20:27:05.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-11-06T20:27:05.870 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-11-06T20:27:05.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-11-06T20:27:05.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-11-06T20:27:05.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-11-06T20:27:05.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-11-06T20:27:05.871 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-11-06T20:27:05.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-11-06T20:27:05.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-11-06T20:27:05.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-11-06T20:27:05.872 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-11-06T20:27:05.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-11-06T20:27:05.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-11-06T20:27:05.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-06T20:27:05.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-11-06T20:27:05.873 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-11-06T20:27:05.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-06T20:27:05.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-11-06T20:27:05.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-11-06T20:27:05.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-11-06T20:27:05.874 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-11-06T20:27:05.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-11-06T20:27:05.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-11-06T20:27:05.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-11-06T20:27:05.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-11-06T20:27:05.875 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-11-06T20:27:05.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-11-06T20:27:05.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-11-06T20:27:05.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-11-06T20:27:05.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-11-06T20:27:05.876 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-11-06T20:27:05.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-06T20:27:05.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-11-06T20:27:05.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-11-06T20:27:05.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-11-06T20:27:05.877 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-11-06T20:27:05.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-11-06T20:27:05.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-11-06T20:27:05.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-11-06T20:27:05.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-11-06T20:27:05.878 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-11-06T20:27:05.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-11-06T20:27:05.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-11-06T20:27:05.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-11-06T20:27:05.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-11-06T20:27:05.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-11-06T20:27:05.879 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-11-06T20:27:05.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-11-06T20:27:05.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-11-06T20:27:05.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-11-06T20:27:05.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-11-06T20:27:05.880 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-11-06T20:27:05.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-11-06T20:27:05.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-11-06T20:27:05.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-11-06T20:27:05.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-11-06T20:27:05.881 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-11-06T20:27:05.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-11-06T20:27:05.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-11-06T20:27:05.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-11-06T20:27:05.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-11-06T20:27:05.882 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-11-06T20:27:05.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-11-06T20:27:05.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-11-06T20:27:05.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-11-06T20:27:05.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-11-06T20:27:05.883 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-11-06T20:27:05.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-11-06T20:27:05.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-11-06T20:27:05.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-11-06T20:27:05.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-11-06T20:27:05.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-11-06T20:27:05.884 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-11-06T20:27:05.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-11-06T20:27:05.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-11-06T20:27:05.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-11-06T20:27:05.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-11-06T20:27:05.885 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-11-06T20:27:05.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-11-06T20:27:05.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-11-06T20:27:05.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-06T20:27:05.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-06T20:27:05.886 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-06T20:27:05.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-06T20:27:05.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-06T20:27:05.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-06T20:27:05.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-06T20:27:05.887 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-06T20:27:05.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-06T20:27:05.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-06T20:27:05.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-06T20:27:05.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-06T20:27:05.888 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-11-06T20:27:05.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-11-06T20:27:05.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-11-06T20:27:05.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-06T20:27:05.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-11-06T20:27:05.889 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-11-06T20:27:05.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-11-06T20:27:05.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-11-06T20:27:05.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-11-06T20:27:05.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-11-06T20:27:05.890 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-11-06T20:27:05.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-11-06T20:27:05.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-11-06T20:27:05.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-11-06T20:27:05.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-11-06T20:27:05.891 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-11-06T20:27:05.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-11-06T20:27:05.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-11-06T20:27:05.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-11-06T20:27:05.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-11-06T20:27:05.892 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-11-06T20:27:05.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-11-06T20:27:05.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-11-06T20:27:05.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-11-06T20:27:05.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-11-06T20:27:05.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-11-06T20:27:05.893 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-11-06T20:27:05.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-11-06T20:27:05.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-11-06T20:27:05.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-11-06T20:27:05.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-11-06T20:27:05.894 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-06T20:27:05.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-06T20:27:05.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-11-06T20:27:05.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-06T20:27:05.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-06T20:27:05.895 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-06T20:27:05.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-06T20:27:05.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-06T20:27:05.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-06T20:27:05.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-06T20:27:05.896 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-06T20:27:05.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-06T20:27:05.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-06T20:27:05.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-06T20:27:05.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-06T20:27:05.897 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-06T20:27:05.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-06T20:27:05.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-06T20:27:05.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-06T20:27:05.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-06T20:27:05.898 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-06T20:27:05.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-06T20:27:05.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-06T20:27:05.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-06T20:27:05.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-06T20:27:05.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-06T20:27:05.899 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-11-06T20:27:05.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-11-06T20:27:05.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-06T20:27:05.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-11-06T20:27:05.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-11-06T20:27:05.900 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-11-06T20:27:05.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-11-06T20:27:05.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-06T20:27:05.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-11-06T20:27:05.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-06T20:27:05.901 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-06T20:27:05.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-06T20:27:05.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-06T20:27:05.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-06T20:27:05.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-11-06T20:27:05.902 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-11-06T20:27:05.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-06T20:27:05.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-11-06T20:27:05.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-06T20:27:05.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-11-06T20:27:05.903 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-06T20:27:05.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-11-06T20:27:05.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-06T20:27:05.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-11-06T20:27:05.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-11-06T20:27:05.904 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-06T20:27:05.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-11-06T20:27:05.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-06T20:27:05.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-11-06T20:27:05.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-06T20:27:05.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-11-06T20:27:05.905 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-06T20:27:05.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-11-06T20:27:05.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-06T20:27:05.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-11-06T20:27:05.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-11-06T20:27:05.906 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-06T20:27:05.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-06T20:27:05.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-11-06T20:27:05.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-11-06T20:27:05.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-11-06T20:27:05.907 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-06T20:27:05.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-11-06T20:27:05.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-11-06T20:27:05.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-11-06T20:27:05.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-11-06T20:27:05.908 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-06T20:27:05.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-11-06T20:27:05.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-06T20:27:05.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-06T20:27:05.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-06T20:27:05.909 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-11-06T20:27:05.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-06T20:27:05.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-11-06T20:27:05.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-11-06T20:27:05.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-11-06T20:27:05.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-06T20:27:05.910 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-11-06T20:27:05.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-11-06T20:27:05.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-11-06T20:27:05.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-11-06T20:27:05.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-06T20:27:05.911 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-11-06T20:27:05.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-11-06T20:27:05.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-11-06T20:27:05.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-11-06T20:27:05.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-11-06T20:27:05.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-11-06T20:27:05.912 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-06T20:27:05.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-06T20:27:05.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-06T20:27:05.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-06T20:27:05.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-06T20:27:05.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-06T20:27:05.913 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-06T20:27:05.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-06T20:27:05.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-11-06T20:27:05.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-11-06T20:27:05.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-11-06T20:27:05.914 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-11-06T20:27:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-11-06T20:27:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-11-06T20:27:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-11-06T20:27:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-11-06T20:27:05.915 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-11-06T20:27:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-11-06T20:27:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-11-06T20:27:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-11-06T20:27:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-11-06T20:27:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-11-06T20:27:05.916 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-11-06T20:27:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-11-06T20:27:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-11-06T20:27:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-11-06T20:27:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-11-06T20:27:05.917 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-11-06T20:27:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-11-06T20:27:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-11-06T20:27:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-06T20:27:05.918 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-11-06T20:27:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-11-06T20:27:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-11-06T20:27:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-11-06T20:27:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-11-06T20:27:05.919 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-11-06T20:27:05.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-11-06T20:27:05.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-11-06T20:27:05.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-11-06T20:27:05.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-11-06T20:27:05.920 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-11-06T20:27:05.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-11-06T20:27:05.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-11-06T20:27:05.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-11-06T20:27:05.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-11-06T20:27:05.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-11-06T20:27:05.921 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-11-06T20:27:05.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-11-06T20:27:05.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-11-06T20:27:05.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-11-06T20:27:05.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-11-06T20:27:05.922 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-11-06T20:27:05.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-11-06T20:27:05.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-06T20:27:05.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-11-06T20:27:05.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-11-06T20:27:05.923 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-11-06T20:27:05.924 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-11-06T20:27:05.924 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-11-06T20:27:05.924 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-11-06T20:27:05.924 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-11-06T20:27:05.924 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-06T20:27:05.925 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-06T20:27:05.925 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-11-06T20:27:05.925 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-11-06T20:27:05.925 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-11-06T20:27:05.925 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-11-06T20:27:05.926 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-11-06T20:27:05.926 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-11-06T20:27:05.926 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-06T20:27:05.926 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-06T20:27:05.926 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-11-06T20:27:05.927 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-11-06T20:27:05.927 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-11-06T20:27:05.927 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-11-06T20:27:05.927 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-11-06T20:27:05.927 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-06T20:27:05.928 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-11-06T20:27:05.928 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-06T20:27:05.928 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-06T20:27:05.928 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-06T20:27:05.928 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-11-06T20:27:05.929 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-11-06T20:27:05.929 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-11-06T20:27:05.929 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-11-06T20:27:05.929 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-11-06T20:27:05.930 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-06T20:27:05.930 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-11-06T20:27:05.930 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-11-06T20:27:05.930 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-11-06T20:27:05.930 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-11-06T20:27:05.931 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-11-06T20:27:05.931 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-06T20:27:05.931 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-06T20:27:05.931 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-06T20:27:05.931 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-11-06T20:27:05.932 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-06T20:27:05.932 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-11-06T20:27:05.932 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-11-06T20:27:05.932 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-11-06T20:27:05.932 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-11-06T20:27:05.933 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-06T20:27:05.933 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-06T20:27:05.933 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-06T20:27:05.933 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-06T20:27:05.933 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-11-06T20:27:05.934 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-06T20:27:05.934 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-06T20:27:05.934 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-06T20:27:05.934 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-06T20:27:05.934 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-11-06T20:27:05.935 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-06T20:27:05.935 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-11-06T20:27:05.935 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-06T20:27:05.935 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-11-06T20:27:05.935 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-06T20:27:05.936 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-11-06T20:27:05.936 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-06T20:27:05.936 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-11-06T20:27:05.936 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-06T20:27:05.937 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-06T20:27:05.937 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-06T20:27:05.937 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-11-06T20:27:05.937 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-11-06T20:27:05.937 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-06T20:27:05.937 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-06T20:27:05.938 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-06T20:27:05.938 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-06T20:27:05.938 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-06T20:27:05.938 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-11-06T20:27:05.939 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-06T20:27:05.939 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-06T20:27:05.939 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-11-06T20:27:05.939 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-06T20:27:05.939 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-06T20:27:05.940 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-06T20:27:05.940 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-06T20:27:05.940 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-06T20:27:05.940 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-11-06T20:27:05.940 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-06T20:27:05.940 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-06T20:27:05.941 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-11-06T20:27:05.941 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-11-06T20:27:05.941 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-06T20:27:05.941 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-11-06T20:27:05.941 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-06T20:27:05.942 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-06T20:27:05.942 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-06T20:27:05.942 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-06T20:27:05.942 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-06T20:27:05.942 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-06T20:27:05.943 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-06T20:27:05.943 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-06T20:27:05.943 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-06T20:27:05.943 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-06T20:27:05.943 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-11-06T20:27:05.944 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-11-06T20:27:05.944 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-06T20:27:05.944 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-11-06T20:27:05.944 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-11-06T20:27:05.944 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-06T20:27:05.944 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-06T20:27:05.945 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-06T20:27:05.945 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-06T20:27:05.945 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-06T20:27:05.945 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-06T20:27:05.945 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-11-06T20:27:05.946 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-06T20:27:05.946 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-06T20:27:05.946 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-11-06T20:27:05.946 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-11-06T20:27:05.946 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-11-06T20:27:05.947 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-06T20:27:05.947 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-11-06T20:27:05.947 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-11-06T20:27:05.947 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-11-06T20:27:05.947 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-06T20:27:05.948 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-11-06T20:27:05.948 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-06T20:27:05.948 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-11-06T20:27:05.948 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-06T20:27:05.948 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-11-06T20:27:05.948 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-06T20:27:05.949 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-11-06T20:27:05.949 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-06T20:27:05.949 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-11-06T20:27:05.949 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-06T20:27:05.949 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-11-06T20:27:05.950 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-06T20:27:05.950 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-11-06T20:27:05.950 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-06T20:27:05.950 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-06T20:27:05.950 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-06T20:27:05.951 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-06T20:27:05.951 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-06T20:27:05.951 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-11-06T20:27:05.951 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-11-06T20:27:05.951 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-06T20:27:05.951 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-06T20:27:05.952 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-06T20:27:05.952 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-11-06T20:27:05.952 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-06T20:27:05.952 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-06T20:27:05.952 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-06T20:27:05.953 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-06T20:27:05.953 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-06T20:27:05.953 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-06T20:27:05.953 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-11-06T20:27:05.953 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-06T20:27:05.954 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-11-06T20:27:05.954 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-06T20:27:05.954 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-06T20:27:05.954 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-11-06T20:27:05.954 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-06T20:27:05.955 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-06T20:27:05.955 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-11-06T20:27:05.955 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-06T20:27:05.955 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-11-06T20:27:05.955 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-06T20:27:05.956 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-11-06T20:27:05.956 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-06T20:27:05.956 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-06T20:27:05.956 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-11-06T20:27:05.956 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-06T20:27:05.957 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-06T20:27:05.957 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-06T20:27:05.957 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-06T20:27:05.957 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-06T20:27:05.957 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-11-06T20:27:05.958 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-06T20:27:05.958 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-11-06T20:27:05.958 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-06T20:27:05.958 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-06T20:27:05.958 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-06T20:27:05.958 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-06T20:27:05.959 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-11-06T20:27:05.959 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-06T20:27:05.959 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-11-06T20:27:05.959 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-06T20:27:05.960 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-11-06T20:27:05.960 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-06T20:27:05.960 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-11-06T20:27:05.960 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-06T20:27:05.960 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-06T20:27:05.961 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-11-06T20:27:05.961 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-06T20:27:05.961 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-06T20:27:05.961 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-06T20:27:05.961 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-06T20:27:05.962 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-11-06T20:27:05.962 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-06T20:27:05.962 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-06T20:27:05.962 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-06T20:27:05.962 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-11-06T20:27:05.963 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-06T20:27:05.963 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-06T20:27:05.963 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-06T20:27:05.963 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-06T20:27:05.963 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-11-06T20:27:05.964 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-06T20:27:05.964 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-11-06T20:27:05.964 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-06T20:27:05.964 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-11-06T20:27:05.964 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-06T20:27:05.965 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-06T20:27:05.965 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-11-06T20:27:05.965 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-06T20:27:05.965 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-06T20:27:05.965 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-11-06T20:27:05.965 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-06T20:27:05.966 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-11-06T20:27:05.966 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-11-06T20:27:05.966 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-06T20:27:05.966 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-11-06T20:27:05.966 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-06T20:27:05.967 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-11-06T20:27:05.967 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-11-06T20:27:05.967 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-06T20:27:05.967 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-11-06T20:27:05.967 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-11-06T20:27:05.968 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-11-06T20:27:05.968 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-11-06T20:27:05.968 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-06T20:27:05.968 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-06T20:27:05.968 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-06T20:27:05.969 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-11-06T20:27:05.969 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-11-06T20:27:05.969 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-11-06T20:27:05.969 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-11-06T20:27:05.969 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-11-06T20:27:05.969 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-06T20:27:05.970 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-06T20:27:05.970 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-11-06T20:27:05.970 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-11-06T20:27:05.970 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-06T20:27:05.970 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-06T20:27:05.971 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-06T20:27:05.971 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-11-06T20:27:05.971 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-11-06T20:27:05.971 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-11-06T20:27:05.971 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-11-06T20:27:05.972 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-11-06T20:27:05.972 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-11-06T20:27:05.972 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-11-06T20:27:05.972 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-11-06T20:27:05.972 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-11-06T20:27:05.973 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-11-06T20:27:05.973 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-11-06T20:27:05.973 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-11-06T20:27:05.973 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-11-06T20:27:05.973 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-11-06T20:27:05.974 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-11-06T20:27:05.974 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-11-06T20:27:05.974 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-11-06T20:27:05.974 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-11-06T20:27:05.974 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-06T20:27:05.975 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-11-06T20:27:05.975 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-06T20:27:05.975 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-06T20:27:05.975 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-06T20:27:05.975 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-06T20:27:05.976 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-11-06T20:27:05.976 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-11-06T20:27:05.976 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-11-06T20:27:05.976 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-11-06T20:27:05.976 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-11-06T20:27:05.977 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-11-06T20:27:05.977 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-11-06T20:27:05.977 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-11-06T20:27:05.977 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-11-06T20:27:05.977 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-11-06T20:27:05.978 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-06T20:27:05.978 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-06T20:27:05.978 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-06T20:27:05.978 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-11-06T20:27:05.978 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-06T20:27:05.979 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-11-06T20:27:05.979 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-11-06T20:27:05.979 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-11-06T20:27:05.979 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-11-06T20:27:05.979 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-06T20:27:05.979 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-11-06T20:27:05.980 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-11-06T20:27:05.980 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-11-06T20:27:05.980 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-06T20:27:05.980 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-11-06T20:27:05.980 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-11-06T20:27:05.981 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-11-06T20:27:05.981 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-11-06T20:27:05.981 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-11-06T20:27:05.981 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-11-06T20:27:05.981 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-11-06T20:27:05.981 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-11-06T20:27:05.982 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-11-06T20:27:05.982 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-11-06T20:27:05.982 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-11-06T20:27:05.982 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-11-06T20:27:05.982 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-11-06T20:27:05.982 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-11-06T20:27:05.983 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-06T20:27:05.983 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-11-06T20:27:05.983 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-06T20:27:05.983 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-11-06T20:27:05.983 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-11-06T20:27:05.983 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-11-06T20:27:05.984 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-11-06T20:27:05.984 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-06T20:27:05.984 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-11-06T20:27:05.984 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-11-06T20:27:05.984 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-06T20:27:05.984 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-11-06T20:27:05.985 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-11-06T20:27:05.985 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-06T20:27:05.985 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-11-06T20:27:05.985 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-06T20:27:05.985 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-06T20:27:05.986 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-06T20:27:05.986 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-06T20:27:05.986 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-06T20:27:05.986 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-06T20:27:05.986 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-06T20:27:05.986 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-06T20:27:05.987 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-06T20:27:05.987 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-11-06T20:27:05.987 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-11-06T20:27:05.987 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-06T20:27:05.987 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-11-06T20:27:05.987 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-06T20:27:05.988 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-06T20:27:05.988 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-06T20:27:05.988 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-06T20:27:05.988 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-06T20:27:05.988 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-06T20:27:05.989 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-06T20:27:05.989 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-06T20:27:05.989 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-06T20:27:05.989 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-11-06T20:27:05.989 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-11-06T20:27:05.990 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-06T20:27:05.990 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-06T20:27:05.990 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-11-06T20:27:05.990 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-11-06T20:27:05.990 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-06T20:27:05.990 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-06T20:27:05.991 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-11-06T20:27:05.991 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-11-06T20:27:05.991 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-11-06T20:27:05.991 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-11-06T20:27:05.991 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-11-06T20:27:05.992 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-11-06T20:27:05.992 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-06T20:27:05.992 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-06T20:27:05.992 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-06T20:27:05.992 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-06T20:27:05.993 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-06T20:27:05.993 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-06T20:27:05.993 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-06T20:27:05.993 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-06T20:27:05.993 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-06T20:27:05.994 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-06T20:27:05.994 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-11-06T20:27:05.994 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-06T20:27:05.994 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-06T20:27:05.994 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-06T20:27:05.995 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-06T20:27:05.995 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-06T20:27:05.995 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-06T20:27:05.995 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-11-06T20:27:05.995 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-06T20:27:05.996 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-06T20:27:05.996 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-06T20:27:05.996 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-06T20:27:05.996 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-06T20:27:05.996 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-06T20:27:05.997 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-06T20:27:05.997 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-06T20:27:05.997 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-06T20:27:05.997 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-06T20:27:05.997 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-06T20:27:05.997 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-06T20:27:05.998 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-11-06T20:27:05.998 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-11-06T20:27:05.998 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-11-06T20:27:05.998 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-11-06T20:27:05.998 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-11-06T20:27:05.999 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-11-06T20:27:05.999 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-11-06T20:27:05.999 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-11-06T20:27:05.999 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-06T20:27:05.999 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-11-06T20:27:06.000 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-11-06T20:27:06.000 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-06T20:27:06.000 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-06T20:27:06.000 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-11-06T20:27:06.000 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-11-06T20:27:06.001 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-06T20:27:06.001 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-11-06T20:27:06.001 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-11-06T20:27:06.001 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-06T20:27:06.001 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-11-06T20:27:06.001 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-11-06T20:27:06.002 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-11-06T20:27:06.002 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-06T20:27:06.002 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-11-06T20:27:06.002 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-06T20:27:06.002 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-11-06T20:27:06.003 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-11-06T20:27:06.003 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-06T20:27:06.003 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-11-06T20:27:06.003 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-06T20:27:06.003 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-06T20:27:06.003 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-06T20:27:06.004 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-06T20:27:06.004 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-06T20:27:06.004 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-11-06T20:27:06.004 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-11-06T20:27:06.004 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-06T20:27:06.005 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-06T20:27:06.005 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-06T20:27:06.005 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-06T20:27:06.005 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-11-06T20:27:06.005 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-11-06T20:27:06.006 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-06T20:27:06.006 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-11-06T20:27:06.006 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-11-06T20:27:06.006 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-06T20:27:06.006 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-06T20:27:06.007 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-11-06T20:27:06.007 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-11-06T20:27:06.007 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-11-06T20:27:06.007 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-11-06T20:27:06.007 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-11-06T20:27:06.007 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-11-06T20:27:06.008 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-11-06T20:27:06.008 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-11-06T20:27:06.008 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-11-06T20:27:06.008 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-11-06T20:27:06.008 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-06T20:27:06.009 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-06T20:27:06.009 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-11-06T20:27:06.009 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-11-06T20:27:06.009 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-11-06T20:27:06.009 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-06T20:27:06.010 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-11-06T20:27:06.010 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-11-06T20:27:06.010 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-11-06T20:27:06.010 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-06T20:27:06.010 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-11-06T20:27:06.011 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-11-06T20:27:06.011 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-06T20:27:06.011 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-06T20:27:06.011 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-06T20:27:06.011 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-11-06T20:27:06.011 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-06T20:27:06.012 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-11-06T20:27:06.012 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-06T20:27:06.012 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-11-06T20:27:06.012 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-11-06T20:27:06.012 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-06T20:27:06.012 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-11-06T20:27:06.013 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-11-06T20:27:06.013 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-11-06T20:27:06.013 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-06T20:27:06.013 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-06T20:27:06.013 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-11-06T20:27:06.014 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-06T20:27:06.014 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-11-06T20:27:06.014 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-11-06T20:27:06.014 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-06T20:27:06.014 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-06T20:27:06.015 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-06T20:27:06.015 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-06T20:27:06.015 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-06T20:27:06.015 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-11-06T20:27:06.015 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-11-06T20:27:06.016 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-11-06T20:27:06.016 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-11-06T20:27:06.016 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-06T20:27:06.016 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-11-06T20:27:06.016 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-06T20:27:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-11-06T20:27:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-11-06T20:27:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-11-06T20:27:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-11-06T20:27:06.017 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-11-06T20:27:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-06T20:27:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-11-06T20:27:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-06T20:27:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-11-06T20:27:06.018 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-11-06T20:27:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-11-06T20:27:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-11-06T20:27:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-11-06T20:27:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-11-06T20:27:06.019 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-06T20:27:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-11-06T20:27:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-11-06T20:27:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-11-06T20:27:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-06T20:27:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-06T20:27:06.020 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-06T20:27:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-06T20:27:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-06T20:27:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-06T20:27:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-06T20:27:06.021 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-06T20:27:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-06T20:27:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-06T20:27:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-06T20:27:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-06T20:27:06.022 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-06T20:27:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-06T20:27:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-06T20:27:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-11-06T20:27:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-11-06T20:27:06.023 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-06T20:27:06.024 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-11-06T20:27:06.024 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-11-06T20:27:06.024 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-11-06T20:27:06.024 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-11-06T20:27:06.024 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-11-06T20:27:06.025 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-06T20:27:06.025 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-11-06T20:27:06.025 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-11-06T20:27:06.025 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-06T20:27:06.025 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-11-06T20:27:06.026 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-11-06T20:27:06.026 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-06T20:27:06.026 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-11-06T20:27:06.026 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-06T20:27:06.026 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-11-06T20:27:06.027 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-11-06T20:27:06.027 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-11-06T20:27:06.027 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-11-06T20:27:06.027 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-11-06T20:27:06.027 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-11-06T20:27:06.028 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-11-06T20:27:06.028 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-11-06T20:27:06.028 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-11-06T20:27:06.028 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-06T20:27:06.028 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-11-06T20:27:06.029 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-11-06T20:27:06.029 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-11-06T20:27:06.029 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-06T20:27:06.029 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-11-06T20:27:06.029 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-11-06T20:27:06.030 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-11-06T20:27:06.030 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-11-06T20:27:06.030 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-11-06T20:27:06.030 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-11-06T20:27:06.030 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-11-06T20:27:06.031 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-11-06T20:27:06.031 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-11-06T20:27:06.031 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-11-06T20:27:06.031 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-11-06T20:27:06.031 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-06T20:27:06.032 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-11-06T20:27:06.032 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-11-06T20:27:06.032 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-11-06T20:27:06.032 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-06T20:27:06.032 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-06T20:27:06.033 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-11-06T20:27:06.033 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-11-06T20:27:06.033 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-06T20:27:06.033 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-11-06T20:27:06.033 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-11-06T20:27:06.033 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-11-06T20:27:06.034 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-11-06T20:27:06.034 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-11-06T20:27:06.034 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-11-06T20:27:06.034 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-11-06T20:27:06.034 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-06T20:27:06.035 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-11-06T20:27:06.035 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-11-06T20:27:06.035 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-11-06T20:27:06.035 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-06T20:27:06.035 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-06T20:27:06.036 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-11-06T20:27:06.036 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-06T20:27:06.036 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-06T20:27:06.036 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-06T20:27:06.036 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-11-06T20:27:06.037 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-11-06T20:27:06.037 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-11-06T20:27:06.037 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-11-06T20:27:06.037 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-06T20:27:06.037 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-11-06T20:27:06.037 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-06T20:27:06.038 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-06T20:27:06.038 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-06T20:27:06.038 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-11-06T20:27:06.038 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-11-06T20:27:06.038 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-11-06T20:27:06.038 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-11-06T20:27:06.039 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-11-06T20:27:06.039 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-11-06T20:27:06.039 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-11-06T20:27:06.039 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-11-06T20:27:06.040 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-11-06T20:27:06.040 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-11-06T20:27:06.040 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-11-06T20:27:06.040 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-11-06T20:27:06.040 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-11-06T20:27:06.040 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-11-06T20:27:06.041 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-06T20:27:06.041 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-11-06T20:27:06.041 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-11-06T20:27:06.041 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-06T20:27:06.041 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-06T20:27:06.042 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-11-06T20:27:06.042 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-06T20:27:06.042 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-11-06T20:27:06.042 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-11-06T20:27:06.042 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-06T20:27:06.043 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-11-06T20:27:06.043 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-11-06T20:27:06.043 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-06T20:27:06.043 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-06T20:27:06.043 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-06T20:27:06.044 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-06T20:27:06.044 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-06T20:27:06.044 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-06T20:27:06.044 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-06T20:27:06.045 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-11-06T20:27:06.045 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-06T20:27:06.045 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-06T20:27:06.045 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-06T20:27:06.045 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-06T20:27:06.045 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-06T20:27:06.046 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-11-06T20:27:06.046 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-11-06T20:27:06.046 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-06T20:27:06.046 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-06T20:27:06.047 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-06T20:27:06.047 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-11-06T20:27:06.047 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-11-06T20:27:06.047 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-06T20:27:06.047 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-06T20:27:06.048 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-11-06T20:27:06.048 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-06T20:27:06.048 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-11-06T20:27:06.048 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-11-06T20:27:06.048 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-06T20:27:06.048 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-11-06T20:27:06.049 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-11-06T20:27:06.049 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-11-06T20:27:06.049 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-11-06T20:27:06.049 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-11-06T20:27:06.049 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-11-06T20:27:06.050 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-06T20:27:06.050 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-11-06T20:27:06.050 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-11-06T20:27:06.050 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-11-06T20:27:06.051 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-11-06T20:27:06.051 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-11-06T20:27:06.051 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-06T20:27:06.051 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-06T20:27:06.051 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-11-06T20:27:06.051 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-11-06T20:27:06.052 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-11-06T20:27:06.052 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-11-06T20:27:06.052 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-11-06T20:27:06.052 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-11-06T20:27:06.052 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-06T20:27:06.053 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-06T20:27:06.053 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-06T20:27:06.053 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-06T20:27:06.053 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-06T20:27:06.053 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-06T20:27:06.054 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-06T20:27:06.054 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-06T20:27:06.054 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-06T20:27:06.054 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-06T20:27:06.054 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-06T20:27:06.054 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-11-06T20:27:06.055 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-11-06T20:27:06.055 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-06T20:27:06.055 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-06T20:27:06.055 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-06T20:27:06.055 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-11-06T20:27:06.055 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-11-06T20:27:06.056 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-06T20:27:06.056 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-06T20:27:06.056 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-11-06T20:27:06.056 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-11-06T20:27:06.056 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-06T20:27:06.057 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-06T20:27:06.057 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-11-06T20:27:06.057 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-11-06T20:27:06.057 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-11-06T20:27:06.057 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-06T20:27:06.057 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-06T20:27:06.058 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-06T20:27:06.058 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-06T20:27:06.058 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-11-06T20:27:06.058 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-06T20:27:06.058 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-11-06T20:27:06.059 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-06T20:27:06.059 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-06T20:27:06.059 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-06T20:27:06.059 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-11-06T20:27:06.059 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-11-06T20:27:06.060 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-11-06T20:27:06.060 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-06T20:27:06.060 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-11-06T20:27:06.060 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-11-06T20:27:06.060 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-11-06T20:27:06.060 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-11-06T20:27:06.061 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-06T20:27:06.061 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-11-06T20:27:06.061 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-11-06T20:27:06.061 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-06T20:27:06.061 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-11-06T20:27:06.061 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-11-06T20:27:06.062 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-11-06T20:27:06.062 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-11-06T20:27:06.062 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-11-06T20:27:06.062 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-11-06T20:27:06.062 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-11-06T20:27:06.063 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-11-06T20:27:06.063 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-11-06T20:27:06.063 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-11-06T20:27:06.063 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-11-06T20:27:06.063 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-11-06T20:27:06.063 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-11-06T20:27:06.064 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-11-06T20:27:06.064 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-11-06T20:27:06.064 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-11-06T20:27:06.064 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-11-06T20:27:06.064 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-11-06T20:27:06.065 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-11-06T20:27:06.065 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-11-06T20:27:06.065 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-11-06T20:27:06.065 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-06T20:27:06.065 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-11-06T20:27:06.066 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-11-06T20:27:06.066 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-11-06T20:27:06.066 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-11-06T20:27:06.066 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-11-06T20:27:06.066 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-11-06T20:27:06.067 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-11-06T20:27:06.067 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-11-06T20:27:06.067 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-11-06T20:27:06.067 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-11-06T20:27:06.067 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-11-06T20:27:06.068 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-11-06T20:27:06.068 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-11-06T20:27:06.068 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-11-06T20:27:06.068 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-11-06T20:27:06.068 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-11-06T20:27:06.069 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-11-06T20:27:06.069 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-11-06T20:27:06.069 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-11-06T20:27:06.069 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-11-06T20:27:06.069 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-11-06T20:27:06.070 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-11-06T20:27:06.070 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-11-06T20:27:06.070 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-11-06T20:27:06.070 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-11-06T20:27:06.070 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-11-06T20:27:06.071 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-11-06T20:27:06.071 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-11-06T20:27:06.071 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-11-06T20:27:06.071 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-06T20:27:06.071 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-11-06T20:27:06.071 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-06T20:27:06.072 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-11-06T20:27:06.072 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-06T20:27:06.072 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-11-06T20:27:06.072 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-11-06T20:27:06.072 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-11-06T20:27:06.073 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-11-06T20:27:06.073 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-11-06T20:27:06.073 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-11-06T20:27:06.073 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-11-06T20:27:06.073 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-11-06T20:27:06.074 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-11-06T20:27:06.074 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-11-06T20:27:06.074 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-11-06T20:27:06.074 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-11-06T20:27:06.074 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-11-06T20:27:06.075 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-11-06T20:27:06.075 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-11-06T20:27:06.075 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-11-06T20:27:06.075 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-11-06T20:27:06.075 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-11-06T20:27:06.076 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-11-06T20:27:06.076 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-11-06T20:27:06.076 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-11-06T20:27:06.076 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-11-06T20:27:06.076 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-11-06T20:27:06.077 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-11-06T20:27:06.077 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-11-06T20:27:06.077 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-11-06T20:27:06.077 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-11-06T20:27:06.077 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-11-06T20:27:06.077 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-11-06T20:27:06.078 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-11-06T20:27:06.078 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-11-06T20:27:06.078 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-11-06T20:27:06.078 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-11-06T20:27:06.078 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-11-06T20:27:06.079 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-11-06T20:27:06.079 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-11-06T20:27:06.079 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-11-06T20:27:06.079 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-11-06T20:27:06.079 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-11-06T20:27:06.080 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-11-06T20:27:06.080 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-11-06T20:27:06.080 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-11-06T20:27:06.080 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-11-06T20:27:06.080 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-11-06T20:27:06.081 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-11-06T20:27:06.081 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-11-06T20:27:06.081 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-11-06T20:27:06.081 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-11-06T20:27:06.081 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-11-06T20:27:06.081 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-11-06T20:27:06.082 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-11-06T20:27:06.082 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-11-06T20:27:06.082 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-11-06T20:27:06.082 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-11-06T20:27:06.082 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-11-06T20:27:06.083 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-11-06T20:27:06.083 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-11-06T20:27:06.083 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-11-06T20:27:06.083 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-11-06T20:27:06.083 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-06T20:27:06.084 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-11-06T20:27:06.084 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-11-06T20:27:06.084 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-11-06T20:27:06.084 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-06T20:27:06.084 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-11-06T20:27:06.084 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-06T20:27:06.085 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-11-06T20:27:06.085 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-06T20:27:06.085 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-11-06T20:27:06.085 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-06T20:27:06.085 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-06T20:27:06.086 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-06T20:27:06.086 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-11-06T20:27:06.086 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-11-06T20:27:06.086 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-11-06T20:27:06.086 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-11-06T20:27:06.087 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-11-06T20:27:06.087 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-11-06T20:27:06.087 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-11-06T20:27:06.087 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-11-06T20:27:06.087 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-11-06T20:27:06.088 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-11-06T20:27:06.088 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-11-06T20:27:06.088 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-11-06T20:27:06.088 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-11-06T20:27:06.088 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-11-06T20:27:06.089 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-11-06T20:27:06.089 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-11-06T20:27:06.089 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-11-06T20:27:06.089 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-11-06T20:27:06.089 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-11-06T20:27:06.090 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-11-06T20:27:06.090 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-11-06T20:27:06.090 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-11-06T20:27:06.090 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-11-06T20:27:06.090 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-11-06T20:27:06.091 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-11-06T20:27:06.091 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-11-06T20:27:06.091 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-11-06T20:27:06.091 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-11-06T20:27:06.091 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-11-06T20:27:06.092 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-06T20:27:06.092 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-11-06T20:27:06.092 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-11-06T20:27:06.092 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-11-06T20:27:06.092 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-11-06T20:27:06.093 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-11-06T20:27:06.093 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-11-06T20:27:06.093 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-11-06T20:27:06.093 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-11-06T20:27:06.093 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-11-06T20:27:06.094 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-11-06T20:27:06.094 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-11-06T20:27:06.094 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-11-06T20:27:06.094 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-11-06T20:27:06.094 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-11-06T20:27:06.095 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-11-06T20:27:06.095 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-11-06T20:27:06.095 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-11-06T20:27:06.095 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-06T20:27:06.095 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-11-06T20:27:06.096 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-11-06T20:27:06.096 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-11-06T20:27:06.096 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-11-06T20:27:06.096 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-11-06T20:27:06.096 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-11-06T20:27:06.096 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-11-06T20:27:06.097 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-11-06T20:27:06.097 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-11-06T20:27:06.097 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-11-06T20:27:06.097 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-11-06T20:27:06.097 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-11-06T20:27:06.097 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-11-06T20:27:06.097 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-06T20:27:06.098 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-11-06T20:27:06.098 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-11-06T20:27:06.098 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-11-06T20:27:06.098 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-11-06T20:27:06.098 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-11-06T20:27:06.099 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-11-06T20:27:06.099 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-11-06T20:27:06.099 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-11-06T20:27:06.099 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-11-06T20:27:06.099 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-11-06T20:27:06.099 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-11-06T20:27:06.100 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-11-06T20:27:06.100 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-11-06T20:27:06.100 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-11-06T20:27:06.100 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-11-06T20:27:06.100 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-11-06T20:27:06.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-11-06T20:27:06.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-11-06T20:27:06.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-11-06T20:27:06.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-11-06T20:27:06.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-11-06T20:27:06.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-11-06T20:27:06.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-06T20:27:06.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-06T20:27:06.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-06T20:27:06.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-06T20:27:06.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-06T20:27:06.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-06T20:27:06.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-06T20:27:06.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-06T20:27:06.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-06T20:27:06.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-06T20:27:06.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-06T20:27:06.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-06T20:27:06.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-06T20:27:06.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-06T20:27:06.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-06T20:27:06.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-06T20:27:06.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-06T20:27:06.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-06T20:27:06.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-06T20:27:06.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-06T20:27:06.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-11-06T20:27:06.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-06T20:27:06.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-06T20:27:06.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-06T20:27:06.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-11-06T20:27:06.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-11-06T20:27:06.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-11-06T20:27:06.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-11-06T20:27:06.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-11-06T20:27:06.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-11-06T20:27:06.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-11-06T20:27:06.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-11-06T20:27:06.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-11-06T20:27:06.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-11-06T20:27:06.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-11-06T20:27:06.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-11-06T20:27:06.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-11-06T20:27:06.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-11-06T20:27:06.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-11-06T20:27:06.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-11-06T20:27:06.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-11-06T20:27:06.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-11-06T20:27:06.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-11-06T20:27:06.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-11-06T20:27:06.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-06T20:27:06.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-11-06T20:27:06.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-06T20:27:06.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-11-06T20:27:06.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-11-06T20:27:06.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-11-06T20:27:06.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-11-06T20:27:06.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-11-06T20:27:06.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-11-06T20:27:06.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-11-06T20:27:06.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-11-06T20:27:06.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-11-06T20:27:06.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-11-06T20:27:06.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-11-06T20:27:06.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-11-06T20:27:06.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-06T20:27:06.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-06T20:27:06.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-06T20:27:06.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-06T20:27:06.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-06T20:27:06.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-06T20:27:06.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-06T20:27:06.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-06T20:27:06.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-06T20:27:06.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-06T20:27:06.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-06T20:27:06.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-06T20:27:06.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-06T20:27:06.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-06T20:27:06.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-06T20:27:06.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-06T20:27:06.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-06T20:27:06.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-06T20:27:06.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-06T20:27:06.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-06T20:27:06.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-06T20:27:06.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-11-06T20:27:06.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-11-06T20:27:06.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-11-06T20:27:06.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-11-06T20:27:06.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-11-06T20:27:06.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-11-06T20:27:06.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-11-06T20:27:06.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-11-06T20:27:06.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-11-06T20:27:06.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-11-06T20:27:06.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-11-06T20:27:06.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-11-06T20:27:06.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-11-06T20:27:06.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-11-06T20:27:06.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-11-06T20:27:06.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-11-06T20:27:06.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-06T20:27:06.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-11-06T20:27:06.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-11-06T20:27:06.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-11-06T20:27:06.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-11-06T20:27:06.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-11-06T20:27:06.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-06T20:27:06.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-11-06T20:27:06.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-11-06T20:27:06.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-11-06T20:27:06.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-06T20:27:06.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-06T20:27:06.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-06T20:27:06.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-06T20:27:06.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-06T20:27:06.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-06T20:27:06.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-06T20:27:06.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-06T20:27:06.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-06T20:27:06.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-06T20:27:06.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-06T20:27:06.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-06T20:27:06.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-06T20:27:06.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-06T20:27:06.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-06T20:27:06.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-06T20:27:06.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-06T20:27:06.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-06T20:27:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-06T20:27:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-06T20:27:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-06T20:27:06.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-06T20:27:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-06T20:27:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-06T20:27:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-11-06T20:27:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-11-06T20:27:06.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-06T20:27:06.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-11-06T20:27:06.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-11-06T20:27:06.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-11-06T20:27:06.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-11-06T20:27:06.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-06T20:27:06.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-11-06T20:27:06.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-11-06T20:27:06.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-11-06T20:27:06.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-11-06T20:27:06.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-11-06T20:27:06.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-11-06T20:27:06.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-11-06T20:27:06.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-11-06T20:27:06.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-11-06T20:27:06.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-11-06T20:27:06.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-11-06T20:27:06.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-11-06T20:27:06.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-11-06T20:27:06.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-11-06T20:27:06.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-06T20:27:06.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-06T20:27:06.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-06T20:27:06.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-06T20:27:06.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-06T20:27:06.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-06T20:27:06.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-06T20:27:06.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-06T20:27:06.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-06T20:27:06.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-06T20:27:06.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-06T20:27:06.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-06T20:27:06.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-06T20:27:06.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-06T20:27:06.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-06T20:27:06.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-11-06T20:27:06.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-06T20:27:06.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-06T20:27:06.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-06T20:27:06.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-06T20:27:06.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-06T20:27:06.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-06T20:27:06.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-06T20:27:06.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-06T20:27:06.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-06T20:27:06.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-06T20:27:06.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-06T20:27:06.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-06T20:27:06.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-06T20:27:06.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-06T20:27:06.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-11-06T20:27:06.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-11-06T20:27:06.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-06T20:27:06.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-06T20:27:06.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-06T20:27:06.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-06T20:27:06.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-06T20:27:06.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-06T20:27:06.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-06T20:27:06.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-06T20:27:06.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-06T20:27:06.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-06T20:27:06.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-06T20:27:06.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-06T20:27:06.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-06T20:27:06.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-11-06T20:27:06.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-06T20:27:06.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-11-06T20:27:06.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-11-06T20:27:06.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-06T20:27:06.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-06T20:27:06.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-11-06T20:27:06.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-11-06T20:27:06.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-06T20:27:06.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-06T20:27:06.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-11-06T20:27:06.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-11-06T20:27:06.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-06T20:27:06.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-11-06T20:27:06.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-06T20:27:06.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-06T20:27:06.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-11-06T20:27:06.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-11-06T20:27:06.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-11-06T20:27:06.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-06T20:27:06.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-11-06T20:27:06.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-06T20:27:06.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-11-06T20:27:06.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-11-06T20:27:06.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-11-06T20:27:06.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-11-06T20:27:06.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-11-06T20:27:06.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-11-06T20:27:06.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-11-06T20:27:06.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-11-06T20:27:06.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-11-06T20:27:06.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-11-06T20:27:06.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-11-06T20:27:06.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-11-06T20:27:06.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-11-06T20:27:06.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-11-06T20:27:06.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-06T20:27:06.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-06T20:27:06.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-06T20:27:06.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-11-06T20:27:06.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-11-06T20:27:06.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-11-06T20:27:06.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-11-06T20:27:06.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-06T20:27:06.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-11-06T20:27:06.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-11-06T20:27:06.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-06T20:27:06.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-11-06T20:27:06.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-11-06T20:27:06.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-06T20:27:06.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-11-06T20:27:06.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-06T20:27:06.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-11-06T20:27:06.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-06T20:27:06.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-06T20:27:06.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-11-06T20:27:06.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-11-06T20:27:06.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-11-06T20:27:06.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-11-06T20:27:06.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-11-06T20:27:06.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-11-06T20:27:06.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-11-06T20:27:06.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-11-06T20:27:06.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-11-06T20:27:06.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-11-06T20:27:06.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-11-06T20:27:06.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-11-06T20:27:06.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-11-06T20:27:06.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-11-06T20:27:06.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-11-06T20:27:06.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-11-06T20:27:06.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-11-06T20:27:06.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-11-06T20:27:06.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-11-06T20:27:06.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-11-06T20:27:06.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-11-06T20:27:06.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-11-06T20:27:06.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-11-06T20:27:06.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-11-06T20:27:06.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-11-06T20:27:06.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-11-06T20:27:06.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-11-06T20:27:06.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-11-06T20:27:06.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-11-06T20:27:06.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-11-06T20:27:06.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-11-06T20:27:06.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-11-06T20:27:06.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-11-06T20:27:06.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-11-06T20:27:06.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-11-06T20:27:06.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-11-06T20:27:06.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-11-06T20:27:06.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-11-06T20:27:06.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-11-06T20:27:06.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-11-06T20:27:06.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-11-06T20:27:06.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-11-06T20:27:06.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-11-06T20:27:06.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-11-06T20:27:06.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-11-06T20:27:06.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-11-06T20:27:06.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-11-06T20:27:06.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-11-06T20:27:06.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-11-06T20:27:06.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-11-06T20:27:06.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-11-06T20:27:06.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-11-06T20:27:06.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-11-06T20:27:06.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-11-06T20:27:06.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-11-06T20:27:06.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-11-06T20:27:06.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-11-06T20:27:06.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-11-06T20:27:06.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-11-06T20:27:06.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-11-06T20:27:06.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-11-06T20:27:06.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-11-06T20:27:06.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-11-06T20:27:06.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-11-06T20:27:06.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-11-06T20:27:06.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-11-06T20:27:06.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-11-06T20:27:06.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-11-06T20:27:06.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-11-06T20:27:06.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-11-06T20:27:06.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-11-06T20:27:06.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-11-06T20:27:06.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-11-06T20:27:06.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-11-06T20:27:06.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-11-06T20:27:06.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-11-06T20:27:06.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-11-06T20:27:06.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-11-06T20:27:06.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-11-06T20:27:06.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-11-06T20:27:06.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-11-06T20:27:06.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-11-06T20:27:06.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-11-06T20:27:06.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-11-06T20:27:06.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-11-06T20:27:06.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-11-06T20:27:06.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-11-06T20:27:06.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-11-06T20:27:06.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-11-06T20:27:06.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-11-06T20:27:06.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-11-06T20:27:06.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-11-06T20:27:06.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-11-06T20:27:06.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-11-06T20:27:06.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-11-06T20:27:06.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-11-06T20:27:06.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-11-06T20:27:06.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-11-06T20:27:06.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-11-06T20:27:06.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-11-06T20:27:06.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-11-06T20:27:06.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-11-06T20:27:06.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-11-06T20:27:06.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-11-06T20:27:06.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-11-06T20:27:06.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-11-06T20:27:06.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-11-06T20:27:06.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-11-06T20:27:06.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-11-06T20:27:06.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-11-06T20:27:06.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-11-06T20:27:06.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-11-06T20:27:06.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-11-06T20:27:06.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-11-06T20:27:06.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-11-06T20:27:06.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-11-06T20:27:06.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-06T20:27:06.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-06T20:27:06.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-11-06T20:27:06.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-11-06T20:27:06.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-06T20:27:06.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-11-06T20:27:06.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-11-06T20:27:06.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-11-06T20:27:06.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-11-06T20:27:06.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-11-06T20:27:06.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-11-06T20:27:06.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-06T20:27:06.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-06T20:27:06.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-11-06T20:27:06.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-06T20:27:06.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-06T20:27:06.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-11-06T20:27:06.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-06T20:27:06.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-11-06T20:27:06.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-11-06T20:27:06.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-11-06T20:27:06.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-06T20:27:06.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-11-06T20:27:06.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-11-06T20:27:06.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-11-06T20:27:06.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-11-06T20:27:06.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-11-06T20:27:06.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-11-06T20:27:06.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-11-06T20:27:06.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-11-06T20:27:06.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-11-06T20:27:06.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-11-06T20:27:06.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-11-06T20:27:06.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-11-06T20:27:06.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-11-06T20:27:06.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-11-06T20:27:06.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-11-06T20:27:06.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-11-06T20:27:06.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-11-06T20:27:06.179 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:06.179 DEBUG:teuthology.orchestra.run.smithi069:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-06T20:27:06.208 INFO:teuthology.orchestra.run.smithi069.stdout:postinstall scriptlet (using /bin/sh): 2023-11-06T20:27:06.208 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-11-06T20:27:06.209 INFO:teuthology.orchestra.run.smithi069.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-06T20:27:06.209 INFO:teuthology.orchestra.run.smithi069.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-06T20:27:06.209 INFO:teuthology.orchestra.run.smithi069.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:06.209 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-06T20:27:06.209 INFO:teuthology.orchestra.run.smithi069.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-06T20:27:06.210 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2023-11-06T20:27:06.210 INFO:teuthology.orchestra.run.smithi069.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-06T20:27:06.210 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-06T20:27:06.210 INFO:teuthology.orchestra.run.smithi069.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-11-06T20:27:06.210 INFO:teuthology.orchestra.run.smithi069.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-06T20:27:06.211 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:06.211 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2023-11-06T20:27:06.211 INFO:teuthology.orchestra.run.smithi069.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-06T20:27:06.211 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-06T20:27:06.211 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:06.212 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2023-11-06T20:27:06.213 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -qi grub2-tools 2023-11-06T20:27:06.292 INFO:teuthology.orchestra.run.smithi069.stdout:Name : grub2-tools 2023-11-06T20:27:06.292 INFO:teuthology.orchestra.run.smithi069.stdout:Epoch : 1 2023-11-06T20:27:06.292 INFO:teuthology.orchestra.run.smithi069.stdout:Version : 2.02 2023-11-06T20:27:06.293 INFO:teuthology.orchestra.run.smithi069.stdout:Release : 129.el8 2023-11-06T20:27:06.293 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: x86_64 2023-11-06T20:27:06.293 INFO:teuthology.orchestra.run.smithi069.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-11-06T20:27:06.293 INFO:teuthology.orchestra.run.smithi069.stdout:Group : System Environment/Base 2023-11-06T20:27:06.293 INFO:teuthology.orchestra.run.smithi069.stdout:Size : 9265586 2023-11-06T20:27:06.294 INFO:teuthology.orchestra.run.smithi069.stdout:License : GPLv3+ 2023-11-06T20:27:06.294 INFO:teuthology.orchestra.run.smithi069.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-11-06T20:27:06.294 INFO:teuthology.orchestra.run.smithi069.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-11-06T20:27:06.294 INFO:teuthology.orchestra.run.smithi069.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-11-06T20:27:06.294 INFO:teuthology.orchestra.run.smithi069.stdout:Build Host : kbuilder.bsys.centos.org 2023-11-06T20:27:06.295 INFO:teuthology.orchestra.run.smithi069.stdout:Relocations : (not relocatable) 2023-11-06T20:27:06.295 INFO:teuthology.orchestra.run.smithi069.stdout:Packager : CentOS BuildSystem 2023-11-06T20:27:06.295 INFO:teuthology.orchestra.run.smithi069.stdout:Vendor : CentOS 2023-11-06T20:27:06.295 INFO:teuthology.orchestra.run.smithi069.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-06T20:27:06.295 INFO:teuthology.orchestra.run.smithi069.stdout:Summary : Support tools for GRUB. 2023-11-06T20:27:06.295 INFO:teuthology.orchestra.run.smithi069.stdout:Description : 2023-11-06T20:27:06.296 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:27:06.296 INFO:teuthology.orchestra.run.smithi069.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-06T20:27:06.296 INFO:teuthology.orchestra.run.smithi069.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-06T20:27:06.296 INFO:teuthology.orchestra.run.smithi069.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-06T20:27:06.296 INFO:teuthology.orchestra.run.smithi069.stdout:hardware devices. 2023-11-06T20:27:06.296 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:27:06.297 INFO:teuthology.orchestra.run.smithi069.stdout:This subpackage provides tools for support of all platforms. 2023-11-06T20:27:06.297 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-06T20:27:06.298 INFO:teuthology.task.kernel:Updating grub on smithi069 to boot 6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:06.298 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-06T20:27:09.408 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2023-11-06T20:27:11.584 INFO:teuthology.orchestra.run.smithi069.stderr:done 2023-11-06T20:27:11.587 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2023-11-06T20:27:11.609 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.bfM26cgWID 2023-11-06T20:27:11.610 DEBUG:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bfM26cgWID 2023-11-06T20:27:11.699 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.bfM26cgWID 2023-11-06T20:27:11.840 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.bfM26cgWID is 6KB 2023-11-06T20:27:11.888 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.bfM26cgWID 2023-11-06T20:27:11.907 DEBUG:teuthology.orchestra.run.smithi069:> sudo /bin/ls /boot/loader/entries || true 2023-11-06T20:27:11.971 INFO:teuthology.orchestra.run.smithi069.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-11-06T20:27:11.972 INFO:teuthology.orchestra.run.smithi069.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-11-06T20:27:11.972 INFO:teuthology.orchestra.run.smithi069.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-11-06T20:27:11.972 INFO:teuthology.orchestra.run.smithi069.stdout:a1b29d6c289747bdb22930598d39874a-6.6.0-rc5-g5b73517e0e2f.conf 2023-11-06T20:27:11.973 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:12.138 DEBUG:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2023-11-06T20:27:25.893 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-06T20:27:25.894 DEBUG:teuthology.orchestra.run.smithi033:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-06T20:27:25.952 INFO:teuthology.orchestra.run.smithi033.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:25.952 INFO:teuthology.orchestra.run.smithi033.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:25.952 INFO:teuthology.orchestra.run.smithi033.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:25.953 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:25.953 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-11-06T20:27:25.953 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-11-06T20:27:25.953 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-11-06T20:27:25.953 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-11-06T20:27:25.954 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-06T20:27:25.954 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-06T20:27:25.954 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-06T20:27:25.954 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-06T20:27:25.954 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-06T20:27:25.955 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-06T20:27:25.955 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-06T20:27:25.955 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-06T20:27:25.955 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-06T20:27:25.955 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-06T20:27:25.956 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-06T20:27:25.956 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-06T20:27:25.956 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-06T20:27:25.956 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-06T20:27:25.956 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-06T20:27:25.957 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-06T20:27:25.957 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-06T20:27:25.957 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-06T20:27:25.957 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-06T20:27:25.957 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-06T20:27:25.958 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-11-06T20:27:25.958 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-11-06T20:27:25.958 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-11-06T20:27:25.958 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-06T20:27:25.958 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-11-06T20:27:25.959 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-06T20:27:25.959 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-06T20:27:25.959 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-11-06T20:27:25.959 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-11-06T20:27:25.960 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-11-06T20:27:25.960 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-11-06T20:27:25.960 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-11-06T20:27:25.960 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-11-06T20:27:25.960 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-11-06T20:27:25.961 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-11-06T20:27:25.961 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-06T20:27:25.961 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-11-06T20:27:25.961 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-11-06T20:27:25.961 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-11-06T20:27:25.962 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-11-06T20:27:25.962 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-11-06T20:27:25.962 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-11-06T20:27:25.962 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-11-06T20:27:25.962 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-11-06T20:27:25.963 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-11-06T20:27:25.963 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-11-06T20:27:25.963 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-11-06T20:27:25.963 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-11-06T20:27:25.963 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-11-06T20:27:25.964 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-11-06T20:27:25.964 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-11-06T20:27:25.964 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-11-06T20:27:25.964 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-11-06T20:27:25.964 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-11-06T20:27:25.964 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-11-06T20:27:25.965 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-11-06T20:27:25.965 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-11-06T20:27:25.965 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-11-06T20:27:25.965 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-11-06T20:27:25.965 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-11-06T20:27:25.966 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-11-06T20:27:25.966 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-11-06T20:27:25.966 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-11-06T20:27:25.966 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-11-06T20:27:25.966 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-11-06T20:27:25.967 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-11-06T20:27:25.967 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-11-06T20:27:25.967 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-11-06T20:27:25.967 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-11-06T20:27:25.967 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-11-06T20:27:25.968 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-11-06T20:27:25.968 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-11-06T20:27:25.968 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-11-06T20:27:25.968 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-11-06T20:27:25.968 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-11-06T20:27:25.969 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-11-06T20:27:25.969 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-11-06T20:27:25.969 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-11-06T20:27:25.969 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-11-06T20:27:25.969 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-11-06T20:27:25.970 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-06T20:27:25.970 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-11-06T20:27:25.970 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-11-06T20:27:25.970 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-11-06T20:27:25.970 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-11-06T20:27:25.971 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-11-06T20:27:25.971 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-11-06T20:27:25.971 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-11-06T20:27:25.971 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-11-06T20:27:25.971 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-11-06T20:27:25.972 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-11-06T20:27:25.972 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-11-06T20:27:25.972 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-11-06T20:27:25.972 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-11-06T20:27:25.972 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-11-06T20:27:25.972 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-11-06T20:27:25.973 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-11-06T20:27:25.973 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-11-06T20:27:25.973 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-11-06T20:27:25.973 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-11-06T20:27:25.973 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-11-06T20:27:25.974 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-11-06T20:27:25.974 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-11-06T20:27:25.974 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-11-06T20:27:25.974 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-11-06T20:27:25.974 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-06T20:27:25.975 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-06T20:27:25.975 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-11-06T20:27:25.975 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-11-06T20:27:25.975 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-11-06T20:27:25.975 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-11-06T20:27:25.976 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-11-06T20:27:25.976 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-11-06T20:27:25.976 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-11-06T20:27:25.976 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-11-06T20:27:25.976 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-11-06T20:27:25.977 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-11-06T20:27:25.977 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-11-06T20:27:25.977 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-11-06T20:27:25.977 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-11-06T20:27:25.977 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-06T20:27:25.978 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-06T20:27:25.978 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-11-06T20:27:25.978 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-11-06T20:27:25.978 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-11-06T20:27:25.978 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-11-06T20:27:25.979 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-11-06T20:27:25.979 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-11-06T20:27:25.979 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-11-06T20:27:25.979 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-11-06T20:27:25.979 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-11-06T20:27:25.979 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-11-06T20:27:25.980 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-11-06T20:27:25.980 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-11-06T20:27:25.980 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-11-06T20:27:25.980 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-11-06T20:27:25.980 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-11-06T20:27:25.980 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-11-06T20:27:25.981 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-11-06T20:27:25.981 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-11-06T20:27:25.981 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-11-06T20:27:25.981 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-11-06T20:27:25.981 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-11-06T20:27:25.982 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-11-06T20:27:25.982 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-11-06T20:27:25.982 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-11-06T20:27:25.982 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-11-06T20:27:25.982 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-11-06T20:27:25.983 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-11-06T20:27:25.983 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-06T20:27:25.983 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-06T20:27:25.983 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-11-06T20:27:25.983 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-11-06T20:27:25.983 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-11-06T20:27:25.984 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-11-06T20:27:25.984 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-11-06T20:27:25.984 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-11-06T20:27:25.984 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-11-06T20:27:25.984 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-11-06T20:27:25.985 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-11-06T20:27:25.985 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-11-06T20:27:25.985 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-11-06T20:27:25.985 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-06T20:27:25.985 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-11-06T20:27:25.985 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-11-06T20:27:25.986 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-11-06T20:27:25.986 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-11-06T20:27:25.986 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-11-06T20:27:25.986 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-11-06T20:27:25.986 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-11-06T20:27:25.987 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-06T20:27:25.987 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-11-06T20:27:25.987 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-11-06T20:27:25.987 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-11-06T20:27:25.987 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-11-06T20:27:25.988 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-11-06T20:27:25.988 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-11-06T20:27:25.988 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-06T20:27:25.988 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-06T20:27:25.988 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-06T20:27:25.989 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-11-06T20:27:25.989 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-06T20:27:25.989 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-11-06T20:27:25.989 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-06T20:27:25.989 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-06T20:27:25.989 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-06T20:27:25.990 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-06T20:27:25.990 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-06T20:27:25.990 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-06T20:27:25.990 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-11-06T20:27:25.990 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-11-06T20:27:25.991 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-11-06T20:27:25.991 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-11-06T20:27:25.991 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-11-06T20:27:25.991 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-11-06T20:27:25.991 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-11-06T20:27:25.992 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-06T20:27:25.992 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-06T20:27:25.992 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-06T20:27:25.992 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-06T20:27:25.992 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-06T20:27:25.993 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-11-06T20:27:25.993 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-11-06T20:27:25.993 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-11-06T20:27:25.993 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-06T20:27:25.993 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-06T20:27:25.994 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-06T20:27:25.994 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-06T20:27:25.994 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-06T20:27:25.994 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-06T20:27:25.994 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-11-06T20:27:25.994 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-11-06T20:27:25.995 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-06T20:27:25.995 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-11-06T20:27:25.995 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-11-06T20:27:25.995 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-11-06T20:27:25.995 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-11-06T20:27:25.996 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-06T20:27:25.996 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-06T20:27:25.996 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-06T20:27:25.996 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-11-06T20:27:25.996 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-11-06T20:27:25.997 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-11-06T20:27:25.997 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-06T20:27:25.997 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-11-06T20:27:25.997 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-11-06T20:27:25.997 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-11-06T20:27:25.998 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-06T20:27:25.998 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-11-06T20:27:25.998 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-11-06T20:27:25.998 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-06T20:27:25.998 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-11-06T20:27:25.999 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-06T20:27:25.999 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-11-06T20:27:25.999 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-11-06T20:27:25.999 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-11-06T20:27:25.999 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-11-06T20:27:25.999 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-11-06T20:27:26.000 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-11-06T20:27:26.000 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-11-06T20:27:26.000 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-11-06T20:27:26.000 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-11-06T20:27:26.000 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-11-06T20:27:26.001 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-11-06T20:27:26.001 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-11-06T20:27:26.001 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-11-06T20:27:26.001 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-11-06T20:27:26.002 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-11-06T20:27:26.002 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-11-06T20:27:26.002 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-06T20:27:26.002 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-11-06T20:27:26.002 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-11-06T20:27:26.002 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-06T20:27:26.003 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-11-06T20:27:26.003 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-11-06T20:27:26.003 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-11-06T20:27:26.003 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-11-06T20:27:26.003 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-11-06T20:27:26.004 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-11-06T20:27:26.004 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-11-06T20:27:26.004 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-11-06T20:27:26.004 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-11-06T20:27:26.004 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-11-06T20:27:26.005 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-11-06T20:27:26.005 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-06T20:27:26.005 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-06T20:27:26.005 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-11-06T20:27:26.006 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-11-06T20:27:26.006 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-11-06T20:27:26.006 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-11-06T20:27:26.006 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-11-06T20:27:26.006 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-11-06T20:27:26.006 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-11-06T20:27:26.007 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-11-06T20:27:26.007 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-06T20:27:26.007 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-06T20:27:26.007 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-11-06T20:27:26.007 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-11-06T20:27:26.008 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-11-06T20:27:26.008 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-11-06T20:27:26.008 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-11-06T20:27:26.008 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-11-06T20:27:26.008 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-11-06T20:27:26.009 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-11-06T20:27:26.009 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-11-06T20:27:26.009 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-06T20:27:26.009 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-11-06T20:27:26.009 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-06T20:27:26.010 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-06T20:27:26.010 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-06T20:27:26.010 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-06T20:27:26.010 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-06T20:27:26.010 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-06T20:27:26.011 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-06T20:27:26.011 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-06T20:27:26.011 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-06T20:27:26.011 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-11-06T20:27:26.011 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-11-06T20:27:26.011 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-11-06T20:27:26.012 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-06T20:27:26.012 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-11-06T20:27:26.012 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-11-06T20:27:26.012 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-11-06T20:27:26.012 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-11-06T20:27:26.013 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-11-06T20:27:26.013 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-06T20:27:26.013 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-11-06T20:27:26.013 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-11-06T20:27:26.013 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-11-06T20:27:26.014 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-11-06T20:27:26.014 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-11-06T20:27:26.014 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-11-06T20:27:26.014 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-11-06T20:27:26.014 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-11-06T20:27:26.015 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-11-06T20:27:26.015 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-11-06T20:27:26.015 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-11-06T20:27:26.015 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-11-06T20:27:26.015 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-11-06T20:27:26.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-11-06T20:27:26.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-11-06T20:27:26.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-11-06T20:27:26.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-11-06T20:27:26.016 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-11-06T20:27:26.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-11-06T20:27:26.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-06T20:27:26.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-11-06T20:27:26.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-11-06T20:27:26.017 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-11-06T20:27:26.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-11-06T20:27:26.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-11-06T20:27:26.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-11-06T20:27:26.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-11-06T20:27:26.018 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-11-06T20:27:26.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-11-06T20:27:26.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-11-06T20:27:26.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-11-06T20:27:26.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-11-06T20:27:26.019 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-11-06T20:27:26.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-11-06T20:27:26.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-11-06T20:27:26.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-11-06T20:27:26.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-11-06T20:27:26.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-11-06T20:27:26.020 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-11-06T20:27:26.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-06T20:27:26.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-11-06T20:27:26.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-11-06T20:27:26.021 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-06T20:27:26.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-11-06T20:27:26.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-11-06T20:27:26.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-11-06T20:27:26.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-11-06T20:27:26.022 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-11-06T20:27:26.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-11-06T20:27:26.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-11-06T20:27:26.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-11-06T20:27:26.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-11-06T20:27:26.023 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-11-06T20:27:26.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-11-06T20:27:26.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-11-06T20:27:26.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-11-06T20:27:26.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-11-06T20:27:26.024 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-06T20:27:26.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-11-06T20:27:26.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-11-06T20:27:26.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-11-06T20:27:26.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-11-06T20:27:26.025 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-11-06T20:27:26.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-11-06T20:27:26.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-11-06T20:27:26.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-11-06T20:27:26.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-11-06T20:27:26.026 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-11-06T20:27:26.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-11-06T20:27:26.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-11-06T20:27:26.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-11-06T20:27:26.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-11-06T20:27:26.027 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-11-06T20:27:26.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-11-06T20:27:26.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-11-06T20:27:26.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-11-06T20:27:26.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-11-06T20:27:26.028 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-11-06T20:27:26.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-11-06T20:27:26.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-11-06T20:27:26.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-11-06T20:27:26.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-11-06T20:27:26.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-11-06T20:27:26.029 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-11-06T20:27:26.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-11-06T20:27:26.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-11-06T20:27:26.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-11-06T20:27:26.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-11-06T20:27:26.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-11-06T20:27:26.030 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-11-06T20:27:26.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-11-06T20:27:26.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-11-06T20:27:26.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-11-06T20:27:26.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-11-06T20:27:26.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-11-06T20:27:26.031 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-11-06T20:27:26.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-11-06T20:27:26.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-11-06T20:27:26.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-11-06T20:27:26.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-11-06T20:27:26.032 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-11-06T20:27:26.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-11-06T20:27:26.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-11-06T20:27:26.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-11-06T20:27:26.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-11-06T20:27:26.033 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-11-06T20:27:26.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-06T20:27:26.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-06T20:27:26.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-06T20:27:26.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-06T20:27:26.034 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-06T20:27:26.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-06T20:27:26.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-06T20:27:26.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-06T20:27:26.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-06T20:27:26.035 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-06T20:27:26.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-06T20:27:26.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-06T20:27:26.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-11-06T20:27:26.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-11-06T20:27:26.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-11-06T20:27:26.036 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-06T20:27:26.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-11-06T20:27:26.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-11-06T20:27:26.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-11-06T20:27:26.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-11-06T20:27:26.037 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-11-06T20:27:26.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-11-06T20:27:26.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-11-06T20:27:26.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-11-06T20:27:26.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-11-06T20:27:26.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-11-06T20:27:26.038 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-11-06T20:27:26.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-11-06T20:27:26.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-11-06T20:27:26.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-11-06T20:27:26.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-11-06T20:27:26.039 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-11-06T20:27:26.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-11-06T20:27:26.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-11-06T20:27:26.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-11-06T20:27:26.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-11-06T20:27:26.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-11-06T20:27:26.040 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-11-06T20:27:26.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-11-06T20:27:26.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-11-06T20:27:26.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-11-06T20:27:26.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-11-06T20:27:26.041 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-11-06T20:27:26.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-06T20:27:26.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-06T20:27:26.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-11-06T20:27:26.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-06T20:27:26.042 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-06T20:27:26.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-06T20:27:26.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-06T20:27:26.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-06T20:27:26.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-06T20:27:26.043 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-06T20:27:26.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-06T20:27:26.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-06T20:27:26.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-06T20:27:26.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-06T20:27:26.044 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-06T20:27:26.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-06T20:27:26.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-06T20:27:26.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-06T20:27:26.045 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-06T20:27:26.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-06T20:27:26.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-06T20:27:26.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-06T20:27:26.046 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-06T20:27:26.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-06T20:27:26.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-06T20:27:26.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-06T20:27:26.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-11-06T20:27:26.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-11-06T20:27:26.047 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-06T20:27:26.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-11-06T20:27:26.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-11-06T20:27:26.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-11-06T20:27:26.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-11-06T20:27:26.048 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-06T20:27:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-11-06T20:27:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-06T20:27:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-06T20:27:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-06T20:27:26.049 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-06T20:27:26.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-06T20:27:26.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-11-06T20:27:26.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-11-06T20:27:26.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-06T20:27:26.050 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-11-06T20:27:26.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-06T20:27:26.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-11-06T20:27:26.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-06T20:27:26.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-11-06T20:27:26.051 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-06T20:27:26.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-11-06T20:27:26.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-11-06T20:27:26.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-06T20:27:26.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-11-06T20:27:26.052 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-06T20:27:26.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-11-06T20:27:26.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-06T20:27:26.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-11-06T20:27:26.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-06T20:27:26.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-11-06T20:27:26.053 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-06T20:27:26.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-11-06T20:27:26.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-11-06T20:27:26.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-06T20:27:26.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-06T20:27:26.054 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-11-06T20:27:26.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-11-06T20:27:26.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-11-06T20:27:26.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-06T20:27:26.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-11-06T20:27:26.055 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-11-06T20:27:26.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-11-06T20:27:26.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-11-06T20:27:26.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-06T20:27:26.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-11-06T20:27:26.056 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-06T20:27:26.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-06T20:27:26.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-06T20:27:26.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-11-06T20:27:26.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-06T20:27:26.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-11-06T20:27:26.057 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-11-06T20:27:26.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-11-06T20:27:26.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-06T20:27:26.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-11-06T20:27:26.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-11-06T20:27:26.058 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-11-06T20:27:26.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-11-06T20:27:26.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-06T20:27:26.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-11-06T20:27:26.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-11-06T20:27:26.059 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-11-06T20:27:26.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-11-06T20:27:26.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-11-06T20:27:26.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-11-06T20:27:26.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-06T20:27:26.060 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-06T20:27:26.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-06T20:27:26.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-06T20:27:26.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-06T20:27:26.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-06T20:27:26.061 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-06T20:27:26.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-06T20:27:26.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-11-06T20:27:26.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-11-06T20:27:26.062 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-11-06T20:27:26.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-11-06T20:27:26.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-11-06T20:27:26.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-11-06T20:27:26.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-11-06T20:27:26.063 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-11-06T20:27:26.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-11-06T20:27:26.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-11-06T20:27:26.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-11-06T20:27:26.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-11-06T20:27:26.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-11-06T20:27:26.064 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-11-06T20:27:26.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-11-06T20:27:26.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-11-06T20:27:26.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-11-06T20:27:26.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-11-06T20:27:26.065 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-11-06T20:27:26.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-11-06T20:27:26.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-11-06T20:27:26.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-11-06T20:27:26.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-06T20:27:26.066 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-11-06T20:27:26.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-11-06T20:27:26.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-11-06T20:27:26.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-11-06T20:27:26.067 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-11-06T20:27:26.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-11-06T20:27:26.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-11-06T20:27:26.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-11-06T20:27:26.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-11-06T20:27:26.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-11-06T20:27:26.068 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-11-06T20:27:26.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-11-06T20:27:26.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-11-06T20:27:26.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-11-06T20:27:26.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-11-06T20:27:26.069 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-11-06T20:27:26.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-11-06T20:27:26.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-11-06T20:27:26.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-11-06T20:27:26.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-11-06T20:27:26.070 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-11-06T20:27:26.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-11-06T20:27:26.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-11-06T20:27:26.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-06T20:27:26.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-11-06T20:27:26.071 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-11-06T20:27:26.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-11-06T20:27:26.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-11-06T20:27:26.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-11-06T20:27:26.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-11-06T20:27:26.072 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-11-06T20:27:26.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-06T20:27:26.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-06T20:27:26.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-11-06T20:27:26.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-11-06T20:27:26.073 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-11-06T20:27:26.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-11-06T20:27:26.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-11-06T20:27:26.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-11-06T20:27:26.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-06T20:27:26.074 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-06T20:27:26.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-11-06T20:27:26.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-11-06T20:27:26.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-11-06T20:27:26.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-11-06T20:27:26.075 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-11-06T20:27:26.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-06T20:27:26.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-11-06T20:27:26.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-06T20:27:26.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-06T20:27:26.076 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-06T20:27:26.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-11-06T20:27:26.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-11-06T20:27:26.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-11-06T20:27:26.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-11-06T20:27:26.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-11-06T20:27:26.077 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-06T20:27:26.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-11-06T20:27:26.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-11-06T20:27:26.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-11-06T20:27:26.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-11-06T20:27:26.078 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-11-06T20:27:26.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-06T20:27:26.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-06T20:27:26.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-06T20:27:26.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-11-06T20:27:26.079 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-06T20:27:26.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-11-06T20:27:26.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-11-06T20:27:26.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-11-06T20:27:26.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-11-06T20:27:26.080 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-06T20:27:26.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-06T20:27:26.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-06T20:27:26.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-06T20:27:26.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-11-06T20:27:26.081 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-06T20:27:26.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-06T20:27:26.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-06T20:27:26.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-06T20:27:26.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-11-06T20:27:26.082 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-06T20:27:26.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-11-06T20:27:26.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-06T20:27:26.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-11-06T20:27:26.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-06T20:27:26.083 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-11-06T20:27:26.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-06T20:27:26.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-11-06T20:27:26.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-06T20:27:26.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-06T20:27:26.084 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-06T20:27:26.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-11-06T20:27:26.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-11-06T20:27:26.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-06T20:27:26.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-06T20:27:26.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-06T20:27:26.085 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-06T20:27:26.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-06T20:27:26.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-11-06T20:27:26.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-06T20:27:26.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-06T20:27:26.086 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-11-06T20:27:26.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-06T20:27:26.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-06T20:27:26.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-06T20:27:26.087 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-06T20:27:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-06T20:27:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-11-06T20:27:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-06T20:27:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-06T20:27:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-11-06T20:27:26.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-11-06T20:27:26.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-06T20:27:26.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-11-06T20:27:26.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-06T20:27:26.089 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-06T20:27:26.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-06T20:27:26.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-06T20:27:26.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-06T20:27:26.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-06T20:27:26.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-06T20:27:26.090 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-06T20:27:26.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-06T20:27:26.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-06T20:27:26.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-11-06T20:27:26.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-11-06T20:27:26.091 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-06T20:27:26.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-11-06T20:27:26.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-11-06T20:27:26.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-06T20:27:26.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-06T20:27:26.092 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-06T20:27:26.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-06T20:27:26.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-06T20:27:26.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-06T20:27:26.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-11-06T20:27:26.093 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-06T20:27:26.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-06T20:27:26.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-11-06T20:27:26.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-11-06T20:27:26.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-11-06T20:27:26.094 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-06T20:27:26.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-11-06T20:27:26.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-11-06T20:27:26.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-11-06T20:27:26.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-06T20:27:26.095 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-11-06T20:27:26.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-06T20:27:26.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-11-06T20:27:26.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-06T20:27:26.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-11-06T20:27:26.096 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-06T20:27:26.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-11-06T20:27:26.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-06T20:27:26.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-11-06T20:27:26.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-06T20:27:26.097 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-11-06T20:27:26.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-06T20:27:26.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-11-06T20:27:26.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-06T20:27:26.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-06T20:27:26.098 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-06T20:27:26.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-06T20:27:26.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-06T20:27:26.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-11-06T20:27:26.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-11-06T20:27:26.099 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-06T20:27:26.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-06T20:27:26.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-06T20:27:26.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-11-06T20:27:26.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-06T20:27:26.100 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-06T20:27:26.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-06T20:27:26.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-06T20:27:26.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-06T20:27:26.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-06T20:27:26.101 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-11-06T20:27:26.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-06T20:27:26.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-11-06T20:27:26.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-06T20:27:26.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-06T20:27:26.102 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-11-06T20:27:26.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-06T20:27:26.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-06T20:27:26.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-11-06T20:27:26.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-06T20:27:26.103 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-11-06T20:27:26.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-06T20:27:26.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-11-06T20:27:26.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-06T20:27:26.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-06T20:27:26.104 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-11-06T20:27:26.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-06T20:27:26.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-06T20:27:26.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-06T20:27:26.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-06T20:27:26.105 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-06T20:27:26.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-11-06T20:27:26.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-06T20:27:26.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-11-06T20:27:26.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-06T20:27:26.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-06T20:27:26.106 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-06T20:27:26.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-06T20:27:26.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-11-06T20:27:26.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-06T20:27:26.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-11-06T20:27:26.107 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-06T20:27:26.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-11-06T20:27:26.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-06T20:27:26.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-11-06T20:27:26.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-06T20:27:26.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-06T20:27:26.108 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-11-06T20:27:26.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-06T20:27:26.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-06T20:27:26.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-06T20:27:26.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-06T20:27:26.109 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-11-06T20:27:26.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-06T20:27:26.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-06T20:27:26.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-06T20:27:26.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-11-06T20:27:26.110 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-06T20:27:26.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-06T20:27:26.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-06T20:27:26.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-06T20:27:26.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-11-06T20:27:26.111 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-06T20:27:26.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-11-06T20:27:26.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-06T20:27:26.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-11-06T20:27:26.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-06T20:27:26.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-06T20:27:26.112 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-11-06T20:27:26.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-06T20:27:26.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-06T20:27:26.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-11-06T20:27:26.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-06T20:27:26.113 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-11-06T20:27:26.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-11-06T20:27:26.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-06T20:27:26.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-11-06T20:27:26.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-06T20:27:26.114 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-11-06T20:27:26.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-11-06T20:27:26.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-06T20:27:26.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-11-06T20:27:26.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-11-06T20:27:26.115 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-11-06T20:27:26.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-11-06T20:27:26.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-06T20:27:26.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-06T20:27:26.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-06T20:27:26.116 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-11-06T20:27:26.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-11-06T20:27:26.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-11-06T20:27:26.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-11-06T20:27:26.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-11-06T20:27:26.117 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-06T20:27:26.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-06T20:27:26.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-11-06T20:27:26.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-11-06T20:27:26.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-06T20:27:26.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-06T20:27:26.118 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-06T20:27:26.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-11-06T20:27:26.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-11-06T20:27:26.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-11-06T20:27:26.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-11-06T20:27:26.119 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-11-06T20:27:26.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-11-06T20:27:26.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-11-06T20:27:26.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-11-06T20:27:26.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-11-06T20:27:26.120 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-11-06T20:27:26.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-11-06T20:27:26.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-11-06T20:27:26.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-11-06T20:27:26.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-11-06T20:27:26.121 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-11-06T20:27:26.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-11-06T20:27:26.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-11-06T20:27:26.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-11-06T20:27:26.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-06T20:27:26.122 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-11-06T20:27:26.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-06T20:27:26.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-06T20:27:26.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-06T20:27:26.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-06T20:27:26.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-11-06T20:27:26.123 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-11-06T20:27:26.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-11-06T20:27:26.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-11-06T20:27:26.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-11-06T20:27:26.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-11-06T20:27:26.124 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-11-06T20:27:26.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-11-06T20:27:26.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-11-06T20:27:26.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-11-06T20:27:26.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-06T20:27:26.125 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-06T20:27:26.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-06T20:27:26.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-11-06T20:27:26.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-06T20:27:26.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-11-06T20:27:26.126 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-11-06T20:27:26.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-11-06T20:27:26.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-11-06T20:27:26.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-06T20:27:26.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-11-06T20:27:26.127 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-11-06T20:27:26.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-11-06T20:27:26.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-06T20:27:26.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-11-06T20:27:26.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-11-06T20:27:26.128 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-11-06T20:27:26.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-11-06T20:27:26.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-11-06T20:27:26.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-11-06T20:27:26.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-11-06T20:27:26.129 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-11-06T20:27:26.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-11-06T20:27:26.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-11-06T20:27:26.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-11-06T20:27:26.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-11-06T20:27:26.130 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-11-06T20:27:26.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-11-06T20:27:26.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-06T20:27:26.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-11-06T20:27:26.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-06T20:27:26.131 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-11-06T20:27:26.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-11-06T20:27:26.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-11-06T20:27:26.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-11-06T20:27:26.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-06T20:27:26.132 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-11-06T20:27:26.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-11-06T20:27:26.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-06T20:27:26.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-11-06T20:27:26.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-11-06T20:27:26.133 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-06T20:27:26.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-11-06T20:27:26.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-06T20:27:26.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-06T20:27:26.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-06T20:27:26.134 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-06T20:27:26.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-06T20:27:26.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-06T20:27:26.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-06T20:27:26.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-06T20:27:26.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-06T20:27:26.135 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-11-06T20:27:26.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-11-06T20:27:26.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-06T20:27:26.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-11-06T20:27:26.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-06T20:27:26.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-06T20:27:26.136 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-06T20:27:26.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-06T20:27:26.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-06T20:27:26.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-06T20:27:26.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-06T20:27:26.137 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-06T20:27:26.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-06T20:27:26.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-11-06T20:27:26.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-11-06T20:27:26.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-06T20:27:26.138 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-06T20:27:26.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-11-06T20:27:26.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-11-06T20:27:26.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-06T20:27:26.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-06T20:27:26.139 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-11-06T20:27:26.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-11-06T20:27:26.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-11-06T20:27:26.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-11-06T20:27:26.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-11-06T20:27:26.140 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-11-06T20:27:26.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-06T20:27:26.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-06T20:27:26.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-06T20:27:26.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-06T20:27:26.141 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-06T20:27:26.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-06T20:27:26.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-06T20:27:26.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-06T20:27:26.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-06T20:27:26.142 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-06T20:27:26.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-11-06T20:27:26.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-06T20:27:26.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-06T20:27:26.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-06T20:27:26.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-06T20:27:26.143 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-06T20:27:26.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-06T20:27:26.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-11-06T20:27:26.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-06T20:27:26.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-06T20:27:26.144 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-06T20:27:26.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-06T20:27:26.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-06T20:27:26.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-06T20:27:26.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-06T20:27:26.145 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-06T20:27:26.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-06T20:27:26.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-06T20:27:26.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-06T20:27:26.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-06T20:27:26.146 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-11-06T20:27:26.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-11-06T20:27:26.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-11-06T20:27:26.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-11-06T20:27:26.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-11-06T20:27:26.147 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-11-06T20:27:26.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-11-06T20:27:26.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-11-06T20:27:26.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-06T20:27:26.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-11-06T20:27:26.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-11-06T20:27:26.148 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-06T20:27:26.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-06T20:27:26.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-11-06T20:27:26.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-11-06T20:27:26.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-06T20:27:26.149 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-11-06T20:27:26.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-11-06T20:27:26.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-06T20:27:26.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-11-06T20:27:26.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-11-06T20:27:26.150 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-11-06T20:27:26.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-06T20:27:26.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-11-06T20:27:26.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-06T20:27:26.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-11-06T20:27:26.151 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-11-06T20:27:26.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-06T20:27:26.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-11-06T20:27:26.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-06T20:27:26.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-06T20:27:26.152 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-06T20:27:26.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-06T20:27:26.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-06T20:27:26.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-11-06T20:27:26.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-11-06T20:27:26.153 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-06T20:27:26.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-06T20:27:26.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-06T20:27:26.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-06T20:27:26.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-11-06T20:27:26.154 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-11-06T20:27:26.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-06T20:27:26.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-11-06T20:27:26.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-11-06T20:27:26.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-06T20:27:26.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-06T20:27:26.155 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-11-06T20:27:26.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-11-06T20:27:26.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-11-06T20:27:26.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-11-06T20:27:26.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-11-06T20:27:26.156 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-11-06T20:27:26.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-11-06T20:27:26.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-11-06T20:27:26.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-11-06T20:27:26.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-11-06T20:27:26.157 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-06T20:27:26.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-06T20:27:26.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-11-06T20:27:26.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-11-06T20:27:26.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-11-06T20:27:26.158 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-06T20:27:26.159 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-11-06T20:27:26.159 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-11-06T20:27:26.159 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-11-06T20:27:26.159 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-06T20:27:26.159 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-11-06T20:27:26.160 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-11-06T20:27:26.160 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-06T20:27:26.160 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-06T20:27:26.160 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-06T20:27:26.160 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-11-06T20:27:26.160 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-06T20:27:26.161 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-11-06T20:27:26.161 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-06T20:27:26.161 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-11-06T20:27:26.161 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-11-06T20:27:26.161 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-06T20:27:26.162 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-11-06T20:27:26.162 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-11-06T20:27:26.162 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-11-06T20:27:26.162 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-06T20:27:26.162 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-06T20:27:26.163 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-11-06T20:27:26.163 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-06T20:27:26.163 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-11-06T20:27:26.163 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-11-06T20:27:26.163 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-06T20:27:26.164 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-06T20:27:26.164 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-06T20:27:26.164 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-06T20:27:26.164 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-06T20:27:26.164 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-11-06T20:27:26.164 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-11-06T20:27:26.165 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-11-06T20:27:26.165 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-11-06T20:27:26.165 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-06T20:27:26.165 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-11-06T20:27:26.165 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-06T20:27:26.165 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-11-06T20:27:26.166 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-11-06T20:27:26.166 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-11-06T20:27:26.166 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-11-06T20:27:26.166 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-11-06T20:27:26.166 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-06T20:27:26.166 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-11-06T20:27:26.167 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-06T20:27:26.167 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-11-06T20:27:26.167 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-11-06T20:27:26.167 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-11-06T20:27:26.167 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-11-06T20:27:26.168 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-11-06T20:27:26.168 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-11-06T20:27:26.168 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-06T20:27:26.168 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-11-06T20:27:26.168 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-11-06T20:27:26.169 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-11-06T20:27:26.169 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-06T20:27:26.169 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-06T20:27:26.169 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-06T20:27:26.169 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-06T20:27:26.170 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-06T20:27:26.170 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-06T20:27:26.170 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-06T20:27:26.170 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-06T20:27:26.170 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-06T20:27:26.170 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-06T20:27:26.171 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-06T20:27:26.171 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-06T20:27:26.171 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-06T20:27:26.171 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-06T20:27:26.171 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-06T20:27:26.172 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-11-06T20:27:26.172 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-11-06T20:27:26.172 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-06T20:27:26.172 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-11-06T20:27:26.172 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-11-06T20:27:26.173 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-11-06T20:27:26.173 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-11-06T20:27:26.173 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-11-06T20:27:26.173 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-06T20:27:26.173 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-11-06T20:27:26.174 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-11-06T20:27:26.174 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-06T20:27:26.174 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-11-06T20:27:26.174 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-11-06T20:27:26.174 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-06T20:27:26.174 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-11-06T20:27:26.175 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-06T20:27:26.175 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-11-06T20:27:26.175 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-11-06T20:27:26.175 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-11-06T20:27:26.175 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-11-06T20:27:26.176 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-11-06T20:27:26.176 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-11-06T20:27:26.176 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-11-06T20:27:26.176 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-11-06T20:27:26.176 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-11-06T20:27:26.177 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-06T20:27:26.177 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-11-06T20:27:26.177 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-11-06T20:27:26.177 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-11-06T20:27:26.177 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-06T20:27:26.178 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-11-06T20:27:26.178 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-11-06T20:27:26.178 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-11-06T20:27:26.178 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-11-06T20:27:26.178 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-11-06T20:27:26.179 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-11-06T20:27:26.179 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-11-06T20:27:26.179 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-11-06T20:27:26.179 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-11-06T20:27:26.179 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-11-06T20:27:26.180 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-11-06T20:27:26.180 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-06T20:27:26.180 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-11-06T20:27:26.180 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-11-06T20:27:26.180 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-11-06T20:27:26.180 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-06T20:27:26.181 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-06T20:27:26.181 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-11-06T20:27:26.181 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-11-06T20:27:26.181 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-06T20:27:26.181 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-11-06T20:27:26.181 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-11-06T20:27:26.182 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-11-06T20:27:26.182 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-11-06T20:27:26.182 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-11-06T20:27:26.182 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-11-06T20:27:26.182 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-11-06T20:27:26.183 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-06T20:27:26.183 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-11-06T20:27:26.183 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-11-06T20:27:26.183 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-11-06T20:27:26.183 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-06T20:27:26.184 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-06T20:27:26.184 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-11-06T20:27:26.184 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-06T20:27:26.184 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-06T20:27:26.184 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-06T20:27:26.185 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-11-06T20:27:26.185 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-11-06T20:27:26.185 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-11-06T20:27:26.185 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-11-06T20:27:26.185 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-06T20:27:26.185 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-11-06T20:27:26.186 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-06T20:27:26.186 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-06T20:27:26.186 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-06T20:27:26.186 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-11-06T20:27:26.186 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-11-06T20:27:26.187 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-11-06T20:27:26.187 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-11-06T20:27:26.187 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-11-06T20:27:26.187 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-11-06T20:27:26.187 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-11-06T20:27:26.187 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-11-06T20:27:26.188 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-11-06T20:27:26.188 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-11-06T20:27:26.188 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-11-06T20:27:26.188 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-11-06T20:27:26.188 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-11-06T20:27:26.188 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-11-06T20:27:26.189 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-06T20:27:26.189 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-11-06T20:27:26.189 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-11-06T20:27:26.189 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-06T20:27:26.189 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-06T20:27:26.190 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-11-06T20:27:26.190 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-06T20:27:26.190 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-11-06T20:27:26.190 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-11-06T20:27:26.190 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-06T20:27:26.191 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-11-06T20:27:26.191 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-11-06T20:27:26.191 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-06T20:27:26.191 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-06T20:27:26.191 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-06T20:27:26.191 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-06T20:27:26.192 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-06T20:27:26.192 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-06T20:27:26.192 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-06T20:27:26.192 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-11-06T20:27:26.192 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-06T20:27:26.193 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-06T20:27:26.193 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-06T20:27:26.193 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-06T20:27:26.193 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-06T20:27:26.193 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-11-06T20:27:26.194 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-11-06T20:27:26.194 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-06T20:27:26.194 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-06T20:27:26.194 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-06T20:27:26.194 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-11-06T20:27:26.195 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-11-06T20:27:26.195 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-06T20:27:26.195 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-06T20:27:26.195 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-11-06T20:27:26.195 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-06T20:27:26.195 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-11-06T20:27:26.196 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-11-06T20:27:26.196 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-06T20:27:26.196 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-11-06T20:27:26.196 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-11-06T20:27:26.196 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-11-06T20:27:26.197 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-11-06T20:27:26.197 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-11-06T20:27:26.197 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-11-06T20:27:26.197 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-06T20:27:26.197 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-11-06T20:27:26.197 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-11-06T20:27:26.198 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-11-06T20:27:26.198 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-11-06T20:27:26.198 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-11-06T20:27:26.198 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-06T20:27:26.198 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-06T20:27:26.198 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-11-06T20:27:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-11-06T20:27:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-11-06T20:27:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-11-06T20:27:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-11-06T20:27:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-11-06T20:27:26.200 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-06T20:27:26.200 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-06T20:27:26.200 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-06T20:27:26.200 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-06T20:27:26.200 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-06T20:27:26.200 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-06T20:27:26.201 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-06T20:27:26.201 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-06T20:27:26.201 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-06T20:27:26.201 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-06T20:27:26.202 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-06T20:27:26.202 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-11-06T20:27:26.202 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-11-06T20:27:26.202 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-06T20:27:26.202 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-06T20:27:26.202 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-06T20:27:26.203 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-11-06T20:27:26.203 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-11-06T20:27:26.203 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-06T20:27:26.203 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-06T20:27:26.203 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-11-06T20:27:26.204 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-11-06T20:27:26.204 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-06T20:27:26.204 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-06T20:27:26.204 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-11-06T20:27:26.204 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-11-06T20:27:26.205 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-11-06T20:27:26.205 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-06T20:27:26.205 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-06T20:27:26.205 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-06T20:27:26.205 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-06T20:27:26.206 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-11-06T20:27:26.206 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-06T20:27:26.206 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-11-06T20:27:26.206 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-06T20:27:26.206 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-06T20:27:26.207 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-06T20:27:26.207 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-11-06T20:27:26.207 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-11-06T20:27:26.207 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-11-06T20:27:26.207 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-06T20:27:26.208 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-11-06T20:27:26.208 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-11-06T20:27:26.208 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-11-06T20:27:26.208 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-11-06T20:27:26.208 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-06T20:27:26.208 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-11-06T20:27:26.209 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-11-06T20:27:26.209 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-06T20:27:26.209 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-11-06T20:27:26.209 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-11-06T20:27:26.209 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-11-06T20:27:26.210 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-11-06T20:27:26.210 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-11-06T20:27:26.210 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-11-06T20:27:26.210 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-11-06T20:27:26.210 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-11-06T20:27:26.210 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-11-06T20:27:26.211 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-11-06T20:27:26.211 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-11-06T20:27:26.211 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-11-06T20:27:26.211 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-11-06T20:27:26.211 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-11-06T20:27:26.212 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-11-06T20:27:26.212 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-11-06T20:27:26.212 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-11-06T20:27:26.212 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-11-06T20:27:26.212 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-11-06T20:27:26.212 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-11-06T20:27:26.213 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-11-06T20:27:26.213 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-06T20:27:26.213 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-11-06T20:27:26.213 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-11-06T20:27:26.213 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-11-06T20:27:26.214 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-11-06T20:27:26.214 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-11-06T20:27:26.214 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-11-06T20:27:26.214 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-11-06T20:27:26.214 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-11-06T20:27:26.215 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-11-06T20:27:26.215 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-11-06T20:27:26.215 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-11-06T20:27:26.215 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-11-06T20:27:26.215 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-11-06T20:27:26.216 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-11-06T20:27:26.216 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-11-06T20:27:26.216 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-11-06T20:27:26.216 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-11-06T20:27:26.216 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-11-06T20:27:26.216 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-11-06T20:27:26.217 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-11-06T20:27:26.217 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-11-06T20:27:26.217 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-11-06T20:27:26.217 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-11-06T20:27:26.217 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-11-06T20:27:26.218 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-11-06T20:27:26.218 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-11-06T20:27:26.218 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-11-06T20:27:26.218 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-11-06T20:27:26.218 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-11-06T20:27:26.219 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-06T20:27:26.219 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-11-06T20:27:26.219 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-06T20:27:26.219 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-11-06T20:27:26.219 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-06T20:27:26.220 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-11-06T20:27:26.220 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-11-06T20:27:26.220 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-11-06T20:27:26.220 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-11-06T20:27:26.220 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-11-06T20:27:26.221 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-11-06T20:27:26.221 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-11-06T20:27:26.221 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-11-06T20:27:26.221 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-11-06T20:27:26.221 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-11-06T20:27:26.222 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-11-06T20:27:26.222 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-11-06T20:27:26.222 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-11-06T20:27:26.222 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-11-06T20:27:26.222 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-11-06T20:27:26.223 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-11-06T20:27:26.223 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-11-06T20:27:26.223 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-11-06T20:27:26.223 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-11-06T20:27:26.223 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-11-06T20:27:26.224 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-11-06T20:27:26.224 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-11-06T20:27:26.224 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-11-06T20:27:26.224 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-11-06T20:27:26.224 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-11-06T20:27:26.225 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-11-06T20:27:26.225 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-11-06T20:27:26.225 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-11-06T20:27:26.225 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-11-06T20:27:26.225 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-11-06T20:27:26.226 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-11-06T20:27:26.226 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-11-06T20:27:26.226 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-11-06T20:27:26.226 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-11-06T20:27:26.226 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-11-06T20:27:26.227 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-11-06T20:27:26.227 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-11-06T20:27:26.227 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-11-06T20:27:26.227 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-11-06T20:27:26.227 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-11-06T20:27:26.227 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-11-06T20:27:26.228 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-11-06T20:27:26.228 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-11-06T20:27:26.228 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-11-06T20:27:26.228 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-11-06T20:27:26.228 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-11-06T20:27:26.229 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-11-06T20:27:26.229 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-11-06T20:27:26.229 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-11-06T20:27:26.229 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-11-06T20:27:26.229 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-11-06T20:27:26.230 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-11-06T20:27:26.230 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-11-06T20:27:26.230 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-11-06T20:27:26.230 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-11-06T20:27:26.230 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-11-06T20:27:26.231 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-11-06T20:27:26.231 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-11-06T20:27:26.231 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-11-06T20:27:26.231 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-06T20:27:26.231 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-11-06T20:27:26.232 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-11-06T20:27:26.232 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-11-06T20:27:26.232 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-06T20:27:26.232 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-11-06T20:27:26.232 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-06T20:27:26.233 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-11-06T20:27:26.233 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-06T20:27:26.233 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-11-06T20:27:26.233 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-06T20:27:26.233 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-06T20:27:26.234 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-06T20:27:26.234 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-11-06T20:27:26.234 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-11-06T20:27:26.234 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-11-06T20:27:26.234 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-11-06T20:27:26.235 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-11-06T20:27:26.235 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-11-06T20:27:26.235 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-11-06T20:27:26.235 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-11-06T20:27:26.235 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-11-06T20:27:26.235 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-11-06T20:27:26.236 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-11-06T20:27:26.236 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-11-06T20:27:26.236 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-11-06T20:27:26.236 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-11-06T20:27:26.236 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-11-06T20:27:26.237 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-11-06T20:27:26.237 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-11-06T20:27:26.237 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-11-06T20:27:26.237 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-11-06T20:27:26.238 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-11-06T20:27:26.238 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-11-06T20:27:26.238 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-11-06T20:27:26.238 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-11-06T20:27:26.238 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-11-06T20:27:26.238 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-11-06T20:27:26.239 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-11-06T20:27:26.239 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-11-06T20:27:26.239 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-11-06T20:27:26.239 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-11-06T20:27:26.239 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-06T20:27:26.240 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-11-06T20:27:26.240 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-11-06T20:27:26.240 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-11-06T20:27:26.240 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-11-06T20:27:26.240 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-11-06T20:27:26.241 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-11-06T20:27:26.241 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-11-06T20:27:26.241 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-11-06T20:27:26.241 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-11-06T20:27:26.241 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-11-06T20:27:26.241 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-11-06T20:27:26.242 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-11-06T20:27:26.242 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-11-06T20:27:26.242 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-11-06T20:27:26.242 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-11-06T20:27:26.242 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-11-06T20:27:26.243 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-11-06T20:27:26.243 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-06T20:27:26.243 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-11-06T20:27:26.243 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-11-06T20:27:26.243 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-11-06T20:27:26.244 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-11-06T20:27:26.244 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-11-06T20:27:26.244 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-11-06T20:27:26.244 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-11-06T20:27:26.244 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-11-06T20:27:26.245 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-11-06T20:27:26.245 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-11-06T20:27:26.245 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-11-06T20:27:26.245 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-11-06T20:27:26.245 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-11-06T20:27:26.246 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-06T20:27:26.246 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-11-06T20:27:26.246 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-11-06T20:27:26.246 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-11-06T20:27:26.246 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-11-06T20:27:26.247 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-11-06T20:27:26.247 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-11-06T20:27:26.247 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-11-06T20:27:26.247 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-11-06T20:27:26.247 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-11-06T20:27:26.247 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-11-06T20:27:26.248 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-11-06T20:27:26.248 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-11-06T20:27:26.248 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-11-06T20:27:26.248 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-11-06T20:27:26.248 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-11-06T20:27:26.249 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-11-06T20:27:26.249 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-11-06T20:27:26.249 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-11-06T20:27:26.249 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-11-06T20:27:26.249 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-11-06T20:27:26.250 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-11-06T20:27:26.250 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-11-06T20:27:26.250 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-06T20:27:26.250 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-06T20:27:26.250 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-06T20:27:26.251 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-06T20:27:26.251 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-06T20:27:26.251 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-06T20:27:26.251 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-06T20:27:26.251 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-06T20:27:26.252 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-06T20:27:26.252 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-06T20:27:26.252 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-06T20:27:26.252 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-06T20:27:26.252 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-06T20:27:26.252 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-06T20:27:26.253 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-06T20:27:26.253 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-06T20:27:26.253 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-06T20:27:26.253 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-06T20:27:26.253 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-06T20:27:26.254 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-06T20:27:26.254 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-11-06T20:27:26.254 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-06T20:27:26.254 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-06T20:27:26.254 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-06T20:27:26.255 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-11-06T20:27:26.255 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-11-06T20:27:26.255 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-11-06T20:27:26.255 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-11-06T20:27:26.255 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-11-06T20:27:26.256 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-11-06T20:27:26.256 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-11-06T20:27:26.256 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-11-06T20:27:26.256 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-11-06T20:27:26.256 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-11-06T20:27:26.257 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-11-06T20:27:26.257 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-11-06T20:27:26.257 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-11-06T20:27:26.257 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-11-06T20:27:26.257 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-11-06T20:27:26.257 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-11-06T20:27:26.258 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-11-06T20:27:26.258 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-11-06T20:27:26.258 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-11-06T20:27:26.258 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-11-06T20:27:26.258 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-06T20:27:26.259 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-11-06T20:27:26.259 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-06T20:27:26.259 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-11-06T20:27:26.259 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-11-06T20:27:26.259 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-11-06T20:27:26.260 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-11-06T20:27:26.260 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-11-06T20:27:26.260 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-11-06T20:27:26.260 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-11-06T20:27:26.260 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-11-06T20:27:26.261 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-11-06T20:27:26.261 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-11-06T20:27:26.261 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-11-06T20:27:26.261 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-11-06T20:27:26.261 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-06T20:27:26.262 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-06T20:27:26.262 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-06T20:27:26.262 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-06T20:27:26.262 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-06T20:27:26.262 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-06T20:27:26.262 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-06T20:27:26.263 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-06T20:27:26.263 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-06T20:27:26.263 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-06T20:27:26.263 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-06T20:27:26.263 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-06T20:27:26.264 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-06T20:27:26.264 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-06T20:27:26.264 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-06T20:27:26.264 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-06T20:27:26.264 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-06T20:27:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-06T20:27:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-06T20:27:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-06T20:27:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-06T20:27:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-11-06T20:27:26.266 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-11-06T20:27:26.266 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-11-06T20:27:26.266 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-11-06T20:27:26.266 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-11-06T20:27:26.267 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-11-06T20:27:26.267 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-11-06T20:27:26.267 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-11-06T20:27:26.267 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-11-06T20:27:26.267 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-11-06T20:27:26.267 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-11-06T20:27:26.268 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-11-06T20:27:26.268 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-11-06T20:27:26.268 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-11-06T20:27:26.268 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-11-06T20:27:26.268 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-11-06T20:27:26.269 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-06T20:27:26.269 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-11-06T20:27:26.269 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-11-06T20:27:26.269 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-11-06T20:27:26.269 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-11-06T20:27:26.270 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-11-06T20:27:26.270 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-06T20:27:26.270 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-11-06T20:27:26.270 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-11-06T20:27:26.270 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-11-06T20:27:26.271 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-06T20:27:26.271 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-06T20:27:26.271 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-06T20:27:26.271 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-06T20:27:26.271 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-06T20:27:26.272 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-06T20:27:26.272 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-06T20:27:26.272 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-06T20:27:26.272 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-06T20:27:26.272 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-06T20:27:26.272 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-06T20:27:26.273 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-06T20:27:26.273 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-06T20:27:26.273 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-06T20:27:26.273 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-06T20:27:26.273 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-06T20:27:26.274 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-06T20:27:26.274 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-06T20:27:26.274 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-06T20:27:26.274 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-06T20:27:26.274 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-06T20:27:26.275 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-06T20:27:26.275 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-06T20:27:26.275 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-06T20:27:26.275 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-11-06T20:27:26.275 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-11-06T20:27:26.276 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-06T20:27:26.276 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-11-06T20:27:26.276 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-11-06T20:27:26.276 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-11-06T20:27:26.276 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-11-06T20:27:26.277 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-06T20:27:26.277 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-11-06T20:27:26.277 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-11-06T20:27:26.277 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-11-06T20:27:26.277 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-11-06T20:27:26.277 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-11-06T20:27:26.278 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-11-06T20:27:26.278 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-11-06T20:27:26.278 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-11-06T20:27:26.278 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-11-06T20:27:26.278 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-11-06T20:27:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-11-06T20:27:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-11-06T20:27:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-11-06T20:27:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-11-06T20:27:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-06T20:27:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-06T20:27:26.280 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-06T20:27:26.280 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-06T20:27:26.280 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-06T20:27:26.280 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-06T20:27:26.280 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-06T20:27:26.281 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-06T20:27:26.281 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-06T20:27:26.281 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-06T20:27:26.281 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-06T20:27:26.281 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-06T20:27:26.282 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-06T20:27:26.282 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-06T20:27:26.282 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-06T20:27:26.282 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-11-06T20:27:26.282 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-06T20:27:26.283 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-06T20:27:26.283 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-06T20:27:26.283 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-06T20:27:26.283 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-06T20:27:26.283 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-06T20:27:26.283 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-06T20:27:26.284 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-06T20:27:26.284 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-06T20:27:26.284 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-06T20:27:26.284 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-06T20:27:26.284 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-06T20:27:26.285 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-06T20:27:26.285 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-06T20:27:26.285 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-11-06T20:27:26.285 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-11-06T20:27:26.285 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-06T20:27:26.286 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-06T20:27:26.286 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-06T20:27:26.286 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-06T20:27:26.286 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-06T20:27:26.286 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-06T20:27:26.287 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-06T20:27:26.287 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-06T20:27:26.287 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-06T20:27:26.287 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-06T20:27:26.287 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-06T20:27:26.287 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-06T20:27:26.288 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-06T20:27:26.288 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-11-06T20:27:26.288 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-06T20:27:26.288 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-11-06T20:27:26.288 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-11-06T20:27:26.289 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-06T20:27:26.289 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-06T20:27:26.289 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-11-06T20:27:26.289 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-11-06T20:27:26.289 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-06T20:27:26.290 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-06T20:27:26.290 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-11-06T20:27:26.290 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-11-06T20:27:26.290 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-06T20:27:26.290 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-11-06T20:27:26.290 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-06T20:27:26.291 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-06T20:27:26.291 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-11-06T20:27:26.291 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-11-06T20:27:26.291 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-11-06T20:27:26.291 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-06T20:27:26.292 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-11-06T20:27:26.292 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-06T20:27:26.292 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-11-06T20:27:26.292 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-11-06T20:27:26.292 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-11-06T20:27:26.293 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-11-06T20:27:26.293 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-11-06T20:27:26.293 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-11-06T20:27:26.293 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-11-06T20:27:26.293 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-11-06T20:27:26.293 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-11-06T20:27:26.294 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-11-06T20:27:26.294 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-11-06T20:27:26.294 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-11-06T20:27:26.294 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-11-06T20:27:26.294 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-11-06T20:27:26.295 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-06T20:27:26.295 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-06T20:27:26.295 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-06T20:27:26.295 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-11-06T20:27:26.295 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-11-06T20:27:26.296 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-11-06T20:27:26.296 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-11-06T20:27:26.296 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-06T20:27:26.296 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-11-06T20:27:26.296 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-11-06T20:27:26.297 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-06T20:27:26.297 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-11-06T20:27:26.297 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-11-06T20:27:26.297 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-06T20:27:26.297 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-11-06T20:27:26.298 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-06T20:27:26.298 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-11-06T20:27:26.298 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-06T20:27:26.298 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-06T20:27:26.298 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-11-06T20:27:26.298 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-11-06T20:27:26.299 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-11-06T20:27:26.299 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-11-06T20:27:26.299 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-11-06T20:27:26.299 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-11-06T20:27:26.299 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-11-06T20:27:26.300 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-11-06T20:27:26.300 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-11-06T20:27:26.300 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-11-06T20:27:26.300 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-11-06T20:27:26.300 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-11-06T20:27:26.301 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-11-06T20:27:26.301 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-11-06T20:27:26.301 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-11-06T20:27:26.301 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-11-06T20:27:26.301 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-11-06T20:27:26.301 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-11-06T20:27:26.302 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-11-06T20:27:26.302 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-11-06T20:27:26.302 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-11-06T20:27:26.302 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-11-06T20:27:26.302 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-11-06T20:27:26.303 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-11-06T20:27:26.303 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-11-06T20:27:26.303 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-11-06T20:27:26.303 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-11-06T20:27:26.303 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-11-06T20:27:26.303 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-11-06T20:27:26.304 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-11-06T20:27:26.304 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-11-06T20:27:26.304 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-11-06T20:27:26.304 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-11-06T20:27:26.304 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-11-06T20:27:26.305 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-11-06T20:27:26.305 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-11-06T20:27:26.305 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-11-06T20:27:26.305 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-11-06T20:27:26.305 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-11-06T20:27:26.306 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-11-06T20:27:26.306 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-11-06T20:27:26.306 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-11-06T20:27:26.306 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-11-06T20:27:26.306 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-11-06T20:27:26.306 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-11-06T20:27:26.307 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-11-06T20:27:26.307 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-11-06T20:27:26.307 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-11-06T20:27:26.307 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-11-06T20:27:26.307 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-11-06T20:27:26.308 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-11-06T20:27:26.308 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-11-06T20:27:26.308 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-11-06T20:27:26.308 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-11-06T20:27:26.308 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-11-06T20:27:26.309 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-11-06T20:27:26.309 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-11-06T20:27:26.309 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-11-06T20:27:26.309 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-11-06T20:27:26.309 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-11-06T20:27:26.310 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-11-06T20:27:26.310 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-11-06T20:27:26.310 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-11-06T20:27:26.310 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-11-06T20:27:26.310 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-11-06T20:27:26.311 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-11-06T20:27:26.311 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-11-06T20:27:26.311 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-11-06T20:27:26.311 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-11-06T20:27:26.311 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-11-06T20:27:26.311 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-11-06T20:27:26.312 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-11-06T20:27:26.312 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-11-06T20:27:26.312 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-11-06T20:27:26.312 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-11-06T20:27:26.313 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-11-06T20:27:26.313 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-11-06T20:27:26.313 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-11-06T20:27:26.313 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-11-06T20:27:26.313 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-11-06T20:27:26.313 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-11-06T20:27:26.314 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-11-06T20:27:26.314 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-11-06T20:27:26.314 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-11-06T20:27:26.314 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-11-06T20:27:26.314 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-11-06T20:27:26.315 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-11-06T20:27:26.315 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-11-06T20:27:26.315 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-11-06T20:27:26.315 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-11-06T20:27:26.315 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-11-06T20:27:26.316 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-11-06T20:27:26.316 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-11-06T20:27:26.316 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-11-06T20:27:26.316 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-11-06T20:27:26.316 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-11-06T20:27:26.316 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-11-06T20:27:26.317 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-11-06T20:27:26.317 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-11-06T20:27:26.317 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-11-06T20:27:26.317 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-11-06T20:27:26.317 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-11-06T20:27:26.318 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-11-06T20:27:26.318 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-11-06T20:27:26.318 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-11-06T20:27:26.318 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-11-06T20:27:26.318 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-11-06T20:27:26.319 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-11-06T20:27:26.319 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-11-06T20:27:26.319 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-11-06T20:27:26.319 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-11-06T20:27:26.319 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-11-06T20:27:26.320 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-11-06T20:27:26.320 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-11-06T20:27:26.320 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-11-06T20:27:26.320 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-11-06T20:27:26.320 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-06T20:27:26.320 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-06T20:27:26.321 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-11-06T20:27:26.321 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-11-06T20:27:26.321 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-06T20:27:26.321 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-11-06T20:27:26.321 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-11-06T20:27:26.322 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-11-06T20:27:26.322 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-11-06T20:27:26.322 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-11-06T20:27:26.322 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-11-06T20:27:26.322 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-06T20:27:26.323 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-06T20:27:26.323 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-11-06T20:27:26.323 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-06T20:27:26.323 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-06T20:27:26.323 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-11-06T20:27:26.324 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-06T20:27:26.324 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-11-06T20:27:26.324 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-11-06T20:27:26.324 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-11-06T20:27:26.324 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-06T20:27:26.325 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-11-06T20:27:26.325 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-11-06T20:27:26.325 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-11-06T20:27:26.325 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-11-06T20:27:26.325 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-11-06T20:27:26.325 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-11-06T20:27:26.326 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-11-06T20:27:26.326 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-11-06T20:27:26.326 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-11-06T20:27:26.326 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-11-06T20:27:26.326 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-11-06T20:27:26.327 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-11-06T20:27:26.327 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-11-06T20:27:26.327 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-11-06T20:27:26.327 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-11-06T20:27:26.327 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-11-06T20:27:26.328 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-11-06T20:27:26.328 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:26.329 DEBUG:teuthology.orchestra.run.smithi033:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-06T20:27:26.358 INFO:teuthology.orchestra.run.smithi033.stdout:postinstall scriptlet (using /bin/sh): 2023-11-06T20:27:26.358 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-11-06T20:27:26.358 INFO:teuthology.orchestra.run.smithi033.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-06T20:27:26.358 INFO:teuthology.orchestra.run.smithi033.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-06T20:27:26.359 INFO:teuthology.orchestra.run.smithi033.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:26.359 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-06T20:27:26.359 INFO:teuthology.orchestra.run.smithi033.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-06T20:27:26.359 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2023-11-06T20:27:26.359 INFO:teuthology.orchestra.run.smithi033.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-06T20:27:26.360 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-06T20:27:26.360 INFO:teuthology.orchestra.run.smithi033.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-11-06T20:27:26.360 INFO:teuthology.orchestra.run.smithi033.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-06T20:27:26.360 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:26.360 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2023-11-06T20:27:26.361 INFO:teuthology.orchestra.run.smithi033.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-06T20:27:26.361 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-06T20:27:26.361 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:26.361 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2023-11-06T20:27:26.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2-tools 2023-11-06T20:27:26.444 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2-tools 2023-11-06T20:27:26.445 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2023-11-06T20:27:26.445 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.02 2023-11-06T20:27:26.445 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 129.el8 2023-11-06T20:27:26.445 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2023-11-06T20:27:26.445 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-11-06T20:27:26.446 INFO:teuthology.orchestra.run.smithi033.stdout:Group : System Environment/Base 2023-11-06T20:27:26.446 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 9265586 2023-11-06T20:27:26.446 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2023-11-06T20:27:26.446 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-11-06T20:27:26.446 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-11-06T20:27:26.447 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-11-06T20:27:26.447 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : kbuilder.bsys.centos.org 2023-11-06T20:27:26.447 INFO:teuthology.orchestra.run.smithi033.stdout:Relocations : (not relocatable) 2023-11-06T20:27:26.447 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : CentOS BuildSystem 2023-11-06T20:27:26.447 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : CentOS 2023-11-06T20:27:26.447 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-06T20:27:26.448 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Support tools for GRUB. 2023-11-06T20:27:26.448 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2023-11-06T20:27:26.448 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:27:26.448 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-06T20:27:26.448 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-06T20:27:26.449 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-06T20:27:26.449 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2023-11-06T20:27:26.449 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:27:26.449 INFO:teuthology.orchestra.run.smithi033.stdout:This subpackage provides tools for support of all platforms. 2023-11-06T20:27:26.450 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-06T20:27:26.451 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:26.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-06T20:27:30.336 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2023-11-06T20:27:32.598 INFO:teuthology.orchestra.run.smithi033.stderr:done 2023-11-06T20:27:32.599 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2023-11-06T20:27:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.k8aUKCsrri 2023-11-06T20:27:32.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.k8aUKCsrri 2023-11-06T20:27:32.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.k8aUKCsrri 2023-11-06T20:27:32.854 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.k8aUKCsrri is 6KB 2023-11-06T20:27:32.903 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.k8aUKCsrri 2023-11-06T20:27:32.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo /bin/ls /boot/loader/entries || true 2023-11-06T20:27:32.995 INFO:teuthology.orchestra.run.smithi033.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-11-06T20:27:32.996 INFO:teuthology.orchestra.run.smithi033.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-11-06T20:27:32.996 INFO:teuthology.orchestra.run.smithi033.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-11-06T20:27:32.996 INFO:teuthology.orchestra.run.smithi033.stdout:a1b29d6c289747bdb22930598d39874a-6.6.0-rc5-g5b73517e0e2f.conf 2023-11-06T20:27:32.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:27:33.155 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2023-11-06T20:27:42.150 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-11-06T20:27:42.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-11-06T20:27:42.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T20:28:00.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-11-06T20:28:03.161 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-06T20:28:03.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-11-06T20:28:03.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T20:28:09.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-11-06T20:28:09.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T20:28:12.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-11-06T20:28:21.499 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-11-06T20:28:24.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-11-06T20:28:24.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T20:28:30.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-11-06T20:28:30.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T20:28:48.890 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-11-06T20:28:56.365 DEBUG:teuthology.orchestra.run.smithi069:> true 2023-11-06T20:28:56.786 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-11-06T20:28:56.787 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-rc5-g5b73517e0e2f"... 2023-11-06T20:28:56.787 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2023-11-06T20:28:56.833 INFO:teuthology.orchestra.run.smithi069.stdout:6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:28:56.833 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:28:56.833 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-06T20:28:56.834 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-06T20:28:57.835 DEBUG:teuthology.parallel:result is None 2023-11-06T20:29:00.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-11-06T20:29:00.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T20:29:03.842 DEBUG:teuthology.orchestra.run.smithi033:> true 2023-11-06T20:29:04.322 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-11-06T20:29:04.322 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g5b73517e0e2f"... 2023-11-06T20:29:04.323 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2023-11-06T20:29:04.371 INFO:teuthology.orchestra.run.smithi033.stdout:6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:29:04.371 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-11-06T20:29:04.372 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-06T20:29:04.372 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-06T20:29:05.373 DEBUG:teuthology.parallel:result is None 2023-11-06T20:29:05.373 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-06T20:29:05.381 INFO:teuthology.task.internal:Creating test directory... 2023-11-06T20:29:05.381 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-06T20:29:05.386 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-06T20:29:05.437 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-06T20:29:05.446 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-06T20:29:05.453 INFO:teuthology.task.internal:Creating archive directory... 2023-11-06T20:29:05.454 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-06T20:29:05.458 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-06T20:29:05.527 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-06T20:29:05.534 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-06T20:29:05.535 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-06T20:29:05.539 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-06T20:29:05.659 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T20:29:05.659 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T20:29:05.691 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T20:29:05.695 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T20:29:05.698 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-06T20:29:05.705 INFO:teuthology.task.internal:Configuring sudo... 2023-11-06T20:29:05.706 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-06T20:29:05.710 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-06T20:29:05.780 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-06T20:29:05.789 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-06T20:29:05.789 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-06T20:29:05.794 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-06T20:29:05.840 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T20:29:05.952 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T20:29:06.024 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T20:29:06.099 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T20:29:06.172 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:29:06.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-06T20:29:06.255 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T20:29:06.331 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T20:29:06.401 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T20:29:06.475 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T20:29:06.544 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:29:06.544 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-06T20:29:06.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2023-11-06T20:29:06.619 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2023-11-06T20:29:06.744 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-06T20:29:06.778 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-06T20:29:07.243 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-06T20:29:07.252 INFO:teuthology.task.internal:Starting timer... 2023-11-06T20:29:07.252 INFO:teuthology.run_tasks:Running task pcp... 2023-11-06T20:29:07.263 INFO:teuthology.run_tasks:Running task selinux... 2023-11-06T20:29:07.271 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-06T20:29:07.271 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2023-11-06T20:29:07.401 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2023-11-06T20:29:07.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2023-11-06T20:29:07.535 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2023-11-06T20:29:07.538 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-06T20:29:07.538 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2023-11-06T20:29:07.597 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2023-11-06T20:29:07.597 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2023-11-06T20:29:07.667 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2023-11-06T20:29:07.667 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi069.front.sepia.ceph.com': 'permissive'} 2023-11-06T20:29:07.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-06T20:29:07.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T20:29:07.741 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-06T20:29:07.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T20:29:07.812 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-06T20:29:07.812 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2023-11-06T20:29:07.883 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2023-11-06T20:29:07.954 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-06T20:29:07.964 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-06T20:29:07.970 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-06T20:29:08.100 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-06T20:29:08.117 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-06T20:29:08.118 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi069.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-06T20:40:02.038 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi069.front.sepia.ceph.com')] 2023-11-06T20:40:02.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-11-06T20:40:02.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T20:40:02.127 DEBUG:teuthology.orchestra.run.smithi033:> true 2023-11-06T20:40:02.251 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-11-06T20:40:02.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-11-06T20:40:02.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T20:40:02.336 DEBUG:teuthology.orchestra.run.smithi069:> true 2023-11-06T20:40:02.451 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-11-06T20:40:02.452 INFO:teuthology.run_tasks:Running task clock... 2023-11-06T20:40:02.464 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-06T20:40:02.465 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-06T20:40:02.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T20:40:02.470 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-06T20:40:02.470 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T20:40:02.544 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-06T20:40:02.568 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-06T20:40:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-06T20:40:02.601 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-06T20:40:02.677 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2023-11-06T20:40:02.686 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2023-11-06T20:40:02.706 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2023-11-06T20:40:02.715 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2023-11-06T20:40:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-06T20:40:02.747 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-06T20:40:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-06T20:40:02.780 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-06T20:40:02.865 INFO:teuthology.orchestra.run.smithi069.stderr:bash: ntpq: command not found 2023-11-06T20:40:02.873 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-06T20:40:02.873 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2023-11-06T20:40:02.873 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T20:40:02.873 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T20:40:02.873 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T20:40:02.874 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T20:40:02.874 INFO:teuthology.orchestra.run.smithi033.stderr:bash: ntpq: command not found 2023-11-06T20:40:02.881 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-06T20:40:02.881 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2023-11-06T20:40:02.881 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T20:40:02.881 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T20:40:02.881 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T20:40:02.882 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T20:40:02.883 INFO:teuthology.run_tasks:Running task install... 2023-11-06T20:40:02.891 DEBUG:teuthology.task.install:project ceph 2023-11-06T20:40:02.892 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4e03306bd00b0abb2be37dbc132f82d281f8973d'}} 2023-11-06T20:40:02.892 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '4e03306bd00b0abb2be37dbc132f82d281f8973d'} 2023-11-06T20:40:02.892 INFO:teuthology.task.install:Using flavor: default 2023-11-06T20:40:02.898 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-06T20:40:02.901 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-06T20:40:02.902 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e03306bd00b0abb2be37dbc132f82d281f8973d', 'tag': None, 'wait_for_package': False} 2023-11-06T20:40:02.902 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4e03306bd00b0abb2be37dbc132f82d281f8973d 2023-11-06T20:40:02.904 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e03306bd00b0abb2be37dbc132f82d281f8973d', 'tag': None, 'wait_for_package': False} 2023-11-06T20:40:02.905 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4e03306bd00b0abb2be37dbc132f82d281f8973d 2023-11-06T20:40:03.188 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-11-02-1726-pacific/4e03306bd00b0abb2be37dbc132f82d281f8973d/centos/8/flavors/default/ 2023-11-06T20:40:03.188 INFO:teuthology.task.install.rpm:Package version is 16.2.14-132.g4e03306b 2023-11-06T20:40:03.201 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-11-02-1726-pacific/4e03306bd00b0abb2be37dbc132f82d281f8973d/centos/8/flavors/default/ 2023-11-06T20:40:03.202 INFO:teuthology.task.install.rpm:Package version is 16.2.14-132.g4e03306b 2023-11-06T20:40:03.311 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-11-02-1726-pacific/4e03306bd00b0abb2be37dbc132f82d281f8973d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-11-02-1726-pacific/4e03306bd00b0abb2be37dbc132f82d281f8973d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-11-02-1726-pacific/4e03306bd00b0abb2be37dbc132f82d281f8973d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-06T20:40:03.312 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:40:03.312 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-06T20:40:03.364 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-11-02-1726-pacific/4e03306bd00b0abb2be37dbc132f82d281f8973d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-11-02-1726-pacific/4e03306bd00b0abb2be37dbc132f82d281f8973d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-11-02-1726-pacific/4e03306bd00b0abb2be37dbc132f82d281f8973d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-06T20:40:03.364 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:40:03.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-06T20:40:03.384 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-11-06T20:40:03.384 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4e03306bd00b0abb2be37dbc132f82d281f8973d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-06T20:40:03.438 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-11-06T20:40:03.439 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4e03306bd00b0abb2be37dbc132f82d281f8973d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-06T20:40:03.496 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-06T20:40:03.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-06T20:40:03.593 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-06T20:40:03.650 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-06T20:40:03.661 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2023-11-06T20:40:03.665 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2023-11-06T20:40:03.722 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2023-11-06T20:40:03.725 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2023-11-06T20:40:04.012 INFO:teuthology.orchestra.run.smithi069.stdout:76 files removed 2023-11-06T20:40:04.048 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-11-06T20:40:04.104 INFO:teuthology.orchestra.run.smithi033.stdout:76 files removed 2023-11-06T20:40:04.141 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-11-06T20:40:04.956 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 206 kB/s | 76 kB 00:00 2023-11-06T20:40:05.073 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 213 kB/s | 76 kB 00:00 2023-11-06T20:40:05.265 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 56 kB/s | 15 kB 00:00 2023-11-06T20:40:05.432 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 48 kB/s | 15 kB 00:00 2023-11-06T20:40:05.616 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 4.4 kB/s | 1.4 kB 00:00 2023-11-06T20:40:05.825 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 4.0 kB/s | 1.4 kB 00:00 2023-11-06T20:40:14.157 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 8 - AppStream 4.0 MB/s | 34 MB 00:08 2023-11-06T20:40:16.324 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - AppStream 3.2 MB/s | 34 MB 00:10 2023-11-06T20:40:24.266 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 8 - BaseOS 16 MB/s | 52 MB 00:03 2023-11-06T20:40:32.324 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 8 - Extras 70 kB/s | 18 kB 00:00 2023-11-06T20:40:32.916 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2023-11-06T20:40:33.742 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 8 - PowerTools 9.8 MB/s | 6.6 MB 00:00 2023-11-06T20:40:33.972 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - BaseOS 4.9 MB/s | 52 MB 00:10 2023-11-06T20:40:35.319 INFO:teuthology.orchestra.run.smithi069.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-06T20:40:42.530 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - Extras 48 kB/s | 18 kB 00:00 2023-11-06T20:40:43.139 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2023-11-06T20:40:44.024 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - PowerTools 9.2 MB/s | 6.6 MB 00:00 2023-11-06T20:40:45.600 INFO:teuthology.orchestra.run.smithi033.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-11-06T20:40:46.743 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-11-06T20:40:46.893 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 16 MB 00:11 2023-11-06T20:40:51.460 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 370 kB/s | 24 kB 00:00 2023-11-06T20:40:51.551 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 422 kB/s | 24 kB 00:00 2023-11-06T20:40:55.875 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-06T20:40:55.876 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-06T20:40:55.890 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-11-06T20:40:55.891 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-11-06T20:40:55.891 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-11-06T20:40:55.893 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-11-06T20:40:55.894 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-06T20:40:55.896 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-06T20:40:55.899 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-06T20:40:55.900 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-06T20:40:56.147 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2023-11-06T20:40:56.161 INFO:teuthology.orchestra.run.smithi069.stdout:==================================================================================================== 2023-11-06T20:40:56.161 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2023-11-06T20:40:56.161 INFO:teuthology.orchestra.run.smithi069.stdout:==================================================================================================== 2023-11-06T20:40:56.161 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2023-11-06T20:40:56.162 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-11-06T20:40:56.162 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:16.2.14-132.g4e03306b.el8 ceph 6.4 k 2023-11-06T20:40:56.162 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:16.2.14-132.g4e03306b.el8 ceph 6.1 M 2023-11-06T20:40:56.162 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:16.2.14-132.g4e03306b.el8 ceph 910 k 2023-11-06T20:40:56.163 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-132.g4e03306b.el8 ceph 161 k 2023-11-06T20:40:56.163 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:16.2.14-132.g4e03306b.el8 ceph 1.6 M 2023-11-06T20:40:56.163 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 116 k 2023-11-06T20:40:56.163 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 3.1 M 2023-11-06T20:40:56.163 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 8.6 M 2023-11-06T20:40:56.163 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 27 k 2023-11-06T20:40:56.164 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:16.2.14-132.g4e03306b.el8 ceph 12 M 2023-11-06T20:40:56.164 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:16.2.14-132.g4e03306b.el8 ceph 52 M 2023-11-06T20:40:56.164 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 85 k 2023-11-06T20:40:56.164 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:16.2.14-132.g4e03306b.el8 ceph 257 k 2023-11-06T20:40:56.165 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 19 k 2023-11-06T20:40:56.165 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-11-06T20:40:56.165 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-11-06T20:40:56.165 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-11-06T20:40:56.165 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:16.2.14-132.g4e03306b.el8 ceph 25 k 2023-11-06T20:40:56.166 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:16.2.14-132.g4e03306b.el8 ceph 824 k 2023-11-06T20:40:56.166 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:16.2.14-132.g4e03306b.el8 ceph 131 k 2023-11-06T20:40:56.166 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-11-06T20:40:56.166 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-11-06T20:40:56.166 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-11-06T20:40:56.167 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:16.2.14-132.g4e03306b.el8 ceph 215 k 2023-11-06T20:40:56.167 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:16.2.14-132.g4e03306b.el8 ceph 390 k 2023-11-06T20:40:56.167 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:16.2.14-132.g4e03306b.el8 ceph 369 k 2023-11-06T20:40:56.167 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:16.2.14-132.g4e03306b.el8 ceph 114 k 2023-11-06T20:40:56.167 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:16.2.14-132.g4e03306b.el8 ceph 88 k 2023-11-06T20:40:56.168 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:16.2.14-132.g4e03306b.el8 ceph 3.7 M 2023-11-06T20:40:56.168 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:16.2.14-132.g4e03306b.el8 ceph 173 k 2023-11-06T20:40:56.168 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-11-06T20:40:56.168 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2023-11-06T20:40:56.168 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-11.el8 baseos 144 k 2023-11-06T20:40:56.169 INFO:teuthology.orchestra.run.smithi069.stdout: acl x86_64 2.2.53-3.el8 baseos 83 k 2023-11-06T20:40:56.169 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-11-06T20:40:56.169 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-11-06T20:40:56.169 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-11-06T20:40:56.170 INFO:teuthology.orchestra.run.smithi069.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-11-06T20:40:56.170 INFO:teuthology.orchestra.run.smithi069.stdout: libacl x86_64 2.2.53-3.el8 baseos 35 k 2023-11-06T20:40:56.170 INFO:teuthology.orchestra.run.smithi069.stdout: libacl-devel x86_64 2.2.53-3.el8 baseos 85 k 2023-11-06T20:40:56.170 INFO:teuthology.orchestra.run.smithi069.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-11-06T20:40:56.171 INFO:teuthology.orchestra.run.smithi069.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-11-06T20:40:56.171 INFO:teuthology.orchestra.run.smithi069.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-11-06T20:40:56.171 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:16.2.14-132.g4e03306b.el8 ceph 3.7 M 2023-11-06T20:40:56.171 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:16.2.14-132.g4e03306b.el8 ceph 4.2 M 2023-11-06T20:40:56.171 INFO:teuthology.orchestra.run.smithi069.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-11-06T20:40:56.172 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-11-06T20:40:56.172 INFO:teuthology.orchestra.run.smithi069.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-11-06T20:40:56.172 INFO:teuthology.orchestra.run.smithi069.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-11-06T20:40:56.172 INFO:teuthology.orchestra.run.smithi069.stdout: xfsdump x86_64 3.1.8-7.el8 baseos 334 k 2023-11-06T20:40:56.172 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-11-06T20:40:56.173 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2023-11-06T20:40:56.173 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-11-06T20:40:56.173 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:16.2.14-132.g4e03306b.el8 ceph 24 M 2023-11-06T20:40:56.173 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 24 k 2023-11-06T20:40:56.173 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:16.2.14-132.g4e03306b.el8 ceph 2.4 M 2023-11-06T20:40:56.174 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 245 k 2023-11-06T20:40:56.174 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:16.2.14-132.g4e03306b.el8 ceph 5.0 M 2023-11-06T20:40:56.174 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:16.2.14-132.g4e03306b.el8 ceph 18 M 2023-11-06T20:40:56.174 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 15 k 2023-11-06T20:40:56.174 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:16.2.14-132.g4e03306b.el8 ceph 26 k 2023-11-06T20:40:56.175 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-11-06T20:40:56.175 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-06T20:40:56.175 INFO:teuthology.orchestra.run.smithi069.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-11-06T20:40:56.175 INFO:teuthology.orchestra.run.smithi069.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-11-06T20:40:56.176 INFO:teuthology.orchestra.run.smithi069.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-11-06T20:40:56.176 INFO:teuthology.orchestra.run.smithi069.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-11-06T20:40:56.176 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:16.2.14-132.g4e03306b.el8 ceph 174 k 2023-11-06T20:40:56.176 INFO:teuthology.orchestra.run.smithi069.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-11-06T20:40:56.176 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2023-11-06T20:40:56.177 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-06T20:40:56.177 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2023-11-06T20:40:56.177 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-11-06T20:40:56.177 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:16.2.14-132.g4e03306b.el8 ceph 494 k 2023-11-06T20:40:56.177 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-11-06T20:40:56.178 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:16.2.14-132.g4e03306b.el8 ceph 3.8 M 2023-11-06T20:40:56.178 INFO:teuthology.orchestra.run.smithi069.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-11-06T20:40:56.178 INFO:teuthology.orchestra.run.smithi069.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-11-06T20:40:56.178 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-06T20:40:56.179 INFO:teuthology.orchestra.run.smithi069.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-11-06T20:40:56.179 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-11-06T20:40:56.179 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-11-06T20:40:56.179 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-11-06T20:40:56.179 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-11-06T20:40:56.179 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-11-06T20:40:56.180 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-11-06T20:40:56.180 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-11-06T20:40:56.180 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-11-06T20:40:56.180 INFO:teuthology.orchestra.run.smithi069.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2023-11-06T20:40:56.180 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-11-06T20:40:56.181 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-06T20:40:56.181 INFO:teuthology.orchestra.run.smithi069.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-06T20:40:56.181 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-06T20:40:56.181 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:16.2.14-132.g4e03306b.el8 ceph 45 k 2023-11-06T20:40:56.181 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:16.2.14-132.g4e03306b.el8 ceph 99 k 2023-11-06T20:40:56.181 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-06T20:40:56.182 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-11-06T20:40:56.182 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-06T20:40:56.182 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-06T20:40:56.182 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-11-06T20:40:56.182 INFO:teuthology.orchestra.run.smithi069.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2023-11-06T20:40:56.183 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-06T20:40:56.183 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-06T20:40:56.183 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-06T20:40:56.183 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-11-06T20:40:56.183 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-11-06T20:40:56.183 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-11-06T20:40:56.184 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-11-06T20:40:56.184 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-06T20:40:56.184 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-06T20:40:56.184 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-11-06T20:40:56.184 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-11-06T20:40:56.185 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-06T20:40:56.185 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-11-06T20:40:56.185 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-11-06T20:40:56.185 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-11-06T20:40:56.185 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-06T20:40:56.186 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-06T20:40:56.186 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-11-06T20:40:56.186 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-11-06T20:40:56.186 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-11-06T20:40:56.187 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-11-06T20:40:56.187 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-11-06T20:40:56.187 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-11-06T20:40:56.187 INFO:teuthology.orchestra.run.smithi069.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2023-11-06T20:40:56.187 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-06T20:40:56.188 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-11-06T20:40:56.188 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-06T20:40:56.188 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-06T20:40:56.188 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-11-06T20:40:56.189 INFO:teuthology.orchestra.run.smithi069.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-06T20:40:56.189 INFO:teuthology.orchestra.run.smithi069.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-06T20:40:56.189 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-06T20:40:56.189 INFO:teuthology.orchestra.run.smithi069.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-06T20:40:56.189 INFO:teuthology.orchestra.run.smithi069.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-06T20:40:56.189 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-06T20:40:56.190 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-06T20:40:56.190 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-06T20:40:56.190 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-11-06T20:40:56.190 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-06T20:40:56.191 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-11-06T20:40:56.191 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-11-06T20:40:56.191 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-11-06T20:40:56.191 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:40:56.191 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2023-11-06T20:40:56.192 INFO:teuthology.orchestra.run.smithi069.stdout:==================================================================================================== 2023-11-06T20:40:56.192 INFO:teuthology.orchestra.run.smithi069.stdout:Install 123 Packages 2023-11-06T20:40:56.192 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 19 Packages 2023-11-06T20:40:56.192 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:40:56.193 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 209 M 2023-11-06T20:40:56.193 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2023-11-06T20:40:56.222 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-06T20:40:56.223 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-06T20:40:56.237 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-11-06T20:40:56.238 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-11-06T20:40:56.238 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-11-06T20:40:56.240 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-11-06T20:40:56.241 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-06T20:40:56.243 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-06T20:40:56.246 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-06T20:40:56.247 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-06T20:40:56.509 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2023-11-06T20:40:56.523 INFO:teuthology.orchestra.run.smithi033.stdout:==================================================================================================== 2023-11-06T20:40:56.524 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2023-11-06T20:40:56.524 INFO:teuthology.orchestra.run.smithi033.stdout:==================================================================================================== 2023-11-06T20:40:56.524 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2023-11-06T20:40:56.524 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-11-06T20:40:56.524 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:16.2.14-132.g4e03306b.el8 ceph 6.4 k 2023-11-06T20:40:56.525 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:16.2.14-132.g4e03306b.el8 ceph 6.1 M 2023-11-06T20:40:56.525 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:16.2.14-132.g4e03306b.el8 ceph 910 k 2023-11-06T20:40:56.525 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-132.g4e03306b.el8 ceph 161 k 2023-11-06T20:40:56.525 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:16.2.14-132.g4e03306b.el8 ceph 1.6 M 2023-11-06T20:40:56.526 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 116 k 2023-11-06T20:40:56.526 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 3.1 M 2023-11-06T20:40:56.526 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 8.6 M 2023-11-06T20:40:56.526 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 27 k 2023-11-06T20:40:56.527 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:16.2.14-132.g4e03306b.el8 ceph 12 M 2023-11-06T20:40:56.527 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:16.2.14-132.g4e03306b.el8 ceph 52 M 2023-11-06T20:40:56.527 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 85 k 2023-11-06T20:40:56.527 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:16.2.14-132.g4e03306b.el8 ceph 257 k 2023-11-06T20:40:56.527 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 19 k 2023-11-06T20:40:56.528 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-11-06T20:40:56.528 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-11-06T20:40:56.528 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-11-06T20:40:56.528 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:16.2.14-132.g4e03306b.el8 ceph 25 k 2023-11-06T20:40:56.528 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:16.2.14-132.g4e03306b.el8 ceph 824 k 2023-11-06T20:40:56.529 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:16.2.14-132.g4e03306b.el8 ceph 131 k 2023-11-06T20:40:56.529 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-11-06T20:40:56.529 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-11-06T20:40:56.529 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-11-06T20:40:56.529 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:16.2.14-132.g4e03306b.el8 ceph 215 k 2023-11-06T20:40:56.530 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:16.2.14-132.g4e03306b.el8 ceph 390 k 2023-11-06T20:40:56.530 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:16.2.14-132.g4e03306b.el8 ceph 369 k 2023-11-06T20:40:56.530 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:16.2.14-132.g4e03306b.el8 ceph 114 k 2023-11-06T20:40:56.530 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:16.2.14-132.g4e03306b.el8 ceph 88 k 2023-11-06T20:40:56.531 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:16.2.14-132.g4e03306b.el8 ceph 3.7 M 2023-11-06T20:40:56.531 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:16.2.14-132.g4e03306b.el8 ceph 173 k 2023-11-06T20:40:56.531 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-11-06T20:40:56.531 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2023-11-06T20:40:56.531 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-11.el8 baseos 144 k 2023-11-06T20:40:56.532 INFO:teuthology.orchestra.run.smithi033.stdout: acl x86_64 2.2.53-3.el8 baseos 83 k 2023-11-06T20:40:56.532 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-11-06T20:40:56.532 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-11-06T20:40:56.532 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-11-06T20:40:56.532 INFO:teuthology.orchestra.run.smithi033.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-11-06T20:40:56.533 INFO:teuthology.orchestra.run.smithi033.stdout: libacl x86_64 2.2.53-3.el8 baseos 35 k 2023-11-06T20:40:56.533 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-devel x86_64 2.2.53-3.el8 baseos 85 k 2023-11-06T20:40:56.533 INFO:teuthology.orchestra.run.smithi033.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-11-06T20:40:56.533 INFO:teuthology.orchestra.run.smithi033.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-11-06T20:40:56.533 INFO:teuthology.orchestra.run.smithi033.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-11-06T20:40:56.534 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:16.2.14-132.g4e03306b.el8 ceph 3.7 M 2023-11-06T20:40:56.534 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:16.2.14-132.g4e03306b.el8 ceph 4.2 M 2023-11-06T20:40:56.534 INFO:teuthology.orchestra.run.smithi033.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-11-06T20:40:56.534 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-11-06T20:40:56.534 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-11-06T20:40:56.535 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-11-06T20:40:56.535 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump x86_64 3.1.8-7.el8 baseos 334 k 2023-11-06T20:40:56.535 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-11-06T20:40:56.535 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2023-11-06T20:40:56.535 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-11-06T20:40:56.536 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:16.2.14-132.g4e03306b.el8 ceph 24 M 2023-11-06T20:40:56.536 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 24 k 2023-11-06T20:40:56.536 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:16.2.14-132.g4e03306b.el8 ceph 2.4 M 2023-11-06T20:40:56.536 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 245 k 2023-11-06T20:40:56.537 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:16.2.14-132.g4e03306b.el8 ceph 5.0 M 2023-11-06T20:40:56.537 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:16.2.14-132.g4e03306b.el8 ceph 18 M 2023-11-06T20:40:56.537 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:16.2.14-132.g4e03306b.el8 ceph-noarch 15 k 2023-11-06T20:40:56.537 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:16.2.14-132.g4e03306b.el8 ceph 26 k 2023-11-06T20:40:56.537 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-11-06T20:40:56.538 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-06T20:40:56.538 INFO:teuthology.orchestra.run.smithi033.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-11-06T20:40:56.538 INFO:teuthology.orchestra.run.smithi033.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-11-06T20:40:56.538 INFO:teuthology.orchestra.run.smithi033.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-11-06T20:40:56.538 INFO:teuthology.orchestra.run.smithi033.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-11-06T20:40:56.539 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:16.2.14-132.g4e03306b.el8 ceph 174 k 2023-11-06T20:40:56.539 INFO:teuthology.orchestra.run.smithi033.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-11-06T20:40:56.539 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2023-11-06T20:40:56.539 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-06T20:40:56.539 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2023-11-06T20:40:56.540 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-11-06T20:40:56.540 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:16.2.14-132.g4e03306b.el8 ceph 494 k 2023-11-06T20:40:56.540 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-11-06T20:40:56.540 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:16.2.14-132.g4e03306b.el8 ceph 3.8 M 2023-11-06T20:40:56.540 INFO:teuthology.orchestra.run.smithi033.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-11-06T20:40:56.541 INFO:teuthology.orchestra.run.smithi033.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-11-06T20:40:56.541 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-06T20:40:56.541 INFO:teuthology.orchestra.run.smithi033.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-11-06T20:40:56.541 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-11-06T20:40:56.542 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-11-06T20:40:56.542 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-11-06T20:40:56.542 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-11-06T20:40:56.542 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-11-06T20:40:56.542 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-11-06T20:40:56.543 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-11-06T20:40:56.543 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-11-06T20:40:56.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2023-11-06T20:40:56.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-11-06T20:40:56.543 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-06T20:40:56.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-06T20:40:56.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-06T20:40:56.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:16.2.14-132.g4e03306b.el8 ceph 45 k 2023-11-06T20:40:56.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:16.2.14-132.g4e03306b.el8 ceph 99 k 2023-11-06T20:40:56.544 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-06T20:40:56.545 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-11-06T20:40:56.545 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-06T20:40:56.545 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-06T20:40:56.545 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-11-06T20:40:56.545 INFO:teuthology.orchestra.run.smithi033.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2023-11-06T20:40:56.546 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-06T20:40:56.546 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-06T20:40:56.546 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-06T20:40:56.546 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-11-06T20:40:56.547 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-11-06T20:40:56.547 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-11-06T20:40:56.547 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-11-06T20:40:56.547 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-06T20:40:56.547 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-06T20:40:56.548 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-11-06T20:40:56.548 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-11-06T20:40:56.548 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-06T20:40:56.548 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-11-06T20:40:56.548 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-11-06T20:40:56.549 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-11-06T20:40:56.549 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-06T20:40:56.549 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-06T20:40:56.549 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-11-06T20:40:56.549 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-11-06T20:40:56.550 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-11-06T20:40:56.550 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-11-06T20:40:56.550 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-11-06T20:40:56.550 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-11-06T20:40:56.550 INFO:teuthology.orchestra.run.smithi033.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2023-11-06T20:40:56.551 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-06T20:40:56.551 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-11-06T20:40:56.551 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-06T20:40:56.551 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-06T20:40:56.551 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-11-06T20:40:56.552 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-06T20:40:56.552 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-06T20:40:56.552 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-06T20:40:56.552 INFO:teuthology.orchestra.run.smithi033.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-06T20:40:56.552 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-06T20:40:56.552 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-06T20:40:56.553 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-06T20:40:56.553 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-06T20:40:56.553 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-11-06T20:40:56.553 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-06T20:40:56.553 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-11-06T20:40:56.554 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-11-06T20:40:56.554 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-11-06T20:40:56.554 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:40:56.554 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2023-11-06T20:40:56.554 INFO:teuthology.orchestra.run.smithi033.stdout:==================================================================================================== 2023-11-06T20:40:56.555 INFO:teuthology.orchestra.run.smithi033.stdout:Install 123 Packages 2023-11-06T20:40:56.555 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 19 Packages 2023-11-06T20:40:56.555 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:40:56.556 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 209 M 2023-11-06T20:40:56.556 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2023-11-06T20:40:56.928 INFO:teuthology.orchestra.run.smithi069.stdout:(1/142): ceph-16.2.14-132.g4e03306b.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2023-11-06T20:40:57.144 INFO:teuthology.orchestra.run.smithi033.stdout:(1/142): ceph-16.2.14-132.g4e03306b.el8.x86_64. 36 kB/s | 6.4 kB 00:00 2023-11-06T20:40:57.637 INFO:teuthology.orchestra.run.smithi069.stdout:(2/142): ceph-fuse-16.2.14-132.g4e03306b.el8.x8 1.3 MB/s | 910 kB 00:00 2023-11-06T20:40:57.845 INFO:teuthology.orchestra.run.smithi069.stdout:(3/142): ceph-immutable-object-cache-16.2.14-13 777 kB/s | 161 kB 00:00 2023-11-06T20:40:57.985 INFO:teuthology.orchestra.run.smithi033.stdout:(2/142): ceph-fuse-16.2.14-132.g4e03306b.el8.x8 1.1 MB/s | 910 kB 00:00 2023-11-06T20:40:58.052 INFO:teuthology.orchestra.run.smithi033.stdout:(3/142): ceph-immutable-object-cache-16.2.14-13 2.4 MB/s | 161 kB 00:00 2023-11-06T20:40:58.170 INFO:teuthology.orchestra.run.smithi069.stdout:(4/142): ceph-common-16.2.14-132.g4e03306b.el8. 17 MB/s | 24 MB 00:01 2023-11-06T20:40:58.295 INFO:teuthology.orchestra.run.smithi069.stdout:(5/142): ceph-mgr-16.2.14-132.g4e03306b.el8.x86 13 MB/s | 1.6 MB 00:00 2023-11-06T20:40:58.629 INFO:teuthology.orchestra.run.smithi069.stdout:(6/142): ceph-mon-16.2.14-132.g4e03306b.el8.x86 15 MB/s | 5.0 MB 00:00 2023-11-06T20:40:58.661 INFO:teuthology.orchestra.run.smithi033.stdout:(4/142): ceph-mds-16.2.14-132.g4e03306b.el8.x86 4.0 MB/s | 2.4 MB 00:00 2023-11-06T20:40:59.094 INFO:teuthology.orchestra.run.smithi033.stdout:(5/142): ceph-mgr-16.2.14-132.g4e03306b.el8.x86 3.7 MB/s | 1.6 MB 00:00 2023-11-06T20:40:59.279 INFO:teuthology.orchestra.run.smithi069.stdout:(7/142): ceph-mds-16.2.14-132.g4e03306b.el8.x86 1.7 MB/s | 2.4 MB 00:01 2023-11-06T20:40:59.398 INFO:teuthology.orchestra.run.smithi069.stdout:(8/142): ceph-base-16.2.14-132.g4e03306b.el8.x8 2.3 MB/s | 6.1 MB 00:02 2023-11-06T20:40:59.513 INFO:teuthology.orchestra.run.smithi069.stdout:(9/142): ceph-selinux-16.2.14-132.g4e03306b.el8 225 kB/s | 26 kB 00:00 2023-11-06T20:40:59.861 INFO:teuthology.orchestra.run.smithi033.stdout:(6/142): ceph-base-16.2.14-132.g4e03306b.el8.x8 2.1 MB/s | 6.1 MB 00:02 2023-11-06T20:41:00.154 INFO:teuthology.orchestra.run.smithi033.stdout:(7/142): ceph-mon-16.2.14-132.g4e03306b.el8.x86 4.7 MB/s | 5.0 MB 00:01 2023-11-06T20:41:00.280 INFO:teuthology.orchestra.run.smithi069.stdout:(10/142): ceph-osd-16.2.14-132.g4e03306b.el8.x8 11 MB/s | 18 MB 00:01 2023-11-06T20:41:00.372 INFO:teuthology.orchestra.run.smithi069.stdout:(11/142): cephfs-mirror-16.2.14-132.g4e03306b.e 2.8 MB/s | 257 kB 00:00 2023-11-06T20:41:00.447 INFO:teuthology.orchestra.run.smithi069.stdout:(12/142): libcephfs-devel-16.2.14-132.g4e03306b 341 kB/s | 25 kB 00:00 2023-11-06T20:41:00.615 INFO:teuthology.orchestra.run.smithi069.stdout:(13/142): libcephfs2-16.2.14-132.g4e03306b.el8. 4.8 MB/s | 824 kB 00:00 2023-11-06T20:41:00.689 INFO:teuthology.orchestra.run.smithi069.stdout:(14/142): libcephsqlite-16.2.14-132.g4e03306b.e 2.3 MB/s | 174 kB 00:00 2023-11-06T20:41:00.756 INFO:teuthology.orchestra.run.smithi069.stdout:(15/142): librados-devel-16.2.14-132.g4e03306b. 1.9 MB/s | 131 kB 00:00 2023-11-06T20:41:00.848 INFO:teuthology.orchestra.run.smithi069.stdout:(16/142): libradosstriper1-16.2.14-132.g4e03306 5.3 MB/s | 494 kB 00:00 2023-11-06T20:41:01.332 INFO:teuthology.orchestra.run.smithi069.stdout:(17/142): librgw2-16.2.14-132.g4e03306b.el8.x86 7.8 MB/s | 3.8 MB 00:00 2023-11-06T20:41:01.424 INFO:teuthology.orchestra.run.smithi069.stdout:(18/142): python3-ceph-argparse-16.2.14-132.g4e 492 kB/s | 45 kB 00:00 2023-11-06T20:41:01.516 INFO:teuthology.orchestra.run.smithi069.stdout:(19/142): python3-ceph-common-16.2.14-132.g4e03 1.1 MB/s | 99 kB 00:00 2023-11-06T20:41:01.591 INFO:teuthology.orchestra.run.smithi069.stdout:(20/142): python3-cephfs-16.2.14-132.g4e03306b. 2.8 MB/s | 215 kB 00:00 2023-11-06T20:41:01.683 INFO:teuthology.orchestra.run.smithi069.stdout:(21/142): python3-rados-16.2.14-132.g4e03306b.e 4.2 MB/s | 390 kB 00:00 2023-11-06T20:41:01.767 INFO:teuthology.orchestra.run.smithi069.stdout:(22/142): python3-rbd-16.2.14-132.g4e03306b.el8 4.3 MB/s | 369 kB 00:00 2023-11-06T20:41:01.842 INFO:teuthology.orchestra.run.smithi069.stdout:(23/142): python3-rgw-16.2.14-132.g4e03306b.el8 1.5 MB/s | 114 kB 00:00 2023-11-06T20:41:01.918 INFO:teuthology.orchestra.run.smithi069.stdout:(24/142): rbd-fuse-16.2.14-132.g4e03306b.el8.x8 1.1 MB/s | 88 kB 00:00 2023-11-06T20:41:02.443 INFO:teuthology.orchestra.run.smithi069.stdout:(25/142): rbd-mirror-16.2.14-132.g4e03306b.el8. 7.0 MB/s | 3.7 MB 00:00 2023-11-06T20:41:02.540 INFO:teuthology.orchestra.run.smithi069.stdout:(26/142): rbd-nbd-16.2.14-132.g4e03306b.el8.x86 1.8 MB/s | 173 kB 00:00 2023-11-06T20:41:02.594 INFO:teuthology.orchestra.run.smithi069.stdout:(27/142): ceph-grafana-dashboards-16.2.14-132.g 455 kB/s | 24 kB 00:00 2023-11-06T20:41:02.612 INFO:teuthology.orchestra.run.smithi033.stdout:(8/142): ceph-radosgw-16.2.14-132.g4e03306b.el8 4.7 MB/s | 12 MB 00:02 2023-11-06T20:41:02.644 INFO:teuthology.orchestra.run.smithi069.stdout:(28/142): ceph-mgr-cephadm-16.2.14-132.g4e03306 2.3 MB/s | 116 kB 00:00 2023-11-06T20:41:02.678 INFO:teuthology.orchestra.run.smithi033.stdout:(9/142): ceph-selinux-16.2.14-132.g4e03306b.el8 386 kB/s | 26 kB 00:00 2023-11-06T20:41:02.938 INFO:teuthology.orchestra.run.smithi033.stdout:(10/142): ceph-osd-16.2.14-132.g4e03306b.el8.x8 6.0 MB/s | 18 MB 00:03 2023-11-06T20:41:02.994 INFO:teuthology.orchestra.run.smithi069.stdout:(29/142): ceph-mgr-dashboard-16.2.14-132.g4e033 8.9 MB/s | 3.1 MB 00:00 2023-11-06T20:41:03.038 INFO:teuthology.orchestra.run.smithi033.stdout:(11/142): cephfs-mirror-16.2.14-132.g4e03306b.e 2.5 MB/s | 257 kB 00:00 2023-11-06T20:41:03.138 INFO:teuthology.orchestra.run.smithi033.stdout:(12/142): libcephfs-devel-16.2.14-132.g4e03306b 256 kB/s | 25 kB 00:00 2023-11-06T20:41:03.338 INFO:teuthology.orchestra.run.smithi033.stdout:(13/142): libcephfs2-16.2.14-132.g4e03306b.el8. 4.0 MB/s | 824 kB 00:00 2023-11-06T20:41:03.439 INFO:teuthology.orchestra.run.smithi033.stdout:(14/142): libcephsqlite-16.2.14-132.g4e03306b.e 1.7 MB/s | 174 kB 00:00 2023-11-06T20:41:03.539 INFO:teuthology.orchestra.run.smithi033.stdout:(15/142): librados-devel-16.2.14-132.g4e03306b. 1.3 MB/s | 131 kB 00:00 2023-11-06T20:41:03.656 INFO:teuthology.orchestra.run.smithi033.stdout:(16/142): libradosstriper1-16.2.14-132.g4e03306 4.1 MB/s | 494 kB 00:00 2023-11-06T20:41:04.103 INFO:teuthology.orchestra.run.smithi069.stdout:(30/142): ceph-mgr-diskprediction-local-16.2.14 7.8 MB/s | 8.6 MB 00:01 2023-11-06T20:41:04.169 INFO:teuthology.orchestra.run.smithi069.stdout:(31/142): ceph-mgr-modules-core-16.2.14-132.g4e 3.6 MB/s | 245 kB 00:00 2023-11-06T20:41:04.237 INFO:teuthology.orchestra.run.smithi069.stdout:(32/142): ceph-mgr-rook-16.2.14-132.g4e03306b.e 409 kB/s | 27 kB 00:00 2023-11-06T20:41:04.278 INFO:teuthology.orchestra.run.smithi069.stdout:(33/142): ceph-prometheus-alerts-16.2.14-132.g4 360 kB/s | 15 kB 00:00 2023-11-06T20:41:04.337 INFO:teuthology.orchestra.run.smithi069.stdout:(34/142): cephadm-16.2.14-132.g4e03306b.el8.noa 1.4 MB/s | 85 kB 00:00 2023-11-06T20:41:04.395 INFO:teuthology.orchestra.run.smithi069.stdout:(35/142): cephfs-top-16.2.14-132.g4e03306b.el8. 320 kB/s | 19 kB 00:00 2023-11-06T20:41:04.481 INFO:teuthology.orchestra.run.smithi033.stdout:(17/142): ceph-common-16.2.14-132.g4e03306b.el8 3.2 MB/s | 24 MB 00:07 2023-11-06T20:41:04.548 INFO:teuthology.orchestra.run.smithi033.stdout:(18/142): librgw2-16.2.14-132.g4e03306b.el8.x86 4.3 MB/s | 3.8 MB 00:00 2023-11-06T20:41:04.573 INFO:teuthology.orchestra.run.smithi033.stdout:(19/142): python3-ceph-argparse-16.2.14-132.g4e 492 kB/s | 45 kB 00:00 2023-11-06T20:41:04.607 INFO:teuthology.orchestra.run.smithi033.stdout:(20/142): python3-ceph-common-16.2.14-132.g4e03 1.7 MB/s | 99 kB 00:00 2023-11-06T20:41:04.691 INFO:teuthology.orchestra.run.smithi033.stdout:(21/142): python3-cephfs-16.2.14-132.g4e03306b. 1.8 MB/s | 215 kB 00:00 2023-11-06T20:41:04.724 INFO:teuthology.orchestra.run.smithi033.stdout:(22/142): python3-rados-16.2.14-132.g4e03306b.e 3.3 MB/s | 390 kB 00:00 2023-11-06T20:41:04.791 INFO:teuthology.orchestra.run.smithi033.stdout:(23/142): python3-rgw-16.2.14-132.g4e03306b.el8 1.7 MB/s | 114 kB 00:00 2023-11-06T20:41:04.816 INFO:teuthology.orchestra.run.smithi033.stdout:(24/142): python3-rbd-16.2.14-132.g4e03306b.el8 2.9 MB/s | 369 kB 00:00 2023-11-06T20:41:04.858 INFO:teuthology.orchestra.run.smithi033.stdout:(25/142): rbd-fuse-16.2.14-132.g4e03306b.el8.x8 1.3 MB/s | 88 kB 00:00 2023-11-06T20:41:04.888 INFO:teuthology.orchestra.run.smithi069.stdout:(36/142): ceph-radosgw-16.2.14-132.g4e03306b.el 2.1 MB/s | 12 MB 00:05 2023-11-06T20:41:04.933 INFO:teuthology.orchestra.run.smithi033.stdout:(26/142): rbd-nbd-16.2.14-132.g4e03306b.el8.x86 2.3 MB/s | 173 kB 00:00 2023-11-06T20:41:05.000 INFO:teuthology.orchestra.run.smithi033.stdout:(27/142): ceph-grafana-dashboards-16.2.14-132.g 366 kB/s | 24 kB 00:00 2023-11-06T20:41:05.076 INFO:teuthology.orchestra.run.smithi033.stdout:(28/142): ceph-mgr-cephadm-16.2.14-132.g4e03306 1.5 MB/s | 116 kB 00:00 2023-11-06T20:41:05.205 INFO:teuthology.orchestra.run.smithi069.stdout:(37/142): automake-1.16.1-8.el8.noarch.rpm 915 kB/s | 740 kB 00:00 2023-11-06T20:41:05.313 INFO:teuthology.orchestra.run.smithi069.stdout:(38/142): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2023-11-06T20:41:05.430 INFO:teuthology.orchestra.run.smithi069.stdout:(39/142): jq-1.6-7.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2023-11-06T20:41:05.518 INFO:teuthology.orchestra.run.smithi033.stdout:(29/142): rbd-mirror-16.2.14-132.g4e03306b.el8. 5.2 MB/s | 3.7 MB 00:00 2023-11-06T20:41:05.539 INFO:teuthology.orchestra.run.smithi069.stdout:(40/142): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-11-06T20:41:05.585 INFO:teuthology.orchestra.run.smithi033.stdout:(30/142): ceph-mgr-dashboard-16.2.14-132.g4e033 6.2 MB/s | 3.1 MB 00:00 2023-11-06T20:41:05.597 INFO:teuthology.orchestra.run.smithi069.stdout:(41/142): bison-3.0.4-10.el8.x86_64.rpm 971 kB/s | 688 kB 00:00 2023-11-06T20:41:05.668 INFO:teuthology.orchestra.run.smithi033.stdout:(31/142): ceph-mgr-modules-core-16.2.14-132.g4e 2.9 MB/s | 245 kB 00:00 2023-11-06T20:41:05.689 INFO:teuthology.orchestra.run.smithi069.stdout:(42/142): libtool-2.4.6-25.el8.x86_64.rpm 4.6 MB/s | 709 kB 00:00 2023-11-06T20:41:05.723 INFO:teuthology.orchestra.run.smithi069.stdout:(43/142): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2023-11-06T20:41:05.730 INFO:teuthology.orchestra.run.smithi033.stdout:(32/142): ceph-mgr-rook-16.2.14-132.g4e03306b.e 469 kB/s | 27 kB 00:00 2023-11-06T20:41:05.794 INFO:teuthology.orchestra.run.smithi033.stdout:(33/142): ceph-prometheus-alerts-16.2.14-132.g4 223 kB/s | 15 kB 00:00 2023-11-06T20:41:05.807 INFO:teuthology.orchestra.run.smithi069.stdout:(44/142): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2023-11-06T20:41:05.853 INFO:teuthology.orchestra.run.smithi033.stdout:(34/142): cephadm-16.2.14-132.g4e03306b.el8.noa 1.4 MB/s | 85 kB 00:00 2023-11-06T20:41:05.953 INFO:teuthology.orchestra.run.smithi033.stdout:(35/142): cephfs-top-16.2.14-132.g4e03306b.el8. 186 kB/s | 19 kB 00:00 2023-11-06T20:41:06.382 INFO:teuthology.orchestra.run.smithi069.stdout:(45/142): openblas-0.3.15-4.el8.x86_64.rpm 7.2 MB/s | 4.7 MB 00:00 2023-11-06T20:41:06.770 INFO:teuthology.orchestra.run.smithi033.stdout:(36/142): automake-1.16.1-8.el8.noarch.rpm 906 kB/s | 740 kB 00:00 2023-11-06T20:41:06.949 INFO:teuthology.orchestra.run.smithi069.stdout:(46/142): ceph-test-16.2.14-132.g4e03306b.el8.x 7.0 MB/s | 52 MB 00:07 2023-11-06T20:41:07.021 INFO:teuthology.orchestra.run.smithi033.stdout:(37/142): ceph-mgr-diskprediction-local-16.2.14 5.7 MB/s | 8.6 MB 00:01 2023-11-06T20:41:07.054 INFO:teuthology.orchestra.run.smithi033.stdout:(38/142): bison-3.0.4-10.el8.x86_64.rpm 2.4 MB/s | 688 kB 00:00 2023-11-06T20:41:07.188 INFO:teuthology.orchestra.run.smithi033.stdout:(39/142): jq-1.6-7.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2023-11-06T20:41:07.305 INFO:teuthology.orchestra.run.smithi033.stdout:(40/142): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-11-06T20:41:07.421 INFO:teuthology.orchestra.run.smithi033.stdout:(41/142): libtool-2.4.6-25.el8.x86_64.rpm 6.0 MB/s | 709 kB 00:00 2023-11-06T20:41:07.530 INFO:teuthology.orchestra.run.smithi033.stdout:(42/142): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.3 MB/s | 259 kB 00:00 2023-11-06T20:41:07.566 INFO:teuthology.orchestra.run.smithi069.stdout:(47/142): python3-jinja2-2.10.1-3.el8.noarch.rp 873 kB/s | 538 kB 00:00 2023-11-06T20:41:07.630 INFO:teuthology.orchestra.run.smithi033.stdout:(43/142): oniguruma-6.8.2-2.el8.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2023-11-06T20:41:07.674 INFO:teuthology.orchestra.run.smithi069.stdout:(48/142): python3-jsonpatch-1.21-2.el8.noarch.r 246 kB/s | 27 kB 00:00 2023-11-06T20:41:07.698 INFO:teuthology.orchestra.run.smithi033.stdout:(44/142): flex-2.6.1-9.el8.x86_64.rpm 473 kB/s | 320 kB 00:00 2023-11-06T20:41:07.783 INFO:teuthology.orchestra.run.smithi069.stdout:(49/142): python3-jsonpointer-1.10-11.el8.noarc 184 kB/s | 20 kB 00:00 2023-11-06T20:41:07.933 INFO:teuthology.orchestra.run.smithi069.stdout:(50/142): openblas-threads-0.3.15-4.el8.x86_64. 2.3 MB/s | 4.9 MB 00:02 2023-11-06T20:41:07.992 INFO:teuthology.orchestra.run.smithi069.stdout:(51/142): python3-mako-1.0.6-14.el8.noarch.rpm 753 kB/s | 157 kB 00:00 2023-11-06T20:41:08.042 INFO:teuthology.orchestra.run.smithi069.stdout:(52/142): python3-markupsafe-0.23-19.el8.x86_64 357 kB/s | 39 kB 00:00 2023-11-06T20:41:08.301 INFO:teuthology.orchestra.run.smithi069.stdout:(53/142): python3-numpy-f2py-1.14.3-10.el8.x86_ 872 kB/s | 225 kB 00:00 2023-11-06T20:41:08.410 INFO:teuthology.orchestra.run.smithi069.stdout:(54/142): python3-prettytable-0.7.2-14.el8.noar 405 kB/s | 44 kB 00:00 2023-11-06T20:41:08.535 INFO:teuthology.orchestra.run.smithi069.stdout:(55/142): python3-pyOpenSSL-19.0.0-1.el8.noarch 829 kB/s | 103 kB 00:00 2023-11-06T20:41:08.598 INFO:teuthology.orchestra.run.smithi033.stdout:(45/142): ceph-test-16.2.14-132.g4e03306b.el8.x 8.7 MB/s | 52 MB 00:05 2023-11-06T20:41:08.727 INFO:teuthology.orchestra.run.smithi069.stdout:(56/142): python3-pyasn1-0.3.7-6.el8.noarch.rpm 660 kB/s | 126 kB 00:00 2023-11-06T20:41:08.827 INFO:teuthology.orchestra.run.smithi069.stdout:(57/142): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-11-06T20:41:08.886 INFO:teuthology.orchestra.run.smithi069.stdout:(58/142): python3-numpy-1.14.3-10.el8.x86_64.rp 4.2 MB/s | 3.7 MB 00:00 2023-11-06T20:41:08.936 INFO:teuthology.orchestra.run.smithi069.stdout:(59/142): python3-pytz-2017.2-11.el8.noarch.rpm 526 kB/s | 57 kB 00:00 2023-11-06T20:41:09.528 INFO:teuthology.orchestra.run.smithi069.stdout:(60/142): python3-werkzeug-0.12.2-4.el8.noarch. 774 kB/s | 457 kB 00:00 2023-11-06T20:41:09.748 INFO:teuthology.orchestra.run.smithi033.stdout:(46/142): python3-babel-2.5.1-7.el8.noarch.rpm 4.2 MB/s | 4.8 MB 00:01 2023-11-06T20:41:09.815 INFO:teuthology.orchestra.run.smithi033.stdout:(47/142): openblas-threads-0.3.15-4.el8.x86_64. 2.3 MB/s | 4.9 MB 00:02 2023-11-06T20:41:09.932 INFO:teuthology.orchestra.run.smithi033.stdout:(48/142): python3-jsonpatch-1.21-2.el8.noarch.r 227 kB/s | 27 kB 00:00 2023-11-06T20:41:09.953 INFO:teuthology.orchestra.run.smithi069.stdout:(61/142): socat-1.7.4.1-1.el8.x86_64.rpm 760 kB/s | 323 kB 00:00 2023-11-06T20:41:10.032 INFO:teuthology.orchestra.run.smithi033.stdout:(49/142): python3-jsonpointer-1.10-11.el8.noarc 200 kB/s | 20 kB 00:00 2023-11-06T20:41:10.099 INFO:teuthology.orchestra.run.smithi033.stdout:(50/142): openblas-0.3.15-4.el8.x86_64.rpm 1.9 MB/s | 4.7 MB 00:02 2023-11-06T20:41:10.112 INFO:teuthology.orchestra.run.smithi069.stdout:(62/142): python3-babel-2.5.1-7.el8.noarch.rpm 1.3 MB/s | 4.8 MB 00:03 2023-11-06T20:41:10.137 INFO:teuthology.orchestra.run.smithi069.stdout:(63/142): xmlstarlet-1.6.1-20.el8.x86_64.rpm 385 kB/s | 71 kB 00:00 2023-11-06T20:41:10.208 INFO:teuthology.orchestra.run.smithi033.stdout:(51/142): python3-markupsafe-0.23-19.el8.x86_64 359 kB/s | 39 kB 00:00 2023-11-06T20:41:10.233 INFO:teuthology.orchestra.run.smithi033.stdout:(52/142): python3-mako-1.0.6-14.el8.noarch.rpm 783 kB/s | 157 kB 00:00 2023-11-06T20:41:10.366 INFO:teuthology.orchestra.run.smithi033.stdout:(53/142): python3-jinja2-2.10.1-3.el8.noarch.rp 870 kB/s | 538 kB 00:00 2023-11-06T20:41:10.455 INFO:teuthology.orchestra.run.smithi069.stdout:(64/142): elfutils-libelf-devel-0.189-3.el8.x86 180 kB/s | 62 kB 00:00 2023-11-06T20:41:10.475 INFO:teuthology.orchestra.run.smithi033.stdout:(54/142): python3-prettytable-0.7.2-14.el8.noar 405 kB/s | 44 kB 00:00 2023-11-06T20:41:10.500 INFO:teuthology.orchestra.run.smithi033.stdout:(55/142): python3-numpy-f2py-1.14.3-10.el8.x86_ 843 kB/s | 225 kB 00:00 2023-11-06T20:41:10.546 INFO:teuthology.orchestra.run.smithi069.stdout:(65/142): fuse-2.9.7-17.el8.x86_64.rpm 203 kB/s | 83 kB 00:00 2023-11-06T20:41:10.571 INFO:teuthology.orchestra.run.smithi069.stdout:(66/142): keyutils-libs-devel-1.5.10-9.el8.x86_ 412 kB/s | 48 kB 00:00 2023-11-06T20:41:10.584 INFO:teuthology.orchestra.run.smithi033.stdout:(56/142): python3-pyOpenSSL-19.0.0-1.el8.noarch 955 kB/s | 103 kB 00:00 2023-11-06T20:41:10.663 INFO:teuthology.orchestra.run.smithi069.stdout:(67/142): libaio-devel-0.3.112-1.el8.x86_64.rpm 204 kB/s | 19 kB 00:00 2023-11-06T20:41:10.684 INFO:teuthology.orchestra.run.smithi033.stdout:(57/142): python3-pyasn1-0.3.7-6.el8.noarch.rpm 688 kB/s | 126 kB 00:00 2023-11-06T20:41:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:(58/142): python3-pyasn1-modules-0.3.7-6.el8.no 879 kB/s | 110 kB 00:00 2023-11-06T20:41:10.764 INFO:teuthology.orchestra.run.smithi069.stdout:(68/142): libcom_err-devel-1.45.6-5.el8.x86_64. 387 kB/s | 39 kB 00:00 2023-11-06T20:41:10.827 INFO:teuthology.orchestra.run.smithi033.stdout:(59/142): python3-pytz-2017.2-11.el8.noarch.rpm 402 kB/s | 57 kB 00:00 2023-11-06T20:41:10.881 INFO:teuthology.orchestra.run.smithi069.stdout:(69/142): krb5-devel-1.18.2-22.el8.x86_64.rpm 1.6 MB/s | 561 kB 00:00 2023-11-06T20:41:10.981 INFO:teuthology.orchestra.run.smithi069.stdout:(70/142): libquadmath-8.5.0-21.el8.x86_64.rpm 1.7 MB/s | 172 kB 00:00 2023-11-06T20:41:11.046 INFO:teuthology.orchestra.run.smithi069.stdout:(71/142): libgfortran-8.5.0-21.el8.x86_64.rpm 2.2 MB/s | 645 kB 00:00 2023-11-06T20:41:11.082 INFO:teuthology.orchestra.run.smithi069.stdout:(72/142): librabbitmq-0.9.0-4.el8.x86_64.rpm 467 kB/s | 47 kB 00:00 2023-11-06T20:41:11.149 INFO:teuthology.orchestra.run.smithi069.stdout:(73/142): libselinux-devel-2.9-8.el8.x86_64.rpm 1.9 MB/s | 200 kB 00:00 2023-11-06T20:41:11.182 INFO:teuthology.orchestra.run.smithi069.stdout:(74/142): libsepol-devel-2.9-3.el8.x86_64.rpm 869 kB/s | 87 kB 00:00 2023-11-06T20:41:11.249 INFO:teuthology.orchestra.run.smithi069.stdout:(75/142): libuuid-devel-2.32.1-43.el8.x86_64.rp 998 kB/s | 100 kB 00:00 2023-11-06T20:41:11.283 INFO:teuthology.orchestra.run.smithi069.stdout:(76/142): libverto-devel-0.3.2-2.el8.x86_64.rpm 180 kB/s | 18 kB 00:00 2023-11-06T20:41:11.350 INFO:teuthology.orchestra.run.smithi069.stdout:(77/142): libzstd-devel-1.4.4-1.el8.x86_64.rpm 438 kB/s | 44 kB 00:00 2023-11-06T20:41:11.410 INFO:teuthology.orchestra.run.smithi033.stdout:(60/142): python3-werkzeug-0.12.2-4.el8.noarch. 785 kB/s | 457 kB 00:00 2023-11-06T20:41:11.467 INFO:teuthology.orchestra.run.smithi069.stdout:(78/142): pcre2-devel-10.32-3.el8.x86_64.rpm 5.1 MB/s | 605 kB 00:00 2023-11-06T20:41:11.567 INFO:teuthology.orchestra.run.smithi069.stdout:(79/142): pcre2-utf16-10.32-3.el8.x86_64.rpm 2.2 MB/s | 229 kB 00:00 2023-11-06T20:41:11.684 INFO:teuthology.orchestra.run.smithi069.stdout:(80/142): openssl-devel-1.1.1k-9.el8.x86_64.rpm 7.9 MB/s | 3.2 MB 00:00 2023-11-06T20:41:11.709 INFO:teuthology.orchestra.run.smithi069.stdout:(81/142): pcre2-utf32-10.32-3.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2023-11-06T20:41:11.784 INFO:teuthology.orchestra.run.smithi069.stdout:(82/142): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.4 MB/s | 249 kB 00:00 2023-11-06T20:41:11.818 INFO:teuthology.orchestra.run.smithi069.stdout:(83/142): python3-cryptography-3.2.1-6.el8.x86_ 5.8 MB/s | 641 kB 00:00 2023-11-06T20:41:11.835 INFO:teuthology.orchestra.run.smithi033.stdout:(61/142): socat-1.7.4.1-1.el8.x86_64.rpm 760 kB/s | 323 kB 00:00 2023-11-06T20:41:11.885 INFO:teuthology.orchestra.run.smithi069.stdout:(84/142): python3-jwt-1.6.1-2.el8.noarch.rpm 426 kB/s | 43 kB 00:00 2023-11-06T20:41:11.918 INFO:teuthology.orchestra.run.smithi069.stdout:(85/142): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2023-11-06T20:41:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:(62/142): xmlstarlet-1.6.1-20.el8.x86_64.rpm 706 kB/s | 71 kB 00:00 2023-11-06T20:41:11.977 INFO:teuthology.orchestra.run.smithi069.stdout:(86/142): python3-pycparser-2.14-14.el8.noarch. 1.2 MB/s | 109 kB 00:00 2023-11-06T20:41:12.019 INFO:teuthology.orchestra.run.smithi069.stdout:(87/142): python3-requests-oauthlib-1.0.0-1.el8 425 kB/s | 43 kB 00:00 2023-11-06T20:41:12.078 INFO:teuthology.orchestra.run.smithi069.stdout:(88/142): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 745 kB/s | 74 kB 00:00 2023-11-06T20:41:12.136 INFO:teuthology.orchestra.run.smithi069.stdout:(89/142): zlib-devel-1.2.11-21.el8.x86_64.rpm 497 kB/s | 58 kB 00:00 2023-11-06T20:41:12.420 INFO:teuthology.orchestra.run.smithi033.stdout:(63/142): elfutils-libelf-devel-0.189-3.el8.x86 128 kB/s | 62 kB 00:00 2023-11-06T20:41:12.595 INFO:teuthology.orchestra.run.smithi033.stdout:(64/142): fuse-2.9.7-17.el8.x86_64.rpm 475 kB/s | 83 kB 00:00 2023-11-06T20:41:12.687 INFO:teuthology.orchestra.run.smithi033.stdout:(65/142): keyutils-libs-devel-1.5.10-9.el8.x86_ 526 kB/s | 48 kB 00:00 2023-11-06T20:41:12.753 INFO:teuthology.orchestra.run.smithi069.stdout:(90/142): leveldb-1.22-1.el8.x86_64.rpm 293 kB/s | 181 kB 00:00 2023-11-06T20:41:12.879 INFO:teuthology.orchestra.run.smithi069.stdout:(91/142): liboath-2.6.2-3.el8.x86_64.rpm 471 kB/s | 59 kB 00:00 2023-11-06T20:41:12.945 INFO:teuthology.orchestra.run.smithi069.stdout:(92/142): gperftools-libs-2.7-9.el8.x86_64.rpm 353 kB/s | 306 kB 00:00 2023-11-06T20:41:12.946 INFO:teuthology.orchestra.run.smithi033.stdout:(66/142): krb5-devel-1.18.2-22.el8.x86_64.rpm 2.1 MB/s | 561 kB 00:00 2023-11-06T20:41:12.988 INFO:teuthology.orchestra.run.smithi069.stdout:(93/142): libunwind-1.3.1-3.el8.x86_64.rpm 691 kB/s | 75 kB 00:00 2023-11-06T20:41:13.037 INFO:teuthology.orchestra.run.smithi033.stdout:(67/142): libaio-devel-0.3.112-1.el8.x86_64.rpm 204 kB/s | 19 kB 00:00 2023-11-06T20:41:13.054 INFO:teuthology.orchestra.run.smithi069.stdout:(94/142): python3-apipkg-1.5-6.el8.noarch.rpm 195 kB/s | 21 kB 00:00 2023-11-06T20:41:13.096 INFO:teuthology.orchestra.run.smithi069.stdout:(95/142): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 403 kB/s | 44 kB 00:00 2023-11-06T20:41:13.138 INFO:teuthology.orchestra.run.smithi033.stdout:(68/142): libcom_err-devel-1.45.6-5.el8.x86_64. 388 kB/s | 39 kB 00:00 2023-11-06T20:41:13.205 INFO:teuthology.orchestra.run.smithi069.stdout:(96/142): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2023-11-06T20:41:13.257 INFO:teuthology.orchestra.run.smithi033.stdout:(69/142): libgfortran-8.5.0-21.el8.x86_64.rpm 5.4 MB/s | 645 kB 00:00 2023-11-06T20:41:13.314 INFO:teuthology.orchestra.run.smithi069.stdout:(97/142): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-11-06T20:41:13.347 INFO:teuthology.orchestra.run.smithi033.stdout:(70/142): libquadmath-8.5.0-21.el8.x86_64.rpm 1.8 MB/s | 172 kB 00:00 2023-11-06T20:41:13.348 INFO:teuthology.orchestra.run.smithi069.stdout:(98/142): python3-beautifulsoup4-4.6.3-2.el8.1. 632 kB/s | 185 kB 00:00 2023-11-06T20:41:13.447 INFO:teuthology.orchestra.run.smithi033.stdout:(71/142): librabbitmq-0.9.0-4.el8.x86_64.rpm 469 kB/s | 47 kB 00:00 2023-11-06T20:41:13.548 INFO:teuthology.orchestra.run.smithi033.stdout:(72/142): libselinux-devel-2.9-8.el8.x86_64.rpm 2.0 MB/s | 200 kB 00:00 2023-11-06T20:41:13.598 INFO:teuthology.orchestra.run.smithi069.stdout:(99/142): python3-cheroot-8.5.2-1.el8.noarch.rp 609 kB/s | 173 kB 00:00 2023-11-06T20:41:13.648 INFO:teuthology.orchestra.run.smithi033.stdout:(73/142): libsepol-devel-2.9-3.el8.x86_64.rpm 873 kB/s | 87 kB 00:00 2023-11-06T20:41:13.765 INFO:teuthology.orchestra.run.smithi033.stdout:(74/142): libuuid-devel-2.32.1-43.el8.x86_64.rp 855 kB/s | 100 kB 00:00 2023-11-06T20:41:13.857 INFO:teuthology.orchestra.run.smithi033.stdout:(75/142): libverto-devel-0.3.2-2.el8.x86_64.rpm 197 kB/s | 18 kB 00:00 2023-11-06T20:41:13.865 INFO:teuthology.orchestra.run.smithi069.stdout:(100/142): python3-cherrypy-18.4.0-1.el8.noarch 744 kB/s | 384 kB 00:00 2023-11-06T20:41:13.949 INFO:teuthology.orchestra.run.smithi033.stdout:(76/142): libzstd-devel-1.4.4-1.el8.x86_64.rpm 480 kB/s | 44 kB 00:00 2023-11-06T20:41:14.065 INFO:teuthology.orchestra.run.smithi069.stdout:(101/142): python3-google-auth-1.1.1-10.el8.noa 406 kB/s | 81 kB 00:00 2023-11-06T20:41:14.099 INFO:teuthology.orchestra.run.smithi069.stdout:(102/142): python3-execnet-1.7.1-1.el8.noarch.r 710 kB/s | 355 kB 00:00 2023-11-06T20:41:14.191 INFO:teuthology.orchestra.run.smithi069.stdout:(103/142): python3-jaraco-6.2-6.el8.noarch.rpm 89 kB/s | 11 kB 00:00 2023-11-06T20:41:14.216 INFO:teuthology.orchestra.run.smithi069.stdout:(104/142): python3-jaraco-functools-2.0-4.el8.n 150 kB/s | 18 kB 00:00 2023-11-06T20:41:14.249 INFO:teuthology.orchestra.run.smithi033.stdout:(77/142): openssl-devel-1.1.1k-9.el8.x86_64.rpm 11 MB/s | 3.2 MB 00:00 2023-11-06T20:41:14.316 INFO:teuthology.orchestra.run.smithi033.stdout:(78/142): python3-numpy-1.14.3-10.el8.x86_64.rp 930 kB/s | 3.7 MB 00:04 2023-11-06T20:41:14.325 INFO:teuthology.orchestra.run.smithi069.stdout:(105/142): python3-logutils-0.3.5-11.el8.noarch 450 kB/s | 49 kB 00:00 2023-11-06T20:41:14.366 INFO:teuthology.orchestra.run.smithi033.stdout:(79/142): pcre2-devel-10.32-3.el8.x86_64.rpm 5.1 MB/s | 605 kB 00:00 2023-11-06T20:41:14.425 INFO:teuthology.orchestra.run.smithi069.stdout:(106/142): python3-more-itertools-7.2.0-3.el8.n 588 kB/s | 59 kB 00:00 2023-11-06T20:41:14.466 INFO:teuthology.orchestra.run.smithi033.stdout:(80/142): pcre2-utf32-10.32-3.el8.x86_64.rpm 2.2 MB/s | 220 kB 00:00 2023-11-06T20:41:14.575 INFO:teuthology.orchestra.run.smithi033.stdout:(81/142): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.2 MB/s | 249 kB 00:00 2023-11-06T20:41:14.701 INFO:teuthology.orchestra.run.smithi033.stdout:(82/142): python3-cryptography-3.2.1-6.el8.x86_ 5.0 MB/s | 641 kB 00:00 2023-11-06T20:41:14.792 INFO:teuthology.orchestra.run.smithi033.stdout:(83/142): pcre2-utf16-10.32-3.el8.x86_64.rpm 481 kB/s | 229 kB 00:00 2023-11-06T20:41:14.817 INFO:teuthology.orchestra.run.smithi033.stdout:(84/142): python3-jwt-1.6.1-2.el8.noarch.rpm 366 kB/s | 43 kB 00:00 2023-11-06T20:41:14.867 INFO:teuthology.orchestra.run.smithi069.stdout:(107/142): python3-pecan-1.3.2-9.el8.noarch.rpm 640 kB/s | 283 kB 00:00 2023-11-06T20:41:14.894 INFO:teuthology.orchestra.run.smithi033.stdout:(85/142): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2023-11-06T20:41:14.909 INFO:teuthology.orchestra.run.smithi033.stdout:(86/142): python3-pycparser-2.14-14.el8.noarch. 1.2 MB/s | 109 kB 00:00 2023-11-06T20:41:14.967 INFO:teuthology.orchestra.run.smithi069.stdout:(108/142): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2023-11-06T20:41:14.993 INFO:teuthology.orchestra.run.smithi033.stdout:(87/142): python3-requests-oauthlib-1.0.0-1.el8 425 kB/s | 43 kB 00:00 2023-11-06T20:41:15.044 INFO:teuthology.orchestra.run.smithi033.stdout:(88/142): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 557 kB/s | 74 kB 00:00 2023-11-06T20:41:15.084 INFO:teuthology.orchestra.run.smithi069.stdout:(109/142): python3-remoto-1.2.1-1.el8.noarch.rp 418 kB/s | 49 kB 00:00 2023-11-06T20:41:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:(89/142): zlib-devel-1.2.11-21.el8.x86_64.rpm 581 kB/s | 58 kB 00:00 2023-11-06T20:41:15.209 INFO:teuthology.orchestra.run.smithi069.stdout:(110/142): python3-repoze-lru-0.7-6.el8.noarch. 270 kB/s | 34 kB 00:00 2023-11-06T20:41:15.470 INFO:teuthology.orchestra.run.smithi033.stdout:(90/142): gperftools-libs-2.7-9.el8.x86_64.rpm 720 kB/s | 306 kB 00:00 2023-11-06T20:41:15.493 INFO:teuthology.orchestra.run.smithi069.stdout:(111/142): python3-routes-2.4.1-12.el8.noarch.r 693 kB/s | 196 kB 00:00 2023-11-06T20:41:15.495 INFO:teuthology.orchestra.run.smithi033.stdout:(91/142): leveldb-1.22-1.el8.x86_64.rpm 452 kB/s | 181 kB 00:00 2023-11-06T20:41:15.578 INFO:teuthology.orchestra.run.smithi033.stdout:(92/142): liboath-2.6.2-3.el8.x86_64.rpm 543 kB/s | 59 kB 00:00 2023-11-06T20:41:15.595 INFO:teuthology.orchestra.run.smithi033.stdout:(93/142): libunwind-1.3.1-3.el8.x86_64.rpm 750 kB/s | 75 kB 00:00 2023-11-06T20:41:15.618 INFO:teuthology.orchestra.run.smithi069.stdout:(112/142): python3-rsa-4.9-2.el8.noarch.rpm 516 kB/s | 64 kB 00:00 2023-11-06T20:41:15.704 INFO:teuthology.orchestra.run.smithi033.stdout:(94/142): python3-apipkg-1.5-6.el8.noarch.rpm 169 kB/s | 21 kB 00:00 2023-11-06T20:41:15.729 INFO:teuthology.orchestra.run.smithi033.stdout:(95/142): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 326 kB/s | 44 kB 00:00 2023-11-06T20:41:15.760 INFO:teuthology.orchestra.run.smithi069.stdout:(113/142): python3-simplegeneric-0.8.1-17.el8.n 132 kB/s | 19 kB 00:00 2023-11-06T20:41:15.802 INFO:teuthology.orchestra.run.smithi069.stdout:(114/142): python3-kubernetes-11.0.0-6.el8.noar 741 kB/s | 1.2 MB 00:01 2023-11-06T20:41:15.838 INFO:teuthology.orchestra.run.smithi033.stdout:(96/142): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-11-06T20:41:15.863 INFO:teuthology.orchestra.run.smithi033.stdout:(97/142): python3-cachetools-3.1.1-4.el8.noarch 247 kB/s | 33 kB 00:00 2023-11-06T20:41:15.919 INFO:teuthology.orchestra.run.smithi069.stdout:(115/142): python3-singledispatch-3.4.0.3-18.el 151 kB/s | 24 kB 00:00 2023-11-06T20:41:15.944 INFO:teuthology.orchestra.run.smithi069.stdout:(116/142): python3-tempora-1.14.1-5.el8.noarch. 206 kB/s | 29 kB 00:00 2023-11-06T20:41:15.947 INFO:teuthology.orchestra.run.smithi033.stdout:(98/142): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-11-06T20:41:15.989 INFO:teuthology.orchestra.run.smithi033.stdout:(99/142): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-11-06T20:41:16.045 INFO:teuthology.orchestra.run.smithi069.stdout:(117/142): python3-trustme-0.6.0-4.el8.noarch.r 220 kB/s | 27 kB 00:00 2023-11-06T20:41:16.081 INFO:teuthology.orchestra.run.smithi033.stdout:(100/142): python3-cherrypy-18.4.0-1.el8.noarch 2.8 MB/s | 384 kB 00:00 2023-11-06T20:41:16.189 INFO:teuthology.orchestra.run.smithi033.stdout:(101/142): python3-execnet-1.7.1-1.el8.noarch.r 1.7 MB/s | 355 kB 00:00 2023-11-06T20:41:16.214 INFO:teuthology.orchestra.run.smithi033.stdout:(102/142): python3-google-auth-1.1.1-10.el8.noa 610 kB/s | 81 kB 00:00 2023-11-06T20:41:16.273 INFO:teuthology.orchestra.run.smithi033.stdout:(103/142): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-11-06T20:41:16.315 INFO:teuthology.orchestra.run.smithi033.stdout:(104/142): python3-jaraco-functools-2.0-4.el8.n 175 kB/s | 18 kB 00:00 2023-11-06T20:41:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:(118/142): python3-waitress-1.4.3-1.el8.noarch. 625 kB/s | 255 kB 00:00 2023-11-06T20:41:16.407 INFO:teuthology.orchestra.run.smithi033.stdout:(105/142): python3-logutils-0.3.5-11.el8.noarch 531 kB/s | 49 kB 00:00 2023-11-06T20:41:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:(119/142): python3-webob-1.8.5-1.el8.1.noarch.r 602 kB/s | 251 kB 00:00 2023-11-06T20:41:16.482 INFO:teuthology.orchestra.run.smithi033.stdout:(106/142): python3-more-itertools-7.2.0-3.el8.n 785 kB/s | 59 kB 00:00 2023-11-06T20:41:16.487 INFO:teuthology.orchestra.run.smithi069.stdout:(120/142): python3-websocket-client-0.56.0-5.el 459 kB/s | 61 kB 00:00 2023-11-06T20:41:16.516 INFO:teuthology.orchestra.run.smithi033.stdout:(107/142): python3-kubernetes-11.0.0-6.el8.noar 4.8 MB/s | 1.2 MB 00:00 2023-11-06T20:41:16.596 INFO:teuthology.orchestra.run.smithi069.stdout:(121/142): python3-zc-lockfile-2.0-2.el8.noarch 212 kB/s | 23 kB 00:00 2023-11-06T20:41:16.608 INFO:teuthology.orchestra.run.smithi033.stdout:(108/142): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-11-06T20:41:16.625 INFO:teuthology.orchestra.run.smithi033.stdout:(109/142): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-11-06T20:41:16.637 INFO:teuthology.orchestra.run.smithi069.stdout:(122/142): python3-webtest-2.0.33-1.el8.noarch. 487 kB/s | 85 kB 00:00 2023-11-06T20:41:16.684 INFO:teuthology.orchestra.run.smithi033.stdout:(110/142): python3-remoto-1.2.1-1.el8.noarch.rp 652 kB/s | 49 kB 00:00 2023-11-06T20:41:16.716 INFO:teuthology.orchestra.run.smithi033.stdout:(111/142): python3-repoze-lru-0.7-6.el8.noarch. 447 kB/s | 34 kB 00:00 2023-11-06T20:41:16.776 INFO:teuthology.orchestra.run.smithi033.stdout:(112/142): python3-routes-2.4.1-12.el8.noarch.r 2.1 MB/s | 196 kB 00:00 2023-11-06T20:41:16.801 INFO:teuthology.orchestra.run.smithi033.stdout:(113/142): python3-rsa-4.9-2.el8.noarch.rpm 640 kB/s | 64 kB 00:00 2023-11-06T20:41:16.851 INFO:teuthology.orchestra.run.smithi033.stdout:(114/142): python3-simplegeneric-0.8.1-17.el8.n 249 kB/s | 19 kB 00:00 2023-11-06T20:41:16.877 INFO:teuthology.orchestra.run.smithi033.stdout:(115/142): python3-singledispatch-3.4.0.3-18.el 318 kB/s | 24 kB 00:00 2023-11-06T20:41:16.927 INFO:teuthology.orchestra.run.smithi033.stdout:(116/142): python3-tempora-1.14.1-5.el8.noarch. 388 kB/s | 29 kB 00:00 2023-11-06T20:41:16.978 INFO:teuthology.orchestra.run.smithi033.stdout:(117/142): python3-trustme-0.6.0-4.el8.noarch.r 274 kB/s | 27 kB 00:00 2023-11-06T20:41:17.045 INFO:teuthology.orchestra.run.smithi033.stdout:(118/142): python3-waitress-1.4.3-1.el8.noarch. 2.1 MB/s | 255 kB 00:00 2023-11-06T20:41:17.070 INFO:teuthology.orchestra.run.smithi033.stdout:(119/142): python3-webob-1.8.5-1.el8.1.noarch.r 2.7 MB/s | 251 kB 00:00 2023-11-06T20:41:17.129 INFO:teuthology.orchestra.run.smithi033.stdout:(120/142): python3-websocket-client-0.56.0-5.el 732 kB/s | 61 kB 00:00 2023-11-06T20:41:17.154 INFO:teuthology.orchestra.run.smithi033.stdout:(121/142): python3-webtest-2.0.33-1.el8.noarch. 1.0 MB/s | 85 kB 00:00 2023-11-06T20:41:17.188 INFO:teuthology.orchestra.run.smithi069.stdout:(123/142): librbd1-16.2.14-132.g4e03306b.el8.x8 7.6 MB/s | 4.2 MB 00:00 2023-11-06T20:41:17.212 INFO:teuthology.orchestra.run.smithi033.stdout:(122/142): python3-zc-lockfile-2.0-2.el8.noarch 275 kB/s | 23 kB 00:00 2023-11-06T20:41:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:(123/142): librados2-16.2.14-132.g4e03306b.el8. 10 MB/s | 3.7 MB 00:00 2023-11-06T20:41:17.613 INFO:teuthology.orchestra.run.smithi069.stdout:(124/142): NetworkManager-initscripts-updown-1. 339 kB/s | 144 kB 00:00 2023-11-06T20:41:17.674 INFO:teuthology.orchestra.run.smithi069.stdout:(125/142): librados2-16.2.14-132.g4e03306b.el8. 3.5 MB/s | 3.7 MB 00:01 2023-11-06T20:41:17.697 INFO:teuthology.orchestra.run.smithi033.stdout:(124/142): python3-scipy-1.0.0-21.module_el8.5. 2.0 MB/s | 14 MB 00:06 2023-11-06T20:41:17.716 INFO:teuthology.orchestra.run.smithi069.stdout:(126/142): acl-2.2.53-3.el8.x86_64.rpm 809 kB/s | 83 kB 00:00 2023-11-06T20:41:17.880 INFO:teuthology.orchestra.run.smithi033.stdout:(125/142): librbd1-16.2.14-132.g4e03306b.el8.x8 6.3 MB/s | 4.2 MB 00:00 2023-11-06T20:41:17.908 INFO:teuthology.orchestra.run.smithi069.stdout:(127/142): elfutils-libelf-0.189-3.el8.x86_64.r 1.2 MB/s | 232 kB 00:00 2023-11-06T20:41:17.922 INFO:teuthology.orchestra.run.smithi033.stdout:(126/142): NetworkManager-initscripts-updown-1. 367 kB/s | 144 kB 00:00 2023-11-06T20:41:17.972 INFO:teuthology.orchestra.run.smithi033.stdout:(127/142): acl-2.2.53-3.el8.x86_64.rpm 300 kB/s | 83 kB 00:00 2023-11-06T20:41:18.033 INFO:teuthology.orchestra.run.smithi069.stdout:(128/142): elfutils-libs-0.189-3.el8.x86_64.rpm 2.4 MB/s | 303 kB 00:00 2023-11-06T20:41:18.058 INFO:teuthology.orchestra.run.smithi069.stdout:(129/142): elfutils-debuginfod-client-0.189-3.e 198 kB/s | 76 kB 00:00 2023-11-06T20:41:18.139 INFO:teuthology.orchestra.run.smithi033.stdout:(128/142): elfutils-libelf-0.189-3.el8.x86_64.r 1.0 MB/s | 232 kB 00:00 2023-11-06T20:41:18.167 INFO:teuthology.orchestra.run.smithi069.stdout:(130/142): libacl-2.2.53-3.el8.x86_64.rpm 321 kB/s | 35 kB 00:00 2023-11-06T20:41:18.248 INFO:teuthology.orchestra.run.smithi033.stdout:(129/142): elfutils-debuginfod-client-0.189-3.e 207 kB/s | 76 kB 00:00 2023-11-06T20:41:18.309 INFO:teuthology.orchestra.run.smithi069.stdout:(131/142): iproute-6.2.0-5.el8.x86_64.rpm 3.0 MB/s | 854 kB 00:00 2023-11-06T20:41:18.331 INFO:teuthology.orchestra.run.smithi033.stdout:(130/142): iproute-6.2.0-5.el8.x86_64.rpm 4.3 MB/s | 854 kB 00:00 2023-11-06T20:41:18.357 INFO:teuthology.orchestra.run.smithi033.stdout:(131/142): libacl-2.2.53-3.el8.x86_64.rpm 321 kB/s | 35 kB 00:00 2023-11-06T20:41:18.368 INFO:teuthology.orchestra.run.smithi069.stdout:(132/142): libacl-devel-2.2.53-3.el8.x86_64.rpm 424 kB/s | 85 kB 00:00 2023-11-06T20:41:18.382 INFO:teuthology.orchestra.run.smithi033.stdout:(132/142): elfutils-libs-0.189-3.el8.x86_64.rpm 740 kB/s | 303 kB 00:00 2023-11-06T20:41:18.409 INFO:teuthology.orchestra.run.smithi069.stdout:(133/142): libblkid-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 220 kB 00:00 2023-11-06T20:41:18.432 INFO:teuthology.orchestra.run.smithi033.stdout:(133/142): libacl-devel-2.2.53-3.el8.x86_64.rpm 849 kB/s | 85 kB 00:00 2023-11-06T20:41:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:(134/142): libmount-2.32.1-43.el8.x86_64.rpm 1.9 MB/s | 236 kB 00:00 2023-11-06T20:41:18.549 INFO:teuthology.orchestra.run.smithi033.stdout:(134/142): libblkid-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2023-11-06T20:41:18.574 INFO:teuthology.orchestra.run.smithi033.stdout:(135/142): libmount-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 236 kB 00:00 2023-11-06T20:41:18.660 INFO:teuthology.orchestra.run.smithi069.stdout:(135/142): libsmartcols-2.32.1-43.el8.x86_64.rp 1.4 MB/s | 179 kB 00:00 2023-11-06T20:41:18.674 INFO:teuthology.orchestra.run.smithi033.stdout:(136/142): libsmartcols-2.32.1-43.el8.x86_64.rp 1.4 MB/s | 179 kB 00:00 2023-11-06T20:41:18.700 INFO:teuthology.orchestra.run.smithi033.stdout:(137/142): libfdisk-2.32.1-43.el8.x86_64.rpm 800 kB/s | 254 kB 00:00 2023-11-06T20:41:18.725 INFO:teuthology.orchestra.run.smithi033.stdout:(138/142): libuuid-2.32.1-43.el8.x86_64.rpm 658 kB/s | 99 kB 00:00 2023-11-06T20:41:18.785 INFO:teuthology.orchestra.run.smithi069.stdout:(136/142): libfdisk-2.32.1-43.el8.x86_64.rpm 608 kB/s | 254 kB 00:00 2023-11-06T20:41:18.810 INFO:teuthology.orchestra.run.smithi069.stdout:(137/142): libuuid-2.32.1-43.el8.x86_64.rpm 660 kB/s | 99 kB 00:00 2023-11-06T20:41:18.876 INFO:teuthology.orchestra.run.smithi033.stdout:(139/142): xfsdump-3.1.8-7.el8.x86_64.rpm 2.2 MB/s | 334 kB 00:00 2023-11-06T20:41:18.926 INFO:teuthology.orchestra.run.smithi033.stdout:(140/142): util-linux-user-2.32.1-43.el8.x86_64 455 kB/s | 103 kB 00:00 2023-11-06T20:41:18.944 INFO:teuthology.orchestra.run.smithi069.stdout:(138/142): util-linux-user-2.32.1-43.el8.x86_64 770 kB/s | 103 kB 00:00 2023-11-06T20:41:19.034 INFO:teuthology.orchestra.run.smithi033.stdout:(141/142): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2023-11-06T20:41:19.069 INFO:teuthology.orchestra.run.smithi069.stdout:(139/142): xfsdump-3.1.8-7.el8.x86_64.rpm 2.6 MB/s | 334 kB 00:00 2023-11-06T20:41:19.595 INFO:teuthology.orchestra.run.smithi069.stdout:(140/142): xfsprogs-5.0.0-12.el8.x86_64.rpm 2.1 MB/s | 1.1 MB 00:00 2023-11-06T20:41:19.684 INFO:teuthology.orchestra.run.smithi033.stdout:(142/142): util-linux-2.32.1-43.el8.x86_64.rpm 2.5 MB/s | 2.5 MB 00:01 2023-11-06T20:41:19.689 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2023-11-06T20:41:19.689 INFO:teuthology.orchestra.run.smithi033.stdout:Total 9.0 MB/s | 209 MB 00:23 2023-11-06T20:41:21.103 INFO:teuthology.orchestra.run.smithi069.stdout:(141/142): util-linux-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 2.5 MB 00:02 2023-11-06T20:41:21.537 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2023-11-06T20:41:21.982 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2023-11-06T20:41:21.983 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2023-11-06T20:41:23.518 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2023-11-06T20:41:23.527 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2023-11-06T20:41:25.670 INFO:teuthology.orchestra.run.smithi069.stdout:(142/142): python3-scipy-1.0.0-21.module_el8.5. 835 kB/s | 14 MB 00:16 2023-11-06T20:41:25.675 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2023-11-06T20:41:25.675 INFO:teuthology.orchestra.run.smithi069.stdout:Total 7.1 MB/s | 209 MB 00:29 2023-11-06T20:41:27.470 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2023-11-06T20:41:27.886 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2023-11-06T20:41:27.887 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2023-11-06T20:41:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2023-11-06T20:41:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/1 2023-11-06T20:41:29.240 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 1/161 2023-11-06T20:41:29.323 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2023-11-06T20:41:29.327 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2023-11-06T20:41:29.342 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/161 2023-11-06T20:41:29.507 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 2/161 2023-11-06T20:41:29.597 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 2/161 2023-11-06T20:41:29.774 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/161 2023-11-06T20:41:29.990 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/161 2023-11-06T20:41:30.179 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 5/161 2023-11-06T20:41:30.364 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/161 2023-11-06T20:41:30.556 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/161 2023-11-06T20:41:31.007 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/161 2023-11-06T20:41:31.189 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:16.2.14-132.g4e03306b.el8.x86_64 8/161 2023-11-06T20:41:31.678 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.14-132.g4e03306b.el8.x86_64 8/161 2023-11-06T20:41:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:16.2.14-132.g4e03306b.el8.x86_64 9/161 2023-11-06T20:41:31.980 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.14-132.g4e03306b.el8.x86_64 9/161 2023-11-06T20:41:32.156 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:16.2.14-132.g4e03306b.el8.x86_64 10/161 2023-11-06T20:41:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:16.2.14-132.g4e03306b.el8.x86_64 10/161 2023-11-06T20:41:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:16.2.14-132.g4e03306b.el8.x86_64 11/161 2023-11-06T20:41:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/161 2023-11-06T20:41:33.122 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/161 2023-11-06T20:41:33.289 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:16.2.14-132.g4e03306b.el8.x86_64 13/161 2023-11-06T20:41:33.529 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:16.2.14-132.g4e03306b.el8.x86_64 13/161 2023-11-06T20:41:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/161 2023-11-06T20:41:33.810 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/161 2023-11-06T20:41:34.044 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/161 2023-11-06T20:41:34.218 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2023-11-06T20:41:34.373 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:16.2.14-132.g4e03306b.el8.x86_64 17/161 2023-11-06T20:41:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:16.2.14-132.g4e03306b.el8.x86_64 17/161 2023-11-06T20:41:34.622 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:16.2.14-132.g4e03306b.el8.x86 18/161 2023-11-06T20:41:34.683 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2023-11-06T20:41:34.702 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:16.2.14-132.g4e03306b.el8.x86 18/161 2023-11-06T20:41:34.742 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/1 2023-11-06T20:41:34.909 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 1/161 2023-11-06T20:41:34.914 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libacl-2.2.53-3.el8.x86_64 19/161 2023-11-06T20:41:35.009 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/161 2023-11-06T20:41:35.085 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/161 2023-11-06T20:41:35.167 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 2/161 2023-11-06T20:41:35.240 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 21/161 2023-11-06T20:41:35.269 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 2/161 2023-11-06T20:41:35.364 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 22/161 2023-11-06T20:41:35.478 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/161 2023-11-06T20:41:35.528 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 22/161 2023-11-06T20:41:35.672 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 23/161 2023-11-06T20:41:35.711 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/161 2023-11-06T20:41:35.939 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 5/161 2023-11-06T20:41:36.144 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/161 2023-11-06T20:41:36.334 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/161 2023-11-06T20:41:36.364 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 23/161 2023-11-06T20:41:36.531 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/161 2023-11-06T20:41:36.758 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/161 2023-11-06T20:41:36.975 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:16.2.14-132.g4e03306b.el8.x86_64 8/161 2023-11-06T20:41:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/161 2023-11-06T20:41:37.419 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.14-132.g4e03306b.el8.x86_64 8/161 2023-11-06T20:41:37.422 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/161 2023-11-06T20:41:37.633 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:16.2.14-132.g4e03306b.el8.x86_64 9/161 2023-11-06T20:41:37.778 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.14-132.g4e03306b.el8.x86_64 9/161 2023-11-06T20:41:38.008 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:16.2.14-132.g4e03306b.el8.x86_64 10/161 2023-11-06T20:41:38.105 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:16.2.14-132.g4e03306b.el8.x86_64 10/161 2023-11-06T20:41:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/161 2023-11-06T20:41:38.327 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:16.2.14-132.g4e03306b.el8.x86_64 11/161 2023-11-06T20:41:38.483 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/161 2023-11-06T20:41:38.649 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/161 2023-11-06T20:41:38.832 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/161 2023-11-06T20:41:38.884 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/161 2023-11-06T20:41:38.989 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 28/161 2023-11-06T20:41:39.055 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:16.2.14-132.g4e03306b.el8.x86_64 13/161 2023-11-06T20:41:39.189 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/161 2023-11-06T20:41:39.285 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:16.2.14-132.g4e03306b.el8.x86_64 13/161 2023-11-06T20:41:39.429 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:16.2.14-132.g4e03306b.el8. 30/161 2023-11-06T20:41:39.508 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/161 2023-11-06T20:41:39.556 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:16.2.14-132.g4e03306b.el 31/161 2023-11-06T20:41:39.595 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/161 2023-11-06T20:41:39.791 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/161 2023-11-06T20:41:39.796 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:16.2.14-132.g4e03306b.el8.x86_6 32/161 2023-11-06T20:41:39.975 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2023-11-06T20:41:40.166 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:16.2.14-132.g4e03306b.el8.x86_64 17/161 2023-11-06T20:41:40.287 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:16.2.14-132.g4e03306b.el8.x86_64 17/161 2023-11-06T20:41:40.650 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:16.2.14-132.g4e03306b.el8.x86 18/161 2023-11-06T20:41:40.729 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:16.2.14-132.g4e03306b.el8.x86 18/161 2023-11-06T20:41:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/161 2023-11-06T20:41:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libacl-2.2.53-3.el8.x86_64 19/161 2023-11-06T20:41:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/161 2023-11-06T20:41:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/161 2023-11-06T20:41:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 21/161 2023-11-06T20:41:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 22/161 2023-11-06T20:41:41.581 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/161 2023-11-06T20:41:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 22/161 2023-11-06T20:41:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 23/161 2023-11-06T20:41:41.771 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/161 2023-11-06T20:41:42.421 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 23/161 2023-11-06T20:41:42.624 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/161 2023-11-06T20:41:43.292 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/161 2023-11-06T20:41:43.507 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/161 2023-11-06T20:41:44.019 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/161 2023-11-06T20:41:44.304 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/161 2023-11-06T20:41:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/161 2023-11-06T20:41:44.526 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/161 2023-11-06T20:41:44.682 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/161 2023-11-06T20:41:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2023-11-06T20:41:44.866 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/161 2023-11-06T20:41:44.885 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:16.2.14-132.g4e03306b.el8.x86_64 40/161 2023-11-06T20:41:45.031 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 28/161 2023-11-06T20:41:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:16.2.14-132.g4e03306b.el8.x86_64 41/161 2023-11-06T20:41:45.250 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:16.2.14-132.g4e03306b.el8.x86_6 42/161 2023-11-06T20:41:45.252 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/161 2023-11-06T20:41:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 43/161 2023-11-06T20:41:45.497 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:16.2.14-132.g4e03306b.el8. 30/161 2023-11-06T20:41:45.616 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 44/161 2023-11-06T20:41:45.631 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:16.2.14-132.g4e03306b.el 31/161 2023-11-06T20:41:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 45/161 2023-11-06T20:41:45.915 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:16.2.14-132.g4e03306b.el8.x86_6 32/161 2023-11-06T20:41:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 45/161 2023-11-06T20:41:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 46/161 2023-11-06T20:41:46.263 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 46/161 2023-11-06T20:41:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 47/161 2023-11-06T20:41:46.587 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 48/161 2023-11-06T20:41:46.990 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/161 2023-11-06T20:41:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 48/161 2023-11-06T20:41:47.346 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/161 2023-11-06T20:41:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 49/161 2023-11-06T20:41:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 49/161 2023-11-06T20:41:47.675 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/161 2023-11-06T20:41:47.680 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 50/161 2023-11-06T20:41:47.882 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/161 2023-11-06T20:41:47.953 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 51/161 2023-11-06T20:41:48.146 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 52/161 2023-11-06T20:41:48.315 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 53/161 2023-11-06T20:41:48.476 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 54/161 2023-11-06T20:41:48.671 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 55/161 2023-11-06T20:41:48.863 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 56/161 2023-11-06T20:41:49.041 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/161 2023-11-06T20:41:49.187 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/161 2023-11-06T20:41:49.354 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/161 2023-11-06T20:41:49.503 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/161 2023-11-06T20:41:49.698 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/161 2023-11-06T20:41:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/161 2023-11-06T20:41:49.983 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/161 2023-11-06T20:41:50.112 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/161 2023-11-06T20:41:50.302 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/161 2023-11-06T20:41:50.391 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/161 2023-11-06T20:41:50.502 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/161 2023-11-06T20:41:50.598 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/161 2023-11-06T20:41:50.677 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2023-11-06T20:41:50.885 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:16.2.14-132.g4e03306b.el8.x86_64 40/161 2023-11-06T20:41:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/161 2023-11-06T20:41:51.079 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:16.2.14-132.g4e03306b.el8.x86_64 41/161 2023-11-06T20:41:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/161 2023-11-06T20:41:51.308 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:16.2.14-132.g4e03306b.el8.x86_6 42/161 2023-11-06T20:41:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 68/161 2023-11-06T20:41:51.511 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 43/161 2023-11-06T20:41:51.693 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 69/161 2023-11-06T20:41:51.767 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 44/161 2023-11-06T20:41:51.882 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 70/161 2023-11-06T20:41:52.014 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 45/161 2023-11-06T20:41:52.093 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 71/161 2023-11-06T20:41:52.111 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 45/161 2023-11-06T20:41:52.306 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 46/161 2023-11-06T20:41:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 72/161 2023-11-06T20:41:52.388 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 46/161 2023-11-06T20:41:52.545 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 47/161 2023-11-06T20:41:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 73/161 2023-11-06T20:41:52.756 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 48/161 2023-11-06T20:41:52.932 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 74/161 2023-11-06T20:41:53.238 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 75/161 2023-11-06T20:41:53.333 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 48/161 2023-11-06T20:41:53.430 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 76/161 2023-11-06T20:41:53.623 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 49/161 2023-11-06T20:41:53.653 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 49/161 2023-11-06T20:41:53.711 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 77/161 2023-11-06T20:41:53.880 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 50/161 2023-11-06T20:41:53.934 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 78/161 2023-11-06T20:41:54.172 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 51/161 2023-11-06T20:41:54.381 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 52/161 2023-11-06T20:41:54.536 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 53/161 2023-11-06T20:41:54.694 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 54/161 2023-11-06T20:41:54.912 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 55/161 2023-11-06T20:41:55.104 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 56/161 2023-11-06T20:41:55.310 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/161 2023-11-06T20:41:55.314 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/161 2023-11-06T20:41:55.463 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/161 2023-11-06T20:41:55.530 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 80/161 2023-11-06T20:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/161 2023-11-06T20:41:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 81/161 2023-11-06T20:41:55.855 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/161 2023-11-06T20:41:56.037 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 82/161 2023-11-06T20:41:56.066 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/161 2023-11-06T20:41:56.280 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/161 2023-11-06T20:41:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/161 2023-11-06T20:41:56.509 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/161 2023-11-06T20:41:56.683 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 84/161 2023-11-06T20:41:56.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/161 2023-11-06T20:41:56.876 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 85/161 2023-11-06T20:41:56.991 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/161 2023-11-06T20:41:57.208 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 86/161 2023-11-06T20:41:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/161 2023-11-06T20:41:57.458 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 87/161 2023-11-06T20:41:57.597 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 88/161 2023-11-06T20:41:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/161 2023-11-06T20:41:57.758 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/161 2023-11-06T20:41:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 68/161 2023-11-06T20:41:58.064 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/161 2023-11-06T20:41:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 69/161 2023-11-06T20:41:58.307 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 91/161 2023-11-06T20:41:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 70/161 2023-11-06T20:41:58.436 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/161 2023-11-06T20:41:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 71/161 2023-11-06T20:41:58.633 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/161 2023-11-06T20:41:58.782 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 72/161 2023-11-06T20:41:58.831 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/161 2023-11-06T20:41:59.037 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 73/161 2023-11-06T20:41:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.7-17.el8.x86_64 95/161 2023-11-06T20:41:59.273 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/161 2023-11-06T20:41:59.438 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 97/161 2023-11-06T20:41:59.480 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 74/161 2023-11-06T20:41:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 98/161 2023-11-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 75/161 2023-11-06T20:41:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 76/161 2023-11-06T20:42:00.208 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 77/161 2023-11-06T20:42:00.441 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 78/161 2023-11-06T20:42:01.709 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/161 2023-11-06T20:42:01.957 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 80/161 2023-11-06T20:42:02.223 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 81/161 2023-11-06T20:42:02.555 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 82/161 2023-11-06T20:42:02.633 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 99/161 2023-11-06T20:42:02.833 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/161 2023-11-06T20:42:02.852 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 99/161 2023-11-06T20:42:03.092 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 84/161 2023-11-06T20:42:03.318 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 85/161 2023-11-06T20:42:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 99/161 2023-11-06T20:42:03.696 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 86/161 2023-11-06T20:42:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 87/161 2023-11-06T20:42:04.068 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:16.2.14-132.g4e03306b.el8.x86_64 100/161 2023-11-06T20:42:04.074 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 88/161 2023-11-06T20:42:04.266 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/161 2023-11-06T20:42:04.283 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:16.2.14-132.g4e03306b.el8.x86_64 100/161 2023-11-06T20:42:04.417 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:16.2.14-132.g4e03306b.el8.x86_64 101/161 2023-11-06T20:42:04.635 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/161 2023-11-06T20:42:04.894 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 91/161 2023-11-06T20:42:05.079 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/161 2023-11-06T20:42:05.272 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/161 2023-11-06T20:42:05.532 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/161 2023-11-06T20:42:05.791 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.7-17.el8.x86_64 95/161 2023-11-06T20:42:06.085 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/161 2023-11-06T20:42:06.315 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 97/161 2023-11-06T20:42:06.454 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 98/161 2023-11-06T20:42:09.204 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 99/161 2023-11-06T20:42:09.471 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 99/161 2023-11-06T20:42:10.254 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 99/161 2023-11-06T20:42:10.550 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:16.2.14-132.g4e03306b.el8.x86_64 100/161 2023-11-06T20:42:10.763 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:16.2.14-132.g4e03306b.el8.x86_64 100/161 2023-11-06T20:42:10.920 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:16.2.14-132.g4e03306b.el8.x86_64 101/161 2023-11-06T20:42:25.321 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:16.2.14-132.g4e03306b.el8.x86_64 101/161 2023-11-06T20:42:25.321 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2023-11-06T20:42:25.322 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2023-11-06T20:42:25.322 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /dev 2023-11-06T20:42:25.322 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /run 2023-11-06T20:42:25.322 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2023-11-06T20:42:25.322 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2023-11-06T20:42:25.322 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2023-11-06T20:42:25.323 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2023-11-06T20:42:25.323 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /dev 2023-11-06T20:42:25.323 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:42:25.756 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:16.2.14-132.g4e03306b.el8.x86_64 102/161 2023-11-06T20:42:26.260 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:16.2.14-132.g4e03306b.el8.x86_64 102/161 2023-11-06T20:42:26.431 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:16.2.14-132.g4e03306b.el8.x86_64 103/161 2023-11-06T20:42:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:16.2.14-132.g4e03306b.el8.x86_64 103/161 2023-11-06T20:42:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:16.2.14-132.g4e03306b.el8.x86_64 104/161 2023-11-06T20:42:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:16.2.14-132.g4e03306b.el8.x86_64 104/161 2023-11-06T20:42:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 105/161 2023-11-06T20:42:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 106/161 2023-11-06T20:42:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 107/161 2023-11-06T20:42:29.198 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 107/161 2023-11-06T20:42:29.475 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : jq-1.6-7.el8.x86_64 108/161 2023-11-06T20:42:29.689 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.1-8.el8.noarch 109/161 2023-11-06T20:42:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:16.2.14-132.g4e03306b.el8.noarch 110/161 2023-11-06T20:42:30.617 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:16.2.14-132.g4e03306b.el8.noarch 110/161 2023-11-06T20:42:30.755 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-132.g4e03306b.e 111/161 2023-11-06T20:42:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-132.g4e03306b. 112/161 2023-11-06T20:42:31.120 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:16.2.14-132.g4e03306b.el8.x86_64 101/161 2023-11-06T20:42:31.121 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2023-11-06T20:42:31.121 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2023-11-06T20:42:31.121 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /dev 2023-11-06T20:42:31.121 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /run 2023-11-06T20:42:31.122 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2023-11-06T20:42:31.122 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2023-11-06T20:42:31.122 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2023-11-06T20:42:31.122 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2023-11-06T20:42:31.123 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /dev 2023-11-06T20:42:31.123 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:42:31.146 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-132.g4e03306b.el8.noa 113/161 2023-11-06T20:42:31.584 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:16.2.14-132.g4e03306b.el8.x86_64 102/161 2023-11-06T20:42:32.058 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:16.2.14-132.g4e03306b.el8.x86_64 102/161 2023-11-06T20:42:32.251 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:16.2.14-132.g4e03306b.el8.x86_64 103/161 2023-11-06T20:42:32.345 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-132.g4e03306b.el8.noa 113/161 2023-11-06T20:42:32.613 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-132.g4e03306b.el8.n 114/161 2023-11-06T20:42:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-132.g4e03306b.el8.n 114/161 2023-11-06T20:42:33.868 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:16.2.14-132.g4e03306b.el8.x86_64 103/161 2023-11-06T20:42:34.046 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-132.g4e0 115/161 2023-11-06T20:42:34.125 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:16.2.14-132.g4e03306b.el8.x86_64 104/161 2023-11-06T20:42:34.180 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-132.g4e0 115/161 2023-11-06T20:42:34.186 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:16.2.14-132.g4e03306b.el8.x86_64 104/161 2023-11-06T20:42:34.392 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 105/161 2023-11-06T20:42:34.565 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-132.g4e03306b.el 116/161 2023-11-06T20:42:34.621 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 106/161 2023-11-06T20:42:34.739 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:16.2.14-132.g4e03306b.el8.x86_64 117/161 2023-11-06T20:42:34.794 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:16.2.14-132.g4e03306b.el8.x86_64 117/161 2023-11-06T20:42:34.808 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 107/161 2023-11-06T20:42:34.930 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 107/161 2023-11-06T20:42:34.951 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:16.2.14-132.g4e03306b.el8.noarch 118/161 2023-11-06T20:42:34.980 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-132.g4e03306b.el8.noarch 118/161 2023-11-06T20:42:35.213 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:16.2.14-132.g4e03306b.el8.x86_64 119/161 2023-11-06T20:42:35.225 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : jq-1.6-7.el8.x86_64 108/161 2023-11-06T20:42:35.362 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/161 2023-11-06T20:42:35.441 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.1-8.el8.noarch 109/161 2023-11-06T20:42:36.027 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:16.2.14-132.g4e03306b.el8.noarch 110/161 2023-11-06T20:42:36.182 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:16.2.14-132.g4e03306b.el8.noarch 110/161 2023-11-06T20:42:36.290 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-132.g4e03306b.e 111/161 2023-11-06T20:42:36.488 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-132.g4e03306b. 112/161 2023-11-06T20:42:36.716 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-132.g4e03306b.el8.noa 113/161 2023-11-06T20:42:37.798 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-132.g4e03306b.el8.noa 113/161 2023-11-06T20:42:38.095 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-132.g4e03306b.el8.n 114/161 2023-11-06T20:42:39.243 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-132.g4e03306b.el8.n 114/161 2023-11-06T20:42:39.449 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-132.g4e0 115/161 2023-11-06T20:42:39.572 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-132.g4e0 115/161 2023-11-06T20:42:39.946 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-132.g4e03306b.el 116/161 2023-11-06T20:42:40.179 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:16.2.14-132.g4e03306b.el8.x86_64 117/161 2023-11-06T20:42:40.232 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:16.2.14-132.g4e03306b.el8.x86_64 117/161 2023-11-06T20:42:40.392 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:16.2.14-132.g4e03306b.el8.noarch 118/161 2023-11-06T20:42:40.420 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-132.g4e03306b.el8.noarch 118/161 2023-11-06T20:42:40.514 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/161 2023-11-06T20:42:40.614 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:16.2.14-132.g4e03306b.el8.x86_64 119/161 2023-11-06T20:42:40.790 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/161 2023-11-06T20:42:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:16.2.14-132.g4e03306b.el8.x86_64 121/161 2023-11-06T20:42:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:16.2.14-132.g4e03306b.el8.x86_64 122/161 2023-11-06T20:42:42.157 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-132.g4e03306b.el8.x86_64 122/161 2023-11-06T20:42:42.328 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-132.g4e033 123/161 2023-11-06T20:42:42.401 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-132.g4e033 123/161 2023-11-06T20:42:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 124/161 2023-11-06T20:42:43.004 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 124/161 2023-11-06T20:42:43.178 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 125/161 2023-11-06T20:42:43.309 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 125/161 2023-11-06T20:42:44.654 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 126/161 2023-11-06T20:42:44.950 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 127/161 2023-11-06T20:42:45.176 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/161 2023-11-06T20:42:45.410 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 129/161 2023-11-06T20:42:45.549 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 130/161 2023-11-06T20:42:45.580 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/161 2023-11-06T20:42:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:16.2.14-132.g4e03306b.el8.x86_ 131/161 2023-11-06T20:42:46.005 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : acl-2.2.53-3.el8.x86_64 132/161 2023-11-06T20:42:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libacl-devel-2.2.53-3.el8.x86_64 133/161 2023-11-06T20:42:46.459 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsdump-3.1.8-7.el8.x86_64 134/161 2023-11-06T20:42:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:16.2.14-132.g4e03306b.el8.noarch 135/161 2023-11-06T20:42:46.858 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:16.2.14-132.g4e03306b.el8.x86_64 121/161 2023-11-06T20:42:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 136/161 2023-11-06T20:42:47.061 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:16.2.14-132.g4e03306b.el8.x86_64 122/161 2023-11-06T20:42:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-132.g4e03306b.el8.x86_64 122/161 2023-11-06T20:42:47.250 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:16.2.14-132.g4e03306b.el8.x86_64 137/161 2023-11-06T20:42:47.406 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-132.g4e033 123/161 2023-11-06T20:42:47.476 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-132.g4e033 123/161 2023-11-06T20:42:47.505 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 138/161 2023-11-06T20:42:47.661 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-11.e 139/161 2023-11-06T20:42:47.665 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 124/161 2023-11-06T20:42:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-11.e 139/161 2023-11-06T20:42:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 140/161 2023-11-06T20:42:48.057 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 124/161 2023-11-06T20:42:48.152 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.1-9.el8.x86_64 141/161 2023-11-06T20:42:48.265 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 141/161 2023-11-06T20:42:48.314 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 125/161 2023-11-06T20:42:48.439 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 125/161 2023-11-06T20:42:48.477 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.0.4-10.el8.x86_64 142/161 2023-11-06T20:42:48.505 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 142/161 2023-11-06T20:42:48.671 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 143/161 2023-11-06T20:42:48.865 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 144/161 2023-11-06T20:42:49.030 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 145/161 2023-11-06T20:42:49.030 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 146/161 2023-11-06T20:42:49.299 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 146/161 2023-11-06T20:42:49.299 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 147/161 2023-11-06T20:42:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 147/161 2023-11-06T20:42:49.620 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libacl-devel-2.2.53-1.el8.x86_64 148/161 2023-11-06T20:42:49.639 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 149/161 2023-11-06T20:42:49.686 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 126/161 2023-11-06T20:42:49.742 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 149/161 2023-11-06T20:42:49.742 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 150/161 2023-11-06T20:42:49.997 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 150/161 2023-11-06T20:42:50.003 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 127/161 2023-11-06T20:42:50.159 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : acl-2.2.53-1.el8.x86_64 151/161 2023-11-06T20:42:50.240 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/161 2023-11-06T20:42:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 152/161 2023-11-06T20:42:50.479 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 153/161 2023-11-06T20:42:50.529 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 129/161 2023-11-06T20:42:50.693 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 130/161 2023-11-06T20:42:50.695 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 154/161 2023-11-06T20:42:50.696 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 155/161 2023-11-06T20:42:50.912 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:16.2.14-132.g4e03306b.el8.x86_ 131/161 2023-11-06T20:42:50.913 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 155/161 2023-11-06T20:42:50.914 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 156/161 2023-11-06T20:42:51.122 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 156/161 2023-11-06T20:42:51.123 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 157/161 2023-11-06T20:42:51.219 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : acl-2.2.53-3.el8.x86_64 132/161 2023-11-06T20:42:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 157/161 2023-11-06T20:42:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 158/161 2023-11-06T20:42:51.440 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libacl-devel-2.2.53-3.el8.x86_64 133/161 2023-11-06T20:42:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 158/161 2023-11-06T20:42:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 159/161 2023-11-06T20:42:51.736 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : xfsdump-3.1.8-7.el8.x86_64 134/161 2023-11-06T20:42:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 159/161 2023-11-06T20:42:51.953 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 160/161 2023-11-06T20:42:51.972 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:16.2.14-132.g4e03306b.el8.noarch 135/161 2023-11-06T20:42:52.062 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libacl-2.2.53-1.el8.x86_64 161/161 2023-11-06T20:42:52.197 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 136/161 2023-11-06T20:42:52.282 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libacl-2.2.53-1.el8.x86_64 161/161 2023-11-06T20:42:52.282 INFO:teuthology.orchestra.run.smithi033.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-06T20:42:52.283 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:42:52.564 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:16.2.14-132.g4e03306b.el8.x86_64 137/161 2023-11-06T20:42:52.849 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 138/161 2023-11-06T20:42:52.997 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-11.e 139/161 2023-11-06T20:42:53.022 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-11.e 139/161 2023-11-06T20:42:53.030 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:16.2.14-132.g4e03306b.el8.x86_64 1/161 2023-11-06T20:42:53.031 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:16.2.14-132.g4e03306b.el8.x86_64 2/161 2023-11-06T20:42:53.031 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 3/161 2023-11-06T20:42:53.031 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 4/161 2023-11-06T20:42:53.031 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-132.g4e033 5/161 2023-11-06T20:42:53.032 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:16.2.14-132.g4e03306b.el8.x86_64 6/161 2023-11-06T20:42:53.032 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:16.2.14-132.g4e03306b.el8.x86_64 7/161 2023-11-06T20:42:53.032 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:16.2.14-132.g4e03306b.el8.x86_64 8/161 2023-11-06T20:42:53.032 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:16.2.14-132.g4e03306b.el8.x86_64 9/161 2023-11-06T20:42:53.032 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:16.2.14-132.g4e03306b.el8.x86_64 10/161 2023-11-06T20:42:53.033 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:16.2.14-132.g4e03306b.el8.x86_64 11/161 2023-11-06T20:42:53.033 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:16.2.14-132.g4e03306b.el8.x86_64 12/161 2023-11-06T20:42:53.033 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 13/161 2023-11-06T20:42:53.033 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:16.2.14-132.g4e03306b.el8.x86_ 14/161 2023-11-06T20:42:53.033 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:16.2.14-132.g4e03306b.el8.x86_64 15/161 2023-11-06T20:42:53.034 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:16.2.14-132.g4e03306b.el8.x86_64 16/161 2023-11-06T20:42:53.034 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:16.2.14-132.g4e03306b.el8.x86_6 17/161 2023-11-06T20:42:53.034 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:16.2.14-132.g4e03306b.el8.x86 18/161 2023-11-06T20:42:53.034 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:16.2.14-132.g4e03306b.el8.x86_64 19/161 2023-11-06T20:42:53.034 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:16.2.14-132.g4e03306b.el 20/161 2023-11-06T20:42:53.034 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:16.2.14-132.g4e03306b.el8. 21/161 2023-11-06T20:42:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:16.2.14-132.g4e03306b.el8.x86_6 22/161 2023-11-06T20:42:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:16.2.14-132.g4e03306b.el8.x86_64 23/161 2023-11-06T20:42:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:16.2.14-132.g4e03306b.el8.x86_64 24/161 2023-11-06T20:42:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:16.2.14-132.g4e03306b.el8.x86_64 25/161 2023-11-06T20:42:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 26/161 2023-11-06T20:42:53.036 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 27/161 2023-11-06T20:42:53.036 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:16.2.14-132.g4e03306b.el8.x86_64 28/161 2023-11-06T20:42:53.036 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-132.g4e03306b. 29/161 2023-11-06T20:42:53.036 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-132.g4e03306b.el8.noa 30/161 2023-11-06T20:42:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-132.g4e03306b.el8.n 31/161 2023-11-06T20:42:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-132.g4e0 32/161 2023-11-06T20:42:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-132.g4e03306b.el 33/161 2023-11-06T20:42:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:16.2.14-132.g4e03306b.el8.noarch 34/161 2023-11-06T20:42:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-132.g4e03306b.e 35/161 2023-11-06T20:42:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:16.2.14-132.g4e03306b.el8.noarch 36/161 2023-11-06T20:42:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:16.2.14-132.g4e03306b.el8.noarch 37/161 2023-11-06T20:42:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.1-8.el8.noarch 38/161 2023-11-06T20:42:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.0.4-10.el8.x86_64 39/161 2023-11-06T20:42:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.1-9.el8.x86_64 40/161 2023-11-06T20:42:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : jq-1.6-7.el8.x86_64 41/161 2023-11-06T20:42:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 42/161 2023-11-06T20:42:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 43/161 2023-11-06T20:42:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 44/161 2023-11-06T20:42:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 45/161 2023-11-06T20:42:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 46/161 2023-11-06T20:42:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 47/161 2023-11-06T20:42:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 48/161 2023-11-06T20:42:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 49/161 2023-11-06T20:42:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 50/161 2023-11-06T20:42:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 51/161 2023-11-06T20:42:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 52/161 2023-11-06T20:42:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 53/161 2023-11-06T20:42:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 54/161 2023-11-06T20:42:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 55/161 2023-11-06T20:42:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 56/161 2023-11-06T20:42:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 57/161 2023-11-06T20:42:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 58/161 2023-11-06T20:42:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 59/161 2023-11-06T20:42:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 60/161 2023-11-06T20:42:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 61/161 2023-11-06T20:42:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 62/161 2023-11-06T20:42:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 63/161 2023-11-06T20:42:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 64/161 2023-11-06T20:42:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 65/161 2023-11-06T20:42:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 66/161 2023-11-06T20:42:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 67/161 2023-11-06T20:42:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 68/161 2023-11-06T20:42:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 69/161 2023-11-06T20:42:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 70/161 2023-11-06T20:42:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 71/161 2023-11-06T20:42:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 72/161 2023-11-06T20:42:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 73/161 2023-11-06T20:42:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 74/161 2023-11-06T20:42:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 75/161 2023-11-06T20:42:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 76/161 2023-11-06T20:42:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 77/161 2023-11-06T20:42:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/161 2023-11-06T20:42:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 79/161 2023-11-06T20:42:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 80/161 2023-11-06T20:42:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 81/161 2023-11-06T20:42:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 82/161 2023-11-06T20:42:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 83/161 2023-11-06T20:42:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 84/161 2023-11-06T20:42:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 85/161 2023-11-06T20:42:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 86/161 2023-11-06T20:42:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 87/161 2023-11-06T20:42:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 88/161 2023-11-06T20:42:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 89/161 2023-11-06T20:42:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 90/161 2023-11-06T20:42:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 91/161 2023-11-06T20:42:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : leveldb-1.22-1.el8.x86_64 92/161 2023-11-06T20:42:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 93/161 2023-11-06T20:42:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 94/161 2023-11-06T20:42:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 95/161 2023-11-06T20:42:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 96/161 2023-11-06T20:42:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2023-11-06T20:42:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 98/161 2023-11-06T20:42:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 99/161 2023-11-06T20:42:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 100/161 2023-11-06T20:42:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 101/161 2023-11-06T20:42:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 102/161 2023-11-06T20:42:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 103/161 2023-11-06T20:42:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 104/161 2023-11-06T20:42:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 105/161 2023-11-06T20:42:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 106/161 2023-11-06T20:42:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 107/161 2023-11-06T20:42:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 108/161 2023-11-06T20:42:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 109/161 2023-11-06T20:42:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-2.6-1.el8.noarch 110/161 2023-11-06T20:42:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 111/161 2023-11-06T20:42:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 112/161 2023-11-06T20:42:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 113/161 2023-11-06T20:42:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 114/161 2023-11-06T20:42:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 115/161 2023-11-06T20:42:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 116/161 2023-11-06T20:42:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 117/161 2023-11-06T20:42:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 118/161 2023-11-06T20:42:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 119/161 2023-11-06T20:42:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 120/161 2023-11-06T20:42:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 121/161 2023-11-06T20:42:53.064 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 122/161 2023-11-06T20:42:53.064 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 123/161 2023-11-06T20:42:53.064 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.14-132.g4e03306b.el8.x86_64 124/161 2023-11-06T20:42:53.064 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 125/161 2023-11-06T20:42:53.064 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.14-132.g4e03306b.el8.x86_64 126/161 2023-11-06T20:42:53.065 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 127/161 2023-11-06T20:42:53.065 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-11.e 128/161 2023-11-06T20:42:53.065 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 129/161 2023-11-06T20:42:53.065 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : acl-2.2.53-3.el8.x86_64 130/161 2023-11-06T20:42:53.065 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : acl-2.2.53-1.el8.x86_64 131/161 2023-11-06T20:42:53.065 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 132/161 2023-11-06T20:42:53.066 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 133/161 2023-11-06T20:42:53.066 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 134/161 2023-11-06T20:42:53.066 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 135/161 2023-11-06T20:42:53.066 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 136/161 2023-11-06T20:42:53.066 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 137/161 2023-11-06T20:42:53.066 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 138/161 2023-11-06T20:42:53.067 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 139/161 2023-11-06T20:42:53.067 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-2.2.53-3.el8.x86_64 140/161 2023-11-06T20:42:53.067 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-2.2.53-1.el8.x86_64 141/161 2023-11-06T20:42:53.067 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-devel-2.2.53-3.el8.x86_64 142/161 2023-11-06T20:42:53.067 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-devel-2.2.53-1.el8.x86_64 143/161 2023-11-06T20:42:53.068 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 144/161 2023-11-06T20:42:53.068 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 145/161 2023-11-06T20:42:53.068 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 146/161 2023-11-06T20:42:53.068 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 147/161 2023-11-06T20:42:53.068 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 148/161 2023-11-06T20:42:53.068 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 149/161 2023-11-06T20:42:53.069 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 150/161 2023-11-06T20:42:53.069 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 151/161 2023-11-06T20:42:53.069 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 152/161 2023-11-06T20:42:53.069 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 153/161 2023-11-06T20:42:53.069 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 154/161 2023-11-06T20:42:53.070 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 155/161 2023-11-06T20:42:53.070 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 156/161 2023-11-06T20:42:53.070 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 157/161 2023-11-06T20:42:53.070 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.8-7.el8.x86_64 158/161 2023-11-06T20:42:53.070 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 159/161 2023-11-06T20:42:53.071 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2023-11-06T20:42:53.221 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 140/161 2023-11-06T20:42:53.463 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.1-9.el8.x86_64 141/161 2023-11-06T20:42:53.571 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 141/161 2023-11-06T20:42:53.830 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.0.4-10.el8.x86_64 142/161 2023-11-06T20:42:53.857 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 142/161 2023-11-06T20:42:54.057 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 143/161 2023-11-06T20:42:54.253 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 144/161 2023-11-06T20:42:54.455 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 145/161 2023-11-06T20:42:54.455 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 146/161 2023-11-06T20:42:54.760 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 146/161 2023-11-06T20:42:54.760 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 147/161 2023-11-06T20:42:55.017 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 147/161 2023-11-06T20:42:55.106 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libacl-devel-2.2.53-1.el8.x86_64 148/161 2023-11-06T20:42:55.124 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 149/161 2023-11-06T20:42:55.236 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 149/161 2023-11-06T20:42:55.237 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 150/161 2023-11-06T20:42:55.276 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2023-11-06T20:42:55.277 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:42:55.277 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2023-11-06T20:42:55.277 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-initscripts-updown-1:1.40.16-11.el8.noarch 2023-11-06T20:42:55.277 INFO:teuthology.orchestra.run.smithi033.stdout: acl-2.2.53-3.el8.x86_64 2023-11-06T20:42:55.278 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-11-06T20:42:55.278 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-11-06T20:42:55.278 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-11-06T20:42:55.278 INFO:teuthology.orchestra.run.smithi033.stdout: iproute-6.2.0-5.el8.x86_64 2023-11-06T20:42:55.278 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-2.2.53-3.el8.x86_64 2023-11-06T20:42:55.279 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-devel-2.2.53-3.el8.x86_64 2023-11-06T20:42:55.279 INFO:teuthology.orchestra.run.smithi033.stdout: libblkid-2.32.1-43.el8.x86_64 2023-11-06T20:42:55.279 INFO:teuthology.orchestra.run.smithi033.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-11-06T20:42:55.279 INFO:teuthology.orchestra.run.smithi033.stdout: libmount-2.32.1-43.el8.x86_64 2023-11-06T20:42:55.279 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.280 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.280 INFO:teuthology.orchestra.run.smithi033.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-11-06T20:42:55.280 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-2.32.1-43.el8.x86_64 2023-11-06T20:42:55.280 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-2.32.1-43.el8.x86_64 2023-11-06T20:42:55.280 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-11-06T20:42:55.281 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump-3.1.8-7.el8.x86_64 2023-11-06T20:42:55.281 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-11-06T20:42:55.281 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2023-11-06T20:42:55.281 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.1-8.el8.noarch 2023-11-06T20:42:55.281 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.0.4-10.el8.x86_64 2023-11-06T20:42:55.282 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.282 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.282 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.282 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.282 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:42:55.283 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.283 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.283 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.283 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:42:55.283 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:42:55.284 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:42:55.284 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:42:55.284 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:42:55.284 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.284 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.285 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:42:55.285 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.285 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.285 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.285 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:42:55.286 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.286 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:42:55.286 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-11-06T20:42:55.286 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.1-9.el8.x86_64 2023-11-06T20:42:55.287 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.7-17.el8.x86_64 2023-11-06T20:42:55.287 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-06T20:42:55.287 INFO:teuthology.orchestra.run.smithi033.stdout: jq-1.6-7.el8.x86_64 2023-11-06T20:42:55.287 INFO:teuthology.orchestra.run.smithi033.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-06T20:42:55.287 INFO:teuthology.orchestra.run.smithi033.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-11-06T20:42:55.288 INFO:teuthology.orchestra.run.smithi033.stdout: leveldb-1.22-1.el8.x86_64 2023-11-06T20:42:55.288 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-06T20:42:55.288 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.288 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.288 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.289 INFO:teuthology.orchestra.run.smithi033.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-11-06T20:42:55.289 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-8.5.0-21.el8.x86_64 2023-11-06T20:42:55.289 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-06T20:42:55.289 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-8.5.0-21.el8.x86_64 2023-11-06T20:42:55.289 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-11-06T20:42:55.290 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.290 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.290 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-06T20:42:55.290 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.290 INFO:teuthology.orchestra.run.smithi033.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-11-06T20:42:55.290 INFO:teuthology.orchestra.run.smithi033.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-06T20:42:55.291 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-06T20:42:55.291 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-06T20:42:55.291 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-11-06T20:42:55.291 INFO:teuthology.orchestra.run.smithi033.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-11-06T20:42:55.291 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-11-06T20:42:55.292 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-06T20:42:55.292 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-06T20:42:55.292 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.15-4.el8.x86_64 2023-11-06T20:42:55.292 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-11-06T20:42:55.292 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-11-06T20:42:55.293 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-11-06T20:42:55.293 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-11-06T20:42:55.293 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-11-06T20:42:55.293 INFO:teuthology.orchestra.run.smithi033.stdout: python3-apipkg-1.5-6.el8.noarch 2023-11-06T20:42:55.293 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-06T20:42:55.293 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-06T20:42:55.294 INFO:teuthology.orchestra.run.smithi033.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-06T20:42:55.294 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-06T20:42:55.294 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.294 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.294 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.295 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-06T20:42:55.295 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-11-06T20:42:55.295 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-06T20:42:55.295 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-06T20:42:55.295 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-11-06T20:42:55.296 INFO:teuthology.orchestra.run.smithi033.stdout: python3-execnet-1.7.1-1.el8.noarch 2023-11-06T20:42:55.296 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-06T20:42:55.296 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-06T20:42:55.296 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-06T20:42:55.296 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-06T20:42:55.297 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-06T20:42:55.297 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-06T20:42:55.297 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-06T20:42:55.297 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-06T20:42:55.297 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-06T20:42:55.298 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.0.6-14.el8.noarch 2023-11-06T20:42:55.298 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-06T20:42:55.298 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-06T20:42:55.298 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-06T20:42:55.298 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-06T20:42:55.299 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-06T20:42:55.299 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-06T20:42:55.299 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-2.6-1.el8.noarch 2023-11-06T20:42:55.299 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-06T20:42:55.299 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-06T20:42:55.300 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-06T20:42:55.300 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-06T20:42:55.300 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-06T20:42:55.300 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2017.2-11.el8.noarch 2023-11-06T20:42:55.300 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.301 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.301 INFO:teuthology.orchestra.run.smithi033.stdout: python3-remoto-1.2.1-1.el8.noarch 2023-11-06T20:42:55.301 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-06T20:42:55.301 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-06T20:42:55.301 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.302 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-06T20:42:55.302 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-06T20:42:55.302 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-11-06T20:42:55.302 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-06T20:42:55.303 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-06T20:42:55.303 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-06T20:42:55.303 INFO:teuthology.orchestra.run.smithi033.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-06T20:42:55.303 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-06T20:42:55.303 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-06T20:42:55.304 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-06T20:42:55.304 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-06T20:42:55.304 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-06T20:42:55.304 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-06T20:42:55.304 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.305 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.305 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:42:55.305 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-06T20:42:55.305 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-11-06T20:42:55.305 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-06T20:42:55.306 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-11-06T20:42:55.306 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:42:55.306 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2023-11-06T20:42:55.533 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 150/161 2023-11-06T20:42:55.581 DEBUG:teuthology.parallel:result is None 2023-11-06T20:42:55.737 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : acl-2.2.53-1.el8.x86_64 151/161 2023-11-06T20:42:55.948 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 152/161 2023-11-06T20:42:56.146 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 153/161 2023-11-06T20:42:56.398 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 154/161 2023-11-06T20:42:56.399 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 155/161 2023-11-06T20:42:56.674 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 155/161 2023-11-06T20:42:56.675 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 156/161 2023-11-06T20:42:56.924 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 156/161 2023-11-06T20:42:56.925 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 157/161 2023-11-06T20:42:57.150 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 157/161 2023-11-06T20:42:57.150 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 158/161 2023-11-06T20:42:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 158/161 2023-11-06T20:42:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 159/161 2023-11-06T20:42:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 159/161 2023-11-06T20:42:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 160/161 2023-11-06T20:42:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libacl-2.2.53-1.el8.x86_64 161/161 2023-11-06T20:42:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libacl-2.2.53-1.el8.x86_64 161/161 2023-11-06T20:42:58.242 INFO:teuthology.orchestra.run.smithi069.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-06T20:42:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:42:58.976 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:16.2.14-132.g4e03306b.el8.x86_64 1/161 2023-11-06T20:42:58.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:16.2.14-132.g4e03306b.el8.x86_64 2/161 2023-11-06T20:42:58.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 3/161 2023-11-06T20:42:58.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 4/161 2023-11-06T20:42:58.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-132.g4e033 5/161 2023-11-06T20:42:58.977 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:16.2.14-132.g4e03306b.el8.x86_64 6/161 2023-11-06T20:42:58.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:16.2.14-132.g4e03306b.el8.x86_64 7/161 2023-11-06T20:42:58.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:16.2.14-132.g4e03306b.el8.x86_64 8/161 2023-11-06T20:42:58.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:16.2.14-132.g4e03306b.el8.x86_64 9/161 2023-11-06T20:42:58.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:16.2.14-132.g4e03306b.el8.x86_64 10/161 2023-11-06T20:42:58.978 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:16.2.14-132.g4e03306b.el8.x86_64 11/161 2023-11-06T20:42:58.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:16.2.14-132.g4e03306b.el8.x86_64 12/161 2023-11-06T20:42:58.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 13/161 2023-11-06T20:42:58.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:16.2.14-132.g4e03306b.el8.x86_ 14/161 2023-11-06T20:42:58.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:16.2.14-132.g4e03306b.el8.x86_64 15/161 2023-11-06T20:42:58.979 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:16.2.14-132.g4e03306b.el8.x86_64 16/161 2023-11-06T20:42:58.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:16.2.14-132.g4e03306b.el8.x86_6 17/161 2023-11-06T20:42:58.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:16.2.14-132.g4e03306b.el8.x86 18/161 2023-11-06T20:42:58.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:16.2.14-132.g4e03306b.el8.x86_64 19/161 2023-11-06T20:42:58.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:16.2.14-132.g4e03306b.el 20/161 2023-11-06T20:42:58.980 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:16.2.14-132.g4e03306b.el8. 21/161 2023-11-06T20:42:58.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:16.2.14-132.g4e03306b.el8.x86_6 22/161 2023-11-06T20:42:58.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:16.2.14-132.g4e03306b.el8.x86_64 23/161 2023-11-06T20:42:58.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:16.2.14-132.g4e03306b.el8.x86_64 24/161 2023-11-06T20:42:58.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:16.2.14-132.g4e03306b.el8.x86_64 25/161 2023-11-06T20:42:58.981 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 26/161 2023-11-06T20:42:58.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 27/161 2023-11-06T20:42:58.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:16.2.14-132.g4e03306b.el8.x86_64 28/161 2023-11-06T20:42:58.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-132.g4e03306b. 29/161 2023-11-06T20:42:58.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-132.g4e03306b.el8.noa 30/161 2023-11-06T20:42:58.982 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-132.g4e03306b.el8.n 31/161 2023-11-06T20:42:58.983 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-132.g4e0 32/161 2023-11-06T20:42:58.983 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-132.g4e03306b.el 33/161 2023-11-06T20:42:58.983 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:16.2.14-132.g4e03306b.el8.noarch 34/161 2023-11-06T20:42:58.983 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-132.g4e03306b.e 35/161 2023-11-06T20:42:58.983 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:16.2.14-132.g4e03306b.el8.noarch 36/161 2023-11-06T20:42:58.984 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:16.2.14-132.g4e03306b.el8.noarch 37/161 2023-11-06T20:42:58.984 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.1-8.el8.noarch 38/161 2023-11-06T20:42:58.984 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.0.4-10.el8.x86_64 39/161 2023-11-06T20:42:58.984 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.1-9.el8.x86_64 40/161 2023-11-06T20:42:58.985 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : jq-1.6-7.el8.x86_64 41/161 2023-11-06T20:42:58.985 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 42/161 2023-11-06T20:42:58.985 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 43/161 2023-11-06T20:42:58.986 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 44/161 2023-11-06T20:42:58.986 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 45/161 2023-11-06T20:42:58.986 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 46/161 2023-11-06T20:42:58.986 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 47/161 2023-11-06T20:42:58.987 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 48/161 2023-11-06T20:42:58.987 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 49/161 2023-11-06T20:42:58.987 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 50/161 2023-11-06T20:42:58.987 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 51/161 2023-11-06T20:42:58.987 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 52/161 2023-11-06T20:42:58.988 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 53/161 2023-11-06T20:42:58.988 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 54/161 2023-11-06T20:42:58.988 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 55/161 2023-11-06T20:42:58.988 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 56/161 2023-11-06T20:42:58.988 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 57/161 2023-11-06T20:42:58.989 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 58/161 2023-11-06T20:42:58.989 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 59/161 2023-11-06T20:42:58.989 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 60/161 2023-11-06T20:42:58.989 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 61/161 2023-11-06T20:42:58.989 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 62/161 2023-11-06T20:42:58.990 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 63/161 2023-11-06T20:42:58.990 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 64/161 2023-11-06T20:42:58.990 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 65/161 2023-11-06T20:42:58.990 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 66/161 2023-11-06T20:42:58.990 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 67/161 2023-11-06T20:42:58.991 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 68/161 2023-11-06T20:42:58.991 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 69/161 2023-11-06T20:42:58.991 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 70/161 2023-11-06T20:42:58.991 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 71/161 2023-11-06T20:42:58.991 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 72/161 2023-11-06T20:42:58.991 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 73/161 2023-11-06T20:42:58.992 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 74/161 2023-11-06T20:42:58.992 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 75/161 2023-11-06T20:42:58.992 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 76/161 2023-11-06T20:42:58.993 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 77/161 2023-11-06T20:42:58.993 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/161 2023-11-06T20:42:58.993 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 79/161 2023-11-06T20:42:58.993 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 80/161 2023-11-06T20:42:58.993 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 81/161 2023-11-06T20:42:58.994 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 82/161 2023-11-06T20:42:58.994 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 83/161 2023-11-06T20:42:58.994 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 84/161 2023-11-06T20:42:58.995 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 85/161 2023-11-06T20:42:58.996 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 86/161 2023-11-06T20:42:58.996 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 87/161 2023-11-06T20:42:58.997 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 88/161 2023-11-06T20:42:58.997 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 89/161 2023-11-06T20:42:58.997 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 90/161 2023-11-06T20:42:58.997 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 91/161 2023-11-06T20:42:58.997 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : leveldb-1.22-1.el8.x86_64 92/161 2023-11-06T20:42:58.998 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 93/161 2023-11-06T20:42:58.998 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 94/161 2023-11-06T20:42:58.998 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 95/161 2023-11-06T20:42:58.998 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 96/161 2023-11-06T20:42:58.999 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2023-11-06T20:42:58.999 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 98/161 2023-11-06T20:42:58.999 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 99/161 2023-11-06T20:42:59.000 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 100/161 2023-11-06T20:42:59.000 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 101/161 2023-11-06T20:42:59.001 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 102/161 2023-11-06T20:42:59.001 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 103/161 2023-11-06T20:42:59.001 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 104/161 2023-11-06T20:42:59.002 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 105/161 2023-11-06T20:42:59.002 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 106/161 2023-11-06T20:42:59.002 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 107/161 2023-11-06T20:42:59.002 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 108/161 2023-11-06T20:42:59.003 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 109/161 2023-11-06T20:42:59.003 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-2.6-1.el8.noarch 110/161 2023-11-06T20:42:59.003 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 111/161 2023-11-06T20:42:59.004 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 112/161 2023-11-06T20:42:59.004 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 113/161 2023-11-06T20:42:59.004 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 114/161 2023-11-06T20:42:59.005 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 115/161 2023-11-06T20:42:59.005 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 116/161 2023-11-06T20:42:59.005 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 117/161 2023-11-06T20:42:59.005 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 118/161 2023-11-06T20:42:59.006 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 119/161 2023-11-06T20:42:59.006 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 120/161 2023-11-06T20:42:59.006 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 121/161 2023-11-06T20:42:59.006 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 122/161 2023-11-06T20:42:59.007 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 123/161 2023-11-06T20:42:59.007 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.14-132.g4e03306b.el8.x86_64 124/161 2023-11-06T20:42:59.008 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 125/161 2023-11-06T20:42:59.008 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.14-132.g4e03306b.el8.x86_64 126/161 2023-11-06T20:42:59.008 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 127/161 2023-11-06T20:42:59.009 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-11.e 128/161 2023-11-06T20:42:59.009 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 129/161 2023-11-06T20:42:59.009 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : acl-2.2.53-3.el8.x86_64 130/161 2023-11-06T20:42:59.009 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : acl-2.2.53-1.el8.x86_64 131/161 2023-11-06T20:42:59.010 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 132/161 2023-11-06T20:42:59.010 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 133/161 2023-11-06T20:42:59.010 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 134/161 2023-11-06T20:42:59.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 135/161 2023-11-06T20:42:59.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 136/161 2023-11-06T20:42:59.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 137/161 2023-11-06T20:42:59.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 138/161 2023-11-06T20:42:59.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 139/161 2023-11-06T20:42:59.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libacl-2.2.53-3.el8.x86_64 140/161 2023-11-06T20:42:59.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libacl-2.2.53-1.el8.x86_64 141/161 2023-11-06T20:42:59.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libacl-devel-2.2.53-3.el8.x86_64 142/161 2023-11-06T20:42:59.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libacl-devel-2.2.53-1.el8.x86_64 143/161 2023-11-06T20:42:59.013 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 144/161 2023-11-06T20:42:59.013 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 145/161 2023-11-06T20:42:59.013 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 146/161 2023-11-06T20:42:59.013 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 147/161 2023-11-06T20:42:59.013 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 148/161 2023-11-06T20:42:59.014 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 149/161 2023-11-06T20:42:59.014 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 150/161 2023-11-06T20:42:59.014 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 151/161 2023-11-06T20:42:59.014 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 152/161 2023-11-06T20:42:59.014 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 153/161 2023-11-06T20:42:59.015 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 154/161 2023-11-06T20:42:59.015 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 155/161 2023-11-06T20:42:59.015 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 156/161 2023-11-06T20:42:59.015 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 157/161 2023-11-06T20:42:59.015 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsdump-3.1.8-7.el8.x86_64 158/161 2023-11-06T20:42:59.016 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 159/161 2023-11-06T20:42:59.016 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2023-11-06T20:43:01.137 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2023-11-06T20:43:01.137 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:01.138 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2023-11-06T20:43:01.138 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-initscripts-updown-1:1.40.16-11.el8.noarch 2023-11-06T20:43:01.138 INFO:teuthology.orchestra.run.smithi069.stdout: acl-2.2.53-3.el8.x86_64 2023-11-06T20:43:01.138 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-11-06T20:43:01.139 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-11-06T20:43:01.139 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-11-06T20:43:01.139 INFO:teuthology.orchestra.run.smithi069.stdout: iproute-6.2.0-5.el8.x86_64 2023-11-06T20:43:01.139 INFO:teuthology.orchestra.run.smithi069.stdout: libacl-2.2.53-3.el8.x86_64 2023-11-06T20:43:01.139 INFO:teuthology.orchestra.run.smithi069.stdout: libacl-devel-2.2.53-3.el8.x86_64 2023-11-06T20:43:01.140 INFO:teuthology.orchestra.run.smithi069.stdout: libblkid-2.32.1-43.el8.x86_64 2023-11-06T20:43:01.140 INFO:teuthology.orchestra.run.smithi069.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-11-06T20:43:01.140 INFO:teuthology.orchestra.run.smithi069.stdout: libmount-2.32.1-43.el8.x86_64 2023-11-06T20:43:01.140 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.141 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.141 INFO:teuthology.orchestra.run.smithi069.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-11-06T20:43:01.141 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-2.32.1-43.el8.x86_64 2023-11-06T20:43:01.141 INFO:teuthology.orchestra.run.smithi069.stdout: util-linux-2.32.1-43.el8.x86_64 2023-11-06T20:43:01.142 INFO:teuthology.orchestra.run.smithi069.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-11-06T20:43:01.142 INFO:teuthology.orchestra.run.smithi069.stdout: xfsdump-3.1.8-7.el8.x86_64 2023-11-06T20:43:01.142 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-11-06T20:43:01.142 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2023-11-06T20:43:01.142 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.1-8.el8.noarch 2023-11-06T20:43:01.143 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.0.4-10.el8.x86_64 2023-11-06T20:43:01.143 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.143 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.143 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.143 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.144 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:43:01.144 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.144 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.144 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.144 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:43:01.145 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:43:01.145 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:43:01.145 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:43:01.145 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:43:01.145 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.146 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.146 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:43:01.146 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.146 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.147 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.147 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:43:01.147 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.147 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:16.2.14-132.g4e03306b.el8.noarch 2023-11-06T20:43:01.147 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-11-06T20:43:01.148 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.1-9.el8.x86_64 2023-11-06T20:43:01.148 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.7-17.el8.x86_64 2023-11-06T20:43:01.148 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-06T20:43:01.148 INFO:teuthology.orchestra.run.smithi069.stdout: jq-1.6-7.el8.x86_64 2023-11-06T20:43:01.148 INFO:teuthology.orchestra.run.smithi069.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-06T20:43:01.149 INFO:teuthology.orchestra.run.smithi069.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-11-06T20:43:01.149 INFO:teuthology.orchestra.run.smithi069.stdout: leveldb-1.22-1.el8.x86_64 2023-11-06T20:43:01.149 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-06T20:43:01.149 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.150 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.150 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.150 INFO:teuthology.orchestra.run.smithi069.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-11-06T20:43:01.150 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-8.5.0-21.el8.x86_64 2023-11-06T20:43:01.150 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-06T20:43:01.151 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-8.5.0-21.el8.x86_64 2023-11-06T20:43:01.151 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-11-06T20:43:01.151 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.151 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.151 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-06T20:43:01.152 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.152 INFO:teuthology.orchestra.run.smithi069.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-11-06T20:43:01.152 INFO:teuthology.orchestra.run.smithi069.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-06T20:43:01.152 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-06T20:43:01.152 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-06T20:43:01.152 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-11-06T20:43:01.153 INFO:teuthology.orchestra.run.smithi069.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-11-06T20:43:01.153 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-11-06T20:43:01.153 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-06T20:43:01.153 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-06T20:43:01.153 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.15-4.el8.x86_64 2023-11-06T20:43:01.154 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-11-06T20:43:01.154 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-11-06T20:43:01.154 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-11-06T20:43:01.154 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-11-06T20:43:01.154 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-11-06T20:43:01.155 INFO:teuthology.orchestra.run.smithi069.stdout: python3-apipkg-1.5-6.el8.noarch 2023-11-06T20:43:01.155 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-06T20:43:01.155 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-06T20:43:01.155 INFO:teuthology.orchestra.run.smithi069.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-06T20:43:01.155 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-06T20:43:01.156 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.156 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.156 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.156 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-06T20:43:01.156 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-11-06T20:43:01.157 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-06T20:43:01.157 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-06T20:43:01.157 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-11-06T20:43:01.157 INFO:teuthology.orchestra.run.smithi069.stdout: python3-execnet-1.7.1-1.el8.noarch 2023-11-06T20:43:01.157 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-06T20:43:01.158 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-06T20:43:01.158 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-06T20:43:01.158 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-06T20:43:01.158 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-06T20:43:01.159 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-06T20:43:01.159 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-06T20:43:01.159 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-06T20:43:01.159 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-06T20:43:01.160 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.0.6-14.el8.noarch 2023-11-06T20:43:01.160 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-06T20:43:01.160 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-06T20:43:01.160 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-06T20:43:01.160 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-06T20:43:01.161 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-06T20:43:01.161 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-06T20:43:01.161 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-2.6-1.el8.noarch 2023-11-06T20:43:01.161 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-06T20:43:01.162 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-06T20:43:01.162 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-06T20:43:01.162 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-06T20:43:01.162 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-06T20:43:01.162 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2017.2-11.el8.noarch 2023-11-06T20:43:01.163 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.163 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.163 INFO:teuthology.orchestra.run.smithi069.stdout: python3-remoto-1.2.1-1.el8.noarch 2023-11-06T20:43:01.163 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-06T20:43:01.163 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-06T20:43:01.164 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.164 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-06T20:43:01.164 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-06T20:43:01.164 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-11-06T20:43:01.164 INFO:teuthology.orchestra.run.smithi069.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-06T20:43:01.165 INFO:teuthology.orchestra.run.smithi069.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-06T20:43:01.165 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-06T20:43:01.165 INFO:teuthology.orchestra.run.smithi069.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-06T20:43:01.165 INFO:teuthology.orchestra.run.smithi069.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-06T20:43:01.165 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-06T20:43:01.166 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-06T20:43:01.166 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-06T20:43:01.166 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-06T20:43:01.166 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-06T20:43:01.166 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.167 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.167 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:16.2.14-132.g4e03306b.el8.x86_64 2023-11-06T20:43:01.167 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-06T20:43:01.167 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-11-06T20:43:01.167 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-06T20:43:01.168 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-11-06T20:43:01.168 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:01.168 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2023-11-06T20:43:01.449 DEBUG:teuthology.parallel:result is None 2023-11-06T20:43:01.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4e03306bd00b0abb2be37dbc132f82d281f8973d 2023-11-06T20:43:01.595 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-06T20:43:01.652 INFO:teuthology.orchestra.run.smithi033.stdout:16.2.14-132.g4e03306b.el8 2023-11-06T20:43:01.652 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-132.g4e03306b.el8 2023-11-06T20:43:01.653 INFO:teuthology.task.install:The correct ceph version 16.2.14-132.g4e03306b is installed. 2023-11-06T20:43:01.653 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4e03306bd00b0abb2be37dbc132f82d281f8973d 2023-11-06T20:43:01.885 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-06T20:43:01.942 INFO:teuthology.orchestra.run.smithi069.stdout:16.2.14-132.g4e03306b.el8 2023-11-06T20:43:01.943 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-132.g4e03306b.el8 2023-11-06T20:43:01.943 INFO:teuthology.task.install:The correct ceph version 16.2.14-132.g4e03306b is installed. 2023-11-06T20:43:01.943 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-06T20:43:01.944 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:01.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-06T20:43:02.020 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:02.020 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-06T20:43:02.094 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-06T20:43:02.095 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:02.095 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2023-11-06T20:43:02.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-06T20:43:02.279 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:02.280 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2023-11-06T20:43:02.353 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-06T20:43:02.465 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-06T20:43:02.465 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:02.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-06T20:43:02.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-06T20:43:02.647 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:02.647 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-06T20:43:02.720 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-06T20:43:02.831 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-06T20:43:02.832 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:02.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2023-11-06T20:43:02.904 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-06T20:43:03.015 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:03.015 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2023-11-06T20:43:03.085 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-06T20:43:03.196 INFO:teuthology.run_tasks:Running task ceph... 2023-11-06T20:43:03.253 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-06T20:43:03.254 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /var/log/ceph 2023-11-06T20:43:03.259 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /var/log/ceph 2023-11-06T20:43:03.331 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-06T20:43:03.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-06T20:43:03.336 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-06T20:43:03.407 INFO:tasks.ceph:Creating extra log directories... 2023-11-06T20:43:03.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-06T20:43:03.411 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-06T20:43:03.489 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-06T20:43:03.490 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-06T20:43:03.490 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-06T20:43:03.490 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:03.491 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-06T20:43:03.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-06T20:43:03.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-06T20:43:03.815 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:03.815 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-06T20:43:03.864 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-06T20:43:04.075 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-06T20:43:04.153 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-06T20:43:04.153 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-06T20:43:04.153 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362', 'branch': 'wip-vshankar-testing1-2023-11-02-1726-pacific', 'description': 'fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7449362', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing1-2023-11-02-1726-pacific'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '4e03306bd00b0abb2be37dbc132f82d281f8973d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4e03306bd00b0abb2be37dbc132f82d281f8973d'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing1-2023-11-02-1726-pacific', 'sha1': '4e03306bd00b0abb2be37dbc132f82d281f8973d'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6875, 'sha1': '4e03306bd00b0abb2be37dbc132f82d281f8973d', 'sleep_before_teardown': 0, 'subset': '378/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing1-2023-11-02-1726-pacific', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_4e03306bd00b0abb2be37dbc132f82d281f8973d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '4e03306bd00b0abb2be37dbc132f82d281f8973d', 'targets': {'smithi033.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBhYb7C6mh7TStE9qnElUtlktEmsqwCqHWi6AuvtIjAc5I+LEPtMn5m6Tc5YouBmPFlV8RWjsDjd/cj8RwnXvFw=', 'smithi069.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUKXzZAO/PeanAIVEqBgMAF/dV5YdwYatA66k8kMbvkXopSNomPc8YE8Wu/L9oujrpYY+2GBQZ6hTHfS4+cXPQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '4e03306bd00b0abb2be37dbc132f82d281f8973d'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-06_07:50:57', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3084822'} 2023-11-06T20:43:04.154 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-06T20:43:04.159 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-06T20:43:04.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/run/ceph 2023-11-06T20:43:04.212 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m0777 -- /var/run/ceph 2023-11-06T20:43:04.284 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:04.285 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2023-11-06T20:43:04.333 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-06T20:43:04.333 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2023-11-06T20:43:04.422 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-06T20:43:04.423 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T20:43:04.423 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-11-06T20:43:04.423 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T20:43:04.423 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T20:43:04.423 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-11-06 20:42:52.501480557 +0000 2023-11-06T20:43:04.424 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-11-06 20:38:58.419169135 +0000 2023-11-06T20:43:04.424 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-11-06 20:38:58.419169135 +0000 2023-11-06T20:43:04.424 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2023-11-06 20:38:58.390169611 +0000 2023-11-06T20:43:04.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-06T20:43:04.493 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-11-06T20:43:04.493 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-11-06T20:43:04.493 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000265781 s, 1.9 MB/s 2023-11-06T20:43:04.495 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-06T20:43:04.544 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2023-11-06T20:43:04.591 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-06T20:43:04.591 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T20:43:04.592 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-11-06T20:43:04.592 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T20:43:04.592 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T20:43:04.592 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-11-06 20:42:52.501480557 +0000 2023-11-06T20:43:04.593 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-11-06 20:38:59.277155047 +0000 2023-11-06T20:43:04.593 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-11-06 20:38:59.277155047 +0000 2023-11-06T20:43:04.593 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2023-11-06 20:38:59.248155523 +0000 2023-11-06T20:43:04.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-06T20:43:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-11-06T20:43:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-11-06T20:43:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000284922 s, 1.8 MB/s 2023-11-06T20:43:04.664 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-06T20:43:04.712 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2023-11-06T20:43:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-06T20:43:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T20:43:04.759 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-11-06T20:43:04.760 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T20:43:04.760 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T20:43:04.760 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-11-06 20:42:52.501480557 +0000 2023-11-06T20:43:04.761 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-11-06 20:39:00.131141025 +0000 2023-11-06T20:43:04.761 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-11-06 20:39:00.131141025 +0000 2023-11-06T20:43:04.761 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2023-11-06 20:39:00.105141452 +0000 2023-11-06T20:43:04.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-06T20:43:04.829 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-11-06T20:43:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-11-06T20:43:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000297764 s, 1.7 MB/s 2023-11-06T20:43:04.832 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-06T20:43:04.881 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2023-11-06T20:43:04.928 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-06T20:43:04.928 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T20:43:04.928 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-11-06T20:43:04.929 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T20:43:04.929 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T20:43:04.929 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-11-06 20:42:52.501480557 +0000 2023-11-06T20:43:04.929 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-11-06 20:39:00.992126889 +0000 2023-11-06T20:43:04.929 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-11-06 20:39:00.992126889 +0000 2023-11-06T20:43:04.930 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2023-11-06 20:39:00.965127332 +0000 2023-11-06T20:43:04.930 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-06T20:43:05.037 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-11-06T20:43:05.037 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-11-06T20:43:05.038 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000477222 s, 1.1 MB/s 2023-11-06T20:43:05.039 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-06T20:43:05.088 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T20:43:05.089 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:05.089 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2023-11-06T20:43:05.142 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-06T20:43:05.142 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2023-11-06T20:43:05.194 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-06T20:43:05.194 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T20:43:05.195 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-11-06T20:43:05.195 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T20:43:05.195 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T20:43:05.195 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-11-06 20:42:58.458048932 +0000 2023-11-06T20:43:05.195 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-11-06 20:38:43.771081229 +0000 2023-11-06T20:43:05.196 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-11-06 20:38:43.771081229 +0000 2023-11-06T20:43:05.197 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2023-11-06 20:38:43.745081847 +0000 2023-11-06T20:43:05.198 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-06T20:43:05.268 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-11-06T20:43:05.269 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-11-06T20:43:05.269 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000287454 s, 1.8 MB/s 2023-11-06T20:43:05.271 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-06T20:43:05.321 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2023-11-06T20:43:05.367 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-06T20:43:05.367 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T20:43:05.368 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-11-06T20:43:05.368 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T20:43:05.368 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T20:43:05.368 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-11-06 20:42:58.458048932 +0000 2023-11-06T20:43:05.369 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-11-06 20:38:44.606061383 +0000 2023-11-06T20:43:05.369 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-11-06 20:38:44.606061383 +0000 2023-11-06T20:43:05.369 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2023-11-06 20:38:44.576062096 +0000 2023-11-06T20:43:05.369 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-06T20:43:05.475 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-11-06T20:43:05.476 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-11-06T20:43:05.476 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000367599 s, 1.4 MB/s 2023-11-06T20:43:05.478 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-06T20:43:05.528 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2023-11-06T20:43:05.576 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-06T20:43:05.576 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T20:43:05.576 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-11-06T20:43:05.577 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T20:43:05.577 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T20:43:05.577 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-11-06 20:42:58.459048908 +0000 2023-11-06T20:43:05.577 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-11-06 20:38:45.437041634 +0000 2023-11-06T20:43:05.577 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-11-06 20:38:45.437041634 +0000 2023-11-06T20:43:05.578 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2023-11-06 20:38:45.411042252 +0000 2023-11-06T20:43:05.578 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-06T20:43:05.684 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-11-06T20:43:05.685 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-11-06T20:43:05.685 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000288599 s, 1.8 MB/s 2023-11-06T20:43:05.687 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-06T20:43:05.736 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2023-11-06T20:43:05.784 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-06T20:43:05.784 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T20:43:05.784 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-11-06T20:43:05.785 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T20:43:05.785 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-11-06T20:43:05.785 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-11-06 20:42:58.459048908 +0000 2023-11-06T20:43:05.785 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-11-06 20:38:46.288021408 +0000 2023-11-06T20:43:05.786 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-11-06 20:38:46.288021408 +0000 2023-11-06T20:43:05.786 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2023-11-06 20:38:46.262022026 +0000 2023-11-06T20:43:05.786 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-06T20:43:05.853 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-11-06T20:43:05.854 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-11-06T20:43:05.854 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000288084 s, 1.8 MB/s 2023-11-06T20:43:05.856 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-06T20:43:05.907 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T20:43:05.907 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi069.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-06T20:43:05.907 INFO:tasks.ceph:Generating config... 2023-11-06T20:43:05.908 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-06T20:43:05.909 INFO:tasks.ceph:[client] debug client = 20 2023-11-06T20:43:05.909 INFO:tasks.ceph:[client] debug ms = 1 2023-11-06T20:43:05.909 INFO:tasks.ceph:[client] pid file = 2023-11-06T20:43:05.910 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-06T20:43:05.910 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-06T20:43:05.910 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-06T20:43:05.910 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-06T20:43:05.911 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-06T20:43:05.911 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-06T20:43:05.912 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-06T20:43:05.912 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-06T20:43:05.912 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-06T20:43:05.913 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-06T20:43:05.913 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-06T20:43:05.913 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-06T20:43:05.914 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-06T20:43:05.914 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-06T20:43:05.914 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-06T20:43:05.914 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-06T20:43:05.915 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-06T20:43:05.915 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-06T20:43:05.915 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-06T20:43:05.915 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-06T20:43:05.916 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-06T20:43:05.916 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-06T20:43:05.916 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-06T20:43:05.916 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-06T20:43:05.917 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-06T20:43:05.917 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-06T20:43:05.917 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-06T20:43:05.917 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-06T20:43:05.917 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-06T20:43:05.918 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-06T20:43:05.918 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-06T20:43:05.918 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-06T20:43:05.918 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-06T20:43:05.919 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-06T20:43:05.919 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-06T20:43:05.919 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-06T20:43:05.920 INFO:tasks.ceph:Setting up mon.a... 2023-11-06T20:43:05.920 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-06T20:43:06.103 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.keyring 2023-11-06T20:43:06.109 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-06T20:43:06.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-06T20:43:06.280 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.33'), ('mon.b', '172.21.15.69'), ('mon.c', '[v2:172.21.15.69:3301,v1:172.21.15.69:6790]')] 2023-11-06T20:43:06.281 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.33,172.21.15.69,[v2:172.21.15.69:3301,v1:172.21.15.69:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-06T20:43:06.282 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:06.283 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-06T20:43:06.329 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.33 --add b 172.21.15.69 --addv c '[v2:172.21.15.69:3301,v1:172.21.15.69:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:06.453 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:06.454 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: generated fsid 097ce162-8fe3-4461-8445-f7551959ee06 2023-11-06T20:43:06.454 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 0 2023-11-06T20:43:06.454 INFO:teuthology.orchestra.run.smithi033.stdout:fsid 097ce162-8fe3-4461-8445-f7551959ee06 2023-11-06T20:43:06.454 INFO:teuthology.orchestra.run.smithi033.stdout:last_changed 2023-11-06T20:43:06.454980+0000 2023-11-06T20:43:06.454 INFO:teuthology.orchestra.run.smithi033.stdout:created 2023-11-06T20:43:06.454980+0000 2023-11-06T20:43:06.455 INFO:teuthology.orchestra.run.smithi033.stdout:min_mon_release 0 (unknown) 2023-11-06T20:43:06.455 INFO:teuthology.orchestra.run.smithi033.stdout:election_strategy: 1 2023-11-06T20:43:06.455 INFO:teuthology.orchestra.run.smithi033.stdout:0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2023-11-06T20:43:06.455 INFO:teuthology.orchestra.run.smithi033.stdout:1: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.b 2023-11-06T20:43:06.456 INFO:teuthology.orchestra.run.smithi033.stdout:2: [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] mon.c 2023-11-06T20:43:06.456 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-06T20:43:06.457 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-06T20:43:06.503 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 097ce162-8fe3-4461-8445-f7551959ee06... 2023-11-06T20:43:06.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T20:43:06.509 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T20:43:06.626 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2023-11-06T20:43:06.626 INFO:teuthology.orchestra.run.smithi033.stdout: chdir = "" 2023-11-06T20:43:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T20:43:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: auth supported = cephx 2023-11-06T20:43:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: filestore xattr use omap = true 2023-11-06T20:43:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: mon clock drift allowed = 1.000 2023-11-06T20:43:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: osd crush chooseleaf type = 0 2023-11-06T20:43:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: auth debug = true 2023-11-06T20:43:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on old message = true 2023-11-06T20:43:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on bug = true 2023-11-06T20:43:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T20:43:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: mon pg warn max object skew = 0 2023-11-06T20:43:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T20:43:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T20:43:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default size = 2 2023-11-06T20:43:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow primary affinity = true 2023-11-06T20:43:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow pg remap = true 2023-11-06T20:43:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on legacy crush tunables = false 2023-11-06T20:43:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on crush straw calc version zero = false 2023-11-06T20:43:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on no sortbitwise = false 2023-11-06T20:43:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on osd down out interval zero = false 2023-11-06T20:43:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on too few osds = false 2023-11-06T20:43:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T20:43:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T20:43:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: mon_allow_pool_size_one = true 2023-11-06T20:43:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-11-06T20:43:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd default data pool replay window = 5 2023-11-06T20:43:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: mon allow pool delete = true 2023-11-06T20:43:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: mon cluster log file level = debug 2023-11-06T20:43:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: debug asserts on shutdown = true 2023-11-06T20:43:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: mon health detail to clog = false 2023-11-06T20:43:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: mon host = "172.21.15.33,172.21.15.69,[v2:172.21.15.69:3301,v1:172.21.15.69:6790]" 2023-11-06T20:43:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on skipped message = False 2023-11-06T20:43:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 097ce162-8fe3-4461-8445-f7551959ee06 2023-11-06T20:43:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.636 INFO:teuthology.orchestra.run.smithi033.stdout:[osd] 2023-11-06T20:43:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: osd journal size = 100 2023-11-06T20:43:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub load threshold = 5.0 2023-11-06T20:43:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub max interval = 600 2023-11-06T20:43:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: osd recover clone overlap = true 2023-11-06T20:43:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: osd recovery max chunk = 1048576 2023-11-06T20:43:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug shutdown = true 2023-11-06T20:43:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug op order = true 2023-11-06T20:43:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify stray on activate = true 2023-11-06T20:43:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: osd open classes on start = true 2023-11-06T20:43:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug pg log writeout = true 2023-11-06T20:43:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: osd deep scrub update digest min age = 30 2023-11-06T20:43:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: osd map max advance = 10 2023-11-06T20:43:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: journal zero on create = true 2023-11-06T20:43:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: filestore ondisk finisher threads = 3 2023-11-06T20:43:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: filestore apply finisher threads = 3 2023-11-06T20:43:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: bdev debug aio = true 2023-11-06T20:43:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug misdirected ops = true 2023-11-06T20:43:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: bdev async discard = True 2023-11-06T20:43:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: bdev enable discard = True 2023-11-06T20:43:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore allocator = bitmap 2023-11-06T20:43:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore block size = 96636764160 2023-11-06T20:43:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore fsck on mount = True 2023-11-06T20:43:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluefs = 1/20 2023-11-06T20:43:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluestore = 1/20 2023-11-06T20:43:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-11-06T20:43:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: debug osd = 20 2023-11-06T20:43:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: debug rocksdb = 4/10 2023-11-06T20:43:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T20:43:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd full ratio = 0.9 2023-11-06T20:43:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd nearfull ratio = 0.8 2023-11-06T20:43:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: osd failsafe full ratio = 0.95 2023-11-06T20:43:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: osd objectstore = bluestore 2023-11-06T20:43:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2023-11-06T20:43:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.644 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr] 2023-11-06T20:43:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-11-06T20:43:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: debug mgr = 20 2023-11-06T20:43:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2023-11-06T20:43:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2023-11-06T20:43:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2023-11-06T20:43:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2023-11-06T20:43:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: mgr/telemetry/nag = false 2023-11-06T20:43:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: debug client = 20 2023-11-06T20:43:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: debug finisher = 20 2023-11-06T20:43:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.646 INFO:teuthology.orchestra.run.smithi033.stdout:[mon] 2023-11-06T20:43:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-11-06T20:43:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2023-11-06T20:43:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: debug paxos = 20 2023-11-06T20:43:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2023-11-06T20:43:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: mon data avail warn = 5 2023-11-06T20:43:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: mon mgr mkfs grace = 240 2023-11-06T20:43:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2023-11-06T20:43:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd reporter subtree level = osd 2023-11-06T20:43:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd prime pg temp = true 2023-11-06T20:43:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2023-11-06T20:43:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T20:43:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T20:43:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T20:43:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T20:43:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T20:43:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T20:43:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: mon op complaint time = 120 2023-11-06T20:43:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:06.651 INFO:teuthology.orchestra.run.smithi033.stdout:[client] 2023-11-06T20:43:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: rgw cache enabled = true 2023-11-06T20:43:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable ops log = true 2023-11-06T20:43:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable usage log = true 2023-11-06T20:43:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T20:43:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T20:43:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: client mount timeout = 600 2023-11-06T20:43:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: debug client = 20 2023-11-06T20:43:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-11-06T20:43:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = "" 2023-11-06T20:43:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2023-11-06T20:43:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2023-11-06T20:43:06.653 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.a] 2023-11-06T20:43:06.653 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.b] 2023-11-06T20:43:06.654 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.c] 2023-11-06T20:43:06.654 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2023-11-06T20:43:06.654 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2023-11-06T20:43:06.654 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2023-11-06T20:43:06.654 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2023-11-06T20:43:06.655 INFO:teuthology.orchestra.run.smithi033.stdout:[mds] 2023-11-06T20:43:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds = 20 2023-11-06T20:43:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-11-06T20:43:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug frag = True 2023-11-06T20:43:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug scatterstat = True 2023-11-06T20:43:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: mds op complaint time = 180 2023-11-06T20:43:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: mds verify scatter = True 2023-11-06T20:43:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2023-11-06T20:43:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2023-11-06T20:43:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2023-11-06T20:43:06.657 INFO:teuthology.orchestra.run.smithi069.stdout:[global] 2023-11-06T20:43:06.657 INFO:teuthology.orchestra.run.smithi069.stdout: chdir = "" 2023-11-06T20:43:06.657 INFO:teuthology.orchestra.run.smithi069.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T20:43:06.658 INFO:teuthology.orchestra.run.smithi069.stdout: auth supported = cephx 2023-11-06T20:43:06.658 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.658 INFO:teuthology.orchestra.run.smithi069.stdout: filestore xattr use omap = true 2023-11-06T20:43:06.658 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.658 INFO:teuthology.orchestra.run.smithi069.stdout: mon clock drift allowed = 1.000 2023-11-06T20:43:06.659 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.659 INFO:teuthology.orchestra.run.smithi069.stdout: osd crush chooseleaf type = 0 2023-11-06T20:43:06.659 INFO:teuthology.orchestra.run.smithi069.stdout: auth debug = true 2023-11-06T20:43:06.659 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.659 INFO:teuthology.orchestra.run.smithi069.stdout: ms die on old message = true 2023-11-06T20:43:06.660 INFO:teuthology.orchestra.run.smithi069.stdout: ms die on bug = true 2023-11-06T20:43:06.660 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.660 INFO:teuthology.orchestra.run.smithi069.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T20:43:06.660 INFO:teuthology.orchestra.run.smithi069.stdout: mon pg warn max object skew = 0 2023-11-06T20:43:06.660 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.660 INFO:teuthology.orchestra.run.smithi069.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T20:43:06.661 INFO:teuthology.orchestra.run.smithi069.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T20:43:06.661 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.661 INFO:teuthology.orchestra.run.smithi069.stdout: osd pool default size = 2 2023-11-06T20:43:06.661 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.661 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd allow primary affinity = true 2023-11-06T20:43:06.662 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd allow pg remap = true 2023-11-06T20:43:06.662 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on legacy crush tunables = false 2023-11-06T20:43:06.662 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on crush straw calc version zero = false 2023-11-06T20:43:06.662 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on no sortbitwise = false 2023-11-06T20:43:06.663 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on osd down out interval zero = false 2023-11-06T20:43:06.663 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on too few osds = false 2023-11-06T20:43:06.663 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T20:43:06.663 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T20:43:06.663 INFO:teuthology.orchestra.run.smithi069.stdout: mon_allow_pool_size_one = true 2023-11-06T20:43:06.663 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.664 INFO:teuthology.orchestra.run.smithi069.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-11-06T20:43:06.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.664 INFO:teuthology.orchestra.run.smithi069.stdout: osd default data pool replay window = 5 2023-11-06T20:43:06.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.665 INFO:teuthology.orchestra.run.smithi069.stdout: mon allow pool delete = true 2023-11-06T20:43:06.665 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.665 INFO:teuthology.orchestra.run.smithi069.stdout: mon cluster log file level = debug 2023-11-06T20:43:06.665 INFO:teuthology.orchestra.run.smithi069.stdout: debug asserts on shutdown = true 2023-11-06T20:43:06.665 INFO:teuthology.orchestra.run.smithi069.stdout: mon health detail to clog = false 2023-11-06T20:43:06.666 INFO:teuthology.orchestra.run.smithi069.stdout: mon host = "172.21.15.33,172.21.15.69,[v2:172.21.15.69:3301,v1:172.21.15.69:6790]" 2023-11-06T20:43:06.666 INFO:teuthology.orchestra.run.smithi069.stdout: ms die on skipped message = False 2023-11-06T20:43:06.666 INFO:teuthology.orchestra.run.smithi069.stdout: fsid = 097ce162-8fe3-4461-8445-f7551959ee06 2023-11-06T20:43:06.667 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.667 INFO:teuthology.orchestra.run.smithi069.stdout:[osd] 2023-11-06T20:43:06.667 INFO:teuthology.orchestra.run.smithi069.stdout: osd journal size = 100 2023-11-06T20:43:06.667 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.667 INFO:teuthology.orchestra.run.smithi069.stdout: osd scrub load threshold = 5.0 2023-11-06T20:43:06.668 INFO:teuthology.orchestra.run.smithi069.stdout: osd scrub max interval = 600 2023-11-06T20:43:06.668 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.668 INFO:teuthology.orchestra.run.smithi069.stdout: osd recover clone overlap = true 2023-11-06T20:43:06.668 INFO:teuthology.orchestra.run.smithi069.stdout: osd recovery max chunk = 1048576 2023-11-06T20:43:06.668 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.669 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug shutdown = true 2023-11-06T20:43:06.669 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug op order = true 2023-11-06T20:43:06.669 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug verify stray on activate = true 2023-11-06T20:43:06.669 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.669 INFO:teuthology.orchestra.run.smithi069.stdout: osd open classes on start = true 2023-11-06T20:43:06.670 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug pg log writeout = true 2023-11-06T20:43:06.670 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.670 INFO:teuthology.orchestra.run.smithi069.stdout: osd deep scrub update digest min age = 30 2023-11-06T20:43:06.670 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.670 INFO:teuthology.orchestra.run.smithi069.stdout: osd map max advance = 10 2023-11-06T20:43:06.671 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.671 INFO:teuthology.orchestra.run.smithi069.stdout: journal zero on create = true 2023-11-06T20:43:06.671 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.671 INFO:teuthology.orchestra.run.smithi069.stdout: filestore ondisk finisher threads = 3 2023-11-06T20:43:06.671 INFO:teuthology.orchestra.run.smithi069.stdout: filestore apply finisher threads = 3 2023-11-06T20:43:06.672 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.672 INFO:teuthology.orchestra.run.smithi069.stdout: bdev debug aio = true 2023-11-06T20:43:06.672 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug misdirected ops = true 2023-11-06T20:43:06.672 INFO:teuthology.orchestra.run.smithi069.stdout: bdev async discard = True 2023-11-06T20:43:06.672 INFO:teuthology.orchestra.run.smithi069.stdout: bdev enable discard = True 2023-11-06T20:43:06.673 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore allocator = bitmap 2023-11-06T20:43:06.673 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore block size = 96636764160 2023-11-06T20:43:06.673 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore fsck on mount = True 2023-11-06T20:43:06.673 INFO:teuthology.orchestra.run.smithi069.stdout: debug bluefs = 1/20 2023-11-06T20:43:06.673 INFO:teuthology.orchestra.run.smithi069.stdout: debug bluestore = 1/20 2023-11-06T20:43:06.674 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2023-11-06T20:43:06.674 INFO:teuthology.orchestra.run.smithi069.stdout: debug osd = 20 2023-11-06T20:43:06.674 INFO:teuthology.orchestra.run.smithi069.stdout: debug rocksdb = 4/10 2023-11-06T20:43:06.674 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T20:43:06.674 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd full ratio = 0.9 2023-11-06T20:43:06.675 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd nearfull ratio = 0.8 2023-11-06T20:43:06.675 INFO:teuthology.orchestra.run.smithi069.stdout: osd failsafe full ratio = 0.95 2023-11-06T20:43:06.675 INFO:teuthology.orchestra.run.smithi069.stdout: osd objectstore = bluestore 2023-11-06T20:43:06.675 INFO:teuthology.orchestra.run.smithi069.stdout: osd op complaint time = 180 2023-11-06T20:43:06.676 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.676 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr] 2023-11-06T20:43:06.676 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2023-11-06T20:43:06.676 INFO:teuthology.orchestra.run.smithi069.stdout: debug mgr = 20 2023-11-06T20:43:06.676 INFO:teuthology.orchestra.run.smithi069.stdout: debug mon = 20 2023-11-06T20:43:06.676 INFO:teuthology.orchestra.run.smithi069.stdout: debug auth = 20 2023-11-06T20:43:06.677 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min pgs per osd = 4 2023-11-06T20:43:06.677 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min bytes per osd = 10 2023-11-06T20:43:06.677 INFO:teuthology.orchestra.run.smithi069.stdout: mgr/telemetry/nag = false 2023-11-06T20:43:06.677 INFO:teuthology.orchestra.run.smithi069.stdout: debug client = 20 2023-11-06T20:43:06.677 INFO:teuthology.orchestra.run.smithi069.stdout: debug finisher = 20 2023-11-06T20:43:06.678 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.678 INFO:teuthology.orchestra.run.smithi069.stdout:[mon] 2023-11-06T20:43:06.678 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2023-11-06T20:43:06.678 INFO:teuthology.orchestra.run.smithi069.stdout: debug mon = 20 2023-11-06T20:43:06.678 INFO:teuthology.orchestra.run.smithi069.stdout: debug paxos = 20 2023-11-06T20:43:06.679 INFO:teuthology.orchestra.run.smithi069.stdout: debug auth = 20 2023-11-06T20:43:06.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon data avail warn = 5 2023-11-06T20:43:06.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon mgr mkfs grace = 240 2023-11-06T20:43:06.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min pgs per osd = 4 2023-11-06T20:43:06.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd reporter subtree level = osd 2023-11-06T20:43:06.680 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd prime pg temp = true 2023-11-06T20:43:06.680 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min bytes per osd = 10 2023-11-06T20:43:06.680 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.680 INFO:teuthology.orchestra.run.smithi069.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T20:43:06.680 INFO:teuthology.orchestra.run.smithi069.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T20:43:06.681 INFO:teuthology.orchestra.run.smithi069.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T20:43:06.681 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.681 INFO:teuthology.orchestra.run.smithi069.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T20:43:06.681 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T20:43:06.681 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T20:43:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: mon op complaint time = 120 2023-11-06T20:43:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-11-06T20:43:06.682 INFO:teuthology.orchestra.run.smithi069.stdout:[client] 2023-11-06T20:43:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: rgw cache enabled = true 2023-11-06T20:43:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: rgw enable ops log = true 2023-11-06T20:43:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: rgw enable usage log = true 2023-11-06T20:43:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T20:43:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T20:43:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: client mount timeout = 600 2023-11-06T20:43:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: debug client = 20 2023-11-06T20:43:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2023-11-06T20:43:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: pid file = "" 2023-11-06T20:43:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: rados mon op timeout = 900 2023-11-06T20:43:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: rados osd op timeout = 900 2023-11-06T20:43:06.684 INFO:teuthology.orchestra.run.smithi069.stdout:[mon.a] 2023-11-06T20:43:06.685 INFO:teuthology.orchestra.run.smithi069.stdout:[mon.b] 2023-11-06T20:43:06.685 INFO:teuthology.orchestra.run.smithi069.stdout:[mon.c] 2023-11-06T20:43:06.685 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.a] 2023-11-06T20:43:06.685 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.c] 2023-11-06T20:43:06.685 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.b] 2023-11-06T20:43:06.685 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.d] 2023-11-06T20:43:06.686 INFO:teuthology.orchestra.run.smithi069.stdout:[mds] 2023-11-06T20:43:06.686 INFO:teuthology.orchestra.run.smithi069.stdout: debug mds = 20 2023-11-06T20:43:06.686 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2023-11-06T20:43:06.686 INFO:teuthology.orchestra.run.smithi069.stdout: mds debug frag = True 2023-11-06T20:43:06.686 INFO:teuthology.orchestra.run.smithi069.stdout: mds debug scatterstat = True 2023-11-06T20:43:06.687 INFO:teuthology.orchestra.run.smithi069.stdout: mds op complaint time = 180 2023-11-06T20:43:06.687 INFO:teuthology.orchestra.run.smithi069.stdout: mds verify scatter = True 2023-11-06T20:43:06.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd op complaint time = 180 2023-11-06T20:43:06.687 INFO:teuthology.orchestra.run.smithi069.stdout: rados mon op timeout = 900 2023-11-06T20:43:06.687 INFO:teuthology.orchestra.run.smithi069.stdout: rados osd op timeout = 900 2023-11-06T20:43:06.689 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-06T20:43:06.689 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-06T20:43:06.797 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-06T20:43:06.798 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:06.798 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-06T20:43:06.845 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:06.845 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-06T20:43:06.930 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:43:06.931 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:06.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-06T20:43:06.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-06T20:43:07.068 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:07.068 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:07.153 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T20:43:07.154 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:07.154 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-06T20:43:07.155 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-06T20:43:07.253 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:07.253 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:07.340 INFO:tasks.ceph:Setting up mon nodes... 2023-11-06T20:43:07.340 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-06T20:43:07.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-06T20:43:07.461 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-06T20:43:07.467 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-06T20:43:07.663 INFO:teuthology.orchestra.run.smithi069.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-06T20:43:07.669 INFO:tasks.ceph:Setting up mds nodes... 2023-11-06T20:43:07.669 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-06T20:43:07.789 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-06T20:43:07.793 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-06T20:43:07.864 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-06T20:43:07.983 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-06T20:43:07.989 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-06T20:43:08.059 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-06T20:43:08.175 INFO:teuthology.orchestra.run.smithi069.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-06T20:43:08.180 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-06T20:43:08.249 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-06T20:43:08.365 INFO:teuthology.orchestra.run.smithi069.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-06T20:43:08.369 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-06T20:43:08.438 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-06T20:43:08.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-06T20:43:08.535 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-06T20:43:08.565 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-06T20:43:08.658 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-06T20:43:08.689 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-06T20:43:08.780 INFO:teuthology.orchestra.run.smithi069.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-06T20:43:08.809 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-06T20:43:08.900 INFO:teuthology.orchestra.run.smithi069.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-06T20:43:08.929 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-06T20:43:08.930 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi069.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-06T20:43:08.930 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-06T20:43:09.002 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T20:43:09.002 INFO:tasks.ceph:role: osd.0 2023-11-06T20:43:09.003 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:43:09.003 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-06T20:43:09.081 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T20:43:09.082 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T20:43:09.082 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T20:43:09.082 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T20:43:09.083 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T20:43:09.083 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-11-06T20:43:09.083 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T20:43:09.083 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T20:43:09.083 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T20:43:09.084 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T20:43:09.510 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2023-11-06T20:43:09.514 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-11-06T20:43:09.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-06T20:43:09.723 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-06T20:43:09.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-06T20:43:09.877 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T20:43:09.878 INFO:tasks.ceph:role: osd.1 2023-11-06T20:43:09.878 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:43:09.879 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-06T20:43:09.957 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T20:43:09.958 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T20:43:09.958 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T20:43:09.958 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T20:43:09.958 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T20:43:09.959 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-11-06T20:43:09.959 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T20:43:09.959 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T20:43:09.959 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T20:43:09.959 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T20:43:10.357 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2023-11-06T20:43:10.362 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-11-06T20:43:10.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-06T20:43:10.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-06T20:43:10.590 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-06T20:43:10.661 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T20:43:10.662 INFO:tasks.ceph:role: osd.2 2023-11-06T20:43:10.662 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:43:10.662 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-06T20:43:10.741 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T20:43:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T20:43:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T20:43:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T20:43:10.742 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T20:43:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-11-06T20:43:10.742 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T20:43:10.742 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T20:43:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T20:43:10.743 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T20:43:11.190 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2023-11-06T20:43:11.194 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-11-06T20:43:11.194 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-06T20:43:11.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-06T20:43:11.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-06T20:43:11.560 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T20:43:11.560 INFO:tasks.ceph:role: osd.3 2023-11-06T20:43:11.560 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:43:11.561 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-06T20:43:11.639 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T20:43:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T20:43:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T20:43:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T20:43:11.640 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T20:43:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-11-06T20:43:11.640 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T20:43:11.640 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T20:43:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T20:43:11.641 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T20:43:12.067 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2023-11-06T20:43:12.072 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-11-06T20:43:12.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-06T20:43:12.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-06T20:43:12.268 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:12.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:12.389+0000 7f362fbf7380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-06T20:43:12.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:12.389+0000 7f362fbf7380 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-06T20:43:12.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:12.389+0000 7f362fbf7380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-06T20:43:12.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:12.389+0000 7f362fbf7380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-06T20:43:12.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:12.389+0000 7f362fbf7380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-06T20:43:12.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:12.392+0000 7f362fbf7380 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-06T20:43:15.523 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-06T20:43:15.600 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:15.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:15.704+0000 7f40d2d3b380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-06T20:43:15.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:15.704+0000 7f40d2d3b380 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-06T20:43:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:15.705+0000 7f40d2d3b380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-06T20:43:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:15.705+0000 7f40d2d3b380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-06T20:43:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:15.705+0000 7f40d2d3b380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-06T20:43:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:15.706+0000 7f40d2d3b380 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-06T20:43:18.830 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-06T20:43:18.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:19.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:19.013+0000 7f19b4085380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-06T20:43:19.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:19.013+0000 7f19b4085380 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-06T20:43:19.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:19.013+0000 7f19b4085380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-06T20:43:19.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:19.013+0000 7f19b4085380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-06T20:43:19.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:19.013+0000 7f19b4085380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-06T20:43:19.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:19.015+0000 7f19b4085380 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-06T20:43:22.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-06T20:43:22.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:22.327+0000 7fc28fba1380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-06T20:43:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:22.328+0000 7fc28fba1380 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-06T20:43:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:22.328+0000 7fc28fba1380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-06T20:43:22.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:22.328+0000 7fc28fba1380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-06T20:43:22.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:22.328+0000 7fc28fba1380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-06T20:43:22.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:22.329+0000 7fc28fba1380 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-06T20:43:25.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-06T20:43:25.534 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-06T20:43:25.612 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T20:43:25.612 INFO:tasks.ceph:role: osd.4 2023-11-06T20:43:25.612 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T20:43:25.613 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-06T20:43:25.692 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T20:43:25.692 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T20:43:25.693 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T20:43:25.693 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T20:43:25.693 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T20:43:25.693 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2023-11-06T20:43:25.693 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T20:43:25.694 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T20:43:25.694 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T20:43:25.694 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T20:43:26.131 INFO:teuthology.orchestra.run.smithi069.stdout:Discarding blocks...Done. 2023-11-06T20:43:26.135 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2023-11-06T20:43:26.135 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-06T20:43:26.319 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-06T20:43:26.399 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-06T20:43:26.470 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T20:43:26.471 INFO:tasks.ceph:role: osd.5 2023-11-06T20:43:26.471 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T20:43:26.471 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-06T20:43:26.548 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T20:43:26.548 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T20:43:26.548 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T20:43:26.549 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T20:43:26.549 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T20:43:26.549 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2023-11-06T20:43:26.549 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T20:43:26.549 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T20:43:26.550 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T20:43:26.550 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T20:43:26.950 INFO:teuthology.orchestra.run.smithi069.stdout:Discarding blocks...Done. 2023-11-06T20:43:26.954 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2023-11-06T20:43:26.954 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-06T20:43:27.097 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-06T20:43:27.174 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-06T20:43:27.244 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T20:43:27.244 INFO:tasks.ceph:role: osd.6 2023-11-06T20:43:27.244 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T20:43:27.245 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-06T20:43:27.321 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T20:43:27.321 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T20:43:27.321 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T20:43:27.321 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T20:43:27.322 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T20:43:27.322 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2023-11-06T20:43:27.322 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T20:43:27.322 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T20:43:27.323 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T20:43:27.323 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T20:43:27.779 INFO:teuthology.orchestra.run.smithi069.stdout:Discarding blocks...Done. 2023-11-06T20:43:27.783 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2023-11-06T20:43:27.783 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-06T20:43:27.955 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-06T20:43:28.034 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-06T20:43:28.104 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T20:43:28.105 INFO:tasks.ceph:role: osd.7 2023-11-06T20:43:28.105 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T20:43:28.105 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-06T20:43:28.182 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T20:43:28.182 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T20:43:28.182 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T20:43:28.183 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T20:43:28.183 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T20:43:28.183 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2023-11-06T20:43:28.183 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T20:43:28.183 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T20:43:28.184 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T20:43:28.184 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T20:43:28.612 INFO:teuthology.orchestra.run.smithi069.stdout:Discarding blocks...Done. 2023-11-06T20:43:28.616 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2023-11-06T20:43:28.616 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-06T20:43:28.752 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-06T20:43:28.827 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:28.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:28.947+0000 7fb5dcc09380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-06T20:43:28.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:28.947+0000 7fb5dcc09380 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-06T20:43:28.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:28.947+0000 7fb5dcc09380 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-06T20:43:28.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:28.947+0000 7fb5dcc09380 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-06T20:43:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:28.947+0000 7fb5dcc09380 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-06T20:43:28.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:28.949+0000 7fb5dcc09380 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-06T20:43:32.070 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-06T20:43:32.142 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:32.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:32.258+0000 7f6352987380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-06T20:43:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:32.259+0000 7f6352987380 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-06T20:43:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:32.259+0000 7f6352987380 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-06T20:43:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:32.259+0000 7f6352987380 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-06T20:43:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:32.259+0000 7f6352987380 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-06T20:43:32.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:32.260+0000 7f6352987380 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-06T20:43:34.155 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:43:34.161 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:43:35.389 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-06T20:43:35.460 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:35.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:35.564+0000 7f73e7c67380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-06T20:43:35.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:35.564+0000 7f73e7c67380 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-06T20:43:35.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:35.564+0000 7f73e7c67380 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-06T20:43:35.565 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:35.564+0000 7f73e7c67380 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-06T20:43:35.565 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:35.564+0000 7f73e7c67380 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-06T20:43:35.565 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:35.566+0000 7f73e7c67380 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-06T20:43:38.686 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-06T20:43:38.756 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:38.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:38.856+0000 7fdd9f24b380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-06T20:43:38.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:38.857+0000 7fdd9f24b380 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-06T20:43:38.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:38.857+0000 7fdd9f24b380 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-06T20:43:38.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:38.857+0000 7fdd9f24b380 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-06T20:43:38.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:38.857+0000 7fdd9f24b380 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-06T20:43:38.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:38.858+0000 7fdd9f24b380 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-06T20:43:41.989 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-06T20:43:42.061 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-06T20:43:42.061 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:42.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-06T20:43:42.140 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:42.141 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-06T20:43:42.215 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:42.215 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-06T20:43:42.286 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:42.286 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-06T20:43:42.396 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:42.397 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-06T20:43:42.507 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:42.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-06T20:43:42.618 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:42.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-06T20:43:42.728 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:42.729 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-06T20:43:42.800 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:42.801 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-06T20:43:42.870 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:42.871 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-06T20:43:42.980 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:42.980 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-06T20:43:43.088 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:43.089 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-06T20:43:43.198 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:43.198 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-06T20:43:43.307 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:43.307 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-06T20:43:43.417 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:43.417 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-06T20:43:43.468 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:43.469 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-06T20:43:43.517 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:43.517 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-06T20:43:43.563 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:43.564 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-06T20:43:43.649 INFO:tasks.ceph:Adding keys to all mons... 2023-11-06T20:43:43.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-06T20:43:43.654 DEBUG:teuthology.orchestra.run.smithi069:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-06T20:43:43.721 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.y] 2023-11-06T20:43:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg== 2023-11-06T20:43:43.722 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2023-11-06T20:43:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg== 2023-11-06T20:43:43.722 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2023-11-06T20:43:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w== 2023-11-06T20:43:43.723 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.0] 2023-11-06T20:43:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw== 2023-11-06T20:43:43.723 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.1] 2023-11-06T20:43:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg== 2023-11-06T20:43:43.724 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.2] 2023-11-06T20:43:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg== 2023-11-06T20:43:43.724 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.3] 2023-11-06T20:43:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ== 2023-11-06T20:43:43.724 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.x] 2023-11-06T20:43:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ== 2023-11-06T20:43:43.725 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2023-11-06T20:43:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ== 2023-11-06T20:43:43.725 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2023-11-06T20:43:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg== 2023-11-06T20:43:43.726 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.4] 2023-11-06T20:43:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw== 2023-11-06T20:43:43.726 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.5] 2023-11-06T20:43:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw== 2023-11-06T20:43:43.726 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.6] 2023-11-06T20:43:43.727 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg== 2023-11-06T20:43:43.727 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.7] 2023-11-06T20:43:43.727 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA== 2023-11-06T20:43:43.727 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2023-11-06T20:43:43.727 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA== 2023-11-06T20:43:43.728 INFO:teuthology.orchestra.run.smithi033.stdout:[client.2] 2023-11-06T20:43:43.728 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg== 2023-11-06T20:43:43.728 INFO:teuthology.orchestra.run.smithi033.stdout:[client.1] 2023-11-06T20:43:43.728 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A== 2023-11-06T20:43:43.728 INFO:teuthology.orchestra.run.smithi033.stdout:[client.3] 2023-11-06T20:43:43.728 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ== 2023-11-06T20:43:43.755 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr.y] 2023-11-06T20:43:43.755 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg== 2023-11-06T20:43:43.756 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.a] 2023-11-06T20:43:43.756 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg== 2023-11-06T20:43:43.756 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.c] 2023-11-06T20:43:43.756 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w== 2023-11-06T20:43:43.756 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.0] 2023-11-06T20:43:43.757 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw== 2023-11-06T20:43:43.757 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.1] 2023-11-06T20:43:43.757 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg== 2023-11-06T20:43:43.757 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.2] 2023-11-06T20:43:43.757 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg== 2023-11-06T20:43:43.758 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.3] 2023-11-06T20:43:43.758 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ== 2023-11-06T20:43:43.758 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr.x] 2023-11-06T20:43:43.758 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ== 2023-11-06T20:43:43.758 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.b] 2023-11-06T20:43:43.759 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ== 2023-11-06T20:43:43.759 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.d] 2023-11-06T20:43:43.759 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg== 2023-11-06T20:43:43.759 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.4] 2023-11-06T20:43:43.759 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw== 2023-11-06T20:43:43.759 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.5] 2023-11-06T20:43:43.760 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw== 2023-11-06T20:43:43.760 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.6] 2023-11-06T20:43:43.760 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg== 2023-11-06T20:43:43.760 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.7] 2023-11-06T20:43:43.761 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA== 2023-11-06T20:43:43.761 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2023-11-06T20:43:43.761 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA== 2023-11-06T20:43:43.761 INFO:teuthology.orchestra.run.smithi069.stdout:[client.2] 2023-11-06T20:43:43.761 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg== 2023-11-06T20:43:43.761 INFO:teuthology.orchestra.run.smithi069.stdout:[client.1] 2023-11-06T20:43:43.762 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A== 2023-11-06T20:43:43.762 INFO:teuthology.orchestra.run.smithi069.stdout:[client.3] 2023-11-06T20:43:43.762 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ== 2023-11-06T20:43:43.763 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T20:43:43.774 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T20:43:43.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T20:43:43.913 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T20:43:44.017 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T20:43:44.022 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T20:43:44.127 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.132 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.239 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.347 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.454 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T20:43:44.561 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T20:43:44.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T20:43:44.669 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T20:43:44.774 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T20:43:44.779 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T20:43:44.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.889 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.991 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:44.995 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:45.097 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:45.102 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:45.206 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:45.211 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T20:43:45.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T20:43:45.318 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T20:43:45.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T20:43:45.427 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T20:43:45.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T20:43:45.536 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T20:43:45.642 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T20:43:45.647 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T20:43:45.751 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-06T20:43:45.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-06T20:43:45.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-06T20:43:45.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-06T20:43:46.016 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-06T20:43:46.088 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-06T20:43:46.203 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-06T20:43:46.274 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-06T20:43:46.344 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-06T20:43:46.459 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-06T20:43:46.530 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:46.535 DEBUG:teuthology.orchestra.run.smithi069:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-06T20:43:46.580 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-06T20:43:46.581 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-06T20:43:46.581 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-06T20:43:46.586 INFO:tasks.ceph.mon.a:Started 2023-11-06T20:43:46.586 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-06T20:43:46.586 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-06T20:43:46.625 INFO:tasks.ceph.mon.b:Started 2023-11-06T20:43:46.625 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-06T20:43:46.625 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-06T20:43:46.629 INFO:tasks.ceph.mon.c:Started 2023-11-06T20:43:46.630 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-06T20:43:46.630 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-06T20:43:46.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-06T20:43:46.635 INFO:tasks.ceph.mgr.y:Started 2023-11-06T20:43:46.636 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-06T20:43:46.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-06T20:43:46.640 INFO:tasks.ceph.mgr.x:Started 2023-11-06T20:43:46.641 INFO:tasks.ceph:Setting crush tunables to default 2023-11-06T20:43:46.641 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd crush tunables default 2023-11-06T20:43:46.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.839+0000 7f60307ff700 1 Processor -- start 2023-11-06T20:43:46.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.839+0000 7f60307ff700 1 -- start start 2023-11-06T20:43:46.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.840+0000 7f60307ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 0x7f60280c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:46.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.840+0000 7f60307ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f60280aaf50 0x7f6028063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:46.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.841+0000 7f60307ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 0x7f6028063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:46.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.841+0000 7f60307ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60280c18f0 con 0x7f60280aaf50 2023-11-06T20:43:46.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.841+0000 7f60307ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6028063b10 con 0x7f60280aa4f0 2023-11-06T20:43:46.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.841+0000 7f60307ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6028158c00 con 0x7f6028157700 2023-11-06T20:43:46.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.841+0000 7f602e59b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 0x7f60280c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:46.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.841+0000 7f602dd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f60280aaf50 0x7f6028063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:46.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.841+0000 7f602e59b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 0x7f60280c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:36742/0 (socket says 172.21.15.33:36742) 2023-11-06T20:43:46.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.841+0000 7f602e59b700 1 -- 172.21.15.33:0/1670448812 learned_addr learned my addr 172.21.15.33:0/1670448812 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:43:46.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.841+0000 7f602ed9c700 1 --2- 172.21.15.33:0/1670448812 >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 0x7f6028063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:46.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.841+0000 7f602e59b700 1 -- 172.21.15.33:0/1670448812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 msgr2=0x7f60280c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T20:43:46.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602e59b700 1 -- 172.21.15.33:0/1670448812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 msgr2=0x7f60280c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T20:43:46.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602e59b700 1 --2- 172.21.15.33:0/1670448812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 0x7f60280c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-06T20:43:46.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602dd9a700 1 -- 172.21.15.33:0/1670448812 >> v2:172.21.15.33:3300/0 conn(0x7f60280aaf50 msgr2=0x7f6028063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-06T20:43:46.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602dd9a700 1 -- 172.21.15.33:0/1670448812 >> v2:172.21.15.33:3300/0 conn(0x7f60280aaf50 msgr2=0x7f6028063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T20:43:46.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602dd9a700 1 --2- 172.21.15.33:0/1670448812 >> v2:172.21.15.33:3300/0 conn(0x7f60280aaf50 0x7f6028063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-06T20:43:46.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602ed9c700 1 -- 172.21.15.33:0/1670448812 >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 msgr2=0x7f6028063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-06T20:43:46.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602ed9c700 1 -- 172.21.15.33:0/1670448812 >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 msgr2=0x7f6028063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T20:43:46.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602ed9c700 1 --2- 172.21.15.33:0/1670448812 >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 0x7f6028063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-06T20:43:46.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602e59b700 1 --2- 172.21.15.33:0/1670448812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 0x7f60280c6f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:43:46.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602ed9c700 1 --2- 172.21.15.33:0/1670448812 >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 0x7f6028063620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:43:46.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:46.842+0000 7f602dd9a700 1 --2- 172.21.15.33:0/1670448812 >> v2:172.21.15.33:3300/0 conn(0x7f60280aaf50 0x7f6028063130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:43:47.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.043+0000 7f602dd9a700 1 --2- 172.21.15.33:0/1670448812 >> v2:172.21.15.33:3300/0 conn(0x7f60280aaf50 0x7f6028063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:47.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.043+0000 7f602e59b700 1 --2- 172.21.15.33:0/1670448812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 0x7f60280c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:47.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.043+0000 7f602ed9c700 1 --2- 172.21.15.33:0/1670448812 >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 0x7f6028063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:47.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.043+0000 7f602dd9a700 1 -- 172.21.15.33:0/1670448812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 msgr2=0x7f60280c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:47.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.043+0000 7f602dd9a700 1 --2- 172.21.15.33:0/1670448812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 0x7f60280c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.043+0000 7f602dd9a700 1 -- 172.21.15.33:0/1670448812 >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 msgr2=0x7f6028063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.043+0000 7f602dd9a700 1 --2- 172.21.15.33:0/1670448812 >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 0x7f6028063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.043+0000 7f602dd9a700 1 -- 172.21.15.33:0/1670448812 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028157ff0 con 0x7f60280aaf50 2023-11-06T20:43:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.043+0000 7f602ed9c700 1 --2- 172.21.15.33:0/1670448812 >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 0x7f6028063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:43:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.043+0000 7f602dd9a700 1 --2- 172.21.15.33:0/1670448812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60280aaf50 0x7f6028063130 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f60280c53d0 tx=0x7f6018008040).ready entity=mon.0 client_cookie=34959e1dc329c7e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.044+0000 7f602d599700 1 -- 172.21.15.33:0/1670448812 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60180104f0 con 0x7f60280aaf50 2023-11-06T20:43:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.044+0000 7f602d599700 1 -- 172.21.15.33:0/1670448812 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6018010650 con 0x7f60280aaf50 2023-11-06T20:43:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.044+0000 7f602d599700 1 -- 172.21.15.33:0/1670448812 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60180109d0 con 0x7f60280aaf50 2023-11-06T20:43:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.044+0000 7f60307ff700 1 -- 172.21.15.33:0/1670448812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60280aaf50 msgr2=0x7f6028063130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.044+0000 7f60307ff700 1 --2- 172.21.15.33:0/1670448812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60280aaf50 0x7f6028063130 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f60280c53d0 tx=0x7f6018008040).stop 2023-11-06T20:43:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.045+0000 7f60307ff700 1 -- 172.21.15.33:0/1670448812 shutdown_connections 2023-11-06T20:43:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.045+0000 7f60307ff700 1 --2- 172.21.15.33:0/1670448812 >> v2:172.21.15.69:3300/0 conn(0x7f60280aa4f0 0x7f6028063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.045+0000 7f60307ff700 1 --2- 172.21.15.33:0/1670448812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6028157700 0x7f60280c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.045+0000 7f60307ff700 1 --2- 172.21.15.33:0/1670448812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60280aaf50 0x7f6028063130 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.045+0000 7f60307ff700 1 -- 172.21.15.33:0/1670448812 >> 172.21.15.33:0/1670448812 conn(0x7f602805ec80 msgr2=0x7f60280cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.045+0000 7f60307ff700 1 -- 172.21.15.33:0/1670448812 shutdown_connections 2023-11-06T20:43:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.045+0000 7f60307ff700 1 -- 172.21.15.33:0/1670448812 wait complete. 2023-11-06T20:43:47.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.046+0000 7f60307ff700 1 Processor -- start 2023-11-06T20:43:47.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.046+0000 7f60307ff700 1 -- start start 2023-11-06T20:43:47.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.047+0000 7f60307ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60280aa4f0 0x7f60280ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:47.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.047+0000 7f60307ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60280aaf50 0x7f6028167d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:47.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.047+0000 7f60307ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6028157700 0x7f60280c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f60307ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028153bf0 con 0x7f6028157700 2023-11-06T20:43:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f60307ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028155480 con 0x7f60280aa4f0 2023-11-06T20:43:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f60307ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6028155760 con 0x7f60280aaf50 2023-11-06T20:43:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6028157700 0x7f60280c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602e59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60280aa4f0 0x7f60280ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:47.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602dd9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60280aaf50 0x7f6028167d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:47.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602e59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60280aa4f0 0x7f60280ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:45344/0 (socket says 172.21.15.33:45344) 2023-11-06T20:43:47.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602e59b700 1 -- 172.21.15.33:0/2771026662 learned_addr learned my addr 172.21.15.33:0/2771026662 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:43:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602ed9c700 1 -- 172.21.15.33:0/2771026662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60280aaf50 msgr2=0x7f6028167d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:47.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602ed9c700 1 --2- 172.21.15.33:0/2771026662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60280aaf50 0x7f6028167d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:47.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602ed9c700 1 -- 172.21.15.33:0/2771026662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60280aa4f0 msgr2=0x7f60280ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:47.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602ed9c700 1 --2- 172.21.15.33:0/2771026662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60280aa4f0 0x7f60280ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:47.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602ed9c700 1 -- 172.21.15.33:0/2771026662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601800e040 con 0x7f6028157700 2023-11-06T20:43:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602dd9a700 1 --2- 172.21.15.33:0/2771026662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60280aaf50 0x7f6028167d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.048+0000 7f602ed9c700 1 --2- 172.21.15.33:0/2771026662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6028157700 0x7f60280c38d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6020007fb0 tx=0x7f6020009930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.049+0000 7f601e7fc700 1 -- 172.21.15.33:0/2771026662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020002d30 con 0x7f6028157700 2023-11-06T20:43:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.049+0000 7f601e7fc700 1 -- 172.21.15.33:0/2771026662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602000f030 con 0x7f6028157700 2023-11-06T20:43:47.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.049+0000 7f601e7fc700 1 -- 172.21.15.33:0/2771026662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020015650 con 0x7f6028157700 2023-11-06T20:43:47.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.049+0000 7f60307ff700 1 -- 172.21.15.33:0/2771026662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6028155a40 con 0x7f6028157700 2023-11-06T20:43:47.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.049+0000 7f60307ff700 1 -- 172.21.15.33:0/2771026662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6028154fc0 con 0x7f6028157700 2023-11-06T20:43:47.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.050+0000 7f601e7fc700 1 -- 172.21.15.33:0/2771026662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f6020003040 con 0x7f6028157700 2023-11-06T20:43:47.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.050+0000 7f601e7fc700 1 -- 172.21.15.33:0/2771026662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f602001e3f0 con 0x7f6028157700 2023-11-06T20:43:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.050+0000 7f60307ff700 1 -- 172.21.15.33:0/2771026662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028156240 con 0x7f6028157700 2023-11-06T20:43:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.055+0000 7f601e7fc700 1 -- 172.21.15.33:0/2771026662 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f6020015a00 con 0x7f6028157700 2023-11-06T20:43:47.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.133+0000 7f60307ff700 1 -- 172.21.15.33:0/2771026662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6028155210 con 0x7f6028157700 2023-11-06T20:43:47.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.137+0000 7f601e7fc700 1 -- 172.21.15.33:0/2771026662 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f602001e680 con 0x7f6028157700 2023-11-06T20:43:47.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.138+0000 7f60307ff700 1 -- 172.21.15.33:0/2771026662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6028157700 msgr2=0x7f60280c38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:47.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.138+0000 7f60307ff700 1 --2- 172.21.15.33:0/2771026662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6028157700 0x7f60280c38d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6020007fb0 tx=0x7f6020009930).stop 2023-11-06T20:43:47.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.138+0000 7f60307ff700 1 -- 172.21.15.33:0/2771026662 shutdown_connections 2023-11-06T20:43:47.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.138+0000 7f60307ff700 1 --2- 172.21.15.33:0/2771026662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6028157700 0x7f60280c38d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:47.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.138+0000 7f60307ff700 1 --2- 172.21.15.33:0/2771026662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60280aa4f0 0x7f60280ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:47.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.138+0000 7f60307ff700 1 --2- 172.21.15.33:0/2771026662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60280aaf50 0x7f6028167d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:47.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.138+0000 7f60307ff700 1 -- 172.21.15.33:0/2771026662 >> 172.21.15.33:0/2771026662 conn(0x7f602805ec80 msgr2=0x7f60280c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:47.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.138+0000 7f60307ff700 1 -- 172.21.15.33:0/2771026662 shutdown_connections 2023-11-06T20:43:47.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:47.138+0000 7f60307ff700 1 -- 172.21.15.33:0/2771026662 wait complete. 2023-11-06T20:43:47.140 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2023-11-06T20:43:47.154 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:47.155+0000 7fa46b635680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:43:47.155 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-06T20:43:47.156 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:47.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-06T20:43:47.180 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:47.180+0000 7f0052f61680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:43:47.233 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:47.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-06T20:43:47.242 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:47.243+0000 7fa46b635680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:43:47.282 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:47.279+0000 7f0052f61680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:43:47.347 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:47.348 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-06T20:43:47.459 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-11-06T20:43:47.459 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-06T20:43:47.569 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:47.570 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-06T20:43:47.643 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:47.643 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-06T20:43:47.759 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:47.759 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-06T20:43:47.775 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:47.775+0000 7fa46b635680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:43:47.835 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-11-06T20:43:47.836 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-06T20:43:47.845 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:47.845+0000 7f0052f61680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:43:47.946 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd new 04ae67d0-5222-4147-8dbe-c91a41c71ce1 0 2023-11-06T20:43:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.172+0000 7f66a47ff700 1 Processor -- start 2023-11-06T20:43:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.172+0000 7f66a47ff700 1 -- start start 2023-11-06T20:43:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.173+0000 7f66a47ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f669c0aaf50 0x7f669c063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.173+0000 7f66a47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aa4f0 0x7f669c063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.173+0000 7f66a47ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669c158c00 con 0x7f669c157700 2023-11-06T20:43:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.173+0000 7f66a47ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669c0c18f0 con 0x7f669c0aaf50 2023-11-06T20:43:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.173+0000 7f66a47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c063b10 con 0x7f669c0aa4f0 2023-11-06T20:43:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a1d9a700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f669c0aaf50 0x7f669c063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a2d9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aa4f0 0x7f669c063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a1d9a700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f669c0aaf50 0x7f669c063130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33196/0 (socket says 172.21.15.69:33196) 2023-11-06T20:43:48.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a1d9a700 1 -- 172.21.15.69:0/1468291483 learned_addr learned my addr 172.21.15.69:0/1468291483 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:48.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a1d9a700 1 -- 172.21.15.69:0/1468291483 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aa4f0 msgr2=0x7f669c063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:48.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a1d9a700 1 --2- 172.21.15.69:0/1468291483 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aa4f0 0x7f669c063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a1d9a700 1 -- 172.21.15.69:0/1468291483 >> v1:172.21.15.69:6789/0 conn(0x7f669c157700 legacy=0x7f669c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a1d9a700 1 -- 172.21.15.69:0/1468291483 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c0c7610 con 0x7f669c0aaf50 2023-11-06T20:43:48.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a1599700 1 -- 172.21.15.69:0/1468291483 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066491633 0 0) 0x7f669c158c00 con 0x7f669c157700 2023-11-06T20:43:48.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a2d9c700 1 --2- 172.21.15.69:0/1468291483 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aa4f0 0x7f669c063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:43:48.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a1d9a700 1 --2- 172.21.15.69:0/1468291483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aaf50 0x7f669c063130 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f668c006fd0 tx=0x7f668c0090f0).ready entity=mon.1 client_cookie=5dc874832f925f4f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:48.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.174+0000 7f66a1599700 1 -- 172.21.15.69:0/1468291483 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c00b600 con 0x7f669c0aaf50 2023-11-06T20:43:48.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.175+0000 7f66a1599700 1 -- 172.21.15.69:0/1468291483 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c00b760 con 0x7f669c0aaf50 2023-11-06T20:43:48.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.175+0000 7f66a1599700 1 -- 172.21.15.69:0/1468291483 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c00bae0 con 0x7f669c0aaf50 2023-11-06T20:43:48.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.175+0000 7f66a47ff700 1 -- 172.21.15.69:0/1468291483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aaf50 msgr2=0x7f669c063130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.175+0000 7f66a47ff700 1 --2- 172.21.15.69:0/1468291483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aaf50 0x7f669c063130 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f668c006fd0 tx=0x7f668c0090f0).stop 2023-11-06T20:43:48.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.175+0000 7f66a47ff700 1 -- 172.21.15.69:0/1468291483 shutdown_connections 2023-11-06T20:43:48.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.175+0000 7f66a47ff700 1 --2- 172.21.15.69:0/1468291483 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aa4f0 0x7f669c063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.175+0000 7f66a47ff700 1 --2- 172.21.15.69:0/1468291483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aaf50 0x7f669c063130 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.175+0000 7f66a47ff700 1 -- 172.21.15.69:0/1468291483 >> 172.21.15.69:0/1468291483 conn(0x7f669c05ec80 msgr2=0x7f669c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.175+0000 7f66a47ff700 1 -- 172.21.15.69:0/1468291483 shutdown_connections 2023-11-06T20:43:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.175+0000 7f66a47ff700 1 -- 172.21.15.69:0/1468291483 wait complete. 2023-11-06T20:43:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.177+0000 7f66a47ff700 1 Processor -- start 2023-11-06T20:43:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.177+0000 7f66a47ff700 1 -- start start 2023-11-06T20:43:48.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a47ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aa4f0 0x7f669c14dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aaf50 0x7f669c0c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a47ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f669c157700 0x7f669c0c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a47ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0c32a0 con 0x7f669c157700 2023-11-06T20:43:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a47ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c15a4d0 con 0x7f669c0aa4f0 2023-11-06T20:43:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0c34a0 con 0x7f669c0aaf50 2023-11-06T20:43:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a1d9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aaf50 0x7f669c0c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a259b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aa4f0 0x7f669c14dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a1d9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aaf50 0x7f669c0c28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.69:59850/0 (socket says 172.21.15.69:59850) 2023-11-06T20:43:48.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a259b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aa4f0 0x7f669c14dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33210/0 (socket says 172.21.15.69:33210) 2023-11-06T20:43:48.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a1d9a700 1 -- 172.21.15.69:0/825399023 learned_addr learned my addr 172.21.15.69:0/825399023 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:48.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.178+0000 7f66a2d9c700 1 --2- 172.21.15.69:0/825399023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f669c157700 0x7f669c0c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66a259b700 1 -- 172.21.15.69:0/825399023 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aaf50 msgr2=0x7f669c0c28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66a259b700 1 --2- 172.21.15.69:0/825399023 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aaf50 0x7f669c0c28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66a259b700 1 -- 172.21.15.69:0/825399023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f669c157700 msgr2=0x7f669c0c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66a259b700 1 --2- 172.21.15.69:0/825399023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f669c157700 0x7f669c0c2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66a259b700 1 -- 172.21.15.69:0/825399023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c009370 con 0x7f669c0aa4f0 2023-11-06T20:43:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66a1d9a700 1 --2- 172.21.15.69:0/825399023 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aaf50 0x7f669c0c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66a259b700 1 --2- 172.21.15.69:0/825399023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aa4f0 0x7f669c14dcd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6698001eb0 tx=0x7f6698007bb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66927fc700 1 -- 172.21.15.69:0/825399023 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698012400 con 0x7f669c0aa4f0 2023-11-06T20:43:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66927fc700 1 -- 172.21.15.69:0/825399023 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6698012580 con 0x7f669c0aa4f0 2023-11-06T20:43:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66a47ff700 1 -- 172.21.15.69:0/825399023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669c0c3a60 con 0x7f669c0aa4f0 2023-11-06T20:43:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66927fc700 1 -- 172.21.15.69:0/825399023 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698012940 con 0x7f669c0aa4f0 2023-11-06T20:43:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.179+0000 7f66a47ff700 1 -- 172.21.15.69:0/825399023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c0ae2c0 con 0x7f669c0aa4f0 2023-11-06T20:43:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.180+0000 7f66927fc700 1 -- 172.21.15.69:0/825399023 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f669801b3b0 con 0x7f669c0aa4f0 2023-11-06T20:43:48.187 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.180+0000 7f66927fc700 1 -- 172.21.15.69:0/825399023 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f669801b9d0 con 0x7f669c0aa4f0 2023-11-06T20:43:48.187 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.180+0000 7f66a47ff700 1 -- 172.21.15.69:0/825399023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669c0ae540 con 0x7f669c0aa4f0 2023-11-06T20:43:48.187 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.183+0000 7f66927fc700 1 -- 172.21.15.69:0/825399023 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f6698021060 con 0x7f669c0aa4f0 2023-11-06T20:43:48.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.264+0000 7f66a47ff700 1 -- 172.21.15.69:0/825399023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd new", "uuid": "04ae67d0-5222-4147-8dbe-c91a41c71ce1", "id": 0} v 0) v1 -- 0x7f669c0cb670 con 0x7f669c0aa4f0 2023-11-06T20:43:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.273+0000 7f66927fc700 1 -- 172.21.15.69:0/825399023 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "04ae67d0-5222-4147-8dbe-c91a41c71ce1", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6698017070 con 0x7f669c0aa4f0 2023-11-06T20:43:48.273 INFO:teuthology.orchestra.run.smithi069.stdout:0 2023-11-06T20:43:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.274+0000 7f66a47ff700 1 -- 172.21.15.69:0/825399023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aa4f0 msgr2=0x7f669c14dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.274+0000 7f66a47ff700 1 --2- 172.21.15.69:0/825399023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aa4f0 0x7f669c14dcd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6698001eb0 tx=0x7f6698007bb0).stop 2023-11-06T20:43:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.274+0000 7f66a47ff700 1 -- 172.21.15.69:0/825399023 shutdown_connections 2023-11-06T20:43:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.274+0000 7f66a47ff700 1 --2- 172.21.15.69:0/825399023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f669c157700 0x7f669c0c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.274+0000 7f66a47ff700 1 --2- 172.21.15.69:0/825399023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f669c0aa4f0 0x7f669c14dcd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.274+0000 7f66a47ff700 1 --2- 172.21.15.69:0/825399023 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f669c0aaf50 0x7f669c0c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.274+0000 7f66a47ff700 1 -- 172.21.15.69:0/825399023 >> 172.21.15.69:0/825399023 conn(0x7f669c05ec80 msgr2=0x7f669c0646d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:48.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.274+0000 7f66a47ff700 1 -- 172.21.15.69:0/825399023 shutdown_connections 2023-11-06T20:43:48.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.274+0000 7f66a47ff700 1 -- 172.21.15.69:0/825399023 wait complete. 2023-11-06T20:43:48.291 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd new 6b36f155-ed6d-4dbd-86a9-82c284781ad3 1 2023-11-06T20:43:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.478+0000 7f48ec3ff700 1 Processor -- start 2023-11-06T20:43:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.478+0000 7f48ec3ff700 1 -- start start 2023-11-06T20:43:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.479+0000 7f48ec3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48e4157700 0x7f48e40c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.479+0000 7f48ec3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f48e40aaf50 0x7f48e4063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.479+0000 7f48ec3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aa4f0 0x7f48e4063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.479+0000 7f48ec3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e40c18f0 con 0x7f48e4157700 2023-11-06T20:43:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.479+0000 7f48ec3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e4063b10 con 0x7f48e40aaf50 2023-11-06T20:43:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.479+0000 7f48ec3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4158c00 con 0x7f48e40aa4f0 2023-11-06T20:43:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.479+0000 7f48e999a700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f48e40aaf50 0x7f48e4063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.479+0000 7f48e999a700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f48e40aaf50 0x7f48e4063130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33212/0 (socket says 172.21.15.69:33212) 2023-11-06T20:43:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48e999a700 1 -- 172.21.15.69:0/3528082294 learned_addr learned my addr 172.21.15.69:0/3528082294 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:48.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.479+0000 7f48ea99c700 1 --2- 172.21.15.69:0/3528082294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aa4f0 0x7f48e4063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48ea19b700 1 --2- 172.21.15.69:0/3528082294 >> v2:172.21.15.33:3300/0 conn(0x7f48e4157700 0x7f48e40c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48e999a700 1 -- 172.21.15.69:0/3528082294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aa4f0 msgr2=0x7f48e4063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:48.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48e999a700 1 --2- 172.21.15.69:0/3528082294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aa4f0 0x7f48e4063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48e999a700 1 -- 172.21.15.69:0/3528082294 >> v2:172.21.15.33:3300/0 conn(0x7f48e4157700 msgr2=0x7f48e40c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48e999a700 1 --2- 172.21.15.69:0/3528082294 >> v2:172.21.15.33:3300/0 conn(0x7f48e4157700 0x7f48e40c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48e999a700 1 -- 172.21.15.69:0/3528082294 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e4157ff0 con 0x7f48e40aaf50 2023-11-06T20:43:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48ea99c700 1 --2- 172.21.15.69:0/3528082294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aa4f0 0x7f48e4063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48e999a700 1 --2- 172.21.15.69:0/3528082294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aaf50 0x7f48e4063130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f48e4064470 tx=0x7f48d40090f0).ready entity=mon.1 client_cookie=78e1f2db90984a30 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48e9199700 1 -- 172.21.15.69:0/3528082294 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400f510 con 0x7f48e40aaf50 2023-11-06T20:43:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.480+0000 7f48e9199700 1 -- 172.21.15.69:0/3528082294 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d400f670 con 0x7f48e40aaf50 2023-11-06T20:43:48.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.481+0000 7f48e9199700 1 -- 172.21.15.69:0/3528082294 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400f9f0 con 0x7f48e40aaf50 2023-11-06T20:43:48.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.481+0000 7f48ec3ff700 1 -- 172.21.15.69:0/3528082294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aaf50 msgr2=0x7f48e4063130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.481+0000 7f48ec3ff700 1 --2- 172.21.15.69:0/3528082294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aaf50 0x7f48e4063130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f48e4064470 tx=0x7f48d40090f0).stop 2023-11-06T20:43:48.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.481+0000 7f48ec3ff700 1 -- 172.21.15.69:0/3528082294 shutdown_connections 2023-11-06T20:43:48.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.481+0000 7f48ec3ff700 1 --2- 172.21.15.69:0/3528082294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aa4f0 0x7f48e4063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.481+0000 7f48ec3ff700 1 --2- 172.21.15.69:0/3528082294 >> v2:172.21.15.33:3300/0 conn(0x7f48e4157700 0x7f48e40c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.481+0000 7f48ec3ff700 1 --2- 172.21.15.69:0/3528082294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aaf50 0x7f48e4063130 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.481+0000 7f48ec3ff700 1 -- 172.21.15.69:0/3528082294 >> 172.21.15.69:0/3528082294 conn(0x7f48e405ec80 msgr2=0x7f48e40cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:48.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.481+0000 7f48ec3ff700 1 -- 172.21.15.69:0/3528082294 shutdown_connections 2023-11-06T20:43:48.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.481+0000 7f48ec3ff700 1 -- 172.21.15.69:0/3528082294 wait complete. 2023-11-06T20:43:48.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.483+0000 7f48ec3ff700 1 Processor -- start 2023-11-06T20:43:48.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.483+0000 7f48ec3ff700 1 -- start start 2023-11-06T20:43:48.489 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ec3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aa4f0 0x7f48e4156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.489 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ec3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aaf50 0x7f48e4154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.489 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ec3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e4157700 0x7f48e41550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.489 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ec3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4156470 con 0x7f48e4157700 2023-11-06T20:43:48.489 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ec3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41555d0 con 0x7f48e40aa4f0 2023-11-06T20:43:48.489 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ec3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4153550 con 0x7f48e40aaf50 2023-11-06T20:43:48.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48e999a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aaf50 0x7f48e4154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ea19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aa4f0 0x7f48e4156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48e999a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aaf50 0x7f48e4154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.69:59854/0 (socket says 172.21.15.69:59854) 2023-11-06T20:43:48.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ea19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aa4f0 0x7f48e4156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33220/0 (socket says 172.21.15.69:33220) 2023-11-06T20:43:48.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48e999a700 1 -- 172.21.15.69:0/331533073 learned_addr learned my addr 172.21.15.69:0/331533073 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:48.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ea99c700 1 --2- 172.21.15.69:0/331533073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e4157700 0x7f48e41550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ea19b700 1 -- 172.21.15.69:0/331533073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aaf50 msgr2=0x7f48e4154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ea19b700 1 --2- 172.21.15.69:0/331533073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aaf50 0x7f48e4154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ea19b700 1 -- 172.21.15.69:0/331533073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e4157700 msgr2=0x7f48e41550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ea19b700 1 --2- 172.21.15.69:0/331533073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e4157700 0x7f48e41550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.492 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.484+0000 7f48ea19b700 1 -- 172.21.15.69:0/331533073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc009a20 con 0x7f48e40aa4f0 2023-11-06T20:43:48.492 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.485+0000 7f48ea19b700 1 --2- 172.21.15.69:0/331533073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aa4f0 0x7f48e4156900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f48e000e6f0 tx=0x7f48e0009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:48.492 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.485+0000 7f48da7fc700 1 -- 172.21.15.69:0/331533073 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0012600 con 0x7f48e40aa4f0 2023-11-06T20:43:48.492 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.485+0000 7f48da7fc700 1 -- 172.21.15.69:0/331533073 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e0012760 con 0x7f48e40aa4f0 2023-11-06T20:43:48.492 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.485+0000 7f48ec3ff700 1 -- 172.21.15.69:0/331533073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d4009370 con 0x7f48e40aa4f0 2023-11-06T20:43:48.492 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.485+0000 7f48da7fc700 1 -- 172.21.15.69:0/331533073 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0012ae0 con 0x7f48e40aa4f0 2023-11-06T20:43:48.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.485+0000 7f48ec3ff700 1 -- 172.21.15.69:0/331533073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e4155e20 con 0x7f48e40aa4f0 2023-11-06T20:43:48.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.485+0000 7f48da7fc700 1 -- 172.21.15.69:0/331533073 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f48e001b660 con 0x7f48e40aa4f0 2023-11-06T20:43:48.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.485+0000 7f48da7fc700 1 -- 172.21.15.69:0/331533073 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f48e001a3b0 con 0x7f48e40aa4f0 2023-11-06T20:43:48.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.486+0000 7f48ec3ff700 1 -- 172.21.15.69:0/331533073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e406c6c0 con 0x7f48e40aa4f0 2023-11-06T20:43:48.494 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.489+0000 7f48da7fc700 1 -- 172.21.15.69:0/331533073 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f48e003ae50 con 0x7f48e40aa4f0 2023-11-06T20:43:48.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.571+0000 7f48ec3ff700 1 -- 172.21.15.69:0/331533073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6b36f155-ed6d-4dbd-86a9-82c284781ad3", "id": 1} v 0) v1 -- 0x7f48e41585b0 con 0x7f48e40aa4f0 2023-11-06T20:43:48.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.579+0000 7f48da7fc700 1 -- 172.21.15.69:0/331533073 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b36f155-ed6d-4dbd-86a9-82c284781ad3", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f48e0029070 con 0x7f48e40aa4f0 2023-11-06T20:43:48.579 INFO:teuthology.orchestra.run.smithi069.stdout:1 2023-11-06T20:43:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.580+0000 7f48ec3ff700 1 -- 172.21.15.69:0/331533073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aa4f0 msgr2=0x7f48e4156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.580+0000 7f48ec3ff700 1 --2- 172.21.15.69:0/331533073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aa4f0 0x7f48e4156900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f48e000e6f0 tx=0x7f48e0009060).stop 2023-11-06T20:43:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.580+0000 7f48ec3ff700 1 -- 172.21.15.69:0/331533073 shutdown_connections 2023-11-06T20:43:48.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.580+0000 7f48ec3ff700 1 --2- 172.21.15.69:0/331533073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e4157700 0x7f48e41550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.580+0000 7f48ec3ff700 1 --2- 172.21.15.69:0/331533073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e40aa4f0 0x7f48e4156900 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.580+0000 7f48ec3ff700 1 --2- 172.21.15.69:0/331533073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48e40aaf50 0x7f48e4154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.580+0000 7f48ec3ff700 1 -- 172.21.15.69:0/331533073 >> 172.21.15.69:0/331533073 conn(0x7f48e405ec80 msgr2=0x7f48e41513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:48.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.580+0000 7f48ec3ff700 1 -- 172.21.15.69:0/331533073 shutdown_connections 2023-11-06T20:43:48.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.580+0000 7f48ec3ff700 1 -- 172.21.15.69:0/331533073 wait complete. 2023-11-06T20:43:48.596 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd new e9f10394-ee41-4786-95c9-76d96c52df77 2 2023-11-06T20:43:48.708 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:48.708+0000 7fa46b635680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:43:48.754 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:48.755+0000 7f0052f61680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:43:48.796 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:48.797+0000 7fa46b635680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:43:48.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.818+0000 7f43b3fff700 1 Processor -- start 2023-11-06T20:43:48.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.818+0000 7f43b3fff700 1 -- start start 2023-11-06T20:43:48.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b3fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f43b41542c0 0x7f43b40644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b40aade0 0x7f43b4064990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b3fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b40c0680 con 0x7f43b40c5b50 2023-11-06T20:43:48.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b3fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b4064e80 con 0x7f43b41542c0 2023-11-06T20:43:48.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b3fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b40c1b20 con 0x7f43b40aade0 2023-11-06T20:43:48.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b2ffd700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f43b41542c0 0x7f43b40644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b27fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b40aade0 0x7f43b4064990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b2ffd700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f43b41542c0 0x7f43b40644a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33236/0 (socket says 172.21.15.69:33236) 2023-11-06T20:43:48.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b2ffd700 1 -- 172.21.15.69:0/1270473060 learned_addr learned my addr 172.21.15.69:0/1270473060 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:48.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b27fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b40aade0 0x7f43b4064990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.69:59868/0 (socket says 172.21.15.69:59868) 2023-11-06T20:43:48.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b2ffd700 1 -- 172.21.15.69:0/1270473060 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b40aade0 msgr2=0x7f43b4064990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:48.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b2ffd700 1 --2- 172.21.15.69:0/1270473060 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b40aade0 0x7f43b4064990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b2ffd700 1 -- 172.21.15.69:0/1270473060 >> v1:172.21.15.33:6789/0 conn(0x7f43b40c5b50 legacy=0x7f43b40c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:48.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b2ffd700 1 -- 172.21.15.69:0/1270473060 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b40bfd90 con 0x7f43b41542c0 2023-11-06T20:43:48.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.821+0000 7f43b27fc700 1 --2- 172.21.15.69:0/1270473060 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b40aade0 0x7f43b4064990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:48.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b2ffd700 1 --2- 172.21.15.69:0/1270473060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b41542c0 0x7f43b40644a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f43a4006fd0 tx=0x7f43a40090f0).ready entity=mon.1 client_cookie=b1e04e5e32f384d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:48.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b1ffb700 1 -- 172.21.15.69:0/1270473060 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a400b520 con 0x7f43b41542c0 2023-11-06T20:43:48.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b1ffb700 1 -- 172.21.15.69:0/1270473060 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a400b680 con 0x7f43b41542c0 2023-11-06T20:43:48.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b1ffb700 1 -- 172.21.15.69:0/1270473060 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a400ba00 con 0x7f43b41542c0 2023-11-06T20:43:48.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b3fff700 1 -- 172.21.15.69:0/1270473060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b41542c0 msgr2=0x7f43b40644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b3fff700 1 --2- 172.21.15.69:0/1270473060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b41542c0 0x7f43b40644a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f43a4006fd0 tx=0x7f43a40090f0).stop 2023-11-06T20:43:48.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b3fff700 1 -- 172.21.15.69:0/1270473060 shutdown_connections 2023-11-06T20:43:48.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b3fff700 1 --2- 172.21.15.69:0/1270473060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b41542c0 0x7f43b40644a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b3fff700 1 --2- 172.21.15.69:0/1270473060 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b40aade0 0x7f43b4064990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b3fff700 1 -- 172.21.15.69:0/1270473060 >> 172.21.15.69:0/1270473060 conn(0x7f43b40cb760 msgr2=0x7f43b40c7730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:48.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b3fff700 1 -- 172.21.15.69:0/1270473060 shutdown_connections 2023-11-06T20:43:48.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.822+0000 7f43b3fff700 1 -- 172.21.15.69:0/1270473060 wait complete. 2023-11-06T20:43:48.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.825+0000 7f43b3fff700 1 Processor -- start 2023-11-06T20:43:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.825+0000 7f43b3fff700 1 -- start start 2023-11-06T20:43:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b40aade0 0x7f43b4155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b40c5b50 0x7f43b40afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b41542c0 0x7f43b413d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b40af7b0 con 0x7f43b40c5b50 2023-11-06T20:43:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b3fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b40c2800 con 0x7f43b40aade0 2023-11-06T20:43:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b3fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4155a10 con 0x7f43b41542c0 2023-11-06T20:43:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b40aade0 0x7f43b4155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b40aade0 0x7f43b4155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33252/0 (socket says 172.21.15.69:33252) 2023-11-06T20:43:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b2ffd700 1 -- 172.21.15.69:0/87964983 learned_addr learned my addr 172.21.15.69:0/87964983 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b27fc700 1 --2- 172.21.15.69:0/87964983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b40c5b50 0x7f43b40afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.826+0000 7f43b37fe700 1 --2- 172.21.15.69:0/87964983 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b41542c0 0x7f43b413d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f43b2ffd700 1 -- 172.21.15.69:0/87964983 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b41542c0 msgr2=0x7f43b413d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f43b2ffd700 1 --2- 172.21.15.69:0/87964983 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b41542c0 0x7f43b413d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f43b2ffd700 1 -- 172.21.15.69:0/87964983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b40c5b50 msgr2=0x7f43b40afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f43b2ffd700 1 --2- 172.21.15.69:0/87964983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b40c5b50 0x7f43b40afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f43b2ffd700 1 -- 172.21.15.69:0/87964983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a80092a0 con 0x7f43b40aade0 2023-11-06T20:43:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f43b27fc700 1 --2- 172.21.15.69:0/87964983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b40c5b50 0x7f43b40afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f43b2ffd700 1 --2- 172.21.15.69:0/87964983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b40aade0 0x7f43b4155520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f43b40841d0 tx=0x7f43a4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f439affd700 1 -- 172.21.15.69:0/87964983 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a400be90 con 0x7f43b40aade0 2023-11-06T20:43:48.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f439affd700 1 -- 172.21.15.69:0/87964983 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a400b7f0 con 0x7f43b40aade0 2023-11-06T20:43:48.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f439affd700 1 -- 172.21.15.69:0/87964983 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a401b620 con 0x7f43b40aade0 2023-11-06T20:43:48.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.827+0000 7f43b3fff700 1 -- 172.21.15.69:0/87964983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a4009370 con 0x7f43b40aade0 2023-11-06T20:43:48.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.828+0000 7f43b3fff700 1 -- 172.21.15.69:0/87964983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b40c21b0 con 0x7f43b40aade0 2023-11-06T20:43:48.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.828+0000 7f439affd700 1 -- 172.21.15.69:0/87964983 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f43a401bc30 con 0x7f43b40aade0 2023-11-06T20:43:48.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.828+0000 7f439affd700 1 -- 172.21.15.69:0/87964983 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f43a4024940 con 0x7f43b40aade0 2023-11-06T20:43:48.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.829+0000 7f43b3fff700 1 -- 172.21.15.69:0/87964983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b406ad80 con 0x7f43b40aade0 2023-11-06T20:43:48.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.831+0000 7f439affd700 1 -- 172.21.15.69:0/87964983 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f43a4029060 con 0x7f43b40aade0 2023-11-06T20:43:48.843 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:48.843+0000 7f0052f61680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:43:48.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.914+0000 7f43b3fff700 1 -- 172.21.15.69:0/87964983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e9f10394-ee41-4786-95c9-76d96c52df77", "id": 2} v 0) v1 -- 0x7f43b40475a0 con 0x7f43b40aade0 2023-11-06T20:43:48.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.922+0000 7f439affd700 1 -- 172.21.15.69:0/87964983 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e9f10394-ee41-4786-95c9-76d96c52df77", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f43a4033020 con 0x7f43b40aade0 2023-11-06T20:43:48.922 INFO:teuthology.orchestra.run.smithi069.stdout:2 2023-11-06T20:43:48.923 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.923+0000 7f43b3fff700 1 -- 172.21.15.69:0/87964983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b40aade0 msgr2=0x7f43b4155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:48.924 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.923+0000 7f43b3fff700 1 --2- 172.21.15.69:0/87964983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b40aade0 0x7f43b4155520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f43b40841d0 tx=0x7f43a4015000).stop 2023-11-06T20:43:48.924 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.923+0000 7f43b3fff700 1 -- 172.21.15.69:0/87964983 shutdown_connections 2023-11-06T20:43:48.924 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.923+0000 7f43b3fff700 1 --2- 172.21.15.69:0/87964983 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b41542c0 0x7f43b413d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.924 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.923+0000 7f43b3fff700 1 --2- 172.21.15.69:0/87964983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b40aade0 0x7f43b4155520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.923+0000 7f43b3fff700 1 --2- 172.21.15.69:0/87964983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b40c5b50 0x7f43b40afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:48.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.923+0000 7f43b3fff700 1 -- 172.21.15.69:0/87964983 >> 172.21.15.69:0/87964983 conn(0x7f43b40cb760 msgr2=0x7f43b40c7730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:48.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.923+0000 7f43b3fff700 1 -- 172.21.15.69:0/87964983 shutdown_connections 2023-11-06T20:43:48.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:48.923+0000 7f43b3fff700 1 -- 172.21.15.69:0/87964983 wait complete. 2023-11-06T20:43:48.939 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd new 64d3860c-4915-4ff6-b485-ad289ed7126c 3 2023-11-06T20:43:49.044 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:49.044+0000 7fa46b635680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:43:49.085 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:49.086+0000 7f0052f61680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:43:49.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.126+0000 7f2e421ff700 1 Processor -- start 2023-11-06T20:43:49.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.126+0000 7f2e421ff700 1 -- start start 2023-11-06T20:43:49.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.127+0000 7f2e421ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e3c157700 0x7f2e3c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e421ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c063840 0x7f2e3c0c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e421ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3c158c00 con 0x7f2e3c0aaf50 2023-11-06T20:43:49.130 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e421ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c0c18f0 con 0x7f2e3c157700 2023-11-06T20:43:49.130 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e421ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c157ff0 con 0x7f2e3c063840 2023-11-06T20:43:49.130 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e3affd700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f2e3c0aaf50 0x7f2e3c063130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:58030/0 (socket says 172.21.15.69:58030) 2023-11-06T20:43:49.130 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e3affd700 1 -- 172.21.15.69:0/664858555 learned_addr learned my addr 172.21.15.69:0/664858555 (peer_addr_for_me v1:172.21.15.69:0/0) 2023-11-06T20:43:49.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e3bfff700 1 --2- 172.21.15.69:0/664858555 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c063840 0x7f2e3c0c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e3b7fe700 1 --2- 172.21.15.69:0/664858555 >> v2:172.21.15.33:3300/0 conn(0x7f2e3c157700 0x7f2e3c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e3bfff700 1 -- 172.21.15.69:0/664858555 >> v2:172.21.15.33:3300/0 conn(0x7f2e3c157700 msgr2=0x7f2e3c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:49.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e3a7fc700 1 -- 172.21.15.69:0/664858555 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216405449 0 0) 0x7f2e3c158c00 con 0x7f2e3c0aaf50 2023-11-06T20:43:49.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e3bfff700 1 --2- 172.21.15.69:0/664858555 >> v2:172.21.15.33:3300/0 conn(0x7f2e3c157700 0x7f2e3c0c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e3bfff700 1 -- 172.21.15.69:0/664858555 >> v1:172.21.15.69:6789/0 conn(0x7f2e3c0aaf50 legacy=0x7f2e3c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.128+0000 7f2e3bfff700 1 -- 172.21.15.69:0/664858555 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c1582d0 con 0x7f2e3c063840 2023-11-06T20:43:49.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e3bfff700 1 --2- 172.21.15.69:0/664858555 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c063840 0x7f2e3c0c5d70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f2e34009e60 tx=0x7f2e34009780).ready entity=mon.2 client_cookie=6b871aa237b04f5e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:49.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e3a7fc700 1 -- 172.21.15.69:0/664858555 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400e070 con 0x7f2e3c063840 2023-11-06T20:43:49.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e3a7fc700 1 -- 172.21.15.69:0/664858555 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3400bb60 con 0x7f2e3c063840 2023-11-06T20:43:49.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e3a7fc700 1 -- 172.21.15.69:0/664858555 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34012410 con 0x7f2e3c063840 2023-11-06T20:43:49.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e421ff700 1 -- 172.21.15.69:0/664858555 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c063840 msgr2=0x7f2e3c0c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e421ff700 1 --2- 172.21.15.69:0/664858555 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c063840 0x7f2e3c0c5d70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f2e34009e60 tx=0x7f2e34009780).stop 2023-11-06T20:43:49.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e421ff700 1 -- 172.21.15.69:0/664858555 shutdown_connections 2023-11-06T20:43:49.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e421ff700 1 --2- 172.21.15.69:0/664858555 >> v2:172.21.15.33:3300/0 conn(0x7f2e3c157700 0x7f2e3c0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e421ff700 1 --2- 172.21.15.69:0/664858555 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c063840 0x7f2e3c0c5d70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e421ff700 1 -- 172.21.15.69:0/664858555 >> 172.21.15.69:0/664858555 conn(0x7f2e3c05ec80 msgr2=0x7f2e3c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:49.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.129+0000 7f2e421ff700 1 -- 172.21.15.69:0/664858555 shutdown_connections 2023-11-06T20:43:49.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.130+0000 7f2e421ff700 1 -- 172.21.15.69:0/664858555 wait complete. 2023-11-06T20:43:49.137 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.131+0000 7f2e421ff700 1 Processor -- start 2023-11-06T20:43:49.137 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.131+0000 7f2e421ff700 1 -- start start 2023-11-06T20:43:49.137 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e421ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e3c063840 0x7f2e3c13c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.137 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e421ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e3c0aaf50 0x7f2e3c0afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.137 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e421ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c157700 0x7f2e3c0b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.138 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e421ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0c3030 con 0x7f2e3c0aaf50 2023-11-06T20:43:49.138 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e421ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0c2460 con 0x7f2e3c063840 2023-11-06T20:43:49.138 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e421ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0c2700 con 0x7f2e3c157700 2023-11-06T20:43:49.138 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e3b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e3c063840 0x7f2e3c13c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.138 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e3bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c157700 0x7f2e3c0b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.139 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e3c0aaf50 0x7f2e3c0afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.139 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e3bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c157700 0x7f2e3c0b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.69:59904/0 (socket says 172.21.15.69:59904) 2023-11-06T20:43:49.139 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.132+0000 7f2e3bfff700 1 -- 172.21.15.69:0/3040319246 learned_addr learned my addr 172.21.15.69:0/3040319246 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:49.139 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e3bfff700 1 -- 172.21.15.69:0/3040319246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e3c063840 msgr2=0x7f2e3c13c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.139 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e3bfff700 1 --2- 172.21.15.69:0/3040319246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e3c063840 0x7f2e3c13c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e3bfff700 1 -- 172.21.15.69:0/3040319246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e3c0aaf50 msgr2=0x7f2e3c0afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e3bfff700 1 --2- 172.21.15.69:0/3040319246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e3c0aaf50 0x7f2e3c0afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e3bfff700 1 -- 172.21.15.69:0/3040319246 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e34009a20 con 0x7f2e3c157700 2023-11-06T20:43:49.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e3b7fe700 1 --2- 172.21.15.69:0/3040319246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e3c063840 0x7f2e3c13c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:49.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e3bfff700 1 --2- 172.21.15.69:0/3040319246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c157700 0x7f2e3c0b00d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f2e3400b6c0 tx=0x7f2e34009780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:49.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e237fe700 1 -- 172.21.15.69:0/3040319246 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34012410 con 0x7f2e3c157700 2023-11-06T20:43:49.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e237fe700 1 -- 172.21.15.69:0/3040319246 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3401d630 con 0x7f2e3c157700 2023-11-06T20:43:49.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e421ff700 1 -- 172.21.15.69:0/3040319246 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c0c29e0 con 0x7f2e3c157700 2023-11-06T20:43:49.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e237fe700 1 -- 172.21.15.69:0/3040319246 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3401d9f0 con 0x7f2e3c157700 2023-11-06T20:43:49.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.133+0000 7f2e421ff700 1 -- 172.21.15.69:0/3040319246 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c0b05c0 con 0x7f2e3c157700 2023-11-06T20:43:49.142 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.134+0000 7f2e237fe700 1 -- 172.21.15.69:0/3040319246 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f2e34003040 con 0x7f2e3c157700 2023-11-06T20:43:49.142 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.134+0000 7f2e237fe700 1 -- 172.21.15.69:0/3040319246 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f2e34026410 con 0x7f2e3c157700 2023-11-06T20:43:49.142 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.134+0000 7f2e421ff700 1 -- 172.21.15.69:0/3040319246 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3c0c31a0 con 0x7f2e3c157700 2023-11-06T20:43:49.142 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.137+0000 7f2e237fe700 1 -- 172.21.15.69:0/3040319246 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f2e34017070 con 0x7f2e3c157700 2023-11-06T20:43:49.221 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.222+0000 7f2e421ff700 1 -- 172.21.15.69:0/3040319246 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd new", "uuid": "64d3860c-4915-4ff6-b485-ad289ed7126c", "id": 3} v 0) v1 -- 0x7f2e3c1585b0 con 0x7f2e3c157700 2023-11-06T20:43:49.229 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.230+0000 7f2e237fe700 1 -- 172.21.15.69:0/3040319246 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64d3860c-4915-4ff6-b485-ad289ed7126c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2e340269e0 con 0x7f2e3c157700 2023-11-06T20:43:49.230 INFO:teuthology.orchestra.run.smithi069.stdout:3 2023-11-06T20:43:49.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.231+0000 7f2e421ff700 1 -- 172.21.15.69:0/3040319246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c157700 msgr2=0x7f2e3c0b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.231+0000 7f2e421ff700 1 --2- 172.21.15.69:0/3040319246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c157700 0x7f2e3c0b00d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f2e3400b6c0 tx=0x7f2e34009780).stop 2023-11-06T20:43:49.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.231+0000 7f2e421ff700 1 -- 172.21.15.69:0/3040319246 shutdown_connections 2023-11-06T20:43:49.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.231+0000 7f2e421ff700 1 --2- 172.21.15.69:0/3040319246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e3c157700 0x7f2e3c0b00d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.231+0000 7f2e421ff700 1 --2- 172.21.15.69:0/3040319246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e3c063840 0x7f2e3c13c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.231+0000 7f2e421ff700 1 --2- 172.21.15.69:0/3040319246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e3c0aaf50 0x7f2e3c0afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.231+0000 7f2e421ff700 1 -- 172.21.15.69:0/3040319246 >> 172.21.15.69:0/3040319246 conn(0x7f2e3c05ec80 msgr2=0x7f2e3c150c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:49.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.231+0000 7f2e421ff700 1 -- 172.21.15.69:0/3040319246 shutdown_connections 2023-11-06T20:43:49.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.231+0000 7f2e421ff700 1 -- 172.21.15.69:0/3040319246 wait complete. 2023-11-06T20:43:49.248 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd new 6abb58b5-ff91-4d81-90c2-6d27605dcdd3 4 2023-11-06T20:43:49.436 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.436+0000 7f02bd1ff700 1 Processor -- start 2023-11-06T20:43:49.437 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.437+0000 7f02bd1ff700 1 -- start start 2023-11-06T20:43:49.438 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.438+0000 7f02bd1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f02b80aaf50 0x7f02b8063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.438 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.438+0000 7f02bd1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b80aa4f0 0x7f02b8063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.438 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.438+0000 7f02bd1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b8158c00 con 0x7f02b8157700 2023-11-06T20:43:49.438 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.438+0000 7f02bd1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b80c18f0 con 0x7f02b80aaf50 2023-11-06T20:43:49.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.438+0000 7f02bd1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8063b10 con 0x7f02b80aa4f0 2023-11-06T20:43:49.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.438+0000 7f02b759e700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b80aa4f0 0x7f02b8063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.438+0000 7f02b759e700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b80aa4f0 0x7f02b8063620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.69:59916/0 (socket says 172.21.15.69:59916) 2023-11-06T20:43:49.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.438+0000 7f02b759e700 1 -- 172.21.15.69:0/3572348115 learned_addr learned my addr 172.21.15.69:0/3572348115 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:49.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.438+0000 7f02b659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f02b80aaf50 0x7f02b8063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.441 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.439+0000 7f02b5d9b700 1 -- 172.21.15.69:0/3572348115 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288648820 0 0) 0x7f02b8158c00 con 0x7f02b8157700 2023-11-06T20:43:49.441 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.439+0000 7f02b5d9b700 1 -- 172.21.15.69:0/3572348115 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a00033f0 con 0x7f02b8157700 2023-11-06T20:43:49.442 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.439+0000 7f02b759e700 1 -- 172.21.15.69:0/3572348115 >> v2:172.21.15.33:3300/0 conn(0x7f02b80aaf50 msgr2=0x7f02b8063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:49.442 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.439+0000 7f02b759e700 1 --2- 172.21.15.69:0/3572348115 >> v2:172.21.15.33:3300/0 conn(0x7f02b80aaf50 0x7f02b8063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.442 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.439+0000 7f02b759e700 1 -- 172.21.15.69:0/3572348115 >> v1:172.21.15.33:6789/0 conn(0x7f02b8157700 legacy=0x7f02b80c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.442 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.439+0000 7f02b759e700 1 -- 172.21.15.69:0/3572348115 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b80c7610 con 0x7f02b80aa4f0 2023-11-06T20:43:49.443 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.439+0000 7f02b759e700 1 --2- 172.21.15.69:0/3572348115 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b80aa4f0 0x7f02b8063620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f02b0009f20 tx=0x7f02b0009840).ready entity=mon.2 client_cookie=81eb3bcf6d9c87a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:49.443 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.439+0000 7f02b5d9b700 1 -- 172.21.15.69:0/3572348115 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000e070 con 0x7f02b80aa4f0 2023-11-06T20:43:49.443 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.439+0000 7f02b5d9b700 1 -- 172.21.15.69:0/3572348115 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b000b970 con 0x7f02b80aa4f0 2023-11-06T20:43:49.443 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.439+0000 7f02b5d9b700 1 -- 172.21.15.69:0/3572348115 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000bcf0 con 0x7f02b80aa4f0 2023-11-06T20:43:49.443 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.440+0000 7f02bd1ff700 1 -- 172.21.15.69:0/3572348115 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b80aa4f0 msgr2=0x7f02b8063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.440+0000 7f02bd1ff700 1 --2- 172.21.15.69:0/3572348115 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b80aa4f0 0x7f02b8063620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f02b0009f20 tx=0x7f02b0009840).stop 2023-11-06T20:43:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.440+0000 7f02bd1ff700 1 -- 172.21.15.69:0/3572348115 shutdown_connections 2023-11-06T20:43:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.440+0000 7f02bd1ff700 1 --2- 172.21.15.69:0/3572348115 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b80aa4f0 0x7f02b8063620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.440+0000 7f02bd1ff700 1 --2- 172.21.15.69:0/3572348115 >> v2:172.21.15.33:3300/0 conn(0x7f02b80aaf50 0x7f02b8063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.440+0000 7f02bd1ff700 1 -- 172.21.15.69:0/3572348115 >> 172.21.15.69:0/3572348115 conn(0x7f02b805ec80 msgr2=0x7f02b80cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:49.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.440+0000 7f02bd1ff700 1 -- 172.21.15.69:0/3572348115 shutdown_connections 2023-11-06T20:43:49.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.440+0000 7f02bd1ff700 1 -- 172.21.15.69:0/3572348115 wait complete. 2023-11-06T20:43:49.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.441+0000 7f02bd1ff700 1 Processor -- start 2023-11-06T20:43:49.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.442+0000 7f02bd1ff700 1 -- start start 2023-11-06T20:43:49.448 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02bd1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02b80aa4f0 0x7f02b814e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.448 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02bd1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b80aaf50 0x7f02b8151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.449 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02bd1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b8157700 0x7f02b814dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.449 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02bd1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b80c3a00 con 0x7f02b80aaf50 2023-11-06T20:43:49.449 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02bd1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8154270 con 0x7f02b80aa4f0 2023-11-06T20:43:49.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02bd1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8154550 con 0x7f02b8157700 2023-11-06T20:43:49.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b6d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02b80aa4f0 0x7f02b814e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b759e700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b8157700 0x7f02b814dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b6d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02b80aa4f0 0x7f02b814e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33280/0 (socket says 172.21.15.69:33280) 2023-11-06T20:43:49.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b6d9d700 1 -- 172.21.15.69:0/1871191392 learned_addr learned my addr 172.21.15.69:0/1871191392 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:49.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b659c700 1 --2- 172.21.15.69:0/1871191392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b80aaf50 0x7f02b8151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b759e700 1 -- 172.21.15.69:0/1871191392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02b80aa4f0 msgr2=0x7f02b814e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b759e700 1 --2- 172.21.15.69:0/1871191392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02b80aa4f0 0x7f02b814e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b759e700 1 -- 172.21.15.69:0/1871191392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b80aaf50 msgr2=0x7f02b8151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b759e700 1 --2- 172.21.15.69:0/1871191392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b80aaf50 0x7f02b8151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b759e700 1 -- 172.21.15.69:0/1871191392 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b0009ae0 con 0x7f02b8157700 2023-11-06T20:43:49.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.443+0000 7f02b6d9d700 1 --2- 172.21.15.69:0/1871191392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02b80aa4f0 0x7f02b814e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:49.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.444+0000 7f02b759e700 1 --2- 172.21.15.69:0/1871191392 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b8157700 0x7f02b814dd20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f02b0013000 tx=0x7f02b0009840).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:49.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.444+0000 7f029effd700 1 -- 172.21.15.69:0/1871191392 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000bee0 con 0x7f02b8157700 2023-11-06T20:43:49.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.444+0000 7f029effd700 1 -- 172.21.15.69:0/1871191392 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b000bcf0 con 0x7f02b8157700 2023-11-06T20:43:49.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.444+0000 7f02bd1ff700 1 -- 172.21.15.69:0/1871191392 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b8154830 con 0x7f02b8157700 2023-11-06T20:43:49.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.444+0000 7f029effd700 1 -- 172.21.15.69:0/1871191392 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b001d6b0 con 0x7f02b8157700 2023-11-06T20:43:49.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.444+0000 7f02bd1ff700 1 -- 172.21.15.69:0/1871191392 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b814e210 con 0x7f02b8157700 2023-11-06T20:43:49.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.444+0000 7f029effd700 1 -- 172.21.15.69:0/1871191392 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f02b0003040 con 0x7f02b8157700 2023-11-06T20:43:49.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.444+0000 7f029effd700 1 -- 172.21.15.69:0/1871191392 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f02b001dd00 con 0x7f02b8157700 2023-11-06T20:43:49.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.445+0000 7f02bd1ff700 1 -- 172.21.15.69:0/1871191392 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b80c3b70 con 0x7f02b8157700 2023-11-06T20:43:49.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.448+0000 7f029effd700 1 -- 172.21.15.69:0/1871191392 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f02b002b060 con 0x7f02b8157700 2023-11-06T20:43:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.529+0000 7f02bd1ff700 1 -- 172.21.15.69:0/1871191392 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6abb58b5-ff91-4d81-90c2-6d27605dcdd3", "id": 4} v 0) v1 -- 0x7f02b806c6c0 con 0x7f02b8157700 2023-11-06T20:43:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.537+0000 7f029effd700 1 -- 172.21.15.69:0/1871191392 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6abb58b5-ff91-4d81-90c2-6d27605dcdd3", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f02b0022070 con 0x7f02b8157700 2023-11-06T20:43:49.537 INFO:teuthology.orchestra.run.smithi069.stdout:4 2023-11-06T20:43:49.538 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.538+0000 7f02bd1ff700 1 -- 172.21.15.69:0/1871191392 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b8157700 msgr2=0x7f02b814dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.538 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.538+0000 7f02bd1ff700 1 --2- 172.21.15.69:0/1871191392 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b8157700 0x7f02b814dd20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f02b0013000 tx=0x7f02b0009840).stop 2023-11-06T20:43:49.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.538+0000 7f02bd1ff700 1 -- 172.21.15.69:0/1871191392 shutdown_connections 2023-11-06T20:43:49.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.538+0000 7f02bd1ff700 1 --2- 172.21.15.69:0/1871191392 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02b8157700 0x7f02b814dd20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.538+0000 7f02bd1ff700 1 --2- 172.21.15.69:0/1871191392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02b80aa4f0 0x7f02b814e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.538+0000 7f02bd1ff700 1 --2- 172.21.15.69:0/1871191392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b80aaf50 0x7f02b8151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.538+0000 7f02bd1ff700 1 -- 172.21.15.69:0/1871191392 >> 172.21.15.69:0/1871191392 conn(0x7f02b805ec80 msgr2=0x7f02b80ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:49.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.538+0000 7f02bd1ff700 1 -- 172.21.15.69:0/1871191392 shutdown_connections 2023-11-06T20:43:49.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.538+0000 7f02bd1ff700 1 -- 172.21.15.69:0/1871191392 wait complete. 2023-11-06T20:43:49.554 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd new 9291f261-440c-40b8-8ae2-45fbec712e20 5 2023-11-06T20:43:49.603 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:49.604+0000 7fa46b635680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:43:49.642 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:49.642+0000 7f0052f61680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:43:49.733 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:49.733+0000 7fa46b635680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:43:49.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.740+0000 7fa2c93ff700 1 Processor -- start 2023-11-06T20:43:49.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.741+0000 7fa2c93ff700 1 -- start start 2023-11-06T20:43:49.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.742+0000 7fa2c93ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 0x7fa2c4158780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.742+0000 7fa2c93ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c40a8330 con 0x7fa2c40c5110 2023-11-06T20:43:49.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.742+0000 7fa2c93ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c4158c70 con 0x7fa2c4066620 2023-11-06T20:43:49.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.742+0000 7fa2c93ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40be210 con 0x7fa2c4066a00 2023-11-06T20:43:49.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.742+0000 7fa2c37fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 0x7fa2c4158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.742+0000 7fa2c2ffd700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fa2c4066620 0x7fa2c4067ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:58036/0 (socket says 172.21.15.69:58036) 2023-11-06T20:43:49.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.742+0000 7fa2c2ffd700 1 -- 172.21.15.69:0/3054187979 learned_addr learned my addr 172.21.15.69:0/3054187979 (peer_addr_for_me v1:172.21.15.69:0/0) 2023-11-06T20:43:49.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.743+0000 7fa2c37fe700 1 -- 172.21.15.69:0/3054187979 >> v1:172.21.15.69:6789/0 conn(0x7fa2c4066620 legacy=0x7fa2c4067ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.743+0000 7fa2c37fe700 1 -- 172.21.15.69:0/3054187979 >> v1:172.21.15.33:6789/0 conn(0x7fa2c40c5110 legacy=0x7fa2c40681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.743+0000 7fa2c1ffb700 1 -- 172.21.15.69:0/3054187979 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064206888 0 0) 0x7fa2c4158c70 con 0x7fa2c4066620 2023-11-06T20:43:49.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.743+0000 7fa2c37fe700 1 -- 172.21.15.69:0/3054187979 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40bdc50 con 0x7fa2c4066a00 2023-11-06T20:43:49.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.743+0000 7fa2c37fe700 1 --2- 172.21.15.69:0/3054187979 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 0x7fa2c4158780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fa2bc009e90 tx=0x7fa2bc009780).ready entity=mon.2 client_cookie=ce1bfa986555dd7d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:49.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.743+0000 7fa2c1ffb700 1 -- 172.21.15.69:0/3054187979 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00e070 con 0x7fa2c4066a00 2023-11-06T20:43:49.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.743+0000 7fa2c1ffb700 1 -- 172.21.15.69:0/3054187979 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc00bb30 con 0x7fa2c4066a00 2023-11-06T20:43:49.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.743+0000 7fa2c1ffb700 1 -- 172.21.15.69:0/3054187979 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00beb0 con 0x7fa2c4066a00 2023-11-06T20:43:49.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.744+0000 7fa2c93ff700 1 -- 172.21.15.69:0/3054187979 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 msgr2=0x7fa2c4158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.744+0000 7fa2c93ff700 1 --2- 172.21.15.69:0/3054187979 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 0x7fa2c4158780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fa2bc009e90 tx=0x7fa2bc009780).stop 2023-11-06T20:43:49.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.744+0000 7fa2c93ff700 1 -- 172.21.15.69:0/3054187979 shutdown_connections 2023-11-06T20:43:49.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.744+0000 7fa2c93ff700 1 --2- 172.21.15.69:0/3054187979 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 0x7fa2c4158780 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.744+0000 7fa2c93ff700 1 -- 172.21.15.69:0/3054187979 >> 172.21.15.69:0/3054187979 conn(0x7fa2c40c8c50 msgr2=0x7fa2c40c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:49.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.744+0000 7fa2c93ff700 1 -- 172.21.15.69:0/3054187979 shutdown_connections 2023-11-06T20:43:49.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.744+0000 7fa2c93ff700 1 -- 172.21.15.69:0/3054187979 wait complete. 2023-11-06T20:43:49.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.745+0000 7fa2c93ff700 1 Processor -- start 2023-11-06T20:43:49.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.745+0000 7fa2c93ff700 1 -- start start 2023-11-06T20:43:49.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.746+0000 7fa2c93ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c4066620 0x7fa2c4158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.746+0000 7fa2c93ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 0x7fa2c40a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.746+0000 7fa2c93ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2c40c5110 0x7fa2c40abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:49.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.746+0000 7fa2c93ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4066e60 con 0x7fa2c40c5110 2023-11-06T20:43:49.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.746+0000 7fa2c93ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40aec90 con 0x7fa2c4066620 2023-11-06T20:43:49.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.746+0000 7fa2c93ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40ab270 con 0x7fa2c4066a00 2023-11-06T20:43:49.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c4066620 0x7fa2c4158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c4066620 0x7fa2c4158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33296/0 (socket says 172.21.15.69:33296) 2023-11-06T20:43:49.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c2ffd700 1 -- 172.21.15.69:0/1250721426 learned_addr learned my addr 172.21.15.69:0/1250721426 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:49.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c27fc700 1 --2- 172.21.15.69:0/1250721426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 0x7fa2c40a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c37fe700 1 --2- 172.21.15.69:0/1250721426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2c40c5110 0x7fa2c40abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:49.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c2ffd700 1 -- 172.21.15.69:0/1250721426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 msgr2=0x7fa2c40a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c2ffd700 1 --2- 172.21.15.69:0/1250721426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 0x7fa2c40a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c2ffd700 1 -- 172.21.15.69:0/1250721426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2c40c5110 msgr2=0x7fa2c40abbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c2ffd700 1 --2- 172.21.15.69:0/1250721426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2c40c5110 0x7fa2c40abbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c2ffd700 1 -- 172.21.15.69:0/1250721426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc009a20 con 0x7fa2c4066620 2023-11-06T20:43:49.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c27fc700 1 --2- 172.21.15.69:0/1250721426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 0x7fa2c40a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:49.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2c2ffd700 1 --2- 172.21.15.69:0/1250721426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c4066620 0x7fa2c4158140 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fa2b400c4c0 tx=0x7fa2b4007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:49.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.747+0000 7fa2aaffd700 1 -- 172.21.15.69:0/1250721426 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400ce30 con 0x7fa2c4066620 2023-11-06T20:43:49.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.748+0000 7fa2aaffd700 1 -- 172.21.15.69:0/1250721426 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b40072e0 con 0x7fa2c4066620 2023-11-06T20:43:49.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.748+0000 7fa2aaffd700 1 -- 172.21.15.69:0/1250721426 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40057a0 con 0x7fa2c4066620 2023-11-06T20:43:49.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.748+0000 7fa2c93ff700 1 -- 172.21.15.69:0/1250721426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c40ae6d0 con 0x7fa2c4066620 2023-11-06T20:43:49.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.748+0000 7fa2c93ff700 1 -- 172.21.15.69:0/1250721426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c40a56d0 con 0x7fa2c4066620 2023-11-06T20:43:49.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.748+0000 7fa2aaffd700 1 -- 172.21.15.69:0/1250721426 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fa2b4013450 con 0x7fa2c4066620 2023-11-06T20:43:49.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.748+0000 7fa2aaffd700 1 -- 172.21.15.69:0/1250721426 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7fa2b4005df0 con 0x7fa2c4066620 2023-11-06T20:43:49.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.749+0000 7fa2c93ff700 1 -- 172.21.15.69:0/1250721426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c40a5530 con 0x7fa2c4066620 2023-11-06T20:43:49.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.751+0000 7fa2aaffd700 1 -- 172.21.15.69:0/1250721426 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fa2c40a5530 con 0x7fa2c4066620 2023-11-06T20:43:49.772 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:49.772+0000 7f0052f61680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:43:49.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.834+0000 7fa2c93ff700 1 -- 172.21.15.69:0/1250721426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9291f261-440c-40b8-8ae2-45fbec712e20", "id": 5} v 0) v1 -- 0x7fa2c405f7a0 con 0x7fa2c4066620 2023-11-06T20:43:49.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.843+0000 7fa2aaffd700 1 -- 172.21.15.69:0/1250721426 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9291f261-440c-40b8-8ae2-45fbec712e20", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa2b4013700 con 0x7fa2c4066620 2023-11-06T20:43:49.843 INFO:teuthology.orchestra.run.smithi069.stdout:5 2023-11-06T20:43:49.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.844+0000 7fa2c93ff700 1 -- 172.21.15.69:0/1250721426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c4066620 msgr2=0x7fa2c4158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:49.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.844+0000 7fa2c93ff700 1 --2- 172.21.15.69:0/1250721426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c4066620 0x7fa2c4158140 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fa2b400c4c0 tx=0x7fa2b4007c10).stop 2023-11-06T20:43:49.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.844+0000 7fa2c93ff700 1 -- 172.21.15.69:0/1250721426 shutdown_connections 2023-11-06T20:43:49.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.844+0000 7fa2c93ff700 1 --2- 172.21.15.69:0/1250721426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2c40c5110 0x7fa2c40abbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.844+0000 7fa2c93ff700 1 --2- 172.21.15.69:0/1250721426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c4066620 0x7fa2c4158140 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.844+0000 7fa2c93ff700 1 --2- 172.21.15.69:0/1250721426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2c4066a00 0x7fa2c40a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.844+0000 7fa2c93ff700 1 -- 172.21.15.69:0/1250721426 >> 172.21.15.69:0/1250721426 conn(0x7fa2c40c8c50 msgr2=0x7fa2c4152b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.844+0000 7fa2c93ff700 1 -- 172.21.15.69:0/1250721426 shutdown_connections 2023-11-06T20:43:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:49.844+0000 7fa2c93ff700 1 -- 172.21.15.69:0/1250721426 wait complete. 2023-11-06T20:43:49.861 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd new f37d1d81-3a4e-41bc-9b26-99744a422362 6 2023-11-06T20:43:49.930 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:49.930+0000 7fa46b635680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:43:49.963 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:49.963+0000 7f0052f61680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:43:50.031 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:50.032+0000 7fa46b635680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:43:50.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.049+0000 7ff37f7ff700 1 Processor -- start 2023-11-06T20:43:50.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.049+0000 7ff37f7ff700 1 -- start start 2023-11-06T20:43:50.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.050+0000 7ff37f7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff378063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.050+0000 7ff37f7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff378158c00 con 0x7ff378157700 2023-11-06T20:43:50.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.050+0000 7ff37f7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3780c7610 con 0x7ff3780aa4f0 2023-11-06T20:43:50.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.050+0000 7ff37f7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780c18f0 con 0x7ff3780aaf50 2023-11-06T20:43:50.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.050+0000 7ff37cd9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff378063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.051+0000 7ff37dd9c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7ff3780aa4f0 0x7ff3780c5d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:58038/0 (socket says 172.21.15.69:58038) 2023-11-06T20:43:50.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.051+0000 7ff37dd9c700 1 -- 172.21.15.69:0/39956825 learned_addr learned my addr 172.21.15.69:0/39956825 (peer_addr_for_me v1:172.21.15.69:0/0) 2023-11-06T20:43:50.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.051+0000 7ff36ffff700 1 -- 172.21.15.69:0/39956825 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836634709 0 0) 0x7ff3780c7610 con 0x7ff3780aa4f0 2023-11-06T20:43:50.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.051+0000 7ff37cd9a700 1 -- 172.21.15.69:0/39956825 >> v1:172.21.15.69:6789/0 conn(0x7ff3780aa4f0 legacy=0x7ff3780c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.051+0000 7ff37cd9a700 1 -- 172.21.15.69:0/39956825 >> v1:172.21.15.33:6789/0 conn(0x7ff378157700 legacy=0x7ff3780c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.051+0000 7ff37cd9a700 1 -- 172.21.15.69:0/39956825 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378157ff0 con 0x7ff3780aaf50 2023-11-06T20:43:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.051+0000 7ff37cd9a700 1 --2- 172.21.15.69:0/39956825 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff378063130 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7ff368009790 tx=0x7ff368009060).ready entity=mon.2 client_cookie=2c6f3e8245254874 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.051+0000 7ff36ffff700 1 -- 172.21.15.69:0/39956825 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36800f510 con 0x7ff3780aaf50 2023-11-06T20:43:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.051+0000 7ff36ffff700 1 -- 172.21.15.69:0/39956825 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36800f670 con 0x7ff3780aaf50 2023-11-06T20:43:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.052+0000 7ff36ffff700 1 -- 172.21.15.69:0/39956825 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36800f9f0 con 0x7ff3780aaf50 2023-11-06T20:43:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.052+0000 7ff37f7ff700 1 -- 172.21.15.69:0/39956825 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 msgr2=0x7ff378063130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.052+0000 7ff37f7ff700 1 --2- 172.21.15.69:0/39956825 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff378063130 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7ff368009790 tx=0x7ff368009060).stop 2023-11-06T20:43:50.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.052+0000 7ff37f7ff700 1 -- 172.21.15.69:0/39956825 shutdown_connections 2023-11-06T20:43:50.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.052+0000 7ff37f7ff700 1 --2- 172.21.15.69:0/39956825 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff378063130 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.052+0000 7ff37f7ff700 1 -- 172.21.15.69:0/39956825 >> 172.21.15.69:0/39956825 conn(0x7ff37805ec80 msgr2=0x7ff3780cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:50.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.052+0000 7ff37f7ff700 1 -- 172.21.15.69:0/39956825 shutdown_connections 2023-11-06T20:43:50.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.052+0000 7ff37f7ff700 1 -- 172.21.15.69:0/39956825 wait complete. 2023-11-06T20:43:50.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.054+0000 7ff37f7ff700 1 Processor -- start 2023-11-06T20:43:50.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.054+0000 7ff37f7ff700 1 -- start start 2023-11-06T20:43:50.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37f7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3780aa4f0 0x7ff37813f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37f7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff3780c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37f7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff378157700 0x7ff378153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37f7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780c4440 con 0x7ff3780aa4f0 2023-11-06T20:43:50.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37f7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780c3830 con 0x7ff378157700 2023-11-06T20:43:50.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37f7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780c3b10 con 0x7ff3780aaf50 2023-11-06T20:43:50.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37cd9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff3780c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37dd9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff378157700 0x7ff378153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3780aa4f0 0x7ff37813f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37cd9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff3780c2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.69:59962/0 (socket says 172.21.15.69:59962) 2023-11-06T20:43:50.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37cd9a700 1 -- 172.21.15.69:0/2350170954 learned_addr learned my addr 172.21.15.69:0/2350170954 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:50.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.055+0000 7ff37d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3780aa4f0 0x7ff37813f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.69:39996/0 (socket says 172.21.15.69:39996) 2023-11-06T20:43:50.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff37dd9c700 1 -- 172.21.15.69:0/2350170954 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 msgr2=0x7ff3780c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff37dd9c700 1 --2- 172.21.15.69:0/2350170954 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff3780c2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff37dd9c700 1 -- 172.21.15.69:0/2350170954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3780aa4f0 msgr2=0x7ff37813f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff37dd9c700 1 --2- 172.21.15.69:0/2350170954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3780aa4f0 0x7ff37813f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff37dd9c700 1 -- 172.21.15.69:0/2350170954 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3680092a0 con 0x7ff378157700 2023-11-06T20:43:50.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff37cd9a700 1 --2- 172.21.15.69:0/2350170954 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff3780c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:50.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff37dd9c700 1 --2- 172.21.15.69:0/2350170954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff378157700 0x7ff378153230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7ff378064050 tx=0x7ff370007f90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:50.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff36d7fa700 1 -- 172.21.15.69:0/2350170954 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370005d30 con 0x7ff378157700 2023-11-06T20:43:50.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff36d7fa700 1 -- 172.21.15.69:0/2350170954 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff370005e90 con 0x7ff378157700 2023-11-06T20:43:50.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff37f7ff700 1 -- 172.21.15.69:0/2350170954 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3780c3df0 con 0x7ff378157700 2023-11-06T20:43:50.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff36d7fa700 1 -- 172.21.15.69:0/2350170954 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370007300 con 0x7ff378157700 2023-11-06T20:43:50.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff37f7ff700 1 -- 172.21.15.69:0/2350170954 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37813f8a0 con 0x7ff378157700 2023-11-06T20:43:50.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff36d7fa700 1 -- 172.21.15.69:0/2350170954 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7ff370013450 con 0x7ff378157700 2023-11-06T20:43:50.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.056+0000 7ff36d7fa700 1 -- 172.21.15.69:0/2350170954 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7ff370007950 con 0x7ff378157700 2023-11-06T20:43:50.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.057+0000 7ff37f7ff700 1 -- 172.21.15.69:0/2350170954 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3780c45b0 con 0x7ff378157700 2023-11-06T20:43:50.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.060+0000 7ff36d7fa700 1 -- 172.21.15.69:0/2350170954 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7ff370034e50 con 0x7ff378157700 2023-11-06T20:43:50.065 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:50.065+0000 7f0052f61680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:43:50.124 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:50.125+0000 7fa46b635680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:43:50.143 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.143+0000 7ff37f7ff700 1 -- 172.21.15.69:0/2350170954 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f37d1d81-3a4e-41bc-9b26-99744a422362", "id": 6} v 0) v1 -- 0x7ff37806c6c0 con 0x7ff378157700 2023-11-06T20:43:50.151 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.151+0000 7ff36d7fa700 1 -- 172.21.15.69:0/2350170954 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f37d1d81-3a4e-41bc-9b26-99744a422362", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff370023070 con 0x7ff378157700 2023-11-06T20:43:50.152 INFO:teuthology.orchestra.run.smithi069.stdout:6 2023-11-06T20:43:50.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.152+0000 7ff37f7ff700 1 -- 172.21.15.69:0/2350170954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff378157700 msgr2=0x7ff378153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.152+0000 7ff37f7ff700 1 --2- 172.21.15.69:0/2350170954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff378157700 0x7ff378153230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7ff378064050 tx=0x7ff370007f90).stop 2023-11-06T20:43:50.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.152+0000 7ff37f7ff700 1 -- 172.21.15.69:0/2350170954 shutdown_connections 2023-11-06T20:43:50.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.153+0000 7ff37f7ff700 1 --2- 172.21.15.69:0/2350170954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff378157700 0x7ff378153230 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.153+0000 7ff37f7ff700 1 --2- 172.21.15.69:0/2350170954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3780aa4f0 0x7ff37813f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.154 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.153+0000 7ff37f7ff700 1 --2- 172.21.15.69:0/2350170954 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3780aaf50 0x7ff3780c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.154 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.153+0000 7ff37f7ff700 1 -- 172.21.15.69:0/2350170954 >> 172.21.15.69:0/2350170954 conn(0x7ff37805ec80 msgr2=0x7ff3781513b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:50.154 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.153+0000 7ff37f7ff700 1 -- 172.21.15.69:0/2350170954 shutdown_connections 2023-11-06T20:43:50.154 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.153+0000 7ff37f7ff700 1 -- 172.21.15.69:0/2350170954 wait complete. 2023-11-06T20:43:50.157 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:50.157+0000 7f0052f61680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:43:50.169 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd new 6f9b4dbb-2660-4430-8ad2-003531697dec 7 2023-11-06T20:43:50.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.353+0000 7fea2b9ff700 1 Processor -- start 2023-11-06T20:43:50.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.354+0000 7fea2b9ff700 1 -- start start 2023-11-06T20:43:50.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.355+0000 7fea2b9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fea24157700 0x7fea240c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.355+0000 7fea2b9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea24158c00 con 0x7fea24063840 2023-11-06T20:43:50.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.355+0000 7fea2b9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea24157ff0 con 0x7fea240aaf50 2023-11-06T20:43:50.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.355+0000 7fea2b9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea240c18f0 con 0x7fea24157700 2023-11-06T20:43:50.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.355+0000 7fea2979b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fea24157700 0x7fea240c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.355+0000 7fea2979b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fea24157700 0x7fea240c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33316/0 (socket says 172.21.15.69:33316) 2023-11-06T20:43:50.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.355+0000 7fea28f9a700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fea240aaf50 0x7fea24063130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:58054/0 (socket says 172.21.15.69:58054) 2023-11-06T20:43:50.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.355+0000 7fea2979b700 1 -- 172.21.15.69:0/1411169913 learned_addr learned my addr 172.21.15.69:0/1411169913 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:50.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.355+0000 7fea2979b700 1 -- 172.21.15.69:0/1411169913 >> v1:172.21.15.69:6789/0 conn(0x7fea240aaf50 legacy=0x7fea24063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.356+0000 7fea2979b700 1 -- 172.21.15.69:0/1411169913 >> v1:172.21.15.33:6789/0 conn(0x7fea24063840 legacy=0x7fea240c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.356+0000 7fea2979b700 1 -- 172.21.15.69:0/1411169913 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea241582d0 con 0x7fea24157700 2023-11-06T20:43:50.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.356+0000 7fea2979b700 1 --2- 172.21.15.69:0/1411169913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 0x7fea240c6f00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fea20006fd0 tx=0x7fea200090f0).ready entity=mon.1 client_cookie=f17fd04336884638 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:50.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.356+0000 7fea1bfff700 1 -- 172.21.15.69:0/1411169913 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2000b600 con 0x7fea24157700 2023-11-06T20:43:50.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.356+0000 7fea1bfff700 1 -- 172.21.15.69:0/1411169913 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2000b760 con 0x7fea24157700 2023-11-06T20:43:50.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.356+0000 7fea1bfff700 1 -- 172.21.15.69:0/1411169913 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2000bae0 con 0x7fea24157700 2023-11-06T20:43:50.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.356+0000 7fea2b9ff700 1 -- 172.21.15.69:0/1411169913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 msgr2=0x7fea240c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.356+0000 7fea2b9ff700 1 --2- 172.21.15.69:0/1411169913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 0x7fea240c6f00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fea20006fd0 tx=0x7fea200090f0).stop 2023-11-06T20:43:50.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.357+0000 7fea2b9ff700 1 -- 172.21.15.69:0/1411169913 shutdown_connections 2023-11-06T20:43:50.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.357+0000 7fea2b9ff700 1 --2- 172.21.15.69:0/1411169913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 0x7fea240c6f00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.357+0000 7fea2b9ff700 1 -- 172.21.15.69:0/1411169913 >> 172.21.15.69:0/1411169913 conn(0x7fea2405ec80 msgr2=0x7fea240cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:50.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.357+0000 7fea2b9ff700 1 -- 172.21.15.69:0/1411169913 shutdown_connections 2023-11-06T20:43:50.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.357+0000 7fea2b9ff700 1 -- 172.21.15.69:0/1411169913 wait complete. 2023-11-06T20:43:50.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.358+0000 7fea2b9ff700 1 Processor -- start 2023-11-06T20:43:50.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.358+0000 7fea2b9ff700 1 -- start start 2023-11-06T20:43:50.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.359+0000 7fea2b9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea24063840 0x7fea24152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.359+0000 7fea2b9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea240aaf50 0x7fea24167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.359+0000 7fea2b9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 0x7fea240c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.359+0000 7fea2b9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea24152580 con 0x7fea240aaf50 2023-11-06T20:43:50.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.359+0000 7fea2b9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea241684d0 con 0x7fea24157700 2023-11-06T20:43:50.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.359+0000 7fea2b9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea241687a0 con 0x7fea24063840 2023-11-06T20:43:50.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.359+0000 7fea29f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 0x7fea240c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea28f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea240aaf50 0x7fea24167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea2979b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea24063840 0x7fea24152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea29f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 0x7fea240c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33328/0 (socket says 172.21.15.69:33328) 2023-11-06T20:43:50.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea29f9c700 1 -- 172.21.15.69:0/586459251 learned_addr learned my addr 172.21.15.69:0/586459251 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:43:50.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea29f9c700 1 -- 172.21.15.69:0/586459251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea24063840 msgr2=0x7fea24152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea29f9c700 1 --2- 172.21.15.69:0/586459251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea24063840 0x7fea24152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea29f9c700 1 -- 172.21.15.69:0/586459251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea240aaf50 msgr2=0x7fea24167fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea29f9c700 1 --2- 172.21.15.69:0/586459251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea240aaf50 0x7fea24167fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea29f9c700 1 -- 172.21.15.69:0/586459251 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea20009370 con 0x7fea24157700 2023-11-06T20:43:50.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea2979b700 1 --2- 172.21.15.69:0/586459251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea24063840 0x7fea24152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:50.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea29f9c700 1 --2- 172.21.15.69:0/586459251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 0x7fea240c8c30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fea1c00c7c0 tx=0x7fea1c0014e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:50.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea197fa700 1 -- 172.21.15.69:0/586459251 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00e4a0 con 0x7fea24157700 2023-11-06T20:43:50.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea197fa700 1 -- 172.21.15.69:0/586459251 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c00e600 con 0x7fea24157700 2023-11-06T20:43:50.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea2b9ff700 1 -- 172.21.15.69:0/586459251 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea24168a80 con 0x7fea24157700 2023-11-06T20:43:50.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.360+0000 7fea197fa700 1 -- 172.21.15.69:0/586459251 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00e9a0 con 0x7fea24157700 2023-11-06T20:43:50.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.361+0000 7fea2b9ff700 1 -- 172.21.15.69:0/586459251 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea241533d0 con 0x7fea24157700 2023-11-06T20:43:50.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.361+0000 7fea197fa700 1 -- 172.21.15.69:0/586459251 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fea1c01b3f0 con 0x7fea24157700 2023-11-06T20:43:50.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.361+0000 7fea197fa700 1 -- 172.21.15.69:0/586459251 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7fea1c012a80 con 0x7fea24157700 2023-11-06T20:43:50.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.362+0000 7fea2b9ff700 1 -- 172.21.15.69:0/586459251 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2406c6c0 con 0x7fea24157700 2023-11-06T20:43:50.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.366+0000 7fea197fa700 1 -- 172.21.15.69:0/586459251 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fea1c020060 con 0x7fea24157700 2023-11-06T20:43:50.449 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.449+0000 7fea2b9ff700 1 -- 172.21.15.69:0/586459251 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6f9b4dbb-2660-4430-8ad2-003531697dec", "id": 7} v 0) v1 -- 0x7fea240bfe50 con 0x7fea24157700 2023-11-06T20:43:50.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.457+0000 7fea197fa700 1 -- 172.21.15.69:0/586459251 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6f9b4dbb-2660-4430-8ad2-003531697dec", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fea1c012d50 con 0x7fea24157700 2023-11-06T20:43:50.457 INFO:teuthology.orchestra.run.smithi069.stdout:7 2023-11-06T20:43:50.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.458+0000 7fea2b9ff700 1 -- 172.21.15.69:0/586459251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 msgr2=0x7fea240c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.458+0000 7fea2b9ff700 1 --2- 172.21.15.69:0/586459251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 0x7fea240c8c30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fea1c00c7c0 tx=0x7fea1c0014e0).stop 2023-11-06T20:43:50.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.458+0000 7fea2b9ff700 1 -- 172.21.15.69:0/586459251 shutdown_connections 2023-11-06T20:43:50.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.458+0000 7fea2b9ff700 1 --2- 172.21.15.69:0/586459251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea24063840 0x7fea24152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.458+0000 7fea2b9ff700 1 --2- 172.21.15.69:0/586459251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea24157700 0x7fea240c8c30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.458+0000 7fea2b9ff700 1 --2- 172.21.15.69:0/586459251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea240aaf50 0x7fea24167fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.458+0000 7fea2b9ff700 1 -- 172.21.15.69:0/586459251 >> 172.21.15.69:0/586459251 conn(0x7fea2405ec80 msgr2=0x7fea240c6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:50.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.458+0000 7fea2b9ff700 1 -- 172.21.15.69:0/586459251 shutdown_connections 2023-11-06T20:43:50.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:43:50.458+0000 7fea2b9ff700 1 -- 172.21.15.69:0/586459251 wait complete. 2023-11-06T20:43:50.465 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:50.466+0000 7fa46b635680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:43:50.475 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-06T20:43:50.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-06T20:43:50.480 INFO:tasks.ceph.osd.0:Started 2023-11-06T20:43:50.481 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-06T20:43:50.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-06T20:43:50.485 INFO:tasks.ceph.osd.1:Started 2023-11-06T20:43:50.486 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-06T20:43:50.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-06T20:43:50.490 INFO:tasks.ceph.osd.2:Started 2023-11-06T20:43:50.491 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-06T20:43:50.491 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-06T20:43:50.492 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:50.492+0000 7f0052f61680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:43:50.497 INFO:tasks.ceph.osd.3:Started 2023-11-06T20:43:50.498 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-06T20:43:50.498 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-06T20:43:50.502 INFO:tasks.ceph.osd.4:Started 2023-11-06T20:43:50.503 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-06T20:43:50.503 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-06T20:43:50.508 INFO:tasks.ceph.osd.5:Started 2023-11-06T20:43:50.508 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-06T20:43:50.508 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-06T20:43:50.514 INFO:tasks.ceph.osd.6:Started 2023-11-06T20:43:50.515 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-06T20:43:50.515 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-06T20:43:50.521 INFO:tasks.ceph.osd.7:Started 2023-11-06T20:43:50.522 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:43:50.739 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:50.739+0000 7fa46b635680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:43:50.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.783+0000 7f25eb9ff700 1 Processor -- start 2023-11-06T20:43:50.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.783+0000 7f25eb9ff700 1 -- start start 2023-11-06T20:43:50.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.784+0000 7f25eb9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f25e41544c0 0x7f25e40c1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25eb9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e40c10c0 con 0x7f25e40c1a40 2023-11-06T20:43:50.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25eb9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e40c04b0 con 0x7f25e41540e0 2023-11-06T20:43:50.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25eb9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e40c0790 con 0x7f25e41544c0 2023-11-06T20:43:50.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25e8f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f25e41544c0 0x7f25e40c1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25e9f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f25e40c1a40 0x7f25e40c5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44110/0 (socket says 172.21.15.33:44110) 2023-11-06T20:43:50.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25e9f9c700 1 -- 172.21.15.33:0/3366061109 learned_addr learned my addr 172.21.15.33:0/3366061109 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:43:50.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25e979b700 1 --1- 172.21.15.33:0/3366061109 >> v1:172.21.15.69:6789/0 conn(0x7f25e41540e0 0x7f25e40c7b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:47172/0 (socket says 172.21.15.33:47172) 2023-11-06T20:43:50.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25e8f9a700 1 -- 172.21.15.33:0/3366061109 >> v1:172.21.15.69:6789/0 conn(0x7f25e41540e0 legacy=0x7f25e40c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25e8f9a700 1 -- 172.21.15.33:0/3366061109 >> v1:172.21.15.33:6789/0 conn(0x7f25e40c1a40 legacy=0x7f25e40c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25dbfff700 1 -- 172.21.15.33:0/3366061109 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924935967 0 0) 0x7f25e40c10c0 con 0x7f25e40c1a40 2023-11-06T20:43:50.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.785+0000 7f25e8f9a700 1 -- 172.21.15.33:0/3366061109 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e40c0a70 con 0x7f25e41544c0 2023-11-06T20:43:50.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.786+0000 7f25e8f9a700 1 --2- 172.21.15.33:0/3366061109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41544c0 0x7f25e40c1550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f25d4009760 tx=0x7f25d4009060).ready entity=mon.0 client_cookie=f5846ec9bcfd325e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:50.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.786+0000 7f25dbfff700 1 -- 172.21.15.33:0/3366061109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d400b580 con 0x7f25e41544c0 2023-11-06T20:43:50.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.786+0000 7f25dbfff700 1 -- 172.21.15.33:0/3366061109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d400b6e0 con 0x7f25e41544c0 2023-11-06T20:43:50.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.786+0000 7f25dbfff700 1 -- 172.21.15.33:0/3366061109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d400ba60 con 0x7f25e41544c0 2023-11-06T20:43:50.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.786+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3366061109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41544c0 msgr2=0x7f25e40c1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.786+0000 7f25eb9ff700 1 --2- 172.21.15.33:0/3366061109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41544c0 0x7f25e40c1550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f25d4009760 tx=0x7f25d4009060).stop 2023-11-06T20:43:50.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.787+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3366061109 shutdown_connections 2023-11-06T20:43:50.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.787+0000 7f25eb9ff700 1 --2- 172.21.15.33:0/3366061109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41544c0 0x7f25e40c1550 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.787+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3366061109 >> 172.21.15.33:0/3366061109 conn(0x7f25e405e1b0 msgr2=0x7f25e405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:50.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.787+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3366061109 shutdown_connections 2023-11-06T20:43:50.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.787+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3366061109 wait complete. 2023-11-06T20:43:50.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.789+0000 7f25eb9ff700 1 Processor -- start 2023-11-06T20:43:50.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.789+0000 7f25eb9ff700 1 -- start start 2023-11-06T20:43:50.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.790+0000 7f25eb9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25e40c1a40 0x7f25e4168370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.790+0000 7f25eb9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41540e0 0x7f25e4156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.790+0000 7f25eb9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f25e41544c0 0x7f25e40ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:50.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.790+0000 7f25eb9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40c4d20 con 0x7f25e41540e0 2023-11-06T20:43:50.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.790+0000 7f25eb9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40c3c80 con 0x7f25e40c1a40 2023-11-06T20:43:50.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.790+0000 7f25eb9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40c3f60 con 0x7f25e41544c0 2023-11-06T20:43:50.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e979b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25e40c1a40 0x7f25e4168370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e8f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41540e0 0x7f25e4156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e979b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25e40c1a40 0x7f25e4168370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:58718/0 (socket says 172.21.15.33:58718) 2023-11-06T20:43:50.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e8f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41540e0 0x7f25e4156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57480/0 (socket says 172.21.15.33:57480) 2023-11-06T20:43:50.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e979b700 1 -- 172.21.15.33:0/3135403497 learned_addr learned my addr 172.21.15.33:0/3135403497 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:43:50.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e9f9c700 1 --2- 172.21.15.33:0/3135403497 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f25e41544c0 0x7f25e40ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:50.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e8f9a700 1 -- 172.21.15.33:0/3135403497 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f25e41544c0 msgr2=0x7f25e40ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e8f9a700 1 --2- 172.21.15.33:0/3135403497 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f25e41544c0 0x7f25e40ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e8f9a700 1 -- 172.21.15.33:0/3135403497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25e40c1a40 msgr2=0x7f25e4168370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e8f9a700 1 --2- 172.21.15.33:0/3135403497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25e40c1a40 0x7f25e4168370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e8f9a700 1 -- 172.21.15.33:0/3135403497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d40092a0 con 0x7f25e41540e0 2023-11-06T20:43:50.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e979b700 1 --2- 172.21.15.33:0/3135403497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25e40c1a40 0x7f25e4168370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:50.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.791+0000 7f25e8f9a700 1 --2- 172.21.15.33:0/3135403497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41540e0 0x7f25e4156200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f25d4014000 tx=0x7f25d4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:50.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.792+0000 7f25d97fa700 1 -- 172.21.15.33:0/3135403497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d400b850 con 0x7f25e41540e0 2023-11-06T20:43:50.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.792+0000 7f25d97fa700 1 -- 172.21.15.33:0/3135403497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d400b9b0 con 0x7f25e41540e0 2023-11-06T20:43:50.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.792+0000 7f25d97fa700 1 -- 172.21.15.33:0/3135403497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d401b660 con 0x7f25e41540e0 2023-11-06T20:43:50.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.792+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3135403497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e40c4240 con 0x7f25e41540e0 2023-11-06T20:43:50.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.792+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3135403497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e40c4800 con 0x7f25e41540e0 2023-11-06T20:43:50.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.792+0000 7f25d97fa700 1 -- 172.21.15.33:0/3135403497 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f25d401bcf0 con 0x7f25e41540e0 2023-11-06T20:43:50.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.792+0000 7f25d97fa700 1 -- 172.21.15.33:0/3135403497 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f25d4024d70 con 0x7f25e41540e0 2023-11-06T20:43:50.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.793+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3135403497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e406c6c0 con 0x7f25e41540e0 2023-11-06T20:43:50.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.796+0000 7f25d97fa700 1 -- 172.21.15.33:0/3135403497 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f25d4029060 con 0x7f25e41540e0 2023-11-06T20:43:50.800 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:50.798+0000 7f0052f61680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:43:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.868+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3135403497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25e4062ac0 con 0x7f25e41540e0 2023-11-06T20:43:50.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.869+0000 7f25d97fa700 1 -- 172.21.15.33:0/3135403497 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7f25d401f070 con 0x7f25e41540e0 2023-11-06T20:43:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:43:50.869 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:43:50.451599+0000","last_up_change":"0.000000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:43:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.870+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3135403497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41540e0 msgr2=0x7f25e4156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.870+0000 7f25eb9ff700 1 --2- 172.21.15.33:0/3135403497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41540e0 0x7f25e4156200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f25d4014000 tx=0x7f25d4017000).stop 2023-11-06T20:43:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.870+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3135403497 shutdown_connections 2023-11-06T20:43:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.870+0000 7f25eb9ff700 1 --2- 172.21.15.33:0/3135403497 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f25e41544c0 0x7f25e40ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.870+0000 7f25eb9ff700 1 --2- 172.21.15.33:0/3135403497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25e40c1a40 0x7f25e4168370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.870+0000 7f25eb9ff700 1 --2- 172.21.15.33:0/3135403497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25e41540e0 0x7f25e4156200 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.870+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3135403497 >> 172.21.15.33:0/3135403497 conn(0x7f25e405e1b0 msgr2=0x7f25e40aa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.870+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3135403497 shutdown_connections 2023-11-06T20:43:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:50.870+0000 7f25eb9ff700 1 -- 172.21.15.33:0/3135403497 wait complete. 2023-11-06T20:43:50.887 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-06T20:43:50.888 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-06T20:43:51.194 INFO:tasks.ceph.osd.4.smithi069.stderr:2023-11-06T20:43:51.194+0000 7f114d62a380 -1 Falling back to public interface 2023-11-06T20:43:51.195 INFO:tasks.ceph.osd.6.smithi069.stderr:2023-11-06T20:43:51.194+0000 7f353e5c2380 -1 Falling back to public interface 2023-11-06T20:43:51.195 INFO:tasks.ceph.osd.4.smithi069.stderr:2023-11-06T20:43:51.195+0000 7f11445fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-11-06T20:43:51.196 INFO:tasks.ceph.osd.6.smithi069.stderr:2023-11-06T20:43:51.195+0000 7f35353fd700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2023-11-06T20:43:51.198 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-11-06T20:43:51.199+0000 7ff7b9c8d380 -1 Falling back to public interface 2023-11-06T20:43:51.207 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-11-06T20:43:51.208+0000 7f06acd04380 -1 Falling back to public interface 2023-11-06T20:43:51.230 INFO:tasks.ceph.osd.5.smithi069.stderr:2023-11-06T20:43:51.231+0000 7ff08a406380 -1 Falling back to public interface 2023-11-06T20:43:51.231 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-11-06T20:43:51.232+0000 7f5c021f0380 -1 Falling back to public interface 2023-11-06T20:43:51.233 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-11-06T20:43:51.233+0000 7f451b7fd380 -1 Falling back to public interface 2023-11-06T20:43:51.234 INFO:tasks.ceph.osd.7.smithi069.stderr:2023-11-06T20:43:51.235+0000 7fe47ef60380 -1 Falling back to public interface 2023-11-06T20:43:51.240 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-11-06T20:43:51.241+0000 7f5bf8ffd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6827/0: (98) Address already in use 2023-11-06T20:43:51.572 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:51.573+0000 7fa46b635680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:43:51.659 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:51.660+0000 7fa46b635680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:43:51.665 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:51.665+0000 7f0052f61680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:43:51.693 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-06T20:43:51.758 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:51.758+0000 7fa46b635680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:43:51.760 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:51.761+0000 7f0052f61680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:43:51.865 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:51.865+0000 7f0052f61680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:43:51.868 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.868+0000 7f9e45dff700 1 Processor -- start 2023-11-06T20:43:51.868 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.868+0000 7f9e45dff700 1 -- start start 2023-11-06T20:43:51.870 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.870+0000 7f9e45dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 0x7f9e40155560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:51.870 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.870+0000 7f9e45dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9e400631f0 0x7f9e40063670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:51.872 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.870+0000 7f9e45dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e40063b60 con 0x7f9e40157eb0 2023-11-06T20:43:51.872 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.870+0000 7f9e45dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e400ada40 con 0x7f9e400631f0 2023-11-06T20:43:51.872 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.870+0000 7f9e45dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e40158de0 con 0x7f9e400aa550 2023-11-06T20:43:51.872 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.870+0000 7f9e3f7fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 0x7f9e40155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:51.873 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.870+0000 7f9e3effd700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9e400631f0 0x7f9e40063670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:51.873 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.870+0000 7f9e3effd700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9e400631f0 0x7f9e40063670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:58720/0 (socket says 172.21.15.33:58720) 2023-11-06T20:43:51.873 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.870+0000 7f9e3effd700 1 -- 172.21.15.33:0/495488090 learned_addr learned my addr 172.21.15.33:0/495488090 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:43:51.873 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.871+0000 7f9e3f7fe700 1 -- 172.21.15.33:0/495488090 >> v2:172.21.15.69:3300/0 conn(0x7f9e400631f0 msgr2=0x7f9e40063670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:51.873 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.871+0000 7f9e3f7fe700 1 --2- 172.21.15.33:0/495488090 >> v2:172.21.15.69:3300/0 conn(0x7f9e400631f0 0x7f9e40063670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:51.874 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.871+0000 7f9e3f7fe700 1 -- 172.21.15.33:0/495488090 >> v1:172.21.15.69:6789/0 conn(0x7f9e40157eb0 legacy=0x7f9e400c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:51.874 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.871+0000 7f9e3f7fe700 1 -- 172.21.15.33:0/495488090 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e40158b00 con 0x7f9e400aa550 2023-11-06T20:43:51.874 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.871+0000 7f9e3f7fe700 1 --2- 172.21.15.33:0/495488090 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 0x7f9e40155560 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f9e30009760 tx=0x7f9e30009060).ready entity=mon.2 client_cookie=85fd45b7d5e56286 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:51.875 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.871+0000 7f9e3e7fc700 1 -- 172.21.15.33:0/495488090 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000f510 con 0x7f9e400aa550 2023-11-06T20:43:51.875 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.871+0000 7f9e3e7fc700 1 -- 172.21.15.33:0/495488090 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3000f670 con 0x7f9e400aa550 2023-11-06T20:43:51.875 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.871+0000 7f9e3e7fc700 1 -- 172.21.15.33:0/495488090 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000f9f0 con 0x7f9e400aa550 2023-11-06T20:43:51.875 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.872+0000 7f9e45dff700 1 -- 172.21.15.33:0/495488090 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 msgr2=0x7f9e40155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:51.875 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.872+0000 7f9e45dff700 1 --2- 172.21.15.33:0/495488090 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 0x7f9e40155560 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f9e30009760 tx=0x7f9e30009060).stop 2023-11-06T20:43:51.876 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.872+0000 7f9e45dff700 1 -- 172.21.15.33:0/495488090 shutdown_connections 2023-11-06T20:43:51.876 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.872+0000 7f9e45dff700 1 --2- 172.21.15.33:0/495488090 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 0x7f9e40155560 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:51.876 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.872+0000 7f9e45dff700 1 --2- 172.21.15.33:0/495488090 >> v2:172.21.15.69:3300/0 conn(0x7f9e400631f0 0x7f9e40063670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:51.876 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.872+0000 7f9e45dff700 1 -- 172.21.15.33:0/495488090 >> 172.21.15.33:0/495488090 conn(0x7f9e400cae80 msgr2=0x7f9e400cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:51.876 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.872+0000 7f9e45dff700 1 -- 172.21.15.33:0/495488090 shutdown_connections 2023-11-06T20:43:51.877 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.872+0000 7f9e45dff700 1 -- 172.21.15.33:0/495488090 wait complete. 2023-11-06T20:43:51.879 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.873+0000 7f9e45dff700 1 Processor -- start 2023-11-06T20:43:51.879 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.874+0000 7f9e45dff700 1 -- start start 2023-11-06T20:43:51.879 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.874+0000 7f9e45dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e400631f0 0x7f9e4013ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:51.879 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.874+0000 7f9e45dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 0x7f9e400c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:51.880 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.874+0000 7f9e45dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e40157eb0 0x7f9e40165f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:51.880 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.874+0000 7f9e45dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e40155ab0 con 0x7f9e40157eb0 2023-11-06T20:43:51.880 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.874+0000 7f9e45dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e40155d90 con 0x7f9e400631f0 2023-11-06T20:43:51.880 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.874+0000 7f9e45dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e40156070 con 0x7f9e400aa550 2023-11-06T20:43:51.881 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e400631f0 0x7f9e4013ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:51.881 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e40157eb0 0x7f9e40165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:51.881 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3effd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 0x7f9e400c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:51.881 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e400631f0 0x7f9e4013ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:58726/0 (socket says 172.21.15.33:58726) 2023-11-06T20:43:51.881 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3f7fe700 1 -- 172.21.15.33:0/2336040713 learned_addr learned my addr 172.21.15.33:0/2336040713 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:43:51.882 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3effd700 1 -- 172.21.15.33:0/2336040713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e400631f0 msgr2=0x7f9e4013ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:51.882 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3effd700 1 --2- 172.21.15.33:0/2336040713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e400631f0 0x7f9e4013ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:51.882 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3effd700 1 -- 172.21.15.33:0/2336040713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e40157eb0 msgr2=0x7f9e40165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:51.882 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3effd700 1 --2- 172.21.15.33:0/2336040713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e40157eb0 0x7f9e40165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:51.882 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3effd700 1 -- 172.21.15.33:0/2336040713 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e300092a0 con 0x7f9e400aa550 2023-11-06T20:43:51.883 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3ffff700 1 --2- 172.21.15.33:0/2336040713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e40157eb0 0x7f9e40165f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:51.883 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.875+0000 7f9e3effd700 1 --2- 172.21.15.33:0/2336040713 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 0x7f9e400c24e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f9e40155a80 tx=0x7f9e340090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:51.883 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.876+0000 7f9e277fe700 1 -- 172.21.15.33:0/2336040713 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34012600 con 0x7f9e400aa550 2023-11-06T20:43:51.883 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.876+0000 7f9e277fe700 1 -- 172.21.15.33:0/2336040713 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e34012760 con 0x7f9e400aa550 2023-11-06T20:43:51.883 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.876+0000 7f9e277fe700 1 -- 172.21.15.33:0/2336040713 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34012ae0 con 0x7f9e400aa550 2023-11-06T20:43:51.883 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.876+0000 7f9e45dff700 1 -- 172.21.15.33:0/2336040713 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e40156350 con 0x7f9e400aa550 2023-11-06T20:43:51.884 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.876+0000 7f9e45dff700 1 -- 172.21.15.33:0/2336040713 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e400c2070 con 0x7f9e400aa550 2023-11-06T20:43:51.884 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.876+0000 7f9e277fe700 1 -- 172.21.15.33:0/2336040713 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f9e3401b660 con 0x7f9e400aa550 2023-11-06T20:43:51.884 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.876+0000 7f9e277fe700 1 -- 172.21.15.33:0/2336040713 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f9e3401b910 con 0x7f9e400aa550 2023-11-06T20:43:51.884 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.877+0000 7f9e45dff700 1 -- 172.21.15.33:0/2336040713 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e401567f0 con 0x7f9e400aa550 2023-11-06T20:43:51.885 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.880+0000 7f9e277fe700 1 -- 172.21.15.33:0/2336040713 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f9e34016020 con 0x7f9e400aa550 2023-11-06T20:43:51.949 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:51.949+0000 7fa46b635680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:43:51.950 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.950+0000 7f9e45dff700 1 -- 172.21.15.33:0/2336040713 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e401569a0 con 0x7f9e400aa550 2023-11-06T20:43:51.951 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.951+0000 7f9e277fe700 1 -- 172.21.15.33:0/2336040713 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7f9e3401bb50 con 0x7f9e400aa550 2023-11-06T20:43:51.951 INFO:teuthology.misc.health.smithi033.stdout: 2023-11-06T20:43:51.951 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":10,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:43:50.451599+0000","last_up_change":"0.000000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:43:51.952 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.952+0000 7f9e45dff700 1 -- 172.21.15.33:0/2336040713 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 msgr2=0x7f9e400c24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:51.953 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.952+0000 7f9e45dff700 1 --2- 172.21.15.33:0/2336040713 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 0x7f9e400c24e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f9e40155a80 tx=0x7f9e340090f0).stop 2023-11-06T20:43:51.953 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.952+0000 7f9e45dff700 1 -- 172.21.15.33:0/2336040713 shutdown_connections 2023-11-06T20:43:51.953 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.952+0000 7f9e45dff700 1 --2- 172.21.15.33:0/2336040713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e40157eb0 0x7f9e40165f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:51.953 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.952+0000 7f9e45dff700 1 --2- 172.21.15.33:0/2336040713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e400631f0 0x7f9e4013ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:51.954 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.952+0000 7f9e45dff700 1 --2- 172.21.15.33:0/2336040713 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e400aa550 0x7f9e400c24e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:51.954 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.952+0000 7f9e45dff700 1 -- 172.21.15.33:0/2336040713 >> 172.21.15.33:0/2336040713 conn(0x7f9e400cae80 msgr2=0x7f9e400bfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:51.954 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.952+0000 7f9e45dff700 1 -- 172.21.15.33:0/2336040713 shutdown_connections 2023-11-06T20:43:51.954 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:51.953+0000 7f9e45dff700 1 -- 172.21.15.33:0/2336040713 wait complete. 2023-11-06T20:43:51.966 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-06T20:43:52.041 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:52.042+0000 7fa46b635680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:43:52.065 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:52.065+0000 7f0052f61680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:43:52.160 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:52.160+0000 7f0052f61680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:43:52.186 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:52.186+0000 7fa46b635680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:43:52.311 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:52.311+0000 7f0052f61680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:43:52.318 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:52.318+0000 7fa46b635680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:43:52.447 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:52.447+0000 7f0052f61680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:43:52.699 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:52.699+0000 7fa46b635680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:43:52.788 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:52.788+0000 7fa46b635680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:43:52.848 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:52.848+0000 7f0052f61680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:43:52.941 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:43:52.941+0000 7f0052f61680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:43:53.809 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:53.809+0000 7fa41b3fa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-06T20:43:53.810 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:43:53.811+0000 7fa41b3fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-06T20:43:54.938 INFO:tasks.ceph.osd.6.smithi069.stderr:2023-11-06T20:43:54.939+0000 7f353e5c2380 -1 osd.6 0 log_to_monitors {default=true} 2023-11-06T20:43:54.964 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-11-06T20:43:54.964+0000 7f06acd04380 -1 osd.0 0 log_to_monitors {default=true} 2023-11-06T20:43:55.000 INFO:tasks.ceph.osd.4.smithi069.stderr:2023-11-06T20:43:55.001+0000 7f114d62a380 -1 osd.4 0 log_to_monitors {default=true} 2023-11-06T20:43:55.019 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-11-06T20:43:55.019+0000 7ff7b9c8d380 -1 osd.1 0 log_to_monitors {default=true} 2023-11-06T20:43:55.027 INFO:tasks.ceph.osd.5.smithi069.stderr:2023-11-06T20:43:55.028+0000 7ff08a406380 -1 osd.5 0 log_to_monitors {default=true} 2023-11-06T20:43:55.034 INFO:tasks.ceph.osd.7.smithi069.stderr:2023-11-06T20:43:55.034+0000 7fe47ef60380 -1 osd.7 0 log_to_monitors {default=true} 2023-11-06T20:43:55.081 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-11-06T20:43:55.081+0000 7f5c021f0380 -1 osd.3 0 log_to_monitors {default=true} 2023-11-06T20:43:55.100 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-11-06T20:43:55.101+0000 7f451b7fd380 -1 osd.2 0 log_to_monitors {default=true} 2023-11-06T20:43:56.827 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-11-06T20:43:56.827+0000 7f5bf57f6700 -1 osd.3 0 waiting for initial osdmap 2023-11-06T20:43:56.828 INFO:tasks.ceph.osd.5.smithi069.stderr:2023-11-06T20:43:56.829+0000 7ff07d9f6700 -1 osd.5 0 waiting for initial osdmap 2023-11-06T20:43:56.830 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-11-06T20:43:56.830+0000 7f06a03f6700 -1 osd.0 0 waiting for initial osdmap 2023-11-06T20:43:56.830 INFO:tasks.ceph.osd.7.smithi069.stderr:2023-11-06T20:43:56.830+0000 7fe4725f6700 -1 osd.7 0 waiting for initial osdmap 2023-11-06T20:43:56.834 INFO:tasks.ceph.osd.4.smithi069.stderr:2023-11-06T20:43:56.834+0000 7f1140df6700 -1 osd.4 0 waiting for initial osdmap 2023-11-06T20:43:56.834 INFO:tasks.ceph.osd.6.smithi069.stderr:2023-11-06T20:43:56.834+0000 7f3531bf6700 -1 osd.6 0 waiting for initial osdmap 2023-11-06T20:43:56.837 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-11-06T20:43:56.838+0000 7ff7ad3f6700 -1 osd.1 0 waiting for initial osdmap 2023-11-06T20:43:56.837 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-11-06T20:43:56.838+0000 7f450edf6700 -1 osd.2 0 waiting for initial osdmap 2023-11-06T20:43:56.840 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-11-06T20:43:56.840+0000 7f5bf1bfa700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T20:43:56.840 INFO:tasks.ceph.osd.5.smithi069.stderr:2023-11-06T20:43:56.841+0000 7ff079dfa700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T20:43:56.841 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-11-06T20:43:56.841+0000 7f069c7fa700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T20:43:56.841 INFO:tasks.ceph.osd.7.smithi069.stderr:2023-11-06T20:43:56.842+0000 7fe46e9fa700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T20:43:56.843 INFO:tasks.ceph.osd.6.smithi069.stderr:2023-11-06T20:43:56.843+0000 7f352dffa700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T20:43:56.843 INFO:tasks.ceph.osd.4.smithi069.stderr:2023-11-06T20:43:56.844+0000 7f113d1fa700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T20:43:56.844 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-11-06T20:43:56.845+0000 7f450b1fa700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T20:43:56.846 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-11-06T20:43:56.847+0000 7ff7a97fa700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T20:43:58.772 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-06T20:43:58.957 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.957+0000 7f5499fff700 1 Processor -- start 2023-11-06T20:43:58.958 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.958+0000 7f5499fff700 1 -- start start 2023-11-06T20:43:58.959 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.959+0000 7f5499fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5494148f10 0x7f5494153110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:58.959 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.959+0000 7f5499fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f5494069c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:58.959 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.959+0000 7f5499fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5494068ee0 con 0x7f549406a100 2023-11-06T20:43:58.960 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.959+0000 7f5499fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549406a570 con 0x7f5494148f10 2023-11-06T20:43:58.960 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.959+0000 7f5499fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5494068310 con 0x7f54940c73a0 2023-11-06T20:43:58.962 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.959+0000 7f5492ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f5494069c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:58.962 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.959+0000 7f54937fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5494148f10 0x7f5494153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:58.962 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f5492ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f5494069c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:57834/0 (socket says 172.21.15.33:57834) 2023-11-06T20:43:58.963 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f54937fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5494148f10 0x7f5494153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:47030/0 (socket says 172.21.15.33:47030) 2023-11-06T20:43:58.963 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f5492ffd700 1 -- 172.21.15.33:0/3709635389 learned_addr learned my addr 172.21.15.33:0/3709635389 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:43:58.963 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f54927fc700 1 -- 172.21.15.33:0/3709635389 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181321793 0 0) 0x7f5494068ee0 con 0x7f549406a100 2023-11-06T20:43:58.963 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f54927fc700 1 -- 172.21.15.33:0/3709635389 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f547c0033f0 con 0x7f549406a100 2023-11-06T20:43:58.963 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f54937fe700 1 -- 172.21.15.33:0/3709635389 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 msgr2=0x7f5494069c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:58.964 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f54937fe700 1 --2- 172.21.15.33:0/3709635389 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f5494069c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:58.964 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f54937fe700 1 -- 172.21.15.33:0/3709635389 >> v1:172.21.15.69:6789/0 conn(0x7f549406a100 legacy=0x7f5494155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:58.964 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f54937fe700 1 -- 172.21.15.33:0/3709635389 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54940685b0 con 0x7f5494148f10 2023-11-06T20:43:58.964 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f5492ffd700 1 --2- 172.21.15.33:0/3709635389 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f5494069c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:43:58.965 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.960+0000 7f54937fe700 1 --2- 172.21.15.33:0/3709635389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5494148f10 0x7f5494153110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5484009760 tx=0x7f5484009060).ready entity=mon.1 client_cookie=ec27b41014a200ff server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:58.965 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f54927fc700 1 -- 172.21.15.33:0/3709635389 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548400d4a0 con 0x7f5494148f10 2023-11-06T20:43:58.965 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f54927fc700 1 -- 172.21.15.33:0/3709635389 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548400d600 con 0x7f5494148f10 2023-11-06T20:43:58.965 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f54927fc700 1 -- 172.21.15.33:0/3709635389 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548400d980 con 0x7f5494148f10 2023-11-06T20:43:58.965 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f5499fff700 1 -- 172.21.15.33:0/3709635389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5494148f10 msgr2=0x7f5494153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:58.966 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f5499fff700 1 --2- 172.21.15.33:0/3709635389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5494148f10 0x7f5494153110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5484009760 tx=0x7f5484009060).stop 2023-11-06T20:43:58.966 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f5499fff700 1 -- 172.21.15.33:0/3709635389 shutdown_connections 2023-11-06T20:43:58.966 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f5499fff700 1 --2- 172.21.15.33:0/3709635389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5494148f10 0x7f5494153110 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:58.966 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f5499fff700 1 --2- 172.21.15.33:0/3709635389 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f5494069c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:58.966 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f5499fff700 1 -- 172.21.15.33:0/3709635389 >> 172.21.15.33:0/3709635389 conn(0x7f5494063240 msgr2=0x7f54940c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:58.967 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f5499fff700 1 -- 172.21.15.33:0/3709635389 shutdown_connections 2023-11-06T20:43:58.967 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.961+0000 7f5499fff700 1 -- 172.21.15.33:0/3709635389 wait complete. 2023-11-06T20:43:58.970 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.963+0000 7f5499fff700 1 Processor -- start 2023-11-06T20:43:58.970 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.964+0000 7f5499fff700 1 -- start start 2023-11-06T20:43:58.970 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f5499fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549406a100 0x7f54940ab370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:58.971 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f5499fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f54940ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:58.971 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f5499fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5494148f10 0x7f54941497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:58.971 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f5499fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54940abd50 con 0x7f549406a100 2023-11-06T20:43:58.971 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f5499fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5494147880 con 0x7f5494148f10 2023-11-06T20:43:58.971 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f5499fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549414add0 con 0x7f54940c73a0 2023-11-06T20:43:58.972 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f54937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549406a100 0x7f54940ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:58.972 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f54937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549406a100 0x7f54940ab370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53842/0 (socket says 172.21.15.33:53842) 2023-11-06T20:43:58.972 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f54937fe700 1 -- 172.21.15.33:0/1616986285 learned_addr learned my addr 172.21.15.33:0/1616986285 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:43:58.972 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f5492ffd700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f54940ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:58.972 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.965+0000 7f5493fff700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5494148f10 0x7f54941497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:58.973 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f54937fe700 1 -- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 msgr2=0x7f54940ab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:58.973 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f54937fe700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f54940ab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:58.973 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f54937fe700 1 -- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5494148f10 msgr2=0x7f54941497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:58.973 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f54937fe700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5494148f10 0x7f54941497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:58.974 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f54937fe700 1 -- 172.21.15.33:0/1616986285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488009370 con 0x7f549406a100 2023-11-06T20:43:58.974 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f5492ffd700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f54940ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:43:58.974 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f5493fff700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5494148f10 0x7f54941497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:43:58.974 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f54937fe700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549406a100 0x7f54940ab370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f5494067c80 tx=0x7f5484016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:58.974 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f547b7fe700 1 -- 172.21.15.33:0/1616986285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548400de10 con 0x7f549406a100 2023-11-06T20:43:58.975 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f547b7fe700 1 -- 172.21.15.33:0/1616986285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548400d770 con 0x7f549406a100 2023-11-06T20:43:58.975 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f5499fff700 1 -- 172.21.15.33:0/1616986285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54840092a0 con 0x7f549406a100 2023-11-06T20:43:58.975 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f547b7fe700 1 -- 172.21.15.33:0/1616986285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548401d680 con 0x7f549406a100 2023-11-06T20:43:58.975 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.966+0000 7f5499fff700 1 -- 172.21.15.33:0/1616986285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5494160020 con 0x7f549406a100 2023-11-06T20:43:58.975 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.967+0000 7f5499fff700 1 -- 172.21.15.33:0/1616986285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5494160300 con 0x7f549406a100 2023-11-06T20:43:58.976 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.968+0000 7f547b7fe700 1 -- 172.21.15.33:0/1616986285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f548401da30 con 0x7f549406a100 2023-11-06T20:43:58.976 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.969+0000 7f547b7fe700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f548005f120 0x7f54800615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:58.976 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.969+0000 7f547b7fe700 1 -- 172.21.15.33:0/1616986285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7f548407b5f0 con 0x7f549406a100 2023-11-06T20:43:58.976 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.969+0000 7f5492ffd700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f548005f120 0x7f54800615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:58.976 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.970+0000 7f5492ffd700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f548005f120 0x7f54800615e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f54940bd4c0 tx=0x7f5488008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:58.977 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:58.974+0000 7f547b7fe700 1 -- 172.21.15.33:0/1616986285 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f548404cd90 con 0x7f549406a100 2023-11-06T20:43:59.159 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.159+0000 7f5499fff700 1 -- 172.21.15.33:0/1616986285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54940475a0 con 0x7f549406a100 2023-11-06T20:43:59.160 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.160+0000 7f547b7fe700 1 -- 172.21.15.33:0/1616986285 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+12228 (secure 0 0 0) 0x7f54840503b0 con 0x7f549406a100 2023-11-06T20:43:59.160 INFO:teuthology.misc.health.smithi033.stdout: 2023-11-06T20:43:59.161 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":14,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:43:58.839022+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:43:59.163 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.162+0000 7f5499fff700 1 -- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f548005f120 msgr2=0x7f54800615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:59.163 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.162+0000 7f5499fff700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f548005f120 0x7f54800615e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f54940bd4c0 tx=0x7f5488008040).stop 2023-11-06T20:43:59.163 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.162+0000 7f5499fff700 1 -- 172.21.15.33:0/1616986285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549406a100 msgr2=0x7f54940ab370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:59.163 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.162+0000 7f5499fff700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549406a100 0x7f54940ab370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f5494067c80 tx=0x7f5484016000).stop 2023-11-06T20:43:59.163 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.162+0000 7f5499fff700 1 -- 172.21.15.33:0/1616986285 shutdown_connections 2023-11-06T20:43:59.164 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.162+0000 7f5499fff700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f548005f120 0x7f54800615e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.164 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.162+0000 7f5499fff700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5494148f10 0x7f54941497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.164 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.162+0000 7f5499fff700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f549406a100 0x7f54940ab370 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.164 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.162+0000 7f5499fff700 1 --2- 172.21.15.33:0/1616986285 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54940c73a0 0x7f54940ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.164 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.162+0000 7f5499fff700 1 -- 172.21.15.33:0/1616986285 >> 172.21.15.33:0/1616986285 conn(0x7f5494063240 msgr2=0x7f54940a89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:59.165 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.163+0000 7f5499fff700 1 -- 172.21.15.33:0/1616986285 shutdown_connections 2023-11-06T20:43:59.165 INFO:teuthology.misc.health.smithi033.stderr:2023-11-06T20:43:59.163+0000 7f5499fff700 1 -- 172.21.15.33:0/1616986285 wait complete. 2023-11-06T20:43:59.177 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-06T20:43:59.178 INFO:tasks.ceph:Creating RBD pool 2023-11-06T20:43:59.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-06T20:43:59.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.368+0000 7f5aec5ff700 1 Processor -- start 2023-11-06T20:43:59.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.368+0000 7f5aec5ff700 1 -- start start 2023-11-06T20:43:59.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.369+0000 7f5aec5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ae4154330 0x7f5ae40c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:59.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.369+0000 7f5aec5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5ae4154710 0x7f5ae40c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:59.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.369+0000 7f5aec5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae40c6b60 con 0x7f5ae40c7ff0 2023-11-06T20:43:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.369+0000 7f5aec5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae40c5cd0 con 0x7f5ae4154330 2023-11-06T20:43:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.369+0000 7f5aec5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae40c5f50 con 0x7f5ae4154710 2023-11-06T20:43:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.369+0000 7f5aeab9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5ae40c7ff0 0x7f5ae40646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41168/0 (socket says 172.21.15.33:41168) 2023-11-06T20:43:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.369+0000 7f5aeab9c700 1 -- 172.21.15.33:0/2106917150 learned_addr learned my addr 172.21.15.33:0/2106917150 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:43:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.369+0000 7f5aea39b700 1 --2- 172.21.15.33:0/2106917150 >> v2:172.21.15.33:3300/0 conn(0x7f5ae4154330 0x7f5ae40c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.370+0000 7f5ae9b9a700 1 --2- 172.21.15.33:0/2106917150 >> v2:172.21.15.69:3300/0 conn(0x7f5ae4154710 0x7f5ae40c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.370+0000 7f5aea39b700 1 -- 172.21.15.33:0/2106917150 >> v2:172.21.15.69:3300/0 conn(0x7f5ae4154710 msgr2=0x7f5ae40c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:43:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.370+0000 7f5aea39b700 1 --2- 172.21.15.33:0/2106917150 >> v2:172.21.15.69:3300/0 conn(0x7f5ae4154710 0x7f5ae40c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.370+0000 7f5aea39b700 1 -- 172.21.15.33:0/2106917150 >> v1:172.21.15.33:6789/0 conn(0x7f5ae40c7ff0 legacy=0x7f5ae40646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:59.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.370+0000 7f5aea39b700 1 -- 172.21.15.33:0/2106917150 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae40c6230 con 0x7f5ae4154330 2023-11-06T20:43:59.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.370+0000 7f5ae9399700 1 -- 172.21.15.33:0/2106917150 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509010297 0 0) 0x7f5ae40c6b60 con 0x7f5ae40c7ff0 2023-11-06T20:43:59.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.370+0000 7f5aea39b700 1 --2- 172.21.15.33:0/2106917150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 0x7f5ae40c57e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5ae40c8d10 tx=0x7f5ae0009060).ready entity=mon.0 client_cookie=4bf08a9d0a77bce0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:59.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.370+0000 7f5ae9399700 1 -- 172.21.15.33:0/2106917150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000f510 con 0x7f5ae4154330 2023-11-06T20:43:59.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.370+0000 7f5ae9399700 1 -- 172.21.15.33:0/2106917150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae000f670 con 0x7f5ae4154330 2023-11-06T20:43:59.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.370+0000 7f5ae9399700 1 -- 172.21.15.33:0/2106917150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000f9f0 con 0x7f5ae4154330 2023-11-06T20:43:59.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.371+0000 7f5aec5ff700 1 -- 172.21.15.33:0/2106917150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 msgr2=0x7f5ae40c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:59.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.371+0000 7f5aec5ff700 1 --2- 172.21.15.33:0/2106917150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 0x7f5ae40c57e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5ae40c8d10 tx=0x7f5ae0009060).stop 2023-11-06T20:43:59.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.371+0000 7f5aec5ff700 1 -- 172.21.15.33:0/2106917150 shutdown_connections 2023-11-06T20:43:59.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.371+0000 7f5aec5ff700 1 --2- 172.21.15.33:0/2106917150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 0x7f5ae40c57e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.371+0000 7f5aec5ff700 1 --2- 172.21.15.33:0/2106917150 >> v2:172.21.15.69:3300/0 conn(0x7f5ae4154710 0x7f5ae40c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.371+0000 7f5aec5ff700 1 -- 172.21.15.33:0/2106917150 >> 172.21.15.33:0/2106917150 conn(0x7f5ae40cb910 msgr2=0x7f5ae40c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:59.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.371+0000 7f5aec5ff700 1 -- 172.21.15.33:0/2106917150 shutdown_connections 2023-11-06T20:43:59.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.371+0000 7f5aec5ff700 1 -- 172.21.15.33:0/2106917150 wait complete. 2023-11-06T20:43:59.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.373+0000 7f5aec5ff700 1 Processor -- start 2023-11-06T20:43:59.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.373+0000 7f5aec5ff700 1 -- start start 2023-11-06T20:43:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5aec5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ae40c7ff0 0x7f5ae4150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5aec5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 0x7f5ae41648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5aec5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae4154710 0x7f5ae40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5aec5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae4155c10 con 0x7f5ae4154330 2023-11-06T20:43:59.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5aec5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae4155ef0 con 0x7f5ae4154710 2023-11-06T20:43:59.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5aec5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae41561d0 con 0x7f5ae40c7ff0 2023-11-06T20:43:59.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5ae9b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 0x7f5ae41648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:59.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5ae9b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 0x7f5ae41648e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53864/0 (socket says 172.21.15.33:53864) 2023-11-06T20:43:59.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5ae9b9a700 1 -- 172.21.15.33:0/972338744 learned_addr learned my addr 172.21.15.33:0/972338744 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:43:59.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5aea39b700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ae40c7ff0 0x7f5ae4150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:59.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5aeab9c700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae4154710 0x7f5ae40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:59.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5ae9b9a700 1 -- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ae40c7ff0 msgr2=0x7f5ae4150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:59.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5ae9b9a700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ae40c7ff0 0x7f5ae4150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5ae9b9a700 1 -- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae4154710 msgr2=0x7f5ae40ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:59.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5ae9b9a700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae4154710 0x7f5ae40ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5ae9b9a700 1 -- 172.21.15.33:0/972338744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae00092a0 con 0x7f5ae4154330 2023-11-06T20:43:59.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.374+0000 7f5ae9b9a700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 0x7f5ae41648e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f5ad400e900 tx=0x7f5ad4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:59.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.375+0000 7f5ada7fc700 1 -- 172.21.15.33:0/972338744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad400eea0 con 0x7f5ae4154330 2023-11-06T20:43:59.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.375+0000 7f5ada7fc700 1 -- 172.21.15.33:0/972338744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad400c850 con 0x7f5ae4154330 2023-11-06T20:43:59.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.375+0000 7f5ada7fc700 1 -- 172.21.15.33:0/972338744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad400cbd0 con 0x7f5ae4154330 2023-11-06T20:43:59.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.375+0000 7f5aec5ff700 1 -- 172.21.15.33:0/972338744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae41564b0 con 0x7f5ae4154330 2023-11-06T20:43:59.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.375+0000 7f5aec5ff700 1 -- 172.21.15.33:0/972338744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae41511a0 con 0x7f5ae4154330 2023-11-06T20:43:59.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.376+0000 7f5aec5ff700 1 -- 172.21.15.33:0/972338744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae40830b0 con 0x7f5ae4154330 2023-11-06T20:43:59.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.377+0000 7f5ada7fc700 1 -- 172.21.15.33:0/972338744 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5ad401b400 con 0x7f5ae4154330 2023-11-06T20:43:59.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.377+0000 7f5ada7fc700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ac805f150 0x7f5ac8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:43:59.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.378+0000 7f5ada7fc700 1 -- 172.21.15.33:0/972338744 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7f5ad407a3c0 con 0x7f5ae4154330 2023-11-06T20:43:59.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.378+0000 7f5aea39b700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ac805f150 0x7f5ac8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:43:59.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.378+0000 7f5aea39b700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ac805f150 0x7f5ac8061610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f5ae40c8d10 tx=0x7f5ae0017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:43:59.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.382+0000 7f5ada7fc700 1 -- 172.21.15.33:0/972338744 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5ad404ba40 con 0x7f5ae4154330 2023-11-06T20:43:59.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.564+0000 7f5aec5ff700 1 -- 172.21.15.33:0/972338744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f5ae4062b00 con 0x7f5ae4154330 2023-11-06T20:43:59.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.848+0000 7f5ada7fc700 1 -- 172.21.15.33:0/972338744 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7f5ad4016070 con 0x7f5ae4154330 2023-11-06T20:43:59.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.851+0000 7f5aec5ff700 1 -- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ac805f150 msgr2=0x7f5ac8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:59.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.851+0000 7f5aec5ff700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ac805f150 0x7f5ac8061610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f5ae40c8d10 tx=0x7f5ae0017000).stop 2023-11-06T20:43:59.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.852+0000 7f5aec5ff700 1 -- 172.21.15.33:0/972338744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 msgr2=0x7f5ae41648e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:43:59.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.852+0000 7f5aec5ff700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 0x7f5ae41648e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f5ad400e900 tx=0x7f5ad4009060).stop 2023-11-06T20:43:59.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.852+0000 7f5aec5ff700 1 -- 172.21.15.33:0/972338744 shutdown_connections 2023-11-06T20:43:59.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.852+0000 7f5aec5ff700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ac805f150 0x7f5ac8061610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.852+0000 7f5aec5ff700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ae40c7ff0 0x7f5ae4150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.852+0000 7f5aec5ff700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae4154710 0x7f5ae40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.852+0000 7f5aec5ff700 1 --2- 172.21.15.33:0/972338744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ae4154330 0x7f5ae41648e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:43:59.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.852+0000 7f5aec5ff700 1 -- 172.21.15.33:0/972338744 >> 172.21.15.33:0/972338744 conn(0x7f5ae40cb910 msgr2=0x7f5ae40c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:43:59.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.853+0000 7f5aec5ff700 1 -- 172.21.15.33:0/972338744 shutdown_connections 2023-11-06T20:43:59.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:43:59.853+0000 7f5aec5ff700 1 -- 172.21.15.33:0/972338744 wait complete. 2023-11-06T20:43:59.855 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'rbd' created 2023-11-06T20:43:59.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-06T20:44:00.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.063+0000 7f7ab53ff700 1 Processor -- start 2023-11-06T20:44:00.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.063+0000 7f7ab53ff700 1 -- start start 2023-11-06T20:44:00.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.064+0000 7f7ab53ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7ab0153ed0 0x7f7ab0158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:00.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.064+0000 7f7ab53ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab00c8ee0 0x7f7ab00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:00.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.064+0000 7f7ab53ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00c9f80 con 0x7f7ab01542b0 2023-11-06T20:44:00.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.064+0000 7f7ab53ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab00c6000 con 0x7f7ab0153ed0 2023-11-06T20:44:00.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.064+0000 7f7ab53ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00c9370 con 0x7f7ab00c8ee0 2023-11-06T20:44:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.065+0000 7f7aaeffd700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7ab0153ed0 0x7f7ab0158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.065+0000 7f7aae7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab00c8ee0 0x7f7ab00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.065+0000 7f7aaeffd700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7ab0153ed0 0x7f7ab0158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:47052/0 (socket says 172.21.15.33:47052) 2023-11-06T20:44:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.065+0000 7f7aaeffd700 1 -- 172.21.15.33:0/888707487 learned_addr learned my addr 172.21.15.33:0/888707487 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.065+0000 7f7aae7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab00c8ee0 0x7f7ab00ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:57856/0 (socket says 172.21.15.33:57856) 2023-11-06T20:44:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7aaeffd700 1 -- 172.21.15.33:0/888707487 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab00c8ee0 msgr2=0x7f7ab00ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7aaeffd700 1 --2- 172.21.15.33:0/888707487 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab00c8ee0 0x7f7ab00ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7aaeffd700 1 -- 172.21.15.33:0/888707487 >> v1:172.21.15.69:6789/0 conn(0x7f7ab01542b0 legacy=0x7f7ab00c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7aaeffd700 1 -- 172.21.15.33:0/888707487 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab00c9650 con 0x7f7ab0153ed0 2023-11-06T20:44:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7aadffb700 1 -- 172.21.15.33:0/888707487 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392509238 0 0) 0x7f7ab00c9f80 con 0x7f7ab01542b0 2023-11-06T20:44:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7aaeffd700 1 --2- 172.21.15.33:0/888707487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ab0153ed0 0x7f7ab0158600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7aa0006fd0 tx=0x7f7aa00090f0).ready entity=mon.1 client_cookie=2a2cea412cdba859 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7aadffb700 1 -- 172.21.15.33:0/888707487 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000b580 con 0x7f7ab0153ed0 2023-11-06T20:44:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7aadffb700 1 -- 172.21.15.33:0/888707487 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa000b6e0 con 0x7f7ab0153ed0 2023-11-06T20:44:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7aadffb700 1 -- 172.21.15.33:0/888707487 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000ba60 con 0x7f7ab0153ed0 2023-11-06T20:44:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7ab53ff700 1 -- 172.21.15.33:0/888707487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ab0153ed0 msgr2=0x7f7ab0158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.066+0000 7f7ab53ff700 1 --2- 172.21.15.33:0/888707487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ab0153ed0 0x7f7ab0158600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7aa0006fd0 tx=0x7f7aa00090f0).stop 2023-11-06T20:44:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.067+0000 7f7ab53ff700 1 -- 172.21.15.33:0/888707487 shutdown_connections 2023-11-06T20:44:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.067+0000 7f7ab53ff700 1 --2- 172.21.15.33:0/888707487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ab0153ed0 0x7f7ab0158600 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.067+0000 7f7ab53ff700 1 --2- 172.21.15.33:0/888707487 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab00c8ee0 0x7f7ab00ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.067+0000 7f7ab53ff700 1 -- 172.21.15.33:0/888707487 >> 172.21.15.33:0/888707487 conn(0x7f7ab005e1b0 msgr2=0x7f7ab005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.067+0000 7f7ab53ff700 1 -- 172.21.15.33:0/888707487 shutdown_connections 2023-11-06T20:44:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.067+0000 7f7ab53ff700 1 -- 172.21.15.33:0/888707487 wait complete. 2023-11-06T20:44:00.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.068+0000 7f7ab53ff700 1 Processor -- start 2023-11-06T20:44:00.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.068+0000 7f7ab53ff700 1 -- start start 2023-11-06T20:44:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.069+0000 7f7ab53ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ab0153ed0 0x7f7ab00a9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.069+0000 7f7ab53ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab01542b0 0x7f7ab0164f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.069+0000 7f7ab53ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab014fda0 0x7f7ab00b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.069+0000 7f7ab53ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab0151ee0 con 0x7f7ab01542b0 2023-11-06T20:44:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.069+0000 7f7ab53ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00a9c60 con 0x7f7ab0153ed0 2023-11-06T20:44:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.069+0000 7f7ab53ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab0165430 con 0x7f7ab014fda0 2023-11-06T20:44:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.069+0000 7f7aae7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab01542b0 0x7f7ab0164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7aae7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab01542b0 0x7f7ab0164f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53880/0 (socket says 172.21.15.33:53880) 2023-11-06T20:44:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7aaeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ab0153ed0 0x7f7ab00a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7aae7fc700 1 -- 172.21.15.33:0/1685683045 learned_addr learned my addr 172.21.15.33:0/1685683045 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:00.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7aaf7fe700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab014fda0 0x7f7ab00b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:00.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7aae7fc700 1 -- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab014fda0 msgr2=0x7f7ab00b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:00.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7aae7fc700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab014fda0 0x7f7ab00b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:00.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7aae7fc700 1 -- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ab0153ed0 msgr2=0x7f7ab00a9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:00.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7aae7fc700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ab0153ed0 0x7f7ab00a9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:00.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7aae7fc700 1 -- 172.21.15.33:0/1685683045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa0009370 con 0x7f7ab01542b0 2023-11-06T20:44:00.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7aae7fc700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab01542b0 0x7f7ab0164f40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7aa400e930 tx=0x7f7aa4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:00.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7a96ffd700 1 -- 172.21.15.33:0/1685683045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4012600 con 0x7f7ab01542b0 2023-11-06T20:44:00.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7a96ffd700 1 -- 172.21.15.33:0/1685683045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa4012760 con 0x7f7ab01542b0 2023-11-06T20:44:00.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7a96ffd700 1 -- 172.21.15.33:0/1685683045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4012ae0 con 0x7f7ab01542b0 2023-11-06T20:44:00.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.070+0000 7f7ab53ff700 1 -- 172.21.15.33:0/1685683045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab0150230 con 0x7f7ab01542b0 2023-11-06T20:44:00.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.071+0000 7f7ab53ff700 1 -- 172.21.15.33:0/1685683045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab01507f0 con 0x7f7ab01542b0 2023-11-06T20:44:00.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.072+0000 7f7ab53ff700 1 -- 172.21.15.33:0/1685683045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab006c6c0 con 0x7f7ab01542b0 2023-11-06T20:44:00.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.072+0000 7f7a96ffd700 1 -- 172.21.15.33:0/1685683045 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7aa401b400 con 0x7f7ab01542b0 2023-11-06T20:44:00.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.074+0000 7f7a96ffd700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a9c05f120 0x7f7a9c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.074+0000 7f7a96ffd700 1 -- 172.21.15.33:0/1685683045 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5258+0+0 (secure 0 0 0) 0x7f7aa407a2e0 con 0x7f7ab01542b0 2023-11-06T20:44:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.074+0000 7f7aaeffd700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a9c05f120 0x7f7a9c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.075+0000 7f7aaeffd700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a9c05f120 0x7f7a9c0615e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7aa0009130 tx=0x7f7aa0017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.079+0000 7f7a96ffd700 1 -- 172.21.15.33:0/1685683045 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7aa404b8b0 con 0x7f7ab01542b0 2023-11-06T20:44:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.260+0000 7f7ab53ff700 1 -- 172.21.15.33:0/1685683045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7ab00c9930 con 0x7f7ab01542b0 2023-11-06T20:44:00.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.850+0000 7f7a96ffd700 1 -- 172.21.15.33:0/1685683045 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7f7aa404eed0 con 0x7f7ab01542b0 2023-11-06T20:44:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.854+0000 7f7ab53ff700 1 -- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a9c05f120 msgr2=0x7f7a9c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.854+0000 7f7ab53ff700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a9c05f120 0x7f7a9c0615e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7aa0009130 tx=0x7f7aa0017000).stop 2023-11-06T20:44:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.854+0000 7f7ab53ff700 1 -- 172.21.15.33:0/1685683045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab01542b0 msgr2=0x7f7ab0164f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.854+0000 7f7ab53ff700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab01542b0 0x7f7ab0164f40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7aa400e930 tx=0x7f7aa4009060).stop 2023-11-06T20:44:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.854+0000 7f7ab53ff700 1 -- 172.21.15.33:0/1685683045 shutdown_connections 2023-11-06T20:44:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.854+0000 7f7ab53ff700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a9c05f120 0x7f7a9c0615e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.854+0000 7f7ab53ff700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ab014fda0 0x7f7ab00b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.854+0000 7f7ab53ff700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ab0153ed0 0x7f7ab00a9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.854+0000 7f7ab53ff700 1 --2- 172.21.15.33:0/1685683045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab01542b0 0x7f7ab0164f40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.854+0000 7f7ab53ff700 1 -- 172.21.15.33:0/1685683045 >> 172.21.15.33:0/1685683045 conn(0x7f7ab005e1b0 msgr2=0x7f7ab014f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.855+0000 7f7ab53ff700 1 -- 172.21.15.33:0/1685683045 shutdown_connections 2023-11-06T20:44:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:00.855+0000 7f7ab53ff700 1 -- 172.21.15.33:0/1685683045 wait complete. 2023-11-06T20:44:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-06T20:44:00.877 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-06T20:44:00.877 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T20:44:00.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T20:44:00.882 INFO:tasks.ceph.mds.a:Started 2023-11-06T20:44:00.883 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T20:44:00.883 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T20:44:00.888 INFO:tasks.ceph.mds.c:Started 2023-11-06T20:44:00.888 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T20:44:00.889 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T20:44:00.895 INFO:tasks.ceph.mds.b:Started 2023-11-06T20:44:00.895 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T20:44:00.895 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T20:44:00.900 INFO:tasks.ceph.mds.d:Started 2023-11-06T20:44:00.900 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-06T20:44:00.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:01.039 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T20:44:01.040 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T20:44:01.051 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T20:44:01.054 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T20:44:01.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.119+0000 7fa1e2fff700 1 Processor -- start 2023-11-06T20:44:01.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.119+0000 7fa1e2fff700 1 -- start start 2023-11-06T20:44:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.120+0000 7fa1e2fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa1dc0c61d0 0x7fa1dc0c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.120+0000 7fa1e2fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc1554a0 con 0x7fa1dc0c1840 2023-11-06T20:44:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.120+0000 7fa1e2fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc155930 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.120+0000 7fa1e2fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc154960 con 0x7fa1dc0c61d0 2023-11-06T20:44:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.120+0000 7fa1e0d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa1dc0c1840 0x7fa1dc0a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41190/0 (socket says 172.21.15.33:41190) 2023-11-06T20:44:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.120+0000 7fa1e0d9b700 1 -- 172.21.15.33:0/216530121 learned_addr learned my addr 172.21.15.33:0/216530121 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1e159c700 1 --2- 172.21.15.33:0/216530121 >> v2:172.21.15.69:3300/0 conn(0x7fa1dc0c61d0 0x7fa1dc0c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1db7fe700 1 -- 172.21.15.33:0/216530121 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350794253 0 0) 0x7fa1dc1554a0 con 0x7fa1dc0c1840 2023-11-06T20:44:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1db7fe700 1 -- 172.21.15.33:0/216530121 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1cc0033f0 con 0x7fa1dc0c1840 2023-11-06T20:44:01.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1e159c700 1 -- 172.21.15.33:0/216530121 >> v1:172.21.15.69:6789/0 conn(0x7fa1dc0c5d50 legacy=0x7fa1dc0bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1db7fe700 1 -- 172.21.15.33:0/216530121 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4042322515 0 0) 0x7fa1cc0033f0 con 0x7fa1dc0c1840 2023-11-06T20:44:01.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1e159c700 1 -- 172.21.15.33:0/216530121 >> v1:172.21.15.33:6789/0 conn(0x7fa1dc0c1840 legacy=0x7fa1dc0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1e159c700 1 -- 172.21.15.33:0/216530121 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0aa4f0 con 0x7fa1dc0c61d0 2023-11-06T20:44:01.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1e159c700 1 --2- 172.21.15.33:0/216530121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc0c61d0 0x7fa1dc0c0480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa1d4009e60 tx=0x7fa1d4009780).ready entity=mon.1 client_cookie=e5de4d6ab3e39e35 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1db7fe700 1 -- 172.21.15.33:0/216530121 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d400e070 con 0x7fa1dc0c61d0 2023-11-06T20:44:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1db7fe700 1 -- 172.21.15.33:0/216530121 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d400b850 con 0x7fa1dc0c61d0 2023-11-06T20:44:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.121+0000 7fa1db7fe700 1 -- 172.21.15.33:0/216530121 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0001470 con 0x7fa1dc0c61d0 2023-11-06T20:44:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.122+0000 7fa1e2fff700 1 -- 172.21.15.33:0/216530121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc0c61d0 msgr2=0x7fa1dc0c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.122+0000 7fa1e2fff700 1 --2- 172.21.15.33:0/216530121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc0c61d0 0x7fa1dc0c0480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa1d4009e60 tx=0x7fa1d4009780).stop 2023-11-06T20:44:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.122+0000 7fa1e2fff700 1 -- 172.21.15.33:0/216530121 shutdown_connections 2023-11-06T20:44:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.122+0000 7fa1e2fff700 1 --2- 172.21.15.33:0/216530121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc0c61d0 0x7fa1dc0c0480 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.122+0000 7fa1e2fff700 1 -- 172.21.15.33:0/216530121 >> 172.21.15.33:0/216530121 conn(0x7fa1dc05ec80 msgr2=0x7fa1dc0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.122+0000 7fa1e2fff700 1 -- 172.21.15.33:0/216530121 shutdown_connections 2023-11-06T20:44:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.122+0000 7fa1e2fff700 1 -- 172.21.15.33:0/216530121 wait complete. 2023-11-06T20:44:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.124+0000 7fa1e2fff700 1 Processor -- start 2023-11-06T20:44:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.124+0000 7fa1e2fff700 1 -- start start 2023-11-06T20:44:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1e2fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1dc0c1840 0x7fa1dc1582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1e2fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc0c5d50 0x7fa1dc0ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1e2fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc0c61d0 0x7fa1dc151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1e2fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0c1f50 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1e2fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0c21f0 con 0x7fa1dc0c61d0 2023-11-06T20:44:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1e2fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0c24d0 con 0x7fa1dc0c1840 2023-11-06T20:44:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1dbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc0c5d50 0x7fa1dc0ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1dbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc0c5d50 0x7fa1dc0ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53924/0 (socket says 172.21.15.33:53924) 2023-11-06T20:44:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1dbfff700 1 -- 172.21.15.33:0/616502426 learned_addr learned my addr 172.21.15.33:0/616502426 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1e0d9b700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1dc0c1840 0x7fa1dc1582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1e159c700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc0c61d0 0x7fa1dc151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1dbfff700 1 -- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1dc0c1840 msgr2=0x7fa1dc1582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1dbfff700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1dc0c1840 0x7fa1dc1582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1dbfff700 1 -- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc0c61d0 msgr2=0x7fa1dc151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1dbfff700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc0c61d0 0x7fa1dc151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.126+0000 7fa1dbfff700 1 -- 172.21.15.33:0/616502426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d4009a20 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.127+0000 7fa1dbfff700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc0c5d50 0x7fa1dc0ae790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa1c800c590 tx=0x7fa1c8007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.127+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/616502426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800f470 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.127+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/616502426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c800f5f0 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.127+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/616502426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800f9b0 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.127+0000 7fa1e2fff700 1 -- 172.21.15.33:0/616502426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc0c27b0 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.127+0000 7fa1e2fff700 1 -- 172.21.15.33:0/616502426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc1587c0 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.128+0000 7fa1e2fff700 1 -- 172.21.15.33:0/616502426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc0c29a0 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.129+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/616502426 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa1c8012400 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.131+0000 7fa1d8ff9700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1c005f150 0x7fa1c0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.131+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/616502426 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fa1c8079440 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.131+0000 7fa1e0d9b700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1c005f150 0x7fa1c0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.131+0000 7fa1e0d9b700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1c005f150 0x7fa1c0061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa1dc0ad080 tx=0x7fa1d0001980).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.134+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/616502426 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa1c804aa00 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.315+0000 7fa1e2fff700 1 -- 172.21.15.33:0/616502426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1dc06c6c0 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.316+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/616502426 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13770 (secure 0 0 0) 0x7fa1c804e020 con 0x7fa1dc0c5d50 2023-11-06T20:44:01.317 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:01.317 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:00.847027+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.318+0000 7fa1e2fff700 1 -- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1c005f150 msgr2=0x7fa1c0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.318+0000 7fa1e2fff700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1c005f150 0x7fa1c0061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa1dc0ad080 tx=0x7fa1d0001980).stop 2023-11-06T20:44:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.318+0000 7fa1e2fff700 1 -- 172.21.15.33:0/616502426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc0c5d50 msgr2=0x7fa1dc0ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.318+0000 7fa1e2fff700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc0c5d50 0x7fa1dc0ae790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa1c800c590 tx=0x7fa1c8007ce0).stop 2023-11-06T20:44:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.319+0000 7fa1e2fff700 1 -- 172.21.15.33:0/616502426 shutdown_connections 2023-11-06T20:44:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.319+0000 7fa1e2fff700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1c005f150 0x7fa1c0061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.319+0000 7fa1e2fff700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1dc0c1840 0x7fa1dc1582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.319+0000 7fa1e2fff700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc0c61d0 0x7fa1dc151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.319+0000 7fa1e2fff700 1 --2- 172.21.15.33:0/616502426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc0c5d50 0x7fa1dc0ae790 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.319+0000 7fa1e2fff700 1 -- 172.21.15.33:0/616502426 >> 172.21.15.33:0/616502426 conn(0x7fa1dc05ec80 msgr2=0x7fa1dc0c9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:01.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.319+0000 7fa1e2fff700 1 -- 172.21.15.33:0/616502426 shutdown_connections 2023-11-06T20:44:01.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.319+0000 7fa1e2fff700 1 -- 172.21.15.33:0/616502426 wait complete. 2023-11-06T20:44:01.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T20:44:01.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:44:01.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.538+0000 7ff8a75ff700 1 Processor -- start 2023-11-06T20:44:01.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.538+0000 7ff8a75ff700 1 -- start start 2023-11-06T20:44:01.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.539+0000 7ff8a75ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff8a0154330 0x7ff8a00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.539+0000 7ff8a75ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8a00c7ff0 0x7ff8a00646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.539+0000 7ff8a75ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a00c6b60 con 0x7ff8a0154710 2023-11-06T20:44:01.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.539+0000 7ff8a75ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a00c7700 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.539+0000 7ff8a75ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a00c5cd0 con 0x7ff8a0154330 2023-11-06T20:44:01.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.539+0000 7ff8a5b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8a00c7ff0 0x7ff8a00646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.539+0000 7ff8a5b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8a00c7ff0 0x7ff8a00646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53940/0 (socket says 172.21.15.33:53940) 2023-11-06T20:44:01.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.539+0000 7ff8a5b9c700 1 -- 172.21.15.33:0/2036278117 learned_addr learned my addr 172.21.15.33:0/2036278117 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:01.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.539+0000 7ff8a539b700 1 --2- 172.21.15.33:0/2036278117 >> v2:172.21.15.69:3300/0 conn(0x7ff8a0154330 0x7ff8a00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.540+0000 7ff897fff700 1 -- 172.21.15.33:0/2036278117 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598252938 0 0) 0x7ff8a00c6b60 con 0x7ff8a0154710 2023-11-06T20:44:01.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.540+0000 7ff897fff700 1 -- 172.21.15.33:0/2036278117 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff88c0033f0 con 0x7ff8a0154710 2023-11-06T20:44:01.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.540+0000 7ff8a5b9c700 1 -- 172.21.15.33:0/2036278117 >> v2:172.21.15.69:3300/0 conn(0x7ff8a0154330 msgr2=0x7ff8a00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:01.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.540+0000 7ff8a5b9c700 1 --2- 172.21.15.33:0/2036278117 >> v2:172.21.15.69:3300/0 conn(0x7ff8a0154330 0x7ff8a00c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.540+0000 7ff8a5b9c700 1 -- 172.21.15.33:0/2036278117 >> v1:172.21.15.69:6789/0 conn(0x7ff8a0154710 legacy=0x7ff8a00c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.540+0000 7ff8a5b9c700 1 -- 172.21.15.33:0/2036278117 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a00c5f50 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.540+0000 7ff8a5b9c700 1 --2- 172.21.15.33:0/2036278117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 0x7ff8a00646c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff8a00605d0 tx=0x7ff898009840).ready entity=mon.0 client_cookie=1ee527eeb1ad396e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:01.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.540+0000 7ff897fff700 1 -- 172.21.15.33:0/2036278117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800e070 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.540+0000 7ff897fff700 1 -- 172.21.15.33:0/2036278117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89800b970 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.540+0000 7ff897fff700 1 -- 172.21.15.33:0/2036278117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800bcf0 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.541+0000 7ff8a75ff700 1 -- 172.21.15.33:0/2036278117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 msgr2=0x7ff8a00646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.541+0000 7ff8a75ff700 1 --2- 172.21.15.33:0/2036278117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 0x7ff8a00646c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff8a00605d0 tx=0x7ff898009840).stop 2023-11-06T20:44:01.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.541+0000 7ff8a75ff700 1 -- 172.21.15.33:0/2036278117 shutdown_connections 2023-11-06T20:44:01.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.541+0000 7ff8a75ff700 1 --2- 172.21.15.33:0/2036278117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 0x7ff8a00646c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.541+0000 7ff8a75ff700 1 --2- 172.21.15.33:0/2036278117 >> v2:172.21.15.69:3300/0 conn(0x7ff8a0154330 0x7ff8a00c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.541+0000 7ff8a75ff700 1 -- 172.21.15.33:0/2036278117 >> 172.21.15.33:0/2036278117 conn(0x7ff8a00cb910 msgr2=0x7ff8a00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:01.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.541+0000 7ff8a75ff700 1 -- 172.21.15.33:0/2036278117 shutdown_connections 2023-11-06T20:44:01.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.541+0000 7ff8a75ff700 1 -- 172.21.15.33:0/2036278117 wait complete. 2023-11-06T20:44:01.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.542+0000 7ff8a75ff700 1 Processor -- start 2023-11-06T20:44:01.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.543+0000 7ff8a75ff700 1 -- start start 2023-11-06T20:44:01.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.543+0000 7ff8a75ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 0x7ff8a0164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a75ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8a0154330 0x7ff8a01592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a75ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8a0154710 0x7ff8a01571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a75ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a0155670 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a75ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00afd90 con 0x7ff8a0154330 2023-11-06T20:44:01.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a75ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a0154cd0 con 0x7ff8a0154710 2023-11-06T20:44:01.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a539b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 0x7ff8a0164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a539b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 0x7ff8a0164900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53946/0 (socket says 172.21.15.33:53946) 2023-11-06T20:44:01.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a539b700 1 -- 172.21.15.33:0/4147670150 learned_addr learned my addr 172.21.15.33:0/4147670150 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:01.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a5b9c700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8a0154710 0x7ff8a01571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a4b9a700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8a0154330 0x7ff8a01592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a539b700 1 -- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8a0154710 msgr2=0x7ff8a01571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a539b700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8a0154710 0x7ff8a01571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a539b700 1 -- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8a0154330 msgr2=0x7ff8a01592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a539b700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8a0154330 0x7ff8a01592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a539b700 1 -- 172.21.15.33:0/4147670150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff898009ae0 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a4b9a700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8a0154330 0x7ff8a01592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:01.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.544+0000 7ff8a539b700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 0x7ff8a0164900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff89c00e930 tx=0x7ff89c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:01.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.545+0000 7ff8957fa700 1 -- 172.21.15.33:0/4147670150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c00c670 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.545+0000 7ff8957fa700 1 -- 172.21.15.33:0/4147670150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c00c7d0 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.545+0000 7ff8a75ff700 1 -- 172.21.15.33:0/4147670150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a0154fb0 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.545+0000 7ff8957fa700 1 -- 172.21.15.33:0/4147670150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c00cb50 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.545+0000 7ff8a75ff700 1 -- 172.21.15.33:0/4147670150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a0164df0 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.546+0000 7ff8a75ff700 1 -- 172.21.15.33:0/4147670150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a0165040 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.547+0000 7ff8957fa700 1 -- 172.21.15.33:0/4147670150 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff89c01b400 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.548+0000 7ff8957fa700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f150 0x7ff884061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.548+0000 7ff8957fa700 1 -- 172.21.15.33:0/4147670150 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff89c07a360 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.548+0000 7ff8a4b9a700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f150 0x7ff884061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.549+0000 7ff8a4b9a700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f150 0x7ff884061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ff890007fd0 tx=0x7ff890009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.551+0000 7ff8957fa700 1 -- 172.21.15.33:0/4147670150 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff89c04b9d0 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.732+0000 7ff8a75ff700 1 -- 172.21.15.33:0/4147670150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff8a01558e0 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.733+0000 7ff8957fa700 1 -- 172.21.15.33:0/4147670150 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v16) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff89c04eff0 con 0x7ff8a00c7ff0 2023-11-06T20:44:01.732 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:44:01.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 -- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f150 msgr2=0x7ff884061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f150 0x7ff884061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ff890007fd0 tx=0x7ff890009040).stop 2023-11-06T20:44:01.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 -- 172.21.15.33:0/4147670150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 msgr2=0x7ff8a0164900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 0x7ff8a0164900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff89c00e930 tx=0x7ff89c009060).stop 2023-11-06T20:44:01.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 -- 172.21.15.33:0/4147670150 shutdown_connections 2023-11-06T20:44:01.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f150 0x7ff884061610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8a0154710 0x7ff8a01571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a00c7ff0 0x7ff8a0164900 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 --2- 172.21.15.33:0/4147670150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8a0154330 0x7ff8a01592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 -- 172.21.15.33:0/4147670150 >> 172.21.15.33:0/4147670150 conn(0x7ff8a00cb910 msgr2=0x7ff8a00c8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:01.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 -- 172.21.15.33:0/4147670150 shutdown_connections 2023-11-06T20:44:01.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.735+0000 7ff8a75ff700 1 -- 172.21.15.33:0/4147670150 wait complete. 2023-11-06T20:44:01.751 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:44:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.955+0000 7fc4425ff700 1 Processor -- start 2023-11-06T20:44:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.955+0000 7fc4425ff700 1 -- start start 2023-11-06T20:44:01.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.956+0000 7fc4425ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc43c154330 0x7fc43c0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.956+0000 7fc4425ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc43c154710 0x7fc43c0c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc4425ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43c0c6b60 con 0x7fc43c0c7ff0 2023-11-06T20:44:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc4425ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43c0c5cd0 con 0x7fc43c154330 2023-11-06T20:44:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc4425ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43c0c5f50 con 0x7fc43c154710 2023-11-06T20:44:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc43bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc43c154330 0x7fc43c0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc43bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc43c154330 0x7fc43c0c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53952/0 (socket says 172.21.15.33:53952) 2023-11-06T20:44:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc43bfff700 1 -- 172.21.15.33:0/984380187 learned_addr learned my addr 172.21.15.33:0/984380187 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc43b7fe700 1 --2- 172.21.15.33:0/984380187 >> v2:172.21.15.69:3300/0 conn(0x7fc43c154710 0x7fc43c0c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc43bfff700 1 -- 172.21.15.33:0/984380187 >> v2:172.21.15.69:3300/0 conn(0x7fc43c154710 msgr2=0x7fc43c0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc43bfff700 1 --2- 172.21.15.33:0/984380187 >> v2:172.21.15.69:3300/0 conn(0x7fc43c154710 0x7fc43c0c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc43bfff700 1 -- 172.21.15.33:0/984380187 >> v1:172.21.15.69:6789/0 conn(0x7fc43c0c7ff0 legacy=0x7fc43c0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.957+0000 7fc43bfff700 1 -- 172.21.15.33:0/984380187 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c0c6230 con 0x7fc43c154330 2023-11-06T20:44:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc43bfff700 1 --2- 172.21.15.33:0/984380187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154330 0x7fc43c0c57e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc43c0c8d10 tx=0x7fc42c0090f0).ready entity=mon.0 client_cookie=de9c83e870c18f8a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:01.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc43affd700 1 -- 172.21.15.33:0/984380187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00b520 con 0x7fc43c154330 2023-11-06T20:44:01.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc43affd700 1 -- 172.21.15.33:0/984380187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c00b680 con 0x7fc43c154330 2023-11-06T20:44:01.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc43affd700 1 -- 172.21.15.33:0/984380187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00ba00 con 0x7fc43c154330 2023-11-06T20:44:01.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc4425ff700 1 -- 172.21.15.33:0/984380187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154330 msgr2=0x7fc43c0c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc4425ff700 1 --2- 172.21.15.33:0/984380187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154330 0x7fc43c0c57e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc43c0c8d10 tx=0x7fc42c0090f0).stop 2023-11-06T20:44:01.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc4425ff700 1 -- 172.21.15.33:0/984380187 shutdown_connections 2023-11-06T20:44:01.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc4425ff700 1 --2- 172.21.15.33:0/984380187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154330 0x7fc43c0c57e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc4425ff700 1 --2- 172.21.15.33:0/984380187 >> v2:172.21.15.69:3300/0 conn(0x7fc43c154710 0x7fc43c0c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc4425ff700 1 -- 172.21.15.33:0/984380187 >> 172.21.15.33:0/984380187 conn(0x7fc43c0cb910 msgr2=0x7fc43c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:01.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc4425ff700 1 -- 172.21.15.33:0/984380187 shutdown_connections 2023-11-06T20:44:01.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.958+0000 7fc4425ff700 1 -- 172.21.15.33:0/984380187 wait complete. 2023-11-06T20:44:01.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.960+0000 7fc4425ff700 1 Processor -- start 2023-11-06T20:44:01.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.960+0000 7fc4425ff700 1 -- start start 2023-11-06T20:44:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc4425ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc43c0c7ff0 0x7fc43c164bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc4425ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc43c154330 0x7fc43c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc4425ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154710 0x7fc43c167580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc4425ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c0af560 con 0x7fc43c154710 2023-11-06T20:44:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc4425ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c156410 con 0x7fc43c0c7ff0 2023-11-06T20:44:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc4425ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c1650b0 con 0x7fc43c154330 2023-11-06T20:44:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc440b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154710 0x7fc43c167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc440b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154710 0x7fc43c167580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53960/0 (socket says 172.21.15.33:53960) 2023-11-06T20:44:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc440b9c700 1 -- 172.21.15.33:0/1775775986 learned_addr learned my addr 172.21.15.33:0/1775775986 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc43b7fe700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc43c154330 0x7fc43c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.961+0000 7fc43bfff700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc43c0c7ff0 0x7fc43c164bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc440b9c700 1 -- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc43c154330 msgr2=0x7fc43c0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc440b9c700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc43c154330 0x7fc43c0ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc440b9c700 1 -- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc43c0c7ff0 msgr2=0x7fc43c164bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc440b9c700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc43c0c7ff0 0x7fc43c164bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc440b9c700 1 -- 172.21.15.33:0/1775775986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c009370 con 0x7fc43c154710 2023-11-06T20:44:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc43b7fe700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc43c154330 0x7fc43c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc440b9c700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154710 0x7fc43c167580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc43400c7f0 tx=0x7fc434001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc423fff700 1 -- 172.21.15.33:0/1775775986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434010510 con 0x7fc43c154710 2023-11-06T20:44:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc423fff700 1 -- 172.21.15.33:0/1775775986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc434010670 con 0x7fc43c154710 2023-11-06T20:44:01.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc423fff700 1 -- 172.21.15.33:0/1775775986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4340109f0 con 0x7fc43c154710 2023-11-06T20:44:01.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc4425ff700 1 -- 172.21.15.33:0/1775775986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c0aefa0 con 0x7fc43c154710 2023-11-06T20:44:01.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.962+0000 7fc4425ff700 1 -- 172.21.15.33:0/1775775986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c1560a0 con 0x7fc43c154710 2023-11-06T20:44:01.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.964+0000 7fc4425ff700 1 -- 172.21.15.33:0/1775775986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43c156580 con 0x7fc43c154710 2023-11-06T20:44:01.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.964+0000 7fc423fff700 1 -- 172.21.15.33:0/1775775986 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc43401b400 con 0x7fc43c154710 2023-11-06T20:44:01.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.965+0000 7fc423fff700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc42805f150 0x7fc428061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.965+0000 7fc423fff700 1 -- 172.21.15.33:0/1775775986 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc434079bf0 con 0x7fc43c154710 2023-11-06T20:44:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.966+0000 7fc43bfff700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc42805f150 0x7fc428061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.966+0000 7fc43bfff700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc42805f150 0x7fc428061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fc42c009130 tx=0x7fc42c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:01.970+0000 7fc423fff700 1 -- 172.21.15.33:0/1775775986 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc43404b1b0 con 0x7fc43c154710 2023-11-06T20:44:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.153+0000 7fc4425ff700 1 -- 172.21.15.33:0/1775775986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc43c0aa6e0 con 0x7fc43c154710 2023-11-06T20:44:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.154+0000 7fc423fff700 1 -- 172.21.15.33:0/1775775986 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc43404e7d0 con 0x7fc43c154710 2023-11-06T20:44:02.154 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:44:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 -- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc42805f150 msgr2=0x7fc428061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc42805f150 0x7fc428061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fc42c009130 tx=0x7fc42c017000).stop 2023-11-06T20:44:02.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 -- 172.21.15.33:0/1775775986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154710 msgr2=0x7fc43c167580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154710 0x7fc43c167580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc43400c7f0 tx=0x7fc434001510).stop 2023-11-06T20:44:02.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 -- 172.21.15.33:0/1775775986 shutdown_connections 2023-11-06T20:44:02.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc42805f150 0x7fc428061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc43c154710 0x7fc43c167580 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc43c0c7ff0 0x7fc43c164bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 --2- 172.21.15.33:0/1775775986 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc43c154330 0x7fc43c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 -- 172.21.15.33:0/1775775986 >> 172.21.15.33:0/1775775986 conn(0x7fc43c0cb910 msgr2=0x7fc43c0aba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:02.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 -- 172.21.15.33:0/1775775986 shutdown_connections 2023-11-06T20:44:02.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.156+0000 7fc4425ff700 1 -- 172.21.15.33:0/1775775986 wait complete. 2023-11-06T20:44:02.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:44:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.368+0000 7f7c3a5ff700 1 Processor -- start 2023-11-06T20:44:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.368+0000 7f7c3a5ff700 1 -- start start 2023-11-06T20:44:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.370+0000 7f7c3a5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c340c1840 0x7f7c340a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.370+0000 7f7c3a5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7c340c6460 0x7f7c340bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.370+0000 7f7c3a5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c341554a0 con 0x7f7c340aa2d0 2023-11-06T20:44:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.370+0000 7f7c3a5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c34154960 con 0x7f7c340c1840 2023-11-06T20:44:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.370+0000 7f7c3a5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c34155930 con 0x7f7c340c6460 2023-11-06T20:44:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.370+0000 7f7c33fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c340c1840 0x7f7c340a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.370+0000 7f7c33fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c340c1840 0x7f7c340a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53964/0 (socket says 172.21.15.33:53964) 2023-11-06T20:44:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.370+0000 7f7c33fff700 1 -- 172.21.15.33:0/618687187 learned_addr learned my addr 172.21.15.33:0/618687187 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.370+0000 7f7c38b9c700 1 --2- 172.21.15.33:0/618687187 >> v2:172.21.15.69:3300/0 conn(0x7f7c340c6460 0x7f7c340bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.371+0000 7f7c33fff700 1 -- 172.21.15.33:0/618687187 >> v2:172.21.15.69:3300/0 conn(0x7f7c340c6460 msgr2=0x7f7c340bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.371+0000 7f7c33fff700 1 --2- 172.21.15.33:0/618687187 >> v2:172.21.15.69:3300/0 conn(0x7f7c340c6460 0x7f7c340bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.371+0000 7f7c33fff700 1 -- 172.21.15.33:0/618687187 >> v1:172.21.15.69:6789/0 conn(0x7f7c340aa2d0 legacy=0x7f7c340c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.371+0000 7f7c33fff700 1 -- 172.21.15.33:0/618687187 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c34154b70 con 0x7f7c340c1840 2023-11-06T20:44:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.371+0000 7f7c33fff700 1 --2- 172.21.15.33:0/618687187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 0x7f7c340a9de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7c340c55f0 tx=0x7f7c240090f0).ready entity=mon.0 client_cookie=d250a9819211ac27 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.371+0000 7f7c32ffd700 1 -- 172.21.15.33:0/618687187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400b520 con 0x7f7c340c1840 2023-11-06T20:44:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.371+0000 7f7c32ffd700 1 -- 172.21.15.33:0/618687187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2400b680 con 0x7f7c340c1840 2023-11-06T20:44:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.371+0000 7f7c32ffd700 1 -- 172.21.15.33:0/618687187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400ba00 con 0x7f7c340c1840 2023-11-06T20:44:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.372+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/618687187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 msgr2=0x7f7c340a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.372+0000 7f7c3a5ff700 1 --2- 172.21.15.33:0/618687187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 0x7f7c340a9de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7c340c55f0 tx=0x7f7c240090f0).stop 2023-11-06T20:44:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.372+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/618687187 shutdown_connections 2023-11-06T20:44:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.372+0000 7f7c3a5ff700 1 --2- 172.21.15.33:0/618687187 >> v2:172.21.15.69:3300/0 conn(0x7f7c340c6460 0x7f7c340bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.372+0000 7f7c3a5ff700 1 --2- 172.21.15.33:0/618687187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 0x7f7c340a9de0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.372+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/618687187 >> 172.21.15.33:0/618687187 conn(0x7f7c3405ec80 msgr2=0x7f7c340cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.372+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/618687187 shutdown_connections 2023-11-06T20:44:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.372+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/618687187 wait complete. 2023-11-06T20:44:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.373+0000 7f7c3a5ff700 1 Processor -- start 2023-11-06T20:44:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.374+0000 7f7c3a5ff700 1 -- start start 2023-11-06T20:44:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.374+0000 7f7c3a5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c340aa2d0 0x7f7c3414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.374+0000 7f7c3a5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 0x7f7c34164580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c3a5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c340c6460 0x7f7c34164a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c3a5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c340b0960 con 0x7f7c340c1840 2023-11-06T20:44:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c3a5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3414a650 con 0x7f7c340c6460 2023-11-06T20:44:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c3a5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3414d4f0 con 0x7f7c340aa2d0 2023-11-06T20:44:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 0x7f7c34164580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 0x7f7c34164580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53976/0 (socket says 172.21.15.33:53976) 2023-11-06T20:44:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c337fe700 1 -- 172.21.15.33:0/2389341334 learned_addr learned my addr 172.21.15.33:0/2389341334 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c33fff700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c340aa2d0 0x7f7c3414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c38b9c700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c340c6460 0x7f7c34164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c337fe700 1 -- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c340aa2d0 msgr2=0x7f7c3414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c337fe700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c340aa2d0 0x7f7c3414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c337fe700 1 -- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c340c6460 msgr2=0x7f7c34164a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c337fe700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c340c6460 0x7f7c34164a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c337fe700 1 -- 172.21.15.33:0/2389341334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c24009370 con 0x7f7c340c1840 2023-11-06T20:44:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.375+0000 7f7c337fe700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 0x7f7c34164580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f7c2800c440 tx=0x7f7c28007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.376+0000 7f7c1bfff700 1 -- 172.21.15.33:0/2389341334 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28007020 con 0x7f7c340c1840 2023-11-06T20:44:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.376+0000 7f7c1bfff700 1 -- 172.21.15.33:0/2389341334 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c28007180 con 0x7f7c340c1840 2023-11-06T20:44:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.376+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/2389341334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3414d7d0 con 0x7f7c340c1840 2023-11-06T20:44:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.376+0000 7f7c1bfff700 1 -- 172.21.15.33:0/2389341334 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28007520 con 0x7f7c340c1840 2023-11-06T20:44:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.376+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/2389341334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c340c2350 con 0x7f7c340c1840 2023-11-06T20:44:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.377+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/2389341334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3406c6c0 con 0x7f7c340c1840 2023-11-06T20:44:02.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.378+0000 7f7c1bfff700 1 -- 172.21.15.33:0/2389341334 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7c280068b0 con 0x7f7c340c1840 2023-11-06T20:44:02.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.378+0000 7f7c1bfff700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7c2005f150 0x7f7c20061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.379+0000 7f7c1bfff700 1 -- 172.21.15.33:0/2389341334 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f7c28079880 con 0x7f7c340c1840 2023-11-06T20:44:02.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.379+0000 7f7c33fff700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7c2005f150 0x7f7c20061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.379+0000 7f7c33fff700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7c2005f150 0x7f7c20061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7c24009130 tx=0x7f7c24017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:02.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.383+0000 7f7c1bfff700 1 -- 172.21.15.33:0/2389341334 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7c2804ade0 con 0x7f7c340c1840 2023-11-06T20:44:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.605+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/2389341334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c34063f20 con 0x7f7c340c1840 2023-11-06T20:44:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.606+0000 7f7c1bfff700 1 -- 172.21.15.33:0/2389341334 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7f7c2804e400 con 0x7f7c340c1840 2023-11-06T20:44:02.605 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:02.605 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4123875619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123875619},{"type":"v1","addr":"172.21.15.33:6834","nonce":4123875619}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3506554570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3506554570},{"type":"v1","addr":"172.21.15.33:6835","nonce":3506554570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/239985290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":239985290},{"type":"v1","addr":"172.21.15.69:6835","nonce":239985290}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2681789033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2681789033},{"type":"v1","addr":"172.21.15.69:6837","nonce":2681789033}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-06T20:44:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7c2005f150 msgr2=0x7f7c20061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7c2005f150 0x7f7c20061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7c24009130 tx=0x7f7c24017000).stop 2023-11-06T20:44:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/2389341334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 msgr2=0x7f7c34164580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 0x7f7c34164580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f7c2800c440 tx=0x7f7c28007dd0).stop 2023-11-06T20:44:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/2389341334 shutdown_connections 2023-11-06T20:44:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7c2005f150 0x7f7c20061610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c340aa2d0 0x7f7c3414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c340c6460 0x7f7c34164a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 --2- 172.21.15.33:0/2389341334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c340c1840 0x7f7c34164580 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/2389341334 >> 172.21.15.33:0/2389341334 conn(0x7f7c3405ec80 msgr2=0x7f7c340ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/2389341334 shutdown_connections 2023-11-06T20:44:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.608+0000 7f7c3a5ff700 1 -- 172.21.15.33:0/2389341334 wait complete. 2023-11-06T20:44:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2023-11-06T20:44:02.624 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:02.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.821+0000 7f84313ff700 1 Processor -- start 2023-11-06T20:44:02.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.821+0000 7f84313ff700 1 -- start start 2023-11-06T20:44:02.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.822+0000 7f84313ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f842c0c1840 0x7f842c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.822+0000 7f84313ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f842c0aa2d0 0x7f842c0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.822+0000 7f84313ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0c6240 0x7f842c157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.822+0000 7f84313ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c154960 con 0x7f842c0c1840 2023-11-06T20:44:02.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.822+0000 7f84313ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c1554a0 con 0x7f842c0aa2d0 2023-11-06T20:44:02.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.822+0000 7f84313ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c155930 con 0x7f842c0c6240 2023-11-06T20:44:02.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f842c0c1840 0x7f842c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f842c0c1840 0x7f842c0a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53984/0 (socket says 172.21.15.33:53984) 2023-11-06T20:44:02.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842affd700 1 -- 172.21.15.33:0/863139676 learned_addr learned my addr 172.21.15.33:0/863139676 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:02.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842b7fe700 1 --2- 172.21.15.33:0/863139676 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0c6240 0x7f842c157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842a7fc700 1 --2- 172.21.15.33:0/863139676 >> v2:172.21.15.69:3300/0 conn(0x7f842c0aa2d0 0x7f842c0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842affd700 1 -- 172.21.15.33:0/863139676 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0c6240 msgr2=0x7f842c157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842affd700 1 --2- 172.21.15.33:0/863139676 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0c6240 0x7f842c157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842affd700 1 -- 172.21.15.33:0/863139676 >> v2:172.21.15.69:3300/0 conn(0x7f842c0aa2d0 msgr2=0x7f842c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842affd700 1 --2- 172.21.15.33:0/863139676 >> v2:172.21.15.69:3300/0 conn(0x7f842c0aa2d0 0x7f842c0c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842affd700 1 -- 172.21.15.33:0/863139676 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0c6690 con 0x7f842c0c1840 2023-11-06T20:44:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842b7fe700 1 --2- 172.21.15.33:0/863139676 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0c6240 0x7f842c157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842a7fc700 1 --2- 172.21.15.33:0/863139676 >> v2:172.21.15.69:3300/0 conn(0x7f842c0aa2d0 0x7f842c0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f842affd700 1 --2- 172.21.15.33:0/863139676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842c0c1840 0x7f842c0a9de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f842c0adb40 tx=0x7f841c0090f0).ready entity=mon.0 client_cookie=9e74b460860bf379 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.823+0000 7f8429ffb700 1 -- 172.21.15.33:0/863139676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00b600 con 0x7f842c0c1840 2023-11-06T20:44:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f8429ffb700 1 -- 172.21.15.33:0/863139676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c00b760 con 0x7f842c0c1840 2023-11-06T20:44:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f8429ffb700 1 -- 172.21.15.33:0/863139676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00bae0 con 0x7f842c0c1840 2023-11-06T20:44:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f84313ff700 1 -- 172.21.15.33:0/863139676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842c0c1840 msgr2=0x7f842c0a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f84313ff700 1 --2- 172.21.15.33:0/863139676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842c0c1840 0x7f842c0a9de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f842c0adb40 tx=0x7f841c0090f0).stop 2023-11-06T20:44:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f84313ff700 1 -- 172.21.15.33:0/863139676 shutdown_connections 2023-11-06T20:44:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f84313ff700 1 --2- 172.21.15.33:0/863139676 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0c6240 0x7f842c157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f84313ff700 1 --2- 172.21.15.33:0/863139676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842c0c1840 0x7f842c0a9de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f84313ff700 1 --2- 172.21.15.33:0/863139676 >> v2:172.21.15.69:3300/0 conn(0x7f842c0aa2d0 0x7f842c0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f84313ff700 1 -- 172.21.15.33:0/863139676 >> 172.21.15.33:0/863139676 conn(0x7f842c05ec80 msgr2=0x7f842c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f84313ff700 1 -- 172.21.15.33:0/863139676 shutdown_connections 2023-11-06T20:44:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.824+0000 7f84313ff700 1 -- 172.21.15.33:0/863139676 wait complete. 2023-11-06T20:44:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.826+0000 7f84313ff700 1 Processor -- start 2023-11-06T20:44:02.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.826+0000 7f84313ff700 1 -- start start 2023-11-06T20:44:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f84313ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0aa2d0 0x7f842c167980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f84313ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c0c1840 0x7f842c13d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f84313ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842c0c6240 0x7f842c0aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f84313ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c1674f0 con 0x7f842c0c6240 2023-11-06T20:44:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f84313ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c167e70 con 0x7f842c0c1840 2023-11-06T20:44:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f84313ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c13dca0 con 0x7f842c0aa2d0 2023-11-06T20:44:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f842affd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0aa2d0 0x7f842c167980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f842b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842c0c6240 0x7f842c0aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f842a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c0c1840 0x7f842c13d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f842affd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0aa2d0 0x7f842c167980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:57948/0 (socket says 172.21.15.33:57948) 2023-11-06T20:44:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f842affd700 1 -- 172.21.15.33:0/869348798 learned_addr learned my addr 172.21.15.33:0/869348798 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f842b7fe700 1 -- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0aa2d0 msgr2=0x7f842c167980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f842b7fe700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0aa2d0 0x7f842c167980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f842b7fe700 1 -- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c0c1840 msgr2=0x7f842c13d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f842b7fe700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c0c1840 0x7f842c13d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.827+0000 7f842b7fe700 1 -- 172.21.15.33:0/869348798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c009370 con 0x7f842c0c6240 2023-11-06T20:44:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.828+0000 7f842b7fe700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842c0c6240 0x7f842c0aeee0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f842400cf90 tx=0x7f8424007840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.828+0000 7f8412ffd700 1 -- 172.21.15.33:0/869348798 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424016470 con 0x7f842c0c6240 2023-11-06T20:44:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.828+0000 7f8412ffd700 1 -- 172.21.15.33:0/869348798 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84240165d0 con 0x7f842c0c6240 2023-11-06T20:44:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.828+0000 7f8412ffd700 1 -- 172.21.15.33:0/869348798 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424016970 con 0x7f842c0c6240 2023-11-06T20:44:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.828+0000 7f84313ff700 1 -- 172.21.15.33:0/869348798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c1668e0 con 0x7f842c0c6240 2023-11-06T20:44:02.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.828+0000 7f84313ff700 1 -- 172.21.15.33:0/869348798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c166ea0 con 0x7f842c0c6240 2023-11-06T20:44:02.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.829+0000 7f84313ff700 1 -- 172.21.15.33:0/869348798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c1676a0 con 0x7f842c0c6240 2023-11-06T20:44:02.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.830+0000 7f8412ffd700 1 -- 172.21.15.33:0/869348798 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8424013070 con 0x7f842c0c6240 2023-11-06T20:44:02.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.831+0000 7f8412ffd700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f841805f150 0x7f8418061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:02.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.832+0000 7f8412ffd700 1 -- 172.21.15.33:0/869348798 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f842407a7a0 con 0x7f842c0c6240 2023-11-06T20:44:02.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.832+0000 7f842affd700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f841805f150 0x7f8418061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:02.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.832+0000 7f842affd700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f841805f150 0x7f8418061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f841c00b3c0 tx=0x7f841c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:02.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:02.836+0000 7f8412ffd700 1 -- 172.21.15.33:0/869348798 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f842c1676a0 con 0x7f842c0c6240 2023-11-06T20:44:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.016+0000 7f84313ff700 1 -- 172.21.15.33:0/869348798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f842c1676a0 con 0x7f842c0c6240 2023-11-06T20:44:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.017+0000 7f8412ffd700 1 -- 172.21.15.33:0/869348798 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13757 (secure 0 0 0) 0x7f842c1676a0 con 0x7f842c0c6240 2023-11-06T20:44:03.016 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:03.016 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:01.859081+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 -- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f841805f150 msgr2=0x7f8418061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f841805f150 0x7f8418061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f841c00b3c0 tx=0x7f841c017000).stop 2023-11-06T20:44:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 -- 172.21.15.33:0/869348798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842c0c6240 msgr2=0x7f842c0aeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842c0c6240 0x7f842c0aeee0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f842400cf90 tx=0x7f8424007840).stop 2023-11-06T20:44:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 -- 172.21.15.33:0/869348798 shutdown_connections 2023-11-06T20:44:03.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f841805f150 0x7f8418061610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f842c0aa2d0 0x7f842c167980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f842c0c6240 0x7f842c0aeee0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 --2- 172.21.15.33:0/869348798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c0c1840 0x7f842c13d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 -- 172.21.15.33:0/869348798 >> 172.21.15.33:0/869348798 conn(0x7f842c05ec80 msgr2=0x7f842c152a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 -- 172.21.15.33:0/869348798 shutdown_connections 2023-11-06T20:44:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.019+0000 7f84313ff700 1 -- 172.21.15.33:0/869348798 wait complete. 2023-11-06T20:44:03.036 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T20:44:03.037 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:44:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.237+0000 7fa1539ff700 1 Processor -- start 2023-11-06T20:44:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.237+0000 7fa1539ff700 1 -- start start 2023-11-06T20:44:03.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.239+0000 7fa1539ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa14c154330 0x7fa14c0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.239+0000 7fa1539ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa14c154710 0x7fa14c0c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.239+0000 7fa1539ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14c0c6b60 con 0x7fa14c0c7ff0 2023-11-06T20:44:03.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.239+0000 7fa1539ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14c0c5cd0 con 0x7fa14c154330 2023-11-06T20:44:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.239+0000 7fa1539ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14c0c5f50 con 0x7fa14c154710 2023-11-06T20:44:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.239+0000 7fa15179b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa14c154330 0x7fa14c0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.239+0000 7fa15179b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa14c154330 0x7fa14c0c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54002/0 (socket says 172.21.15.33:54002) 2023-11-06T20:44:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.239+0000 7fa15179b700 1 -- 172.21.15.33:0/2457239375 learned_addr learned my addr 172.21.15.33:0/2457239375 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.239+0000 7fa150f9a700 1 --2- 172.21.15.33:0/2457239375 >> v2:172.21.15.69:3300/0 conn(0x7fa14c154710 0x7fa14c0c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.240+0000 7fa15179b700 1 -- 172.21.15.33:0/2457239375 >> v2:172.21.15.69:3300/0 conn(0x7fa14c154710 msgr2=0x7fa14c0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:03.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.240+0000 7fa15179b700 1 --2- 172.21.15.33:0/2457239375 >> v2:172.21.15.69:3300/0 conn(0x7fa14c154710 0x7fa14c0c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.240+0000 7fa15179b700 1 -- 172.21.15.33:0/2457239375 >> v1:172.21.15.69:6789/0 conn(0x7fa14c0c7ff0 legacy=0x7fa14c0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.240+0000 7fa15179b700 1 -- 172.21.15.33:0/2457239375 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0c6230 con 0x7fa14c154330 2023-11-06T20:44:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.240+0000 7fa15179b700 1 --2- 172.21.15.33:0/2457239375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c154330 0x7fa14c0c57e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa14c0c8d10 tx=0x7fa1480090f0).ready entity=mon.0 client_cookie=df964648076ae04b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.240+0000 7fa143fff700 1 -- 172.21.15.33:0/2457239375 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14800b600 con 0x7fa14c154330 2023-11-06T20:44:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.240+0000 7fa143fff700 1 -- 172.21.15.33:0/2457239375 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14800b760 con 0x7fa14c154330 2023-11-06T20:44:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.240+0000 7fa143fff700 1 -- 172.21.15.33:0/2457239375 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14800bae0 con 0x7fa14c154330 2023-11-06T20:44:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.240+0000 7fa1539ff700 1 -- 172.21.15.33:0/2457239375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c154330 msgr2=0x7fa14c0c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.240+0000 7fa1539ff700 1 --2- 172.21.15.33:0/2457239375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c154330 0x7fa14c0c57e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa14c0c8d10 tx=0x7fa1480090f0).stop 2023-11-06T20:44:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.241+0000 7fa1539ff700 1 -- 172.21.15.33:0/2457239375 shutdown_connections 2023-11-06T20:44:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.241+0000 7fa1539ff700 1 --2- 172.21.15.33:0/2457239375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c154330 0x7fa14c0c57e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.241+0000 7fa1539ff700 1 --2- 172.21.15.33:0/2457239375 >> v2:172.21.15.69:3300/0 conn(0x7fa14c154710 0x7fa14c0c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.241+0000 7fa1539ff700 1 -- 172.21.15.33:0/2457239375 >> 172.21.15.33:0/2457239375 conn(0x7fa14c0cb910 msgr2=0x7fa14c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.241+0000 7fa1539ff700 1 -- 172.21.15.33:0/2457239375 shutdown_connections 2023-11-06T20:44:03.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.241+0000 7fa1539ff700 1 -- 172.21.15.33:0/2457239375 wait complete. 2023-11-06T20:44:03.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.242+0000 7fa1539ff700 1 Processor -- start 2023-11-06T20:44:03.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.242+0000 7fa1539ff700 1 -- start start 2023-11-06T20:44:03.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.243+0000 7fa1539ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa14c0c7ff0 0x7fa14c1649b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.243+0000 7fa1539ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa14c154330 0x7fa14c1689d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.243+0000 7fa1539ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c154710 0x7fa14c149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.243+0000 7fa1539ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c168ec0 con 0x7fa14c154710 2023-11-06T20:44:03.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.243+0000 7fa1539ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0adfe0 con 0x7fa14c154330 2023-11-06T20:44:03.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.243+0000 7fa1539ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c165000 con 0x7fa14c0c7ff0 2023-11-06T20:44:03.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa15179b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa14c0c7ff0 0x7fa14c1649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa151f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c154710 0x7fa14c149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa15179b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa14c0c7ff0 0x7fa14c1649b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:57952/0 (socket says 172.21.15.33:57952) 2023-11-06T20:44:03.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa15179b700 1 -- 172.21.15.33:0/217204781 learned_addr learned my addr 172.21.15.33:0/217204781 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:03.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa150f9a700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa14c154330 0x7fa14c1689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa151f9c700 1 -- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa14c0c7ff0 msgr2=0x7fa14c1649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa151f9c700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa14c0c7ff0 0x7fa14c1649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa151f9c700 1 -- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa14c154330 msgr2=0x7fa14c1689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa151f9c700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa14c154330 0x7fa14c1689d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa151f9c700 1 -- 172.21.15.33:0/217204781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa148009370 con 0x7fa14c154710 2023-11-06T20:44:03.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa151f9c700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c154710 0x7fa14c149670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa14400c790 tx=0x7fa144001460).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:03.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa1417fa700 1 -- 172.21.15.33:0/217204781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400f470 con 0x7fa14c154710 2023-11-06T20:44:03.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa1417fa700 1 -- 172.21.15.33:0/217204781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14400f5d0 con 0x7fa14c154710 2023-11-06T20:44:03.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa1417fa700 1 -- 172.21.15.33:0/217204781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400f950 con 0x7fa14c154710 2023-11-06T20:44:03.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.244+0000 7fa1539ff700 1 -- 172.21.15.33:0/217204781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14c1652e0 con 0x7fa14c154710 2023-11-06T20:44:03.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.245+0000 7fa1539ff700 1 -- 172.21.15.33:0/217204781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c1658a0 con 0x7fa14c154710 2023-11-06T20:44:03.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.246+0000 7fa1539ff700 1 -- 172.21.15.33:0/217204781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14c0ae200 con 0x7fa14c154710 2023-11-06T20:44:03.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.246+0000 7fa1417fa700 1 -- 172.21.15.33:0/217204781 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa14400fd00 con 0x7fa14c154710 2023-11-06T20:44:03.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.248+0000 7fa1417fa700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13005f150 0x7fa130061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.248+0000 7fa1417fa700 1 -- 172.21.15.33:0/217204781 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fa1440793c0 con 0x7fa14c154710 2023-11-06T20:44:03.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.248+0000 7fa15179b700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13005f150 0x7fa130061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.249+0000 7fa15179b700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13005f150 0x7fa130061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa14800b3c0 tx=0x7fa148017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:03.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.251+0000 7fa1417fa700 1 -- 172.21.15.33:0/217204781 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa144048af0 con 0x7fa14c154710 2023-11-06T20:44:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.431+0000 7fa1539ff700 1 -- 172.21.15.33:0/217204781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa14c0830b0 con 0x7fa14c154710 2023-11-06T20:44:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.432+0000 7fa1417fa700 1 -- 172.21.15.33:0/217204781 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa14c0ae200 con 0x7fa14c154710 2023-11-06T20:44:03.432 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:44:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.433+0000 7fa1539ff700 1 -- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13005f150 msgr2=0x7fa130061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.433+0000 7fa1539ff700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13005f150 0x7fa130061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa14800b3c0 tx=0x7fa148017000).stop 2023-11-06T20:44:03.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.433+0000 7fa1539ff700 1 -- 172.21.15.33:0/217204781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c154710 msgr2=0x7fa14c149670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.433+0000 7fa1539ff700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c154710 0x7fa14c149670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa14400c790 tx=0x7fa144001460).stop 2023-11-06T20:44:03.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.434+0000 7fa1539ff700 1 -- 172.21.15.33:0/217204781 shutdown_connections 2023-11-06T20:44:03.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.434+0000 7fa1539ff700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13005f150 0x7fa130061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.434+0000 7fa1539ff700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa14c0c7ff0 0x7fa14c1649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.434+0000 7fa1539ff700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14c154710 0x7fa14c149670 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.434+0000 7fa1539ff700 1 --2- 172.21.15.33:0/217204781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa14c154330 0x7fa14c1689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.434+0000 7fa1539ff700 1 -- 172.21.15.33:0/217204781 >> 172.21.15.33:0/217204781 conn(0x7fa14c0cb910 msgr2=0x7fa14c0ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:03.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.434+0000 7fa1539ff700 1 -- 172.21.15.33:0/217204781 shutdown_connections 2023-11-06T20:44:03.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.434+0000 7fa1539ff700 1 -- 172.21.15.33:0/217204781 wait complete. 2023-11-06T20:44:03.450 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:44:03.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.648+0000 7f9ba67ff700 1 Processor -- start 2023-11-06T20:44:03.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.648+0000 7f9ba67ff700 1 -- start start 2023-11-06T20:44:03.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.650+0000 7f9ba67ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ba0154710 0x7f9ba00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.650+0000 7f9ba67ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba00c6b60 con 0x7f9ba00c7ff0 2023-11-06T20:44:03.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.650+0000 7f9ba67ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba00c7700 con 0x7f9ba0154330 2023-11-06T20:44:03.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.650+0000 7f9ba67ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba00c5cd0 con 0x7f9ba0154710 2023-11-06T20:44:03.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.650+0000 7f9b9f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ba0154710 0x7f9ba00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.650+0000 7f9ba4d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9ba00c7ff0 0x7f9ba00646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41192/0 (socket says 172.21.15.33:41192) 2023-11-06T20:44:03.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.650+0000 7f9ba4d9c700 1 -- 172.21.15.33:0/2643171956 learned_addr learned my addr 172.21.15.33:0/2643171956 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:03.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.650+0000 7f9b9f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ba0154710 0x7f9ba00c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54014/0 (socket says 172.21.15.33:54014) 2023-11-06T20:44:03.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.651+0000 7f9b9f7fe700 1 -- 172.21.15.33:0/2643171956 >> v1:172.21.15.69:6789/0 conn(0x7f9ba0154330 legacy=0x7f9ba00c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.651+0000 7f9b9f7fe700 1 -- 172.21.15.33:0/2643171956 >> v1:172.21.15.33:6789/0 conn(0x7f9ba00c7ff0 legacy=0x7f9ba00646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.651+0000 7f9b9f7fe700 1 -- 172.21.15.33:0/2643171956 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba00c5f50 con 0x7f9ba0154710 2023-11-06T20:44:03.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.651+0000 7f9b9effd700 1 -- 172.21.15.33:0/2643171956 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326434334 0 0) 0x7f9ba00c6b60 con 0x7f9ba00c7ff0 2023-11-06T20:44:03.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.651+0000 7f9b9f7fe700 1 --2- 172.21.15.33:0/2643171956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154710 0x7f9ba00c57e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f9b94009760 tx=0x7f9b94009060).ready entity=mon.0 client_cookie=aaea0cafac2835bc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:03.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.651+0000 7f9b9effd700 1 -- 172.21.15.33:0/2643171956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400b580 con 0x7f9ba0154710 2023-11-06T20:44:03.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.651+0000 7f9b9effd700 1 -- 172.21.15.33:0/2643171956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9400b6e0 con 0x7f9ba0154710 2023-11-06T20:44:03.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.651+0000 7f9b9effd700 1 -- 172.21.15.33:0/2643171956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400ba60 con 0x7f9ba0154710 2023-11-06T20:44:03.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.651+0000 7f9ba67ff700 1 -- 172.21.15.33:0/2643171956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154710 msgr2=0x7f9ba00c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.651+0000 7f9ba67ff700 1 --2- 172.21.15.33:0/2643171956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154710 0x7f9ba00c57e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f9b94009760 tx=0x7f9b94009060).stop 2023-11-06T20:44:03.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.652+0000 7f9ba67ff700 1 -- 172.21.15.33:0/2643171956 shutdown_connections 2023-11-06T20:44:03.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.652+0000 7f9ba67ff700 1 --2- 172.21.15.33:0/2643171956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154710 0x7f9ba00c57e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.652+0000 7f9ba67ff700 1 -- 172.21.15.33:0/2643171956 >> 172.21.15.33:0/2643171956 conn(0x7f9ba00cb910 msgr2=0x7f9ba00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:03.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.652+0000 7f9ba67ff700 1 -- 172.21.15.33:0/2643171956 shutdown_connections 2023-11-06T20:44:03.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.652+0000 7f9ba67ff700 1 -- 172.21.15.33:0/2643171956 wait complete. 2023-11-06T20:44:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.653+0000 7f9ba67ff700 1 Processor -- start 2023-11-06T20:44:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.653+0000 7f9ba67ff700 1 -- start start 2023-11-06T20:44:03.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.654+0000 7f9ba67ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ba00c7ff0 0x7f9ba00ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.654+0000 7f9ba67ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154330 0x7f9ba0165f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.654+0000 7f9ba67ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ba0154710 0x7f9ba00b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.654+0000 7f9ba67ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba01393b0 con 0x7f9ba0154330 2023-11-06T20:44:03.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.654+0000 7f9ba67ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c38f0 con 0x7f9ba00c7ff0 2023-11-06T20:44:03.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.654+0000 7f9ba67ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00b0df0 con 0x7f9ba0154710 2023-11-06T20:44:03.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.654+0000 7f9b9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154330 0x7f9ba0165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154330 0x7f9ba0165f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54020/0 (socket says 172.21.15.33:54020) 2023-11-06T20:44:03.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b9f7fe700 1 -- 172.21.15.33:0/1829187235 learned_addr learned my addr 172.21.15.33:0/1829187235 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:03.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9ba4d9c700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ba0154710 0x7f9ba00b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b9ffff700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ba00c7ff0 0x7f9ba00ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b9f7fe700 1 -- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ba0154710 msgr2=0x7f9ba00b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b9f7fe700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ba0154710 0x7f9ba00b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b9f7fe700 1 -- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ba00c7ff0 msgr2=0x7f9ba00ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b9f7fe700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ba00c7ff0 0x7f9ba00ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b9f7fe700 1 -- 172.21.15.33:0/1829187235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940092a0 con 0x7f9ba0154330 2023-11-06T20:44:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9ba4d9c700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ba0154710 0x7f9ba00b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b9f7fe700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154330 0x7f9ba0165f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9b94011000 tx=0x7f9b94017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b87fff700 1 -- 172.21.15.33:0/1829187235 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400b850 con 0x7f9ba0154330 2023-11-06T20:44:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b87fff700 1 -- 172.21.15.33:0/1829187235 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9400b9b0 con 0x7f9ba0154330 2023-11-06T20:44:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9b87fff700 1 -- 172.21.15.33:0/1829187235 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9401c630 con 0x7f9ba0154330 2023-11-06T20:44:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.655+0000 7f9ba67ff700 1 -- 172.21.15.33:0/1829187235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba00c2fc0 con 0x7f9ba0154330 2023-11-06T20:44:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.656+0000 7f9ba67ff700 1 -- 172.21.15.33:0/1829187235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba00c3580 con 0x7f9ba0154330 2023-11-06T20:44:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.657+0000 7f9ba67ff700 1 -- 172.21.15.33:0/1829187235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba00c3a80 con 0x7f9ba0154330 2023-11-06T20:44:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.657+0000 7f9b87fff700 1 -- 172.21.15.33:0/1829187235 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9b9401c9e0 con 0x7f9ba0154330 2023-11-06T20:44:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.658+0000 7f9b87fff700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b8c05f150 0x7f9b8c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.659+0000 7f9b87fff700 1 -- 172.21.15.33:0/1829187235 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f9b9407bd10 con 0x7f9ba0154330 2023-11-06T20:44:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.659+0000 7f9b9ffff700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b8c05f150 0x7f9b8c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.659+0000 7f9b9ffff700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b8c05f150 0x7f9b8c061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9b90003900 tx=0x7f9b90007d40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.663+0000 7f9b87fff700 1 -- 172.21.15.33:0/1829187235 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9b9404d270 con 0x7f9ba0154330 2023-11-06T20:44:03.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.844+0000 7f9ba67ff700 1 -- 172.21.15.33:0/1829187235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9ba00652d0 con 0x7f9ba0154330 2023-11-06T20:44:03.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.845+0000 7f9b87fff700 1 -- 172.21.15.33:0/1829187235 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9b9402a020 con 0x7f9ba0154330 2023-11-06T20:44:03.844 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:44:03.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 -- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b8c05f150 msgr2=0x7f9b8c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b8c05f150 0x7f9b8c061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9b90003900 tx=0x7f9b90007d40).stop 2023-11-06T20:44:03.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 -- 172.21.15.33:0/1829187235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154330 msgr2=0x7f9ba0165f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:03.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154330 0x7f9ba0165f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9b94011000 tx=0x7f9b94017000).stop 2023-11-06T20:44:03.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 -- 172.21.15.33:0/1829187235 shutdown_connections 2023-11-06T20:44:03.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b8c05f150 0x7f9b8c061610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ba0154710 0x7f9ba00b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ba00c7ff0 0x7f9ba00ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 --2- 172.21.15.33:0/1829187235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba0154330 0x7f9ba0165f80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 -- 172.21.15.33:0/1829187235 >> 172.21.15.33:0/1829187235 conn(0x7f9ba00cb910 msgr2=0x7f9ba0149d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 -- 172.21.15.33:0/1829187235 shutdown_connections 2023-11-06T20:44:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:03.847+0000 7f9ba67ff700 1 -- 172.21.15.33:0/1829187235 wait complete. 2023-11-06T20:44:03.864 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T20:44:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.060+0000 7f214cdff700 1 Processor -- start 2023-11-06T20:44:04.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.060+0000 7f214cdff700 1 -- start start 2023-11-06T20:44:04.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.062+0000 7f214cdff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f21480c61d0 0x7f21480c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.062+0000 7f214cdff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21481554a0 con 0x7f21480c1840 2023-11-06T20:44:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.062+0000 7f214cdff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2148155930 con 0x7f21480c5d50 2023-11-06T20:44:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.062+0000 7f214cdff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2148154960 con 0x7f21480c61d0 2023-11-06T20:44:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.062+0000 7f214659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f21480c1840 0x7f21480a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41204/0 (socket says 172.21.15.33:41204) 2023-11-06T20:44:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.062+0000 7f214659c700 1 -- 172.21.15.33:0/2249520037 learned_addr learned my addr 172.21.15.33:0/2249520037 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.062+0000 7f2146d9d700 1 --2- 172.21.15.33:0/2249520037 >> v2:172.21.15.33:3300/0 conn(0x7f21480c61d0 0x7f21480c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.062+0000 7f214559a700 1 -- 172.21.15.33:0/2249520037 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949643504 0 0) 0x7f21481554a0 con 0x7f21480c1840 2023-11-06T20:44:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.063+0000 7f214559a700 1 -- 172.21.15.33:0/2249520037 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21300033f0 con 0x7f21480c1840 2023-11-06T20:44:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.063+0000 7f2146d9d700 1 -- 172.21.15.33:0/2249520037 >> v1:172.21.15.69:6789/0 conn(0x7f21480c5d50 legacy=0x7f21480bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.063+0000 7f2146d9d700 1 -- 172.21.15.33:0/2249520037 >> v1:172.21.15.33:6789/0 conn(0x7f21480c1840 legacy=0x7f21480a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.063+0000 7f2146d9d700 1 -- 172.21.15.33:0/2249520037 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21480aa4f0 con 0x7f21480c61d0 2023-11-06T20:44:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.063+0000 7f2146d9d700 1 --2- 172.21.15.33:0/2249520037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 0x7f21480c0480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2140009e60 tx=0x7f2140009780).ready entity=mon.0 client_cookie=be8eb1f09a509352 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.063+0000 7f214559a700 1 -- 172.21.15.33:0/2249520037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214000e070 con 0x7f21480c61d0 2023-11-06T20:44:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.063+0000 7f214559a700 1 -- 172.21.15.33:0/2249520037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214000bb60 con 0x7f21480c61d0 2023-11-06T20:44:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.063+0000 7f214559a700 1 -- 172.21.15.33:0/2249520037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2140012410 con 0x7f21480c61d0 2023-11-06T20:44:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.063+0000 7f214cdff700 1 -- 172.21.15.33:0/2249520037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 msgr2=0x7f21480c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.063+0000 7f214cdff700 1 --2- 172.21.15.33:0/2249520037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 0x7f21480c0480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2140009e60 tx=0x7f2140009780).stop 2023-11-06T20:44:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.064+0000 7f214cdff700 1 -- 172.21.15.33:0/2249520037 shutdown_connections 2023-11-06T20:44:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.064+0000 7f214cdff700 1 --2- 172.21.15.33:0/2249520037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 0x7f21480c0480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.064+0000 7f214cdff700 1 -- 172.21.15.33:0/2249520037 >> 172.21.15.33:0/2249520037 conn(0x7f214805ec80 msgr2=0x7f21480cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.064+0000 7f214cdff700 1 -- 172.21.15.33:0/2249520037 shutdown_connections 2023-11-06T20:44:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.064+0000 7f214cdff700 1 -- 172.21.15.33:0/2249520037 wait complete. 2023-11-06T20:44:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.065+0000 7f214cdff700 1 Processor -- start 2023-11-06T20:44:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.065+0000 7f214cdff700 1 -- start start 2023-11-06T20:44:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.066+0000 7f214cdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f21480c1840 0x7f21480b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.066+0000 7f214cdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21480c5d50 0x7f21481645f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.066+0000 7f214cdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 0x7f21481682c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.066+0000 7f214cdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2148159170 con 0x7f21480c61d0 2023-11-06T20:44:04.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.066+0000 7f214cdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2148158840 con 0x7f21480c5d50 2023-11-06T20:44:04.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.066+0000 7f214cdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2148158b20 con 0x7f21480c1840 2023-11-06T20:44:04.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.066+0000 7f2146d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 0x7f21481682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.066+0000 7f2146d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 0x7f21481682c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54036/0 (socket says 172.21.15.33:54036) 2023-11-06T20:44:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.066+0000 7f2146d9d700 1 -- 172.21.15.33:0/3930662646 learned_addr learned my addr 172.21.15.33:0/3930662646 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f214659c700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f21480c1840 0x7f21480b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f2145d9b700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21480c5d50 0x7f21481645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f2146d9d700 1 -- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f21480c1840 msgr2=0x7f21480b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f2146d9d700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f21480c1840 0x7f21480b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f2146d9d700 1 -- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21480c5d50 msgr2=0x7f21481645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f2146d9d700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21480c5d50 0x7f21481645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f2146d9d700 1 -- 172.21.15.33:0/3930662646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2140009a20 con 0x7f21480c61d0 2023-11-06T20:44:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f2146d9d700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 0x7f21481682c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f214000b6c0 tx=0x7f2148155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f21367fc700 1 -- 172.21.15.33:0/3930662646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214001d630 con 0x7f21480c61d0 2023-11-06T20:44:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f21367fc700 1 -- 172.21.15.33:0/3930662646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214001d790 con 0x7f21480c61d0 2023-11-06T20:44:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f214cdff700 1 -- 172.21.15.33:0/3930662646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2148158e00 con 0x7f21480c61d0 2023-11-06T20:44:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.067+0000 7f21367fc700 1 -- 172.21.15.33:0/3930662646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214001db10 con 0x7f21480c61d0 2023-11-06T20:44:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.068+0000 7f214cdff700 1 -- 172.21.15.33:0/3930662646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2148164ae0 con 0x7f21480c61d0 2023-11-06T20:44:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.069+0000 7f214cdff700 1 -- 172.21.15.33:0/3930662646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2148159500 con 0x7f21480c61d0 2023-11-06T20:44:04.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.069+0000 7f21367fc700 1 -- 172.21.15.33:0/3930662646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2140027460 con 0x7f21480c61d0 2023-11-06T20:44:04.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.070+0000 7f21367fc700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f212805f150 0x7f2128061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.070+0000 7f21367fc700 1 -- 172.21.15.33:0/3930662646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f214007bec0 con 0x7f21480c61d0 2023-11-06T20:44:04.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.071+0000 7f214659c700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f212805f150 0x7f2128061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.071+0000 7f214659c700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f212805f150 0x7f2128061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2138005750 tx=0x7f2138009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:04.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.074+0000 7f21367fc700 1 -- 172.21.15.33:0/3930662646 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f214004d3c0 con 0x7f21480c61d0 2023-11-06T20:44:04.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.256+0000 7f214cdff700 1 -- 172.21.15.33:0/3930662646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f214806c6c0 con 0x7f21480c61d0 2023-11-06T20:44:04.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.256+0000 7f21367fc700 1 -- 172.21.15.33:0/3930662646 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+143 (secure 0 0 0) 0x7f21400509e0 con 0x7f21480c61d0 2023-11-06T20:44:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T20:44:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T20:44:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T20:44:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T20:44:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T20:44:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:04.260 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T20:44:04.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 -- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f212805f150 msgr2=0x7f2128061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f212805f150 0x7f2128061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2138005750 tx=0x7f2138009040).stop 2023-11-06T20:44:04.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 -- 172.21.15.33:0/3930662646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 msgr2=0x7f21481682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 0x7f21481682c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f214000b6c0 tx=0x7f2148155410).stop 2023-11-06T20:44:04.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 -- 172.21.15.33:0/3930662646 shutdown_connections 2023-11-06T20:44:04.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f212805f150 0x7f2128061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f21480c1840 0x7f21480b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480c61d0 0x7f21481682c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 --2- 172.21.15.33:0/3930662646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21480c5d50 0x7f21481645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 -- 172.21.15.33:0/3930662646 >> 172.21.15.33:0/3930662646 conn(0x7f214805ec80 msgr2=0x7f21480ad290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:04.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 -- 172.21.15.33:0/3930662646 shutdown_connections 2023-11-06T20:44:04.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.258+0000 7f214cdff700 1 -- 172.21.15.33:0/3930662646 wait complete. 2023-11-06T20:44:04.275 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T20:44:04.275 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T20:44:04.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:44:04.283 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:44:04.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.477+0000 7f10105ff700 1 Processor -- start 2023-11-06T20:44:04.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.477+0000 7f10105ff700 1 -- start start 2023-11-06T20:44:04.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.478+0000 7f10105ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1008154720 0x7f10080aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.478+0000 7f10105ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008158720 0x7f1008063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.478+0000 7f10105ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10080c0e00 con 0x7f1008154340 2023-11-06T20:44:04.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.478+0000 7f10105ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1008158bf0 con 0x7f1008154720 2023-11-06T20:44:04.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.478+0000 7f10105ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1008158ed0 con 0x7f1008158720 2023-11-06T20:44:04.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100db9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1008154720 0x7f10080aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100db9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1008154720 0x7f10080aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54052/0 (socket says 172.21.15.33:54052) 2023-11-06T20:44:04.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100e39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1008154340 0x7f1008158010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41212/0 (socket says 172.21.15.33:41212) 2023-11-06T20:44:04.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100db9a700 1 -- 172.21.15.33:0/1732919481 learned_addr learned my addr 172.21.15.33:0/1732919481 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:04.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100eb9c700 1 --2- 172.21.15.33:0/1732919481 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008158720 0x7f1008063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100db9a700 1 -- 172.21.15.33:0/1732919481 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008158720 msgr2=0x7f1008063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:04.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100db9a700 1 --2- 172.21.15.33:0/1732919481 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008158720 0x7f1008063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100db9a700 1 -- 172.21.15.33:0/1732919481 >> v1:172.21.15.33:6789/0 conn(0x7f1008154340 legacy=0x7f1008158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100db9a700 1 -- 172.21.15.33:0/1732919481 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10080c5d80 con 0x7f1008154720 2023-11-06T20:44:04.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100d399700 1 -- 172.21.15.33:0/1732919481 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231637830 0 0) 0x7f10080c0e00 con 0x7f1008154340 2023-11-06T20:44:04.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100db9a700 1 --2- 172.21.15.33:0/1732919481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008154720 0x7f10080aae40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0ff8006fd0 tx=0x7f0ff80090f0).ready entity=mon.0 client_cookie=135b6a70dfe4ed2c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:04.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100d399700 1 -- 172.21.15.33:0/1732919481 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800b600 con 0x7f1008154720 2023-11-06T20:44:04.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100d399700 1 -- 172.21.15.33:0/1732919481 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff800b760 con 0x7f1008154720 2023-11-06T20:44:04.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.479+0000 7f100d399700 1 -- 172.21.15.33:0/1732919481 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800bae0 con 0x7f1008154720 2023-11-06T20:44:04.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.480+0000 7f10105ff700 1 -- 172.21.15.33:0/1732919481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008154720 msgr2=0x7f10080aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.480+0000 7f10105ff700 1 --2- 172.21.15.33:0/1732919481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008154720 0x7f10080aae40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0ff8006fd0 tx=0x7f0ff80090f0).stop 2023-11-06T20:44:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.480+0000 7f10105ff700 1 -- 172.21.15.33:0/1732919481 shutdown_connections 2023-11-06T20:44:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.480+0000 7f10105ff700 1 --2- 172.21.15.33:0/1732919481 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008158720 0x7f1008063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.480+0000 7f10105ff700 1 --2- 172.21.15.33:0/1732919481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008154720 0x7f10080aae40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.480+0000 7f10105ff700 1 -- 172.21.15.33:0/1732919481 >> 172.21.15.33:0/1732919481 conn(0x7f100805ec80 msgr2=0x7f10080cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:04.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.480+0000 7f10105ff700 1 -- 172.21.15.33:0/1732919481 shutdown_connections 2023-11-06T20:44:04.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.480+0000 7f10105ff700 1 -- 172.21.15.33:0/1732919481 wait complete. 2023-11-06T20:44:04.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.482+0000 7f10105ff700 1 Processor -- start 2023-11-06T20:44:04.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.482+0000 7f10105ff700 1 -- start start 2023-11-06T20:44:04.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f10105ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1008154340 0x7f1008155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f10105ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008154720 0x7f100814cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f10105ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008158720 0x7f10080cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f10105ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10080cb7d0 con 0x7f1008158720 2023-11-06T20:44:04.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f10105ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1008155590 con 0x7f1008154340 2023-11-06T20:44:04.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f10105ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10080b06e0 con 0x7f1008154720 2023-11-06T20:44:04.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100eb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008158720 0x7f10080cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100eb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008158720 0x7f10080cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54062/0 (socket says 172.21.15.33:54062) 2023-11-06T20:44:04.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100eb9c700 1 -- 172.21.15.33:0/527041515 learned_addr learned my addr 172.21.15.33:0/527041515 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:04.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100db9a700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008154720 0x7f100814cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100e39b700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1008154340 0x7f1008155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100eb9c700 1 -- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008154720 msgr2=0x7f100814cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100eb9c700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008154720 0x7f100814cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100eb9c700 1 -- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1008154340 msgr2=0x7f1008155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100eb9c700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1008154340 0x7f1008155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100eb9c700 1 -- 172.21.15.33:0/527041515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff8009370 con 0x7f1008158720 2023-11-06T20:44:04.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100db9a700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008154720 0x7f100814cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.483+0000 7f100e39b700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1008154340 0x7f1008155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.484+0000 7f100eb9c700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008158720 0x7f10080cabf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f1000007f50 tx=0x7f1000009800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.484+0000 7f0ffe7fc700 1 -- 172.21.15.33:0/527041515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100000ec70 con 0x7f1008158720 2023-11-06T20:44:04.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.484+0000 7f0ffe7fc700 1 -- 172.21.15.33:0/527041515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100000edd0 con 0x7f1008158720 2023-11-06T20:44:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.484+0000 7f0ffe7fc700 1 -- 172.21.15.33:0/527041515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000014610 con 0x7f1008158720 2023-11-06T20:44:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.484+0000 7f10105ff700 1 -- 172.21.15.33:0/527041515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1008155f10 con 0x7f1008158720 2023-11-06T20:44:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.484+0000 7f10105ff700 1 -- 172.21.15.33:0/527041515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1008154f40 con 0x7f1008158720 2023-11-06T20:44:04.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.485+0000 7f10105ff700 1 -- 172.21.15.33:0/527041515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1008155700 con 0x7f1008158720 2023-11-06T20:44:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.486+0000 7f0ffe7fc700 1 -- 172.21.15.33:0/527041515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f100001d3b0 con 0x7f1008158720 2023-11-06T20:44:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.487+0000 7f0ffe7fc700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fec05f120 0x7f0fec0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.487+0000 7f0ffe7fc700 1 -- 172.21.15.33:0/527041515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f100007a460 con 0x7f1008158720 2023-11-06T20:44:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.487+0000 7f100e39b700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fec05f120 0x7f0fec0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.488+0000 7f100e39b700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fec05f120 0x7f0fec0615e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1004001750 tx=0x7f1004009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:04.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.490+0000 7f0ffe7fc700 1 -- 172.21.15.33:0/527041515 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f100004b960 con 0x7f1008158720 2023-11-06T20:44:04.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.671+0000 7f10105ff700 1 -- 172.21.15.33:0/527041515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f10080cbcf0 con 0x7f1008158720 2023-11-06T20:44:04.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.881+0000 7f0ffe7fc700 1 -- 172.21.15.33:0/527041515 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 152+0+0 (secure 0 0 0) 0x7f100004ef80 con 0x7f1008158720 2023-11-06T20:44:04.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 -- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fec05f120 msgr2=0x7f0fec0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fec05f120 0x7f0fec0615e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1004001750 tx=0x7f1004009040).stop 2023-11-06T20:44:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 -- 172.21.15.33:0/527041515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008158720 msgr2=0x7f10080cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008158720 0x7f10080cabf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f1000007f50 tx=0x7f1000009800).stop 2023-11-06T20:44:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 -- 172.21.15.33:0/527041515 shutdown_connections 2023-11-06T20:44:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fec05f120 0x7f0fec0615e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1008158720 0x7f10080cabf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1008154340 0x7f1008155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 --2- 172.21.15.33:0/527041515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1008154720 0x7f100814cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:04.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 -- 172.21.15.33:0/527041515 >> 172.21.15.33:0/527041515 conn(0x7f100805ec80 msgr2=0x7f1008152580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:04.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 -- 172.21.15.33:0/527041515 shutdown_connections 2023-11-06T20:44:04.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:04.884+0000 7f10105ff700 1 -- 172.21.15.33:0/527041515 wait complete. 2023-11-06T20:44:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T20:44:04.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T20:44:05.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.112+0000 7fa0591ff700 1 Processor -- start 2023-11-06T20:44:05.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.112+0000 7fa0591ff700 1 -- start start 2023-11-06T20:44:05.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.113+0000 7fa0591ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa054154710 0x7fa0540c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:05.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.113+0000 7fa0591ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540c6b60 con 0x7fa054154330 2023-11-06T20:44:05.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.113+0000 7fa0591ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540c7700 con 0x7fa0540c7ff0 2023-11-06T20:44:05.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.113+0000 7fa0591ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0540c5cd0 con 0x7fa054154710 2023-11-06T20:44:05.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.114+0000 7fa05259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa054154710 0x7fa0540c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:05.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.114+0000 7fa05259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa054154710 0x7fa0540c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54072/0 (socket says 172.21.15.33:54072) 2023-11-06T20:44:05.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.114+0000 7fa05259c700 1 -- 172.21.15.33:0/168300611 learned_addr learned my addr 172.21.15.33:0/168300611 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:05.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.114+0000 7fa05259c700 1 -- 172.21.15.33:0/168300611 >> v1:172.21.15.69:6789/0 conn(0x7fa0540c7ff0 legacy=0x7fa0540646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:05.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.114+0000 7fa05259c700 1 -- 172.21.15.33:0/168300611 >> v1:172.21.15.33:6789/0 conn(0x7fa054154330 legacy=0x7fa0540c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:05.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.114+0000 7fa051d9b700 1 -- 172.21.15.33:0/168300611 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030467808 0 0) 0x7fa0540c6b60 con 0x7fa054154330 2023-11-06T20:44:05.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.114+0000 7fa05259c700 1 -- 172.21.15.33:0/168300611 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540c5f50 con 0x7fa054154710 2023-11-06T20:44:05.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.114+0000 7fa05259c700 1 --2- 172.21.15.33:0/168300611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa054154710 0x7fa0540c57e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fa048006fd0 tx=0x7fa0480090f0).ready entity=mon.0 client_cookie=1c466638b2a91622 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:05.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.115+0000 7fa051d9b700 1 -- 172.21.15.33:0/168300611 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800b600 con 0x7fa054154710 2023-11-06T20:44:05.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.115+0000 7fa051d9b700 1 -- 172.21.15.33:0/168300611 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04800b760 con 0x7fa054154710 2023-11-06T20:44:05.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.115+0000 7fa051d9b700 1 -- 172.21.15.33:0/168300611 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800bae0 con 0x7fa054154710 2023-11-06T20:44:05.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.115+0000 7fa0591ff700 1 -- 172.21.15.33:0/168300611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa054154710 msgr2=0x7fa0540c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:05.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.115+0000 7fa0591ff700 1 --2- 172.21.15.33:0/168300611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa054154710 0x7fa0540c57e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fa048006fd0 tx=0x7fa0480090f0).stop 2023-11-06T20:44:05.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.115+0000 7fa0591ff700 1 -- 172.21.15.33:0/168300611 shutdown_connections 2023-11-06T20:44:05.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.115+0000 7fa0591ff700 1 --2- 172.21.15.33:0/168300611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa054154710 0x7fa0540c57e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:05.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.115+0000 7fa0591ff700 1 -- 172.21.15.33:0/168300611 >> 172.21.15.33:0/168300611 conn(0x7fa0540cb910 msgr2=0x7fa0540c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:05.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.115+0000 7fa0591ff700 1 -- 172.21.15.33:0/168300611 shutdown_connections 2023-11-06T20:44:05.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.115+0000 7fa0591ff700 1 -- 172.21.15.33:0/168300611 wait complete. 2023-11-06T20:44:05.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.117+0000 7fa0591ff700 1 Processor -- start 2023-11-06T20:44:05.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.117+0000 7fa0591ff700 1 -- start start 2023-11-06T20:44:05.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa0591ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0540c7ff0 0x7fa054159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:05.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa0591ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa054154330 0x7fa0540c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:05.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa0591ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa054154710 0x7fa054148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:05.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa0591ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0541674f0 con 0x7fa0540c7ff0 2023-11-06T20:44:05.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa0591ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa054167980 con 0x7fa054154330 2023-11-06T20:44:05.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa0591ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540c47a0 con 0x7fa054154710 2023-11-06T20:44:05.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0540c7ff0 0x7fa054159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:05.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0540c7ff0 0x7fa054159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54082/0 (socket says 172.21.15.33:54082) 2023-11-06T20:44:05.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa052d9d700 1 -- 172.21.15.33:0/3531903287 learned_addr learned my addr 172.21.15.33:0/3531903287 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:05.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa05259c700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa054154330 0x7fa0540c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:05.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.118+0000 7fa05359e700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa054154710 0x7fa054148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:05.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa052d9d700 1 -- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa054154710 msgr2=0x7fa054148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:05.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa052d9d700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa054154710 0x7fa054148400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:05.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa052d9d700 1 -- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa054154330 msgr2=0x7fa0540c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:05.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa052d9d700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa054154330 0x7fa0540c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:05.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa052d9d700 1 -- 172.21.15.33:0/3531903287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048009370 con 0x7fa0540c7ff0 2023-11-06T20:44:05.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa052d9d700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0540c7ff0 0x7fa054159810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fa044001eb0 tx=0x7fa044007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:05.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa03affd700 1 -- 172.21.15.33:0/3531903287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04400ce60 con 0x7fa0540c7ff0 2023-11-06T20:44:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa03affd700 1 -- 172.21.15.33:0/3531903287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04400f470 con 0x7fa0540c7ff0 2023-11-06T20:44:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa03affd700 1 -- 172.21.15.33:0/3531903287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04400f810 con 0x7fa0540c7ff0 2023-11-06T20:44:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa0591ff700 1 -- 172.21.15.33:0/3531903287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0541558c0 con 0x7fa0540c7ff0 2023-11-06T20:44:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.119+0000 7fa0591ff700 1 -- 172.21.15.33:0/3531903287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa054166bc0 con 0x7fa0540c7ff0 2023-11-06T20:44:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.120+0000 7fa0591ff700 1 -- 172.21.15.33:0/3531903287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0541677d0 con 0x7fa0540c7ff0 2023-11-06T20:44:05.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.121+0000 7fa03affd700 1 -- 172.21.15.33:0/3531903287 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa04400fb80 con 0x7fa0540c7ff0 2023-11-06T20:44:05.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.122+0000 7fa03affd700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa04005f150 0x7fa040061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:05.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.122+0000 7fa03affd700 1 -- 172.21.15.33:0/3531903287 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5639+0+0 (secure 0 0 0) 0x7fa0440795e0 con 0x7fa0540c7ff0 2023-11-06T20:44:05.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.122+0000 7fa05259c700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa04005f150 0x7fa040061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:05.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.123+0000 7fa05259c700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa04005f150 0x7fa040061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa054154b70 tx=0x7fa048017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:05.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.126+0000 7fa03affd700 1 -- 172.21.15.33:0/3531903287 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa04404a9d0 con 0x7fa0540c7ff0 2023-11-06T20:44:05.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.306+0000 7fa0591ff700 1 -- 172.21.15.33:0/3531903287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa0540830b0 con 0x7fa0540c7ff0 2023-11-06T20:44:05.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.893+0000 7fa03affd700 1 -- 172.21.15.33:0/3531903287 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa04404dff0 con 0x7fa0540c7ff0 2023-11-06T20:44:05.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.898+0000 7fa0591ff700 1 -- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa04005f150 msgr2=0x7fa040061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:05.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.898+0000 7fa0591ff700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa04005f150 0x7fa040061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa054154b70 tx=0x7fa048017000).stop 2023-11-06T20:44:05.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.898+0000 7fa0591ff700 1 -- 172.21.15.33:0/3531903287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0540c7ff0 msgr2=0x7fa054159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:05.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.898+0000 7fa0591ff700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0540c7ff0 0x7fa054159810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fa044001eb0 tx=0x7fa044007c70).stop 2023-11-06T20:44:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.901+0000 7fa0591ff700 1 -- 172.21.15.33:0/3531903287 shutdown_connections 2023-11-06T20:44:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.902+0000 7fa0591ff700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa04005f150 0x7fa040061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.902+0000 7fa0591ff700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa054154710 0x7fa054148400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.902+0000 7fa0591ff700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0540c7ff0 0x7fa054159810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.902+0000 7fa0591ff700 1 --2- 172.21.15.33:0/3531903287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa054154330 0x7fa0540c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.902+0000 7fa0591ff700 1 -- 172.21.15.33:0/3531903287 >> 172.21.15.33:0/3531903287 conn(0x7fa0540cb910 msgr2=0x7fa0540c01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.903+0000 7fa0591ff700 1 -- 172.21.15.33:0/3531903287 shutdown_connections 2023-11-06T20:44:05.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:05.904+0000 7fa0591ff700 1 -- 172.21.15.33:0/3531903287 wait complete. 2023-11-06T20:44:05.911 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T20:44:05.945 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T20:44:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.147+0000 7f5bbffff700 1 Processor -- start 2023-11-06T20:44:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.147+0000 7f5bbffff700 1 -- start start 2023-11-06T20:44:06.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.148+0000 7f5bbffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5bb8154710 0x7f5bb80c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.148+0000 7f5bbffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb80c7ff0 0x7f5bb80646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.148+0000 7f5bbffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb80c6b60 con 0x7f5bb8154330 2023-11-06T20:44:06.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.148+0000 7f5bbffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb80c7700 con 0x7f5bb8154710 2023-11-06T20:44:06.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.148+0000 7f5bbffff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb80c5cd0 con 0x7f5bb80c7ff0 2023-11-06T20:44:06.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbd59a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5bb8154710 0x7f5bb80c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbdd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5bb8154330 0x7f5bb80c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41226/0 (socket says 172.21.15.33:41226) 2023-11-06T20:44:06.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbdd9b700 1 -- 172.21.15.33:0/2624877134 learned_addr learned my addr 172.21.15.33:0/2624877134 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:06.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbe59c700 1 --2- 172.21.15.33:0/2624877134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb80c7ff0 0x7f5bb80646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbd59a700 1 -- 172.21.15.33:0/2624877134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb80c7ff0 msgr2=0x7f5bb80646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:06.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbd59a700 1 --2- 172.21.15.33:0/2624877134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb80c7ff0 0x7f5bb80646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbd59a700 1 -- 172.21.15.33:0/2624877134 >> v1:172.21.15.33:6789/0 conn(0x7f5bb8154330 legacy=0x7f5bb80c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbcd99700 1 -- 172.21.15.33:0/2624877134 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587324253 0 0) 0x7f5bb80c6b60 con 0x7f5bb8154330 2023-11-06T20:44:06.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbd59a700 1 -- 172.21.15.33:0/2624877134 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb80c5f50 con 0x7f5bb8154710 2023-11-06T20:44:06.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbe59c700 1 --2- 172.21.15.33:0/2624877134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb80c7ff0 0x7f5bb80646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:06.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.149+0000 7f5bbd59a700 1 --2- 172.21.15.33:0/2624877134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 0x7f5bb80c57e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5ba8009760 tx=0x7f5ba8009060).ready entity=mon.0 client_cookie=df672cd55fc7abdc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:06.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbcd99700 1 -- 172.21.15.33:0/2624877134 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800b580 con 0x7f5bb8154710 2023-11-06T20:44:06.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbcd99700 1 -- 172.21.15.33:0/2624877134 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba800b6e0 con 0x7f5bb8154710 2023-11-06T20:44:06.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbcd99700 1 -- 172.21.15.33:0/2624877134 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800ba60 con 0x7f5bb8154710 2023-11-06T20:44:06.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbffff700 1 -- 172.21.15.33:0/2624877134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 msgr2=0x7f5bb80c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbffff700 1 --2- 172.21.15.33:0/2624877134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 0x7f5bb80c57e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5ba8009760 tx=0x7f5ba8009060).stop 2023-11-06T20:44:06.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbffff700 1 -- 172.21.15.33:0/2624877134 shutdown_connections 2023-11-06T20:44:06.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbffff700 1 --2- 172.21.15.33:0/2624877134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb80c7ff0 0x7f5bb80646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbffff700 1 --2- 172.21.15.33:0/2624877134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 0x7f5bb80c57e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbffff700 1 -- 172.21.15.33:0/2624877134 >> 172.21.15.33:0/2624877134 conn(0x7f5bb80cb910 msgr2=0x7f5bb80c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbffff700 1 -- 172.21.15.33:0/2624877134 shutdown_connections 2023-11-06T20:44:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.150+0000 7f5bbffff700 1 -- 172.21.15.33:0/2624877134 wait complete. 2023-11-06T20:44:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.152+0000 7f5bbffff700 1 Processor -- start 2023-11-06T20:44:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.152+0000 7f5bbffff700 1 -- start start 2023-11-06T20:44:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.153+0000 7f5bbffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb80c7ff0 0x7f5bb81578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.153+0000 7f5bbffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb8154330 0x7f5bb8145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.153+0000 7f5bbffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 0x7f5bb8145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.153+0000 7f5bbffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb81594d0 con 0x7f5bb8154710 2023-11-06T20:44:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.153+0000 7f5bbffff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb80afbe0 con 0x7f5bb80c7ff0 2023-11-06T20:44:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.153+0000 7f5bbffff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8157d90 con 0x7f5bb8154330 2023-11-06T20:44:06.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.153+0000 7f5bbe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 0x7f5bb8145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.153+0000 7f5bbe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 0x7f5bb8145730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54108/0 (socket says 172.21.15.33:54108) 2023-11-06T20:44:06.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.153+0000 7f5bbe59c700 1 -- 172.21.15.33:0/1069836980 learned_addr learned my addr 172.21.15.33:0/1069836980 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:06.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5bbdd9b700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb80c7ff0 0x7f5bb81578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5bbd59a700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb8154330 0x7f5bb8145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5bbe59c700 1 -- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb8154330 msgr2=0x7f5bb8145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5bbe59c700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb8154330 0x7f5bb8145240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5bbe59c700 1 -- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb80c7ff0 msgr2=0x7f5bb81578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5bbe59c700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb80c7ff0 0x7f5bb81578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5bbe59c700 1 -- 172.21.15.33:0/1069836980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80092a0 con 0x7f5bb8154710 2023-11-06T20:44:06.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5bbe59c700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 0x7f5bb8145730 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f5bb000cef0 tx=0x7f5bb0007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:06.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5badffb700 1 -- 172.21.15.33:0/1069836980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000ae00 con 0x7f5bb8154710 2023-11-06T20:44:06.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5badffb700 1 -- 172.21.15.33:0/1069836980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb0016400 con 0x7f5bb8154710 2023-11-06T20:44:06.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5bbffff700 1 -- 172.21.15.33:0/1069836980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb80aecf0 con 0x7f5bb8154710 2023-11-06T20:44:06.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5badffb700 1 -- 172.21.15.33:0/1069836980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0016780 con 0x7f5bb8154710 2023-11-06T20:44:06.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.154+0000 7f5bbffff700 1 -- 172.21.15.33:0/1069836980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb80af2b0 con 0x7f5bb8154710 2023-11-06T20:44:06.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.155+0000 7f5bbffff700 1 -- 172.21.15.33:0/1069836980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb80afd70 con 0x7f5bb8154710 2023-11-06T20:44:06.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.156+0000 7f5badffb700 1 -- 172.21.15.33:0/1069836980 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5bb001f3f0 con 0x7f5bb8154710 2023-11-06T20:44:06.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.157+0000 7f5badffb700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b9c05f150 0x7f5b9c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.157+0000 7f5badffb700 1 -- 172.21.15.33:0/1069836980 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f5bb007aa10 con 0x7f5bb8154710 2023-11-06T20:44:06.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.158+0000 7f5bbdd9b700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b9c05f150 0x7f5b9c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.158+0000 7f5bbdd9b700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b9c05f150 0x7f5b9c061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f5bb40011e0 tx=0x7f5bb4009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:06.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.162+0000 7f5badffb700 1 -- 172.21.15.33:0/1069836980 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5bb004bc20 con 0x7f5bb8154710 2023-11-06T20:44:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.382+0000 7f5bbffff700 1 -- 172.21.15.33:0/1069836980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5bb80830b0 con 0x7f5bb8154710 2023-11-06T20:44:06.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.387+0000 7f5badffb700 1 -- 172.21.15.33:0/1069836980 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5bb001b020 con 0x7f5bb8154710 2023-11-06T20:44:06.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.391+0000 7f5bbffff700 1 -- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b9c05f150 msgr2=0x7f5b9c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.391+0000 7f5bbffff700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b9c05f150 0x7f5b9c061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f5bb40011e0 tx=0x7f5bb4009040).stop 2023-11-06T20:44:06.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.392+0000 7f5bbffff700 1 -- 172.21.15.33:0/1069836980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 msgr2=0x7f5bb8145730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.392+0000 7f5bbffff700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 0x7f5bb8145730 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f5bb000cef0 tx=0x7f5bb0007860).stop 2023-11-06T20:44:06.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.393+0000 7f5bbffff700 1 -- 172.21.15.33:0/1069836980 shutdown_connections 2023-11-06T20:44:06.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.393+0000 7f5bbffff700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b9c05f150 0x7f5b9c061610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.393+0000 7f5bbffff700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8154710 0x7f5bb8145730 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.393+0000 7f5bbffff700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb80c7ff0 0x7f5bb81578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.393+0000 7f5bbffff700 1 --2- 172.21.15.33:0/1069836980 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5bb8154330 0x7f5bb8145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.393+0000 7f5bbffff700 1 -- 172.21.15.33:0/1069836980 >> 172.21.15.33:0/1069836980 conn(0x7f5bb80cb910 msgr2=0x7f5bb80ca620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:06.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.395+0000 7f5bbffff700 1 -- 172.21.15.33:0/1069836980 shutdown_connections 2023-11-06T20:44:06.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.396+0000 7f5bbffff700 1 -- 172.21.15.33:0/1069836980 wait complete. 2023-11-06T20:44:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 3 and data pool 4 2023-11-06T20:44:06.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:06.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.625+0000 7f1ead9ff700 1 Processor -- start 2023-11-06T20:44:06.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.626+0000 7f1ead9ff700 1 -- start start 2023-11-06T20:44:06.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.627+0000 7f1ead9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ea80c1840 0x7f1ea80a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.627+0000 7f1ead9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 0x7f1ea80c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.627+0000 7f1ead9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea81554a0 con 0x7f1ea80c6240 2023-11-06T20:44:06.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.627+0000 7f1ead9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea8154960 con 0x7f1ea80c1840 2023-11-06T20:44:06.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.627+0000 7f1ead9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8155930 con 0x7f1ea80aa2d0 2023-11-06T20:44:06.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.627+0000 7f1ea6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ea80c1840 0x7f1ea80a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.627+0000 7f1ea6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ea80c1840 0x7f1ea80a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54116/0 (socket says 172.21.15.33:54116) 2023-11-06T20:44:06.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.627+0000 7f1ea6ffd700 1 -- 172.21.15.33:0/766516551 learned_addr learned my addr 172.21.15.33:0/766516551 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:06.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.627+0000 7f1ea67fc700 1 --2- 172.21.15.33:0/766516551 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 0x7f1ea80c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.628+0000 7f1ea6ffd700 1 -- 172.21.15.33:0/766516551 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 msgr2=0x7f1ea80c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:06.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.628+0000 7f1ea6ffd700 1 --2- 172.21.15.33:0/766516551 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 0x7f1ea80c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.628+0000 7f1ea6ffd700 1 -- 172.21.15.33:0/766516551 >> v1:172.21.15.69:6789/0 conn(0x7f1ea80c6240 legacy=0x7f1ea8157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.628+0000 7f1ea6ffd700 1 -- 172.21.15.33:0/766516551 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea80c66b0 con 0x7f1ea80c1840 2023-11-06T20:44:06.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.628+0000 7f1ea67fc700 1 --2- 172.21.15.33:0/766516551 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 0x7f1ea80c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:06.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.628+0000 7f1ea6ffd700 1 --2- 172.21.15.33:0/766516551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 0x7f1ea80a9de0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f1e98006fd0 tx=0x7f1e980090f0).ready entity=mon.0 client_cookie=7b321c3740ee5528 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:06.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.628+0000 7f1ea5ffb700 1 -- 172.21.15.33:0/766516551 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9800b600 con 0x7f1ea80c1840 2023-11-06T20:44:06.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.628+0000 7f1ea5ffb700 1 -- 172.21.15.33:0/766516551 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9800b760 con 0x7f1ea80c1840 2023-11-06T20:44:06.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.628+0000 7f1ea5ffb700 1 -- 172.21.15.33:0/766516551 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9800bae0 con 0x7f1ea80c1840 2023-11-06T20:44:06.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.629+0000 7f1ead9ff700 1 -- 172.21.15.33:0/766516551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 msgr2=0x7f1ea80a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.629+0000 7f1ead9ff700 1 --2- 172.21.15.33:0/766516551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 0x7f1ea80a9de0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f1e98006fd0 tx=0x7f1e980090f0).stop 2023-11-06T20:44:06.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.629+0000 7f1ead9ff700 1 -- 172.21.15.33:0/766516551 shutdown_connections 2023-11-06T20:44:06.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.629+0000 7f1ead9ff700 1 --2- 172.21.15.33:0/766516551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 0x7f1ea80a9de0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.629+0000 7f1ead9ff700 1 --2- 172.21.15.33:0/766516551 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 0x7f1ea80c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.629+0000 7f1ead9ff700 1 -- 172.21.15.33:0/766516551 >> 172.21.15.33:0/766516551 conn(0x7f1ea805ec80 msgr2=0x7f1ea80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:06.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.629+0000 7f1ead9ff700 1 -- 172.21.15.33:0/766516551 shutdown_connections 2023-11-06T20:44:06.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.629+0000 7f1ead9ff700 1 -- 172.21.15.33:0/766516551 wait complete. 2023-11-06T20:44:06.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.630+0000 7f1ead9ff700 1 Processor -- start 2023-11-06T20:44:06.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.630+0000 7f1ead9ff700 1 -- start start 2023-11-06T20:44:06.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.631+0000 7f1ead9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 0x7f1ea8144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.631+0000 7f1ead9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 0x7f1ea80aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.631+0000 7f1ead9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ea80c6240 0x7f1ea8153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.631+0000 7f1ead9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea81451d0 con 0x7f1ea80c1840 2023-11-06T20:44:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.631+0000 7f1ead9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea80afd80 con 0x7f1ea80c6240 2023-11-06T20:44:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.631+0000 7f1ead9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8155f20 con 0x7f1ea80aa2d0 2023-11-06T20:44:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 0x7f1ea80aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 0x7f1ea80aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54132/0 (socket says 172.21.15.33:54132) 2023-11-06T20:44:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea67fc700 1 -- 172.21.15.33:0/1662596623 learned_addr learned my addr 172.21.15.33:0/1662596623 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea6ffd700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 0x7f1ea8144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea77fe700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ea80c6240 0x7f1ea8153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea67fc700 1 -- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 msgr2=0x7f1ea8144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea67fc700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 0x7f1ea8144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea67fc700 1 -- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ea80c6240 msgr2=0x7f1ea8153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea67fc700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ea80c6240 0x7f1ea8153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea67fc700 1 -- 172.21.15.33:0/1662596623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e98009370 con 0x7f1ea80c1840 2023-11-06T20:44:06.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.632+0000 7f1ea67fc700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 0x7f1ea80aed70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1e9c00d860 tx=0x7f1e9c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:06.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.633+0000 7f1e8effd700 1 -- 172.21.15.33:0/1662596623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c016070 con 0x7f1ea80c1840 2023-11-06T20:44:06.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.633+0000 7f1e8effd700 1 -- 172.21.15.33:0/1662596623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c011590 con 0x7f1ea80c1840 2023-11-06T20:44:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.633+0000 7f1e8effd700 1 -- 172.21.15.33:0/1662596623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c011910 con 0x7f1ea80c1840 2023-11-06T20:44:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.633+0000 7f1ead9ff700 1 -- 172.21.15.33:0/1662596623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea80af7c0 con 0x7f1ea80c1840 2023-11-06T20:44:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.633+0000 7f1ead9ff700 1 -- 172.21.15.33:0/1662596623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea8155bb0 con 0x7f1ea80c1840 2023-11-06T20:44:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.634+0000 7f1ead9ff700 1 -- 172.21.15.33:0/1662596623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea806c6c0 con 0x7f1ea80c1840 2023-11-06T20:44:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.634+0000 7f1e8effd700 1 -- 172.21.15.33:0/1662596623 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1e9c011cc0 con 0x7f1ea80c1840 2023-11-06T20:44:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.635+0000 7f1e8effd700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e9405f150 0x7f1e94061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.635+0000 7f1e8effd700 1 -- 172.21.15.33:0/1662596623 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1e9c079960 con 0x7f1ea80c1840 2023-11-06T20:44:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.636+0000 7f1ea6ffd700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e9405f150 0x7f1e94061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.636+0000 7f1ea6ffd700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e9405f150 0x7f1e94061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1e9800b3c0 tx=0x7f1e98017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.639+0000 7f1e8effd700 1 -- 172.21.15.33:0/1662596623 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1e9c04ab70 con 0x7f1ea80c1840 2023-11-06T20:44:06.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.824+0000 7f1ead9ff700 1 -- 172.21.15.33:0/1662596623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ea8156060 con 0x7f1ea80c1840 2023-11-06T20:44:06.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.825+0000 7f1e8effd700 1 -- 172.21.15.33:0/1662596623 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f1e9c04e190 con 0x7f1ea80c1840 2023-11-06T20:44:06.825 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:06.825 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:06.383394+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:06.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 -- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e9405f150 msgr2=0x7f1e94061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e9405f150 0x7f1e94061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1e9800b3c0 tx=0x7f1e98017000).stop 2023-11-06T20:44:06.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 -- 172.21.15.33:0/1662596623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 msgr2=0x7f1ea80aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:06.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 0x7f1ea80aed70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1e9c00d860 tx=0x7f1e9c008040).stop 2023-11-06T20:44:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 -- 172.21.15.33:0/1662596623 shutdown_connections 2023-11-06T20:44:06.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e9405f150 0x7f1e94061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ea80aa2d0 0x7f1ea8144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ea80c6240 0x7f1ea8153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 --2- 172.21.15.33:0/1662596623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ea80c1840 0x7f1ea80aed70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:06.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 -- 172.21.15.33:0/1662596623 >> 172.21.15.33:0/1662596623 conn(0x7f1ea805ec80 msgr2=0x7f1ea8150cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:06.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 -- 172.21.15.33:0/1662596623 shutdown_connections 2023-11-06T20:44:06.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:06.827+0000 7f1ead9ff700 1 -- 172.21.15.33:0/1662596623 wait complete. 2023-11-06T20:44:06.844 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:07.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.043+0000 7f812a1ff700 1 Processor -- start 2023-11-06T20:44:07.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.043+0000 7f812a1ff700 1 -- start start 2023-11-06T20:44:07.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f812a1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81240c1840 0x7f81240a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f812a1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81241554a0 con 0x7f81240c6460 2023-11-06T20:44:07.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f812a1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8124155930 con 0x7f81240aa2d0 2023-11-06T20:44:07.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f812a1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8124154960 con 0x7f81240c1840 2023-11-06T20:44:07.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f81237fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81240c1840 0x7f81240a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f81237fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81240c1840 0x7f81240a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54138/0 (socket says 172.21.15.33:54138) 2023-11-06T20:44:07.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f8123fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f81240c6460 0x7f81240bfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41234/0 (socket says 172.21.15.33:41234) 2023-11-06T20:44:07.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f81237fe700 1 -- 172.21.15.33:0/3728721358 learned_addr learned my addr 172.21.15.33:0/3728721358 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:07.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f81237fe700 1 -- 172.21.15.33:0/3728721358 >> v1:172.21.15.69:6789/0 conn(0x7f81240aa2d0 legacy=0x7f81240c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f81237fe700 1 -- 172.21.15.33:0/3728721358 >> v1:172.21.15.33:6789/0 conn(0x7f81240c6460 legacy=0x7f81240bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.045+0000 7f81227fc700 1 -- 172.21.15.33:0/3728721358 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134584627 0 0) 0x7f81241554a0 con 0x7f81240c6460 2023-11-06T20:44:07.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.046+0000 7f81237fe700 1 -- 172.21.15.33:0/3728721358 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8124154b70 con 0x7f81240c1840 2023-11-06T20:44:07.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.046+0000 7f81237fe700 1 --2- 172.21.15.33:0/3728721358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 0x7f81240a9de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f8114006fd0 tx=0x7f81140090f0).ready entity=mon.0 client_cookie=4928deba8fcb230d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:07.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.046+0000 7f81227fc700 1 -- 172.21.15.33:0/3728721358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811400b600 con 0x7f81240c1840 2023-11-06T20:44:07.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.046+0000 7f81227fc700 1 -- 172.21.15.33:0/3728721358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811400b760 con 0x7f81240c1840 2023-11-06T20:44:07.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.046+0000 7f81227fc700 1 -- 172.21.15.33:0/3728721358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811400bae0 con 0x7f81240c1840 2023-11-06T20:44:07.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.046+0000 7f812a1ff700 1 -- 172.21.15.33:0/3728721358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 msgr2=0x7f81240a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.046+0000 7f812a1ff700 1 --2- 172.21.15.33:0/3728721358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 0x7f81240a9de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f8114006fd0 tx=0x7f81140090f0).stop 2023-11-06T20:44:07.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.046+0000 7f812a1ff700 1 -- 172.21.15.33:0/3728721358 shutdown_connections 2023-11-06T20:44:07.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.046+0000 7f812a1ff700 1 --2- 172.21.15.33:0/3728721358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 0x7f81240a9de0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.046+0000 7f812a1ff700 1 -- 172.21.15.33:0/3728721358 >> 172.21.15.33:0/3728721358 conn(0x7f812405ec80 msgr2=0x7f81240cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:07.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.047+0000 7f812a1ff700 1 -- 172.21.15.33:0/3728721358 shutdown_connections 2023-11-06T20:44:07.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.047+0000 7f812a1ff700 1 -- 172.21.15.33:0/3728721358 wait complete. 2023-11-06T20:44:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.049+0000 7f812a1ff700 1 Processor -- start 2023-11-06T20:44:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.049+0000 7f812a1ff700 1 -- start start 2023-11-06T20:44:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.050+0000 7f812a1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81240aa2d0 0x7f8124150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.050+0000 7f812a1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 0x7f812414dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.050+0000 7f812a1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81240c6460 0x7f81240af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.050+0000 7f812a1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8124150ef0 con 0x7f81240c1840 2023-11-06T20:44:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.050+0000 7f812a1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8124150750 con 0x7f81240aa2d0 2023-11-06T20:44:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.050+0000 7f812a1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812414e250 con 0x7f81240c6460 2023-11-06T20:44:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.050+0000 7f8122ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 0x7f812414dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.050+0000 7f8122ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 0x7f812414dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54154/0 (socket says 172.21.15.33:54154) 2023-11-06T20:44:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.050+0000 7f8122ffd700 1 -- 172.21.15.33:0/259786283 learned_addr learned my addr 172.21.15.33:0/259786283 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f81237fe700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81240aa2d0 0x7f8124150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f8123fff700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81240c6460 0x7f81240af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f8122ffd700 1 -- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81240c6460 msgr2=0x7f81240af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f8122ffd700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81240c6460 0x7f81240af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f8122ffd700 1 -- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81240aa2d0 msgr2=0x7f8124150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f8122ffd700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81240aa2d0 0x7f8124150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f8122ffd700 1 -- 172.21.15.33:0/259786283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8114009370 con 0x7f81240c1840 2023-11-06T20:44:07.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f81237fe700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81240aa2d0 0x7f8124150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f8122ffd700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 0x7f812414dd60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f811800c4c0 tx=0x7f8118007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f810b7fe700 1 -- 172.21.15.33:0/259786283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118007020 con 0x7f81240c1840 2023-11-06T20:44:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f810b7fe700 1 -- 172.21.15.33:0/259786283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8118007180 con 0x7f81240c1840 2023-11-06T20:44:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f810b7fe700 1 -- 172.21.15.33:0/259786283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118007520 con 0x7f81240c1840 2023-11-06T20:44:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f812a1ff700 1 -- 172.21.15.33:0/259786283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81240af230 con 0x7f81240c1840 2023-11-06T20:44:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.051+0000 7f812a1ff700 1 -- 172.21.15.33:0/259786283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8124150c10 con 0x7f81240c1840 2023-11-06T20:44:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.052+0000 7f812a1ff700 1 -- 172.21.15.33:0/259786283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812406c6c0 con 0x7f81240c1840 2023-11-06T20:44:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.053+0000 7f810b7fe700 1 -- 172.21.15.33:0/259786283 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f81180068b0 con 0x7f81240c1840 2023-11-06T20:44:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.054+0000 7f810b7fe700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f811005f150 0x7f8110061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.054+0000 7f810b7fe700 1 -- 172.21.15.33:0/259786283 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f81180795d0 con 0x7f81240c1840 2023-11-06T20:44:07.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.054+0000 7f81237fe700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f811005f150 0x7f8110061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.055+0000 7f81237fe700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f811005f150 0x7f8110061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8114009130 tx=0x7f8114017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:07.063 INFO:tasks.ceph.mds.d.smithi069.stderr:2023-11-06T20:44:07.059+0000 7f33c81f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T20:44:07.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.058+0000 7f810b7fe700 1 -- 172.21.15.33:0/259786283 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8118047af0 con 0x7f81240c1840 2023-11-06T20:44:07.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.238+0000 7f812a1ff700 1 -- 172.21.15.33:0/259786283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81240aef80 con 0x7f81240c1840 2023-11-06T20:44:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.239+0000 7f810b7fe700 1 -- 172.21.15.33:0/259786283 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f811804a8a0 con 0x7f81240c1840 2023-11-06T20:44:07.238 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:07.239 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:06.383394+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 -- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f811005f150 msgr2=0x7f8110061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f811005f150 0x7f8110061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8114009130 tx=0x7f8114017000).stop 2023-11-06T20:44:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 -- 172.21.15.33:0/259786283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 msgr2=0x7f812414dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 0x7f812414dd60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f811800c4c0 tx=0x7f8118007dd0).stop 2023-11-06T20:44:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 -- 172.21.15.33:0/259786283 shutdown_connections 2023-11-06T20:44:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f811005f150 0x7f8110061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81240c6460 0x7f81240af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81240aa2d0 0x7f8124150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 --2- 172.21.15.33:0/259786283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81240c1840 0x7f812414dd60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 -- 172.21.15.33:0/259786283 >> 172.21.15.33:0/259786283 conn(0x7f812405ec80 msgr2=0x7f8124152830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 -- 172.21.15.33:0/259786283 shutdown_connections 2023-11-06T20:44:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.241+0000 7f812a1ff700 1 -- 172.21.15.33:0/259786283 wait complete. 2023-11-06T20:44:07.258 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T20:44:07.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.460+0000 7f3fe1bff700 1 Processor -- start 2023-11-06T20:44:07.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.460+0000 7f3fe1bff700 1 -- start start 2023-11-06T20:44:07.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.461+0000 7f3fe1bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3fdc154710 0x7f3fdc0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.461+0000 7f3fe1bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 0x7f3fdc0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.461+0000 7f3fe1bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc0c6b60 con 0x7f3fdc154330 2023-11-06T20:44:07.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.461+0000 7f3fe1bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fdc0c7700 con 0x7f3fdc154710 2023-11-06T20:44:07.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.462+0000 7f3fe1bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0c5cd0 con 0x7f3fdc0c7ff0 2023-11-06T20:44:07.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.462+0000 7f3fdbfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 0x7f3fdc0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.462+0000 7f3fdbfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 0x7f3fdc0646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:58062/0 (socket says 172.21.15.33:58062) 2023-11-06T20:44:07.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.462+0000 7f3fdbfff700 1 -- 172.21.15.33:0/1585224057 learned_addr learned my addr 172.21.15.33:0/1585224057 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:07.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.462+0000 7f3fdaffd700 1 --2- 172.21.15.33:0/1585224057 >> v2:172.21.15.69:3300/0 conn(0x7f3fdc154710 0x7f3fdc0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.462+0000 7f3fdbfff700 1 -- 172.21.15.33:0/1585224057 >> v2:172.21.15.69:3300/0 conn(0x7f3fdc154710 msgr2=0x7f3fdc0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:07.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.462+0000 7f3fdbfff700 1 --2- 172.21.15.33:0/1585224057 >> v2:172.21.15.69:3300/0 conn(0x7f3fdc154710 0x7f3fdc0c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.462+0000 7f3fdbfff700 1 -- 172.21.15.33:0/1585224057 >> v1:172.21.15.69:6789/0 conn(0x7f3fdc154330 legacy=0x7f3fdc0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.462+0000 7f3fdbfff700 1 -- 172.21.15.33:0/1585224057 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc0c5f50 con 0x7f3fdc0c7ff0 2023-11-06T20:44:07.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.462+0000 7f3fda7fc700 1 -- 172.21.15.33:0/1585224057 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802095432 0 0) 0x7f3fdc0c6b60 con 0x7f3fdc154330 2023-11-06T20:44:07.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.463+0000 7f3fdbfff700 1 --2- 172.21.15.33:0/1585224057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 0x7f3fdc0646c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f3fd4009f20 tx=0x7f3fd4009840).ready entity=mon.2 client_cookie=aa4c3cf33d3cb31d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:07.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.463+0000 7f3fda7fc700 1 -- 172.21.15.33:0/1585224057 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400e070 con 0x7f3fdc0c7ff0 2023-11-06T20:44:07.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.463+0000 7f3fda7fc700 1 -- 172.21.15.33:0/1585224057 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd400b9d0 con 0x7f3fdc0c7ff0 2023-11-06T20:44:07.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.463+0000 7f3fda7fc700 1 -- 172.21.15.33:0/1585224057 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400bd50 con 0x7f3fdc0c7ff0 2023-11-06T20:44:07.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.464+0000 7f3fe1bff700 1 -- 172.21.15.33:0/1585224057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 msgr2=0x7f3fdc0646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.464+0000 7f3fe1bff700 1 --2- 172.21.15.33:0/1585224057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 0x7f3fdc0646c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f3fd4009f20 tx=0x7f3fd4009840).stop 2023-11-06T20:44:07.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.464+0000 7f3fe1bff700 1 -- 172.21.15.33:0/1585224057 shutdown_connections 2023-11-06T20:44:07.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.464+0000 7f3fe1bff700 1 --2- 172.21.15.33:0/1585224057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 0x7f3fdc0646c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.464+0000 7f3fe1bff700 1 --2- 172.21.15.33:0/1585224057 >> v2:172.21.15.69:3300/0 conn(0x7f3fdc154710 0x7f3fdc0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.464+0000 7f3fe1bff700 1 -- 172.21.15.33:0/1585224057 >> 172.21.15.33:0/1585224057 conn(0x7f3fdc0cb910 msgr2=0x7f3fdc0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:07.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.464+0000 7f3fe1bff700 1 -- 172.21.15.33:0/1585224057 shutdown_connections 2023-11-06T20:44:07.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.464+0000 7f3fe1bff700 1 -- 172.21.15.33:0/1585224057 wait complete. 2023-11-06T20:44:07.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.466+0000 7f3fe1bff700 1 Processor -- start 2023-11-06T20:44:07.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.466+0000 7f3fe1bff700 1 -- start start 2023-11-06T20:44:07.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fe1bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 0x7f3fdc146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fe1bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc154330 0x7f3fdc0ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fe1bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc154710 0x7f3fdc0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fe1bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc147100 con 0x7f3fdc154330 2023-11-06T20:44:07.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fe1bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0af180 con 0x7f3fdc154710 2023-11-06T20:44:07.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fe1bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc153140 con 0x7f3fdc0c7ff0 2023-11-06T20:44:07.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fdaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc154330 0x7f3fdc0ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fdaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc154330 0x7f3fdc0ae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54164/0 (socket says 172.21.15.33:54164) 2023-11-06T20:44:07.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fdaffd700 1 -- 172.21.15.33:0/3832720371 learned_addr learned my addr 172.21.15.33:0/3832720371 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:07.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fdbfff700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc154710 0x7f3fdc0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fdb7fe700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 0x7f3fdc146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fdaffd700 1 -- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 msgr2=0x7f3fdc146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fdaffd700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 0x7f3fdc146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fdaffd700 1 -- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc154710 msgr2=0x7f3fdc0aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:07.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fdaffd700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc154710 0x7f3fdc0aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:07.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.468+0000 7f3fdaffd700 1 -- 172.21.15.33:0/3832720371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4009ae0 con 0x7f3fdc154330 2023-11-06T20:44:07.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.469+0000 7f3fdaffd700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc154330 0x7f3fdc0ae7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f3fd000e8d0 tx=0x7f3fd0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:07.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.469+0000 7f3fc37fe700 1 -- 172.21.15.33:0/3832720371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0012600 con 0x7f3fdc154330 2023-11-06T20:44:07.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.469+0000 7f3fc37fe700 1 -- 172.21.15.33:0/3832720371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd0012760 con 0x7f3fdc154330 2023-11-06T20:44:07.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.469+0000 7f3fe1bff700 1 -- 172.21.15.33:0/3832720371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc0af420 con 0x7f3fdc154330 2023-11-06T20:44:07.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.469+0000 7f3fc37fe700 1 -- 172.21.15.33:0/3832720371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0012ae0 con 0x7f3fdc154330 2023-11-06T20:44:07.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.469+0000 7f3fe1bff700 1 -- 172.21.15.33:0/3832720371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc0afc60 con 0x7f3fdc154330 2023-11-06T20:44:07.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.470+0000 7f3fe1bff700 1 -- 172.21.15.33:0/3832720371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc0830b0 con 0x7f3fdc154330 2023-11-06T20:44:07.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.471+0000 7f3fc37fe700 1 -- 172.21.15.33:0/3832720371 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3fd001b400 con 0x7f3fdc154330 2023-11-06T20:44:07.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.472+0000 7f3fc37fe700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fc805f150 0x7f3fc8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:07.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.472+0000 7f3fc37fe700 1 -- 172.21.15.33:0/3832720371 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3fd007a550 con 0x7f3fdc154330 2023-11-06T20:44:07.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.472+0000 7f3fdb7fe700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fc805f150 0x7f3fc8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:07.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.473+0000 7f3fdb7fe700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fc805f150 0x7f3fc8061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f3fcc007fd0 tx=0x7f3fcc007bb0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:07.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.476+0000 7f3fc37fe700 1 -- 172.21.15.33:0/3832720371 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3fd004b820 con 0x7f3fdc154330 2023-11-06T20:44:07.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:07.701+0000 7f3fe1bff700 1 -- 172.21.15.33:0/3832720371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3fdc06ab40 con 0x7f3fdc154330 2023-11-06T20:44:08.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.390+0000 7f3fc37fe700 1 -- 172.21.15.33:0/3832720371 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3fd0016020 con 0x7f3fdc154330 2023-11-06T20:44:08.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 -- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fc805f150 msgr2=0x7f3fc8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:08.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fc805f150 0x7f3fc8061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f3fcc007fd0 tx=0x7f3fcc007bb0).stop 2023-11-06T20:44:08.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 -- 172.21.15.33:0/3832720371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc154330 msgr2=0x7f3fdc0ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:08.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc154330 0x7f3fdc0ae7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f3fd000e8d0 tx=0x7f3fd0009060).stop 2023-11-06T20:44:08.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 -- 172.21.15.33:0/3832720371 shutdown_connections 2023-11-06T20:44:08.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fc805f150 0x7f3fc8061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3fdc0c7ff0 0x7f3fdc146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc154710 0x7f3fdc0aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 --2- 172.21.15.33:0/3832720371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc154330 0x7f3fdc0ae7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 -- 172.21.15.33:0/3832720371 >> 172.21.15.33:0/3832720371 conn(0x7f3fdc0cb910 msgr2=0x7f3fdc0646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:08.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 -- 172.21.15.33:0/3832720371 shutdown_connections 2023-11-06T20:44:08.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.393+0000 7f3fe1bff700 1 -- 172.21.15.33:0/3832720371 wait complete. 2023-11-06T20:44:08.412 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T20:44:08.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T20:44:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.619+0000 7f319efff700 1 Processor -- start 2023-11-06T20:44:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.619+0000 7f319efff700 1 -- start start 2023-11-06T20:44:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.620+0000 7f319efff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f319814e0b0 0x7f319814e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.620+0000 7f319efff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31980bc210 0x7f31980bc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.620+0000 7f319efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980c17f0 0x7f31980c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.620+0000 7f319efff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31980c2170 con 0x7f31980bc210 2023-11-06T20:44:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.620+0000 7f319efff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31980bcc90 con 0x7f319814e0b0 2023-11-06T20:44:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.620+0000 7f319efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31980bcf70 con 0x7f31980c17f0 2023-11-06T20:44:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f3197fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31980bc210 0x7f31980bc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f3197fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31980bc210 0x7f31980bc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42626/0 (socket says 172.21.15.33:42626) 2023-11-06T20:44:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f3197fff700 1 -- 172.21.15.33:0/2869036700 learned_addr learned my addr 172.21.15.33:0/2869036700 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f319d59c700 1 --2- 172.21.15.33:0/2869036700 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980c17f0 0x7f31980c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f3197fff700 1 -- 172.21.15.33:0/2869036700 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980c17f0 msgr2=0x7f31980c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f319cd9b700 1 --2- 172.21.15.33:0/2869036700 >> v2:172.21.15.69:3300/0 conn(0x7f319814e0b0 0x7f319814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f3197fff700 1 --2- 172.21.15.33:0/2869036700 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980c17f0 0x7f31980c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f3197fff700 1 -- 172.21.15.33:0/2869036700 >> v2:172.21.15.69:3300/0 conn(0x7f319814e0b0 msgr2=0x7f319814e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:08.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f3197fff700 1 --2- 172.21.15.33:0/2869036700 >> v2:172.21.15.69:3300/0 conn(0x7f319814e0b0 0x7f319814e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f3197fff700 1 -- 172.21.15.33:0/2869036700 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31980bd250 con 0x7f31980bc210 2023-11-06T20:44:08.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f319cd9b700 1 --2- 172.21.15.33:0/2869036700 >> v2:172.21.15.69:3300/0 conn(0x7f319814e0b0 0x7f319814e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:08.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f319d59c700 1 --2- 172.21.15.33:0/2869036700 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980c17f0 0x7f31980c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:08.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.621+0000 7f3197fff700 1 --2- 172.21.15.33:0/2869036700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31980bc210 0x7f31980bc670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f31980676b0 tx=0x7f31840090f0).ready entity=mon.0 client_cookie=5eee9f380c7557cf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f31977fe700 1 -- 172.21.15.33:0/2869036700 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318400b600 con 0x7f31980bc210 2023-11-06T20:44:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f31977fe700 1 -- 172.21.15.33:0/2869036700 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318400b760 con 0x7f31980bc210 2023-11-06T20:44:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f31977fe700 1 -- 172.21.15.33:0/2869036700 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318400bae0 con 0x7f31980bc210 2023-11-06T20:44:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f319efff700 1 -- 172.21.15.33:0/2869036700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31980bc210 msgr2=0x7f31980bc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f319efff700 1 --2- 172.21.15.33:0/2869036700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31980bc210 0x7f31980bc670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f31980676b0 tx=0x7f31840090f0).stop 2023-11-06T20:44:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f319efff700 1 -- 172.21.15.33:0/2869036700 shutdown_connections 2023-11-06T20:44:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f319efff700 1 --2- 172.21.15.33:0/2869036700 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980c17f0 0x7f31980c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f319efff700 1 --2- 172.21.15.33:0/2869036700 >> v2:172.21.15.69:3300/0 conn(0x7f319814e0b0 0x7f319814e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f319efff700 1 --2- 172.21.15.33:0/2869036700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31980bc210 0x7f31980bc670 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f319efff700 1 -- 172.21.15.33:0/2869036700 >> 172.21.15.33:0/2869036700 conn(0x7f3198062710 msgr2=0x7f3198062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f319efff700 1 -- 172.21.15.33:0/2869036700 shutdown_connections 2023-11-06T20:44:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.622+0000 7f319efff700 1 -- 172.21.15.33:0/2869036700 wait complete. 2023-11-06T20:44:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.624+0000 7f319efff700 1 Processor -- start 2023-11-06T20:44:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.624+0000 7f319efff700 1 -- start start 2023-11-06T20:44:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980bc210 0x7f31980be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319efff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31980c17f0 0x7f31980a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319efff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319814e0b0 0x7f31980bfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319efff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980bf7d0 con 0x7f319814e0b0 2023-11-06T20:44:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319efff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980a0a70 con 0x7f31980c17f0 2023-11-06T20:44:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31980be2b0 con 0x7f31980bc210 2023-11-06T20:44:08.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319d59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319814e0b0 0x7f31980bfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:08.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319d59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319814e0b0 0x7f31980bfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42638/0 (socket says 172.21.15.33:42638) 2023-11-06T20:44:08.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319d59c700 1 -- 172.21.15.33:0/3776983620 learned_addr learned my addr 172.21.15.33:0/3776983620 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:08.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319cd9b700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980bc210 0x7f31980be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:08.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f3197fff700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31980c17f0 0x7f31980a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319d59c700 1 -- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980bc210 msgr2=0x7f31980be5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319d59c700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980bc210 0x7f31980be5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319d59c700 1 -- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31980c17f0 msgr2=0x7f31980a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.625+0000 7f319d59c700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31980c17f0 0x7f31980a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.626+0000 7f319d59c700 1 -- 172.21.15.33:0/3776983620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184009370 con 0x7f319814e0b0 2023-11-06T20:44:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.626+0000 7f3197fff700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31980c17f0 0x7f31980a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.626+0000 7f319d59c700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319814e0b0 0x7f31980bfc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f319000a380 tx=0x7f3190007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.626+0000 7f3194ff9700 1 -- 172.21.15.33:0/3776983620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000ae10 con 0x7f319814e0b0 2023-11-06T20:44:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.626+0000 7f3194ff9700 1 -- 172.21.15.33:0/3776983620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319000d650 con 0x7f319814e0b0 2023-11-06T20:44:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.626+0000 7f319efff700 1 -- 172.21.15.33:0/3776983620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31980bebc0 con 0x7f319814e0b0 2023-11-06T20:44:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.626+0000 7f3194ff9700 1 -- 172.21.15.33:0/3776983620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000d9d0 con 0x7f319814e0b0 2023-11-06T20:44:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.626+0000 7f319efff700 1 -- 172.21.15.33:0/3776983620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31980bf180 con 0x7f319814e0b0 2023-11-06T20:44:08.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.627+0000 7f319efff700 1 -- 172.21.15.33:0/3776983620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31980bf940 con 0x7f319814e0b0 2023-11-06T20:44:08.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.628+0000 7f3194ff9700 1 -- 172.21.15.33:0/3776983620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3190013070 con 0x7f319814e0b0 2023-11-06T20:44:08.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.629+0000 7f3194ff9700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f317c05f150 0x7f317c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.629+0000 7f3194ff9700 1 -- 172.21.15.33:0/3776983620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f319007ad20 con 0x7f319814e0b0 2023-11-06T20:44:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.630+0000 7f319cd9b700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f317c05f150 0x7f317c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.630+0000 7f319cd9b700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f317c05f150 0x7f317c061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f318c00ab40 tx=0x7f318c008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.633+0000 7f3194ff9700 1 -- 172.21.15.33:0/3776983620 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f319004bf30 con 0x7f319814e0b0 2023-11-06T20:44:08.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:08.859+0000 7f319efff700 1 -- 172.21.15.33:0/3776983620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f31980bbd30 con 0x7f319814e0b0 2023-11-06T20:44:09.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.392+0000 7f3194ff9700 1 -- 172.21.15.33:0/3776983620 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f319004f550 con 0x7f319814e0b0 2023-11-06T20:44:09.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.395+0000 7f319efff700 1 -- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f317c05f150 msgr2=0x7f317c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:09.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.395+0000 7f319efff700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f317c05f150 0x7f317c061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f318c00ab40 tx=0x7f318c008040).stop 2023-11-06T20:44:09.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.395+0000 7f319efff700 1 -- 172.21.15.33:0/3776983620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319814e0b0 msgr2=0x7f31980bfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:09.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.395+0000 7f319efff700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319814e0b0 0x7f31980bfc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f319000a380 tx=0x7f3190007900).stop 2023-11-06T20:44:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.396+0000 7f319efff700 1 -- 172.21.15.33:0/3776983620 shutdown_connections 2023-11-06T20:44:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.396+0000 7f319efff700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f317c05f150 0x7f317c061610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.396+0000 7f319efff700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f31980bc210 0x7f31980be5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.396+0000 7f319efff700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319814e0b0 0x7f31980bfc60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.396+0000 7f319efff700 1 --2- 172.21.15.33:0/3776983620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31980c17f0 0x7f31980a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.396+0000 7f319efff700 1 -- 172.21.15.33:0/3776983620 >> 172.21.15.33:0/3776983620 conn(0x7f3198062710 msgr2=0x7f3198062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.396+0000 7f319efff700 1 -- 172.21.15.33:0/3776983620 shutdown_connections 2023-11-06T20:44:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.396+0000 7f319efff700 1 -- 172.21.15.33:0/3776983620 wait complete. 2023-11-06T20:44:09.415 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T20:44:09.416 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:44:09.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.616+0000 7f45437ff700 1 Processor -- start 2023-11-06T20:44:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.616+0000 7f45437ff700 1 -- start start 2023-11-06T20:44:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f45437ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c5d50 0x7f453c0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f45437ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f453c0bfd70 0x7f453c0c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f45437ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c1554a0 con 0x7f453c0c1840 2023-11-06T20:44:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f45437ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453c154960 con 0x7f453c0bfd70 2023-11-06T20:44:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f45437ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0c66c0 con 0x7f453c0c5d50 2023-11-06T20:44:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f4541d9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f453c0bfd70 0x7f453c0c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f4540d9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c5d50 0x7f453c0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f4540d9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c5d50 0x7f453c0c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:51194/0 (socket says 172.21.15.33:51194) 2023-11-06T20:44:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f454159b700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f453c0c1840 0x7f453c0a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:46902/0 (socket says 172.21.15.33:46902) 2023-11-06T20:44:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f4540d9a700 1 -- 172.21.15.33:0/329846386 learned_addr learned my addr 172.21.15.33:0/329846386 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.618+0000 7f4541d9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f453c0bfd70 0x7f453c0c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:51806/0 (socket says 172.21.15.33:51806) 2023-11-06T20:44:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4533fff700 1 -- 172.21.15.33:0/329846386 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500796272 0 0) 0x7f453c1554a0 con 0x7f453c0c1840 2023-11-06T20:44:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4533fff700 1 -- 172.21.15.33:0/329846386 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45280033f0 con 0x7f453c0c1840 2023-11-06T20:44:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4540d9a700 1 -- 172.21.15.33:0/329846386 >> v2:172.21.15.69:3300/0 conn(0x7f453c0bfd70 msgr2=0x7f453c0c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4540d9a700 1 --2- 172.21.15.33:0/329846386 >> v2:172.21.15.69:3300/0 conn(0x7f453c0bfd70 0x7f453c0c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4540d9a700 1 -- 172.21.15.33:0/329846386 >> v1:172.21.15.69:6789/0 conn(0x7f453c0c1840 legacy=0x7f453c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4540d9a700 1 -- 172.21.15.33:0/329846386 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c155930 con 0x7f453c0c5d50 2023-11-06T20:44:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4541d9c700 1 --2- 172.21.15.33:0/329846386 >> v2:172.21.15.69:3300/0 conn(0x7f453c0bfd70 0x7f453c0c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4540d9a700 1 --2- 172.21.15.33:0/329846386 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c5d50 0x7f453c0c61d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f452c006fd0 tx=0x7f452c0090f0).ready entity=mon.2 client_cookie=98a04d6c25f74ef7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4533fff700 1 -- 172.21.15.33:0/329846386 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c00f510 con 0x7f453c0c5d50 2023-11-06T20:44:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4533fff700 1 -- 172.21.15.33:0/329846386 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c00f670 con 0x7f453c0c5d50 2023-11-06T20:44:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.619+0000 7f4533fff700 1 -- 172.21.15.33:0/329846386 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c00f9f0 con 0x7f453c0c5d50 2023-11-06T20:44:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.620+0000 7f45437ff700 1 -- 172.21.15.33:0/329846386 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c5d50 msgr2=0x7f453c0c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.620+0000 7f45437ff700 1 --2- 172.21.15.33:0/329846386 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c5d50 0x7f453c0c61d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f452c006fd0 tx=0x7f452c0090f0).stop 2023-11-06T20:44:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.620+0000 7f45437ff700 1 -- 172.21.15.33:0/329846386 shutdown_connections 2023-11-06T20:44:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.620+0000 7f45437ff700 1 --2- 172.21.15.33:0/329846386 >> v2:172.21.15.69:3300/0 conn(0x7f453c0bfd70 0x7f453c0c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.620+0000 7f45437ff700 1 --2- 172.21.15.33:0/329846386 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c5d50 0x7f453c0c61d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.620+0000 7f45437ff700 1 -- 172.21.15.33:0/329846386 >> 172.21.15.33:0/329846386 conn(0x7f453c05ec80 msgr2=0x7f453c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:09.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.620+0000 7f45437ff700 1 -- 172.21.15.33:0/329846386 shutdown_connections 2023-11-06T20:44:09.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.620+0000 7f45437ff700 1 -- 172.21.15.33:0/329846386 wait complete. 2023-11-06T20:44:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.621+0000 7f45437ff700 1 Processor -- start 2023-11-06T20:44:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.622+0000 7f45437ff700 1 -- start start 2023-11-06T20:44:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.622+0000 7f45437ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f453c0bfd70 0x7f453c0c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.622+0000 7f45437ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c1840 0x7f453c14e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.622+0000 7f45437ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f453c0c5d50 0x7f453c1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.622+0000 7f45437ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c166520 con 0x7f453c0c5d50 2023-11-06T20:44:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.622+0000 7f45437ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c14eba0 con 0x7f453c0bfd70 2023-11-06T20:44:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.622+0000 7f45437ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c165910 con 0x7f453c0c1840 2023-11-06T20:44:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4541d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f453c0c5d50 0x7f453c1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4541d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f453c0c5d50 0x7f453c1564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42648/0 (socket says 172.21.15.33:42648) 2023-11-06T20:44:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4541d9c700 1 -- 172.21.15.33:0/552304134 learned_addr learned my addr 172.21.15.33:0/552304134 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4540d9a700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c1840 0x7f453c14e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f454159b700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f453c0bfd70 0x7f453c0c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4541d9c700 1 -- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c1840 msgr2=0x7f453c14e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4541d9c700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c1840 0x7f453c14e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4541d9c700 1 -- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f453c0bfd70 msgr2=0x7f453c0c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4541d9c700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f453c0bfd70 0x7f453c0c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4541d9c700 1 -- 172.21.15.33:0/552304134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4534009a20 con 0x7f453c0c5d50 2023-11-06T20:44:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4540d9a700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c1840 0x7f453c14e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.623+0000 7f4541d9c700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f453c0c5d50 0x7f453c1564c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f4534009f20 tx=0x7f453400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.624+0000 7f45317fa700 1 -- 172.21.15.33:0/552304134 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534009c80 con 0x7f453c0c5d50 2023-11-06T20:44:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.624+0000 7f45317fa700 1 -- 172.21.15.33:0/552304134 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453400f040 con 0x7f453c0c5d50 2023-11-06T20:44:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.624+0000 7f45437ff700 1 -- 172.21.15.33:0/552304134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c009370 con 0x7f453c0c5d50 2023-11-06T20:44:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.624+0000 7f45317fa700 1 -- 172.21.15.33:0/552304134 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400ce90 con 0x7f453c0c5d50 2023-11-06T20:44:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.624+0000 7f45437ff700 1 -- 172.21.15.33:0/552304134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c165ed0 con 0x7f453c0c5d50 2023-11-06T20:44:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.625+0000 7f45437ff700 1 -- 172.21.15.33:0/552304134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453c166690 con 0x7f453c0c5d50 2023-11-06T20:44:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.626+0000 7f45317fa700 1 -- 172.21.15.33:0/552304134 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4534013610 con 0x7f453c0c5d50 2023-11-06T20:44:09.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.627+0000 7f45317fa700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452005f150 0x7f4520061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:09.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.627+0000 7f45317fa700 1 -- 172.21.15.33:0/552304134 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f453407ab40 con 0x7f453c0c5d50 2023-11-06T20:44:09.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.627+0000 7f454159b700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452005f150 0x7f4520061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:09.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.628+0000 7f454159b700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452005f150 0x7f4520061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f453c0ad2b0 tx=0x7f4538000bc0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:09.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.631+0000 7f45317fa700 1 -- 172.21.15.33:0/552304134 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f453404be10 con 0x7f453c0c5d50 2023-11-06T20:44:09.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.852+0000 7f45437ff700 1 -- 172.21.15.33:0/552304134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f453c06c6c0 con 0x7f453c0c5d50 2023-11-06T20:44:09.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.853+0000 7f45317fa700 1 -- 172.21.15.33:0/552304134 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4480 (secure 0 0 0) 0x7f453404f430 con 0x7f453c0c5d50 2023-11-06T20:44:09.853 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:09.853 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4123875619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123875619},{"type":"v1","addr":"172.21.15.33:6834","nonce":4123875619}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3506554570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3506554570},{"type":"v1","addr":"172.21.15.33:6835","nonce":3506554570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/239985290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":239985290},{"type":"v1","addr":"172.21.15.69:6835","nonce":239985290}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:44:06.383743+0000","modified":"2023-11-06T20:44:09.389408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/2681789033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2681789033},{"type":"v1","addr":"172.21.15.69:6837","nonce":2681789033}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-06T20:44:09.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 -- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452005f150 msgr2=0x7f4520061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:09.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452005f150 0x7f4520061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f453c0ad2b0 tx=0x7f4538000bc0).stop 2023-11-06T20:44:09.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 -- 172.21.15.33:0/552304134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f453c0c5d50 msgr2=0x7f453c1564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:09.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f453c0c5d50 0x7f453c1564c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f4534009f20 tx=0x7f453400b040).stop 2023-11-06T20:44:09.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 -- 172.21.15.33:0/552304134 shutdown_connections 2023-11-06T20:44:09.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452005f150 0x7f4520061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f453c0c5d50 0x7f453c1564c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f453c0bfd70 0x7f453c0c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 --2- 172.21.15.33:0/552304134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f453c0c1840 0x7f453c14e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:09.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 -- 172.21.15.33:0/552304134 >> 172.21.15.33:0/552304134 conn(0x7f453c05ec80 msgr2=0x7f453c151620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:09.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 -- 172.21.15.33:0/552304134 shutdown_connections 2023-11-06T20:44:09.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:09.855+0000 7f45437ff700 1 -- 172.21.15.33:0/552304134 wait complete. 2023-11-06T20:44:09.860 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2023-11-06T20:44:09.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.068+0000 7fa7935ff700 1 Processor -- start 2023-11-06T20:44:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.069+0000 7fa7935ff700 1 -- start start 2023-11-06T20:44:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.070+0000 7fa7935ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa78c0c5d50 0x7fa78c0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.070+0000 7fa7935ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c1554a0 con 0x7fa78c0bfd70 2023-11-06T20:44:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.070+0000 7fa7935ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c0c66c0 con 0x7fa78c0c1840 2023-11-06T20:44:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.070+0000 7fa7935ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78c154960 con 0x7fa78c0c5d50 2023-11-06T20:44:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.070+0000 7fa791b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa78c0bfd70 0x7fa78c0c0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59704/0 (socket says 172.21.15.33:59704) 2023-11-06T20:44:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.070+0000 7fa791b9c700 1 -- 172.21.15.33:0/3659815176 learned_addr learned my addr 172.21.15.33:0/3659815176 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.071+0000 7fa790b9a700 1 --2- 172.21.15.33:0/3659815176 >> v2:172.21.15.69:3300/0 conn(0x7fa78c0c5d50 0x7fa78c0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.071+0000 7fa783fff700 1 -- 172.21.15.33:0/3659815176 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895135546 0 0) 0x7fa78c1554a0 con 0x7fa78c0bfd70 2023-11-06T20:44:10.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.071+0000 7fa783fff700 1 -- 172.21.15.33:0/3659815176 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7780033f0 con 0x7fa78c0bfd70 2023-11-06T20:44:10.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.071+0000 7fa783fff700 1 -- 172.21.15.33:0/3659815176 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921107863 0 0) 0x7fa78c0c66c0 con 0x7fa78c0c1840 2023-11-06T20:44:10.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.071+0000 7fa783fff700 1 -- 172.21.15.33:0/3659815176 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa78c1554a0 con 0x7fa78c0c1840 2023-11-06T20:44:10.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.071+0000 7fa783fff700 1 -- 172.21.15.33:0/3659815176 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 199386363 0 0) 0x7fa7780033f0 con 0x7fa78c0bfd70 2023-11-06T20:44:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.071+0000 7fa790b9a700 1 -- 172.21.15.33:0/3659815176 >> v1:172.21.15.69:6789/0 conn(0x7fa78c0c1840 legacy=0x7fa78c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.071+0000 7fa790b9a700 1 -- 172.21.15.33:0/3659815176 >> v1:172.21.15.33:6789/0 conn(0x7fa78c0bfd70 legacy=0x7fa78c0c0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.071+0000 7fa790b9a700 1 -- 172.21.15.33:0/3659815176 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c155930 con 0x7fa78c0c5d50 2023-11-06T20:44:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.071+0000 7fa790b9a700 1 --2- 172.21.15.33:0/3659815176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa78c0c5d50 0x7fa78c0c61d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fa77c009760 tx=0x7fa77c009060).ready entity=mon.1 client_cookie=3655cdc21d187b1c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.072+0000 7fa783fff700 1 -- 172.21.15.33:0/3659815176 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7840017f0 con 0x7fa78c0c5d50 2023-11-06T20:44:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.072+0000 7fa783fff700 1 -- 172.21.15.33:0/3659815176 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c011400 con 0x7fa78c0c5d50 2023-11-06T20:44:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.072+0000 7fa783fff700 1 -- 172.21.15.33:0/3659815176 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c011780 con 0x7fa78c0c5d50 2023-11-06T20:44:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.072+0000 7fa7935ff700 1 -- 172.21.15.33:0/3659815176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa78c0c5d50 msgr2=0x7fa78c0c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.072+0000 7fa7935ff700 1 --2- 172.21.15.33:0/3659815176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa78c0c5d50 0x7fa78c0c61d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fa77c009760 tx=0x7fa77c009060).stop 2023-11-06T20:44:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.072+0000 7fa7935ff700 1 -- 172.21.15.33:0/3659815176 shutdown_connections 2023-11-06T20:44:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.072+0000 7fa7935ff700 1 --2- 172.21.15.33:0/3659815176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa78c0c5d50 0x7fa78c0c61d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.072+0000 7fa7935ff700 1 -- 172.21.15.33:0/3659815176 >> 172.21.15.33:0/3659815176 conn(0x7fa78c05ec80 msgr2=0x7fa78c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.072+0000 7fa7935ff700 1 -- 172.21.15.33:0/3659815176 shutdown_connections 2023-11-06T20:44:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.072+0000 7fa7935ff700 1 -- 172.21.15.33:0/3659815176 wait complete. 2023-11-06T20:44:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.074+0000 7fa7935ff700 1 Processor -- start 2023-11-06T20:44:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.074+0000 7fa7935ff700 1 -- start start 2023-11-06T20:44:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa7935ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa78c0bfd70 0x7fa78c149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa7935ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78c0c1840 0x7fa78c0c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa7935ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa78c0c5d50 0x7fa78c167cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa7935ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0c46d0 con 0x7fa78c0c1840 2023-11-06T20:44:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa7935ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0c49b0 con 0x7fa78c0bfd70 2023-11-06T20:44:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa7935ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0c4c90 con 0x7fa78c0c5d50 2023-11-06T20:44:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa790b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78c0c1840 0x7fa78c0c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa790b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78c0c1840 0x7fa78c0c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42662/0 (socket says 172.21.15.33:42662) 2023-11-06T20:44:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa790b9a700 1 -- 172.21.15.33:0/3790616421 learned_addr learned my addr 172.21.15.33:0/3790616421 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa79139b700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa78c0bfd70 0x7fa78c149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa791b9c700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa78c0c5d50 0x7fa78c167cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa790b9a700 1 -- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa78c0c5d50 msgr2=0x7fa78c167cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa790b9a700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa78c0c5d50 0x7fa78c167cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa790b9a700 1 -- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa78c0bfd70 msgr2=0x7fa78c149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa790b9a700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa78c0bfd70 0x7fa78c149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa790b9a700 1 -- 172.21.15.33:0/3790616421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c0092a0 con 0x7fa78c0c1840 2023-11-06T20:44:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa79139b700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa78c0bfd70 0x7fa78c149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.075+0000 7fa791b9c700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa78c0c5d50 0x7fa78c167cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.076+0000 7fa790b9a700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78c0c1840 0x7fa78c0c1eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa77c009d50 tx=0x7fa77c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.076+0000 7fa7817fa700 1 -- 172.21.15.33:0/3790616421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0108e0 con 0x7fa78c0c1840 2023-11-06T20:44:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.076+0000 7fa7817fa700 1 -- 172.21.15.33:0/3790616421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c010a40 con 0x7fa78c0c1840 2023-11-06T20:44:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.076+0000 7fa7817fa700 1 -- 172.21.15.33:0/3790616421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c010de0 con 0x7fa78c0c1840 2023-11-06T20:44:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.076+0000 7fa7935ff700 1 -- 172.21.15.33:0/3790616421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c0c4f70 con 0x7fa78c0c1840 2023-11-06T20:44:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.076+0000 7fa7935ff700 1 -- 172.21.15.33:0/3790616421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c149ee0 con 0x7fa78c0c1840 2023-11-06T20:44:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.077+0000 7fa7935ff700 1 -- 172.21.15.33:0/3790616421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78c153600 con 0x7fa78c0c1840 2023-11-06T20:44:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.078+0000 7fa7817fa700 1 -- 172.21.15.33:0/3790616421 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa77c01d640 con 0x7fa78c0c1840 2023-11-06T20:44:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.079+0000 7fa7817fa700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa77005f150 0x7fa770061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.079+0000 7fa7817fa700 1 -- 172.21.15.33:0/3790616421 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa77c07b600 con 0x7fa78c0c1840 2023-11-06T20:44:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.079+0000 7fa79139b700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa77005f150 0x7fa770061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.080+0000 7fa79139b700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa77005f150 0x7fa770061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fa788002750 tx=0x7fa788009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.082+0000 7fa7817fa700 1 -- 172.21.15.33:0/3790616421 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa77c04c8d0 con 0x7fa78c0c1840 2023-11-06T20:44:10.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.265+0000 7fa7935ff700 1 -- 172.21.15.33:0/3790616421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa78c149ee0 con 0x7fa78c0c1840 2023-11-06T20:44:10.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.266+0000 7fa7817fa700 1 -- 172.21.15.33:0/3790616421 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fa78c149ee0 con 0x7fa78c0c1840 2023-11-06T20:44:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:10.266 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.268+0000 7fa7935ff700 1 -- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa77005f150 msgr2=0x7fa770061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.268+0000 7fa7935ff700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa77005f150 0x7fa770061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fa788002750 tx=0x7fa788009040).stop 2023-11-06T20:44:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.268+0000 7fa7935ff700 1 -- 172.21.15.33:0/3790616421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78c0c1840 msgr2=0x7fa78c0c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.268+0000 7fa7935ff700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78c0c1840 0x7fa78c0c1eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa77c009d50 tx=0x7fa77c016000).stop 2023-11-06T20:44:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.269+0000 7fa7935ff700 1 -- 172.21.15.33:0/3790616421 shutdown_connections 2023-11-06T20:44:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.269+0000 7fa7935ff700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa77005f150 0x7fa770061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.269+0000 7fa7935ff700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa78c0c5d50 0x7fa78c167cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.269+0000 7fa7935ff700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa78c0bfd70 0x7fa78c149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.269+0000 7fa7935ff700 1 --2- 172.21.15.33:0/3790616421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78c0c1840 0x7fa78c0c1eb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.269+0000 7fa7935ff700 1 -- 172.21.15.33:0/3790616421 >> 172.21.15.33:0/3790616421 conn(0x7fa78c05ec80 msgr2=0x7fa78c151bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.269+0000 7fa7935ff700 1 -- 172.21.15.33:0/3790616421 shutdown_connections 2023-11-06T20:44:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.269+0000 7fa7935ff700 1 -- 172.21.15.33:0/3790616421 wait complete. 2023-11-06T20:44:10.285 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T20:44:10.286 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:44:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.483+0000 7fe81d1ff700 1 Processor -- start 2023-11-06T20:44:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.483+0000 7fe81d1ff700 1 -- start start 2023-11-06T20:44:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.484+0000 7fe81d1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8180c1840 0x7fe8180a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.484+0000 7fe81d1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8180aa2d0 0x7fe8180c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.484+0000 7fe81d1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8181554a0 con 0x7fe8180c6240 2023-11-06T20:44:10.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.484+0000 7fe81d1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe818154960 con 0x7fe8180c1840 2023-11-06T20:44:10.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.484+0000 7fe81d1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe818155930 con 0x7fe8180aa2d0 2023-11-06T20:44:10.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.484+0000 7fe816d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8180c1840 0x7fe8180a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.485+0000 7fe816d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8180c1840 0x7fe8180a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42678/0 (socket says 172.21.15.33:42678) 2023-11-06T20:44:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.485+0000 7fe816d9d700 1 -- 172.21.15.33:0/2067313561 learned_addr learned my addr 172.21.15.33:0/2067313561 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.485+0000 7fe81659c700 1 --2- 172.21.15.33:0/2067313561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8180aa2d0 0x7fe8180c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.485+0000 7fe816d9d700 1 -- 172.21.15.33:0/2067313561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8180aa2d0 msgr2=0x7fe8180c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.485+0000 7fe816d9d700 1 --2- 172.21.15.33:0/2067313561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8180aa2d0 0x7fe8180c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.485+0000 7fe816d9d700 1 -- 172.21.15.33:0/2067313561 >> v1:172.21.15.33:6789/0 conn(0x7fe8180c6240 legacy=0x7fe818157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.485+0000 7fe815d9b700 1 -- 172.21.15.33:0/2067313561 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656735133 0 0) 0x7fe8181554a0 con 0x7fe8180c6240 2023-11-06T20:44:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.485+0000 7fe816d9d700 1 -- 172.21.15.33:0/2067313561 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8180c66b0 con 0x7fe8180c1840 2023-11-06T20:44:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.485+0000 7fe816d9d700 1 --2- 172.21.15.33:0/2067313561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c1840 0x7fe8180a9de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fe808006fd0 tx=0x7fe8080090f0).ready entity=mon.0 client_cookie=c7762b68a06262cb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.485+0000 7fe815d9b700 1 -- 172.21.15.33:0/2067313561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80800b580 con 0x7fe8180c1840 2023-11-06T20:44:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.486+0000 7fe815d9b700 1 -- 172.21.15.33:0/2067313561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80800b6e0 con 0x7fe8180c1840 2023-11-06T20:44:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.486+0000 7fe815d9b700 1 -- 172.21.15.33:0/2067313561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80800ba60 con 0x7fe8180c1840 2023-11-06T20:44:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.486+0000 7fe81d1ff700 1 -- 172.21.15.33:0/2067313561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c1840 msgr2=0x7fe8180a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.486+0000 7fe81d1ff700 1 --2- 172.21.15.33:0/2067313561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c1840 0x7fe8180a9de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fe808006fd0 tx=0x7fe8080090f0).stop 2023-11-06T20:44:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.486+0000 7fe81d1ff700 1 -- 172.21.15.33:0/2067313561 shutdown_connections 2023-11-06T20:44:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.486+0000 7fe81d1ff700 1 --2- 172.21.15.33:0/2067313561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c1840 0x7fe8180a9de0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.486+0000 7fe81d1ff700 1 --2- 172.21.15.33:0/2067313561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8180aa2d0 0x7fe8180c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.486+0000 7fe81d1ff700 1 -- 172.21.15.33:0/2067313561 >> 172.21.15.33:0/2067313561 conn(0x7fe81805ec80 msgr2=0x7fe8180cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.486+0000 7fe81d1ff700 1 -- 172.21.15.33:0/2067313561 shutdown_connections 2023-11-06T20:44:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.486+0000 7fe81d1ff700 1 -- 172.21.15.33:0/2067313561 wait complete. 2023-11-06T20:44:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.488+0000 7fe81d1ff700 1 Processor -- start 2023-11-06T20:44:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.488+0000 7fe81d1ff700 1 -- start start 2023-11-06T20:44:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81d1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8180aa2d0 0x7fe818156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81d1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8180c1840 0x7fe8180ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81d1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c6240 0x7fe818164cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81d1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe818156e80 con 0x7fe8180c6240 2023-11-06T20:44:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81d1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe818157a00 con 0x7fe8180c1840 2023-11-06T20:44:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81d1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe818157ce0 con 0x7fe8180aa2d0 2023-11-06T20:44:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c6240 0x7fe818164cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c6240 0x7fe818164cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42680/0 (socket says 172.21.15.33:42680) 2023-11-06T20:44:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81759e700 1 -- 172.21.15.33:0/3073074021 learned_addr learned my addr 172.21.15.33:0/3073074021 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81659c700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8180c1840 0x7fe8180ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe816d9d700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8180aa2d0 0x7fe818156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81759e700 1 -- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8180aa2d0 msgr2=0x7fe818156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81759e700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8180aa2d0 0x7fe818156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81759e700 1 -- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8180c1840 msgr2=0x7fe8180ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81759e700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8180c1840 0x7fe8180ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.490+0000 7fe81759e700 1 -- 172.21.15.33:0/3073074021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808009370 con 0x7fe8180c6240 2023-11-06T20:44:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.491+0000 7fe81759e700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c6240 0x7fe818164cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fe810002230 tx=0x7fe810007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.491+0000 7fe7feffd700 1 -- 172.21.15.33:0/3073074021 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810010510 con 0x7fe8180c6240 2023-11-06T20:44:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.491+0000 7fe7feffd700 1 -- 172.21.15.33:0/3073074021 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe810010670 con 0x7fe8180c6240 2023-11-06T20:44:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.491+0000 7fe7feffd700 1 -- 172.21.15.33:0/3073074021 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810010a10 con 0x7fe8180c6240 2023-11-06T20:44:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.491+0000 7fe81d1ff700 1 -- 172.21.15.33:0/3073074021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe818157fc0 con 0x7fe8180c6240 2023-11-06T20:44:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.491+0000 7fe81d1ff700 1 -- 172.21.15.33:0/3073074021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe818158580 con 0x7fe8180c6240 2023-11-06T20:44:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.492+0000 7fe81d1ff700 1 -- 172.21.15.33:0/3073074021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8181587a0 con 0x7fe8180c6240 2023-11-06T20:44:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.493+0000 7fe7feffd700 1 -- 172.21.15.33:0/3073074021 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe81001b400 con 0x7fe8180c6240 2023-11-06T20:44:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.494+0000 7fe7feffd700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe80405f120 0x7fe8040615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.494+0000 7fe7feffd700 1 -- 172.21.15.33:0/3073074021 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe810079fe0 con 0x7fe8180c6240 2023-11-06T20:44:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.494+0000 7fe816d9d700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe80405f120 0x7fe8040615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.495+0000 7fe816d9d700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe80405f120 0x7fe8040615e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe808009fd0 tx=0x7fe808017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.498+0000 7fe7feffd700 1 -- 172.21.15.33:0/3073074021 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe8100a8af0 con 0x7fe8180c6240 2023-11-06T20:44:10.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.674+0000 7fe81d1ff700 1 -- 172.21.15.33:0/3073074021 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe818157e50 con 0x7fe80405f120 2023-11-06T20:44:10.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.686+0000 7fe7feffd700 1 -- 172.21.15.33:0/3073074021 <== mgr.4103 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+327838 (secure 0 0 0) 0x7fe818157e50 con 0x7fe80405f120 2023-11-06T20:44:10.690 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:10.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.690+0000 7fe81d1ff700 1 -- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe80405f120 msgr2=0x7fe8040615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe80405f120 0x7fe8040615e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe808009fd0 tx=0x7fe808017000).stop 2023-11-06T20:44:10.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 -- 172.21.15.33:0/3073074021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c6240 msgr2=0x7fe818164cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c6240 0x7fe818164cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fe810002230 tx=0x7fe810007f70).stop 2023-11-06T20:44:10.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 -- 172.21.15.33:0/3073074021 shutdown_connections 2023-11-06T20:44:10.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe80405f120 0x7fe8040615e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8180aa2d0 0x7fe818156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180c6240 0x7fe818164cc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 --2- 172.21.15.33:0/3073074021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8180c1840 0x7fe8180ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 -- 172.21.15.33:0/3073074021 >> 172.21.15.33:0/3073074021 conn(0x7fe81805ec80 msgr2=0x7fe818152a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:10.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 -- 172.21.15.33:0/3073074021 shutdown_connections 2023-11-06T20:44:10.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.691+0000 7fe81d1ff700 1 -- 172.21.15.33:0/3073074021 wait complete. 2023-11-06T20:44:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:44:10.707 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-11-06T20:44:08.807458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":76,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378132,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596588,"statfs":{"total":773094113280,"available":770658906112,"internally_reserved":0,"allocated":1396736,"data_stored":451855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T20:44:05.926783+0000","last_change":"2023-11-06T20:44:05.918231+0000","last_active":"2023-11-06T20:44:05.926783+0000","last_peered":"2023-11-06T20:44:05.926783+0000","last_clean":"2023-11-06T20:44:05.926783+0000","last_became_active":"2023-11-06T20:44:05.917999+0000","last_became_peered":"2023-11-06T20:44:05.917999+0000","last_unstale":"2023-11-06T20:44:05.926783+0000","last_undegraded":"2023-11-06T20:44:05.926783+0000","last_fullsized":"2023-11-06T20:44:05.926783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395338+0000","last_change":"2023-11-06T20:44:05.930203+0000","last_active":"2023-11-06T20:44:06.395338+0000","last_peered":"2023-11-06T20:44:06.395338+0000","last_clean":"2023-11-06T20:44:06.395338+0000","last_became_active":"2023-11-06T20:44:05.929810+0000","last_became_peered":"2023-11-06T20:44:05.929810+0000","last_unstale":"2023-11-06T20:44:06.395338+0000","last_undegraded":"2023-11-06T20:44:06.395338+0000","last_fullsized":"2023-11-06T20:44:06.395338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416540+0000","last_change":"2023-11-06T20:44:06.406652+0000","last_active":"2023-11-06T20:44:06.416540+0000","last_peered":"2023-11-06T20:44:06.416540+0000","last_clean":"2023-11-06T20:44:06.416540+0000","last_became_active":"2023-11-06T20:44:06.404607+0000","last_became_peered":"2023-11-06T20:44:06.404607+0000","last_unstale":"2023-11-06T20:44:06.416540+0000","last_undegraded":"2023-11-06T20:44:06.416540+0000","last_fullsized":"2023-11-06T20:44:06.416540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"20'4","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426884+0000","last_change":"2023-11-06T20:44:05.909856+0000","last_active":"2023-11-06T20:44:06.426884+0000","last_peered":"2023-11-06T20:44:06.426884+0000","last_clean":"2023-11-06T20:44:06.426884+0000","last_became_active":"2023-11-06T20:44:05.909547+0000","last_became_peered":"2023-11-06T20:44:05.909547+0000","last_unstale":"2023-11-06T20:44:06.426884+0000","last_undegraded":"2023-11-06T20:44:06.426884+0000","last_fullsized":"2023-11-06T20:44:06.426884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412089+0000","last_change":"2023-11-06T20:44:06.412089+0000","last_active":"2023-11-06T20:44:06.412089+0000","last_peered":"2023-11-06T20:44:06.412089+0000","last_clean":"2023-11-06T20:44:06.412089+0000","last_became_active":"2023-11-06T20:44:06.410838+0000","last_became_peered":"2023-11-06T20:44:06.410838+0000","last_unstale":"2023-11-06T20:44:06.412089+0000","last_undegraded":"2023-11-06T20:44:06.412089+0000","last_fullsized":"2023-11-06T20:44:06.412089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421847+0000","last_change":"2023-11-06T20:44:05.926188+0000","last_active":"2023-11-06T20:44:06.421847+0000","last_peered":"2023-11-06T20:44:06.421847+0000","last_clean":"2023-11-06T20:44:06.421847+0000","last_became_active":"2023-11-06T20:44:05.925868+0000","last_became_peered":"2023-11-06T20:44:05.925868+0000","last_unstale":"2023-11-06T20:44:06.421847+0000","last_undegraded":"2023-11-06T20:44:06.421847+0000","last_fullsized":"2023-11-06T20:44:06.421847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.401739+0000","last_change":"2023-11-06T20:44:06.397805+0000","last_active":"2023-11-06T20:44:06.401739+0000","last_peered":"2023-11-06T20:44:06.401739+0000","last_clean":"2023-11-06T20:44:06.401739+0000","last_became_active":"2023-11-06T20:44:06.397479+0000","last_became_peered":"2023-11-06T20:44:06.397479+0000","last_unstale":"2023-11-06T20:44:06.401739+0000","last_undegraded":"2023-11-06T20:44:06.401739+0000","last_fullsized":"2023-11-06T20:44:06.401739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412317+0000","last_change":"2023-11-06T20:44:06.405453+0000","last_active":"2023-11-06T20:44:06.412317+0000","last_peered":"2023-11-06T20:44:06.412317+0000","last_clean":"2023-11-06T20:44:06.412317+0000","last_became_active":"2023-11-06T20:44:06.405101+0000","last_became_peered":"2023-11-06T20:44:06.405101+0000","last_unstale":"2023-11-06T20:44:06.412317+0000","last_undegraded":"2023-11-06T20:44:06.412317+0000","last_fullsized":"2023-11-06T20:44:06.412317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-11-06T20:44:00.955849+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:44:00.955849+0000","last_peered":"2023-11-06T20:44:00.955849+0000","last_clean":"2023-11-06T20:44:00.955849+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:44:00.955849+0000","last_undegraded":"2023-11-06T20:44:00.955849+0000","last_fullsized":"2023-11-06T20:44:00.955849+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393545+0000","last_change":"2023-11-06T20:44:05.916204+0000","last_active":"2023-11-06T20:44:06.393545+0000","last_peered":"2023-11-06T20:44:06.393545+0000","last_clean":"2023-11-06T20:44:06.393545+0000","last_became_active":"2023-11-06T20:44:05.915592+0000","last_became_peered":"2023-11-06T20:44:05.915592+0000","last_unstale":"2023-11-06T20:44:06.393545+0000","last_undegraded":"2023-11-06T20:44:06.393545+0000","last_fullsized":"2023-11-06T20:44:06.393545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-11-06T20:44:00.958148+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:44:00.958148+0000","last_peered":"2023-11-06T20:44:00.958148+0000","last_clean":"2023-11-06T20:44:00.958148+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:44:00.958148+0000","last_undegraded":"2023-11-06T20:44:00.958148+0000","last_fullsized":"2023-11-06T20:44:00.958148+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-11-06T20:44:00.860662+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:44:00.860662+0000","last_peered":"2023-11-06T20:44:00.860662+0000","last_clean":"2023-11-06T20:44:00.860662+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:44:00.860662+0000","last_undegraded":"2023-11-06T20:44:00.860662+0000","last_fullsized":"2023-11-06T20:44:00.860662+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-11-06T20:44:01.006242+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:44:01.006242+0000","last_peered":"2023-11-06T20:44:01.006242+0000","last_clean":"2023-11-06T20:44:01.006242+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:44:01.006242+0000","last_undegraded":"2023-11-06T20:44:01.006242+0000","last_fullsized":"2023-11-06T20:44:01.006242+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T20:44:05.912183+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:44:05.912183+0000","last_peered":"2023-11-06T20:44:05.912183+0000","last_clean":"2023-11-06T20:44:05.912183+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:44:05.912183+0000","last_undegraded":"2023-11-06T20:44:05.912183+0000","last_fullsized":"2023-11-06T20:44:05.912183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-11-06T20:44:00.861678+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:44:00.861678+0000","last_peered":"2023-11-06T20:44:00.861678+0000","last_clean":"2023-11-06T20:44:00.861678+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:44:00.861678+0000","last_undegraded":"2023-11-06T20:44:00.861678+0000","last_fullsized":"2023-11-06T20:44:00.861678+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-11-06T20:44:00.864517+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:44:00.864517+0000","last_peered":"2023-11-06T20:44:00.864517+0000","last_clean":"2023-11-06T20:44:00.864517+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:44:00.864517+0000","last_undegraded":"2023-11-06T20:44:00.864517+0000","last_fullsized":"2023-11-06T20:44:00.864517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-11-06T20:44:00.969382+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:44:00.969382+0000","last_peered":"2023-11-06T20:44:00.969382+0000","last_clean":"2023-11-06T20:44:00.969382+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:44:00.969382+0000","last_undegraded":"2023-11-06T20:44:00.969382+0000","last_fullsized":"2023-11-06T20:44:00.969382+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-11-06T20:44:00.971844+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:44:00.971844+0000","last_peered":"2023-11-06T20:44:00.971844+0000","last_clean":"2023-11-06T20:44:00.971844+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:44:00.971844+0000","last_undegraded":"2023-11-06T20:44:00.971844+0000","last_fullsized":"2023-11-06T20:44:00.971844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.402668+0000","last_change":"2023-11-06T20:44:06.399566+0000","last_active":"2023-11-06T20:44:06.402668+0000","last_peered":"2023-11-06T20:44:06.402668+0000","last_clean":"2023-11-06T20:44:06.402668+0000","last_became_active":"2023-11-06T20:44:06.399283+0000","last_became_peered":"2023-11-06T20:44:06.399283+0000","last_unstale":"2023-11-06T20:44:06.402668+0000","last_undegraded":"2023-11-06T20:44:06.402668+0000","last_fullsized":"2023-11-06T20:44:06.402668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416558+0000","last_change":"2023-11-06T20:44:06.407233+0000","last_active":"2023-11-06T20:44:06.416558+0000","last_peered":"2023-11-06T20:44:06.416558+0000","last_clean":"2023-11-06T20:44:06.416558+0000","last_became_active":"2023-11-06T20:44:06.406927+0000","last_became_peered":"2023-11-06T20:44:06.406927+0000","last_unstale":"2023-11-06T20:44:06.416558+0000","last_undegraded":"2023-11-06T20:44:06.416558+0000","last_fullsized":"2023-11-06T20:44:06.416558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T20:44:05.930658+0000","last_change":"2023-11-06T20:44:05.925954+0000","last_active":"2023-11-06T20:44:05.930658+0000","last_peered":"2023-11-06T20:44:05.930658+0000","last_clean":"2023-11-06T20:44:05.930658+0000","last_became_active":"2023-11-06T20:44:05.925626+0000","last_became_peered":"2023-11-06T20:44:05.925626+0000","last_unstale":"2023-11-06T20:44:05.930658+0000","last_undegraded":"2023-11-06T20:44:05.930658+0000","last_fullsized":"2023-11-06T20:44:05.930658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420472+0000","last_change":"2023-11-06T20:44:06.417902+0000","last_active":"2023-11-06T20:44:06.420472+0000","last_peered":"2023-11-06T20:44:06.420472+0000","last_clean":"2023-11-06T20:44:06.420472+0000","last_became_active":"2023-11-06T20:44:06.417309+0000","last_became_peered":"2023-11-06T20:44:06.417309+0000","last_unstale":"2023-11-06T20:44:06.420472+0000","last_undegraded":"2023-11-06T20:44:06.420472+0000","last_fullsized":"2023-11-06T20:44:06.420472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.415635+0000","last_change":"2023-11-06T20:44:06.412706+0000","last_active":"2023-11-06T20:44:06.415635+0000","last_peered":"2023-11-06T20:44:06.415635+0000","last_clean":"2023-11-06T20:44:06.415635+0000","last_became_active":"2023-11-06T20:44:06.412286+0000","last_became_peered":"2023-11-06T20:44:06.412286+0000","last_unstale":"2023-11-06T20:44:06.415635+0000","last_undegraded":"2023-11-06T20:44:06.415635+0000","last_fullsized":"2023-11-06T20:44:06.415635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.394177+0000","last_change":"2023-11-06T20:44:05.916945+0000","last_active":"2023-11-06T20:44:06.394177+0000","last_peered":"2023-11-06T20:44:06.394177+0000","last_clean":"2023-11-06T20:44:06.394177+0000","last_became_active":"2023-11-06T20:44:05.916459+0000","last_became_peered":"2023-11-06T20:44:05.916459+0000","last_unstale":"2023-11-06T20:44:06.394177+0000","last_undegraded":"2023-11-06T20:44:06.394177+0000","last_fullsized":"2023-11-06T20:44:06.394177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395535+0000","last_change":"2023-11-06T20:44:05.922285+0000","last_active":"2023-11-06T20:44:06.395535+0000","last_peered":"2023-11-06T20:44:06.395535+0000","last_clean":"2023-11-06T20:44:06.395535+0000","last_became_active":"2023-11-06T20:44:05.921952+0000","last_became_peered":"2023-11-06T20:44:05.921952+0000","last_unstale":"2023-11-06T20:44:06.395535+0000","last_undegraded":"2023-11-06T20:44:06.395535+0000","last_fullsized":"2023-11-06T20:44:06.395535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.411774+0000","last_change":"2023-11-06T20:44:06.411774+0000","last_active":"2023-11-06T20:44:06.411774+0000","last_peered":"2023-11-06T20:44:06.411774+0000","last_clean":"2023-11-06T20:44:06.411774+0000","last_became_active":"2023-11-06T20:44:06.411426+0000","last_became_peered":"2023-11-06T20:44:06.411426+0000","last_unstale":"2023-11-06T20:44:06.411774+0000","last_undegraded":"2023-11-06T20:44:06.411774+0000","last_fullsized":"2023-11-06T20:44:06.411774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T20:44:05.930903+0000","last_change":"2023-11-06T20:44:05.928517+0000","last_active":"2023-11-06T20:44:05.930903+0000","last_peered":"2023-11-06T20:44:05.930903+0000","last_clean":"2023-11-06T20:44:05.930903+0000","last_became_active":"2023-11-06T20:44:05.928197+0000","last_became_peered":"2023-11-06T20:44:05.928197+0000","last_unstale":"2023-11-06T20:44:05.930903+0000","last_undegraded":"2023-11-06T20:44:05.930903+0000","last_fullsized":"2023-11-06T20:44:05.930903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.414429+0000","last_change":"2023-11-06T20:44:06.414429+0000","last_active":"2023-11-06T20:44:06.414429+0000","last_peered":"2023-11-06T20:44:06.414429+0000","last_clean":"2023-11-06T20:44:06.414429+0000","last_became_active":"2023-11-06T20:44:06.411089+0000","last_became_peered":"2023-11-06T20:44:06.411089+0000","last_unstale":"2023-11-06T20:44:06.414429+0000","last_undegraded":"2023-11-06T20:44:06.414429+0000","last_fullsized":"2023-11-06T20:44:06.414429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.409877+0000","last_change":"2023-11-06T20:44:06.409877+0000","last_active":"2023-11-06T20:44:06.409877+0000","last_peered":"2023-11-06T20:44:06.409877+0000","last_clean":"2023-11-06T20:44:06.409877+0000","last_became_active":"2023-11-06T20:44:06.408631+0000","last_became_peered":"2023-11-06T20:44:06.408631+0000","last_unstale":"2023-11-06T20:44:06.409877+0000","last_undegraded":"2023-11-06T20:44:06.409877+0000","last_fullsized":"2023-11-06T20:44:06.409877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407493+0000","last_change":"2023-11-06T20:44:06.407493+0000","last_active":"2023-11-06T20:44:06.407493+0000","last_peered":"2023-11-06T20:44:06.407493+0000","last_clean":"2023-11-06T20:44:06.407493+0000","last_became_active":"2023-11-06T20:44:06.407167+0000","last_became_peered":"2023-11-06T20:44:06.407167+0000","last_unstale":"2023-11-06T20:44:06.407493+0000","last_undegraded":"2023-11-06T20:44:06.407493+0000","last_fullsized":"2023-11-06T20:44:06.407493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407092+0000","last_change":"2023-11-06T20:44:06.407092+0000","last_active":"2023-11-06T20:44:06.407092+0000","last_peered":"2023-11-06T20:44:06.407092+0000","last_clean":"2023-11-06T20:44:06.407092+0000","last_became_active":"2023-11-06T20:44:06.406767+0000","last_became_peered":"2023-11-06T20:44:06.406767+0000","last_unstale":"2023-11-06T20:44:06.407092+0000","last_undegraded":"2023-11-06T20:44:06.407092+0000","last_fullsized":"2023-11-06T20:44:06.407092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419551+0000","last_change":"2023-11-06T20:44:06.419551+0000","last_active":"2023-11-06T20:44:06.419551+0000","last_peered":"2023-11-06T20:44:06.419551+0000","last_clean":"2023-11-06T20:44:06.419551+0000","last_became_active":"2023-11-06T20:44:06.419244+0000","last_became_peered":"2023-11-06T20:44:06.419244+0000","last_unstale":"2023-11-06T20:44:06.419551+0000","last_undegraded":"2023-11-06T20:44:06.419551+0000","last_fullsized":"2023-11-06T20:44:06.419551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399159+0000","last_change":"2023-11-06T20:44:06.399159+0000","last_active":"2023-11-06T20:44:06.399159+0000","last_peered":"2023-11-06T20:44:06.399159+0000","last_clean":"2023-11-06T20:44:06.399159+0000","last_became_active":"2023-11-06T20:44:06.398305+0000","last_became_peered":"2023-11-06T20:44:06.398305+0000","last_unstale":"2023-11-06T20:44:06.399159+0000","last_undegraded":"2023-11-06T20:44:06.399159+0000","last_fullsized":"2023-11-06T20:44:06.399159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T20:44:05.940218+0000","last_change":"2023-11-06T20:44:05.939218+0000","last_active":"2023-11-06T20:44:05.940218+0000","last_peered":"2023-11-06T20:44:05.940218+0000","last_clean":"2023-11-06T20:44:05.940218+0000","last_became_active":"2023-11-06T20:44:05.938856+0000","last_became_peered":"2023-11-06T20:44:05.938856+0000","last_unstale":"2023-11-06T20:44:05.940218+0000","last_undegraded":"2023-11-06T20:44:05.940218+0000","last_fullsized":"2023-11-06T20:44:05.940218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416917+0000","last_change":"2023-11-06T20:44:06.412444+0000","last_active":"2023-11-06T20:44:06.416917+0000","last_peered":"2023-11-06T20:44:06.416917+0000","last_clean":"2023-11-06T20:44:06.416917+0000","last_became_active":"2023-11-06T20:44:06.412113+0000","last_became_peered":"2023-11-06T20:44:06.412113+0000","last_unstale":"2023-11-06T20:44:06.416917+0000","last_undegraded":"2023-11-06T20:44:06.416917+0000","last_fullsized":"2023-11-06T20:44:06.416917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420060+0000","last_change":"2023-11-06T20:44:05.909687+0000","last_active":"2023-11-06T20:44:06.420060+0000","last_peered":"2023-11-06T20:44:06.420060+0000","last_clean":"2023-11-06T20:44:06.420060+0000","last_became_active":"2023-11-06T20:44:05.907015+0000","last_became_peered":"2023-11-06T20:44:05.907015+0000","last_unstale":"2023-11-06T20:44:06.420060+0000","last_undegraded":"2023-11-06T20:44:06.420060+0000","last_fullsized":"2023-11-06T20:44:06.420060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420499+0000","last_change":"2023-11-06T20:44:05.919149+0000","last_active":"2023-11-06T20:44:06.420499+0000","last_peered":"2023-11-06T20:44:06.420499+0000","last_clean":"2023-11-06T20:44:06.420499+0000","last_became_active":"2023-11-06T20:44:05.918833+0000","last_became_peered":"2023-11-06T20:44:05.918833+0000","last_unstale":"2023-11-06T20:44:06.420499+0000","last_undegraded":"2023-11-06T20:44:06.420499+0000","last_fullsized":"2023-11-06T20:44:06.420499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:05.880924+0000","last_change":"2023-11-06T20:44:05.880924+0000","last_active":"2023-11-06T20:44:05.880924+0000","last_peered":"2023-11-06T20:44:05.880924+0000","last_clean":"2023-11-06T20:44:05.880924+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:05.880924+0000","last_undegraded":"2023-11-06T20:44:05.880924+0000","last_fullsized":"2023-11-06T20:44:05.880924+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.410070+0000","last_change":"2023-11-06T20:44:06.410070+0000","last_active":"2023-11-06T20:44:06.410070+0000","last_peered":"2023-11-06T20:44:06.410070+0000","last_clean":"2023-11-06T20:44:06.410070+0000","last_became_active":"2023-11-06T20:44:06.409768+0000","last_became_peered":"2023-11-06T20:44:06.409768+0000","last_unstale":"2023-11-06T20:44:06.410070+0000","last_undegraded":"2023-11-06T20:44:06.410070+0000","last_fullsized":"2023-11-06T20:44:06.410070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:44:04.874230+0000","last_change":"2023-11-06T20:44:04.874230+0000","last_active":"2023-11-06T20:44:04.874230+0000","last_peered":"2023-11-06T20:44:04.874230+0000","last_clean":"2023-11-06T20:44:04.874230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:44:04.874230+0000","last_undegraded":"2023-11-06T20:44:04.874230+0000","last_fullsized":"2023-11-06T20:44:04.874230+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":34,"acting":34,"num_store_stats":2},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":24,"acting":24,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574850,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297260,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074580,"statfs":{"total":96636764160,"available":96332369920,"internally_reserved":0,"allocated":167936,"data_stored":53442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297248,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074592,"statfs":{"total":96636764160,"available":96332382208,"internally_reserved":0,"allocated":155648,"data_stored":47643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":74435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297248,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074592,"statfs":{"total":96636764160,"available":96332382208,"internally_reserved":0,"allocated":155648,"data_stored":47643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":74165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297260,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074580,"statfs":{"total":96636764160,"available":96332369920,"internally_reserved":0,"allocated":167936,"data_stored":53442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297260,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074580,"statfs":{"total":96636764160,"available":96332369920,"internally_reserved":0,"allocated":167936,"data_stored":53442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297248,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074592,"statfs":{"total":96636764160,"available":96332382208,"internally_reserved":0,"allocated":155648,"data_stored":47643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:44:10.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:44:10.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.944+0000 7f0cd87ff700 1 Processor -- start 2023-11-06T20:44:10.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.944+0000 7f0cd87ff700 1 -- start start 2023-11-06T20:44:10.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd87ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd00c5d50 0x7f0cd00c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd87ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0cd00bfd70 0x7f0cd00c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd87ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd01554a0 con 0x7f0cd00c1840 2023-11-06T20:44:10.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd87ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd0154960 con 0x7f0cd00bfd70 2023-11-06T20:44:10.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd87ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd00c66c0 con 0x7f0cd00c5d50 2023-11-06T20:44:10.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd659b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0cd00c1840 0x7f0cd00a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59718/0 (socket says 172.21.15.33:59718) 2023-11-06T20:44:10.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd659b700 1 -- 172.21.15.33:0/3495450010 learned_addr learned my addr 172.21.15.33:0/3495450010 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:10.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd6d9c700 1 --2- 172.21.15.33:0/3495450010 >> v2:172.21.15.69:3300/0 conn(0x7f0cd00bfd70 0x7f0cd00c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854224249 0 0) 0x7f0cd01554a0 con 0x7f0cd00c1840 2023-11-06T20:44:10.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd5d9a700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd00c5d50 0x7f0cd00c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.946+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cbc0033f0 con 0x7f0cd00c1840 2023-11-06T20:44:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1247986060 0 0) 0x7f0cbc0033f0 con 0x7f0cd00c1840 2023-11-06T20:44:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd00c5d50 msgr2=0x7f0cd00c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd00c5d50 0x7f0cd00c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 >> v2:172.21.15.69:3300/0 conn(0x7f0cd00bfd70 msgr2=0x7f0cd00c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 --2- 172.21.15.33:0/3495450010 >> v2:172.21.15.69:3300/0 conn(0x7f0cd00bfd70 0x7f0cd00c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0155930 con 0x7f0cd00c1840 2023-11-06T20:44:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f0ccc001470 con 0x7f0cd00c1840 2023-11-06T20:44:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 >> v1:172.21.15.33:6789/0 conn(0x7f0cd00c1840 legacy=0x7f0cd00a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd00c5d50 0x7f0cbc0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cbc003e10 0x7f0cbc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc004b80 0x7f0cbc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0154960 con 0x7f0cbc004b80 2023-11-06T20:44:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd00c66c0 con 0x7f0cbc003e10 2023-11-06T20:44:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc00b8e0 con 0x7f0cd00c5d50 2023-11-06T20:44:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd6d9c700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc004b80 0x7f0cbc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd659b700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd00c5d50 0x7f0cbc0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5d9a700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cbc003e10 0x7f0cbc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd6d9c700 1 -- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd00c5d50 msgr2=0x7f0cbc0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:10.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd6d9c700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd00c5d50 0x7f0cbc0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd6d9c700 1 -- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cbc003e10 msgr2=0x7f0cbc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:10.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd6d9c700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cbc003e10 0x7f0cbc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd6d9c700 1 -- 172.21.15.33:0/3495450010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc0033f0 con 0x7f0cbc004b80 2023-11-06T20:44:10.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.947+0000 7f0cd5d9a700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cbc003e10 0x7f0cbc004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:10.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.948+0000 7f0cd6d9c700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc004b80 0x7f0cbc005000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f0cc800e240 tx=0x7f0cc800b040).ready entity=mon.0 client_cookie=30aef3b01946361c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:10.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.948+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8009ac0 con 0x7f0cbc004b80 2023-11-06T20:44:10.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.948+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc800f040 con 0x7f0cbc004b80 2023-11-06T20:44:10.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.948+0000 7f0cd5599700 1 -- 172.21.15.33:0/3495450010 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800ed00 con 0x7f0cbc004b80 2023-11-06T20:44:10.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.948+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3495450010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc004b80 msgr2=0x7f0cbc005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.948+0000 7f0cd87ff700 1 --2- 172.21.15.33:0/3495450010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc004b80 0x7f0cbc005000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f0cc800e240 tx=0x7f0cc800b040).stop 2023-11-06T20:44:10.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.948+0000 7f0cd6d9c700 1 -- 172.21.15.33:0/3495450010 reap_dead start 2023-11-06T20:44:10.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.948+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3495450010 shutdown_connections 2023-11-06T20:44:10.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.948+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3495450010 >> 172.21.15.33:0/3495450010 conn(0x7f0cd005ec80 msgr2=0x7f0cd00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:10.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.949+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3495450010 shutdown_connections 2023-11-06T20:44:10.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.949+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3495450010 wait complete. 2023-11-06T20:44:10.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.950+0000 7f0cd87ff700 1 Processor -- start 2023-11-06T20:44:10.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.950+0000 7f0cd87ff700 1 -- start start 2023-11-06T20:44:10.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.951+0000 7f0cd87ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cbc004b80 0x7f0cd00abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.951+0000 7f0cd87ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd01651e0 0x7f0cd0164b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.951+0000 7f0cd87ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0157f70 0x7f0cd01583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.951+0000 7f0cd87ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd00ac300 con 0x7f0cd0157f70 2023-11-06T20:44:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.951+0000 7f0cd87ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd00aee70 con 0x7f0cbc004b80 2023-11-06T20:44:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.951+0000 7f0cd87ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0164ff0 con 0x7f0cd01651e0 2023-11-06T20:44:10.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.951+0000 7f0cd6d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0157f70 0x7f0cd01583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.951+0000 7f0cd659b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cbc004b80 0x7f0cd00abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.951+0000 7f0cd5d9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd01651e0 0x7f0cd0164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd5d9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd01651e0 0x7f0cd0164b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:51250/0 (socket says 172.21.15.33:51250) 2023-11-06T20:44:10.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd5d9a700 1 -- 172.21.15.33:0/3722544498 learned_addr learned my addr 172.21.15.33:0/3722544498 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:10.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd6d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0157f70 0x7f0cd01583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42694/0 (socket says 172.21.15.33:42694) 2023-11-06T20:44:10.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd6d9c700 1 -- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd01651e0 msgr2=0x7f0cd0164b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd6d9c700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd01651e0 0x7f0cd0164b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd6d9c700 1 -- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cbc004b80 msgr2=0x7f0cd00abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:10.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd6d9c700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cbc004b80 0x7f0cd00abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:10.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd6d9c700 1 -- 172.21.15.33:0/3722544498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd00af000 con 0x7f0cd0157f70 2023-11-06T20:44:10.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd6d9c700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0157f70 0x7f0cd01583f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f0cc8017000 tx=0x7f0cc801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:10.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cc67fc700 1 -- 172.21.15.33:0/3722544498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc80138f0 con 0x7f0cd0157f70 2023-11-06T20:44:10.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cc67fc700 1 -- 172.21.15.33:0/3722544498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc8021070 con 0x7f0cd0157f70 2023-11-06T20:44:10.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3722544498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd00ac4f0 con 0x7f0cd0157f70 2023-11-06T20:44:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cc67fc700 1 -- 172.21.15.33:0/3722544498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8013a90 con 0x7f0cd0157f70 2023-11-06T20:44:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.952+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3722544498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd00acab0 con 0x7f0cd0157f70 2023-11-06T20:44:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.953+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3722544498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd00af190 con 0x7f0cd0157f70 2023-11-06T20:44:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.954+0000 7f0cc67fc700 1 -- 172.21.15.33:0/3722544498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0cc8026420 con 0x7f0cd0157f70 2023-11-06T20:44:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.955+0000 7f0cc67fc700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cb405f150 0x7f0cb4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.955+0000 7f0cc67fc700 1 -- 172.21.15.33:0/3722544498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0cc807b360 con 0x7f0cd0157f70 2023-11-06T20:44:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.956+0000 7f0cd659b700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cb405f150 0x7f0cb4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.956+0000 7f0cd659b700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cb405f150 0x7f0cb4061610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0ccc0010e0 tx=0x7f0cd0155410).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:10.960+0000 7f0cc67fc700 1 -- 172.21.15.33:0/3722544498 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0cc804c510 con 0x7f0cd0157f70 2023-11-06T20:44:11.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.140+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3722544498 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0cd006c6c0 con 0x7f0cb405f150 2023-11-06T20:44:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.150+0000 7f0cc67fc700 1 -- 172.21.15.33:0/3722544498 <== mgr.4103 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336499 (secure 0 0 0) 0x7f0cd006c6c0 con 0x7f0cb405f150 2023-11-06T20:44:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:11.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.154+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cb405f150 msgr2=0x7f0cb4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.154+0000 7f0cd87ff700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cb405f150 0x7f0cb4061610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0ccc0010e0 tx=0x7f0cd0155410).stop 2023-11-06T20:44:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.154+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3722544498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0157f70 msgr2=0x7f0cd01583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.154+0000 7f0cd87ff700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0157f70 0x7f0cd01583f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f0cc8017000 tx=0x7f0cc801a000).stop 2023-11-06T20:44:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.155+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3722544498 shutdown_connections 2023-11-06T20:44:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.155+0000 7f0cd87ff700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cb405f150 0x7f0cb4061610 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.155+0000 7f0cd87ff700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cd0157f70 0x7f0cd01583f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.155+0000 7f0cd87ff700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cbc004b80 0x7f0cd00abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.155+0000 7f0cd87ff700 1 --2- 172.21.15.33:0/3722544498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cd01651e0 0x7f0cd0164b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.155+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3722544498 >> 172.21.15.33:0/3722544498 conn(0x7f0cd005ec80 msgr2=0x7f0cd00c0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.155+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3722544498 shutdown_connections 2023-11-06T20:44:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.155+0000 7f0cd87ff700 1 -- 172.21.15.33:0/3722544498 wait complete. 2023-11-06T20:44:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:44:11.171 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-06T20:44:10.807917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":244,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378468,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596252,"statfs":{"total":773094113280,"available":770658562048,"internally_reserved":0,"allocated":1740800,"data_stored":612892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000459"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427856+0000","last_change":"2023-11-06T20:44:06.424455+0000","last_active":"2023-11-06T20:44:06.427856+0000","last_peered":"2023-11-06T20:44:06.427856+0000","last_clean":"2023-11-06T20:44:06.427856+0000","last_became_active":"2023-11-06T20:44:06.424120+0000","last_became_peered":"2023-11-06T20:44:06.424120+0000","last_unstale":"2023-11-06T20:44:06.427856+0000","last_undegraded":"2023-11-06T20:44:06.427856+0000","last_fullsized":"2023-11-06T20:44:06.427856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T20:44:05.926783+0000","last_change":"2023-11-06T20:44:05.918231+0000","last_active":"2023-11-06T20:44:05.926783+0000","last_peered":"2023-11-06T20:44:05.926783+0000","last_clean":"2023-11-06T20:44:05.926783+0000","last_became_active":"2023-11-06T20:44:05.917999+0000","last_became_peered":"2023-11-06T20:44:05.917999+0000","last_unstale":"2023-11-06T20:44:05.926783+0000","last_undegraded":"2023-11-06T20:44:05.926783+0000","last_fullsized":"2023-11-06T20:44:05.926783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420202+0000","last_change":"2023-11-06T20:44:06.413802+0000","last_active":"2023-11-06T20:44:06.420202+0000","last_peered":"2023-11-06T20:44:06.420202+0000","last_clean":"2023-11-06T20:44:06.420202+0000","last_became_active":"2023-11-06T20:44:06.412830+0000","last_became_peered":"2023-11-06T20:44:06.412830+0000","last_unstale":"2023-11-06T20:44:06.420202+0000","last_undegraded":"2023-11-06T20:44:06.420202+0000","last_fullsized":"2023-11-06T20:44:06.420202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.402079+0000","last_change":"2023-11-06T20:44:05.922105+0000","last_active":"2023-11-06T20:44:06.402079+0000","last_peered":"2023-11-06T20:44:06.402079+0000","last_clean":"2023-11-06T20:44:06.402079+0000","last_became_active":"2023-11-06T20:44:05.921727+0000","last_became_peered":"2023-11-06T20:44:05.921727+0000","last_unstale":"2023-11-06T20:44:06.402079+0000","last_undegraded":"2023-11-06T20:44:06.402079+0000","last_fullsized":"2023-11-06T20:44:06.402079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426592+0000","last_change":"2023-11-06T20:44:06.411322+0000","last_active":"2023-11-06T20:44:06.426592+0000","last_peered":"2023-11-06T20:44:06.426592+0000","last_clean":"2023-11-06T20:44:06.426592+0000","last_became_active":"2023-11-06T20:44:06.409787+0000","last_became_peered":"2023-11-06T20:44:06.409787+0000","last_unstale":"2023-11-06T20:44:06.426592+0000","last_undegraded":"2023-11-06T20:44:06.426592+0000","last_fullsized":"2023-11-06T20:44:06.426592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393182+0000","last_change":"2023-11-06T20:44:05.920323+0000","last_active":"2023-11-06T20:44:06.393182+0000","last_peered":"2023-11-06T20:44:06.393182+0000","last_clean":"2023-11-06T20:44:06.393182+0000","last_became_active":"2023-11-06T20:44:05.919781+0000","last_became_peered":"2023-11-06T20:44:05.919781+0000","last_unstale":"2023-11-06T20:44:06.393182+0000","last_undegraded":"2023-11-06T20:44:06.393182+0000","last_fullsized":"2023-11-06T20:44:06.393182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.428239+0000","last_change":"2023-11-06T20:44:06.426498+0000","last_active":"2023-11-06T20:44:06.428239+0000","last_peered":"2023-11-06T20:44:06.428239+0000","last_clean":"2023-11-06T20:44:06.428239+0000","last_became_active":"2023-11-06T20:44:06.426148+0000","last_became_peered":"2023-11-06T20:44:06.426148+0000","last_unstale":"2023-11-06T20:44:06.428239+0000","last_undegraded":"2023-11-06T20:44:06.428239+0000","last_fullsized":"2023-11-06T20:44:06.428239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395364+0000","last_change":"2023-11-06T20:44:05.934770+0000","last_active":"2023-11-06T20:44:06.395364+0000","last_peered":"2023-11-06T20:44:06.395364+0000","last_clean":"2023-11-06T20:44:06.395364+0000","last_became_active":"2023-11-06T20:44:05.934479+0000","last_became_peered":"2023-11-06T20:44:05.934479+0000","last_unstale":"2023-11-06T20:44:06.395364+0000","last_undegraded":"2023-11-06T20:44:06.395364+0000","last_fullsized":"2023-11-06T20:44:06.395364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425272+0000","last_change":"2023-11-06T20:44:06.422189+0000","last_active":"2023-11-06T20:44:06.425272+0000","last_peered":"2023-11-06T20:44:06.425272+0000","last_clean":"2023-11-06T20:44:06.425272+0000","last_became_active":"2023-11-06T20:44:06.421946+0000","last_became_peered":"2023-11-06T20:44:06.421946+0000","last_unstale":"2023-11-06T20:44:06.425272+0000","last_undegraded":"2023-11-06T20:44:06.425272+0000","last_fullsized":"2023-11-06T20:44:06.425272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395338+0000","last_change":"2023-11-06T20:44:05.930203+0000","last_active":"2023-11-06T20:44:06.395338+0000","last_peered":"2023-11-06T20:44:06.395338+0000","last_clean":"2023-11-06T20:44:06.395338+0000","last_became_active":"2023-11-06T20:44:05.929810+0000","last_became_peered":"2023-11-06T20:44:05.929810+0000","last_unstale":"2023-11-06T20:44:06.395338+0000","last_undegraded":"2023-11-06T20:44:06.395338+0000","last_fullsized":"2023-11-06T20:44:06.395338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.428278+0000","last_change":"2023-11-06T20:44:06.425515+0000","last_active":"2023-11-06T20:44:06.428278+0000","last_peered":"2023-11-06T20:44:06.428278+0000","last_clean":"2023-11-06T20:44:06.428278+0000","last_became_active":"2023-11-06T20:44:06.425144+0000","last_became_peered":"2023-11-06T20:44:06.425144+0000","last_unstale":"2023-11-06T20:44:06.428278+0000","last_undegraded":"2023-11-06T20:44:06.428278+0000","last_fullsized":"2023-11-06T20:44:06.428278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424086+0000","last_change":"2023-11-06T20:44:05.930915+0000","last_active":"2023-11-06T20:44:06.424086+0000","last_peered":"2023-11-06T20:44:06.424086+0000","last_clean":"2023-11-06T20:44:06.424086+0000","last_became_active":"2023-11-06T20:44:05.928587+0000","last_became_peered":"2023-11-06T20:44:05.928587+0000","last_unstale":"2023-11-06T20:44:06.424086+0000","last_undegraded":"2023-11-06T20:44:06.424086+0000","last_fullsized":"2023-11-06T20:44:06.424086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416540+0000","last_change":"2023-11-06T20:44:06.406652+0000","last_active":"2023-11-06T20:44:06.416540+0000","last_peered":"2023-11-06T20:44:06.416540+0000","last_clean":"2023-11-06T20:44:06.416540+0000","last_became_active":"2023-11-06T20:44:06.404607+0000","last_became_peered":"2023-11-06T20:44:06.404607+0000","last_unstale":"2023-11-06T20:44:06.416540+0000","last_undegraded":"2023-11-06T20:44:06.416540+0000","last_fullsized":"2023-11-06T20:44:06.416540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.428170+0000","last_change":"2023-11-06T20:44:05.921294+0000","last_active":"2023-11-06T20:44:06.428170+0000","last_peered":"2023-11-06T20:44:06.428170+0000","last_clean":"2023-11-06T20:44:06.428170+0000","last_became_active":"2023-11-06T20:44:05.921040+0000","last_became_peered":"2023-11-06T20:44:05.921040+0000","last_unstale":"2023-11-06T20:44:06.428170+0000","last_undegraded":"2023-11-06T20:44:06.428170+0000","last_fullsized":"2023-11-06T20:44:06.428170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420418+0000","last_change":"2023-11-06T20:44:06.414350+0000","last_active":"2023-11-06T20:44:06.420418+0000","last_peered":"2023-11-06T20:44:06.420418+0000","last_clean":"2023-11-06T20:44:06.420418+0000","last_became_active":"2023-11-06T20:44:06.413471+0000","last_became_peered":"2023-11-06T20:44:06.413471+0000","last_unstale":"2023-11-06T20:44:06.420418+0000","last_undegraded":"2023-11-06T20:44:06.420418+0000","last_fullsized":"2023-11-06T20:44:06.420418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427433+0000","last_change":"2023-11-06T20:44:05.931968+0000","last_active":"2023-11-06T20:44:06.427433+0000","last_peered":"2023-11-06T20:44:06.427433+0000","last_clean":"2023-11-06T20:44:06.427433+0000","last_became_active":"2023-11-06T20:44:05.931666+0000","last_became_peered":"2023-11-06T20:44:05.931666+0000","last_unstale":"2023-11-06T20:44:06.427433+0000","last_undegraded":"2023-11-06T20:44:06.427433+0000","last_fullsized":"2023-11-06T20:44:06.427433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427742+0000","last_change":"2023-11-06T20:44:06.421789+0000","last_active":"2023-11-06T20:44:06.427742+0000","last_peered":"2023-11-06T20:44:06.427742+0000","last_clean":"2023-11-06T20:44:06.427742+0000","last_became_active":"2023-11-06T20:44:06.421065+0000","last_became_peered":"2023-11-06T20:44:06.421065+0000","last_unstale":"2023-11-06T20:44:06.427742+0000","last_undegraded":"2023-11-06T20:44:06.427742+0000","last_fullsized":"2023-11-06T20:44:06.427742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.398924+0000","last_change":"2023-11-06T20:44:05.936868+0000","last_active":"2023-11-06T20:44:06.398924+0000","last_peered":"2023-11-06T20:44:06.398924+0000","last_clean":"2023-11-06T20:44:06.398924+0000","last_became_active":"2023-11-06T20:44:05.936617+0000","last_became_peered":"2023-11-06T20:44:05.936617+0000","last_unstale":"2023-11-06T20:44:06.398924+0000","last_undegraded":"2023-11-06T20:44:06.398924+0000","last_fullsized":"2023-11-06T20:44:06.398924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421281+0000","last_change":"2023-11-06T20:44:06.409854+0000","last_active":"2023-11-06T20:44:06.421281+0000","last_peered":"2023-11-06T20:44:06.421281+0000","last_clean":"2023-11-06T20:44:06.421281+0000","last_became_active":"2023-11-06T20:44:06.409400+0000","last_became_peered":"2023-11-06T20:44:06.409400+0000","last_unstale":"2023-11-06T20:44:06.421281+0000","last_undegraded":"2023-11-06T20:44:06.421281+0000","last_fullsized":"2023-11-06T20:44:06.421281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.390859+0000","last_change":"2023-11-06T20:44:05.920464+0000","last_active":"2023-11-06T20:44:06.390859+0000","last_peered":"2023-11-06T20:44:06.390859+0000","last_clean":"2023-11-06T20:44:06.390859+0000","last_became_active":"2023-11-06T20:44:05.920179+0000","last_became_peered":"2023-11-06T20:44:05.920179+0000","last_unstale":"2023-11-06T20:44:06.390859+0000","last_undegraded":"2023-11-06T20:44:06.390859+0000","last_fullsized":"2023-11-06T20:44:06.390859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.410888+0000","last_change":"2023-11-06T20:44:06.404170+0000","last_active":"2023-11-06T20:44:06.410888+0000","last_peered":"2023-11-06T20:44:06.410888+0000","last_clean":"2023-11-06T20:44:06.410888+0000","last_became_active":"2023-11-06T20:44:06.403838+0000","last_became_peered":"2023-11-06T20:44:06.403838+0000","last_unstale":"2023-11-06T20:44:06.410888+0000","last_undegraded":"2023-11-06T20:44:06.410888+0000","last_fullsized":"2023-11-06T20:44:06.410888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'4","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426884+0000","last_change":"2023-11-06T20:44:05.909856+0000","last_active":"2023-11-06T20:44:06.426884+0000","last_peered":"2023-11-06T20:44:06.426884+0000","last_clean":"2023-11-06T20:44:06.426884+0000","last_became_active":"2023-11-06T20:44:05.909547+0000","last_became_peered":"2023-11-06T20:44:05.909547+0000","last_unstale":"2023-11-06T20:44:06.426884+0000","last_undegraded":"2023-11-06T20:44:06.426884+0000","last_fullsized":"2023-11-06T20:44:06.426884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412684+0000","last_change":"2023-11-06T20:44:06.401237+0000","last_active":"2023-11-06T20:44:06.412684+0000","last_peered":"2023-11-06T20:44:06.412684+0000","last_clean":"2023-11-06T20:44:06.412684+0000","last_became_active":"2023-11-06T20:44:06.399700+0000","last_became_peered":"2023-11-06T20:44:06.399700+0000","last_unstale":"2023-11-06T20:44:06.412684+0000","last_undegraded":"2023-11-06T20:44:06.412684+0000","last_fullsized":"2023-11-06T20:44:06.412684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421048+0000","last_change":"2023-11-06T20:44:05.930204+0000","last_active":"2023-11-06T20:44:06.421048+0000","last_peered":"2023-11-06T20:44:06.421048+0000","last_clean":"2023-11-06T20:44:06.421048+0000","last_became_active":"2023-11-06T20:44:05.929904+0000","last_became_peered":"2023-11-06T20:44:05.929904+0000","last_unstale":"2023-11-06T20:44:06.421048+0000","last_undegraded":"2023-11-06T20:44:06.421048+0000","last_fullsized":"2023-11-06T20:44:06.421048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412089+0000","last_change":"2023-11-06T20:44:06.412089+0000","last_active":"2023-11-06T20:44:06.412089+0000","last_peered":"2023-11-06T20:44:06.412089+0000","last_clean":"2023-11-06T20:44:06.412089+0000","last_became_active":"2023-11-06T20:44:06.410838+0000","last_became_peered":"2023-11-06T20:44:06.410838+0000","last_unstale":"2023-11-06T20:44:06.412089+0000","last_undegraded":"2023-11-06T20:44:06.412089+0000","last_fullsized":"2023-11-06T20:44:06.412089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421847+0000","last_change":"2023-11-06T20:44:05.926188+0000","last_active":"2023-11-06T20:44:06.421847+0000","last_peered":"2023-11-06T20:44:06.421847+0000","last_clean":"2023-11-06T20:44:06.421847+0000","last_became_active":"2023-11-06T20:44:05.925868+0000","last_became_peered":"2023-11-06T20:44:05.925868+0000","last_unstale":"2023-11-06T20:44:06.421847+0000","last_undegraded":"2023-11-06T20:44:06.421847+0000","last_fullsized":"2023-11-06T20:44:06.421847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.401739+0000","last_change":"2023-11-06T20:44:06.397805+0000","last_active":"2023-11-06T20:44:06.401739+0000","last_peered":"2023-11-06T20:44:06.401739+0000","last_clean":"2023-11-06T20:44:06.401739+0000","last_became_active":"2023-11-06T20:44:06.397479+0000","last_became_peered":"2023-11-06T20:44:06.397479+0000","last_unstale":"2023-11-06T20:44:06.401739+0000","last_undegraded":"2023-11-06T20:44:06.401739+0000","last_fullsized":"2023-11-06T20:44:06.401739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389927+0000","last_change":"2023-11-06T20:44:05.925515+0000","last_active":"2023-11-06T20:44:06.389927+0000","last_peered":"2023-11-06T20:44:06.389927+0000","last_clean":"2023-11-06T20:44:06.389927+0000","last_became_active":"2023-11-06T20:44:05.921696+0000","last_became_peered":"2023-11-06T20:44:05.921696+0000","last_unstale":"2023-11-06T20:44:06.389927+0000","last_undegraded":"2023-11-06T20:44:06.389927+0000","last_fullsized":"2023-11-06T20:44:06.389927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425392+0000","last_change":"2023-11-06T20:44:06.421420+0000","last_active":"2023-11-06T20:44:06.425392+0000","last_peered":"2023-11-06T20:44:06.425392+0000","last_clean":"2023-11-06T20:44:06.425392+0000","last_became_active":"2023-11-06T20:44:06.421106+0000","last_became_peered":"2023-11-06T20:44:06.421106+0000","last_unstale":"2023-11-06T20:44:06.425392+0000","last_undegraded":"2023-11-06T20:44:06.425392+0000","last_fullsized":"2023-11-06T20:44:06.425392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404911+0000","last_change":"2023-11-06T20:44:05.921460+0000","last_active":"2023-11-06T20:44:06.404911+0000","last_peered":"2023-11-06T20:44:06.404911+0000","last_clean":"2023-11-06T20:44:06.404911+0000","last_became_active":"2023-11-06T20:44:05.921090+0000","last_became_peered":"2023-11-06T20:44:05.921090+0000","last_unstale":"2023-11-06T20:44:06.404911+0000","last_undegraded":"2023-11-06T20:44:06.404911+0000","last_fullsized":"2023-11-06T20:44:06.404911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421245+0000","last_change":"2023-11-06T20:44:06.415154+0000","last_active":"2023-11-06T20:44:06.421245+0000","last_peered":"2023-11-06T20:44:06.421245+0000","last_clean":"2023-11-06T20:44:06.421245+0000","last_became_active":"2023-11-06T20:44:06.414473+0000","last_became_peered":"2023-11-06T20:44:06.414473+0000","last_unstale":"2023-11-06T20:44:06.421245+0000","last_undegraded":"2023-11-06T20:44:06.421245+0000","last_fullsized":"2023-11-06T20:44:06.421245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420865+0000","last_change":"2023-11-06T20:44:05.922405+0000","last_active":"2023-11-06T20:44:06.420865+0000","last_peered":"2023-11-06T20:44:06.420865+0000","last_clean":"2023-11-06T20:44:06.420865+0000","last_became_active":"2023-11-06T20:44:05.921125+0000","last_became_peered":"2023-11-06T20:44:05.921125+0000","last_unstale":"2023-11-06T20:44:06.420865+0000","last_undegraded":"2023-11-06T20:44:06.420865+0000","last_fullsized":"2023-11-06T20:44:06.420865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407539+0000","last_change":"2023-11-06T20:44:06.399863+0000","last_active":"2023-11-06T20:44:06.407539+0000","last_peered":"2023-11-06T20:44:06.407539+0000","last_clean":"2023-11-06T20:44:06.407539+0000","last_became_active":"2023-11-06T20:44:06.399513+0000","last_became_peered":"2023-11-06T20:44:06.399513+0000","last_unstale":"2023-11-06T20:44:06.407539+0000","last_undegraded":"2023-11-06T20:44:06.407539+0000","last_fullsized":"2023-11-06T20:44:06.407539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388728+0000","last_change":"2023-11-06T20:44:05.936043+0000","last_active":"2023-11-06T20:44:06.388728+0000","last_peered":"2023-11-06T20:44:06.388728+0000","last_clean":"2023-11-06T20:44:06.388728+0000","last_became_active":"2023-11-06T20:44:05.935749+0000","last_became_peered":"2023-11-06T20:44:05.935749+0000","last_unstale":"2023-11-06T20:44:06.388728+0000","last_undegraded":"2023-11-06T20:44:06.388728+0000","last_fullsized":"2023-11-06T20:44:06.388728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412317+0000","last_change":"2023-11-06T20:44:06.405453+0000","last_active":"2023-11-06T20:44:06.412317+0000","last_peered":"2023-11-06T20:44:06.412317+0000","last_clean":"2023-11-06T20:44:06.412317+0000","last_became_active":"2023-11-06T20:44:06.405101+0000","last_became_peered":"2023-11-06T20:44:06.405101+0000","last_unstale":"2023-11-06T20:44:06.412317+0000","last_undegraded":"2023-11-06T20:44:06.412317+0000","last_fullsized":"2023-11-06T20:44:06.412317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389916+0000","last_change":"2023-11-06T20:44:05.929392+0000","last_active":"2023-11-06T20:44:06.389916+0000","last_peered":"2023-11-06T20:44:06.389916+0000","last_clean":"2023-11-06T20:44:06.389916+0000","last_became_active":"2023-11-06T20:44:05.928892+0000","last_became_peered":"2023-11-06T20:44:05.928892+0000","last_unstale":"2023-11-06T20:44:06.389916+0000","last_undegraded":"2023-11-06T20:44:06.389916+0000","last_fullsized":"2023-11-06T20:44:06.389916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420520+0000","last_change":"2023-11-06T20:44:06.415261+0000","last_active":"2023-11-06T20:44:06.420520+0000","last_peered":"2023-11-06T20:44:06.420520+0000","last_clean":"2023-11-06T20:44:06.420520+0000","last_became_active":"2023-11-06T20:44:06.410860+0000","last_became_peered":"2023-11-06T20:44:06.410860+0000","last_unstale":"2023-11-06T20:44:06.420520+0000","last_undegraded":"2023-11-06T20:44:06.420520+0000","last_fullsized":"2023-11-06T20:44:06.420520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.406449+0000","last_change":"2023-11-06T20:44:05.912367+0000","last_active":"2023-11-06T20:44:06.406449+0000","last_peered":"2023-11-06T20:44:06.406449+0000","last_clean":"2023-11-06T20:44:06.406449+0000","last_became_active":"2023-11-06T20:44:05.911944+0000","last_became_peered":"2023-11-06T20:44:05.911944+0000","last_unstale":"2023-11-06T20:44:06.406449+0000","last_undegraded":"2023-11-06T20:44:06.406449+0000","last_fullsized":"2023-11-06T20:44:06.406449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393110+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:44:06.393110+0000","last_peered":"2023-11-06T20:44:06.393110+0000","last_clean":"2023-11-06T20:44:06.393110+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:44:06.393110+0000","last_undegraded":"2023-11-06T20:44:06.393110+0000","last_fullsized":"2023-11-06T20:44:06.393110+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393545+0000","last_change":"2023-11-06T20:44:05.916204+0000","last_active":"2023-11-06T20:44:06.393545+0000","last_peered":"2023-11-06T20:44:06.393545+0000","last_clean":"2023-11-06T20:44:06.393545+0000","last_became_active":"2023-11-06T20:44:05.915592+0000","last_became_peered":"2023-11-06T20:44:05.915592+0000","last_unstale":"2023-11-06T20:44:06.393545+0000","last_undegraded":"2023-11-06T20:44:06.393545+0000","last_fullsized":"2023-11-06T20:44:06.393545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.422726+0000","last_change":"2023-11-06T20:44:06.418081+0000","last_active":"2023-11-06T20:44:06.422726+0000","last_peered":"2023-11-06T20:44:06.422726+0000","last_clean":"2023-11-06T20:44:06.422726+0000","last_became_active":"2023-11-06T20:44:06.416191+0000","last_became_peered":"2023-11-06T20:44:06.416191+0000","last_unstale":"2023-11-06T20:44:06.422726+0000","last_undegraded":"2023-11-06T20:44:06.422726+0000","last_fullsized":"2023-11-06T20:44:06.422726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388876+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:44:06.388876+0000","last_peered":"2023-11-06T20:44:06.388876+0000","last_clean":"2023-11-06T20:44:06.388876+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:44:06.388876+0000","last_undegraded":"2023-11-06T20:44:06.388876+0000","last_fullsized":"2023-11-06T20:44:06.388876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404240+0000","last_change":"2023-11-06T20:44:05.926886+0000","last_active":"2023-11-06T20:44:06.404240+0000","last_peered":"2023-11-06T20:44:06.404240+0000","last_clean":"2023-11-06T20:44:06.404240+0000","last_became_active":"2023-11-06T20:44:05.925506+0000","last_became_peered":"2023-11-06T20:44:05.925506+0000","last_unstale":"2023-11-06T20:44:06.404240+0000","last_undegraded":"2023-11-06T20:44:06.404240+0000","last_fullsized":"2023-11-06T20:44:06.404240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420444+0000","last_change":"2023-11-06T20:44:06.415342+0000","last_active":"2023-11-06T20:44:06.420444+0000","last_peered":"2023-11-06T20:44:06.420444+0000","last_clean":"2023-11-06T20:44:06.420444+0000","last_became_active":"2023-11-06T20:44:06.414787+0000","last_became_peered":"2023-11-06T20:44:06.414787+0000","last_unstale":"2023-11-06T20:44:06.420444+0000","last_undegraded":"2023-11-06T20:44:06.420444+0000","last_fullsized":"2023-11-06T20:44:06.420444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425033+0000","last_change":"2023-11-06T20:44:06.422594+0000","last_active":"2023-11-06T20:44:06.425033+0000","last_peered":"2023-11-06T20:44:06.425033+0000","last_clean":"2023-11-06T20:44:06.425033+0000","last_became_active":"2023-11-06T20:44:06.422264+0000","last_became_peered":"2023-11-06T20:44:06.422264+0000","last_unstale":"2023-11-06T20:44:06.425033+0000","last_undegraded":"2023-11-06T20:44:06.425033+0000","last_fullsized":"2023-11-06T20:44:06.425033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.400201+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:44:06.400201+0000","last_peered":"2023-11-06T20:44:06.400201+0000","last_clean":"2023-11-06T20:44:06.400201+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:44:06.400201+0000","last_undegraded":"2023-11-06T20:44:06.400201+0000","last_fullsized":"2023-11-06T20:44:06.400201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.392931+0000","last_change":"2023-11-06T20:44:05.912746+0000","last_active":"2023-11-06T20:44:06.392931+0000","last_peered":"2023-11-06T20:44:06.392931+0000","last_clean":"2023-11-06T20:44:06.392931+0000","last_became_active":"2023-11-06T20:44:05.912443+0000","last_became_peered":"2023-11-06T20:44:05.912443+0000","last_unstale":"2023-11-06T20:44:06.392931+0000","last_undegraded":"2023-11-06T20:44:06.392931+0000","last_fullsized":"2023-11-06T20:44:06.392931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412853+0000","last_change":"2023-11-06T20:44:06.400524+0000","last_active":"2023-11-06T20:44:06.412853+0000","last_peered":"2023-11-06T20:44:06.412853+0000","last_clean":"2023-11-06T20:44:06.412853+0000","last_became_active":"2023-11-06T20:44:06.399624+0000","last_became_peered":"2023-11-06T20:44:06.399624+0000","last_unstale":"2023-11-06T20:44:06.412853+0000","last_undegraded":"2023-11-06T20:44:06.412853+0000","last_fullsized":"2023-11-06T20:44:06.412853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388619+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:44:06.388619+0000","last_peered":"2023-11-06T20:44:06.388619+0000","last_clean":"2023-11-06T20:44:06.388619+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:44:06.388619+0000","last_undegraded":"2023-11-06T20:44:06.388619+0000","last_fullsized":"2023-11-06T20:44:06.388619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T20:44:05.912183+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:44:05.912183+0000","last_peered":"2023-11-06T20:44:05.912183+0000","last_clean":"2023-11-06T20:44:05.912183+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:44:05.912183+0000","last_undegraded":"2023-11-06T20:44:05.912183+0000","last_fullsized":"2023-11-06T20:44:05.912183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.401330+0000","last_change":"2023-11-06T20:44:05.915540+0000","last_active":"2023-11-06T20:44:06.401330+0000","last_peered":"2023-11-06T20:44:06.401330+0000","last_clean":"2023-11-06T20:44:06.401330+0000","last_became_active":"2023-11-06T20:44:05.915264+0000","last_became_peered":"2023-11-06T20:44:05.915264+0000","last_unstale":"2023-11-06T20:44:06.401330+0000","last_undegraded":"2023-11-06T20:44:06.401330+0000","last_fullsized":"2023-11-06T20:44:06.401330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.414058+0000","last_change":"2023-11-06T20:44:06.406952+0000","last_active":"2023-11-06T20:44:06.414058+0000","last_peered":"2023-11-06T20:44:06.414058+0000","last_clean":"2023-11-06T20:44:06.414058+0000","last_became_active":"2023-11-06T20:44:06.406478+0000","last_became_peered":"2023-11-06T20:44:06.406478+0000","last_unstale":"2023-11-06T20:44:06.414058+0000","last_undegraded":"2023-11-06T20:44:06.414058+0000","last_fullsized":"2023-11-06T20:44:06.414058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389490+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:44:06.389490+0000","last_peered":"2023-11-06T20:44:06.389490+0000","last_clean":"2023-11-06T20:44:06.389490+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:44:06.389490+0000","last_undegraded":"2023-11-06T20:44:06.389490+0000","last_fullsized":"2023-11-06T20:44:06.389490+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396293+0000","last_change":"2023-11-06T20:44:05.916288+0000","last_active":"2023-11-06T20:44:06.396293+0000","last_peered":"2023-11-06T20:44:06.396293+0000","last_clean":"2023-11-06T20:44:06.396293+0000","last_became_active":"2023-11-06T20:44:05.915873+0000","last_became_peered":"2023-11-06T20:44:05.915873+0000","last_unstale":"2023-11-06T20:44:06.396293+0000","last_undegraded":"2023-11-06T20:44:06.396293+0000","last_fullsized":"2023-11-06T20:44:06.396293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.390009+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:44:06.390009+0000","last_peered":"2023-11-06T20:44:06.390009+0000","last_clean":"2023-11-06T20:44:06.390009+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:44:06.390009+0000","last_undegraded":"2023-11-06T20:44:06.390009+0000","last_fullsized":"2023-11-06T20:44:06.390009+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399349+0000","last_change":"2023-11-06T20:44:05.914799+0000","last_active":"2023-11-06T20:44:06.399349+0000","last_peered":"2023-11-06T20:44:06.399349+0000","last_clean":"2023-11-06T20:44:06.399349+0000","last_became_active":"2023-11-06T20:44:05.914449+0000","last_became_peered":"2023-11-06T20:44:05.914449+0000","last_unstale":"2023-11-06T20:44:06.399349+0000","last_undegraded":"2023-11-06T20:44:06.399349+0000","last_fullsized":"2023-11-06T20:44:06.399349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412824+0000","last_change":"2023-11-06T20:44:06.408012+0000","last_active":"2023-11-06T20:44:06.412824+0000","last_peered":"2023-11-06T20:44:06.412824+0000","last_clean":"2023-11-06T20:44:06.412824+0000","last_became_active":"2023-11-06T20:44:06.407679+0000","last_became_peered":"2023-11-06T20:44:06.407679+0000","last_unstale":"2023-11-06T20:44:06.412824+0000","last_undegraded":"2023-11-06T20:44:06.412824+0000","last_fullsized":"2023-11-06T20:44:06.412824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421500+0000","last_change":"2023-11-06T20:44:05.919819+0000","last_active":"2023-11-06T20:44:06.421500+0000","last_peered":"2023-11-06T20:44:06.421500+0000","last_clean":"2023-11-06T20:44:06.421500+0000","last_became_active":"2023-11-06T20:44:05.919286+0000","last_became_peered":"2023-11-06T20:44:05.919286+0000","last_unstale":"2023-11-06T20:44:06.421500+0000","last_undegraded":"2023-11-06T20:44:06.421500+0000","last_fullsized":"2023-11-06T20:44:06.421500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.405378+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:44:06.405378+0000","last_peered":"2023-11-06T20:44:06.405378+0000","last_clean":"2023-11-06T20:44:06.405378+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:44:06.405378+0000","last_undegraded":"2023-11-06T20:44:06.405378+0000","last_fullsized":"2023-11-06T20:44:06.405378+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.423170+0000","last_change":"2023-11-06T20:44:06.420584+0000","last_active":"2023-11-06T20:44:06.423170+0000","last_peered":"2023-11-06T20:44:06.423170+0000","last_clean":"2023-11-06T20:44:06.423170+0000","last_became_active":"2023-11-06T20:44:06.420289+0000","last_became_peered":"2023-11-06T20:44:06.420289+0000","last_unstale":"2023-11-06T20:44:06.423170+0000","last_undegraded":"2023-11-06T20:44:06.423170+0000","last_fullsized":"2023-11-06T20:44:06.423170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389064+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:44:06.389064+0000","last_peered":"2023-11-06T20:44:06.389064+0000","last_clean":"2023-11-06T20:44:06.389064+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:44:06.389064+0000","last_undegraded":"2023-11-06T20:44:06.389064+0000","last_fullsized":"2023-11-06T20:44:06.389064+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.405496+0000","last_change":"2023-11-06T20:44:05.911916+0000","last_active":"2023-11-06T20:44:06.405496+0000","last_peered":"2023-11-06T20:44:06.405496+0000","last_clean":"2023-11-06T20:44:06.405496+0000","last_became_active":"2023-11-06T20:44:05.911373+0000","last_became_peered":"2023-11-06T20:44:05.911373+0000","last_unstale":"2023-11-06T20:44:06.405496+0000","last_undegraded":"2023-11-06T20:44:06.405496+0000","last_fullsized":"2023-11-06T20:44:06.405496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.402668+0000","last_change":"2023-11-06T20:44:06.399566+0000","last_active":"2023-11-06T20:44:06.402668+0000","last_peered":"2023-11-06T20:44:06.402668+0000","last_clean":"2023-11-06T20:44:06.402668+0000","last_became_active":"2023-11-06T20:44:06.399283+0000","last_became_peered":"2023-11-06T20:44:06.399283+0000","last_unstale":"2023-11-06T20:44:06.402668+0000","last_undegraded":"2023-11-06T20:44:06.402668+0000","last_fullsized":"2023-11-06T20:44:06.402668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420260+0000","last_change":"2023-11-06T20:44:06.413736+0000","last_active":"2023-11-06T20:44:06.420260+0000","last_peered":"2023-11-06T20:44:06.420260+0000","last_clean":"2023-11-06T20:44:06.420260+0000","last_became_active":"2023-11-06T20:44:06.413520+0000","last_became_peered":"2023-11-06T20:44:06.413520+0000","last_unstale":"2023-11-06T20:44:06.420260+0000","last_undegraded":"2023-11-06T20:44:06.420260+0000","last_fullsized":"2023-11-06T20:44:06.420260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.403806+0000","last_change":"2023-11-06T20:44:05.929692+0000","last_active":"2023-11-06T20:44:06.403806+0000","last_peered":"2023-11-06T20:44:06.403806+0000","last_clean":"2023-11-06T20:44:06.403806+0000","last_became_active":"2023-11-06T20:44:05.929364+0000","last_became_peered":"2023-11-06T20:44:05.929364+0000","last_unstale":"2023-11-06T20:44:06.403806+0000","last_undegraded":"2023-11-06T20:44:06.403806+0000","last_fullsized":"2023-11-06T20:44:06.403806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416558+0000","last_change":"2023-11-06T20:44:06.407233+0000","last_active":"2023-11-06T20:44:06.416558+0000","last_peered":"2023-11-06T20:44:06.416558+0000","last_clean":"2023-11-06T20:44:06.416558+0000","last_became_active":"2023-11-06T20:44:06.406927+0000","last_became_peered":"2023-11-06T20:44:06.406927+0000","last_unstale":"2023-11-06T20:44:06.416558+0000","last_undegraded":"2023-11-06T20:44:06.416558+0000","last_fullsized":"2023-11-06T20:44:06.416558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T20:44:05.930658+0000","last_change":"2023-11-06T20:44:05.925954+0000","last_active":"2023-11-06T20:44:05.930658+0000","last_peered":"2023-11-06T20:44:05.930658+0000","last_clean":"2023-11-06T20:44:05.930658+0000","last_became_active":"2023-11-06T20:44:05.925626+0000","last_became_peered":"2023-11-06T20:44:05.925626+0000","last_unstale":"2023-11-06T20:44:05.930658+0000","last_undegraded":"2023-11-06T20:44:05.930658+0000","last_fullsized":"2023-11-06T20:44:05.930658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420472+0000","last_change":"2023-11-06T20:44:06.417902+0000","last_active":"2023-11-06T20:44:06.420472+0000","last_peered":"2023-11-06T20:44:06.420472+0000","last_clean":"2023-11-06T20:44:06.420472+0000","last_became_active":"2023-11-06T20:44:06.417309+0000","last_became_peered":"2023-11-06T20:44:06.417309+0000","last_unstale":"2023-11-06T20:44:06.420472+0000","last_undegraded":"2023-11-06T20:44:06.420472+0000","last_fullsized":"2023-11-06T20:44:06.420472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427611+0000","last_change":"2023-11-06T20:44:05.935705+0000","last_active":"2023-11-06T20:44:06.427611+0000","last_peered":"2023-11-06T20:44:06.427611+0000","last_clean":"2023-11-06T20:44:06.427611+0000","last_became_active":"2023-11-06T20:44:05.935436+0000","last_became_peered":"2023-11-06T20:44:05.935436+0000","last_unstale":"2023-11-06T20:44:06.427611+0000","last_undegraded":"2023-11-06T20:44:06.427611+0000","last_fullsized":"2023-11-06T20:44:06.427611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419962+0000","last_change":"2023-11-06T20:44:06.414767+0000","last_active":"2023-11-06T20:44:06.419962+0000","last_peered":"2023-11-06T20:44:06.419962+0000","last_clean":"2023-11-06T20:44:06.419962+0000","last_became_active":"2023-11-06T20:44:06.414452+0000","last_became_peered":"2023-11-06T20:44:06.414452+0000","last_unstale":"2023-11-06T20:44:06.419962+0000","last_undegraded":"2023-11-06T20:44:06.419962+0000","last_fullsized":"2023-11-06T20:44:06.419962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.430730+0000","last_change":"2023-11-06T20:44:05.916430+0000","last_active":"2023-11-06T20:44:06.430730+0000","last_peered":"2023-11-06T20:44:06.430730+0000","last_clean":"2023-11-06T20:44:06.430730+0000","last_became_active":"2023-11-06T20:44:05.916163+0000","last_became_peered":"2023-11-06T20:44:05.916163+0000","last_unstale":"2023-11-06T20:44:06.430730+0000","last_undegraded":"2023-11-06T20:44:06.430730+0000","last_fullsized":"2023-11-06T20:44:06.430730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.415635+0000","last_change":"2023-11-06T20:44:06.412706+0000","last_active":"2023-11-06T20:44:06.415635+0000","last_peered":"2023-11-06T20:44:06.415635+0000","last_clean":"2023-11-06T20:44:06.415635+0000","last_became_active":"2023-11-06T20:44:06.412286+0000","last_became_peered":"2023-11-06T20:44:06.412286+0000","last_unstale":"2023-11-06T20:44:06.415635+0000","last_undegraded":"2023-11-06T20:44:06.415635+0000","last_fullsized":"2023-11-06T20:44:06.415635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.430724+0000","last_change":"2023-11-06T20:44:05.920163+0000","last_active":"2023-11-06T20:44:06.430724+0000","last_peered":"2023-11-06T20:44:06.430724+0000","last_clean":"2023-11-06T20:44:06.430724+0000","last_became_active":"2023-11-06T20:44:05.919817+0000","last_became_peered":"2023-11-06T20:44:05.919817+0000","last_unstale":"2023-11-06T20:44:06.430724+0000","last_undegraded":"2023-11-06T20:44:06.430724+0000","last_fullsized":"2023-11-06T20:44:06.430724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420853+0000","last_change":"2023-11-06T20:44:06.415900+0000","last_active":"2023-11-06T20:44:06.420853+0000","last_peered":"2023-11-06T20:44:06.420853+0000","last_clean":"2023-11-06T20:44:06.420853+0000","last_became_active":"2023-11-06T20:44:06.415623+0000","last_became_peered":"2023-11-06T20:44:06.415623+0000","last_unstale":"2023-11-06T20:44:06.420853+0000","last_undegraded":"2023-11-06T20:44:06.420853+0000","last_fullsized":"2023-11-06T20:44:06.420853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.394177+0000","last_change":"2023-11-06T20:44:05.916945+0000","last_active":"2023-11-06T20:44:06.394177+0000","last_peered":"2023-11-06T20:44:06.394177+0000","last_clean":"2023-11-06T20:44:06.394177+0000","last_became_active":"2023-11-06T20:44:05.916459+0000","last_became_peered":"2023-11-06T20:44:05.916459+0000","last_unstale":"2023-11-06T20:44:06.394177+0000","last_undegraded":"2023-11-06T20:44:06.394177+0000","last_fullsized":"2023-11-06T20:44:06.394177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420482+0000","last_change":"2023-11-06T20:44:06.414181+0000","last_active":"2023-11-06T20:44:06.420482+0000","last_peered":"2023-11-06T20:44:06.420482+0000","last_clean":"2023-11-06T20:44:06.420482+0000","last_became_active":"2023-11-06T20:44:06.411516+0000","last_became_peered":"2023-11-06T20:44:06.411516+0000","last_unstale":"2023-11-06T20:44:06.420482+0000","last_undegraded":"2023-11-06T20:44:06.420482+0000","last_fullsized":"2023-11-06T20:44:06.420482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.400331+0000","last_change":"2023-11-06T20:44:05.923629+0000","last_active":"2023-11-06T20:44:06.400331+0000","last_peered":"2023-11-06T20:44:06.400331+0000","last_clean":"2023-11-06T20:44:06.400331+0000","last_became_active":"2023-11-06T20:44:05.923258+0000","last_became_peered":"2023-11-06T20:44:05.923258+0000","last_unstale":"2023-11-06T20:44:06.400331+0000","last_undegraded":"2023-11-06T20:44:06.400331+0000","last_fullsized":"2023-11-06T20:44:06.400331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427759+0000","last_change":"2023-11-06T20:44:06.421947+0000","last_active":"2023-11-06T20:44:06.427759+0000","last_peered":"2023-11-06T20:44:06.427759+0000","last_clean":"2023-11-06T20:44:06.427759+0000","last_became_active":"2023-11-06T20:44:06.421497+0000","last_became_peered":"2023-11-06T20:44:06.421497+0000","last_unstale":"2023-11-06T20:44:06.427759+0000","last_undegraded":"2023-11-06T20:44:06.427759+0000","last_fullsized":"2023-11-06T20:44:06.427759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388963+0000","last_change":"2023-11-06T20:44:05.926582+0000","last_active":"2023-11-06T20:44:06.388963+0000","last_peered":"2023-11-06T20:44:06.388963+0000","last_clean":"2023-11-06T20:44:06.388963+0000","last_became_active":"2023-11-06T20:44:05.926291+0000","last_became_peered":"2023-11-06T20:44:05.926291+0000","last_unstale":"2023-11-06T20:44:06.388963+0000","last_undegraded":"2023-11-06T20:44:06.388963+0000","last_fullsized":"2023-11-06T20:44:06.388963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.408755+0000","last_change":"2023-11-06T20:44:06.401465+0000","last_active":"2023-11-06T20:44:06.408755+0000","last_peered":"2023-11-06T20:44:06.408755+0000","last_clean":"2023-11-06T20:44:06.408755+0000","last_became_active":"2023-11-06T20:44:06.401100+0000","last_became_peered":"2023-11-06T20:44:06.401100+0000","last_unstale":"2023-11-06T20:44:06.408755+0000","last_undegraded":"2023-11-06T20:44:06.408755+0000","last_fullsized":"2023-11-06T20:44:06.408755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.392747+0000","last_change":"2023-11-06T20:44:05.910900+0000","last_active":"2023-11-06T20:44:06.392747+0000","last_peered":"2023-11-06T20:44:06.392747+0000","last_clean":"2023-11-06T20:44:06.392747+0000","last_became_active":"2023-11-06T20:44:05.910285+0000","last_became_peered":"2023-11-06T20:44:05.910285+0000","last_unstale":"2023-11-06T20:44:06.392747+0000","last_undegraded":"2023-11-06T20:44:06.392747+0000","last_fullsized":"2023-11-06T20:44:06.392747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.410850+0000","last_change":"2023-11-06T20:44:06.403555+0000","last_active":"2023-11-06T20:44:06.410850+0000","last_peered":"2023-11-06T20:44:06.410850+0000","last_clean":"2023-11-06T20:44:06.410850+0000","last_became_active":"2023-11-06T20:44:06.403246+0000","last_became_peered":"2023-11-06T20:44:06.403246+0000","last_unstale":"2023-11-06T20:44:06.410850+0000","last_undegraded":"2023-11-06T20:44:06.410850+0000","last_fullsized":"2023-11-06T20:44:06.410850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399175+0000","last_change":"2023-11-06T20:44:05.929537+0000","last_active":"2023-11-06T20:44:06.399175+0000","last_peered":"2023-11-06T20:44:06.399175+0000","last_clean":"2023-11-06T20:44:06.399175+0000","last_became_active":"2023-11-06T20:44:05.929231+0000","last_became_peered":"2023-11-06T20:44:05.929231+0000","last_unstale":"2023-11-06T20:44:06.399175+0000","last_undegraded":"2023-11-06T20:44:06.399175+0000","last_fullsized":"2023-11-06T20:44:06.399175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.429096+0000","last_change":"2023-11-06T20:44:06.425230+0000","last_active":"2023-11-06T20:44:06.429096+0000","last_peered":"2023-11-06T20:44:06.429096+0000","last_clean":"2023-11-06T20:44:06.429096+0000","last_became_active":"2023-11-06T20:44:06.424973+0000","last_became_peered":"2023-11-06T20:44:06.424973+0000","last_unstale":"2023-11-06T20:44:06.429096+0000","last_undegraded":"2023-11-06T20:44:06.429096+0000","last_fullsized":"2023-11-06T20:44:06.429096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399278+0000","last_change":"2023-11-06T20:44:05.918773+0000","last_active":"2023-11-06T20:44:06.399278+0000","last_peered":"2023-11-06T20:44:06.399278+0000","last_clean":"2023-11-06T20:44:06.399278+0000","last_became_active":"2023-11-06T20:44:05.916793+0000","last_became_peered":"2023-11-06T20:44:05.916793+0000","last_unstale":"2023-11-06T20:44:06.399278+0000","last_undegraded":"2023-11-06T20:44:06.399278+0000","last_fullsized":"2023-11-06T20:44:06.399278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.423633+0000","last_change":"2023-11-06T20:44:06.419145+0000","last_active":"2023-11-06T20:44:06.423633+0000","last_peered":"2023-11-06T20:44:06.423633+0000","last_clean":"2023-11-06T20:44:06.423633+0000","last_became_active":"2023-11-06T20:44:06.417610+0000","last_became_peered":"2023-11-06T20:44:06.417610+0000","last_unstale":"2023-11-06T20:44:06.423633+0000","last_undegraded":"2023-11-06T20:44:06.423633+0000","last_fullsized":"2023-11-06T20:44:06.423633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395535+0000","last_change":"2023-11-06T20:44:05.922285+0000","last_active":"2023-11-06T20:44:06.395535+0000","last_peered":"2023-11-06T20:44:06.395535+0000","last_clean":"2023-11-06T20:44:06.395535+0000","last_became_active":"2023-11-06T20:44:05.921952+0000","last_became_peered":"2023-11-06T20:44:05.921952+0000","last_unstale":"2023-11-06T20:44:06.395535+0000","last_undegraded":"2023-11-06T20:44:06.395535+0000","last_fullsized":"2023-11-06T20:44:06.395535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.411774+0000","last_change":"2023-11-06T20:44:06.411774+0000","last_active":"2023-11-06T20:44:06.411774+0000","last_peered":"2023-11-06T20:44:06.411774+0000","last_clean":"2023-11-06T20:44:06.411774+0000","last_became_active":"2023-11-06T20:44:06.411426+0000","last_became_peered":"2023-11-06T20:44:06.411426+0000","last_unstale":"2023-11-06T20:44:06.411774+0000","last_undegraded":"2023-11-06T20:44:06.411774+0000","last_fullsized":"2023-11-06T20:44:06.411774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395460+0000","last_change":"2023-11-06T20:44:05.922681+0000","last_active":"2023-11-06T20:44:06.395460+0000","last_peered":"2023-11-06T20:44:06.395460+0000","last_clean":"2023-11-06T20:44:06.395460+0000","last_became_active":"2023-11-06T20:44:05.920900+0000","last_became_peered":"2023-11-06T20:44:05.920900+0000","last_unstale":"2023-11-06T20:44:06.395460+0000","last_undegraded":"2023-11-06T20:44:06.395460+0000","last_fullsized":"2023-11-06T20:44:06.395460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420536+0000","last_change":"2023-11-06T20:44:06.416541+0000","last_active":"2023-11-06T20:44:06.420536+0000","last_peered":"2023-11-06T20:44:06.420536+0000","last_clean":"2023-11-06T20:44:06.420536+0000","last_became_active":"2023-11-06T20:44:06.416192+0000","last_became_peered":"2023-11-06T20:44:06.416192+0000","last_unstale":"2023-11-06T20:44:06.420536+0000","last_undegraded":"2023-11-06T20:44:06.420536+0000","last_fullsized":"2023-11-06T20:44:06.420536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389587+0000","last_change":"2023-11-06T20:44:05.935474+0000","last_active":"2023-11-06T20:44:06.389587+0000","last_peered":"2023-11-06T20:44:06.389587+0000","last_clean":"2023-11-06T20:44:06.389587+0000","last_became_active":"2023-11-06T20:44:05.930110+0000","last_became_peered":"2023-11-06T20:44:05.930110+0000","last_unstale":"2023-11-06T20:44:06.389587+0000","last_undegraded":"2023-11-06T20:44:06.389587+0000","last_fullsized":"2023-11-06T20:44:06.389587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421313+0000","last_change":"2023-11-06T20:44:06.416913+0000","last_active":"2023-11-06T20:44:06.421313+0000","last_peered":"2023-11-06T20:44:06.421313+0000","last_clean":"2023-11-06T20:44:06.421313+0000","last_became_active":"2023-11-06T20:44:06.416623+0000","last_became_peered":"2023-11-06T20:44:06.416623+0000","last_unstale":"2023-11-06T20:44:06.421313+0000","last_undegraded":"2023-11-06T20:44:06.421313+0000","last_fullsized":"2023-11-06T20:44:06.421313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.394473+0000","last_change":"2023-11-06T20:44:05.917793+0000","last_active":"2023-11-06T20:44:06.394473+0000","last_peered":"2023-11-06T20:44:06.394473+0000","last_clean":"2023-11-06T20:44:06.394473+0000","last_became_active":"2023-11-06T20:44:05.917376+0000","last_became_peered":"2023-11-06T20:44:05.917376+0000","last_unstale":"2023-11-06T20:44:06.394473+0000","last_undegraded":"2023-11-06T20:44:06.394473+0000","last_fullsized":"2023-11-06T20:44:06.394473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.409294+0000","last_change":"2023-11-06T20:44:06.400632+0000","last_active":"2023-11-06T20:44:06.409294+0000","last_peered":"2023-11-06T20:44:06.409294+0000","last_clean":"2023-11-06T20:44:06.409294+0000","last_became_active":"2023-11-06T20:44:06.397240+0000","last_became_peered":"2023-11-06T20:44:06.397240+0000","last_unstale":"2023-11-06T20:44:06.409294+0000","last_undegraded":"2023-11-06T20:44:06.409294+0000","last_fullsized":"2023-11-06T20:44:06.409294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T20:44:05.930903+0000","last_change":"2023-11-06T20:44:05.928517+0000","last_active":"2023-11-06T20:44:05.930903+0000","last_peered":"2023-11-06T20:44:05.930903+0000","last_clean":"2023-11-06T20:44:05.930903+0000","last_became_active":"2023-11-06T20:44:05.928197+0000","last_became_peered":"2023-11-06T20:44:05.928197+0000","last_unstale":"2023-11-06T20:44:05.930903+0000","last_undegraded":"2023-11-06T20:44:05.930903+0000","last_fullsized":"2023-11-06T20:44:05.930903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.414429+0000","last_change":"2023-11-06T20:44:06.414429+0000","last_active":"2023-11-06T20:44:06.414429+0000","last_peered":"2023-11-06T20:44:06.414429+0000","last_clean":"2023-11-06T20:44:06.414429+0000","last_became_active":"2023-11-06T20:44:06.411089+0000","last_became_peered":"2023-11-06T20:44:06.411089+0000","last_unstale":"2023-11-06T20:44:06.414429+0000","last_undegraded":"2023-11-06T20:44:06.414429+0000","last_fullsized":"2023-11-06T20:44:06.414429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.417271+0000","last_change":"2023-11-06T20:44:05.917588+0000","last_active":"2023-11-06T20:44:06.417271+0000","last_peered":"2023-11-06T20:44:06.417271+0000","last_clean":"2023-11-06T20:44:06.417271+0000","last_became_active":"2023-11-06T20:44:05.917276+0000","last_became_peered":"2023-11-06T20:44:05.917276+0000","last_unstale":"2023-11-06T20:44:06.417271+0000","last_undegraded":"2023-11-06T20:44:06.417271+0000","last_fullsized":"2023-11-06T20:44:06.417271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.409877+0000","last_change":"2023-11-06T20:44:06.409877+0000","last_active":"2023-11-06T20:44:06.409877+0000","last_peered":"2023-11-06T20:44:06.409877+0000","last_clean":"2023-11-06T20:44:06.409877+0000","last_became_active":"2023-11-06T20:44:06.408631+0000","last_became_peered":"2023-11-06T20:44:06.408631+0000","last_unstale":"2023-11-06T20:44:06.409877+0000","last_undegraded":"2023-11-06T20:44:06.409877+0000","last_fullsized":"2023-11-06T20:44:06.409877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404793+0000","last_change":"2023-11-06T20:44:05.932767+0000","last_active":"2023-11-06T20:44:06.404793+0000","last_peered":"2023-11-06T20:44:06.404793+0000","last_clean":"2023-11-06T20:44:06.404793+0000","last_became_active":"2023-11-06T20:44:05.932095+0000","last_became_peered":"2023-11-06T20:44:05.932095+0000","last_unstale":"2023-11-06T20:44:06.404793+0000","last_undegraded":"2023-11-06T20:44:06.404793+0000","last_fullsized":"2023-11-06T20:44:06.404793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412317+0000","last_change":"2023-11-06T20:44:06.410542+0000","last_active":"2023-11-06T20:44:06.412317+0000","last_peered":"2023-11-06T20:44:06.412317+0000","last_clean":"2023-11-06T20:44:06.412317+0000","last_became_active":"2023-11-06T20:44:06.410258+0000","last_became_peered":"2023-11-06T20:44:06.410258+0000","last_unstale":"2023-11-06T20:44:06.412317+0000","last_undegraded":"2023-11-06T20:44:06.412317+0000","last_fullsized":"2023-11-06T20:44:06.412317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427701+0000","last_change":"2023-11-06T20:44:05.927848+0000","last_active":"2023-11-06T20:44:06.427701+0000","last_peered":"2023-11-06T20:44:06.427701+0000","last_clean":"2023-11-06T20:44:06.427701+0000","last_became_active":"2023-11-06T20:44:05.927576+0000","last_became_peered":"2023-11-06T20:44:05.927576+0000","last_unstale":"2023-11-06T20:44:06.427701+0000","last_undegraded":"2023-11-06T20:44:06.427701+0000","last_fullsized":"2023-11-06T20:44:06.427701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416668+0000","last_change":"2023-11-06T20:44:06.409302+0000","last_active":"2023-11-06T20:44:06.416668+0000","last_peered":"2023-11-06T20:44:06.416668+0000","last_clean":"2023-11-06T20:44:06.416668+0000","last_became_active":"2023-11-06T20:44:06.409049+0000","last_became_peered":"2023-11-06T20:44:06.409049+0000","last_unstale":"2023-11-06T20:44:06.416668+0000","last_undegraded":"2023-11-06T20:44:06.416668+0000","last_fullsized":"2023-11-06T20:44:06.416668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.398113+0000","last_change":"2023-11-06T20:44:05.915170+0000","last_active":"2023-11-06T20:44:06.398113+0000","last_peered":"2023-11-06T20:44:06.398113+0000","last_clean":"2023-11-06T20:44:06.398113+0000","last_became_active":"2023-11-06T20:44:05.914850+0000","last_became_peered":"2023-11-06T20:44:05.914850+0000","last_unstale":"2023-11-06T20:44:06.398113+0000","last_undegraded":"2023-11-06T20:44:06.398113+0000","last_fullsized":"2023-11-06T20:44:06.398113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407493+0000","last_change":"2023-11-06T20:44:06.407493+0000","last_active":"2023-11-06T20:44:06.407493+0000","last_peered":"2023-11-06T20:44:06.407493+0000","last_clean":"2023-11-06T20:44:06.407493+0000","last_became_active":"2023-11-06T20:44:06.407167+0000","last_became_peered":"2023-11-06T20:44:06.407167+0000","last_unstale":"2023-11-06T20:44:06.407493+0000","last_undegraded":"2023-11-06T20:44:06.407493+0000","last_fullsized":"2023-11-06T20:44:06.407493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389137+0000","last_change":"2023-11-06T20:44:05.934566+0000","last_active":"2023-11-06T20:44:06.389137+0000","last_peered":"2023-11-06T20:44:06.389137+0000","last_clean":"2023-11-06T20:44:06.389137+0000","last_became_active":"2023-11-06T20:44:05.934165+0000","last_became_peered":"2023-11-06T20:44:05.934165+0000","last_unstale":"2023-11-06T20:44:06.389137+0000","last_undegraded":"2023-11-06T20:44:06.389137+0000","last_fullsized":"2023-11-06T20:44:06.389137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.423114+0000","last_change":"2023-11-06T20:44:06.417767+0000","last_active":"2023-11-06T20:44:06.423114+0000","last_peered":"2023-11-06T20:44:06.423114+0000","last_clean":"2023-11-06T20:44:06.423114+0000","last_became_active":"2023-11-06T20:44:06.417468+0000","last_became_peered":"2023-11-06T20:44:06.417468+0000","last_unstale":"2023-11-06T20:44:06.423114+0000","last_undegraded":"2023-11-06T20:44:06.423114+0000","last_fullsized":"2023-11-06T20:44:06.423114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.30","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427626+0000","last_change":"2023-11-06T20:44:05.930422+0000","last_active":"2023-11-06T20:44:06.427626+0000","last_peered":"2023-11-06T20:44:06.427626+0000","last_clean":"2023-11-06T20:44:06.427626+0000","last_became_active":"2023-11-06T20:44:05.930070+0000","last_became_peered":"2023-11-06T20:44:05.930070+0000","last_unstale":"2023-11-06T20:44:06.427626+0000","last_undegraded":"2023-11-06T20:44:06.427626+0000","last_fullsized":"2023-11-06T20:44:06.427626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407092+0000","last_change":"2023-11-06T20:44:06.407092+0000","last_active":"2023-11-06T20:44:06.407092+0000","last_peered":"2023-11-06T20:44:06.407092+0000","last_clean":"2023-11-06T20:44:06.407092+0000","last_became_active":"2023-11-06T20:44:06.406767+0000","last_became_peered":"2023-11-06T20:44:06.406767+0000","last_unstale":"2023-11-06T20:44:06.407092+0000","last_undegraded":"2023-11-06T20:44:06.407092+0000","last_fullsized":"2023-11-06T20:44:06.407092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393919+0000","last_change":"2023-11-06T20:44:05.923803+0000","last_active":"2023-11-06T20:44:06.393919+0000","last_peered":"2023-11-06T20:44:06.393919+0000","last_clean":"2023-11-06T20:44:06.393919+0000","last_became_active":"2023-11-06T20:44:05.923361+0000","last_became_peered":"2023-11-06T20:44:05.923361+0000","last_unstale":"2023-11-06T20:44:06.393919+0000","last_undegraded":"2023-11-06T20:44:06.393919+0000","last_fullsized":"2023-11-06T20:44:06.393919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419551+0000","last_change":"2023-11-06T20:44:06.419551+0000","last_active":"2023-11-06T20:44:06.419551+0000","last_peered":"2023-11-06T20:44:06.419551+0000","last_clean":"2023-11-06T20:44:06.419551+0000","last_became_active":"2023-11-06T20:44:06.419244+0000","last_became_peered":"2023-11-06T20:44:06.419244+0000","last_unstale":"2023-11-06T20:44:06.419551+0000","last_undegraded":"2023-11-06T20:44:06.419551+0000","last_fullsized":"2023-11-06T20:44:06.419551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.398750+0000","last_change":"2023-11-06T20:44:05.915503+0000","last_active":"2023-11-06T20:44:06.398750+0000","last_peered":"2023-11-06T20:44:06.398750+0000","last_clean":"2023-11-06T20:44:06.398750+0000","last_became_active":"2023-11-06T20:44:05.914672+0000","last_became_peered":"2023-11-06T20:44:05.914672+0000","last_unstale":"2023-11-06T20:44:06.398750+0000","last_undegraded":"2023-11-06T20:44:06.398750+0000","last_fullsized":"2023-11-06T20:44:06.398750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424374+0000","last_change":"2023-11-06T20:44:06.418248+0000","last_active":"2023-11-06T20:44:06.424374+0000","last_peered":"2023-11-06T20:44:06.424374+0000","last_clean":"2023-11-06T20:44:06.424374+0000","last_became_active":"2023-11-06T20:44:06.408712+0000","last_became_peered":"2023-11-06T20:44:06.408712+0000","last_unstale":"2023-11-06T20:44:06.424374+0000","last_undegraded":"2023-11-06T20:44:06.424374+0000","last_fullsized":"2023-11-06T20:44:06.424374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.33","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425162+0000","last_change":"2023-11-06T20:44:05.927963+0000","last_active":"2023-11-06T20:44:06.425162+0000","last_peered":"2023-11-06T20:44:06.425162+0000","last_clean":"2023-11-06T20:44:06.425162+0000","last_became_active":"2023-11-06T20:44:05.927646+0000","last_became_peered":"2023-11-06T20:44:05.927646+0000","last_unstale":"2023-11-06T20:44:06.425162+0000","last_undegraded":"2023-11-06T20:44:06.425162+0000","last_fullsized":"2023-11-06T20:44:06.425162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420270+0000","last_change":"2023-11-06T20:44:06.413775+0000","last_active":"2023-11-06T20:44:06.420270+0000","last_peered":"2023-11-06T20:44:06.420270+0000","last_clean":"2023-11-06T20:44:06.420270+0000","last_became_active":"2023-11-06T20:44:06.413505+0000","last_became_peered":"2023-11-06T20:44:06.413505+0000","last_unstale":"2023-11-06T20:44:06.420270+0000","last_undegraded":"2023-11-06T20:44:06.420270+0000","last_fullsized":"2023-11-06T20:44:06.420270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.429184+0000","last_change":"2023-11-06T20:44:05.931664+0000","last_active":"2023-11-06T20:44:06.429184+0000","last_peered":"2023-11-06T20:44:06.429184+0000","last_clean":"2023-11-06T20:44:06.429184+0000","last_became_active":"2023-11-06T20:44:05.930383+0000","last_became_peered":"2023-11-06T20:44:05.930383+0000","last_unstale":"2023-11-06T20:44:06.429184+0000","last_undegraded":"2023-11-06T20:44:06.429184+0000","last_fullsized":"2023-11-06T20:44:06.429184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399159+0000","last_change":"2023-11-06T20:44:06.399159+0000","last_active":"2023-11-06T20:44:06.399159+0000","last_peered":"2023-11-06T20:44:06.399159+0000","last_clean":"2023-11-06T20:44:06.399159+0000","last_became_active":"2023-11-06T20:44:06.398305+0000","last_became_peered":"2023-11-06T20:44:06.398305+0000","last_unstale":"2023-11-06T20:44:06.399159+0000","last_undegraded":"2023-11-06T20:44:06.399159+0000","last_fullsized":"2023-11-06T20:44:06.399159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404918+0000","last_change":"2023-11-06T20:44:05.934808+0000","last_active":"2023-11-06T20:44:06.404918+0000","last_peered":"2023-11-06T20:44:06.404918+0000","last_clean":"2023-11-06T20:44:06.404918+0000","last_became_active":"2023-11-06T20:44:05.934503+0000","last_became_peered":"2023-11-06T20:44:05.934503+0000","last_unstale":"2023-11-06T20:44:06.404918+0000","last_undegraded":"2023-11-06T20:44:06.404918+0000","last_fullsized":"2023-11-06T20:44:06.404918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421422+0000","last_change":"2023-11-06T20:44:06.413884+0000","last_active":"2023-11-06T20:44:06.421422+0000","last_peered":"2023-11-06T20:44:06.421422+0000","last_clean":"2023-11-06T20:44:06.421422+0000","last_became_active":"2023-11-06T20:44:06.413013+0000","last_became_peered":"2023-11-06T20:44:06.413013+0000","last_unstale":"2023-11-06T20:44:06.421422+0000","last_undegraded":"2023-11-06T20:44:06.421422+0000","last_fullsized":"2023-11-06T20:44:06.421422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407560+0000","last_change":"2023-11-06T20:44:05.920301+0000","last_active":"2023-11-06T20:44:06.407560+0000","last_peered":"2023-11-06T20:44:06.407560+0000","last_clean":"2023-11-06T20:44:06.407560+0000","last_became_active":"2023-11-06T20:44:05.917401+0000","last_became_peered":"2023-11-06T20:44:05.917401+0000","last_unstale":"2023-11-06T20:44:06.407560+0000","last_undegraded":"2023-11-06T20:44:06.407560+0000","last_fullsized":"2023-11-06T20:44:06.407560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424639+0000","last_change":"2023-11-06T20:44:06.406242+0000","last_active":"2023-11-06T20:44:06.424639+0000","last_peered":"2023-11-06T20:44:06.424639+0000","last_clean":"2023-11-06T20:44:06.424639+0000","last_became_active":"2023-11-06T20:44:06.405979+0000","last_became_peered":"2023-11-06T20:44:06.405979+0000","last_unstale":"2023-11-06T20:44:06.424639+0000","last_undegraded":"2023-11-06T20:44:06.424639+0000","last_fullsized":"2023-11-06T20:44:06.424639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T20:44:05.940218+0000","last_change":"2023-11-06T20:44:05.939218+0000","last_active":"2023-11-06T20:44:05.940218+0000","last_peered":"2023-11-06T20:44:05.940218+0000","last_clean":"2023-11-06T20:44:05.940218+0000","last_became_active":"2023-11-06T20:44:05.938856+0000","last_became_peered":"2023-11-06T20:44:05.938856+0000","last_unstale":"2023-11-06T20:44:05.940218+0000","last_undegraded":"2023-11-06T20:44:05.940218+0000","last_fullsized":"2023-11-06T20:44:05.940218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424019+0000","last_change":"2023-11-06T20:44:06.416017+0000","last_active":"2023-11-06T20:44:06.424019+0000","last_peered":"2023-11-06T20:44:06.424019+0000","last_clean":"2023-11-06T20:44:06.424019+0000","last_became_active":"2023-11-06T20:44:06.414205+0000","last_became_peered":"2023-11-06T20:44:06.414205+0000","last_unstale":"2023-11-06T20:44:06.424019+0000","last_undegraded":"2023-11-06T20:44:06.424019+0000","last_fullsized":"2023-11-06T20:44:06.424019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393064+0000","last_change":"2023-11-06T20:44:05.928182+0000","last_active":"2023-11-06T20:44:06.393064+0000","last_peered":"2023-11-06T20:44:06.393064+0000","last_clean":"2023-11-06T20:44:06.393064+0000","last_became_active":"2023-11-06T20:44:05.927847+0000","last_became_peered":"2023-11-06T20:44:05.927847+0000","last_unstale":"2023-11-06T20:44:06.393064+0000","last_undegraded":"2023-11-06T20:44:06.393064+0000","last_fullsized":"2023-11-06T20:44:06.393064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416917+0000","last_change":"2023-11-06T20:44:06.412444+0000","last_active":"2023-11-06T20:44:06.416917+0000","last_peered":"2023-11-06T20:44:06.416917+0000","last_clean":"2023-11-06T20:44:06.416917+0000","last_became_active":"2023-11-06T20:44:06.412113+0000","last_became_peered":"2023-11-06T20:44:06.412113+0000","last_unstale":"2023-11-06T20:44:06.416917+0000","last_undegraded":"2023-11-06T20:44:06.416917+0000","last_fullsized":"2023-11-06T20:44:06.416917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.403846+0000","last_change":"2023-11-06T20:44:05.929538+0000","last_active":"2023-11-06T20:44:06.403846+0000","last_peered":"2023-11-06T20:44:06.403846+0000","last_clean":"2023-11-06T20:44:06.403846+0000","last_became_active":"2023-11-06T20:44:05.922763+0000","last_became_peered":"2023-11-06T20:44:05.922763+0000","last_unstale":"2023-11-06T20:44:06.403846+0000","last_undegraded":"2023-11-06T20:44:06.403846+0000","last_fullsized":"2023-11-06T20:44:06.403846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426978+0000","last_change":"2023-11-06T20:44:06.423931+0000","last_active":"2023-11-06T20:44:06.426978+0000","last_peered":"2023-11-06T20:44:06.426978+0000","last_clean":"2023-11-06T20:44:06.426978+0000","last_became_active":"2023-11-06T20:44:06.423644+0000","last_became_peered":"2023-11-06T20:44:06.423644+0000","last_unstale":"2023-11-06T20:44:06.426978+0000","last_undegraded":"2023-11-06T20:44:06.426978+0000","last_fullsized":"2023-11-06T20:44:06.426978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420060+0000","last_change":"2023-11-06T20:44:05.909687+0000","last_active":"2023-11-06T20:44:06.420060+0000","last_peered":"2023-11-06T20:44:06.420060+0000","last_clean":"2023-11-06T20:44:06.420060+0000","last_became_active":"2023-11-06T20:44:05.907015+0000","last_became_peered":"2023-11-06T20:44:05.907015+0000","last_unstale":"2023-11-06T20:44:06.420060+0000","last_undegraded":"2023-11-06T20:44:06.420060+0000","last_fullsized":"2023-11-06T20:44:06.420060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421264+0000","last_change":"2023-11-06T20:44:06.415345+0000","last_active":"2023-11-06T20:44:06.421264+0000","last_peered":"2023-11-06T20:44:06.421264+0000","last_clean":"2023-11-06T20:44:06.421264+0000","last_became_active":"2023-11-06T20:44:06.414850+0000","last_became_peered":"2023-11-06T20:44:06.414850+0000","last_unstale":"2023-11-06T20:44:06.421264+0000","last_undegraded":"2023-11-06T20:44:06.421264+0000","last_fullsized":"2023-11-06T20:44:06.421264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421636+0000","last_change":"2023-11-06T20:44:05.921703+0000","last_active":"2023-11-06T20:44:06.421636+0000","last_peered":"2023-11-06T20:44:06.421636+0000","last_clean":"2023-11-06T20:44:06.421636+0000","last_became_active":"2023-11-06T20:44:05.920678+0000","last_became_peered":"2023-11-06T20:44:05.920678+0000","last_unstale":"2023-11-06T20:44:06.421636+0000","last_undegraded":"2023-11-06T20:44:06.421636+0000","last_fullsized":"2023-11-06T20:44:06.421636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419871+0000","last_change":"2023-11-06T20:44:06.412730+0000","last_active":"2023-11-06T20:44:06.419871+0000","last_peered":"2023-11-06T20:44:06.419871+0000","last_clean":"2023-11-06T20:44:06.419871+0000","last_became_active":"2023-11-06T20:44:06.411623+0000","last_became_peered":"2023-11-06T20:44:06.411623+0000","last_unstale":"2023-11-06T20:44:06.419871+0000","last_undegraded":"2023-11-06T20:44:06.419871+0000","last_fullsized":"2023-11-06T20:44:06.419871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420499+0000","last_change":"2023-11-06T20:44:05.919149+0000","last_active":"2023-11-06T20:44:06.420499+0000","last_peered":"2023-11-06T20:44:06.420499+0000","last_clean":"2023-11-06T20:44:06.420499+0000","last_became_active":"2023-11-06T20:44:05.918833+0000","last_became_peered":"2023-11-06T20:44:05.918833+0000","last_unstale":"2023-11-06T20:44:06.420499+0000","last_undegraded":"2023-11-06T20:44:06.420499+0000","last_fullsized":"2023-11-06T20:44:06.420499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407334+0000","last_change":"2023-11-06T20:44:06.399010+0000","last_active":"2023-11-06T20:44:06.407334+0000","last_peered":"2023-11-06T20:44:06.407334+0000","last_clean":"2023-11-06T20:44:06.407334+0000","last_became_active":"2023-11-06T20:44:06.398749+0000","last_became_peered":"2023-11-06T20:44:06.398749+0000","last_unstale":"2023-11-06T20:44:06.407334+0000","last_undegraded":"2023-11-06T20:44:06.407334+0000","last_fullsized":"2023-11-06T20:44:06.407334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.394817+0000","last_change":"2023-11-06T20:44:05.911533+0000","last_active":"2023-11-06T20:44:06.394817+0000","last_peered":"2023-11-06T20:44:06.394817+0000","last_clean":"2023-11-06T20:44:06.394817+0000","last_became_active":"2023-11-06T20:44:05.911170+0000","last_became_peered":"2023-11-06T20:44:05.911170+0000","last_unstale":"2023-11-06T20:44:06.394817+0000","last_undegraded":"2023-11-06T20:44:06.394817+0000","last_fullsized":"2023-11-06T20:44:06.394817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426932+0000","last_change":"2023-11-06T20:44:06.413837+0000","last_active":"2023-11-06T20:44:06.426932+0000","last_peered":"2023-11-06T20:44:06.426932+0000","last_clean":"2023-11-06T20:44:06.426932+0000","last_became_active":"2023-11-06T20:44:06.413575+0000","last_became_peered":"2023-11-06T20:44:06.413575+0000","last_unstale":"2023-11-06T20:44:06.426932+0000","last_undegraded":"2023-11-06T20:44:06.426932+0000","last_fullsized":"2023-11-06T20:44:06.426932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395088+0000","last_change":"2023-11-06T20:44:05.919710+0000","last_active":"2023-11-06T20:44:06.395088+0000","last_peered":"2023-11-06T20:44:06.395088+0000","last_clean":"2023-11-06T20:44:06.395088+0000","last_became_active":"2023-11-06T20:44:05.919424+0000","last_became_peered":"2023-11-06T20:44:05.919424+0000","last_unstale":"2023-11-06T20:44:06.395088+0000","last_undegraded":"2023-11-06T20:44:06.395088+0000","last_fullsized":"2023-11-06T20:44:06.395088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.410070+0000","last_change":"2023-11-06T20:44:06.410070+0000","last_active":"2023-11-06T20:44:06.410070+0000","last_peered":"2023-11-06T20:44:06.410070+0000","last_clean":"2023-11-06T20:44:06.410070+0000","last_became_active":"2023-11-06T20:44:06.409768+0000","last_became_peered":"2023-11-06T20:44:06.409768+0000","last_unstale":"2023-11-06T20:44:06.410070+0000","last_undegraded":"2023-11-06T20:44:06.410070+0000","last_fullsized":"2023-11-06T20:44:06.410070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427907+0000","last_change":"2023-11-06T20:44:05.937805+0000","last_active":"2023-11-06T20:44:06.427907+0000","last_peered":"2023-11-06T20:44:06.427907+0000","last_clean":"2023-11-06T20:44:06.427907+0000","last_became_active":"2023-11-06T20:44:05.937296+0000","last_became_peered":"2023-11-06T20:44:05.937296+0000","last_unstale":"2023-11-06T20:44:06.427907+0000","last_undegraded":"2023-11-06T20:44:06.427907+0000","last_fullsized":"2023-11-06T20:44:06.427907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297308,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074532,"statfs":{"total":96636764160,"available":96332320768,"internally_reserved":0,"allocated":217088,"data_stored":80225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297300,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074540,"statfs":{"total":96636764160,"available":96332328960,"internally_reserved":0,"allocated":208896,"data_stored":73611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":74435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":75113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":74165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297328,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074512,"statfs":{"total":96636764160,"available":96332300288,"internally_reserved":0,"allocated":237568,"data_stored":81527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297312,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074528,"statfs":{"total":96636764160,"available":96332316672,"internally_reserved":0,"allocated":221184,"data_stored":80295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297296,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074544,"statfs":{"total":96636764160,"available":96332333056,"internally_reserved":0,"allocated":204800,"data_stored":73521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:44:11.177 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T20:44:11.178 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-06T20:44:11.178 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-06T20:44:11.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-06T20:44:11.179 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-06T20:44:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.407+0000 7fb2ebbff700 1 Processor -- start 2023-11-06T20:44:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.407+0000 7fb2ebbff700 1 -- start start 2023-11-06T20:44:11.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.408+0000 7fb2ebbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 0x7fb2e40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.408+0000 7fb2ebbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb2e40c6460 0x7fb2e40bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.408+0000 7fb2ebbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e41554a0 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.408+0000 7fb2ebbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e4154960 con 0x7fb2e40c6460 2023-11-06T20:44:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.408+0000 7fb2ebbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4155930 con 0x7fb2e40c1840 2023-11-06T20:44:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.409+0000 7fb2e999b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 0x7fb2e40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.409+0000 7fb2ea19c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb2e40c6460 0x7fb2e40bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.409+0000 7fb2ea19c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb2e40c6460 0x7fb2e40bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:51876/0 (socket says 172.21.15.33:51876) 2023-11-06T20:44:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.409+0000 7fb2ea19c700 1 -- 172.21.15.33:0/3235132992 learned_addr learned my addr 172.21.15.33:0/3235132992 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.409+0000 7fb2ea19c700 1 -- 172.21.15.33:0/3235132992 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 msgr2=0x7fb2e40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.409+0000 7fb2ea19c700 1 --2- 172.21.15.33:0/3235132992 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 0x7fb2e40a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.409+0000 7fb2ea19c700 1 -- 172.21.15.33:0/3235132992 >> v1:172.21.15.69:6789/0 conn(0x7fb2e40aa2d0 legacy=0x7fb2e40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.409+0000 7fb2ea19c700 1 -- 172.21.15.33:0/3235132992 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e4154b70 con 0x7fb2e40c6460 2023-11-06T20:44:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.410+0000 7fb2ea19c700 1 --2- 172.21.15.33:0/3235132992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2e40c6460 0x7fb2e40bfd70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fb2dc009f20 tx=0x7fb2dc009840).ready entity=mon.1 client_cookie=9a231bd58566b5dd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.410+0000 7fb2e8999700 1 -- 172.21.15.33:0/3235132992 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00e070 con 0x7fb2e40c6460 2023-11-06T20:44:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.410+0000 7fb2e8999700 1 -- 172.21.15.33:0/3235132992 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc00b9d0 con 0x7fb2e40c6460 2023-11-06T20:44:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.410+0000 7fb2e8999700 1 -- 172.21.15.33:0/3235132992 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00bd50 con 0x7fb2e40c6460 2023-11-06T20:44:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.410+0000 7fb2ebbff700 1 -- 172.21.15.33:0/3235132992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2e40c6460 msgr2=0x7fb2e40bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.410+0000 7fb2ebbff700 1 --2- 172.21.15.33:0/3235132992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2e40c6460 0x7fb2e40bfd70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fb2dc009f20 tx=0x7fb2dc009840).stop 2023-11-06T20:44:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.410+0000 7fb2ebbff700 1 -- 172.21.15.33:0/3235132992 shutdown_connections 2023-11-06T20:44:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.410+0000 7fb2ebbff700 1 --2- 172.21.15.33:0/3235132992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2e40c6460 0x7fb2e40bfd70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.410+0000 7fb2ebbff700 1 --2- 172.21.15.33:0/3235132992 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 0x7fb2e40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.410+0000 7fb2ebbff700 1 -- 172.21.15.33:0/3235132992 >> 172.21.15.33:0/3235132992 conn(0x7fb2e405ec80 msgr2=0x7fb2e40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.411+0000 7fb2ebbff700 1 -- 172.21.15.33:0/3235132992 shutdown_connections 2023-11-06T20:44:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.411+0000 7fb2ebbff700 1 -- 172.21.15.33:0/3235132992 wait complete. 2023-11-06T20:44:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.413+0000 7fb2ebbff700 1 Processor -- start 2023-11-06T20:44:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.413+0000 7fb2ebbff700 1 -- start start 2023-11-06T20:44:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2ebbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e40aa2d0 0x7fb2e40c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2ebbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 0x7fb2e4150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2ebbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2e40c6460 0x7fb2e40ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2ebbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40b0db0 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2ebbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40b0ad0 con 0x7fb2e40c6460 2023-11-06T20:44:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2ebbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c3930 con 0x7fb2e40c1840 2023-11-06T20:44:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2e999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e40aa2d0 0x7fb2e40c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2e999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e40aa2d0 0x7fb2e40c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42708/0 (socket says 172.21.15.33:42708) 2023-11-06T20:44:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2e999b700 1 -- 172.21.15.33:0/398288091 learned_addr learned my addr 172.21.15.33:0/398288091 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2ea19c700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2e40c6460 0x7fb2e40ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.414+0000 7fb2e919a700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 0x7fb2e4150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2e999b700 1 -- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 msgr2=0x7fb2e4150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2e999b700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 0x7fb2e4150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2e999b700 1 -- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2e40c6460 msgr2=0x7fb2e40ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2e999b700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2e40c6460 0x7fb2e40ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2e999b700 1 -- 172.21.15.33:0/398288091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc009ae0 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2e919a700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 0x7fb2e4150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2e999b700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e40aa2d0 0x7fb2e40c4410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fb2e000e930 tx=0x7fb2e0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2d9ffb700 1 -- 172.21.15.33:0/398288091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e000c670 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2d9ffb700 1 -- 172.21.15.33:0/398288091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e000c7d0 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2d9ffb700 1 -- 172.21.15.33:0/398288091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e000cb50 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2ebbff700 1 -- 172.21.15.33:0/398288091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e40c3c10 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.415+0000 7fb2ebbff700 1 -- 172.21.15.33:0/398288091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e40b07a0 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.416+0000 7fb2ebbff700 1 -- 172.21.15.33:0/398288091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e40c40b0 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.417+0000 7fb2d9ffb700 1 -- 172.21.15.33:0/398288091 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb2e001b400 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.418+0000 7fb2d9ffb700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2c805f150 0x7fb2c8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.418+0000 7fb2d9ffb700 1 -- 172.21.15.33:0/398288091 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb2e007a6b0 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.418+0000 7fb2e919a700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2c805f150 0x7fb2c8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.419+0000 7fb2e919a700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2c805f150 0x7fb2c8061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb2e40ca760 tx=0x7fb2d4009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:11.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.422+0000 7fb2d9ffb700 1 -- 172.21.15.33:0/398288091 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb2e004b9d0 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.650+0000 7fb2ebbff700 1 -- 172.21.15.33:0/398288091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb2e40c4260 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.653+0000 7fb2d9ffb700 1 -- 172.21.15.33:0/398288091 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+148953 (secure 0 0 0) 0x7fb2e004eff0 con 0x7fb2e40aa2d0 2023-11-06T20:44:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 -- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2c805f150 msgr2=0x7fb2c8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2c805f150 0x7fb2c8061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb2e40ca760 tx=0x7fb2d4009040).stop 2023-11-06T20:44:11.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 -- 172.21.15.33:0/398288091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e40aa2d0 msgr2=0x7fb2e40c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e40aa2d0 0x7fb2e40c4410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fb2e000e930 tx=0x7fb2e0009060).stop 2023-11-06T20:44:11.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 -- 172.21.15.33:0/398288091 shutdown_connections 2023-11-06T20:44:11.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2c805f150 0x7fb2c8061610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2e40c6460 0x7fb2e40ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e40aa2d0 0x7fb2e40c4410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 --2- 172.21.15.33:0/398288091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2e40c1840 0x7fb2e4150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 -- 172.21.15.33:0/398288091 >> 172.21.15.33:0/398288091 conn(0x7fb2e405ec80 msgr2=0x7fb2e40c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 -- 172.21.15.33:0/398288091 shutdown_connections 2023-11-06T20:44:11.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.656+0000 7fb2ebbff700 1 -- 172.21.15.33:0/398288091 wait complete. 2023-11-06T20:44:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":114912},{"type":"v1","addr":"172.21.15.69:6833","nonce":114912}]},"active_addr":"172.21.15.69:6833/114912","active_change":"2023-11-06T20:43:52.792458+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":1201458877}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":1049662608}]}]} 2023-11-06T20:44:11.676 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-06T20:44:11.676 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-06T20:44:11.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:11.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.872+0000 7fc1133ff700 1 Processor -- start 2023-11-06T20:44:11.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.872+0000 7fc1133ff700 1 -- start start 2023-11-06T20:44:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.873+0000 7fc1133ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc10c0c1840 0x7fc10c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.873+0000 7fc1133ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 0x7fc10c0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.873+0000 7fc1133ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c1554a0 con 0x7fc10c0c6240 2023-11-06T20:44:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.873+0000 7fc1133ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10c154960 con 0x7fc10c0c1840 2023-11-06T20:44:11.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.873+0000 7fc1133ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c155930 con 0x7fc10c0aa2d0 2023-11-06T20:44:11.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.873+0000 7fc11119b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc10c0c1840 0x7fc10c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc11119b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc10c0c1840 0x7fc10c0a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42712/0 (socket says 172.21.15.33:42712) 2023-11-06T20:44:11.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc11119b700 1 -- 172.21.15.33:0/3039197502 learned_addr learned my addr 172.21.15.33:0/3039197502 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:11.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc11099a700 1 --2- 172.21.15.33:0/3039197502 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 0x7fc10c0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc11119b700 1 -- 172.21.15.33:0/3039197502 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 msgr2=0x7fc10c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc11119b700 1 --2- 172.21.15.33:0/3039197502 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 0x7fc10c0c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc11119b700 1 -- 172.21.15.33:0/3039197502 >> v1:172.21.15.69:6789/0 conn(0x7fc10c0c6240 legacy=0x7fc10c157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc11119b700 1 -- 172.21.15.33:0/3039197502 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0c66b0 con 0x7fc10c0c1840 2023-11-06T20:44:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc11119b700 1 --2- 172.21.15.33:0/3039197502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 0x7fc10c0a9de0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fc108006fd0 tx=0x7fc1080090f0).ready entity=mon.0 client_cookie=77bdc4b73886869d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:11.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc103fff700 1 -- 172.21.15.33:0/3039197502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800b580 con 0x7fc10c0c1840 2023-11-06T20:44:11.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc103fff700 1 -- 172.21.15.33:0/3039197502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10800b6e0 con 0x7fc10c0c1840 2023-11-06T20:44:11.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.874+0000 7fc103fff700 1 -- 172.21.15.33:0/3039197502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800ba60 con 0x7fc10c0c1840 2023-11-06T20:44:11.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.875+0000 7fc1133ff700 1 -- 172.21.15.33:0/3039197502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 msgr2=0x7fc10c0a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.875+0000 7fc1133ff700 1 --2- 172.21.15.33:0/3039197502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 0x7fc10c0a9de0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fc108006fd0 tx=0x7fc1080090f0).stop 2023-11-06T20:44:11.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.875+0000 7fc1133ff700 1 -- 172.21.15.33:0/3039197502 shutdown_connections 2023-11-06T20:44:11.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.875+0000 7fc1133ff700 1 --2- 172.21.15.33:0/3039197502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 0x7fc10c0a9de0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.875+0000 7fc1133ff700 1 --2- 172.21.15.33:0/3039197502 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 0x7fc10c0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.875+0000 7fc1133ff700 1 -- 172.21.15.33:0/3039197502 >> 172.21.15.33:0/3039197502 conn(0x7fc10c05ec80 msgr2=0x7fc10c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:11.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.875+0000 7fc1133ff700 1 -- 172.21.15.33:0/3039197502 shutdown_connections 2023-11-06T20:44:11.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.875+0000 7fc1133ff700 1 -- 172.21.15.33:0/3039197502 wait complete. 2023-11-06T20:44:11.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.877+0000 7fc1133ff700 1 Processor -- start 2023-11-06T20:44:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.877+0000 7fc1133ff700 1 -- start start 2023-11-06T20:44:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc1133ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 0x7fc10c0c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc1133ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 0x7fc10c0c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc1133ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc10c0c6240 0x7fc10c0c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc1133ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0c2890 con 0x7fc10c0c1840 2023-11-06T20:44:11.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc1133ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c14dcb0 con 0x7fc10c0c6240 2023-11-06T20:44:11.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc1133ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0c47d0 con 0x7fc10c0aa2d0 2023-11-06T20:44:11.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11099a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 0x7fc10c0c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11099a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 0x7fc10c0c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42720/0 (socket says 172.21.15.33:42720) 2023-11-06T20:44:11.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11099a700 1 -- 172.21.15.33:0/861981905 learned_addr learned my addr 172.21.15.33:0/861981905 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11119b700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 0x7fc10c0c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11199c700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc10c0c6240 0x7fc10c0c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11099a700 1 -- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 msgr2=0x7fc10c0c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11099a700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 0x7fc10c0c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11099a700 1 -- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc10c0c6240 msgr2=0x7fc10c0c23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11099a700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc10c0c6240 0x7fc10c0c23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11099a700 1 -- 172.21.15.33:0/861981905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc108009370 con 0x7fc10c0c1840 2023-11-06T20:44:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11119b700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 0x7fc10c0c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11199c700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc10c0c6240 0x7fc10c0c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.878+0000 7fc11099a700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 0x7fc10c0c1eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fc0fc00e8d0 tx=0x7fc0fc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.879+0000 7fc1017fa700 1 -- 172.21.15.33:0/861981905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00c640 con 0x7fc10c0c1840 2023-11-06T20:44:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.879+0000 7fc1017fa700 1 -- 172.21.15.33:0/861981905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc00c7a0 con 0x7fc10c0c1840 2023-11-06T20:44:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.879+0000 7fc1017fa700 1 -- 172.21.15.33:0/861981905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00cb20 con 0x7fc10c0c1840 2023-11-06T20:44:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.879+0000 7fc1133ff700 1 -- 172.21.15.33:0/861981905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c0c4ab0 con 0x7fc10c0c1840 2023-11-06T20:44:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.879+0000 7fc1133ff700 1 -- 172.21.15.33:0/861981905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c0c5070 con 0x7fc10c0c1840 2023-11-06T20:44:11.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.880+0000 7fc1133ff700 1 -- 172.21.15.33:0/861981905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10c14de40 con 0x7fc10c0c1840 2023-11-06T20:44:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.881+0000 7fc1017fa700 1 -- 172.21.15.33:0/861981905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc0fc01b400 con 0x7fc10c0c1840 2023-11-06T20:44:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.882+0000 7fc1017fa700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0f005f150 0x7fc0f0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.882+0000 7fc1017fa700 1 -- 172.21.15.33:0/861981905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc0fc07a630 con 0x7fc10c0c1840 2023-11-06T20:44:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.882+0000 7fc11119b700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0f005f150 0x7fc0f0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:11.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.883+0000 7fc11119b700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0f005f150 0x7fc0f0061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc108009fd0 tx=0x7fc108017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:11.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:11.886+0000 7fc1017fa700 1 -- 172.21.15.33:0/861981905 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc0fc04b890 con 0x7fc10c0c1840 2023-11-06T20:44:12.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.067+0000 7fc1133ff700 1 -- 172.21.15.33:0/861981905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc10c1652d0 con 0x7fc10c0c1840 2023-11-06T20:44:12.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.068+0000 7fc1017fa700 1 -- 172.21.15.33:0/861981905 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fc0fc04eeb0 con 0x7fc10c0c1840 2023-11-06T20:44:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:12.068 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:12.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.070+0000 7fc1133ff700 1 -- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0f005f150 msgr2=0x7fc0f0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.070+0000 7fc1133ff700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0f005f150 0x7fc0f0061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc108009fd0 tx=0x7fc108017000).stop 2023-11-06T20:44:12.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.070+0000 7fc1133ff700 1 -- 172.21.15.33:0/861981905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 msgr2=0x7fc10c0c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.070+0000 7fc1133ff700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 0x7fc10c0c1eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fc0fc00e8d0 tx=0x7fc0fc009060).stop 2023-11-06T20:44:12.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.070+0000 7fc1133ff700 1 -- 172.21.15.33:0/861981905 shutdown_connections 2023-11-06T20:44:12.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.070+0000 7fc1133ff700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0f005f150 0x7fc0f0061610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.070+0000 7fc1133ff700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc10c0aa2d0 0x7fc10c0c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.070+0000 7fc1133ff700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc10c0c6240 0x7fc10c0c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.070+0000 7fc1133ff700 1 --2- 172.21.15.33:0/861981905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc10c0c1840 0x7fc10c0c1eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.070+0000 7fc1133ff700 1 -- 172.21.15.33:0/861981905 >> 172.21.15.33:0/861981905 conn(0x7fc10c05ec80 msgr2=0x7fc10c152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:12.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.071+0000 7fc1133ff700 1 -- 172.21.15.33:0/861981905 shutdown_connections 2023-11-06T20:44:12.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.071+0000 7fc1133ff700 1 -- 172.21.15.33:0/861981905 wait complete. 2023-11-06T20:44:12.087 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-06T20:44:12.087 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.285+0000 7f1554fff700 1 Processor -- start 2023-11-06T20:44:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.285+0000 7f1554fff700 1 -- start start 2023-11-06T20:44:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.286+0000 7f1554fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15500c6de0 0x7f15500c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.286+0000 7f1554fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15500c0830 0x7f15500c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.286+0000 7f1554fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15500c9bd0 con 0x7f15501542e0 2023-11-06T20:44:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.286+0000 7f1554fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155005f0a0 con 0x7f15500c6de0 2023-11-06T20:44:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.286+0000 7f1554fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15500a8300 con 0x7f15500c0830 2023-11-06T20:44:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15500c6de0 0x7f15500c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15500c6de0 0x7f15500c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42726/0 (socket says 172.21.15.33:42726) 2023-11-06T20:44:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154dd9b700 1 -- 172.21.15.33:0/788825755 learned_addr learned my addr 172.21.15.33:0/788825755 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154ed9d700 1 --2- 172.21.15.33:0/788825755 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15500c0830 0x7f15500c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154dd9b700 1 -- 172.21.15.33:0/788825755 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15500c0830 msgr2=0x7f15500c0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154dd9b700 1 --2- 172.21.15.33:0/788825755 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15500c0830 0x7f15500c0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154dd9b700 1 -- 172.21.15.33:0/788825755 >> v1:172.21.15.69:6789/0 conn(0x7f15501542e0 legacy=0x7f15500a7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154dd9b700 1 -- 172.21.15.33:0/788825755 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15500c11b0 con 0x7f15500c6de0 2023-11-06T20:44:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154dd9b700 1 --2- 172.21.15.33:0/788825755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 0x7f15500c7290 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1544009830 tx=0x7f15440090f0).ready entity=mon.0 client_cookie=33be5dc35f02dd63 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154d59a700 1 -- 172.21.15.33:0/788825755 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154400b580 con 0x7f15500c6de0 2023-11-06T20:44:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154d59a700 1 -- 172.21.15.33:0/788825755 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154400b6e0 con 0x7f15500c6de0 2023-11-06T20:44:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.287+0000 7f154d59a700 1 -- 172.21.15.33:0/788825755 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154400ba60 con 0x7f15500c6de0 2023-11-06T20:44:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.288+0000 7f1554fff700 1 -- 172.21.15.33:0/788825755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 msgr2=0x7f15500c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.288+0000 7f1554fff700 1 --2- 172.21.15.33:0/788825755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 0x7f15500c7290 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1544009830 tx=0x7f15440090f0).stop 2023-11-06T20:44:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.288+0000 7f1554fff700 1 -- 172.21.15.33:0/788825755 shutdown_connections 2023-11-06T20:44:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.288+0000 7f1554fff700 1 --2- 172.21.15.33:0/788825755 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15500c0830 0x7f15500c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.288+0000 7f1554fff700 1 --2- 172.21.15.33:0/788825755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 0x7f15500c7290 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.288+0000 7f1554fff700 1 -- 172.21.15.33:0/788825755 >> 172.21.15.33:0/788825755 conn(0x7f155005e990 msgr2=0x7f15500cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.288+0000 7f1554fff700 1 -- 172.21.15.33:0/788825755 shutdown_connections 2023-11-06T20:44:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.288+0000 7f1554fff700 1 -- 172.21.15.33:0/788825755 wait complete. 2023-11-06T20:44:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.290+0000 7f1554fff700 1 Processor -- start 2023-11-06T20:44:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.290+0000 7f1554fff700 1 -- start start 2023-11-06T20:44:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f1554fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15500c0830 0x7f15500ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f1554fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 0x7f1550164f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f1554fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15501542e0 0x7f1550150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f1554fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1550098780 con 0x7f15500c6de0 2023-11-06T20:44:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f1554fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15500ad7d0 con 0x7f15500c0830 2023-11-06T20:44:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f1554fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1550165430 con 0x7f15501542e0 2023-11-06T20:44:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f154dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 0x7f1550164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f154dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 0x7f1550164f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42728/0 (socket says 172.21.15.33:42728) 2023-11-06T20:44:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f154dd9b700 1 -- 172.21.15.33:0/1016356959 learned_addr learned my addr 172.21.15.33:0/1016356959 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f154e59c700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15500c0830 0x7f15500ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f154ed9d700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15501542e0 0x7f1550150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f154dd9b700 1 -- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15501542e0 msgr2=0x7f1550150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.291+0000 7f154dd9b700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15501542e0 0x7f1550150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f154dd9b700 1 -- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15500c0830 msgr2=0x7f15500ac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f154dd9b700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15500c0830 0x7f15500ac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f154dd9b700 1 -- 172.21.15.33:0/1016356959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1544009370 con 0x7f15500c6de0 2023-11-06T20:44:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f154e59c700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15500c0830 0x7f15500ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f154ed9d700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15501542e0 0x7f1550150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f154dd9b700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 0x7f1550164f40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1550083ab0 tx=0x7f1544017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f153e7fc700 1 -- 172.21.15.33:0/1016356959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154400b850 con 0x7f15500c6de0 2023-11-06T20:44:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f153e7fc700 1 -- 172.21.15.33:0/1016356959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154400b9b0 con 0x7f15500c6de0 2023-11-06T20:44:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f153e7fc700 1 -- 172.21.15.33:0/1016356959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154401c670 con 0x7f15500c6de0 2023-11-06T20:44:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f1554fff700 1 -- 172.21.15.33:0/1016356959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15500acbc0 con 0x7f15500c6de0 2023-11-06T20:44:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.292+0000 7f1554fff700 1 -- 172.21.15.33:0/1016356959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15500ad180 con 0x7f15500c6de0 2023-11-06T20:44:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.293+0000 7f1554fff700 1 -- 172.21.15.33:0/1016356959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155006c6c0 con 0x7f15500c6de0 2023-11-06T20:44:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.294+0000 7f153e7fc700 1 -- 172.21.15.33:0/1016356959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f154401ca20 con 0x7f15500c6de0 2023-11-06T20:44:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.295+0000 7f153e7fc700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f153005f120 0x7f15300615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.295+0000 7f153e7fc700 1 -- 172.21.15.33:0/1016356959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f154407bfa0 con 0x7f15500c6de0 2023-11-06T20:44:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.295+0000 7f154e59c700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f153005f120 0x7f15300615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.296+0000 7f154e59c700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f153005f120 0x7f15300615e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f1540001690 tx=0x7f1540009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.299+0000 7f153e7fc700 1 -- 172.21.15.33:0/1016356959 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f154404d1b0 con 0x7f15500c6de0 2023-11-06T20:44:12.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.482+0000 7f1554fff700 1 -- 172.21.15.33:0/1016356959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15500c8ce0 con 0x7f15500c6de0 2023-11-06T20:44:12.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.483+0000 7f153e7fc700 1 -- 172.21.15.33:0/1016356959 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f154402a070 con 0x7f15500c6de0 2023-11-06T20:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 -- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f153005f120 msgr2=0x7f15300615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f153005f120 0x7f15300615e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f1540001690 tx=0x7f1540009040).stop 2023-11-06T20:44:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 -- 172.21.15.33:0/1016356959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 msgr2=0x7f1550164f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 0x7f1550164f40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1550083ab0 tx=0x7f1544017000).stop 2023-11-06T20:44:12.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 -- 172.21.15.33:0/1016356959 shutdown_connections 2023-11-06T20:44:12.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f153005f120 0x7f15300615e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15501542e0 0x7f1550150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15500c0830 0x7f15500ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 --2- 172.21.15.33:0/1016356959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15500c6de0 0x7f1550164f40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 -- 172.21.15.33:0/1016356959 >> 172.21.15.33:0/1016356959 conn(0x7f155005e990 msgr2=0x7f15500a7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:12.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 -- 172.21.15.33:0/1016356959 shutdown_connections 2023-11-06T20:44:12.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.485+0000 7f1554fff700 1 -- 172.21.15.33:0/1016356959 wait complete. 2023-11-06T20:44:12.502 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-06T20:44:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.699+0000 7f06ff3ff700 1 Processor -- start 2023-11-06T20:44:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.699+0000 7f06ff3ff700 1 -- start start 2023-11-06T20:44:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.700+0000 7f06ff3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f06f80c6d90 0x7f06f80c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.700+0000 7f06ff3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06f80bfd60 0x7f06f80c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.700+0000 7f06ff3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f80c7700 con 0x7f06f8154210 2023-11-06T20:44:12.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.700+0000 7f06ff3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f80c06c0 con 0x7f06f80bfd60 2023-11-06T20:44:12.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.700+0000 7f06ff3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f805efc0 con 0x7f06f80c6d90 2023-11-06T20:44:12.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.700+0000 7f06fd99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06f80bfd60 0x7f06f80c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.700+0000 7f06fd99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06f80bfd60 0x7f06f80c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42742/0 (socket says 172.21.15.33:42742) 2023-11-06T20:44:12.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.700+0000 7f06fd99c700 1 -- 172.21.15.33:0/4205620118 learned_addr learned my addr 172.21.15.33:0/4205620118 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:12.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.701+0000 7f06fc99a700 1 --2- 172.21.15.33:0/4205620118 >> v2:172.21.15.69:3300/0 conn(0x7f06f80c6d90 0x7f06f80c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.701+0000 7f06fd99c700 1 -- 172.21.15.33:0/4205620118 >> v2:172.21.15.69:3300/0 conn(0x7f06f80c6d90 msgr2=0x7f06f80c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:12.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.701+0000 7f06fd99c700 1 --2- 172.21.15.33:0/4205620118 >> v2:172.21.15.69:3300/0 conn(0x7f06f80c6d90 0x7f06f80c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.701+0000 7f06fd99c700 1 -- 172.21.15.33:0/4205620118 >> v1:172.21.15.69:6789/0 conn(0x7f06f8154210 legacy=0x7f06f80a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.701+0000 7f06fd99c700 1 -- 172.21.15.33:0/4205620118 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f80c8ea0 con 0x7f06f80bfd60 2023-11-06T20:44:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.701+0000 7f06fd99c700 1 --2- 172.21.15.33:0/4205620118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f80bfd60 0x7f06f80c01d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f06f80cbf10 tx=0x7f06f0009840).ready entity=mon.0 client_cookie=80f56e885e83db22 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.701+0000 7f06effff700 1 -- 172.21.15.33:0/4205620118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f000e070 con 0x7f06f80bfd60 2023-11-06T20:44:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.701+0000 7f06effff700 1 -- 172.21.15.33:0/4205620118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f000ba20 con 0x7f06f80bfd60 2023-11-06T20:44:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.701+0000 7f06effff700 1 -- 172.21.15.33:0/4205620118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f000bda0 con 0x7f06f80bfd60 2023-11-06T20:44:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.702+0000 7f06ff3ff700 1 -- 172.21.15.33:0/4205620118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f80bfd60 msgr2=0x7f06f80c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.702+0000 7f06ff3ff700 1 --2- 172.21.15.33:0/4205620118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f80bfd60 0x7f06f80c01d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f06f80cbf10 tx=0x7f06f0009840).stop 2023-11-06T20:44:12.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.702+0000 7f06ff3ff700 1 -- 172.21.15.33:0/4205620118 shutdown_connections 2023-11-06T20:44:12.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.702+0000 7f06ff3ff700 1 --2- 172.21.15.33:0/4205620118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f80bfd60 0x7f06f80c01d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.702+0000 7f06ff3ff700 1 --2- 172.21.15.33:0/4205620118 >> v2:172.21.15.69:3300/0 conn(0x7f06f80c6d90 0x7f06f80c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.702+0000 7f06ff3ff700 1 -- 172.21.15.33:0/4205620118 >> 172.21.15.33:0/4205620118 conn(0x7f06f805e990 msgr2=0x7f06f80cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:12.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.702+0000 7f06ff3ff700 1 -- 172.21.15.33:0/4205620118 shutdown_connections 2023-11-06T20:44:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.702+0000 7f06ff3ff700 1 -- 172.21.15.33:0/4205620118 wait complete. 2023-11-06T20:44:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.704+0000 7f06ff3ff700 1 Processor -- start 2023-11-06T20:44:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.704+0000 7f06ff3ff700 1 -- start start 2023-11-06T20:44:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06ff3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f80bfd60 0x7f06f81560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06ff3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f80c6d90 0x7f06f80c2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06ff3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06f8154210 0x7f06f80c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06ff3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f80c3f10 con 0x7f06f80bfd60 2023-11-06T20:44:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06ff3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f8157280 con 0x7f06f80c6d90 2023-11-06T20:44:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06ff3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f814e500 con 0x7f06f8154210 2023-11-06T20:44:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06fd19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f80bfd60 0x7f06f81560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06fd19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f80bfd60 0x7f06f81560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42758/0 (socket says 172.21.15.33:42758) 2023-11-06T20:44:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06fd19b700 1 -- 172.21.15.33:0/577420498 learned_addr learned my addr 172.21.15.33:0/577420498 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06fc99a700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f80c6d90 0x7f06f80c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06fd99c700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06f8154210 0x7f06f80c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06fd19b700 1 -- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06f8154210 msgr2=0x7f06f80c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06fd19b700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06f8154210 0x7f06f80c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06fd19b700 1 -- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f80c6d90 msgr2=0x7f06f80c2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06fd19b700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f80c6d90 0x7f06f80c2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.705+0000 7f06fd19b700 1 -- 172.21.15.33:0/577420498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f0009ae0 con 0x7f06f80bfd60 2023-11-06T20:44:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.706+0000 7f06fd19b700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f80bfd60 0x7f06f81560e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f06f400c440 tx=0x7f06f4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.706+0000 7f06ed7fa700 1 -- 172.21.15.33:0/577420498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4007020 con 0x7f06f80bfd60 2023-11-06T20:44:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.706+0000 7f06ed7fa700 1 -- 172.21.15.33:0/577420498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f4007180 con 0x7f06f80bfd60 2023-11-06T20:44:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.706+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f8156670 con 0x7f06f80bfd60 2023-11-06T20:44:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.706+0000 7f06ed7fa700 1 -- 172.21.15.33:0/577420498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4007520 con 0x7f06f80bfd60 2023-11-06T20:44:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.706+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f8156c30 con 0x7f06f80bfd60 2023-11-06T20:44:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.707+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f06f806c6c0 con 0x7f06f80bfd60 2023-11-06T20:44:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.708+0000 7f06ed7fa700 1 -- 172.21.15.33:0/577420498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f06f40068b0 con 0x7f06f80bfd60 2023-11-06T20:44:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.709+0000 7f06ed7fa700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f06dc05f150 0x7f06dc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.709+0000 7f06ed7fa700 1 -- 172.21.15.33:0/577420498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f06f4079320 con 0x7f06f80bfd60 2023-11-06T20:44:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.709+0000 7f06ed7fa700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] conn(0x7f06dc0684f0 0x7f06dc06a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.709+0000 7f06ed7fa700 1 -- 172.21.15.33:0/577420498 --> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f06dc06afd0 con 0x7f06dc0684f0 2023-11-06T20:44:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.709+0000 7f06ed7fa700 1 -- 172.21.15.33:0/577420498 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06f407b880 con 0x7f06f80bfd60 2023-11-06T20:44:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.709+0000 7f06fc99a700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f06dc05f150 0x7f06dc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.710+0000 7f06fd99c700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] conn(0x7f06dc0684f0 0x7f06dc06a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.710+0000 7f06fc99a700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f06dc05f150 0x7f06dc061610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f06f80c5240 tx=0x7f06e8009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.710+0000 7f06fd99c700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] conn(0x7f06dc0684f0 0x7f06dc06a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.711+0000 7f06ed7fa700 1 -- 172.21.15.33:0/577420498 <== osd.0 v2:172.21.15.33:6808/115395 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f06dc06afd0 con 0x7f06dc0684f0 2023-11-06T20:44:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.726+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 --> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f06f80c7700 con 0x7f06dc0684f0 2023-11-06T20:44:12.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.727+0000 7f06ed7fa700 1 -- 172.21.15.33:0/577420498 <== osd.0 v2:172.21.15.33:6808/115395 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f06f80c7700 con 0x7f06dc0684f0 2023-11-06T20:44:12.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.727+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 >> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] conn(0x7f06dc0684f0 msgr2=0x7f06dc06a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.727+0000 7f06ff3ff700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] conn(0x7f06dc0684f0 0x7f06dc06a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.727+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f06dc05f150 msgr2=0x7f06dc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.727+0000 7f06ff3ff700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f06dc05f150 0x7f06dc061610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f06f80c5240 tx=0x7f06e8009060).stop 2023-11-06T20:44:12.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.727+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f80bfd60 msgr2=0x7f06f81560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.727+0000 7f06ff3ff700 1 --2- 172.21.15.33:0/577420498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f80bfd60 0x7f06f81560e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f06f400c440 tx=0x7f06f4007e00).stop 2023-11-06T20:44:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.727+0000 7f06fd99c700 1 -- 172.21.15.33:0/577420498 reap_dead start 2023-11-06T20:44:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.727+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 shutdown_connections 2023-11-06T20:44:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.727+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 >> 172.21.15.33:0/577420498 conn(0x7f06f805e990 msgr2=0x7f06f8151390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:12.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.728+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 shutdown_connections 2023-11-06T20:44:12.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.728+0000 7f06ff3ff700 1 -- 172.21.15.33:0/577420498 wait complete. 2023-11-06T20:44:12.743 INFO:teuthology.orchestra.run.smithi033.stdout:55834574852 2023-11-06T20:44:12.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-06T20:44:12.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.941+0000 7fa341bff700 1 Processor -- start 2023-11-06T20:44:12.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.941+0000 7fa341bff700 1 -- start start 2023-11-06T20:44:12.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.942+0000 7fa341bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa33c154210 0x7fa33c0a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.942+0000 7fa341bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c05efc0 con 0x7fa33c0ca610 2023-11-06T20:44:12.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.942+0000 7fa341bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c0c8ea0 con 0x7fa33c0bfd60 2023-11-06T20:44:12.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.942+0000 7fa341bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa33c0caa80 con 0x7fa33c154210 2023-11-06T20:44:12.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.943+0000 7fa33bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa33c0ca610 0x7fa33c0c6d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59726/0 (socket says 172.21.15.33:59726) 2023-11-06T20:44:12.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.943+0000 7fa33bfff700 1 -- 172.21.15.33:0/859173343 learned_addr learned my addr 172.21.15.33:0/859173343 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:12.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.943+0000 7fa33b7fe700 1 --2- 172.21.15.33:0/859173343 >> v2:172.21.15.69:3300/0 conn(0x7fa33c154210 0x7fa33c0a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.943+0000 7fa33a7fc700 1 -- 172.21.15.33:0/859173343 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868744610 0 0) 0x7fa33c05efc0 con 0x7fa33c0ca610 2023-11-06T20:44:12.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.943+0000 7fa33a7fc700 1 -- 172.21.15.33:0/859173343 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3240033f0 con 0x7fa33c0ca610 2023-11-06T20:44:12.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.943+0000 7fa33a7fc700 1 -- 172.21.15.33:0/859173343 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725383967 0 0) 0x7fa33c0c8ea0 con 0x7fa33c0bfd60 2023-11-06T20:44:12.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.943+0000 7fa33a7fc700 1 -- 172.21.15.33:0/859173343 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa33c05efc0 con 0x7fa33c0bfd60 2023-11-06T20:44:12.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.943+0000 7fa33a7fc700 1 -- 172.21.15.33:0/859173343 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899941918 0 0) 0x7fa3240033f0 con 0x7fa33c0ca610 2023-11-06T20:44:12.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.943+0000 7fa33b7fe700 1 -- 172.21.15.33:0/859173343 >> v1:172.21.15.69:6789/0 conn(0x7fa33c0bfd60 legacy=0x7fa33c0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.943+0000 7fa33b7fe700 1 -- 172.21.15.33:0/859173343 >> v1:172.21.15.33:6789/0 conn(0x7fa33c0ca610 legacy=0x7fa33c0c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.944+0000 7fa33b7fe700 1 -- 172.21.15.33:0/859173343 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c0c9140 con 0x7fa33c154210 2023-11-06T20:44:12.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.944+0000 7fa33b7fe700 1 --2- 172.21.15.33:0/859173343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa33c154210 0x7fa33c0a7bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa32c009760 tx=0x7fa32c009060).ready entity=mon.1 client_cookie=1013d549cbe0d789 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.944+0000 7fa33a7fc700 1 -- 172.21.15.33:0/859173343 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3340017f0 con 0x7fa33c154210 2023-11-06T20:44:12.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.944+0000 7fa33a7fc700 1 -- 172.21.15.33:0/859173343 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c011400 con 0x7fa33c154210 2023-11-06T20:44:12.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.944+0000 7fa33a7fc700 1 -- 172.21.15.33:0/859173343 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c011780 con 0x7fa33c154210 2023-11-06T20:44:12.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.944+0000 7fa341bff700 1 -- 172.21.15.33:0/859173343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa33c154210 msgr2=0x7fa33c0a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.944+0000 7fa341bff700 1 --2- 172.21.15.33:0/859173343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa33c154210 0x7fa33c0a7bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa32c009760 tx=0x7fa32c009060).stop 2023-11-06T20:44:12.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.945+0000 7fa341bff700 1 -- 172.21.15.33:0/859173343 shutdown_connections 2023-11-06T20:44:12.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.945+0000 7fa341bff700 1 --2- 172.21.15.33:0/859173343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa33c154210 0x7fa33c0a7bd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.945+0000 7fa341bff700 1 -- 172.21.15.33:0/859173343 >> 172.21.15.33:0/859173343 conn(0x7fa33c05e990 msgr2=0x7fa33c0cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:12.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.945+0000 7fa341bff700 1 -- 172.21.15.33:0/859173343 shutdown_connections 2023-11-06T20:44:12.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.945+0000 7fa341bff700 1 -- 172.21.15.33:0/859173343 wait complete. 2023-11-06T20:44:12.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.946+0000 7fa341bff700 1 Processor -- start 2023-11-06T20:44:12.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.946+0000 7fa341bff700 1 -- start start 2023-11-06T20:44:12.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.947+0000 7fa341bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c0bfd60 0x7fa33c0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.947+0000 7fa341bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa33c0ca610 0x7fa33c0a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.947+0000 7fa341bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa33c154210 0x7fa33c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.947+0000 7fa341bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0af000 con 0x7fa33c0bfd60 2023-11-06T20:44:12.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.947+0000 7fa341bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0af2e0 con 0x7fa33c0ca610 2023-11-06T20:44:12.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.947+0000 7fa341bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0ae6b0 con 0x7fa33c154210 2023-11-06T20:44:12.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.947+0000 7fa33b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c0bfd60 0x7fa33c0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa33b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c0bfd60 0x7fa33c0addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42766/0 (socket says 172.21.15.33:42766) 2023-11-06T20:44:12.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa33b7fe700 1 -- 172.21.15.33:0/220859193 learned_addr learned my addr 172.21.15.33:0/220859193 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa33affd700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa33c0ca610 0x7fa33c0a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa33b7fe700 1 -- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa33c154210 msgr2=0x7fa33c1568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa33b7fe700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa33c154210 0x7fa33c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:44:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa33b7fe700 1 -- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa33c0ca610 msgr2=0x7fa33c0a8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa33b7fe700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa33c0ca610 0x7fa33c0a8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa33b7fe700 1 -- 172.21.15.33:0/220859193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c0092a0 con 0x7fa33c0bfd60 2023-11-06T20:44:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa33affd700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa33c0ca610 0x7fa33c0a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa33b7fe700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c0bfd60 0x7fa33c0addb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa32c0118a0 tx=0x7fa32c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa3237fe700 1 -- 172.21.15.33:0/220859193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0108e0 con 0x7fa33c0bfd60 2023-11-06T20:44:12.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa3237fe700 1 -- 172.21.15.33:0/220859193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c010a40 con 0x7fa33c0bfd60 2023-11-06T20:44:12.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa3237fe700 1 -- 172.21.15.33:0/220859193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c010de0 con 0x7fa33c0bfd60 2023-11-06T20:44:12.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.948+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33c0ae990 con 0x7fa33c0bfd60 2023-11-06T20:44:12.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.949+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33c0ae2a0 con 0x7fa33c0bfd60 2023-11-06T20:44:12.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.949+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa33c06c6c0 con 0x7fa33c0bfd60 2023-11-06T20:44:12.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.950+0000 7fa3237fe700 1 -- 172.21.15.33:0/220859193 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa32c01d640 con 0x7fa33c0bfd60 2023-11-06T20:44:12.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.951+0000 7fa3237fe700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa32805f150 0x7fa328061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.951+0000 7fa3237fe700 1 -- 172.21.15.33:0/220859193 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa32c07b7c0 con 0x7fa33c0bfd60 2023-11-06T20:44:12.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.951+0000 7fa3237fe700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] conn(0x7fa3280684f0 0x7fa32806a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:12.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.951+0000 7fa3237fe700 1 -- 172.21.15.33:0/220859193 --> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa32806afd0 con 0x7fa3280684f0 2023-11-06T20:44:12.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.951+0000 7fa3237fe700 1 -- 172.21.15.33:0/220859193 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa32c07dd20 con 0x7fa33c0bfd60 2023-11-06T20:44:12.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.951+0000 7fa33affd700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa32805f150 0x7fa328061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.952+0000 7fa33bfff700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] conn(0x7fa3280684f0 0x7fa32806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:12.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.952+0000 7fa33affd700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa32805f150 0x7fa328061610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fa33c150920 tx=0x7fa330009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.952+0000 7fa33bfff700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] conn(0x7fa3280684f0 0x7fa32806a950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:12.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.953+0000 7fa3237fe700 1 -- 172.21.15.33:0/220859193 <== osd.1 v2:172.21.15.33:6800/115394 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fa32806afd0 con 0x7fa3280684f0 2023-11-06T20:44:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.968+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 --> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa33c0af420 con 0x7fa3280684f0 2023-11-06T20:44:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.968+0000 7fa3237fe700 1 -- 172.21.15.33:0/220859193 <== osd.1 v2:172.21.15.33:6800/115394 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa334007e10 con 0x7fa3280684f0 2023-11-06T20:44:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.968+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 >> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] conn(0x7fa3280684f0 msgr2=0x7fa32806a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.968+0000 7fa341bff700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] conn(0x7fa3280684f0 0x7fa32806a950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.969+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa32805f150 msgr2=0x7fa328061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.969+0000 7fa341bff700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa32805f150 0x7fa328061610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fa33c150920 tx=0x7fa330009040).stop 2023-11-06T20:44:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.969+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c0bfd60 msgr2=0x7fa33c0addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.969+0000 7fa341bff700 1 --2- 172.21.15.33:0/220859193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c0bfd60 0x7fa33c0addb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa32c0118a0 tx=0x7fa32c016000).stop 2023-11-06T20:44:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.969+0000 7fa33bfff700 1 -- 172.21.15.33:0/220859193 reap_dead start 2023-11-06T20:44:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.969+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 shutdown_connections 2023-11-06T20:44:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.969+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 >> 172.21.15.33:0/220859193 conn(0x7fa33c05e990 msgr2=0x7fa33c0c6850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.969+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 shutdown_connections 2023-11-06T20:44:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:12.969+0000 7fa341bff700 1 -- 172.21.15.33:0/220859193 wait complete. 2023-11-06T20:44:12.984 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-11-06T20:44:12.984 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-06T20:44:13.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.181+0000 7ff2935ff700 1 Processor -- start 2023-11-06T20:44:13.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.182+0000 7ff2935ff700 1 -- start start 2023-11-06T20:44:13.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.183+0000 7ff2935ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff28c0632f0 0x7ff28c0636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.183+0000 7ff2935ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff28c158960 0x7ff28c1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.183+0000 7ff2935ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28c1584d0 con 0x7ff28c154ec0 2023-11-06T20:44:13.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.183+0000 7ff2935ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28c1559c0 con 0x7ff28c158960 2023-11-06T20:44:13.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.183+0000 7ff2935ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28c155400 con 0x7ff28c0632f0 2023-11-06T20:44:13.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.183+0000 7ff290b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff28c158960 0x7ff28c1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.183+0000 7ff290b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff28c158960 0x7ff28c1549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42778/0 (socket says 172.21.15.33:42778) 2023-11-06T20:44:13.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.183+0000 7ff290b9a700 1 -- 172.21.15.33:0/3198039962 learned_addr learned my addr 172.21.15.33:0/3198039962 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:13.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.184+0000 7ff29139b700 1 --2- 172.21.15.33:0/3198039962 >> v2:172.21.15.69:3300/0 conn(0x7ff28c0632f0 0x7ff28c0636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.184+0000 7ff290b9a700 1 -- 172.21.15.33:0/3198039962 >> v2:172.21.15.69:3300/0 conn(0x7ff28c0632f0 msgr2=0x7ff28c0636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:13.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.184+0000 7ff290b9a700 1 --2- 172.21.15.33:0/3198039962 >> v2:172.21.15.69:3300/0 conn(0x7ff28c0632f0 0x7ff28c0636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.184+0000 7ff290b9a700 1 -- 172.21.15.33:0/3198039962 >> v1:172.21.15.33:6789/0 conn(0x7ff28c154ec0 legacy=0x7ff28c14e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.184+0000 7ff290b9a700 1 -- 172.21.15.33:0/3198039962 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c1556e0 con 0x7ff28c158960 2023-11-06T20:44:13.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.184+0000 7ff283fff700 1 -- 172.21.15.33:0/3198039962 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245731193 0 0) 0x7ff28c1584d0 con 0x7ff28c154ec0 2023-11-06T20:44:13.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.184+0000 7ff290b9a700 1 --2- 172.21.15.33:0/3198039962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28c158960 0x7ff28c1549d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ff28c0c56b0 tx=0x7ff27c0090f0).ready entity=mon.0 client_cookie=9b9b36ab67d9fb3b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.184+0000 7ff283fff700 1 -- 172.21.15.33:0/3198039962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00b600 con 0x7ff28c158960 2023-11-06T20:44:13.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.184+0000 7ff283fff700 1 -- 172.21.15.33:0/3198039962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27c00b760 con 0x7ff28c158960 2023-11-06T20:44:13.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.184+0000 7ff283fff700 1 -- 172.21.15.33:0/3198039962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00bae0 con 0x7ff28c158960 2023-11-06T20:44:13.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.185+0000 7ff2935ff700 1 -- 172.21.15.33:0/3198039962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28c158960 msgr2=0x7ff28c1549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.185+0000 7ff2935ff700 1 --2- 172.21.15.33:0/3198039962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28c158960 0x7ff28c1549d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ff28c0c56b0 tx=0x7ff27c0090f0).stop 2023-11-06T20:44:13.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.185+0000 7ff2935ff700 1 -- 172.21.15.33:0/3198039962 shutdown_connections 2023-11-06T20:44:13.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.185+0000 7ff2935ff700 1 --2- 172.21.15.33:0/3198039962 >> v2:172.21.15.69:3300/0 conn(0x7ff28c0632f0 0x7ff28c0636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.185+0000 7ff2935ff700 1 --2- 172.21.15.33:0/3198039962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28c158960 0x7ff28c1549d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.185+0000 7ff2935ff700 1 -- 172.21.15.33:0/3198039962 >> 172.21.15.33:0/3198039962 conn(0x7ff28c0cb910 msgr2=0x7ff28c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:13.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.185+0000 7ff2935ff700 1 -- 172.21.15.33:0/3198039962 shutdown_connections 2023-11-06T20:44:13.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.185+0000 7ff2935ff700 1 -- 172.21.15.33:0/3198039962 wait complete. 2023-11-06T20:44:13.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.186+0000 7ff2935ff700 1 Processor -- start 2023-11-06T20:44:13.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.187+0000 7ff2935ff700 1 -- start start 2023-11-06T20:44:13.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.187+0000 7ff2935ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28c0632f0 0x7ff28c0c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff2935ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c154ec0 0x7ff28c0c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff2935ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff28c158960 0x7ff28c0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff2935ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c0aff00 con 0x7ff28c0632f0 2023-11-06T20:44:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff2935ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c168970 con 0x7ff28c154ec0 2023-11-06T20:44:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff2935ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c0af8b0 con 0x7ff28c158960 2023-11-06T20:44:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff29139b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28c0632f0 0x7ff28c0c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff29139b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28c0632f0 0x7ff28c0c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42794/0 (socket says 172.21.15.33:42794) 2023-11-06T20:44:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff29139b700 1 -- 172.21.15.33:0/3367132545 learned_addr learned my addr 172.21.15.33:0/3367132545 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff29139b700 1 -- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff28c158960 msgr2=0x7ff28c0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff29139b700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff28c158960 0x7ff28c0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:44:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff29139b700 1 -- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c154ec0 msgr2=0x7ff28c0c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff290b9a700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c154ec0 0x7ff28c0c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff29139b700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c154ec0 0x7ff28c0c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff29139b700 1 -- 172.21.15.33:0/3367132545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c009370 con 0x7ff28c0632f0 2023-11-06T20:44:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff290b9a700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c154ec0 0x7ff28c0c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.188+0000 7ff29139b700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28c0632f0 0x7ff28c0c1ee0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7ff28800d890 tx=0x7ff288008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.189+0000 7ff2817fa700 1 -- 172.21.15.33:0/3367132545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880125f0 con 0x7ff28c0632f0 2023-11-06T20:44:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.189+0000 7ff2817fa700 1 -- 172.21.15.33:0/3367132545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff288012750 con 0x7ff28c0632f0 2023-11-06T20:44:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.189+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28c0afb90 con 0x7ff28c0632f0 2023-11-06T20:44:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.189+0000 7ff2817fa700 1 -- 172.21.15.33:0/3367132545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288012ad0 con 0x7ff28c0632f0 2023-11-06T20:44:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.189+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28c0c23d0 con 0x7ff28c0632f0 2023-11-06T20:44:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.190+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff28c0af710 con 0x7ff28c0632f0 2023-11-06T20:44:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.191+0000 7ff2817fa700 1 -- 172.21.15.33:0/3367132545 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff288013480 con 0x7ff28c0632f0 2023-11-06T20:44:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.192+0000 7ff2817fa700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27005f180 0x7ff270061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.192+0000 7ff2817fa700 1 -- 172.21.15.33:0/3367132545 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff28807a010 con 0x7ff28c0632f0 2023-11-06T20:44:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.192+0000 7ff2817fa700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] conn(0x7ff270068520 0x7ff27006a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.192+0000 7ff2817fa700 1 -- 172.21.15.33:0/3367132545 --> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff27006b000 con 0x7ff270068520 2023-11-06T20:44:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.192+0000 7ff2817fa700 1 -- 172.21.15.33:0/3367132545 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff28807c630 con 0x7ff28c0632f0 2023-11-06T20:44:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.192+0000 7ff290b9a700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27005f180 0x7ff270061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.192+0000 7ff291b9c700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] conn(0x7ff270068520 0x7ff27006a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.193+0000 7ff290b9a700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27005f180 0x7ff270061640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff28c0ca8a0 tx=0x7ff27c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.193+0000 7ff291b9c700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] conn(0x7ff270068520 0x7ff27006a980 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.194+0000 7ff2817fa700 1 -- 172.21.15.33:0/3367132545 <== osd.2 v2:172.21.15.33:6818/115398 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7ff27006b000 con 0x7ff270068520 2023-11-06T20:44:13.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.209+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 --> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff28c0b0040 con 0x7ff270068520 2023-11-06T20:44:13.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.209+0000 7ff2817fa700 1 -- 172.21.15.33:0/3367132545 <== osd.2 v2:172.21.15.33:6818/115398 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff28400d150 con 0x7ff270068520 2023-11-06T20:44:13.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.209+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 >> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] conn(0x7ff270068520 msgr2=0x7ff27006a980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.209+0000 7ff2935ff700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] conn(0x7ff270068520 0x7ff27006a980 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.210+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27005f180 msgr2=0x7ff270061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.210+0000 7ff2935ff700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27005f180 0x7ff270061640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff28c0ca8a0 tx=0x7ff27c017000).stop 2023-11-06T20:44:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.210+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28c0632f0 msgr2=0x7ff28c0c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.210+0000 7ff2935ff700 1 --2- 172.21.15.33:0/3367132545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28c0632f0 0x7ff28c0c1ee0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7ff28800d890 tx=0x7ff288008040).stop 2023-11-06T20:44:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.210+0000 7ff291b9c700 1 -- 172.21.15.33:0/3367132545 reap_dead start 2023-11-06T20:44:13.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.210+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 shutdown_connections 2023-11-06T20:44:13.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.210+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 >> 172.21.15.33:0/3367132545 conn(0x7ff28c0cb910 msgr2=0x7ff28c155b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:13.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.210+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 shutdown_connections 2023-11-06T20:44:13.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.210+0000 7ff2935ff700 1 -- 172.21.15.33:0/3367132545 wait complete. 2023-11-06T20:44:13.225 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-11-06T20:44:13.225 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-06T20:44:13.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.422+0000 7ff70e3ff700 1 Processor -- start 2023-11-06T20:44:13.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.422+0000 7ff70e3ff700 1 -- start start 2023-11-06T20:44:13.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff70e3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7080c6d90 0x7ff7080c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff70e3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff7080bfd60 0x7ff7080c01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff70e3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7080c7700 con 0x7ff708154210 2023-11-06T20:44:13.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff70e3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7080c06e0 con 0x7ff7080bfd60 2023-11-06T20:44:13.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff70e3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70805efc0 con 0x7ff7080c6d90 2023-11-06T20:44:13.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff707fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff708154210 0x7ff7080a7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59740/0 (socket says 172.21.15.33:59740) 2023-11-06T20:44:13.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff707fff700 1 -- 172.21.15.33:0/4014917620 learned_addr learned my addr 172.21.15.33:0/4014917620 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:13.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff70c99c700 1 --2- 172.21.15.33:0/4014917620 >> v2:172.21.15.69:3300/0 conn(0x7ff7080bfd60 0x7ff7080c01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff7077fe700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7080c6d90 0x7ff7080c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895458752 0 0) 0x7ff6f8001110 con 0x7ff708154210 2023-11-06T20:44:13.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.424+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f00033f0 con 0x7ff708154210 2023-11-06T20:44:13.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1197681513 0 0) 0x7ff6f00033f0 con 0x7ff708154210 2023-11-06T20:44:13.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7080c6d90 msgr2=0x7ff7080c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:13.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7080c6d90 0x7ff7080c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 >> v2:172.21.15.69:3300/0 conn(0x7ff7080bfd60 msgr2=0x7ff7080c01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:13.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 --2- 172.21.15.33:0/4014917620 >> v2:172.21.15.69:3300/0 conn(0x7ff7080bfd60 0x7ff7080c01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7080c8ea0 con 0x7ff708154210 2023-11-06T20:44:13.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7ff6f8001600 con 0x7ff708154210 2023-11-06T20:44:13.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff70c99c700 1 --2- 172.21.15.33:0/4014917620 >> v2:172.21.15.69:3300/0 conn(0x7ff7080bfd60 0x7ff7080c01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:13.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 >> v1:172.21.15.33:6789/0 conn(0x7ff708154210 legacy=0x7ff7080a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7080c6d90 0x7ff6f00038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f0003e10 0x7ff6f0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6f0004b80 0x7ff6f0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7080c06e0 con 0x7ff6f0004b80 2023-11-06T20:44:13.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70805efc0 con 0x7ff6f0003e10 2023-11-06T20:44:13.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f000b8e0 con 0x7ff7080c6d90 2023-11-06T20:44:13.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff70c99c700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6f0004b80 0x7ff6f0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff7077fe700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f0003e10 0x7ff6f0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.425+0000 7ff707fff700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7080c6d90 0x7ff6f00038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.426+0000 7ff70c99c700 1 -- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7080c6d90 msgr2=0x7ff6f00038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:13.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.426+0000 7ff70c99c700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7080c6d90 0x7ff6f00038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.426+0000 7ff70c99c700 1 -- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f0003e10 msgr2=0x7ff6f0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:13.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.426+0000 7ff70c99c700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f0003e10 0x7ff6f0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.426+0000 7ff70c99c700 1 -- 172.21.15.33:0/4014917620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f00033f0 con 0x7ff6f0004b80 2023-11-06T20:44:13.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.426+0000 7ff70c99c700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6f0004b80 0x7ff6f0005000 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7ff70000e3f0 tx=0x7ff70000b040).ready entity=mon.0 client_cookie=7c1b20b3419515ef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.426+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700009c50 con 0x7ff6f0004b80 2023-11-06T20:44:13.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.426+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70000f040 con 0x7ff6f0004b80 2023-11-06T20:44:13.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.426+0000 7ff706ffd700 1 -- 172.21.15.33:0/4014917620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70000eeb0 con 0x7ff6f0004b80 2023-11-06T20:44:13.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.427+0000 7ff70e3ff700 1 -- 172.21.15.33:0/4014917620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6f0004b80 msgr2=0x7ff6f0005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.427+0000 7ff70e3ff700 1 --2- 172.21.15.33:0/4014917620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6f0004b80 0x7ff6f0005000 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7ff70000e3f0 tx=0x7ff70000b040).stop 2023-11-06T20:44:13.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.427+0000 7ff70c99c700 1 -- 172.21.15.33:0/4014917620 reap_dead start 2023-11-06T20:44:13.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.427+0000 7ff70e3ff700 1 -- 172.21.15.33:0/4014917620 shutdown_connections 2023-11-06T20:44:13.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.427+0000 7ff70e3ff700 1 -- 172.21.15.33:0/4014917620 >> 172.21.15.33:0/4014917620 conn(0x7ff70805e990 msgr2=0x7ff7080cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:13.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.427+0000 7ff70e3ff700 1 -- 172.21.15.33:0/4014917620 shutdown_connections 2023-11-06T20:44:13.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.427+0000 7ff70e3ff700 1 -- 172.21.15.33:0/4014917620 wait complete. 2023-11-06T20:44:13.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.428+0000 7ff70e3ff700 1 Processor -- start 2023-11-06T20:44:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.428+0000 7ff70e3ff700 1 -- start start 2023-11-06T20:44:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.429+0000 7ff70e3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6f0004b80 0x7ff70814f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.429+0000 7ff70e3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7080c37c0 0x7ff7080c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.429+0000 7ff70e3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7080b0310 0x7ff7080b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.429+0000 7ff70e3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7080c4130 con 0x7ff7080b0310 2023-11-06T20:44:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.429+0000 7ff70e3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7080c4f30 con 0x7ff7080c37c0 2023-11-06T20:44:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.429+0000 7ff70e3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7080b0c80 con 0x7ff6f0004b80 2023-11-06T20:44:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff70c99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7080b0310 0x7ff7080b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff70c99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7080b0310 0x7ff7080b0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42810/0 (socket says 172.21.15.33:42810) 2023-11-06T20:44:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff70c99c700 1 -- 172.21.15.33:0/1245111054 learned_addr learned my addr 172.21.15.33:0/1245111054 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff707fff700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6f0004b80 0x7ff70814f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff7077fe700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7080c37c0 0x7ff7080c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff70c99c700 1 -- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6f0004b80 msgr2=0x7ff70814f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff70c99c700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6f0004b80 0x7ff70814f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff70c99c700 1 -- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7080c37c0 msgr2=0x7ff7080c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff70c99c700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7080c37c0 0x7ff7080c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff70c99c700 1 -- 172.21.15.33:0/1245111054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7080c4320 con 0x7ff7080b0310 2023-11-06T20:44:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff7077fe700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7080c37c0 0x7ff7080c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.430+0000 7ff70c99c700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7080b0310 0x7ff7080b0790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ff6f00045a0 tx=0x7ff70001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.431+0000 7ff6effff700 1 -- 172.21.15.33:0/1245111054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700021030 con 0x7ff7080b0310 2023-11-06T20:44:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.431+0000 7ff6effff700 1 -- 172.21.15.33:0/1245111054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff700009c50 con 0x7ff7080b0310 2023-11-06T20:44:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.431+0000 7ff6effff700 1 -- 172.21.15.33:0/1245111054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7000093a0 con 0x7ff7080b0310 2023-11-06T20:44:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.431+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7080c4600 con 0x7ff7080b0310 2023-11-06T20:44:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.431+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7080c4bc0 con 0x7ff7080b0310 2023-11-06T20:44:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.432+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff708084250 con 0x7ff7080b0310 2023-11-06T20:44:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.433+0000 7ff6effff700 1 -- 172.21.15.33:0/1245111054 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff700024070 con 0x7ff7080b0310 2023-11-06T20:44:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.433+0000 7ff6effff700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff6f405f150 0x7ff6f4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.434+0000 7ff6effff700 1 -- 172.21.15.33:0/1245111054 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff700022070 con 0x7ff7080b0310 2023-11-06T20:44:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.434+0000 7ff6effff700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] conn(0x7ff6f40684f0 0x7ff6f406a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.434+0000 7ff6effff700 1 -- 172.21.15.33:0/1245111054 --> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6f406afd0 con 0x7ff6f40684f0 2023-11-06T20:44:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.434+0000 7ff6effff700 1 -- 172.21.15.33:0/1245111054 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70007dd40 con 0x7ff7080b0310 2023-11-06T20:44:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.434+0000 7ff707fff700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff6f405f150 0x7ff6f4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.434+0000 7ff7077fe700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] conn(0x7ff6f40684f0 0x7ff6f406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.434+0000 7ff707fff700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff6f405f150 0x7ff6f4061610 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff6f8001ed0 tx=0x7ff6f8007900).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.434+0000 7ff7077fe700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] conn(0x7ff6f40684f0 0x7ff6f406a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.435+0000 7ff6effff700 1 -- 172.21.15.33:0/1245111054 <== osd.3 v2:172.21.15.33:6816/115401 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7ff6f406afd0 con 0x7ff6f40684f0 2023-11-06T20:44:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.450+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 --> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff70806c6c0 con 0x7ff6f40684f0 2023-11-06T20:44:13.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.451+0000 7ff6effff700 1 -- 172.21.15.33:0/1245111054 <== osd.3 v2:172.21.15.33:6816/115401 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff70806c6c0 con 0x7ff6f40684f0 2023-11-06T20:44:13.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.451+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 >> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] conn(0x7ff6f40684f0 msgr2=0x7ff6f406a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.451+0000 7ff70e3ff700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] conn(0x7ff6f40684f0 0x7ff6f406a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.451+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff6f405f150 msgr2=0x7ff6f4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.451+0000 7ff70e3ff700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff6f405f150 0x7ff6f4061610 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff6f8001ed0 tx=0x7ff6f8007900).stop 2023-11-06T20:44:13.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.451+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7080b0310 msgr2=0x7ff7080b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.451+0000 7ff70e3ff700 1 --2- 172.21.15.33:0/1245111054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7080b0310 0x7ff7080b0790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ff6f00045a0 tx=0x7ff70001a000).stop 2023-11-06T20:44:13.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.451+0000 7ff70c99c700 1 -- 172.21.15.33:0/1245111054 reap_dead start 2023-11-06T20:44:13.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.451+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 shutdown_connections 2023-11-06T20:44:13.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.451+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 >> 172.21.15.33:0/1245111054 conn(0x7ff70805e990 msgr2=0x7ff7080cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:13.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.452+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 shutdown_connections 2023-11-06T20:44:13.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.452+0000 7ff70e3ff700 1 -- 172.21.15.33:0/1245111054 wait complete. 2023-11-06T20:44:13.467 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-11-06T20:44:13.467 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-06T20:44:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.664+0000 7f2fbf1ff700 1 Processor -- start 2023-11-06T20:44:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.664+0000 7f2fbf1ff700 1 -- start start 2023-11-06T20:44:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.665+0000 7f2fbf1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2fb80c0470 0x7f2fb80c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.665+0000 7f2fbf1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb80c7700 con 0x7f2fb8154210 2023-11-06T20:44:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.665+0000 7f2fbf1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb805efc0 con 0x7f2fb80c6d90 2023-11-06T20:44:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.665+0000 7f2fbf1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb80c8ea0 con 0x7f2fb80c0470 2023-11-06T20:44:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fbcf9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2fb8154210 0x7f2fb80a7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59750/0 (socket says 172.21.15.33:59750) 2023-11-06T20:44:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fbcf9b700 1 -- 172.21.15.33:0/392288283 learned_addr learned my addr 172.21.15.33:0/392288283 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609094247 0 0) 0x7f2fac001110 con 0x7f2fb8154210 2023-11-06T20:44:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa80033f0 con 0x7f2fb8154210 2023-11-06T20:44:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fbd79c700 1 --2- 172.21.15.33:0/392288283 >> v2:172.21.15.69:3300/0 conn(0x7f2fb80c0470 0x7f2fb80c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2185432345 0 0) 0x7f2fa80033f0 con 0x7f2fb8154210 2023-11-06T20:44:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 >> v2:172.21.15.69:3300/0 conn(0x7f2fb80c0470 msgr2=0x7f2fb80c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fb77fe700 1 --2- 172.21.15.33:0/392288283 >> v2:172.21.15.69:3300/0 conn(0x7f2fb80c0470 0x7f2fb80c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 >> v1:172.21.15.69:6789/0 conn(0x7f2fb80c6d90 legacy=0x7f2fb80bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb80c9140 con 0x7f2fb8154210 2023-11-06T20:44:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f2fac001600 con 0x7f2fb8154210 2023-11-06T20:44:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.666+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 >> v1:172.21.15.33:6789/0 conn(0x7f2fb8154210 legacy=0x7f2fb80a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb77fe700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fa80038e0 0x7f2fa8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb77fe700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 0x7f2fa8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb77fe700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fa8005000 0x7f2fa800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb80c8ea0 con 0x7f2fa8004690 2023-11-06T20:44:13.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa800beb0 con 0x7f2fa8005000 2023-11-06T20:44:13.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa800c070 con 0x7f2fa80038e0 2023-11-06T20:44:13.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb7fff700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 0x7f2fa8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fbcf9b700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fa80038e0 0x7f2fa8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fbd79c700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fa8005000 0x7f2fa800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb7fff700 1 -- 172.21.15.33:0/392288283 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fa80038e0 msgr2=0x7f2fa8003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:13.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb7fff700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fa80038e0 0x7f2fa8003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb7fff700 1 -- 172.21.15.33:0/392288283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fa8005000 msgr2=0x7f2fa800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:13.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb7fff700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fa8005000 0x7f2fa800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb7fff700 1 -- 172.21.15.33:0/392288283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa80033f0 con 0x7f2fa8004690 2023-11-06T20:44:13.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fbd79c700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fa8005000 0x7f2fa800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fbd79c700 1 -- 172.21.15.33:0/392288283 reap_dead start 2023-11-06T20:44:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.667+0000 7f2fb7fff700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 0x7f2fa8004b10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f2fa4001730 tx=0x7f2fa4007bb0).ready entity=mon.0 client_cookie=a57cfd6c4bd23c28 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.668+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4007020 con 0x7f2fa8004690 2023-11-06T20:44:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.668+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa4007180 con 0x7f2fa8004690 2023-11-06T20:44:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.668+0000 7f2fb77fe700 1 -- 172.21.15.33:0/392288283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4007520 con 0x7f2fa8004690 2023-11-06T20:44:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.668+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/392288283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 msgr2=0x7f2fa8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.668+0000 7f2fbf1ff700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 0x7f2fa8004b10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f2fa4001730 tx=0x7f2fa4007bb0).stop 2023-11-06T20:44:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.668+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/392288283 shutdown_connections 2023-11-06T20:44:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.668+0000 7f2fbf1ff700 1 --2- 172.21.15.33:0/392288283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 0x7f2fa8004b10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.668+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/392288283 >> 172.21.15.33:0/392288283 conn(0x7f2fb805e990 msgr2=0x7f2fb80cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.668+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/392288283 shutdown_connections 2023-11-06T20:44:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.668+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/392288283 wait complete. 2023-11-06T20:44:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.670+0000 7f2fbf1ff700 1 Processor -- start 2023-11-06T20:44:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.670+0000 7f2fbf1ff700 1 -- start start 2023-11-06T20:44:13.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbf1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 0x7f2fb80c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbf1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb80c29e0 0x7f2fb8164d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbf1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fb80c2e60 0x7f2fb8165230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbf1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb80c2550 con 0x7f2fa8004690 2023-11-06T20:44:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbf1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb80c1f20 con 0x7f2fb80c29e0 2023-11-06T20:44:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbf1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81658b0 con 0x7f2fb80c2e60 2023-11-06T20:44:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbcf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 0x7f2fb80c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbcf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 0x7f2fb80c48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42824/0 (socket says 172.21.15.33:42824) 2023-11-06T20:44:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbcf9b700 1 -- 172.21.15.33:0/4157355361 learned_addr learned my addr 172.21.15.33:0/4157355361 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbd79c700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fb80c2e60 0x7f2fb8165230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fb7fff700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb80c29e0 0x7f2fb8164d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbcf9b700 1 -- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fb80c2e60 msgr2=0x7f2fb8165230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbcf9b700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fb80c2e60 0x7f2fb8165230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbcf9b700 1 -- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb80c29e0 msgr2=0x7f2fb8164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbcf9b700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb80c29e0 0x7f2fb8164d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fbcf9b700 1 -- 172.21.15.33:0/4157355361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8165b90 con 0x7f2fa8004690 2023-11-06T20:44:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.671+0000 7f2fb7fff700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb80c29e0 0x7f2fb8164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.672+0000 7f2fbcf9b700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 0x7f2fb80c48c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2fac001e70 tx=0x7f2fac0078d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.672+0000 7f2fb4ff9700 1 -- 172.21.15.33:0/4157355361 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00f660 con 0x7f2fa8004690 2023-11-06T20:44:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.672+0000 7f2fb4ff9700 1 -- 172.21.15.33:0/4157355361 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac00f7c0 con 0x7f2fa8004690 2023-11-06T20:44:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.672+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb80c21e0 con 0x7f2fa8004690 2023-11-06T20:44:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.672+0000 7f2fb4ff9700 1 -- 172.21.15.33:0/4157355361 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00fb40 con 0x7f2fa8004690 2023-11-06T20:44:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.672+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb80c3910 con 0x7f2fa8004690 2023-11-06T20:44:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.673+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2fb806c6c0 con 0x7f2fa8004690 2023-11-06T20:44:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.674+0000 7f2fb4ff9700 1 -- 172.21.15.33:0/4157355361 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2fac01d3b0 con 0x7f2fa8004690 2023-11-06T20:44:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.675+0000 7f2fb4ff9700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f9c05f150 0x7f2f9c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.675+0000 7f2fb4ff9700 1 -- 172.21.15.33:0/4157355361 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2fac0799a0 con 0x7f2fa8004690 2023-11-06T20:44:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.675+0000 7f2fb4ff9700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] conn(0x7f2f9c0684f0 0x7f2f9c06a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.675+0000 7f2fb4ff9700 1 -- 172.21.15.33:0/4157355361 --> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f9c06afd0 con 0x7f2f9c0684f0 2023-11-06T20:44:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.675+0000 7f2fb4ff9700 1 -- 172.21.15.33:0/4157355361 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fac07e220 con 0x7f2fa8004690 2023-11-06T20:44:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.675+0000 7f2fb7fff700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f9c05f150 0x7f2f9c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.676+0000 7f2fbd79c700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] conn(0x7f2f9c0684f0 0x7f2f9c06a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.676+0000 7f2fb7fff700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f9c05f150 0x7f2f9c061610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f2fb80b0bf0 tx=0x7f2fa4015000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.676+0000 7f2fbd79c700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] conn(0x7f2f9c0684f0 0x7f2f9c06a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.677+0000 7f2fb4ff9700 1 -- 172.21.15.33:0/4157355361 <== osd.4 v2:172.21.15.69:6801/115620 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f2f9c06afd0 con 0x7f2f9c0684f0 2023-11-06T20:44:13.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.692+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 --> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2fb80c2690 con 0x7f2f9c0684f0 2023-11-06T20:44:13.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.693+0000 7f2fb4ff9700 1 -- 172.21.15.33:0/4157355361 <== osd.4 v2:172.21.15.69:6801/115620 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2fb000f040 con 0x7f2f9c0684f0 2023-11-06T20:44:13.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.693+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] conn(0x7f2f9c0684f0 msgr2=0x7f2f9c06a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.693+0000 7f2fbf1ff700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] conn(0x7f2f9c0684f0 0x7f2f9c06a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.694+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f9c05f150 msgr2=0x7f2f9c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.694+0000 7f2fbf1ff700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f9c05f150 0x7f2f9c061610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f2fb80b0bf0 tx=0x7f2fa4015000).stop 2023-11-06T20:44:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.694+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 msgr2=0x7f2fb80c48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.694+0000 7f2fbf1ff700 1 --2- 172.21.15.33:0/4157355361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa8004690 0x7f2fb80c48c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2fac001e70 tx=0x7f2fac0078d0).stop 2023-11-06T20:44:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.694+0000 7f2fbd79c700 1 -- 172.21.15.33:0/4157355361 reap_dead start 2023-11-06T20:44:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.694+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 shutdown_connections 2023-11-06T20:44:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.694+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 >> 172.21.15.33:0/4157355361 conn(0x7f2fb805e990 msgr2=0x7f2fb80631c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.694+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 shutdown_connections 2023-11-06T20:44:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.694+0000 7f2fbf1ff700 1 -- 172.21.15.33:0/4157355361 wait complete. 2023-11-06T20:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout:55834574852 2023-11-06T20:44:13.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-06T20:44:13.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.907+0000 7fc42bdff700 1 Processor -- start 2023-11-06T20:44:13.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.907+0000 7fc42bdff700 1 -- start start 2023-11-06T20:44:13.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.908+0000 7fc42bdff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc424154210 0x7fc4240a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.908+0000 7fc42bdff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4240bfd60 0x7fc4240c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.908+0000 7fc42bdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4240c9f00 0x7fc4240ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.908+0000 7fc42bdff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4240c06d0 con 0x7fc4240bfd60 2023-11-06T20:44:13.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.908+0000 7fc42bdff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc42405efc0 con 0x7fc424154210 2023-11-06T20:44:13.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.908+0000 7fc42bdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240c8ea0 con 0x7fc4240c9f00 2023-11-06T20:44:13.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.908+0000 7fc42939a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4240bfd60 0x7fc4240c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.908+0000 7fc42939a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4240bfd60 0x7fc4240c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42834/0 (socket says 172.21.15.33:42834) 2023-11-06T20:44:13.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.908+0000 7fc42939a700 1 -- 172.21.15.33:0/3470947753 learned_addr learned my addr 172.21.15.33:0/3470947753 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:13.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc429b9b700 1 --2- 172.21.15.33:0/3470947753 >> v2:172.21.15.69:3300/0 conn(0x7fc424154210 0x7fc4240a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc42a39c700 1 --2- 172.21.15.33:0/3470947753 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4240c9f00 0x7fc4240ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc42939a700 1 -- 172.21.15.33:0/3470947753 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4240c9f00 msgr2=0x7fc4240ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:13.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc42939a700 1 --2- 172.21.15.33:0/3470947753 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4240c9f00 0x7fc4240ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc42939a700 1 -- 172.21.15.33:0/3470947753 >> v2:172.21.15.69:3300/0 conn(0x7fc424154210 msgr2=0x7fc4240a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc42939a700 1 --2- 172.21.15.33:0/3470947753 >> v2:172.21.15.69:3300/0 conn(0x7fc424154210 0x7fc4240a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc42939a700 1 -- 172.21.15.33:0/3470947753 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240c9140 con 0x7fc4240bfd60 2023-11-06T20:44:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc42939a700 1 --2- 172.21.15.33:0/3470947753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240bfd60 0x7fc4240c01e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fc4240adab0 tx=0x7fc4140090f0).ready entity=mon.0 client_cookie=6de374dc288e148e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc428b99700 1 -- 172.21.15.33:0/3470947753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41400b600 con 0x7fc4240bfd60 2023-11-06T20:44:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc428b99700 1 -- 172.21.15.33:0/3470947753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41400b760 con 0x7fc4240bfd60 2023-11-06T20:44:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.909+0000 7fc428b99700 1 -- 172.21.15.33:0/3470947753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41400bae0 con 0x7fc4240bfd60 2023-11-06T20:44:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.910+0000 7fc42bdff700 1 -- 172.21.15.33:0/3470947753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240bfd60 msgr2=0x7fc4240c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.910+0000 7fc42bdff700 1 --2- 172.21.15.33:0/3470947753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240bfd60 0x7fc4240c01e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fc4240adab0 tx=0x7fc4140090f0).stop 2023-11-06T20:44:13.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.910+0000 7fc42bdff700 1 -- 172.21.15.33:0/3470947753 shutdown_connections 2023-11-06T20:44:13.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.910+0000 7fc42bdff700 1 --2- 172.21.15.33:0/3470947753 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4240c9f00 0x7fc4240ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.910+0000 7fc42bdff700 1 --2- 172.21.15.33:0/3470947753 >> v2:172.21.15.69:3300/0 conn(0x7fc424154210 0x7fc4240a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.910+0000 7fc42bdff700 1 --2- 172.21.15.33:0/3470947753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240bfd60 0x7fc4240c01e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.910+0000 7fc42bdff700 1 -- 172.21.15.33:0/3470947753 >> 172.21.15.33:0/3470947753 conn(0x7fc42405e990 msgr2=0x7fc4240cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:13.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.910+0000 7fc42bdff700 1 -- 172.21.15.33:0/3470947753 shutdown_connections 2023-11-06T20:44:13.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.910+0000 7fc42bdff700 1 -- 172.21.15.33:0/3470947753 wait complete. 2023-11-06T20:44:13.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.911+0000 7fc42bdff700 1 Processor -- start 2023-11-06T20:44:13.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.912+0000 7fc42bdff700 1 -- start start 2023-11-06T20:44:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.912+0000 7fc42bdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240bfd60 0x7fc4240ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.912+0000 7fc42bdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4240c9f00 0x7fc42414fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc42bdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc424154210 0x7fc424064050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc42bdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240ab740 con 0x7fc4240bfd60 2023-11-06T20:44:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc42bdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240aba20 con 0x7fc424154210 2023-11-06T20:44:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc42bdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240abd00 con 0x7fc4240c9f00 2023-11-06T20:44:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc429b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240bfd60 0x7fc4240ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc429b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240bfd60 0x7fc4240ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42836/0 (socket says 172.21.15.33:42836) 2023-11-06T20:44:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc429b9b700 1 -- 172.21.15.33:0/2015315378 learned_addr learned my addr 172.21.15.33:0/2015315378 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc42a39c700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc424154210 0x7fc424064050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc42939a700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4240c9f00 0x7fc42414fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc429b9b700 1 -- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4240c9f00 msgr2=0x7fc42414fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc429b9b700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4240c9f00 0x7fc42414fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc429b9b700 1 -- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc424154210 msgr2=0x7fc424064050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc429b9b700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc424154210 0x7fc424064050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc429b9b700 1 -- 172.21.15.33:0/2015315378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414009370 con 0x7fc4240bfd60 2023-11-06T20:44:13.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.913+0000 7fc429b9b700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240bfd60 0x7fc4240ad2b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fc42000d890 tx=0x7fc420008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.914+0000 7fc419ffb700 1 -- 172.21.15.33:0/2015315378 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420011590 con 0x7fc4240bfd60 2023-11-06T20:44:13.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.914+0000 7fc419ffb700 1 -- 172.21.15.33:0/2015315378 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4200116f0 con 0x7fc4240bfd60 2023-11-06T20:44:13.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.914+0000 7fc419ffb700 1 -- 172.21.15.33:0/2015315378 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420011a70 con 0x7fc4240bfd60 2023-11-06T20:44:13.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.914+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4240abfe0 con 0x7fc4240bfd60 2023-11-06T20:44:13.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.914+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4240ad7a0 con 0x7fc4240bfd60 2023-11-06T20:44:13.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.915+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc42406c6c0 con 0x7fc4240bfd60 2023-11-06T20:44:13.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.916+0000 7fc419ffb700 1 -- 172.21.15.33:0/2015315378 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc420012420 con 0x7fc4240bfd60 2023-11-06T20:44:13.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.916+0000 7fc419ffb700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc40805f150 0x7fc408061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.917+0000 7fc419ffb700 1 -- 172.21.15.33:0/2015315378 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc420079ba0 con 0x7fc4240bfd60 2023-11-06T20:44:13.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.917+0000 7fc42939a700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc40805f150 0x7fc408061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.917+0000 7fc419ffb700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] conn(0x7fc4080684f0 0x7fc40806a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:13.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.917+0000 7fc419ffb700 1 -- 172.21.15.33:0/2015315378 --> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc40806afd0 con 0x7fc4080684f0 2023-11-06T20:44:13.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.917+0000 7fc419ffb700 1 -- 172.21.15.33:0/2015315378 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc42007e220 con 0x7fc4240bfd60 2023-11-06T20:44:13.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.917+0000 7fc42a39c700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] conn(0x7fc4080684f0 0x7fc40806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:13.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.918+0000 7fc42939a700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc40805f150 0x7fc408061610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fc414009130 tx=0x7fc414017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.918+0000 7fc42a39c700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] conn(0x7fc4080684f0 0x7fc40806a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:13.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.919+0000 7fc419ffb700 1 -- 172.21.15.33:0/2015315378 <== osd.5 v2:172.21.15.69:6816/115642 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fc40806afd0 con 0x7fc4080684f0 2023-11-06T20:44:13.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.936+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 --> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc4240ac630 con 0x7fc4080684f0 2023-11-06T20:44:13.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc419ffb700 1 -- 172.21.15.33:0/2015315378 <== osd.5 v2:172.21.15.69:6816/115642 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc4240ac630 con 0x7fc4080684f0 2023-11-06T20:44:13.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] conn(0x7fc4080684f0 msgr2=0x7fc40806a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc42bdff700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] conn(0x7fc4080684f0 0x7fc40806a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:13.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc40805f150 msgr2=0x7fc408061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc42bdff700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc40805f150 0x7fc408061610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fc414009130 tx=0x7fc414017000).stop 2023-11-06T20:44:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240bfd60 msgr2=0x7fc4240ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc42bdff700 1 --2- 172.21.15.33:0/2015315378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240bfd60 0x7fc4240ad2b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fc42000d890 tx=0x7fc420008040).stop 2023-11-06T20:44:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc42a39c700 1 -- 172.21.15.33:0/2015315378 reap_dead start 2023-11-06T20:44:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 shutdown_connections 2023-11-06T20:44:13.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 >> 172.21.15.33:0/2015315378 conn(0x7fc42405e990 msgr2=0x7fc4240a7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:13.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.937+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 shutdown_connections 2023-11-06T20:44:13.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:13.938+0000 7fc42bdff700 1 -- 172.21.15.33:0/2015315378 wait complete. 2023-11-06T20:44:13.953 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2023-11-06T20:44:13.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-06T20:44:14.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.151+0000 7f20385ff700 1 Processor -- start 2023-11-06T20:44:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.151+0000 7f20385ff700 1 -- start start 2023-11-06T20:44:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.153+0000 7f20385ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2030154210 0x7f20300a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.153+0000 7f20385ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20300bfd60 0x7f20300c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.153+0000 7f20385ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20300c06d0 con 0x7f20300c9f00 2023-11-06T20:44:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.153+0000 7f20385ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20300caaa0 con 0x7f2030154210 2023-11-06T20:44:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.153+0000 7f20385ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203005efc0 con 0x7f20300bfd60 2023-11-06T20:44:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.153+0000 7f2035b9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20300bfd60 0x7f20300c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.153+0000 7f203639b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2030154210 0x7f20300a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.153+0000 7f2035b9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20300bfd60 0x7f20300c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:51426/0 (socket says 172.21.15.33:51426) 2023-11-06T20:44:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.153+0000 7f2035b9a700 1 -- 172.21.15.33:0/816589299 learned_addr learned my addr 172.21.15.33:0/816589299 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.153+0000 7f2036b9c700 1 --1- 172.21.15.33:0/816589299 >> v1:172.21.15.69:6789/0 conn(0x7f20300c9f00 0x7f20300ca390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:46968/0 (socket says 172.21.15.33:46968) 2023-11-06T20:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.154+0000 7f2035399700 1 -- 172.21.15.33:0/816589299 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134754623 0 0) 0x7f20300c06d0 con 0x7f20300c9f00 2023-11-06T20:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.154+0000 7f2035b9a700 1 -- 172.21.15.33:0/816589299 >> v2:172.21.15.69:3300/0 conn(0x7f2030154210 msgr2=0x7f20300a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.154+0000 7f2035b9a700 1 --2- 172.21.15.33:0/816589299 >> v2:172.21.15.69:3300/0 conn(0x7f2030154210 0x7f20300a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.154+0000 7f2035b9a700 1 -- 172.21.15.33:0/816589299 >> v1:172.21.15.69:6789/0 conn(0x7f20300c9f00 legacy=0x7f20300ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.154+0000 7f2035b9a700 1 -- 172.21.15.33:0/816589299 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20300c8ea0 con 0x7f20300bfd60 2023-11-06T20:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.154+0000 7f2035b9a700 1 --2- 172.21.15.33:0/816589299 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20300bfd60 0x7f20300c01e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f2020006fd0 tx=0x7f20200090f0).ready entity=mon.2 client_cookie=3264d474a9ddbe2a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.154+0000 7f2035399700 1 -- 172.21.15.33:0/816589299 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000b580 con 0x7f20300bfd60 2023-11-06T20:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.154+0000 7f2035399700 1 -- 172.21.15.33:0/816589299 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000b6e0 con 0x7f20300bfd60 2023-11-06T20:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.154+0000 7f2035399700 1 -- 172.21.15.33:0/816589299 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000ba60 con 0x7f20300bfd60 2023-11-06T20:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.155+0000 7f20385ff700 1 -- 172.21.15.33:0/816589299 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20300bfd60 msgr2=0x7f20300c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.155+0000 7f20385ff700 1 --2- 172.21.15.33:0/816589299 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20300bfd60 0x7f20300c01e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f2020006fd0 tx=0x7f20200090f0).stop 2023-11-06T20:44:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.155+0000 7f20385ff700 1 -- 172.21.15.33:0/816589299 shutdown_connections 2023-11-06T20:44:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.155+0000 7f20385ff700 1 --2- 172.21.15.33:0/816589299 >> v2:172.21.15.69:3300/0 conn(0x7f2030154210 0x7f20300a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.155+0000 7f20385ff700 1 --2- 172.21.15.33:0/816589299 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20300bfd60 0x7f20300c01e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.155+0000 7f20385ff700 1 -- 172.21.15.33:0/816589299 >> 172.21.15.33:0/816589299 conn(0x7f203005e990 msgr2=0x7f20300cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.155+0000 7f20385ff700 1 -- 172.21.15.33:0/816589299 shutdown_connections 2023-11-06T20:44:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.155+0000 7f20385ff700 1 -- 172.21.15.33:0/816589299 wait complete. 2023-11-06T20:44:14.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.156+0000 7f20385ff700 1 Processor -- start 2023-11-06T20:44:14.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.156+0000 7f20385ff700 1 -- start start 2023-11-06T20:44:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.157+0000 7f20385ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20300bfd60 0x7f20300af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.157+0000 7f20385ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20300c9f00 0x7f20301426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.157+0000 7f20385ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2030154210 0x7f2030064050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.157+0000 7f20385ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030167b80 con 0x7f20300bfd60 2023-11-06T20:44:14.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.157+0000 7f20385ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030167dd0 con 0x7f20300c9f00 2023-11-06T20:44:14.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.157+0000 7f20385ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20301680b0 con 0x7f2030154210 2023-11-06T20:44:14.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f203639b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20300bfd60 0x7f20300af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f203639b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20300bfd60 0x7f20300af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42838/0 (socket says 172.21.15.33:42838) 2023-11-06T20:44:14.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f203639b700 1 -- 172.21.15.33:0/1179059023 learned_addr learned my addr 172.21.15.33:0/1179059023 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f2035b9a700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20300c9f00 0x7f20301426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f2036b9c700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2030154210 0x7f2030064050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f203639b700 1 -- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2030154210 msgr2=0x7f2030064050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f203639b700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2030154210 0x7f2030064050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f203639b700 1 -- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20300c9f00 msgr2=0x7f20301426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f203639b700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20300c9f00 0x7f20301426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f203639b700 1 -- 172.21.15.33:0/1179059023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020009370 con 0x7f20300bfd60 2023-11-06T20:44:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f2035b9a700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20300c9f00 0x7f20301426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f2036b9c700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2030154210 0x7f2030064050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f203639b700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20300bfd60 0x7f20300af5b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f202c00e900 tx=0x7f202c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.158+0000 7f20267fc700 1 -- 172.21.15.33:0/1179059023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00c640 con 0x7f20300bfd60 2023-11-06T20:44:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.159+0000 7f20267fc700 1 -- 172.21.15.33:0/1179059023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c00c7a0 con 0x7f20300bfd60 2023-11-06T20:44:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.159+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2030168390 con 0x7f20300bfd60 2023-11-06T20:44:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.159+0000 7f20267fc700 1 -- 172.21.15.33:0/1179059023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00cb20 con 0x7f20300bfd60 2023-11-06T20:44:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.159+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2030168950 con 0x7f20300bfd60 2023-11-06T20:44:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.159+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f203006c6c0 con 0x7f20300bfd60 2023-11-06T20:44:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.160+0000 7f20267fc700 1 -- 172.21.15.33:0/1179059023 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f202c01b400 con 0x7f20300bfd60 2023-11-06T20:44:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.161+0000 7f20267fc700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f201405f150 0x7f2014061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.161+0000 7f20267fc700 1 -- 172.21.15.33:0/1179059023 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f202c07a700 con 0x7f20300bfd60 2023-11-06T20:44:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.162+0000 7f20267fc700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] conn(0x7f20140684f0 0x7f201406a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.162+0000 7f2035b9a700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f201405f150 0x7f2014061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.162+0000 7f20267fc700 1 -- 172.21.15.33:0/1179059023 --> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f201406afd0 con 0x7f20140684f0 2023-11-06T20:44:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.162+0000 7f20267fc700 1 -- 172.21.15.33:0/1179059023 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f202c07cc60 con 0x7f20300bfd60 2023-11-06T20:44:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.162+0000 7f2036b9c700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] conn(0x7f20140684f0 0x7f201406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.162+0000 7f2035b9a700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f201405f150 0x7f2014061610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f20300afac0 tx=0x7f2020017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.162+0000 7f2036b9c700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] conn(0x7f20140684f0 0x7f201406a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.163+0000 7f20267fc700 1 -- 172.21.15.33:0/1179059023 <== osd.6 v2:172.21.15.69:6800/115621 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f201406afd0 con 0x7f20140684f0 2023-11-06T20:44:14.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.178+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 --> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2030164620 con 0x7f20140684f0 2023-11-06T20:44:14.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.179+0000 7f20267fc700 1 -- 172.21.15.33:0/1179059023 <== osd.6 v2:172.21.15.69:6800/115621 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2030164620 con 0x7f20140684f0 2023-11-06T20:44:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.179+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] conn(0x7f20140684f0 msgr2=0x7f201406a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.179+0000 7f20385ff700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] conn(0x7f20140684f0 0x7f201406a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.179+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f201405f150 msgr2=0x7f2014061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.179+0000 7f20385ff700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f201405f150 0x7f2014061610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f20300afac0 tx=0x7f2020017000).stop 2023-11-06T20:44:14.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.179+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20300bfd60 msgr2=0x7f20300af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.179+0000 7f20385ff700 1 --2- 172.21.15.33:0/1179059023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20300bfd60 0x7f20300af5b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f202c00e900 tx=0x7f202c009060).stop 2023-11-06T20:44:14.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.179+0000 7f2036b9c700 1 -- 172.21.15.33:0/1179059023 reap_dead start 2023-11-06T20:44:14.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.179+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 shutdown_connections 2023-11-06T20:44:14.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.180+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 >> 172.21.15.33:0/1179059023 conn(0x7f203005e990 msgr2=0x7f20300c6470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:14.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.180+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 shutdown_connections 2023-11-06T20:44:14.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.180+0000 7f20385ff700 1 -- 172.21.15.33:0/1179059023 wait complete. 2023-11-06T20:44:14.195 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-11-06T20:44:14.195 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-06T20:44:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.392+0000 7fb80a1ff700 1 Processor -- start 2023-11-06T20:44:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.392+0000 7fb80a1ff700 1 -- start start 2023-11-06T20:44:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb80a1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb8040c0470 0x7fb8040c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb80a1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8040c7700 con 0x7fb8040c6d90 2023-11-06T20:44:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb80a1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80405efc0 con 0x7fb804154210 2023-11-06T20:44:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb80a1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8040c8ea0 con 0x7fb8040c0470 2023-11-06T20:44:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb802ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb8040c6d90 0x7fb8040bfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59758/0 (socket says 172.21.15.33:59758) 2023-11-06T20:44:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb802ffd700 1 -- 172.21.15.33:0/1571891582 learned_addr learned my addr 172.21.15.33:0/1571891582 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb803fff700 1 --2- 172.21.15.33:0/1571891582 >> v2:172.21.15.69:3300/0 conn(0x7fb8040c0470 0x7fb8040c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63764870 0 0) 0x7fb7f8001110 con 0x7fb8040c6d90 2023-11-06T20:44:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7ec0033f0 con 0x7fb8040c6d90 2023-11-06T20:44:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849291647 0 0) 0x7fb80405efc0 con 0x7fb804154210 2023-11-06T20:44:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.394+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f8001110 con 0x7fb804154210 2023-11-06T20:44:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2677910513 0 0) 0x7fb7ec0033f0 con 0x7fb8040c6d90 2023-11-06T20:44:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 >> v2:172.21.15.69:3300/0 conn(0x7fb8040c0470 msgr2=0x7fb8040c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 --2- 172.21.15.33:0/1571891582 >> v2:172.21.15.69:3300/0 conn(0x7fb8040c0470 0x7fb8040c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 >> v1:172.21.15.69:6789/0 conn(0x7fb804154210 legacy=0x7fb8040a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040c9140 con 0x7fb8040c6d90 2023-11-06T20:44:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fb7f8001600 con 0x7fb8040c6d90 2023-11-06T20:44:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 >> v1:172.21.15.33:6789/0 conn(0x7fb8040c6d90 legacy=0x7fb8040bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb7ec003a70 0x7fb7ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 0x7fb7ec004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7ec005000 0x7fb7ec00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040c8ea0 con 0x7fb7ec004690 2023-11-06T20:44:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec00be80 con 0x7fb7ec005000 2023-11-06T20:44:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec00c040 con 0x7fb7ec003a70 2023-11-06T20:44:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb803fff700 1 --2- 172.21.15.33:0/1571891582 >> v2:172.21.15.69:3300/0 conn(0x7fb8040c0470 0x7fb8040c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb802ffd700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 0x7fb7ec004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb8037fe700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb7ec003a70 0x7fb7ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb803fff700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7ec005000 0x7fb7ec00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb802ffd700 1 -- 172.21.15.33:0/1571891582 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb7ec003a70 msgr2=0x7fb7ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb802ffd700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb7ec003a70 0x7fb7ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb802ffd700 1 -- 172.21.15.33:0/1571891582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7ec005000 msgr2=0x7fb7ec00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb802ffd700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7ec005000 0x7fb7ec00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb802ffd700 1 -- 172.21.15.33:0/1571891582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec0033f0 con 0x7fb7ec004690 2023-11-06T20:44:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb803fff700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7ec005000 0x7fb7ec00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.395+0000 7fb803fff700 1 -- 172.21.15.33:0/1571891582 reap_dead start 2023-11-06T20:44:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.396+0000 7fb802ffd700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 0x7fb7ec004b10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fb7f8007fd0 tx=0x7fb7f8007880).ready entity=mon.0 client_cookie=7fb0c98160394f28 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.396+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8007b50 con 0x7fb7ec004690 2023-11-06T20:44:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.396+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f800cba0 con 0x7fb7ec004690 2023-11-06T20:44:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.396+0000 7fb8027fc700 1 -- 172.21.15.33:0/1571891582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80070d0 con 0x7fb7ec004690 2023-11-06T20:44:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.396+0000 7fb80a1ff700 1 -- 172.21.15.33:0/1571891582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 msgr2=0x7fb7ec004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.396+0000 7fb80a1ff700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 0x7fb7ec004b10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fb7f8007fd0 tx=0x7fb7f8007880).stop 2023-11-06T20:44:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.396+0000 7fb80a1ff700 1 -- 172.21.15.33:0/1571891582 shutdown_connections 2023-11-06T20:44:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.396+0000 7fb80a1ff700 1 --2- 172.21.15.33:0/1571891582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 0x7fb7ec004b10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.396+0000 7fb80a1ff700 1 -- 172.21.15.33:0/1571891582 >> 172.21.15.33:0/1571891582 conn(0x7fb80405e990 msgr2=0x7fb8040cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.397+0000 7fb80a1ff700 1 -- 172.21.15.33:0/1571891582 shutdown_connections 2023-11-06T20:44:14.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.397+0000 7fb80a1ff700 1 -- 172.21.15.33:0/1571891582 wait complete. 2023-11-06T20:44:14.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.398+0000 7fb80a1ff700 1 Processor -- start 2023-11-06T20:44:14.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.398+0000 7fb80a1ff700 1 -- start start 2023-11-06T20:44:14.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.399+0000 7fb80a1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 0x7fb8040b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.399+0000 7fb80a1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80414cf90 0x7fb80414d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.399+0000 7fb80a1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb80414d900 0x7fb8041683e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.399+0000 7fb80a1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040c4c00 con 0x7fb7ec004690 2023-11-06T20:44:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.399+0000 7fb80a1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040c5090 con 0x7fb80414cf90 2023-11-06T20:44:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.399+0000 7fb80a1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80414de40 con 0x7fb80414d900 2023-11-06T20:44:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb8037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 0x7fb8040b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb8037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 0x7fb8040b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42856/0 (socket says 172.21.15.33:42856) 2023-11-06T20:44:14.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb8037fe700 1 -- 172.21.15.33:0/12787248 learned_addr learned my addr 172.21.15.33:0/12787248 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:14.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb803fff700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb80414d900 0x7fb8041683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb802ffd700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80414cf90 0x7fb80414d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb8037fe700 1 -- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb80414d900 msgr2=0x7fb8041683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb8037fe700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb80414d900 0x7fb8041683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb8037fe700 1 -- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80414cf90 msgr2=0x7fb80414d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb8037fe700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb80414cf90 0x7fb80414d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb8037fe700 1 -- 172.21.15.33:0/12787248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80414e120 con 0x7fb7ec004690 2023-11-06T20:44:14.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb803fff700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb80414d900 0x7fb8041683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:14.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.400+0000 7fb8037fe700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 0x7fb8040b06d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb7f4002a70 tx=0x7fb7f4007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.401+0000 7fb7eb7fe700 1 -- 172.21.15.33:0/12787248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400f8d0 con 0x7fb7ec004690 2023-11-06T20:44:14.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.401+0000 7fb7eb7fe700 1 -- 172.21.15.33:0/12787248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f400fa30 con 0x7fb7ec004690 2023-11-06T20:44:14.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.401+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8040c4890 con 0x7fb7ec004690 2023-11-06T20:44:14.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.401+0000 7fb7eb7fe700 1 -- 172.21.15.33:0/12787248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400fdb0 con 0x7fb7ec004690 2023-11-06T20:44:14.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.401+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8040b0d80 con 0x7fb7ec004690 2023-11-06T20:44:14.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.402+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb80406c6c0 con 0x7fb7ec004690 2023-11-06T20:44:14.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.403+0000 7fb7eb7fe700 1 -- 172.21.15.33:0/12787248 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb7f401d3b0 con 0x7fb7ec004690 2023-11-06T20:44:14.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.403+0000 7fb7eb7fe700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb7f005f150 0x7fb7f0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.403+0000 7fb7eb7fe700 1 -- 172.21.15.33:0/12787248 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb7f407a2a0 con 0x7fb7ec004690 2023-11-06T20:44:14.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.404+0000 7fb802ffd700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb7f005f150 0x7fb7f0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.404+0000 7fb7eb7fe700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] conn(0x7fb7f00684f0 0x7fb7f006a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.404+0000 7fb7eb7fe700 1 -- 172.21.15.33:0/12787248 --> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7f006afd0 con 0x7fb7f00684f0 2023-11-06T20:44:14.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.404+0000 7fb7eb7fe700 1 -- 172.21.15.33:0/12787248 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7f407c800 con 0x7fb7ec004690 2023-11-06T20:44:14.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.404+0000 7fb803fff700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] conn(0x7fb7f00684f0 0x7fb7f006a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.404+0000 7fb802ffd700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb7f005f150 0x7fb7f0061610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fb7f800cb70 tx=0x7fb7f8015000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.405+0000 7fb803fff700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] conn(0x7fb7f00684f0 0x7fb7f006a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.406+0000 7fb7eb7fe700 1 -- 172.21.15.33:0/12787248 <== osd.7 v2:172.21.15.69:6821/115651 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fb7f006afd0 con 0x7fb7f00684f0 2023-11-06T20:44:14.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.423+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 --> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb8040c4ee0 con 0x7fb7f00684f0 2023-11-06T20:44:14.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.423+0000 7fb7eb7fe700 1 -- 172.21.15.33:0/12787248 <== osd.7 v2:172.21.15.69:6821/115651 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb8040c4ee0 con 0x7fb7f00684f0 2023-11-06T20:44:14.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] conn(0x7fb7f00684f0 msgr2=0x7fb7f006a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb80a1ff700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] conn(0x7fb7f00684f0 0x7fb7f006a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb7f005f150 msgr2=0x7fb7f0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb80a1ff700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb7f005f150 0x7fb7f0061610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fb7f800cb70 tx=0x7fb7f8015000).stop 2023-11-06T20:44:14.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 msgr2=0x7fb8040b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb80a1ff700 1 --2- 172.21.15.33:0/12787248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7ec004690 0x7fb8040b06d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb7f4002a70 tx=0x7fb7f4007b90).stop 2023-11-06T20:44:14.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb803fff700 1 -- 172.21.15.33:0/12787248 reap_dead start 2023-11-06T20:44:14.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 shutdown_connections 2023-11-06T20:44:14.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 >> 172.21.15.33:0/12787248 conn(0x7fb80405e990 msgr2=0x7fb804152700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:14.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 shutdown_connections 2023-11-06T20:44:14.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.424+0000 7fb80a1ff700 1 -- 172.21.15.33:0/12787248 wait complete. 2023-11-06T20:44:14.440 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-11-06T20:44:14.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-06T20:44:14.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.638+0000 7ff43d9ff700 1 Processor -- start 2023-11-06T20:44:14.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.638+0000 7ff43d9ff700 1 -- start start 2023-11-06T20:44:14.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.639+0000 7ff43d9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4381373a0 0x7ff438137780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.639+0000 7ff43d9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4380c3990 0x7ff438069440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff43d9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43806a470 con 0x7ff4380c1020 2023-11-06T20:44:14.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff43d9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43806a190 con 0x7ff4381373a0 2023-11-06T20:44:14.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff43d9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4380bbab0 con 0x7ff4380c3990 2023-11-06T20:44:14.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff436ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4381373a0 0x7ff438137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff436ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4381373a0 0x7ff438137780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42870/0 (socket says 172.21.15.33:42870) 2023-11-06T20:44:14.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff436ffd700 1 -- 172.21.15.33:0/171267775 learned_addr learned my addr 172.21.15.33:0/171267775 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:14.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff4367fc700 1 --1- 172.21.15.33:0/171267775 >> v1:172.21.15.33:6789/0 conn(0x7ff4380c1020 0x7ff4380c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59766/0 (socket says 172.21.15.33:59766) 2023-11-06T20:44:14.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff4377fe700 1 --2- 172.21.15.33:0/171267775 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4380c3990 0x7ff438069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff436ffd700 1 -- 172.21.15.33:0/171267775 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4380c3990 msgr2=0x7ff438069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:14.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff436ffd700 1 --2- 172.21.15.33:0/171267775 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4380c3990 0x7ff438069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff436ffd700 1 -- 172.21.15.33:0/171267775 >> v1:172.21.15.33:6789/0 conn(0x7ff4380c1020 legacy=0x7ff4380c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff436ffd700 1 -- 172.21.15.33:0/171267775 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4380bbd90 con 0x7ff4381373a0 2023-11-06T20:44:14.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff435ffb700 1 -- 172.21.15.33:0/171267775 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3670155859 0 0) 0x7ff43806a470 con 0x7ff4380c1020 2023-11-06T20:44:14.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.640+0000 7ff436ffd700 1 --2- 172.21.15.33:0/171267775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4381373a0 0x7ff438137780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7ff428006fd0 tx=0x7ff4280090f0).ready entity=mon.0 client_cookie=67f5966438f457c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff435ffb700 1 -- 172.21.15.33:0/171267775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42800b600 con 0x7ff4381373a0 2023-11-06T20:44:14.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff435ffb700 1 -- 172.21.15.33:0/171267775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42800b760 con 0x7ff4381373a0 2023-11-06T20:44:14.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff435ffb700 1 -- 172.21.15.33:0/171267775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42800bae0 con 0x7ff4381373a0 2023-11-06T20:44:14.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff43d9ff700 1 -- 172.21.15.33:0/171267775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4381373a0 msgr2=0x7ff438137780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff43d9ff700 1 --2- 172.21.15.33:0/171267775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4381373a0 0x7ff438137780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7ff428006fd0 tx=0x7ff4280090f0).stop 2023-11-06T20:44:14.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff43d9ff700 1 -- 172.21.15.33:0/171267775 shutdown_connections 2023-11-06T20:44:14.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff43d9ff700 1 --2- 172.21.15.33:0/171267775 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4380c3990 0x7ff438069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff43d9ff700 1 --2- 172.21.15.33:0/171267775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4381373a0 0x7ff438137780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff43d9ff700 1 -- 172.21.15.33:0/171267775 >> 172.21.15.33:0/171267775 conn(0x7ff438062710 msgr2=0x7ff438062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:14.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff43d9ff700 1 -- 172.21.15.33:0/171267775 shutdown_connections 2023-11-06T20:44:14.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.641+0000 7ff43d9ff700 1 -- 172.21.15.33:0/171267775 wait complete. 2023-11-06T20:44:14.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.643+0000 7ff43d9ff700 1 Processor -- start 2023-11-06T20:44:14.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.643+0000 7ff43d9ff700 1 -- start start 2023-11-06T20:44:14.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff43d9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4380c1020 0x7ff438150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff43d9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4380c3990 0x7ff4381665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff43d9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4381373a0 0x7ff43814e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff43d9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4380a9a10 con 0x7ff4380c1020 2023-11-06T20:44:14.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff43d9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff438151300 con 0x7ff4381373a0 2023-11-06T20:44:14.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff43d9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff438165930 con 0x7ff4380c3990 2023-11-06T20:44:14.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff436ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4380c1020 0x7ff438150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff436ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4380c1020 0x7ff438150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42880/0 (socket says 172.21.15.33:42880) 2023-11-06T20:44:14.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff436ffd700 1 -- 172.21.15.33:0/1191630377 learned_addr learned my addr 172.21.15.33:0/1191630377 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:14.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff4367fc700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4380c3990 0x7ff4381665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.644+0000 7ff4377fe700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4381373a0 0x7ff43814e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff436ffd700 1 -- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4380c3990 msgr2=0x7ff4381665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff436ffd700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4380c3990 0x7ff4381665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff436ffd700 1 -- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4381373a0 msgr2=0x7ff43814e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff436ffd700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4381373a0 0x7ff43814e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff436ffd700 1 -- 172.21.15.33:0/1191630377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff428009370 con 0x7ff4380c1020 2023-11-06T20:44:14.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff4377fe700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4381373a0 0x7ff43814e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:14.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff436ffd700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4380c1020 0x7ff438150e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7ff43805efa0 tx=0x7ff428017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff41effd700 1 -- 172.21.15.33:0/1191630377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42800b8d0 con 0x7ff4380c1020 2023-11-06T20:44:14.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff41effd700 1 -- 172.21.15.33:0/1191630377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42800ba30 con 0x7ff4380c1020 2023-11-06T20:44:14.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff41effd700 1 -- 172.21.15.33:0/1191630377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42801d680 con 0x7ff4380c1020 2023-11-06T20:44:14.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff43d9ff700 1 -- 172.21.15.33:0/1191630377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4380aa270 con 0x7ff4380c1020 2023-11-06T20:44:14.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.645+0000 7ff43d9ff700 1 -- 172.21.15.33:0/1191630377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff438166aa0 con 0x7ff4380c1020 2023-11-06T20:44:14.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.646+0000 7ff43d9ff700 1 -- 172.21.15.33:0/1191630377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff438152560 con 0x7ff4380c1020 2023-11-06T20:44:14.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.647+0000 7ff41effd700 1 -- 172.21.15.33:0/1191630377 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff42801da30 con 0x7ff4380c1020 2023-11-06T20:44:14.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.648+0000 7ff41effd700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff42405f150 0x7ff424061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:14.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.648+0000 7ff41effd700 1 -- 172.21.15.33:0/1191630377 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff42807c620 con 0x7ff4380c1020 2023-11-06T20:44:14.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.648+0000 7ff4367fc700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff42405f150 0x7ff424061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:14.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.649+0000 7ff4367fc700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff42405f150 0x7ff424061610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7ff4380abaf0 tx=0x7ff42c007c90).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:14.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.652+0000 7ff41effd700 1 -- 172.21.15.33:0/1191630377 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff42804d830 con 0x7ff4380c1020 2023-11-06T20:44:14.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.833+0000 7ff43d9ff700 1 -- 172.21.15.33:0/1191630377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff438065a00 con 0x7ff4380c1020 2023-11-06T20:44:14.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.833+0000 7ff41effd700 1 -- 172.21.15.33:0/1191630377 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff428021020 con 0x7ff4380c1020 2023-11-06T20:44:14.833 INFO:teuthology.orchestra.run.smithi033.stdout:55834574852 2023-11-06T20:44:14.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 -- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff42405f150 msgr2=0x7ff424061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff42405f150 0x7ff424061610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7ff4380abaf0 tx=0x7ff42c007c90).stop 2023-11-06T20:44:14.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 -- 172.21.15.33:0/1191630377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4380c1020 msgr2=0x7ff438150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:14.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4380c1020 0x7ff438150e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7ff43805efa0 tx=0x7ff428017000).stop 2023-11-06T20:44:14.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 -- 172.21.15.33:0/1191630377 shutdown_connections 2023-11-06T20:44:14.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff42405f150 0x7ff424061610 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4381373a0 0x7ff43814e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4380c1020 0x7ff438150e10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 --2- 172.21.15.33:0/1191630377 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4380c3990 0x7ff4381665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:14.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 -- 172.21.15.33:0/1191630377 >> 172.21.15.33:0/1191630377 conn(0x7ff438062710 msgr2=0x7ff438136690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:14.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.835+0000 7ff43d9ff700 1 -- 172.21.15.33:0/1191630377 shutdown_connections 2023-11-06T20:44:14.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:14.836+0000 7ff43d9ff700 1 -- 172.21.15.33:0/1191630377 wait complete. 2023-11-06T20:44:14.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2023-11-06T20:44:14.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-06T20:44:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.051+0000 7f7b7f5ff700 1 Processor -- start 2023-11-06T20:44:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.051+0000 7f7b7f5ff700 1 -- start start 2023-11-06T20:44:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.052+0000 7f7b7f5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7b78154210 0x7f7b780a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.052+0000 7f7b7f5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780bfd60 0x7f7b780c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.052+0000 7f7b7f5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b780c06d0 con 0x7f7b780c9f00 2023-11-06T20:44:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7f5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780caaa0 con 0x7f7b78154210 2023-11-06T20:44:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7f5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7805efc0 con 0x7f7b780bfd60 2023-11-06T20:44:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7cb9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780bfd60 0x7f7b780c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7d39b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7b78154210 0x7f7b780a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7cb9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780bfd60 0x7f7b780c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:51482/0 (socket says 172.21.15.33:51482) 2023-11-06T20:44:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7cb9a700 1 -- 172.21.15.33:0/2802034499 learned_addr learned my addr 172.21.15.33:0/2802034499 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7d39b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7b78154210 0x7f7b780a7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52086/0 (socket says 172.21.15.33:52086) 2023-11-06T20:44:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7d39b700 1 -- 172.21.15.33:0/2802034499 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780bfd60 msgr2=0x7f7b780c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7d39b700 1 --2- 172.21.15.33:0/2802034499 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780bfd60 0x7f7b780c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7d39b700 1 -- 172.21.15.33:0/2802034499 >> v1:172.21.15.69:6789/0 conn(0x7f7b780c9f00 legacy=0x7f7b780ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b7d39b700 1 -- 172.21.15.33:0/2802034499 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b780c8ea0 con 0x7f7b78154210 2023-11-06T20:44:15.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.053+0000 7f7b6ffff700 1 -- 172.21.15.33:0/2802034499 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934301321 0 0) 0x7f7b780c06d0 con 0x7f7b780c9f00 2023-11-06T20:44:15.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.054+0000 7f7b7cb9a700 1 --2- 172.21.15.33:0/2802034499 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780bfd60 0x7f7b780c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:15.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.054+0000 7f7b7d39b700 1 --2- 172.21.15.33:0/2802034499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b78154210 0x7f7b780a7bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f7b74009760 tx=0x7f7b74009060).ready entity=mon.1 client_cookie=f6ea5451c8e6251d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:15.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.054+0000 7f7b6ffff700 1 -- 172.21.15.33:0/2802034499 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7400f510 con 0x7f7b78154210 2023-11-06T20:44:15.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.054+0000 7f7b6ffff700 1 -- 172.21.15.33:0/2802034499 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7400f670 con 0x7f7b78154210 2023-11-06T20:44:15.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.054+0000 7f7b6ffff700 1 -- 172.21.15.33:0/2802034499 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7400f9f0 con 0x7f7b78154210 2023-11-06T20:44:15.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.054+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/2802034499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b78154210 msgr2=0x7f7b780a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.055+0000 7f7b7f5ff700 1 --2- 172.21.15.33:0/2802034499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b78154210 0x7f7b780a7bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f7b74009760 tx=0x7f7b74009060).stop 2023-11-06T20:44:15.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.055+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/2802034499 shutdown_connections 2023-11-06T20:44:15.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.055+0000 7f7b7f5ff700 1 --2- 172.21.15.33:0/2802034499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b78154210 0x7f7b780a7bd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.055+0000 7f7b7f5ff700 1 --2- 172.21.15.33:0/2802034499 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780bfd60 0x7f7b780c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.055+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/2802034499 >> 172.21.15.33:0/2802034499 conn(0x7f7b7805e990 msgr2=0x7f7b780cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:15.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.055+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/2802034499 shutdown_connections 2023-11-06T20:44:15.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.055+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/2802034499 wait complete. 2023-11-06T20:44:15.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.056+0000 7f7b7f5ff700 1 Processor -- start 2023-11-06T20:44:15.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.056+0000 7f7b7f5ff700 1 -- start start 2023-11-06T20:44:15.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.057+0000 7f7b7f5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b780bfd60 0x7f7b780c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.057+0000 7f7b7f5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780c9f00 0x7f7b780b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.057+0000 7f7b7f5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b78154210 0x7f7b780c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.057+0000 7f7b7f5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78168110 con 0x7f7b780bfd60 2023-11-06T20:44:15.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.057+0000 7f7b7f5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78165040 con 0x7f7b78154210 2023-11-06T20:44:15.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.057+0000 7f7b7f5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78165320 con 0x7f7b780c9f00 2023-11-06T20:44:15.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7d39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b780bfd60 0x7f7b780c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7d39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b780bfd60 0x7f7b780c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42886/0 (socket says 172.21.15.33:42886) 2023-11-06T20:44:15.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7d39b700 1 -- 172.21.15.33:0/1948375122 learned_addr learned my addr 172.21.15.33:0/1948375122 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:15.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7cb9a700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780c9f00 0x7f7b780b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7db9c700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b78154210 0x7f7b780c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7d39b700 1 -- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780c9f00 msgr2=0x7f7b780b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7d39b700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780c9f00 0x7f7b780b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7d39b700 1 -- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b78154210 msgr2=0x7f7b780c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7d39b700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b78154210 0x7f7b780c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7d39b700 1 -- 172.21.15.33:0/1948375122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b68009370 con 0x7f7b780bfd60 2023-11-06T20:44:15.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.058+0000 7f7b7d39b700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b780bfd60 0x7f7b780c4130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f7b74009e00 tx=0x7f7b74017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:15.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.059+0000 7f7b6d7fa700 1 -- 172.21.15.33:0/1948375122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7400f7e0 con 0x7f7b780bfd60 2023-11-06T20:44:15.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.059+0000 7f7b6d7fa700 1 -- 172.21.15.33:0/1948375122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7400f940 con 0x7f7b780bfd60 2023-11-06T20:44:15.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.059+0000 7f7b6d7fa700 1 -- 172.21.15.33:0/1948375122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7401a650 con 0x7f7b780bfd60 2023-11-06T20:44:15.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.059+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/1948375122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b740092a0 con 0x7f7b780bfd60 2023-11-06T20:44:15.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.059+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/1948375122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b781658e0 con 0x7f7b780bfd60 2023-11-06T20:44:15.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.060+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/1948375122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b781683f0 con 0x7f7b780bfd60 2023-11-06T20:44:15.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.061+0000 7f7b6d7fa700 1 -- 172.21.15.33:0/1948375122 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7b7401a9c0 con 0x7f7b780bfd60 2023-11-06T20:44:15.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.062+0000 7f7b6d7fa700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f150 0x7f7b5c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.062+0000 7f7b6d7fa700 1 -- 172.21.15.33:0/1948375122 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7b7407b500 con 0x7f7b780bfd60 2023-11-06T20:44:15.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.062+0000 7f7b7cb9a700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f150 0x7f7b5c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.063+0000 7f7b7cb9a700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f150 0x7f7b5c061610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f7b68009340 tx=0x7f7b68008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:15.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.065+0000 7f7b6d7fa700 1 -- 172.21.15.33:0/1948375122 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7b7404c710 con 0x7f7b780bfd60 2023-11-06T20:44:15.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.244+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/1948375122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7b7806c6c0 con 0x7f7b780bfd60 2023-11-06T20:44:15.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.245+0000 7f7b6d7fa700 1 -- 172.21.15.33:0/1948375122 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b7401f090 con 0x7f7b780bfd60 2023-11-06T20:44:15.244 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-11-06T20:44:15.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f150 msgr2=0x7f7b5c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f150 0x7f7b5c061610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f7b68009340 tx=0x7f7b68008040).stop 2023-11-06T20:44:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/1948375122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b780bfd60 msgr2=0x7f7b780c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b780bfd60 0x7f7b780c4130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f7b74009e00 tx=0x7f7b74017000).stop 2023-11-06T20:44:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/1948375122 shutdown_connections 2023-11-06T20:44:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f150 0x7f7b5c061610 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b78154210 0x7f7b780c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b780bfd60 0x7f7b780c4130 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 --2- 172.21.15.33:0/1948375122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b780c9f00 0x7f7b780b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/1948375122 >> 172.21.15.33:0/1948375122 conn(0x7f7b7805e990 msgr2=0x7f7b78151140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/1948375122 shutdown_connections 2023-11-06T20:44:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.247+0000 7f7b7f5ff700 1 -- 172.21.15.33:0/1948375122 wait complete. 2023-11-06T20:44:15.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2023-11-06T20:44:15.264 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-06T20:44:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.461+0000 7fe3c53ff700 1 Processor -- start 2023-11-06T20:44:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.461+0000 7fe3c53ff700 1 -- start start 2023-11-06T20:44:15.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.462+0000 7fe3c53ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe3c0154210 0x7fe3c00a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.462+0000 7fe3c53ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe3c00ca610 0x7fe3c00c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3c53ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c00c06d0 con 0x7fe3c00bfd60 2023-11-06T20:44:15.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3c53ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c00caa80 con 0x7fe3c0154210 2023-11-06T20:44:15.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3c53ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c005efc0 con 0x7fe3c00ca610 2023-11-06T20:44:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3beffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe3c0154210 0x7fe3c00a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3be7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe3c00bfd60 0x7fe3c00c9f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59768/0 (socket says 172.21.15.33:59768) 2023-11-06T20:44:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3be7fc700 1 -- 172.21.15.33:0/3646211866 learned_addr learned my addr 172.21.15.33:0/3646211866 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3beffd700 1 -- 172.21.15.33:0/3646211866 >> v2:172.21.15.69:3300/0 conn(0x7fe3c00ca610 msgr2=0x7fe3c00c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3bf7fe700 1 --2- 172.21.15.33:0/3646211866 >> v2:172.21.15.69:3300/0 conn(0x7fe3c00ca610 0x7fe3c00c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3beffd700 1 --2- 172.21.15.33:0/3646211866 >> v2:172.21.15.69:3300/0 conn(0x7fe3c00ca610 0x7fe3c00c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3beffd700 1 -- 172.21.15.33:0/3646211866 >> v1:172.21.15.33:6789/0 conn(0x7fe3c00bfd60 legacy=0x7fe3c00c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3bdffb700 1 -- 172.21.15.33:0/3646211866 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423169625 0 0) 0x7fe3c00c06d0 con 0x7fe3c00bfd60 2023-11-06T20:44:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3beffd700 1 -- 172.21.15.33:0/3646211866 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c00c8ea0 con 0x7fe3c0154210 2023-11-06T20:44:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.463+0000 7fe3beffd700 1 --2- 172.21.15.33:0/3646211866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 0x7fe3c00a7bd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fe3c0083a30 tx=0x7fe3b0009060).ready entity=mon.0 client_cookie=ff0ee06857262f78 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3bdffb700 1 -- 172.21.15.33:0/3646211866 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000b580 con 0x7fe3c0154210 2023-11-06T20:44:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3bdffb700 1 -- 172.21.15.33:0/3646211866 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b000b6e0 con 0x7fe3c0154210 2023-11-06T20:44:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3bdffb700 1 -- 172.21.15.33:0/3646211866 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000ba60 con 0x7fe3c0154210 2023-11-06T20:44:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3c53ff700 1 -- 172.21.15.33:0/3646211866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 msgr2=0x7fe3c00a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3c53ff700 1 --2- 172.21.15.33:0/3646211866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 0x7fe3c00a7bd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fe3c0083a30 tx=0x7fe3b0009060).stop 2023-11-06T20:44:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3c53ff700 1 -- 172.21.15.33:0/3646211866 shutdown_connections 2023-11-06T20:44:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3c53ff700 1 --2- 172.21.15.33:0/3646211866 >> v2:172.21.15.69:3300/0 conn(0x7fe3c00ca610 0x7fe3c00c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3c53ff700 1 --2- 172.21.15.33:0/3646211866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 0x7fe3c00a7bd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3c53ff700 1 -- 172.21.15.33:0/3646211866 >> 172.21.15.33:0/3646211866 conn(0x7fe3c005e990 msgr2=0x7fe3c00cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3c53ff700 1 -- 172.21.15.33:0/3646211866 shutdown_connections 2023-11-06T20:44:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.464+0000 7fe3c53ff700 1 -- 172.21.15.33:0/3646211866 wait complete. 2023-11-06T20:44:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.467+0000 7fe3c53ff700 1 Processor -- start 2023-11-06T20:44:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.467+0000 7fe3c53ff700 1 -- start start 2023-11-06T20:44:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3c53ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3c00bfd60 0x7fe3c016ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3c53ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3c00ca610 0x7fe3c014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3c53ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 0x7fe3c00aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3c53ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c00aeb60 con 0x7fe3c0154210 2023-11-06T20:44:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3c53ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c016d3a0 con 0x7fe3c00bfd60 2023-11-06T20:44:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3c53ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c00adee0 con 0x7fe3c00ca610 2023-11-06T20:44:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 0x7fe3c00aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 0x7fe3c00aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42894/0 (socket says 172.21.15.33:42894) 2023-11-06T20:44:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3bf7fe700 1 -- 172.21.15.33:0/1145612431 learned_addr learned my addr 172.21.15.33:0/1145612431 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3be7fc700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3c00ca610 0x7fe3c014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.468+0000 7fe3beffd700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3c00bfd60 0x7fe3c016ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3bf7fe700 1 -- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3c00ca610 msgr2=0x7fe3c014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3bf7fe700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3c00ca610 0x7fe3c014ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3bf7fe700 1 -- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3c00bfd60 msgr2=0x7fe3c016ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3bf7fe700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3c00bfd60 0x7fe3c016ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3bf7fe700 1 -- 172.21.15.33:0/1145612431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00092a0 con 0x7fe3c0154210 2023-11-06T20:44:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3bf7fe700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 0x7fe3c00aef80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fe3b800ceb0 tx=0x7fe3b8007870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3a6ffd700 1 -- 172.21.15.33:0/1145612431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800ad90 con 0x7fe3c0154210 2023-11-06T20:44:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3a6ffd700 1 -- 172.21.15.33:0/1145612431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b8016400 con 0x7fe3c0154210 2023-11-06T20:44:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3c53ff700 1 -- 172.21.15.33:0/1145612431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c00ae1c0 con 0x7fe3c0154210 2023-11-06T20:44:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3a6ffd700 1 -- 172.21.15.33:0/1145612431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8016780 con 0x7fe3c0154210 2023-11-06T20:44:15.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.469+0000 7fe3c53ff700 1 -- 172.21.15.33:0/1145612431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c00ae780 con 0x7fe3c0154210 2023-11-06T20:44:15.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.471+0000 7fe3c53ff700 1 -- 172.21.15.33:0/1145612431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c00aeca0 con 0x7fe3c0154210 2023-11-06T20:44:15.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.471+0000 7fe3a6ffd700 1 -- 172.21.15.33:0/1145612431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe3b8009030 con 0x7fe3c0154210 2023-11-06T20:44:15.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.472+0000 7fe3a6ffd700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe3ac05f150 0x7fe3ac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.472+0000 7fe3a6ffd700 1 -- 172.21.15.33:0/1145612431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe3b807a950 con 0x7fe3c0154210 2023-11-06T20:44:15.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.472+0000 7fe3beffd700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe3ac05f150 0x7fe3ac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.473+0000 7fe3beffd700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe3ac05f150 0x7fe3ac061610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fe3b0009fd0 tx=0x7fe3b0017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:15.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.476+0000 7fe3a6ffd700 1 -- 172.21.15.33:0/1145612431 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe3c00aeca0 con 0x7fe3c0154210 2023-11-06T20:44:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.658+0000 7fe3c53ff700 1 -- 172.21.15.33:0/1145612431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe3c006c6c0 con 0x7fe3c0154210 2023-11-06T20:44:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.659+0000 7fe3a6ffd700 1 -- 172.21.15.33:0/1145612431 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe3b804bb60 con 0x7fe3c0154210 2023-11-06T20:44:15.658 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-11-06T20:44:15.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 -- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe3ac05f150 msgr2=0x7fe3ac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe3ac05f150 0x7fe3ac061610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fe3b0009fd0 tx=0x7fe3b0017000).stop 2023-11-06T20:44:15.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 -- 172.21.15.33:0/1145612431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 msgr2=0x7fe3c00aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 0x7fe3c00aef80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fe3b800ceb0 tx=0x7fe3b8007870).stop 2023-11-06T20:44:15.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 -- 172.21.15.33:0/1145612431 shutdown_connections 2023-11-06T20:44:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe3ac05f150 0x7fe3ac061610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c0154210 0x7fe3c00aef80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3c00bfd60 0x7fe3c016ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 --2- 172.21.15.33:0/1145612431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3c00ca610 0x7fe3c014ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 -- 172.21.15.33:0/1145612431 >> 172.21.15.33:0/1145612431 conn(0x7fe3c005e990 msgr2=0x7fe3c00c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 -- 172.21.15.33:0/1145612431 shutdown_connections 2023-11-06T20:44:15.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.661+0000 7fe3c53ff700 1 -- 172.21.15.33:0/1145612431 wait complete. 2023-11-06T20:44:15.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2023-11-06T20:44:15.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-06T20:44:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.875+0000 7f2fe31ff700 1 Processor -- start 2023-11-06T20:44:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.875+0000 7f2fe31ff700 1 -- start start 2023-11-06T20:44:15.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.876+0000 7f2fe31ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2fdc154210 0x7f2fdc0a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.876+0000 7f2fe31ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fdc0bfd60 0x7f2fdc0c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe31ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fdc0c06d0 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe31ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc0caaa0 con 0x7f2fdc154210 2023-11-06T20:44:15.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe31ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc05efc0 con 0x7f2fdc0bfd60 2023-11-06T20:44:15.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe0f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2fdc154210 0x7f2fdc0a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe0f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2fdc154210 0x7f2fdc0a7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42898/0 (socket says 172.21.15.33:42898) 2023-11-06T20:44:15.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe0f9b700 1 -- 172.21.15.33:0/3400654156 learned_addr learned my addr 172.21.15.33:0/3400654156 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:15.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fdbfff700 1 --2- 172.21.15.33:0/3400654156 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fdc0bfd60 0x7f2fdc0c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe0f9b700 1 -- 172.21.15.33:0/3400654156 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fdc0bfd60 msgr2=0x7f2fdc0c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe0f9b700 1 --2- 172.21.15.33:0/3400654156 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fdc0bfd60 0x7f2fdc0c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe0f9b700 1 -- 172.21.15.33:0/3400654156 >> v1:172.21.15.69:6789/0 conn(0x7f2fdc0c9f00 legacy=0x7f2fdc0ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe0f9b700 1 -- 172.21.15.33:0/3400654156 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc0c8ea0 con 0x7f2fdc154210 2023-11-06T20:44:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.877+0000 7f2fe0f9b700 1 --2- 172.21.15.33:0/3400654156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc154210 0x7f2fdc0a7bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f2fd0006fd0 tx=0x7f2fd00090f0).ready entity=mon.0 client_cookie=4886a281dc148eec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fdb7fe700 1 -- 172.21.15.33:0/3400654156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000b600 con 0x7f2fdc154210 2023-11-06T20:44:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fdb7fe700 1 -- 172.21.15.33:0/3400654156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd000b760 con 0x7f2fdc154210 2023-11-06T20:44:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fdb7fe700 1 -- 172.21.15.33:0/3400654156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000bae0 con 0x7f2fdc154210 2023-11-06T20:44:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fe31ff700 1 -- 172.21.15.33:0/3400654156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc154210 msgr2=0x7f2fdc0a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fe31ff700 1 --2- 172.21.15.33:0/3400654156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc154210 0x7f2fdc0a7bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f2fd0006fd0 tx=0x7f2fd00090f0).stop 2023-11-06T20:44:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fe31ff700 1 -- 172.21.15.33:0/3400654156 shutdown_connections 2023-11-06T20:44:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fe31ff700 1 --2- 172.21.15.33:0/3400654156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc154210 0x7f2fdc0a7bd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fe31ff700 1 --2- 172.21.15.33:0/3400654156 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fdc0bfd60 0x7f2fdc0c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fe31ff700 1 -- 172.21.15.33:0/3400654156 >> 172.21.15.33:0/3400654156 conn(0x7f2fdc05e990 msgr2=0x7f2fdc0cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fe31ff700 1 -- 172.21.15.33:0/3400654156 shutdown_connections 2023-11-06T20:44:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.878+0000 7f2fe31ff700 1 -- 172.21.15.33:0/3400654156 wait complete. 2023-11-06T20:44:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.880+0000 7f2fe31ff700 1 Processor -- start 2023-11-06T20:44:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.880+0000 7f2fe31ff700 1 -- start start 2023-11-06T20:44:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fe31ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fdc0bfd60 0x7f2fdc0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fe31ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0c9f00 0x7f2fdc0c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fe31ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fdc154210 0x7f2fdc14eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fe31ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0b02a0 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fe31ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0b0730 con 0x7f2fdc154210 2023-11-06T20:44:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fe31ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0af690 con 0x7f2fdc0bfd60 2023-11-06T20:44:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0c9f00 0x7f2fdc0c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0c9f00 0x7f2fdc0c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42906/0 (socket says 172.21.15.33:42906) 2023-11-06T20:44:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fdbfff700 1 -- 172.21.15.33:0/276109784 learned_addr learned my addr 172.21.15.33:0/276109784 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fe0f9b700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fdc0bfd60 0x7f2fdc0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.881+0000 7f2fe179c700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fdc154210 0x7f2fdc14eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fdbfff700 1 -- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fdc0bfd60 msgr2=0x7f2fdc0af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fdbfff700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fdc0bfd60 0x7f2fdc0af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fdbfff700 1 -- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fdc154210 msgr2=0x7f2fdc14eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fdbfff700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fdc154210 0x7f2fdc14eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fdbfff700 1 -- 172.21.15.33:0/276109784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd0009370 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fe179c700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fdc154210 0x7f2fdc14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fdbfff700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0c9f00 0x7f2fdc0c2fe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f2fc800d890 tx=0x7f2fc8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fd8ff9700 1 -- 172.21.15.33:0/276109784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8011590 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fd8ff9700 1 -- 172.21.15.33:0/276109784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc80116f0 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fd8ff9700 1 -- 172.21.15.33:0/276109784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8011a70 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fe31ff700 1 -- 172.21.15.33:0/276109784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc0af970 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.882+0000 7f2fe31ff700 1 -- 172.21.15.33:0/276109784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc0aff30 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.883+0000 7f2fe31ff700 1 -- 172.21.15.33:0/276109784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc06c6c0 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.884+0000 7f2fd8ff9700 1 -- 172.21.15.33:0/276109784 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2fc801c460 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.885+0000 7f2fd8ff9700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2fc005f150 0x7f2fc0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.885+0000 7f2fd8ff9700 1 -- 172.21.15.33:0/276109784 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2fc807a260 con 0x7f2fdc0c9f00 2023-11-06T20:44:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.885+0000 7f2fe0f9b700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2fc005f150 0x7f2fc0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.886+0000 7f2fe0f9b700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2fc005f150 0x7f2fc0061610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2fd0009130 tx=0x7f2fd0017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:15.889+0000 7f2fd8ff9700 1 -- 172.21.15.33:0/276109784 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2fc804b4d0 con 0x7f2fdc0c9f00 2023-11-06T20:44:16.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.074+0000 7f2fe31ff700 1 -- 172.21.15.33:0/276109784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2fdc0c9140 con 0x7f2fdc0c9f00 2023-11-06T20:44:16.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.074+0000 7f2fd8ff9700 1 -- 172.21.15.33:0/276109784 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2fc804eaf0 con 0x7f2fdc0c9f00 2023-11-06T20:44:16.074 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2023-11-06T20:44:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.076+0000 7f2fe31ff700 1 -- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2fc005f150 msgr2=0x7f2fc0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.076+0000 7f2fe31ff700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2fc005f150 0x7f2fc0061610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2fd0009130 tx=0x7f2fd0017000).stop 2023-11-06T20:44:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.077+0000 7f2fe31ff700 1 -- 172.21.15.33:0/276109784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0c9f00 msgr2=0x7f2fdc0c2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.077+0000 7f2fe31ff700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0c9f00 0x7f2fdc0c2fe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f2fc800d890 tx=0x7f2fc8008040).stop 2023-11-06T20:44:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.077+0000 7f2fe31ff700 1 -- 172.21.15.33:0/276109784 shutdown_connections 2023-11-06T20:44:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.077+0000 7f2fe31ff700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2fc005f150 0x7f2fc0061610 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.077+0000 7f2fe31ff700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fdc0bfd60 0x7f2fdc0af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.077+0000 7f2fe31ff700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fdc154210 0x7f2fdc14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.077+0000 7f2fe31ff700 1 --2- 172.21.15.33:0/276109784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0c9f00 0x7f2fdc0c2fe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.077+0000 7f2fe31ff700 1 -- 172.21.15.33:0/276109784 >> 172.21.15.33:0/276109784 conn(0x7f2fdc05e990 msgr2=0x7f2fdc064050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:16.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.077+0000 7f2fe31ff700 1 -- 172.21.15.33:0/276109784 shutdown_connections 2023-11-06T20:44:16.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.077+0000 7f2fe31ff700 1 -- 172.21.15.33:0/276109784 wait complete. 2023-11-06T20:44:16.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.3 2023-11-06T20:44:16.093 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-06T20:44:16.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.290+0000 7fe7b5dff700 1 Processor -- start 2023-11-06T20:44:16.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.290+0000 7fe7b5dff700 1 -- start start 2023-11-06T20:44:16.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.291+0000 7fe7b5dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe7b00c6d90 0x7fe7b00c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.291+0000 7fe7b5dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b00c7700 con 0x7fe7b00bfd60 2023-11-06T20:44:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.291+0000 7fe7b5dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b005efc0 con 0x7fe7b0154210 2023-11-06T20:44:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.291+0000 7fe7b5dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b00c8ea0 con 0x7fe7b00c6d90 2023-11-06T20:44:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.291+0000 7fe7affff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe7b00bfd60 0x7fe7b00c9f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59778/0 (socket says 172.21.15.33:59778) 2023-11-06T20:44:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.291+0000 7fe7affff700 1 -- 172.21.15.33:0/3288381296 learned_addr learned my addr 172.21.15.33:0/3288381296 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.291+0000 7fe7aeffd700 1 --2- 172.21.15.33:0/3288381296 >> v2:172.21.15.69:3300/0 conn(0x7fe7b00c6d90 0x7fe7b00c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.292+0000 7fe7ae7fc700 1 -- 172.21.15.33:0/3288381296 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198656438 0 0) 0x7fe7a8001490 con 0x7fe7b00bfd60 2023-11-06T20:44:16.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.292+0000 7fe7ae7fc700 1 -- 172.21.15.33:0/3288381296 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7980033f0 con 0x7fe7b00bfd60 2023-11-06T20:44:16.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.292+0000 7fe7aeffd700 1 -- 172.21.15.33:0/3288381296 >> v1:172.21.15.69:6789/0 conn(0x7fe7b0154210 legacy=0x7fe7b00a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.292+0000 7fe7ae7fc700 1 -- 172.21.15.33:0/3288381296 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304934567 0 0) 0x7fe7b005efc0 con 0x7fe7b0154210 2023-11-06T20:44:16.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.292+0000 7fe7aeffd700 1 -- 172.21.15.33:0/3288381296 >> v1:172.21.15.33:6789/0 conn(0x7fe7b00bfd60 legacy=0x7fe7b00c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.292+0000 7fe7aeffd700 1 -- 172.21.15.33:0/3288381296 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00c9140 con 0x7fe7b00c6d90 2023-11-06T20:44:16.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.292+0000 7fe7aeffd700 1 --2- 172.21.15.33:0/3288381296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7b00c6d90 0x7fe7b00c7210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fe7a4009760 tx=0x7fe7a4009060).ready entity=mon.1 client_cookie=de35392f77a94aa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:16.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.292+0000 7fe7ae7fc700 1 -- 172.21.15.33:0/3288381296 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a400f510 con 0x7fe7b00c6d90 2023-11-06T20:44:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.293+0000 7fe7ae7fc700 1 -- 172.21.15.33:0/3288381296 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a400f670 con 0x7fe7b00c6d90 2023-11-06T20:44:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.293+0000 7fe7ae7fc700 1 -- 172.21.15.33:0/3288381296 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a400f9f0 con 0x7fe7b00c6d90 2023-11-06T20:44:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.293+0000 7fe7b5dff700 1 -- 172.21.15.33:0/3288381296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7b00c6d90 msgr2=0x7fe7b00c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.293+0000 7fe7b5dff700 1 --2- 172.21.15.33:0/3288381296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7b00c6d90 0x7fe7b00c7210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fe7a4009760 tx=0x7fe7a4009060).stop 2023-11-06T20:44:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.293+0000 7fe7b5dff700 1 -- 172.21.15.33:0/3288381296 shutdown_connections 2023-11-06T20:44:16.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.293+0000 7fe7b5dff700 1 --2- 172.21.15.33:0/3288381296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7b00c6d90 0x7fe7b00c7210 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.293+0000 7fe7b5dff700 1 -- 172.21.15.33:0/3288381296 >> 172.21.15.33:0/3288381296 conn(0x7fe7b005e990 msgr2=0x7fe7b00cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:16.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.293+0000 7fe7b5dff700 1 -- 172.21.15.33:0/3288381296 shutdown_connections 2023-11-06T20:44:16.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.293+0000 7fe7b5dff700 1 -- 172.21.15.33:0/3288381296 wait complete. 2023-11-06T20:44:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.295+0000 7fe7b5dff700 1 Processor -- start 2023-11-06T20:44:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.295+0000 7fe7b5dff700 1 -- start start 2023-11-06T20:44:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7b5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7b00bfd60 0x7fe7b014fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7b5dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b00c6d90 0x7fe7b0155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7b5dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7b0154210 0x7fe7b0155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7b5dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b0150200 con 0x7fe7b00c6d90 2023-11-06T20:44:16.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7b5dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b0155200 con 0x7fe7b0154210 2023-11-06T20:44:16.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7b5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b0155b80 con 0x7fe7b00bfd60 2023-11-06T20:44:16.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b00c6d90 0x7fe7b0155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b00c6d90 0x7fe7b0155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42918/0 (socket says 172.21.15.33:42918) 2023-11-06T20:44:16.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7aeffd700 1 -- 172.21.15.33:0/2904562654 learned_addr learned my addr 172.21.15.33:0/2904562654 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:16.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7affff700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7b0154210 0x7fe7b0155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7af7fe700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7b00bfd60 0x7fe7b014fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7aeffd700 1 -- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7b00bfd60 msgr2=0x7fe7b014fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7aeffd700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7b00bfd60 0x7fe7b014fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7aeffd700 1 -- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7b0154210 msgr2=0x7fe7b0155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7aeffd700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7b0154210 0x7fe7b0155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7aeffd700 1 -- 172.21.15.33:0/2904562654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a40092a0 con 0x7fe7b00c6d90 2023-11-06T20:44:16.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7af7fe700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7b00bfd60 0x7fe7b014fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:16.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.296+0000 7fe7aeffd700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b00c6d90 0x7fe7b0155690 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fe7a400f590 tx=0x7fe7a4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:16.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.297+0000 7fe7977fe700 1 -- 172.21.15.33:0/2904562654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a400f7e0 con 0x7fe7b00c6d90 2023-11-06T20:44:16.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.297+0000 7fe7977fe700 1 -- 172.21.15.33:0/2904562654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a400f940 con 0x7fe7b00c6d90 2023-11-06T20:44:16.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.297+0000 7fe7b5dff700 1 -- 172.21.15.33:0/2904562654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b01545f0 con 0x7fe7b00c6d90 2023-11-06T20:44:16.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.297+0000 7fe7977fe700 1 -- 172.21.15.33:0/2904562654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a401b640 con 0x7fe7b00c6d90 2023-11-06T20:44:16.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.297+0000 7fe7b5dff700 1 -- 172.21.15.33:0/2904562654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b0154bb0 con 0x7fe7b00c6d90 2023-11-06T20:44:16.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.298+0000 7fe7b5dff700 1 -- 172.21.15.33:0/2904562654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b006c6c0 con 0x7fe7b00c6d90 2023-11-06T20:44:16.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.299+0000 7fe7977fe700 1 -- 172.21.15.33:0/2904562654 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe7a401b9f0 con 0x7fe7b00c6d90 2023-11-06T20:44:16.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.299+0000 7fe7977fe700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe79c05f180 0x7fe79c061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.300+0000 7fe7977fe700 1 -- 172.21.15.33:0/2904562654 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe7a407bde0 con 0x7fe7b00c6d90 2023-11-06T20:44:16.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.300+0000 7fe7af7fe700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe79c05f180 0x7fe79c061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.301+0000 7fe7af7fe700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe79c05f180 0x7fe79c061640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fe7a0001640 tx=0x7fe7a0009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:16.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.304+0000 7fe7977fe700 1 -- 172.21.15.33:0/2904562654 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe7a404cff0 con 0x7fe7b00c6d90 2023-11-06T20:44:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.486+0000 7fe7b5dff700 1 -- 172.21.15.33:0/2904562654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe7b00c9420 con 0x7fe7b00c6d90 2023-11-06T20:44:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.486+0000 7fe7977fe700 1 -- 172.21.15.33:0/2904562654 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe7a4050610 con 0x7fe7b00c6d90 2023-11-06T20:44:16.486 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-11-06T20:44:16.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 -- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe79c05f180 msgr2=0x7fe79c061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe79c05f180 0x7fe79c061640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fe7a0001640 tx=0x7fe7a0009040).stop 2023-11-06T20:44:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 -- 172.21.15.33:0/2904562654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b00c6d90 msgr2=0x7fe7b0155690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b00c6d90 0x7fe7b0155690 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fe7a400f590 tx=0x7fe7a4017000).stop 2023-11-06T20:44:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 -- 172.21.15.33:0/2904562654 shutdown_connections 2023-11-06T20:44:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe79c05f180 0x7fe79c061640 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7b00bfd60 0x7fe7b014fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7b0154210 0x7fe7b0155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 --2- 172.21.15.33:0/2904562654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b00c6d90 0x7fe7b0155690 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 -- 172.21.15.33:0/2904562654 >> 172.21.15.33:0/2904562654 conn(0x7fe7b005e990 msgr2=0x7fe7b0151240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:16.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 -- 172.21.15.33:0/2904562654 shutdown_connections 2023-11-06T20:44:16.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.488+0000 7fe7b5dff700 1 -- 172.21.15.33:0/2904562654 wait complete. 2023-11-06T20:44:16.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.4 2023-11-06T20:44:16.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-06T20:44:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.703+0000 7fcd9a1ff700 1 Processor -- start 2023-11-06T20:44:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.703+0000 7fcd9a1ff700 1 -- start start 2023-11-06T20:44:16.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.704+0000 7fcd9a1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcd94154210 0x7fcd940a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.704+0000 7fcd9a1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd940bfd60 0x7fcd940c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.704+0000 7fcd9a1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd940c9f00 0x7fcd940ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.704+0000 7fcd9a1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940c06d0 con 0x7fcd940c9f00 2023-11-06T20:44:16.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.704+0000 7fcd9a1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9405efc0 con 0x7fcd94154210 2023-11-06T20:44:16.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.704+0000 7fcd9a1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940c8ea0 con 0x7fcd940bfd60 2023-11-06T20:44:16.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd93fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd940c9f00 0x7fcd940ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd93fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd940c9f00 0x7fcd940ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42932/0 (socket says 172.21.15.33:42932) 2023-11-06T20:44:16.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd93fff700 1 -- 172.21.15.33:0/2145038099 learned_addr learned my addr 172.21.15.33:0/2145038099 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:16.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd937fe700 1 --2- 172.21.15.33:0/2145038099 >> v2:172.21.15.69:3300/0 conn(0x7fcd94154210 0x7fcd940a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd92ffd700 1 --2- 172.21.15.33:0/2145038099 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd940bfd60 0x7fcd940c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd93fff700 1 -- 172.21.15.33:0/2145038099 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd940bfd60 msgr2=0x7fcd940c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:16.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd93fff700 1 --2- 172.21.15.33:0/2145038099 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd940bfd60 0x7fcd940c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd93fff700 1 -- 172.21.15.33:0/2145038099 >> v2:172.21.15.69:3300/0 conn(0x7fcd94154210 msgr2=0x7fcd940a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:16.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd93fff700 1 --2- 172.21.15.33:0/2145038099 >> v2:172.21.15.69:3300/0 conn(0x7fcd94154210 0x7fcd940a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd93fff700 1 -- 172.21.15.33:0/2145038099 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd940c9140 con 0x7fcd940c9f00 2023-11-06T20:44:16.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.705+0000 7fcd93fff700 1 --2- 172.21.15.33:0/2145038099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940c9f00 0x7fcd940ca390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fcd940adab0 tx=0x7fcd8c009840).ready entity=mon.0 client_cookie=fc46a3d512457120 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:16.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd927fc700 1 -- 172.21.15.33:0/2145038099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00e070 con 0x7fcd940c9f00 2023-11-06T20:44:16.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd927fc700 1 -- 172.21.15.33:0/2145038099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c00ba50 con 0x7fcd940c9f00 2023-11-06T20:44:16.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd927fc700 1 -- 172.21.15.33:0/2145038099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00bdd0 con 0x7fcd940c9f00 2023-11-06T20:44:16.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/2145038099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940c9f00 msgr2=0x7fcd940ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd9a1ff700 1 --2- 172.21.15.33:0/2145038099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940c9f00 0x7fcd940ca390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fcd940adab0 tx=0x7fcd8c009840).stop 2023-11-06T20:44:16.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/2145038099 shutdown_connections 2023-11-06T20:44:16.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd9a1ff700 1 --2- 172.21.15.33:0/2145038099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940c9f00 0x7fcd940ca390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd9a1ff700 1 --2- 172.21.15.33:0/2145038099 >> v2:172.21.15.69:3300/0 conn(0x7fcd94154210 0x7fcd940a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd9a1ff700 1 --2- 172.21.15.33:0/2145038099 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd940bfd60 0x7fcd940c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/2145038099 >> 172.21.15.33:0/2145038099 conn(0x7fcd9405e990 msgr2=0x7fcd940cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:16.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/2145038099 shutdown_connections 2023-11-06T20:44:16.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.706+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/2145038099 wait complete. 2023-11-06T20:44:16.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.708+0000 7fcd9a1ff700 1 Processor -- start 2023-11-06T20:44:16.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.708+0000 7fcd9a1ff700 1 -- start start 2023-11-06T20:44:16.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd9a1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940bfd60 0x7fcd940af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd9a1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd940c9f00 0x7fcd94156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd9a1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd94154210 0x7fcd94156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd9a1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940af1f0 con 0x7fcd940bfd60 2023-11-06T20:44:16.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd9a1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940ae5e0 con 0x7fcd940c9f00 2023-11-06T20:44:16.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd9a1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940ae8c0 con 0x7fcd94154210 2023-11-06T20:44:16.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940bfd60 0x7fcd940af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940bfd60 0x7fcd940af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42940/0 (socket says 172.21.15.33:42940) 2023-11-06T20:44:16.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd937fe700 1 -- 172.21.15.33:0/177267629 learned_addr learned my addr 172.21.15.33:0/177267629 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:16.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd92ffd700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd940c9f00 0x7fcd94156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.709+0000 7fcd93fff700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd94154210 0x7fcd94156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd937fe700 1 -- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd94154210 msgr2=0x7fcd94156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd937fe700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd94154210 0x7fcd94156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd937fe700 1 -- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd940c9f00 msgr2=0x7fcd94156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd937fe700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd940c9f00 0x7fcd94156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd937fe700 1 -- 172.21.15.33:0/177267629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c009ae0 con 0x7fcd940bfd60 2023-11-06T20:44:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd937fe700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940bfd60 0x7fcd940af680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fcd8400d890 tx=0x7fcd84008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd7b7fe700 1 -- 172.21.15.33:0/177267629 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84011590 con 0x7fcd940bfd60 2023-11-06T20:44:16.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd7b7fe700 1 -- 172.21.15.33:0/177267629 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd840116f0 con 0x7fcd940bfd60 2023-11-06T20:44:16.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd7b7fe700 1 -- 172.21.15.33:0/177267629 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84011a70 con 0x7fcd940bfd60 2023-11-06T20:44:16.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/177267629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd940aeba0 con 0x7fcd940bfd60 2023-11-06T20:44:16.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.710+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/177267629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd940ae070 con 0x7fcd940bfd60 2023-11-06T20:44:16.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.711+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/177267629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9406c6c0 con 0x7fcd940bfd60 2023-11-06T20:44:16.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.712+0000 7fcd7b7fe700 1 -- 172.21.15.33:0/177267629 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcd8401c460 con 0x7fcd940bfd60 2023-11-06T20:44:16.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.713+0000 7fcd7b7fe700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd8005f150 0x7fcd80061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:16.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.713+0000 7fcd7b7fe700 1 -- 172.21.15.33:0/177267629 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcd84079e10 con 0x7fcd940bfd60 2023-11-06T20:44:16.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.713+0000 7fcd92ffd700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd8005f150 0x7fcd80061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:16.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.714+0000 7fcd92ffd700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd8005f150 0x7fcd80061610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fcd941676d0 tx=0x7fcd88008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:16.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.717+0000 7fcd7b7fe700 1 -- 172.21.15.33:0/177267629 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcd8404b080 con 0x7fcd940bfd60 2023-11-06T20:44:16.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.898+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/177267629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fcd940c9420 con 0x7fcd940bfd60 2023-11-06T20:44:16.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.899+0000 7fcd7b7fe700 1 -- 172.21.15.33:0/177267629 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcd84011e60 con 0x7fcd940bfd60 2023-11-06T20:44:16.898 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2023-11-06T20:44:16.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd8005f150 msgr2=0x7fcd80061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd8005f150 0x7fcd80061610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fcd941676d0 tx=0x7fcd88008040).stop 2023-11-06T20:44:16.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/177267629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940bfd60 msgr2=0x7fcd940af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:16.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940bfd60 0x7fcd940af680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fcd8400d890 tx=0x7fcd84008040).stop 2023-11-06T20:44:16.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/177267629 shutdown_connections 2023-11-06T20:44:16.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd8005f150 0x7fcd80061610 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd94154210 0x7fcd94156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd940bfd60 0x7fcd940af680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 --2- 172.21.15.33:0/177267629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd940c9f00 0x7fcd94156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:16.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/177267629 >> 172.21.15.33:0/177267629 conn(0x7fcd9405e990 msgr2=0x7fcd9414f510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:16.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/177267629 shutdown_connections 2023-11-06T20:44:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:16.901+0000 7fcd9a1ff700 1 -- 172.21.15.33:0/177267629 wait complete. 2023-11-06T20:44:16.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-11-06T20:44:16.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-06T20:44:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.116+0000 7fbeb67ff700 1 Processor -- start 2023-11-06T20:44:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.116+0000 7fbeb67ff700 1 -- start start 2023-11-06T20:44:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.118+0000 7fbeb67ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbeb0154210 0x7fbeb00a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.118+0000 7fbeb67ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbeb00bfd60 0x7fbeb00c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.118+0000 7fbeb67ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb00c06d0 con 0x7fbeb00c9f00 2023-11-06T20:44:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.118+0000 7fbeb67ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb00caa80 con 0x7fbeb00bfd60 2023-11-06T20:44:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.118+0000 7fbeb67ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb005efc0 con 0x7fbeb0154210 2023-11-06T20:44:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.118+0000 7fbeaf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbeb00bfd60 0x7fbeb00c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.118+0000 7fbeaf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbeb00bfd60 0x7fbeb00c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42950/0 (socket says 172.21.15.33:42950) 2023-11-06T20:44:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.118+0000 7fbeaf7fe700 1 -- 172.21.15.33:0/3432342381 learned_addr learned my addr 172.21.15.33:0/3432342381 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.118+0000 7fbeaffff700 1 --2- 172.21.15.33:0/3432342381 >> v2:172.21.15.69:3300/0 conn(0x7fbeb0154210 0x7fbeb00a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.119+0000 7fbeaf7fe700 1 -- 172.21.15.33:0/3432342381 >> v2:172.21.15.69:3300/0 conn(0x7fbeb0154210 msgr2=0x7fbeb00a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.119+0000 7fbeaf7fe700 1 --2- 172.21.15.33:0/3432342381 >> v2:172.21.15.69:3300/0 conn(0x7fbeb0154210 0x7fbeb00a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.119+0000 7fbeaf7fe700 1 -- 172.21.15.33:0/3432342381 >> v1:172.21.15.69:6789/0 conn(0x7fbeb00c9f00 legacy=0x7fbeb00ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.119+0000 7fbeaf7fe700 1 -- 172.21.15.33:0/3432342381 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb00c8ea0 con 0x7fbeb00bfd60 2023-11-06T20:44:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.119+0000 7fbeaf7fe700 1 --2- 172.21.15.33:0/3432342381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00bfd60 0x7fbeb00c01e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fbeb00adab0 tx=0x7fbea40090f0).ready entity=mon.0 client_cookie=31fa3d087f91b485 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:17.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.119+0000 7fbeaeffd700 1 -- 172.21.15.33:0/3432342381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400b520 con 0x7fbeb00bfd60 2023-11-06T20:44:17.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.119+0000 7fbeaeffd700 1 -- 172.21.15.33:0/3432342381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea400b680 con 0x7fbeb00bfd60 2023-11-06T20:44:17.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.119+0000 7fbeaeffd700 1 -- 172.21.15.33:0/3432342381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400ba00 con 0x7fbeb00bfd60 2023-11-06T20:44:17.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.120+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3432342381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00bfd60 msgr2=0x7fbeb00c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.120+0000 7fbeb67ff700 1 --2- 172.21.15.33:0/3432342381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00bfd60 0x7fbeb00c01e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fbeb00adab0 tx=0x7fbea40090f0).stop 2023-11-06T20:44:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.120+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3432342381 shutdown_connections 2023-11-06T20:44:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.120+0000 7fbeb67ff700 1 --2- 172.21.15.33:0/3432342381 >> v2:172.21.15.69:3300/0 conn(0x7fbeb0154210 0x7fbeb00a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.120+0000 7fbeb67ff700 1 --2- 172.21.15.33:0/3432342381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00bfd60 0x7fbeb00c01e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.120+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3432342381 >> 172.21.15.33:0/3432342381 conn(0x7fbeb005e990 msgr2=0x7fbeb00cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.120+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3432342381 shutdown_connections 2023-11-06T20:44:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.120+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3432342381 wait complete. 2023-11-06T20:44:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.121+0000 7fbeb67ff700 1 Processor -- start 2023-11-06T20:44:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.121+0000 7fbeb67ff700 1 -- start start 2023-11-06T20:44:17.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.122+0000 7fbeb67ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb00bfd60 0x7fbeb014e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.122+0000 7fbeb67ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00c9f00 0x7fbeb00abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.122+0000 7fbeb67ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbeb0154210 0x7fbeb00ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.122+0000 7fbeb67ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00ab840 con 0x7fbeb00c9f00 2023-11-06T20:44:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.122+0000 7fbeb67ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00aac30 con 0x7fbeb00bfd60 2023-11-06T20:44:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.122+0000 7fbeb67ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00aaf10 con 0x7fbeb0154210 2023-11-06T20:44:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00c9f00 0x7fbeb00abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb00bfd60 0x7fbeb014e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00c9f00 0x7fbeb00abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42956/0 (socket says 172.21.15.33:42956) 2023-11-06T20:44:17.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb00bfd60 0x7fbeb014e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52176/0 (socket says 172.21.15.33:52176) 2023-11-06T20:44:17.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaf7fe700 1 -- 172.21.15.33:0/3509696775 learned_addr learned my addr 172.21.15.33:0/3509696775 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:17.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeb4d9c700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbeb0154210 0x7fbeb00ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaffff700 1 -- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbeb0154210 msgr2=0x7fbeb00ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaffff700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbeb0154210 0x7fbeb00ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaffff700 1 -- 172.21.15.33:0/3509696775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00c9f00 msgr2=0x7fbeb00abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaffff700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00c9f00 0x7fbeb00abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaffff700 1 -- 172.21.15.33:0/3509696775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea4009370 con 0x7fbeb00bfd60 2023-11-06T20:44:17.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeb4d9c700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbeb0154210 0x7fbeb00ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:17.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaf7fe700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00c9f00 0x7fbeb00abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.123+0000 7fbeaffff700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb00bfd60 0x7fbeb014e830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fbea000e6f0 tx=0x7fbea0009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.124+0000 7fbe97fff700 1 -- 172.21.15.33:0/3509696775 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0012600 con 0x7fbeb00bfd60 2023-11-06T20:44:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.124+0000 7fbe97fff700 1 -- 172.21.15.33:0/3509696775 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea0012760 con 0x7fbeb00bfd60 2023-11-06T20:44:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.124+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3509696775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb00ab1f0 con 0x7fbeb00bfd60 2023-11-06T20:44:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.124+0000 7fbe97fff700 1 -- 172.21.15.33:0/3509696775 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0012ae0 con 0x7fbeb00bfd60 2023-11-06T20:44:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.124+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3509696775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb00c2060 con 0x7fbeb00bfd60 2023-11-06T20:44:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.125+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3509696775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb00c5160 con 0x7fbeb00bfd60 2023-11-06T20:44:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.125+0000 7fbe97fff700 1 -- 172.21.15.33:0/3509696775 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fbea001b400 con 0x7fbeb00bfd60 2023-11-06T20:44:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.126+0000 7fbe97fff700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe9c05f150 0x7fbe9c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.126+0000 7fbe97fff700 1 -- 172.21.15.33:0/3509696775 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbea007a560 con 0x7fbeb00bfd60 2023-11-06T20:44:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.127+0000 7fbeaf7fe700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe9c05f150 0x7fbe9c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.127+0000 7fbeaf7fe700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe9c05f150 0x7fbe9c061610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fbeb0157d20 tx=0x7fbea4017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:17.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.132+0000 7fbe97fff700 1 -- 172.21.15.33:0/3509696775 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbea004b710 con 0x7fbeb00bfd60 2023-11-06T20:44:17.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.313+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3509696775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbeb006c6c0 con 0x7fbeb00bfd60 2023-11-06T20:44:17.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.313+0000 7fbe97fff700 1 -- 172.21.15.33:0/3509696775 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbea004ed30 con 0x7fbeb00bfd60 2023-11-06T20:44:17.313 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-11-06T20:44:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe9c05f150 msgr2=0x7fbe9c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe9c05f150 0x7fbe9c061610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fbeb0157d20 tx=0x7fbea4017000).stop 2023-11-06T20:44:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb00bfd60 msgr2=0x7fbeb014e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb00bfd60 0x7fbeb014e830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fbea000e6f0 tx=0x7fbea0009060).stop 2023-11-06T20:44:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3509696775 shutdown_connections 2023-11-06T20:44:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe9c05f150 0x7fbe9c061610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbeb0154210 0x7fbeb00ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbeb00bfd60 0x7fbeb014e830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 --2- 172.21.15.33:0/3509696775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb00c9f00 0x7fbeb00abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3509696775 >> 172.21.15.33:0/3509696775 conn(0x7fbeb005e990 msgr2=0x7fbeb00ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3509696775 shutdown_connections 2023-11-06T20:44:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.316+0000 7fbeb67ff700 1 -- 172.21.15.33:0/3509696775 wait complete. 2023-11-06T20:44:17.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2023-11-06T20:44:17.332 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-06T20:44:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.530+0000 7ffa94bff700 1 Processor -- start 2023-11-06T20:44:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.530+0000 7ffa94bff700 1 -- start start 2023-11-06T20:44:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.531+0000 7ffa94bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa900c6d90 0x7ffa900c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.531+0000 7ffa94bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa900c7700 con 0x7ffa900bfd60 2023-11-06T20:44:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.531+0000 7ffa94bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9005efc0 con 0x7ffa90154210 2023-11-06T20:44:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.531+0000 7ffa94bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900c8ea0 con 0x7ffa900c6d90 2023-11-06T20:44:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.531+0000 7ffa8ed9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffa900bfd60 0x7ffa900c9f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59788/0 (socket says 172.21.15.33:59788) 2023-11-06T20:44:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.531+0000 7ffa8ed9d700 1 -- 172.21.15.33:0/4109919214 learned_addr learned my addr 172.21.15.33:0/4109919214 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:17.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.532+0000 7ffa8dd9b700 1 --2- 172.21.15.33:0/4109919214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa900c6d90 0x7ffa900c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.532+0000 7ffa8d59a700 1 -- 172.21.15.33:0/4109919214 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774350140 0 0) 0x7ffa88001490 con 0x7ffa900bfd60 2023-11-06T20:44:17.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.532+0000 7ffa8d59a700 1 -- 172.21.15.33:0/4109919214 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa780033f0 con 0x7ffa900bfd60 2023-11-06T20:44:17.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.532+0000 7ffa8d59a700 1 -- 172.21.15.33:0/4109919214 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021413191 0 0) 0x7ffa9005efc0 con 0x7ffa90154210 2023-11-06T20:44:17.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.532+0000 7ffa8d59a700 1 -- 172.21.15.33:0/4109919214 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa88001490 con 0x7ffa90154210 2023-11-06T20:44:17.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.532+0000 7ffa8dd9b700 1 -- 172.21.15.33:0/4109919214 >> v1:172.21.15.69:6789/0 conn(0x7ffa90154210 legacy=0x7ffa900a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.532+0000 7ffa8d59a700 1 -- 172.21.15.33:0/4109919214 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 118398058 0 0) 0x7ffa780033f0 con 0x7ffa900bfd60 2023-11-06T20:44:17.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.532+0000 7ffa8dd9b700 1 -- 172.21.15.33:0/4109919214 >> v1:172.21.15.33:6789/0 conn(0x7ffa900bfd60 legacy=0x7ffa900c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.532+0000 7ffa8dd9b700 1 -- 172.21.15.33:0/4109919214 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa900c9140 con 0x7ffa900c6d90 2023-11-06T20:44:17.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.532+0000 7ffa8dd9b700 1 --2- 172.21.15.33:0/4109919214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa900c6d90 0x7ffa900c7210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ffa84009790 tx=0x7ffa84009060).ready entity=mon.2 client_cookie=f570b0d41d13672d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:17.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.533+0000 7ffa8d59a700 1 -- 172.21.15.33:0/4109919214 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88001980 con 0x7ffa900c6d90 2023-11-06T20:44:17.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.533+0000 7ffa8d59a700 1 -- 172.21.15.33:0/4109919214 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa84011400 con 0x7ffa900c6d90 2023-11-06T20:44:17.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.533+0000 7ffa8d59a700 1 -- 172.21.15.33:0/4109919214 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84011780 con 0x7ffa900c6d90 2023-11-06T20:44:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.533+0000 7ffa94bff700 1 -- 172.21.15.33:0/4109919214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa900c6d90 msgr2=0x7ffa900c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.533+0000 7ffa94bff700 1 --2- 172.21.15.33:0/4109919214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa900c6d90 0x7ffa900c7210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ffa84009790 tx=0x7ffa84009060).stop 2023-11-06T20:44:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.533+0000 7ffa94bff700 1 -- 172.21.15.33:0/4109919214 shutdown_connections 2023-11-06T20:44:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.533+0000 7ffa94bff700 1 --2- 172.21.15.33:0/4109919214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa900c6d90 0x7ffa900c7210 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.533+0000 7ffa94bff700 1 -- 172.21.15.33:0/4109919214 >> 172.21.15.33:0/4109919214 conn(0x7ffa9005e990 msgr2=0x7ffa900cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.533+0000 7ffa94bff700 1 -- 172.21.15.33:0/4109919214 shutdown_connections 2023-11-06T20:44:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.533+0000 7ffa94bff700 1 -- 172.21.15.33:0/4109919214 wait complete. 2023-11-06T20:44:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.535+0000 7ffa94bff700 1 Processor -- start 2023-11-06T20:44:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.535+0000 7ffa94bff700 1 -- start start 2023-11-06T20:44:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa94bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa900bfd60 0x7ffa900c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa94bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa900c6d90 0x7ffa9014f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa94bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa90154210 0x7ffa901521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa94bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa901526d0 con 0x7ffa900c6d90 2023-11-06T20:44:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa94bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9014e620 con 0x7ffa900bfd60 2023-11-06T20:44:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa94bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9014fc90 con 0x7ffa90154210 2023-11-06T20:44:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa8dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa900c6d90 0x7ffa9014f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa8dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa900c6d90 0x7ffa9014f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42970/0 (socket says 172.21.15.33:42970) 2023-11-06T20:44:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa8dd9b700 1 -- 172.21.15.33:0/3437671339 learned_addr learned my addr 172.21.15.33:0/3437671339 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa8e59c700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa900bfd60 0x7ffa900c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.536+0000 7ffa8ed9d700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa90154210 0x7ffa901521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa8dd9b700 1 -- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa90154210 msgr2=0x7ffa901521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa8dd9b700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa90154210 0x7ffa901521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa8dd9b700 1 -- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa900bfd60 msgr2=0x7ffa900c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa8dd9b700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa900bfd60 0x7ffa900c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa8dd9b700 1 -- 172.21.15.33:0/3437671339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa840092a0 con 0x7ffa900c6d90 2023-11-06T20:44:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa8dd9b700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa900c6d90 0x7ffa9014f600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7ffa90083660 tx=0x7ffa84016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa7e7fc700 1 -- 172.21.15.33:0/3437671339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84010870 con 0x7ffa900c6d90 2023-11-06T20:44:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa7e7fc700 1 -- 172.21.15.33:0/3437671339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa840109d0 con 0x7ffa900c6d90 2023-11-06T20:44:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa7e7fc700 1 -- 172.21.15.33:0/3437671339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84010d90 con 0x7ffa900c6d90 2023-11-06T20:44:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa94bff700 1 -- 172.21.15.33:0/3437671339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa900c3900 con 0x7ffa900c6d90 2023-11-06T20:44:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.537+0000 7ffa94bff700 1 -- 172.21.15.33:0/3437671339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa900c3ec0 con 0x7ffa900c6d90 2023-11-06T20:44:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.538+0000 7ffa94bff700 1 -- 172.21.15.33:0/3437671339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9006c6c0 con 0x7ffa900c6d90 2023-11-06T20:44:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.539+0000 7ffa7e7fc700 1 -- 172.21.15.33:0/3437671339 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ffa840263b0 con 0x7ffa900c6d90 2023-11-06T20:44:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.540+0000 7ffa7e7fc700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7005f150 0x7ffa70061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.540+0000 7ffa7e7fc700 1 -- 172.21.15.33:0/3437671339 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ffa8407afd0 con 0x7ffa900c6d90 2023-11-06T20:44:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.540+0000 7ffa8e59c700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7005f150 0x7ffa70061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.541+0000 7ffa8e59c700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7005f150 0x7ffa70061610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7ffa9014edb0 tx=0x7ffa80009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.544+0000 7ffa7e7fc700 1 -- 172.21.15.33:0/3437671339 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffa8404c180 con 0x7ffa900c6d90 2023-11-06T20:44:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.724+0000 7ffa94bff700 1 -- 172.21.15.33:0/3437671339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ffa9005eda0 con 0x7ffa900c6d90 2023-11-06T20:44:17.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.725+0000 7ffa7e7fc700 1 -- 172.21.15.33:0/3437671339 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffa84022090 con 0x7ffa900c6d90 2023-11-06T20:44:17.724 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-11-06T20:44:17.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 -- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7005f150 msgr2=0x7ffa70061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7005f150 0x7ffa70061610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7ffa9014edb0 tx=0x7ffa80009040).stop 2023-11-06T20:44:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 -- 172.21.15.33:0/3437671339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa900c6d90 msgr2=0x7ffa9014f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa900c6d90 0x7ffa9014f600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7ffa90083660 tx=0x7ffa84016000).stop 2023-11-06T20:44:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 -- 172.21.15.33:0/3437671339 shutdown_connections 2023-11-06T20:44:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7005f150 0x7ffa70061610 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa90154210 0x7ffa901521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa900bfd60 0x7ffa900c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 --2- 172.21.15.33:0/3437671339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa900c6d90 0x7ffa9014f600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 -- 172.21.15.33:0/3437671339 >> 172.21.15.33:0/3437671339 conn(0x7ffa9005e990 msgr2=0x7ffa900c9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 -- 172.21.15.33:0/3437671339 shutdown_connections 2023-11-06T20:44:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.727+0000 7ffa94bff700 1 -- 172.21.15.33:0/3437671339 wait complete. 2023-11-06T20:44:17.743 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2023-11-06T20:44:17.744 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-06T20:44:17.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:44:17.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.942+0000 7f4d2a7ff700 1 Processor -- start 2023-11-06T20:44:17.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.942+0000 7f4d2a7ff700 1 -- start start 2023-11-06T20:44:17.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d2a7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240c1840 0x7f4d240a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d2a7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4d240c6460 0x7f4d240bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d2a7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d241554a0 con 0x7f4d240aa2d0 2023-11-06T20:44:17.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d2a7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d24154960 con 0x7f4d240c6460 2023-11-06T20:44:17.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d2a7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24155930 con 0x7f4d240c1840 2023-11-06T20:44:17.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d237fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4d240aa2d0 0x7f4d240c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51634/0 (socket says 172.21.15.33:51634) 2023-11-06T20:44:17.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d237fe700 1 -- 172.21.15.33:0/1803204769 learned_addr learned my addr 172.21.15.33:0/1803204769 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:17.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d23fff700 1 --2- 172.21.15.33:0/1803204769 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240c1840 0x7f4d240a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d28d9c700 1 --2- 172.21.15.33:0/1803204769 >> v2:172.21.15.69:3300/0 conn(0x7f4d240c6460 0x7f4d240bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d22ffd700 1 -- 172.21.15.33:0/1803204769 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968290542 0 0) 0x7f4d241554a0 con 0x7f4d240aa2d0 2023-11-06T20:44:17.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.944+0000 7f4d22ffd700 1 -- 172.21.15.33:0/1803204769 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d0c0033f0 con 0x7f4d240aa2d0 2023-11-06T20:44:17.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.945+0000 7f4d28d9c700 1 -- 172.21.15.33:0/1803204769 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240c1840 msgr2=0x7f4d240a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:17.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.945+0000 7f4d28d9c700 1 --2- 172.21.15.33:0/1803204769 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240c1840 0x7f4d240a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.945+0000 7f4d28d9c700 1 -- 172.21.15.33:0/1803204769 >> v1:172.21.15.33:6789/0 conn(0x7f4d240aa2d0 legacy=0x7f4d240c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.945+0000 7f4d28d9c700 1 -- 172.21.15.33:0/1803204769 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d24154b70 con 0x7f4d240c6460 2023-11-06T20:44:17.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.945+0000 7f4d23fff700 1 --2- 172.21.15.33:0/1803204769 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240c1840 0x7f4d240a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:17.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.945+0000 7f4d28d9c700 1 --2- 172.21.15.33:0/1803204769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d240c6460 0x7f4d240bfd70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4d1c009e60 tx=0x7f4d1c009780).ready entity=mon.1 client_cookie=9b7e374888c87f7f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:17.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.945+0000 7f4d22ffd700 1 -- 172.21.15.33:0/1803204769 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c00e070 con 0x7f4d240c6460 2023-11-06T20:44:17.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.945+0000 7f4d22ffd700 1 -- 172.21.15.33:0/1803204769 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c00bab0 con 0x7f4d240c6460 2023-11-06T20:44:17.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.945+0000 7f4d22ffd700 1 -- 172.21.15.33:0/1803204769 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c00be30 con 0x7f4d240c6460 2023-11-06T20:44:17.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.946+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/1803204769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d240c6460 msgr2=0x7f4d240bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.946+0000 7f4d2a7ff700 1 --2- 172.21.15.33:0/1803204769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d240c6460 0x7f4d240bfd70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4d1c009e60 tx=0x7f4d1c009780).stop 2023-11-06T20:44:17.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.946+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/1803204769 shutdown_connections 2023-11-06T20:44:17.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.946+0000 7f4d2a7ff700 1 --2- 172.21.15.33:0/1803204769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d240c6460 0x7f4d240bfd70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.946+0000 7f4d2a7ff700 1 --2- 172.21.15.33:0/1803204769 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240c1840 0x7f4d240a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.946+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/1803204769 >> 172.21.15.33:0/1803204769 conn(0x7f4d2405ec80 msgr2=0x7f4d240cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:17.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.946+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/1803204769 shutdown_connections 2023-11-06T20:44:17.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.946+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/1803204769 wait complete. 2023-11-06T20:44:17.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.947+0000 7f4d2a7ff700 1 Processor -- start 2023-11-06T20:44:17.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.948+0000 7f4d2a7ff700 1 -- start start 2023-11-06T20:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.948+0000 7f4d2a7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240aa2d0 0x7f4d2413ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.948+0000 7f4d2a7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d240c1840 0x7f4d2413b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d2a7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d240c6460 0x7f4d24139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d2a7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24149dc0 con 0x7f4d240c6460 2023-11-06T20:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d2a7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24138cb0 con 0x7f4d240c1840 2023-11-06T20:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d2a7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24138f90 con 0x7f4d240aa2d0 2023-11-06T20:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d28d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d240c6460 0x7f4d24139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d28d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d240c6460 0x7f4d24139d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33556/0 (socket says 172.21.15.33:33556) 2023-11-06T20:44:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d28d9c700 1 -- 172.21.15.33:0/2474345520 learned_addr learned my addr 172.21.15.33:0/2474345520 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d23fff700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240aa2d0 0x7f4d2413ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d237fe700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d240c1840 0x7f4d2413b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d28d9c700 1 -- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240aa2d0 msgr2=0x7f4d2413ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d28d9c700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240aa2d0 0x7f4d2413ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d28d9c700 1 -- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d240c1840 msgr2=0x7f4d2413b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:17.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d28d9c700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d240c1840 0x7f4d2413b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:17.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d28d9c700 1 -- 172.21.15.33:0/2474345520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d140092a0 con 0x7f4d240c6460 2023-11-06T20:44:17.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.949+0000 7f4d28d9c700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d240c6460 0x7f4d24139d50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f4d1c0097c0 tx=0x7f4d24155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.950+0000 7f4d0bfff700 1 -- 172.21.15.33:0/2474345520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c01d630 con 0x7f4d240c6460 2023-11-06T20:44:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.950+0000 7f4d0bfff700 1 -- 172.21.15.33:0/2474345520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c01d790 con 0x7f4d240c6460 2023-11-06T20:44:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.950+0000 7f4d0bfff700 1 -- 172.21.15.33:0/2474345520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c01db30 con 0x7f4d240c6460 2023-11-06T20:44:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.950+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/2474345520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c009a20 con 0x7f4d240c6460 2023-11-06T20:44:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.950+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/2474345520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d24139550 con 0x7f4d240c6460 2023-11-06T20:44:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.951+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/2474345520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2406c6c0 con 0x7f4d240c6460 2023-11-06T20:44:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.952+0000 7f4d0bfff700 1 -- 172.21.15.33:0/2474345520 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4d1c027460 con 0x7f4d240c6460 2023-11-06T20:44:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.952+0000 7f4d0bfff700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d1005f150 0x7f4d10061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.952+0000 7f4d0bfff700 1 -- 172.21.15.33:0/2474345520 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4d1c07c160 con 0x7f4d240c6460 2023-11-06T20:44:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.953+0000 7f4d23fff700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d1005f150 0x7f4d10061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.953+0000 7f4d23fff700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d1005f150 0x7f4d10061610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f4d14009270 tx=0x7f4d14008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:17.956+0000 7f4d0bfff700 1 -- 172.21.15.33:0/2474345520 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4d1c04d3d0 con 0x7f4d240c6460 2023-11-06T20:44:18.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.134+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/2474345520 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d24063f20 con 0x7f4d1005f150 2023-11-06T20:44:18.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.145+0000 7f4d0bfff700 1 -- 172.21.15.33:0/2474345520 <== mgr.4103 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336367 (secure 0 0 0) 0x7f4d24063f20 con 0x7f4d1005f150 2023-11-06T20:44:18.149 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.149+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d1005f150 msgr2=0x7f4d10061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.149+0000 7f4d2a7ff700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d1005f150 0x7f4d10061610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f4d14009270 tx=0x7f4d14008040).stop 2023-11-06T20:44:18.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.149+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/2474345520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d240c6460 msgr2=0x7f4d24139d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.149+0000 7f4d2a7ff700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d240c6460 0x7f4d24139d50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f4d1c0097c0 tx=0x7f4d24155410).stop 2023-11-06T20:44:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.150+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/2474345520 shutdown_connections 2023-11-06T20:44:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.150+0000 7f4d2a7ff700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d1005f150 0x7f4d10061610 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.150+0000 7f4d2a7ff700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d240aa2d0 0x7f4d2413ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.150+0000 7f4d2a7ff700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d240c6460 0x7f4d24139d50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.150+0000 7f4d2a7ff700 1 --2- 172.21.15.33:0/2474345520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d240c1840 0x7f4d2413b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.150+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/2474345520 >> 172.21.15.33:0/2474345520 conn(0x7f4d2405ec80 msgr2=0x7f4d24155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:18.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.150+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/2474345520 shutdown_connections 2023-11-06T20:44:18.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.150+0000 7f4d2a7ff700 1 -- 172.21.15.33:0/2474345520 wait complete. 2023-11-06T20:44:18.155 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:44:18.166 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-06T20:44:16.809634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378528,"kb_used_data":1760,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596192,"statfs":{"total":773094113280,"available":770658500608,"internally_reserved":0,"allocated":1802240,"data_stored":646412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002175"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427856+0000","last_change":"2023-11-06T20:44:06.424455+0000","last_active":"2023-11-06T20:44:06.427856+0000","last_peered":"2023-11-06T20:44:06.427856+0000","last_clean":"2023-11-06T20:44:06.427856+0000","last_became_active":"2023-11-06T20:44:06.424120+0000","last_became_peered":"2023-11-06T20:44:06.424120+0000","last_unstale":"2023-11-06T20:44:06.427856+0000","last_undegraded":"2023-11-06T20:44:06.427856+0000","last_fullsized":"2023-11-06T20:44:06.427856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396063+0000","last_change":"2023-11-06T20:44:05.918231+0000","last_active":"2023-11-06T20:44:06.396063+0000","last_peered":"2023-11-06T20:44:06.396063+0000","last_clean":"2023-11-06T20:44:06.396063+0000","last_became_active":"2023-11-06T20:44:05.917999+0000","last_became_peered":"2023-11-06T20:44:05.917999+0000","last_unstale":"2023-11-06T20:44:06.396063+0000","last_undegraded":"2023-11-06T20:44:06.396063+0000","last_fullsized":"2023-11-06T20:44:06.396063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420202+0000","last_change":"2023-11-06T20:44:06.413802+0000","last_active":"2023-11-06T20:44:06.420202+0000","last_peered":"2023-11-06T20:44:06.420202+0000","last_clean":"2023-11-06T20:44:06.420202+0000","last_became_active":"2023-11-06T20:44:06.412830+0000","last_became_peered":"2023-11-06T20:44:06.412830+0000","last_unstale":"2023-11-06T20:44:06.420202+0000","last_undegraded":"2023-11-06T20:44:06.420202+0000","last_fullsized":"2023-11-06T20:44:06.420202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.402079+0000","last_change":"2023-11-06T20:44:05.922105+0000","last_active":"2023-11-06T20:44:06.402079+0000","last_peered":"2023-11-06T20:44:06.402079+0000","last_clean":"2023-11-06T20:44:06.402079+0000","last_became_active":"2023-11-06T20:44:05.921727+0000","last_became_peered":"2023-11-06T20:44:05.921727+0000","last_unstale":"2023-11-06T20:44:06.402079+0000","last_undegraded":"2023-11-06T20:44:06.402079+0000","last_fullsized":"2023-11-06T20:44:06.402079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426592+0000","last_change":"2023-11-06T20:44:06.411322+0000","last_active":"2023-11-06T20:44:06.426592+0000","last_peered":"2023-11-06T20:44:06.426592+0000","last_clean":"2023-11-06T20:44:06.426592+0000","last_became_active":"2023-11-06T20:44:06.409787+0000","last_became_peered":"2023-11-06T20:44:06.409787+0000","last_unstale":"2023-11-06T20:44:06.426592+0000","last_undegraded":"2023-11-06T20:44:06.426592+0000","last_fullsized":"2023-11-06T20:44:06.426592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393182+0000","last_change":"2023-11-06T20:44:05.920323+0000","last_active":"2023-11-06T20:44:06.393182+0000","last_peered":"2023-11-06T20:44:06.393182+0000","last_clean":"2023-11-06T20:44:06.393182+0000","last_became_active":"2023-11-06T20:44:05.919781+0000","last_became_peered":"2023-11-06T20:44:05.919781+0000","last_unstale":"2023-11-06T20:44:06.393182+0000","last_undegraded":"2023-11-06T20:44:06.393182+0000","last_fullsized":"2023-11-06T20:44:06.393182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.428239+0000","last_change":"2023-11-06T20:44:06.426498+0000","last_active":"2023-11-06T20:44:06.428239+0000","last_peered":"2023-11-06T20:44:06.428239+0000","last_clean":"2023-11-06T20:44:06.428239+0000","last_became_active":"2023-11-06T20:44:06.426148+0000","last_became_peered":"2023-11-06T20:44:06.426148+0000","last_unstale":"2023-11-06T20:44:06.428239+0000","last_undegraded":"2023-11-06T20:44:06.428239+0000","last_fullsized":"2023-11-06T20:44:06.428239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395364+0000","last_change":"2023-11-06T20:44:05.934770+0000","last_active":"2023-11-06T20:44:06.395364+0000","last_peered":"2023-11-06T20:44:06.395364+0000","last_clean":"2023-11-06T20:44:06.395364+0000","last_became_active":"2023-11-06T20:44:05.934479+0000","last_became_peered":"2023-11-06T20:44:05.934479+0000","last_unstale":"2023-11-06T20:44:06.395364+0000","last_undegraded":"2023-11-06T20:44:06.395364+0000","last_fullsized":"2023-11-06T20:44:06.395364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425272+0000","last_change":"2023-11-06T20:44:06.422189+0000","last_active":"2023-11-06T20:44:06.425272+0000","last_peered":"2023-11-06T20:44:06.425272+0000","last_clean":"2023-11-06T20:44:06.425272+0000","last_became_active":"2023-11-06T20:44:06.421946+0000","last_became_peered":"2023-11-06T20:44:06.421946+0000","last_unstale":"2023-11-06T20:44:06.425272+0000","last_undegraded":"2023-11-06T20:44:06.425272+0000","last_fullsized":"2023-11-06T20:44:06.425272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395338+0000","last_change":"2023-11-06T20:44:05.930203+0000","last_active":"2023-11-06T20:44:06.395338+0000","last_peered":"2023-11-06T20:44:06.395338+0000","last_clean":"2023-11-06T20:44:06.395338+0000","last_became_active":"2023-11-06T20:44:05.929810+0000","last_became_peered":"2023-11-06T20:44:05.929810+0000","last_unstale":"2023-11-06T20:44:06.395338+0000","last_undegraded":"2023-11-06T20:44:06.395338+0000","last_fullsized":"2023-11-06T20:44:06.395338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.428278+0000","last_change":"2023-11-06T20:44:06.425515+0000","last_active":"2023-11-06T20:44:06.428278+0000","last_peered":"2023-11-06T20:44:06.428278+0000","last_clean":"2023-11-06T20:44:06.428278+0000","last_became_active":"2023-11-06T20:44:06.425144+0000","last_became_peered":"2023-11-06T20:44:06.425144+0000","last_unstale":"2023-11-06T20:44:06.428278+0000","last_undegraded":"2023-11-06T20:44:06.428278+0000","last_fullsized":"2023-11-06T20:44:06.428278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424086+0000","last_change":"2023-11-06T20:44:05.930915+0000","last_active":"2023-11-06T20:44:06.424086+0000","last_peered":"2023-11-06T20:44:06.424086+0000","last_clean":"2023-11-06T20:44:06.424086+0000","last_became_active":"2023-11-06T20:44:05.928587+0000","last_became_peered":"2023-11-06T20:44:05.928587+0000","last_unstale":"2023-11-06T20:44:06.424086+0000","last_undegraded":"2023-11-06T20:44:06.424086+0000","last_fullsized":"2023-11-06T20:44:06.424086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416540+0000","last_change":"2023-11-06T20:44:06.406652+0000","last_active":"2023-11-06T20:44:06.416540+0000","last_peered":"2023-11-06T20:44:06.416540+0000","last_clean":"2023-11-06T20:44:06.416540+0000","last_became_active":"2023-11-06T20:44:06.404607+0000","last_became_peered":"2023-11-06T20:44:06.404607+0000","last_unstale":"2023-11-06T20:44:06.416540+0000","last_undegraded":"2023-11-06T20:44:06.416540+0000","last_fullsized":"2023-11-06T20:44:06.416540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.428170+0000","last_change":"2023-11-06T20:44:05.921294+0000","last_active":"2023-11-06T20:44:06.428170+0000","last_peered":"2023-11-06T20:44:06.428170+0000","last_clean":"2023-11-06T20:44:06.428170+0000","last_became_active":"2023-11-06T20:44:05.921040+0000","last_became_peered":"2023-11-06T20:44:05.921040+0000","last_unstale":"2023-11-06T20:44:06.428170+0000","last_undegraded":"2023-11-06T20:44:06.428170+0000","last_fullsized":"2023-11-06T20:44:06.428170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420418+0000","last_change":"2023-11-06T20:44:06.414350+0000","last_active":"2023-11-06T20:44:06.420418+0000","last_peered":"2023-11-06T20:44:06.420418+0000","last_clean":"2023-11-06T20:44:06.420418+0000","last_became_active":"2023-11-06T20:44:06.413471+0000","last_became_peered":"2023-11-06T20:44:06.413471+0000","last_unstale":"2023-11-06T20:44:06.420418+0000","last_undegraded":"2023-11-06T20:44:06.420418+0000","last_fullsized":"2023-11-06T20:44:06.420418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427433+0000","last_change":"2023-11-06T20:44:05.931968+0000","last_active":"2023-11-06T20:44:06.427433+0000","last_peered":"2023-11-06T20:44:06.427433+0000","last_clean":"2023-11-06T20:44:06.427433+0000","last_became_active":"2023-11-06T20:44:05.931666+0000","last_became_peered":"2023-11-06T20:44:05.931666+0000","last_unstale":"2023-11-06T20:44:06.427433+0000","last_undegraded":"2023-11-06T20:44:06.427433+0000","last_fullsized":"2023-11-06T20:44:06.427433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427742+0000","last_change":"2023-11-06T20:44:06.421789+0000","last_active":"2023-11-06T20:44:06.427742+0000","last_peered":"2023-11-06T20:44:06.427742+0000","last_clean":"2023-11-06T20:44:06.427742+0000","last_became_active":"2023-11-06T20:44:06.421065+0000","last_became_peered":"2023-11-06T20:44:06.421065+0000","last_unstale":"2023-11-06T20:44:06.427742+0000","last_undegraded":"2023-11-06T20:44:06.427742+0000","last_fullsized":"2023-11-06T20:44:06.427742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.398924+0000","last_change":"2023-11-06T20:44:05.936868+0000","last_active":"2023-11-06T20:44:06.398924+0000","last_peered":"2023-11-06T20:44:06.398924+0000","last_clean":"2023-11-06T20:44:06.398924+0000","last_became_active":"2023-11-06T20:44:05.936617+0000","last_became_peered":"2023-11-06T20:44:05.936617+0000","last_unstale":"2023-11-06T20:44:06.398924+0000","last_undegraded":"2023-11-06T20:44:06.398924+0000","last_fullsized":"2023-11-06T20:44:06.398924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421281+0000","last_change":"2023-11-06T20:44:06.409854+0000","last_active":"2023-11-06T20:44:06.421281+0000","last_peered":"2023-11-06T20:44:06.421281+0000","last_clean":"2023-11-06T20:44:06.421281+0000","last_became_active":"2023-11-06T20:44:06.409400+0000","last_became_peered":"2023-11-06T20:44:06.409400+0000","last_unstale":"2023-11-06T20:44:06.421281+0000","last_undegraded":"2023-11-06T20:44:06.421281+0000","last_fullsized":"2023-11-06T20:44:06.421281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.390859+0000","last_change":"2023-11-06T20:44:05.920464+0000","last_active":"2023-11-06T20:44:06.390859+0000","last_peered":"2023-11-06T20:44:06.390859+0000","last_clean":"2023-11-06T20:44:06.390859+0000","last_became_active":"2023-11-06T20:44:05.920179+0000","last_became_peered":"2023-11-06T20:44:05.920179+0000","last_unstale":"2023-11-06T20:44:06.390859+0000","last_undegraded":"2023-11-06T20:44:06.390859+0000","last_fullsized":"2023-11-06T20:44:06.390859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.410888+0000","last_change":"2023-11-06T20:44:06.404170+0000","last_active":"2023-11-06T20:44:06.410888+0000","last_peered":"2023-11-06T20:44:06.410888+0000","last_clean":"2023-11-06T20:44:06.410888+0000","last_became_active":"2023-11-06T20:44:06.403838+0000","last_became_peered":"2023-11-06T20:44:06.403838+0000","last_unstale":"2023-11-06T20:44:06.410888+0000","last_undegraded":"2023-11-06T20:44:06.410888+0000","last_fullsized":"2023-11-06T20:44:06.410888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427011+0000","last_change":"2023-11-06T20:44:05.909856+0000","last_active":"2023-11-06T20:44:06.427011+0000","last_peered":"2023-11-06T20:44:06.427011+0000","last_clean":"2023-11-06T20:44:06.427011+0000","last_became_active":"2023-11-06T20:44:05.909547+0000","last_became_peered":"2023-11-06T20:44:05.909547+0000","last_unstale":"2023-11-06T20:44:06.427011+0000","last_undegraded":"2023-11-06T20:44:06.427011+0000","last_fullsized":"2023-11-06T20:44:06.427011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412684+0000","last_change":"2023-11-06T20:44:06.401237+0000","last_active":"2023-11-06T20:44:06.412684+0000","last_peered":"2023-11-06T20:44:06.412684+0000","last_clean":"2023-11-06T20:44:06.412684+0000","last_became_active":"2023-11-06T20:44:06.399700+0000","last_became_peered":"2023-11-06T20:44:06.399700+0000","last_unstale":"2023-11-06T20:44:06.412684+0000","last_undegraded":"2023-11-06T20:44:06.412684+0000","last_fullsized":"2023-11-06T20:44:06.412684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421048+0000","last_change":"2023-11-06T20:44:05.930204+0000","last_active":"2023-11-06T20:44:06.421048+0000","last_peered":"2023-11-06T20:44:06.421048+0000","last_clean":"2023-11-06T20:44:06.421048+0000","last_became_active":"2023-11-06T20:44:05.929904+0000","last_became_peered":"2023-11-06T20:44:05.929904+0000","last_unstale":"2023-11-06T20:44:06.421048+0000","last_undegraded":"2023-11-06T20:44:06.421048+0000","last_fullsized":"2023-11-06T20:44:06.421048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.417505+0000","last_change":"2023-11-06T20:44:06.412089+0000","last_active":"2023-11-06T20:44:06.417505+0000","last_peered":"2023-11-06T20:44:06.417505+0000","last_clean":"2023-11-06T20:44:06.417505+0000","last_became_active":"2023-11-06T20:44:06.410838+0000","last_became_peered":"2023-11-06T20:44:06.410838+0000","last_unstale":"2023-11-06T20:44:06.417505+0000","last_undegraded":"2023-11-06T20:44:06.417505+0000","last_fullsized":"2023-11-06T20:44:06.417505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421847+0000","last_change":"2023-11-06T20:44:05.926188+0000","last_active":"2023-11-06T20:44:06.421847+0000","last_peered":"2023-11-06T20:44:06.421847+0000","last_clean":"2023-11-06T20:44:06.421847+0000","last_became_active":"2023-11-06T20:44:05.925868+0000","last_became_peered":"2023-11-06T20:44:05.925868+0000","last_unstale":"2023-11-06T20:44:06.421847+0000","last_undegraded":"2023-11-06T20:44:06.421847+0000","last_fullsized":"2023-11-06T20:44:06.421847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.401739+0000","last_change":"2023-11-06T20:44:06.397805+0000","last_active":"2023-11-06T20:44:06.401739+0000","last_peered":"2023-11-06T20:44:06.401739+0000","last_clean":"2023-11-06T20:44:06.401739+0000","last_became_active":"2023-11-06T20:44:06.397479+0000","last_became_peered":"2023-11-06T20:44:06.397479+0000","last_unstale":"2023-11-06T20:44:06.401739+0000","last_undegraded":"2023-11-06T20:44:06.401739+0000","last_fullsized":"2023-11-06T20:44:06.401739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389927+0000","last_change":"2023-11-06T20:44:05.925515+0000","last_active":"2023-11-06T20:44:06.389927+0000","last_peered":"2023-11-06T20:44:06.389927+0000","last_clean":"2023-11-06T20:44:06.389927+0000","last_became_active":"2023-11-06T20:44:05.921696+0000","last_became_peered":"2023-11-06T20:44:05.921696+0000","last_unstale":"2023-11-06T20:44:06.389927+0000","last_undegraded":"2023-11-06T20:44:06.389927+0000","last_fullsized":"2023-11-06T20:44:06.389927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425392+0000","last_change":"2023-11-06T20:44:06.421420+0000","last_active":"2023-11-06T20:44:06.425392+0000","last_peered":"2023-11-06T20:44:06.425392+0000","last_clean":"2023-11-06T20:44:06.425392+0000","last_became_active":"2023-11-06T20:44:06.421106+0000","last_became_peered":"2023-11-06T20:44:06.421106+0000","last_unstale":"2023-11-06T20:44:06.425392+0000","last_undegraded":"2023-11-06T20:44:06.425392+0000","last_fullsized":"2023-11-06T20:44:06.425392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404911+0000","last_change":"2023-11-06T20:44:05.921460+0000","last_active":"2023-11-06T20:44:06.404911+0000","last_peered":"2023-11-06T20:44:06.404911+0000","last_clean":"2023-11-06T20:44:06.404911+0000","last_became_active":"2023-11-06T20:44:05.921090+0000","last_became_peered":"2023-11-06T20:44:05.921090+0000","last_unstale":"2023-11-06T20:44:06.404911+0000","last_undegraded":"2023-11-06T20:44:06.404911+0000","last_fullsized":"2023-11-06T20:44:06.404911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421245+0000","last_change":"2023-11-06T20:44:06.415154+0000","last_active":"2023-11-06T20:44:06.421245+0000","last_peered":"2023-11-06T20:44:06.421245+0000","last_clean":"2023-11-06T20:44:06.421245+0000","last_became_active":"2023-11-06T20:44:06.414473+0000","last_became_peered":"2023-11-06T20:44:06.414473+0000","last_unstale":"2023-11-06T20:44:06.421245+0000","last_undegraded":"2023-11-06T20:44:06.421245+0000","last_fullsized":"2023-11-06T20:44:06.421245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420865+0000","last_change":"2023-11-06T20:44:05.922405+0000","last_active":"2023-11-06T20:44:06.420865+0000","last_peered":"2023-11-06T20:44:06.420865+0000","last_clean":"2023-11-06T20:44:06.420865+0000","last_became_active":"2023-11-06T20:44:05.921125+0000","last_became_peered":"2023-11-06T20:44:05.921125+0000","last_unstale":"2023-11-06T20:44:06.420865+0000","last_undegraded":"2023-11-06T20:44:06.420865+0000","last_fullsized":"2023-11-06T20:44:06.420865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407539+0000","last_change":"2023-11-06T20:44:06.399863+0000","last_active":"2023-11-06T20:44:06.407539+0000","last_peered":"2023-11-06T20:44:06.407539+0000","last_clean":"2023-11-06T20:44:06.407539+0000","last_became_active":"2023-11-06T20:44:06.399513+0000","last_became_peered":"2023-11-06T20:44:06.399513+0000","last_unstale":"2023-11-06T20:44:06.407539+0000","last_undegraded":"2023-11-06T20:44:06.407539+0000","last_fullsized":"2023-11-06T20:44:06.407539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388728+0000","last_change":"2023-11-06T20:44:05.936043+0000","last_active":"2023-11-06T20:44:06.388728+0000","last_peered":"2023-11-06T20:44:06.388728+0000","last_clean":"2023-11-06T20:44:06.388728+0000","last_became_active":"2023-11-06T20:44:05.935749+0000","last_became_peered":"2023-11-06T20:44:05.935749+0000","last_unstale":"2023-11-06T20:44:06.388728+0000","last_undegraded":"2023-11-06T20:44:06.388728+0000","last_fullsized":"2023-11-06T20:44:06.388728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412317+0000","last_change":"2023-11-06T20:44:06.405453+0000","last_active":"2023-11-06T20:44:06.412317+0000","last_peered":"2023-11-06T20:44:06.412317+0000","last_clean":"2023-11-06T20:44:06.412317+0000","last_became_active":"2023-11-06T20:44:06.405101+0000","last_became_peered":"2023-11-06T20:44:06.405101+0000","last_unstale":"2023-11-06T20:44:06.412317+0000","last_undegraded":"2023-11-06T20:44:06.412317+0000","last_fullsized":"2023-11-06T20:44:06.412317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389916+0000","last_change":"2023-11-06T20:44:05.929392+0000","last_active":"2023-11-06T20:44:06.389916+0000","last_peered":"2023-11-06T20:44:06.389916+0000","last_clean":"2023-11-06T20:44:06.389916+0000","last_became_active":"2023-11-06T20:44:05.928892+0000","last_became_peered":"2023-11-06T20:44:05.928892+0000","last_unstale":"2023-11-06T20:44:06.389916+0000","last_undegraded":"2023-11-06T20:44:06.389916+0000","last_fullsized":"2023-11-06T20:44:06.389916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420520+0000","last_change":"2023-11-06T20:44:06.415261+0000","last_active":"2023-11-06T20:44:06.420520+0000","last_peered":"2023-11-06T20:44:06.420520+0000","last_clean":"2023-11-06T20:44:06.420520+0000","last_became_active":"2023-11-06T20:44:06.410860+0000","last_became_peered":"2023-11-06T20:44:06.410860+0000","last_unstale":"2023-11-06T20:44:06.420520+0000","last_undegraded":"2023-11-06T20:44:06.420520+0000","last_fullsized":"2023-11-06T20:44:06.420520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.406449+0000","last_change":"2023-11-06T20:44:05.912367+0000","last_active":"2023-11-06T20:44:06.406449+0000","last_peered":"2023-11-06T20:44:06.406449+0000","last_clean":"2023-11-06T20:44:06.406449+0000","last_became_active":"2023-11-06T20:44:05.911944+0000","last_became_peered":"2023-11-06T20:44:05.911944+0000","last_unstale":"2023-11-06T20:44:06.406449+0000","last_undegraded":"2023-11-06T20:44:06.406449+0000","last_fullsized":"2023-11-06T20:44:06.406449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393110+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:44:06.393110+0000","last_peered":"2023-11-06T20:44:06.393110+0000","last_clean":"2023-11-06T20:44:06.393110+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:44:06.393110+0000","last_undegraded":"2023-11-06T20:44:06.393110+0000","last_fullsized":"2023-11-06T20:44:06.393110+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393545+0000","last_change":"2023-11-06T20:44:05.916204+0000","last_active":"2023-11-06T20:44:06.393545+0000","last_peered":"2023-11-06T20:44:06.393545+0000","last_clean":"2023-11-06T20:44:06.393545+0000","last_became_active":"2023-11-06T20:44:05.915592+0000","last_became_peered":"2023-11-06T20:44:05.915592+0000","last_unstale":"2023-11-06T20:44:06.393545+0000","last_undegraded":"2023-11-06T20:44:06.393545+0000","last_fullsized":"2023-11-06T20:44:06.393545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.422726+0000","last_change":"2023-11-06T20:44:06.418081+0000","last_active":"2023-11-06T20:44:06.422726+0000","last_peered":"2023-11-06T20:44:06.422726+0000","last_clean":"2023-11-06T20:44:06.422726+0000","last_became_active":"2023-11-06T20:44:06.416191+0000","last_became_peered":"2023-11-06T20:44:06.416191+0000","last_unstale":"2023-11-06T20:44:06.422726+0000","last_undegraded":"2023-11-06T20:44:06.422726+0000","last_fullsized":"2023-11-06T20:44:06.422726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388876+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:44:06.388876+0000","last_peered":"2023-11-06T20:44:06.388876+0000","last_clean":"2023-11-06T20:44:06.388876+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:44:06.388876+0000","last_undegraded":"2023-11-06T20:44:06.388876+0000","last_fullsized":"2023-11-06T20:44:06.388876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404240+0000","last_change":"2023-11-06T20:44:05.926886+0000","last_active":"2023-11-06T20:44:06.404240+0000","last_peered":"2023-11-06T20:44:06.404240+0000","last_clean":"2023-11-06T20:44:06.404240+0000","last_became_active":"2023-11-06T20:44:05.925506+0000","last_became_peered":"2023-11-06T20:44:05.925506+0000","last_unstale":"2023-11-06T20:44:06.404240+0000","last_undegraded":"2023-11-06T20:44:06.404240+0000","last_fullsized":"2023-11-06T20:44:06.404240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420444+0000","last_change":"2023-11-06T20:44:06.415342+0000","last_active":"2023-11-06T20:44:06.420444+0000","last_peered":"2023-11-06T20:44:06.420444+0000","last_clean":"2023-11-06T20:44:06.420444+0000","last_became_active":"2023-11-06T20:44:06.414787+0000","last_became_peered":"2023-11-06T20:44:06.414787+0000","last_unstale":"2023-11-06T20:44:06.420444+0000","last_undegraded":"2023-11-06T20:44:06.420444+0000","last_fullsized":"2023-11-06T20:44:06.420444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425033+0000","last_change":"2023-11-06T20:44:06.422594+0000","last_active":"2023-11-06T20:44:06.425033+0000","last_peered":"2023-11-06T20:44:06.425033+0000","last_clean":"2023-11-06T20:44:06.425033+0000","last_became_active":"2023-11-06T20:44:06.422264+0000","last_became_peered":"2023-11-06T20:44:06.422264+0000","last_unstale":"2023-11-06T20:44:06.425033+0000","last_undegraded":"2023-11-06T20:44:06.425033+0000","last_fullsized":"2023-11-06T20:44:06.425033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.400201+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:44:06.400201+0000","last_peered":"2023-11-06T20:44:06.400201+0000","last_clean":"2023-11-06T20:44:06.400201+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:44:06.400201+0000","last_undegraded":"2023-11-06T20:44:06.400201+0000","last_fullsized":"2023-11-06T20:44:06.400201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.392931+0000","last_change":"2023-11-06T20:44:05.912746+0000","last_active":"2023-11-06T20:44:06.392931+0000","last_peered":"2023-11-06T20:44:06.392931+0000","last_clean":"2023-11-06T20:44:06.392931+0000","last_became_active":"2023-11-06T20:44:05.912443+0000","last_became_peered":"2023-11-06T20:44:05.912443+0000","last_unstale":"2023-11-06T20:44:06.392931+0000","last_undegraded":"2023-11-06T20:44:06.392931+0000","last_fullsized":"2023-11-06T20:44:06.392931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412853+0000","last_change":"2023-11-06T20:44:06.400524+0000","last_active":"2023-11-06T20:44:06.412853+0000","last_peered":"2023-11-06T20:44:06.412853+0000","last_clean":"2023-11-06T20:44:06.412853+0000","last_became_active":"2023-11-06T20:44:06.399624+0000","last_became_peered":"2023-11-06T20:44:06.399624+0000","last_unstale":"2023-11-06T20:44:06.412853+0000","last_undegraded":"2023-11-06T20:44:06.412853+0000","last_fullsized":"2023-11-06T20:44:06.412853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388619+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:44:06.388619+0000","last_peered":"2023-11-06T20:44:06.388619+0000","last_clean":"2023-11-06T20:44:06.388619+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:44:06.388619+0000","last_undegraded":"2023-11-06T20:44:06.388619+0000","last_fullsized":"2023-11-06T20:44:06.388619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395865+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:44:06.395865+0000","last_peered":"2023-11-06T20:44:06.395865+0000","last_clean":"2023-11-06T20:44:06.395865+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:44:06.395865+0000","last_undegraded":"2023-11-06T20:44:06.395865+0000","last_fullsized":"2023-11-06T20:44:06.395865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.401330+0000","last_change":"2023-11-06T20:44:05.915540+0000","last_active":"2023-11-06T20:44:06.401330+0000","last_peered":"2023-11-06T20:44:06.401330+0000","last_clean":"2023-11-06T20:44:06.401330+0000","last_became_active":"2023-11-06T20:44:05.915264+0000","last_became_peered":"2023-11-06T20:44:05.915264+0000","last_unstale":"2023-11-06T20:44:06.401330+0000","last_undegraded":"2023-11-06T20:44:06.401330+0000","last_fullsized":"2023-11-06T20:44:06.401330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.414058+0000","last_change":"2023-11-06T20:44:06.406952+0000","last_active":"2023-11-06T20:44:06.414058+0000","last_peered":"2023-11-06T20:44:06.414058+0000","last_clean":"2023-11-06T20:44:06.414058+0000","last_became_active":"2023-11-06T20:44:06.406478+0000","last_became_peered":"2023-11-06T20:44:06.406478+0000","last_unstale":"2023-11-06T20:44:06.414058+0000","last_undegraded":"2023-11-06T20:44:06.414058+0000","last_fullsized":"2023-11-06T20:44:06.414058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389490+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:44:06.389490+0000","last_peered":"2023-11-06T20:44:06.389490+0000","last_clean":"2023-11-06T20:44:06.389490+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:44:06.389490+0000","last_undegraded":"2023-11-06T20:44:06.389490+0000","last_fullsized":"2023-11-06T20:44:06.389490+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396293+0000","last_change":"2023-11-06T20:44:05.916288+0000","last_active":"2023-11-06T20:44:06.396293+0000","last_peered":"2023-11-06T20:44:06.396293+0000","last_clean":"2023-11-06T20:44:06.396293+0000","last_became_active":"2023-11-06T20:44:05.915873+0000","last_became_peered":"2023-11-06T20:44:05.915873+0000","last_unstale":"2023-11-06T20:44:06.396293+0000","last_undegraded":"2023-11-06T20:44:06.396293+0000","last_fullsized":"2023-11-06T20:44:06.396293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.390009+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:44:06.390009+0000","last_peered":"2023-11-06T20:44:06.390009+0000","last_clean":"2023-11-06T20:44:06.390009+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:44:06.390009+0000","last_undegraded":"2023-11-06T20:44:06.390009+0000","last_fullsized":"2023-11-06T20:44:06.390009+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399349+0000","last_change":"2023-11-06T20:44:05.914799+0000","last_active":"2023-11-06T20:44:06.399349+0000","last_peered":"2023-11-06T20:44:06.399349+0000","last_clean":"2023-11-06T20:44:06.399349+0000","last_became_active":"2023-11-06T20:44:05.914449+0000","last_became_peered":"2023-11-06T20:44:05.914449+0000","last_unstale":"2023-11-06T20:44:06.399349+0000","last_undegraded":"2023-11-06T20:44:06.399349+0000","last_fullsized":"2023-11-06T20:44:06.399349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412824+0000","last_change":"2023-11-06T20:44:06.408012+0000","last_active":"2023-11-06T20:44:06.412824+0000","last_peered":"2023-11-06T20:44:06.412824+0000","last_clean":"2023-11-06T20:44:06.412824+0000","last_became_active":"2023-11-06T20:44:06.407679+0000","last_became_peered":"2023-11-06T20:44:06.407679+0000","last_unstale":"2023-11-06T20:44:06.412824+0000","last_undegraded":"2023-11-06T20:44:06.412824+0000","last_fullsized":"2023-11-06T20:44:06.412824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421500+0000","last_change":"2023-11-06T20:44:05.919819+0000","last_active":"2023-11-06T20:44:06.421500+0000","last_peered":"2023-11-06T20:44:06.421500+0000","last_clean":"2023-11-06T20:44:06.421500+0000","last_became_active":"2023-11-06T20:44:05.919286+0000","last_became_peered":"2023-11-06T20:44:05.919286+0000","last_unstale":"2023-11-06T20:44:06.421500+0000","last_undegraded":"2023-11-06T20:44:06.421500+0000","last_fullsized":"2023-11-06T20:44:06.421500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.405378+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:44:06.405378+0000","last_peered":"2023-11-06T20:44:06.405378+0000","last_clean":"2023-11-06T20:44:06.405378+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:44:06.405378+0000","last_undegraded":"2023-11-06T20:44:06.405378+0000","last_fullsized":"2023-11-06T20:44:06.405378+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.423170+0000","last_change":"2023-11-06T20:44:06.420584+0000","last_active":"2023-11-06T20:44:06.423170+0000","last_peered":"2023-11-06T20:44:06.423170+0000","last_clean":"2023-11-06T20:44:06.423170+0000","last_became_active":"2023-11-06T20:44:06.420289+0000","last_became_peered":"2023-11-06T20:44:06.420289+0000","last_unstale":"2023-11-06T20:44:06.423170+0000","last_undegraded":"2023-11-06T20:44:06.423170+0000","last_fullsized":"2023-11-06T20:44:06.423170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389064+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:44:06.389064+0000","last_peered":"2023-11-06T20:44:06.389064+0000","last_clean":"2023-11-06T20:44:06.389064+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:44:06.389064+0000","last_undegraded":"2023-11-06T20:44:06.389064+0000","last_fullsized":"2023-11-06T20:44:06.389064+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.405496+0000","last_change":"2023-11-06T20:44:05.911916+0000","last_active":"2023-11-06T20:44:06.405496+0000","last_peered":"2023-11-06T20:44:06.405496+0000","last_clean":"2023-11-06T20:44:06.405496+0000","last_became_active":"2023-11-06T20:44:05.911373+0000","last_became_peered":"2023-11-06T20:44:05.911373+0000","last_unstale":"2023-11-06T20:44:06.405496+0000","last_undegraded":"2023-11-06T20:44:06.405496+0000","last_fullsized":"2023-11-06T20:44:06.405496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.402668+0000","last_change":"2023-11-06T20:44:06.399566+0000","last_active":"2023-11-06T20:44:06.402668+0000","last_peered":"2023-11-06T20:44:06.402668+0000","last_clean":"2023-11-06T20:44:06.402668+0000","last_became_active":"2023-11-06T20:44:06.399283+0000","last_became_peered":"2023-11-06T20:44:06.399283+0000","last_unstale":"2023-11-06T20:44:06.402668+0000","last_undegraded":"2023-11-06T20:44:06.402668+0000","last_fullsized":"2023-11-06T20:44:06.402668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420260+0000","last_change":"2023-11-06T20:44:06.413736+0000","last_active":"2023-11-06T20:44:06.420260+0000","last_peered":"2023-11-06T20:44:06.420260+0000","last_clean":"2023-11-06T20:44:06.420260+0000","last_became_active":"2023-11-06T20:44:06.413520+0000","last_became_peered":"2023-11-06T20:44:06.413520+0000","last_unstale":"2023-11-06T20:44:06.420260+0000","last_undegraded":"2023-11-06T20:44:06.420260+0000","last_fullsized":"2023-11-06T20:44:06.420260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.403806+0000","last_change":"2023-11-06T20:44:05.929692+0000","last_active":"2023-11-06T20:44:06.403806+0000","last_peered":"2023-11-06T20:44:06.403806+0000","last_clean":"2023-11-06T20:44:06.403806+0000","last_became_active":"2023-11-06T20:44:05.929364+0000","last_became_peered":"2023-11-06T20:44:05.929364+0000","last_unstale":"2023-11-06T20:44:06.403806+0000","last_undegraded":"2023-11-06T20:44:06.403806+0000","last_fullsized":"2023-11-06T20:44:06.403806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416558+0000","last_change":"2023-11-06T20:44:06.407233+0000","last_active":"2023-11-06T20:44:06.416558+0000","last_peered":"2023-11-06T20:44:06.416558+0000","last_clean":"2023-11-06T20:44:06.416558+0000","last_became_active":"2023-11-06T20:44:06.406927+0000","last_became_peered":"2023-11-06T20:44:06.406927+0000","last_unstale":"2023-11-06T20:44:06.416558+0000","last_undegraded":"2023-11-06T20:44:06.416558+0000","last_fullsized":"2023-11-06T20:44:06.416558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396117+0000","last_change":"2023-11-06T20:44:05.925954+0000","last_active":"2023-11-06T20:44:06.396117+0000","last_peered":"2023-11-06T20:44:06.396117+0000","last_clean":"2023-11-06T20:44:06.396117+0000","last_became_active":"2023-11-06T20:44:05.925626+0000","last_became_peered":"2023-11-06T20:44:05.925626+0000","last_unstale":"2023-11-06T20:44:06.396117+0000","last_undegraded":"2023-11-06T20:44:06.396117+0000","last_fullsized":"2023-11-06T20:44:06.396117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420472+0000","last_change":"2023-11-06T20:44:06.417902+0000","last_active":"2023-11-06T20:44:06.420472+0000","last_peered":"2023-11-06T20:44:06.420472+0000","last_clean":"2023-11-06T20:44:06.420472+0000","last_became_active":"2023-11-06T20:44:06.417309+0000","last_became_peered":"2023-11-06T20:44:06.417309+0000","last_unstale":"2023-11-06T20:44:06.420472+0000","last_undegraded":"2023-11-06T20:44:06.420472+0000","last_fullsized":"2023-11-06T20:44:06.420472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427611+0000","last_change":"2023-11-06T20:44:05.935705+0000","last_active":"2023-11-06T20:44:06.427611+0000","last_peered":"2023-11-06T20:44:06.427611+0000","last_clean":"2023-11-06T20:44:06.427611+0000","last_became_active":"2023-11-06T20:44:05.935436+0000","last_became_peered":"2023-11-06T20:44:05.935436+0000","last_unstale":"2023-11-06T20:44:06.427611+0000","last_undegraded":"2023-11-06T20:44:06.427611+0000","last_fullsized":"2023-11-06T20:44:06.427611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419962+0000","last_change":"2023-11-06T20:44:06.414767+0000","last_active":"2023-11-06T20:44:06.419962+0000","last_peered":"2023-11-06T20:44:06.419962+0000","last_clean":"2023-11-06T20:44:06.419962+0000","last_became_active":"2023-11-06T20:44:06.414452+0000","last_became_peered":"2023-11-06T20:44:06.414452+0000","last_unstale":"2023-11-06T20:44:06.419962+0000","last_undegraded":"2023-11-06T20:44:06.419962+0000","last_fullsized":"2023-11-06T20:44:06.419962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.430730+0000","last_change":"2023-11-06T20:44:05.916430+0000","last_active":"2023-11-06T20:44:06.430730+0000","last_peered":"2023-11-06T20:44:06.430730+0000","last_clean":"2023-11-06T20:44:06.430730+0000","last_became_active":"2023-11-06T20:44:05.916163+0000","last_became_peered":"2023-11-06T20:44:05.916163+0000","last_unstale":"2023-11-06T20:44:06.430730+0000","last_undegraded":"2023-11-06T20:44:06.430730+0000","last_fullsized":"2023-11-06T20:44:06.430730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.415635+0000","last_change":"2023-11-06T20:44:06.412706+0000","last_active":"2023-11-06T20:44:06.415635+0000","last_peered":"2023-11-06T20:44:06.415635+0000","last_clean":"2023-11-06T20:44:06.415635+0000","last_became_active":"2023-11-06T20:44:06.412286+0000","last_became_peered":"2023-11-06T20:44:06.412286+0000","last_unstale":"2023-11-06T20:44:06.415635+0000","last_undegraded":"2023-11-06T20:44:06.415635+0000","last_fullsized":"2023-11-06T20:44:06.415635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.430724+0000","last_change":"2023-11-06T20:44:05.920163+0000","last_active":"2023-11-06T20:44:06.430724+0000","last_peered":"2023-11-06T20:44:06.430724+0000","last_clean":"2023-11-06T20:44:06.430724+0000","last_became_active":"2023-11-06T20:44:05.919817+0000","last_became_peered":"2023-11-06T20:44:05.919817+0000","last_unstale":"2023-11-06T20:44:06.430724+0000","last_undegraded":"2023-11-06T20:44:06.430724+0000","last_fullsized":"2023-11-06T20:44:06.430724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420853+0000","last_change":"2023-11-06T20:44:06.415900+0000","last_active":"2023-11-06T20:44:06.420853+0000","last_peered":"2023-11-06T20:44:06.420853+0000","last_clean":"2023-11-06T20:44:06.420853+0000","last_became_active":"2023-11-06T20:44:06.415623+0000","last_became_peered":"2023-11-06T20:44:06.415623+0000","last_unstale":"2023-11-06T20:44:06.420853+0000","last_undegraded":"2023-11-06T20:44:06.420853+0000","last_fullsized":"2023-11-06T20:44:06.420853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.394177+0000","last_change":"2023-11-06T20:44:05.916945+0000","last_active":"2023-11-06T20:44:06.394177+0000","last_peered":"2023-11-06T20:44:06.394177+0000","last_clean":"2023-11-06T20:44:06.394177+0000","last_became_active":"2023-11-06T20:44:05.916459+0000","last_became_peered":"2023-11-06T20:44:05.916459+0000","last_unstale":"2023-11-06T20:44:06.394177+0000","last_undegraded":"2023-11-06T20:44:06.394177+0000","last_fullsized":"2023-11-06T20:44:06.394177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420482+0000","last_change":"2023-11-06T20:44:06.414181+0000","last_active":"2023-11-06T20:44:06.420482+0000","last_peered":"2023-11-06T20:44:06.420482+0000","last_clean":"2023-11-06T20:44:06.420482+0000","last_became_active":"2023-11-06T20:44:06.411516+0000","last_became_peered":"2023-11-06T20:44:06.411516+0000","last_unstale":"2023-11-06T20:44:06.420482+0000","last_undegraded":"2023-11-06T20:44:06.420482+0000","last_fullsized":"2023-11-06T20:44:06.420482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.400331+0000","last_change":"2023-11-06T20:44:05.923629+0000","last_active":"2023-11-06T20:44:06.400331+0000","last_peered":"2023-11-06T20:44:06.400331+0000","last_clean":"2023-11-06T20:44:06.400331+0000","last_became_active":"2023-11-06T20:44:05.923258+0000","last_became_peered":"2023-11-06T20:44:05.923258+0000","last_unstale":"2023-11-06T20:44:06.400331+0000","last_undegraded":"2023-11-06T20:44:06.400331+0000","last_fullsized":"2023-11-06T20:44:06.400331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427759+0000","last_change":"2023-11-06T20:44:06.421947+0000","last_active":"2023-11-06T20:44:06.427759+0000","last_peered":"2023-11-06T20:44:06.427759+0000","last_clean":"2023-11-06T20:44:06.427759+0000","last_became_active":"2023-11-06T20:44:06.421497+0000","last_became_peered":"2023-11-06T20:44:06.421497+0000","last_unstale":"2023-11-06T20:44:06.427759+0000","last_undegraded":"2023-11-06T20:44:06.427759+0000","last_fullsized":"2023-11-06T20:44:06.427759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388963+0000","last_change":"2023-11-06T20:44:05.926582+0000","last_active":"2023-11-06T20:44:06.388963+0000","last_peered":"2023-11-06T20:44:06.388963+0000","last_clean":"2023-11-06T20:44:06.388963+0000","last_became_active":"2023-11-06T20:44:05.926291+0000","last_became_peered":"2023-11-06T20:44:05.926291+0000","last_unstale":"2023-11-06T20:44:06.388963+0000","last_undegraded":"2023-11-06T20:44:06.388963+0000","last_fullsized":"2023-11-06T20:44:06.388963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.408755+0000","last_change":"2023-11-06T20:44:06.401465+0000","last_active":"2023-11-06T20:44:06.408755+0000","last_peered":"2023-11-06T20:44:06.408755+0000","last_clean":"2023-11-06T20:44:06.408755+0000","last_became_active":"2023-11-06T20:44:06.401100+0000","last_became_peered":"2023-11-06T20:44:06.401100+0000","last_unstale":"2023-11-06T20:44:06.408755+0000","last_undegraded":"2023-11-06T20:44:06.408755+0000","last_fullsized":"2023-11-06T20:44:06.408755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.392747+0000","last_change":"2023-11-06T20:44:05.910900+0000","last_active":"2023-11-06T20:44:06.392747+0000","last_peered":"2023-11-06T20:44:06.392747+0000","last_clean":"2023-11-06T20:44:06.392747+0000","last_became_active":"2023-11-06T20:44:05.910285+0000","last_became_peered":"2023-11-06T20:44:05.910285+0000","last_unstale":"2023-11-06T20:44:06.392747+0000","last_undegraded":"2023-11-06T20:44:06.392747+0000","last_fullsized":"2023-11-06T20:44:06.392747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.410850+0000","last_change":"2023-11-06T20:44:06.403555+0000","last_active":"2023-11-06T20:44:06.410850+0000","last_peered":"2023-11-06T20:44:06.410850+0000","last_clean":"2023-11-06T20:44:06.410850+0000","last_became_active":"2023-11-06T20:44:06.403246+0000","last_became_peered":"2023-11-06T20:44:06.403246+0000","last_unstale":"2023-11-06T20:44:06.410850+0000","last_undegraded":"2023-11-06T20:44:06.410850+0000","last_fullsized":"2023-11-06T20:44:06.410850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399175+0000","last_change":"2023-11-06T20:44:05.929537+0000","last_active":"2023-11-06T20:44:06.399175+0000","last_peered":"2023-11-06T20:44:06.399175+0000","last_clean":"2023-11-06T20:44:06.399175+0000","last_became_active":"2023-11-06T20:44:05.929231+0000","last_became_peered":"2023-11-06T20:44:05.929231+0000","last_unstale":"2023-11-06T20:44:06.399175+0000","last_undegraded":"2023-11-06T20:44:06.399175+0000","last_fullsized":"2023-11-06T20:44:06.399175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.429096+0000","last_change":"2023-11-06T20:44:06.425230+0000","last_active":"2023-11-06T20:44:06.429096+0000","last_peered":"2023-11-06T20:44:06.429096+0000","last_clean":"2023-11-06T20:44:06.429096+0000","last_became_active":"2023-11-06T20:44:06.424973+0000","last_became_peered":"2023-11-06T20:44:06.424973+0000","last_unstale":"2023-11-06T20:44:06.429096+0000","last_undegraded":"2023-11-06T20:44:06.429096+0000","last_fullsized":"2023-11-06T20:44:06.429096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399278+0000","last_change":"2023-11-06T20:44:05.918773+0000","last_active":"2023-11-06T20:44:06.399278+0000","last_peered":"2023-11-06T20:44:06.399278+0000","last_clean":"2023-11-06T20:44:06.399278+0000","last_became_active":"2023-11-06T20:44:05.916793+0000","last_became_peered":"2023-11-06T20:44:05.916793+0000","last_unstale":"2023-11-06T20:44:06.399278+0000","last_undegraded":"2023-11-06T20:44:06.399278+0000","last_fullsized":"2023-11-06T20:44:06.399278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.423633+0000","last_change":"2023-11-06T20:44:06.419145+0000","last_active":"2023-11-06T20:44:06.423633+0000","last_peered":"2023-11-06T20:44:06.423633+0000","last_clean":"2023-11-06T20:44:06.423633+0000","last_became_active":"2023-11-06T20:44:06.417610+0000","last_became_peered":"2023-11-06T20:44:06.417610+0000","last_unstale":"2023-11-06T20:44:06.423633+0000","last_undegraded":"2023-11-06T20:44:06.423633+0000","last_fullsized":"2023-11-06T20:44:06.423633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-06T20:44:07.410165+0000","last_change":"2023-11-06T20:44:05.922285+0000","last_active":"2023-11-06T20:44:07.410165+0000","last_peered":"2023-11-06T20:44:07.410165+0000","last_clean":"2023-11-06T20:44:07.410165+0000","last_became_active":"2023-11-06T20:44:05.921952+0000","last_became_peered":"2023-11-06T20:44:05.921952+0000","last_unstale":"2023-11-06T20:44:07.410165+0000","last_undegraded":"2023-11-06T20:44:07.410165+0000","last_fullsized":"2023-11-06T20:44:07.410165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419384+0000","last_change":"2023-11-06T20:44:06.411774+0000","last_active":"2023-11-06T20:44:06.419384+0000","last_peered":"2023-11-06T20:44:06.419384+0000","last_clean":"2023-11-06T20:44:06.419384+0000","last_became_active":"2023-11-06T20:44:06.411426+0000","last_became_peered":"2023-11-06T20:44:06.411426+0000","last_unstale":"2023-11-06T20:44:06.419384+0000","last_undegraded":"2023-11-06T20:44:06.419384+0000","last_fullsized":"2023-11-06T20:44:06.419384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395460+0000","last_change":"2023-11-06T20:44:05.922681+0000","last_active":"2023-11-06T20:44:06.395460+0000","last_peered":"2023-11-06T20:44:06.395460+0000","last_clean":"2023-11-06T20:44:06.395460+0000","last_became_active":"2023-11-06T20:44:05.920900+0000","last_became_peered":"2023-11-06T20:44:05.920900+0000","last_unstale":"2023-11-06T20:44:06.395460+0000","last_undegraded":"2023-11-06T20:44:06.395460+0000","last_fullsized":"2023-11-06T20:44:06.395460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420536+0000","last_change":"2023-11-06T20:44:06.416541+0000","last_active":"2023-11-06T20:44:06.420536+0000","last_peered":"2023-11-06T20:44:06.420536+0000","last_clean":"2023-11-06T20:44:06.420536+0000","last_became_active":"2023-11-06T20:44:06.416192+0000","last_became_peered":"2023-11-06T20:44:06.416192+0000","last_unstale":"2023-11-06T20:44:06.420536+0000","last_undegraded":"2023-11-06T20:44:06.420536+0000","last_fullsized":"2023-11-06T20:44:06.420536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389587+0000","last_change":"2023-11-06T20:44:05.935474+0000","last_active":"2023-11-06T20:44:06.389587+0000","last_peered":"2023-11-06T20:44:06.389587+0000","last_clean":"2023-11-06T20:44:06.389587+0000","last_became_active":"2023-11-06T20:44:05.930110+0000","last_became_peered":"2023-11-06T20:44:05.930110+0000","last_unstale":"2023-11-06T20:44:06.389587+0000","last_undegraded":"2023-11-06T20:44:06.389587+0000","last_fullsized":"2023-11-06T20:44:06.389587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421313+0000","last_change":"2023-11-06T20:44:06.416913+0000","last_active":"2023-11-06T20:44:06.421313+0000","last_peered":"2023-11-06T20:44:06.421313+0000","last_clean":"2023-11-06T20:44:06.421313+0000","last_became_active":"2023-11-06T20:44:06.416623+0000","last_became_peered":"2023-11-06T20:44:06.416623+0000","last_unstale":"2023-11-06T20:44:06.421313+0000","last_undegraded":"2023-11-06T20:44:06.421313+0000","last_fullsized":"2023-11-06T20:44:06.421313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.394473+0000","last_change":"2023-11-06T20:44:05.917793+0000","last_active":"2023-11-06T20:44:06.394473+0000","last_peered":"2023-11-06T20:44:06.394473+0000","last_clean":"2023-11-06T20:44:06.394473+0000","last_became_active":"2023-11-06T20:44:05.917376+0000","last_became_peered":"2023-11-06T20:44:05.917376+0000","last_unstale":"2023-11-06T20:44:06.394473+0000","last_undegraded":"2023-11-06T20:44:06.394473+0000","last_fullsized":"2023-11-06T20:44:06.394473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.409294+0000","last_change":"2023-11-06T20:44:06.400632+0000","last_active":"2023-11-06T20:44:06.409294+0000","last_peered":"2023-11-06T20:44:06.409294+0000","last_clean":"2023-11-06T20:44:06.409294+0000","last_became_active":"2023-11-06T20:44:06.397240+0000","last_became_peered":"2023-11-06T20:44:06.397240+0000","last_unstale":"2023-11-06T20:44:06.409294+0000","last_undegraded":"2023-11-06T20:44:06.409294+0000","last_fullsized":"2023-11-06T20:44:06.409294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396107+0000","last_change":"2023-11-06T20:44:05.928517+0000","last_active":"2023-11-06T20:44:06.396107+0000","last_peered":"2023-11-06T20:44:06.396107+0000","last_clean":"2023-11-06T20:44:06.396107+0000","last_became_active":"2023-11-06T20:44:05.928197+0000","last_became_peered":"2023-11-06T20:44:05.928197+0000","last_unstale":"2023-11-06T20:44:06.396107+0000","last_undegraded":"2023-11-06T20:44:06.396107+0000","last_fullsized":"2023-11-06T20:44:06.396107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.417520+0000","last_change":"2023-11-06T20:44:06.414429+0000","last_active":"2023-11-06T20:44:06.417520+0000","last_peered":"2023-11-06T20:44:06.417520+0000","last_clean":"2023-11-06T20:44:06.417520+0000","last_became_active":"2023-11-06T20:44:06.411089+0000","last_became_peered":"2023-11-06T20:44:06.411089+0000","last_unstale":"2023-11-06T20:44:06.417520+0000","last_undegraded":"2023-11-06T20:44:06.417520+0000","last_fullsized":"2023-11-06T20:44:06.417520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.417271+0000","last_change":"2023-11-06T20:44:05.917588+0000","last_active":"2023-11-06T20:44:06.417271+0000","last_peered":"2023-11-06T20:44:06.417271+0000","last_clean":"2023-11-06T20:44:06.417271+0000","last_became_active":"2023-11-06T20:44:05.917276+0000","last_became_peered":"2023-11-06T20:44:05.917276+0000","last_unstale":"2023-11-06T20:44:06.417271+0000","last_undegraded":"2023-11-06T20:44:06.417271+0000","last_fullsized":"2023-11-06T20:44:06.417271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.411678+0000","last_change":"2023-11-06T20:44:06.409877+0000","last_active":"2023-11-06T20:44:06.411678+0000","last_peered":"2023-11-06T20:44:06.411678+0000","last_clean":"2023-11-06T20:44:06.411678+0000","last_became_active":"2023-11-06T20:44:06.408631+0000","last_became_peered":"2023-11-06T20:44:06.408631+0000","last_unstale":"2023-11-06T20:44:06.411678+0000","last_undegraded":"2023-11-06T20:44:06.411678+0000","last_fullsized":"2023-11-06T20:44:06.411678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404793+0000","last_change":"2023-11-06T20:44:05.932767+0000","last_active":"2023-11-06T20:44:06.404793+0000","last_peered":"2023-11-06T20:44:06.404793+0000","last_clean":"2023-11-06T20:44:06.404793+0000","last_became_active":"2023-11-06T20:44:05.932095+0000","last_became_peered":"2023-11-06T20:44:05.932095+0000","last_unstale":"2023-11-06T20:44:06.404793+0000","last_undegraded":"2023-11-06T20:44:06.404793+0000","last_fullsized":"2023-11-06T20:44:06.404793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412317+0000","last_change":"2023-11-06T20:44:06.410542+0000","last_active":"2023-11-06T20:44:06.412317+0000","last_peered":"2023-11-06T20:44:06.412317+0000","last_clean":"2023-11-06T20:44:06.412317+0000","last_became_active":"2023-11-06T20:44:06.410258+0000","last_became_peered":"2023-11-06T20:44:06.410258+0000","last_unstale":"2023-11-06T20:44:06.412317+0000","last_undegraded":"2023-11-06T20:44:06.412317+0000","last_fullsized":"2023-11-06T20:44:06.412317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427701+0000","last_change":"2023-11-06T20:44:05.927848+0000","last_active":"2023-11-06T20:44:06.427701+0000","last_peered":"2023-11-06T20:44:06.427701+0000","last_clean":"2023-11-06T20:44:06.427701+0000","last_became_active":"2023-11-06T20:44:05.927576+0000","last_became_peered":"2023-11-06T20:44:05.927576+0000","last_unstale":"2023-11-06T20:44:06.427701+0000","last_undegraded":"2023-11-06T20:44:06.427701+0000","last_fullsized":"2023-11-06T20:44:06.427701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416668+0000","last_change":"2023-11-06T20:44:06.409302+0000","last_active":"2023-11-06T20:44:06.416668+0000","last_peered":"2023-11-06T20:44:06.416668+0000","last_clean":"2023-11-06T20:44:06.416668+0000","last_became_active":"2023-11-06T20:44:06.409049+0000","last_became_peered":"2023-11-06T20:44:06.409049+0000","last_unstale":"2023-11-06T20:44:06.416668+0000","last_undegraded":"2023-11-06T20:44:06.416668+0000","last_fullsized":"2023-11-06T20:44:06.416668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.398113+0000","last_change":"2023-11-06T20:44:05.915170+0000","last_active":"2023-11-06T20:44:06.398113+0000","last_peered":"2023-11-06T20:44:06.398113+0000","last_clean":"2023-11-06T20:44:06.398113+0000","last_became_active":"2023-11-06T20:44:05.914850+0000","last_became_peered":"2023-11-06T20:44:05.914850+0000","last_unstale":"2023-11-06T20:44:06.398113+0000","last_undegraded":"2023-11-06T20:44:06.398113+0000","last_fullsized":"2023-11-06T20:44:06.398113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419324+0000","last_change":"2023-11-06T20:44:06.407493+0000","last_active":"2023-11-06T20:44:06.419324+0000","last_peered":"2023-11-06T20:44:06.419324+0000","last_clean":"2023-11-06T20:44:06.419324+0000","last_became_active":"2023-11-06T20:44:06.407167+0000","last_became_peered":"2023-11-06T20:44:06.407167+0000","last_unstale":"2023-11-06T20:44:06.419324+0000","last_undegraded":"2023-11-06T20:44:06.419324+0000","last_fullsized":"2023-11-06T20:44:06.419324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389137+0000","last_change":"2023-11-06T20:44:05.934566+0000","last_active":"2023-11-06T20:44:06.389137+0000","last_peered":"2023-11-06T20:44:06.389137+0000","last_clean":"2023-11-06T20:44:06.389137+0000","last_became_active":"2023-11-06T20:44:05.934165+0000","last_became_peered":"2023-11-06T20:44:05.934165+0000","last_unstale":"2023-11-06T20:44:06.389137+0000","last_undegraded":"2023-11-06T20:44:06.389137+0000","last_fullsized":"2023-11-06T20:44:06.389137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.423114+0000","last_change":"2023-11-06T20:44:06.417767+0000","last_active":"2023-11-06T20:44:06.423114+0000","last_peered":"2023-11-06T20:44:06.423114+0000","last_clean":"2023-11-06T20:44:06.423114+0000","last_became_active":"2023-11-06T20:44:06.417468+0000","last_became_peered":"2023-11-06T20:44:06.417468+0000","last_unstale":"2023-11-06T20:44:06.423114+0000","last_undegraded":"2023-11-06T20:44:06.423114+0000","last_fullsized":"2023-11-06T20:44:06.423114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.30","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427626+0000","last_change":"2023-11-06T20:44:05.930422+0000","last_active":"2023-11-06T20:44:06.427626+0000","last_peered":"2023-11-06T20:44:06.427626+0000","last_clean":"2023-11-06T20:44:06.427626+0000","last_became_active":"2023-11-06T20:44:05.930070+0000","last_became_peered":"2023-11-06T20:44:05.930070+0000","last_unstale":"2023-11-06T20:44:06.427626+0000","last_undegraded":"2023-11-06T20:44:06.427626+0000","last_fullsized":"2023-11-06T20:44:06.427626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.411716+0000","last_change":"2023-11-06T20:44:06.407092+0000","last_active":"2023-11-06T20:44:06.411716+0000","last_peered":"2023-11-06T20:44:06.411716+0000","last_clean":"2023-11-06T20:44:06.411716+0000","last_became_active":"2023-11-06T20:44:06.406767+0000","last_became_peered":"2023-11-06T20:44:06.406767+0000","last_unstale":"2023-11-06T20:44:06.411716+0000","last_undegraded":"2023-11-06T20:44:06.411716+0000","last_fullsized":"2023-11-06T20:44:06.411716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393919+0000","last_change":"2023-11-06T20:44:05.923803+0000","last_active":"2023-11-06T20:44:06.393919+0000","last_peered":"2023-11-06T20:44:06.393919+0000","last_clean":"2023-11-06T20:44:06.393919+0000","last_became_active":"2023-11-06T20:44:05.923361+0000","last_became_peered":"2023-11-06T20:44:05.923361+0000","last_unstale":"2023-11-06T20:44:06.393919+0000","last_undegraded":"2023-11-06T20:44:06.393919+0000","last_fullsized":"2023-11-06T20:44:06.393919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420576+0000","last_change":"2023-11-06T20:44:06.419551+0000","last_active":"2023-11-06T20:44:06.420576+0000","last_peered":"2023-11-06T20:44:06.420576+0000","last_clean":"2023-11-06T20:44:06.420576+0000","last_became_active":"2023-11-06T20:44:06.419244+0000","last_became_peered":"2023-11-06T20:44:06.419244+0000","last_unstale":"2023-11-06T20:44:06.420576+0000","last_undegraded":"2023-11-06T20:44:06.420576+0000","last_fullsized":"2023-11-06T20:44:06.420576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.398750+0000","last_change":"2023-11-06T20:44:05.915503+0000","last_active":"2023-11-06T20:44:06.398750+0000","last_peered":"2023-11-06T20:44:06.398750+0000","last_clean":"2023-11-06T20:44:06.398750+0000","last_became_active":"2023-11-06T20:44:05.914672+0000","last_became_peered":"2023-11-06T20:44:05.914672+0000","last_unstale":"2023-11-06T20:44:06.398750+0000","last_undegraded":"2023-11-06T20:44:06.398750+0000","last_fullsized":"2023-11-06T20:44:06.398750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424374+0000","last_change":"2023-11-06T20:44:06.418248+0000","last_active":"2023-11-06T20:44:06.424374+0000","last_peered":"2023-11-06T20:44:06.424374+0000","last_clean":"2023-11-06T20:44:06.424374+0000","last_became_active":"2023-11-06T20:44:06.408712+0000","last_became_peered":"2023-11-06T20:44:06.408712+0000","last_unstale":"2023-11-06T20:44:06.424374+0000","last_undegraded":"2023-11-06T20:44:06.424374+0000","last_fullsized":"2023-11-06T20:44:06.424374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.33","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425162+0000","last_change":"2023-11-06T20:44:05.927963+0000","last_active":"2023-11-06T20:44:06.425162+0000","last_peered":"2023-11-06T20:44:06.425162+0000","last_clean":"2023-11-06T20:44:06.425162+0000","last_became_active":"2023-11-06T20:44:05.927646+0000","last_became_peered":"2023-11-06T20:44:05.927646+0000","last_unstale":"2023-11-06T20:44:06.425162+0000","last_undegraded":"2023-11-06T20:44:06.425162+0000","last_fullsized":"2023-11-06T20:44:06.425162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420270+0000","last_change":"2023-11-06T20:44:06.413775+0000","last_active":"2023-11-06T20:44:06.420270+0000","last_peered":"2023-11-06T20:44:06.420270+0000","last_clean":"2023-11-06T20:44:06.420270+0000","last_became_active":"2023-11-06T20:44:06.413505+0000","last_became_peered":"2023-11-06T20:44:06.413505+0000","last_unstale":"2023-11-06T20:44:06.420270+0000","last_undegraded":"2023-11-06T20:44:06.420270+0000","last_fullsized":"2023-11-06T20:44:06.420270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.429184+0000","last_change":"2023-11-06T20:44:05.931664+0000","last_active":"2023-11-06T20:44:06.429184+0000","last_peered":"2023-11-06T20:44:06.429184+0000","last_clean":"2023-11-06T20:44:06.429184+0000","last_became_active":"2023-11-06T20:44:05.930383+0000","last_became_peered":"2023-11-06T20:44:05.930383+0000","last_unstale":"2023-11-06T20:44:06.429184+0000","last_undegraded":"2023-11-06T20:44:06.429184+0000","last_fullsized":"2023-11-06T20:44:06.429184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.401400+0000","last_change":"2023-11-06T20:44:06.399159+0000","last_active":"2023-11-06T20:44:06.401400+0000","last_peered":"2023-11-06T20:44:06.401400+0000","last_clean":"2023-11-06T20:44:06.401400+0000","last_became_active":"2023-11-06T20:44:06.398305+0000","last_became_peered":"2023-11-06T20:44:06.398305+0000","last_unstale":"2023-11-06T20:44:06.401400+0000","last_undegraded":"2023-11-06T20:44:06.401400+0000","last_fullsized":"2023-11-06T20:44:06.401400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404918+0000","last_change":"2023-11-06T20:44:05.934808+0000","last_active":"2023-11-06T20:44:06.404918+0000","last_peered":"2023-11-06T20:44:06.404918+0000","last_clean":"2023-11-06T20:44:06.404918+0000","last_became_active":"2023-11-06T20:44:05.934503+0000","last_became_peered":"2023-11-06T20:44:05.934503+0000","last_unstale":"2023-11-06T20:44:06.404918+0000","last_undegraded":"2023-11-06T20:44:06.404918+0000","last_fullsized":"2023-11-06T20:44:06.404918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421422+0000","last_change":"2023-11-06T20:44:06.413884+0000","last_active":"2023-11-06T20:44:06.421422+0000","last_peered":"2023-11-06T20:44:06.421422+0000","last_clean":"2023-11-06T20:44:06.421422+0000","last_became_active":"2023-11-06T20:44:06.413013+0000","last_became_peered":"2023-11-06T20:44:06.413013+0000","last_unstale":"2023-11-06T20:44:06.421422+0000","last_undegraded":"2023-11-06T20:44:06.421422+0000","last_fullsized":"2023-11-06T20:44:06.421422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407560+0000","last_change":"2023-11-06T20:44:05.920301+0000","last_active":"2023-11-06T20:44:06.407560+0000","last_peered":"2023-11-06T20:44:06.407560+0000","last_clean":"2023-11-06T20:44:06.407560+0000","last_became_active":"2023-11-06T20:44:05.917401+0000","last_became_peered":"2023-11-06T20:44:05.917401+0000","last_unstale":"2023-11-06T20:44:06.407560+0000","last_undegraded":"2023-11-06T20:44:06.407560+0000","last_fullsized":"2023-11-06T20:44:06.407560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424639+0000","last_change":"2023-11-06T20:44:06.406242+0000","last_active":"2023-11-06T20:44:06.424639+0000","last_peered":"2023-11-06T20:44:06.424639+0000","last_clean":"2023-11-06T20:44:06.424639+0000","last_became_active":"2023-11-06T20:44:06.405979+0000","last_became_peered":"2023-11-06T20:44:06.405979+0000","last_unstale":"2023-11-06T20:44:06.424639+0000","last_undegraded":"2023-11-06T20:44:06.424639+0000","last_fullsized":"2023-11-06T20:44:06.424639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396663+0000","last_change":"2023-11-06T20:44:05.939218+0000","last_active":"2023-11-06T20:44:06.396663+0000","last_peered":"2023-11-06T20:44:06.396663+0000","last_clean":"2023-11-06T20:44:06.396663+0000","last_became_active":"2023-11-06T20:44:05.938856+0000","last_became_peered":"2023-11-06T20:44:05.938856+0000","last_unstale":"2023-11-06T20:44:06.396663+0000","last_undegraded":"2023-11-06T20:44:06.396663+0000","last_fullsized":"2023-11-06T20:44:06.396663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424019+0000","last_change":"2023-11-06T20:44:06.416017+0000","last_active":"2023-11-06T20:44:06.424019+0000","last_peered":"2023-11-06T20:44:06.424019+0000","last_clean":"2023-11-06T20:44:06.424019+0000","last_became_active":"2023-11-06T20:44:06.414205+0000","last_became_peered":"2023-11-06T20:44:06.414205+0000","last_unstale":"2023-11-06T20:44:06.424019+0000","last_undegraded":"2023-11-06T20:44:06.424019+0000","last_fullsized":"2023-11-06T20:44:06.424019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393064+0000","last_change":"2023-11-06T20:44:05.928182+0000","last_active":"2023-11-06T20:44:06.393064+0000","last_peered":"2023-11-06T20:44:06.393064+0000","last_clean":"2023-11-06T20:44:06.393064+0000","last_became_active":"2023-11-06T20:44:05.927847+0000","last_became_peered":"2023-11-06T20:44:05.927847+0000","last_unstale":"2023-11-06T20:44:06.393064+0000","last_undegraded":"2023-11-06T20:44:06.393064+0000","last_fullsized":"2023-11-06T20:44:06.393064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416917+0000","last_change":"2023-11-06T20:44:06.412444+0000","last_active":"2023-11-06T20:44:06.416917+0000","last_peered":"2023-11-06T20:44:06.416917+0000","last_clean":"2023-11-06T20:44:06.416917+0000","last_became_active":"2023-11-06T20:44:06.412113+0000","last_became_peered":"2023-11-06T20:44:06.412113+0000","last_unstale":"2023-11-06T20:44:06.416917+0000","last_undegraded":"2023-11-06T20:44:06.416917+0000","last_fullsized":"2023-11-06T20:44:06.416917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.403846+0000","last_change":"2023-11-06T20:44:05.929538+0000","last_active":"2023-11-06T20:44:06.403846+0000","last_peered":"2023-11-06T20:44:06.403846+0000","last_clean":"2023-11-06T20:44:06.403846+0000","last_became_active":"2023-11-06T20:44:05.922763+0000","last_became_peered":"2023-11-06T20:44:05.922763+0000","last_unstale":"2023-11-06T20:44:06.403846+0000","last_undegraded":"2023-11-06T20:44:06.403846+0000","last_fullsized":"2023-11-06T20:44:06.403846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426978+0000","last_change":"2023-11-06T20:44:06.423931+0000","last_active":"2023-11-06T20:44:06.426978+0000","last_peered":"2023-11-06T20:44:06.426978+0000","last_clean":"2023-11-06T20:44:06.426978+0000","last_became_active":"2023-11-06T20:44:06.423644+0000","last_became_peered":"2023-11-06T20:44:06.423644+0000","last_unstale":"2023-11-06T20:44:06.426978+0000","last_undegraded":"2023-11-06T20:44:06.426978+0000","last_fullsized":"2023-11-06T20:44:06.426978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420060+0000","last_change":"2023-11-06T20:44:05.909687+0000","last_active":"2023-11-06T20:44:06.420060+0000","last_peered":"2023-11-06T20:44:06.420060+0000","last_clean":"2023-11-06T20:44:06.420060+0000","last_became_active":"2023-11-06T20:44:05.907015+0000","last_became_peered":"2023-11-06T20:44:05.907015+0000","last_unstale":"2023-11-06T20:44:06.420060+0000","last_undegraded":"2023-11-06T20:44:06.420060+0000","last_fullsized":"2023-11-06T20:44:06.420060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421264+0000","last_change":"2023-11-06T20:44:06.415345+0000","last_active":"2023-11-06T20:44:06.421264+0000","last_peered":"2023-11-06T20:44:06.421264+0000","last_clean":"2023-11-06T20:44:06.421264+0000","last_became_active":"2023-11-06T20:44:06.414850+0000","last_became_peered":"2023-11-06T20:44:06.414850+0000","last_unstale":"2023-11-06T20:44:06.421264+0000","last_undegraded":"2023-11-06T20:44:06.421264+0000","last_fullsized":"2023-11-06T20:44:06.421264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421636+0000","last_change":"2023-11-06T20:44:05.921703+0000","last_active":"2023-11-06T20:44:06.421636+0000","last_peered":"2023-11-06T20:44:06.421636+0000","last_clean":"2023-11-06T20:44:06.421636+0000","last_became_active":"2023-11-06T20:44:05.920678+0000","last_became_peered":"2023-11-06T20:44:05.920678+0000","last_unstale":"2023-11-06T20:44:06.421636+0000","last_undegraded":"2023-11-06T20:44:06.421636+0000","last_fullsized":"2023-11-06T20:44:06.421636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419871+0000","last_change":"2023-11-06T20:44:06.412730+0000","last_active":"2023-11-06T20:44:06.419871+0000","last_peered":"2023-11-06T20:44:06.419871+0000","last_clean":"2023-11-06T20:44:06.419871+0000","last_became_active":"2023-11-06T20:44:06.411623+0000","last_became_peered":"2023-11-06T20:44:06.411623+0000","last_unstale":"2023-11-06T20:44:06.419871+0000","last_undegraded":"2023-11-06T20:44:06.419871+0000","last_fullsized":"2023-11-06T20:44:06.419871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420499+0000","last_change":"2023-11-06T20:44:05.919149+0000","last_active":"2023-11-06T20:44:06.420499+0000","last_peered":"2023-11-06T20:44:06.420499+0000","last_clean":"2023-11-06T20:44:06.420499+0000","last_became_active":"2023-11-06T20:44:05.918833+0000","last_became_peered":"2023-11-06T20:44:05.918833+0000","last_unstale":"2023-11-06T20:44:06.420499+0000","last_undegraded":"2023-11-06T20:44:06.420499+0000","last_fullsized":"2023-11-06T20:44:06.420499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407334+0000","last_change":"2023-11-06T20:44:06.399010+0000","last_active":"2023-11-06T20:44:06.407334+0000","last_peered":"2023-11-06T20:44:06.407334+0000","last_clean":"2023-11-06T20:44:06.407334+0000","last_became_active":"2023-11-06T20:44:06.398749+0000","last_became_peered":"2023-11-06T20:44:06.398749+0000","last_unstale":"2023-11-06T20:44:06.407334+0000","last_undegraded":"2023-11-06T20:44:06.407334+0000","last_fullsized":"2023-11-06T20:44:06.407334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.394817+0000","last_change":"2023-11-06T20:44:05.911533+0000","last_active":"2023-11-06T20:44:06.394817+0000","last_peered":"2023-11-06T20:44:06.394817+0000","last_clean":"2023-11-06T20:44:06.394817+0000","last_became_active":"2023-11-06T20:44:05.911170+0000","last_became_peered":"2023-11-06T20:44:05.911170+0000","last_unstale":"2023-11-06T20:44:06.394817+0000","last_undegraded":"2023-11-06T20:44:06.394817+0000","last_fullsized":"2023-11-06T20:44:06.394817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426932+0000","last_change":"2023-11-06T20:44:06.413837+0000","last_active":"2023-11-06T20:44:06.426932+0000","last_peered":"2023-11-06T20:44:06.426932+0000","last_clean":"2023-11-06T20:44:06.426932+0000","last_became_active":"2023-11-06T20:44:06.413575+0000","last_became_peered":"2023-11-06T20:44:06.413575+0000","last_unstale":"2023-11-06T20:44:06.426932+0000","last_undegraded":"2023-11-06T20:44:06.426932+0000","last_fullsized":"2023-11-06T20:44:06.426932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395088+0000","last_change":"2023-11-06T20:44:05.919710+0000","last_active":"2023-11-06T20:44:06.395088+0000","last_peered":"2023-11-06T20:44:06.395088+0000","last_clean":"2023-11-06T20:44:06.395088+0000","last_became_active":"2023-11-06T20:44:05.919424+0000","last_became_peered":"2023-11-06T20:44:05.919424+0000","last_unstale":"2023-11-06T20:44:06.395088+0000","last_undegraded":"2023-11-06T20:44:06.395088+0000","last_fullsized":"2023-11-06T20:44:06.395088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419344+0000","last_change":"2023-11-06T20:44:06.410070+0000","last_active":"2023-11-06T20:44:06.419344+0000","last_peered":"2023-11-06T20:44:06.419344+0000","last_clean":"2023-11-06T20:44:06.419344+0000","last_became_active":"2023-11-06T20:44:06.409768+0000","last_became_peered":"2023-11-06T20:44:06.409768+0000","last_unstale":"2023-11-06T20:44:06.419344+0000","last_undegraded":"2023-11-06T20:44:06.419344+0000","last_fullsized":"2023-11-06T20:44:06.419344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427907+0000","last_change":"2023-11-06T20:44:05.937805+0000","last_active":"2023-11-06T20:44:06.427907+0000","last_peered":"2023-11-06T20:44:06.427907+0000","last_clean":"2023-11-06T20:44:06.427907+0000","last_became_active":"2023-11-06T20:44:05.937296+0000","last_became_peered":"2023-11-06T20:44:05.937296+0000","last_unstale":"2023-11-06T20:44:06.427907+0000","last_undegraded":"2023-11-06T20:44:06.427907+0000","last_fullsized":"2023-11-06T20:44:06.427907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297308,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074532,"statfs":{"total":96636764160,"available":96332320768,"internally_reserved":0,"allocated":217088,"data_stored":80225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297312,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074528,"statfs":{"total":96636764160,"available":96332316672,"internally_reserved":0,"allocated":221184,"data_stored":80315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":81139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297328,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074512,"statfs":{"total":96636764160,"available":96332300288,"internally_reserved":0,"allocated":237568,"data_stored":81817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":80869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297328,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074512,"statfs":{"total":96636764160,"available":96332300288,"internally_reserved":0,"allocated":237568,"data_stored":81527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297312,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074528,"statfs":{"total":96636764160,"available":96332316672,"internally_reserved":0,"allocated":221184,"data_stored":80295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297308,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074532,"statfs":{"total":96636764160,"available":96332320768,"internally_reserved":0,"allocated":217088,"data_stored":80225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:44:18.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:44:18.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.403+0000 7f146a5ff700 1 Processor -- start 2023-11-06T20:44:18.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.403+0000 7f146a5ff700 1 -- start start 2023-11-06T20:44:18.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f146a5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14640c1840 0x7f14640a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f146a5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f14640aa2d0 0x7f14640c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f146a5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14640c6240 0x7f1464157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f146a5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1464154960 con 0x7f14640c1840 2023-11-06T20:44:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f146a5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14641554a0 con 0x7f14640aa2d0 2023-11-06T20:44:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f146a5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1464155930 con 0x7f14640c6240 2023-11-06T20:44:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f1463fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14640c1840 0x7f14640a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f1463fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14640c1840 0x7f14640a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33572/0 (socket says 172.21.15.33:33572) 2023-11-06T20:44:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f1463fff700 1 -- 172.21.15.33:0/723990433 learned_addr learned my addr 172.21.15.33:0/723990433 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f1468b9c700 1 --2- 172.21.15.33:0/723990433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14640c6240 0x7f1464157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.405+0000 7f14637fe700 1 --2- 172.21.15.33:0/723990433 >> v2:172.21.15.69:3300/0 conn(0x7f14640aa2d0 0x7f14640c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.406+0000 7f1463fff700 1 -- 172.21.15.33:0/723990433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14640c6240 msgr2=0x7f1464157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:18.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.406+0000 7f1463fff700 1 --2- 172.21.15.33:0/723990433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14640c6240 0x7f1464157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.406+0000 7f1463fff700 1 -- 172.21.15.33:0/723990433 >> v2:172.21.15.69:3300/0 conn(0x7f14640aa2d0 msgr2=0x7f14640c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:18.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.406+0000 7f1463fff700 1 --2- 172.21.15.33:0/723990433 >> v2:172.21.15.69:3300/0 conn(0x7f14640aa2d0 0x7f14640c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.406+0000 7f1463fff700 1 -- 172.21.15.33:0/723990433 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14640c6690 con 0x7f14640c1840 2023-11-06T20:44:18.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.406+0000 7f14637fe700 1 --2- 172.21.15.33:0/723990433 >> v2:172.21.15.69:3300/0 conn(0x7f14640aa2d0 0x7f14640c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:18.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.406+0000 7f1463fff700 1 --2- 172.21.15.33:0/723990433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c1840 0x7f14640a9de0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f14640adb40 tx=0x7f14540090f0).ready entity=mon.0 client_cookie=4812af0164f5ab76 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:18.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.406+0000 7f1462ffd700 1 -- 172.21.15.33:0/723990433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400b520 con 0x7f14640c1840 2023-11-06T20:44:18.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.406+0000 7f1462ffd700 1 -- 172.21.15.33:0/723990433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145400b680 con 0x7f14640c1840 2023-11-06T20:44:18.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.406+0000 7f1462ffd700 1 -- 172.21.15.33:0/723990433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400ba00 con 0x7f14640c1840 2023-11-06T20:44:18.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.407+0000 7f146a5ff700 1 -- 172.21.15.33:0/723990433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c1840 msgr2=0x7f14640a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.407+0000 7f146a5ff700 1 --2- 172.21.15.33:0/723990433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c1840 0x7f14640a9de0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f14640adb40 tx=0x7f14540090f0).stop 2023-11-06T20:44:18.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.407+0000 7f146a5ff700 1 -- 172.21.15.33:0/723990433 shutdown_connections 2023-11-06T20:44:18.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.407+0000 7f146a5ff700 1 --2- 172.21.15.33:0/723990433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14640c6240 0x7f1464157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.407+0000 7f146a5ff700 1 --2- 172.21.15.33:0/723990433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c1840 0x7f14640a9de0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.407+0000 7f146a5ff700 1 --2- 172.21.15.33:0/723990433 >> v2:172.21.15.69:3300/0 conn(0x7f14640aa2d0 0x7f14640c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.407+0000 7f146a5ff700 1 -- 172.21.15.33:0/723990433 >> 172.21.15.33:0/723990433 conn(0x7f146405ec80 msgr2=0x7f14640cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:18.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.407+0000 7f146a5ff700 1 -- 172.21.15.33:0/723990433 shutdown_connections 2023-11-06T20:44:18.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.407+0000 7f146a5ff700 1 -- 172.21.15.33:0/723990433 wait complete. 2023-11-06T20:44:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.408+0000 7f146a5ff700 1 Processor -- start 2023-11-06T20:44:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.408+0000 7f146a5ff700 1 -- start start 2023-11-06T20:44:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.409+0000 7f146a5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14640aa2d0 0x7f1464150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.409+0000 7f146a5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14640c1840 0x7f14641515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.409+0000 7f146a5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c6240 0x7f1464151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.409+0000 7f146a5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14640aff40 con 0x7f14640c6240 2023-11-06T20:44:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.409+0000 7f146a5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1464164dc0 con 0x7f14640c1840 2023-11-06T20:44:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.409+0000 7f146a5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1464151fb0 con 0x7f14640aa2d0 2023-11-06T20:44:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.409+0000 7f1468b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c6240 0x7f1464151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f1468b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c6240 0x7f1464151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33576/0 (socket says 172.21.15.33:33576) 2023-11-06T20:44:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f1468b9c700 1 -- 172.21.15.33:0/1224532607 learned_addr learned my addr 172.21.15.33:0/1224532607 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f14637fe700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14640c1840 0x7f14641515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f1463fff700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14640aa2d0 0x7f1464150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f1468b9c700 1 -- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14640aa2d0 msgr2=0x7f1464150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f1468b9c700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14640aa2d0 0x7f1464150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f1468b9c700 1 -- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14640c1840 msgr2=0x7f14641515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f1468b9c700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14640c1840 0x7f14641515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f1468b9c700 1 -- 172.21.15.33:0/1224532607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454009370 con 0x7f14640c6240 2023-11-06T20:44:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f1468b9c700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c6240 0x7f1464151ac0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f145c009e90 tx=0x7f145c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f144bfff700 1 -- 172.21.15.33:0/1224532607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c00caa0 con 0x7f14640c6240 2023-11-06T20:44:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f144bfff700 1 -- 172.21.15.33:0/1224532607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c00f040 con 0x7f14640c6240 2023-11-06T20:44:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f146a5ff700 1 -- 172.21.15.33:0/1224532607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1464156570 con 0x7f14640c6240 2023-11-06T20:44:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.410+0000 7f144bfff700 1 -- 172.21.15.33:0/1224532607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c00ce10 con 0x7f14640c6240 2023-11-06T20:44:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.411+0000 7f146a5ff700 1 -- 172.21.15.33:0/1224532607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1464155f20 con 0x7f14640c6240 2023-11-06T20:44:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.412+0000 7f146a5ff700 1 -- 172.21.15.33:0/1224532607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1464156700 con 0x7f14640c6240 2023-11-06T20:44:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.412+0000 7f144bfff700 1 -- 172.21.15.33:0/1224532607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f145c013610 con 0x7f14640c6240 2023-11-06T20:44:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.413+0000 7f144bfff700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f145005f150 0x7f1450061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.413+0000 7f144bfff700 1 -- 172.21.15.33:0/1224532607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f145c07ab00 con 0x7f14640c6240 2023-11-06T20:44:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.413+0000 7f1463fff700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f145005f150 0x7f1450061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.414+0000 7f1463fff700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f145005f150 0x7f1450061610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1454009130 tx=0x7f1454017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.417+0000 7f144bfff700 1 -- 172.21.15.33:0/1224532607 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f145c07ce60 con 0x7f14640c6240 2023-11-06T20:44:18.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.596+0000 7f146a5ff700 1 -- 172.21.15.33:0/1224532607 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f146406c6c0 con 0x7f145005f150 2023-11-06T20:44:18.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.608+0000 7f144bfff700 1 -- 172.21.15.33:0/1224532607 <== mgr.4103 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336367 (secure 0 0 0) 0x7f146406c6c0 con 0x7f145005f150 2023-11-06T20:44:18.613 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:18.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.612+0000 7f146a5ff700 1 -- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f145005f150 msgr2=0x7f1450061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.612+0000 7f146a5ff700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f145005f150 0x7f1450061610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1454009130 tx=0x7f1454017000).stop 2023-11-06T20:44:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.612+0000 7f146a5ff700 1 -- 172.21.15.33:0/1224532607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c6240 msgr2=0x7f1464151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.612+0000 7f146a5ff700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c6240 0x7f1464151ac0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f145c009e90 tx=0x7f145c00b040).stop 2023-11-06T20:44:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.612+0000 7f146a5ff700 1 -- 172.21.15.33:0/1224532607 shutdown_connections 2023-11-06T20:44:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.612+0000 7f146a5ff700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f145005f150 0x7f1450061610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.612+0000 7f146a5ff700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14640aa2d0 0x7f1464150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.612+0000 7f146a5ff700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14640c6240 0x7f1464151ac0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.612+0000 7f146a5ff700 1 --2- 172.21.15.33:0/1224532607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14640c1840 0x7f14641515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.612+0000 7f146a5ff700 1 -- 172.21.15.33:0/1224532607 >> 172.21.15.33:0/1224532607 conn(0x7f146405ec80 msgr2=0x7f14640ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.613+0000 7f146a5ff700 1 -- 172.21.15.33:0/1224532607 shutdown_connections 2023-11-06T20:44:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.613+0000 7f146a5ff700 1 -- 172.21.15.33:0/1224532607 wait complete. 2023-11-06T20:44:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:44:18.629 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-06T20:44:16.809634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378528,"kb_used_data":1760,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596192,"statfs":{"total":773094113280,"available":770658500608,"internally_reserved":0,"allocated":1802240,"data_stored":646412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002175"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427856+0000","last_change":"2023-11-06T20:44:06.424455+0000","last_active":"2023-11-06T20:44:06.427856+0000","last_peered":"2023-11-06T20:44:06.427856+0000","last_clean":"2023-11-06T20:44:06.427856+0000","last_became_active":"2023-11-06T20:44:06.424120+0000","last_became_peered":"2023-11-06T20:44:06.424120+0000","last_unstale":"2023-11-06T20:44:06.427856+0000","last_undegraded":"2023-11-06T20:44:06.427856+0000","last_fullsized":"2023-11-06T20:44:06.427856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396063+0000","last_change":"2023-11-06T20:44:05.918231+0000","last_active":"2023-11-06T20:44:06.396063+0000","last_peered":"2023-11-06T20:44:06.396063+0000","last_clean":"2023-11-06T20:44:06.396063+0000","last_became_active":"2023-11-06T20:44:05.917999+0000","last_became_peered":"2023-11-06T20:44:05.917999+0000","last_unstale":"2023-11-06T20:44:06.396063+0000","last_undegraded":"2023-11-06T20:44:06.396063+0000","last_fullsized":"2023-11-06T20:44:06.396063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420202+0000","last_change":"2023-11-06T20:44:06.413802+0000","last_active":"2023-11-06T20:44:06.420202+0000","last_peered":"2023-11-06T20:44:06.420202+0000","last_clean":"2023-11-06T20:44:06.420202+0000","last_became_active":"2023-11-06T20:44:06.412830+0000","last_became_peered":"2023-11-06T20:44:06.412830+0000","last_unstale":"2023-11-06T20:44:06.420202+0000","last_undegraded":"2023-11-06T20:44:06.420202+0000","last_fullsized":"2023-11-06T20:44:06.420202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.402079+0000","last_change":"2023-11-06T20:44:05.922105+0000","last_active":"2023-11-06T20:44:06.402079+0000","last_peered":"2023-11-06T20:44:06.402079+0000","last_clean":"2023-11-06T20:44:06.402079+0000","last_became_active":"2023-11-06T20:44:05.921727+0000","last_became_peered":"2023-11-06T20:44:05.921727+0000","last_unstale":"2023-11-06T20:44:06.402079+0000","last_undegraded":"2023-11-06T20:44:06.402079+0000","last_fullsized":"2023-11-06T20:44:06.402079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426592+0000","last_change":"2023-11-06T20:44:06.411322+0000","last_active":"2023-11-06T20:44:06.426592+0000","last_peered":"2023-11-06T20:44:06.426592+0000","last_clean":"2023-11-06T20:44:06.426592+0000","last_became_active":"2023-11-06T20:44:06.409787+0000","last_became_peered":"2023-11-06T20:44:06.409787+0000","last_unstale":"2023-11-06T20:44:06.426592+0000","last_undegraded":"2023-11-06T20:44:06.426592+0000","last_fullsized":"2023-11-06T20:44:06.426592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393182+0000","last_change":"2023-11-06T20:44:05.920323+0000","last_active":"2023-11-06T20:44:06.393182+0000","last_peered":"2023-11-06T20:44:06.393182+0000","last_clean":"2023-11-06T20:44:06.393182+0000","last_became_active":"2023-11-06T20:44:05.919781+0000","last_became_peered":"2023-11-06T20:44:05.919781+0000","last_unstale":"2023-11-06T20:44:06.393182+0000","last_undegraded":"2023-11-06T20:44:06.393182+0000","last_fullsized":"2023-11-06T20:44:06.393182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.428239+0000","last_change":"2023-11-06T20:44:06.426498+0000","last_active":"2023-11-06T20:44:06.428239+0000","last_peered":"2023-11-06T20:44:06.428239+0000","last_clean":"2023-11-06T20:44:06.428239+0000","last_became_active":"2023-11-06T20:44:06.426148+0000","last_became_peered":"2023-11-06T20:44:06.426148+0000","last_unstale":"2023-11-06T20:44:06.428239+0000","last_undegraded":"2023-11-06T20:44:06.428239+0000","last_fullsized":"2023-11-06T20:44:06.428239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395364+0000","last_change":"2023-11-06T20:44:05.934770+0000","last_active":"2023-11-06T20:44:06.395364+0000","last_peered":"2023-11-06T20:44:06.395364+0000","last_clean":"2023-11-06T20:44:06.395364+0000","last_became_active":"2023-11-06T20:44:05.934479+0000","last_became_peered":"2023-11-06T20:44:05.934479+0000","last_unstale":"2023-11-06T20:44:06.395364+0000","last_undegraded":"2023-11-06T20:44:06.395364+0000","last_fullsized":"2023-11-06T20:44:06.395364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425272+0000","last_change":"2023-11-06T20:44:06.422189+0000","last_active":"2023-11-06T20:44:06.425272+0000","last_peered":"2023-11-06T20:44:06.425272+0000","last_clean":"2023-11-06T20:44:06.425272+0000","last_became_active":"2023-11-06T20:44:06.421946+0000","last_became_peered":"2023-11-06T20:44:06.421946+0000","last_unstale":"2023-11-06T20:44:06.425272+0000","last_undegraded":"2023-11-06T20:44:06.425272+0000","last_fullsized":"2023-11-06T20:44:06.425272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395338+0000","last_change":"2023-11-06T20:44:05.930203+0000","last_active":"2023-11-06T20:44:06.395338+0000","last_peered":"2023-11-06T20:44:06.395338+0000","last_clean":"2023-11-06T20:44:06.395338+0000","last_became_active":"2023-11-06T20:44:05.929810+0000","last_became_peered":"2023-11-06T20:44:05.929810+0000","last_unstale":"2023-11-06T20:44:06.395338+0000","last_undegraded":"2023-11-06T20:44:06.395338+0000","last_fullsized":"2023-11-06T20:44:06.395338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.428278+0000","last_change":"2023-11-06T20:44:06.425515+0000","last_active":"2023-11-06T20:44:06.428278+0000","last_peered":"2023-11-06T20:44:06.428278+0000","last_clean":"2023-11-06T20:44:06.428278+0000","last_became_active":"2023-11-06T20:44:06.425144+0000","last_became_peered":"2023-11-06T20:44:06.425144+0000","last_unstale":"2023-11-06T20:44:06.428278+0000","last_undegraded":"2023-11-06T20:44:06.428278+0000","last_fullsized":"2023-11-06T20:44:06.428278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424086+0000","last_change":"2023-11-06T20:44:05.930915+0000","last_active":"2023-11-06T20:44:06.424086+0000","last_peered":"2023-11-06T20:44:06.424086+0000","last_clean":"2023-11-06T20:44:06.424086+0000","last_became_active":"2023-11-06T20:44:05.928587+0000","last_became_peered":"2023-11-06T20:44:05.928587+0000","last_unstale":"2023-11-06T20:44:06.424086+0000","last_undegraded":"2023-11-06T20:44:06.424086+0000","last_fullsized":"2023-11-06T20:44:06.424086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416540+0000","last_change":"2023-11-06T20:44:06.406652+0000","last_active":"2023-11-06T20:44:06.416540+0000","last_peered":"2023-11-06T20:44:06.416540+0000","last_clean":"2023-11-06T20:44:06.416540+0000","last_became_active":"2023-11-06T20:44:06.404607+0000","last_became_peered":"2023-11-06T20:44:06.404607+0000","last_unstale":"2023-11-06T20:44:06.416540+0000","last_undegraded":"2023-11-06T20:44:06.416540+0000","last_fullsized":"2023-11-06T20:44:06.416540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.428170+0000","last_change":"2023-11-06T20:44:05.921294+0000","last_active":"2023-11-06T20:44:06.428170+0000","last_peered":"2023-11-06T20:44:06.428170+0000","last_clean":"2023-11-06T20:44:06.428170+0000","last_became_active":"2023-11-06T20:44:05.921040+0000","last_became_peered":"2023-11-06T20:44:05.921040+0000","last_unstale":"2023-11-06T20:44:06.428170+0000","last_undegraded":"2023-11-06T20:44:06.428170+0000","last_fullsized":"2023-11-06T20:44:06.428170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420418+0000","last_change":"2023-11-06T20:44:06.414350+0000","last_active":"2023-11-06T20:44:06.420418+0000","last_peered":"2023-11-06T20:44:06.420418+0000","last_clean":"2023-11-06T20:44:06.420418+0000","last_became_active":"2023-11-06T20:44:06.413471+0000","last_became_peered":"2023-11-06T20:44:06.413471+0000","last_unstale":"2023-11-06T20:44:06.420418+0000","last_undegraded":"2023-11-06T20:44:06.420418+0000","last_fullsized":"2023-11-06T20:44:06.420418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427433+0000","last_change":"2023-11-06T20:44:05.931968+0000","last_active":"2023-11-06T20:44:06.427433+0000","last_peered":"2023-11-06T20:44:06.427433+0000","last_clean":"2023-11-06T20:44:06.427433+0000","last_became_active":"2023-11-06T20:44:05.931666+0000","last_became_peered":"2023-11-06T20:44:05.931666+0000","last_unstale":"2023-11-06T20:44:06.427433+0000","last_undegraded":"2023-11-06T20:44:06.427433+0000","last_fullsized":"2023-11-06T20:44:06.427433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427742+0000","last_change":"2023-11-06T20:44:06.421789+0000","last_active":"2023-11-06T20:44:06.427742+0000","last_peered":"2023-11-06T20:44:06.427742+0000","last_clean":"2023-11-06T20:44:06.427742+0000","last_became_active":"2023-11-06T20:44:06.421065+0000","last_became_peered":"2023-11-06T20:44:06.421065+0000","last_unstale":"2023-11-06T20:44:06.427742+0000","last_undegraded":"2023-11-06T20:44:06.427742+0000","last_fullsized":"2023-11-06T20:44:06.427742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.398924+0000","last_change":"2023-11-06T20:44:05.936868+0000","last_active":"2023-11-06T20:44:06.398924+0000","last_peered":"2023-11-06T20:44:06.398924+0000","last_clean":"2023-11-06T20:44:06.398924+0000","last_became_active":"2023-11-06T20:44:05.936617+0000","last_became_peered":"2023-11-06T20:44:05.936617+0000","last_unstale":"2023-11-06T20:44:06.398924+0000","last_undegraded":"2023-11-06T20:44:06.398924+0000","last_fullsized":"2023-11-06T20:44:06.398924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421281+0000","last_change":"2023-11-06T20:44:06.409854+0000","last_active":"2023-11-06T20:44:06.421281+0000","last_peered":"2023-11-06T20:44:06.421281+0000","last_clean":"2023-11-06T20:44:06.421281+0000","last_became_active":"2023-11-06T20:44:06.409400+0000","last_became_peered":"2023-11-06T20:44:06.409400+0000","last_unstale":"2023-11-06T20:44:06.421281+0000","last_undegraded":"2023-11-06T20:44:06.421281+0000","last_fullsized":"2023-11-06T20:44:06.421281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.390859+0000","last_change":"2023-11-06T20:44:05.920464+0000","last_active":"2023-11-06T20:44:06.390859+0000","last_peered":"2023-11-06T20:44:06.390859+0000","last_clean":"2023-11-06T20:44:06.390859+0000","last_became_active":"2023-11-06T20:44:05.920179+0000","last_became_peered":"2023-11-06T20:44:05.920179+0000","last_unstale":"2023-11-06T20:44:06.390859+0000","last_undegraded":"2023-11-06T20:44:06.390859+0000","last_fullsized":"2023-11-06T20:44:06.390859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.410888+0000","last_change":"2023-11-06T20:44:06.404170+0000","last_active":"2023-11-06T20:44:06.410888+0000","last_peered":"2023-11-06T20:44:06.410888+0000","last_clean":"2023-11-06T20:44:06.410888+0000","last_became_active":"2023-11-06T20:44:06.403838+0000","last_became_peered":"2023-11-06T20:44:06.403838+0000","last_unstale":"2023-11-06T20:44:06.410888+0000","last_undegraded":"2023-11-06T20:44:06.410888+0000","last_fullsized":"2023-11-06T20:44:06.410888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427011+0000","last_change":"2023-11-06T20:44:05.909856+0000","last_active":"2023-11-06T20:44:06.427011+0000","last_peered":"2023-11-06T20:44:06.427011+0000","last_clean":"2023-11-06T20:44:06.427011+0000","last_became_active":"2023-11-06T20:44:05.909547+0000","last_became_peered":"2023-11-06T20:44:05.909547+0000","last_unstale":"2023-11-06T20:44:06.427011+0000","last_undegraded":"2023-11-06T20:44:06.427011+0000","last_fullsized":"2023-11-06T20:44:06.427011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412684+0000","last_change":"2023-11-06T20:44:06.401237+0000","last_active":"2023-11-06T20:44:06.412684+0000","last_peered":"2023-11-06T20:44:06.412684+0000","last_clean":"2023-11-06T20:44:06.412684+0000","last_became_active":"2023-11-06T20:44:06.399700+0000","last_became_peered":"2023-11-06T20:44:06.399700+0000","last_unstale":"2023-11-06T20:44:06.412684+0000","last_undegraded":"2023-11-06T20:44:06.412684+0000","last_fullsized":"2023-11-06T20:44:06.412684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421048+0000","last_change":"2023-11-06T20:44:05.930204+0000","last_active":"2023-11-06T20:44:06.421048+0000","last_peered":"2023-11-06T20:44:06.421048+0000","last_clean":"2023-11-06T20:44:06.421048+0000","last_became_active":"2023-11-06T20:44:05.929904+0000","last_became_peered":"2023-11-06T20:44:05.929904+0000","last_unstale":"2023-11-06T20:44:06.421048+0000","last_undegraded":"2023-11-06T20:44:06.421048+0000","last_fullsized":"2023-11-06T20:44:06.421048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.417505+0000","last_change":"2023-11-06T20:44:06.412089+0000","last_active":"2023-11-06T20:44:06.417505+0000","last_peered":"2023-11-06T20:44:06.417505+0000","last_clean":"2023-11-06T20:44:06.417505+0000","last_became_active":"2023-11-06T20:44:06.410838+0000","last_became_peered":"2023-11-06T20:44:06.410838+0000","last_unstale":"2023-11-06T20:44:06.417505+0000","last_undegraded":"2023-11-06T20:44:06.417505+0000","last_fullsized":"2023-11-06T20:44:06.417505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421847+0000","last_change":"2023-11-06T20:44:05.926188+0000","last_active":"2023-11-06T20:44:06.421847+0000","last_peered":"2023-11-06T20:44:06.421847+0000","last_clean":"2023-11-06T20:44:06.421847+0000","last_became_active":"2023-11-06T20:44:05.925868+0000","last_became_peered":"2023-11-06T20:44:05.925868+0000","last_unstale":"2023-11-06T20:44:06.421847+0000","last_undegraded":"2023-11-06T20:44:06.421847+0000","last_fullsized":"2023-11-06T20:44:06.421847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.401739+0000","last_change":"2023-11-06T20:44:06.397805+0000","last_active":"2023-11-06T20:44:06.401739+0000","last_peered":"2023-11-06T20:44:06.401739+0000","last_clean":"2023-11-06T20:44:06.401739+0000","last_became_active":"2023-11-06T20:44:06.397479+0000","last_became_peered":"2023-11-06T20:44:06.397479+0000","last_unstale":"2023-11-06T20:44:06.401739+0000","last_undegraded":"2023-11-06T20:44:06.401739+0000","last_fullsized":"2023-11-06T20:44:06.401739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389927+0000","last_change":"2023-11-06T20:44:05.925515+0000","last_active":"2023-11-06T20:44:06.389927+0000","last_peered":"2023-11-06T20:44:06.389927+0000","last_clean":"2023-11-06T20:44:06.389927+0000","last_became_active":"2023-11-06T20:44:05.921696+0000","last_became_peered":"2023-11-06T20:44:05.921696+0000","last_unstale":"2023-11-06T20:44:06.389927+0000","last_undegraded":"2023-11-06T20:44:06.389927+0000","last_fullsized":"2023-11-06T20:44:06.389927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425392+0000","last_change":"2023-11-06T20:44:06.421420+0000","last_active":"2023-11-06T20:44:06.425392+0000","last_peered":"2023-11-06T20:44:06.425392+0000","last_clean":"2023-11-06T20:44:06.425392+0000","last_became_active":"2023-11-06T20:44:06.421106+0000","last_became_peered":"2023-11-06T20:44:06.421106+0000","last_unstale":"2023-11-06T20:44:06.425392+0000","last_undegraded":"2023-11-06T20:44:06.425392+0000","last_fullsized":"2023-11-06T20:44:06.425392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404911+0000","last_change":"2023-11-06T20:44:05.921460+0000","last_active":"2023-11-06T20:44:06.404911+0000","last_peered":"2023-11-06T20:44:06.404911+0000","last_clean":"2023-11-06T20:44:06.404911+0000","last_became_active":"2023-11-06T20:44:05.921090+0000","last_became_peered":"2023-11-06T20:44:05.921090+0000","last_unstale":"2023-11-06T20:44:06.404911+0000","last_undegraded":"2023-11-06T20:44:06.404911+0000","last_fullsized":"2023-11-06T20:44:06.404911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421245+0000","last_change":"2023-11-06T20:44:06.415154+0000","last_active":"2023-11-06T20:44:06.421245+0000","last_peered":"2023-11-06T20:44:06.421245+0000","last_clean":"2023-11-06T20:44:06.421245+0000","last_became_active":"2023-11-06T20:44:06.414473+0000","last_became_peered":"2023-11-06T20:44:06.414473+0000","last_unstale":"2023-11-06T20:44:06.421245+0000","last_undegraded":"2023-11-06T20:44:06.421245+0000","last_fullsized":"2023-11-06T20:44:06.421245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420865+0000","last_change":"2023-11-06T20:44:05.922405+0000","last_active":"2023-11-06T20:44:06.420865+0000","last_peered":"2023-11-06T20:44:06.420865+0000","last_clean":"2023-11-06T20:44:06.420865+0000","last_became_active":"2023-11-06T20:44:05.921125+0000","last_became_peered":"2023-11-06T20:44:05.921125+0000","last_unstale":"2023-11-06T20:44:06.420865+0000","last_undegraded":"2023-11-06T20:44:06.420865+0000","last_fullsized":"2023-11-06T20:44:06.420865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407539+0000","last_change":"2023-11-06T20:44:06.399863+0000","last_active":"2023-11-06T20:44:06.407539+0000","last_peered":"2023-11-06T20:44:06.407539+0000","last_clean":"2023-11-06T20:44:06.407539+0000","last_became_active":"2023-11-06T20:44:06.399513+0000","last_became_peered":"2023-11-06T20:44:06.399513+0000","last_unstale":"2023-11-06T20:44:06.407539+0000","last_undegraded":"2023-11-06T20:44:06.407539+0000","last_fullsized":"2023-11-06T20:44:06.407539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388728+0000","last_change":"2023-11-06T20:44:05.936043+0000","last_active":"2023-11-06T20:44:06.388728+0000","last_peered":"2023-11-06T20:44:06.388728+0000","last_clean":"2023-11-06T20:44:06.388728+0000","last_became_active":"2023-11-06T20:44:05.935749+0000","last_became_peered":"2023-11-06T20:44:05.935749+0000","last_unstale":"2023-11-06T20:44:06.388728+0000","last_undegraded":"2023-11-06T20:44:06.388728+0000","last_fullsized":"2023-11-06T20:44:06.388728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412317+0000","last_change":"2023-11-06T20:44:06.405453+0000","last_active":"2023-11-06T20:44:06.412317+0000","last_peered":"2023-11-06T20:44:06.412317+0000","last_clean":"2023-11-06T20:44:06.412317+0000","last_became_active":"2023-11-06T20:44:06.405101+0000","last_became_peered":"2023-11-06T20:44:06.405101+0000","last_unstale":"2023-11-06T20:44:06.412317+0000","last_undegraded":"2023-11-06T20:44:06.412317+0000","last_fullsized":"2023-11-06T20:44:06.412317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389916+0000","last_change":"2023-11-06T20:44:05.929392+0000","last_active":"2023-11-06T20:44:06.389916+0000","last_peered":"2023-11-06T20:44:06.389916+0000","last_clean":"2023-11-06T20:44:06.389916+0000","last_became_active":"2023-11-06T20:44:05.928892+0000","last_became_peered":"2023-11-06T20:44:05.928892+0000","last_unstale":"2023-11-06T20:44:06.389916+0000","last_undegraded":"2023-11-06T20:44:06.389916+0000","last_fullsized":"2023-11-06T20:44:06.389916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420520+0000","last_change":"2023-11-06T20:44:06.415261+0000","last_active":"2023-11-06T20:44:06.420520+0000","last_peered":"2023-11-06T20:44:06.420520+0000","last_clean":"2023-11-06T20:44:06.420520+0000","last_became_active":"2023-11-06T20:44:06.410860+0000","last_became_peered":"2023-11-06T20:44:06.410860+0000","last_unstale":"2023-11-06T20:44:06.420520+0000","last_undegraded":"2023-11-06T20:44:06.420520+0000","last_fullsized":"2023-11-06T20:44:06.420520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.406449+0000","last_change":"2023-11-06T20:44:05.912367+0000","last_active":"2023-11-06T20:44:06.406449+0000","last_peered":"2023-11-06T20:44:06.406449+0000","last_clean":"2023-11-06T20:44:06.406449+0000","last_became_active":"2023-11-06T20:44:05.911944+0000","last_became_peered":"2023-11-06T20:44:05.911944+0000","last_unstale":"2023-11-06T20:44:06.406449+0000","last_undegraded":"2023-11-06T20:44:06.406449+0000","last_fullsized":"2023-11-06T20:44:06.406449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393110+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:44:06.393110+0000","last_peered":"2023-11-06T20:44:06.393110+0000","last_clean":"2023-11-06T20:44:06.393110+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:44:06.393110+0000","last_undegraded":"2023-11-06T20:44:06.393110+0000","last_fullsized":"2023-11-06T20:44:06.393110+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393545+0000","last_change":"2023-11-06T20:44:05.916204+0000","last_active":"2023-11-06T20:44:06.393545+0000","last_peered":"2023-11-06T20:44:06.393545+0000","last_clean":"2023-11-06T20:44:06.393545+0000","last_became_active":"2023-11-06T20:44:05.915592+0000","last_became_peered":"2023-11-06T20:44:05.915592+0000","last_unstale":"2023-11-06T20:44:06.393545+0000","last_undegraded":"2023-11-06T20:44:06.393545+0000","last_fullsized":"2023-11-06T20:44:06.393545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.422726+0000","last_change":"2023-11-06T20:44:06.418081+0000","last_active":"2023-11-06T20:44:06.422726+0000","last_peered":"2023-11-06T20:44:06.422726+0000","last_clean":"2023-11-06T20:44:06.422726+0000","last_became_active":"2023-11-06T20:44:06.416191+0000","last_became_peered":"2023-11-06T20:44:06.416191+0000","last_unstale":"2023-11-06T20:44:06.422726+0000","last_undegraded":"2023-11-06T20:44:06.422726+0000","last_fullsized":"2023-11-06T20:44:06.422726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388876+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:44:06.388876+0000","last_peered":"2023-11-06T20:44:06.388876+0000","last_clean":"2023-11-06T20:44:06.388876+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:44:06.388876+0000","last_undegraded":"2023-11-06T20:44:06.388876+0000","last_fullsized":"2023-11-06T20:44:06.388876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404240+0000","last_change":"2023-11-06T20:44:05.926886+0000","last_active":"2023-11-06T20:44:06.404240+0000","last_peered":"2023-11-06T20:44:06.404240+0000","last_clean":"2023-11-06T20:44:06.404240+0000","last_became_active":"2023-11-06T20:44:05.925506+0000","last_became_peered":"2023-11-06T20:44:05.925506+0000","last_unstale":"2023-11-06T20:44:06.404240+0000","last_undegraded":"2023-11-06T20:44:06.404240+0000","last_fullsized":"2023-11-06T20:44:06.404240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420444+0000","last_change":"2023-11-06T20:44:06.415342+0000","last_active":"2023-11-06T20:44:06.420444+0000","last_peered":"2023-11-06T20:44:06.420444+0000","last_clean":"2023-11-06T20:44:06.420444+0000","last_became_active":"2023-11-06T20:44:06.414787+0000","last_became_peered":"2023-11-06T20:44:06.414787+0000","last_unstale":"2023-11-06T20:44:06.420444+0000","last_undegraded":"2023-11-06T20:44:06.420444+0000","last_fullsized":"2023-11-06T20:44:06.420444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425033+0000","last_change":"2023-11-06T20:44:06.422594+0000","last_active":"2023-11-06T20:44:06.425033+0000","last_peered":"2023-11-06T20:44:06.425033+0000","last_clean":"2023-11-06T20:44:06.425033+0000","last_became_active":"2023-11-06T20:44:06.422264+0000","last_became_peered":"2023-11-06T20:44:06.422264+0000","last_unstale":"2023-11-06T20:44:06.425033+0000","last_undegraded":"2023-11-06T20:44:06.425033+0000","last_fullsized":"2023-11-06T20:44:06.425033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.400201+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:44:06.400201+0000","last_peered":"2023-11-06T20:44:06.400201+0000","last_clean":"2023-11-06T20:44:06.400201+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:44:06.400201+0000","last_undegraded":"2023-11-06T20:44:06.400201+0000","last_fullsized":"2023-11-06T20:44:06.400201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.392931+0000","last_change":"2023-11-06T20:44:05.912746+0000","last_active":"2023-11-06T20:44:06.392931+0000","last_peered":"2023-11-06T20:44:06.392931+0000","last_clean":"2023-11-06T20:44:06.392931+0000","last_became_active":"2023-11-06T20:44:05.912443+0000","last_became_peered":"2023-11-06T20:44:05.912443+0000","last_unstale":"2023-11-06T20:44:06.392931+0000","last_undegraded":"2023-11-06T20:44:06.392931+0000","last_fullsized":"2023-11-06T20:44:06.392931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412853+0000","last_change":"2023-11-06T20:44:06.400524+0000","last_active":"2023-11-06T20:44:06.412853+0000","last_peered":"2023-11-06T20:44:06.412853+0000","last_clean":"2023-11-06T20:44:06.412853+0000","last_became_active":"2023-11-06T20:44:06.399624+0000","last_became_peered":"2023-11-06T20:44:06.399624+0000","last_unstale":"2023-11-06T20:44:06.412853+0000","last_undegraded":"2023-11-06T20:44:06.412853+0000","last_fullsized":"2023-11-06T20:44:06.412853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388619+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:44:06.388619+0000","last_peered":"2023-11-06T20:44:06.388619+0000","last_clean":"2023-11-06T20:44:06.388619+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:44:06.388619+0000","last_undegraded":"2023-11-06T20:44:06.388619+0000","last_fullsized":"2023-11-06T20:44:06.388619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395865+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:44:06.395865+0000","last_peered":"2023-11-06T20:44:06.395865+0000","last_clean":"2023-11-06T20:44:06.395865+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:44:06.395865+0000","last_undegraded":"2023-11-06T20:44:06.395865+0000","last_fullsized":"2023-11-06T20:44:06.395865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.401330+0000","last_change":"2023-11-06T20:44:05.915540+0000","last_active":"2023-11-06T20:44:06.401330+0000","last_peered":"2023-11-06T20:44:06.401330+0000","last_clean":"2023-11-06T20:44:06.401330+0000","last_became_active":"2023-11-06T20:44:05.915264+0000","last_became_peered":"2023-11-06T20:44:05.915264+0000","last_unstale":"2023-11-06T20:44:06.401330+0000","last_undegraded":"2023-11-06T20:44:06.401330+0000","last_fullsized":"2023-11-06T20:44:06.401330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.414058+0000","last_change":"2023-11-06T20:44:06.406952+0000","last_active":"2023-11-06T20:44:06.414058+0000","last_peered":"2023-11-06T20:44:06.414058+0000","last_clean":"2023-11-06T20:44:06.414058+0000","last_became_active":"2023-11-06T20:44:06.406478+0000","last_became_peered":"2023-11-06T20:44:06.406478+0000","last_unstale":"2023-11-06T20:44:06.414058+0000","last_undegraded":"2023-11-06T20:44:06.414058+0000","last_fullsized":"2023-11-06T20:44:06.414058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389490+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:44:06.389490+0000","last_peered":"2023-11-06T20:44:06.389490+0000","last_clean":"2023-11-06T20:44:06.389490+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:44:06.389490+0000","last_undegraded":"2023-11-06T20:44:06.389490+0000","last_fullsized":"2023-11-06T20:44:06.389490+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396293+0000","last_change":"2023-11-06T20:44:05.916288+0000","last_active":"2023-11-06T20:44:06.396293+0000","last_peered":"2023-11-06T20:44:06.396293+0000","last_clean":"2023-11-06T20:44:06.396293+0000","last_became_active":"2023-11-06T20:44:05.915873+0000","last_became_peered":"2023-11-06T20:44:05.915873+0000","last_unstale":"2023-11-06T20:44:06.396293+0000","last_undegraded":"2023-11-06T20:44:06.396293+0000","last_fullsized":"2023-11-06T20:44:06.396293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.390009+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:44:06.390009+0000","last_peered":"2023-11-06T20:44:06.390009+0000","last_clean":"2023-11-06T20:44:06.390009+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:44:06.390009+0000","last_undegraded":"2023-11-06T20:44:06.390009+0000","last_fullsized":"2023-11-06T20:44:06.390009+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399349+0000","last_change":"2023-11-06T20:44:05.914799+0000","last_active":"2023-11-06T20:44:06.399349+0000","last_peered":"2023-11-06T20:44:06.399349+0000","last_clean":"2023-11-06T20:44:06.399349+0000","last_became_active":"2023-11-06T20:44:05.914449+0000","last_became_peered":"2023-11-06T20:44:05.914449+0000","last_unstale":"2023-11-06T20:44:06.399349+0000","last_undegraded":"2023-11-06T20:44:06.399349+0000","last_fullsized":"2023-11-06T20:44:06.399349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412824+0000","last_change":"2023-11-06T20:44:06.408012+0000","last_active":"2023-11-06T20:44:06.412824+0000","last_peered":"2023-11-06T20:44:06.412824+0000","last_clean":"2023-11-06T20:44:06.412824+0000","last_became_active":"2023-11-06T20:44:06.407679+0000","last_became_peered":"2023-11-06T20:44:06.407679+0000","last_unstale":"2023-11-06T20:44:06.412824+0000","last_undegraded":"2023-11-06T20:44:06.412824+0000","last_fullsized":"2023-11-06T20:44:06.412824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421500+0000","last_change":"2023-11-06T20:44:05.919819+0000","last_active":"2023-11-06T20:44:06.421500+0000","last_peered":"2023-11-06T20:44:06.421500+0000","last_clean":"2023-11-06T20:44:06.421500+0000","last_became_active":"2023-11-06T20:44:05.919286+0000","last_became_peered":"2023-11-06T20:44:05.919286+0000","last_unstale":"2023-11-06T20:44:06.421500+0000","last_undegraded":"2023-11-06T20:44:06.421500+0000","last_fullsized":"2023-11-06T20:44:06.421500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.405378+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:44:06.405378+0000","last_peered":"2023-11-06T20:44:06.405378+0000","last_clean":"2023-11-06T20:44:06.405378+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:44:06.405378+0000","last_undegraded":"2023-11-06T20:44:06.405378+0000","last_fullsized":"2023-11-06T20:44:06.405378+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.423170+0000","last_change":"2023-11-06T20:44:06.420584+0000","last_active":"2023-11-06T20:44:06.423170+0000","last_peered":"2023-11-06T20:44:06.423170+0000","last_clean":"2023-11-06T20:44:06.423170+0000","last_became_active":"2023-11-06T20:44:06.420289+0000","last_became_peered":"2023-11-06T20:44:06.420289+0000","last_unstale":"2023-11-06T20:44:06.423170+0000","last_undegraded":"2023-11-06T20:44:06.423170+0000","last_fullsized":"2023-11-06T20:44:06.423170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389064+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:44:06.389064+0000","last_peered":"2023-11-06T20:44:06.389064+0000","last_clean":"2023-11-06T20:44:06.389064+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:44:06.389064+0000","last_undegraded":"2023-11-06T20:44:06.389064+0000","last_fullsized":"2023-11-06T20:44:06.389064+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.405496+0000","last_change":"2023-11-06T20:44:05.911916+0000","last_active":"2023-11-06T20:44:06.405496+0000","last_peered":"2023-11-06T20:44:06.405496+0000","last_clean":"2023-11-06T20:44:06.405496+0000","last_became_active":"2023-11-06T20:44:05.911373+0000","last_became_peered":"2023-11-06T20:44:05.911373+0000","last_unstale":"2023-11-06T20:44:06.405496+0000","last_undegraded":"2023-11-06T20:44:06.405496+0000","last_fullsized":"2023-11-06T20:44:06.405496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.402668+0000","last_change":"2023-11-06T20:44:06.399566+0000","last_active":"2023-11-06T20:44:06.402668+0000","last_peered":"2023-11-06T20:44:06.402668+0000","last_clean":"2023-11-06T20:44:06.402668+0000","last_became_active":"2023-11-06T20:44:06.399283+0000","last_became_peered":"2023-11-06T20:44:06.399283+0000","last_unstale":"2023-11-06T20:44:06.402668+0000","last_undegraded":"2023-11-06T20:44:06.402668+0000","last_fullsized":"2023-11-06T20:44:06.402668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420260+0000","last_change":"2023-11-06T20:44:06.413736+0000","last_active":"2023-11-06T20:44:06.420260+0000","last_peered":"2023-11-06T20:44:06.420260+0000","last_clean":"2023-11-06T20:44:06.420260+0000","last_became_active":"2023-11-06T20:44:06.413520+0000","last_became_peered":"2023-11-06T20:44:06.413520+0000","last_unstale":"2023-11-06T20:44:06.420260+0000","last_undegraded":"2023-11-06T20:44:06.420260+0000","last_fullsized":"2023-11-06T20:44:06.420260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.403806+0000","last_change":"2023-11-06T20:44:05.929692+0000","last_active":"2023-11-06T20:44:06.403806+0000","last_peered":"2023-11-06T20:44:06.403806+0000","last_clean":"2023-11-06T20:44:06.403806+0000","last_became_active":"2023-11-06T20:44:05.929364+0000","last_became_peered":"2023-11-06T20:44:05.929364+0000","last_unstale":"2023-11-06T20:44:06.403806+0000","last_undegraded":"2023-11-06T20:44:06.403806+0000","last_fullsized":"2023-11-06T20:44:06.403806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416558+0000","last_change":"2023-11-06T20:44:06.407233+0000","last_active":"2023-11-06T20:44:06.416558+0000","last_peered":"2023-11-06T20:44:06.416558+0000","last_clean":"2023-11-06T20:44:06.416558+0000","last_became_active":"2023-11-06T20:44:06.406927+0000","last_became_peered":"2023-11-06T20:44:06.406927+0000","last_unstale":"2023-11-06T20:44:06.416558+0000","last_undegraded":"2023-11-06T20:44:06.416558+0000","last_fullsized":"2023-11-06T20:44:06.416558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396117+0000","last_change":"2023-11-06T20:44:05.925954+0000","last_active":"2023-11-06T20:44:06.396117+0000","last_peered":"2023-11-06T20:44:06.396117+0000","last_clean":"2023-11-06T20:44:06.396117+0000","last_became_active":"2023-11-06T20:44:05.925626+0000","last_became_peered":"2023-11-06T20:44:05.925626+0000","last_unstale":"2023-11-06T20:44:06.396117+0000","last_undegraded":"2023-11-06T20:44:06.396117+0000","last_fullsized":"2023-11-06T20:44:06.396117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420472+0000","last_change":"2023-11-06T20:44:06.417902+0000","last_active":"2023-11-06T20:44:06.420472+0000","last_peered":"2023-11-06T20:44:06.420472+0000","last_clean":"2023-11-06T20:44:06.420472+0000","last_became_active":"2023-11-06T20:44:06.417309+0000","last_became_peered":"2023-11-06T20:44:06.417309+0000","last_unstale":"2023-11-06T20:44:06.420472+0000","last_undegraded":"2023-11-06T20:44:06.420472+0000","last_fullsized":"2023-11-06T20:44:06.420472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427611+0000","last_change":"2023-11-06T20:44:05.935705+0000","last_active":"2023-11-06T20:44:06.427611+0000","last_peered":"2023-11-06T20:44:06.427611+0000","last_clean":"2023-11-06T20:44:06.427611+0000","last_became_active":"2023-11-06T20:44:05.935436+0000","last_became_peered":"2023-11-06T20:44:05.935436+0000","last_unstale":"2023-11-06T20:44:06.427611+0000","last_undegraded":"2023-11-06T20:44:06.427611+0000","last_fullsized":"2023-11-06T20:44:06.427611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419962+0000","last_change":"2023-11-06T20:44:06.414767+0000","last_active":"2023-11-06T20:44:06.419962+0000","last_peered":"2023-11-06T20:44:06.419962+0000","last_clean":"2023-11-06T20:44:06.419962+0000","last_became_active":"2023-11-06T20:44:06.414452+0000","last_became_peered":"2023-11-06T20:44:06.414452+0000","last_unstale":"2023-11-06T20:44:06.419962+0000","last_undegraded":"2023-11-06T20:44:06.419962+0000","last_fullsized":"2023-11-06T20:44:06.419962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.430730+0000","last_change":"2023-11-06T20:44:05.916430+0000","last_active":"2023-11-06T20:44:06.430730+0000","last_peered":"2023-11-06T20:44:06.430730+0000","last_clean":"2023-11-06T20:44:06.430730+0000","last_became_active":"2023-11-06T20:44:05.916163+0000","last_became_peered":"2023-11-06T20:44:05.916163+0000","last_unstale":"2023-11-06T20:44:06.430730+0000","last_undegraded":"2023-11-06T20:44:06.430730+0000","last_fullsized":"2023-11-06T20:44:06.430730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.415635+0000","last_change":"2023-11-06T20:44:06.412706+0000","last_active":"2023-11-06T20:44:06.415635+0000","last_peered":"2023-11-06T20:44:06.415635+0000","last_clean":"2023-11-06T20:44:06.415635+0000","last_became_active":"2023-11-06T20:44:06.412286+0000","last_became_peered":"2023-11-06T20:44:06.412286+0000","last_unstale":"2023-11-06T20:44:06.415635+0000","last_undegraded":"2023-11-06T20:44:06.415635+0000","last_fullsized":"2023-11-06T20:44:06.415635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.430724+0000","last_change":"2023-11-06T20:44:05.920163+0000","last_active":"2023-11-06T20:44:06.430724+0000","last_peered":"2023-11-06T20:44:06.430724+0000","last_clean":"2023-11-06T20:44:06.430724+0000","last_became_active":"2023-11-06T20:44:05.919817+0000","last_became_peered":"2023-11-06T20:44:05.919817+0000","last_unstale":"2023-11-06T20:44:06.430724+0000","last_undegraded":"2023-11-06T20:44:06.430724+0000","last_fullsized":"2023-11-06T20:44:06.430724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420853+0000","last_change":"2023-11-06T20:44:06.415900+0000","last_active":"2023-11-06T20:44:06.420853+0000","last_peered":"2023-11-06T20:44:06.420853+0000","last_clean":"2023-11-06T20:44:06.420853+0000","last_became_active":"2023-11-06T20:44:06.415623+0000","last_became_peered":"2023-11-06T20:44:06.415623+0000","last_unstale":"2023-11-06T20:44:06.420853+0000","last_undegraded":"2023-11-06T20:44:06.420853+0000","last_fullsized":"2023-11-06T20:44:06.420853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.394177+0000","last_change":"2023-11-06T20:44:05.916945+0000","last_active":"2023-11-06T20:44:06.394177+0000","last_peered":"2023-11-06T20:44:06.394177+0000","last_clean":"2023-11-06T20:44:06.394177+0000","last_became_active":"2023-11-06T20:44:05.916459+0000","last_became_peered":"2023-11-06T20:44:05.916459+0000","last_unstale":"2023-11-06T20:44:06.394177+0000","last_undegraded":"2023-11-06T20:44:06.394177+0000","last_fullsized":"2023-11-06T20:44:06.394177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420482+0000","last_change":"2023-11-06T20:44:06.414181+0000","last_active":"2023-11-06T20:44:06.420482+0000","last_peered":"2023-11-06T20:44:06.420482+0000","last_clean":"2023-11-06T20:44:06.420482+0000","last_became_active":"2023-11-06T20:44:06.411516+0000","last_became_peered":"2023-11-06T20:44:06.411516+0000","last_unstale":"2023-11-06T20:44:06.420482+0000","last_undegraded":"2023-11-06T20:44:06.420482+0000","last_fullsized":"2023-11-06T20:44:06.420482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.400331+0000","last_change":"2023-11-06T20:44:05.923629+0000","last_active":"2023-11-06T20:44:06.400331+0000","last_peered":"2023-11-06T20:44:06.400331+0000","last_clean":"2023-11-06T20:44:06.400331+0000","last_became_active":"2023-11-06T20:44:05.923258+0000","last_became_peered":"2023-11-06T20:44:05.923258+0000","last_unstale":"2023-11-06T20:44:06.400331+0000","last_undegraded":"2023-11-06T20:44:06.400331+0000","last_fullsized":"2023-11-06T20:44:06.400331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427759+0000","last_change":"2023-11-06T20:44:06.421947+0000","last_active":"2023-11-06T20:44:06.427759+0000","last_peered":"2023-11-06T20:44:06.427759+0000","last_clean":"2023-11-06T20:44:06.427759+0000","last_became_active":"2023-11-06T20:44:06.421497+0000","last_became_peered":"2023-11-06T20:44:06.421497+0000","last_unstale":"2023-11-06T20:44:06.427759+0000","last_undegraded":"2023-11-06T20:44:06.427759+0000","last_fullsized":"2023-11-06T20:44:06.427759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.388963+0000","last_change":"2023-11-06T20:44:05.926582+0000","last_active":"2023-11-06T20:44:06.388963+0000","last_peered":"2023-11-06T20:44:06.388963+0000","last_clean":"2023-11-06T20:44:06.388963+0000","last_became_active":"2023-11-06T20:44:05.926291+0000","last_became_peered":"2023-11-06T20:44:05.926291+0000","last_unstale":"2023-11-06T20:44:06.388963+0000","last_undegraded":"2023-11-06T20:44:06.388963+0000","last_fullsized":"2023-11-06T20:44:06.388963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.408755+0000","last_change":"2023-11-06T20:44:06.401465+0000","last_active":"2023-11-06T20:44:06.408755+0000","last_peered":"2023-11-06T20:44:06.408755+0000","last_clean":"2023-11-06T20:44:06.408755+0000","last_became_active":"2023-11-06T20:44:06.401100+0000","last_became_peered":"2023-11-06T20:44:06.401100+0000","last_unstale":"2023-11-06T20:44:06.408755+0000","last_undegraded":"2023-11-06T20:44:06.408755+0000","last_fullsized":"2023-11-06T20:44:06.408755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.392747+0000","last_change":"2023-11-06T20:44:05.910900+0000","last_active":"2023-11-06T20:44:06.392747+0000","last_peered":"2023-11-06T20:44:06.392747+0000","last_clean":"2023-11-06T20:44:06.392747+0000","last_became_active":"2023-11-06T20:44:05.910285+0000","last_became_peered":"2023-11-06T20:44:05.910285+0000","last_unstale":"2023-11-06T20:44:06.392747+0000","last_undegraded":"2023-11-06T20:44:06.392747+0000","last_fullsized":"2023-11-06T20:44:06.392747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.410850+0000","last_change":"2023-11-06T20:44:06.403555+0000","last_active":"2023-11-06T20:44:06.410850+0000","last_peered":"2023-11-06T20:44:06.410850+0000","last_clean":"2023-11-06T20:44:06.410850+0000","last_became_active":"2023-11-06T20:44:06.403246+0000","last_became_peered":"2023-11-06T20:44:06.403246+0000","last_unstale":"2023-11-06T20:44:06.410850+0000","last_undegraded":"2023-11-06T20:44:06.410850+0000","last_fullsized":"2023-11-06T20:44:06.410850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399175+0000","last_change":"2023-11-06T20:44:05.929537+0000","last_active":"2023-11-06T20:44:06.399175+0000","last_peered":"2023-11-06T20:44:06.399175+0000","last_clean":"2023-11-06T20:44:06.399175+0000","last_became_active":"2023-11-06T20:44:05.929231+0000","last_became_peered":"2023-11-06T20:44:05.929231+0000","last_unstale":"2023-11-06T20:44:06.399175+0000","last_undegraded":"2023-11-06T20:44:06.399175+0000","last_fullsized":"2023-11-06T20:44:06.399175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.429096+0000","last_change":"2023-11-06T20:44:06.425230+0000","last_active":"2023-11-06T20:44:06.429096+0000","last_peered":"2023-11-06T20:44:06.429096+0000","last_clean":"2023-11-06T20:44:06.429096+0000","last_became_active":"2023-11-06T20:44:06.424973+0000","last_became_peered":"2023-11-06T20:44:06.424973+0000","last_unstale":"2023-11-06T20:44:06.429096+0000","last_undegraded":"2023-11-06T20:44:06.429096+0000","last_fullsized":"2023-11-06T20:44:06.429096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.399278+0000","last_change":"2023-11-06T20:44:05.918773+0000","last_active":"2023-11-06T20:44:06.399278+0000","last_peered":"2023-11-06T20:44:06.399278+0000","last_clean":"2023-11-06T20:44:06.399278+0000","last_became_active":"2023-11-06T20:44:05.916793+0000","last_became_peered":"2023-11-06T20:44:05.916793+0000","last_unstale":"2023-11-06T20:44:06.399278+0000","last_undegraded":"2023-11-06T20:44:06.399278+0000","last_fullsized":"2023-11-06T20:44:06.399278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.423633+0000","last_change":"2023-11-06T20:44:06.419145+0000","last_active":"2023-11-06T20:44:06.423633+0000","last_peered":"2023-11-06T20:44:06.423633+0000","last_clean":"2023-11-06T20:44:06.423633+0000","last_became_active":"2023-11-06T20:44:06.417610+0000","last_became_peered":"2023-11-06T20:44:06.417610+0000","last_unstale":"2023-11-06T20:44:06.423633+0000","last_undegraded":"2023-11-06T20:44:06.423633+0000","last_fullsized":"2023-11-06T20:44:06.423633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-06T20:44:07.410165+0000","last_change":"2023-11-06T20:44:05.922285+0000","last_active":"2023-11-06T20:44:07.410165+0000","last_peered":"2023-11-06T20:44:07.410165+0000","last_clean":"2023-11-06T20:44:07.410165+0000","last_became_active":"2023-11-06T20:44:05.921952+0000","last_became_peered":"2023-11-06T20:44:05.921952+0000","last_unstale":"2023-11-06T20:44:07.410165+0000","last_undegraded":"2023-11-06T20:44:07.410165+0000","last_fullsized":"2023-11-06T20:44:07.410165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419384+0000","last_change":"2023-11-06T20:44:06.411774+0000","last_active":"2023-11-06T20:44:06.419384+0000","last_peered":"2023-11-06T20:44:06.419384+0000","last_clean":"2023-11-06T20:44:06.419384+0000","last_became_active":"2023-11-06T20:44:06.411426+0000","last_became_peered":"2023-11-06T20:44:06.411426+0000","last_unstale":"2023-11-06T20:44:06.419384+0000","last_undegraded":"2023-11-06T20:44:06.419384+0000","last_fullsized":"2023-11-06T20:44:06.419384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395460+0000","last_change":"2023-11-06T20:44:05.922681+0000","last_active":"2023-11-06T20:44:06.395460+0000","last_peered":"2023-11-06T20:44:06.395460+0000","last_clean":"2023-11-06T20:44:06.395460+0000","last_became_active":"2023-11-06T20:44:05.920900+0000","last_became_peered":"2023-11-06T20:44:05.920900+0000","last_unstale":"2023-11-06T20:44:06.395460+0000","last_undegraded":"2023-11-06T20:44:06.395460+0000","last_fullsized":"2023-11-06T20:44:06.395460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420536+0000","last_change":"2023-11-06T20:44:06.416541+0000","last_active":"2023-11-06T20:44:06.420536+0000","last_peered":"2023-11-06T20:44:06.420536+0000","last_clean":"2023-11-06T20:44:06.420536+0000","last_became_active":"2023-11-06T20:44:06.416192+0000","last_became_peered":"2023-11-06T20:44:06.416192+0000","last_unstale":"2023-11-06T20:44:06.420536+0000","last_undegraded":"2023-11-06T20:44:06.420536+0000","last_fullsized":"2023-11-06T20:44:06.420536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389587+0000","last_change":"2023-11-06T20:44:05.935474+0000","last_active":"2023-11-06T20:44:06.389587+0000","last_peered":"2023-11-06T20:44:06.389587+0000","last_clean":"2023-11-06T20:44:06.389587+0000","last_became_active":"2023-11-06T20:44:05.930110+0000","last_became_peered":"2023-11-06T20:44:05.930110+0000","last_unstale":"2023-11-06T20:44:06.389587+0000","last_undegraded":"2023-11-06T20:44:06.389587+0000","last_fullsized":"2023-11-06T20:44:06.389587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421313+0000","last_change":"2023-11-06T20:44:06.416913+0000","last_active":"2023-11-06T20:44:06.421313+0000","last_peered":"2023-11-06T20:44:06.421313+0000","last_clean":"2023-11-06T20:44:06.421313+0000","last_became_active":"2023-11-06T20:44:06.416623+0000","last_became_peered":"2023-11-06T20:44:06.416623+0000","last_unstale":"2023-11-06T20:44:06.421313+0000","last_undegraded":"2023-11-06T20:44:06.421313+0000","last_fullsized":"2023-11-06T20:44:06.421313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.394473+0000","last_change":"2023-11-06T20:44:05.917793+0000","last_active":"2023-11-06T20:44:06.394473+0000","last_peered":"2023-11-06T20:44:06.394473+0000","last_clean":"2023-11-06T20:44:06.394473+0000","last_became_active":"2023-11-06T20:44:05.917376+0000","last_became_peered":"2023-11-06T20:44:05.917376+0000","last_unstale":"2023-11-06T20:44:06.394473+0000","last_undegraded":"2023-11-06T20:44:06.394473+0000","last_fullsized":"2023-11-06T20:44:06.394473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.409294+0000","last_change":"2023-11-06T20:44:06.400632+0000","last_active":"2023-11-06T20:44:06.409294+0000","last_peered":"2023-11-06T20:44:06.409294+0000","last_clean":"2023-11-06T20:44:06.409294+0000","last_became_active":"2023-11-06T20:44:06.397240+0000","last_became_peered":"2023-11-06T20:44:06.397240+0000","last_unstale":"2023-11-06T20:44:06.409294+0000","last_undegraded":"2023-11-06T20:44:06.409294+0000","last_fullsized":"2023-11-06T20:44:06.409294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396107+0000","last_change":"2023-11-06T20:44:05.928517+0000","last_active":"2023-11-06T20:44:06.396107+0000","last_peered":"2023-11-06T20:44:06.396107+0000","last_clean":"2023-11-06T20:44:06.396107+0000","last_became_active":"2023-11-06T20:44:05.928197+0000","last_became_peered":"2023-11-06T20:44:05.928197+0000","last_unstale":"2023-11-06T20:44:06.396107+0000","last_undegraded":"2023-11-06T20:44:06.396107+0000","last_fullsized":"2023-11-06T20:44:06.396107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.417520+0000","last_change":"2023-11-06T20:44:06.414429+0000","last_active":"2023-11-06T20:44:06.417520+0000","last_peered":"2023-11-06T20:44:06.417520+0000","last_clean":"2023-11-06T20:44:06.417520+0000","last_became_active":"2023-11-06T20:44:06.411089+0000","last_became_peered":"2023-11-06T20:44:06.411089+0000","last_unstale":"2023-11-06T20:44:06.417520+0000","last_undegraded":"2023-11-06T20:44:06.417520+0000","last_fullsized":"2023-11-06T20:44:06.417520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.417271+0000","last_change":"2023-11-06T20:44:05.917588+0000","last_active":"2023-11-06T20:44:06.417271+0000","last_peered":"2023-11-06T20:44:06.417271+0000","last_clean":"2023-11-06T20:44:06.417271+0000","last_became_active":"2023-11-06T20:44:05.917276+0000","last_became_peered":"2023-11-06T20:44:05.917276+0000","last_unstale":"2023-11-06T20:44:06.417271+0000","last_undegraded":"2023-11-06T20:44:06.417271+0000","last_fullsized":"2023-11-06T20:44:06.417271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.411678+0000","last_change":"2023-11-06T20:44:06.409877+0000","last_active":"2023-11-06T20:44:06.411678+0000","last_peered":"2023-11-06T20:44:06.411678+0000","last_clean":"2023-11-06T20:44:06.411678+0000","last_became_active":"2023-11-06T20:44:06.408631+0000","last_became_peered":"2023-11-06T20:44:06.408631+0000","last_unstale":"2023-11-06T20:44:06.411678+0000","last_undegraded":"2023-11-06T20:44:06.411678+0000","last_fullsized":"2023-11-06T20:44:06.411678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404793+0000","last_change":"2023-11-06T20:44:05.932767+0000","last_active":"2023-11-06T20:44:06.404793+0000","last_peered":"2023-11-06T20:44:06.404793+0000","last_clean":"2023-11-06T20:44:06.404793+0000","last_became_active":"2023-11-06T20:44:05.932095+0000","last_became_peered":"2023-11-06T20:44:05.932095+0000","last_unstale":"2023-11-06T20:44:06.404793+0000","last_undegraded":"2023-11-06T20:44:06.404793+0000","last_fullsized":"2023-11-06T20:44:06.404793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.412317+0000","last_change":"2023-11-06T20:44:06.410542+0000","last_active":"2023-11-06T20:44:06.412317+0000","last_peered":"2023-11-06T20:44:06.412317+0000","last_clean":"2023-11-06T20:44:06.412317+0000","last_became_active":"2023-11-06T20:44:06.410258+0000","last_became_peered":"2023-11-06T20:44:06.410258+0000","last_unstale":"2023-11-06T20:44:06.412317+0000","last_undegraded":"2023-11-06T20:44:06.412317+0000","last_fullsized":"2023-11-06T20:44:06.412317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427701+0000","last_change":"2023-11-06T20:44:05.927848+0000","last_active":"2023-11-06T20:44:06.427701+0000","last_peered":"2023-11-06T20:44:06.427701+0000","last_clean":"2023-11-06T20:44:06.427701+0000","last_became_active":"2023-11-06T20:44:05.927576+0000","last_became_peered":"2023-11-06T20:44:05.927576+0000","last_unstale":"2023-11-06T20:44:06.427701+0000","last_undegraded":"2023-11-06T20:44:06.427701+0000","last_fullsized":"2023-11-06T20:44:06.427701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416668+0000","last_change":"2023-11-06T20:44:06.409302+0000","last_active":"2023-11-06T20:44:06.416668+0000","last_peered":"2023-11-06T20:44:06.416668+0000","last_clean":"2023-11-06T20:44:06.416668+0000","last_became_active":"2023-11-06T20:44:06.409049+0000","last_became_peered":"2023-11-06T20:44:06.409049+0000","last_unstale":"2023-11-06T20:44:06.416668+0000","last_undegraded":"2023-11-06T20:44:06.416668+0000","last_fullsized":"2023-11-06T20:44:06.416668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.398113+0000","last_change":"2023-11-06T20:44:05.915170+0000","last_active":"2023-11-06T20:44:06.398113+0000","last_peered":"2023-11-06T20:44:06.398113+0000","last_clean":"2023-11-06T20:44:06.398113+0000","last_became_active":"2023-11-06T20:44:05.914850+0000","last_became_peered":"2023-11-06T20:44:05.914850+0000","last_unstale":"2023-11-06T20:44:06.398113+0000","last_undegraded":"2023-11-06T20:44:06.398113+0000","last_fullsized":"2023-11-06T20:44:06.398113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419324+0000","last_change":"2023-11-06T20:44:06.407493+0000","last_active":"2023-11-06T20:44:06.419324+0000","last_peered":"2023-11-06T20:44:06.419324+0000","last_clean":"2023-11-06T20:44:06.419324+0000","last_became_active":"2023-11-06T20:44:06.407167+0000","last_became_peered":"2023-11-06T20:44:06.407167+0000","last_unstale":"2023-11-06T20:44:06.419324+0000","last_undegraded":"2023-11-06T20:44:06.419324+0000","last_fullsized":"2023-11-06T20:44:06.419324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.389137+0000","last_change":"2023-11-06T20:44:05.934566+0000","last_active":"2023-11-06T20:44:06.389137+0000","last_peered":"2023-11-06T20:44:06.389137+0000","last_clean":"2023-11-06T20:44:06.389137+0000","last_became_active":"2023-11-06T20:44:05.934165+0000","last_became_peered":"2023-11-06T20:44:05.934165+0000","last_unstale":"2023-11-06T20:44:06.389137+0000","last_undegraded":"2023-11-06T20:44:06.389137+0000","last_fullsized":"2023-11-06T20:44:06.389137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.423114+0000","last_change":"2023-11-06T20:44:06.417767+0000","last_active":"2023-11-06T20:44:06.423114+0000","last_peered":"2023-11-06T20:44:06.423114+0000","last_clean":"2023-11-06T20:44:06.423114+0000","last_became_active":"2023-11-06T20:44:06.417468+0000","last_became_peered":"2023-11-06T20:44:06.417468+0000","last_unstale":"2023-11-06T20:44:06.423114+0000","last_undegraded":"2023-11-06T20:44:06.423114+0000","last_fullsized":"2023-11-06T20:44:06.423114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.30","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427626+0000","last_change":"2023-11-06T20:44:05.930422+0000","last_active":"2023-11-06T20:44:06.427626+0000","last_peered":"2023-11-06T20:44:06.427626+0000","last_clean":"2023-11-06T20:44:06.427626+0000","last_became_active":"2023-11-06T20:44:05.930070+0000","last_became_peered":"2023-11-06T20:44:05.930070+0000","last_unstale":"2023-11-06T20:44:06.427626+0000","last_undegraded":"2023-11-06T20:44:06.427626+0000","last_fullsized":"2023-11-06T20:44:06.427626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.411716+0000","last_change":"2023-11-06T20:44:06.407092+0000","last_active":"2023-11-06T20:44:06.411716+0000","last_peered":"2023-11-06T20:44:06.411716+0000","last_clean":"2023-11-06T20:44:06.411716+0000","last_became_active":"2023-11-06T20:44:06.406767+0000","last_became_peered":"2023-11-06T20:44:06.406767+0000","last_unstale":"2023-11-06T20:44:06.411716+0000","last_undegraded":"2023-11-06T20:44:06.411716+0000","last_fullsized":"2023-11-06T20:44:06.411716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393919+0000","last_change":"2023-11-06T20:44:05.923803+0000","last_active":"2023-11-06T20:44:06.393919+0000","last_peered":"2023-11-06T20:44:06.393919+0000","last_clean":"2023-11-06T20:44:06.393919+0000","last_became_active":"2023-11-06T20:44:05.923361+0000","last_became_peered":"2023-11-06T20:44:05.923361+0000","last_unstale":"2023-11-06T20:44:06.393919+0000","last_undegraded":"2023-11-06T20:44:06.393919+0000","last_fullsized":"2023-11-06T20:44:06.393919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420576+0000","last_change":"2023-11-06T20:44:06.419551+0000","last_active":"2023-11-06T20:44:06.420576+0000","last_peered":"2023-11-06T20:44:06.420576+0000","last_clean":"2023-11-06T20:44:06.420576+0000","last_became_active":"2023-11-06T20:44:06.419244+0000","last_became_peered":"2023-11-06T20:44:06.419244+0000","last_unstale":"2023-11-06T20:44:06.420576+0000","last_undegraded":"2023-11-06T20:44:06.420576+0000","last_fullsized":"2023-11-06T20:44:06.420576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.398750+0000","last_change":"2023-11-06T20:44:05.915503+0000","last_active":"2023-11-06T20:44:06.398750+0000","last_peered":"2023-11-06T20:44:06.398750+0000","last_clean":"2023-11-06T20:44:06.398750+0000","last_became_active":"2023-11-06T20:44:05.914672+0000","last_became_peered":"2023-11-06T20:44:05.914672+0000","last_unstale":"2023-11-06T20:44:06.398750+0000","last_undegraded":"2023-11-06T20:44:06.398750+0000","last_fullsized":"2023-11-06T20:44:06.398750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424374+0000","last_change":"2023-11-06T20:44:06.418248+0000","last_active":"2023-11-06T20:44:06.424374+0000","last_peered":"2023-11-06T20:44:06.424374+0000","last_clean":"2023-11-06T20:44:06.424374+0000","last_became_active":"2023-11-06T20:44:06.408712+0000","last_became_peered":"2023-11-06T20:44:06.408712+0000","last_unstale":"2023-11-06T20:44:06.424374+0000","last_undegraded":"2023-11-06T20:44:06.424374+0000","last_fullsized":"2023-11-06T20:44:06.424374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.33","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.425162+0000","last_change":"2023-11-06T20:44:05.927963+0000","last_active":"2023-11-06T20:44:06.425162+0000","last_peered":"2023-11-06T20:44:06.425162+0000","last_clean":"2023-11-06T20:44:06.425162+0000","last_became_active":"2023-11-06T20:44:05.927646+0000","last_became_peered":"2023-11-06T20:44:05.927646+0000","last_unstale":"2023-11-06T20:44:06.425162+0000","last_undegraded":"2023-11-06T20:44:06.425162+0000","last_fullsized":"2023-11-06T20:44:06.425162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420270+0000","last_change":"2023-11-06T20:44:06.413775+0000","last_active":"2023-11-06T20:44:06.420270+0000","last_peered":"2023-11-06T20:44:06.420270+0000","last_clean":"2023-11-06T20:44:06.420270+0000","last_became_active":"2023-11-06T20:44:06.413505+0000","last_became_peered":"2023-11-06T20:44:06.413505+0000","last_unstale":"2023-11-06T20:44:06.420270+0000","last_undegraded":"2023-11-06T20:44:06.420270+0000","last_fullsized":"2023-11-06T20:44:06.420270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.429184+0000","last_change":"2023-11-06T20:44:05.931664+0000","last_active":"2023-11-06T20:44:06.429184+0000","last_peered":"2023-11-06T20:44:06.429184+0000","last_clean":"2023-11-06T20:44:06.429184+0000","last_became_active":"2023-11-06T20:44:05.930383+0000","last_became_peered":"2023-11-06T20:44:05.930383+0000","last_unstale":"2023-11-06T20:44:06.429184+0000","last_undegraded":"2023-11-06T20:44:06.429184+0000","last_fullsized":"2023-11-06T20:44:06.429184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.401400+0000","last_change":"2023-11-06T20:44:06.399159+0000","last_active":"2023-11-06T20:44:06.401400+0000","last_peered":"2023-11-06T20:44:06.401400+0000","last_clean":"2023-11-06T20:44:06.401400+0000","last_became_active":"2023-11-06T20:44:06.398305+0000","last_became_peered":"2023-11-06T20:44:06.398305+0000","last_unstale":"2023-11-06T20:44:06.401400+0000","last_undegraded":"2023-11-06T20:44:06.401400+0000","last_fullsized":"2023-11-06T20:44:06.401400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.404918+0000","last_change":"2023-11-06T20:44:05.934808+0000","last_active":"2023-11-06T20:44:06.404918+0000","last_peered":"2023-11-06T20:44:06.404918+0000","last_clean":"2023-11-06T20:44:06.404918+0000","last_became_active":"2023-11-06T20:44:05.934503+0000","last_became_peered":"2023-11-06T20:44:05.934503+0000","last_unstale":"2023-11-06T20:44:06.404918+0000","last_undegraded":"2023-11-06T20:44:06.404918+0000","last_fullsized":"2023-11-06T20:44:06.404918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421422+0000","last_change":"2023-11-06T20:44:06.413884+0000","last_active":"2023-11-06T20:44:06.421422+0000","last_peered":"2023-11-06T20:44:06.421422+0000","last_clean":"2023-11-06T20:44:06.421422+0000","last_became_active":"2023-11-06T20:44:06.413013+0000","last_became_peered":"2023-11-06T20:44:06.413013+0000","last_unstale":"2023-11-06T20:44:06.421422+0000","last_undegraded":"2023-11-06T20:44:06.421422+0000","last_fullsized":"2023-11-06T20:44:06.421422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407560+0000","last_change":"2023-11-06T20:44:05.920301+0000","last_active":"2023-11-06T20:44:06.407560+0000","last_peered":"2023-11-06T20:44:06.407560+0000","last_clean":"2023-11-06T20:44:06.407560+0000","last_became_active":"2023-11-06T20:44:05.917401+0000","last_became_peered":"2023-11-06T20:44:05.917401+0000","last_unstale":"2023-11-06T20:44:06.407560+0000","last_undegraded":"2023-11-06T20:44:06.407560+0000","last_fullsized":"2023-11-06T20:44:06.407560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424639+0000","last_change":"2023-11-06T20:44:06.406242+0000","last_active":"2023-11-06T20:44:06.424639+0000","last_peered":"2023-11-06T20:44:06.424639+0000","last_clean":"2023-11-06T20:44:06.424639+0000","last_became_active":"2023-11-06T20:44:06.405979+0000","last_became_peered":"2023-11-06T20:44:06.405979+0000","last_unstale":"2023-11-06T20:44:06.424639+0000","last_undegraded":"2023-11-06T20:44:06.424639+0000","last_fullsized":"2023-11-06T20:44:06.424639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.396663+0000","last_change":"2023-11-06T20:44:05.939218+0000","last_active":"2023-11-06T20:44:06.396663+0000","last_peered":"2023-11-06T20:44:06.396663+0000","last_clean":"2023-11-06T20:44:06.396663+0000","last_became_active":"2023-11-06T20:44:05.938856+0000","last_became_peered":"2023-11-06T20:44:05.938856+0000","last_unstale":"2023-11-06T20:44:06.396663+0000","last_undegraded":"2023-11-06T20:44:06.396663+0000","last_fullsized":"2023-11-06T20:44:06.396663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.424019+0000","last_change":"2023-11-06T20:44:06.416017+0000","last_active":"2023-11-06T20:44:06.424019+0000","last_peered":"2023-11-06T20:44:06.424019+0000","last_clean":"2023-11-06T20:44:06.424019+0000","last_became_active":"2023-11-06T20:44:06.414205+0000","last_became_peered":"2023-11-06T20:44:06.414205+0000","last_unstale":"2023-11-06T20:44:06.424019+0000","last_undegraded":"2023-11-06T20:44:06.424019+0000","last_fullsized":"2023-11-06T20:44:06.424019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.393064+0000","last_change":"2023-11-06T20:44:05.928182+0000","last_active":"2023-11-06T20:44:06.393064+0000","last_peered":"2023-11-06T20:44:06.393064+0000","last_clean":"2023-11-06T20:44:06.393064+0000","last_became_active":"2023-11-06T20:44:05.927847+0000","last_became_peered":"2023-11-06T20:44:05.927847+0000","last_unstale":"2023-11-06T20:44:06.393064+0000","last_undegraded":"2023-11-06T20:44:06.393064+0000","last_fullsized":"2023-11-06T20:44:06.393064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.416917+0000","last_change":"2023-11-06T20:44:06.412444+0000","last_active":"2023-11-06T20:44:06.416917+0000","last_peered":"2023-11-06T20:44:06.416917+0000","last_clean":"2023-11-06T20:44:06.416917+0000","last_became_active":"2023-11-06T20:44:06.412113+0000","last_became_peered":"2023-11-06T20:44:06.412113+0000","last_unstale":"2023-11-06T20:44:06.416917+0000","last_undegraded":"2023-11-06T20:44:06.416917+0000","last_fullsized":"2023-11-06T20:44:06.416917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.403846+0000","last_change":"2023-11-06T20:44:05.929538+0000","last_active":"2023-11-06T20:44:06.403846+0000","last_peered":"2023-11-06T20:44:06.403846+0000","last_clean":"2023-11-06T20:44:06.403846+0000","last_became_active":"2023-11-06T20:44:05.922763+0000","last_became_peered":"2023-11-06T20:44:05.922763+0000","last_unstale":"2023-11-06T20:44:06.403846+0000","last_undegraded":"2023-11-06T20:44:06.403846+0000","last_fullsized":"2023-11-06T20:44:06.403846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426978+0000","last_change":"2023-11-06T20:44:06.423931+0000","last_active":"2023-11-06T20:44:06.426978+0000","last_peered":"2023-11-06T20:44:06.426978+0000","last_clean":"2023-11-06T20:44:06.426978+0000","last_became_active":"2023-11-06T20:44:06.423644+0000","last_became_peered":"2023-11-06T20:44:06.423644+0000","last_unstale":"2023-11-06T20:44:06.426978+0000","last_undegraded":"2023-11-06T20:44:06.426978+0000","last_fullsized":"2023-11-06T20:44:06.426978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420060+0000","last_change":"2023-11-06T20:44:05.909687+0000","last_active":"2023-11-06T20:44:06.420060+0000","last_peered":"2023-11-06T20:44:06.420060+0000","last_clean":"2023-11-06T20:44:06.420060+0000","last_became_active":"2023-11-06T20:44:05.907015+0000","last_became_peered":"2023-11-06T20:44:05.907015+0000","last_unstale":"2023-11-06T20:44:06.420060+0000","last_undegraded":"2023-11-06T20:44:06.420060+0000","last_fullsized":"2023-11-06T20:44:06.420060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421264+0000","last_change":"2023-11-06T20:44:06.415345+0000","last_active":"2023-11-06T20:44:06.421264+0000","last_peered":"2023-11-06T20:44:06.421264+0000","last_clean":"2023-11-06T20:44:06.421264+0000","last_became_active":"2023-11-06T20:44:06.414850+0000","last_became_peered":"2023-11-06T20:44:06.414850+0000","last_unstale":"2023-11-06T20:44:06.421264+0000","last_undegraded":"2023-11-06T20:44:06.421264+0000","last_fullsized":"2023-11-06T20:44:06.421264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.421636+0000","last_change":"2023-11-06T20:44:05.921703+0000","last_active":"2023-11-06T20:44:06.421636+0000","last_peered":"2023-11-06T20:44:06.421636+0000","last_clean":"2023-11-06T20:44:06.421636+0000","last_became_active":"2023-11-06T20:44:05.920678+0000","last_became_peered":"2023-11-06T20:44:05.920678+0000","last_unstale":"2023-11-06T20:44:06.421636+0000","last_undegraded":"2023-11-06T20:44:06.421636+0000","last_fullsized":"2023-11-06T20:44:06.421636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419871+0000","last_change":"2023-11-06T20:44:06.412730+0000","last_active":"2023-11-06T20:44:06.419871+0000","last_peered":"2023-11-06T20:44:06.419871+0000","last_clean":"2023-11-06T20:44:06.419871+0000","last_became_active":"2023-11-06T20:44:06.411623+0000","last_became_peered":"2023-11-06T20:44:06.411623+0000","last_unstale":"2023-11-06T20:44:06.419871+0000","last_undegraded":"2023-11-06T20:44:06.419871+0000","last_fullsized":"2023-11-06T20:44:06.419871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.420499+0000","last_change":"2023-11-06T20:44:05.919149+0000","last_active":"2023-11-06T20:44:06.420499+0000","last_peered":"2023-11-06T20:44:06.420499+0000","last_clean":"2023-11-06T20:44:06.420499+0000","last_became_active":"2023-11-06T20:44:05.918833+0000","last_became_peered":"2023-11-06T20:44:05.918833+0000","last_unstale":"2023-11-06T20:44:06.420499+0000","last_undegraded":"2023-11-06T20:44:06.420499+0000","last_fullsized":"2023-11-06T20:44:06.420499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.407334+0000","last_change":"2023-11-06T20:44:06.399010+0000","last_active":"2023-11-06T20:44:06.407334+0000","last_peered":"2023-11-06T20:44:06.407334+0000","last_clean":"2023-11-06T20:44:06.407334+0000","last_became_active":"2023-11-06T20:44:06.398749+0000","last_became_peered":"2023-11-06T20:44:06.398749+0000","last_unstale":"2023-11-06T20:44:06.407334+0000","last_undegraded":"2023-11-06T20:44:06.407334+0000","last_fullsized":"2023-11-06T20:44:06.407334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.394817+0000","last_change":"2023-11-06T20:44:05.911533+0000","last_active":"2023-11-06T20:44:06.394817+0000","last_peered":"2023-11-06T20:44:06.394817+0000","last_clean":"2023-11-06T20:44:06.394817+0000","last_became_active":"2023-11-06T20:44:05.911170+0000","last_became_peered":"2023-11-06T20:44:05.911170+0000","last_unstale":"2023-11-06T20:44:06.394817+0000","last_undegraded":"2023-11-06T20:44:06.394817+0000","last_fullsized":"2023-11-06T20:44:06.394817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.426932+0000","last_change":"2023-11-06T20:44:06.413837+0000","last_active":"2023-11-06T20:44:06.426932+0000","last_peered":"2023-11-06T20:44:06.426932+0000","last_clean":"2023-11-06T20:44:06.426932+0000","last_became_active":"2023-11-06T20:44:06.413575+0000","last_became_peered":"2023-11-06T20:44:06.413575+0000","last_unstale":"2023-11-06T20:44:06.426932+0000","last_undegraded":"2023-11-06T20:44:06.426932+0000","last_fullsized":"2023-11-06T20:44:06.426932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.395088+0000","last_change":"2023-11-06T20:44:05.919710+0000","last_active":"2023-11-06T20:44:06.395088+0000","last_peered":"2023-11-06T20:44:06.395088+0000","last_clean":"2023-11-06T20:44:06.395088+0000","last_became_active":"2023-11-06T20:44:05.919424+0000","last_became_peered":"2023-11-06T20:44:05.919424+0000","last_unstale":"2023-11-06T20:44:06.395088+0000","last_undegraded":"2023-11-06T20:44:06.395088+0000","last_fullsized":"2023-11-06T20:44:06.395088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.419344+0000","last_change":"2023-11-06T20:44:06.410070+0000","last_active":"2023-11-06T20:44:06.419344+0000","last_peered":"2023-11-06T20:44:06.419344+0000","last_clean":"2023-11-06T20:44:06.419344+0000","last_became_active":"2023-11-06T20:44:06.409768+0000","last_became_peered":"2023-11-06T20:44:06.409768+0000","last_unstale":"2023-11-06T20:44:06.419344+0000","last_undegraded":"2023-11-06T20:44:06.419344+0000","last_fullsized":"2023-11-06T20:44:06.419344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:05.880924+0000","last_clean_scrub_stamp":"2023-11-06T20:44:05.880924+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-06T20:44:06.427907+0000","last_change":"2023-11-06T20:44:05.937805+0000","last_active":"2023-11-06T20:44:06.427907+0000","last_peered":"2023-11-06T20:44:06.427907+0000","last_clean":"2023-11-06T20:44:06.427907+0000","last_became_active":"2023-11-06T20:44:05.937296+0000","last_became_peered":"2023-11-06T20:44:05.937296+0000","last_unstale":"2023-11-06T20:44:06.427907+0000","last_undegraded":"2023-11-06T20:44:06.427907+0000","last_fullsized":"2023-11-06T20:44:06.427907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:44:04.874230+0000","last_clean_scrub_stamp":"2023-11-06T20:44:04.874230+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297308,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074532,"statfs":{"total":96636764160,"available":96332320768,"internally_reserved":0,"allocated":217088,"data_stored":80225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297312,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074528,"statfs":{"total":96636764160,"available":96332316672,"internally_reserved":0,"allocated":221184,"data_stored":80315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":81139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297328,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074512,"statfs":{"total":96636764160,"available":96332300288,"internally_reserved":0,"allocated":237568,"data_stored":81817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":80869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297328,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074512,"statfs":{"total":96636764160,"available":96332300288,"internally_reserved":0,"allocated":237568,"data_stored":81527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297312,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074528,"statfs":{"total":96636764160,"available":96332316672,"internally_reserved":0,"allocated":221184,"data_stored":80295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297308,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074532,"statfs":{"total":96636764160,"available":96332320768,"internally_reserved":0,"allocated":217088,"data_stored":80225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:44:18.636 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-06T20:44:18.636 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-06T20:44:18.636 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-06T20:44:18.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-06T20:44:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.864+0000 7ff6ab9ff700 1 Processor -- start 2023-11-06T20:44:18.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.864+0000 7ff6ab9ff700 1 -- start start 2023-11-06T20:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.865+0000 7ff6ab9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c1840 0x7ff6a40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.865+0000 7ff6ab9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a41554a0 con 0x7ff6a40aa2d0 2023-11-06T20:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.865+0000 7ff6ab9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a4155930 con 0x7ff6a40c6460 2023-11-06T20:44:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.865+0000 7ff6ab9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4154960 con 0x7ff6a40c1840 2023-11-06T20:44:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff6a8f9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff6a40aa2d0 0x7ff6a40c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51642/0 (socket says 172.21.15.33:51642) 2023-11-06T20:44:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff6a8f9a700 1 -- 172.21.15.33:0/1482034084 learned_addr learned my addr 172.21.15.33:0/1482034084 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff6a979b700 1 --2- 172.21.15.33:0/1482034084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c1840 0x7ff6a40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff69bfff700 1 -- 172.21.15.33:0/1482034084 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550316131 0 0) 0x7ff6a41554a0 con 0x7ff6a40aa2d0 2023-11-06T20:44:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff69bfff700 1 -- 172.21.15.33:0/1482034084 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6900033f0 con 0x7ff6a40aa2d0 2023-11-06T20:44:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff69bfff700 1 -- 172.21.15.33:0/1482034084 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154729396 0 0) 0x7ff6a4155930 con 0x7ff6a40c6460 2023-11-06T20:44:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff69bfff700 1 -- 172.21.15.33:0/1482034084 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a41554a0 con 0x7ff6a40c6460 2023-11-06T20:44:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff6a979b700 1 -- 172.21.15.33:0/1482034084 >> v1:172.21.15.69:6789/0 conn(0x7ff6a40c6460 legacy=0x7ff6a40bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff6a979b700 1 -- 172.21.15.33:0/1482034084 >> v1:172.21.15.33:6789/0 conn(0x7ff6a40aa2d0 legacy=0x7ff6a40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff6a979b700 1 -- 172.21.15.33:0/1482034084 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a4154b70 con 0x7ff6a40c1840 2023-11-06T20:44:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.866+0000 7ff6a979b700 1 --2- 172.21.15.33:0/1482034084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c1840 0x7ff6a40a9de0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff6a0009790 tx=0x7ff6a0009060).ready entity=mon.2 client_cookie=ccbfd00256e6cabc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.867+0000 7ff69bfff700 1 -- 172.21.15.33:0/1482034084 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a000f510 con 0x7ff6a40c1840 2023-11-06T20:44:18.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.867+0000 7ff69bfff700 1 -- 172.21.15.33:0/1482034084 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a000f670 con 0x7ff6a40c1840 2023-11-06T20:44:18.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.867+0000 7ff69bfff700 1 -- 172.21.15.33:0/1482034084 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a000f9f0 con 0x7ff6a40c1840 2023-11-06T20:44:18.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.867+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/1482034084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c1840 msgr2=0x7ff6a40a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.867+0000 7ff6ab9ff700 1 --2- 172.21.15.33:0/1482034084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c1840 0x7ff6a40a9de0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff6a0009790 tx=0x7ff6a0009060).stop 2023-11-06T20:44:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.867+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/1482034084 shutdown_connections 2023-11-06T20:44:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.867+0000 7ff6ab9ff700 1 --2- 172.21.15.33:0/1482034084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c1840 0x7ff6a40a9de0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.867+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/1482034084 >> 172.21.15.33:0/1482034084 conn(0x7ff6a405ec80 msgr2=0x7ff6a40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.867+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/1482034084 shutdown_connections 2023-11-06T20:44:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.867+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/1482034084 wait complete. 2023-11-06T20:44:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.869+0000 7ff6ab9ff700 1 Processor -- start 2023-11-06T20:44:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.869+0000 7ff6ab9ff700 1 -- start start 2023-11-06T20:44:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.870+0000 7ff6ab9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40aa2d0 0x7ff6a40b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.870+0000 7ff6ab9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a40c1840 0x7ff6a40c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.870+0000 7ff6ab9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40c6460 0x7ff6a40c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.870+0000 7ff6ab9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4165cf0 con 0x7ff6a40c6460 2023-11-06T20:44:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.870+0000 7ff6ab9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40c9600 con 0x7ff6a40c1840 2023-11-06T20:44:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.870+0000 7ff6ab9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4156be0 con 0x7ff6a40aa2d0 2023-11-06T20:44:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.870+0000 7ff6a9f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40c6460 0x7ff6a40c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.870+0000 7ff6a9f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40c6460 0x7ff6a40c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33580/0 (socket says 172.21.15.33:33580) 2023-11-06T20:44:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.870+0000 7ff6a9f9c700 1 -- 172.21.15.33:0/2075256110 learned_addr learned my addr 172.21.15.33:0/2075256110 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6a8f9a700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a40c1840 0x7ff6a40c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6a979b700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40aa2d0 0x7ff6a40b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6a9f9c700 1 -- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40aa2d0 msgr2=0x7ff6a40b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6a9f9c700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40aa2d0 0x7ff6a40b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6a9f9c700 1 -- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a40c1840 msgr2=0x7ff6a40c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:18.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6a9f9c700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a40c1840 0x7ff6a40c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:18.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6a9f9c700 1 -- 172.21.15.33:0/2075256110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a00092a0 con 0x7ff6a40c6460 2023-11-06T20:44:18.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6a979b700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40aa2d0 0x7ff6a40b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:18.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6a8f9a700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a40c1840 0x7ff6a40c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:18.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6a9f9c700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40c6460 0x7ff6a40c9110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7ff69c00c7c0 tx=0x7ff69c007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:18.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6997fa700 1 -- 172.21.15.33:0/2075256110 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c010510 con 0x7ff6a40c6460 2023-11-06T20:44:18.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6997fa700 1 -- 172.21.15.33:0/2075256110 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c010670 con 0x7ff6a40c6460 2023-11-06T20:44:18.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/2075256110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a4164ce0 con 0x7ff6a40c6460 2023-11-06T20:44:18.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6997fa700 1 -- 172.21.15.33:0/2075256110 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c010a30 con 0x7ff6a40c6460 2023-11-06T20:44:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.871+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/2075256110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a4165210 con 0x7ff6a40c6460 2023-11-06T20:44:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.873+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/2075256110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a406c6c0 con 0x7ff6a40c6460 2023-11-06T20:44:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.873+0000 7ff6997fa700 1 -- 172.21.15.33:0/2075256110 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff69c01b400 con 0x7ff6a40c6460 2023-11-06T20:44:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.875+0000 7ff6997fa700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f120 0x7ff6880615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.875+0000 7ff6997fa700 1 -- 172.21.15.33:0/2075256110 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff69c07a010 con 0x7ff6a40c6460 2023-11-06T20:44:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.875+0000 7ff6a979b700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f120 0x7ff6880615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.876+0000 7ff6a979b700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f120 0x7ff6880615e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7ff6a00096e0 tx=0x7ff6a0017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:18.878+0000 7ff6997fa700 1 -- 172.21.15.33:0/2075256110 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff69c04b280 con 0x7ff6a40c6460 2023-11-06T20:44:19.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.097+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/2075256110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff6a40cb2d0 con 0x7ff6a40c6460 2023-11-06T20:44:19.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.097+0000 7ff6997fa700 1 -- 172.21.15.33:0/2075256110 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7ff69c050040 con 0x7ff6a40c6460 2023-11-06T20:44:19.097 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:19.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f120 msgr2=0x7ff6880615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f120 0x7ff6880615e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7ff6a00096e0 tx=0x7ff6a0017000).stop 2023-11-06T20:44:19.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/2075256110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40c6460 msgr2=0x7ff6a40c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40c6460 0x7ff6a40c9110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7ff69c00c7c0 tx=0x7ff69c007f50).stop 2023-11-06T20:44:19.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/2075256110 shutdown_connections 2023-11-06T20:44:19.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f120 0x7ff6880615e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40aa2d0 0x7ff6a40b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40c6460 0x7ff6a40c9110 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 --2- 172.21.15.33:0/2075256110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a40c1840 0x7ff6a40c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/2075256110 >> 172.21.15.33:0/2075256110 conn(0x7ff6a405ec80 msgr2=0x7ff6a4063140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:19.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.099+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/2075256110 shutdown_connections 2023-11-06T20:44:19.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.100+0000 7ff6ab9ff700 1 -- 172.21.15.33:0/2075256110 wait complete. 2023-11-06T20:44:19.116 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-06T20:44:19.116 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-06T20:44:19.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:19.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.313+0000 7f4e16dff700 1 Processor -- start 2023-11-06T20:44:19.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.313+0000 7f4e16dff700 1 -- start start 2023-11-06T20:44:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.314+0000 7f4e16dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e100c5d50 0x7f4e100c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.314+0000 7f4e16dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e101554a0 con 0x7f4e100c1840 2023-11-06T20:44:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.314+0000 7f4e16dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e100c66c0 con 0x7f4e100bfd70 2023-11-06T20:44:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.314+0000 7f4e16dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e10154960 con 0x7f4e100c5d50 2023-11-06T20:44:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.315+0000 7f4e14b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4e100c1840 0x7f4e100a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51656/0 (socket says 172.21.15.33:51656) 2023-11-06T20:44:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.315+0000 7f4e14b9b700 1 -- 172.21.15.33:0/2849258135 learned_addr learned my addr 172.21.15.33:0/2849258135 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.315+0000 7f4e0ffff700 1 --2- 172.21.15.33:0/2849258135 >> v2:172.21.15.33:3300/0 conn(0x7f4e100c5d50 0x7f4e100c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.315+0000 7f4e0f7fe700 1 -- 172.21.15.33:0/2849258135 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958126014 0 0) 0x7f4e101554a0 con 0x7f4e100c1840 2023-11-06T20:44:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.315+0000 7f4e0f7fe700 1 -- 172.21.15.33:0/2849258135 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e000033f0 con 0x7f4e100c1840 2023-11-06T20:44:19.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.315+0000 7f4e0ffff700 1 -- 172.21.15.33:0/2849258135 >> v1:172.21.15.69:6789/0 conn(0x7f4e100bfd70 legacy=0x7f4e100c0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.315+0000 7f4e0ffff700 1 -- 172.21.15.33:0/2849258135 >> v1:172.21.15.33:6789/0 conn(0x7f4e100c1840 legacy=0x7f4e100a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.315+0000 7f4e0ffff700 1 -- 172.21.15.33:0/2849258135 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e10155930 con 0x7f4e100c5d50 2023-11-06T20:44:19.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.315+0000 7f4e0ffff700 1 --2- 172.21.15.33:0/2849258135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 0x7f4e100c61d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f4dfc009760 tx=0x7f4dfc009060).ready entity=mon.0 client_cookie=3109d7d4b64ad686 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:19.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.316+0000 7f4e0f7fe700 1 -- 172.21.15.33:0/2849258135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00f510 con 0x7f4e100c5d50 2023-11-06T20:44:19.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.316+0000 7f4e0f7fe700 1 -- 172.21.15.33:0/2849258135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dfc00f670 con 0x7f4e100c5d50 2023-11-06T20:44:19.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.316+0000 7f4e0f7fe700 1 -- 172.21.15.33:0/2849258135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00f9f0 con 0x7f4e100c5d50 2023-11-06T20:44:19.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.316+0000 7f4e16dff700 1 -- 172.21.15.33:0/2849258135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 msgr2=0x7f4e100c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.316+0000 7f4e16dff700 1 --2- 172.21.15.33:0/2849258135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 0x7f4e100c61d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f4dfc009760 tx=0x7f4dfc009060).stop 2023-11-06T20:44:19.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.316+0000 7f4e16dff700 1 -- 172.21.15.33:0/2849258135 shutdown_connections 2023-11-06T20:44:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.316+0000 7f4e16dff700 1 --2- 172.21.15.33:0/2849258135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 0x7f4e100c61d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.316+0000 7f4e16dff700 1 -- 172.21.15.33:0/2849258135 >> 172.21.15.33:0/2849258135 conn(0x7f4e1005ec80 msgr2=0x7f4e100cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.316+0000 7f4e16dff700 1 -- 172.21.15.33:0/2849258135 shutdown_connections 2023-11-06T20:44:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.316+0000 7f4e16dff700 1 -- 172.21.15.33:0/2849258135 wait complete. 2023-11-06T20:44:19.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.318+0000 7f4e16dff700 1 Processor -- start 2023-11-06T20:44:19.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.318+0000 7f4e16dff700 1 -- start start 2023-11-06T20:44:19.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e16dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e100bfd70 0x7f4e10156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e16dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e100c1840 0x7f4e1014e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e16dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 0x7f4e101528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e16dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e101657c0 con 0x7f4e100c5d50 2023-11-06T20:44:19.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e16dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1014e950 con 0x7f4e100c1840 2023-11-06T20:44:19.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e16dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e10155c40 con 0x7f4e100bfd70 2023-11-06T20:44:19.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e1539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 0x7f4e101528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e1539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 0x7f4e101528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33594/0 (socket says 172.21.15.33:33594) 2023-11-06T20:44:19.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e1539c700 1 -- 172.21.15.33:0/3965711918 learned_addr learned my addr 172.21.15.33:0/3965711918 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:19.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e0ffff700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e100c1840 0x7f4e1014e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.319+0000 7f4e14b9b700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e100bfd70 0x7f4e10156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e1539c700 1 -- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e100bfd70 msgr2=0x7f4e10156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e1539c700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e100bfd70 0x7f4e10156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e1539c700 1 -- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e100c1840 msgr2=0x7f4e1014e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e1539c700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e100c1840 0x7f4e1014e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e1539c700 1 -- 172.21.15.33:0/3965711918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc0092a0 con 0x7f4e100c5d50 2023-11-06T20:44:19.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e14b9b700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e100bfd70 0x7f4e10156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:19.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e0ffff700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e100c1840 0x7f4e1014e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:19.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e1539c700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 0x7f4e101528c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f4e08002230 tx=0x7f4e080015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:19.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e0cff9700 1 -- 172.21.15.33:0/3965711918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08010510 con 0x7f4e100c5d50 2023-11-06T20:44:19.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e0cff9700 1 -- 172.21.15.33:0/3965711918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e08010670 con 0x7f4e100c5d50 2023-11-06T20:44:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e16dff700 1 -- 172.21.15.33:0/3965711918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e10155f20 con 0x7f4e100c5d50 2023-11-06T20:44:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e0cff9700 1 -- 172.21.15.33:0/3965711918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e080109f0 con 0x7f4e100c5d50 2023-11-06T20:44:19.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.320+0000 7f4e16dff700 1 -- 172.21.15.33:0/3965711918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e101564e0 con 0x7f4e100c5d50 2023-11-06T20:44:19.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.321+0000 7f4e16dff700 1 -- 172.21.15.33:0/3965711918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e10156700 con 0x7f4e100c5d50 2023-11-06T20:44:19.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.322+0000 7f4e0cff9700 1 -- 172.21.15.33:0/3965711918 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4e0801b400 con 0x7f4e100c5d50 2023-11-06T20:44:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.323+0000 7f4e0cff9700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4df405f150 0x7f4df4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.323+0000 7f4e0cff9700 1 -- 172.21.15.33:0/3965711918 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4e0807a110 con 0x7f4e100c5d50 2023-11-06T20:44:19.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.323+0000 7f4e14b9b700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4df405f150 0x7f4df4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.324+0000 7f4e14b9b700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4df405f150 0x7f4df4061610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f4e04005750 tx=0x7f4e04009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:19.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.327+0000 7f4e0cff9700 1 -- 172.21.15.33:0/3965711918 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4e0804b2c0 con 0x7f4e100c5d50 2023-11-06T20:44:19.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.510+0000 7f4e16dff700 1 -- 172.21.15.33:0/3965711918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e1006c6c0 con 0x7f4e100c5d50 2023-11-06T20:44:19.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.511+0000 7f4e0cff9700 1 -- 172.21.15.33:0/3965711918 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f4e0804e8e0 con 0x7f4e100c5d50 2023-11-06T20:44:19.511 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:19.511 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:19.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 -- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4df405f150 msgr2=0x7f4df4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4df405f150 0x7f4df4061610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f4e04005750 tx=0x7f4e04009040).stop 2023-11-06T20:44:19.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 -- 172.21.15.33:0/3965711918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 msgr2=0x7f4e101528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 0x7f4e101528c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f4e08002230 tx=0x7f4e080015e0).stop 2023-11-06T20:44:19.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 -- 172.21.15.33:0/3965711918 shutdown_connections 2023-11-06T20:44:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4df405f150 0x7f4df4061610 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e100bfd70 0x7f4e10156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e100c5d50 0x7f4e101528c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 --2- 172.21.15.33:0/3965711918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e100c1840 0x7f4e1014e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 -- 172.21.15.33:0/3965711918 >> 172.21.15.33:0/3965711918 conn(0x7f4e1005ec80 msgr2=0x7f4e10150b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 -- 172.21.15.33:0/3965711918 shutdown_connections 2023-11-06T20:44:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.513+0000 7f4e16dff700 1 -- 172.21.15.33:0/3965711918 wait complete. 2023-11-06T20:44:19.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:44:04.716205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:44:05.351480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T20:44:19.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:44:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.731+0000 7fcea97ff700 1 Processor -- start 2023-11-06T20:44:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.731+0000 7fcea97ff700 1 -- start start 2023-11-06T20:44:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.733+0000 7fcea97ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcea414e420 0x7fcea414e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.733+0000 7fcea97ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea40c5dc0 0x7fcea40c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.733+0000 7fcea97ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcea40c7640 0x7fcea40c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.733+0000 7fcea97ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40ad980 con 0x7fcea414e420 2023-11-06T20:44:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.733+0000 7fcea97ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea4158880 con 0x7fcea40c7640 2023-11-06T20:44:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.733+0000 7fcea97ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4157c90 con 0x7fcea40c5dc0 2023-11-06T20:44:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.733+0000 7fcea2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcea414e420 0x7fcea414e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.733+0000 7fcea2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcea414e420 0x7fcea414e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33598/0 (socket says 172.21.15.33:33598) 2023-11-06T20:44:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.733+0000 7fcea2ffd700 1 -- 172.21.15.33:0/3718787901 learned_addr learned my addr 172.21.15.33:0/3718787901 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea2ffd700 1 -- 172.21.15.33:0/3718787901 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea40c5dc0 msgr2=0x7fcea40c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.733+0000 7fcea27fc700 1 --2- 172.21.15.33:0/3718787901 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea40c5dc0 0x7fcea40c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea37fe700 1 --2- 172.21.15.33:0/3718787901 >> v2:172.21.15.69:3300/0 conn(0x7fcea40c7640 0x7fcea40c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea2ffd700 1 --2- 172.21.15.33:0/3718787901 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea40c5dc0 0x7fcea40c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea2ffd700 1 -- 172.21.15.33:0/3718787901 >> v2:172.21.15.69:3300/0 conn(0x7fcea40c7640 msgr2=0x7fcea40c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea2ffd700 1 --2- 172.21.15.33:0/3718787901 >> v2:172.21.15.69:3300/0 conn(0x7fcea40c7640 0x7fcea40c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea2ffd700 1 -- 172.21.15.33:0/3718787901 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea4157f50 con 0x7fcea414e420 2023-11-06T20:44:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea37fe700 1 --2- 172.21.15.33:0/3718787901 >> v2:172.21.15.69:3300/0 conn(0x7fcea40c7640 0x7fcea40c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea27fc700 1 --2- 172.21.15.33:0/3718787901 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea40c5dc0 0x7fcea40c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea2ffd700 1 --2- 172.21.15.33:0/3718787901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea414e420 0x7fcea414e800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fcea40cbdc0 tx=0x7fce940090f0).ready entity=mon.0 client_cookie=7c9edf875250bbbe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea1ffb700 1 -- 172.21.15.33:0/3718787901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400b600 con 0x7fcea414e420 2023-11-06T20:44:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea1ffb700 1 -- 172.21.15.33:0/3718787901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9400b760 con 0x7fcea414e420 2023-11-06T20:44:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea1ffb700 1 -- 172.21.15.33:0/3718787901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400bae0 con 0x7fcea414e420 2023-11-06T20:44:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.734+0000 7fcea97ff700 1 -- 172.21.15.33:0/3718787901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea414e420 msgr2=0x7fcea414e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.735+0000 7fcea97ff700 1 --2- 172.21.15.33:0/3718787901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea414e420 0x7fcea414e800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fcea40cbdc0 tx=0x7fce940090f0).stop 2023-11-06T20:44:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.735+0000 7fcea97ff700 1 -- 172.21.15.33:0/3718787901 shutdown_connections 2023-11-06T20:44:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.735+0000 7fcea97ff700 1 --2- 172.21.15.33:0/3718787901 >> v2:172.21.15.69:3300/0 conn(0x7fcea40c7640 0x7fcea40c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.735+0000 7fcea97ff700 1 --2- 172.21.15.33:0/3718787901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea414e420 0x7fcea414e800 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.735+0000 7fcea97ff700 1 --2- 172.21.15.33:0/3718787901 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea40c5dc0 0x7fcea40c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.735+0000 7fcea97ff700 1 -- 172.21.15.33:0/3718787901 >> 172.21.15.33:0/3718787901 conn(0x7fcea405e1b0 msgr2=0x7fcea405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.735+0000 7fcea97ff700 1 -- 172.21.15.33:0/3718787901 shutdown_connections 2023-11-06T20:44:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.735+0000 7fcea97ff700 1 -- 172.21.15.33:0/3718787901 wait complete. 2023-11-06T20:44:19.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.737+0000 7fcea97ff700 1 Processor -- start 2023-11-06T20:44:19.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.737+0000 7fcea97ff700 1 -- start start 2023-11-06T20:44:19.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.738+0000 7fcea97ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea40c5dc0 0x7fcea4144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.738+0000 7fcea97ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea40c7640 0x7fcea40c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.738+0000 7fcea97ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea414e420 0x7fcea40b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.738+0000 7fcea97ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40aff20 con 0x7fcea40c7640 2023-11-06T20:44:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.738+0000 7fcea97ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40af370 con 0x7fcea40c5dc0 2023-11-06T20:44:19.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.738+0000 7fcea97ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4145470 con 0x7fcea414e420 2023-11-06T20:44:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea40c7640 0x7fcea40c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea40c7640 0x7fcea40c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33612/0 (socket says 172.21.15.33:33612) 2023-11-06T20:44:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea27fc700 1 -- 172.21.15.33:0/3813966382 learned_addr learned my addr 172.21.15.33:0/3813966382 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea2ffd700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea40c5dc0 0x7fcea4144f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea37fe700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea414e420 0x7fcea40b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea27fc700 1 -- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea414e420 msgr2=0x7fcea40b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea27fc700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea414e420 0x7fcea40b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea27fc700 1 -- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea40c5dc0 msgr2=0x7fcea4144f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea27fc700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea40c5dc0 0x7fcea4144f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea27fc700 1 -- 172.21.15.33:0/3813966382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce94009370 con 0x7fcea40c7640 2023-11-06T20:44:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea37fe700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea414e420 0x7fcea40b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea27fc700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea40c7640 0x7fcea40c4470 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fce9800c8b0 tx=0x7fce98008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fce8affd700 1 -- 172.21.15.33:0/3813966382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98010580 con 0x7fcea40c7640 2023-11-06T20:44:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fce8affd700 1 -- 172.21.15.33:0/3813966382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce980106e0 con 0x7fcea40c7640 2023-11-06T20:44:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fcea97ff700 1 -- 172.21.15.33:0/3813966382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea40af5f0 con 0x7fcea40c7640 2023-11-06T20:44:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.739+0000 7fce8affd700 1 -- 172.21.15.33:0/3813966382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98010a60 con 0x7fcea40c7640 2023-11-06T20:44:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.740+0000 7fcea97ff700 1 -- 172.21.15.33:0/3813966382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea40afbb0 con 0x7fcea40c7640 2023-11-06T20:44:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.741+0000 7fcea97ff700 1 -- 172.21.15.33:0/3813966382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea406c6c0 con 0x7fcea40c7640 2023-11-06T20:44:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.741+0000 7fce8affd700 1 -- 172.21.15.33:0/3813966382 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fce9801b460 con 0x7fcea40c7640 2023-11-06T20:44:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.742+0000 7fce8affd700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce9005f150 0x7fce90061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.742+0000 7fce8affd700 1 -- 172.21.15.33:0/3813966382 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fce9807a210 con 0x7fcea40c7640 2023-11-06T20:44:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.743+0000 7fcea2ffd700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce9005f150 0x7fce90061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.743+0000 7fcea2ffd700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce9005f150 0x7fce90061610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fce94009130 tx=0x7fce94017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.746+0000 7fce8affd700 1 -- 172.21.15.33:0/3813966382 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fce9804b3c0 con 0x7fcea40c7640 2023-11-06T20:44:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.926+0000 7fcea97ff700 1 -- 172.21.15.33:0/3813966382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcea40b0060 con 0x7fcea40c7640 2023-11-06T20:44:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.926+0000 7fce8affd700 1 -- 172.21.15.33:0/3813966382 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7fce9804e9e0 con 0x7fcea40c7640 2023-11-06T20:44:19.926 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:44:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 -- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce9005f150 msgr2=0x7fce90061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce9005f150 0x7fce90061610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fce94009130 tx=0x7fce94017000).stop 2023-11-06T20:44:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 -- 172.21.15.33:0/3813966382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea40c7640 msgr2=0x7fcea40c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea40c7640 0x7fcea40c4470 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fce9800c8b0 tx=0x7fce98008040).stop 2023-11-06T20:44:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 -- 172.21.15.33:0/3813966382 shutdown_connections 2023-11-06T20:44:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce9005f150 0x7fce90061610 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcea414e420 0x7fcea40b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea40c5dc0 0x7fcea4144f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 --2- 172.21.15.33:0/3813966382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea40c7640 0x7fcea40c4470 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:19.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 -- 172.21.15.33:0/3813966382 >> 172.21.15.33:0/3813966382 conn(0x7fcea405e1b0 msgr2=0x7fcea4157010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:19.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 -- 172.21.15.33:0/3813966382 shutdown_connections 2023-11-06T20:44:19.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:19.929+0000 7fcea97ff700 1 -- 172.21.15.33:0/3813966382 wait complete. 2023-11-06T20:44:19.945 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:44:20.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.144+0000 7fe5409ff700 1 Processor -- start 2023-11-06T20:44:20.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.144+0000 7fe5409ff700 1 -- start start 2023-11-06T20:44:20.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.145+0000 7fe5409ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe53c154330 0x7fe53c0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.145+0000 7fe5409ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe53c154710 0x7fe53c0c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.145+0000 7fe5409ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53c0c6b60 con 0x7fe53c0c7ff0 2023-11-06T20:44:20.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.145+0000 7fe5409ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53c0c5cd0 con 0x7fe53c154710 2023-11-06T20:44:20.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.145+0000 7fe5409ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53c0c5f50 con 0x7fe53c154330 2023-11-06T20:44:20.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.145+0000 7fe539d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe53c154710 0x7fe53c0c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.145+0000 7fe539d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe53c154710 0x7fe53c0c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33616/0 (socket says 172.21.15.33:33616) 2023-11-06T20:44:20.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.145+0000 7fe539d9b700 1 -- 172.21.15.33:0/4066097537 learned_addr learned my addr 172.21.15.33:0/4066097537 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:20.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.146+0000 7fe53a59c700 1 --2- 172.21.15.33:0/4066097537 >> v2:172.21.15.69:3300/0 conn(0x7fe53c154330 0x7fe53c0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.146+0000 7fe539d9b700 1 -- 172.21.15.33:0/4066097537 >> v2:172.21.15.69:3300/0 conn(0x7fe53c154330 msgr2=0x7fe53c0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.146+0000 7fe539d9b700 1 --2- 172.21.15.33:0/4066097537 >> v2:172.21.15.69:3300/0 conn(0x7fe53c154330 0x7fe53c0c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.146+0000 7fe539d9b700 1 -- 172.21.15.33:0/4066097537 >> v1:172.21.15.69:6789/0 conn(0x7fe53c0c7ff0 legacy=0x7fe53c0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.146+0000 7fe539d9b700 1 -- 172.21.15.33:0/4066097537 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c0c6230 con 0x7fe53c154710 2023-11-06T20:44:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.146+0000 7fe53a59c700 1 --2- 172.21.15.33:0/4066097537 >> v2:172.21.15.69:3300/0 conn(0x7fe53c154330 0x7fe53c0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:20.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.146+0000 7fe539d9b700 1 --2- 172.21.15.33:0/4066097537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 0x7fe53c0c6ff0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fe53c0605d0 tx=0x7fe5300090f0).ready entity=mon.0 client_cookie=ef24d1fc781cafed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.146+0000 7fe53959a700 1 -- 172.21.15.33:0/4066097537 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53000b600 con 0x7fe53c154710 2023-11-06T20:44:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.146+0000 7fe53959a700 1 -- 172.21.15.33:0/4066097537 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53000b760 con 0x7fe53c154710 2023-11-06T20:44:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.146+0000 7fe53959a700 1 -- 172.21.15.33:0/4066097537 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53000bae0 con 0x7fe53c154710 2023-11-06T20:44:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.147+0000 7fe5409ff700 1 -- 172.21.15.33:0/4066097537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 msgr2=0x7fe53c0c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.147+0000 7fe5409ff700 1 --2- 172.21.15.33:0/4066097537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 0x7fe53c0c6ff0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fe53c0605d0 tx=0x7fe5300090f0).stop 2023-11-06T20:44:20.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.147+0000 7fe5409ff700 1 -- 172.21.15.33:0/4066097537 shutdown_connections 2023-11-06T20:44:20.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.147+0000 7fe5409ff700 1 --2- 172.21.15.33:0/4066097537 >> v2:172.21.15.69:3300/0 conn(0x7fe53c154330 0x7fe53c0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.147+0000 7fe5409ff700 1 --2- 172.21.15.33:0/4066097537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 0x7fe53c0c6ff0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.147+0000 7fe5409ff700 1 -- 172.21.15.33:0/4066097537 >> 172.21.15.33:0/4066097537 conn(0x7fe53c0cb910 msgr2=0x7fe53c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.147+0000 7fe5409ff700 1 -- 172.21.15.33:0/4066097537 shutdown_connections 2023-11-06T20:44:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.147+0000 7fe5409ff700 1 -- 172.21.15.33:0/4066097537 wait complete. 2023-11-06T20:44:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.148+0000 7fe5409ff700 1 Processor -- start 2023-11-06T20:44:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.149+0000 7fe5409ff700 1 -- start start 2023-11-06T20:44:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.149+0000 7fe5409ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe53c0c7ff0 0x7fe53c155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.149+0000 7fe5409ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe53c154330 0x7fe53c147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.149+0000 7fe5409ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 0x7fe53c0c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe5409ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c155930 con 0x7fe53c154710 2023-11-06T20:44:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe5409ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c0c1ee0 con 0x7fe53c154330 2023-11-06T20:44:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe5409ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c0c2780 con 0x7fe53c0c7ff0 2023-11-06T20:44:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe53ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 0x7fe53c0c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe53ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 0x7fe53c0c40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33626/0 (socket says 172.21.15.33:33626) 2023-11-06T20:44:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe53ad9d700 1 -- 172.21.15.33:0/4172220425 learned_addr learned my addr 172.21.15.33:0/4172220425 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe539d9b700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe53c154330 0x7fe53c147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe53a59c700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe53c0c7ff0 0x7fe53c155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe53ad9d700 1 -- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe53c0c7ff0 msgr2=0x7fe53c155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe53ad9d700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe53c0c7ff0 0x7fe53c155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe53ad9d700 1 -- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe53c154330 msgr2=0x7fe53c147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe53ad9d700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe53c154330 0x7fe53c147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe53ad9d700 1 -- 172.21.15.33:0/4172220425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe530009370 con 0x7fe53c154710 2023-11-06T20:44:20.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.150+0000 7fe53ad9d700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 0x7fe53c0c40d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe53400c7f0 tx=0x7fe534001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:20.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.151+0000 7fe52a7fc700 1 -- 172.21.15.33:0/4172220425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400e4a0 con 0x7fe53c154710 2023-11-06T20:44:20.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.151+0000 7fe52a7fc700 1 -- 172.21.15.33:0/4172220425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53400e600 con 0x7fe53c154710 2023-11-06T20:44:20.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.151+0000 7fe52a7fc700 1 -- 172.21.15.33:0/4172220425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400e980 con 0x7fe53c154710 2023-11-06T20:44:20.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.151+0000 7fe5409ff700 1 -- 172.21.15.33:0/4172220425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53c0b0230 con 0x7fe53c154710 2023-11-06T20:44:20.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.151+0000 7fe5409ff700 1 -- 172.21.15.33:0/4172220425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53c0c2410 con 0x7fe53c154710 2023-11-06T20:44:20.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.152+0000 7fe5409ff700 1 -- 172.21.15.33:0/4172220425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53c0c28c0 con 0x7fe53c154710 2023-11-06T20:44:20.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.153+0000 7fe52a7fc700 1 -- 172.21.15.33:0/4172220425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe5340123f0 con 0x7fe53c154710 2023-11-06T20:44:20.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.153+0000 7fe52a7fc700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe51c05f150 0x7fe51c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.154+0000 7fe52a7fc700 1 -- 172.21.15.33:0/4172220425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe534079a70 con 0x7fe53c154710 2023-11-06T20:44:20.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.154+0000 7fe53a59c700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe51c05f150 0x7fe51c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.154+0000 7fe53a59c700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe51c05f150 0x7fe51c061610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fe52c006d80 tx=0x7fe52c008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:20.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.157+0000 7fe52a7fc700 1 -- 172.21.15.33:0/4172220425 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe534047af0 con 0x7fe53c154710 2023-11-06T20:44:20.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.335+0000 7fe5409ff700 1 -- 172.21.15.33:0/4172220425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe53c0652d0 con 0x7fe53c154710 2023-11-06T20:44:20.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.335+0000 7fe52a7fc700 1 -- 172.21.15.33:0/4172220425 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe53404ac20 con 0x7fe53c154710 2023-11-06T20:44:20.335 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:44:20.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 -- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe51c05f150 msgr2=0x7fe51c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe51c05f150 0x7fe51c061610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fe52c006d80 tx=0x7fe52c008040).stop 2023-11-06T20:44:20.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 -- 172.21.15.33:0/4172220425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 msgr2=0x7fe53c0c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 0x7fe53c0c40d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe53400c7f0 tx=0x7fe534001510).stop 2023-11-06T20:44:20.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 -- 172.21.15.33:0/4172220425 shutdown_connections 2023-11-06T20:44:20.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe51c05f150 0x7fe51c061610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe53c0c7ff0 0x7fe53c155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53c154710 0x7fe53c0c40d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 --2- 172.21.15.33:0/4172220425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe53c154330 0x7fe53c147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 -- 172.21.15.33:0/4172220425 >> 172.21.15.33:0/4172220425 conn(0x7fe53c0cb910 msgr2=0x7fe53c0646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:20.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 -- 172.21.15.33:0/4172220425 shutdown_connections 2023-11-06T20:44:20.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.337+0000 7fe5409ff700 1 -- 172.21.15.33:0/4172220425 wait complete. 2023-11-06T20:44:20.354 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T20:44:20.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.553+0000 7fcaf4fff700 1 Processor -- start 2023-11-06T20:44:20.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.553+0000 7fcaf4fff700 1 -- start start 2023-11-06T20:44:20.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.554+0000 7fcaf4fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcaf0154710 0x7fcaf00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.554+0000 7fcaf4fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00c6b60 con 0x7fcaf0154330 2023-11-06T20:44:20.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.554+0000 7fcaf4fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00c7700 con 0x7fcaf00c7ff0 2023-11-06T20:44:20.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.554+0000 7fcaf4fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf00c5cd0 con 0x7fcaf0154710 2023-11-06T20:44:20.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.555+0000 7fcaee59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcaf0154330 0x7fcaf00c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51664/0 (socket says 172.21.15.33:51664) 2023-11-06T20:44:20.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.555+0000 7fcaee59c700 1 -- 172.21.15.33:0/174395578 learned_addr learned my addr 172.21.15.33:0/174395578 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:20.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.555+0000 7fcaedd9b700 1 --2- 172.21.15.33:0/174395578 >> v2:172.21.15.69:3300/0 conn(0x7fcaf0154710 0x7fcaf00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.555+0000 7fcaed59a700 1 -- 172.21.15.33:0/174395578 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566911353 0 0) 0x7fcaf00c6b60 con 0x7fcaf0154330 2023-11-06T20:44:20.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.555+0000 7fcaed59a700 1 -- 172.21.15.33:0/174395578 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcad80033f0 con 0x7fcaf0154330 2023-11-06T20:44:20.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.555+0000 7fcaedd9b700 1 -- 172.21.15.33:0/174395578 >> v1:172.21.15.69:6789/0 conn(0x7fcaf00c7ff0 legacy=0x7fcaf00646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.555+0000 7fcaedd9b700 1 -- 172.21.15.33:0/174395578 >> v1:172.21.15.33:6789/0 conn(0x7fcaf0154330 legacy=0x7fcaf00c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.555+0000 7fcaedd9b700 1 -- 172.21.15.33:0/174395578 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00c5f50 con 0x7fcaf0154710 2023-11-06T20:44:20.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.555+0000 7fcaedd9b700 1 --2- 172.21.15.33:0/174395578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf0154710 0x7fcaf00c57e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fcae4009760 tx=0x7fcae4009060).ready entity=mon.1 client_cookie=1ce0b617d6234a36 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:20.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.556+0000 7fcaed59a700 1 -- 172.21.15.33:0/174395578 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae400f510 con 0x7fcaf0154710 2023-11-06T20:44:20.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.556+0000 7fcaed59a700 1 -- 172.21.15.33:0/174395578 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae400f670 con 0x7fcaf0154710 2023-11-06T20:44:20.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.556+0000 7fcaed59a700 1 -- 172.21.15.33:0/174395578 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae400f9f0 con 0x7fcaf0154710 2023-11-06T20:44:20.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.556+0000 7fcaf4fff700 1 -- 172.21.15.33:0/174395578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf0154710 msgr2=0x7fcaf00c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.556+0000 7fcaf4fff700 1 --2- 172.21.15.33:0/174395578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf0154710 0x7fcaf00c57e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fcae4009760 tx=0x7fcae4009060).stop 2023-11-06T20:44:20.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.556+0000 7fcaf4fff700 1 -- 172.21.15.33:0/174395578 shutdown_connections 2023-11-06T20:44:20.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.556+0000 7fcaf4fff700 1 --2- 172.21.15.33:0/174395578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf0154710 0x7fcaf00c57e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.556+0000 7fcaf4fff700 1 -- 172.21.15.33:0/174395578 >> 172.21.15.33:0/174395578 conn(0x7fcaf00cb910 msgr2=0x7fcaf00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:20.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.556+0000 7fcaf4fff700 1 -- 172.21.15.33:0/174395578 shutdown_connections 2023-11-06T20:44:20.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.556+0000 7fcaf4fff700 1 -- 172.21.15.33:0/174395578 wait complete. 2023-11-06T20:44:20.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.558+0000 7fcaf4fff700 1 Processor -- start 2023-11-06T20:44:20.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.558+0000 7fcaf4fff700 1 -- start start 2023-11-06T20:44:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaf4fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaf00c7ff0 0x7fcaf0158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaf4fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf0154330 0x7fcaf0154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaf4fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf0154710 0x7fcaf00c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaf4fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0168050 con 0x7fcaf0154710 2023-11-06T20:44:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaf4fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0157100 con 0x7fcaf0154330 2023-11-06T20:44:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaf4fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf01573e0 con 0x7fcaf00c7ff0 2023-11-06T20:44:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaeed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf0154710 0x7fcaf00c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaeed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf0154710 0x7fcaf00c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33634/0 (socket says 172.21.15.33:33634) 2023-11-06T20:44:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaeed9d700 1 -- 172.21.15.33:0/1823679951 learned_addr learned my addr 172.21.15.33:0/1823679951 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaedd9b700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf0154330 0x7fcaf0154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.559+0000 7fcaee59c700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaf00c7ff0 0x7fcaf0158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcaeed9d700 1 -- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaf00c7ff0 msgr2=0x7fcaf0158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcaeed9d700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaf00c7ff0 0x7fcaf0158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcaeed9d700 1 -- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf0154330 msgr2=0x7fcaf0154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcaeed9d700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf0154330 0x7fcaf0154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcaeed9d700 1 -- 172.21.15.33:0/1823679951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae40092a0 con 0x7fcaf0154710 2023-11-06T20:44:20.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcaeed9d700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf0154710 0x7fcaf00c9130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fcae800c7c0 tx=0x7fcae8001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:20.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcade7fc700 1 -- 172.21.15.33:0/1823679951 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae800e4a0 con 0x7fcaf0154710 2023-11-06T20:44:20.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcade7fc700 1 -- 172.21.15.33:0/1823679951 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae800e600 con 0x7fcaf0154710 2023-11-06T20:44:20.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcaf4fff700 1 -- 172.21.15.33:0/1823679951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf01576c0 con 0x7fcaf0154710 2023-11-06T20:44:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcade7fc700 1 -- 172.21.15.33:0/1823679951 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae800e980 con 0x7fcaf0154710 2023-11-06T20:44:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.560+0000 7fcaf4fff700 1 -- 172.21.15.33:0/1823679951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf01499c0 con 0x7fcaf0154710 2023-11-06T20:44:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.561+0000 7fcaf4fff700 1 -- 172.21.15.33:0/1823679951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf0149bb0 con 0x7fcaf0154710 2023-11-06T20:44:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.562+0000 7fcade7fc700 1 -- 172.21.15.33:0/1823679951 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcae80123f0 con 0x7fcaf0154710 2023-11-06T20:44:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.563+0000 7fcade7fc700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcad005f150 0x7fcad0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.563+0000 7fcade7fc700 1 -- 172.21.15.33:0/1823679951 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcae8079a50 con 0x7fcaf0154710 2023-11-06T20:44:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.563+0000 7fcaee59c700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcad005f150 0x7fcad0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.564+0000 7fcaee59c700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcad005f150 0x7fcad0061610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fcae0007fd0 tx=0x7fcae0009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.567+0000 7fcade7fc700 1 -- 172.21.15.33:0/1823679951 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcae8047af0 con 0x7fcaf0154710 2023-11-06T20:44:20.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.746+0000 7fcaf4fff700 1 -- 172.21.15.33:0/1823679951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcaf00652d0 con 0x7fcaf0154710 2023-11-06T20:44:20.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.746+0000 7fcade7fc700 1 -- 172.21.15.33:0/1823679951 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcae804ac00 con 0x7fcaf0154710 2023-11-06T20:44:20.746 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:20.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 -- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcad005f150 msgr2=0x7fcad0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcad005f150 0x7fcad0061610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fcae0007fd0 tx=0x7fcae0009040).stop 2023-11-06T20:44:20.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 -- 172.21.15.33:0/1823679951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf0154710 msgr2=0x7fcaf00c9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf0154710 0x7fcaf00c9130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fcae800c7c0 tx=0x7fcae8001510).stop 2023-11-06T20:44:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 -- 172.21.15.33:0/1823679951 shutdown_connections 2023-11-06T20:44:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcad005f150 0x7fcad0061610 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaf00c7ff0 0x7fcaf0158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf0154710 0x7fcaf00c9130 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 --2- 172.21.15.33:0/1823679951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaf0154330 0x7fcaf0154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 -- 172.21.15.33:0/1823679951 >> 172.21.15.33:0/1823679951 conn(0x7fcaf00cb910 msgr2=0x7fcaf00c0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.748+0000 7fcaf4fff700 1 -- 172.21.15.33:0/1823679951 shutdown_connections 2023-11-06T20:44:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.749+0000 7fcaf4fff700 1 -- 172.21.15.33:0/1823679951 wait complete. 2023-11-06T20:44:20.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T20:44:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.966+0000 7fb0ee5ff700 1 Processor -- start 2023-11-06T20:44:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.966+0000 7fb0ee5ff700 1 -- start start 2023-11-06T20:44:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ee5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb0e8154710 0x7fb0e80c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ee5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb0e80c7ff0 0x7fb0e80646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ee5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e80c6b60 con 0x7fb0e8154330 2023-11-06T20:44:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ee5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e80c7700 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ee5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e80c5cd0 con 0x7fb0e8154710 2023-11-06T20:44:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ecb9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb0e80c7ff0 0x7fb0e80646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ecb9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb0e80c7ff0 0x7fb0e80646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33648/0 (socket says 172.21.15.33:33648) 2023-11-06T20:44:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ecb9c700 1 -- 172.21.15.33:0/2992205733 learned_addr learned my addr 172.21.15.33:0/2992205733 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0e7fff700 1 --1- 172.21.15.33:0/2992205733 >> v1:172.21.15.33:6789/0 conn(0x7fb0e8154330 0x7fb0e80c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51670/0 (socket says 172.21.15.33:51670) 2023-11-06T20:44:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0e77fe700 1 --2- 172.21.15.33:0/2992205733 >> v2:172.21.15.69:3300/0 conn(0x7fb0e8154710 0x7fb0e80c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ecb9c700 1 -- 172.21.15.33:0/2992205733 >> v2:172.21.15.69:3300/0 conn(0x7fb0e8154710 msgr2=0x7fb0e80c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ecb9c700 1 --2- 172.21.15.33:0/2992205733 >> v2:172.21.15.69:3300/0 conn(0x7fb0e8154710 0x7fb0e80c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0ecb9c700 1 -- 172.21.15.33:0/2992205733 >> v1:172.21.15.33:6789/0 conn(0x7fb0e8154330 legacy=0x7fb0e80c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.968+0000 7fb0e6ffd700 1 -- 172.21.15.33:0/2992205733 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662968073 0 0) 0x7fb0e80c6b60 con 0x7fb0e8154330 2023-11-06T20:44:20.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.969+0000 7fb0ecb9c700 1 -- 172.21.15.33:0/2992205733 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e80c5f50 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.969+0000 7fb0ecb9c700 1 --2- 172.21.15.33:0/2992205733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 0x7fb0e80646c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fb0e80c8d10 tx=0x7fb0e0009840).ready entity=mon.0 client_cookie=ce14879f1358f4d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:20.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.969+0000 7fb0e6ffd700 1 -- 172.21.15.33:0/2992205733 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000e070 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.969+0000 7fb0e6ffd700 1 -- 172.21.15.33:0/2992205733 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e000ba20 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.969+0000 7fb0e6ffd700 1 -- 172.21.15.33:0/2992205733 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000bda0 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.969+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/2992205733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 msgr2=0x7fb0e80646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.969+0000 7fb0ee5ff700 1 --2- 172.21.15.33:0/2992205733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 0x7fb0e80646c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fb0e80c8d10 tx=0x7fb0e0009840).stop 2023-11-06T20:44:20.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.970+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/2992205733 shutdown_connections 2023-11-06T20:44:20.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.970+0000 7fb0ee5ff700 1 --2- 172.21.15.33:0/2992205733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 0x7fb0e80646c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.970+0000 7fb0ee5ff700 1 --2- 172.21.15.33:0/2992205733 >> v2:172.21.15.69:3300/0 conn(0x7fb0e8154710 0x7fb0e80c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.970+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/2992205733 >> 172.21.15.33:0/2992205733 conn(0x7fb0e80cb910 msgr2=0x7fb0e80c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:20.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.970+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/2992205733 shutdown_connections 2023-11-06T20:44:20.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.970+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/2992205733 wait complete. 2023-11-06T20:44:20.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.972+0000 7fb0ee5ff700 1 Processor -- start 2023-11-06T20:44:20.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.972+0000 7fb0ee5ff700 1 -- start start 2023-11-06T20:44:20.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.973+0000 7fb0ee5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 0x7fb0e80b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.973+0000 7fb0ee5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0e8154330 0x7fb0e8165090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.973+0000 7fb0ee5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0e8154710 0x7fb0e81533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.973+0000 7fb0ee5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80b0a40 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.973+0000 7fb0ee5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8165580 con 0x7fb0e8154330 2023-11-06T20:44:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.973+0000 7fb0ee5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e81554d0 con 0x7fb0e8154710 2023-11-06T20:44:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.973+0000 7fb0e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 0x7fb0e80b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 0x7fb0e80b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33654/0 (socket says 172.21.15.33:33654) 2023-11-06T20:44:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0e7fff700 1 -- 172.21.15.33:0/3582117211 learned_addr learned my addr 172.21.15.33:0/3582117211 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:20.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0e77fe700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0e8154330 0x7fb0e8165090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0ecb9c700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0e8154710 0x7fb0e81533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0e7fff700 1 -- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0e8154710 msgr2=0x7fb0e81533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0e7fff700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0e8154710 0x7fb0e81533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0e7fff700 1 -- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0e8154330 msgr2=0x7fb0e8165090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:20.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0e7fff700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0e8154330 0x7fb0e8165090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:20.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0e7fff700 1 -- 172.21.15.33:0/3582117211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e0009ae0 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0e7fff700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 0x7fb0e80b0550 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fb0d800c590 tx=0x7fb0d8007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:20.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0cffff700 1 -- 172.21.15.33:0/3582117211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800cea0 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0cffff700 1 -- 172.21.15.33:0/3582117211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80072e0 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.974+0000 7fb0cffff700 1 -- 172.21.15.33:0/3582117211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80056e0 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.975+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/3582117211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e81538c0 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.975+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/3582117211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e8154ba0 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.976+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/3582117211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e80c8d40 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.976+0000 7fb0cffff700 1 -- 172.21.15.33:0/3582117211 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb0d8005ad0 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.977+0000 7fb0cffff700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0d405f150 0x7fb0d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.977+0000 7fb0cffff700 1 -- 172.21.15.33:0/3582117211 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb0d8079980 con 0x7fb0e80c7ff0 2023-11-06T20:44:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.978+0000 7fb0e77fe700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0d405f150 0x7fb0d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:20.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.978+0000 7fb0e77fe700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0d405f150 0x7fb0d4061610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fb0e80a2510 tx=0x7fb0dc009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:20.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:20.981+0000 7fb0cffff700 1 -- 172.21.15.33:0/3582117211 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb0d804ab90 con 0x7fb0e80c7ff0 2023-11-06T20:44:21.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.162+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/3582117211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0e80830b0 con 0x7fb0e80c7ff0 2023-11-06T20:44:21.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.163+0000 7fb0cffff700 1 -- 172.21.15.33:0/3582117211 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0d804e1b0 con 0x7fb0e80c7ff0 2023-11-06T20:44:21.162 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:21.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0d405f150 msgr2=0x7fb0d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0d405f150 0x7fb0d4061610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fb0e80a2510 tx=0x7fb0dc009060).stop 2023-11-06T20:44:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/3582117211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 msgr2=0x7fb0e80b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 0x7fb0e80b0550 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fb0d800c590 tx=0x7fb0d8007c90).stop 2023-11-06T20:44:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/3582117211 shutdown_connections 2023-11-06T20:44:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0d405f150 0x7fb0d4061610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0e8154710 0x7fb0e81533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0e80c7ff0 0x7fb0e80b0550 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 --2- 172.21.15.33:0/3582117211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0e8154330 0x7fb0e8165090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/3582117211 >> 172.21.15.33:0/3582117211 conn(0x7fb0e80cb910 msgr2=0x7fb0e80ac050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/3582117211 shutdown_connections 2023-11-06T20:44:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.165+0000 7fb0ee5ff700 1 -- 172.21.15.33:0/3582117211 wait complete. 2023-11-06T20:44:21.181 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:44:21.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.378+0000 7f4e833ff700 1 Processor -- start 2023-11-06T20:44:21.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.378+0000 7f4e833ff700 1 -- start start 2023-11-06T20:44:21.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.379+0000 7f4e833ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e7c154590 0x7f4e7c066460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.379+0000 7f4e833ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7c1554e0 con 0x7f4e7c1541b0 2023-11-06T20:44:21.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.379+0000 7f4e833ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7c155970 con 0x7f4e7c095e80 2023-11-06T20:44:21.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.380+0000 7f4e833ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7c154bb0 con 0x7f4e7c154590 2023-11-06T20:44:21.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.380+0000 7f4e8099a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e7c154590 0x7f4e7c066460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.380+0000 7f4e8119b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4e7c1541b0 0x7f4e7c0a7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51682/0 (socket says 172.21.15.33:51682) 2023-11-06T20:44:21.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.380+0000 7f4e8119b700 1 -- 172.21.15.33:0/659628117 learned_addr learned my addr 172.21.15.33:0/659628117 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:21.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.380+0000 7f4e73fff700 1 -- 172.21.15.33:0/659628117 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335209333 0 0) 0x7f4e7c1554e0 con 0x7f4e7c1541b0 2023-11-06T20:44:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.380+0000 7f4e73fff700 1 -- 172.21.15.33:0/659628117 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e680033f0 con 0x7f4e7c1541b0 2023-11-06T20:44:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.380+0000 7f4e8099a700 1 -- 172.21.15.33:0/659628117 >> v1:172.21.15.69:6789/0 conn(0x7f4e7c095e80 legacy=0x7f4e7c0be580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.380+0000 7f4e8099a700 1 -- 172.21.15.33:0/659628117 >> v1:172.21.15.33:6789/0 conn(0x7f4e7c1541b0 legacy=0x7f4e7c0a7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.380+0000 7f4e8099a700 1 -- 172.21.15.33:0/659628117 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c154e90 con 0x7f4e7c154590 2023-11-06T20:44:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.380+0000 7f4e8099a700 1 --2- 172.21.15.33:0/659628117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 0x7f4e7c066460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f4e6c009760 tx=0x7f4e6c009060).ready entity=mon.0 client_cookie=682873b34ee7bdd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.381+0000 7f4e73fff700 1 -- 172.21.15.33:0/659628117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00f510 con 0x7f4e7c154590 2023-11-06T20:44:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.381+0000 7f4e73fff700 1 -- 172.21.15.33:0/659628117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c00f670 con 0x7f4e7c154590 2023-11-06T20:44:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.381+0000 7f4e73fff700 1 -- 172.21.15.33:0/659628117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00f9f0 con 0x7f4e7c154590 2023-11-06T20:44:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.381+0000 7f4e833ff700 1 -- 172.21.15.33:0/659628117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 msgr2=0x7f4e7c066460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.381+0000 7f4e833ff700 1 --2- 172.21.15.33:0/659628117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 0x7f4e7c066460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f4e6c009760 tx=0x7f4e6c009060).stop 2023-11-06T20:44:21.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.381+0000 7f4e833ff700 1 -- 172.21.15.33:0/659628117 shutdown_connections 2023-11-06T20:44:21.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.381+0000 7f4e833ff700 1 --2- 172.21.15.33:0/659628117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 0x7f4e7c066460 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.381+0000 7f4e833ff700 1 -- 172.21.15.33:0/659628117 >> 172.21.15.33:0/659628117 conn(0x7f4e7c05ec30 msgr2=0x7f4e7c0c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:21.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.381+0000 7f4e833ff700 1 -- 172.21.15.33:0/659628117 shutdown_connections 2023-11-06T20:44:21.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.381+0000 7f4e833ff700 1 -- 172.21.15.33:0/659628117 wait complete. 2023-11-06T20:44:21.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.383+0000 7f4e833ff700 1 Processor -- start 2023-11-06T20:44:21.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.383+0000 7f4e833ff700 1 -- start start 2023-11-06T20:44:21.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.384+0000 7f4e833ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e7c095e80 0x7f4e7c0acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.384+0000 7f4e833ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e7c1541b0 0x7f4e7c064ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.384+0000 7f4e833ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 0x7f4e7c0c3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.384+0000 7f4e833ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c0ad5f0 con 0x7f4e7c154590 2023-11-06T20:44:21.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.384+0000 7f4e833ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c0651d0 con 0x7f4e7c095e80 2023-11-06T20:44:21.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.384+0000 7f4e833ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c14d210 con 0x7f4e7c1541b0 2023-11-06T20:44:21.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.384+0000 7f4e8199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 0x7f4e7c0c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.384+0000 7f4e8199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 0x7f4e7c0c3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33658/0 (socket says 172.21.15.33:33658) 2023-11-06T20:44:21.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.384+0000 7f4e8199c700 1 -- 172.21.15.33:0/636113799 learned_addr learned my addr 172.21.15.33:0/636113799 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:21.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e8099a700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e7c1541b0 0x7f4e7c064ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e8119b700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e7c095e80 0x7f4e7c0acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e8199c700 1 -- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e7c1541b0 msgr2=0x7f4e7c064ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e8199c700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e7c1541b0 0x7f4e7c064ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e8199c700 1 -- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e7c095e80 msgr2=0x7f4e7c0acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e8199c700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e7c095e80 0x7f4e7c0acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e8199c700 1 -- 172.21.15.33:0/636113799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0092a0 con 0x7f4e7c154590 2023-11-06T20:44:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e8119b700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e7c095e80 0x7f4e7c0acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e8199c700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 0x7f4e7c0c3300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f4e7400c7c0 tx=0x7f4e74001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e717fa700 1 -- 172.21.15.33:0/636113799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400e4a0 con 0x7f4e7c154590 2023-11-06T20:44:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e717fa700 1 -- 172.21.15.33:0/636113799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7400e600 con 0x7f4e7c154590 2023-11-06T20:44:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e833ff700 1 -- 172.21.15.33:0/636113799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7c0c56e0 con 0x7f4e7c154590 2023-11-06T20:44:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e717fa700 1 -- 172.21.15.33:0/636113799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400e980 con 0x7f4e7c154590 2023-11-06T20:44:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.385+0000 7f4e833ff700 1 -- 172.21.15.33:0/636113799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7c0c5090 con 0x7f4e7c154590 2023-11-06T20:44:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.387+0000 7f4e833ff700 1 -- 172.21.15.33:0/636113799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7c0c4f00 con 0x7f4e7c154590 2023-11-06T20:44:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.387+0000 7f4e717fa700 1 -- 172.21.15.33:0/636113799 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4e7400ed30 con 0x7f4e7c154590 2023-11-06T20:44:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.389+0000 7f4e717fa700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e6005f180 0x7f4e60061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.389+0000 7f4e717fa700 1 -- 172.21.15.33:0/636113799 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4e74079800 con 0x7f4e7c154590 2023-11-06T20:44:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.390+0000 7f4e8119b700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e6005f180 0x7f4e60061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.390+0000 7f4e8119b700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e6005f180 0x7f4e60061640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f4e78001360 tx=0x7f4e78009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.393+0000 7f4e717fa700 1 -- 172.21.15.33:0/636113799 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4e74048af0 con 0x7f4e7c154590 2023-11-06T20:44:21.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.617+0000 7f4e833ff700 1 -- 172.21.15.33:0/636113799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e7c0c58a0 con 0x7f4e7c154590 2023-11-06T20:44:21.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.617+0000 7f4e717fa700 1 -- 172.21.15.33:0/636113799 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4480 (secure 0 0 0) 0x7f4e74017020 con 0x7f4e7c154590 2023-11-06T20:44:21.617 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4123875619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123875619},{"type":"v1","addr":"172.21.15.33:6834","nonce":4123875619}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3506554570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3506554570},{"type":"v1","addr":"172.21.15.33:6835","nonce":3506554570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/239985290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":239985290},{"type":"v1","addr":"172.21.15.69:6835","nonce":239985290}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:44:06.383743+0000","modified":"2023-11-06T20:44:09.389408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/2681789033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2681789033},{"type":"v1","addr":"172.21.15.69:6837","nonce":2681789033}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-06T20:44:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.619+0000 7f4e833ff700 1 -- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e6005f180 msgr2=0x7f4e60061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.619+0000 7f4e833ff700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e6005f180 0x7f4e60061640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f4e78001360 tx=0x7f4e78009040).stop 2023-11-06T20:44:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.619+0000 7f4e833ff700 1 -- 172.21.15.33:0/636113799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 msgr2=0x7f4e7c0c3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.620+0000 7f4e833ff700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 0x7f4e7c0c3300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f4e7400c7c0 tx=0x7f4e74001510).stop 2023-11-06T20:44:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.620+0000 7f4e833ff700 1 -- 172.21.15.33:0/636113799 shutdown_connections 2023-11-06T20:44:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.620+0000 7f4e833ff700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e6005f180 0x7f4e60061640 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.620+0000 7f4e833ff700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e7c154590 0x7f4e7c0c3300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.620+0000 7f4e833ff700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e7c095e80 0x7f4e7c0acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.620+0000 7f4e833ff700 1 --2- 172.21.15.33:0/636113799 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e7c1541b0 0x7f4e7c064ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.620+0000 7f4e833ff700 1 -- 172.21.15.33:0/636113799 >> 172.21.15.33:0/636113799 conn(0x7f4e7c05ec30 msgr2=0x7f4e7c0a7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:21.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.620+0000 7f4e833ff700 1 -- 172.21.15.33:0/636113799 shutdown_connections 2023-11-06T20:44:21.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.620+0000 7f4e833ff700 1 -- 172.21.15.33:0/636113799 wait complete. 2023-11-06T20:44:21.622 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2023-11-06T20:44:21.636 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:21.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.833+0000 7fb1019ff700 1 Processor -- start 2023-11-06T20:44:21.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.833+0000 7fb1019ff700 1 -- start start 2023-11-06T20:44:21.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.834+0000 7fb1019ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0c5d50 0x7fb0fc0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.834+0000 7fb1019ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb0fc0bfd70 0x7fb0fc0c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.834+0000 7fb1019ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0fc1554a0 con 0x7fb0fc0c1840 2023-11-06T20:44:21.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.834+0000 7fb1019ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0fc154960 con 0x7fb0fc0bfd70 2023-11-06T20:44:21.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.834+0000 7fb1019ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0c66c0 con 0x7fb0fc0c5d50 2023-11-06T20:44:21.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0fb7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb0fc0bfd70 0x7fb0fc0c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0fa7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0c5d50 0x7fb0fc0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0fb7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb0fc0bfd70 0x7fb0fc0c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:35936/0 (socket says 172.21.15.33:35936) 2023-11-06T20:44:21.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0fb7fe700 1 -- 172.21.15.33:0/3189613137 learned_addr learned my addr 172.21.15.33:0/3189613137 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:21.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0fa7fc700 1 --2- 172.21.15.33:0/3189613137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0c5d50 0x7fb0fc0c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:47712/0 (socket says 172.21.15.33:47712) 2023-11-06T20:44:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0fb7fe700 1 -- 172.21.15.33:0/3189613137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0c5d50 msgr2=0x7fb0fc0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0fb7fe700 1 --2- 172.21.15.33:0/3189613137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0c5d50 0x7fb0fc0c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0fb7fe700 1 -- 172.21.15.33:0/3189613137 >> v1:172.21.15.69:6789/0 conn(0x7fb0fc0c1840 legacy=0x7fb0fc0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0fb7fe700 1 -- 172.21.15.33:0/3189613137 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc155930 con 0x7fb0fc0bfd70 2023-11-06T20:44:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0f9ffb700 1 -- 172.21.15.33:0/3189613137 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003990983 0 0) 0x7fb0fc1554a0 con 0x7fb0fc0c1840 2023-11-06T20:44:21.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0fb7fe700 1 --2- 172.21.15.33:0/3189613137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc0bfd70 0x7fb0fc0c01e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fb0f4009f20 tx=0x7fb0f4009840).ready entity=mon.1 client_cookie=fab671f7d512acfe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:21.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0f9ffb700 1 -- 172.21.15.33:0/3189613137 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f400e070 con 0x7fb0fc0bfd70 2023-11-06T20:44:21.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0f9ffb700 1 -- 172.21.15.33:0/3189613137 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f400b9d0 con 0x7fb0fc0bfd70 2023-11-06T20:44:21.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.835+0000 7fb0f9ffb700 1 -- 172.21.15.33:0/3189613137 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f400bd50 con 0x7fb0fc0bfd70 2023-11-06T20:44:21.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.836+0000 7fb1019ff700 1 -- 172.21.15.33:0/3189613137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc0bfd70 msgr2=0x7fb0fc0c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.836+0000 7fb1019ff700 1 --2- 172.21.15.33:0/3189613137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc0bfd70 0x7fb0fc0c01e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fb0f4009f20 tx=0x7fb0f4009840).stop 2023-11-06T20:44:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.836+0000 7fb1019ff700 1 -- 172.21.15.33:0/3189613137 shutdown_connections 2023-11-06T20:44:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.836+0000 7fb1019ff700 1 --2- 172.21.15.33:0/3189613137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc0bfd70 0x7fb0fc0c01e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.836+0000 7fb1019ff700 1 --2- 172.21.15.33:0/3189613137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0c5d50 0x7fb0fc0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.836+0000 7fb1019ff700 1 -- 172.21.15.33:0/3189613137 >> 172.21.15.33:0/3189613137 conn(0x7fb0fc05ec80 msgr2=0x7fb0fc0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.836+0000 7fb1019ff700 1 -- 172.21.15.33:0/3189613137 shutdown_connections 2023-11-06T20:44:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.836+0000 7fb1019ff700 1 -- 172.21.15.33:0/3189613137 wait complete. 2023-11-06T20:44:21.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.838+0000 7fb1019ff700 1 Processor -- start 2023-11-06T20:44:21.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.838+0000 7fb1019ff700 1 -- start start 2023-11-06T20:44:21.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb1019ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0bfd70 0x7fb0fc14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb1019ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0c1840 0x7fb0fc0addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb1019ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc0c5d50 0x7fb0fc150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb1019ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0c3db0 con 0x7fb0fc0c1840 2023-11-06T20:44:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb1019ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0c31a0 con 0x7fb0fc0c5d50 2023-11-06T20:44:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb1019ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0c3480 con 0x7fb0fc0bfd70 2023-11-06T20:44:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb0fa7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0c1840 0x7fb0fc0addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb0fa7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0c1840 0x7fb0fc0addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33672/0 (socket says 172.21.15.33:33672) 2023-11-06T20:44:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb0fa7fc700 1 -- 172.21.15.33:0/3182401440 learned_addr learned my addr 172.21.15.33:0/3182401440 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb0faffd700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0bfd70 0x7fb0fc14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb0fb7fe700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc0c5d50 0x7fb0fc150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb0fa7fc700 1 -- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0bfd70 msgr2=0x7fb0fc14f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb0fa7fc700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0bfd70 0x7fb0fc14f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb0fa7fc700 1 -- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc0c5d50 msgr2=0x7fb0fc150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb0fa7fc700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc0c5d50 0x7fb0fc150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.839+0000 7fb0fa7fc700 1 -- 172.21.15.33:0/3182401440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f4009ae0 con 0x7fb0fc0c1840 2023-11-06T20:44:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.840+0000 7fb0fa7fc700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0c1840 0x7fb0fc0addc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fb0f000e930 tx=0x7fb0f0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.840+0000 7fb0e2ffd700 1 -- 172.21.15.33:0/3182401440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0012600 con 0x7fb0fc0c1840 2023-11-06T20:44:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.840+0000 7fb0e2ffd700 1 -- 172.21.15.33:0/3182401440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f0012760 con 0x7fb0fc0c1840 2023-11-06T20:44:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.840+0000 7fb0e2ffd700 1 -- 172.21.15.33:0/3182401440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0012ae0 con 0x7fb0fc0c1840 2023-11-06T20:44:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.840+0000 7fb1019ff700 1 -- 172.21.15.33:0/3182401440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc0c3760 con 0x7fb0fc0c1840 2023-11-06T20:44:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.840+0000 7fb1019ff700 1 -- 172.21.15.33:0/3182401440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc0ae2b0 con 0x7fb0fc0c1840 2023-11-06T20:44:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.841+0000 7fb1019ff700 1 -- 172.21.15.33:0/3182401440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc0c3f20 con 0x7fb0fc0c1840 2023-11-06T20:44:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.842+0000 7fb0e2ffd700 1 -- 172.21.15.33:0/3182401440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb0f001b400 con 0x7fb0fc0c1840 2023-11-06T20:44:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.843+0000 7fb0e2ffd700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0e805f150 0x7fb0e8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.843+0000 7fb0e2ffd700 1 -- 172.21.15.33:0/3182401440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb0f007a700 con 0x7fb0fc0c1840 2023-11-06T20:44:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.843+0000 7fb0faffd700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0e805f150 0x7fb0e8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.844+0000 7fb0faffd700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0e805f150 0x7fb0e8061610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fb0ec007fd0 tx=0x7fb0ec007bb0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:21.848+0000 7fb0e2ffd700 1 -- 172.21.15.33:0/3182401440 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb0f004b8b0 con 0x7fb0fc0c1840 2023-11-06T20:44:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.037+0000 7fb1019ff700 1 -- 172.21.15.33:0/3182401440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0fc0aa410 con 0x7fb0fc0c1840 2023-11-06T20:44:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.037+0000 7fb0e2ffd700 1 -- 172.21.15.33:0/3182401440 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fb0f004eed0 con 0x7fb0fc0c1840 2023-11-06T20:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:22.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 -- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0e805f150 msgr2=0x7fb0e8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0e805f150 0x7fb0e8061610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fb0ec007fd0 tx=0x7fb0ec007bb0).stop 2023-11-06T20:44:22.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 -- 172.21.15.33:0/3182401440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0c1840 msgr2=0x7fb0fc0addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0c1840 0x7fb0fc0addc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fb0f000e930 tx=0x7fb0f0009060).stop 2023-11-06T20:44:22.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 -- 172.21.15.33:0/3182401440 shutdown_connections 2023-11-06T20:44:22.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb0e805f150 0x7fb0e8061610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0fc0bfd70 0x7fb0fc14f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0fc0c5d50 0x7fb0fc150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 --2- 172.21.15.33:0/3182401440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0c1840 0x7fb0fc0addc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 -- 172.21.15.33:0/3182401440 >> 172.21.15.33:0/3182401440 conn(0x7fb0fc05ec80 msgr2=0x7fb0fc153880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:22.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 -- 172.21.15.33:0/3182401440 shutdown_connections 2023-11-06T20:44:22.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.040+0000 7fb1019ff700 1 -- 172.21.15.33:0/3182401440 wait complete. 2023-11-06T20:44:22.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:44:04.716205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:44:05.351480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T20:44:22.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:44:22.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.255+0000 7f3f2b7ff700 1 Processor -- start 2023-11-06T20:44:22.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.256+0000 7f3f2b7ff700 1 -- start start 2023-11-06T20:44:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.257+0000 7f3f2b7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f24154710 0x7f3f240c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.257+0000 7f3f2b7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f240c6b60 con 0x7f3f24154330 2023-11-06T20:44:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.257+0000 7f3f2b7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f240c7700 con 0x7f3f240c7ff0 2023-11-06T20:44:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.257+0000 7f3f2b7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f240c5cd0 con 0x7f3f24154710 2023-11-06T20:44:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.257+0000 7f3f2959b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3f24154330 0x7f3f240c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51698/0 (socket says 172.21.15.33:51698) 2023-11-06T20:44:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.257+0000 7f3f2959b700 1 -- 172.21.15.33:0/3274702796 learned_addr learned my addr 172.21.15.33:0/3274702796 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.257+0000 7f3f28d9a700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f24154710 0x7f3f240c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855426494 0 0) 0x7f3f240c6b60 con 0x7f3f24154330 2023-11-06T20:44:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f100033f0 con 0x7f3f24154330 2023-11-06T20:44:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007596806 0 0) 0x7f3f240c7700 con 0x7f3f240c7ff0 2023-11-06T20:44:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f240c6b60 con 0x7f3f240c7ff0 2023-11-06T20:44:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1312195949 0 0) 0x7f3f100033f0 con 0x7f3f24154330 2023-11-06T20:44:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f24154710 msgr2=0x7f3f240c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:22.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f24154710 0x7f3f240c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 >> v1:172.21.15.69:6789/0 conn(0x7f3f240c7ff0 legacy=0x7f3f240646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240c5f50 con 0x7f3f24154330 2023-11-06T20:44:22.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f3f20001470 con 0x7f3f24154330 2023-11-06T20:44:22.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 >> v1:172.21.15.33:6789/0 conn(0x7f3f24154330 legacy=0x7f3f240c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f24154710 0x7f3f10003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f10003fb0 0x7f3f10004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f10004b80 0x7f3f10005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f240c5cd0 con 0x7f3f10003fb0 2023-11-06T20:44:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1000b990 con 0x7f3f10004b80 2023-11-06T20:44:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1000bb50 con 0x7f3f24154710 2023-11-06T20:44:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.258+0000 7f3f28d9a700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f10003fb0 0x7f3f10004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f2959b700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f24154710 0x7f3f10003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f29d9c700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f10004b80 0x7f3f10005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f28d9a700 1 -- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f24154710 msgr2=0x7f3f10003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f28d9a700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f24154710 0x7f3f10003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f28d9a700 1 -- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f10004b80 msgr2=0x7f3f10005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f28d9a700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f10004b80 0x7f3f10005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f28d9a700 1 -- 172.21.15.33:0/3274702796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f100033f0 con 0x7f3f10003fb0 2023-11-06T20:44:22.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f28d9a700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f10003fb0 0x7f3f10004690 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f3f24154b70 tx=0x7f3f14009060).ready entity=mon.0 client_cookie=5c846951802c8b8e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:22.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14009e90 con 0x7f3f10003fb0 2023-11-06T20:44:22.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14009430 con 0x7f3f10003fb0 2023-11-06T20:44:22.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.259+0000 7f3f1bfff700 1 -- 172.21.15.33:0/3274702796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400e850 con 0x7f3f10003fb0 2023-11-06T20:44:22.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.260+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/3274702796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f10003fb0 msgr2=0x7f3f10004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.260+0000 7f3f2b7ff700 1 --2- 172.21.15.33:0/3274702796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f10003fb0 0x7f3f10004690 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f3f24154b70 tx=0x7f3f14009060).stop 2023-11-06T20:44:22.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.260+0000 7f3f29d9c700 1 -- 172.21.15.33:0/3274702796 reap_dead start 2023-11-06T20:44:22.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.260+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/3274702796 shutdown_connections 2023-11-06T20:44:22.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.260+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/3274702796 >> 172.21.15.33:0/3274702796 conn(0x7f3f240cb910 msgr2=0x7f3f240c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:22.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.260+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/3274702796 shutdown_connections 2023-11-06T20:44:22.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.260+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/3274702796 wait complete. 2023-11-06T20:44:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.261+0000 7f3f2b7ff700 1 Processor -- start 2023-11-06T20:44:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.262+0000 7f3f2b7ff700 1 -- start start 2023-11-06T20:44:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f2b7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f10003fb0 0x7f3f241563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f2b7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f241568c0 0x7f3f240c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f2b7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24156d40 0x7f3f240c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f2b7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f240addd0 con 0x7f3f241568c0 2023-11-06T20:44:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f2b7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f24157260 con 0x7f3f24156d40 2023-11-06T20:44:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f2b7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f24157540 con 0x7f3f10003fb0 2023-11-06T20:44:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f28d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f241568c0 0x7f3f240c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f28d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f241568c0 0x7f3f240c2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33698/0 (socket says 172.21.15.33:33698) 2023-11-06T20:44:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f2959b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f10003fb0 0x7f3f241563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f28d9a700 1 -- 172.21.15.33:0/1617797191 learned_addr learned my addr 172.21.15.33:0/1617797191 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f29d9c700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24156d40 0x7f3f240c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f28d9a700 1 -- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f10003fb0 msgr2=0x7f3f241563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f28d9a700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f10003fb0 0x7f3f241563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f28d9a700 1 -- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24156d40 msgr2=0x7f3f240c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f28d9a700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24156d40 0x7f3f240c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f28d9a700 1 -- 172.21.15.33:0/1617797191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f24157820 con 0x7f3f241568c0 2023-11-06T20:44:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.263+0000 7f3f29d9c700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24156d40 0x7f3f240c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.264+0000 7f3f28d9a700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f241568c0 0x7f3f240c2eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3f1400e4b0 tx=0x7f3f1401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.264+0000 7f3f197fa700 1 -- 172.21.15.33:0/1617797191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1401d980 con 0x7f3f241568c0 2023-11-06T20:44:22.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.264+0000 7f3f197fa700 1 -- 172.21.15.33:0/1617797191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1401db00 con 0x7f3f241568c0 2023-11-06T20:44:22.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.264+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/1617797191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f24157b00 con 0x7f3f241568c0 2023-11-06T20:44:22.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.264+0000 7f3f197fa700 1 -- 172.21.15.33:0/1617797191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14026440 con 0x7f3f241568c0 2023-11-06T20:44:22.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.264+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/1617797191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f240c3890 con 0x7f3f241568c0 2023-11-06T20:44:22.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.265+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/1617797191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f240830b0 con 0x7f3f241568c0 2023-11-06T20:44:22.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.266+0000 7f3f197fa700 1 -- 172.21.15.33:0/1617797191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3f1402f440 con 0x7f3f241568c0 2023-11-06T20:44:22.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.267+0000 7f3f197fa700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f0805f150 0x7f3f08061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.267+0000 7f3f197fa700 1 -- 172.21.15.33:0/1617797191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3f1407be30 con 0x7f3f241568c0 2023-11-06T20:44:22.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.267+0000 7f3f2959b700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f0805f150 0x7f3f08061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.268+0000 7f3f2959b700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f0805f150 0x7f3f08061610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f3f20007d70 tx=0x7f3f20001e10).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:22.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.271+0000 7f3f197fa700 1 -- 172.21.15.33:0/1617797191 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3f1404d020 con 0x7f3f241568c0 2023-11-06T20:44:22.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.452+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/1617797191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3f240652d0 con 0x7f3f241568c0 2023-11-06T20:44:22.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.453+0000 7f3f197fa700 1 -- 172.21.15.33:0/1617797191 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3f14016020 con 0x7f3f241568c0 2023-11-06T20:44:22.452 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:44:22.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.454+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f0805f150 msgr2=0x7f3f08061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.454+0000 7f3f2b7ff700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f0805f150 0x7f3f08061610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f3f20007d70 tx=0x7f3f20001e10).stop 2023-11-06T20:44:22.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.454+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/1617797191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f241568c0 msgr2=0x7f3f240c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.454+0000 7f3f2b7ff700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f241568c0 0x7f3f240c2eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3f1400e4b0 tx=0x7f3f1401a000).stop 2023-11-06T20:44:22.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.455+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/1617797191 shutdown_connections 2023-11-06T20:44:22.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.455+0000 7f3f2b7ff700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f0805f150 0x7f3f08061610 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.455+0000 7f3f2b7ff700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f10003fb0 0x7f3f241563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.455+0000 7f3f2b7ff700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f24156d40 0x7f3f240c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.455+0000 7f3f2b7ff700 1 --2- 172.21.15.33:0/1617797191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f241568c0 0x7f3f240c2eb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.455+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/1617797191 >> 172.21.15.33:0/1617797191 conn(0x7f3f240cb910 msgr2=0x7f3f240aaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:22.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.455+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/1617797191 shutdown_connections 2023-11-06T20:44:22.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.455+0000 7f3f2b7ff700 1 -- 172.21.15.33:0/1617797191 wait complete. 2023-11-06T20:44:22.471 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:44:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.671+0000 7faf26dff700 1 Processor -- start 2023-11-06T20:44:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.671+0000 7faf26dff700 1 -- start start 2023-11-06T20:44:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.672+0000 7faf26dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7faf20154710 0x7faf200c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.672+0000 7faf26dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200c7ff0 0x7faf200646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.672+0000 7faf26dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf200c6b60 con 0x7faf20154330 2023-11-06T20:44:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.672+0000 7faf26dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf200c7700 con 0x7faf20154710 2023-11-06T20:44:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.672+0000 7faf26dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf200c5cd0 con 0x7faf200c7ff0 2023-11-06T20:44:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf2539c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200c7ff0 0x7faf200646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf1ffff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7faf20154710 0x7faf200c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf2539c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200c7ff0 0x7faf200646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:47748/0 (socket says 172.21.15.33:47748) 2023-11-06T20:44:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf2539c700 1 -- 172.21.15.33:0/783214407 learned_addr learned my addr 172.21.15.33:0/783214407 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf24b9b700 1 --1- 172.21.15.33:0/783214407 >> v1:172.21.15.69:6789/0 conn(0x7faf20154330 0x7faf200c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:38418/0 (socket says 172.21.15.33:38418) 2023-11-06T20:44:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf2539c700 1 -- 172.21.15.33:0/783214407 >> v2:172.21.15.69:3300/0 conn(0x7faf20154710 msgr2=0x7faf200c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf2539c700 1 --2- 172.21.15.33:0/783214407 >> v2:172.21.15.69:3300/0 conn(0x7faf20154710 0x7faf200c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf2539c700 1 -- 172.21.15.33:0/783214407 >> v1:172.21.15.69:6789/0 conn(0x7faf20154330 legacy=0x7faf200c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf2539c700 1 -- 172.21.15.33:0/783214407 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf200c5f50 con 0x7faf200c7ff0 2023-11-06T20:44:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf2539c700 1 --2- 172.21.15.33:0/783214407 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200c7ff0 0x7faf200646c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7faf18009f20 tx=0x7faf18009840).ready entity=mon.2 client_cookie=1b8085423757ea96 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.673+0000 7faf1f7fe700 1 -- 172.21.15.33:0/783214407 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800e070 con 0x7faf200c7ff0 2023-11-06T20:44:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.674+0000 7faf1f7fe700 1 -- 172.21.15.33:0/783214407 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1800b9d0 con 0x7faf200c7ff0 2023-11-06T20:44:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.674+0000 7faf1f7fe700 1 -- 172.21.15.33:0/783214407 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800bd50 con 0x7faf200c7ff0 2023-11-06T20:44:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.674+0000 7faf26dff700 1 -- 172.21.15.33:0/783214407 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200c7ff0 msgr2=0x7faf200646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.674+0000 7faf26dff700 1 --2- 172.21.15.33:0/783214407 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200c7ff0 0x7faf200646c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7faf18009f20 tx=0x7faf18009840).stop 2023-11-06T20:44:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.674+0000 7faf26dff700 1 -- 172.21.15.33:0/783214407 shutdown_connections 2023-11-06T20:44:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.674+0000 7faf26dff700 1 --2- 172.21.15.33:0/783214407 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200c7ff0 0x7faf200646c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.674+0000 7faf26dff700 1 --2- 172.21.15.33:0/783214407 >> v2:172.21.15.69:3300/0 conn(0x7faf20154710 0x7faf200c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.674+0000 7faf26dff700 1 -- 172.21.15.33:0/783214407 >> 172.21.15.33:0/783214407 conn(0x7faf200cb910 msgr2=0x7faf200c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.674+0000 7faf26dff700 1 -- 172.21.15.33:0/783214407 shutdown_connections 2023-11-06T20:44:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.674+0000 7faf26dff700 1 -- 172.21.15.33:0/783214407 wait complete. 2023-11-06T20:44:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.676+0000 7faf26dff700 1 Processor -- start 2023-11-06T20:44:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.676+0000 7faf26dff700 1 -- start start 2023-11-06T20:44:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf26dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c7ff0 0x7faf2014a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf26dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20154330 0x7faf2014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf26dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf20154710 0x7faf201571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf26dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf20157a20 con 0x7faf20154330 2023-11-06T20:44:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf26dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf200c3050 con 0x7faf200c7ff0 2023-11-06T20:44:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf26dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2014aae0 con 0x7faf20154710 2023-11-06T20:44:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf1ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20154330 0x7faf2014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf1ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20154330 0x7faf2014a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33704/0 (socket says 172.21.15.33:33704) 2023-11-06T20:44:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf1ffff700 1 -- 172.21.15.33:0/2976483413 learned_addr learned my addr 172.21.15.33:0/2976483413 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf2539c700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf20154710 0x7faf201571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf24b9b700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c7ff0 0x7faf2014a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf1ffff700 1 -- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf20154710 msgr2=0x7faf201571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf1ffff700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf20154710 0x7faf201571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf1ffff700 1 -- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c7ff0 msgr2=0x7faf2014a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf1ffff700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c7ff0 0x7faf2014a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf1ffff700 1 -- 172.21.15.33:0/2976483413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf18009ae0 con 0x7faf20154330 2023-11-06T20:44:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.677+0000 7faf24b9b700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c7ff0 0x7faf2014a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.678+0000 7faf1ffff700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20154330 0x7faf2014a5f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7faf0c00e8d0 tx=0x7faf0c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.678+0000 7faf1cff9700 1 -- 172.21.15.33:0/2976483413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00c640 con 0x7faf20154330 2023-11-06T20:44:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.678+0000 7faf1cff9700 1 -- 172.21.15.33:0/2976483413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c00c7a0 con 0x7faf20154330 2023-11-06T20:44:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.678+0000 7faf26dff700 1 -- 172.21.15.33:0/2976483413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf200c2720 con 0x7faf20154330 2023-11-06T20:44:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.678+0000 7faf1cff9700 1 -- 172.21.15.33:0/2976483413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00cb20 con 0x7faf20154330 2023-11-06T20:44:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.678+0000 7faf26dff700 1 -- 172.21.15.33:0/2976483413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf200c2ce0 con 0x7faf20154330 2023-11-06T20:44:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.679+0000 7faf26dff700 1 -- 172.21.15.33:0/2976483413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf200aa5d0 con 0x7faf20154330 2023-11-06T20:44:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.680+0000 7faf1cff9700 1 -- 172.21.15.33:0/2976483413 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7faf0c01b400 con 0x7faf20154330 2023-11-06T20:44:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.681+0000 7faf1cff9700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf0405f150 0x7faf04061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.681+0000 7faf1cff9700 1 -- 172.21.15.33:0/2976483413 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7faf0c07a630 con 0x7faf20154330 2023-11-06T20:44:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.681+0000 7faf24b9b700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf0405f150 0x7faf04061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.682+0000 7faf24b9b700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf0405f150 0x7faf04061610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7faf140016b0 tx=0x7faf14009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.685+0000 7faf1cff9700 1 -- 172.21.15.33:0/2976483413 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf0c04b890 con 0x7faf20154330 2023-11-06T20:44:22.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.865+0000 7faf26dff700 1 -- 172.21.15.33:0/2976483413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faf200830b0 con 0x7faf20154330 2023-11-06T20:44:22.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.865+0000 7faf1cff9700 1 -- 172.21.15.33:0/2976483413 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7faf0c04eeb0 con 0x7faf20154330 2023-11-06T20:44:22.865 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:44:22.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.867+0000 7faf26dff700 1 -- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf0405f150 msgr2=0x7faf04061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.867+0000 7faf26dff700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf0405f150 0x7faf04061610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7faf140016b0 tx=0x7faf14009040).stop 2023-11-06T20:44:22.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.867+0000 7faf26dff700 1 -- 172.21.15.33:0/2976483413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20154330 msgr2=0x7faf2014a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:22.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.867+0000 7faf26dff700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20154330 0x7faf2014a5f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7faf0c00e8d0 tx=0x7faf0c009060).stop 2023-11-06T20:44:22.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.867+0000 7faf26dff700 1 -- 172.21.15.33:0/2976483413 shutdown_connections 2023-11-06T20:44:22.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.867+0000 7faf26dff700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf0405f150 0x7faf04061610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.867+0000 7faf26dff700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf20154710 0x7faf201571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.867+0000 7faf26dff700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c7ff0 0x7faf2014a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.867+0000 7faf26dff700 1 --2- 172.21.15.33:0/2976483413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20154330 0x7faf2014a5f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:22.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.867+0000 7faf26dff700 1 -- 172.21.15.33:0/2976483413 >> 172.21.15.33:0/2976483413 conn(0x7faf200cb910 msgr2=0x7faf200646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:22.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.868+0000 7faf26dff700 1 -- 172.21.15.33:0/2976483413 shutdown_connections 2023-11-06T20:44:22.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:22.868+0000 7faf26dff700 1 -- 172.21.15.33:0/2976483413 wait complete. 2023-11-06T20:44:22.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T20:44:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.084+0000 7f6c3c9ff700 1 Processor -- start 2023-11-06T20:44:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.084+0000 7f6c3c9ff700 1 -- start start 2023-11-06T20:44:23.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.085+0000 7f6c3c9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c38154710 0x7f6c380c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.085+0000 7f6c3c9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c380c7ff0 0x7f6c380646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c3c9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c380c6b60 con 0x7f6c38154330 2023-11-06T20:44:23.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c3c9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c380c7700 con 0x7f6c380c7ff0 2023-11-06T20:44:23.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c3c9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380c5cd0 con 0x7f6c38154710 2023-11-06T20:44:23.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c36d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c380c7ff0 0x7f6c380646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c3659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6c38154330 0x7f6c380c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51702/0 (socket says 172.21.15.33:51702) 2023-11-06T20:44:23.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c3659c700 1 -- 172.21.15.33:0/1674792143 learned_addr learned my addr 172.21.15.33:0/1674792143 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:23.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c35d9b700 1 --2- 172.21.15.33:0/1674792143 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c38154710 0x7f6c380c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c36d9d700 1 -- 172.21.15.33:0/1674792143 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c38154710 msgr2=0x7f6c380c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:23.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c36d9d700 1 --2- 172.21.15.33:0/1674792143 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c38154710 0x7f6c380c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c36d9d700 1 -- 172.21.15.33:0/1674792143 >> v1:172.21.15.33:6789/0 conn(0x7f6c38154330 legacy=0x7f6c380c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c36d9d700 1 -- 172.21.15.33:0/1674792143 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c380c5f50 con 0x7f6c380c7ff0 2023-11-06T20:44:23.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c3559a700 1 -- 172.21.15.33:0/1674792143 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036054187 0 0) 0x7f6c380c6b60 con 0x7f6c38154330 2023-11-06T20:44:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.086+0000 7f6c36d9d700 1 --2- 172.21.15.33:0/1674792143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c380c7ff0 0x7f6c380646c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f6c30009e60 tx=0x7f6c30009780).ready entity=mon.0 client_cookie=4d0140b2667960af server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3559a700 1 -- 172.21.15.33:0/1674792143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000e070 con 0x7f6c380c7ff0 2023-11-06T20:44:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3559a700 1 -- 172.21.15.33:0/1674792143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3000bb60 con 0x7f6c380c7ff0 2023-11-06T20:44:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3559a700 1 -- 172.21.15.33:0/1674792143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30012410 con 0x7f6c380c7ff0 2023-11-06T20:44:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/1674792143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c380c7ff0 msgr2=0x7f6c380646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3c9ff700 1 --2- 172.21.15.33:0/1674792143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c380c7ff0 0x7f6c380646c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f6c30009e60 tx=0x7f6c30009780).stop 2023-11-06T20:44:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/1674792143 shutdown_connections 2023-11-06T20:44:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3c9ff700 1 --2- 172.21.15.33:0/1674792143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c380c7ff0 0x7f6c380646c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3c9ff700 1 --2- 172.21.15.33:0/1674792143 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c38154710 0x7f6c380c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/1674792143 >> 172.21.15.33:0/1674792143 conn(0x7f6c380cb910 msgr2=0x7f6c380c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/1674792143 shutdown_connections 2023-11-06T20:44:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.087+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/1674792143 wait complete. 2023-11-06T20:44:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.089+0000 7f6c3c9ff700 1 Processor -- start 2023-11-06T20:44:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.089+0000 7f6c3c9ff700 1 -- start start 2023-11-06T20:44:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c3c9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c380c7ff0 0x7f6c380a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c3c9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c38154330 0x7f6c380addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c3c9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c38154710 0x7f6c380ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c3c9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380b0990 con 0x7f6c38154710 2023-11-06T20:44:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c3c9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c38155320 con 0x7f6c380c7ff0 2023-11-06T20:44:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c3c9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380b06b0 con 0x7f6c38154330 2023-11-06T20:44:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c36d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c38154710 0x7f6c380ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c36d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c38154710 0x7f6c380ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33726/0 (socket says 172.21.15.33:33726) 2023-11-06T20:44:23.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c36d9d700 1 -- 172.21.15.33:0/617601766 learned_addr learned my addr 172.21.15.33:0/617601766 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:23.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c35d9b700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c38154330 0x7f6c380addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.090+0000 7f6c3659c700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c380c7ff0 0x7f6c380a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c36d9d700 1 -- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c38154330 msgr2=0x7f6c380addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c36d9d700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c38154330 0x7f6c380addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c36d9d700 1 -- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c380c7ff0 msgr2=0x7f6c380a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c36d9d700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c380c7ff0 0x7f6c380a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c36d9d700 1 -- 172.21.15.33:0/617601766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30009a20 con 0x7f6c38154710 2023-11-06T20:44:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c36d9d700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c38154710 0x7f6c380ae2c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f6c300094c0 tx=0x7f6c30009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c267fc700 1 -- 172.21.15.33:0/617601766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30012410 con 0x7f6c38154710 2023-11-06T20:44:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c267fc700 1 -- 172.21.15.33:0/617601766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3001d630 con 0x7f6c38154710 2023-11-06T20:44:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c267fc700 1 -- 172.21.15.33:0/617601766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3001d9f0 con 0x7f6c38154710 2023-11-06T20:44:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/617601766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c38154cd0 con 0x7f6c38154710 2023-11-06T20:44:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.091+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/617601766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c380b02d0 con 0x7f6c38154710 2023-11-06T20:44:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.092+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/617601766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c381554b0 con 0x7f6c38154710 2023-11-06T20:44:23.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.093+0000 7f6c267fc700 1 -- 172.21.15.33:0/617601766 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6c30003040 con 0x7f6c38154710 2023-11-06T20:44:23.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.094+0000 7f6c267fc700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c1805f150 0x7f6c18061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.094+0000 7f6c267fc700 1 -- 172.21.15.33:0/617601766 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6c3007c0f0 con 0x7f6c38154710 2023-11-06T20:44:23.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.094+0000 7f6c3659c700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c1805f150 0x7f6c18061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.095+0000 7f6c3659c700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c1805f150 0x7f6c18061610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f6c280011e0 tx=0x7f6c28007bb0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:23.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.098+0000 7f6c267fc700 1 -- 172.21.15.33:0/617601766 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6c3004d380 con 0x7f6c38154710 2023-11-06T20:44:23.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.276+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/617601766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c380830b0 con 0x7f6c38154710 2023-11-06T20:44:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.276+0000 7f6c267fc700 1 -- 172.21.15.33:0/617601766 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c300509a0 con 0x7f6c38154710 2023-11-06T20:44:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:23.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.278+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c1805f150 msgr2=0x7f6c18061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.278+0000 7f6c3c9ff700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c1805f150 0x7f6c18061610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f6c280011e0 tx=0x7f6c28007bb0).stop 2023-11-06T20:44:23.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.278+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/617601766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c38154710 msgr2=0x7f6c380ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.278+0000 7f6c3c9ff700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c38154710 0x7f6c380ae2c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f6c300094c0 tx=0x7f6c30009780).stop 2023-11-06T20:44:23.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.278+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/617601766 shutdown_connections 2023-11-06T20:44:23.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.278+0000 7f6c3c9ff700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c1805f150 0x7f6c18061610 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.278+0000 7f6c3c9ff700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c38154710 0x7f6c380ae2c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.278+0000 7f6c3c9ff700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c380c7ff0 0x7f6c380a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.278+0000 7f6c3c9ff700 1 --2- 172.21.15.33:0/617601766 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c38154330 0x7f6c380addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.278+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/617601766 >> 172.21.15.33:0/617601766 conn(0x7f6c380cb910 msgr2=0x7f6c380ab8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:23.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.279+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/617601766 shutdown_connections 2023-11-06T20:44:23.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.279+0000 7f6c3c9ff700 1 -- 172.21.15.33:0/617601766 wait complete. 2023-11-06T20:44:23.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T20:44:23.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.494+0000 7f5286bff700 1 Processor -- start 2023-11-06T20:44:23.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.494+0000 7f5286bff700 1 -- start start 2023-11-06T20:44:23.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.495+0000 7f5286bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154710 0x7f52800c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.495+0000 7f5286bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f52800c7ff0 0x7f52800646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.495+0000 7f5286bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52800c6b60 con 0x7f5280154330 2023-11-06T20:44:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.495+0000 7f5286bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52800c7700 con 0x7f52800c7ff0 2023-11-06T20:44:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.495+0000 7f5286bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52800c5cd0 con 0x7f5280154710 2023-11-06T20:44:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f528519c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f52800c7ff0 0x7f52800646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f528519c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f52800c7ff0 0x7f52800646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33740/0 (socket says 172.21.15.33:33740) 2023-11-06T20:44:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f528519c700 1 -- 172.21.15.33:0/1144264614 learned_addr learned my addr 172.21.15.33:0/1144264614 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f527ffff700 1 --2- 172.21.15.33:0/1144264614 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154710 0x7f52800c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f528519c700 1 -- 172.21.15.33:0/1144264614 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154710 msgr2=0x7f52800c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f528519c700 1 --2- 172.21.15.33:0/1144264614 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154710 0x7f52800c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f528519c700 1 -- 172.21.15.33:0/1144264614 >> v1:172.21.15.69:6789/0 conn(0x7f5280154330 legacy=0x7f52800c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f528519c700 1 -- 172.21.15.33:0/1144264614 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52800c5f50 con 0x7f52800c7ff0 2023-11-06T20:44:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f527ffff700 1 --2- 172.21.15.33:0/1144264614 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154710 0x7f52800c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f528519c700 1 --2- 172.21.15.33:0/1144264614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7ff0 0x7f52800646c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f5278009f20 tx=0x7f5278009840).ready entity=mon.0 client_cookie=bcd2454f2e42865b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.496+0000 7f527f7fe700 1 -- 172.21.15.33:0/1144264614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527800e070 con 0x7f52800c7ff0 2023-11-06T20:44:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.497+0000 7f527f7fe700 1 -- 172.21.15.33:0/1144264614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527800ba80 con 0x7f52800c7ff0 2023-11-06T20:44:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.497+0000 7f527f7fe700 1 -- 172.21.15.33:0/1144264614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527800be00 con 0x7f52800c7ff0 2023-11-06T20:44:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.497+0000 7f5286bff700 1 -- 172.21.15.33:0/1144264614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7ff0 msgr2=0x7f52800646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.497+0000 7f5286bff700 1 --2- 172.21.15.33:0/1144264614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7ff0 0x7f52800646c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f5278009f20 tx=0x7f5278009840).stop 2023-11-06T20:44:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.497+0000 7f5286bff700 1 -- 172.21.15.33:0/1144264614 shutdown_connections 2023-11-06T20:44:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.497+0000 7f5286bff700 1 --2- 172.21.15.33:0/1144264614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7ff0 0x7f52800646c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.497+0000 7f5286bff700 1 --2- 172.21.15.33:0/1144264614 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154710 0x7f52800c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.497+0000 7f5286bff700 1 -- 172.21.15.33:0/1144264614 >> 172.21.15.33:0/1144264614 conn(0x7f52800cb910 msgr2=0x7f52800c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.497+0000 7f5286bff700 1 -- 172.21.15.33:0/1144264614 shutdown_connections 2023-11-06T20:44:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.497+0000 7f5286bff700 1 -- 172.21.15.33:0/1144264614 wait complete. 2023-11-06T20:44:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.499+0000 7f5286bff700 1 Processor -- start 2023-11-06T20:44:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.499+0000 7f5286bff700 1 -- start start 2023-11-06T20:44:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f5286bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52800c7ff0 0x7f52801525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f5286bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154330 0x7f52801574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f5286bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5280154710 0x7f52801579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f5286bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52800aea80 con 0x7f5280154710 2023-11-06T20:44:23.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f5286bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52800ade70 con 0x7f52800c7ff0 2023-11-06T20:44:23.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f5286bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52800ae150 con 0x7f5280154330 2023-11-06T20:44:23.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f528519c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5280154710 0x7f52801579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f528519c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5280154710 0x7f52801579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33752/0 (socket says 172.21.15.33:33752) 2023-11-06T20:44:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f528519c700 1 -- 172.21.15.33:0/1250350902 learned_addr learned my addr 172.21.15.33:0/1250350902 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f527ffff700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154330 0x7f52801574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f528499b700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52800c7ff0 0x7f52801525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f528519c700 1 -- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154330 msgr2=0x7f52801574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f528519c700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154330 0x7f52801574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f528519c700 1 -- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52800c7ff0 msgr2=0x7f52801525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f528519c700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52800c7ff0 0x7f52801525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f528519c700 1 -- 172.21.15.33:0/1250350902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5278009ae0 con 0x7f5280154710 2023-11-06T20:44:23.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.500+0000 7f528499b700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52800c7ff0 0x7f52801525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:23.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.501+0000 7f528519c700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5280154710 0x7f52801579e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f527800b6c0 tx=0x7f5278009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:23.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.501+0000 7f527cff9700 1 -- 172.21.15.33:0/1250350902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527800be00 con 0x7f5280154710 2023-11-06T20:44:23.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.501+0000 7f527cff9700 1 -- 172.21.15.33:0/1250350902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52780032f0 con 0x7f5280154710 2023-11-06T20:44:23.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.501+0000 7f5286bff700 1 -- 172.21.15.33:0/1250350902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52800ae430 con 0x7f5280154710 2023-11-06T20:44:23.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.501+0000 7f527cff9700 1 -- 172.21.15.33:0/1250350902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527801d690 con 0x7f5280154710 2023-11-06T20:44:23.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.501+0000 7f5286bff700 1 -- 172.21.15.33:0/1250350902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5280152ae0 con 0x7f5280154710 2023-11-06T20:44:23.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.502+0000 7f5286bff700 1 -- 172.21.15.33:0/1250350902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52800ca750 con 0x7f5280154710 2023-11-06T20:44:23.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.503+0000 7f527cff9700 1 -- 172.21.15.33:0/1250350902 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5278003040 con 0x7f5280154710 2023-11-06T20:44:23.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.504+0000 7f527cff9700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f526405f150 0x7f5264061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.505+0000 7f527cff9700 1 -- 172.21.15.33:0/1250350902 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f527807b560 con 0x7f5280154710 2023-11-06T20:44:23.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.505+0000 7f528499b700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f526405f150 0x7f5264061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.505+0000 7f528499b700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f526405f150 0x7f5264061610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5274007fd0 tx=0x7f5274009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:23.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.508+0000 7f527cff9700 1 -- 172.21.15.33:0/1250350902 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f527804c830 con 0x7f5280154710 2023-11-06T20:44:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.689+0000 7f5286bff700 1 -- 172.21.15.33:0/1250350902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52800830b0 con 0x7f5280154710 2023-11-06T20:44:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.689+0000 7f527cff9700 1 -- 172.21.15.33:0/1250350902 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5278022070 con 0x7f5280154710 2023-11-06T20:44:23.690 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.691+0000 7f5286bff700 1 -- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f526405f150 msgr2=0x7f5264061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.691+0000 7f5286bff700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f526405f150 0x7f5264061610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5274007fd0 tx=0x7f5274009040).stop 2023-11-06T20:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.691+0000 7f5286bff700 1 -- 172.21.15.33:0/1250350902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5280154710 msgr2=0x7f52801579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.691+0000 7f5286bff700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5280154710 0x7f52801579e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f527800b6c0 tx=0x7f5278009840).stop 2023-11-06T20:44:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.691+0000 7f5286bff700 1 -- 172.21.15.33:0/1250350902 shutdown_connections 2023-11-06T20:44:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.691+0000 7f5286bff700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f526405f150 0x7f5264061610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.691+0000 7f5286bff700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5280154710 0x7f52801579e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.691+0000 7f5286bff700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52800c7ff0 0x7f52801525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.691+0000 7f5286bff700 1 --2- 172.21.15.33:0/1250350902 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5280154330 0x7f52801574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.691+0000 7f5286bff700 1 -- 172.21.15.33:0/1250350902 >> 172.21.15.33:0/1250350902 conn(0x7f52800cb910 msgr2=0x7f52800c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.692+0000 7f5286bff700 1 -- 172.21.15.33:0/1250350902 shutdown_connections 2023-11-06T20:44:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.692+0000 7f5286bff700 1 -- 172.21.15.33:0/1250350902 wait complete. 2023-11-06T20:44:23.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:44:23.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.904+0000 7f16ea5ff700 1 Processor -- start 2023-11-06T20:44:23.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.904+0000 7f16ea5ff700 1 -- start start 2023-11-06T20:44:23.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16ea5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f16e40c1840 0x7f16e40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16ea5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16e40aa2d0 0x7f16e40c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16ea5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 0x7f16e4157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16ea5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e4154960 con 0x7f16e40aa2d0 2023-11-06T20:44:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16ea5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e41554a0 con 0x7f16e40c1840 2023-11-06T20:44:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16ea5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e4155930 con 0x7f16e40c6240 2023-11-06T20:44:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16e37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16e40aa2d0 0x7f16e40c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16e37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16e40aa2d0 0x7f16e40c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33758/0 (socket says 172.21.15.33:33758) 2023-11-06T20:44:23.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16e37fe700 1 -- 172.21.15.33:0/3219829578 learned_addr learned my addr 172.21.15.33:0/3219829578 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:23.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16e3fff700 1 --2- 172.21.15.33:0/3219829578 >> v2:172.21.15.69:3300/0 conn(0x7f16e40c1840 0x7f16e40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16e8b9c700 1 --2- 172.21.15.33:0/3219829578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 0x7f16e4157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16e37fe700 1 -- 172.21.15.33:0/3219829578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 msgr2=0x7f16e4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16e37fe700 1 --2- 172.21.15.33:0/3219829578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 0x7f16e4157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16e37fe700 1 -- 172.21.15.33:0/3219829578 >> v2:172.21.15.69:3300/0 conn(0x7f16e40c1840 msgr2=0x7f16e40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16e37fe700 1 --2- 172.21.15.33:0/3219829578 >> v2:172.21.15.69:3300/0 conn(0x7f16e40c1840 0x7f16e40a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.906+0000 7f16e37fe700 1 -- 172.21.15.33:0/3219829578 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e40c6690 con 0x7f16e40aa2d0 2023-11-06T20:44:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.907+0000 7f16e3fff700 1 --2- 172.21.15.33:0/3219829578 >> v2:172.21.15.69:3300/0 conn(0x7f16e40c1840 0x7f16e40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.907+0000 7f16e8b9c700 1 --2- 172.21.15.33:0/3219829578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 0x7f16e4157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.907+0000 7f16e37fe700 1 --2- 172.21.15.33:0/3219829578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40aa2d0 0x7f16e40c5d50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f16e40840b0 tx=0x7f16d80090f0).ready entity=mon.0 client_cookie=2e4bd3a4af390dd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.907+0000 7f16e2ffd700 1 -- 172.21.15.33:0/3219829578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800b600 con 0x7f16e40aa2d0 2023-11-06T20:44:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.907+0000 7f16e2ffd700 1 -- 172.21.15.33:0/3219829578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d800b760 con 0x7f16e40aa2d0 2023-11-06T20:44:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.907+0000 7f16e2ffd700 1 -- 172.21.15.33:0/3219829578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800bae0 con 0x7f16e40aa2d0 2023-11-06T20:44:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.907+0000 7f16ea5ff700 1 -- 172.21.15.33:0/3219829578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40aa2d0 msgr2=0x7f16e40c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.907+0000 7f16ea5ff700 1 --2- 172.21.15.33:0/3219829578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40aa2d0 0x7f16e40c5d50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f16e40840b0 tx=0x7f16d80090f0).stop 2023-11-06T20:44:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.908+0000 7f16ea5ff700 1 -- 172.21.15.33:0/3219829578 shutdown_connections 2023-11-06T20:44:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.908+0000 7f16ea5ff700 1 --2- 172.21.15.33:0/3219829578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 0x7f16e4157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.908+0000 7f16ea5ff700 1 --2- 172.21.15.33:0/3219829578 >> v2:172.21.15.69:3300/0 conn(0x7f16e40c1840 0x7f16e40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.908+0000 7f16ea5ff700 1 --2- 172.21.15.33:0/3219829578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40aa2d0 0x7f16e40c5d50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.908+0000 7f16ea5ff700 1 -- 172.21.15.33:0/3219829578 >> 172.21.15.33:0/3219829578 conn(0x7f16e405ec80 msgr2=0x7f16e40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.908+0000 7f16ea5ff700 1 -- 172.21.15.33:0/3219829578 shutdown_connections 2023-11-06T20:44:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.908+0000 7f16ea5ff700 1 -- 172.21.15.33:0/3219829578 wait complete. 2023-11-06T20:44:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.909+0000 7f16ea5ff700 1 Processor -- start 2023-11-06T20:44:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.909+0000 7f16ea5ff700 1 -- start start 2023-11-06T20:44:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.910+0000 7f16ea5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e40aa2d0 0x7f16e4155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.910+0000 7f16ea5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40c1840 0x7f16e40c2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.910+0000 7f16ea5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 0x7f16e40c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.910+0000 7f16ea5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40c2640 con 0x7f16e40c1840 2023-11-06T20:44:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.910+0000 7f16ea5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e414fb10 con 0x7f16e40aa2d0 2023-11-06T20:44:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.910+0000 7f16ea5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40ac7e0 con 0x7f16e40c6240 2023-11-06T20:44:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.910+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40c1840 0x7f16e40c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.910+0000 7f16e3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e40aa2d0 0x7f16e4155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.911+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40c1840 0x7f16e40c2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33770/0 (socket says 172.21.15.33:33770) 2023-11-06T20:44:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.911+0000 7f16e37fe700 1 -- 172.21.15.33:0/877355764 learned_addr learned my addr 172.21.15.33:0/877355764 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.911+0000 7f16e8b9c700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 0x7f16e40c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.911+0000 7f16e37fe700 1 -- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 msgr2=0x7f16e40c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.911+0000 7f16e37fe700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 0x7f16e40c27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.911+0000 7f16e37fe700 1 -- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e40aa2d0 msgr2=0x7f16e4155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.911+0000 7f16e37fe700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e40aa2d0 0x7f16e4155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.911+0000 7f16e37fe700 1 -- 172.21.15.33:0/877355764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d8009370 con 0x7f16e40c1840 2023-11-06T20:44:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.911+0000 7f16e37fe700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40c1840 0x7f16e40c2150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f16e406bc90 tx=0x7f16d8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.912+0000 7f16cbfff700 1 -- 172.21.15.33:0/877355764 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800bea0 con 0x7f16e40c1840 2023-11-06T20:44:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.912+0000 7f16cbfff700 1 -- 172.21.15.33:0/877355764 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d800b8d0 con 0x7f16e40c1840 2023-11-06T20:44:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.912+0000 7f16cbfff700 1 -- 172.21.15.33:0/877355764 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d801b690 con 0x7f16e40c1840 2023-11-06T20:44:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.912+0000 7f16ea5ff700 1 -- 172.21.15.33:0/877355764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e414fd80 con 0x7f16e40c1840 2023-11-06T20:44:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.912+0000 7f16ea5ff700 1 -- 172.21.15.33:0/877355764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e40c2cd0 con 0x7f16e40c1840 2023-11-06T20:44:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.914+0000 7f16ea5ff700 1 -- 172.21.15.33:0/877355764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e40ac9d0 con 0x7f16e40c1840 2023-11-06T20:44:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.914+0000 7f16cbfff700 1 -- 172.21.15.33:0/877355764 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f16d801ba00 con 0x7f16e40c1840 2023-11-06T20:44:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.915+0000 7f16cbfff700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16d005f150 0x7f16d0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.915+0000 7f16cbfff700 1 -- 172.21.15.33:0/877355764 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f16d807b810 con 0x7f16e40c1840 2023-11-06T20:44:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.915+0000 7f16e3fff700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16d005f150 0x7f16d0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.916+0000 7f16e3fff700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16d005f150 0x7f16d0061610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f16d4006d80 tx=0x7f16d4008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:23.919+0000 7f16cbfff700 1 -- 172.21.15.33:0/877355764 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f16e40ac9d0 con 0x7f16e40c1840 2023-11-06T20:44:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.141+0000 7f16ea5ff700 1 -- 172.21.15.33:0/877355764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16e40c2e60 con 0x7f16e40c1840 2023-11-06T20:44:24.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.141+0000 7f16cbfff700 1 -- 172.21.15.33:0/877355764 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4480 (secure 0 0 0) 0x7f16e40c2e60 con 0x7f16e40c1840 2023-11-06T20:44:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:24.142 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4123875619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123875619},{"type":"v1","addr":"172.21.15.33:6834","nonce":4123875619}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3506554570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3506554570},{"type":"v1","addr":"172.21.15.33:6835","nonce":3506554570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/239985290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":239985290},{"type":"v1","addr":"172.21.15.69:6835","nonce":239985290}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:44:06.383743+0000","modified":"2023-11-06T20:44:09.389408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/2681789033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2681789033},{"type":"v1","addr":"172.21.15.69:6837","nonce":2681789033}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-06T20:44:24.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 -- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16d005f150 msgr2=0x7f16d0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16d005f150 0x7f16d0061610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f16d4006d80 tx=0x7f16d4008040).stop 2023-11-06T20:44:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 -- 172.21.15.33:0/877355764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40c1840 msgr2=0x7f16e40c2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40c1840 0x7f16e40c2150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f16e406bc90 tx=0x7f16d8017000).stop 2023-11-06T20:44:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 -- 172.21.15.33:0/877355764 shutdown_connections 2023-11-06T20:44:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16d005f150 0x7f16d0061610 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16e40c6240 0x7f16e40c27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e40aa2d0 0x7f16e4155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 --2- 172.21.15.33:0/877355764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16e40c1840 0x7f16e40c2150 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 -- 172.21.15.33:0/877355764 >> 172.21.15.33:0/877355764 conn(0x7f16e405ec80 msgr2=0x7f16e40bfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 -- 172.21.15.33:0/877355764 shutdown_connections 2023-11-06T20:44:24.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.143+0000 7f16ea5ff700 1 -- 172.21.15.33:0/877355764 wait complete. 2023-11-06T20:44:24.146 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2023-11-06T20:44:24.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:24.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.356+0000 7f4636bff700 1 Processor -- start 2023-11-06T20:44:24.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.356+0000 7f4636bff700 1 -- start start 2023-11-06T20:44:24.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f4636bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46300c5d50 0x7f46300c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f4636bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46300bfd70 0x7f46300c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f4636bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46301554a0 con 0x7f46300c1840 2023-11-06T20:44:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f4636bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4630154960 con 0x7f46300c5d50 2023-11-06T20:44:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f4636bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46300c66c0 con 0x7f46300bfd70 2023-11-06T20:44:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f462ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46300c5d50 0x7f46300c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f462ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46300c5d50 0x7f46300c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33786/0 (socket says 172.21.15.33:33786) 2023-11-06T20:44:24.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f462ffff700 1 -- 172.21.15.33:0/1793770506 learned_addr learned my addr 172.21.15.33:0/1793770506 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:24.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f463519c700 1 --2- 172.21.15.33:0/1793770506 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46300bfd70 0x7f46300c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f462ffff700 1 -- 172.21.15.33:0/1793770506 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46300bfd70 msgr2=0x7f46300c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f462ffff700 1 --2- 172.21.15.33:0/1793770506 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46300bfd70 0x7f46300c01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.358+0000 7f462ffff700 1 -- 172.21.15.33:0/1793770506 >> v1:172.21.15.69:6789/0 conn(0x7f46300c1840 legacy=0x7f46300a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.359+0000 7f462ffff700 1 -- 172.21.15.33:0/1793770506 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4630155930 con 0x7f46300c5d50 2023-11-06T20:44:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.359+0000 7f462ffff700 1 --2- 172.21.15.33:0/1793770506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 0x7f46300c61d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f461c006fd0 tx=0x7f461c0090f0).ready entity=mon.0 client_cookie=2975081f5efd256e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:24.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.359+0000 7f462f7fe700 1 -- 172.21.15.33:0/1793770506 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c00b580 con 0x7f46300c5d50 2023-11-06T20:44:24.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.359+0000 7f462f7fe700 1 -- 172.21.15.33:0/1793770506 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c00b6e0 con 0x7f46300c5d50 2023-11-06T20:44:24.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.359+0000 7f462f7fe700 1 -- 172.21.15.33:0/1793770506 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c00ba60 con 0x7f46300c5d50 2023-11-06T20:44:24.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.359+0000 7f4636bff700 1 -- 172.21.15.33:0/1793770506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 msgr2=0x7f46300c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.359+0000 7f4636bff700 1 --2- 172.21.15.33:0/1793770506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 0x7f46300c61d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f461c006fd0 tx=0x7f461c0090f0).stop 2023-11-06T20:44:24.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.360+0000 7f4636bff700 1 -- 172.21.15.33:0/1793770506 shutdown_connections 2023-11-06T20:44:24.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.360+0000 7f4636bff700 1 --2- 172.21.15.33:0/1793770506 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46300bfd70 0x7f46300c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.360+0000 7f4636bff700 1 --2- 172.21.15.33:0/1793770506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 0x7f46300c61d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.360+0000 7f4636bff700 1 -- 172.21.15.33:0/1793770506 >> 172.21.15.33:0/1793770506 conn(0x7f463005ec80 msgr2=0x7f46300cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:24.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.360+0000 7f4636bff700 1 -- 172.21.15.33:0/1793770506 shutdown_connections 2023-11-06T20:44:24.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.360+0000 7f4636bff700 1 -- 172.21.15.33:0/1793770506 wait complete. 2023-11-06T20:44:24.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.361+0000 7f4636bff700 1 Processor -- start 2023-11-06T20:44:24.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.361+0000 7f4636bff700 1 -- start start 2023-11-06T20:44:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.362+0000 7f4636bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46300bfd70 0x7f4630146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.362+0000 7f4636bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46300c1840 0x7f4630149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.362+0000 7f4636bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 0x7f46300c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.362+0000 7f4636bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630156770 con 0x7f46300c5d50 2023-11-06T20:44:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.362+0000 7f4636bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46301470d0 con 0x7f46300c1840 2023-11-06T20:44:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.362+0000 7f4636bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4630155e40 con 0x7f46300bfd70 2023-11-06T20:44:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.362+0000 7f463519c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 0x7f46300c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.362+0000 7f463519c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 0x7f46300c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33796/0 (socket says 172.21.15.33:33796) 2023-11-06T20:44:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.362+0000 7f463519c700 1 -- 172.21.15.33:0/1514784584 learned_addr learned my addr 172.21.15.33:0/1514784584 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f462ffff700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46300c1840 0x7f4630149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f463499b700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46300bfd70 0x7f4630146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f463519c700 1 -- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46300bfd70 msgr2=0x7f4630146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f463519c700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46300bfd70 0x7f4630146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f463519c700 1 -- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46300c1840 msgr2=0x7f4630149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f463519c700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46300c1840 0x7f4630149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f463519c700 1 -- 172.21.15.33:0/1514784584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c009370 con 0x7f46300c5d50 2023-11-06T20:44:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f463519c700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 0x7f46300c9da0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f4628009ec0 tx=0x7f462800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f462cff9700 1 -- 172.21.15.33:0/1514784584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800cb70 con 0x7f46300c5d50 2023-11-06T20:44:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f462cff9700 1 -- 172.21.15.33:0/1514784584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462800f040 con 0x7f46300c5d50 2023-11-06T20:44:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f4636bff700 1 -- 172.21.15.33:0/1514784584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4630156120 con 0x7f46300c5d50 2023-11-06T20:44:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f462cff9700 1 -- 172.21.15.33:0/1514784584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800cee0 con 0x7f46300c5d50 2023-11-06T20:44:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.363+0000 7f4636bff700 1 -- 172.21.15.33:0/1514784584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4630148900 con 0x7f46300c5d50 2023-11-06T20:44:24.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.365+0000 7f4636bff700 1 -- 172.21.15.33:0/1514784584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46301565c0 con 0x7f46300c5d50 2023-11-06T20:44:24.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.365+0000 7f462cff9700 1 -- 172.21.15.33:0/1514784584 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4628013610 con 0x7f46300c5d50 2023-11-06T20:44:24.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.367+0000 7f462cff9700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f461405f150 0x7f4614061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.367+0000 7f462cff9700 1 -- 172.21.15.33:0/1514784584 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f462807aa00 con 0x7f46300c5d50 2023-11-06T20:44:24.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.367+0000 7f463499b700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f461405f150 0x7f4614061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.368+0000 7f463499b700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f461405f150 0x7f4614061610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4624003870 tx=0x7f4624007d40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:24.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.371+0000 7f462cff9700 1 -- 172.21.15.33:0/1514784584 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f462804bcd0 con 0x7f46300c5d50 2023-11-06T20:44:24.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.553+0000 7f4636bff700 1 -- 172.21.15.33:0/1514784584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f463006c6c0 con 0x7f46300c5d50 2023-11-06T20:44:24.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.554+0000 7f462cff9700 1 -- 172.21.15.33:0/1514784584 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f462804f2f0 con 0x7f46300c5d50 2023-11-06T20:44:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:24.554 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:24.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 -- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f461405f150 msgr2=0x7f4614061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f461405f150 0x7f4614061610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4624003870 tx=0x7f4624007d40).stop 2023-11-06T20:44:24.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 -- 172.21.15.33:0/1514784584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 msgr2=0x7f46300c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 0x7f46300c9da0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f4628009ec0 tx=0x7f462800b040).stop 2023-11-06T20:44:24.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 -- 172.21.15.33:0/1514784584 shutdown_connections 2023-11-06T20:44:24.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f461405f150 0x7f4614061610 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46300bfd70 0x7f4630146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46300c5d50 0x7f46300c9da0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 --2- 172.21.15.33:0/1514784584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46300c1840 0x7f4630149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 -- 172.21.15.33:0/1514784584 >> 172.21.15.33:0/1514784584 conn(0x7f463005ec80 msgr2=0x7f46300c02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:24.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 -- 172.21.15.33:0/1514784584 shutdown_connections 2023-11-06T20:44:24.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.556+0000 7f4636bff700 1 -- 172.21.15.33:0/1514784584 wait complete. 2023-11-06T20:44:24.574 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:44:24.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.768+0000 7fb2fc3ff700 1 Processor -- start 2023-11-06T20:44:24.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.768+0000 7fb2fc3ff700 1 -- start start 2023-11-06T20:44:24.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.770+0000 7fb2fc3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb2f40c1840 0x7fb2f40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.770+0000 7fb2fc3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40aa2d0 0x7fb2f40c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.770+0000 7fb2fc3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f41554a0 con 0x7fb2f40c6240 2023-11-06T20:44:24.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.770+0000 7fb2fc3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f4154960 con 0x7fb2f40c1840 2023-11-06T20:44:24.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.770+0000 7fb2fc3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4155930 con 0x7fb2f40aa2d0 2023-11-06T20:44:24.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.770+0000 7fb2f999a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40aa2d0 0x7fb2f40c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.770+0000 7fb2fa19b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb2f40c1840 0x7fb2f40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.770+0000 7fb2fa19b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb2f40c1840 0x7fb2f40a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:36024/0 (socket says 172.21.15.33:36024) 2023-11-06T20:44:24.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.770+0000 7fb2fa19b700 1 -- 172.21.15.33:0/846471013 learned_addr learned my addr 172.21.15.33:0/846471013 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:24.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.771+0000 7fb2fa19b700 1 -- 172.21.15.33:0/846471013 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40aa2d0 msgr2=0x7fb2f40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.771+0000 7fb2fa19b700 1 --2- 172.21.15.33:0/846471013 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40aa2d0 0x7fb2f40c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.771+0000 7fb2fa19b700 1 -- 172.21.15.33:0/846471013 >> v1:172.21.15.69:6789/0 conn(0x7fb2f40c6240 legacy=0x7fb2f4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.771+0000 7fb2fa19b700 1 -- 172.21.15.33:0/846471013 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f40c66b0 con 0x7fb2f40c1840 2023-11-06T20:44:24.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.771+0000 7fb2fa19b700 1 --2- 172.21.15.33:0/846471013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40c1840 0x7fb2f40a9de0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fb2f0006fd0 tx=0x7fb2f00090f0).ready entity=mon.1 client_cookie=641467f35f1a5c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.771+0000 7fb2f9199700 1 -- 172.21.15.33:0/846471013 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000b580 con 0x7fb2f40c1840 2023-11-06T20:44:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.771+0000 7fb2f9199700 1 -- 172.21.15.33:0/846471013 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f000b6e0 con 0x7fb2f40c1840 2023-11-06T20:44:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.771+0000 7fb2f9199700 1 -- 172.21.15.33:0/846471013 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000ba60 con 0x7fb2f40c1840 2023-11-06T20:44:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.772+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/846471013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40c1840 msgr2=0x7fb2f40a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.772+0000 7fb2fc3ff700 1 --2- 172.21.15.33:0/846471013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40c1840 0x7fb2f40a9de0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fb2f0006fd0 tx=0x7fb2f00090f0).stop 2023-11-06T20:44:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.772+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/846471013 shutdown_connections 2023-11-06T20:44:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.772+0000 7fb2fc3ff700 1 --2- 172.21.15.33:0/846471013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40c1840 0x7fb2f40a9de0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.772+0000 7fb2fc3ff700 1 --2- 172.21.15.33:0/846471013 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40aa2d0 0x7fb2f40c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.772+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/846471013 >> 172.21.15.33:0/846471013 conn(0x7fb2f405ec80 msgr2=0x7fb2f40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:24.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.772+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/846471013 shutdown_connections 2023-11-06T20:44:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.772+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/846471013 wait complete. 2023-11-06T20:44:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.773+0000 7fb2fc3ff700 1 Processor -- start 2023-11-06T20:44:24.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.774+0000 7fb2fc3ff700 1 -- start start 2023-11-06T20:44:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.774+0000 7fb2fc3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40aa2d0 0x7fb2f4063300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.774+0000 7fb2fc3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40c1840 0x7fb2f4149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.774+0000 7fb2fc3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2f40c6240 0x7fb2f4149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.774+0000 7fb2fc3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f414a160 con 0x7fb2f40c6240 2023-11-06T20:44:24.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2fc3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f40c2f80 con 0x7fb2f40aa2d0 2023-11-06T20:44:24.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2fc3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f40af340 con 0x7fb2f40c1840 2023-11-06T20:44:24.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2f999a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40c1840 0x7fb2f4149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2fa99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2f40c6240 0x7fb2f4149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2fa19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40aa2d0 0x7fb2f4063300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2f999a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40c1840 0x7fb2f4149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:47858/0 (socket says 172.21.15.33:47858) 2023-11-06T20:44:24.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2f999a700 1 -- 172.21.15.33:0/370529682 learned_addr learned my addr 172.21.15.33:0/370529682 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:24.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2f999a700 1 -- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40aa2d0 msgr2=0x7fb2f4063300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2f999a700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40aa2d0 0x7fb2f4063300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2f999a700 1 -- 172.21.15.33:0/370529682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2f40c6240 msgr2=0x7fb2f4149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:24.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2f999a700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2f40c6240 0x7fb2f4149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:24.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2f999a700 1 -- 172.21.15.33:0/370529682 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f0009370 con 0x7fb2f40c1840 2023-11-06T20:44:24.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.775+0000 7fb2fa99c700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2f40c6240 0x7fb2f4149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:24.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.776+0000 7fb2f999a700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40c1840 0x7fb2f4149780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fb2e400e8d0 tx=0x7fb2e4009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.776+0000 7fb2ea7fc700 1 -- 172.21.15.33:0/370529682 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4012600 con 0x7fb2f40c1840 2023-11-06T20:44:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.776+0000 7fb2ea7fc700 1 -- 172.21.15.33:0/370529682 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e4012760 con 0x7fb2f40c1840 2023-11-06T20:44:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.776+0000 7fb2ea7fc700 1 -- 172.21.15.33:0/370529682 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4012ae0 con 0x7fb2f40c1840 2023-11-06T20:44:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.776+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/370529682 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f40c3240 con 0x7fb2f40c1840 2023-11-06T20:44:24.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.776+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/370529682 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f40af060 con 0x7fb2f40c1840 2023-11-06T20:44:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.777+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/370529682 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f40c33d0 con 0x7fb2f40c1840 2023-11-06T20:44:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.778+0000 7fb2ea7fc700 1 -- 172.21.15.33:0/370529682 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb2e401b400 con 0x7fb2f40c1840 2023-11-06T20:44:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.779+0000 7fb2ea7fc700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2d805f120 0x7fb2d80615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.779+0000 7fb2ea7fc700 1 -- 172.21.15.33:0/370529682 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb2e407a590 con 0x7fb2f40c1840 2023-11-06T20:44:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.779+0000 7fb2fa19b700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2d805f120 0x7fb2d80615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.780+0000 7fb2fa19b700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2d805f120 0x7fb2d80615e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fb2f0009fd0 tx=0x7fb2f0017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:24.784+0000 7fb2ea7fc700 1 -- 172.21.15.33:0/370529682 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb2e404b7a0 con 0x7fb2f40c1840 2023-11-06T20:44:25.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.011+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/370529682 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2f406c6c0 con 0x7fb2f40c1840 2023-11-06T20:44:25.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.012+0000 7fb2ea7fc700 1 -- 172.21.15.33:0/370529682 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4480 (secure 0 0 0) 0x7fb2e4016020 con 0x7fb2f40c1840 2023-11-06T20:44:25.013 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:25.013 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4123875619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123875619},{"type":"v1","addr":"172.21.15.33:6834","nonce":4123875619}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3506554570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3506554570},{"type":"v1","addr":"172.21.15.33:6835","nonce":3506554570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/239985290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":239985290},{"type":"v1","addr":"172.21.15.69:6835","nonce":239985290}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:44:06.383743+0000","modified":"2023-11-06T20:44:09.389408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/2681789033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2681789033},{"type":"v1","addr":"172.21.15.69:6837","nonce":2681789033}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-06T20:44:25.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2d805f120 msgr2=0x7fb2d80615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2d805f120 0x7fb2d80615e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fb2f0009fd0 tx=0x7fb2f0017000).stop 2023-11-06T20:44:25.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40c1840 msgr2=0x7fb2f4149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40c1840 0x7fb2f4149780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fb2e400e8d0 tx=0x7fb2e4009060).stop 2023-11-06T20:44:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/370529682 shutdown_connections 2023-11-06T20:44:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2d805f120 0x7fb2d80615e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2f40c6240 0x7fb2f4149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40aa2d0 0x7fb2f4063300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 --2- 172.21.15.33:0/370529682 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2f40c1840 0x7fb2f4149780 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/370529682 >> 172.21.15.33:0/370529682 conn(0x7fb2f405ec80 msgr2=0x7fb2f40ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:25.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/370529682 shutdown_connections 2023-11-06T20:44:25.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.015+0000 7fb2fc3ff700 1 -- 172.21.15.33:0/370529682 wait complete. 2023-11-06T20:44:25.018 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2023-11-06T20:44:25.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T20:44:06.383743+0000', 'modified': '2023-11-06T20:44:09.389408+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4257}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4257': {'gid': 4257, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.69:6837/2681789033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6836', 'nonce': 2681789033}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 2681789033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T20:44:25.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T20:44:25.436 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-06T20:44:25.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.634+0000 7fcea39ff700 1 Processor -- start 2023-11-06T20:44:25.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.634+0000 7fcea39ff700 1 -- start start 2023-11-06T20:44:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fcea39ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c0c7420 0x7fce9c05eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fcea39ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fce9c064cb0 0x7fce9c0c5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fcea39ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce9c155480 con 0x7fce9c0a9e10 2023-11-06T20:44:25.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fcea39ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9c0aa5f0 con 0x7fce9c064cb0 2023-11-06T20:44:25.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fcea39ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c154b50 con 0x7fce9c0c7420 2023-11-06T20:44:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fcea0f9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fce9c0a9e10 0x7fce9c0645a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51716/0 (socket says 172.21.15.33:51716) 2023-11-06T20:44:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fcea0f9a700 1 -- 172.21.15.33:0/3070809545 learned_addr learned my addr 172.21.15.33:0/3070809545 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/3070809545 >> v2:172.21.15.69:3300/0 conn(0x7fce9c064cb0 0x7fce9c0c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fcea179b700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c0c7420 0x7fce9c05eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353789240 0 0) 0x7fce9c155480 con 0x7fce9c0a9e10 2023-11-06T20:44:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.636+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce880033f0 con 0x7fce9c0a9e10 2023-11-06T20:44:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606030746 0 0) 0x7fce880033f0 con 0x7fce9c0a9e10 2023-11-06T20:44:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c0c7420 msgr2=0x7fce9c05eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c0c7420 0x7fce9c05eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 >> v2:172.21.15.69:3300/0 conn(0x7fce9c064cb0 msgr2=0x7fce9c0c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 --2- 172.21.15.33:0/3070809545 >> v2:172.21.15.69:3300/0 conn(0x7fce9c064cb0 0x7fce9c0c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c154e30 con 0x7fce9c0a9e10 2023-11-06T20:44:25.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fce8c001470 con 0x7fce9c0a9e10 2023-11-06T20:44:25.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 >> v1:172.21.15.33:6789/0 conn(0x7fce9c0a9e10 legacy=0x7fce9c0645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c0c7420 0x7fce880038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce88003e10 0x7fce88004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce88004b80 0x7fce88005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c0aa5f0 con 0x7fce88004b80 2023-11-06T20:44:25.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c154b50 con 0x7fce88003e10 2023-11-06T20:44:25.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8800b8e0 con 0x7fce9c0c7420 2023-11-06T20:44:25.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce88004b80 0x7fce88005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fcea179b700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c0c7420 0x7fce880038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.637+0000 7fcea0f9a700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce88003e10 0x7fce88004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fcea1f9c700 1 -- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c0c7420 msgr2=0x7fce880038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c0c7420 0x7fce880038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fcea1f9c700 1 -- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce88003e10 msgr2=0x7fce88004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce88003e10 0x7fce88004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fcea1f9c700 1 -- 172.21.15.33:0/3070809545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce880033f0 con 0x7fce88004b80 2023-11-06T20:44:25.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce88004b80 0x7fce88005000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fce9400e240 tx=0x7fce9400b040).ready entity=mon.0 client_cookie=67d19ba5453e1f6e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:25.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94009ac0 con 0x7fce88004b80 2023-11-06T20:44:25.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9400f040 con 0x7fce88004b80 2023-11-06T20:44:25.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fce93fff700 1 -- 172.21.15.33:0/3070809545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400ed00 con 0x7fce88004b80 2023-11-06T20:44:25.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fcea39ff700 1 -- 172.21.15.33:0/3070809545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce88004b80 msgr2=0x7fce88005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fcea39ff700 1 --2- 172.21.15.33:0/3070809545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce88004b80 0x7fce88005000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fce9400e240 tx=0x7fce9400b040).stop 2023-11-06T20:44:25.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.638+0000 7fcea1f9c700 1 -- 172.21.15.33:0/3070809545 reap_dead start 2023-11-06T20:44:25.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.639+0000 7fcea39ff700 1 -- 172.21.15.33:0/3070809545 shutdown_connections 2023-11-06T20:44:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.639+0000 7fcea39ff700 1 -- 172.21.15.33:0/3070809545 >> 172.21.15.33:0/3070809545 conn(0x7fce9c0c8320 msgr2=0x7fce9c05e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.639+0000 7fcea39ff700 1 -- 172.21.15.33:0/3070809545 shutdown_connections 2023-11-06T20:44:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.639+0000 7fcea39ff700 1 -- 172.21.15.33:0/3070809545 wait complete. 2023-11-06T20:44:25.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.640+0000 7fcea39ff700 1 Processor -- start 2023-11-06T20:44:25.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.640+0000 7fcea39ff700 1 -- start start 2023-11-06T20:44:25.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea39ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce88004b80 0x7fce9c0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea39ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce9c0af320 0x7fce9c0add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea39ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c142940 0x7fce9c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea39ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c0c37a0 con 0x7fce9c0af320 2023-11-06T20:44:25.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea39ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c0c4e90 con 0x7fce88004b80 2023-11-06T20:44:25.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea39ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c0af7a0 con 0x7fce9c142940 2023-11-06T20:44:25.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea0f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce9c0af320 0x7fce9c0add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea0f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce9c0af320 0x7fce9c0add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33828/0 (socket says 172.21.15.33:33828) 2023-11-06T20:44:25.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea179b700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce88004b80 0x7fce9c0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea0f9a700 1 -- 172.21.15.33:0/2299598246 learned_addr learned my addr 172.21.15.33:0/2299598246 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:25.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.641+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c142940 0x7fce9c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fcea0f9a700 1 -- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c142940 msgr2=0x7fce9c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fcea0f9a700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c142940 0x7fce9c1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fcea0f9a700 1 -- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce88004b80 msgr2=0x7fce9c0c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fcea0f9a700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce88004b80 0x7fce9c0c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fcea0f9a700 1 -- 172.21.15.33:0/2299598246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c0ae250 con 0x7fce9c0af320 2023-11-06T20:44:25.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fcea0f9a700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce9c0af320 0x7fce9c0add60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fce8c0049e0 tx=0x7fce8c001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:25.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fce917fa700 1 -- 172.21.15.33:0/2299598246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c011590 con 0x7fce9c0af320 2023-11-06T20:44:25.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fce917fa700 1 -- 172.21.15.33:0/2299598246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c0116f0 con 0x7fce9c0af320 2023-11-06T20:44:25.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fcea39ff700 1 -- 172.21.15.33:0/2299598246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9c0c4b60 con 0x7fce9c0af320 2023-11-06T20:44:25.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fce917fa700 1 -- 172.21.15.33:0/2299598246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c011a70 con 0x7fce9c0af320 2023-11-06T20:44:25.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.642+0000 7fcea39ff700 1 -- 172.21.15.33:0/2299598246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c0ae7b0 con 0x7fce9c0af320 2023-11-06T20:44:25.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.644+0000 7fce917fa700 1 -- 172.21.15.33:0/2299598246 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fce8c01d3b0 con 0x7fce9c0af320 2023-11-06T20:44:25.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.645+0000 7fce917fa700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce8005f150 0x7fce80061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.645+0000 7fce917fa700 1 -- 172.21.15.33:0/2299598246 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fce8c079b10 con 0x7fce9c0af320 2023-11-06T20:44:25.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.645+0000 7fcea179b700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce8005f150 0x7fce80061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.646+0000 7fcea179b700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce8005f150 0x7fce80061610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fce98006d80 tx=0x7fce98009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:25.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.647+0000 7fcebac81b80 1 Processor -- start 2023-11-06T20:44:25.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.647+0000 7fcebac81b80 1 -- start start 2023-11-06T20:44:25.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.648+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 0x560534762460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.648+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605347aabe0 0x56053475cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.648+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56053478a4c0 0x56053475d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.648+0000 7fcebac81b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053469e2f0 con 0x56053478a4c0 2023-11-06T20:44:25.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.648+0000 7fcebac81b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605346b37e0 con 0x560534762080 2023-11-06T20:44:25.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.648+0000 7fcebac81b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605346b3920 con 0x5605347aabe0 2023-11-06T20:44:25.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea1f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56053478a4c0 0x56053475d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea179b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 0x560534762460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea0f9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605347aabe0 0x56053475cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea1f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56053478a4c0 0x56053475d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33832/0 (socket says 172.21.15.33:33832) 2023-11-06T20:44:25.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea1f9c700 1 -- 172.21.15.33:0/916518232 learned_addr learned my addr 172.21.15.33:0/916518232 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:25.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea1f9c700 1 -- 172.21.15.33:0/916518232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605347aabe0 msgr2=0x56053475cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/916518232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605347aabe0 0x56053475cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea1f9c700 1 -- 172.21.15.33:0/916518232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 msgr2=0x560534762460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/916518232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 0x560534762460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea1f9c700 1 -- 172.21.15.33:0/916518232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605346ac720 con 0x56053478a4c0 2023-11-06T20:44:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/916518232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56053478a4c0 0x56053475d3e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fce94014000 tx=0x7fce9401a000).ready entity=mon.0 client_cookie=f2f410271ed46b9e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fce867fc700 1 -- 172.21.15.33:0/916518232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94022070 con 0x56053478a4c0 2023-11-06T20:44:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fce867fc700 1 -- 172.21.15.33:0/916518232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce94009ac0 con 0x56053478a4c0 2023-11-06T20:44:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.649+0000 7fce867fc700 1 -- 172.21.15.33:0/916518232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400ec30 con 0x56053478a4c0 2023-11-06T20:44:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.650+0000 7fcebac81b80 1 -- 172.21.15.33:0/916518232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56053478a4c0 msgr2=0x56053475d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.650+0000 7fcebac81b80 1 --2- 172.21.15.33:0/916518232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56053478a4c0 0x56053475d3e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fce94014000 tx=0x7fce9401a000).stop 2023-11-06T20:44:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.650+0000 7fcebac81b80 1 -- 172.21.15.33:0/916518232 shutdown_connections 2023-11-06T20:44:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.650+0000 7fcebac81b80 1 --2- 172.21.15.33:0/916518232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56053478a4c0 0x56053475d3e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.650+0000 7fcebac81b80 1 --2- 172.21.15.33:0/916518232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 0x560534762460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.650+0000 7fcebac81b80 1 --2- 172.21.15.33:0/916518232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605347aabe0 0x56053475cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.650+0000 7fcebac81b80 1 -- 172.21.15.33:0/916518232 >> 172.21.15.33:0/916518232 conn(0x5605345cb940 msgr2=0x5605347ab070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.650+0000 7fcebac81b80 1 -- 172.21.15.33:0/916518232 shutdown_connections 2023-11-06T20:44:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.650+0000 7fcebac81b80 1 -- 172.21.15.33:0/916518232 wait complete. 2023-11-06T20:44:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.651+0000 7fcebac81b80 1 -- start start 2023-11-06T20:44:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.651+0000 7fcebac81b80 1 Processor -- start 2023-11-06T20:44:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 0x560534783710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x560534783c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605347aabe0 0x56053478cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcebac81b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605346b3ba0 con 0x5605347aabe0 2023-11-06T20:44:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcebac81b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053467ee60 con 0x560534762080 2023-11-06T20:44:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcebac81b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605347a8830 con 0x56053478a4c0 2023-11-06T20:44:25.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcebac81b80 10 client.0 ll_register_callbacks cb 0x560534653ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T20:44:25.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcea1f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605347aabe0 0x56053478cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcea179b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 0x560534783710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcea0f9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x560534783c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcea1f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605347aabe0 0x56053478cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33840/0 (socket says 172.21.15.33:33840) 2023-11-06T20:44:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcea179b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 0x560534783710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:36080/0 (socket says 172.21.15.33:36080) 2023-11-06T20:44:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.652+0000 7fcea1f9c700 1 -- 172.21.15.33:0/3026894743 learned_addr learned my addr 172.21.15.33:0/3026894743 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fcea1f9c700 1 -- 172.21.15.33:0/3026894743 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 msgr2=0x560534783c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x560534783c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fcea1f9c700 1 -- 172.21.15.33:0/3026894743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 msgr2=0x560534783710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 0x560534783710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fcea1f9c700 1 -- 172.21.15.33:0/3026894743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce880033f0 con 0x5605347aabe0 2023-11-06T20:44:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605347aabe0 0x56053478cb40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fce9400afd0 tx=0x7fce94022000).ready entity=mon.0 client_cookie=18afee0129a5ebe9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fce7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T20:44:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fce7b7fe700 1 -- 172.21.15.33:0/3026894743 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940139c0 con 0x5605347aabe0 2023-11-06T20:44:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fce7b7fe700 1 -- 172.21.15.33:0/3026894743 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fce94031c90 con 0x5605347aabe0 2023-11-06T20:44:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fcebac81b80 1 -- 172.21.15.33:0/3026894743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560534755e20 con 0x5605347aabe0 2023-11-06T20:44:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fce7b7fe700 1 -- 172.21.15.33:0/3026894743 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce94031e50 con 0x5605347aabe0 2023-11-06T20:44:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.653+0000 7fcebac81b80 10 client.4617 fetch_fsmap learned FSMap version 7 2023-11-06T20:44:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.654+0000 7fcebac81b80 1 -- 172.21.15.33:0/3026894743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5605346a7ac0 con 0x5605347aabe0 2023-11-06T20:44:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.654+0000 7fce7b7fe700 1 -- 172.21.15.33:0/3026894743 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560534755e20 con 0x5605347aabe0 2023-11-06T20:44:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.654+0000 7fce7b7fe700 1 -- 172.21.15.33:0/3026894743 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fce9403a470 con 0x5605347aabe0 2023-11-06T20:44:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.654+0000 7fcebac81b80 10 client.4617 fetch_fsmap finished waiting for FSMap version 7 2023-11-06T20:44:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.654+0000 7fcebac81b80 10 client.4617 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T20:44:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.654+0000 7fcebac81b80 20 client.4617 populate_metadata read hostname 'smithi033' 2023-11-06T20:44:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.654+0000 7fcebac81b80 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.69:6836/2681789033,v1:172.21.15.69:6837/2681789033] conn(0x5605347a96b0 0x5605347a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.654+0000 7fcebac81b80 4 client.4617 mds_command: new command op to 4257 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T20:44:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.654+0000 7fcebac81b80 1 -- 172.21.15.33:0/3026894743 --> [v2:172.21.15.69:6836/2681789033,v1:172.21.15.69:6837/2681789033] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56053451bef0 con 0x5605347a96b0 2023-11-06T20:44:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.654+0000 7fcea179b700 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.69:6836/2681789033,v1:172.21.15.69:6837/2681789033] conn(0x5605347a96b0 0x5605347a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.655+0000 7fcea179b700 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.69:6836/2681789033,v1:172.21.15.69:6837/2681789033] conn(0x5605347a96b0 0x5605347a9a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9c4021cc7bd7d66e server_cookie=df14ed8ab19e407f in_seq=0 out_seq=0 2023-11-06T20:44:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.655+0000 7fce7b7fe700 10 client.4617 ms_handle_connect on v2:172.21.15.69:6836/2681789033 2023-11-06T20:44:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fce7b7fe700 1 -- 172.21.15.33:0/3026894743 <== mds.0 v2:172.21.15.69:6836/2681789033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x56053451bef0 con 0x5605347a96b0 2023-11-06T20:44:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fce7b7fe700 10 client.4617 handle_command_reply: tid=0 2023-11-06T20:44:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 client.4617 shutdown 2023-11-06T20:44:25.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 -- 172.21.15.33:0/3026894743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605347aabe0 msgr2=0x56053478cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605347aabe0 0x56053478cb40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fce9400afd0 tx=0x7fce94022000).stop 2023-11-06T20:44:25.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 -- 172.21.15.33:0/3026894743 shutdown_connections 2023-11-06T20:44:25.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.69:6836/2681789033,v1:172.21.15.69:6837/2681789033] conn(0x5605347a96b0 0x5605347a9a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605347aabe0 0x56053478cb40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560534762080 0x560534783710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 --2- 172.21.15.33:0/3026894743 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x560534783c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fce7b7fe700 0 client.4617 ms_handle_reset on v2:172.21.15.69:6836/2681789033 2023-11-06T20:44:25.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 -- 172.21.15.33:0/3026894743 >> 172.21.15.33:0/3026894743 conn(0x5605345cb940 msgr2=0x560534760d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:25.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 -- 172.21.15.33:0/3026894743 shutdown_connections 2023-11-06T20:44:25.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.656+0000 7fcebac81b80 1 -- 172.21.15.33:0/3026894743 wait complete. 2023-11-06T20:44:25.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.657+0000 7fcebac81b80 20 client.4617 trim_cache size 0 max 16384 2023-11-06T20:44:25.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.673+0000 7fcebac81b80 1 Processor -- start 2023-11-06T20:44:25.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.674+0000 7fcebac81b80 1 -- start start 2023-11-06T20:44:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.674+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x560534760e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.674+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x5605347a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.674+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605347aabe0 0x5605347a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.674+0000 7fcebac81b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560534763380 con 0x560534762080 2023-11-06T20:44:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.674+0000 7fcebac81b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560534761bd0 con 0x5605347aabe0 2023-11-06T20:44:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.674+0000 7fcebac81b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605347646b0 con 0x56053478a4c0 2023-11-06T20:44:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x560534760e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x560534760e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33852/0 (socket says 172.21.15.33:33852) 2023-11-06T20:44:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea179b700 1 -- 172.21.15.33:0/575850609 learned_addr learned my addr 172.21.15.33:0/575850609 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea0f9a700 1 --2- 172.21.15.33:0/575850609 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x5605347a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/575850609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605347aabe0 0x5605347a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea179b700 1 -- 172.21.15.33:0/575850609 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 msgr2=0x5605347a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea179b700 1 --2- 172.21.15.33:0/575850609 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x5605347a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea179b700 1 -- 172.21.15.33:0/575850609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605347aabe0 msgr2=0x5605347a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea179b700 1 --2- 172.21.15.33:0/575850609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605347aabe0 0x5605347a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea179b700 1 -- 172.21.15.33:0/575850609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605346a7810 con 0x560534762080 2023-11-06T20:44:25.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fcea179b700 1 --2- 172.21.15.33:0/575850609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x560534760e70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fce98014ed0 tx=0x7fce98019000).ready entity=mon.0 client_cookie=6b1d42686f7261d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:25.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fce86ffd700 1 -- 172.21.15.33:0/575850609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98025070 con 0x560534762080 2023-11-06T20:44:25.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fce86ffd700 1 -- 172.21.15.33:0/575850609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce98026040 con 0x560534762080 2023-11-06T20:44:25.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.675+0000 7fce86ffd700 1 -- 172.21.15.33:0/575850609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98029280 con 0x560534762080 2023-11-06T20:44:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.676+0000 7fcebac81b80 1 -- 172.21.15.33:0/575850609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 msgr2=0x560534760e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.676+0000 7fcebac81b80 1 --2- 172.21.15.33:0/575850609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x560534760e70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fce98014ed0 tx=0x7fce98019000).stop 2023-11-06T20:44:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.676+0000 7fcebac81b80 1 -- 172.21.15.33:0/575850609 shutdown_connections 2023-11-06T20:44:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.676+0000 7fcebac81b80 1 --2- 172.21.15.33:0/575850609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605347aabe0 0x5605347a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.676+0000 7fcebac81b80 1 --2- 172.21.15.33:0/575850609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x560534760e70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.676+0000 7fcebac81b80 1 --2- 172.21.15.33:0/575850609 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x5605347a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.676+0000 7fcebac81b80 1 -- 172.21.15.33:0/575850609 >> 172.21.15.33:0/575850609 conn(0x5605345cb940 msgr2=0x56053475fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.676+0000 7fcebac81b80 1 -- 172.21.15.33:0/575850609 shutdown_connections 2023-11-06T20:44:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.676+0000 7fcebac81b80 1 -- 172.21.15.33:0/575850609 wait complete. 2023-11-06T20:44:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.677+0000 7fcebac81b80 1 -- start start 2023-11-06T20:44:25.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.677+0000 7fcebac81b80 1 Processor -- start 2023-11-06T20:44:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x5605347600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x5605348061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcebac81b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605347aabe0 0x560534783fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcebac81b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605347874d0 con 0x560534762080 2023-11-06T20:44:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcebac81b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560534790a00 con 0x5605347aabe0 2023-11-06T20:44:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcebac81b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605348066d0 con 0x56053478a4c0 2023-11-06T20:44:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcebac81b80 10 client.0 ll_register_callbacks cb 0x560534653ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T20:44:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcea179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x5605347600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcea179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x5605347600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33854/0 (socket says 172.21.15.33:33854) 2023-11-06T20:44:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcea0f9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x5605348061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcea179b700 1 -- 172.21.15.33:0/797095662 learned_addr learned my addr 172.21.15.33:0/797095662 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.678+0000 7fcea1f9c700 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605347aabe0 0x560534783fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fcea179b700 1 -- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 msgr2=0x5605348061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fcea179b700 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x5605348061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fcea179b700 1 -- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605347aabe0 msgr2=0x560534783fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fcea179b700 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605347aabe0 0x560534783fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fcea179b700 1 -- 172.21.15.33:0/797095662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce98014d40 con 0x560534762080 2023-11-06T20:44:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fcea179b700 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x5605347600f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fce98009690 tx=0x7fce9802a000).ready entity=mon.0 client_cookie=f2cbfc78912c6f84 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:25.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fce7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T20:44:25.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fce7b7fe700 1 -- 172.21.15.33:0/797095662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98037070 con 0x560534762080 2023-11-06T20:44:25.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fcebac81b80 1 -- 172.21.15.33:0/797095662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce94009ac0 con 0x560534762080 2023-11-06T20:44:25.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fce7b7fe700 1 -- 172.21.15.33:0/797095662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fce9802a8b0 con 0x560534762080 2023-11-06T20:44:25.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fce7b7fe700 1 -- 172.21.15.33:0/797095662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce98014860 con 0x560534762080 2023-11-06T20:44:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fcebac81b80 10 client.4623 fetch_fsmap learned FSMap version 7 2023-11-06T20:44:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.679+0000 7fcebac81b80 1 -- 172.21.15.33:0/797095662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5605345211d0 con 0x560534762080 2023-11-06T20:44:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.680+0000 7fce7b7fe700 1 -- 172.21.15.33:0/797095662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce94009ac0 con 0x560534762080 2023-11-06T20:44:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.680+0000 7fce7b7fe700 1 -- 172.21.15.33:0/797095662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fce98033460 con 0x560534762080 2023-11-06T20:44:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.680+0000 7fcebac81b80 10 client.4623 fetch_fsmap finished waiting for FSMap version 7 2023-11-06T20:44:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.680+0000 7fcebac81b80 10 client.4623 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T20:44:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.680+0000 7fcebac81b80 20 client.4623 populate_metadata read hostname 'smithi033' 2023-11-06T20:44:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.680+0000 7fcebac81b80 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:6836/2681789033,v1:172.21.15.69:6837/2681789033] conn(0x56053475bf60 0x56053475c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.680+0000 7fcebac81b80 4 client.4623 mds_command: new command op to 4257 tid=0[{"prefix": "status"}] 2023-11-06T20:44:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.680+0000 7fcebac81b80 1 -- 172.21.15.33:0/797095662 --> [v2:172.21.15.69:6836/2681789033,v1:172.21.15.69:6837/2681789033] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560534551040 con 0x56053475bf60 2023-11-06T20:44:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.680+0000 7fcea0f9a700 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:6836/2681789033,v1:172.21.15.69:6837/2681789033] conn(0x56053475bf60 0x56053475c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcea0f9a700 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:6836/2681789033,v1:172.21.15.69:6837/2681789033] conn(0x56053475bf60 0x56053475c340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=668cfcf2ca32e708 server_cookie=6ddc5727824c20da in_seq=0 out_seq=0 2023-11-06T20:44:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fce7b7fe700 10 client.4623 ms_handle_connect on v2:172.21.15.69:6836/2681789033 2023-11-06T20:44:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fce7b7fe700 1 -- 172.21.15.33:0/797095662 <== mds.0 v2:172.21.15.69:6836/2681789033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x560534551040 con 0x56053475bf60 2023-11-06T20:44:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fce7b7fe700 10 client.4623 handle_command_reply: tid=0 2023-11-06T20:44:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 client.4623 shutdown 2023-11-06T20:44:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 -- 172.21.15.33:0/797095662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 msgr2=0x5605347600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x5605347600f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fce98009690 tx=0x7fce9802a000).stop 2023-11-06T20:44:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 -- 172.21.15.33:0/797095662 shutdown_connections 2023-11-06T20:44:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:6836/2681789033,v1:172.21.15.69:6837/2681789033] conn(0x56053475bf60 0x56053475c340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605347aabe0 0x560534783fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560534762080 0x5605347600f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 --2- 172.21.15.33:0/797095662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56053478a4c0 0x5605348061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fce7b7fe700 0 client.4623 ms_handle_reset on v2:172.21.15.69:6836/2681789033 2023-11-06T20:44:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 -- 172.21.15.33:0/797095662 >> 172.21.15.33:0/797095662 conn(0x5605345cb940 msgr2=0x560534762460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 -- 172.21.15.33:0/797095662 shutdown_connections 2023-11-06T20:44:25.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 1 -- 172.21.15.33:0/797095662 wait complete. 2023-11-06T20:44:25.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.681+0000 7fcebac81b80 20 client.4623 trim_cache size 0 max 16384 2023-11-06T20:44:25.700 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T20:44:25.700 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T20:44:25.701 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T20:44:25.701 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4257, 2023-11-06T20:44:25.701 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T20:44:25.701 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T20:44:25.701 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T20:44:25.701 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 19.296002573999999, 2023-11-06T20:44:25.702 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 7, 2023-11-06T20:44:25.702 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 21, 2023-11-06T20:44:25.702 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 21, 2023-11-06T20:44:25.702 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 24.621142260999999 2023-11-06T20:44:25.702 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T20:44:25.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 -- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce8005f150 msgr2=0x7fce80061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce8005f150 0x7fce80061610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fce98006d80 tx=0x7fce98009060).stop 2023-11-06T20:44:25.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 -- 172.21.15.33:0/2299598246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce9c0af320 msgr2=0x7fce9c0add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:25.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce9c0af320 0x7fce9c0add60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fce8c0049e0 tx=0x7fce8c001e90).stop 2023-11-06T20:44:25.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 -- 172.21.15.33:0/2299598246 shutdown_connections 2023-11-06T20:44:25.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce8005f150 0x7fce80061610 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce9c142940 0x7fce9c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce88004b80 0x7fce9c0c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 --2- 172.21.15.33:0/2299598246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce9c0af320 0x7fce9c0add60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:25.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 -- 172.21.15.33:0/2299598246 >> 172.21.15.33:0/2299598246 conn(0x7fce9c0c8320 msgr2=0x7fce9c153e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:25.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 -- 172.21.15.33:0/2299598246 shutdown_connections 2023-11-06T20:44:25.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:25.682+0000 7fcea39ff700 1 -- 172.21.15.33:0/2299598246 wait complete. 2023-11-06T20:44:25.706 INFO:teuthology.run_tasks:Running task kclient... 2023-11-06T20:44:25.728 INFO:tasks.kclient:Mounting kernel clients... 2023-11-06T20:44:25.728 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-06T20:44:25.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:25.729 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T20:44:25.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:25.778 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2023-11-06T20:44:25.853 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2023-11-06T20:44:25.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T20:44:25.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:25.857 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T20:44:25.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:25.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2023-11-06T20:44:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2023-11-06T20:44:25.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T20:44:25.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:25.983 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2023-11-06T20:44:26.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:26.037 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2023-11-06T20:44:26.114 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2023-11-06T20:44:26.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T20:44:26.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:26.117 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2023-11-06T20:44:26.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:26.165 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2023-11-06T20:44:26.238 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2023-11-06T20:44:26.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T20:44:26.241 INFO:tasks.kclient:client.0 config is {} 2023-11-06T20:44:26.242 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T20:44:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.406+0000 7fb54e3ff700 1 Processor -- start 2023-11-06T20:44:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.406+0000 7fb54e3ff700 1 -- start start 2023-11-06T20:44:26.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.408+0000 7fb54e3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5480a9ee0 0x7fb5480c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:26.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.408+0000 7fb54e3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5480c5f80 con 0x7fb5480c07b0 2023-11-06T20:44:26.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.408+0000 7fb54e3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5480aa740 con 0x7fb5481541a0 2023-11-06T20:44:26.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.408+0000 7fb54e3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480c6800 con 0x7fb5480a9ee0 2023-11-06T20:44:26.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.408+0000 7fb5477fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5480a9ee0 0x7fb5480c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:26.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.408+0000 7fb54c99c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb5480c07b0 0x7fb5480a7c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51722/0 (socket says 172.21.15.33:51722) 2023-11-06T20:44:26.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.408+0000 7fb5477fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5480a9ee0 0x7fb5480c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33858/0 (socket says 172.21.15.33:33858) 2023-11-06T20:44:26.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.408+0000 7fb54c99c700 1 -- 172.21.15.33:0/1493897751 learned_addr learned my addr 172.21.15.33:0/1493897751 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:26.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.408+0000 7fb5477fe700 1 -- 172.21.15.33:0/1493897751 >> v1:172.21.15.69:6789/0 conn(0x7fb5481541a0 legacy=0x7fb5480c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:26.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.408+0000 7fb5477fe700 1 -- 172.21.15.33:0/1493897751 >> v1:172.21.15.33:6789/0 conn(0x7fb5480c07b0 legacy=0x7fb5480a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:26.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.409+0000 7fb5477fe700 1 -- 172.21.15.33:0/1493897751 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480aa460 con 0x7fb5480a9ee0 2023-11-06T20:44:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.409+0000 7fb546ffd700 1 -- 172.21.15.33:0/1493897751 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205425166 0 0) 0x7fb5480c5f80 con 0x7fb5480c07b0 2023-11-06T20:44:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.409+0000 7fb5477fe700 1 --2- 172.21.15.33:0/1493897751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480a9ee0 0x7fb5480c6c90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fb53c009760 tx=0x7fb53c009060).ready entity=mon.0 client_cookie=496612fc733ac96 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.409+0000 7fb546ffd700 1 -- 172.21.15.33:0/1493897751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00b580 con 0x7fb5480a9ee0 2023-11-06T20:44:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.409+0000 7fb546ffd700 1 -- 172.21.15.33:0/1493897751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c00b6e0 con 0x7fb5480a9ee0 2023-11-06T20:44:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.409+0000 7fb546ffd700 1 -- 172.21.15.33:0/1493897751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00ba60 con 0x7fb5480a9ee0 2023-11-06T20:44:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.409+0000 7fb54e3ff700 1 -- 172.21.15.33:0/1493897751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480a9ee0 msgr2=0x7fb5480c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.409+0000 7fb54e3ff700 1 --2- 172.21.15.33:0/1493897751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480a9ee0 0x7fb5480c6c90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fb53c009760 tx=0x7fb53c009060).stop 2023-11-06T20:44:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.410+0000 7fb54e3ff700 1 -- 172.21.15.33:0/1493897751 shutdown_connections 2023-11-06T20:44:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.410+0000 7fb54e3ff700 1 --2- 172.21.15.33:0/1493897751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480a9ee0 0x7fb5480c6c90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.410+0000 7fb54e3ff700 1 -- 172.21.15.33:0/1493897751 >> 172.21.15.33:0/1493897751 conn(0x7fb5480944e0 msgr2=0x7fb54806a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.410+0000 7fb54e3ff700 1 -- 172.21.15.33:0/1493897751 shutdown_connections 2023-11-06T20:44:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.410+0000 7fb54e3ff700 1 -- 172.21.15.33:0/1493897751 wait complete. 2023-11-06T20:44:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.411+0000 7fb54e3ff700 1 Processor -- start 2023-11-06T20:44:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.411+0000 7fb54e3ff700 1 -- start start 2023-11-06T20:44:26.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.412+0000 7fb54e3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5480a9ee0 0x7fb548163f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:26.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.412+0000 7fb54e3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480c07b0 0x7fb5480aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:26.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.412+0000 7fb54e3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb5481541a0 0x7fb5480ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.412+0000 7fb54e3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548164480 con 0x7fb5480c07b0 2023-11-06T20:44:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.412+0000 7fb54e3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480ab1d0 con 0x7fb5480a9ee0 2023-11-06T20:44:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.412+0000 7fb54e3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480ab4b0 con 0x7fb5481541a0 2023-11-06T20:44:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.412+0000 7fb5477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480c07b0 0x7fb5480aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:26.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb547fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5480a9ee0 0x7fb548163f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:26.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb5477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480c07b0 0x7fb5480aa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33864/0 (socket says 172.21.15.33:33864) 2023-11-06T20:44:26.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb5477fe700 1 -- 172.21.15.33:0/4111867055 learned_addr learned my addr 172.21.15.33:0/4111867055 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:26.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb54c99c700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb5481541a0 0x7fb5480ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:26.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb547fff700 1 -- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb5481541a0 msgr2=0x7fb5480ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:26.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb547fff700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb5481541a0 0x7fb5480ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:26.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb547fff700 1 -- 172.21.15.33:0/4111867055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480c07b0 msgr2=0x7fb5480aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:26.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb547fff700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480c07b0 0x7fb5480aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:26.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb547fff700 1 -- 172.21.15.33:0/4111867055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c0092a0 con 0x7fb5480a9ee0 2023-11-06T20:44:26.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb5477fe700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480c07b0 0x7fb5480aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:26.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb547fff700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5480a9ee0 0x7fb548163f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb53800c440 tx=0x7fb538007d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:26.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb52ffff700 1 -- 172.21.15.33:0/4111867055 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538007020 con 0x7fb5480a9ee0 2023-11-06T20:44:26.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb52ffff700 1 -- 172.21.15.33:0/4111867055 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb538007180 con 0x7fb5480a9ee0 2023-11-06T20:44:26.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb54e3ff700 1 -- 172.21.15.33:0/4111867055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5480ab790 con 0x7fb5480a9ee0 2023-11-06T20:44:26.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.413+0000 7fb52ffff700 1 -- 172.21.15.33:0/4111867055 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538007520 con 0x7fb5480a9ee0 2023-11-06T20:44:26.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.414+0000 7fb54e3ff700 1 -- 172.21.15.33:0/4111867055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5480abd50 con 0x7fb5480a9ee0 2023-11-06T20:44:26.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.415+0000 7fb54e3ff700 1 -- 172.21.15.33:0/4111867055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5480842b0 con 0x7fb5480a9ee0 2023-11-06T20:44:26.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.415+0000 7fb52ffff700 1 -- 172.21.15.33:0/4111867055 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb538006900 con 0x7fb5480a9ee0 2023-11-06T20:44:26.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.416+0000 7fb52ffff700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb53405f150 0x7fb534061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:26.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.416+0000 7fb52ffff700 1 -- 172.21.15.33:0/4111867055 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb5380793c0 con 0x7fb5480a9ee0 2023-11-06T20:44:26.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.416+0000 7fb5477fe700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb53405f150 0x7fb534061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:26.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.417+0000 7fb5477fe700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb53405f150 0x7fb534061610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb53c006e00 tx=0x7fb53c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:26.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.421+0000 7fb52ffff700 1 -- 172.21.15.33:0/4111867055 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb538047af0 con 0x7fb5480a9ee0 2023-11-06T20:44:26.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.649+0000 7fb54e3ff700 1 -- 172.21.15.33:0/4111867055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5480475a0 con 0x7fb5480a9ee0 2023-11-06T20:44:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.650+0000 7fb52ffff700 1 -- 172.21.15.33:0/4111867055 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb53804a690 con 0x7fb5480a9ee0 2023-11-06T20:44:26.651 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T20:44:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 -- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb53405f150 msgr2=0x7fb534061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb53405f150 0x7fb534061610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb53c006e00 tx=0x7fb53c017000).stop 2023-11-06T20:44:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 -- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5480a9ee0 msgr2=0x7fb548163f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5480a9ee0 0x7fb548163f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb53800c440 tx=0x7fb538007d90).stop 2023-11-06T20:44:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 -- 172.21.15.33:0/4111867055 shutdown_connections 2023-11-06T20:44:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb53405f150 0x7fb534061610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb5481541a0 0x7fb5480ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5480a9ee0 0x7fb548163f90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 --2- 172.21.15.33:0/4111867055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5480c07b0 0x7fb5480aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 -- 172.21.15.33:0/4111867055 >> 172.21.15.33:0/4111867055 conn(0x7fb5480944e0 msgr2=0x7fb5480c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 -- 172.21.15.33:0/4111867055 shutdown_connections 2023-11-06T20:44:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:26.653+0000 7fb54e3ff700 1 -- 172.21.15.33:0/4111867055 wait complete. 2023-11-06T20:44:26.667 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T20:44:26.668 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T20:44:26.668 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T20:44:26.668 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T20:44:26.668 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T20:44:26.669 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T20:44:26.669 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T20:44:26.669 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T20:44:26.669 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T20:44:26.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:26.670 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T20:44:26.717 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:44:26.717 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:44:26.718 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:44:26.718 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:26.718 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T20:44:26.718 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:26.718 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:26.719 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:26.719 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T20:44:26.719 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:26.719 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:26.719 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T20:44:26.719 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T20:44:26.720 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:26.720 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T20:44:26.720 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T20:44:26.720 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42277sec preferred_lft 42277sec 2023-11-06T20:44:26.720 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T20:44:26.721 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:26.721 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:26.721 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:26.721 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T20:44:26.722 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-06T20:44:26.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:26.723 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:44:26.723 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:44:26.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2023-11-06T20:44:26.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2023-11-06T20:44:26.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2023-11-06T20:44:26.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-06T20:44:26.725 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:44:27.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:27.158 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T20:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T20:44:27.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:27.237 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T20:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T20:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T20:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T20:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T20:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T20:44:27.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:27.311 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:44:27.312 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:44:27.312 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T20:44:27.312 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T20:44:27.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T20:44:27.313 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:44:28.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:28.015 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T20:44:28.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:28.064 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2023-11-06T20:44:28.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:28.113 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:44:28.113 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:44:28.114 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T20:44:28.114 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-06T20:44:28.114 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:44:28.261 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-06T20:44:28.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:28.262 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:44:28.262 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:44:28.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-06T20:44:28.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-06T20:44:28.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-06T20:44:28.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-06T20:44:28.264 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-06T20:44:28.264 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:44:28.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:28.532 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:44:28.532 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:44:28.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T20:44:28.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2023-11-06T20:44:28.533 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:44:28.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:28.708 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T20:44:28.755 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T20:44:28.756 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:28.858 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:28.858 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:44:29.310 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:29.311 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:44:29.315 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T20:44:29.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:29.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T20:44:29.388 INFO:tasks.kclient:client.2 config is {} 2023-11-06T20:44:29.389 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T20:44:29.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.553+0000 7f8d427ff700 1 Processor -- start 2023-11-06T20:44:29.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.553+0000 7f8d427ff700 1 -- start start 2023-11-06T20:44:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.555+0000 7f8d427ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 0x7f8d3c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.555+0000 7f8d427ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3c0c4d10 con 0x7f8d3c13b860 2023-11-06T20:44:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.555+0000 7f8d427ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3c0c4a10 con 0x7f8d3c155200 2023-11-06T20:44:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.555+0000 7f8d427ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c069ab0 con 0x7f8d3c0c61b0 2023-11-06T20:44:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.555+0000 7f8d3b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8d3c13b860 0x7f8d3c0683b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37904/0 (socket says 172.21.15.33:37904) 2023-11-06T20:44:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.555+0000 7f8d3b7fe700 1 -- 172.21.15.33:0/1848726589 learned_addr learned my addr 172.21.15.33:0/1848726589 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.555+0000 7f8d40d9c700 1 --2- 172.21.15.33:0/1848726589 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 0x7f8d3c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.555+0000 7f8d3affd700 1 -- 172.21.15.33:0/1848726589 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734702947 0 0) 0x7f8d3c0c4d10 con 0x7f8d3c13b860 2023-11-06T20:44:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.556+0000 7f8d3affd700 1 -- 172.21.15.33:0/1848726589 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d240033f0 con 0x7f8d3c13b860 2023-11-06T20:44:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.556+0000 7f8d3affd700 1 -- 172.21.15.33:0/1848726589 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099827786 0 0) 0x7f8d3c0c4a10 con 0x7f8d3c155200 2023-11-06T20:44:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.556+0000 7f8d3affd700 1 -- 172.21.15.33:0/1848726589 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d3c0c4d10 con 0x7f8d3c155200 2023-11-06T20:44:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.556+0000 7f8d40d9c700 1 -- 172.21.15.33:0/1848726589 >> v1:172.21.15.69:6789/0 conn(0x7f8d3c155200 legacy=0x7f8d3c0bc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.556+0000 7f8d40d9c700 1 -- 172.21.15.33:0/1848726589 >> v1:172.21.15.33:6789/0 conn(0x7f8d3c13b860 legacy=0x7f8d3c0683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.556+0000 7f8d40d9c700 1 -- 172.21.15.33:0/1848726589 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c0bcef0 con 0x7f8d3c0c61b0 2023-11-06T20:44:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.556+0000 7f8d40d9c700 1 --2- 172.21.15.33:0/1848726589 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 0x7f8d3c15ccb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f8d34009e90 tx=0x7f8d34009780).ready entity=mon.2 client_cookie=1bbcb5b19589cbc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.556+0000 7f8d3affd700 1 -- 172.21.15.33:0/1848726589 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400e070 con 0x7f8d3c0c61b0 2023-11-06T20:44:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.556+0000 7f8d3affd700 1 -- 172.21.15.33:0/1848726589 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3400bab0 con 0x7f8d3c0c61b0 2023-11-06T20:44:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.556+0000 7f8d3affd700 1 -- 172.21.15.33:0/1848726589 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400be30 con 0x7f8d3c0c61b0 2023-11-06T20:44:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.557+0000 7f8d427ff700 1 -- 172.21.15.33:0/1848726589 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 msgr2=0x7f8d3c15ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.557+0000 7f8d427ff700 1 --2- 172.21.15.33:0/1848726589 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 0x7f8d3c15ccb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f8d34009e90 tx=0x7f8d34009780).stop 2023-11-06T20:44:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.557+0000 7f8d427ff700 1 -- 172.21.15.33:0/1848726589 shutdown_connections 2023-11-06T20:44:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.557+0000 7f8d427ff700 1 --2- 172.21.15.33:0/1848726589 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 0x7f8d3c15ccb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.557+0000 7f8d427ff700 1 -- 172.21.15.33:0/1848726589 >> 172.21.15.33:0/1848726589 conn(0x7f8d3c0c3510 msgr2=0x7f8d3c0c3920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.557+0000 7f8d427ff700 1 -- 172.21.15.33:0/1848726589 shutdown_connections 2023-11-06T20:44:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.557+0000 7f8d427ff700 1 -- 172.21.15.33:0/1848726589 wait complete. 2023-11-06T20:44:29.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.559+0000 7f8d427ff700 1 Processor -- start 2023-11-06T20:44:29.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.559+0000 7f8d427ff700 1 -- start start 2023-11-06T20:44:29.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.560+0000 7f8d427ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 0x7f8d3c135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:29.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.560+0000 7f8d427ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d3c13b860 0x7f8d3c0643f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:29.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.560+0000 7f8d427ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c155200 0x7f8d3c0648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.560+0000 7f8d427ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c064dd0 con 0x7f8d3c155200 2023-11-06T20:44:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d427ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0c2b00 con 0x7f8d3c13b860 2023-11-06T20:44:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d427ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0c1ef0 con 0x7f8d3c0c61b0 2023-11-06T20:44:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d40d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c155200 0x7f8d3c0648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d40d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c155200 0x7f8d3c0648e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43906/0 (socket says 172.21.15.33:43906) 2023-11-06T20:44:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d40d9c700 1 -- 172.21.15.33:0/3431227767 learned_addr learned my addr 172.21.15.33:0/3431227767 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d3b7fe700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d3c13b860 0x7f8d3c0643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d3bfff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 0x7f8d3c135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d40d9c700 1 -- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 msgr2=0x7f8d3c135560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d40d9c700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 0x7f8d3c135560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d40d9c700 1 -- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d3c13b860 msgr2=0x7f8d3c0643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d40d9c700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d3c13b860 0x7f8d3c0643f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d40d9c700 1 -- 172.21.15.33:0/3431227767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d34009a20 con 0x7f8d3c155200 2023-11-06T20:44:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d3bfff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 0x7f8d3c135560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.561+0000 7f8d40d9c700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c155200 0x7f8d3c0648e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f8d3c067ae0 tx=0x7f8d3c0c4c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.562+0000 7f8d23fff700 1 -- 172.21.15.33:0/3431227767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3401d630 con 0x7f8d3c155200 2023-11-06T20:44:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.562+0000 7f8d23fff700 1 -- 172.21.15.33:0/3431227767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3401d790 con 0x7f8d3c155200 2023-11-06T20:44:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.562+0000 7f8d23fff700 1 -- 172.21.15.33:0/3431227767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3401db50 con 0x7f8d3c155200 2023-11-06T20:44:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.562+0000 7f8d427ff700 1 -- 172.21.15.33:0/3431227767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c0c21d0 con 0x7f8d3c155200 2023-11-06T20:44:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.562+0000 7f8d427ff700 1 -- 172.21.15.33:0/3431227767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c0c2790 con 0x7f8d3c155200 2023-11-06T20:44:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.563+0000 7f8d427ff700 1 -- 172.21.15.33:0/3431227767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3c0c2c90 con 0x7f8d3c155200 2023-11-06T20:44:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.564+0000 7f8d23fff700 1 -- 172.21.15.33:0/3431227767 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8d34003040 con 0x7f8d3c155200 2023-11-06T20:44:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.564+0000 7f8d23fff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d2805f120 0x7f8d280615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.564+0000 7f8d23fff700 1 -- 172.21.15.33:0/3431227767 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8d3407c120 con 0x7f8d3c155200 2023-11-06T20:44:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.565+0000 7f8d3bfff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d2805f120 0x7f8d280615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.565+0000 7f8d3bfff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d2805f120 0x7f8d280615e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f8d3c1383e0 tx=0x7f8d2c009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.569+0000 7f8d23fff700 1 -- 172.21.15.33:0/3431227767 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8d3404d330 con 0x7f8d3c155200 2023-11-06T20:44:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.797+0000 7f8d427ff700 1 -- 172.21.15.33:0/3431227767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8d3c0c6610 con 0x7f8d3c155200 2023-11-06T20:44:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.797+0000 7f8d23fff700 1 -- 172.21.15.33:0/3431227767 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8d34050950 con 0x7f8d3c155200 2023-11-06T20:44:29.798 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T20:44:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.799+0000 7f8d427ff700 1 -- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d2805f120 msgr2=0x7f8d280615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.799+0000 7f8d427ff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d2805f120 0x7f8d280615e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f8d3c1383e0 tx=0x7f8d2c009040).stop 2023-11-06T20:44:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.800+0000 7f8d427ff700 1 -- 172.21.15.33:0/3431227767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c155200 msgr2=0x7f8d3c0648e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.800+0000 7f8d427ff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c155200 0x7f8d3c0648e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f8d3c067ae0 tx=0x7f8d3c0c4c80).stop 2023-11-06T20:44:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.800+0000 7f8d427ff700 1 -- 172.21.15.33:0/3431227767 shutdown_connections 2023-11-06T20:44:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.800+0000 7f8d427ff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d2805f120 0x7f8d280615e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.800+0000 7f8d427ff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d3c0c61b0 0x7f8d3c135560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.800+0000 7f8d427ff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d3c155200 0x7f8d3c0648e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.800+0000 7f8d427ff700 1 --2- 172.21.15.33:0/3431227767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d3c13b860 0x7f8d3c0643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.800+0000 7f8d427ff700 1 -- 172.21.15.33:0/3431227767 >> 172.21.15.33:0/3431227767 conn(0x7f8d3c0c3510 msgr2=0x7f8d3c0683b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.800+0000 7f8d427ff700 1 -- 172.21.15.33:0/3431227767 shutdown_connections 2023-11-06T20:44:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:29.800+0000 7f8d427ff700 1 -- 172.21.15.33:0/3431227767 wait complete. 2023-11-06T20:44:29.815 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T20:44:29.815 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T20:44:29.815 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T20:44:29.815 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-06T20:44:29.816 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-06T20:44:29.816 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T20:44:29.816 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T20:44:29.816 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T20:44:29.817 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-06T20:44:29.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:29.817 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T20:44:29.866 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:44:29.866 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:44:29.867 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:44:29.867 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:29.867 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T20:44:29.867 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:29.868 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:29.868 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:29.868 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T20:44:29.868 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:29.868 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:29.869 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T20:44:29.869 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T20:44:29.869 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:29.869 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T20:44:29.869 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T20:44:29.870 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42274sec preferred_lft 42274sec 2023-11-06T20:44:29.870 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T20:44:29.870 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:29.870 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:29.870 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:29.871 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T20:44:29.871 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T20:44:29.871 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:29.871 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T20:44:29.871 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:29.872 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T20:44:29.872 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:29.872 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T20:44:29.872 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T20:44:29.872 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link tentative 2023-11-06T20:44:29.873 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:29.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:29.873 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T20:44:29.946 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T20:44:29.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:29.949 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T20:44:30.000 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T20:44:30.000 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T20:44:30.001 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T20:44:30.001 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T20:44:30.001 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T20:44:30.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:30.003 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:44:30.003 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:44:30.003 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T20:44:30.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T20:44:30.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T20:44:30.004 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:44:30.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:30.178 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T20:44:30.225 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T20:44:30.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:30.227 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2023-11-06T20:44:30.274 INFO:teuthology.orchestra.run.smithi033.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-06T20:44:30.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:30.276 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:44:30.276 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:44:30.276 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T20:44:30.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-06T20:44:30.277 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:44:30.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:30.427 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-06T20:44:30.500 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:44:30.500 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:44:30.501 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:44:30.501 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:30.501 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T20:44:30.501 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:30.501 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T20:44:30.502 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether be:a8:6e:a0:73:f2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T20:44:30.502 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T20:44:30.502 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:30.502 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::bca8:6eff:fea0:73f2/64 scope link tentative 2023-11-06T20:44:30.502 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:30.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:30.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-06T20:44:30.577 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:44:30.577 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:44:30.578 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:44:30.578 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:30.578 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T20:44:30.578 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:30.578 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T20:44:30.579 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether be:a8:6e:a0:73:f2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T20:44:30.579 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T20:44:30.579 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:30.579 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::bca8:6eff:fea0:73f2/64 scope link 2023-11-06T20:44:30.579 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:30.581 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-06T20:44:30.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:30.581 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:44:30.582 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:44:30.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-06T20:44:30.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-06T20:44:30.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-06T20:44:30.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-06T20:44:30.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-06T20:44:30.584 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:44:30.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:30.840 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:44:30.840 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:44:30.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.1 up 2023-11-06T20:44:30.841 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.1 master ceph-brx 2023-11-06T20:44:30.841 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:44:31.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:31.018 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.2 2023-11-06T20:44:31.065 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T20:44:31.066 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:31.165 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:31.166 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=2,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:44:31.194 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:31.194 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=2,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:44:31.198 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T20:44:31.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:31.198 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-06T20:44:31.270 INFO:tasks.kclient:client.1 config is {} 2023-11-06T20:44:31.271 DEBUG:teuthology.orchestra.run.smithi069:> ceph fs ls 2023-11-06T20:44:31.440 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.440+0000 7f03517ff700 1 Processor -- start 2023-11-06T20:44:31.440 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.440+0000 7f03517ff700 1 -- start start 2023-11-06T20:44:31.441 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f03517ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f034c1541a0 0x7f034c0c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:31.441 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f03517ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0a9ee0 0x7f034c0c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:31.442 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f03517ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c0c5f80 con 0x7f034c0c07b0 2023-11-06T20:44:31.442 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f03517ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c0aa740 con 0x7f034c1541a0 2023-11-06T20:44:31.442 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f03517ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0c6800 con 0x7f034c0a9ee0 2023-11-06T20:44:31.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f0343fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0a9ee0 0x7f034c0c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:31.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f034b7fe700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f034c0c07b0 0x7f034c0a7c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44042/0 (socket says 172.21.15.69:44042) 2023-11-06T20:44:31.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f034b7fe700 1 -- 172.21.15.69:0/2927723379 learned_addr learned my addr 172.21.15.69:0/2927723379 (peer_addr_for_me v1:172.21.15.69:0/0) 2023-11-06T20:44:31.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f034affd700 1 --2- 172.21.15.69:0/2927723379 >> v2:172.21.15.33:3300/0 conn(0x7f034c1541a0 0x7f034c0c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f034a7fc700 1 -- 172.21.15.69:0/2927723379 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225986159 0 0) 0x7f034c0c5f80 con 0x7f034c0c07b0 2023-11-06T20:44:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f0343fff700 1 -- 172.21.15.69:0/2927723379 >> v2:172.21.15.33:3300/0 conn(0x7f034c1541a0 msgr2=0x7f034c0c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f0343fff700 1 --2- 172.21.15.69:0/2927723379 >> v2:172.21.15.33:3300/0 conn(0x7f034c1541a0 0x7f034c0c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f0343fff700 1 -- 172.21.15.69:0/2927723379 >> v1:172.21.15.69:6789/0 conn(0x7f034c0c07b0 legacy=0x7f034c0a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.442+0000 7f0343fff700 1 -- 172.21.15.69:0/2927723379 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c0aa460 con 0x7f034c0a9ee0 2023-11-06T20:44:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f0343fff700 1 --2- 172.21.15.69:0/2927723379 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0a9ee0 0x7f034c0c6c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f0334009760 tx=0x7f0334009060).ready entity=mon.2 client_cookie=3e173d815731ceec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f034a7fc700 1 -- 172.21.15.69:0/2927723379 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033400b580 con 0x7f034c0a9ee0 2023-11-06T20:44:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f034a7fc700 1 -- 172.21.15.69:0/2927723379 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033400b6e0 con 0x7f034c0a9ee0 2023-11-06T20:44:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f034a7fc700 1 -- 172.21.15.69:0/2927723379 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033400ba60 con 0x7f034c0a9ee0 2023-11-06T20:44:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f03517ff700 1 -- 172.21.15.69:0/2927723379 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0a9ee0 msgr2=0x7f034c0c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f03517ff700 1 --2- 172.21.15.69:0/2927723379 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0a9ee0 0x7f034c0c6c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f0334009760 tx=0x7f0334009060).stop 2023-11-06T20:44:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f03517ff700 1 -- 172.21.15.69:0/2927723379 shutdown_connections 2023-11-06T20:44:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f03517ff700 1 --2- 172.21.15.69:0/2927723379 >> v2:172.21.15.33:3300/0 conn(0x7f034c1541a0 0x7f034c0c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f03517ff700 1 --2- 172.21.15.69:0/2927723379 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0a9ee0 0x7f034c0c6c90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f03517ff700 1 -- 172.21.15.69:0/2927723379 >> 172.21.15.69:0/2927723379 conn(0x7f034c0944e0 msgr2=0x7f034c06a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:31.448 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.443+0000 7f03517ff700 1 -- 172.21.15.69:0/2927723379 shutdown_connections 2023-11-06T20:44:31.448 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.444+0000 7f03517ff700 1 -- 172.21.15.69:0/2927723379 wait complete. 2023-11-06T20:44:31.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.445+0000 7f03517ff700 1 Processor -- start 2023-11-06T20:44:31.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.445+0000 7f03517ff700 1 -- start start 2023-11-06T20:44:31.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f03517ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f034c0a9ee0 0x7f034c1684d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:31.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f03517ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0c07b0 0x7f034c14ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:31.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f03517ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c1541a0 0x7f034c0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:31.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f03517ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0ade10 con 0x7f034c1541a0 2023-11-06T20:44:31.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f03517ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0aec30 con 0x7f034c0a9ee0 2023-11-06T20:44:31.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f03517ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c1689c0 con 0x7f034c0c07b0 2023-11-06T20:44:31.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f034affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f034c0a9ee0 0x7f034c1684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:31.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f0343fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0c07b0 0x7f034c14ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:31.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f034affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f034c0a9ee0 0x7f034c1684d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57440/0 (socket says 172.21.15.69:57440) 2023-11-06T20:44:31.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f034affd700 1 -- 172.21.15.69:0/3240019964 learned_addr learned my addr 172.21.15.69:0/3240019964 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:44:31.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.446+0000 7f034b7fe700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c1541a0 0x7f034c0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:31.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f0343fff700 1 -- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f034c0a9ee0 msgr2=0x7f034c1684d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:31.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f0343fff700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f034c0a9ee0 0x7f034c1684d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:31.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f0343fff700 1 -- 172.21.15.69:0/3240019964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c1541a0 msgr2=0x7f034c0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:31.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f0343fff700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c1541a0 0x7f034c0c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:31.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f0343fff700 1 -- 172.21.15.69:0/3240019964 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03340092a0 con 0x7f034c0c07b0 2023-11-06T20:44:31.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f034affd700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f034c0a9ee0 0x7f034c1684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:31.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f0343fff700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0c07b0 0x7f034c14ca70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f0334014000 tx=0x7f0334017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:31.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f0342ffd700 1 -- 172.21.15.69:0/3240019964 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334000bc0 con 0x7f034c0c07b0 2023-11-06T20:44:31.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f0342ffd700 1 -- 172.21.15.69:0/3240019964 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033400b850 con 0x7f034c0c07b0 2023-11-06T20:44:31.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f03517ff700 1 -- 172.21.15.69:0/3240019964 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c0ae020 con 0x7f034c0c07b0 2023-11-06T20:44:31.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f0342ffd700 1 -- 172.21.15.69:0/3240019964 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334000d20 con 0x7f034c0c07b0 2023-11-06T20:44:31.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.447+0000 7f03517ff700 1 -- 172.21.15.69:0/3240019964 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c0ae5e0 con 0x7f034c0c07b0 2023-11-06T20:44:31.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.448+0000 7f03517ff700 1 -- 172.21.15.69:0/3240019964 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c0aedc0 con 0x7f034c0c07b0 2023-11-06T20:44:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.449+0000 7f0342ffd700 1 -- 172.21.15.69:0/3240019964 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f033401bb00 con 0x7f034c0c07b0 2023-11-06T20:44:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.450+0000 7f0342ffd700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f032c05f150 0x7f032c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.450+0000 7f0342ffd700 1 -- 172.21.15.69:0/3240019964 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f033407bdd0 con 0x7f034c0c07b0 2023-11-06T20:44:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.450+0000 7f034affd700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f032c05f150 0x7f032c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.450+0000 7f034affd700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f032c05f150 0x7f032c061610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f033c006fd0 tx=0x7f033c008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:31.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.455+0000 7f0342ffd700 1 -- 172.21.15.69:0/3240019964 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f033407f030 con 0x7f034c0c07b0 2023-11-06T20:44:31.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.665+0000 7f03517ff700 1 -- 172.21.15.69:0/3240019964 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f034c0475a0 con 0x7f034c0c07b0 2023-11-06T20:44:31.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.665+0000 7f0342ffd700 1 -- 172.21.15.69:0/3240019964 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f033401f020 con 0x7f034c0c07b0 2023-11-06T20:44:31.665 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T20:44:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.667+0000 7f03517ff700 1 -- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f032c05f150 msgr2=0x7f032c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.667+0000 7f03517ff700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f032c05f150 0x7f032c061610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f033c006fd0 tx=0x7f033c008040).stop 2023-11-06T20:44:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.667+0000 7f03517ff700 1 -- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0c07b0 msgr2=0x7f034c14ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.667+0000 7f03517ff700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0c07b0 0x7f034c14ca70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f0334014000 tx=0x7f0334017000).stop 2023-11-06T20:44:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.667+0000 7f03517ff700 1 -- 172.21.15.69:0/3240019964 shutdown_connections 2023-11-06T20:44:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.668+0000 7f03517ff700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f032c05f150 0x7f032c061610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:31.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.668+0000 7f03517ff700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c1541a0 0x7f034c0c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:31.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.668+0000 7f03517ff700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f034c0a9ee0 0x7f034c1684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:31.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.668+0000 7f03517ff700 1 --2- 172.21.15.69:0/3240019964 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f034c0c07b0 0x7f034c14ca70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:31.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.668+0000 7f03517ff700 1 -- 172.21.15.69:0/3240019964 >> 172.21.15.69:0/3240019964 conn(0x7f034c0944e0 msgr2=0x7f034c0c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:31.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.668+0000 7f03517ff700 1 -- 172.21.15.69:0/3240019964 shutdown_connections 2023-11-06T20:44:31.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:31.668+0000 7f03517ff700 1 -- 172.21.15.69:0/3240019964 wait complete. 2023-11-06T20:44:31.682 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T20:44:31.682 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T20:44:31.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2023-11-06T20:44:31.683 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-06T20:44:31.683 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-06T20:44:31.683 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T20:44:31.683 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T20:44:31.683 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T20:44:31.684 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-06T20:44:31.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:31.684 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2023-11-06T20:44:31.731 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:44:31.731 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:44:31.731 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:44:31.732 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:31.732 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2023-11-06T20:44:31.732 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:31.732 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:31.733 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:31.733 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2023-11-06T20:44:31.733 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:31.733 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:31.733 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2023-11-06T20:44:31.734 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:31.734 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:31.734 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T20:44:31.734 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:31.734 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-06T20:44:31.735 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42259sec preferred_lft 42259sec 2023-11-06T20:44:31.735 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2023-11-06T20:44:31.735 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:31.736 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-06T20:44:31.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:31.737 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-11-06T20:44:31.737 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-11-06T20:44:31.737 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2023-11-06T20:44:31.737 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2023-11-06T20:44:31.738 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2023-11-06T20:44:31.738 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-06T20:44:31.738 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-11-06T20:44:32.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:32.157 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T20:44:32.228 INFO:teuthology.orchestra.run.smithi069.stdout:1 2023-11-06T20:44:32.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:32.232 DEBUG:teuthology.orchestra.run.smithi069:> route 2023-11-06T20:44:32.306 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2023-11-06T20:44:32.307 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T20:44:32.307 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T20:44:32.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T20:44:32.307 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T20:44:32.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:32.309 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-11-06T20:44:32.309 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-11-06T20:44:32.310 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T20:44:32.310 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T20:44:32.310 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T20:44:32.311 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-11-06T20:44:33.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:33.024 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2023-11-06T20:44:33.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:33.073 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2023-11-06T20:44:33.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:33.121 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-11-06T20:44:33.121 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-11-06T20:44:33.121 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-06T20:44:33.122 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-06T20:44:33.122 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-11-06T20:44:33.266 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-06T20:44:33.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:33.267 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-11-06T20:44:33.268 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-11-06T20:44:33.268 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-06T20:44:33.268 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-06T20:44:33.269 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-06T20:44:33.269 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-06T20:44:33.269 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-06T20:44:33.269 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-11-06T20:44:33.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:33.529 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-11-06T20:44:33.530 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-11-06T20:44:33.530 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2023-11-06T20:44:33.530 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2023-11-06T20:44:33.531 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-11-06T20:44:33.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:33.705 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p /home/ubuntu/cephtest/mnt.1 2023-11-06T20:44:33.752 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T20:44:33.752 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:33.891 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:33.892 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:44:34.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:44:34.394 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:44:34.418 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:34.418 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:44:34.422 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T20:44:34.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:34.423 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-06T20:44:34.532 INFO:tasks.kclient:client.3 config is {} 2023-11-06T20:44:34.532 DEBUG:teuthology.orchestra.run.smithi069:> ceph fs ls 2023-11-06T20:44:34.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.731+0000 7fbc521ff700 1 Processor -- start 2023-11-06T20:44:34.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.732+0000 7fbc521ff700 1 -- start start 2023-11-06T20:44:34.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.733+0000 7fbc521ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbc4c150440 0x7fbc4c068970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:34.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.733+0000 7fbc521ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c0c2230 0x7fbc4c068e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:34.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.733+0000 7fbc521ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4c06a060 con 0x7fbc4c0c6630 2023-11-06T20:44:34.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.733+0000 7fbc521ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c069450 con 0x7fbc4c150440 2023-11-06T20:44:34.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.733+0000 7fbc521ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c069730 con 0x7fbc4c0c2230 2023-11-06T20:44:34.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c0c2230 0x7fbc4c068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:34.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c0c2230 0x7fbc4c068e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.69:38534/0 (socket says 172.21.15.69:38534) 2023-11-06T20:44:34.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4bfff700 1 -- 172.21.15.69:0/3873943669 learned_addr learned my addr 172.21.15.69:0/3873943669 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:44:34.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4bfff700 1 -- 172.21.15.69:0/3873943669 >> v2:172.21.15.33:3300/0 conn(0x7fbc4c150440 msgr2=0x7fbc4c068970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:34.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4b7fe700 1 --2- 172.21.15.69:0/3873943669 >> v2:172.21.15.33:3300/0 conn(0x7fbc4c150440 0x7fbc4c068970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:34.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4bfff700 1 --2- 172.21.15.69:0/3873943669 >> v2:172.21.15.33:3300/0 conn(0x7fbc4c150440 0x7fbc4c068970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:34.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4bfff700 1 -- 172.21.15.69:0/3873943669 >> v1:172.21.15.33:6789/0 conn(0x7fbc4c0c6630 legacy=0x7fbc4c0c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:34.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4bfff700 1 -- 172.21.15.69:0/3873943669 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c069a10 con 0x7fbc4c0c2230 2023-11-06T20:44:34.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4b7fe700 1 --2- 172.21.15.69:0/3873943669 >> v2:172.21.15.33:3300/0 conn(0x7fbc4c150440 0x7fbc4c068970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:34.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4bfff700 1 --2- 172.21.15.69:0/3873943669 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c0c2230 0x7fbc4c068e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fbc44009f20 tx=0x7fbc44009840).ready entity=mon.2 client_cookie=a0f468e89857feff server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:34.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.734+0000 7fbc4a7fc700 1 -- 172.21.15.69:0/3873943669 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400e070 con 0x7fbc4c0c2230 2023-11-06T20:44:34.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.735+0000 7fbc4a7fc700 1 -- 172.21.15.69:0/3873943669 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4400ba40 con 0x7fbc4c0c2230 2023-11-06T20:44:34.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.735+0000 7fbc4a7fc700 1 -- 172.21.15.69:0/3873943669 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400bdc0 con 0x7fbc4c0c2230 2023-11-06T20:44:34.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.735+0000 7fbc521ff700 1 -- 172.21.15.69:0/3873943669 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c0c2230 msgr2=0x7fbc4c068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:34.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.735+0000 7fbc521ff700 1 --2- 172.21.15.69:0/3873943669 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c0c2230 0x7fbc4c068e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fbc44009f20 tx=0x7fbc44009840).stop 2023-11-06T20:44:34.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.735+0000 7fbc521ff700 1 -- 172.21.15.69:0/3873943669 shutdown_connections 2023-11-06T20:44:34.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.735+0000 7fbc521ff700 1 --2- 172.21.15.69:0/3873943669 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c0c2230 0x7fbc4c068e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:34.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.735+0000 7fbc521ff700 1 --2- 172.21.15.69:0/3873943669 >> v2:172.21.15.33:3300/0 conn(0x7fbc4c150440 0x7fbc4c068970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:34.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.735+0000 7fbc521ff700 1 -- 172.21.15.69:0/3873943669 >> 172.21.15.69:0/3873943669 conn(0x7fbc4c0c3510 msgr2=0x7fbc4c0c3920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.735+0000 7fbc521ff700 1 -- 172.21.15.69:0/3873943669 shutdown_connections 2023-11-06T20:44:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.735+0000 7fbc521ff700 1 -- 172.21.15.69:0/3873943669 wait complete. 2023-11-06T20:44:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.737+0000 7fbc521ff700 1 Processor -- start 2023-11-06T20:44:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.737+0000 7fbc521ff700 1 -- start start 2023-11-06T20:44:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc521ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc4c0c2230 0x7fbc4c0bd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc521ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c150440 0x7fbc4c14def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc521ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c163d70 0x7fbc4c0abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc521ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c0aa510 con 0x7fbc4c0c2230 2023-11-06T20:44:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc521ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c0ac810 con 0x7fbc4c163d70 2023-11-06T20:44:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc521ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c0a9c20 con 0x7fbc4c150440 2023-11-06T20:44:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4affd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c150440 0x7fbc4c14def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c163d70 0x7fbc4c0abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4affd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c150440 0x7fbc4c14def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.69:38544/0 (socket says 172.21.15.69:38544) 2023-11-06T20:44:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4affd700 1 -- 172.21.15.69:0/1865080134 learned_addr learned my addr 172.21.15.69:0/1865080134 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-11-06T20:44:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4b7fe700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc4c0c2230 0x7fbc4c0bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4bfff700 1 -- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c150440 msgr2=0x7fbc4c14def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4bfff700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c150440 0x7fbc4c14def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4bfff700 1 -- 172.21.15.69:0/1865080134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc4c0c2230 msgr2=0x7fbc4c0bd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4bfff700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc4c0c2230 0x7fbc4c0bd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4bfff700 1 -- 172.21.15.69:0/1865080134 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44009ae0 con 0x7fbc4c163d70 2023-11-06T20:44:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.738+0000 7fbc4b7fe700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc4c0c2230 0x7fbc4c0bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.739+0000 7fbc4bfff700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c163d70 0x7fbc4c0abe30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fbc44013000 tx=0x7fbc44009840).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.739+0000 7fbc337fe700 1 -- 172.21.15.69:0/1865080134 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc440092e0 con 0x7fbc4c163d70 2023-11-06T20:44:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.739+0000 7fbc337fe700 1 -- 172.21.15.69:0/1865080134 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4401e470 con 0x7fbc4c163d70 2023-11-06T20:44:34.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.739+0000 7fbc521ff700 1 -- 172.21.15.69:0/1865080134 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c0a9ec0 con 0x7fbc4c163d70 2023-11-06T20:44:34.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.739+0000 7fbc337fe700 1 -- 172.21.15.69:0/1865080134 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4401e830 con 0x7fbc4c163d70 2023-11-06T20:44:34.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.739+0000 7fbc521ff700 1 -- 172.21.15.69:0/1865080134 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c1641f0 con 0x7fbc4c163d70 2023-11-06T20:44:34.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.740+0000 7fbc521ff700 1 -- 172.21.15.69:0/1865080134 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4c0475a0 con 0x7fbc4c163d70 2023-11-06T20:44:34.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.741+0000 7fbc337fe700 1 -- 172.21.15.69:0/1865080134 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fbc44003040 con 0x7fbc4c163d70 2023-11-06T20:44:34.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.742+0000 7fbc337fe700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc3805f120 0x7fbc380615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:34.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.742+0000 7fbc337fe700 1 -- 172.21.15.69:0/1865080134 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbc4407b950 con 0x7fbc4c163d70 2023-11-06T20:44:34.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.742+0000 7fbc4b7fe700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc3805f120 0x7fbc380615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:34.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.742+0000 7fbc4b7fe700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc3805f120 0x7fbc380615e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fbc3c006fd0 tx=0x7fbc3c008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:34.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.747+0000 7fbc337fe700 1 -- 172.21.15.69:0/1865080134 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbc4404cb60 con 0x7fbc4c163d70 2023-11-06T20:44:34.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.962+0000 7fbc521ff700 1 -- 172.21.15.69:0/1865080134 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc4c0aa650 con 0x7fbc4c163d70 2023-11-06T20:44:34.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.963+0000 7fbc337fe700 1 -- 172.21.15.69:0/1865080134 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbc44023030 con 0x7fbc4c163d70 2023-11-06T20:44:34.963 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T20:44:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.964+0000 7fbc521ff700 1 -- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc3805f120 msgr2=0x7fbc380615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:34.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.964+0000 7fbc521ff700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc3805f120 0x7fbc380615e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fbc3c006fd0 tx=0x7fbc3c008040).stop 2023-11-06T20:44:34.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.965+0000 7fbc521ff700 1 -- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c163d70 msgr2=0x7fbc4c0abe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:34.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.965+0000 7fbc521ff700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c163d70 0x7fbc4c0abe30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fbc44013000 tx=0x7fbc44009840).stop 2023-11-06T20:44:34.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.965+0000 7fbc521ff700 1 -- 172.21.15.69:0/1865080134 shutdown_connections 2023-11-06T20:44:34.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.965+0000 7fbc521ff700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc3805f120 0x7fbc380615e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:34.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.965+0000 7fbc521ff700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c163d70 0x7fbc4c0abe30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:34.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.965+0000 7fbc521ff700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc4c0c2230 0x7fbc4c0bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:34.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.965+0000 7fbc521ff700 1 --2- 172.21.15.69:0/1865080134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbc4c150440 0x7fbc4c14def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:34.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.965+0000 7fbc521ff700 1 -- 172.21.15.69:0/1865080134 >> 172.21.15.69:0/1865080134 conn(0x7fbc4c0c3510 msgr2=0x7fbc4c065c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:34.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.965+0000 7fbc521ff700 1 -- 172.21.15.69:0/1865080134 shutdown_connections 2023-11-06T20:44:34.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-11-06T20:44:34.965+0000 7fbc521ff700 1 -- 172.21.15.69:0/1865080134 wait complete. 2023-11-06T20:44:34.979 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T20:44:34.980 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T20:44:34.980 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2023-11-06T20:44:34.980 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-06T20:44:34.980 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-06T20:44:34.981 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T20:44:34.981 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T20:44:34.981 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T20:44:34.981 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-06T20:44:34.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:34.982 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2023-11-06T20:44:35.031 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:44:35.032 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:44:35.032 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:44:35.032 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.032 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2023-11-06T20:44:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.033 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2023-11-06T20:44:35.033 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:35.034 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:35.034 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2023-11-06T20:44:35.034 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:44:35.034 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:35.034 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T20:44:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-06T20:44:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42256sec preferred_lft 42256sec 2023-11-06T20:44:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2023-11-06T20:44:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.036 INFO:teuthology.orchestra.run.smithi069.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T20:44:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 6a:f1:72:7e:db:ff brd ff:ff:ff:ff:ff:ff 2023-11-06T20:44:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T20:44:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::b007:49ff:fe92:ddbb/64 scope link 2023-11-06T20:44:35.037 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.037 INFO:teuthology.orchestra.run.smithi069.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T20:44:35.037 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 6a:f1:72:7e:db:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-06T20:44:35.037 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::68f1:72ff:fe7e:dbff/64 scope link 2023-11-06T20:44:35.037 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:35.038 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T20:44:35.106 INFO:teuthology.orchestra.run.smithi069.stdout:1 2023-11-06T20:44:35.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:35.109 DEBUG:teuthology.orchestra.run.smithi069:> route 2023-11-06T20:44:35.158 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2023-11-06T20:44:35.158 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T20:44:35.159 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T20:44:35.159 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T20:44:35.159 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T20:44:35.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:35.161 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-11-06T20:44:35.161 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-11-06T20:44:35.161 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T20:44:35.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T20:44:35.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T20:44:35.162 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-11-06T20:44:35.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:35.331 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2023-11-06T20:44:35.378 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-06T20:44:35.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:35.380 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2023-11-06T20:44:35.426 INFO:teuthology.orchestra.run.smithi069.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-06T20:44:35.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:35.428 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-11-06T20:44:35.428 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-11-06T20:44:35.428 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-06T20:44:35.428 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-06T20:44:35.429 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-11-06T20:44:35.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:35.573 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-06T20:44:35.646 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:44:35.646 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:44:35.646 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:44:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2023-11-06T20:44:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.647 INFO:teuthology.orchestra.run.smithi069.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T20:44:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 36:74:32:75:3a:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T20:44:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T20:44:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::3474:32ff:fe75:3a47/64 scope link 2023-11-06T20:44:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:35.650 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-06T20:44:35.721 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:44:35.721 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:44:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:44:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2023-11-06T20:44:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.723 INFO:teuthology.orchestra.run.smithi069.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T20:44:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 36:74:32:75:3a:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T20:44:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T20:44:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::3474:32ff:fe75:3a47/64 scope link 2023-11-06T20:44:35.724 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:44:35.725 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-06T20:44:35.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:35.726 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-11-06T20:44:35.726 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-11-06T20:44:35.726 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-06T20:44:35.727 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-06T20:44:35.727 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-06T20:44:35.727 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-06T20:44:35.728 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-06T20:44:35.728 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-11-06T20:44:35.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:35.969 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-11-06T20:44:35.969 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-11-06T20:44:35.970 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.1 up 2023-11-06T20:44:35.970 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.1 master ceph-brx 2023-11-06T20:44:35.970 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-11-06T20:44:36.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:36.143 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p /home/ubuntu/cephtest/mnt.3 2023-11-06T20:44:36.189 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T20:44:36.190 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:36.327 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:36.328 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,name=3,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:44:36.354 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:44:36.354 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,name=3,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:44:36.360 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T20:44:36.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:36.361 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-06T20:44:36.430 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-06T20:44:36.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:36.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.655+0000 7f085c1ff700 1 Processor -- start 2023-11-06T20:44:36.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.655+0000 7f085c1ff700 1 -- start start 2023-11-06T20:44:36.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.656+0000 7f085c1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f085414c640 0x7f08540bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:36.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.656+0000 7f085c1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 0x7f0854065160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:36.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.656+0000 7f085c1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0854153470 con 0x7f08540c33f0 2023-11-06T20:44:36.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.656+0000 7f085c1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08540bd8f0 con 0x7f085414c640 2023-11-06T20:44:36.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.656+0000 7f085c1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08540643a0 con 0x7f0854152ff0 2023-11-06T20:44:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f085979a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 0x7f0854065160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f0859f9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f085414c640 0x7f08540bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f085979a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 0x7f0854065160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:48266/0 (socket says 172.21.15.33:48266) 2023-11-06T20:44:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f085979a700 1 -- 172.21.15.33:0/86051101 learned_addr learned my addr 172.21.15.33:0/86051101 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f085a79c700 1 --1- 172.21.15.33:0/86051101 >> v1:172.21.15.69:6789/0 conn(0x7f08540c33f0 0x7f08540c3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:34020/0 (socket says 172.21.15.33:34020) 2023-11-06T20:44:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f0858f99700 1 -- 172.21.15.33:0/86051101 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422124967 0 0) 0x7f0854153470 con 0x7f08540c33f0 2023-11-06T20:44:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f0858f99700 1 -- 172.21.15.33:0/86051101 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08400033f0 con 0x7f08540c33f0 2023-11-06T20:44:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f085979a700 1 -- 172.21.15.33:0/86051101 >> v2:172.21.15.69:3300/0 conn(0x7f085414c640 msgr2=0x7f08540bb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f085979a700 1 --2- 172.21.15.33:0/86051101 >> v2:172.21.15.69:3300/0 conn(0x7f085414c640 0x7f08540bb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f085979a700 1 -- 172.21.15.33:0/86051101 >> v1:172.21.15.69:6789/0 conn(0x7f08540c33f0 legacy=0x7f08540c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f085979a700 1 -- 172.21.15.33:0/86051101 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0854064680 con 0x7f0854152ff0 2023-11-06T20:44:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f0859f9b700 1 --2- 172.21.15.33:0/86051101 >> v2:172.21.15.69:3300/0 conn(0x7f085414c640 0x7f08540bb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.657+0000 7f085979a700 1 --2- 172.21.15.33:0/86051101 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 0x7f0854065160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f0844006fd0 tx=0x7f08440090f0).ready entity=mon.2 client_cookie=94d7c9f672bbe8f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f0858f99700 1 -- 172.21.15.33:0/86051101 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084400f510 con 0x7f0854152ff0 2023-11-06T20:44:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f0858f99700 1 -- 172.21.15.33:0/86051101 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084400f670 con 0x7f0854152ff0 2023-11-06T20:44:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f0858f99700 1 -- 172.21.15.33:0/86051101 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084400f9f0 con 0x7f0854152ff0 2023-11-06T20:44:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f085c1ff700 1 -- 172.21.15.33:0/86051101 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 msgr2=0x7f0854065160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f085c1ff700 1 --2- 172.21.15.33:0/86051101 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 0x7f0854065160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f0844006fd0 tx=0x7f08440090f0).stop 2023-11-06T20:44:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f085c1ff700 1 -- 172.21.15.33:0/86051101 shutdown_connections 2023-11-06T20:44:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f085c1ff700 1 --2- 172.21.15.33:0/86051101 >> v2:172.21.15.69:3300/0 conn(0x7f085414c640 0x7f08540bb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f085c1ff700 1 --2- 172.21.15.33:0/86051101 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 0x7f0854065160 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f085c1ff700 1 -- 172.21.15.33:0/86051101 >> 172.21.15.33:0/86051101 conn(0x7f0854062710 msgr2=0x7f0854062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f085c1ff700 1 -- 172.21.15.33:0/86051101 shutdown_connections 2023-11-06T20:44:36.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.658+0000 7f085c1ff700 1 -- 172.21.15.33:0/86051101 wait complete. 2023-11-06T20:44:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.660+0000 7f085c1ff700 1 Processor -- start 2023-11-06T20:44:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.660+0000 7f085c1ff700 1 -- start start 2023-11-06T20:44:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f085c1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08540c33f0 0x7f085413f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f085c1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085414c640 0x7f08540c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f085c1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 0x7f08540c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f085c1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08540c6de0 con 0x7f08540c33f0 2023-11-06T20:44:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f085c1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08540c70c0 con 0x7f085414c640 2023-11-06T20:44:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f085c1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08540c73a0 con 0x7f0854152ff0 2023-11-06T20:44:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f0859f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08540c33f0 0x7f085413f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f085979a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085414c640 0x7f08540c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f0859f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08540c33f0 0x7f085413f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43916/0 (socket says 172.21.15.33:43916) 2023-11-06T20:44:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f0859f9b700 1 -- 172.21.15.33:0/2449714107 learned_addr learned my addr 172.21.15.33:0/2449714107 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.661+0000 7f085a79c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 0x7f08540c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f085979a700 1 -- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 msgr2=0x7f08540c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:36.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f085979a700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 0x7f08540c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:36.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f085979a700 1 -- 172.21.15.33:0/2449714107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08540c33f0 msgr2=0x7f085413f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:36.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f085979a700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08540c33f0 0x7f085413f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:36.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f085979a700 1 -- 172.21.15.33:0/2449714107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08500092a0 con 0x7f085414c640 2023-11-06T20:44:36.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f085979a700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085414c640 0x7f08540c5390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f0844014000 tx=0x7f0844017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f0849ffb700 1 -- 172.21.15.33:0/2449714107 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084400fe80 con 0x7f085414c640 2023-11-06T20:44:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f0849ffb700 1 -- 172.21.15.33:0/2449714107 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0844006a40 con 0x7f085414c640 2023-11-06T20:44:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f0849ffb700 1 -- 172.21.15.33:0/2449714107 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084401c420 con 0x7f085414c640 2023-11-06T20:44:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f085c1ff700 1 -- 172.21.15.33:0/2449714107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0844009370 con 0x7f085414c640 2023-11-06T20:44:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.662+0000 7f085c1ff700 1 -- 172.21.15.33:0/2449714107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085413fb40 con 0x7f085414c640 2023-11-06T20:44:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.663+0000 7f085c1ff700 1 -- 172.21.15.33:0/2449714107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0854064960 con 0x7f085414c640 2023-11-06T20:44:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.664+0000 7f0849ffb700 1 -- 172.21.15.33:0/2449714107 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f084401c790 con 0x7f085414c640 2023-11-06T20:44:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.665+0000 7f0849ffb700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f083805f150 0x7f0838061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.665+0000 7f0849ffb700 1 -- 172.21.15.33:0/2449714107 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f084407b8e0 con 0x7f085414c640 2023-11-06T20:44:36.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.665+0000 7f0859f9b700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f083805f150 0x7f0838061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:36.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.666+0000 7f0859f9b700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f083805f150 0x7f0838061610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0850009ec0 tx=0x7f0850009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:36.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.669+0000 7f0849ffb700 1 -- 172.21.15.33:0/2449714107 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f084404caf0 con 0x7f085414c640 2023-11-06T20:44:36.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.848+0000 7f085c1ff700 1 -- 172.21.15.33:0/2449714107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08540c2ec0 con 0x7f085414c640 2023-11-06T20:44:36.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.850+0000 7f0849ffb700 1 -- 172.21.15.33:0/2449714107 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f0844020020 con 0x7f085414c640 2023-11-06T20:44:36.851 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:36.851 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:36.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 -- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f083805f150 msgr2=0x7f0838061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:36.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f083805f150 0x7f0838061610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0850009ec0 tx=0x7f0850009060).stop 2023-11-06T20:44:36.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 -- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085414c640 msgr2=0x7f08540c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:36.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085414c640 0x7f08540c5390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f0844014000 tx=0x7f0844017000).stop 2023-11-06T20:44:36.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 -- 172.21.15.33:0/2449714107 shutdown_connections 2023-11-06T20:44:36.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f083805f150 0x7f0838061610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:36.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0854152ff0 0x7f08540c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:36.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08540c33f0 0x7f085413f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 --2- 172.21.15.33:0/2449714107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085414c640 0x7f08540c5390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 -- 172.21.15.33:0/2449714107 >> 172.21.15.33:0/2449714107 conn(0x7f0854062710 msgr2=0x7f08540c3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 -- 172.21.15.33:0/2449714107 shutdown_connections 2023-11-06T20:44:36.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:36.852+0000 7f085c1ff700 1 -- 172.21.15.33:0/2449714107 wait complete. 2023-11-06T20:44:36.871 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:44:04.716205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:44:05.351480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T20:44:36.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:44:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.078+0000 7fac94bff700 1 Processor -- start 2023-11-06T20:44:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.078+0000 7fac94bff700 1 -- start start 2023-11-06T20:44:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.080+0000 7fac94bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fac90154710 0x7fac900c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.080+0000 7fac94bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac900c6b60 con 0x7fac900c7ff0 2023-11-06T20:44:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.080+0000 7fac94bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac900c7700 con 0x7fac90154330 2023-11-06T20:44:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.080+0000 7fac94bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac900c5cd0 con 0x7fac90154710 2023-11-06T20:44:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.080+0000 7fac8ed9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fac900c7ff0 0x7fac900646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37914/0 (socket says 172.21.15.33:37914) 2023-11-06T20:44:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.080+0000 7fac8ed9d700 1 -- 172.21.15.33:0/769621057 learned_addr learned my addr 172.21.15.33:0/769621057 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.080+0000 7fac8dd9b700 1 --2- 172.21.15.33:0/769621057 >> v2:172.21.15.69:3300/0 conn(0x7fac90154710 0x7fac900c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8d59a700 1 -- 172.21.15.33:0/769621057 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772274955 0 0) 0x7fac900c6b60 con 0x7fac900c7ff0 2023-11-06T20:44:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8d59a700 1 -- 172.21.15.33:0/769621057 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac780033f0 con 0x7fac900c7ff0 2023-11-06T20:44:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8d59a700 1 -- 172.21.15.33:0/769621057 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793640574 0 0) 0x7fac900c7700 con 0x7fac90154330 2023-11-06T20:44:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8d59a700 1 -- 172.21.15.33:0/769621057 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac900c6b60 con 0x7fac90154330 2023-11-06T20:44:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8dd9b700 1 -- 172.21.15.33:0/769621057 >> v1:172.21.15.69:6789/0 conn(0x7fac90154330 legacy=0x7fac900c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8dd9b700 1 -- 172.21.15.33:0/769621057 >> v1:172.21.15.33:6789/0 conn(0x7fac900c7ff0 legacy=0x7fac900646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8d59a700 1 -- 172.21.15.33:0/769621057 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2968768260 0 0) 0x7fac780033f0 con 0x7fac900c7ff0 2023-11-06T20:44:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8dd9b700 1 -- 172.21.15.33:0/769621057 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac900c5f50 con 0x7fac90154710 2023-11-06T20:44:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8dd9b700 1 --2- 172.21.15.33:0/769621057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac90154710 0x7fac900c57e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fac84009760 tx=0x7fac84009060).ready entity=mon.1 client_cookie=9dfa9932e6d7cf55 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8d59a700 1 -- 172.21.15.33:0/769621057 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac880017f0 con 0x7fac90154710 2023-11-06T20:44:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8d59a700 1 -- 172.21.15.33:0/769621057 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac84011400 con 0x7fac90154710 2023-11-06T20:44:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.081+0000 7fac8d59a700 1 -- 172.21.15.33:0/769621057 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac84011780 con 0x7fac90154710 2023-11-06T20:44:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.082+0000 7fac94bff700 1 -- 172.21.15.33:0/769621057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac90154710 msgr2=0x7fac900c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.082+0000 7fac94bff700 1 --2- 172.21.15.33:0/769621057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac90154710 0x7fac900c57e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fac84009760 tx=0x7fac84009060).stop 2023-11-06T20:44:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.082+0000 7fac94bff700 1 -- 172.21.15.33:0/769621057 shutdown_connections 2023-11-06T20:44:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.082+0000 7fac94bff700 1 --2- 172.21.15.33:0/769621057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac90154710 0x7fac900c57e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.082+0000 7fac94bff700 1 -- 172.21.15.33:0/769621057 >> 172.21.15.33:0/769621057 conn(0x7fac900cb910 msgr2=0x7fac900c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.082+0000 7fac94bff700 1 -- 172.21.15.33:0/769621057 shutdown_connections 2023-11-06T20:44:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.082+0000 7fac94bff700 1 -- 172.21.15.33:0/769621057 wait complete. 2023-11-06T20:44:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.084+0000 7fac94bff700 1 Processor -- start 2023-11-06T20:44:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.084+0000 7fac94bff700 1 -- start start 2023-11-06T20:44:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac94bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c7ff0 0x7fac90147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac94bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90154330 0x7fac901555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac94bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90154710 0x7fac900c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac94bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90148260 con 0x7fac90154710 2023-11-06T20:44:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac94bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90155eb0 con 0x7fac900c7ff0 2023-11-06T20:44:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac94bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac90156190 con 0x7fac90154330 2023-11-06T20:44:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac8ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90154710 0x7fac900c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac8ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90154710 0x7fac900c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43930/0 (socket says 172.21.15.33:43930) 2023-11-06T20:44:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac8ed9d700 1 -- 172.21.15.33:0/4266967796 learned_addr learned my addr 172.21.15.33:0/4266967796 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac8e59c700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c7ff0 0x7fac90147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac8dd9b700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90154330 0x7fac901555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac8ed9d700 1 -- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90154330 msgr2=0x7fac901555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac8ed9d700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90154330 0x7fac901555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac8ed9d700 1 -- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c7ff0 msgr2=0x7fac90147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac8ed9d700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c7ff0 0x7fac90147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.085+0000 7fac8ed9d700 1 -- 172.21.15.33:0/4266967796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac840092a0 con 0x7fac90154710 2023-11-06T20:44:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.086+0000 7fac8ed9d700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90154710 0x7fac900c01d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fac8800cb40 tx=0x7fac88007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.086+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4266967796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8800ce50 con 0x7fac90154710 2023-11-06T20:44:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.086+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4266967796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac88004c40 con 0x7fac90154710 2023-11-06T20:44:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.086+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4266967796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88007170 con 0x7fac90154710 2023-11-06T20:44:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.086+0000 7fac94bff700 1 -- 172.21.15.33:0/4266967796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac90156470 con 0x7fac90154710 2023-11-06T20:44:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.086+0000 7fac94bff700 1 -- 172.21.15.33:0/4266967796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac90156a30 con 0x7fac90154710 2023-11-06T20:44:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.087+0000 7fac94bff700 1 -- 172.21.15.33:0/4266967796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac90156c20 con 0x7fac90154710 2023-11-06T20:44:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.088+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4266967796 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fac88007520 con 0x7fac90154710 2023-11-06T20:44:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.089+0000 7fac7e7fc700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fac7005f150 0x7fac70061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.089+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4266967796 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fac88079fa0 con 0x7fac90154710 2023-11-06T20:44:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.089+0000 7fac8e59c700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fac7005f150 0x7fac70061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.090+0000 7fac8e59c700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fac7005f150 0x7fac70061610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fac80003930 tx=0x7fac80007bd0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:37.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.093+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4266967796 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fac8804b1b0 con 0x7fac90154710 2023-11-06T20:44:37.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.273+0000 7fac94bff700 1 -- 172.21.15.33:0/4266967796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fac900830b0 con 0x7fac90154710 2023-11-06T20:44:37.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.274+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4266967796 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7fac8804e7d0 con 0x7fac90154710 2023-11-06T20:44:37.274 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:44:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 -- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fac7005f150 msgr2=0x7fac70061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fac7005f150 0x7fac70061610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fac80003930 tx=0x7fac80007bd0).stop 2023-11-06T20:44:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 -- 172.21.15.33:0/4266967796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90154710 msgr2=0x7fac900c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90154710 0x7fac900c01d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fac8800cb40 tx=0x7fac88007ba0).stop 2023-11-06T20:44:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 -- 172.21.15.33:0/4266967796 shutdown_connections 2023-11-06T20:44:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fac7005f150 0x7fac70061610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90154710 0x7fac900c01d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c7ff0 0x7fac90147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 --2- 172.21.15.33:0/4266967796 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90154330 0x7fac901555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 -- 172.21.15.33:0/4266967796 >> 172.21.15.33:0/4266967796 conn(0x7fac900cb910 msgr2=0x7fac90152c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 -- 172.21.15.33:0/4266967796 shutdown_connections 2023-11-06T20:44:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.276+0000 7fac94bff700 1 -- 172.21.15.33:0/4266967796 wait complete. 2023-11-06T20:44:37.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:44:37.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.494+0000 7f876b5ff700 1 Processor -- start 2023-11-06T20:44:37.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.494+0000 7f876b5ff700 1 -- start start 2023-11-06T20:44:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.495+0000 7f876b5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87640646c0 0x7f8764064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.495+0000 7f876b5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87640c57e0 0x7f87640c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.495+0000 7f876b5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87641556d0 con 0x7f87641544f0 2023-11-06T20:44:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.495+0000 7f876b5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87641548d0 con 0x7f87640c57e0 2023-11-06T20:44:37.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f876b5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8764065030 con 0x7f87640646c0 2023-11-06T20:44:37.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f876939b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f87641544f0 0x7f87640c0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37928/0 (socket says 172.21.15.33:37928) 2023-11-06T20:44:37.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f876939b700 1 -- 172.21.15.33:0/794784476 learned_addr learned my addr 172.21.15.33:0/794784476 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:37.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f8769b9c700 1 --2- 172.21.15.33:0/794784476 >> v2:172.21.15.33:3300/0 conn(0x7f87640c57e0 0x7f87640c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f8768b9a700 1 --2- 172.21.15.33:0/794784476 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87640646c0 0x7f8764064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f875bfff700 1 -- 172.21.15.33:0/794784476 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787040999 0 0) 0x7f87641556d0 con 0x7f87641544f0 2023-11-06T20:44:37.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f875bfff700 1 -- 172.21.15.33:0/794784476 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87500033f0 con 0x7f87641544f0 2023-11-06T20:44:37.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f8769b9c700 1 -- 172.21.15.33:0/794784476 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87640646c0 msgr2=0x7f8764064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f8769b9c700 1 --2- 172.21.15.33:0/794784476 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87640646c0 0x7f8764064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f8769b9c700 1 -- 172.21.15.33:0/794784476 >> v1:172.21.15.33:6789/0 conn(0x7f87641544f0 legacy=0x7f87640c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.496+0000 7f8769b9c700 1 -- 172.21.15.33:0/794784476 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764154ac0 con 0x7f87640c57e0 2023-11-06T20:44:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f8769b9c700 1 --2- 172.21.15.33:0/794784476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 0x7f87640c5c40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f875c009e60 tx=0x7f875c009780).ready entity=mon.0 client_cookie=f3b2ea17ff85bf0c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f875bfff700 1 -- 172.21.15.33:0/794784476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00e070 con 0x7f87640c57e0 2023-11-06T20:44:37.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f875bfff700 1 -- 172.21.15.33:0/794784476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00bab0 con 0x7f87640c57e0 2023-11-06T20:44:37.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f875bfff700 1 -- 172.21.15.33:0/794784476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00be30 con 0x7f87640c57e0 2023-11-06T20:44:37.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f876b5ff700 1 -- 172.21.15.33:0/794784476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 msgr2=0x7f87640c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f876b5ff700 1 --2- 172.21.15.33:0/794784476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 0x7f87640c5c40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f875c009e60 tx=0x7f875c009780).stop 2023-11-06T20:44:37.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f876b5ff700 1 -- 172.21.15.33:0/794784476 shutdown_connections 2023-11-06T20:44:37.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f876b5ff700 1 --2- 172.21.15.33:0/794784476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 0x7f87640c5c40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f876b5ff700 1 --2- 172.21.15.33:0/794784476 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87640646c0 0x7f8764064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f876b5ff700 1 -- 172.21.15.33:0/794784476 >> 172.21.15.33:0/794784476 conn(0x7f87640cb900 msgr2=0x7f87640c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:37.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.497+0000 7f876b5ff700 1 -- 172.21.15.33:0/794784476 shutdown_connections 2023-11-06T20:44:37.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.498+0000 7f876b5ff700 1 -- 172.21.15.33:0/794784476 wait complete. 2023-11-06T20:44:37.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.499+0000 7f876b5ff700 1 Processor -- start 2023-11-06T20:44:37.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.499+0000 7f876b5ff700 1 -- start start 2023-11-06T20:44:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.500+0000 7f876b5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87640646c0 0x7f87640b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.500+0000 7f876b5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 0x7f87640c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.500+0000 7f876b5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87641544f0 0x7f87641469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.500+0000 7f876b5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c4170 con 0x7f87640c57e0 2023-11-06T20:44:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.500+0000 7f876b5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c3240 con 0x7f87640646c0 2023-11-06T20:44:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.500+0000 7f876b5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c34f0 con 0x7f87641544f0 2023-11-06T20:44:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.500+0000 7f8768b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 0x7f87640c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.500+0000 7f8768b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 0x7f87640c2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43942/0 (socket says 172.21.15.33:43942) 2023-11-06T20:44:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.500+0000 7f8768b9a700 1 -- 172.21.15.33:0/661393463 learned_addr learned my addr 172.21.15.33:0/661393463 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f8769b9c700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87641544f0 0x7f87641469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f876939b700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87640646c0 0x7f87640b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f8768b9a700 1 -- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87641544f0 msgr2=0x7f87641469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f8768b9a700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87641544f0 0x7f87641469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f8768b9a700 1 -- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87640646c0 msgr2=0x7f87640b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f8768b9a700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87640646c0 0x7f87640b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f8768b9a700 1 -- 172.21.15.33:0/661393463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c009a20 con 0x7f87640c57e0 2023-11-06T20:44:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f8768b9a700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 0x7f87640c2d50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f875400e930 tx=0x7f8754009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f87597fa700 1 -- 172.21.15.33:0/661393463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754012600 con 0x7f87640c57e0 2023-11-06T20:44:37.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f87597fa700 1 -- 172.21.15.33:0/661393463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8754012760 con 0x7f87640c57e0 2023-11-06T20:44:37.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f87597fa700 1 -- 172.21.15.33:0/661393463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754012ae0 con 0x7f87640c57e0 2023-11-06T20:44:37.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f876b5ff700 1 -- 172.21.15.33:0/661393463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87640c37d0 con 0x7f87640c57e0 2023-11-06T20:44:37.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.501+0000 7f876b5ff700 1 -- 172.21.15.33:0/661393463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87640c3d90 con 0x7f87640c57e0 2023-11-06T20:44:37.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.503+0000 7f876b5ff700 1 -- 172.21.15.33:0/661393463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87640c42b0 con 0x7f87640c57e0 2023-11-06T20:44:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.503+0000 7f87597fa700 1 -- 172.21.15.33:0/661393463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f875401b400 con 0x7f87640c57e0 2023-11-06T20:44:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.504+0000 7f87597fa700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f150 0x7f8748061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.505+0000 7f87597fa700 1 -- 172.21.15.33:0/661393463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f875407a650 con 0x7f87640c57e0 2023-11-06T20:44:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.505+0000 7f876939b700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f150 0x7f8748061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.505+0000 7f876939b700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f150 0x7f8748061610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f8760007fd0 tx=0x7f8760009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.508+0000 7f87597fa700 1 -- 172.21.15.33:0/661393463 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f87640c42b0 con 0x7f87640c57e0 2023-11-06T20:44:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.688+0000 7f876b5ff700 1 -- 172.21.15.33:0/661393463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87640c42b0 con 0x7f87640c57e0 2023-11-06T20:44:37.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.689+0000 7f87597fa700 1 -- 172.21.15.33:0/661393463 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f87640c42b0 con 0x7f87640c57e0 2023-11-06T20:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:44:37.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 -- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f150 msgr2=0x7f8748061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f150 0x7f8748061610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f8760007fd0 tx=0x7f8760009040).stop 2023-11-06T20:44:37.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 -- 172.21.15.33:0/661393463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 msgr2=0x7f87640c2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 0x7f87640c2d50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f875400e930 tx=0x7f8754009060).stop 2023-11-06T20:44:37.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 -- 172.21.15.33:0/661393463 shutdown_connections 2023-11-06T20:44:37.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f150 0x7f8748061610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87641544f0 0x7f87641469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87640646c0 0x7f87640b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 --2- 172.21.15.33:0/661393463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c57e0 0x7f87640c2d50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 -- 172.21.15.33:0/661393463 >> 172.21.15.33:0/661393463 conn(0x7f87640cb900 msgr2=0x7f8764152d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:37.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 -- 172.21.15.33:0/661393463 shutdown_connections 2023-11-06T20:44:37.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.691+0000 7f876b5ff700 1 -- 172.21.15.33:0/661393463 wait complete. 2023-11-06T20:44:37.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T20:44:37.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.908+0000 7fe87c1ff700 1 Processor -- start 2023-11-06T20:44:37.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.908+0000 7fe87c1ff700 1 -- start start 2023-11-06T20:44:37.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.909+0000 7fe87c1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe874154330 0x7fe8740c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.909+0000 7fe87c1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe874154710 0x7fe8740c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.909+0000 7fe87c1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740c6b60 con 0x7fe8740c7ff0 2023-11-06T20:44:37.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.909+0000 7fe87c1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8740c5cd0 con 0x7fe874154330 2023-11-06T20:44:37.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.909+0000 7fe87c1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740c5f50 con 0x7fe874154710 2023-11-06T20:44:37.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.910+0000 7fe879f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe874154330 0x7fe8740c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.910+0000 7fe87a79c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe8740c7ff0 0x7fe8740646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43808/0 (socket says 172.21.15.33:43808) 2023-11-06T20:44:37.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.910+0000 7fe87a79c700 1 -- 172.21.15.33:0/644833841 learned_addr learned my addr 172.21.15.33:0/644833841 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:37.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.910+0000 7fe87979a700 1 --2- 172.21.15.33:0/644833841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe874154710 0x7fe8740c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.910+0000 7fe879f9b700 1 -- 172.21.15.33:0/644833841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe874154710 msgr2=0x7fe8740c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:37.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.910+0000 7fe879f9b700 1 --2- 172.21.15.33:0/644833841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe874154710 0x7fe8740c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.910+0000 7fe879f9b700 1 -- 172.21.15.33:0/644833841 >> v1:172.21.15.33:6789/0 conn(0x7fe8740c7ff0 legacy=0x7fe8740646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.910+0000 7fe879f9b700 1 -- 172.21.15.33:0/644833841 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740c6230 con 0x7fe874154330 2023-11-06T20:44:37.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.910+0000 7fe878f99700 1 -- 172.21.15.33:0/644833841 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232900529 0 0) 0x7fe8740c6b60 con 0x7fe8740c7ff0 2023-11-06T20:44:37.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.910+0000 7fe879f9b700 1 --2- 172.21.15.33:0/644833841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe874154330 0x7fe8740c57e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fe870009760 tx=0x7fe870009060).ready entity=mon.0 client_cookie=a13dc572b7e75d2e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:37.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe878f99700 1 -- 172.21.15.33:0/644833841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000f510 con 0x7fe874154330 2023-11-06T20:44:37.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe878f99700 1 -- 172.21.15.33:0/644833841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87000f670 con 0x7fe874154330 2023-11-06T20:44:37.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe878f99700 1 -- 172.21.15.33:0/644833841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000f9f0 con 0x7fe874154330 2023-11-06T20:44:37.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe87c1ff700 1 -- 172.21.15.33:0/644833841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe874154330 msgr2=0x7fe8740c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe87c1ff700 1 --2- 172.21.15.33:0/644833841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe874154330 0x7fe8740c57e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fe870009760 tx=0x7fe870009060).stop 2023-11-06T20:44:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe87c1ff700 1 -- 172.21.15.33:0/644833841 shutdown_connections 2023-11-06T20:44:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe87c1ff700 1 --2- 172.21.15.33:0/644833841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe874154330 0x7fe8740c57e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe87c1ff700 1 --2- 172.21.15.33:0/644833841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe874154710 0x7fe8740c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe87c1ff700 1 -- 172.21.15.33:0/644833841 >> 172.21.15.33:0/644833841 conn(0x7fe8740cb910 msgr2=0x7fe8740c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe87c1ff700 1 -- 172.21.15.33:0/644833841 shutdown_connections 2023-11-06T20:44:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.911+0000 7fe87c1ff700 1 -- 172.21.15.33:0/644833841 wait complete. 2023-11-06T20:44:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.913+0000 7fe87c1ff700 1 Processor -- start 2023-11-06T20:44:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.913+0000 7fe87c1ff700 1 -- start start 2023-11-06T20:44:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe87c1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8740c7ff0 0x7fe874149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe87c1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe874154330 0x7fe874151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe87c1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe874154710 0x7fe8740b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe87c1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe874155ab0 con 0x7fe8740c7ff0 2023-11-06T20:44:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe87c1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe874155780 con 0x7fe874154710 2023-11-06T20:44:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe87c1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741523b0 con 0x7fe874154330 2023-11-06T20:44:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe879f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8740c7ff0 0x7fe874149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe879f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8740c7ff0 0x7fe874149b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36944/0 (socket says 172.21.15.33:36944) 2023-11-06T20:44:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe879f9b700 1 -- 172.21.15.33:0/3034478613 learned_addr learned my addr 172.21.15.33:0/3034478613 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe87979a700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe874154330 0x7fe874151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe87a79c700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe874154710 0x7fe8740b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe879f9b700 1 -- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe874154330 msgr2=0x7fe874151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.914+0000 7fe879f9b700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe874154330 0x7fe874151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.915+0000 7fe879f9b700 1 -- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe874154710 msgr2=0x7fe8740b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:37.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.915+0000 7fe879f9b700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe874154710 0x7fe8740b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:37.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.915+0000 7fe879f9b700 1 -- 172.21.15.33:0/3034478613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700092a0 con 0x7fe8740c7ff0 2023-11-06T20:44:37.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.915+0000 7fe879f9b700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8740c7ff0 0x7fe874149b30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fe87406c170 tx=0x7fe870017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:37.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.915+0000 7fe869ffb700 1 -- 172.21.15.33:0/3034478613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000f7e0 con 0x7fe8740c7ff0 2023-11-06T20:44:37.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.915+0000 7fe869ffb700 1 -- 172.21.15.33:0/3034478613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87000f940 con 0x7fe8740c7ff0 2023-11-06T20:44:37.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.915+0000 7fe87c1ff700 1 -- 172.21.15.33:0/3034478613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe874152690 con 0x7fe8740c7ff0 2023-11-06T20:44:37.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.915+0000 7fe869ffb700 1 -- 172.21.15.33:0/3034478613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87001b690 con 0x7fe8740c7ff0 2023-11-06T20:44:37.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.915+0000 7fe87c1ff700 1 -- 172.21.15.33:0/3034478613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87414a020 con 0x7fe8740c7ff0 2023-11-06T20:44:37.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.916+0000 7fe87c1ff700 1 -- 172.21.15.33:0/3034478613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8740830b0 con 0x7fe8740c7ff0 2023-11-06T20:44:37.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.917+0000 7fe869ffb700 1 -- 172.21.15.33:0/3034478613 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe87001ba80 con 0x7fe8740c7ff0 2023-11-06T20:44:37.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.918+0000 7fe869ffb700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe85805f120 0x7fe8580615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:37.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.918+0000 7fe869ffb700 1 -- 172.21.15.33:0/3034478613 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe87007b950 con 0x7fe8740c7ff0 2023-11-06T20:44:37.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.919+0000 7fe87979a700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe85805f120 0x7fe8580615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:37.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.919+0000 7fe87979a700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe85805f120 0x7fe8580615e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fe8740aa080 tx=0x7fe864009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:37.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:37.923+0000 7fe869ffb700 1 -- 172.21.15.33:0/3034478613 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe87004cb60 con 0x7fe8740c7ff0 2023-11-06T20:44:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.106+0000 7fe87c1ff700 1 -- 172.21.15.33:0/3034478613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8740652d0 con 0x7fe8740c7ff0 2023-11-06T20:44:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.106+0000 7fe869ffb700 1 -- 172.21.15.33:0/3034478613 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8700280a0 con 0x7fe8740c7ff0 2023-11-06T20:44:38.107 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 -- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe85805f120 msgr2=0x7fe8580615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe85805f120 0x7fe8580615e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fe8740aa080 tx=0x7fe864009060).stop 2023-11-06T20:44:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 -- 172.21.15.33:0/3034478613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8740c7ff0 msgr2=0x7fe874149b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8740c7ff0 0x7fe874149b30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fe87406c170 tx=0x7fe870017000).stop 2023-11-06T20:44:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 -- 172.21.15.33:0/3034478613 shutdown_connections 2023-11-06T20:44:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe85805f120 0x7fe8580615e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe874154710 0x7fe8740b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8740c7ff0 0x7fe874149b30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 --2- 172.21.15.33:0/3034478613 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe874154330 0x7fe874151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 -- 172.21.15.33:0/3034478613 >> 172.21.15.33:0/3034478613 conn(0x7fe8740cb910 msgr2=0x7fe8740aa200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 -- 172.21.15.33:0/3034478613 shutdown_connections 2023-11-06T20:44:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.109+0000 7fe87c1ff700 1 -- 172.21.15.33:0/3034478613 wait complete. 2023-11-06T20:44:38.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T20:44:38.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.324+0000 7f2f899ff700 1 Processor -- start 2023-11-06T20:44:38.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.325+0000 7f2f899ff700 1 -- start start 2023-11-06T20:44:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.326+0000 7f2f899ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f84154330 0x7f2f840c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.326+0000 7f2f899ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2f84154710 0x7f2f840c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.326+0000 7f2f899ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f840c6b60 con 0x7f2f840c7ff0 2023-11-06T20:44:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.326+0000 7f2f899ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f840c5cd0 con 0x7f2f84154330 2023-11-06T20:44:38.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.326+0000 7f2f899ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f840c5f50 con 0x7f2f84154710 2023-11-06T20:44:38.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.326+0000 7f2f82ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f84154330 0x7f2f840c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.326+0000 7f2f82ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f84154330 0x7f2f840c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36950/0 (socket says 172.21.15.33:36950) 2023-11-06T20:44:38.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.326+0000 7f2f82ffd700 1 -- 172.21.15.33:0/1626761561 learned_addr learned my addr 172.21.15.33:0/1626761561 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.326+0000 7f2f827fc700 1 --2- 172.21.15.33:0/1626761561 >> v2:172.21.15.69:3300/0 conn(0x7f2f84154710 0x7f2f840c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.327+0000 7f2f82ffd700 1 -- 172.21.15.33:0/1626761561 >> v2:172.21.15.69:3300/0 conn(0x7f2f84154710 msgr2=0x7f2f840c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.327+0000 7f2f82ffd700 1 --2- 172.21.15.33:0/1626761561 >> v2:172.21.15.69:3300/0 conn(0x7f2f84154710 0x7f2f840c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.327+0000 7f2f82ffd700 1 -- 172.21.15.33:0/1626761561 >> v1:172.21.15.69:6789/0 conn(0x7f2f840c7ff0 legacy=0x7f2f840646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.327+0000 7f2f82ffd700 1 -- 172.21.15.33:0/1626761561 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f840c6230 con 0x7f2f84154330 2023-11-06T20:44:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.327+0000 7f2f82ffd700 1 --2- 172.21.15.33:0/1626761561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154330 0x7f2f840c57e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f2f840c8d10 tx=0x7f2f740090f0).ready entity=mon.0 client_cookie=4edc0110602517c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.327+0000 7f2f81ffb700 1 -- 172.21.15.33:0/1626761561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400b5a0 con 0x7f2f84154330 2023-11-06T20:44:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.327+0000 7f2f81ffb700 1 -- 172.21.15.33:0/1626761561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7400b700 con 0x7f2f84154330 2023-11-06T20:44:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.327+0000 7f2f81ffb700 1 -- 172.21.15.33:0/1626761561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400ba80 con 0x7f2f84154330 2023-11-06T20:44:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.327+0000 7f2f899ff700 1 -- 172.21.15.33:0/1626761561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154330 msgr2=0x7f2f840c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.328+0000 7f2f899ff700 1 --2- 172.21.15.33:0/1626761561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154330 0x7f2f840c57e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f2f840c8d10 tx=0x7f2f740090f0).stop 2023-11-06T20:44:38.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.328+0000 7f2f899ff700 1 -- 172.21.15.33:0/1626761561 shutdown_connections 2023-11-06T20:44:38.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.328+0000 7f2f899ff700 1 --2- 172.21.15.33:0/1626761561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154330 0x7f2f840c57e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.328+0000 7f2f899ff700 1 --2- 172.21.15.33:0/1626761561 >> v2:172.21.15.69:3300/0 conn(0x7f2f84154710 0x7f2f840c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.328+0000 7f2f899ff700 1 -- 172.21.15.33:0/1626761561 >> 172.21.15.33:0/1626761561 conn(0x7f2f840cb910 msgr2=0x7f2f840c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.328+0000 7f2f899ff700 1 -- 172.21.15.33:0/1626761561 shutdown_connections 2023-11-06T20:44:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.328+0000 7f2f899ff700 1 -- 172.21.15.33:0/1626761561 wait complete. 2023-11-06T20:44:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.329+0000 7f2f899ff700 1 Processor -- start 2023-11-06T20:44:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.329+0000 7f2f899ff700 1 -- start start 2023-11-06T20:44:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.330+0000 7f2f899ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f840c7ff0 0x7f2f841572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.330+0000 7f2f899ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f84154330 0x7f2f840b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.330+0000 7f2f899ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154710 0x7f2f840646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.330+0000 7f2f899ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84164ad0 con 0x7f2f84154710 2023-11-06T20:44:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.330+0000 7f2f899ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84164db0 con 0x7f2f840c7ff0 2023-11-06T20:44:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.330+0000 7f2f899ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84165090 con 0x7f2f84154330 2023-11-06T20:44:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154710 0x7f2f840646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154710 0x7f2f840646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36962/0 (socket says 172.21.15.33:36962) 2023-11-06T20:44:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f837fe700 1 -- 172.21.15.33:0/3984496331 learned_addr learned my addr 172.21.15.33:0/3984496331 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f82ffd700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f840c7ff0 0x7f2f841572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f827fc700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f84154330 0x7f2f840b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f837fe700 1 -- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f84154330 msgr2=0x7f2f840b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f837fe700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f84154330 0x7f2f840b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f837fe700 1 -- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f840c7ff0 msgr2=0x7f2f841572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f837fe700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f840c7ff0 0x7f2f841572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f837fe700 1 -- 172.21.15.33:0/3984496331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f74009370 con 0x7f2f84154710 2023-11-06T20:44:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.331+0000 7f2f837fe700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154710 0x7f2f840646c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f2f7c00b810 tx=0x7f2f7c009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.332+0000 7f2f6affd700 1 -- 172.21.15.33:0/3984496331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00d600 con 0x7f2f84154710 2023-11-06T20:44:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.332+0000 7f2f6affd700 1 -- 172.21.15.33:0/3984496331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c00d780 con 0x7f2f84154710 2023-11-06T20:44:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.332+0000 7f2f6affd700 1 -- 172.21.15.33:0/3984496331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00db40 con 0x7f2f84154710 2023-11-06T20:44:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.332+0000 7f2f899ff700 1 -- 172.21.15.33:0/3984496331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f84165370 con 0x7f2f84154710 2023-11-06T20:44:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.332+0000 7f2f899ff700 1 -- 172.21.15.33:0/3984496331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f841577d0 con 0x7f2f84154710 2023-11-06T20:44:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.333+0000 7f2f899ff700 1 -- 172.21.15.33:0/3984496331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f84165890 con 0x7f2f84154710 2023-11-06T20:44:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.334+0000 7f2f6affd700 1 -- 172.21.15.33:0/3984496331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2f7c019460 con 0x7f2f84154710 2023-11-06T20:44:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.334+0000 7f2f6affd700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f7005f150 0x7f2f70061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.334+0000 7f2f6affd700 1 -- 172.21.15.33:0/3984496331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2f7c07a2d0 con 0x7f2f84154710 2023-11-06T20:44:38.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.335+0000 7f2f82ffd700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f7005f150 0x7f2f70061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.335+0000 7f2f82ffd700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f7005f150 0x7f2f70061610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f2f74009130 tx=0x7f2f74015000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:38.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.338+0000 7f2f6affd700 1 -- 172.21.15.33:0/3984496331 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2f84165890 con 0x7f2f84154710 2023-11-06T20:44:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.523+0000 7f2f899ff700 1 -- 172.21.15.33:0/3984496331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f84165890 con 0x7f2f84154710 2023-11-06T20:44:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.523+0000 7f2f6affd700 1 -- 172.21.15.33:0/3984496331 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f84165890 con 0x7f2f84154710 2023-11-06T20:44:38.524 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:38.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.525+0000 7f2f899ff700 1 -- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f7005f150 msgr2=0x7f2f70061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.525+0000 7f2f899ff700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f7005f150 0x7f2f70061610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f2f74009130 tx=0x7f2f74015000).stop 2023-11-06T20:44:38.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.525+0000 7f2f899ff700 1 -- 172.21.15.33:0/3984496331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154710 msgr2=0x7f2f840646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.525+0000 7f2f899ff700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154710 0x7f2f840646c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f2f7c00b810 tx=0x7f2f7c009290).stop 2023-11-06T20:44:38.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.526+0000 7f2f899ff700 1 -- 172.21.15.33:0/3984496331 shutdown_connections 2023-11-06T20:44:38.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.526+0000 7f2f899ff700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2f7005f150 0x7f2f70061610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.526+0000 7f2f899ff700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f84154710 0x7f2f840646c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.526+0000 7f2f899ff700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f840c7ff0 0x7f2f841572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.526+0000 7f2f899ff700 1 --2- 172.21.15.33:0/3984496331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f84154330 0x7f2f840b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.526+0000 7f2f899ff700 1 -- 172.21.15.33:0/3984496331 >> 172.21.15.33:0/3984496331 conn(0x7f2f840cb910 msgr2=0x7f2f840c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:38.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.526+0000 7f2f899ff700 1 -- 172.21.15.33:0/3984496331 shutdown_connections 2023-11-06T20:44:38.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.526+0000 7f2f899ff700 1 -- 172.21.15.33:0/3984496331 wait complete. 2023-11-06T20:44:38.544 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:38.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.739+0000 7fd509fff700 1 Processor -- start 2023-11-06T20:44:38.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.739+0000 7fd509fff700 1 -- start start 2023-11-06T20:44:38.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.740+0000 7fd509fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 0x7fd5040a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.740+0000 7fd509fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5040aa2d0 0x7fd5040c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.740+0000 7fd509fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd5040c6240 0x7fd504157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.740+0000 7fd509fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd504154960 con 0x7fd5040aa2d0 2023-11-06T20:44:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.740+0000 7fd509fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5041554a0 con 0x7fd5040c6240 2023-11-06T20:44:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.740+0000 7fd509fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd504155930 con 0x7fd5040c1840 2023-11-06T20:44:38.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd502ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5040aa2d0 0x7fd5040c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd502ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5040aa2d0 0x7fd5040c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36970/0 (socket says 172.21.15.33:36970) 2023-11-06T20:44:38.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd502ffd700 1 -- 172.21.15.33:0/1484333426 learned_addr learned my addr 172.21.15.33:0/1484333426 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd503fff700 1 --2- 172.21.15.33:0/1484333426 >> v2:172.21.15.69:3300/0 conn(0x7fd5040c6240 0x7fd504157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd5037fe700 1 --2- 172.21.15.33:0/1484333426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 0x7fd5040a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd502ffd700 1 -- 172.21.15.33:0/1484333426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 msgr2=0x7fd5040a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:38.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd502ffd700 1 --2- 172.21.15.33:0/1484333426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 0x7fd5040a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd502ffd700 1 -- 172.21.15.33:0/1484333426 >> v2:172.21.15.69:3300/0 conn(0x7fd5040c6240 msgr2=0x7fd504157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:38.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd502ffd700 1 --2- 172.21.15.33:0/1484333426 >> v2:172.21.15.69:3300/0 conn(0x7fd5040c6240 0x7fd504157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd502ffd700 1 -- 172.21.15.33:0/1484333426 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd504154b70 con 0x7fd5040aa2d0 2023-11-06T20:44:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd503fff700 1 --2- 172.21.15.33:0/1484333426 >> v2:172.21.15.69:3300/0 conn(0x7fd5040c6240 0x7fd504157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd5037fe700 1 --2- 172.21.15.33:0/1484333426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 0x7fd5040a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd502ffd700 1 --2- 172.21.15.33:0/1484333426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040aa2d0 0x7fd5040c5d50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fd504159040 tx=0x7fd4f80090f0).ready entity=mon.0 client_cookie=515c36c9b4b24ccc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.741+0000 7fd5027fc700 1 -- 172.21.15.33:0/1484333426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800b600 con 0x7fd5040aa2d0 2023-11-06T20:44:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd5027fc700 1 -- 172.21.15.33:0/1484333426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f800b760 con 0x7fd5040aa2d0 2023-11-06T20:44:38.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd5027fc700 1 -- 172.21.15.33:0/1484333426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800bae0 con 0x7fd5040aa2d0 2023-11-06T20:44:38.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd509fff700 1 -- 172.21.15.33:0/1484333426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040aa2d0 msgr2=0x7fd5040c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd509fff700 1 --2- 172.21.15.33:0/1484333426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040aa2d0 0x7fd5040c5d50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fd504159040 tx=0x7fd4f80090f0).stop 2023-11-06T20:44:38.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd509fff700 1 -- 172.21.15.33:0/1484333426 shutdown_connections 2023-11-06T20:44:38.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd509fff700 1 --2- 172.21.15.33:0/1484333426 >> v2:172.21.15.69:3300/0 conn(0x7fd5040c6240 0x7fd504157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd509fff700 1 --2- 172.21.15.33:0/1484333426 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 0x7fd5040a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd509fff700 1 --2- 172.21.15.33:0/1484333426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040aa2d0 0x7fd5040c5d50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd509fff700 1 -- 172.21.15.33:0/1484333426 >> 172.21.15.33:0/1484333426 conn(0x7fd50405ec80 msgr2=0x7fd5040cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd509fff700 1 -- 172.21.15.33:0/1484333426 shutdown_connections 2023-11-06T20:44:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.742+0000 7fd509fff700 1 -- 172.21.15.33:0/1484333426 wait complete. 2023-11-06T20:44:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.744+0000 7fd509fff700 1 Processor -- start 2023-11-06T20:44:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.744+0000 7fd509fff700 1 -- start start 2023-11-06T20:44:38.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd509fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5040aa2d0 0x7fd50414e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd509fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 0x7fd5040c2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd509fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040c6240 0x7fd5040c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd509fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040c3f90 con 0x7fd5040c6240 2023-11-06T20:44:38.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd509fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040c3380 con 0x7fd5040aa2d0 2023-11-06T20:44:38.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd509fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040c3660 con 0x7fd5040c1840 2023-11-06T20:44:38.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd503fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040c6240 0x7fd5040c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd503fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040c6240 0x7fd5040c2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36986/0 (socket says 172.21.15.33:36986) 2023-11-06T20:44:38.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd503fff700 1 -- 172.21.15.33:0/2713362086 learned_addr learned my addr 172.21.15.33:0/2713362086 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:38.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd5037fe700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5040aa2d0 0x7fd50414e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.745+0000 7fd502ffd700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 0x7fd5040c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd503fff700 1 -- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 msgr2=0x7fd5040c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd503fff700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 0x7fd5040c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd503fff700 1 -- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5040aa2d0 msgr2=0x7fd50414e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd503fff700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5040aa2d0 0x7fd50414e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd503fff700 1 -- 172.21.15.33:0/2713362086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8009370 con 0x7fd5040c6240 2023-11-06T20:44:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd5037fe700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5040aa2d0 0x7fd50414e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd502ffd700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 0x7fd5040c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd503fff700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040c6240 0x7fd5040c2d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fd4fc00cef0 tx=0x7fd4fc007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd4eb7fe700 1 -- 172.21.15.33:0/2713362086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc00ae00 con 0x7fd5040c6240 2023-11-06T20:44:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd4eb7fe700 1 -- 172.21.15.33:0/2713362086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc016400 con 0x7fd5040c6240 2023-11-06T20:44:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd509fff700 1 -- 172.21.15.33:0/2713362086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5040c3940 con 0x7fd5040c6240 2023-11-06T20:44:38.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd4eb7fe700 1 -- 172.21.15.33:0/2713362086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc016780 con 0x7fd5040c6240 2023-11-06T20:44:38.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.746+0000 7fd509fff700 1 -- 172.21.15.33:0/2713362086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50414ee20 con 0x7fd5040c6240 2023-11-06T20:44:38.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.747+0000 7fd509fff700 1 -- 172.21.15.33:0/2713362086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50406c6c0 con 0x7fd5040c6240 2023-11-06T20:44:38.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.748+0000 7fd4eb7fe700 1 -- 172.21.15.33:0/2713362086 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd4fc01f3f0 con 0x7fd5040c6240 2023-11-06T20:44:38.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.749+0000 7fd4eb7fe700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4f005f150 0x7fd4f0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:38.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.749+0000 7fd4eb7fe700 1 -- 172.21.15.33:0/2713362086 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd4fc07a910 con 0x7fd5040c6240 2023-11-06T20:44:38.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.749+0000 7fd5037fe700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4f005f150 0x7fd4f0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:38.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.750+0000 7fd5037fe700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4f005f150 0x7fd4f0061610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fd4f400aa90 tx=0x7fd4f4008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:38.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.753+0000 7fd4eb7fe700 1 -- 172.21.15.33:0/2713362086 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd4fc04bb80 con 0x7fd5040c6240 2023-11-06T20:44:38.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.942+0000 7fd509fff700 1 -- 172.21.15.33:0/2713362086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd504063f20 con 0x7fd5040c6240 2023-11-06T20:44:38.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.943+0000 7fd4eb7fe700 1 -- 172.21.15.33:0/2713362086 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fd4fc04f1a0 con 0x7fd5040c6240 2023-11-06T20:44:38.943 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:38.943 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:38.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 -- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4f005f150 msgr2=0x7fd4f0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4f005f150 0x7fd4f0061610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fd4f400aa90 tx=0x7fd4f4008040).stop 2023-11-06T20:44:38.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 -- 172.21.15.33:0/2713362086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040c6240 msgr2=0x7fd5040c2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:38.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040c6240 0x7fd5040c2d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fd4fc00cef0 tx=0x7fd4fc007860).stop 2023-11-06T20:44:38.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 -- 172.21.15.33:0/2713362086 shutdown_connections 2023-11-06T20:44:38.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4f005f150 0x7fd4f0061610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5040c6240 0x7fd5040c2d70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5040aa2d0 0x7fd50414e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 --2- 172.21.15.33:0/2713362086 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5040c1840 0x7fd5040c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:38.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 -- 172.21.15.33:0/2713362086 >> 172.21.15.33:0/2713362086 conn(0x7fd50405ec80 msgr2=0x7fd504152d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:38.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 -- 172.21.15.33:0/2713362086 shutdown_connections 2023-11-06T20:44:38.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:38.945+0000 7fd509fff700 1 -- 172.21.15.33:0/2713362086 wait complete. 2023-11-06T20:44:38.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:44:04.716205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:44:05.351480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T20:44:38.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:44:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.162+0000 7efd7adff700 1 Processor -- start 2023-11-06T20:44:39.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.163+0000 7efd7adff700 1 -- start start 2023-11-06T20:44:39.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.164+0000 7efd7adff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd74154710 0x7efd740c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.164+0000 7efd7adff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efd740c7ff0 0x7efd740646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.164+0000 7efd7adff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd740c6b60 con 0x7efd74154330 2023-11-06T20:44:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.164+0000 7efd7adff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd740c7700 con 0x7efd74154710 2023-11-06T20:44:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.164+0000 7efd7adff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd740c5cd0 con 0x7efd740c7ff0 2023-11-06T20:44:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.164+0000 7efd73fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd74154710 0x7efd740c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.164+0000 7efd73fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd74154710 0x7efd740c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36988/0 (socket says 172.21.15.33:36988) 2023-11-06T20:44:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.164+0000 7efd73fff700 1 -- 172.21.15.33:0/4182086523 learned_addr learned my addr 172.21.15.33:0/4182086523 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:39.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.164+0000 7efd7939c700 1 --2- 172.21.15.33:0/4182086523 >> v2:172.21.15.69:3300/0 conn(0x7efd740c7ff0 0x7efd740646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.165+0000 7efd73fff700 1 -- 172.21.15.33:0/4182086523 >> v2:172.21.15.69:3300/0 conn(0x7efd740c7ff0 msgr2=0x7efd740646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:39.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.165+0000 7efd73fff700 1 --2- 172.21.15.33:0/4182086523 >> v2:172.21.15.69:3300/0 conn(0x7efd740c7ff0 0x7efd740646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.165+0000 7efd73fff700 1 -- 172.21.15.33:0/4182086523 >> v1:172.21.15.33:6789/0 conn(0x7efd74154330 legacy=0x7efd740c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.165+0000 7efd73fff700 1 -- 172.21.15.33:0/4182086523 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd740c5f50 con 0x7efd74154710 2023-11-06T20:44:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.165+0000 7efd737fe700 1 -- 172.21.15.33:0/4182086523 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997270339 0 0) 0x7efd740c6b60 con 0x7efd74154330 2023-11-06T20:44:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.165+0000 7efd73fff700 1 --2- 172.21.15.33:0/4182086523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 0x7efd740c57e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7efd740c1e60 tx=0x7efd600090f0).ready entity=mon.0 client_cookie=ebb7a4e71c7ba3b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.165+0000 7efd737fe700 1 -- 172.21.15.33:0/4182086523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6000b520 con 0x7efd74154710 2023-11-06T20:44:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.165+0000 7efd737fe700 1 -- 172.21.15.33:0/4182086523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6000b680 con 0x7efd74154710 2023-11-06T20:44:39.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.165+0000 7efd737fe700 1 -- 172.21.15.33:0/4182086523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6000ba00 con 0x7efd74154710 2023-11-06T20:44:39.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.166+0000 7efd7adff700 1 -- 172.21.15.33:0/4182086523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 msgr2=0x7efd740c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.166+0000 7efd7adff700 1 --2- 172.21.15.33:0/4182086523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 0x7efd740c57e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7efd740c1e60 tx=0x7efd600090f0).stop 2023-11-06T20:44:39.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.166+0000 7efd7adff700 1 -- 172.21.15.33:0/4182086523 shutdown_connections 2023-11-06T20:44:39.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.166+0000 7efd7adff700 1 --2- 172.21.15.33:0/4182086523 >> v2:172.21.15.69:3300/0 conn(0x7efd740c7ff0 0x7efd740646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.166+0000 7efd7adff700 1 --2- 172.21.15.33:0/4182086523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 0x7efd740c57e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.166+0000 7efd7adff700 1 -- 172.21.15.33:0/4182086523 >> 172.21.15.33:0/4182086523 conn(0x7efd740cb910 msgr2=0x7efd740c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:39.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.166+0000 7efd7adff700 1 -- 172.21.15.33:0/4182086523 shutdown_connections 2023-11-06T20:44:39.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.166+0000 7efd7adff700 1 -- 172.21.15.33:0/4182086523 wait complete. 2023-11-06T20:44:39.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.168+0000 7efd7adff700 1 Processor -- start 2023-11-06T20:44:39.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.168+0000 7efd7adff700 1 -- start start 2023-11-06T20:44:39.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.169+0000 7efd7adff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd740c7ff0 0x7efd740ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.169+0000 7efd7adff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd74154330 0x7efd741509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.169+0000 7efd7adff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 0x7efd7414dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.169+0000 7efd7adff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd741569a0 con 0x7efd74154710 2023-11-06T20:44:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.169+0000 7efd7adff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740aed80 con 0x7efd74154330 2023-11-06T20:44:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.169+0000 7efd7adff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd74156070 con 0x7efd740c7ff0 2023-11-06T20:44:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd7939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 0x7efd7414dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd7939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 0x7efd7414dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36994/0 (socket says 172.21.15.33:36994) 2023-11-06T20:44:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd7939c700 1 -- 172.21.15.33:0/1318723893 learned_addr learned my addr 172.21.15.33:0/1318723893 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd73fff700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd74154330 0x7efd741509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd78b9b700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd740c7ff0 0x7efd740ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd7939c700 1 -- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd740c7ff0 msgr2=0x7efd740ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd7939c700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd740c7ff0 0x7efd740ae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd7939c700 1 -- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd74154330 msgr2=0x7efd741509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd7939c700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd74154330 0x7efd741509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd7939c700 1 -- 172.21.15.33:0/1318723893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd60009370 con 0x7efd74154710 2023-11-06T20:44:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.170+0000 7efd7939c700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 0x7efd7414dc50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7efd6c009e90 tx=0x7efd6c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.171+0000 7efd70ff9700 1 -- 172.21.15.33:0/1318723893 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00caa0 con 0x7efd74154710 2023-11-06T20:44:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.171+0000 7efd70ff9700 1 -- 172.21.15.33:0/1318723893 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c00f040 con 0x7efd74154710 2023-11-06T20:44:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.171+0000 7efd70ff9700 1 -- 172.21.15.33:0/1318723893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00ce10 con 0x7efd74154710 2023-11-06T20:44:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.171+0000 7efd7adff700 1 -- 172.21.15.33:0/1318723893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd74156350 con 0x7efd74154710 2023-11-06T20:44:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.171+0000 7efd7adff700 1 -- 172.21.15.33:0/1318723893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd74150ee0 con 0x7efd74154710 2023-11-06T20:44:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.172+0000 7efd7adff700 1 -- 172.21.15.33:0/1318723893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd74156b10 con 0x7efd74154710 2023-11-06T20:44:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.173+0000 7efd70ff9700 1 -- 172.21.15.33:0/1318723893 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7efd6c013610 con 0x7efd74154710 2023-11-06T20:44:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.174+0000 7efd70ff9700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd5805f150 0x7efd58061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.174+0000 7efd70ff9700 1 -- 172.21.15.33:0/1318723893 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efd6c07ab10 con 0x7efd74154710 2023-11-06T20:44:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.174+0000 7efd78b9b700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd5805f150 0x7efd58061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.175+0000 7efd78b9b700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd5805f150 0x7efd58061610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7efd68007fd0 tx=0x7efd68009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.178+0000 7efd70ff9700 1 -- 172.21.15.33:0/1318723893 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efd6c04bd20 con 0x7efd74154710 2023-11-06T20:44:39.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.356+0000 7efd7adff700 1 -- 172.21.15.33:0/1318723893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efd740830b0 con 0x7efd74154710 2023-11-06T20:44:39.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.357+0000 7efd70ff9700 1 -- 172.21.15.33:0/1318723893 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7efd6c04f340 con 0x7efd74154710 2023-11-06T20:44:39.357 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:44:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.358+0000 7efd7adff700 1 -- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd5805f150 msgr2=0x7efd58061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.358+0000 7efd7adff700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd5805f150 0x7efd58061610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7efd68007fd0 tx=0x7efd68009040).stop 2023-11-06T20:44:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.358+0000 7efd7adff700 1 -- 172.21.15.33:0/1318723893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 msgr2=0x7efd7414dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.358+0000 7efd7adff700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 0x7efd7414dc50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7efd6c009e90 tx=0x7efd6c00b040).stop 2023-11-06T20:44:39.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.359+0000 7efd7adff700 1 -- 172.21.15.33:0/1318723893 shutdown_connections 2023-11-06T20:44:39.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.359+0000 7efd7adff700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd5805f150 0x7efd58061610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.359+0000 7efd7adff700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd740c7ff0 0x7efd740ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.359+0000 7efd7adff700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74154710 0x7efd7414dc50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.359+0000 7efd7adff700 1 --2- 172.21.15.33:0/1318723893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd74154330 0x7efd741509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.359+0000 7efd7adff700 1 -- 172.21.15.33:0/1318723893 >> 172.21.15.33:0/1318723893 conn(0x7efd740cb910 msgr2=0x7efd740ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:39.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.359+0000 7efd7adff700 1 -- 172.21.15.33:0/1318723893 shutdown_connections 2023-11-06T20:44:39.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.359+0000 7efd7adff700 1 -- 172.21.15.33:0/1318723893 wait complete. 2023-11-06T20:44:39.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:44:39.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.574+0000 7fd0477ff700 1 Processor -- start 2023-11-06T20:44:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.574+0000 7fd0477ff700 1 -- start start 2023-11-06T20:44:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.576+0000 7fd0477ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd04014e420 0x7fd04014e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.576+0000 7fd0477ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd0400c7d50 0x7fd0400c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.576+0000 7fd0477ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0400ad980 con 0x7fd0400c5dc0 2023-11-06T20:44:39.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.576+0000 7fd0477ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd040158880 con 0x7fd0400c7d50 2023-11-06T20:44:39.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.576+0000 7fd0477ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd040157c90 con 0x7fd04014e420 2023-11-06T20:44:39.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.576+0000 7fd044d9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd0400c5dc0 0x7fd0400c7640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43824/0 (socket says 172.21.15.33:43824) 2023-11-06T20:44:39.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.576+0000 7fd044d9a700 1 -- 172.21.15.33:0/949333852 learned_addr learned my addr 172.21.15.33:0/949333852 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:39.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.576+0000 7fd045d9c700 1 --2- 172.21.15.33:0/949333852 >> v2:172.21.15.69:3300/0 conn(0x7fd0400c7d50 0x7fd0400c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.576+0000 7fd04559b700 1 --2- 172.21.15.33:0/949333852 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd04014e420 0x7fd04014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.577+0000 7fd037fff700 1 -- 172.21.15.33:0/949333852 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986685070 0 0) 0x7fd0400ad980 con 0x7fd0400c5dc0 2023-11-06T20:44:39.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.577+0000 7fd037fff700 1 -- 172.21.15.33:0/949333852 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd02c0033f0 con 0x7fd0400c5dc0 2023-11-06T20:44:39.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.577+0000 7fd045d9c700 1 -- 172.21.15.33:0/949333852 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd04014e420 msgr2=0x7fd04014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:39.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.577+0000 7fd045d9c700 1 --2- 172.21.15.33:0/949333852 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd04014e420 0x7fd04014e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.577+0000 7fd045d9c700 1 -- 172.21.15.33:0/949333852 >> v1:172.21.15.33:6789/0 conn(0x7fd0400c5dc0 legacy=0x7fd0400c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.577+0000 7fd045d9c700 1 -- 172.21.15.33:0/949333852 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd040157f50 con 0x7fd0400c7d50 2023-11-06T20:44:39.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.577+0000 7fd045d9c700 1 --2- 172.21.15.33:0/949333852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0400c7d50 0x7fd0400c6240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd038009e60 tx=0x7fd038009780).ready entity=mon.1 client_cookie=d2b70696914a53a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:39.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.577+0000 7fd037fff700 1 -- 172.21.15.33:0/949333852 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03800e070 con 0x7fd0400c7d50 2023-11-06T20:44:39.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.577+0000 7fd037fff700 1 -- 172.21.15.33:0/949333852 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03800bab0 con 0x7fd0400c7d50 2023-11-06T20:44:39.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.577+0000 7fd037fff700 1 -- 172.21.15.33:0/949333852 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03800be30 con 0x7fd0400c7d50 2023-11-06T20:44:39.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.578+0000 7fd0477ff700 1 -- 172.21.15.33:0/949333852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0400c7d50 msgr2=0x7fd0400c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.578+0000 7fd0477ff700 1 --2- 172.21.15.33:0/949333852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0400c7d50 0x7fd0400c6240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd038009e60 tx=0x7fd038009780).stop 2023-11-06T20:44:39.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.578+0000 7fd0477ff700 1 -- 172.21.15.33:0/949333852 shutdown_connections 2023-11-06T20:44:39.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.578+0000 7fd0477ff700 1 --2- 172.21.15.33:0/949333852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0400c7d50 0x7fd0400c6240 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.578+0000 7fd0477ff700 1 --2- 172.21.15.33:0/949333852 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd04014e420 0x7fd04014e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.578+0000 7fd0477ff700 1 -- 172.21.15.33:0/949333852 >> 172.21.15.33:0/949333852 conn(0x7fd04005e1b0 msgr2=0x7fd04005e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:39.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.578+0000 7fd0477ff700 1 -- 172.21.15.33:0/949333852 shutdown_connections 2023-11-06T20:44:39.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.578+0000 7fd0477ff700 1 -- 172.21.15.33:0/949333852 wait complete. 2023-11-06T20:44:39.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.579+0000 7fd0477ff700 1 Processor -- start 2023-11-06T20:44:39.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.579+0000 7fd0477ff700 1 -- start start 2023-11-06T20:44:39.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.580+0000 7fd0477ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400c5dc0 0x7fd040150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.580+0000 7fd0477ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0400c7d50 0x7fd04014dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.580+0000 7fd0477ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04014e420 0x7fd04014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.580+0000 7fd0477ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04014fcf0 con 0x7fd0400c5dc0 2023-11-06T20:44:39.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.580+0000 7fd0477ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04014e0a0 con 0x7fd04014e420 2023-11-06T20:44:39.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd0477ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04014f0e0 con 0x7fd0400c7d50 2023-11-06T20:44:39.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd04559b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400c5dc0 0x7fd040150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd04559b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400c5dc0 0x7fd040150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37002/0 (socket says 172.21.15.33:37002) 2023-11-06T20:44:39.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd04559b700 1 -- 172.21.15.33:0/715295415 learned_addr learned my addr 172.21.15.33:0/715295415 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:39.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd044d9a700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0400c7d50 0x7fd04014dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd045d9c700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04014e420 0x7fd04014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd04559b700 1 -- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0400c7d50 msgr2=0x7fd04014dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd04559b700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0400c7d50 0x7fd04014dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd04559b700 1 -- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04014e420 msgr2=0x7fd04014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd04559b700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04014e420 0x7fd04014e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd04559b700 1 -- 172.21.15.33:0/715295415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd038009a20 con 0x7fd0400c5dc0 2023-11-06T20:44:39.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd044d9a700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0400c7d50 0x7fd04014dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:39.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd045d9c700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04014e420 0x7fd04014e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:39.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.581+0000 7fd04559b700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400c5dc0 0x7fd040150180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fd03c00e900 tx=0x7fd03c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:39.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.582+0000 7fd0357fa700 1 -- 172.21.15.33:0/715295415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00c640 con 0x7fd0400c5dc0 2023-11-06T20:44:39.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.582+0000 7fd0357fa700 1 -- 172.21.15.33:0/715295415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c00c7a0 con 0x7fd0400c5dc0 2023-11-06T20:44:39.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.582+0000 7fd0477ff700 1 -- 172.21.15.33:0/715295415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04014f3c0 con 0x7fd0400c5dc0 2023-11-06T20:44:39.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.582+0000 7fd0357fa700 1 -- 172.21.15.33:0/715295415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00cb20 con 0x7fd0400c5dc0 2023-11-06T20:44:39.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.582+0000 7fd0477ff700 1 -- 172.21.15.33:0/715295415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04014f980 con 0x7fd0400c5dc0 2023-11-06T20:44:39.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.583+0000 7fd0477ff700 1 -- 172.21.15.33:0/715295415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04014ffd0 con 0x7fd0400c5dc0 2023-11-06T20:44:39.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.584+0000 7fd0357fa700 1 -- 172.21.15.33:0/715295415 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd03c01b400 con 0x7fd0400c5dc0 2023-11-06T20:44:39.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.585+0000 7fd0357fa700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd02405f150 0x7fd024061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.585+0000 7fd0357fa700 1 -- 172.21.15.33:0/715295415 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd03c07a640 con 0x7fd0400c5dc0 2023-11-06T20:44:39.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.585+0000 7fd044d9a700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd02405f150 0x7fd024061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.586+0000 7fd044d9a700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd02405f150 0x7fd024061610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fd0400c4e50 tx=0x7fd030009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:39.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.589+0000 7fd0357fa700 1 -- 172.21.15.33:0/715295415 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd03c04b8b0 con 0x7fd0400c5dc0 2023-11-06T20:44:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.769+0000 7fd0477ff700 1 -- 172.21.15.33:0/715295415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd04006c6c0 con 0x7fd0400c5dc0 2023-11-06T20:44:39.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.770+0000 7fd0357fa700 1 -- 172.21.15.33:0/715295415 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd03c016070 con 0x7fd0400c5dc0 2023-11-06T20:44:39.770 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:44:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.771+0000 7fd0477ff700 1 -- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd02405f150 msgr2=0x7fd024061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.771+0000 7fd0477ff700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd02405f150 0x7fd024061610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fd0400c4e50 tx=0x7fd030009040).stop 2023-11-06T20:44:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.771+0000 7fd0477ff700 1 -- 172.21.15.33:0/715295415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400c5dc0 msgr2=0x7fd040150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.771+0000 7fd0477ff700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400c5dc0 0x7fd040150180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fd03c00e900 tx=0x7fd03c009060).stop 2023-11-06T20:44:39.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.771+0000 7fd0477ff700 1 -- 172.21.15.33:0/715295415 shutdown_connections 2023-11-06T20:44:39.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.772+0000 7fd0477ff700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd02405f150 0x7fd024061610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.772+0000 7fd0477ff700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04014e420 0x7fd04014e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.772+0000 7fd0477ff700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400c5dc0 0x7fd040150180 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.772+0000 7fd0477ff700 1 --2- 172.21.15.33:0/715295415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0400c7d50 0x7fd04014dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.772+0000 7fd0477ff700 1 -- 172.21.15.33:0/715295415 >> 172.21.15.33:0/715295415 conn(0x7fd04005e1b0 msgr2=0x7fd040156fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:39.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.772+0000 7fd0477ff700 1 -- 172.21.15.33:0/715295415 shutdown_connections 2023-11-06T20:44:39.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.772+0000 7fd0477ff700 1 -- 172.21.15.33:0/715295415 wait complete. 2023-11-06T20:44:39.791 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T20:44:39.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.989+0000 7fe1f2dff700 1 Processor -- start 2023-11-06T20:44:39.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.989+0000 7fe1f2dff700 1 -- start start 2023-11-06T20:44:39.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.990+0000 7fe1f2dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1ec154330 0x7fe1ec0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.990+0000 7fe1f2dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 0x7fe1ec0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:39.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.990+0000 7fe1f2dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ec0c6b60 con 0x7fe1ec154710 2023-11-06T20:44:39.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.990+0000 7fe1f2dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ec0c7700 con 0x7fe1ec154330 2023-11-06T20:44:39.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.990+0000 7fe1f2dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec0c5cd0 con 0x7fe1ec0c7ff0 2023-11-06T20:44:39.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.990+0000 7fe1f0b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1ec154330 0x7fe1ec0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1f0b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1ec154330 0x7fe1ec0c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37014/0 (socket says 172.21.15.33:37014) 2023-11-06T20:44:39.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1f0b9b700 1 -- 172.21.15.33:0/3138108480 learned_addr learned my addr 172.21.15.33:0/3138108480 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:39.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1f139c700 1 --2- 172.21.15.33:0/3138108480 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 0x7fe1ec0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:39.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1f0b9b700 1 -- 172.21.15.33:0/3138108480 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 msgr2=0x7fe1ec0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:39.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1f0b9b700 1 --2- 172.21.15.33:0/3138108480 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 0x7fe1ec0646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1f0b9b700 1 -- 172.21.15.33:0/3138108480 >> v1:172.21.15.69:6789/0 conn(0x7fe1ec154710 legacy=0x7fe1ec0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1f0b9b700 1 -- 172.21.15.33:0/3138108480 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec0c5f50 con 0x7fe1ec154330 2023-11-06T20:44:39.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1f139c700 1 --2- 172.21.15.33:0/3138108480 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 0x7fe1ec0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:39.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1f0b9b700 1 --2- 172.21.15.33:0/3138108480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154330 0x7fe1ec0c57e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fe1e0006fd0 tx=0x7fe1e00090f0).ready entity=mon.0 client_cookie=7177a18a04400c86 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:39.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1eb7fe700 1 -- 172.21.15.33:0/3138108480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000b600 con 0x7fe1ec154330 2023-11-06T20:44:39.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1eb7fe700 1 -- 172.21.15.33:0/3138108480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e000b760 con 0x7fe1ec154330 2023-11-06T20:44:39.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.991+0000 7fe1eb7fe700 1 -- 172.21.15.33:0/3138108480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000bae0 con 0x7fe1ec154330 2023-11-06T20:44:39.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.992+0000 7fe1f2dff700 1 -- 172.21.15.33:0/3138108480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154330 msgr2=0x7fe1ec0c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:39.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.992+0000 7fe1f2dff700 1 --2- 172.21.15.33:0/3138108480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154330 0x7fe1ec0c57e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fe1e0006fd0 tx=0x7fe1e00090f0).stop 2023-11-06T20:44:39.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.992+0000 7fe1f2dff700 1 -- 172.21.15.33:0/3138108480 shutdown_connections 2023-11-06T20:44:39.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.992+0000 7fe1f2dff700 1 --2- 172.21.15.33:0/3138108480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154330 0x7fe1ec0c57e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.992+0000 7fe1f2dff700 1 --2- 172.21.15.33:0/3138108480 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 0x7fe1ec0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:39.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.992+0000 7fe1f2dff700 1 -- 172.21.15.33:0/3138108480 >> 172.21.15.33:0/3138108480 conn(0x7fe1ec0cb910 msgr2=0x7fe1ec0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:39.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.992+0000 7fe1f2dff700 1 -- 172.21.15.33:0/3138108480 shutdown_connections 2023-11-06T20:44:39.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.992+0000 7fe1f2dff700 1 -- 172.21.15.33:0/3138108480 wait complete. 2023-11-06T20:44:40.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.993+0000 7fe1f2dff700 1 Processor -- start 2023-11-06T20:44:40.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.994+0000 7fe1f2dff700 1 -- start start 2023-11-06T20:44:40.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.994+0000 7fe1f2dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 0x7fe1ec0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f2dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1ec154330 0x7fe1ec0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f2dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154710 0x7fe1ec14cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f2dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec0b0d60 con 0x7fe1ec154710 2023-11-06T20:44:40.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f2dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec154c40 con 0x7fe1ec154330 2023-11-06T20:44:40.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f2dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec154f20 con 0x7fe1ec0c7ff0 2023-11-06T20:44:40.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f139c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154710 0x7fe1ec14cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f139c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154710 0x7fe1ec14cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37030/0 (socket says 172.21.15.33:37030) 2023-11-06T20:44:40.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f139c700 1 -- 172.21.15.33:0/2246487202 learned_addr learned my addr 172.21.15.33:0/2246487202 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:40.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f0b9b700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 0x7fe1ec0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1ebfff700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1ec154330 0x7fe1ec0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f139c700 1 -- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 msgr2=0x7fe1ec0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f139c700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 0x7fe1ec0b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f139c700 1 -- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1ec154330 msgr2=0x7fe1ec0addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f139c700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1ec154330 0x7fe1ec0addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f139c700 1 -- 172.21.15.33:0/2246487202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e0009370 con 0x7fe1ec154710 2023-11-06T20:44:40.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1ebfff700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1ec154330 0x7fe1ec0addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:40.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.995+0000 7fe1f139c700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154710 0x7fe1ec14cdd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fe1e400d180 tx=0x7fe1e4009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:40.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.996+0000 7fe1e8ff9700 1 -- 172.21.15.33:0/2246487202 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400de20 con 0x7fe1ec154710 2023-11-06T20:44:40.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.996+0000 7fe1e8ff9700 1 -- 172.21.15.33:0/2246487202 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e400f040 con 0x7fe1ec154710 2023-11-06T20:44:40.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.996+0000 7fe1e8ff9700 1 -- 172.21.15.33:0/2246487202 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4013600 con 0x7fe1ec154710 2023-11-06T20:44:40.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.996+0000 7fe1f2dff700 1 -- 172.21.15.33:0/2246487202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ec155200 con 0x7fe1ec154710 2023-11-06T20:44:40.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.996+0000 7fe1f2dff700 1 -- 172.21.15.33:0/2246487202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec0ae2c0 con 0x7fe1ec154710 2023-11-06T20:44:40.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.997+0000 7fe1f2dff700 1 -- 172.21.15.33:0/2246487202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec1531e0 con 0x7fe1ec154710 2023-11-06T20:44:40.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.998+0000 7fe1e8ff9700 1 -- 172.21.15.33:0/2246487202 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe1e40139f0 con 0x7fe1ec154710 2023-11-06T20:44:40.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.999+0000 7fe1e8ff9700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe1d005f150 0x7fe1d0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.999+0000 7fe1e8ff9700 1 -- 172.21.15.33:0/2246487202 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe1e407aa50 con 0x7fe1ec154710 2023-11-06T20:44:40.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:39.999+0000 7fe1f0b9b700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe1d005f150 0x7fe1d0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.000+0000 7fe1f0b9b700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe1d005f150 0x7fe1d0061610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fe1e0009130 tx=0x7fe1e0017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:40.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.002+0000 7fe1e8ff9700 1 -- 172.21.15.33:0/2246487202 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe1e404bc00 con 0x7fe1ec154710 2023-11-06T20:44:40.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.183+0000 7fe1f2dff700 1 -- 172.21.15.33:0/2246487202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1ec0652d0 con 0x7fe1ec154710 2023-11-06T20:44:40.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.184+0000 7fe1e8ff9700 1 -- 172.21.15.33:0/2246487202 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1e404f220 con 0x7fe1ec154710 2023-11-06T20:44:40.184 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:40.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 -- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe1d005f150 msgr2=0x7fe1d0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe1d005f150 0x7fe1d0061610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fe1e0009130 tx=0x7fe1e0017000).stop 2023-11-06T20:44:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 -- 172.21.15.33:0/2246487202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154710 msgr2=0x7fe1ec14cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154710 0x7fe1ec14cdd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fe1e400d180 tx=0x7fe1e4009a50).stop 2023-11-06T20:44:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 -- 172.21.15.33:0/2246487202 shutdown_connections 2023-11-06T20:44:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe1d005f150 0x7fe1d0061610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe1ec0c7ff0 0x7fe1ec0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1ec154710 0x7fe1ec14cdd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 --2- 172.21.15.33:0/2246487202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1ec154330 0x7fe1ec0addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 -- 172.21.15.33:0/2246487202 >> 172.21.15.33:0/2246487202 conn(0x7fe1ec0cb910 msgr2=0x7fe1ec0646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:40.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 -- 172.21.15.33:0/2246487202 shutdown_connections 2023-11-06T20:44:40.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.186+0000 7fe1f2dff700 1 -- 172.21.15.33:0/2246487202 wait complete. 2023-11-06T20:44:40.204 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T20:44:40.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.401+0000 7f108efff700 1 Processor -- start 2023-11-06T20:44:40.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.402+0000 7f108efff700 1 -- start start 2023-11-06T20:44:40.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.403+0000 7f108efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10881544f0 0x7f10880c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.403+0000 7f108efff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f10880c0c40 0x7f10880646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.403+0000 7f108efff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1088064bb0 0x7f1088157f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.403+0000 7f108efff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10881548d0 con 0x7f1088064bb0 2023-11-06T20:44:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.403+0000 7f108efff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10881556d0 con 0x7f10880c0c40 2023-11-06T20:44:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.403+0000 7f108efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1088065030 con 0x7f10881544f0 2023-11-06T20:44:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.403+0000 7f108d59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1088064bb0 0x7f1088157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.403+0000 7f108d59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1088064bb0 0x7f1088157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37036/0 (socket says 172.21.15.33:37036) 2023-11-06T20:44:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.403+0000 7f108d59c700 1 -- 172.21.15.33:0/4281864042 learned_addr learned my addr 172.21.15.33:0/4281864042 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f1087fff700 1 --2- 172.21.15.33:0/4281864042 >> v2:172.21.15.69:3300/0 conn(0x7f10880c0c40 0x7f10880646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f108cd9b700 1 --2- 172.21.15.33:0/4281864042 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10881544f0 0x7f10880c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f108d59c700 1 -- 172.21.15.33:0/4281864042 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10881544f0 msgr2=0x7f10880c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f108d59c700 1 --2- 172.21.15.33:0/4281864042 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10881544f0 0x7f10880c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f108d59c700 1 -- 172.21.15.33:0/4281864042 >> v2:172.21.15.69:3300/0 conn(0x7f10880c0c40 msgr2=0x7f10880646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f108d59c700 1 --2- 172.21.15.33:0/4281864042 >> v2:172.21.15.69:3300/0 conn(0x7f10880c0c40 0x7f10880646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f108d59c700 1 -- 172.21.15.33:0/4281864042 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1088154ac0 con 0x7f1088064bb0 2023-11-06T20:44:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f1087fff700 1 --2- 172.21.15.33:0/4281864042 >> v2:172.21.15.69:3300/0 conn(0x7f10880c0c40 0x7f10880646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f108cd9b700 1 --2- 172.21.15.33:0/4281864042 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10881544f0 0x7f10880c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f108d59c700 1 --2- 172.21.15.33:0/4281864042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1088064bb0 0x7f1088157f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f108806c1a0 tx=0x7f1080009840).ready entity=mon.0 client_cookie=e2091db100d7e228 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f10877fe700 1 -- 172.21.15.33:0/4281864042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000e070 con 0x7f1088064bb0 2023-11-06T20:44:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f10877fe700 1 -- 172.21.15.33:0/4281864042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108000ba50 con 0x7f1088064bb0 2023-11-06T20:44:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.404+0000 7f10877fe700 1 -- 172.21.15.33:0/4281864042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000bdd0 con 0x7f1088064bb0 2023-11-06T20:44:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.405+0000 7f108efff700 1 -- 172.21.15.33:0/4281864042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1088064bb0 msgr2=0x7f1088157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.405+0000 7f108efff700 1 --2- 172.21.15.33:0/4281864042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1088064bb0 0x7f1088157f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f108806c1a0 tx=0x7f1080009840).stop 2023-11-06T20:44:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.405+0000 7f108efff700 1 -- 172.21.15.33:0/4281864042 shutdown_connections 2023-11-06T20:44:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.405+0000 7f108efff700 1 --2- 172.21.15.33:0/4281864042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1088064bb0 0x7f1088157f20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.405+0000 7f108efff700 1 --2- 172.21.15.33:0/4281864042 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10881544f0 0x7f10880c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.405+0000 7f108efff700 1 --2- 172.21.15.33:0/4281864042 >> v2:172.21.15.69:3300/0 conn(0x7f10880c0c40 0x7f10880646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.405+0000 7f108efff700 1 -- 172.21.15.33:0/4281864042 >> 172.21.15.33:0/4281864042 conn(0x7f10880cb900 msgr2=0x7f10880c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.405+0000 7f108efff700 1 -- 172.21.15.33:0/4281864042 shutdown_connections 2023-11-06T20:44:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.405+0000 7f108efff700 1 -- 172.21.15.33:0/4281864042 wait complete. 2023-11-06T20:44:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.406+0000 7f108efff700 1 Processor -- start 2023-11-06T20:44:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.407+0000 7f108efff700 1 -- start start 2023-11-06T20:44:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.407+0000 7f108efff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1088064bb0 0x7f1088165050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.407+0000 7f108efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10880c0c40 0x7f1088166cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108efff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10881544f0 0x7f1088167510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108efff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10880af7e0 con 0x7f10881544f0 2023-11-06T20:44:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108efff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10880c5180 con 0x7f1088064bb0 2023-11-06T20:44:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10880aee40 con 0x7f10880c0c40 2023-11-06T20:44:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108d59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10881544f0 0x7f1088167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108d59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10881544f0 0x7f1088167510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37040/0 (socket says 172.21.15.33:37040) 2023-11-06T20:44:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108d59c700 1 -- 172.21.15.33:0/2657806659 learned_addr learned my addr 172.21.15.33:0/2657806659 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f1087fff700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10880c0c40 0x7f1088166cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108cd9b700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1088064bb0 0x7f1088165050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108d59c700 1 -- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10880c0c40 msgr2=0x7f1088166cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108d59c700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10880c0c40 0x7f1088166cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108d59c700 1 -- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1088064bb0 msgr2=0x7f1088165050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108d59c700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1088064bb0 0x7f1088165050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108d59c700 1 -- 172.21.15.33:0/2657806659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1080009ae0 con 0x7f10881544f0 2023-11-06T20:44:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.408+0000 7f108d59c700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10881544f0 0x7f1088167510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f1080009fd0 tx=0x7f1080009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.409+0000 7f1084ff9700 1 -- 172.21.15.33:0/2657806659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080013e60 con 0x7f10881544f0 2023-11-06T20:44:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.409+0000 7f1084ff9700 1 -- 172.21.15.33:0/2657806659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108001e470 con 0x7f10881544f0 2023-11-06T20:44:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.409+0000 7f108efff700 1 -- 172.21.15.33:0/2657806659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10880af120 con 0x7f10881544f0 2023-11-06T20:44:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.409+0000 7f1084ff9700 1 -- 172.21.15.33:0/2657806659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108001e830 con 0x7f10881544f0 2023-11-06T20:44:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.409+0000 7f108efff700 1 -- 172.21.15.33:0/2657806659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1088165540 con 0x7f10881544f0 2023-11-06T20:44:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.410+0000 7f108efff700 1 -- 172.21.15.33:0/2657806659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10880af920 con 0x7f10881544f0 2023-11-06T20:44:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.411+0000 7f1084ff9700 1 -- 172.21.15.33:0/2657806659 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1080003040 con 0x7f10881544f0 2023-11-06T20:44:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.412+0000 7f1084ff9700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f106c05f180 0x7f106c061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.412+0000 7f1084ff9700 1 -- 172.21.15.33:0/2657806659 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f108007b990 con 0x7f10881544f0 2023-11-06T20:44:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.412+0000 7f108cd9b700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f106c05f180 0x7f106c061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.413+0000 7f108cd9b700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f106c05f180 0x7f106c061640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f107c00ab40 tx=0x7f107c008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.416+0000 7f1084ff9700 1 -- 172.21.15.33:0/2657806659 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f10880af920 con 0x7f10881544f0 2023-11-06T20:44:40.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.596+0000 7f108efff700 1 -- 172.21.15.33:0/2657806659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10880af920 con 0x7f10881544f0 2023-11-06T20:44:40.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.596+0000 7f1084ff9700 1 -- 172.21.15.33:0/2657806659 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10880af920 con 0x7f10881544f0 2023-11-06T20:44:40.597 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:40.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.598+0000 7f108efff700 1 -- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f106c05f180 msgr2=0x7f106c061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.598+0000 7f108efff700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f106c05f180 0x7f106c061640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f107c00ab40 tx=0x7f107c008040).stop 2023-11-06T20:44:40.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.598+0000 7f108efff700 1 -- 172.21.15.33:0/2657806659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10881544f0 msgr2=0x7f1088167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.598+0000 7f108efff700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10881544f0 0x7f1088167510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f1080009fd0 tx=0x7f1080009840).stop 2023-11-06T20:44:40.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.598+0000 7f108efff700 1 -- 172.21.15.33:0/2657806659 shutdown_connections 2023-11-06T20:44:40.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.598+0000 7f108efff700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f106c05f180 0x7f106c061640 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.598+0000 7f108efff700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10881544f0 0x7f1088167510 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.598+0000 7f108efff700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1088064bb0 0x7f1088165050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.598+0000 7f108efff700 1 --2- 172.21.15.33:0/2657806659 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f10880c0c40 0x7f1088166cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.598+0000 7f108efff700 1 -- 172.21.15.33:0/2657806659 >> 172.21.15.33:0/2657806659 conn(0x7f10880cb900 msgr2=0x7f10880c5650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:40.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.599+0000 7f108efff700 1 -- 172.21.15.33:0/2657806659 shutdown_connections 2023-11-06T20:44:40.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.599+0000 7f108efff700 1 -- 172.21.15.33:0/2657806659 wait complete. 2023-11-06T20:44:40.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:40.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.813+0000 7f712cfff700 1 Processor -- start 2023-11-06T20:44:40.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.813+0000 7f712cfff700 1 -- start start 2023-11-06T20:44:40.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.814+0000 7f712cfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71280c1840 0x7f71280a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.814+0000 7f712cfff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f71280aa2d0 0x7f71280c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.814+0000 7f712cfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71281554a0 con 0x7f71280c6240 2023-11-06T20:44:40.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.814+0000 7f712cfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7128154960 con 0x7f71280c1840 2023-11-06T20:44:40.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.814+0000 7f712cfff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7128155930 con 0x7f71280aa2d0 2023-11-06T20:44:40.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.814+0000 7f712659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71280c1840 0x7f71280a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.814+0000 7f7126d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f71280c6240 0x7f7128157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43832/0 (socket says 172.21.15.33:43832) 2023-11-06T20:44:40.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.814+0000 7f7126d9d700 1 -- 172.21.15.33:0/3310735708 learned_addr learned my addr 172.21.15.33:0/3310735708 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:40.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.815+0000 7f7125d9b700 1 --2- 172.21.15.33:0/3310735708 >> v2:172.21.15.69:3300/0 conn(0x7f71280aa2d0 0x7f71280c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.815+0000 7f712659c700 1 -- 172.21.15.33:0/3310735708 >> v2:172.21.15.69:3300/0 conn(0x7f71280aa2d0 msgr2=0x7f71280c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:40.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.815+0000 7f712659c700 1 --2- 172.21.15.33:0/3310735708 >> v2:172.21.15.69:3300/0 conn(0x7f71280aa2d0 0x7f71280c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.815+0000 7f712659c700 1 -- 172.21.15.33:0/3310735708 >> v1:172.21.15.33:6789/0 conn(0x7f71280c6240 legacy=0x7f7128157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.815+0000 7f712659c700 1 -- 172.21.15.33:0/3310735708 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7128154b70 con 0x7f71280c1840 2023-11-06T20:44:40.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.815+0000 7f712659c700 1 --2- 172.21.15.33:0/3310735708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c1840 0x7f71280a9de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f71280adb40 tx=0x7f7118009060).ready entity=mon.0 client_cookie=7b44cea0d7a1e43a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:40.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.815+0000 7f712559a700 1 -- 172.21.15.33:0/3310735708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800b580 con 0x7f71280c1840 2023-11-06T20:44:40.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.815+0000 7f712559a700 1 -- 172.21.15.33:0/3310735708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711800b6e0 con 0x7f71280c1840 2023-11-06T20:44:40.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.815+0000 7f712559a700 1 -- 172.21.15.33:0/3310735708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800ba60 con 0x7f71280c1840 2023-11-06T20:44:40.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.816+0000 7f712cfff700 1 -- 172.21.15.33:0/3310735708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c1840 msgr2=0x7f71280a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.816+0000 7f712cfff700 1 --2- 172.21.15.33:0/3310735708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c1840 0x7f71280a9de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f71280adb40 tx=0x7f7118009060).stop 2023-11-06T20:44:40.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.816+0000 7f712cfff700 1 -- 172.21.15.33:0/3310735708 shutdown_connections 2023-11-06T20:44:40.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.816+0000 7f712cfff700 1 --2- 172.21.15.33:0/3310735708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c1840 0x7f71280a9de0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.816+0000 7f712cfff700 1 --2- 172.21.15.33:0/3310735708 >> v2:172.21.15.69:3300/0 conn(0x7f71280aa2d0 0x7f71280c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.816+0000 7f712cfff700 1 -- 172.21.15.33:0/3310735708 >> 172.21.15.33:0/3310735708 conn(0x7f712805ec80 msgr2=0x7f71280cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.816+0000 7f712cfff700 1 -- 172.21.15.33:0/3310735708 shutdown_connections 2023-11-06T20:44:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.816+0000 7f712cfff700 1 -- 172.21.15.33:0/3310735708 wait complete. 2023-11-06T20:44:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.817+0000 7f712cfff700 1 Processor -- start 2023-11-06T20:44:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.818+0000 7f712cfff700 1 -- start start 2023-11-06T20:44:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.818+0000 7f712cfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71280aa2d0 0x7f7128152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.818+0000 7f712cfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71280c1840 0x7f7128140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.818+0000 7f712cfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c6240 0x7f7128140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.818+0000 7f712cfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7128141010 con 0x7f71280c6240 2023-11-06T20:44:40.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.818+0000 7f712cfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71280adda0 con 0x7f71280aa2d0 2023-11-06T20:44:40.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.818+0000 7f712cfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7128158190 con 0x7f71280c1840 2023-11-06T20:44:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f7126d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c6240 0x7f7128140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f7126d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c6240 0x7f7128140b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37066/0 (socket says 172.21.15.33:37066) 2023-11-06T20:44:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f7126d9d700 1 -- 172.21.15.33:0/73621980 learned_addr learned my addr 172.21.15.33:0/73621980 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f712659c700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71280aa2d0 0x7f7128152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f7125d9b700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71280c1840 0x7f7128140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f7126d9d700 1 -- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71280c1840 msgr2=0x7f7128140630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f7126d9d700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71280c1840 0x7f7128140630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f7126d9d700 1 -- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71280aa2d0 msgr2=0x7f7128152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:40.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f7126d9d700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71280aa2d0 0x7f7128152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:40.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f7126d9d700 1 -- 172.21.15.33:0/73621980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180092a0 con 0x7f71280c6240 2023-11-06T20:44:40.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.819+0000 7f7126d9d700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c6240 0x7f7128140b20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f712000c7c0 tx=0x7f7120001560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:40.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.820+0000 7f71167fc700 1 -- 172.21.15.33:0/73621980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120005d30 con 0x7f71280c6240 2023-11-06T20:44:40.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.820+0000 7f71167fc700 1 -- 172.21.15.33:0/73621980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120005e90 con 0x7f71280c6240 2023-11-06T20:44:40.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.820+0000 7f71167fc700 1 -- 172.21.15.33:0/73621980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200072e0 con 0x7f71280c6240 2023-11-06T20:44:40.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.820+0000 7f712cfff700 1 -- 172.21.15.33:0/73621980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7128158470 con 0x7f71280c6240 2023-11-06T20:44:40.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.820+0000 7f712cfff700 1 -- 172.21.15.33:0/73621980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71280ae320 con 0x7f71280c6240 2023-11-06T20:44:40.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.821+0000 7f712cfff700 1 -- 172.21.15.33:0/73621980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71280ae540 con 0x7f71280c6240 2023-11-06T20:44:40.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.822+0000 7f71167fc700 1 -- 172.21.15.33:0/73621980 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f71200076d0 con 0x7f71280c6240 2023-11-06T20:44:40.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.822+0000 7f71167fc700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f710805f150 0x7f7108061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:40.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.823+0000 7f71167fc700 1 -- 172.21.15.33:0/73621980 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7120079b20 con 0x7f71280c6240 2023-11-06T20:44:40.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.823+0000 7f712659c700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f710805f150 0x7f7108061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:40.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.824+0000 7f712659c700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f710805f150 0x7f7108061610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f711800b340 tx=0x7f7118017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:40.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:40.826+0000 7f71167fc700 1 -- 172.21.15.33:0/73621980 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f712004acd0 con 0x7f71280c6240 2023-11-06T20:44:41.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.009+0000 7f712cfff700 1 -- 172.21.15.33:0/73621980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f712806c6c0 con 0x7f71280c6240 2023-11-06T20:44:41.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.010+0000 7f71167fc700 1 -- 172.21.15.33:0/73621980 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f712004e2f0 con 0x7f71280c6240 2023-11-06T20:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:41.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 -- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f710805f150 msgr2=0x7f7108061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f710805f150 0x7f7108061610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f711800b340 tx=0x7f7118017000).stop 2023-11-06T20:44:41.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 -- 172.21.15.33:0/73621980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c6240 msgr2=0x7f7128140b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c6240 0x7f7128140b20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f712000c7c0 tx=0x7f7120001560).stop 2023-11-06T20:44:41.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 -- 172.21.15.33:0/73621980 shutdown_connections 2023-11-06T20:44:41.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f710805f150 0x7f7108061610 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71280c6240 0x7f7128140b20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71280aa2d0 0x7f7128152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 --2- 172.21.15.33:0/73621980 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71280c1840 0x7f7128140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 -- 172.21.15.33:0/73621980 >> 172.21.15.33:0/73621980 conn(0x7f712805ec80 msgr2=0x7f712814f910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.012+0000 7f712cfff700 1 -- 172.21.15.33:0/73621980 shutdown_connections 2023-11-06T20:44:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.013+0000 7f712cfff700 1 -- 172.21.15.33:0/73621980 wait complete. 2023-11-06T20:44:41.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:44:04.716205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:44:05.351480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T20:44:41.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:44:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.228+0000 7f51c39ff700 1 Processor -- start 2023-11-06T20:44:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.229+0000 7f51c39ff700 1 -- start start 2023-11-06T20:44:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.230+0000 7f51c39ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 0x7f51bc0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.230+0000 7f51c39ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f51bc154710 0x7f51bc0c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.230+0000 7f51c39ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f51bc0c7ff0 0x7f51bc0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.230+0000 7f51c39ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc0c6b60 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.230+0000 7f51c39ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc0c5cd0 con 0x7f51bc154710 2023-11-06T20:44:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.230+0000 7f51c39ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0c5f50 con 0x7f51bc154330 2023-11-06T20:44:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.230+0000 7f51c1f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f51bc0c7ff0 0x7f51bc0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.230+0000 7f51c1f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f51bc0c7ff0 0x7f51bc0646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37068/0 (socket says 172.21.15.33:37068) 2023-11-06T20:44:41.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.230+0000 7f51c1f9c700 1 -- 172.21.15.33:0/1928733412 learned_addr learned my addr 172.21.15.33:0/1928733412 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:41.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51c179b700 1 --2- 172.21.15.33:0/1928733412 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 0x7f51bc0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51c0f9a700 1 --2- 172.21.15.33:0/1928733412 >> v2:172.21.15.69:3300/0 conn(0x7f51bc154710 0x7f51bc0c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51c1f9c700 1 -- 172.21.15.33:0/1928733412 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 msgr2=0x7f51bc0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:41.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51c1f9c700 1 --2- 172.21.15.33:0/1928733412 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 0x7f51bc0c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51c1f9c700 1 -- 172.21.15.33:0/1928733412 >> v2:172.21.15.69:3300/0 conn(0x7f51bc154710 msgr2=0x7f51bc0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:41.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51c1f9c700 1 --2- 172.21.15.33:0/1928733412 >> v2:172.21.15.69:3300/0 conn(0x7f51bc154710 0x7f51bc0c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51c1f9c700 1 -- 172.21.15.33:0/1928733412 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0c6230 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51c179b700 1 --2- 172.21.15.33:0/1928733412 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 0x7f51bc0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:41.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51c0f9a700 1 --2- 172.21.15.33:0/1928733412 >> v2:172.21.15.69:3300/0 conn(0x7f51bc154710 0x7f51bc0c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:41.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51c1f9c700 1 --2- 172.21.15.33:0/1928733412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 0x7f51bc0646c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f51bc0627e0 tx=0x7f51b4009840).ready entity=mon.0 client_cookie=56061dcbff055fce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:41.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51b3fff700 1 -- 172.21.15.33:0/1928733412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b400e070 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51b3fff700 1 -- 172.21.15.33:0/1928733412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b400ba50 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.231+0000 7f51b3fff700 1 -- 172.21.15.33:0/1928733412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b400bdd0 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.232+0000 7f51c39ff700 1 -- 172.21.15.33:0/1928733412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 msgr2=0x7f51bc0646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.232+0000 7f51c39ff700 1 --2- 172.21.15.33:0/1928733412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 0x7f51bc0646c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f51bc0627e0 tx=0x7f51b4009840).stop 2023-11-06T20:44:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.232+0000 7f51c39ff700 1 -- 172.21.15.33:0/1928733412 shutdown_connections 2023-11-06T20:44:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.232+0000 7f51c39ff700 1 --2- 172.21.15.33:0/1928733412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 0x7f51bc0646c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.232+0000 7f51c39ff700 1 --2- 172.21.15.33:0/1928733412 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 0x7f51bc0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.232+0000 7f51c39ff700 1 --2- 172.21.15.33:0/1928733412 >> v2:172.21.15.69:3300/0 conn(0x7f51bc154710 0x7f51bc0c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.232+0000 7f51c39ff700 1 -- 172.21.15.33:0/1928733412 >> 172.21.15.33:0/1928733412 conn(0x7f51bc0cb910 msgr2=0x7f51bc0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.232+0000 7f51c39ff700 1 -- 172.21.15.33:0/1928733412 shutdown_connections 2023-11-06T20:44:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.232+0000 7f51c39ff700 1 -- 172.21.15.33:0/1928733412 wait complete. 2023-11-06T20:44:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.233+0000 7f51c39ff700 1 Processor -- start 2023-11-06T20:44:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.233+0000 7f51c39ff700 1 -- start start 2023-11-06T20:44:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.234+0000 7f51c39ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 0x7f51bc153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.234+0000 7f51c39ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 0x7f51bc0ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.234+0000 7f51c39ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc154710 0x7f51bc155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.234+0000 7f51c39ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc1647c0 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.234+0000 7f51c39ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0af210 con 0x7f51bc154710 2023-11-06T20:44:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.234+0000 7f51c39ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0af4f0 con 0x7f51bc154330 2023-11-06T20:44:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 0x7f51bc153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 0x7f51bc153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37082/0 (socket says 172.21.15.33:37082) 2023-11-06T20:44:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c179b700 1 -- 172.21.15.33:0/3632200418 learned_addr learned my addr 172.21.15.33:0/3632200418 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c1f9c700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc154710 0x7f51bc155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c0f9a700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 0x7f51bc0ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c179b700 1 -- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 msgr2=0x7f51bc0ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c179b700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 0x7f51bc0ae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c179b700 1 -- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc154710 msgr2=0x7f51bc155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c179b700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc154710 0x7f51bc155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c179b700 1 -- 172.21.15.33:0/3632200418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4009ae0 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c1f9c700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc154710 0x7f51bc155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c0f9a700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 0x7f51bc0ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51c179b700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 0x7f51bc153ac0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f51b800c880 tx=0x7f51b8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.235+0000 7f51b17fa700 1 -- 172.21.15.33:0/3632200418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8010580 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.236+0000 7f51b17fa700 1 -- 172.21.15.33:0/3632200418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b80106e0 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.236+0000 7f51b17fa700 1 -- 172.21.15.33:0/3632200418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8010a60 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.236+0000 7f51c39ff700 1 -- 172.21.15.33:0/3632200418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc0af7d0 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.236+0000 7f51c39ff700 1 -- 172.21.15.33:0/3632200418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc158930 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.237+0000 7f51c39ff700 1 -- 172.21.15.33:0/3632200418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc0c48b0 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.238+0000 7f51b17fa700 1 -- 172.21.15.33:0/3632200418 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f51b801b460 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.239+0000 7f51b17fa700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51a005f150 0x7f51a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.239+0000 7f51b17fa700 1 -- 172.21.15.33:0/3632200418 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f51b807a320 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.239+0000 7f51c0f9a700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51a005f150 0x7f51a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.240+0000 7f51c0f9a700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51a005f150 0x7f51a0061610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f51ac006fd0 tx=0x7f51ac008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.242+0000 7f51b17fa700 1 -- 172.21.15.33:0/3632200418 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f51b804b4d0 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.421+0000 7f51c39ff700 1 -- 172.21.15.33:0/3632200418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f51bc0652d0 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.421+0000 7f51b17fa700 1 -- 172.21.15.33:0/3632200418 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7f51b804eaf0 con 0x7f51bc0c7ff0 2023-11-06T20:44:41.422 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:44:41.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.423+0000 7f51c39ff700 1 -- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51a005f150 msgr2=0x7f51a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.423+0000 7f51c39ff700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51a005f150 0x7f51a0061610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f51ac006fd0 tx=0x7f51ac008040).stop 2023-11-06T20:44:41.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.423+0000 7f51c39ff700 1 -- 172.21.15.33:0/3632200418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 msgr2=0x7f51bc153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.423+0000 7f51c39ff700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 0x7f51bc153ac0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f51b800c880 tx=0x7f51b8008040).stop 2023-11-06T20:44:41.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.423+0000 7f51c39ff700 1 -- 172.21.15.33:0/3632200418 shutdown_connections 2023-11-06T20:44:41.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.423+0000 7f51c39ff700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51a005f150 0x7f51a0061610 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.423+0000 7f51c39ff700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc154710 0x7f51bc155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.423+0000 7f51c39ff700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51bc0c7ff0 0x7f51bc153ac0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.423+0000 7f51c39ff700 1 --2- 172.21.15.33:0/3632200418 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51bc154330 0x7f51bc0ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.423+0000 7f51c39ff700 1 -- 172.21.15.33:0/3632200418 >> 172.21.15.33:0/3632200418 conn(0x7f51bc0cb910 msgr2=0x7f51bc0ac570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:41.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.424+0000 7f51c39ff700 1 -- 172.21.15.33:0/3632200418 shutdown_connections 2023-11-06T20:44:41.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.424+0000 7f51c39ff700 1 -- 172.21.15.33:0/3632200418 wait complete. 2023-11-06T20:44:41.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:44:41.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.639+0000 7f2ba47ff700 1 Processor -- start 2023-11-06T20:44:41.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.639+0000 7f2ba47ff700 1 -- start start 2023-11-06T20:44:41.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.640+0000 7f2ba47ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2b9c154710 0x7f2b9c0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.640+0000 7f2ba47ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c0c6b60 con 0x7f2b9c154330 2023-11-06T20:44:41.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.640+0000 7f2ba47ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c0c7700 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.640+0000 7f2ba47ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c0c5cd0 con 0x7f2b9c154710 2023-11-06T20:44:41.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.640+0000 7f2ba259b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2b9c154330 0x7f2b9c0c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43838/0 (socket says 172.21.15.33:43838) 2023-11-06T20:44:41.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.640+0000 7f2ba259b700 1 -- 172.21.15.33:0/949671372 learned_addr learned my addr 172.21.15.33:0/949671372 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:41.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.641+0000 7f2ba1d9a700 1 --2- 172.21.15.33:0/949671372 >> v2:172.21.15.69:3300/0 conn(0x7f2b9c154710 0x7f2b9c0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.641+0000 7f2ba1599700 1 -- 172.21.15.33:0/949671372 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289906201 0 0) 0x7f2b9c0c6b60 con 0x7f2b9c154330 2023-11-06T20:44:41.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.641+0000 7f2ba1599700 1 -- 172.21.15.33:0/949671372 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b880033f0 con 0x7f2b9c154330 2023-11-06T20:44:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.641+0000 7f2ba1d9a700 1 -- 172.21.15.33:0/949671372 >> v1:172.21.15.69:6789/0 conn(0x7f2b9c0c7ff0 legacy=0x7f2b9c0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.641+0000 7f2ba1599700 1 -- 172.21.15.33:0/949671372 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753052707 0 0) 0x7f2b9c0c7700 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.641+0000 7f2ba1d9a700 1 -- 172.21.15.33:0/949671372 >> v1:172.21.15.33:6789/0 conn(0x7f2b9c154330 legacy=0x7f2b9c0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.641+0000 7f2ba1d9a700 1 -- 172.21.15.33:0/949671372 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0c5f50 con 0x7f2b9c154710 2023-11-06T20:44:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.641+0000 7f2ba1d9a700 1 --2- 172.21.15.33:0/949671372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b9c154710 0x7f2b9c0c57e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f2b8c009760 tx=0x7f2b8c009060).ready entity=mon.1 client_cookie=d6f802828cfcee37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.642+0000 7f2ba1599700 1 -- 172.21.15.33:0/949671372 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c00f510 con 0x7f2b9c154710 2023-11-06T20:44:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.642+0000 7f2ba1599700 1 -- 172.21.15.33:0/949671372 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c00f670 con 0x7f2b9c154710 2023-11-06T20:44:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.642+0000 7f2ba1599700 1 -- 172.21.15.33:0/949671372 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c00f9f0 con 0x7f2b9c154710 2023-11-06T20:44:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.642+0000 7f2ba47ff700 1 -- 172.21.15.33:0/949671372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b9c154710 msgr2=0x7f2b9c0c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.642+0000 7f2ba47ff700 1 --2- 172.21.15.33:0/949671372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b9c154710 0x7f2b9c0c57e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f2b8c009760 tx=0x7f2b8c009060).stop 2023-11-06T20:44:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.642+0000 7f2ba47ff700 1 -- 172.21.15.33:0/949671372 shutdown_connections 2023-11-06T20:44:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.642+0000 7f2ba47ff700 1 --2- 172.21.15.33:0/949671372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b9c154710 0x7f2b9c0c57e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.642+0000 7f2ba47ff700 1 -- 172.21.15.33:0/949671372 >> 172.21.15.33:0/949671372 conn(0x7f2b9c0cb910 msgr2=0x7f2b9c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.642+0000 7f2ba47ff700 1 -- 172.21.15.33:0/949671372 shutdown_connections 2023-11-06T20:44:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.642+0000 7f2ba47ff700 1 -- 172.21.15.33:0/949671372 wait complete. 2023-11-06T20:44:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.644+0000 7f2ba47ff700 1 Processor -- start 2023-11-06T20:44:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.644+0000 7f2ba47ff700 1 -- start start 2023-11-06T20:44:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba47ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b9c0c7ff0 0x7f2b9c14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba47ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b9c154330 0x7f2b9c0ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2b9c154710 0x7f2b9c0c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba47ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c156c00 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba47ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c154fb0 con 0x7f2b9c154330 2023-11-06T20:44:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c156640 con 0x7f2b9c154710 2023-11-06T20:44:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b9c0c7ff0 0x7f2b9c14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b9c0c7ff0 0x7f2b9c14ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37086/0 (socket says 172.21.15.33:37086) 2023-11-06T20:44:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba259b700 1 -- 172.21.15.33:0/3094491189 learned_addr learned my addr 172.21.15.33:0/3094491189 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba1d9a700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b9c154330 0x7f2b9c0ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba2d9c700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2b9c154710 0x7f2b9c0c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba259b700 1 -- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2b9c154710 msgr2=0x7f2b9c0c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba259b700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2b9c154710 0x7f2b9c0c3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba259b700 1 -- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b9c154330 msgr2=0x7f2b9c0ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba259b700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b9c154330 0x7f2b9c0ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba259b700 1 -- 172.21.15.33:0/3094491189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c0092a0 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.645+0000 7f2ba1d9a700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b9c154330 0x7f2b9c0ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:41.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.646+0000 7f2ba259b700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b9c0c7ff0 0x7f2b9c14ca90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f2b98001ed0 tx=0x7f2b98007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:41.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.646+0000 7f2b927fc700 1 -- 172.21.15.33:0/3094491189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b980072e0 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.646+0000 7f2b927fc700 1 -- 172.21.15.33:0/3094491189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b98007440 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.646+0000 7f2ba47ff700 1 -- 172.21.15.33:0/3094491189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c156920 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.646+0000 7f2b927fc700 1 -- 172.21.15.33:0/3094491189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b980057a0 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.646+0000 7f2ba47ff700 1 -- 172.21.15.33:0/3094491189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c14cf80 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.647+0000 7f2ba47ff700 1 -- 172.21.15.33:0/3094491189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c0830b0 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.648+0000 7f2b927fc700 1 -- 172.21.15.33:0/3094491189 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2b98005b50 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.649+0000 7f2b927fc700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2b8005f150 0x7f2b80061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:41.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.649+0000 7f2b927fc700 1 -- 172.21.15.33:0/3094491189 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2b98079970 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.649+0000 7f2ba1d9a700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2b8005f150 0x7f2b80061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:41.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.650+0000 7f2ba1d9a700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2b8005f150 0x7f2b80061610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f2b9c154b70 tx=0x7f2b8c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:41.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.653+0000 7f2b927fc700 1 -- 172.21.15.33:0/3094491189 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2b9804ac40 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.833+0000 7f2ba47ff700 1 -- 172.21.15.33:0/3094491189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2b9c0652d0 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.833+0000 7f2b927fc700 1 -- 172.21.15.33:0/3094491189 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2b98018030 con 0x7f2b9c0c7ff0 2023-11-06T20:44:41.834 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:44:41.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 -- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2b8005f150 msgr2=0x7f2b80061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2b8005f150 0x7f2b80061610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f2b9c154b70 tx=0x7f2b8c017000).stop 2023-11-06T20:44:41.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 -- 172.21.15.33:0/3094491189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b9c0c7ff0 msgr2=0x7f2b9c14ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:41.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b9c0c7ff0 0x7f2b9c14ca90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f2b98001ed0 tx=0x7f2b98007c70).stop 2023-11-06T20:44:41.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 -- 172.21.15.33:0/3094491189 shutdown_connections 2023-11-06T20:44:41.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2b8005f150 0x7f2b80061610 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2b9c154710 0x7f2b9c0c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b9c0c7ff0 0x7f2b9c14ca90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 --2- 172.21.15.33:0/3094491189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b9c154330 0x7f2b9c0ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 -- 172.21.15.33:0/3094491189 >> 172.21.15.33:0/3094491189 conn(0x7f2b9c0cb910 msgr2=0x7f2b9c0c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 -- 172.21.15.33:0/3094491189 shutdown_connections 2023-11-06T20:44:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:41.836+0000 7f2ba47ff700 1 -- 172.21.15.33:0/3094491189 wait complete. 2023-11-06T20:44:41.853 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T20:44:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.051+0000 7f29507ff700 1 Processor -- start 2023-11-06T20:44:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.051+0000 7f29507ff700 1 -- start start 2023-11-06T20:44:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.052+0000 7f29507ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2948154330 0x7f29480c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.052+0000 7f29507ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2948154710 0x7f29480c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.052+0000 7f29507ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29480c6b60 con 0x7f29480c7ff0 2023-11-06T20:44:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.052+0000 7f29507ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29480c5cd0 con 0x7f2948154710 2023-11-06T20:44:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.052+0000 7f29507ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29480c5f50 con 0x7f2948154330 2023-11-06T20:44:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.052+0000 7f294dd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2948154710 0x7f29480c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.052+0000 7f294dd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2948154710 0x7f29480c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37094/0 (socket says 172.21.15.33:37094) 2023-11-06T20:44:42.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.052+0000 7f294dd9a700 1 -- 172.21.15.33:0/1587804746 learned_addr learned my addr 172.21.15.33:0/1587804746 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:42.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.053+0000 7f294e59b700 1 --2- 172.21.15.33:0/1587804746 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2948154330 0x7f29480c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.053+0000 7f294dd9a700 1 -- 172.21.15.33:0/1587804746 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2948154330 msgr2=0x7f29480c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.053+0000 7f294dd9a700 1 --2- 172.21.15.33:0/1587804746 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2948154330 0x7f29480c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.053+0000 7f294dd9a700 1 -- 172.21.15.33:0/1587804746 >> v1:172.21.15.69:6789/0 conn(0x7f29480c7ff0 legacy=0x7f29480646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.053+0000 7f294dd9a700 1 -- 172.21.15.33:0/1587804746 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29480c6230 con 0x7f2948154710 2023-11-06T20:44:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.053+0000 7f294dd9a700 1 --2- 172.21.15.33:0/1587804746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 0x7f29480c6ff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f2938006fd0 tx=0x7f29380090f0).ready entity=mon.0 client_cookie=b303eb274079f4bc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.053+0000 7f294d599700 1 -- 172.21.15.33:0/1587804746 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293800b600 con 0x7f2948154710 2023-11-06T20:44:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.053+0000 7f294d599700 1 -- 172.21.15.33:0/1587804746 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293800b760 con 0x7f2948154710 2023-11-06T20:44:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.053+0000 7f294d599700 1 -- 172.21.15.33:0/1587804746 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293800bae0 con 0x7f2948154710 2023-11-06T20:44:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.054+0000 7f29507ff700 1 -- 172.21.15.33:0/1587804746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 msgr2=0x7f29480c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.054+0000 7f29507ff700 1 --2- 172.21.15.33:0/1587804746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 0x7f29480c6ff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f2938006fd0 tx=0x7f29380090f0).stop 2023-11-06T20:44:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.054+0000 7f29507ff700 1 -- 172.21.15.33:0/1587804746 shutdown_connections 2023-11-06T20:44:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.054+0000 7f29507ff700 1 --2- 172.21.15.33:0/1587804746 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2948154330 0x7f29480c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.054+0000 7f29507ff700 1 --2- 172.21.15.33:0/1587804746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 0x7f29480c6ff0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.054+0000 7f29507ff700 1 -- 172.21.15.33:0/1587804746 >> 172.21.15.33:0/1587804746 conn(0x7f29480cb910 msgr2=0x7f29480c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:42.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.054+0000 7f29507ff700 1 -- 172.21.15.33:0/1587804746 shutdown_connections 2023-11-06T20:44:42.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.054+0000 7f29507ff700 1 -- 172.21.15.33:0/1587804746 wait complete. 2023-11-06T20:44:42.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.055+0000 7f29507ff700 1 Processor -- start 2023-11-06T20:44:42.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.056+0000 7f29507ff700 1 -- start start 2023-11-06T20:44:42.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.056+0000 7f29507ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29480c7ff0 0x7f2948150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.056+0000 7f29507ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2948154330 0x7f29480b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f29507ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 0x7f29480b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f29507ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29480b0af0 con 0x7f2948154710 2023-11-06T20:44:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f29507ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294814d9b0 con 0x7f2948154330 2023-11-06T20:44:42.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f29507ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29480b0cd0 con 0x7f29480c7ff0 2023-11-06T20:44:42.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 0x7f29480b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 0x7f29480b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37102/0 (socket says 172.21.15.33:37102) 2023-11-06T20:44:42.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294ed9c700 1 -- 172.21.15.33:0/1993997790 learned_addr learned my addr 172.21.15.33:0/1993997790 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294dd9a700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2948154330 0x7f29480b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294e59b700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29480c7ff0 0x7f2948150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294ed9c700 1 -- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29480c7ff0 msgr2=0x7f2948150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294ed9c700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29480c7ff0 0x7f2948150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294ed9c700 1 -- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2948154330 msgr2=0x7f29480b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294ed9c700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2948154330 0x7f29480b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294ed9c700 1 -- 172.21.15.33:0/1993997790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938009370 con 0x7f2948154710 2023-11-06T20:44:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.057+0000 7f294ed9c700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 0x7f29480b0600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f294000c7c0 tx=0x7f29400014e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.058+0000 7f293e7fc700 1 -- 172.21.15.33:0/1993997790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940010510 con 0x7f2948154710 2023-11-06T20:44:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.058+0000 7f293e7fc700 1 -- 172.21.15.33:0/1993997790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940010670 con 0x7f2948154710 2023-11-06T20:44:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.058+0000 7f293e7fc700 1 -- 172.21.15.33:0/1993997790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940010a10 con 0x7f2948154710 2023-11-06T20:44:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.058+0000 7f29507ff700 1 -- 172.21.15.33:0/1993997790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294814dc50 con 0x7f2948154710 2023-11-06T20:44:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.058+0000 7f29507ff700 1 -- 172.21.15.33:0/1993997790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294814e210 con 0x7f2948154710 2023-11-06T20:44:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.059+0000 7f29507ff700 1 -- 172.21.15.33:0/1993997790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29480830b0 con 0x7f2948154710 2023-11-06T20:44:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.060+0000 7f293e7fc700 1 -- 172.21.15.33:0/1993997790 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f294001b3b0 con 0x7f2948154710 2023-11-06T20:44:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.061+0000 7f293e7fc700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f292c05f150 0x7f292c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.061+0000 7f293e7fc700 1 -- 172.21.15.33:0/1993997790 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2940079c20 con 0x7f2948154710 2023-11-06T20:44:42.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.061+0000 7f294e59b700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f292c05f150 0x7f292c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.062+0000 7f294e59b700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f292c05f150 0x7f292c061610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f294400bb20 tx=0x7f2944008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:42.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.065+0000 7f293e7fc700 1 -- 172.21.15.33:0/1993997790 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f294004add0 con 0x7f2948154710 2023-11-06T20:44:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.241+0000 7f29507ff700 1 -- 172.21.15.33:0/1993997790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29480652d0 con 0x7f2948154710 2023-11-06T20:44:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.242+0000 7f293e7fc700 1 -- 172.21.15.33:0/1993997790 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f294004e3f0 con 0x7f2948154710 2023-11-06T20:44:42.243 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 -- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f292c05f150 msgr2=0x7f292c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f292c05f150 0x7f292c061610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f294400bb20 tx=0x7f2944008040).stop 2023-11-06T20:44:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 -- 172.21.15.33:0/1993997790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 msgr2=0x7f29480b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 0x7f29480b0600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f294000c7c0 tx=0x7f29400014e0).stop 2023-11-06T20:44:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 -- 172.21.15.33:0/1993997790 shutdown_connections 2023-11-06T20:44:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f292c05f150 0x7f292c061610 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29480c7ff0 0x7f2948150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2948154710 0x7f29480b0600 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 --2- 172.21.15.33:0/1993997790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2948154330 0x7f29480b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 -- 172.21.15.33:0/1993997790 >> 172.21.15.33:0/1993997790 conn(0x7f29480cb910 msgr2=0x7f29480c8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 -- 172.21.15.33:0/1993997790 shutdown_connections 2023-11-06T20:44:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.244+0000 7f29507ff700 1 -- 172.21.15.33:0/1993997790 wait complete. 2023-11-06T20:44:42.261 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T20:44:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.459+0000 7f462fbff700 1 Processor -- start 2023-11-06T20:44:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.459+0000 7f462fbff700 1 -- start start 2023-11-06T20:44:42.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462fbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4628064dd0 0x7f46280c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462fbff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46280c6b60 con 0x7f4628154330 2023-11-06T20:44:42.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462fbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46280c7700 con 0x7f4628154710 2023-11-06T20:44:42.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462fbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46280c5cd0 con 0x7f4628064dd0 2023-11-06T20:44:42.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462d99b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4628154330 0x7f46280c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43854/0 (socket says 172.21.15.33:43854) 2023-11-06T20:44:42.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462d99b700 1 -- 172.21.15.33:0/108853528 learned_addr learned my addr 172.21.15.33:0/108853528 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:42.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462e19c700 1 --2- 172.21.15.33:0/108853528 >> v2:172.21.15.33:3300/0 conn(0x7f4628064dd0 0x7f46280c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462e19c700 1 -- 172.21.15.33:0/108853528 >> v1:172.21.15.69:6789/0 conn(0x7f4628154710 legacy=0x7f46280646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462e19c700 1 -- 172.21.15.33:0/108853528 >> v1:172.21.15.33:6789/0 conn(0x7f4628154330 legacy=0x7f46280c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462e19c700 1 -- 172.21.15.33:0/108853528 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46280c5f50 con 0x7f4628064dd0 2023-11-06T20:44:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.461+0000 7f462c999700 1 -- 172.21.15.33:0/108853528 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925398862 0 0) 0x7f46280c6b60 con 0x7f4628154330 2023-11-06T20:44:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.462+0000 7f462e19c700 1 --2- 172.21.15.33:0/108853528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628064dd0 0x7f46280c57e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f4620009e60 tx=0x7f4620009780).ready entity=mon.0 client_cookie=7d4f7dc0e655e592 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.462+0000 7f462c999700 1 -- 172.21.15.33:0/108853528 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000e070 con 0x7f4628064dd0 2023-11-06T20:44:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.462+0000 7f462c999700 1 -- 172.21.15.33:0/108853528 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462000bc30 con 0x7f4628064dd0 2023-11-06T20:44:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.462+0000 7f462c999700 1 -- 172.21.15.33:0/108853528 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620012410 con 0x7f4628064dd0 2023-11-06T20:44:42.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.462+0000 7f462fbff700 1 -- 172.21.15.33:0/108853528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628064dd0 msgr2=0x7f46280c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.462+0000 7f462fbff700 1 --2- 172.21.15.33:0/108853528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628064dd0 0x7f46280c57e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f4620009e60 tx=0x7f4620009780).stop 2023-11-06T20:44:42.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.462+0000 7f462fbff700 1 -- 172.21.15.33:0/108853528 shutdown_connections 2023-11-06T20:44:42.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.462+0000 7f462fbff700 1 --2- 172.21.15.33:0/108853528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628064dd0 0x7f46280c57e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.462+0000 7f462fbff700 1 -- 172.21.15.33:0/108853528 >> 172.21.15.33:0/108853528 conn(0x7f46280cb910 msgr2=0x7f46280c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:42.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.463+0000 7f462fbff700 1 -- 172.21.15.33:0/108853528 shutdown_connections 2023-11-06T20:44:42.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.463+0000 7f462fbff700 1 -- 172.21.15.33:0/108853528 wait complete. 2023-11-06T20:44:42.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.464+0000 7f462fbff700 1 Processor -- start 2023-11-06T20:44:42.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.464+0000 7f462fbff700 1 -- start start 2023-11-06T20:44:42.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.465+0000 7f462fbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4628064dd0 0x7f4628146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.465+0000 7f462fbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4628154330 0x7f46280aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.465+0000 7f462fbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628154710 0x7f4628155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.465+0000 7f462fbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628157af0 con 0x7f4628154710 2023-11-06T20:44:42.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.465+0000 7f462fbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628157290 con 0x7f4628154330 2023-11-06T20:44:42.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.465+0000 7f462fbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4628147100 con 0x7f4628064dd0 2023-11-06T20:44:42.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.465+0000 7f462e19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628154710 0x7f4628155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.465+0000 7f462e19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628154710 0x7f4628155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37128/0 (socket says 172.21.15.33:37128) 2023-11-06T20:44:42.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.465+0000 7f462e19c700 1 -- 172.21.15.33:0/2735114383 learned_addr learned my addr 172.21.15.33:0/2735114383 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:42.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.466+0000 7f462d99b700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4628064dd0 0x7f4628146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.466+0000 7f462d19a700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4628154330 0x7f46280aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.466+0000 7f462e19c700 1 -- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4628064dd0 msgr2=0x7f4628146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.466+0000 7f462e19c700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4628064dd0 0x7f4628146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.466+0000 7f462e19c700 1 -- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4628154330 msgr2=0x7f46280aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.466+0000 7f462e19c700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4628154330 0x7f46280aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.466+0000 7f462e19c700 1 -- 172.21.15.33:0/2735114383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620009a20 con 0x7f4628154710 2023-11-06T20:44:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.466+0000 7f462e19c700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628154710 0x7f4628155430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f462806c170 tx=0x7f4620009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.466+0000 7f461dffb700 1 -- 172.21.15.33:0/2735114383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620006320 con 0x7f4628154710 2023-11-06T20:44:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.467+0000 7f461dffb700 1 -- 172.21.15.33:0/2735114383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4620006480 con 0x7f4628154710 2023-11-06T20:44:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.467+0000 7f461dffb700 1 -- 172.21.15.33:0/2735114383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620006820 con 0x7f4628154710 2023-11-06T20:44:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.467+0000 7f462fbff700 1 -- 172.21.15.33:0/2735114383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4628165450 con 0x7f4628154710 2023-11-06T20:44:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.467+0000 7f462fbff700 1 -- 172.21.15.33:0/2735114383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4628165980 con 0x7f4628154710 2023-11-06T20:44:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.468+0000 7f462fbff700 1 -- 172.21.15.33:0/2735114383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46280af5c0 con 0x7f4628154710 2023-11-06T20:44:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.468+0000 7f461dffb700 1 -- 172.21.15.33:0/2735114383 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4620027460 con 0x7f4628154710 2023-11-06T20:44:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.469+0000 7f461dffb700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f460c05f150 0x7f460c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.469+0000 7f461dffb700 1 -- 172.21.15.33:0/2735114383 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f462007c660 con 0x7f4628154710 2023-11-06T20:44:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.470+0000 7f462d99b700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f460c05f150 0x7f460c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.470+0000 7f462d99b700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f460c05f150 0x7f460c061610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f46240011e0 tx=0x7f4624007b90).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.473+0000 7f461dffb700 1 -- 172.21.15.33:0/2735114383 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f46280af5c0 con 0x7f4628154710 2023-11-06T20:44:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.653+0000 7f462fbff700 1 -- 172.21.15.33:0/2735114383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46280af5c0 con 0x7f4628154710 2023-11-06T20:44:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.654+0000 7f461dffb700 1 -- 172.21.15.33:0/2735114383 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46280af5c0 con 0x7f4628154710 2023-11-06T20:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 -- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f460c05f150 msgr2=0x7f460c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f460c05f150 0x7f460c061610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f46240011e0 tx=0x7f4624007b90).stop 2023-11-06T20:44:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 -- 172.21.15.33:0/2735114383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628154710 msgr2=0x7f4628155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628154710 0x7f4628155430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f462806c170 tx=0x7f4620009780).stop 2023-11-06T20:44:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 -- 172.21.15.33:0/2735114383 shutdown_connections 2023-11-06T20:44:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f460c05f150 0x7f460c061610 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4628064dd0 0x7f4628146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4628154710 0x7f4628155430 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 --2- 172.21.15.33:0/2735114383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4628154330 0x7f46280aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 -- 172.21.15.33:0/2735114383 >> 172.21.15.33:0/2735114383 conn(0x7f46280cb910 msgr2=0x7f46280a9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 -- 172.21.15.33:0/2735114383 shutdown_connections 2023-11-06T20:44:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.656+0000 7f462fbff700 1 -- 172.21.15.33:0/2735114383 wait complete. 2023-11-06T20:44:42.673 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.869+0000 7ff595bff700 1 Processor -- start 2023-11-06T20:44:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.869+0000 7ff595bff700 1 -- start start 2023-11-06T20:44:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.871+0000 7ff595bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59005e1b0 0x7ff590063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.871+0000 7ff595bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5900c3680 0x7ff590154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.871+0000 7ff595bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5900c31f0 con 0x7ff590063140 2023-11-06T20:44:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.871+0000 7ff595bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59006a0f0 con 0x7ff5900c3680 2023-11-06T20:44:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.871+0000 7ff595bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900c2920 con 0x7ff59005e1b0 2023-11-06T20:44:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.871+0000 7ff58ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5900c3680 0x7ff590154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.871+0000 7ff58ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5900c3680 0x7ff590154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37140/0 (socket says 172.21.15.33:37140) 2023-11-06T20:44:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.871+0000 7ff58ffff700 1 -- 172.21.15.33:0/2849242819 learned_addr learned my addr 172.21.15.33:0/2849242819 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.871+0000 7ff58f7fe700 1 --1- 172.21.15.33:0/2849242819 >> v1:172.21.15.33:6789/0 conn(0x7ff590063140 0x7ff5900c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43856/0 (socket says 172.21.15.33:43856) 2023-11-06T20:44:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.871+0000 7ff58effd700 1 --2- 172.21.15.33:0/2849242819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59005e1b0 0x7ff590063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.872+0000 7ff58ffff700 1 -- 172.21.15.33:0/2849242819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59005e1b0 msgr2=0x7ff590063520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.872+0000 7ff58ffff700 1 --2- 172.21.15.33:0/2849242819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59005e1b0 0x7ff590063520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.872+0000 7ff58ffff700 1 -- 172.21.15.33:0/2849242819 >> v1:172.21.15.33:6789/0 conn(0x7ff590063140 legacy=0x7ff5900c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.872+0000 7ff58e7fc700 1 -- 172.21.15.33:0/2849242819 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601631162 0 0) 0x7ff5900c31f0 con 0x7ff590063140 2023-11-06T20:44:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.872+0000 7ff58ffff700 1 -- 172.21.15.33:0/2849242819 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59005e640 con 0x7ff5900c3680 2023-11-06T20:44:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.872+0000 7ff58ffff700 1 --2- 172.21.15.33:0/2849242819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5900c3680 0x7ff590154050 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7ff588009f20 tx=0x7ff588009840).ready entity=mon.0 client_cookie=7cf2f354f0c09dfb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.872+0000 7ff58e7fc700 1 -- 172.21.15.33:0/2849242819 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58800e070 con 0x7ff5900c3680 2023-11-06T20:44:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.872+0000 7ff58e7fc700 1 -- 172.21.15.33:0/2849242819 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58800b9d0 con 0x7ff5900c3680 2023-11-06T20:44:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.872+0000 7ff58e7fc700 1 -- 172.21.15.33:0/2849242819 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58800bd50 con 0x7ff5900c3680 2023-11-06T20:44:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.873+0000 7ff595bff700 1 -- 172.21.15.33:0/2849242819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5900c3680 msgr2=0x7ff590154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.873+0000 7ff595bff700 1 --2- 172.21.15.33:0/2849242819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5900c3680 0x7ff590154050 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7ff588009f20 tx=0x7ff588009840).stop 2023-11-06T20:44:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.873+0000 7ff595bff700 1 -- 172.21.15.33:0/2849242819 shutdown_connections 2023-11-06T20:44:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.873+0000 7ff595bff700 1 --2- 172.21.15.33:0/2849242819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5900c3680 0x7ff590154050 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.873+0000 7ff595bff700 1 --2- 172.21.15.33:0/2849242819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59005e1b0 0x7ff590063520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.873+0000 7ff595bff700 1 -- 172.21.15.33:0/2849242819 >> 172.21.15.33:0/2849242819 conn(0x7ff5900c7d60 msgr2=0x7ff5900c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.873+0000 7ff595bff700 1 -- 172.21.15.33:0/2849242819 shutdown_connections 2023-11-06T20:44:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.873+0000 7ff595bff700 1 -- 172.21.15.33:0/2849242819 wait complete. 2023-11-06T20:44:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.874+0000 7ff595bff700 1 Processor -- start 2023-11-06T20:44:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.875+0000 7ff595bff700 1 -- start start 2023-11-06T20:44:42.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.875+0000 7ff595bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff590063140 0x7ff59014b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.875+0000 7ff595bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5900c3680 0x7ff590168120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.875+0000 7ff595bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59014b870 0x7ff590168610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.875+0000 7ff595bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff590149f70 con 0x7ff590063140 2023-11-06T20:44:42.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.875+0000 7ff595bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff590147030 con 0x7ff5900c3680 2023-11-06T20:44:42.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.875+0000 7ff595bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff590147890 con 0x7ff59014b870 2023-11-06T20:44:42.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff590063140 0x7ff59014b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff590063140 0x7ff59014b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37144/0 (socket says 172.21.15.33:37144) 2023-11-06T20:44:42.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58f7fe700 1 -- 172.21.15.33:0/1613123363 learned_addr learned my addr 172.21.15.33:0/1613123363 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:42.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58ffff700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59014b870 0x7ff590168610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58effd700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5900c3680 0x7ff590168120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58f7fe700 1 -- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59014b870 msgr2=0x7ff590168610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58f7fe700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59014b870 0x7ff590168610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58f7fe700 1 -- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5900c3680 msgr2=0x7ff590168120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58f7fe700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5900c3680 0x7ff590168120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58f7fe700 1 -- 172.21.15.33:0/1613123363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff588009ae0 con 0x7ff590063140 2023-11-06T20:44:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58ffff700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59014b870 0x7ff590168610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58effd700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5900c3680 0x7ff590168120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff58f7fe700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff590063140 0x7ff59014b380 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7ff58000c560 tx=0x7ff580007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.876+0000 7ff5777fe700 1 -- 172.21.15.33:0/1613123363 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5800072e0 con 0x7ff590063140 2023-11-06T20:44:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.877+0000 7ff5777fe700 1 -- 172.21.15.33:0/1613123363 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580007440 con 0x7ff590063140 2023-11-06T20:44:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.877+0000 7ff5777fe700 1 -- 172.21.15.33:0/1613123363 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5800058f0 con 0x7ff590063140 2023-11-06T20:44:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.877+0000 7ff595bff700 1 -- 172.21.15.33:0/1613123363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff590149c90 con 0x7ff590063140 2023-11-06T20:44:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.877+0000 7ff595bff700 1 -- 172.21.15.33:0/1613123363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff590147520 con 0x7ff590063140 2023-11-06T20:44:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.878+0000 7ff595bff700 1 -- 172.21.15.33:0/1613123363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff590064420 con 0x7ff590063140 2023-11-06T20:44:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.878+0000 7ff5777fe700 1 -- 172.21.15.33:0/1613123363 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff580005ce0 con 0x7ff590063140 2023-11-06T20:44:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.879+0000 7ff5777fe700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff57c05f150 0x7ff57c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.879+0000 7ff5777fe700 1 -- 172.21.15.33:0/1613123363 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff580079a30 con 0x7ff590063140 2023-11-06T20:44:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.880+0000 7ff58effd700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff57c05f150 0x7ff57c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.880+0000 7ff58effd700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff57c05f150 0x7ff57c061610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7ff5900c6c20 tx=0x7ff584008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:42.883+0000 7ff5777fe700 1 -- 172.21.15.33:0/1613123363 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff58004ad00 con 0x7ff590063140 2023-11-06T20:44:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.070+0000 7ff595bff700 1 -- 172.21.15.33:0/1613123363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff59006bfd0 con 0x7ff590063140 2023-11-06T20:44:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.071+0000 7ff5777fe700 1 -- 172.21.15.33:0/1613123363 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7ff58004e320 con 0x7ff590063140 2023-11-06T20:44:43.071 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:43.071 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:43.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.072+0000 7ff595bff700 1 -- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff57c05f150 msgr2=0x7ff57c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.072+0000 7ff595bff700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff57c05f150 0x7ff57c061610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7ff5900c6c20 tx=0x7ff584008040).stop 2023-11-06T20:44:43.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.072+0000 7ff595bff700 1 -- 172.21.15.33:0/1613123363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff590063140 msgr2=0x7ff59014b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.072+0000 7ff595bff700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff590063140 0x7ff59014b380 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7ff58000c560 tx=0x7ff580007c90).stop 2023-11-06T20:44:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.073+0000 7ff595bff700 1 -- 172.21.15.33:0/1613123363 shutdown_connections 2023-11-06T20:44:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.073+0000 7ff595bff700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff57c05f150 0x7ff57c061610 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.073+0000 7ff595bff700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff59014b870 0x7ff590168610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.073+0000 7ff595bff700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff590063140 0x7ff59014b380 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.073+0000 7ff595bff700 1 --2- 172.21.15.33:0/1613123363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5900c3680 0x7ff590168120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.073+0000 7ff595bff700 1 -- 172.21.15.33:0/1613123363 >> 172.21.15.33:0/1613123363 conn(0x7ff5900c7d60 msgr2=0x7ff590151700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:43.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.073+0000 7ff595bff700 1 -- 172.21.15.33:0/1613123363 shutdown_connections 2023-11-06T20:44:43.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.073+0000 7ff595bff700 1 -- 172.21.15.33:0/1613123363 wait complete. 2023-11-06T20:44:43.090 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:44:04.716205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:44:05.351480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T20:44:43.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:44:43.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.289+0000 7fcf329ff700 1 Processor -- start 2023-11-06T20:44:43.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.290+0000 7fcf329ff700 1 -- start start 2023-11-06T20:44:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.291+0000 7fcf329ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcf2c1546a0 0x7fcf2c064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.291+0000 7fcf329ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 0x7fcf2c066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.291+0000 7fcf329ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2c0641f0 con 0x7fcf2c066560 2023-11-06T20:44:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.291+0000 7fcf329ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2c154a80 con 0x7fcf2c1546a0 2023-11-06T20:44:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.291+0000 7fcf329ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c0638c0 con 0x7fcf2c0be2c0 2023-11-06T20:44:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.291+0000 7fcf2bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcf2c1546a0 0x7fcf2c064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcf2c1546a0 0x7fcf2c064680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37146/0 (socket says 172.21.15.33:37146) 2023-11-06T20:44:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2bfff700 1 -- 172.21.15.33:0/1583245228 learned_addr learned my addr 172.21.15.33:0/1583245228 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2b7fe700 1 --1- 172.21.15.33:0/1583245228 >> v1:172.21.15.33:6789/0 conn(0x7fcf2c066560 0x7fcf2c0bdbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43860/0 (socket says 172.21.15.33:43860) 2023-11-06T20:44:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf30f9c700 1 --2- 172.21.15.33:0/1583245228 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 0x7fcf2c066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2bfff700 1 -- 172.21.15.33:0/1583245228 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 msgr2=0x7fcf2c066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2bfff700 1 --2- 172.21.15.33:0/1583245228 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 0x7fcf2c066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2bfff700 1 -- 172.21.15.33:0/1583245228 >> v1:172.21.15.33:6789/0 conn(0x7fcf2c066560 legacy=0x7fcf2c0bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2bfff700 1 -- 172.21.15.33:0/1583245228 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c063ba0 con 0x7fcf2c1546a0 2023-11-06T20:44:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf30f9c700 1 --2- 172.21.15.33:0/1583245228 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 0x7fcf2c066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2bfff700 1 --2- 172.21.15.33:0/1583245228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf2c1546a0 0x7fcf2c064680 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fcf1c006fd0 tx=0x7fcf1c0090f0).ready entity=mon.0 client_cookie=47f4e0f09be5676d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2affd700 1 -- 172.21.15.33:0/1583245228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00b600 con 0x7fcf2c1546a0 2023-11-06T20:44:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2affd700 1 -- 172.21.15.33:0/1583245228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c00b760 con 0x7fcf2c1546a0 2023-11-06T20:44:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.292+0000 7fcf2affd700 1 -- 172.21.15.33:0/1583245228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00bae0 con 0x7fcf2c1546a0 2023-11-06T20:44:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.293+0000 7fcf329ff700 1 -- 172.21.15.33:0/1583245228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf2c1546a0 msgr2=0x7fcf2c064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.293+0000 7fcf329ff700 1 --2- 172.21.15.33:0/1583245228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf2c1546a0 0x7fcf2c064680 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fcf1c006fd0 tx=0x7fcf1c0090f0).stop 2023-11-06T20:44:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.293+0000 7fcf329ff700 1 -- 172.21.15.33:0/1583245228 shutdown_connections 2023-11-06T20:44:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.293+0000 7fcf329ff700 1 --2- 172.21.15.33:0/1583245228 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 0x7fcf2c066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.293+0000 7fcf329ff700 1 --2- 172.21.15.33:0/1583245228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf2c1546a0 0x7fcf2c064680 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.293+0000 7fcf329ff700 1 -- 172.21.15.33:0/1583245228 >> 172.21.15.33:0/1583245228 conn(0x7fcf2c0c8c50 msgr2=0x7fcf2c0c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.293+0000 7fcf329ff700 1 -- 172.21.15.33:0/1583245228 shutdown_connections 2023-11-06T20:44:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.293+0000 7fcf329ff700 1 -- 172.21.15.33:0/1583245228 wait complete. 2023-11-06T20:44:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.295+0000 7fcf329ff700 1 Processor -- start 2023-11-06T20:44:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.295+0000 7fcf329ff700 1 -- start start 2023-11-06T20:44:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.296+0000 7fcf329ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf2c066560 0x7fcf2c0c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.296+0000 7fcf329ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 0x7fcf2c151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.296+0000 7fcf329ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf2c1546a0 0x7fcf2c167d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.296+0000 7fcf329ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c0a8750 con 0x7fcf2c1546a0 2023-11-06T20:44:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.296+0000 7fcf329ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c0a8d10 con 0x7fcf2c066560 2023-11-06T20:44:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.296+0000 7fcf329ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c168810 con 0x7fcf2c0be2c0 2023-11-06T20:44:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf2c066560 0x7fcf2c0c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf30f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf2c1546a0 0x7fcf2c167d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf2c066560 0x7fcf2c0c12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:45748/0 (socket says 172.21.15.33:45748) 2023-11-06T20:44:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf2bfff700 1 -- 172.21.15.33:0/4165876942 learned_addr learned my addr 172.21.15.33:0/4165876942 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf2b7fe700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 0x7fcf2c151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf30f9c700 1 -- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 msgr2=0x7fcf2c151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf30f9c700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 0x7fcf2c151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf30f9c700 1 -- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf2c066560 msgr2=0x7fcf2c0c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf30f9c700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf2c066560 0x7fcf2c0c12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf30f9c700 1 -- 172.21.15.33:0/4165876942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c009370 con 0x7fcf2c1546a0 2023-11-06T20:44:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf30f9c700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf2c1546a0 0x7fcf2c167d50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fcf24009f70 tx=0x7fcf240099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.297+0000 7fcf13fff700 1 -- 172.21.15.33:0/4165876942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2400db80 con 0x7fcf2c1546a0 2023-11-06T20:44:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.298+0000 7fcf13fff700 1 -- 172.21.15.33:0/4165876942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2400f040 con 0x7fcf2c1546a0 2023-11-06T20:44:43.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.298+0000 7fcf329ff700 1 -- 172.21.15.33:0/4165876942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c168af0 con 0x7fcf2c1546a0 2023-11-06T20:44:43.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.298+0000 7fcf13fff700 1 -- 172.21.15.33:0/4165876942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf240133f0 con 0x7fcf2c1546a0 2023-11-06T20:44:43.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.298+0000 7fcf329ff700 1 -- 172.21.15.33:0/4165876942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c0aac50 con 0x7fcf2c1546a0 2023-11-06T20:44:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.299+0000 7fcf329ff700 1 -- 172.21.15.33:0/4165876942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2c066000 con 0x7fcf2c1546a0 2023-11-06T20:44:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.299+0000 7fcf13fff700 1 -- 172.21.15.33:0/4165876942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcf240137e0 con 0x7fcf2c1546a0 2023-11-06T20:44:43.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.300+0000 7fcf13fff700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcf1805f150 0x7fcf18061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.300+0000 7fcf13fff700 1 -- 172.21.15.33:0/4165876942 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcf2407a7a0 con 0x7fcf2c1546a0 2023-11-06T20:44:43.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.301+0000 7fcf2bfff700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcf1805f150 0x7fcf18061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.301+0000 7fcf2bfff700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcf1805f150 0x7fcf18061610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fcf1c009130 tx=0x7fcf1c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:43.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.304+0000 7fcf13fff700 1 -- 172.21.15.33:0/4165876942 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcf2404b9b0 con 0x7fcf2c1546a0 2023-11-06T20:44:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.486+0000 7fcf329ff700 1 -- 172.21.15.33:0/4165876942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcf2c0a8ea0 con 0x7fcf2c1546a0 2023-11-06T20:44:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.487+0000 7fcf13fff700 1 -- 172.21.15.33:0/4165876942 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcf2404efd0 con 0x7fcf2c1546a0 2023-11-06T20:44:43.487 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:44:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 -- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcf1805f150 msgr2=0x7fcf18061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcf1805f150 0x7fcf18061610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fcf1c009130 tx=0x7fcf1c017000).stop 2023-11-06T20:44:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 -- 172.21.15.33:0/4165876942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf2c1546a0 msgr2=0x7fcf2c167d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf2c1546a0 0x7fcf2c167d50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fcf24009f70 tx=0x7fcf240099b0).stop 2023-11-06T20:44:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 -- 172.21.15.33:0/4165876942 shutdown_connections 2023-11-06T20:44:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcf1805f150 0x7fcf18061610 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf2c1546a0 0x7fcf2c167d50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf2c066560 0x7fcf2c0c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 --2- 172.21.15.33:0/4165876942 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf2c0be2c0 0x7fcf2c151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 -- 172.21.15.33:0/4165876942 >> 172.21.15.33:0/4165876942 conn(0x7fcf2c0c8c50 msgr2=0x7fcf2c152b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 -- 172.21.15.33:0/4165876942 shutdown_connections 2023-11-06T20:44:43.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.489+0000 7fcf329ff700 1 -- 172.21.15.33:0/4165876942 wait complete. 2023-11-06T20:44:43.506 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:44:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.704+0000 7fad117ff700 1 Processor -- start 2023-11-06T20:44:43.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.704+0000 7fad117ff700 1 -- start start 2023-11-06T20:44:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad117ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fad0c064dd0 0x7fad0c0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad117ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0c6b60 con 0x7fad0c154330 2023-11-06T20:44:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad117ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0c7700 con 0x7fad0c154710 2023-11-06T20:44:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad117ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c0c5cd0 con 0x7fad0c064dd0 2023-11-06T20:44:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad0affd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fad0c154330 0x7fad0c0c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43870/0 (socket says 172.21.15.33:43870) 2023-11-06T20:44:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad0affd700 1 -- 172.21.15.33:0/3668534690 learned_addr learned my addr 172.21.15.33:0/3668534690 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad0b7fe700 1 --2- 172.21.15.33:0/3668534690 >> v2:172.21.15.69:3300/0 conn(0x7fad0c064dd0 0x7fad0c0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad09ffb700 1 -- 172.21.15.33:0/3668534690 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267003051 0 0) 0x7fad0c0c6b60 con 0x7fad0c154330 2023-11-06T20:44:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad09ffb700 1 -- 172.21.15.33:0/3668534690 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf40033f0 con 0x7fad0c154330 2023-11-06T20:44:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad0b7fe700 1 -- 172.21.15.33:0/3668534690 >> v1:172.21.15.69:6789/0 conn(0x7fad0c154710 legacy=0x7fad0c0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad0b7fe700 1 -- 172.21.15.33:0/3668534690 >> v1:172.21.15.33:6789/0 conn(0x7fad0c154330 legacy=0x7fad0c0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.706+0000 7fad0b7fe700 1 -- 172.21.15.33:0/3668534690 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0c5f50 con 0x7fad0c064dd0 2023-11-06T20:44:43.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.707+0000 7fad0b7fe700 1 --2- 172.21.15.33:0/3668534690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad0c064dd0 0x7fad0c0c57e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fad04009e60 tx=0x7fad04009780).ready entity=mon.1 client_cookie=2cfb49a997588b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:43.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.707+0000 7fad09ffb700 1 -- 172.21.15.33:0/3668534690 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0400e070 con 0x7fad0c064dd0 2023-11-06T20:44:43.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.707+0000 7fad09ffb700 1 -- 172.21.15.33:0/3668534690 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0400bb60 con 0x7fad0c064dd0 2023-11-06T20:44:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.707+0000 7fad09ffb700 1 -- 172.21.15.33:0/3668534690 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04012410 con 0x7fad0c064dd0 2023-11-06T20:44:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.707+0000 7fad117ff700 1 -- 172.21.15.33:0/3668534690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad0c064dd0 msgr2=0x7fad0c0c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.707+0000 7fad117ff700 1 --2- 172.21.15.33:0/3668534690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad0c064dd0 0x7fad0c0c57e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fad04009e60 tx=0x7fad04009780).stop 2023-11-06T20:44:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.708+0000 7fad117ff700 1 -- 172.21.15.33:0/3668534690 shutdown_connections 2023-11-06T20:44:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.708+0000 7fad117ff700 1 --2- 172.21.15.33:0/3668534690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad0c064dd0 0x7fad0c0c57e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.708+0000 7fad117ff700 1 -- 172.21.15.33:0/3668534690 >> 172.21.15.33:0/3668534690 conn(0x7fad0c0cb910 msgr2=0x7fad0c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.708+0000 7fad117ff700 1 -- 172.21.15.33:0/3668534690 shutdown_connections 2023-11-06T20:44:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.708+0000 7fad117ff700 1 -- 172.21.15.33:0/3668534690 wait complete. 2023-11-06T20:44:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.709+0000 7fad117ff700 1 Processor -- start 2023-11-06T20:44:43.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.709+0000 7fad117ff700 1 -- start start 2023-11-06T20:44:43.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.710+0000 7fad117ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fad0c064dd0 0x7fad0c145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.710+0000 7fad117ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad0c154330 0x7fad0c0ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.710+0000 7fad117ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c154710 0x7fad0c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.710+0000 7fad117ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0c2af0 con 0x7fad0c154710 2023-11-06T20:44:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.710+0000 7fad117ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c145960 con 0x7fad0c154330 2023-11-06T20:44:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.710+0000 7fad117ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0c2200 con 0x7fad0c064dd0 2023-11-06T20:44:43.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.710+0000 7fad0b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c154710 0x7fad0c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.710+0000 7fad0b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c154710 0x7fad0c152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37166/0 (socket says 172.21.15.33:37166) 2023-11-06T20:44:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.710+0000 7fad0b7fe700 1 -- 172.21.15.33:0/759870939 learned_addr learned my addr 172.21.15.33:0/759870939 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7fad0affd700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fad0c064dd0 0x7fad0c145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7fad0a7fc700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad0c154330 0x7fad0c0ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7fad0b7fe700 1 -- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fad0c064dd0 msgr2=0x7fad0c145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7fad0b7fe700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fad0c064dd0 0x7fad0c145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7fad0b7fe700 1 -- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad0c154330 msgr2=0x7fad0c0ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7fad0b7fe700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad0c154330 0x7fad0c0ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7fad0b7fe700 1 -- 172.21.15.33:0/759870939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad04009a20 con 0x7fad0c154710 2023-11-06T20:44:43.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7fad0affd700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fad0c064dd0 0x7fad0c145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7fad0a7fc700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad0c154330 0x7fad0c0ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7fad0b7fe700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c154710 0x7fad0c152880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fad04011000 tx=0x7fad04009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7facf2ffd700 1 -- 172.21.15.33:0/759870939 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0401d630 con 0x7fad0c154710 2023-11-06T20:44:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.711+0000 7facf2ffd700 1 -- 172.21.15.33:0/759870939 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0401d790 con 0x7fad0c154710 2023-11-06T20:44:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.712+0000 7fad117ff700 1 -- 172.21.15.33:0/759870939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c0aeab0 con 0x7fad0c154710 2023-11-06T20:44:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.712+0000 7facf2ffd700 1 -- 172.21.15.33:0/759870939 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0401db30 con 0x7fad0c154710 2023-11-06T20:44:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.712+0000 7fad117ff700 1 -- 172.21.15.33:0/759870939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c0c2780 con 0x7fad0c154710 2023-11-06T20:44:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.713+0000 7fad117ff700 1 -- 172.21.15.33:0/759870939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c0c2c60 con 0x7fad0c154710 2023-11-06T20:44:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.714+0000 7facf2ffd700 1 -- 172.21.15.33:0/759870939 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fad04026410 con 0x7fad0c154710 2023-11-06T20:44:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.714+0000 7facf2ffd700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facf805f150 0x7facf8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.715+0000 7facf2ffd700 1 -- 172.21.15.33:0/759870939 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fad0407c110 con 0x7fad0c154710 2023-11-06T20:44:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.715+0000 7fad0affd700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facf805f150 0x7facf8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.715+0000 7fad0affd700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facf805f150 0x7facf8061610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7facfc007fd0 tx=0x7facfc009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.719+0000 7facf2ffd700 1 -- 172.21.15.33:0/759870939 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fad0404d420 con 0x7fad0c154710 2023-11-06T20:44:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.899+0000 7fad117ff700 1 -- 172.21.15.33:0/759870939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fad0c0652d0 con 0x7fad0c154710 2023-11-06T20:44:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.900+0000 7facf2ffd700 1 -- 172.21.15.33:0/759870939 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fad04050a40 con 0x7fad0c154710 2023-11-06T20:44:43.900 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:44:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 -- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facf805f150 msgr2=0x7facf8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facf805f150 0x7facf8061610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7facfc007fd0 tx=0x7facfc009040).stop 2023-11-06T20:44:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 -- 172.21.15.33:0/759870939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c154710 msgr2=0x7fad0c152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c154710 0x7fad0c152880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fad04011000 tx=0x7fad04009780).stop 2023-11-06T20:44:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 -- 172.21.15.33:0/759870939 shutdown_connections 2023-11-06T20:44:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facf805f150 0x7facf8061610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fad0c064dd0 0x7fad0c145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad0c154710 0x7fad0c152880 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 --2- 172.21.15.33:0/759870939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad0c154330 0x7fad0c0ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 -- 172.21.15.33:0/759870939 >> 172.21.15.33:0/759870939 conn(0x7fad0c0cb910 msgr2=0x7fad0c0c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 -- 172.21.15.33:0/759870939 shutdown_connections 2023-11-06T20:44:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:43.902+0000 7fad117ff700 1 -- 172.21.15.33:0/759870939 wait complete. 2023-11-06T20:44:43.920 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T20:44:44.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.118+0000 7f29ccdff700 1 Processor -- start 2023-11-06T20:44:44.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.118+0000 7f29ccdff700 1 -- start start 2023-11-06T20:44:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29ccdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29c8154330 0x7f29c80c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29ccdff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f29c8154710 0x7f29c80c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29ccdff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f29c80c7ff0 0x7f29c80646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29ccdff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c80c6b60 con 0x7f29c8154710 2023-11-06T20:44:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29ccdff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c80c5cd0 con 0x7f29c80c7ff0 2023-11-06T20:44:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29ccdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80c5f50 con 0x7f29c8154330 2023-11-06T20:44:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29c5d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f29c8154710 0x7f29c80c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29c5d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f29c8154710 0x7f29c80c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37172/0 (socket says 172.21.15.33:37172) 2023-11-06T20:44:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29c5d9b700 1 -- 172.21.15.33:0/3061702290 learned_addr learned my addr 172.21.15.33:0/3061702290 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29c6d9d700 1 --2- 172.21.15.33:0/3061702290 >> v2:172.21.15.69:3300/0 conn(0x7f29c80c7ff0 0x7f29c80646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.120+0000 7f29c659c700 1 --2- 172.21.15.33:0/3061702290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29c8154330 0x7f29c80c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.121+0000 7f29c5d9b700 1 -- 172.21.15.33:0/3061702290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29c8154330 msgr2=0x7f29c80c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.121+0000 7f29c5d9b700 1 --2- 172.21.15.33:0/3061702290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29c8154330 0x7f29c80c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.121+0000 7f29c5d9b700 1 -- 172.21.15.33:0/3061702290 >> v2:172.21.15.69:3300/0 conn(0x7f29c80c7ff0 msgr2=0x7f29c80646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.121+0000 7f29c5d9b700 1 --2- 172.21.15.33:0/3061702290 >> v2:172.21.15.69:3300/0 conn(0x7f29c80c7ff0 0x7f29c80646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.121+0000 7f29c5d9b700 1 -- 172.21.15.33:0/3061702290 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c80c6230 con 0x7f29c8154710 2023-11-06T20:44:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.121+0000 7f29c5d9b700 1 --2- 172.21.15.33:0/3061702290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 0x7f29c80c6ff0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f29c80c1e90 tx=0x7f29bc0090f0).ready entity=mon.0 client_cookie=3ce02c404f75e36 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.121+0000 7f29c559a700 1 -- 172.21.15.33:0/3061702290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00f510 con 0x7f29c8154710 2023-11-06T20:44:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.121+0000 7f29c559a700 1 -- 172.21.15.33:0/3061702290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc00f670 con 0x7f29c8154710 2023-11-06T20:44:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.121+0000 7f29c559a700 1 -- 172.21.15.33:0/3061702290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00f9f0 con 0x7f29c8154710 2023-11-06T20:44:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.122+0000 7f29ccdff700 1 -- 172.21.15.33:0/3061702290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 msgr2=0x7f29c80c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.122+0000 7f29ccdff700 1 --2- 172.21.15.33:0/3061702290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 0x7f29c80c6ff0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f29c80c1e90 tx=0x7f29bc0090f0).stop 2023-11-06T20:44:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.122+0000 7f29ccdff700 1 -- 172.21.15.33:0/3061702290 shutdown_connections 2023-11-06T20:44:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.122+0000 7f29ccdff700 1 --2- 172.21.15.33:0/3061702290 >> v2:172.21.15.69:3300/0 conn(0x7f29c80c7ff0 0x7f29c80646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.122+0000 7f29ccdff700 1 --2- 172.21.15.33:0/3061702290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29c8154330 0x7f29c80c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.122+0000 7f29ccdff700 1 --2- 172.21.15.33:0/3061702290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 0x7f29c80c6ff0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.122+0000 7f29ccdff700 1 -- 172.21.15.33:0/3061702290 >> 172.21.15.33:0/3061702290 conn(0x7f29c80cb910 msgr2=0x7f29c80c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.122+0000 7f29ccdff700 1 -- 172.21.15.33:0/3061702290 shutdown_connections 2023-11-06T20:44:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.122+0000 7f29ccdff700 1 -- 172.21.15.33:0/3061702290 wait complete. 2023-11-06T20:44:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.123+0000 7f29ccdff700 1 Processor -- start 2023-11-06T20:44:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.124+0000 7f29ccdff700 1 -- start start 2023-11-06T20:44:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.124+0000 7f29ccdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29c80c7ff0 0x7f29c81432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.124+0000 7f29ccdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c8154330 0x7f29c81531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29ccdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 0x7f29c80ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29ccdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80aee70 con 0x7f29c8154710 2023-11-06T20:44:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29ccdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80aebf0 con 0x7f29c8154330 2023-11-06T20:44:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29ccdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80c31a0 con 0x7f29c80c7ff0 2023-11-06T20:44:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 0x7f29c80ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 0x7f29c80ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37186/0 (socket says 172.21.15.33:37186) 2023-11-06T20:44:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c6d9d700 1 -- 172.21.15.33:0/453196152 learned_addr learned my addr 172.21.15.33:0/453196152 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c659c700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29c80c7ff0 0x7f29c81432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c5d9b700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c8154330 0x7f29c81531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c6d9d700 1 -- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29c80c7ff0 msgr2=0x7f29c81432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c6d9d700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29c80c7ff0 0x7f29c81432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c6d9d700 1 -- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c8154330 msgr2=0x7f29c81531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c6d9d700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c8154330 0x7f29c81531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c6d9d700 1 -- 172.21.15.33:0/453196152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc009370 con 0x7f29c8154710 2023-11-06T20:44:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c5d9b700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c8154330 0x7f29c81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.125+0000 7f29c6d9d700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 0x7f29c80ae700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f29c0009f20 tx=0x7f29c000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.126+0000 7f29b67fc700 1 -- 172.21.15.33:0/453196152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c0009c50 con 0x7f29c8154710 2023-11-06T20:44:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.126+0000 7f29b67fc700 1 -- 172.21.15.33:0/453196152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c000f040 con 0x7f29c8154710 2023-11-06T20:44:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.126+0000 7f29ccdff700 1 -- 172.21.15.33:0/453196152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c80c3480 con 0x7f29c8154710 2023-11-06T20:44:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.126+0000 7f29b67fc700 1 -- 172.21.15.33:0/453196152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c000ceb0 con 0x7f29c8154710 2023-11-06T20:44:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.126+0000 7f29ccdff700 1 -- 172.21.15.33:0/453196152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c80c3a40 con 0x7f29c8154710 2023-11-06T20:44:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.127+0000 7f29ccdff700 1 -- 172.21.15.33:0/453196152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c80c3c60 con 0x7f29c8154710 2023-11-06T20:44:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.128+0000 7f29b67fc700 1 -- 172.21.15.33:0/453196152 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f29c0013610 con 0x7f29c8154710 2023-11-06T20:44:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.128+0000 7f29b67fc700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a805f150 0x7f29a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.128+0000 7f29b67fc700 1 -- 172.21.15.33:0/453196152 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f29c007aad0 con 0x7f29c8154710 2023-11-06T20:44:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.129+0000 7f29c659c700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a805f150 0x7f29a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.129+0000 7f29c659c700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a805f150 0x7f29a8061610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f29b8009b30 tx=0x7f29b8008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.132+0000 7f29b67fc700 1 -- 172.21.15.33:0/453196152 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f29c004bda0 con 0x7f29c8154710 2023-11-06T20:44:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.312+0000 7f29ccdff700 1 -- 172.21.15.33:0/453196152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29c80652d0 con 0x7f29c8154710 2023-11-06T20:44:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.313+0000 7f29b67fc700 1 -- 172.21.15.33:0/453196152 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29c004f3c0 con 0x7f29c8154710 2023-11-06T20:44:44.313 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:44.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.314+0000 7f29ccdff700 1 -- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a805f150 msgr2=0x7f29a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.314+0000 7f29ccdff700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a805f150 0x7f29a8061610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f29b8009b30 tx=0x7f29b8008040).stop 2023-11-06T20:44:44.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.314+0000 7f29ccdff700 1 -- 172.21.15.33:0/453196152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 msgr2=0x7f29c80ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.314+0000 7f29ccdff700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 0x7f29c80ae700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f29c0009f20 tx=0x7f29c000b040).stop 2023-11-06T20:44:44.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.315+0000 7f29ccdff700 1 -- 172.21.15.33:0/453196152 shutdown_connections 2023-11-06T20:44:44.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.315+0000 7f29ccdff700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a805f150 0x7f29a8061610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.315+0000 7f29ccdff700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29c80c7ff0 0x7f29c81432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.315+0000 7f29ccdff700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29c8154710 0x7f29c80ae700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.315+0000 7f29ccdff700 1 --2- 172.21.15.33:0/453196152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c8154330 0x7f29c81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.315+0000 7f29ccdff700 1 -- 172.21.15.33:0/453196152 >> 172.21.15.33:0/453196152 conn(0x7f29c80cb910 msgr2=0x7f29c80c9350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.315+0000 7f29ccdff700 1 -- 172.21.15.33:0/453196152 shutdown_connections 2023-11-06T20:44:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.315+0000 7f29ccdff700 1 -- 172.21.15.33:0/453196152 wait complete. 2023-11-06T20:44:44.332 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T20:44:44.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.530+0000 7f42493ff700 1 Processor -- start 2023-11-06T20:44:44.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.530+0000 7f42493ff700 1 -- start start 2023-11-06T20:44:44.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.532+0000 7f42493ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4244154330 0x7f42440c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.532+0000 7f42493ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42440c6b60 con 0x7f4244154710 2023-11-06T20:44:44.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.532+0000 7f42493ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42440c7700 con 0x7f42440c7ff0 2023-11-06T20:44:44.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.532+0000 7f42493ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42440c5cd0 con 0x7f4244154330 2023-11-06T20:44:44.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.532+0000 7f42427fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4244154710 0x7f42440c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43882/0 (socket says 172.21.15.33:43882) 2023-11-06T20:44:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.532+0000 7f42427fc700 1 -- 172.21.15.33:0/1376255653 learned_addr learned my addr 172.21.15.33:0/1376255653 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.532+0000 7f4242ffd700 1 --2- 172.21.15.33:0/1376255653 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4244154330 0x7f42440c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.532+0000 7f4241ffb700 1 -- 172.21.15.33:0/1376255653 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161707828 0 0) 0x7f42440c6b60 con 0x7f4244154710 2023-11-06T20:44:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.532+0000 7f4241ffb700 1 -- 172.21.15.33:0/1376255653 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f422c0033f0 con 0x7f4244154710 2023-11-06T20:44:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.533+0000 7f4241ffb700 1 -- 172.21.15.33:0/1376255653 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521351726 0 0) 0x7f42440c7700 con 0x7f42440c7ff0 2023-11-06T20:44:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.533+0000 7f4241ffb700 1 -- 172.21.15.33:0/1376255653 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42440c6b60 con 0x7f42440c7ff0 2023-11-06T20:44:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.533+0000 7f4242ffd700 1 -- 172.21.15.33:0/1376255653 >> v1:172.21.15.69:6789/0 conn(0x7f42440c7ff0 legacy=0x7f42440646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.533+0000 7f4242ffd700 1 -- 172.21.15.33:0/1376255653 >> v1:172.21.15.33:6789/0 conn(0x7f4244154710 legacy=0x7f42440c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.533+0000 7f4242ffd700 1 -- 172.21.15.33:0/1376255653 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440c5f50 con 0x7f4244154330 2023-11-06T20:44:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.533+0000 7f4242ffd700 1 --2- 172.21.15.33:0/1376255653 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4244154330 0x7f42440c57e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4234009790 tx=0x7f4234009060).ready entity=mon.2 client_cookie=26821c1735fc8e72 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.533+0000 7f4241ffb700 1 -- 172.21.15.33:0/1376255653 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423400f510 con 0x7f4244154330 2023-11-06T20:44:44.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.533+0000 7f4241ffb700 1 -- 172.21.15.33:0/1376255653 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423400f670 con 0x7f4244154330 2023-11-06T20:44:44.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.533+0000 7f4241ffb700 1 -- 172.21.15.33:0/1376255653 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423400f9f0 con 0x7f4244154330 2023-11-06T20:44:44.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.534+0000 7f42493ff700 1 -- 172.21.15.33:0/1376255653 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4244154330 msgr2=0x7f42440c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.534+0000 7f42493ff700 1 --2- 172.21.15.33:0/1376255653 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4244154330 0x7f42440c57e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4234009790 tx=0x7f4234009060).stop 2023-11-06T20:44:44.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.534+0000 7f42493ff700 1 -- 172.21.15.33:0/1376255653 shutdown_connections 2023-11-06T20:44:44.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.534+0000 7f42493ff700 1 --2- 172.21.15.33:0/1376255653 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4244154330 0x7f42440c57e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.534+0000 7f42493ff700 1 -- 172.21.15.33:0/1376255653 >> 172.21.15.33:0/1376255653 conn(0x7f42440cb910 msgr2=0x7f42440c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:44.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.534+0000 7f42493ff700 1 -- 172.21.15.33:0/1376255653 shutdown_connections 2023-11-06T20:44:44.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.534+0000 7f42493ff700 1 -- 172.21.15.33:0/1376255653 wait complete. 2023-11-06T20:44:44.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.535+0000 7f42493ff700 1 Processor -- start 2023-11-06T20:44:44.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.535+0000 7f42493ff700 1 -- start start 2023-11-06T20:44:44.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.536+0000 7f42493ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42440c7ff0 0x7f42440b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.536+0000 7f42493ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4244154330 0x7f4244167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.536+0000 7f42493ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4244154710 0x7f4244164d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.536+0000 7f42493ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4244154f10 con 0x7f4244154710 2023-11-06T20:44:44.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.536+0000 7f42493ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42441551f0 con 0x7f4244154330 2023-11-06T20:44:44.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.536+0000 7f42493ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42441554d0 con 0x7f42440c7ff0 2023-11-06T20:44:44.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4244154710 0x7f4244164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4244154710 0x7f4244164d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37200/0 (socket says 172.21.15.33:37200) 2023-11-06T20:44:44.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42437fe700 1 -- 172.21.15.33:0/3785905570 learned_addr learned my addr 172.21.15.33:0/3785905570 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:44.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42427fc700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4244154330 0x7f4244167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f4242ffd700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42440c7ff0 0x7f42440b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42437fe700 1 -- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42440c7ff0 msgr2=0x7f42440b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42437fe700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42440c7ff0 0x7f42440b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42437fe700 1 -- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4244154330 msgr2=0x7f4244167ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42437fe700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4244154330 0x7f4244167ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42437fe700 1 -- 172.21.15.33:0/3785905570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340092a0 con 0x7f4244154710 2023-11-06T20:44:44.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42427fc700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4244154330 0x7f4244167ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:44.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f42437fe700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4244154710 0x7f4244164d30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f423c00c7c0 tx=0x7f423c007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.537+0000 7f422affd700 1 -- 172.21.15.33:0/3785905570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c002240 con 0x7f4244154710 2023-11-06T20:44:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.538+0000 7f422affd700 1 -- 172.21.15.33:0/3785905570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c0023a0 con 0x7f4244154710 2023-11-06T20:44:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.538+0000 7f42493ff700 1 -- 172.21.15.33:0/3785905570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42441557b0 con 0x7f4244154710 2023-11-06T20:44:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.538+0000 7f422affd700 1 -- 172.21.15.33:0/3785905570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0017e0 con 0x7f4244154710 2023-11-06T20:44:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.538+0000 7f42493ff700 1 -- 172.21.15.33:0/3785905570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4244167fb0 con 0x7f4244154710 2023-11-06T20:44:44.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.539+0000 7f42493ff700 1 -- 172.21.15.33:0/3785905570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42440b0c70 con 0x7f4244154710 2023-11-06T20:44:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.540+0000 7f422affd700 1 -- 172.21.15.33:0/3785905570 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f423c002510 con 0x7f4244154710 2023-11-06T20:44:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.541+0000 7f422affd700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f423005f150 0x7f4230061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.541+0000 7f422affd700 1 -- 172.21.15.33:0/3785905570 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f423c07a170 con 0x7f4244154710 2023-11-06T20:44:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.541+0000 7f4242ffd700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f423005f150 0x7f4230061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.542+0000 7f4242ffd700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f423005f150 0x7f4230061610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f4234006d30 tx=0x7f4234017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:44.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.545+0000 7f422affd700 1 -- 172.21.15.33:0/3785905570 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f423c04b380 con 0x7f4244154710 2023-11-06T20:44:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.724+0000 7f42493ff700 1 -- 172.21.15.33:0/3785905570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42440830b0 con 0x7f4244154710 2023-11-06T20:44:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.724+0000 7f422affd700 1 -- 172.21.15.33:0/3785905570 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f423c04e9a0 con 0x7f4244154710 2023-11-06T20:44:44.725 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:44.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 -- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f423005f150 msgr2=0x7f4230061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f423005f150 0x7f4230061610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f4234006d30 tx=0x7f4234017000).stop 2023-11-06T20:44:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 -- 172.21.15.33:0/3785905570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4244154710 msgr2=0x7f4244164d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4244154710 0x7f4244164d30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f423c00c7c0 tx=0x7f423c007f50).stop 2023-11-06T20:44:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 -- 172.21.15.33:0/3785905570 shutdown_connections 2023-11-06T20:44:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f423005f150 0x7f4230061610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42440c7ff0 0x7f42440b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4244154710 0x7f4244164d30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 --2- 172.21.15.33:0/3785905570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4244154330 0x7f4244167ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 -- 172.21.15.33:0/3785905570 >> 172.21.15.33:0/3785905570 conn(0x7f42440cb910 msgr2=0x7f42440ac9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 -- 172.21.15.33:0/3785905570 shutdown_connections 2023-11-06T20:44:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.726+0000 7f42493ff700 1 -- 172.21.15.33:0/3785905570 wait complete. 2023-11-06T20:44:44.771 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-11-06T20:44:44.772 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-06T20:44:44.772 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-11-06T20:44:44.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.969+0000 7fd9107ff700 1 Processor -- start 2023-11-06T20:44:44.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.969+0000 7fd9107ff700 1 -- start start 2023-11-06T20:44:44.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.971+0000 7fd9107ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd9080c5d50 0x7fd9080c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.971+0000 7fd9107ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9080bfd70 0x7fd9080c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.971+0000 7fd9107ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9081554a0 con 0x7fd9080c1840 2023-11-06T20:44:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.971+0000 7fd9107ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd908154960 con 0x7fd9080c5d50 2023-11-06T20:44:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.971+0000 7fd9107ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080c66c0 con 0x7fd9080bfd70 2023-11-06T20:44:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.971+0000 7fd90e59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd9080c1840 0x7fd9080a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43896/0 (socket says 172.21.15.33:43896) 2023-11-06T20:44:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.971+0000 7fd90e59b700 1 -- 172.21.15.33:0/1347878038 learned_addr learned my addr 172.21.15.33:0/1347878038 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:44.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.971+0000 7fd90ed9c700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9080bfd70 0x7fd9080c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.971+0000 7fd90dd9a700 1 --2- 172.21.15.33:0/1347878038 >> v2:172.21.15.69:3300/0 conn(0x7fd9080c5d50 0x7fd9080c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.971+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90187032 0 0) 0x7fd9081554a0 con 0x7fd9080c1840 2023-11-06T20:44:44.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8f40033f0 con 0x7fd9080c1840 2023-11-06T20:44:44.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1788995887 0 0) 0x7fd8f40033f0 con 0x7fd9080c1840 2023-11-06T20:44:44.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9080bfd70 msgr2=0x7fd9080c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:44.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9080bfd70 0x7fd9080c01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 >> v2:172.21.15.69:3300/0 conn(0x7fd9080c5d50 msgr2=0x7fd9080c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:44.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 --2- 172.21.15.33:0/1347878038 >> v2:172.21.15.69:3300/0 conn(0x7fd9080c5d50 0x7fd9080c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908155930 con 0x7fd9080c1840 2023-11-06T20:44:44.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fd904001470 con 0x7fd9080c1840 2023-11-06T20:44:44.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90ed9c700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9080bfd70 0x7fd9080c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:44.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 >> v1:172.21.15.33:6789/0 conn(0x7fd9080c1840 legacy=0x7fd9080a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8f40038e0 0x7fd8f4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 0x7fd8f4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9080bfd70 0x7fd8f4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908154960 con 0x7fd8f4004690 2023-11-06T20:44:44.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080c66c0 con 0x7fd8f40038e0 2023-11-06T20:44:44.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f40044f0 con 0x7fd9080bfd70 2023-11-06T20:44:44.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90dd9a700 1 --2- 172.21.15.33:0/1347878038 >> v2:172.21.15.69:3300/0 conn(0x7fd9080c5d50 0x7fd9080c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:44.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90e59b700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8f40038e0 0x7fd8f4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90dd9a700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 0x7fd8f4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.972+0000 7fd90ed9c700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9080bfd70 0x7fd8f4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.973+0000 7fd90dd9a700 1 -- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9080bfd70 msgr2=0x7fd8f4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:44.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.973+0000 7fd90dd9a700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9080bfd70 0x7fd8f4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.973+0000 7fd90dd9a700 1 -- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8f40038e0 msgr2=0x7fd8f4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:44.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.973+0000 7fd90dd9a700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8f40038e0 0x7fd8f4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.973+0000 7fd90dd9a700 1 -- 172.21.15.33:0/1347878038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd900009a20 con 0x7fd8f4004690 2023-11-06T20:44:44.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.973+0000 7fd90e59b700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8f40038e0 0x7fd8f4003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:44.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.973+0000 7fd90dd9a700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 0x7fd8f4004b10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd8f8009c30 tx=0x7fd8f8009060).ready entity=mon.0 client_cookie=ac190ce410d923a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:44.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.973+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80095c0 con 0x7fd8f4004690 2023-11-06T20:44:44.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.973+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f800e710 con 0x7fd8f4004690 2023-11-06T20:44:44.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.973+0000 7fd90d599700 1 -- 172.21.15.33:0/1347878038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800ea90 con 0x7fd8f4004690 2023-11-06T20:44:44.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.974+0000 7fd9107ff700 1 -- 172.21.15.33:0/1347878038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 msgr2=0x7fd8f4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.974+0000 7fd9107ff700 1 --2- 172.21.15.33:0/1347878038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 0x7fd8f4004b10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd8f8009c30 tx=0x7fd8f8009060).stop 2023-11-06T20:44:44.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.974+0000 7fd90ed9c700 1 -- 172.21.15.33:0/1347878038 reap_dead start 2023-11-06T20:44:44.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.974+0000 7fd9107ff700 1 -- 172.21.15.33:0/1347878038 shutdown_connections 2023-11-06T20:44:44.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.974+0000 7fd9107ff700 1 -- 172.21.15.33:0/1347878038 >> 172.21.15.33:0/1347878038 conn(0x7fd90805ec80 msgr2=0x7fd9080cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:44.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.974+0000 7fd9107ff700 1 -- 172.21.15.33:0/1347878038 shutdown_connections 2023-11-06T20:44:44.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.974+0000 7fd9107ff700 1 -- 172.21.15.33:0/1347878038 wait complete. 2023-11-06T20:44:44.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.976+0000 7fd9107ff700 1 Processor -- start 2023-11-06T20:44:44.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.976+0000 7fd9107ff700 1 -- start start 2023-11-06T20:44:44.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd9107ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 0x7fd9080ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd9107ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd908157740 0x7fd9081564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd9107ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd908149ca0 0x7fd90814e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd9107ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908156d30 con 0x7fd8f4004690 2023-11-06T20:44:44.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd9107ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080ad110 con 0x7fd908149ca0 2023-11-06T20:44:44.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd9107ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080ac7e0 con 0x7fd908157740 2023-11-06T20:44:44.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd90e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 0x7fd9080ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd90e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 0x7fd9080ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37218/0 (socket says 172.21.15.33:37218) 2023-11-06T20:44:44.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd90e59b700 1 -- 172.21.15.33:0/3969665201 learned_addr learned my addr 172.21.15.33:0/3969665201 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:44.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd90dd9a700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd908157740 0x7fd9081564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd90ed9c700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd908149ca0 0x7fd90814e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd90e59b700 1 -- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd908157740 msgr2=0x7fd9081564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.977+0000 7fd90e59b700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd908157740 0x7fd9081564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.978+0000 7fd90e59b700 1 -- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd908149ca0 msgr2=0x7fd90814e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:44.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.978+0000 7fd90e59b700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd908149ca0 0x7fd90814e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:44.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.978+0000 7fd90e59b700 1 -- 172.21.15.33:0/3969665201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080acac0 con 0x7fd8f4004690 2023-11-06T20:44:44.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.978+0000 7fd90ed9c700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd908149ca0 0x7fd90814e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:44.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.978+0000 7fd90e59b700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 0x7fd9080ad5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fd904001010 tx=0x7fd908155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:44.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.978+0000 7fd8fe7fc700 1 -- 172.21.15.33:0/3969665201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90400f840 con 0x7fd8f4004690 2023-11-06T20:44:44.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.978+0000 7fd8fe7fc700 1 -- 172.21.15.33:0/3969665201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90400f9a0 con 0x7fd8f4004690 2023-11-06T20:44:44.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.978+0000 7fd8fe7fc700 1 -- 172.21.15.33:0/3969665201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90400fd20 con 0x7fd8f4004690 2023-11-06T20:44:44.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.978+0000 7fd9107ff700 1 -- 172.21.15.33:0/3969665201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9080acda0 con 0x7fd8f4004690 2023-11-06T20:44:44.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.978+0000 7fd9107ff700 1 -- 172.21.15.33:0/3969665201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90814a120 con 0x7fd8f4004690 2023-11-06T20:44:44.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.979+0000 7fd9107ff700 1 -- 172.21.15.33:0/3969665201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd908157b20 con 0x7fd8f4004690 2023-11-06T20:44:44.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.980+0000 7fd8fe7fc700 1 -- 172.21.15.33:0/3969665201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd904012560 con 0x7fd8f4004690 2023-11-06T20:44:44.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.981+0000 7fd8fe7fc700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd8ec05f150 0x7fd8ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:44.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.981+0000 7fd8fe7fc700 1 -- 172.21.15.33:0/3969665201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd904079cd0 con 0x7fd8f4004690 2023-11-06T20:44:44.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.981+0000 7fd90dd9a700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd8ec05f150 0x7fd8ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:44.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.982+0000 7fd90dd9a700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd8ec05f150 0x7fd8ec061610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fd908156e90 tx=0x7fd8f8019000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:44.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:44.985+0000 7fd8fe7fc700 1 -- 172.21.15.33:0/3969665201 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd90404b050 con 0x7fd8f4004690 2023-11-06T20:44:45.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.234+0000 7fd9107ff700 1 -- 172.21.15.33:0/3969665201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fd90806c6c0 con 0x7fd8f4004690 2023-11-06T20:44:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.806+0000 7fd8fe7fc700 1 -- 172.21.15.33:0/3969665201 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd904019030 con 0x7fd8f4004690 2023-11-06T20:44:45.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.808+0000 7fd9107ff700 1 -- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd8ec05f150 msgr2=0x7fd8ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:45.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.808+0000 7fd9107ff700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd8ec05f150 0x7fd8ec061610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fd908156e90 tx=0x7fd8f8019000).stop 2023-11-06T20:44:45.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.808+0000 7fd9107ff700 1 -- 172.21.15.33:0/3969665201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 msgr2=0x7fd9080ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:45.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.808+0000 7fd9107ff700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 0x7fd9080ad5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fd904001010 tx=0x7fd908155410).stop 2023-11-06T20:44:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.809+0000 7fd9107ff700 1 -- 172.21.15.33:0/3969665201 shutdown_connections 2023-11-06T20:44:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.809+0000 7fd9107ff700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd8ec05f150 0x7fd8ec061610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.809+0000 7fd9107ff700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd908149ca0 0x7fd90814e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.809+0000 7fd9107ff700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4004690 0x7fd9080ad5a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.809+0000 7fd9107ff700 1 --2- 172.21.15.33:0/3969665201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd908157740 0x7fd9081564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.809+0000 7fd9107ff700 1 -- 172.21.15.33:0/3969665201 >> 172.21.15.33:0/3969665201 conn(0x7fd90805ec80 msgr2=0x7fd9081503d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.809+0000 7fd9107ff700 1 -- 172.21.15.33:0/3969665201 shutdown_connections 2023-11-06T20:44:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:45.809+0000 7fd9107ff700 1 -- 172.21.15.33:0/3969665201 wait complete. 2023-11-06T20:44:45.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T20:44:46.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.033+0000 7ff6695ff700 1 Processor -- start 2023-11-06T20:44:46.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.033+0000 7ff6695ff700 1 -- start start 2023-11-06T20:44:46.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.034+0000 7ff6695ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff664133f10 0x7ff6641520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:46.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.034+0000 7ff6695ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6641342f0 0x7ff6640c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:46.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.034+0000 7ff6695ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff6640c3940 0x7ff6640c1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:46.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.034+0000 7ff6695ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66414bc20 con 0x7ff664133f10 2023-11-06T20:44:46.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.034+0000 7ff6695ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6640c3dd0 con 0x7ff6640c3940 2023-11-06T20:44:46.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.034+0000 7ff6695ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6641525e0 con 0x7ff6641342f0 2023-11-06T20:44:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff662ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff664133f10 0x7ff6641520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff662ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff664133f10 0x7ff6641520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37232/0 (socket says 172.21.15.33:37232) 2023-11-06T20:44:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff662ffd700 1 -- 172.21.15.33:0/40685624 learned_addr learned my addr 172.21.15.33:0/40685624 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff6627fc700 1 --2- 172.21.15.33:0/40685624 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6641342f0 0x7ff6640c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff6637fe700 1 --2- 172.21.15.33:0/40685624 >> v2:172.21.15.69:3300/0 conn(0x7ff6640c3940 0x7ff6640c1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:46.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff662ffd700 1 -- 172.21.15.33:0/40685624 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6641342f0 msgr2=0x7ff6640c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:46.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff662ffd700 1 --2- 172.21.15.33:0/40685624 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6641342f0 0x7ff6640c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:46.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff662ffd700 1 -- 172.21.15.33:0/40685624 >> v2:172.21.15.69:3300/0 conn(0x7ff6640c3940 msgr2=0x7ff6640c1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:46.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff662ffd700 1 --2- 172.21.15.33:0/40685624 >> v2:172.21.15.69:3300/0 conn(0x7ff6640c3940 0x7ff6640c1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:46.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff662ffd700 1 -- 172.21.15.33:0/40685624 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff664154f90 con 0x7ff664133f10 2023-11-06T20:44:46.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff662ffd700 1 --2- 172.21.15.33:0/40685624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 0x7ff6641520f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7ff664066410 tx=0x7ff6540090f0).ready entity=mon.0 client_cookie=2770b27191e45510 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:46.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.035+0000 7ff661ffb700 1 -- 172.21.15.33:0/40685624 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65400b600 con 0x7ff664133f10 2023-11-06T20:44:46.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff661ffb700 1 -- 172.21.15.33:0/40685624 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65400b760 con 0x7ff664133f10 2023-11-06T20:44:46.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff661ffb700 1 -- 172.21.15.33:0/40685624 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65400bae0 con 0x7ff664133f10 2023-11-06T20:44:46.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff6695ff700 1 -- 172.21.15.33:0/40685624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 msgr2=0x7ff6641520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:46.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff6695ff700 1 --2- 172.21.15.33:0/40685624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 0x7ff6641520f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7ff664066410 tx=0x7ff6540090f0).stop 2023-11-06T20:44:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff6695ff700 1 -- 172.21.15.33:0/40685624 shutdown_connections 2023-11-06T20:44:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff6695ff700 1 --2- 172.21.15.33:0/40685624 >> v2:172.21.15.69:3300/0 conn(0x7ff6640c3940 0x7ff6640c1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff6695ff700 1 --2- 172.21.15.33:0/40685624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 0x7ff6641520f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff6695ff700 1 --2- 172.21.15.33:0/40685624 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6641342f0 0x7ff6640c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff6695ff700 1 -- 172.21.15.33:0/40685624 >> 172.21.15.33:0/40685624 conn(0x7ff664062710 msgr2=0x7ff664062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff6695ff700 1 -- 172.21.15.33:0/40685624 shutdown_connections 2023-11-06T20:44:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.036+0000 7ff6695ff700 1 -- 172.21.15.33:0/40685624 wait complete. 2023-11-06T20:44:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.038+0000 7ff6695ff700 1 Processor -- start 2023-11-06T20:44:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.038+0000 7ff6695ff700 1 -- start start 2023-11-06T20:44:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.039+0000 7ff6695ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6640c3940 0x7ff664162ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.039+0000 7ff6695ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 0x7ff6641600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.039+0000 7ff6695ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6641342f0 0x7ff6640c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.039+0000 7ff6695ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640aa990 con 0x7ff664133f10 2023-11-06T20:44:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.039+0000 7ff6695ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640a9d80 con 0x7ff6640c3940 2023-11-06T20:44:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.039+0000 7ff6695ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640aa060 con 0x7ff6641342f0 2023-11-06T20:44:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.039+0000 7ff6627fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 0x7ff6641600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.039+0000 7ff6627fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 0x7ff6641600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37236/0 (socket says 172.21.15.33:37236) 2023-11-06T20:44:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.039+0000 7ff6627fc700 1 -- 172.21.15.33:0/495592157 learned_addr learned my addr 172.21.15.33:0/495592157 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:46.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff6627fc700 1 -- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6641342f0 msgr2=0x7ff6640c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:46.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff6627fc700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6641342f0 0x7ff6640c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:44:46.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff6627fc700 1 -- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6640c3940 msgr2=0x7ff664162ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:46.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff6627fc700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6640c3940 0x7ff664162ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:44:46.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff6627fc700 1 -- 172.21.15.33:0/495592157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff654009370 con 0x7ff664133f10 2023-11-06T20:44:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff6627fc700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 0x7ff6641600f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7ff658006980 tx=0x7ff658008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff64affd700 1 -- 172.21.15.33:0/495592157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6580125f0 con 0x7ff664133f10 2023-11-06T20:44:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff64affd700 1 -- 172.21.15.33:0/495592157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658012750 con 0x7ff664133f10 2023-11-06T20:44:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff6695ff700 1 -- 172.21.15.33:0/495592157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6640aa340 con 0x7ff664133f10 2023-11-06T20:44:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff64affd700 1 -- 172.21.15.33:0/495592157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658012ad0 con 0x7ff664133f10 2023-11-06T20:44:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.040+0000 7ff6695ff700 1 -- 172.21.15.33:0/495592157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff664163390 con 0x7ff664133f10 2023-11-06T20:44:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.041+0000 7ff6695ff700 1 -- 172.21.15.33:0/495592157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff664155270 con 0x7ff664133f10 2023-11-06T20:44:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.042+0000 7ff64affd700 1 -- 172.21.15.33:0/495592157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff658013480 con 0x7ff664133f10 2023-11-06T20:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.043+0000 7ff64affd700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff65005f150 0x7ff650061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.043+0000 7ff64affd700 1 -- 172.21.15.33:0/495592157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff658079f90 con 0x7ff664133f10 2023-11-06T20:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.044+0000 7ff662ffd700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff65005f150 0x7ff650061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.044+0000 7ff662ffd700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff65005f150 0x7ff650061610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff654009fd0 tx=0x7ff654017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.047+0000 7ff64affd700 1 -- 172.21.15.33:0/495592157 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff65804b140 con 0x7ff664133f10 2023-11-06T20:44:46.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.227+0000 7ff6695ff700 1 -- 172.21.15.33:0/495592157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff664063c00 con 0x7ff664133f10 2023-11-06T20:44:46.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.231+0000 7ff64affd700 1 -- 172.21.15.33:0/495592157 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff65804e760 con 0x7ff664133f10 2023-11-06T20:44:46.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.233+0000 7ff6695ff700 1 -- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff65005f150 msgr2=0x7ff650061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:46.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.233+0000 7ff6695ff700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff65005f150 0x7ff650061610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff654009fd0 tx=0x7ff654017000).stop 2023-11-06T20:44:46.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.233+0000 7ff6695ff700 1 -- 172.21.15.33:0/495592157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 msgr2=0x7ff6641600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:46.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.233+0000 7ff6695ff700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 0x7ff6641600f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7ff658006980 tx=0x7ff658008040).stop 2023-11-06T20:44:46.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.233+0000 7ff6695ff700 1 -- 172.21.15.33:0/495592157 shutdown_connections 2023-11-06T20:44:46.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.233+0000 7ff6695ff700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff65005f150 0x7ff650061610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:46.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.233+0000 7ff6695ff700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6641342f0 0x7ff6640c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:44:46.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.233+0000 7ff6695ff700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6640c3940 0x7ff664162ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:44:46.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.233+0000 7ff6695ff700 1 --2- 172.21.15.33:0/495592157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664133f10 0x7ff6641600f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:46.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.233+0000 7ff6695ff700 1 -- 172.21.15.33:0/495592157 >> 172.21.15.33:0/495592157 conn(0x7ff664062710 msgr2=0x7ff6640a8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:46.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.234+0000 7ff6695ff700 1 -- 172.21.15.33:0/495592157 shutdown_connections 2023-11-06T20:44:46.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:46.234+0000 7ff6695ff700 1 -- 172.21.15.33:0/495592157 wait complete. 2023-11-06T20:44:46.251 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T20:44:46.252 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:44:46.352 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:44:46.451 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T20:44:46.451 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T20:44:46.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:44:46.452 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-06T20:44:46.530 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:44:46.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:46.531 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T20:44:46.587 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-06T20:44:46.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:44:46.588 DEBUG:teuthology.orchestra.run.smithi069:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-06T20:44:46.666 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T20:44:46.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:46.667 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T20:44:46.725 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-06T20:44:46.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:44:46.726 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-06T20:44:46.802 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:44:46.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:46.803 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T20:44:46.852 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-06T20:44:46.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:44:46.853 DEBUG:teuthology.orchestra.run.smithi069:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-06T20:44:46.927 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T20:44:46.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:44:46.928 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T20:44:46.975 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T20:44:46.975 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:44:47.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.170+0000 7fa1d89ff700 1 Processor -- start 2023-11-06T20:44:47.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.170+0000 7fa1d89ff700 1 -- start start 2023-11-06T20:44:47.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.171+0000 7fa1d89ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa1d40c1840 0x7fa1d40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.171+0000 7fa1d89ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40c6460 0x7fa1d40bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.171+0000 7fa1d89ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d41554a0 con 0x7fa1d40aa2d0 2023-11-06T20:44:47.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.171+0000 7fa1d89ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d4154960 con 0x7fa1d40c1840 2023-11-06T20:44:47.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.171+0000 7fa1d89ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4155930 con 0x7fa1d40c6460 2023-11-06T20:44:47.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d1d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa1d40aa2d0 0x7fa1d40c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43906/0 (socket says 172.21.15.33:43906) 2023-11-06T20:44:47.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d1d9b700 1 -- 172.21.15.33:0/3659345073 learned_addr learned my addr 172.21.15.33:0/3659345073 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:47.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d2d9d700 1 --2- 172.21.15.33:0/3659345073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40c6460 0x7fa1d40bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d259c700 1 --2- 172.21.15.33:0/3659345073 >> v2:172.21.15.69:3300/0 conn(0x7fa1d40c1840 0x7fa1d40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d159a700 1 -- 172.21.15.33:0/3659345073 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657025056 0 0) 0x7fa1d41554a0 con 0x7fa1d40aa2d0 2023-11-06T20:44:47.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d159a700 1 -- 172.21.15.33:0/3659345073 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1bc0033f0 con 0x7fa1d40aa2d0 2023-11-06T20:44:47.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d2d9d700 1 -- 172.21.15.33:0/3659345073 >> v2:172.21.15.69:3300/0 conn(0x7fa1d40c1840 msgr2=0x7fa1d40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:47.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d2d9d700 1 --2- 172.21.15.33:0/3659345073 >> v2:172.21.15.69:3300/0 conn(0x7fa1d40c1840 0x7fa1d40a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d2d9d700 1 -- 172.21.15.33:0/3659345073 >> v1:172.21.15.33:6789/0 conn(0x7fa1d40aa2d0 legacy=0x7fa1d40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d2d9d700 1 -- 172.21.15.33:0/3659345073 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d4154b70 con 0x7fa1d40c6460 2023-11-06T20:44:47.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d259c700 1 --2- 172.21.15.33:0/3659345073 >> v2:172.21.15.69:3300/0 conn(0x7fa1d40c1840 0x7fa1d40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:47.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.172+0000 7fa1d2d9d700 1 --2- 172.21.15.33:0/3659345073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40c6460 0x7fa1d40bfd70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa1cc009e60 tx=0x7fa1cc009780).ready entity=mon.2 client_cookie=600ff8187093a9b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:47.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d159a700 1 -- 172.21.15.33:0/3659345073 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00e070 con 0x7fa1d40c6460 2023-11-06T20:44:47.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d159a700 1 -- 172.21.15.33:0/3659345073 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc00bab0 con 0x7fa1d40c6460 2023-11-06T20:44:47.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d159a700 1 -- 172.21.15.33:0/3659345073 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00be30 con 0x7fa1d40c6460 2023-11-06T20:44:47.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d89ff700 1 -- 172.21.15.33:0/3659345073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40c6460 msgr2=0x7fa1d40bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d89ff700 1 --2- 172.21.15.33:0/3659345073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40c6460 0x7fa1d40bfd70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa1cc009e60 tx=0x7fa1cc009780).stop 2023-11-06T20:44:47.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d89ff700 1 -- 172.21.15.33:0/3659345073 shutdown_connections 2023-11-06T20:44:47.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d89ff700 1 --2- 172.21.15.33:0/3659345073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40c6460 0x7fa1d40bfd70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d89ff700 1 --2- 172.21.15.33:0/3659345073 >> v2:172.21.15.69:3300/0 conn(0x7fa1d40c1840 0x7fa1d40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d89ff700 1 -- 172.21.15.33:0/3659345073 >> 172.21.15.33:0/3659345073 conn(0x7fa1d405ec80 msgr2=0x7fa1d40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:47.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d89ff700 1 -- 172.21.15.33:0/3659345073 shutdown_connections 2023-11-06T20:44:47.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.173+0000 7fa1d89ff700 1 -- 172.21.15.33:0/3659345073 wait complete. 2023-11-06T20:44:47.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.175+0000 7fa1d89ff700 1 Processor -- start 2023-11-06T20:44:47.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.176+0000 7fa1d89ff700 1 -- start start 2023-11-06T20:44:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d89ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40aa2d0 0x7fa1d40c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d89ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d40c1840 0x7fa1d40afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d89ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1d40c6460 0x7fa1d40addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d89ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d40b0550 con 0x7fa1d40c1840 2023-11-06T20:44:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d89ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4155c40 con 0x7fa1d40c6460 2023-11-06T20:44:47.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d89ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4155ee0 con 0x7fa1d40aa2d0 2023-11-06T20:44:47.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d40c1840 0x7fa1d40afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d40c1840 0x7fa1d40afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37246/0 (socket says 172.21.15.33:37246) 2023-11-06T20:44:47.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d1d9b700 1 -- 172.21.15.33:0/1899559024 learned_addr learned my addr 172.21.15.33:0/1899559024 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:47.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d259c700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40aa2d0 0x7fa1d40c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.177+0000 7fa1d2d9d700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1d40c6460 0x7fa1d40addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1d1d9b700 1 -- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40aa2d0 msgr2=0x7fa1d40c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1d1d9b700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40aa2d0 0x7fa1d40c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1d1d9b700 1 -- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1d40c6460 msgr2=0x7fa1d40addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1d1d9b700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1d40c6460 0x7fa1d40addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1d1d9b700 1 -- 172.21.15.33:0/1899559024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c40092a0 con 0x7fa1d40c1840 2023-11-06T20:44:47.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1d1d9b700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d40c1840 0x7fa1d40afd70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fa1c800c4c0 tx=0x7fa1c8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:47.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1c27fc700 1 -- 172.21.15.33:0/1899559024 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800ce70 con 0x7fa1d40c1840 2023-11-06T20:44:47.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1c27fc700 1 -- 172.21.15.33:0/1899559024 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c80072e0 con 0x7fa1d40c1840 2023-11-06T20:44:47.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1c27fc700 1 -- 172.21.15.33:0/1899559024 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80057a0 con 0x7fa1d40c1840 2023-11-06T20:44:47.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1d89ff700 1 -- 172.21.15.33:0/1899559024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1cc009a20 con 0x7fa1d40c1840 2023-11-06T20:44:47.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.178+0000 7fa1d89ff700 1 -- 172.21.15.33:0/1899559024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d41564a0 con 0x7fa1d40c1840 2023-11-06T20:44:47.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.180+0000 7fa1d89ff700 1 -- 172.21.15.33:0/1899559024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d406c6c0 con 0x7fa1d40c1840 2023-11-06T20:44:47.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.180+0000 7fa1c27fc700 1 -- 172.21.15.33:0/1899559024 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa1c8005bd0 con 0x7fa1d40c1840 2023-11-06T20:44:47.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.181+0000 7fa1c27fc700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1b405f150 0x7fa1b4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.182+0000 7fa1c27fc700 1 -- 172.21.15.33:0/1899559024 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa1c8079890 con 0x7fa1d40c1840 2023-11-06T20:44:47.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.182+0000 7fa1d259c700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1b405f150 0x7fa1b4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.182+0000 7fa1d259c700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1b405f150 0x7fa1b4061610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fa1d40c48e0 tx=0x7fa1c4009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:47.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.186+0000 7fa1c27fc700 1 -- 172.21.15.33:0/1899559024 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa1c804aaa0 con 0x7fa1d40c1840 2023-11-06T20:44:47.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.408+0000 7fa1d89ff700 1 -- 172.21.15.33:0/1899559024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1d4063f20 con 0x7fa1d40c1840 2023-11-06T20:44:47.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.408+0000 7fa1c27fc700 1 -- 172.21.15.33:0/1899559024 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4480 (secure 0 0 0) 0x7fa1c804e0c0 con 0x7fa1d40c1840 2023-11-06T20:44:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:47.409 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4123875619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123875619},{"type":"v1","addr":"172.21.15.33:6834","nonce":4123875619}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3506554570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3506554570},{"type":"v1","addr":"172.21.15.33:6835","nonce":3506554570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/239985290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":239985290},{"type":"v1","addr":"172.21.15.69:6835","nonce":239985290}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:44:06.383743+0000","modified":"2023-11-06T20:44:09.389408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/2681789033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2681789033},{"type":"v1","addr":"172.21.15.69:6837","nonce":2681789033}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-06T20:44:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 -- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1b405f150 msgr2=0x7fa1b4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1b405f150 0x7fa1b4061610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fa1d40c48e0 tx=0x7fa1c4009060).stop 2023-11-06T20:44:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 -- 172.21.15.33:0/1899559024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d40c1840 msgr2=0x7fa1d40afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d40c1840 0x7fa1d40afd70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fa1c800c4c0 tx=0x7fa1c8007c50).stop 2023-11-06T20:44:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 -- 172.21.15.33:0/1899559024 shutdown_connections 2023-11-06T20:44:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1b405f150 0x7fa1b4061610 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa1d40aa2d0 0x7fa1d40c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1d40c6460 0x7fa1d40addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 --2- 172.21.15.33:0/1899559024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d40c1840 0x7fa1d40afd70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 -- 172.21.15.33:0/1899559024 >> 172.21.15.33:0/1899559024 conn(0x7fa1d405ec80 msgr2=0x7fa1d4152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 -- 172.21.15.33:0/1899559024 shutdown_connections 2023-11-06T20:44:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.410+0000 7fa1d89ff700 1 -- 172.21.15.33:0/1899559024 wait complete. 2023-11-06T20:44:47.414 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2023-11-06T20:44:47.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.624+0000 7fd5055ff700 1 Processor -- start 2023-11-06T20:44:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.624+0000 7fd5055ff700 1 -- start start 2023-11-06T20:44:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.625+0000 7fd5055ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd500151dc0 0x7fd5000c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.625+0000 7fd5055ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd500154e90 con 0x7fd5001521a0 2023-11-06T20:44:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.625+0000 7fd5055ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5000639d0 con 0x7fd500155320 2023-11-06T20:44:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.625+0000 7fd5055ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd500063cb0 con 0x7fd500151dc0 2023-11-06T20:44:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.626+0000 7fd4feffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd500151dc0 0x7fd5000c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.626+0000 7fd4feffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd500151dc0 0x7fd5000c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37254/0 (socket says 172.21.15.33:37254) 2023-11-06T20:44:47.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.626+0000 7fd4feffd700 1 -- 172.21.15.33:0/715654371 learned_addr learned my addr 172.21.15.33:0/715654371 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:47.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.626+0000 7fd4fe7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd5001521a0 0x7fd500063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43912/0 (socket says 172.21.15.33:43912) 2023-11-06T20:44:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.626+0000 7fd4feffd700 1 -- 172.21.15.33:0/715654371 >> v1:172.21.15.69:6789/0 conn(0x7fd500155320 legacy=0x7fd5000c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.626+0000 7fd4feffd700 1 -- 172.21.15.33:0/715654371 >> v1:172.21.15.33:6789/0 conn(0x7fd5001521a0 legacy=0x7fd500063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.626+0000 7fd4feffd700 1 -- 172.21.15.33:0/715654371 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd500063f90 con 0x7fd500151dc0 2023-11-06T20:44:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.626+0000 7fd4fdffb700 1 -- 172.21.15.33:0/715654371 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154010805 0 0) 0x7fd500154e90 con 0x7fd5001521a0 2023-11-06T20:44:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.626+0000 7fd4feffd700 1 --2- 172.21.15.33:0/715654371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500151dc0 0x7fd5000c0c50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fd4f0006fd0 tx=0x7fd4f00090f0).ready entity=mon.0 client_cookie=1d3403998ee13820 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.627+0000 7fd4fdffb700 1 -- 172.21.15.33:0/715654371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000b600 con 0x7fd500151dc0 2023-11-06T20:44:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.627+0000 7fd4fdffb700 1 -- 172.21.15.33:0/715654371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f000b760 con 0x7fd500151dc0 2023-11-06T20:44:47.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.627+0000 7fd4fdffb700 1 -- 172.21.15.33:0/715654371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000bae0 con 0x7fd500151dc0 2023-11-06T20:44:47.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.627+0000 7fd5055ff700 1 -- 172.21.15.33:0/715654371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500151dc0 msgr2=0x7fd5000c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.627+0000 7fd5055ff700 1 --2- 172.21.15.33:0/715654371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500151dc0 0x7fd5000c0c50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fd4f0006fd0 tx=0x7fd4f00090f0).stop 2023-11-06T20:44:47.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.627+0000 7fd5055ff700 1 -- 172.21.15.33:0/715654371 shutdown_connections 2023-11-06T20:44:47.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.627+0000 7fd5055ff700 1 --2- 172.21.15.33:0/715654371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd500151dc0 0x7fd5000c0c50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.627+0000 7fd5055ff700 1 -- 172.21.15.33:0/715654371 >> 172.21.15.33:0/715654371 conn(0x7fd50005e1b0 msgr2=0x7fd50005e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:47.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.627+0000 7fd5055ff700 1 -- 172.21.15.33:0/715654371 shutdown_connections 2023-11-06T20:44:47.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.627+0000 7fd5055ff700 1 -- 172.21.15.33:0/715654371 wait complete. 2023-11-06T20:44:47.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.629+0000 7fd5055ff700 1 Processor -- start 2023-11-06T20:44:47.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.629+0000 7fd5055ff700 1 -- start start 2023-11-06T20:44:47.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd5055ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd500151dc0 0x7fd5000c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd5055ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5001521a0 0x7fd500150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd5055ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd500155320 0x7fd5000c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd5055ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000aecd0 con 0x7fd5001521a0 2023-11-06T20:44:47.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd5055ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000c26f0 con 0x7fd500155320 2023-11-06T20:44:47.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd5055ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000c29d0 con 0x7fd500151dc0 2023-11-06T20:44:47.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd4fe7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5001521a0 0x7fd500150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd4fe7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5001521a0 0x7fd500150a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37262/0 (socket says 172.21.15.33:37262) 2023-11-06T20:44:47.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd4fe7fc700 1 -- 172.21.15.33:0/2294646837 learned_addr learned my addr 172.21.15.33:0/2294646837 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:47.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd4ff7fe700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd500155320 0x7fd5000c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd4feffd700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd500151dc0 0x7fd5000c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd4fe7fc700 1 -- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd500151dc0 msgr2=0x7fd5000c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.630+0000 7fd4fe7fc700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd500151dc0 0x7fd5000c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.631+0000 7fd4fe7fc700 1 -- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd500155320 msgr2=0x7fd5000c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.631+0000 7fd4fe7fc700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd500155320 0x7fd5000c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.631+0000 7fd4fe7fc700 1 -- 172.21.15.33:0/2294646837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0009370 con 0x7fd5001521a0 2023-11-06T20:44:47.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.631+0000 7fd4fe7fc700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5001521a0 0x7fd500150a90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fd4f400c4c0 tx=0x7fd4f4007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:47.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.631+0000 7fd4e6ffd700 1 -- 172.21.15.33:0/2294646837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f400ce30 con 0x7fd5001521a0 2023-11-06T20:44:47.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.631+0000 7fd4e6ffd700 1 -- 172.21.15.33:0/2294646837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f40072e0 con 0x7fd5001521a0 2023-11-06T20:44:47.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.631+0000 7fd4e6ffd700 1 -- 172.21.15.33:0/2294646837 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40056e0 con 0x7fd5001521a0 2023-11-06T20:44:47.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.631+0000 7fd5055ff700 1 -- 172.21.15.33:0/2294646837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5000c2cb0 con 0x7fd5001521a0 2023-11-06T20:44:47.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.631+0000 7fd5055ff700 1 -- 172.21.15.33:0/2294646837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5000c23b0 con 0x7fd5001521a0 2023-11-06T20:44:47.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.632+0000 7fd5055ff700 1 -- 172.21.15.33:0/2294646837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5000aee80 con 0x7fd5001521a0 2023-11-06T20:44:47.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.633+0000 7fd4e6ffd700 1 -- 172.21.15.33:0/2294646837 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd4f4005a90 con 0x7fd5001521a0 2023-11-06T20:44:47.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.634+0000 7fd4e6ffd700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4ec05f150 0x7fd4ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:47.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.634+0000 7fd4e6ffd700 1 -- 172.21.15.33:0/2294646837 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd4f4079910 con 0x7fd5001521a0 2023-11-06T20:44:47.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.634+0000 7fd4feffd700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4ec05f150 0x7fd4ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:47.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.635+0000 7fd4feffd700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4ec05f150 0x7fd4ec061610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd4f0009130 tx=0x7fd4f0017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:47.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.638+0000 7fd4e6ffd700 1 -- 172.21.15.33:0/2294646837 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd5000aee80 con 0x7fd5001521a0 2023-11-06T20:44:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.819+0000 7fd5055ff700 1 -- 172.21.15.33:0/2294646837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5000aeaa0 con 0x7fd5001521a0 2023-11-06T20:44:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.820+0000 7fd4e6ffd700 1 -- 172.21.15.33:0/2294646837 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fd4f404ab20 con 0x7fd5001521a0 2023-11-06T20:44:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:47.820 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:47.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 -- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4ec05f150 msgr2=0x7fd4ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4ec05f150 0x7fd4ec061610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd4f0009130 tx=0x7fd4f0017000).stop 2023-11-06T20:44:47.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 -- 172.21.15.33:0/2294646837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5001521a0 msgr2=0x7fd500150a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:47.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5001521a0 0x7fd500150a90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fd4f400c4c0 tx=0x7fd4f4007c10).stop 2023-11-06T20:44:47.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 -- 172.21.15.33:0/2294646837 shutdown_connections 2023-11-06T20:44:47.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd4ec05f150 0x7fd4ec061610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd500151dc0 0x7fd5000c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd500155320 0x7fd5000c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 --2- 172.21.15.33:0/2294646837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5001521a0 0x7fd500150a90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:47.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 -- 172.21.15.33:0/2294646837 >> 172.21.15.33:0/2294646837 conn(0x7fd50005e1b0 msgr2=0x7fd50005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:47.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 -- 172.21.15.33:0/2294646837 shutdown_connections 2023-11-06T20:44:47.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:47.822+0000 7fd5055ff700 1 -- 172.21.15.33:0/2294646837 wait complete. 2023-11-06T20:44:47.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:44:04.716205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:44:05.351480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T20:44:47.841 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:44:48.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.039+0000 7f76691ff700 1 Processor -- start 2023-11-06T20:44:48.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.039+0000 7f76691ff700 1 -- start start 2023-11-06T20:44:48.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.041+0000 7f76691ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7664154710 0x7f76640c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.041+0000 7f76691ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76640c7ff0 0x7f76640646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.041+0000 7f76691ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76640c6b60 con 0x7f7664154330 2023-11-06T20:44:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.041+0000 7f76691ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76640c7700 con 0x7f7664154710 2023-11-06T20:44:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.041+0000 7f76691ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76640c5cd0 con 0x7f76640c7ff0 2023-11-06T20:44:48.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.041+0000 7f766259c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7664154710 0x7f76640c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.041+0000 7f766359e700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76640c7ff0 0x7f76640646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.041+0000 7f766259c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7664154710 0x7f76640c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52028/0 (socket says 172.21.15.33:52028) 2023-11-06T20:44:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.041+0000 7f766259c700 1 -- 172.21.15.33:0/1624361032 learned_addr learned my addr 172.21.15.33:0/1624361032 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.042+0000 7f766359e700 1 -- 172.21.15.33:0/1624361032 >> v2:172.21.15.69:3300/0 conn(0x7f7664154710 msgr2=0x7f76640c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.042+0000 7f766359e700 1 --2- 172.21.15.33:0/1624361032 >> v2:172.21.15.69:3300/0 conn(0x7f7664154710 0x7f76640c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.042+0000 7f766359e700 1 -- 172.21.15.33:0/1624361032 >> v1:172.21.15.69:6789/0 conn(0x7f7664154330 legacy=0x7f76640c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.042+0000 7f766359e700 1 -- 172.21.15.33:0/1624361032 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76640c5f50 con 0x7f76640c7ff0 2023-11-06T20:44:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.042+0000 7f766259c700 1 --2- 172.21.15.33:0/1624361032 >> v2:172.21.15.69:3300/0 conn(0x7f7664154710 0x7f76640c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.042+0000 7f766359e700 1 --2- 172.21.15.33:0/1624361032 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76640c7ff0 0x7f76640646c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f765c009e60 tx=0x7f765c009780).ready entity=mon.2 client_cookie=1e2184c7dca31a5f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.042+0000 7f7661d9b700 1 -- 172.21.15.33:0/1624361032 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c00e070 con 0x7f76640c7ff0 2023-11-06T20:44:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.042+0000 7f7661d9b700 1 -- 172.21.15.33:0/1624361032 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c00bb60 con 0x7f76640c7ff0 2023-11-06T20:44:48.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.042+0000 7f7661d9b700 1 -- 172.21.15.33:0/1624361032 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c012410 con 0x7f76640c7ff0 2023-11-06T20:44:48.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.043+0000 7f76691ff700 1 -- 172.21.15.33:0/1624361032 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76640c7ff0 msgr2=0x7f76640646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.043+0000 7f76691ff700 1 --2- 172.21.15.33:0/1624361032 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76640c7ff0 0x7f76640646c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f765c009e60 tx=0x7f765c009780).stop 2023-11-06T20:44:48.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.043+0000 7f76691ff700 1 -- 172.21.15.33:0/1624361032 shutdown_connections 2023-11-06T20:44:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.043+0000 7f76691ff700 1 --2- 172.21.15.33:0/1624361032 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76640c7ff0 0x7f76640646c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.043+0000 7f76691ff700 1 --2- 172.21.15.33:0/1624361032 >> v2:172.21.15.69:3300/0 conn(0x7f7664154710 0x7f76640c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.043+0000 7f76691ff700 1 -- 172.21.15.33:0/1624361032 >> 172.21.15.33:0/1624361032 conn(0x7f76640cb910 msgr2=0x7f76640c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.043+0000 7f76691ff700 1 -- 172.21.15.33:0/1624361032 shutdown_connections 2023-11-06T20:44:48.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.043+0000 7f76691ff700 1 -- 172.21.15.33:0/1624361032 wait complete. 2023-11-06T20:44:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.044+0000 7f76691ff700 1 Processor -- start 2023-11-06T20:44:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.044+0000 7f76691ff700 1 -- start start 2023-11-06T20:44:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.045+0000 7f76691ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76640c7ff0 0x7f7664149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.045+0000 7f76691ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7664154330 0x7f7664151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.045+0000 7f76691ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7664154710 0x7f76640c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.045+0000 7f76691ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640b0e50 con 0x7f76640c7ff0 2023-11-06T20:44:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.045+0000 7f76691ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640aec90 con 0x7f7664154710 2023-11-06T20:44:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.045+0000 7f76691ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76641518c0 con 0x7f7664154330 2023-11-06T20:44:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f7662d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76640c7ff0 0x7f7664149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f7662d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76640c7ff0 0x7f7664149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40938/0 (socket says 172.21.15.33:40938) 2023-11-06T20:44:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f7662d9d700 1 -- 172.21.15.33:0/2983635304 learned_addr learned my addr 172.21.15.33:0/2983635304 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f766359e700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7664154710 0x7f76640c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f766259c700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7664154330 0x7f7664151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f7662d9d700 1 -- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7664154330 msgr2=0x7f7664151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f7662d9d700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7664154330 0x7f7664151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f7662d9d700 1 -- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7664154710 msgr2=0x7f76640c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f7662d9d700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7664154710 0x7f76640c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f7662d9d700 1 -- 172.21.15.33:0/2983635304 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658009370 con 0x7f76640c7ff0 2023-11-06T20:44:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.046+0000 7f7662d9d700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76640c7ff0 0x7f7664149b10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f765400c4f0 tx=0x7f7654007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.047+0000 7f764affd700 1 -- 172.21.15.33:0/2983635304 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765400ce60 con 0x7f76640c7ff0 2023-11-06T20:44:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.047+0000 7f764affd700 1 -- 172.21.15.33:0/2983635304 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7654001e90 con 0x7f76640c7ff0 2023-11-06T20:44:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.047+0000 7f76691ff700 1 -- 172.21.15.33:0/2983635304 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c009a20 con 0x7f76640c7ff0 2023-11-06T20:44:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.047+0000 7f764affd700 1 -- 172.21.15.33:0/2983635304 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7654007250 con 0x7f76640c7ff0 2023-11-06T20:44:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.047+0000 7f76691ff700 1 -- 172.21.15.33:0/2983635304 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7664151550 con 0x7f76640c7ff0 2023-11-06T20:44:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.048+0000 7f76691ff700 1 -- 172.21.15.33:0/2983635304 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76641510b0 con 0x7f76640c7ff0 2023-11-06T20:44:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.049+0000 7f764affd700 1 -- 172.21.15.33:0/2983635304 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f76540066a0 con 0x7f76640c7ff0 2023-11-06T20:44:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.050+0000 7f764affd700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f765005f150 0x7f7650061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.050+0000 7f764affd700 1 -- 172.21.15.33:0/2983635304 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7654079a90 con 0x7f76640c7ff0 2023-11-06T20:44:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.050+0000 7f766259c700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f765005f150 0x7f7650061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.050+0000 7f766259c700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f765005f150 0x7f7650061610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f76641597a0 tx=0x7f76580090f0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:48.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.053+0000 7f764affd700 1 -- 172.21.15.33:0/2983635304 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f765404aca0 con 0x7f76640c7ff0 2023-11-06T20:44:48.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.234+0000 7f76691ff700 1 -- 172.21.15.33:0/2983635304 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7664151ba0 con 0x7f76640c7ff0 2023-11-06T20:44:48.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.235+0000 7f764affd700 1 -- 172.21.15.33:0/2983635304 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7654079020 con 0x7f76640c7ff0 2023-11-06T20:44:48.235 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:44:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 -- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f765005f150 msgr2=0x7f7650061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f765005f150 0x7f7650061610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f76641597a0 tx=0x7f76580090f0).stop 2023-11-06T20:44:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 -- 172.21.15.33:0/2983635304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76640c7ff0 msgr2=0x7f7664149b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76640c7ff0 0x7f7664149b10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f765400c4f0 tx=0x7f7654007e00).stop 2023-11-06T20:44:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 -- 172.21.15.33:0/2983635304 shutdown_connections 2023-11-06T20:44:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f765005f150 0x7f7650061610 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7664154710 0x7f76640c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76640c7ff0 0x7f7664149b10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 --2- 172.21.15.33:0/2983635304 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7664154330 0x7f7664151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 -- 172.21.15.33:0/2983635304 >> 172.21.15.33:0/2983635304 conn(0x7f76640cb910 msgr2=0x7f76640c88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 -- 172.21.15.33:0/2983635304 shutdown_connections 2023-11-06T20:44:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.237+0000 7f76691ff700 1 -- 172.21.15.33:0/2983635304 wait complete. 2023-11-06T20:44:48.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:44:48.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.453+0000 7f67e8bff700 1 Processor -- start 2023-11-06T20:44:48.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.453+0000 7f67e8bff700 1 -- start start 2023-11-06T20:44:48.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e8bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f67e4154330 0x7f67e40c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e8bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e40c7ff0 0x7f67e40646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e8bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e40c6b60 con 0x7f67e4154710 2023-11-06T20:44:48.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e8bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e40c7700 con 0x7f67e4154330 2023-11-06T20:44:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e8bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40c5cd0 con 0x7f67e40c7ff0 2023-11-06T20:44:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f67e4154330 0x7f67e40c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f67e4154330 0x7f67e40c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40950/0 (socket says 172.21.15.33:40950) 2023-11-06T20:44:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e259c700 1 -- 172.21.15.33:0/1747214230 learned_addr learned my addr 172.21.15.33:0/1747214230 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:48.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e2d9d700 1 --2- 172.21.15.33:0/1747214230 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e40c7ff0 0x7f67e40646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e259c700 1 -- 172.21.15.33:0/1747214230 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e40c7ff0 msgr2=0x7f67e40646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:48.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e259c700 1 --2- 172.21.15.33:0/1747214230 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e40c7ff0 0x7f67e40646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.455+0000 7f67e259c700 1 -- 172.21.15.33:0/1747214230 >> v1:172.21.15.69:6789/0 conn(0x7f67e4154710 legacy=0x7f67e40c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:48.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.456+0000 7f67e259c700 1 -- 172.21.15.33:0/1747214230 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e40c5f50 con 0x7f67e4154330 2023-11-06T20:44:48.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.456+0000 7f67e2d9d700 1 --2- 172.21.15.33:0/1747214230 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e40c7ff0 0x7f67e40646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:48.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.456+0000 7f67e259c700 1 --2- 172.21.15.33:0/1747214230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4154330 0x7f67e40c57e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f67d4006fd0 tx=0x7f67d40090f0).ready entity=mon.0 client_cookie=51ec14857cbf99b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:48.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.456+0000 7f67e159a700 1 -- 172.21.15.33:0/1747214230 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400b600 con 0x7f67e4154330 2023-11-06T20:44:48.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.456+0000 7f67e159a700 1 -- 172.21.15.33:0/1747214230 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d400b760 con 0x7f67e4154330 2023-11-06T20:44:48.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.456+0000 7f67e159a700 1 -- 172.21.15.33:0/1747214230 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400bae0 con 0x7f67e4154330 2023-11-06T20:44:48.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.456+0000 7f67e8bff700 1 -- 172.21.15.33:0/1747214230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4154330 msgr2=0x7f67e40c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.456+0000 7f67e8bff700 1 --2- 172.21.15.33:0/1747214230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4154330 0x7f67e40c57e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f67d4006fd0 tx=0x7f67d40090f0).stop 2023-11-06T20:44:48.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.456+0000 7f67e8bff700 1 -- 172.21.15.33:0/1747214230 shutdown_connections 2023-11-06T20:44:48.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.457+0000 7f67e8bff700 1 --2- 172.21.15.33:0/1747214230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4154330 0x7f67e40c57e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.457+0000 7f67e8bff700 1 --2- 172.21.15.33:0/1747214230 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e40c7ff0 0x7f67e40646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.457+0000 7f67e8bff700 1 -- 172.21.15.33:0/1747214230 >> 172.21.15.33:0/1747214230 conn(0x7f67e40cb910 msgr2=0x7f67e40c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:48.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.457+0000 7f67e8bff700 1 -- 172.21.15.33:0/1747214230 shutdown_connections 2023-11-06T20:44:48.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.457+0000 7f67e8bff700 1 -- 172.21.15.33:0/1747214230 wait complete. 2023-11-06T20:44:48.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.459+0000 7f67e8bff700 1 Processor -- start 2023-11-06T20:44:48.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.459+0000 7f67e8bff700 1 -- start start 2023-11-06T20:44:48.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.460+0000 7f67e8bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40c7ff0 0x7f67e4168910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.460+0000 7f67e8bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e4154330 0x7f67e40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.460+0000 7f67e8bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e4154710 0x7f67e4151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.460+0000 7f67e8bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40c46a0 con 0x7f67e40c7ff0 2023-11-06T20:44:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.460+0000 7f67e8bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4168e00 con 0x7f67e4154330 2023-11-06T20:44:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.460+0000 7f67e8bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40aa500 con 0x7f67e4154710 2023-11-06T20:44:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.460+0000 7f67e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40c7ff0 0x7f67e4168910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.460+0000 7f67e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40c7ff0 0x7f67e4168910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40962/0 (socket says 172.21.15.33:40962) 2023-11-06T20:44:48.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.460+0000 7f67e259c700 1 -- 172.21.15.33:0/1485384912 learned_addr learned my addr 172.21.15.33:0/1485384912 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:48.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67e2d9d700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e4154710 0x7f67e4151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67e1d9b700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e4154330 0x7f67e40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67e259c700 1 -- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e4154710 msgr2=0x7f67e4151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67e259c700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e4154710 0x7f67e4151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67e259c700 1 -- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e4154330 msgr2=0x7f67e40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67e259c700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e4154330 0x7f67e40ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67e259c700 1 -- 172.21.15.33:0/1485384912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4009370 con 0x7f67e40c7ff0 2023-11-06T20:44:48.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67e259c700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40c7ff0 0x7f67e4168910 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f67e406c170 tx=0x7f67d4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:48.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67d27fc700 1 -- 172.21.15.33:0/1485384912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400bae0 con 0x7f67e40c7ff0 2023-11-06T20:44:48.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67d27fc700 1 -- 172.21.15.33:0/1485384912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d4000c90 con 0x7f67e40c7ff0 2023-11-06T20:44:48.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67e8bff700 1 -- 172.21.15.33:0/1485384912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e40c48b0 con 0x7f67e40c7ff0 2023-11-06T20:44:48.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.461+0000 7f67d27fc700 1 -- 172.21.15.33:0/1485384912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d401c670 con 0x7f67e40c7ff0 2023-11-06T20:44:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.462+0000 7f67e8bff700 1 -- 172.21.15.33:0/1485384912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e40b0840 con 0x7f67e40c7ff0 2023-11-06T20:44:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.462+0000 7f67e8bff700 1 -- 172.21.15.33:0/1485384912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e40830b0 con 0x7f67e40c7ff0 2023-11-06T20:44:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.463+0000 7f67d27fc700 1 -- 172.21.15.33:0/1485384912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f67d401c9e0 con 0x7f67e40c7ff0 2023-11-06T20:44:48.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.464+0000 7f67d27fc700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67c405f150 0x7f67c4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.465+0000 7f67d27fc700 1 -- 172.21.15.33:0/1485384912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f67d407c1a0 con 0x7f67e40c7ff0 2023-11-06T20:44:48.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.465+0000 7f67e1d9b700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67c405f150 0x7f67c4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.466+0000 7f67e1d9b700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67c405f150 0x7f67c4061610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f67e4154b70 tx=0x7f67d8008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:48.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.469+0000 7f67d27fc700 1 -- 172.21.15.33:0/1485384912 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f67d404d3b0 con 0x7f67e40c7ff0 2023-11-06T20:44:48.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.646+0000 7f67e8bff700 1 -- 172.21.15.33:0/1485384912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f67e4062b00 con 0x7f67e40c7ff0 2023-11-06T20:44:48.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.646+0000 7f67d27fc700 1 -- 172.21.15.33:0/1485384912 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f67d402a070 con 0x7f67e40c7ff0 2023-11-06T20:44:48.647 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:44:48.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 -- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67c405f150 msgr2=0x7f67c4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67c405f150 0x7f67c4061610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f67e4154b70 tx=0x7f67d8008040).stop 2023-11-06T20:44:48.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 -- 172.21.15.33:0/1485384912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40c7ff0 msgr2=0x7f67e4168910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40c7ff0 0x7f67e4168910 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f67e406c170 tx=0x7f67d4017000).stop 2023-11-06T20:44:48.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 -- 172.21.15.33:0/1485384912 shutdown_connections 2023-11-06T20:44:48.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67c405f150 0x7f67c4061610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f67e4154710 0x7f67e4151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e40c7ff0 0x7f67e4168910 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 --2- 172.21.15.33:0/1485384912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e4154330 0x7f67e40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 -- 172.21.15.33:0/1485384912 >> 172.21.15.33:0/1485384912 conn(0x7f67e40cb910 msgr2=0x7f67e40abb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 -- 172.21.15.33:0/1485384912 shutdown_connections 2023-11-06T20:44:48.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.648+0000 7f67e8bff700 1 -- 172.21.15.33:0/1485384912 wait complete. 2023-11-06T20:44:48.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T20:44:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.866+0000 7f0ea6dff700 1 Processor -- start 2023-11-06T20:44:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.866+0000 7f0ea6dff700 1 -- start start 2023-11-06T20:44:48.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.867+0000 7f0ea6dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea0154330 0x7f0ea00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.868+0000 7f0ea6dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0ea0154710 0x7f0ea00c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.868+0000 7f0ea6dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea00c6b60 con 0x7f0ea00c7ff0 2023-11-06T20:44:48.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.868+0000 7f0ea6dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea00c5cd0 con 0x7f0ea0154710 2023-11-06T20:44:48.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.868+0000 7f0ea6dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00c5f50 con 0x7f0ea0154330 2023-11-06T20:44:48.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.868+0000 7f0ea4b9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea0154330 0x7f0ea00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.868+0000 7f0e9ffff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0ea0154710 0x7f0ea00c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.868+0000 7f0ea4b9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea0154330 0x7f0ea00c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:48608/0 (socket says 172.21.15.33:48608) 2023-11-06T20:44:48.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.868+0000 7f0ea4b9b700 1 -- 172.21.15.33:0/2913945702 learned_addr learned my addr 172.21.15.33:0/2913945702 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0e9ffff700 1 -- 172.21.15.33:0/2913945702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea0154330 msgr2=0x7f0ea00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0e9ffff700 1 --2- 172.21.15.33:0/2913945702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea0154330 0x7f0ea00c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0e9ffff700 1 -- 172.21.15.33:0/2913945702 >> v1:172.21.15.69:6789/0 conn(0x7f0ea00c7ff0 legacy=0x7f0ea00646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0e9ffff700 1 -- 172.21.15.33:0/2913945702 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea00c6230 con 0x7f0ea0154710 2023-11-06T20:44:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0e9f7fe700 1 -- 172.21.15.33:0/2913945702 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680197245 0 0) 0x7f0ea00c6b60 con 0x7f0ea00c7ff0 2023-11-06T20:44:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0ea4b9b700 1 --2- 172.21.15.33:0/2913945702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea0154330 0x7f0ea00c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0e9ffff700 1 --2- 172.21.15.33:0/2913945702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea0154710 0x7f0ea00c6ff0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f0e8c009760 tx=0x7f0e8c009060).ready entity=mon.1 client_cookie=f59de3584afdcf29 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0e9f7fe700 1 -- 172.21.15.33:0/2913945702 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c00f510 con 0x7f0ea0154710 2023-11-06T20:44:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0e9f7fe700 1 -- 172.21.15.33:0/2913945702 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c00f670 con 0x7f0ea0154710 2023-11-06T20:44:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0e9f7fe700 1 -- 172.21.15.33:0/2913945702 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c00f9f0 con 0x7f0ea0154710 2023-11-06T20:44:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0ea6dff700 1 -- 172.21.15.33:0/2913945702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea0154710 msgr2=0x7f0ea00c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.869+0000 7f0ea6dff700 1 --2- 172.21.15.33:0/2913945702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea0154710 0x7f0ea00c6ff0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f0e8c009760 tx=0x7f0e8c009060).stop 2023-11-06T20:44:48.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.870+0000 7f0ea6dff700 1 -- 172.21.15.33:0/2913945702 shutdown_connections 2023-11-06T20:44:48.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.870+0000 7f0ea6dff700 1 --2- 172.21.15.33:0/2913945702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea0154330 0x7f0ea00c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.870+0000 7f0ea6dff700 1 --2- 172.21.15.33:0/2913945702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea0154710 0x7f0ea00c6ff0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.870+0000 7f0ea6dff700 1 -- 172.21.15.33:0/2913945702 >> 172.21.15.33:0/2913945702 conn(0x7f0ea00cb910 msgr2=0x7f0ea00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:48.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.870+0000 7f0ea6dff700 1 -- 172.21.15.33:0/2913945702 shutdown_connections 2023-11-06T20:44:48.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.870+0000 7f0ea6dff700 1 -- 172.21.15.33:0/2913945702 wait complete. 2023-11-06T20:44:48.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.872+0000 7f0ea6dff700 1 Processor -- start 2023-11-06T20:44:48.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.872+0000 7f0ea6dff700 1 -- start start 2023-11-06T20:44:48.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.873+0000 7f0ea6dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea00c7ff0 0x7f0ea0159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.873+0000 7f0ea6dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ea0154330 0x7f0ea01674a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.873+0000 7f0ea6dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea0154710 0x7f0ea01524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.873+0000 7f0ea6dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea0167010 con 0x7f0ea0154330 2023-11-06T20:44:48.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.873+0000 7f0ea6dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea01579d0 con 0x7f0ea0154710 2023-11-06T20:44:48.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.873+0000 7f0ea6dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea0157cb0 con 0x7f0ea00c7ff0 2023-11-06T20:44:48.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ea0154330 0x7f0ea01674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ea0154330 0x7f0ea01674a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40966/0 (socket says 172.21.15.33:40966) 2023-11-06T20:44:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9ffff700 1 -- 172.21.15.33:0/1920351694 learned_addr learned my addr 172.21.15.33:0/1920351694 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0ea4b9b700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea00c7ff0 0x7f0ea0159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0ea539c700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea0154710 0x7f0ea01524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9ffff700 1 -- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea00c7ff0 msgr2=0x7f0ea0159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9ffff700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea00c7ff0 0x7f0ea0159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9ffff700 1 -- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea0154710 msgr2=0x7f0ea01524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9ffff700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea0154710 0x7f0ea01524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9ffff700 1 -- 172.21.15.33:0/1920351694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e94009370 con 0x7f0ea0154330 2023-11-06T20:44:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0ea539c700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea0154710 0x7f0ea01524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9ffff700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ea0154330 0x7f0ea01674a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f0ea006c170 tx=0x7f0e8c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9cff9700 1 -- 172.21.15.33:0/1920351694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c00f7e0 con 0x7f0ea0154330 2023-11-06T20:44:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0e9cff9700 1 -- 172.21.15.33:0/1920351694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c00f940 con 0x7f0ea0154330 2023-11-06T20:44:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.874+0000 7f0ea6dff700 1 -- 172.21.15.33:0/1920351694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c0092a0 con 0x7f0ea0154330 2023-11-06T20:44:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.875+0000 7f0e9cff9700 1 -- 172.21.15.33:0/1920351694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c01c670 con 0x7f0ea0154330 2023-11-06T20:44:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.875+0000 7f0ea6dff700 1 -- 172.21.15.33:0/1920351694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea01529c0 con 0x7f0ea0154330 2023-11-06T20:44:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.876+0000 7f0ea6dff700 1 -- 172.21.15.33:0/1920351694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea00830b0 con 0x7f0ea0154330 2023-11-06T20:44:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.877+0000 7f0e9cff9700 1 -- 172.21.15.33:0/1920351694 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0e8c01ca60 con 0x7f0ea0154330 2023-11-06T20:44:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.878+0000 7f0e9cff9700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e8405f150 0x7f0e84061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.878+0000 7f0e9cff9700 1 -- 172.21.15.33:0/1920351694 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0e8c07be90 con 0x7f0ea0154330 2023-11-06T20:44:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.878+0000 7f0ea4b9b700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e8405f150 0x7f0e84061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.879+0000 7f0ea4b9b700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e8405f150 0x7f0e84061610 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f0e94006fd0 tx=0x7f0e940090f0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:48.881+0000 7f0e9cff9700 1 -- 172.21.15.33:0/1920351694 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0e8c04d0a0 con 0x7f0ea0154330 2023-11-06T20:44:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.060+0000 7f0ea6dff700 1 -- 172.21.15.33:0/1920351694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ea0062b00 con 0x7f0ea0154330 2023-11-06T20:44:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.061+0000 7f0e9cff9700 1 -- 172.21.15.33:0/1920351694 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e8c07d020 con 0x7f0ea0154330 2023-11-06T20:44:49.061 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:49.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 -- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e8405f150 msgr2=0x7f0e84061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e8405f150 0x7f0e84061610 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f0e94006fd0 tx=0x7f0e940090f0).stop 2023-11-06T20:44:49.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 -- 172.21.15.33:0/1920351694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ea0154330 msgr2=0x7f0ea01674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ea0154330 0x7f0ea01674a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f0ea006c170 tx=0x7f0e8c017000).stop 2023-11-06T20:44:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 -- 172.21.15.33:0/1920351694 shutdown_connections 2023-11-06T20:44:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e8405f150 0x7f0e84061610 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ea00c7ff0 0x7f0ea0159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea0154710 0x7f0ea01524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 --2- 172.21.15.33:0/1920351694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ea0154330 0x7f0ea01674a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 -- 172.21.15.33:0/1920351694 >> 172.21.15.33:0/1920351694 conn(0x7f0ea00cb910 msgr2=0x7f0ea00ab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 -- 172.21.15.33:0/1920351694 shutdown_connections 2023-11-06T20:44:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.063+0000 7f0ea6dff700 1 -- 172.21.15.33:0/1920351694 wait complete. 2023-11-06T20:44:49.080 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T20:44:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.278+0000 7f7977bff700 1 Processor -- start 2023-11-06T20:44:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.279+0000 7f7977bff700 1 -- start start 2023-11-06T20:44:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.280+0000 7f7977bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7970154710 0x7f79700c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.280+0000 7f7977bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f79700c7ff0 0x7f79700646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.280+0000 7f7977bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79700c6b60 con 0x7f7970154330 2023-11-06T20:44:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.280+0000 7f7977bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79700c7700 con 0x7f79700c7ff0 2023-11-06T20:44:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.280+0000 7f7977bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79700c5cd0 con 0x7f7970154710 2023-11-06T20:44:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.280+0000 7f797619c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f79700c7ff0 0x7f79700646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.280+0000 7f797619c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f79700c7ff0 0x7f79700646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52060/0 (socket says 172.21.15.33:52060) 2023-11-06T20:44:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.280+0000 7f797619c700 1 -- 172.21.15.33:0/479301825 learned_addr learned my addr 172.21.15.33:0/479301825 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.280+0000 7f797519a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7970154710 0x7f79700c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.280+0000 7f797599b700 1 --1- 172.21.15.33:0/479301825 >> v1:172.21.15.69:6789/0 conn(0x7f7970154330 0x7f79700c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:34568/0 (socket says 172.21.15.33:34568) 2023-11-06T20:44:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.281+0000 7f797619c700 1 -- 172.21.15.33:0/479301825 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7970154710 msgr2=0x7f79700c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.281+0000 7f797619c700 1 --2- 172.21.15.33:0/479301825 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7970154710 0x7f79700c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.281+0000 7f797619c700 1 -- 172.21.15.33:0/479301825 >> v1:172.21.15.69:6789/0 conn(0x7f7970154330 legacy=0x7f79700c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.281+0000 7f797619c700 1 -- 172.21.15.33:0/479301825 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79700c5f50 con 0x7f79700c7ff0 2023-11-06T20:44:49.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.281+0000 7f797619c700 1 --2- 172.21.15.33:0/479301825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79700c7ff0 0x7f79700646c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f7968009f20 tx=0x7f7968009840).ready entity=mon.1 client_cookie=86f2d8b7c71ae182 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:49.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.281+0000 7f7974999700 1 -- 172.21.15.33:0/479301825 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800e070 con 0x7f79700c7ff0 2023-11-06T20:44:49.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.281+0000 7f7974999700 1 -- 172.21.15.33:0/479301825 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796800b9d0 con 0x7f79700c7ff0 2023-11-06T20:44:49.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.281+0000 7f7974999700 1 -- 172.21.15.33:0/479301825 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800bd50 con 0x7f79700c7ff0 2023-11-06T20:44:49.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.282+0000 7f7977bff700 1 -- 172.21.15.33:0/479301825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79700c7ff0 msgr2=0x7f79700646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.282+0000 7f7977bff700 1 --2- 172.21.15.33:0/479301825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79700c7ff0 0x7f79700646c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f7968009f20 tx=0x7f7968009840).stop 2023-11-06T20:44:49.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.282+0000 7f7977bff700 1 -- 172.21.15.33:0/479301825 shutdown_connections 2023-11-06T20:44:49.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.282+0000 7f7977bff700 1 --2- 172.21.15.33:0/479301825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79700c7ff0 0x7f79700646c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.282+0000 7f7977bff700 1 --2- 172.21.15.33:0/479301825 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7970154710 0x7f79700c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.282+0000 7f7977bff700 1 -- 172.21.15.33:0/479301825 >> 172.21.15.33:0/479301825 conn(0x7f79700cb910 msgr2=0x7f79700c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:49.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.282+0000 7f7977bff700 1 -- 172.21.15.33:0/479301825 shutdown_connections 2023-11-06T20:44:49.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.282+0000 7f7977bff700 1 -- 172.21.15.33:0/479301825 wait complete. 2023-11-06T20:44:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.284+0000 7f7977bff700 1 Processor -- start 2023-11-06T20:44:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.284+0000 7f7977bff700 1 -- start start 2023-11-06T20:44:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f7977bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79700c7ff0 0x7f79700a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f7977bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7970154330 0x7f79700ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f7977bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970154710 0x7f79700aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f7977bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79700af1c0 con 0x7f7970154330 2023-11-06T20:44:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f7977bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970157dd0 con 0x7f7970154710 2023-11-06T20:44:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f7977bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79700affd0 con 0x7f79700c7ff0 2023-11-06T20:44:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f797519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7970154330 0x7f79700ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f797519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7970154330 0x7f79700ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40974/0 (socket says 172.21.15.33:40974) 2023-11-06T20:44:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f797519a700 1 -- 172.21.15.33:0/755354214 learned_addr learned my addr 172.21.15.33:0/755354214 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f797619c700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970154710 0x7f79700aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f797599b700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79700c7ff0 0x7f79700a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f797519a700 1 -- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79700c7ff0 msgr2=0x7f79700a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.285+0000 7f797519a700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79700c7ff0 0x7f79700a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.286+0000 7f797519a700 1 -- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970154710 msgr2=0x7f79700aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.286+0000 7f797519a700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970154710 0x7f79700aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.286+0000 7f797519a700 1 -- 172.21.15.33:0/755354214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968009ae0 con 0x7f7970154330 2023-11-06T20:44:49.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.286+0000 7f797599b700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79700c7ff0 0x7f79700a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:49.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.286+0000 7f797519a700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7970154330 0x7f79700ae7e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f796000e900 tx=0x7f7960009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:49.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.286+0000 7f7965ffb700 1 -- 172.21.15.33:0/755354214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796000c670 con 0x7f7970154330 2023-11-06T20:44:49.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.286+0000 7f7965ffb700 1 -- 172.21.15.33:0/755354214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796000c7d0 con 0x7f7970154330 2023-11-06T20:44:49.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.286+0000 7f7965ffb700 1 -- 172.21.15.33:0/755354214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796000cb50 con 0x7f7970154330 2023-11-06T20:44:49.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.286+0000 7f7977bff700 1 -- 172.21.15.33:0/755354214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79700af3c0 con 0x7f7970154330 2023-11-06T20:44:49.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.286+0000 7f7977bff700 1 -- 172.21.15.33:0/755354214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79700af980 con 0x7f7970154330 2023-11-06T20:44:49.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.287+0000 7f7977bff700 1 -- 172.21.15.33:0/755354214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79700b0110 con 0x7f7970154330 2023-11-06T20:44:49.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.288+0000 7f7965ffb700 1 -- 172.21.15.33:0/755354214 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f796001b400 con 0x7f7970154330 2023-11-06T20:44:49.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.289+0000 7f7965ffb700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f795405f150 0x7f7954061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.289+0000 7f7965ffb700 1 -- 172.21.15.33:0/755354214 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f796007a770 con 0x7f7970154330 2023-11-06T20:44:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.289+0000 7f797599b700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f795405f150 0x7f7954061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.290+0000 7f797599b700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f795405f150 0x7f7954061610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f796c005790 tx=0x7f796c007c90).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.293+0000 7f7965ffb700 1 -- 172.21.15.33:0/755354214 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f796004b920 con 0x7f7970154330 2023-11-06T20:44:49.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.472+0000 7f7977bff700 1 -- 172.21.15.33:0/755354214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79700652d0 con 0x7f7970154330 2023-11-06T20:44:49.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.473+0000 7f7965ffb700 1 -- 172.21.15.33:0/755354214 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7960016070 con 0x7f7970154330 2023-11-06T20:44:49.473 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:44:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 -- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f795405f150 msgr2=0x7f7954061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f795405f150 0x7f7954061610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f796c005790 tx=0x7f796c007c90).stop 2023-11-06T20:44:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 -- 172.21.15.33:0/755354214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7970154330 msgr2=0x7f79700ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7970154330 0x7f79700ae7e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f796000e900 tx=0x7f7960009060).stop 2023-11-06T20:44:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 -- 172.21.15.33:0/755354214 shutdown_connections 2023-11-06T20:44:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f795405f150 0x7f7954061610 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79700c7ff0 0x7f79700a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970154710 0x7f79700aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 --2- 172.21.15.33:0/755354214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7970154330 0x7f79700ae7e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 -- 172.21.15.33:0/755354214 >> 172.21.15.33:0/755354214 conn(0x7f79700cb910 msgr2=0x7f79700c8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 -- 172.21.15.33:0/755354214 shutdown_connections 2023-11-06T20:44:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.475+0000 7f7977bff700 1 -- 172.21.15.33:0/755354214 wait complete. 2023-11-06T20:44:49.492 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:44:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.687+0000 7ff4759ff700 1 Processor -- start 2023-11-06T20:44:49.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.687+0000 7ff4759ff700 1 -- start start 2023-11-06T20:44:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff4759ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4700c1840 0x7ff4700a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff4759ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 0x7ff4700c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff4759ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4701554a0 con 0x7ff4700c6240 2023-11-06T20:44:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff4759ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff470154960 con 0x7ff4700c1840 2023-11-06T20:44:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff4759ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff470155930 con 0x7ff4700aa2d0 2023-11-06T20:44:49.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff46effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4700c1840 0x7ff4700a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff46effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4700c1840 0x7ff4700a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40990/0 (socket says 172.21.15.33:40990) 2023-11-06T20:44:49.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff46effd700 1 -- 172.21.15.33:0/3853698113 learned_addr learned my addr 172.21.15.33:0/3853698113 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:49.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff46e7fc700 1 --2- 172.21.15.33:0/3853698113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 0x7ff4700c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff46effd700 1 -- 172.21.15.33:0/3853698113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 msgr2=0x7ff4700c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:49.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff46effd700 1 --2- 172.21.15.33:0/3853698113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 0x7ff4700c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff46effd700 1 -- 172.21.15.33:0/3853698113 >> v1:172.21.15.69:6789/0 conn(0x7ff4700c6240 legacy=0x7ff470157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.689+0000 7ff46effd700 1 -- 172.21.15.33:0/3853698113 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4700c66b0 con 0x7ff4700c1840 2023-11-06T20:44:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.690+0000 7ff46effd700 1 --2- 172.21.15.33:0/3853698113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4700c1840 0x7ff4700a9de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7ff460006fd0 tx=0x7ff4600090f0).ready entity=mon.0 client_cookie=69f301a1fa1613c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.690+0000 7ff46dffb700 1 -- 172.21.15.33:0/3853698113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46000b600 con 0x7ff4700c1840 2023-11-06T20:44:49.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.690+0000 7ff46dffb700 1 -- 172.21.15.33:0/3853698113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46000b760 con 0x7ff4700c1840 2023-11-06T20:44:49.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.690+0000 7ff46dffb700 1 -- 172.21.15.33:0/3853698113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46000bae0 con 0x7ff4700c1840 2023-11-06T20:44:49.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.690+0000 7ff4759ff700 1 -- 172.21.15.33:0/3853698113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4700c1840 msgr2=0x7ff4700a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.690+0000 7ff4759ff700 1 --2- 172.21.15.33:0/3853698113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4700c1840 0x7ff4700a9de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7ff460006fd0 tx=0x7ff4600090f0).stop 2023-11-06T20:44:49.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.690+0000 7ff4759ff700 1 -- 172.21.15.33:0/3853698113 shutdown_connections 2023-11-06T20:44:49.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.690+0000 7ff4759ff700 1 --2- 172.21.15.33:0/3853698113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4700c1840 0x7ff4700a9de0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.690+0000 7ff4759ff700 1 --2- 172.21.15.33:0/3853698113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 0x7ff4700c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.690+0000 7ff4759ff700 1 -- 172.21.15.33:0/3853698113 >> 172.21.15.33:0/3853698113 conn(0x7ff47005ec80 msgr2=0x7ff4700cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:49.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.691+0000 7ff4759ff700 1 -- 172.21.15.33:0/3853698113 shutdown_connections 2023-11-06T20:44:49.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.691+0000 7ff4759ff700 1 -- 172.21.15.33:0/3853698113 wait complete. 2023-11-06T20:44:49.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.692+0000 7ff4759ff700 1 Processor -- start 2023-11-06T20:44:49.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.692+0000 7ff4759ff700 1 -- start start 2023-11-06T20:44:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.693+0000 7ff4759ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 0x7ff470165f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.693+0000 7ff4759ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4700c1840 0x7ff4701684e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.693+0000 7ff4759ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4700c6240 0x7ff470167750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.693+0000 7ff4759ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700c51a0 con 0x7ff4700c6240 2023-11-06T20:44:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.693+0000 7ff4759ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700ab5c0 con 0x7ff4700c1840 2023-11-06T20:44:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.693+0000 7ff4759ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700c4ec0 con 0x7ff4700aa2d0 2023-11-06T20:44:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46effd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 0x7ff470165f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4700c6240 0x7ff470167750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46e7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4700c1840 0x7ff4701684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46effd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 0x7ff470165f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:48668/0 (socket says 172.21.15.33:48668) 2023-11-06T20:44:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46effd700 1 -- 172.21.15.33:0/193902058 learned_addr learned my addr 172.21.15.33:0/193902058 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46f7fe700 1 -- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 msgr2=0x7ff470165f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46f7fe700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 0x7ff470165f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46f7fe700 1 -- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4700c1840 msgr2=0x7ff4701684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46f7fe700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4700c1840 0x7ff4701684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46f7fe700 1 -- 172.21.15.33:0/193902058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460009370 con 0x7ff4700c6240 2023-11-06T20:44:49.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff46f7fe700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4700c6240 0x7ff470167750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7ff4680013d0 tx=0x7ff468001530).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:49.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff456ffd700 1 -- 172.21.15.33:0/193902058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800e4a0 con 0x7ff4700c6240 2023-11-06T20:44:49.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff456ffd700 1 -- 172.21.15.33:0/193902058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46800e600 con 0x7ff4700c6240 2023-11-06T20:44:49.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.694+0000 7ff456ffd700 1 -- 172.21.15.33:0/193902058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800e980 con 0x7ff4700c6240 2023-11-06T20:44:49.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.695+0000 7ff4759ff700 1 -- 172.21.15.33:0/193902058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4700aaf70 con 0x7ff4700c6240 2023-11-06T20:44:49.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.695+0000 7ff4759ff700 1 -- 172.21.15.33:0/193902058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4701690a0 con 0x7ff4700c6240 2023-11-06T20:44:49.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.696+0000 7ff4759ff700 1 -- 172.21.15.33:0/193902058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4700ab770 con 0x7ff4700c6240 2023-11-06T20:44:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.696+0000 7ff456ffd700 1 -- 172.21.15.33:0/193902058 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff4680123f0 con 0x7ff4700c6240 2023-11-06T20:44:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.697+0000 7ff456ffd700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff45c05f150 0x7ff45c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.697+0000 7ff456ffd700 1 -- 172.21.15.33:0/193902058 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff468079a40 con 0x7ff4700c6240 2023-11-06T20:44:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.698+0000 7ff46effd700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff45c05f150 0x7ff45c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.698+0000 7ff46effd700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff45c05f150 0x7ff45c061610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7ff46000b3c0 tx=0x7ff460017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.701+0000 7ff456ffd700 1 -- 172.21.15.33:0/193902058 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff468047af0 con 0x7ff4700c6240 2023-11-06T20:44:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.927+0000 7ff4759ff700 1 -- 172.21.15.33:0/193902058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff47006c6c0 con 0x7ff4700c6240 2023-11-06T20:44:49.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.927+0000 7ff456ffd700 1 -- 172.21.15.33:0/193902058 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4480 (secure 0 0 0) 0x7ff4700ab770 con 0x7ff4700c6240 2023-11-06T20:44:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:49.928 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4123875619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123875619},{"type":"v1","addr":"172.21.15.33:6834","nonce":4123875619}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3506554570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3506554570},{"type":"v1","addr":"172.21.15.33:6835","nonce":3506554570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/239985290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":239985290},{"type":"v1","addr":"172.21.15.69:6835","nonce":239985290}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:44:06.383743+0000","modified":"2023-11-06T20:44:09.389408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/2681789033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2681789033},{"type":"v1","addr":"172.21.15.69:6837","nonce":2681789033}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-06T20:44:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.929+0000 7ff4759ff700 1 -- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff45c05f150 msgr2=0x7ff45c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.929+0000 7ff4759ff700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff45c05f150 0x7ff45c061610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7ff46000b3c0 tx=0x7ff460017000).stop 2023-11-06T20:44:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.929+0000 7ff4759ff700 1 -- 172.21.15.33:0/193902058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4700c6240 msgr2=0x7ff470167750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.929+0000 7ff4759ff700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4700c6240 0x7ff470167750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7ff4680013d0 tx=0x7ff468001530).stop 2023-11-06T20:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.930+0000 7ff4759ff700 1 -- 172.21.15.33:0/193902058 shutdown_connections 2023-11-06T20:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.930+0000 7ff4759ff700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff45c05f150 0x7ff45c061610 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.930+0000 7ff4759ff700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff4700aa2d0 0x7ff470165f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.930+0000 7ff4759ff700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4700c6240 0x7ff470167750 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.930+0000 7ff4759ff700 1 --2- 172.21.15.33:0/193902058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4700c1840 0x7ff4701684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.930+0000 7ff4759ff700 1 -- 172.21.15.33:0/193902058 >> 172.21.15.33:0/193902058 conn(0x7ff47005ec80 msgr2=0x7ff4700ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.930+0000 7ff4759ff700 1 -- 172.21.15.33:0/193902058 shutdown_connections 2023-11-06T20:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:49.930+0000 7ff4759ff700 1 -- 172.21.15.33:0/193902058 wait complete. 2023-11-06T20:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2023-11-06T20:44:49.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:50.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.143+0000 7fdb2f7ff700 1 Processor -- start 2023-11-06T20:44:50.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.143+0000 7fdb2f7ff700 1 -- start start 2023-11-06T20:44:50.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.145+0000 7fdb2f7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdb28063280 0x7fdb28063660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.145+0000 7fdb2f7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2814e4a0 0x7fdb2814e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.145+0000 7fdb2f7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2814ee10 con 0x7fdb280645d0 2023-11-06T20:44:50.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.145+0000 7fdb2f7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb28158e20 con 0x7fdb28063280 2023-11-06T20:44:50.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.145+0000 7fdb2f7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb281582a0 con 0x7fdb2814e4a0 2023-11-06T20:44:50.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.145+0000 7fdb2dd9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdb280645d0 0x7fdb28064a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36012/0 (socket says 172.21.15.33:36012) 2023-11-06T20:44:50.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.145+0000 7fdb2dd9c700 1 -- 172.21.15.33:0/1548224049 learned_addr learned my addr 172.21.15.33:0/1548224049 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:50.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.145+0000 7fdb2cd9a700 1 --2- 172.21.15.33:0/1548224049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2814e4a0 0x7fdb2814e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.145+0000 7fdb2d59b700 1 --2- 172.21.15.33:0/1548224049 >> v2:172.21.15.69:3300/0 conn(0x7fdb28063280 0x7fdb28063660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb1ffff700 1 -- 172.21.15.33:0/1548224049 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914777319 0 0) 0x7fdb2814ee10 con 0x7fdb280645d0 2023-11-06T20:44:50.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb1ffff700 1 -- 172.21.15.33:0/1548224049 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb140033f0 con 0x7fdb280645d0 2023-11-06T20:44:50.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb2d59b700 1 -- 172.21.15.33:0/1548224049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2814e4a0 msgr2=0x7fdb2814e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:50.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb2d59b700 1 --2- 172.21.15.33:0/1548224049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2814e4a0 0x7fdb2814e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb2d59b700 1 -- 172.21.15.33:0/1548224049 >> v1:172.21.15.33:6789/0 conn(0x7fdb280645d0 legacy=0x7fdb28064a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb2d59b700 1 -- 172.21.15.33:0/1548224049 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28158580 con 0x7fdb28063280 2023-11-06T20:44:50.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb2cd9a700 1 --2- 172.21.15.33:0/1548224049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2814e4a0 0x7fdb2814e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:50.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb2d59b700 1 --2- 172.21.15.33:0/1548224049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb28063280 0x7fdb28063660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fdb24009760 tx=0x7fdb24009060).ready entity=mon.1 client_cookie=a9199c877c835050 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:50.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb1ffff700 1 -- 172.21.15.33:0/1548224049 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400d4a0 con 0x7fdb28063280 2023-11-06T20:44:50.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb1ffff700 1 -- 172.21.15.33:0/1548224049 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2400d600 con 0x7fdb28063280 2023-11-06T20:44:50.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.146+0000 7fdb1ffff700 1 -- 172.21.15.33:0/1548224049 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400d980 con 0x7fdb28063280 2023-11-06T20:44:50.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.147+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1548224049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb28063280 msgr2=0x7fdb28063660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.147+0000 7fdb2f7ff700 1 --2- 172.21.15.33:0/1548224049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb28063280 0x7fdb28063660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fdb24009760 tx=0x7fdb24009060).stop 2023-11-06T20:44:50.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.147+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1548224049 shutdown_connections 2023-11-06T20:44:50.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.147+0000 7fdb2f7ff700 1 --2- 172.21.15.33:0/1548224049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb28063280 0x7fdb28063660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.147+0000 7fdb2f7ff700 1 --2- 172.21.15.33:0/1548224049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2814e4a0 0x7fdb2814e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.147+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1548224049 >> 172.21.15.33:0/1548224049 conn(0x7fdb280cade0 msgr2=0x7fdb280cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:50.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.147+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1548224049 shutdown_connections 2023-11-06T20:44:50.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.147+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1548224049 wait complete. 2023-11-06T20:44:50.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.148+0000 7fdb2f7ff700 1 Processor -- start 2023-11-06T20:44:50.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.149+0000 7fdb2f7ff700 1 -- start start 2023-11-06T20:44:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2f7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb28063280 0x7fdb281649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2f7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb280645d0 0x7fdb2814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2f7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb2814e4a0 0x7fdb2814d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2f7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb280b0d70 con 0x7fdb28063280 2023-11-06T20:44:50.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2f7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb280b07b0 con 0x7fdb2814e4a0 2023-11-06T20:44:50.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2f7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb280b0a90 con 0x7fdb280645d0 2023-11-06T20:44:50.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb28063280 0x7fdb281649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb28063280 0x7fdb281649f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41000/0 (socket says 172.21.15.33:41000) 2023-11-06T20:44:50.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2d59b700 1 -- 172.21.15.33:0/1806274224 learned_addr learned my addr 172.21.15.33:0/1806274224 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:50.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2cd9a700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb280645d0 0x7fdb2814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2d59b700 1 -- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb280645d0 msgr2=0x7fdb2814aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2d59b700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb280645d0 0x7fdb2814aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2dd9c700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb2814e4a0 0x7fdb2814d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2d59b700 1 -- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb2814e4a0 msgr2=0x7fdb2814d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2d59b700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb2814e4a0 0x7fdb2814d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2d59b700 1 -- 172.21.15.33:0/1806274224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb180092a0 con 0x7fdb28063280 2023-11-06T20:44:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2cd9a700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb280645d0 0x7fdb2814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2dd9c700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb2814e4a0 0x7fdb2814d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.150+0000 7fdb2d59b700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb28063280 0x7fdb281649f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fdb24009d50 tx=0x7fdb24016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.151+0000 7fdb1d7fa700 1 -- 172.21.15.33:0/1806274224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24000c90 con 0x7fdb28063280 2023-11-06T20:44:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.151+0000 7fdb1d7fa700 1 -- 172.21.15.33:0/1806274224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2400d770 con 0x7fdb28063280 2023-11-06T20:44:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.151+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1806274224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb240092a0 con 0x7fdb28063280 2023-11-06T20:44:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.151+0000 7fdb1d7fa700 1 -- 172.21.15.33:0/1806274224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2401d640 con 0x7fdb28063280 2023-11-06T20:44:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.151+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1806274224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb280ae0f0 con 0x7fdb28063280 2023-11-06T20:44:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.153+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1806274224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2806c6c0 con 0x7fdb28063280 2023-11-06T20:44:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.153+0000 7fdb1d7fa700 1 -- 172.21.15.33:0/1806274224 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdb2401da70 con 0x7fdb28063280 2023-11-06T20:44:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.154+0000 7fdb1d7fa700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdb0c05f150 0x7fdb0c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.154+0000 7fdb1d7fa700 1 -- 172.21.15.33:0/1806274224 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdb2407ba10 con 0x7fdb28063280 2023-11-06T20:44:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.155+0000 7fdb2cd9a700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdb0c05f150 0x7fdb0c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.155+0000 7fdb2cd9a700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdb0c05f150 0x7fdb0c061610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fdb18009270 tx=0x7fdb18008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.158+0000 7fdb1d7fa700 1 -- 172.21.15.33:0/1806274224 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdb2404cd80 con 0x7fdb28063280 2023-11-06T20:44:50.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.344+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1806274224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb28158b30 con 0x7fdb28063280 2023-11-06T20:44:50.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.345+0000 7fdb1d7fa700 1 -- 172.21.15.33:0/1806274224 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fdb240503a0 con 0x7fdb28063280 2023-11-06T20:44:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:50.345 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:50.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdb0c05f150 msgr2=0x7fdb0c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdb0c05f150 0x7fdb0c061610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fdb18009270 tx=0x7fdb18008040).stop 2023-11-06T20:44:50.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1806274224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb28063280 msgr2=0x7fdb281649f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb28063280 0x7fdb281649f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fdb24009d50 tx=0x7fdb24016000).stop 2023-11-06T20:44:50.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1806274224 shutdown_connections 2023-11-06T20:44:50.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdb0c05f150 0x7fdb0c061610 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb2814e4a0 0x7fdb2814d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb28063280 0x7fdb281649f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 --2- 172.21.15.33:0/1806274224 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb280645d0 0x7fdb2814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1806274224 >> 172.21.15.33:0/1806274224 conn(0x7fdb280cade0 msgr2=0x7fdb280ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1806274224 shutdown_connections 2023-11-06T20:44:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.347+0000 7fdb2f7ff700 1 -- 172.21.15.33:0/1806274224 wait complete. 2023-11-06T20:44:50.365 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T20:44:50.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:44:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.561+0000 7faf6e7ff700 1 Processor -- start 2023-11-06T20:44:50.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.561+0000 7faf6e7ff700 1 -- start start 2023-11-06T20:44:50.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.563+0000 7faf6e7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 0x7faf680a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.563+0000 7faf6e7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf681554a0 con 0x7faf680aa2d0 2023-11-06T20:44:50.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.563+0000 7faf6e7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf68155930 con 0x7faf680c6460 2023-11-06T20:44:50.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.563+0000 7faf6e7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf68154960 con 0x7faf680c1840 2023-11-06T20:44:50.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.563+0000 7faf677fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faf680aa2d0 0x7faf680c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36020/0 (socket says 172.21.15.33:36020) 2023-11-06T20:44:50.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.563+0000 7faf677fe700 1 -- 172.21.15.33:0/4238257668 learned_addr learned my addr 172.21.15.33:0/4238257668 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:50.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.563+0000 7faf67fff700 1 --2- 172.21.15.33:0/4238257668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 0x7faf680a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.564+0000 7faf66ffd700 1 -- 172.21.15.33:0/4238257668 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257217379 0 0) 0x7faf681554a0 con 0x7faf680aa2d0 2023-11-06T20:44:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.564+0000 7faf66ffd700 1 -- 172.21.15.33:0/4238257668 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf500033f0 con 0x7faf680aa2d0 2023-11-06T20:44:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.564+0000 7faf66ffd700 1 -- 172.21.15.33:0/4238257668 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127110286 0 0) 0x7faf68155930 con 0x7faf680c6460 2023-11-06T20:44:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.564+0000 7faf66ffd700 1 -- 172.21.15.33:0/4238257668 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf681554a0 con 0x7faf680c6460 2023-11-06T20:44:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.564+0000 7faf67fff700 1 -- 172.21.15.33:0/4238257668 >> v1:172.21.15.69:6789/0 conn(0x7faf680c6460 legacy=0x7faf680bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.564+0000 7faf67fff700 1 -- 172.21.15.33:0/4238257668 >> v1:172.21.15.33:6789/0 conn(0x7faf680aa2d0 legacy=0x7faf680c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.564+0000 7faf67fff700 1 -- 172.21.15.33:0/4238257668 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf68154b70 con 0x7faf680c1840 2023-11-06T20:44:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.564+0000 7faf67fff700 1 --2- 172.21.15.33:0/4238257668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 0x7faf680a9de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7faf58009790 tx=0x7faf58009060).ready entity=mon.2 client_cookie=4f5287faa971d5df server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.565+0000 7faf66ffd700 1 -- 172.21.15.33:0/4238257668 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5800f510 con 0x7faf680c1840 2023-11-06T20:44:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.565+0000 7faf66ffd700 1 -- 172.21.15.33:0/4238257668 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5800f670 con 0x7faf680c1840 2023-11-06T20:44:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.565+0000 7faf66ffd700 1 -- 172.21.15.33:0/4238257668 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5800f9f0 con 0x7faf680c1840 2023-11-06T20:44:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.565+0000 7faf6e7ff700 1 -- 172.21.15.33:0/4238257668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 msgr2=0x7faf680a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.565+0000 7faf6e7ff700 1 --2- 172.21.15.33:0/4238257668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 0x7faf680a9de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7faf58009790 tx=0x7faf58009060).stop 2023-11-06T20:44:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.565+0000 7faf6e7ff700 1 -- 172.21.15.33:0/4238257668 shutdown_connections 2023-11-06T20:44:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.565+0000 7faf6e7ff700 1 --2- 172.21.15.33:0/4238257668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 0x7faf680a9de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.565+0000 7faf6e7ff700 1 -- 172.21.15.33:0/4238257668 >> 172.21.15.33:0/4238257668 conn(0x7faf6805ec80 msgr2=0x7faf680cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.565+0000 7faf6e7ff700 1 -- 172.21.15.33:0/4238257668 shutdown_connections 2023-11-06T20:44:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.565+0000 7faf6e7ff700 1 -- 172.21.15.33:0/4238257668 wait complete. 2023-11-06T20:44:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.567+0000 7faf6e7ff700 1 Processor -- start 2023-11-06T20:44:50.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.567+0000 7faf6e7ff700 1 -- start start 2023-11-06T20:44:50.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf6e7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680aa2d0 0x7faf680b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf6e7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 0x7faf680c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf6e7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf680c6460 0x7faf680c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf6e7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf68165cf0 con 0x7faf680aa2d0 2023-11-06T20:44:50.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf6e7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf680c9600 con 0x7faf680c6460 2023-11-06T20:44:50.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf6e7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf68156be0 con 0x7faf680c1840 2023-11-06T20:44:50.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf67fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680aa2d0 0x7faf680b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf67fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680aa2d0 0x7faf680b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41014/0 (socket says 172.21.15.33:41014) 2023-11-06T20:44:50.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf67fff700 1 -- 172.21.15.33:0/946246438 learned_addr learned my addr 172.21.15.33:0/946246438 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:50.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf677fe700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 0x7faf680c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf6cd9c700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf680c6460 0x7faf680c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf67fff700 1 -- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 msgr2=0x7faf680c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf67fff700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 0x7faf680c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf67fff700 1 -- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf680c6460 msgr2=0x7faf680c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf67fff700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf680c6460 0x7faf680c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf67fff700 1 -- 172.21.15.33:0/946246438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580092a0 con 0x7faf680aa2d0 2023-11-06T20:44:50.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.568+0000 7faf677fe700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 0x7faf680c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:50.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.569+0000 7faf67fff700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680aa2d0 0x7faf680b01b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7faf6806bc90 tx=0x7faf58017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:50.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.569+0000 7faf4ffff700 1 -- 172.21.15.33:0/946246438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5800f7e0 con 0x7faf680aa2d0 2023-11-06T20:44:50.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.569+0000 7faf4ffff700 1 -- 172.21.15.33:0/946246438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5800f940 con 0x7faf680aa2d0 2023-11-06T20:44:50.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.569+0000 7faf6e7ff700 1 -- 172.21.15.33:0/946246438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf68164ce0 con 0x7faf680aa2d0 2023-11-06T20:44:50.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.569+0000 7faf4ffff700 1 -- 172.21.15.33:0/946246438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5801b660 con 0x7faf680aa2d0 2023-11-06T20:44:50.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.569+0000 7faf6e7ff700 1 -- 172.21.15.33:0/946246438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf68165860 con 0x7faf680aa2d0 2023-11-06T20:44:50.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.570+0000 7faf6e7ff700 1 -- 172.21.15.33:0/946246438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf68165b40 con 0x7faf680aa2d0 2023-11-06T20:44:50.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.571+0000 7faf4ffff700 1 -- 172.21.15.33:0/946246438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7faf5801ba10 con 0x7faf680aa2d0 2023-11-06T20:44:50.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.571+0000 7faf4ffff700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf5405f120 0x7faf540615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:50.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.572+0000 7faf4ffff700 1 -- 172.21.15.33:0/946246438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7faf5807bb40 con 0x7faf680aa2d0 2023-11-06T20:44:50.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.572+0000 7faf677fe700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf5405f120 0x7faf540615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:50.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.573+0000 7faf677fe700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf5405f120 0x7faf540615e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7faf5c005750 tx=0x7faf5c009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:50.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.575+0000 7faf4ffff700 1 -- 172.21.15.33:0/946246438 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf5804cd50 con 0x7faf680aa2d0 2023-11-06T20:44:50.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.798+0000 7faf6e7ff700 1 -- 172.21.15.33:0/946246438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf6806c6c0 con 0x7faf680aa2d0 2023-11-06T20:44:50.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.799+0000 7faf4ffff700 1 -- 172.21.15.33:0/946246438 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4480 (secure 0 0 0) 0x7faf58029020 con 0x7faf680aa2d0 2023-11-06T20:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4123875619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123875619},{"type":"v1","addr":"172.21.15.33:6834","nonce":4123875619}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3506554570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3506554570},{"type":"v1","addr":"172.21.15.33:6835","nonce":3506554570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/239985290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":239985290},{"type":"v1","addr":"172.21.15.69:6835","nonce":239985290}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:44:06.383743+0000","modified":"2023-11-06T20:44:09.389408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/2681789033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2681789033},{"type":"v1","addr":"172.21.15.69:6837","nonce":2681789033}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-06T20:44:50.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 -- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf5405f120 msgr2=0x7faf540615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf5405f120 0x7faf540615e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7faf5c005750 tx=0x7faf5c009040).stop 2023-11-06T20:44:50.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 -- 172.21.15.33:0/946246438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680aa2d0 msgr2=0x7faf680b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:50.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680aa2d0 0x7faf680b01b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7faf6806bc90 tx=0x7faf58017000).stop 2023-11-06T20:44:50.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 -- 172.21.15.33:0/946246438 shutdown_connections 2023-11-06T20:44:50.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf5405f120 0x7faf540615e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf680c6460 0x7faf680c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680aa2d0 0x7faf680b01b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 --2- 172.21.15.33:0/946246438 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680c1840 0x7faf680c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 -- 172.21.15.33:0/946246438 >> 172.21.15.33:0/946246438 conn(0x7faf6805ec80 msgr2=0x7faf68063140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 -- 172.21.15.33:0/946246438 shutdown_connections 2023-11-06T20:44:50.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:50.801+0000 7faf6e7ff700 1 -- 172.21.15.33:0/946246438 wait complete. 2023-11-06T20:44:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2023-11-06T20:44:50.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:44:51.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.015+0000 7fc0e3fff700 1 Processor -- start 2023-11-06T20:44:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.015+0000 7fc0e3fff700 1 -- start start 2023-11-06T20:44:51.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.016+0000 7fc0e3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0dc154280 0x7fc0dc154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e3fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc0dc0ad6e0 0x7fc0dc0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e3fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0dc0c5db0 con 0x7fc0dc0a9f20 2023-11-06T20:44:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0dc0c6930 con 0x7fc0dc154280 2023-11-06T20:44:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e3fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0dc0c6000 con 0x7fc0dc0ad6e0 2023-11-06T20:44:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0dc154280 0x7fc0dc154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0dc154280 0x7fc0dc154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41020/0 (socket says 172.21.15.33:41020) 2023-11-06T20:44:51.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e1d9b700 1 -- 172.21.15.33:0/1795284817 learned_addr learned my addr 172.21.15.33:0/1795284817 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:51.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e159a700 1 --2- 172.21.15.33:0/1795284817 >> v2:172.21.15.69:3300/0 conn(0x7fc0dc0ad6e0 0x7fc0dc0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e1d9b700 1 -- 172.21.15.33:0/1795284817 >> v2:172.21.15.69:3300/0 conn(0x7fc0dc0ad6e0 msgr2=0x7fc0dc0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:51.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e1d9b700 1 --2- 172.21.15.33:0/1795284817 >> v2:172.21.15.69:3300/0 conn(0x7fc0dc0ad6e0 0x7fc0dc0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e1d9b700 1 -- 172.21.15.33:0/1795284817 >> v1:172.21.15.69:6789/0 conn(0x7fc0dc0a9f20 legacy=0x7fc0dc0c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:51.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e1d9b700 1 -- 172.21.15.33:0/1795284817 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc0c62e0 con 0x7fc0dc154280 2023-11-06T20:44:51.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.017+0000 7fc0e159a700 1 --2- 172.21.15.33:0/1795284817 >> v2:172.21.15.69:3300/0 conn(0x7fc0dc0ad6e0 0x7fc0dc0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e1d9b700 1 --2- 172.21.15.33:0/1795284817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 0x7fc0dc154660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fc0dc0c8910 tx=0x7fc0d80090f0).ready entity=mon.0 client_cookie=2c03b3d29e19b1b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e0d99700 1 -- 172.21.15.33:0/1795284817 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d800b5a0 con 0x7fc0dc154280 2023-11-06T20:44:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e0d99700 1 -- 172.21.15.33:0/1795284817 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d800b700 con 0x7fc0dc154280 2023-11-06T20:44:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e0d99700 1 -- 172.21.15.33:0/1795284817 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d800ba80 con 0x7fc0dc154280 2023-11-06T20:44:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e3fff700 1 -- 172.21.15.33:0/1795284817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 msgr2=0x7fc0dc154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e3fff700 1 --2- 172.21.15.33:0/1795284817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 0x7fc0dc154660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fc0dc0c8910 tx=0x7fc0d80090f0).stop 2023-11-06T20:44:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e3fff700 1 -- 172.21.15.33:0/1795284817 shutdown_connections 2023-11-06T20:44:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e3fff700 1 --2- 172.21.15.33:0/1795284817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 0x7fc0dc154660 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e3fff700 1 --2- 172.21.15.33:0/1795284817 >> v2:172.21.15.69:3300/0 conn(0x7fc0dc0ad6e0 0x7fc0dc0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e3fff700 1 -- 172.21.15.33:0/1795284817 >> 172.21.15.33:0/1795284817 conn(0x7fc0dc05e1b0 msgr2=0x7fc0dc05e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.018+0000 7fc0e3fff700 1 -- 172.21.15.33:0/1795284817 shutdown_connections 2023-11-06T20:44:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.019+0000 7fc0e3fff700 1 -- 172.21.15.33:0/1795284817 wait complete. 2023-11-06T20:44:51.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.021+0000 7fc0e3fff700 1 Processor -- start 2023-11-06T20:44:51.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.021+0000 7fc0e3fff700 1 -- start start 2023-11-06T20:44:51.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.022+0000 7fc0e3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc0a9f20 0x7fc0dc164f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.022+0000 7fc0e3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0dc0ad6e0 0x7fc0dc167b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.022+0000 7fc0e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 0x7fc0dc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.022+0000 7fc0e3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc164580 con 0x7fc0dc154280 2023-11-06T20:44:51.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.022+0000 7fc0e3fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc164b00 con 0x7fc0dc0a9f20 2023-11-06T20:44:51.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.022+0000 7fc0e3fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc165480 con 0x7fc0dc0ad6e0 2023-11-06T20:44:51.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.022+0000 7fc0e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 0x7fc0dc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.022+0000 7fc0e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 0x7fc0dc0af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41032/0 (socket says 172.21.15.33:41032) 2023-11-06T20:44:51.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.022+0000 7fc0e259c700 1 -- 172.21.15.33:0/102014786 learned_addr learned my addr 172.21.15.33:0/102014786 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:51.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e1d9b700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc0a9f20 0x7fc0dc164f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e159a700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0dc0ad6e0 0x7fc0dc167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e259c700 1 -- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0dc0ad6e0 msgr2=0x7fc0dc167b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e259c700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0dc0ad6e0 0x7fc0dc167b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e259c700 1 -- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc0a9f20 msgr2=0x7fc0dc164f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e259c700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc0a9f20 0x7fc0dc164f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e259c700 1 -- 172.21.15.33:0/102014786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d8009370 con 0x7fc0dc154280 2023-11-06T20:44:51.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e159a700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0dc0ad6e0 0x7fc0dc167b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:44:51.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e259c700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 0x7fc0dc0af9a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fc0d400c840 tx=0x7fc0d4002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:51.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0d1ffb700 1 -- 172.21.15.33:0/102014786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4010510 con 0x7fc0dc154280 2023-11-06T20:44:51.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0d1ffb700 1 -- 172.21.15.33:0/102014786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d4010690 con 0x7fc0dc154280 2023-11-06T20:44:51.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e3fff700 1 -- 172.21.15.33:0/102014786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0dc1647b0 con 0x7fc0dc154280 2023-11-06T20:44:51.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0d1ffb700 1 -- 172.21.15.33:0/102014786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4010a50 con 0x7fc0dc154280 2023-11-06T20:44:51.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.023+0000 7fc0e3fff700 1 -- 172.21.15.33:0/102014786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc168830 con 0x7fc0dc154280 2023-11-06T20:44:51.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.024+0000 7fc0e3fff700 1 -- 172.21.15.33:0/102014786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0dc164c90 con 0x7fc0dc154280 2023-11-06T20:44:51.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.025+0000 7fc0d1ffb700 1 -- 172.21.15.33:0/102014786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc0d4012440 con 0x7fc0dc154280 2023-11-06T20:44:51.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.026+0000 7fc0d1ffb700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0c005f150 0x7fc0c0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.026+0000 7fc0d1ffb700 1 -- 172.21.15.33:0/102014786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc0d4079ce0 con 0x7fc0dc154280 2023-11-06T20:44:51.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.027+0000 7fc0e1d9b700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0c005f150 0x7fc0c0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.027+0000 7fc0e1d9b700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0c005f150 0x7fc0c0061610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fc0d8009130 tx=0x7fc0d8015000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:51.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.030+0000 7fc0d1ffb700 1 -- 172.21.15.33:0/102014786 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc0d4048af0 con 0x7fc0dc154280 2023-11-06T20:44:51.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.254+0000 7fc0e3fff700 1 -- 172.21.15.33:0/102014786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0dc0c65c0 con 0x7fc0dc154280 2023-11-06T20:44:51.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.255+0000 7fc0d1ffb700 1 -- 172.21.15.33:0/102014786 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4480 (secure 0 0 0) 0x7fc0d404af40 con 0x7fc0dc154280 2023-11-06T20:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4123875619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123875619},{"type":"v1","addr":"172.21.15.33:6834","nonce":4123875619}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3506554570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3506554570},{"type":"v1","addr":"172.21.15.33:6835","nonce":3506554570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/239985290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":239985290},{"type":"v1","addr":"172.21.15.69:6835","nonce":239985290}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:44:06.383743+0000","modified":"2023-11-06T20:44:09.389408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/2681789033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2681789033},{"type":"v1","addr":"172.21.15.69:6837","nonce":2681789033}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-06T20:44:51.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 -- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0c005f150 msgr2=0x7fc0c0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0c005f150 0x7fc0c0061610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fc0d8009130 tx=0x7fc0d8015000).stop 2023-11-06T20:44:51.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 -- 172.21.15.33:0/102014786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 msgr2=0x7fc0dc0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 0x7fc0dc0af9a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fc0d400c840 tx=0x7fc0d4002560).stop 2023-11-06T20:44:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 -- 172.21.15.33:0/102014786 shutdown_connections 2023-11-06T20:44:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc0c005f150 0x7fc0c0061610 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0dc154280 0x7fc0dc0af9a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0dc0a9f20 0x7fc0dc164f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 --2- 172.21.15.33:0/102014786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0dc0ad6e0 0x7fc0dc167b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 -- 172.21.15.33:0/102014786 >> 172.21.15.33:0/102014786 conn(0x7fc0dc05e1b0 msgr2=0x7fc0dc0ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:51.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 -- 172.21.15.33:0/102014786 shutdown_connections 2023-11-06T20:44:51.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.257+0000 7fc0e3fff700 1 -- 172.21.15.33:0/102014786 wait complete. 2023-11-06T20:44:51.259 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2023-11-06T20:44:51.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:44:51.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.470+0000 7f86015ff700 1 Processor -- start 2023-11-06T20:44:51.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.471+0000 7f86015ff700 1 -- start start 2023-11-06T20:44:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.472+0000 7f86015ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f85fc0c1840 0x7f85fc0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.472+0000 7f86015ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85fc0aa2d0 0x7f85fc0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.472+0000 7f86015ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc1554a0 con 0x7f85fc0c6240 2023-11-06T20:44:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.472+0000 7f86015ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc154960 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.472+0000 7f86015ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc155930 con 0x7f85fc0c1840 2023-11-06T20:44:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.472+0000 7f85fa7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85fc0aa2d0 0x7f85fc0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.472+0000 7f85fa7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85fc0aa2d0 0x7f85fc0c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41048/0 (socket says 172.21.15.33:41048) 2023-11-06T20:44:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.472+0000 7f85fa7fc700 1 -- 172.21.15.33:0/2536479009 learned_addr learned my addr 172.21.15.33:0/2536479009 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.472+0000 7f85faffd700 1 --2- 172.21.15.33:0/2536479009 >> v2:172.21.15.69:3300/0 conn(0x7f85fc0c1840 0x7f85fc0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.473+0000 7f85fa7fc700 1 -- 172.21.15.33:0/2536479009 >> v2:172.21.15.69:3300/0 conn(0x7f85fc0c1840 msgr2=0x7f85fc0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:51.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.473+0000 7f85fa7fc700 1 --2- 172.21.15.33:0/2536479009 >> v2:172.21.15.69:3300/0 conn(0x7f85fc0c1840 0x7f85fc0a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.473+0000 7f85fa7fc700 1 -- 172.21.15.33:0/2536479009 >> v1:172.21.15.69:6789/0 conn(0x7f85fc0c6240 legacy=0x7f85fc157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.473+0000 7f85fa7fc700 1 -- 172.21.15.33:0/2536479009 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc154b70 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.473+0000 7f85fa7fc700 1 --2- 172.21.15.33:0/2536479009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 0x7f85fc0c5d50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f85fc0840b0 tx=0x7f85f00090f0).ready entity=mon.0 client_cookie=65d13dac010c8867 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:51.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.473+0000 7f85f9ffb700 1 -- 172.21.15.33:0/2536479009 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f000b520 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.473+0000 7f85f9ffb700 1 -- 172.21.15.33:0/2536479009 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f000b680 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.473+0000 7f85f9ffb700 1 -- 172.21.15.33:0/2536479009 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f000ba00 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.474+0000 7f86015ff700 1 -- 172.21.15.33:0/2536479009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 msgr2=0x7f85fc0c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.474+0000 7f86015ff700 1 --2- 172.21.15.33:0/2536479009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 0x7f85fc0c5d50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f85fc0840b0 tx=0x7f85f00090f0).stop 2023-11-06T20:44:51.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.474+0000 7f86015ff700 1 -- 172.21.15.33:0/2536479009 shutdown_connections 2023-11-06T20:44:51.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.474+0000 7f86015ff700 1 --2- 172.21.15.33:0/2536479009 >> v2:172.21.15.69:3300/0 conn(0x7f85fc0c1840 0x7f85fc0a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.474+0000 7f86015ff700 1 --2- 172.21.15.33:0/2536479009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 0x7f85fc0c5d50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.474+0000 7f86015ff700 1 -- 172.21.15.33:0/2536479009 >> 172.21.15.33:0/2536479009 conn(0x7f85fc05ec80 msgr2=0x7f85fc0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:51.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.474+0000 7f86015ff700 1 -- 172.21.15.33:0/2536479009 shutdown_connections 2023-11-06T20:44:51.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.474+0000 7f86015ff700 1 -- 172.21.15.33:0/2536479009 wait complete. 2023-11-06T20:44:51.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.475+0000 7f86015ff700 1 Processor -- start 2023-11-06T20:44:51.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.476+0000 7f86015ff700 1 -- start start 2023-11-06T20:44:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.476+0000 7f86015ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 0x7f85fc158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f86015ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85fc0c1840 0x7f85fc144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f86015ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85fc0c6240 0x7f85fc0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f86015ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc157020 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f86015ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc157300 con 0x7f85fc0c6240 2023-11-06T20:44:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f86015ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc1575e0 con 0x7f85fc0c1840 2023-11-06T20:44:51.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 0x7f85fc158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 0x7f85fc158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41064/0 (socket says 172.21.15.33:41064) 2023-11-06T20:44:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85faffd700 1 -- 172.21.15.33:0/3274322098 learned_addr learned my addr 172.21.15.33:0/3274322098 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85fa7fc700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85fc0c1840 0x7f85fc144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85fb7fe700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85fc0c6240 0x7f85fc0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85faffd700 1 -- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85fc0c1840 msgr2=0x7f85fc144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85faffd700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85fc0c1840 0x7f85fc144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85faffd700 1 -- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85fc0c6240 msgr2=0x7f85fc0c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85faffd700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85fc0c6240 0x7f85fc0c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85faffd700 1 -- 172.21.15.33:0/3274322098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f0009370 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85fb7fe700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85fc0c6240 0x7f85fc0c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:51.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.477+0000 7f85faffd700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 0x7f85fc158390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f85ec00e930 tx=0x7f85ec009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.478+0000 7f85e2ffd700 1 -- 172.21.15.33:0/3274322098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec00c670 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.478+0000 7f85e2ffd700 1 -- 172.21.15.33:0/3274322098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec00c7d0 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.478+0000 7f85e2ffd700 1 -- 172.21.15.33:0/3274322098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec00cb50 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.478+0000 7f86015ff700 1 -- 172.21.15.33:0/3274322098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc1578c0 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.478+0000 7f86015ff700 1 -- 172.21.15.33:0/3274322098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc144fa0 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.479+0000 7f86015ff700 1 -- 172.21.15.33:0/3274322098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc0ae180 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.480+0000 7f85e2ffd700 1 -- 172.21.15.33:0/3274322098 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f85ec01b400 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.480+0000 7f85e2ffd700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f85e805f150 0x7f85e8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.481+0000 7f85e2ffd700 1 -- 172.21.15.33:0/3274322098 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f85ec07a6b0 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.481+0000 7f85fa7fc700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f85e805f150 0x7f85e8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.481+0000 7f85fa7fc700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f85e805f150 0x7f85e8061610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f85fc0aa6d0 tx=0x7f85f0017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.485+0000 7f85e2ffd700 1 -- 172.21.15.33:0/3274322098 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f85ec04b9d0 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.669+0000 7f86015ff700 1 -- 172.21.15.33:0/3274322098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85fc1451f0 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.670+0000 7f85e2ffd700 1 -- 172.21.15.33:0/3274322098 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f85ec04eff0 con 0x7f85fc0aa2d0 2023-11-06T20:44:51.670 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:51.670 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:07.385695+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:44:04.716205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T20:44:05.351480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:44:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 -- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f85e805f150 msgr2=0x7f85e8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f85e805f150 0x7f85e8061610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f85fc0aa6d0 tx=0x7f85f0017000).stop 2023-11-06T20:44:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 -- 172.21.15.33:0/3274322098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 msgr2=0x7f85fc158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 0x7f85fc158390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f85ec00e930 tx=0x7f85ec009060).stop 2023-11-06T20:44:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 -- 172.21.15.33:0/3274322098 shutdown_connections 2023-11-06T20:44:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f85e805f150 0x7f85e8061610 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85fc0c6240 0x7f85fc0c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85fc0aa2d0 0x7f85fc158390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 --2- 172.21.15.33:0/3274322098 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85fc0c1840 0x7f85fc144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 -- 172.21.15.33:0/3274322098 >> 172.21.15.33:0/3274322098 conn(0x7f85fc05ec80 msgr2=0x7f85fc150dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:51.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 -- 172.21.15.33:0/3274322098 shutdown_connections 2023-11-06T20:44:51.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.672+0000 7f86015ff700 1 -- 172.21.15.33:0/3274322098 wait complete. 2023-11-06T20:44:51.690 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T20:44:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.887+0000 7f61a65ff700 1 Processor -- start 2023-11-06T20:44:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.887+0000 7f61a65ff700 1 -- start start 2023-11-06T20:44:51.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.889+0000 7f61a65ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a0154210 0x7f61a00a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.889+0000 7f61a65ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a005efc0 con 0x7f61a00bfd60 2023-11-06T20:44:51.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.889+0000 7f61a65ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a00c8ea0 con 0x7f61a00ca610 2023-11-06T20:44:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.889+0000 7f61a65ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00c9140 con 0x7f61a0154210 2023-11-06T20:44:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.889+0000 7f619f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f61a00bfd60 0x7f61a00c9f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36028/0 (socket says 172.21.15.33:36028) 2023-11-06T20:44:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.889+0000 7f619f7fe700 1 -- 172.21.15.33:0/1506568157 learned_addr learned my addr 172.21.15.33:0/1506568157 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.889+0000 7f619ffff700 1 --2- 172.21.15.33:0/1506568157 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a0154210 0x7f61a00a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.889+0000 7f619effd700 1 -- 172.21.15.33:0/1506568157 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363205951 0 0) 0x7f61a005efc0 con 0x7f61a00bfd60 2023-11-06T20:44:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.890+0000 7f619effd700 1 -- 172.21.15.33:0/1506568157 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61880033f0 con 0x7f61a00bfd60 2023-11-06T20:44:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.890+0000 7f619effd700 1 -- 172.21.15.33:0/1506568157 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61390814 0 0) 0x7f61a00c8ea0 con 0x7f61a00ca610 2023-11-06T20:44:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.890+0000 7f619ffff700 1 -- 172.21.15.33:0/1506568157 >> v1:172.21.15.69:6789/0 conn(0x7f61a00ca610 legacy=0x7f61a00c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.890+0000 7f619ffff700 1 -- 172.21.15.33:0/1506568157 >> v1:172.21.15.33:6789/0 conn(0x7f61a00bfd60 legacy=0x7f61a00c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.890+0000 7f619ffff700 1 -- 172.21.15.33:0/1506568157 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a00c9420 con 0x7f61a0154210 2023-11-06T20:44:51.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.890+0000 7f619ffff700 1 --2- 172.21.15.33:0/1506568157 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a0154210 0x7f61a00a7bd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f6190009790 tx=0x7f6190009060).ready entity=mon.2 client_cookie=34932fa683eeb2db server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:51.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.890+0000 7f619effd700 1 -- 172.21.15.33:0/1506568157 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000f510 con 0x7f61a0154210 2023-11-06T20:44:51.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.890+0000 7f619effd700 1 -- 172.21.15.33:0/1506568157 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619000f670 con 0x7f61a0154210 2023-11-06T20:44:51.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.890+0000 7f619effd700 1 -- 172.21.15.33:0/1506568157 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000f9f0 con 0x7f61a0154210 2023-11-06T20:44:51.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.891+0000 7f61a65ff700 1 -- 172.21.15.33:0/1506568157 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a0154210 msgr2=0x7f61a00a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.891+0000 7f61a65ff700 1 --2- 172.21.15.33:0/1506568157 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a0154210 0x7f61a00a7bd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f6190009790 tx=0x7f6190009060).stop 2023-11-06T20:44:51.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.891+0000 7f61a65ff700 1 -- 172.21.15.33:0/1506568157 shutdown_connections 2023-11-06T20:44:51.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.891+0000 7f61a65ff700 1 --2- 172.21.15.33:0/1506568157 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a0154210 0x7f61a00a7bd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.891+0000 7f61a65ff700 1 -- 172.21.15.33:0/1506568157 >> 172.21.15.33:0/1506568157 conn(0x7f61a005e990 msgr2=0x7f61a00cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.891+0000 7f61a65ff700 1 -- 172.21.15.33:0/1506568157 shutdown_connections 2023-11-06T20:44:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.891+0000 7f61a65ff700 1 -- 172.21.15.33:0/1506568157 wait complete. 2023-11-06T20:44:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.893+0000 7f61a65ff700 1 Processor -- start 2023-11-06T20:44:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.893+0000 7f61a65ff700 1 -- start start 2023-11-06T20:44:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.894+0000 7f61a65ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a00bfd60 0x7f61a01655a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f61a65ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a00ca610 0x7f61a014fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f61a65ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61a0154210 0x7f61a00ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f61a65ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00aaaa0 con 0x7f61a00bfd60 2023-11-06T20:44:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f61a65ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01501e0 con 0x7f61a0154210 2023-11-06T20:44:51.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f61a65ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00aca30 con 0x7f61a00ca610 2023-11-06T20:44:51.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f619ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a00bfd60 0x7f61a01655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f619ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a00bfd60 0x7f61a01655a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41076/0 (socket says 172.21.15.33:41076) 2023-11-06T20:44:51.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f619ffff700 1 -- 172.21.15.33:0/2935833185 learned_addr learned my addr 172.21.15.33:0/2935833185 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:51.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f61a4b9c700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61a0154210 0x7f61a00ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f619f7fe700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a00ca610 0x7f61a014fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f619ffff700 1 -- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a00ca610 msgr2=0x7f61a014fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f619ffff700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a00ca610 0x7f61a014fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f619ffff700 1 -- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61a0154210 msgr2=0x7f61a00ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f619ffff700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61a0154210 0x7f61a00ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.895+0000 7f619ffff700 1 -- 172.21.15.33:0/2935833185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61900092a0 con 0x7f61a00bfd60 2023-11-06T20:44:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.896+0000 7f619ffff700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a00bfd60 0x7f61a01655a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f61a0083660 tx=0x7f6190017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.896+0000 7f6187fff700 1 -- 172.21.15.33:0/2935833185 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000fe80 con 0x7f61a00bfd60 2023-11-06T20:44:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.896+0000 7f6187fff700 1 -- 172.21.15.33:0/2935833185 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619000f7e0 con 0x7f61a00bfd60 2023-11-06T20:44:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.896+0000 7f6187fff700 1 -- 172.21.15.33:0/2935833185 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61900246a0 con 0x7f61a00bfd60 2023-11-06T20:44:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.896+0000 7f61a65ff700 1 -- 172.21.15.33:0/2935833185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a00acd00 con 0x7f61a00bfd60 2023-11-06T20:44:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.896+0000 7f61a65ff700 1 -- 172.21.15.33:0/2935833185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a00ad2c0 con 0x7f61a00bfd60 2023-11-06T20:44:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.897+0000 7f61a65ff700 1 -- 172.21.15.33:0/2935833185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a00aad80 con 0x7f61a00bfd60 2023-11-06T20:44:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.898+0000 7f6187fff700 1 -- 172.21.15.33:0/2935833185 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6190021070 con 0x7f61a00bfd60 2023-11-06T20:44:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.899+0000 7f6187fff700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f618c05f150 0x7f618c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.899+0000 7f6187fff700 1 -- 172.21.15.33:0/2935833185 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f619007bd20 con 0x7f61a00bfd60 2023-11-06T20:44:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.900+0000 7f619f7fe700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f618c05f150 0x7f618c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:51.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.900+0000 7f619f7fe700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f618c05f150 0x7f618c061610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f61a00c8900 tx=0x7f6194009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:51.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:51.904+0000 7f6187fff700 1 -- 172.21.15.33:0/2935833185 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f619004cf30 con 0x7f61a00bfd60 2023-11-06T20:44:52.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.126+0000 7f61a65ff700 1 -- 172.21.15.33:0/2935833185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f61a006c6c0 con 0x7f61a00bfd60 2023-11-06T20:44:52.826 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T20:44:52.826+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T20:44:52.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.833+0000 7f6187fff700 1 -- 172.21.15.33:0/2935833185 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6190029070 con 0x7f61a00bfd60 2023-11-06T20:44:52.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 -- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f618c05f150 msgr2=0x7f618c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:52.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f618c05f150 0x7f618c061610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f61a00c8900 tx=0x7f6194009040).stop 2023-11-06T20:44:52.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 -- 172.21.15.33:0/2935833185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a00bfd60 msgr2=0x7f61a01655a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:52.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a00bfd60 0x7f61a01655a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f61a0083660 tx=0x7f6190017000).stop 2023-11-06T20:44:52.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 -- 172.21.15.33:0/2935833185 shutdown_connections 2023-11-06T20:44:52.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f618c05f150 0x7f618c061610 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:52.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61a0154210 0x7f61a00ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:52.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61a00bfd60 0x7f61a01655a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:52.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 --2- 172.21.15.33:0/2935833185 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61a00ca610 0x7f61a014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:52.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 -- 172.21.15.33:0/2935833185 >> 172.21.15.33:0/2935833185 conn(0x7f61a005e990 msgr2=0x7f61a0153160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:52.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 -- 172.21.15.33:0/2935833185 shutdown_connections 2023-11-06T20:44:52.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:52.835+0000 7f61a65ff700 1 -- 172.21.15.33:0/2935833185 wait complete. 2023-11-06T20:44:52.838 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T20:44:52.839 INFO:tasks.ceph.mds.d.smithi069.stderr: -2281> 2023-11-06T20:44:07.059+0000 7f33c81f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T20:44:52.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T20:44:52.882 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T20:44:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.056+0000 7ffb2b1ff700 1 Processor -- start 2023-11-06T20:44:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.056+0000 7ffb2b1ff700 1 -- start start 2023-11-06T20:44:53.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.058+0000 7ffb2b1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb24155410 0x7ffb240644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:53.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.058+0000 7ffb2b1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb241547d0 0x7ffb240649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:53.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.058+0000 7ffb2b1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb24063c00 con 0x7ffb24155030 2023-11-06T20:44:53.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.058+0000 7ffb2b1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb24064090 con 0x7ffb24155410 2023-11-06T20:44:53.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.058+0000 7ffb2b1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb24064ed0 con 0x7ffb241547d0 2023-11-06T20:44:53.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.058+0000 7ffb23fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb24155410 0x7ffb240644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:53.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.058+0000 7ffb23fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb24155410 0x7ffb240644f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41082/0 (socket says 172.21.15.33:41082) 2023-11-06T20:44:53.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.058+0000 7ffb23fff700 1 -- 172.21.15.33:0/2953488910 learned_addr learned my addr 172.21.15.33:0/2953488910 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:53.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.058+0000 7ffb2979c700 1 --2- 172.21.15.33:0/2953488910 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb241547d0 0x7ffb240649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:53.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.059+0000 7ffb23fff700 1 -- 172.21.15.33:0/2953488910 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb241547d0 msgr2=0x7ffb240649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:53.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.059+0000 7ffb23fff700 1 --2- 172.21.15.33:0/2953488910 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb241547d0 0x7ffb240649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:53.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.059+0000 7ffb23fff700 1 -- 172.21.15.33:0/2953488910 >> v1:172.21.15.69:6789/0 conn(0x7ffb24155030 legacy=0x7ffb24062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:53.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.059+0000 7ffb23fff700 1 -- 172.21.15.33:0/2953488910 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb24062ff0 con 0x7ffb24155410 2023-11-06T20:44:53.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.059+0000 7ffb2979c700 1 --2- 172.21.15.33:0/2953488910 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb241547d0 0x7ffb240649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:53.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.059+0000 7ffb23fff700 1 --2- 172.21.15.33:0/2953488910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb24155410 0x7ffb240644f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7ffb10006fd0 tx=0x7ffb100090f0).ready entity=mon.0 client_cookie=a979cc3f50bc023d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:53.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.059+0000 7ffb237fe700 1 -- 172.21.15.33:0/2953488910 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000b600 con 0x7ffb24155410 2023-11-06T20:44:53.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.059+0000 7ffb237fe700 1 -- 172.21.15.33:0/2953488910 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1000b760 con 0x7ffb24155410 2023-11-06T20:44:53.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.059+0000 7ffb237fe700 1 -- 172.21.15.33:0/2953488910 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000bae0 con 0x7ffb24155410 2023-11-06T20:44:53.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.060+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/2953488910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb24155410 msgr2=0x7ffb240644f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:53.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.060+0000 7ffb2b1ff700 1 --2- 172.21.15.33:0/2953488910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb24155410 0x7ffb240644f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7ffb10006fd0 tx=0x7ffb100090f0).stop 2023-11-06T20:44:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.060+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/2953488910 shutdown_connections 2023-11-06T20:44:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.060+0000 7ffb2b1ff700 1 --2- 172.21.15.33:0/2953488910 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb241547d0 0x7ffb240649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.060+0000 7ffb2b1ff700 1 --2- 172.21.15.33:0/2953488910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb24155410 0x7ffb240644f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.060+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/2953488910 >> 172.21.15.33:0/2953488910 conn(0x7ffb240c3d30 msgr2=0x7ffb240c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.060+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/2953488910 shutdown_connections 2023-11-06T20:44:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.060+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/2953488910 wait complete. 2023-11-06T20:44:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.061+0000 7ffb2b1ff700 1 Processor -- start 2023-11-06T20:44:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.061+0000 7ffb2b1ff700 1 -- start start 2023-11-06T20:44:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.062+0000 7ffb2b1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb241547d0 0x7ffb24135f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.062+0000 7ffb2b1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb24155030 0x7ffb240c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.062+0000 7ffb2b1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb24155410 0x7ffb240c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.062+0000 7ffb2b1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb240c66e0 con 0x7ffb241547d0 2023-11-06T20:44:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.062+0000 7ffb2b1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb240c0ba0 con 0x7ffb24155410 2023-11-06T20:44:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.062+0000 7ffb2b1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb240c0e80 con 0x7ffb24155030 2023-11-06T20:44:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb28f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb241547d0 0x7ffb24135f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb28f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb241547d0 0x7ffb24135f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41084/0 (socket says 172.21.15.33:41084) 2023-11-06T20:44:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb28f9b700 1 -- 172.21.15.33:0/848610205 learned_addr learned my addr 172.21.15.33:0/848610205 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb23fff700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb24155030 0x7ffb240c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb2979c700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb24155410 0x7ffb240c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:53.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb28f9b700 1 -- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb24155030 msgr2=0x7ffb240c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:53.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb28f9b700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb24155030 0x7ffb240c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:53.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb28f9b700 1 -- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb24155410 msgr2=0x7ffb240c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:53.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb28f9b700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb24155410 0x7ffb240c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:53.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb28f9b700 1 -- 172.21.15.33:0/848610205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb10009370 con 0x7ffb241547d0 2023-11-06T20:44:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb2979c700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb24155410 0x7ffb240c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb28f9b700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb241547d0 0x7ffb24135f10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7ffb1800c4c0 tx=0x7ffb18007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb20ff9700 1 -- 172.21.15.33:0/848610205 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1800ce70 con 0x7ffb241547d0 2023-11-06T20:44:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.064+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/848610205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb240c1160 con 0x7ffb241547d0 2023-11-06T20:44:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.063+0000 7ffb20ff9700 1 -- 172.21.15.33:0/848610205 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb18001e90 con 0x7ffb241547d0 2023-11-06T20:44:53.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.064+0000 7ffb20ff9700 1 -- 172.21.15.33:0/848610205 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18007250 con 0x7ffb241547d0 2023-11-06T20:44:53.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.064+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/848610205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb240aa620 con 0x7ffb241547d0 2023-11-06T20:44:53.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.065+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/848610205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb240aa7b0 con 0x7ffb241547d0 2023-11-06T20:44:53.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.065+0000 7ffb20ff9700 1 -- 172.21.15.33:0/848610205 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ffb180066a0 con 0x7ffb241547d0 2023-11-06T20:44:53.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.066+0000 7ffb20ff9700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffb0805f150 0x7ffb08061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:53.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.066+0000 7ffb20ff9700 1 -- 172.21.15.33:0/848610205 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ffb18079ab0 con 0x7ffb241547d0 2023-11-06T20:44:53.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.067+0000 7ffb23fff700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffb0805f150 0x7ffb08061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:53.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.067+0000 7ffb23fff700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffb0805f150 0x7ffb08061610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7ffb1000b3c0 tx=0x7ffb10017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:53.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.070+0000 7ffb20ff9700 1 -- 172.21.15.33:0/848610205 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffb1804ad30 con 0x7ffb241547d0 2023-11-06T20:44:53.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.295+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/848610205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffb2405e280 con 0x7ffb241547d0 2023-11-06T20:44:53.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.835+0000 7ffb20ff9700 1 -- 172.21.15.33:0/848610205 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffb1804e350 con 0x7ffb241547d0 2023-11-06T20:44:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.837+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffb0805f150 msgr2=0x7ffb08061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.837+0000 7ffb2b1ff700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffb0805f150 0x7ffb08061610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7ffb1000b3c0 tx=0x7ffb10017000).stop 2023-11-06T20:44:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.837+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/848610205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb241547d0 msgr2=0x7ffb24135f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:53.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.837+0000 7ffb2b1ff700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb241547d0 0x7ffb24135f10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7ffb1800c4c0 tx=0x7ffb18007e00).stop 2023-11-06T20:44:53.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.838+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/848610205 shutdown_connections 2023-11-06T20:44:53.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.838+0000 7ffb2b1ff700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffb0805f150 0x7ffb08061610 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:53.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.838+0000 7ffb2b1ff700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb24155410 0x7ffb240c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:53.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.838+0000 7ffb2b1ff700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb241547d0 0x7ffb24135f10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:53.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.838+0000 7ffb2b1ff700 1 --2- 172.21.15.33:0/848610205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb24155030 0x7ffb240c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:53.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.838+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/848610205 >> 172.21.15.33:0/848610205 conn(0x7ffb240c3d30 msgr2=0x7ffb241536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:53.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.838+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/848610205 shutdown_connections 2023-11-06T20:44:53.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:53.838+0000 7ffb2b1ff700 1 -- 172.21.15.33:0/848610205 wait complete. 2023-11-06T20:44:53.858 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T20:44:54.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.064+0000 7ffa925ff700 1 Processor -- start 2023-11-06T20:44:54.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.065+0000 7ffa925ff700 1 -- start start 2023-11-06T20:44:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.066+0000 7ffa925ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa8c0c8ee0 0x7ffa8c0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:54.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.066+0000 7ffa925ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ffa8c1542b0 0x7ffa8c0c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:54.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.066+0000 7ffa925ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8c0c9f80 con 0x7ffa8c153ed0 2023-11-06T20:44:54.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.066+0000 7ffa925ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8c0c6000 con 0x7ffa8c0c8ee0 2023-11-06T20:44:54.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.066+0000 7ffa925ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8c0c9370 con 0x7ffa8c1542b0 2023-11-06T20:44:54.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.066+0000 7ffa8b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa8c0c8ee0 0x7ffa8c0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.066+0000 7ffa8bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffa8c153ed0 0x7ffa8c158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36030/0 (socket says 172.21.15.33:36030) 2023-11-06T20:44:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.067+0000 7ffa8bfff700 1 -- 172.21.15.33:0/821725468 learned_addr learned my addr 172.21.15.33:0/821725468 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.066+0000 7ffa8b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa8c0c8ee0 0x7ffa8c0ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41094/0 (socket says 172.21.15.33:41094) 2023-11-06T20:44:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.067+0000 7ffa90b9c700 1 --2- 172.21.15.33:0/821725468 >> v2:172.21.15.69:3300/0 conn(0x7ffa8c1542b0 0x7ffa8c0c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.067+0000 7ffa8b7fe700 1 -- 172.21.15.33:0/821725468 >> v2:172.21.15.69:3300/0 conn(0x7ffa8c1542b0 msgr2=0x7ffa8c0c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:54.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.067+0000 7ffa8b7fe700 1 --2- 172.21.15.33:0/821725468 >> v2:172.21.15.69:3300/0 conn(0x7ffa8c1542b0 0x7ffa8c0c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:54.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.067+0000 7ffa8b7fe700 1 -- 172.21.15.33:0/821725468 >> v1:172.21.15.33:6789/0 conn(0x7ffa8c153ed0 legacy=0x7ffa8c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:54.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.067+0000 7ffa8b7fe700 1 -- 172.21.15.33:0/821725468 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c0c9650 con 0x7ffa8c0c8ee0 2023-11-06T20:44:54.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.067+0000 7ffa8b7fe700 1 --2- 172.21.15.33:0/821725468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c0c8ee0 0x7ffa8c0ca410 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7ffa8c0c85d0 tx=0x7ffa80009060).ready entity=mon.0 client_cookie=acd1b72ec55e7d66 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:54.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.067+0000 7ffa8affd700 1 -- 172.21.15.33:0/821725468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000b580 con 0x7ffa8c0c8ee0 2023-11-06T20:44:54.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.067+0000 7ffa8affd700 1 -- 172.21.15.33:0/821725468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8000b6e0 con 0x7ffa8c0c8ee0 2023-11-06T20:44:54.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.067+0000 7ffa8affd700 1 -- 172.21.15.33:0/821725468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000ba60 con 0x7ffa8c0c8ee0 2023-11-06T20:44:54.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.068+0000 7ffa925ff700 1 -- 172.21.15.33:0/821725468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c0c8ee0 msgr2=0x7ffa8c0ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:54.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.068+0000 7ffa925ff700 1 --2- 172.21.15.33:0/821725468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c0c8ee0 0x7ffa8c0ca410 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7ffa8c0c85d0 tx=0x7ffa80009060).stop 2023-11-06T20:44:54.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.068+0000 7ffa925ff700 1 -- 172.21.15.33:0/821725468 shutdown_connections 2023-11-06T20:44:54.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.068+0000 7ffa925ff700 1 --2- 172.21.15.33:0/821725468 >> v2:172.21.15.69:3300/0 conn(0x7ffa8c1542b0 0x7ffa8c0c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:54.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.068+0000 7ffa925ff700 1 --2- 172.21.15.33:0/821725468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c0c8ee0 0x7ffa8c0ca410 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:54.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.068+0000 7ffa925ff700 1 -- 172.21.15.33:0/821725468 >> 172.21.15.33:0/821725468 conn(0x7ffa8c05e1b0 msgr2=0x7ffa8c05e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:54.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.068+0000 7ffa925ff700 1 -- 172.21.15.33:0/821725468 shutdown_connections 2023-11-06T20:44:54.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.068+0000 7ffa925ff700 1 -- 172.21.15.33:0/821725468 wait complete. 2023-11-06T20:44:54.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.070+0000 7ffa925ff700 1 Processor -- start 2023-11-06T20:44:54.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.070+0000 7ffa925ff700 1 -- start start 2023-11-06T20:44:54.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa925ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa8c153ed0 0x7ffa8c14cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:54.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa925ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c1542b0 0x7ffa8c14d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:54.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa925ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa8c164370 0x7ffa8c0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:54.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa925ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c1508a0 con 0x7ffa8c1542b0 2023-11-06T20:44:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa925ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c14f9d0 con 0x7ffa8c153ed0 2023-11-06T20:44:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa925ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c14fc90 con 0x7ffa8c164370 2023-11-06T20:44:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa8b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c1542b0 0x7ffa8c14d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa8b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c1542b0 0x7ffa8c14d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41102/0 (socket says 172.21.15.33:41102) 2023-11-06T20:44:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa8b7fe700 1 -- 172.21.15.33:0/913818625 learned_addr learned my addr 172.21.15.33:0/913818625 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa8bfff700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa8c153ed0 0x7ffa8c14cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa90b9c700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa8c164370 0x7ffa8c0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa8b7fe700 1 -- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa8c164370 msgr2=0x7ffa8c0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa8b7fe700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa8c164370 0x7ffa8c0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa8b7fe700 1 -- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa8c153ed0 msgr2=0x7ffa8c14cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa8b7fe700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa8c153ed0 0x7ffa8c14cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:54.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa8b7fe700 1 -- 172.21.15.33:0/913818625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa800092a0 con 0x7ffa8c1542b0 2023-11-06T20:44:54.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.072+0000 7ffa90b9c700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa8c164370 0x7ffa8c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:54.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.073+0000 7ffa8b7fe700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c1542b0 0x7ffa8c14d370 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7ffa8c06be10 tx=0x7ffa80017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:54.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.073+0000 7ffa73fff700 1 -- 172.21.15.33:0/913818625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000bee0 con 0x7ffa8c1542b0 2023-11-06T20:44:54.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.073+0000 7ffa73fff700 1 -- 172.21.15.33:0/913818625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8000b850 con 0x7ffa8c1542b0 2023-11-06T20:44:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.073+0000 7ffa925ff700 1 -- 172.21.15.33:0/913818625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8c14ff70 con 0x7ffa8c1542b0 2023-11-06T20:44:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.073+0000 7ffa73fff700 1 -- 172.21.15.33:0/913818625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8001b410 con 0x7ffa8c1542b0 2023-11-06T20:44:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.073+0000 7ffa925ff700 1 -- 172.21.15.33:0/913818625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c150530 con 0x7ffa8c1542b0 2023-11-06T20:44:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.074+0000 7ffa925ff700 1 -- 172.21.15.33:0/913818625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c0c64c0 con 0x7ffa8c1542b0 2023-11-06T20:44:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.075+0000 7ffa73fff700 1 -- 172.21.15.33:0/913818625 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ffa8001b7c0 con 0x7ffa8c1542b0 2023-11-06T20:44:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.076+0000 7ffa73fff700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7805f180 0x7ffa78061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:54.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.076+0000 7ffa73fff700 1 -- 172.21.15.33:0/913818625 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6236+0+0 (secure 0 0 0) 0x7ffa8007bc90 con 0x7ffa8c1542b0 2023-11-06T20:44:54.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.076+0000 7ffa8bfff700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7805f180 0x7ffa78061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:54.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.077+0000 7ffa8bfff700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7805f180 0x7ffa78061640 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7ffa7c007fd0 tx=0x7ffa7c007d30).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:54.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.080+0000 7ffa73fff700 1 -- 172.21.15.33:0/913818625 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffa8004ce30 con 0x7ffa8c1542b0 2023-11-06T20:44:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:54.262+0000 7ffa925ff700 1 -- 172.21.15.33:0/913818625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffa8c06c6c0 con 0x7ffa8c1542b0 2023-11-06T20:44:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.146+0000 7ffa73fff700 1 -- 172.21.15.33:0/913818625 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7ffa80028070 con 0x7ffa8c1542b0 2023-11-06T20:44:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.150+0000 7ffa925ff700 1 -- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7805f180 msgr2=0x7ffa78061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.150+0000 7ffa925ff700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7805f180 0x7ffa78061640 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7ffa7c007fd0 tx=0x7ffa7c007d30).stop 2023-11-06T20:44:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.150+0000 7ffa925ff700 1 -- 172.21.15.33:0/913818625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c1542b0 msgr2=0x7ffa8c14d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.150+0000 7ffa925ff700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c1542b0 0x7ffa8c14d370 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7ffa8c06be10 tx=0x7ffa80017000).stop 2023-11-06T20:44:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.150+0000 7ffa925ff700 1 -- 172.21.15.33:0/913818625 shutdown_connections 2023-11-06T20:44:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.150+0000 7ffa925ff700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa7805f180 0x7ffa78061640 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:55.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.150+0000 7ffa925ff700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa8c164370 0x7ffa8c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:55.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.150+0000 7ffa925ff700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa8c153ed0 0x7ffa8c14cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:55.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.150+0000 7ffa925ff700 1 --2- 172.21.15.33:0/913818625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa8c1542b0 0x7ffa8c14d370 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:55.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.150+0000 7ffa925ff700 1 -- 172.21.15.33:0/913818625 >> 172.21.15.33:0/913818625 conn(0x7ffa8c05e1b0 msgr2=0x7ffa8c14e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:55.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.151+0000 7ffa925ff700 1 -- 172.21.15.33:0/913818625 shutdown_connections 2023-11-06T20:44:55.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.151+0000 7ffa925ff700 1 -- 172.21.15.33:0/913818625 wait complete. 2023-11-06T20:44:55.153 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T20:44:55.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T20:44:55.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.376+0000 7fd98f3ff700 1 Processor -- start 2023-11-06T20:44:55.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.376+0000 7fd98f3ff700 1 -- start start 2023-11-06T20:44:55.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.377+0000 7fd98f3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd988158770 0x7fd9880c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:55.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd98f3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9880a3da0 con 0x7fd98813dea0 2023-11-06T20:44:55.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd98f3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd988065090 con 0x7fd9880c1d00 2023-11-06T20:44:55.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd98f3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9880c3200 con 0x7fd988158770 2023-11-06T20:44:55.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd98c99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd988158770 0x7fd9880c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:55.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd98c99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd988158770 0x7fd9880c1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41116/0 (socket says 172.21.15.33:41116) 2023-11-06T20:44:55.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd98c99a700 1 -- 172.21.15.33:0/1080543851 learned_addr learned my addr 172.21.15.33:0/1080543851 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd98d19b700 1 --1- 172.21.15.33:0/1080543851 >> v1:172.21.15.33:6789/0 conn(0x7fd98813dea0 0x7fd988158060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36036/0 (socket says 172.21.15.33:36036) 2023-11-06T20:44:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd98c99a700 1 -- 172.21.15.33:0/1080543851 >> v1:172.21.15.69:6789/0 conn(0x7fd9880c1d00 legacy=0x7fd9880c2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd98c99a700 1 -- 172.21.15.33:0/1080543851 >> v1:172.21.15.33:6789/0 conn(0x7fd98813dea0 legacy=0x7fd988158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd98c99a700 1 -- 172.21.15.33:0/1080543851 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988064760 con 0x7fd988158770 2023-11-06T20:44:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.378+0000 7fd97ffff700 1 -- 172.21.15.33:0/1080543851 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524717275 0 0) 0x7fd9880a3da0 con 0x7fd98813dea0 2023-11-06T20:44:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.379+0000 7fd98c99a700 1 --2- 172.21.15.33:0/1080543851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 0x7fd9880c1810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fd978006fd0 tx=0x7fd9780090f0).ready entity=mon.0 client_cookie=abf443c9b34e7251 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.379+0000 7fd97ffff700 1 -- 172.21.15.33:0/1080543851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800b600 con 0x7fd988158770 2023-11-06T20:44:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.379+0000 7fd97ffff700 1 -- 172.21.15.33:0/1080543851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97800b760 con 0x7fd988158770 2023-11-06T20:44:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.379+0000 7fd97ffff700 1 -- 172.21.15.33:0/1080543851 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800bb00 con 0x7fd988158770 2023-11-06T20:44:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.379+0000 7fd98f3ff700 1 -- 172.21.15.33:0/1080543851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 msgr2=0x7fd9880c1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.379+0000 7fd98f3ff700 1 --2- 172.21.15.33:0/1080543851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 0x7fd9880c1810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fd978006fd0 tx=0x7fd9780090f0).stop 2023-11-06T20:44:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.379+0000 7fd98f3ff700 1 -- 172.21.15.33:0/1080543851 shutdown_connections 2023-11-06T20:44:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.379+0000 7fd98f3ff700 1 --2- 172.21.15.33:0/1080543851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 0x7fd9880c1810 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.379+0000 7fd98f3ff700 1 -- 172.21.15.33:0/1080543851 >> 172.21.15.33:0/1080543851 conn(0x7fd9880c6e70 msgr2=0x7fd9880c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.379+0000 7fd98f3ff700 1 -- 172.21.15.33:0/1080543851 shutdown_connections 2023-11-06T20:44:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.380+0000 7fd98f3ff700 1 -- 172.21.15.33:0/1080543851 wait complete. 2023-11-06T20:44:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.381+0000 7fd98f3ff700 1 Processor -- start 2023-11-06T20:44:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.381+0000 7fd98f3ff700 1 -- start start 2023-11-06T20:44:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.382+0000 7fd98f3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9880c1d00 0x7fd9881643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.382+0000 7fd98f3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd98813dea0 0x7fd9880659f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.382+0000 7fd98f3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 0x7fd988065ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:55.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.382+0000 7fd98f3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880663d0 con 0x7fd988158770 2023-11-06T20:44:55.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.382+0000 7fd98f3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9881680f0 con 0x7fd98813dea0 2023-11-06T20:44:55.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.382+0000 7fd98f3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd988168c70 con 0x7fd9880c1d00 2023-11-06T20:44:55.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.382+0000 7fd98d99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 0x7fd988065ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:55.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.382+0000 7fd98d99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 0x7fd988065ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41120/0 (socket says 172.21.15.33:41120) 2023-11-06T20:44:55.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.382+0000 7fd98d99c700 1 -- 172.21.15.33:0/2511495734 learned_addr learned my addr 172.21.15.33:0/2511495734 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:55.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd98d19b700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9880c1d00 0x7fd9881643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:55.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd98c99a700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd98813dea0 0x7fd9880659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:55.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd98d99c700 1 -- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9880c1d00 msgr2=0x7fd9881643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd98d99c700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9880c1d00 0x7fd9881643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd98d99c700 1 -- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd98813dea0 msgr2=0x7fd9880659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd98d99c700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd98813dea0 0x7fd9880659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd98d99c700 1 -- 172.21.15.33:0/2511495734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd978009370 con 0x7fd988158770 2023-11-06T20:44:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd98c99a700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd98813dea0 0x7fd9880659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd98d99c700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 0x7fd988065ee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fd980002230 tx=0x7fd9800015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd97d7fa700 1 -- 172.21.15.33:0/2511495734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980010510 con 0x7fd988158770 2023-11-06T20:44:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd97d7fa700 1 -- 172.21.15.33:0/2511495734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd980010670 con 0x7fd988158770 2023-11-06T20:44:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd97d7fa700 1 -- 172.21.15.33:0/2511495734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800109f0 con 0x7fd988158770 2023-11-06T20:44:55.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.383+0000 7fd98f3ff700 1 -- 172.21.15.33:0/2511495734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd988168340 con 0x7fd988158770 2023-11-06T20:44:55.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.384+0000 7fd98f3ff700 1 -- 172.21.15.33:0/2511495734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd988168900 con 0x7fd988158770 2023-11-06T20:44:55.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.385+0000 7fd98f3ff700 1 -- 172.21.15.33:0/2511495734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd988168de0 con 0x7fd988158770 2023-11-06T20:44:55.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.385+0000 7fd97d7fa700 1 -- 172.21.15.33:0/2511495734 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd98001b400 con 0x7fd988158770 2023-11-06T20:44:55.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.387+0000 7fd97d7fa700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f150 0x7fd96c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:55.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.387+0000 7fd97d7fa700 1 -- 172.21.15.33:0/2511495734 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5769+0+0 (secure 0 0 0) 0x7fd980079e50 con 0x7fd988158770 2023-11-06T20:44:55.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.388+0000 7fd98d19b700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f150 0x7fd96c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:55.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.388+0000 7fd98d19b700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f150 0x7fd96c061610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd98813d660 tx=0x7fd984007bb0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:55.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.391+0000 7fd97d7fa700 1 -- 172.21.15.33:0/2511495734 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd98004b220 con 0x7fd988158770 2023-11-06T20:44:55.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:55.570+0000 7fd98f3ff700 1 -- 172.21.15.33:0/2511495734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd988064a40 con 0x7fd988158770 2023-11-06T20:44:56.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.146+0000 7fd97d7fa700 1 -- 172.21.15.33:0/2511495734 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd98004e840 con 0x7fd988158770 2023-11-06T20:44:56.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.148+0000 7fd98f3ff700 1 -- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f150 msgr2=0x7fd96c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:56.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.148+0000 7fd98f3ff700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f150 0x7fd96c061610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd98813d660 tx=0x7fd984007bb0).stop 2023-11-06T20:44:56.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.148+0000 7fd98f3ff700 1 -- 172.21.15.33:0/2511495734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 msgr2=0x7fd988065ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:56.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.148+0000 7fd98f3ff700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 0x7fd988065ee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fd980002230 tx=0x7fd9800015e0).stop 2023-11-06T20:44:56.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.149+0000 7fd98f3ff700 1 -- 172.21.15.33:0/2511495734 shutdown_connections 2023-11-06T20:44:56.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.149+0000 7fd98f3ff700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f150 0x7fd96c061610 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:56.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.149+0000 7fd98f3ff700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9880c1d00 0x7fd9881643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:56.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.149+0000 7fd98f3ff700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd988158770 0x7fd988065ee0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.149+0000 7fd98f3ff700 1 --2- 172.21.15.33:0/2511495734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd98813dea0 0x7fd9880659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.149+0000 7fd98f3ff700 1 -- 172.21.15.33:0/2511495734 >> 172.21.15.33:0/2511495734 conn(0x7fd9880c6e70 msgr2=0x7fd9880a5750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.149+0000 7fd98f3ff700 1 -- 172.21.15.33:0/2511495734 shutdown_connections 2023-11-06T20:44:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.149+0000 7fd98f3ff700 1 -- 172.21.15.33:0/2511495734 wait complete. 2023-11-06T20:44:56.152 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T20:44:56.170 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T20:44:56.170 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T20:44:56.171 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T20:44:56.171 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:44:56.171 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T20:44:56.171 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T20:44:56.172 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T20:44:56.172 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:44:56.172 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T20:44:56.173 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T20:44:56.173 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T20:44:56.173 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:44:56.173 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T20:44:56.174 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T20:44:56.174 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T20:44:56.174 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:44:56.187 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T20:44:56.187 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T20:44:56.188 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T20:44:56.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T20:44:56.190 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T20:44:56.190 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T20:44:56.191 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T20:44:56.191 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T20:44:56.234 INFO:tasks.ceph.mds.a:Started 2023-11-06T20:44:56.234 DEBUG:teuthology.parallel:result is None 2023-11-06T20:44:56.235 INFO:tasks.ceph.mds.d:Started 2023-11-06T20:44:56.236 DEBUG:teuthology.parallel:result is None 2023-11-06T20:44:56.238 INFO:tasks.ceph.mds.c:Started 2023-11-06T20:44:56.238 DEBUG:teuthology.parallel:result is None 2023-11-06T20:44:56.239 INFO:tasks.ceph.mds.b:Started 2023-11-06T20:44:56.239 DEBUG:teuthology.parallel:result is None 2023-11-06T20:44:56.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T20:44:56.384 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T20:44:56.384 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T20:44:56.386 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T20:44:56.388 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T20:44:56.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.452+0000 7fcab199d700 1 Processor -- start 2023-11-06T20:44:56.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.453+0000 7fcab199d700 1 -- start start 2023-11-06T20:44:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.454+0000 7fcab199d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac1542c0 0x7fcaac0644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.454+0000 7fcab199d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaac0c0680 con 0x7fcaac0aade0 2023-11-06T20:44:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.454+0000 7fcab199d700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaac0bfd90 con 0x7fcaac0c6260 2023-11-06T20:44:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.454+0000 7fcab199d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0c67f0 con 0x7fcaac1542c0 2023-11-06T20:44:56.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcaabfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcaac0aade0 0x7fcaac0c5b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36066/0 (socket says 172.21.15.33:36066) 2023-11-06T20:44:56.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcaabfff700 1 -- 172.21.15.33:0/1068683055 learned_addr learned my addr 172.21.15.33:0/1068683055 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:56.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcab119c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fcaac0c6260 0x7fcaac0c0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:34640/0 (socket says 172.21.15.33:34640) 2023-11-06T20:44:56.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcab099b700 1 --2- 172.21.15.33:0/1068683055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac1542c0 0x7fcaac0644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:56.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcaab7fe700 1 -- 172.21.15.33:0/1068683055 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791781357 0 0) 0x7fcaac0bfd90 con 0x7fcaac0c6260 2023-11-06T20:44:56.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcaab7fe700 1 -- 172.21.15.33:0/1068683055 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca9c0033f0 con 0x7fcaac0c6260 2023-11-06T20:44:56.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcaab7fe700 1 -- 172.21.15.33:0/1068683055 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683906629 0 0) 0x7fcaac0c0680 con 0x7fcaac0aade0 2023-11-06T20:44:56.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcaab7fe700 1 -- 172.21.15.33:0/1068683055 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaac0bfd90 con 0x7fcaac0aade0 2023-11-06T20:44:56.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcab099b700 1 -- 172.21.15.33:0/1068683055 >> v1:172.21.15.69:6789/0 conn(0x7fcaac0c6260 legacy=0x7fcaac0c0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:56.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcab099b700 1 -- 172.21.15.33:0/1068683055 >> v1:172.21.15.33:6789/0 conn(0x7fcaac0aade0 legacy=0x7fcaac0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:56.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.455+0000 7fcab099b700 1 -- 172.21.15.33:0/1068683055 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac0c0030 con 0x7fcaac1542c0 2023-11-06T20:44:56.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.456+0000 7fcab099b700 1 --2- 172.21.15.33:0/1068683055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac1542c0 0x7fcaac0644a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fcaa0009790 tx=0x7fcaa0009060).ready entity=mon.2 client_cookie=a14c61a2b60d131d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:56.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.456+0000 7fcaab7fe700 1 -- 172.21.15.33:0/1068683055 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000f510 con 0x7fcaac1542c0 2023-11-06T20:44:56.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.457+0000 7fcaab7fe700 1 -- 172.21.15.33:0/1068683055 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa000f670 con 0x7fcaac1542c0 2023-11-06T20:44:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.457+0000 7fcaab7fe700 1 -- 172.21.15.33:0/1068683055 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000f9f0 con 0x7fcaac1542c0 2023-11-06T20:44:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.457+0000 7fcab199d700 1 -- 172.21.15.33:0/1068683055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac1542c0 msgr2=0x7fcaac0644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.457+0000 7fcab199d700 1 --2- 172.21.15.33:0/1068683055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac1542c0 0x7fcaac0644a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fcaa0009790 tx=0x7fcaa0009060).stop 2023-11-06T20:44:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.458+0000 7fcab199d700 1 -- 172.21.15.33:0/1068683055 shutdown_connections 2023-11-06T20:44:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.458+0000 7fcab199d700 1 --2- 172.21.15.33:0/1068683055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac1542c0 0x7fcaac0644a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.458+0000 7fcab199d700 1 -- 172.21.15.33:0/1068683055 >> 172.21.15.33:0/1068683055 conn(0x7fcaac0cb760 msgr2=0x7fcaac0c7730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.458+0000 7fcab199d700 1 -- 172.21.15.33:0/1068683055 shutdown_connections 2023-11-06T20:44:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.458+0000 7fcab199d700 1 -- 172.21.15.33:0/1068683055 wait complete. 2023-11-06T20:44:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.460+0000 7fcab199d700 1 Processor -- start 2023-11-06T20:44:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.460+0000 7fcab199d700 1 -- start start 2023-11-06T20:44:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.461+0000 7fcab199d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaac0aade0 0x7fcaac0aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.461+0000 7fcab199d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac0c6260 0x7fcaac0acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.461+0000 7fcab199d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaac1542c0 0x7fcaac151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.461+0000 7fcab199d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac1507d0 con 0x7fcaac1542c0 2023-11-06T20:44:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.461+0000 7fcab199d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0afe90 con 0x7fcaac0aade0 2023-11-06T20:44:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.461+0000 7fcab199d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0b0170 con 0x7fcaac0c6260 2023-11-06T20:44:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.462+0000 7fcab119c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaac1542c0 0x7fcaac151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.462+0000 7fcab099b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaac0aade0 0x7fcaac0aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.462+0000 7fcab099b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaac0aade0 0x7fcaac0aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52214/0 (socket says 172.21.15.33:52214) 2023-11-06T20:44:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.462+0000 7fcaabfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac0c6260 0x7fcaac0acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.462+0000 7fcab099b700 1 -- 172.21.15.33:0/686380465 learned_addr learned my addr 172.21.15.33:0/686380465 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.462+0000 7fcab099b700 1 -- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac0c6260 msgr2=0x7fcaac0acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.462+0000 7fcab099b700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac0c6260 0x7fcaac0acf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.462+0000 7fcab099b700 1 -- 172.21.15.33:0/686380465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaac1542c0 msgr2=0x7fcaac151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.462+0000 7fcab099b700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaac1542c0 0x7fcaac151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.462+0000 7fcab099b700 1 -- 172.21.15.33:0/686380465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa00092a0 con 0x7fcaac0aade0 2023-11-06T20:44:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.463+0000 7fcab099b700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaac0aade0 0x7fcaac0aea60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fcaa0009fd0 tx=0x7fcaa0017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.463+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/686380465 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0000c90 con 0x7fcaac0aade0 2023-11-06T20:44:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.463+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/686380465 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa000f7e0 con 0x7fcaac0aade0 2023-11-06T20:44:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.463+0000 7fcab199d700 1 -- 172.21.15.33:0/686380465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaac0b0450 con 0x7fcaac0aade0 2023-11-06T20:44:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.463+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/686380465 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa001a650 con 0x7fcaac0aade0 2023-11-06T20:44:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.463+0000 7fcab199d700 1 -- 172.21.15.33:0/686380465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac0ad400 con 0x7fcaac0aade0 2023-11-06T20:44:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.464+0000 7fcab199d700 1 -- 172.21.15.33:0/686380465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaac0b07c0 con 0x7fcaac0aade0 2023-11-06T20:44:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.465+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/686380465 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcaa001aa00 con 0x7fcaac0aade0 2023-11-06T20:44:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.466+0000 7fcaa8ff9700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca9005f150 0x7fca90061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.466+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/686380465 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5355+0+0 (secure 0 0 0) 0x7fcaa007b360 con 0x7fcaac0aade0 2023-11-06T20:44:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.466+0000 7fcaabfff700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca9005f150 0x7fca90061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.467+0000 7fcaabfff700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca9005f150 0x7fca90061610 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fcaac0ae870 tx=0x7fca98007c40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.471+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/686380465 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcaa004c870 con 0x7fcaac0aade0 2023-11-06T20:44:56.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:56.668+0000 7fcab199d700 1 -- 172.21.15.33:0/686380465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcaac06ad80 con 0x7fcaac0aade0 2023-11-06T20:44:57.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.428+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/686380465 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcaa0029080 con 0x7fcaac0aade0 2023-11-06T20:44:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.430+0000 7fcab199d700 1 -- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca9005f150 msgr2=0x7fca90061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.430+0000 7fcab199d700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca9005f150 0x7fca90061610 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fcaac0ae870 tx=0x7fca98007c40).stop 2023-11-06T20:44:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.430+0000 7fcab199d700 1 -- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaac0aade0 msgr2=0x7fcaac0aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.430+0000 7fcab199d700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaac0aade0 0x7fcaac0aea60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fcaa0009fd0 tx=0x7fcaa0017000).stop 2023-11-06T20:44:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.431+0000 7fcab199d700 1 -- 172.21.15.33:0/686380465 shutdown_connections 2023-11-06T20:44:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.431+0000 7fcab199d700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca9005f150 0x7fca90061610 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.431+0000 7fcab199d700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaac1542c0 0x7fcaac151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.431+0000 7fcab199d700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcaac0aade0 0x7fcaac0aea60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.431+0000 7fcab199d700 1 --2- 172.21.15.33:0/686380465 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcaac0c6260 0x7fcaac0acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.431+0000 7fcab199d700 1 -- 172.21.15.33:0/686380465 >> 172.21.15.33:0/686380465 conn(0x7fcaac0cb760 msgr2=0x7fcaac0ca3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.431+0000 7fcab199d700 1 -- 172.21.15.33:0/686380465 shutdown_connections 2023-11-06T20:44:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.431+0000 7fcab199d700 1 -- 172.21.15.33:0/686380465 wait complete. 2023-11-06T20:44:57.433 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T20:44:57.450 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T20:44:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.648+0000 7ff30edff700 1 Processor -- start 2023-11-06T20:44:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.648+0000 7ff30edff700 1 -- start start 2023-11-06T20:44:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.649+0000 7ff30edff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3080c3600 0x7ff3080c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.649+0000 7ff30edff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3080c17e0 con 0x7ff308136f50 2023-11-06T20:44:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.649+0000 7ff30edff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30806a740 con 0x7ff308069410 2023-11-06T20:44:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.649+0000 7ff30edff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff308069f90 con 0x7ff3080c3600 2023-11-06T20:44:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.650+0000 7ff307fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff308136f50 0x7ff3081373d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36068/0 (socket says 172.21.15.33:36068) 2023-11-06T20:44:57.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.650+0000 7ff307fff700 1 -- 172.21.15.33:0/2971014000 learned_addr learned my addr 172.21.15.33:0/2971014000 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:57.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.650+0000 7ff30cb9b700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3080c3600 0x7ff3080c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.650+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597845914 0 0) 0x7ff3080c17e0 con 0x7ff308136f50 2023-11-06T20:44:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.650+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f80033f0 con 0x7ff308136f50 2023-11-06T20:44:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.650+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4010129160 0 0) 0x7ff2f80033f0 con 0x7ff308136f50 2023-11-06T20:44:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.650+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3080c3600 msgr2=0x7ff3080c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.650+0000 7ff3077fe700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3080c3600 0x7ff3080c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.650+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 >> v1:172.21.15.69:6789/0 conn(0x7ff308069410 legacy=0x7ff308069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:57.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30806a180 con 0x7ff308136f50 2023-11-06T20:44:57.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7ff2f4001470 con 0x7ff308136f50 2023-11-06T20:44:57.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 >> v1:172.21.15.33:6789/0 conn(0x7ff308136f50 legacy=0x7ff3081373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:57.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff3077fe700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f8003910 0x7ff2f8003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff3077fe700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3080c3600 0x7ff2f8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff3077fe700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f8004b80 0x7ff2f8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f800b8e0 con 0x7ff2f8003910 2023-11-06T20:44:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f800ba20 con 0x7ff2f8004b80 2023-11-06T20:44:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f800bb90 con 0x7ff3080c3600 2023-11-06T20:44:57.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff30cb9b700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f8003910 0x7ff2f8003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:57.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff307fff700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3080c3600 0x7ff2f8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:57.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff30d39c700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f8004b80 0x7ff2f8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:57.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff30cb9b700 1 -- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3080c3600 msgr2=0x7ff2f8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:57.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff30cb9b700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3080c3600 0x7ff2f8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff30cb9b700 1 -- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f8004b80 msgr2=0x7ff2f8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff30cb9b700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f8004b80 0x7ff2f8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.651+0000 7ff30cb9b700 1 -- 172.21.15.33:0/2971014000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f80033f0 con 0x7ff2f8003910 2023-11-06T20:44:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.652+0000 7ff30cb9b700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f8003910 0x7ff2f8003cf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7ff2fc009a80 tx=0x7ff2fc009060).ready entity=mon.0 client_cookie=ad86ec75b40339a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.652+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc009430 con 0x7ff2f8003910 2023-11-06T20:44:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.652+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc00e5e0 con 0x7ff2f8003910 2023-11-06T20:44:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.652+0000 7ff3077fe700 1 -- 172.21.15.33:0/2971014000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00e960 con 0x7ff2f8003910 2023-11-06T20:44:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.652+0000 7ff30edff700 1 -- 172.21.15.33:0/2971014000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f8003910 msgr2=0x7ff2f8003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.652+0000 7ff30edff700 1 --2- 172.21.15.33:0/2971014000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f8003910 0x7ff2f8003cf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7ff2fc009a80 tx=0x7ff2fc009060).stop 2023-11-06T20:44:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.652+0000 7ff30d39c700 1 -- 172.21.15.33:0/2971014000 reap_dead start 2023-11-06T20:44:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.652+0000 7ff30edff700 1 -- 172.21.15.33:0/2971014000 shutdown_connections 2023-11-06T20:44:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.652+0000 7ff30edff700 1 -- 172.21.15.33:0/2971014000 >> 172.21.15.33:0/2971014000 conn(0x7ff308062710 msgr2=0x7ff308062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.653+0000 7ff30edff700 1 -- 172.21.15.33:0/2971014000 shutdown_connections 2023-11-06T20:44:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.653+0000 7ff30edff700 1 -- 172.21.15.33:0/2971014000 wait complete. 2023-11-06T20:44:57.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.654+0000 7ff30edff700 1 Processor -- start 2023-11-06T20:44:57.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.654+0000 7ff30edff700 1 -- start start 2023-11-06T20:44:57.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff30edff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f8003910 0x7ff308163050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:57.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff30edff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff308153de0 0x7ff3080c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:57.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff30edff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3081515b0 0x7ff3081519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:57.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff30edff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff308153950 con 0x7ff308153de0 2023-11-06T20:44:57.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff30edff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff308151ee0 con 0x7ff2f8003910 2023-11-06T20:44:57.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff30edff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff308152a60 con 0x7ff3081515b0 2023-11-06T20:44:57.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff307fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff308153de0 0x7ff3080c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:57.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff30cb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f8003910 0x7ff308163050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:57.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff307fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff308153de0 0x7ff3080c0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41172/0 (socket says 172.21.15.33:41172) 2023-11-06T20:44:57.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff30cb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f8003910 0x7ff308163050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52228/0 (socket says 172.21.15.33:52228) 2023-11-06T20:44:57.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff307fff700 1 -- 172.21.15.33:0/1221065066 learned_addr learned my addr 172.21.15.33:0/1221065066 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:57.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.655+0000 7ff30d39c700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3081515b0 0x7ff3081519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:57.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff307fff700 1 -- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3081515b0 msgr2=0x7ff3081519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:57.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff307fff700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3081515b0 0x7ff3081519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff307fff700 1 -- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f8003910 msgr2=0x7ff308163050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:57.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff307fff700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f8003910 0x7ff308163050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff307fff700 1 -- 172.21.15.33:0/1221065066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff308152d40 con 0x7ff308153de0 2023-11-06T20:44:57.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff307fff700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff308153de0 0x7ff3080c0130 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7ff2f40047b0 tx=0x7ff2f4001df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:57.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff304ff9700 1 -- 172.21.15.33:0/1221065066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4011590 con 0x7ff308153de0 2023-11-06T20:44:57.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff304ff9700 1 -- 172.21.15.33:0/1221065066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f40116f0 con 0x7ff308153de0 2023-11-06T20:44:57.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff304ff9700 1 -- 172.21.15.33:0/1221065066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4011a70 con 0x7ff308153de0 2023-11-06T20:44:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff30edff700 1 -- 172.21.15.33:0/1221065066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff308153020 con 0x7ff308153de0 2023-11-06T20:44:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.656+0000 7ff30edff700 1 -- 172.21.15.33:0/1221065066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3081535e0 con 0x7ff308153de0 2023-11-06T20:44:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.658+0000 7ff30edff700 1 -- 172.21.15.33:0/1221065066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3080c0d80 con 0x7ff308153de0 2023-11-06T20:44:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.658+0000 7ff304ff9700 1 -- 172.21.15.33:0/1221065066 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff2f401c360 con 0x7ff308153de0 2023-11-06T20:44:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.660+0000 7ff304ff9700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2ec05f150 0x7ff2ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:57.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.660+0000 7ff304ff9700 1 -- 172.21.15.33:0/1221065066 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff2f4079570 con 0x7ff308153de0 2023-11-06T20:44:57.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.660+0000 7ff30cb9b700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2ec05f150 0x7ff2ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:57.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.661+0000 7ff30cb9b700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2ec05f150 0x7ff2ec061610 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7ff2fc006d50 tx=0x7ff2fc01a000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:57.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.664+0000 7ff304ff9700 1 -- 172.21.15.33:0/1221065066 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff2f4049af0 con 0x7ff308153de0 2023-11-06T20:44:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.912+0000 7ff30edff700 1 -- 172.21.15.33:0/1221065066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff3080c2ce0 con 0x7ff308153de0 2023-11-06T20:44:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.912+0000 7ff304ff9700 1 -- 172.21.15.33:0/1221065066 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff2f4019020 con 0x7ff308153de0 2023-11-06T20:44:57.913 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:44:57.913 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T20:44:57.913 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T20:44:57.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.914 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T20:44:57.914 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T20:44:57.914 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.914 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:44:57.915 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:44:57.915 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:44:57.915 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.916 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T20:44:57.916 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T20:44:57.916 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.917 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:44:57.917 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:44:57.917 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:44:57.917 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.918 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T20:44:57.919 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T20:44:57.919 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.919 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:44:57.919 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:44:57.919 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:44:57.920 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.920 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T20:44:57.921 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T20:44:57.921 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.921 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:44:57.921 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:44:57.921 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:44:57.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.922 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T20:44:57.923 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T20:44:57.923 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.923 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:44:57.923 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:44:57.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.924 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.924 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T20:44:57.924 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T20:44:57.925 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.925 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:44:57.925 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:44:57.925 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.926 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T20:44:57.926 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T20:44:57.927 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.927 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:44:57.927 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:44:57.927 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.928 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T20:44:57.928 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T20:44:57.928 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.929 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:44:57.929 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:44:57.929 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.929 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.930 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T20:44:57.930 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T20:44:57.930 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.930 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:44:57.931 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:44:57.931 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.932 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T20:44:57.932 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T20:44:57.932 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.932 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:44:57.933 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:44:57.933 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.934 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T20:44:57.934 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T20:44:57.934 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.934 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:44:57.935 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:44:57.935 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.936 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T20:44:57.936 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T20:44:57.936 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.936 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:44:57.937 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:44:57.937 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.938 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T20:44:57.938 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T20:44:57.938 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.938 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:44:57.938 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:44:57.939 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:44:57.939 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:44:57.939 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.940 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T20:44:57.940 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T20:44:57.940 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.940 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:44:57.940 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:44:57.941 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:44:57.941 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:44:57.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.942 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T20:44:57.942 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T20:44:57.942 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.942 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:44:57.942 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:44:57.943 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:44:57.943 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:44:57.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.944 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T20:44:57.944 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T20:44:57.944 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.944 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:44:57.944 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:44:57.945 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:44:57.945 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:44:57.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.946 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T20:44:57.946 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T20:44:57.946 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.946 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:44:57.947 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:44:57.947 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:44:57.947 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.948 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-mds", 2023-11-06T20:44:57.948 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQACUEllOJUfOhAA8RDAjqGI2OYPjVDsPZTC0Q==", 2023-11-06T20:44:57.949 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.949 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-mds" 2023-11-06T20:44:57.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.950 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-mgr", 2023-11-06T20:44:57.950 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQACUEllYqAfOhAA4iiHoZHybDj08b+jyRfFrQ==", 2023-11-06T20:44:57.950 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.950 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-06T20:44:57.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.951 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-osd", 2023-11-06T20:44:57.951 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQACUEllFKkfOhAAGzBjYidYc7nuLt7zYj1EiQ==", 2023-11-06T20:44:57.952 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.952 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-osd" 2023-11-06T20:44:57.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.953 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rbd", 2023-11-06T20:44:57.953 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQACUElla7EfOhAA6vDf+asGzV/mkRP2ARRZyQ==", 2023-11-06T20:44:57.953 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.953 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-06T20:44:57.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.954 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-06T20:44:57.955 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQACUEll9rkfOhAArQEyS1Zm+UXm/PgVGCoscQ==", 2023-11-06T20:44:57.955 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.955 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-06T20:44:57.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.956 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rgw", 2023-11-06T20:44:57.956 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQACUEllpcIfOhAA6w+S4cN6/mRuqhPd/Cr2Fg==", 2023-11-06T20:44:57.957 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.957 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-06T20:44:57.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.958 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T20:44:57.958 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T20:44:57.958 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.958 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:44:57.961 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T20:44:57.961 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.961 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:44:57.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:44:57.962 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T20:44:57.962 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T20:44:57.963 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:44:57.963 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:44:57.963 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T20:44:57.963 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:44:57.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:44:57.964 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T20:44:57.964 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T20:44:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.914+0000 7ff30edff700 1 -- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2ec05f150 msgr2=0x7ff2ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.914+0000 7ff30edff700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2ec05f150 0x7ff2ec061610 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7ff2fc006d50 tx=0x7ff2fc01a000).stop 2023-11-06T20:44:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.914+0000 7ff30edff700 1 -- 172.21.15.33:0/1221065066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff308153de0 msgr2=0x7ff3080c0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.914+0000 7ff30edff700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff308153de0 0x7ff3080c0130 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7ff2f40047b0 tx=0x7ff2f4001df0).stop 2023-11-06T20:44:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.914+0000 7ff30edff700 1 -- 172.21.15.33:0/1221065066 shutdown_connections 2023-11-06T20:44:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.914+0000 7ff30edff700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2ec05f150 0x7ff2ec061610 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.914+0000 7ff30edff700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3081515b0 0x7ff3081519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.914+0000 7ff30edff700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2f8003910 0x7ff308163050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.914+0000 7ff30edff700 1 --2- 172.21.15.33:0/1221065066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff308153de0 0x7ff3080c0130 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:57.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.914+0000 7ff30edff700 1 -- 172.21.15.33:0/1221065066 >> 172.21.15.33:0/1221065066 conn(0x7ff308062710 msgr2=0x7ff308062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:57.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.915+0000 7ff30edff700 1 -- 172.21.15.33:0/1221065066 shutdown_connections 2023-11-06T20:44:57.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:57.915+0000 7ff30edff700 1 -- 172.21.15.33:0/1221065066 wait complete. 2023-11-06T20:44:57.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-06T20:44:58.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.167+0000 7f46875ff700 1 Processor -- start 2023-11-06T20:44:58.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.167+0000 7f46875ff700 1 -- start start 2023-11-06T20:44:58.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.169+0000 7f46875ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46800c0470 0x7f46800c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.169+0000 7f46875ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46800c7700 con 0x7f4680154210 2023-11-06T20:44:58.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.169+0000 7f46875ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468005efc0 con 0x7f46800c6d90 2023-11-06T20:44:58.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.169+0000 7f46875ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800c8ea0 con 0x7f46800c0470 2023-11-06T20:44:58.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.169+0000 7f468539b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4680154210 0x7f46800a7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34038/0 (socket says 172.21.15.33:34038) 2023-11-06T20:44:58.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.169+0000 7f468539b700 1 -- 172.21.15.33:0/3613808847 learned_addr learned my addr 172.21.15.33:0/3613808847 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:58.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.169+0000 7f4685b9c700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46800c0470 0x7f46800c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.169+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519905760 0 0) 0x7f467c001110 con 0x7f4680154210 2023-11-06T20:44:58.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f466c0033f0 con 0x7f4680154210 2023-11-06T20:44:58.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1899307904 0 0) 0x7f466c0033f0 con 0x7f4680154210 2023-11-06T20:44:58.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46800c0470 msgr2=0x7f46800c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:58.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46800c0470 0x7f46800c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 >> v1:172.21.15.69:6789/0 conn(0x7f46800c6d90 legacy=0x7f46800bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800c9140 con 0x7f4680154210 2023-11-06T20:44:58.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f467c001600 con 0x7f4680154210 2023-11-06T20:44:58.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 >> v1:172.21.15.33:6789/0 conn(0x7f4680154210 legacy=0x7f46800a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 0x7f466c003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46800c0470 0x7f466c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f466c004b80 0x7f466c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800c8ea0 con 0x7f466c003910 2023-11-06T20:44:58.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c00b8e0 con 0x7f466c004b80 2023-11-06T20:44:58.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c00ba70 con 0x7f46800c0470 2023-11-06T20:44:58.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f468539b700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 0x7f466c003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4685b9c700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f466c004b80 0x7f466c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.170+0000 7f4684b9a700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46800c0470 0x7f466c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.171+0000 7f468539b700 1 -- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46800c0470 msgr2=0x7f466c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:58.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.171+0000 7f468539b700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46800c0470 0x7f466c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.171+0000 7f468539b700 1 -- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f466c004b80 msgr2=0x7f466c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:58.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.171+0000 7f468539b700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f466c004b80 0x7f466c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.171+0000 7f468539b700 1 -- 172.21.15.33:0/3613808847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0033f0 con 0x7f466c003910 2023-11-06T20:44:58.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.171+0000 7f4684b9a700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46800c0470 0x7f466c004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:58.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.171+0000 7f468539b700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 0x7f466c003cf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f467c00c2d0 tx=0x7f467c007850).ready entity=mon.0 client_cookie=d35c291e41b1e79c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:58.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.171+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c007b60 con 0x7f466c003910 2023-11-06T20:44:58.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.171+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c00cc00 con 0x7f466c003910 2023-11-06T20:44:58.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.171+0000 7f4677fff700 1 -- 172.21.15.33:0/3613808847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0070d0 con 0x7f466c003910 2023-11-06T20:44:58.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.172+0000 7f46875ff700 1 -- 172.21.15.33:0/3613808847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 msgr2=0x7f466c003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.172+0000 7f46875ff700 1 --2- 172.21.15.33:0/3613808847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 0x7f466c003cf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f467c00c2d0 tx=0x7f467c007850).stop 2023-11-06T20:44:58.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.172+0000 7f4685b9c700 1 -- 172.21.15.33:0/3613808847 reap_dead start 2023-11-06T20:44:58.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.172+0000 7f46875ff700 1 -- 172.21.15.33:0/3613808847 shutdown_connections 2023-11-06T20:44:58.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.172+0000 7f46875ff700 1 -- 172.21.15.33:0/3613808847 >> 172.21.15.33:0/3613808847 conn(0x7f468005e990 msgr2=0x7f46800cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:58.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.172+0000 7f46875ff700 1 -- 172.21.15.33:0/3613808847 shutdown_connections 2023-11-06T20:44:58.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.172+0000 7f46875ff700 1 -- 172.21.15.33:0/3613808847 wait complete. 2023-11-06T20:44:58.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.173+0000 7f46875ff700 1 Processor -- start 2023-11-06T20:44:58.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.173+0000 7f46875ff700 1 -- start start 2023-11-06T20:44:58.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.174+0000 7f46875ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 0x7f4680098930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.174+0000 7f46875ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46801545f0 0x7f46801653d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.174+0000 7f46875ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4680155f40 0x7f46801563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.174+0000 7f46875ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4680150460 con 0x7f466c003910 2023-11-06T20:44:58.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.174+0000 7f46875ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4680155750 con 0x7f4680155f40 2023-11-06T20:44:58.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.174+0000 7f46875ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46801568b0 con 0x7f46801545f0 2023-11-06T20:44:58.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f468539b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 0x7f4680098930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f468539b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 0x7f4680098930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42082/0 (socket says 172.21.15.33:42082) 2023-11-06T20:44:58.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f468539b700 1 -- 172.21.15.33:0/2360108978 learned_addr learned my addr 172.21.15.33:0/2360108978 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:58.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f4684b9a700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46801545f0 0x7f46801653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f4685b9c700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4680155f40 0x7f46801563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f468539b700 1 -- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46801545f0 msgr2=0x7f46801653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f468539b700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46801545f0 0x7f46801653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f468539b700 1 -- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4680155f40 msgr2=0x7f46801563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f468539b700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4680155f40 0x7f46801563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f468539b700 1 -- 172.21.15.33:0/2360108978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46801658c0 con 0x7f466c003910 2023-11-06T20:44:58.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f468539b700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 0x7f4680098930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f467c00c8c0 tx=0x7f467c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:58.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f46757fa700 1 -- 172.21.15.33:0/2360108978 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c007440 con 0x7f466c003910 2023-11-06T20:44:58.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f46757fa700 1 -- 172.21.15.33:0/2360108978 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c0075a0 con 0x7f466c003910 2023-11-06T20:44:58.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f46875ff700 1 -- 172.21.15.33:0/2360108978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4680155be0 con 0x7f466c003910 2023-11-06T20:44:58.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.175+0000 7f46757fa700 1 -- 172.21.15.33:0/2360108978 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c004900 con 0x7f466c003910 2023-11-06T20:44:58.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.176+0000 7f46875ff700 1 -- 172.21.15.33:0/2360108978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4680154e20 con 0x7f466c003910 2023-11-06T20:44:58.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.177+0000 7f46875ff700 1 -- 172.21.15.33:0/2360108978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4680155a30 con 0x7f466c003910 2023-11-06T20:44:58.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.177+0000 7f46757fa700 1 -- 172.21.15.33:0/2360108978 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f467c004cb0 con 0x7f466c003910 2023-11-06T20:44:58.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.178+0000 7f46757fa700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f466405f150 0x7f4664061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.179+0000 7f46757fa700 1 -- 172.21.15.33:0/2360108978 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f467c07ab50 con 0x7f466c003910 2023-11-06T20:44:58.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.179+0000 7f4684b9a700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f466405f150 0x7f4664061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.179+0000 7f4684b9a700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f466405f150 0x7f4664061610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f46800c4f40 tx=0x7f4670009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:58.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.182+0000 7f46757fa700 1 -- 172.21.15.33:0/2360108978 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f467c04c0b0 con 0x7f466c003910 2023-11-06T20:44:58.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.428+0000 7f46875ff700 1 -- 172.21.15.33:0/2360108978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f46800c7700 con 0x7f466c003910 2023-11-06T20:44:58.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.432+0000 7f46757fa700 1 -- 172.21.15.33:0/2360108978 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v3) v1 ==== 97+0+0 (secure 0 0 0) 0x7f46800c7700 con 0x7f466c003910 2023-11-06T20:44:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.434+0000 7f46875ff700 1 -- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f466405f150 msgr2=0x7f4664061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.434+0000 7f46875ff700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f466405f150 0x7f4664061610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f46800c4f40 tx=0x7f4670009040).stop 2023-11-06T20:44:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.434+0000 7f46875ff700 1 -- 172.21.15.33:0/2360108978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 msgr2=0x7f4680098930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.434+0000 7f46875ff700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 0x7f4680098930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f467c00c8c0 tx=0x7f467c015000).stop 2023-11-06T20:44:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.435+0000 7f46875ff700 1 -- 172.21.15.33:0/2360108978 shutdown_connections 2023-11-06T20:44:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.435+0000 7f46875ff700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f466405f150 0x7f4664061610 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.435+0000 7f46875ff700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4680155f40 0x7f46801563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.435+0000 7f46875ff700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f466c003910 0x7f4680098930 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.435+0000 7f46875ff700 1 --2- 172.21.15.33:0/2360108978 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f46801545f0 0x7f46801653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.435+0000 7f46875ff700 1 -- 172.21.15.33:0/2360108978 >> 172.21.15.33:0/2360108978 conn(0x7f468005e990 msgr2=0x7f4680153190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:58.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.435+0000 7f46875ff700 1 -- 172.21.15.33:0/2360108978 shutdown_connections 2023-11-06T20:44:58.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.435+0000 7f46875ff700 1 -- 172.21.15.33:0/2360108978 wait complete. 2023-11-06T20:44:58.438 INFO:teuthology.orchestra.run.smithi033.stderr:updated 2023-11-06T20:44:58.452 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-06T20:44:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.648+0000 7fbf2a5ff700 1 Processor -- start 2023-11-06T20:44:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.649+0000 7fbf2a5ff700 1 -- start start 2023-11-06T20:44:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.650+0000 7fbf2a5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbf2414e6a0 0x7fbf24066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.650+0000 7fbf2a5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240640c0 con 0x7fbf241583b0 2023-11-06T20:44:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.650+0000 7fbf2a5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240c5a10 con 0x7fbf240c5ea0 2023-11-06T20:44:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.650+0000 7fbf2a5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf24064d10 con 0x7fbf2414e6a0 2023-11-06T20:44:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.650+0000 7fbf28b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbf241583b0 0x7fbf240c3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34052/0 (socket says 172.21.15.33:34052) 2023-11-06T20:44:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.651+0000 7fbf28b9c700 1 -- 172.21.15.33:0/1075448387 learned_addr learned my addr 172.21.15.33:0/1075448387 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.651+0000 7fbf23fff700 1 --2- 172.21.15.33:0/1075448387 >> v2:172.21.15.69:3300/0 conn(0x7fbf2414e6a0 0x7fbf24066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.651+0000 7fbf22ffd700 1 -- 172.21.15.33:0/1075448387 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364715343 0 0) 0x7fbf1c001490 con 0x7fbf241583b0 2023-11-06T20:44:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.651+0000 7fbf22ffd700 1 -- 172.21.15.33:0/1075448387 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0c0033f0 con 0x7fbf241583b0 2023-11-06T20:44:58.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.651+0000 7fbf22ffd700 1 -- 172.21.15.33:0/1075448387 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706235021 0 0) 0x7fbf240c5a10 con 0x7fbf240c5ea0 2023-11-06T20:44:58.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.651+0000 7fbf22ffd700 1 -- 172.21.15.33:0/1075448387 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf1c001490 con 0x7fbf240c5ea0 2023-11-06T20:44:58.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.651+0000 7fbf23fff700 1 -- 172.21.15.33:0/1075448387 >> v1:172.21.15.69:6789/0 conn(0x7fbf240c5ea0 legacy=0x7fbf24157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.651+0000 7fbf23fff700 1 -- 172.21.15.33:0/1075448387 >> v1:172.21.15.33:6789/0 conn(0x7fbf241583b0 legacy=0x7fbf240c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.651+0000 7fbf23fff700 1 -- 172.21.15.33:0/1075448387 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24063820 con 0x7fbf2414e6a0 2023-11-06T20:44:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.652+0000 7fbf23fff700 1 --2- 172.21.15.33:0/1075448387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf2414e6a0 0x7fbf24066e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fbf14009760 tx=0x7fbf14009060).ready entity=mon.1 client_cookie=78d986be3ed632c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.652+0000 7fbf22ffd700 1 -- 172.21.15.33:0/1075448387 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400d4a0 con 0x7fbf2414e6a0 2023-11-06T20:44:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.652+0000 7fbf22ffd700 1 -- 172.21.15.33:0/1075448387 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1400d600 con 0x7fbf2414e6a0 2023-11-06T20:44:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.652+0000 7fbf22ffd700 1 -- 172.21.15.33:0/1075448387 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400d980 con 0x7fbf2414e6a0 2023-11-06T20:44:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.652+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/1075448387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf2414e6a0 msgr2=0x7fbf24066e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.652+0000 7fbf2a5ff700 1 --2- 172.21.15.33:0/1075448387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf2414e6a0 0x7fbf24066e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fbf14009760 tx=0x7fbf14009060).stop 2023-11-06T20:44:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.652+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/1075448387 shutdown_connections 2023-11-06T20:44:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.652+0000 7fbf2a5ff700 1 --2- 172.21.15.33:0/1075448387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf2414e6a0 0x7fbf24066e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.652+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/1075448387 >> 172.21.15.33:0/1075448387 conn(0x7fbf2405e1b0 msgr2=0x7fbf2405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.652+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/1075448387 shutdown_connections 2023-11-06T20:44:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.653+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/1075448387 wait complete. 2023-11-06T20:44:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.654+0000 7fbf2a5ff700 1 Processor -- start 2023-11-06T20:44:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.654+0000 7fbf2a5ff700 1 -- start start 2023-11-06T20:44:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.655+0000 7fbf2a5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf240c5ea0 0x7fbf2413c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.655+0000 7fbf2a5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf2414e6a0 0x7fbf240bfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.655+0000 7fbf2a5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbf241583b0 0x7fbf240abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.655+0000 7fbf2a5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240c12a0 con 0x7fbf240c5ea0 2023-11-06T20:44:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.655+0000 7fbf2a5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240c0690 con 0x7fbf2414e6a0 2023-11-06T20:44:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.655+0000 7fbf2a5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240c0970 con 0x7fbf241583b0 2023-11-06T20:44:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.655+0000 7fbf23fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf240c5ea0 0x7fbf2413c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.655+0000 7fbf23fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf240c5ea0 0x7fbf2413c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42096/0 (socket says 172.21.15.33:42096) 2023-11-06T20:44:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.655+0000 7fbf23fff700 1 -- 172.21.15.33:0/445943156 learned_addr learned my addr 172.21.15.33:0/445943156 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.655+0000 7fbf237fe700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf2414e6a0 0x7fbf240bfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf28b9c700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbf241583b0 0x7fbf240abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf23fff700 1 -- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbf241583b0 msgr2=0x7fbf240abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf23fff700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbf241583b0 0x7fbf240abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf23fff700 1 -- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf2414e6a0 msgr2=0x7fbf240bfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf23fff700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf2414e6a0 0x7fbf240bfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf23fff700 1 -- 172.21.15.33:0/445943156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf140092a0 con 0x7fbf240c5ea0 2023-11-06T20:44:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf237fe700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf2414e6a0 0x7fbf240bfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:44:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf23fff700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf240c5ea0 0x7fbf2413c870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fbf14009630 tx=0x7fbf14016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf0bfff700 1 -- 172.21.15.33:0/445943156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400d770 con 0x7fbf240c5ea0 2023-11-06T20:44:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf0bfff700 1 -- 172.21.15.33:0/445943156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1400d8d0 con 0x7fbf240c5ea0 2023-11-06T20:44:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/445943156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf240c0c50 con 0x7fbf240c5ea0 2023-11-06T20:44:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf0bfff700 1 -- 172.21.15.33:0/445943156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1401d660 con 0x7fbf240c5ea0 2023-11-06T20:44:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.656+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/445943156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2413cd60 con 0x7fbf240c5ea0 2023-11-06T20:44:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.657+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/445943156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf24063b00 con 0x7fbf240c5ea0 2023-11-06T20:44:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.658+0000 7fbf0bfff700 1 -- 172.21.15.33:0/445943156 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fbf140263b0 con 0x7fbf240c5ea0 2023-11-06T20:44:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.659+0000 7fbf0bfff700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf1005f150 0x7fbf10061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.659+0000 7fbf0bfff700 1 -- 172.21.15.33:0/445943156 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fbf1407aec0 con 0x7fbf240c5ea0 2023-11-06T20:44:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.659+0000 7fbf237fe700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf1005f150 0x7fbf10061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.660+0000 7fbf237fe700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf1005f150 0x7fbf10061610 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fbf18007fd0 tx=0x7fbf18009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.663+0000 7fbf0bfff700 1 -- 172.21.15.33:0/445943156 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbf1404c520 con 0x7fbf240c5ea0 2023-11-06T20:44:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.908+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/445943156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fbf2405eda0 con 0x7fbf240c5ea0 2023-11-06T20:44:58.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.911+0000 7fbf0bfff700 1 -- 172.21.15.33:0/445943156 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v4) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbf14022070 con 0x7fbf240c5ea0 2023-11-06T20:44:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.913+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf1005f150 msgr2=0x7fbf10061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.913+0000 7fbf2a5ff700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf1005f150 0x7fbf10061610 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fbf18007fd0 tx=0x7fbf18009040).stop 2023-11-06T20:44:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.913+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/445943156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf240c5ea0 msgr2=0x7fbf2413c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.913+0000 7fbf2a5ff700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf240c5ea0 0x7fbf2413c870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fbf14009630 tx=0x7fbf14016000).stop 2023-11-06T20:44:58.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.914+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/445943156 shutdown_connections 2023-11-06T20:44:58.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.914+0000 7fbf2a5ff700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf1005f150 0x7fbf10061610 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.914+0000 7fbf2a5ff700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbf241583b0 0x7fbf240abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.914+0000 7fbf2a5ff700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf240c5ea0 0x7fbf2413c870 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.914+0000 7fbf2a5ff700 1 --2- 172.21.15.33:0/445943156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf2414e6a0 0x7fbf240bfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.914+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/445943156 >> 172.21.15.33:0/445943156 conn(0x7fbf2405e1b0 msgr2=0x7fbf2413d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.914+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/445943156 shutdown_connections 2023-11-06T20:44:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:58.914+0000 7fbf2a5ff700 1 -- 172.21.15.33:0/445943156 wait complete. 2023-11-06T20:44:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:updated 2023-11-06T20:44:58.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-06T20:44:59.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.131+0000 7f82fc7ff700 1 Processor -- start 2023-11-06T20:44:59.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.131+0000 7f82fc7ff700 1 -- start start 2023-11-06T20:44:59.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.132+0000 7f82fc7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f82f413d230 0x7f82f41586f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.132+0000 7f82fc7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f4062840 con 0x7f82f413ce50 2023-11-06T20:44:59.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.132+0000 7f82fc7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f4062b20 con 0x7f82f40c4c70 2023-11-06T20:44:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.132+0000 7f82fc7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f4158d50 con 0x7f82f413d230 2023-11-06T20:44:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82fa59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f82f413ce50 0x7f82f40c4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34062/0 (socket says 172.21.15.33:34062) 2023-11-06T20:44:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82fa59b700 1 -- 172.21.15.33:0/4010861311 learned_addr learned my addr 172.21.15.33:0/4010861311 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82fad9c700 1 --2- 172.21.15.33:0/4010861311 >> v2:172.21.15.69:3300/0 conn(0x7f82f413d230 0x7f82f41586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718180081 0 0) 0x7f82f4062840 con 0x7f82f413ce50 2023-11-06T20:44:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e00033f0 con 0x7f82f413ce50 2023-11-06T20:44:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3456362741 0 0) 0x7f82e00033f0 con 0x7f82f413ce50 2023-11-06T20:44:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 >> v2:172.21.15.69:3300/0 conn(0x7f82f413d230 msgr2=0x7f82f41586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82f9599700 1 --2- 172.21.15.33:0/4010861311 >> v2:172.21.15.69:3300/0 conn(0x7f82f413d230 0x7f82f41586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 >> v1:172.21.15.69:6789/0 conn(0x7f82f40c4c70 legacy=0x7f82f4157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4062e00 con 0x7f82f413ce50 2023-11-06T20:44:59.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.133+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f82f0001470 con 0x7f82f413ce50 2023-11-06T20:44:59.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 >> v1:172.21.15.33:6789/0 conn(0x7f82f413ce50 legacy=0x7f82f40c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82f9599700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f82e00038e0 0x7f82e0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82f9599700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82e0004690 0x7f82e0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82f9599700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82e0005000 0x7f82e000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f4158d50 con 0x7f82e0005000 2023-11-06T20:44:59.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e000bf60 con 0x7f82e0004690 2023-11-06T20:44:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e000c0f0 con 0x7f82e00038e0 2023-11-06T20:44:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82fad9c700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82e0005000 0x7f82e000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82f9d9a700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82e0004690 0x7f82e0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82fa59b700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f82e00038e0 0x7f82e0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82fad9c700 1 -- 172.21.15.33:0/4010861311 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f82e00038e0 msgr2=0x7f82e0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82fad9c700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f82e00038e0 0x7f82e0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82fad9c700 1 -- 172.21.15.33:0/4010861311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82e0004690 msgr2=0x7f82e0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82fad9c700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82e0004690 0x7f82e0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.134+0000 7f82fad9c700 1 -- 172.21.15.33:0/4010861311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00033f0 con 0x7f82e0005000 2023-11-06T20:44:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82fad9c700 1 -- 172.21.15.33:0/4010861311 reap_dead start 2023-11-06T20:44:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82fad9c700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82e0005000 0x7f82e000b8e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f82ec00e210 tx=0x7f82ec00b040).ready entity=mon.0 client_cookie=52049a8210668a4e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec009ac0 con 0x7f82e0005000 2023-11-06T20:44:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec00f040 con 0x7f82e0005000 2023-11-06T20:44:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82f9599700 1 -- 172.21.15.33:0/4010861311 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00ecd0 con 0x7f82e0005000 2023-11-06T20:44:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82fc7ff700 1 -- 172.21.15.33:0/4010861311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82e0005000 msgr2=0x7f82e000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82fc7ff700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82e0005000 0x7f82e000b8e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f82ec00e210 tx=0x7f82ec00b040).stop 2023-11-06T20:44:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82fc7ff700 1 -- 172.21.15.33:0/4010861311 shutdown_connections 2023-11-06T20:44:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82fc7ff700 1 --2- 172.21.15.33:0/4010861311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82e0005000 0x7f82e000b8e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82fc7ff700 1 -- 172.21.15.33:0/4010861311 >> 172.21.15.33:0/4010861311 conn(0x7f82f40c7dc0 msgr2=0x7f82f40c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.135+0000 7f82fc7ff700 1 -- 172.21.15.33:0/4010861311 shutdown_connections 2023-11-06T20:44:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.136+0000 7f82fc7ff700 1 -- 172.21.15.33:0/4010861311 wait complete. 2023-11-06T20:44:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.137+0000 7f82fc7ff700 1 Processor -- start 2023-11-06T20:44:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.137+0000 7f82fc7ff700 1 -- start start 2023-11-06T20:44:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82fc7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f82e0005000 0x7f82f4132160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82fc7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f4135060 0x7f82f41354e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82fc7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82f41397b0 0x7f82f4139c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82fc7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f4132650 con 0x7f82f41397b0 2023-11-06T20:44:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82fc7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f4134ec0 con 0x7f82f4135060 2023-11-06T20:44:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82fc7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f413a120 con 0x7f82e0005000 2023-11-06T20:44:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82fad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82f41397b0 0x7f82f4139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82fad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82f41397b0 0x7f82f4139c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42120/0 (socket says 172.21.15.33:42120) 2023-11-06T20:44:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82fad9c700 1 -- 172.21.15.33:0/2336597957 learned_addr learned my addr 172.21.15.33:0/2336597957 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82f9d9a700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f4135060 0x7f82f41354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.138+0000 7f82fa59b700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f82e0005000 0x7f82f4132160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82fad9c700 1 -- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f82e0005000 msgr2=0x7f82f4132160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82fad9c700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f82e0005000 0x7f82f4132160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82fad9c700 1 -- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f4135060 msgr2=0x7f82f41354e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82fad9c700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f4135060 0x7f82f41354e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82fad9c700 1 -- 172.21.15.33:0/2336597957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40aa6a0 con 0x7f82f41397b0 2023-11-06T20:44:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82fad9c700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82f41397b0 0x7f82f4139c30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f82ec00afd0 tx=0x7f82ec018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82ea7fc700 1 -- 172.21.15.33:0/2336597957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec009ac0 con 0x7f82f41397b0 2023-11-06T20:44:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82ea7fc700 1 -- 172.21.15.33:0/2336597957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec01e470 con 0x7f82f41397b0 2023-11-06T20:44:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82fc7ff700 1 -- 172.21.15.33:0/2336597957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f40c1490 con 0x7f82f41397b0 2023-11-06T20:44:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82ea7fc700 1 -- 172.21.15.33:0/2336597957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec01e810 con 0x7f82f41397b0 2023-11-06T20:44:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.139+0000 7f82fc7ff700 1 -- 172.21.15.33:0/2336597957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f4134900 con 0x7f82f41397b0 2023-11-06T20:44:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.141+0000 7f82fc7ff700 1 -- 172.21.15.33:0/2336597957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f40aa9d0 con 0x7f82f41397b0 2023-11-06T20:44:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.141+0000 7f82ea7fc700 1 -- 172.21.15.33:0/2336597957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f82ec003020 con 0x7f82f41397b0 2023-11-06T20:44:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.142+0000 7f82ea7fc700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f82d805f150 0x7f82d8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.142+0000 7f82ea7fc700 1 -- 172.21.15.33:0/2336597957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f82ec07bc50 con 0x7f82f41397b0 2023-11-06T20:44:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.143+0000 7f82fa59b700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f82d805f150 0x7f82d8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.144+0000 7f82fa59b700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f82d805f150 0x7f82d8061610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f82f0001050 tx=0x7f82f0001e50).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.147+0000 7f82ea7fc700 1 -- 172.21.15.33:0/2336597957 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f82ec04d270 con 0x7f82f41397b0 2023-11-06T20:44:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.399+0000 7f82fc7ff700 1 -- 172.21.15.33:0/2336597957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f82f40aa500 con 0x7f82f41397b0 2023-11-06T20:44:59.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.403+0000 7f82ea7fc700 1 -- 172.21.15.33:0/2336597957 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v5) v1 ==== 97+0+0 (secure 0 0 0) 0x7f82f40aa500 con 0x7f82f41397b0 2023-11-06T20:44:59.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 -- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f82d805f150 msgr2=0x7f82d8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f82d805f150 0x7f82d8061610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f82f0001050 tx=0x7f82f0001e50).stop 2023-11-06T20:44:59.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 -- 172.21.15.33:0/2336597957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82f41397b0 msgr2=0x7f82f4139c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82f41397b0 0x7f82f4139c30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f82ec00afd0 tx=0x7f82ec018000).stop 2023-11-06T20:44:59.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 -- 172.21.15.33:0/2336597957 shutdown_connections 2023-11-06T20:44:59.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f82d805f150 0x7f82d8061610 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f82e0005000 0x7f82f4132160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82f41397b0 0x7f82f4139c30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 --2- 172.21.15.33:0/2336597957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f4135060 0x7f82f41354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 -- 172.21.15.33:0/2336597957 >> 172.21.15.33:0/2336597957 conn(0x7f82f40c7dc0 msgr2=0x7f82f4138140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:59.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 -- 172.21.15.33:0/2336597957 shutdown_connections 2023-11-06T20:44:59.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.405+0000 7f82fc7ff700 1 -- 172.21.15.33:0/2336597957 wait complete. 2023-11-06T20:44:59.408 INFO:teuthology.orchestra.run.smithi033.stderr:updated 2023-11-06T20:44:59.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-06T20:44:59.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.619+0000 7fa99d1ff700 1 Processor -- start 2023-11-06T20:44:59.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.619+0000 7fa99d1ff700 1 -- start start 2023-11-06T20:44:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.621+0000 7fa99d1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa9980be2c0 0x7fa998068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.621+0000 7fa99d1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa998068100 con 0x7fa998152310 2023-11-06T20:44:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.621+0000 7fa99d1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9980658c0 con 0x7fa998151f30 2023-11-06T20:44:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.621+0000 7fa99d1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa998068a80 con 0x7fa9980be2c0 2023-11-06T20:44:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.621+0000 7fa99759e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa998152310 0x7fa998154b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34072/0 (socket says 172.21.15.33:34072) 2023-11-06T20:44:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.621+0000 7fa99759e700 1 -- 172.21.15.33:0/3437844054 learned_addr learned my addr 172.21.15.33:0/3437844054 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:44:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.621+0000 7fa99659c700 1 --2- 172.21.15.33:0/3437844054 >> v2:172.21.15.69:3300/0 conn(0x7fa9980be2c0 0x7fa998068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003998570 0 0) 0x7fa998068100 con 0x7fa998152310 2023-11-06T20:44:59.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9800033f0 con 0x7fa998152310 2023-11-06T20:44:59.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256796368 0 0) 0x7fa9980658c0 con 0x7fa998151f30 2023-11-06T20:44:59.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa998068100 con 0x7fa998151f30 2023-11-06T20:44:59.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805176383 0 0) 0x7fa9800033f0 con 0x7fa998152310 2023-11-06T20:44:59.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 >> v2:172.21.15.69:3300/0 conn(0x7fa9980be2c0 msgr2=0x7fa998068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:59.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 --2- 172.21.15.33:0/3437844054 >> v2:172.21.15.69:3300/0 conn(0x7fa9980be2c0 0x7fa998068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 >> v1:172.21.15.69:6789/0 conn(0x7fa998151f30 legacy=0x7fa9980bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa998155750 con 0x7fa998152310 2023-11-06T20:44:59.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fa9900017f0 con 0x7fa998152310 2023-11-06T20:44:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 >> v1:172.21.15.33:6789/0 conn(0x7fa998152310 legacy=0x7fa998154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa980003a70 0x7fa980003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa980004690 0x7fa980004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980005000 0x7fa98000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa998068a80 con 0x7fa980005000 2023-11-06T20:44:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98000bf60 con 0x7fa980004690 2023-11-06T20:44:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98000c0d0 con 0x7fa980003a70 2023-11-06T20:44:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa99759e700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980005000 0x7fa98000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.622+0000 7fa99659c700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa980004690 0x7fa980004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa996d9d700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa980003a70 0x7fa980003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa99759e700 1 -- 172.21.15.33:0/3437844054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa980003a70 msgr2=0x7fa980003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa99759e700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa980003a70 0x7fa980003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa99759e700 1 -- 172.21.15.33:0/3437844054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa980004690 msgr2=0x7fa980004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:44:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa99759e700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa980004690 0x7fa980004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa99759e700 1 -- 172.21.15.33:0/3437844054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800033f0 con 0x7fa980005000 2023-11-06T20:44:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa99659c700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa980004690 0x7fa980004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:44:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa99759e700 1 -- 172.21.15.33:0/3437844054 reap_dead start 2023-11-06T20:44:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa99759e700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980005000 0x7fa98000b8e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fa99000c5c0 tx=0x7fa990007f40).ready entity=mon.0 client_cookie=7c9e65e993f1fbb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99000cdc0 con 0x7fa980005000 2023-11-06T20:44:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa990005b20 con 0x7fa980005000 2023-11-06T20:44:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.623+0000 7fa995d9b700 1 -- 172.21.15.33:0/3437844054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9900070d0 con 0x7fa980005000 2023-11-06T20:44:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.624+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3437844054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980005000 msgr2=0x7fa98000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.624+0000 7fa99d1ff700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980005000 0x7fa98000b8e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fa99000c5c0 tx=0x7fa990007f40).stop 2023-11-06T20:44:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.624+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3437844054 shutdown_connections 2023-11-06T20:44:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.624+0000 7fa99d1ff700 1 --2- 172.21.15.33:0/3437844054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980005000 0x7fa98000b8e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.624+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3437844054 >> 172.21.15.33:0/3437844054 conn(0x7fa9980c56f0 msgr2=0x7fa9980c5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.624+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3437844054 shutdown_connections 2023-11-06T20:44:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.624+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3437844054 wait complete. 2023-11-06T20:44:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.625+0000 7fa99d1ff700 1 Processor -- start 2023-11-06T20:44:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.625+0000 7fa99d1ff700 1 -- start start 2023-11-06T20:44:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.626+0000 7fa99d1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa980005000 0x7fa998150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.626+0000 7fa99d1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9980ad8e0 0x7fa998150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.626+0000 7fa99d1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa998150d80 0x7fa998159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.626+0000 7fa99d1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9980ad450 con 0x7fa9980ad8e0 2023-11-06T20:44:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.626+0000 7fa99d1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa998157050 con 0x7fa998150d80 2023-11-06T20:44:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.626+0000 7fa99d1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9980ac5b0 con 0x7fa980005000 2023-11-06T20:44:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9980ad8e0 0x7fa998150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9980ad8e0 0x7fa998150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42148/0 (socket says 172.21.15.33:42148) 2023-11-06T20:44:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99659c700 1 -- 172.21.15.33:0/3747078465 learned_addr learned my addr 172.21.15.33:0/3747078465 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:44:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa996d9d700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa980005000 0x7fa998150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99759e700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa998150d80 0x7fa998159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99659c700 1 -- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa980005000 msgr2=0x7fa998150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99659c700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa980005000 0x7fa998150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99659c700 1 -- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa998150d80 msgr2=0x7fa998159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99659c700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa998150d80 0x7fa998159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99659c700 1 -- 172.21.15.33:0/3747078465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9980ac840 con 0x7fa9980ad8e0 2023-11-06T20:44:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99659c700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9980ad8e0 0x7fa998150890 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fa98c011af0 tx=0x7fa98c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa97effd700 1 -- 172.21.15.33:0/3747078465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00fa10 con 0x7fa9980ad8e0 2023-11-06T20:44:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa97effd700 1 -- 172.21.15.33:0/3747078465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c00fb70 con 0x7fa9980ad8e0 2023-11-06T20:44:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa97effd700 1 -- 172.21.15.33:0/3747078465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c014440 con 0x7fa9980ad8e0 2023-11-06T20:44:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.627+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3747078465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9980acb20 con 0x7fa9980ad8e0 2023-11-06T20:44:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.628+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3747078465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9980ad0e0 con 0x7fa9980ad8e0 2023-11-06T20:44:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.629+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3747078465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa998066000 con 0x7fa9980ad8e0 2023-11-06T20:44:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.630+0000 7fa97effd700 1 -- 172.21.15.33:0/3747078465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa98c014830 con 0x7fa9980ad8e0 2023-11-06T20:44:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.631+0000 7fa97effd700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa98405f150 0x7fa984061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:44:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.631+0000 7fa97effd700 1 -- 172.21.15.33:0/3747078465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fa98c07a7d0 con 0x7fa9980ad8e0 2023-11-06T20:44:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.631+0000 7fa996d9d700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa98405f150 0x7fa984061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:44:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.632+0000 7fa996d9d700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa98405f150 0x7fa984061610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fa988007fd0 tx=0x7fa988009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:44:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.635+0000 7fa97effd700 1 -- 172.21.15.33:0/3747078465 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa98c04bd30 con 0x7fa9980ad8e0 2023-11-06T20:44:59.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.883+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3747078465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fa998155a30 con 0x7fa9980ad8e0 2023-11-06T20:44:59.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.887+0000 7fa97effd700 1 -- 172.21.15.33:0/3747078465 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v6) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa98c04f350 con 0x7fa9980ad8e0 2023-11-06T20:44:59.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa98405f150 msgr2=0x7fa984061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa98405f150 0x7fa984061610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fa988007fd0 tx=0x7fa988009040).stop 2023-11-06T20:44:59.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3747078465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9980ad8e0 msgr2=0x7fa998150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:44:59.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9980ad8e0 0x7fa998150890 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fa98c011af0 tx=0x7fa98c009060).stop 2023-11-06T20:44:59.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3747078465 shutdown_connections 2023-11-06T20:44:59.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa98405f150 0x7fa984061610 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa980005000 0x7fa998150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa998150d80 0x7fa998159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 --2- 172.21.15.33:0/3747078465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9980ad8e0 0x7fa998150890 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:44:59.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3747078465 >> 172.21.15.33:0/3747078465 conn(0x7fa9980c56f0 msgr2=0x7fa9980644d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:44:59.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3747078465 shutdown_connections 2023-11-06T20:44:59.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:44:59.889+0000 7fa99d1ff700 1 -- 172.21.15.33:0/3747078465 wait complete. 2023-11-06T20:44:59.892 INFO:teuthology.orchestra.run.smithi033.stderr:updated 2023-11-06T20:44:59.906 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-06T20:45:00.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.104+0000 7f54405ff700 1 Processor -- start 2023-11-06T20:45:00.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.104+0000 7f54405ff700 1 -- start start 2023-11-06T20:45:00.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.105+0000 7f54405ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 0x7f54380c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.105+0000 7f54405ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5438064dd0 0x7f54380c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.105+0000 7f54405ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54381556d0 con 0x7f54380c0c40 2023-11-06T20:45:00.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.105+0000 7f54405ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54381548d0 con 0x7f5438064dd0 2023-11-06T20:45:00.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.105+0000 7f54405ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5438154ac0 con 0x7f54381544f0 2023-11-06T20:45:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543e39b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 0x7f54380c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543db9a700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f54380c0c40 0x7f54380646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:49262/0 (socket says 172.21.15.33:49262) 2023-11-06T20:45:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543db9a700 1 -- 172.21.15.33:0/2657420819 learned_addr learned my addr 172.21.15.33:0/2657420819 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543eb9c700 1 --2- 172.21.15.33:0/2657420819 >> v2:172.21.15.69:3300/0 conn(0x7f5438064dd0 0x7f54380c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543d399700 1 -- 172.21.15.33:0/2657420819 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536804374 0 0) 0x7f54381556d0 con 0x7f54380c0c40 2023-11-06T20:45:00.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543d399700 1 -- 172.21.15.33:0/2657420819 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54240033f0 con 0x7f54380c0c40 2023-11-06T20:45:00.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543eb9c700 1 -- 172.21.15.33:0/2657420819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 msgr2=0x7f54380c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:00.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543eb9c700 1 --2- 172.21.15.33:0/2657420819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 0x7f54380c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543eb9c700 1 -- 172.21.15.33:0/2657420819 >> v1:172.21.15.69:6789/0 conn(0x7f54380c0c40 legacy=0x7f54380646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543eb9c700 1 -- 172.21.15.33:0/2657420819 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5438154da0 con 0x7f5438064dd0 2023-11-06T20:45:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543e39b700 1 --2- 172.21.15.33:0/2657420819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 0x7f54380c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.106+0000 7f543eb9c700 1 --2- 172.21.15.33:0/2657420819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5438064dd0 0x7f54380c57e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f5430009e60 tx=0x7f5430009780).ready entity=mon.1 client_cookie=bb1b4d7b0e8d23ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f543d399700 1 -- 172.21.15.33:0/2657420819 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543000e070 con 0x7f5438064dd0 2023-11-06T20:45:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f543d399700 1 -- 172.21.15.33:0/2657420819 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543000bab0 con 0x7f5438064dd0 2023-11-06T20:45:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f543d399700 1 -- 172.21.15.33:0/2657420819 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543000be30 con 0x7f5438064dd0 2023-11-06T20:45:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f54405ff700 1 -- 172.21.15.33:0/2657420819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5438064dd0 msgr2=0x7f54380c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f54405ff700 1 --2- 172.21.15.33:0/2657420819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5438064dd0 0x7f54380c57e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f5430009e60 tx=0x7f5430009780).stop 2023-11-06T20:45:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f54405ff700 1 -- 172.21.15.33:0/2657420819 shutdown_connections 2023-11-06T20:45:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f54405ff700 1 --2- 172.21.15.33:0/2657420819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5438064dd0 0x7f54380c57e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f54405ff700 1 --2- 172.21.15.33:0/2657420819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 0x7f54380c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f54405ff700 1 -- 172.21.15.33:0/2657420819 >> 172.21.15.33:0/2657420819 conn(0x7f54380cb900 msgr2=0x7f54380c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f54405ff700 1 -- 172.21.15.33:0/2657420819 shutdown_connections 2023-11-06T20:45:00.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.107+0000 7f54405ff700 1 -- 172.21.15.33:0/2657420819 wait complete. 2023-11-06T20:45:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.109+0000 7f54405ff700 1 Processor -- start 2023-11-06T20:45:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.109+0000 7f54405ff700 1 -- start start 2023-11-06T20:45:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.110+0000 7f54405ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5438064dd0 0x7f54381644f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.110+0000 7f54405ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380c0c40 0x7f54380c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.110+0000 7f54405ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 0x7f5438152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.110+0000 7f54405ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54380ae220 con 0x7f54380c0c40 2023-11-06T20:45:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.110+0000 7f54405ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54380ae500 con 0x7f5438064dd0 2023-11-06T20:45:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.110+0000 7f54405ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54380ae7e0 con 0x7f54381544f0 2023-11-06T20:45:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.110+0000 7f543db9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380c0c40 0x7f54380c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.110+0000 7f543db9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380c0c40 0x7f54380c28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42152/0 (socket says 172.21.15.33:42152) 2023-11-06T20:45:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.110+0000 7f543db9a700 1 -- 172.21.15.33:0/4040422006 learned_addr learned my addr 172.21.15.33:0/4040422006 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f543eb9c700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 0x7f5438152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f543e39b700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5438064dd0 0x7f54381644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f543db9a700 1 -- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 msgr2=0x7f5438152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f543db9a700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 0x7f5438152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f543db9a700 1 -- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5438064dd0 msgr2=0x7f54381644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f543db9a700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5438064dd0 0x7f54381644f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f543db9a700 1 -- 172.21.15.33:0/4040422006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54340092a0 con 0x7f54380c0c40 2023-11-06T20:45:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f543db9a700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380c0c40 0x7f54380c28b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f542800c4c0 tx=0x7f5428007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f542e7fc700 1 -- 172.21.15.33:0/4040422006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54280072e0 con 0x7f54380c0c40 2023-11-06T20:45:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f542e7fc700 1 -- 172.21.15.33:0/4040422006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428007440 con 0x7f54380c0c40 2023-11-06T20:45:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f54405ff700 1 -- 172.21.15.33:0/4040422006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5430009a20 con 0x7f54380c0c40 2023-11-06T20:45:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f542e7fc700 1 -- 172.21.15.33:0/4040422006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54280057a0 con 0x7f54380c0c40 2023-11-06T20:45:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.111+0000 7f54405ff700 1 -- 172.21.15.33:0/4040422006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54380aeda0 con 0x7f54380c0c40 2023-11-06T20:45:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.112+0000 7f54405ff700 1 -- 172.21.15.33:0/4040422006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54380aef90 con 0x7f54380c0c40 2023-11-06T20:45:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.113+0000 7f542e7fc700 1 -- 172.21.15.33:0/4040422006 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5428005bd0 con 0x7f54380c0c40 2023-11-06T20:45:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.114+0000 7f542e7fc700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f541c05f150 0x7f541c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.114+0000 7f542e7fc700 1 -- 172.21.15.33:0/4040422006 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f5428079870 con 0x7f54380c0c40 2023-11-06T20:45:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.114+0000 7f543e39b700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f541c05f150 0x7f541c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.115+0000 7f543e39b700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f541c05f150 0x7f541c061610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f5434009270 tx=0x7f5434008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:00.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.118+0000 7f542e7fc700 1 -- 172.21.15.33:0/4040422006 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f542804add0 con 0x7f54380c0c40 2023-11-06T20:45:00.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.369+0000 7f54405ff700 1 -- 172.21.15.33:0/4040422006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f54380c8760 con 0x7f54380c0c40 2023-11-06T20:45:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.373+0000 7f542e7fc700 1 -- 172.21.15.33:0/4040422006 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5428018030 con 0x7f54380c0c40 2023-11-06T20:45:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 -- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f541c05f150 msgr2=0x7f541c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f541c05f150 0x7f541c061610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f5434009270 tx=0x7f5434008040).stop 2023-11-06T20:45:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 -- 172.21.15.33:0/4040422006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380c0c40 msgr2=0x7f54380c28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380c0c40 0x7f54380c28b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f542800c4c0 tx=0x7f5428007c50).stop 2023-11-06T20:45:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 -- 172.21.15.33:0/4040422006 shutdown_connections 2023-11-06T20:45:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f541c05f150 0x7f541c061610 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54381544f0 0x7f5438152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5438064dd0 0x7f54381644f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 --2- 172.21.15.33:0/4040422006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380c0c40 0x7f54380c28b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 -- 172.21.15.33:0/4040422006 >> 172.21.15.33:0/4040422006 conn(0x7f54380cb900 msgr2=0x7f5438153a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 -- 172.21.15.33:0/4040422006 shutdown_connections 2023-11-06T20:45:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.375+0000 7f54405ff700 1 -- 172.21.15.33:0/4040422006 wait complete. 2023-11-06T20:45:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:updated 2023-11-06T20:45:00.392 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-06T20:45:00.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.589+0000 7f0799fff700 1 Processor -- start 2023-11-06T20:45:00.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.589+0000 7f0799fff700 1 -- start start 2023-11-06T20:45:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.590+0000 7f0799fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0794154210 0x7f07940a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.590+0000 7f0799fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f07940bfd60 0x7f07940c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.590+0000 7f0799fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f07940c9f00 0x7f07940ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.590+0000 7f0799fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07940c06d0 con 0x7f0794154210 2023-11-06T20:45:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.590+0000 7f0799fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079405efc0 con 0x7f07940c9f00 2023-11-06T20:45:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.590+0000 7f0799fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07940c8ea0 con 0x7f07940bfd60 2023-11-06T20:45:00.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f07937fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0794154210 0x7f07940a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f07937fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0794154210 0x7f07940a7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42168/0 (socket says 172.21.15.33:42168) 2023-11-06T20:45:00.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f07937fe700 1 -- 172.21.15.33:0/2079470238 learned_addr learned my addr 172.21.15.33:0/2079470238 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:00.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f0792ffd700 1 --2- 172.21.15.33:0/2079470238 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f07940bfd60 0x7f07940c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f0793fff700 1 --2- 172.21.15.33:0/2079470238 >> v2:172.21.15.69:3300/0 conn(0x7f07940c9f00 0x7f07940ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f07937fe700 1 -- 172.21.15.33:0/2079470238 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f07940bfd60 msgr2=0x7f07940c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:00.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f07937fe700 1 --2- 172.21.15.33:0/2079470238 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f07940bfd60 0x7f07940c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f07937fe700 1 -- 172.21.15.33:0/2079470238 >> v2:172.21.15.69:3300/0 conn(0x7f07940c9f00 msgr2=0x7f07940ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:00.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f07937fe700 1 --2- 172.21.15.33:0/2079470238 >> v2:172.21.15.69:3300/0 conn(0x7f07940c9f00 0x7f07940ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f07937fe700 1 -- 172.21.15.33:0/2079470238 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07940c9140 con 0x7f0794154210 2023-11-06T20:45:00.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.591+0000 7f07937fe700 1 --2- 172.21.15.33:0/2079470238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0794154210 0x7f07940a7bd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f07940855b0 tx=0x7f07840090f0).ready entity=mon.0 client_cookie=b08f9788840f6941 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:00.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f07927fc700 1 -- 172.21.15.33:0/2079470238 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400f510 con 0x7f0794154210 2023-11-06T20:45:00.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f07927fc700 1 -- 172.21.15.33:0/2079470238 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078400f670 con 0x7f0794154210 2023-11-06T20:45:00.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f07927fc700 1 -- 172.21.15.33:0/2079470238 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400f9f0 con 0x7f0794154210 2023-11-06T20:45:00.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f0799fff700 1 -- 172.21.15.33:0/2079470238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0794154210 msgr2=0x7f07940a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f0799fff700 1 --2- 172.21.15.33:0/2079470238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0794154210 0x7f07940a7bd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f07940855b0 tx=0x7f07840090f0).stop 2023-11-06T20:45:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f0799fff700 1 -- 172.21.15.33:0/2079470238 shutdown_connections 2023-11-06T20:45:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f0799fff700 1 --2- 172.21.15.33:0/2079470238 >> v2:172.21.15.69:3300/0 conn(0x7f07940c9f00 0x7f07940ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f0799fff700 1 --2- 172.21.15.33:0/2079470238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0794154210 0x7f07940a7bd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f0799fff700 1 --2- 172.21.15.33:0/2079470238 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f07940bfd60 0x7f07940c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f0799fff700 1 -- 172.21.15.33:0/2079470238 >> 172.21.15.33:0/2079470238 conn(0x7f079405e990 msgr2=0x7f07940cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:00.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f0799fff700 1 -- 172.21.15.33:0/2079470238 shutdown_connections 2023-11-06T20:45:00.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.592+0000 7f0799fff700 1 -- 172.21.15.33:0/2079470238 wait complete. 2023-11-06T20:45:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.594+0000 7f0799fff700 1 Processor -- start 2023-11-06T20:45:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.595+0000 7f0799fff700 1 -- start start 2023-11-06T20:45:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f0799fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07940bfd60 0x7f07940c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f0799fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07940c9f00 0x7f0794152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f0799fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0794154210 0x7f0794152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f0799fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07940c31b0 con 0x7f07940c9f00 2023-11-06T20:45:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f0799fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0794152bf0 con 0x7f07940bfd60 2023-11-06T20:45:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f0799fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0794155dc0 con 0x7f0794154210 2023-11-06T20:45:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f0792ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07940c9f00 0x7f0794152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f0792ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07940c9f00 0x7f0794152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42180/0 (socket says 172.21.15.33:42180) 2023-11-06T20:45:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f0792ffd700 1 -- 172.21.15.33:0/3485192469 learned_addr learned my addr 172.21.15.33:0/3485192469 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f07937fe700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07940bfd60 0x7f07940c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.596+0000 7f0793fff700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0794154210 0x7f0794152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f0792ffd700 1 -- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0794154210 msgr2=0x7f0794152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f0792ffd700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0794154210 0x7f0794152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f0792ffd700 1 -- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07940bfd60 msgr2=0x7f07940c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f0792ffd700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07940bfd60 0x7f07940c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f0792ffd700 1 -- 172.21.15.33:0/3485192469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784009370 con 0x7f07940c9f00 2023-11-06T20:45:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f0792ffd700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07940c9f00 0x7f0794152210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f078800e930 tx=0x7f0788009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f077b7fe700 1 -- 172.21.15.33:0/3485192469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078800c670 con 0x7f07940c9f00 2023-11-06T20:45:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f077b7fe700 1 -- 172.21.15.33:0/3485192469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078800c7d0 con 0x7f07940c9f00 2023-11-06T20:45:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f0799fff700 1 -- 172.21.15.33:0/3485192469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07941560a0 con 0x7f07940c9f00 2023-11-06T20:45:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f077b7fe700 1 -- 172.21.15.33:0/3485192469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078800cb50 con 0x7f07940c9f00 2023-11-06T20:45:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.597+0000 7f0799fff700 1 -- 172.21.15.33:0/3485192469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0794156660 con 0x7f07940c9f00 2023-11-06T20:45:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.598+0000 7f0799fff700 1 -- 172.21.15.33:0/3485192469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079406c6c0 con 0x7f07940c9f00 2023-11-06T20:45:00.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.599+0000 7f077b7fe700 1 -- 172.21.15.33:0/3485192469 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f078801b400 con 0x7f07940c9f00 2023-11-06T20:45:00.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.600+0000 7f077b7fe700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f078005f150 0x7f0780061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:00.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.600+0000 7f077b7fe700 1 -- 172.21.15.33:0/3485192469 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f078807a420 con 0x7f07940c9f00 2023-11-06T20:45:00.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.600+0000 7f07937fe700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f078005f150 0x7f0780061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:00.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.601+0000 7f07937fe700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f078005f150 0x7f0780061610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0784009130 tx=0x7f0784017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:00.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.604+0000 7f077b7fe700 1 -- 172.21.15.33:0/3485192469 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f078804b9d0 con 0x7f07940c9f00 2023-11-06T20:45:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.855+0000 7f0799fff700 1 -- 172.21.15.33:0/3485192469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f07940c9420 con 0x7f07940c9f00 2023-11-06T20:45:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.858+0000 7f077b7fe700 1 -- 172.21.15.33:0/3485192469 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f078804eff0 con 0x7f07940c9f00 2023-11-06T20:45:00.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 -- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f078005f150 msgr2=0x7f0780061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f078005f150 0x7f0780061610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0784009130 tx=0x7f0784017000).stop 2023-11-06T20:45:00.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 -- 172.21.15.33:0/3485192469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07940c9f00 msgr2=0x7f0794152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:00.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07940c9f00 0x7f0794152210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f078800e930 tx=0x7f0788009060).stop 2023-11-06T20:45:00.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 -- 172.21.15.33:0/3485192469 shutdown_connections 2023-11-06T20:45:00.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f078005f150 0x7f0780061610 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0794154210 0x7f0794152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07940bfd60 0x7f07940c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 --2- 172.21.15.33:0/3485192469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07940c9f00 0x7f0794152210 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:00.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 -- 172.21.15.33:0/3485192469 >> 172.21.15.33:0/3485192469 conn(0x7f079405e990 msgr2=0x7f0794064050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:00.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 -- 172.21.15.33:0/3485192469 shutdown_connections 2023-11-06T20:45:00.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:00.861+0000 7f0799fff700 1 -- 172.21.15.33:0/3485192469 wait complete. 2023-11-06T20:45:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:updated 2023-11-06T20:45:00.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:45:01.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.075+0000 7f49425ff700 1 Processor -- start 2023-11-06T20:45:01.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.076+0000 7f49425ff700 1 -- start start 2023-11-06T20:45:01.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.077+0000 7f49425ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f493c0c1840 0x7f493c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.077+0000 7f49425ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0c6460 0x7f493c0bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.077+0000 7f49425ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c1554a0 con 0x7f493c0aa2d0 2023-11-06T20:45:01.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.077+0000 7f49425ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c154960 con 0x7f493c0c1840 2023-11-06T20:45:01.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.077+0000 7f49425ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c155930 con 0x7f493c0c6460 2023-11-06T20:45:01.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.078+0000 7f493bfff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f493c0c1840 0x7f493c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.078+0000 7f4940b9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0c6460 0x7f493c0bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.078+0000 7f4940b9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0c6460 0x7f493c0bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:48088/0 (socket says 172.21.15.33:48088) 2023-11-06T20:45:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.078+0000 7f4940b9c700 1 -- 172.21.15.33:0/3102307836 learned_addr learned my addr 172.21.15.33:0/3102307836 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.078+0000 7f493bfff700 1 -- 172.21.15.33:0/3102307836 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0c6460 msgr2=0x7f493c0bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:01.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.078+0000 7f493bfff700 1 --2- 172.21.15.33:0/3102307836 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0c6460 0x7f493c0bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.078+0000 7f493bfff700 1 -- 172.21.15.33:0/3102307836 >> v1:172.21.15.69:6789/0 conn(0x7f493c0aa2d0 legacy=0x7f493c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.078+0000 7f493bfff700 1 -- 172.21.15.33:0/3102307836 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c154b70 con 0x7f493c0c1840 2023-11-06T20:45:01.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.078+0000 7f4940b9c700 1 --2- 172.21.15.33:0/3102307836 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0c6460 0x7f493c0bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:01.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.078+0000 7f493bfff700 1 --2- 172.21.15.33:0/3102307836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f493c0c1840 0x7f493c0a9de0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f492c009760 tx=0x7f492c009060).ready entity=mon.1 client_cookie=d4806569210575d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:01.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f493affd700 1 -- 172.21.15.33:0/3102307836 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00f510 con 0x7f493c0c1840 2023-11-06T20:45:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f493affd700 1 -- 172.21.15.33:0/3102307836 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00f670 con 0x7f493c0c1840 2023-11-06T20:45:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f493affd700 1 -- 172.21.15.33:0/3102307836 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00f9f0 con 0x7f493c0c1840 2023-11-06T20:45:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f49425ff700 1 -- 172.21.15.33:0/3102307836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f493c0c1840 msgr2=0x7f493c0a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f49425ff700 1 --2- 172.21.15.33:0/3102307836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f493c0c1840 0x7f493c0a9de0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f492c009760 tx=0x7f492c009060).stop 2023-11-06T20:45:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f49425ff700 1 -- 172.21.15.33:0/3102307836 shutdown_connections 2023-11-06T20:45:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f49425ff700 1 --2- 172.21.15.33:0/3102307836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f493c0c1840 0x7f493c0a9de0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f49425ff700 1 --2- 172.21.15.33:0/3102307836 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0c6460 0x7f493c0bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f49425ff700 1 -- 172.21.15.33:0/3102307836 >> 172.21.15.33:0/3102307836 conn(0x7f493c05ec80 msgr2=0x7f493c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f49425ff700 1 -- 172.21.15.33:0/3102307836 shutdown_connections 2023-11-06T20:45:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.079+0000 7f49425ff700 1 -- 172.21.15.33:0/3102307836 wait complete. 2023-11-06T20:45:01.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.081+0000 7f49425ff700 1 Processor -- start 2023-11-06T20:45:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.081+0000 7f49425ff700 1 -- start start 2023-11-06T20:45:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.082+0000 7f49425ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0aa2d0 0x7f493c158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.082+0000 7f49425ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f493c0c1840 0x7f493c146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.082+0000 7f49425ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c0c6460 0x7f493c0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.082+0000 7f49425ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0aff70 con 0x7f493c0c6460 2023-11-06T20:45:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.082+0000 7f49425ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c155fe0 con 0x7f493c0c1840 2023-11-06T20:45:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.082+0000 7f49425ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0af920 con 0x7f493c0aa2d0 2023-11-06T20:45:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.082+0000 7f4940b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c0c6460 0x7f493c0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.082+0000 7f4940b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c0c6460 0x7f493c0aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42188/0 (socket says 172.21.15.33:42188) 2023-11-06T20:45:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.082+0000 7f4940b9c700 1 -- 172.21.15.33:0/1695531160 learned_addr learned my addr 172.21.15.33:0/1695531160 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:01.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f493b7fe700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f493c0c1840 0x7f493c146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f493bfff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0aa2d0 0x7f493c158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f4940b9c700 1 -- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0aa2d0 msgr2=0x7f493c158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f4940b9c700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0aa2d0 0x7f493c158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f4940b9c700 1 -- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f493c0c1840 msgr2=0x7f493c146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f4940b9c700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f493c0c1840 0x7f493c146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f4940b9c700 1 -- 172.21.15.33:0/1695531160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4934009ae0 con 0x7f493c0c6460 2023-11-06T20:45:01.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f493bfff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0aa2d0 0x7f493c158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:01.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f4940b9c700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c0c6460 0x7f493c0aae40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f4934009fb0 tx=0x7f493400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:01.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f4923fff700 1 -- 172.21.15.33:0/1695531160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400cb20 con 0x7f493c0c6460 2023-11-06T20:45:01.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f4923fff700 1 -- 172.21.15.33:0/1695531160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493400f040 con 0x7f493c0c6460 2023-11-06T20:45:01.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f49425ff700 1 -- 172.21.15.33:0/1695531160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c0092a0 con 0x7f493c0c6460 2023-11-06T20:45:01.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f4923fff700 1 -- 172.21.15.33:0/1695531160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400ce90 con 0x7f493c0c6460 2023-11-06T20:45:01.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.083+0000 7f49425ff700 1 -- 172.21.15.33:0/1695531160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c0af5d0 con 0x7f493c0c6460 2023-11-06T20:45:01.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.084+0000 7f49425ff700 1 -- 172.21.15.33:0/1695531160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c0b00e0 con 0x7f493c0c6460 2023-11-06T20:45:01.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.085+0000 7f4923fff700 1 -- 172.21.15.33:0/1695531160 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4934014670 con 0x7f493c0c6460 2023-11-06T20:45:01.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.086+0000 7f4923fff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f492805f120 0x7f49280615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.086+0000 7f4923fff700 1 -- 172.21.15.33:0/1695531160 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f4934079ff0 con 0x7f493c0c6460 2023-11-06T20:45:01.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.086+0000 7f493bfff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f492805f120 0x7f49280615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.087+0000 7f493bfff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f492805f120 0x7f49280615e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f493c152f90 tx=0x7f492c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:01.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.090+0000 7f4923fff700 1 -- 172.21.15.33:0/1695531160 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f493404aaf0 con 0x7f493c0c6460 2023-11-06T20:45:01.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.274+0000 7f49425ff700 1 -- 172.21.15.33:0/1695531160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f493c06c6c0 con 0x7f493c0c6460 2023-11-06T20:45:01.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.275+0000 7f4923fff700 1 -- 172.21.15.33:0/1695531160 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+13760 (secure 0 0 0) 0x7f493404b600 con 0x7f493c0c6460 2023-11-06T20:45:01.275 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:01.275 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":31,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:44:57.422445+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:45:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 -- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f492805f120 msgr2=0x7f49280615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f492805f120 0x7f49280615e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f493c152f90 tx=0x7f492c017000).stop 2023-11-06T20:45:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 -- 172.21.15.33:0/1695531160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c0c6460 msgr2=0x7f493c0aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c0c6460 0x7f493c0aae40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f4934009fb0 tx=0x7f493400b040).stop 2023-11-06T20:45:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 -- 172.21.15.33:0/1695531160 shutdown_connections 2023-11-06T20:45:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f492805f120 0x7f49280615e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f493c0aa2d0 0x7f493c158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c0c6460 0x7f493c0aae40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 --2- 172.21.15.33:0/1695531160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f493c0c1840 0x7f493c146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 -- 172.21.15.33:0/1695531160 >> 172.21.15.33:0/1695531160 conn(0x7f493c05ec80 msgr2=0x7f493c0ad420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 -- 172.21.15.33:0/1695531160 shutdown_connections 2023-11-06T20:45:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.277+0000 7f49425ff700 1 -- 172.21.15.33:0/1695531160 wait complete. 2023-11-06T20:45:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T20:45:01.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:45:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.495+0000 7f2a41fff700 1 Processor -- start 2023-11-06T20:45:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.495+0000 7f2a41fff700 1 -- start start 2023-11-06T20:45:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a41fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2a3c14faa0 0x7f2a3c064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a41fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c14fe80 0x7f2a3c152750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a41fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a3c152c40 0x7f2a3c0c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a41fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c064270 con 0x7f2a3c152c40 2023-11-06T20:45:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a41fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c0651b0 con 0x7f2a3c14faa0 2023-11-06T20:45:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a41fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c1531f0 con 0x7f2a3c14fe80 2023-11-06T20:45:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a3bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a3c152c40 0x7f2a3c0c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a3bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a3c152c40 0x7f2a3c0c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42196/0 (socket says 172.21.15.33:42196) 2023-11-06T20:45:01.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a3bfff700 1 -- 172.21.15.33:0/1885029236 learned_addr learned my addr 172.21.15.33:0/1885029236 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a3b7fe700 1 --2- 172.21.15.33:0/1885029236 >> v2:172.21.15.69:3300/0 conn(0x7f2a3c14faa0 0x7f2a3c064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.497+0000 7f2a3affd700 1 --2- 172.21.15.33:0/1885029236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c14fe80 0x7f2a3c152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3bfff700 1 -- 172.21.15.33:0/1885029236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c14fe80 msgr2=0x7f2a3c152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3bfff700 1 --2- 172.21.15.33:0/1885029236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c14fe80 0x7f2a3c152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3bfff700 1 -- 172.21.15.33:0/1885029236 >> v2:172.21.15.69:3300/0 conn(0x7f2a3c14faa0 msgr2=0x7f2a3c064cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3bfff700 1 --2- 172.21.15.33:0/1885029236 >> v2:172.21.15.69:3300/0 conn(0x7f2a3c14faa0 0x7f2a3c064cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3bfff700 1 -- 172.21.15.33:0/1885029236 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c1534d0 con 0x7f2a3c152c40 2023-11-06T20:45:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3affd700 1 --2- 172.21.15.33:0/1885029236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c14fe80 0x7f2a3c152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3b7fe700 1 --2- 172.21.15.33:0/1885029236 >> v2:172.21.15.69:3300/0 conn(0x7f2a3c14faa0 0x7f2a3c064cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3bfff700 1 --2- 172.21.15.33:0/1885029236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c152c40 0x7f2a3c0c3450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f2a3c0676b0 tx=0x7f2a34009840).ready entity=mon.0 client_cookie=4fe54e786f71b0e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3a7fc700 1 -- 172.21.15.33:0/1885029236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400e070 con 0x7f2a3c152c40 2023-11-06T20:45:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3a7fc700 1 -- 172.21.15.33:0/1885029236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3400ba50 con 0x7f2a3c152c40 2023-11-06T20:45:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.498+0000 7f2a3a7fc700 1 -- 172.21.15.33:0/1885029236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400bdd0 con 0x7f2a3c152c40 2023-11-06T20:45:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.499+0000 7f2a41fff700 1 -- 172.21.15.33:0/1885029236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c152c40 msgr2=0x7f2a3c0c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.499+0000 7f2a41fff700 1 --2- 172.21.15.33:0/1885029236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c152c40 0x7f2a3c0c3450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f2a3c0676b0 tx=0x7f2a34009840).stop 2023-11-06T20:45:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.499+0000 7f2a41fff700 1 -- 172.21.15.33:0/1885029236 shutdown_connections 2023-11-06T20:45:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.499+0000 7f2a41fff700 1 --2- 172.21.15.33:0/1885029236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c152c40 0x7f2a3c0c3450 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.499+0000 7f2a41fff700 1 --2- 172.21.15.33:0/1885029236 >> v2:172.21.15.69:3300/0 conn(0x7f2a3c14faa0 0x7f2a3c064cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.499+0000 7f2a41fff700 1 --2- 172.21.15.33:0/1885029236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c14fe80 0x7f2a3c152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.499+0000 7f2a41fff700 1 -- 172.21.15.33:0/1885029236 >> 172.21.15.33:0/1885029236 conn(0x7f2a3c062710 msgr2=0x7f2a3c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.499+0000 7f2a41fff700 1 -- 172.21.15.33:0/1885029236 shutdown_connections 2023-11-06T20:45:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.499+0000 7f2a41fff700 1 -- 172.21.15.33:0/1885029236 wait complete. 2023-11-06T20:45:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.500+0000 7f2a41fff700 1 Processor -- start 2023-11-06T20:45:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.500+0000 7f2a41fff700 1 -- start start 2023-11-06T20:45:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.501+0000 7f2a41fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a3c14faa0 0x7f2a3c0aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.501+0000 7f2a41fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c14fe80 0x7f2a3c147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.501+0000 7f2a41fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c152c40 0x7f2a3c158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.501+0000 7f2a41fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c157570 con 0x7f2a3c14fe80 2023-11-06T20:45:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.501+0000 7f2a41fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c147d30 con 0x7f2a3c14faa0 2023-11-06T20:45:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.501+0000 7f2a41fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0a9dd0 con 0x7f2a3c152c40 2023-11-06T20:45:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c14fe80 0x7f2a3c147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c14fe80 0x7f2a3c147840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42208/0 (socket says 172.21.15.33:42208) 2023-11-06T20:45:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3affd700 1 -- 172.21.15.33:0/1891412326 learned_addr learned my addr 172.21.15.33:0/1891412326 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3b7fe700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a3c14faa0 0x7f2a3c0aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3bfff700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c152c40 0x7f2a3c158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3affd700 1 -- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c152c40 msgr2=0x7f2a3c158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3affd700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c152c40 0x7f2a3c158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3affd700 1 -- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a3c14faa0 msgr2=0x7f2a3c0aae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3affd700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a3c14faa0 0x7f2a3c0aae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3affd700 1 -- 172.21.15.33:0/1891412326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34009ae0 con 0x7f2a3c14fe80 2023-11-06T20:45:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a3affd700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c14fe80 0x7f2a3c147840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f2a3000c880 tx=0x7f2a30008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a237fe700 1 -- 172.21.15.33:0/1891412326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30010580 con 0x7f2a3c14fe80 2023-11-06T20:45:01.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a237fe700 1 -- 172.21.15.33:0/1891412326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a300106e0 con 0x7f2a3c14fe80 2023-11-06T20:45:01.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a41fff700 1 -- 172.21.15.33:0/1891412326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c0aa0b0 con 0x7f2a3c14fe80 2023-11-06T20:45:01.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.502+0000 7f2a237fe700 1 -- 172.21.15.33:0/1891412326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30010a60 con 0x7f2a3c14fe80 2023-11-06T20:45:01.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.503+0000 7f2a41fff700 1 -- 172.21.15.33:0/1891412326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c0aa670 con 0x7f2a3c14fe80 2023-11-06T20:45:01.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.504+0000 7f2a41fff700 1 -- 172.21.15.33:0/1891412326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c0aacc0 con 0x7f2a3c14fe80 2023-11-06T20:45:01.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.504+0000 7f2a237fe700 1 -- 172.21.15.33:0/1891412326 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2a3001b460 con 0x7f2a3c14fe80 2023-11-06T20:45:01.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.505+0000 7f2a237fe700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a2805f150 0x7f2a28061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.506+0000 7f2a237fe700 1 -- 172.21.15.33:0/1891412326 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f2a30079f00 con 0x7f2a3c14fe80 2023-11-06T20:45:01.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.506+0000 7f2a3b7fe700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a2805f150 0x7f2a28061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.506+0000 7f2a3b7fe700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a2805f150 0x7f2a28061610 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f2a2c00aa70 tx=0x7f2a2c008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:01.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.509+0000 7f2a237fe700 1 -- 172.21.15.33:0/1891412326 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2a3004b400 con 0x7f2a3c14fe80 2023-11-06T20:45:01.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.689+0000 7f2a41fff700 1 -- 172.21.15.33:0/1891412326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2a3c063c00 con 0x7f2a3c14fe80 2023-11-06T20:45:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.689+0000 7f2a237fe700 1 -- 172.21.15.33:0/1891412326 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v31) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2a3004ea20 con 0x7f2a3c14fe80 2023-11-06T20:45:01.690 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:45:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.691+0000 7f2a41fff700 1 -- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a2805f150 msgr2=0x7f2a28061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.691+0000 7f2a41fff700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a2805f150 0x7f2a28061610 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f2a2c00aa70 tx=0x7f2a2c008040).stop 2023-11-06T20:45:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.691+0000 7f2a41fff700 1 -- 172.21.15.33:0/1891412326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c14fe80 msgr2=0x7f2a3c147840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.691+0000 7f2a41fff700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c14fe80 0x7f2a3c147840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f2a3000c880 tx=0x7f2a30008040).stop 2023-11-06T20:45:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.691+0000 7f2a41fff700 1 -- 172.21.15.33:0/1891412326 shutdown_connections 2023-11-06T20:45:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.691+0000 7f2a41fff700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a2805f150 0x7f2a28061610 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.691+0000 7f2a41fff700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a3c152c40 0x7f2a3c158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.691+0000 7f2a41fff700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a3c14faa0 0x7f2a3c0aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.692+0000 7f2a41fff700 1 --2- 172.21.15.33:0/1891412326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a3c14fe80 0x7f2a3c147840 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.692+0000 7f2a41fff700 1 -- 172.21.15.33:0/1891412326 >> 172.21.15.33:0/1891412326 conn(0x7f2a3c062710 msgr2=0x7f2a3c152280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.692+0000 7f2a41fff700 1 -- 172.21.15.33:0/1891412326 shutdown_connections 2023-11-06T20:45:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.692+0000 7f2a41fff700 1 -- 172.21.15.33:0/1891412326 wait complete. 2023-11-06T20:45:01.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:45:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.909+0000 7ff0e85ff700 1 Processor -- start 2023-11-06T20:45:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.909+0000 7ff0e85ff700 1 -- start start 2023-11-06T20:45:01.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e85ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e00c1f00 0x7ff0e015b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e85ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e00bcc90 con 0x7ff0e014e0b0 2023-11-06T20:45:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e85ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e00bcf70 con 0x7ff0e014e490 2023-11-06T20:45:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e85ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00bd250 con 0x7ff0e00c1f00 2023-11-06T20:45:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e639b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff0e014e0b0 0x7ff0e00bc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34078/0 (socket says 172.21.15.33:34078) 2023-11-06T20:45:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e639b700 1 -- 172.21.15.33:0/3944178942 learned_addr learned my addr 172.21.15.33:0/3944178942 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e6b9c700 1 --2- 172.21.15.33:0/3944178942 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e00c1f00 0x7ff0e015b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e5399700 1 -- 172.21.15.33:0/3944178942 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930444292 0 0) 0x7ff0e00bcc90 con 0x7ff0e014e0b0 2023-11-06T20:45:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e5399700 1 -- 172.21.15.33:0/3944178942 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0cc0033f0 con 0x7ff0e014e0b0 2023-11-06T20:45:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e5399700 1 -- 172.21.15.33:0/3944178942 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898418526 0 0) 0x7ff0e00bcf70 con 0x7ff0e014e490 2023-11-06T20:45:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.911+0000 7ff0e5399700 1 -- 172.21.15.33:0/3944178942 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e00bcc90 con 0x7ff0e014e490 2023-11-06T20:45:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.912+0000 7ff0e6b9c700 1 -- 172.21.15.33:0/3944178942 >> v1:172.21.15.69:6789/0 conn(0x7ff0e014e490 legacy=0x7ff0e00c17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.912+0000 7ff0e6b9c700 1 -- 172.21.15.33:0/3944178942 >> v1:172.21.15.33:6789/0 conn(0x7ff0e014e0b0 legacy=0x7ff0e00bc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.912+0000 7ff0e6b9c700 1 -- 172.21.15.33:0/3944178942 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00bd530 con 0x7ff0e00c1f00 2023-11-06T20:45:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.912+0000 7ff0e6b9c700 1 --2- 172.21.15.33:0/3944178942 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e00c1f00 0x7ff0e015b740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff0d800afa0 tx=0x7ff0d800a890).ready entity=mon.2 client_cookie=a836f6af40219552 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.912+0000 7ff0e5399700 1 -- 172.21.15.33:0/3944178942 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800f070 con 0x7ff0e00c1f00 2023-11-06T20:45:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.912+0000 7ff0e5399700 1 -- 172.21.15.33:0/3944178942 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d800a2e0 con 0x7ff0e00c1f00 2023-11-06T20:45:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.912+0000 7ff0e5399700 1 -- 172.21.15.33:0/3944178942 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800cbb0 con 0x7ff0e00c1f00 2023-11-06T20:45:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.913+0000 7ff0e85ff700 1 -- 172.21.15.33:0/3944178942 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e00c1f00 msgr2=0x7ff0e015b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.913+0000 7ff0e85ff700 1 --2- 172.21.15.33:0/3944178942 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e00c1f00 0x7ff0e015b740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff0d800afa0 tx=0x7ff0d800a890).stop 2023-11-06T20:45:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.913+0000 7ff0e85ff700 1 -- 172.21.15.33:0/3944178942 shutdown_connections 2023-11-06T20:45:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.913+0000 7ff0e85ff700 1 --2- 172.21.15.33:0/3944178942 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e00c1f00 0x7ff0e015b740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.913+0000 7ff0e85ff700 1 -- 172.21.15.33:0/3944178942 >> 172.21.15.33:0/3944178942 conn(0x7ff0e0062710 msgr2=0x7ff0e0062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.913+0000 7ff0e85ff700 1 -- 172.21.15.33:0/3944178942 shutdown_connections 2023-11-06T20:45:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.913+0000 7ff0e85ff700 1 -- 172.21.15.33:0/3944178942 wait complete. 2023-11-06T20:45:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.914+0000 7ff0e85ff700 1 Processor -- start 2023-11-06T20:45:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.914+0000 7ff0e85ff700 1 -- start start 2023-11-06T20:45:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.915+0000 7ff0e85ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e00c1f00 0x7ff0e0140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.915+0000 7ff0e85ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e014e0b0 0x7ff0e0161af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.915+0000 7ff0e85ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e014e490 0x7ff0e00c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.915+0000 7ff0e85ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00c76a0 con 0x7ff0e00c1f00 2023-11-06T20:45:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.915+0000 7ff0e85ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00be490 con 0x7ff0e014e490 2023-11-06T20:45:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.915+0000 7ff0e85ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00bdb60 con 0x7ff0e014e0b0 2023-11-06T20:45:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e639b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e00c1f00 0x7ff0e0140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e639b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e00c1f00 0x7ff0e0140020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42214/0 (socket says 172.21.15.33:42214) 2023-11-06T20:45:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e639b700 1 -- 172.21.15.33:0/4140676629 learned_addr learned my addr 172.21.15.33:0/4140676629 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e6b9c700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e014e490 0x7ff0e00c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e5b9a700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e014e0b0 0x7ff0e0161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e639b700 1 -- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e014e0b0 msgr2=0x7ff0e0161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e639b700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e014e0b0 0x7ff0e0161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e639b700 1 -- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e014e490 msgr2=0x7ff0e00c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e639b700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e014e490 0x7ff0e00c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e639b700 1 -- 172.21.15.33:0/4140676629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d800ab30 con 0x7ff0e00c1f00 2023-11-06T20:45:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e6b9c700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e014e490 0x7ff0e00c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.916+0000 7ff0e639b700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e00c1f00 0x7ff0e0140020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7ff0dc00c590 tx=0x7ff0dc007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.917+0000 7ff0d67fc700 1 -- 172.21.15.33:0/4140676629 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00ce60 con 0x7ff0e00c1f00 2023-11-06T20:45:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.917+0000 7ff0d67fc700 1 -- 172.21.15.33:0/4140676629 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc0072e0 con 0x7ff0e00c1f00 2023-11-06T20:45:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.917+0000 7ff0d67fc700 1 -- 172.21.15.33:0/4140676629 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc007660 con 0x7ff0e00c1f00 2023-11-06T20:45:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.917+0000 7ff0e85ff700 1 -- 172.21.15.33:0/4140676629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e00bde40 con 0x7ff0e00c1f00 2023-11-06T20:45:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.917+0000 7ff0e85ff700 1 -- 172.21.15.33:0/4140676629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e0140510 con 0x7ff0e00c1f00 2023-11-06T20:45:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.918+0000 7ff0e85ff700 1 -- 172.21.15.33:0/4140676629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e0151260 con 0x7ff0e00c1f00 2023-11-06T20:45:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.919+0000 7ff0d67fc700 1 -- 172.21.15.33:0/4140676629 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff0dc0059b0 con 0x7ff0e00c1f00 2023-11-06T20:45:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.920+0000 7ff0d67fc700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0c405f150 0x7ff0c4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.920+0000 7ff0d67fc700 1 -- 172.21.15.33:0/4140676629 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff0dc079520 con 0x7ff0e00c1f00 2023-11-06T20:45:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.921+0000 7ff0e5b9a700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0c405f150 0x7ff0c4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.921+0000 7ff0e5b9a700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0c405f150 0x7ff0c4061610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ff0d0007fd0 tx=0x7ff0d0007b90).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:01.925+0000 7ff0d67fc700 1 -- 172.21.15.33:0/4140676629 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff0dc04aac0 con 0x7ff0e00c1f00 2023-11-06T20:45:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.105+0000 7ff0e85ff700 1 -- 172.21.15.33:0/4140676629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff0e00bbd30 con 0x7ff0e00c1f00 2023-11-06T20:45:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.106+0000 7ff0d67fc700 1 -- 172.21.15.33:0/4140676629 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff0dc018020 con 0x7ff0e00c1f00 2023-11-06T20:45:02.106 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:45:02.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 -- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0c405f150 msgr2=0x7ff0c4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0c405f150 0x7ff0c4061610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ff0d0007fd0 tx=0x7ff0d0007b90).stop 2023-11-06T20:45:02.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 -- 172.21.15.33:0/4140676629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e00c1f00 msgr2=0x7ff0e0140020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e00c1f00 0x7ff0e0140020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7ff0dc00c590 tx=0x7ff0dc007d20).stop 2023-11-06T20:45:02.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 -- 172.21.15.33:0/4140676629 shutdown_connections 2023-11-06T20:45:02.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0c405f150 0x7ff0c4061610 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e014e490 0x7ff0e00c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e00c1f00 0x7ff0e0140020 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 --2- 172.21.15.33:0/4140676629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0e014e0b0 0x7ff0e0161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 -- 172.21.15.33:0/4140676629 >> 172.21.15.33:0/4140676629 conn(0x7ff0e0062710 msgr2=0x7ff0e0153510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:02.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 -- 172.21.15.33:0/4140676629 shutdown_connections 2023-11-06T20:45:02.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.108+0000 7ff0e85ff700 1 -- 172.21.15.33:0/4140676629 wait complete. 2023-11-06T20:45:02.125 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T20:45:02.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.322+0000 7fb743bff700 1 Processor -- start 2023-11-06T20:45:02.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.322+0000 7fb743bff700 1 -- start start 2023-11-06T20:45:02.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.322+0000 7fb743bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb73c0c5d50 0x7fb73c0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.323+0000 7fb743bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0bfd70 0x7fb73c0c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.323+0000 7fb743bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73c1554a0 con 0x7fb73c0c1840 2023-11-06T20:45:02.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.323+0000 7fb743bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73c154960 con 0x7fb73c0c5d50 2023-11-06T20:45:02.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.323+0000 7fb743bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0c66c0 con 0x7fb73c0bfd70 2023-11-06T20:45:02.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.323+0000 7fb74119a700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb73c0c5d50 0x7fb73c0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.323+0000 7fb74219c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0bfd70 0x7fb73c0c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.323+0000 7fb74219c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0bfd70 0x7fb73c0c01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:48158/0 (socket says 172.21.15.33:48158) 2023-11-06T20:45:02.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.323+0000 7fb74119a700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb73c0c5d50 0x7fb73c0c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:49486/0 (socket says 172.21.15.33:49486) 2023-11-06T20:45:02.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.323+0000 7fb74219c700 1 -- 172.21.15.33:0/4099501123 learned_addr learned my addr 172.21.15.33:0/4099501123 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:02.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.324+0000 7fb74219c700 1 -- 172.21.15.33:0/4099501123 >> v2:172.21.15.69:3300/0 conn(0x7fb73c0c5d50 msgr2=0x7fb73c0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:02.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.324+0000 7fb74219c700 1 --2- 172.21.15.33:0/4099501123 >> v2:172.21.15.69:3300/0 conn(0x7fb73c0c5d50 0x7fb73c0c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.324+0000 7fb74219c700 1 -- 172.21.15.33:0/4099501123 >> v1:172.21.15.69:6789/0 conn(0x7fb73c0c1840 legacy=0x7fb73c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.324+0000 7fb74219c700 1 -- 172.21.15.33:0/4099501123 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c155930 con 0x7fb73c0bfd70 2023-11-06T20:45:02.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.324+0000 7fb74119a700 1 --2- 172.21.15.33:0/4099501123 >> v2:172.21.15.69:3300/0 conn(0x7fb73c0c5d50 0x7fb73c0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:02.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.324+0000 7fb740999700 1 -- 172.21.15.33:0/4099501123 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951419246 0 0) 0x7fb73c1554a0 con 0x7fb73c0c1840 2023-11-06T20:45:02.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.324+0000 7fb74219c700 1 --2- 172.21.15.33:0/4099501123 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0bfd70 0x7fb73c0c01c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb734009f20 tx=0x7fb734009840).ready entity=mon.2 client_cookie=7b0fe3e134360a29 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:02.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.324+0000 7fb740999700 1 -- 172.21.15.33:0/4099501123 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73400e070 con 0x7fb73c0bfd70 2023-11-06T20:45:02.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.324+0000 7fb740999700 1 -- 172.21.15.33:0/4099501123 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73400b9d0 con 0x7fb73c0bfd70 2023-11-06T20:45:02.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.324+0000 7fb740999700 1 -- 172.21.15.33:0/4099501123 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73400bd50 con 0x7fb73c0bfd70 2023-11-06T20:45:02.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.326+0000 7fb743bff700 1 -- 172.21.15.33:0/4099501123 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0bfd70 msgr2=0x7fb73c0c01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.326+0000 7fb743bff700 1 --2- 172.21.15.33:0/4099501123 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0bfd70 0x7fb73c0c01c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb734009f20 tx=0x7fb734009840).stop 2023-11-06T20:45:02.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.326+0000 7fb743bff700 1 -- 172.21.15.33:0/4099501123 shutdown_connections 2023-11-06T20:45:02.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.326+0000 7fb743bff700 1 --2- 172.21.15.33:0/4099501123 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0bfd70 0x7fb73c0c01c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.326+0000 7fb743bff700 1 --2- 172.21.15.33:0/4099501123 >> v2:172.21.15.69:3300/0 conn(0x7fb73c0c5d50 0x7fb73c0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.326+0000 7fb743bff700 1 -- 172.21.15.33:0/4099501123 >> 172.21.15.33:0/4099501123 conn(0x7fb73c05ec80 msgr2=0x7fb73c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:02.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.326+0000 7fb743bff700 1 -- 172.21.15.33:0/4099501123 shutdown_connections 2023-11-06T20:45:02.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.326+0000 7fb743bff700 1 -- 172.21.15.33:0/4099501123 wait complete. 2023-11-06T20:45:02.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.327+0000 7fb743bff700 1 Processor -- start 2023-11-06T20:45:02.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.327+0000 7fb743bff700 1 -- start start 2023-11-06T20:45:02.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb743bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb73c0bfd70 0x7fb73c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb743bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb73c0c1840 0x7fb73c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb743bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0c5d50 0x7fb73c164410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb743bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0af550 con 0x7fb73c0bfd70 2023-11-06T20:45:02.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb743bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c14fa70 con 0x7fb73c0c1840 2023-11-06T20:45:02.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb743bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0c3bb0 con 0x7fb73c0c5d50 2023-11-06T20:45:02.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb73c0bfd70 0x7fb73c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb73c0bfd70 0x7fb73c0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42226/0 (socket says 172.21.15.33:42226) 2023-11-06T20:45:02.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74199b700 1 -- 172.21.15.33:0/1275773415 learned_addr learned my addr 172.21.15.33:0/1275773415 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:02.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74119a700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb73c0c1840 0x7fb73c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74219c700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0c5d50 0x7fb73c164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74199b700 1 -- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0c5d50 msgr2=0x7fb73c164410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74199b700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0c5d50 0x7fb73c164410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74199b700 1 -- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb73c0c1840 msgr2=0x7fb73c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74199b700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb73c0c1840 0x7fb73c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74199b700 1 -- 172.21.15.33:0/1275773415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c0092a0 con 0x7fb73c0bfd70 2023-11-06T20:45:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb74199b700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb73c0bfd70 0x7fb73c0b0950 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fb73800c4e0 tx=0x7fb738007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb731ffb700 1 -- 172.21.15.33:0/1275773415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73800ce50 con 0x7fb73c0bfd70 2023-11-06T20:45:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb731ffb700 1 -- 172.21.15.33:0/1275773415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73800f470 con 0x7fb73c0bfd70 2023-11-06T20:45:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb731ffb700 1 -- 172.21.15.33:0/1275773415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73800f830 con 0x7fb73c0bfd70 2023-11-06T20:45:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb743bff700 1 -- 172.21.15.33:0/1275773415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb734009ae0 con 0x7fb73c0bfd70 2023-11-06T20:45:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.329+0000 7fb743bff700 1 -- 172.21.15.33:0/1275773415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73c0c3890 con 0x7fb73c0bfd70 2023-11-06T20:45:02.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.330+0000 7fb743bff700 1 -- 172.21.15.33:0/1275773415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73c0c3e90 con 0x7fb73c0bfd70 2023-11-06T20:45:02.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.331+0000 7fb731ffb700 1 -- 172.21.15.33:0/1275773415 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb73800fba0 con 0x7fb73c0bfd70 2023-11-06T20:45:02.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.332+0000 7fb731ffb700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72005f150 0x7fb720061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.332+0000 7fb731ffb700 1 -- 172.21.15.33:0/1275773415 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fb738079380 con 0x7fb73c0bfd70 2023-11-06T20:45:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.332+0000 7fb74119a700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72005f150 0x7fb720061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.333+0000 7fb74119a700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72005f150 0x7fb720061610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fb73c0aa020 tx=0x7fb72c009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.337+0000 7fb731ffb700 1 -- 172.21.15.33:0/1275773415 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb738048af0 con 0x7fb73c0bfd70 2023-11-06T20:45:02.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.523+0000 7fb743bff700 1 -- 172.21.15.33:0/1275773415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb73c0af3b0 con 0x7fb73c0bfd70 2023-11-06T20:45:02.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.523+0000 7fb731ffb700 1 -- 172.21.15.33:0/1275773415 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+143 (secure 0 0 0) 0x7fb73c0af3b0 con 0x7fb73c0bfd70 2023-11-06T20:45:02.524 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:02.524 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T20:45:02.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:45:02.524 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T20:45:02.525 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T20:45:02.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:45:02.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:45:02.525 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T20:45:02.525 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T20:45:02.526 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:45:02.526 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T20:45:02.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 -- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72005f150 msgr2=0x7fb720061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72005f150 0x7fb720061610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fb73c0aa020 tx=0x7fb72c009060).stop 2023-11-06T20:45:02.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 -- 172.21.15.33:0/1275773415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb73c0bfd70 msgr2=0x7fb73c0b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb73c0bfd70 0x7fb73c0b0950 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fb73800c4e0 tx=0x7fb738007c70).stop 2023-11-06T20:45:02.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 -- 172.21.15.33:0/1275773415 shutdown_connections 2023-11-06T20:45:02.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72005f150 0x7fb720061610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb73c0c5d50 0x7fb73c164410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb73c0bfd70 0x7fb73c0b0950 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 --2- 172.21.15.33:0/1275773415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb73c0c1840 0x7fb73c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 -- 172.21.15.33:0/1275773415 >> 172.21.15.33:0/1275773415 conn(0x7fb73c05ec80 msgr2=0x7fb73c0c0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:02.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 -- 172.21.15.33:0/1275773415 shutdown_connections 2023-11-06T20:45:02.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.526+0000 7fb743bff700 1 -- 172.21.15.33:0/1275773415 wait complete. 2023-11-06T20:45:02.543 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T20:45:02.544 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T20:45:02.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.744+0000 7f51e89ff700 1 Processor -- start 2023-11-06T20:45:02.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.744+0000 7f51e89ff700 1 -- start start 2023-11-06T20:45:02.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e89ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f51e40c8ee0 0x7f51e40ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e89ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f51e41542b0 0x7f51e40c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e89ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e40c9f80 con 0x7f51e4153ed0 2023-11-06T20:45:02.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e89ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e40c6000 con 0x7f51e40c8ee0 2023-11-06T20:45:02.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e89ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e40c9370 con 0x7f51e41542b0 2023-11-06T20:45:02.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f51e40c8ee0 0x7f51e40ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f51e40c8ee0 0x7f51e40ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42240/0 (socket says 172.21.15.33:42240) 2023-11-06T20:45:02.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e1d9b700 1 -- 172.21.15.33:0/2587505527 learned_addr learned my addr 172.21.15.33:0/2587505527 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:02.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e2d9d700 1 --2- 172.21.15.33:0/2587505527 >> v2:172.21.15.69:3300/0 conn(0x7f51e41542b0 0x7f51e40c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e1d9b700 1 -- 172.21.15.33:0/2587505527 >> v2:172.21.15.69:3300/0 conn(0x7f51e41542b0 msgr2=0x7f51e40c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:02.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e1d9b700 1 --2- 172.21.15.33:0/2587505527 >> v2:172.21.15.69:3300/0 conn(0x7f51e41542b0 0x7f51e40c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e1d9b700 1 -- 172.21.15.33:0/2587505527 >> v1:172.21.15.33:6789/0 conn(0x7f51e4153ed0 legacy=0x7f51e4158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e1d9b700 1 -- 172.21.15.33:0/2587505527 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40c9650 con 0x7f51e40c8ee0 2023-11-06T20:45:02.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e1d9b700 1 --2- 172.21.15.33:0/2587505527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e40c8ee0 0x7f51e40ca410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f51e40c85d0 tx=0x7f51d80090f0).ready entity=mon.0 client_cookie=f9391070bdec6571 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:02.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e159a700 1 -- 172.21.15.33:0/2587505527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800b600 con 0x7f51e40c8ee0 2023-11-06T20:45:02.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e159a700 1 -- 172.21.15.33:0/2587505527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d800b760 con 0x7f51e40c8ee0 2023-11-06T20:45:02.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.746+0000 7f51e159a700 1 -- 172.21.15.33:0/2587505527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800bae0 con 0x7f51e40c8ee0 2023-11-06T20:45:02.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.747+0000 7f51e89ff700 1 -- 172.21.15.33:0/2587505527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e40c8ee0 msgr2=0x7f51e40ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.747+0000 7f51e89ff700 1 --2- 172.21.15.33:0/2587505527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e40c8ee0 0x7f51e40ca410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f51e40c85d0 tx=0x7f51d80090f0).stop 2023-11-06T20:45:02.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.747+0000 7f51e89ff700 1 -- 172.21.15.33:0/2587505527 shutdown_connections 2023-11-06T20:45:02.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.747+0000 7f51e89ff700 1 --2- 172.21.15.33:0/2587505527 >> v2:172.21.15.69:3300/0 conn(0x7f51e41542b0 0x7f51e40c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.747+0000 7f51e89ff700 1 --2- 172.21.15.33:0/2587505527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e40c8ee0 0x7f51e40ca410 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.747+0000 7f51e89ff700 1 -- 172.21.15.33:0/2587505527 >> 172.21.15.33:0/2587505527 conn(0x7f51e405e1b0 msgr2=0x7f51e405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:02.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.747+0000 7f51e89ff700 1 -- 172.21.15.33:0/2587505527 shutdown_connections 2023-11-06T20:45:02.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.747+0000 7f51e89ff700 1 -- 172.21.15.33:0/2587505527 wait complete. 2023-11-06T20:45:02.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.749+0000 7f51e89ff700 1 Processor -- start 2023-11-06T20:45:02.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.749+0000 7f51e89ff700 1 -- start start 2023-11-06T20:45:02.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e89ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4153ed0 0x7f51e414cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e89ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51e41542b0 0x7f51e414d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e89ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4164370 0x7f51e40cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e89ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e41508a0 con 0x7f51e4164370 2023-11-06T20:45:02.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e89ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e414f9d0 con 0x7f51e4153ed0 2023-11-06T20:45:02.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e89ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e414fc90 con 0x7f51e41542b0 2023-11-06T20:45:02.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4164370 0x7f51e40cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4164370 0x7f51e40cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42252/0 (socket says 172.21.15.33:42252) 2023-11-06T20:45:02.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e2d9d700 1 -- 172.21.15.33:0/4078186561 learned_addr learned my addr 172.21.15.33:0/4078186561 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:02.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e259c700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4153ed0 0x7f51e414cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e1d9b700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51e41542b0 0x7f51e414d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e2d9d700 1 -- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51e41542b0 msgr2=0x7f51e414d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e2d9d700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51e41542b0 0x7f51e414d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e2d9d700 1 -- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4153ed0 msgr2=0x7f51e414cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:02.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e2d9d700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4153ed0 0x7f51e414cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:02.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.750+0000 7f51e2d9d700 1 -- 172.21.15.33:0/4078186561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d8009370 con 0x7f51e4164370 2023-11-06T20:45:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.751+0000 7f51e2d9d700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4164370 0x7f51e40cae00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f51dc007ef0 tx=0x7f51dc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.751+0000 7f51d27fc700 1 -- 172.21.15.33:0/4078186561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00ebc0 con 0x7f51e4164370 2023-11-06T20:45:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.751+0000 7f51d27fc700 1 -- 172.21.15.33:0/4078186561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc00ed20 con 0x7f51e4164370 2023-11-06T20:45:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.751+0000 7f51d27fc700 1 -- 172.21.15.33:0/4078186561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc014610 con 0x7f51e4164370 2023-11-06T20:45:02.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.751+0000 7f51e89ff700 1 -- 172.21.15.33:0/4078186561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e414ff70 con 0x7f51e4164370 2023-11-06T20:45:02.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.751+0000 7f51e89ff700 1 -- 172.21.15.33:0/4078186561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e4150530 con 0x7f51e4164370 2023-11-06T20:45:02.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.752+0000 7f51e89ff700 1 -- 172.21.15.33:0/4078186561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e40c64c0 con 0x7f51e4164370 2023-11-06T20:45:02.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.753+0000 7f51d27fc700 1 -- 172.21.15.33:0/4078186561 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f51dc014a40 con 0x7f51e4164370 2023-11-06T20:45:02.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.755+0000 7f51d27fc700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51c405f180 0x7f51c4061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:02.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.755+0000 7f51d27fc700 1 -- 172.21.15.33:0/4078186561 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f51dc07a3a0 con 0x7f51e4164370 2023-11-06T20:45:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.755+0000 7f51e259c700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51c405f180 0x7f51c4061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.755+0000 7f51e259c700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51c405f180 0x7f51c4061640 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f51d4003900 tx=0x7f51d4007d40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.759+0000 7f51d27fc700 1 -- 172.21.15.33:0/4078186561 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f51dc04b900 con 0x7f51e4164370 2023-11-06T20:45:02.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:02.940+0000 7f51e89ff700 1 -- 172.21.15.33:0/4078186561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f51e40c9930 con 0x7f51e4164370 2023-11-06T20:45:03.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.169+0000 7f51d27fc700 1 -- 172.21.15.33:0/4078186561 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 152+0+0 (secure 0 0 0) 0x7f51dc04ef20 con 0x7f51e4164370 2023-11-06T20:45:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 -- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51c405f180 msgr2=0x7f51c4061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51c405f180 0x7f51c4061640 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f51d4003900 tx=0x7f51d4007d40).stop 2023-11-06T20:45:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 -- 172.21.15.33:0/4078186561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4164370 msgr2=0x7f51e40cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4164370 0x7f51e40cae00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f51dc007ef0 tx=0x7f51dc009840).stop 2023-11-06T20:45:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 -- 172.21.15.33:0/4078186561 shutdown_connections 2023-11-06T20:45:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f51c405f180 0x7f51c4061640 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4164370 0x7f51e40cae00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51e4153ed0 0x7f51e414cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 --2- 172.21.15.33:0/4078186561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f51e41542b0 0x7f51e414d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 -- 172.21.15.33:0/4078186561 >> 172.21.15.33:0/4078186561 conn(0x7f51e405e1b0 msgr2=0x7f51e414e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 -- 172.21.15.33:0/4078186561 shutdown_connections 2023-11-06T20:45:03.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.172+0000 7f51e89ff700 1 -- 172.21.15.33:0/4078186561 wait complete. 2023-11-06T20:45:03.174 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T20:45:03.195 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T20:45:03.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.398+0000 7fbade9ff700 1 Processor -- start 2023-11-06T20:45:03.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.398+0000 7fbade9ff700 1 -- start start 2023-11-06T20:45:03.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.399+0000 7fbade9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbad813dff0 0x7fbad8157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:03.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.399+0000 7fbade9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbad80c4b00 0x7fbad80c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:03.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.399+0000 7fbade9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad8064bf0 con 0x7fbad80c4720 2023-11-06T20:45:03.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.399+0000 7fbade9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad805e490 con 0x7fbad813dff0 2023-11-06T20:45:03.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.399+0000 7fbade9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad8062e10 con 0x7fbad80c4b00 2023-11-06T20:45:03.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.400+0000 7fbad77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbad813dff0 0x7fbad8157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:03.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.400+0000 7fbadcf9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbad80c4b00 0x7fbad80c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:03.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.400+0000 7fbad77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbad813dff0 0x7fbad8157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42260/0 (socket says 172.21.15.33:42260) 2023-11-06T20:45:03.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.400+0000 7fbad77fe700 1 -- 172.21.15.33:0/1971440286 learned_addr learned my addr 172.21.15.33:0/1971440286 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:03.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.400+0000 7fbad77fe700 1 -- 172.21.15.33:0/1971440286 >> v2:172.21.15.69:3300/0 conn(0x7fbad80c4b00 msgr2=0x7fbad80c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:03.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.400+0000 7fbad77fe700 1 --2- 172.21.15.33:0/1971440286 >> v2:172.21.15.69:3300/0 conn(0x7fbad80c4b00 0x7fbad80c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:03.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.400+0000 7fbad77fe700 1 -- 172.21.15.33:0/1971440286 >> v1:172.21.15.69:6789/0 conn(0x7fbad80c4720 legacy=0x7fbad813d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:03.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.400+0000 7fbad77fe700 1 -- 172.21.15.33:0/1971440286 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80630f0 con 0x7fbad813dff0 2023-11-06T20:45:03.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.400+0000 7fbad77fe700 1 --2- 172.21.15.33:0/1971440286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 0x7fbad8157e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fbad80c23a0 tx=0x7fbacc0090f0).ready entity=mon.0 client_cookie=9959acd1b75dd38c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:03.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbad6ffd700 1 -- 172.21.15.33:0/1971440286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00b520 con 0x7fbad813dff0 2023-11-06T20:45:03.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbad6ffd700 1 -- 172.21.15.33:0/1971440286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc00b680 con 0x7fbad813dff0 2023-11-06T20:45:03.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbad6ffd700 1 -- 172.21.15.33:0/1971440286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00ba00 con 0x7fbad813dff0 2023-11-06T20:45:03.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbade9ff700 1 -- 172.21.15.33:0/1971440286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 msgr2=0x7fbad8157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:03.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbade9ff700 1 --2- 172.21.15.33:0/1971440286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 0x7fbad8157e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fbad80c23a0 tx=0x7fbacc0090f0).stop 2023-11-06T20:45:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbade9ff700 1 -- 172.21.15.33:0/1971440286 shutdown_connections 2023-11-06T20:45:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbade9ff700 1 --2- 172.21.15.33:0/1971440286 >> v2:172.21.15.69:3300/0 conn(0x7fbad80c4b00 0x7fbad80c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbade9ff700 1 --2- 172.21.15.33:0/1971440286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 0x7fbad8157e20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:03.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbade9ff700 1 -- 172.21.15.33:0/1971440286 >> 172.21.15.33:0/1971440286 conn(0x7fbad80c7dc0 msgr2=0x7fbad80c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:03.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbade9ff700 1 -- 172.21.15.33:0/1971440286 shutdown_connections 2023-11-06T20:45:03.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.401+0000 7fbade9ff700 1 -- 172.21.15.33:0/1971440286 wait complete. 2023-11-06T20:45:03.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.403+0000 7fbade9ff700 1 Processor -- start 2023-11-06T20:45:03.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.403+0000 7fbade9ff700 1 -- start start 2023-11-06T20:45:03.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbade9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80c4720 0x7fbad8136430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:03.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbade9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad80c4b00 0x7fbad8138ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:03.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbade9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 0x7fbad80c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbade9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8134b60 con 0x7fbad813dff0 2023-11-06T20:45:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbade9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8134810 con 0x7fbad80c4b00 2023-11-06T20:45:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbade9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8136e10 con 0x7fbad80c4720 2023-11-06T20:45:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbadcf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 0x7fbad80c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbadcf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 0x7fbad80c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42274/0 (socket says 172.21.15.33:42274) 2023-11-06T20:45:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbadcf9c700 1 -- 172.21.15.33:0/337624422 learned_addr learned my addr 172.21.15.33:0/337624422 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbad7fff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80c4720 0x7fbad8136430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.404+0000 7fbad77fe700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad80c4b00 0x7fbad8138ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbadcf9c700 1 -- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80c4720 msgr2=0x7fbad8136430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbadcf9c700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80c4720 0x7fbad8136430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbadcf9c700 1 -- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad80c4b00 msgr2=0x7fbad8138ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbadcf9c700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad80c4b00 0x7fbad8138ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbadcf9c700 1 -- 172.21.15.33:0/337624422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc009370 con 0x7fbad813dff0 2023-11-06T20:45:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbad7fff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80c4720 0x7fbad8136430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbadcf9c700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 0x7fbad80c5a90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fbad0009e90 tx=0x7fbad000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbabffff700 1 -- 172.21.15.33:0/337624422 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad000cb50 con 0x7fbad813dff0 2023-11-06T20:45:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbabffff700 1 -- 172.21.15.33:0/337624422 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad000f040 con 0x7fbad813dff0 2023-11-06T20:45:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbade9ff700 1 -- 172.21.15.33:0/337624422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad81370f0 con 0x7fbad813dff0 2023-11-06T20:45:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbabffff700 1 -- 172.21.15.33:0/337624422 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad000cec0 con 0x7fbad813dff0 2023-11-06T20:45:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.405+0000 7fbade9ff700 1 -- 172.21.15.33:0/337624422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad80acf70 con 0x7fbad813dff0 2023-11-06T20:45:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.406+0000 7fbade9ff700 1 -- 172.21.15.33:0/337624422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad80ad160 con 0x7fbad813dff0 2023-11-06T20:45:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.407+0000 7fbabffff700 1 -- 172.21.15.33:0/337624422 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fbad0013610 con 0x7fbad813dff0 2023-11-06T20:45:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.408+0000 7fbabffff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbac405f180 0x7fbac4061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.408+0000 7fbabffff700 1 -- 172.21.15.33:0/337624422 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5639+0+0 (secure 0 0 0) 0x7fbad007a8e0 con 0x7fbad813dff0 2023-11-06T20:45:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.408+0000 7fbad7fff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbac405f180 0x7fbac4061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.409+0000 7fbad7fff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbac405f180 0x7fbac4061640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fbac8001600 tx=0x7fbac8009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.413+0000 7fbabffff700 1 -- 172.21.15.33:0/337624422 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbad80ad160 con 0x7fbad813dff0 2023-11-06T20:45:03.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:03.592+0000 7fbade9ff700 1 -- 172.21.15.33:0/337624422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbad80ad160 con 0x7fbad813dff0 2023-11-06T20:45:04.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.177+0000 7fbabffff700 1 -- 172.21.15.33:0/337624422 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbad80ad160 con 0x7fbad813dff0 2023-11-06T20:45:04.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.186+0000 7fbade9ff700 1 -- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbac405f180 msgr2=0x7fbac4061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.186+0000 7fbade9ff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbac405f180 0x7fbac4061640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fbac8001600 tx=0x7fbac8009040).stop 2023-11-06T20:45:04.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.186+0000 7fbade9ff700 1 -- 172.21.15.33:0/337624422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 msgr2=0x7fbad80c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.186+0000 7fbade9ff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 0x7fbad80c5a90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fbad0009e90 tx=0x7fbad000b040).stop 2023-11-06T20:45:04.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.188+0000 7fbade9ff700 1 -- 172.21.15.33:0/337624422 shutdown_connections 2023-11-06T20:45:04.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.188+0000 7fbade9ff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbac405f180 0x7fbac4061640 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.188+0000 7fbade9ff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80c4720 0x7fbad8136430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.188+0000 7fbade9ff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad813dff0 0x7fbad80c5a90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.188+0000 7fbade9ff700 1 --2- 172.21.15.33:0/337624422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad80c4b00 0x7fbad8138ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.188+0000 7fbade9ff700 1 -- 172.21.15.33:0/337624422 >> 172.21.15.33:0/337624422 conn(0x7fbad80c7dc0 msgr2=0x7fbad81570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.188+0000 7fbade9ff700 1 -- 172.21.15.33:0/337624422 shutdown_connections 2023-11-06T20:45:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.188+0000 7fbade9ff700 1 -- 172.21.15.33:0/337624422 wait complete. 2023-11-06T20:45:04.191 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T20:45:04.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T20:45:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.423+0000 7f8d0e7ff700 1 Processor -- start 2023-11-06T20:45:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.423+0000 7f8d0e7ff700 1 -- start start 2023-11-06T20:45:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.425+0000 7f8d0e7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8d08154710 0x7f8d080c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.425+0000 7f8d0e7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d080c6b60 con 0x7f8d08154330 2023-11-06T20:45:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.425+0000 7f8d0e7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d080c7700 con 0x7f8d080c7ff0 2023-11-06T20:45:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.425+0000 7f8d0e7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d080c5cd0 con 0x7f8d08154710 2023-11-06T20:45:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.425+0000 7f8d07fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8d08154330 0x7f8d080c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34094/0 (socket says 172.21.15.33:34094) 2023-11-06T20:45:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.425+0000 7f8d07fff700 1 -- 172.21.15.33:0/1071468239 learned_addr learned my addr 172.21.15.33:0/1071468239 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.425+0000 7f8d077fe700 1 --2- 172.21.15.33:0/1071468239 >> v2:172.21.15.69:3300/0 conn(0x7f8d08154710 0x7f8d080c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.426+0000 7f8d06ffd700 1 -- 172.21.15.33:0/1071468239 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737225816 0 0) 0x7f8d080c6b60 con 0x7f8d08154330 2023-11-06T20:45:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.426+0000 7f8d06ffd700 1 -- 172.21.15.33:0/1071468239 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cf00033f0 con 0x7f8d08154330 2023-11-06T20:45:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.426+0000 7f8d077fe700 1 -- 172.21.15.33:0/1071468239 >> v1:172.21.15.69:6789/0 conn(0x7f8d080c7ff0 legacy=0x7f8d080646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.426+0000 7f8d06ffd700 1 -- 172.21.15.33:0/1071468239 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 60916678 0 0) 0x7f8cf00033f0 con 0x7f8d08154330 2023-11-06T20:45:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.426+0000 7f8d077fe700 1 -- 172.21.15.33:0/1071468239 >> v1:172.21.15.33:6789/0 conn(0x7f8d08154330 legacy=0x7f8d080c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.426+0000 7f8d077fe700 1 -- 172.21.15.33:0/1071468239 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d080c5f50 con 0x7f8d08154710 2023-11-06T20:45:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.426+0000 7f8d077fe700 1 --2- 172.21.15.33:0/1071468239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d08154710 0x7f8d080c57e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8cfc009760 tx=0x7f8cfc009060).ready entity=mon.1 client_cookie=1796609344e3befc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.426+0000 7f8d06ffd700 1 -- 172.21.15.33:0/1071468239 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8001470 con 0x7f8d08154710 2023-11-06T20:45:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.427+0000 7f8d06ffd700 1 -- 172.21.15.33:0/1071468239 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cfc011400 con 0x7f8d08154710 2023-11-06T20:45:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.427+0000 7f8d06ffd700 1 -- 172.21.15.33:0/1071468239 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc011780 con 0x7f8d08154710 2023-11-06T20:45:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.427+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/1071468239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d08154710 msgr2=0x7f8d080c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.427+0000 7f8d0e7ff700 1 --2- 172.21.15.33:0/1071468239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d08154710 0x7f8d080c57e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8cfc009760 tx=0x7f8cfc009060).stop 2023-11-06T20:45:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.427+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/1071468239 shutdown_connections 2023-11-06T20:45:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.427+0000 7f8d0e7ff700 1 --2- 172.21.15.33:0/1071468239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d08154710 0x7f8d080c57e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.427+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/1071468239 >> 172.21.15.33:0/1071468239 conn(0x7f8d080cb910 msgr2=0x7f8d080c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.427+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/1071468239 shutdown_connections 2023-11-06T20:45:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.427+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/1071468239 wait complete. 2023-11-06T20:45:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.429+0000 7f8d0e7ff700 1 Processor -- start 2023-11-06T20:45:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.429+0000 7f8d0e7ff700 1 -- start start 2023-11-06T20:45:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d0e7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d080c7ff0 0x7f8d081578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d0e7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d08154330 0x7f8d08148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d0e7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d08154710 0x7f8d08152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d0e7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08155a00 con 0x7f8d08154330 2023-11-06T20:45:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d0e7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08155ce0 con 0x7f8d080c7ff0 2023-11-06T20:45:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d0e7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08155fc0 con 0x7f8d08154710 2023-11-06T20:45:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d08154330 0x7f8d08148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d08154330 0x7f8d08148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42278/0 (socket says 172.21.15.33:42278) 2023-11-06T20:45:04.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d077fe700 1 -- 172.21.15.33:0/2519671035 learned_addr learned my addr 172.21.15.33:0/2519671035 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:04.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d07fff700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d080c7ff0 0x7f8d081578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d0cd9c700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d08154710 0x7f8d08152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d077fe700 1 -- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d08154710 msgr2=0x7f8d08152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d077fe700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d08154710 0x7f8d08152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d077fe700 1 -- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d080c7ff0 msgr2=0x7f8d081578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d077fe700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d080c7ff0 0x7f8d081578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d077fe700 1 -- 172.21.15.33:0/2519671035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc0092a0 con 0x7f8d08154330 2023-11-06T20:45:04.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.430+0000 7f8d0cd9c700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d08154710 0x7f8d08152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:04.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.431+0000 7f8d077fe700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d08154330 0x7f8d08148ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f8d0806c170 tx=0x7f8cfc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.431+0000 7f8ceffff700 1 -- 172.21.15.33:0/2519671035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc010960 con 0x7f8d08154330 2023-11-06T20:45:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.431+0000 7f8ceffff700 1 -- 172.21.15.33:0/2519671035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cfc010ac0 con 0x7f8d08154330 2023-11-06T20:45:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.431+0000 7f8ceffff700 1 -- 172.21.15.33:0/2519671035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc010e80 con 0x7f8d08154330 2023-11-06T20:45:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.431+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/2519671035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d081562a0 con 0x7f8d08154330 2023-11-06T20:45:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.431+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/2519671035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d08154e30 con 0x7f8d08154330 2023-11-06T20:45:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.432+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/2519671035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d080af8a0 con 0x7f8d08154330 2023-11-06T20:45:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.433+0000 7f8ceffff700 1 -- 172.21.15.33:0/2519671035 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8cfc01d640 con 0x7f8d08154330 2023-11-06T20:45:04.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.434+0000 7f8ceffff700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cf405f150 0x7f8cf4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.434+0000 7f8ceffff700 1 -- 172.21.15.33:0/2519671035 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f8cfc07c2f0 con 0x7f8d08154330 2023-11-06T20:45:04.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.435+0000 7f8d07fff700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cf405f150 0x7f8cf4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.435+0000 7f8d07fff700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cf405f150 0x7f8cf4061610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f8cf80011e0 tx=0x7f8cf8000bc0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:04.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.438+0000 7f8ceffff700 1 -- 172.21.15.33:0/2519671035 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8cfc04d5e0 con 0x7f8d08154330 2023-11-06T20:45:04.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:45:04.508 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:45:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.668+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/2519671035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8d080af540 con 0x7f8d08154330 2023-11-06T20:45:04.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.674+0000 7f8ceffff700 1 -- 172.21.15.33:0/2519671035 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8d080af540 con 0x7f8d08154330 2023-11-06T20:45:04.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.678+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cf405f150 msgr2=0x7f8cf4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.678+0000 7f8d0e7ff700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cf405f150 0x7f8cf4061610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f8cf80011e0 tx=0x7f8cf8000bc0).stop 2023-11-06T20:45:04.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.678+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/2519671035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d08154330 msgr2=0x7f8d08148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.678+0000 7f8d0e7ff700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d08154330 0x7f8d08148ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f8d0806c170 tx=0x7f8cfc014000).stop 2023-11-06T20:45:04.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.679+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/2519671035 shutdown_connections 2023-11-06T20:45:04.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.679+0000 7f8d0e7ff700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cf405f150 0x7f8cf4061610 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.679+0000 7f8d0e7ff700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8d08154710 0x7f8d08152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.679+0000 7f8d0e7ff700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d080c7ff0 0x7f8d081578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.679+0000 7f8d0e7ff700 1 --2- 172.21.15.33:0/2519671035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d08154330 0x7f8d08148ff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.679+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/2519671035 >> 172.21.15.33:0/2519671035 conn(0x7f8d080cb910 msgr2=0x7f8d08153bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:04.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.680+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/2519671035 shutdown_connections 2023-11-06T20:45:04.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.690+0000 7f8d0e7ff700 1 -- 172.21.15.33:0/2519671035 wait complete. 2023-11-06T20:45:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 5 and data pool 6 2023-11-06T20:45:04.725 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:45:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.922+0000 7f42691ff700 1 Processor -- start 2023-11-06T20:45:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.922+0000 7f42691ff700 1 -- start start 2023-11-06T20:45:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.924+0000 7f42691ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4264154280 0x7f4264154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.924+0000 7f42691ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f42640ad6e0 0x7f42640c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.924+0000 7f42691ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42640c5db0 con 0x7f42640a9f20 2023-11-06T20:45:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.924+0000 7f42691ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42640c6930 con 0x7f42640ad6e0 2023-11-06T20:45:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.924+0000 7f42691ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42640c6000 con 0x7f4264154280 2023-11-06T20:45:04.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.925+0000 7f4262d9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4264154280 0x7f4264154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.925+0000 7f426259c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f42640ad6e0 0x7f42640c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.925+0000 7f426359e700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f42640a9f20 0x7f42640c7470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:49314/0 (socket says 172.21.15.33:49314) 2023-11-06T20:45:04.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.925+0000 7f426359e700 1 -- 172.21.15.33:0/3479219104 learned_addr learned my addr 172.21.15.33:0/3479219104 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:04.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.925+0000 7f4261d9b700 1 -- 172.21.15.33:0/3479219104 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272310712 0 0) 0x7f42640c5db0 con 0x7f42640a9f20 2023-11-06T20:45:04.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.925+0000 7f4262d9d700 1 -- 172.21.15.33:0/3479219104 >> v2:172.21.15.69:3300/0 conn(0x7f42640ad6e0 msgr2=0x7f42640c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:04.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.925+0000 7f4262d9d700 1 --2- 172.21.15.33:0/3479219104 >> v2:172.21.15.69:3300/0 conn(0x7f42640ad6e0 0x7f42640c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.925+0000 7f4262d9d700 1 -- 172.21.15.33:0/3479219104 >> v1:172.21.15.69:6789/0 conn(0x7f42640a9f20 legacy=0x7f42640c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.925+0000 7f4262d9d700 1 -- 172.21.15.33:0/3479219104 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42640c62e0 con 0x7f4264154280 2023-11-06T20:45:04.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.925+0000 7f4262d9d700 1 --2- 172.21.15.33:0/3479219104 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4264154280 0x7f4264154660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f4254009790 tx=0x7f4254009060).ready entity=mon.2 client_cookie=604ad37b9fdaff37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:04.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f4261d9b700 1 -- 172.21.15.33:0/3479219104 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425400f510 con 0x7f4264154280 2023-11-06T20:45:04.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f4261d9b700 1 -- 172.21.15.33:0/3479219104 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425400f670 con 0x7f4264154280 2023-11-06T20:45:04.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f4261d9b700 1 -- 172.21.15.33:0/3479219104 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425400f9f0 con 0x7f4264154280 2023-11-06T20:45:04.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f42691ff700 1 -- 172.21.15.33:0/3479219104 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4264154280 msgr2=0x7f4264154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f42691ff700 1 --2- 172.21.15.33:0/3479219104 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4264154280 0x7f4264154660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f4254009790 tx=0x7f4254009060).stop 2023-11-06T20:45:04.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f42691ff700 1 -- 172.21.15.33:0/3479219104 shutdown_connections 2023-11-06T20:45:04.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f42691ff700 1 --2- 172.21.15.33:0/3479219104 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4264154280 0x7f4264154660 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f42691ff700 1 --2- 172.21.15.33:0/3479219104 >> v2:172.21.15.69:3300/0 conn(0x7f42640ad6e0 0x7f42640c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f42691ff700 1 -- 172.21.15.33:0/3479219104 >> 172.21.15.33:0/3479219104 conn(0x7f426405e1b0 msgr2=0x7f426405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:04.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f42691ff700 1 -- 172.21.15.33:0/3479219104 shutdown_connections 2023-11-06T20:45:04.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.926+0000 7f42691ff700 1 -- 172.21.15.33:0/3479219104 wait complete. 2023-11-06T20:45:04.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.928+0000 7f42691ff700 1 Processor -- start 2023-11-06T20:45:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.928+0000 7f42691ff700 1 -- start start 2023-11-06T20:45:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.929+0000 7f42691ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42640a9f20 0x7f42640ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.929+0000 7f42691ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42640ad6e0 0x7f42640a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.929+0000 7f42691ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4264154280 0x7f42640a2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.929+0000 7f42691ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42640a2540 con 0x7f42640ad6e0 2023-11-06T20:45:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.929+0000 7f42691ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42641659a0 con 0x7f4264154280 2023-11-06T20:45:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.929+0000 7f42691ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42640cb700 con 0x7f42640a9f20 2023-11-06T20:45:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.929+0000 7f426259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42640ad6e0 0x7f42640a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.929+0000 7f426259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42640ad6e0 0x7f42640a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42294/0 (socket says 172.21.15.33:42294) 2023-11-06T20:45:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.929+0000 7f426259c700 1 -- 172.21.15.33:0/3861645589 learned_addr learned my addr 172.21.15.33:0/3861645589 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f426359e700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4264154280 0x7f42640a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f4262d9d700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42640a9f20 0x7f42640ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f426259c700 1 -- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42640a9f20 msgr2=0x7f42640ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f426259c700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42640a9f20 0x7f42640ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f426259c700 1 -- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4264154280 msgr2=0x7f42640a2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f426259c700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4264154280 0x7f42640a2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f426259c700 1 -- 172.21.15.33:0/3861645589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42540092a0 con 0x7f42640ad6e0 2023-11-06T20:45:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f426259c700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42640ad6e0 0x7f42640a1b60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f425800e8d0 tx=0x7f4258009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f424affd700 1 -- 172.21.15.33:0/3861645589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42580094b0 con 0x7f42640ad6e0 2023-11-06T20:45:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f424affd700 1 -- 172.21.15.33:0/3861645589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4258012600 con 0x7f42640ad6e0 2023-11-06T20:45:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f424affd700 1 -- 172.21.15.33:0/3861645589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258012980 con 0x7f42640ad6e0 2023-11-06T20:45:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.930+0000 7f42691ff700 1 -- 172.21.15.33:0/3861645589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42640cae60 con 0x7f42640ad6e0 2023-11-06T20:45:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.931+0000 7f42691ff700 1 -- 172.21.15.33:0/3861645589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42640cb420 con 0x7f42640ad6e0 2023-11-06T20:45:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.932+0000 7f42691ff700 1 -- 172.21.15.33:0/3861645589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4264165b30 con 0x7f42640ad6e0 2023-11-06T20:45:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.933+0000 7f424affd700 1 -- 172.21.15.33:0/3861645589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4258012cf0 con 0x7f42640ad6e0 2023-11-06T20:45:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.933+0000 7f424affd700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425005f150 0x7f4250061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.933+0000 7f424affd700 1 -- 172.21.15.33:0/3861645589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f425807a600 con 0x7f42640ad6e0 2023-11-06T20:45:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.934+0000 7f4262d9d700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425005f150 0x7f4250061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.934+0000 7f4262d9d700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425005f150 0x7f4250061610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f4254009790 tx=0x7f4254017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:04.937+0000 7f424affd700 1 -- 172.21.15.33:0/3861645589 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f425804b810 con 0x7f42640ad6e0 2023-11-06T20:45:05.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.124+0000 7f42691ff700 1 -- 172.21.15.33:0/3861645589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4264064090 con 0x7f42640ad6e0 2023-11-06T20:45:05.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.125+0000 7f424affd700 1 -- 172.21.15.33:0/3861645589 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f4258016020 con 0x7f42640ad6e0 2023-11-06T20:45:05.124 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:05.125 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:45:04.668947+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:45:02.985899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T20:45:03.637532+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:45:05.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 -- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425005f150 msgr2=0x7f4250061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425005f150 0x7f4250061610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f4254009790 tx=0x7f4254017000).stop 2023-11-06T20:45:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 -- 172.21.15.33:0/3861645589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42640ad6e0 msgr2=0x7f42640a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42640ad6e0 0x7f42640a1b60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f425800e8d0 tx=0x7f4258009060).stop 2023-11-06T20:45:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 -- 172.21.15.33:0/3861645589 shutdown_connections 2023-11-06T20:45:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425005f150 0x7f4250061610 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42640a9f20 0x7f42640ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4264154280 0x7f42640a2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 --2- 172.21.15.33:0/3861645589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42640ad6e0 0x7f42640a1b60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 -- 172.21.15.33:0/3861645589 >> 172.21.15.33:0/3861645589 conn(0x7f426405e1b0 msgr2=0x7f42641535b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:05.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 -- 172.21.15.33:0/3861645589 shutdown_connections 2023-11-06T20:45:05.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.127+0000 7f42691ff700 1 -- 172.21.15.33:0/3861645589 wait complete. 2023-11-06T20:45:05.144 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:45:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.341+0000 7f9e8a7ff700 1 Processor -- start 2023-11-06T20:45:05.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.341+0000 7f9e8a7ff700 1 -- start start 2023-11-06T20:45:05.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.343+0000 7f9e8a7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9e84137660 0x7f9e840c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.343+0000 7f9e8a7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9e84069410 0x7f9e84069860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.343+0000 7f9e8a7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e840c17e0 con 0x7f9e840c3600 2023-11-06T20:45:05.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.343+0000 7f9e8a7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8406a740 con 0x7f9e84137660 2023-11-06T20:45:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.343+0000 7f9e8a7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8406a180 con 0x7f9e84069410 2023-11-06T20:45:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.343+0000 7f9e837fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9e84137660 0x7f9e840c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.343+0000 7f9e837fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9e84137660 0x7f9e840c39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42308/0 (socket says 172.21.15.33:42308) 2023-11-06T20:45:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.343+0000 7f9e837fe700 1 -- 172.21.15.33:0/3176014186 learned_addr learned my addr 172.21.15.33:0/3176014186 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:05.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.343+0000 7f9e83fff700 1 --1- 172.21.15.33:0/3176014186 >> v1:172.21.15.33:6789/0 conn(0x7f9e840c3600 0x7f9e84136f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34104/0 (socket says 172.21.15.33:34104) 2023-11-06T20:45:05.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e88d9c700 1 --2- 172.21.15.33:0/3176014186 >> v2:172.21.15.69:3300/0 conn(0x7f9e84069410 0x7f9e84069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e837fe700 1 -- 172.21.15.33:0/3176014186 >> v2:172.21.15.69:3300/0 conn(0x7f9e84069410 msgr2=0x7f9e84069860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:05.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e837fe700 1 --2- 172.21.15.33:0/3176014186 >> v2:172.21.15.69:3300/0 conn(0x7f9e84069410 0x7f9e84069860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e837fe700 1 -- 172.21.15.33:0/3176014186 >> v1:172.21.15.33:6789/0 conn(0x7f9e840c3600 legacy=0x7f9e84136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e837fe700 1 -- 172.21.15.33:0/3176014186 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8406a460 con 0x7f9e84137660 2023-11-06T20:45:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e88d9c700 1 --2- 172.21.15.33:0/3176014186 >> v2:172.21.15.69:3300/0 conn(0x7f9e84069410 0x7f9e84069860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e82ffd700 1 -- 172.21.15.33:0/3176014186 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867404981 0 0) 0x7f9e840c17e0 con 0x7f9e840c3600 2023-11-06T20:45:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e837fe700 1 --2- 172.21.15.33:0/3176014186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e84137660 0x7f9e840c39e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f9e84137b40 tx=0x7f9e780090f0).ready entity=mon.0 client_cookie=d71f81f04e9d0e2c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:05.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e82ffd700 1 -- 172.21.15.33:0/3176014186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800b600 con 0x7f9e84137660 2023-11-06T20:45:05.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e82ffd700 1 -- 172.21.15.33:0/3176014186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7800b760 con 0x7f9e84137660 2023-11-06T20:45:05.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.344+0000 7f9e82ffd700 1 -- 172.21.15.33:0/3176014186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800bae0 con 0x7f9e84137660 2023-11-06T20:45:05.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.345+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/3176014186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e84137660 msgr2=0x7f9e840c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.345+0000 7f9e8a7ff700 1 --2- 172.21.15.33:0/3176014186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e84137660 0x7f9e840c39e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f9e84137b40 tx=0x7f9e780090f0).stop 2023-11-06T20:45:05.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.345+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/3176014186 shutdown_connections 2023-11-06T20:45:05.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.345+0000 7f9e8a7ff700 1 --2- 172.21.15.33:0/3176014186 >> v2:172.21.15.69:3300/0 conn(0x7f9e84069410 0x7f9e84069860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.345+0000 7f9e8a7ff700 1 --2- 172.21.15.33:0/3176014186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e84137660 0x7f9e840c39e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.345+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/3176014186 >> 172.21.15.33:0/3176014186 conn(0x7f9e84062710 msgr2=0x7f9e84062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.345+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/3176014186 shutdown_connections 2023-11-06T20:45:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.345+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/3176014186 wait complete. 2023-11-06T20:45:05.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.347+0000 7f9e8a7ff700 1 Processor -- start 2023-11-06T20:45:05.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.347+0000 7f9e8a7ff700 1 -- start start 2023-11-06T20:45:05.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e8a7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e84069410 0x7f9e840c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e8a7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e840c3600 0x7f9e840bdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e8a7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e84137660 0x7f9e8414d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e8a7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e84167940 con 0x7f9e840c3600 2023-11-06T20:45:05.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e8a7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e84167c20 con 0x7f9e84069410 2023-11-06T20:45:05.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e8a7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e84167f00 con 0x7f9e84137660 2023-11-06T20:45:05.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e840c3600 0x7f9e840bdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e840c3600 0x7f9e840bdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42310/0 (socket says 172.21.15.33:42310) 2023-11-06T20:45:05.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e837fe700 1 -- 172.21.15.33:0/1567774049 learned_addr learned my addr 172.21.15.33:0/1567774049 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:05.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e83fff700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e84069410 0x7f9e840c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e88d9c700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e84137660 0x7f9e8414d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e837fe700 1 -- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e84137660 msgr2=0x7f9e8414d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e837fe700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e84137660 0x7f9e8414d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e837fe700 1 -- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e84069410 msgr2=0x7f9e840c0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e837fe700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e84069410 0x7f9e840c0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e837fe700 1 -- 172.21.15.33:0/1567774049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e78009370 con 0x7f9e840c3600 2023-11-06T20:45:05.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.348+0000 7f9e88d9c700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e84137660 0x7f9e8414d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:05.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.349+0000 7f9e837fe700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e840c3600 0x7f9e840bdad0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9e840c4600 tx=0x7f9e78017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:05.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.349+0000 7f9e6bfff700 1 -- 172.21.15.33:0/1567774049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800bae0 con 0x7f9e840c3600 2023-11-06T20:45:05.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.349+0000 7f9e6bfff700 1 -- 172.21.15.33:0/1567774049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e78000c90 con 0x7f9e840c3600 2023-11-06T20:45:05.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.349+0000 7f9e6bfff700 1 -- 172.21.15.33:0/1567774049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7801b620 con 0x7f9e840c3600 2023-11-06T20:45:05.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.349+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/1567774049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e841681e0 con 0x7f9e840c3600 2023-11-06T20:45:05.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.349+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/1567774049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8414d990 con 0x7f9e840c3600 2023-11-06T20:45:05.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.350+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/1567774049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e84168700 con 0x7f9e840c3600 2023-11-06T20:45:05.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.351+0000 7f9e6bfff700 1 -- 172.21.15.33:0/1567774049 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9e7801ba10 con 0x7f9e840c3600 2023-11-06T20:45:05.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.351+0000 7f9e6bfff700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9e7005f150 0x7f9e70061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.352+0000 7f9e6bfff700 1 -- 172.21.15.33:0/1567774049 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9e7807be80 con 0x7f9e840c3600 2023-11-06T20:45:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.352+0000 7f9e83fff700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9e7005f150 0x7f9e70061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.352+0000 7f9e83fff700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9e7005f150 0x7f9e70061610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f9e740011e0 tx=0x7f9e74007180).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.356+0000 7f9e6bfff700 1 -- 172.21.15.33:0/1567774049 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9e84168700 con 0x7f9e840c3600 2023-11-06T20:45:05.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.539+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/1567774049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e84168700 con 0x7f9e840c3600 2023-11-06T20:45:05.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.540+0000 7f9e6bfff700 1 -- 172.21.15.33:0/1567774049 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f9e84168700 con 0x7f9e840c3600 2023-11-06T20:45:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:05.540 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:45:04.668947+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:45:02.985899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T20:45:03.637532+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:45:05.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9e7005f150 msgr2=0x7f9e70061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9e7005f150 0x7f9e70061610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f9e740011e0 tx=0x7f9e74007180).stop 2023-11-06T20:45:05.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/1567774049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e840c3600 msgr2=0x7f9e840bdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e840c3600 0x7f9e840bdad0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9e840c4600 tx=0x7f9e78017000).stop 2023-11-06T20:45:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/1567774049 shutdown_connections 2023-11-06T20:45:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9e7005f150 0x7f9e70061610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9e84137660 0x7f9e8414d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e84069410 0x7f9e840c0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 --2- 172.21.15.33:0/1567774049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e840c3600 0x7f9e840bdad0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/1567774049 >> 172.21.15.33:0/1567774049 conn(0x7f9e84062710 msgr2=0x7f9e84136140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:05.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/1567774049 shutdown_connections 2023-11-06T20:45:05.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.543+0000 7f9e8a7ff700 1 -- 172.21.15.33:0/1567774049 wait complete. 2023-11-06T20:45:05.559 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T20:45:05.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.761+0000 7f00761ff700 1 Processor -- start 2023-11-06T20:45:05.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.761+0000 7f00761ff700 1 -- start start 2023-11-06T20:45:05.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f00761ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0070154330 0x7f00700c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f00761ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f00700c7ff0 0x7f00700646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f00761ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00700c6b60 con 0x7f0070154710 2023-11-06T20:45:05.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f00761ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00700c7700 con 0x7f00700c7ff0 2023-11-06T20:45:05.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f00761ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00700c5cd0 con 0x7f0070154330 2023-11-06T20:45:05.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f006effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0070154710 0x7f00700c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34114/0 (socket says 172.21.15.33:34114) 2023-11-06T20:45:05.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f006ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f00700c7ff0 0x7f00700646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f006effd700 1 -- 172.21.15.33:0/602147555 learned_addr learned my addr 172.21.15.33:0/602147555 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f006f7fe700 1 --2- 172.21.15.33:0/602147555 >> v2:172.21.15.69:3300/0 conn(0x7f0070154330 0x7f00700c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f006ffff700 1 -- 172.21.15.33:0/602147555 >> v2:172.21.15.69:3300/0 conn(0x7f0070154330 msgr2=0x7f00700c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:05.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f006ffff700 1 --2- 172.21.15.33:0/602147555 >> v2:172.21.15.69:3300/0 conn(0x7f0070154330 0x7f00700c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f006ffff700 1 -- 172.21.15.33:0/602147555 >> v1:172.21.15.33:6789/0 conn(0x7f0070154710 legacy=0x7f00700c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f006ffff700 1 -- 172.21.15.33:0/602147555 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00700c5f50 con 0x7f00700c7ff0 2023-11-06T20:45:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.763+0000 7f006f7fe700 1 --2- 172.21.15.33:0/602147555 >> v2:172.21.15.69:3300/0 conn(0x7f0070154330 0x7f00700c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.764+0000 7f006ffff700 1 --2- 172.21.15.33:0/602147555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00700c7ff0 0x7f00700646c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f00700605d0 tx=0x7f0068009780).ready entity=mon.0 client_cookie=4b2e6a867beeb08 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.764+0000 7f006e7fc700 1 -- 172.21.15.33:0/602147555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006800e070 con 0x7f00700c7ff0 2023-11-06T20:45:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.764+0000 7f006e7fc700 1 -- 172.21.15.33:0/602147555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006800bc30 con 0x7f00700c7ff0 2023-11-06T20:45:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.764+0000 7f006e7fc700 1 -- 172.21.15.33:0/602147555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068012410 con 0x7f00700c7ff0 2023-11-06T20:45:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.764+0000 7f00761ff700 1 -- 172.21.15.33:0/602147555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00700c7ff0 msgr2=0x7f00700646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.764+0000 7f00761ff700 1 --2- 172.21.15.33:0/602147555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00700c7ff0 0x7f00700646c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f00700605d0 tx=0x7f0068009780).stop 2023-11-06T20:45:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.765+0000 7f00761ff700 1 -- 172.21.15.33:0/602147555 shutdown_connections 2023-11-06T20:45:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.765+0000 7f00761ff700 1 --2- 172.21.15.33:0/602147555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00700c7ff0 0x7f00700646c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.765+0000 7f00761ff700 1 --2- 172.21.15.33:0/602147555 >> v2:172.21.15.69:3300/0 conn(0x7f0070154330 0x7f00700c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.765+0000 7f00761ff700 1 -- 172.21.15.33:0/602147555 >> 172.21.15.33:0/602147555 conn(0x7f00700cb910 msgr2=0x7f00700c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:05.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.765+0000 7f00761ff700 1 -- 172.21.15.33:0/602147555 shutdown_connections 2023-11-06T20:45:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.765+0000 7f00761ff700 1 -- 172.21.15.33:0/602147555 wait complete. 2023-11-06T20:45:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.766+0000 7f00761ff700 1 Processor -- start 2023-11-06T20:45:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.766+0000 7f00761ff700 1 -- start start 2023-11-06T20:45:05.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.767+0000 7f00761ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00700c7ff0 0x7f0070149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.767+0000 7f00761ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070154330 0x7f0070156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.767+0000 7f00761ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0070154710 0x7f00701574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.767+0000 7f00761ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00700ae920 con 0x7f0070154330 2023-11-06T20:45:05.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.767+0000 7f00761ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0070157a80 con 0x7f00700c7ff0 2023-11-06T20:45:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.767+0000 7f00761ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00700aebc0 con 0x7f0070154710 2023-11-06T20:45:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.767+0000 7f006effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070154330 0x7f0070156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.767+0000 7f006effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070154330 0x7f0070156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42328/0 (socket says 172.21.15.33:42328) 2023-11-06T20:45:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.767+0000 7f006effd700 1 -- 172.21.15.33:0/1034780683 learned_addr learned my addr 172.21.15.33:0/1034780683 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f006ffff700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0070154710 0x7f00701574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f006f7fe700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00700c7ff0 0x7f0070149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f006effd700 1 -- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0070154710 msgr2=0x7f00701574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f006effd700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0070154710 0x7f00701574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f006effd700 1 -- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00700c7ff0 msgr2=0x7f0070149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f006effd700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00700c7ff0 0x7f0070149810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f006effd700 1 -- 172.21.15.33:0/1034780683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068009a20 con 0x7f0070154330 2023-11-06T20:45:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f006f7fe700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00700c7ff0 0x7f0070149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f006effd700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070154330 0x7f0070156fc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f006400c440 tx=0x7f0064007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f00577fe700 1 -- 172.21.15.33:0/1034780683 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006400ce40 con 0x7f0070154330 2023-11-06T20:45:05.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f00577fe700 1 -- 172.21.15.33:0/1034780683 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00640072e0 con 0x7f0070154330 2023-11-06T20:45:05.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f00761ff700 1 -- 172.21.15.33:0/1034780683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00700aeea0 con 0x7f0070154330 2023-11-06T20:45:05.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f00577fe700 1 -- 172.21.15.33:0/1034780683 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064007660 con 0x7f0070154330 2023-11-06T20:45:05.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.768+0000 7f00761ff700 1 -- 172.21.15.33:0/1034780683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0070149d00 con 0x7f0070154330 2023-11-06T20:45:05.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.769+0000 7f00761ff700 1 -- 172.21.15.33:0/1034780683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00700830b0 con 0x7f0070154330 2023-11-06T20:45:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.770+0000 7f00577fe700 1 -- 172.21.15.33:0/1034780683 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f00640059b0 con 0x7f0070154330 2023-11-06T20:45:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.771+0000 7f00577fe700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f150 0x7f005c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.771+0000 7f00577fe700 1 -- 172.21.15.33:0/1034780683 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0064079920 con 0x7f0070154330 2023-11-06T20:45:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.771+0000 7f006f7fe700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f150 0x7f005c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.772+0000 7f006f7fe700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f150 0x7f005c061610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f006000aa90 tx=0x7f0060008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:05.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:05.775+0000 7f00577fe700 1 -- 172.21.15.33:0/1034780683 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f006404ab30 con 0x7f0070154330 2023-11-06T20:45:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.006+0000 7f00761ff700 1 -- 172.21.15.33:0/1034780683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f00700652d0 con 0x7f0070154330 2023-11-06T20:45:06.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.687+0000 7f00577fe700 1 -- 172.21.15.33:0/1034780683 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f006404e150 con 0x7f0070154330 2023-11-06T20:45:06.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 -- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f150 msgr2=0x7f005c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:06.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f150 0x7f005c061610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f006000aa90 tx=0x7f0060008040).stop 2023-11-06T20:45:06.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 -- 172.21.15.33:0/1034780683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070154330 msgr2=0x7f0070156fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:06.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070154330 0x7f0070156fc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f006400c440 tx=0x7f0064007e20).stop 2023-11-06T20:45:06.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 -- 172.21.15.33:0/1034780683 shutdown_connections 2023-11-06T20:45:06.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f150 0x7f005c061610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:06.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0070154710 0x7f00701574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:06.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00700c7ff0 0x7f0070149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:06.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 --2- 172.21.15.33:0/1034780683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070154330 0x7f0070156fc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:06.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 -- 172.21.15.33:0/1034780683 >> 172.21.15.33:0/1034780683 conn(0x7f00700cb910 msgr2=0x7f00700a9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:06.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 -- 172.21.15.33:0/1034780683 shutdown_connections 2023-11-06T20:45:06.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.690+0000 7f00761ff700 1 -- 172.21.15.33:0/1034780683 wait complete. 2023-11-06T20:45:06.709 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T20:45:06.710 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T20:45:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.914+0000 7f32a65ff700 1 Processor -- start 2023-11-06T20:45:06.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.915+0000 7f32a65ff700 1 -- start start 2023-11-06T20:45:06.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.916+0000 7f32a65ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f32a0154330 0x7f32a00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:06.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.916+0000 7f32a65ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a00c6b60 con 0x7f32a0154710 2023-11-06T20:45:06.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.916+0000 7f32a65ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a00c7700 con 0x7f32a00c7ff0 2023-11-06T20:45:06.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.916+0000 7f32a65ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c5cd0 con 0x7f32a0154330 2023-11-06T20:45:06.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.916+0000 7f329f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f32a0154710 0x7f32a00c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34122/0 (socket says 172.21.15.33:34122) 2023-11-06T20:45:06.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.916+0000 7f329f7fe700 1 -- 172.21.15.33:0/581671356 learned_addr learned my addr 172.21.15.33:0/581671356 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:06.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329ffff700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f32a0154330 0x7f32a00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:06.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770696120 0 0) 0x7f32a00c6b60 con 0x7f32a0154710 2023-11-06T20:45:06.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32880033f0 con 0x7f32a0154710 2023-11-06T20:45:06.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3622395503 0 0) 0x7f32880033f0 con 0x7f32a0154710 2023-11-06T20:45:06.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f32a0154330 msgr2=0x7f32a00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:06.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f32a0154330 0x7f32a00c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:06.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 >> v1:172.21.15.69:6789/0 conn(0x7f32a00c7ff0 legacy=0x7f32a00646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:06.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00c5f50 con 0x7f32a0154710 2023-11-06T20:45:06.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f3294001470 con 0x7f32a0154710 2023-11-06T20:45:06.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 >> v1:172.21.15.33:6789/0 conn(0x7f32a0154710 legacy=0x7f32a00c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3288003910 0x7f3288003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3288004690 0x7f3288004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f32a0154330 0x7f3288005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c5cd0 con 0x7f3288003910 2023-11-06T20:45:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328800b8e0 con 0x7f3288004690 2023-11-06T20:45:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.917+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328800ba50 con 0x7f32a0154330 2023-11-06T20:45:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.918+0000 7f329ffff700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3288003910 0x7f3288003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.918+0000 7f329f7fe700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3288004690 0x7f3288004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:06.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.918+0000 7f32a4b9c700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f32a0154330 0x7f3288005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.918+0000 7f329ffff700 1 -- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f32a0154330 msgr2=0x7f3288005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.918+0000 7f329ffff700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f32a0154330 0x7f3288005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.918+0000 7f329ffff700 1 -- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3288004690 msgr2=0x7f3288004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:06.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.918+0000 7f329ffff700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3288004690 0x7f3288004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.918+0000 7f329ffff700 1 -- 172.21.15.33:0/581671356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32880033f0 con 0x7f3288003910 2023-11-06T20:45:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.918+0000 7f329ffff700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3288003910 0x7f3288003cf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f3290009a80 tx=0x7f3290009060).ready entity=mon.0 client_cookie=db6c646564b86767 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.919+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3290009430 con 0x7f3288003910 2023-11-06T20:45:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.919+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329000e5e0 con 0x7f3288003910 2023-11-06T20:45:06.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.919+0000 7f329effd700 1 -- 172.21.15.33:0/581671356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329000e960 con 0x7f3288003910 2023-11-06T20:45:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.919+0000 7f32a65ff700 1 -- 172.21.15.33:0/581671356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3288003910 msgr2=0x7f3288003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.919+0000 7f32a65ff700 1 --2- 172.21.15.33:0/581671356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3288003910 0x7f3288003cf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f3290009a80 tx=0x7f3290009060).stop 2023-11-06T20:45:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.919+0000 7f32a4b9c700 1 -- 172.21.15.33:0/581671356 reap_dead start 2023-11-06T20:45:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.919+0000 7f32a65ff700 1 -- 172.21.15.33:0/581671356 shutdown_connections 2023-11-06T20:45:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.919+0000 7f32a65ff700 1 -- 172.21.15.33:0/581671356 >> 172.21.15.33:0/581671356 conn(0x7f32a00cb910 msgr2=0x7f32a00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.919+0000 7f32a65ff700 1 -- 172.21.15.33:0/581671356 shutdown_connections 2023-11-06T20:45:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.919+0000 7f32a65ff700 1 -- 172.21.15.33:0/581671356 wait complete. 2023-11-06T20:45:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.921+0000 7f32a65ff700 1 Processor -- start 2023-11-06T20:45:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.921+0000 7f32a65ff700 1 -- start start 2023-11-06T20:45:06.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f32a65ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3288003910 0x7f32a0156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:06.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f32a65ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a013d7e0 0x7f32a013dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:06.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f32a65ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32a0154ba0 0x7f32a0155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:06.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f32a65ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a0156e20 con 0x7f32a013d7e0 2023-11-06T20:45:06.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f32a65ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a0155890 con 0x7f32a0154ba0 2023-11-06T20:45:06.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f32a65ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a0155b70 con 0x7f3288003910 2023-11-06T20:45:06.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f329f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a013d7e0 0x7f32a013dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:06.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f329f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a013d7e0 0x7f32a013dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42344/0 (socket says 172.21.15.33:42344) 2023-11-06T20:45:06.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f329f7fe700 1 -- 172.21.15.33:0/1302226680 learned_addr learned my addr 172.21.15.33:0/1302226680 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:06.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f329ffff700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3288003910 0x7f32a0156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:06.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f329f7fe700 1 -- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3288003910 msgr2=0x7f32a0156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:06.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f329f7fe700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3288003910 0x7f32a0156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:06.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f329f7fe700 1 -- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32a0154ba0 msgr2=0x7f32a0155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:06.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f32a4b9c700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32a0154ba0 0x7f32a0155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:06.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f329f7fe700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32a0154ba0 0x7f32a0155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:06.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f329f7fe700 1 -- 172.21.15.33:0/1302226680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0155e50 con 0x7f32a013d7e0 2023-11-06T20:45:06.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.922+0000 7f32a4b9c700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32a0154ba0 0x7f32a0155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:06.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.923+0000 7f329f7fe700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a013d7e0 0x7f32a013dc60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f32940047e0 tx=0x7f3294001e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:06.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.923+0000 7f3287fff700 1 -- 172.21.15.33:0/1302226680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400f5b0 con 0x7f32a013d7e0 2023-11-06T20:45:06.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.923+0000 7f3287fff700 1 -- 172.21.15.33:0/1302226680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329400f710 con 0x7f32a013d7e0 2023-11-06T20:45:06.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.923+0000 7f32a65ff700 1 -- 172.21.15.33:0/1302226680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a0156130 con 0x7f32a013d7e0 2023-11-06T20:45:06.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.923+0000 7f3287fff700 1 -- 172.21.15.33:0/1302226680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400fa90 con 0x7f32a013d7e0 2023-11-06T20:45:06.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.923+0000 7f32a65ff700 1 -- 172.21.15.33:0/1302226680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a0155510 con 0x7f32a013d7e0 2023-11-06T20:45:06.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.924+0000 7f32a65ff700 1 -- 172.21.15.33:0/1302226680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a0156650 con 0x7f32a013d7e0 2023-11-06T20:45:06.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.925+0000 7f3287fff700 1 -- 172.21.15.33:0/1302226680 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f329401d3b0 con 0x7f32a013d7e0 2023-11-06T20:45:06.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.925+0000 7f3287fff700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f328c05f150 0x7f328c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.926+0000 7f3287fff700 1 -- 172.21.15.33:0/1302226680 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3294079ce0 con 0x7f32a013d7e0 2023-11-06T20:45:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.926+0000 7f329ffff700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f328c05f150 0x7f328c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.926+0000 7f329ffff700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f328c05f150 0x7f328c061610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f3290006d50 tx=0x7f329001a000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:06.930+0000 7f3287fff700 1 -- 172.21.15.33:0/1302226680 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f32a0156650 con 0x7f32a013d7e0 2023-11-06T20:45:07.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.154+0000 7f32a65ff700 1 -- 172.21.15.33:0/1302226680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f32a0156650 con 0x7f32a013d7e0 2023-11-06T20:45:07.158 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-11-06T20:45:07.159+0000 7f940fbf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T20:45:07.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.780+0000 7f3287fff700 1 -- 172.21.15.33:0/1302226680 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f32a0156650 con 0x7f32a013d7e0 2023-11-06T20:45:07.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 -- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f328c05f150 msgr2=0x7f328c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:07.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f328c05f150 0x7f328c061610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f3290006d50 tx=0x7f329001a000).stop 2023-11-06T20:45:07.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 -- 172.21.15.33:0/1302226680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a013d7e0 msgr2=0x7f32a013dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:07.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a013d7e0 0x7f32a013dc60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f32940047e0 tx=0x7f3294001e70).stop 2023-11-06T20:45:07.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 -- 172.21.15.33:0/1302226680 shutdown_connections 2023-11-06T20:45:07.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f328c05f150 0x7f328c061610 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:07.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3288003910 0x7f32a0156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:07.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32a0154ba0 0x7f32a0155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:07.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 --2- 172.21.15.33:0/1302226680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a013d7e0 0x7f32a013dc60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:07.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 -- 172.21.15.33:0/1302226680 >> 172.21.15.33:0/1302226680 conn(0x7f32a00cb910 msgr2=0x7f32a00ab610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:07.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.783+0000 7f32a65ff700 1 -- 172.21.15.33:0/1302226680 shutdown_connections 2023-11-06T20:45:07.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:07.784+0000 7f32a65ff700 1 -- 172.21.15.33:0/1302226680 wait complete. 2023-11-06T20:45:07.803 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T20:45:07.803 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:45:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.005+0000 7f8c6b9ff700 1 Processor -- start 2023-11-06T20:45:08.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.005+0000 7f8c6b9ff700 1 -- start start 2023-11-06T20:45:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.006+0000 7f8c6b9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c640c5d50 0x7f8c640c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.006+0000 7f8c6b9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c641554a0 con 0x7f8c640c1840 2023-11-06T20:45:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.006+0000 7f8c6b9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c640c66c0 con 0x7f8c640bfd70 2023-11-06T20:45:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.006+0000 7f8c6b9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c64154960 con 0x7f8c640c5d50 2023-11-06T20:45:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.006+0000 7f8c68f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c640c5d50 0x7f8c640c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.006+0000 7f8c6979b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8c640c1840 0x7f8c640a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57956/0 (socket says 172.21.15.33:57956) 2023-11-06T20:45:08.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.006+0000 7f8c6979b700 1 -- 172.21.15.33:0/2976477509 learned_addr learned my addr 172.21.15.33:0/2976477509 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.007+0000 7f8c68f9a700 1 -- 172.21.15.33:0/2976477509 >> v1:172.21.15.69:6789/0 conn(0x7f8c640bfd70 legacy=0x7f8c640c0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.007+0000 7f8c68f9a700 1 -- 172.21.15.33:0/2976477509 >> v1:172.21.15.33:6789/0 conn(0x7f8c640c1840 legacy=0x7f8c640a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.007+0000 7f8c68f9a700 1 -- 172.21.15.33:0/2976477509 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c64155930 con 0x7f8c640c5d50 2023-11-06T20:45:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.007+0000 7f8c5bfff700 1 -- 172.21.15.33:0/2976477509 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960798171 0 0) 0x7f8c641554a0 con 0x7f8c640c1840 2023-11-06T20:45:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.007+0000 7f8c68f9a700 1 --2- 172.21.15.33:0/2976477509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640c5d50 0x7f8c640c61d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f8c54009760 tx=0x7f8c54009060).ready entity=mon.0 client_cookie=f27a05e5884defb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.007+0000 7f8c5bfff700 1 -- 172.21.15.33:0/2976477509 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400b580 con 0x7f8c640c5d50 2023-11-06T20:45:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.007+0000 7f8c5bfff700 1 -- 172.21.15.33:0/2976477509 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5400b6e0 con 0x7f8c640c5d50 2023-11-06T20:45:08.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.007+0000 7f8c5bfff700 1 -- 172.21.15.33:0/2976477509 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400ba60 con 0x7f8c640c5d50 2023-11-06T20:45:08.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.008+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/2976477509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640c5d50 msgr2=0x7f8c640c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.008+0000 7f8c6b9ff700 1 --2- 172.21.15.33:0/2976477509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640c5d50 0x7f8c640c61d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f8c54009760 tx=0x7f8c54009060).stop 2023-11-06T20:45:08.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.008+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/2976477509 shutdown_connections 2023-11-06T20:45:08.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.008+0000 7f8c6b9ff700 1 --2- 172.21.15.33:0/2976477509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640c5d50 0x7f8c640c61d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.008+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/2976477509 >> 172.21.15.33:0/2976477509 conn(0x7f8c6405ec80 msgr2=0x7f8c640cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:08.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.008+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/2976477509 shutdown_connections 2023-11-06T20:45:08.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.008+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/2976477509 wait complete. 2023-11-06T20:45:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.010+0000 7f8c6b9ff700 1 Processor -- start 2023-11-06T20:45:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.010+0000 7f8c6b9ff700 1 -- start start 2023-11-06T20:45:08.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6b9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640bfd70 0x7f8c640af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6b9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c640c1840 0x7f8c641580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6b9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8c640c5d50 0x7f8c6414f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6b9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640afe20 con 0x7f8c640bfd70 2023-11-06T20:45:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6b9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c64156150 con 0x7f8c640c1840 2023-11-06T20:45:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6b9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c64155d40 con 0x7f8c640c5d50 2023-11-06T20:45:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6979b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640bfd70 0x7f8c640af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6979b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640bfd70 0x7f8c640af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36374/0 (socket says 172.21.15.33:36374) 2023-11-06T20:45:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6979b700 1 -- 172.21.15.33:0/224475945 learned_addr learned my addr 172.21.15.33:0/224475945 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c68f9a700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c640c1840 0x7f8c641580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c69f9c700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8c640c5d50 0x7f8c6414f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6979b700 1 -- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8c640c5d50 msgr2=0x7f8c6414f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6979b700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8c640c5d50 0x7f8c6414f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6979b700 1 -- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c640c1840 msgr2=0x7f8c641580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6979b700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c640c1840 0x7f8c641580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c6979b700 1 -- 172.21.15.33:0/224475945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540092a0 con 0x7f8c640bfd70 2023-11-06T20:45:08.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.011+0000 7f8c69f9c700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8c640c5d50 0x7f8c6414f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:08.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.012+0000 7f8c6979b700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640bfd70 0x7f8c640af930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f8c6000c4c0 tx=0x7f8c60007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:08.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.012+0000 7f8c597fa700 1 -- 172.21.15.33:0/224475945 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c600072e0 con 0x7f8c640bfd70 2023-11-06T20:45:08.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.012+0000 7f8c597fa700 1 -- 172.21.15.33:0/224475945 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c60007440 con 0x7f8c640bfd70 2023-11-06T20:45:08.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.012+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/224475945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c640b0050 con 0x7f8c640bfd70 2023-11-06T20:45:08.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.012+0000 7f8c597fa700 1 -- 172.21.15.33:0/224475945 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c600057a0 con 0x7f8c640bfd70 2023-11-06T20:45:08.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.012+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/224475945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c640b0610 con 0x7f8c640bfd70 2023-11-06T20:45:08.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.013+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/224475945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6406c6c0 con 0x7f8c640bfd70 2023-11-06T20:45:08.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.014+0000 7f8c597fa700 1 -- 172.21.15.33:0/224475945 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8c60005b90 con 0x7f8c640bfd70 2023-11-06T20:45:08.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.015+0000 7f8c597fa700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8c4805f150 0x7f8c48061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.015+0000 7f8c597fa700 1 -- 172.21.15.33:0/224475945 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8c60079ae0 con 0x7f8c640bfd70 2023-11-06T20:45:08.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.016+0000 7f8c68f9a700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8c4805f150 0x7f8c48061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.016+0000 7f8c68f9a700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8c4805f150 0x7f8c48061610 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8c6414a180 tx=0x7f8c54017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:08.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.019+0000 7f8c597fa700 1 -- 172.21.15.33:0/224475945 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8c6004acf0 con 0x7f8c640bfd70 2023-11-06T20:45:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.246+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/224475945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c64063f20 con 0x7f8c640bfd70 2023-11-06T20:45:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.246+0000 7f8c597fa700 1 -- 172.21.15.33:0/224475945 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f8c6004e310 con 0x7f8c640bfd70 2023-11-06T20:45:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:08.246 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/2191444463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2191444463},{"type":"v1","addr":"172.21.15.69:6836","nonce":2191444463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3920247513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3920247513},{"type":"v1","addr":"172.21.15.33:6834","nonce":3920247513}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2046477314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2046477314},{"type":"v1","addr":"172.21.15.69:6837","nonce":2046477314}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:45:04.669326+0000","modified":"2023-11-06T20:45:07.773858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/3657599674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3657599674},{"type":"v1","addr":"172.21.15.33:6835","nonce":3657599674}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-06T20:45:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.248+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8c4805f150 msgr2=0x7f8c48061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.248+0000 7f8c6b9ff700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8c4805f150 0x7f8c48061610 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8c6414a180 tx=0x7f8c54017000).stop 2023-11-06T20:45:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.248+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/224475945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640bfd70 msgr2=0x7f8c640af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.248+0000 7f8c6b9ff700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640bfd70 0x7f8c640af930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f8c6000c4c0 tx=0x7f8c60007c10).stop 2023-11-06T20:45:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.248+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/224475945 shutdown_connections 2023-11-06T20:45:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.248+0000 7f8c6b9ff700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8c4805f150 0x7f8c48061610 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.248+0000 7f8c6b9ff700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8c640c5d50 0x7f8c6414f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.248+0000 7f8c6b9ff700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c640bfd70 0x7f8c640af930 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.248+0000 7f8c6b9ff700 1 --2- 172.21.15.33:0/224475945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c640c1840 0x7f8c641580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.248+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/224475945 >> 172.21.15.33:0/224475945 conn(0x7f8c6405ec80 msgr2=0x7f8c64152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.249+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/224475945 shutdown_connections 2023-11-06T20:45:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.249+0000 7f8c6b9ff700 1 -- 172.21.15.33:0/224475945 wait complete. 2023-11-06T20:45:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 15 2023-11-06T20:45:08.265 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:45:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.461+0000 7f34775ff700 1 Processor -- start 2023-11-06T20:45:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.461+0000 7f34775ff700 1 -- start start 2023-11-06T20:45:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.462+0000 7f34775ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3470151dc0 0x7f34700c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.462+0000 7f34775ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f34701521a0 0x7f3470063110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.462+0000 7f34775ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3470155320 0x7f34700c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f34775ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3470154e90 con 0x7f3470151dc0 2023-11-06T20:45:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f34775ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34700639d0 con 0x7f34701521a0 2023-11-06T20:45:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f34775ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3470063cb0 con 0x7f3470155320 2023-11-06T20:45:08.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f347539b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3470151dc0 0x7f34700c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f347539b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3470151dc0 0x7f34700c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36382/0 (socket says 172.21.15.33:36382) 2023-11-06T20:45:08.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f347539b700 1 -- 172.21.15.33:0/2668462627 learned_addr learned my addr 172.21.15.33:0/2668462627 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:08.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f3475b9c700 1 --2- 172.21.15.33:0/2668462627 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3470155320 0x7f34700c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f3474b9a700 1 --2- 172.21.15.33:0/2668462627 >> v2:172.21.15.69:3300/0 conn(0x7f34701521a0 0x7f3470063110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f347539b700 1 -- 172.21.15.33:0/2668462627 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3470155320 msgr2=0x7f34700c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f347539b700 1 --2- 172.21.15.33:0/2668462627 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3470155320 0x7f34700c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f347539b700 1 -- 172.21.15.33:0/2668462627 >> v2:172.21.15.69:3300/0 conn(0x7f34701521a0 msgr2=0x7f3470063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f347539b700 1 --2- 172.21.15.33:0/2668462627 >> v2:172.21.15.69:3300/0 conn(0x7f34701521a0 0x7f3470063110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f347539b700 1 -- 172.21.15.33:0/2668462627 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470063f90 con 0x7f3470151dc0 2023-11-06T20:45:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f3475b9c700 1 --2- 172.21.15.33:0/2668462627 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3470155320 0x7f34700c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.463+0000 7f3474b9a700 1 --2- 172.21.15.33:0/2668462627 >> v2:172.21.15.69:3300/0 conn(0x7f34701521a0 0x7f3470063110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f347539b700 1 --2- 172.21.15.33:0/2668462627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470151dc0 0x7f34700c0c50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f34700c0680 tx=0x7f346c0090f0).ready entity=mon.0 client_cookie=f3573faefcd2c154 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:08.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f3467fff700 1 -- 172.21.15.33:0/2668462627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c00b600 con 0x7f3470151dc0 2023-11-06T20:45:08.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f3467fff700 1 -- 172.21.15.33:0/2668462627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c00b760 con 0x7f3470151dc0 2023-11-06T20:45:08.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f3467fff700 1 -- 172.21.15.33:0/2668462627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c00bae0 con 0x7f3470151dc0 2023-11-06T20:45:08.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f34775ff700 1 -- 172.21.15.33:0/2668462627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470151dc0 msgr2=0x7f34700c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f34775ff700 1 --2- 172.21.15.33:0/2668462627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470151dc0 0x7f34700c0c50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f34700c0680 tx=0x7f346c0090f0).stop 2023-11-06T20:45:08.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f34775ff700 1 -- 172.21.15.33:0/2668462627 shutdown_connections 2023-11-06T20:45:08.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f34775ff700 1 --2- 172.21.15.33:0/2668462627 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3470155320 0x7f34700c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f34775ff700 1 --2- 172.21.15.33:0/2668462627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470151dc0 0x7f34700c0c50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f34775ff700 1 --2- 172.21.15.33:0/2668462627 >> v2:172.21.15.69:3300/0 conn(0x7f34701521a0 0x7f3470063110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f34775ff700 1 -- 172.21.15.33:0/2668462627 >> 172.21.15.33:0/2668462627 conn(0x7f347005e1b0 msgr2=0x7f347005e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:08.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.464+0000 7f34775ff700 1 -- 172.21.15.33:0/2668462627 shutdown_connections 2023-11-06T20:45:08.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.465+0000 7f34775ff700 1 -- 172.21.15.33:0/2668462627 wait complete. 2023-11-06T20:45:08.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.466+0000 7f34775ff700 1 Processor -- start 2023-11-06T20:45:08.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.466+0000 7f34775ff700 1 -- start start 2023-11-06T20:45:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.467+0000 7f34775ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3470151dc0 0x7f34700ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.467+0000 7f34775ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34701521a0 0x7f34700abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.467+0000 7f34775ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470155320 0x7f34701641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.467+0000 7f34775ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34701506c0 con 0x7f3470155320 2023-11-06T20:45:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.467+0000 7f34775ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700ac440 con 0x7f3470151dc0 2023-11-06T20:45:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.467+0000 7f34775ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34700ac720 con 0x7f34701521a0 2023-11-06T20:45:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.467+0000 7f3475b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470155320 0x7f34701641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f3475b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470155320 0x7f34701641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36384/0 (socket says 172.21.15.33:36384) 2023-11-06T20:45:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f3475b9c700 1 -- 172.21.15.33:0/2284858509 learned_addr learned my addr 172.21.15.33:0/2284858509 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f347539b700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3470151dc0 0x7f34700ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f3474b9a700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34701521a0 0x7f34700abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f3475b9c700 1 -- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34701521a0 msgr2=0x7f34700abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f3475b9c700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34701521a0 0x7f34700abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f3475b9c700 1 -- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3470151dc0 msgr2=0x7f34700ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f3475b9c700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3470151dc0 0x7f34700ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f3475b9c700 1 -- 172.21.15.33:0/2284858509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c009370 con 0x7f3470155320 2023-11-06T20:45:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f347539b700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3470151dc0 0x7f34700ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f3474b9a700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34701521a0 0x7f34700abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f3475b9c700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470155320 0x7f34701641e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f346800cef0 tx=0x7f3468007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f34657fa700 1 -- 172.21.15.33:0/2284858509 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800ae00 con 0x7f3470155320 2023-11-06T20:45:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f34657fa700 1 -- 172.21.15.33:0/2284858509 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3468016400 con 0x7f3470155320 2023-11-06T20:45:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f34657fa700 1 -- 172.21.15.33:0/2284858509 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468016780 con 0x7f3470155320 2023-11-06T20:45:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.468+0000 7f34775ff700 1 -- 172.21.15.33:0/2284858509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34700aca00 con 0x7f3470155320 2023-11-06T20:45:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.469+0000 7f34775ff700 1 -- 172.21.15.33:0/2284858509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34700ac0f0 con 0x7f3470155320 2023-11-06T20:45:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.470+0000 7f34775ff700 1 -- 172.21.15.33:0/2284858509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34700acf20 con 0x7f3470155320 2023-11-06T20:45:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.470+0000 7f34657fa700 1 -- 172.21.15.33:0/2284858509 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f34680203f0 con 0x7f3470155320 2023-11-06T20:45:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.471+0000 7f34657fa700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f345405f150 0x7f3454061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.471+0000 7f34657fa700 1 -- 172.21.15.33:0/2284858509 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f346807aca0 con 0x7f3470155320 2023-11-06T20:45:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.472+0000 7f347539b700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f345405f150 0x7f3454061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.472+0000 7f347539b700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f345405f150 0x7f3454061610 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f346c009130 tx=0x7f346c017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.475+0000 7f34657fa700 1 -- 172.21.15.33:0/2284858509 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f34700acf20 con 0x7f3470155320 2023-11-06T20:45:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.656+0000 7f34775ff700 1 -- 172.21.15.33:0/2284858509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34700acf20 con 0x7f3470155320 2023-11-06T20:45:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.657+0000 7f34657fa700 1 -- 172.21.15.33:0/2284858509 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f34700acf20 con 0x7f3470155320 2023-11-06T20:45:08.657 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:08.657 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:45:05.671805+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:45:02.985899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T20:45:03.637532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:45:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 -- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f345405f150 msgr2=0x7f3454061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f345405f150 0x7f3454061610 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f346c009130 tx=0x7f346c017000).stop 2023-11-06T20:45:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 -- 172.21.15.33:0/2284858509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470155320 msgr2=0x7f34701641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470155320 0x7f34701641e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f346800cef0 tx=0x7f3468007860).stop 2023-11-06T20:45:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 -- 172.21.15.33:0/2284858509 shutdown_connections 2023-11-06T20:45:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f345405f150 0x7f3454061610 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470155320 0x7f34701641e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3470151dc0 0x7f34700ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 --2- 172.21.15.33:0/2284858509 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34701521a0 0x7f34700abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 -- 172.21.15.33:0/2284858509 >> 172.21.15.33:0/2284858509 conn(0x7f347005e1b0 msgr2=0x7f3470157170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 -- 172.21.15.33:0/2284858509 shutdown_connections 2023-11-06T20:45:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.660+0000 7f34775ff700 1 -- 172.21.15.33:0/2284858509 wait complete. 2023-11-06T20:45:08.677 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T20:45:08.677 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:45:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.875+0000 7f44d5bff700 1 Processor -- start 2023-11-06T20:45:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.875+0000 7f44d5bff700 1 -- start start 2023-11-06T20:45:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.876+0000 7f44d5bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f44d00c1840 0x7f44d00a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.876+0000 7f44d5bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f44d00aa2d0 0x7f44d00c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.876+0000 7f44d5bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d01554a0 con 0x7f44d00c6240 2023-11-06T20:45:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.876+0000 7f44d5bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d0154960 con 0x7f44d00aa2d0 2023-11-06T20:45:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.876+0000 7f44d5bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d0155930 con 0x7f44d00c1840 2023-11-06T20:45:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44ceffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f44d00aa2d0 0x7f44d00c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44ceffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f44d00aa2d0 0x7f44d00c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36398/0 (socket says 172.21.15.33:36398) 2023-11-06T20:45:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44ceffd700 1 -- 172.21.15.33:0/3262847247 learned_addr learned my addr 172.21.15.33:0/3262847247 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44cffff700 1 --1- 172.21.15.33:0/3262847247 >> v1:172.21.15.33:6789/0 conn(0x7f44d00c6240 0x7f44d0157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57958/0 (socket says 172.21.15.33:57958) 2023-11-06T20:45:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44cf7fe700 1 --2- 172.21.15.33:0/3262847247 >> v2:172.21.15.69:3300/0 conn(0x7f44d00c1840 0x7f44d00a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44ceffd700 1 -- 172.21.15.33:0/3262847247 >> v2:172.21.15.69:3300/0 conn(0x7f44d00c1840 msgr2=0x7f44d00a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44ceffd700 1 --2- 172.21.15.33:0/3262847247 >> v2:172.21.15.69:3300/0 conn(0x7f44d00c1840 0x7f44d00a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44ceffd700 1 -- 172.21.15.33:0/3262847247 >> v1:172.21.15.33:6789/0 conn(0x7f44d00c6240 legacy=0x7f44d0157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44ceffd700 1 -- 172.21.15.33:0/3262847247 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d0154b70 con 0x7f44d00aa2d0 2023-11-06T20:45:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44ce7fc700 1 -- 172.21.15.33:0/3262847247 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878245209 0 0) 0x7f44d01554a0 con 0x7f44d00c6240 2023-11-06T20:45:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.877+0000 7f44ceffd700 1 --2- 172.21.15.33:0/3262847247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00aa2d0 0x7f44d00c5d50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f44d00840b0 tx=0x7f44c40090f0).ready entity=mon.0 client_cookie=385e5b3113f12c48 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44ce7fc700 1 -- 172.21.15.33:0/3262847247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c400b520 con 0x7f44d00aa2d0 2023-11-06T20:45:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44ce7fc700 1 -- 172.21.15.33:0/3262847247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c400b680 con 0x7f44d00aa2d0 2023-11-06T20:45:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44ce7fc700 1 -- 172.21.15.33:0/3262847247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c400ba00 con 0x7f44d00aa2d0 2023-11-06T20:45:08.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44d5bff700 1 -- 172.21.15.33:0/3262847247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00aa2d0 msgr2=0x7f44d00c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44d5bff700 1 --2- 172.21.15.33:0/3262847247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00aa2d0 0x7f44d00c5d50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f44d00840b0 tx=0x7f44c40090f0).stop 2023-11-06T20:45:08.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44d5bff700 1 -- 172.21.15.33:0/3262847247 shutdown_connections 2023-11-06T20:45:08.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44d5bff700 1 --2- 172.21.15.33:0/3262847247 >> v2:172.21.15.69:3300/0 conn(0x7f44d00c1840 0x7f44d00a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44d5bff700 1 --2- 172.21.15.33:0/3262847247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00aa2d0 0x7f44d00c5d50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44d5bff700 1 -- 172.21.15.33:0/3262847247 >> 172.21.15.33:0/3262847247 conn(0x7f44d005ec80 msgr2=0x7f44d00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:08.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44d5bff700 1 -- 172.21.15.33:0/3262847247 shutdown_connections 2023-11-06T20:45:08.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.878+0000 7f44d5bff700 1 -- 172.21.15.33:0/3262847247 wait complete. 2023-11-06T20:45:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.880+0000 7f44d5bff700 1 Processor -- start 2023-11-06T20:45:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.880+0000 7f44d5bff700 1 -- start start 2023-11-06T20:45:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44d5bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44d00aa2d0 0x7f44d00c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44d5bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00c1840 0x7f44d014c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44d5bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44d00c6240 0x7f44d014c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44d5bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d014cc70 con 0x7f44d00c1840 2023-11-06T20:45:08.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44d5bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d014cf50 con 0x7f44d00aa2d0 2023-11-06T20:45:08.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44d5bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d014d230 con 0x7f44d00c6240 2023-11-06T20:45:08.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44ceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00c1840 0x7f44d014c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44ceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00c1840 0x7f44d014c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36412/0 (socket says 172.21.15.33:36412) 2023-11-06T20:45:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44ceffd700 1 -- 172.21.15.33:0/1785421175 learned_addr learned my addr 172.21.15.33:0/1785421175 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44cf7fe700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44d00aa2d0 0x7f44d00c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44cffff700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44d00c6240 0x7f44d014c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44ceffd700 1 -- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44d00c6240 msgr2=0x7f44d014c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44ceffd700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44d00c6240 0x7f44d014c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44ceffd700 1 -- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44d00aa2d0 msgr2=0x7f44d00c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44ceffd700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44d00aa2d0 0x7f44d00c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44ceffd700 1 -- 172.21.15.33:0/1785421175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c4009370 con 0x7f44d00c1840 2023-11-06T20:45:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.881+0000 7f44cffff700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44d00c6240 0x7f44d014c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.882+0000 7f44ceffd700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00c1840 0x7f44d014c120 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f44c4009130 tx=0x7f44c4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:08.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.882+0000 7f44b77fe700 1 -- 172.21.15.33:0/1785421175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c400be90 con 0x7f44d00c1840 2023-11-06T20:45:08.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.882+0000 7f44b77fe700 1 -- 172.21.15.33:0/1785421175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c400b7f0 con 0x7f44d00c1840 2023-11-06T20:45:08.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.882+0000 7f44d5bff700 1 -- 172.21.15.33:0/1785421175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d014d510 con 0x7f44d00c1840 2023-11-06T20:45:08.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.882+0000 7f44b77fe700 1 -- 172.21.15.33:0/1785421175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c401b660 con 0x7f44d00c1840 2023-11-06T20:45:08.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.882+0000 7f44d5bff700 1 -- 172.21.15.33:0/1785421175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d014a9f0 con 0x7f44d00c1840 2023-11-06T20:45:08.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.883+0000 7f44d5bff700 1 -- 172.21.15.33:0/1785421175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d00aa110 con 0x7f44d00c1840 2023-11-06T20:45:08.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.884+0000 7f44b77fe700 1 -- 172.21.15.33:0/1785421175 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f44c401ba10 con 0x7f44d00c1840 2023-11-06T20:45:08.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.885+0000 7f44b77fe700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f44bc05f150 0x7f44bc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:08.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.885+0000 7f44b77fe700 1 -- 172.21.15.33:0/1785421175 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f44c407bb70 con 0x7f44d00c1840 2023-11-06T20:45:08.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.885+0000 7f44cf7fe700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f44bc05f150 0x7f44bc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:08.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.886+0000 7f44cf7fe700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f44bc05f150 0x7f44bc061610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f44c0009bc0 tx=0x7f44c0008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:08.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:08.889+0000 7f44b77fe700 1 -- 172.21.15.33:0/1785421175 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f44c404cd80 con 0x7f44d00c1840 2023-11-06T20:45:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.067+0000 7f44d5bff700 1 -- 172.21.15.33:0/1785421175 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44d006c6c0 con 0x7f44bc05f150 2023-11-06T20:45:09.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.079+0000 7f44b77fe700 1 -- 172.21.15.33:0/1785421175 <== mgr.4103 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354516 (secure 0 0 0) 0x7f44d006c6c0 con 0x7f44bc05f150 2023-11-06T20:45:09.083 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:09.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 -- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f44bc05f150 msgr2=0x7f44bc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f44bc05f150 0x7f44bc061610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f44c0009bc0 tx=0x7f44c0008040).stop 2023-11-06T20:45:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 -- 172.21.15.33:0/1785421175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00c1840 msgr2=0x7f44d014c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00c1840 0x7f44d014c120 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f44c4009130 tx=0x7f44c4017000).stop 2023-11-06T20:45:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 -- 172.21.15.33:0/1785421175 shutdown_connections 2023-11-06T20:45:09.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f44bc05f150 0x7f44bc061610 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44d00c6240 0x7f44d014c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44d00aa2d0 0x7f44d00c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 --2- 172.21.15.33:0/1785421175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d00c1840 0x7f44d014c120 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 -- 172.21.15.33:0/1785421175 >> 172.21.15.33:0/1785421175 conn(0x7f44d005ec80 msgr2=0x7f44d00cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 -- 172.21.15.33:0/1785421175 shutdown_connections 2023-11-06T20:45:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.084+0000 7f44d5bff700 1 -- 172.21.15.33:0/1785421175 wait complete. 2023-11-06T20:45:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:45:09.100 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-06T20:45:08.823293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379644,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595076,"statfs":{"total":773094113280,"available":770657357824,"internally_reserved":0,"allocated":2945024,"data_stored":1192131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4861,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.325528"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3f","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712002+0000","last_change":"2023-11-06T20:45:04.213243+0000","last_active":"2023-11-06T20:45:04.712002+0000","last_peered":"2023-11-06T20:45:04.712002+0000","last_clean":"2023-11-06T20:45:04.712002+0000","last_became_active":"2023-11-06T20:45:04.212975+0000","last_became_peered":"2023-11-06T20:45:04.212975+0000","last_unstale":"2023-11-06T20:45:04.712002+0000","last_undegraded":"2023-11-06T20:45:04.712002+0000","last_fullsized":"2023-11-06T20:45:04.712002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173152+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699604+0000","last_change":"2023-11-06T20:45:04.196531+0000","last_active":"2023-11-06T20:45:04.699604+0000","last_peered":"2023-11-06T20:45:04.699604+0000","last_clean":"2023-11-06T20:45:04.699604+0000","last_became_active":"2023-11-06T20:45:04.194888+0000","last_became_peered":"2023-11-06T20:45:04.194888+0000","last_unstale":"2023-11-06T20:45:04.699604+0000","last_undegraded":"2023-11-06T20:45:04.699604+0000","last_fullsized":"2023-11-06T20:45:04.699604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175110+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174673+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703778+0000","last_change":"2023-11-06T20:45:04.699346+0000","last_active":"2023-11-06T20:45:04.703778+0000","last_peered":"2023-11-06T20:45:04.703778+0000","last_clean":"2023-11-06T20:45:04.703778+0000","last_became_active":"2023-11-06T20:45:04.698931+0000","last_became_peered":"2023-11-06T20:45:04.698931+0000","last_unstale":"2023-11-06T20:45:04.703778+0000","last_undegraded":"2023-11-06T20:45:04.703778+0000","last_fullsized":"2023-11-06T20:45:04.703778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.176517+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680925+0000","last_change":"2023-11-06T20:45:04.212407+0000","last_active":"2023-11-06T20:45:04.680925+0000","last_peered":"2023-11-06T20:45:04.680925+0000","last_clean":"2023-11-06T20:45:04.680925+0000","last_became_active":"2023-11-06T20:45:04.211888+0000","last_became_peered":"2023-11-06T20:45:04.211888+0000","last_unstale":"2023-11-06T20:45:04.680925+0000","last_undegraded":"2023-11-06T20:45:04.680925+0000","last_fullsized":"2023-11-06T20:45:04.680925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680573+0000","last_change":"2023-11-06T20:45:04.203081+0000","last_active":"2023-11-06T20:45:04.680573+0000","last_peered":"2023-11-06T20:45:04.680573+0000","last_clean":"2023-11-06T20:45:04.680573+0000","last_became_active":"2023-11-06T20:45:04.199530+0000","last_became_peered":"2023-11-06T20:45:04.199530+0000","last_unstale":"2023-11-06T20:45:04.680573+0000","last_undegraded":"2023-11-06T20:45:04.680573+0000","last_fullsized":"2023-11-06T20:45:04.680573+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700837+0000","last_change":"2023-11-06T20:45:04.207865+0000","last_active":"2023-11-06T20:45:04.700837+0000","last_peered":"2023-11-06T20:45:04.700837+0000","last_clean":"2023-11-06T20:45:04.700837+0000","last_became_active":"2023-11-06T20:45:04.207584+0000","last_became_peered":"2023-11-06T20:45:04.207584+0000","last_unstale":"2023-11-06T20:45:04.700837+0000","last_undegraded":"2023-11-06T20:45:04.700837+0000","last_fullsized":"2023-11-06T20:45:04.700837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175778+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715752+0000","last_change":"2023-11-06T20:45:04.711145+0000","last_active":"2023-11-06T20:45:04.715752+0000","last_peered":"2023-11-06T20:45:04.715752+0000","last_clean":"2023-11-06T20:45:04.715752+0000","last_became_active":"2023-11-06T20:45:04.709234+0000","last_became_peered":"2023-11-06T20:45:04.709234+0000","last_unstale":"2023-11-06T20:45:04.715752+0000","last_undegraded":"2023-11-06T20:45:04.715752+0000","last_fullsized":"2023-11-06T20:45:04.715752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182188+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715259+0000","last_change":"2023-11-06T20:45:04.206902+0000","last_active":"2023-11-06T20:45:04.715259+0000","last_peered":"2023-11-06T20:45:04.715259+0000","last_clean":"2023-11-06T20:45:04.715259+0000","last_became_active":"2023-11-06T20:45:04.206583+0000","last_became_peered":"2023-11-06T20:45:04.206583+0000","last_unstale":"2023-11-06T20:45:04.715259+0000","last_undegraded":"2023-11-06T20:45:04.715259+0000","last_fullsized":"2023-11-06T20:45:04.715259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712482+0000","last_change":"2023-11-06T20:45:04.707038+0000","last_active":"2023-11-06T20:45:04.712482+0000","last_peered":"2023-11-06T20:45:04.712482+0000","last_clean":"2023-11-06T20:45:04.712482+0000","last_became_active":"2023-11-06T20:45:04.704732+0000","last_became_peered":"2023-11-06T20:45:04.704732+0000","last_unstale":"2023-11-06T20:45:04.712482+0000","last_undegraded":"2023-11-06T20:45:04.712482+0000","last_fullsized":"2023-11-06T20:45:04.712482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178534+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174676+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699919+0000","last_change":"2023-11-06T20:45:04.698880+0000","last_active":"2023-11-06T20:45:04.699919+0000","last_peered":"2023-11-06T20:45:04.699919+0000","last_clean":"2023-11-06T20:45:04.699919+0000","last_became_active":"2023-11-06T20:45:04.697976+0000","last_became_peered":"2023-11-06T20:45:04.697976+0000","last_unstale":"2023-11-06T20:45:04.699919+0000","last_undegraded":"2023-11-06T20:45:04.699919+0000","last_fullsized":"2023-11-06T20:45:04.699919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179850+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703867+0000","last_change":"2023-11-06T20:45:04.701051+0000","last_active":"2023-11-06T20:45:04.703867+0000","last_peered":"2023-11-06T20:45:04.703867+0000","last_clean":"2023-11-06T20:45:04.703867+0000","last_became_active":"2023-11-06T20:45:04.700684+0000","last_became_peered":"2023-11-06T20:45:04.700684+0000","last_unstale":"2023-11-06T20:45:04.703867+0000","last_undegraded":"2023-11-06T20:45:04.703867+0000","last_fullsized":"2023-11-06T20:45:04.703867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179400+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182863+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178154+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180329+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180016+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180343+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681053+0000","last_change":"2023-11-06T20:45:04.208631+0000","last_active":"2023-11-06T20:45:04.681053+0000","last_peered":"2023-11-06T20:45:04.681053+0000","last_clean":"2023-11-06T20:45:04.681053+0000","last_became_active":"2023-11-06T20:45:04.208313+0000","last_became_peered":"2023-11-06T20:45:04.208313+0000","last_unstale":"2023-11-06T20:45:04.681053+0000","last_undegraded":"2023-11-06T20:45:04.681053+0000","last_fullsized":"2023-11-06T20:45:04.681053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178850+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680469+0000","last_change":"2023-11-06T20:45:04.208028+0000","last_active":"2023-11-06T20:45:04.680469+0000","last_peered":"2023-11-06T20:45:04.680469+0000","last_clean":"2023-11-06T20:45:04.680469+0000","last_became_active":"2023-11-06T20:45:04.207152+0000","last_became_peered":"2023-11-06T20:45:04.207152+0000","last_unstale":"2023-11-06T20:45:04.680469+0000","last_undegraded":"2023-11-06T20:45:04.680469+0000","last_fullsized":"2023-11-06T20:45:04.680469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714410+0000","last_change":"2023-11-06T20:45:04.710247+0000","last_active":"2023-11-06T20:45:04.714410+0000","last_peered":"2023-11-06T20:45:04.714410+0000","last_clean":"2023-11-06T20:45:04.714410+0000","last_became_active":"2023-11-06T20:45:04.709893+0000","last_became_peered":"2023-11-06T20:45:04.709893+0000","last_unstale":"2023-11-06T20:45:04.714410+0000","last_undegraded":"2023-11-06T20:45:04.714410+0000","last_fullsized":"2023-11-06T20:45:04.714410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.176372+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175234+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173525+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700044+0000","last_change":"2023-11-06T20:45:04.693933+0000","last_active":"2023-11-06T20:45:04.700044+0000","last_peered":"2023-11-06T20:45:04.700044+0000","last_clean":"2023-11-06T20:45:04.700044+0000","last_became_active":"2023-11-06T20:45:04.693204+0000","last_became_peered":"2023-11-06T20:45:04.693204+0000","last_unstale":"2023-11-06T20:45:04.700044+0000","last_undegraded":"2023-11-06T20:45:04.700044+0000","last_fullsized":"2023-11-06T20:45:04.700044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681743+0000","last_change":"2023-11-06T20:45:04.213724+0000","last_active":"2023-11-06T20:45:04.681743+0000","last_peered":"2023-11-06T20:45:04.681743+0000","last_clean":"2023-11-06T20:45:04.681743+0000","last_became_active":"2023-11-06T20:45:04.211299+0000","last_became_peered":"2023-11-06T20:45:04.211299+0000","last_unstale":"2023-11-06T20:45:04.681743+0000","last_undegraded":"2023-11-06T20:45:04.681743+0000","last_fullsized":"2023-11-06T20:45:04.681743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179941+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703797+0000","last_change":"2023-11-06T20:45:04.701216+0000","last_active":"2023-11-06T20:45:04.703797+0000","last_peered":"2023-11-06T20:45:04.703797+0000","last_clean":"2023-11-06T20:45:04.703797+0000","last_became_active":"2023-11-06T20:45:04.700956+0000","last_became_peered":"2023-11-06T20:45:04.700956+0000","last_unstale":"2023-11-06T20:45:04.703797+0000","last_undegraded":"2023-11-06T20:45:04.703797+0000","last_fullsized":"2023-11-06T20:45:04.703797+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180325+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680966+0000","last_change":"2023-11-06T20:45:04.205274+0000","last_active":"2023-11-06T20:45:04.680966+0000","last_peered":"2023-11-06T20:45:04.680966+0000","last_clean":"2023-11-06T20:45:04.680966+0000","last_became_active":"2023-11-06T20:45:04.204948+0000","last_became_peered":"2023-11-06T20:45:04.204948+0000","last_unstale":"2023-11-06T20:45:04.680966+0000","last_undegraded":"2023-11-06T20:45:04.680966+0000","last_fullsized":"2023-11-06T20:45:04.680966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.181516+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699237+0000","last_change":"2023-11-06T20:45:04.199975+0000","last_active":"2023-11-06T20:45:04.699237+0000","last_peered":"2023-11-06T20:45:04.699237+0000","last_clean":"2023-11-06T20:45:04.699237+0000","last_became_active":"2023-11-06T20:45:04.195195+0000","last_became_peered":"2023-11-06T20:45:04.195195+0000","last_unstale":"2023-11-06T20:45:04.699237+0000","last_undegraded":"2023-11-06T20:45:04.699237+0000","last_fullsized":"2023-11-06T20:45:04.699237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179244+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174491+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681802+0000","last_change":"2023-11-06T20:45:04.209004+0000","last_active":"2023-11-06T20:45:04.681802+0000","last_peered":"2023-11-06T20:45:04.681802+0000","last_clean":"2023-11-06T20:45:04.681802+0000","last_became_active":"2023-11-06T20:45:04.208295+0000","last_became_peered":"2023-11-06T20:45:04.208295+0000","last_unstale":"2023-11-06T20:45:04.681802+0000","last_undegraded":"2023-11-06T20:45:04.681802+0000","last_fullsized":"2023-11-06T20:45:04.681802+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680947+0000","last_change":"2023-11-06T20:45:04.200869+0000","last_active":"2023-11-06T20:45:04.680947+0000","last_peered":"2023-11-06T20:45:04.680947+0000","last_clean":"2023-11-06T20:45:04.680947+0000","last_became_active":"2023-11-06T20:45:04.200557+0000","last_became_peered":"2023-11-06T20:45:04.200557+0000","last_unstale":"2023-11-06T20:45:04.680947+0000","last_undegraded":"2023-11-06T20:45:04.680947+0000","last_fullsized":"2023-11-06T20:45:04.680947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696708+0000","last_change":"2023-11-06T20:45:04.205999+0000","last_active":"2023-11-06T20:45:04.696708+0000","last_peered":"2023-11-06T20:45:04.696708+0000","last_clean":"2023-11-06T20:45:04.696708+0000","last_became_active":"2023-11-06T20:45:04.205244+0000","last_became_peered":"2023-11-06T20:45:04.205244+0000","last_unstale":"2023-11-06T20:45:04.696708+0000","last_undegraded":"2023-11-06T20:45:04.696708+0000","last_fullsized":"2023-11-06T20:45:04.696708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174499+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696679+0000","last_change":"2023-11-06T20:45:04.194284+0000","last_active":"2023-11-06T20:45:04.696679+0000","last_peered":"2023-11-06T20:45:04.696679+0000","last_clean":"2023-11-06T20:45:04.696679+0000","last_became_active":"2023-11-06T20:45:04.193972+0000","last_became_peered":"2023-11-06T20:45:04.193972+0000","last_unstale":"2023-11-06T20:45:04.696679+0000","last_undegraded":"2023-11-06T20:45:04.696679+0000","last_fullsized":"2023-11-06T20:45:04.696679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175433+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700556+0000","last_change":"2023-11-06T20:45:04.699519+0000","last_active":"2023-11-06T20:45:04.700556+0000","last_peered":"2023-11-06T20:45:04.700556+0000","last_clean":"2023-11-06T20:45:04.700556+0000","last_became_active":"2023-11-06T20:45:04.699244+0000","last_became_peered":"2023-11-06T20:45:04.699244+0000","last_unstale":"2023-11-06T20:45:04.700556+0000","last_undegraded":"2023-11-06T20:45:04.700556+0000","last_fullsized":"2023-11-06T20:45:04.700556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182638+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182611+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698113+0000","last_change":"2023-11-06T20:45:04.694154+0000","last_active":"2023-11-06T20:45:04.698113+0000","last_peered":"2023-11-06T20:45:04.698113+0000","last_clean":"2023-11-06T20:45:04.698113+0000","last_became_active":"2023-11-06T20:45:04.693824+0000","last_became_peered":"2023-11-06T20:45:04.693824+0000","last_unstale":"2023-11-06T20:45:04.698113+0000","last_undegraded":"2023-11-06T20:45:04.698113+0000","last_fullsized":"2023-11-06T20:45:04.698113+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174583+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175488+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179685+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.184627+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714056+0000","last_change":"2023-11-06T20:45:04.199156+0000","last_active":"2023-11-06T20:45:04.714056+0000","last_peered":"2023-11-06T20:45:04.714056+0000","last_clean":"2023-11-06T20:45:04.714056+0000","last_became_active":"2023-11-06T20:45:04.198858+0000","last_became_peered":"2023-11-06T20:45:04.198858+0000","last_unstale":"2023-11-06T20:45:04.714056+0000","last_undegraded":"2023-11-06T20:45:04.714056+0000","last_fullsized":"2023-11-06T20:45:04.714056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715339+0000","last_change":"2023-11-06T20:45:04.199860+0000","last_active":"2023-11-06T20:45:04.715339+0000","last_peered":"2023-11-06T20:45:04.715339+0000","last_clean":"2023-11-06T20:45:04.715339+0000","last_became_active":"2023-11-06T20:45:04.197409+0000","last_became_peered":"2023-11-06T20:45:04.197409+0000","last_unstale":"2023-11-06T20:45:04.715339+0000","last_undegraded":"2023-11-06T20:45:04.715339+0000","last_fullsized":"2023-11-06T20:45:04.715339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713635+0000","last_change":"2023-11-06T20:45:04.707405+0000","last_active":"2023-11-06T20:45:04.713635+0000","last_peered":"2023-11-06T20:45:04.713635+0000","last_clean":"2023-11-06T20:45:04.713635+0000","last_became_active":"2023-11-06T20:45:04.704832+0000","last_became_peered":"2023-11-06T20:45:04.704832+0000","last_unstale":"2023-11-06T20:45:04.713635+0000","last_undegraded":"2023-11-06T20:45:04.713635+0000","last_fullsized":"2023-11-06T20:45:04.713635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182164+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713102+0000","last_change":"2023-11-06T20:45:04.703423+0000","last_active":"2023-11-06T20:45:04.713102+0000","last_peered":"2023-11-06T20:45:04.713102+0000","last_clean":"2023-11-06T20:45:04.713102+0000","last_became_active":"2023-11-06T20:45:04.703108+0000","last_became_peered":"2023-11-06T20:45:04.703108+0000","last_unstale":"2023-11-06T20:45:04.713102+0000","last_undegraded":"2023-11-06T20:45:04.713102+0000","last_fullsized":"2023-11-06T20:45:04.713102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179821+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174040+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.172938+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178482+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698151+0000","last_change":"2023-11-06T20:45:04.207106+0000","last_active":"2023-11-06T20:45:04.698151+0000","last_peered":"2023-11-06T20:45:04.698151+0000","last_clean":"2023-11-06T20:45:04.698151+0000","last_became_active":"2023-11-06T20:45:04.206814+0000","last_became_peered":"2023-11-06T20:45:04.206814+0000","last_unstale":"2023-11-06T20:45:04.698151+0000","last_undegraded":"2023-11-06T20:45:04.698151+0000","last_fullsized":"2023-11-06T20:45:04.698151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178510+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173953+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:57.427499+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:44:57.427499+0000","last_peered":"2023-11-06T20:44:57.427499+0000","last_clean":"2023-11-06T20:44:57.427499+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:44:57.427499+0000","last_undegraded":"2023-11-06T20:44:57.427499+0000","last_fullsized":"2023-11-06T20:44:57.427499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.177557+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046724+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:44:59.046724+0000","last_peered":"2023-11-06T20:44:59.046724+0000","last_clean":"2023-11-06T20:44:59.046724+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:44:59.046724+0000","last_undegraded":"2023-11-06T20:44:59.046724+0000","last_fullsized":"2023-11-06T20:44:59.046724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.176976+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.047320+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:44:59.047320+0000","last_peered":"2023-11-06T20:44:59.047320+0000","last_clean":"2023-11-06T20:44:59.047320+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:44:59.047320+0000","last_undegraded":"2023-11-06T20:44:59.047320+0000","last_fullsized":"2023-11-06T20:44:59.047320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179510+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046727+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:44:59.046727+0000","last_peered":"2023-11-06T20:44:59.046727+0000","last_clean":"2023-11-06T20:44:59.046727+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:44:59.046727+0000","last_undegraded":"2023-11-06T20:44:59.046727+0000","last_fullsized":"2023-11-06T20:44:59.046727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696761+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:45:04.696761+0000","last_peered":"2023-11-06T20:45:04.696761+0000","last_clean":"2023-11-06T20:45:04.696761+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:45:04.696761+0000","last_undegraded":"2023-11-06T20:45:04.696761+0000","last_fullsized":"2023-11-06T20:45:04.696761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182200+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046740+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:44:59.046740+0000","last_peered":"2023-11-06T20:44:59.046740+0000","last_clean":"2023-11-06T20:44:59.046740+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:44:59.046740+0000","last_undegraded":"2023-11-06T20:44:59.046740+0000","last_fullsized":"2023-11-06T20:44:59.046740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179915+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046755+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:44:59.046755+0000","last_peered":"2023-11-06T20:44:59.046755+0000","last_clean":"2023-11-06T20:44:59.046755+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:44:59.046755+0000","last_undegraded":"2023-11-06T20:44:59.046755+0000","last_fullsized":"2023-11-06T20:44:59.046755+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.184211+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.047396+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:44:59.047396+0000","last_peered":"2023-11-06T20:44:59.047396+0000","last_clean":"2023-11-06T20:44:59.047396+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:44:59.047396+0000","last_undegraded":"2023-11-06T20:44:59.047396+0000","last_fullsized":"2023-11-06T20:44:59.047396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173926+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046729+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:44:59.046729+0000","last_peered":"2023-11-06T20:44:59.046729+0000","last_clean":"2023-11-06T20:44:59.046729+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:44:59.046729+0000","last_undegraded":"2023-11-06T20:44:59.046729+0000","last_fullsized":"2023-11-06T20:44:59.046729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":2},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":1468,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.1749999999999998,"5min":5.1749999999999998,"15min":5.1749999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.624}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65600000000000003}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.202,"5min":5.202,"15min":5.202},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.77200000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72999999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.75700000000000001}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46700000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.88700000000000001}]},{"osd":2,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.82999999999999996}]},{"osd":3,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.78300000000000003}]},{"osd":7,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.749}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":409600,"data_stored":167482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.42499999999999999}]},{"osd":2,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.439}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.38200000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.60499999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297436,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074404,"statfs":{"total":96636764160,"available":96332189696,"internally_reserved":0,"allocated":348160,"data_stored":139715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.51000000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.5}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.746},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.73299999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.70699999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":409600,"data_stored":167696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.38100000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.69299999999999995}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297436,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074404,"statfs":{"total":96636764160,"available":96332189696,"internally_reserved":0,"allocated":348160,"data_stored":139715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.38},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.44}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.53500000000000003}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64700000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.621}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.76100000000000001}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.86499999999999999}]},{"osd":1,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.93200000000000005}]},{"osd":2,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.95699999999999996}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.748},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.76000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297436,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074404,"statfs":{"total":96636764160,"available":96332189696,"internally_reserved":0,"allocated":348160,"data_stored":139715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66200000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68000000000000005}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.745}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.69899999999999995}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.70799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:45:09.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:45:09.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.336+0000 7f49addff700 1 Processor -- start 2023-11-06T20:45:09.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.336+0000 7f49addff700 1 -- start start 2023-11-06T20:45:09.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.337+0000 7f49addff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49a80c5d50 0x7f49a80c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.337+0000 7f49addff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a81554a0 con 0x7f49a80c1840 2023-11-06T20:45:09.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.337+0000 7f49addff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a80c66c0 con 0x7f49a80bfd70 2023-11-06T20:45:09.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.337+0000 7f49addff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a8154960 con 0x7f49a80c5d50 2023-11-06T20:45:09.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.338+0000 7f49a6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49a80c5d50 0x7f49a80c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.338+0000 7f49a6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49a80c5d50 0x7f49a80c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36426/0 (socket says 172.21.15.33:36426) 2023-11-06T20:45:09.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.338+0000 7f49a6ffd700 1 -- 172.21.15.33:0/3476107531 learned_addr learned my addr 172.21.15.33:0/3476107531 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:09.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.338+0000 7f49a77fe700 1 --1- 172.21.15.33:0/3476107531 >> v1:172.21.15.33:6789/0 conn(0x7f49a80c1840 0x7f49a80a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57964/0 (socket says 172.21.15.33:57964) 2023-11-06T20:45:09.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.338+0000 7f49a6ffd700 1 -- 172.21.15.33:0/3476107531 >> v1:172.21.15.69:6789/0 conn(0x7f49a80bfd70 legacy=0x7f49a80c0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.338+0000 7f49a6ffd700 1 -- 172.21.15.33:0/3476107531 >> v1:172.21.15.33:6789/0 conn(0x7f49a80c1840 legacy=0x7f49a80a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.338+0000 7f49a6ffd700 1 -- 172.21.15.33:0/3476107531 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a8155930 con 0x7f49a80c5d50 2023-11-06T20:45:09.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.338+0000 7f49a67fc700 1 -- 172.21.15.33:0/3476107531 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122521125 0 0) 0x7f49a81554a0 con 0x7f49a80c1840 2023-11-06T20:45:09.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.338+0000 7f49a6ffd700 1 --2- 172.21.15.33:0/3476107531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c5d50 0x7f49a80c61d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f499c006fd0 tx=0x7f499c0090f0).ready entity=mon.0 client_cookie=beed5765a4574685 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:09.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.339+0000 7f49a67fc700 1 -- 172.21.15.33:0/3476107531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c00b600 con 0x7f49a80c5d50 2023-11-06T20:45:09.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.339+0000 7f49a67fc700 1 -- 172.21.15.33:0/3476107531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c00b760 con 0x7f49a80c5d50 2023-11-06T20:45:09.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.339+0000 7f49a67fc700 1 -- 172.21.15.33:0/3476107531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c00bae0 con 0x7f49a80c5d50 2023-11-06T20:45:09.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.339+0000 7f49addff700 1 -- 172.21.15.33:0/3476107531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c5d50 msgr2=0x7f49a80c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.339+0000 7f49addff700 1 --2- 172.21.15.33:0/3476107531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c5d50 0x7f49a80c61d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f499c006fd0 tx=0x7f499c0090f0).stop 2023-11-06T20:45:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.339+0000 7f49addff700 1 -- 172.21.15.33:0/3476107531 shutdown_connections 2023-11-06T20:45:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.339+0000 7f49addff700 1 --2- 172.21.15.33:0/3476107531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c5d50 0x7f49a80c61d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.339+0000 7f49addff700 1 -- 172.21.15.33:0/3476107531 >> 172.21.15.33:0/3476107531 conn(0x7f49a805ec80 msgr2=0x7f49a80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.339+0000 7f49addff700 1 -- 172.21.15.33:0/3476107531 shutdown_connections 2023-11-06T20:45:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.339+0000 7f49addff700 1 -- 172.21.15.33:0/3476107531 wait complete. 2023-11-06T20:45:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.341+0000 7f49addff700 1 Processor -- start 2023-11-06T20:45:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.341+0000 7f49addff700 1 -- start start 2023-11-06T20:45:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49addff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a80bfd70 0x7f49a8150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49addff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c1840 0x7f49a80afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49addff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f49a80c5d50 0x7f49a8153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49addff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a8158cf0 con 0x7f49a80c1840 2023-11-06T20:45:09.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49addff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a8158fd0 con 0x7f49a80bfd70 2023-11-06T20:45:09.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49addff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a81592b0 con 0x7f49a80c5d50 2023-11-06T20:45:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c1840 0x7f49a80afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c1840 0x7f49a80afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36428/0 (socket says 172.21.15.33:36428) 2023-11-06T20:45:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a6ffd700 1 -- 172.21.15.33:0/1458185394 learned_addr learned my addr 172.21.15.33:0/1458185394 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a77fe700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a80bfd70 0x7f49a8150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a7fff700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f49a80c5d50 0x7f49a8153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a6ffd700 1 -- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f49a80c5d50 msgr2=0x7f49a8153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a6ffd700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f49a80c5d50 0x7f49a8153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a6ffd700 1 -- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a80bfd70 msgr2=0x7f49a8150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a6ffd700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a80bfd70 0x7f49a8150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a6ffd700 1 -- 172.21.15.33:0/1458185394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c009370 con 0x7f49a80c1840 2023-11-06T20:45:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a77fe700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a80bfd70 0x7f49a8150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.342+0000 7f49a7fff700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f49a80c5d50 0x7f49a8153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.343+0000 7f49a6ffd700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c1840 0x7f49a80afb20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f49a806bc90 tx=0x7f499c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.343+0000 7f498f7fe700 1 -- 172.21.15.33:0/1458185394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c00bae0 con 0x7f49a80c1840 2023-11-06T20:45:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.343+0000 7f498f7fe700 1 -- 172.21.15.33:0/1458185394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c000c90 con 0x7f49a80c1840 2023-11-06T20:45:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.343+0000 7f49addff700 1 -- 172.21.15.33:0/1458185394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a8159590 con 0x7f49a80c1840 2023-11-06T20:45:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.343+0000 7f498f7fe700 1 -- 172.21.15.33:0/1458185394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c01b640 con 0x7f49a80c1840 2023-11-06T20:45:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.343+0000 7f49addff700 1 -- 172.21.15.33:0/1458185394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a8158910 con 0x7f49a80c1840 2023-11-06T20:45:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.344+0000 7f49addff700 1 -- 172.21.15.33:0/1458185394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a8159ab0 con 0x7f49a80c1840 2023-11-06T20:45:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.345+0000 7f498f7fe700 1 -- 172.21.15.33:0/1458185394 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f499c01ba30 con 0x7f49a80c1840 2023-11-06T20:45:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.346+0000 7f498f7fe700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f499405f150 0x7f4994061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.346+0000 7f498f7fe700 1 -- 172.21.15.33:0/1458185394 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f499c07bda0 con 0x7f49a80c1840 2023-11-06T20:45:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.346+0000 7f49a77fe700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f499405f150 0x7f4994061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.347+0000 7f49a77fe700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f499405f150 0x7f4994061610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f4998000c00 tx=0x7f4998009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.350+0000 7f498f7fe700 1 -- 172.21.15.33:0/1458185394 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f49a8159ab0 con 0x7f49a80c1840 2023-11-06T20:45:09.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.524+0000 7f49addff700 1 -- 172.21.15.33:0/1458185394 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49a8159ab0 con 0x7f499405f150 2023-11-06T20:45:09.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.535+0000 7f498f7fe700 1 -- 172.21.15.33:0/1458185394 <== mgr.4103 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354516 (secure 0 0 0) 0x7f49a8159ab0 con 0x7f499405f150 2023-11-06T20:45:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:09.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 -- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f499405f150 msgr2=0x7f4994061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f499405f150 0x7f4994061610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f4998000c00 tx=0x7f4998009040).stop 2023-11-06T20:45:09.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 -- 172.21.15.33:0/1458185394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c1840 msgr2=0x7f49a80afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c1840 0x7f49a80afb20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f49a806bc90 tx=0x7f499c017000).stop 2023-11-06T20:45:09.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 -- 172.21.15.33:0/1458185394 shutdown_connections 2023-11-06T20:45:09.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f499405f150 0x7f4994061610 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f49a80c5d50 0x7f49a8153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a80bfd70 0x7f49a8150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 --2- 172.21.15.33:0/1458185394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a80c1840 0x7f49a80afb20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 -- 172.21.15.33:0/1458185394 >> 172.21.15.33:0/1458185394 conn(0x7f49a805ec80 msgr2=0x7f49a80ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:09.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.539+0000 7f49addff700 1 -- 172.21.15.33:0/1458185394 shutdown_connections 2023-11-06T20:45:09.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.540+0000 7f49addff700 1 -- 172.21.15.33:0/1458185394 wait complete. 2023-11-06T20:45:09.544 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:45:09.556 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-06T20:45:08.823293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379644,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595076,"statfs":{"total":773094113280,"available":770657357824,"internally_reserved":0,"allocated":2945024,"data_stored":1192131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4861,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.325528"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3f","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712002+0000","last_change":"2023-11-06T20:45:04.213243+0000","last_active":"2023-11-06T20:45:04.712002+0000","last_peered":"2023-11-06T20:45:04.712002+0000","last_clean":"2023-11-06T20:45:04.712002+0000","last_became_active":"2023-11-06T20:45:04.212975+0000","last_became_peered":"2023-11-06T20:45:04.212975+0000","last_unstale":"2023-11-06T20:45:04.712002+0000","last_undegraded":"2023-11-06T20:45:04.712002+0000","last_fullsized":"2023-11-06T20:45:04.712002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173152+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699604+0000","last_change":"2023-11-06T20:45:04.196531+0000","last_active":"2023-11-06T20:45:04.699604+0000","last_peered":"2023-11-06T20:45:04.699604+0000","last_clean":"2023-11-06T20:45:04.699604+0000","last_became_active":"2023-11-06T20:45:04.194888+0000","last_became_peered":"2023-11-06T20:45:04.194888+0000","last_unstale":"2023-11-06T20:45:04.699604+0000","last_undegraded":"2023-11-06T20:45:04.699604+0000","last_fullsized":"2023-11-06T20:45:04.699604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175110+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174673+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703778+0000","last_change":"2023-11-06T20:45:04.699346+0000","last_active":"2023-11-06T20:45:04.703778+0000","last_peered":"2023-11-06T20:45:04.703778+0000","last_clean":"2023-11-06T20:45:04.703778+0000","last_became_active":"2023-11-06T20:45:04.698931+0000","last_became_peered":"2023-11-06T20:45:04.698931+0000","last_unstale":"2023-11-06T20:45:04.703778+0000","last_undegraded":"2023-11-06T20:45:04.703778+0000","last_fullsized":"2023-11-06T20:45:04.703778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.176517+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680925+0000","last_change":"2023-11-06T20:45:04.212407+0000","last_active":"2023-11-06T20:45:04.680925+0000","last_peered":"2023-11-06T20:45:04.680925+0000","last_clean":"2023-11-06T20:45:04.680925+0000","last_became_active":"2023-11-06T20:45:04.211888+0000","last_became_peered":"2023-11-06T20:45:04.211888+0000","last_unstale":"2023-11-06T20:45:04.680925+0000","last_undegraded":"2023-11-06T20:45:04.680925+0000","last_fullsized":"2023-11-06T20:45:04.680925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680573+0000","last_change":"2023-11-06T20:45:04.203081+0000","last_active":"2023-11-06T20:45:04.680573+0000","last_peered":"2023-11-06T20:45:04.680573+0000","last_clean":"2023-11-06T20:45:04.680573+0000","last_became_active":"2023-11-06T20:45:04.199530+0000","last_became_peered":"2023-11-06T20:45:04.199530+0000","last_unstale":"2023-11-06T20:45:04.680573+0000","last_undegraded":"2023-11-06T20:45:04.680573+0000","last_fullsized":"2023-11-06T20:45:04.680573+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700837+0000","last_change":"2023-11-06T20:45:04.207865+0000","last_active":"2023-11-06T20:45:04.700837+0000","last_peered":"2023-11-06T20:45:04.700837+0000","last_clean":"2023-11-06T20:45:04.700837+0000","last_became_active":"2023-11-06T20:45:04.207584+0000","last_became_peered":"2023-11-06T20:45:04.207584+0000","last_unstale":"2023-11-06T20:45:04.700837+0000","last_undegraded":"2023-11-06T20:45:04.700837+0000","last_fullsized":"2023-11-06T20:45:04.700837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175778+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715752+0000","last_change":"2023-11-06T20:45:04.711145+0000","last_active":"2023-11-06T20:45:04.715752+0000","last_peered":"2023-11-06T20:45:04.715752+0000","last_clean":"2023-11-06T20:45:04.715752+0000","last_became_active":"2023-11-06T20:45:04.709234+0000","last_became_peered":"2023-11-06T20:45:04.709234+0000","last_unstale":"2023-11-06T20:45:04.715752+0000","last_undegraded":"2023-11-06T20:45:04.715752+0000","last_fullsized":"2023-11-06T20:45:04.715752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182188+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715259+0000","last_change":"2023-11-06T20:45:04.206902+0000","last_active":"2023-11-06T20:45:04.715259+0000","last_peered":"2023-11-06T20:45:04.715259+0000","last_clean":"2023-11-06T20:45:04.715259+0000","last_became_active":"2023-11-06T20:45:04.206583+0000","last_became_peered":"2023-11-06T20:45:04.206583+0000","last_unstale":"2023-11-06T20:45:04.715259+0000","last_undegraded":"2023-11-06T20:45:04.715259+0000","last_fullsized":"2023-11-06T20:45:04.715259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712482+0000","last_change":"2023-11-06T20:45:04.707038+0000","last_active":"2023-11-06T20:45:04.712482+0000","last_peered":"2023-11-06T20:45:04.712482+0000","last_clean":"2023-11-06T20:45:04.712482+0000","last_became_active":"2023-11-06T20:45:04.704732+0000","last_became_peered":"2023-11-06T20:45:04.704732+0000","last_unstale":"2023-11-06T20:45:04.712482+0000","last_undegraded":"2023-11-06T20:45:04.712482+0000","last_fullsized":"2023-11-06T20:45:04.712482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178534+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174676+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699919+0000","last_change":"2023-11-06T20:45:04.698880+0000","last_active":"2023-11-06T20:45:04.699919+0000","last_peered":"2023-11-06T20:45:04.699919+0000","last_clean":"2023-11-06T20:45:04.699919+0000","last_became_active":"2023-11-06T20:45:04.697976+0000","last_became_peered":"2023-11-06T20:45:04.697976+0000","last_unstale":"2023-11-06T20:45:04.699919+0000","last_undegraded":"2023-11-06T20:45:04.699919+0000","last_fullsized":"2023-11-06T20:45:04.699919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179850+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703867+0000","last_change":"2023-11-06T20:45:04.701051+0000","last_active":"2023-11-06T20:45:04.703867+0000","last_peered":"2023-11-06T20:45:04.703867+0000","last_clean":"2023-11-06T20:45:04.703867+0000","last_became_active":"2023-11-06T20:45:04.700684+0000","last_became_peered":"2023-11-06T20:45:04.700684+0000","last_unstale":"2023-11-06T20:45:04.703867+0000","last_undegraded":"2023-11-06T20:45:04.703867+0000","last_fullsized":"2023-11-06T20:45:04.703867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179400+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182863+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178154+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180329+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180016+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180343+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681053+0000","last_change":"2023-11-06T20:45:04.208631+0000","last_active":"2023-11-06T20:45:04.681053+0000","last_peered":"2023-11-06T20:45:04.681053+0000","last_clean":"2023-11-06T20:45:04.681053+0000","last_became_active":"2023-11-06T20:45:04.208313+0000","last_became_peered":"2023-11-06T20:45:04.208313+0000","last_unstale":"2023-11-06T20:45:04.681053+0000","last_undegraded":"2023-11-06T20:45:04.681053+0000","last_fullsized":"2023-11-06T20:45:04.681053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178850+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680469+0000","last_change":"2023-11-06T20:45:04.208028+0000","last_active":"2023-11-06T20:45:04.680469+0000","last_peered":"2023-11-06T20:45:04.680469+0000","last_clean":"2023-11-06T20:45:04.680469+0000","last_became_active":"2023-11-06T20:45:04.207152+0000","last_became_peered":"2023-11-06T20:45:04.207152+0000","last_unstale":"2023-11-06T20:45:04.680469+0000","last_undegraded":"2023-11-06T20:45:04.680469+0000","last_fullsized":"2023-11-06T20:45:04.680469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714410+0000","last_change":"2023-11-06T20:45:04.710247+0000","last_active":"2023-11-06T20:45:04.714410+0000","last_peered":"2023-11-06T20:45:04.714410+0000","last_clean":"2023-11-06T20:45:04.714410+0000","last_became_active":"2023-11-06T20:45:04.709893+0000","last_became_peered":"2023-11-06T20:45:04.709893+0000","last_unstale":"2023-11-06T20:45:04.714410+0000","last_undegraded":"2023-11-06T20:45:04.714410+0000","last_fullsized":"2023-11-06T20:45:04.714410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.176372+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175234+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173525+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700044+0000","last_change":"2023-11-06T20:45:04.693933+0000","last_active":"2023-11-06T20:45:04.700044+0000","last_peered":"2023-11-06T20:45:04.700044+0000","last_clean":"2023-11-06T20:45:04.700044+0000","last_became_active":"2023-11-06T20:45:04.693204+0000","last_became_peered":"2023-11-06T20:45:04.693204+0000","last_unstale":"2023-11-06T20:45:04.700044+0000","last_undegraded":"2023-11-06T20:45:04.700044+0000","last_fullsized":"2023-11-06T20:45:04.700044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681743+0000","last_change":"2023-11-06T20:45:04.213724+0000","last_active":"2023-11-06T20:45:04.681743+0000","last_peered":"2023-11-06T20:45:04.681743+0000","last_clean":"2023-11-06T20:45:04.681743+0000","last_became_active":"2023-11-06T20:45:04.211299+0000","last_became_peered":"2023-11-06T20:45:04.211299+0000","last_unstale":"2023-11-06T20:45:04.681743+0000","last_undegraded":"2023-11-06T20:45:04.681743+0000","last_fullsized":"2023-11-06T20:45:04.681743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179941+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703797+0000","last_change":"2023-11-06T20:45:04.701216+0000","last_active":"2023-11-06T20:45:04.703797+0000","last_peered":"2023-11-06T20:45:04.703797+0000","last_clean":"2023-11-06T20:45:04.703797+0000","last_became_active":"2023-11-06T20:45:04.700956+0000","last_became_peered":"2023-11-06T20:45:04.700956+0000","last_unstale":"2023-11-06T20:45:04.703797+0000","last_undegraded":"2023-11-06T20:45:04.703797+0000","last_fullsized":"2023-11-06T20:45:04.703797+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180325+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680966+0000","last_change":"2023-11-06T20:45:04.205274+0000","last_active":"2023-11-06T20:45:04.680966+0000","last_peered":"2023-11-06T20:45:04.680966+0000","last_clean":"2023-11-06T20:45:04.680966+0000","last_became_active":"2023-11-06T20:45:04.204948+0000","last_became_peered":"2023-11-06T20:45:04.204948+0000","last_unstale":"2023-11-06T20:45:04.680966+0000","last_undegraded":"2023-11-06T20:45:04.680966+0000","last_fullsized":"2023-11-06T20:45:04.680966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.181516+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699237+0000","last_change":"2023-11-06T20:45:04.199975+0000","last_active":"2023-11-06T20:45:04.699237+0000","last_peered":"2023-11-06T20:45:04.699237+0000","last_clean":"2023-11-06T20:45:04.699237+0000","last_became_active":"2023-11-06T20:45:04.195195+0000","last_became_peered":"2023-11-06T20:45:04.195195+0000","last_unstale":"2023-11-06T20:45:04.699237+0000","last_undegraded":"2023-11-06T20:45:04.699237+0000","last_fullsized":"2023-11-06T20:45:04.699237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179244+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174491+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681802+0000","last_change":"2023-11-06T20:45:04.209004+0000","last_active":"2023-11-06T20:45:04.681802+0000","last_peered":"2023-11-06T20:45:04.681802+0000","last_clean":"2023-11-06T20:45:04.681802+0000","last_became_active":"2023-11-06T20:45:04.208295+0000","last_became_peered":"2023-11-06T20:45:04.208295+0000","last_unstale":"2023-11-06T20:45:04.681802+0000","last_undegraded":"2023-11-06T20:45:04.681802+0000","last_fullsized":"2023-11-06T20:45:04.681802+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680947+0000","last_change":"2023-11-06T20:45:04.200869+0000","last_active":"2023-11-06T20:45:04.680947+0000","last_peered":"2023-11-06T20:45:04.680947+0000","last_clean":"2023-11-06T20:45:04.680947+0000","last_became_active":"2023-11-06T20:45:04.200557+0000","last_became_peered":"2023-11-06T20:45:04.200557+0000","last_unstale":"2023-11-06T20:45:04.680947+0000","last_undegraded":"2023-11-06T20:45:04.680947+0000","last_fullsized":"2023-11-06T20:45:04.680947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696708+0000","last_change":"2023-11-06T20:45:04.205999+0000","last_active":"2023-11-06T20:45:04.696708+0000","last_peered":"2023-11-06T20:45:04.696708+0000","last_clean":"2023-11-06T20:45:04.696708+0000","last_became_active":"2023-11-06T20:45:04.205244+0000","last_became_peered":"2023-11-06T20:45:04.205244+0000","last_unstale":"2023-11-06T20:45:04.696708+0000","last_undegraded":"2023-11-06T20:45:04.696708+0000","last_fullsized":"2023-11-06T20:45:04.696708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174499+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696679+0000","last_change":"2023-11-06T20:45:04.194284+0000","last_active":"2023-11-06T20:45:04.696679+0000","last_peered":"2023-11-06T20:45:04.696679+0000","last_clean":"2023-11-06T20:45:04.696679+0000","last_became_active":"2023-11-06T20:45:04.193972+0000","last_became_peered":"2023-11-06T20:45:04.193972+0000","last_unstale":"2023-11-06T20:45:04.696679+0000","last_undegraded":"2023-11-06T20:45:04.696679+0000","last_fullsized":"2023-11-06T20:45:04.696679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175433+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700556+0000","last_change":"2023-11-06T20:45:04.699519+0000","last_active":"2023-11-06T20:45:04.700556+0000","last_peered":"2023-11-06T20:45:04.700556+0000","last_clean":"2023-11-06T20:45:04.700556+0000","last_became_active":"2023-11-06T20:45:04.699244+0000","last_became_peered":"2023-11-06T20:45:04.699244+0000","last_unstale":"2023-11-06T20:45:04.700556+0000","last_undegraded":"2023-11-06T20:45:04.700556+0000","last_fullsized":"2023-11-06T20:45:04.700556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182638+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182611+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698113+0000","last_change":"2023-11-06T20:45:04.694154+0000","last_active":"2023-11-06T20:45:04.698113+0000","last_peered":"2023-11-06T20:45:04.698113+0000","last_clean":"2023-11-06T20:45:04.698113+0000","last_became_active":"2023-11-06T20:45:04.693824+0000","last_became_peered":"2023-11-06T20:45:04.693824+0000","last_unstale":"2023-11-06T20:45:04.698113+0000","last_undegraded":"2023-11-06T20:45:04.698113+0000","last_fullsized":"2023-11-06T20:45:04.698113+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174583+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175488+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179685+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.184627+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714056+0000","last_change":"2023-11-06T20:45:04.199156+0000","last_active":"2023-11-06T20:45:04.714056+0000","last_peered":"2023-11-06T20:45:04.714056+0000","last_clean":"2023-11-06T20:45:04.714056+0000","last_became_active":"2023-11-06T20:45:04.198858+0000","last_became_peered":"2023-11-06T20:45:04.198858+0000","last_unstale":"2023-11-06T20:45:04.714056+0000","last_undegraded":"2023-11-06T20:45:04.714056+0000","last_fullsized":"2023-11-06T20:45:04.714056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715339+0000","last_change":"2023-11-06T20:45:04.199860+0000","last_active":"2023-11-06T20:45:04.715339+0000","last_peered":"2023-11-06T20:45:04.715339+0000","last_clean":"2023-11-06T20:45:04.715339+0000","last_became_active":"2023-11-06T20:45:04.197409+0000","last_became_peered":"2023-11-06T20:45:04.197409+0000","last_unstale":"2023-11-06T20:45:04.715339+0000","last_undegraded":"2023-11-06T20:45:04.715339+0000","last_fullsized":"2023-11-06T20:45:04.715339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713635+0000","last_change":"2023-11-06T20:45:04.707405+0000","last_active":"2023-11-06T20:45:04.713635+0000","last_peered":"2023-11-06T20:45:04.713635+0000","last_clean":"2023-11-06T20:45:04.713635+0000","last_became_active":"2023-11-06T20:45:04.704832+0000","last_became_peered":"2023-11-06T20:45:04.704832+0000","last_unstale":"2023-11-06T20:45:04.713635+0000","last_undegraded":"2023-11-06T20:45:04.713635+0000","last_fullsized":"2023-11-06T20:45:04.713635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182164+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713102+0000","last_change":"2023-11-06T20:45:04.703423+0000","last_active":"2023-11-06T20:45:04.713102+0000","last_peered":"2023-11-06T20:45:04.713102+0000","last_clean":"2023-11-06T20:45:04.713102+0000","last_became_active":"2023-11-06T20:45:04.703108+0000","last_became_peered":"2023-11-06T20:45:04.703108+0000","last_unstale":"2023-11-06T20:45:04.713102+0000","last_undegraded":"2023-11-06T20:45:04.713102+0000","last_fullsized":"2023-11-06T20:45:04.713102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179821+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174040+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.172938+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178482+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698151+0000","last_change":"2023-11-06T20:45:04.207106+0000","last_active":"2023-11-06T20:45:04.698151+0000","last_peered":"2023-11-06T20:45:04.698151+0000","last_clean":"2023-11-06T20:45:04.698151+0000","last_became_active":"2023-11-06T20:45:04.206814+0000","last_became_peered":"2023-11-06T20:45:04.206814+0000","last_unstale":"2023-11-06T20:45:04.698151+0000","last_undegraded":"2023-11-06T20:45:04.698151+0000","last_fullsized":"2023-11-06T20:45:04.698151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178510+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173953+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:57.427499+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:44:57.427499+0000","last_peered":"2023-11-06T20:44:57.427499+0000","last_clean":"2023-11-06T20:44:57.427499+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:44:57.427499+0000","last_undegraded":"2023-11-06T20:44:57.427499+0000","last_fullsized":"2023-11-06T20:44:57.427499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.177557+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046724+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:44:59.046724+0000","last_peered":"2023-11-06T20:44:59.046724+0000","last_clean":"2023-11-06T20:44:59.046724+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:44:59.046724+0000","last_undegraded":"2023-11-06T20:44:59.046724+0000","last_fullsized":"2023-11-06T20:44:59.046724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.176976+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.047320+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:44:59.047320+0000","last_peered":"2023-11-06T20:44:59.047320+0000","last_clean":"2023-11-06T20:44:59.047320+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:44:59.047320+0000","last_undegraded":"2023-11-06T20:44:59.047320+0000","last_fullsized":"2023-11-06T20:44:59.047320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179510+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046727+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:44:59.046727+0000","last_peered":"2023-11-06T20:44:59.046727+0000","last_clean":"2023-11-06T20:44:59.046727+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:44:59.046727+0000","last_undegraded":"2023-11-06T20:44:59.046727+0000","last_fullsized":"2023-11-06T20:44:59.046727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696761+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:45:04.696761+0000","last_peered":"2023-11-06T20:45:04.696761+0000","last_clean":"2023-11-06T20:45:04.696761+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:45:04.696761+0000","last_undegraded":"2023-11-06T20:45:04.696761+0000","last_fullsized":"2023-11-06T20:45:04.696761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182200+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046740+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:44:59.046740+0000","last_peered":"2023-11-06T20:44:59.046740+0000","last_clean":"2023-11-06T20:44:59.046740+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:44:59.046740+0000","last_undegraded":"2023-11-06T20:44:59.046740+0000","last_fullsized":"2023-11-06T20:44:59.046740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179915+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046755+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:44:59.046755+0000","last_peered":"2023-11-06T20:44:59.046755+0000","last_clean":"2023-11-06T20:44:59.046755+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:44:59.046755+0000","last_undegraded":"2023-11-06T20:44:59.046755+0000","last_fullsized":"2023-11-06T20:44:59.046755+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.184211+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.047396+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:44:59.047396+0000","last_peered":"2023-11-06T20:44:59.047396+0000","last_clean":"2023-11-06T20:44:59.047396+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:44:59.047396+0000","last_undegraded":"2023-11-06T20:44:59.047396+0000","last_fullsized":"2023-11-06T20:44:59.047396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173926+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046729+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:44:59.046729+0000","last_peered":"2023-11-06T20:44:59.046729+0000","last_clean":"2023-11-06T20:44:59.046729+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:44:59.046729+0000","last_undegraded":"2023-11-06T20:44:59.046729+0000","last_fullsized":"2023-11-06T20:44:59.046729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":2},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":1468,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.1749999999999998,"5min":5.1749999999999998,"15min":5.1749999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.624}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65600000000000003}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.202,"5min":5.202,"15min":5.202},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.77200000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72999999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.75700000000000001}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46700000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.88700000000000001}]},{"osd":2,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.82999999999999996}]},{"osd":3,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.78300000000000003}]},{"osd":7,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.749}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":409600,"data_stored":167482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.42499999999999999}]},{"osd":2,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.439}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.38200000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.60499999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297436,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074404,"statfs":{"total":96636764160,"available":96332189696,"internally_reserved":0,"allocated":348160,"data_stored":139715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.51000000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.5}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.746},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.73299999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.70699999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":409600,"data_stored":167696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.38100000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.69299999999999995}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297436,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074404,"statfs":{"total":96636764160,"available":96332189696,"internally_reserved":0,"allocated":348160,"data_stored":139715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.38},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.44}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.53500000000000003}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64700000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.621}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.76100000000000001}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.86499999999999999}]},{"osd":1,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.93200000000000005}]},{"osd":2,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.95699999999999996}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.748},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.76000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297436,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074404,"statfs":{"total":96636764160,"available":96332189696,"internally_reserved":0,"allocated":348160,"data_stored":139715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66200000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68000000000000005}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.745}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.69899999999999995}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.70799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:45:09.563 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:45:09.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.792+0000 7f18eb3ff700 1 Processor -- start 2023-11-06T20:45:09.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.793+0000 7f18eb3ff700 1 -- start start 2023-11-06T20:45:09.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.794+0000 7f18eb3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f18e40c5d50 0x7f18e40c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.794+0000 7f18eb3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f18e40bfd70 0x7f18e40c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.794+0000 7f18eb3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e41554a0 con 0x7f18e40c1840 2023-11-06T20:45:09.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.794+0000 7f18eb3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e4154960 con 0x7f18e40bfd70 2023-11-06T20:45:09.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.794+0000 7f18eb3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e40c66c0 con 0x7f18e40c5d50 2023-11-06T20:45:09.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.794+0000 7f18e919b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f18e40c1840 0x7f18e40a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57980/0 (socket says 172.21.15.33:57980) 2023-11-06T20:45:09.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.794+0000 7f18e919b700 1 -- 172.21.15.33:0/938936814 learned_addr learned my addr 172.21.15.33:0/938936814 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:09.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.794+0000 7f18e999c700 1 --2- 172.21.15.33:0/938936814 >> v2:172.21.15.33:3300/0 conn(0x7f18e40bfd70 0x7f18e40c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.794+0000 7f18e899a700 1 --2- 172.21.15.33:0/938936814 >> v2:172.21.15.69:3300/0 conn(0x7f18e40c5d50 0x7f18e40c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.795+0000 7f18dbfff700 1 -- 172.21.15.33:0/938936814 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096651717 0 0) 0x7f18e41554a0 con 0x7f18e40c1840 2023-11-06T20:45:09.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.795+0000 7f18dbfff700 1 -- 172.21.15.33:0/938936814 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d00033f0 con 0x7f18e40c1840 2023-11-06T20:45:09.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.795+0000 7f18e999c700 1 -- 172.21.15.33:0/938936814 >> v2:172.21.15.69:3300/0 conn(0x7f18e40c5d50 msgr2=0x7f18e40c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:09.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.795+0000 7f18e999c700 1 --2- 172.21.15.33:0/938936814 >> v2:172.21.15.69:3300/0 conn(0x7f18e40c5d50 0x7f18e40c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.795+0000 7f18e999c700 1 -- 172.21.15.33:0/938936814 >> v1:172.21.15.33:6789/0 conn(0x7f18e40c1840 legacy=0x7f18e40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.795+0000 7f18e999c700 1 -- 172.21.15.33:0/938936814 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4155930 con 0x7f18e40bfd70 2023-11-06T20:45:09.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.795+0000 7f18e999c700 1 --2- 172.21.15.33:0/938936814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40bfd70 0x7f18e40c0180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f18e40adb40 tx=0x7f18dc009780).ready entity=mon.0 client_cookie=821131766549ff11 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:09.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.795+0000 7f18dbfff700 1 -- 172.21.15.33:0/938936814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00e070 con 0x7f18e40bfd70 2023-11-06T20:45:09.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.795+0000 7f18dbfff700 1 -- 172.21.15.33:0/938936814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc00bab0 con 0x7f18e40bfd70 2023-11-06T20:45:09.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.795+0000 7f18dbfff700 1 -- 172.21.15.33:0/938936814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00be30 con 0x7f18e40bfd70 2023-11-06T20:45:09.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.796+0000 7f18eb3ff700 1 -- 172.21.15.33:0/938936814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40bfd70 msgr2=0x7f18e40c0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.796+0000 7f18eb3ff700 1 --2- 172.21.15.33:0/938936814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40bfd70 0x7f18e40c0180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f18e40adb40 tx=0x7f18dc009780).stop 2023-11-06T20:45:09.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.796+0000 7f18eb3ff700 1 -- 172.21.15.33:0/938936814 shutdown_connections 2023-11-06T20:45:09.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.796+0000 7f18eb3ff700 1 --2- 172.21.15.33:0/938936814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40bfd70 0x7f18e40c0180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.796+0000 7f18eb3ff700 1 --2- 172.21.15.33:0/938936814 >> v2:172.21.15.69:3300/0 conn(0x7f18e40c5d50 0x7f18e40c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.796+0000 7f18eb3ff700 1 -- 172.21.15.33:0/938936814 >> 172.21.15.33:0/938936814 conn(0x7f18e405ec80 msgr2=0x7f18e40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:09.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.796+0000 7f18eb3ff700 1 -- 172.21.15.33:0/938936814 shutdown_connections 2023-11-06T20:45:09.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.796+0000 7f18eb3ff700 1 -- 172.21.15.33:0/938936814 wait complete. 2023-11-06T20:45:09.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.797+0000 7f18eb3ff700 1 Processor -- start 2023-11-06T20:45:09.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.798+0000 7f18eb3ff700 1 -- start start 2023-11-06T20:45:09.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18eb3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18e40bfd70 0x7f18e414b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18eb3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18e40c1840 0x7f18e4148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18eb3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40c5d50 0x7f18e4159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18eb3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e41483e0 con 0x7f18e40c5d50 2023-11-06T20:45:09.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18eb3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e41498c0 con 0x7f18e40c1840 2023-11-06T20:45:09.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18eb3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40aeb30 con 0x7f18e40bfd70 2023-11-06T20:45:09.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40c5d50 0x7f18e4159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40c5d50 0x7f18e4159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36446/0 (socket says 172.21.15.33:36446) 2023-11-06T20:45:09.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e999c700 1 -- 172.21.15.33:0/1653547629 learned_addr learned my addr 172.21.15.33:0/1653547629 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:09.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e919b700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18e40bfd70 0x7f18e414b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e899a700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18e40c1840 0x7f18e4148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e999c700 1 -- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18e40bfd70 msgr2=0x7f18e414b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e999c700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18e40bfd70 0x7f18e414b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e999c700 1 -- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18e40c1840 msgr2=0x7f18e4148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:09.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e999c700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18e40c1840 0x7f18e4148e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:09.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e999c700 1 -- 172.21.15.33:0/1653547629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc009a20 con 0x7f18e40c5d50 2023-11-06T20:45:09.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e899a700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18e40c1840 0x7f18e4148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:09.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.799+0000 7f18e999c700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40c5d50 0x7f18e4159240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f18e406bc90 tx=0x7f18e4155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:09.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.800+0000 7f18d97fa700 1 -- 172.21.15.33:0/1653547629 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00b640 con 0x7f18e40c5d50 2023-11-06T20:45:09.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.800+0000 7f18d97fa700 1 -- 172.21.15.33:0/1653547629 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc01d630 con 0x7f18e40c5d50 2023-11-06T20:45:09.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.800+0000 7f18d97fa700 1 -- 172.21.15.33:0/1653547629 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc01d9b0 con 0x7f18e40c5d50 2023-11-06T20:45:09.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.800+0000 7f18eb3ff700 1 -- 172.21.15.33:0/1653547629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e40ae200 con 0x7f18e40c5d50 2023-11-06T20:45:09.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.800+0000 7f18eb3ff700 1 -- 172.21.15.33:0/1653547629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e40ae7c0 con 0x7f18e40c5d50 2023-11-06T20:45:09.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.801+0000 7f18eb3ff700 1 -- 172.21.15.33:0/1653547629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e406c6c0 con 0x7f18e40c5d50 2023-11-06T20:45:09.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.802+0000 7f18d97fa700 1 -- 172.21.15.33:0/1653547629 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f18dc003040 con 0x7f18e40c5d50 2023-11-06T20:45:09.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.803+0000 7f18d97fa700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18c805f150 0x7f18c8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:09.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.803+0000 7f18d97fa700 1 -- 172.21.15.33:0/1653547629 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f18dc07bff0 con 0x7f18e40c5d50 2023-11-06T20:45:09.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.803+0000 7f18e919b700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18c805f150 0x7f18c8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:09.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.804+0000 7f18e919b700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18c805f150 0x7f18c8061610 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f18e414d780 tx=0x7f18e0009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:09.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.807+0000 7f18d97fa700 1 -- 172.21.15.33:0/1653547629 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f18dc0aaaf0 con 0x7f18e40c5d50 2023-11-06T20:45:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.984+0000 7f18eb3ff700 1 -- 172.21.15.33:0/1653547629 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18e40aec70 con 0x7f18c805f150 2023-11-06T20:45:09.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.995+0000 7f18d97fa700 1 -- 172.21.15.33:0/1653547629 <== mgr.4103 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354516 (secure 0 0 0) 0x7f18e40aec70 con 0x7f18c805f150 2023-11-06T20:45:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.999+0000 7f18eb3ff700 1 -- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18c805f150 msgr2=0x7f18c8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.999+0000 7f18eb3ff700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18c805f150 0x7f18c8061610 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f18e414d780 tx=0x7f18e0009040).stop 2023-11-06T20:45:10.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.999+0000 7f18eb3ff700 1 -- 172.21.15.33:0/1653547629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40c5d50 msgr2=0x7f18e4159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:09.999+0000 7f18eb3ff700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40c5d50 0x7f18e4159240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f18e406bc90 tx=0x7f18e4155410).stop 2023-11-06T20:45:10.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.000+0000 7f18eb3ff700 1 -- 172.21.15.33:0/1653547629 shutdown_connections 2023-11-06T20:45:10.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.000+0000 7f18eb3ff700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18c805f150 0x7f18c8061610 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.000+0000 7f18eb3ff700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18e40bfd70 0x7f18e414b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.000+0000 7f18eb3ff700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e40c5d50 0x7f18e4159240 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.000+0000 7f18eb3ff700 1 --2- 172.21.15.33:0/1653547629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18e40c1840 0x7f18e4148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.000+0000 7f18eb3ff700 1 -- 172.21.15.33:0/1653547629 >> 172.21.15.33:0/1653547629 conn(0x7f18e405ec80 msgr2=0x7f18e4152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.000+0000 7f18eb3ff700 1 -- 172.21.15.33:0/1653547629 shutdown_connections 2023-11-06T20:45:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.000+0000 7f18eb3ff700 1 -- 172.21.15.33:0/1653547629 wait complete. 2023-11-06T20:45:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:45:10.016 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-06T20:45:08.823293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379644,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595076,"statfs":{"total":773094113280,"available":770657357824,"internally_reserved":0,"allocated":2945024,"data_stored":1192131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4861,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.325528"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3f","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712002+0000","last_change":"2023-11-06T20:45:04.213243+0000","last_active":"2023-11-06T20:45:04.712002+0000","last_peered":"2023-11-06T20:45:04.712002+0000","last_clean":"2023-11-06T20:45:04.712002+0000","last_became_active":"2023-11-06T20:45:04.212975+0000","last_became_peered":"2023-11-06T20:45:04.212975+0000","last_unstale":"2023-11-06T20:45:04.712002+0000","last_undegraded":"2023-11-06T20:45:04.712002+0000","last_fullsized":"2023-11-06T20:45:04.712002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173152+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699604+0000","last_change":"2023-11-06T20:45:04.196531+0000","last_active":"2023-11-06T20:45:04.699604+0000","last_peered":"2023-11-06T20:45:04.699604+0000","last_clean":"2023-11-06T20:45:04.699604+0000","last_became_active":"2023-11-06T20:45:04.194888+0000","last_became_peered":"2023-11-06T20:45:04.194888+0000","last_unstale":"2023-11-06T20:45:04.699604+0000","last_undegraded":"2023-11-06T20:45:04.699604+0000","last_fullsized":"2023-11-06T20:45:04.699604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175110+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174673+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703778+0000","last_change":"2023-11-06T20:45:04.699346+0000","last_active":"2023-11-06T20:45:04.703778+0000","last_peered":"2023-11-06T20:45:04.703778+0000","last_clean":"2023-11-06T20:45:04.703778+0000","last_became_active":"2023-11-06T20:45:04.698931+0000","last_became_peered":"2023-11-06T20:45:04.698931+0000","last_unstale":"2023-11-06T20:45:04.703778+0000","last_undegraded":"2023-11-06T20:45:04.703778+0000","last_fullsized":"2023-11-06T20:45:04.703778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.176517+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680925+0000","last_change":"2023-11-06T20:45:04.212407+0000","last_active":"2023-11-06T20:45:04.680925+0000","last_peered":"2023-11-06T20:45:04.680925+0000","last_clean":"2023-11-06T20:45:04.680925+0000","last_became_active":"2023-11-06T20:45:04.211888+0000","last_became_peered":"2023-11-06T20:45:04.211888+0000","last_unstale":"2023-11-06T20:45:04.680925+0000","last_undegraded":"2023-11-06T20:45:04.680925+0000","last_fullsized":"2023-11-06T20:45:04.680925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680573+0000","last_change":"2023-11-06T20:45:04.203081+0000","last_active":"2023-11-06T20:45:04.680573+0000","last_peered":"2023-11-06T20:45:04.680573+0000","last_clean":"2023-11-06T20:45:04.680573+0000","last_became_active":"2023-11-06T20:45:04.199530+0000","last_became_peered":"2023-11-06T20:45:04.199530+0000","last_unstale":"2023-11-06T20:45:04.680573+0000","last_undegraded":"2023-11-06T20:45:04.680573+0000","last_fullsized":"2023-11-06T20:45:04.680573+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700837+0000","last_change":"2023-11-06T20:45:04.207865+0000","last_active":"2023-11-06T20:45:04.700837+0000","last_peered":"2023-11-06T20:45:04.700837+0000","last_clean":"2023-11-06T20:45:04.700837+0000","last_became_active":"2023-11-06T20:45:04.207584+0000","last_became_peered":"2023-11-06T20:45:04.207584+0000","last_unstale":"2023-11-06T20:45:04.700837+0000","last_undegraded":"2023-11-06T20:45:04.700837+0000","last_fullsized":"2023-11-06T20:45:04.700837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175778+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715752+0000","last_change":"2023-11-06T20:45:04.711145+0000","last_active":"2023-11-06T20:45:04.715752+0000","last_peered":"2023-11-06T20:45:04.715752+0000","last_clean":"2023-11-06T20:45:04.715752+0000","last_became_active":"2023-11-06T20:45:04.709234+0000","last_became_peered":"2023-11-06T20:45:04.709234+0000","last_unstale":"2023-11-06T20:45:04.715752+0000","last_undegraded":"2023-11-06T20:45:04.715752+0000","last_fullsized":"2023-11-06T20:45:04.715752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182188+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715259+0000","last_change":"2023-11-06T20:45:04.206902+0000","last_active":"2023-11-06T20:45:04.715259+0000","last_peered":"2023-11-06T20:45:04.715259+0000","last_clean":"2023-11-06T20:45:04.715259+0000","last_became_active":"2023-11-06T20:45:04.206583+0000","last_became_peered":"2023-11-06T20:45:04.206583+0000","last_unstale":"2023-11-06T20:45:04.715259+0000","last_undegraded":"2023-11-06T20:45:04.715259+0000","last_fullsized":"2023-11-06T20:45:04.715259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712482+0000","last_change":"2023-11-06T20:45:04.707038+0000","last_active":"2023-11-06T20:45:04.712482+0000","last_peered":"2023-11-06T20:45:04.712482+0000","last_clean":"2023-11-06T20:45:04.712482+0000","last_became_active":"2023-11-06T20:45:04.704732+0000","last_became_peered":"2023-11-06T20:45:04.704732+0000","last_unstale":"2023-11-06T20:45:04.712482+0000","last_undegraded":"2023-11-06T20:45:04.712482+0000","last_fullsized":"2023-11-06T20:45:04.712482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178534+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174676+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699919+0000","last_change":"2023-11-06T20:45:04.698880+0000","last_active":"2023-11-06T20:45:04.699919+0000","last_peered":"2023-11-06T20:45:04.699919+0000","last_clean":"2023-11-06T20:45:04.699919+0000","last_became_active":"2023-11-06T20:45:04.697976+0000","last_became_peered":"2023-11-06T20:45:04.697976+0000","last_unstale":"2023-11-06T20:45:04.699919+0000","last_undegraded":"2023-11-06T20:45:04.699919+0000","last_fullsized":"2023-11-06T20:45:04.699919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179850+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703867+0000","last_change":"2023-11-06T20:45:04.701051+0000","last_active":"2023-11-06T20:45:04.703867+0000","last_peered":"2023-11-06T20:45:04.703867+0000","last_clean":"2023-11-06T20:45:04.703867+0000","last_became_active":"2023-11-06T20:45:04.700684+0000","last_became_peered":"2023-11-06T20:45:04.700684+0000","last_unstale":"2023-11-06T20:45:04.703867+0000","last_undegraded":"2023-11-06T20:45:04.703867+0000","last_fullsized":"2023-11-06T20:45:04.703867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179400+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182863+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178154+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180329+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180016+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180343+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681053+0000","last_change":"2023-11-06T20:45:04.208631+0000","last_active":"2023-11-06T20:45:04.681053+0000","last_peered":"2023-11-06T20:45:04.681053+0000","last_clean":"2023-11-06T20:45:04.681053+0000","last_became_active":"2023-11-06T20:45:04.208313+0000","last_became_peered":"2023-11-06T20:45:04.208313+0000","last_unstale":"2023-11-06T20:45:04.681053+0000","last_undegraded":"2023-11-06T20:45:04.681053+0000","last_fullsized":"2023-11-06T20:45:04.681053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178850+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680469+0000","last_change":"2023-11-06T20:45:04.208028+0000","last_active":"2023-11-06T20:45:04.680469+0000","last_peered":"2023-11-06T20:45:04.680469+0000","last_clean":"2023-11-06T20:45:04.680469+0000","last_became_active":"2023-11-06T20:45:04.207152+0000","last_became_peered":"2023-11-06T20:45:04.207152+0000","last_unstale":"2023-11-06T20:45:04.680469+0000","last_undegraded":"2023-11-06T20:45:04.680469+0000","last_fullsized":"2023-11-06T20:45:04.680469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714410+0000","last_change":"2023-11-06T20:45:04.710247+0000","last_active":"2023-11-06T20:45:04.714410+0000","last_peered":"2023-11-06T20:45:04.714410+0000","last_clean":"2023-11-06T20:45:04.714410+0000","last_became_active":"2023-11-06T20:45:04.709893+0000","last_became_peered":"2023-11-06T20:45:04.709893+0000","last_unstale":"2023-11-06T20:45:04.714410+0000","last_undegraded":"2023-11-06T20:45:04.714410+0000","last_fullsized":"2023-11-06T20:45:04.714410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.176372+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175234+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173525+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700044+0000","last_change":"2023-11-06T20:45:04.693933+0000","last_active":"2023-11-06T20:45:04.700044+0000","last_peered":"2023-11-06T20:45:04.700044+0000","last_clean":"2023-11-06T20:45:04.700044+0000","last_became_active":"2023-11-06T20:45:04.693204+0000","last_became_peered":"2023-11-06T20:45:04.693204+0000","last_unstale":"2023-11-06T20:45:04.700044+0000","last_undegraded":"2023-11-06T20:45:04.700044+0000","last_fullsized":"2023-11-06T20:45:04.700044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681743+0000","last_change":"2023-11-06T20:45:04.213724+0000","last_active":"2023-11-06T20:45:04.681743+0000","last_peered":"2023-11-06T20:45:04.681743+0000","last_clean":"2023-11-06T20:45:04.681743+0000","last_became_active":"2023-11-06T20:45:04.211299+0000","last_became_peered":"2023-11-06T20:45:04.211299+0000","last_unstale":"2023-11-06T20:45:04.681743+0000","last_undegraded":"2023-11-06T20:45:04.681743+0000","last_fullsized":"2023-11-06T20:45:04.681743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179941+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703797+0000","last_change":"2023-11-06T20:45:04.701216+0000","last_active":"2023-11-06T20:45:04.703797+0000","last_peered":"2023-11-06T20:45:04.703797+0000","last_clean":"2023-11-06T20:45:04.703797+0000","last_became_active":"2023-11-06T20:45:04.700956+0000","last_became_peered":"2023-11-06T20:45:04.700956+0000","last_unstale":"2023-11-06T20:45:04.703797+0000","last_undegraded":"2023-11-06T20:45:04.703797+0000","last_fullsized":"2023-11-06T20:45:04.703797+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.180325+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680966+0000","last_change":"2023-11-06T20:45:04.205274+0000","last_active":"2023-11-06T20:45:04.680966+0000","last_peered":"2023-11-06T20:45:04.680966+0000","last_clean":"2023-11-06T20:45:04.680966+0000","last_became_active":"2023-11-06T20:45:04.204948+0000","last_became_peered":"2023-11-06T20:45:04.204948+0000","last_unstale":"2023-11-06T20:45:04.680966+0000","last_undegraded":"2023-11-06T20:45:04.680966+0000","last_fullsized":"2023-11-06T20:45:04.680966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.181516+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699237+0000","last_change":"2023-11-06T20:45:04.199975+0000","last_active":"2023-11-06T20:45:04.699237+0000","last_peered":"2023-11-06T20:45:04.699237+0000","last_clean":"2023-11-06T20:45:04.699237+0000","last_became_active":"2023-11-06T20:45:04.195195+0000","last_became_peered":"2023-11-06T20:45:04.195195+0000","last_unstale":"2023-11-06T20:45:04.699237+0000","last_undegraded":"2023-11-06T20:45:04.699237+0000","last_fullsized":"2023-11-06T20:45:04.699237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179244+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174491+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681802+0000","last_change":"2023-11-06T20:45:04.209004+0000","last_active":"2023-11-06T20:45:04.681802+0000","last_peered":"2023-11-06T20:45:04.681802+0000","last_clean":"2023-11-06T20:45:04.681802+0000","last_became_active":"2023-11-06T20:45:04.208295+0000","last_became_peered":"2023-11-06T20:45:04.208295+0000","last_unstale":"2023-11-06T20:45:04.681802+0000","last_undegraded":"2023-11-06T20:45:04.681802+0000","last_fullsized":"2023-11-06T20:45:04.681802+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680947+0000","last_change":"2023-11-06T20:45:04.200869+0000","last_active":"2023-11-06T20:45:04.680947+0000","last_peered":"2023-11-06T20:45:04.680947+0000","last_clean":"2023-11-06T20:45:04.680947+0000","last_became_active":"2023-11-06T20:45:04.200557+0000","last_became_peered":"2023-11-06T20:45:04.200557+0000","last_unstale":"2023-11-06T20:45:04.680947+0000","last_undegraded":"2023-11-06T20:45:04.680947+0000","last_fullsized":"2023-11-06T20:45:04.680947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696708+0000","last_change":"2023-11-06T20:45:04.205999+0000","last_active":"2023-11-06T20:45:04.696708+0000","last_peered":"2023-11-06T20:45:04.696708+0000","last_clean":"2023-11-06T20:45:04.696708+0000","last_became_active":"2023-11-06T20:45:04.205244+0000","last_became_peered":"2023-11-06T20:45:04.205244+0000","last_unstale":"2023-11-06T20:45:04.696708+0000","last_undegraded":"2023-11-06T20:45:04.696708+0000","last_fullsized":"2023-11-06T20:45:04.696708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174499+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696679+0000","last_change":"2023-11-06T20:45:04.194284+0000","last_active":"2023-11-06T20:45:04.696679+0000","last_peered":"2023-11-06T20:45:04.696679+0000","last_clean":"2023-11-06T20:45:04.696679+0000","last_became_active":"2023-11-06T20:45:04.193972+0000","last_became_peered":"2023-11-06T20:45:04.193972+0000","last_unstale":"2023-11-06T20:45:04.696679+0000","last_undegraded":"2023-11-06T20:45:04.696679+0000","last_fullsized":"2023-11-06T20:45:04.696679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175433+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700556+0000","last_change":"2023-11-06T20:45:04.699519+0000","last_active":"2023-11-06T20:45:04.700556+0000","last_peered":"2023-11-06T20:45:04.700556+0000","last_clean":"2023-11-06T20:45:04.700556+0000","last_became_active":"2023-11-06T20:45:04.699244+0000","last_became_peered":"2023-11-06T20:45:04.699244+0000","last_unstale":"2023-11-06T20:45:04.700556+0000","last_undegraded":"2023-11-06T20:45:04.700556+0000","last_fullsized":"2023-11-06T20:45:04.700556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182638+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182611+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698113+0000","last_change":"2023-11-06T20:45:04.694154+0000","last_active":"2023-11-06T20:45:04.698113+0000","last_peered":"2023-11-06T20:45:04.698113+0000","last_clean":"2023-11-06T20:45:04.698113+0000","last_became_active":"2023-11-06T20:45:04.693824+0000","last_became_peered":"2023-11-06T20:45:04.693824+0000","last_unstale":"2023-11-06T20:45:04.698113+0000","last_undegraded":"2023-11-06T20:45:04.698113+0000","last_fullsized":"2023-11-06T20:45:04.698113+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174583+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.175488+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179685+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.184627+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714056+0000","last_change":"2023-11-06T20:45:04.199156+0000","last_active":"2023-11-06T20:45:04.714056+0000","last_peered":"2023-11-06T20:45:04.714056+0000","last_clean":"2023-11-06T20:45:04.714056+0000","last_became_active":"2023-11-06T20:45:04.198858+0000","last_became_peered":"2023-11-06T20:45:04.198858+0000","last_unstale":"2023-11-06T20:45:04.714056+0000","last_undegraded":"2023-11-06T20:45:04.714056+0000","last_fullsized":"2023-11-06T20:45:04.714056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715339+0000","last_change":"2023-11-06T20:45:04.199860+0000","last_active":"2023-11-06T20:45:04.715339+0000","last_peered":"2023-11-06T20:45:04.715339+0000","last_clean":"2023-11-06T20:45:04.715339+0000","last_became_active":"2023-11-06T20:45:04.197409+0000","last_became_peered":"2023-11-06T20:45:04.197409+0000","last_unstale":"2023-11-06T20:45:04.715339+0000","last_undegraded":"2023-11-06T20:45:04.715339+0000","last_fullsized":"2023-11-06T20:45:04.715339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713635+0000","last_change":"2023-11-06T20:45:04.707405+0000","last_active":"2023-11-06T20:45:04.713635+0000","last_peered":"2023-11-06T20:45:04.713635+0000","last_clean":"2023-11-06T20:45:04.713635+0000","last_became_active":"2023-11-06T20:45:04.704832+0000","last_became_peered":"2023-11-06T20:45:04.704832+0000","last_unstale":"2023-11-06T20:45:04.713635+0000","last_undegraded":"2023-11-06T20:45:04.713635+0000","last_fullsized":"2023-11-06T20:45:04.713635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182164+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713102+0000","last_change":"2023-11-06T20:45:04.703423+0000","last_active":"2023-11-06T20:45:04.713102+0000","last_peered":"2023-11-06T20:45:04.713102+0000","last_clean":"2023-11-06T20:45:04.713102+0000","last_became_active":"2023-11-06T20:45:04.703108+0000","last_became_peered":"2023-11-06T20:45:04.703108+0000","last_unstale":"2023-11-06T20:45:04.713102+0000","last_undegraded":"2023-11-06T20:45:04.713102+0000","last_fullsized":"2023-11-06T20:45:04.713102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179821+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.174040+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.172938+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178482+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698151+0000","last_change":"2023-11-06T20:45:04.207106+0000","last_active":"2023-11-06T20:45:04.698151+0000","last_peered":"2023-11-06T20:45:04.698151+0000","last_clean":"2023-11-06T20:45:04.698151+0000","last_became_active":"2023-11-06T20:45:04.206814+0000","last_became_peered":"2023-11-06T20:45:04.206814+0000","last_unstale":"2023-11-06T20:45:04.698151+0000","last_undegraded":"2023-11-06T20:45:04.698151+0000","last_fullsized":"2023-11-06T20:45:04.698151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.178510+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173953+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:57.427499+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:44:57.427499+0000","last_peered":"2023-11-06T20:44:57.427499+0000","last_clean":"2023-11-06T20:44:57.427499+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:44:57.427499+0000","last_undegraded":"2023-11-06T20:44:57.427499+0000","last_fullsized":"2023-11-06T20:44:57.427499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.177557+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046724+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:44:59.046724+0000","last_peered":"2023-11-06T20:44:59.046724+0000","last_clean":"2023-11-06T20:44:59.046724+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:44:59.046724+0000","last_undegraded":"2023-11-06T20:44:59.046724+0000","last_fullsized":"2023-11-06T20:44:59.046724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.176976+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.047320+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:44:59.047320+0000","last_peered":"2023-11-06T20:44:59.047320+0000","last_clean":"2023-11-06T20:44:59.047320+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:44:59.047320+0000","last_undegraded":"2023-11-06T20:44:59.047320+0000","last_fullsized":"2023-11-06T20:44:59.047320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179510+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046727+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:44:59.046727+0000","last_peered":"2023-11-06T20:44:59.046727+0000","last_clean":"2023-11-06T20:44:59.046727+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:44:59.046727+0000","last_undegraded":"2023-11-06T20:44:59.046727+0000","last_fullsized":"2023-11-06T20:44:59.046727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696761+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:45:04.696761+0000","last_peered":"2023-11-06T20:45:04.696761+0000","last_clean":"2023-11-06T20:45:04.696761+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:45:04.696761+0000","last_undegraded":"2023-11-06T20:45:04.696761+0000","last_fullsized":"2023-11-06T20:45:04.696761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.182200+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046740+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:44:59.046740+0000","last_peered":"2023-11-06T20:44:59.046740+0000","last_clean":"2023-11-06T20:44:59.046740+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:44:59.046740+0000","last_undegraded":"2023-11-06T20:44:59.046740+0000","last_fullsized":"2023-11-06T20:44:59.046740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.179915+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046755+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:44:59.046755+0000","last_peered":"2023-11-06T20:44:59.046755+0000","last_clean":"2023-11-06T20:44:59.046755+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:44:59.046755+0000","last_undegraded":"2023-11-06T20:44:59.046755+0000","last_fullsized":"2023-11-06T20:44:59.046755+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.184211+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.047396+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:44:59.047396+0000","last_peered":"2023-11-06T20:44:59.047396+0000","last_clean":"2023-11-06T20:44:59.047396+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:44:59.047396+0000","last_undegraded":"2023-11-06T20:44:59.047396+0000","last_fullsized":"2023-11-06T20:44:59.047396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:45:04.168407+0000","last_change":"2023-11-06T20:45:04.168407+0000","last_active":"2023-11-06T20:45:04.168407+0000","last_peered":"2023-11-06T20:45:04.168407+0000","last_clean":"2023-11-06T20:45:04.168407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:04.168407+0000","last_undegraded":"2023-11-06T20:45:04.168407+0000","last_fullsized":"2023-11-06T20:45:04.168407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-06T20:45:03.162825+0000","last_change":"2023-11-06T20:45:03.173926+0000","last_active":"2023-11-06T20:45:03.162825+0000","last_peered":"2023-11-06T20:45:03.162825+0000","last_clean":"2023-11-06T20:45:03.162825+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:45:03.162825+0000","last_undegraded":"2023-11-06T20:45:03.162825+0000","last_fullsized":"2023-11-06T20:45:03.162825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-06T20:44:59.046729+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:44:59.046729+0000","last_peered":"2023-11-06T20:44:59.046729+0000","last_clean":"2023-11-06T20:44:59.046729+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:44:59.046729+0000","last_undegraded":"2023-11-06T20:44:59.046729+0000","last_fullsized":"2023-11-06T20:44:59.046729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":2},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":1468,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.1749999999999998,"5min":5.1749999999999998,"15min":5.1749999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.624}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65600000000000003}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.202,"5min":5.202,"15min":5.202},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.77200000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72999999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.75700000000000001}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46700000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.88700000000000001}]},{"osd":2,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.82999999999999996}]},{"osd":3,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.78300000000000003}]},{"osd":7,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.749}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":409600,"data_stored":167482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.42499999999999999}]},{"osd":2,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.439}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.38200000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.60499999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297436,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074404,"statfs":{"total":96636764160,"available":96332189696,"internally_reserved":0,"allocated":348160,"data_stored":139715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.51000000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.5}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.746},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.73299999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.70699999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":409600,"data_stored":167696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.38100000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.69299999999999995}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297436,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074404,"statfs":{"total":96636764160,"available":96332189696,"internally_reserved":0,"allocated":348160,"data_stored":139715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.38},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.44}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.53500000000000003}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64700000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.621}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.76100000000000001}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.86499999999999999}]},{"osd":1,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.93200000000000005}]},{"osd":2,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.95699999999999996}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.748},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.76000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297436,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074404,"statfs":{"total":96636764160,"available":96332189696,"internally_reserved":0,"allocated":348160,"data_stored":139715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66200000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68000000000000005}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.745}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.69899999999999995}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.70799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:45:10.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2023-11-06T20:45:10.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2023-11-06T20:45:10.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-11-06T20:45:10.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.173152+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T20:45:10.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2023-11-06T20:45:10.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2023-11-06T20:45:10.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-11-06T20:45:10.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.175110+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2023-11-06T20:45:10.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-11-06T20:45:10.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.174673+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T20:45:10.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-11-06T20:45:10.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.176517+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T20:45:10.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2023-11-06T20:45:10.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2023-11-06T20:45:10.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2023-11-06T20:45:10.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2023-11-06T20:45:10.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-11-06T20:45:10.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.175778+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-11-06T20:45:10.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.182188+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T20:45:10.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2023-11-06T20:45:10.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-11-06T20:45:10.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.178534+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T20:45:10.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2023-11-06T20:45:10.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-11-06T20:45:10.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.174676+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-11-06T20:45:10.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.179850+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T20:45:10.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-11-06T20:45:10.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.179400+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T20:45:10.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2023-11-06T20:45:10.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-11-06T20:45:10.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.182863+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T20:45:10.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2023-11-06T20:45:10.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-11-06T20:45:10.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.178154+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T20:45:10.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2023-11-06T20:45:10.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-11-06T20:45:10.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.180329+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T20:45:10.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2023-11-06T20:45:10.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-11-06T20:45:10.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.180016+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T20:45:10.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2023-11-06T20:45:10.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-11-06T20:45:10.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.180343+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T20:45:10.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2023-11-06T20:45:10.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2023-11-06T20:45:10.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-11-06T20:45:10.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.178850+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T20:45:10.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2023-11-06T20:45:10.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-11-06T20:45:10.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.176372+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2023-11-06T20:45:10.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-11-06T20:45:10.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.175234+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2023-11-06T20:45:10.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-11-06T20:45:10.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.173525+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T20:45:10.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2023-11-06T20:45:10.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-11-06T20:45:10.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.179941+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T20:45:10.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-11-06T20:45:10.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.180325+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T20:45:10.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2023-11-06T20:45:10.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2023-11-06T20:45:10.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-11-06T20:45:10.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.181516+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T20:45:10.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-11-06T20:45:10.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-11-06T20:45:10.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-11-06T20:45:10.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.179244+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T20:45:10.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-11-06T20:45:10.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-11-06T20:45:10.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.174491+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-11-06T20:45:10.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-11-06T20:45:10.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-11-06T20:45:10.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-11-06T20:45:10.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-11-06T20:45:10.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.174499+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-11-06T20:45:10.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-11-06T20:45:10.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-11-06T20:45:10.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.175433+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-11-06T20:45:10.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.182638+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T20:45:10.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-11-06T20:45:10.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-11-06T20:45:10.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.182611+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T20:45:10.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-11-06T20:45:10.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.174583+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-11-06T20:45:10.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-11-06T20:45:10.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.175488+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-11-06T20:45:10.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-11-06T20:45:10.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.179685+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T20:45:10.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-11-06T20:45:10.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-11-06T20:45:10.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.184627+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T20:45:10.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-11-06T20:45:10.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-11-06T20:45:10.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-11-06T20:45:10.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.182164+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T20:45:10.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-11-06T20:45:10.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.179821+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T20:45:10.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-11-06T20:45:10.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-11-06T20:45:10.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.174040+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T20:45:10.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-11-06T20:45:10.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-11-06T20:45:10.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.172938+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T20:45:10.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-11-06T20:45:10.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-11-06T20:45:10.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.178482+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T20:45:10.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-11-06T20:45:10.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-11-06T20:45:10.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-11-06T20:45:10.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.178510+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T20:45:10.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-11-06T20:45:10.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-11-06T20:45:10.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.173953+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-11-06T20:45:10.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-11-06T20:45:10.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.177557+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T20:45:10.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-11-06T20:45:10.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-11-06T20:45:10.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.176976+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T20:45:10.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-11-06T20:45:10.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-11-06T20:45:10.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.179510+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T20:45:10.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-11-06T20:45:10.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-11-06T20:45:10.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.182200+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T20:45:10.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-11-06T20:45:10.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-11-06T20:45:10.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.179915+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T20:45:10.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-11-06T20:45:10.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-11-06T20:45:10.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.184211+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T20:45:10.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-11-06T20:45:10.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T20:45:04.168407+0000', 'last_change': '2023-11-06T20:45:04.168407+0000', 'last_active': '2023-11-06T20:45:04.168407+0000', 'last_peered': '2023-11-06T20:45:04.168407+0000', 'last_clean': '2023-11-06T20:45:04.168407+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:04.168407+0000', 'last_undegraded': '2023-11-06T20:45:04.168407+0000', 'last_fullsized': '2023-11-06T20:45:04.168407+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:04.168407+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T20:45:10.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-11-06T20:45:10.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-06T20:45:03.162825+0000', 'last_change': '2023-11-06T20:45:03.173926+0000', 'last_active': '2023-11-06T20:45:03.162825+0000', 'last_peered': '2023-11-06T20:45:03.162825+0000', 'last_clean': '2023-11-06T20:45:03.162825+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T20:45:03.162825+0000', 'last_undegraded': '2023-11-06T20:45:03.162825+0000', 'last_fullsized': '2023-11-06T20:45:03.162825+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'last_clean_scrub_stamp': '2023-11-06T20:45:03.162825+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T20:45:10.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T20:45:10.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.304+0000 7ff0a5dff700 1 Processor -- start 2023-11-06T20:45:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.304+0000 7ff0a5dff700 1 -- start start 2023-11-06T20:45:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.306+0000 7ff0a5dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff0a00c1840 0x7ff0a00a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.306+0000 7ff0a5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00aa2d0 0x7ff0a00c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.306+0000 7ff0a5dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a01554a0 con 0x7ff0a00c6240 2023-11-06T20:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.306+0000 7ff0a5dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a0154960 con 0x7ff0a00c1840 2023-11-06T20:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.306+0000 7ff0a5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0155930 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.306+0000 7ff09effd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00aa2d0 0x7ff0a00c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.306+0000 7ff09f7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff0a00c1840 0x7ff0a00a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.306+0000 7ff09ffff700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7ff0a00c6240 0x7ff0a0157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:46040/0 (socket says 172.21.15.33:46040) 2023-11-06T20:45:10.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.306+0000 7ff09ffff700 1 -- 172.21.15.33:0/2391004707 learned_addr learned my addr 172.21.15.33:0/2391004707 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:10.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09e7fc700 1 -- 172.21.15.33:0/2391004707 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907387573 0 0) 0x7ff0a01554a0 con 0x7ff0a00c6240 2023-11-06T20:45:10.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09e7fc700 1 -- 172.21.15.33:0/2391004707 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0880033f0 con 0x7ff0a00c6240 2023-11-06T20:45:10.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09f7fe700 1 -- 172.21.15.33:0/2391004707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00aa2d0 msgr2=0x7ff0a00c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:10.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09f7fe700 1 --2- 172.21.15.33:0/2391004707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00aa2d0 0x7ff0a00c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09f7fe700 1 -- 172.21.15.33:0/2391004707 >> v1:172.21.15.69:6789/0 conn(0x7ff0a00c6240 legacy=0x7ff0a0157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09f7fe700 1 -- 172.21.15.33:0/2391004707 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00c66b0 con 0x7ff0a00c1840 2023-11-06T20:45:10.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09effd700 1 --2- 172.21.15.33:0/2391004707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00aa2d0 0x7ff0a00c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09f7fe700 1 --2- 172.21.15.33:0/2391004707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0a00c1840 0x7ff0a00a9de0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7ff090009760 tx=0x7ff090009060).ready entity=mon.1 client_cookie=9efaeaf3dac53526 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09e7fc700 1 -- 172.21.15.33:0/2391004707 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09000d4a0 con 0x7ff0a00c1840 2023-11-06T20:45:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09e7fc700 1 -- 172.21.15.33:0/2391004707 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09000d600 con 0x7ff0a00c1840 2023-11-06T20:45:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.307+0000 7ff09e7fc700 1 -- 172.21.15.33:0/2391004707 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09000d980 con 0x7ff0a00c1840 2023-11-06T20:45:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.308+0000 7ff0a5dff700 1 -- 172.21.15.33:0/2391004707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0a00c1840 msgr2=0x7ff0a00a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.308+0000 7ff0a5dff700 1 --2- 172.21.15.33:0/2391004707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0a00c1840 0x7ff0a00a9de0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7ff090009760 tx=0x7ff090009060).stop 2023-11-06T20:45:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.308+0000 7ff0a5dff700 1 -- 172.21.15.33:0/2391004707 shutdown_connections 2023-11-06T20:45:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.308+0000 7ff0a5dff700 1 --2- 172.21.15.33:0/2391004707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0a00c1840 0x7ff0a00a9de0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.308+0000 7ff0a5dff700 1 --2- 172.21.15.33:0/2391004707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00aa2d0 0x7ff0a00c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.308+0000 7ff0a5dff700 1 -- 172.21.15.33:0/2391004707 >> 172.21.15.33:0/2391004707 conn(0x7ff0a005ec80 msgr2=0x7ff0a00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.308+0000 7ff0a5dff700 1 -- 172.21.15.33:0/2391004707 shutdown_connections 2023-11-06T20:45:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.308+0000 7ff0a5dff700 1 -- 172.21.15.33:0/2391004707 wait complete. 2023-11-06T20:45:10.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.309+0000 7ff0a5dff700 1 Processor -- start 2023-11-06T20:45:10.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.310+0000 7ff0a5dff700 1 -- start start 2023-11-06T20:45:10.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.310+0000 7ff0a5dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a00aa2d0 0x7ff0a01439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.310+0000 7ff0a5dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0a00c1840 0x7ff0a014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.310+0000 7ff0a5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00c6240 0x7ff0a00ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.310+0000 7ff0a5dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0165300 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.310+0000 7ff0a5dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a00b0b50 con 0x7ff0a00c1840 2023-11-06T20:45:10.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.310+0000 7ff0a5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a01646f0 con 0x7ff0a00c6240 2023-11-06T20:45:10.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a00aa2d0 0x7ff0a01439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a00aa2d0 0x7ff0a01439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36460/0 (socket says 172.21.15.33:36460) 2023-11-06T20:45:10.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09f7fe700 1 -- 172.21.15.33:0/958900405 learned_addr learned my addr 172.21.15.33:0/958900405 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:10.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09ffff700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00c6240 0x7ff0a00ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09effd700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0a00c1840 0x7ff0a014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09f7fe700 1 -- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00c6240 msgr2=0x7ff0a00ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09f7fe700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00c6240 0x7ff0a00ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09f7fe700 1 -- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0a00c1840 msgr2=0x7ff0a014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09f7fe700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0a00c1840 0x7ff0a014e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09f7fe700 1 -- 172.21.15.33:0/958900405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940092a0 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09effd700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0a00c1840 0x7ff0a014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09ffff700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00c6240 0x7ff0a00ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:10.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.311+0000 7ff09f7fe700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a00aa2d0 0x7ff0a01439d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ff090009d50 tx=0x7ff090016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:10.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.312+0000 7ff0877fe700 1 -- 172.21.15.33:0/958900405 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090000c90 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.312+0000 7ff0877fe700 1 -- 172.21.15.33:0/958900405 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09000d770 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.312+0000 7ff0877fe700 1 -- 172.21.15.33:0/958900405 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09001d640 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.312+0000 7ff0a5dff700 1 -- 172.21.15.33:0/958900405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0900092a0 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.312+0000 7ff0a5dff700 1 -- 172.21.15.33:0/958900405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a0164cb0 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.313+0000 7ff0a5dff700 1 -- 172.21.15.33:0/958900405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a006c6c0 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.314+0000 7ff0877fe700 1 -- 172.21.15.33:0/958900405 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff09001da70 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.315+0000 7ff0877fe700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff08c05f120 0x7ff08c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.315+0000 7ff0877fe700 1 -- 172.21.15.33:0/958900405 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff09007b9b0 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.315+0000 7ff09effd700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff08c05f120 0x7ff08c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.316+0000 7ff09effd700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff08c05f120 0x7ff08c0615e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff094009270 tx=0x7ff094008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.319+0000 7ff0877fe700 1 -- 172.21.15.33:0/958900405 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff09004cc80 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.568+0000 7ff0a5dff700 1 -- 172.21.15.33:0/958900405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff0a00c5640 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.568+0000 7ff0877fe700 1 -- 172.21.15.33:0/958900405 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1659 (secure 0 0 0) 0x7ff0900502a0 con 0x7ff0a00aa2d0 2023-11-06T20:45:10.568 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:10.568 INFO:teuthology.orchestra.run.smithi033.stdout:{"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1699303437,"num_in_osds":8,"osd_in_since":1699303430,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":52},{"state_name":"creating+peering","count":46},{"state_name":"active+clean","count":39}],"num_pgs":137,"num_pools":4,"num_objects":7,"data_bytes":1468,"bytes_used":2436755456,"bytes_avail":770657357824,"bytes_total":773094113280,"unknown_pgs_ratio":0.37956205010414124,"inactive_pgs_ratio":0.33576643466949463},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4926}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-11-06T20:45:00.821584+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T20:45:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 -- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff08c05f120 msgr2=0x7ff08c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff08c05f120 0x7ff08c0615e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff094009270 tx=0x7ff094008040).stop 2023-11-06T20:45:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 -- 172.21.15.33:0/958900405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a00aa2d0 msgr2=0x7ff0a01439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a00aa2d0 0x7ff0a01439d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ff090009d50 tx=0x7ff090016000).stop 2023-11-06T20:45:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 -- 172.21.15.33:0/958900405 shutdown_connections 2023-11-06T20:45:10.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff08c05f120 0x7ff08c0615e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0a00c6240 0x7ff0a00ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a00aa2d0 0x7ff0a01439d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 --2- 172.21.15.33:0/958900405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0a00c1840 0x7ff0a014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 -- 172.21.15.33:0/958900405 >> 172.21.15.33:0/958900405 conn(0x7ff0a005ec80 msgr2=0x7ff0a01503f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:10.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 -- 172.21.15.33:0/958900405 shutdown_connections 2023-11-06T20:45:10.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.570+0000 7ff0a5dff700 1 -- 172.21.15.33:0/958900405 wait complete. 2023-11-06T20:45:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T20:45:10.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:45:10.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.783+0000 7f3ffebff700 1 Processor -- start 2023-11-06T20:45:10.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.783+0000 7f3ffebff700 1 -- start start 2023-11-06T20:45:10.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.784+0000 7f3ffebff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3ff80c1840 0x7f3ff80a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.784+0000 7f3ffebff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3ff80aa2d0 0x7f3ff80c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.784+0000 7f3ffebff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff81554a0 con 0x7f3ff80c6240 2023-11-06T20:45:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.784+0000 7f3ffebff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff8154960 con 0x7f3ff80aa2d0 2023-11-06T20:45:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.784+0000 7f3ffebff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff8155930 con 0x7f3ff80c1840 2023-11-06T20:45:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ff7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3ff80aa2d0 0x7f3ff80c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ffd19c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3ff80c6240 0x7f3ff8157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57994/0 (socket says 172.21.15.33:57994) 2023-11-06T20:45:10.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ffd19c700 1 -- 172.21.15.33:0/1361471872 learned_addr learned my addr 172.21.15.33:0/1361471872 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:10.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ff7fff700 1 --2- 172.21.15.33:0/1361471872 >> v2:172.21.15.33:3300/0 conn(0x7f3ff80aa2d0 0x7f3ff80c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36466/0 (socket says 172.21.15.33:36466) 2023-11-06T20:45:10.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ffc99b700 1 --2- 172.21.15.33:0/1361471872 >> v2:172.21.15.69:3300/0 conn(0x7f3ff80c1840 0x7f3ff80a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ff7fff700 1 -- 172.21.15.33:0/1361471872 >> v2:172.21.15.69:3300/0 conn(0x7f3ff80c1840 msgr2=0x7f3ff80a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:10.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ff7fff700 1 --2- 172.21.15.33:0/1361471872 >> v2:172.21.15.69:3300/0 conn(0x7f3ff80c1840 0x7f3ff80a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ff7fff700 1 -- 172.21.15.33:0/1361471872 >> v1:172.21.15.33:6789/0 conn(0x7f3ff80c6240 legacy=0x7f3ff8157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ff7fff700 1 -- 172.21.15.33:0/1361471872 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff8154b70 con 0x7f3ff80aa2d0 2023-11-06T20:45:10.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ff77fe700 1 -- 172.21.15.33:0/1361471872 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167360399 0 0) 0x7f3ff81554a0 con 0x7f3ff80c6240 2023-11-06T20:45:10.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.785+0000 7f3ff7fff700 1 --2- 172.21.15.33:0/1361471872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80aa2d0 0x7f3ff80c5d50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f3ff80840b0 tx=0x7f3fe4009060).ready entity=mon.0 client_cookie=b8a98fb8a5920116 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:10.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ff77fe700 1 -- 172.21.15.33:0/1361471872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400f510 con 0x7f3ff80aa2d0 2023-11-06T20:45:10.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ff77fe700 1 -- 172.21.15.33:0/1361471872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe400f670 con 0x7f3ff80aa2d0 2023-11-06T20:45:10.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ff77fe700 1 -- 172.21.15.33:0/1361471872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400f9f0 con 0x7f3ff80aa2d0 2023-11-06T20:45:10.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ffebff700 1 -- 172.21.15.33:0/1361471872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80aa2d0 msgr2=0x7f3ff80c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ffebff700 1 --2- 172.21.15.33:0/1361471872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80aa2d0 0x7f3ff80c5d50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f3ff80840b0 tx=0x7f3fe4009060).stop 2023-11-06T20:45:10.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ffebff700 1 -- 172.21.15.33:0/1361471872 shutdown_connections 2023-11-06T20:45:10.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ffebff700 1 --2- 172.21.15.33:0/1361471872 >> v2:172.21.15.69:3300/0 conn(0x7f3ff80c1840 0x7f3ff80a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ffebff700 1 --2- 172.21.15.33:0/1361471872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80aa2d0 0x7f3ff80c5d50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ffebff700 1 -- 172.21.15.33:0/1361471872 >> 172.21.15.33:0/1361471872 conn(0x7f3ff805ec80 msgr2=0x7f3ff80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:10.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ffebff700 1 -- 172.21.15.33:0/1361471872 shutdown_connections 2023-11-06T20:45:10.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.786+0000 7f3ffebff700 1 -- 172.21.15.33:0/1361471872 wait complete. 2023-11-06T20:45:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.788+0000 7f3ffebff700 1 Processor -- start 2023-11-06T20:45:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.788+0000 7f3ffebff700 1 -- start start 2023-11-06T20:45:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ffebff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff80aa2d0 0x7f3ff80c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ffebff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80c1840 0x7f3ff814c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ffebff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ff80c6240 0x7f3ff814c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ffebff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff814cc70 con 0x7f3ff80c1840 2023-11-06T20:45:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ffebff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff814cf50 con 0x7f3ff80aa2d0 2023-11-06T20:45:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ffebff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff814d230 con 0x7f3ff80c6240 2023-11-06T20:45:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ff7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80c1840 0x7f3ff814c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ff7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80c1840 0x7f3ff814c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36474/0 (socket says 172.21.15.33:36474) 2023-11-06T20:45:10.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ff7fff700 1 -- 172.21.15.33:0/1602945619 learned_addr learned my addr 172.21.15.33:0/1602945619 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:10.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ffc99b700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff80aa2d0 0x7f3ff80c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.789+0000 7f3ffd19c700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ff80c6240 0x7f3ff814c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ff7fff700 1 -- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ff80c6240 msgr2=0x7f3ff814c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ff7fff700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ff80c6240 0x7f3ff814c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ff7fff700 1 -- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff80aa2d0 msgr2=0x7f3ff80c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ff7fff700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff80aa2d0 0x7f3ff80c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ff7fff700 1 -- 172.21.15.33:0/1602945619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe40092a0 con 0x7f3ff80c1840 2023-11-06T20:45:10.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ff7fff700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80c1840 0x7f3ff814c120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f3ff806bc90 tx=0x7f3fe4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:10.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ff4ff9700 1 -- 172.21.15.33:0/1602945619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400f7e0 con 0x7f3ff80c1840 2023-11-06T20:45:10.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ff4ff9700 1 -- 172.21.15.33:0/1602945619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe400f940 con 0x7f3ff80c1840 2023-11-06T20:45:10.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ffebff700 1 -- 172.21.15.33:0/1602945619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff814d510 con 0x7f3ff80c1840 2023-11-06T20:45:10.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ff4ff9700 1 -- 172.21.15.33:0/1602945619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe401b690 con 0x7f3ff80c1840 2023-11-06T20:45:10.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.790+0000 7f3ffebff700 1 -- 172.21.15.33:0/1602945619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff814a9f0 con 0x7f3ff80c1840 2023-11-06T20:45:10.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.791+0000 7f3ffebff700 1 -- 172.21.15.33:0/1602945619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff80aa110 con 0x7f3ff80c1840 2023-11-06T20:45:10.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.792+0000 7f3ff4ff9700 1 -- 172.21.15.33:0/1602945619 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3fe401ba00 con 0x7f3ff80c1840 2023-11-06T20:45:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.793+0000 7f3ff4ff9700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fdc05f150 0x7f3fdc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.793+0000 7f3ff4ff9700 1 -- 172.21.15.33:0/1602945619 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3fe407b7a0 con 0x7f3ff80c1840 2023-11-06T20:45:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.794+0000 7f3ffc99b700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fdc05f150 0x7f3fdc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.794+0000 7f3ffc99b700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fdc05f150 0x7f3fdc061610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f3fec009c30 tx=0x7f3fec009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.797+0000 7f3ff4ff9700 1 -- 172.21.15.33:0/1602945619 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3fe404ca60 con 0x7f3ff80c1840 2023-11-06T20:45:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.973+0000 7f3ffebff700 1 -- 172.21.15.33:0/1602945619 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ff806c6c0 con 0x7f3fdc05f150 2023-11-06T20:45:10.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.984+0000 7f3ff4ff9700 1 -- 172.21.15.33:0/1602945619 <== mgr.4103 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+360913 (secure 0 0 0) 0x7f3ff806c6c0 con 0x7f3fdc05f150 2023-11-06T20:45:10.988 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.988+0000 7f3ffebff700 1 -- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fdc05f150 msgr2=0x7f3fdc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.988+0000 7f3ffebff700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fdc05f150 0x7f3fdc061610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f3fec009c30 tx=0x7f3fec009060).stop 2023-11-06T20:45:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.989+0000 7f3ffebff700 1 -- 172.21.15.33:0/1602945619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80c1840 msgr2=0x7f3ff814c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:10.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.989+0000 7f3ffebff700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80c1840 0x7f3ff814c120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f3ff806bc90 tx=0x7f3fe4017000).stop 2023-11-06T20:45:10.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.989+0000 7f3ffebff700 1 -- 172.21.15.33:0/1602945619 shutdown_connections 2023-11-06T20:45:10.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.989+0000 7f3ffebff700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3fdc05f150 0x7f3fdc061610 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.989+0000 7f3ffebff700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ff80c6240 0x7f3ff814c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.989+0000 7f3ffebff700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff80aa2d0 0x7f3ff80c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.989+0000 7f3ffebff700 1 --2- 172.21.15.33:0/1602945619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff80c1840 0x7f3ff814c120 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:10.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.989+0000 7f3ffebff700 1 -- 172.21.15.33:0/1602945619 >> 172.21.15.33:0/1602945619 conn(0x7f3ff805ec80 msgr2=0x7f3ff80cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:10.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.989+0000 7f3ffebff700 1 -- 172.21.15.33:0/1602945619 shutdown_connections 2023-11-06T20:45:10.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:10.989+0000 7f3ffebff700 1 -- 172.21.15.33:0/1602945619 wait complete. 2023-11-06T20:45:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:45:11.005 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-11-06T20:45:10.823879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":218,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379936,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752594784,"statfs":{"total":773094113280,"available":770657058816,"internally_reserved":0,"allocated":3244032,"data_stored":1336596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4023,"num_objects":-7,"num_object_clones":0,"num_object_copies":-14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-26,"num_write_kb":-14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.009765"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700327+0000","last_change":"2023-11-06T20:45:04.698159+0000","last_active":"2023-11-06T20:45:04.700327+0000","last_peered":"2023-11-06T20:45:04.700327+0000","last_clean":"2023-11-06T20:45:04.700327+0000","last_became_active":"2023-11-06T20:45:04.695684+0000","last_became_peered":"2023-11-06T20:45:04.695684+0000","last_unstale":"2023-11-06T20:45:04.700327+0000","last_undegraded":"2023-11-06T20:45:04.700327+0000","last_fullsized":"2023-11-06T20:45:04.700327+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3f","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712002+0000","last_change":"2023-11-06T20:45:04.213243+0000","last_active":"2023-11-06T20:45:04.712002+0000","last_peered":"2023-11-06T20:45:04.712002+0000","last_clean":"2023-11-06T20:45:04.712002+0000","last_became_active":"2023-11-06T20:45:04.212975+0000","last_became_peered":"2023-11-06T20:45:04.212975+0000","last_unstale":"2023-11-06T20:45:04.712002+0000","last_undegraded":"2023-11-06T20:45:04.712002+0000","last_fullsized":"2023-11-06T20:45:04.712002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.705009+0000","last_change":"2023-11-06T20:45:04.700122+0000","last_active":"2023-11-06T20:45:04.705009+0000","last_peered":"2023-11-06T20:45:04.705009+0000","last_clean":"2023-11-06T20:45:04.705009+0000","last_became_active":"2023-11-06T20:45:04.699789+0000","last_became_peered":"2023-11-06T20:45:04.699789+0000","last_unstale":"2023-11-06T20:45:04.705009+0000","last_undegraded":"2023-11-06T20:45:04.705009+0000","last_fullsized":"2023-11-06T20:45:04.705009+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.683303+0000","last_change":"2023-11-06T20:45:04.205007+0000","last_active":"2023-11-06T20:45:04.683303+0000","last_peered":"2023-11-06T20:45:04.683303+0000","last_clean":"2023-11-06T20:45:04.683303+0000","last_became_active":"2023-11-06T20:45:04.204674+0000","last_became_peered":"2023-11-06T20:45:04.204674+0000","last_unstale":"2023-11-06T20:45:04.683303+0000","last_undegraded":"2023-11-06T20:45:04.683303+0000","last_fullsized":"2023-11-06T20:45:04.683303+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.688667+0000","last_change":"2023-11-06T20:45:04.682887+0000","last_active":"2023-11-06T20:45:04.688667+0000","last_peered":"2023-11-06T20:45:04.688667+0000","last_clean":"2023-11-06T20:45:04.688667+0000","last_became_active":"2023-11-06T20:45:04.682623+0000","last_became_peered":"2023-11-06T20:45:04.682623+0000","last_unstale":"2023-11-06T20:45:04.688667+0000","last_undegraded":"2023-11-06T20:45:04.688667+0000","last_fullsized":"2023-11-06T20:45:04.688667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699604+0000","last_change":"2023-11-06T20:45:04.196531+0000","last_active":"2023-11-06T20:45:04.699604+0000","last_peered":"2023-11-06T20:45:04.699604+0000","last_clean":"2023-11-06T20:45:04.699604+0000","last_became_active":"2023-11-06T20:45:04.194888+0000","last_became_peered":"2023-11-06T20:45:04.194888+0000","last_unstale":"2023-11-06T20:45:04.699604+0000","last_undegraded":"2023-11-06T20:45:04.699604+0000","last_fullsized":"2023-11-06T20:45:04.699604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.689481+0000","last_change":"2023-11-06T20:45:04.685895+0000","last_active":"2023-11-06T20:45:04.689481+0000","last_peered":"2023-11-06T20:45:04.689481+0000","last_clean":"2023-11-06T20:45:04.689481+0000","last_became_active":"2023-11-06T20:45:04.685580+0000","last_became_peered":"2023-11-06T20:45:04.685580+0000","last_unstale":"2023-11-06T20:45:04.689481+0000","last_undegraded":"2023-11-06T20:45:04.689481+0000","last_fullsized":"2023-11-06T20:45:04.689481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3c","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.706361+0000","last_change":"2023-11-06T20:45:04.203616+0000","last_active":"2023-11-06T20:45:04.706361+0000","last_peered":"2023-11-06T20:45:04.706361+0000","last_clean":"2023-11-06T20:45:04.706361+0000","last_became_active":"2023-11-06T20:45:04.202876+0000","last_became_peered":"2023-11-06T20:45:04.202876+0000","last_unstale":"2023-11-06T20:45:04.706361+0000","last_undegraded":"2023-11-06T20:45:04.706361+0000","last_fullsized":"2023-11-06T20:45:04.706361+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712228+0000","last_change":"2023-11-06T20:45:04.709013+0000","last_active":"2023-11-06T20:45:04.712228+0000","last_peered":"2023-11-06T20:45:04.712228+0000","last_clean":"2023-11-06T20:45:04.712228+0000","last_became_active":"2023-11-06T20:45:04.708683+0000","last_became_peered":"2023-11-06T20:45:04.708683+0000","last_unstale":"2023-11-06T20:45:04.712228+0000","last_undegraded":"2023-11-06T20:45:04.712228+0000","last_fullsized":"2023-11-06T20:45:04.712228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700181+0000","last_change":"2023-11-06T20:45:04.202634+0000","last_active":"2023-11-06T20:45:04.700181+0000","last_peered":"2023-11-06T20:45:04.700181+0000","last_clean":"2023-11-06T20:45:04.700181+0000","last_became_active":"2023-11-06T20:45:04.201207+0000","last_became_peered":"2023-11-06T20:45:04.201207+0000","last_unstale":"2023-11-06T20:45:04.700181+0000","last_undegraded":"2023-11-06T20:45:04.700181+0000","last_fullsized":"2023-11-06T20:45:04.700181+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703778+0000","last_change":"2023-11-06T20:45:04.699346+0000","last_active":"2023-11-06T20:45:04.703778+0000","last_peered":"2023-11-06T20:45:04.703778+0000","last_clean":"2023-11-06T20:45:04.703778+0000","last_became_active":"2023-11-06T20:45:04.698931+0000","last_became_peered":"2023-11-06T20:45:04.698931+0000","last_unstale":"2023-11-06T20:45:04.703778+0000","last_undegraded":"2023-11-06T20:45:04.703778+0000","last_fullsized":"2023-11-06T20:45:04.703778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3a","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.709968+0000","last_change":"2023-11-06T20:45:04.210337+0000","last_active":"2023-11-06T20:45:04.709968+0000","last_peered":"2023-11-06T20:45:04.709968+0000","last_clean":"2023-11-06T20:45:04.709968+0000","last_became_active":"2023-11-06T20:45:04.209950+0000","last_became_peered":"2023-11-06T20:45:04.209950+0000","last_unstale":"2023-11-06T20:45:04.709968+0000","last_undegraded":"2023-11-06T20:45:04.709968+0000","last_fullsized":"2023-11-06T20:45:04.709968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.691246+0000","last_change":"2023-11-06T20:45:04.688979+0000","last_active":"2023-11-06T20:45:04.691246+0000","last_peered":"2023-11-06T20:45:04.691246+0000","last_clean":"2023-11-06T20:45:04.691246+0000","last_became_active":"2023-11-06T20:45:04.688710+0000","last_became_peered":"2023-11-06T20:45:04.688710+0000","last_unstale":"2023-11-06T20:45:04.691246+0000","last_undegraded":"2023-11-06T20:45:04.691246+0000","last_fullsized":"2023-11-06T20:45:04.691246+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680925+0000","last_change":"2023-11-06T20:45:04.212407+0000","last_active":"2023-11-06T20:45:04.680925+0000","last_peered":"2023-11-06T20:45:04.680925+0000","last_clean":"2023-11-06T20:45:04.680925+0000","last_became_active":"2023-11-06T20:45:04.211888+0000","last_became_peered":"2023-11-06T20:45:04.211888+0000","last_unstale":"2023-11-06T20:45:04.680925+0000","last_undegraded":"2023-11-06T20:45:04.680925+0000","last_fullsized":"2023-11-06T20:45:04.680925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703585+0000","last_change":"2023-11-06T20:45:04.700775+0000","last_active":"2023-11-06T20:45:04.703585+0000","last_peered":"2023-11-06T20:45:04.703585+0000","last_clean":"2023-11-06T20:45:04.703585+0000","last_became_active":"2023-11-06T20:45:04.699051+0000","last_became_peered":"2023-11-06T20:45:04.699051+0000","last_unstale":"2023-11-06T20:45:04.703585+0000","last_undegraded":"2023-11-06T20:45:04.703585+0000","last_fullsized":"2023-11-06T20:45:04.703585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680573+0000","last_change":"2023-11-06T20:45:04.203081+0000","last_active":"2023-11-06T20:45:04.680573+0000","last_peered":"2023-11-06T20:45:04.680573+0000","last_clean":"2023-11-06T20:45:04.680573+0000","last_became_active":"2023-11-06T20:45:04.199530+0000","last_became_peered":"2023-11-06T20:45:04.199530+0000","last_unstale":"2023-11-06T20:45:04.680573+0000","last_undegraded":"2023-11-06T20:45:04.680573+0000","last_fullsized":"2023-11-06T20:45:04.680573+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690168+0000","last_change":"2023-11-06T20:45:04.686241+0000","last_active":"2023-11-06T20:45:04.690168+0000","last_peered":"2023-11-06T20:45:04.690168+0000","last_clean":"2023-11-06T20:45:04.690168+0000","last_became_active":"2023-11-06T20:45:04.685794+0000","last_became_peered":"2023-11-06T20:45:04.685794+0000","last_unstale":"2023-11-06T20:45:04.690168+0000","last_undegraded":"2023-11-06T20:45:04.690168+0000","last_fullsized":"2023-11-06T20:45:04.690168+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700837+0000","last_change":"2023-11-06T20:45:04.207865+0000","last_active":"2023-11-06T20:45:04.700837+0000","last_peered":"2023-11-06T20:45:04.700837+0000","last_clean":"2023-11-06T20:45:04.700837+0000","last_became_active":"2023-11-06T20:45:04.207584+0000","last_became_peered":"2023-11-06T20:45:04.207584+0000","last_unstale":"2023-11-06T20:45:04.700837+0000","last_undegraded":"2023-11-06T20:45:04.700837+0000","last_fullsized":"2023-11-06T20:45:04.700837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.711974+0000","last_change":"2023-11-06T20:45:04.707600+0000","last_active":"2023-11-06T20:45:04.711974+0000","last_peered":"2023-11-06T20:45:04.711974+0000","last_clean":"2023-11-06T20:45:04.711974+0000","last_became_active":"2023-11-06T20:45:04.707007+0000","last_became_peered":"2023-11-06T20:45:04.707007+0000","last_unstale":"2023-11-06T20:45:04.711974+0000","last_undegraded":"2023-11-06T20:45:04.711974+0000","last_fullsized":"2023-11-06T20:45:04.711974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.683774+0000","last_change":"2023-11-06T20:45:04.204211+0000","last_active":"2023-11-06T20:45:04.683774+0000","last_peered":"2023-11-06T20:45:04.683774+0000","last_clean":"2023-11-06T20:45:04.683774+0000","last_became_active":"2023-11-06T20:45:04.203955+0000","last_became_peered":"2023-11-06T20:45:04.203955+0000","last_unstale":"2023-11-06T20:45:04.683774+0000","last_undegraded":"2023-11-06T20:45:04.683774+0000","last_fullsized":"2023-11-06T20:45:04.683774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715752+0000","last_change":"2023-11-06T20:45:04.711145+0000","last_active":"2023-11-06T20:45:04.715752+0000","last_peered":"2023-11-06T20:45:04.715752+0000","last_clean":"2023-11-06T20:45:04.715752+0000","last_became_active":"2023-11-06T20:45:04.709234+0000","last_became_peered":"2023-11-06T20:45:04.709234+0000","last_unstale":"2023-11-06T20:45:04.715752+0000","last_undegraded":"2023-11-06T20:45:04.715752+0000","last_fullsized":"2023-11-06T20:45:04.715752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.678195+0000","last_change":"2023-11-06T20:45:04.203691+0000","last_active":"2023-11-06T20:45:04.678195+0000","last_peered":"2023-11-06T20:45:04.678195+0000","last_clean":"2023-11-06T20:45:04.678195+0000","last_became_active":"2023-11-06T20:45:04.203407+0000","last_became_peered":"2023-11-06T20:45:04.203407+0000","last_unstale":"2023-11-06T20:45:04.678195+0000","last_undegraded":"2023-11-06T20:45:04.678195+0000","last_fullsized":"2023-11-06T20:45:04.678195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.697860+0000","last_change":"2023-11-06T20:45:04.695397+0000","last_active":"2023-11-06T20:45:04.697860+0000","last_peered":"2023-11-06T20:45:04.697860+0000","last_clean":"2023-11-06T20:45:04.697860+0000","last_became_active":"2023-11-06T20:45:04.695135+0000","last_became_peered":"2023-11-06T20:45:04.695135+0000","last_unstale":"2023-11-06T20:45:04.697860+0000","last_undegraded":"2023-11-06T20:45:04.697860+0000","last_fullsized":"2023-11-06T20:45:04.697860+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715259+0000","last_change":"2023-11-06T20:45:04.206902+0000","last_active":"2023-11-06T20:45:04.715259+0000","last_peered":"2023-11-06T20:45:04.715259+0000","last_clean":"2023-11-06T20:45:04.715259+0000","last_became_active":"2023-11-06T20:45:04.206583+0000","last_became_peered":"2023-11-06T20:45:04.206583+0000","last_unstale":"2023-11-06T20:45:04.715259+0000","last_undegraded":"2023-11-06T20:45:04.715259+0000","last_fullsized":"2023-11-06T20:45:04.715259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712482+0000","last_change":"2023-11-06T20:45:04.707038+0000","last_active":"2023-11-06T20:45:04.712482+0000","last_peered":"2023-11-06T20:45:04.712482+0000","last_clean":"2023-11-06T20:45:04.712482+0000","last_became_active":"2023-11-06T20:45:04.704732+0000","last_became_peered":"2023-11-06T20:45:04.704732+0000","last_unstale":"2023-11-06T20:45:04.712482+0000","last_undegraded":"2023-11-06T20:45:04.712482+0000","last_fullsized":"2023-11-06T20:45:04.712482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.706215+0000","last_change":"2023-11-06T20:45:04.210698+0000","last_active":"2023-11-06T20:45:04.706215+0000","last_peered":"2023-11-06T20:45:04.706215+0000","last_clean":"2023-11-06T20:45:04.706215+0000","last_became_active":"2023-11-06T20:45:04.209602+0000","last_became_peered":"2023-11-06T20:45:04.209602+0000","last_unstale":"2023-11-06T20:45:04.706215+0000","last_undegraded":"2023-11-06T20:45:04.706215+0000","last_fullsized":"2023-11-06T20:45:04.706215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.693012+0000","last_change":"2023-11-06T20:45:04.685737+0000","last_active":"2023-11-06T20:45:04.693012+0000","last_peered":"2023-11-06T20:45:04.693012+0000","last_clean":"2023-11-06T20:45:04.693012+0000","last_became_active":"2023-11-06T20:45:04.685417+0000","last_became_peered":"2023-11-06T20:45:04.685417+0000","last_unstale":"2023-11-06T20:45:04.693012+0000","last_undegraded":"2023-11-06T20:45:04.693012+0000","last_fullsized":"2023-11-06T20:45:04.693012+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681583+0000","last_change":"2023-11-06T20:45:04.197005+0000","last_active":"2023-11-06T20:45:04.681583+0000","last_peered":"2023-11-06T20:45:04.681583+0000","last_clean":"2023-11-06T20:45:04.681583+0000","last_became_active":"2023-11-06T20:45:04.196738+0000","last_became_peered":"2023-11-06T20:45:04.196738+0000","last_unstale":"2023-11-06T20:45:04.681583+0000","last_undegraded":"2023-11-06T20:45:04.681583+0000","last_fullsized":"2023-11-06T20:45:04.681583+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699919+0000","last_change":"2023-11-06T20:45:04.698880+0000","last_active":"2023-11-06T20:45:04.699919+0000","last_peered":"2023-11-06T20:45:04.699919+0000","last_clean":"2023-11-06T20:45:04.699919+0000","last_became_active":"2023-11-06T20:45:04.697976+0000","last_became_peered":"2023-11-06T20:45:04.697976+0000","last_unstale":"2023-11-06T20:45:04.699919+0000","last_undegraded":"2023-11-06T20:45:04.699919+0000","last_fullsized":"2023-11-06T20:45:04.699919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.694686+0000","last_change":"2023-11-06T20:45:04.204378+0000","last_active":"2023-11-06T20:45:04.694686+0000","last_peered":"2023-11-06T20:45:04.694686+0000","last_clean":"2023-11-06T20:45:04.694686+0000","last_became_active":"2023-11-06T20:45:04.204086+0000","last_became_peered":"2023-11-06T20:45:04.204086+0000","last_unstale":"2023-11-06T20:45:04.694686+0000","last_undegraded":"2023-11-06T20:45:04.694686+0000","last_fullsized":"2023-11-06T20:45:04.694686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703867+0000","last_change":"2023-11-06T20:45:04.701051+0000","last_active":"2023-11-06T20:45:04.703867+0000","last_peered":"2023-11-06T20:45:04.703867+0000","last_clean":"2023-11-06T20:45:04.703867+0000","last_became_active":"2023-11-06T20:45:04.700684+0000","last_became_peered":"2023-11-06T20:45:04.700684+0000","last_unstale":"2023-11-06T20:45:04.703867+0000","last_undegraded":"2023-11-06T20:45:04.703867+0000","last_fullsized":"2023-11-06T20:45:04.703867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.30","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.720414+0000","last_change":"2023-11-06T20:45:04.199618+0000","last_active":"2023-11-06T20:45:04.720414+0000","last_peered":"2023-11-06T20:45:04.720414+0000","last_clean":"2023-11-06T20:45:04.720414+0000","last_became_active":"2023-11-06T20:45:04.198972+0000","last_became_peered":"2023-11-06T20:45:04.198972+0000","last_unstale":"2023-11-06T20:45:04.720414+0000","last_undegraded":"2023-11-06T20:45:04.720414+0000","last_fullsized":"2023-11-06T20:45:04.720414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714421+0000","last_change":"2023-11-06T20:45:04.709196+0000","last_active":"2023-11-06T20:45:04.714421+0000","last_peered":"2023-11-06T20:45:04.714421+0000","last_clean":"2023-11-06T20:45:04.714421+0000","last_became_active":"2023-11-06T20:45:04.708923+0000","last_became_peered":"2023-11-06T20:45:04.708923+0000","last_unstale":"2023-11-06T20:45:04.714421+0000","last_undegraded":"2023-11-06T20:45:04.714421+0000","last_fullsized":"2023-11-06T20:45:04.714421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680315+0000","last_change":"2023-11-06T20:45:04.198617+0000","last_active":"2023-11-06T20:45:04.680315+0000","last_peered":"2023-11-06T20:45:04.680315+0000","last_clean":"2023-11-06T20:45:04.680315+0000","last_became_active":"2023-11-06T20:45:04.196842+0000","last_became_peered":"2023-11-06T20:45:04.196842+0000","last_unstale":"2023-11-06T20:45:04.680315+0000","last_undegraded":"2023-11-06T20:45:04.680315+0000","last_fullsized":"2023-11-06T20:45:04.680315+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698492+0000","last_change":"2023-11-06T20:45:04.692466+0000","last_active":"2023-11-06T20:45:04.698492+0000","last_peered":"2023-11-06T20:45:04.698492+0000","last_clean":"2023-11-06T20:45:04.698492+0000","last_became_active":"2023-11-06T20:45:04.689566+0000","last_became_peered":"2023-11-06T20:45:04.689566+0000","last_unstale":"2023-11-06T20:45:04.698492+0000","last_undegraded":"2023-11-06T20:45:04.698492+0000","last_fullsized":"2023-11-06T20:45:04.698492+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677732+0000","last_change":"2023-11-06T20:45:04.212646+0000","last_active":"2023-11-06T20:45:04.677732+0000","last_peered":"2023-11-06T20:45:04.677732+0000","last_clean":"2023-11-06T20:45:04.677732+0000","last_became_active":"2023-11-06T20:45:04.212353+0000","last_became_peered":"2023-11-06T20:45:04.212353+0000","last_unstale":"2023-11-06T20:45:04.677732+0000","last_undegraded":"2023-11-06T20:45:04.677732+0000","last_fullsized":"2023-11-06T20:45:04.677732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.701538+0000","last_change":"2023-11-06T20:45:04.691702+0000","last_active":"2023-11-06T20:45:04.701538+0000","last_peered":"2023-11-06T20:45:04.701538+0000","last_clean":"2023-11-06T20:45:04.701538+0000","last_became_active":"2023-11-06T20:45:04.691407+0000","last_became_peered":"2023-11-06T20:45:04.691407+0000","last_unstale":"2023-11-06T20:45:04.701538+0000","last_undegraded":"2023-11-06T20:45:04.701538+0000","last_fullsized":"2023-11-06T20:45:04.701538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715714+0000","last_change":"2023-11-06T20:45:04.216348+0000","last_active":"2023-11-06T20:45:04.715714+0000","last_peered":"2023-11-06T20:45:04.715714+0000","last_clean":"2023-11-06T20:45:04.715714+0000","last_became_active":"2023-11-06T20:45:04.215851+0000","last_became_peered":"2023-11-06T20:45:04.215851+0000","last_unstale":"2023-11-06T20:45:04.715714+0000","last_undegraded":"2023-11-06T20:45:04.715714+0000","last_fullsized":"2023-11-06T20:45:04.715714+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713218+0000","last_change":"2023-11-06T20:45:04.709714+0000","last_active":"2023-11-06T20:45:04.713218+0000","last_peered":"2023-11-06T20:45:04.713218+0000","last_clean":"2023-11-06T20:45:04.713218+0000","last_became_active":"2023-11-06T20:45:04.709422+0000","last_became_peered":"2023-11-06T20:45:04.709422+0000","last_unstale":"2023-11-06T20:45:04.713218+0000","last_undegraded":"2023-11-06T20:45:04.713218+0000","last_fullsized":"2023-11-06T20:45:04.713218+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677364+0000","last_change":"2023-11-06T20:45:04.214892+0000","last_active":"2023-11-06T20:45:04.677364+0000","last_peered":"2023-11-06T20:45:04.677364+0000","last_clean":"2023-11-06T20:45:04.677364+0000","last_became_active":"2023-11-06T20:45:04.214629+0000","last_became_peered":"2023-11-06T20:45:04.214629+0000","last_unstale":"2023-11-06T20:45:04.677364+0000","last_undegraded":"2023-11-06T20:45:04.677364+0000","last_fullsized":"2023-11-06T20:45:04.677364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690595+0000","last_change":"2023-11-06T20:45:04.687633+0000","last_active":"2023-11-06T20:45:04.690595+0000","last_peered":"2023-11-06T20:45:04.690595+0000","last_clean":"2023-11-06T20:45:04.690595+0000","last_became_active":"2023-11-06T20:45:04.687278+0000","last_became_peered":"2023-11-06T20:45:04.687278+0000","last_unstale":"2023-11-06T20:45:04.690595+0000","last_undegraded":"2023-11-06T20:45:04.690595+0000","last_fullsized":"2023-11-06T20:45:04.690595+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2b","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715714+0000","last_change":"2023-11-06T20:45:04.213253+0000","last_active":"2023-11-06T20:45:04.715714+0000","last_peered":"2023-11-06T20:45:04.715714+0000","last_clean":"2023-11-06T20:45:04.715714+0000","last_became_active":"2023-11-06T20:45:04.212919+0000","last_became_peered":"2023-11-06T20:45:04.212919+0000","last_unstale":"2023-11-06T20:45:04.715714+0000","last_undegraded":"2023-11-06T20:45:04.715714+0000","last_fullsized":"2023-11-06T20:45:04.715714+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.694384+0000","last_change":"2023-11-06T20:45:04.686984+0000","last_active":"2023-11-06T20:45:04.694384+0000","last_peered":"2023-11-06T20:45:04.694384+0000","last_clean":"2023-11-06T20:45:04.694384+0000","last_became_active":"2023-11-06T20:45:04.686703+0000","last_became_peered":"2023-11-06T20:45:04.686703+0000","last_unstale":"2023-11-06T20:45:04.694384+0000","last_undegraded":"2023-11-06T20:45:04.694384+0000","last_fullsized":"2023-11-06T20:45:04.694384+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681053+0000","last_change":"2023-11-06T20:45:04.208631+0000","last_active":"2023-11-06T20:45:04.681053+0000","last_peered":"2023-11-06T20:45:04.681053+0000","last_clean":"2023-11-06T20:45:04.681053+0000","last_became_active":"2023-11-06T20:45:04.208313+0000","last_became_peered":"2023-11-06T20:45:04.208313+0000","last_unstale":"2023-11-06T20:45:04.681053+0000","last_undegraded":"2023-11-06T20:45:04.681053+0000","last_fullsized":"2023-11-06T20:45:04.681053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.708756+0000","last_change":"2023-11-06T20:45:04.692680+0000","last_active":"2023-11-06T20:45:04.708756+0000","last_peered":"2023-11-06T20:45:04.708756+0000","last_clean":"2023-11-06T20:45:04.708756+0000","last_became_active":"2023-11-06T20:45:04.692335+0000","last_became_peered":"2023-11-06T20:45:04.692335+0000","last_unstale":"2023-11-06T20:45:04.708756+0000","last_undegraded":"2023-11-06T20:45:04.708756+0000","last_fullsized":"2023-11-06T20:45:04.708756+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677027+0000","last_change":"2023-11-06T20:45:04.207900+0000","last_active":"2023-11-06T20:45:04.677027+0000","last_peered":"2023-11-06T20:45:04.677027+0000","last_clean":"2023-11-06T20:45:04.677027+0000","last_became_active":"2023-11-06T20:45:04.207618+0000","last_became_peered":"2023-11-06T20:45:04.207618+0000","last_unstale":"2023-11-06T20:45:04.677027+0000","last_undegraded":"2023-11-06T20:45:04.677027+0000","last_fullsized":"2023-11-06T20:45:04.677027+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.709909+0000","last_change":"2023-11-06T20:45:04.697841+0000","last_active":"2023-11-06T20:45:04.709909+0000","last_peered":"2023-11-06T20:45:04.709909+0000","last_clean":"2023-11-06T20:45:04.709909+0000","last_became_active":"2023-11-06T20:45:04.697442+0000","last_became_peered":"2023-11-06T20:45:04.697442+0000","last_unstale":"2023-11-06T20:45:04.709909+0000","last_undegraded":"2023-11-06T20:45:04.709909+0000","last_fullsized":"2023-11-06T20:45:04.709909+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680469+0000","last_change":"2023-11-06T20:45:04.208028+0000","last_active":"2023-11-06T20:45:04.680469+0000","last_peered":"2023-11-06T20:45:04.680469+0000","last_clean":"2023-11-06T20:45:04.680469+0000","last_became_active":"2023-11-06T20:45:04.207152+0000","last_became_peered":"2023-11-06T20:45:04.207152+0000","last_unstale":"2023-11-06T20:45:04.680469+0000","last_undegraded":"2023-11-06T20:45:04.680469+0000","last_fullsized":"2023-11-06T20:45:04.680469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714410+0000","last_change":"2023-11-06T20:45:04.710247+0000","last_active":"2023-11-06T20:45:04.714410+0000","last_peered":"2023-11-06T20:45:04.714410+0000","last_clean":"2023-11-06T20:45:04.714410+0000","last_became_active":"2023-11-06T20:45:04.709893+0000","last_became_peered":"2023-11-06T20:45:04.709893+0000","last_unstale":"2023-11-06T20:45:04.714410+0000","last_undegraded":"2023-11-06T20:45:04.714410+0000","last_fullsized":"2023-11-06T20:45:04.714410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.684789+0000","last_change":"2023-11-06T20:45:04.201589+0000","last_active":"2023-11-06T20:45:04.684789+0000","last_peered":"2023-11-06T20:45:04.684789+0000","last_clean":"2023-11-06T20:45:04.684789+0000","last_became_active":"2023-11-06T20:45:04.201293+0000","last_became_peered":"2023-11-06T20:45:04.201293+0000","last_unstale":"2023-11-06T20:45:04.684789+0000","last_undegraded":"2023-11-06T20:45:04.684789+0000","last_fullsized":"2023-11-06T20:45:04.684789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.709998+0000","last_change":"2023-11-06T20:45:04.699009+0000","last_active":"2023-11-06T20:45:04.709998+0000","last_peered":"2023-11-06T20:45:04.709998+0000","last_clean":"2023-11-06T20:45:04.709998+0000","last_became_active":"2023-11-06T20:45:04.698741+0000","last_became_peered":"2023-11-06T20:45:04.698741+0000","last_unstale":"2023-11-06T20:45:04.709998+0000","last_undegraded":"2023-11-06T20:45:04.709998+0000","last_fullsized":"2023-11-06T20:45:04.709998+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.26","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-06T20:45:06.403062+0000","last_change":"2023-11-06T20:45:04.216159+0000","last_active":"2023-11-06T20:45:06.403062+0000","last_peered":"2023-11-06T20:45:06.403062+0000","last_clean":"2023-11-06T20:45:06.403062+0000","last_became_active":"2023-11-06T20:45:04.215706+0000","last_became_peered":"2023-11-06T20:45:04.215706+0000","last_unstale":"2023-11-06T20:45:06.403062+0000","last_undegraded":"2023-11-06T20:45:06.403062+0000","last_fullsized":"2023-11-06T20:45:06.403062+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690338+0000","last_change":"2023-11-06T20:45:04.687607+0000","last_active":"2023-11-06T20:45:04.690338+0000","last_peered":"2023-11-06T20:45:04.690338+0000","last_clean":"2023-11-06T20:45:04.690338+0000","last_became_active":"2023-11-06T20:45:04.687273+0000","last_became_peered":"2023-11-06T20:45:04.687273+0000","last_unstale":"2023-11-06T20:45:04.690338+0000","last_undegraded":"2023-11-06T20:45:04.690338+0000","last_fullsized":"2023-11-06T20:45:04.690338+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.683094+0000","last_change":"2023-11-06T20:45:04.202691+0000","last_active":"2023-11-06T20:45:04.683094+0000","last_peered":"2023-11-06T20:45:04.683094+0000","last_clean":"2023-11-06T20:45:04.683094+0000","last_became_active":"2023-11-06T20:45:04.202362+0000","last_became_peered":"2023-11-06T20:45:04.202362+0000","last_unstale":"2023-11-06T20:45:04.683094+0000","last_undegraded":"2023-11-06T20:45:04.683094+0000","last_fullsized":"2023-11-06T20:45:04.683094+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700044+0000","last_change":"2023-11-06T20:45:04.693933+0000","last_active":"2023-11-06T20:45:04.700044+0000","last_peered":"2023-11-06T20:45:04.700044+0000","last_clean":"2023-11-06T20:45:04.700044+0000","last_became_active":"2023-11-06T20:45:04.693204+0000","last_became_peered":"2023-11-06T20:45:04.693204+0000","last_unstale":"2023-11-06T20:45:04.700044+0000","last_undegraded":"2023-11-06T20:45:04.700044+0000","last_fullsized":"2023-11-06T20:45:04.700044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681743+0000","last_change":"2023-11-06T20:45:04.213724+0000","last_active":"2023-11-06T20:45:04.681743+0000","last_peered":"2023-11-06T20:45:04.681743+0000","last_clean":"2023-11-06T20:45:04.681743+0000","last_became_active":"2023-11-06T20:45:04.211299+0000","last_became_peered":"2023-11-06T20:45:04.211299+0000","last_unstale":"2023-11-06T20:45:04.681743+0000","last_undegraded":"2023-11-06T20:45:04.681743+0000","last_fullsized":"2023-11-06T20:45:04.681743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700142+0000","last_change":"2023-11-06T20:45:04.697198+0000","last_active":"2023-11-06T20:45:04.700142+0000","last_peered":"2023-11-06T20:45:04.700142+0000","last_clean":"2023-11-06T20:45:04.700142+0000","last_became_active":"2023-11-06T20:45:04.696745+0000","last_became_peered":"2023-11-06T20:45:04.696745+0000","last_unstale":"2023-11-06T20:45:04.700142+0000","last_undegraded":"2023-11-06T20:45:04.700142+0000","last_fullsized":"2023-11-06T20:45:04.700142+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.686459+0000","last_change":"2023-11-06T20:45:04.205989+0000","last_active":"2023-11-06T20:45:04.686459+0000","last_peered":"2023-11-06T20:45:04.686459+0000","last_clean":"2023-11-06T20:45:04.686459+0000","last_became_active":"2023-11-06T20:45:04.205719+0000","last_became_peered":"2023-11-06T20:45:04.205719+0000","last_unstale":"2023-11-06T20:45:04.686459+0000","last_undegraded":"2023-11-06T20:45:04.686459+0000","last_fullsized":"2023-11-06T20:45:04.686459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703797+0000","last_change":"2023-11-06T20:45:04.701216+0000","last_active":"2023-11-06T20:45:04.703797+0000","last_peered":"2023-11-06T20:45:04.703797+0000","last_clean":"2023-11-06T20:45:04.703797+0000","last_became_active":"2023-11-06T20:45:04.700956+0000","last_became_peered":"2023-11-06T20:45:04.700956+0000","last_unstale":"2023-11-06T20:45:04.703797+0000","last_undegraded":"2023-11-06T20:45:04.703797+0000","last_fullsized":"2023-11-06T20:45:04.703797+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.685550+0000","last_change":"2023-11-06T20:45:04.200776+0000","last_active":"2023-11-06T20:45:04.685550+0000","last_peered":"2023-11-06T20:45:04.685550+0000","last_clean":"2023-11-06T20:45:04.685550+0000","last_became_active":"2023-11-06T20:45:04.200461+0000","last_became_peered":"2023-11-06T20:45:04.200461+0000","last_unstale":"2023-11-06T20:45:04.685550+0000","last_undegraded":"2023-11-06T20:45:04.685550+0000","last_fullsized":"2023-11-06T20:45:04.685550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.702198+0000","last_change":"2023-11-06T20:45:04.695957+0000","last_active":"2023-11-06T20:45:04.702198+0000","last_peered":"2023-11-06T20:45:04.702198+0000","last_clean":"2023-11-06T20:45:04.702198+0000","last_became_active":"2023-11-06T20:45:04.695673+0000","last_became_peered":"2023-11-06T20:45:04.695673+0000","last_unstale":"2023-11-06T20:45:04.702198+0000","last_undegraded":"2023-11-06T20:45:04.702198+0000","last_fullsized":"2023-11-06T20:45:04.702198+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680966+0000","last_change":"2023-11-06T20:45:04.205274+0000","last_active":"2023-11-06T20:45:04.680966+0000","last_peered":"2023-11-06T20:45:04.680966+0000","last_clean":"2023-11-06T20:45:04.680966+0000","last_became_active":"2023-11-06T20:45:04.204948+0000","last_became_peered":"2023-11-06T20:45:04.204948+0000","last_unstale":"2023-11-06T20:45:04.680966+0000","last_undegraded":"2023-11-06T20:45:04.680966+0000","last_fullsized":"2023-11-06T20:45:04.680966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.702112+0000","last_change":"2023-11-06T20:45:04.695538+0000","last_active":"2023-11-06T20:45:04.702112+0000","last_peered":"2023-11-06T20:45:04.702112+0000","last_clean":"2023-11-06T20:45:04.702112+0000","last_became_active":"2023-11-06T20:45:04.695238+0000","last_became_peered":"2023-11-06T20:45:04.695238+0000","last_unstale":"2023-11-06T20:45:04.702112+0000","last_undegraded":"2023-11-06T20:45:04.702112+0000","last_fullsized":"2023-11-06T20:45:04.702112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.674214+0000","last_change":"2023-11-06T20:45:04.201871+0000","last_active":"2023-11-06T20:45:04.674214+0000","last_peered":"2023-11-06T20:45:04.674214+0000","last_clean":"2023-11-06T20:45:04.674214+0000","last_became_active":"2023-11-06T20:45:04.200787+0000","last_became_peered":"2023-11-06T20:45:04.200787+0000","last_unstale":"2023-11-06T20:45:04.674214+0000","last_undegraded":"2023-11-06T20:45:04.674214+0000","last_fullsized":"2023-11-06T20:45:04.674214+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698732+0000","last_change":"2023-11-06T20:45:04.688853+0000","last_active":"2023-11-06T20:45:04.698732+0000","last_peered":"2023-11-06T20:45:04.698732+0000","last_clean":"2023-11-06T20:45:04.698732+0000","last_became_active":"2023-11-06T20:45:04.688554+0000","last_became_peered":"2023-11-06T20:45:04.688554+0000","last_unstale":"2023-11-06T20:45:04.698732+0000","last_undegraded":"2023-11-06T20:45:04.698732+0000","last_fullsized":"2023-11-06T20:45:04.698732+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699237+0000","last_change":"2023-11-06T20:45:04.199975+0000","last_active":"2023-11-06T20:45:04.699237+0000","last_peered":"2023-11-06T20:45:04.699237+0000","last_clean":"2023-11-06T20:45:04.699237+0000","last_became_active":"2023-11-06T20:45:04.195195+0000","last_became_peered":"2023-11-06T20:45:04.195195+0000","last_unstale":"2023-11-06T20:45:04.699237+0000","last_undegraded":"2023-11-06T20:45:04.699237+0000","last_fullsized":"2023-11-06T20:45:04.699237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.697553+0000","last_change":"2023-11-06T20:45:04.692362+0000","last_active":"2023-11-06T20:45:04.697553+0000","last_peered":"2023-11-06T20:45:04.697553+0000","last_clean":"2023-11-06T20:45:04.697553+0000","last_became_active":"2023-11-06T20:45:04.691882+0000","last_became_peered":"2023-11-06T20:45:04.691882+0000","last_unstale":"2023-11-06T20:45:04.697553+0000","last_undegraded":"2023-11-06T20:45:04.697553+0000","last_fullsized":"2023-11-06T20:45:04.697553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713067+0000","last_change":"2023-11-06T20:45:04.209292+0000","last_active":"2023-11-06T20:45:04.713067+0000","last_peered":"2023-11-06T20:45:04.713067+0000","last_clean":"2023-11-06T20:45:04.713067+0000","last_became_active":"2023-11-06T20:45:04.208357+0000","last_became_peered":"2023-11-06T20:45:04.208357+0000","last_unstale":"2023-11-06T20:45:04.713067+0000","last_undegraded":"2023-11-06T20:45:04.713067+0000","last_fullsized":"2023-11-06T20:45:04.713067+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.694110+0000","last_change":"2023-11-06T20:45:04.687618+0000","last_active":"2023-11-06T20:45:04.694110+0000","last_peered":"2023-11-06T20:45:04.694110+0000","last_clean":"2023-11-06T20:45:04.694110+0000","last_became_active":"2023-11-06T20:45:04.687325+0000","last_became_peered":"2023-11-06T20:45:04.687325+0000","last_unstale":"2023-11-06T20:45:04.694110+0000","last_undegraded":"2023-11-06T20:45:04.694110+0000","last_fullsized":"2023-11-06T20:45:04.694110+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700723+0000","last_change":"2023-11-06T20:45:04.211925+0000","last_active":"2023-11-06T20:45:04.700723+0000","last_peered":"2023-11-06T20:45:04.700723+0000","last_clean":"2023-11-06T20:45:04.700723+0000","last_became_active":"2023-11-06T20:45:04.211610+0000","last_became_peered":"2023-11-06T20:45:04.211610+0000","last_unstale":"2023-11-06T20:45:04.700723+0000","last_undegraded":"2023-11-06T20:45:04.700723+0000","last_fullsized":"2023-11-06T20:45:04.700723+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.709752+0000","last_change":"2023-11-06T20:45:04.706215+0000","last_active":"2023-11-06T20:45:04.709752+0000","last_peered":"2023-11-06T20:45:04.709752+0000","last_clean":"2023-11-06T20:45:04.709752+0000","last_became_active":"2023-11-06T20:45:04.705922+0000","last_became_peered":"2023-11-06T20:45:04.705922+0000","last_unstale":"2023-11-06T20:45:04.709752+0000","last_undegraded":"2023-11-06T20:45:04.709752+0000","last_fullsized":"2023-11-06T20:45:04.709752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681802+0000","last_change":"2023-11-06T20:45:04.209004+0000","last_active":"2023-11-06T20:45:04.681802+0000","last_peered":"2023-11-06T20:45:04.681802+0000","last_clean":"2023-11-06T20:45:04.681802+0000","last_became_active":"2023-11-06T20:45:04.208295+0000","last_became_peered":"2023-11-06T20:45:04.208295+0000","last_unstale":"2023-11-06T20:45:04.681802+0000","last_undegraded":"2023-11-06T20:45:04.681802+0000","last_fullsized":"2023-11-06T20:45:04.681802+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690207+0000","last_change":"2023-11-06T20:45:04.686606+0000","last_active":"2023-11-06T20:45:04.690207+0000","last_peered":"2023-11-06T20:45:04.690207+0000","last_clean":"2023-11-06T20:45:04.690207+0000","last_became_active":"2023-11-06T20:45:04.686244+0000","last_became_peered":"2023-11-06T20:45:04.686244+0000","last_unstale":"2023-11-06T20:45:04.690207+0000","last_undegraded":"2023-11-06T20:45:04.690207+0000","last_fullsized":"2023-11-06T20:45:04.690207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680947+0000","last_change":"2023-11-06T20:45:04.200869+0000","last_active":"2023-11-06T20:45:04.680947+0000","last_peered":"2023-11-06T20:45:04.680947+0000","last_clean":"2023-11-06T20:45:04.680947+0000","last_became_active":"2023-11-06T20:45:04.200557+0000","last_became_peered":"2023-11-06T20:45:04.200557+0000","last_unstale":"2023-11-06T20:45:04.680947+0000","last_undegraded":"2023-11-06T20:45:04.680947+0000","last_fullsized":"2023-11-06T20:45:04.680947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.710657+0000","last_change":"2023-11-06T20:45:04.696624+0000","last_active":"2023-11-06T20:45:04.710657+0000","last_peered":"2023-11-06T20:45:04.710657+0000","last_clean":"2023-11-06T20:45:04.710657+0000","last_became_active":"2023-11-06T20:45:04.696081+0000","last_became_peered":"2023-11-06T20:45:04.696081+0000","last_unstale":"2023-11-06T20:45:04.710657+0000","last_undegraded":"2023-11-06T20:45:04.710657+0000","last_fullsized":"2023-11-06T20:45:04.710657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696708+0000","last_change":"2023-11-06T20:45:04.205999+0000","last_active":"2023-11-06T20:45:04.696708+0000","last_peered":"2023-11-06T20:45:04.696708+0000","last_clean":"2023-11-06T20:45:04.696708+0000","last_became_active":"2023-11-06T20:45:04.205244+0000","last_became_peered":"2023-11-06T20:45:04.205244+0000","last_unstale":"2023-11-06T20:45:04.696708+0000","last_undegraded":"2023-11-06T20:45:04.696708+0000","last_fullsized":"2023-11-06T20:45:04.696708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.704806+0000","last_change":"2023-11-06T20:45:04.702026+0000","last_active":"2023-11-06T20:45:04.704806+0000","last_peered":"2023-11-06T20:45:04.704806+0000","last_clean":"2023-11-06T20:45:04.704806+0000","last_became_active":"2023-11-06T20:45:04.701450+0000","last_became_peered":"2023-11-06T20:45:04.701450+0000","last_unstale":"2023-11-06T20:45:04.704806+0000","last_undegraded":"2023-11-06T20:45:04.704806+0000","last_fullsized":"2023-11-06T20:45:04.704806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680922+0000","last_change":"2023-11-06T20:45:04.202263+0000","last_active":"2023-11-06T20:45:04.680922+0000","last_peered":"2023-11-06T20:45:04.680922+0000","last_clean":"2023-11-06T20:45:04.680922+0000","last_became_active":"2023-11-06T20:45:04.201949+0000","last_became_peered":"2023-11-06T20:45:04.201949+0000","last_unstale":"2023-11-06T20:45:04.680922+0000","last_undegraded":"2023-11-06T20:45:04.680922+0000","last_fullsized":"2023-11-06T20:45:04.680922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.691959+0000","last_change":"2023-11-06T20:45:04.684016+0000","last_active":"2023-11-06T20:45:04.691959+0000","last_peered":"2023-11-06T20:45:04.691959+0000","last_clean":"2023-11-06T20:45:04.691959+0000","last_became_active":"2023-11-06T20:45:04.683735+0000","last_became_peered":"2023-11-06T20:45:04.683735+0000","last_unstale":"2023-11-06T20:45:04.691959+0000","last_undegraded":"2023-11-06T20:45:04.691959+0000","last_fullsized":"2023-11-06T20:45:04.691959+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696679+0000","last_change":"2023-11-06T20:45:04.194284+0000","last_active":"2023-11-06T20:45:04.696679+0000","last_peered":"2023-11-06T20:45:04.696679+0000","last_clean":"2023-11-06T20:45:04.696679+0000","last_became_active":"2023-11-06T20:45:04.193972+0000","last_became_peered":"2023-11-06T20:45:04.193972+0000","last_unstale":"2023-11-06T20:45:04.696679+0000","last_undegraded":"2023-11-06T20:45:04.696679+0000","last_fullsized":"2023-11-06T20:45:04.696679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700309+0000","last_change":"2023-11-06T20:45:04.697932+0000","last_active":"2023-11-06T20:45:04.700309+0000","last_peered":"2023-11-06T20:45:04.700309+0000","last_clean":"2023-11-06T20:45:04.700309+0000","last_became_active":"2023-11-06T20:45:04.695307+0000","last_became_peered":"2023-11-06T20:45:04.695307+0000","last_unstale":"2023-11-06T20:45:04.700309+0000","last_undegraded":"2023-11-06T20:45:04.700309+0000","last_fullsized":"2023-11-06T20:45:04.700309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.684880+0000","last_change":"2023-11-06T20:45:04.195073+0000","last_active":"2023-11-06T20:45:04.684880+0000","last_peered":"2023-11-06T20:45:04.684880+0000","last_clean":"2023-11-06T20:45:04.684880+0000","last_became_active":"2023-11-06T20:45:04.194548+0000","last_became_peered":"2023-11-06T20:45:04.194548+0000","last_unstale":"2023-11-06T20:45:04.684880+0000","last_undegraded":"2023-11-06T20:45:04.684880+0000","last_fullsized":"2023-11-06T20:45:04.684880+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700556+0000","last_change":"2023-11-06T20:45:04.699519+0000","last_active":"2023-11-06T20:45:04.700556+0000","last_peered":"2023-11-06T20:45:04.700556+0000","last_clean":"2023-11-06T20:45:04.700556+0000","last_became_active":"2023-11-06T20:45:04.699244+0000","last_became_peered":"2023-11-06T20:45:04.699244+0000","last_unstale":"2023-11-06T20:45:04.700556+0000","last_undegraded":"2023-11-06T20:45:04.700556+0000","last_fullsized":"2023-11-06T20:45:04.700556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677623+0000","last_change":"2023-11-06T20:45:04.193135+0000","last_active":"2023-11-06T20:45:04.677623+0000","last_peered":"2023-11-06T20:45:04.677623+0000","last_clean":"2023-11-06T20:45:04.677623+0000","last_became_active":"2023-11-06T20:45:04.192814+0000","last_became_peered":"2023-11-06T20:45:04.192814+0000","last_unstale":"2023-11-06T20:45:04.677623+0000","last_undegraded":"2023-11-06T20:45:04.677623+0000","last_fullsized":"2023-11-06T20:45:04.677623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.705951+0000","last_change":"2023-11-06T20:45:04.704375+0000","last_active":"2023-11-06T20:45:04.705951+0000","last_peered":"2023-11-06T20:45:04.705951+0000","last_clean":"2023-11-06T20:45:04.705951+0000","last_became_active":"2023-11-06T20:45:04.703514+0000","last_became_peered":"2023-11-06T20:45:04.703514+0000","last_unstale":"2023-11-06T20:45:04.705951+0000","last_undegraded":"2023-11-06T20:45:04.705951+0000","last_fullsized":"2023-11-06T20:45:04.705951+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713284+0000","last_change":"2023-11-06T20:45:04.209762+0000","last_active":"2023-11-06T20:45:04.713284+0000","last_peered":"2023-11-06T20:45:04.713284+0000","last_clean":"2023-11-06T20:45:04.713284+0000","last_became_active":"2023-11-06T20:45:04.209472+0000","last_became_peered":"2023-11-06T20:45:04.209472+0000","last_unstale":"2023-11-06T20:45:04.713284+0000","last_undegraded":"2023-11-06T20:45:04.713284+0000","last_fullsized":"2023-11-06T20:45:04.713284+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698113+0000","last_change":"2023-11-06T20:45:04.694154+0000","last_active":"2023-11-06T20:45:04.698113+0000","last_peered":"2023-11-06T20:45:04.698113+0000","last_clean":"2023-11-06T20:45:04.698113+0000","last_became_active":"2023-11-06T20:45:04.693824+0000","last_became_peered":"2023-11-06T20:45:04.693824+0000","last_unstale":"2023-11-06T20:45:04.698113+0000","last_undegraded":"2023-11-06T20:45:04.698113+0000","last_fullsized":"2023-11-06T20:45:04.698113+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.707587+0000","last_change":"2023-11-06T20:45:04.202088+0000","last_active":"2023-11-06T20:45:04.707587+0000","last_peered":"2023-11-06T20:45:04.707587+0000","last_clean":"2023-11-06T20:45:04.707587+0000","last_became_active":"2023-11-06T20:45:04.201823+0000","last_became_peered":"2023-11-06T20:45:04.201823+0000","last_unstale":"2023-11-06T20:45:04.707587+0000","last_undegraded":"2023-11-06T20:45:04.707587+0000","last_fullsized":"2023-11-06T20:45:04.707587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690572+0000","last_change":"2023-11-06T20:45:04.687056+0000","last_active":"2023-11-06T20:45:04.690572+0000","last_peered":"2023-11-06T20:45:04.690572+0000","last_clean":"2023-11-06T20:45:04.690572+0000","last_became_active":"2023-11-06T20:45:04.686668+0000","last_became_peered":"2023-11-06T20:45:04.686668+0000","last_unstale":"2023-11-06T20:45:04.690572+0000","last_undegraded":"2023-11-06T20:45:04.690572+0000","last_fullsized":"2023-11-06T20:45:04.690572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.702755+0000","last_change":"2023-11-06T20:45:04.212831+0000","last_active":"2023-11-06T20:45:04.702755+0000","last_peered":"2023-11-06T20:45:04.702755+0000","last_clean":"2023-11-06T20:45:04.702755+0000","last_became_active":"2023-11-06T20:45:04.212067+0000","last_became_peered":"2023-11-06T20:45:04.212067+0000","last_unstale":"2023-11-06T20:45:04.702755+0000","last_undegraded":"2023-11-06T20:45:04.702755+0000","last_fullsized":"2023-11-06T20:45:04.702755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.709918+0000","last_change":"2023-11-06T20:45:04.707036+0000","last_active":"2023-11-06T20:45:04.709918+0000","last_peered":"2023-11-06T20:45:04.709918+0000","last_clean":"2023-11-06T20:45:04.709918+0000","last_became_active":"2023-11-06T20:45:04.706688+0000","last_became_peered":"2023-11-06T20:45:04.706688+0000","last_unstale":"2023-11-06T20:45:04.709918+0000","last_undegraded":"2023-11-06T20:45:04.709918+0000","last_fullsized":"2023-11-06T20:45:04.709918+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.676833+0000","last_change":"2023-11-06T20:45:04.206935+0000","last_active":"2023-11-06T20:45:04.676833+0000","last_peered":"2023-11-06T20:45:04.676833+0000","last_clean":"2023-11-06T20:45:04.676833+0000","last_became_active":"2023-11-06T20:45:04.205748+0000","last_became_peered":"2023-11-06T20:45:04.205748+0000","last_unstale":"2023-11-06T20:45:04.676833+0000","last_undegraded":"2023-11-06T20:45:04.676833+0000","last_fullsized":"2023-11-06T20:45:04.676833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.710033+0000","last_change":"2023-11-06T20:45:04.698202+0000","last_active":"2023-11-06T20:45:04.710033+0000","last_peered":"2023-11-06T20:45:04.710033+0000","last_clean":"2023-11-06T20:45:04.710033+0000","last_became_active":"2023-11-06T20:45:04.688664+0000","last_became_peered":"2023-11-06T20:45:04.688664+0000","last_unstale":"2023-11-06T20:45:04.710033+0000","last_undegraded":"2023-11-06T20:45:04.710033+0000","last_fullsized":"2023-11-06T20:45:04.710033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.674795+0000","last_change":"2023-11-06T20:45:04.200818+0000","last_active":"2023-11-06T20:45:04.674795+0000","last_peered":"2023-11-06T20:45:04.674795+0000","last_clean":"2023-11-06T20:45:04.674795+0000","last_became_active":"2023-11-06T20:45:04.200522+0000","last_became_peered":"2023-11-06T20:45:04.200522+0000","last_unstale":"2023-11-06T20:45:04.674795+0000","last_undegraded":"2023-11-06T20:45:04.674795+0000","last_fullsized":"2023-11-06T20:45:04.674795+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.697088+0000","last_change":"2023-11-06T20:45:04.692936+0000","last_active":"2023-11-06T20:45:04.697088+0000","last_peered":"2023-11-06T20:45:04.697088+0000","last_clean":"2023-11-06T20:45:04.697088+0000","last_became_active":"2023-11-06T20:45:04.692621+0000","last_became_peered":"2023-11-06T20:45:04.692621+0000","last_unstale":"2023-11-06T20:45:04.697088+0000","last_undegraded":"2023-11-06T20:45:04.697088+0000","last_fullsized":"2023-11-06T20:45:04.697088+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714056+0000","last_change":"2023-11-06T20:45:04.199156+0000","last_active":"2023-11-06T20:45:04.714056+0000","last_peered":"2023-11-06T20:45:04.714056+0000","last_clean":"2023-11-06T20:45:04.714056+0000","last_became_active":"2023-11-06T20:45:04.198858+0000","last_became_peered":"2023-11-06T20:45:04.198858+0000","last_unstale":"2023-11-06T20:45:04.714056+0000","last_undegraded":"2023-11-06T20:45:04.714056+0000","last_fullsized":"2023-11-06T20:45:04.714056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690506+0000","last_change":"2023-11-06T20:45:04.689007+0000","last_active":"2023-11-06T20:45:04.690506+0000","last_peered":"2023-11-06T20:45:04.690506+0000","last_clean":"2023-11-06T20:45:04.690506+0000","last_became_active":"2023-11-06T20:45:04.688418+0000","last_became_peered":"2023-11-06T20:45:04.688418+0000","last_unstale":"2023-11-06T20:45:04.690506+0000","last_undegraded":"2023-11-06T20:45:04.690506+0000","last_fullsized":"2023-11-06T20:45:04.690506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715339+0000","last_change":"2023-11-06T20:45:04.199860+0000","last_active":"2023-11-06T20:45:04.715339+0000","last_peered":"2023-11-06T20:45:04.715339+0000","last_clean":"2023-11-06T20:45:04.715339+0000","last_became_active":"2023-11-06T20:45:04.197409+0000","last_became_peered":"2023-11-06T20:45:04.197409+0000","last_unstale":"2023-11-06T20:45:04.715339+0000","last_undegraded":"2023-11-06T20:45:04.715339+0000","last_fullsized":"2023-11-06T20:45:04.715339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713635+0000","last_change":"2023-11-06T20:45:04.707405+0000","last_active":"2023-11-06T20:45:04.713635+0000","last_peered":"2023-11-06T20:45:04.713635+0000","last_clean":"2023-11-06T20:45:04.713635+0000","last_became_active":"2023-11-06T20:45:04.704832+0000","last_became_peered":"2023-11-06T20:45:04.704832+0000","last_unstale":"2023-11-06T20:45:04.713635+0000","last_undegraded":"2023-11-06T20:45:04.713635+0000","last_fullsized":"2023-11-06T20:45:04.713635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712816+0000","last_change":"2023-11-06T20:45:04.198033+0000","last_active":"2023-11-06T20:45:04.712816+0000","last_peered":"2023-11-06T20:45:04.712816+0000","last_clean":"2023-11-06T20:45:04.712816+0000","last_became_active":"2023-11-06T20:45:04.197715+0000","last_became_peered":"2023-11-06T20:45:04.197715+0000","last_unstale":"2023-11-06T20:45:04.712816+0000","last_undegraded":"2023-11-06T20:45:04.712816+0000","last_fullsized":"2023-11-06T20:45:04.712816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713102+0000","last_change":"2023-11-06T20:45:04.703423+0000","last_active":"2023-11-06T20:45:04.713102+0000","last_peered":"2023-11-06T20:45:04.713102+0000","last_clean":"2023-11-06T20:45:04.713102+0000","last_became_active":"2023-11-06T20:45:04.703108+0000","last_became_peered":"2023-11-06T20:45:04.703108+0000","last_unstale":"2023-11-06T20:45:04.713102+0000","last_undegraded":"2023-11-06T20:45:04.713102+0000","last_fullsized":"2023-11-06T20:45:04.713102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677408+0000","last_change":"2023-11-06T20:45:04.212009+0000","last_active":"2023-11-06T20:45:04.677408+0000","last_peered":"2023-11-06T20:45:04.677408+0000","last_clean":"2023-11-06T20:45:04.677408+0000","last_became_active":"2023-11-06T20:45:04.211204+0000","last_became_peered":"2023-11-06T20:45:04.211204+0000","last_unstale":"2023-11-06T20:45:04.677408+0000","last_undegraded":"2023-11-06T20:45:04.677408+0000","last_fullsized":"2023-11-06T20:45:04.677408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698546+0000","last_change":"2023-11-06T20:45:04.690373+0000","last_active":"2023-11-06T20:45:04.698546+0000","last_peered":"2023-11-06T20:45:04.698546+0000","last_clean":"2023-11-06T20:45:04.698546+0000","last_became_active":"2023-11-06T20:45:04.690070+0000","last_became_peered":"2023-11-06T20:45:04.690070+0000","last_unstale":"2023-11-06T20:45:04.698546+0000","last_undegraded":"2023-11-06T20:45:04.698546+0000","last_fullsized":"2023-11-06T20:45:04.698546+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.682857+0000","last_change":"2023-11-06T20:45:04.189721+0000","last_active":"2023-11-06T20:45:04.682857+0000","last_peered":"2023-11-06T20:45:04.682857+0000","last_clean":"2023-11-06T20:45:04.682857+0000","last_became_active":"2023-11-06T20:45:04.189438+0000","last_became_peered":"2023-11-06T20:45:04.189438+0000","last_unstale":"2023-11-06T20:45:04.682857+0000","last_undegraded":"2023-11-06T20:45:04.682857+0000","last_fullsized":"2023-11-06T20:45:04.682857+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.697517+0000","last_change":"2023-11-06T20:45:04.690882+0000","last_active":"2023-11-06T20:45:04.697517+0000","last_peered":"2023-11-06T20:45:04.697517+0000","last_clean":"2023-11-06T20:45:04.697517+0000","last_became_active":"2023-11-06T20:45:04.690615+0000","last_became_peered":"2023-11-06T20:45:04.690615+0000","last_unstale":"2023-11-06T20:45:04.697517+0000","last_undegraded":"2023-11-06T20:45:04.697517+0000","last_fullsized":"2023-11-06T20:45:04.697517+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.701162+0000","last_change":"2023-11-06T20:45:04.198810+0000","last_active":"2023-11-06T20:45:04.701162+0000","last_peered":"2023-11-06T20:45:04.701162+0000","last_clean":"2023-11-06T20:45:04.701162+0000","last_became_active":"2023-11-06T20:45:04.198504+0000","last_became_peered":"2023-11-06T20:45:04.198504+0000","last_unstale":"2023-11-06T20:45:04.701162+0000","last_undegraded":"2023-11-06T20:45:04.701162+0000","last_fullsized":"2023-11-06T20:45:04.701162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.705898+0000","last_change":"2023-11-06T20:45:04.703877+0000","last_active":"2023-11-06T20:45:04.705898+0000","last_peered":"2023-11-06T20:45:04.705898+0000","last_clean":"2023-11-06T20:45:04.705898+0000","last_became_active":"2023-11-06T20:45:04.703375+0000","last_became_peered":"2023-11-06T20:45:04.703375+0000","last_unstale":"2023-11-06T20:45:04.705898+0000","last_undegraded":"2023-11-06T20:45:04.705898+0000","last_fullsized":"2023-11-06T20:45:04.705898+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.676925+0000","last_change":"2023-11-06T20:45:04.206537+0000","last_active":"2023-11-06T20:45:04.676925+0000","last_peered":"2023-11-06T20:45:04.676925+0000","last_clean":"2023-11-06T20:45:04.676925+0000","last_became_active":"2023-11-06T20:45:04.206260+0000","last_became_peered":"2023-11-06T20:45:04.206260+0000","last_unstale":"2023-11-06T20:45:04.676925+0000","last_undegraded":"2023-11-06T20:45:04.676925+0000","last_fullsized":"2023-11-06T20:45:04.676925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714350+0000","last_change":"2023-11-06T20:45:04.708217+0000","last_active":"2023-11-06T20:45:04.714350+0000","last_peered":"2023-11-06T20:45:04.714350+0000","last_clean":"2023-11-06T20:45:04.714350+0000","last_became_active":"2023-11-06T20:45:04.707923+0000","last_became_peered":"2023-11-06T20:45:04.707923+0000","last_unstale":"2023-11-06T20:45:04.714350+0000","last_undegraded":"2023-11-06T20:45:04.714350+0000","last_fullsized":"2023-11-06T20:45:04.714350+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698151+0000","last_change":"2023-11-06T20:45:04.207106+0000","last_active":"2023-11-06T20:45:04.698151+0000","last_peered":"2023-11-06T20:45:04.698151+0000","last_clean":"2023-11-06T20:45:04.698151+0000","last_became_active":"2023-11-06T20:45:04.206814+0000","last_became_peered":"2023-11-06T20:45:04.206814+0000","last_unstale":"2023-11-06T20:45:04.698151+0000","last_undegraded":"2023-11-06T20:45:04.698151+0000","last_fullsized":"2023-11-06T20:45:04.698151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.704307+0000","last_change":"2023-11-06T20:45:04.699458+0000","last_active":"2023-11-06T20:45:04.704307+0000","last_peered":"2023-11-06T20:45:04.704307+0000","last_clean":"2023-11-06T20:45:04.704307+0000","last_became_active":"2023-11-06T20:45:04.699133+0000","last_became_peered":"2023-11-06T20:45:04.699133+0000","last_unstale":"2023-11-06T20:45:04.704307+0000","last_undegraded":"2023-11-06T20:45:04.704307+0000","last_fullsized":"2023-11-06T20:45:04.704307+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677627+0000","last_change":"2023-11-06T20:45:04.213785+0000","last_active":"2023-11-06T20:45:04.677627+0000","last_peered":"2023-11-06T20:45:04.677627+0000","last_clean":"2023-11-06T20:45:04.677627+0000","last_became_active":"2023-11-06T20:45:04.213502+0000","last_became_peered":"2023-11-06T20:45:04.213502+0000","last_unstale":"2023-11-06T20:45:04.677627+0000","last_undegraded":"2023-11-06T20:45:04.677627+0000","last_fullsized":"2023-11-06T20:45:04.677627+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690833+0000","last_change":"2023-11-06T20:45:04.688924+0000","last_active":"2023-11-06T20:45:04.690833+0000","last_peered":"2023-11-06T20:45:04.690833+0000","last_clean":"2023-11-06T20:45:04.690833+0000","last_became_active":"2023-11-06T20:45:04.687906+0000","last_became_peered":"2023-11-06T20:45:04.687906+0000","last_unstale":"2023-11-06T20:45:04.690833+0000","last_undegraded":"2023-11-06T20:45:04.690833+0000","last_fullsized":"2023-11-06T20:45:04.690833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.682036+0000","last_change":"2023-11-06T20:45:04.212581+0000","last_active":"2023-11-06T20:45:04.682036+0000","last_peered":"2023-11-06T20:45:04.682036+0000","last_clean":"2023-11-06T20:45:04.682036+0000","last_became_active":"2023-11-06T20:45:04.211576+0000","last_became_peered":"2023-11-06T20:45:04.211576+0000","last_unstale":"2023-11-06T20:45:04.682036+0000","last_undegraded":"2023-11-06T20:45:04.682036+0000","last_fullsized":"2023-11-06T20:45:04.682036+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.682936+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:45:04.682936+0000","last_peered":"2023-11-06T20:45:04.682936+0000","last_clean":"2023-11-06T20:45:04.682936+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:45:04.682936+0000","last_undegraded":"2023-11-06T20:45:04.682936+0000","last_fullsized":"2023-11-06T20:45:04.682936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700094+0000","last_change":"2023-11-06T20:45:04.696479+0000","last_active":"2023-11-06T20:45:04.700094+0000","last_peered":"2023-11-06T20:45:04.700094+0000","last_clean":"2023-11-06T20:45:04.700094+0000","last_became_active":"2023-11-06T20:45:04.696147+0000","last_became_peered":"2023-11-06T20:45:04.696147+0000","last_unstale":"2023-11-06T20:45:04.700094+0000","last_undegraded":"2023-11-06T20:45:04.700094+0000","last_fullsized":"2023-11-06T20:45:04.700094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677513+0000","last_change":"2023-11-06T20:45:04.205346+0000","last_active":"2023-11-06T20:45:04.677513+0000","last_peered":"2023-11-06T20:45:04.677513+0000","last_clean":"2023-11-06T20:45:04.677513+0000","last_became_active":"2023-11-06T20:45:04.204784+0000","last_became_peered":"2023-11-06T20:45:04.204784+0000","last_unstale":"2023-11-06T20:45:04.677513+0000","last_undegraded":"2023-11-06T20:45:04.677513+0000","last_fullsized":"2023-11-06T20:45:04.677513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.676193+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:45:04.676193+0000","last_peered":"2023-11-06T20:45:04.676193+0000","last_clean":"2023-11-06T20:45:04.676193+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:45:04.676193+0000","last_undegraded":"2023-11-06T20:45:04.676193+0000","last_fullsized":"2023-11-06T20:45:04.676193+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.711997+0000","last_change":"2023-11-06T20:45:04.707748+0000","last_active":"2023-11-06T20:45:04.711997+0000","last_peered":"2023-11-06T20:45:04.711997+0000","last_clean":"2023-11-06T20:45:04.711997+0000","last_became_active":"2023-11-06T20:45:04.707342+0000","last_became_peered":"2023-11-06T20:45:04.707342+0000","last_unstale":"2023-11-06T20:45:04.711997+0000","last_undegraded":"2023-11-06T20:45:04.711997+0000","last_fullsized":"2023-11-06T20:45:04.711997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677874+0000","last_change":"2023-11-06T20:45:04.211464+0000","last_active":"2023-11-06T20:45:04.677874+0000","last_peered":"2023-11-06T20:45:04.677874+0000","last_clean":"2023-11-06T20:45:04.677874+0000","last_became_active":"2023-11-06T20:45:04.211164+0000","last_became_peered":"2023-11-06T20:45:04.211164+0000","last_unstale":"2023-11-06T20:45:04.677874+0000","last_undegraded":"2023-11-06T20:45:04.677874+0000","last_fullsized":"2023-11-06T20:45:04.677874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.675956+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:45:04.675956+0000","last_peered":"2023-11-06T20:45:04.675956+0000","last_clean":"2023-11-06T20:45:04.675956+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:45:04.675956+0000","last_undegraded":"2023-11-06T20:45:04.675956+0000","last_fullsized":"2023-11-06T20:45:04.675956+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.702155+0000","last_change":"2023-11-06T20:45:04.695633+0000","last_active":"2023-11-06T20:45:04.702155+0000","last_peered":"2023-11-06T20:45:04.702155+0000","last_clean":"2023-11-06T20:45:04.702155+0000","last_became_active":"2023-11-06T20:45:04.695375+0000","last_became_peered":"2023-11-06T20:45:04.695375+0000","last_unstale":"2023-11-06T20:45:04.702155+0000","last_undegraded":"2023-11-06T20:45:04.702155+0000","last_fullsized":"2023-11-06T20:45:04.702155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.693697+0000","last_change":"2023-11-06T20:45:04.202896+0000","last_active":"2023-11-06T20:45:04.693697+0000","last_peered":"2023-11-06T20:45:04.693697+0000","last_clean":"2023-11-06T20:45:04.693697+0000","last_became_active":"2023-11-06T20:45:04.202582+0000","last_became_peered":"2023-11-06T20:45:04.202582+0000","last_unstale":"2023-11-06T20:45:04.693697+0000","last_undegraded":"2023-11-06T20:45:04.693697+0000","last_fullsized":"2023-11-06T20:45:04.693697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677868+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:45:04.677868+0000","last_peered":"2023-11-06T20:45:04.677868+0000","last_clean":"2023-11-06T20:45:04.677868+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:45:04.677868+0000","last_undegraded":"2023-11-06T20:45:04.677868+0000","last_fullsized":"2023-11-06T20:45:04.677868+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696761+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:45:04.696761+0000","last_peered":"2023-11-06T20:45:04.696761+0000","last_clean":"2023-11-06T20:45:04.696761+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:45:04.696761+0000","last_undegraded":"2023-11-06T20:45:04.696761+0000","last_fullsized":"2023-11-06T20:45:04.696761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.695930+0000","last_change":"2023-11-06T20:45:04.690089+0000","last_active":"2023-11-06T20:45:04.695930+0000","last_peered":"2023-11-06T20:45:04.695930+0000","last_clean":"2023-11-06T20:45:04.695930+0000","last_became_active":"2023-11-06T20:45:04.689827+0000","last_became_peered":"2023-11-06T20:45:04.689827+0000","last_unstale":"2023-11-06T20:45:04.695930+0000","last_undegraded":"2023-11-06T20:45:04.695930+0000","last_fullsized":"2023-11-06T20:45:04.695930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.711425+0000","last_change":"2023-11-06T20:45:04.207979+0000","last_active":"2023-11-06T20:45:04.711425+0000","last_peered":"2023-11-06T20:45:04.711425+0000","last_clean":"2023-11-06T20:45:04.711425+0000","last_became_active":"2023-11-06T20:45:04.207746+0000","last_became_peered":"2023-11-06T20:45:04.207746+0000","last_unstale":"2023-11-06T20:45:04.711425+0000","last_undegraded":"2023-11-06T20:45:04.711425+0000","last_fullsized":"2023-11-06T20:45:04.711425+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.674309+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:45:04.674309+0000","last_peered":"2023-11-06T20:45:04.674309+0000","last_clean":"2023-11-06T20:45:04.674309+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:45:04.674309+0000","last_undegraded":"2023-11-06T20:45:04.674309+0000","last_fullsized":"2023-11-06T20:45:04.674309+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698734+0000","last_change":"2023-11-06T20:45:04.694142+0000","last_active":"2023-11-06T20:45:04.698734+0000","last_peered":"2023-11-06T20:45:04.698734+0000","last_clean":"2023-11-06T20:45:04.698734+0000","last_became_active":"2023-11-06T20:45:04.693797+0000","last_became_peered":"2023-11-06T20:45:04.693797+0000","last_unstale":"2023-11-06T20:45:04.698734+0000","last_undegraded":"2023-11-06T20:45:04.698734+0000","last_fullsized":"2023-11-06T20:45:04.698734+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.685443+0000","last_change":"2023-11-06T20:45:04.202320+0000","last_active":"2023-11-06T20:45:04.685443+0000","last_peered":"2023-11-06T20:45:04.685443+0000","last_clean":"2023-11-06T20:45:04.685443+0000","last_became_active":"2023-11-06T20:45:04.201999+0000","last_became_peered":"2023-11-06T20:45:04.201999+0000","last_unstale":"2023-11-06T20:45:04.685443+0000","last_undegraded":"2023-11-06T20:45:04.685443+0000","last_fullsized":"2023-11-06T20:45:04.685443+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.678282+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:45:04.678282+0000","last_peered":"2023-11-06T20:45:04.678282+0000","last_clean":"2023-11-06T20:45:04.678282+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:45:04.678282+0000","last_undegraded":"2023-11-06T20:45:04.678282+0000","last_fullsized":"2023-11-06T20:45:04.678282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.697448+0000","last_change":"2023-11-06T20:45:04.687776+0000","last_active":"2023-11-06T20:45:04.697448+0000","last_peered":"2023-11-06T20:45:04.697448+0000","last_clean":"2023-11-06T20:45:04.697448+0000","last_became_active":"2023-11-06T20:45:04.687513+0000","last_became_peered":"2023-11-06T20:45:04.687513+0000","last_unstale":"2023-11-06T20:45:04.697448+0000","last_undegraded":"2023-11-06T20:45:04.697448+0000","last_fullsized":"2023-11-06T20:45:04.697448+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.674829+0000","last_change":"2023-11-06T20:45:04.200502+0000","last_active":"2023-11-06T20:45:04.674829+0000","last_peered":"2023-11-06T20:45:04.674829+0000","last_clean":"2023-11-06T20:45:04.674829+0000","last_became_active":"2023-11-06T20:45:04.200132+0000","last_became_peered":"2023-11-06T20:45:04.200132+0000","last_unstale":"2023-11-06T20:45:04.674829+0000","last_undegraded":"2023-11-06T20:45:04.674829+0000","last_fullsized":"2023-11-06T20:45:04.674829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.683507+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:45:04.683507+0000","last_peered":"2023-11-06T20:45:04.683507+0000","last_clean":"2023-11-06T20:45:04.683507+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:45:04.683507+0000","last_undegraded":"2023-11-06T20:45:04.683507+0000","last_fullsized":"2023-11-06T20:45:04.683507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.710859+0000","last_change":"2023-11-06T20:45:04.700015+0000","last_active":"2023-11-06T20:45:04.710859+0000","last_peered":"2023-11-06T20:45:04.710859+0000","last_clean":"2023-11-06T20:45:04.710859+0000","last_became_active":"2023-11-06T20:45:04.699762+0000","last_became_peered":"2023-11-06T20:45:04.699762+0000","last_unstale":"2023-11-06T20:45:04.710859+0000","last_undegraded":"2023-11-06T20:45:04.710859+0000","last_fullsized":"2023-11-06T20:45:04.710859+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680983+0000","last_change":"2023-11-06T20:45:04.214975+0000","last_active":"2023-11-06T20:45:04.680983+0000","last_peered":"2023-11-06T20:45:04.680983+0000","last_clean":"2023-11-06T20:45:04.680983+0000","last_became_active":"2023-11-06T20:45:04.214482+0000","last_became_peered":"2023-11-06T20:45:04.214482+0000","last_unstale":"2023-11-06T20:45:04.680983+0000","last_undegraded":"2023-11-06T20:45:04.680983+0000","last_fullsized":"2023-11-06T20:45:04.680983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680415+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:45:04.680415+0000","last_peered":"2023-11-06T20:45:04.680415+0000","last_clean":"2023-11-06T20:45:04.680415+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:45:04.680415+0000","last_undegraded":"2023-11-06T20:45:04.680415+0000","last_fullsized":"2023-11-06T20:45:04.680415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":401408,"data_stored":166568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.61599999999999999}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.1749999999999998,"5min":5.1749999999999998,"15min":5.1749999999999998},"last":0.495},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.63800000000000001}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.73999999999999999}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.202,"5min":5.202,"15min":5.202},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.59499999999999997}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65300000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.77800000000000002}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":405504,"data_stored":167424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.623}]},{"osd":1,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.874},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.82499999999999996}]},{"osd":5,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.55200000000000005}]},{"osd":7,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":409600,"data_stored":167482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.42499999999999999}]},{"osd":2,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.439}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.38200000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.60499999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":405504,"data_stored":167074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55600000000000005}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.73599999999999999}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.747},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.80700000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":409600,"data_stored":167696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.38100000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.69299999999999995}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":401408,"data_stored":166588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.33100000000000002}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.313},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.435}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.39500000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.89300000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.84299999999999997}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.879},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.872}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":413696,"data_stored":167266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.442},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.505},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.69099999999999995}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.879}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.753},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.77300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":397312,"data_stored":166498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.495},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51200000000000001}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.64200000000000002}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.54100000000000004}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60499999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.52600000000000002}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.56200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:45:14.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:45:14.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.224+0000 7f4df5dff700 1 Processor -- start 2023-11-06T20:45:14.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.224+0000 7f4df5dff700 1 -- start start 2023-11-06T20:45:14.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.225+0000 7f4df5dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4df0063140 0x7f4df0063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.225+0000 7f4df5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4df005e1b0 0x7f4df00c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.225+0000 7f4df5dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df00c31f0 con 0x7f4df00c4490 2023-11-06T20:45:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.225+0000 7f4df5dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df006a0f0 con 0x7f4df0063140 2023-11-06T20:45:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.225+0000 7f4df5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00c2920 con 0x7f4df005e1b0 2023-11-06T20:45:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4def7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4df0063140 0x7f4df0063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4deffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4df00c4490 0x7f4df0154050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58010/0 (socket says 172.21.15.33:58010) 2023-11-06T20:45:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4deffff700 1 -- 172.21.15.33:0/1613311884 learned_addr learned my addr 172.21.15.33:0/1613311884 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4deeffd700 1 --2- 172.21.15.33:0/1613311884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4df005e1b0 0x7f4df00c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4dee7fc700 1 -- 172.21.15.33:0/1613311884 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741586475 0 0) 0x7f4df00c31f0 con 0x7f4df00c4490 2023-11-06T20:45:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4dee7fc700 1 -- 172.21.15.33:0/1613311884 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dd80033f0 con 0x7f4df00c4490 2023-11-06T20:45:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4def7fe700 1 -- 172.21.15.33:0/1613311884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4df005e1b0 msgr2=0x7f4df00c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4def7fe700 1 --2- 172.21.15.33:0/1613311884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4df005e1b0 0x7f4df00c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4def7fe700 1 -- 172.21.15.33:0/1613311884 >> v1:172.21.15.33:6789/0 conn(0x7f4df00c4490 legacy=0x7f4df0154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4def7fe700 1 -- 172.21.15.33:0/1613311884 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00c3b70 con 0x7f4df0063140 2023-11-06T20:45:14.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.226+0000 7f4def7fe700 1 --2- 172.21.15.33:0/1613311884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 0x7f4df0063520 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f4de0009760 tx=0x7f4de0009060).ready entity=mon.0 client_cookie=c000c00ae28037f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4dee7fc700 1 -- 172.21.15.33:0/1613311884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000d4a0 con 0x7f4df0063140 2023-11-06T20:45:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4dee7fc700 1 -- 172.21.15.33:0/1613311884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de000d600 con 0x7f4df0063140 2023-11-06T20:45:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4dee7fc700 1 -- 172.21.15.33:0/1613311884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000d980 con 0x7f4df0063140 2023-11-06T20:45:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4df5dff700 1 -- 172.21.15.33:0/1613311884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 msgr2=0x7f4df0063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4df5dff700 1 --2- 172.21.15.33:0/1613311884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 0x7f4df0063520 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f4de0009760 tx=0x7f4de0009060).stop 2023-11-06T20:45:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4df5dff700 1 -- 172.21.15.33:0/1613311884 shutdown_connections 2023-11-06T20:45:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4df5dff700 1 --2- 172.21.15.33:0/1613311884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 0x7f4df0063520 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4df5dff700 1 --2- 172.21.15.33:0/1613311884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4df005e1b0 0x7f4df00c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4df5dff700 1 -- 172.21.15.33:0/1613311884 >> 172.21.15.33:0/1613311884 conn(0x7f4df00c7d60 msgr2=0x7f4df00c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4df5dff700 1 -- 172.21.15.33:0/1613311884 shutdown_connections 2023-11-06T20:45:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.227+0000 7f4df5dff700 1 -- 172.21.15.33:0/1613311884 wait complete. 2023-11-06T20:45:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.229+0000 7f4df5dff700 1 Processor -- start 2023-11-06T20:45:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.230+0000 7f4df5dff700 1 -- start start 2023-11-06T20:45:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4df5dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 0x7f4df014d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4df5dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df00c4490 0x7f4df0158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4df5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4df0158370 0x7f4df0139270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4df5dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0155ef0 con 0x7f4df0063140 2023-11-06T20:45:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4df5dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0156a70 con 0x7f4df00c4490 2023-11-06T20:45:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4df5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0156140 con 0x7f4df0158370 2023-11-06T20:45:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4def7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 0x7f4df014d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4def7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 0x7f4df014d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36488/0 (socket says 172.21.15.33:36488) 2023-11-06T20:45:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4def7fe700 1 -- 172.21.15.33:0/762019226 learned_addr learned my addr 172.21.15.33:0/762019226 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4deffff700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4df0158370 0x7f4df0139270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.231+0000 7f4deeffd700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df00c4490 0x7f4df0158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4def7fe700 1 -- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4df0158370 msgr2=0x7f4df0139270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4def7fe700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4df0158370 0x7f4df0139270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4def7fe700 1 -- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df00c4490 msgr2=0x7f4df0158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4def7fe700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df00c4490 0x7f4df0158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4def7fe700 1 -- 172.21.15.33:0/762019226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de00092a0 con 0x7f4df0063140 2023-11-06T20:45:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4deeffd700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df00c4490 0x7f4df0158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4def7fe700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 0x7f4df014d0d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f4de0011fb0 tx=0x7f4de0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4dd77fe700 1 -- 172.21.15.33:0/762019226 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0000c90 con 0x7f4df0063140 2023-11-06T20:45:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4dd77fe700 1 -- 172.21.15.33:0/762019226 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de000d770 con 0x7f4df0063140 2023-11-06T20:45:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4df5dff700 1 -- 172.21.15.33:0/762019226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df0156420 con 0x7f4df0063140 2023-11-06T20:45:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4dd77fe700 1 -- 172.21.15.33:0/762019226 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de001d430 con 0x7f4df0063140 2023-11-06T20:45:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.232+0000 7f4df5dff700 1 -- 172.21.15.33:0/762019226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df014d5c0 con 0x7f4df0063140 2023-11-06T20:45:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.233+0000 7f4df5dff700 1 -- 172.21.15.33:0/762019226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df0064420 con 0x7f4df0063140 2023-11-06T20:45:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.234+0000 7f4dd77fe700 1 -- 172.21.15.33:0/762019226 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4de001d860 con 0x7f4df0063140 2023-11-06T20:45:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.235+0000 7f4dd77fe700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ddc05f150 0x7f4ddc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.235+0000 7f4dd77fe700 1 -- 172.21.15.33:0/762019226 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4de007ba40 con 0x7f4df0063140 2023-11-06T20:45:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.235+0000 7f4deeffd700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ddc05f150 0x7f4ddc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.236+0000 7f4deeffd700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ddc05f150 0x7f4ddc061610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f4de4006d80 tx=0x7f4de4008040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.240+0000 7f4dd77fe700 1 -- 172.21.15.33:0/762019226 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4de007de60 con 0x7f4df0063140 2023-11-06T20:45:14.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.415+0000 7f4df5dff700 1 -- 172.21.15.33:0/762019226 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4df006bfd0 con 0x7f4ddc05f150 2023-11-06T20:45:14.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.427+0000 7f4dd77fe700 1 -- 172.21.15.33:0/762019226 <== mgr.4103 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+360913 (secure 0 0 0) 0x7f4df006bfd0 con 0x7f4ddc05f150 2023-11-06T20:45:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:14.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 -- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ddc05f150 msgr2=0x7f4ddc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ddc05f150 0x7f4ddc061610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f4de4006d80 tx=0x7f4de4008040).stop 2023-11-06T20:45:14.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 -- 172.21.15.33:0/762019226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 msgr2=0x7f4df014d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 0x7f4df014d0d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f4de0011fb0 tx=0x7f4de0016000).stop 2023-11-06T20:45:14.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 -- 172.21.15.33:0/762019226 shutdown_connections 2023-11-06T20:45:14.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ddc05f150 0x7f4ddc061610 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4df0158370 0x7f4df0139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0063140 0x7f4df014d0d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 --2- 172.21.15.33:0/762019226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df00c4490 0x7f4df0158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 -- 172.21.15.33:0/762019226 >> 172.21.15.33:0/762019226 conn(0x7f4df00c7d60 msgr2=0x7f4df00c0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 -- 172.21.15.33:0/762019226 shutdown_connections 2023-11-06T20:45:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.431+0000 7f4df5dff700 1 -- 172.21.15.33:0/762019226 wait complete. 2023-11-06T20:45:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:45:14.447 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-11-06T20:45:12.824466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":218,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379936,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752594784,"statfs":{"total":773094113280,"available":770657058816,"internally_reserved":0,"allocated":3244032,"data_stored":1336596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4023,"num_objects":-7,"num_object_clones":0,"num_object_copies":-14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-26,"num_write_kb":-14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.326062"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700327+0000","last_change":"2023-11-06T20:45:04.698159+0000","last_active":"2023-11-06T20:45:04.700327+0000","last_peered":"2023-11-06T20:45:04.700327+0000","last_clean":"2023-11-06T20:45:04.700327+0000","last_became_active":"2023-11-06T20:45:04.695684+0000","last_became_peered":"2023-11-06T20:45:04.695684+0000","last_unstale":"2023-11-06T20:45:04.700327+0000","last_undegraded":"2023-11-06T20:45:04.700327+0000","last_fullsized":"2023-11-06T20:45:04.700327+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3f","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712002+0000","last_change":"2023-11-06T20:45:04.213243+0000","last_active":"2023-11-06T20:45:04.712002+0000","last_peered":"2023-11-06T20:45:04.712002+0000","last_clean":"2023-11-06T20:45:04.712002+0000","last_became_active":"2023-11-06T20:45:04.212975+0000","last_became_peered":"2023-11-06T20:45:04.212975+0000","last_unstale":"2023-11-06T20:45:04.712002+0000","last_undegraded":"2023-11-06T20:45:04.712002+0000","last_fullsized":"2023-11-06T20:45:04.712002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.705009+0000","last_change":"2023-11-06T20:45:04.700122+0000","last_active":"2023-11-06T20:45:04.705009+0000","last_peered":"2023-11-06T20:45:04.705009+0000","last_clean":"2023-11-06T20:45:04.705009+0000","last_became_active":"2023-11-06T20:45:04.699789+0000","last_became_peered":"2023-11-06T20:45:04.699789+0000","last_unstale":"2023-11-06T20:45:04.705009+0000","last_undegraded":"2023-11-06T20:45:04.705009+0000","last_fullsized":"2023-11-06T20:45:04.705009+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.683303+0000","last_change":"2023-11-06T20:45:04.205007+0000","last_active":"2023-11-06T20:45:04.683303+0000","last_peered":"2023-11-06T20:45:04.683303+0000","last_clean":"2023-11-06T20:45:04.683303+0000","last_became_active":"2023-11-06T20:45:04.204674+0000","last_became_peered":"2023-11-06T20:45:04.204674+0000","last_unstale":"2023-11-06T20:45:04.683303+0000","last_undegraded":"2023-11-06T20:45:04.683303+0000","last_fullsized":"2023-11-06T20:45:04.683303+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.688667+0000","last_change":"2023-11-06T20:45:04.682887+0000","last_active":"2023-11-06T20:45:04.688667+0000","last_peered":"2023-11-06T20:45:04.688667+0000","last_clean":"2023-11-06T20:45:04.688667+0000","last_became_active":"2023-11-06T20:45:04.682623+0000","last_became_peered":"2023-11-06T20:45:04.682623+0000","last_unstale":"2023-11-06T20:45:04.688667+0000","last_undegraded":"2023-11-06T20:45:04.688667+0000","last_fullsized":"2023-11-06T20:45:04.688667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699604+0000","last_change":"2023-11-06T20:45:04.196531+0000","last_active":"2023-11-06T20:45:04.699604+0000","last_peered":"2023-11-06T20:45:04.699604+0000","last_clean":"2023-11-06T20:45:04.699604+0000","last_became_active":"2023-11-06T20:45:04.194888+0000","last_became_peered":"2023-11-06T20:45:04.194888+0000","last_unstale":"2023-11-06T20:45:04.699604+0000","last_undegraded":"2023-11-06T20:45:04.699604+0000","last_fullsized":"2023-11-06T20:45:04.699604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.689481+0000","last_change":"2023-11-06T20:45:04.685895+0000","last_active":"2023-11-06T20:45:04.689481+0000","last_peered":"2023-11-06T20:45:04.689481+0000","last_clean":"2023-11-06T20:45:04.689481+0000","last_became_active":"2023-11-06T20:45:04.685580+0000","last_became_peered":"2023-11-06T20:45:04.685580+0000","last_unstale":"2023-11-06T20:45:04.689481+0000","last_undegraded":"2023-11-06T20:45:04.689481+0000","last_fullsized":"2023-11-06T20:45:04.689481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3c","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.706361+0000","last_change":"2023-11-06T20:45:04.203616+0000","last_active":"2023-11-06T20:45:04.706361+0000","last_peered":"2023-11-06T20:45:04.706361+0000","last_clean":"2023-11-06T20:45:04.706361+0000","last_became_active":"2023-11-06T20:45:04.202876+0000","last_became_peered":"2023-11-06T20:45:04.202876+0000","last_unstale":"2023-11-06T20:45:04.706361+0000","last_undegraded":"2023-11-06T20:45:04.706361+0000","last_fullsized":"2023-11-06T20:45:04.706361+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712228+0000","last_change":"2023-11-06T20:45:04.709013+0000","last_active":"2023-11-06T20:45:04.712228+0000","last_peered":"2023-11-06T20:45:04.712228+0000","last_clean":"2023-11-06T20:45:04.712228+0000","last_became_active":"2023-11-06T20:45:04.708683+0000","last_became_peered":"2023-11-06T20:45:04.708683+0000","last_unstale":"2023-11-06T20:45:04.712228+0000","last_undegraded":"2023-11-06T20:45:04.712228+0000","last_fullsized":"2023-11-06T20:45:04.712228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700181+0000","last_change":"2023-11-06T20:45:04.202634+0000","last_active":"2023-11-06T20:45:04.700181+0000","last_peered":"2023-11-06T20:45:04.700181+0000","last_clean":"2023-11-06T20:45:04.700181+0000","last_became_active":"2023-11-06T20:45:04.201207+0000","last_became_peered":"2023-11-06T20:45:04.201207+0000","last_unstale":"2023-11-06T20:45:04.700181+0000","last_undegraded":"2023-11-06T20:45:04.700181+0000","last_fullsized":"2023-11-06T20:45:04.700181+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703778+0000","last_change":"2023-11-06T20:45:04.699346+0000","last_active":"2023-11-06T20:45:04.703778+0000","last_peered":"2023-11-06T20:45:04.703778+0000","last_clean":"2023-11-06T20:45:04.703778+0000","last_became_active":"2023-11-06T20:45:04.698931+0000","last_became_peered":"2023-11-06T20:45:04.698931+0000","last_unstale":"2023-11-06T20:45:04.703778+0000","last_undegraded":"2023-11-06T20:45:04.703778+0000","last_fullsized":"2023-11-06T20:45:04.703778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3a","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.709968+0000","last_change":"2023-11-06T20:45:04.210337+0000","last_active":"2023-11-06T20:45:04.709968+0000","last_peered":"2023-11-06T20:45:04.709968+0000","last_clean":"2023-11-06T20:45:04.709968+0000","last_became_active":"2023-11-06T20:45:04.209950+0000","last_became_peered":"2023-11-06T20:45:04.209950+0000","last_unstale":"2023-11-06T20:45:04.709968+0000","last_undegraded":"2023-11-06T20:45:04.709968+0000","last_fullsized":"2023-11-06T20:45:04.709968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.691246+0000","last_change":"2023-11-06T20:45:04.688979+0000","last_active":"2023-11-06T20:45:04.691246+0000","last_peered":"2023-11-06T20:45:04.691246+0000","last_clean":"2023-11-06T20:45:04.691246+0000","last_became_active":"2023-11-06T20:45:04.688710+0000","last_became_peered":"2023-11-06T20:45:04.688710+0000","last_unstale":"2023-11-06T20:45:04.691246+0000","last_undegraded":"2023-11-06T20:45:04.691246+0000","last_fullsized":"2023-11-06T20:45:04.691246+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680925+0000","last_change":"2023-11-06T20:45:04.212407+0000","last_active":"2023-11-06T20:45:04.680925+0000","last_peered":"2023-11-06T20:45:04.680925+0000","last_clean":"2023-11-06T20:45:04.680925+0000","last_became_active":"2023-11-06T20:45:04.211888+0000","last_became_peered":"2023-11-06T20:45:04.211888+0000","last_unstale":"2023-11-06T20:45:04.680925+0000","last_undegraded":"2023-11-06T20:45:04.680925+0000","last_fullsized":"2023-11-06T20:45:04.680925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703585+0000","last_change":"2023-11-06T20:45:04.700775+0000","last_active":"2023-11-06T20:45:04.703585+0000","last_peered":"2023-11-06T20:45:04.703585+0000","last_clean":"2023-11-06T20:45:04.703585+0000","last_became_active":"2023-11-06T20:45:04.699051+0000","last_became_peered":"2023-11-06T20:45:04.699051+0000","last_unstale":"2023-11-06T20:45:04.703585+0000","last_undegraded":"2023-11-06T20:45:04.703585+0000","last_fullsized":"2023-11-06T20:45:04.703585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680573+0000","last_change":"2023-11-06T20:45:04.203081+0000","last_active":"2023-11-06T20:45:04.680573+0000","last_peered":"2023-11-06T20:45:04.680573+0000","last_clean":"2023-11-06T20:45:04.680573+0000","last_became_active":"2023-11-06T20:45:04.199530+0000","last_became_peered":"2023-11-06T20:45:04.199530+0000","last_unstale":"2023-11-06T20:45:04.680573+0000","last_undegraded":"2023-11-06T20:45:04.680573+0000","last_fullsized":"2023-11-06T20:45:04.680573+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690168+0000","last_change":"2023-11-06T20:45:04.686241+0000","last_active":"2023-11-06T20:45:04.690168+0000","last_peered":"2023-11-06T20:45:04.690168+0000","last_clean":"2023-11-06T20:45:04.690168+0000","last_became_active":"2023-11-06T20:45:04.685794+0000","last_became_peered":"2023-11-06T20:45:04.685794+0000","last_unstale":"2023-11-06T20:45:04.690168+0000","last_undegraded":"2023-11-06T20:45:04.690168+0000","last_fullsized":"2023-11-06T20:45:04.690168+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700837+0000","last_change":"2023-11-06T20:45:04.207865+0000","last_active":"2023-11-06T20:45:04.700837+0000","last_peered":"2023-11-06T20:45:04.700837+0000","last_clean":"2023-11-06T20:45:04.700837+0000","last_became_active":"2023-11-06T20:45:04.207584+0000","last_became_peered":"2023-11-06T20:45:04.207584+0000","last_unstale":"2023-11-06T20:45:04.700837+0000","last_undegraded":"2023-11-06T20:45:04.700837+0000","last_fullsized":"2023-11-06T20:45:04.700837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.711974+0000","last_change":"2023-11-06T20:45:04.707600+0000","last_active":"2023-11-06T20:45:04.711974+0000","last_peered":"2023-11-06T20:45:04.711974+0000","last_clean":"2023-11-06T20:45:04.711974+0000","last_became_active":"2023-11-06T20:45:04.707007+0000","last_became_peered":"2023-11-06T20:45:04.707007+0000","last_unstale":"2023-11-06T20:45:04.711974+0000","last_undegraded":"2023-11-06T20:45:04.711974+0000","last_fullsized":"2023-11-06T20:45:04.711974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.683774+0000","last_change":"2023-11-06T20:45:04.204211+0000","last_active":"2023-11-06T20:45:04.683774+0000","last_peered":"2023-11-06T20:45:04.683774+0000","last_clean":"2023-11-06T20:45:04.683774+0000","last_became_active":"2023-11-06T20:45:04.203955+0000","last_became_peered":"2023-11-06T20:45:04.203955+0000","last_unstale":"2023-11-06T20:45:04.683774+0000","last_undegraded":"2023-11-06T20:45:04.683774+0000","last_fullsized":"2023-11-06T20:45:04.683774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715752+0000","last_change":"2023-11-06T20:45:04.711145+0000","last_active":"2023-11-06T20:45:04.715752+0000","last_peered":"2023-11-06T20:45:04.715752+0000","last_clean":"2023-11-06T20:45:04.715752+0000","last_became_active":"2023-11-06T20:45:04.709234+0000","last_became_peered":"2023-11-06T20:45:04.709234+0000","last_unstale":"2023-11-06T20:45:04.715752+0000","last_undegraded":"2023-11-06T20:45:04.715752+0000","last_fullsized":"2023-11-06T20:45:04.715752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.678195+0000","last_change":"2023-11-06T20:45:04.203691+0000","last_active":"2023-11-06T20:45:04.678195+0000","last_peered":"2023-11-06T20:45:04.678195+0000","last_clean":"2023-11-06T20:45:04.678195+0000","last_became_active":"2023-11-06T20:45:04.203407+0000","last_became_peered":"2023-11-06T20:45:04.203407+0000","last_unstale":"2023-11-06T20:45:04.678195+0000","last_undegraded":"2023-11-06T20:45:04.678195+0000","last_fullsized":"2023-11-06T20:45:04.678195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.697860+0000","last_change":"2023-11-06T20:45:04.695397+0000","last_active":"2023-11-06T20:45:04.697860+0000","last_peered":"2023-11-06T20:45:04.697860+0000","last_clean":"2023-11-06T20:45:04.697860+0000","last_became_active":"2023-11-06T20:45:04.695135+0000","last_became_peered":"2023-11-06T20:45:04.695135+0000","last_unstale":"2023-11-06T20:45:04.697860+0000","last_undegraded":"2023-11-06T20:45:04.697860+0000","last_fullsized":"2023-11-06T20:45:04.697860+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715259+0000","last_change":"2023-11-06T20:45:04.206902+0000","last_active":"2023-11-06T20:45:04.715259+0000","last_peered":"2023-11-06T20:45:04.715259+0000","last_clean":"2023-11-06T20:45:04.715259+0000","last_became_active":"2023-11-06T20:45:04.206583+0000","last_became_peered":"2023-11-06T20:45:04.206583+0000","last_unstale":"2023-11-06T20:45:04.715259+0000","last_undegraded":"2023-11-06T20:45:04.715259+0000","last_fullsized":"2023-11-06T20:45:04.715259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712482+0000","last_change":"2023-11-06T20:45:04.707038+0000","last_active":"2023-11-06T20:45:04.712482+0000","last_peered":"2023-11-06T20:45:04.712482+0000","last_clean":"2023-11-06T20:45:04.712482+0000","last_became_active":"2023-11-06T20:45:04.704732+0000","last_became_peered":"2023-11-06T20:45:04.704732+0000","last_unstale":"2023-11-06T20:45:04.712482+0000","last_undegraded":"2023-11-06T20:45:04.712482+0000","last_fullsized":"2023-11-06T20:45:04.712482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.706215+0000","last_change":"2023-11-06T20:45:04.210698+0000","last_active":"2023-11-06T20:45:04.706215+0000","last_peered":"2023-11-06T20:45:04.706215+0000","last_clean":"2023-11-06T20:45:04.706215+0000","last_became_active":"2023-11-06T20:45:04.209602+0000","last_became_peered":"2023-11-06T20:45:04.209602+0000","last_unstale":"2023-11-06T20:45:04.706215+0000","last_undegraded":"2023-11-06T20:45:04.706215+0000","last_fullsized":"2023-11-06T20:45:04.706215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.693012+0000","last_change":"2023-11-06T20:45:04.685737+0000","last_active":"2023-11-06T20:45:04.693012+0000","last_peered":"2023-11-06T20:45:04.693012+0000","last_clean":"2023-11-06T20:45:04.693012+0000","last_became_active":"2023-11-06T20:45:04.685417+0000","last_became_peered":"2023-11-06T20:45:04.685417+0000","last_unstale":"2023-11-06T20:45:04.693012+0000","last_undegraded":"2023-11-06T20:45:04.693012+0000","last_fullsized":"2023-11-06T20:45:04.693012+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681583+0000","last_change":"2023-11-06T20:45:04.197005+0000","last_active":"2023-11-06T20:45:04.681583+0000","last_peered":"2023-11-06T20:45:04.681583+0000","last_clean":"2023-11-06T20:45:04.681583+0000","last_became_active":"2023-11-06T20:45:04.196738+0000","last_became_peered":"2023-11-06T20:45:04.196738+0000","last_unstale":"2023-11-06T20:45:04.681583+0000","last_undegraded":"2023-11-06T20:45:04.681583+0000","last_fullsized":"2023-11-06T20:45:04.681583+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699919+0000","last_change":"2023-11-06T20:45:04.698880+0000","last_active":"2023-11-06T20:45:04.699919+0000","last_peered":"2023-11-06T20:45:04.699919+0000","last_clean":"2023-11-06T20:45:04.699919+0000","last_became_active":"2023-11-06T20:45:04.697976+0000","last_became_peered":"2023-11-06T20:45:04.697976+0000","last_unstale":"2023-11-06T20:45:04.699919+0000","last_undegraded":"2023-11-06T20:45:04.699919+0000","last_fullsized":"2023-11-06T20:45:04.699919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.694686+0000","last_change":"2023-11-06T20:45:04.204378+0000","last_active":"2023-11-06T20:45:04.694686+0000","last_peered":"2023-11-06T20:45:04.694686+0000","last_clean":"2023-11-06T20:45:04.694686+0000","last_became_active":"2023-11-06T20:45:04.204086+0000","last_became_peered":"2023-11-06T20:45:04.204086+0000","last_unstale":"2023-11-06T20:45:04.694686+0000","last_undegraded":"2023-11-06T20:45:04.694686+0000","last_fullsized":"2023-11-06T20:45:04.694686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703867+0000","last_change":"2023-11-06T20:45:04.701051+0000","last_active":"2023-11-06T20:45:04.703867+0000","last_peered":"2023-11-06T20:45:04.703867+0000","last_clean":"2023-11-06T20:45:04.703867+0000","last_became_active":"2023-11-06T20:45:04.700684+0000","last_became_peered":"2023-11-06T20:45:04.700684+0000","last_unstale":"2023-11-06T20:45:04.703867+0000","last_undegraded":"2023-11-06T20:45:04.703867+0000","last_fullsized":"2023-11-06T20:45:04.703867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.30","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.720414+0000","last_change":"2023-11-06T20:45:04.199618+0000","last_active":"2023-11-06T20:45:04.720414+0000","last_peered":"2023-11-06T20:45:04.720414+0000","last_clean":"2023-11-06T20:45:04.720414+0000","last_became_active":"2023-11-06T20:45:04.198972+0000","last_became_peered":"2023-11-06T20:45:04.198972+0000","last_unstale":"2023-11-06T20:45:04.720414+0000","last_undegraded":"2023-11-06T20:45:04.720414+0000","last_fullsized":"2023-11-06T20:45:04.720414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714421+0000","last_change":"2023-11-06T20:45:04.709196+0000","last_active":"2023-11-06T20:45:04.714421+0000","last_peered":"2023-11-06T20:45:04.714421+0000","last_clean":"2023-11-06T20:45:04.714421+0000","last_became_active":"2023-11-06T20:45:04.708923+0000","last_became_peered":"2023-11-06T20:45:04.708923+0000","last_unstale":"2023-11-06T20:45:04.714421+0000","last_undegraded":"2023-11-06T20:45:04.714421+0000","last_fullsized":"2023-11-06T20:45:04.714421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680315+0000","last_change":"2023-11-06T20:45:04.198617+0000","last_active":"2023-11-06T20:45:04.680315+0000","last_peered":"2023-11-06T20:45:04.680315+0000","last_clean":"2023-11-06T20:45:04.680315+0000","last_became_active":"2023-11-06T20:45:04.196842+0000","last_became_peered":"2023-11-06T20:45:04.196842+0000","last_unstale":"2023-11-06T20:45:04.680315+0000","last_undegraded":"2023-11-06T20:45:04.680315+0000","last_fullsized":"2023-11-06T20:45:04.680315+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698492+0000","last_change":"2023-11-06T20:45:04.692466+0000","last_active":"2023-11-06T20:45:04.698492+0000","last_peered":"2023-11-06T20:45:04.698492+0000","last_clean":"2023-11-06T20:45:04.698492+0000","last_became_active":"2023-11-06T20:45:04.689566+0000","last_became_peered":"2023-11-06T20:45:04.689566+0000","last_unstale":"2023-11-06T20:45:04.698492+0000","last_undegraded":"2023-11-06T20:45:04.698492+0000","last_fullsized":"2023-11-06T20:45:04.698492+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677732+0000","last_change":"2023-11-06T20:45:04.212646+0000","last_active":"2023-11-06T20:45:04.677732+0000","last_peered":"2023-11-06T20:45:04.677732+0000","last_clean":"2023-11-06T20:45:04.677732+0000","last_became_active":"2023-11-06T20:45:04.212353+0000","last_became_peered":"2023-11-06T20:45:04.212353+0000","last_unstale":"2023-11-06T20:45:04.677732+0000","last_undegraded":"2023-11-06T20:45:04.677732+0000","last_fullsized":"2023-11-06T20:45:04.677732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.701538+0000","last_change":"2023-11-06T20:45:04.691702+0000","last_active":"2023-11-06T20:45:04.701538+0000","last_peered":"2023-11-06T20:45:04.701538+0000","last_clean":"2023-11-06T20:45:04.701538+0000","last_became_active":"2023-11-06T20:45:04.691407+0000","last_became_peered":"2023-11-06T20:45:04.691407+0000","last_unstale":"2023-11-06T20:45:04.701538+0000","last_undegraded":"2023-11-06T20:45:04.701538+0000","last_fullsized":"2023-11-06T20:45:04.701538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715714+0000","last_change":"2023-11-06T20:45:04.216348+0000","last_active":"2023-11-06T20:45:04.715714+0000","last_peered":"2023-11-06T20:45:04.715714+0000","last_clean":"2023-11-06T20:45:04.715714+0000","last_became_active":"2023-11-06T20:45:04.215851+0000","last_became_peered":"2023-11-06T20:45:04.215851+0000","last_unstale":"2023-11-06T20:45:04.715714+0000","last_undegraded":"2023-11-06T20:45:04.715714+0000","last_fullsized":"2023-11-06T20:45:04.715714+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713218+0000","last_change":"2023-11-06T20:45:04.709714+0000","last_active":"2023-11-06T20:45:04.713218+0000","last_peered":"2023-11-06T20:45:04.713218+0000","last_clean":"2023-11-06T20:45:04.713218+0000","last_became_active":"2023-11-06T20:45:04.709422+0000","last_became_peered":"2023-11-06T20:45:04.709422+0000","last_unstale":"2023-11-06T20:45:04.713218+0000","last_undegraded":"2023-11-06T20:45:04.713218+0000","last_fullsized":"2023-11-06T20:45:04.713218+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677364+0000","last_change":"2023-11-06T20:45:04.214892+0000","last_active":"2023-11-06T20:45:04.677364+0000","last_peered":"2023-11-06T20:45:04.677364+0000","last_clean":"2023-11-06T20:45:04.677364+0000","last_became_active":"2023-11-06T20:45:04.214629+0000","last_became_peered":"2023-11-06T20:45:04.214629+0000","last_unstale":"2023-11-06T20:45:04.677364+0000","last_undegraded":"2023-11-06T20:45:04.677364+0000","last_fullsized":"2023-11-06T20:45:04.677364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690595+0000","last_change":"2023-11-06T20:45:04.687633+0000","last_active":"2023-11-06T20:45:04.690595+0000","last_peered":"2023-11-06T20:45:04.690595+0000","last_clean":"2023-11-06T20:45:04.690595+0000","last_became_active":"2023-11-06T20:45:04.687278+0000","last_became_peered":"2023-11-06T20:45:04.687278+0000","last_unstale":"2023-11-06T20:45:04.690595+0000","last_undegraded":"2023-11-06T20:45:04.690595+0000","last_fullsized":"2023-11-06T20:45:04.690595+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2b","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715714+0000","last_change":"2023-11-06T20:45:04.213253+0000","last_active":"2023-11-06T20:45:04.715714+0000","last_peered":"2023-11-06T20:45:04.715714+0000","last_clean":"2023-11-06T20:45:04.715714+0000","last_became_active":"2023-11-06T20:45:04.212919+0000","last_became_peered":"2023-11-06T20:45:04.212919+0000","last_unstale":"2023-11-06T20:45:04.715714+0000","last_undegraded":"2023-11-06T20:45:04.715714+0000","last_fullsized":"2023-11-06T20:45:04.715714+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.694384+0000","last_change":"2023-11-06T20:45:04.686984+0000","last_active":"2023-11-06T20:45:04.694384+0000","last_peered":"2023-11-06T20:45:04.694384+0000","last_clean":"2023-11-06T20:45:04.694384+0000","last_became_active":"2023-11-06T20:45:04.686703+0000","last_became_peered":"2023-11-06T20:45:04.686703+0000","last_unstale":"2023-11-06T20:45:04.694384+0000","last_undegraded":"2023-11-06T20:45:04.694384+0000","last_fullsized":"2023-11-06T20:45:04.694384+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681053+0000","last_change":"2023-11-06T20:45:04.208631+0000","last_active":"2023-11-06T20:45:04.681053+0000","last_peered":"2023-11-06T20:45:04.681053+0000","last_clean":"2023-11-06T20:45:04.681053+0000","last_became_active":"2023-11-06T20:45:04.208313+0000","last_became_peered":"2023-11-06T20:45:04.208313+0000","last_unstale":"2023-11-06T20:45:04.681053+0000","last_undegraded":"2023-11-06T20:45:04.681053+0000","last_fullsized":"2023-11-06T20:45:04.681053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.708756+0000","last_change":"2023-11-06T20:45:04.692680+0000","last_active":"2023-11-06T20:45:04.708756+0000","last_peered":"2023-11-06T20:45:04.708756+0000","last_clean":"2023-11-06T20:45:04.708756+0000","last_became_active":"2023-11-06T20:45:04.692335+0000","last_became_peered":"2023-11-06T20:45:04.692335+0000","last_unstale":"2023-11-06T20:45:04.708756+0000","last_undegraded":"2023-11-06T20:45:04.708756+0000","last_fullsized":"2023-11-06T20:45:04.708756+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677027+0000","last_change":"2023-11-06T20:45:04.207900+0000","last_active":"2023-11-06T20:45:04.677027+0000","last_peered":"2023-11-06T20:45:04.677027+0000","last_clean":"2023-11-06T20:45:04.677027+0000","last_became_active":"2023-11-06T20:45:04.207618+0000","last_became_peered":"2023-11-06T20:45:04.207618+0000","last_unstale":"2023-11-06T20:45:04.677027+0000","last_undegraded":"2023-11-06T20:45:04.677027+0000","last_fullsized":"2023-11-06T20:45:04.677027+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.709909+0000","last_change":"2023-11-06T20:45:04.697841+0000","last_active":"2023-11-06T20:45:04.709909+0000","last_peered":"2023-11-06T20:45:04.709909+0000","last_clean":"2023-11-06T20:45:04.709909+0000","last_became_active":"2023-11-06T20:45:04.697442+0000","last_became_peered":"2023-11-06T20:45:04.697442+0000","last_unstale":"2023-11-06T20:45:04.709909+0000","last_undegraded":"2023-11-06T20:45:04.709909+0000","last_fullsized":"2023-11-06T20:45:04.709909+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680469+0000","last_change":"2023-11-06T20:45:04.208028+0000","last_active":"2023-11-06T20:45:04.680469+0000","last_peered":"2023-11-06T20:45:04.680469+0000","last_clean":"2023-11-06T20:45:04.680469+0000","last_became_active":"2023-11-06T20:45:04.207152+0000","last_became_peered":"2023-11-06T20:45:04.207152+0000","last_unstale":"2023-11-06T20:45:04.680469+0000","last_undegraded":"2023-11-06T20:45:04.680469+0000","last_fullsized":"2023-11-06T20:45:04.680469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714410+0000","last_change":"2023-11-06T20:45:04.710247+0000","last_active":"2023-11-06T20:45:04.714410+0000","last_peered":"2023-11-06T20:45:04.714410+0000","last_clean":"2023-11-06T20:45:04.714410+0000","last_became_active":"2023-11-06T20:45:04.709893+0000","last_became_peered":"2023-11-06T20:45:04.709893+0000","last_unstale":"2023-11-06T20:45:04.714410+0000","last_undegraded":"2023-11-06T20:45:04.714410+0000","last_fullsized":"2023-11-06T20:45:04.714410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.684789+0000","last_change":"2023-11-06T20:45:04.201589+0000","last_active":"2023-11-06T20:45:04.684789+0000","last_peered":"2023-11-06T20:45:04.684789+0000","last_clean":"2023-11-06T20:45:04.684789+0000","last_became_active":"2023-11-06T20:45:04.201293+0000","last_became_peered":"2023-11-06T20:45:04.201293+0000","last_unstale":"2023-11-06T20:45:04.684789+0000","last_undegraded":"2023-11-06T20:45:04.684789+0000","last_fullsized":"2023-11-06T20:45:04.684789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.709998+0000","last_change":"2023-11-06T20:45:04.699009+0000","last_active":"2023-11-06T20:45:04.709998+0000","last_peered":"2023-11-06T20:45:04.709998+0000","last_clean":"2023-11-06T20:45:04.709998+0000","last_became_active":"2023-11-06T20:45:04.698741+0000","last_became_peered":"2023-11-06T20:45:04.698741+0000","last_unstale":"2023-11-06T20:45:04.709998+0000","last_undegraded":"2023-11-06T20:45:04.709998+0000","last_fullsized":"2023-11-06T20:45:04.709998+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.26","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-06T20:45:06.403062+0000","last_change":"2023-11-06T20:45:04.216159+0000","last_active":"2023-11-06T20:45:06.403062+0000","last_peered":"2023-11-06T20:45:06.403062+0000","last_clean":"2023-11-06T20:45:06.403062+0000","last_became_active":"2023-11-06T20:45:04.215706+0000","last_became_peered":"2023-11-06T20:45:04.215706+0000","last_unstale":"2023-11-06T20:45:06.403062+0000","last_undegraded":"2023-11-06T20:45:06.403062+0000","last_fullsized":"2023-11-06T20:45:06.403062+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690338+0000","last_change":"2023-11-06T20:45:04.687607+0000","last_active":"2023-11-06T20:45:04.690338+0000","last_peered":"2023-11-06T20:45:04.690338+0000","last_clean":"2023-11-06T20:45:04.690338+0000","last_became_active":"2023-11-06T20:45:04.687273+0000","last_became_peered":"2023-11-06T20:45:04.687273+0000","last_unstale":"2023-11-06T20:45:04.690338+0000","last_undegraded":"2023-11-06T20:45:04.690338+0000","last_fullsized":"2023-11-06T20:45:04.690338+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.683094+0000","last_change":"2023-11-06T20:45:04.202691+0000","last_active":"2023-11-06T20:45:04.683094+0000","last_peered":"2023-11-06T20:45:04.683094+0000","last_clean":"2023-11-06T20:45:04.683094+0000","last_became_active":"2023-11-06T20:45:04.202362+0000","last_became_peered":"2023-11-06T20:45:04.202362+0000","last_unstale":"2023-11-06T20:45:04.683094+0000","last_undegraded":"2023-11-06T20:45:04.683094+0000","last_fullsized":"2023-11-06T20:45:04.683094+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700044+0000","last_change":"2023-11-06T20:45:04.693933+0000","last_active":"2023-11-06T20:45:04.700044+0000","last_peered":"2023-11-06T20:45:04.700044+0000","last_clean":"2023-11-06T20:45:04.700044+0000","last_became_active":"2023-11-06T20:45:04.693204+0000","last_became_peered":"2023-11-06T20:45:04.693204+0000","last_unstale":"2023-11-06T20:45:04.700044+0000","last_undegraded":"2023-11-06T20:45:04.700044+0000","last_fullsized":"2023-11-06T20:45:04.700044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681743+0000","last_change":"2023-11-06T20:45:04.213724+0000","last_active":"2023-11-06T20:45:04.681743+0000","last_peered":"2023-11-06T20:45:04.681743+0000","last_clean":"2023-11-06T20:45:04.681743+0000","last_became_active":"2023-11-06T20:45:04.211299+0000","last_became_peered":"2023-11-06T20:45:04.211299+0000","last_unstale":"2023-11-06T20:45:04.681743+0000","last_undegraded":"2023-11-06T20:45:04.681743+0000","last_fullsized":"2023-11-06T20:45:04.681743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700142+0000","last_change":"2023-11-06T20:45:04.697198+0000","last_active":"2023-11-06T20:45:04.700142+0000","last_peered":"2023-11-06T20:45:04.700142+0000","last_clean":"2023-11-06T20:45:04.700142+0000","last_became_active":"2023-11-06T20:45:04.696745+0000","last_became_peered":"2023-11-06T20:45:04.696745+0000","last_unstale":"2023-11-06T20:45:04.700142+0000","last_undegraded":"2023-11-06T20:45:04.700142+0000","last_fullsized":"2023-11-06T20:45:04.700142+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.686459+0000","last_change":"2023-11-06T20:45:04.205989+0000","last_active":"2023-11-06T20:45:04.686459+0000","last_peered":"2023-11-06T20:45:04.686459+0000","last_clean":"2023-11-06T20:45:04.686459+0000","last_became_active":"2023-11-06T20:45:04.205719+0000","last_became_peered":"2023-11-06T20:45:04.205719+0000","last_unstale":"2023-11-06T20:45:04.686459+0000","last_undegraded":"2023-11-06T20:45:04.686459+0000","last_fullsized":"2023-11-06T20:45:04.686459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.703797+0000","last_change":"2023-11-06T20:45:04.701216+0000","last_active":"2023-11-06T20:45:04.703797+0000","last_peered":"2023-11-06T20:45:04.703797+0000","last_clean":"2023-11-06T20:45:04.703797+0000","last_became_active":"2023-11-06T20:45:04.700956+0000","last_became_peered":"2023-11-06T20:45:04.700956+0000","last_unstale":"2023-11-06T20:45:04.703797+0000","last_undegraded":"2023-11-06T20:45:04.703797+0000","last_fullsized":"2023-11-06T20:45:04.703797+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.685550+0000","last_change":"2023-11-06T20:45:04.200776+0000","last_active":"2023-11-06T20:45:04.685550+0000","last_peered":"2023-11-06T20:45:04.685550+0000","last_clean":"2023-11-06T20:45:04.685550+0000","last_became_active":"2023-11-06T20:45:04.200461+0000","last_became_peered":"2023-11-06T20:45:04.200461+0000","last_unstale":"2023-11-06T20:45:04.685550+0000","last_undegraded":"2023-11-06T20:45:04.685550+0000","last_fullsized":"2023-11-06T20:45:04.685550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.702198+0000","last_change":"2023-11-06T20:45:04.695957+0000","last_active":"2023-11-06T20:45:04.702198+0000","last_peered":"2023-11-06T20:45:04.702198+0000","last_clean":"2023-11-06T20:45:04.702198+0000","last_became_active":"2023-11-06T20:45:04.695673+0000","last_became_peered":"2023-11-06T20:45:04.695673+0000","last_unstale":"2023-11-06T20:45:04.702198+0000","last_undegraded":"2023-11-06T20:45:04.702198+0000","last_fullsized":"2023-11-06T20:45:04.702198+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680966+0000","last_change":"2023-11-06T20:45:04.205274+0000","last_active":"2023-11-06T20:45:04.680966+0000","last_peered":"2023-11-06T20:45:04.680966+0000","last_clean":"2023-11-06T20:45:04.680966+0000","last_became_active":"2023-11-06T20:45:04.204948+0000","last_became_peered":"2023-11-06T20:45:04.204948+0000","last_unstale":"2023-11-06T20:45:04.680966+0000","last_undegraded":"2023-11-06T20:45:04.680966+0000","last_fullsized":"2023-11-06T20:45:04.680966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.702112+0000","last_change":"2023-11-06T20:45:04.695538+0000","last_active":"2023-11-06T20:45:04.702112+0000","last_peered":"2023-11-06T20:45:04.702112+0000","last_clean":"2023-11-06T20:45:04.702112+0000","last_became_active":"2023-11-06T20:45:04.695238+0000","last_became_peered":"2023-11-06T20:45:04.695238+0000","last_unstale":"2023-11-06T20:45:04.702112+0000","last_undegraded":"2023-11-06T20:45:04.702112+0000","last_fullsized":"2023-11-06T20:45:04.702112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.674214+0000","last_change":"2023-11-06T20:45:04.201871+0000","last_active":"2023-11-06T20:45:04.674214+0000","last_peered":"2023-11-06T20:45:04.674214+0000","last_clean":"2023-11-06T20:45:04.674214+0000","last_became_active":"2023-11-06T20:45:04.200787+0000","last_became_peered":"2023-11-06T20:45:04.200787+0000","last_unstale":"2023-11-06T20:45:04.674214+0000","last_undegraded":"2023-11-06T20:45:04.674214+0000","last_fullsized":"2023-11-06T20:45:04.674214+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698732+0000","last_change":"2023-11-06T20:45:04.688853+0000","last_active":"2023-11-06T20:45:04.698732+0000","last_peered":"2023-11-06T20:45:04.698732+0000","last_clean":"2023-11-06T20:45:04.698732+0000","last_became_active":"2023-11-06T20:45:04.688554+0000","last_became_peered":"2023-11-06T20:45:04.688554+0000","last_unstale":"2023-11-06T20:45:04.698732+0000","last_undegraded":"2023-11-06T20:45:04.698732+0000","last_fullsized":"2023-11-06T20:45:04.698732+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.699237+0000","last_change":"2023-11-06T20:45:04.199975+0000","last_active":"2023-11-06T20:45:04.699237+0000","last_peered":"2023-11-06T20:45:04.699237+0000","last_clean":"2023-11-06T20:45:04.699237+0000","last_became_active":"2023-11-06T20:45:04.195195+0000","last_became_peered":"2023-11-06T20:45:04.195195+0000","last_unstale":"2023-11-06T20:45:04.699237+0000","last_undegraded":"2023-11-06T20:45:04.699237+0000","last_fullsized":"2023-11-06T20:45:04.699237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.697553+0000","last_change":"2023-11-06T20:45:04.692362+0000","last_active":"2023-11-06T20:45:04.697553+0000","last_peered":"2023-11-06T20:45:04.697553+0000","last_clean":"2023-11-06T20:45:04.697553+0000","last_became_active":"2023-11-06T20:45:04.691882+0000","last_became_peered":"2023-11-06T20:45:04.691882+0000","last_unstale":"2023-11-06T20:45:04.697553+0000","last_undegraded":"2023-11-06T20:45:04.697553+0000","last_fullsized":"2023-11-06T20:45:04.697553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713067+0000","last_change":"2023-11-06T20:45:04.209292+0000","last_active":"2023-11-06T20:45:04.713067+0000","last_peered":"2023-11-06T20:45:04.713067+0000","last_clean":"2023-11-06T20:45:04.713067+0000","last_became_active":"2023-11-06T20:45:04.208357+0000","last_became_peered":"2023-11-06T20:45:04.208357+0000","last_unstale":"2023-11-06T20:45:04.713067+0000","last_undegraded":"2023-11-06T20:45:04.713067+0000","last_fullsized":"2023-11-06T20:45:04.713067+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.694110+0000","last_change":"2023-11-06T20:45:04.687618+0000","last_active":"2023-11-06T20:45:04.694110+0000","last_peered":"2023-11-06T20:45:04.694110+0000","last_clean":"2023-11-06T20:45:04.694110+0000","last_became_active":"2023-11-06T20:45:04.687325+0000","last_became_peered":"2023-11-06T20:45:04.687325+0000","last_unstale":"2023-11-06T20:45:04.694110+0000","last_undegraded":"2023-11-06T20:45:04.694110+0000","last_fullsized":"2023-11-06T20:45:04.694110+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700723+0000","last_change":"2023-11-06T20:45:04.211925+0000","last_active":"2023-11-06T20:45:04.700723+0000","last_peered":"2023-11-06T20:45:04.700723+0000","last_clean":"2023-11-06T20:45:04.700723+0000","last_became_active":"2023-11-06T20:45:04.211610+0000","last_became_peered":"2023-11-06T20:45:04.211610+0000","last_unstale":"2023-11-06T20:45:04.700723+0000","last_undegraded":"2023-11-06T20:45:04.700723+0000","last_fullsized":"2023-11-06T20:45:04.700723+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.709752+0000","last_change":"2023-11-06T20:45:04.706215+0000","last_active":"2023-11-06T20:45:04.709752+0000","last_peered":"2023-11-06T20:45:04.709752+0000","last_clean":"2023-11-06T20:45:04.709752+0000","last_became_active":"2023-11-06T20:45:04.705922+0000","last_became_peered":"2023-11-06T20:45:04.705922+0000","last_unstale":"2023-11-06T20:45:04.709752+0000","last_undegraded":"2023-11-06T20:45:04.709752+0000","last_fullsized":"2023-11-06T20:45:04.709752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.681802+0000","last_change":"2023-11-06T20:45:04.209004+0000","last_active":"2023-11-06T20:45:04.681802+0000","last_peered":"2023-11-06T20:45:04.681802+0000","last_clean":"2023-11-06T20:45:04.681802+0000","last_became_active":"2023-11-06T20:45:04.208295+0000","last_became_peered":"2023-11-06T20:45:04.208295+0000","last_unstale":"2023-11-06T20:45:04.681802+0000","last_undegraded":"2023-11-06T20:45:04.681802+0000","last_fullsized":"2023-11-06T20:45:04.681802+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690207+0000","last_change":"2023-11-06T20:45:04.686606+0000","last_active":"2023-11-06T20:45:04.690207+0000","last_peered":"2023-11-06T20:45:04.690207+0000","last_clean":"2023-11-06T20:45:04.690207+0000","last_became_active":"2023-11-06T20:45:04.686244+0000","last_became_peered":"2023-11-06T20:45:04.686244+0000","last_unstale":"2023-11-06T20:45:04.690207+0000","last_undegraded":"2023-11-06T20:45:04.690207+0000","last_fullsized":"2023-11-06T20:45:04.690207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680947+0000","last_change":"2023-11-06T20:45:04.200869+0000","last_active":"2023-11-06T20:45:04.680947+0000","last_peered":"2023-11-06T20:45:04.680947+0000","last_clean":"2023-11-06T20:45:04.680947+0000","last_became_active":"2023-11-06T20:45:04.200557+0000","last_became_peered":"2023-11-06T20:45:04.200557+0000","last_unstale":"2023-11-06T20:45:04.680947+0000","last_undegraded":"2023-11-06T20:45:04.680947+0000","last_fullsized":"2023-11-06T20:45:04.680947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.710657+0000","last_change":"2023-11-06T20:45:04.696624+0000","last_active":"2023-11-06T20:45:04.710657+0000","last_peered":"2023-11-06T20:45:04.710657+0000","last_clean":"2023-11-06T20:45:04.710657+0000","last_became_active":"2023-11-06T20:45:04.696081+0000","last_became_peered":"2023-11-06T20:45:04.696081+0000","last_unstale":"2023-11-06T20:45:04.710657+0000","last_undegraded":"2023-11-06T20:45:04.710657+0000","last_fullsized":"2023-11-06T20:45:04.710657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696708+0000","last_change":"2023-11-06T20:45:04.205999+0000","last_active":"2023-11-06T20:45:04.696708+0000","last_peered":"2023-11-06T20:45:04.696708+0000","last_clean":"2023-11-06T20:45:04.696708+0000","last_became_active":"2023-11-06T20:45:04.205244+0000","last_became_peered":"2023-11-06T20:45:04.205244+0000","last_unstale":"2023-11-06T20:45:04.696708+0000","last_undegraded":"2023-11-06T20:45:04.696708+0000","last_fullsized":"2023-11-06T20:45:04.696708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.704806+0000","last_change":"2023-11-06T20:45:04.702026+0000","last_active":"2023-11-06T20:45:04.704806+0000","last_peered":"2023-11-06T20:45:04.704806+0000","last_clean":"2023-11-06T20:45:04.704806+0000","last_became_active":"2023-11-06T20:45:04.701450+0000","last_became_peered":"2023-11-06T20:45:04.701450+0000","last_unstale":"2023-11-06T20:45:04.704806+0000","last_undegraded":"2023-11-06T20:45:04.704806+0000","last_fullsized":"2023-11-06T20:45:04.704806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680922+0000","last_change":"2023-11-06T20:45:04.202263+0000","last_active":"2023-11-06T20:45:04.680922+0000","last_peered":"2023-11-06T20:45:04.680922+0000","last_clean":"2023-11-06T20:45:04.680922+0000","last_became_active":"2023-11-06T20:45:04.201949+0000","last_became_peered":"2023-11-06T20:45:04.201949+0000","last_unstale":"2023-11-06T20:45:04.680922+0000","last_undegraded":"2023-11-06T20:45:04.680922+0000","last_fullsized":"2023-11-06T20:45:04.680922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.691959+0000","last_change":"2023-11-06T20:45:04.684016+0000","last_active":"2023-11-06T20:45:04.691959+0000","last_peered":"2023-11-06T20:45:04.691959+0000","last_clean":"2023-11-06T20:45:04.691959+0000","last_became_active":"2023-11-06T20:45:04.683735+0000","last_became_peered":"2023-11-06T20:45:04.683735+0000","last_unstale":"2023-11-06T20:45:04.691959+0000","last_undegraded":"2023-11-06T20:45:04.691959+0000","last_fullsized":"2023-11-06T20:45:04.691959+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696679+0000","last_change":"2023-11-06T20:45:04.194284+0000","last_active":"2023-11-06T20:45:04.696679+0000","last_peered":"2023-11-06T20:45:04.696679+0000","last_clean":"2023-11-06T20:45:04.696679+0000","last_became_active":"2023-11-06T20:45:04.193972+0000","last_became_peered":"2023-11-06T20:45:04.193972+0000","last_unstale":"2023-11-06T20:45:04.696679+0000","last_undegraded":"2023-11-06T20:45:04.696679+0000","last_fullsized":"2023-11-06T20:45:04.696679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700309+0000","last_change":"2023-11-06T20:45:04.697932+0000","last_active":"2023-11-06T20:45:04.700309+0000","last_peered":"2023-11-06T20:45:04.700309+0000","last_clean":"2023-11-06T20:45:04.700309+0000","last_became_active":"2023-11-06T20:45:04.695307+0000","last_became_peered":"2023-11-06T20:45:04.695307+0000","last_unstale":"2023-11-06T20:45:04.700309+0000","last_undegraded":"2023-11-06T20:45:04.700309+0000","last_fullsized":"2023-11-06T20:45:04.700309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.684880+0000","last_change":"2023-11-06T20:45:04.195073+0000","last_active":"2023-11-06T20:45:04.684880+0000","last_peered":"2023-11-06T20:45:04.684880+0000","last_clean":"2023-11-06T20:45:04.684880+0000","last_became_active":"2023-11-06T20:45:04.194548+0000","last_became_peered":"2023-11-06T20:45:04.194548+0000","last_unstale":"2023-11-06T20:45:04.684880+0000","last_undegraded":"2023-11-06T20:45:04.684880+0000","last_fullsized":"2023-11-06T20:45:04.684880+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700556+0000","last_change":"2023-11-06T20:45:04.699519+0000","last_active":"2023-11-06T20:45:04.700556+0000","last_peered":"2023-11-06T20:45:04.700556+0000","last_clean":"2023-11-06T20:45:04.700556+0000","last_became_active":"2023-11-06T20:45:04.699244+0000","last_became_peered":"2023-11-06T20:45:04.699244+0000","last_unstale":"2023-11-06T20:45:04.700556+0000","last_undegraded":"2023-11-06T20:45:04.700556+0000","last_fullsized":"2023-11-06T20:45:04.700556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677623+0000","last_change":"2023-11-06T20:45:04.193135+0000","last_active":"2023-11-06T20:45:04.677623+0000","last_peered":"2023-11-06T20:45:04.677623+0000","last_clean":"2023-11-06T20:45:04.677623+0000","last_became_active":"2023-11-06T20:45:04.192814+0000","last_became_peered":"2023-11-06T20:45:04.192814+0000","last_unstale":"2023-11-06T20:45:04.677623+0000","last_undegraded":"2023-11-06T20:45:04.677623+0000","last_fullsized":"2023-11-06T20:45:04.677623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.705951+0000","last_change":"2023-11-06T20:45:04.704375+0000","last_active":"2023-11-06T20:45:04.705951+0000","last_peered":"2023-11-06T20:45:04.705951+0000","last_clean":"2023-11-06T20:45:04.705951+0000","last_became_active":"2023-11-06T20:45:04.703514+0000","last_became_peered":"2023-11-06T20:45:04.703514+0000","last_unstale":"2023-11-06T20:45:04.705951+0000","last_undegraded":"2023-11-06T20:45:04.705951+0000","last_fullsized":"2023-11-06T20:45:04.705951+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713284+0000","last_change":"2023-11-06T20:45:04.209762+0000","last_active":"2023-11-06T20:45:04.713284+0000","last_peered":"2023-11-06T20:45:04.713284+0000","last_clean":"2023-11-06T20:45:04.713284+0000","last_became_active":"2023-11-06T20:45:04.209472+0000","last_became_peered":"2023-11-06T20:45:04.209472+0000","last_unstale":"2023-11-06T20:45:04.713284+0000","last_undegraded":"2023-11-06T20:45:04.713284+0000","last_fullsized":"2023-11-06T20:45:04.713284+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698113+0000","last_change":"2023-11-06T20:45:04.694154+0000","last_active":"2023-11-06T20:45:04.698113+0000","last_peered":"2023-11-06T20:45:04.698113+0000","last_clean":"2023-11-06T20:45:04.698113+0000","last_became_active":"2023-11-06T20:45:04.693824+0000","last_became_peered":"2023-11-06T20:45:04.693824+0000","last_unstale":"2023-11-06T20:45:04.698113+0000","last_undegraded":"2023-11-06T20:45:04.698113+0000","last_fullsized":"2023-11-06T20:45:04.698113+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.707587+0000","last_change":"2023-11-06T20:45:04.202088+0000","last_active":"2023-11-06T20:45:04.707587+0000","last_peered":"2023-11-06T20:45:04.707587+0000","last_clean":"2023-11-06T20:45:04.707587+0000","last_became_active":"2023-11-06T20:45:04.201823+0000","last_became_peered":"2023-11-06T20:45:04.201823+0000","last_unstale":"2023-11-06T20:45:04.707587+0000","last_undegraded":"2023-11-06T20:45:04.707587+0000","last_fullsized":"2023-11-06T20:45:04.707587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690572+0000","last_change":"2023-11-06T20:45:04.687056+0000","last_active":"2023-11-06T20:45:04.690572+0000","last_peered":"2023-11-06T20:45:04.690572+0000","last_clean":"2023-11-06T20:45:04.690572+0000","last_became_active":"2023-11-06T20:45:04.686668+0000","last_became_peered":"2023-11-06T20:45:04.686668+0000","last_unstale":"2023-11-06T20:45:04.690572+0000","last_undegraded":"2023-11-06T20:45:04.690572+0000","last_fullsized":"2023-11-06T20:45:04.690572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.702755+0000","last_change":"2023-11-06T20:45:04.212831+0000","last_active":"2023-11-06T20:45:04.702755+0000","last_peered":"2023-11-06T20:45:04.702755+0000","last_clean":"2023-11-06T20:45:04.702755+0000","last_became_active":"2023-11-06T20:45:04.212067+0000","last_became_peered":"2023-11-06T20:45:04.212067+0000","last_unstale":"2023-11-06T20:45:04.702755+0000","last_undegraded":"2023-11-06T20:45:04.702755+0000","last_fullsized":"2023-11-06T20:45:04.702755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.709918+0000","last_change":"2023-11-06T20:45:04.707036+0000","last_active":"2023-11-06T20:45:04.709918+0000","last_peered":"2023-11-06T20:45:04.709918+0000","last_clean":"2023-11-06T20:45:04.709918+0000","last_became_active":"2023-11-06T20:45:04.706688+0000","last_became_peered":"2023-11-06T20:45:04.706688+0000","last_unstale":"2023-11-06T20:45:04.709918+0000","last_undegraded":"2023-11-06T20:45:04.709918+0000","last_fullsized":"2023-11-06T20:45:04.709918+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.676833+0000","last_change":"2023-11-06T20:45:04.206935+0000","last_active":"2023-11-06T20:45:04.676833+0000","last_peered":"2023-11-06T20:45:04.676833+0000","last_clean":"2023-11-06T20:45:04.676833+0000","last_became_active":"2023-11-06T20:45:04.205748+0000","last_became_peered":"2023-11-06T20:45:04.205748+0000","last_unstale":"2023-11-06T20:45:04.676833+0000","last_undegraded":"2023-11-06T20:45:04.676833+0000","last_fullsized":"2023-11-06T20:45:04.676833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.710033+0000","last_change":"2023-11-06T20:45:04.698202+0000","last_active":"2023-11-06T20:45:04.710033+0000","last_peered":"2023-11-06T20:45:04.710033+0000","last_clean":"2023-11-06T20:45:04.710033+0000","last_became_active":"2023-11-06T20:45:04.688664+0000","last_became_peered":"2023-11-06T20:45:04.688664+0000","last_unstale":"2023-11-06T20:45:04.710033+0000","last_undegraded":"2023-11-06T20:45:04.710033+0000","last_fullsized":"2023-11-06T20:45:04.710033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.674795+0000","last_change":"2023-11-06T20:45:04.200818+0000","last_active":"2023-11-06T20:45:04.674795+0000","last_peered":"2023-11-06T20:45:04.674795+0000","last_clean":"2023-11-06T20:45:04.674795+0000","last_became_active":"2023-11-06T20:45:04.200522+0000","last_became_peered":"2023-11-06T20:45:04.200522+0000","last_unstale":"2023-11-06T20:45:04.674795+0000","last_undegraded":"2023-11-06T20:45:04.674795+0000","last_fullsized":"2023-11-06T20:45:04.674795+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.697088+0000","last_change":"2023-11-06T20:45:04.692936+0000","last_active":"2023-11-06T20:45:04.697088+0000","last_peered":"2023-11-06T20:45:04.697088+0000","last_clean":"2023-11-06T20:45:04.697088+0000","last_became_active":"2023-11-06T20:45:04.692621+0000","last_became_peered":"2023-11-06T20:45:04.692621+0000","last_unstale":"2023-11-06T20:45:04.697088+0000","last_undegraded":"2023-11-06T20:45:04.697088+0000","last_fullsized":"2023-11-06T20:45:04.697088+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714056+0000","last_change":"2023-11-06T20:45:04.199156+0000","last_active":"2023-11-06T20:45:04.714056+0000","last_peered":"2023-11-06T20:45:04.714056+0000","last_clean":"2023-11-06T20:45:04.714056+0000","last_became_active":"2023-11-06T20:45:04.198858+0000","last_became_peered":"2023-11-06T20:45:04.198858+0000","last_unstale":"2023-11-06T20:45:04.714056+0000","last_undegraded":"2023-11-06T20:45:04.714056+0000","last_fullsized":"2023-11-06T20:45:04.714056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690506+0000","last_change":"2023-11-06T20:45:04.689007+0000","last_active":"2023-11-06T20:45:04.690506+0000","last_peered":"2023-11-06T20:45:04.690506+0000","last_clean":"2023-11-06T20:45:04.690506+0000","last_became_active":"2023-11-06T20:45:04.688418+0000","last_became_peered":"2023-11-06T20:45:04.688418+0000","last_unstale":"2023-11-06T20:45:04.690506+0000","last_undegraded":"2023-11-06T20:45:04.690506+0000","last_fullsized":"2023-11-06T20:45:04.690506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.715339+0000","last_change":"2023-11-06T20:45:04.199860+0000","last_active":"2023-11-06T20:45:04.715339+0000","last_peered":"2023-11-06T20:45:04.715339+0000","last_clean":"2023-11-06T20:45:04.715339+0000","last_became_active":"2023-11-06T20:45:04.197409+0000","last_became_peered":"2023-11-06T20:45:04.197409+0000","last_unstale":"2023-11-06T20:45:04.715339+0000","last_undegraded":"2023-11-06T20:45:04.715339+0000","last_fullsized":"2023-11-06T20:45:04.715339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713635+0000","last_change":"2023-11-06T20:45:04.707405+0000","last_active":"2023-11-06T20:45:04.713635+0000","last_peered":"2023-11-06T20:45:04.713635+0000","last_clean":"2023-11-06T20:45:04.713635+0000","last_became_active":"2023-11-06T20:45:04.704832+0000","last_became_peered":"2023-11-06T20:45:04.704832+0000","last_unstale":"2023-11-06T20:45:04.713635+0000","last_undegraded":"2023-11-06T20:45:04.713635+0000","last_fullsized":"2023-11-06T20:45:04.713635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.712816+0000","last_change":"2023-11-06T20:45:04.198033+0000","last_active":"2023-11-06T20:45:04.712816+0000","last_peered":"2023-11-06T20:45:04.712816+0000","last_clean":"2023-11-06T20:45:04.712816+0000","last_became_active":"2023-11-06T20:45:04.197715+0000","last_became_peered":"2023-11-06T20:45:04.197715+0000","last_unstale":"2023-11-06T20:45:04.712816+0000","last_undegraded":"2023-11-06T20:45:04.712816+0000","last_fullsized":"2023-11-06T20:45:04.712816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.713102+0000","last_change":"2023-11-06T20:45:04.703423+0000","last_active":"2023-11-06T20:45:04.713102+0000","last_peered":"2023-11-06T20:45:04.713102+0000","last_clean":"2023-11-06T20:45:04.713102+0000","last_became_active":"2023-11-06T20:45:04.703108+0000","last_became_peered":"2023-11-06T20:45:04.703108+0000","last_unstale":"2023-11-06T20:45:04.713102+0000","last_undegraded":"2023-11-06T20:45:04.713102+0000","last_fullsized":"2023-11-06T20:45:04.713102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677408+0000","last_change":"2023-11-06T20:45:04.212009+0000","last_active":"2023-11-06T20:45:04.677408+0000","last_peered":"2023-11-06T20:45:04.677408+0000","last_clean":"2023-11-06T20:45:04.677408+0000","last_became_active":"2023-11-06T20:45:04.211204+0000","last_became_peered":"2023-11-06T20:45:04.211204+0000","last_unstale":"2023-11-06T20:45:04.677408+0000","last_undegraded":"2023-11-06T20:45:04.677408+0000","last_fullsized":"2023-11-06T20:45:04.677408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698546+0000","last_change":"2023-11-06T20:45:04.690373+0000","last_active":"2023-11-06T20:45:04.698546+0000","last_peered":"2023-11-06T20:45:04.698546+0000","last_clean":"2023-11-06T20:45:04.698546+0000","last_became_active":"2023-11-06T20:45:04.690070+0000","last_became_peered":"2023-11-06T20:45:04.690070+0000","last_unstale":"2023-11-06T20:45:04.698546+0000","last_undegraded":"2023-11-06T20:45:04.698546+0000","last_fullsized":"2023-11-06T20:45:04.698546+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.682857+0000","last_change":"2023-11-06T20:45:04.189721+0000","last_active":"2023-11-06T20:45:04.682857+0000","last_peered":"2023-11-06T20:45:04.682857+0000","last_clean":"2023-11-06T20:45:04.682857+0000","last_became_active":"2023-11-06T20:45:04.189438+0000","last_became_peered":"2023-11-06T20:45:04.189438+0000","last_unstale":"2023-11-06T20:45:04.682857+0000","last_undegraded":"2023-11-06T20:45:04.682857+0000","last_fullsized":"2023-11-06T20:45:04.682857+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.697517+0000","last_change":"2023-11-06T20:45:04.690882+0000","last_active":"2023-11-06T20:45:04.697517+0000","last_peered":"2023-11-06T20:45:04.697517+0000","last_clean":"2023-11-06T20:45:04.697517+0000","last_became_active":"2023-11-06T20:45:04.690615+0000","last_became_peered":"2023-11-06T20:45:04.690615+0000","last_unstale":"2023-11-06T20:45:04.697517+0000","last_undegraded":"2023-11-06T20:45:04.697517+0000","last_fullsized":"2023-11-06T20:45:04.697517+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.701162+0000","last_change":"2023-11-06T20:45:04.198810+0000","last_active":"2023-11-06T20:45:04.701162+0000","last_peered":"2023-11-06T20:45:04.701162+0000","last_clean":"2023-11-06T20:45:04.701162+0000","last_became_active":"2023-11-06T20:45:04.198504+0000","last_became_peered":"2023-11-06T20:45:04.198504+0000","last_unstale":"2023-11-06T20:45:04.701162+0000","last_undegraded":"2023-11-06T20:45:04.701162+0000","last_fullsized":"2023-11-06T20:45:04.701162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.705898+0000","last_change":"2023-11-06T20:45:04.703877+0000","last_active":"2023-11-06T20:45:04.705898+0000","last_peered":"2023-11-06T20:45:04.705898+0000","last_clean":"2023-11-06T20:45:04.705898+0000","last_became_active":"2023-11-06T20:45:04.703375+0000","last_became_peered":"2023-11-06T20:45:04.703375+0000","last_unstale":"2023-11-06T20:45:04.705898+0000","last_undegraded":"2023-11-06T20:45:04.705898+0000","last_fullsized":"2023-11-06T20:45:04.705898+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.676925+0000","last_change":"2023-11-06T20:45:04.206537+0000","last_active":"2023-11-06T20:45:04.676925+0000","last_peered":"2023-11-06T20:45:04.676925+0000","last_clean":"2023-11-06T20:45:04.676925+0000","last_became_active":"2023-11-06T20:45:04.206260+0000","last_became_peered":"2023-11-06T20:45:04.206260+0000","last_unstale":"2023-11-06T20:45:04.676925+0000","last_undegraded":"2023-11-06T20:45:04.676925+0000","last_fullsized":"2023-11-06T20:45:04.676925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.714350+0000","last_change":"2023-11-06T20:45:04.708217+0000","last_active":"2023-11-06T20:45:04.714350+0000","last_peered":"2023-11-06T20:45:04.714350+0000","last_clean":"2023-11-06T20:45:04.714350+0000","last_became_active":"2023-11-06T20:45:04.707923+0000","last_became_peered":"2023-11-06T20:45:04.707923+0000","last_unstale":"2023-11-06T20:45:04.714350+0000","last_undegraded":"2023-11-06T20:45:04.714350+0000","last_fullsized":"2023-11-06T20:45:04.714350+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698151+0000","last_change":"2023-11-06T20:45:04.207106+0000","last_active":"2023-11-06T20:45:04.698151+0000","last_peered":"2023-11-06T20:45:04.698151+0000","last_clean":"2023-11-06T20:45:04.698151+0000","last_became_active":"2023-11-06T20:45:04.206814+0000","last_became_peered":"2023-11-06T20:45:04.206814+0000","last_unstale":"2023-11-06T20:45:04.698151+0000","last_undegraded":"2023-11-06T20:45:04.698151+0000","last_fullsized":"2023-11-06T20:45:04.698151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.704307+0000","last_change":"2023-11-06T20:45:04.699458+0000","last_active":"2023-11-06T20:45:04.704307+0000","last_peered":"2023-11-06T20:45:04.704307+0000","last_clean":"2023-11-06T20:45:04.704307+0000","last_became_active":"2023-11-06T20:45:04.699133+0000","last_became_peered":"2023-11-06T20:45:04.699133+0000","last_unstale":"2023-11-06T20:45:04.704307+0000","last_undegraded":"2023-11-06T20:45:04.704307+0000","last_fullsized":"2023-11-06T20:45:04.704307+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677627+0000","last_change":"2023-11-06T20:45:04.213785+0000","last_active":"2023-11-06T20:45:04.677627+0000","last_peered":"2023-11-06T20:45:04.677627+0000","last_clean":"2023-11-06T20:45:04.677627+0000","last_became_active":"2023-11-06T20:45:04.213502+0000","last_became_peered":"2023-11-06T20:45:04.213502+0000","last_unstale":"2023-11-06T20:45:04.677627+0000","last_undegraded":"2023-11-06T20:45:04.677627+0000","last_fullsized":"2023-11-06T20:45:04.677627+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.690833+0000","last_change":"2023-11-06T20:45:04.688924+0000","last_active":"2023-11-06T20:45:04.690833+0000","last_peered":"2023-11-06T20:45:04.690833+0000","last_clean":"2023-11-06T20:45:04.690833+0000","last_became_active":"2023-11-06T20:45:04.687906+0000","last_became_peered":"2023-11-06T20:45:04.687906+0000","last_unstale":"2023-11-06T20:45:04.690833+0000","last_undegraded":"2023-11-06T20:45:04.690833+0000","last_fullsized":"2023-11-06T20:45:04.690833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.682036+0000","last_change":"2023-11-06T20:45:04.212581+0000","last_active":"2023-11-06T20:45:04.682036+0000","last_peered":"2023-11-06T20:45:04.682036+0000","last_clean":"2023-11-06T20:45:04.682036+0000","last_became_active":"2023-11-06T20:45:04.211576+0000","last_became_peered":"2023-11-06T20:45:04.211576+0000","last_unstale":"2023-11-06T20:45:04.682036+0000","last_undegraded":"2023-11-06T20:45:04.682036+0000","last_fullsized":"2023-11-06T20:45:04.682036+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.682936+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:45:04.682936+0000","last_peered":"2023-11-06T20:45:04.682936+0000","last_clean":"2023-11-06T20:45:04.682936+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:45:04.682936+0000","last_undegraded":"2023-11-06T20:45:04.682936+0000","last_fullsized":"2023-11-06T20:45:04.682936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.700094+0000","last_change":"2023-11-06T20:45:04.696479+0000","last_active":"2023-11-06T20:45:04.700094+0000","last_peered":"2023-11-06T20:45:04.700094+0000","last_clean":"2023-11-06T20:45:04.700094+0000","last_became_active":"2023-11-06T20:45:04.696147+0000","last_became_peered":"2023-11-06T20:45:04.696147+0000","last_unstale":"2023-11-06T20:45:04.700094+0000","last_undegraded":"2023-11-06T20:45:04.700094+0000","last_fullsized":"2023-11-06T20:45:04.700094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677513+0000","last_change":"2023-11-06T20:45:04.205346+0000","last_active":"2023-11-06T20:45:04.677513+0000","last_peered":"2023-11-06T20:45:04.677513+0000","last_clean":"2023-11-06T20:45:04.677513+0000","last_became_active":"2023-11-06T20:45:04.204784+0000","last_became_peered":"2023-11-06T20:45:04.204784+0000","last_unstale":"2023-11-06T20:45:04.677513+0000","last_undegraded":"2023-11-06T20:45:04.677513+0000","last_fullsized":"2023-11-06T20:45:04.677513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.676193+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:45:04.676193+0000","last_peered":"2023-11-06T20:45:04.676193+0000","last_clean":"2023-11-06T20:45:04.676193+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:45:04.676193+0000","last_undegraded":"2023-11-06T20:45:04.676193+0000","last_fullsized":"2023-11-06T20:45:04.676193+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.711997+0000","last_change":"2023-11-06T20:45:04.707748+0000","last_active":"2023-11-06T20:45:04.711997+0000","last_peered":"2023-11-06T20:45:04.711997+0000","last_clean":"2023-11-06T20:45:04.711997+0000","last_became_active":"2023-11-06T20:45:04.707342+0000","last_became_peered":"2023-11-06T20:45:04.707342+0000","last_unstale":"2023-11-06T20:45:04.711997+0000","last_undegraded":"2023-11-06T20:45:04.711997+0000","last_fullsized":"2023-11-06T20:45:04.711997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677874+0000","last_change":"2023-11-06T20:45:04.211464+0000","last_active":"2023-11-06T20:45:04.677874+0000","last_peered":"2023-11-06T20:45:04.677874+0000","last_clean":"2023-11-06T20:45:04.677874+0000","last_became_active":"2023-11-06T20:45:04.211164+0000","last_became_peered":"2023-11-06T20:45:04.211164+0000","last_unstale":"2023-11-06T20:45:04.677874+0000","last_undegraded":"2023-11-06T20:45:04.677874+0000","last_fullsized":"2023-11-06T20:45:04.677874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.675956+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:45:04.675956+0000","last_peered":"2023-11-06T20:45:04.675956+0000","last_clean":"2023-11-06T20:45:04.675956+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:45:04.675956+0000","last_undegraded":"2023-11-06T20:45:04.675956+0000","last_fullsized":"2023-11-06T20:45:04.675956+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.702155+0000","last_change":"2023-11-06T20:45:04.695633+0000","last_active":"2023-11-06T20:45:04.702155+0000","last_peered":"2023-11-06T20:45:04.702155+0000","last_clean":"2023-11-06T20:45:04.702155+0000","last_became_active":"2023-11-06T20:45:04.695375+0000","last_became_peered":"2023-11-06T20:45:04.695375+0000","last_unstale":"2023-11-06T20:45:04.702155+0000","last_undegraded":"2023-11-06T20:45:04.702155+0000","last_fullsized":"2023-11-06T20:45:04.702155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.693697+0000","last_change":"2023-11-06T20:45:04.202896+0000","last_active":"2023-11-06T20:45:04.693697+0000","last_peered":"2023-11-06T20:45:04.693697+0000","last_clean":"2023-11-06T20:45:04.693697+0000","last_became_active":"2023-11-06T20:45:04.202582+0000","last_became_peered":"2023-11-06T20:45:04.202582+0000","last_unstale":"2023-11-06T20:45:04.693697+0000","last_undegraded":"2023-11-06T20:45:04.693697+0000","last_fullsized":"2023-11-06T20:45:04.693697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.677868+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:45:04.677868+0000","last_peered":"2023-11-06T20:45:04.677868+0000","last_clean":"2023-11-06T20:45:04.677868+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:45:04.677868+0000","last_undegraded":"2023-11-06T20:45:04.677868+0000","last_fullsized":"2023-11-06T20:45:04.677868+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.696761+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:45:04.696761+0000","last_peered":"2023-11-06T20:45:04.696761+0000","last_clean":"2023-11-06T20:45:04.696761+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:45:04.696761+0000","last_undegraded":"2023-11-06T20:45:04.696761+0000","last_fullsized":"2023-11-06T20:45:04.696761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.695930+0000","last_change":"2023-11-06T20:45:04.690089+0000","last_active":"2023-11-06T20:45:04.695930+0000","last_peered":"2023-11-06T20:45:04.695930+0000","last_clean":"2023-11-06T20:45:04.695930+0000","last_became_active":"2023-11-06T20:45:04.689827+0000","last_became_peered":"2023-11-06T20:45:04.689827+0000","last_unstale":"2023-11-06T20:45:04.695930+0000","last_undegraded":"2023-11-06T20:45:04.695930+0000","last_fullsized":"2023-11-06T20:45:04.695930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.711425+0000","last_change":"2023-11-06T20:45:04.207979+0000","last_active":"2023-11-06T20:45:04.711425+0000","last_peered":"2023-11-06T20:45:04.711425+0000","last_clean":"2023-11-06T20:45:04.711425+0000","last_became_active":"2023-11-06T20:45:04.207746+0000","last_became_peered":"2023-11-06T20:45:04.207746+0000","last_unstale":"2023-11-06T20:45:04.711425+0000","last_undegraded":"2023-11-06T20:45:04.711425+0000","last_fullsized":"2023-11-06T20:45:04.711425+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.674309+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:45:04.674309+0000","last_peered":"2023-11-06T20:45:04.674309+0000","last_clean":"2023-11-06T20:45:04.674309+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:45:04.674309+0000","last_undegraded":"2023-11-06T20:45:04.674309+0000","last_fullsized":"2023-11-06T20:45:04.674309+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.698734+0000","last_change":"2023-11-06T20:45:04.694142+0000","last_active":"2023-11-06T20:45:04.698734+0000","last_peered":"2023-11-06T20:45:04.698734+0000","last_clean":"2023-11-06T20:45:04.698734+0000","last_became_active":"2023-11-06T20:45:04.693797+0000","last_became_peered":"2023-11-06T20:45:04.693797+0000","last_unstale":"2023-11-06T20:45:04.698734+0000","last_undegraded":"2023-11-06T20:45:04.698734+0000","last_fullsized":"2023-11-06T20:45:04.698734+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.685443+0000","last_change":"2023-11-06T20:45:04.202320+0000","last_active":"2023-11-06T20:45:04.685443+0000","last_peered":"2023-11-06T20:45:04.685443+0000","last_clean":"2023-11-06T20:45:04.685443+0000","last_became_active":"2023-11-06T20:45:04.201999+0000","last_became_peered":"2023-11-06T20:45:04.201999+0000","last_unstale":"2023-11-06T20:45:04.685443+0000","last_undegraded":"2023-11-06T20:45:04.685443+0000","last_fullsized":"2023-11-06T20:45:04.685443+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.678282+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:45:04.678282+0000","last_peered":"2023-11-06T20:45:04.678282+0000","last_clean":"2023-11-06T20:45:04.678282+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:45:04.678282+0000","last_undegraded":"2023-11-06T20:45:04.678282+0000","last_fullsized":"2023-11-06T20:45:04.678282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.697448+0000","last_change":"2023-11-06T20:45:04.687776+0000","last_active":"2023-11-06T20:45:04.697448+0000","last_peered":"2023-11-06T20:45:04.697448+0000","last_clean":"2023-11-06T20:45:04.697448+0000","last_became_active":"2023-11-06T20:45:04.687513+0000","last_became_peered":"2023-11-06T20:45:04.687513+0000","last_unstale":"2023-11-06T20:45:04.697448+0000","last_undegraded":"2023-11-06T20:45:04.697448+0000","last_fullsized":"2023-11-06T20:45:04.697448+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.674829+0000","last_change":"2023-11-06T20:45:04.200502+0000","last_active":"2023-11-06T20:45:04.674829+0000","last_peered":"2023-11-06T20:45:04.674829+0000","last_clean":"2023-11-06T20:45:04.674829+0000","last_became_active":"2023-11-06T20:45:04.200132+0000","last_became_peered":"2023-11-06T20:45:04.200132+0000","last_unstale":"2023-11-06T20:45:04.674829+0000","last_undegraded":"2023-11-06T20:45:04.674829+0000","last_fullsized":"2023-11-06T20:45:04.674829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.683507+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:45:04.683507+0000","last_peered":"2023-11-06T20:45:04.683507+0000","last_clean":"2023-11-06T20:45:04.683507+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:45:04.683507+0000","last_undegraded":"2023-11-06T20:45:04.683507+0000","last_fullsized":"2023-11-06T20:45:04.683507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.710859+0000","last_change":"2023-11-06T20:45:04.700015+0000","last_active":"2023-11-06T20:45:04.710859+0000","last_peered":"2023-11-06T20:45:04.710859+0000","last_clean":"2023-11-06T20:45:04.710859+0000","last_became_active":"2023-11-06T20:45:04.699762+0000","last_became_peered":"2023-11-06T20:45:04.699762+0000","last_unstale":"2023-11-06T20:45:04.710859+0000","last_undegraded":"2023-11-06T20:45:04.710859+0000","last_fullsized":"2023-11-06T20:45:04.710859+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:04.168407+0000","last_clean_scrub_stamp":"2023-11-06T20:45:04.168407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680983+0000","last_change":"2023-11-06T20:45:04.214975+0000","last_active":"2023-11-06T20:45:04.680983+0000","last_peered":"2023-11-06T20:45:04.680983+0000","last_clean":"2023-11-06T20:45:04.680983+0000","last_became_active":"2023-11-06T20:45:04.214482+0000","last_became_peered":"2023-11-06T20:45:04.214482+0000","last_unstale":"2023-11-06T20:45:04.680983+0000","last_undegraded":"2023-11-06T20:45:04.680983+0000","last_fullsized":"2023-11-06T20:45:04.680983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:45:03.162825+0000","last_clean_scrub_stamp":"2023-11-06T20:45:03.162825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-06T20:45:04.680415+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:45:04.680415+0000","last_peered":"2023-11-06T20:45:04.680415+0000","last_clean":"2023-11-06T20:45:04.680415+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:45:04.680415+0000","last_undegraded":"2023-11-06T20:45:04.680415+0000","last_fullsized":"2023-11-06T20:45:04.680415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":401408,"data_stored":166568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.61599999999999999}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.1749999999999998,"5min":5.1749999999999998,"15min":5.1749999999999998},"last":0.495},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.63800000000000001}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.73999999999999999}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.202,"5min":5.202,"15min":5.202},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.59499999999999997}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65300000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.77800000000000002}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":405504,"data_stored":167424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.623}]},{"osd":1,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.874},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.82499999999999996}]},{"osd":5,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.55200000000000005}]},{"osd":7,"last update":"Mon Nov 6 20:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":409600,"data_stored":167482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.42499999999999999}]},{"osd":2,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.439}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.38200000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.60499999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":405504,"data_stored":167074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55600000000000005}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.73599999999999999}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.747},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.80700000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":409600,"data_stored":167696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.38100000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.69299999999999995}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":401408,"data_stored":166588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.33100000000000002}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.313},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.435}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.39500000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.89300000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.84299999999999997}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.879},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.872}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":413696,"data_stored":167266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.442},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.505},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.69099999999999995}]},{"osd":3,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63}]},{"osd":5,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.879}]},{"osd":7,"last update":"Mon Nov 6 20:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.753},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.77300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":397312,"data_stored":166498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.495},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51200000000000001}]},{"osd":1,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.64200000000000002}]},{"osd":2,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.54100000000000004}]},{"osd":4,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60499999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.52600000000000002}]},{"osd":7,"last update":"Mon Nov 6 20:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.56200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:45:14.455 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T20:45:14.456 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T20:45:14.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.691+0000 7f9c01dff700 1 Processor -- start 2023-11-06T20:45:14.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.691+0000 7f9c01dff700 1 -- start start 2023-11-06T20:45:14.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.693+0000 7f9c01dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9bfc1544f0 0x7f9bfc0c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.693+0000 7f9c01dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9bfc0c0c40 0x7f9bfc0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.693+0000 7f9c01dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc1556d0 con 0x7f9bfc064bb0 2023-11-06T20:45:14.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.693+0000 7f9c01dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc1548d0 con 0x7f9bfc1544f0 2023-11-06T20:45:14.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.693+0000 7f9c01dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc065040 con 0x7f9bfc0c0c40 2023-11-06T20:45:14.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.693+0000 7f9bfb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9bfc1544f0 0x7f9bfc0c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.693+0000 7f9bfb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9bfc1544f0 0x7f9bfc0c0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36500/0 (socket says 172.21.15.33:36500) 2023-11-06T20:45:14.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.693+0000 7f9bfb7fe700 1 -- 172.21.15.33:0/2183411987 learned_addr learned my addr 172.21.15.33:0/2183411987 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:14.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.693+0000 7f9bfaffd700 1 --2- 172.21.15.33:0/2183411987 >> v2:172.21.15.69:3300/0 conn(0x7f9bfc0c0c40 0x7f9bfc0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.694+0000 7f9bfb7fe700 1 -- 172.21.15.33:0/2183411987 >> v2:172.21.15.69:3300/0 conn(0x7f9bfc0c0c40 msgr2=0x7f9bfc0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:14.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.694+0000 7f9bfb7fe700 1 --2- 172.21.15.33:0/2183411987 >> v2:172.21.15.69:3300/0 conn(0x7f9bfc0c0c40 0x7f9bfc0646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.694+0000 7f9bfb7fe700 1 -- 172.21.15.33:0/2183411987 >> v1:172.21.15.33:6789/0 conn(0x7f9bfc064bb0 legacy=0x7f9bfc157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.694+0000 7f9bfb7fe700 1 -- 172.21.15.33:0/2183411987 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc154ac0 con 0x7f9bfc1544f0 2023-11-06T20:45:14.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.694+0000 7f9bfaffd700 1 --2- 172.21.15.33:0/2183411987 >> v2:172.21.15.69:3300/0 conn(0x7f9bfc0c0c40 0x7f9bfc0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:14.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.694+0000 7f9bfa7fc700 1 -- 172.21.15.33:0/2183411987 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192709373 0 0) 0x7f9bfc1556d0 con 0x7f9bfc064bb0 2023-11-06T20:45:14.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.694+0000 7f9bfb7fe700 1 --2- 172.21.15.33:0/2183411987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 0x7f9bfc0c0750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f9bfc0adab0 tx=0x7f9bec0090f0).ready entity=mon.0 client_cookie=394f2f77162e5b7b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:14.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.694+0000 7f9bfa7fc700 1 -- 172.21.15.33:0/2183411987 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00b600 con 0x7f9bfc1544f0 2023-11-06T20:45:14.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.694+0000 7f9bfa7fc700 1 -- 172.21.15.33:0/2183411987 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec00b760 con 0x7f9bfc1544f0 2023-11-06T20:45:14.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.694+0000 7f9bfa7fc700 1 -- 172.21.15.33:0/2183411987 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00bae0 con 0x7f9bfc1544f0 2023-11-06T20:45:14.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.695+0000 7f9c01dff700 1 -- 172.21.15.33:0/2183411987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 msgr2=0x7f9bfc0c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.695+0000 7f9c01dff700 1 --2- 172.21.15.33:0/2183411987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 0x7f9bfc0c0750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f9bfc0adab0 tx=0x7f9bec0090f0).stop 2023-11-06T20:45:14.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.695+0000 7f9c01dff700 1 -- 172.21.15.33:0/2183411987 shutdown_connections 2023-11-06T20:45:14.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.695+0000 7f9c01dff700 1 --2- 172.21.15.33:0/2183411987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 0x7f9bfc0c0750 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.695+0000 7f9c01dff700 1 --2- 172.21.15.33:0/2183411987 >> v2:172.21.15.69:3300/0 conn(0x7f9bfc0c0c40 0x7f9bfc0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.695+0000 7f9c01dff700 1 -- 172.21.15.33:0/2183411987 >> 172.21.15.33:0/2183411987 conn(0x7f9bfc0cb900 msgr2=0x7f9bfc0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:14.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.695+0000 7f9c01dff700 1 -- 172.21.15.33:0/2183411987 shutdown_connections 2023-11-06T20:45:14.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.695+0000 7f9c01dff700 1 -- 172.21.15.33:0/2183411987 wait complete. 2023-11-06T20:45:14.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.696+0000 7f9c01dff700 1 Processor -- start 2023-11-06T20:45:14.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.697+0000 7f9c01dff700 1 -- start start 2023-11-06T20:45:14.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9c01dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bfc064bb0 0x7f9bfc147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9c01dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bfc0c0c40 0x7f9bfc0c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9c01dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 0x7f9bfc0c4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9c01dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c44f0 con 0x7f9bfc1544f0 2023-11-06T20:45:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9c01dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0addd0 con 0x7f9bfc0c0c40 2023-11-06T20:45:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9c01dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c2fa0 con 0x7f9bfc064bb0 2023-11-06T20:45:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9bfbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 0x7f9bfc0c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9bfbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 0x7f9bfc0c4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36502/0 (socket says 172.21.15.33:36502) 2023-11-06T20:45:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9bfbfff700 1 -- 172.21.15.33:0/2965826664 learned_addr learned my addr 172.21.15.33:0/2965826664 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9bfb7fe700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bfc064bb0 0x7f9bfc147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9bfaffd700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bfc0c0c40 0x7f9bfc0c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9bfbfff700 1 -- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bfc064bb0 msgr2=0x7f9bfc147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9bfbfff700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bfc064bb0 0x7f9bfc147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9bfbfff700 1 -- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bfc0c0c40 msgr2=0x7f9bfc0c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9bfbfff700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bfc0c0c40 0x7f9bfc0c3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.698+0000 7f9bfbfff700 1 -- 172.21.15.33:0/2965826664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec009370 con 0x7f9bfc1544f0 2023-11-06T20:45:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.699+0000 7f9bfbfff700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 0x7f9bfc0c4000 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f9bf400c760 tx=0x7f9bf4007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.699+0000 7f9be37fe700 1 -- 172.21.15.33:0/2965826664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4010510 con 0x7f9bfc1544f0 2023-11-06T20:45:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.699+0000 7f9be37fe700 1 -- 172.21.15.33:0/2965826664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf4010690 con 0x7f9bfc1544f0 2023-11-06T20:45:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.699+0000 7f9c01dff700 1 -- 172.21.15.33:0/2965826664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc0c3280 con 0x7f9bfc1544f0 2023-11-06T20:45:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.699+0000 7f9be37fe700 1 -- 172.21.15.33:0/2965826664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4010a50 con 0x7f9bfc1544f0 2023-11-06T20:45:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.699+0000 7f9c01dff700 1 -- 172.21.15.33:0/2965826664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc0ae120 con 0x7f9bfc1544f0 2023-11-06T20:45:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.700+0000 7f9c01dff700 1 -- 172.21.15.33:0/2965826664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc0ae590 con 0x7f9bfc1544f0 2023-11-06T20:45:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.701+0000 7f9be37fe700 1 -- 172.21.15.33:0/2965826664 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9bf401b400 con 0x7f9bfc1544f0 2023-11-06T20:45:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.702+0000 7f9be37fe700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9be805f150 0x7f9be8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.702+0000 7f9be37fe700 1 -- 172.21.15.33:0/2965826664 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9bf4079ff0 con 0x7f9bfc1544f0 2023-11-06T20:45:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.702+0000 7f9bfb7fe700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9be805f150 0x7f9be8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.703+0000 7f9bfb7fe700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9be805f150 0x7f9be8061610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f9bec009130 tx=0x7f9bec017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:14.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.706+0000 7f9be37fe700 1 -- 172.21.15.33:0/2965826664 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9bfc0ae590 con 0x7f9bfc1544f0 2023-11-06T20:45:14.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.954+0000 7f9c01dff700 1 -- 172.21.15.33:0/2965826664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9bfc0c6ee0 con 0x7f9bfc1544f0 2023-11-06T20:45:14.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.958+0000 7f9be37fe700 1 -- 172.21.15.33:0/2965826664 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9bf404b260 con 0x7f9bfc1544f0 2023-11-06T20:45:14.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 -- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9be805f150 msgr2=0x7f9be8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9be805f150 0x7f9be8061610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f9bec009130 tx=0x7f9bec017000).stop 2023-11-06T20:45:14.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 -- 172.21.15.33:0/2965826664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 msgr2=0x7f9bfc0c4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:14.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 0x7f9bfc0c4000 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f9bf400c760 tx=0x7f9bf4007f10).stop 2023-11-06T20:45:14.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 -- 172.21.15.33:0/2965826664 shutdown_connections 2023-11-06T20:45:14.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9be805f150 0x7f9be8061610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bfc064bb0 0x7f9bfc147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bfc1544f0 0x7f9bfc0c4000 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 --2- 172.21.15.33:0/2965826664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bfc0c0c40 0x7f9bfc0c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:14.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 -- 172.21.15.33:0/2965826664 >> 172.21.15.33:0/2965826664 conn(0x7f9bfc0cb900 msgr2=0x7f9bfc0ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:14.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 -- 172.21.15.33:0/2965826664 shutdown_connections 2023-11-06T20:45:14.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:14.960+0000 7f9c01dff700 1 -- 172.21.15.33:0/2965826664 wait complete. 2023-11-06T20:45:14.964 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T20:45:14.978 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:45:15.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.175+0000 7f11defff700 1 Processor -- start 2023-11-06T20:45:15.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.175+0000 7f11defff700 1 -- start start 2023-11-06T20:45:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.176+0000 7f11defff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f11d8064ce0 0x7f11d81544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.177+0000 7f11defff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d80c0880 con 0x7f11d8154110 2023-11-06T20:45:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.177+0000 7f11defff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d80c8260 con 0x7f11d80aade0 2023-11-06T20:45:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.177+0000 7f11defff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d80bfd40 con 0x7f11d8064ce0 2023-11-06T20:45:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.177+0000 7f11dcd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f11d8154110 0x7f11d80645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58024/0 (socket says 172.21.15.33:58024) 2023-11-06T20:45:15.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.177+0000 7f11dcd9b700 1 -- 172.21.15.33:0/3198839336 learned_addr learned my addr 172.21.15.33:0/3198839336 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:15.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.177+0000 7f11d7fff700 1 --2- 172.21.15.33:0/3198839336 >> v2:172.21.15.69:3300/0 conn(0x7f11d8064ce0 0x7f11d81544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.177+0000 7f11d77fe700 1 -- 172.21.15.33:0/3198839336 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211677404 0 0) 0x7f11d80c0880 con 0x7f11d8154110 2023-11-06T20:45:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.177+0000 7f11d77fe700 1 -- 172.21.15.33:0/3198839336 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11c80033f0 con 0x7f11d8154110 2023-11-06T20:45:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.178+0000 7f11d7fff700 1 -- 172.21.15.33:0/3198839336 >> v1:172.21.15.69:6789/0 conn(0x7f11d80aade0 legacy=0x7f11d80c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.178+0000 7f11d7fff700 1 -- 172.21.15.33:0/3198839336 >> v1:172.21.15.33:6789/0 conn(0x7f11d8154110 legacy=0x7f11d80645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.178+0000 7f11d77fe700 1 -- 172.21.15.33:0/3198839336 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1313485162 0 0) 0x7f11c80033f0 con 0x7f11d8154110 2023-11-06T20:45:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.178+0000 7f11d7fff700 1 -- 172.21.15.33:0/3198839336 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d80bff50 con 0x7f11d8064ce0 2023-11-06T20:45:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.178+0000 7f11d7fff700 1 --2- 172.21.15.33:0/3198839336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d8064ce0 0x7f11d81544f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f11c4009760 tx=0x7f11c4009060).ready entity=mon.1 client_cookie=6f6dbb655c9ed662 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.178+0000 7f11d77fe700 1 -- 172.21.15.33:0/3198839336 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc001470 con 0x7f11d8064ce0 2023-11-06T20:45:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.178+0000 7f11d77fe700 1 -- 172.21.15.33:0/3198839336 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c4011400 con 0x7f11d8064ce0 2023-11-06T20:45:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.178+0000 7f11d77fe700 1 -- 172.21.15.33:0/3198839336 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4011780 con 0x7f11d8064ce0 2023-11-06T20:45:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.179+0000 7f11defff700 1 -- 172.21.15.33:0/3198839336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d8064ce0 msgr2=0x7f11d81544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.179+0000 7f11defff700 1 --2- 172.21.15.33:0/3198839336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d8064ce0 0x7f11d81544f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f11c4009760 tx=0x7f11c4009060).stop 2023-11-06T20:45:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.179+0000 7f11defff700 1 -- 172.21.15.33:0/3198839336 shutdown_connections 2023-11-06T20:45:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.179+0000 7f11defff700 1 --2- 172.21.15.33:0/3198839336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d8064ce0 0x7f11d81544f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.179+0000 7f11defff700 1 -- 172.21.15.33:0/3198839336 >> 172.21.15.33:0/3198839336 conn(0x7f11d80cb8b0 msgr2=0x7f11d80c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.179+0000 7f11defff700 1 -- 172.21.15.33:0/3198839336 shutdown_connections 2023-11-06T20:45:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.179+0000 7f11defff700 1 -- 172.21.15.33:0/3198839336 wait complete. 2023-11-06T20:45:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.180+0000 7f11defff700 1 Processor -- start 2023-11-06T20:45:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.181+0000 7f11defff700 1 -- start start 2023-11-06T20:45:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.181+0000 7f11defff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f11d8064ce0 0x7f11d815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.181+0000 7f11defff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d80aade0 0x7f11d80a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.181+0000 7f11defff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d8154110 0x7f11d80a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.181+0000 7f11defff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80a5310 con 0x7f11d80aade0 2023-11-06T20:45:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11defff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80c4820 con 0x7f11d8154110 2023-11-06T20:45:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11defff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80c3c10 con 0x7f11d8064ce0 2023-11-06T20:45:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d80aade0 0x7f11d80a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d80aade0 0x7f11d80a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36508/0 (socket says 172.21.15.33:36508) 2023-11-06T20:45:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11d7fff700 1 -- 172.21.15.33:0/2326978010 learned_addr learned my addr 172.21.15.33:0/2326978010 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11dcd9b700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f11d8064ce0 0x7f11d815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11dd59c700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d8154110 0x7f11d80a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11d7fff700 1 -- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f11d8064ce0 msgr2=0x7f11d815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11d7fff700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f11d8064ce0 0x7f11d815deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11d7fff700 1 -- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d8154110 msgr2=0x7f11d80a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11d7fff700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d8154110 0x7f11d80a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11d7fff700 1 -- 172.21.15.33:0/2326978010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40092a0 con 0x7f11d80aade0 2023-11-06T20:45:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.182+0000 7f11d7fff700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d80aade0 0x7f11d80a4930 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f11d8084390 tx=0x7f11c4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.183+0000 7f11d4ff9700 1 -- 172.21.15.33:0/2326978010 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c40108c0 con 0x7f11d80aade0 2023-11-06T20:45:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.183+0000 7f11d4ff9700 1 -- 172.21.15.33:0/2326978010 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c4010a20 con 0x7f11d80aade0 2023-11-06T20:45:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.183+0000 7f11d4ff9700 1 -- 172.21.15.33:0/2326978010 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4010da0 con 0x7f11d80aade0 2023-11-06T20:45:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.183+0000 7f11defff700 1 -- 172.21.15.33:0/2326978010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d80c3ef0 con 0x7f11d80aade0 2023-11-06T20:45:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.183+0000 7f11defff700 1 -- 172.21.15.33:0/2326978010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d80c44b0 con 0x7f11d80aade0 2023-11-06T20:45:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.184+0000 7f11defff700 1 -- 172.21.15.33:0/2326978010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d806c6c0 con 0x7f11d80aade0 2023-11-06T20:45:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.185+0000 7f11d4ff9700 1 -- 172.21.15.33:0/2326978010 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f11c401d640 con 0x7f11d80aade0 2023-11-06T20:45:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.186+0000 7f11d4ff9700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11bc05f150 0x7f11bc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.186+0000 7f11d4ff9700 1 -- 172.21.15.33:0/2326978010 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f11c407b940 con 0x7f11d80aade0 2023-11-06T20:45:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.186+0000 7f11dcd9b700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11bc05f150 0x7f11bc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.187+0000 7f11dcd9b700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11bc05f150 0x7f11bc061610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f11cc000c00 tx=0x7f11cc001660).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.190+0000 7f11d4ff9700 1 -- 172.21.15.33:0/2326978010 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f11c404cb50 con 0x7f11d80aade0 2023-11-06T20:45:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.413+0000 7f11defff700 1 -- 172.21.15.33:0/2326978010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11d80c0230 con 0x7f11d80aade0 2023-11-06T20:45:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.414+0000 7f11d4ff9700 1 -- 172.21.15.33:0/2326978010 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f11c4016030 con 0x7f11d80aade0 2023-11-06T20:45:15.413 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:45:15.413 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/2191444463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2191444463},{"type":"v1","addr":"172.21.15.69:6836","nonce":2191444463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3920247513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3920247513},{"type":"v1","addr":"172.21.15.33:6834","nonce":3920247513}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2046477314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2046477314},{"type":"v1","addr":"172.21.15.69:6837","nonce":2046477314}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:45:04.669326+0000","modified":"2023-11-06T20:45:07.773858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/3657599674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3657599674},{"type":"v1","addr":"172.21.15.33:6835","nonce":3657599674}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-06T20:45:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 -- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11bc05f150 msgr2=0x7f11bc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11bc05f150 0x7f11bc061610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f11cc000c00 tx=0x7f11cc001660).stop 2023-11-06T20:45:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 -- 172.21.15.33:0/2326978010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d80aade0 msgr2=0x7f11d80a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d80aade0 0x7f11d80a4930 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f11d8084390 tx=0x7f11c4014000).stop 2023-11-06T20:45:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 -- 172.21.15.33:0/2326978010 shutdown_connections 2023-11-06T20:45:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11bc05f150 0x7f11bc061610 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:15.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f11d8064ce0 0x7f11d815deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:15.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d8154110 0x7f11d80a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:15.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 --2- 172.21.15.33:0/2326978010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d80aade0 0x7f11d80a4930 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:15.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 -- 172.21.15.33:0/2326978010 >> 172.21.15.33:0/2326978010 conn(0x7f11d80cb8b0 msgr2=0x7f11d80afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:15.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 -- 172.21.15.33:0/2326978010 shutdown_connections 2023-11-06T20:45:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:15.416+0000 7f11defff700 1 -- 172.21.15.33:0/2326978010 wait complete. 2023-11-06T20:45:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 15 2023-11-06T20:45:15.835 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T20:45:04.669326+0000', 'modified': '2023-11-06T20:45:07.773858+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4926}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4926': {'gid': 4926, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.33:6835/3657599674', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6833', 'nonce': 3657599674}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 3657599674}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T20:45:15.835 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T20:45:15.836 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T20:45:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.036+0000 7f53cadff700 1 Processor -- start 2023-11-06T20:45:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.036+0000 7f53cadff700 1 -- start start 2023-11-06T20:45:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.038+0000 7f53cadff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c414e6a0 0x7f53c4066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.038+0000 7f53cadff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f53c40c5ea0 0x7f53c4157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.038+0000 7f53cadff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c40640c0 con 0x7f53c4158190 2023-11-06T20:45:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.038+0000 7f53cadff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c4064d10 con 0x7f53c40c5ea0 2023-11-06T20:45:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.038+0000 7f53cadff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4158640 con 0x7f53c414e6a0 2023-11-06T20:45:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.038+0000 7f53c939c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f53c4158190 0x7f53c40c3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58032/0 (socket says 172.21.15.33:58032) 2023-11-06T20:45:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.038+0000 7f53c939c700 1 -- 172.21.15.33:0/64377209 learned_addr learned my addr 172.21.15.33:0/64377209 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.038+0000 7f53c3fff700 1 --2- 172.21.15.33:0/64377209 >> v2:172.21.15.69:3300/0 conn(0x7f53c40c5ea0 0x7f53c4157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c414e6a0 0x7f53c4066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599116836 0 0) 0x7f53bc001490 con 0x7f53c4158190 2023-11-06T20:45:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b40033f0 con 0x7f53c4158190 2023-11-06T20:45:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1085157057 0 0) 0x7f53b40033f0 con 0x7f53c4158190 2023-11-06T20:45:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c414e6a0 msgr2=0x7f53c4066e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c414e6a0 0x7f53c4066e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 >> v2:172.21.15.69:3300/0 conn(0x7f53c40c5ea0 msgr2=0x7f53c4157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 --2- 172.21.15.33:0/64377209 >> v2:172.21.15.69:3300/0 conn(0x7f53c40c5ea0 0x7f53c4157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40c5a10 con 0x7f53c4158190 2023-11-06T20:45:16.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f53bc001980 con 0x7f53c4158190 2023-11-06T20:45:16.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c3fff700 1 --2- 172.21.15.33:0/64377209 >> v2:172.21.15.69:3300/0 conn(0x7f53c40c5ea0 0x7f53c4157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:16.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c414e6a0 0x7f53c4066e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:16.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 >> v1:172.21.15.33:6789/0 conn(0x7f53c4158190 legacy=0x7f53c40c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53b40038e0 0x7f53b4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c414e6a0 0x7f53b4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b4004b80 0x7f53b4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4064d10 con 0x7f53b4004b80 2023-11-06T20:45:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4158640 con 0x7f53b40038e0 2023-11-06T20:45:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4004500 con 0x7f53c414e6a0 2023-11-06T20:45:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.039+0000 7f53c939c700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b4004b80 0x7f53b4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53b40038e0 0x7f53b4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c3fff700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c414e6a0 0x7f53b4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c939c700 1 -- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c414e6a0 msgr2=0x7f53b4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c939c700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c414e6a0 0x7f53b4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c939c700 1 -- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53b40038e0 msgr2=0x7f53b4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c939c700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53b40038e0 0x7f53b4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c939c700 1 -- 172.21.15.33:0/64377209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b80092a0 con 0x7f53b4004b80 2023-11-06T20:45:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c939c700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b4004b80 0x7f53b4005000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f53bc00c6d0 tx=0x7f53bc007c80).ready entity=mon.0 client_cookie=2151058917415a13 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc005b20 con 0x7f53b4004b80 2023-11-06T20:45:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc005c80 con 0x7f53b4004b80 2023-11-06T20:45:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.040+0000 7f53c37fe700 1 -- 172.21.15.33:0/64377209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc007110 con 0x7f53b4004b80 2023-11-06T20:45:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.041+0000 7f53cadff700 1 -- 172.21.15.33:0/64377209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b4004b80 msgr2=0x7f53b4005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.041+0000 7f53cadff700 1 --2- 172.21.15.33:0/64377209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b4004b80 0x7f53b4005000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f53bc00c6d0 tx=0x7f53bc007c80).stop 2023-11-06T20:45:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.041+0000 7f53c939c700 1 -- 172.21.15.33:0/64377209 reap_dead start 2023-11-06T20:45:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.041+0000 7f53cadff700 1 -- 172.21.15.33:0/64377209 shutdown_connections 2023-11-06T20:45:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.041+0000 7f53cadff700 1 -- 172.21.15.33:0/64377209 >> 172.21.15.33:0/64377209 conn(0x7f53c405e1b0 msgr2=0x7f53c405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.041+0000 7f53cadff700 1 -- 172.21.15.33:0/64377209 shutdown_connections 2023-11-06T20:45:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.041+0000 7f53cadff700 1 -- 172.21.15.33:0/64377209 wait complete. 2023-11-06T20:45:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.042+0000 7f53cadff700 1 Processor -- start 2023-11-06T20:45:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.043+0000 7f53cadff700 1 -- start start 2023-11-06T20:45:16.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53cadff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53b4004b80 0x7f53c40c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53cadff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c40c2500 0x7f53c41648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53cadff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c4164dd0 0x7f53c413c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53cadff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40c16f0 con 0x7f53c40c2500 2023-11-06T20:45:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53cadff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40c1260 con 0x7f53b4004b80 2023-11-06T20:45:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53cadff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40c06a0 con 0x7f53c4164dd0 2023-11-06T20:45:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c40c2500 0x7f53c41648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c40c2500 0x7f53c41648e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36520/0 (socket says 172.21.15.33:36520) 2023-11-06T20:45:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c3fff700 1 -- 172.21.15.33:0/1987695413 learned_addr learned my addr 172.21.15.33:0/1987695413 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53b4004b80 0x7f53c40c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c939c700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c4164dd0 0x7f53c413c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c3fff700 1 -- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c4164dd0 msgr2=0x7f53c413c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c3fff700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c4164dd0 0x7f53c413c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c3fff700 1 -- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53b4004b80 msgr2=0x7f53c40c1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c3fff700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53b4004b80 0x7f53c40c1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c3fff700 1 -- 172.21.15.33:0/1987695413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40c0930 con 0x7f53c40c2500 2023-11-06T20:45:16.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c939c700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c4164dd0 0x7f53c413c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.044+0000 7f53c3fff700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c40c2500 0x7f53c41648e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f53b0011a20 tx=0x7f53b0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.045+0000 7f53c0ff9700 1 -- 172.21.15.33:0/1987695413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b000f940 con 0x7f53c40c2500 2023-11-06T20:45:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.045+0000 7f53c0ff9700 1 -- 172.21.15.33:0/1987695413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b000faa0 con 0x7f53c40c2500 2023-11-06T20:45:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.045+0000 7f53c0ff9700 1 -- 172.21.15.33:0/1987695413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b000fe20 con 0x7f53c40c2500 2023-11-06T20:45:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.045+0000 7f53cadff700 1 -- 172.21.15.33:0/1987695413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c40c0c10 con 0x7f53c40c2500 2023-11-06T20:45:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.045+0000 7f53cadff700 1 -- 172.21.15.33:0/1987695413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c40c2980 con 0x7f53c40c2500 2023-11-06T20:45:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.047+0000 7f53c0ff9700 1 -- 172.21.15.33:0/1987695413 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f53b0014650 con 0x7f53c40c2500 2023-11-06T20:45:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.048+0000 7f53c0ff9700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53a805f150 0x7f53a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.048+0000 7f53c0ff9700 1 -- 172.21.15.33:0/1987695413 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f53b007ac90 con 0x7f53c40c2500 2023-11-06T20:45:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.048+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53a805f150 0x7f53a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.049+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53a805f150 0x7f53a8061610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f53b8006d80 tx=0x7f53b8009060).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.050+0000 7f53e212eb80 1 Processor -- start 2023-11-06T20:45:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.050+0000 7f53e212eb80 1 -- start start 2023-11-06T20:45:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x5605423e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x5605423dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054240d4c0 0x5605423e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53e212eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605423212f0 con 0x56054242dbe0 2023-11-06T20:45:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53e212eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605423367e0 con 0x56054240d4c0 2023-11-06T20:45:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53e212eb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x560542336920 con 0x5605423e5080 2023-11-06T20:45:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x5605423dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x5605423dfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36526/0 (socket says 172.21.15.33:36526) 2023-11-06T20:45:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53c3fff700 1 -- 172.21.15.33:0/3414357873 learned_addr learned my addr 172.21.15.33:0/3414357873 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/3414357873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x5605423e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53c939c700 1 --2- 172.21.15.33:0/3414357873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054240d4c0 0x5605423e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53c3fff700 1 -- 172.21.15.33:0/3414357873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 msgr2=0x5605423e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53c3fff700 1 --2- 172.21.15.33:0/3414357873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x5605423e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53c3fff700 1 -- 172.21.15.33:0/3414357873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054240d4c0 msgr2=0x5605423e03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53c3fff700 1 --2- 172.21.15.33:0/3414357873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054240d4c0 0x5605423e03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.051+0000 7f53c3fff700 1 -- 172.21.15.33:0/3414357873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56054232f720 con 0x56054242dbe0 2023-11-06T20:45:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.052+0000 7f53c3fff700 1 --2- 172.21.15.33:0/3414357873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x5605423dfef0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f53b00790e0 tx=0x7f53b0014000).ready entity=mon.0 client_cookie=235316853a8b38fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.052+0000 7f53ad9fe700 1 -- 172.21.15.33:0/3414357873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b00552a0 con 0x56054242dbe0 2023-11-06T20:45:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.052+0000 7f53ad9fe700 1 -- 172.21.15.33:0/3414357873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b004df00 con 0x56054242dbe0 2023-11-06T20:45:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.052+0000 7f53ad9fe700 1 -- 172.21.15.33:0/3414357873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b004f050 con 0x56054242dbe0 2023-11-06T20:45:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.052+0000 7f53e212eb80 1 -- 172.21.15.33:0/3414357873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 msgr2=0x5605423dfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.052+0000 7f53e212eb80 1 --2- 172.21.15.33:0/3414357873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x5605423dfef0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f53b00790e0 tx=0x7f53b0014000).stop 2023-11-06T20:45:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.053+0000 7f53e212eb80 1 -- 172.21.15.33:0/3414357873 shutdown_connections 2023-11-06T20:45:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.053+0000 7f53e212eb80 1 --2- 172.21.15.33:0/3414357873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x5605423e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.053+0000 7f53e212eb80 1 --2- 172.21.15.33:0/3414357873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054240d4c0 0x5605423e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.053+0000 7f53e212eb80 1 --2- 172.21.15.33:0/3414357873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x5605423dfef0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.053+0000 7f53e212eb80 1 -- 172.21.15.33:0/3414357873 >> 172.21.15.33:0/3414357873 conn(0x56054224e940 msgr2=0x56054242e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.053+0000 7f53e212eb80 1 -- 172.21.15.33:0/3414357873 shutdown_connections 2023-11-06T20:45:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.053+0000 7f53e212eb80 1 -- 172.21.15.33:0/3414357873 wait complete. 2023-11-06T20:45:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.053+0000 7f53e212eb80 1 -- start start 2023-11-06T20:45:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.054+0000 7f53e212eb80 1 Processor -- start 2023-11-06T20:45:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605423e5080 0x56054240f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56054240d4c0 0x560542411c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x560542427860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53e212eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560542336ba0 con 0x56054242dbe0 2023-11-06T20:45:16.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53e212eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560542301e60 con 0x5605423e5080 2023-11-06T20:45:16.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53e212eb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x560542410960 con 0x56054240d4c0 2023-11-06T20:45:16.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53e212eb80 10 client.0 ll_register_callbacks cb 0x5605422d6ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T20:45:16.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x560542427860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x560542427860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36530/0 (socket says 172.21.15.33:36530) 2023-11-06T20:45:16.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c939c700 1 -- 172.21.15.33:0/2970471352 learned_addr learned my addr 172.21.15.33:0/2970471352 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:16.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c3fff700 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56054240d4c0 0x560542411c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605423e5080 0x56054240f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c939c700 1 -- 172.21.15.33:0/2970471352 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56054240d4c0 msgr2=0x560542411c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c939c700 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56054240d4c0 0x560542411c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c939c700 1 -- 172.21.15.33:0/2970471352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605423e5080 msgr2=0x56054240f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c939c700 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605423e5080 0x56054240f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c939c700 1 -- 172.21.15.33:0/2970471352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c40c2980 con 0x56054242dbe0 2023-11-06T20:45:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.055+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605423e5080 0x56054240f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53c939c700 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x560542427860 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f53bc007fd0 tx=0x7f53bc015000).ready entity=mon.0 client_cookie=cea5662bdf1b712d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T20:45:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2970471352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc001940 con 0x56054242dbe0 2023-11-06T20:45:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2970471352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f53bc01c040 con 0x56054242dbe0 2023-11-06T20:45:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53e212eb80 1 -- 172.21.15.33:0/2970471352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605423d8e20 con 0x56054242dbe0 2023-11-06T20:45:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2970471352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53bc007bf0 con 0x56054242dbe0 2023-11-06T20:45:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53e212eb80 10 client.5127 fetch_fsmap learned FSMap version 15 2023-11-06T20:45:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53e212eb80 1 -- 172.21.15.33:0/2970471352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x56054232aac0 con 0x56054242dbe0 2023-11-06T20:45:16.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2970471352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605423d8e20 con 0x56054242dbe0 2023-11-06T20:45:16.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2970471352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 15) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f53bc02b070 con 0x56054242dbe0 2023-11-06T20:45:16.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53e212eb80 10 client.5127 fetch_fsmap finished waiting for FSMap version 15 2023-11-06T20:45:16.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53e212eb80 10 client.5127 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T20:45:16.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.056+0000 7f53e212eb80 20 client.5127 populate_metadata read hostname 'smithi033' 2023-11-06T20:45:16.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.057+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.33:6833/3657599674,v1:172.21.15.33:6835/3657599674] conn(0x560542409440 0x560542409820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.057+0000 7f53e212eb80 4 client.5127 mds_command: new command op to 4926 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T20:45:16.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.057+0000 7f53e212eb80 1 -- 172.21.15.33:0/2970471352 --> [v2:172.21.15.33:6833/3657599674,v1:172.21.15.33:6835/3657599674] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56054219eef0 con 0x560542409440 2023-11-06T20:45:16.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.057+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.33:6833/3657599674,v1:172.21.15.33:6835/3657599674] conn(0x560542409440 0x560542409820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.057+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.33:6833/3657599674,v1:172.21.15.33:6835/3657599674] conn(0x560542409440 0x560542409820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7a75305b5da092ce server_cookie=2ddbd7a140e469e0 in_seq=0 out_seq=0 2023-11-06T20:45:16.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.057+0000 7f53a2ffd700 10 client.5127 ms_handle_connect on v2:172.21.15.33:6833/3657599674 2023-11-06T20:45:16.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2970471352 <== mds.0 v2:172.21.15.33:6833/3657599674 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x56054219eef0 con 0x560542409440 2023-11-06T20:45:16.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53a2ffd700 10 client.5127 handle_command_reply: tid=0 2023-11-06T20:45:16.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53e212eb80 1 client.5127 shutdown 2023-11-06T20:45:16.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53e212eb80 1 -- 172.21.15.33:0/2970471352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 msgr2=0x560542427860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x560542427860 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f53bc007fd0 tx=0x7f53bc015000).stop 2023-11-06T20:45:16.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53e212eb80 1 -- 172.21.15.33:0/2970471352 shutdown_connections 2023-11-06T20:45:16.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.33:6833/3657599674,v1:172.21.15.33:6835/3657599674] conn(0x560542409440 0x560542409820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x560542427860 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5605423e5080 0x56054240f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53a2ffd700 0 client.5127 ms_handle_reset on v2:172.21.15.33:6833/3657599674 2023-11-06T20:45:16.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2970471352 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x56054240d4c0 0x560542411c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.058+0000 7f53e212eb80 1 -- 172.21.15.33:0/2970471352 >> 172.21.15.33:0/2970471352 conn(0x56054224e940 msgr2=0x5605424286e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:16.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.059+0000 7f53e212eb80 1 -- 172.21.15.33:0/2970471352 shutdown_connections 2023-11-06T20:45:16.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.059+0000 7f53e212eb80 1 -- 172.21.15.33:0/2970471352 wait complete. 2023-11-06T20:45:16.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.059+0000 7f53e212eb80 20 client.5127 trim_cache size 0 max 16384 2023-11-06T20:45:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.076+0000 7f53e212eb80 1 Processor -- start 2023-11-06T20:45:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.076+0000 7f53e212eb80 1 -- start start 2023-11-06T20:45:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x56054242e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054240d4c0 0x560542407e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054242dbe0 0x560542408300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53e212eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605423e48e0 con 0x56054240d4c0 2023-11-06T20:45:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53e212eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605423e5e30 con 0x56054242dbe0 2023-11-06T20:45:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53e212eb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605423db4c0 con 0x5605423e5080 2023-11-06T20:45:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054240d4c0 0x560542407e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054240d4c0 0x560542407e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36542/0 (socket says 172.21.15.33:36542) 2023-11-06T20:45:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c3fff700 1 -- 172.21.15.33:0/3050718383 learned_addr learned my addr 172.21.15.33:0/3050718383 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c939c700 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054242dbe0 0x560542408300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x56054242e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c3fff700 1 -- 172.21.15.33:0/3050718383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 msgr2=0x56054242e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c3fff700 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x56054242e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c3fff700 1 -- 172.21.15.33:0/3050718383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054242dbe0 msgr2=0x560542408300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c3fff700 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054242dbe0 0x560542408300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c3fff700 1 -- 172.21.15.33:0/3050718383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605423e1f10 con 0x56054240d4c0 2023-11-06T20:45:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c939c700 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054242dbe0 0x560542408300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.077+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x56054242e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53c3fff700 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054240d4c0 0x560542407e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f53b0051010 tx=0x7f53b0057000).ready entity=mon.0 client_cookie=31c4fa92af2a1c32 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53ae1ff700 1 -- 172.21.15.33:0/3050718383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0066070 con 0x56054240d4c0 2023-11-06T20:45:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53ae1ff700 1 -- 172.21.15.33:0/3050718383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b004f050 con 0x56054240d4c0 2023-11-06T20:45:16.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53ae1ff700 1 -- 172.21.15.33:0/3050718383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b004c270 con 0x56054240d4c0 2023-11-06T20:45:16.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53e212eb80 1 -- 172.21.15.33:0/3050718383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054240d4c0 msgr2=0x560542407e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53e212eb80 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054240d4c0 0x560542407e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f53b0051010 tx=0x7f53b0057000).stop 2023-11-06T20:45:16.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53e212eb80 1 -- 172.21.15.33:0/3050718383 shutdown_connections 2023-11-06T20:45:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53e212eb80 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x56054242e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53e212eb80 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054242dbe0 0x560542408300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53e212eb80 1 --2- 172.21.15.33:0/3050718383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054240d4c0 0x560542407e10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53e212eb80 1 -- 172.21.15.33:0/3050718383 >> 172.21.15.33:0/3050718383 conn(0x56054224e940 msgr2=0x56054240f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53e212eb80 1 -- 172.21.15.33:0/3050718383 shutdown_connections 2023-11-06T20:45:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.078+0000 7f53e212eb80 1 -- 172.21.15.33:0/3050718383 wait complete. 2023-11-06T20:45:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.079+0000 7f53e212eb80 1 -- start start 2023-11-06T20:45:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.079+0000 7f53e212eb80 1 Processor -- start 2023-11-06T20:45:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.080+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x56054248c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.080+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054240d4c0 0x560542490380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.080+0000 7f53e212eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x56054240ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.080+0000 7f53e212eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56054240bf00 con 0x56054242dbe0 2023-11-06T20:45:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.080+0000 7f53e212eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605424909c0 con 0x56054240d4c0 2023-11-06T20:45:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.080+0000 7f53e212eb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x56054248f4d0 con 0x5605423e5080 2023-11-06T20:45:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.080+0000 7f53e212eb80 10 client.0 ll_register_callbacks cb 0x5605422d6ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T20:45:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.080+0000 7f53c939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x56054240ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53c939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x56054240ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36548/0 (socket says 172.21.15.33:36548) 2023-11-06T20:45:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53c939c700 1 -- 172.21.15.33:0/2629475626 learned_addr learned my addr 172.21.15.33:0/2629475626 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x56054248c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53c3fff700 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054240d4c0 0x560542490380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53c939c700 1 -- 172.21.15.33:0/2629475626 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 msgr2=0x56054248c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53c939c700 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x56054248c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53c939c700 1 -- 172.21.15.33:0/2629475626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054240d4c0 msgr2=0x560542490380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53c939c700 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054240d4c0 0x560542490380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53c939c700 1 -- 172.21.15.33:0/2629475626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56054232f720 con 0x56054242dbe0 2023-11-06T20:45:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53c939c700 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x56054240ba10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f53bc00ccc0 tx=0x7f53bc022000).ready entity=mon.0 client_cookie=f8a2cf90af4b0417 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T20:45:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2629475626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc001490 con 0x56054242dbe0 2023-11-06T20:45:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53e212eb80 1 -- 172.21.15.33:0/2629475626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53b004df00 con 0x56054242dbe0 2023-11-06T20:45:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.081+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2629475626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f53bc007810 con 0x56054242dbe0 2023-11-06T20:45:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2629475626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53bc0079d0 con 0x56054242dbe0 2023-11-06T20:45:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53e212eb80 10 client.5133 fetch_fsmap learned FSMap version 15 2023-11-06T20:45:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53e212eb80 1 -- 172.21.15.33:0/2629475626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x56054232a810 con 0x56054242dbe0 2023-11-06T20:45:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2629475626 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b004df00 con 0x56054242dbe0 2023-11-06T20:45:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2629475626 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 15) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f53bc02b030 con 0x56054242dbe0 2023-11-06T20:45:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53e212eb80 10 client.5133 fetch_fsmap finished waiting for FSMap version 15 2023-11-06T20:45:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53e212eb80 10 client.5133 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T20:45:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53e212eb80 20 client.5133 populate_metadata read hostname 'smithi033' 2023-11-06T20:45:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.33:6833/3657599674,v1:172.21.15.33:6835/3657599674] conn(0x56054240de30 0x56054240e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53e212eb80 4 client.5133 mds_command: new command op to 4926 tid=0[{"prefix": "status"}] 2023-11-06T20:45:16.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53e212eb80 1 -- 172.21.15.33:0/2629475626 --> [v2:172.21.15.33:6833/3657599674,v1:172.21.15.33:6835/3657599674] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5605421d4040 con 0x56054240de30 2023-11-06T20:45:16.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.082+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.33:6833/3657599674,v1:172.21.15.33:6835/3657599674] conn(0x56054240de30 0x56054240e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53a2ffd700 10 client.5133 ms_handle_connect on v2:172.21.15.33:6833/3657599674 2023-11-06T20:45:16.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53c8b9b700 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.33:6833/3657599674,v1:172.21.15.33:6835/3657599674] conn(0x56054240de30 0x56054240e210 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8aaee17d13f27eee server_cookie=9baf504236c1a5f6 in_seq=0 out_seq=0 2023-11-06T20:45:16.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53a2ffd700 1 -- 172.21.15.33:0/2629475626 <== mds.0 v2:172.21.15.33:6833/3657599674 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5605421d4040 con 0x56054240de30 2023-11-06T20:45:16.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53a2ffd700 10 client.5133 handle_command_reply: tid=0 2023-11-06T20:45:16.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53e212eb80 1 client.5133 shutdown 2023-11-06T20:45:16.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53e212eb80 1 -- 172.21.15.33:0/2629475626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 msgr2=0x56054240ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x56054240ba10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f53bc00ccc0 tx=0x7f53bc022000).stop 2023-11-06T20:45:16.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53e212eb80 1 -- 172.21.15.33:0/2629475626 shutdown_connections 2023-11-06T20:45:16.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5605423e5080 0x56054248c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.33:6833/3657599674,v1:172.21.15.33:6835/3657599674] conn(0x56054240de30 0x56054240e210 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56054242dbe0 0x56054240ba10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53e212eb80 1 --2- 172.21.15.33:0/2629475626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56054240d4c0 0x560542490380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53a2ffd700 0 client.5133 ms_handle_reset on v2:172.21.15.33:6833/3657599674 2023-11-06T20:45:16.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.083+0000 7f53e212eb80 1 -- 172.21.15.33:0/2629475626 >> 172.21.15.33:0/2629475626 conn(0x56054224e940 msgr2=0x560542411a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:16.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53e212eb80 1 -- 172.21.15.33:0/2629475626 shutdown_connections 2023-11-06T20:45:16.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53e212eb80 1 -- 172.21.15.33:0/2629475626 wait complete. 2023-11-06T20:45:16.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53e212eb80 20 client.5133 trim_cache size 0 max 16384 2023-11-06T20:45:16.109 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T20:45:16.109 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T20:45:16.110 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T20:45:16.110 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4926, 2023-11-06T20:45:16.110 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T20:45:16.110 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T20:45:16.110 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T20:45:16.111 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 11.411841730000001, 2023-11-06T20:45:16.111 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 15, 2023-11-06T20:45:16.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 35, 2023-11-06T20:45:16.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 35, 2023-11-06T20:45:16.112 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 19.689070193999999 2023-11-06T20:45:16.112 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T20:45:16.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 -- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53a805f150 msgr2=0x7f53a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53a805f150 0x7f53a8061610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f53b8006d80 tx=0x7f53b8009060).stop 2023-11-06T20:45:16.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 -- 172.21.15.33:0/1987695413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c40c2500 msgr2=0x7f53c41648e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c40c2500 0x7f53c41648e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f53b0011a20 tx=0x7f53b0009060).stop 2023-11-06T20:45:16.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 -- 172.21.15.33:0/1987695413 shutdown_connections 2023-11-06T20:45:16.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53a805f150 0x7f53a8061610 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53c4164dd0 0x7f53c413c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53b4004b80 0x7f53c40c1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 --2- 172.21.15.33:0/1987695413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c40c2500 0x7f53c41648e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 -- 172.21.15.33:0/1987695413 >> 172.21.15.33:0/1987695413 conn(0x7f53c405e1b0 msgr2=0x7f53c413d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:16.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 -- 172.21.15.33:0/1987695413 shutdown_connections 2023-11-06T20:45:16.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.084+0000 7f53cadff700 1 -- 172.21.15.33:0/1987695413 wait complete. 2023-11-06T20:45:16.116 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T20:45:16.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.282+0000 7fccc3fff700 1 Processor -- start 2023-11-06T20:45:16.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.282+0000 7fccc3fff700 1 -- start start 2023-11-06T20:45:16.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.283+0000 7fccc3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fccbc154320 0x7fccbc0a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.283+0000 7fccc3fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fccbc0c0e60 0x7fccbc15de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.283+0000 7fccc3fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccbc0a9820 con 0x7fccbc0aa150 2023-11-06T20:45:16.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.283+0000 7fccc3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccbc154700 con 0x7fccbc154320 2023-11-06T20:45:16.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.283+0000 7fccc3fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccbc0a8c10 con 0x7fccbc0c0e60 2023-11-06T20:45:16.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.284+0000 7fccc1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fccbc154320 0x7fccbc0a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.284+0000 7fccc1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fccbc154320 0x7fccbc0a8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36564/0 (socket says 172.21.15.33:36564) 2023-11-06T20:45:16.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.284+0000 7fccc1d9b700 1 -- 172.21.15.33:0/2209798565 learned_addr learned my addr 172.21.15.33:0/2209798565 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:16.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.284+0000 7fccc159a700 1 --1- 172.21.15.33:0/2209798565 >> v1:172.21.15.33:6789/0 conn(0x7fccbc0aa150 0x7fccbc0c0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58044/0 (socket says 172.21.15.33:58044) 2023-11-06T20:45:16.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.284+0000 7fccc259c700 1 --2- 172.21.15.33:0/2209798565 >> v2:172.21.15.69:3300/0 conn(0x7fccbc0c0e60 0x7fccbc15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.284+0000 7fccc1d9b700 1 -- 172.21.15.33:0/2209798565 >> v2:172.21.15.69:3300/0 conn(0x7fccbc0c0e60 msgr2=0x7fccbc15de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:16.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.284+0000 7fccc1d9b700 1 --2- 172.21.15.33:0/2209798565 >> v2:172.21.15.69:3300/0 conn(0x7fccbc0c0e60 0x7fccbc15de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.284+0000 7fccc1d9b700 1 -- 172.21.15.33:0/2209798565 >> v1:172.21.15.33:6789/0 conn(0x7fccbc0aa150 legacy=0x7fccbc0c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.284+0000 7fccc1d9b700 1 -- 172.21.15.33:0/2209798565 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc0a8ef0 con 0x7fccbc154320 2023-11-06T20:45:16.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.284+0000 7fccc1d9b700 1 --2- 172.21.15.33:0/2209798565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc154320 0x7fccbc0a8670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fccbc085770 tx=0x7fccb80090f0).ready entity=mon.0 client_cookie=1426d975348252ca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:16.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc0d99700 1 -- 172.21.15.33:0/2209798565 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb800b520 con 0x7fccbc154320 2023-11-06T20:45:16.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc0d99700 1 -- 172.21.15.33:0/2209798565 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb800b680 con 0x7fccbc154320 2023-11-06T20:45:16.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc0d99700 1 -- 172.21.15.33:0/2209798565 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb800ba00 con 0x7fccbc154320 2023-11-06T20:45:16.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc3fff700 1 -- 172.21.15.33:0/2209798565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc154320 msgr2=0x7fccbc0a8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc3fff700 1 --2- 172.21.15.33:0/2209798565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc154320 0x7fccbc0a8670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fccbc085770 tx=0x7fccb80090f0).stop 2023-11-06T20:45:16.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc3fff700 1 -- 172.21.15.33:0/2209798565 shutdown_connections 2023-11-06T20:45:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc3fff700 1 --2- 172.21.15.33:0/2209798565 >> v2:172.21.15.69:3300/0 conn(0x7fccbc0c0e60 0x7fccbc15de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc3fff700 1 --2- 172.21.15.33:0/2209798565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc154320 0x7fccbc0a8670 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc3fff700 1 -- 172.21.15.33:0/2209798565 >> 172.21.15.33:0/2209798565 conn(0x7fccbc094530 msgr2=0x7fccbc06a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc3fff700 1 -- 172.21.15.33:0/2209798565 shutdown_connections 2023-11-06T20:45:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.285+0000 7fccc3fff700 1 -- 172.21.15.33:0/2209798565 wait complete. 2023-11-06T20:45:16.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.287+0000 7fccc3fff700 1 Processor -- start 2023-11-06T20:45:16.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.288+0000 7fccc3fff700 1 -- start start 2023-11-06T20:45:16.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccbc0aa150 0x7fccbc151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc0c0e60 0x7fccbc0ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fccbc154320 0x7fccbc147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc0ac020 con 0x7fccbc0c0e60 2023-11-06T20:45:16.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc3fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc0ac300 con 0x7fccbc0aa150 2023-11-06T20:45:16.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc3fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc0ac5e0 con 0x7fccbc154320 2023-11-06T20:45:16.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc159a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc0c0e60 0x7fccbc0ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc159a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc0c0e60 0x7fccbc0ad0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36572/0 (socket says 172.21.15.33:36572) 2023-11-06T20:45:16.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc159a700 1 -- 172.21.15.33:0/2489610326 learned_addr learned my addr 172.21.15.33:0/2489610326 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:16.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc1d9b700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccbc0aa150 0x7fccbc151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.289+0000 7fccc259c700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fccbc154320 0x7fccbc147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccc159a700 1 -- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fccbc154320 msgr2=0x7fccbc147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccc159a700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fccbc154320 0x7fccbc147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccc159a700 1 -- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccbc0aa150 msgr2=0x7fccbc151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccc159a700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccbc0aa150 0x7fccbc151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccc159a700 1 -- 172.21.15.33:0/2489610326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8009370 con 0x7fccbc0c0e60 2023-11-06T20:45:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccc1d9b700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccbc0aa150 0x7fccbc151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccc159a700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc0c0e60 0x7fccbc0ad0c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fccac007fd0 tx=0x7fccac007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccb1ffb700 1 -- 172.21.15.33:0/2489610326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac00ce10 con 0x7fccbc0c0e60 2023-11-06T20:45:16.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccb1ffb700 1 -- 172.21.15.33:0/2489610326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccac0072e0 con 0x7fccbc0c0e60 2023-11-06T20:45:16.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccc3fff700 1 -- 172.21.15.33:0/2489610326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccbc0ac8c0 con 0x7fccbc0c0e60 2023-11-06T20:45:16.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccb1ffb700 1 -- 172.21.15.33:0/2489610326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac0056e0 con 0x7fccbc0c0e60 2023-11-06T20:45:16.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.290+0000 7fccc3fff700 1 -- 172.21.15.33:0/2489610326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccbc0ade20 con 0x7fccbc0c0e60 2023-11-06T20:45:16.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.291+0000 7fccc3fff700 1 -- 172.21.15.33:0/2489610326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccbc0475a0 con 0x7fccbc0c0e60 2023-11-06T20:45:16.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.292+0000 7fccb1ffb700 1 -- 172.21.15.33:0/2489610326 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fccac005ad0 con 0x7fccbc0c0e60 2023-11-06T20:45:16.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.293+0000 7fccb1ffb700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcca005f150 0x7fcca0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:16.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.293+0000 7fccb1ffb700 1 -- 172.21.15.33:0/2489610326 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fccac0798f0 con 0x7fccbc0c0e60 2023-11-06T20:45:16.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.293+0000 7fccc1d9b700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcca005f150 0x7fcca0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:16.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.294+0000 7fccc1d9b700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcca005f150 0x7fcca0061610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fccb8009130 tx=0x7fccb8017000).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:16.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.297+0000 7fccb1ffb700 1 -- 172.21.15.33:0/2489610326 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fccac04ab00 con 0x7fccbc0c0e60 2023-11-06T20:45:16.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.524+0000 7fccc3fff700 1 -- 172.21.15.33:0/2489610326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fccbc0a91d0 con 0x7fccbc0c0e60 2023-11-06T20:45:16.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.524+0000 7fccb1ffb700 1 -- 172.21.15.33:0/2489610326 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7fccac04e120 con 0x7fccbc0c0e60 2023-11-06T20:45:16.524 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T20:45:16.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.526+0000 7fccc3fff700 1 -- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcca005f150 msgr2=0x7fcca0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.526+0000 7fccc3fff700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcca005f150 0x7fcca0061610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fccb8009130 tx=0x7fccb8017000).stop 2023-11-06T20:45:16.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.526+0000 7fccc3fff700 1 -- 172.21.15.33:0/2489610326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc0c0e60 msgr2=0x7fccbc0ad0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:16.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.527+0000 7fccc3fff700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc0c0e60 0x7fccbc0ad0c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fccac007fd0 tx=0x7fccac007df0).stop 2023-11-06T20:45:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.527+0000 7fccc3fff700 1 -- 172.21.15.33:0/2489610326 shutdown_connections 2023-11-06T20:45:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.527+0000 7fccc3fff700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcca005f150 0x7fcca0061610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.527+0000 7fccc3fff700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fccbc154320 0x7fccbc147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.527+0000 7fccc3fff700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccbc0aa150 0x7fccbc151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.527+0000 7fccc3fff700 1 --2- 172.21.15.33:0/2489610326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccbc0c0e60 0x7fccbc0ad0c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:16.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.527+0000 7fccc3fff700 1 -- 172.21.15.33:0/2489610326 >> 172.21.15.33:0/2489610326 conn(0x7fccbc094530 msgr2=0x7fccbc0c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:16.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.527+0000 7fccc3fff700 1 -- 172.21.15.33:0/2489610326 shutdown_connections 2023-11-06T20:45:16.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:16.527+0000 7fccc3fff700 1 -- 172.21.15.33:0/2489610326 wait complete. 2023-11-06T20:45:16.540 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T20:45:16.541 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T20:45:16.541 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T20:45:16.542 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T20:45:16.542 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T20:45:16.542 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T20:45:16.543 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T20:45:16.543 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T20:45:16.543 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T20:45:16.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:45:16.544 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T20:45:16.592 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:45:16.592 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:45:16.592 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:45:16.592 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:45:16.593 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T20:45:16.593 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:45:16.593 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:45:16.593 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:45:16.593 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T20:45:16.594 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:45:16.594 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:45:16.594 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T20:45:16.594 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T20:45:16.594 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:45:16.595 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T20:45:16.595 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T20:45:16.595 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42227sec preferred_lft 42227sec 2023-11-06T20:45:16.595 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T20:45:16.595 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:45:16.596 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:45:16.596 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:45:16.596 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T20:45:16.596 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T20:45:16.596 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T20:45:16.597 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T20:45:16.597 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:45:16.597 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T20:45:16.597 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:45:16.597 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T20:45:16.598 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T20:45:16.598 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T20:45:16.598 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:45:16.598 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T20:45:16.598 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T20:45:16.599 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T20:45:16.599 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:45:16.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:45:16.600 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T20:45:16.672 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T20:45:16.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:45:16.675 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T20:45:16.725 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T20:45:16.725 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T20:45:16.725 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T20:45:16.725 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T20:45:16.726 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T20:45:16.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:45:16.727 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:45:16.727 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:45:16.728 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T20:45:16.728 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T20:45:16.728 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T20:45:16.729 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:45:16.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:45:16.903 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T20:45:16.951 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T20:45:16.951 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T20:45:16.952 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T20:45:16.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:45:16.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T20:45:17.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:45:17.027 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T20:45:17.074 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T20:45:17.075 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:45:17.215 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:45:17.215 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:45:17.242 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:45:17.242 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:45:17.247 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T20:45:17.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:45:17.248 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T20:45:17.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T20:45:17.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.518+0000 7f9242dff700 1 Processor -- start 2023-11-06T20:45:17.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.518+0000 7f9242dff700 1 -- start start 2023-11-06T20:45:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.519+0000 7f9242dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 0x7f923c137780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.519+0000 7f9242dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923c06a470 con 0x7f923c0c1020 2023-11-06T20:45:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.519+0000 7f9242dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923c06a190 con 0x7f923c0c3990 2023-11-06T20:45:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.519+0000 7f9242dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0bbab0 con 0x7f923c1373a0 2023-11-06T20:45:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.520+0000 7f923bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f923c0c1020 0x7f923c0c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58048/0 (socket says 172.21.15.33:58048) 2023-11-06T20:45:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.520+0000 7f923bfff700 1 -- 172.21.15.33:0/3327744553 learned_addr learned my addr 172.21.15.33:0/3327744553 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:17.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.520+0000 7f9240b9b700 1 --2- 172.21.15.33:0/3327744553 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 0x7f923c137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.520+0000 7f923b7fe700 1 -- 172.21.15.33:0/3327744553 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256168358 0 0) 0x7f923c06a470 con 0x7f923c0c1020 2023-11-06T20:45:17.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.520+0000 7f923b7fe700 1 -- 172.21.15.33:0/3327744553 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f922c0033f0 con 0x7f923c0c1020 2023-11-06T20:45:17.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.520+0000 7f923b7fe700 1 -- 172.21.15.33:0/3327744553 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058448399 0 0) 0x7f923c06a190 con 0x7f923c0c3990 2023-11-06T20:45:17.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.520+0000 7f923b7fe700 1 -- 172.21.15.33:0/3327744553 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f923c06a470 con 0x7f923c0c3990 2023-11-06T20:45:17.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.520+0000 7f9240b9b700 1 -- 172.21.15.33:0/3327744553 >> v1:172.21.15.69:6789/0 conn(0x7f923c0c3990 legacy=0x7f923c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.520+0000 7f9240b9b700 1 -- 172.21.15.33:0/3327744553 >> v1:172.21.15.33:6789/0 conn(0x7f923c0c1020 legacy=0x7f923c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.520+0000 7f9240b9b700 1 -- 172.21.15.33:0/3327744553 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c0bbd90 con 0x7f923c1373a0 2023-11-06T20:45:17.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.521+0000 7f9240b9b700 1 --2- 172.21.15.33:0/3327744553 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 0x7f923c137780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f9230009790 tx=0x7f9230009060).ready entity=mon.2 client_cookie=7f303dd5f3982c91 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:17.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.521+0000 7f923b7fe700 1 -- 172.21.15.33:0/3327744553 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000f510 con 0x7f923c1373a0 2023-11-06T20:45:17.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.521+0000 7f923b7fe700 1 -- 172.21.15.33:0/3327744553 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923000f670 con 0x7f923c1373a0 2023-11-06T20:45:17.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.521+0000 7f923b7fe700 1 -- 172.21.15.33:0/3327744553 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000f9f0 con 0x7f923c1373a0 2023-11-06T20:45:17.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.521+0000 7f9242dff700 1 -- 172.21.15.33:0/3327744553 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 msgr2=0x7f923c137780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.521+0000 7f9242dff700 1 --2- 172.21.15.33:0/3327744553 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 0x7f923c137780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f9230009790 tx=0x7f9230009060).stop 2023-11-06T20:45:17.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.522+0000 7f9242dff700 1 -- 172.21.15.33:0/3327744553 shutdown_connections 2023-11-06T20:45:17.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.522+0000 7f9242dff700 1 --2- 172.21.15.33:0/3327744553 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 0x7f923c137780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.522+0000 7f9242dff700 1 -- 172.21.15.33:0/3327744553 >> 172.21.15.33:0/3327744553 conn(0x7f923c062710 msgr2=0x7f923c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:17.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.522+0000 7f9242dff700 1 -- 172.21.15.33:0/3327744553 shutdown_connections 2023-11-06T20:45:17.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.522+0000 7f9242dff700 1 -- 172.21.15.33:0/3327744553 wait complete. 2023-11-06T20:45:17.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.523+0000 7f9242dff700 1 Processor -- start 2023-11-06T20:45:17.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.523+0000 7f9242dff700 1 -- start start 2023-11-06T20:45:17.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.524+0000 7f9242dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f923c0c1020 0x7f923c168a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.524+0000 7f9242dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923c0c3990 0x7f923c0be310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.524+0000 7f9242dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 0x7f923c0a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.524+0000 7f9242dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0bf3e0 con 0x7f923c0c1020 2023-11-06T20:45:17.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.524+0000 7f9242dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c152300 con 0x7f923c0c3990 2023-11-06T20:45:17.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.524+0000 7f9242dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c153f70 con 0x7f923c1373a0 2023-11-06T20:45:17.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.524+0000 7f9240b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f923c0c1020 0x7f923c168a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9240b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f923c0c1020 0x7f923c168a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36586/0 (socket says 172.21.15.33:36586) 2023-11-06T20:45:17.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9240b9b700 1 -- 172.21.15.33:0/2378315051 learned_addr learned my addr 172.21.15.33:0/2378315051 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:17.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f923bfff700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923c0c3990 0x7f923c0be310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f924139c700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 0x7f923c0a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9240b9b700 1 -- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 msgr2=0x7f923c0a8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9240b9b700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 0x7f923c0a8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9240b9b700 1 -- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923c0c3990 msgr2=0x7f923c0be310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9240b9b700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923c0c3990 0x7f923c0be310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9240b9b700 1 -- 172.21.15.33:0/2378315051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92300092a0 con 0x7f923c0c1020 2023-11-06T20:45:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f924139c700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 0x7f923c0a8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9240b9b700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f923c0c1020 0x7f923c168a80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f923c05efa0 tx=0x7f9230017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9238ff9700 1 -- 172.21.15.33:0/2378315051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000f7e0 con 0x7f923c0c1020 2023-11-06T20:45:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9238ff9700 1 -- 172.21.15.33:0/2378315051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923000f940 con 0x7f923c0c1020 2023-11-06T20:45:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9242dff700 1 -- 172.21.15.33:0/2378315051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c0beb40 con 0x7f923c0c1020 2023-11-06T20:45:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.525+0000 7f9238ff9700 1 -- 172.21.15.33:0/2378315051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923001b660 con 0x7f923c0c1020 2023-11-06T20:45:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.526+0000 7f9242dff700 1 -- 172.21.15.33:0/2378315051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c0bf100 con 0x7f923c0c1020 2023-11-06T20:45:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.527+0000 7f9242dff700 1 -- 172.21.15.33:0/2378315051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923c063280 con 0x7f923c0c1020 2023-11-06T20:45:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.527+0000 7f9238ff9700 1 -- 172.21.15.33:0/2378315051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f923001ba10 con 0x7f923c0c1020 2023-11-06T20:45:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.528+0000 7f9238ff9700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f922005f120 0x7f92200615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.529+0000 7f9238ff9700 1 -- 172.21.15.33:0/2378315051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f923007b5d0 con 0x7f923c0c1020 2023-11-06T20:45:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.529+0000 7f923bfff700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f922005f120 0x7f92200615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.530+0000 7f923bfff700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f922005f120 0x7f92200615e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f9228005750 tx=0x7f9228009040).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.533+0000 7f9238ff9700 1 -- 172.21.15.33:0/2378315051 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f923004c7e0 con 0x7f923c0c1020 2023-11-06T20:45:17.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.743+0000 7f9242dff700 1 -- 172.21.15.33:0/2378315051 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f923c065a00 con 0x7f922005f120 2023-11-06T20:45:17.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.745+0000 7f9238ff9700 1 -- 172.21.15.33:0/2378315051 <== mgr.4103 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f923c065a00 con 0x7f922005f120 2023-11-06T20:45:17.744 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T20:45:17.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:45:17.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T20:45:17.745 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:45:17.745 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T20:45:17.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 -- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f922005f120 msgr2=0x7f92200615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f922005f120 0x7f92200615e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f9228005750 tx=0x7f9228009040).stop 2023-11-06T20:45:17.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 -- 172.21.15.33:0/2378315051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f923c0c1020 msgr2=0x7f923c168a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f923c0c1020 0x7f923c168a80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f923c05efa0 tx=0x7f9230017000).stop 2023-11-06T20:45:17.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 -- 172.21.15.33:0/2378315051 shutdown_connections 2023-11-06T20:45:17.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f922005f120 0x7f92200615e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f923c1373a0 0x7f923c0a8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f923c0c1020 0x7f923c168a80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 --2- 172.21.15.33:0/2378315051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923c0c3990 0x7f923c0be310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 -- 172.21.15.33:0/2378315051 >> 172.21.15.33:0/2378315051 conn(0x7f923c062710 msgr2=0x7f923c0bc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:17.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 -- 172.21.15.33:0/2378315051 shutdown_connections 2023-11-06T20:45:17.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.747+0000 7f9242dff700 1 -- 172.21.15.33:0/2378315051 wait complete. 2023-11-06T20:45:17.764 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T20:45:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.964+0000 7f38615ff700 1 Processor -- start 2023-11-06T20:45:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.964+0000 7f38615ff700 1 -- start start 2023-11-06T20:45:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.965+0000 7f38615ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f385c157700 0x7f385c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.965+0000 7f38615ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f385c063840 0x7f385c0c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.965+0000 7f38615ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385c158c00 con 0x7f385c0aaf50 2023-11-06T20:45:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.965+0000 7f38615ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c0c18f0 con 0x7f385c157700 2023-11-06T20:45:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.965+0000 7f38615ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c157ff0 con 0x7f385c063840 2023-11-06T20:45:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f385c157700 0x7f385c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f385c157700 0x7f385c0c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60234/0 (socket says 172.21.15.33:60234) 2023-11-06T20:45:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385affd700 1 -- 172.21.15.33:0/28993337 learned_addr learned my addr 172.21.15.33:0/28993337 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385a7fc700 1 --1- 172.21.15.33:0/28993337 >> v1:172.21.15.33:6789/0 conn(0x7f385c0aaf50 0x7f385c063130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36538/0 (socket says 172.21.15.33:36538) 2023-11-06T20:45:17.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385b7fe700 1 --2- 172.21.15.33:0/28993337 >> v2:172.21.15.69:3300/0 conn(0x7f385c063840 0x7f385c0c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385affd700 1 -- 172.21.15.33:0/28993337 >> v2:172.21.15.69:3300/0 conn(0x7f385c063840 msgr2=0x7f385c0c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385affd700 1 --2- 172.21.15.33:0/28993337 >> v2:172.21.15.69:3300/0 conn(0x7f385c063840 0x7f385c0c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385affd700 1 -- 172.21.15.33:0/28993337 >> v1:172.21.15.33:6789/0 conn(0x7f385c0aaf50 legacy=0x7f385c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f3859ffb700 1 -- 172.21.15.33:0/28993337 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541083659 0 0) 0x7f385c158c00 con 0x7f385c0aaf50 2023-11-06T20:45:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385affd700 1 -- 172.21.15.33:0/28993337 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c1582d0 con 0x7f385c157700 2023-11-06T20:45:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385b7fe700 1 --2- 172.21.15.33:0/28993337 >> v2:172.21.15.69:3300/0 conn(0x7f385c063840 0x7f385c0c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.966+0000 7f385affd700 1 --2- 172.21.15.33:0/28993337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c157700 0x7f385c0c6f00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f385c0c5400 tx=0x7f384c0090f0).ready entity=mon.0 client_cookie=783ae21e62bcbbe3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f3859ffb700 1 -- 172.21.15.33:0/28993337 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c00b600 con 0x7f385c157700 2023-11-06T20:45:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f3859ffb700 1 -- 172.21.15.33:0/28993337 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c00b760 con 0x7f385c157700 2023-11-06T20:45:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f3859ffb700 1 -- 172.21.15.33:0/28993337 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c00bae0 con 0x7f385c157700 2023-11-06T20:45:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f38615ff700 1 -- 172.21.15.33:0/28993337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c157700 msgr2=0x7f385c0c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f38615ff700 1 --2- 172.21.15.33:0/28993337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c157700 0x7f385c0c6f00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f385c0c5400 tx=0x7f384c0090f0).stop 2023-11-06T20:45:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f38615ff700 1 -- 172.21.15.33:0/28993337 shutdown_connections 2023-11-06T20:45:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f38615ff700 1 --2- 172.21.15.33:0/28993337 >> v2:172.21.15.69:3300/0 conn(0x7f385c063840 0x7f385c0c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f38615ff700 1 --2- 172.21.15.33:0/28993337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c157700 0x7f385c0c6f00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f38615ff700 1 -- 172.21.15.33:0/28993337 >> 172.21.15.33:0/28993337 conn(0x7f385c05ec80 msgr2=0x7f385c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f38615ff700 1 -- 172.21.15.33:0/28993337 shutdown_connections 2023-11-06T20:45:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.967+0000 7f38615ff700 1 -- 172.21.15.33:0/28993337 wait complete. 2023-11-06T20:45:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.969+0000 7f38615ff700 1 Processor -- start 2023-11-06T20:45:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.969+0000 7f38615ff700 1 -- start start 2023-11-06T20:45:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f38615ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c063840 0x7f385c0c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f38615ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f385c0aaf50 0x7f385c0c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f38615ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f385c157700 0x7f385c166ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f38615ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0c2500 con 0x7f385c063840 2023-11-06T20:45:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f38615ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0c1ef0 con 0x7f385c157700 2023-11-06T20:45:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f38615ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c167690 con 0x7f385c0aaf50 2023-11-06T20:45:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f385affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c063840 0x7f385c0c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f385affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c063840 0x7f385c0c2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60236/0 (socket says 172.21.15.33:60236) 2023-11-06T20:45:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f385affd700 1 -- 172.21.15.33:0/2554090264 learned_addr learned my addr 172.21.15.33:0/2554090264 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f385a7fc700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f385c0aaf50 0x7f385c0c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.970+0000 7f385b7fe700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f385c157700 0x7f385c166ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f385affd700 1 -- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f385c0aaf50 msgr2=0x7f385c0c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f385affd700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f385c0aaf50 0x7f385c0c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f385affd700 1 -- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f385c157700 msgr2=0x7f385c166ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f385affd700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f385c157700 0x7f385c166ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f385affd700 1 -- 172.21.15.33:0/2554090264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c009370 con 0x7f385c063840 2023-11-06T20:45:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f385a7fc700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f385c0aaf50 0x7f385c0c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f385b7fe700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f385c157700 0x7f385c166ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f385affd700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c063840 0x7f385c0c2990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f384c009130 tx=0x7f384c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f3842ffd700 1 -- 172.21.15.33:0/2554090264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c00b8d0 con 0x7f385c063840 2023-11-06T20:45:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f3842ffd700 1 -- 172.21.15.33:0/2554090264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c00ba30 con 0x7f385c063840 2023-11-06T20:45:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f38615ff700 1 -- 172.21.15.33:0/2554090264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c167970 con 0x7f385c063840 2023-11-06T20:45:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f3842ffd700 1 -- 172.21.15.33:0/2554090264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c01c670 con 0x7f385c063840 2023-11-06T20:45:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.971+0000 7f38615ff700 1 -- 172.21.15.33:0/2554090264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c14e000 con 0x7f385c063840 2023-11-06T20:45:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.972+0000 7f38615ff700 1 -- 172.21.15.33:0/2554090264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385c0c27e0 con 0x7f385c063840 2023-11-06T20:45:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.973+0000 7f3842ffd700 1 -- 172.21.15.33:0/2554090264 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f384c01ca20 con 0x7f385c063840 2023-11-06T20:45:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.974+0000 7f3842ffd700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384805f150 0x7f3848061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.974+0000 7f3842ffd700 1 -- 172.21.15.33:0/2554090264 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f384c07c0e0 con 0x7f385c063840 2023-11-06T20:45:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.975+0000 7f385a7fc700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384805f150 0x7f3848061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.975+0000 7f385a7fc700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384805f150 0x7f3848061610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f385c1571b0 tx=0x7f3850007b10).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:17.978+0000 7f3842ffd700 1 -- 172.21.15.33:0/2554090264 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f384c04d2f0 con 0x7f385c063840 2023-11-06T20:45:18.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.209+0000 7f38615ff700 1 -- 172.21.15.33:0/2554090264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f385c06c6c0 con 0x7f385c063840 2023-11-06T20:45:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.901+0000 7f3842ffd700 1 -- 172.21.15.33:0/2554090264 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 5) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f384c079df0 con 0x7f385c063840 2023-11-06T20:45:18.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.902+0000 7f3842ffd700 1 -- 172.21.15.33:0/2554090264 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v5) v1 ==== 92+0+0 (secure 0 0 0) 0x7f384c042af0 con 0x7f385c063840 2023-11-06T20:45:18.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.905+0000 7f38615ff700 1 -- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384805f150 msgr2=0x7f3848061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:18.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.905+0000 7f38615ff700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384805f150 0x7f3848061610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f385c1571b0 tx=0x7f3850007b10).stop 2023-11-06T20:45:18.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.906+0000 7f38615ff700 1 -- 172.21.15.33:0/2554090264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c063840 msgr2=0x7f385c0c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:18.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.906+0000 7f38615ff700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c063840 0x7f385c0c2990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f384c009130 tx=0x7f384c017000).stop 2023-11-06T20:45:18.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.906+0000 7f38615ff700 1 -- 172.21.15.33:0/2554090264 shutdown_connections 2023-11-06T20:45:18.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.906+0000 7f38615ff700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384805f150 0x7f3848061610 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:18.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.906+0000 7f38615ff700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f385c157700 0x7f385c166ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:18.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.906+0000 7f38615ff700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f385c063840 0x7f385c0c2990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:18.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.906+0000 7f38615ff700 1 --2- 172.21.15.33:0/2554090264 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f385c0aaf50 0x7f385c0c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:18.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.906+0000 7f38615ff700 1 -- 172.21.15.33:0/2554090264 >> 172.21.15.33:0/2554090264 conn(0x7f385c05ec80 msgr2=0x7f385c0c9830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:18.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.906+0000 7f38615ff700 1 -- 172.21.15.33:0/2554090264 shutdown_connections 2023-11-06T20:45:18.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:18.906+0000 7f38615ff700 1 -- 172.21.15.33:0/2554090264 wait complete. 2023-11-06T20:45:18.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T20:45:19.112 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:19.112+0000 7f80dac1b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:45:19.132 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:19.133+0000 7f701d6a0680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:45:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.141+0000 7efc45fff700 1 Processor -- start 2023-11-06T20:45:19.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.141+0000 7efc45fff700 1 -- start start 2023-11-06T20:45:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.142+0000 7efc45fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efc40153f50 0x7efc40154330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.142+0000 7efc45fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 0x7efc400a5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc45fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc400bc010 con 0x7efc40064270 2023-11-06T20:45:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc45fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc40064e30 con 0x7efc40153f50 2023-11-06T20:45:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc45fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc40065110 con 0x7efc400a53c0 2023-11-06T20:45:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc3ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efc40064270 0x7efc400c3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36552/0 (socket says 172.21.15.33:36552) 2023-11-06T20:45:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc3ffff700 1 -- 172.21.15.33:0/443330838 learned_addr learned my addr 172.21.15.33:0/443330838 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc3effd700 1 --2- 172.21.15.33:0/443330838 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 0x7efc400a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc3f7fe700 1 --2- 172.21.15.33:0/443330838 >> v2:172.21.15.69:3300/0 conn(0x7efc40153f50 0x7efc40154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc3e7fc700 1 -- 172.21.15.33:0/443330838 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446602928 0 0) 0x7efc400bc010 con 0x7efc40064270 2023-11-06T20:45:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc3e7fc700 1 -- 172.21.15.33:0/443330838 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc280033f0 con 0x7efc40064270 2023-11-06T20:45:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc3effd700 1 -- 172.21.15.33:0/443330838 >> v2:172.21.15.69:3300/0 conn(0x7efc40153f50 msgr2=0x7efc40154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc3effd700 1 --2- 172.21.15.33:0/443330838 >> v2:172.21.15.69:3300/0 conn(0x7efc40153f50 0x7efc40154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc3effd700 1 -- 172.21.15.33:0/443330838 >> v1:172.21.15.33:6789/0 conn(0x7efc40064270 legacy=0x7efc400c3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.143+0000 7efc3effd700 1 -- 172.21.15.33:0/443330838 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc400a48e0 con 0x7efc400a53c0 2023-11-06T20:45:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.144+0000 7efc3f7fe700 1 --2- 172.21.15.33:0/443330838 >> v2:172.21.15.69:3300/0 conn(0x7efc40153f50 0x7efc40154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.144+0000 7efc3effd700 1 --2- 172.21.15.33:0/443330838 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 0x7efc400a5820 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7efc34009760 tx=0x7efc34009060).ready entity=mon.2 client_cookie=5e7a04efbfcda4e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.144+0000 7efc3e7fc700 1 -- 172.21.15.33:0/443330838 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3400d4a0 con 0x7efc400a53c0 2023-11-06T20:45:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.144+0000 7efc3e7fc700 1 -- 172.21.15.33:0/443330838 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3400d600 con 0x7efc400a53c0 2023-11-06T20:45:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.144+0000 7efc3e7fc700 1 -- 172.21.15.33:0/443330838 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3400d980 con 0x7efc400a53c0 2023-11-06T20:45:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.144+0000 7efc45fff700 1 -- 172.21.15.33:0/443330838 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 msgr2=0x7efc400a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.144+0000 7efc45fff700 1 --2- 172.21.15.33:0/443330838 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 0x7efc400a5820 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7efc34009760 tx=0x7efc34009060).stop 2023-11-06T20:45:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.145+0000 7efc45fff700 1 -- 172.21.15.33:0/443330838 shutdown_connections 2023-11-06T20:45:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.145+0000 7efc45fff700 1 --2- 172.21.15.33:0/443330838 >> v2:172.21.15.69:3300/0 conn(0x7efc40153f50 0x7efc40154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.145+0000 7efc45fff700 1 --2- 172.21.15.33:0/443330838 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 0x7efc400a5820 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.145+0000 7efc45fff700 1 -- 172.21.15.33:0/443330838 >> 172.21.15.33:0/443330838 conn(0x7efc40062710 msgr2=0x7efc40062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.145+0000 7efc45fff700 1 -- 172.21.15.33:0/443330838 shutdown_connections 2023-11-06T20:45:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.145+0000 7efc45fff700 1 -- 172.21.15.33:0/443330838 wait complete. 2023-11-06T20:45:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.146+0000 7efc45fff700 1 Processor -- start 2023-11-06T20:45:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.146+0000 7efc45fff700 1 -- start start 2023-11-06T20:45:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.147+0000 7efc45fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc40064270 0x7efc400bf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.147+0000 7efc45fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 0x7efc400bf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.147+0000 7efc45fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc40153f50 0x7efc400a7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.147+0000 7efc45fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc400a9c40 con 0x7efc40153f50 2023-11-06T20:45:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.147+0000 7efc45fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc400bfef0 con 0x7efc40064270 2023-11-06T20:45:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.147+0000 7efc45fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc400c0190 con 0x7efc400a53c0 2023-11-06T20:45:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.147+0000 7efc3ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc40153f50 0x7efc400a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc3effd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 0x7efc400bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc3f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc40064270 0x7efc400bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc3ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc40153f50 0x7efc400a7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60264/0 (socket says 172.21.15.33:60264) 2023-11-06T20:45:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc3ffff700 1 -- 172.21.15.33:0/1997973179 learned_addr learned my addr 172.21.15.33:0/1997973179 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc3ffff700 1 -- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 msgr2=0x7efc400bf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc3ffff700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 0x7efc400bf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc3ffff700 1 -- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc40064270 msgr2=0x7efc400bf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc3ffff700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc40064270 0x7efc400bf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc3ffff700 1 -- 172.21.15.33:0/1997973179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300092a0 con 0x7efc40153f50 2023-11-06T20:45:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc3ffff700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc40153f50 0x7efc400a7570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7efc38005ad0 tx=0x7efc38001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc277fe700 1 -- 172.21.15.33:0/1997973179 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38005d30 con 0x7efc40153f50 2023-11-06T20:45:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.148+0000 7efc277fe700 1 -- 172.21.15.33:0/1997973179 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc38005e90 con 0x7efc40153f50 2023-11-06T20:45:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.149+0000 7efc277fe700 1 -- 172.21.15.33:0/1997973179 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38007320 con 0x7efc40153f50 2023-11-06T20:45:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.149+0000 7efc45fff700 1 -- 172.21.15.33:0/1997973179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc340092a0 con 0x7efc40153f50 2023-11-06T20:45:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.149+0000 7efc45fff700 1 -- 172.21.15.33:0/1997973179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc400bfbf0 con 0x7efc40153f50 2023-11-06T20:45:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.150+0000 7efc45fff700 1 -- 172.21.15.33:0/1997973179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc400a9a10 con 0x7efc40153f50 2023-11-06T20:45:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.150+0000 7efc277fe700 1 -- 172.21.15.33:0/1997973179 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 75463+0+0 (secure 0 0 0) 0x7efc38007710 con 0x7efc40153f50 2023-11-06T20:45:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.152+0000 7efc277fe700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc2c05f170 0x7efc2c061630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.152+0000 7efc277fe700 1 -- 172.21.15.33:0/1997973179 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efc3807a220 con 0x7efc40153f50 2023-11-06T20:45:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.152+0000 7efc3f7fe700 1 -- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc2c05f170 msgr2=0x7efc2c061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:45:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.152+0000 7efc3f7fe700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc2c05f170 0x7efc2c061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:45:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.156+0000 7efc277fe700 1 -- 172.21.15.33:0/1997973179 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efc400a9a10 con 0x7efc40153f50 2023-11-06T20:45:19.191 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:19.191+0000 7f80dac1b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:45:19.218 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:19.219+0000 7f701d6a0680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:45:19.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.337+0000 7efc45fff700 1 -- 172.21.15.33:0/1997973179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7efc400a9a10 con 0x7efc40153f50 2023-11-06T20:45:19.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.342+0000 7efc277fe700 1 -- 172.21.15.33:0/1997973179 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v3)=0 v3) v1 ==== 138+0+0 (secure 0 0 0) 0x7efc400a9a10 con 0x7efc40153f50 2023-11-06T20:45:19.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 -- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc2c05f170 msgr2=0x7efc2c061630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:45:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc2c05f170 0x7efc2c061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:45:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 -- 172.21.15.33:0/1997973179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc40153f50 msgr2=0x7efc400a7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc40153f50 0x7efc400a7570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7efc38005ad0 tx=0x7efc38001650).stop 2023-11-06T20:45:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 -- 172.21.15.33:0/1997973179 shutdown_connections 2023-11-06T20:45:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc2c05f170 0x7efc2c061630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:45:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc40153f50 0x7efc400a7570 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc40064270 0x7efc400bf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 --2- 172.21.15.33:0/1997973179 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc400a53c0 0x7efc400bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 -- 172.21.15.33:0/1997973179 >> 172.21.15.33:0/1997973179 conn(0x7efc40062710 msgr2=0x7efc401520c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 -- 172.21.15.33:0/1997973179 shutdown_connections 2023-11-06T20:45:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.344+0000 7efc45fff700 1 -- 172.21.15.33:0/1997973179 wait complete. 2023-11-06T20:45:19.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:19.362 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-06T20:45:19.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:19.418 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-11-06T20:45:19.468 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-11-06T20:45:19.522 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:19.522+0000 7f80dac1b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:45:19.576 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:19.577+0000 7f701d6a0680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:45:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.674+0000 7f0fd9dff700 1 Processor -- start 2023-11-06T20:45:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.674+0000 7f0fd9dff700 1 -- start start 2023-11-06T20:45:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.675+0000 7f0fd9dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0fd40c8ee0 0x7f0fd40ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.676+0000 7f0fd9dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd40c9f80 con 0x7f0fd4153ed0 2023-11-06T20:45:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.676+0000 7f0fd9dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd40c6000 con 0x7f0fd41542b0 2023-11-06T20:45:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.676+0000 7f0fd9dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd40c9370 con 0x7f0fd40c8ee0 2023-11-06T20:45:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.676+0000 7f0fd37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0fd4153ed0 0x7f0fd4158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36558/0 (socket says 172.21.15.33:36558) 2023-11-06T20:45:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.676+0000 7f0fd37fe700 1 -- 172.21.15.33:0/2776900021 learned_addr learned my addr 172.21.15.33:0/2776900021 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.676+0000 7f0fd2ffd700 1 --2- 172.21.15.33:0/2776900021 >> v2:172.21.15.69:3300/0 conn(0x7f0fd40c8ee0 0x7f0fd40ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.676+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228903474 0 0) 0x7f0fd40c9f80 con 0x7f0fd4153ed0 2023-11-06T20:45:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fbc0033f0 con 0x7f0fd4153ed0 2023-11-06T20:45:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3237205124 0 0) 0x7f0fbc0033f0 con 0x7f0fd4153ed0 2023-11-06T20:45:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 >> v2:172.21.15.69:3300/0 conn(0x7f0fd40c8ee0 msgr2=0x7f0fd40ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 --2- 172.21.15.33:0/2776900021 >> v2:172.21.15.69:3300/0 conn(0x7f0fd40c8ee0 0x7f0fd40ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 >> v1:172.21.15.69:6789/0 conn(0x7f0fd41542b0 legacy=0x7f0fd40c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd40c9650 con 0x7f0fd4153ed0 2023-11-06T20:45:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f0fc4001470 con 0x7f0fd4153ed0 2023-11-06T20:45:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 >> v1:172.21.15.33:6789/0 conn(0x7f0fd4153ed0 legacy=0x7f0fd4158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fbc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fbc004690 0x7f0fbc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fbc005000 0x7f0fbc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40c9370 con 0x7f0fbc0038e0 2023-11-06T20:45:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc00bf60 con 0x7f0fbc004690 2023-11-06T20:45:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc00c0f0 con 0x7f0fbc005000 2023-11-06T20:45:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fbc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd2ffd700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fbc004690 0x7f0fbc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.677+0000 7f0fd3fff700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fbc005000 0x7f0fbc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd37fe700 1 -- 172.21.15.33:0/2776900021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fbc005000 msgr2=0x7f0fbc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:19.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fbc005000 0x7f0fbc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd37fe700 1 -- 172.21.15.33:0/2776900021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fbc004690 msgr2=0x7f0fbc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:19.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fbc004690 0x7f0fbc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd37fe700 1 -- 172.21.15.33:0/2776900021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0033f0 con 0x7f0fbc0038e0 2023-11-06T20:45:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd3fff700 1 -- 172.21.15.33:0/2776900021 reap_dead start 2023-11-06T20:45:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fbc003d00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f0fc400c190 tx=0x7f0fc4001e50).ready entity=mon.0 client_cookie=1fe51d4376346261 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4007a20 con 0x7f0fbc0038e0 2023-11-06T20:45:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc400cac0 con 0x7f0fbc0038e0 2023-11-06T20:45:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.678+0000 7f0fd27fc700 1 -- 172.21.15.33:0/2776900021 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc400ce80 con 0x7f0fbc0038e0 2023-11-06T20:45:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.679+0000 7f0fd9dff700 1 -- 172.21.15.33:0/2776900021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 msgr2=0x7f0fbc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.679+0000 7f0fd9dff700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fbc003d00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f0fc400c190 tx=0x7f0fc4001e50).stop 2023-11-06T20:45:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.679+0000 7f0fd9dff700 1 -- 172.21.15.33:0/2776900021 shutdown_connections 2023-11-06T20:45:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.679+0000 7f0fd9dff700 1 --2- 172.21.15.33:0/2776900021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fbc003d00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.679+0000 7f0fd9dff700 1 -- 172.21.15.33:0/2776900021 >> 172.21.15.33:0/2776900021 conn(0x7f0fd405e1b0 msgr2=0x7f0fd405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.679+0000 7f0fd9dff700 1 -- 172.21.15.33:0/2776900021 shutdown_connections 2023-11-06T20:45:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.679+0000 7f0fd9dff700 1 -- 172.21.15.33:0/2776900021 wait complete. 2023-11-06T20:45:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.681+0000 7f0fd9dff700 1 Processor -- start 2023-11-06T20:45:19.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.681+0000 7f0fd9dff700 1 -- start start 2023-11-06T20:45:19.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd9dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fd414ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd9dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd4156dd0 0x7f0fd4154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd9dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd4157250 0x7f0fd40aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd9dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40aea30 con 0x7f0fbc0038e0 2023-11-06T20:45:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd9dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd4164340 con 0x7f0fd4156dd0 2023-11-06T20:45:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd9dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40ae190 con 0x7f0fd4157250 2023-11-06T20:45:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fd414ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd4157250 0x7f0fd40aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd4156dd0 0x7f0fd4154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:19.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fd414ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60278/0 (socket says 172.21.15.33:60278) 2023-11-06T20:45:19.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.682+0000 7f0fd37fe700 1 -- 172.21.15.33:0/3205387842 learned_addr learned my addr 172.21.15.33:0/3205387842 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:19.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fd2ffd700 1 -- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd4157250 msgr2=0x7f0fd40aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:19.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fd2ffd700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd4157250 0x7f0fd40aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fd2ffd700 1 -- 172.21.15.33:0/3205387842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 msgr2=0x7f0fd414ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fd2ffd700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fd414ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fd2ffd700 1 -- 172.21.15.33:0/3205387842 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd40ae470 con 0x7f0fd4156dd0 2023-11-06T20:45:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fd3fff700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd4157250 0x7f0fd40aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fd414ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fd2ffd700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd4156dd0 0x7f0fd4154ed0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f0fd4071e50 tx=0x7f0fc8009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8013680 con 0x7f0fd4156dd0 2023-11-06T20:45:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc80137e0 con 0x7f0fd4156dd0 2023-11-06T20:45:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fd9dff700 1 -- 172.21.15.33:0/3205387842 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd40ae750 con 0x7f0fd4156dd0 2023-11-06T20:45:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8013b60 con 0x7f0fd4156dd0 2023-11-06T20:45:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.683+0000 7f0fd9dff700 1 -- 172.21.15.33:0/3205387842 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd41553c0 con 0x7f0fd4156dd0 2023-11-06T20:45:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.685+0000 7f0fd9dff700 1 -- 172.21.15.33:0/3205387842 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd4155670 con 0x7f0fd4156dd0 2023-11-06T20:45:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.685+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f0fc8014440 con 0x7f0fd4156dd0 2023-11-06T20:45:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.686+0000 7f0fbb7fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc0061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.686+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0fc807aa90 con 0x7f0fd4156dd0 2023-11-06T20:45:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.686+0000 7f0fd37fe700 1 -- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 msgr2=0x7f0fc0061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:45:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.686+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:45:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.691+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0fc804bc80 con 0x7f0fd4156dd0 2023-11-06T20:45:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.887+0000 7f0fd37fe700 1 -- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 msgr2=0x7f0fc0061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:45:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.887+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:45:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:19.901+0000 7f0fd9dff700 1 -- 172.21.15.33:0/3205387842 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fd406c6c0 con 0x7f0fc005f1a0 2023-11-06T20:45:20.269 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:20.269+0000 7f80dac1b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:45:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:20.288+0000 7f0fd37fe700 1 -- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 msgr2=0x7f0fc0061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:45:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:20.288+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T20:45:20.351 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:20.352+0000 7f80dac1b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:45:20.369 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:20.370+0000 7f701d6a0680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:45:20.457 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:20.458+0000 7f701d6a0680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:45:20.580 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:20.581+0000 7f80dac1b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:45:20.697 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:20.698+0000 7f701d6a0680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:45:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:21.089+0000 7f0fd37fe700 1 -- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 msgr2=0x7f0fc0061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:45:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:21.089+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T20:45:21.093 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:21.093+0000 7f80dac1b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:45:21.216 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:21.216+0000 7f80dac1b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:45:21.233 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:21.234+0000 7f701d6a0680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:45:21.365 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:21.366+0000 7f701d6a0680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:45:21.399 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:21.399+0000 7f80dac1b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:45:21.495 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:21.495+0000 7f80dac1b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:45:21.556 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:21.557+0000 7f701d6a0680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:45:21.581 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:21.581+0000 7f80dac1b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:45:21.655 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:21.656+0000 7f701d6a0680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:45:21.744 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:21.745+0000 7f701d6a0680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:45:21.887 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:21.887+0000 7f80dac1b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:45:22.073 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:22.074+0000 7f701d6a0680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:45:22.106 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:22.106+0000 7f80dac1b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:45:22.303 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:22.304+0000 7f701d6a0680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:45:22.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:22.691+0000 7f0fd37fe700 1 -- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 msgr2=0x7f0fc0061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:45:22.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:22.691+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T20:45:22.906 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:22.907+0000 7f80dac1b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:45:22.992 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:22.992+0000 7f80dac1b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:45:23.084 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:23.084+0000 7f80dac1b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:45:23.148 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:23.149+0000 7f701d6a0680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:45:23.238 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:23.239+0000 7f701d6a0680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:45:23.267 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:23.267+0000 7f80dac1b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:45:23.335 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:23.336+0000 7f701d6a0680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:45:23.354 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:23.354+0000 7f80dac1b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:45:23.492 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:23.493+0000 7f80dac1b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:45:23.527 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:23.528+0000 7f701d6a0680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:45:23.619 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:23.620+0000 7f701d6a0680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:45:23.621 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:23.622+0000 7f80dac1b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:45:23.766 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:23.768+0000 7f701d6a0680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:45:23.903 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:23.904+0000 7f701d6a0680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:45:23.962 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:23.963+0000 7f80dac1b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:45:24.050 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.051+0000 7f80dac1b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:45:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:24.063+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 6) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f0fc80787e0 con 0x7f0fd4156dd0 2023-11-06T20:45:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:24.063+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 msgr2=0x7f0fc0061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:45:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:24.063+0000 7f0fbb7fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:45:24.138 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.138+0000 7f8050707700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:45:24.138 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.138+0000 7f8050707700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:45:24.139 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.138+0000 7f8050707700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:45:24.139 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.138+0000 7f8050707700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:45:24.139 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.138+0000 7f8050707700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:45:24.140 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.140+0000 7f804d701700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:45:24.140 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.140+0000 7f804d701700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:45:24.140 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.140+0000 7f804d701700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:45:24.140 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.140+0000 7f804d701700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:45:24.141 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:45:24.140+0000 7f804d701700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:45:24.265 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:24.266+0000 7f701d6a0680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:45:24.358 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:45:24.359+0000 7f701d6a0680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:45:25.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.069+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 <== mon.1 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f0fc8078b60 con 0x7f0fd4156dd0 2023-11-06T20:45:25.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.069+0000 7f0fbb7fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.069+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fd406c6c0 con 0x7f0fc005f1a0 2023-11-06T20:45:25.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.070+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.074+0000 7f0fd37fe700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc00615e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f0fc400ca90 tx=0x7f0fc4015000).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:25.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.103+0000 7f0fbb7fe700 1 -- 172.21.15.33:0/3205387842 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f0fd406c6c0 con 0x7f0fc005f1a0 2023-11-06T20:45:25.103 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/dir0 2023-11-06T20:45:25.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 -- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 msgr2=0x7f0fc00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc00615e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f0fc400ca90 tx=0x7f0fc4015000).stop 2023-11-06T20:45:25.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 -- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd4156dd0 msgr2=0x7f0fd4154ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd4156dd0 0x7f0fd4154ed0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f0fd4071e50 tx=0x7f0fc8009060).stop 2023-11-06T20:45:25.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 -- 172.21.15.33:0/3205387842 shutdown_connections 2023-11-06T20:45:25.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fc005f1a0 0x7f0fc00615e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd4157250 0x7f0fd40aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc0038e0 0x7f0fd414ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 --2- 172.21.15.33:0/3205387842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd4156dd0 0x7f0fd4154ed0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 -- 172.21.15.33:0/3205387842 >> 172.21.15.33:0/3205387842 conn(0x7f0fd405e1b0 msgr2=0x7f0fd40c0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:25.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 -- 172.21.15.33:0/3205387842 shutdown_connections 2023-11-06T20:45:25.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.106+0000 7f0fd9dff700 1 -- 172.21.15.33:0/3205387842 wait complete. 2023-11-06T20:45:25.128 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-11-06T20:45:25.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:25.128 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-11-06T20:45:25.180 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-11-06T20:45:25.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.382+0000 7efe0b9ff700 1 Processor -- start 2023-11-06T20:45:25.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.382+0000 7efe0b9ff700 1 -- start start 2023-11-06T20:45:25.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.383+0000 7efe0b9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efe04153ed0 0x7efe04158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.383+0000 7efe0b9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe040c9f80 con 0x7efe040c8ee0 2023-11-06T20:45:25.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.383+0000 7efe0b9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe040cab20 con 0x7efe041542b0 2023-11-06T20:45:25.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.383+0000 7efe0b9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe040c6000 con 0x7efe04153ed0 2023-11-06T20:45:25.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.383+0000 7efe08f9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efe040c8ee0 0x7efe040ca410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36572/0 (socket says 172.21.15.33:36572) 2023-11-06T20:45:25.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.383+0000 7efe08f9a700 1 -- 172.21.15.33:0/460220617 learned_addr learned my addr 172.21.15.33:0/460220617 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:25.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efe0979b700 1 --2- 172.21.15.33:0/460220617 >> v2:172.21.15.69:3300/0 conn(0x7efe04153ed0 0x7efe04158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677403079 0 0) 0x7efe040c9f80 con 0x7efe040c8ee0 2023-11-06T20:45:25.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdf00033f0 con 0x7efe040c8ee0 2023-11-06T20:45:25.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4186755887 0 0) 0x7efdf00033f0 con 0x7efe040c8ee0 2023-11-06T20:45:25.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 >> v2:172.21.15.69:3300/0 conn(0x7efe04153ed0 msgr2=0x7efe04158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 --2- 172.21.15.33:0/460220617 >> v2:172.21.15.69:3300/0 conn(0x7efe04153ed0 0x7efe04158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 >> v1:172.21.15.69:6789/0 conn(0x7efe041542b0 legacy=0x7efe040c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe040c9370 con 0x7efe040c8ee0 2023-11-06T20:45:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7efdf4001470 con 0x7efe040c8ee0 2023-11-06T20:45:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 >> v1:172.21.15.33:6789/0 conn(0x7efe040c8ee0 legacy=0x7efe040ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efdf00038e0 0x7efdf0003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdf0004690 0x7efdf0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdf0005000 0x7efdf000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe040c6000 con 0x7efdf0004690 2023-11-06T20:45:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf000beb0 con 0x7efdf0005000 2023-11-06T20:45:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.384+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf000c070 con 0x7efdf00038e0 2023-11-06T20:45:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efe08f9a700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdf0004690 0x7efdf0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efe0979b700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efdf00038e0 0x7efdf0003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efe09f9c700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdf0005000 0x7efdf000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efe08f9a700 1 -- 172.21.15.33:0/460220617 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efdf00038e0 msgr2=0x7efdf0003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efe08f9a700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efdf00038e0 0x7efdf0003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efe08f9a700 1 -- 172.21.15.33:0/460220617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdf0005000 msgr2=0x7efdf000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efe08f9a700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdf0005000 0x7efdf000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efe08f9a700 1 -- 172.21.15.33:0/460220617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf00033f0 con 0x7efdf0004690 2023-11-06T20:45:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efe09f9c700 1 -- 172.21.15.33:0/460220617 reap_dead start 2023-11-06T20:45:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efe08f9a700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdf0004690 0x7efdf0004b10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7efdf400c190 tx=0x7efdf4001e70).ready entity=mon.0 client_cookie=f747a273a2d9dfab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4007a40 con 0x7efdf0004690 2023-11-06T20:45:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf400ca60 con 0x7efdf0004690 2023-11-06T20:45:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.385+0000 7efdfbfff700 1 -- 172.21.15.33:0/460220617 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400ce00 con 0x7efdf0004690 2023-11-06T20:45:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.386+0000 7efe0b9ff700 1 -- 172.21.15.33:0/460220617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdf0004690 msgr2=0x7efdf0004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.386+0000 7efe0b9ff700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdf0004690 0x7efdf0004b10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7efdf400c190 tx=0x7efdf4001e70).stop 2023-11-06T20:45:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.386+0000 7efe0b9ff700 1 -- 172.21.15.33:0/460220617 shutdown_connections 2023-11-06T20:45:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.386+0000 7efe0b9ff700 1 --2- 172.21.15.33:0/460220617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdf0004690 0x7efdf0004b10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.386+0000 7efe0b9ff700 1 -- 172.21.15.33:0/460220617 >> 172.21.15.33:0/460220617 conn(0x7efe0405e1b0 msgr2=0x7efe0405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.386+0000 7efe0b9ff700 1 -- 172.21.15.33:0/460220617 shutdown_connections 2023-11-06T20:45:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.386+0000 7efe0b9ff700 1 -- 172.21.15.33:0/460220617 wait complete. 2023-11-06T20:45:25.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.387+0000 7efe0b9ff700 1 Processor -- start 2023-11-06T20:45:25.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.388+0000 7efe0b9ff700 1 -- start start 2023-11-06T20:45:25.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.388+0000 7efe0b9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdf0004690 0x7efe0414fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.388+0000 7efe0b9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe04156960 0x7efe040addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe0b9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe040b01b0 0x7efe040b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe0b9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe040aea80 con 0x7efe04156960 2023-11-06T20:45:25.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe0b9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe040c2fe0 con 0x7efdf0004690 2023-11-06T20:45:25.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe0b9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe040c3960 con 0x7efe040b01b0 2023-11-06T20:45:25.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe08f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe04156960 0x7efe040addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe08f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe04156960 0x7efe040addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60300/0 (socket says 172.21.15.33:60300) 2023-11-06T20:45:25.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe08f9a700 1 -- 172.21.15.33:0/3937361168 learned_addr learned my addr 172.21.15.33:0/3937361168 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:25.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe0979b700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdf0004690 0x7efe0414fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe09f9c700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe040b01b0 0x7efe040b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe08f9a700 1 -- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe040b01b0 msgr2=0x7efe040b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe08f9a700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe040b01b0 0x7efe040b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe08f9a700 1 -- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdf0004690 msgr2=0x7efe0414fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe08f9a700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdf0004690 0x7efe0414fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe08f9a700 1 -- 172.21.15.33:0/3937361168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe040c2d00 con 0x7efe04156960 2023-11-06T20:45:25.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.389+0000 7efe08f9a700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe04156960 0x7efe040addd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7efdf400c780 tx=0x7efdf4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:25.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.390+0000 7efdf97fa700 1 -- 172.21.15.33:0/3937361168 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf40074f0 con 0x7efe04156960 2023-11-06T20:45:25.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.390+0000 7efdf97fa700 1 -- 172.21.15.33:0/3937361168 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf4018070 con 0x7efe04156960 2023-11-06T20:45:25.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.390+0000 7efdf97fa700 1 -- 172.21.15.33:0/3937361168 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf40046b0 con 0x7efe04156960 2023-11-06T20:45:25.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.390+0000 7efe0b9ff700 1 -- 172.21.15.33:0/3937361168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe040c3310 con 0x7efe04156960 2023-11-06T20:45:25.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.390+0000 7efe0b9ff700 1 -- 172.21.15.33:0/3937361168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe041501c0 con 0x7efe04156960 2023-11-06T20:45:25.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.391+0000 7efe0b9ff700 1 -- 172.21.15.33:0/3937361168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0406c6c0 con 0x7efe04156960 2023-11-06T20:45:25.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.391+0000 7efdf97fa700 1 -- 172.21.15.33:0/3937361168 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7efdf4004ae0 con 0x7efe04156960 2023-11-06T20:45:25.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.393+0000 7efdf97fa700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde805f080 0x7efde8061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.393+0000 7efdf97fa700 1 -- 172.21.15.33:0/3937361168 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7efdf407aa60 con 0x7efe04156960 2023-11-06T20:45:25.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.393+0000 7efe0979b700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde805f080 0x7efde8061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.394+0000 7efe0979b700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde805f080 0x7efde8061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7efe0405dd60 tx=0x7efe00009060).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:25.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.397+0000 7efdf97fa700 1 -- 172.21.15.33:0/3937361168 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efdf404bdc0 con 0x7efe04156960 2023-11-06T20:45:25.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.606+0000 7efe0b9ff700 1 -- 172.21.15.33:0/3937361168 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efe040c9650 con 0x7efde805f080 2023-11-06T20:45:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.613+0000 7efdf97fa700 1 -- 172.21.15.33:0/3937361168 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7efe040c9650 con 0x7efde805f080 2023-11-06T20:45:25.613 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/dir1 2023-11-06T20:45:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.614+0000 7efe0b9ff700 1 -- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde805f080 msgr2=0x7efde8061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.614+0000 7efe0b9ff700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde805f080 0x7efde8061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7efe0405dd60 tx=0x7efe00009060).stop 2023-11-06T20:45:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.614+0000 7efe0b9ff700 1 -- 172.21.15.33:0/3937361168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe04156960 msgr2=0x7efe040addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.614+0000 7efe0b9ff700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe04156960 0x7efe040addd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7efdf400c780 tx=0x7efdf4015000).stop 2023-11-06T20:45:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.615+0000 7efe0b9ff700 1 -- 172.21.15.33:0/3937361168 shutdown_connections 2023-11-06T20:45:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.615+0000 7efe0b9ff700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde805f080 0x7efde8061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.615+0000 7efe0b9ff700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe040b01b0 0x7efe040b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.615+0000 7efe0b9ff700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdf0004690 0x7efe0414fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.615+0000 7efe0b9ff700 1 --2- 172.21.15.33:0/3937361168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe04156960 0x7efe040addd0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.615+0000 7efe0b9ff700 1 -- 172.21.15.33:0/3937361168 >> 172.21.15.33:0/3937361168 conn(0x7efe0405e1b0 msgr2=0x7efe040c16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.615+0000 7efe0b9ff700 1 -- 172.21.15.33:0/3937361168 shutdown_connections 2023-11-06T20:45:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.615+0000 7efe0b9ff700 1 -- 172.21.15.33:0/3937361168 wait complete. 2023-11-06T20:45:25.632 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-11-06T20:45:25.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:25.633 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-11-06T20:45:25.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-11-06T20:45:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.883+0000 7f3e6bfff700 1 Processor -- start 2023-11-06T20:45:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.883+0000 7f3e6bfff700 1 -- start start 2023-11-06T20:45:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.884+0000 7f3e6bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e6c0ca410 0x7f3e6c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.884+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3e6c0c60c0 0x7f3e6c064510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.884+0000 7f3e6bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6c155170 con 0x7f3e6c064a00 2023-11-06T20:45:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.884+0000 7f3e6bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6c065240 con 0x7f3e6c0ca410 2023-11-06T20:45:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.884+0000 7f3e6bfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c154630 con 0x7f3e6c0c60c0 2023-11-06T20:45:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e6affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e6c0ca410 0x7f3e6c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e6b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3e6c064a00 0x7f3e6c14def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36584/0 (socket says 172.21.15.33:36584) 2023-11-06T20:45:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e6b7fe700 1 -- 172.21.15.33:0/4158181855 learned_addr learned my addr 172.21.15.33:0/4158181855 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e6a7fc700 1 --2- 172.21.15.33:0/4158181855 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3e6c0c60c0 0x7f3e6c064510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e69ffb700 1 -- 172.21.15.33:0/4158181855 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791492242 0 0) 0x7f3e6c155170 con 0x7f3e6c064a00 2023-11-06T20:45:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e69ffb700 1 -- 172.21.15.33:0/4158181855 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e540033f0 con 0x7f3e6c064a00 2023-11-06T20:45:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e6affd700 1 -- 172.21.15.33:0/4158181855 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3e6c0c60c0 msgr2=0x7f3e6c064510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e6affd700 1 --2- 172.21.15.33:0/4158181855 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3e6c0c60c0 0x7f3e6c064510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e6affd700 1 -- 172.21.15.33:0/4158181855 >> v1:172.21.15.33:6789/0 conn(0x7f3e6c064a00 legacy=0x7f3e6c14def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e6affd700 1 -- 172.21.15.33:0/4158181855 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0ca7f0 con 0x7f3e6c0ca410 2023-11-06T20:45:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.885+0000 7f3e6affd700 1 --2- 172.21.15.33:0/4158181855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c0ca410 0x7f3e6c0c5bd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f3e5c009760 tx=0x7f3e5c009060).ready entity=mon.0 client_cookie=71dfedac015b74c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e69ffb700 1 -- 172.21.15.33:0/4158181855 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00d4a0 con 0x7f3e6c0ca410 2023-11-06T20:45:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e69ffb700 1 -- 172.21.15.33:0/4158181855 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c00d600 con 0x7f3e6c0ca410 2023-11-06T20:45:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e69ffb700 1 -- 172.21.15.33:0/4158181855 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00d980 con 0x7f3e6c0ca410 2023-11-06T20:45:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e6bfff700 1 -- 172.21.15.33:0/4158181855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c0ca410 msgr2=0x7f3e6c0c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e6bfff700 1 --2- 172.21.15.33:0/4158181855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c0ca410 0x7f3e6c0c5bd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f3e5c009760 tx=0x7f3e5c009060).stop 2023-11-06T20:45:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e6bfff700 1 -- 172.21.15.33:0/4158181855 shutdown_connections 2023-11-06T20:45:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e6bfff700 1 --2- 172.21.15.33:0/4158181855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c0ca410 0x7f3e6c0c5bd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e6bfff700 1 --2- 172.21.15.33:0/4158181855 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3e6c0c60c0 0x7f3e6c064510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e6bfff700 1 -- 172.21.15.33:0/4158181855 >> 172.21.15.33:0/4158181855 conn(0x7f3e6c0cb6f0 msgr2=0x7f3e6c0c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e6bfff700 1 -- 172.21.15.33:0/4158181855 shutdown_connections 2023-11-06T20:45:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.886+0000 7f3e6bfff700 1 -- 172.21.15.33:0/4158181855 wait complete. 2023-11-06T20:45:25.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.888+0000 7f3e6bfff700 1 Processor -- start 2023-11-06T20:45:25.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.888+0000 7f3e6bfff700 1 -- start start 2023-11-06T20:45:25.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c064a00 0x7f3e6c14e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c0c60c0 0x7f3e6c156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3e6c0ca410 0x7f3e6c156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c156c00 con 0x7f3e6c064a00 2023-11-06T20:45:25.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6bfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0ae850 con 0x7f3e6c0c60c0 2023-11-06T20:45:25.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6bfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0ac1f0 con 0x7f3e6c0ca410 2023-11-06T20:45:25.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c064a00 0x7f3e6c14e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c064a00 0x7f3e6c14e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60318/0 (socket says 172.21.15.33:60318) 2023-11-06T20:45:25.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6affd700 1 -- 172.21.15.33:0/532518736 learned_addr learned my addr 172.21.15.33:0/532518736 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:25.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6b7fe700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3e6c0ca410 0x7f3e6c156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.889+0000 7f3e6a7fc700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c0c60c0 0x7f3e6c156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e6affd700 1 -- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3e6c0ca410 msgr2=0x7f3e6c156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e6affd700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3e6c0ca410 0x7f3e6c156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e6affd700 1 -- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c0c60c0 msgr2=0x7f3e6c156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:25.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e6affd700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c0c60c0 0x7f3e6c156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:25.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e6affd700 1 -- 172.21.15.33:0/532518736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0092a0 con 0x7f3e6c064a00 2023-11-06T20:45:25.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e6a7fc700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c0c60c0 0x7f3e6c156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:25.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e6affd700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c064a00 0x7f3e6c14e8c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f3e6c084210 tx=0x7f3e5c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:25.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e52ffd700 1 -- 172.21.15.33:0/532518736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c000c90 con 0x7f3e6c064a00 2023-11-06T20:45:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e52ffd700 1 -- 172.21.15.33:0/532518736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c00d770 con 0x7f3e6c064a00 2023-11-06T20:45:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e6bfff700 1 -- 172.21.15.33:0/532518736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c0acce0 con 0x7f3e6c064a00 2023-11-06T20:45:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e52ffd700 1 -- 172.21.15.33:0/532518736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c01d430 con 0x7f3e6c064a00 2023-11-06T20:45:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.890+0000 7f3e6bfff700 1 -- 172.21.15.33:0/532518736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c0aeb10 con 0x7f3e6c064a00 2023-11-06T20:45:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.891+0000 7f3e6bfff700 1 -- 172.21.15.33:0/532518736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c0475a0 con 0x7f3e6c064a00 2023-11-06T20:45:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.892+0000 7f3e52ffd700 1 -- 172.21.15.33:0/532518736 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f3e5c01d820 con 0x7f3e6c064a00 2023-11-06T20:45:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.893+0000 7f3e52ffd700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e5805f050 0x7f3e58061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.893+0000 7f3e52ffd700 1 -- 172.21.15.33:0/532518736 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f3e5c07b9c0 con 0x7f3e6c064a00 2023-11-06T20:45:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.893+0000 7f3e6a7fc700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e5805f050 0x7f3e58061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:25.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.894+0000 7f3e6a7fc700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e5805f050 0x7f3e58061510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3e60006d80 tx=0x7f3e60008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:25.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:25.897+0000 7f3e52ffd700 1 -- 172.21.15.33:0/532518736 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e5c04cc40 con 0x7f3e6c064a00 2023-11-06T20:45:26.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.104+0000 7f3e6bfff700 1 -- 172.21.15.33:0/532518736 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e6c0c1910 con 0x7f3e5805f050 2023-11-06T20:45:26.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.110+0000 7f3e52ffd700 1 -- 172.21.15.33:0/532518736 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f3e6c0c1910 con 0x7f3e5805f050 2023-11-06T20:45:26.110 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/dir2 2023-11-06T20:45:26.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 -- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e5805f050 msgr2=0x7f3e58061510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e5805f050 0x7f3e58061510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3e60006d80 tx=0x7f3e60008040).stop 2023-11-06T20:45:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 -- 172.21.15.33:0/532518736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c064a00 msgr2=0x7f3e6c14e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c064a00 0x7f3e6c14e8c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f3e6c084210 tx=0x7f3e5c016000).stop 2023-11-06T20:45:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 -- 172.21.15.33:0/532518736 shutdown_connections 2023-11-06T20:45:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e5805f050 0x7f3e58061510 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3e6c0ca410 0x7f3e6c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e6c064a00 0x7f3e6c14e8c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 --2- 172.21.15.33:0/532518736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e6c0c60c0 0x7f3e6c156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 -- 172.21.15.33:0/532518736 >> 172.21.15.33:0/532518736 conn(0x7f3e6c0cb6f0 msgr2=0x7f3e6c152280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 -- 172.21.15.33:0/532518736 shutdown_connections 2023-11-06T20:45:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.112+0000 7f3e6bfff700 1 -- 172.21.15.33:0/532518736 wait complete. 2023-11-06T20:45:26.134 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-11-06T20:45:26.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:26.135 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-11-06T20:45:26.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-11-06T20:45:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.388+0000 7ff1c99ff700 1 Processor -- start 2023-11-06T20:45:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.388+0000 7ff1c99ff700 1 -- start start 2023-11-06T20:45:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c99ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 0x7ff1c4158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c99ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1c40c8ee0 0x7ff1c40ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c99ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff1c41542b0 0x7ff1c40c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c99ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c40c9f80 con 0x7ff1c40c8ee0 2023-11-06T20:45:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c99ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c40c6000 con 0x7ff1c41542b0 2023-11-06T20:45:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c99ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40c9370 con 0x7ff1c4153ed0 2023-11-06T20:45:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c27fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1c40c8ee0 0x7ff1c40ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c27fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1c40c8ee0 0x7ff1c40ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60322/0 (socket says 172.21.15.33:60322) 2023-11-06T20:45:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c27fc700 1 -- 172.21.15.33:0/1452309816 learned_addr learned my addr 172.21.15.33:0/1452309816 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c2ffd700 1 --2- 172.21.15.33:0/1452309816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 0x7ff1c4158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c37fe700 1 --2- 172.21.15.33:0/1452309816 >> v2:172.21.15.69:3300/0 conn(0x7ff1c41542b0 0x7ff1c40c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c27fc700 1 -- 172.21.15.33:0/1452309816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 msgr2=0x7ff1c4158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:26.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.390+0000 7ff1c27fc700 1 --2- 172.21.15.33:0/1452309816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 0x7ff1c4158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c27fc700 1 -- 172.21.15.33:0/1452309816 >> v2:172.21.15.69:3300/0 conn(0x7ff1c41542b0 msgr2=0x7ff1c40c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:26.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c27fc700 1 --2- 172.21.15.33:0/1452309816 >> v2:172.21.15.69:3300/0 conn(0x7ff1c41542b0 0x7ff1c40c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c27fc700 1 -- 172.21.15.33:0/1452309816 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c40c9650 con 0x7ff1c40c8ee0 2023-11-06T20:45:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c37fe700 1 --2- 172.21.15.33:0/1452309816 >> v2:172.21.15.69:3300/0 conn(0x7ff1c41542b0 0x7ff1c40c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c2ffd700 1 --2- 172.21.15.33:0/1452309816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 0x7ff1c4158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c27fc700 1 --2- 172.21.15.33:0/1452309816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c40c8ee0 0x7ff1c40ca410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7ff1c40c8c60 tx=0x7ff1b80090f0).ready entity=mon.0 client_cookie=cd3d69f9c0ff76a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c1ffb700 1 -- 172.21.15.33:0/1452309816 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800b600 con 0x7ff1c40c8ee0 2023-11-06T20:45:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c1ffb700 1 -- 172.21.15.33:0/1452309816 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b800b760 con 0x7ff1c40c8ee0 2023-11-06T20:45:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c1ffb700 1 -- 172.21.15.33:0/1452309816 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800bae0 con 0x7ff1c40c8ee0 2023-11-06T20:45:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c99ff700 1 -- 172.21.15.33:0/1452309816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c40c8ee0 msgr2=0x7ff1c40ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c99ff700 1 --2- 172.21.15.33:0/1452309816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c40c8ee0 0x7ff1c40ca410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7ff1c40c8c60 tx=0x7ff1b80090f0).stop 2023-11-06T20:45:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c99ff700 1 -- 172.21.15.33:0/1452309816 shutdown_connections 2023-11-06T20:45:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c99ff700 1 --2- 172.21.15.33:0/1452309816 >> v2:172.21.15.69:3300/0 conn(0x7ff1c41542b0 0x7ff1c40c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c99ff700 1 --2- 172.21.15.33:0/1452309816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 0x7ff1c4158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c99ff700 1 --2- 172.21.15.33:0/1452309816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c40c8ee0 0x7ff1c40ca410 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.391+0000 7ff1c99ff700 1 -- 172.21.15.33:0/1452309816 >> 172.21.15.33:0/1452309816 conn(0x7ff1c405e1b0 msgr2=0x7ff1c405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.392+0000 7ff1c99ff700 1 -- 172.21.15.33:0/1452309816 shutdown_connections 2023-11-06T20:45:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.392+0000 7ff1c99ff700 1 -- 172.21.15.33:0/1452309816 wait complete. 2023-11-06T20:45:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.393+0000 7ff1c99ff700 1 Processor -- start 2023-11-06T20:45:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.393+0000 7ff1c99ff700 1 -- start start 2023-11-06T20:45:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.394+0000 7ff1c99ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 0x7ff1c4146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.394+0000 7ff1c99ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c41542b0 0x7ff1c40c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.394+0000 7ff1c99ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1c40c1240 0x7ff1c40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.394+0000 7ff1c99ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40aed70 con 0x7ff1c41542b0 2023-11-06T20:45:26.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.394+0000 7ff1c99ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c41507a0 con 0x7ff1c40c1240 2023-11-06T20:45:26.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.394+0000 7ff1c99ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40c1680 con 0x7ff1c4153ed0 2023-11-06T20:45:26.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.394+0000 7ff1c27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c41542b0 0x7ff1c40c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.394+0000 7ff1c27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c41542b0 0x7ff1c40c0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60336/0 (socket says 172.21.15.33:60336) 2023-11-06T20:45:26.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.394+0000 7ff1c27fc700 1 -- 172.21.15.33:0/2435312777 learned_addr learned my addr 172.21.15.33:0/2435312777 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:26.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c2ffd700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 0x7ff1c4146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c37fe700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1c40c1240 0x7ff1c40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c27fc700 1 -- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 msgr2=0x7ff1c4146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c27fc700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 0x7ff1c4146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c27fc700 1 -- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1c40c1240 msgr2=0x7ff1c40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c27fc700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1c40c1240 0x7ff1c40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c27fc700 1 -- 172.21.15.33:0/2435312777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009370 con 0x7ff1c41542b0 2023-11-06T20:45:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c37fe700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1c40c1240 0x7ff1c40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c27fc700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c41542b0 0x7ff1c40c0d50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7ff1b8011000 tx=0x7ff1b8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1aaffd700 1 -- 172.21.15.33:0/2435312777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8006a40 con 0x7ff1c41542b0 2023-11-06T20:45:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1aaffd700 1 -- 172.21.15.33:0/2435312777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b800b8d0 con 0x7ff1c41542b0 2023-11-06T20:45:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c99ff700 1 -- 172.21.15.33:0/2435312777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c414fe70 con 0x7ff1c41542b0 2023-11-06T20:45:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1aaffd700 1 -- 172.21.15.33:0/2435312777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b801c670 con 0x7ff1c41542b0 2023-11-06T20:45:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.395+0000 7ff1c99ff700 1 -- 172.21.15.33:0/2435312777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c4150430 con 0x7ff1c41542b0 2023-11-06T20:45:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.396+0000 7ff1c99ff700 1 -- 172.21.15.33:0/2435312777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c4150930 con 0x7ff1c41542b0 2023-11-06T20:45:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.397+0000 7ff1aaffd700 1 -- 172.21.15.33:0/2435312777 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7ff1b801ca60 con 0x7ff1c41542b0 2023-11-06T20:45:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.398+0000 7ff1aaffd700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff1b005f320 0x7ff1b00617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.398+0000 7ff1aaffd700 1 -- 172.21.15.33:0/2435312777 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7ff1b807bfb0 con 0x7ff1c41542b0 2023-11-06T20:45:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.399+0000 7ff1c2ffd700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff1b005f320 0x7ff1b00617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.399+0000 7ff1c2ffd700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff1b005f320 0x7ff1b00617e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff1b4006d80 tx=0x7ff1b4008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:26.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.403+0000 7ff1aaffd700 1 -- 172.21.15.33:0/2435312777 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff1b804cf50 con 0x7ff1c41542b0 2023-11-06T20:45:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.612+0000 7ff1c99ff700 1 -- 172.21.15.33:0/2435312777 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1c40c9160 con 0x7ff1b005f320 2023-11-06T20:45:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.618+0000 7ff1aaffd700 1 -- 172.21.15.33:0/2435312777 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff1c40c9160 con 0x7ff1b005f320 2023-11-06T20:45:26.618 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/dir3 2023-11-06T20:45:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 -- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff1b005f320 msgr2=0x7ff1b00617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff1b005f320 0x7ff1b00617e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff1b4006d80 tx=0x7ff1b4008040).stop 2023-11-06T20:45:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 -- 172.21.15.33:0/2435312777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c41542b0 msgr2=0x7ff1c40c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c41542b0 0x7ff1c40c0d50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7ff1b8011000 tx=0x7ff1b8017000).stop 2023-11-06T20:45:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 -- 172.21.15.33:0/2435312777 shutdown_connections 2023-11-06T20:45:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff1b005f320 0x7ff1b00617e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff1c4153ed0 0x7ff1c4146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1c40c1240 0x7ff1c40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 --2- 172.21.15.33:0/2435312777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1c41542b0 0x7ff1c40c0d50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 -- 172.21.15.33:0/2435312777 >> 172.21.15.33:0/2435312777 conn(0x7ff1c405e1b0 msgr2=0x7ff1c414e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 -- 172.21.15.33:0/2435312777 shutdown_connections 2023-11-06T20:45:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.620+0000 7ff1c99ff700 1 -- 172.21.15.33:0/2435312777 wait complete. 2023-11-06T20:45:26.637 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-11-06T20:45:26.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:26.638 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-11-06T20:45:26.687 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-11-06T20:45:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.928+0000 7f1f63dff700 1 Processor -- start 2023-11-06T20:45:26.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.928+0000 7f1f63dff700 1 -- start start 2023-11-06T20:45:26.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.929+0000 7f1f63dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 0x7f1f5c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.929+0000 7f1f63dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1f5c1550d0 0x7f1f5c0c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.929+0000 7f1f63dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c1586a0 con 0x7f1f5c0631a0 2023-11-06T20:45:26.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.929+0000 7f1f63dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c6590 con 0x7f1f5c1550d0 2023-11-06T20:45:26.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.929+0000 7f1f63dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c1544c0 con 0x7f1f5c0acfc0 2023-11-06T20:45:26.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.930+0000 7f1f6139a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1f5c1550d0 0x7f1f5c0c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.930+0000 7f1f6239c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1f5c0631a0 0x7f1f5c064680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36592/0 (socket says 172.21.15.33:36592) 2023-11-06T20:45:26.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.930+0000 7f1f6239c700 1 -- 172.21.15.33:0/3946262409 learned_addr learned my addr 172.21.15.33:0/3946262409 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.930+0000 7f1f61b9b700 1 --2- 172.21.15.33:0/3946262409 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 0x7f1f5c0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.930+0000 7f1f6139a700 1 -- 172.21.15.33:0/3946262409 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 msgr2=0x7f1f5c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.930+0000 7f1f6139a700 1 --2- 172.21.15.33:0/3946262409 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 0x7f1f5c0ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.930+0000 7f1f6139a700 1 -- 172.21.15.33:0/3946262409 >> v1:172.21.15.33:6789/0 conn(0x7f1f5c0631a0 legacy=0x7f1f5c064680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.930+0000 7f1f6139a700 1 -- 172.21.15.33:0/3946262409 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c155550 con 0x7f1f5c1550d0 2023-11-06T20:45:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.930+0000 7f1f60b99700 1 -- 172.21.15.33:0/3946262409 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663659576 0 0) 0x7f1f5c1586a0 con 0x7f1f5c0631a0 2023-11-06T20:45:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.930+0000 7f1f6139a700 1 --2- 172.21.15.33:0/3946262409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c1550d0 0x7f1f5c0c5e20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f1f4c009760 tx=0x7f1f4c009060).ready entity=mon.0 client_cookie=a8fe7847111dd40f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f60b99700 1 -- 172.21.15.33:0/3946262409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00b580 con 0x7f1f5c1550d0 2023-11-06T20:45:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f60b99700 1 -- 172.21.15.33:0/3946262409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00b6e0 con 0x7f1f5c1550d0 2023-11-06T20:45:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f60b99700 1 -- 172.21.15.33:0/3946262409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00ba60 con 0x7f1f5c1550d0 2023-11-06T20:45:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f63dff700 1 -- 172.21.15.33:0/3946262409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c1550d0 msgr2=0x7f1f5c0c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f63dff700 1 --2- 172.21.15.33:0/3946262409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c1550d0 0x7f1f5c0c5e20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f1f4c009760 tx=0x7f1f4c009060).stop 2023-11-06T20:45:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f63dff700 1 -- 172.21.15.33:0/3946262409 shutdown_connections 2023-11-06T20:45:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f63dff700 1 --2- 172.21.15.33:0/3946262409 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 0x7f1f5c0ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f63dff700 1 --2- 172.21.15.33:0/3946262409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c1550d0 0x7f1f5c0c5e20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f63dff700 1 -- 172.21.15.33:0/3946262409 >> 172.21.15.33:0/3946262409 conn(0x7f1f5c05ec80 msgr2=0x7f1f5c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f63dff700 1 -- 172.21.15.33:0/3946262409 shutdown_connections 2023-11-06T20:45:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.931+0000 7f1f63dff700 1 -- 172.21.15.33:0/3946262409 wait complete. 2023-11-06T20:45:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.933+0000 7f1f63dff700 1 Processor -- start 2023-11-06T20:45:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.933+0000 7f1f63dff700 1 -- start start 2023-11-06T20:45:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f63dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0631a0 0x7f1f5c150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f63dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 0x7f1f5c0c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f63dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c1550d0 0x7f1f5c0c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f63dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c157040 con 0x7f1f5c0631a0 2023-11-06T20:45:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f63dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c156430 con 0x7f1f5c1550d0 2023-11-06T20:45:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f63dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c156710 con 0x7f1f5c0acfc0 2023-11-06T20:45:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f61b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0631a0 0x7f1f5c150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f61b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0631a0 0x7f1f5c150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60350/0 (socket says 172.21.15.33:60350) 2023-11-06T20:45:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f61b9b700 1 -- 172.21.15.33:0/3187475492 learned_addr learned my addr 172.21.15.33:0/3187475492 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f6139a700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 0x7f1f5c0c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.934+0000 7f1f6239c700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c1550d0 0x7f1f5c0c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f61b9b700 1 -- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 msgr2=0x7f1f5c0c2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f61b9b700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 0x7f1f5c0c2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f61b9b700 1 -- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c1550d0 msgr2=0x7f1f5c0c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f61b9b700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c1550d0 0x7f1f5c0c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f61b9b700 1 -- 172.21.15.33:0/3187475492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0092a0 con 0x7f1f5c0631a0 2023-11-06T20:45:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f6139a700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 0x7f1f5c0c2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f6239c700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c1550d0 0x7f1f5c0c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f61b9b700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0631a0 0x7f1f5c150b00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f1f5800d890 tx=0x7f1f58008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f51ffb700 1 -- 172.21.15.33:0/3187475492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580125f0 con 0x7f1f5c0631a0 2023-11-06T20:45:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f51ffb700 1 -- 172.21.15.33:0/3187475492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f58012750 con 0x7f1f5c0631a0 2023-11-06T20:45:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f51ffb700 1 -- 172.21.15.33:0/3187475492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58012ad0 con 0x7f1f5c0631a0 2023-11-06T20:45:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f63dff700 1 -- 172.21.15.33:0/3187475492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c1569f0 con 0x7f1f5c0631a0 2023-11-06T20:45:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.935+0000 7f1f63dff700 1 -- 172.21.15.33:0/3187475492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c1560e0 con 0x7f1f5c0631a0 2023-11-06T20:45:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.936+0000 7f1f63dff700 1 -- 172.21.15.33:0/3187475492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c0cae00 con 0x7f1f5c0631a0 2023-11-06T20:45:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.937+0000 7f1f51ffb700 1 -- 172.21.15.33:0/3187475492 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f1f58013480 con 0x7f1f5c0631a0 2023-11-06T20:45:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.938+0000 7f1f51ffb700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f4005f380 0x7f1f40061840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.938+0000 7f1f51ffb700 1 -- 172.21.15.33:0/3187475492 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f1f5807a4c0 con 0x7f1f5c0631a0 2023-11-06T20:45:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.938+0000 7f1f6139a700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f4005f380 0x7f1f40061840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.939+0000 7f1f6139a700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f4005f380 0x7f1f40061840 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f1f4c009760 tx=0x7f1f4c017000).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:26.942+0000 7f1f51ffb700 1 -- 172.21.15.33:0/3187475492 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1f5804b380 con 0x7f1f5c0631a0 2023-11-06T20:45:27.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.153+0000 7f1f63dff700 1 -- 172.21.15.33:0/3187475492 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f5c154eb0 con 0x7f1f4005f380 2023-11-06T20:45:27.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.160+0000 7f1f51ffb700 1 -- 172.21.15.33:0/3187475492 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1f5c154eb0 con 0x7f1f4005f380 2023-11-06T20:45:27.160 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/dir4 2023-11-06T20:45:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 -- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f4005f380 msgr2=0x7f1f40061840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f4005f380 0x7f1f40061840 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f1f4c009760 tx=0x7f1f4c017000).stop 2023-11-06T20:45:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 -- 172.21.15.33:0/3187475492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0631a0 msgr2=0x7f1f5c150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0631a0 0x7f1f5c150b00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f1f5800d890 tx=0x7f1f58008040).stop 2023-11-06T20:45:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 -- 172.21.15.33:0/3187475492 shutdown_connections 2023-11-06T20:45:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f4005f380 0x7f1f40061840 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c1550d0 0x7f1f5c0c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0631a0 0x7f1f5c150b00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 --2- 172.21.15.33:0/3187475492 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c0acfc0 0x7f1f5c0c2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 -- 172.21.15.33:0/3187475492 >> 172.21.15.33:0/3187475492 conn(0x7f1f5c05ec80 msgr2=0x7f1f5c064680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:27.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 -- 172.21.15.33:0/3187475492 shutdown_connections 2023-11-06T20:45:27.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.162+0000 7f1f63dff700 1 -- 172.21.15.33:0/3187475492 wait complete. 2023-11-06T20:45:27.180 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-11-06T20:45:27.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:27.180 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-11-06T20:45:27.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-11-06T20:45:27.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.431+0000 7ff0f3fff700 1 Processor -- start 2023-11-06T20:45:27.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.431+0000 7ff0f3fff700 1 -- start start 2023-11-06T20:45:27.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.432+0000 7ff0f3fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff0ec154340 0x7ff0ec0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.432+0000 7ff0f3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0ec154720 0x7ff0ec158010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.432+0000 7ff0f3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff0ec158500 0x7ff0ec063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.432+0000 7ff0f3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ec158960 con 0x7ff0ec158500 2023-11-06T20:45:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.432+0000 7ff0f3fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ec0c0e00 con 0x7ff0ec154340 2023-11-06T20:45:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.432+0000 7ff0f3fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec158bf0 con 0x7ff0ec154720 2023-11-06T20:45:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.433+0000 7ff0f259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff0ec158500 0x7ff0ec063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.433+0000 7ff0f259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff0ec158500 0x7ff0ec063200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60352/0 (socket says 172.21.15.33:60352) 2023-11-06T20:45:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.433+0000 7ff0f259c700 1 -- 172.21.15.33:0/3932355556 learned_addr learned my addr 172.21.15.33:0/3932355556 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.433+0000 7ff0f159a700 1 --2- 172.21.15.33:0/3932355556 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0ec154720 0x7ff0ec158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.433+0000 7ff0f1d9b700 1 --2- 172.21.15.33:0/3932355556 >> v2:172.21.15.69:3300/0 conn(0x7ff0ec154340 0x7ff0ec0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.433+0000 7ff0f259c700 1 -- 172.21.15.33:0/3932355556 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0ec154720 msgr2=0x7ff0ec158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.433+0000 7ff0f259c700 1 --2- 172.21.15.33:0/3932355556 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0ec154720 0x7ff0ec158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.433+0000 7ff0f259c700 1 -- 172.21.15.33:0/3932355556 >> v2:172.21.15.69:3300/0 conn(0x7ff0ec154340 msgr2=0x7ff0ec0aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.433+0000 7ff0f259c700 1 --2- 172.21.15.33:0/3932355556 >> v2:172.21.15.69:3300/0 conn(0x7ff0ec154340 0x7ff0ec0aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.433+0000 7ff0f259c700 1 -- 172.21.15.33:0/3932355556 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec158ed0 con 0x7ff0ec158500 2023-11-06T20:45:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f259c700 1 --2- 172.21.15.33:0/3932355556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec158500 0x7ff0ec063200 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ff0ec05f090 tx=0x7ff0e4009840).ready entity=mon.0 client_cookie=a63babae161c6200 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f0d99700 1 -- 172.21.15.33:0/3932355556 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400e070 con 0x7ff0ec158500 2023-11-06T20:45:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f0d99700 1 -- 172.21.15.33:0/3932355556 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e400ba50 con 0x7ff0ec158500 2023-11-06T20:45:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f0d99700 1 -- 172.21.15.33:0/3932355556 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400bdd0 con 0x7ff0ec158500 2023-11-06T20:45:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3932355556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec158500 msgr2=0x7ff0ec063200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f3fff700 1 --2- 172.21.15.33:0/3932355556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec158500 0x7ff0ec063200 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ff0ec05f090 tx=0x7ff0e4009840).stop 2023-11-06T20:45:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3932355556 shutdown_connections 2023-11-06T20:45:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f3fff700 1 --2- 172.21.15.33:0/3932355556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec158500 0x7ff0ec063200 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f3fff700 1 --2- 172.21.15.33:0/3932355556 >> v2:172.21.15.69:3300/0 conn(0x7ff0ec154340 0x7ff0ec0aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f3fff700 1 --2- 172.21.15.33:0/3932355556 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0ec154720 0x7ff0ec158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3932355556 >> 172.21.15.33:0/3932355556 conn(0x7ff0ec05ec80 msgr2=0x7ff0ec0cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3932355556 shutdown_connections 2023-11-06T20:45:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.434+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3932355556 wait complete. 2023-11-06T20:45:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.436+0000 7ff0f3fff700 1 Processor -- start 2023-11-06T20:45:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.436+0000 7ff0f3fff700 1 -- start start 2023-11-06T20:45:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0ec154340 0x7ff0ec0ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec154720 0x7ff0ec14e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0ec158500 0x7ff0ec0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec14fc90 con 0x7ff0ec154720 2023-11-06T20:45:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f3fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec14edd0 con 0x7ff0ec154340 2023-11-06T20:45:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f3fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec14f080 con 0x7ff0ec158500 2023-11-06T20:45:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f159a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec154720 0x7ff0ec14e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f159a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec154720 0x7ff0ec14e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60368/0 (socket says 172.21.15.33:60368) 2023-11-06T20:45:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f159a700 1 -- 172.21.15.33:0/3885968651 learned_addr learned my addr 172.21.15.33:0/3885968651 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f259c700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0ec158500 0x7ff0ec0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.437+0000 7ff0f1d9b700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0ec154340 0x7ff0ec0ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0f159a700 1 -- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0ec158500 msgr2=0x7ff0ec0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0f159a700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0ec158500 0x7ff0ec0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0f159a700 1 -- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0ec154340 msgr2=0x7ff0ec0ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0f159a700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0ec154340 0x7ff0ec0ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0f159a700 1 -- 172.21.15.33:0/3885968651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e4009ae0 con 0x7ff0ec154720 2023-11-06T20:45:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0f159a700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec154720 0x7ff0ec14e8e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7ff0dc00d860 tx=0x7ff0dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0e1ffb700 1 -- 172.21.15.33:0/3885968651 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc011590 con 0x7ff0ec154720 2023-11-06T20:45:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0e1ffb700 1 -- 172.21.15.33:0/3885968651 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc0116f0 con 0x7ff0ec154720 2023-11-06T20:45:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3885968651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ec14f360 con 0x7ff0ec154720 2023-11-06T20:45:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0e1ffb700 1 -- 172.21.15.33:0/3885968651 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc011a70 con 0x7ff0ec154720 2023-11-06T20:45:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.438+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3885968651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec14f920 con 0x7ff0ec154720 2023-11-06T20:45:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.440+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3885968651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ec14fe00 con 0x7ff0ec154720 2023-11-06T20:45:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.440+0000 7ff0e1ffb700 1 -- 172.21.15.33:0/3885968651 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7ff0dc012420 con 0x7ff0ec154720 2023-11-06T20:45:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.441+0000 7ff0e1ffb700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0d005f320 0x7ff0d00617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.441+0000 7ff0e1ffb700 1 -- 172.21.15.33:0/3885968651 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7ff0dc079fb0 con 0x7ff0ec154720 2023-11-06T20:45:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.442+0000 7ff0f1d9b700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0d005f320 0x7ff0d00617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.442+0000 7ff0f1d9b700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0d005f320 0x7ff0d00617e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff0e800bbc0 tx=0x7ff0e8008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.445+0000 7ff0e1ffb700 1 -- 172.21.15.33:0/3885968651 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff0dc04ae70 con 0x7ff0ec154720 2023-11-06T20:45:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.651+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3885968651 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0ec0cbcf0 con 0x7ff0d005f320 2023-11-06T20:45:27.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.657+0000 7ff0e1ffb700 1 -- 172.21.15.33:0/3885968651 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff0ec0cbcf0 con 0x7ff0d005f320 2023-11-06T20:45:27.657 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/dir5 2023-11-06T20:45:27.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0d005f320 msgr2=0x7ff0d00617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0d005f320 0x7ff0d00617e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff0e800bbc0 tx=0x7ff0e8008040).stop 2023-11-06T20:45:27.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3885968651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec154720 msgr2=0x7ff0ec14e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec154720 0x7ff0ec14e8e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7ff0dc00d860 tx=0x7ff0dc008040).stop 2023-11-06T20:45:27.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3885968651 shutdown_connections 2023-11-06T20:45:27.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0d005f320 0x7ff0d00617e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff0ec158500 0x7ff0ec0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0ec154340 0x7ff0ec0ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 --2- 172.21.15.33:0/3885968651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0ec154720 0x7ff0ec14e8e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3885968651 >> 172.21.15.33:0/3885968651 conn(0x7ff0ec05ec80 msgr2=0x7ff0ec14d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:27.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3885968651 shutdown_connections 2023-11-06T20:45:27.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.659+0000 7ff0f3fff700 1 -- 172.21.15.33:0/3885968651 wait complete. 2023-11-06T20:45:27.676 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-11-06T20:45:27.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:27.677 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-11-06T20:45:27.728 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-11-06T20:45:27.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.928+0000 7f45d69ff700 1 Processor -- start 2023-11-06T20:45:27.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.929+0000 7f45d69ff700 1 -- start start 2023-11-06T20:45:27.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.930+0000 7f45d69ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f45d0153ed0 0x7f45d0158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.930+0000 7f45d69ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d00c8ee0 0x7f45d00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.930+0000 7f45d69ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d00c9f80 con 0x7f45d01542b0 2023-11-06T20:45:27.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.930+0000 7f45d69ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00c6000 con 0x7f45d0153ed0 2023-11-06T20:45:27.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.930+0000 7f45d69ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00c9370 con 0x7f45d00c8ee0 2023-11-06T20:45:27.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45cf7fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d00c8ee0 0x7f45d00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45d4f9c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f45d01542b0 0x7f45d00c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:37170/0 (socket says 172.21.15.33:37170) 2023-11-06T20:45:27.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45cffff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f45d0153ed0 0x7f45d0158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45cf7fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d00c8ee0 0x7f45d00ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:35216/0 (socket says 172.21.15.33:35216) 2023-11-06T20:45:27.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45d4f9c700 1 -- 172.21.15.33:0/2677206325 learned_addr learned my addr 172.21.15.33:0/2677206325 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:27.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45ceffd700 1 -- 172.21.15.33:0/2677206325 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668618698 0 0) 0x7f45d00c9f80 con 0x7f45d01542b0 2023-11-06T20:45:27.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45ceffd700 1 -- 172.21.15.33:0/2677206325 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45b80033f0 con 0x7f45d01542b0 2023-11-06T20:45:27.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45cffff700 1 -- 172.21.15.33:0/2677206325 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d00c8ee0 msgr2=0x7f45d00ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:27.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45cffff700 1 --2- 172.21.15.33:0/2677206325 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d00c8ee0 0x7f45d00ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45cffff700 1 -- 172.21.15.33:0/2677206325 >> v1:172.21.15.69:6789/0 conn(0x7f45d01542b0 legacy=0x7f45d00c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45cffff700 1 -- 172.21.15.33:0/2677206325 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00c9650 con 0x7f45d0153ed0 2023-11-06T20:45:27.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.931+0000 7f45cffff700 1 --2- 172.21.15.33:0/2677206325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0153ed0 0x7f45d0158600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f45c0009760 tx=0x7f45c0009060).ready entity=mon.1 client_cookie=7bfe33ed6c0f5295 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:27.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45ceffd700 1 -- 172.21.15.33:0/2677206325 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c000d4a0 con 0x7f45d0153ed0 2023-11-06T20:45:27.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45ceffd700 1 -- 172.21.15.33:0/2677206325 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c000d600 con 0x7f45d0153ed0 2023-11-06T20:45:27.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45ceffd700 1 -- 172.21.15.33:0/2677206325 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c000d980 con 0x7f45d0153ed0 2023-11-06T20:45:27.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45d69ff700 1 -- 172.21.15.33:0/2677206325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0153ed0 msgr2=0x7f45d0158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45d69ff700 1 --2- 172.21.15.33:0/2677206325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0153ed0 0x7f45d0158600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f45c0009760 tx=0x7f45c0009060).stop 2023-11-06T20:45:27.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45d69ff700 1 -- 172.21.15.33:0/2677206325 shutdown_connections 2023-11-06T20:45:27.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45d69ff700 1 --2- 172.21.15.33:0/2677206325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0153ed0 0x7f45d0158600 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45d69ff700 1 --2- 172.21.15.33:0/2677206325 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d00c8ee0 0x7f45d00ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45d69ff700 1 -- 172.21.15.33:0/2677206325 >> 172.21.15.33:0/2677206325 conn(0x7f45d005e1b0 msgr2=0x7f45d005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:27.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45d69ff700 1 -- 172.21.15.33:0/2677206325 shutdown_connections 2023-11-06T20:45:27.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.932+0000 7f45d69ff700 1 -- 172.21.15.33:0/2677206325 wait complete. 2023-11-06T20:45:27.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.934+0000 7f45d69ff700 1 Processor -- start 2023-11-06T20:45:27.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.934+0000 7f45d69ff700 1 -- start start 2023-11-06T20:45:27.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.935+0000 7f45d69ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0153ed0 0x7f45d00b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.935+0000 7f45d69ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d01542b0 0x7f45d014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.935+0000 7f45d69ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d014de30 0x7f45d00addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.935+0000 7f45d69ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00a9d50 con 0x7f45d01542b0 2023-11-06T20:45:27.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.935+0000 7f45d69ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d01502b0 con 0x7f45d0153ed0 2023-11-06T20:45:27.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.935+0000 7f45d69ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00a9790 con 0x7f45d014de30 2023-11-06T20:45:27.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.935+0000 7f45cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d01542b0 0x7f45d014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.935+0000 7f45cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d01542b0 0x7f45d014d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45146/0 (socket says 172.21.15.33:45146) 2023-11-06T20:45:27.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.935+0000 7f45cf7fe700 1 -- 172.21.15.33:0/1014787600 learned_addr learned my addr 172.21.15.33:0/1014787600 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:27.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45d4f9c700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d014de30 0x7f45d00addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45cffff700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0153ed0 0x7f45d00b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45cf7fe700 1 -- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d014de30 msgr2=0x7f45d00addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45cf7fe700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d014de30 0x7f45d00addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45cf7fe700 1 -- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0153ed0 msgr2=0x7f45d00b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:27.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45cf7fe700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0153ed0 0x7f45d00b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:27.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45cf7fe700 1 -- 172.21.15.33:0/1014787600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c00092a0 con 0x7f45d01542b0 2023-11-06T20:45:27.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45cf7fe700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d01542b0 0x7f45d014d940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f45c400e900 tx=0x7f45c4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:27.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45b7fff700 1 -- 172.21.15.33:0/1014787600 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c400c640 con 0x7f45d01542b0 2023-11-06T20:45:27.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45b7fff700 1 -- 172.21.15.33:0/1014787600 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c400c7a0 con 0x7f45d01542b0 2023-11-06T20:45:27.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45d69ff700 1 -- 172.21.15.33:0/1014787600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d00a9a70 con 0x7f45d01542b0 2023-11-06T20:45:27.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45b7fff700 1 -- 172.21.15.33:0/1014787600 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c400cb20 con 0x7f45d01542b0 2023-11-06T20:45:27.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.936+0000 7f45d69ff700 1 -- 172.21.15.33:0/1014787600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d00b0af0 con 0x7f45d01542b0 2023-11-06T20:45:27.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.937+0000 7f45d69ff700 1 -- 172.21.15.33:0/1014787600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d0150420 con 0x7f45d01542b0 2023-11-06T20:45:27.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.938+0000 7f45b7fff700 1 -- 172.21.15.33:0/1014787600 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f45c401b400 con 0x7f45d01542b0 2023-11-06T20:45:27.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.939+0000 7f45b7fff700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f45bc05f350 0x7f45bc061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:27.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.940+0000 7f45b7fff700 1 -- 172.21.15.33:0/1014787600 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f45c407aa40 con 0x7f45d01542b0 2023-11-06T20:45:27.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.940+0000 7f45cffff700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f45bc05f350 0x7f45bc061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:27.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.940+0000 7f45cffff700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f45bc05f350 0x7f45bc061810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f45c000d260 tx=0x7f45c0016000).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:27.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:27.944+0000 7f45b7fff700 1 -- 172.21.15.33:0/1014787600 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f45c404b940 con 0x7f45d01542b0 2023-11-06T20:45:28.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.154+0000 7f45d69ff700 1 -- 172.21.15.33:0/1014787600 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f45d006c6c0 con 0x7f45bc05f350 2023-11-06T20:45:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.160+0000 7f45b7fff700 1 -- 172.21.15.33:0/1014787600 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f45d006c6c0 con 0x7f45bc05f350 2023-11-06T20:45:28.160 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/dir6 2023-11-06T20:45:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 -- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f45bc05f350 msgr2=0x7f45bc061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f45bc05f350 0x7f45bc061810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f45c000d260 tx=0x7f45c0016000).stop 2023-11-06T20:45:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 -- 172.21.15.33:0/1014787600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d01542b0 msgr2=0x7f45d014d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d01542b0 0x7f45d014d940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f45c400e900 tx=0x7f45c4009060).stop 2023-11-06T20:45:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 -- 172.21.15.33:0/1014787600 shutdown_connections 2023-11-06T20:45:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f45bc05f350 0x7f45bc061810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45d014de30 0x7f45d00addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45d0153ed0 0x7f45d00b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 --2- 172.21.15.33:0/1014787600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d01542b0 0x7f45d014d940 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 -- 172.21.15.33:0/1014787600 >> 172.21.15.33:0/1014787600 conn(0x7f45d005e1b0 msgr2=0x7f45d014f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.162+0000 7f45d69ff700 1 -- 172.21.15.33:0/1014787600 shutdown_connections 2023-11-06T20:45:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.163+0000 7f45d69ff700 1 -- 172.21.15.33:0/1014787600 wait complete. 2023-11-06T20:45:28.180 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-11-06T20:45:28.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:28.180 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-11-06T20:45:28.232 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-11-06T20:45:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.433+0000 7fe497bff700 1 Processor -- start 2023-11-06T20:45:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.434+0000 7fe497bff700 1 -- start start 2023-11-06T20:45:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.435+0000 7fe497bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4900bc930 0x7fe490069000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.435+0000 7fe497bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4900634c0 con 0x7fe49005e3c0 2023-11-06T20:45:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.435+0000 7fe497bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4900c4600 con 0x7fe490068c20 2023-11-06T20:45:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.435+0000 7fe497bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4900628b0 con 0x7fe4900bc930 2023-11-06T20:45:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.435+0000 7fe49619c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4900bc930 0x7fe490069000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.435+0000 7fe49519a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe49005e3c0 0x7fe4900bc220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46602/0 (socket says 172.21.15.33:46602) 2023-11-06T20:45:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.435+0000 7fe49519a700 1 -- 172.21.15.33:0/4186092100 learned_addr learned my addr 172.21.15.33:0/4186092100 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:28.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.436+0000 7fe49619c700 1 -- 172.21.15.33:0/4186092100 >> v1:172.21.15.69:6789/0 conn(0x7fe490068c20 legacy=0x7fe490063950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.436+0000 7fe49619c700 1 -- 172.21.15.33:0/4186092100 >> v1:172.21.15.33:6789/0 conn(0x7fe49005e3c0 legacy=0x7fe4900bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.436+0000 7fe49619c700 1 -- 172.21.15.33:0/4186092100 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe490062b90 con 0x7fe4900bc930 2023-11-06T20:45:28.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.436+0000 7fe494999700 1 -- 172.21.15.33:0/4186092100 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202745368 0 0) 0x7fe4900634c0 con 0x7fe49005e3c0 2023-11-06T20:45:28.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.436+0000 7fe49619c700 1 --2- 172.21.15.33:0/4186092100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 0x7fe490069000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fe488009e60 tx=0x7fe488009780).ready entity=mon.0 client_cookie=e470ebd5d15628ac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:28.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.436+0000 7fe494999700 1 -- 172.21.15.33:0/4186092100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48800e070 con 0x7fe4900bc930 2023-11-06T20:45:28.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.436+0000 7fe494999700 1 -- 172.21.15.33:0/4186092100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48800bc30 con 0x7fe4900bc930 2023-11-06T20:45:28.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.436+0000 7fe494999700 1 -- 172.21.15.33:0/4186092100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe488012410 con 0x7fe4900bc930 2023-11-06T20:45:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.437+0000 7fe497bff700 1 -- 172.21.15.33:0/4186092100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 msgr2=0x7fe490069000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.437+0000 7fe497bff700 1 --2- 172.21.15.33:0/4186092100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 0x7fe490069000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fe488009e60 tx=0x7fe488009780).stop 2023-11-06T20:45:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.437+0000 7fe497bff700 1 -- 172.21.15.33:0/4186092100 shutdown_connections 2023-11-06T20:45:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.437+0000 7fe497bff700 1 --2- 172.21.15.33:0/4186092100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 0x7fe490069000 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.437+0000 7fe497bff700 1 -- 172.21.15.33:0/4186092100 >> 172.21.15.33:0/4186092100 conn(0x7fe4900c7d10 msgr2=0x7fe4900c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:28.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.437+0000 7fe497bff700 1 -- 172.21.15.33:0/4186092100 shutdown_connections 2023-11-06T20:45:28.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.437+0000 7fe497bff700 1 -- 172.21.15.33:0/4186092100 wait complete. 2023-11-06T20:45:28.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.438+0000 7fe497bff700 1 Processor -- start 2023-11-06T20:45:28.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.439+0000 7fe497bff700 1 -- start start 2023-11-06T20:45:28.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.439+0000 7fe497bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49005e3c0 0x7fe4900bed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.439+0000 7fe497bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe490068c20 0x7fe49014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.439+0000 7fe497bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 0x7fe4900a7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.439+0000 7fe497bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900bf4d0 con 0x7fe4900bc930 2023-11-06T20:45:28.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.439+0000 7fe497bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900c0330 con 0x7fe490068c20 2023-11-06T20:45:28.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.439+0000 7fe497bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900bf720 con 0x7fe49005e3c0 2023-11-06T20:45:28.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49619c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 0x7fe4900a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49619c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 0x7fe4900a7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45176/0 (socket says 172.21.15.33:45176) 2023-11-06T20:45:28.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49619c700 1 -- 172.21.15.33:0/1865944456 learned_addr learned my addr 172.21.15.33:0/1865944456 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:28.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49599b700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49005e3c0 0x7fe4900bed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49519a700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe490068c20 0x7fe49014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49619c700 1 -- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49005e3c0 msgr2=0x7fe4900bed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49619c700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49005e3c0 0x7fe4900bed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49619c700 1 -- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe490068c20 msgr2=0x7fe49014fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49619c700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe490068c20 0x7fe49014fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49619c700 1 -- 172.21.15.33:0/1865944456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe488009a20 con 0x7fe4900bc930 2023-11-06T20:45:28.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.440+0000 7fe49619c700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 0x7fe4900a7400 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fe4900616d0 tx=0x7fe488009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:28.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.441+0000 7fe485ffb700 1 -- 172.21.15.33:0/1865944456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48800b6f0 con 0x7fe4900bc930 2023-11-06T20:45:28.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.441+0000 7fe485ffb700 1 -- 172.21.15.33:0/1865944456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe488013e00 con 0x7fe4900bc930 2023-11-06T20:45:28.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.441+0000 7fe485ffb700 1 -- 172.21.15.33:0/1865944456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe488006af0 con 0x7fe4900bc930 2023-11-06T20:45:28.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.441+0000 7fe497bff700 1 -- 172.21.15.33:0/1865944456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4900bfa00 con 0x7fe4900bc930 2023-11-06T20:45:28.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.441+0000 7fe497bff700 1 -- 172.21.15.33:0/1865944456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4900bffc0 con 0x7fe4900bc930 2023-11-06T20:45:28.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.442+0000 7fe497bff700 1 -- 172.21.15.33:0/1865944456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe490062e70 con 0x7fe4900bc930 2023-11-06T20:45:28.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.443+0000 7fe485ffb700 1 -- 172.21.15.33:0/1865944456 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7fe48801d630 con 0x7fe4900bc930 2023-11-06T20:45:28.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.444+0000 7fe485ffb700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe47405f320 0x7fe4740617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.444+0000 7fe485ffb700 1 -- 172.21.15.33:0/1865944456 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fe48807cd50 con 0x7fe4900bc930 2023-11-06T20:45:28.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.444+0000 7fe49599b700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe47405f320 0x7fe4740617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.445+0000 7fe49599b700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe47405f320 0x7fe4740617e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe48c0036d0 tx=0x7fe48c009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:28.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.448+0000 7fe485ffb700 1 -- 172.21.15.33:0/1865944456 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe48804dc60 con 0x7fe4900bc930 2023-11-06T20:45:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.653+0000 7fe497bff700 1 -- 172.21.15.33:0/1865944456 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4900c62a0 con 0x7fe47405f320 2023-11-06T20:45:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.659+0000 7fe485ffb700 1 -- 172.21.15.33:0/1865944456 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe4900c62a0 con 0x7fe47405f320 2023-11-06T20:45:28.659 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/dir7 2023-11-06T20:45:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 -- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe47405f320 msgr2=0x7fe4740617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe47405f320 0x7fe4740617e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe48c0036d0 tx=0x7fe48c009040).stop 2023-11-06T20:45:28.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 -- 172.21.15.33:0/1865944456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 msgr2=0x7fe4900a7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 0x7fe4900a7400 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fe4900616d0 tx=0x7fe488009780).stop 2023-11-06T20:45:28.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 -- 172.21.15.33:0/1865944456 shutdown_connections 2023-11-06T20:45:28.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe47405f320 0x7fe4740617e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49005e3c0 0x7fe4900bed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4900bc930 0x7fe4900a7400 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 --2- 172.21.15.33:0/1865944456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe490068c20 0x7fe49014fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 -- 172.21.15.33:0/1865944456 >> 172.21.15.33:0/1865944456 conn(0x7fe4900c7d10 msgr2=0x7fe4900bc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 -- 172.21.15.33:0/1865944456 shutdown_connections 2023-11-06T20:45:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.661+0000 7fe497bff700 1 -- 172.21.15.33:0/1865944456 wait complete. 2023-11-06T20:45:28.678 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-11-06T20:45:28.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:28.679 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-11-06T20:45:28.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-11-06T20:45:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.932+0000 7f50a73ff700 1 Processor -- start 2023-11-06T20:45:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.932+0000 7f50a73ff700 1 -- start start 2023-11-06T20:45:28.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.933+0000 7f50a73ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f50a00c8ee0 0x7f50a00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.933+0000 7f50a73ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f50a01542b0 0x7f50a00c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.933+0000 7f50a73ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a00c9f80 con 0x7f50a0153ed0 2023-11-06T20:45:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.933+0000 7f50a73ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a00c6000 con 0x7f50a00c8ee0 2023-11-06T20:45:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.933+0000 7f50a73ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a00c9370 con 0x7f50a01542b0 2023-11-06T20:45:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f50a499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f50a00c8ee0 0x7f50a00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f50a519b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f50a0153ed0 0x7f50a0158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46616/0 (socket says 172.21.15.33:46616) 2023-11-06T20:45:28.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f50a519b700 1 -- 172.21.15.33:0/3215129202 learned_addr learned my addr 172.21.15.33:0/3215129202 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f50a599c700 1 --2- 172.21.15.33:0/3215129202 >> v2:172.21.15.69:3300/0 conn(0x7f50a01542b0 0x7f50a00c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f5097fff700 1 -- 172.21.15.33:0/3215129202 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602870263 0 0) 0x7f50a00c9f80 con 0x7f50a0153ed0 2023-11-06T20:45:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f5097fff700 1 -- 172.21.15.33:0/3215129202 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f508c0033f0 con 0x7f50a0153ed0 2023-11-06T20:45:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f50a499a700 1 -- 172.21.15.33:0/3215129202 >> v2:172.21.15.69:3300/0 conn(0x7f50a01542b0 msgr2=0x7f50a00c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:28.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f50a499a700 1 --2- 172.21.15.33:0/3215129202 >> v2:172.21.15.69:3300/0 conn(0x7f50a01542b0 0x7f50a00c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f50a499a700 1 -- 172.21.15.33:0/3215129202 >> v1:172.21.15.33:6789/0 conn(0x7f50a0153ed0 legacy=0x7f50a0158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f50a499a700 1 -- 172.21.15.33:0/3215129202 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a00c9650 con 0x7f50a00c8ee0 2023-11-06T20:45:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.934+0000 7f50a499a700 1 --2- 172.21.15.33:0/3215129202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a00c8ee0 0x7f50a00ca410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f50a00c85d0 tx=0x7f5090009060).ready entity=mon.0 client_cookie=8a6629aa48dc62a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f5097fff700 1 -- 172.21.15.33:0/3215129202 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000d4a0 con 0x7f50a00c8ee0 2023-11-06T20:45:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f5097fff700 1 -- 172.21.15.33:0/3215129202 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509000d600 con 0x7f50a00c8ee0 2023-11-06T20:45:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f5097fff700 1 -- 172.21.15.33:0/3215129202 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000d980 con 0x7f50a00c8ee0 2023-11-06T20:45:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f50a73ff700 1 -- 172.21.15.33:0/3215129202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a00c8ee0 msgr2=0x7f50a00ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f50a73ff700 1 --2- 172.21.15.33:0/3215129202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a00c8ee0 0x7f50a00ca410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f50a00c85d0 tx=0x7f5090009060).stop 2023-11-06T20:45:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f50a73ff700 1 -- 172.21.15.33:0/3215129202 shutdown_connections 2023-11-06T20:45:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f50a73ff700 1 --2- 172.21.15.33:0/3215129202 >> v2:172.21.15.69:3300/0 conn(0x7f50a01542b0 0x7f50a00c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f50a73ff700 1 --2- 172.21.15.33:0/3215129202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a00c8ee0 0x7f50a00ca410 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f50a73ff700 1 -- 172.21.15.33:0/3215129202 >> 172.21.15.33:0/3215129202 conn(0x7f50a005e1b0 msgr2=0x7f50a005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f50a73ff700 1 -- 172.21.15.33:0/3215129202 shutdown_connections 2023-11-06T20:45:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.935+0000 7f50a73ff700 1 -- 172.21.15.33:0/3215129202 wait complete. 2023-11-06T20:45:28.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.937+0000 7f50a73ff700 1 Processor -- start 2023-11-06T20:45:28.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.937+0000 7f50a73ff700 1 -- start start 2023-11-06T20:45:28.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a73ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50a0153ed0 0x7f50a0153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a73ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50a01542b0 0x7f50a01546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a73ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a0157d70 0x7f50a01581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a73ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a0155c10 con 0x7f50a0157d70 2023-11-06T20:45:28.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a73ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a0156700 con 0x7f50a0153ed0 2023-11-06T20:45:28.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a73ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a00adf90 con 0x7f50a01542b0 2023-11-06T20:45:28.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a0157d70 0x7f50a01581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a0157d70 0x7f50a01581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45192/0 (socket says 172.21.15.33:45192) 2023-11-06T20:45:28.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a599c700 1 -- 172.21.15.33:0/2190040015 learned_addr learned my addr 172.21.15.33:0/2190040015 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:28.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a519b700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50a0153ed0 0x7f50a0153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.938+0000 7f50a499a700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50a01542b0 0x7f50a01546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50a599c700 1 -- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50a01542b0 msgr2=0x7f50a01546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50a599c700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50a01542b0 0x7f50a01546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50a599c700 1 -- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50a0153ed0 msgr2=0x7f50a0153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:28.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50a599c700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50a0153ed0 0x7f50a0153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:28.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50a599c700 1 -- 172.21.15.33:0/2190040015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50900092a0 con 0x7f50a0157d70 2023-11-06T20:45:28.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50a599c700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a0157d70 0x7f50a01581f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f5098009da0 tx=0x7f509800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:28.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50957fa700 1 -- 172.21.15.33:0/2190040015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098009ad0 con 0x7f50a0157d70 2023-11-06T20:45:28.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50957fa700 1 -- 172.21.15.33:0/2190040015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509800f040 con 0x7f50a0157d70 2023-11-06T20:45:28.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50a73ff700 1 -- 172.21.15.33:0/2190040015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a0155e60 con 0x7f50a0157d70 2023-11-06T20:45:28.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50957fa700 1 -- 172.21.15.33:0/2190040015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800cd80 con 0x7f50a0157d70 2023-11-06T20:45:28.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.939+0000 7f50a73ff700 1 -- 172.21.15.33:0/2190040015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a0156420 con 0x7f50a0157d70 2023-11-06T20:45:28.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.940+0000 7f50a73ff700 1 -- 172.21.15.33:0/2190040015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a006c6c0 con 0x7f50a0157d70 2023-11-06T20:45:28.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.941+0000 7f50957fa700 1 -- 172.21.15.33:0/2190040015 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f5098013610 con 0x7f50a0157d70 2023-11-06T20:45:28.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.942+0000 7f50957fa700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508405f350 0x7f5084061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:28.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.942+0000 7f50957fa700 1 -- 172.21.15.33:0/2190040015 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f509807ae50 con 0x7f50a0157d70 2023-11-06T20:45:28.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.943+0000 7f50a519b700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508405f350 0x7f5084061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:28.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.943+0000 7f50a519b700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508405f350 0x7f5084061810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f509c005750 tx=0x7f509c009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:28.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:28.946+0000 7f50957fa700 1 -- 172.21.15.33:0/2190040015 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f509804bc60 con 0x7f50a0157d70 2023-11-06T20:45:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.151+0000 7f50a73ff700 1 -- 172.21.15.33:0/2190040015 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f50a00ae0d0 con 0x7f508405f350 2023-11-06T20:45:29.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.156+0000 7f50957fa700 1 -- 172.21.15.33:0/2190040015 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f50a00ae0d0 con 0x7f508405f350 2023-11-06T20:45:29.157 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/dir8 2023-11-06T20:45:29.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 -- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508405f350 msgr2=0x7f5084061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508405f350 0x7f5084061810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f509c005750 tx=0x7f509c009040).stop 2023-11-06T20:45:29.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 -- 172.21.15.33:0/2190040015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a0157d70 msgr2=0x7f50a01581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a0157d70 0x7f50a01581f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f5098009da0 tx=0x7f509800b040).stop 2023-11-06T20:45:29.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 -- 172.21.15.33:0/2190040015 shutdown_connections 2023-11-06T20:45:29.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508405f350 0x7f5084061810 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a0157d70 0x7f50a01581f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50a0153ed0 0x7f50a0153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 --2- 172.21.15.33:0/2190040015 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50a01542b0 0x7f50a01546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 -- 172.21.15.33:0/2190040015 >> 172.21.15.33:0/2190040015 conn(0x7f50a005e1b0 msgr2=0x7f50a014d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:29.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 -- 172.21.15.33:0/2190040015 shutdown_connections 2023-11-06T20:45:29.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.159+0000 7f50a73ff700 1 -- 172.21.15.33:0/2190040015 wait complete. 2023-11-06T20:45:29.177 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-11-06T20:45:29.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:45:29.177 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-11-06T20:45:29.227 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-11-06T20:45:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.428+0000 7f4c389ff700 1 Processor -- start 2023-11-06T20:45:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.428+0000 7f4c389ff700 1 -- start start 2023-11-06T20:45:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.429+0000 7f4c389ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c340bc090 0x7f4c3415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.429+0000 7f4c389ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c34154c60 con 0x7f4c341514c0 2023-11-06T20:45:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.429+0000 7f4c389ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c34068f50 con 0x7f4c341510e0 2023-11-06T20:45:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.429+0000 7f4c389ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c34151dc0 con 0x7f4c340bc090 2023-11-06T20:45:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c32d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c340bc090 0x7f4c3415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c31d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4c341514c0 0x7f4c340bb980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46622/0 (socket says 172.21.15.33:46622) 2023-11-06T20:45:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c31d9b700 1 -- 172.21.15.33:0/50256256 learned_addr learned my addr 172.21.15.33:0/50256256 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c32d9d700 1 -- 172.21.15.33:0/50256256 >> v1:172.21.15.69:6789/0 conn(0x7f4c341510e0 legacy=0x7f4c340a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c32d9d700 1 -- 172.21.15.33:0/50256256 >> v1:172.21.15.33:6789/0 conn(0x7f4c341514c0 legacy=0x7f4c340bb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c32d9d700 1 -- 172.21.15.33:0/50256256 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c341520a0 con 0x7f4c340bc090 2023-11-06T20:45:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c3159a700 1 -- 172.21.15.33:0/50256256 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341895420 0 0) 0x7f4c34154c60 con 0x7f4c341514c0 2023-11-06T20:45:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c32d9d700 1 --2- 172.21.15.33:0/50256256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 0x7f4c3415bea0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f4c2c009e60 tx=0x7f4c2c009780).ready entity=mon.0 client_cookie=e7f0221eece37a02 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c3159a700 1 -- 172.21.15.33:0/50256256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00e070 con 0x7f4c340bc090 2023-11-06T20:45:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c3159a700 1 -- 172.21.15.33:0/50256256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c00bbf0 con 0x7f4c340bc090 2023-11-06T20:45:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.430+0000 7f4c3159a700 1 -- 172.21.15.33:0/50256256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c012410 con 0x7f4c340bc090 2023-11-06T20:45:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.431+0000 7f4c389ff700 1 -- 172.21.15.33:0/50256256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 msgr2=0x7f4c3415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.431+0000 7f4c389ff700 1 --2- 172.21.15.33:0/50256256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 0x7f4c3415bea0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f4c2c009e60 tx=0x7f4c2c009780).stop 2023-11-06T20:45:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.431+0000 7f4c389ff700 1 -- 172.21.15.33:0/50256256 shutdown_connections 2023-11-06T20:45:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.431+0000 7f4c389ff700 1 --2- 172.21.15.33:0/50256256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 0x7f4c3415bea0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.431+0000 7f4c389ff700 1 -- 172.21.15.33:0/50256256 >> 172.21.15.33:0/50256256 conn(0x7f4c34062710 msgr2=0x7f4c34062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.431+0000 7f4c389ff700 1 -- 172.21.15.33:0/50256256 shutdown_connections 2023-11-06T20:45:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.431+0000 7f4c389ff700 1 -- 172.21.15.33:0/50256256 wait complete. 2023-11-06T20:45:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.433+0000 7f4c389ff700 1 Processor -- start 2023-11-06T20:45:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.433+0000 7f4c389ff700 1 -- start start 2023-11-06T20:45:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c389ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 0x7f4c340a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c389ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c341510e0 0x7f4c3414d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c389ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c341514c0 0x7f4c340aa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c389ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340a9dc0 con 0x7f4c340bc090 2023-11-06T20:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c389ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340c2530 con 0x7f4c341510e0 2023-11-06T20:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c389ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340c2810 con 0x7f4c341514c0 2023-11-06T20:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c3259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 0x7f4c340a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c3259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 0x7f4c340a8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45216/0 (socket says 172.21.15.33:45216) 2023-11-06T20:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c3259c700 1 -- 172.21.15.33:0/2090713073 learned_addr learned my addr 172.21.15.33:0/2090713073 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c31d9b700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c341510e0 0x7f4c3414d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c32d9d700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c341514c0 0x7f4c340aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c3259c700 1 -- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c341514c0 msgr2=0x7f4c340aa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c3259c700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c341514c0 0x7f4c340aa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c3259c700 1 -- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c341510e0 msgr2=0x7f4c3414d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c3259c700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c341510e0 0x7f4c3414d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.434+0000 7f4c3259c700 1 -- 172.21.15.33:0/2090713073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c009a20 con 0x7f4c340bc090 2023-11-06T20:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.435+0000 7f4c32d9d700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c341514c0 0x7f4c340aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.435+0000 7f4c31d9b700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c341510e0 0x7f4c3414d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.435+0000 7f4c3259c700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 0x7f4c340a8f90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f4c2400c440 tx=0x7f4c24007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.435+0000 7f4c227fc700 1 -- 172.21.15.33:0/2090713073 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400f470 con 0x7f4c340bc090 2023-11-06T20:45:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.435+0000 7f4c227fc700 1 -- 172.21.15.33:0/2090713073 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2400f5f0 con 0x7f4c340bc090 2023-11-06T20:45:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.435+0000 7f4c389ff700 1 -- 172.21.15.33:0/2090713073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c340a9a50 con 0x7f4c340bc090 2023-11-06T20:45:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.435+0000 7f4c227fc700 1 -- 172.21.15.33:0/2090713073 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400f9b0 con 0x7f4c340bc090 2023-11-06T20:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.435+0000 7f4c389ff700 1 -- 172.21.15.33:0/2090713073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c340a9480 con 0x7f4c340bc090 2023-11-06T20:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.436+0000 7f4c389ff700 1 -- 172.21.15.33:0/2090713073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c340aa0a0 con 0x7f4c340bc090 2023-11-06T20:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.437+0000 7f4c227fc700 1 -- 172.21.15.33:0/2090713073 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f4c24012400 con 0x7f4c340bc090 2023-11-06T20:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.438+0000 7f4c227fc700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c1405f350 0x7f4c14061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.438+0000 7f4c227fc700 1 -- 172.21.15.33:0/2090713073 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f4c24079ad0 con 0x7f4c340bc090 2023-11-06T20:45:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.438+0000 7f4c31d9b700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c1405f350 0x7f4c14061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.439+0000 7f4c31d9b700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c1405f350 0x7f4c14061810 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f4c340c1f90 tx=0x7f4c28009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.442+0000 7f4c227fc700 1 -- 172.21.15.33:0/2090713073 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4c24048af0 con 0x7f4c340bc090 2023-11-06T20:45:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.650+0000 7f4c389ff700 1 -- 172.21.15.33:0/2090713073 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c340a9f00 con 0x7f4c1405f350 2023-11-06T20:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.655+0000 7f4c227fc700 1 -- 172.21.15.33:0/2090713073 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4c280012b0 con 0x7f4c1405f350 2023-11-06T20:45:29.655 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/dir9 2023-11-06T20:45:29.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 -- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c1405f350 msgr2=0x7f4c14061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c1405f350 0x7f4c14061810 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f4c340c1f90 tx=0x7f4c28009040).stop 2023-11-06T20:45:29.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 -- 172.21.15.33:0/2090713073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 msgr2=0x7f4c340a8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 0x7f4c340a8f90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f4c2400c440 tx=0x7f4c24007de0).stop 2023-11-06T20:45:29.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 -- 172.21.15.33:0/2090713073 shutdown_connections 2023-11-06T20:45:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c1405f350 0x7f4c14061810 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c341514c0 0x7f4c340aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340bc090 0x7f4c340a8f90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 --2- 172.21.15.33:0/2090713073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c341510e0 0x7f4c3414d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 -- 172.21.15.33:0/2090713073 >> 172.21.15.33:0/2090713073 conn(0x7f4c34062710 msgr2=0x7f4c3414e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.657+0000 7f4c389ff700 1 -- 172.21.15.33:0/2090713073 shutdown_connections 2023-11-06T20:45:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.658+0000 7f4c389ff700 1 -- 172.21.15.33:0/2090713073 wait complete. 2023-11-06T20:45:29.675 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-11-06T20:45:29.675 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-11-06T20:45:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.877+0000 7f09aa3ff700 1 Processor -- start 2023-11-06T20:45:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.877+0000 7f09aa3ff700 1 -- start start 2023-11-06T20:45:29.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.879+0000 7f09aa3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f09a4153ed0 0x7f09a4158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.879+0000 7f09aa3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f09a41542b0 0x7f09a40c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.879+0000 7f09aa3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40c9f80 con 0x7f09a40c8ee0 2023-11-06T20:45:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.879+0000 7f09aa3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a40cab20 con 0x7f09a4153ed0 2023-11-06T20:45:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.879+0000 7f09aa3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40c6000 con 0x7f09a41542b0 2023-11-06T20:45:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.879+0000 7f09a3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f09a4153ed0 0x7f09a4158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.879+0000 7f09a3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f09a4153ed0 0x7f09a4158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45230/0 (socket says 172.21.15.33:45230) 2023-11-06T20:45:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.879+0000 7f09a3fff700 1 -- 172.21.15.33:0/2314157675 learned_addr learned my addr 172.21.15.33:0/2314157675 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.879+0000 7f09a899c700 1 --2- 172.21.15.33:0/2314157675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f09a41542b0 0x7f09a40c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.880+0000 7f09a3fff700 1 -- 172.21.15.33:0/2314157675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f09a41542b0 msgr2=0x7f09a40c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.880+0000 7f09a3fff700 1 --2- 172.21.15.33:0/2314157675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f09a41542b0 0x7f09a40c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.880+0000 7f09a3fff700 1 -- 172.21.15.33:0/2314157675 >> v1:172.21.15.69:6789/0 conn(0x7f09a40c8ee0 legacy=0x7f09a40ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.880+0000 7f09a3fff700 1 -- 172.21.15.33:0/2314157675 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40c9370 con 0x7f09a4153ed0 2023-11-06T20:45:29.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.880+0000 7f09a3fff700 1 --2- 172.21.15.33:0/2314157675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4153ed0 0x7f09a4158600 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f0994006fd0 tx=0x7f09940090f0).ready entity=mon.0 client_cookie=3fe0a02068dfafb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:29.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.880+0000 7f09a2ffd700 1 -- 172.21.15.33:0/2314157675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099400b600 con 0x7f09a4153ed0 2023-11-06T20:45:29.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.880+0000 7f09a2ffd700 1 -- 172.21.15.33:0/2314157675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099400b760 con 0x7f09a4153ed0 2023-11-06T20:45:29.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.880+0000 7f09a2ffd700 1 -- 172.21.15.33:0/2314157675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099400bae0 con 0x7f09a4153ed0 2023-11-06T20:45:29.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.881+0000 7f09aa3ff700 1 -- 172.21.15.33:0/2314157675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4153ed0 msgr2=0x7f09a4158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.881+0000 7f09aa3ff700 1 --2- 172.21.15.33:0/2314157675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4153ed0 0x7f09a4158600 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f0994006fd0 tx=0x7f09940090f0).stop 2023-11-06T20:45:29.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.881+0000 7f09aa3ff700 1 -- 172.21.15.33:0/2314157675 shutdown_connections 2023-11-06T20:45:29.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.881+0000 7f09aa3ff700 1 --2- 172.21.15.33:0/2314157675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4153ed0 0x7f09a4158600 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.881+0000 7f09aa3ff700 1 --2- 172.21.15.33:0/2314157675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f09a41542b0 0x7f09a40c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.881+0000 7f09aa3ff700 1 -- 172.21.15.33:0/2314157675 >> 172.21.15.33:0/2314157675 conn(0x7f09a405e1b0 msgr2=0x7f09a405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:29.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.881+0000 7f09aa3ff700 1 -- 172.21.15.33:0/2314157675 shutdown_connections 2023-11-06T20:45:29.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.881+0000 7f09aa3ff700 1 -- 172.21.15.33:0/2314157675 wait complete. 2023-11-06T20:45:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.883+0000 7f09aa3ff700 1 Processor -- start 2023-11-06T20:45:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.883+0000 7f09aa3ff700 1 -- start start 2023-11-06T20:45:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.884+0000 7f09aa3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a4153ed0 0x7f09a4145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.884+0000 7f09aa3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a41542b0 0x7f09a4158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.884+0000 7f09aa3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f09a4168160 0x7f09a4156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.884+0000 7f09aa3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41595e0 con 0x7f09a41542b0 2023-11-06T20:45:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.884+0000 7f09aa3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41598c0 con 0x7f09a4153ed0 2023-11-06T20:45:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.884+0000 7f09aa3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4159ba0 con 0x7f09a4168160 2023-11-06T20:45:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a41542b0 0x7f09a4158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a4153ed0 0x7f09a4145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a41542b0 0x7f09a4158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45244/0 (socket says 172.21.15.33:45244) 2023-11-06T20:45:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a4153ed0 0x7f09a4145900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:60518/0 (socket says 172.21.15.33:60518) 2023-11-06T20:45:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a37fe700 1 -- 172.21.15.33:0/251842593 learned_addr learned my addr 172.21.15.33:0/251842593 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a899c700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f09a4168160 0x7f09a4156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a37fe700 1 -- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f09a4168160 msgr2=0x7f09a4156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a37fe700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f09a4168160 0x7f09a4156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a37fe700 1 -- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a4153ed0 msgr2=0x7f09a4145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a37fe700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a4153ed0 0x7f09a4145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a37fe700 1 -- 172.21.15.33:0/251842593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994009370 con 0x7f09a41542b0 2023-11-06T20:45:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.885+0000 7f09a37fe700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a41542b0 0x7f09a4158b70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f099800c4c0 tx=0x7f0998007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.886+0000 7f098bfff700 1 -- 172.21.15.33:0/251842593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800ce70 con 0x7f09a41542b0 2023-11-06T20:45:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.886+0000 7f098bfff700 1 -- 172.21.15.33:0/251842593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0998001e90 con 0x7f09a41542b0 2023-11-06T20:45:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.886+0000 7f098bfff700 1 -- 172.21.15.33:0/251842593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998007250 con 0x7f09a41542b0 2023-11-06T20:45:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.886+0000 7f09aa3ff700 1 -- 172.21.15.33:0/251842593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a40c3010 con 0x7f09a41542b0 2023-11-06T20:45:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.886+0000 7f09aa3ff700 1 -- 172.21.15.33:0/251842593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a4159060 con 0x7f09a41542b0 2023-11-06T20:45:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.887+0000 7f09aa3ff700 1 -- 172.21.15.33:0/251842593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a40c3250 con 0x7f09a41542b0 2023-11-06T20:45:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.888+0000 7f098bfff700 1 -- 172.21.15.33:0/251842593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f09980066a0 con 0x7f09a41542b0 2023-11-06T20:45:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.889+0000 7f098bfff700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f099005f350 0x7f0990061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.889+0000 7f098bfff700 1 -- 172.21.15.33:0/251842593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f0998079f10 con 0x7f09a41542b0 2023-11-06T20:45:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.889+0000 7f09a3fff700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f099005f350 0x7f0990061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.890+0000 7f09a3fff700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f099005f350 0x7f0990061810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f0994009130 tx=0x7f0994017000).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:29.893+0000 7f098bfff700 1 -- 172.21.15.33:0/251842593 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f09a40c3250 con 0x7f09a41542b0 2023-11-06T20:45:30.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.104+0000 7f09aa3ff700 1 -- 172.21.15.33:0/251842593 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f09a40c3250 con 0x7f099005f350 2023-11-06T20:45:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.111+0000 7f098bfff700 1 -- 172.21.15.33:0/251842593 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f09a40c3250 con 0x7f099005f350 2023-11-06T20:45:30.111 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-11-06T20:45:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 -- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f099005f350 msgr2=0x7f0990061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f099005f350 0x7f0990061810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f0994009130 tx=0x7f0994017000).stop 2023-11-06T20:45:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 -- 172.21.15.33:0/251842593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a41542b0 msgr2=0x7f09a4158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a41542b0 0x7f09a4158b70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f099800c4c0 tx=0x7f0998007e00).stop 2023-11-06T20:45:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 -- 172.21.15.33:0/251842593 shutdown_connections 2023-11-06T20:45:30.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f099005f350 0x7f0990061810 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f09a4168160 0x7f09a4156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a4153ed0 0x7f09a4145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 --2- 172.21.15.33:0/251842593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a41542b0 0x7f09a4158b70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 -- 172.21.15.33:0/251842593 >> 172.21.15.33:0/251842593 conn(0x7f09a405e1b0 msgr2=0x7f09a40abcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:30.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 -- 172.21.15.33:0/251842593 shutdown_connections 2023-11-06T20:45:30.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.113+0000 7f09aa3ff700 1 -- 172.21.15.33:0/251842593 wait complete. 2023-11-06T20:45:30.130 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-11-06T20:45:30.131 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-11-06T20:45:30.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.334+0000 7f7259bff700 1 Processor -- start 2023-11-06T20:45:30.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.334+0000 7f7259bff700 1 -- start start 2023-11-06T20:45:30.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.335+0000 7f7259bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 0x7f7254158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.335+0000 7f7259bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f72540c8ee0 0x7f72540ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.335+0000 7f7259bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72540c9f80 con 0x7f72541542b0 2023-11-06T20:45:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.335+0000 7f7259bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72540c6000 con 0x7f72540c8ee0 2023-11-06T20:45:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.335+0000 7f7259bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72540c9370 con 0x7f7254153ed0 2023-11-06T20:45:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.336+0000 7f7252ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f72540c8ee0 0x7f72540ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.336+0000 7f7253fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f72541542b0 0x7f72540c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46632/0 (socket says 172.21.15.33:46632) 2023-11-06T20:45:30.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.336+0000 7f7253fff700 1 -- 172.21.15.33:0/3591319356 learned_addr learned my addr 172.21.15.33:0/3591319356 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:30.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.336+0000 7f72537fe700 1 --2- 172.21.15.33:0/3591319356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 0x7f7254158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.336+0000 7f72527fc700 1 -- 172.21.15.33:0/3591319356 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554575968 0 0) 0x7f72540c9f80 con 0x7f72541542b0 2023-11-06T20:45:30.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.336+0000 7f7252ffd700 1 -- 172.21.15.33:0/3591319356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 msgr2=0x7f7254158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:30.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.336+0000 7f7252ffd700 1 --2- 172.21.15.33:0/3591319356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 0x7f7254158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.336+0000 7f7252ffd700 1 -- 172.21.15.33:0/3591319356 >> v1:172.21.15.33:6789/0 conn(0x7f72541542b0 legacy=0x7f72540c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.336+0000 7f7252ffd700 1 -- 172.21.15.33:0/3591319356 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72540c9650 con 0x7f72540c8ee0 2023-11-06T20:45:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f7252ffd700 1 --2- 172.21.15.33:0/3591319356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72540c8ee0 0x7f72540ca410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f7248009760 tx=0x7f7248009060).ready entity=mon.0 client_cookie=32fad4689733974c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f72527fc700 1 -- 172.21.15.33:0/3591319356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724800b580 con 0x7f72540c8ee0 2023-11-06T20:45:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f72527fc700 1 -- 172.21.15.33:0/3591319356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724800b6e0 con 0x7f72540c8ee0 2023-11-06T20:45:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f72527fc700 1 -- 172.21.15.33:0/3591319356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724800ba60 con 0x7f72540c8ee0 2023-11-06T20:45:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f7259bff700 1 -- 172.21.15.33:0/3591319356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72540c8ee0 msgr2=0x7f72540ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f7259bff700 1 --2- 172.21.15.33:0/3591319356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72540c8ee0 0x7f72540ca410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f7248009760 tx=0x7f7248009060).stop 2023-11-06T20:45:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f7259bff700 1 -- 172.21.15.33:0/3591319356 shutdown_connections 2023-11-06T20:45:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f7259bff700 1 --2- 172.21.15.33:0/3591319356 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 0x7f7254158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f7259bff700 1 --2- 172.21.15.33:0/3591319356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72540c8ee0 0x7f72540ca410 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f7259bff700 1 -- 172.21.15.33:0/3591319356 >> 172.21.15.33:0/3591319356 conn(0x7f725405e1b0 msgr2=0x7f725405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:30.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.337+0000 7f7259bff700 1 -- 172.21.15.33:0/3591319356 shutdown_connections 2023-11-06T20:45:30.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.338+0000 7f7259bff700 1 -- 172.21.15.33:0/3591319356 wait complete. 2023-11-06T20:45:30.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.339+0000 7f7259bff700 1 Processor -- start 2023-11-06T20:45:30.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.339+0000 7f7259bff700 1 -- start start 2023-11-06T20:45:30.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f7259bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 0x7f7254150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f7259bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72541542b0 0x7f72540afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f7259bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72540b0320 0x7f7254156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f7259bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7254156850 con 0x7f72541542b0 2023-11-06T20:45:30.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f7259bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540b0780 con 0x7f72540b0320 2023-11-06T20:45:30.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f7259bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7254155c40 con 0x7f7254153ed0 2023-11-06T20:45:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f7252ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72541542b0 0x7f72540afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f7252ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72541542b0 0x7f72540afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45266/0 (socket says 172.21.15.33:45266) 2023-11-06T20:45:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f7252ffd700 1 -- 172.21.15.33:0/2304717993 learned_addr learned my addr 172.21.15.33:0/2304717993 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f72537fe700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 0x7f7254150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.340+0000 7f7253fff700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72540b0320 0x7f7254156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f7252ffd700 1 -- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 msgr2=0x7f7254150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f7252ffd700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 0x7f7254150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f7252ffd700 1 -- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72540b0320 msgr2=0x7f7254156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f7252ffd700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72540b0320 0x7f7254156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f7252ffd700 1 -- 172.21.15.33:0/2304717993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480092a0 con 0x7f72541542b0 2023-11-06T20:45:30.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f72537fe700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 0x7f7254150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:30.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f7252ffd700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72541542b0 0x7f72540afe30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f7248011000 tx=0x7f7248017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:30.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f723b7fe700 1 -- 172.21.15.33:0/2304717993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724800b850 con 0x7f72541542b0 2023-11-06T20:45:30.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f723b7fe700 1 -- 172.21.15.33:0/2304717993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724800b9b0 con 0x7f72541542b0 2023-11-06T20:45:30.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f7259bff700 1 -- 172.21.15.33:0/2304717993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7254155f20 con 0x7f72541542b0 2023-11-06T20:45:30.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f723b7fe700 1 -- 172.21.15.33:0/2304717993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724801b690 con 0x7f72541542b0 2023-11-06T20:45:30.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.341+0000 7f7259bff700 1 -- 172.21.15.33:0/2304717993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72541564e0 con 0x7f72541542b0 2023-11-06T20:45:30.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.342+0000 7f7259bff700 1 -- 172.21.15.33:0/2304717993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72541569c0 con 0x7f72541542b0 2023-11-06T20:45:30.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.343+0000 7f723b7fe700 1 -- 172.21.15.33:0/2304717993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f724801bac0 con 0x7f72541542b0 2023-11-06T20:45:30.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.344+0000 7f723b7fe700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724005f350 0x7f7240061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.344+0000 7f723b7fe700 1 -- 172.21.15.33:0/2304717993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f724807bf40 con 0x7f72541542b0 2023-11-06T20:45:30.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.344+0000 7f72537fe700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724005f350 0x7f7240061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.345+0000 7f72537fe700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724005f350 0x7f7240061810 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f724400bb60 tx=0x7f7244008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:30.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.348+0000 7f723b7fe700 1 -- 172.21.15.33:0/2304717993 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f724804ce60 con 0x7f72541542b0 2023-11-06T20:45:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.557+0000 7f7259bff700 1 -- 172.21.15.33:0/2304717993 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f725406c6c0 con 0x7f724005f350 2023-11-06T20:45:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.563+0000 7f723b7fe700 1 -- 172.21.15.33:0/2304717993 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f725406c6c0 con 0x7f724005f350 2023-11-06T20:45:30.563 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-11-06T20:45:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.564+0000 7f7259bff700 1 -- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724005f350 msgr2=0x7f7240061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.564+0000 7f7259bff700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724005f350 0x7f7240061810 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f724400bb60 tx=0x7f7244008040).stop 2023-11-06T20:45:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.564+0000 7f7259bff700 1 -- 172.21.15.33:0/2304717993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72541542b0 msgr2=0x7f72540afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.564+0000 7f7259bff700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72541542b0 0x7f72540afe30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f7248011000 tx=0x7f7248017000).stop 2023-11-06T20:45:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.564+0000 7f7259bff700 1 -- 172.21.15.33:0/2304717993 shutdown_connections 2023-11-06T20:45:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.564+0000 7f7259bff700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724005f350 0x7f7240061810 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.564+0000 7f7259bff700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7254153ed0 0x7f7254150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.564+0000 7f7259bff700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72540b0320 0x7f7254156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.564+0000 7f7259bff700 1 --2- 172.21.15.33:0/2304717993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72541542b0 0x7f72540afe30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.564+0000 7f7259bff700 1 -- 172.21.15.33:0/2304717993 >> 172.21.15.33:0/2304717993 conn(0x7f725405e1b0 msgr2=0x7f72540cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.565+0000 7f7259bff700 1 -- 172.21.15.33:0/2304717993 shutdown_connections 2023-11-06T20:45:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.565+0000 7f7259bff700 1 -- 172.21.15.33:0/2304717993 wait complete. 2023-11-06T20:45:30.582 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-11-06T20:45:30.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-11-06T20:45:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.783+0000 7f246f5ff700 1 Processor -- start 2023-11-06T20:45:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.783+0000 7f246f5ff700 1 -- start start 2023-11-06T20:45:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.784+0000 7f246f5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2468064810 0x7f2468064bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.784+0000 7f246f5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2468155370 0x7f24680c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.784+0000 7f246f5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 0x7f24680c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.784+0000 7f246f5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2468154ee0 con 0x7f2468064810 2023-11-06T20:45:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.784+0000 7f246f5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2468154bb0 con 0x7f2468155370 2023-11-06T20:45:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.784+0000 7f246f5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24680c0f10 con 0x7f24680c5770 2023-11-06T20:45:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.784+0000 7f246d39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2468064810 0x7f2468064bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.784+0000 7f246d39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2468064810 0x7f2468064bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45280/0 (socket says 172.21.15.33:45280) 2023-11-06T20:45:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f246d39b700 1 -- 172.21.15.33:0/311501423 learned_addr learned my addr 172.21.15.33:0/311501423 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f246cb9a700 1 --2- 172.21.15.33:0/311501423 >> v2:172.21.15.69:3300/0 conn(0x7f2468155370 0x7f24680c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f246db9c700 1 --2- 172.21.15.33:0/311501423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 0x7f24680c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f246d39b700 1 -- 172.21.15.33:0/311501423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 msgr2=0x7f24680c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f246d39b700 1 --2- 172.21.15.33:0/311501423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 0x7f24680c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f246d39b700 1 -- 172.21.15.33:0/311501423 >> v2:172.21.15.69:3300/0 conn(0x7f2468155370 msgr2=0x7f24680c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:30.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f246d39b700 1 --2- 172.21.15.33:0/311501423 >> v2:172.21.15.69:3300/0 conn(0x7f2468155370 0x7f24680c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f246d39b700 1 -- 172.21.15.33:0/311501423 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24680c0950 con 0x7f2468064810 2023-11-06T20:45:30.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f246db9c700 1 --2- 172.21.15.33:0/311501423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 0x7f24680c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:30.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f246d39b700 1 --2- 172.21.15.33:0/311501423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 0x7f2468064bf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f24680bffd0 tx=0x7f24640090f0).ready entity=mon.0 client_cookie=6cd771bbb57b8a96 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:30.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f245ffff700 1 -- 172.21.15.33:0/311501423 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246400b600 con 0x7f2468064810 2023-11-06T20:45:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f245ffff700 1 -- 172.21.15.33:0/311501423 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246400b760 con 0x7f2468064810 2023-11-06T20:45:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.785+0000 7f245ffff700 1 -- 172.21.15.33:0/311501423 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246400bae0 con 0x7f2468064810 2023-11-06T20:45:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.786+0000 7f246f5ff700 1 -- 172.21.15.33:0/311501423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 msgr2=0x7f2468064bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.786+0000 7f246f5ff700 1 --2- 172.21.15.33:0/311501423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 0x7f2468064bf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f24680bffd0 tx=0x7f24640090f0).stop 2023-11-06T20:45:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.786+0000 7f246f5ff700 1 -- 172.21.15.33:0/311501423 shutdown_connections 2023-11-06T20:45:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.786+0000 7f246f5ff700 1 --2- 172.21.15.33:0/311501423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 0x7f24680c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.786+0000 7f246f5ff700 1 --2- 172.21.15.33:0/311501423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 0x7f2468064bf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.786+0000 7f246f5ff700 1 --2- 172.21.15.33:0/311501423 >> v2:172.21.15.69:3300/0 conn(0x7f2468155370 0x7f24680c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.786+0000 7f246f5ff700 1 -- 172.21.15.33:0/311501423 >> 172.21.15.33:0/311501423 conn(0x7f24680cb850 msgr2=0x7f24680c7870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.786+0000 7f246f5ff700 1 -- 172.21.15.33:0/311501423 shutdown_connections 2023-11-06T20:45:30.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.786+0000 7f246f5ff700 1 -- 172.21.15.33:0/311501423 wait complete. 2023-11-06T20:45:30.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.788+0000 7f246f5ff700 1 Processor -- start 2023-11-06T20:45:30.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.788+0000 7f246f5ff700 1 -- start start 2023-11-06T20:45:30.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246f5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 0x7f2468144980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246f5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 0x7f24680ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246f5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2468155370 0x7f24680c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246f5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24680c2280 con 0x7f2468064810 2023-11-06T20:45:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246f5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24680aff40 con 0x7f2468155370 2023-11-06T20:45:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246f5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24680b0220 con 0x7f24680c5770 2023-11-06T20:45:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246d39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 0x7f2468144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246d39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 0x7f2468144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45282/0 (socket says 172.21.15.33:45282) 2023-11-06T20:45:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246d39b700 1 -- 172.21.15.33:0/3390695630 learned_addr learned my addr 172.21.15.33:0/3390695630 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246cb9a700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 0x7f24680ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246db9c700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2468155370 0x7f24680c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246d39b700 1 -- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 msgr2=0x7f24680ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246d39b700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 0x7f24680ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246d39b700 1 -- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2468155370 msgr2=0x7f24680c4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:30.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246d39b700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2468155370 0x7f24680c4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:30.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.790+0000 7f246d39b700 1 -- 172.21.15.33:0/3390695630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2464009370 con 0x7f2468064810 2023-11-06T20:45:30.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.791+0000 7f246d39b700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 0x7f2468144980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f2464009130 tx=0x7f2464017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:30.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.791+0000 7f245d7fa700 1 -- 172.21.15.33:0/3390695630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246400b8d0 con 0x7f2468064810 2023-11-06T20:45:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.791+0000 7f245d7fa700 1 -- 172.21.15.33:0/3390695630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246400ba30 con 0x7f2468064810 2023-11-06T20:45:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.791+0000 7f246f5ff700 1 -- 172.21.15.33:0/3390695630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24680c1f10 con 0x7f2468064810 2023-11-06T20:45:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.791+0000 7f245d7fa700 1 -- 172.21.15.33:0/3390695630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246401b660 con 0x7f2468064810 2023-11-06T20:45:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.791+0000 7f246f5ff700 1 -- 172.21.15.33:0/3390695630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2468144e70 con 0x7f2468064810 2023-11-06T20:45:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.792+0000 7f246f5ff700 1 -- 172.21.15.33:0/3390695630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24680c0c30 con 0x7f2468064810 2023-11-06T20:45:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.793+0000 7f245d7fa700 1 -- 172.21.15.33:0/3390695630 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f246401b9d0 con 0x7f2468064810 2023-11-06T20:45:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.794+0000 7f245d7fa700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f244c05f350 0x7f244c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.794+0000 7f245d7fa700 1 -- 172.21.15.33:0/3390695630 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f246407beb0 con 0x7f2468064810 2023-11-06T20:45:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.794+0000 7f246cb9a700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f244c05f350 0x7f244c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.795+0000 7f246cb9a700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f244c05f350 0x7f244c061810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f24680cbe60 tx=0x7f2458008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:30.798+0000 7f245d7fa700 1 -- 172.21.15.33:0/3390695630 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f246404ced0 con 0x7f2468064810 2023-11-06T20:45:31.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.009+0000 7f246f5ff700 1 -- 172.21.15.33:0/3390695630 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f24680c23c0 con 0x7f244c05f350 2023-11-06T20:45:31.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.014+0000 7f245d7fa700 1 -- 172.21.15.33:0/3390695630 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f245800b800 con 0x7f244c05f350 2023-11-06T20:45:31.014 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-11-06T20:45:31.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 -- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f244c05f350 msgr2=0x7f244c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f244c05f350 0x7f244c061810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f24680cbe60 tx=0x7f2458008040).stop 2023-11-06T20:45:31.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 -- 172.21.15.33:0/3390695630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 msgr2=0x7f2468144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 0x7f2468144980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f2464009130 tx=0x7f2464017000).stop 2023-11-06T20:45:31.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 -- 172.21.15.33:0/3390695630 shutdown_connections 2023-11-06T20:45:31.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f244c05f350 0x7f244c061810 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2468155370 0x7f24680c4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2468064810 0x7f2468144980 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 --2- 172.21.15.33:0/3390695630 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24680c5770 0x7f24680ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 -- 172.21.15.33:0/3390695630 >> 172.21.15.33:0/3390695630 conn(0x7f24680cb850 msgr2=0x7f24680c8050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:31.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 -- 172.21.15.33:0/3390695630 shutdown_connections 2023-11-06T20:45:31.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.016+0000 7f246f5ff700 1 -- 172.21.15.33:0/3390695630 wait complete. 2023-11-06T20:45:31.034 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-11-06T20:45:31.035 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-11-06T20:45:31.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.238+0000 7f8fd5fff700 1 Processor -- start 2023-11-06T20:45:31.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.238+0000 7f8fd5fff700 1 -- start start 2023-11-06T20:45:31.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.239+0000 7f8fd5fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8fd00c8ee0 0x7f8fd00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.239+0000 7f8fd5fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd00c9f80 con 0x7f8fd01542b0 2023-11-06T20:45:31.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.239+0000 7f8fd5fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd00c6000 con 0x7f8fd0153ed0 2023-11-06T20:45:31.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.239+0000 7f8fd5fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd00c9370 con 0x7f8fd00c8ee0 2023-11-06T20:45:31.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.240+0000 7f8fceffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8fd00c8ee0 0x7f8fd00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.240+0000 7f8fcffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8fd01542b0 0x7f8fd00c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46646/0 (socket says 172.21.15.33:46646) 2023-11-06T20:45:31.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.240+0000 7f8fcffff700 1 -- 172.21.15.33:0/770196475 learned_addr learned my addr 172.21.15.33:0/770196475 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:31.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.240+0000 7f8fceffd700 1 -- 172.21.15.33:0/770196475 >> v1:172.21.15.69:6789/0 conn(0x7f8fd0153ed0 legacy=0x7f8fd0158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:31.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.240+0000 7f8fceffd700 1 -- 172.21.15.33:0/770196475 >> v1:172.21.15.33:6789/0 conn(0x7f8fd01542b0 legacy=0x7f8fd00c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.240+0000 7f8fceffd700 1 -- 172.21.15.33:0/770196475 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd00c9650 con 0x7f8fd00c8ee0 2023-11-06T20:45:31.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.240+0000 7f8fce7fc700 1 -- 172.21.15.33:0/770196475 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966138208 0 0) 0x7f8fd00c9f80 con 0x7f8fd01542b0 2023-11-06T20:45:31.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.240+0000 7f8fceffd700 1 --2- 172.21.15.33:0/770196475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd00c8ee0 0x7f8fd00ca410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f8fc4009760 tx=0x7f8fc4009060).ready entity=mon.0 client_cookie=dc91e7baebc51fcb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:31.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.241+0000 7f8fce7fc700 1 -- 172.21.15.33:0/770196475 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400b520 con 0x7f8fd00c8ee0 2023-11-06T20:45:31.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.241+0000 7f8fce7fc700 1 -- 172.21.15.33:0/770196475 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc400b680 con 0x7f8fd00c8ee0 2023-11-06T20:45:31.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.241+0000 7f8fce7fc700 1 -- 172.21.15.33:0/770196475 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400ba00 con 0x7f8fd00c8ee0 2023-11-06T20:45:31.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.241+0000 7f8fd5fff700 1 -- 172.21.15.33:0/770196475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd00c8ee0 msgr2=0x7f8fd00ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.241+0000 7f8fd5fff700 1 --2- 172.21.15.33:0/770196475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd00c8ee0 0x7f8fd00ca410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f8fc4009760 tx=0x7f8fc4009060).stop 2023-11-06T20:45:31.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.241+0000 7f8fd5fff700 1 -- 172.21.15.33:0/770196475 shutdown_connections 2023-11-06T20:45:31.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.241+0000 7f8fd5fff700 1 --2- 172.21.15.33:0/770196475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd00c8ee0 0x7f8fd00ca410 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.241+0000 7f8fd5fff700 1 -- 172.21.15.33:0/770196475 >> 172.21.15.33:0/770196475 conn(0x7f8fd005e1b0 msgr2=0x7f8fd005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:31.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.241+0000 7f8fd5fff700 1 -- 172.21.15.33:0/770196475 shutdown_connections 2023-11-06T20:45:31.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.241+0000 7f8fd5fff700 1 -- 172.21.15.33:0/770196475 wait complete. 2023-11-06T20:45:31.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.243+0000 7f8fd5fff700 1 Processor -- start 2023-11-06T20:45:31.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.243+0000 7f8fd5fff700 1 -- start start 2023-11-06T20:45:31.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.244+0000 7f8fd5fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8fd0153ed0 0x7f8fd01659f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.244+0000 7f8fd5fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd01542b0 0x7f8fd00b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.244+0000 7f8fd5fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fd0167910 0x7f8fd00ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.244+0000 7f8fd5fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0155e90 con 0x7f8fd01542b0 2023-11-06T20:45:31.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.244+0000 7f8fd5fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0165ee0 con 0x7f8fd0167910 2023-11-06T20:45:31.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.244+0000 7f8fd5fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0165810 con 0x7f8fd0153ed0 2023-11-06T20:45:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.244+0000 7f8fceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd01542b0 0x7f8fd00b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.244+0000 7f8fceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd01542b0 0x7f8fd00b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45290/0 (socket says 172.21.15.33:45290) 2023-11-06T20:45:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.244+0000 7f8fceffd700 1 -- 172.21.15.33:0/2292179687 learned_addr learned my addr 172.21.15.33:0/2292179687 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.244+0000 7f8fcf7fe700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8fd0153ed0 0x7f8fd01659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fcffff700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fd0167910 0x7f8fd00ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fceffd700 1 -- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8fd0153ed0 msgr2=0x7f8fd01659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fceffd700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8fd0153ed0 0x7f8fd01659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fceffd700 1 -- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fd0167910 msgr2=0x7f8fd00ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fceffd700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fd0167910 0x7f8fd00ac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fceffd700 1 -- 172.21.15.33:0/2292179687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40092a0 con 0x7f8fd01542b0 2023-11-06T20:45:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fcf7fe700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8fd0153ed0 0x7f8fd01659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fcffff700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fd0167910 0x7f8fd00ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fceffd700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd01542b0 0x7f8fd00b02c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8fc4009eb0 tx=0x7f8fc4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fb77fe700 1 -- 172.21.15.33:0/2292179687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400b7f0 con 0x7f8fd01542b0 2023-11-06T20:45:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fb77fe700 1 -- 172.21.15.33:0/2292179687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc400b950 con 0x7f8fd01542b0 2023-11-06T20:45:31.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fb77fe700 1 -- 172.21.15.33:0/2292179687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc401b650 con 0x7f8fd01542b0 2023-11-06T20:45:31.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fd5fff700 1 -- 172.21.15.33:0/2292179687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd0164c00 con 0x7f8fd01542b0 2023-11-06T20:45:31.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.245+0000 7f8fd5fff700 1 -- 172.21.15.33:0/2292179687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd01651c0 con 0x7f8fd01542b0 2023-11-06T20:45:31.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.246+0000 7f8fd5fff700 1 -- 172.21.15.33:0/2292179687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd00aef50 con 0x7f8fd01542b0 2023-11-06T20:45:31.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.247+0000 7f8fb77fe700 1 -- 172.21.15.33:0/2292179687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f8fc401b9c0 con 0x7f8fd01542b0 2023-11-06T20:45:31.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.248+0000 7f8fb77fe700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fbc05f350 0x7f8fbc061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.249+0000 7f8fb77fe700 1 -- 172.21.15.33:0/2292179687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f8fc407bfe0 con 0x7f8fd01542b0 2023-11-06T20:45:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.249+0000 7f8fcf7fe700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fbc05f350 0x7f8fbc061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.249+0000 7f8fcf7fe700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fbc05f350 0x7f8fbc061810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f8fd00b0250 tx=0x7f8fc0009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.253+0000 7f8fb77fe700 1 -- 172.21.15.33:0/2292179687 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8fd00aef50 con 0x7f8fd01542b0 2023-11-06T20:45:31.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.463+0000 7f8fd5fff700 1 -- 172.21.15.33:0/2292179687 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fd00c9930 con 0x7f8fbc05f350 2023-11-06T20:45:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.469+0000 7f8fb77fe700 1 -- 172.21.15.33:0/2292179687 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8fd00c9930 con 0x7f8fbc05f350 2023-11-06T20:45:31.469 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-11-06T20:45:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 -- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fbc05f350 msgr2=0x7f8fbc061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fbc05f350 0x7f8fbc061810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f8fd00b0250 tx=0x7f8fc0009040).stop 2023-11-06T20:45:31.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 -- 172.21.15.33:0/2292179687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd01542b0 msgr2=0x7f8fd00b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd01542b0 0x7f8fd00b02c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8fc4009eb0 tx=0x7f8fc4017000).stop 2023-11-06T20:45:31.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 -- 172.21.15.33:0/2292179687 shutdown_connections 2023-11-06T20:45:31.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fbc05f350 0x7f8fbc061810 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8fd0153ed0 0x7f8fd01659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fd0167910 0x7f8fd00ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 --2- 172.21.15.33:0/2292179687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fd01542b0 0x7f8fd00b02c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 -- 172.21.15.33:0/2292179687 >> 172.21.15.33:0/2292179687 conn(0x7f8fd005e1b0 msgr2=0x7f8fd00ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:31.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 -- 172.21.15.33:0/2292179687 shutdown_connections 2023-11-06T20:45:31.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.471+0000 7f8fd5fff700 1 -- 172.21.15.33:0/2292179687 wait complete. 2023-11-06T20:45:31.488 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-11-06T20:45:31.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-11-06T20:45:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.690+0000 7f64d7bff700 1 Processor -- start 2023-11-06T20:45:31.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.690+0000 7f64d7bff700 1 -- start start 2023-11-06T20:45:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.691+0000 7f64d7bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d0153ed0 0x7f64d0158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.691+0000 7f64d7bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f64d00c8ee0 0x7f64d00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.691+0000 7f64d7bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d00c9f80 con 0x7f64d01542b0 2023-11-06T20:45:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.691+0000 7f64d7bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d00c6000 con 0x7f64d00c8ee0 2023-11-06T20:45:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.691+0000 7f64d7bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00c9370 con 0x7f64d0153ed0 2023-11-06T20:45:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.692+0000 7f64d619c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f64d01542b0 0x7f64d00c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46656/0 (socket says 172.21.15.33:46656) 2023-11-06T20:45:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.692+0000 7f64d619c700 1 -- 172.21.15.33:0/3482623202 learned_addr learned my addr 172.21.15.33:0/3482623202 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.692+0000 7f64d599b700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d0153ed0 0x7f64d0158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.692+0000 7f64d519a700 1 --2- 172.21.15.33:0/3482623202 >> v2:172.21.15.69:3300/0 conn(0x7f64d00c8ee0 0x7f64d00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.692+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457197303 0 0) 0x7f64d00c9f80 con 0x7f64d01542b0 2023-11-06T20:45:31.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.692+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64bc0033f0 con 0x7f64d01542b0 2023-11-06T20:45:31.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.692+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3305189092 0 0) 0x7f64bc0033f0 con 0x7f64d01542b0 2023-11-06T20:45:31.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.692+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d0153ed0 msgr2=0x7f64d0158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:31.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.692+0000 7f64d4999700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d0153ed0 0x7f64d0158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 >> v2:172.21.15.69:3300/0 conn(0x7f64d00c8ee0 msgr2=0x7f64d00ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:31.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 --2- 172.21.15.33:0/3482623202 >> v2:172.21.15.69:3300/0 conn(0x7f64d00c8ee0 0x7f64d00ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00c9650 con 0x7f64d01542b0 2023-11-06T20:45:31.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f64c80017f0 con 0x7f64d01542b0 2023-11-06T20:45:31.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 >> v1:172.21.15.33:6789/0 conn(0x7f64d01542b0 legacy=0x7f64d00c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 0x7f64bc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d0153ed0 0x7f64bc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64bc004b80 0x7f64bc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00c6000 con 0x7f64bc0038e0 2023-11-06T20:45:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00c9370 con 0x7f64bc004b80 2023-11-06T20:45:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc00b8e0 con 0x7f64d0153ed0 2023-11-06T20:45:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d599b700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 0x7f64bc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d519a700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d0153ed0 0x7f64bc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d619c700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64bc004b80 0x7f64bc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d599b700 1 -- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d0153ed0 msgr2=0x7f64bc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:31.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d599b700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d0153ed0 0x7f64bc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d599b700 1 -- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64bc004b80 msgr2=0x7f64bc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:31.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d599b700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64bc004b80 0x7f64bc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d599b700 1 -- 172.21.15.33:0/3482623202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc0033f0 con 0x7f64bc0038e0 2023-11-06T20:45:31.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.693+0000 7f64d619c700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64bc004b80 0x7f64bc005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.694+0000 7f64d599b700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 0x7f64bc003d00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f64cc009a80 tx=0x7f64cc009060).ready entity=mon.0 client_cookie=3d9d7cee6674eaf1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.694+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc009430 con 0x7f64bc0038e0 2023-11-06T20:45:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.694+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc00e640 con 0x7f64bc0038e0 2023-11-06T20:45:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.694+0000 7f64d4999700 1 -- 172.21.15.33:0/3482623202 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00e9c0 con 0x7f64bc0038e0 2023-11-06T20:45:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.694+0000 7f64d7bff700 1 -- 172.21.15.33:0/3482623202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 msgr2=0x7f64bc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.694+0000 7f64d7bff700 1 --2- 172.21.15.33:0/3482623202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 0x7f64bc003d00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f64cc009a80 tx=0x7f64cc009060).stop 2023-11-06T20:45:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.694+0000 7f64d619c700 1 -- 172.21.15.33:0/3482623202 reap_dead start 2023-11-06T20:45:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.694+0000 7f64d7bff700 1 -- 172.21.15.33:0/3482623202 shutdown_connections 2023-11-06T20:45:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.694+0000 7f64d7bff700 1 -- 172.21.15.33:0/3482623202 >> 172.21.15.33:0/3482623202 conn(0x7f64d005e1b0 msgr2=0x7f64d005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.695+0000 7f64d7bff700 1 -- 172.21.15.33:0/3482623202 shutdown_connections 2023-11-06T20:45:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.695+0000 7f64d7bff700 1 -- 172.21.15.33:0/3482623202 wait complete. 2023-11-06T20:45:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.697+0000 7f64d7bff700 1 Processor -- start 2023-11-06T20:45:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.697+0000 7f64d7bff700 1 -- start start 2023-11-06T20:45:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.698+0000 7f64d7bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 0x7f64d00afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.698+0000 7f64d7bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d014c680 0x7f64d00addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.698+0000 7f64d7bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64d00ae2c0 0x7f64d01506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.698+0000 7f64d7bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00c3eb0 con 0x7f64bc0038e0 2023-11-06T20:45:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.698+0000 7f64d7bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00af5c0 con 0x7f64d00ae2c0 2023-11-06T20:45:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.698+0000 7f64d7bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00ae740 con 0x7f64d014c680 2023-11-06T20:45:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.698+0000 7f64d599b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 0x7f64d00afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.698+0000 7f64d599b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 0x7f64d00afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45310/0 (socket says 172.21.15.33:45310) 2023-11-06T20:45:31.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.698+0000 7f64d599b700 1 -- 172.21.15.33:0/3310180981 learned_addr learned my addr 172.21.15.33:0/3310180981 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:31.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64d519a700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d014c680 0x7f64d00addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64d619c700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64d00ae2c0 0x7f64d01506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64d599b700 1 -- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d014c680 msgr2=0x7f64d00addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64d599b700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d014c680 0x7f64d00addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64d599b700 1 -- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64d00ae2c0 msgr2=0x7f64d01506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64d599b700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64d00ae2c0 0x7f64d01506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64d599b700 1 -- 172.21.15.33:0/3310180981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00ae9b0 con 0x7f64bc0038e0 2023-11-06T20:45:31.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64d619c700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64d00ae2c0 0x7f64d01506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64d599b700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 0x7f64d00afa50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f64cc00e4b0 tx=0x7f64cc01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64c5ffb700 1 -- 172.21.15.33:0/3310180981 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00ed30 con 0x7f64bc0038e0 2023-11-06T20:45:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64c5ffb700 1 -- 172.21.15.33:0/3310180981 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc00ee90 con 0x7f64bc0038e0 2023-11-06T20:45:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64d7bff700 1 -- 172.21.15.33:0/3310180981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d00aec90 con 0x7f64bc0038e0 2023-11-06T20:45:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.699+0000 7f64c5ffb700 1 -- 172.21.15.33:0/3310180981 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc026440 con 0x7f64bc0038e0 2023-11-06T20:45:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.700+0000 7f64d7bff700 1 -- 172.21.15.33:0/3310180981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d00af250 con 0x7f64bc0038e0 2023-11-06T20:45:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.701+0000 7f64d7bff700 1 -- 172.21.15.33:0/3310180981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d006c6c0 con 0x7f64bc0038e0 2023-11-06T20:45:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.701+0000 7f64c5ffb700 1 -- 172.21.15.33:0/3310180981 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f64cc02f440 con 0x7f64bc0038e0 2023-11-06T20:45:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.702+0000 7f64c5ffb700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f64b405f350 0x7f64b4061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.702+0000 7f64c5ffb700 1 -- 172.21.15.33:0/3310180981 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f64cc07c320 con 0x7f64bc0038e0 2023-11-06T20:45:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.703+0000 7f64d519a700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f64b405f350 0x7f64b4061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.703+0000 7f64d519a700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f64b405f350 0x7f64b4061810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f64c0006c40 tx=0x7f64c0008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.706+0000 7f64c5ffb700 1 -- 172.21.15.33:0/3310180981 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f64cc04d2e0 con 0x7f64bc0038e0 2023-11-06T20:45:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.912+0000 7f64d7bff700 1 -- 172.21.15.33:0/3310180981 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f64d00c8cc0 con 0x7f64b405f350 2023-11-06T20:45:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.918+0000 7f64c5ffb700 1 -- 172.21.15.33:0/3310180981 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f64d00c8cc0 con 0x7f64b405f350 2023-11-06T20:45:31.918 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-11-06T20:45:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.919+0000 7f64d7bff700 1 -- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f64b405f350 msgr2=0x7f64b4061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.919+0000 7f64d7bff700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f64b405f350 0x7f64b4061810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f64c0006c40 tx=0x7f64c0008040).stop 2023-11-06T20:45:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.919+0000 7f64d7bff700 1 -- 172.21.15.33:0/3310180981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 msgr2=0x7f64d00afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.919+0000 7f64d7bff700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 0x7f64d00afa50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f64cc00e4b0 tx=0x7f64cc01a000).stop 2023-11-06T20:45:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.920+0000 7f64d7bff700 1 -- 172.21.15.33:0/3310180981 shutdown_connections 2023-11-06T20:45:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.920+0000 7f64d7bff700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f64b405f350 0x7f64b4061810 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.920+0000 7f64d7bff700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64d00ae2c0 0x7f64d01506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.920+0000 7f64d7bff700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64bc0038e0 0x7f64d00afa50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.920+0000 7f64d7bff700 1 --2- 172.21.15.33:0/3310180981 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64d014c680 0x7f64d00addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.920+0000 7f64d7bff700 1 -- 172.21.15.33:0/3310180981 >> 172.21.15.33:0/3310180981 conn(0x7f64d005e1b0 msgr2=0x7f64d015a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:31.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.920+0000 7f64d7bff700 1 -- 172.21.15.33:0/3310180981 shutdown_connections 2023-11-06T20:45:31.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:31.920+0000 7f64d7bff700 1 -- 172.21.15.33:0/3310180981 wait complete. 2023-11-06T20:45:31.937 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-11-06T20:45:31.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-11-06T20:45:32.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.139+0000 7f1a263ff700 1 Processor -- start 2023-11-06T20:45:32.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.140+0000 7f1a263ff700 1 -- start start 2023-11-06T20:45:32.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.141+0000 7f1a263ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a200c8ee0 0x7f1a200ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.141+0000 7f1a263ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1a201542b0 0x7f1a200c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.141+0000 7f1a263ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a200c9f80 con 0x7f1a20153ed0 2023-11-06T20:45:32.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.141+0000 7f1a263ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a200c6000 con 0x7f1a200c8ee0 2023-11-06T20:45:32.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.141+0000 7f1a263ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a200c9370 con 0x7f1a201542b0 2023-11-06T20:45:32.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.141+0000 7f1a1f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a200c8ee0 0x7f1a200ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.142+0000 7f1a1f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a200c8ee0 0x7f1a200ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45316/0 (socket says 172.21.15.33:45316) 2023-11-06T20:45:32.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.142+0000 7f1a1f7fe700 1 -- 172.21.15.33:0/786979316 learned_addr learned my addr 172.21.15.33:0/786979316 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:32.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.142+0000 7f1a2499c700 1 --2- 172.21.15.33:0/786979316 >> v2:172.21.15.69:3300/0 conn(0x7f1a201542b0 0x7f1a200c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.142+0000 7f1a1f7fe700 1 -- 172.21.15.33:0/786979316 >> v2:172.21.15.69:3300/0 conn(0x7f1a201542b0 msgr2=0x7f1a200c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:32.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.142+0000 7f1a1f7fe700 1 --2- 172.21.15.33:0/786979316 >> v2:172.21.15.69:3300/0 conn(0x7f1a201542b0 0x7f1a200c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.142+0000 7f1a1f7fe700 1 -- 172.21.15.33:0/786979316 >> v1:172.21.15.69:6789/0 conn(0x7f1a20153ed0 legacy=0x7f1a20158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.142+0000 7f1a1f7fe700 1 -- 172.21.15.33:0/786979316 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a200c9650 con 0x7f1a200c8ee0 2023-11-06T20:45:32.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.142+0000 7f1a1f7fe700 1 --2- 172.21.15.33:0/786979316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a200c8ee0 0x7f1a200ca410 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f1a200c85d0 tx=0x7f1a140090f0).ready entity=mon.0 client_cookie=1626da77c906e710 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:32.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a1effd700 1 -- 172.21.15.33:0/786979316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1400b600 con 0x7f1a200c8ee0 2023-11-06T20:45:32.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a1effd700 1 -- 172.21.15.33:0/786979316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1400b760 con 0x7f1a200c8ee0 2023-11-06T20:45:32.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a1effd700 1 -- 172.21.15.33:0/786979316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1400bae0 con 0x7f1a200c8ee0 2023-11-06T20:45:32.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a263ff700 1 -- 172.21.15.33:0/786979316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a200c8ee0 msgr2=0x7f1a200ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a263ff700 1 --2- 172.21.15.33:0/786979316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a200c8ee0 0x7f1a200ca410 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f1a200c85d0 tx=0x7f1a140090f0).stop 2023-11-06T20:45:32.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a263ff700 1 -- 172.21.15.33:0/786979316 shutdown_connections 2023-11-06T20:45:32.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a263ff700 1 --2- 172.21.15.33:0/786979316 >> v2:172.21.15.69:3300/0 conn(0x7f1a201542b0 0x7f1a200c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a263ff700 1 --2- 172.21.15.33:0/786979316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a200c8ee0 0x7f1a200ca410 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a263ff700 1 -- 172.21.15.33:0/786979316 >> 172.21.15.33:0/786979316 conn(0x7f1a2005e1b0 msgr2=0x7f1a2005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:32.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a263ff700 1 -- 172.21.15.33:0/786979316 shutdown_connections 2023-11-06T20:45:32.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.143+0000 7f1a263ff700 1 -- 172.21.15.33:0/786979316 wait complete. 2023-11-06T20:45:32.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.145+0000 7f1a263ff700 1 Processor -- start 2023-11-06T20:45:32.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.146+0000 7f1a263ff700 1 -- start start 2023-11-06T20:45:32.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a263ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1a20153ed0 0x7f1a2014cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a263ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a201542b0 0x7f1a2014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a263ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a20164370 0x7f1a200cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a263ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a201508a0 con 0x7f1a20164370 2023-11-06T20:45:32.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a263ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2014f9d0 con 0x7f1a201542b0 2023-11-06T20:45:32.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a263ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2014fc90 con 0x7f1a20153ed0 2023-11-06T20:45:32.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a2499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a20164370 0x7f1a200cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a2499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a20164370 0x7f1a200cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45318/0 (socket says 172.21.15.33:45318) 2023-11-06T20:45:32.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a2499c700 1 -- 172.21.15.33:0/3970654869 learned_addr learned my addr 172.21.15.33:0/3970654869 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:32.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a1ffff700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1a20153ed0 0x7f1a2014cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.147+0000 7f1a1f7fe700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a201542b0 0x7f1a2014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a2499c700 1 -- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1a20153ed0 msgr2=0x7f1a2014cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a2499c700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1a20153ed0 0x7f1a2014cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a2499c700 1 -- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a201542b0 msgr2=0x7f1a2014d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a2499c700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a201542b0 0x7f1a2014d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a2499c700 1 -- 172.21.15.33:0/3970654869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14009370 con 0x7f1a20164370 2023-11-06T20:45:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a1f7fe700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a201542b0 0x7f1a2014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a2499c700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a20164370 0x7f1a200cae00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f1a18007ef0 tx=0x7f1a18009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a07fff700 1 -- 172.21.15.33:0/3970654869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1800eca0 con 0x7f1a20164370 2023-11-06T20:45:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a07fff700 1 -- 172.21.15.33:0/3970654869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1800ee00 con 0x7f1a20164370 2023-11-06T20:45:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a263ff700 1 -- 172.21.15.33:0/3970654869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2014ff70 con 0x7f1a20164370 2023-11-06T20:45:32.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a07fff700 1 -- 172.21.15.33:0/3970654869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18014610 con 0x7f1a20164370 2023-11-06T20:45:32.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.148+0000 7f1a263ff700 1 -- 172.21.15.33:0/3970654869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a20150530 con 0x7f1a20164370 2023-11-06T20:45:32.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.149+0000 7f1a263ff700 1 -- 172.21.15.33:0/3970654869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a200c64c0 con 0x7f1a20164370 2023-11-06T20:45:32.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.150+0000 7f1a07fff700 1 -- 172.21.15.33:0/3970654869 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f1a18014a40 con 0x7f1a20164370 2023-11-06T20:45:32.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.151+0000 7f1a07fff700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1a0c05f380 0x7f1a0c061840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.151+0000 7f1a07fff700 1 -- 172.21.15.33:0/3970654869 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f1a1807ac00 con 0x7f1a20164370 2023-11-06T20:45:32.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.151+0000 7f1a1ffff700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1a0c05f380 0x7f1a0c061840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.152+0000 7f1a1ffff700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1a0c05f380 0x7f1a0c061840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f1a100036d0 tx=0x7f1a10009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:32.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.155+0000 7f1a07fff700 1 -- 172.21.15.33:0/3970654869 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1a1804bb20 con 0x7f1a20164370 2023-11-06T20:45:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.361+0000 7f1a263ff700 1 -- 172.21.15.33:0/3970654869 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a200c9930 con 0x7f1a0c05f380 2023-11-06T20:45:32.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.367+0000 7f1a07fff700 1 -- 172.21.15.33:0/3970654869 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f1a200c9930 con 0x7f1a0c05f380 2023-11-06T20:45:32.367 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-11-06T20:45:32.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 -- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1a0c05f380 msgr2=0x7f1a0c061840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1a0c05f380 0x7f1a0c061840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f1a100036d0 tx=0x7f1a10009040).stop 2023-11-06T20:45:32.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 -- 172.21.15.33:0/3970654869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a20164370 msgr2=0x7f1a200cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a20164370 0x7f1a200cae00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f1a18007ef0 tx=0x7f1a18009840).stop 2023-11-06T20:45:32.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 -- 172.21.15.33:0/3970654869 shutdown_connections 2023-11-06T20:45:32.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1a0c05f380 0x7f1a0c061840 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1a20153ed0 0x7f1a2014cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a20164370 0x7f1a200cae00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 --2- 172.21.15.33:0/3970654869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a201542b0 0x7f1a2014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 -- 172.21.15.33:0/3970654869 >> 172.21.15.33:0/3970654869 conn(0x7f1a2005e1b0 msgr2=0x7f1a2014e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:32.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 -- 172.21.15.33:0/3970654869 shutdown_connections 2023-11-06T20:45:32.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.369+0000 7f1a263ff700 1 -- 172.21.15.33:0/3970654869 wait complete. 2023-11-06T20:45:32.387 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-11-06T20:45:32.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-11-06T20:45:32.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.588+0000 7fa2355ff700 1 Processor -- start 2023-11-06T20:45:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.588+0000 7fa2355ff700 1 -- start start 2023-11-06T20:45:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.590+0000 7fa2355ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa230153ed0 0x7fa230158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.590+0000 7fa2355ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300c8ee0 0x7fa2300ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.590+0000 7fa2355ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2300c9f80 con 0x7fa2301542b0 2023-11-06T20:45:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.590+0000 7fa2355ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2300c6000 con 0x7fa230153ed0 2023-11-06T20:45:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.590+0000 7fa2355ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2300c9370 con 0x7fa2300c8ee0 2023-11-06T20:45:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.590+0000 7fa22e7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300c8ee0 0x7fa2300ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.590+0000 7fa22e7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300c8ee0 0x7fa2300ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:35364/0 (socket says 172.21.15.33:35364) 2023-11-06T20:45:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.590+0000 7fa22e7fc700 1 -- 172.21.15.33:0/2643491601 learned_addr learned my addr 172.21.15.33:0/2643491601 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.590+0000 7fa22effd700 1 --2- 172.21.15.33:0/2643491601 >> v2:172.21.15.69:3300/0 conn(0x7fa230153ed0 0x7fa230158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22dffb700 1 -- 172.21.15.33:0/2643491601 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630881881 0 0) 0x7fa2300c9f80 con 0x7fa2301542b0 2023-11-06T20:45:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22dffb700 1 -- 172.21.15.33:0/2643491601 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2180033f0 con 0x7fa2301542b0 2023-11-06T20:45:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22e7fc700 1 -- 172.21.15.33:0/2643491601 >> v2:172.21.15.69:3300/0 conn(0x7fa230153ed0 msgr2=0x7fa230158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22e7fc700 1 --2- 172.21.15.33:0/2643491601 >> v2:172.21.15.69:3300/0 conn(0x7fa230153ed0 0x7fa230158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22e7fc700 1 -- 172.21.15.33:0/2643491601 >> v1:172.21.15.69:6789/0 conn(0x7fa2301542b0 legacy=0x7fa2300c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22e7fc700 1 -- 172.21.15.33:0/2643491601 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2300c9650 con 0x7fa2300c8ee0 2023-11-06T20:45:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22effd700 1 --2- 172.21.15.33:0/2643491601 >> v2:172.21.15.69:3300/0 conn(0x7fa230153ed0 0x7fa230158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:45:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22e7fc700 1 --2- 172.21.15.33:0/2643491601 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300c8ee0 0x7fa2300ca410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fa224006fd0 tx=0x7fa2240090f0).ready entity=mon.2 client_cookie=cf9270d1f013781c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22dffb700 1 -- 172.21.15.33:0/2643491601 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400f510 con 0x7fa2300c8ee0 2023-11-06T20:45:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22dffb700 1 -- 172.21.15.33:0/2643491601 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22400f670 con 0x7fa2300c8ee0 2023-11-06T20:45:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.591+0000 7fa22dffb700 1 -- 172.21.15.33:0/2643491601 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400f9f0 con 0x7fa2300c8ee0 2023-11-06T20:45:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.592+0000 7fa2355ff700 1 -- 172.21.15.33:0/2643491601 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300c8ee0 msgr2=0x7fa2300ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.592+0000 7fa2355ff700 1 --2- 172.21.15.33:0/2643491601 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300c8ee0 0x7fa2300ca410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fa224006fd0 tx=0x7fa2240090f0).stop 2023-11-06T20:45:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.592+0000 7fa2355ff700 1 -- 172.21.15.33:0/2643491601 shutdown_connections 2023-11-06T20:45:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.592+0000 7fa2355ff700 1 --2- 172.21.15.33:0/2643491601 >> v2:172.21.15.69:3300/0 conn(0x7fa230153ed0 0x7fa230158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.592+0000 7fa2355ff700 1 --2- 172.21.15.33:0/2643491601 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300c8ee0 0x7fa2300ca410 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.592+0000 7fa2355ff700 1 -- 172.21.15.33:0/2643491601 >> 172.21.15.33:0/2643491601 conn(0x7fa23005e1b0 msgr2=0x7fa23005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.592+0000 7fa2355ff700 1 -- 172.21.15.33:0/2643491601 shutdown_connections 2023-11-06T20:45:32.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.592+0000 7fa2355ff700 1 -- 172.21.15.33:0/2643491601 wait complete. 2023-11-06T20:45:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.593+0000 7fa2355ff700 1 Processor -- start 2023-11-06T20:45:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.594+0000 7fa2355ff700 1 -- start start 2023-11-06T20:45:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa2355ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa230153ed0 0x7fa23014dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa2355ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2301542b0 0x7fa2300b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa2355ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300b0600 0x7fa2301516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa2355ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2301523d0 con 0x7fa2301542b0 2023-11-06T20:45:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa2355ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2300b0a80 con 0x7fa230153ed0 2023-11-06T20:45:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa2355ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa230167a80 con 0x7fa2300b0600 2023-11-06T20:45:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2301542b0 0x7fa2300b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2301542b0 0x7fa2300b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45334/0 (socket says 172.21.15.33:45334) 2023-11-06T20:45:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22e7fc700 1 -- 172.21.15.33:0/820894041 learned_addr learned my addr 172.21.15.33:0/820894041 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22effd700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa230153ed0 0x7fa23014dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22f7fe700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300b0600 0x7fa2301516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22e7fc700 1 -- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300b0600 msgr2=0x7fa2301516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22e7fc700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300b0600 0x7fa2301516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22e7fc700 1 -- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa230153ed0 msgr2=0x7fa23014dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22e7fc700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa230153ed0 0x7fa23014dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22e7fc700 1 -- 172.21.15.33:0/820894041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2200092a0 con 0x7fa2301542b0 2023-11-06T20:45:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.595+0000 7fa22effd700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa230153ed0 0x7fa23014dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.596+0000 7fa22e7fc700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2301542b0 0x7fa2300b0110 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fa224009df0 tx=0x7fa224017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.596+0000 7fa216ffd700 1 -- 172.21.15.33:0/820894041 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224006a40 con 0x7fa2301542b0 2023-11-06T20:45:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.596+0000 7fa216ffd700 1 -- 172.21.15.33:0/820894041 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22400f7e0 con 0x7fa2301542b0 2023-11-06T20:45:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.596+0000 7fa2355ff700 1 -- 172.21.15.33:0/820894041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa224009370 con 0x7fa2301542b0 2023-11-06T20:45:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.596+0000 7fa216ffd700 1 -- 172.21.15.33:0/820894041 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22401c650 con 0x7fa2301542b0 2023-11-06T20:45:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.596+0000 7fa2355ff700 1 -- 172.21.15.33:0/820894041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa230168040 con 0x7fa2301542b0 2023-11-06T20:45:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.597+0000 7fa2355ff700 1 -- 172.21.15.33:0/820894041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23006c6c0 con 0x7fa2301542b0 2023-11-06T20:45:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.598+0000 7fa216ffd700 1 -- 172.21.15.33:0/820894041 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7fa22401ca40 con 0x7fa2301542b0 2023-11-06T20:45:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.599+0000 7fa216ffd700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa21c05f350 0x7fa21c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.599+0000 7fa216ffd700 1 -- 172.21.15.33:0/820894041 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fa22407c120 con 0x7fa2301542b0 2023-11-06T20:45:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.600+0000 7fa22effd700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa21c05f350 0x7fa21c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:32.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.600+0000 7fa22effd700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa21c05f350 0x7fa21c061810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa220009270 tx=0x7fa220008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:32.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.603+0000 7fa216ffd700 1 -- 172.21.15.33:0/820894041 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa22404d140 con 0x7fa2301542b0 2023-11-06T20:45:32.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.814+0000 7fa2355ff700 1 -- 172.21.15.33:0/820894041 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2300c1730 con 0x7fa21c05f350 2023-11-06T20:45:32.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.820+0000 7fa216ffd700 1 -- 172.21.15.33:0/820894041 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa2300c1730 con 0x7fa21c05f350 2023-11-06T20:45:32.820 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-11-06T20:45:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 -- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa21c05f350 msgr2=0x7fa21c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa21c05f350 0x7fa21c061810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa220009270 tx=0x7fa220008040).stop 2023-11-06T20:45:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 -- 172.21.15.33:0/820894041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2301542b0 msgr2=0x7fa2300b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2301542b0 0x7fa2300b0110 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fa224009df0 tx=0x7fa224017000).stop 2023-11-06T20:45:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 -- 172.21.15.33:0/820894041 shutdown_connections 2023-11-06T20:45:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa21c05f350 0x7fa21c061810 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2300b0600 0x7fa2301516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa230153ed0 0x7fa23014dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 --2- 172.21.15.33:0/820894041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2301542b0 0x7fa2300b0110 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 -- 172.21.15.33:0/820894041 >> 172.21.15.33:0/820894041 conn(0x7fa23005e1b0 msgr2=0x7fa23014f400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 -- 172.21.15.33:0/820894041 shutdown_connections 2023-11-06T20:45:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:32.822+0000 7fa2355ff700 1 -- 172.21.15.33:0/820894041 wait complete. 2023-11-06T20:45:32.839 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-11-06T20:45:32.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-11-06T20:45:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.041+0000 7f279edff700 1 Processor -- start 2023-11-06T20:45:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.041+0000 7f279edff700 1 -- start start 2023-11-06T20:45:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.042+0000 7f279edff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27980c8ee0 0x7f27980ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.042+0000 7f279edff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27981542b0 0x7f27980c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.042+0000 7f279edff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27980c9f80 con 0x7f2798153ed0 2023-11-06T20:45:33.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.042+0000 7f279edff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27980c6000 con 0x7f27981542b0 2023-11-06T20:45:33.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.042+0000 7f279edff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c9370 con 0x7f27980c8ee0 2023-11-06T20:45:33.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f279cb9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2798153ed0 0x7f2798158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46672/0 (socket says 172.21.15.33:46672) 2023-11-06T20:45:33.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f279cb9b700 1 -- 172.21.15.33:0/1737246248 learned_addr learned my addr 172.21.15.33:0/1737246248 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:33.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f279d39c700 1 --2- 172.21.15.33:0/1737246248 >> v2:172.21.15.33:3300/0 conn(0x7f27981542b0 0x7f27980c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f2797fff700 1 --2- 172.21.15.33:0/1737246248 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27980c8ee0 0x7f27980ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f27977fe700 1 -- 172.21.15.33:0/1737246248 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65450685 0 0) 0x7f27980c9f80 con 0x7f2798153ed0 2023-11-06T20:45:33.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f27977fe700 1 -- 172.21.15.33:0/1737246248 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27880033f0 con 0x7f2798153ed0 2023-11-06T20:45:33.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f279d39c700 1 -- 172.21.15.33:0/1737246248 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27980c8ee0 msgr2=0x7f27980ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:33.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f279d39c700 1 --2- 172.21.15.33:0/1737246248 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27980c8ee0 0x7f27980ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f279d39c700 1 -- 172.21.15.33:0/1737246248 >> v1:172.21.15.33:6789/0 conn(0x7f2798153ed0 legacy=0x7f2798158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f279d39c700 1 -- 172.21.15.33:0/1737246248 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27980c9650 con 0x7f27981542b0 2023-11-06T20:45:33.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.043+0000 7f279d39c700 1 --2- 172.21.15.33:0/1737246248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27981542b0 0x7f27980c76f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f2790009e60 tx=0x7f2790009780).ready entity=mon.0 client_cookie=d8eaa70b47529ad1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f27977fe700 1 -- 172.21.15.33:0/1737246248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000e070 con 0x7f27981542b0 2023-11-06T20:45:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f27977fe700 1 -- 172.21.15.33:0/1737246248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279000bb60 con 0x7f27981542b0 2023-11-06T20:45:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f27977fe700 1 -- 172.21.15.33:0/1737246248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790012410 con 0x7f27981542b0 2023-11-06T20:45:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f279edff700 1 -- 172.21.15.33:0/1737246248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27981542b0 msgr2=0x7f27980c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f279edff700 1 --2- 172.21.15.33:0/1737246248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27981542b0 0x7f27980c76f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f2790009e60 tx=0x7f2790009780).stop 2023-11-06T20:45:33.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f279edff700 1 -- 172.21.15.33:0/1737246248 shutdown_connections 2023-11-06T20:45:33.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f279edff700 1 --2- 172.21.15.33:0/1737246248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27981542b0 0x7f27980c76f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f279edff700 1 --2- 172.21.15.33:0/1737246248 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27980c8ee0 0x7f27980ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f279edff700 1 -- 172.21.15.33:0/1737246248 >> 172.21.15.33:0/1737246248 conn(0x7f279805e1b0 msgr2=0x7f279805e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:33.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f279edff700 1 -- 172.21.15.33:0/1737246248 shutdown_connections 2023-11-06T20:45:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.044+0000 7f279edff700 1 -- 172.21.15.33:0/1737246248 wait complete. 2023-11-06T20:45:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.046+0000 7f279edff700 1 Processor -- start 2023-11-06T20:45:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.046+0000 7f279edff700 1 -- start start 2023-11-06T20:45:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.047+0000 7f279edff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2798153ed0 0x7f27980aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.047+0000 7f279edff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27981542b0 0x7f27980ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.047+0000 7f279edff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27980c3410 0x7f279814a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.047+0000 7f279edff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980afa80 con 0x7f27980c3410 2023-11-06T20:45:33.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.047+0000 7f279edff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980aee70 con 0x7f2798153ed0 2023-11-06T20:45:33.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.047+0000 7f279edff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980af150 con 0x7f27981542b0 2023-11-06T20:45:33.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.047+0000 7f279d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27980c3410 0x7f279814a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.047+0000 7f279d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27980c3410 0x7f279814a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45350/0 (socket says 172.21.15.33:45350) 2023-11-06T20:45:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.047+0000 7f279d39c700 1 -- 172.21.15.33:0/3474784805 learned_addr learned my addr 172.21.15.33:0/3474784805 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f2797fff700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27981542b0 0x7f27980ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f279d39c700 1 -- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27981542b0 msgr2=0x7f27980ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f279d39c700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27981542b0 0x7f27980ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f279d39c700 1 -- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2798153ed0 msgr2=0x7f27980aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f279cb9b700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2798153ed0 0x7f27980aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f279d39c700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2798153ed0 0x7f27980aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f279d39c700 1 -- 172.21.15.33:0/3474784805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2790009a20 con 0x7f27980c3410 2023-11-06T20:45:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f2797fff700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27981542b0 0x7f27980ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:33.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f279cb9b700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2798153ed0 0x7f27980aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:33.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f279d39c700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27980c3410 0x7f279814a820 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f279000b6c0 tx=0x7f2790009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:33.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f2794ff9700 1 -- 172.21.15.33:0/3474784805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279001d700 con 0x7f27980c3410 2023-11-06T20:45:33.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f2794ff9700 1 -- 172.21.15.33:0/3474784805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279001d860 con 0x7f27980c3410 2023-11-06T20:45:33.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f279edff700 1 -- 172.21.15.33:0/3474784805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27980af430 con 0x7f27980c3410 2023-11-06T20:45:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f2794ff9700 1 -- 172.21.15.33:0/3474784805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279001dc20 con 0x7f27980c3410 2023-11-06T20:45:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.048+0000 7f279edff700 1 -- 172.21.15.33:0/3474784805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27980c3850 con 0x7f27980c3410 2023-11-06T20:45:33.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.049+0000 7f279edff700 1 -- 172.21.15.33:0/3474784805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27980afbf0 con 0x7f27980c3410 2023-11-06T20:45:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.050+0000 7f2794ff9700 1 -- 172.21.15.33:0/3474784805 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f2790003040 con 0x7f27980c3410 2023-11-06T20:45:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.051+0000 7f2794ff9700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f277c05f320 0x7f277c0617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.051+0000 7f2794ff9700 1 -- 172.21.15.33:0/3474784805 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f279007c6e0 con 0x7f27980c3410 2023-11-06T20:45:33.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.052+0000 7f279cb9b700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f277c05f320 0x7f277c0617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.052+0000 7f279cb9b700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f277c05f320 0x7f277c0617e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f278c005750 tx=0x7f278c009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:33.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.055+0000 7f2794ff9700 1 -- 172.21.15.33:0/3474784805 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f279004d5b0 con 0x7f27980c3410 2023-11-06T20:45:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.265+0000 7f279edff700 1 -- 172.21.15.33:0/3474784805 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f279806c6c0 con 0x7f277c05f320 2023-11-06T20:45:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.270+0000 7f2794ff9700 1 -- 172.21.15.33:0/3474784805 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f279806c6c0 con 0x7f277c05f320 2023-11-06T20:45:33.270 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-11-06T20:45:33.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.272+0000 7f279edff700 1 -- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f277c05f320 msgr2=0x7f277c0617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.272+0000 7f279edff700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f277c05f320 0x7f277c0617e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f278c005750 tx=0x7f278c009040).stop 2023-11-06T20:45:33.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.272+0000 7f279edff700 1 -- 172.21.15.33:0/3474784805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27980c3410 msgr2=0x7f279814a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.272+0000 7f279edff700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27980c3410 0x7f279814a820 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f279000b6c0 tx=0x7f2790009780).stop 2023-11-06T20:45:33.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.272+0000 7f279edff700 1 -- 172.21.15.33:0/3474784805 shutdown_connections 2023-11-06T20:45:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.272+0000 7f279edff700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f277c05f320 0x7f277c0617e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.272+0000 7f279edff700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27980c3410 0x7f279814a820 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.272+0000 7f279edff700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2798153ed0 0x7f27980aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.272+0000 7f279edff700 1 --2- 172.21.15.33:0/3474784805 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27981542b0 0x7f27980ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.272+0000 7f279edff700 1 -- 172.21.15.33:0/3474784805 >> 172.21.15.33:0/3474784805 conn(0x7f279805e1b0 msgr2=0x7f27980c7610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.273+0000 7f279edff700 1 -- 172.21.15.33:0/3474784805 shutdown_connections 2023-11-06T20:45:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.273+0000 7f279edff700 1 -- 172.21.15.33:0/3474784805 wait complete. 2023-11-06T20:45:33.290 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-11-06T20:45:33.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-11-06T20:45:33.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.491+0000 7f211d7ff700 1 Processor -- start 2023-11-06T20:45:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.492+0000 7f211d7ff700 1 -- start start 2023-11-06T20:45:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.493+0000 7f211d7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f21180c8ee0 0x7f21180ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.493+0000 7f211d7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21180c9f80 con 0x7f21181542b0 2023-11-06T20:45:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.493+0000 7f211d7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21180c6000 con 0x7f2118153ed0 2023-11-06T20:45:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.493+0000 7f211d7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21180c9370 con 0x7f21180c8ee0 2023-11-06T20:45:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.493+0000 7f21167fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f21180c8ee0 0x7f21180ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.493+0000 7f21177fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f21181542b0 0x7f21180c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46684/0 (socket says 172.21.15.33:46684) 2023-11-06T20:45:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.493+0000 7f21177fe700 1 -- 172.21.15.33:0/1564358950 learned_addr learned my addr 172.21.15.33:0/1564358950 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:45:33.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.494+0000 7f21167fc700 1 -- 172.21.15.33:0/1564358950 >> v1:172.21.15.69:6789/0 conn(0x7f2118153ed0 legacy=0x7f2118158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.494+0000 7f21167fc700 1 -- 172.21.15.33:0/1564358950 >> v1:172.21.15.33:6789/0 conn(0x7f21181542b0 legacy=0x7f21180c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.494+0000 7f21167fc700 1 -- 172.21.15.33:0/1564358950 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21180c9650 con 0x7f21180c8ee0 2023-11-06T20:45:33.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.494+0000 7f2115ffb700 1 -- 172.21.15.33:0/1564358950 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377185404 0 0) 0x7f21180c9f80 con 0x7f21181542b0 2023-11-06T20:45:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.494+0000 7f21167fc700 1 --2- 172.21.15.33:0/1564358950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c8ee0 0x7f21180ca410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f210c009760 tx=0x7f210c009060).ready entity=mon.0 client_cookie=9e1dd811fa3418ef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.494+0000 7f2115ffb700 1 -- 172.21.15.33:0/1564358950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c00b580 con 0x7f21180c8ee0 2023-11-06T20:45:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.494+0000 7f2115ffb700 1 -- 172.21.15.33:0/1564358950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c00b6e0 con 0x7f21180c8ee0 2023-11-06T20:45:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.494+0000 7f2115ffb700 1 -- 172.21.15.33:0/1564358950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c00ba60 con 0x7f21180c8ee0 2023-11-06T20:45:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.495+0000 7f211d7ff700 1 -- 172.21.15.33:0/1564358950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c8ee0 msgr2=0x7f21180ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.495+0000 7f211d7ff700 1 --2- 172.21.15.33:0/1564358950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c8ee0 0x7f21180ca410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f210c009760 tx=0x7f210c009060).stop 2023-11-06T20:45:33.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.495+0000 7f211d7ff700 1 -- 172.21.15.33:0/1564358950 shutdown_connections 2023-11-06T20:45:33.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.495+0000 7f211d7ff700 1 --2- 172.21.15.33:0/1564358950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c8ee0 0x7f21180ca410 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.495+0000 7f211d7ff700 1 -- 172.21.15.33:0/1564358950 >> 172.21.15.33:0/1564358950 conn(0x7f211805e1b0 msgr2=0x7f211805e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:33.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.495+0000 7f211d7ff700 1 -- 172.21.15.33:0/1564358950 shutdown_connections 2023-11-06T20:45:33.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.495+0000 7f211d7ff700 1 -- 172.21.15.33:0/1564358950 wait complete. 2023-11-06T20:45:33.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.496+0000 7f211d7ff700 1 Processor -- start 2023-11-06T20:45:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.496+0000 7f211d7ff700 1 -- start start 2023-11-06T20:45:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.497+0000 7f211d7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2118153ed0 0x7f211814d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.497+0000 7f211d7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21181542b0 0x7f21180c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.497+0000 7f211d7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c41d0 0x7f21180ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.497+0000 7f211d7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21180aa780 con 0x7f21180c41d0 2023-11-06T20:45:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.497+0000 7f211d7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211814d740 con 0x7f21181542b0 2023-11-06T20:45:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.497+0000 7f211d7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2118154d90 con 0x7f2118153ed0 2023-11-06T20:45:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c41d0 0x7f21180ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c41d0 0x7f21180ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45362/0 (socket says 172.21.15.33:45362) 2023-11-06T20:45:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21177fe700 1 -- 172.21.15.33:0/3551423499 learned_addr learned my addr 172.21.15.33:0/3551423499 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21167fc700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21181542b0 0x7f21180c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21177fe700 1 -- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2118153ed0 msgr2=0x7f211814d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f2116ffd700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2118153ed0 0x7f211814d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21177fe700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2118153ed0 0x7f211814d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21177fe700 1 -- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21181542b0 msgr2=0x7f21180c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21177fe700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21181542b0 0x7f21180c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21177fe700 1 -- 172.21.15.33:0/3551423499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c0092a0 con 0x7f21180c41d0 2023-11-06T20:45:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21167fc700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21181542b0 0x7f21180c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f2116ffd700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2118153ed0 0x7f211814d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:45:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f21177fe700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c41d0 0x7f21180ade40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f211000c810 tx=0x7f21100014d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f20feffd700 1 -- 172.21.15.33:0/3551423499 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110005d30 con 0x7f21180c41d0 2023-11-06T20:45:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f20feffd700 1 -- 172.21.15.33:0/3551423499 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2110007110 con 0x7f21180c41d0 2023-11-06T20:45:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f211d7ff700 1 -- 172.21.15.33:0/3551423499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21180c4650 con 0x7f21180c41d0 2023-11-06T20:45:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.498+0000 7f20feffd700 1 -- 172.21.15.33:0/3551423499 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21100074d0 con 0x7f21180c41d0 2023-11-06T20:45:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.499+0000 7f211d7ff700 1 -- 172.21.15.33:0/3551423499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21181547d0 con 0x7f21180c41d0 2023-11-06T20:45:33.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.500+0000 7f211d7ff700 1 -- 172.21.15.33:0/3551423499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211806c6c0 con 0x7f21180c41d0 2023-11-06T20:45:33.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.500+0000 7f20feffd700 1 -- 172.21.15.33:0/3551423499 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f2110013410 con 0x7f21180c41d0 2023-11-06T20:45:33.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.502+0000 7f20feffd700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f210405f350 0x7f2104061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.502+0000 7f20feffd700 1 -- 172.21.15.33:0/3551423499 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f211007a580 con 0x7f21180c41d0 2023-11-06T20:45:33.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.502+0000 7f2116ffd700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f210405f350 0x7f2104061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.503+0000 7f2116ffd700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f210405f350 0x7f2104061810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f211814e2d0 tx=0x7f2108009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:33.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.506+0000 7f20feffd700 1 -- 172.21.15.33:0/3551423499 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f211004b4c0 con 0x7f21180c41d0 2023-11-06T20:45:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.710+0000 7f211d7ff700 1 -- 172.21.15.33:0/3551423499 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2118154ae0 con 0x7f210405f350 2023-11-06T20:45:33.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.716+0000 7f20feffd700 1 -- 172.21.15.33:0/3551423499 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f2118154ae0 con 0x7f210405f350 2023-11-06T20:45:33.716 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-11-06T20:45:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.717+0000 7f211d7ff700 1 -- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f210405f350 msgr2=0x7f2104061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f210405f350 0x7f2104061810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f211814e2d0 tx=0x7f2108009040).stop 2023-11-06T20:45:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 -- 172.21.15.33:0/3551423499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c41d0 msgr2=0x7f21180ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c41d0 0x7f21180ade40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f211000c810 tx=0x7f21100014d0).stop 2023-11-06T20:45:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 -- 172.21.15.33:0/3551423499 shutdown_connections 2023-11-06T20:45:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f210405f350 0x7f2104061810 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2118153ed0 0x7f211814d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21180c41d0 0x7f21180ade40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 --2- 172.21.15.33:0/3551423499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21181542b0 0x7f21180c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 -- 172.21.15.33:0/3551423499 >> 172.21.15.33:0/3551423499 conn(0x7f211805e1b0 msgr2=0x7f211814eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:33.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 -- 172.21.15.33:0/3551423499 shutdown_connections 2023-11-06T20:45:33.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.718+0000 7f211d7ff700 1 -- 172.21.15.33:0/3551423499 wait complete. 2023-11-06T20:45:33.735 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-11-06T20:45:33.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-11-06T20:45:33.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.938+0000 7fc7733ff700 1 Processor -- start 2023-11-06T20:45:33.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.938+0000 7fc7733ff700 1 -- start start 2023-11-06T20:45:33.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.940+0000 7fc7733ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154340 0x7fc76c0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.940+0000 7fc7733ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc76c158720 0x7fc76c063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.940+0000 7fc7733ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76c0c0e00 con 0x7fc76c154720 2023-11-06T20:45:33.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.940+0000 7fc7733ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76c158bf0 con 0x7fc76c158720 2023-11-06T20:45:33.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.940+0000 7fc7733ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c158ed0 con 0x7fc76c154340 2023-11-06T20:45:33.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.940+0000 7fc77119b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154340 0x7fc76c0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.940+0000 7fc77119b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154340 0x7fc76c0aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:35422/0 (socket says 172.21.15.33:35422) 2023-11-06T20:45:33.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.940+0000 7fc77119b700 1 -- 172.21.15.33:0/3614820644 learned_addr learned my addr 172.21.15.33:0/3614820644 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:33.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.940+0000 7fc77199c700 1 --2- 172.21.15.33:0/3614820644 >> v2:172.21.15.33:3300/0 conn(0x7fc76c158720 0x7fc76c063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.941+0000 7fc77199c700 1 -- 172.21.15.33:0/3614820644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154340 msgr2=0x7fc76c0aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:45:33.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.941+0000 7fc77199c700 1 --2- 172.21.15.33:0/3614820644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154340 0x7fc76c0aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.941+0000 7fc77199c700 1 -- 172.21.15.33:0/3614820644 >> v1:172.21.15.69:6789/0 conn(0x7fc76c154720 legacy=0x7fc76c158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.941+0000 7fc77199c700 1 -- 172.21.15.33:0/3614820644 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c0c5d80 con 0x7fc76c158720 2023-11-06T20:45:33.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.941+0000 7fc77199c700 1 --2- 172.21.15.33:0/3614820644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 0x7fc76c063200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fc764009e60 tx=0x7fc764009780).ready entity=mon.0 client_cookie=ba05030c5605131d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:33.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.941+0000 7fc763fff700 1 -- 172.21.15.33:0/3614820644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76400e070 con 0x7fc76c158720 2023-11-06T20:45:33.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.941+0000 7fc763fff700 1 -- 172.21.15.33:0/3614820644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76400bb60 con 0x7fc76c158720 2023-11-06T20:45:33.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.941+0000 7fc763fff700 1 -- 172.21.15.33:0/3614820644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764012410 con 0x7fc76c158720 2023-11-06T20:45:33.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.942+0000 7fc7733ff700 1 -- 172.21.15.33:0/3614820644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 msgr2=0x7fc76c063200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.942+0000 7fc7733ff700 1 --2- 172.21.15.33:0/3614820644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 0x7fc76c063200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fc764009e60 tx=0x7fc764009780).stop 2023-11-06T20:45:33.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.942+0000 7fc7733ff700 1 -- 172.21.15.33:0/3614820644 shutdown_connections 2023-11-06T20:45:33.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.942+0000 7fc7733ff700 1 --2- 172.21.15.33:0/3614820644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 0x7fc76c063200 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.942+0000 7fc7733ff700 1 --2- 172.21.15.33:0/3614820644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154340 0x7fc76c0aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.942+0000 7fc7733ff700 1 -- 172.21.15.33:0/3614820644 >> 172.21.15.33:0/3614820644 conn(0x7fc76c05ec80 msgr2=0x7fc76c0cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:33.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.942+0000 7fc7733ff700 1 -- 172.21.15.33:0/3614820644 shutdown_connections 2023-11-06T20:45:33.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.942+0000 7fc7733ff700 1 -- 172.21.15.33:0/3614820644 wait complete. 2023-11-06T20:45:33.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.943+0000 7fc7733ff700 1 Processor -- start 2023-11-06T20:45:33.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.944+0000 7fc7733ff700 1 -- start start 2023-11-06T20:45:33.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.944+0000 7fc7733ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c154340 0x7fc76c153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.944+0000 7fc7733ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154720 0x7fc76c149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.944+0000 7fc7733ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 0x7fc76c0c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.944+0000 7fc7733ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c150510 con 0x7fc76c158720 2023-11-06T20:45:33.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.944+0000 7fc7733ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0c90e0 con 0x7fc76c154340 2023-11-06T20:45:33.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.944+0000 7fc7733ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0c92f0 con 0x7fc76c154720 2023-11-06T20:45:33.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 0x7fc76c0c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 0x7fc76c0c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45382/0 (socket says 172.21.15.33:45382) 2023-11-06T20:45:33.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77199c700 1 -- 172.21.15.33:0/686365062 learned_addr learned my addr 172.21.15.33:0/686365062 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:45:33.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77119b700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c154340 0x7fc76c153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77099a700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154720 0x7fc76c149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77199c700 1 -- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154720 msgr2=0x7fc76c149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77199c700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154720 0x7fc76c149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77199c700 1 -- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c154340 msgr2=0x7fc76c153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:33.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77199c700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c154340 0x7fc76c153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:33.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77199c700 1 -- 172.21.15.33:0/686365062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764009a20 con 0x7fc76c158720 2023-11-06T20:45:33.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77119b700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c154340 0x7fc76c153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:45:33.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc77199c700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 0x7fc76c0c8bf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fc76c065a00 tx=0x7fc764009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:33.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.945+0000 7fc7617fa700 1 -- 172.21.15.33:0/686365062 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640116d0 con 0x7fc76c158720 2023-11-06T20:45:33.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.946+0000 7fc7617fa700 1 -- 172.21.15.33:0/686365062 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc764011830 con 0x7fc76c158720 2023-11-06T20:45:33.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.946+0000 7fc7733ff700 1 -- 172.21.15.33:0/686365062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c0c95d0 con 0x7fc76c158720 2023-11-06T20:45:33.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.946+0000 7fc7617fa700 1 -- 172.21.15.33:0/686365062 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764011bf0 con 0x7fc76c158720 2023-11-06T20:45:33.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.946+0000 7fc7733ff700 1 -- 172.21.15.33:0/686365062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c0c9b90 con 0x7fc76c158720 2023-11-06T20:45:33.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.947+0000 7fc7733ff700 1 -- 172.21.15.33:0/686365062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76c0c6060 con 0x7fc76c158720 2023-11-06T20:45:33.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.947+0000 7fc7617fa700 1 -- 172.21.15.33:0/686365062 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7fc764003040 con 0x7fc76c158720 2023-11-06T20:45:33.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.949+0000 7fc7617fa700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc75005f350 0x7fc750061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:45:33.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.949+0000 7fc7617fa700 1 -- 172.21.15.33:0/686365062 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fc76407c550 con 0x7fc76c158720 2023-11-06T20:45:33.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.949+0000 7fc77119b700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc75005f350 0x7fc750061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:45:33.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.950+0000 7fc77119b700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc75005f350 0x7fc750061810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc768006bc0 tx=0x7fc768008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:45:33.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:33.952+0000 7fc7617fa700 1 -- 172.21.15.33:0/686365062 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc76404d460 con 0x7fc76c158720 2023-11-06T20:45:34.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.162+0000 7fc7733ff700 1 -- 172.21.15.33:0/686365062 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc76c1500c0 con 0x7fc75005f350 2023-11-06T20:45:34.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.168+0000 7fc7617fa700 1 -- 172.21.15.33:0/686365062 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc76c1500c0 con 0x7fc75005f350 2023-11-06T20:45:34.167 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-11-06T20:45:34.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.169+0000 7fc7733ff700 1 -- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc75005f350 msgr2=0x7fc750061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:34.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.169+0000 7fc7733ff700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc75005f350 0x7fc750061810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc768006bc0 tx=0x7fc768008040).stop 2023-11-06T20:45:34.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.169+0000 7fc7733ff700 1 -- 172.21.15.33:0/686365062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 msgr2=0x7fc76c0c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:45:34.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.169+0000 7fc7733ff700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 0x7fc76c0c8bf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fc76c065a00 tx=0x7fc764009780).stop 2023-11-06T20:45:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.170+0000 7fc7733ff700 1 -- 172.21.15.33:0/686365062 shutdown_connections 2023-11-06T20:45:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.170+0000 7fc7733ff700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc75005f350 0x7fc750061810 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.170+0000 7fc7733ff700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c158720 0x7fc76c0c8bf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.170+0000 7fc7733ff700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c154340 0x7fc76c153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.170+0000 7fc7733ff700 1 --2- 172.21.15.33:0/686365062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc76c154720 0x7fc76c149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:45:34.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.170+0000 7fc7733ff700 1 -- 172.21.15.33:0/686365062 >> 172.21.15.33:0/686365062 conn(0x7fc76c05ec80 msgr2=0x7fc76c14cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:45:34.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.170+0000 7fc7733ff700 1 -- 172.21.15.33:0/686365062 shutdown_connections 2023-11-06T20:45:34.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:45:34.170+0000 7fc7733ff700 1 -- 172.21.15.33:0/686365062 wait complete. 2023-11-06T20:45:34.187 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-11-06T20:45:34.620 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:45:34.625 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:46:00.020 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:46:00.020+0000 7f803f6e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:46:00.020 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:46:00.020+0000 7f803f6e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:46:00.021 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:46:00.020+0000 7f803f6e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:46:00.021 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:46:00.020+0000 7f803f6e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:46:00.021 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:46:00.020+0000 7f803f6e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:46:04.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:46:04.755 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:46:34.871 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:46:34.877 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:47:05.144 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:47:05.150 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:47:35.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:47:35.263 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:48:05.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:48:05.406 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:48:35.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:48:35.521 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:49:05.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:49:05.644 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:49:35.751 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:49:35.757 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:50:05.873 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:50:05.879 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:50:36.003 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:50:36.009 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:51:06.116 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:51:06.122 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:51:36.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:51:36.235 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:52:06.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:52:06.499 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:52:36.606 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:52:36.612 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:53:06.719 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:53:06.725 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:53:36.841 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:53:36.847 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:54:06.954 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:54:06.960 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:54:37.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:54:37.074 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:55:07.181 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:55:07.187 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:55:37.314 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:55:37.320 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:56:07.433 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:56:07.439 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:56:20.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-11-06T20:56:20.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.401+0000 7fc4d7fff700 1 Processor -- start 2023-11-06T20:56:20.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.401+0000 7fc4d7fff700 1 -- start start 2023-11-06T20:56:20.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.402+0000 7fc4d7fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 0x7fc4d00aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.402+0000 7fc4d7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d00c02c0 con 0x7fc4d01542b0 2023-11-06T20:56:20.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.402+0000 7fc4d7fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d0158460 con 0x7fc4d00638d0 2023-11-06T20:56:20.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.402+0000 7fc4d7fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0158740 con 0x7fc4d0153ed0 2023-11-06T20:56:20.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d559a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc4d01542b0 0x7fc4d00631c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53312/0 (socket says 172.21.15.33:53312) 2023-11-06T20:56:20.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d559a700 1 -- 172.21.15.33:0/2785726797 learned_addr learned my addr 172.21.15.33:0/2785726797 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:20.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d5d9b700 1 --2- 172.21.15.33:0/2785726797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 0x7fc4d00aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d4d99700 1 -- 172.21.15.33:0/2785726797 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454591428 0 0) 0x7fc4d00c02c0 con 0x7fc4d01542b0 2023-11-06T20:56:20.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d4d99700 1 -- 172.21.15.33:0/2785726797 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4bc0033f0 con 0x7fc4d01542b0 2023-11-06T20:56:20.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d4d99700 1 -- 172.21.15.33:0/2785726797 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737360923 0 0) 0x7fc4d0158460 con 0x7fc4d00638d0 2023-11-06T20:56:20.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d4d99700 1 -- 172.21.15.33:0/2785726797 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4d00c02c0 con 0x7fc4d00638d0 2023-11-06T20:56:20.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d5d9b700 1 -- 172.21.15.33:0/2785726797 >> v1:172.21.15.69:6789/0 conn(0x7fc4d00638d0 legacy=0x7fc4d00c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d4d99700 1 -- 172.21.15.33:0/2785726797 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 943281070 0 0) 0x7fc4bc0033f0 con 0x7fc4d01542b0 2023-11-06T20:56:20.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d5d9b700 1 -- 172.21.15.33:0/2785726797 >> v1:172.21.15.33:6789/0 conn(0x7fc4d01542b0 legacy=0x7fc4d00631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.403+0000 7fc4d5d9b700 1 -- 172.21.15.33:0/2785726797 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d0158a20 con 0x7fc4d0153ed0 2023-11-06T20:56:20.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.404+0000 7fc4d5d9b700 1 --2- 172.21.15.33:0/2785726797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 0x7fc4d00aa2c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc4cc009790 tx=0x7fc4cc009060).ready entity=mon.2 client_cookie=f5acf42484f10e30 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:20.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.404+0000 7fc4d4d99700 1 -- 172.21.15.33:0/2785726797 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0001470 con 0x7fc4d0153ed0 2023-11-06T20:56:20.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.404+0000 7fc4d4d99700 1 -- 172.21.15.33:0/2785726797 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc011400 con 0x7fc4d0153ed0 2023-11-06T20:56:20.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.404+0000 7fc4d4d99700 1 -- 172.21.15.33:0/2785726797 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc011780 con 0x7fc4d0153ed0 2023-11-06T20:56:20.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.404+0000 7fc4d7fff700 1 -- 172.21.15.33:0/2785726797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 msgr2=0x7fc4d00aa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.404+0000 7fc4d7fff700 1 --2- 172.21.15.33:0/2785726797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 0x7fc4d00aa2c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc4cc009790 tx=0x7fc4cc009060).stop 2023-11-06T20:56:20.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.405+0000 7fc4d7fff700 1 -- 172.21.15.33:0/2785726797 shutdown_connections 2023-11-06T20:56:20.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.405+0000 7fc4d7fff700 1 --2- 172.21.15.33:0/2785726797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 0x7fc4d00aa2c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.405+0000 7fc4d7fff700 1 -- 172.21.15.33:0/2785726797 >> 172.21.15.33:0/2785726797 conn(0x7fc4d005ec80 msgr2=0x7fc4d00cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:20.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.405+0000 7fc4d7fff700 1 -- 172.21.15.33:0/2785726797 shutdown_connections 2023-11-06T20:56:20.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.405+0000 7fc4d7fff700 1 -- 172.21.15.33:0/2785726797 wait complete. 2023-11-06T20:56:20.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.406+0000 7fc4d7fff700 1 Processor -- start 2023-11-06T20:56:20.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.406+0000 7fc4d7fff700 1 -- start start 2023-11-06T20:56:20.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.407+0000 7fc4d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4d00638d0 0x7fc4d0153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.407+0000 7fc4d7fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 0x7fc4d0154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.407+0000 7fc4d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d01542b0 0x7fc4d0154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.407+0000 7fc4d7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0155110 con 0x7fc4d01542b0 2023-11-06T20:56:20.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.407+0000 7fc4d7fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d00c4090 con 0x7fc4d00638d0 2023-11-06T20:56:20.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.407+0000 7fc4d7fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d01552f0 con 0x7fc4d0153ed0 2023-11-06T20:56:20.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.407+0000 7fc4d659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d01542b0 0x7fc4d0154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d01542b0 0x7fc4d0154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45786/0 (socket says 172.21.15.33:45786) 2023-11-06T20:56:20.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d659c700 1 -- 172.21.15.33:0/3492103290 learned_addr learned my addr 172.21.15.33:0/3492103290 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:20.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d559a700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 0x7fc4d0154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d5d9b700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4d00638d0 0x7fc4d0153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d659c700 1 -- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 msgr2=0x7fc4d0154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d659c700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 0x7fc4d0154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d659c700 1 -- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4d00638d0 msgr2=0x7fc4d0153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d659c700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4d00638d0 0x7fc4d0153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d659c700 1 -- 172.21.15.33:0/3492103290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc0092a0 con 0x7fc4d01542b0 2023-11-06T20:56:20.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d659c700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d01542b0 0x7fc4d0154c20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fc4c800c7c0 tx=0x7fc4c8007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:20.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4c5ffb700 1 -- 172.21.15.33:0/3492103290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c800e4a0 con 0x7fc4d01542b0 2023-11-06T20:56:20.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4c5ffb700 1 -- 172.21.15.33:0/3492103290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c800e600 con 0x7fc4d01542b0 2023-11-06T20:56:20.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4d7fff700 1 -- 172.21.15.33:0/3492103290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d01555d0 con 0x7fc4d01542b0 2023-11-06T20:56:20.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.408+0000 7fc4c5ffb700 1 -- 172.21.15.33:0/3492103290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c800e980 con 0x7fc4d01542b0 2023-11-06T20:56:20.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.409+0000 7fc4d7fff700 1 -- 172.21.15.33:0/3492103290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d0155b90 con 0x7fc4d01542b0 2023-11-06T20:56:20.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.410+0000 7fc4d7fff700 1 -- 172.21.15.33:0/3492103290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d0157cf0 con 0x7fc4d01542b0 2023-11-06T20:56:20.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.410+0000 7fc4c5ffb700 1 -- 172.21.15.33:0/3492103290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fc4c800ed30 con 0x7fc4d01542b0 2023-11-06T20:56:20.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.411+0000 7fc4c5ffb700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4b405f2c0 0x7fc4b4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.411+0000 7fc4c5ffb700 1 -- 172.21.15.33:0/3492103290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fc4c8079d50 con 0x7fc4d01542b0 2023-11-06T20:56:20.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.412+0000 7fc4d5d9b700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4b405f2c0 0x7fc4b4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.412+0000 7fc4d5d9b700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4b405f2c0 0x7fc4b4061780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc4cc0096e0 tx=0x7fc4cc016000).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:20.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.416+0000 7fc4c5ffb700 1 -- 172.21.15.33:0/3492103290 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc4c8048af0 con 0x7fc4d01542b0 2023-11-06T20:56:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.625+0000 7fc4d7fff700 1 -- 172.21.15.33:0/3492103290 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4d00bfe70 con 0x7fc4b405f2c0 2023-11-06T20:56:20.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.631+0000 7fc4c5ffb700 1 -- 172.21.15.33:0/3492103290 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc4d00bfe70 con 0x7fc4b405f2c0 2023-11-06T20:56:20.631 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-11-06T20:56:20.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 -- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4b405f2c0 msgr2=0x7fc4b4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4b405f2c0 0x7fc4b4061780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc4cc0096e0 tx=0x7fc4cc016000).stop 2023-11-06T20:56:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 -- 172.21.15.33:0/3492103290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d01542b0 msgr2=0x7fc4d0154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d01542b0 0x7fc4d0154c20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fc4c800c7c0 tx=0x7fc4c8007f50).stop 2023-11-06T20:56:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 -- 172.21.15.33:0/3492103290 shutdown_connections 2023-11-06T20:56:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4b405f2c0 0x7fc4b4061780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d01542b0 0x7fc4d0154c20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4d00638d0 0x7fc4d0153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 --2- 172.21.15.33:0/3492103290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc4d0153ed0 0x7fc4d0154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 -- 172.21.15.33:0/3492103290 >> 172.21.15.33:0/3492103290 conn(0x7fc4d005ec80 msgr2=0x7fc4d00ca420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 -- 172.21.15.33:0/3492103290 shutdown_connections 2023-11-06T20:56:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.633+0000 7fc4d7fff700 1 -- 172.21.15.33:0/3492103290 wait complete. 2023-11-06T20:56:20.651 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-11-06T20:56:20.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-11-06T20:56:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.852+0000 7fcbbc3ff700 1 Processor -- start 2023-11-06T20:56:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.852+0000 7fcbbc3ff700 1 -- start start 2023-11-06T20:56:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbbc3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcbb4154720 0x7fcbb40aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbbc3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcbb4158720 0x7fcbb4063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbbc3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb40c0e00 con 0x7fcbb4154340 2023-11-06T20:56:20.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbbc3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb4158bf0 con 0x7fcbb4154720 2023-11-06T20:56:20.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbbc3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb4158ed0 con 0x7fcbb4158720 2023-11-06T20:56:20.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbb999a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcbb4154720 0x7fcbb40aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbb999a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcbb4154720 0x7fcbb40aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45802/0 (socket says 172.21.15.33:45802) 2023-11-06T20:56:20.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbb999a700 1 -- 172.21.15.33:0/1466040081 learned_addr learned my addr 172.21.15.33:0/1466040081 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbba99c700 1 --2- 172.21.15.33:0/1466040081 >> v2:172.21.15.69:3300/0 conn(0x7fcbb4158720 0x7fcbb4063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbb999a700 1 -- 172.21.15.33:0/1466040081 >> v2:172.21.15.69:3300/0 conn(0x7fcbb4158720 msgr2=0x7fcbb4063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbb999a700 1 --2- 172.21.15.33:0/1466040081 >> v2:172.21.15.69:3300/0 conn(0x7fcbb4158720 0x7fcbb4063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbb999a700 1 -- 172.21.15.33:0/1466040081 >> v1:172.21.15.69:6789/0 conn(0x7fcbb4154340 legacy=0x7fcbb4158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.854+0000 7fcbb999a700 1 -- 172.21.15.33:0/1466040081 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40c5d80 con 0x7fcbb4154720 2023-11-06T20:56:20.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.855+0000 7fcbb999a700 1 --2- 172.21.15.33:0/1466040081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 0x7fcbb40aae40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fcbb40adba0 tx=0x7fcba40090f0).ready entity=mon.0 client_cookie=9189b5440992927c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:20.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.855+0000 7fcbb9199700 1 -- 172.21.15.33:0/1466040081 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba400b520 con 0x7fcbb4154720 2023-11-06T20:56:20.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.855+0000 7fcbb9199700 1 -- 172.21.15.33:0/1466040081 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba400b680 con 0x7fcbb4154720 2023-11-06T20:56:20.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.855+0000 7fcbb9199700 1 -- 172.21.15.33:0/1466040081 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba400ba00 con 0x7fcbb4154720 2023-11-06T20:56:20.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.855+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/1466040081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 msgr2=0x7fcbb40aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.855+0000 7fcbbc3ff700 1 --2- 172.21.15.33:0/1466040081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 0x7fcbb40aae40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fcbb40adba0 tx=0x7fcba40090f0).stop 2023-11-06T20:56:20.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.856+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/1466040081 shutdown_connections 2023-11-06T20:56:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.856+0000 7fcbbc3ff700 1 --2- 172.21.15.33:0/1466040081 >> v2:172.21.15.69:3300/0 conn(0x7fcbb4158720 0x7fcbb4063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.856+0000 7fcbbc3ff700 1 --2- 172.21.15.33:0/1466040081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 0x7fcbb40aae40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.856+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/1466040081 >> 172.21.15.33:0/1466040081 conn(0x7fcbb405ec80 msgr2=0x7fcbb40cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.856+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/1466040081 shutdown_connections 2023-11-06T20:56:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.856+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/1466040081 wait complete. 2023-11-06T20:56:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.857+0000 7fcbbc3ff700 1 Processor -- start 2023-11-06T20:56:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.857+0000 7fcbbc3ff700 1 -- start start 2023-11-06T20:56:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.858+0000 7fcbbc3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbb4154340 0x7fcbb4155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.858+0000 7fcbbc3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 0x7fcbb40c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.858+0000 7fcbbc3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbb4158720 0x7fcbb414eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.858+0000 7fcbbc3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40c8c30 con 0x7fcbb4154720 2023-11-06T20:56:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.858+0000 7fcbbc3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40c4aa0 con 0x7fcbb4154340 2023-11-06T20:56:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.858+0000 7fcbbc3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40c8e80 con 0x7fcbb4158720 2023-11-06T20:56:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbb999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 0x7fcbb40c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbb999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 0x7fcbb40c45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45810/0 (socket says 172.21.15.33:45810) 2023-11-06T20:56:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbb999a700 1 -- 172.21.15.33:0/2526220533 learned_addr learned my addr 172.21.15.33:0/2526220533 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbba19b700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbb4154340 0x7fcbb4155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbba99c700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbb4158720 0x7fcbb414eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbb999a700 1 -- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbb4158720 msgr2=0x7fcbb414eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbb999a700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbb4158720 0x7fcbb414eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbb999a700 1 -- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbb4154340 msgr2=0x7fcbb4155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbb999a700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbb4154340 0x7fcbb4155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbb999a700 1 -- 172.21.15.33:0/2526220533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba4009370 con 0x7fcbb4154720 2023-11-06T20:56:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbba19b700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbb4154340 0x7fcbb4155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.859+0000 7fcbb999a700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 0x7fcbb40c45b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fcba4011000 tx=0x7fcba4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.860+0000 7fcbaa7fc700 1 -- 172.21.15.33:0/2526220533 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba400b7f0 con 0x7fcbb4154720 2023-11-06T20:56:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.860+0000 7fcbaa7fc700 1 -- 172.21.15.33:0/2526220533 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba400b950 con 0x7fcbb4154720 2023-11-06T20:56:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.860+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/2526220533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb40c9160 con 0x7fcbb4154720 2023-11-06T20:56:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.860+0000 7fcbaa7fc700 1 -- 172.21.15.33:0/2526220533 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba401c670 con 0x7fcbb4154720 2023-11-06T20:56:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.860+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/2526220533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb40c9720 con 0x7fcbb4154720 2023-11-06T20:56:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.861+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/2526220533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb40c9940 con 0x7fcbb4154720 2023-11-06T20:56:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.861+0000 7fcbaa7fc700 1 -- 172.21.15.33:0/2526220533 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fcba401ca20 con 0x7fcbb4154720 2023-11-06T20:56:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.862+0000 7fcbaa7fc700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb9805f2c0 0x7fcb98061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.862+0000 7fcbaa7fc700 1 -- 172.21.15.33:0/2526220533 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fcba407c160 con 0x7fcbb4154720 2023-11-06T20:56:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.863+0000 7fcbba19b700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb9805f2c0 0x7fcb98061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.863+0000 7fcbba19b700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb9805f2c0 0x7fcb98061780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fcbb40a9ff0 tx=0x7fcbb0009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:20.866+0000 7fcbaa7fc700 1 -- 172.21.15.33:0/2526220533 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcba404d090 con 0x7fcbb4154720 2023-11-06T20:56:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.074+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/2526220533 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbb40c6060 con 0x7fcb9805f2c0 2023-11-06T20:56:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.079+0000 7fcbaa7fc700 1 -- 172.21.15.33:0/2526220533 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fcbb40c6060 con 0x7fcb9805f2c0 2023-11-06T20:56:21.080 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-11-06T20:56:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb9805f2c0 msgr2=0x7fcb98061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb9805f2c0 0x7fcb98061780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fcbb40a9ff0 tx=0x7fcbb0009040).stop 2023-11-06T20:56:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/2526220533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 msgr2=0x7fcbb40c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 0x7fcbb40c45b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fcba4011000 tx=0x7fcba4017000).stop 2023-11-06T20:56:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/2526220533 shutdown_connections 2023-11-06T20:56:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb9805f2c0 0x7fcb98061780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbb4158720 0x7fcbb414eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbb4154340 0x7fcbb4155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 --2- 172.21.15.33:0/2526220533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb4154720 0x7fcbb40c45b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/2526220533 >> 172.21.15.33:0/2526220533 conn(0x7fcbb405ec80 msgr2=0x7fcbb4153410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/2526220533 shutdown_connections 2023-11-06T20:56:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.081+0000 7fcbbc3ff700 1 -- 172.21.15.33:0/2526220533 wait complete. 2023-11-06T20:56:21.099 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-11-06T20:56:21.099 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-11-06T20:56:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.301+0000 7f4867fff700 1 Processor -- start 2023-11-06T20:56:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.301+0000 7f4867fff700 1 -- start start 2023-11-06T20:56:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f4867fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4860068c20 0x7f4860069000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f4867fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4860069ac0 0x7f48600bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f4867fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48600634c0 con 0x7f486005e3c0 2023-11-06T20:56:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f4867fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48600c4600 con 0x7f4860069ac0 2023-11-06T20:56:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f4867fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48600628b0 con 0x7f4860068c20 2023-11-06T20:56:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f486559a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f486005e3c0 0x7f4860063950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53326/0 (socket says 172.21.15.33:53326) 2023-11-06T20:56:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f486559a700 1 -- 172.21.15.33:0/3490733768 learned_addr learned my addr 172.21.15.33:0/3490733768 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f486659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4860069ac0 0x7f48600bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f4865d9b700 1 --2- 172.21.15.33:0/3490733768 >> v2:172.21.15.69:3300/0 conn(0x7f4860068c20 0x7f4860069000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f486659c700 1 -- 172.21.15.33:0/3490733768 >> v2:172.21.15.69:3300/0 conn(0x7f4860068c20 msgr2=0x7f4860069000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.303+0000 7f486659c700 1 --2- 172.21.15.33:0/3490733768 >> v2:172.21.15.69:3300/0 conn(0x7f4860068c20 0x7f4860069000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.304+0000 7f486659c700 1 -- 172.21.15.33:0/3490733768 >> v1:172.21.15.33:6789/0 conn(0x7f486005e3c0 legacy=0x7f4860063950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.304+0000 7f486659c700 1 -- 172.21.15.33:0/3490733768 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4860062b90 con 0x7f4860069ac0 2023-11-06T20:56:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.304+0000 7f486659c700 1 --2- 172.21.15.33:0/3490733768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 0x7f48600bc220 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f486006c180 tx=0x7f4858009780).ready entity=mon.0 client_cookie=c6dde6c1093cd344 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:21.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.304+0000 7f4864d99700 1 -- 172.21.15.33:0/3490733768 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485800e070 con 0x7f4860069ac0 2023-11-06T20:56:21.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.304+0000 7f4864d99700 1 -- 172.21.15.33:0/3490733768 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485800bc30 con 0x7f4860069ac0 2023-11-06T20:56:21.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.304+0000 7f4864d99700 1 -- 172.21.15.33:0/3490733768 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858012410 con 0x7f4860069ac0 2023-11-06T20:56:21.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.304+0000 7f4867fff700 1 -- 172.21.15.33:0/3490733768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 msgr2=0x7f48600bc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.304+0000 7f4867fff700 1 --2- 172.21.15.33:0/3490733768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 0x7f48600bc220 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f486006c180 tx=0x7f4858009780).stop 2023-11-06T20:56:21.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.305+0000 7f4867fff700 1 -- 172.21.15.33:0/3490733768 shutdown_connections 2023-11-06T20:56:21.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.305+0000 7f4867fff700 1 --2- 172.21.15.33:0/3490733768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 0x7f48600bc220 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.305+0000 7f4867fff700 1 --2- 172.21.15.33:0/3490733768 >> v2:172.21.15.69:3300/0 conn(0x7f4860068c20 0x7f4860069000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.305+0000 7f4867fff700 1 -- 172.21.15.33:0/3490733768 >> 172.21.15.33:0/3490733768 conn(0x7f48600c7d10 msgr2=0x7f48600c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:21.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.305+0000 7f4867fff700 1 -- 172.21.15.33:0/3490733768 shutdown_connections 2023-11-06T20:56:21.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.305+0000 7f4867fff700 1 -- 172.21.15.33:0/3490733768 wait complete. 2023-11-06T20:56:21.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.306+0000 7f4867fff700 1 Processor -- start 2023-11-06T20:56:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.306+0000 7f4867fff700 1 -- start start 2023-11-06T20:56:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.307+0000 7f4867fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f486005e3c0 0x7f48600a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.307+0000 7f4867fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4860068c20 0x7f4860167ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.307+0000 7f4867fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 0x7f4860150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.307+0000 7f4867fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48600bf850 con 0x7f4860069ac0 2023-11-06T20:56:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.307+0000 7f4867fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48600bfaa0 con 0x7f4860068c20 2023-11-06T20:56:21.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.307+0000 7f4867fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48600bfd80 con 0x7f486005e3c0 2023-11-06T20:56:21.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f486659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 0x7f4860150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f486659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 0x7f4860150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45832/0 (socket says 172.21.15.33:45832) 2023-11-06T20:56:21.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f486659c700 1 -- 172.21.15.33:0/2051945816 learned_addr learned my addr 172.21.15.33:0/2051945816 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:21.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f486559a700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4860068c20 0x7f4860167ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f4865d9b700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f486005e3c0 0x7f48600a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f486659c700 1 -- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f486005e3c0 msgr2=0x7f48600a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f486659c700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f486005e3c0 0x7f48600a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f486659c700 1 -- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4860068c20 msgr2=0x7f4860167ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f486659c700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4860068c20 0x7f4860167ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f486659c700 1 -- 172.21.15.33:0/2051945816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4858009a20 con 0x7f4860069ac0 2023-11-06T20:56:21.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f486659c700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 0x7f4860150a10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f48580097c0 tx=0x7f4860063430).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:21.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.308+0000 7f4855ffb700 1 -- 172.21.15.33:0/2051945816 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485801d700 con 0x7f4860069ac0 2023-11-06T20:56:21.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.309+0000 7f4855ffb700 1 -- 172.21.15.33:0/2051945816 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485801d860 con 0x7f4860069ac0 2023-11-06T20:56:21.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.309+0000 7f4867fff700 1 -- 172.21.15.33:0/2051945816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48600c0060 con 0x7f4860069ac0 2023-11-06T20:56:21.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.309+0000 7f4855ffb700 1 -- 172.21.15.33:0/2051945816 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485801dc20 con 0x7f4860069ac0 2023-11-06T20:56:21.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.309+0000 7f4867fff700 1 -- 172.21.15.33:0/2051945816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48601683e0 con 0x7f4860069ac0 2023-11-06T20:56:21.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.310+0000 7f4867fff700 1 -- 172.21.15.33:0/2051945816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48600c0560 con 0x7f4860069ac0 2023-11-06T20:56:21.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.311+0000 7f4855ffb700 1 -- 172.21.15.33:0/2051945816 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f4858003040 con 0x7f4860069ac0 2023-11-06T20:56:21.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.311+0000 7f4855ffb700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f484405f2c0 0x7f4844061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.311+0000 7f4855ffb700 1 -- 172.21.15.33:0/2051945816 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f485807c440 con 0x7f4860069ac0 2023-11-06T20:56:21.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.312+0000 7f4865d9b700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f484405f2c0 0x7f4844061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.313+0000 7f4865d9b700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f484405f2c0 0x7f4844061780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f485c00bbc0 tx=0x7f485c008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:21.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.315+0000 7f4855ffb700 1 -- 172.21.15.33:0/2051945816 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f485804d430 con 0x7f4860069ac0 2023-11-06T20:56:21.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.525+0000 7f4867fff700 1 -- 172.21.15.33:0/2051945816 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4860062e70 con 0x7f484405f2c0 2023-11-06T20:56:21.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.532+0000 7f4855ffb700 1 -- 172.21.15.33:0/2051945816 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4860062e70 con 0x7f484405f2c0 2023-11-06T20:56:21.533 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-11-06T20:56:21.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 -- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f484405f2c0 msgr2=0x7f4844061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f484405f2c0 0x7f4844061780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f485c00bbc0 tx=0x7f485c008040).stop 2023-11-06T20:56:21.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 -- 172.21.15.33:0/2051945816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 msgr2=0x7f4860150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 0x7f4860150a10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f48580097c0 tx=0x7f4860063430).stop 2023-11-06T20:56:21.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 -- 172.21.15.33:0/2051945816 shutdown_connections 2023-11-06T20:56:21.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f484405f2c0 0x7f4844061780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f486005e3c0 0x7f48600a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4860069ac0 0x7f4860150a10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 --2- 172.21.15.33:0/2051945816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4860068c20 0x7f4860167ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 -- 172.21.15.33:0/2051945816 >> 172.21.15.33:0/2051945816 conn(0x7f48600c7d10 msgr2=0x7f48600bd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.534+0000 7f4867fff700 1 -- 172.21.15.33:0/2051945816 shutdown_connections 2023-11-06T20:56:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.535+0000 7f4867fff700 1 -- 172.21.15.33:0/2051945816 wait complete. 2023-11-06T20:56:21.552 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-11-06T20:56:21.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-11-06T20:56:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.753+0000 7f53dffff700 1 Processor -- start 2023-11-06T20:56:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.753+0000 7f53dffff700 1 -- start start 2023-11-06T20:56:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.754+0000 7f53dffff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f53d8154340 0x7f53d80aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.754+0000 7f53dffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53d8154720 0x7f53d8158010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.754+0000 7f53dffff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d8158990 con 0x7f53d8158500 2023-11-06T20:56:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.754+0000 7f53dffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d80c0e00 con 0x7f53d8154720 2023-11-06T20:56:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.754+0000 7f53dffff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d8158bf0 con 0x7f53d8154340 2023-11-06T20:56:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.755+0000 7f53dd59a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53d8154720 0x7f53d8158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.755+0000 7f53dd59a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53d8154720 0x7f53d8158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45846/0 (socket says 172.21.15.33:45846) 2023-11-06T20:56:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.755+0000 7f53dd59a700 1 -- 172.21.15.33:0/1062522635 learned_addr learned my addr 172.21.15.33:0/1062522635 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.755+0000 7f53ddd9b700 1 --2- 172.21.15.33:0/1062522635 >> v2:172.21.15.69:3300/0 conn(0x7f53d8154340 0x7f53d80aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.755+0000 7f53dd59a700 1 -- 172.21.15.33:0/1062522635 >> v2:172.21.15.69:3300/0 conn(0x7f53d8154340 msgr2=0x7f53d80aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.755+0000 7f53dd59a700 1 --2- 172.21.15.33:0/1062522635 >> v2:172.21.15.69:3300/0 conn(0x7f53d8154340 0x7f53d80aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.755+0000 7f53dd59a700 1 -- 172.21.15.33:0/1062522635 >> v1:172.21.15.69:6789/0 conn(0x7f53d8158500 legacy=0x7f53d8063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.755+0000 7f53dd59a700 1 -- 172.21.15.33:0/1062522635 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d8158ed0 con 0x7f53d8154720 2023-11-06T20:56:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.755+0000 7f53dd59a700 1 --2- 172.21.15.33:0/1062522635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 0x7f53d8158010 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f53d805f090 tx=0x7f53c80090f0).ready entity=mon.0 client_cookie=a2427154e735b38d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dcd99700 1 -- 172.21.15.33:0/1062522635 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800b520 con 0x7f53d8154720 2023-11-06T20:56:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dcd99700 1 -- 172.21.15.33:0/1062522635 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c800b680 con 0x7f53d8154720 2023-11-06T20:56:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dcd99700 1 -- 172.21.15.33:0/1062522635 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800ba00 con 0x7f53d8154720 2023-11-06T20:56:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dffff700 1 -- 172.21.15.33:0/1062522635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 msgr2=0x7f53d8158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dffff700 1 --2- 172.21.15.33:0/1062522635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 0x7f53d8158010 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f53d805f090 tx=0x7f53c80090f0).stop 2023-11-06T20:56:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dffff700 1 -- 172.21.15.33:0/1062522635 shutdown_connections 2023-11-06T20:56:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dffff700 1 --2- 172.21.15.33:0/1062522635 >> v2:172.21.15.69:3300/0 conn(0x7f53d8154340 0x7f53d80aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dffff700 1 --2- 172.21.15.33:0/1062522635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 0x7f53d8158010 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dffff700 1 -- 172.21.15.33:0/1062522635 >> 172.21.15.33:0/1062522635 conn(0x7f53d805ec80 msgr2=0x7f53d80cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dffff700 1 -- 172.21.15.33:0/1062522635 shutdown_connections 2023-11-06T20:56:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.756+0000 7f53dffff700 1 -- 172.21.15.33:0/1062522635 wait complete. 2023-11-06T20:56:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.758+0000 7f53dffff700 1 Processor -- start 2023-11-06T20:56:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.758+0000 7f53dffff700 1 -- start start 2023-11-06T20:56:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53dffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53d8154340 0x7f53d814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53dffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 0x7f53d8156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53dffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53d8158500 0x7f53d8156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53dffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8157410 con 0x7f53d8154720 2023-11-06T20:56:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53dffff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d80aed10 con 0x7f53d8154340 2023-11-06T20:56:21.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53dffff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d80c9a40 con 0x7f53d8158500 2023-11-06T20:56:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53dd59a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 0x7f53d8156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53dd59a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 0x7f53d8156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45852/0 (socket says 172.21.15.33:45852) 2023-11-06T20:56:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53dd59a700 1 -- 172.21.15.33:0/3455679691 learned_addr learned my addr 172.21.15.33:0/3455679691 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53ddd9b700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53d8154340 0x7f53d814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.759+0000 7f53de59c700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53d8158500 0x7f53d8156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53dd59a700 1 -- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53d8158500 msgr2=0x7f53d8156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53dd59a700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53d8158500 0x7f53d8156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53dd59a700 1 -- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53d8154340 msgr2=0x7f53d814cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53dd59a700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53d8154340 0x7f53d814cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53dd59a700 1 -- 172.21.15.33:0/3455679691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c8009370 con 0x7f53d8154720 2023-11-06T20:56:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53de59c700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53d8158500 0x7f53d8156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53dd59a700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 0x7f53d8156a30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f53c8009130 tx=0x7f53c8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53cdffb700 1 -- 172.21.15.33:0/3455679691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800be90 con 0x7f53d8154720 2023-11-06T20:56:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53cdffb700 1 -- 172.21.15.33:0/3455679691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c800b7f0 con 0x7f53d8154720 2023-11-06T20:56:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53cdffb700 1 -- 172.21.15.33:0/3455679691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c801c670 con 0x7f53d8154720 2023-11-06T20:56:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53dffff700 1 -- 172.21.15.33:0/3455679691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d80c9d20 con 0x7f53d8154720 2023-11-06T20:56:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.760+0000 7f53dffff700 1 -- 172.21.15.33:0/3455679691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d80ca2e0 con 0x7f53d8154720 2023-11-06T20:56:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.761+0000 7f53dffff700 1 -- 172.21.15.33:0/3455679691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d80aeea0 con 0x7f53d8154720 2023-11-06T20:56:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.762+0000 7f53cdffb700 1 -- 172.21.15.33:0/3455679691 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f53c801c9e0 con 0x7f53d8154720 2023-11-06T20:56:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.763+0000 7f53cdffb700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53bc05f2c0 0x7f53bc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.763+0000 7f53cdffb700 1 -- 172.21.15.33:0/3455679691 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f53c807c2f0 con 0x7f53d8154720 2023-11-06T20:56:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.764+0000 7f53ddd9b700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53bc05f2c0 0x7f53bc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.764+0000 7f53ddd9b700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53bc05f2c0 0x7f53bc061780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f53d4009c30 tx=0x7f53d4009060).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:21.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.767+0000 7f53cdffb700 1 -- 172.21.15.33:0/3455679691 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f53d80aeea0 con 0x7f53d8154720 2023-11-06T20:56:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.977+0000 7f53dffff700 1 -- 172.21.15.33:0/3455679691 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f53d80aeea0 con 0x7f53bc05f2c0 2023-11-06T20:56:21.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.983+0000 7f53cdffb700 1 -- 172.21.15.33:0/3455679691 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f53d80aeea0 con 0x7f53bc05f2c0 2023-11-06T20:56:21.984 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-11-06T20:56:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 -- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53bc05f2c0 msgr2=0x7f53bc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53bc05f2c0 0x7f53bc061780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f53d4009c30 tx=0x7f53d4009060).stop 2023-11-06T20:56:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 -- 172.21.15.33:0/3455679691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 msgr2=0x7f53d8156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:21.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 0x7f53d8156a30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f53c8009130 tx=0x7f53c8017000).stop 2023-11-06T20:56:21.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 -- 172.21.15.33:0/3455679691 shutdown_connections 2023-11-06T20:56:21.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53bc05f2c0 0x7f53bc061780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53d8158500 0x7f53d8156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53d8154340 0x7f53d814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 --2- 172.21.15.33:0/3455679691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53d8154720 0x7f53d8156a30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:21.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 -- 172.21.15.33:0/3455679691 >> 172.21.15.33:0/3455679691 conn(0x7f53d805ec80 msgr2=0x7f53d814dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:21.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 -- 172.21.15.33:0/3455679691 shutdown_connections 2023-11-06T20:56:21.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:21.985+0000 7f53dffff700 1 -- 172.21.15.33:0/3455679691 wait complete. 2023-11-06T20:56:22.003 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-11-06T20:56:22.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-11-06T20:56:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.205+0000 7fd0b8dff700 1 Processor -- start 2023-11-06T20:56:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.205+0000 7fd0b8dff700 1 -- start start 2023-11-06T20:56:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.206+0000 7fd0b8dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0b4154340 0x7fd0b40aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.206+0000 7fd0b8dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0b4154720 0x7fd0b4158010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.206+0000 7fd0b8dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b41589b0 con 0x7fd0b4158500 2023-11-06T20:56:22.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.206+0000 7fd0b8dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b40c0e00 con 0x7fd0b4154720 2023-11-06T20:56:22.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.206+0000 7fd0b8dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4158bf0 con 0x7fd0b4154340 2023-11-06T20:56:22.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.207+0000 7fd0b1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0b4154720 0x7fd0b4158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.207+0000 7fd0b1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0b4154720 0x7fd0b4158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45856/0 (socket says 172.21.15.33:45856) 2023-11-06T20:56:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.207+0000 7fd0b1d9b700 1 -- 172.21.15.33:0/2999298959 learned_addr learned my addr 172.21.15.33:0/2999298959 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.207+0000 7fd0b259c700 1 --2- 172.21.15.33:0/2999298959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0b4154340 0x7fd0b40aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.207+0000 7fd0b1d9b700 1 -- 172.21.15.33:0/2999298959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0b4154340 msgr2=0x7fd0b40aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.207+0000 7fd0b1d9b700 1 --2- 172.21.15.33:0/2999298959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0b4154340 0x7fd0b40aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.207+0000 7fd0b1d9b700 1 -- 172.21.15.33:0/2999298959 >> v1:172.21.15.69:6789/0 conn(0x7fd0b4158500 legacy=0x7fd0b4063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.207+0000 7fd0b1d9b700 1 -- 172.21.15.33:0/2999298959 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b4158ed0 con 0x7fd0b4154720 2023-11-06T20:56:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.207+0000 7fd0b1d9b700 1 --2- 172.21.15.33:0/2999298959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4154720 0x7fd0b4158010 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fd0a8006fd0 tx=0x7fd0a80090f0).ready entity=mon.0 client_cookie=dd4a54a380cf953d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b159a700 1 -- 172.21.15.33:0/2999298959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800b580 con 0x7fd0b4154720 2023-11-06T20:56:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b159a700 1 -- 172.21.15.33:0/2999298959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a800b6e0 con 0x7fd0b4154720 2023-11-06T20:56:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b159a700 1 -- 172.21.15.33:0/2999298959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800ba60 con 0x7fd0b4154720 2023-11-06T20:56:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b8dff700 1 -- 172.21.15.33:0/2999298959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4154720 msgr2=0x7fd0b4158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b8dff700 1 --2- 172.21.15.33:0/2999298959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4154720 0x7fd0b4158010 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fd0a8006fd0 tx=0x7fd0a80090f0).stop 2023-11-06T20:56:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b8dff700 1 -- 172.21.15.33:0/2999298959 shutdown_connections 2023-11-06T20:56:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b8dff700 1 --2- 172.21.15.33:0/2999298959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0b4154340 0x7fd0b40aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b8dff700 1 --2- 172.21.15.33:0/2999298959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4154720 0x7fd0b4158010 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b8dff700 1 -- 172.21.15.33:0/2999298959 >> 172.21.15.33:0/2999298959 conn(0x7fd0b405ec80 msgr2=0x7fd0b40cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b8dff700 1 -- 172.21.15.33:0/2999298959 shutdown_connections 2023-11-06T20:56:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.208+0000 7fd0b8dff700 1 -- 172.21.15.33:0/2999298959 wait complete. 2023-11-06T20:56:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.210+0000 7fd0b8dff700 1 Processor -- start 2023-11-06T20:56:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.210+0000 7fd0b8dff700 1 -- start start 2023-11-06T20:56:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b8dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b4154340 0x7fd0b4156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b8dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0b4154720 0x7fd0b4154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b8dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4158500 0x7fd0b40c3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b8dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4151ab0 con 0x7fd0b4158500 2023-11-06T20:56:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b8dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b414dee0 con 0x7fd0b4154340 2023-11-06T20:56:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b8dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40c3be0 con 0x7fd0b4154720 2023-11-06T20:56:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4158500 0x7fd0b40c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4158500 0x7fd0b40c3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45870/0 (socket says 172.21.15.33:45870) 2023-11-06T20:56:22.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b2d9d700 1 -- 172.21.15.33:0/4263153401 learned_addr learned my addr 172.21.15.33:0/4263153401 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:22.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b1d9b700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0b4154720 0x7fd0b4154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.211+0000 7fd0b259c700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b4154340 0x7fd0b4156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0b2d9d700 1 -- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0b4154720 msgr2=0x7fd0b4154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0b2d9d700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0b4154720 0x7fd0b4154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0b2d9d700 1 -- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b4154340 msgr2=0x7fd0b4156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0b2d9d700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b4154340 0x7fd0b4156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0b2d9d700 1 -- 172.21.15.33:0/4263153401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a8009370 con 0x7fd0b4158500 2023-11-06T20:56:22.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0b259c700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b4154340 0x7fd0b4156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:22.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0b2d9d700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4158500 0x7fd0b40c3090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fd0ac002230 tx=0x7fd0ac0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0a27fc700 1 -- 172.21.15.33:0/4263153401 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac010510 con 0x7fd0b4158500 2023-11-06T20:56:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0a27fc700 1 -- 172.21.15.33:0/4263153401 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac010670 con 0x7fd0b4158500 2023-11-06T20:56:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0b8dff700 1 -- 172.21.15.33:0/4263153401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b40c3ec0 con 0x7fd0b4158500 2023-11-06T20:56:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0a27fc700 1 -- 172.21.15.33:0/4263153401 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0109f0 con 0x7fd0b4158500 2023-11-06T20:56:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.212+0000 7fd0b8dff700 1 -- 172.21.15.33:0/4263153401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b4151360 con 0x7fd0b4158500 2023-11-06T20:56:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.213+0000 7fd0b8dff700 1 -- 172.21.15.33:0/4263153401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b41515b0 con 0x7fd0b4158500 2023-11-06T20:56:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.214+0000 7fd0a27fc700 1 -- 172.21.15.33:0/4263153401 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fd0ac01b400 con 0x7fd0b4158500 2023-11-06T20:56:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.215+0000 7fd0a27fc700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd09405f2c0 0x7fd094061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.215+0000 7fd0a27fc700 1 -- 172.21.15.33:0/4263153401 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fd0ac07a1c0 con 0x7fd0b4158500 2023-11-06T20:56:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.215+0000 7fd0b259c700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd09405f2c0 0x7fd094061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.216+0000 7fd0b259c700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd09405f2c0 0x7fd094061780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd0a4009b60 tx=0x7fd0a4008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.219+0000 7fd0a27fc700 1 -- 172.21.15.33:0/4263153401 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd0ac04b150 con 0x7fd0b4158500 2023-11-06T20:56:22.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.429+0000 7fd0b8dff700 1 -- 172.21.15.33:0/4263153401 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0b4151bf0 con 0x7fd09405f2c0 2023-11-06T20:56:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.434+0000 7fd0a27fc700 1 -- 172.21.15.33:0/4263153401 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd0b4151bf0 con 0x7fd09405f2c0 2023-11-06T20:56:22.434 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-11-06T20:56:22.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 -- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd09405f2c0 msgr2=0x7fd094061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd09405f2c0 0x7fd094061780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd0a4009b60 tx=0x7fd0a4008040).stop 2023-11-06T20:56:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 -- 172.21.15.33:0/4263153401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4158500 msgr2=0x7fd0b40c3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4158500 0x7fd0b40c3090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fd0ac002230 tx=0x7fd0ac0015e0).stop 2023-11-06T20:56:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 -- 172.21.15.33:0/4263153401 shutdown_connections 2023-11-06T20:56:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd09405f2c0 0x7fd094061780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4158500 0x7fd0b40c3090 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b4154340 0x7fd0b4156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 --2- 172.21.15.33:0/4263153401 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0b4154720 0x7fd0b4154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 -- 172.21.15.33:0/4263153401 >> 172.21.15.33:0/4263153401 conn(0x7fd0b405ec80 msgr2=0x7fd0b40ad3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:22.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 -- 172.21.15.33:0/4263153401 shutdown_connections 2023-11-06T20:56:22.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.436+0000 7fd0b8dff700 1 -- 172.21.15.33:0/4263153401 wait complete. 2023-11-06T20:56:22.453 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-11-06T20:56:22.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-11-06T20:56:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.655+0000 7fead53ff700 1 Processor -- start 2023-11-06T20:56:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.655+0000 7fead53ff700 1 -- start start 2023-11-06T20:56:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.656+0000 7fead53ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fead01510e0 0x7fead00a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.656+0000 7fead53ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fead01514c0 0x7fead00bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.656+0000 7fead53ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead0154c60 con 0x7fead00bbe70 2023-11-06T20:56:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.656+0000 7fead53ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead0068f50 con 0x7fead01514c0 2023-11-06T20:56:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.656+0000 7fead53ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead00bc2c0 con 0x7fead01510e0 2023-11-06T20:56:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.656+0000 7feace7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fead01514c0 0x7fead00bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feace7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fead01514c0 0x7fead00bb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45872/0 (socket says 172.21.15.33:45872) 2023-11-06T20:56:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feace7fc700 1 -- 172.21.15.33:0/3473320676 learned_addr learned my addr 172.21.15.33:0/3473320676 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feaceffd700 1 --2- 172.21.15.33:0/3473320676 >> v2:172.21.15.69:3300/0 conn(0x7fead01510e0 0x7fead00a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feace7fc700 1 -- 172.21.15.33:0/3473320676 >> v2:172.21.15.69:3300/0 conn(0x7fead01510e0 msgr2=0x7fead00a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feace7fc700 1 --2- 172.21.15.33:0/3473320676 >> v2:172.21.15.69:3300/0 conn(0x7fead01510e0 0x7fead00a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feace7fc700 1 -- 172.21.15.33:0/3473320676 >> v1:172.21.15.69:6789/0 conn(0x7fead00bbe70 legacy=0x7fead0069430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feace7fc700 1 -- 172.21.15.33:0/3473320676 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead0151dc0 con 0x7fead01514c0 2023-11-06T20:56:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feace7fc700 1 --2- 172.21.15.33:0/3473320676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01514c0 0x7fead00bb980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fead006be70 tx=0x7feac40090f0).ready entity=mon.0 client_cookie=2aca205accfc58d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feacdffb700 1 -- 172.21.15.33:0/3473320676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac400b600 con 0x7fead01514c0 2023-11-06T20:56:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feacdffb700 1 -- 172.21.15.33:0/3473320676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac400b760 con 0x7fead01514c0 2023-11-06T20:56:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.657+0000 7feacdffb700 1 -- 172.21.15.33:0/3473320676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac400bae0 con 0x7fead01514c0 2023-11-06T20:56:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.658+0000 7fead53ff700 1 -- 172.21.15.33:0/3473320676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01514c0 msgr2=0x7fead00bb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.658+0000 7fead53ff700 1 --2- 172.21.15.33:0/3473320676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01514c0 0x7fead00bb980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fead006be70 tx=0x7feac40090f0).stop 2023-11-06T20:56:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.658+0000 7fead53ff700 1 -- 172.21.15.33:0/3473320676 shutdown_connections 2023-11-06T20:56:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.658+0000 7fead53ff700 1 --2- 172.21.15.33:0/3473320676 >> v2:172.21.15.69:3300/0 conn(0x7fead01510e0 0x7fead00a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.658+0000 7fead53ff700 1 --2- 172.21.15.33:0/3473320676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01514c0 0x7fead00bb980 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.658+0000 7fead53ff700 1 -- 172.21.15.33:0/3473320676 >> 172.21.15.33:0/3473320676 conn(0x7fead0062710 msgr2=0x7fead0062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.658+0000 7fead53ff700 1 -- 172.21.15.33:0/3473320676 shutdown_connections 2023-11-06T20:56:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.658+0000 7fead53ff700 1 -- 172.21.15.33:0/3473320676 wait complete. 2023-11-06T20:56:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.660+0000 7fead53ff700 1 Processor -- start 2023-11-06T20:56:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.660+0000 7fead53ff700 1 -- start start 2023-11-06T20:56:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7fead53ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead00bbe70 0x7fead00a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7fead53ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01510e0 0x7fead0168150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7fead53ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fead01514c0 0x7fead0168640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7fead53ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead0168b30 con 0x7fead01510e0 2023-11-06T20:56:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7fead53ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead0165d50 con 0x7fead00bbe70 2023-11-06T20:56:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7fead53ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead00ab420 con 0x7fead01514c0 2023-11-06T20:56:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7feace7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01510e0 0x7fead0168150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7feace7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01510e0 0x7fead0168150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45886/0 (socket says 172.21.15.33:45886) 2023-11-06T20:56:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7feace7fc700 1 -- 172.21.15.33:0/352170290 learned_addr learned my addr 172.21.15.33:0/352170290 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7feaceffd700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead00bbe70 0x7fead00a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7feacf7fe700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fead01514c0 0x7fead0168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7feace7fc700 1 -- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fead01514c0 msgr2=0x7fead0168640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7feace7fc700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fead01514c0 0x7fead0168640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7feace7fc700 1 -- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead00bbe70 msgr2=0x7fead00a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7feace7fc700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead00bbe70 0x7fead00a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.661+0000 7feace7fc700 1 -- 172.21.15.33:0/352170290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac4009370 con 0x7fead01510e0 2023-11-06T20:56:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.662+0000 7feace7fc700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01510e0 0x7fead0168150 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fead00c4530 tx=0x7feac4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.662+0000 7feab6ffd700 1 -- 172.21.15.33:0/352170290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac400b8d0 con 0x7fead01510e0 2023-11-06T20:56:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.662+0000 7feab6ffd700 1 -- 172.21.15.33:0/352170290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac400ba30 con 0x7fead01510e0 2023-11-06T20:56:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.662+0000 7feab6ffd700 1 -- 172.21.15.33:0/352170290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac401c670 con 0x7fead01510e0 2023-11-06T20:56:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.662+0000 7fead53ff700 1 -- 172.21.15.33:0/352170290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead00c0560 con 0x7fead01510e0 2023-11-06T20:56:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.662+0000 7fead53ff700 1 -- 172.21.15.33:0/352170290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead00ab0b0 con 0x7fead01510e0 2023-11-06T20:56:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.663+0000 7fead53ff700 1 -- 172.21.15.33:0/352170290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead01520a0 con 0x7fead01510e0 2023-11-06T20:56:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.664+0000 7feab6ffd700 1 -- 172.21.15.33:0/352170290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7feac401c9e0 con 0x7fead01510e0 2023-11-06T20:56:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.665+0000 7feab6ffd700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feabc05f2c0 0x7feabc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.665+0000 7feab6ffd700 1 -- 172.21.15.33:0/352170290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7feac407be80 con 0x7fead01510e0 2023-11-06T20:56:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.665+0000 7feaceffd700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feabc05f2c0 0x7feabc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.666+0000 7feaceffd700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feabc05f2c0 0x7feabc061780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7feac000bbc0 tx=0x7feac0008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.669+0000 7feab6ffd700 1 -- 172.21.15.33:0/352170290 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feac404cdb0 con 0x7fead01510e0 2023-11-06T20:56:22.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.878+0000 7fead53ff700 1 -- 172.21.15.33:0/352170290 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fead0063c50 con 0x7feabc05f2c0 2023-11-06T20:56:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.883+0000 7feab6ffd700 1 -- 172.21.15.33:0/352170290 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fead0063c50 con 0x7feabc05f2c0 2023-11-06T20:56:22.883 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-11-06T20:56:22.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 -- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feabc05f2c0 msgr2=0x7feabc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feabc05f2c0 0x7feabc061780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7feac000bbc0 tx=0x7feac0008040).stop 2023-11-06T20:56:22.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 -- 172.21.15.33:0/352170290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01510e0 msgr2=0x7fead0168150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:22.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01510e0 0x7fead0168150 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fead00c4530 tx=0x7feac4017000).stop 2023-11-06T20:56:22.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 -- 172.21.15.33:0/352170290 shutdown_connections 2023-11-06T20:56:22.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feabc05f2c0 0x7feabc061780 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fead01514c0 0x7fead0168640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead00bbe70 0x7fead00a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 --2- 172.21.15.33:0/352170290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01510e0 0x7fead0168150 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 -- 172.21.15.33:0/352170290 >> 172.21.15.33:0/352170290 conn(0x7fead0062710 msgr2=0x7fead0152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 -- 172.21.15.33:0/352170290 shutdown_connections 2023-11-06T20:56:22.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:22.885+0000 7fead53ff700 1 -- 172.21.15.33:0/352170290 wait complete. 2023-11-06T20:56:22.902 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-11-06T20:56:22.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-11-06T20:56:23.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.104+0000 7f1043bff700 1 Processor -- start 2023-11-06T20:56:23.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.104+0000 7f1043bff700 1 -- start start 2023-11-06T20:56:23.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f1043bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f103c0c8ee0 0x7f103c0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f1043bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f103c1542b0 0x7f103c0c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f1043bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f103c0c9f80 con 0x7f103c153ed0 2023-11-06T20:56:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f1043bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103c0c6000 con 0x7f103c1542b0 2023-11-06T20:56:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f1043bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103c0c9370 con 0x7f103c0c8ee0 2023-11-06T20:56:23.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f104219c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f103c1542b0 0x7f103c0c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f104219c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f103c1542b0 0x7f103c0c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45902/0 (socket says 172.21.15.33:45902) 2023-11-06T20:56:23.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f104219c700 1 -- 172.21.15.33:0/4187655664 learned_addr learned my addr 172.21.15.33:0/4187655664 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:23.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f104119a700 1 --2- 172.21.15.33:0/4187655664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f103c0c8ee0 0x7f103c0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f104219c700 1 -- 172.21.15.33:0/4187655664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f103c0c8ee0 msgr2=0x7f103c0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f104219c700 1 --2- 172.21.15.33:0/4187655664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f103c0c8ee0 0x7f103c0ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f104219c700 1 -- 172.21.15.33:0/4187655664 >> v1:172.21.15.69:6789/0 conn(0x7f103c153ed0 legacy=0x7f103c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.106+0000 7f104219c700 1 -- 172.21.15.33:0/4187655664 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c0c9650 con 0x7f103c1542b0 2023-11-06T20:56:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.107+0000 7f104219c700 1 --2- 172.21.15.33:0/4187655664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 0x7f103c0c76f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f1034009f20 tx=0x7f1034009840).ready entity=mon.0 client_cookie=bdd99f8abd8f0f1c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:23.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.107+0000 7f1040999700 1 -- 172.21.15.33:0/4187655664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400e070 con 0x7f103c1542b0 2023-11-06T20:56:23.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.107+0000 7f1040999700 1 -- 172.21.15.33:0/4187655664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103400ba80 con 0x7f103c1542b0 2023-11-06T20:56:23.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.107+0000 7f1040999700 1 -- 172.21.15.33:0/4187655664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400be00 con 0x7f103c1542b0 2023-11-06T20:56:23.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.107+0000 7f1043bff700 1 -- 172.21.15.33:0/4187655664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 msgr2=0x7f103c0c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.107+0000 7f1043bff700 1 --2- 172.21.15.33:0/4187655664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 0x7f103c0c76f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f1034009f20 tx=0x7f1034009840).stop 2023-11-06T20:56:23.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.107+0000 7f1043bff700 1 -- 172.21.15.33:0/4187655664 shutdown_connections 2023-11-06T20:56:23.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.107+0000 7f1043bff700 1 --2- 172.21.15.33:0/4187655664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 0x7f103c0c76f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.107+0000 7f1043bff700 1 --2- 172.21.15.33:0/4187655664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f103c0c8ee0 0x7f103c0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.107+0000 7f1043bff700 1 -- 172.21.15.33:0/4187655664 >> 172.21.15.33:0/4187655664 conn(0x7f103c05e1b0 msgr2=0x7f103c05e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:23.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.108+0000 7f1043bff700 1 -- 172.21.15.33:0/4187655664 shutdown_connections 2023-11-06T20:56:23.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.108+0000 7f1043bff700 1 -- 172.21.15.33:0/4187655664 wait complete. 2023-11-06T20:56:23.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.109+0000 7f1043bff700 1 Processor -- start 2023-11-06T20:56:23.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.109+0000 7f1043bff700 1 -- start start 2023-11-06T20:56:23.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.110+0000 7f1043bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f103c153ed0 0x7f103c14cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.110+0000 7f1043bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 0x7f103c164370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.110+0000 7f1043bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f103c164860 0x7f103c0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.110+0000 7f1043bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c14f690 con 0x7f103c1542b0 2023-11-06T20:56:23.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.110+0000 7f1043bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1504b0 con 0x7f103c164860 2023-11-06T20:56:23.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.110+0000 7f1043bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103c14d260 con 0x7f103c153ed0 2023-11-06T20:56:23.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.110+0000 7f104119a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 0x7f103c164370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.110+0000 7f104119a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 0x7f103c164370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45918/0 (socket says 172.21.15.33:45918) 2023-11-06T20:56:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.110+0000 7f104119a700 1 -- 172.21.15.33:0/1598589894 learned_addr learned my addr 172.21.15.33:0/1598589894 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f104219c700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f103c164860 0x7f103c0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f104199b700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f103c153ed0 0x7f103c14cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f104119a700 1 -- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f103c153ed0 msgr2=0x7f103c14cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f104119a700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f103c153ed0 0x7f103c14cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f104119a700 1 -- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f103c164860 msgr2=0x7f103c0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f104119a700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f103c164860 0x7f103c0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f104119a700 1 -- 172.21.15.33:0/1598589894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1034009ae0 con 0x7f103c1542b0 2023-11-06T20:56:23.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f104119a700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 0x7f103c164370 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f102c00d890 tx=0x7f102c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:23.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f1031ffb700 1 -- 172.21.15.33:0/1598589894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c011590 con 0x7f103c1542b0 2023-11-06T20:56:23.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f1031ffb700 1 -- 172.21.15.33:0/1598589894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c0116f0 con 0x7f103c1542b0 2023-11-06T20:56:23.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f1043bff700 1 -- 172.21.15.33:0/1598589894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103c14f8a0 con 0x7f103c1542b0 2023-11-06T20:56:23.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f1031ffb700 1 -- 172.21.15.33:0/1598589894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c011a70 con 0x7f103c1542b0 2023-11-06T20:56:23.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.111+0000 7f1043bff700 1 -- 172.21.15.33:0/1598589894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103c14fe60 con 0x7f103c1542b0 2023-11-06T20:56:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.113+0000 7f1043bff700 1 -- 172.21.15.33:0/1598589894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103c150660 con 0x7f103c1542b0 2023-11-06T20:56:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.113+0000 7f1031ffb700 1 -- 172.21.15.33:0/1598589894 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f102c01c460 con 0x7f103c1542b0 2023-11-06T20:56:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.114+0000 7f1031ffb700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f102005f2c0 0x7f1020061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.114+0000 7f1031ffb700 1 -- 172.21.15.33:0/1598589894 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f102c07a2d0 con 0x7f103c1542b0 2023-11-06T20:56:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.115+0000 7f104199b700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f102005f2c0 0x7f1020061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.115+0000 7f104199b700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f102005f2c0 0x7f1020061780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1038003900 tx=0x7f1038007d40).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.118+0000 7f1031ffb700 1 -- 172.21.15.33:0/1598589894 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f103c150660 con 0x7f103c1542b0 2023-11-06T20:56:23.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.328+0000 7f1043bff700 1 -- 172.21.15.33:0/1598589894 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f103c150660 con 0x7f102005f2c0 2023-11-06T20:56:23.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.333+0000 7f1031ffb700 1 -- 172.21.15.33:0/1598589894 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f103c150660 con 0x7f102005f2c0 2023-11-06T20:56:23.333 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-11-06T20:56:23.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 -- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f102005f2c0 msgr2=0x7f1020061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f102005f2c0 0x7f1020061780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1038003900 tx=0x7f1038007d40).stop 2023-11-06T20:56:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 -- 172.21.15.33:0/1598589894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 msgr2=0x7f103c164370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 0x7f103c164370 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f102c00d890 tx=0x7f102c008040).stop 2023-11-06T20:56:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 -- 172.21.15.33:0/1598589894 shutdown_connections 2023-11-06T20:56:23.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f102005f2c0 0x7f1020061780 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f103c153ed0 0x7f103c14cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f103c164860 0x7f103c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 --2- 172.21.15.33:0/1598589894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f103c1542b0 0x7f103c164370 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 -- 172.21.15.33:0/1598589894 >> 172.21.15.33:0/1598589894 conn(0x7f103c05e1b0 msgr2=0x7f103c14d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:23.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 -- 172.21.15.33:0/1598589894 shutdown_connections 2023-11-06T20:56:23.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.335+0000 7f1043bff700 1 -- 172.21.15.33:0/1598589894 wait complete. 2023-11-06T20:56:23.353 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-11-06T20:56:23.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-11-06T20:56:23.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.554+0000 7fcbe3fff700 1 Processor -- start 2023-11-06T20:56:23.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.554+0000 7fcbe3fff700 1 -- start start 2023-11-06T20:56:23.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.556+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbdc0c8ee0 0x7fcbdc0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.556+0000 7fcbe3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcbdc1542b0 0x7fcbdc0c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.556+0000 7fcbe3fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbdc0c9f80 con 0x7fcbdc153ed0 2023-11-06T20:56:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.556+0000 7fcbe3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbdc0c6000 con 0x7fcbdc1542b0 2023-11-06T20:56:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.556+0000 7fcbe3fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0c9370 con 0x7fcbdc0c8ee0 2023-11-06T20:56:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.556+0000 7fcbe259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcbdc1542b0 0x7fcbdc0c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.556+0000 7fcbe259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcbdc1542b0 0x7fcbdc0c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45930/0 (socket says 172.21.15.33:45930) 2023-11-06T20:56:23.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.556+0000 7fcbe259c700 1 -- 172.21.15.33:0/3773335191 learned_addr learned my addr 172.21.15.33:0/3773335191 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:23.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.556+0000 7fcbe159a700 1 --2- 172.21.15.33:0/3773335191 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbdc0c8ee0 0x7fcbdc0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.557+0000 7fcbe259c700 1 -- 172.21.15.33:0/3773335191 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbdc0c8ee0 msgr2=0x7fcbdc0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.557+0000 7fcbe259c700 1 --2- 172.21.15.33:0/3773335191 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbdc0c8ee0 0x7fcbdc0ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.557+0000 7fcbe259c700 1 -- 172.21.15.33:0/3773335191 >> v1:172.21.15.69:6789/0 conn(0x7fcbdc153ed0 legacy=0x7fcbdc158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.557+0000 7fcbe259c700 1 -- 172.21.15.33:0/3773335191 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0c9650 con 0x7fcbdc1542b0 2023-11-06T20:56:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.557+0000 7fcbe259c700 1 --2- 172.21.15.33:0/3773335191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc1542b0 0x7fcbdc0c76f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fcbd4009f20 tx=0x7fcbd4009840).ready entity=mon.0 client_cookie=387924d3d0ddde82 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.557+0000 7fcbe0d99700 1 -- 172.21.15.33:0/3773335191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400e070 con 0x7fcbdc1542b0 2023-11-06T20:56:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.557+0000 7fcbe0d99700 1 -- 172.21.15.33:0/3773335191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd400ba80 con 0x7fcbdc1542b0 2023-11-06T20:56:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.557+0000 7fcbe0d99700 1 -- 172.21.15.33:0/3773335191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400be00 con 0x7fcbdc1542b0 2023-11-06T20:56:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.558+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3773335191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc1542b0 msgr2=0x7fcbdc0c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.558+0000 7fcbe3fff700 1 --2- 172.21.15.33:0/3773335191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc1542b0 0x7fcbdc0c76f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fcbd4009f20 tx=0x7fcbd4009840).stop 2023-11-06T20:56:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.558+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3773335191 shutdown_connections 2023-11-06T20:56:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.558+0000 7fcbe3fff700 1 --2- 172.21.15.33:0/3773335191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc1542b0 0x7fcbdc0c76f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.558+0000 7fcbe3fff700 1 --2- 172.21.15.33:0/3773335191 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbdc0c8ee0 0x7fcbdc0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.558+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3773335191 >> 172.21.15.33:0/3773335191 conn(0x7fcbdc05e1b0 msgr2=0x7fcbdc05e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.558+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3773335191 shutdown_connections 2023-11-06T20:56:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.558+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3773335191 wait complete. 2023-11-06T20:56:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.559+0000 7fcbe3fff700 1 Processor -- start 2023-11-06T20:56:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.559+0000 7fcbe3fff700 1 -- start start 2023-11-06T20:56:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.560+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbdc153ed0 0x7fcbdc0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.560+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbdc1542b0 0x7fcbdc0c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.560+0000 7fcbe3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0afdf0 0x7fcbdc0b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.560+0000 7fcbe3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0c37e0 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.560+0000 7fcbe3fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0c4d00 con 0x7fcbdc1542b0 2023-11-06T20:56:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.560+0000 7fcbe3fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0c2f40 con 0x7fcbdc153ed0 2023-11-06T20:56:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0afdf0 0x7fcbdc0b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0afdf0 0x7fcbdc0b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45942/0 (socket says 172.21.15.33:45942) 2023-11-06T20:56:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe259c700 1 -- 172.21.15.33:0/3603796204 learned_addr learned my addr 172.21.15.33:0/3603796204 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe159a700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbdc1542b0 0x7fcbdc0c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe1d9b700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbdc153ed0 0x7fcbdc0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe259c700 1 -- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbdc153ed0 msgr2=0x7fcbdc0c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe259c700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbdc153ed0 0x7fcbdc0c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe259c700 1 -- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbdc1542b0 msgr2=0x7fcbdc0c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe259c700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbdc1542b0 0x7fcbdc0c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe259c700 1 -- 172.21.15.33:0/3603796204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd4009ae0 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe259c700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0afdf0 0x7fcbdc0b0250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fcbd400b6c0 tx=0x7fcbd4009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:23.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbd1ffb700 1 -- 172.21.15.33:0/3603796204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4007170 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbd1ffb700 1 -- 172.21.15.33:0/3603796204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd40092e0 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.561+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3603796204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbdc0c3220 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.562+0000 7fcbd1ffb700 1 -- 172.21.15.33:0/3603796204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4007860 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.562+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3603796204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbdc146320 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.563+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3603796204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbdc146690 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.563+0000 7fcbd1ffb700 1 -- 172.21.15.33:0/3603796204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fcbd4003040 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.564+0000 7fcbd1ffb700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcbc005f290 0x7fcbc0061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.564+0000 7fcbd1ffb700 1 -- 172.21.15.33:0/3603796204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fcbd407bf10 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.565+0000 7fcbe1d9b700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcbc005f290 0x7fcbc0061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.565+0000 7fcbe1d9b700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcbc005f290 0x7fcbc0061750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fcbd8003900 tx=0x7fcbd8007d40).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:23.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.568+0000 7fcbd1ffb700 1 -- 172.21.15.33:0/3603796204 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcbd404cfb0 con 0x7fcbdc0afdf0 2023-11-06T20:56:23.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.783+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3603796204 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbdc0c90e0 con 0x7fcbc005f290 2023-11-06T20:56:23.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.788+0000 7fcbd1ffb700 1 -- 172.21.15.33:0/3603796204 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fcbdc0c90e0 con 0x7fcbc005f290 2023-11-06T20:56:23.788 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-11-06T20:56:23.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcbc005f290 msgr2=0x7fcbc0061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcbc005f290 0x7fcbc0061750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fcbd8003900 tx=0x7fcbd8007d40).stop 2023-11-06T20:56:23.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3603796204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0afdf0 msgr2=0x7fcbdc0b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:23.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0afdf0 0x7fcbdc0b0250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fcbd400b6c0 tx=0x7fcbd4009840).stop 2023-11-06T20:56:23.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3603796204 shutdown_connections 2023-11-06T20:56:23.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcbc005f290 0x7fcbc0061750 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcbdc153ed0 0x7fcbdc0c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0afdf0 0x7fcbdc0b0250 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 --2- 172.21.15.33:0/3603796204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbdc1542b0 0x7fcbdc0c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3603796204 >> 172.21.15.33:0/3603796204 conn(0x7fcbdc05e1b0 msgr2=0x7fcbdc0a9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3603796204 shutdown_connections 2023-11-06T20:56:23.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:23.790+0000 7fcbe3fff700 1 -- 172.21.15.33:0/3603796204 wait complete. 2023-11-06T20:56:23.807 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-11-06T20:56:23.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-11-06T20:56:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.008+0000 7ff7a3dff700 1 Processor -- start 2023-11-06T20:56:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.008+0000 7ff7a3dff700 1 -- start start 2023-11-06T20:56:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.010+0000 7ff7a3dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150c50 0x7ff79c0bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.010+0000 7ff7a3dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79c0bb980 con 0x7ff79c150870 2023-11-06T20:56:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.010+0000 7ff7a3dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79c1553b0 con 0x7ff79c064940 2023-11-06T20:56:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.010+0000 7ff7a3dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c0bbbd0 con 0x7ff79c150c50 2023-11-06T20:56:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.010+0000 7ff7a1b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff79c150870 0x7ff79c064230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53332/0 (socket says 172.21.15.33:53332) 2023-11-06T20:56:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.010+0000 7ff7a1b9b700 1 -- 172.21.15.33:0/3801520843 learned_addr learned my addr 172.21.15.33:0/3801520843 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.010+0000 7ff7a239c700 1 --2- 172.21.15.33:0/3801520843 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150c50 0x7ff79c0bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.010+0000 7ff7a0b99700 1 -- 172.21.15.33:0/3801520843 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527681250 0 0) 0x7ff79c0bb980 con 0x7ff79c150870 2023-11-06T20:56:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a0b99700 1 -- 172.21.15.33:0/3801520843 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7880033f0 con 0x7ff79c150870 2023-11-06T20:56:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a0b99700 1 -- 172.21.15.33:0/3801520843 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764488490 0 0) 0x7ff79c1553b0 con 0x7ff79c064940 2023-11-06T20:56:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a0b99700 1 -- 172.21.15.33:0/3801520843 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff79c0bb980 con 0x7ff79c064940 2023-11-06T20:56:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a0b99700 1 -- 172.21.15.33:0/3801520843 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1292213683 0 0) 0x7ff7880033f0 con 0x7ff79c150870 2023-11-06T20:56:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a239c700 1 -- 172.21.15.33:0/3801520843 >> v1:172.21.15.69:6789/0 conn(0x7ff79c064940 legacy=0x7ff79c154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a239c700 1 -- 172.21.15.33:0/3801520843 >> v1:172.21.15.33:6789/0 conn(0x7ff79c150870 legacy=0x7ff79c064230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a239c700 1 -- 172.21.15.33:0/3801520843 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c0bbeb0 con 0x7ff79c150c50 2023-11-06T20:56:24.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a239c700 1 --2- 172.21.15.33:0/3801520843 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150c50 0x7ff79c0bc690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7ff79400afa0 tx=0x7ff79400a890).ready entity=mon.2 client_cookie=b46e53fd9ce1c60b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:24.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a0b99700 1 -- 172.21.15.33:0/3801520843 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79400f070 con 0x7ff79c150c50 2023-11-06T20:56:24.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a0b99700 1 -- 172.21.15.33:0/3801520843 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79400a0d0 con 0x7ff79c150c50 2023-11-06T20:56:24.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.011+0000 7ff7a0b99700 1 -- 172.21.15.33:0/3801520843 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798001470 con 0x7ff79c150c50 2023-11-06T20:56:24.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.012+0000 7ff7a3dff700 1 -- 172.21.15.33:0/3801520843 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150c50 msgr2=0x7ff79c0bc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.012+0000 7ff7a3dff700 1 --2- 172.21.15.33:0/3801520843 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150c50 0x7ff79c0bc690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7ff79400afa0 tx=0x7ff79400a890).stop 2023-11-06T20:56:24.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.012+0000 7ff7a3dff700 1 -- 172.21.15.33:0/3801520843 shutdown_connections 2023-11-06T20:56:24.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.012+0000 7ff7a3dff700 1 --2- 172.21.15.33:0/3801520843 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150c50 0x7ff79c0bc690 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.012+0000 7ff7a3dff700 1 -- 172.21.15.33:0/3801520843 >> 172.21.15.33:0/3801520843 conn(0x7ff79c062710 msgr2=0x7ff79c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:24.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.012+0000 7ff7a3dff700 1 -- 172.21.15.33:0/3801520843 shutdown_connections 2023-11-06T20:56:24.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.012+0000 7ff7a3dff700 1 -- 172.21.15.33:0/3801520843 wait complete. 2023-11-06T20:56:24.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.014+0000 7ff7a3dff700 1 Processor -- start 2023-11-06T20:56:24.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.014+0000 7ff7a3dff700 1 -- start start 2023-11-06T20:56:24.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a3dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff79c064940 0x7ff79c166450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a3dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150870 0x7ff79c0a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a3dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff79c150c50 0x7ff79c0a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a3dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c166940 con 0x7ff79c150c50 2023-11-06T20:56:24.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a3dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c0a7cb0 con 0x7ff79c064940 2023-11-06T20:56:24.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a3dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c0a7f90 con 0x7ff79c150870 2023-11-06T20:56:24.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a239c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff79c150c50 0x7ff79c0a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a1b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff79c064940 0x7ff79c166450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a239c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff79c150c50 0x7ff79c0a99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45948/0 (socket says 172.21.15.33:45948) 2023-11-06T20:56:24.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a239c700 1 -- 172.21.15.33:0/1331136218 learned_addr learned my addr 172.21.15.33:0/1331136218 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:24.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a139a700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150870 0x7ff79c0a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a239c700 1 -- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150870 msgr2=0x7ff79c0a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a239c700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150870 0x7ff79c0a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a239c700 1 -- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff79c064940 msgr2=0x7ff79c166450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a239c700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff79c064940 0x7ff79c166450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.015+0000 7ff7a239c700 1 -- 172.21.15.33:0/1331136218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79400ab30 con 0x7ff79c150c50 2023-11-06T20:56:24.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.016+0000 7ff7a239c700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff79c150c50 0x7ff79c0a99b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7ff794011000 tx=0x7ff79400a890).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:24.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.016+0000 7ff791ffb700 1 -- 172.21.15.33:0/1331136218 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794014920 con 0x7ff79c150c50 2023-11-06T20:56:24.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.016+0000 7ff791ffb700 1 -- 172.21.15.33:0/1331136218 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff794014aa0 con 0x7ff79c150c50 2023-11-06T20:56:24.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.016+0000 7ff791ffb700 1 -- 172.21.15.33:0/1331136218 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794014e60 con 0x7ff79c150c50 2023-11-06T20:56:24.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.016+0000 7ff7a3dff700 1 -- 172.21.15.33:0/1331136218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c0a8270 con 0x7ff79c150c50 2023-11-06T20:56:24.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.016+0000 7ff7a3dff700 1 -- 172.21.15.33:0/1331136218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c0a8830 con 0x7ff79c150c50 2023-11-06T20:56:24.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.017+0000 7ff7a3dff700 1 -- 172.21.15.33:0/1331136218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79c0a8ba0 con 0x7ff79c150c50 2023-11-06T20:56:24.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.018+0000 7ff791ffb700 1 -- 172.21.15.33:0/1331136218 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7ff79401d070 con 0x7ff79c150c50 2023-11-06T20:56:24.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.019+0000 7ff791ffb700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff78005f2c0 0x7ff780061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.019+0000 7ff791ffb700 1 -- 172.21.15.33:0/1331136218 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7ff79407b150 con 0x7ff79c150c50 2023-11-06T20:56:24.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.019+0000 7ff7a1b9b700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff78005f2c0 0x7ff780061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.020+0000 7ff7a1b9b700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff78005f2c0 0x7ff780061780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff7980015a0 tx=0x7ff798000bc0).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:24.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.023+0000 7ff791ffb700 1 -- 172.21.15.33:0/1331136218 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff79404c140 con 0x7ff79c150c50 2023-11-06T20:56:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.235+0000 7ff7a3dff700 1 -- 172.21.15.33:0/1331136218 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff79c0c42a0 con 0x7ff78005f2c0 2023-11-06T20:56:24.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.240+0000 7ff791ffb700 1 -- 172.21.15.33:0/1331136218 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff79c0c42a0 con 0x7ff78005f2c0 2023-11-06T20:56:24.241 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-11-06T20:56:24.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 -- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff78005f2c0 msgr2=0x7ff780061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff78005f2c0 0x7ff780061780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff7980015a0 tx=0x7ff798000bc0).stop 2023-11-06T20:56:24.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 -- 172.21.15.33:0/1331136218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff79c150c50 msgr2=0x7ff79c0a99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff79c150c50 0x7ff79c0a99b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7ff794011000 tx=0x7ff79400a890).stop 2023-11-06T20:56:24.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 -- 172.21.15.33:0/1331136218 shutdown_connections 2023-11-06T20:56:24.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff78005f2c0 0x7ff780061780 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff79c150c50 0x7ff79c0a99b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff79c064940 0x7ff79c166450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 --2- 172.21.15.33:0/1331136218 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff79c150870 0x7ff79c0a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 -- 172.21.15.33:0/1331136218 >> 172.21.15.33:0/1331136218 conn(0x7ff79c062710 msgr2=0x7ff79c068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 -- 172.21.15.33:0/1331136218 shutdown_connections 2023-11-06T20:56:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.243+0000 7ff7a3dff700 1 -- 172.21.15.33:0/1331136218 wait complete. 2023-11-06T20:56:24.261 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-11-06T20:56:24.261 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-11-06T20:56:24.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.462+0000 7fc1e81ff700 1 Processor -- start 2023-11-06T20:56:24.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.462+0000 7fc1e81ff700 1 -- start start 2023-11-06T20:56:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.463+0000 7fc1e81ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc1e01545e0 0x7fc1e0062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.463+0000 7fc1e81ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1e0155420 0x7fc1e0062c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.463+0000 7fc1e81ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e0069c90 con 0x7fc1e006a120 2023-11-06T20:56:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.463+0000 7fc1e81ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e00630f0 con 0x7fc1e0155420 2023-11-06T20:56:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.463+0000 7fc1e81ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e01549c0 con 0x7fc1e01545e0 2023-11-06T20:56:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.463+0000 7fc1e679c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1e0155420 0x7fc1e0062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.463+0000 7fc1e579a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc1e006a120 0x7fc1e0154d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53336/0 (socket says 172.21.15.33:53336) 2023-11-06T20:56:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.463+0000 7fc1e579a700 1 -- 172.21.15.33:0/2484581503 learned_addr learned my addr 172.21.15.33:0/2484581503 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.464+0000 7fc1e5f9b700 1 --2- 172.21.15.33:0/2484581503 >> v2:172.21.15.69:3300/0 conn(0x7fc1e01545e0 0x7fc1e0062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.464+0000 7fc1e679c700 1 -- 172.21.15.33:0/2484581503 >> v2:172.21.15.69:3300/0 conn(0x7fc1e01545e0 msgr2=0x7fc1e0062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.464+0000 7fc1e679c700 1 --2- 172.21.15.33:0/2484581503 >> v2:172.21.15.69:3300/0 conn(0x7fc1e01545e0 0x7fc1e0062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.464+0000 7fc1e679c700 1 -- 172.21.15.33:0/2484581503 >> v1:172.21.15.33:6789/0 conn(0x7fc1e006a120 legacy=0x7fc1e0154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.464+0000 7fc1e679c700 1 -- 172.21.15.33:0/2484581503 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e0155aa0 con 0x7fc1e0155420 2023-11-06T20:56:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.464+0000 7fc1e4f99700 1 -- 172.21.15.33:0/2484581503 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665549814 0 0) 0x7fc1e0069c90 con 0x7fc1e006a120 2023-11-06T20:56:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.464+0000 7fc1e679c700 1 --2- 172.21.15.33:0/2484581503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 0x7fc1e0062c00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fc1e00c5640 tx=0x7fc1d8009780).ready entity=mon.0 client_cookie=56fbfb9c9699308f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.464+0000 7fc1e4f99700 1 -- 172.21.15.33:0/2484581503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d800e070 con 0x7fc1e0155420 2023-11-06T20:56:24.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.464+0000 7fc1e4f99700 1 -- 172.21.15.33:0/2484581503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d800bc30 con 0x7fc1e0155420 2023-11-06T20:56:24.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.464+0000 7fc1e4f99700 1 -- 172.21.15.33:0/2484581503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8012410 con 0x7fc1e0155420 2023-11-06T20:56:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.465+0000 7fc1e81ff700 1 -- 172.21.15.33:0/2484581503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 msgr2=0x7fc1e0062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.465+0000 7fc1e81ff700 1 --2- 172.21.15.33:0/2484581503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 0x7fc1e0062c00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fc1e00c5640 tx=0x7fc1d8009780).stop 2023-11-06T20:56:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.465+0000 7fc1e81ff700 1 -- 172.21.15.33:0/2484581503 shutdown_connections 2023-11-06T20:56:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.465+0000 7fc1e81ff700 1 --2- 172.21.15.33:0/2484581503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 0x7fc1e0062c00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.465+0000 7fc1e81ff700 1 --2- 172.21.15.33:0/2484581503 >> v2:172.21.15.69:3300/0 conn(0x7fc1e01545e0 0x7fc1e0062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.465+0000 7fc1e81ff700 1 -- 172.21.15.33:0/2484581503 >> 172.21.15.33:0/2484581503 conn(0x7fc1e00c7b30 msgr2=0x7fc1e00c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.465+0000 7fc1e81ff700 1 -- 172.21.15.33:0/2484581503 shutdown_connections 2023-11-06T20:56:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.465+0000 7fc1e81ff700 1 -- 172.21.15.33:0/2484581503 wait complete. 2023-11-06T20:56:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.466+0000 7fc1e81ff700 1 Processor -- start 2023-11-06T20:56:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.467+0000 7fc1e81ff700 1 -- start start 2023-11-06T20:56:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.467+0000 7fc1e81ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e006a120 0x7fc1e013f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.467+0000 7fc1e81ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e01545e0 0x7fc1e00aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e81ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 0x7fc1e00c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e81ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e0152020 con 0x7fc1e0155420 2023-11-06T20:56:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e81ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e00abba0 con 0x7fc1e006a120 2023-11-06T20:56:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e81ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e00ab8a0 con 0x7fc1e01545e0 2023-11-06T20:56:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e679c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 0x7fc1e00c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e679c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 0x7fc1e00c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45976/0 (socket says 172.21.15.33:45976) 2023-11-06T20:56:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e679c700 1 -- 172.21.15.33:0/3924137563 learned_addr learned my addr 172.21.15.33:0/3924137563 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e5f9b700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e006a120 0x7fc1e013f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e579a700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e01545e0 0x7fc1e00aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e679c700 1 -- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e01545e0 msgr2=0x7fc1e00aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e679c700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e01545e0 0x7fc1e00aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e679c700 1 -- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e006a120 msgr2=0x7fc1e013f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e679c700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e006a120 0x7fc1e013f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e679c700 1 -- 172.21.15.33:0/3924137563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d8009a20 con 0x7fc1e0155420 2023-11-06T20:56:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e5f9b700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e006a120 0x7fc1e013f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e579a700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e01545e0 0x7fc1e00aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.468+0000 7fc1e679c700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 0x7fc1e00c6960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fc1d800b6c0 tx=0x7fc1d8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.469+0000 7fc1d5ffb700 1 -- 172.21.15.33:0/3924137563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d801d630 con 0x7fc1e0155420 2023-11-06T20:56:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.469+0000 7fc1d5ffb700 1 -- 172.21.15.33:0/3924137563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d801d790 con 0x7fc1e0155420 2023-11-06T20:56:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.469+0000 7fc1e81ff700 1 -- 172.21.15.33:0/3924137563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e013f7e0 con 0x7fc1e0155420 2023-11-06T20:56:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.469+0000 7fc1d5ffb700 1 -- 172.21.15.33:0/3924137563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d801db30 con 0x7fc1e0155420 2023-11-06T20:56:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.469+0000 7fc1e81ff700 1 -- 172.21.15.33:0/3924137563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e00aa2a0 con 0x7fc1e0155420 2023-11-06T20:56:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.470+0000 7fc1e81ff700 1 -- 172.21.15.33:0/3924137563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e00abd10 con 0x7fc1e0155420 2023-11-06T20:56:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.471+0000 7fc1d5ffb700 1 -- 172.21.15.33:0/3924137563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fc1d8027460 con 0x7fc1e0155420 2023-11-06T20:56:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.471+0000 7fc1d5ffb700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1c405f2c0 0x7fc1c4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.472+0000 7fc1d5ffb700 1 -- 172.21.15.33:0/3924137563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fc1d807c5c0 con 0x7fc1e0155420 2023-11-06T20:56:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.472+0000 7fc1e5f9b700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1c405f2c0 0x7fc1c4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.472+0000 7fc1e5f9b700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1c405f2c0 0x7fc1c4061780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fc1dc0098c0 tx=0x7fc1dc008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.476+0000 7fc1d5ffb700 1 -- 172.21.15.33:0/3924137563 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc1d804d490 con 0x7fc1e0155420 2023-11-06T20:56:24.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.686+0000 7fc1e81ff700 1 -- 172.21.15.33:0/3924137563 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1e0063390 con 0x7fc1c405f2c0 2023-11-06T20:56:24.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.691+0000 7fc1d5ffb700 1 -- 172.21.15.33:0/3924137563 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc1e0063390 con 0x7fc1c405f2c0 2023-11-06T20:56:24.691 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-11-06T20:56:24.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 -- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1c405f2c0 msgr2=0x7fc1c4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1c405f2c0 0x7fc1c4061780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fc1dc0098c0 tx=0x7fc1dc008040).stop 2023-11-06T20:56:24.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 -- 172.21.15.33:0/3924137563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 msgr2=0x7fc1e00c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 0x7fc1e00c6960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fc1d800b6c0 tx=0x7fc1d8009780).stop 2023-11-06T20:56:24.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 -- 172.21.15.33:0/3924137563 shutdown_connections 2023-11-06T20:56:24.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1c405f2c0 0x7fc1c4061780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0155420 0x7fc1e00c6960 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e006a120 0x7fc1e013f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 --2- 172.21.15.33:0/3924137563 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e01545e0 0x7fc1e00aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 -- 172.21.15.33:0/3924137563 >> 172.21.15.33:0/3924137563 conn(0x7fc1e00c7b30 msgr2=0x7fc1e0153800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:24.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 -- 172.21.15.33:0/3924137563 shutdown_connections 2023-11-06T20:56:24.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.693+0000 7fc1e81ff700 1 -- 172.21.15.33:0/3924137563 wait complete. 2023-11-06T20:56:24.712 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-11-06T20:56:24.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:24.713 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-11-06T20:56:24.764 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:24.765 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:24.765 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:24.765 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:24.765 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:24.765 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:24.766 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:24.766 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:24.766 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:24.766 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:24.766 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:24.767 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-20_46_00_UTC', 'scheduled-2023-11-06-20_47_00_UTC', 'scheduled-2023-11-06-20_48_00_UTC', 'scheduled-2023-11-06-20_49_00_UTC', 'scheduled-2023-11-06-20_50_00_UTC', 'scheduled-2023-11-06-20_51_00_UTC', 'scheduled-2023-11-06-20_52_00_UTC', 'scheduled-2023-11-06-20_53_00_UTC', 'scheduled-2023-11-06-20_54_00_UTC', 'scheduled-2023-11-06-20_55_00_UTC', 'scheduled-2023-11-06-20_56_00_UTC'] 2023-11-06T20:56:24.768 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-11-06T20:56:24.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.967+0000 7f8ff93ff700 1 Processor -- start 2023-11-06T20:56:24.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.967+0000 7f8ff93ff700 1 -- start start 2023-11-06T20:56:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.968+0000 7f8ff93ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4153ed0 0x7f8ff4158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.968+0000 7f8ff93ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8ff40c8ee0 0x7f8ff40ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.968+0000 7f8ff93ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff40c9f80 con 0x7f8ff41542b0 2023-11-06T20:56:24.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.968+0000 7f8ff93ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff40c6000 con 0x7f8ff40c8ee0 2023-11-06T20:56:24.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.968+0000 7f8ff93ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40c9370 con 0x7f8ff4153ed0 2023-11-06T20:56:24.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8ff41542b0 0x7f8ff40c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53342/0 (socket says 172.21.15.33:53342) 2023-11-06T20:56:24.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff37fe700 1 -- 172.21.15.33:0/1117895828 learned_addr learned my addr 172.21.15.33:0/1117895828 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:24.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff2ffd700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4153ed0 0x7f8ff4158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff27fc700 1 --2- 172.21.15.33:0/1117895828 >> v2:172.21.15.69:3300/0 conn(0x7f8ff40c8ee0 0x7f8ff40ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20559889 0 0) 0x7f8ff40c9f80 con 0x7f8ff41542b0 2023-11-06T20:56:24.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fdc0033f0 con 0x7f8ff41542b0 2023-11-06T20:56:24.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4040901753 0 0) 0x7f8fdc0033f0 con 0x7f8ff41542b0 2023-11-06T20:56:24.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4153ed0 msgr2=0x7f8ff4158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:24.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4153ed0 0x7f8ff4158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 >> v2:172.21.15.69:3300/0 conn(0x7f8ff40c8ee0 msgr2=0x7f8ff40ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:24.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 --2- 172.21.15.33:0/1117895828 >> v2:172.21.15.69:3300/0 conn(0x7f8ff40c8ee0 0x7f8ff40ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff40c9650 con 0x7f8ff41542b0 2023-11-06T20:56:24.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f8fec0017f0 con 0x7f8ff41542b0 2023-11-06T20:56:24.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 >> v1:172.21.15.33:6789/0 conn(0x7f8ff41542b0 legacy=0x7f8ff40c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fdc0038e0 0x7f8fdc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.969+0000 7f8ff1ffb700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4153ed0 0x7f8fdc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff1ffb700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 0x7f8fdc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40c6000 con 0x7f8fdc004b80 2023-11-06T20:56:24.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40c9370 con 0x7f8fdc0038e0 2023-11-06T20:56:24.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc00b8e0 con 0x7f8ff4153ed0 2023-11-06T20:56:24.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff37fe700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 0x7f8fdc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff27fc700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4153ed0 0x7f8fdc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff2ffd700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fdc0038e0 0x7f8fdc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff37fe700 1 -- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4153ed0 msgr2=0x7f8fdc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:24.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff37fe700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4153ed0 0x7f8fdc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff37fe700 1 -- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fdc0038e0 msgr2=0x7f8fdc003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:24.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff37fe700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fdc0038e0 0x7f8fdc003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff37fe700 1 -- 172.21.15.33:0/1117895828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc0033f0 con 0x7f8fdc004b80 2023-11-06T20:56:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.970+0000 7f8ff37fe700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 0x7f8fdc005000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f8fec00c500 tx=0x7f8fec002210).ready entity=mon.0 client_cookie=71f0532fea0b58fe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.971+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec007de0 con 0x7f8fdc004b80 2023-11-06T20:56:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.971+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec00ce30 con 0x7f8fdc004b80 2023-11-06T20:56:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.971+0000 7f8ff1ffb700 1 -- 172.21.15.33:0/1117895828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec005d50 con 0x7f8fdc004b80 2023-11-06T20:56:24.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.971+0000 7f8ff93ff700 1 -- 172.21.15.33:0/1117895828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 msgr2=0x7f8fdc005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.971+0000 7f8ff93ff700 1 --2- 172.21.15.33:0/1117895828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 0x7f8fdc005000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f8fec00c500 tx=0x7f8fec002210).stop 2023-11-06T20:56:24.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.971+0000 7f8ff37fe700 1 -- 172.21.15.33:0/1117895828 reap_dead start 2023-11-06T20:56:24.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.971+0000 7f8ff93ff700 1 -- 172.21.15.33:0/1117895828 shutdown_connections 2023-11-06T20:56:24.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.971+0000 7f8ff93ff700 1 -- 172.21.15.33:0/1117895828 >> 172.21.15.33:0/1117895828 conn(0x7f8ff405e1b0 msgr2=0x7f8ff405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:24.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.971+0000 7f8ff93ff700 1 -- 172.21.15.33:0/1117895828 shutdown_connections 2023-11-06T20:56:24.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.971+0000 7f8ff93ff700 1 -- 172.21.15.33:0/1117895828 wait complete. 2023-11-06T20:56:24.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.973+0000 7f8ff93ff700 1 Processor -- start 2023-11-06T20:56:24.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.973+0000 7f8ff93ff700 1 -- start start 2023-11-06T20:56:24.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff93ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 0x7f8ff4142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff93ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff41505f0 0x7f8ff4167680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff93ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4150a70 0x7f8ff4167b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff93ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4168060 con 0x7f8fdc004b80 2023-11-06T20:56:24.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff93ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4150f80 con 0x7f8ff41505f0 2023-11-06T20:56:24.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff93ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4151540 con 0x7f8ff4150a70 2023-11-06T20:56:24.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 0x7f8ff4142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 0x7f8ff4142900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45996/0 (socket says 172.21.15.33:45996) 2023-11-06T20:56:24.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff2ffd700 1 -- 172.21.15.33:0/2114766527 learned_addr learned my addr 172.21.15.33:0/2114766527 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:24.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff27fc700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff41505f0 0x7f8ff4167680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.974+0000 7f8ff37fe700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4150a70 0x7f8ff4167b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8ff2ffd700 1 -- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4150a70 msgr2=0x7f8ff4167b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8ff2ffd700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4150a70 0x7f8ff4167b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8ff2ffd700 1 -- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff41505f0 msgr2=0x7f8ff4167680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:24.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8ff2ffd700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff41505f0 0x7f8ff4167680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:24.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8ff2ffd700 1 -- 172.21.15.33:0/2114766527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4142df0 con 0x7f8fdc004b80 2023-11-06T20:56:24.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8ff2ffd700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 0x7f8ff4142900 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f8fe4011a20 tx=0x7f8fe4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:24.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8fdaffd700 1 -- 172.21.15.33:0/2114766527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4011eb0 con 0x7f8fdc004b80 2023-11-06T20:56:24.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8fdaffd700 1 -- 172.21.15.33:0/2114766527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400f8e0 con 0x7f8fdc004b80 2023-11-06T20:56:24.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8fdaffd700 1 -- 172.21.15.33:0/2114766527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400fc60 con 0x7f8fdc004b80 2023-11-06T20:56:24.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8ff93ff700 1 -- 172.21.15.33:0/2114766527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff4153390 con 0x7f8fdc004b80 2023-11-06T20:56:24.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.975+0000 7f8ff93ff700 1 -- 172.21.15.33:0/2114766527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff41511d0 con 0x7f8fdc004b80 2023-11-06T20:56:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.976+0000 7f8ff93ff700 1 -- 172.21.15.33:0/2114766527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff406c6c0 con 0x7f8fdc004b80 2023-11-06T20:56:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.977+0000 7f8fdaffd700 1 -- 172.21.15.33:0/2114766527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f8fe4014650 con 0x7f8fdc004b80 2023-11-06T20:56:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.978+0000 7f8fdaffd700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fe005f2c0 0x7f8fe0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:24.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.978+0000 7f8fdaffd700 1 -- 172.21.15.33:0/2114766527 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f8fe407b190 con 0x7f8fdc004b80 2023-11-06T20:56:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.978+0000 7f8ff27fc700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fe005f2c0 0x7f8fe0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.979+0000 7f8ff27fc700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fe005f2c0 0x7f8fe0061780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f8ff40c1ac0 tx=0x7f8fe8009060).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:24.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:24.982+0000 7f8fdaffd700 1 -- 172.21.15.33:0/2114766527 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8fe404c0c0 con 0x7f8fdc004b80 2023-11-06T20:56:25.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.192+0000 7f8ff93ff700 1 -- 172.21.15.33:0/2114766527 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ff4151680 con 0x7f8fe005f2c0 2023-11-06T20:56:25.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.194+0000 7f8fdaffd700 1 -- 172.21.15.33:0/2114766527 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f8fe8019070 con 0x7f8fe005f2c0 2023-11-06T20:56:25.195 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:25.195 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:25", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:25.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.196+0000 7f8ff93ff700 1 -- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fe005f2c0 msgr2=0x7f8fe0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.196+0000 7f8ff93ff700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fe005f2c0 0x7f8fe0061780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f8ff40c1ac0 tx=0x7f8fe8009060).stop 2023-11-06T20:56:25.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.196+0000 7f8ff93ff700 1 -- 172.21.15.33:0/2114766527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 msgr2=0x7f8ff4142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.196+0000 7f8ff93ff700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 0x7f8ff4142900 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f8fe4011a20 tx=0x7f8fe4009060).stop 2023-11-06T20:56:25.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.196+0000 7f8ff93ff700 1 -- 172.21.15.33:0/2114766527 shutdown_connections 2023-11-06T20:56:25.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.196+0000 7f8ff93ff700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8fe005f2c0 0x7f8fe0061780 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.196+0000 7f8ff93ff700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ff4150a70 0x7f8ff4167b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.196+0000 7f8ff93ff700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fdc004b80 0x7f8ff4142900 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.196+0000 7f8ff93ff700 1 --2- 172.21.15.33:0/2114766527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff41505f0 0x7f8ff4167680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.196+0000 7f8ff93ff700 1 -- 172.21.15.33:0/2114766527 >> 172.21.15.33:0/2114766527 conn(0x7f8ff405e1b0 msgr2=0x7f8ff40ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:25.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.197+0000 7f8ff93ff700 1 -- 172.21.15.33:0/2114766527 shutdown_connections 2023-11-06T20:56:25.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.197+0000 7f8ff93ff700 1 -- 172.21.15.33:0/2114766527 wait complete. 2023-11-06T20:56:25.214 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:25", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:25.214 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T20:45:25', 'first': '2023-11-06T20:46:00', 'last': '2023-11-06T20:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-06T20:56:25.215 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T20:56:25.215 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T20:56:25.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:25.216 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-11-06T20:56:25.265 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:25.266 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:25.266 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:25.266 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:25.266 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:25.266 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:25.267 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:25.267 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:25.267 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:25.267 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:25.267 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:25.268 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-20_46_00_UTC', 'scheduled-2023-11-06-20_47_00_UTC', 'scheduled-2023-11-06-20_48_00_UTC', 'scheduled-2023-11-06-20_49_00_UTC', 'scheduled-2023-11-06-20_50_00_UTC', 'scheduled-2023-11-06-20_51_00_UTC', 'scheduled-2023-11-06-20_52_00_UTC', 'scheduled-2023-11-06-20_53_00_UTC', 'scheduled-2023-11-06-20_54_00_UTC', 'scheduled-2023-11-06-20_55_00_UTC', 'scheduled-2023-11-06-20_56_00_UTC'] 2023-11-06T20:56:25.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-11-06T20:56:25.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.469+0000 7fca15fff700 1 Processor -- start 2023-11-06T20:56:25.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.469+0000 7fca15fff700 1 -- start start 2023-11-06T20:56:25.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.470+0000 7fca15fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fca10153ed0 0x7fca10158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.470+0000 7fca15fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca100c8ee0 0x7fca100ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.470+0000 7fca15fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca100c9f80 con 0x7fca101542b0 2023-11-06T20:56:25.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.470+0000 7fca15fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca100c6000 con 0x7fca100c8ee0 2023-11-06T20:56:25.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.470+0000 7fca15fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca100c9370 con 0x7fca10153ed0 2023-11-06T20:56:25.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.470+0000 7fca0effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca100c8ee0 0x7fca100ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fca101542b0 0x7fca100c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53344/0 (socket says 172.21.15.33:53344) 2023-11-06T20:56:25.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0ffff700 1 -- 172.21.15.33:0/784370446 learned_addr learned my addr 172.21.15.33:0/784370446 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:25.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0f7fe700 1 --2- 172.21.15.33:0/784370446 >> v2:172.21.15.69:3300/0 conn(0x7fca10153ed0 0x7fca10158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0effd700 1 -- 172.21.15.33:0/784370446 >> v2:172.21.15.69:3300/0 conn(0x7fca10153ed0 msgr2=0x7fca10158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:25.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0effd700 1 --2- 172.21.15.33:0/784370446 >> v2:172.21.15.69:3300/0 conn(0x7fca10153ed0 0x7fca10158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0effd700 1 -- 172.21.15.33:0/784370446 >> v1:172.21.15.33:6789/0 conn(0x7fca101542b0 legacy=0x7fca100c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0effd700 1 -- 172.21.15.33:0/784370446 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca100c9650 con 0x7fca100c8ee0 2023-11-06T20:56:25.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0e7fc700 1 -- 172.21.15.33:0/784370446 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712998021 0 0) 0x7fca100c9f80 con 0x7fca101542b0 2023-11-06T20:56:25.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0f7fe700 1 --2- 172.21.15.33:0/784370446 >> v2:172.21.15.69:3300/0 conn(0x7fca10153ed0 0x7fca10158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:25.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0effd700 1 --2- 172.21.15.33:0/784370446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca100c8ee0 0x7fca100ca410 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fca1006bdb0 tx=0x7fca04009060).ready entity=mon.0 client_cookie=1c3a5350017e63bb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:25.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.471+0000 7fca0e7fc700 1 -- 172.21.15.33:0/784370446 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400b580 con 0x7fca100c8ee0 2023-11-06T20:56:25.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.472+0000 7fca0e7fc700 1 -- 172.21.15.33:0/784370446 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0400b6e0 con 0x7fca100c8ee0 2023-11-06T20:56:25.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.472+0000 7fca0e7fc700 1 -- 172.21.15.33:0/784370446 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400ba60 con 0x7fca100c8ee0 2023-11-06T20:56:25.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.472+0000 7fca15fff700 1 -- 172.21.15.33:0/784370446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca100c8ee0 msgr2=0x7fca100ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.472+0000 7fca15fff700 1 --2- 172.21.15.33:0/784370446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca100c8ee0 0x7fca100ca410 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fca1006bdb0 tx=0x7fca04009060).stop 2023-11-06T20:56:25.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.472+0000 7fca15fff700 1 -- 172.21.15.33:0/784370446 shutdown_connections 2023-11-06T20:56:25.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.472+0000 7fca15fff700 1 --2- 172.21.15.33:0/784370446 >> v2:172.21.15.69:3300/0 conn(0x7fca10153ed0 0x7fca10158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.472+0000 7fca15fff700 1 --2- 172.21.15.33:0/784370446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca100c8ee0 0x7fca100ca410 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.472+0000 7fca15fff700 1 -- 172.21.15.33:0/784370446 >> 172.21.15.33:0/784370446 conn(0x7fca1005e1b0 msgr2=0x7fca1005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:25.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.472+0000 7fca15fff700 1 -- 172.21.15.33:0/784370446 shutdown_connections 2023-11-06T20:56:25.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.472+0000 7fca15fff700 1 -- 172.21.15.33:0/784370446 wait complete. 2023-11-06T20:56:25.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.474+0000 7fca15fff700 1 Processor -- start 2023-11-06T20:56:25.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.474+0000 7fca15fff700 1 -- start start 2023-11-06T20:56:25.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca15fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca10153ed0 0x7fca100b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca15fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca101542b0 0x7fca1014fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca15fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca101517f0 0x7fca100c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca15fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1014e500 con 0x7fca101542b0 2023-11-06T20:56:25.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca15fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca10150e30 con 0x7fca101517f0 2023-11-06T20:56:25.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca15fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca10164820 con 0x7fca10153ed0 2023-11-06T20:56:25.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca0effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca101542b0 0x7fca1014fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca0effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca101542b0 0x7fca1014fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46010/0 (socket says 172.21.15.33:46010) 2023-11-06T20:56:25.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca0effd700 1 -- 172.21.15.33:0/126446572 learned_addr learned my addr 172.21.15.33:0/126446572 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:25.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca0ffff700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca101517f0 0x7fca100c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.475+0000 7fca0f7fe700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca10153ed0 0x7fca100b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fca0effd700 1 -- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca10153ed0 msgr2=0x7fca100b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fca0effd700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca10153ed0 0x7fca100b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fca0effd700 1 -- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca101517f0 msgr2=0x7fca100c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fca0effd700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca101517f0 0x7fca100c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fca0effd700 1 -- 172.21.15.33:0/126446572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040092a0 con 0x7fca101542b0 2023-11-06T20:56:25.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fca0f7fe700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca10153ed0 0x7fca100b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:25.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fca0effd700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca101542b0 0x7fca1014fd40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fca04009fd0 tx=0x7fca04017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:25.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fc9f77fe700 1 -- 172.21.15.33:0/126446572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400b850 con 0x7fca101542b0 2023-11-06T20:56:25.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fc9f77fe700 1 -- 172.21.15.33:0/126446572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0400b9b0 con 0x7fca101542b0 2023-11-06T20:56:25.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fca15fff700 1 -- 172.21.15.33:0/126446572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca10150590 con 0x7fca101542b0 2023-11-06T20:56:25.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fc9f77fe700 1 -- 172.21.15.33:0/126446572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0401b640 con 0x7fca101542b0 2023-11-06T20:56:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.476+0000 7fca15fff700 1 -- 172.21.15.33:0/126446572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca10150b50 con 0x7fca101542b0 2023-11-06T20:56:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.477+0000 7fca15fff700 1 -- 172.21.15.33:0/126446572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca10164990 con 0x7fca101542b0 2023-11-06T20:56:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.478+0000 7fc9f77fe700 1 -- 172.21.15.33:0/126446572 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fca0401ba70 con 0x7fca101542b0 2023-11-06T20:56:25.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.479+0000 7fc9f77fe700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc9fc05f2c0 0x7fc9fc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.479+0000 7fc9f77fe700 1 -- 172.21.15.33:0/126446572 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fca0407c140 con 0x7fca101542b0 2023-11-06T20:56:25.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.480+0000 7fca0f7fe700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc9fc05f2c0 0x7fc9fc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.480+0000 7fca0f7fe700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc9fc05f2c0 0x7fc9fc061780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fca00006d80 tx=0x7fca00008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:25.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.484+0000 7fc9f77fe700 1 -- 172.21.15.33:0/126446572 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fca0404d120 con 0x7fca101542b0 2023-11-06T20:56:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.693+0000 7fca15fff700 1 -- 172.21.15.33:0/126446572 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca10164590 con 0x7fc9fc05f2c0 2023-11-06T20:56:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.695+0000 7fc9f77fe700 1 -- 172.21.15.33:0/126446572 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fca10164590 con 0x7fc9fc05f2c0 2023-11-06T20:56:25.695 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:25.695 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:25", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 -- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc9fc05f2c0 msgr2=0x7fc9fc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc9fc05f2c0 0x7fc9fc061780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fca00006d80 tx=0x7fca00008040).stop 2023-11-06T20:56:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 -- 172.21.15.33:0/126446572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca101542b0 msgr2=0x7fca1014fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca101542b0 0x7fca1014fd40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fca04009fd0 tx=0x7fca04017000).stop 2023-11-06T20:56:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 -- 172.21.15.33:0/126446572 shutdown_connections 2023-11-06T20:56:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc9fc05f2c0 0x7fc9fc061780 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca10153ed0 0x7fca100b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca101517f0 0x7fca100c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 --2- 172.21.15.33:0/126446572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca101542b0 0x7fca1014fd40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 -- 172.21.15.33:0/126446572 >> 172.21.15.33:0/126446572 conn(0x7fca1005e1b0 msgr2=0x7fca100cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:25.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 -- 172.21.15.33:0/126446572 shutdown_connections 2023-11-06T20:56:25.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.697+0000 7fca15fff700 1 -- 172.21.15.33:0/126446572 wait complete. 2023-11-06T20:56:25.715 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:25", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:25.715 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T20:45:25', 'first': '2023-11-06T20:46:00', 'last': '2023-11-06T20:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-06T20:56:25.716 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T20:56:25.716 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T20:56:25.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:25.717 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-11-06T20:56:25.765 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:25.766 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:25.766 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:25.766 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:25.766 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:25.767 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:25.767 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:25.767 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:25.768 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:25.768 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:25.768 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:25.769 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-20_46_00_UTC', 'scheduled-2023-11-06-20_47_00_UTC', 'scheduled-2023-11-06-20_48_00_UTC', 'scheduled-2023-11-06-20_49_00_UTC', 'scheduled-2023-11-06-20_50_00_UTC', 'scheduled-2023-11-06-20_51_00_UTC', 'scheduled-2023-11-06-20_52_00_UTC', 'scheduled-2023-11-06-20_53_00_UTC', 'scheduled-2023-11-06-20_54_00_UTC', 'scheduled-2023-11-06-20_55_00_UTC', 'scheduled-2023-11-06-20_56_00_UTC'] 2023-11-06T20:56:25.770 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-11-06T20:56:25.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.969+0000 7f08671ff700 1 Processor -- start 2023-11-06T20:56:25.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.969+0000 7f08671ff700 1 -- start start 2023-11-06T20:56:25.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.971+0000 7f08671ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0860063910 0x7f08600aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.971+0000 7f08671ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08600c0e00 con 0x7f0860154720 2023-11-06T20:56:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.971+0000 7f08671ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0860158bf0 con 0x7f0860154340 2023-11-06T20:56:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.971+0000 7f08671ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0860158ed0 con 0x7f0860063910 2023-11-06T20:56:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.971+0000 7f086579c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0860063910 0x7f08600aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.971+0000 7f085ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0860154720 0x7f0860063200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53352/0 (socket says 172.21.15.33:53352) 2023-11-06T20:56:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.971+0000 7f085ffff700 1 -- 172.21.15.33:0/1752825047 learned_addr learned my addr 172.21.15.33:0/1752825047 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.972+0000 7f086579c700 1 -- 172.21.15.33:0/1752825047 >> v1:172.21.15.69:6789/0 conn(0x7f0860154340 legacy=0x7f0860158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.972+0000 7f086579c700 1 -- 172.21.15.33:0/1752825047 >> v1:172.21.15.33:6789/0 conn(0x7f0860154720 legacy=0x7f0860063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.972+0000 7f086579c700 1 -- 172.21.15.33:0/1752825047 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08600c5d80 con 0x7f0860063910 2023-11-06T20:56:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.972+0000 7f086579c700 1 --2- 172.21.15.33:0/1752825047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 0x7f08600aae40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f0858009e60 tx=0x7f0858009780).ready entity=mon.0 client_cookie=b6b9857640ef0060 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:25.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.972+0000 7f085f7fe700 1 -- 172.21.15.33:0/1752825047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085800e070 con 0x7f0860063910 2023-11-06T20:56:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.972+0000 7f085f7fe700 1 -- 172.21.15.33:0/1752825047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085800bc30 con 0x7f0860063910 2023-11-06T20:56:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.972+0000 7f085f7fe700 1 -- 172.21.15.33:0/1752825047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858012410 con 0x7f0860063910 2023-11-06T20:56:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.972+0000 7f08671ff700 1 -- 172.21.15.33:0/1752825047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 msgr2=0x7f08600aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.972+0000 7f08671ff700 1 --2- 172.21.15.33:0/1752825047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 0x7f08600aae40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f0858009e60 tx=0x7f0858009780).stop 2023-11-06T20:56:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.973+0000 7f08671ff700 1 -- 172.21.15.33:0/1752825047 shutdown_connections 2023-11-06T20:56:25.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.973+0000 7f08671ff700 1 --2- 172.21.15.33:0/1752825047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 0x7f08600aae40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.973+0000 7f08671ff700 1 -- 172.21.15.33:0/1752825047 >> 172.21.15.33:0/1752825047 conn(0x7f086005ec80 msgr2=0x7f08600cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:25.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.973+0000 7f08671ff700 1 -- 172.21.15.33:0/1752825047 shutdown_connections 2023-11-06T20:56:25.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.973+0000 7f08671ff700 1 -- 172.21.15.33:0/1752825047 wait complete. 2023-11-06T20:56:25.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.974+0000 7f08671ff700 1 Processor -- start 2023-11-06T20:56:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.974+0000 7f08671ff700 1 -- start start 2023-11-06T20:56:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.975+0000 7f08671ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 0x7f086014c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.975+0000 7f08671ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0860154340 0x7f08600c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.975+0000 7f08671ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0860154720 0x7f08600c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.975+0000 7f08671ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08600c4040 con 0x7f0860063910 2023-11-06T20:56:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.975+0000 7f08671ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08600ae5a0 con 0x7f0860154720 2023-11-06T20:56:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.975+0000 7f08671ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0860154b30 con 0x7f0860154340 2023-11-06T20:56:25.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f0864f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 0x7f086014c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f0864f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 0x7f086014c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46032/0 (socket says 172.21.15.33:46032) 2023-11-06T20:56:25.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f0864f9b700 1 -- 172.21.15.33:0/2363782515 learned_addr learned my addr 172.21.15.33:0/2363782515 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:25.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f085ffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0860154340 0x7f08600c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f086579c700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0860154720 0x7f08600c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f0864f9b700 1 -- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0860154340 msgr2=0x7f08600c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f0864f9b700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0860154340 0x7f08600c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f0864f9b700 1 -- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0860154720 msgr2=0x7f08600c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f0864f9b700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0860154720 0x7f08600c3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f0864f9b700 1 -- 172.21.15.33:0/2363782515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858009a20 con 0x7f0860063910 2023-11-06T20:56:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f0864f9b700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 0x7f086014c0b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f085400c4c0 tx=0x7f0854007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.976+0000 7f085cff9700 1 -- 172.21.15.33:0/2363782515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085400ce70 con 0x7f0860063910 2023-11-06T20:56:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.977+0000 7f085cff9700 1 -- 172.21.15.33:0/2363782515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0854001e90 con 0x7f0860063910 2023-11-06T20:56:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.977+0000 7f08671ff700 1 -- 172.21.15.33:0/2363782515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0860154e10 con 0x7f0860063910 2023-11-06T20:56:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.977+0000 7f085cff9700 1 -- 172.21.15.33:0/2363782515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854007250 con 0x7f0860063910 2023-11-06T20:56:25.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.977+0000 7f08671ff700 1 -- 172.21.15.33:0/2363782515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08600c2dd0 con 0x7f0860063910 2023-11-06T20:56:25.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.978+0000 7f08671ff700 1 -- 172.21.15.33:0/2363782515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08600c6060 con 0x7f0860063910 2023-11-06T20:56:25.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.978+0000 7f085cff9700 1 -- 172.21.15.33:0/2363782515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f08540066a0 con 0x7f0860063910 2023-11-06T20:56:25.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.980+0000 7f085cff9700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084405f2c0 0x7f0844061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:25.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.980+0000 7f085cff9700 1 -- 172.21.15.33:0/2363782515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f0854079ed0 con 0x7f0860063910 2023-11-06T20:56:25.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.980+0000 7f085ffff700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084405f2c0 0x7f0844061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:25.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.981+0000 7f085ffff700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084405f2c0 0x7f0844061780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0860167ce0 tx=0x7f084c009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:25.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:25.983+0000 7f085cff9700 1 -- 172.21.15.33:0/2363782515 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f085404ae00 con 0x7f0860063910 2023-11-06T20:56:26.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.187+0000 7f08671ff700 1 -- 172.21.15.33:0/2363782515 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0860062ba0 con 0x7f084405f2c0 2023-11-06T20:56:26.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.190+0000 7f085cff9700 1 -- 172.21.15.33:0/2363782515 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f0860062ba0 con 0x7f084405f2c0 2023-11-06T20:56:26.190 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:26.190 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:26", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 -- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084405f2c0 msgr2=0x7f0844061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084405f2c0 0x7f0844061780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0860167ce0 tx=0x7f084c009040).stop 2023-11-06T20:56:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 -- 172.21.15.33:0/2363782515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 msgr2=0x7f086014c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 0x7f086014c0b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f085400c4c0 tx=0x7f0854007e00).stop 2023-11-06T20:56:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 -- 172.21.15.33:0/2363782515 shutdown_connections 2023-11-06T20:56:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084405f2c0 0x7f0844061780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0860154720 0x7f08600c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0860063910 0x7f086014c0b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 --2- 172.21.15.33:0/2363782515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0860154340 0x7f08600c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 -- 172.21.15.33:0/2363782515 >> 172.21.15.33:0/2363782515 conn(0x7f086005ec80 msgr2=0x7f086014cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.191+0000 7f08671ff700 1 -- 172.21.15.33:0/2363782515 shutdown_connections 2023-11-06T20:56:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.192+0000 7f08671ff700 1 -- 172.21.15.33:0/2363782515 wait complete. 2023-11-06T20:56:26.209 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:26", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:26.210 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T20:45:26', 'first': '2023-11-06T20:46:00', 'last': '2023-11-06T20:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-06T20:56:26.210 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T20:56:26.211 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T20:56:26.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:26.212 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-11-06T20:56:26.263 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:26.264 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:26.264 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:26.264 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:26.266 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:26.266 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:26.267 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-20_46_00_UTC', 'scheduled-2023-11-06-20_47_00_UTC', 'scheduled-2023-11-06-20_48_00_UTC', 'scheduled-2023-11-06-20_49_00_UTC', 'scheduled-2023-11-06-20_50_00_UTC', 'scheduled-2023-11-06-20_51_00_UTC', 'scheduled-2023-11-06-20_52_00_UTC', 'scheduled-2023-11-06-20_53_00_UTC', 'scheduled-2023-11-06-20_54_00_UTC', 'scheduled-2023-11-06-20_55_00_UTC', 'scheduled-2023-11-06-20_56_00_UTC'] 2023-11-06T20:56:26.268 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-11-06T20:56:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.468+0000 7fab231ff700 1 Processor -- start 2023-11-06T20:56:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.468+0000 7fab231ff700 1 -- start start 2023-11-06T20:56:26.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.470+0000 7fab231ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab1c0c9340 0x7fab1c155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.470+0000 7fab231ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1c063950 con 0x7fab1c13a960 2023-11-06T20:56:26.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.470+0000 7fab231ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1c063c30 con 0x7fab1c066ba0 2023-11-06T20:56:26.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.470+0000 7fab231ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c064cc0 con 0x7fab1c0c9340 2023-11-06T20:56:26.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab20f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab1c13a960 0x7fab1c066490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53366/0 (socket says 172.21.15.33:53366) 2023-11-06T20:56:26.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab20f9b700 1 -- 172.21.15.33:0/2782187811 learned_addr learned my addr 172.21.15.33:0/2782187811 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:26.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab2179c700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab1c0c9340 0x7fab1c155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802376566 0 0) 0x7fab1c063950 con 0x7fab1c13a960 2023-11-06T20:56:26.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab0c0033f0 con 0x7fab1c13a960 2023-11-06T20:56:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819817763 0 0) 0x7fab1c063c30 con 0x7fab1c066ba0 2023-11-06T20:56:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab1c063950 con 0x7fab1c066ba0 2023-11-06T20:56:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 774146199 0 0) 0x7fab0c0033f0 con 0x7fab1c13a960 2023-11-06T20:56:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab1c0c9340 msgr2=0x7fab1c155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab1c0c9340 0x7fab1c155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 >> v1:172.21.15.69:6789/0 conn(0x7fab1c066ba0 legacy=0x7fab1c0c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c13ad40 con 0x7fab1c13a960 2023-11-06T20:56:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fab10001470 con 0x7fab1c13a960 2023-11-06T20:56:26.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 >> v1:172.21.15.33:6789/0 conn(0x7fab1c13a960 legacy=0x7fab1c066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.471+0000 7fab1b7fe700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab0c003aa0 0x7fab0c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1b7fe700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c004690 0x7fab0c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1b7fe700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c005000 0x7fab0c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c064cc0 con 0x7fab0c004690 2023-11-06T20:56:26.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c00be80 con 0x7fab0c003aa0 2023-11-06T20:56:26.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c00c040 con 0x7fab0c005000 2023-11-06T20:56:26.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab2179c700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab1c0c9340 0x7fab1c155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:26.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1bfff700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c004690 0x7fab0c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab20f9b700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab0c003aa0 0x7fab0c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab2179c700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c005000 0x7fab0c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1bfff700 1 -- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c005000 msgr2=0x7fab0c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1bfff700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c005000 0x7fab0c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1bfff700 1 -- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab0c003aa0 msgr2=0x7fab0c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1bfff700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab0c003aa0 0x7fab0c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1bfff700 1 -- 172.21.15.33:0/2782187811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c0033f0 con 0x7fab0c004690 2023-11-06T20:56:26.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab2179c700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c005000 0x7fab0c00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:26.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.472+0000 7fab1bfff700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c004690 0x7fab0c004b10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fab080018a0 tx=0x7fab08007bb0).ready entity=mon.0 client_cookie=fb2f542753c25db5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:26.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.473+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08007020 con 0x7fab0c004690 2023-11-06T20:56:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.473+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab08007180 con 0x7fab0c004690 2023-11-06T20:56:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.473+0000 7fab1b7fe700 1 -- 172.21.15.33:0/2782187811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08007500 con 0x7fab0c004690 2023-11-06T20:56:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.473+0000 7fab231ff700 1 -- 172.21.15.33:0/2782187811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c004690 msgr2=0x7fab0c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.473+0000 7fab231ff700 1 --2- 172.21.15.33:0/2782187811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab0c004690 0x7fab0c004b10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fab080018a0 tx=0x7fab08007bb0).stop 2023-11-06T20:56:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.473+0000 7fab2179c700 1 -- 172.21.15.33:0/2782187811 reap_dead start 2023-11-06T20:56:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.473+0000 7fab231ff700 1 -- 172.21.15.33:0/2782187811 shutdown_connections 2023-11-06T20:56:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.473+0000 7fab231ff700 1 -- 172.21.15.33:0/2782187811 >> 172.21.15.33:0/2782187811 conn(0x7fab1c05ec70 msgr2=0x7fab1c0c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.473+0000 7fab231ff700 1 -- 172.21.15.33:0/2782187811 shutdown_connections 2023-11-06T20:56:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.473+0000 7fab231ff700 1 -- 172.21.15.33:0/2782187811 wait complete. 2023-11-06T20:56:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.475+0000 7fab231ff700 1 Processor -- start 2023-11-06T20:56:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.475+0000 7fab231ff700 1 -- start start 2023-11-06T20:56:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab231ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c004690 0x7fab1c1399e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab231ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1c0c7930 0x7fab1c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab231ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab1c0acde0 0x7fab1c0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab231ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c155930 con 0x7fab1c0c7930 2023-11-06T20:56:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab231ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0c5a00 con 0x7fab1c0acde0 2023-11-06T20:56:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab231ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0c82a0 con 0x7fab0c004690 2023-11-06T20:56:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1c0c7930 0x7fab1c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1c0c7930 0x7fab1c0c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46056/0 (socket says 172.21.15.33:46056) 2023-11-06T20:56:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab1bfff700 1 -- 172.21.15.33:0/78966912 learned_addr learned my addr 172.21.15.33:0/78966912 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab20f9b700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c004690 0x7fab1c1399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.476+0000 7fab2179c700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab1c0acde0 0x7fab1c0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab1bfff700 1 -- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c004690 msgr2=0x7fab1c1399e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab1bfff700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c004690 0x7fab1c1399e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab1bfff700 1 -- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab1c0acde0 msgr2=0x7fab1c0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab1bfff700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab1c0acde0 0x7fab1c0ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab1bfff700 1 -- 172.21.15.33:0/78966912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c0c56d0 con 0x7fab1c0c7930 2023-11-06T20:56:26.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab20f9b700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c004690 0x7fab1c1399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:26.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab1bfff700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1c0c7930 0x7fab1c0c7db0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fab08001e90 tx=0x7fab08016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:26.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab18ff9700 1 -- 172.21.15.33:0/78966912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08007020 con 0x7fab1c0c7930 2023-11-06T20:56:26.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab18ff9700 1 -- 172.21.15.33:0/78966912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0801d440 con 0x7fab1c0c7930 2023-11-06T20:56:26.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab231ff700 1 -- 172.21.15.33:0/78966912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c0adaa0 con 0x7fab1c0c7930 2023-11-06T20:56:26.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab18ff9700 1 -- 172.21.15.33:0/78966912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0801d7e0 con 0x7fab1c0c7930 2023-11-06T20:56:26.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.477+0000 7fab231ff700 1 -- 172.21.15.33:0/78966912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c0ae060 con 0x7fab1c0c7930 2023-11-06T20:56:26.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.478+0000 7fab231ff700 1 -- 172.21.15.33:0/78966912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c0ae280 con 0x7fab1c0c7930 2023-11-06T20:56:26.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.479+0000 7fab18ff9700 1 -- 172.21.15.33:0/78966912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fab08012400 con 0x7fab1c0c7930 2023-11-06T20:56:26.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.480+0000 7fab18ff9700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fab0005f2c0 0x7fab00061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.480+0000 7fab18ff9700 1 -- 172.21.15.33:0/78966912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fab0807b310 con 0x7fab1c0c7930 2023-11-06T20:56:26.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.481+0000 7fab20f9b700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fab0005f2c0 0x7fab00061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.481+0000 7fab20f9b700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fab0005f2c0 0x7fab00061780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fab10001c80 tx=0x7fab10001df0).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:26.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.484+0000 7fab18ff9700 1 -- 172.21.15.33:0/78966912 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fab1c0ae280 con 0x7fab1c0c7930 2023-11-06T20:56:26.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.689+0000 7fab231ff700 1 -- 172.21.15.33:0/78966912 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab1c0ae280 con 0x7fab0005f2c0 2023-11-06T20:56:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.691+0000 7fab18ff9700 1 -- 172.21.15.33:0/78966912 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fab1c0ae280 con 0x7fab0005f2c0 2023-11-06T20:56:26.691 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:26.692 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:26", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:26.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 -- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fab0005f2c0 msgr2=0x7fab00061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fab0005f2c0 0x7fab00061780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fab10001c80 tx=0x7fab10001df0).stop 2023-11-06T20:56:26.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 -- 172.21.15.33:0/78966912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1c0c7930 msgr2=0x7fab1c0c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1c0c7930 0x7fab1c0c7db0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fab08001e90 tx=0x7fab08016000).stop 2023-11-06T20:56:26.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 -- 172.21.15.33:0/78966912 shutdown_connections 2023-11-06T20:56:26.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fab0005f2c0 0x7fab00061780 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fab0c004690 0x7fab1c1399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab1c0acde0 0x7fab1c0ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 --2- 172.21.15.33:0/78966912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1c0c7930 0x7fab1c0c7db0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 -- 172.21.15.33:0/78966912 >> 172.21.15.33:0/78966912 conn(0x7fab1c05ec70 msgr2=0x7fab1c0c1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:26.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 -- 172.21.15.33:0/78966912 shutdown_connections 2023-11-06T20:56:26.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.693+0000 7fab231ff700 1 -- 172.21.15.33:0/78966912 wait complete. 2023-11-06T20:56:26.711 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:26", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:26.711 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T20:45:26', 'first': '2023-11-06T20:46:00', 'last': '2023-11-06T20:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-06T20:56:26.712 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T20:56:26.712 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T20:56:26.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:26.713 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-11-06T20:56:26.761 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:26.761 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:26.761 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:26.762 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:26.762 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:26.762 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:26.762 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:26.762 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:26.763 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_01_UTC 2023-11-06T20:56:26.763 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:26.763 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:26.764 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-20_46_00_UTC', 'scheduled-2023-11-06-20_47_00_UTC', 'scheduled-2023-11-06-20_48_00_UTC', 'scheduled-2023-11-06-20_49_00_UTC', 'scheduled-2023-11-06-20_50_00_UTC', 'scheduled-2023-11-06-20_51_00_UTC', 'scheduled-2023-11-06-20_52_00_UTC', 'scheduled-2023-11-06-20_53_00_UTC', 'scheduled-2023-11-06-20_54_01_UTC', 'scheduled-2023-11-06-20_55_00_UTC', 'scheduled-2023-11-06-20_56_00_UTC'] 2023-11-06T20:56:26.764 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-11-06T20:56:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.965+0000 7f249c9ff700 1 Processor -- start 2023-11-06T20:56:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.965+0000 7f249c9ff700 1 -- start start 2023-11-06T20:56:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.966+0000 7f249c9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f249813a960 0x7f2498155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.966+0000 7f249c9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2498063950 con 0x7f24980c9340 2023-11-06T20:56:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.966+0000 7f249c9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2498063c30 con 0x7f2498066ba0 2023-11-06T20:56:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.966+0000 7f249c9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2498064cc0 con 0x7f249813a960 2023-11-06T20:56:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f249659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f249813a960 0x7f2498155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f2495d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f24980c9340 0x7f2498066490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53380/0 (socket says 172.21.15.33:53380) 2023-11-06T20:56:26.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f2495d9b700 1 -- 172.21.15.33:0/103902140 learned_addr learned my addr 172.21.15.33:0/103902140 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:26.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f249659c700 1 -- 172.21.15.33:0/103902140 >> v1:172.21.15.69:6789/0 conn(0x7f2498066ba0 legacy=0x7f24980c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f249659c700 1 -- 172.21.15.33:0/103902140 >> v1:172.21.15.33:6789/0 conn(0x7f24980c9340 legacy=0x7f2498066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f249659c700 1 -- 172.21.15.33:0/103902140 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249813ad40 con 0x7f249813a960 2023-11-06T20:56:26.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f249559a700 1 -- 172.21.15.33:0/103902140 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837031859 0 0) 0x7f2498063950 con 0x7f24980c9340 2023-11-06T20:56:26.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f249659c700 1 --2- 172.21.15.33:0/103902140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 0x7f2498155380 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f2488009760 tx=0x7f2488009060).ready entity=mon.0 client_cookie=df55e027008a35d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f249559a700 1 -- 172.21.15.33:0/103902140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800b580 con 0x7f249813a960 2023-11-06T20:56:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f249559a700 1 -- 172.21.15.33:0/103902140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248800b6e0 con 0x7f249813a960 2023-11-06T20:56:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.967+0000 7f249559a700 1 -- 172.21.15.33:0/103902140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800ba60 con 0x7f249813a960 2023-11-06T20:56:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.968+0000 7f249c9ff700 1 -- 172.21.15.33:0/103902140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 msgr2=0x7f2498155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.968+0000 7f249c9ff700 1 --2- 172.21.15.33:0/103902140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 0x7f2498155380 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f2488009760 tx=0x7f2488009060).stop 2023-11-06T20:56:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.968+0000 7f249c9ff700 1 -- 172.21.15.33:0/103902140 shutdown_connections 2023-11-06T20:56:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.968+0000 7f249c9ff700 1 --2- 172.21.15.33:0/103902140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 0x7f2498155380 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.968+0000 7f249c9ff700 1 -- 172.21.15.33:0/103902140 >> 172.21.15.33:0/103902140 conn(0x7f249805ec70 msgr2=0x7f24980c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.968+0000 7f249c9ff700 1 -- 172.21.15.33:0/103902140 shutdown_connections 2023-11-06T20:56:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.968+0000 7f249c9ff700 1 -- 172.21.15.33:0/103902140 wait complete. 2023-11-06T20:56:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.970+0000 7f249c9ff700 1 Processor -- start 2023-11-06T20:56:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.970+0000 7f249c9ff700 1 -- start start 2023-11-06T20:56:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f249c9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2498066ba0 0x7f24980a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f249c9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24980c9340 0x7f24980a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f249c9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 0x7f24980a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f249c9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2498155930 con 0x7f249813a960 2023-11-06T20:56:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f249c9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24980a96b0 con 0x7f2498066ba0 2023-11-06T20:56:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f249c9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24980a52c0 con 0x7f24980c9340 2023-11-06T20:56:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f2496d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 0x7f24980a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f2496d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 0x7f24980a6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46064/0 (socket says 172.21.15.33:46064) 2023-11-06T20:56:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f2496d9d700 1 -- 172.21.15.33:0/3060542718 learned_addr learned my addr 172.21.15.33:0/3060542718 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f249659c700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2498066ba0 0x7f24980a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f2495d9b700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24980c9340 0x7f24980a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f2496d9d700 1 -- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24980c9340 msgr2=0x7f24980a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.971+0000 7f2496d9d700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24980c9340 0x7f24980a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.972+0000 7f2496d9d700 1 -- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2498066ba0 msgr2=0x7f24980a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.972+0000 7f2496d9d700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2498066ba0 0x7f24980a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.972+0000 7f2496d9d700 1 -- 172.21.15.33:0/3060542718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880092a0 con 0x7f249813a960 2023-11-06T20:56:26.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.972+0000 7f2495d9b700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24980c9340 0x7f24980a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:26.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.972+0000 7f2496d9d700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 0x7f24980a6750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f249000c790 tx=0x7f24900014e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:26.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.972+0000 7f24867fc700 1 -- 172.21.15.33:0/3060542718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249000e4a0 con 0x7f249813a960 2023-11-06T20:56:26.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.972+0000 7f24867fc700 1 -- 172.21.15.33:0/3060542718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249000e600 con 0x7f249813a960 2023-11-06T20:56:26.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.972+0000 7f24867fc700 1 -- 172.21.15.33:0/3060542718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249000e980 con 0x7f249813a960 2023-11-06T20:56:26.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.972+0000 7f249c9ff700 1 -- 172.21.15.33:0/3060542718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24980ae6c0 con 0x7f249813a960 2023-11-06T20:56:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.972+0000 7f249c9ff700 1 -- 172.21.15.33:0/3060542718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24980a8ff0 con 0x7f249813a960 2023-11-06T20:56:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.973+0000 7f249c9ff700 1 -- 172.21.15.33:0/3060542718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24980a97f0 con 0x7f249813a960 2023-11-06T20:56:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.974+0000 7f24867fc700 1 -- 172.21.15.33:0/3060542718 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f249000ed30 con 0x7f249813a960 2023-11-06T20:56:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.975+0000 7f24867fc700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f247805f2c0 0x7f2478061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:26.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.975+0000 7f24867fc700 1 -- 172.21.15.33:0/3060542718 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f2490079e70 con 0x7f249813a960 2023-11-06T20:56:26.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.975+0000 7f249659c700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f247805f2c0 0x7f2478061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:26.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.976+0000 7f249659c700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f247805f2c0 0x7f2478061780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f248800b340 tx=0x7f2488017000).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:26.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:26.979+0000 7f24867fc700 1 -- 172.21.15.33:0/3060542718 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2490047af0 con 0x7f249813a960 2023-11-06T20:56:27.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.187+0000 7f249c9ff700 1 -- 172.21.15.33:0/3060542718 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24980634c0 con 0x7f247805f2c0 2023-11-06T20:56:27.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.189+0000 7f24867fc700 1 -- 172.21.15.33:0/3060542718 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f24980634c0 con 0x7f247805f2c0 2023-11-06T20:56:27.190 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:27.190 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:27", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:27.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.191+0000 7f249c9ff700 1 -- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f247805f2c0 msgr2=0x7f2478061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.191+0000 7f249c9ff700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f247805f2c0 0x7f2478061780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f248800b340 tx=0x7f2488017000).stop 2023-11-06T20:56:27.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.191+0000 7f249c9ff700 1 -- 172.21.15.33:0/3060542718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 msgr2=0x7f24980a6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.191+0000 7f249c9ff700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 0x7f24980a6750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f249000c790 tx=0x7f24900014e0).stop 2023-11-06T20:56:27.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.192+0000 7f249c9ff700 1 -- 172.21.15.33:0/3060542718 shutdown_connections 2023-11-06T20:56:27.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.192+0000 7f249c9ff700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f247805f2c0 0x7f2478061780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.192+0000 7f249c9ff700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f249813a960 0x7f24980a6750 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.192+0000 7f249c9ff700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2498066ba0 0x7f24980a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.192+0000 7f249c9ff700 1 --2- 172.21.15.33:0/3060542718 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f24980c9340 0x7f24980a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.192+0000 7f249c9ff700 1 -- 172.21.15.33:0/3060542718 >> 172.21.15.33:0/3060542718 conn(0x7f249805ec70 msgr2=0x7f24980c0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:27.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.192+0000 7f249c9ff700 1 -- 172.21.15.33:0/3060542718 shutdown_connections 2023-11-06T20:56:27.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.192+0000 7f249c9ff700 1 -- 172.21.15.33:0/3060542718 wait complete. 2023-11-06T20:56:27.209 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:27", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:27.210 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T20:45:27', 'first': '2023-11-06T20:46:00', 'last': '2023-11-06T20:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-06T20:56:27.210 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T20:56:27.210 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T20:56:27.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:27.211 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-11-06T20:56:27.260 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:27.261 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:27.261 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:27.261 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:27.261 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:27.261 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:27.262 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:27.262 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:27.262 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:27.262 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:27.262 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:27.263 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-20_46_00_UTC', 'scheduled-2023-11-06-20_47_00_UTC', 'scheduled-2023-11-06-20_48_00_UTC', 'scheduled-2023-11-06-20_49_00_UTC', 'scheduled-2023-11-06-20_50_00_UTC', 'scheduled-2023-11-06-20_51_00_UTC', 'scheduled-2023-11-06-20_52_00_UTC', 'scheduled-2023-11-06-20_53_00_UTC', 'scheduled-2023-11-06-20_54_00_UTC', 'scheduled-2023-11-06-20_55_00_UTC', 'scheduled-2023-11-06-20_56_00_UTC'] 2023-11-06T20:56:27.264 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-11-06T20:56:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.463+0000 7f76175ff700 1 Processor -- start 2023-11-06T20:56:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.463+0000 7f76175ff700 1 -- start start 2023-11-06T20:56:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f76175ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7610068c20 0x7f7610069000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f76175ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f761005e3c0 0x7f7610063950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f76175ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7610063e40 0x7f76100bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f76175ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76100634c0 con 0x7f761005e3c0 2023-11-06T20:56:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f76175ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76100c4600 con 0x7f7610063e40 2023-11-06T20:56:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f76175ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76100628b0 con 0x7f7610068c20 2023-11-06T20:56:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f7614b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f761005e3c0 0x7f7610063950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f7614b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f761005e3c0 0x7f7610063950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46072/0 (socket says 172.21.15.33:46072) 2023-11-06T20:56:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f7614b9a700 1 -- 172.21.15.33:0/3797250946 learned_addr learned my addr 172.21.15.33:0/3797250946 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f7615b9c700 1 --2- 172.21.15.33:0/3797250946 >> v2:172.21.15.69:3300/0 conn(0x7f7610063e40 0x7f76100bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f761539b700 1 --2- 172.21.15.33:0/3797250946 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7610068c20 0x7f7610069000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f7614b9a700 1 -- 172.21.15.33:0/3797250946 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7610068c20 msgr2=0x7f7610069000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f7614b9a700 1 --2- 172.21.15.33:0/3797250946 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7610068c20 0x7f7610069000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f7614b9a700 1 -- 172.21.15.33:0/3797250946 >> v2:172.21.15.69:3300/0 conn(0x7f7610063e40 msgr2=0x7f76100bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f7614b9a700 1 --2- 172.21.15.33:0/3797250946 >> v2:172.21.15.69:3300/0 conn(0x7f7610063e40 0x7f76100bc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f7614b9a700 1 -- 172.21.15.33:0/3797250946 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7610062b90 con 0x7f761005e3c0 2023-11-06T20:56:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.465+0000 7f761539b700 1 --2- 172.21.15.33:0/3797250946 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7610068c20 0x7f7610069000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.466+0000 7f7614b9a700 1 --2- 172.21.15.33:0/3797250946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761005e3c0 0x7f7610063950 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f7610066a20 tx=0x7f76000090f0).ready entity=mon.0 client_cookie=1f2c4c82adf5887f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.466+0000 7f7607fff700 1 -- 172.21.15.33:0/3797250946 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760000b520 con 0x7f761005e3c0 2023-11-06T20:56:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.466+0000 7f7607fff700 1 -- 172.21.15.33:0/3797250946 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760000b680 con 0x7f761005e3c0 2023-11-06T20:56:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.466+0000 7f7607fff700 1 -- 172.21.15.33:0/3797250946 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760000ba00 con 0x7f761005e3c0 2023-11-06T20:56:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.466+0000 7f76175ff700 1 -- 172.21.15.33:0/3797250946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761005e3c0 msgr2=0x7f7610063950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.466+0000 7f76175ff700 1 --2- 172.21.15.33:0/3797250946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761005e3c0 0x7f7610063950 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f7610066a20 tx=0x7f76000090f0).stop 2023-11-06T20:56:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.467+0000 7f76175ff700 1 -- 172.21.15.33:0/3797250946 shutdown_connections 2023-11-06T20:56:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.467+0000 7f76175ff700 1 --2- 172.21.15.33:0/3797250946 >> v2:172.21.15.69:3300/0 conn(0x7f7610063e40 0x7f76100bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.467+0000 7f76175ff700 1 --2- 172.21.15.33:0/3797250946 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7610068c20 0x7f7610069000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.467+0000 7f76175ff700 1 --2- 172.21.15.33:0/3797250946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761005e3c0 0x7f7610063950 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.467+0000 7f76175ff700 1 -- 172.21.15.33:0/3797250946 >> 172.21.15.33:0/3797250946 conn(0x7f76100c7d10 msgr2=0x7f76100c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.467+0000 7f76175ff700 1 -- 172.21.15.33:0/3797250946 shutdown_connections 2023-11-06T20:56:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.467+0000 7f76175ff700 1 -- 172.21.15.33:0/3797250946 wait complete. 2023-11-06T20:56:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.468+0000 7f76175ff700 1 Processor -- start 2023-11-06T20:56:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.468+0000 7f76175ff700 1 -- start start 2023-11-06T20:56:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.469+0000 7f76175ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f761005e3c0 0x7f761013de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.469+0000 7f76175ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7610063e40 0x7f76100aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.469+0000 7f76175ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7610068c20 0x7f761014e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.469+0000 7f76175ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7610152b50 con 0x7f7610063e40 2023-11-06T20:56:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.469+0000 7f76175ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761014b4f0 con 0x7f7610068c20 2023-11-06T20:56:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.469+0000 7f76175ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7610151f40 con 0x7f761005e3c0 2023-11-06T20:56:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.469+0000 7f7614b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7610063e40 0x7f76100aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.469+0000 7f7614b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7610063e40 0x7f76100aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46082/0 (socket says 172.21.15.33:46082) 2023-11-06T20:56:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.469+0000 7f7614b9a700 1 -- 172.21.15.33:0/469198256 learned_addr learned my addr 172.21.15.33:0/469198256 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f761539b700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f761005e3c0 0x7f761013de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f7615b9c700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7610068c20 0x7f761014e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f7614b9a700 1 -- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f761005e3c0 msgr2=0x7f761013de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f7614b9a700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f761005e3c0 0x7f761013de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f7614b9a700 1 -- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7610068c20 msgr2=0x7f761014e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f7614b9a700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7610068c20 0x7f761014e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f7614b9a700 1 -- 172.21.15.33:0/469198256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7600009370 con 0x7f7610063e40 2023-11-06T20:56:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f761539b700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f761005e3c0 0x7f761013de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f7614b9a700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7610063e40 0x7f76100aad00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f7600011000 tx=0x7f7600017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f76057fa700 1 -- 172.21.15.33:0/469198256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600000bc0 con 0x7f7610063e40 2023-11-06T20:56:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f76057fa700 1 -- 172.21.15.33:0/469198256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760000b7f0 con 0x7f7610063e40 2023-11-06T20:56:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f76175ff700 1 -- 172.21.15.33:0/469198256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7610152220 con 0x7f7610063e40 2023-11-06T20:56:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f76057fa700 1 -- 172.21.15.33:0/469198256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600000d20 con 0x7f7610063e40 2023-11-06T20:56:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.470+0000 7f76175ff700 1 -- 172.21.15.33:0/469198256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76101527e0 con 0x7f7610063e40 2023-11-06T20:56:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.472+0000 7f76175ff700 1 -- 172.21.15.33:0/469198256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7610062e70 con 0x7f7610063e40 2023-11-06T20:56:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.472+0000 7f76057fa700 1 -- 172.21.15.33:0/469198256 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f760001c8c0 con 0x7f7610063e40 2023-11-06T20:56:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.473+0000 7f76057fa700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75f405f2c0 0x7f75f4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.473+0000 7f76057fa700 1 -- 172.21.15.33:0/469198256 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f760007c160 con 0x7f7610063e40 2023-11-06T20:56:27.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.474+0000 7f761539b700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75f405f2c0 0x7f75f4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.474+0000 7f761539b700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75f405f2c0 0x7f75f4061780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f760c006d80 tx=0x7f760c008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:27.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.477+0000 7f76057fa700 1 -- 172.21.15.33:0/469198256 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f760004d090 con 0x7f7610063e40 2023-11-06T20:56:27.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.685+0000 7f76175ff700 1 -- 172.21.15.33:0/469198256 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7610065bc0 con 0x7f75f405f2c0 2023-11-06T20:56:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.687+0000 7f76057fa700 1 -- 172.21.15.33:0/469198256 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f7610065bc0 con 0x7f75f405f2c0 2023-11-06T20:56:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:27.688 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:27", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:27.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 -- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75f405f2c0 msgr2=0x7f75f4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75f405f2c0 0x7f75f4061780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f760c006d80 tx=0x7f760c008040).stop 2023-11-06T20:56:27.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 -- 172.21.15.33:0/469198256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7610063e40 msgr2=0x7f76100aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7610063e40 0x7f76100aad00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f7600011000 tx=0x7f7600017000).stop 2023-11-06T20:56:27.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 -- 172.21.15.33:0/469198256 shutdown_connections 2023-11-06T20:56:27.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75f405f2c0 0x7f75f4061780 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f761005e3c0 0x7f761013de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7610068c20 0x7f761014e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 --2- 172.21.15.33:0/469198256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7610063e40 0x7f76100aad00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 -- 172.21.15.33:0/469198256 >> 172.21.15.33:0/469198256 conn(0x7f76100c7d10 msgr2=0x7f7610150460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 -- 172.21.15.33:0/469198256 shutdown_connections 2023-11-06T20:56:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.689+0000 7f76175ff700 1 -- 172.21.15.33:0/469198256 wait complete. 2023-11-06T20:56:27.707 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:27", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:27.707 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T20:45:27', 'first': '2023-11-06T20:46:00', 'last': '2023-11-06T20:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-06T20:56:27.708 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T20:56:27.708 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T20:56:27.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:27.709 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-11-06T20:56:27.757 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:27.757 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:27.758 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:27.758 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:27.758 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:27.758 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:27.758 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:27.759 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:27.759 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:27.759 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:27.759 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:27.760 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-20_46_00_UTC', 'scheduled-2023-11-06-20_47_00_UTC', 'scheduled-2023-11-06-20_48_00_UTC', 'scheduled-2023-11-06-20_49_00_UTC', 'scheduled-2023-11-06-20_50_00_UTC', 'scheduled-2023-11-06-20_51_00_UTC', 'scheduled-2023-11-06-20_52_00_UTC', 'scheduled-2023-11-06-20_53_00_UTC', 'scheduled-2023-11-06-20_54_00_UTC', 'scheduled-2023-11-06-20_55_00_UTC', 'scheduled-2023-11-06-20_56_00_UTC'] 2023-11-06T20:56:27.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-11-06T20:56:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.960+0000 7f576bbff700 1 Processor -- start 2023-11-06T20:56:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.960+0000 7f576bbff700 1 -- start start 2023-11-06T20:56:27.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.961+0000 7f576bbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f57640c8ee0 0x7f57640ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576bbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57641542b0 0x7f57640c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576bbff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640c9f80 con 0x7f5764153ed0 2023-11-06T20:56:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576bbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57640c6000 con 0x7f57641542b0 2023-11-06T20:56:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576bbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57640c9370 con 0x7f57640c8ee0 2023-11-06T20:56:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576999b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5764153ed0 0x7f5764158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54948/0 (socket says 172.21.15.33:54948) 2023-11-06T20:56:27.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576999b700 1 -- 172.21.15.33:0/2459293838 learned_addr learned my addr 172.21.15.33:0/2459293838 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:27.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576a19c700 1 --2- 172.21.15.33:0/2459293838 >> v2:172.21.15.33:3300/0 conn(0x7f57641542b0 0x7f57640c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576919a700 1 --2- 172.21.15.33:0/2459293838 >> v2:172.21.15.69:3300/0 conn(0x7f57640c8ee0 0x7f57640ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f5768999700 1 -- 172.21.15.33:0/2459293838 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322681356 0 0) 0x7f57640c9f80 con 0x7f5764153ed0 2023-11-06T20:56:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f5768999700 1 -- 172.21.15.33:0/2459293838 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57500033f0 con 0x7f5764153ed0 2023-11-06T20:56:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576a19c700 1 -- 172.21.15.33:0/2459293838 >> v2:172.21.15.69:3300/0 conn(0x7f57640c8ee0 msgr2=0x7f57640ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576a19c700 1 --2- 172.21.15.33:0/2459293838 >> v2:172.21.15.69:3300/0 conn(0x7f57640c8ee0 0x7f57640ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576a19c700 1 -- 172.21.15.33:0/2459293838 >> v1:172.21.15.33:6789/0 conn(0x7f5764153ed0 legacy=0x7f5764158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.962+0000 7f576a19c700 1 -- 172.21.15.33:0/2459293838 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640c9650 con 0x7f57641542b0 2023-11-06T20:56:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.963+0000 7f576a19c700 1 --2- 172.21.15.33:0/2459293838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57641542b0 0x7f57640c76f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f57640c8c90 tx=0x7f575c009780).ready entity=mon.0 client_cookie=99f879eb28392a0e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:27.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.963+0000 7f5768999700 1 -- 172.21.15.33:0/2459293838 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c00e070 con 0x7f57641542b0 2023-11-06T20:56:27.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.963+0000 7f5768999700 1 -- 172.21.15.33:0/2459293838 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c00bab0 con 0x7f57641542b0 2023-11-06T20:56:27.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.963+0000 7f5768999700 1 -- 172.21.15.33:0/2459293838 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c00be30 con 0x7f57641542b0 2023-11-06T20:56:27.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.963+0000 7f576bbff700 1 -- 172.21.15.33:0/2459293838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57641542b0 msgr2=0x7f57640c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.963+0000 7f576bbff700 1 --2- 172.21.15.33:0/2459293838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57641542b0 0x7f57640c76f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f57640c8c90 tx=0x7f575c009780).stop 2023-11-06T20:56:27.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.963+0000 7f576bbff700 1 -- 172.21.15.33:0/2459293838 shutdown_connections 2023-11-06T20:56:27.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.963+0000 7f576bbff700 1 --2- 172.21.15.33:0/2459293838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57641542b0 0x7f57640c76f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.963+0000 7f576bbff700 1 --2- 172.21.15.33:0/2459293838 >> v2:172.21.15.69:3300/0 conn(0x7f57640c8ee0 0x7f57640ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.963+0000 7f576bbff700 1 -- 172.21.15.33:0/2459293838 >> 172.21.15.33:0/2459293838 conn(0x7f576405e1b0 msgr2=0x7f576405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:27.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.964+0000 7f576bbff700 1 -- 172.21.15.33:0/2459293838 shutdown_connections 2023-11-06T20:56:27.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.964+0000 7f576bbff700 1 -- 172.21.15.33:0/2459293838 wait complete. 2023-11-06T20:56:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.965+0000 7f576bbff700 1 Processor -- start 2023-11-06T20:56:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.965+0000 7f576bbff700 1 -- start start 2023-11-06T20:56:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.966+0000 7f576bbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5764153ed0 0x7f5764158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.966+0000 7f576bbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57641542b0 0x7f57640afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.966+0000 7f576bbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5764163600 0x7f57640ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.966+0000 7f576bbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640b0dd0 con 0x7f5764163600 2023-11-06T20:56:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.966+0000 7f576bbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640b02b0 con 0x7f57641542b0 2023-11-06T20:56:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.966+0000 7f576bbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640b0530 con 0x7f5764153ed0 2023-11-06T20:56:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.966+0000 7f576a19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5764163600 0x7f57640ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.966+0000 7f576a19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5764163600 0x7f57640ac5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42416/0 (socket says 172.21.15.33:42416) 2023-11-06T20:56:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.966+0000 7f576a19c700 1 -- 172.21.15.33:0/2793055721 learned_addr learned my addr 172.21.15.33:0/2793055721 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.966+0000 7f576999b700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5764153ed0 0x7f5764158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f576919a700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57641542b0 0x7f57640afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f576a19c700 1 -- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5764153ed0 msgr2=0x7f5764158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f576a19c700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5764153ed0 0x7f5764158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f576a19c700 1 -- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57641542b0 msgr2=0x7f57640afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f576a19c700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57641542b0 0x7f57640afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f576a19c700 1 -- 172.21.15.33:0/2793055721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c009a20 con 0x7f5764163600 2023-11-06T20:56:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f576a19c700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5764163600 0x7f57640ac5a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f575c011000 tx=0x7f575c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f5759ffb700 1 -- 172.21.15.33:0/2793055721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c013e60 con 0x7f5764163600 2023-11-06T20:56:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f5759ffb700 1 -- 172.21.15.33:0/2793055721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c01d630 con 0x7f5764163600 2023-11-06T20:56:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f576bbff700 1 -- 172.21.15.33:0/2793055721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57640b0810 con 0x7f5764163600 2023-11-06T20:56:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.967+0000 7f5759ffb700 1 -- 172.21.15.33:0/2793055721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c01d9d0 con 0x7f5764163600 2023-11-06T20:56:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.968+0000 7f576bbff700 1 -- 172.21.15.33:0/2793055721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57641547f0 con 0x7f5764163600 2023-11-06T20:56:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.969+0000 7f576bbff700 1 -- 172.21.15.33:0/2793055721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5764154b60 con 0x7f5764163600 2023-11-06T20:56:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.969+0000 7f5759ffb700 1 -- 172.21.15.33:0/2793055721 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f575c003040 con 0x7f5764163600 2023-11-06T20:56:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.970+0000 7f5759ffb700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f574805f2c0 0x7f5748061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.970+0000 7f5759ffb700 1 -- 172.21.15.33:0/2793055721 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f575c07c230 con 0x7f5764163600 2023-11-06T20:56:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.971+0000 7f576999b700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f574805f2c0 0x7f5748061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.971+0000 7f576999b700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f574805f2c0 0x7f5748061780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f5760005750 tx=0x7f5760009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:27.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:27.974+0000 7f5759ffb700 1 -- 172.21.15.33:0/2793055721 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f575c04d220 con 0x7f5764163600 2023-11-06T20:56:28.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.184+0000 7f576bbff700 1 -- 172.21.15.33:0/2793055721 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f576406c6c0 con 0x7f574805f2c0 2023-11-06T20:56:28.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.187+0000 7f5759ffb700 1 -- 172.21.15.33:0/2793055721 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f576406c6c0 con 0x7f574805f2c0 2023-11-06T20:56:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:28.188 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:28", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:28.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.188+0000 7f576bbff700 1 -- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f574805f2c0 msgr2=0x7f5748061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.188+0000 7f576bbff700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f574805f2c0 0x7f5748061780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f5760005750 tx=0x7f5760009040).stop 2023-11-06T20:56:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.189+0000 7f576bbff700 1 -- 172.21.15.33:0/2793055721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5764163600 msgr2=0x7f57640ac5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.189+0000 7f576bbff700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5764163600 0x7f57640ac5a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f575c011000 tx=0x7f575c009780).stop 2023-11-06T20:56:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.189+0000 7f576bbff700 1 -- 172.21.15.33:0/2793055721 shutdown_connections 2023-11-06T20:56:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.189+0000 7f576bbff700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f574805f2c0 0x7f5748061780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.189+0000 7f576bbff700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5764153ed0 0x7f5764158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.189+0000 7f576bbff700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5764163600 0x7f57640ac5a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.189+0000 7f576bbff700 1 --2- 172.21.15.33:0/2793055721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57641542b0 0x7f57640afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.189+0000 7f576bbff700 1 -- 172.21.15.33:0/2793055721 >> 172.21.15.33:0/2793055721 conn(0x7f576405e1b0 msgr2=0x7f57640c0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:28.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.189+0000 7f576bbff700 1 -- 172.21.15.33:0/2793055721 shutdown_connections 2023-11-06T20:56:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.189+0000 7f576bbff700 1 -- 172.21.15.33:0/2793055721 wait complete. 2023-11-06T20:56:28.207 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:28", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:28.207 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T20:45:28', 'first': '2023-11-06T20:46:00', 'last': '2023-11-06T20:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-06T20:56:28.207 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T20:56:28.208 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T20:56:28.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:28.209 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-11-06T20:56:28.258 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:28.258 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:28.258 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:28.259 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:28.259 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:28.259 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:28.259 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:28.260 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:28.261 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:28.261 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:28.261 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:28.262 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-20_46_00_UTC', 'scheduled-2023-11-06-20_47_00_UTC', 'scheduled-2023-11-06-20_48_00_UTC', 'scheduled-2023-11-06-20_49_00_UTC', 'scheduled-2023-11-06-20_50_00_UTC', 'scheduled-2023-11-06-20_51_00_UTC', 'scheduled-2023-11-06-20_52_00_UTC', 'scheduled-2023-11-06-20_53_00_UTC', 'scheduled-2023-11-06-20_54_00_UTC', 'scheduled-2023-11-06-20_55_00_UTC', 'scheduled-2023-11-06-20_56_00_UTC'] 2023-11-06T20:56:28.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-11-06T20:56:28.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.462+0000 7f7b80bff700 1 Processor -- start 2023-11-06T20:56:28.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.462+0000 7f7b80bff700 1 -- start start 2023-11-06T20:56:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.463+0000 7f7b80bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b7c0c8ee0 0x7f7b7c0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.463+0000 7f7b80bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7c0c9f80 con 0x7f7b7c1542b0 2023-11-06T20:56:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.464+0000 7f7b80bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7c0c6000 con 0x7f7b7c153ed0 2023-11-06T20:56:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.464+0000 7f7b80bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b7c0c9370 con 0x7f7b7c0c8ee0 2023-11-06T20:56:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.464+0000 7f7b79d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b7c0c8ee0 0x7f7b7c0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.464+0000 7f7b7ad9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7b7c1542b0 0x7f7b7c0c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54958/0 (socket says 172.21.15.33:54958) 2023-11-06T20:56:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.464+0000 7f7b7ad9d700 1 -- 172.21.15.33:0/454207251 learned_addr learned my addr 172.21.15.33:0/454207251 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.464+0000 7f7b79d9b700 1 -- 172.21.15.33:0/454207251 >> v1:172.21.15.69:6789/0 conn(0x7f7b7c153ed0 legacy=0x7f7b7c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.464+0000 7f7b79d9b700 1 -- 172.21.15.33:0/454207251 >> v1:172.21.15.33:6789/0 conn(0x7f7b7c1542b0 legacy=0x7f7b7c0c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.464+0000 7f7b7959a700 1 -- 172.21.15.33:0/454207251 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164770119 0 0) 0x7f7b7c0c9f80 con 0x7f7b7c1542b0 2023-11-06T20:56:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.464+0000 7f7b79d9b700 1 -- 172.21.15.33:0/454207251 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c0c9650 con 0x7f7b7c0c8ee0 2023-11-06T20:56:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b79d9b700 1 --2- 172.21.15.33:0/454207251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c0c8ee0 0x7f7b7c0ca410 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f7b70009760 tx=0x7f7b70009060).ready entity=mon.0 client_cookie=f01f3db3e59eba79 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b7959a700 1 -- 172.21.15.33:0/454207251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000b520 con 0x7f7b7c0c8ee0 2023-11-06T20:56:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b7959a700 1 -- 172.21.15.33:0/454207251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7000b680 con 0x7f7b7c0c8ee0 2023-11-06T20:56:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b7959a700 1 -- 172.21.15.33:0/454207251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000ba00 con 0x7f7b7c0c8ee0 2023-11-06T20:56:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b80bff700 1 -- 172.21.15.33:0/454207251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c0c8ee0 msgr2=0x7f7b7c0ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b80bff700 1 --2- 172.21.15.33:0/454207251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c0c8ee0 0x7f7b7c0ca410 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f7b70009760 tx=0x7f7b70009060).stop 2023-11-06T20:56:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b80bff700 1 -- 172.21.15.33:0/454207251 shutdown_connections 2023-11-06T20:56:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b80bff700 1 --2- 172.21.15.33:0/454207251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c0c8ee0 0x7f7b7c0ca410 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b80bff700 1 -- 172.21.15.33:0/454207251 >> 172.21.15.33:0/454207251 conn(0x7f7b7c05e1b0 msgr2=0x7f7b7c05e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b80bff700 1 -- 172.21.15.33:0/454207251 shutdown_connections 2023-11-06T20:56:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.465+0000 7f7b80bff700 1 -- 172.21.15.33:0/454207251 wait complete. 2023-11-06T20:56:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.467+0000 7f7b80bff700 1 Processor -- start 2023-11-06T20:56:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.467+0000 7f7b80bff700 1 -- start start 2023-11-06T20:56:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b80bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c153ed0 0x7f7b7c165940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b80bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b7c1542b0 0x7f7b7c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b80bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b7c164730 0x7f7b7c1674d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b80bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c1657b0 con 0x7f7b7c153ed0 2023-11-06T20:56:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b80bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c156000 con 0x7f7b7c164730 2023-11-06T20:56:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b80bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c156960 con 0x7f7b7c1542b0 2023-11-06T20:56:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b7a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c153ed0 0x7f7b7c165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b7a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c153ed0 0x7f7b7c165940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42430/0 (socket says 172.21.15.33:42430) 2023-11-06T20:56:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b7a59c700 1 -- 172.21.15.33:0/3129203049 learned_addr learned my addr 172.21.15.33:0/3129203049 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b79d9b700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b7c1542b0 0x7f7b7c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.468+0000 7f7b7ad9d700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b7c164730 0x7f7b7c1674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b7a59c700 1 -- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b7c1542b0 msgr2=0x7f7b7c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b7a59c700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b7c1542b0 0x7f7b7c155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b7a59c700 1 -- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b7c164730 msgr2=0x7f7b7c1674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b7a59c700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b7c164730 0x7f7b7c1674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b7a59c700 1 -- 172.21.15.33:0/3129203049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b700092a0 con 0x7f7b7c153ed0 2023-11-06T20:56:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b7a59c700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c153ed0 0x7f7b7c165940 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f7b6c00c440 tx=0x7f7b6c007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b6a7fc700 1 -- 172.21.15.33:0/3129203049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c012400 con 0x7f7b7c153ed0 2023-11-06T20:56:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b6a7fc700 1 -- 172.21.15.33:0/3129203049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c012580 con 0x7f7b7c153ed0 2023-11-06T20:56:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b6a7fc700 1 -- 172.21.15.33:0/3129203049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c012940 con 0x7f7b7c153ed0 2023-11-06T20:56:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b80bff700 1 -- 172.21.15.33:0/3129203049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7c164f10 con 0x7f7b7c153ed0 2023-11-06T20:56:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.469+0000 7f7b80bff700 1 -- 172.21.15.33:0/3129203049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7c1654d0 con 0x7f7b7c153ed0 2023-11-06T20:56:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.471+0000 7f7b80bff700 1 -- 172.21.15.33:0/3129203049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7c158b50 con 0x7f7b7c153ed0 2023-11-06T20:56:28.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.472+0000 7f7b6a7fc700 1 -- 172.21.15.33:0/3129203049 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f7b6c012d30 con 0x7f7b7c153ed0 2023-11-06T20:56:28.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.472+0000 7f7b6a7fc700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f2c0 0x7f7b5c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.473+0000 7f7b6a7fc700 1 -- 172.21.15.33:0/3129203049 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f7b6c0798f0 con 0x7f7b7c153ed0 2023-11-06T20:56:28.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.473+0000 7f7b79d9b700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f2c0 0x7f7b5c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.473+0000 7f7b79d9b700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f2c0 0x7f7b5c061780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7b7000b2e0 tx=0x7f7b70017000).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:28.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.476+0000 7f7b6a7fc700 1 -- 172.21.15.33:0/3129203049 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7b6c048af0 con 0x7f7b7c153ed0 2023-11-06T20:56:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.684+0000 7f7b80bff700 1 -- 172.21.15.33:0/3129203049 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b7c0c90e0 con 0x7f7b5c05f2c0 2023-11-06T20:56:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.686+0000 7f7b6a7fc700 1 -- 172.21.15.33:0/3129203049 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f7b7c0c90e0 con 0x7f7b5c05f2c0 2023-11-06T20:56:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:28.687 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:28", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 -- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f2c0 msgr2=0x7f7b5c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f2c0 0x7f7b5c061780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7b7000b2e0 tx=0x7f7b70017000).stop 2023-11-06T20:56:28.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 -- 172.21.15.33:0/3129203049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c153ed0 msgr2=0x7f7b7c165940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c153ed0 0x7f7b7c165940 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f7b6c00c440 tx=0x7f7b6c007de0).stop 2023-11-06T20:56:28.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 -- 172.21.15.33:0/3129203049 shutdown_connections 2023-11-06T20:56:28.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b5c05f2c0 0x7f7b5c061780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b7c164730 0x7f7b7c1674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b7c153ed0 0x7f7b7c165940 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 --2- 172.21.15.33:0/3129203049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b7c1542b0 0x7f7b7c155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 -- 172.21.15.33:0/3129203049 >> 172.21.15.33:0/3129203049 conn(0x7f7b7c05e1b0 msgr2=0x7f7b7c157db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:28.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 -- 172.21.15.33:0/3129203049 shutdown_connections 2023-11-06T20:56:28.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.688+0000 7f7b80bff700 1 -- 172.21.15.33:0/3129203049 wait complete. 2023-11-06T20:56:28.705 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:28", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:28.706 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T20:45:28', 'first': '2023-11-06T20:46:00', 'last': '2023-11-06T20:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-06T20:56:28.706 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T20:56:28.707 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T20:56:28.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:28.707 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-11-06T20:56:28.759 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:28.760 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:28.760 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:28.760 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:28.760 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:28.761 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:28.761 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:28.761 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:28.761 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:28.761 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:28.762 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:28.763 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-20_46_00_UTC', 'scheduled-2023-11-06-20_47_00_UTC', 'scheduled-2023-11-06-20_48_00_UTC', 'scheduled-2023-11-06-20_49_00_UTC', 'scheduled-2023-11-06-20_50_00_UTC', 'scheduled-2023-11-06-20_51_00_UTC', 'scheduled-2023-11-06-20_52_00_UTC', 'scheduled-2023-11-06-20_53_00_UTC', 'scheduled-2023-11-06-20_54_00_UTC', 'scheduled-2023-11-06-20_55_00_UTC', 'scheduled-2023-11-06-20_56_00_UTC'] 2023-11-06T20:56:28.763 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-11-06T20:56:28.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.964+0000 7f0447dff700 1 Processor -- start 2023-11-06T20:56:28.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.964+0000 7f0447dff700 1 -- start start 2023-11-06T20:56:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.966+0000 7f0447dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0440153ed0 0x7f0440158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.966+0000 7f0447dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04401542b0 0x7f04400c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.966+0000 7f0447dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04400c9f80 con 0x7f04400c8ee0 2023-11-06T20:56:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.966+0000 7f0447dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04400cab20 con 0x7f0440153ed0 2023-11-06T20:56:28.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.966+0000 7f0447dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04400c6000 con 0x7f04401542b0 2023-11-06T20:56:28.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.966+0000 7f044639c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04401542b0 0x7f04400c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.966+0000 7f0445b9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0440153ed0 0x7f0440158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.966+0000 7f044639c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04401542b0 0x7f04400c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:37094/0 (socket says 172.21.15.33:37094) 2023-11-06T20:56:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.966+0000 7f044639c700 1 -- 172.21.15.33:0/3659896752 learned_addr learned my addr 172.21.15.33:0/3659896752 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.966+0000 7f044539a700 1 --1- 172.21.15.33:0/3659896752 >> v1:172.21.15.69:6789/0 conn(0x7f04400c8ee0 0x7f04400ca410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:44566/0 (socket says 172.21.15.33:44566) 2023-11-06T20:56:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.967+0000 7f044639c700 1 -- 172.21.15.33:0/3659896752 >> v2:172.21.15.69:3300/0 conn(0x7f0440153ed0 msgr2=0x7f0440158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.967+0000 7f044639c700 1 --2- 172.21.15.33:0/3659896752 >> v2:172.21.15.69:3300/0 conn(0x7f0440153ed0 0x7f0440158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.967+0000 7f044639c700 1 -- 172.21.15.33:0/3659896752 >> v1:172.21.15.69:6789/0 conn(0x7f04400c8ee0 legacy=0x7f04400ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.967+0000 7f044639c700 1 -- 172.21.15.33:0/3659896752 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04400c9370 con 0x7f04401542b0 2023-11-06T20:56:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.967+0000 7f044639c700 1 --2- 172.21.15.33:0/3659896752 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04401542b0 0x7f04400c76f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f0438009f20 tx=0x7f0438009840).ready entity=mon.2 client_cookie=cfd60175169eb07f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.967+0000 7f0444b99700 1 -- 172.21.15.33:0/3659896752 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043800e070 con 0x7f04401542b0 2023-11-06T20:56:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.967+0000 7f0444b99700 1 -- 172.21.15.33:0/3659896752 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043800b9d0 con 0x7f04401542b0 2023-11-06T20:56:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.967+0000 7f0444b99700 1 -- 172.21.15.33:0/3659896752 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043800bd50 con 0x7f04401542b0 2023-11-06T20:56:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.968+0000 7f0447dff700 1 -- 172.21.15.33:0/3659896752 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04401542b0 msgr2=0x7f04400c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.968+0000 7f0447dff700 1 --2- 172.21.15.33:0/3659896752 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04401542b0 0x7f04400c76f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f0438009f20 tx=0x7f0438009840).stop 2023-11-06T20:56:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.968+0000 7f0447dff700 1 -- 172.21.15.33:0/3659896752 shutdown_connections 2023-11-06T20:56:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.968+0000 7f0447dff700 1 --2- 172.21.15.33:0/3659896752 >> v2:172.21.15.69:3300/0 conn(0x7f0440153ed0 0x7f0440158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.968+0000 7f0447dff700 1 --2- 172.21.15.33:0/3659896752 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04401542b0 0x7f04400c76f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.968+0000 7f0447dff700 1 -- 172.21.15.33:0/3659896752 >> 172.21.15.33:0/3659896752 conn(0x7f044005e1b0 msgr2=0x7f044005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.968+0000 7f0447dff700 1 -- 172.21.15.33:0/3659896752 shutdown_connections 2023-11-06T20:56:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.968+0000 7f0447dff700 1 -- 172.21.15.33:0/3659896752 wait complete. 2023-11-06T20:56:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.969+0000 7f0447dff700 1 Processor -- start 2023-11-06T20:56:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.969+0000 7f0447dff700 1 -- start start 2023-11-06T20:56:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.970+0000 7f0447dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0440153ed0 0x7f04400b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.970+0000 7f0447dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04401542b0 0x7f04400addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.970+0000 7f0447dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0440158090 0x7f04401584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.970+0000 7f0447dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04400ae850 con 0x7f04401542b0 2023-11-06T20:56:28.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.970+0000 7f0447dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04400c3470 con 0x7f0440158090 2023-11-06T20:56:28.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.970+0000 7f0447dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04400c3750 con 0x7f0440153ed0 2023-11-06T20:56:28.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f044539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04401542b0 0x7f04400addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f044539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04401542b0 0x7f04400addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42442/0 (socket says 172.21.15.33:42442) 2023-11-06T20:56:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f044539a700 1 -- 172.21.15.33:0/2590463065 learned_addr learned my addr 172.21.15.33:0/2590463065 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f0445b9b700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0440153ed0 0x7f04400b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f044639c700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0440158090 0x7f04401584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f044539a700 1 -- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0440153ed0 msgr2=0x7f04400b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f044539a700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0440153ed0 0x7f04400b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f044539a700 1 -- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0440158090 msgr2=0x7f04401584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f044539a700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0440158090 0x7f04401584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f044539a700 1 -- 172.21.15.33:0/2590463065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0438009ae0 con 0x7f04401542b0 2023-11-06T20:56:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f0445b9b700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0440153ed0 0x7f04400b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f044539a700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04401542b0 0x7f04400addd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f0430001eb0 tx=0x7f0430007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.971+0000 7f0435ffb700 1 -- 172.21.15.33:0/2590463065 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300070d0 con 0x7f04401542b0 2023-11-06T20:56:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.972+0000 7f0435ffb700 1 -- 172.21.15.33:0/2590463065 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0430007230 con 0x7f04401542b0 2023-11-06T20:56:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.972+0000 7f0435ffb700 1 -- 172.21.15.33:0/2590463065 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300075b0 con 0x7f04401542b0 2023-11-06T20:56:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.972+0000 7f0447dff700 1 -- 172.21.15.33:0/2590463065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04400c3a30 con 0x7f04401542b0 2023-11-06T20:56:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.972+0000 7f0447dff700 1 -- 172.21.15.33:0/2590463065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04401589a0 con 0x7f04401542b0 2023-11-06T20:56:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.973+0000 7f0447dff700 1 -- 172.21.15.33:0/2590463065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04400ae620 con 0x7f04401542b0 2023-11-06T20:56:28.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.973+0000 7f0435ffb700 1 -- 172.21.15.33:0/2590463065 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f04300058f0 con 0x7f04401542b0 2023-11-06T20:56:28.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.974+0000 7f0435ffb700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f042405f2c0 0x7f0424061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.974+0000 7f0435ffb700 1 -- 172.21.15.33:0/2590463065 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f0430079b60 con 0x7f04401542b0 2023-11-06T20:56:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.975+0000 7f0445b9b700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f042405f2c0 0x7f0424061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.975+0000 7f0445b9b700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f042405f2c0 0x7f0424061780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f043c011310 tx=0x7f043c009060).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:28.978+0000 7f0435ffb700 1 -- 172.21.15.33:0/2590463065 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f04400ae620 con 0x7f04401542b0 2023-11-06T20:56:29.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.182+0000 7f0447dff700 1 -- 172.21.15.33:0/2590463065 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04400ae620 con 0x7f042405f2c0 2023-11-06T20:56:29.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.184+0000 7f0435ffb700 1 -- 172.21.15.33:0/2590463065 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f04400ae620 con 0x7f042405f2c0 2023-11-06T20:56:29.184 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:29.184 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:29", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:29.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 -- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f042405f2c0 msgr2=0x7f0424061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f042405f2c0 0x7f0424061780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f043c011310 tx=0x7f043c009060).stop 2023-11-06T20:56:29.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 -- 172.21.15.33:0/2590463065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04401542b0 msgr2=0x7f04400addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04401542b0 0x7f04400addd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f0430001eb0 tx=0x7f0430007df0).stop 2023-11-06T20:56:29.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 -- 172.21.15.33:0/2590463065 shutdown_connections 2023-11-06T20:56:29.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f042405f2c0 0x7f0424061780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0440153ed0 0x7f04400b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0440158090 0x7f04401584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 --2- 172.21.15.33:0/2590463065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04401542b0 0x7f04400addd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 -- 172.21.15.33:0/2590463065 >> 172.21.15.33:0/2590463065 conn(0x7f044005e1b0 msgr2=0x7f04400aabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 -- 172.21.15.33:0/2590463065 shutdown_connections 2023-11-06T20:56:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.186+0000 7f0447dff700 1 -- 172.21.15.33:0/2590463065 wait complete. 2023-11-06T20:56:29.204 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:29", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:29.204 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T20:45:29', 'first': '2023-11-06T20:46:00', 'last': '2023-11-06T20:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-06T20:56:29.204 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T20:56:29.205 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T20:56:29.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:29.206 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-11-06T20:56:29.255 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:29.255 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:29.255 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:29.255 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:29.256 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:29.256 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:29.256 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:29.256 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:29.256 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:29.257 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:29.257 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:29.258 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-20_46_00_UTC', 'scheduled-2023-11-06-20_47_00_UTC', 'scheduled-2023-11-06-20_48_00_UTC', 'scheduled-2023-11-06-20_49_00_UTC', 'scheduled-2023-11-06-20_50_00_UTC', 'scheduled-2023-11-06-20_51_00_UTC', 'scheduled-2023-11-06-20_52_00_UTC', 'scheduled-2023-11-06-20_53_00_UTC', 'scheduled-2023-11-06-20_54_00_UTC', 'scheduled-2023-11-06-20_55_00_UTC', 'scheduled-2023-11-06-20_56_00_UTC'] 2023-11-06T20:56:29.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-11-06T20:56:29.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.459+0000 7f5f5d9ff700 1 Processor -- start 2023-11-06T20:56:29.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.459+0000 7f5f5d9ff700 1 -- start start 2023-11-06T20:56:29.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.460+0000 7f5f5d9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5f580a8c10 0x7f5f58153460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.460+0000 7f5f5d9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f580a5de0 0x7f5f580c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.460+0000 7f5f5d9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f580c3dd0 con 0x7f5f580c3940 2023-11-06T20:56:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.460+0000 7f5f5d9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f58153950 con 0x7f5f580a5de0 2023-11-06T20:56:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.460+0000 7f5f5d9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f580bd490 con 0x7f5f580a8c10 2023-11-06T20:56:29.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.460+0000 7f5f567fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f580a5de0 0x7f5f580c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.461+0000 7f5f567fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f580a5de0 0x7f5f580c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42450/0 (socket says 172.21.15.33:42450) 2023-11-06T20:56:29.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.461+0000 7f5f567fc700 1 -- 172.21.15.33:0/1337733850 learned_addr learned my addr 172.21.15.33:0/1337733850 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:29.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.461+0000 7f5f56ffd700 1 --2- 172.21.15.33:0/1337733850 >> v2:172.21.15.69:3300/0 conn(0x7f5f580a8c10 0x7f5f58153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.461+0000 7f5f567fc700 1 -- 172.21.15.33:0/1337733850 >> v2:172.21.15.69:3300/0 conn(0x7f5f580a8c10 msgr2=0x7f5f58153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:29.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.461+0000 7f5f567fc700 1 --2- 172.21.15.33:0/1337733850 >> v2:172.21.15.69:3300/0 conn(0x7f5f580a8c10 0x7f5f58153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.461+0000 7f5f567fc700 1 -- 172.21.15.33:0/1337733850 >> v1:172.21.15.69:6789/0 conn(0x7f5f580c3940 legacy=0x7f5f58155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:29.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.461+0000 7f5f567fc700 1 -- 172.21.15.33:0/1337733850 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f580bd770 con 0x7f5f580a5de0 2023-11-06T20:56:29.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.461+0000 7f5f56ffd700 1 --2- 172.21.15.33:0/1337733850 >> v2:172.21.15.69:3300/0 conn(0x7f5f580a8c10 0x7f5f58153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:29.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.461+0000 7f5f567fc700 1 --2- 172.21.15.33:0/1337733850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a5de0 0x7f5f580c3450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f5f5806be70 tx=0x7f5f4c0090f0).ready entity=mon.0 client_cookie=55e2090cfdf9a373 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.461+0000 7f5f55ffb700 1 -- 172.21.15.33:0/1337733850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00b5a0 con 0x7f5f580a5de0 2023-11-06T20:56:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.462+0000 7f5f55ffb700 1 -- 172.21.15.33:0/1337733850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c00b700 con 0x7f5f580a5de0 2023-11-06T20:56:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.462+0000 7f5f55ffb700 1 -- 172.21.15.33:0/1337733850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00ba80 con 0x7f5f580a5de0 2023-11-06T20:56:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.462+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/1337733850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a5de0 msgr2=0x7f5f580c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.462+0000 7f5f5d9ff700 1 --2- 172.21.15.33:0/1337733850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a5de0 0x7f5f580c3450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f5f5806be70 tx=0x7f5f4c0090f0).stop 2023-11-06T20:56:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.462+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/1337733850 shutdown_connections 2023-11-06T20:56:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.462+0000 7f5f5d9ff700 1 --2- 172.21.15.33:0/1337733850 >> v2:172.21.15.69:3300/0 conn(0x7f5f580a8c10 0x7f5f58153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.462+0000 7f5f5d9ff700 1 --2- 172.21.15.33:0/1337733850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a5de0 0x7f5f580c3450 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.462+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/1337733850 >> 172.21.15.33:0/1337733850 conn(0x7f5f58062710 msgr2=0x7f5f58062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.462+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/1337733850 shutdown_connections 2023-11-06T20:56:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.462+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/1337733850 wait complete. 2023-11-06T20:56:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.464+0000 7f5f5d9ff700 1 Processor -- start 2023-11-06T20:56:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.464+0000 7f5f5d9ff700 1 -- start start 2023-11-06T20:56:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f5d9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f580a5de0 0x7f5f58159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f5d9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a8c10 0x7f5f5814bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f5d9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5f580c3940 0x7f5f581655f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f5d9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5815a4a0 con 0x7f5f580a8c10 2023-11-06T20:56:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f5d9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58164be0 con 0x7f5f580a5de0 2023-11-06T20:56:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f5d9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58165160 con 0x7f5f580c3940 2023-11-06T20:56:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f567fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a8c10 0x7f5f5814bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f567fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a8c10 0x7f5f5814bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42462/0 (socket says 172.21.15.33:42462) 2023-11-06T20:56:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f567fc700 1 -- 172.21.15.33:0/444143262 learned_addr learned my addr 172.21.15.33:0/444143262 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:29.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f56ffd700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f580a5de0 0x7f5f58159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f577fe700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5f580c3940 0x7f5f581655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f567fc700 1 -- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5f580c3940 msgr2=0x7f5f581655f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f567fc700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5f580c3940 0x7f5f581655f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f567fc700 1 -- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f580a5de0 msgr2=0x7f5f58159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f567fc700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f580a5de0 0x7f5f58159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f567fc700 1 -- 172.21.15.33:0/444143262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c009370 con 0x7f5f580a8c10 2023-11-06T20:56:29.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f56ffd700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f580a5de0 0x7f5f58159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:29.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.465+0000 7f5f577fe700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5f580c3940 0x7f5f581655f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:29.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.466+0000 7f5f567fc700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a8c10 0x7f5f5814bb40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f5f4c009130 tx=0x7f5f4c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:29.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.466+0000 7f5f3effd700 1 -- 172.21.15.33:0/444143262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00ba80 con 0x7f5f580a8c10 2023-11-06T20:56:29.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.466+0000 7f5f3effd700 1 -- 172.21.15.33:0/444143262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c000d10 con 0x7f5f580a8c10 2023-11-06T20:56:29.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.466+0000 7f5f3effd700 1 -- 172.21.15.33:0/444143262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c01c670 con 0x7f5f580a8c10 2023-11-06T20:56:29.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.466+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/444143262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5814c030 con 0x7f5f580a8c10 2023-11-06T20:56:29.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.466+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/444143262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5815a1c0 con 0x7f5f580a8c10 2023-11-06T20:56:29.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.467+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/444143262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f58165310 con 0x7f5f580a8c10 2023-11-06T20:56:29.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.468+0000 7f5f3effd700 1 -- 172.21.15.33:0/444143262 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f5f4c01c9e0 con 0x7f5f580a8c10 2023-11-06T20:56:29.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.469+0000 7f5f3effd700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5f4405f290 0x7f5f44061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.469+0000 7f5f3effd700 1 -- 172.21.15.33:0/444143262 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f5f4c07c430 con 0x7f5f580a8c10 2023-11-06T20:56:29.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.469+0000 7f5f56ffd700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5f4405f290 0x7f5f44061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.470+0000 7f5f56ffd700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5f4405f290 0x7f5f44061750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f5f48008990 tx=0x7f5f48008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:29.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.473+0000 7f5f3effd700 1 -- 172.21.15.33:0/444143262 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5f58165310 con 0x7f5f580a8c10 2023-11-06T20:56:29.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.678+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/444143262 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f581652d0 con 0x7f5f4405f290 2023-11-06T20:56:29.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.681+0000 7f5f3effd700 1 -- 172.21.15.33:0/444143262 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f5f581652d0 con 0x7f5f4405f290 2023-11-06T20:56:29.681 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:29.681 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:29", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:29.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.682+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5f4405f290 msgr2=0x7f5f44061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.682+0000 7f5f5d9ff700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5f4405f290 0x7f5f44061750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f5f48008990 tx=0x7f5f48008040).stop 2023-11-06T20:56:29.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.682+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/444143262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a8c10 msgr2=0x7f5f5814bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.682+0000 7f5f5d9ff700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a8c10 0x7f5f5814bb40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f5f4c009130 tx=0x7f5f4c015000).stop 2023-11-06T20:56:29.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.682+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/444143262 shutdown_connections 2023-11-06T20:56:29.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.682+0000 7f5f5d9ff700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5f4405f290 0x7f5f44061750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.682+0000 7f5f5d9ff700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5f580c3940 0x7f5f581655f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.682+0000 7f5f5d9ff700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f580a5de0 0x7f5f58159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.682+0000 7f5f5d9ff700 1 --2- 172.21.15.33:0/444143262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f580a8c10 0x7f5f5814bb40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.682+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/444143262 >> 172.21.15.33:0/444143262 conn(0x7f5f58062710 msgr2=0x7f5f580c55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:29.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.683+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/444143262 shutdown_connections 2023-11-06T20:56:29.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.683+0000 7f5f5d9ff700 1 -- 172.21.15.33:0/444143262 wait complete. 2023-11-06T20:56:29.700 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T20:45:29", "first": "2023-11-06T20:46:00", "last": "2023-11-06T20:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-06T20:56:29.700 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T20:45:29', 'first': '2023-11-06T20:46:00', 'last': '2023-11-06T20:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-06T20:56:29.701 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T20:56:29.701 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T20:56:29.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-11-06T20:56:29.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.901+0000 7f6de6dff700 1 Processor -- start 2023-11-06T20:56:29.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.901+0000 7f6de6dff700 1 -- start start 2023-11-06T20:56:29.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.903+0000 7f6de6dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154340 0x7f6de00aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.903+0000 7f6de6dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6de0158720 0x7f6de0063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.903+0000 7f6de6dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de00c0e00 con 0x7f6de0154720 2023-11-06T20:56:29.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.903+0000 7f6de6dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de0158bf0 con 0x7f6de0158720 2023-11-06T20:56:29.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.903+0000 7f6de6dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0158ed0 con 0x7f6de0154340 2023-11-06T20:56:29.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.904+0000 7f6de539c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6de0158720 0x7f6de0063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.904+0000 7f6de4b9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154340 0x7f6de00aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.904+0000 7f6de539c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6de0158720 0x7f6de0063200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:38680/0 (socket says 172.21.15.33:38680) 2023-11-06T20:56:29.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.904+0000 7f6de539c700 1 -- 172.21.15.33:0/35091668 learned_addr learned my addr 172.21.15.33:0/35091668 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:29.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.904+0000 7f6de4b9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154340 0x7f6de00aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:37120/0 (socket says 172.21.15.33:37120) 2023-11-06T20:56:29.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.904+0000 7f6ddffff700 1 --1- 172.21.15.33:0/35091668 >> v1:172.21.15.69:6789/0 conn(0x7f6de0154720 0x7f6de0158010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:44582/0 (socket says 172.21.15.33:44582) 2023-11-06T20:56:29.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.904+0000 7f6de539c700 1 -- 172.21.15.33:0/35091668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154340 msgr2=0x7f6de00aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:29.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.904+0000 7f6de539c700 1 --2- 172.21.15.33:0/35091668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154340 0x7f6de00aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.904+0000 7f6de539c700 1 -- 172.21.15.33:0/35091668 >> v1:172.21.15.69:6789/0 conn(0x7f6de0154720 legacy=0x7f6de0158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.905+0000 7f6de539c700 1 -- 172.21.15.33:0/35091668 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de00c5d80 con 0x7f6de0158720 2023-11-06T20:56:29.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.904+0000 7f6ddf7fe700 1 -- 172.21.15.33:0/35091668 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096881017 0 0) 0x7f6de00c0e00 con 0x7f6de0154720 2023-11-06T20:56:29.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.905+0000 7f6de4b9b700 1 --2- 172.21.15.33:0/35091668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154340 0x7f6de00aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:29.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.905+0000 7f6de539c700 1 --2- 172.21.15.33:0/35091668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6de0158720 0x7f6de0063200 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f6dd8009f20 tx=0x7f6dd8009840).ready entity=mon.1 client_cookie=3d897699216d974b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:29.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.905+0000 7f6ddf7fe700 1 -- 172.21.15.33:0/35091668 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800e070 con 0x7f6de0158720 2023-11-06T20:56:29.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.905+0000 7f6ddf7fe700 1 -- 172.21.15.33:0/35091668 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd800b9d0 con 0x7f6de0158720 2023-11-06T20:56:29.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.905+0000 7f6ddf7fe700 1 -- 172.21.15.33:0/35091668 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800bd50 con 0x7f6de0158720 2023-11-06T20:56:29.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.906+0000 7f6de6dff700 1 -- 172.21.15.33:0/35091668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6de0158720 msgr2=0x7f6de0063200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.906+0000 7f6de6dff700 1 --2- 172.21.15.33:0/35091668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6de0158720 0x7f6de0063200 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f6dd8009f20 tx=0x7f6dd8009840).stop 2023-11-06T20:56:29.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.906+0000 7f6de6dff700 1 -- 172.21.15.33:0/35091668 shutdown_connections 2023-11-06T20:56:29.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.906+0000 7f6de6dff700 1 --2- 172.21.15.33:0/35091668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6de0158720 0x7f6de0063200 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.906+0000 7f6de6dff700 1 --2- 172.21.15.33:0/35091668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154340 0x7f6de00aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.906+0000 7f6de6dff700 1 -- 172.21.15.33:0/35091668 >> 172.21.15.33:0/35091668 conn(0x7f6de005ec80 msgr2=0x7f6de00cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:29.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.906+0000 7f6de6dff700 1 -- 172.21.15.33:0/35091668 shutdown_connections 2023-11-06T20:56:29.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.906+0000 7f6de6dff700 1 -- 172.21.15.33:0/35091668 wait complete. 2023-11-06T20:56:29.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.907+0000 7f6de6dff700 1 Processor -- start 2023-11-06T20:56:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.907+0000 7f6de6dff700 1 -- start start 2023-11-06T20:56:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.908+0000 7f6de6dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6de0154340 0x7f6de014bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.908+0000 7f6de6dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154720 0x7f6de00abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.908+0000 7f6de6dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de0158720 0x7f6de00c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.908+0000 7f6de6dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0155170 con 0x7f6de0158720 2023-11-06T20:56:29.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.908+0000 7f6de6dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01556f0 con 0x7f6de0154340 2023-11-06T20:56:29.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.908+0000 7f6de6dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00acd40 con 0x7f6de0154720 2023-11-06T20:56:29.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6de539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de0158720 0x7f6de00c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6de539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de0158720 0x7f6de00c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42474/0 (socket says 172.21.15.33:42474) 2023-11-06T20:56:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6de539c700 1 -- 172.21.15.33:0/1479342748 learned_addr learned my addr 172.21.15.33:0/1479342748 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6ddffff700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154720 0x7f6de00abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6de4b9b700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6de0154340 0x7f6de014bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6de539c700 1 -- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154720 msgr2=0x7f6de00abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6de539c700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154720 0x7f6de00abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6de539c700 1 -- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6de0154340 msgr2=0x7f6de014bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6de539c700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6de0154340 0x7f6de014bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6de539c700 1 -- 172.21.15.33:0/1479342748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd40092a0 con 0x7f6de0158720 2023-11-06T20:56:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.909+0000 7f6de539c700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de0158720 0x7f6de00c9f10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f6dd80072d0 tx=0x7f6dd8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.910+0000 7f6ddcff9700 1 -- 172.21.15.33:0/1479342748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800b640 con 0x7f6de0158720 2023-11-06T20:56:29.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.910+0000 7f6ddcff9700 1 -- 172.21.15.33:0/1479342748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd8013e70 con 0x7f6de0158720 2023-11-06T20:56:29.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.910+0000 7f6ddcff9700 1 -- 172.21.15.33:0/1479342748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8007820 con 0x7f6de0158720 2023-11-06T20:56:29.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.910+0000 7f6de6dff700 1 -- 172.21.15.33:0/1479342748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd8009ae0 con 0x7f6de0158720 2023-11-06T20:56:29.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.910+0000 7f6de6dff700 1 -- 172.21.15.33:0/1479342748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de00aca60 con 0x7f6de0158720 2023-11-06T20:56:29.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.911+0000 7f6de6dff700 1 -- 172.21.15.33:0/1479342748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de0155880 con 0x7f6de0158720 2023-11-06T20:56:29.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.912+0000 7f6ddcff9700 1 -- 172.21.15.33:0/1479342748 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f6dd8003040 con 0x7f6de0158720 2023-11-06T20:56:29.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.913+0000 7f6ddcff9700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dc405f2c0 0x7f6dc4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:29.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.913+0000 7f6ddcff9700 1 -- 172.21.15.33:0/1479342748 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f6dd807bd30 con 0x7f6de0158720 2023-11-06T20:56:29.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.913+0000 7f6de4b9b700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dc405f2c0 0x7f6dc4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:29.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.914+0000 7f6de4b9b700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dc405f2c0 0x7f6dc4061780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f6dd4009dc0 tx=0x7f6dd4009060).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:29.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:29.917+0000 7f6ddcff9700 1 -- 172.21.15.33:0/1479342748 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6dd804cdd0 con 0x7f6de0158720 2023-11-06T20:56:30.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.126+0000 7f6de6dff700 1 -- 172.21.15.33:0/1479342748 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6de00cbcf0 con 0x7f6dc405f2c0 2023-11-06T20:56:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.131+0000 7f6ddcff9700 1 -- 172.21.15.33:0/1479342748 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f6de00cbcf0 con 0x7f6dc405f2c0 2023-11-06T20:56:30.131 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-11-06T20:56:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 -- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dc405f2c0 msgr2=0x7f6dc4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dc405f2c0 0x7f6dc4061780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f6dd4009dc0 tx=0x7f6dd4009060).stop 2023-11-06T20:56:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 -- 172.21.15.33:0/1479342748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de0158720 msgr2=0x7f6de00c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de0158720 0x7f6de00c9f10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f6dd80072d0 tx=0x7f6dd8009840).stop 2023-11-06T20:56:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 -- 172.21.15.33:0/1479342748 shutdown_connections 2023-11-06T20:56:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dc405f2c0 0x7f6dc4061780 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6de0158720 0x7f6de00c9f10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6de0154340 0x7f6de014bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 --2- 172.21.15.33:0/1479342748 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6de0154720 0x7f6de00abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 -- 172.21.15.33:0/1479342748 >> 172.21.15.33:0/1479342748 conn(0x7f6de005ec80 msgr2=0x7f6de0152250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 -- 172.21.15.33:0/1479342748 shutdown_connections 2023-11-06T20:56:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:30.133+0000 7f6de6dff700 1 -- 172.21.15.33:0/1479342748 wait complete. 2023-11-06T20:56:30.151 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-11-06T20:56:30.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.152 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-11-06T20:56:30.200 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:30.200 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:30.200 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:30.200 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:30.201 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:30.201 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:30.201 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:30.201 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:30.202 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:30.202 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:30.202 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:30.203 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:30.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.204 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_46_00_UTC) 2023-11-06T20:56:30.261 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:30.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.262 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_47_00_UTC) 2023-11-06T20:56:30.318 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:30.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.320 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_48_00_UTC) 2023-11-06T20:56:30.375 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:30.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.375 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_49_00_UTC) 2023-11-06T20:56:30.431 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:30.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.432 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_50_00_UTC) 2023-11-06T20:56:30.488 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:30.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.488 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_51_00_UTC) 2023-11-06T20:56:30.544 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:30.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.545 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_52_00_UTC) 2023-11-06T20:56:30.602 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:30.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.603 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_53_00_UTC) 2023-11-06T20:56:30.660 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:30.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.661 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_54_00_UTC) 2023-11-06T20:56:30.716 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:30.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.717 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_55_00_UTC) 2023-11-06T20:56:30.772 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:30.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.773 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-06-20_56_00_UTC) 2023-11-06T20:56:30.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:30.828 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-11-06T20:56:30.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2023-11-06T20:56:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.080+0000 7fe015bff700 1 Processor -- start 2023-11-06T20:56:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.081+0000 7fe015bff700 1 -- start start 2023-11-06T20:56:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.082+0000 7fe015bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe0101545e0 0x7fe010062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.082+0000 7fe015bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe010155420 0x7fe010062c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.082+0000 7fe015bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe010069c90 con 0x7fe01006a120 2023-11-06T20:56:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.082+0000 7fe015bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0100630f0 con 0x7fe010155420 2023-11-06T20:56:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.082+0000 7fe015bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0101549c0 con 0x7fe0101545e0 2023-11-06T20:56:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.082+0000 7fe00effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe01006a120 0x7fe010154d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54974/0 (socket says 172.21.15.33:54974) 2023-11-06T20:56:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.082+0000 7fe00effd700 1 -- 172.21.15.33:0/1515275824 learned_addr learned my addr 172.21.15.33:0/1515275824 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.082+0000 7fe00ffff700 1 --2- 172.21.15.33:0/1515275824 >> v2:172.21.15.33:3300/0 conn(0x7fe010155420 0x7fe010062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.082+0000 7fe00f7fe700 1 --2- 172.21.15.33:0/1515275824 >> v2:172.21.15.69:3300/0 conn(0x7fe0101545e0 0x7fe010062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.083+0000 7fe00e7fc700 1 -- 172.21.15.33:0/1515275824 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455794165 0 0) 0x7fe010069c90 con 0x7fe01006a120 2023-11-06T20:56:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.083+0000 7fe00e7fc700 1 -- 172.21.15.33:0/1515275824 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff80033f0 con 0x7fe01006a120 2023-11-06T20:56:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.083+0000 7fe00ffff700 1 -- 172.21.15.33:0/1515275824 >> v2:172.21.15.69:3300/0 conn(0x7fe0101545e0 msgr2=0x7fe010062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.083+0000 7fe00ffff700 1 --2- 172.21.15.33:0/1515275824 >> v2:172.21.15.69:3300/0 conn(0x7fe0101545e0 0x7fe010062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.083+0000 7fe00ffff700 1 -- 172.21.15.33:0/1515275824 >> v1:172.21.15.33:6789/0 conn(0x7fe01006a120 legacy=0x7fe010154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.083+0000 7fe00ffff700 1 -- 172.21.15.33:0/1515275824 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe010155aa0 con 0x7fe010155420 2023-11-06T20:56:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.083+0000 7fe00ffff700 1 --2- 172.21.15.33:0/1515275824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 0x7fe010062c00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fe0100c5640 tx=0x7fe008009780).ready entity=mon.0 client_cookie=6463bbb90de59db2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.083+0000 7fe00e7fc700 1 -- 172.21.15.33:0/1515275824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800e070 con 0x7fe010155420 2023-11-06T20:56:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.083+0000 7fe00e7fc700 1 -- 172.21.15.33:0/1515275824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00800bab0 con 0x7fe010155420 2023-11-06T20:56:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.083+0000 7fe00e7fc700 1 -- 172.21.15.33:0/1515275824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800be30 con 0x7fe010155420 2023-11-06T20:56:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.084+0000 7fe015bff700 1 -- 172.21.15.33:0/1515275824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 msgr2=0x7fe010062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.084+0000 7fe015bff700 1 --2- 172.21.15.33:0/1515275824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 0x7fe010062c00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fe0100c5640 tx=0x7fe008009780).stop 2023-11-06T20:56:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.084+0000 7fe015bff700 1 -- 172.21.15.33:0/1515275824 shutdown_connections 2023-11-06T20:56:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.084+0000 7fe015bff700 1 --2- 172.21.15.33:0/1515275824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 0x7fe010062c00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.084+0000 7fe015bff700 1 --2- 172.21.15.33:0/1515275824 >> v2:172.21.15.69:3300/0 conn(0x7fe0101545e0 0x7fe010062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.084+0000 7fe015bff700 1 -- 172.21.15.33:0/1515275824 >> 172.21.15.33:0/1515275824 conn(0x7fe0100c7b30 msgr2=0x7fe0100c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:31.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.084+0000 7fe015bff700 1 -- 172.21.15.33:0/1515275824 shutdown_connections 2023-11-06T20:56:31.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.084+0000 7fe015bff700 1 -- 172.21.15.33:0/1515275824 wait complete. 2023-11-06T20:56:31.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.085+0000 7fe015bff700 1 Processor -- start 2023-11-06T20:56:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.085+0000 7fe015bff700 1 -- start start 2023-11-06T20:56:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.086+0000 7fe015bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01006a120 0x7fe0100a7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.086+0000 7fe015bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0101545e0 0x7fe0100ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.086+0000 7fe015bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 0x7fe010167ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.086+0000 7fe015bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe010159480 con 0x7fe010155420 2023-11-06T20:56:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.086+0000 7fe015bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100a7950 con 0x7fe01006a120 2023-11-06T20:56:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.086+0000 7fe015bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe010158b50 con 0x7fe0101545e0 2023-11-06T20:56:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 0x7fe010167ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 0x7fe010167ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42490/0 (socket says 172.21.15.33:42490) 2023-11-06T20:56:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00ffff700 1 -- 172.21.15.33:0/1496052683 learned_addr learned my addr 172.21.15.33:0/1496052683 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00f7fe700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01006a120 0x7fe0100a7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00effd700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0101545e0 0x7fe0100ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00ffff700 1 -- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0101545e0 msgr2=0x7fe0100ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00ffff700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0101545e0 0x7fe0100ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00ffff700 1 -- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01006a120 msgr2=0x7fe0100a7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00ffff700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01006a120 0x7fe0100a7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00ffff700 1 -- 172.21.15.33:0/1496052683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008009a20 con 0x7fe010155420 2023-11-06T20:56:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.087+0000 7fe00ffff700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 0x7fe010167ef0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fe00800b610 tx=0x7fe008009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.088+0000 7fdff77fe700 1 -- 172.21.15.33:0/1496052683 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800b640 con 0x7fe010155420 2023-11-06T20:56:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.088+0000 7fdff77fe700 1 -- 172.21.15.33:0/1496052683 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00801d630 con 0x7fe010155420 2023-11-06T20:56:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.088+0000 7fdff77fe700 1 -- 172.21.15.33:0/1496052683 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00801d9d0 con 0x7fe010155420 2023-11-06T20:56:31.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.088+0000 7fe015bff700 1 -- 172.21.15.33:0/1496052683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe010158e30 con 0x7fe010155420 2023-11-06T20:56:31.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.088+0000 7fe015bff700 1 -- 172.21.15.33:0/1496052683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0100aa560 con 0x7fe010155420 2023-11-06T20:56:31.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.089+0000 7fe015bff700 1 -- 172.21.15.33:0/1496052683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0101595f0 con 0x7fe010155420 2023-11-06T20:56:31.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.090+0000 7fdff77fe700 1 -- 172.21.15.33:0/1496052683 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fe008026410 con 0x7fe010155420 2023-11-06T20:56:31.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.091+0000 7fdff77fe700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdffc05f2c0 0x7fdffc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:31.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.091+0000 7fdff77fe700 1 -- 172.21.15.33:0/1496052683 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fe00807c290 con 0x7fe010155420 2023-11-06T20:56:31.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.091+0000 7fe00f7fe700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdffc05f2c0 0x7fdffc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:31.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.092+0000 7fe00f7fe700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdffc05f2c0 0x7fdffc061780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fe000009c30 tx=0x7fe000009060).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:31.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.095+0000 7fdff77fe700 1 -- 172.21.15.33:0/1496052683 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe00804d160 con 0x7fe010155420 2023-11-06T20:56:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.305+0000 7fe015bff700 1 -- 172.21.15.33:0/1496052683 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe010063390 con 0x7fdffc05f2c0 2023-11-06T20:56:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.310+0000 7fdff77fe700 1 -- 172.21.15.33:0/1496052683 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fe010063390 con 0x7fdffc05f2c0 2023-11-06T20:56:31.310 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-11-06T20:56:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 -- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdffc05f2c0 msgr2=0x7fdffc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdffc05f2c0 0x7fdffc061780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fe000009c30 tx=0x7fe000009060).stop 2023-11-06T20:56:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 -- 172.21.15.33:0/1496052683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 msgr2=0x7fe010167ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 0x7fe010167ef0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fe00800b610 tx=0x7fe008009780).stop 2023-11-06T20:56:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 -- 172.21.15.33:0/1496052683 shutdown_connections 2023-11-06T20:56:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdffc05f2c0 0x7fdffc061780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:31.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe010155420 0x7fe010167ef0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:31.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01006a120 0x7fe0100a7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:31.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 --2- 172.21.15.33:0/1496052683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0101545e0 0x7fe0100ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:31.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 -- 172.21.15.33:0/1496052683 >> 172.21.15.33:0/1496052683 conn(0x7fe0100c7b30 msgr2=0x7fe0100a5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 -- 172.21.15.33:0/1496052683 shutdown_connections 2023-11-06T20:56:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:31.312+0000 7fe015bff700 1 -- 172.21.15.33:0/1496052683 wait complete. 2023-11-06T20:56:31.330 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-11-06T20:56:31.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.330 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-11-06T20:56:31.379 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:31.379 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:31.379 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:31.380 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:31.380 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:31.380 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:31.380 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:31.380 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:31.381 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:31.381 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:31.381 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:31.382 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:31.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.383 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_46_00_UTC) 2023-11-06T20:56:31.438 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:31.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.439 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_47_00_UTC) 2023-11-06T20:56:31.495 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:31.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.496 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_48_00_UTC) 2023-11-06T20:56:31.553 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:31.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.554 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_49_00_UTC) 2023-11-06T20:56:31.609 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:31.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.610 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_50_00_UTC) 2023-11-06T20:56:31.665 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:31.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.666 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_51_00_UTC) 2023-11-06T20:56:31.723 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:31.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.725 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_52_00_UTC) 2023-11-06T20:56:31.781 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:31.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.782 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_53_00_UTC) 2023-11-06T20:56:31.836 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:31.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.837 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_54_00_UTC) 2023-11-06T20:56:31.892 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:31.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.893 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_55_00_UTC) 2023-11-06T20:56:31.949 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:31.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:31.950 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-06-20_56_00_UTC) 2023-11-06T20:56:32.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:32.006 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-11-06T20:56:32.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2023-11-06T20:56:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.295+0000 7fdd687ff700 1 Processor -- start 2023-11-06T20:56:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.296+0000 7fdd687ff700 1 -- start start 2023-11-06T20:56:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.297+0000 7fdd687ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdd60153ed0 0x7fdd60158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.297+0000 7fdd687ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdd600c8ee0 0x7fdd600ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.297+0000 7fdd687ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd600c9f80 con 0x7fdd601542b0 2023-11-06T20:56:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.297+0000 7fdd687ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd600c6000 con 0x7fdd600c8ee0 2023-11-06T20:56:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.297+0000 7fdd687ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd600c9370 con 0x7fdd60153ed0 2023-11-06T20:56:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.297+0000 7fdd65d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdd600c8ee0 0x7fdd600ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.297+0000 7fdd66d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdd601542b0 0x7fdd600c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54988/0 (socket says 172.21.15.33:54988) 2023-11-06T20:56:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.297+0000 7fdd66d9c700 1 -- 172.21.15.33:0/2149021631 learned_addr learned my addr 172.21.15.33:0/2149021631 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.297+0000 7fdd6659b700 1 --2- 172.21.15.33:0/2149021631 >> v2:172.21.15.69:3300/0 conn(0x7fdd60153ed0 0x7fdd60158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.298+0000 7fdd65d9a700 1 -- 172.21.15.33:0/2149021631 >> v2:172.21.15.69:3300/0 conn(0x7fdd60153ed0 msgr2=0x7fdd60158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.298+0000 7fdd65d9a700 1 --2- 172.21.15.33:0/2149021631 >> v2:172.21.15.69:3300/0 conn(0x7fdd60153ed0 0x7fdd60158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.298+0000 7fdd65d9a700 1 -- 172.21.15.33:0/2149021631 >> v1:172.21.15.33:6789/0 conn(0x7fdd601542b0 legacy=0x7fdd600c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.298+0000 7fdd65d9a700 1 -- 172.21.15.33:0/2149021631 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600c9650 con 0x7fdd600c8ee0 2023-11-06T20:56:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.298+0000 7fdd65599700 1 -- 172.21.15.33:0/2149021631 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546125665 0 0) 0x7fdd600c9f80 con 0x7fdd601542b0 2023-11-06T20:56:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.298+0000 7fdd65d9a700 1 --2- 172.21.15.33:0/2149021631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd600c8ee0 0x7fdd600ca410 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fdd6006bdb0 tx=0x7fdd50009060).ready entity=mon.0 client_cookie=6f378ea5140f6505 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.298+0000 7fdd65599700 1 -- 172.21.15.33:0/2149021631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5000f510 con 0x7fdd600c8ee0 2023-11-06T20:56:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.298+0000 7fdd65599700 1 -- 172.21.15.33:0/2149021631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5000f670 con 0x7fdd600c8ee0 2023-11-06T20:56:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.298+0000 7fdd65599700 1 -- 172.21.15.33:0/2149021631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5000f9f0 con 0x7fdd600c8ee0 2023-11-06T20:56:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.299+0000 7fdd687ff700 1 -- 172.21.15.33:0/2149021631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd600c8ee0 msgr2=0x7fdd600ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.299+0000 7fdd687ff700 1 --2- 172.21.15.33:0/2149021631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd600c8ee0 0x7fdd600ca410 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fdd6006bdb0 tx=0x7fdd50009060).stop 2023-11-06T20:56:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.299+0000 7fdd687ff700 1 -- 172.21.15.33:0/2149021631 shutdown_connections 2023-11-06T20:56:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.299+0000 7fdd687ff700 1 --2- 172.21.15.33:0/2149021631 >> v2:172.21.15.69:3300/0 conn(0x7fdd60153ed0 0x7fdd60158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.299+0000 7fdd687ff700 1 --2- 172.21.15.33:0/2149021631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd600c8ee0 0x7fdd600ca410 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.299+0000 7fdd687ff700 1 -- 172.21.15.33:0/2149021631 >> 172.21.15.33:0/2149021631 conn(0x7fdd6005e1b0 msgr2=0x7fdd6005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.299+0000 7fdd687ff700 1 -- 172.21.15.33:0/2149021631 shutdown_connections 2023-11-06T20:56:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.299+0000 7fdd687ff700 1 -- 172.21.15.33:0/2149021631 wait complete. 2023-11-06T20:56:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.300+0000 7fdd687ff700 1 Processor -- start 2023-11-06T20:56:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.300+0000 7fdd687ff700 1 -- start start 2023-11-06T20:56:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.301+0000 7fdd687ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd60153ed0 0x7fdd6013d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.301+0000 7fdd687ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd601542b0 0x7fdd6014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.301+0000 7fdd687ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd6014f650 0x7fdd600addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.301+0000 7fdd687ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60165c90 con 0x7fdd601542b0 2023-11-06T20:56:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.301+0000 7fdd687ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6014e3a0 con 0x7fdd60153ed0 2023-11-06T20:56:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.301+0000 7fdd687ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6014e680 con 0x7fdd6014f650 2023-11-06T20:56:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd65d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd601542b0 0x7fdd6014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd65d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd601542b0 0x7fdd6014f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42522/0 (socket says 172.21.15.33:42522) 2023-11-06T20:56:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd65d9a700 1 -- 172.21.15.33:0/1908173245 learned_addr learned my addr 172.21.15.33:0/1908173245 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd6659b700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd60153ed0 0x7fdd6013d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd65d9a700 1 -- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd6014f650 msgr2=0x7fdd600addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd65d9a700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd6014f650 0x7fdd600addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd65d9a700 1 -- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd60153ed0 msgr2=0x7fdd6013d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd65d9a700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd60153ed0 0x7fdd6013d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd65d9a700 1 -- 172.21.15.33:0/1908173245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd500092a0 con 0x7fdd601542b0 2023-11-06T20:56:32.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd6659b700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd60153ed0 0x7fdd6013d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:32.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.302+0000 7fdd65d9a700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd601542b0 0x7fdd6014f160 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fdd50009dd0 tx=0x7fdd50017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:32.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.303+0000 7fdd567fc700 1 -- 172.21.15.33:0/1908173245 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5000f7e0 con 0x7fdd601542b0 2023-11-06T20:56:32.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.303+0000 7fdd567fc700 1 -- 172.21.15.33:0/1908173245 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5000f940 con 0x7fdd601542b0 2023-11-06T20:56:32.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.303+0000 7fdd567fc700 1 -- 172.21.15.33:0/1908173245 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5001b660 con 0x7fdd601542b0 2023-11-06T20:56:32.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.303+0000 7fdd687ff700 1 -- 172.21.15.33:0/1908173245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6014e960 con 0x7fdd601542b0 2023-11-06T20:56:32.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.303+0000 7fdd687ff700 1 -- 172.21.15.33:0/1908173245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6013dcb0 con 0x7fdd601542b0 2023-11-06T20:56:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.304+0000 7fdd687ff700 1 -- 172.21.15.33:0/1908173245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6014ee80 con 0x7fdd601542b0 2023-11-06T20:56:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.305+0000 7fdd567fc700 1 -- 172.21.15.33:0/1908173245 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fdd5001ba10 con 0x7fdd601542b0 2023-11-06T20:56:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.305+0000 7fdd567fc700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd4405f2c0 0x7fdd44061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.305+0000 7fdd567fc700 1 -- 172.21.15.33:0/1908173245 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6843+0+0 (secure 0 0 0) 0x7fdd5007bff0 con 0x7fdd601542b0 2023-11-06T20:56:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.306+0000 7fdd6659b700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd4405f2c0 0x7fdd44061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.306+0000 7fdd6659b700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd4405f2c0 0x7fdd44061780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fdd5c006d80 tx=0x7fdd5c008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.310+0000 7fdd567fc700 1 -- 172.21.15.33:0/1908173245 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdd500aaaf0 con 0x7fdd601542b0 2023-11-06T20:56:32.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.516+0000 7fdd687ff700 1 -- 172.21.15.33:0/1908173245 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd6006c6c0 con 0x7fdd4405f2c0 2023-11-06T20:56:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.524+0000 7fdd567fc700 1 -- 172.21.15.33:0/1908173245 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fdd6006c6c0 con 0x7fdd4405f2c0 2023-11-06T20:56:32.525 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-11-06T20:56:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.526+0000 7fdd687ff700 1 -- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd4405f2c0 msgr2=0x7fdd44061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.526+0000 7fdd687ff700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd4405f2c0 0x7fdd44061780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fdd5c006d80 tx=0x7fdd5c008040).stop 2023-11-06T20:56:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.526+0000 7fdd687ff700 1 -- 172.21.15.33:0/1908173245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd601542b0 msgr2=0x7fdd6014f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.526+0000 7fdd687ff700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd601542b0 0x7fdd6014f160 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fdd50009dd0 tx=0x7fdd50017000).stop 2023-11-06T20:56:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.526+0000 7fdd687ff700 1 -- 172.21.15.33:0/1908173245 shutdown_connections 2023-11-06T20:56:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.526+0000 7fdd687ff700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd4405f2c0 0x7fdd44061780 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.526+0000 7fdd687ff700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd6014f650 0x7fdd600addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.526+0000 7fdd687ff700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd60153ed0 0x7fdd6013d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.526+0000 7fdd687ff700 1 --2- 172.21.15.33:0/1908173245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd601542b0 0x7fdd6014f160 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.526+0000 7fdd687ff700 1 -- 172.21.15.33:0/1908173245 >> 172.21.15.33:0/1908173245 conn(0x7fdd6005e1b0 msgr2=0x7fdd600aabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.527+0000 7fdd687ff700 1 -- 172.21.15.33:0/1908173245 shutdown_connections 2023-11-06T20:56:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:32.527+0000 7fdd687ff700 1 -- 172.21.15.33:0/1908173245 wait complete. 2023-11-06T20:56:32.554 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-11-06T20:56:32.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:32.555 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-11-06T20:56:32.611 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:32.612 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:32.612 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:32.612 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:32.613 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:32.613 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:32.613 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:32.613 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:32.613 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:32.613 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:32.614 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:32.615 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:32.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:32.616 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_46_00_UTC) 2023-11-06T20:56:32.672 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:32.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:32.673 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_47_00_UTC) 2023-11-06T20:56:32.730 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:32.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:32.731 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_48_00_UTC) 2023-11-06T20:56:32.795 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:32.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:32.796 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_49_00_UTC) 2023-11-06T20:56:32.853 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:32.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:32.854 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_50_00_UTC) 2023-11-06T20:56:32.912 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:32.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:32.913 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_51_00_UTC) 2023-11-06T20:56:32.969 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:32.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:32.970 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_52_00_UTC) 2023-11-06T20:56:33.025 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:33.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:33.026 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_53_00_UTC) 2023-11-06T20:56:33.082 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:33.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:33.085 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_54_00_UTC) 2023-11-06T20:56:33.200 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:33.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:33.201 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_55_00_UTC) 2023-11-06T20:56:33.260 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:33.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:33.261 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-06-20_56_00_UTC) 2023-11-06T20:56:33.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:33.318 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-11-06T20:56:33.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2023-11-06T20:56:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.599+0000 7f6d539ff700 1 Processor -- start 2023-11-06T20:56:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.599+0000 7f6d539ff700 1 -- start start 2023-11-06T20:56:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.600+0000 7f6d539ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d4c0a5de0 0x7f6d4c153460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.600+0000 7f6d539ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 0x7f6d4c0c15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.600+0000 7f6d539ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4c153950 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.600+0000 7f6d539ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4c0bd490 con 0x7f6d4c0a5de0 2023-11-06T20:56:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.600+0000 7f6d539ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0bd770 con 0x7f6d4c0c3b60 2023-11-06T20:56:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.600+0000 7f6d50f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d4c0a5de0 0x7f6d4c153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.600+0000 7f6d51f9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 0x7f6d4c0c15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.600+0000 7f6d50f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d4c0a5de0 0x7f6d4c153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42524/0 (socket says 172.21.15.33:42524) 2023-11-06T20:56:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.600+0000 7f6d50f9a700 1 -- 172.21.15.33:0/3211531064 learned_addr learned my addr 172.21.15.33:0/3211531064 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.601+0000 7f6d51f9c700 1 -- 172.21.15.33:0/3211531064 >> v2:172.21.15.33:3300/0 conn(0x7f6d4c0a5de0 msgr2=0x7f6d4c153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.601+0000 7f6d51f9c700 1 --2- 172.21.15.33:0/3211531064 >> v2:172.21.15.33:3300/0 conn(0x7f6d4c0a5de0 0x7f6d4c153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.601+0000 7f6d51f9c700 1 -- 172.21.15.33:0/3211531064 >> v1:172.21.15.69:6789/0 conn(0x7f6d4c0a8c10 legacy=0x7f6d4c0c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.601+0000 7f6d51f9c700 1 -- 172.21.15.33:0/3211531064 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c0c2b50 con 0x7f6d4c0c3b60 2023-11-06T20:56:33.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.601+0000 7f6d50f9a700 1 --2- 172.21.15.33:0/3211531064 >> v2:172.21.15.33:3300/0 conn(0x7f6d4c0a5de0 0x7f6d4c153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:33.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.601+0000 7f6d43fff700 1 -- 172.21.15.33:0/3211531064 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774325468 0 0) 0x7f6d4c153950 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.601+0000 7f6d51f9c700 1 --2- 172.21.15.33:0/3211531064 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 0x7f6d4c0c15d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6d44009e60 tx=0x7f6d44009780).ready entity=mon.2 client_cookie=3bfaab011bf7f3c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:33.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.601+0000 7f6d43fff700 1 -- 172.21.15.33:0/3211531064 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400e070 con 0x7f6d4c0c3b60 2023-11-06T20:56:33.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.601+0000 7f6d43fff700 1 -- 172.21.15.33:0/3211531064 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4400bb60 con 0x7f6d4c0c3b60 2023-11-06T20:56:33.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.601+0000 7f6d43fff700 1 -- 172.21.15.33:0/3211531064 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d44012410 con 0x7f6d4c0c3b60 2023-11-06T20:56:33.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.602+0000 7f6d539ff700 1 -- 172.21.15.33:0/3211531064 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 msgr2=0x7f6d4c0c15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:33.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.602+0000 7f6d539ff700 1 --2- 172.21.15.33:0/3211531064 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 0x7f6d4c0c15d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6d44009e60 tx=0x7f6d44009780).stop 2023-11-06T20:56:33.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.602+0000 7f6d539ff700 1 -- 172.21.15.33:0/3211531064 shutdown_connections 2023-11-06T20:56:33.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.602+0000 7f6d539ff700 1 --2- 172.21.15.33:0/3211531064 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 0x7f6d4c0c15d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:33.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.602+0000 7f6d539ff700 1 --2- 172.21.15.33:0/3211531064 >> v2:172.21.15.33:3300/0 conn(0x7f6d4c0a5de0 0x7f6d4c153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:33.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.602+0000 7f6d539ff700 1 -- 172.21.15.33:0/3211531064 >> 172.21.15.33:0/3211531064 conn(0x7f6d4c062710 msgr2=0x7f6d4c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:33.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.602+0000 7f6d539ff700 1 -- 172.21.15.33:0/3211531064 shutdown_connections 2023-11-06T20:56:33.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.602+0000 7f6d539ff700 1 -- 172.21.15.33:0/3211531064 wait complete. 2023-11-06T20:56:33.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.603+0000 7f6d539ff700 1 Processor -- start 2023-11-06T20:56:33.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.604+0000 7f6d539ff700 1 -- start start 2023-11-06T20:56:33.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.604+0000 7f6d539ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d4c0a5de0 0x7f6d4c1679c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:33.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.604+0000 7f6d539ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d4c0a8c10 0x7f6d4c0ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d539ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 0x7f6d4c0abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d539ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c167eb0 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d539ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0a6500 con 0x7f6d4c0a5de0 2023-11-06T20:56:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d539ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0a7360 con 0x7f6d4c0c3b60 2023-11-06T20:56:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d50f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d4c0a8c10 0x7f6d4c0ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d50f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d4c0a8c10 0x7f6d4c0ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42528/0 (socket says 172.21.15.33:42528) 2023-11-06T20:56:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d50f9a700 1 -- 172.21.15.33:0/3810931710 learned_addr learned my addr 172.21.15.33:0/3810931710 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d51f9c700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 0x7f6d4c0abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d5179b700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d4c0a5de0 0x7f6d4c1679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d50f9a700 1 -- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 msgr2=0x7f6d4c0abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d50f9a700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 0x7f6d4c0abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:33.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d50f9a700 1 -- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d4c0a5de0 msgr2=0x7f6d4c1679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:33.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d50f9a700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d4c0a5de0 0x7f6d4c1679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:33.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d50f9a700 1 -- 172.21.15.33:0/3810931710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c009370 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d51f9c700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 0x7f6d4c0abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:33.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.605+0000 7f6d50f9a700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d4c0a8c10 0x7f6d4c0ab8f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f6d3c006d00 tx=0x7f6d3c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:33.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.606+0000 7f6d417fa700 1 -- 172.21.15.33:0/3810931710 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c00eee0 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.606+0000 7f6d417fa700 1 -- 172.21.15.33:0/3810931710 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c00c850 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.606+0000 7f6d417fa700 1 -- 172.21.15.33:0/3810931710 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c00cbd0 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.606+0000 7f6d539ff700 1 -- 172.21.15.33:0/3810931710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d44009a20 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.606+0000 7f6d539ff700 1 -- 172.21.15.33:0/3810931710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c0a6a30 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.607+0000 7f6d539ff700 1 -- 172.21.15.33:0/3810931710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c0c2e30 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.608+0000 7f6d417fa700 1 -- 172.21.15.33:0/3810931710 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f6d3c01b400 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.609+0000 7f6d417fa700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d3005f2c0 0x7f6d30061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:33.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.609+0000 7f6d417fa700 1 -- 172.21.15.33:0/3810931710 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7035+0+0 (secure 0 0 0) 0x7f6d3c07ae80 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.609+0000 7f6d5179b700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d3005f2c0 0x7f6d30061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:33.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.610+0000 7f6d5179b700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d3005f2c0 0x7f6d30061780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6d4c167920 tx=0x7f6d48009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:33.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.612+0000 7f6d417fa700 1 -- 172.21.15.33:0/3810931710 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6d3c04bb20 con 0x7f6d4c0a8c10 2023-11-06T20:56:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.816+0000 7f6d539ff700 1 -- 172.21.15.33:0/3810931710 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d4c0a94e0 con 0x7f6d3005f2c0 2023-11-06T20:56:33.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.820+0000 7f6d417fa700 1 -- 172.21.15.33:0/3810931710 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f6d4c0a94e0 con 0x7f6d3005f2c0 2023-11-06T20:56:33.821 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-11-06T20:56:33.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 -- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d3005f2c0 msgr2=0x7f6d30061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:33.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d3005f2c0 0x7f6d30061780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6d4c167920 tx=0x7f6d48009040).stop 2023-11-06T20:56:33.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 -- 172.21.15.33:0/3810931710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d4c0a8c10 msgr2=0x7f6d4c0ab8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:33.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d4c0a8c10 0x7f6d4c0ab8f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f6d3c006d00 tx=0x7f6d3c0090f0).stop 2023-11-06T20:56:33.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 -- 172.21.15.33:0/3810931710 shutdown_connections 2023-11-06T20:56:33.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d3005f2c0 0x7f6d30061780 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:33.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d4c0c3b60 0x7f6d4c0abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:33.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d4c0a5de0 0x7f6d4c1679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:33.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 --2- 172.21.15.33:0/3810931710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d4c0a8c10 0x7f6d4c0ab8f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:33.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 -- 172.21.15.33:0/3810931710 >> 172.21.15.33:0/3810931710 conn(0x7f6d4c062710 msgr2=0x7f6d4c0c33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:33.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 -- 172.21.15.33:0/3810931710 shutdown_connections 2023-11-06T20:56:33.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:33.822+0000 7f6d539ff700 1 -- 172.21.15.33:0/3810931710 wait complete. 2023-11-06T20:56:33.840 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-11-06T20:56:33.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:33.841 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-11-06T20:56:33.890 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:33.890 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:33.890 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:33.890 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:33.891 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:33.891 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:33.891 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:33.891 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:33.891 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:33.892 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:33.892 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:33.893 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:33.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:33.894 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_46_00_UTC) 2023-11-06T20:56:33.951 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:33.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:33.952 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_47_00_UTC) 2023-11-06T20:56:34.007 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:34.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:34.008 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_48_00_UTC) 2023-11-06T20:56:34.062 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:34.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:34.063 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_49_00_UTC) 2023-11-06T20:56:34.143 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:34.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:34.144 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_50_00_UTC) 2023-11-06T20:56:34.199 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:34.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:34.200 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_51_00_UTC) 2023-11-06T20:56:34.255 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:34.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:34.256 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_52_00_UTC) 2023-11-06T20:56:34.310 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:34.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:34.311 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_53_00_UTC) 2023-11-06T20:56:34.371 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:34.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:34.372 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_54_00_UTC) 2023-11-06T20:56:34.457 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:34.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:34.458 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_55_00_UTC) 2023-11-06T20:56:34.512 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:34.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:34.513 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-06-20_56_00_UTC) 2023-11-06T20:56:34.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:34.568 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-11-06T20:56:34.622 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2023-11-06T20:56:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.821+0000 7f048ffff700 1 Processor -- start 2023-11-06T20:56:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.821+0000 7f048ffff700 1 -- start start 2023-11-06T20:56:34.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.823+0000 7f048ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04880638d0 0x7f04880aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:34.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.823+0000 7f048ffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04881542b0 0x7f0488157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:34.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.823+0000 7f048ffff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04880c02c0 con 0x7f0488153ed0 2023-11-06T20:56:34.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.823+0000 7f048ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04881581e0 con 0x7f04880638d0 2023-11-06T20:56:34.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.823+0000 7f048ffff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0488158460 con 0x7f04881542b0 2023-11-06T20:56:34.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.823+0000 7f048d59a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04880638d0 0x7f04880aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:34.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.823+0000 7f048d59a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04880638d0 0x7f04880aa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42540/0 (socket says 172.21.15.33:42540) 2023-11-06T20:56:34.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.823+0000 7f048d59a700 1 -- 172.21.15.33:0/3649984623 learned_addr learned my addr 172.21.15.33:0/3649984623 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:34.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.823+0000 7f048e59c700 1 --2- 172.21.15.33:0/3649984623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04881542b0 0x7f0488157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:34.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.824+0000 7f048cd99700 1 -- 172.21.15.33:0/3649984623 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258965725 0 0) 0x7f04880c02c0 con 0x7f0488153ed0 2023-11-06T20:56:34.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.824+0000 7f048d59a700 1 -- 172.21.15.33:0/3649984623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04881542b0 msgr2=0x7f0488157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:34.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.824+0000 7f048d59a700 1 --2- 172.21.15.33:0/3649984623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04881542b0 0x7f0488157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:34.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.824+0000 7f048d59a700 1 -- 172.21.15.33:0/3649984623 >> v1:172.21.15.69:6789/0 conn(0x7f0488153ed0 legacy=0x7f04880631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:34.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.824+0000 7f048d59a700 1 -- 172.21.15.33:0/3649984623 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0488158740 con 0x7f04880638d0 2023-11-06T20:56:34.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.824+0000 7f048e59c700 1 --2- 172.21.15.33:0/3649984623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04881542b0 0x7f0488157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:34.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.824+0000 7f048d59a700 1 --2- 172.21.15.33:0/3649984623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 0x7f04880aa2c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f0478006fd0 tx=0x7f04780090f0).ready entity=mon.0 client_cookie=c23562b774f58598 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:34.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.824+0000 7f048cd99700 1 -- 172.21.15.33:0/3649984623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047800b580 con 0x7f04880638d0 2023-11-06T20:56:34.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.824+0000 7f048cd99700 1 -- 172.21.15.33:0/3649984623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047800b6e0 con 0x7f04880638d0 2023-11-06T20:56:34.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.824+0000 7f048cd99700 1 -- 172.21.15.33:0/3649984623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047800ba60 con 0x7f04880638d0 2023-11-06T20:56:34.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.825+0000 7f048ffff700 1 -- 172.21.15.33:0/3649984623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 msgr2=0x7f04880aa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:34.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.825+0000 7f048ffff700 1 --2- 172.21.15.33:0/3649984623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 0x7f04880aa2c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f0478006fd0 tx=0x7f04780090f0).stop 2023-11-06T20:56:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.825+0000 7f048ffff700 1 -- 172.21.15.33:0/3649984623 shutdown_connections 2023-11-06T20:56:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.825+0000 7f048ffff700 1 --2- 172.21.15.33:0/3649984623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f04881542b0 0x7f0488157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.825+0000 7f048ffff700 1 --2- 172.21.15.33:0/3649984623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 0x7f04880aa2c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.825+0000 7f048ffff700 1 -- 172.21.15.33:0/3649984623 >> 172.21.15.33:0/3649984623 conn(0x7f048805ec80 msgr2=0x7f04880cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.825+0000 7f048ffff700 1 -- 172.21.15.33:0/3649984623 shutdown_connections 2023-11-06T20:56:34.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.825+0000 7f048ffff700 1 -- 172.21.15.33:0/3649984623 wait complete. 2023-11-06T20:56:34.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.827+0000 7f048ffff700 1 Processor -- start 2023-11-06T20:56:34.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.827+0000 7f048ffff700 1 -- start start 2023-11-06T20:56:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.828+0000 7f048ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 0x7f048814db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:34.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.828+0000 7f048ffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0488153ed0 0x7f04880addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:34.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.828+0000 7f048ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04881542b0 0x7f04880ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:34.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.828+0000 7f048ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880ae7b0 con 0x7f04880638d0 2023-11-06T20:56:34.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.828+0000 7f048ffff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04881557e0 con 0x7f04881542b0 2023-11-06T20:56:34.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.828+0000 7f048ffff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04880c48d0 con 0x7f0488153ed0 2023-11-06T20:56:34.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 0x7f048814db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:34.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 0x7f048814db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42546/0 (socket says 172.21.15.33:42546) 2023-11-06T20:56:34.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048dd9b700 1 -- 172.21.15.33:0/1857435059 learned_addr learned my addr 172.21.15.33:0/1857435059 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:34.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048d59a700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0488153ed0 0x7f04880addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:34.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048e59c700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04881542b0 0x7f04880ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:34.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048dd9b700 1 -- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0488153ed0 msgr2=0x7f04880addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:34.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048dd9b700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0488153ed0 0x7f04880addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:34.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048dd9b700 1 -- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04881542b0 msgr2=0x7f04880ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:34.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048dd9b700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04881542b0 0x7f04880ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048dd9b700 1 -- 172.21.15.33:0/1857435059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480009a20 con 0x7f04880638d0 2023-11-06T20:56:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.829+0000 7f048dd9b700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 0x7f048814db00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0484001eb0 tx=0x7f0484007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.830+0000 7f047dffb700 1 -- 172.21.15.33:0/1857435059 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048400f470 con 0x7f04880638d0 2023-11-06T20:56:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.830+0000 7f047dffb700 1 -- 172.21.15.33:0/1857435059 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048400f5f0 con 0x7f04880638d0 2023-11-06T20:56:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.830+0000 7f048ffff700 1 -- 172.21.15.33:0/1857435059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0478009370 con 0x7f04880638d0 2023-11-06T20:56:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.830+0000 7f047dffb700 1 -- 172.21.15.33:0/1857435059 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048400f9b0 con 0x7f04880638d0 2023-11-06T20:56:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.830+0000 7f048ffff700 1 -- 172.21.15.33:0/1857435059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04880aec90 con 0x7f04880638d0 2023-11-06T20:56:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.831+0000 7f048ffff700 1 -- 172.21.15.33:0/1857435059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0488158a20 con 0x7f04880638d0 2023-11-06T20:56:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.832+0000 7f047dffb700 1 -- 172.21.15.33:0/1857435059 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f048400fd20 con 0x7f04880638d0 2023-11-06T20:56:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.833+0000 7f047dffb700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046c05f2c0 0x7f046c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.833+0000 7f047dffb700 1 -- 172.21.15.33:0/1857435059 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f0484079d70 con 0x7f04880638d0 2023-11-06T20:56:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.833+0000 7f048d59a700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046c05f2c0 0x7f046c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.834+0000 7f048d59a700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046c05f2c0 0x7f046c061780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0478006fd0 tx=0x7f0478017000).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:34.837+0000 7f047dffb700 1 -- 172.21.15.33:0/1857435059 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f048404ab50 con 0x7f04880638d0 2023-11-06T20:56:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.047+0000 7f048ffff700 1 -- 172.21.15.33:0/1857435059 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f04880af2a0 con 0x7f046c05f2c0 2023-11-06T20:56:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.051+0000 7f047dffb700 1 -- 172.21.15.33:0/1857435059 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f04880af2a0 con 0x7f046c05f2c0 2023-11-06T20:56:35.051 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-11-06T20:56:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 -- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046c05f2c0 msgr2=0x7f046c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046c05f2c0 0x7f046c061780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0478006fd0 tx=0x7f0478017000).stop 2023-11-06T20:56:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 -- 172.21.15.33:0/1857435059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 msgr2=0x7f048814db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 0x7f048814db00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0484001eb0 tx=0x7f0484007c70).stop 2023-11-06T20:56:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 -- 172.21.15.33:0/1857435059 shutdown_connections 2023-11-06T20:56:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046c05f2c0 0x7f046c061780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04881542b0 0x7f04880ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04880638d0 0x7f048814db00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 --2- 172.21.15.33:0/1857435059 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0488153ed0 0x7f04880addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 -- 172.21.15.33:0/1857435059 >> 172.21.15.33:0/1857435059 conn(0x7f048805ec80 msgr2=0x7f04880c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:35.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 -- 172.21.15.33:0/1857435059 shutdown_connections 2023-11-06T20:56:35.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:35.053+0000 7f048ffff700 1 -- 172.21.15.33:0/1857435059 wait complete. 2023-11-06T20:56:35.071 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-11-06T20:56:35.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.072 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-11-06T20:56:35.125 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:35.125 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:35.125 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:35.125 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:35.125 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:35.126 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:35.126 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:35.126 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:35.127 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_01_UTC 2023-11-06T20:56:35.127 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:35.127 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:35.128 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:35.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.129 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_46_00_UTC) 2023-11-06T20:56:35.188 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:35.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.189 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_47_00_UTC) 2023-11-06T20:56:35.243 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:35.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.245 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_48_00_UTC) 2023-11-06T20:56:35.299 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:35.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.300 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_49_00_UTC) 2023-11-06T20:56:35.355 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:35.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.356 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_50_00_UTC) 2023-11-06T20:56:35.448 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:35.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.449 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_51_00_UTC) 2023-11-06T20:56:35.504 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:35.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.505 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_52_00_UTC) 2023-11-06T20:56:35.560 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:35.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.561 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_53_00_UTC) 2023-11-06T20:56:35.615 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_54_01_UTC 2023-11-06T20:56:35.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.616 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_54_01_UTC) 2023-11-06T20:56:35.670 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:35.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.671 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_55_00_UTC) 2023-11-06T20:56:35.726 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:35.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.727 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-06-20_56_00_UTC) 2023-11-06T20:56:35.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:35.784 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-11-06T20:56:35.834 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2023-11-06T20:56:36.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.035+0000 7f81481ff700 1 Processor -- start 2023-11-06T20:56:36.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.035+0000 7f81481ff700 1 -- start start 2023-11-06T20:56:36.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f81481ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81400c8ee0 0x7f81400ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:36.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f81481ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f81401542b0 0x7f81400c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:36.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f81481ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81400c9f80 con 0x7f8140153ed0 2023-11-06T20:56:36.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f81481ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81400c6000 con 0x7f81400c8ee0 2023-11-06T20:56:36.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f81481ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81400c9370 con 0x7f81401542b0 2023-11-06T20:56:36.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f814579a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81400c8ee0 0x7f81400ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:36.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f8145f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8140153ed0 0x7f8140158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54996/0 (socket says 172.21.15.33:54996) 2023-11-06T20:56:36.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f8145f9b700 1 -- 172.21.15.33:0/4114983961 learned_addr learned my addr 172.21.15.33:0/4114983961 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:36.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f814679c700 1 --2- 172.21.15.33:0/4114983961 >> v2:172.21.15.69:3300/0 conn(0x7f81401542b0 0x7f81400c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:36.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f8144f99700 1 -- 172.21.15.33:0/4114983961 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945090753 0 0) 0x7f81400c9f80 con 0x7f8140153ed0 2023-11-06T20:56:36.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f8144f99700 1 -- 172.21.15.33:0/4114983961 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f812c0033f0 con 0x7f8140153ed0 2023-11-06T20:56:36.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f814579a700 1 -- 172.21.15.33:0/4114983961 >> v2:172.21.15.69:3300/0 conn(0x7f81401542b0 msgr2=0x7f81400c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:36.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.037+0000 7f814579a700 1 --2- 172.21.15.33:0/4114983961 >> v2:172.21.15.69:3300/0 conn(0x7f81401542b0 0x7f81400c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:36.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.038+0000 7f814579a700 1 -- 172.21.15.33:0/4114983961 >> v1:172.21.15.33:6789/0 conn(0x7f8140153ed0 legacy=0x7f8140158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:36.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.038+0000 7f814579a700 1 -- 172.21.15.33:0/4114983961 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81400c9650 con 0x7f81400c8ee0 2023-11-06T20:56:36.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.038+0000 7f814679c700 1 --2- 172.21.15.33:0/4114983961 >> v2:172.21.15.69:3300/0 conn(0x7f81401542b0 0x7f81400c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:36.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.038+0000 7f814579a700 1 --2- 172.21.15.33:0/4114983961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81400c8ee0 0x7f81400ca410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f81400c85d0 tx=0x7f8130009060).ready entity=mon.0 client_cookie=1973b9901368af62 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:36.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.038+0000 7f8144f99700 1 -- 172.21.15.33:0/4114983961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000f510 con 0x7f81400c8ee0 2023-11-06T20:56:36.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.038+0000 7f8144f99700 1 -- 172.21.15.33:0/4114983961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813000f670 con 0x7f81400c8ee0 2023-11-06T20:56:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.038+0000 7f8144f99700 1 -- 172.21.15.33:0/4114983961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000f9f0 con 0x7f81400c8ee0 2023-11-06T20:56:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.038+0000 7f81481ff700 1 -- 172.21.15.33:0/4114983961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81400c8ee0 msgr2=0x7f81400ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.038+0000 7f81481ff700 1 --2- 172.21.15.33:0/4114983961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81400c8ee0 0x7f81400ca410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f81400c85d0 tx=0x7f8130009060).stop 2023-11-06T20:56:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.039+0000 7f81481ff700 1 -- 172.21.15.33:0/4114983961 shutdown_connections 2023-11-06T20:56:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.039+0000 7f81481ff700 1 --2- 172.21.15.33:0/4114983961 >> v2:172.21.15.69:3300/0 conn(0x7f81401542b0 0x7f81400c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.039+0000 7f81481ff700 1 --2- 172.21.15.33:0/4114983961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81400c8ee0 0x7f81400ca410 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.039+0000 7f81481ff700 1 -- 172.21.15.33:0/4114983961 >> 172.21.15.33:0/4114983961 conn(0x7f814005e1b0 msgr2=0x7f814005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.039+0000 7f81481ff700 1 -- 172.21.15.33:0/4114983961 shutdown_connections 2023-11-06T20:56:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.039+0000 7f81481ff700 1 -- 172.21.15.33:0/4114983961 wait complete. 2023-11-06T20:56:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.040+0000 7f81481ff700 1 Processor -- start 2023-11-06T20:56:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.040+0000 7f81481ff700 1 -- start start 2023-11-06T20:56:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.041+0000 7f81481ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8140153ed0 0x7f814014d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.041+0000 7f81481ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81401542b0 0x7f8140164320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.041+0000 7f81481ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f814014dd30 0x7f81401565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.041+0000 7f81481ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8140165300 con 0x7f81401542b0 2023-11-06T20:56:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.041+0000 7f81481ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8140164d40 con 0x7f814014dd30 2023-11-06T20:56:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.041+0000 7f81481ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8140165020 con 0x7f8140153ed0 2023-11-06T20:56:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.041+0000 7f814579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81401542b0 0x7f8140164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.041+0000 7f814579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81401542b0 0x7f8140164320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42562/0 (socket says 172.21.15.33:42562) 2023-11-06T20:56:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.041+0000 7f814579a700 1 -- 172.21.15.33:0/1265112452 learned_addr learned my addr 172.21.15.33:0/1265112452 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f8145f9b700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8140153ed0 0x7f814014d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f814679c700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f814014dd30 0x7f81401565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f814579a700 1 -- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8140153ed0 msgr2=0x7f814014d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f814579a700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8140153ed0 0x7f814014d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f814579a700 1 -- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f814014dd30 msgr2=0x7f81401565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f814579a700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f814014dd30 0x7f81401565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f814579a700 1 -- 172.21.15.33:0/1265112452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300092a0 con 0x7f81401542b0 2023-11-06T20:56:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f814679c700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f814014dd30 0x7f81401565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f814579a700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81401542b0 0x7f8140164320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f8130009dd0 tx=0x7f8130017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f8135ffb700 1 -- 172.21.15.33:0/1265112452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000f7e0 con 0x7f81401542b0 2023-11-06T20:56:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f8135ffb700 1 -- 172.21.15.33:0/1265112452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813000f940 con 0x7f81401542b0 2023-11-06T20:56:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f81481ff700 1 -- 172.21.15.33:0/1265112452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8140155de0 con 0x7f81401542b0 2023-11-06T20:56:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f8135ffb700 1 -- 172.21.15.33:0/1265112452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813001b670 con 0x7f81401542b0 2023-11-06T20:56:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.042+0000 7f81481ff700 1 -- 172.21.15.33:0/1265112452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814014da10 con 0x7f81401542b0 2023-11-06T20:56:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.044+0000 7f81481ff700 1 -- 172.21.15.33:0/1265112452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8140155c40 con 0x7f81401542b0 2023-11-06T20:56:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.044+0000 7f8135ffb700 1 -- 172.21.15.33:0/1265112452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f813001b9e0 con 0x7f81401542b0 2023-11-06T20:56:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.045+0000 7f8135ffb700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f812405f290 0x7f8124061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.045+0000 7f8135ffb700 1 -- 172.21.15.33:0/1265112452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6971+0+0 (secure 0 0 0) 0x7f813007bd00 con 0x7f81401542b0 2023-11-06T20:56:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.045+0000 7f8145f9b700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f812405f290 0x7f8124061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.046+0000 7f8145f9b700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f812405f290 0x7f8124061750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f813c005750 tx=0x7f813c009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.049+0000 7f8135ffb700 1 -- 172.21.15.33:0/1265112452 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8140155c40 con 0x7f81401542b0 2023-11-06T20:56:36.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.319+0000 7f81481ff700 1 -- 172.21.15.33:0/1265112452 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8140155c40 con 0x7f812405f290 2023-11-06T20:56:36.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.323+0000 7f8135ffb700 1 -- 172.21.15.33:0/1265112452 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f8140155c40 con 0x7f812405f290 2023-11-06T20:56:36.324 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-11-06T20:56:36.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 -- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f812405f290 msgr2=0x7f8124061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:36.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f812405f290 0x7f8124061750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f813c005750 tx=0x7f813c009040).stop 2023-11-06T20:56:36.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 -- 172.21.15.33:0/1265112452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81401542b0 msgr2=0x7f8140164320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:36.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81401542b0 0x7f8140164320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f8130009dd0 tx=0x7f8130017000).stop 2023-11-06T20:56:36.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 -- 172.21.15.33:0/1265112452 shutdown_connections 2023-11-06T20:56:36.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f812405f290 0x7f8124061750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:36.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8140153ed0 0x7f814014d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:36.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f814014dd30 0x7f81401565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:36.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 --2- 172.21.15.33:0/1265112452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81401542b0 0x7f8140164320 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:36.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 -- 172.21.15.33:0/1265112452 >> 172.21.15.33:0/1265112452 conn(0x7f814005e1b0 msgr2=0x7f814014ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:36.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 -- 172.21.15.33:0/1265112452 shutdown_connections 2023-11-06T20:56:36.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:36.326+0000 7f81481ff700 1 -- 172.21.15.33:0/1265112452 wait complete. 2023-11-06T20:56:36.343 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-11-06T20:56:36.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.344 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-11-06T20:56:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:36.396 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:36.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.397 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_46_00_UTC) 2023-11-06T20:56:36.452 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:36.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.453 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_47_00_UTC) 2023-11-06T20:56:36.508 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:36.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.509 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_48_00_UTC) 2023-11-06T20:56:36.564 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:36.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.565 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_49_00_UTC) 2023-11-06T20:56:36.620 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:36.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.621 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_50_00_UTC) 2023-11-06T20:56:36.676 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:36.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.676 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_51_00_UTC) 2023-11-06T20:56:36.731 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:36.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.732 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_52_00_UTC) 2023-11-06T20:56:36.787 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:36.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.788 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_53_00_UTC) 2023-11-06T20:56:36.842 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:36.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.843 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_54_00_UTC) 2023-11-06T20:56:36.898 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:36.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.899 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_55_00_UTC) 2023-11-06T20:56:36.955 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:36.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:36.956 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-06-20_56_00_UTC) 2023-11-06T20:56:37.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:37.011 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-11-06T20:56:37.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2023-11-06T20:56:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.290+0000 7f2a31bff700 1 Processor -- start 2023-11-06T20:56:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.290+0000 7f2a31bff700 1 -- start start 2023-11-06T20:56:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.291+0000 7f2a31bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a2c154340 0x7f2a2c0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.291+0000 7f2a31bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2a2c154720 0x7f2a2c158010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.291+0000 7f2a31bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c158990 con 0x7f2a2c158500 2023-11-06T20:56:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.291+0000 7f2a31bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c0c0e00 con 0x7f2a2c154340 2023-11-06T20:56:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.291+0000 7f2a31bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c158bf0 con 0x7f2a2c154720 2023-11-06T20:56:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a2c154340 0x7f2a2c0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2a2c158500 0x7f2a2c063200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55008/0 (socket says 172.21.15.33:55008) 2023-11-06T20:56:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2bfff700 1 -- 172.21.15.33:0/3514864318 learned_addr learned my addr 172.21.15.33:0/3514864318 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2affd700 1 --2- 172.21.15.33:0/3514864318 >> v2:172.21.15.69:3300/0 conn(0x7f2a2c154720 0x7f2a2c158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2a7fc700 1 -- 172.21.15.33:0/3514864318 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329467903 0 0) 0x7f2a2c158990 con 0x7f2a2c158500 2023-11-06T20:56:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2a7fc700 1 -- 172.21.15.33:0/3514864318 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a140033f0 con 0x7f2a2c158500 2023-11-06T20:56:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2b7fe700 1 -- 172.21.15.33:0/3514864318 >> v2:172.21.15.69:3300/0 conn(0x7f2a2c154720 msgr2=0x7f2a2c158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2b7fe700 1 --2- 172.21.15.33:0/3514864318 >> v2:172.21.15.69:3300/0 conn(0x7f2a2c154720 0x7f2a2c158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2b7fe700 1 -- 172.21.15.33:0/3514864318 >> v1:172.21.15.33:6789/0 conn(0x7f2a2c158500 legacy=0x7f2a2c063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2b7fe700 1 -- 172.21.15.33:0/3514864318 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c158ed0 con 0x7f2a2c154340 2023-11-06T20:56:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.292+0000 7f2a2b7fe700 1 --2- 172.21.15.33:0/3514864318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154340 0x7f2a2c0aae40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f2a2c05f0c0 tx=0x7f2a1c009060).ready entity=mon.0 client_cookie=cca3e98b07e705a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a2a7fc700 1 -- 172.21.15.33:0/3514864318 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00d4a0 con 0x7f2a2c154340 2023-11-06T20:56:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a2a7fc700 1 -- 172.21.15.33:0/3514864318 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c00d600 con 0x7f2a2c154340 2023-11-06T20:56:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a2a7fc700 1 -- 172.21.15.33:0/3514864318 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00d980 con 0x7f2a2c154340 2023-11-06T20:56:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a31bff700 1 -- 172.21.15.33:0/3514864318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154340 msgr2=0x7f2a2c0aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a31bff700 1 --2- 172.21.15.33:0/3514864318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154340 0x7f2a2c0aae40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f2a2c05f0c0 tx=0x7f2a1c009060).stop 2023-11-06T20:56:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a31bff700 1 -- 172.21.15.33:0/3514864318 shutdown_connections 2023-11-06T20:56:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a31bff700 1 --2- 172.21.15.33:0/3514864318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154340 0x7f2a2c0aae40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a31bff700 1 --2- 172.21.15.33:0/3514864318 >> v2:172.21.15.69:3300/0 conn(0x7f2a2c154720 0x7f2a2c158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a31bff700 1 -- 172.21.15.33:0/3514864318 >> 172.21.15.33:0/3514864318 conn(0x7f2a2c05ec80 msgr2=0x7f2a2c0cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a31bff700 1 -- 172.21.15.33:0/3514864318 shutdown_connections 2023-11-06T20:56:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.293+0000 7f2a31bff700 1 -- 172.21.15.33:0/3514864318 wait complete. 2023-11-06T20:56:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.295+0000 7f2a31bff700 1 Processor -- start 2023-11-06T20:56:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.295+0000 7f2a31bff700 1 -- start start 2023-11-06T20:56:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a31bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2c154340 0x7f2a2c168850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a31bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154720 0x7f2a2c144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a31bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a2c158500 0x7f2a2c159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a31bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c168d40 con 0x7f2a2c154720 2023-11-06T20:56:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a31bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0af600 con 0x7f2a2c158500 2023-11-06T20:56:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a31bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0af8e0 con 0x7f2a2c154340 2023-11-06T20:56:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a2affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154720 0x7f2a2c144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a2affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154720 0x7f2a2c144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42592/0 (socket says 172.21.15.33:42592) 2023-11-06T20:56:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a2affd700 1 -- 172.21.15.33:0/3117704821 learned_addr learned my addr 172.21.15.33:0/3117704821 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a2bfff700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a2c158500 0x7f2a2c159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a2b7fe700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2c154340 0x7f2a2c168850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a2affd700 1 -- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2c154340 msgr2=0x7f2a2c168850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.296+0000 7f2a2affd700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2c154340 0x7f2a2c168850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.297+0000 7f2a2affd700 1 -- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a2c158500 msgr2=0x7f2a2c159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.297+0000 7f2a2affd700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a2c158500 0x7f2a2c159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.297+0000 7f2a2affd700 1 -- 172.21.15.33:0/3117704821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c0092a0 con 0x7f2a2c154720 2023-11-06T20:56:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.297+0000 7f2a2affd700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154720 0x7f2a2c144db0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f2a2000e900 tx=0x7f2a20009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.297+0000 7f2a137fe700 1 -- 172.21.15.33:0/3117704821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20012600 con 0x7f2a2c154720 2023-11-06T20:56:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.297+0000 7f2a137fe700 1 -- 172.21.15.33:0/3117704821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a20012760 con 0x7f2a2c154720 2023-11-06T20:56:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.297+0000 7f2a137fe700 1 -- 172.21.15.33:0/3117704821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20012ae0 con 0x7f2a2c154720 2023-11-06T20:56:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.297+0000 7f2a31bff700 1 -- 172.21.15.33:0/3117704821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c0afbc0 con 0x7f2a2c154720 2023-11-06T20:56:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.297+0000 7f2a31bff700 1 -- 172.21.15.33:0/3117704821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c0b0180 con 0x7f2a2c154720 2023-11-06T20:56:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.298+0000 7f2a31bff700 1 -- 172.21.15.33:0/3117704821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c0b04f0 con 0x7f2a2c154720 2023-11-06T20:56:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.299+0000 7f2a137fe700 1 -- 172.21.15.33:0/3117704821 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f2a2001b400 con 0x7f2a2c154720 2023-11-06T20:56:37.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.300+0000 7f2a137fe700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1805f2c0 0x7f2a18061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.300+0000 7f2a137fe700 1 -- 172.21.15.33:0/3117704821 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 7035+0+0 (secure 0 0 0) 0x7f2a2007ab80 con 0x7f2a2c154720 2023-11-06T20:56:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.300+0000 7f2a2b7fe700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1805f2c0 0x7f2a18061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.301+0000 7f2a2b7fe700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1805f2c0 0x7f2a18061780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f2a1c009fd0 tx=0x7f2a1c016000).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.304+0000 7f2a137fe700 1 -- 172.21.15.33:0/3117704821 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2a2004b8e0 con 0x7f2a2c154720 2023-11-06T20:56:37.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.533+0000 7f2a31bff700 1 -- 172.21.15.33:0/3117704821 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a2c0c5d80 con 0x7f2a1805f2c0 2023-11-06T20:56:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.541+0000 7f2a137fe700 1 -- 172.21.15.33:0/3117704821 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f2a2c0c5d80 con 0x7f2a1805f2c0 2023-11-06T20:56:37.542 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-11-06T20:56:37.546 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:56:37.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.546+0000 7f2a31bff700 1 -- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1805f2c0 msgr2=0x7f2a18061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:37.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.546+0000 7f2a31bff700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1805f2c0 0x7f2a18061780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f2a1c009fd0 tx=0x7f2a1c016000).stop 2023-11-06T20:56:37.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.546+0000 7f2a31bff700 1 -- 172.21.15.33:0/3117704821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154720 msgr2=0x7f2a2c144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.546+0000 7f2a31bff700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154720 0x7f2a2c144db0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f2a2000e900 tx=0x7f2a20009060).stop 2023-11-06T20:56:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.546+0000 7f2a31bff700 1 -- 172.21.15.33:0/3117704821 shutdown_connections 2023-11-06T20:56:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.546+0000 7f2a31bff700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1805f2c0 0x7f2a18061780 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.546+0000 7f2a31bff700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2c154340 0x7f2a2c168850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.546+0000 7f2a31bff700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a2c158500 0x7f2a2c159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.546+0000 7f2a31bff700 1 --2- 172.21.15.33:0/3117704821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a2c154720 0x7f2a2c144db0 secure :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0x7f2a2000e900 tx=0x7f2a20009060).stop 2023-11-06T20:56:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.547+0000 7f2a31bff700 1 -- 172.21.15.33:0/3117704821 >> 172.21.15.33:0/3117704821 conn(0x7f2a2c05ec80 msgr2=0x7f2a2c0ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:37.554 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:56:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.548+0000 7f2a31bff700 1 -- 172.21.15.33:0/3117704821 shutdown_connections 2023-11-06T20:56:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:37.549+0000 7f2a31bff700 1 -- 172.21.15.33:0/3117704821 wait complete. 2023-11-06T20:56:37.584 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-11-06T20:56:37.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:37.585 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-11-06T20:56:37.675 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:37.675 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:37.676 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:37.676 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:37.676 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:37.676 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:37.676 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:37.677 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:37.677 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:37.677 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:37.677 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:37.679 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:37.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:37.680 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_46_00_UTC) 2023-11-06T20:56:37.777 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:37.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:37.778 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_47_00_UTC) 2023-11-06T20:56:37.834 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:37.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:37.835 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_48_00_UTC) 2023-11-06T20:56:37.894 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:37.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:37.895 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_49_00_UTC) 2023-11-06T20:56:37.950 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:37.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:37.951 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_50_00_UTC) 2023-11-06T20:56:38.006 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:38.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:38.007 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_51_00_UTC) 2023-11-06T20:56:38.062 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:38.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:38.063 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_52_00_UTC) 2023-11-06T20:56:38.157 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:38.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:38.158 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_53_00_UTC) 2023-11-06T20:56:38.220 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:38.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:38.221 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_54_00_UTC) 2023-11-06T20:56:38.275 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:38.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:38.276 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_55_00_UTC) 2023-11-06T20:56:38.331 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:38.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:38.332 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-06-20_56_00_UTC) 2023-11-06T20:56:38.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:38.386 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-11-06T20:56:38.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2023-11-06T20:56:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.669+0000 7f6c459ff700 1 Processor -- start 2023-11-06T20:56:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.669+0000 7f6c459ff700 1 -- start start 2023-11-06T20:56:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c459ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c40154720 0x7f6c400aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c459ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c40158720 0x7f6c40063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c459ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c400c0e00 con 0x7f6c40154340 2023-11-06T20:56:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c459ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c40158bf0 con 0x7f6c40154720 2023-11-06T20:56:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c459ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c40158ed0 con 0x7f6c40158720 2023-11-06T20:56:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c3e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c40154720 0x7f6c400aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c3effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6c40154340 0x7f6c40158010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50542/0 (socket says 172.21.15.33:50542) 2023-11-06T20:56:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c3effd700 1 -- 172.21.15.33:0/3021629400 learned_addr learned my addr 172.21.15.33:0/3021629400 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c3f7fe700 1 --2- 172.21.15.33:0/3021629400 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c40158720 0x7f6c40063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c3e7fc700 1 -- 172.21.15.33:0/3021629400 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c40158720 msgr2=0x7f6c40063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c3e7fc700 1 --2- 172.21.15.33:0/3021629400 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c40158720 0x7f6c40063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.671+0000 7f6c3e7fc700 1 -- 172.21.15.33:0/3021629400 >> v1:172.21.15.33:6789/0 conn(0x7f6c40154340 legacy=0x7f6c40158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.672+0000 7f6c3e7fc700 1 -- 172.21.15.33:0/3021629400 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400c5d80 con 0x7f6c40154720 2023-11-06T20:56:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.672+0000 7f6c3dffb700 1 -- 172.21.15.33:0/3021629400 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804836606 0 0) 0x7f6c400c0e00 con 0x7f6c40154340 2023-11-06T20:56:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.672+0000 7f6c3e7fc700 1 --2- 172.21.15.33:0/3021629400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 0x7f6c400aae40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f6c34009760 tx=0x7f6c34009060).ready entity=mon.0 client_cookie=29bb2b74d757db47 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.672+0000 7f6c3dffb700 1 -- 172.21.15.33:0/3021629400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400b580 con 0x7f6c40154720 2023-11-06T20:56:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.672+0000 7f6c3dffb700 1 -- 172.21.15.33:0/3021629400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3400b6e0 con 0x7f6c40154720 2023-11-06T20:56:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.672+0000 7f6c3dffb700 1 -- 172.21.15.33:0/3021629400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400ba60 con 0x7f6c40154720 2023-11-06T20:56:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.672+0000 7f6c459ff700 1 -- 172.21.15.33:0/3021629400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 msgr2=0x7f6c400aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.672+0000 7f6c459ff700 1 --2- 172.21.15.33:0/3021629400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 0x7f6c400aae40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f6c34009760 tx=0x7f6c34009060).stop 2023-11-06T20:56:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.673+0000 7f6c459ff700 1 -- 172.21.15.33:0/3021629400 shutdown_connections 2023-11-06T20:56:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.673+0000 7f6c459ff700 1 --2- 172.21.15.33:0/3021629400 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c40158720 0x7f6c40063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.673+0000 7f6c459ff700 1 --2- 172.21.15.33:0/3021629400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 0x7f6c400aae40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.673+0000 7f6c459ff700 1 -- 172.21.15.33:0/3021629400 >> 172.21.15.33:0/3021629400 conn(0x7f6c4005ec80 msgr2=0x7f6c400cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.673+0000 7f6c459ff700 1 -- 172.21.15.33:0/3021629400 shutdown_connections 2023-11-06T20:56:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.673+0000 7f6c459ff700 1 -- 172.21.15.33:0/3021629400 wait complete. 2023-11-06T20:56:38.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.674+0000 7f6c459ff700 1 Processor -- start 2023-11-06T20:56:38.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.674+0000 7f6c459ff700 1 -- start start 2023-11-06T20:56:38.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.675+0000 7f6c459ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c40154340 0x7f6c401558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:38.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.675+0000 7f6c459ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 0x7f6c4014cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:38.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.675+0000 7f6c459ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c40158720 0x7f6c40156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:38.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.675+0000 7f6c459ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400b0540 con 0x7f6c40154720 2023-11-06T20:56:38.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.675+0000 7f6c459ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c40155460 con 0x7f6c40158720 2023-11-06T20:56:38.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.675+0000 7f6c459ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c40155de0 con 0x7f6c40154340 2023-11-06T20:56:38.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.675+0000 7f6c3e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 0x7f6c4014cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:38.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.675+0000 7f6c3e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 0x7f6c4014cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40270/0 (socket says 172.21.15.33:40270) 2023-11-06T20:56:38.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.675+0000 7f6c3e7fc700 1 -- 172.21.15.33:0/159276645 learned_addr learned my addr 172.21.15.33:0/159276645 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:38.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c3e7fc700 1 -- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c40154340 msgr2=0x7f6c401558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c3e7fc700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c40154340 0x7f6c401558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c3e7fc700 1 -- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c40158720 msgr2=0x7f6c40156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c3e7fc700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c40158720 0x7f6c40156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c3e7fc700 1 -- 172.21.15.33:0/159276645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340092a0 con 0x7f6c40154720 2023-11-06T20:56:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c3e7fc700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 0x7f6c4014cb30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f6c34011000 tx=0x7f6c34017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c26ffd700 1 -- 172.21.15.33:0/159276645 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400b850 con 0x7f6c40154720 2023-11-06T20:56:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c26ffd700 1 -- 172.21.15.33:0/159276645 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3400b9b0 con 0x7f6c40154720 2023-11-06T20:56:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c459ff700 1 -- 172.21.15.33:0/159276645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c401570b0 con 0x7f6c40154720 2023-11-06T20:56:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c26ffd700 1 -- 172.21.15.33:0/159276645 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3401b660 con 0x7f6c40154720 2023-11-06T20:56:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.676+0000 7f6c459ff700 1 -- 172.21.15.33:0/159276645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c40154e10 con 0x7f6c40154720 2023-11-06T20:56:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.678+0000 7f6c459ff700 1 -- 172.21.15.33:0/159276645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c400c6060 con 0x7f6c40154720 2023-11-06T20:56:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.678+0000 7f6c26ffd700 1 -- 172.21.15.33:0/159276645 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f6c3401ba10 con 0x7f6c40154720 2023-11-06T20:56:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.679+0000 7f6c26ffd700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c2c05f2c0 0x7f6c2c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.679+0000 7f6c26ffd700 1 -- 172.21.15.33:0/159276645 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 7051+0+0 (secure 0 0 0) 0x7f6c3407c430 con 0x7f6c40154720 2023-11-06T20:56:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.679+0000 7f6c3effd700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c2c05f2c0 0x7f6c2c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.680+0000 7f6c3effd700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c2c05f2c0 0x7f6c2c061780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6c300011e0 tx=0x7f6c30007850).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.683+0000 7f6c26ffd700 1 -- 172.21.15.33:0/159276645 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6c3404d120 con 0x7f6c40154720 2023-11-06T20:56:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.891+0000 7f6c459ff700 1 -- 172.21.15.33:0/159276645 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c401555d0 con 0x7f6c2c05f2c0 2023-11-06T20:56:38.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.895+0000 7f6c26ffd700 1 -- 172.21.15.33:0/159276645 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f6c401555d0 con 0x7f6c2c05f2c0 2023-11-06T20:56:38.896 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-11-06T20:56:38.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 -- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c2c05f2c0 msgr2=0x7f6c2c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:38.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c2c05f2c0 0x7f6c2c061780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6c300011e0 tx=0x7f6c30007850).stop 2023-11-06T20:56:38.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 -- 172.21.15.33:0/159276645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 msgr2=0x7f6c4014cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:38.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 0x7f6c4014cb30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f6c34011000 tx=0x7f6c34017000).stop 2023-11-06T20:56:38.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 -- 172.21.15.33:0/159276645 shutdown_connections 2023-11-06T20:56:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c2c05f2c0 0x7f6c2c061780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c40154340 0x7f6c401558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c40158720 0x7f6c40156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 --2- 172.21.15.33:0/159276645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c40154720 0x7f6c4014cb30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 -- 172.21.15.33:0/159276645 >> 172.21.15.33:0/159276645 conn(0x7f6c4005ec80 msgr2=0x7f6c400ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:38.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 -- 172.21.15.33:0/159276645 shutdown_connections 2023-11-06T20:56:38.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:38.897+0000 7f6c459ff700 1 -- 172.21.15.33:0/159276645 wait complete. 2023-11-06T20:56:38.914 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-11-06T20:56:38.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:38.915 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-11-06T20:56:38.964 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:38.965 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:38.965 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:38.965 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:38.965 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:38.967 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:38.967 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:38.968 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:38.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:38.969 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_46_00_UTC) 2023-11-06T20:56:39.025 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:39.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.026 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_47_00_UTC) 2023-11-06T20:56:39.081 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:39.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.082 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_48_00_UTC) 2023-11-06T20:56:39.157 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:39.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.158 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_49_00_UTC) 2023-11-06T20:56:39.217 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:39.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.218 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_50_00_UTC) 2023-11-06T20:56:39.295 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:39.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.296 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_51_00_UTC) 2023-11-06T20:56:39.352 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:39.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.353 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_52_00_UTC) 2023-11-06T20:56:39.409 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:39.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.410 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_53_00_UTC) 2023-11-06T20:56:39.465 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:39.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.466 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_54_00_UTC) 2023-11-06T20:56:39.521 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:39.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.522 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_55_00_UTC) 2023-11-06T20:56:39.576 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:39.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.577 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-06-20_56_00_UTC) 2023-11-06T20:56:39.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:39.632 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-11-06T20:56:39.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2023-11-06T20:56:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.885+0000 7fc8161ff700 1 Processor -- start 2023-11-06T20:56:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.885+0000 7fc8161ff700 1 -- start start 2023-11-06T20:56:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.886+0000 7fc8161ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc81005e3c0 0x7fc810069000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.886+0000 7fc8161ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8100634c0 con 0x7fc810068c20 2023-11-06T20:56:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.886+0000 7fc8161ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8100c4600 con 0x7fc810069ac0 2023-11-06T20:56:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.886+0000 7fc8161ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8100628b0 con 0x7fc81005e3c0 2023-11-06T20:56:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.886+0000 7fc80f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc810068c20 0x7fc810063950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50552/0 (socket says 172.21.15.33:50552) 2023-11-06T20:56:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.886+0000 7fc80f7fe700 1 -- 172.21.15.33:0/2665505257 learned_addr learned my addr 172.21.15.33:0/2665505257 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:39.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592430313 0 0) 0x7fc8100634c0 con 0x7fc810068c20 2023-11-06T20:56:39.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80effd700 1 --2- 172.21.15.33:0/2665505257 >> v2:172.21.15.69:3300/0 conn(0x7fc81005e3c0 0x7fc810069000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:39.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f80033f0 con 0x7fc810068c20 2023-11-06T20:56:39.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3499208973 0 0) 0x7fc7f80033f0 con 0x7fc810068c20 2023-11-06T20:56:39.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 >> v2:172.21.15.69:3300/0 conn(0x7fc81005e3c0 msgr2=0x7fc810069000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:39.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 --2- 172.21.15.33:0/2665505257 >> v2:172.21.15.69:3300/0 conn(0x7fc81005e3c0 0x7fc810069000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:39.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 >> v1:172.21.15.69:6789/0 conn(0x7fc810069ac0 legacy=0x7fc8100bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:39.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc810062b90 con 0x7fc810068c20 2023-11-06T20:56:39.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fc800001470 con 0x7fc810068c20 2023-11-06T20:56:39.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 >> v1:172.21.15.33:6789/0 conn(0x7fc810068c20 legacy=0x7fc810063950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:39.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7f80038e0 0x7fc7f8003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:39.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f8004690 0x7fc7f8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:39.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7f8005000 0x7fc7f800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:39.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100628b0 con 0x7fc7f8004690 2023-11-06T20:56:39.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f800be00 con 0x7fc7f8005000 2023-11-06T20:56:39.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f800bfc0 con 0x7fc7f80038e0 2023-11-06T20:56:39.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.887+0000 7fc80effd700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f8004690 0x7fc7f8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:39.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80f7fe700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7f80038e0 0x7fc7f8003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:39.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80ffff700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7f8005000 0x7fc7f800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:39.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80effd700 1 -- 172.21.15.33:0/2665505257 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7f80038e0 msgr2=0x7fc7f8003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:39.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80effd700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7f80038e0 0x7fc7f8003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:39.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80effd700 1 -- 172.21.15.33:0/2665505257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7f8005000 msgr2=0x7fc7f800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:39.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80effd700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7f8005000 0x7fc7f800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:39.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80effd700 1 -- 172.21.15.33:0/2665505257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80033f0 con 0x7fc7f8004690 2023-11-06T20:56:39.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80f7fe700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7f80038e0 0x7fc7f8003cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:39.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80ffff700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7f8005000 0x7fc7f800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:39.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80ffff700 1 -- 172.21.15.33:0/2665505257 reap_dead start 2023-11-06T20:56:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80effd700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f8004690 0x7fc7f8004b10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fc81005e840 tx=0x7fc804009060).ready entity=mon.0 client_cookie=8212927e1a572363 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804009e90 con 0x7fc7f8004690 2023-11-06T20:56:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc804009430 con 0x7fc7f8004690 2023-11-06T20:56:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.888+0000 7fc80e7fc700 1 -- 172.21.15.33:0/2665505257 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80400e850 con 0x7fc7f8004690 2023-11-06T20:56:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.889+0000 7fc8161ff700 1 -- 172.21.15.33:0/2665505257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f8004690 msgr2=0x7fc7f8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:39.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.889+0000 7fc8161ff700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f8004690 0x7fc7f8004b10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fc81005e840 tx=0x7fc804009060).stop 2023-11-06T20:56:39.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.889+0000 7fc8161ff700 1 -- 172.21.15.33:0/2665505257 shutdown_connections 2023-11-06T20:56:39.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.889+0000 7fc8161ff700 1 --2- 172.21.15.33:0/2665505257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f8004690 0x7fc7f8004b10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:39.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.889+0000 7fc8161ff700 1 -- 172.21.15.33:0/2665505257 >> 172.21.15.33:0/2665505257 conn(0x7fc8100c7d10 msgr2=0x7fc8100c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:39.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.889+0000 7fc8161ff700 1 -- 172.21.15.33:0/2665505257 shutdown_connections 2023-11-06T20:56:39.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.889+0000 7fc8161ff700 1 -- 172.21.15.33:0/2665505257 wait complete. 2023-11-06T20:56:39.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.890+0000 7fc8161ff700 1 Processor -- start 2023-11-06T20:56:39.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.891+0000 7fc8161ff700 1 -- start start 2023-11-06T20:56:39.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.891+0000 7fc8161ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7f8004690 0x7fc81013cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:39.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.891+0000 7fc8161ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc81013d1d0 0x7fc810151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:39.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc8161ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8101523a0 0x7fc8100be330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:39.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc8161ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810167ab0 con 0x7fc8101523a0 2023-11-06T20:56:39.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc8161ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810168350 con 0x7fc81013d1d0 2023-11-06T20:56:39.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc8161ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc810152800 con 0x7fc7f8004690 2023-11-06T20:56:39.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8101523a0 0x7fc8100be330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:39.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8101523a0 0x7fc8100be330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40286/0 (socket says 172.21.15.33:40286) 2023-11-06T20:56:39.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80ffff700 1 -- 172.21.15.33:0/2612158963 learned_addr learned my addr 172.21.15.33:0/2612158963 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:39.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80f7fe700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7f8004690 0x7fc81013cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:39.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80effd700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc81013d1d0 0x7fc810151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:39.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80ffff700 1 -- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7f8004690 msgr2=0x7fc81013cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:39.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80ffff700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7f8004690 0x7fc81013cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:39.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80ffff700 1 -- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc81013d1d0 msgr2=0x7fc810151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:39.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80ffff700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc81013d1d0 0x7fc810151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:39.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80ffff700 1 -- 172.21.15.33:0/2612158963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8100bf060 con 0x7fc8101523a0 2023-11-06T20:56:39.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.892+0000 7fc80ffff700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8101523a0 0x7fc8100be330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc81005f360 tx=0x7fc808009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:39.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.893+0000 7fc7f77fe700 1 -- 172.21.15.33:0/2612158963 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808010520 con 0x7fc8101523a0 2023-11-06T20:56:39.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.893+0000 7fc7f77fe700 1 -- 172.21.15.33:0/2612158963 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808010680 con 0x7fc8101523a0 2023-11-06T20:56:39.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.893+0000 7fc7f77fe700 1 -- 172.21.15.33:0/2612158963 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808010a40 con 0x7fc8101523a0 2023-11-06T20:56:39.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.893+0000 7fc8161ff700 1 -- 172.21.15.33:0/2612158963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc810167d00 con 0x7fc8101523a0 2023-11-06T20:56:39.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.893+0000 7fc8161ff700 1 -- 172.21.15.33:0/2612158963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8101687e0 con 0x7fc8101523a0 2023-11-06T20:56:39.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.894+0000 7fc8161ff700 1 -- 172.21.15.33:0/2612158963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8101689d0 con 0x7fc8101523a0 2023-11-06T20:56:39.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.895+0000 7fc7f77fe700 1 -- 172.21.15.33:0/2612158963 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fc808003070 con 0x7fc8101523a0 2023-11-06T20:56:39.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.896+0000 7fc7f77fe700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7fc05f290 0x7fc7fc061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:39.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.896+0000 7fc7f77fe700 1 -- 172.21.15.33:0/2612158963 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6971+0+0 (secure 0 0 0) 0x7fc80807a1a0 con 0x7fc8101523a0 2023-11-06T20:56:39.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.896+0000 7fc80f7fe700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7fc05f290 0x7fc7fc061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:39.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.897+0000 7fc80f7fe700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7fc05f290 0x7fc7fc061750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc800000e10 tx=0x7fc800009040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:39.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:39.900+0000 7fc7f77fe700 1 -- 172.21.15.33:0/2612158963 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc808049af0 con 0x7fc8101523a0 2023-11-06T20:56:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.109+0000 7fc8161ff700 1 -- 172.21.15.33:0/2612158963 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8101684e0 con 0x7fc7fc05f290 2023-11-06T20:56:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.117+0000 7fc7f77fe700 1 -- 172.21.15.33:0/2612158963 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc8101684e0 con 0x7fc7fc05f290 2023-11-06T20:56:40.118 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-11-06T20:56:40.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.121+0000 7fc8161ff700 1 -- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7fc05f290 msgr2=0x7fc7fc061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:40.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.121+0000 7fc8161ff700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7fc05f290 0x7fc7fc061750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc800000e10 tx=0x7fc800009040).stop 2023-11-06T20:56:40.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.121+0000 7fc8161ff700 1 -- 172.21.15.33:0/2612158963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8101523a0 msgr2=0x7fc8100be330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:40.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.121+0000 7fc8161ff700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8101523a0 0x7fc8100be330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc81005f360 tx=0x7fc808009040).stop 2023-11-06T20:56:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.121+0000 7fc8161ff700 1 -- 172.21.15.33:0/2612158963 shutdown_connections 2023-11-06T20:56:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.121+0000 7fc8161ff700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7fc05f290 0x7fc7fc061750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.121+0000 7fc8161ff700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7f8004690 0x7fc81013cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.121+0000 7fc8161ff700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8101523a0 0x7fc8100be330 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.121+0000 7fc8161ff700 1 --2- 172.21.15.33:0/2612158963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc81013d1d0 0x7fc810151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.121+0000 7fc8161ff700 1 -- 172.21.15.33:0/2612158963 >> 172.21.15.33:0/2612158963 conn(0x7fc8100c7d10 msgr2=0x7fc8100c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.122+0000 7fc8161ff700 1 -- 172.21.15.33:0/2612158963 shutdown_connections 2023-11-06T20:56:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:40.122+0000 7fc8161ff700 1 -- 172.21.15.33:0/2612158963 wait complete. 2023-11-06T20:56:40.155 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-11-06T20:56:40.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.157 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-11-06T20:56:40.227 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:40.228 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:40.228 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:40.228 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:40.228 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:40.228 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:40.229 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:40.229 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:40.229 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:40.229 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:40.229 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:40.230 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:40.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.231 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_46_00_UTC) 2023-11-06T20:56:40.289 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:40.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.290 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_47_00_UTC) 2023-11-06T20:56:40.347 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:40.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.347 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_48_00_UTC) 2023-11-06T20:56:40.402 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:40.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.403 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_49_00_UTC) 2023-11-06T20:56:40.458 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:40.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.459 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_50_00_UTC) 2023-11-06T20:56:40.515 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:40.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.516 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_51_00_UTC) 2023-11-06T20:56:40.571 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:40.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.572 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_52_00_UTC) 2023-11-06T20:56:40.628 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:40.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.629 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_53_00_UTC) 2023-11-06T20:56:40.684 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:40.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.685 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_54_00_UTC) 2023-11-06T20:56:40.742 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:40.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.743 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_55_00_UTC) 2023-11-06T20:56:40.799 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:40.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.800 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-06-20_56_00_UTC) 2023-11-06T20:56:40.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:40.857 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-11-06T20:56:40.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2023-11-06T20:56:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.111+0000 7f8b767ff700 1 Processor -- start 2023-11-06T20:56:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.111+0000 7f8b767ff700 1 -- start start 2023-11-06T20:56:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.114+0000 7f8b767ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b7013a960 0x7f8b70155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.114+0000 7f8b767ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8b700c9340 0x7f8b70066490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.114+0000 7f8b767ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b70066980 0x7f8b700679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.114+0000 7f8b767ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b70064cc0 con 0x7f8b7013a960 2023-11-06T20:56:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.114+0000 7f8b767ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b70155930 con 0x7f8b700c9340 2023-11-06T20:56:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.114+0000 7f8b767ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c3360 con 0x7f8b70066980 2023-11-06T20:56:41.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.115+0000 7f8b6f7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8b700c9340 0x7f8b70066490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:41.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.115+0000 7f8b6f7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8b700c9340 0x7f8b70066490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:40338/0 (socket says 172.21.15.33:40338) 2023-11-06T20:56:41.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.115+0000 7f8b6f7fe700 1 -- 172.21.15.33:0/2592094643 learned_addr learned my addr 172.21.15.33:0/2592094643 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:41.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.115+0000 7f8b74d9c700 1 --2- 172.21.15.33:0/2592094643 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b70066980 0x7f8b700679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.115+0000 7f8b6ffff700 1 --2- 172.21.15.33:0/2592094643 >> v2:172.21.15.33:3300/0 conn(0x7f8b7013a960 0x7f8b70155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.116+0000 7f8b6f7fe700 1 -- 172.21.15.33:0/2592094643 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b70066980 msgr2=0x7f8b700679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.116+0000 7f8b6f7fe700 1 --2- 172.21.15.33:0/2592094643 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b70066980 0x7f8b700679f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.116+0000 7f8b6f7fe700 1 -- 172.21.15.33:0/2592094643 >> v2:172.21.15.33:3300/0 conn(0x7f8b7013a960 msgr2=0x7f8b70155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.116+0000 7f8b6f7fe700 1 --2- 172.21.15.33:0/2592094643 >> v2:172.21.15.33:3300/0 conn(0x7f8b7013a960 0x7f8b70155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.116+0000 7f8b6f7fe700 1 -- 172.21.15.33:0/2592094643 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70066e10 con 0x7f8b700c9340 2023-11-06T20:56:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.116+0000 7f8b6ffff700 1 --2- 172.21.15.33:0/2592094643 >> v2:172.21.15.33:3300/0 conn(0x7f8b7013a960 0x7f8b70155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.116+0000 7f8b6f7fe700 1 --2- 172.21.15.33:0/2592094643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b700c9340 0x7f8b70066490 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f8b7006bd80 tx=0x7f8b640090f0).ready entity=mon.1 client_cookie=1553ffd9a63498e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.117+0000 7f8b6effd700 1 -- 172.21.15.33:0/2592094643 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400b520 con 0x7f8b700c9340 2023-11-06T20:56:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.117+0000 7f8b6effd700 1 -- 172.21.15.33:0/2592094643 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6400b680 con 0x7f8b700c9340 2023-11-06T20:56:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.117+0000 7f8b6effd700 1 -- 172.21.15.33:0/2592094643 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400ba00 con 0x7f8b700c9340 2023-11-06T20:56:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.118+0000 7f8b767ff700 1 -- 172.21.15.33:0/2592094643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b700c9340 msgr2=0x7f8b70066490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.118+0000 7f8b767ff700 1 --2- 172.21.15.33:0/2592094643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b700c9340 0x7f8b70066490 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f8b7006bd80 tx=0x7f8b640090f0).stop 2023-11-06T20:56:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.119+0000 7f8b767ff700 1 -- 172.21.15.33:0/2592094643 shutdown_connections 2023-11-06T20:56:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.119+0000 7f8b767ff700 1 --2- 172.21.15.33:0/2592094643 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b70066980 0x7f8b700679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.119+0000 7f8b767ff700 1 --2- 172.21.15.33:0/2592094643 >> v2:172.21.15.33:3300/0 conn(0x7f8b7013a960 0x7f8b70155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.119+0000 7f8b767ff700 1 --2- 172.21.15.33:0/2592094643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b700c9340 0x7f8b70066490 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.119+0000 7f8b767ff700 1 -- 172.21.15.33:0/2592094643 >> 172.21.15.33:0/2592094643 conn(0x7f8b7005ec70 msgr2=0x7f8b700c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.119+0000 7f8b767ff700 1 -- 172.21.15.33:0/2592094643 shutdown_connections 2023-11-06T20:56:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.119+0000 7f8b767ff700 1 -- 172.21.15.33:0/2592094643 wait complete. 2023-11-06T20:56:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.123+0000 7f8b767ff700 1 Processor -- start 2023-11-06T20:56:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.123+0000 7f8b767ff700 1 -- start start 2023-11-06T20:56:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.125+0000 7f8b767ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b70066980 0x7f8b700adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.125+0000 7f8b767ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b700c9340 0x7f8b700a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.125+0000 7f8b767ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b7013a960 0x7f8b701397e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.125+0000 7f8b767ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700a93f0 con 0x7f8b7013a960 2023-11-06T20:56:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.125+0000 7f8b767ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700a9600 con 0x7f8b700c9340 2023-11-06T20:56:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.125+0000 7f8b767ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700a98e0 con 0x7f8b70066980 2023-11-06T20:56:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.125+0000 7f8b74d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b7013a960 0x7f8b701397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.125+0000 7f8b74d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b7013a960 0x7f8b701397e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40300/0 (socket says 172.21.15.33:40300) 2023-11-06T20:56:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.125+0000 7f8b74d9c700 1 -- 172.21.15.33:0/3328502587 learned_addr learned my addr 172.21.15.33:0/3328502587 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.126+0000 7f8b6ffff700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b70066980 0x7f8b700adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.126+0000 7f8b6f7fe700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b700c9340 0x7f8b700a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.127+0000 7f8b74d9c700 1 -- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b70066980 msgr2=0x7f8b700adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.127+0000 7f8b74d9c700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b70066980 0x7f8b700adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.127+0000 7f8b74d9c700 1 -- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b700c9340 msgr2=0x7f8b700a8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.127+0000 7f8b74d9c700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b700c9340 0x7f8b700a8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.127+0000 7f8b74d9c700 1 -- 172.21.15.33:0/3328502587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b64009370 con 0x7f8b7013a960 2023-11-06T20:56:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.128+0000 7f8b74d9c700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b7013a960 0x7f8b701397e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f8b68009e90 tx=0x7f8b6800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.129+0000 7f8b57fff700 1 -- 172.21.15.33:0/3328502587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800cc20 con 0x7f8b7013a960 2023-11-06T20:56:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.129+0000 7f8b57fff700 1 -- 172.21.15.33:0/3328502587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6800f040 con 0x7f8b7013a960 2023-11-06T20:56:41.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.129+0000 7f8b767ff700 1 -- 172.21.15.33:0/3328502587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b700a9bc0 con 0x7f8b7013a960 2023-11-06T20:56:41.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.130+0000 7f8b767ff700 1 -- 172.21.15.33:0/3328502587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b700aa180 con 0x7f8b7013a960 2023-11-06T20:56:41.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.132+0000 7f8b57fff700 1 -- 172.21.15.33:0/3328502587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68013400 con 0x7f8b7013a960 2023-11-06T20:56:41.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.133+0000 7f8b767ff700 1 -- 172.21.15.33:0/3328502587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b50005120 con 0x7f8b7013a960 2023-11-06T20:56:41.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.134+0000 7f8b57fff700 1 -- 172.21.15.33:0/3328502587 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f8b680137b0 con 0x7f8b7013a960 2023-11-06T20:56:41.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.136+0000 7f8b57fff700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b5c05f310 0x7f8b5c0617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.137+0000 7f8b57fff700 1 -- 172.21.15.33:0/3328502587 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7019+0+0 (secure 0 0 0) 0x7f8b6807af70 con 0x7f8b7013a960 2023-11-06T20:56:41.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.139+0000 7f8b6ffff700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b5c05f310 0x7f8b5c0617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.143+0000 7f8b6ffff700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b5c05f310 0x7f8b5c0617d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f8b60006d80 tx=0x7f8b60008040).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:41.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.147+0000 7f8b57fff700 1 -- 172.21.15.33:0/3328502587 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8b680a9af0 con 0x7f8b7013a960 2023-11-06T20:56:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.370+0000 7f8b767ff700 1 -- 172.21.15.33:0/3328502587 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b50000bc0 con 0x7f8b5c05f310 2023-11-06T20:56:41.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.374+0000 7f8b57fff700 1 -- 172.21.15.33:0/3328502587 <== mgr.4823 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f8b50000bc0 con 0x7f8b5c05f310 2023-11-06T20:56:41.374 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-11-06T20:56:41.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 -- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b5c05f310 msgr2=0x7f8b5c0617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:41.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b5c05f310 0x7f8b5c0617d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f8b60006d80 tx=0x7f8b60008040).stop 2023-11-06T20:56:41.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 -- 172.21.15.33:0/3328502587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b7013a960 msgr2=0x7f8b701397e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:41.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b7013a960 0x7f8b701397e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f8b68009e90 tx=0x7f8b6800b040).stop 2023-11-06T20:56:41.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 -- 172.21.15.33:0/3328502587 shutdown_connections 2023-11-06T20:56:41.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b5c05f310 0x7f8b5c0617d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b70066980 0x7f8b700adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b7013a960 0x7f8b701397e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 --2- 172.21.15.33:0/3328502587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b700c9340 0x7f8b700a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 -- 172.21.15.33:0/3328502587 >> 172.21.15.33:0/3328502587 conn(0x7f8b7005ec70 msgr2=0x7f8b700c06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 -- 172.21.15.33:0/3328502587 shutdown_connections 2023-11-06T20:56:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:41.376+0000 7f8b767ff700 1 -- 172.21.15.33:0/3328502587 wait complete. 2023-11-06T20:56:41.404 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-11-06T20:56:41.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:41.405 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-11-06T20:56:41.469 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:41.469 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:41.470 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:41.470 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:41.470 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:41.470 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:41.470 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:41.471 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:41.471 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:41.471 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:41.471 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:41.472 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_46_00_UTC 2023-11-06T20:56:41.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:41.473 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_46_00_UTC) 2023-11-06T20:56:41.528 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_47_00_UTC 2023-11-06T20:56:41.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:41.529 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_47_00_UTC) 2023-11-06T20:56:41.585 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_48_00_UTC 2023-11-06T20:56:41.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:41.586 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_48_00_UTC) 2023-11-06T20:56:41.642 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_49_00_UTC 2023-11-06T20:56:41.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:41.643 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_49_00_UTC) 2023-11-06T20:56:41.699 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_50_00_UTC 2023-11-06T20:56:41.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:41.700 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_50_00_UTC) 2023-11-06T20:56:41.756 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_51_00_UTC 2023-11-06T20:56:41.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:41.757 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_51_00_UTC) 2023-11-06T20:56:41.812 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_52_00_UTC 2023-11-06T20:56:41.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:41.813 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_52_00_UTC) 2023-11-06T20:56:41.867 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_53_00_UTC 2023-11-06T20:56:41.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:41.868 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_53_00_UTC) 2023-11-06T20:56:41.923 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_54_00_UTC 2023-11-06T20:56:41.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:41.923 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_54_00_UTC) 2023-11-06T20:56:42.027 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_55_00_UTC 2023-11-06T20:56:42.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:42.027 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_55_00_UTC) 2023-11-06T20:56:42.099 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_56_00_UTC 2023-11-06T20:56:42.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:42.100 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-06-20_56_00_UTC) 2023-11-06T20:56:42.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:42.179 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-11-06T20:56:42.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T20:56:42.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.435+0000 7fa1583ff700 1 Processor -- start 2023-11-06T20:56:42.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.435+0000 7fa1583ff700 1 -- start start 2023-11-06T20:56:42.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.437+0000 7fa1583ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa150066620 0x7fa1500c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:42.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.437+0000 7fa1583ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1500a8330 con 0x7fa150158780 2023-11-06T20:56:42.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.437+0000 7fa1583ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1500bdc50 con 0x7fa150066a00 2023-11-06T20:56:42.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.437+0000 7fa1583ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1500be210 con 0x7fa150066620 2023-11-06T20:56:42.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.437+0000 7fa15619b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa150066620 0x7fa1500c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:42.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.437+0000 7fa15699c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa150158780 0x7fa1500681e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50564/0 (socket says 172.21.15.33:50564) 2023-11-06T20:56:42.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.437+0000 7fa15699c700 1 -- 172.21.15.33:0/2332346380 learned_addr learned my addr 172.21.15.33:0/2332346380 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:42.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.437+0000 7fa15619b700 1 -- 172.21.15.33:0/2332346380 >> v1:172.21.15.69:6789/0 conn(0x7fa150066a00 legacy=0x7fa150067ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:42.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.438+0000 7fa15619b700 1 -- 172.21.15.33:0/2332346380 >> v1:172.21.15.33:6789/0 conn(0x7fa150158780 legacy=0x7fa1500681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:42.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.437+0000 7fa155199700 1 -- 172.21.15.33:0/2332346380 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502789396 0 0) 0x7fa1500bdc50 con 0x7fa150066a00 2023-11-06T20:56:42.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.438+0000 7fa15619b700 1 -- 172.21.15.33:0/2332346380 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500bdf30 con 0x7fa150066620 2023-11-06T20:56:42.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.438+0000 7fa15619b700 1 --2- 172.21.15.33:0/2332346380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 0x7fa1500c5110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fa14c009760 tx=0x7fa14c009060).ready entity=mon.0 client_cookie=c9a9626bf856935 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:42.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.438+0000 7fa155199700 1 -- 172.21.15.33:0/2332346380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00f510 con 0x7fa150066620 2023-11-06T20:56:42.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.438+0000 7fa155199700 1 -- 172.21.15.33:0/2332346380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c00f670 con 0x7fa150066620 2023-11-06T20:56:42.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.438+0000 7fa155199700 1 -- 172.21.15.33:0/2332346380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00f9f0 con 0x7fa150066620 2023-11-06T20:56:42.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.438+0000 7fa1583ff700 1 -- 172.21.15.33:0/2332346380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 msgr2=0x7fa1500c5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:42.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.438+0000 7fa1583ff700 1 --2- 172.21.15.33:0/2332346380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 0x7fa1500c5110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fa14c009760 tx=0x7fa14c009060).stop 2023-11-06T20:56:42.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.439+0000 7fa1583ff700 1 -- 172.21.15.33:0/2332346380 shutdown_connections 2023-11-06T20:56:42.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.439+0000 7fa1583ff700 1 --2- 172.21.15.33:0/2332346380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 0x7fa1500c5110 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:42.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.439+0000 7fa1583ff700 1 -- 172.21.15.33:0/2332346380 >> 172.21.15.33:0/2332346380 conn(0x7fa1500c8c50 msgr2=0x7fa1500c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.439+0000 7fa1583ff700 1 -- 172.21.15.33:0/2332346380 shutdown_connections 2023-11-06T20:56:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.439+0000 7fa1583ff700 1 -- 172.21.15.33:0/2332346380 wait complete. 2023-11-06T20:56:42.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.441+0000 7fa1583ff700 1 Processor -- start 2023-11-06T20:56:42.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.441+0000 7fa1583ff700 1 -- start start 2023-11-06T20:56:42.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.442+0000 7fa1583ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 0x7fa150156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:42.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.442+0000 7fa1583ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa150066a00 0x7fa15014d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:42.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.442+0000 7fa1583ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa150158780 0x7fa15014d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:42.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.442+0000 7fa1583ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15014ddf0 con 0x7fa150066620 2023-11-06T20:56:42.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.442+0000 7fa1583ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500c00d0 con 0x7fa150158780 2023-11-06T20:56:42.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.442+0000 7fa1583ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500c03b0 con 0x7fa150066a00 2023-11-06T20:56:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.442+0000 7fa15619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 0x7fa150156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 0x7fa150156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40330/0 (socket says 172.21.15.33:40330) 2023-11-06T20:56:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15619b700 1 -- 172.21.15.33:0/1280414724 learned_addr learned my addr 172.21.15.33:0/1280414724 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15599a700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa150066a00 0x7fa15014d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15699c700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa150158780 0x7fa15014d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15619b700 1 -- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa150066a00 msgr2=0x7fa15014d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15619b700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa150066a00 0x7fa15014d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15619b700 1 -- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa150158780 msgr2=0x7fa15014d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15619b700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa150158780 0x7fa15014d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15619b700 1 -- 172.21.15.33:0/1280414724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0092a0 con 0x7fa150066620 2023-11-06T20:56:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15699c700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa150158780 0x7fa15014d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa15619b700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 0x7fa150156b10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7fa14c009eb0 tx=0x7fa14c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa1467fc700 1 -- 172.21.15.33:0/1280414724 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00f7e0 con 0x7fa150066620 2023-11-06T20:56:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa1467fc700 1 -- 172.21.15.33:0/1280414724 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c00f940 con 0x7fa150066620 2023-11-06T20:56:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa1583ff700 1 -- 172.21.15.33:0/1280414724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1500c0690 con 0x7fa150066620 2023-11-06T20:56:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.443+0000 7fa1467fc700 1 -- 172.21.15.33:0/1280414724 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c01a430 con 0x7fa150066620 2023-11-06T20:56:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.444+0000 7fa1583ff700 1 -- 172.21.15.33:0/1280414724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1500bfcf0 con 0x7fa150066620 2023-11-06T20:56:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.445+0000 7fa1583ff700 1 -- 172.21.15.33:0/1280414724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1500bf020 con 0x7fa150066620 2023-11-06T20:56:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.445+0000 7fa1467fc700 1 -- 172.21.15.33:0/1280414724 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fa14c021020 con 0x7fa150066620 2023-11-06T20:56:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.446+0000 7fa1467fc700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13405f2c0 0x7fa134061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.446+0000 7fa1467fc700 1 -- 172.21.15.33:0/1280414724 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7431+0+0 (secure 0 0 0) 0x7fa14c07bfb0 con 0x7fa150066620 2023-11-06T20:56:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.447+0000 7fa15599a700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13405f2c0 0x7fa134061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.447+0000 7fa15599a700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13405f2c0 0x7fa134061780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fa140007fd0 tx=0x7fa140007bb0).ready entity=mgr.4823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.451+0000 7fa1467fc700 1 -- 172.21.15.33:0/1280414724 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa14c04cb20 con 0x7fa150066620 2023-11-06T20:56:42.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:42.677+0000 7fa1583ff700 1 -- 172.21.15.33:0/1280414724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa15005f7a0 con 0x7fa150066620 2023-11-06T20:56:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.119+0000 7fa1467fc700 1 -- 172.21.15.33:0/1280414724 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fa14c079620 con 0x7fa150066620 2023-11-06T20:56:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.120+0000 7fa1467fc700 1 -- 172.21.15.33:0/1280414724 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v11) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa14c0418f0 con 0x7fa150066620 2023-11-06T20:56:43.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 -- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13405f2c0 msgr2=0x7fa134061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:43.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13405f2c0 0x7fa134061780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fa140007fd0 tx=0x7fa140007bb0).stop 2023-11-06T20:56:43.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 -- 172.21.15.33:0/1280414724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 msgr2=0x7fa150156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:43.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 0x7fa150156b10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7fa14c009eb0 tx=0x7fa14c017000).stop 2023-11-06T20:56:43.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 -- 172.21.15.33:0/1280414724 shutdown_connections 2023-11-06T20:56:43.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa13405f2c0 0x7fa134061780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa150158780 0x7fa15014d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa150066620 0x7fa150156b10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 --2- 172.21.15.33:0/1280414724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa150066a00 0x7fa15014d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 -- 172.21.15.33:0/1280414724 >> 172.21.15.33:0/1280414724 conn(0x7fa1500c8c50 msgr2=0x7fa1500c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:43.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 -- 172.21.15.33:0/1280414724 shutdown_connections 2023-11-06T20:56:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.124+0000 7fa1583ff700 1 -- 172.21.15.33:0/1280414724 wait complete. 2023-11-06T20:56:43.150 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T20:56:43.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:56:43.244 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:56:43.344 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:43.344+0000 7fe2df576680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:56:43.348 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T20:56:43.348 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T20:56:43.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:56:43.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T20:56:43.350 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:43.351+0000 7ffbd7dbc680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:56:43.430 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:43.430+0000 7fe2df576680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:56:43.435 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:56:43.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:56:43.436 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T20:56:43.444 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:43.440+0000 7ffbd7dbc680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:56:43.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:56:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.717+0000 7ff870dff700 1 Processor -- start 2023-11-06T20:56:43.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.717+0000 7ff870dff700 1 -- start start 2023-11-06T20:56:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.718+0000 7ff870dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff86c063140 0x7ff86c063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.718+0000 7ff870dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff86c05e1b0 0x7ff86c0c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.718+0000 7ff870dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff86c0c31f0 con 0x7ff86c0c4490 2023-11-06T20:56:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.718+0000 7ff870dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff86c06a0f0 con 0x7ff86c063140 2023-11-06T20:56:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.718+0000 7ff870dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c0c2920 con 0x7ff86c05e1b0 2023-11-06T20:56:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.719+0000 7ff86a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff86c063140 0x7ff86c063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.719+0000 7ff86a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff86c063140 0x7ff86c063520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40360/0 (socket says 172.21.15.33:40360) 2023-11-06T20:56:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.719+0000 7ff86a59c700 1 -- 172.21.15.33:0/43284125 learned_addr learned my addr 172.21.15.33:0/43284125 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.719+0000 7ff869d9b700 1 --2- 172.21.15.33:0/43284125 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff86c05e1b0 0x7ff86c0c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.719+0000 7ff86a59c700 1 -- 172.21.15.33:0/43284125 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff86c05e1b0 msgr2=0x7ff86c0c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.719+0000 7ff86a59c700 1 --2- 172.21.15.33:0/43284125 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff86c05e1b0 0x7ff86c0c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.719+0000 7ff86a59c700 1 -- 172.21.15.33:0/43284125 >> v1:172.21.15.69:6789/0 conn(0x7ff86c0c4490 legacy=0x7ff86c154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.719+0000 7ff86a59c700 1 -- 172.21.15.33:0/43284125 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c0c3b70 con 0x7ff86c063140 2023-11-06T20:56:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.719+0000 7ff86959a700 1 -- 172.21.15.33:0/43284125 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647597154 0 0) 0x7ff86c0c31f0 con 0x7ff86c0c4490 2023-11-06T20:56:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.719+0000 7ff86a59c700 1 --2- 172.21.15.33:0/43284125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 0x7ff86c063520 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7ff85c006fd0 tx=0x7ff85c0090f0).ready entity=mon.0 client_cookie=b3c390a4098e80fd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff86959a700 1 -- 172.21.15.33:0/43284125 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c00b580 con 0x7ff86c063140 2023-11-06T20:56:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff86959a700 1 -- 172.21.15.33:0/43284125 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85c00b6e0 con 0x7ff86c063140 2023-11-06T20:56:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff86959a700 1 -- 172.21.15.33:0/43284125 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c00ba60 con 0x7ff86c063140 2023-11-06T20:56:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff870dff700 1 -- 172.21.15.33:0/43284125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 msgr2=0x7ff86c063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff870dff700 1 --2- 172.21.15.33:0/43284125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 0x7ff86c063520 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7ff85c006fd0 tx=0x7ff85c0090f0).stop 2023-11-06T20:56:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff870dff700 1 -- 172.21.15.33:0/43284125 shutdown_connections 2023-11-06T20:56:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff870dff700 1 --2- 172.21.15.33:0/43284125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 0x7ff86c063520 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff870dff700 1 --2- 172.21.15.33:0/43284125 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff86c05e1b0 0x7ff86c0c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff870dff700 1 -- 172.21.15.33:0/43284125 >> 172.21.15.33:0/43284125 conn(0x7ff86c0c7d60 msgr2=0x7ff86c0c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff870dff700 1 -- 172.21.15.33:0/43284125 shutdown_connections 2023-11-06T20:56:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.720+0000 7ff870dff700 1 -- 172.21.15.33:0/43284125 wait complete. 2023-11-06T20:56:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.722+0000 7ff870dff700 1 Processor -- start 2023-11-06T20:56:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.722+0000 7ff870dff700 1 -- start start 2023-11-06T20:56:43.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.723+0000 7ff870dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 0x7ff86c165c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:43.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.723+0000 7ff870dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff86c0c4490 0x7ff86c1626f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:43.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.723+0000 7ff870dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff86c162be0 0x7ff86c167520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:43.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.723+0000 7ff870dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c167090 con 0x7ff86c063140 2023-11-06T20:56:43.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.723+0000 7ff870dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c166480 con 0x7ff86c0c4490 2023-11-06T20:56:43.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.723+0000 7ff870dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c166760 con 0x7ff86c162be0 2023-11-06T20:56:43.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.723+0000 7ff86a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 0x7ff86c165c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:43.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.723+0000 7ff86a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 0x7ff86c165c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40374/0 (socket says 172.21.15.33:40374) 2023-11-06T20:56:43.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.723+0000 7ff86a59c700 1 -- 172.21.15.33:0/359243460 learned_addr learned my addr 172.21.15.33:0/359243460 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:43.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.723+0000 7ff869d9b700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff86c0c4490 0x7ff86c1626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:43.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff86ad9d700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff86c162be0 0x7ff86c167520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:43.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff869d9b700 1 -- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff86c162be0 msgr2=0x7ff86c167520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:43.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff869d9b700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff86c162be0 0x7ff86c167520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff869d9b700 1 -- 172.21.15.33:0/359243460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 msgr2=0x7ff86c165c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:43.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff869d9b700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 0x7ff86c165c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff869d9b700 1 -- 172.21.15.33:0/359243460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c009370 con 0x7ff86c0c4490 2023-11-06T20:56:43.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff86a59c700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 0x7ff86c165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:43.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff869d9b700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff86c0c4490 0x7ff86c1626f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7ff86000e6c0 tx=0x7ff860009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:43.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff85a7fc700 1 -- 172.21.15.33:0/359243460 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff860012600 con 0x7ff86c0c4490 2023-11-06T20:56:43.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff85a7fc700 1 -- 172.21.15.33:0/359243460 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff860012760 con 0x7ff86c0c4490 2023-11-06T20:56:43.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.724+0000 7ff85a7fc700 1 -- 172.21.15.33:0/359243460 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff860012ae0 con 0x7ff86c0c4490 2023-11-06T20:56:43.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.725+0000 7ff870dff700 1 -- 172.21.15.33:0/359243460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86c166a40 con 0x7ff86c0c4490 2023-11-06T20:56:43.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.725+0000 7ff870dff700 1 -- 172.21.15.33:0/359243460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86c166140 con 0x7ff86c0c4490 2023-11-06T20:56:43.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.726+0000 7ff870dff700 1 -- 172.21.15.33:0/359243460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86c167240 con 0x7ff86c0c4490 2023-11-06T20:56:43.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.726+0000 7ff85a7fc700 1 -- 172.21.15.33:0/359243460 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7ff86001b400 con 0x7ff86c0c4490 2023-11-06T20:56:43.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.727+0000 7ff85a7fc700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff84c05f270 0x7ff84c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:43.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.727+0000 7ff85a7fc700 1 -- 172.21.15.33:0/359243460 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7003+0+0 (secure 0 0 0) 0x7ff86007a9a0 con 0x7ff86c0c4490 2023-11-06T20:56:43.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.727+0000 7ff86a59c700 1 -- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff84c05f270 msgr2=0x7ff84c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:43.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.727+0000 7ff86a59c700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff84c05f270 0x7ff84c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:43.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.732+0000 7ff85a7fc700 1 -- 172.21.15.33:0/359243460 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff86c167240 con 0x7ff86c0c4490 2023-11-06T20:56:43.775 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:43.775+0000 7fe2df576680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:56:43.840 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:43.840+0000 7ffbd7dbc680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:56:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.928+0000 7ff86a59c700 1 -- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff84c05f270 msgr2=0x7ff84c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.928+0000 7ff86a59c700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff84c05f270 0x7ff84c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:56:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.985+0000 7ff870dff700 1 -- 172.21.15.33:0/359243460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff86c064420 con 0x7ff86c0c4490 2023-11-06T20:56:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.985+0000 7ff85a7fc700 1 -- 172.21.15.33:0/359243460 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4489 (secure 0 0 0) 0x7ff86004f150 con 0x7ff86c0c4490 2023-11-06T20:56:43.987 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:43.987 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/2191444463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2191444463},{"type":"v1","addr":"172.21.15.69:6836","nonce":2191444463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3920247513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3920247513},{"type":"v1","addr":"172.21.15.33:6834","nonce":3920247513}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2046477314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2046477314},{"type":"v1","addr":"172.21.15.69:6837","nonce":2046477314}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:45:04.669326+0000","modified":"2023-11-06T20:45:07.773858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/3657599674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3657599674},{"type":"v1","addr":"172.21.15.33:6835","nonce":3657599674}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-06T20:56:43.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 -- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff84c05f270 msgr2=0x7ff84c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:43.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff84c05f270 0x7ff84c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:43.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 -- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff86c0c4490 msgr2=0x7ff86c1626f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:43.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff86c0c4490 0x7ff86c1626f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7ff86000e6c0 tx=0x7ff860009060).stop 2023-11-06T20:56:43.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 -- 172.21.15.33:0/359243460 shutdown_connections 2023-11-06T20:56:43.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff84c05f270 0x7ff84c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:43.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff86c162be0 0x7ff86c167520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff86c063140 0x7ff86c165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 --2- 172.21.15.33:0/359243460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff86c0c4490 0x7ff86c1626f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:43.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 -- 172.21.15.33:0/359243460 >> 172.21.15.33:0/359243460 conn(0x7ff86c0c7d60 msgr2=0x7ff86c151c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:43.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 -- 172.21.15.33:0/359243460 shutdown_connections 2023-11-06T20:56:43.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:43.988+0000 7ff870dff700 1 -- 172.21.15.33:0/359243460 wait complete. 2023-11-06T20:56:43.994 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 15 2023-11-06T20:56:44.009 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:56:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.219+0000 7f6d2cdff700 1 Processor -- start 2023-11-06T20:56:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.219+0000 7f6d2cdff700 1 -- start start 2023-11-06T20:56:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.220+0000 7f6d2cdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d28066c50 0x7f6d28067d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.220+0000 7f6d2cdff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6d28138910 0x7f6d28068250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.220+0000 7f6d2cdff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d280c35b0 con 0x7f6d28138530 2023-11-06T20:56:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.220+0000 7f6d2cdff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d280c4130 con 0x7f6d28138910 2023-11-06T20:56:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.220+0000 7f6d2cdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d280c3800 con 0x7f6d28066c50 2023-11-06T20:56:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.221+0000 7f6d26d9d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6d28138910 0x7f6d28068250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.221+0000 7f6d25d9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d28066c50 0x7f6d28067d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.221+0000 7f6d2659c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f6d28138530 0x7f6d28066540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:42590/0 (socket says 172.21.15.33:42590) 2023-11-06T20:56:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.221+0000 7f6d2659c700 1 -- 172.21.15.33:0/4101844871 learned_addr learned my addr 172.21.15.33:0/4101844871 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.221+0000 7f6d26d9d700 1 -- 172.21.15.33:0/4101844871 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d28066c50 msgr2=0x7f6d28067d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.221+0000 7f6d26d9d700 1 --2- 172.21.15.33:0/4101844871 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d28066c50 0x7f6d28067d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.221+0000 7f6d26d9d700 1 -- 172.21.15.33:0/4101844871 >> v1:172.21.15.69:6789/0 conn(0x7f6d28138530 legacy=0x7f6d28066540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.221+0000 7f6d26d9d700 1 -- 172.21.15.33:0/4101844871 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d280c3ae0 con 0x7f6d28138910 2023-11-06T20:56:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.221+0000 7f6d2559a700 1 -- 172.21.15.33:0/4101844871 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948933262 0 0) 0x7f6d280c35b0 con 0x7f6d28138530 2023-11-06T20:56:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.221+0000 7f6d26d9d700 1 --2- 172.21.15.33:0/4101844871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d28138910 0x7f6d28068250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f6d20009e60 tx=0x7f6d20009780).ready entity=mon.1 client_cookie=15d81dcf67ba248a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2559a700 1 -- 172.21.15.33:0/4101844871 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2000e070 con 0x7f6d28138910 2023-11-06T20:56:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2559a700 1 -- 172.21.15.33:0/4101844871 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2000bb60 con 0x7f6d28138910 2023-11-06T20:56:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2559a700 1 -- 172.21.15.33:0/4101844871 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20012410 con 0x7f6d28138910 2023-11-06T20:56:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2cdff700 1 -- 172.21.15.33:0/4101844871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d28138910 msgr2=0x7f6d28068250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2cdff700 1 --2- 172.21.15.33:0/4101844871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d28138910 0x7f6d28068250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f6d20009e60 tx=0x7f6d20009780).stop 2023-11-06T20:56:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2cdff700 1 -- 172.21.15.33:0/4101844871 shutdown_connections 2023-11-06T20:56:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2cdff700 1 --2- 172.21.15.33:0/4101844871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d28138910 0x7f6d28068250 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2cdff700 1 --2- 172.21.15.33:0/4101844871 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d28066c50 0x7f6d28067d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2cdff700 1 -- 172.21.15.33:0/4101844871 >> 172.21.15.33:0/4101844871 conn(0x7f6d280c8c10 msgr2=0x7f6d280c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2cdff700 1 -- 172.21.15.33:0/4101844871 shutdown_connections 2023-11-06T20:56:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.222+0000 7f6d2cdff700 1 -- 172.21.15.33:0/4101844871 wait complete. 2023-11-06T20:56:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.224+0000 7f6d2cdff700 1 Processor -- start 2023-11-06T20:56:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.224+0000 7f6d2cdff700 1 -- start start 2023-11-06T20:56:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d2cdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d28066c50 0x7f6d28158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d2cdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d28138530 0x7f6d280c05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d2cdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d28138910 0x7f6d280aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d2cdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d280c0a90 con 0x7f6d28066c50 2023-11-06T20:56:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d2cdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d280ab240 con 0x7f6d28138910 2023-11-06T20:56:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d2cdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28158be0 con 0x7f6d28138530 2023-11-06T20:56:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d28066c50 0x7f6d28158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d28066c50 0x7f6d28158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40378/0 (socket says 172.21.15.33:40378) 2023-11-06T20:56:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d2659c700 1 -- 172.21.15.33:0/121896004 learned_addr learned my addr 172.21.15.33:0/121896004 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d25d9b700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d28138530 0x7f6d280c05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.225+0000 7f6d26d9d700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d28138910 0x7f6d280aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d2659c700 1 -- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d28138530 msgr2=0x7f6d280c05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d2659c700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d28138530 0x7f6d280c05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d2659c700 1 -- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d28138910 msgr2=0x7f6d280aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d2659c700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d28138910 0x7f6d280aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d2659c700 1 -- 172.21.15.33:0/121896004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d20009a20 con 0x7f6d28066c50 2023-11-06T20:56:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d2659c700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d28066c50 0x7f6d28158600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f6d1800c4e0 tx=0x7f6d18007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d167fc700 1 -- 172.21.15.33:0/121896004 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1800ce90 con 0x7f6d28066c50 2023-11-06T20:56:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d167fc700 1 -- 172.21.15.33:0/121896004 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d180072e0 con 0x7f6d28066c50 2023-11-06T20:56:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d2cdff700 1 -- 172.21.15.33:0/121896004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d28158ec0 con 0x7f6d28066c50 2023-11-06T20:56:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d167fc700 1 -- 172.21.15.33:0/121896004 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18007660 con 0x7f6d28066c50 2023-11-06T20:56:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.226+0000 7f6d2cdff700 1 -- 172.21.15.33:0/121896004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d28159480 con 0x7f6d28066c50 2023-11-06T20:56:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.228+0000 7f6d2cdff700 1 -- 172.21.15.33:0/121896004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d280c3dc0 con 0x7f6d28066c50 2023-11-06T20:56:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.228+0000 7f6d167fc700 1 -- 172.21.15.33:0/121896004 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f6d180059b0 con 0x7f6d28066c50 2023-11-06T20:56:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.229+0000 7f6d167fc700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0805f270 0x7f6d08061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.230+0000 7f6d167fc700 1 -- 172.21.15.33:0/121896004 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6923+0+0 (secure 0 0 0) 0x7f6d18079e50 con 0x7f6d28066c50 2023-11-06T20:56:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.230+0000 7f6d25d9b700 1 -- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0805f270 msgr2=0x7f6d08061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.230+0000 7f6d25d9b700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0805f270 0x7f6d08061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.234+0000 7f6d167fc700 1 -- 172.21.15.33:0/121896004 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6d1804abc0 con 0x7f6d28066c50 2023-11-06T20:56:44.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.415+0000 7f6d2cdff700 1 -- 172.21.15.33:0/121896004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d280c4800 con 0x7f6d28066c50 2023-11-06T20:56:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.416+0000 7f6d167fc700 1 -- 172.21.15.33:0/121896004 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18220 (secure 0 0 0) 0x7f6d1804e1e0 con 0x7f6d28066c50 2023-11-06T20:56:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:44.417 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":51,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:56:43.935779+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:45:02.985899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T20:45:03.637532+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.33","mappings":[{"from":4,"to":5}]},{"pgid":"5.35","mappings":[{"from":4,"to":5}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]},{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.18","mappings":[{"from":4,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6832/114912":"2023-11-07T20:45:24.054860+0000","172.21.15.69:6833/114912":"2023-11-07T20:45:24.054860+0000","172.21.15.69:0/1201458877":"2023-11-07T20:45:24.054860+0000","172.21.15.69:0/1049662608":"2023-11-07T20:45:24.054860+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":4,"length":2},{"begin":9,"length":1},{"begin":14,"length":2},{"begin":19,"length":1},{"begin":24,"length":2},{"begin":29,"length":1},{"begin":34,"length":2},{"begin":39,"length":1},{"begin":44,"length":2},{"begin":49,"length":1},{"begin":54,"length":2},{"begin":59,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":74,"length":2},{"begin":79,"length":2},{"begin":84,"length":2},{"begin":89,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":106,"length":1},{"begin":110,"length":2},{"begin":183,"length":38}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":74,"length":1},{"begin":84,"length":1},{"begin":95,"length":1},{"begin":218,"length":3}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:56:44.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.418+0000 7f6d2cdff700 1 -- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0805f270 msgr2=0x7f6d08061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:44.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.418+0000 7f6d2cdff700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0805f270 0x7f6d08061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.418+0000 7f6d2cdff700 1 -- 172.21.15.33:0/121896004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d28066c50 msgr2=0x7f6d28158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.418+0000 7f6d2cdff700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d28066c50 0x7f6d28158600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f6d1800c4e0 tx=0x7f6d18007c70).stop 2023-11-06T20:56:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.418+0000 7f6d2cdff700 1 -- 172.21.15.33:0/121896004 shutdown_connections 2023-11-06T20:56:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.418+0000 7f6d2cdff700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0805f270 0x7f6d08061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.418+0000 7f6d2cdff700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d28138910 0x7f6d280aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.418+0000 7f6d2cdff700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d28066c50 0x7f6d28158600 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.418+0000 7f6d2cdff700 1 --2- 172.21.15.33:0/121896004 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d28138530 0x7f6d280c05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.418+0000 7f6d2cdff700 1 -- 172.21.15.33:0/121896004 >> 172.21.15.33:0/121896004 conn(0x7f6d280c8c10 msgr2=0x7f6d28063820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.419+0000 7f6d2cdff700 1 -- 172.21.15.33:0/121896004 shutdown_connections 2023-11-06T20:56:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.419+0000 7f6d2cdff700 1 -- 172.21.15.33:0/121896004 wait complete. 2023-11-06T20:56:44.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:45:02.985899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:45:03.637532+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T20:56:44.438 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:56:44.544 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:44.544+0000 7fe2df576680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:56:44.628 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:44.628+0000 7fe2df576680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:56:44.666 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:44.666+0000 7ffbd7dbc680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:56:44.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.682+0000 7fbad5fff700 1 Processor -- start 2023-11-06T20:56:44.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.682+0000 7fbad5fff700 1 -- start start 2023-11-06T20:56:44.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.684+0000 7fbad5fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154710 0x7fbad00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.684+0000 7fbad5fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbad00c7ff0 0x7fbad00646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.684+0000 7fbad5fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad00c6b60 con 0x7fbad0154330 2023-11-06T20:56:44.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.684+0000 7fbad5fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad00c7700 con 0x7fbad00c7ff0 2023-11-06T20:56:44.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.684+0000 7fbad5fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00c5cd0 con 0x7fbad0154710 2023-11-06T20:56:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.684+0000 7fbacffff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbad00c7ff0 0x7fbad00646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.684+0000 7fbaceffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154710 0x7fbad00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.684+0000 7fbacffff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbad00c7ff0 0x7fbad00646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:40408/0 (socket says 172.21.15.33:40408) 2023-11-06T20:56:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.684+0000 7fbacffff700 1 -- 172.21.15.33:0/3054382486 learned_addr learned my addr 172.21.15.33:0/3054382486 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.684+0000 7fbacf7fe700 1 --1- 172.21.15.33:0/3054382486 >> v1:172.21.15.69:6789/0 conn(0x7fbad0154330 0x7fbad00c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:42602/0 (socket says 172.21.15.33:42602) 2023-11-06T20:56:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.685+0000 7fbacffff700 1 -- 172.21.15.33:0/3054382486 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154710 msgr2=0x7fbad00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.685+0000 7fbacffff700 1 --2- 172.21.15.33:0/3054382486 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154710 0x7fbad00c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.685+0000 7fbacffff700 1 -- 172.21.15.33:0/3054382486 >> v1:172.21.15.69:6789/0 conn(0x7fbad0154330 legacy=0x7fbad00c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.685+0000 7fbacffff700 1 -- 172.21.15.33:0/3054382486 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad00c5f50 con 0x7fbad00c7ff0 2023-11-06T20:56:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.685+0000 7fbace7fc700 1 -- 172.21.15.33:0/3054382486 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071990516 0 0) 0x7fbad00c6b60 con 0x7fbad0154330 2023-11-06T20:56:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.685+0000 7fbacffff700 1 --2- 172.21.15.33:0/3054382486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad00c7ff0 0x7fbad00646c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fbac8009f20 tx=0x7fbac8009840).ready entity=mon.1 client_cookie=25842945f9536a04 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.685+0000 7fbace7fc700 1 -- 172.21.15.33:0/3054382486 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800e070 con 0x7fbad00c7ff0 2023-11-06T20:56:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.685+0000 7fbace7fc700 1 -- 172.21.15.33:0/3054382486 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac800b9d0 con 0x7fbad00c7ff0 2023-11-06T20:56:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.685+0000 7fbace7fc700 1 -- 172.21.15.33:0/3054382486 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800bd50 con 0x7fbad00c7ff0 2023-11-06T20:56:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.686+0000 7fbad5fff700 1 -- 172.21.15.33:0/3054382486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad00c7ff0 msgr2=0x7fbad00646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.686+0000 7fbad5fff700 1 --2- 172.21.15.33:0/3054382486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad00c7ff0 0x7fbad00646c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fbac8009f20 tx=0x7fbac8009840).stop 2023-11-06T20:56:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.686+0000 7fbad5fff700 1 -- 172.21.15.33:0/3054382486 shutdown_connections 2023-11-06T20:56:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.686+0000 7fbad5fff700 1 --2- 172.21.15.33:0/3054382486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad00c7ff0 0x7fbad00646c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.686+0000 7fbad5fff700 1 --2- 172.21.15.33:0/3054382486 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154710 0x7fbad00c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.686+0000 7fbad5fff700 1 -- 172.21.15.33:0/3054382486 >> 172.21.15.33:0/3054382486 conn(0x7fbad00cb910 msgr2=0x7fbad00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.686+0000 7fbad5fff700 1 -- 172.21.15.33:0/3054382486 shutdown_connections 2023-11-06T20:56:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.686+0000 7fbad5fff700 1 -- 172.21.15.33:0/3054382486 wait complete. 2023-11-06T20:56:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.687+0000 7fbad5fff700 1 Processor -- start 2023-11-06T20:56:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.688+0000 7fbad5fff700 1 -- start start 2023-11-06T20:56:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.688+0000 7fbad5fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad00c7ff0 0x7fbad00aa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbad5fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154330 0x7fbad0153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbad5fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad0154710 0x7fbad0156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbad5fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00af770 con 0x7fbad0154710 2023-11-06T20:56:44.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbad5fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad01560b0 con 0x7fbad00c7ff0 2023-11-06T20:56:44.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbad5fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00b0280 con 0x7fbad0154330 2023-11-06T20:56:44.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbacffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad0154710 0x7fbad0156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:44.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbacffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad0154710 0x7fbad0156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40388/0 (socket says 172.21.15.33:40388) 2023-11-06T20:56:44.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbacffff700 1 -- 172.21.15.33:0/3046152472 learned_addr learned my addr 172.21.15.33:0/3046152472 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:44.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbacf7fe700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad00c7ff0 0x7fbad00aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:44.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbaceffd700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154330 0x7fbad0153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:44.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbacffff700 1 -- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154330 msgr2=0x7fbad0153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:44.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbacffff700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154330 0x7fbad0153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbacffff700 1 -- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad00c7ff0 msgr2=0x7fbad00aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbacffff700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad00c7ff0 0x7fbad00aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbacffff700 1 -- 172.21.15.33:0/3046152472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac8009ae0 con 0x7fbad0154710 2023-11-06T20:56:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbaceffd700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154330 0x7fbad0153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.689+0000 7fbacf7fe700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad00c7ff0 0x7fbad00aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.690+0000 7fbacffff700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad0154710 0x7fbad0156310 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fbac800b610 tx=0x7fbac8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.690+0000 7fbab77fe700 1 -- 172.21.15.33:0/3046152472 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800bd50 con 0x7fbad0154710 2023-11-06T20:56:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.690+0000 7fbab77fe700 1 -- 172.21.15.33:0/3046152472 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac800b640 con 0x7fbad0154710 2023-11-06T20:56:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.690+0000 7fbad5fff700 1 -- 172.21.15.33:0/3046152472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad00af950 con 0x7fbad0154710 2023-11-06T20:56:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.690+0000 7fbab77fe700 1 -- 172.21.15.33:0/3046152472 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac801d6b0 con 0x7fbad0154710 2023-11-06T20:56:44.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.690+0000 7fbad5fff700 1 -- 172.21.15.33:0/3046152472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad00aff10 con 0x7fbad0154710 2023-11-06T20:56:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.691+0000 7fbad5fff700 1 -- 172.21.15.33:0/3046152472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad00b03f0 con 0x7fbad0154710 2023-11-06T20:56:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.692+0000 7fbab77fe700 1 -- 172.21.15.33:0/3046152472 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fbac8003040 con 0x7fbad0154710 2023-11-06T20:56:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.693+0000 7fbab77fe700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbabc05f270 0x7fbabc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.693+0000 7fbab77fe700 1 -- 172.21.15.33:0/3046152472 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6923+0+0 (secure 0 0 0) 0x7fbac807ba60 con 0x7fbad0154710 2023-11-06T20:56:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.694+0000 7fbacf7fe700 1 -- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbabc05f270 msgr2=0x7fbabc061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:44.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.694+0000 7fbacf7fe700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbabc05f270 0x7fbabc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:44.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.698+0000 7fbab77fe700 1 -- 172.21.15.33:0/3046152472 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbac804c7d0 con 0x7fbad0154710 2023-11-06T20:56:44.757 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:44.757+0000 7ffbd7dbc680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:56:44.859 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:44.859+0000 7fe2df576680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:56:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.878+0000 7fbad5fff700 1 -- 172.21.15.33:0/3046152472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbad00830b0 con 0x7fbad0154710 2023-11-06T20:56:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.878+0000 7fbab77fe700 1 -- 172.21.15.33:0/3046152472 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v51) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbac8022070 con 0x7fbad0154710 2023-11-06T20:56:44.879 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.880+0000 7fbad5fff700 1 -- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbabc05f270 msgr2=0x7fbabc061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.880+0000 7fbad5fff700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbabc05f270 0x7fbabc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.880+0000 7fbad5fff700 1 -- 172.21.15.33:0/3046152472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad0154710 msgr2=0x7fbad0156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.880+0000 7fbad5fff700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad0154710 0x7fbad0156310 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fbac800b610 tx=0x7fbac8009840).stop 2023-11-06T20:56:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.881+0000 7fbad5fff700 1 -- 172.21.15.33:0/3046152472 shutdown_connections 2023-11-06T20:56:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.881+0000 7fbad5fff700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbabc05f270 0x7fbabc061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.881+0000 7fbad5fff700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad0154710 0x7fbad0156310 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.881+0000 7fbad5fff700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad00c7ff0 0x7fbad00aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.881+0000 7fbad5fff700 1 --2- 172.21.15.33:0/3046152472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad0154330 0x7fbad0153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.881+0000 7fbad5fff700 1 -- 172.21.15.33:0/3046152472 >> 172.21.15.33:0/3046152472 conn(0x7fbad00cb910 msgr2=0x7fbad00646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.881+0000 7fbad5fff700 1 -- 172.21.15.33:0/3046152472 shutdown_connections 2023-11-06T20:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:44.881+0000 7fbad5fff700 1 -- 172.21.15.33:0/3046152472 wait complete. 2023-11-06T20:56:44.899 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:56:45.022 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:45.022+0000 7ffbd7dbc680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:56:45.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.116+0000 7f1c279ff700 1 Processor -- start 2023-11-06T20:56:45.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.116+0000 7f1c279ff700 1 -- start start 2023-11-06T20:56:45.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.118+0000 7f1c279ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c20154330 0x7f1c200c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.118+0000 7f1c279ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1c200c7ff0 0x7f1c200646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.118+0000 7f1c279ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c200c6b60 con 0x7f1c20154710 2023-11-06T20:56:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.118+0000 7f1c279ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c200c7700 con 0x7f1c200c7ff0 2023-11-06T20:56:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.118+0000 7f1c279ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c5cd0 con 0x7f1c20154330 2023-11-06T20:56:45.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.118+0000 7f1c25f9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1c200c7ff0 0x7f1c200646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.118+0000 7f1c2579b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c20154330 0x7f1c200c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.118+0000 7f1c25f9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1c200c7ff0 0x7f1c200646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:40422/0 (socket says 172.21.15.33:40422) 2023-11-06T20:56:45.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.118+0000 7f1c25f9c700 1 -- 172.21.15.33:0/3410980707 learned_addr learned my addr 172.21.15.33:0/3410980707 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:45.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.119+0000 7f1c25f9c700 1 -- 172.21.15.33:0/3410980707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c20154330 msgr2=0x7f1c200c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:45.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.119+0000 7f1c25f9c700 1 --2- 172.21.15.33:0/3410980707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c20154330 0x7f1c200c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.119+0000 7f1c25f9c700 1 -- 172.21.15.33:0/3410980707 >> v1:172.21.15.69:6789/0 conn(0x7f1c20154710 legacy=0x7f1c200c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.119+0000 7f1c25f9c700 1 -- 172.21.15.33:0/3410980707 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200c5f50 con 0x7f1c200c7ff0 2023-11-06T20:56:45.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.119+0000 7f1c17fff700 1 -- 172.21.15.33:0/3410980707 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289564562 0 0) 0x7f1c200c6b60 con 0x7f1c20154710 2023-11-06T20:56:45.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.119+0000 7f1c2579b700 1 --2- 172.21.15.33:0/3410980707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c20154330 0x7f1c200c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:45.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.119+0000 7f1c25f9c700 1 --2- 172.21.15.33:0/3410980707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c200c7ff0 0x7f1c200646c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f1c18009f20 tx=0x7f1c18009840).ready entity=mon.1 client_cookie=f0f2b56ad901fb38 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.119+0000 7f1c17fff700 1 -- 172.21.15.33:0/3410980707 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800e070 con 0x7f1c200c7ff0 2023-11-06T20:56:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.119+0000 7f1c17fff700 1 -- 172.21.15.33:0/3410980707 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1800b9d0 con 0x7f1c200c7ff0 2023-11-06T20:56:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.119+0000 7f1c17fff700 1 -- 172.21.15.33:0/3410980707 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800bd50 con 0x7f1c200c7ff0 2023-11-06T20:56:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.120+0000 7f1c279ff700 1 -- 172.21.15.33:0/3410980707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c200c7ff0 msgr2=0x7f1c200646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.120+0000 7f1c279ff700 1 --2- 172.21.15.33:0/3410980707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c200c7ff0 0x7f1c200646c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f1c18009f20 tx=0x7f1c18009840).stop 2023-11-06T20:56:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.120+0000 7f1c279ff700 1 -- 172.21.15.33:0/3410980707 shutdown_connections 2023-11-06T20:56:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.120+0000 7f1c279ff700 1 --2- 172.21.15.33:0/3410980707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c200c7ff0 0x7f1c200646c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.120+0000 7f1c279ff700 1 --2- 172.21.15.33:0/3410980707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c20154330 0x7f1c200c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.120+0000 7f1c279ff700 1 -- 172.21.15.33:0/3410980707 >> 172.21.15.33:0/3410980707 conn(0x7f1c200cb910 msgr2=0x7f1c200c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.120+0000 7f1c279ff700 1 -- 172.21.15.33:0/3410980707 shutdown_connections 2023-11-06T20:56:45.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.120+0000 7f1c279ff700 1 -- 172.21.15.33:0/3410980707 wait complete. 2023-11-06T20:56:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.122+0000 7f1c279ff700 1 Processor -- start 2023-11-06T20:56:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.122+0000 7f1c279ff700 1 -- start start 2023-11-06T20:56:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c279ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c200c7ff0 0x7f1c20153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c279ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c20154330 0x7f1c200aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c279ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c20154710 0x7f1c200ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c279ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c20159970 con 0x7f1c20154710 2023-11-06T20:56:45.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c279ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200ab8e0 con 0x7f1c20154330 2023-11-06T20:56:45.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c279ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c20157c20 con 0x7f1c200c7ff0 2023-11-06T20:56:45.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c25f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c20154710 0x7f1c200ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c25f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c20154710 0x7f1c200ab3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40390/0 (socket says 172.21.15.33:40390) 2023-11-06T20:56:45.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c25f9c700 1 -- 172.21.15.33:0/3918169761 learned_addr learned my addr 172.21.15.33:0/3918169761 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:45.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c24f9a700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c20154330 0x7f1c200aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.123+0000 7f1c2579b700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c200c7ff0 0x7f1c20153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c25f9c700 1 -- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c200c7ff0 msgr2=0x7f1c20153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c25f9c700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c200c7ff0 0x7f1c20153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c25f9c700 1 -- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c20154330 msgr2=0x7f1c200aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c25f9c700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c20154330 0x7f1c200aaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c25f9c700 1 -- 172.21.15.33:0/3918169761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c0092a0 con 0x7f1c20154710 2023-11-06T20:56:45.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c24f9a700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c20154330 0x7f1c200aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:45.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c25f9c700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c20154710 0x7f1c200ab3f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f1c2006c170 tx=0x7f1c18009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:45.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c157fa700 1 -- 172.21.15.33:0/3918169761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180092e0 con 0x7f1c20154710 2023-11-06T20:56:45.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c157fa700 1 -- 172.21.15.33:0/3918169761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1801e470 con 0x7f1c20154710 2023-11-06T20:56:45.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c279ff700 1 -- 172.21.15.33:0/3918169761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c18009ae0 con 0x7f1c20154710 2023-11-06T20:56:45.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c157fa700 1 -- 172.21.15.33:0/3918169761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1801e830 con 0x7f1c20154710 2023-11-06T20:56:45.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.124+0000 7f1c279ff700 1 -- 172.21.15.33:0/3918169761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c201572f0 con 0x7f1c20154710 2023-11-06T20:56:45.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.125+0000 7f1c279ff700 1 -- 172.21.15.33:0/3918169761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c20164490 con 0x7f1c20154710 2023-11-06T20:56:45.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.126+0000 7f1c157fa700 1 -- 172.21.15.33:0/3918169761 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f1c18003040 con 0x7f1c20154710 2023-11-06T20:56:45.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.127+0000 7f1c157fa700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c0405f270 0x7f1c04061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.128+0000 7f1c157fa700 1 -- 172.21.15.33:0/3918169761 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6891+0+0 (secure 0 0 0) 0x7f1c1807bb20 con 0x7f1c20154710 2023-11-06T20:56:45.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.128+0000 7f1c2579b700 1 -- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c0405f270 msgr2=0x7f1c04061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:45.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.128+0000 7f1c2579b700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c0405f270 0x7f1c04061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:45.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.131+0000 7f1c157fa700 1 -- 172.21.15.33:0/3918169761 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1c20164490 con 0x7f1c20154710 2023-11-06T20:56:45.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.312+0000 7f1c279ff700 1 -- 172.21.15.33:0/3918169761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1c20164490 con 0x7f1c20154710 2023-11-06T20:56:45.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.312+0000 7f1c157fa700 1 -- 172.21.15.33:0/3918169761 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v52) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1c20164490 con 0x7f1c20154710 2023-11-06T20:56:45.313 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:56:45.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.314+0000 7f1c279ff700 1 -- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c0405f270 msgr2=0x7f1c04061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:45.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.314+0000 7f1c279ff700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c0405f270 0x7f1c04061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:45.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.314+0000 7f1c279ff700 1 -- 172.21.15.33:0/3918169761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c20154710 msgr2=0x7f1c200ab3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.314+0000 7f1c279ff700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c20154710 0x7f1c200ab3f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f1c2006c170 tx=0x7f1c18009840).stop 2023-11-06T20:56:45.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.315+0000 7f1c279ff700 1 -- 172.21.15.33:0/3918169761 shutdown_connections 2023-11-06T20:56:45.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.315+0000 7f1c279ff700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c0405f270 0x7f1c04061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:45.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.315+0000 7f1c279ff700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c200c7ff0 0x7f1c20153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.315+0000 7f1c279ff700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c20154710 0x7f1c200ab3f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.315+0000 7f1c279ff700 1 --2- 172.21.15.33:0/3918169761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c20154330 0x7f1c200aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.315+0000 7f1c279ff700 1 -- 172.21.15.33:0/3918169761 >> 172.21.15.33:0/3918169761 conn(0x7f1c200cb910 msgr2=0x7f1c200ac4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:45.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.315+0000 7f1c279ff700 1 -- 172.21.15.33:0/3918169761 shutdown_connections 2023-11-06T20:56:45.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.315+0000 7f1c279ff700 1 -- 172.21.15.33:0/3918169761 wait complete. 2023-11-06T20:56:45.333 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T20:56:45.383 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:45.383+0000 7fe2df576680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:56:45.508 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:45.508+0000 7fe2df576680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:56:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.534+0000 7fdf0afff700 1 Processor -- start 2023-11-06T20:56:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.535+0000 7fdf0afff700 1 -- start start 2023-11-06T20:56:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.536+0000 7fdf0afff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdf041544f0 0x7fdf040c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.536+0000 7fdf0afff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf04064dd0 0x7fdf040c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.536+0000 7fdf0afff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf041556d0 con 0x7fdf040c0c40 2023-11-06T20:56:45.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.536+0000 7fdf0afff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf041548d0 con 0x7fdf04064dd0 2023-11-06T20:56:45.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.536+0000 7fdf0afff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf04154ac0 con 0x7fdf041544f0 2023-11-06T20:56:45.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.536+0000 7fdf0959c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf04064dd0 0x7fdf040c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.536+0000 7fdf08d9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdf041544f0 0x7fdf040c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.536+0000 7fdf0959c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf04064dd0 0x7fdf040c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40394/0 (socket says 172.21.15.33:40394) 2023-11-06T20:56:45.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.536+0000 7fdf0959c700 1 -- 172.21.15.33:0/196007824 learned_addr learned my addr 172.21.15.33:0/196007824 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:45.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.537+0000 7fdf08d9b700 1 -- 172.21.15.33:0/196007824 >> v2:172.21.15.33:3300/0 conn(0x7fdf04064dd0 msgr2=0x7fdf040c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:45.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.537+0000 7fdf08d9b700 1 --2- 172.21.15.33:0/196007824 >> v2:172.21.15.33:3300/0 conn(0x7fdf04064dd0 0x7fdf040c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.537+0000 7fdf08d9b700 1 -- 172.21.15.33:0/196007824 >> v1:172.21.15.69:6789/0 conn(0x7fdf040c0c40 legacy=0x7fdf040646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.537+0000 7fdf08d9b700 1 -- 172.21.15.33:0/196007824 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04154da0 con 0x7fdf041544f0 2023-11-06T20:56:45.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.537+0000 7fdf0959c700 1 --2- 172.21.15.33:0/196007824 >> v2:172.21.15.33:3300/0 conn(0x7fdf04064dd0 0x7fdf040c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:45.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.537+0000 7fdf037fe700 1 -- 172.21.15.33:0/196007824 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977258155 0 0) 0x7fdf041556d0 con 0x7fdf040c0c40 2023-11-06T20:56:45.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.537+0000 7fdf08d9b700 1 --2- 172.21.15.33:0/196007824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf041544f0 0x7fdf040c0750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fdf04084660 tx=0x7fdef8009060).ready entity=mon.1 client_cookie=f9eaa7b0d1a7dea7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:45.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.537+0000 7fdf037fe700 1 -- 172.21.15.33:0/196007824 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef800f510 con 0x7fdf041544f0 2023-11-06T20:56:45.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.537+0000 7fdf037fe700 1 -- 172.21.15.33:0/196007824 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef800f670 con 0x7fdf041544f0 2023-11-06T20:56:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.537+0000 7fdf037fe700 1 -- 172.21.15.33:0/196007824 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef800f9f0 con 0x7fdf041544f0 2023-11-06T20:56:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.538+0000 7fdf0afff700 1 -- 172.21.15.33:0/196007824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf041544f0 msgr2=0x7fdf040c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.538+0000 7fdf0afff700 1 --2- 172.21.15.33:0/196007824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf041544f0 0x7fdf040c0750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fdf04084660 tx=0x7fdef8009060).stop 2023-11-06T20:56:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.538+0000 7fdf0afff700 1 -- 172.21.15.33:0/196007824 shutdown_connections 2023-11-06T20:56:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.538+0000 7fdf0afff700 1 --2- 172.21.15.33:0/196007824 >> v2:172.21.15.33:3300/0 conn(0x7fdf04064dd0 0x7fdf040c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.538+0000 7fdf0afff700 1 --2- 172.21.15.33:0/196007824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf041544f0 0x7fdf040c0750 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.538+0000 7fdf0afff700 1 -- 172.21.15.33:0/196007824 >> 172.21.15.33:0/196007824 conn(0x7fdf040cb900 msgr2=0x7fdf040c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.538+0000 7fdf0afff700 1 -- 172.21.15.33:0/196007824 shutdown_connections 2023-11-06T20:56:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.538+0000 7fdf0afff700 1 -- 172.21.15.33:0/196007824 wait complete. 2023-11-06T20:56:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.540+0000 7fdf0afff700 1 Processor -- start 2023-11-06T20:56:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.540+0000 7fdf0afff700 1 -- start start 2023-11-06T20:56:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf0afff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf04064dd0 0x7fdf040ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf0afff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf040c0c40 0x7fdf040ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf0afff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf041544f0 0x7fdf04164e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf0afff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040a9bf0 con 0x7fdf04064dd0 2023-11-06T20:56:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf0afff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041534c0 con 0x7fdf040c0c40 2023-11-06T20:56:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf0afff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040aed60 con 0x7fdf041544f0 2023-11-06T20:56:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf08d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf04064dd0 0x7fdf040ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf08d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf04064dd0 0x7fdf040ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40398/0 (socket says 172.21.15.33:40398) 2023-11-06T20:56:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf08d9b700 1 -- 172.21.15.33:0/3066161824 learned_addr learned my addr 172.21.15.33:0/3066161824 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf0959c700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf041544f0 0x7fdf04164e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf03fff700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf040c0c40 0x7fdf040ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf08d9b700 1 -- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf041544f0 msgr2=0x7fdf04164e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf08d9b700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf041544f0 0x7fdf04164e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf08d9b700 1 -- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf040c0c40 msgr2=0x7fdf040ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf08d9b700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf040c0c40 0x7fdf040ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.541+0000 7fdf08d9b700 1 -- 172.21.15.33:0/3066161824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc009ae0 con 0x7fdf04064dd0 2023-11-06T20:56:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.542+0000 7fdf08d9b700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf04064dd0 0x7fdf040ae280 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fdef8009dd0 tx=0x7fdef8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.542+0000 7fdf00ff9700 1 -- 172.21.15.33:0/3066161824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef800f7e0 con 0x7fdf04064dd0 2023-11-06T20:56:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.542+0000 7fdf00ff9700 1 -- 172.21.15.33:0/3066161824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef800f940 con 0x7fdf04064dd0 2023-11-06T20:56:45.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.542+0000 7fdf0afff700 1 -- 172.21.15.33:0/3066161824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef80092a0 con 0x7fdf04064dd0 2023-11-06T20:56:45.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.542+0000 7fdf00ff9700 1 -- 172.21.15.33:0/3066161824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef801b690 con 0x7fdf04064dd0 2023-11-06T20:56:45.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.542+0000 7fdf0afff700 1 -- 172.21.15.33:0/3066161824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf040af320 con 0x7fdf04064dd0 2023-11-06T20:56:45.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.543+0000 7fdf0afff700 1 -- 172.21.15.33:0/3066161824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf04155080 con 0x7fdf04064dd0 2023-11-06T20:56:45.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.544+0000 7fdf00ff9700 1 -- 172.21.15.33:0/3066161824 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fdef801ba40 con 0x7fdf04064dd0 2023-11-06T20:56:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.545+0000 7fdf00ff9700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdee805f270 0x7fdee8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.545+0000 7fdf00ff9700 1 -- 172.21.15.33:0/3066161824 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6891+0+0 (secure 0 0 0) 0x7fdef807bd60 con 0x7fdf04064dd0 2023-11-06T20:56:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.545+0000 7fdf03fff700 1 -- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdee805f270 msgr2=0x7fdee8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:45.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.545+0000 7fdf03fff700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdee805f270 0x7fdee8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:45.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.553+0000 7fdf00ff9700 1 -- 172.21.15.33:0/3066161824 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdef804caf0 con 0x7fdf04064dd0 2023-11-06T20:56:45.581 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:45.581+0000 7ffbd7dbc680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:56:45.692 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:45.693+0000 7fe2df576680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:56:45.716 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:45.715+0000 7ffbd7dbc680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:56:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.732+0000 7fdf0afff700 1 -- 172.21.15.33:0/3066161824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf04065340 con 0x7fdf04064dd0 2023-11-06T20:56:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.733+0000 7fdf00ff9700 1 -- 172.21.15.33:0/3066161824 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdef80280a0 con 0x7fdf04064dd0 2023-11-06T20:56:45.733 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:56:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 -- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdee805f270 msgr2=0x7fdee8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdee805f270 0x7fdee8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 -- 172.21.15.33:0/3066161824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf04064dd0 msgr2=0x7fdf040ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf04064dd0 0x7fdf040ae280 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fdef8009dd0 tx=0x7fdef8017000).stop 2023-11-06T20:56:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 -- 172.21.15.33:0/3066161824 shutdown_connections 2023-11-06T20:56:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdee805f270 0x7fdee8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf041544f0 0x7fdf04164e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf04064dd0 0x7fdf040ae280 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 --2- 172.21.15.33:0/3066161824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf040c0c40 0x7fdf040ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 -- 172.21.15.33:0/3066161824 >> 172.21.15.33:0/3066161824 conn(0x7fdf040cb900 msgr2=0x7fdf040a9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 -- 172.21.15.33:0/3066161824 shutdown_connections 2023-11-06T20:56:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.735+0000 7fdf0afff700 1 -- 172.21.15.33:0/3066161824 wait complete. 2023-11-06T20:56:45.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T20:56:45.787 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:45.787+0000 7fe2df576680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:56:45.873 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:45.872+0000 7fe2df576680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:56:45.911 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:45.911+0000 7ffbd7dbc680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:56:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.958+0000 7f9d77dff700 1 Processor -- start 2023-11-06T20:56:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.958+0000 7f9d77dff700 1 -- start start 2023-11-06T20:56:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.959+0000 7f9d77dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9d701546a0 0x7f9d70064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.959+0000 7f9d77dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d70066560 0x7f9d700669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.959+0000 7f9d77dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d700641f0 con 0x7f9d700bdbb0 2023-11-06T20:56:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.959+0000 7f9d77dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d70154a80 con 0x7f9d701546a0 2023-11-06T20:56:45.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.959+0000 7f9d77dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700638c0 con 0x7f9d70066560 2023-11-06T20:56:45.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.960+0000 7f9d7639c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9d700bdbb0 0x7f9d700c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50572/0 (socket says 172.21.15.33:50572) 2023-11-06T20:56:45.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.960+0000 7f9d7639c700 1 -- 172.21.15.33:0/530265277 learned_addr learned my addr 172.21.15.33:0/530265277 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:45.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.960+0000 7f9d75b9b700 1 --2- 172.21.15.33:0/530265277 >> v2:172.21.15.69:3300/0 conn(0x7f9d701546a0 0x7f9d70064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.960+0000 7f9d7539a700 1 --2- 172.21.15.33:0/530265277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d70066560 0x7f9d700669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.960+0000 7f9d75b9b700 1 -- 172.21.15.33:0/530265277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d70066560 msgr2=0x7f9d700669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:45.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.960+0000 7f9d75b9b700 1 --2- 172.21.15.33:0/530265277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d70066560 0x7f9d700669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.960+0000 7f9d75b9b700 1 -- 172.21.15.33:0/530265277 >> v1:172.21.15.33:6789/0 conn(0x7f9d700bdbb0 legacy=0x7f9d700c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.960+0000 7f9d74b99700 1 -- 172.21.15.33:0/530265277 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946301398 0 0) 0x7f9d700641f0 con 0x7f9d700bdbb0 2023-11-06T20:56:45.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.960+0000 7f9d75b9b700 1 -- 172.21.15.33:0/530265277 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70063ba0 con 0x7f9d701546a0 2023-11-06T20:56:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.960+0000 7f9d75b9b700 1 --2- 172.21.15.33:0/530265277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d701546a0 0x7f9d70064680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f9d6c009760 tx=0x7f9d6c009060).ready entity=mon.1 client_cookie=e1dda4edb57aeef8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d74b99700 1 -- 172.21.15.33:0/530265277 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c00f510 con 0x7f9d701546a0 2023-11-06T20:56:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d74b99700 1 -- 172.21.15.33:0/530265277 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c00f670 con 0x7f9d701546a0 2023-11-06T20:56:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d74b99700 1 -- 172.21.15.33:0/530265277 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c00f9f0 con 0x7f9d701546a0 2023-11-06T20:56:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d77dff700 1 -- 172.21.15.33:0/530265277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d701546a0 msgr2=0x7f9d70064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d77dff700 1 --2- 172.21.15.33:0/530265277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d701546a0 0x7f9d70064680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f9d6c009760 tx=0x7f9d6c009060).stop 2023-11-06T20:56:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d77dff700 1 -- 172.21.15.33:0/530265277 shutdown_connections 2023-11-06T20:56:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d77dff700 1 --2- 172.21.15.33:0/530265277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d701546a0 0x7f9d70064680 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d77dff700 1 --2- 172.21.15.33:0/530265277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d70066560 0x7f9d700669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d77dff700 1 -- 172.21.15.33:0/530265277 >> 172.21.15.33:0/530265277 conn(0x7f9d700c8c50 msgr2=0x7f9d700c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d77dff700 1 -- 172.21.15.33:0/530265277 shutdown_connections 2023-11-06T20:56:45.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.961+0000 7f9d77dff700 1 -- 172.21.15.33:0/530265277 wait complete. 2023-11-06T20:56:45.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.963+0000 7f9d77dff700 1 Processor -- start 2023-11-06T20:56:45.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.963+0000 7f9d77dff700 1 -- start start 2023-11-06T20:56:45.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d77dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d70066560 0x7f9d70168290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d77dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d700bdbb0 0x7f9d70145550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d77dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d701546a0 0x7f9d70067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d77dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700a79d0 con 0x7f9d701546a0 2023-11-06T20:56:45.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d77dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700a7540 con 0x7f9d70066560 2023-11-06T20:56:45.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d77dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700a6930 con 0x7f9d700bdbb0 2023-11-06T20:56:45.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d7639c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d701546a0 0x7f9d70067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d7539a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d700bdbb0 0x7f9d70145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d75b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d70066560 0x7f9d70168290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:45.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d7539a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d700bdbb0 0x7f9d70145550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:33076/0 (socket says 172.21.15.33:33076) 2023-11-06T20:56:45.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.964+0000 7f9d7539a700 1 -- 172.21.15.33:0/3599130293 learned_addr learned my addr 172.21.15.33:0/3599130293 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:45.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d7539a700 1 -- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d70066560 msgr2=0x7f9d70168290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d7539a700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d70066560 0x7f9d70168290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d7539a700 1 -- 172.21.15.33:0/3599130293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d701546a0 msgr2=0x7f9d70067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:45.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d7539a700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d701546a0 0x7f9d70067ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:45.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d7539a700 1 -- 172.21.15.33:0/3599130293 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0092a0 con 0x7f9d700bdbb0 2023-11-06T20:56:45.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d7539a700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d700bdbb0 0x7f9d70145550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9d6000e960 tx=0x7f9d60009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:45.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d65ffb700 1 -- 172.21.15.33:0/3599130293 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60012600 con 0x7f9d700bdbb0 2023-11-06T20:56:45.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d65ffb700 1 -- 172.21.15.33:0/3599130293 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d60012760 con 0x7f9d700bdbb0 2023-11-06T20:56:45.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d77dff700 1 -- 172.21.15.33:0/3599130293 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d700a6c10 con 0x7f9d700bdbb0 2023-11-06T20:56:45.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d65ffb700 1 -- 172.21.15.33:0/3599130293 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60012ae0 con 0x7f9d700bdbb0 2023-11-06T20:56:45.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.965+0000 7f9d77dff700 1 -- 172.21.15.33:0/3599130293 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d700a71d0 con 0x7f9d700bdbb0 2023-11-06T20:56:45.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.967+0000 7f9d77dff700 1 -- 172.21.15.33:0/3599130293 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d70066000 con 0x7f9d700bdbb0 2023-11-06T20:56:45.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.968+0000 7f9d65ffb700 1 -- 172.21.15.33:0/3599130293 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f9d6001b400 con 0x7f9d700bdbb0 2023-11-06T20:56:45.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.969+0000 7f9d65ffb700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d5405f270 0x7f9d54061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:45.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.969+0000 7f9d65ffb700 1 -- 172.21.15.33:0/3599130293 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6891+0+0 (secure 0 0 0) 0x7f9d6007aaf0 con 0x7f9d700bdbb0 2023-11-06T20:56:45.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.969+0000 7f9d75b9b700 1 -- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d5405f270 msgr2=0x7f9d54061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:45.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.969+0000 7f9d75b9b700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d5405f270 0x7f9d54061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:45.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:45.978+0000 7f9d65ffb700 1 -- 172.21.15.33:0/3599130293 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9d6004b8e0 con 0x7f9d700bdbb0 2023-11-06T20:56:46.013 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:46.013+0000 7ffbd7dbc680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:56:46.104 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:46.103+0000 7ffbd7dbc680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:56:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.165+0000 7f9d77dff700 1 -- 172.21.15.33:0/3599130293 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d700a7680 con 0x7f9d700bdbb0 2023-11-06T20:56:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.166+0000 7f9d65ffb700 1 -- 172.21.15.33:0/3599130293 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d6004ef00 con 0x7f9d700bdbb0 2023-11-06T20:56:46.166 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:56:46.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 -- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d5405f270 msgr2=0x7f9d54061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:46.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d5405f270 0x7f9d54061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:46.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 -- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d700bdbb0 msgr2=0x7f9d70145550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:46.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d700bdbb0 0x7f9d70145550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9d6000e960 tx=0x7f9d60009060).stop 2023-11-06T20:56:46.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 -- 172.21.15.33:0/3599130293 shutdown_connections 2023-11-06T20:56:46.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d5405f270 0x7f9d54061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:46.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d701546a0 0x7f9d70067ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d70066560 0x7f9d70168290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 --2- 172.21.15.33:0/3599130293 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d700bdbb0 0x7f9d70145550 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 -- 172.21.15.33:0/3599130293 >> 172.21.15.33:0/3599130293 conn(0x7f9d700c8c50 msgr2=0x7f9d700c3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:46.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 -- 172.21.15.33:0/3599130293 shutdown_connections 2023-11-06T20:56:46.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.168+0000 7f9d77dff700 1 -- 172.21.15.33:0/3599130293 wait complete. 2023-11-06T20:56:46.187 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:56:46.190 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:46.190+0000 7fe2df576680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:56:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.384+0000 7f7b9d7ff700 1 Processor -- start 2023-11-06T20:56:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.384+0000 7f7b9d7ff700 1 -- start start 2023-11-06T20:56:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.385+0000 7f7b9d7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7b9814c640 0x7f7b980bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.385+0000 7f7b9d7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b98152ff0 0x7f7b98065160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.385+0000 7f7b9d7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b980c33f0 0x7f7b980c3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.385+0000 7f7b9d7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b980bd8f0 con 0x7f7b980c33f0 2023-11-06T20:56:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.385+0000 7f7b9d7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b98153470 con 0x7f7b9814c640 2023-11-06T20:56:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.385+0000 7f7b9d7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980c3d70 con 0x7f7b98152ff0 2023-11-06T20:56:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b977fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b980c33f0 0x7f7b980c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b977fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b980c33f0 0x7f7b980c3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40406/0 (socket says 172.21.15.33:40406) 2023-11-06T20:56:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b977fe700 1 -- 172.21.15.33:0/2727184341 learned_addr learned my addr 172.21.15.33:0/2727184341 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b967fc700 1 --2- 172.21.15.33:0/2727184341 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b98152ff0 0x7f7b98065160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b96ffd700 1 --2- 172.21.15.33:0/2727184341 >> v2:172.21.15.69:3300/0 conn(0x7f7b9814c640 0x7f7b980bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b977fe700 1 -- 172.21.15.33:0/2727184341 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b98152ff0 msgr2=0x7f7b98065160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b977fe700 1 --2- 172.21.15.33:0/2727184341 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b98152ff0 0x7f7b98065160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b977fe700 1 -- 172.21.15.33:0/2727184341 >> v2:172.21.15.69:3300/0 conn(0x7f7b9814c640 msgr2=0x7f7b980bb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b977fe700 1 --2- 172.21.15.33:0/2727184341 >> v2:172.21.15.69:3300/0 conn(0x7f7b9814c640 0x7f7b980bb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b977fe700 1 -- 172.21.15.33:0/2727184341 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980643a0 con 0x7f7b980c33f0 2023-11-06T20:56:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.386+0000 7f7b977fe700 1 --2- 172.21.15.33:0/2727184341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b980c33f0 0x7f7b980c3880 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f7b98063bd0 tx=0x7f7b90009840).ready entity=mon.0 client_cookie=e896145a07cfb0ba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b95ffb700 1 -- 172.21.15.33:0/2727184341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000e070 con 0x7f7b980c33f0 2023-11-06T20:56:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b95ffb700 1 -- 172.21.15.33:0/2727184341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9000b970 con 0x7f7b980c33f0 2023-11-06T20:56:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b95ffb700 1 -- 172.21.15.33:0/2727184341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000bcf0 con 0x7f7b980c33f0 2023-11-06T20:56:46.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/2727184341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b980c33f0 msgr2=0x7f7b980c3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:46.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b9d7ff700 1 --2- 172.21.15.33:0/2727184341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b980c33f0 0x7f7b980c3880 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f7b98063bd0 tx=0x7f7b90009840).stop 2023-11-06T20:56:46.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/2727184341 shutdown_connections 2023-11-06T20:56:46.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b9d7ff700 1 --2- 172.21.15.33:0/2727184341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b980c33f0 0x7f7b980c3880 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b9d7ff700 1 --2- 172.21.15.33:0/2727184341 >> v2:172.21.15.69:3300/0 conn(0x7f7b9814c640 0x7f7b980bb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b9d7ff700 1 --2- 172.21.15.33:0/2727184341 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b98152ff0 0x7f7b98065160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/2727184341 >> 172.21.15.33:0/2727184341 conn(0x7f7b98062710 msgr2=0x7f7b98062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.387+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/2727184341 shutdown_connections 2023-11-06T20:56:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.388+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/2727184341 wait complete. 2023-11-06T20:56:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.390+0000 7f7b9d7ff700 1 Processor -- start 2023-11-06T20:56:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.390+0000 7f7b9d7ff700 1 -- start start 2023-11-06T20:56:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b9d7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b980c33f0 0x7f7b980be4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b9d7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b9814c640 0x7f7b980aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b9d7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b98152ff0 0x7f7b980ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b9d7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980aa990 con 0x7f7b9814c640 2023-11-06T20:56:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b9d7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980a9d80 con 0x7f7b980c33f0 2023-11-06T20:56:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b9d7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980aa060 con 0x7f7b98152ff0 2023-11-06T20:56:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b9814c640 0x7f7b980aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b96ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b980c33f0 0x7f7b980be4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b9814c640 0x7f7b980aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40414/0 (socket says 172.21.15.33:40414) 2023-11-06T20:56:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b96ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b980c33f0 0x7f7b980be4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:40492/0 (socket says 172.21.15.33:40492) 2023-11-06T20:56:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.391+0000 7f7b967fc700 1 -- 172.21.15.33:0/1335115903 learned_addr learned my addr 172.21.15.33:0/1335115903 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b977fe700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b98152ff0 0x7f7b980ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b967fc700 1 -- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b98152ff0 msgr2=0x7f7b980ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b967fc700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b98152ff0 0x7f7b980ab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b967fc700 1 -- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b980c33f0 msgr2=0x7f7b980be4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:46.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b967fc700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b980c33f0 0x7f7b980be4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b967fc700 1 -- 172.21.15.33:0/1335115903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90009ae0 con 0x7f7b9814c640 2023-11-06T20:56:46.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b967fc700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b9814c640 0x7f7b980aae20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f7b8c00e900 tx=0x7f7b8c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:46.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b7effd700 1 -- 172.21.15.33:0/1335115903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c012600 con 0x7f7b9814c640 2023-11-06T20:56:46.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b7effd700 1 -- 172.21.15.33:0/1335115903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c012760 con 0x7f7b9814c640 2023-11-06T20:56:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b7effd700 1 -- 172.21.15.33:0/1335115903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c012ae0 con 0x7f7b9814c640 2023-11-06T20:56:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/1335115903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b980aa340 con 0x7f7b9814c640 2023-11-06T20:56:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.392+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/1335115903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b980a99c0 con 0x7f7b9814c640 2023-11-06T20:56:46.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.394+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/1335115903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b980aaad0 con 0x7f7b9814c640 2023-11-06T20:56:46.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.394+0000 7f7b7effd700 1 -- 172.21.15.33:0/1335115903 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f7b8c01b400 con 0x7f7b9814c640 2023-11-06T20:56:46.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.395+0000 7f7b7effd700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b8405f270 0x7f7b84061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.395+0000 7f7b7effd700 1 -- 172.21.15.33:0/1335115903 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6891+0+0 (secure 0 0 0) 0x7f7b8c07aaa0 con 0x7f7b9814c640 2023-11-06T20:56:46.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.395+0000 7f7b96ffd700 1 -- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b8405f270 msgr2=0x7f7b84061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:46.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.395+0000 7f7b96ffd700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b8405f270 0x7f7b84061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:46.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.403+0000 7f7b7effd700 1 -- 172.21.15.33:0/1335115903 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7b8c04b7d0 con 0x7f7b9814c640 2023-11-06T20:56:46.412 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:46.412+0000 7fe2df576680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:56:46.446 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:46.446+0000 7ffbd7dbc680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:56:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.596+0000 7f7b96ffd700 1 -- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b8405f270 msgr2=0x7f7b84061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.596+0000 7f7b96ffd700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b8405f270 0x7f7b84061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:56:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.633+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/1335115903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b9805f0b0 con 0x7f7b9814c640 2023-11-06T20:56:46.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.634+0000 7f7b7effd700 1 -- 172.21.15.33:0/1335115903 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f7b8c04edf0 con 0x7f7b9814c640 2023-11-06T20:56:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:46.635 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/2191444463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2191444463},{"type":"v1","addr":"172.21.15.69:6836","nonce":2191444463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3920247513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3920247513},{"type":"v1","addr":"172.21.15.33:6834","nonce":3920247513}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2046477314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2046477314},{"type":"v1","addr":"172.21.15.69:6837","nonce":2046477314}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:45:04.669326+0000","modified":"2023-11-06T20:45:07.773858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/3657599674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3657599674},{"type":"v1","addr":"172.21.15.33:6835","nonce":3657599674}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-06T20:56:46.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b8405f270 msgr2=0x7f7b84061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:46.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b8405f270 0x7f7b84061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:46.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/1335115903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b9814c640 msgr2=0x7f7b980aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b9814c640 0x7f7b980aae20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f7b8c00e900 tx=0x7f7b8c009060).stop 2023-11-06T20:56:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/1335115903 shutdown_connections 2023-11-06T20:56:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b8405f270 0x7f7b84061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b98152ff0 0x7f7b980ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b980c33f0 0x7f7b980be4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 --2- 172.21.15.33:0/1335115903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b9814c640 0x7f7b980aae20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/1335115903 >> 172.21.15.33:0/1335115903 conn(0x7f7b98062710 msgr2=0x7f7b9814a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.636+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/1335115903 shutdown_connections 2023-11-06T20:56:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.637+0000 7f7b9d7ff700 1 -- 172.21.15.33:0/1335115903 wait complete. 2023-11-06T20:56:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 15 2023-11-06T20:56:46.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:56:46.679 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:46.679+0000 7ffbd7dbc680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:56:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.858+0000 7fae42fff700 1 Processor -- start 2023-11-06T20:56:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.858+0000 7fae42fff700 1 -- start start 2023-11-06T20:56:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.859+0000 7fae42fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fae3c0bc7e0 0x7fae3c064400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.859+0000 7fae42fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fae3c0648f0 0x7fae3c0c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.859+0000 7fae42fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c0c1940 con 0x7fae3c14ae80 2023-11-06T20:56:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.859+0000 7fae42fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c14b260 con 0x7fae3c0bc7e0 2023-11-06T20:56:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.859+0000 7fae42fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c151cf0 con 0x7fae3c0648f0 2023-11-06T20:56:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.860+0000 7fae3bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fae3c0bc7e0 0x7fae3c064400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.860+0000 7fae3bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fae3c0bc7e0 0x7fae3c064400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40424/0 (socket says 172.21.15.33:40424) 2023-11-06T20:56:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.860+0000 7fae3bfff700 1 -- 172.21.15.33:0/3960737659 learned_addr learned my addr 172.21.15.33:0/3960737659 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.860+0000 7fae4159c700 1 --2- 172.21.15.33:0/3960737659 >> v2:172.21.15.69:3300/0 conn(0x7fae3c0648f0 0x7fae3c0c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.860+0000 7fae3bfff700 1 -- 172.21.15.33:0/3960737659 >> v2:172.21.15.69:3300/0 conn(0x7fae3c0648f0 msgr2=0x7fae3c0c1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.860+0000 7fae3bfff700 1 --2- 172.21.15.33:0/3960737659 >> v2:172.21.15.69:3300/0 conn(0x7fae3c0648f0 0x7fae3c0c1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.860+0000 7fae3bfff700 1 -- 172.21.15.33:0/3960737659 >> v1:172.21.15.69:6789/0 conn(0x7fae3c14ae80 legacy=0x7fae3c0bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.860+0000 7fae3bfff700 1 -- 172.21.15.33:0/3960737659 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c065160 con 0x7fae3c0bc7e0 2023-11-06T20:56:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.860+0000 7fae3bfff700 1 --2- 172.21.15.33:0/3960737659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c0bc7e0 0x7fae3c064400 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fae3c0bbdd0 tx=0x7fae280090f0).ready entity=mon.0 client_cookie=258e5aca339c4636 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae3b7fe700 1 -- 172.21.15.33:0/3960737659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2800b520 con 0x7fae3c0bc7e0 2023-11-06T20:56:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae3b7fe700 1 -- 172.21.15.33:0/3960737659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2800b680 con 0x7fae3c0bc7e0 2023-11-06T20:56:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae3b7fe700 1 -- 172.21.15.33:0/3960737659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2800ba00 con 0x7fae3c0bc7e0 2023-11-06T20:56:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae42fff700 1 -- 172.21.15.33:0/3960737659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c0bc7e0 msgr2=0x7fae3c064400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae42fff700 1 --2- 172.21.15.33:0/3960737659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c0bc7e0 0x7fae3c064400 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fae3c0bbdd0 tx=0x7fae280090f0).stop 2023-11-06T20:56:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae42fff700 1 -- 172.21.15.33:0/3960737659 shutdown_connections 2023-11-06T20:56:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae42fff700 1 --2- 172.21.15.33:0/3960737659 >> v2:172.21.15.69:3300/0 conn(0x7fae3c0648f0 0x7fae3c0c1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae42fff700 1 --2- 172.21.15.33:0/3960737659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c0bc7e0 0x7fae3c064400 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae42fff700 1 -- 172.21.15.33:0/3960737659 >> 172.21.15.33:0/3960737659 conn(0x7fae3c062710 msgr2=0x7fae3c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:46.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae42fff700 1 -- 172.21.15.33:0/3960737659 shutdown_connections 2023-11-06T20:56:46.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.861+0000 7fae42fff700 1 -- 172.21.15.33:0/3960737659 wait complete. 2023-11-06T20:56:46.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.863+0000 7fae42fff700 1 Processor -- start 2023-11-06T20:56:46.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.863+0000 7fae42fff700 1 -- start start 2023-11-06T20:56:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae42fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae3c0648f0 0x7fae3c164cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae42fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fae3c0bc7e0 0x7fae3c14b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae42fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c14ae80 0x7fae3c14b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae42fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c14bd80 con 0x7fae3c14ae80 2023-11-06T20:56:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae42fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c161c90 con 0x7fae3c0648f0 2023-11-06T20:56:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae42fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c14d6c0 con 0x7fae3c0bc7e0 2023-11-06T20:56:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae4159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c14ae80 0x7fae3c14b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae40d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae3c0648f0 0x7fae3c164cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae4159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c14ae80 0x7fae3c14b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40432/0 (socket says 172.21.15.33:40432) 2023-11-06T20:56:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae4159c700 1 -- 172.21.15.33:0/1802453313 learned_addr learned my addr 172.21.15.33:0/1802453313 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.864+0000 7fae3bfff700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fae3c0bc7e0 0x7fae3c14b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae4159c700 1 -- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fae3c0bc7e0 msgr2=0x7fae3c14b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae4159c700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fae3c0bc7e0 0x7fae3c14b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae4159c700 1 -- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae3c0648f0 msgr2=0x7fae3c164cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae4159c700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae3c0648f0 0x7fae3c164cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae4159c700 1 -- 172.21.15.33:0/1802453313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae28009370 con 0x7fae3c14ae80 2023-11-06T20:56:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae4159c700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c14ae80 0x7fae3c14b890 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fae34009ec0 tx=0x7fae3400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae38ff9700 1 -- 172.21.15.33:0/1802453313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400cb10 con 0x7fae3c14ae80 2023-11-06T20:56:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae38ff9700 1 -- 172.21.15.33:0/1802453313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3400f040 con 0x7fae3c14ae80 2023-11-06T20:56:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae38ff9700 1 -- 172.21.15.33:0/1802453313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400ce80 con 0x7fae3c14ae80 2023-11-06T20:56:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae42fff700 1 -- 172.21.15.33:0/1802453313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c14c180 con 0x7fae3c14ae80 2023-11-06T20:56:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.865+0000 7fae42fff700 1 -- 172.21.15.33:0/1802453313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c14c9a0 con 0x7fae3c14ae80 2023-11-06T20:56:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.867+0000 7fae42fff700 1 -- 172.21.15.33:0/1802453313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c14d530 con 0x7fae3c14ae80 2023-11-06T20:56:46.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.867+0000 7fae38ff9700 1 -- 172.21.15.33:0/1802453313 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fae34013610 con 0x7fae3c14ae80 2023-11-06T20:56:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.869+0000 7fae38ff9700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae2005f270 0x7fae20061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.869+0000 7fae38ff9700 1 -- 172.21.15.33:0/1802453313 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6891+0+0 (secure 0 0 0) 0x7fae3407aef0 con 0x7fae3c14ae80 2023-11-06T20:56:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.869+0000 7fae40d9b700 1 -- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae2005f270 msgr2=0x7fae20061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.869+0000 7fae40d9b700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae2005f270 0x7fae20061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:46.873+0000 7fae38ff9700 1 -- 172.21.15.33:0/1802453313 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fae3404bc80 con 0x7fae3c14ae80 2023-11-06T20:56:47.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.058+0000 7fae42fff700 1 -- 172.21.15.33:0/1802453313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae3c05f0b0 con 0x7fae3c14ae80 2023-11-06T20:56:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.059+0000 7fae38ff9700 1 -- 172.21.15.33:0/1802453313 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+18173 (secure 0 0 0) 0x7fae3404f2a0 con 0x7fae3c14ae80 2023-11-06T20:56:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:47.060 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":52,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:56:44.937429+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:45:02.985899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T20:45:03.637532+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":52,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.33","mappings":[{"from":4,"to":5}]},{"pgid":"5.35","mappings":[{"from":4,"to":5}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]},{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.18","mappings":[{"from":4,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6832/114912":"2023-11-07T20:45:24.054860+0000","172.21.15.69:6833/114912":"2023-11-07T20:45:24.054860+0000","172.21.15.69:0/1201458877":"2023-11-07T20:45:24.054860+0000","172.21.15.69:0/1049662608":"2023-11-07T20:45:24.054860+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":4,"length":2},{"begin":9,"length":1},{"begin":14,"length":2},{"begin":19,"length":1},{"begin":24,"length":2},{"begin":29,"length":1},{"begin":34,"length":2},{"begin":39,"length":1},{"begin":44,"length":2},{"begin":49,"length":1},{"begin":54,"length":2},{"begin":59,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":74,"length":2},{"begin":79,"length":2},{"begin":84,"length":2},{"begin":89,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":183,"length":39}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":105,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:56:47.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 -- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae2005f270 msgr2=0x7fae20061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:47.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae2005f270 0x7fae20061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:47.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 -- 172.21.15.33:0/1802453313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c14ae80 msgr2=0x7fae3c14b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c14ae80 0x7fae3c14b890 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fae34009ec0 tx=0x7fae3400b040).stop 2023-11-06T20:56:47.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 -- 172.21.15.33:0/1802453313 shutdown_connections 2023-11-06T20:56:47.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae2005f270 0x7fae20061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:47.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c14ae80 0x7fae3c14b890 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae3c0648f0 0x7fae3c164cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 --2- 172.21.15.33:0/1802453313 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fae3c0bc7e0 0x7fae3c14b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 -- 172.21.15.33:0/1802453313 >> 172.21.15.33:0/1802453313 conn(0x7fae3c062710 msgr2=0x7fae3c0c5670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:47.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 -- 172.21.15.33:0/1802453313 shutdown_connections 2023-11-06T20:56:47.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.061+0000 7fae42fff700 1 -- 172.21.15.33:0/1802453313 wait complete. 2023-11-06T20:56:47.080 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T20:56:47.080 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:56:47.223 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:47.223+0000 7fe2df576680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:56:47.309 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:47.310+0000 7fe2df576680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:56:47.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.325+0000 7fa7327ff700 1 Processor -- start 2023-11-06T20:56:47.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.325+0000 7fa7327ff700 1 -- start start 2023-11-06T20:56:47.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.327+0000 7fa7327ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c39e0 0x7fa72c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.327+0000 7fa7327ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c0c17e0 con 0x7fa72c137660 2023-11-06T20:56:47.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.327+0000 7fa7327ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c06a740 con 0x7fa72c0c3600 2023-11-06T20:56:47.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.327+0000 7fa7327ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c06a180 con 0x7fa72c0c39e0 2023-11-06T20:56:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.327+0000 7fa72b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa72c137660 0x7fa72c069410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50582/0 (socket says 172.21.15.33:50582) 2023-11-06T20:56:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.327+0000 7fa72b7fe700 1 -- 172.21.15.33:0/1642784229 learned_addr learned my addr 172.21.15.33:0/1642784229 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.327+0000 7fa730d9c700 1 --2- 172.21.15.33:0/1642784229 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c39e0 0x7fa72c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.327+0000 7fa72affd700 1 -- 172.21.15.33:0/1642784229 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261486154 0 0) 0x7fa72c0c17e0 con 0x7fa72c137660 2023-11-06T20:56:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.327+0000 7fa72affd700 1 -- 172.21.15.33:0/1642784229 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7140033f0 con 0x7fa72c137660 2023-11-06T20:56:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.327+0000 7fa72affd700 1 -- 172.21.15.33:0/1642784229 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235582018 0 0) 0x7fa72c06a740 con 0x7fa72c0c3600 2023-11-06T20:56:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.328+0000 7fa72affd700 1 -- 172.21.15.33:0/1642784229 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa72c0c17e0 con 0x7fa72c0c3600 2023-11-06T20:56:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.328+0000 7fa730d9c700 1 -- 172.21.15.33:0/1642784229 >> v1:172.21.15.69:6789/0 conn(0x7fa72c0c3600 legacy=0x7fa72c136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.328+0000 7fa730d9c700 1 -- 172.21.15.33:0/1642784229 >> v1:172.21.15.33:6789/0 conn(0x7fa72c137660 legacy=0x7fa72c069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.328+0000 7fa730d9c700 1 -- 172.21.15.33:0/1642784229 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c06a460 con 0x7fa72c0c39e0 2023-11-06T20:56:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.328+0000 7fa730d9c700 1 --2- 172.21.15.33:0/1642784229 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c39e0 0x7fa72c069b20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa724009e90 tx=0x7fa724009780).ready entity=mon.2 client_cookie=51ba1981dadc4d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.328+0000 7fa72affd700 1 -- 172.21.15.33:0/1642784229 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72400e070 con 0x7fa72c0c39e0 2023-11-06T20:56:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.328+0000 7fa72affd700 1 -- 172.21.15.33:0/1642784229 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72400bab0 con 0x7fa72c0c39e0 2023-11-06T20:56:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.328+0000 7fa72affd700 1 -- 172.21.15.33:0/1642784229 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72400be30 con 0x7fa72c0c39e0 2023-11-06T20:56:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.329+0000 7fa7327ff700 1 -- 172.21.15.33:0/1642784229 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c39e0 msgr2=0x7fa72c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.329+0000 7fa7327ff700 1 --2- 172.21.15.33:0/1642784229 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c39e0 0x7fa72c069b20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa724009e90 tx=0x7fa724009780).stop 2023-11-06T20:56:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.329+0000 7fa7327ff700 1 -- 172.21.15.33:0/1642784229 shutdown_connections 2023-11-06T20:56:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.329+0000 7fa7327ff700 1 --2- 172.21.15.33:0/1642784229 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c39e0 0x7fa72c069b20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.329+0000 7fa7327ff700 1 -- 172.21.15.33:0/1642784229 >> 172.21.15.33:0/1642784229 conn(0x7fa72c062710 msgr2=0x7fa72c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.329+0000 7fa7327ff700 1 -- 172.21.15.33:0/1642784229 shutdown_connections 2023-11-06T20:56:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.329+0000 7fa7327ff700 1 -- 172.21.15.33:0/1642784229 wait complete. 2023-11-06T20:56:47.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.330+0000 7fa7327ff700 1 Processor -- start 2023-11-06T20:56:47.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.331+0000 7fa7327ff700 1 -- start start 2023-11-06T20:56:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.331+0000 7fa7327ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c3600 0x7fa72c165120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa7327ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72c0c39e0 0x7fa72c0c0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa7327ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c137660 0x7fa72c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa7327ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0c0b10 con 0x7fa72c137660 2023-11-06T20:56:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa7327ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0bfbe0 con 0x7fa72c0c39e0 2023-11-06T20:56:47.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa7327ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c1656c0 con 0x7fa72c0c3600 2023-11-06T20:56:47.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa730d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c137660 0x7fa72c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:47.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa72b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72c0c39e0 0x7fa72c0c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:47.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa730d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c137660 0x7fa72c0a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40444/0 (socket says 172.21.15.33:40444) 2023-11-06T20:56:47.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa730d9c700 1 -- 172.21.15.33:0/2302080437 learned_addr learned my addr 172.21.15.33:0/2302080437 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:47.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa72bfff700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c3600 0x7fa72c165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:47.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa730d9c700 1 -- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c3600 msgr2=0x7fa72c165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa730d9c700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c3600 0x7fa72c165120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa730d9c700 1 -- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72c0c39e0 msgr2=0x7fa72c0c0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa730d9c700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72c0c39e0 0x7fa72c0c0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.332+0000 7fa730d9c700 1 -- 172.21.15.33:0/2302080437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724009a20 con 0x7fa72c137660 2023-11-06T20:56:47.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.333+0000 7fa730d9c700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c137660 0x7fa72c0a52c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7fa72400b610 tx=0x7fa72c06a8f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:47.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.333+0000 7fa713fff700 1 -- 172.21.15.33:0/2302080437 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72401d630 con 0x7fa72c137660 2023-11-06T20:56:47.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.333+0000 7fa713fff700 1 -- 172.21.15.33:0/2302080437 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72401d790 con 0x7fa72c137660 2023-11-06T20:56:47.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.333+0000 7fa7327ff700 1 -- 172.21.15.33:0/2302080437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c0bf2b0 con 0x7fa72c137660 2023-11-06T20:56:47.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.333+0000 7fa713fff700 1 -- 172.21.15.33:0/2302080437 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72401db50 con 0x7fa72c137660 2023-11-06T20:56:47.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.333+0000 7fa7327ff700 1 -- 172.21.15.33:0/2302080437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c0bf870 con 0x7fa72c137660 2023-11-06T20:56:47.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.334+0000 7fa7327ff700 1 -- 172.21.15.33:0/2302080437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c0bfd70 con 0x7fa72c137660 2023-11-06T20:56:47.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.335+0000 7fa713fff700 1 -- 172.21.15.33:0/2302080437 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fa724003040 con 0x7fa72c137660 2023-11-06T20:56:47.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.336+0000 7fa713fff700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa71805f270 0x7fa718061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.336+0000 7fa713fff700 1 -- 172.21.15.33:0/2302080437 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6891+0+0 (secure 0 0 0) 0x7fa72407c530 con 0x7fa72c137660 2023-11-06T20:56:47.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.336+0000 7fa72bfff700 1 -- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa71805f270 msgr2=0x7fa718061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:47.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.336+0000 7fa72bfff700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa71805f270 0x7fa718061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:47.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.340+0000 7fa713fff700 1 -- 172.21.15.33:0/2302080437 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa72404d380 con 0x7fa72c137660 2023-11-06T20:56:47.404 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:47.404+0000 7fe2df576680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:56:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.537+0000 7fa72bfff700 1 -- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa71805f270 msgr2=0x7fa718061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.537+0000 7fa72bfff700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa71805f270 0x7fa718061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:56:47.554 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:47.554+0000 7ffbd7dbc680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:56:47.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.565+0000 7fa7327ff700 1 -- 172.21.15.33:0/2302080437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa72c065750 con 0x7fa72c137660 2023-11-06T20:56:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.566+0000 7fa713fff700 1 -- 172.21.15.33:0/2302080437 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4489 (secure 0 0 0) 0x7fa7240509a0 con 0x7fa72c137660 2023-11-06T20:56:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:47.567 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/2191444463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2191444463},{"type":"v1","addr":"172.21.15.69:6836","nonce":2191444463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3920247513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3920247513},{"type":"v1","addr":"172.21.15.33:6834","nonce":3920247513}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2046477314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2046477314},{"type":"v1","addr":"172.21.15.69:6837","nonce":2046477314}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:45:04.669326+0000","modified":"2023-11-06T20:45:07.773858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/3657599674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3657599674},{"type":"v1","addr":"172.21.15.33:6835","nonce":3657599674}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-06T20:56:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.568+0000 7fa7327ff700 1 -- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa71805f270 msgr2=0x7fa718061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.568+0000 7fa7327ff700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa71805f270 0x7fa718061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.568+0000 7fa7327ff700 1 -- 172.21.15.33:0/2302080437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c137660 msgr2=0x7fa72c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.568+0000 7fa7327ff700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c137660 0x7fa72c0a52c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7fa72400b610 tx=0x7fa72c06a8f0).stop 2023-11-06T20:56:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.568+0000 7fa7327ff700 1 -- 172.21.15.33:0/2302080437 shutdown_connections 2023-11-06T20:56:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.568+0000 7fa7327ff700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa71805f270 0x7fa718061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.568+0000 7fa7327ff700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa72c0c3600 0x7fa72c165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.568+0000 7fa7327ff700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c137660 0x7fa72c0a52c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.568+0000 7fa7327ff700 1 --2- 172.21.15.33:0/2302080437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72c0c39e0 0x7fa72c0c0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.568+0000 7fa7327ff700 1 -- 172.21.15.33:0/2302080437 >> 172.21.15.33:0/2302080437 conn(0x7fa72c062710 msgr2=0x7fa72c14fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.569+0000 7fa7327ff700 1 -- 172.21.15.33:0/2302080437 shutdown_connections 2023-11-06T20:56:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.569+0000 7fa7327ff700 1 -- 172.21.15.33:0/2302080437 wait complete. 2023-11-06T20:56:47.572 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 15 2023-11-06T20:56:47.585 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:47.585+0000 7fe2df576680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:56:47.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:56:47.650 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:47.650+0000 7ffbd7dbc680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:56:47.672 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:47.673+0000 7fe2df576680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:56:47.751 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:47.751+0000 7ffbd7dbc680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:56:47.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.785+0000 7f68865ff700 1 Processor -- start 2023-11-06T20:56:47.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.786+0000 7f68865ff700 1 -- start start 2023-11-06T20:56:47.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.787+0000 7f68865ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6880152250 0x7f68800638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.787+0000 7f68865ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f68800c4490 0x7f68800c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.787+0000 7f68865ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68800c3380 con 0x7f68800ac690 2023-11-06T20:56:47.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.787+0000 7f68865ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68800c20d0 con 0x7f6880152250 2023-11-06T20:56:47.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.787+0000 7f68865ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688005e520 con 0x7f68800c4490 2023-11-06T20:56:47.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.788+0000 7f687ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6880152250 0x7f68800638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:47.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.788+0000 7f687ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6880152250 0x7f68800638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40446/0 (socket says 172.21.15.33:40446) 2023-11-06T20:56:47.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.788+0000 7f687ffff700 1 -- 172.21.15.33:0/1500741784 learned_addr learned my addr 172.21.15.33:0/1500741784 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:47.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.788+0000 7f687f7fe700 1 --2- 172.21.15.33:0/1500741784 >> v2:172.21.15.69:3300/0 conn(0x7f68800c4490 0x7f68800c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.788+0000 7f687ffff700 1 -- 172.21.15.33:0/1500741784 >> v2:172.21.15.69:3300/0 conn(0x7f68800c4490 msgr2=0x7f68800c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.788+0000 7f687ffff700 1 --2- 172.21.15.33:0/1500741784 >> v2:172.21.15.69:3300/0 conn(0x7f68800c4490 0x7f68800c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.788+0000 7f687ffff700 1 -- 172.21.15.33:0/1500741784 >> v1:172.21.15.69:6789/0 conn(0x7f68800ac690 legacy=0x7f68800acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.788+0000 7f687ffff700 1 -- 172.21.15.33:0/1500741784 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688005e800 con 0x7f6880152250 2023-11-06T20:56:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.788+0000 7f687effd700 1 -- 172.21.15.33:0/1500741784 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143660124 0 0) 0x7f68800c3380 con 0x7f68800ac690 2023-11-06T20:56:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.788+0000 7f687ffff700 1 --2- 172.21.15.33:0/1500741784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880152250 0x7f68800638b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f6880068530 tx=0x7f68700090f0).ready entity=mon.0 client_cookie=8b0a30b467e0cbd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.789+0000 7f687effd700 1 -- 172.21.15.33:0/1500741784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000b520 con 0x7f6880152250 2023-11-06T20:56:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.789+0000 7f687effd700 1 -- 172.21.15.33:0/1500741784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687000b680 con 0x7f6880152250 2023-11-06T20:56:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.789+0000 7f687effd700 1 -- 172.21.15.33:0/1500741784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000ba00 con 0x7f6880152250 2023-11-06T20:56:47.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.789+0000 7f68865ff700 1 -- 172.21.15.33:0/1500741784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880152250 msgr2=0x7f68800638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.789+0000 7f68865ff700 1 --2- 172.21.15.33:0/1500741784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880152250 0x7f68800638b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f6880068530 tx=0x7f68700090f0).stop 2023-11-06T20:56:47.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.789+0000 7f68865ff700 1 -- 172.21.15.33:0/1500741784 shutdown_connections 2023-11-06T20:56:47.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.789+0000 7f68865ff700 1 --2- 172.21.15.33:0/1500741784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880152250 0x7f68800638b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.789+0000 7f68865ff700 1 --2- 172.21.15.33:0/1500741784 >> v2:172.21.15.69:3300/0 conn(0x7f68800c4490 0x7f68800c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.789+0000 7f68865ff700 1 -- 172.21.15.33:0/1500741784 >> 172.21.15.33:0/1500741784 conn(0x7f68800c7d60 msgr2=0x7f68800c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:47.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.790+0000 7f68865ff700 1 -- 172.21.15.33:0/1500741784 shutdown_connections 2023-11-06T20:56:47.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.790+0000 7f68865ff700 1 -- 172.21.15.33:0/1500741784 wait complete. 2023-11-06T20:56:47.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.792+0000 7f68865ff700 1 Processor -- start 2023-11-06T20:56:47.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.792+0000 7f68865ff700 1 -- start start 2023-11-06T20:56:47.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f68865ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68800ac690 0x7f6880156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f68865ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68800c4490 0x7f6880168900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f68865ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6880152250 0x7f68800a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f68865ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6880168140 con 0x7f68800ac690 2023-11-06T20:56:47.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f68865ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68801572e0 con 0x7f6880152250 2023-11-06T20:56:47.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f68865ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68801575c0 con 0x7f68800c4490 2023-11-06T20:56:47.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f687ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68800ac690 0x7f6880156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:47.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f687ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68800ac690 0x7f6880156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40452/0 (socket says 172.21.15.33:40452) 2023-11-06T20:56:47.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f687ffff700 1 -- 172.21.15.33:0/1401006428 learned_addr learned my addr 172.21.15.33:0/1401006428 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:47.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f6884b9c700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6880152250 0x7f68800a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:47.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f687f7fe700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68800c4490 0x7f6880168900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:47.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f687ffff700 1 -- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68800c4490 msgr2=0x7f6880168900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f687ffff700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68800c4490 0x7f6880168900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f687ffff700 1 -- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6880152250 msgr2=0x7f68800a8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:47.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f687ffff700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6880152250 0x7f68800a8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:47.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f687ffff700 1 -- 172.21.15.33:0/1401006428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6870009370 con 0x7f68800ac690 2023-11-06T20:56:47.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.793+0000 7f687f7fe700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68800c4490 0x7f6880168900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:47.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.794+0000 7f687ffff700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68800ac690 0x7f6880156ab0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f6870009130 tx=0x7f6870017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:47.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.794+0000 7f6867fff700 1 -- 172.21.15.33:0/1401006428 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000be90 con 0x7f68800ac690 2023-11-06T20:56:47.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.794+0000 7f6867fff700 1 -- 172.21.15.33:0/1401006428 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687000b7f0 con 0x7f68800ac690 2023-11-06T20:56:47.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.794+0000 7f6867fff700 1 -- 172.21.15.33:0/1401006428 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687001b660 con 0x7f68800ac690 2023-11-06T20:56:47.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.794+0000 7f68865ff700 1 -- 172.21.15.33:0/1401006428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68801578a0 con 0x7f68800ac690 2023-11-06T20:56:47.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.794+0000 7f68865ff700 1 -- 172.21.15.33:0/1401006428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6880156fa0 con 0x7f68800ac690 2023-11-06T20:56:47.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.795+0000 7f68865ff700 1 -- 172.21.15.33:0/1401006428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68800c42d0 con 0x7f68800ac690 2023-11-06T20:56:47.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.796+0000 7f6867fff700 1 -- 172.21.15.33:0/1401006428 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f687001b9d0 con 0x7f68800ac690 2023-11-06T20:56:47.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.797+0000 7f6867fff700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f686c05f270 0x7f686c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:47.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.797+0000 7f6867fff700 1 -- 172.21.15.33:0/1401006428 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6891+0+0 (secure 0 0 0) 0x7f687007bee0 con 0x7f68800ac690 2023-11-06T20:56:47.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.797+0000 7f687f7fe700 1 -- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f686c05f270 msgr2=0x7f686c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.797+0000 7f687f7fe700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f686c05f270 0x7f686c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.801+0000 7f6867fff700 1 -- 172.21.15.33:0/1401006428 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f687004cd30 con 0x7f68800ac690 2023-11-06T20:56:47.814 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:47.815+0000 7fe2df576680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:56:47.943 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:47.943+0000 7fe2df576680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:56:47.949 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:47.949+0000 7ffbd7dbc680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:56:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.998+0000 7f687f7fe700 1 -- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f686c05f270 msgr2=0x7f686c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:47.998+0000 7f687f7fe700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f686c05f270 0x7f686c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:56:48.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.027+0000 7f68865ff700 1 -- 172.21.15.33:0/1401006428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6880064420 con 0x7f68800ac690 2023-11-06T20:56:48.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.028+0000 7f6867fff700 1 -- 172.21.15.33:0/1401006428 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f6870050350 con 0x7f68800ac690 2023-11-06T20:56:48.028 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:48.028 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/2191444463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2191444463},{"type":"v1","addr":"172.21.15.69:6836","nonce":2191444463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3920247513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3920247513},{"type":"v1","addr":"172.21.15.33:6834","nonce":3920247513}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2046477314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2046477314},{"type":"v1","addr":"172.21.15.69:6837","nonce":2046477314}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:45:04.669326+0000","modified":"2023-11-06T20:45:07.773858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/3657599674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3657599674},{"type":"v1","addr":"172.21.15.33:6835","nonce":3657599674}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-06T20:56:48.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.030+0000 7f68865ff700 1 -- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f686c05f270 msgr2=0x7f686c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:48.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.030+0000 7f68865ff700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f686c05f270 0x7f686c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:48.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.030+0000 7f68865ff700 1 -- 172.21.15.33:0/1401006428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68800ac690 msgr2=0x7f6880156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:48.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.030+0000 7f68865ff700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68800ac690 0x7f6880156ab0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f6870009130 tx=0x7f6870017000).stop 2023-11-06T20:56:48.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.030+0000 7f68865ff700 1 -- 172.21.15.33:0/1401006428 shutdown_connections 2023-11-06T20:56:48.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.030+0000 7f68865ff700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f686c05f270 0x7f686c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:48.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.030+0000 7f68865ff700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6880152250 0x7f68800a8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.030+0000 7f68865ff700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68800ac690 0x7f6880156ab0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.030+0000 7f68865ff700 1 --2- 172.21.15.33:0/1401006428 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68800c4490 0x7f6880168900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.030+0000 7f68865ff700 1 -- 172.21.15.33:0/1401006428 >> 172.21.15.33:0/1401006428 conn(0x7f68800c7d60 msgr2=0x7f68801511e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:48.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.031+0000 7f68865ff700 1 -- 172.21.15.33:0/1401006428 shutdown_connections 2023-11-06T20:56:48.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.031+0000 7f68865ff700 1 -- 172.21.15.33:0/1401006428 wait complete. 2023-11-06T20:56:48.033 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 15 2023-11-06T20:56:48.044 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:48.044+0000 7ffbd7dbc680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:56:48.048 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:56:48.199 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:48.199+0000 7ffbd7dbc680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:56:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.250+0000 7f4a6bfff700 1 Processor -- start 2023-11-06T20:56:48.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.250+0000 7f4a6bfff700 1 -- start start 2023-11-06T20:56:48.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.252+0000 7f4a6bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a6c151d90 0x7f4a6c0c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.252+0000 7f4a6bfff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4a6c152170 0x7f4a6c0c6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.252+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 0x7f4a6c0645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.252+0000 7f4a6bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6c154960 con 0x7f4a6c151d90 2023-11-06T20:56:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.252+0000 7f4a6bfff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6c0c6900 con 0x7f4a6c152170 2023-11-06T20:56:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.252+0000 7f4a6bfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c1551a0 con 0x7f4a6c155630 2023-11-06T20:56:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a6c151d90 0x7f4a6c0c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a6c151d90 0x7f4a6c0c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47808/0 (socket says 172.21.15.33:47808) 2023-11-06T20:56:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6affd700 1 -- 172.21.15.33:0/149903498 learned_addr learned my addr 172.21.15.33:0/149903498 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6b7fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 0x7f4a6c0645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6a7fc700 1 --2- 172.21.15.33:0/149903498 >> v2:172.21.15.69:3300/0 conn(0x7f4a6c152170 0x7f4a6c0c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6affd700 1 -- 172.21.15.33:0/149903498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 msgr2=0x7f4a6c0645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6affd700 1 --2- 172.21.15.33:0/149903498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 0x7f4a6c0645e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6affd700 1 -- 172.21.15.33:0/149903498 >> v2:172.21.15.69:3300/0 conn(0x7f4a6c152170 msgr2=0x7f4a6c0c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6affd700 1 --2- 172.21.15.33:0/149903498 >> v2:172.21.15.69:3300/0 conn(0x7f4a6c152170 0x7f4a6c0c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6affd700 1 -- 172.21.15.33:0/149903498 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c0c6620 con 0x7f4a6c151d90 2023-11-06T20:56:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6b7fe700 1 --2- 172.21.15.33:0/149903498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 0x7f4a6c0645e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.253+0000 7f4a6affd700 1 --2- 172.21.15.33:0/149903498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 0x7f4a6c0c5b80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f4a6c0ad880 tx=0x7f4a5c0090f0).ready entity=mon.0 client_cookie=bd77a968804a413e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a69ffb700 1 -- 172.21.15.33:0/149903498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00f510 con 0x7f4a6c151d90 2023-11-06T20:56:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a69ffb700 1 -- 172.21.15.33:0/149903498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5c00f670 con 0x7f4a6c151d90 2023-11-06T20:56:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a69ffb700 1 -- 172.21.15.33:0/149903498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00f9f0 con 0x7f4a6c151d90 2023-11-06T20:56:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a6bfff700 1 -- 172.21.15.33:0/149903498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 msgr2=0x7f4a6c0c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a6bfff700 1 --2- 172.21.15.33:0/149903498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 0x7f4a6c0c5b80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f4a6c0ad880 tx=0x7f4a5c0090f0).stop 2023-11-06T20:56:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a6bfff700 1 -- 172.21.15.33:0/149903498 shutdown_connections 2023-11-06T20:56:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a6bfff700 1 --2- 172.21.15.33:0/149903498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 0x7f4a6c0645e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a6bfff700 1 --2- 172.21.15.33:0/149903498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 0x7f4a6c0c5b80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a6bfff700 1 --2- 172.21.15.33:0/149903498 >> v2:172.21.15.69:3300/0 conn(0x7f4a6c152170 0x7f4a6c0c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a6bfff700 1 -- 172.21.15.33:0/149903498 >> 172.21.15.33:0/149903498 conn(0x7f4a6c0cb780 msgr2=0x7f4a6c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a6bfff700 1 -- 172.21.15.33:0/149903498 shutdown_connections 2023-11-06T20:56:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.254+0000 7f4a6bfff700 1 -- 172.21.15.33:0/149903498 wait complete. 2023-11-06T20:56:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.256+0000 7f4a6bfff700 1 Processor -- start 2023-11-06T20:56:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.256+0000 7f4a6bfff700 1 -- start start 2023-11-06T20:56:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 0x7f4a6c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a6c152170 0x7f4a6c14c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 0x7f4a6c14c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c14caa0 con 0x7f4a6c151d90 2023-11-06T20:56:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6bfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0af390 con 0x7f4a6c152170 2023-11-06T20:56:48.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6bfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0ae560 con 0x7f4a6c155630 2023-11-06T20:56:48.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 0x7f4a6c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 0x7f4a6c0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47810/0 (socket says 172.21.15.33:47810) 2023-11-06T20:56:48.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6affd700 1 -- 172.21.15.33:0/2991964153 learned_addr learned my addr 172.21.15.33:0/2991964153 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:48.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6b7fe700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 0x7f4a6c14c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.257+0000 7f4a6a7fc700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a6c152170 0x7f4a6c14c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a6affd700 1 -- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 msgr2=0x7f4a6c14c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:48.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a6affd700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 0x7f4a6c14c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a6affd700 1 -- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a6c152170 msgr2=0x7f4a6c14c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:48.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a6affd700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a6c152170 0x7f4a6c14c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a6affd700 1 -- 172.21.15.33:0/2991964153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a64009a20 con 0x7f4a6c151d90 2023-11-06T20:56:48.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a6affd700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 0x7f4a6c0adc60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f4a6c06c2d0 tx=0x7f4a5c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a52ffd700 1 -- 172.21.15.33:0/2991964153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c006a40 con 0x7f4a6c151d90 2023-11-06T20:56:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a52ffd700 1 -- 172.21.15.33:0/2991964153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5c00f7e0 con 0x7f4a6c151d90 2023-11-06T20:56:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a6bfff700 1 -- 172.21.15.33:0/2991964153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c009370 con 0x7f4a6c151d90 2023-11-06T20:56:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a52ffd700 1 -- 172.21.15.33:0/2991964153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c01b660 con 0x7f4a6c151d90 2023-11-06T20:56:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.258+0000 7f4a6bfff700 1 -- 172.21.15.33:0/2991964153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c0aeb20 con 0x7f4a6c151d90 2023-11-06T20:56:48.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.260+0000 7f4a6bfff700 1 -- 172.21.15.33:0/2991964153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6c060830 con 0x7f4a6c151d90 2023-11-06T20:56:48.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.260+0000 7f4a52ffd700 1 -- 172.21.15.33:0/2991964153 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f4a5c01ba50 con 0x7f4a6c151d90 2023-11-06T20:56:48.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.261+0000 7f4a52ffd700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a5805f2a0 0x7f4a58061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.261+0000 7f4a52ffd700 1 -- 172.21.15.33:0/2991964153 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6891+0+0 (secure 0 0 0) 0x7f4a5c07be30 con 0x7f4a6c151d90 2023-11-06T20:56:48.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.261+0000 7f4a6a7fc700 1 -- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a5805f2a0 msgr2=0x7f4a58061760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:56:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.261+0000 7f4a6a7fc700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a5805f2a0 0x7f4a58061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:56:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.265+0000 7f4a52ffd700 1 -- 172.21.15.33:0/2991964153 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4a5c04cbc0 con 0x7f4a6c151d90 2023-11-06T20:56:48.290 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.290+0000 7fe2df576680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:56:48.344 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:48.344+0000 7ffbd7dbc680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:56:48.377 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.378+0000 7fe2df576680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:56:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.389+0000 7f4a52ffd700 1 -- 172.21.15.33:0/2991964153 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 12) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f4a5c01bd00 con 0x7f4a6c151d90 2023-11-06T20:56:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.389+0000 7f4a52ffd700 1 -- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a5805f2a0 msgr2=0x7f4a58061760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:56:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.389+0000 7f4a52ffd700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a5805f2a0 0x7f4a58061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:48.451 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.451+0000 7fe25704b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:56:48.451 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.451+0000 7fe25704b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:56:48.452 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.451+0000 7fe25704b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:56:48.452 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.451+0000 7fe25704b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:56:48.452 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.451+0000 7fe25704b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:56:48.452 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.451+0000 7fe25a051700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:56:48.452 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.451+0000 7fe25a051700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:56:48.452 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.451+0000 7fe25a051700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:56:48.453 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.451+0000 7fe25a051700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:56:48.453 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:56:48.451+0000 7fe25a051700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:56:48.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.459+0000 7f4a6bfff700 1 -- 172.21.15.33:0/2991964153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a6c0475a0 con 0x7f4a6c151d90 2023-11-06T20:56:48.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.460+0000 7f4a52ffd700 1 -- 172.21.15.33:0/2991964153 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18348 (secure 0 0 0) 0x7f4a5c054bd0 con 0x7f4a6c151d90 2023-11-06T20:56:48.461 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":53,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:56:48.381333+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:45:02.985899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T20:45:03.637532+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":52,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.33","mappings":[{"from":4,"to":5}]},{"pgid":"5.35","mappings":[{"from":4,"to":5}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]},{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.18","mappings":[{"from":4,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/216718810":"2023-11-07T20:56:48.381300+0000","172.21.15.69:0/785816995":"2023-11-07T20:56:48.381300+0000","172.21.15.69:0/634089083":"2023-11-07T20:56:48.381300+0000","172.21.15.69:0/3787072062":"2023-11-07T20:56:48.381300+0000","172.21.15.69:0/1049662608":"2023-11-07T20:45:24.054860+0000","172.21.15.69:0/1201458877":"2023-11-07T20:45:24.054860+0000","172.21.15.69:6833/114912":"2023-11-07T20:45:24.054860+0000","172.21.15.69:6832/114912":"2023-11-07T20:45:24.054860+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":4,"length":2},{"begin":9,"length":1},{"begin":14,"length":2},{"begin":19,"length":1},{"begin":24,"length":2},{"begin":29,"length":1},{"begin":34,"length":2},{"begin":39,"length":1},{"begin":44,"length":2},{"begin":49,"length":1},{"begin":54,"length":2},{"begin":59,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":74,"length":2},{"begin":79,"length":2},{"begin":84,"length":2},{"begin":89,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":183,"length":39}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:56:48.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.463+0000 7f4a6bfff700 1 -- 172.21.15.33:0/2991964153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 msgr2=0x7f4a6c0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:48.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.463+0000 7f4a6bfff700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 0x7f4a6c0adc60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f4a6c06c2d0 tx=0x7f4a5c017000).stop 2023-11-06T20:56:48.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.463+0000 7f4a6bfff700 1 -- 172.21.15.33:0/2991964153 shutdown_connections 2023-11-06T20:56:48.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.463+0000 7f4a6bfff700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a5805f2a0 0x7f4a58061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:48.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.463+0000 7f4a6bfff700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a6c155630 0x7f4a6c14c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.463+0000 7f4a6bfff700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a6c151d90 0x7f4a6c0adc60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.463+0000 7f4a6bfff700 1 --2- 172.21.15.33:0/2991964153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a6c152170 0x7f4a6c14c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.463+0000 7f4a6bfff700 1 -- 172.21.15.33:0/2991964153 >> 172.21.15.33:0/2991964153 conn(0x7f4a6c0cb780 msgr2=0x7f4a6c0c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.463+0000 7f4a6bfff700 1 -- 172.21.15.33:0/2991964153 shutdown_connections 2023-11-06T20:56:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.463+0000 7f4a6bfff700 1 -- 172.21.15.33:0/2991964153 wait complete. 2023-11-06T20:56:48.482 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T20:56:48.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.687+0000 7f4b65fff700 1 Processor -- start 2023-11-06T20:56:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.687+0000 7f4b65fff700 1 -- start start 2023-11-06T20:56:48.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.688+0000 7f4b65fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4b00 0x7f4b600abf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.688+0000 7f4b65fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b600ac460 0x7f4b600c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.688+0000 7f4b65fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b60063290 con 0x7f4b600c4720 2023-11-06T20:56:48.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.688+0000 7f4b65fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b60062710 con 0x7f4b600ac460 2023-11-06T20:56:48.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.688+0000 7f4b65fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b60062960 con 0x7f4b600c4b00 2023-11-06T20:56:48.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.689+0000 7f4b5ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b600ac460 0x7f4b600c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.689+0000 7f4b5f7fe700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f4b600c4720 0x7f4b60063720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:49650/0 (socket says 172.21.15.33:49650) 2023-11-06T20:56:48.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.689+0000 7f4b5f7fe700 1 -- 172.21.15.33:0/3488132365 learned_addr learned my addr 172.21.15.33:0/3488132365 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:48.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.689+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4b00 0x7f4b600abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.689+0000 7f4b5e7fc700 1 -- 172.21.15.33:0/3488132365 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861759038 0 0) 0x7f4b60063290 con 0x7f4b600c4720 2023-11-06T20:56:48.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.689+0000 7f4b5e7fc700 1 -- 172.21.15.33:0/3488132365 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b480033f0 con 0x7f4b600c4720 2023-11-06T20:56:48.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.689+0000 7f4b5effd700 1 -- 172.21.15.33:0/3488132365 >> v2:172.21.15.33:3300/0 conn(0x7f4b600ac460 msgr2=0x7f4b600c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:48.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.689+0000 7f4b5effd700 1 --2- 172.21.15.33:0/3488132365 >> v2:172.21.15.33:3300/0 conn(0x7f4b600ac460 0x7f4b600c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.689+0000 7f4b5effd700 1 -- 172.21.15.33:0/3488132365 >> v1:172.21.15.69:6789/0 conn(0x7f4b600c4720 legacy=0x7f4b60063720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:48.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.689+0000 7f4b5effd700 1 -- 172.21.15.33:0/3488132365 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b60062c40 con 0x7f4b600c4b00 2023-11-06T20:56:48.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.690+0000 7f4b5effd700 1 --2- 172.21.15.33:0/3488132365 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4b00 0x7f4b600abf70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f4b54009760 tx=0x7f4b54009060).ready entity=mon.2 client_cookie=74605612efe65a1c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:48.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.690+0000 7f4b5e7fc700 1 -- 172.21.15.33:0/3488132365 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540090d0 con 0x7f4b600c4b00 2023-11-06T20:56:48.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.690+0000 7f4b5e7fc700 1 -- 172.21.15.33:0/3488132365 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5400d4a0 con 0x7f4b600c4b00 2023-11-06T20:56:48.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.690+0000 7f4b5e7fc700 1 -- 172.21.15.33:0/3488132365 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400d820 con 0x7f4b600c4b00 2023-11-06T20:56:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.690+0000 7f4b65fff700 1 -- 172.21.15.33:0/3488132365 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4b00 msgr2=0x7f4b600abf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.690+0000 7f4b65fff700 1 --2- 172.21.15.33:0/3488132365 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4b00 0x7f4b600abf70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f4b54009760 tx=0x7f4b54009060).stop 2023-11-06T20:56:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.691+0000 7f4b65fff700 1 -- 172.21.15.33:0/3488132365 shutdown_connections 2023-11-06T20:56:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.691+0000 7f4b65fff700 1 --2- 172.21.15.33:0/3488132365 >> v2:172.21.15.33:3300/0 conn(0x7f4b600ac460 0x7f4b600c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.691+0000 7f4b65fff700 1 --2- 172.21.15.33:0/3488132365 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4b00 0x7f4b600abf70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.691+0000 7f4b65fff700 1 -- 172.21.15.33:0/3488132365 >> 172.21.15.33:0/3488132365 conn(0x7f4b600c7dc0 msgr2=0x7f4b600c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:48.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.691+0000 7f4b65fff700 1 -- 172.21.15.33:0/3488132365 shutdown_connections 2023-11-06T20:56:48.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.691+0000 7f4b65fff700 1 -- 172.21.15.33:0/3488132365 wait complete. 2023-11-06T20:56:48.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.692+0000 7f4b65fff700 1 Processor -- start 2023-11-06T20:56:48.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.692+0000 7f4b65fff700 1 -- start start 2023-11-06T20:56:48.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.693+0000 7f4b65fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b600ac460 0x7f4b6013ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.693+0000 7f4b65fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4720 0x7f4b60150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.693+0000 7f4b65fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b600c4b00 0x7f4b60154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:48.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.693+0000 7f4b65fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b60153590 con 0x7f4b600c4b00 2023-11-06T20:56:48.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.693+0000 7f4b65fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601543d0 con 0x7f4b600ac460 2023-11-06T20:56:48.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.693+0000 7f4b65fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b60151a20 con 0x7f4b600c4720 2023-11-06T20:56:48.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.693+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b600c4b00 0x7f4b60154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b600c4b00 0x7f4b60154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47820/0 (socket says 172.21.15.33:47820) 2023-11-06T20:56:48.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5ffff700 1 -- 172.21.15.33:0/436131810 learned_addr learned my addr 172.21.15.33:0/436131810 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:48.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5f7fe700 1 --2- 172.21.15.33:0/436131810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b600ac460 0x7f4b6013ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5effd700 1 --2- 172.21.15.33:0/436131810 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4720 0x7f4b60150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5ffff700 1 -- 172.21.15.33:0/436131810 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4720 msgr2=0x7f4b60150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5ffff700 1 --2- 172.21.15.33:0/436131810 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4720 0x7f4b60150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5ffff700 1 -- 172.21.15.33:0/436131810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b600ac460 msgr2=0x7f4b6013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5ffff700 1 --2- 172.21.15.33:0/436131810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b600ac460 0x7f4b6013ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5ffff700 1 -- 172.21.15.33:0/436131810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b540092a0 con 0x7f4b600c4b00 2023-11-06T20:56:48.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5f7fe700 1 --2- 172.21.15.33:0/436131810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b600ac460 0x7f4b6013ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:48.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b5ffff700 1 --2- 172.21.15.33:0/436131810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b600c4b00 0x7f4b60154860 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f4b5800e0f0 tx=0x7f4b5800d040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:48.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b477fe700 1 -- 172.21.15.33:0/436131810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800b2e0 con 0x7f4b600c4b00 2023-11-06T20:56:48.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b477fe700 1 -- 172.21.15.33:0/436131810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b58011040 con 0x7f4b600c4b00 2023-11-06T20:56:48.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b65fff700 1 -- 172.21.15.33:0/436131810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b601537c0 con 0x7f4b600c4b00 2023-11-06T20:56:48.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.694+0000 7f4b477fe700 1 -- 172.21.15.33:0/436131810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800ece0 con 0x7f4b600c4b00 2023-11-06T20:56:48.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.695+0000 7f4b65fff700 1 -- 172.21.15.33:0/436131810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b60153d80 con 0x7f4b600c4b00 2023-11-06T20:56:48.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.696+0000 7f4b65fff700 1 -- 172.21.15.33:0/436131810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b60064180 con 0x7f4b600c4b00 2023-11-06T20:56:48.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.696+0000 7f4b477fe700 1 -- 172.21.15.33:0/436131810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f4b58009760 con 0x7f4b600c4b00 2023-11-06T20:56:48.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.697+0000 7f4b477fe700 1 -- 172.21.15.33:0/436131810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7019+0+0 (secure 0 0 0) 0x7f4b5807c3e0 con 0x7f4b600c4b00 2023-11-06T20:56:48.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.701+0000 7f4b477fe700 1 -- 172.21.15.33:0/436131810 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4b5804caf0 con 0x7f4b600c4b00 2023-11-06T20:56:48.721 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:48.721+0000 7ffbd7dbc680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:56:48.816 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:56:48.816+0000 7ffbd7dbc680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:56:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:48.930+0000 7f4b65fff700 1 -- 172.21.15.33:0/436131810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4b60063350 con 0x7f4b600c4b00 2023-11-06T20:56:49.383 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T20:56:49.383+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T20:56:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.386+0000 7f4b477fe700 1 -- 172.21.15.33:0/436131810 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4b5804d4a0 con 0x7f4b600c4b00 2023-11-06T20:56:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.389+0000 7f4b65fff700 1 -- 172.21.15.33:0/436131810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b600c4b00 msgr2=0x7f4b60154860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.389+0000 7f4b65fff700 1 --2- 172.21.15.33:0/436131810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b600c4b00 0x7f4b60154860 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f4b5800e0f0 tx=0x7f4b5800d040).stop 2023-11-06T20:56:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.389+0000 7f4b65fff700 1 -- 172.21.15.33:0/436131810 shutdown_connections 2023-11-06T20:56:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.389+0000 7f4b65fff700 1 --2- 172.21.15.33:0/436131810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b600c4b00 0x7f4b60154860 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.389+0000 7f4b65fff700 1 --2- 172.21.15.33:0/436131810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b600ac460 0x7f4b6013ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.389+0000 7f4b65fff700 1 --2- 172.21.15.33:0/436131810 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b600c4720 0x7f4b60150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.389+0000 7f4b65fff700 1 -- 172.21.15.33:0/436131810 >> 172.21.15.33:0/436131810 conn(0x7f4b600c7dc0 msgr2=0x7f4b600c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.389+0000 7f4b65fff700 1 -- 172.21.15.33:0/436131810 shutdown_connections 2023-11-06T20:56:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.389+0000 7f4b65fff700 1 -- 172.21.15.33:0/436131810 wait complete. 2023-11-06T20:56:49.392 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T20:56:49.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T20:56:49.455 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T20:56:49.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.620+0000 7ff2a09ff700 1 Processor -- start 2023-11-06T20:56:49.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.620+0000 7ff2a09ff700 1 -- start start 2023-11-06T20:56:49.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.621+0000 7ff2a09ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 0x7ff29c063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:49.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.621+0000 7ff2a09ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff29c0aa4f0 0x7ff29c063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:49.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.621+0000 7ff2a09ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29c158c00 con 0x7ff29c157700 2023-11-06T20:56:49.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.621+0000 7ff2a09ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29c0c18f0 con 0x7ff29c0aa4f0 2023-11-06T20:56:49.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.621+0000 7ff2a09ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c063b10 con 0x7ff29c0aaf50 2023-11-06T20:56:49.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff29a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff29c157700 0x7ff29c0c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50466/0 (socket says 172.21.15.33:50466) 2023-11-06T20:56:49.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff29a59c700 1 -- 172.21.15.33:0/2043350928 learned_addr learned my addr 172.21.15.33:0/2043350928 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:49.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff299d9b700 1 --2- 172.21.15.33:0/2043350928 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 0x7ff29c063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:49.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff29ad9d700 1 --2- 172.21.15.33:0/2043350928 >> v2:172.21.15.69:3300/0 conn(0x7ff29c0aa4f0 0x7ff29c063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff29959a700 1 -- 172.21.15.33:0/2043350928 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586186681 0 0) 0x7ff29c158c00 con 0x7ff29c157700 2023-11-06T20:56:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff29959a700 1 -- 172.21.15.33:0/2043350928 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2840033f0 con 0x7ff29c157700 2023-11-06T20:56:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff29ad9d700 1 -- 172.21.15.33:0/2043350928 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 msgr2=0x7ff29c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff29ad9d700 1 --2- 172.21.15.33:0/2043350928 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 0x7ff29c063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff29ad9d700 1 -- 172.21.15.33:0/2043350928 >> v1:172.21.15.33:6789/0 conn(0x7ff29c157700 legacy=0x7ff29c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff29ad9d700 1 -- 172.21.15.33:0/2043350928 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c0c7610 con 0x7ff29c0aa4f0 2023-11-06T20:56:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.622+0000 7ff299d9b700 1 --2- 172.21.15.33:0/2043350928 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 0x7ff29c063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff29ad9d700 1 --2- 172.21.15.33:0/2043350928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff29c0aa4f0 0x7ff29c063620 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7ff294009e90 tx=0x7ff294009780).ready entity=mon.1 client_cookie=39f569c556116257 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff29959a700 1 -- 172.21.15.33:0/2043350928 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29400e070 con 0x7ff29c0aa4f0 2023-11-06T20:56:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff29959a700 1 -- 172.21.15.33:0/2043350928 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29400ba70 con 0x7ff29c0aa4f0 2023-11-06T20:56:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff29959a700 1 -- 172.21.15.33:0/2043350928 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29400bdf0 con 0x7ff29c0aa4f0 2023-11-06T20:56:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff2a09ff700 1 -- 172.21.15.33:0/2043350928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff29c0aa4f0 msgr2=0x7ff29c063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff2a09ff700 1 --2- 172.21.15.33:0/2043350928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff29c0aa4f0 0x7ff29c063620 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7ff294009e90 tx=0x7ff294009780).stop 2023-11-06T20:56:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff2a09ff700 1 -- 172.21.15.33:0/2043350928 shutdown_connections 2023-11-06T20:56:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff2a09ff700 1 --2- 172.21.15.33:0/2043350928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff29c0aa4f0 0x7ff29c063620 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff2a09ff700 1 --2- 172.21.15.33:0/2043350928 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 0x7ff29c063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:49.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff2a09ff700 1 -- 172.21.15.33:0/2043350928 >> 172.21.15.33:0/2043350928 conn(0x7ff29c05ec80 msgr2=0x7ff29c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:49.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff2a09ff700 1 -- 172.21.15.33:0/2043350928 shutdown_connections 2023-11-06T20:56:49.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.623+0000 7ff2a09ff700 1 -- 172.21.15.33:0/2043350928 wait complete. 2023-11-06T20:56:49.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.625+0000 7ff2a09ff700 1 Processor -- start 2023-11-06T20:56:49.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.625+0000 7ff2a09ff700 1 -- start start 2023-11-06T20:56:49.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.626+0000 7ff2a09ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff29c0aa4f0 0x7ff29c0b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:49.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.626+0000 7ff2a09ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 0x7ff29c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:49.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.626+0000 7ff2a09ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c157700 0x7ff29c0ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:49.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.626+0000 7ff2a09ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0b0ae0 con 0x7ff29c157700 2023-11-06T20:56:49.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.626+0000 7ff2a09ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0b0fa0 con 0x7ff29c0aa4f0 2023-11-06T20:56:49.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.626+0000 7ff2a09ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c1504a0 con 0x7ff29c0aaf50 2023-11-06T20:56:49.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.626+0000 7ff29ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c157700 0x7ff29c0ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:49.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.626+0000 7ff29ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c157700 0x7ff29c0ac320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47848/0 (socket says 172.21.15.33:47848) 2023-11-06T20:56:49.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.626+0000 7ff29ad9d700 1 -- 172.21.15.33:0/1713154014 learned_addr learned my addr 172.21.15.33:0/1713154014 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:49.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff299d9b700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 0x7ff29c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff29a59c700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff29c0aa4f0 0x7ff29c0b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff29ad9d700 1 -- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 msgr2=0x7ff29c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff29ad9d700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 0x7ff29c0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff29ad9d700 1 -- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff29c0aa4f0 msgr2=0x7ff29c0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff29ad9d700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff29c0aa4f0 0x7ff29c0b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff29ad9d700 1 -- 172.21.15.33:0/1713154014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2900092a0 con 0x7ff29c157700 2023-11-06T20:56:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff299d9b700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 0x7ff29c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff29a59c700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff29c0aa4f0 0x7ff29c0b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff29ad9d700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c157700 0x7ff29c0ac320 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7ff29c05f0e0 tx=0x7ff294009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff28a7fc700 1 -- 172.21.15.33:0/1713154014 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29401d630 con 0x7ff29c157700 2023-11-06T20:56:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff28a7fc700 1 -- 172.21.15.33:0/1713154014 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29401d790 con 0x7ff29c157700 2023-11-06T20:56:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff28a7fc700 1 -- 172.21.15.33:0/1713154014 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29401db50 con 0x7ff29c157700 2023-11-06T20:56:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff2a09ff700 1 -- 172.21.15.33:0/1713154014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff294009a20 con 0x7ff29c157700 2023-11-06T20:56:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.627+0000 7ff2a09ff700 1 -- 172.21.15.33:0/1713154014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29c0c2090 con 0x7ff29c157700 2023-11-06T20:56:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.629+0000 7ff2a09ff700 1 -- 172.21.15.33:0/1713154014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29c0c29c0 con 0x7ff29c157700 2023-11-06T20:56:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.629+0000 7ff28a7fc700 1 -- 172.21.15.33:0/1713154014 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 75340+0+0 (secure 0 0 0) 0x7ff294003040 con 0x7ff29c157700 2023-11-06T20:56:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.630+0000 7ff28a7fc700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27c05f030 0x7ff27c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.630+0000 7ff28a7fc700 1 -- 172.21.15.33:0/1713154014 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7105+0+0 (secure 0 0 0) 0x7ff29407c220 con 0x7ff29c157700 2023-11-06T20:56:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.630+0000 7ff29a59c700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27c05f030 0x7ff27c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.631+0000 7ff29a59c700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27c05f030 0x7ff27c0614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7ff28c007fd0 tx=0x7ff28c009040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.635+0000 7ff28a7fc700 1 -- 172.21.15.33:0/1713154014 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff29404d2e0 con 0x7ff29c157700 2023-11-06T20:56:49.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:49.862+0000 7ff2a09ff700 1 -- 172.21.15.33:0/1713154014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff29c0c1ef0 con 0x7ff29c157700 2023-11-06T20:56:50.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.400+0000 7ff28a7fc700 1 -- 172.21.15.33:0/1713154014 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff29c0c1ef0 con 0x7ff29c157700 2023-11-06T20:56:50.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.402+0000 7ff2a09ff700 1 -- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27c05f030 msgr2=0x7ff27c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:50.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.402+0000 7ff2a09ff700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27c05f030 0x7ff27c0614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7ff28c007fd0 tx=0x7ff28c009040).stop 2023-11-06T20:56:50.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.402+0000 7ff2a09ff700 1 -- 172.21.15.33:0/1713154014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c157700 msgr2=0x7ff29c0ac320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:50.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.402+0000 7ff2a09ff700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c157700 0x7ff29c0ac320 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7ff29c05f0e0 tx=0x7ff294009780).stop 2023-11-06T20:56:50.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.402+0000 7ff2a09ff700 1 -- 172.21.15.33:0/1713154014 shutdown_connections 2023-11-06T20:56:50.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.402+0000 7ff2a09ff700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff27c05f030 0x7ff27c0614f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.402+0000 7ff2a09ff700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c157700 0x7ff29c0ac320 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.402+0000 7ff2a09ff700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff29c0aa4f0 0x7ff29c0b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.402+0000 7ff2a09ff700 1 --2- 172.21.15.33:0/1713154014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff29c0aaf50 0x7ff29c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.402+0000 7ff2a09ff700 1 -- 172.21.15.33:0/1713154014 >> 172.21.15.33:0/1713154014 conn(0x7ff29c05ec80 msgr2=0x7ff29c0c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:50.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.403+0000 7ff2a09ff700 1 -- 172.21.15.33:0/1713154014 shutdown_connections 2023-11-06T20:56:50.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.403+0000 7ff2a09ff700 1 -- 172.21.15.33:0/1713154014 wait complete. 2023-11-06T20:56:50.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T20:56:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.664+0000 7fc1e67ff700 1 Processor -- start 2023-11-06T20:56:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.664+0000 7fc1e67ff700 1 -- start start 2023-11-06T20:56:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.666+0000 7fc1e67ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc1e0153ed0 0x7fc1e0158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.666+0000 7fc1e67ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e00c8ee0 0x7fc1e00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.666+0000 7fc1e67ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1e01542b0 0x7fc1e00c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.666+0000 7fc1e67ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e00c9f80 con 0x7fc1e01542b0 2023-11-06T20:56:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.666+0000 7fc1e67ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e00c6000 con 0x7fc1e0153ed0 2023-11-06T20:56:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.666+0000 7fc1e67ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e00c9370 con 0x7fc1e00c8ee0 2023-11-06T20:56:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.666+0000 7fc1e4d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1e01542b0 0x7fc1e00c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.666+0000 7fc1e4d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1e01542b0 0x7fc1e00c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47862/0 (socket says 172.21.15.33:47862) 2023-11-06T20:56:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.666+0000 7fc1e4d9c700 1 -- 172.21.15.33:0/3699218167 learned_addr learned my addr 172.21.15.33:0/3699218167 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1df7fe700 1 --2- 172.21.15.33:0/3699218167 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e00c8ee0 0x7fc1e00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1dffff700 1 --2- 172.21.15.33:0/3699218167 >> v2:172.21.15.69:3300/0 conn(0x7fc1e0153ed0 0x7fc1e0158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1e4d9c700 1 -- 172.21.15.33:0/3699218167 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e00c8ee0 msgr2=0x7fc1e00ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1e4d9c700 1 --2- 172.21.15.33:0/3699218167 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e00c8ee0 0x7fc1e00ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1e4d9c700 1 -- 172.21.15.33:0/3699218167 >> v2:172.21.15.69:3300/0 conn(0x7fc1e0153ed0 msgr2=0x7fc1e0158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1e4d9c700 1 --2- 172.21.15.33:0/3699218167 >> v2:172.21.15.69:3300/0 conn(0x7fc1e0153ed0 0x7fc1e0158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1e4d9c700 1 -- 172.21.15.33:0/3699218167 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e00c9650 con 0x7fc1e01542b0 2023-11-06T20:56:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1dffff700 1 --2- 172.21.15.33:0/3699218167 >> v2:172.21.15.69:3300/0 conn(0x7fc1e0153ed0 0x7fc1e0158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1e4d9c700 1 --2- 172.21.15.33:0/3699218167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e01542b0 0x7fc1e00c76f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fc1e006bdb0 tx=0x7fc1d8009840).ready entity=mon.0 client_cookie=15b498f5b07a7107 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1deffd700 1 -- 172.21.15.33:0/3699218167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d800e070 con 0x7fc1e01542b0 2023-11-06T20:56:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1deffd700 1 -- 172.21.15.33:0/3699218167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d800ba20 con 0x7fc1e01542b0 2023-11-06T20:56:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.667+0000 7fc1deffd700 1 -- 172.21.15.33:0/3699218167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d800bda0 con 0x7fc1e01542b0 2023-11-06T20:56:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.668+0000 7fc1e67ff700 1 -- 172.21.15.33:0/3699218167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e01542b0 msgr2=0x7fc1e00c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.668+0000 7fc1e67ff700 1 --2- 172.21.15.33:0/3699218167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e01542b0 0x7fc1e00c76f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fc1e006bdb0 tx=0x7fc1d8009840).stop 2023-11-06T20:56:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.668+0000 7fc1e67ff700 1 -- 172.21.15.33:0/3699218167 shutdown_connections 2023-11-06T20:56:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.668+0000 7fc1e67ff700 1 --2- 172.21.15.33:0/3699218167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e01542b0 0x7fc1e00c76f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.668+0000 7fc1e67ff700 1 --2- 172.21.15.33:0/3699218167 >> v2:172.21.15.69:3300/0 conn(0x7fc1e0153ed0 0x7fc1e0158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.668+0000 7fc1e67ff700 1 --2- 172.21.15.33:0/3699218167 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e00c8ee0 0x7fc1e00ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.668+0000 7fc1e67ff700 1 -- 172.21.15.33:0/3699218167 >> 172.21.15.33:0/3699218167 conn(0x7fc1e005e1b0 msgr2=0x7fc1e005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:50.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.668+0000 7fc1e67ff700 1 -- 172.21.15.33:0/3699218167 shutdown_connections 2023-11-06T20:56:50.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.668+0000 7fc1e67ff700 1 -- 172.21.15.33:0/3699218167 wait complete. 2023-11-06T20:56:50.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.670+0000 7fc1e67ff700 1 Processor -- start 2023-11-06T20:56:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.670+0000 7fc1e67ff700 1 -- start start 2023-11-06T20:56:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1e67ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0153ed0 0x7fc1e01685d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1e67ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e01542b0 0x7fc1e0157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1e67ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e00adf90 0x7fc1e00ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1e67ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e0158000 con 0x7fc1e0153ed0 2023-11-06T20:56:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1e67ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e01582c0 con 0x7fc1e01542b0 2023-11-06T20:56:50.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1e67ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e01585a0 con 0x7fc1e00adf90 2023-11-06T20:56:50.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1dffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0153ed0 0x7fc1e01685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:50.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1dffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0153ed0 0x7fc1e01685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47874/0 (socket says 172.21.15.33:47874) 2023-11-06T20:56:50.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1dffff700 1 -- 172.21.15.33:0/1196274751 learned_addr learned my addr 172.21.15.33:0/1196274751 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:50.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1e4d9c700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e00adf90 0x7fc1e00ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1df7fe700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e01542b0 0x7fc1e0157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1dffff700 1 -- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e00adf90 msgr2=0x7fc1e00ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1dffff700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e00adf90 0x7fc1e00ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1dffff700 1 -- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e01542b0 msgr2=0x7fc1e0157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1dffff700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e01542b0 0x7fc1e0157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.671+0000 7fc1dffff700 1 -- 172.21.15.33:0/1196274751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d8009ae0 con 0x7fc1e0153ed0 2023-11-06T20:56:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.672+0000 7fc1dffff700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0153ed0 0x7fc1e01685d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fc1d000d860 tx=0x7fc1d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.672+0000 7fc1c7fff700 1 -- 172.21.15.33:0/1196274751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0011590 con 0x7fc1e0153ed0 2023-11-06T20:56:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.672+0000 7fc1c7fff700 1 -- 172.21.15.33:0/1196274751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d00116f0 con 0x7fc1e0153ed0 2023-11-06T20:56:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.672+0000 7fc1c7fff700 1 -- 172.21.15.33:0/1196274751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0011a70 con 0x7fc1e0153ed0 2023-11-06T20:56:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.672+0000 7fc1e67ff700 1 -- 172.21.15.33:0/1196274751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e0158880 con 0x7fc1e0153ed0 2023-11-06T20:56:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.672+0000 7fc1e67ff700 1 -- 172.21.15.33:0/1196274751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e0157d00 con 0x7fc1e0153ed0 2023-11-06T20:56:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.673+0000 7fc1e67ff700 1 -- 172.21.15.33:0/1196274751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e006c6c0 con 0x7fc1e0153ed0 2023-11-06T20:56:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.674+0000 7fc1c7fff700 1 -- 172.21.15.33:0/1196274751 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fc1d0012420 con 0x7fc1e0153ed0 2023-11-06T20:56:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.674+0000 7fc1c7fff700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1cc05f030 0x7fc1cc0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.675+0000 7fc1c7fff700 1 -- 172.21.15.33:0/1196274751 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7105+0+0 (secure 0 0 0) 0x7fc1d0079ff0 con 0x7fc1e0153ed0 2023-11-06T20:56:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.675+0000 7fc1df7fe700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1cc05f030 0x7fc1cc0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:50.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.675+0000 7fc1df7fe700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1cc05f030 0x7fc1cc0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc1e00c18f0 tx=0x7fc1d4008040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:50.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.678+0000 7fc1c7fff700 1 -- 172.21.15.33:0/1196274751 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc1d004b020 con 0x7fc1e0153ed0 2023-11-06T20:56:50.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:50.860+0000 7fc1e67ff700 1 -- 172.21.15.33:0/1196274751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc1e00c90e0 con 0x7fc1e0153ed0 2023-11-06T20:56:51.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.147+0000 7fc1c7fff700 1 -- 172.21.15.33:0/1196274751 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc1d0077b20 con 0x7fc1e0153ed0 2023-11-06T20:56:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.409+0000 7fc1c7fff700 1 -- 172.21.15.33:0/1196274751 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc1d0072f60 con 0x7fc1e0153ed0 2023-11-06T20:56:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.412+0000 7fc1e67ff700 1 -- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1cc05f030 msgr2=0x7fc1cc0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.413+0000 7fc1e67ff700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1cc05f030 0x7fc1cc0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc1e00c18f0 tx=0x7fc1d4008040).stop 2023-11-06T20:56:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.413+0000 7fc1e67ff700 1 -- 172.21.15.33:0/1196274751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0153ed0 msgr2=0x7fc1e01685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.413+0000 7fc1e67ff700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0153ed0 0x7fc1e01685d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fc1d000d860 tx=0x7fc1d0008040).stop 2023-11-06T20:56:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.414+0000 7fc1e67ff700 1 -- 172.21.15.33:0/1196274751 shutdown_connections 2023-11-06T20:56:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.414+0000 7fc1e67ff700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc1cc05f030 0x7fc1cc0614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.414+0000 7fc1e67ff700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1e00adf90 0x7fc1e00ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.414+0000 7fc1e67ff700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1e0153ed0 0x7fc1e01685d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.414+0000 7fc1e67ff700 1 --2- 172.21.15.33:0/1196274751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1e01542b0 0x7fc1e0157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.414+0000 7fc1e67ff700 1 -- 172.21.15.33:0/1196274751 >> 172.21.15.33:0/1196274751 conn(0x7fc1e005e1b0 msgr2=0x7fc1e00ab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:51.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.415+0000 7fc1e67ff700 1 -- 172.21.15.33:0/1196274751 shutdown_connections 2023-11-06T20:56:51.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.416+0000 7fc1e67ff700 1 -- 172.21.15.33:0/1196274751 wait complete. 2023-11-06T20:56:51.420 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T20:56:51.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T20:56:51.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.641+0000 7f3a659ff700 1 Processor -- start 2023-11-06T20:56:51.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.642+0000 7f3a659ff700 1 -- start start 2023-11-06T20:56:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.643+0000 7f3a659ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3a600c8ee0 0x7f3a600ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.643+0000 7f3a659ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 0x7f3a600c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.643+0000 7f3a659ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a600c9f80 con 0x7f3a60153ed0 2023-11-06T20:56:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.643+0000 7f3a659ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a600c6000 con 0x7f3a600c8ee0 2023-11-06T20:56:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.643+0000 7f3a659ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600c9370 con 0x7f3a601542b0 2023-11-06T20:56:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.643+0000 7f3a5e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3a600c8ee0 0x7f3a600ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.643+0000 7f3a5e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3a600c8ee0 0x7f3a600ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47886/0 (socket says 172.21.15.33:47886) 2023-11-06T20:56:51.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.643+0000 7f3a5e7fc700 1 -- 172.21.15.33:0/4206517485 learned_addr learned my addr 172.21.15.33:0/4206517485 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:51.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.643+0000 7f3a5effd700 1 --1- 172.21.15.33:0/4206517485 >> v1:172.21.15.33:6789/0 conn(0x7f3a60153ed0 0x7f3a60158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50470/0 (socket says 172.21.15.33:50470) 2023-11-06T20:56:51.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5e7fc700 1 -- 172.21.15.33:0/4206517485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 msgr2=0x7f3a600c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:51.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5f7fe700 1 --2- 172.21.15.33:0/4206517485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 0x7f3a600c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:51.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5dffb700 1 -- 172.21.15.33:0/4206517485 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141849035 0 0) 0x7f3a600c9f80 con 0x7f3a60153ed0 2023-11-06T20:56:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5e7fc700 1 --2- 172.21.15.33:0/4206517485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 0x7f3a600c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5e7fc700 1 -- 172.21.15.33:0/4206517485 >> v1:172.21.15.33:6789/0 conn(0x7f3a60153ed0 legacy=0x7f3a60158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5e7fc700 1 -- 172.21.15.33:0/4206517485 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a600c9650 con 0x7f3a600c8ee0 2023-11-06T20:56:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5f7fe700 1 --2- 172.21.15.33:0/4206517485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 0x7f3a600c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5e7fc700 1 --2- 172.21.15.33:0/4206517485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a600c8ee0 0x7f3a600ca410 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3a54006fd0 tx=0x7f3a540090f0).ready entity=mon.0 client_cookie=7fc29fbd0601f093 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5dffb700 1 -- 172.21.15.33:0/4206517485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400b600 con 0x7f3a600c8ee0 2023-11-06T20:56:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5dffb700 1 -- 172.21.15.33:0/4206517485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5400b760 con 0x7f3a600c8ee0 2023-11-06T20:56:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.644+0000 7f3a5dffb700 1 -- 172.21.15.33:0/4206517485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400bae0 con 0x7f3a600c8ee0 2023-11-06T20:56:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.645+0000 7f3a659ff700 1 -- 172.21.15.33:0/4206517485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a600c8ee0 msgr2=0x7f3a600ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.645+0000 7f3a659ff700 1 --2- 172.21.15.33:0/4206517485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a600c8ee0 0x7f3a600ca410 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3a54006fd0 tx=0x7f3a540090f0).stop 2023-11-06T20:56:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.645+0000 7f3a659ff700 1 -- 172.21.15.33:0/4206517485 shutdown_connections 2023-11-06T20:56:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.645+0000 7f3a659ff700 1 --2- 172.21.15.33:0/4206517485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 0x7f3a600c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.645+0000 7f3a659ff700 1 --2- 172.21.15.33:0/4206517485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a600c8ee0 0x7f3a600ca410 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.645+0000 7f3a659ff700 1 -- 172.21.15.33:0/4206517485 >> 172.21.15.33:0/4206517485 conn(0x7f3a6005e1b0 msgr2=0x7f3a6005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.645+0000 7f3a659ff700 1 -- 172.21.15.33:0/4206517485 shutdown_connections 2023-11-06T20:56:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.645+0000 7f3a659ff700 1 -- 172.21.15.33:0/4206517485 wait complete. 2023-11-06T20:56:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.646+0000 7f3a659ff700 1 Processor -- start 2023-11-06T20:56:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.646+0000 7f3a659ff700 1 -- start start 2023-11-06T20:56:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.647+0000 7f3a659ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a60153ed0 0x7f3a600ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.647+0000 7f3a659ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 0x7f3a600acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.647+0000 7f3a659ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a600b0ac0 0x7f3a60154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.647+0000 7f3a659ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600ad2b0 con 0x7f3a60153ed0 2023-11-06T20:56:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.647+0000 7f3a659ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600adee0 con 0x7f3a600b0ac0 2023-11-06T20:56:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.647+0000 7f3a659ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a601554c0 con 0x7f3a601542b0 2023-11-06T20:56:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a60153ed0 0x7f3a600ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a60153ed0 0x7f3a600ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47892/0 (socket says 172.21.15.33:47892) 2023-11-06T20:56:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5effd700 1 -- 172.21.15.33:0/3280546009 learned_addr learned my addr 172.21.15.33:0/3280546009 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5e7fc700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 0x7f3a600acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5f7fe700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a600b0ac0 0x7f3a60154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5effd700 1 -- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 msgr2=0x7f3a600acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5effd700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 0x7f3a600acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5effd700 1 -- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a600b0ac0 msgr2=0x7f3a60154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5effd700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a600b0ac0 0x7f3a60154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5effd700 1 -- 172.21.15.33:0/3280546009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54009370 con 0x7f3a60153ed0 2023-11-06T20:56:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5e7fc700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 0x7f3a600acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5f7fe700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a600b0ac0 0x7f3a60154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.648+0000 7f3a5effd700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a60153ed0 0x7f3a600ac8d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f3a5000c560 tx=0x7f3a50007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.649+0000 7f3a46ffd700 1 -- 172.21.15.33:0/3280546009 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500072e0 con 0x7f3a60153ed0 2023-11-06T20:56:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.649+0000 7f3a46ffd700 1 -- 172.21.15.33:0/3280546009 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50007440 con 0x7f3a60153ed0 2023-11-06T20:56:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.649+0000 7f3a46ffd700 1 -- 172.21.15.33:0/3280546009 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500058f0 con 0x7f3a60153ed0 2023-11-06T20:56:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.649+0000 7f3a659ff700 1 -- 172.21.15.33:0/3280546009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a601557a0 con 0x7f3a60153ed0 2023-11-06T20:56:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.649+0000 7f3a659ff700 1 -- 172.21.15.33:0/3280546009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a60155d60 con 0x7f3a60153ed0 2023-11-06T20:56:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.650+0000 7f3a659ff700 1 -- 172.21.15.33:0/3280546009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6006c6c0 con 0x7f3a60153ed0 2023-11-06T20:56:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.650+0000 7f3a46ffd700 1 -- 172.21.15.33:0/3280546009 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3a50005ce0 con 0x7f3a60153ed0 2023-11-06T20:56:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.651+0000 7f3a46ffd700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4c05f270 0x7f3a4c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.651+0000 7f3a46ffd700 1 -- 172.21.15.33:0/3280546009 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6551+0+0 (secure 0 0 0) 0x7f3a50079f00 con 0x7f3a60153ed0 2023-11-06T20:56:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.652+0000 7f3a5e7fc700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4c05f270 0x7f3a4c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.652+0000 7f3a5e7fc700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4c05f270 0x7f3a4c061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3a6015b0e0 tx=0x7f3a54017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:51.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.655+0000 7f3a46ffd700 1 -- 172.21.15.33:0/3280546009 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3a5004ade0 con 0x7f3a60153ed0 2023-11-06T20:56:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.836+0000 7f3a659ff700 1 -- 172.21.15.33:0/3280546009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3a601563a0 con 0x7f3a60153ed0 2023-11-06T20:56:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.940+0000 7f3a46ffd700 1 -- 172.21.15.33:0/3280546009 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3a601563a0 con 0x7f3a60153ed0 2023-11-06T20:56:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.943+0000 7f3a659ff700 1 -- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4c05f270 msgr2=0x7f3a4c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.943+0000 7f3a659ff700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4c05f270 0x7f3a4c061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3a6015b0e0 tx=0x7f3a54017000).stop 2023-11-06T20:56:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.943+0000 7f3a659ff700 1 -- 172.21.15.33:0/3280546009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a60153ed0 msgr2=0x7f3a600ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.943+0000 7f3a659ff700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a60153ed0 0x7f3a600ac8d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f3a5000c560 tx=0x7f3a50007c90).stop 2023-11-06T20:56:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.943+0000 7f3a659ff700 1 -- 172.21.15.33:0/3280546009 shutdown_connections 2023-11-06T20:56:51.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.943+0000 7f3a659ff700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4c05f270 0x7f3a4c061730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.943+0000 7f3a659ff700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a600b0ac0 0x7f3a60154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.943+0000 7f3a659ff700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a60153ed0 0x7f3a600ac8d0 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0x7f3a5000c560 tx=0x7f3a50007c90).stop 2023-11-06T20:56:51.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.943+0000 7f3a659ff700 1 --2- 172.21.15.33:0/3280546009 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a601542b0 0x7f3a600acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:51.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.943+0000 7f3a659ff700 1 -- 172.21.15.33:0/3280546009 >> 172.21.15.33:0/3280546009 conn(0x7f3a6005e1b0 msgr2=0x7f3a600cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:51.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.944+0000 7f3a659ff700 1 -- 172.21.15.33:0/3280546009 shutdown_connections 2023-11-06T20:56:51.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:51.944+0000 7f3a659ff700 1 -- 172.21.15.33:0/3280546009 wait complete. 2023-11-06T20:56:51.946 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T20:56:51.965 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T20:56:52.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.178+0000 7fe5ddbff700 1 Processor -- start 2023-11-06T20:56:52.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.178+0000 7fe5ddbff700 1 -- start start 2023-11-06T20:56:52.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.180+0000 7fe5ddbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aaf50 0x7fe5d8063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.180+0000 7fe5ddbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe5d80aa4f0 0x7fe5d8063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.180+0000 7fe5ddbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d8158c00 con 0x7fe5d8157700 2023-11-06T20:56:52.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.180+0000 7fe5ddbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d80c18f0 con 0x7fe5d80aa4f0 2023-11-06T20:56:52.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.180+0000 7fe5ddbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d8063b10 con 0x7fe5d80aaf50 2023-11-06T20:56:52.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.180+0000 7fe5d6ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aaf50 0x7fe5d8063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.180+0000 7fe5d7fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe5d80aa4f0 0x7fe5d8063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.180+0000 7fe5d7fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe5d80aa4f0 0x7fe5d8063620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:36794/0 (socket says 172.21.15.33:36794) 2023-11-06T20:56:52.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.180+0000 7fe5d7fff700 1 -- 172.21.15.33:0/2364446108 learned_addr learned my addr 172.21.15.33:0/2364446108 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:52.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.180+0000 7fe5d77fe700 1 --1- 172.21.15.33:0/2364446108 >> v1:172.21.15.69:6789/0 conn(0x7fe5d8157700 0x7fe5d80c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:49652/0 (socket says 172.21.15.33:49652) 2023-11-06T20:56:52.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.181+0000 7fe5d7fff700 1 -- 172.21.15.33:0/2364446108 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aaf50 msgr2=0x7fe5d8063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:52.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.181+0000 7fe5d7fff700 1 --2- 172.21.15.33:0/2364446108 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aaf50 0x7fe5d8063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.181+0000 7fe5d67fc700 1 -- 172.21.15.33:0/2364446108 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449527815 0 0) 0x7fe5d8158c00 con 0x7fe5d8157700 2023-11-06T20:56:52.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.181+0000 7fe5d7fff700 1 -- 172.21.15.33:0/2364446108 >> v1:172.21.15.69:6789/0 conn(0x7fe5d8157700 legacy=0x7fe5d80c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.181+0000 7fe5d7fff700 1 -- 172.21.15.33:0/2364446108 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d80c7610 con 0x7fe5d80aa4f0 2023-11-06T20:56:52.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.181+0000 7fe5d6ffd700 1 --2- 172.21.15.33:0/2364446108 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aaf50 0x7fe5d8063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:52.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.181+0000 7fe5d7fff700 1 --2- 172.21.15.33:0/2364446108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d80aa4f0 0x7fe5d8063620 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fe5d0009f50 tx=0x7fe5d0009840).ready entity=mon.1 client_cookie=6af91ccb65c02b4c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:52.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.181+0000 7fe5d67fc700 1 -- 172.21.15.33:0/2364446108 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000e070 con 0x7fe5d80aa4f0 2023-11-06T20:56:52.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.181+0000 7fe5d67fc700 1 -- 172.21.15.33:0/2364446108 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d000b9c0 con 0x7fe5d80aa4f0 2023-11-06T20:56:52.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.181+0000 7fe5d67fc700 1 -- 172.21.15.33:0/2364446108 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000bd40 con 0x7fe5d80aa4f0 2023-11-06T20:56:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.182+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2364446108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d80aa4f0 msgr2=0x7fe5d8063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.182+0000 7fe5ddbff700 1 --2- 172.21.15.33:0/2364446108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d80aa4f0 0x7fe5d8063620 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fe5d0009f50 tx=0x7fe5d0009840).stop 2023-11-06T20:56:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.182+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2364446108 shutdown_connections 2023-11-06T20:56:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.182+0000 7fe5ddbff700 1 --2- 172.21.15.33:0/2364446108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d80aa4f0 0x7fe5d8063620 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.182+0000 7fe5ddbff700 1 --2- 172.21.15.33:0/2364446108 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aaf50 0x7fe5d8063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.182+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2364446108 >> 172.21.15.33:0/2364446108 conn(0x7fe5d805ec80 msgr2=0x7fe5d80cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:52.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.182+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2364446108 shutdown_connections 2023-11-06T20:56:52.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.182+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2364446108 wait complete. 2023-11-06T20:56:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.183+0000 7fe5ddbff700 1 Processor -- start 2023-11-06T20:56:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.184+0000 7fe5ddbff700 1 -- start start 2023-11-06T20:56:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5ddbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aa4f0 0x7fe5d80ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5ddbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d80aaf50 0x7fe5d813f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5ddbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5d8157700 0x7fe5d80c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5ddbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80ae4b0 con 0x7fe5d8157700 2023-11-06T20:56:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5ddbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80adea0 con 0x7fe5d80aaf50 2023-11-06T20:56:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5ddbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80c4550 con 0x7fe5d80aa4f0 2023-11-06T20:56:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d77fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aa4f0 0x7fe5d80ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5d8157700 0x7fe5d80c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d80aaf50 0x7fe5d813f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d77fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aa4f0 0x7fe5d80ae940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:46938/0 (socket says 172.21.15.33:46938) 2023-11-06T20:56:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d77fe700 1 -- 172.21.15.33:0/2059211500 learned_addr learned my addr 172.21.15.33:0/2059211500 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d7fff700 1 -- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aa4f0 msgr2=0x7fe5d80ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d7fff700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aa4f0 0x7fe5d80ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d7fff700 1 -- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d80aaf50 msgr2=0x7fe5d813f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d7fff700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d80aaf50 0x7fe5d813f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d7fff700 1 -- 172.21.15.33:0/2059211500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc0092a0 con 0x7fe5d8157700 2023-11-06T20:56:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d77fe700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aa4f0 0x7fe5d80ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.185+0000 7fe5d6ffd700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d80aaf50 0x7fe5d813f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.186+0000 7fe5d7fff700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5d8157700 0x7fe5d80c3b90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fe5d0013000 tx=0x7fe5d0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.186+0000 7fe5bf7fe700 1 -- 172.21.15.33:0/2059211500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000bd40 con 0x7fe5d8157700 2023-11-06T20:56:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.186+0000 7fe5bf7fe700 1 -- 172.21.15.33:0/2059211500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d00092e0 con 0x7fe5d8157700 2023-11-06T20:56:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.186+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2059211500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d0009ae0 con 0x7fe5d8157700 2023-11-06T20:56:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.186+0000 7fe5bf7fe700 1 -- 172.21.15.33:0/2059211500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d001d6b0 con 0x7fe5d8157700 2023-11-06T20:56:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.186+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2059211500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d80c4830 con 0x7fe5d8157700 2023-11-06T20:56:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.187+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2059211500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d80c0e30 con 0x7fe5d8157700 2023-11-06T20:56:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.188+0000 7fe5bf7fe700 1 -- 172.21.15.33:0/2059211500 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe5d0003040 con 0x7fe5d8157700 2023-11-06T20:56:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.189+0000 7fe5bf7fe700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe5c405f270 0x7fe5c4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.189+0000 7fe5bf7fe700 1 -- 172.21.15.33:0/2059211500 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fe5d007b1d0 con 0x7fe5d8157700 2023-11-06T20:56:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.189+0000 7fe5d77fe700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe5c405f270 0x7fe5c4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.190+0000 7fe5d77fe700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe5c405f270 0x7fe5c4061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fe5c8007fd0 tx=0x7fe5c8007c90).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:52.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.193+0000 7fe5bf7fe700 1 -- 172.21.15.33:0/2059211500 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe5d004c340 con 0x7fe5d8157700 2023-11-06T20:56:52.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.375+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2059211500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fe5d806c6c0 con 0x7fe5d8157700 2023-11-06T20:56:52.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.379+0000 7fe5bf7fe700 1 -- 172.21.15.33:0/2059211500 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v4) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe5d00220b0 con 0x7fe5d8157700 2023-11-06T20:56:52.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.381+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe5c405f270 msgr2=0x7fe5c4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.381+0000 7fe5ddbff700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe5c405f270 0x7fe5c4061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fe5c8007fd0 tx=0x7fe5c8007c90).stop 2023-11-06T20:56:52.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.381+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2059211500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5d8157700 msgr2=0x7fe5d80c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.381+0000 7fe5ddbff700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5d8157700 0x7fe5d80c3b90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fe5d0013000 tx=0x7fe5d0009840).stop 2023-11-06T20:56:52.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.381+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2059211500 shutdown_connections 2023-11-06T20:56:52.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.381+0000 7fe5ddbff700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe5c405f270 0x7fe5c4061730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.381+0000 7fe5ddbff700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5d80aa4f0 0x7fe5d80ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.381+0000 7fe5ddbff700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5d8157700 0x7fe5d80c3b90 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.381+0000 7fe5ddbff700 1 --2- 172.21.15.33:0/2059211500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d80aaf50 0x7fe5d813f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.381+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2059211500 >> 172.21.15.33:0/2059211500 conn(0x7fe5d805ec80 msgr2=0x7fe5d80ad360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:52.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.382+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2059211500 shutdown_connections 2023-11-06T20:56:52.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.382+0000 7fe5ddbff700 1 -- 172.21.15.33:0/2059211500 wait complete. 2023-11-06T20:56:52.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T20:56:52.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.598+0000 7f5a1c9ff700 1 Processor -- start 2023-11-06T20:56:52.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.599+0000 7f5a1c9ff700 1 -- start start 2023-11-06T20:56:52.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.600+0000 7f5a1c9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 0x7f5a180c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.600+0000 7f5a1c9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5a18063840 0x7f5a180c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.600+0000 7f5a1c9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a18158c00 con 0x7f5a180aaf50 2023-11-06T20:56:52.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.600+0000 7f5a1c9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a180c18f0 con 0x7f5a18063840 2023-11-06T20:56:52.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.600+0000 7f5a1c9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a18157ff0 con 0x7f5a18157700 2023-11-06T20:56:52.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.600+0000 7f5a1659c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 0x7f5a180c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.600+0000 7f5a16d9d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5a18063840 0x7f5a180c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.600+0000 7f5a1659c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 0x7f5a180c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:46944/0 (socket says 172.21.15.33:46944) 2023-11-06T20:56:52.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.600+0000 7f5a1659c700 1 -- 172.21.15.33:0/3090594881 learned_addr learned my addr 172.21.15.33:0/3090594881 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:52.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.600+0000 7f5a15d9b700 1 --1- 172.21.15.33:0/3090594881 >> v1:172.21.15.69:6789/0 conn(0x7f5a180aaf50 0x7f5a18063130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:49654/0 (socket says 172.21.15.33:49654) 2023-11-06T20:56:52.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.601+0000 7f5a1659c700 1 -- 172.21.15.33:0/3090594881 >> v2:172.21.15.69:3300/0 conn(0x7f5a18063840 msgr2=0x7f5a180c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:52.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.601+0000 7f5a1659c700 1 --2- 172.21.15.33:0/3090594881 >> v2:172.21.15.69:3300/0 conn(0x7f5a18063840 0x7f5a180c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.601+0000 7f5a1659c700 1 -- 172.21.15.33:0/3090594881 >> v1:172.21.15.69:6789/0 conn(0x7f5a180aaf50 legacy=0x7f5a18063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.601+0000 7f5a1659c700 1 -- 172.21.15.33:0/3090594881 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a181582d0 con 0x7f5a18157700 2023-11-06T20:56:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.601+0000 7f5a1659c700 1 --2- 172.21.15.33:0/3090594881 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 0x7f5a180c6f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f5a08006fd0 tx=0x7f5a080090f0).ready entity=mon.2 client_cookie=f657fbdc3b547210 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.601+0000 7f5a1559a700 1 -- 172.21.15.33:0/3090594881 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800b580 con 0x7f5a18157700 2023-11-06T20:56:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.601+0000 7f5a1559a700 1 -- 172.21.15.33:0/3090594881 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0800b6e0 con 0x7f5a18157700 2023-11-06T20:56:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.601+0000 7f5a1559a700 1 -- 172.21.15.33:0/3090594881 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800ba60 con 0x7f5a18157700 2023-11-06T20:56:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.602+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/3090594881 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 msgr2=0x7f5a180c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.602+0000 7f5a1c9ff700 1 --2- 172.21.15.33:0/3090594881 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 0x7f5a180c6f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f5a08006fd0 tx=0x7f5a080090f0).stop 2023-11-06T20:56:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.602+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/3090594881 shutdown_connections 2023-11-06T20:56:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.602+0000 7f5a1c9ff700 1 --2- 172.21.15.33:0/3090594881 >> v2:172.21.15.69:3300/0 conn(0x7f5a18063840 0x7f5a180c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.602+0000 7f5a1c9ff700 1 --2- 172.21.15.33:0/3090594881 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 0x7f5a180c6f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.602+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/3090594881 >> 172.21.15.33:0/3090594881 conn(0x7f5a1805ec80 msgr2=0x7f5a180cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:52.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.602+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/3090594881 shutdown_connections 2023-11-06T20:56:52.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.602+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/3090594881 wait complete. 2023-11-06T20:56:52.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.603+0000 7f5a1c9ff700 1 Processor -- start 2023-11-06T20:56:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.604+0000 7f5a1c9ff700 1 -- start start 2023-11-06T20:56:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.604+0000 7f5a1c9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a18063840 0x7f5a180af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.604+0000 7f5a1c9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a180aaf50 0x7f5a18150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a1c9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 0x7f5a18150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a1c9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180b0eb0 con 0x7f5a180aaf50 2023-11-06T20:56:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a1c9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180ae280 con 0x7f5a18063840 2023-11-06T20:56:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a1c9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180b08f0 con 0x7f5a18157700 2023-11-06T20:56:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a15d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a180aaf50 0x7f5a18150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a15d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a180aaf50 0x7f5a18150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47900/0 (socket says 172.21.15.33:47900) 2023-11-06T20:56:52.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a15d9b700 1 -- 172.21.15.33:0/2388560537 learned_addr learned my addr 172.21.15.33:0/2388560537 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:52.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a1659c700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a18063840 0x7f5a180af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a16d9d700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 0x7f5a18150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a1659c700 1 -- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 msgr2=0x7f5a18150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a1659c700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 0x7f5a18150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a1659c700 1 -- 172.21.15.33:0/2388560537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a180aaf50 msgr2=0x7f5a18150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a1659c700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a180aaf50 0x7f5a18150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a1659c700 1 -- 172.21.15.33:0/2388560537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a08009370 con 0x7f5a18063840 2023-11-06T20:56:52.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.605+0000 7f5a15d9b700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a180aaf50 0x7f5a18150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:52.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.606+0000 7f5a1659c700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a18063840 0x7f5a180af910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f5a08014000 tx=0x7f5a08017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.606+0000 7f5a067fc700 1 -- 172.21.15.33:0/2388560537 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08000bc0 con 0x7f5a18063840 2023-11-06T20:56:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.606+0000 7f5a067fc700 1 -- 172.21.15.33:0/2388560537 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0800b850 con 0x7f5a18063840 2023-11-06T20:56:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.606+0000 7f5a067fc700 1 -- 172.21.15.33:0/2388560537 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08000d20 con 0x7f5a18063840 2023-11-06T20:56:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.606+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/2388560537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a180b0bd0 con 0x7f5a18063840 2023-11-06T20:56:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.606+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/2388560537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a18151170 con 0x7f5a18063840 2023-11-06T20:56:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.607+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/2388560537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a180ae560 con 0x7f5a18063840 2023-11-06T20:56:52.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.608+0000 7f5a067fc700 1 -- 172.21.15.33:0/2388560537 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5a0801d890 con 0x7f5a18063840 2023-11-06T20:56:52.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.609+0000 7f5a067fc700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f59f805f270 0x7f59f8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:52.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.609+0000 7f5a067fc700 1 -- 172.21.15.33:0/2388560537 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f5a0807bf60 con 0x7f5a18063840 2023-11-06T20:56:52.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.609+0000 7f5a15d9b700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f59f805f270 0x7f59f8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.610+0000 7f5a15d9b700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f59f805f270 0x7f59f8061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f5a18154840 tx=0x7f5a0c007d30).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.614+0000 7f5a067fc700 1 -- 172.21.15.33:0/2388560537 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5a0804d020 con 0x7f5a18063840 2023-11-06T20:56:52.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.793+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/2388560537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5a1806c6c0 con 0x7f5a18063840 2023-11-06T20:56:52.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.799+0000 7f5a067fc700 1 -- 172.21.15.33:0/2388560537 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5a080210a0 con 0x7f5a18063840 2023-11-06T20:56:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f59f805f270 msgr2=0x7f59f8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f59f805f270 0x7f59f8061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f5a18154840 tx=0x7f5a0c007d30).stop 2023-11-06T20:56:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a18063840 msgr2=0x7f5a180af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a18063840 0x7f5a180af910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f5a08014000 tx=0x7f5a08017000).stop 2023-11-06T20:56:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/2388560537 shutdown_connections 2023-11-06T20:56:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f59f805f270 0x7f59f8061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5a18157700 0x7f5a18150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a18063840 0x7f5a180af910 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 --2- 172.21.15.33:0/2388560537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a180aaf50 0x7f5a18150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/2388560537 >> 172.21.15.33:0/2388560537 conn(0x7f5a1805ec80 msgr2=0x7f5a180c5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/2388560537 shutdown_connections 2023-11-06T20:56:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:52.801+0000 7f5a1c9ff700 1 -- 172.21.15.33:0/2388560537 wait complete. 2023-11-06T20:56:52.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-11-06T20:56:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.017+0000 7f42b1fff700 1 Processor -- start 2023-11-06T20:56:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.017+0000 7f42b1fff700 1 -- start start 2023-11-06T20:56:53.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.018+0000 7f42b1fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42ac157720 0x7f42ac158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:53.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.018+0000 7f42b1fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f42ac0bccc0 0x7f42ac0a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:53.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.018+0000 7f42b1fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ac0bbfb0 con 0x7f42ac0a4e60 2023-11-06T20:56:53.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.018+0000 7f42b1fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ac0a52d0 con 0x7f42ac157720 2023-11-06T20:56:53.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.018+0000 7f42b1fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ac158d30 con 0x7f42ac0bccc0 2023-11-06T20:56:53.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.018+0000 7f42ab7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42ac157720 0x7f42ac158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:53.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.018+0000 7f42ab7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42ac157720 0x7f42ac158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47914/0 (socket says 172.21.15.33:47914) 2023-11-06T20:56:53.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.018+0000 7f42ab7fe700 1 -- 172.21.15.33:0/1384989977 learned_addr learned my addr 172.21.15.33:0/1384989977 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:53.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.019+0000 7f42aaffd700 1 --2- 172.21.15.33:0/1384989977 >> v2:172.21.15.69:3300/0 conn(0x7f42ac0bccc0 0x7f42ac0a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.019+0000 7f42ab7fe700 1 -- 172.21.15.33:0/1384989977 >> v2:172.21.15.69:3300/0 conn(0x7f42ac0bccc0 msgr2=0x7f42ac0a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.019+0000 7f42ab7fe700 1 --2- 172.21.15.33:0/1384989977 >> v2:172.21.15.69:3300/0 conn(0x7f42ac0bccc0 0x7f42ac0a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.019+0000 7f42ab7fe700 1 -- 172.21.15.33:0/1384989977 >> v1:172.21.15.69:6789/0 conn(0x7f42ac0a4e60 legacy=0x7f42ac0c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.019+0000 7f42ab7fe700 1 -- 172.21.15.33:0/1384989977 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac0bdb90 con 0x7f42ac157720 2023-11-06T20:56:53.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.019+0000 7f42ab7fe700 1 --2- 172.21.15.33:0/1384989977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac157720 0x7f42ac158840 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f42ac0657a0 tx=0x7f429c0090f0).ready entity=mon.0 client_cookie=929a0bdf349ced61 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.019+0000 7f42aa7fc700 1 -- 172.21.15.33:0/1384989977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00b5a0 con 0x7f42ac157720 2023-11-06T20:56:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.019+0000 7f42aa7fc700 1 -- 172.21.15.33:0/1384989977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c00b700 con 0x7f42ac157720 2023-11-06T20:56:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.019+0000 7f42aa7fc700 1 -- 172.21.15.33:0/1384989977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00ba80 con 0x7f42ac157720 2023-11-06T20:56:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.020+0000 7f42b1fff700 1 -- 172.21.15.33:0/1384989977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac157720 msgr2=0x7f42ac158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.020+0000 7f42b1fff700 1 --2- 172.21.15.33:0/1384989977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac157720 0x7f42ac158840 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f42ac0657a0 tx=0x7f429c0090f0).stop 2023-11-06T20:56:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.020+0000 7f42b1fff700 1 -- 172.21.15.33:0/1384989977 shutdown_connections 2023-11-06T20:56:53.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.020+0000 7f42b1fff700 1 --2- 172.21.15.33:0/1384989977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac157720 0x7f42ac158840 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.020+0000 7f42b1fff700 1 --2- 172.21.15.33:0/1384989977 >> v2:172.21.15.69:3300/0 conn(0x7f42ac0bccc0 0x7f42ac0a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.020+0000 7f42b1fff700 1 -- 172.21.15.33:0/1384989977 >> 172.21.15.33:0/1384989977 conn(0x7f42ac0631e0 msgr2=0x7f42ac0c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.020+0000 7f42b1fff700 1 -- 172.21.15.33:0/1384989977 shutdown_connections 2023-11-06T20:56:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.020+0000 7f42b1fff700 1 -- 172.21.15.33:0/1384989977 wait complete. 2023-11-06T20:56:53.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.021+0000 7f42b1fff700 1 Processor -- start 2023-11-06T20:56:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.022+0000 7f42b1fff700 1 -- start start 2023-11-06T20:56:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.022+0000 7f42b1fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42ac0a4e60 0x7f42ac0aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.022+0000 7f42b1fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac0bccc0 0x7f42ac14aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.022+0000 7f42b1fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42ac157720 0x7f42ac06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.022+0000 7f42b1fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0aacf0 con 0x7f42ac0bccc0 2023-11-06T20:56:53.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42b1fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0ac310 con 0x7f42ac157720 2023-11-06T20:56:53.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42b1fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0a8f90 con 0x7f42ac0a4e60 2023-11-06T20:56:53.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42aaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac0bccc0 0x7f42ac14aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:53.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42aaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac0bccc0 0x7f42ac14aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47918/0 (socket says 172.21.15.33:47918) 2023-11-06T20:56:53.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42aaffd700 1 -- 172.21.15.33:0/2688485120 learned_addr learned my addr 172.21.15.33:0/2688485120 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:53.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42ab7fe700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42ac0a4e60 0x7f42ac0aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:53.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42abfff700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42ac157720 0x7f42ac06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:53.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42aaffd700 1 -- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42ac0a4e60 msgr2=0x7f42ac0aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:53.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42aaffd700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42ac0a4e60 0x7f42ac0aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:53.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42aaffd700 1 -- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42ac157720 msgr2=0x7f42ac06a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:53.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42aaffd700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42ac157720 0x7f42ac06a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:53.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42aaffd700 1 -- 172.21.15.33:0/2688485120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c009370 con 0x7f42ac0bccc0 2023-11-06T20:56:53.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.023+0000 7f42aaffd700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac0bccc0 0x7f42ac14aee0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f42a000d890 tx=0x7f42a0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:53.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.024+0000 7f42937fe700 1 -- 172.21.15.33:0/2688485120 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0011590 con 0x7f42ac0bccc0 2023-11-06T20:56:53.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.024+0000 7f42937fe700 1 -- 172.21.15.33:0/2688485120 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a00116f0 con 0x7f42ac0bccc0 2023-11-06T20:56:53.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.024+0000 7f42b1fff700 1 -- 172.21.15.33:0/2688485120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac14b3d0 con 0x7f42ac0bccc0 2023-11-06T20:56:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.024+0000 7f42937fe700 1 -- 172.21.15.33:0/2688485120 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0011a70 con 0x7f42ac0bccc0 2023-11-06T20:56:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.024+0000 7f42b1fff700 1 -- 172.21.15.33:0/2688485120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac0a8660 con 0x7f42ac0bccc0 2023-11-06T20:56:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.025+0000 7f42b1fff700 1 -- 172.21.15.33:0/2688485120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ac0bc1e0 con 0x7f42ac0bccc0 2023-11-06T20:56:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.026+0000 7f42937fe700 1 -- 172.21.15.33:0/2688485120 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f42a001c460 con 0x7f42ac0bccc0 2023-11-06T20:56:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.027+0000 7f42937fe700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f429805f270 0x7f4298061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.027+0000 7f42937fe700 1 -- 172.21.15.33:0/2688485120 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f42a007a4b0 con 0x7f42ac0bccc0 2023-11-06T20:56:53.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.027+0000 7f42ab7fe700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f429805f270 0x7f4298061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:53.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.028+0000 7f42ab7fe700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f429805f270 0x7f4298061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f429c009130 tx=0x7f429c015000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:53.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.031+0000 7f42937fe700 1 -- 172.21.15.33:0/2688485120 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f42a004b510 con 0x7f42ac0bccc0 2023-11-06T20:56:53.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:53.286+0000 7f42b1fff700 1 -- 172.21.15.33:0/2688485120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f42ac060690 con 0x7f42ac0bccc0 2023-11-06T20:56:54.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.164+0000 7f42937fe700 1 -- 172.21.15.33:0/2688485120 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v453) v1 ==== 158+0+0 (secure 0 0 0) 0x7f42a0011e60 con 0x7f42ac0bccc0 2023-11-06T20:56:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 -- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f429805f270 msgr2=0x7f4298061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f429805f270 0x7f4298061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f429c009130 tx=0x7f429c015000).stop 2023-11-06T20:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 -- 172.21.15.33:0/2688485120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac0bccc0 msgr2=0x7f42ac14aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac0bccc0 0x7f42ac14aee0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f42a000d890 tx=0x7f42a0008040).stop 2023-11-06T20:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 -- 172.21.15.33:0/2688485120 shutdown_connections 2023-11-06T20:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f429805f270 0x7f4298061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f42ac0a4e60 0x7f42ac0aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42ac157720 0x7f42ac06a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 --2- 172.21.15.33:0/2688485120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac0bccc0 0x7f42ac14aee0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 -- 172.21.15.33:0/2688485120 >> 172.21.15.33:0/2688485120 conn(0x7f42ac0631e0 msgr2=0x7f42ac14cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 -- 172.21.15.33:0/2688485120 shutdown_connections 2023-11-06T20:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.167+0000 7f42b1fff700 1 -- 172.21.15.33:0/2688485120 wait complete. 2023-11-06T20:56:54.188 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-11-06T20:56:54.188 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-11-06T20:56:54.189 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-06T20:56:54.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-11-06T20:56:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.390+0000 7fb33afff700 1 Processor -- start 2023-11-06T20:56:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.390+0000 7fb33afff700 1 -- start start 2023-11-06T20:56:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.392+0000 7fb33afff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb334154760 0x7fb334063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.392+0000 7fb33afff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3340c31f0 con 0x7fb334063140 2023-11-06T20:56:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.392+0000 7fb33afff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3340c2920 con 0x7fb33405e1b0 2023-11-06T20:56:54.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.392+0000 7fb33afff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33406a0f0 con 0x7fb334154760 2023-11-06T20:56:54.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.392+0000 7fb338d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb334063140 0x7fb3340c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50474/0 (socket says 172.21.15.33:50474) 2023-11-06T20:56:54.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.392+0000 7fb338d9b700 1 -- 172.21.15.33:0/2491789628 learned_addr learned my addr 172.21.15.33:0/2491789628 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:54.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.392+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497215369 0 0) 0x7fb3340c31f0 con 0x7fb334063140 2023-11-06T20:56:54.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.392+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3240033f0 con 0x7fb334063140 2023-11-06T20:56:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.392+0000 7fb33959c700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb334154760 0x7fb334063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3451621090 0 0) 0x7fb3240033f0 con 0x7fb334063140 2023-11-06T20:56:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb334154760 msgr2=0x7fb334063520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb334154760 0x7fb334063520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 >> v1:172.21.15.69:6789/0 conn(0x7fb33405e1b0 legacy=0x7fb334154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33405e640 con 0x7fb334063140 2023-11-06T20:56:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fb328001470 con 0x7fb334063140 2023-11-06T20:56:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 >> v1:172.21.15.33:6789/0 conn(0x7fb334063140 legacy=0x7fb3340c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb334154760 0x7fb324003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:54.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324003e40 0x7fb324004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb324004b80 0x7fb324005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33406a0f0 con 0x7fb324004b80 2023-11-06T20:56:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32400b990 con 0x7fb324003e40 2023-11-06T20:56:54.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32400bb50 con 0x7fb334154760 2023-11-06T20:56:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb33959c700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb324004b80 0x7fb324005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb338d9b700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb334154760 0x7fb324003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.393+0000 7fb333fff700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324003e40 0x7fb324004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb33959c700 1 -- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb334154760 msgr2=0x7fb324003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:54.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb33959c700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb334154760 0x7fb324003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb33959c700 1 -- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324003e40 msgr2=0x7fb324004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb33959c700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324003e40 0x7fb324004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb33959c700 1 -- 172.21.15.33:0/2491789628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3240033f0 con 0x7fb324004b80 2023-11-06T20:56:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb333fff700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324003e40 0x7fb324004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:54.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb33959c700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb324004b80 0x7fb324005000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fb32c00e210 tx=0x7fb32c00b040).ready entity=mon.0 client_cookie=cabfd016c2f10a76 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c009ac0 con 0x7fb324004b80 2023-11-06T20:56:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c00f040 con 0x7fb324004b80 2023-11-06T20:56:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb3337fe700 1 -- 172.21.15.33:0/2491789628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c00ecd0 con 0x7fb324004b80 2023-11-06T20:56:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.394+0000 7fb33afff700 1 -- 172.21.15.33:0/2491789628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb324004b80 msgr2=0x7fb324005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.395+0000 7fb33afff700 1 --2- 172.21.15.33:0/2491789628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb324004b80 0x7fb324005000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fb32c00e210 tx=0x7fb32c00b040).stop 2023-11-06T20:56:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.395+0000 7fb33959c700 1 -- 172.21.15.33:0/2491789628 reap_dead start 2023-11-06T20:56:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.395+0000 7fb33afff700 1 -- 172.21.15.33:0/2491789628 shutdown_connections 2023-11-06T20:56:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.395+0000 7fb33afff700 1 -- 172.21.15.33:0/2491789628 >> 172.21.15.33:0/2491789628 conn(0x7fb3340c7d60 msgr2=0x7fb3340c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.395+0000 7fb33afff700 1 -- 172.21.15.33:0/2491789628 shutdown_connections 2023-11-06T20:56:54.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.395+0000 7fb33afff700 1 -- 172.21.15.33:0/2491789628 wait complete. 2023-11-06T20:56:54.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.396+0000 7fb33afff700 1 Processor -- start 2023-11-06T20:56:54.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.396+0000 7fb33afff700 1 -- start start 2023-11-06T20:56:54.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.397+0000 7fb33afff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb324004b80 0x7fb33413d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:54.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.397+0000 7fb33afff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb33413dc80 0x7fb334155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:54.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.397+0000 7fb33afff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334156120 0x7fb3341565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:54.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.397+0000 7fb33afff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33413ed80 con 0x7fb334156120 2023-11-06T20:56:54.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.397+0000 7fb33afff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb334156a90 con 0x7fb33413dc80 2023-11-06T20:56:54.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.397+0000 7fb33afff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33413e170 con 0x7fb324004b80 2023-11-06T20:56:54.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb33959c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334156120 0x7fb3341565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:54.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb338d9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb324004b80 0x7fb33413d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:54.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb33959c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334156120 0x7fb3341565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47926/0 (socket says 172.21.15.33:47926) 2023-11-06T20:56:54.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb33959c700 1 -- 172.21.15.33:0/3138369581 learned_addr learned my addr 172.21.15.33:0/3138369581 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:54.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb333fff700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb33413dc80 0x7fb334155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:54.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb33959c700 1 -- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb324004b80 msgr2=0x7fb33413d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:54.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb33959c700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb324004b80 0x7fb33413d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:54.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb33959c700 1 -- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb33413dc80 msgr2=0x7fb334155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:54.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb33959c700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb33413dc80 0x7fb334155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:54.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb33959c700 1 -- 172.21.15.33:0/3138369581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33413e450 con 0x7fb334156120 2023-11-06T20:56:54.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb33959c700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334156120 0x7fb3341565a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fb32c014000 tx=0x7fb32c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:54.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.398+0000 7fb330ff9700 1 -- 172.21.15.33:0/3138369581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c00ecd0 con 0x7fb334156120 2023-11-06T20:56:54.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.399+0000 7fb330ff9700 1 -- 172.21.15.33:0/3138369581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c01e410 con 0x7fb334156120 2023-11-06T20:56:54.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.399+0000 7fb330ff9700 1 -- 172.21.15.33:0/3138369581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c01e600 con 0x7fb334156120 2023-11-06T20:56:54.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.399+0000 7fb33afff700 1 -- 172.21.15.33:0/3138369581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33413e730 con 0x7fb334156120 2023-11-06T20:56:54.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.399+0000 7fb33afff700 1 -- 172.21.15.33:0/3138369581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33413d370 con 0x7fb334156120 2023-11-06T20:56:54.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.400+0000 7fb33afff700 1 -- 172.21.15.33:0/3138369581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb334153af0 con 0x7fb334156120 2023-11-06T20:56:54.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.400+0000 7fb330ff9700 1 -- 172.21.15.33:0/3138369581 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb32c024030 con 0x7fb334156120 2023-11-06T20:56:54.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.402+0000 7fb330ff9700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb31805f270 0x7fb318061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:54.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.402+0000 7fb330ff9700 1 -- 172.21.15.33:0/3138369581 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fb32c07c400 con 0x7fb334156120 2023-11-06T20:56:54.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.402+0000 7fb338d9b700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb31805f270 0x7fb318061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:54.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.403+0000 7fb338d9b700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb31805f270 0x7fb318061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb3280010e0 tx=0x7fb328001e50).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:54.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.406+0000 7fb330ff9700 1 -- 172.21.15.33:0/3138369581 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb32c04d460 con 0x7fb334156120 2023-11-06T20:56:54.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:54.658+0000 7fb33afff700 1 -- 172.21.15.33:0/3138369581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fb334064420 con 0x7fb334156120 2023-11-06T20:56:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.168+0000 7fb330ff9700 1 -- 172.21.15.33:0/3138369581 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v454) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb32c02c020 con 0x7fb334156120 2023-11-06T20:56:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 -- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb31805f270 msgr2=0x7fb318061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb31805f270 0x7fb318061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb3280010e0 tx=0x7fb328001e50).stop 2023-11-06T20:56:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 -- 172.21.15.33:0/3138369581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334156120 msgr2=0x7fb3341565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334156120 0x7fb3341565a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fb32c014000 tx=0x7fb32c01a000).stop 2023-11-06T20:56:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 -- 172.21.15.33:0/3138369581 shutdown_connections 2023-11-06T20:56:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb31805f270 0x7fb318061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb324004b80 0x7fb33413d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334156120 0x7fb3341565a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 --2- 172.21.15.33:0/3138369581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb33413dc80 0x7fb334155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 -- 172.21.15.33:0/3138369581 >> 172.21.15.33:0/3138369581 conn(0x7fb3340c7d60 msgr2=0x7fb3340c6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 -- 172.21.15.33:0/3138369581 shutdown_connections 2023-11-06T20:56:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.171+0000 7fb33afff700 1 -- 172.21.15.33:0/3138369581 wait complete. 2023-11-06T20:56:55.190 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T20:56:55.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.390+0000 7f7154fff700 1 Processor -- start 2023-11-06T20:56:55.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.391+0000 7f7154fff700 1 -- start start 2023-11-06T20:56:55.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.392+0000 7f7154fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7150064680 0x7f71500bdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:55.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.392+0000 7f7154fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71500641f0 con 0x7f71501546a0 2023-11-06T20:56:55.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.392+0000 7f7154fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71500be4f0 con 0x7f71500be0a0 2023-11-06T20:56:55.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.392+0000 7f7154fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7150154a80 con 0x7f7150064680 2023-11-06T20:56:55.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.392+0000 7f714e59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f71501546a0 0x7f7150066560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50480/0 (socket says 172.21.15.33:50480) 2023-11-06T20:56:55.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.392+0000 7f714e59c700 1 -- 172.21.15.33:0/1946993352 learned_addr learned my addr 172.21.15.33:0/1946993352 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:55.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.392+0000 7f714dd9b700 1 --2- 172.21.15.33:0/1946993352 >> v2:172.21.15.69:3300/0 conn(0x7f7150064680 0x7f71500bdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.393+0000 7f714d59a700 1 -- 172.21.15.33:0/1946993352 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273923120 0 0) 0x7f71500641f0 con 0x7f71501546a0 2023-11-06T20:56:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.393+0000 7f714d59a700 1 -- 172.21.15.33:0/1946993352 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71380033f0 con 0x7f71501546a0 2023-11-06T20:56:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.393+0000 7f714dd9b700 1 -- 172.21.15.33:0/1946993352 >> v1:172.21.15.69:6789/0 conn(0x7f71500be0a0 legacy=0x7f71500c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.393+0000 7f714dd9b700 1 -- 172.21.15.33:0/1946993352 >> v1:172.21.15.33:6789/0 conn(0x7f71501546a0 legacy=0x7f7150066560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.393+0000 7f714dd9b700 1 -- 172.21.15.33:0/1946993352 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71500638c0 con 0x7f7150064680 2023-11-06T20:56:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.393+0000 7f714dd9b700 1 --2- 172.21.15.33:0/1946993352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7150064680 0x7f71500bdbb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f7144009760 tx=0x7f7144009060).ready entity=mon.1 client_cookie=c0f83e1ecf9135a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.393+0000 7f714d59a700 1 -- 172.21.15.33:0/1946993352 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714400f510 con 0x7f7150064680 2023-11-06T20:56:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.393+0000 7f714d59a700 1 -- 172.21.15.33:0/1946993352 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714400f670 con 0x7f7150064680 2023-11-06T20:56:55.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.393+0000 7f714d59a700 1 -- 172.21.15.33:0/1946993352 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714400f9f0 con 0x7f7150064680 2023-11-06T20:56:55.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.394+0000 7f7154fff700 1 -- 172.21.15.33:0/1946993352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7150064680 msgr2=0x7f71500bdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:55.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.394+0000 7f7154fff700 1 --2- 172.21.15.33:0/1946993352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7150064680 0x7f71500bdbb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f7144009760 tx=0x7f7144009060).stop 2023-11-06T20:56:55.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.394+0000 7f7154fff700 1 -- 172.21.15.33:0/1946993352 shutdown_connections 2023-11-06T20:56:55.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.394+0000 7f7154fff700 1 --2- 172.21.15.33:0/1946993352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7150064680 0x7f71500bdbb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.394+0000 7f7154fff700 1 -- 172.21.15.33:0/1946993352 >> 172.21.15.33:0/1946993352 conn(0x7f71500c8c50 msgr2=0x7f71500c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:55.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.394+0000 7f7154fff700 1 -- 172.21.15.33:0/1946993352 shutdown_connections 2023-11-06T20:56:55.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.394+0000 7f7154fff700 1 -- 172.21.15.33:0/1946993352 wait complete. 2023-11-06T20:56:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.396+0000 7f7154fff700 1 Processor -- start 2023-11-06T20:56:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.396+0000 7f7154fff700 1 -- start start 2023-11-06T20:56:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f7154fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7150064680 0x7f7150067c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f7154fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71500be0a0 0x7f7150154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f7154fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71501546a0 0x7f71501550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f7154fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71501562a0 con 0x7f71501546a0 2023-11-06T20:56:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f7154fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7150155690 con 0x7f71500be0a0 2023-11-06T20:56:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f7154fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7150155970 con 0x7f7150064680 2023-11-06T20:56:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71501546a0 0x7f71501550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71501546a0 0x7f71501550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47942/0 (socket says 172.21.15.33:47942) 2023-11-06T20:56:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714ed9d700 1 -- 172.21.15.33:0/3683939294 learned_addr learned my addr 172.21.15.33:0/3683939294 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714dd9b700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71500be0a0 0x7f7150154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714e59c700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7150064680 0x7f7150067c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714ed9d700 1 -- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7150064680 msgr2=0x7f7150067c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714ed9d700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7150064680 0x7f7150067c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714ed9d700 1 -- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71500be0a0 msgr2=0x7f7150154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714ed9d700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71500be0a0 0x7f7150154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714ed9d700 1 -- 172.21.15.33:0/3683939294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71440092a0 con 0x7f71501546a0 2023-11-06T20:56:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714e59c700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7150064680 0x7f7150067c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.398+0000 7f714dd9b700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71500be0a0 0x7f7150154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.399+0000 7f714ed9d700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71501546a0 0x7f71501550c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f7148002230 tx=0x7f71480015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.399+0000 7f713e7fc700 1 -- 172.21.15.33:0/3683939294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148010510 con 0x7f71501546a0 2023-11-06T20:56:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.399+0000 7f713e7fc700 1 -- 172.21.15.33:0/3683939294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7148010670 con 0x7f71501546a0 2023-11-06T20:56:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.399+0000 7f7154fff700 1 -- 172.21.15.33:0/3683939294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7150155c50 con 0x7f71501546a0 2023-11-06T20:56:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.399+0000 7f713e7fc700 1 -- 172.21.15.33:0/3683939294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71480109f0 con 0x7f71501546a0 2023-11-06T20:56:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.399+0000 7f7154fff700 1 -- 172.21.15.33:0/3683939294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71500c1760 con 0x7f71501546a0 2023-11-06T20:56:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.400+0000 7f7154fff700 1 -- 172.21.15.33:0/3683939294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7150066000 con 0x7f71501546a0 2023-11-06T20:56:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.401+0000 7f713e7fc700 1 -- 172.21.15.33:0/3683939294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f714801b400 con 0x7f71501546a0 2023-11-06T20:56:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.401+0000 7f713e7fc700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f713005f270 0x7f7130061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:55.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.402+0000 7f713e7fc700 1 -- 172.21.15.33:0/3683939294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f714807a310 con 0x7f71501546a0 2023-11-06T20:56:55.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.402+0000 7f714e59c700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f713005f270 0x7f7130061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:55.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.403+0000 7f714e59c700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f713005f270 0x7f7130061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f7140007fd0 tx=0x7f7140009040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:55.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.406+0000 7f713e7fc700 1 -- 172.21.15.33:0/3683939294 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f714804b370 con 0x7f71501546a0 2023-11-06T20:56:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.584+0000 7f7154fff700 1 -- 172.21.15.33:0/3683939294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f71500c6f20 con 0x7f71501546a0 2023-11-06T20:56:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.590+0000 7f713e7fc700 1 -- 172.21.15.33:0/3683939294 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f714804e990 con 0x7f71501546a0 2023-11-06T20:56:55.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 -- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f713005f270 msgr2=0x7f7130061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:55.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f713005f270 0x7f7130061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f7140007fd0 tx=0x7f7140009040).stop 2023-11-06T20:56:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 -- 172.21.15.33:0/3683939294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71501546a0 msgr2=0x7f71501550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71501546a0 0x7f71501550c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f7148002230 tx=0x7f71480015e0).stop 2023-11-06T20:56:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 -- 172.21.15.33:0/3683939294 shutdown_connections 2023-11-06T20:56:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f713005f270 0x7f7130061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7150064680 0x7f7150067c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71501546a0 0x7f71501550c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 --2- 172.21.15.33:0/3683939294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71500be0a0 0x7f7150154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 -- 172.21.15.33:0/3683939294 >> 172.21.15.33:0/3683939294 conn(0x7f71500c8c50 msgr2=0x7f71500ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 -- 172.21.15.33:0/3683939294 shutdown_connections 2023-11-06T20:56:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:55.592+0000 7f7154fff700 1 -- 172.21.15.33:0/3683939294 wait complete. 2023-11-06T20:56:55.610 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T20:56:55.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:56:55.710 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:56:55.803 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T20:56:55.804 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T20:56:55.804 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:56:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.000+0000 7f4e4e7ff700 1 Processor -- start 2023-11-06T20:56:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.000+0000 7f4e4e7ff700 1 -- start start 2023-11-06T20:56:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.001+0000 7f4e4e7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e480c1840 0x7f4e480a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.001+0000 7f4e4e7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4e480aa2d0 0x7f4e480c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.001+0000 7f4e4e7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e481554a0 con 0x7f4e480c6240 2023-11-06T20:56:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.001+0000 7f4e4e7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e48154960 con 0x7f4e480c1840 2023-11-06T20:56:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.001+0000 7f4e4e7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e48155930 con 0x7f4e480aa2d0 2023-11-06T20:56:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e47fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e480c1840 0x7f4e480a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e47fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e480c1840 0x7f4e480a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47948/0 (socket says 172.21.15.33:47948) 2023-11-06T20:56:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e47fff700 1 -- 172.21.15.33:0/2485005296 learned_addr learned my addr 172.21.15.33:0/2485005296 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e4cd9c700 1 --1- 172.21.15.33:0/2485005296 >> v1:172.21.15.33:6789/0 conn(0x7f4e480c6240 0x7f4e48157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50492/0 (socket says 172.21.15.33:50492) 2023-11-06T20:56:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e477fe700 1 --2- 172.21.15.33:0/2485005296 >> v2:172.21.15.69:3300/0 conn(0x7f4e480aa2d0 0x7f4e480c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:56.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e47fff700 1 -- 172.21.15.33:0/2485005296 >> v2:172.21.15.69:3300/0 conn(0x7f4e480aa2d0 msgr2=0x7f4e480c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:56.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e47fff700 1 --2- 172.21.15.33:0/2485005296 >> v2:172.21.15.69:3300/0 conn(0x7f4e480aa2d0 0x7f4e480c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:56.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e47fff700 1 -- 172.21.15.33:0/2485005296 >> v1:172.21.15.33:6789/0 conn(0x7f4e480c6240 legacy=0x7f4e48157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:56.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e47fff700 1 -- 172.21.15.33:0/2485005296 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e48154b70 con 0x7f4e480c1840 2023-11-06T20:56:56.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e46ffd700 1 -- 172.21.15.33:0/2485005296 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690183104 0 0) 0x7f4e481554a0 con 0x7f4e480c6240 2023-11-06T20:56:56.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e477fe700 1 --2- 172.21.15.33:0/2485005296 >> v2:172.21.15.69:3300/0 conn(0x7f4e480aa2d0 0x7f4e480c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:56.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.002+0000 7f4e47fff700 1 --2- 172.21.15.33:0/2485005296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480c1840 0x7f4e480a9de0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f4e480adb40 tx=0x7f4e380090f0).ready entity=mon.0 client_cookie=25e3ee0096adb7cf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:56.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e46ffd700 1 -- 172.21.15.33:0/2485005296 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800b600 con 0x7f4e480c1840 2023-11-06T20:56:56.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e46ffd700 1 -- 172.21.15.33:0/2485005296 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3800b760 con 0x7f4e480c1840 2023-11-06T20:56:56.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e46ffd700 1 -- 172.21.15.33:0/2485005296 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800bae0 con 0x7f4e480c1840 2023-11-06T20:56:56.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/2485005296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480c1840 msgr2=0x7f4e480a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:56.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e4e7ff700 1 --2- 172.21.15.33:0/2485005296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480c1840 0x7f4e480a9de0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f4e480adb40 tx=0x7f4e380090f0).stop 2023-11-06T20:56:56.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/2485005296 shutdown_connections 2023-11-06T20:56:56.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e4e7ff700 1 --2- 172.21.15.33:0/2485005296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480c1840 0x7f4e480a9de0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:56.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e4e7ff700 1 --2- 172.21.15.33:0/2485005296 >> v2:172.21.15.69:3300/0 conn(0x7f4e480aa2d0 0x7f4e480c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:56.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/2485005296 >> 172.21.15.33:0/2485005296 conn(0x7f4e4805ec80 msgr2=0x7f4e480cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:56.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/2485005296 shutdown_connections 2023-11-06T20:56:56.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.003+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/2485005296 wait complete. 2023-11-06T20:56:56.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.005+0000 7f4e4e7ff700 1 Processor -- start 2023-11-06T20:56:56.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.005+0000 7f4e4e7ff700 1 -- start start 2023-11-06T20:56:56.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e4e7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480aa2d0 0x7f4e48165880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e4e7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e480c1840 0x7f4e480c2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e4e7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e480c6240 0x7f4e48167410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e4e7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48165d70 con 0x7f4e480aa2d0 2023-11-06T20:56:56.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e4e7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e480a9f20 con 0x7f4e480c1840 2023-11-06T20:56:56.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e4e7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e480a9880 con 0x7f4e480c6240 2023-11-06T20:56:56.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e47fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480aa2d0 0x7f4e48165880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:56.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e47fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480aa2d0 0x7f4e48165880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47956/0 (socket says 172.21.15.33:47956) 2023-11-06T20:56:56.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e47fff700 1 -- 172.21.15.33:0/538500348 learned_addr learned my addr 172.21.15.33:0/538500348 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:56.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e47fff700 1 -- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e480c6240 msgr2=0x7f4e48167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:56.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e47fff700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e480c6240 0x7f4e48167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:56.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e47fff700 1 -- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e480c1840 msgr2=0x7f4e480c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:56.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e47fff700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e480c1840 0x7f4e480c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:56.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.006+0000 7f4e47fff700 1 -- 172.21.15.33:0/538500348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38009370 con 0x7f4e480aa2d0 2023-11-06T20:56:56.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.007+0000 7f4e47fff700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480aa2d0 0x7f4e48165880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f4e4806bc90 tx=0x7f4e38017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:56.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.007+0000 7f4e2ffff700 1 -- 172.21.15.33:0/538500348 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800b8d0 con 0x7f4e480aa2d0 2023-11-06T20:56:56.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.007+0000 7f4e2ffff700 1 -- 172.21.15.33:0/538500348 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3800ba30 con 0x7f4e480aa2d0 2023-11-06T20:56:56.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.007+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/538500348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e480c2050 con 0x7f4e480aa2d0 2023-11-06T20:56:56.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.007+0000 7f4e2ffff700 1 -- 172.21.15.33:0/538500348 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3801c670 con 0x7f4e480aa2d0 2023-11-06T20:56:56.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.007+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/538500348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e480a9b40 con 0x7f4e480aa2d0 2023-11-06T20:56:56.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.008+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/538500348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4806c6c0 con 0x7f4e480aa2d0 2023-11-06T20:56:56.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.009+0000 7f4e2ffff700 1 -- 172.21.15.33:0/538500348 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4e3801c9e0 con 0x7f4e480aa2d0 2023-11-06T20:56:56.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.009+0000 7f4e2ffff700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e3405f270 0x7f4e34061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.010+0000 7f4e2ffff700 1 -- 172.21.15.33:0/538500348 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f4e3807bbf0 con 0x7f4e480aa2d0 2023-11-06T20:56:56.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.010+0000 7f4e477fe700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e3405f270 0x7f4e34061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:56.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.010+0000 7f4e477fe700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e3405f270 0x7f4e34061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f4e3c006d80 tx=0x7f4e3c008040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:56.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.013+0000 7f4e2ffff700 1 -- 172.21.15.33:0/538500348 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4e38082080 con 0x7f4e480aa2d0 2023-11-06T20:56:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.241+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/538500348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e48063f20 con 0x7f4e480aa2d0 2023-11-06T20:56:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.241+0000 7f4e2ffff700 1 -- 172.21.15.33:0/538500348 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f4e3804cd70 con 0x7f4e480aa2d0 2023-11-06T20:56:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:56.242 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/2191444463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2191444463},{"type":"v1","addr":"172.21.15.69:6836","nonce":2191444463}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3920247513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3920247513},{"type":"v1","addr":"172.21.15.33:6834","nonce":3920247513}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2046477314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2046477314},{"type":"v1","addr":"172.21.15.69:6837","nonce":2046477314}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2247388400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2247388400},{"type":"v1","addr":"172.21.15.33:6835","nonce":2247388400}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[]} 2023-11-06T20:56:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.243+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e3405f270 msgr2=0x7f4e34061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.243+0000 7f4e4e7ff700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e3405f270 0x7f4e34061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f4e3c006d80 tx=0x7f4e3c008040).stop 2023-11-06T20:56:56.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.243+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/538500348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480aa2d0 msgr2=0x7f4e48165880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.243+0000 7f4e4e7ff700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480aa2d0 0x7f4e48165880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f4e4806bc90 tx=0x7f4e38017000).stop 2023-11-06T20:56:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.244+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/538500348 shutdown_connections 2023-11-06T20:56:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.244+0000 7f4e4e7ff700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4e3405f270 0x7f4e34061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.244+0000 7f4e4e7ff700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4e480c6240 0x7f4e48167410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.244+0000 7f4e4e7ff700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e480aa2d0 0x7f4e48165880 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.244+0000 7f4e4e7ff700 1 --2- 172.21.15.33:0/538500348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e480c1840 0x7f4e480c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.244+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/538500348 >> 172.21.15.33:0/538500348 conn(0x7f4e4805ec80 msgr2=0x7f4e480c0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.244+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/538500348 shutdown_connections 2023-11-06T20:56:56.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.244+0000 7f4e4e7ff700 1 -- 172.21.15.33:0/538500348 wait complete. 2023-11-06T20:56:56.247 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2023-11-06T20:56:56.262 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T20:56:56.262 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T20:56:56.263 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T20:56:56.263 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:56:56.263 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T20:56:56.263 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T20:56:56.264 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T20:56:56.264 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:56:56.264 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T20:56:56.264 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T20:56:56.265 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T20:56:56.265 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:56:56.265 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T20:56:56.266 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T20:56:56.266 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T20:56:56.266 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:56:56.276 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T20:56:56.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T20:56:56.278 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T20:56:56.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T20:56:56.288 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T20:56:56.288 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T20:56:56.289 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T20:56:56.289 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T20:56:56.322 INFO:tasks.ceph.mds.c:Started 2023-11-06T20:56:56.323 DEBUG:teuthology.parallel:result is None 2023-11-06T20:56:56.326 INFO:tasks.ceph.mds.a:Started 2023-11-06T20:56:56.326 DEBUG:teuthology.parallel:result is None 2023-11-06T20:56:56.333 INFO:tasks.ceph.mds.b:Started 2023-11-06T20:56:56.334 DEBUG:teuthology.parallel:result is None 2023-11-06T20:56:56.336 INFO:tasks.ceph.mds.d:Started 2023-11-06T20:56:56.337 DEBUG:teuthology.parallel:result is None 2023-11-06T20:56:56.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T20:56:56.470 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T20:56:56.472 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T20:56:56.489 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T20:56:56.489 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T20:56:56.491 INFO:tasks.ceph.mds.d.smithi069.stderr:2023-11-06T20:56:56.491+0000 7f288a9fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-06T20:56:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.553+0000 7ff6ab7ff700 1 Processor -- start 2023-11-06T20:56:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.553+0000 7ff6ab7ff700 1 -- start start 2023-11-06T20:56:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.555+0000 7ff6ab7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c6d90 0x7ff6a40c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.555+0000 7ff6ab7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff6a40bfd60 0x7ff6a40c01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.555+0000 7ff6ab7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a40c7700 con 0x7ff6a4154210 2023-11-06T20:56:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.555+0000 7ff6ab7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a40c06e0 con 0x7ff6a40bfd60 2023-11-06T20:56:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.555+0000 7ff6ab7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a405efc0 con 0x7ff6a40c6d90 2023-11-06T20:56:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.555+0000 7ff6a959b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff6a4154210 0x7ff6a40a7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50520/0 (socket says 172.21.15.33:50520) 2023-11-06T20:56:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.555+0000 7ff6a959b700 1 -- 172.21.15.33:0/1321906275 learned_addr learned my addr 172.21.15.33:0/1321906275 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:56.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.555+0000 7ff6a9d9c700 1 --2- 172.21.15.33:0/1321906275 >> v2:172.21.15.69:3300/0 conn(0x7ff6a40bfd60 0x7ff6a40c01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:56.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.555+0000 7ff6a8d9a700 1 --2- 172.21.15.33:0/1321906275 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c6d90 0x7ff6a40c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:56.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.556+0000 7ff69bfff700 1 -- 172.21.15.33:0/1321906275 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231826292 0 0) 0x7ff6a0001110 con 0x7ff6a4154210 2023-11-06T20:56:56.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.556+0000 7ff69bfff700 1 -- 172.21.15.33:0/1321906275 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6900033f0 con 0x7ff6a4154210 2023-11-06T20:56:56.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.556+0000 7ff6a9d9c700 1 -- 172.21.15.33:0/1321906275 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c6d90 msgr2=0x7ff6a40c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.556+0000 7ff6a9d9c700 1 --2- 172.21.15.33:0/1321906275 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c6d90 0x7ff6a40c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.556+0000 7ff6a9d9c700 1 -- 172.21.15.33:0/1321906275 >> v1:172.21.15.33:6789/0 conn(0x7ff6a4154210 legacy=0x7ff6a40a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.556+0000 7ff6a9d9c700 1 -- 172.21.15.33:0/1321906275 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a40c8ea0 con 0x7ff6a40bfd60 2023-11-06T20:56:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.556+0000 7ff6a9d9c700 1 --2- 172.21.15.33:0/1321906275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a40bfd60 0x7ff6a40c01f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7ff69c009e60 tx=0x7ff69c009780).ready entity=mon.1 client_cookie=2656058419b0cec7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.556+0000 7ff69bfff700 1 -- 172.21.15.33:0/1321906275 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00e070 con 0x7ff6a40bfd60 2023-11-06T20:56:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.556+0000 7ff69bfff700 1 -- 172.21.15.33:0/1321906275 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c00bab0 con 0x7ff6a40bfd60 2023-11-06T20:56:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.556+0000 7ff69bfff700 1 -- 172.21.15.33:0/1321906275 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00be30 con 0x7ff6a40bfd60 2023-11-06T20:56:56.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.557+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/1321906275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a40bfd60 msgr2=0x7ff6a40c01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:56.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.557+0000 7ff6ab7ff700 1 --2- 172.21.15.33:0/1321906275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a40bfd60 0x7ff6a40c01f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7ff69c009e60 tx=0x7ff69c009780).stop 2023-11-06T20:56:56.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.557+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/1321906275 shutdown_connections 2023-11-06T20:56:56.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.557+0000 7ff6ab7ff700 1 --2- 172.21.15.33:0/1321906275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a40bfd60 0x7ff6a40c01f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:56.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.557+0000 7ff6ab7ff700 1 --2- 172.21.15.33:0/1321906275 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c6d90 0x7ff6a40c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:56.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.557+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/1321906275 >> 172.21.15.33:0/1321906275 conn(0x7ff6a405e990 msgr2=0x7ff6a40cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:56.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.557+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/1321906275 shutdown_connections 2023-11-06T20:56:56.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.557+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/1321906275 wait complete. 2023-11-06T20:56:56.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.558+0000 7ff6ab7ff700 1 Processor -- start 2023-11-06T20:56:56.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.558+0000 7ff6ab7ff700 1 -- start start 2023-11-06T20:56:56.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.559+0000 7ff6ab7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40bfd60 0x7ff6a41685c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.559+0000 7ff6ab7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c6d90 0x7ff6a40c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.559+0000 7ff6ab7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a4154210 0x7ff6a41426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.559+0000 7ff6ab7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40c3280 con 0x7ff6a40bfd60 2023-11-06T20:56:56.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.559+0000 7ff6ab7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4156170 con 0x7ff6a4154210 2023-11-06T20:56:56.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.559+0000 7ff6ab7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4167b70 con 0x7ff6a40c6d90 2023-11-06T20:56:56.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a959b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40bfd60 0x7ff6a41685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:56.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a959b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40bfd60 0x7ff6a41685c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48002/0 (socket says 172.21.15.33:48002) 2023-11-06T20:56:56.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a959b700 1 -- 172.21.15.33:0/69929150 learned_addr learned my addr 172.21.15.33:0/69929150 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:56.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a8d9a700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c6d90 0x7ff6a40c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:56.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a9d9c700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a4154210 0x7ff6a41426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:56.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a959b700 1 -- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c6d90 msgr2=0x7ff6a40c2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:56.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a959b700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c6d90 0x7ff6a40c2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:56.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a959b700 1 -- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a4154210 msgr2=0x7ff6a41426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:56.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a959b700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a4154210 0x7ff6a41426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:56.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a959b700 1 -- 172.21.15.33:0/69929150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c009a20 con 0x7ff6a40bfd60 2023-11-06T20:56:56.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6a959b700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40bfd60 0x7ff6a41685c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7ff6a000c470 tx=0x7ff6a0007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:56.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.560+0000 7ff6997fa700 1 -- 172.21.15.33:0/69929150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00070d0 con 0x7ff6a40bfd60 2023-11-06T20:56:56.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.561+0000 7ff6997fa700 1 -- 172.21.15.33:0/69929150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a0007230 con 0x7ff6a40bfd60 2023-11-06T20:56:56.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.561+0000 7ff6997fa700 1 -- 172.21.15.33:0/69929150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00075f0 con 0x7ff6a40bfd60 2023-11-06T20:56:56.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.561+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/69929150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a4168130 con 0x7ff6a40bfd60 2023-11-06T20:56:56.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.561+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/69929150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a40c3750 con 0x7ff6a40bfd60 2023-11-06T20:56:56.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.562+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/69929150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a406c6c0 con 0x7ff6a40bfd60 2023-11-06T20:56:56.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.563+0000 7ff6997fa700 1 -- 172.21.15.33:0/69929150 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff6a00059b0 con 0x7ff6a40bfd60 2023-11-06T20:56:56.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.564+0000 7ff6997fa700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f270 0x7ff688061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:56.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.564+0000 7ff6997fa700 1 -- 172.21.15.33:0/69929150 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5699+0+0 (secure 0 0 0) 0x7ff6a00796e0 con 0x7ff6a40bfd60 2023-11-06T20:56:56.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.564+0000 7ff6a8d9a700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f270 0x7ff688061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:56.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.565+0000 7ff6a8d9a700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f270 0x7ff688061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff6a40c2cc0 tx=0x7ff694009060).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:56.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.567+0000 7ff6997fa700 1 -- 172.21.15.33:0/69929150 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff6a004a910 con 0x7ff6a40bfd60 2023-11-06T20:56:56.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:56.764+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/69929150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff6a40c9140 con 0x7ff6a40bfd60 2023-11-06T20:56:57.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.516+0000 7ff6997fa700 1 -- 172.21.15.33:0/69929150 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff6a004df30 con 0x7ff6a40bfd60 2023-11-06T20:56:57.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f270 msgr2=0x7ff688061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:57.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f270 0x7ff688061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff6a40c2cc0 tx=0x7ff694009060).stop 2023-11-06T20:56:57.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/69929150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40bfd60 msgr2=0x7ff6a41685c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:57.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40bfd60 0x7ff6a41685c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7ff6a000c470 tx=0x7ff6a0007af0).stop 2023-11-06T20:56:57.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/69929150 shutdown_connections 2023-11-06T20:56:57.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff68805f270 0x7ff688061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:57.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a4154210 0x7ff6a41426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:57.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6a40bfd60 0x7ff6a41685c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:57.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 --2- 172.21.15.33:0/69929150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6a40c6d90 0x7ff6a40c2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:57.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/69929150 >> 172.21.15.33:0/69929150 conn(0x7ff6a405e990 msgr2=0x7ff6a40a7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/69929150 shutdown_connections 2023-11-06T20:56:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.519+0000 7ff6ab7ff700 1 -- 172.21.15.33:0/69929150 wait complete. 2023-11-06T20:56:57.521 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T20:56:57.539 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T20:56:57.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.739+0000 7fd98fdff700 1 Processor -- start 2023-11-06T20:56:57.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.740+0000 7fd98fdff700 1 -- start start 2023-11-06T20:56:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.741+0000 7fd98fdff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9881521a0 0x7fd9880c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:57.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.741+0000 7fd98fdff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd988154e90 con 0x7fd988151dc0 2023-11-06T20:56:57.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.741+0000 7fd98fdff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9880639d0 con 0x7fd988155320 2023-11-06T20:56:57.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.741+0000 7fd98fdff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd988063cb0 con 0x7fd9881521a0 2023-11-06T20:56:57.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.741+0000 7fd98d39a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9881521a0 0x7fd9880c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:57.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.741+0000 7fd98db9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd988151dc0 0x7fd988063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50532/0 (socket says 172.21.15.33:50532) 2023-11-06T20:56:57.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.741+0000 7fd98db9b700 1 -- 172.21.15.33:0/115229681 learned_addr learned my addr 172.21.15.33:0/115229681 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:57.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.742+0000 7fd98d39a700 1 -- 172.21.15.33:0/115229681 >> v1:172.21.15.69:6789/0 conn(0x7fd988155320 legacy=0x7fd9880c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:57.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.742+0000 7fd98d39a700 1 -- 172.21.15.33:0/115229681 >> v1:172.21.15.33:6789/0 conn(0x7fd988151dc0 legacy=0x7fd988063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:57.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.742+0000 7fd98d39a700 1 -- 172.21.15.33:0/115229681 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988063f90 con 0x7fd9881521a0 2023-11-06T20:56:57.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.742+0000 7fd98d39a700 1 --2- 172.21.15.33:0/115229681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 0x7fd9880c0c50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fd978009760 tx=0x7fd978009060).ready entity=mon.0 client_cookie=b02e427619645a04 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:57.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.742+0000 7fd98cb99700 1 -- 172.21.15.33:0/115229681 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800b580 con 0x7fd9881521a0 2023-11-06T20:56:57.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.742+0000 7fd98cb99700 1 -- 172.21.15.33:0/115229681 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97800b6e0 con 0x7fd9881521a0 2023-11-06T20:56:57.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.742+0000 7fd98cb99700 1 -- 172.21.15.33:0/115229681 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800ba60 con 0x7fd9881521a0 2023-11-06T20:56:57.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.743+0000 7fd98fdff700 1 -- 172.21.15.33:0/115229681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 msgr2=0x7fd9880c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:57.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.743+0000 7fd98fdff700 1 --2- 172.21.15.33:0/115229681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 0x7fd9880c0c50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fd978009760 tx=0x7fd978009060).stop 2023-11-06T20:56:57.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.743+0000 7fd98fdff700 1 -- 172.21.15.33:0/115229681 shutdown_connections 2023-11-06T20:56:57.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.743+0000 7fd98fdff700 1 --2- 172.21.15.33:0/115229681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 0x7fd9880c0c50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:57.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.743+0000 7fd98fdff700 1 -- 172.21.15.33:0/115229681 >> 172.21.15.33:0/115229681 conn(0x7fd98805e1b0 msgr2=0x7fd98805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:57.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.743+0000 7fd98fdff700 1 -- 172.21.15.33:0/115229681 shutdown_connections 2023-11-06T20:56:57.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.743+0000 7fd98fdff700 1 -- 172.21.15.33:0/115229681 wait complete. 2023-11-06T20:56:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.745+0000 7fd98fdff700 1 Processor -- start 2023-11-06T20:56:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.745+0000 7fd98fdff700 1 -- start start 2023-11-06T20:56:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98fdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd988151dc0 0x7fd9880a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98fdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 0x7fd988150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98fdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd988155320 0x7fd98805e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98fdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880aa340 con 0x7fd9881521a0 2023-11-06T20:56:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98fdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880ade50 con 0x7fd988155320 2023-11-06T20:56:57.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98fdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880c3d00 con 0x7fd988151dc0 2023-11-06T20:56:57.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98d39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 0x7fd988150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:57.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98d39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 0x7fd988150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48026/0 (socket says 172.21.15.33:48026) 2023-11-06T20:56:57.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98d39a700 1 -- 172.21.15.33:0/1417256290 learned_addr learned my addr 172.21.15.33:0/1417256290 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:57.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98e39c700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd988155320 0x7fd98805e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98db9b700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd988151dc0 0x7fd9880a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98d39a700 1 -- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd988151dc0 msgr2=0x7fd9880a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98d39a700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd988151dc0 0x7fd9880a9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98d39a700 1 -- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd988155320 msgr2=0x7fd98805e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98d39a700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd988155320 0x7fd98805e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98d39a700 1 -- 172.21.15.33:0/1417256290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780092a0 con 0x7fd9881521a0 2023-11-06T20:56:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.746+0000 7fd98e39c700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd988155320 0x7fd98805e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.747+0000 7fd98d39a700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 0x7fd988150ca0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fd97800b340 tx=0x7fd978017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.747+0000 7fd97dffb700 1 -- 172.21.15.33:0/1417256290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800ba60 con 0x7fd9881521a0 2023-11-06T20:56:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.747+0000 7fd97dffb700 1 -- 172.21.15.33:0/1417256290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd978000c90 con 0x7fd9881521a0 2023-11-06T20:56:57.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.747+0000 7fd98fdff700 1 -- 172.21.15.33:0/1417256290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9880c3fe0 con 0x7fd9881521a0 2023-11-06T20:56:57.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.747+0000 7fd97dffb700 1 -- 172.21.15.33:0/1417256290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97801b640 con 0x7fd9881521a0 2023-11-06T20:56:57.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.747+0000 7fd98fdff700 1 -- 172.21.15.33:0/1417256290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9880ae3d0 con 0x7fd9881521a0 2023-11-06T20:56:57.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.748+0000 7fd98fdff700 1 -- 172.21.15.33:0/1417256290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9880ae740 con 0x7fd9881521a0 2023-11-06T20:56:57.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.749+0000 7fd97dffb700 1 -- 172.21.15.33:0/1417256290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd97801ba70 con 0x7fd9881521a0 2023-11-06T20:56:57.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.749+0000 7fd97dffb700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f240 0x7fd96c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:57.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.750+0000 7fd97dffb700 1 -- 172.21.15.33:0/1417256290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fd97807bdf0 con 0x7fd9881521a0 2023-11-06T20:56:57.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.750+0000 7fd98db9b700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f240 0x7fd96c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:57.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.751+0000 7fd98db9b700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f240 0x7fd96c061700 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fd9840016a0 tx=0x7fd984009040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:57.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.754+0000 7fd97dffb700 1 -- 172.21.15.33:0/1417256290 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd97804d1d0 con 0x7fd9881521a0 2023-11-06T20:56:57.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.998+0000 7fd98fdff700 1 -- 172.21.15.33:0/1417256290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd9880c0a30 con 0x7fd9881521a0 2023-11-06T20:56:57.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:57.998+0000 7fd97dffb700 1 -- 172.21.15.33:0/1417256290 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd978028070 con 0x7fd9881521a0 2023-11-06T20:56:57.998 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:57.998 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T20:56:57.999 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T20:56:57.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:57.999 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T20:56:57.999 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T20:56:58.000 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.000 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:56:58.000 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:56:58.000 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:56:58.000 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.001 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T20:56:58.001 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T20:56:58.001 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.002 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:56:58.002 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:56:58.002 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:56:58.002 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.002 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.003 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T20:56:58.003 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T20:56:58.003 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.004 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:56:58.004 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:56:58.004 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:56:58.004 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.005 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T20:56:58.005 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T20:56:58.005 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.006 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:56:58.006 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:56:58.006 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:56:58.006 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.006 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.007 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T20:56:58.007 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T20:56:58.008 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.008 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:56:58.008 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:56:58.008 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.009 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T20:56:58.009 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T20:56:58.010 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.010 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:56:58.010 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:56:58.010 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.011 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T20:56:58.011 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T20:56:58.011 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.011 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:56:58.012 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:56:58.012 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.013 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T20:56:58.013 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T20:56:58.013 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.013 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:56:58.013 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:56:58.014 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.014 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T20:56:58.015 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T20:56:58.015 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.015 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:56:58.015 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:56:58.015 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.016 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T20:56:58.016 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T20:56:58.017 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.017 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:56:58.017 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:56:58.017 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.018 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T20:56:58.018 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T20:56:58.018 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.019 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:56:58.019 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:56:58.019 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.020 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T20:56:58.020 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T20:56:58.020 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.020 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:56:58.021 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:56:58.021 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.022 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T20:56:58.022 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T20:56:58.022 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.022 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:56:58.022 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-11-06T20:56:58.023 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T20:56:58.023 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.023 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T20:56:58.023 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T20:56:58.024 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.024 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:56:58.024 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:56:58.024 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:56:58.024 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:56:58.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.025 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T20:56:58.025 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T20:56:58.025 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.026 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:56:58.026 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:56:58.026 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:56:58.026 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:56:58.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.027 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T20:56:58.027 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T20:56:58.027 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.028 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:56:58.028 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:56:58.028 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:56:58.028 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:56:58.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.029 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T20:56:58.029 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T20:56:58.029 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.029 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:56:58.030 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:56:58.030 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:56:58.030 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.031 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T20:56:58.031 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T20:56:58.031 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.031 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:56:58.032 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T20:56:58.032 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:58.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:58.033 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T20:56:58.033 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T20:56:58.033 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:56:58.033 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:56:58.033 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T20:56:58.034 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:56:58.034 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.034 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:58.034 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T20:56:58.034 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T20:56:58.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 -- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f240 msgr2=0x7fd96c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f240 0x7fd96c061700 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fd9840016a0 tx=0x7fd984009040).stop 2023-11-06T20:56:58.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 -- 172.21.15.33:0/1417256290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 msgr2=0x7fd988150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 0x7fd988150ca0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fd97800b340 tx=0x7fd978017000).stop 2023-11-06T20:56:58.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 -- 172.21.15.33:0/1417256290 shutdown_connections 2023-11-06T20:56:58.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd96c05f240 0x7fd96c061700 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd988151dc0 0x7fd9880a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd988155320 0x7fd98805e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 --2- 172.21.15.33:0/1417256290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9881521a0 0x7fd988150ca0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 -- 172.21.15.33:0/1417256290 >> 172.21.15.33:0/1417256290 conn(0x7fd98805e1b0 msgr2=0x7fd9880c7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:58.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 -- 172.21.15.33:0/1417256290 shutdown_connections 2023-11-06T20:56:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.001+0000 7fd98fdff700 1 -- 172.21.15.33:0/1417256290 wait complete. 2023-11-06T20:56:58.041 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:56:58.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.240+0000 7f4ac63ff700 1 Processor -- start 2023-11-06T20:56:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.240+0000 7f4ac63ff700 1 -- start start 2023-11-06T20:56:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.242+0000 7f4ac63ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac00c4e60 0x7f4ac0066540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.242+0000 7f4ac63ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac01553b0 con 0x7f4ac00be280 2023-11-06T20:56:58.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.242+0000 7f4ac63ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac0154ac0 con 0x7f4ac0066a30 2023-11-06T20:56:58.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.242+0000 7f4ac63ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac0154d60 con 0x7f4ac00c4e60 2023-11-06T20:56:58.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.242+0000 7f4ac499c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4ac00be280 0x7f4ac00c35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48186/0 (socket says 172.21.15.33:48186) 2023-11-06T20:56:58.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.242+0000 7f4ac499c700 1 -- 172.21.15.33:0/3717138280 learned_addr learned my addr 172.21.15.33:0/3717138280 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:58.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.243+0000 7f4abffff700 1 --2- 172.21.15.33:0/3717138280 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac00c4e60 0x7f4ac0066540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.243+0000 7f4abeffd700 1 -- 172.21.15.33:0/3717138280 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895970121 0 0) 0x7f4ac01553b0 con 0x7f4ac00be280 2023-11-06T20:56:58.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.243+0000 7f4abeffd700 1 -- 172.21.15.33:0/3717138280 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aa80033f0 con 0x7f4ac00be280 2023-11-06T20:56:58.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.243+0000 7f4abeffd700 1 -- 172.21.15.33:0/3717138280 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479443019 0 0) 0x7f4ac0154ac0 con 0x7f4ac0066a30 2023-11-06T20:56:58.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.243+0000 7f4abeffd700 1 -- 172.21.15.33:0/3717138280 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ac01553b0 con 0x7f4ac0066a30 2023-11-06T20:56:58.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.243+0000 7f4abffff700 1 -- 172.21.15.33:0/3717138280 >> v1:172.21.15.69:6789/0 conn(0x7f4ac0066a30 legacy=0x7f4ac00bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.243+0000 7f4abffff700 1 -- 172.21.15.33:0/3717138280 >> v1:172.21.15.33:6789/0 conn(0x7f4ac00be280 legacy=0x7f4ac00c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.243+0000 7f4abeffd700 1 -- 172.21.15.33:0/3717138280 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1827853263 0 0) 0x7f4aa80033f0 con 0x7f4ac00be280 2023-11-06T20:56:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.243+0000 7f4abffff700 1 -- 172.21.15.33:0/3717138280 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac0155040 con 0x7f4ac00c4e60 2023-11-06T20:56:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.243+0000 7f4abffff700 1 --2- 172.21.15.33:0/3717138280 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac00c4e60 0x7f4ac0066540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f4ab0009790 tx=0x7f4ab0009060).ready entity=mon.2 client_cookie=2adec4e563270a1d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.244+0000 7f4abeffd700 1 -- 172.21.15.33:0/3717138280 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80017f0 con 0x7f4ac00c4e60 2023-11-06T20:56:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.244+0000 7f4abeffd700 1 -- 172.21.15.33:0/3717138280 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab0011400 con 0x7f4ac00c4e60 2023-11-06T20:56:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.244+0000 7f4abeffd700 1 -- 172.21.15.33:0/3717138280 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0011780 con 0x7f4ac00c4e60 2023-11-06T20:56:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.244+0000 7f4ac63ff700 1 -- 172.21.15.33:0/3717138280 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac00c4e60 msgr2=0x7f4ac0066540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.244+0000 7f4ac63ff700 1 --2- 172.21.15.33:0/3717138280 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac00c4e60 0x7f4ac0066540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f4ab0009790 tx=0x7f4ab0009060).stop 2023-11-06T20:56:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.244+0000 7f4ac63ff700 1 -- 172.21.15.33:0/3717138280 shutdown_connections 2023-11-06T20:56:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.244+0000 7f4ac63ff700 1 --2- 172.21.15.33:0/3717138280 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac00c4e60 0x7f4ac0066540 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.244+0000 7f4ac63ff700 1 -- 172.21.15.33:0/3717138280 >> 172.21.15.33:0/3717138280 conn(0x7f4ac00c8c10 msgr2=0x7f4ac00c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:58.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.244+0000 7f4ac63ff700 1 -- 172.21.15.33:0/3717138280 shutdown_connections 2023-11-06T20:56:58.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.244+0000 7f4ac63ff700 1 -- 172.21.15.33:0/3717138280 wait complete. 2023-11-06T20:56:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.246+0000 7f4ac63ff700 1 Processor -- start 2023-11-06T20:56:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.246+0000 7f4ac63ff700 1 -- start start 2023-11-06T20:56:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4ac63ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac0066a30 0x7f4ac0139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4ac63ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac00be280 0x7f4ac00a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4ac63ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac00c4e60 0x7f4ac0064c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4ac63ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac00a50b0 con 0x7f4ac00c4e60 2023-11-06T20:56:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4ac63ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac0138f90 con 0x7f4ac00be280 2023-11-06T20:56:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4ac63ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac00ac850 con 0x7f4ac0066a30 2023-11-06T20:56:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4ac499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac00c4e60 0x7f4ac0064c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4abffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac0066a30 0x7f4ac0139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4ac499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac00c4e60 0x7f4ac0064c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43730/0 (socket says 172.21.15.33:43730) 2023-11-06T20:56:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4ac499c700 1 -- 172.21.15.33:0/2126753029 learned_addr learned my addr 172.21.15.33:0/2126753029 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.247+0000 7f4abf7fe700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac00be280 0x7f4ac00a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4ac499c700 1 -- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac0066a30 msgr2=0x7f4ac0139670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4ac499c700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac0066a30 0x7f4ac0139670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4ac499c700 1 -- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac00be280 msgr2=0x7f4ac00a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4ac499c700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac00be280 0x7f4ac00a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4ac499c700 1 -- 172.21.15.33:0/2126753029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00092a0 con 0x7f4ac00c4e60 2023-11-06T20:56:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4ac499c700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac00c4e60 0x7f4ac0064c00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f4ab800cb90 tx=0x7f4ab8001e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4aa7fff700 1 -- 172.21.15.33:0/2126753029 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8004c40 con 0x7f4ac00c4e60 2023-11-06T20:56:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4aa7fff700 1 -- 172.21.15.33:0/2126753029 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab8004da0 con 0x7f4ac00c4e60 2023-11-06T20:56:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4ac63ff700 1 -- 172.21.15.33:0/2126753029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac00abc40 con 0x7f4ac00c4e60 2023-11-06T20:56:58.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4aa7fff700 1 -- 172.21.15.33:0/2126753029 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80073a0 con 0x7f4ac00c4e60 2023-11-06T20:56:58.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.248+0000 7f4ac63ff700 1 -- 172.21.15.33:0/2126753029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac00ac200 con 0x7f4ac00c4e60 2023-11-06T20:56:58.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.250+0000 7f4ac63ff700 1 -- 172.21.15.33:0/2126753029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac00ac9c0 con 0x7f4ac00c4e60 2023-11-06T20:56:58.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.250+0000 7f4aa7fff700 1 -- 172.21.15.33:0/2126753029 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4ab8007750 con 0x7f4ac00c4e60 2023-11-06T20:56:58.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.251+0000 7f4aa7fff700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4aac05f270 0x7f4aac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.251+0000 7f4aa7fff700 1 -- 172.21.15.33:0/2126753029 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f4ab8079f00 con 0x7f4ac00c4e60 2023-11-06T20:56:58.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.252+0000 7f4abffff700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4aac05f270 0x7f4aac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.252+0000 7f4abffff700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4aac05f270 0x7f4aac061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4ab00096e0 tx=0x7f4ab0016000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.255+0000 7f4aa7fff700 1 -- 172.21.15.33:0/2126753029 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4ab804b3a0 con 0x7f4ac00c4e60 2023-11-06T20:56:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.437+0000 7f4ac63ff700 1 -- 172.21.15.33:0/2126753029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ac00c6130 con 0x7f4ac00c4e60 2023-11-06T20:56:58.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.438+0000 7f4aa7fff700 1 -- 172.21.15.33:0/2126753029 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+13760 (secure 0 0 0) 0x7f4ab804e9c0 con 0x7f4ac00c4e60 2023-11-06T20:56:58.438 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:58.438 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:56:57.512494+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:56:58.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 -- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4aac05f270 msgr2=0x7f4aac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4aac05f270 0x7f4aac061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4ab00096e0 tx=0x7f4ab0016000).stop 2023-11-06T20:56:58.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 -- 172.21.15.33:0/2126753029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac00c4e60 msgr2=0x7f4ac0064c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac00c4e60 0x7f4ac0064c00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f4ab800cb90 tx=0x7f4ab8001e80).stop 2023-11-06T20:56:58.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 -- 172.21.15.33:0/2126753029 shutdown_connections 2023-11-06T20:56:58.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4aac05f270 0x7f4aac061730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ac0066a30 0x7f4ac0139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac00c4e60 0x7f4ac0064c00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 --2- 172.21.15.33:0/2126753029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac00be280 0x7f4ac00a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 -- 172.21.15.33:0/2126753029 >> 172.21.15.33:0/2126753029 conn(0x7f4ac00c8c10 msgr2=0x7f4ac0152380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:58.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 -- 172.21.15.33:0/2126753029 shutdown_connections 2023-11-06T20:56:58.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.440+0000 7f4ac63ff700 1 -- 172.21.15.33:0/2126753029 wait complete. 2023-11-06T20:56:58.457 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T20:56:58.458 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:56:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.658+0000 7fbe6fdff700 1 Processor -- start 2023-11-06T20:56:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.658+0000 7fbe6fdff700 1 -- start start 2023-11-06T20:56:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.659+0000 7fbe6fdff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbe680aafc0 0x7fbe681543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.659+0000 7fbe6fdff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbe68064670 0x7fbe68064b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.659+0000 7fbe6fdff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe68065010 con 0x7fbe68154010 2023-11-06T20:56:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.659+0000 7fbe6fdff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe680c6280 con 0x7fbe680aafc0 2023-11-06T20:56:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.659+0000 7fbe6fdff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe680c0d90 con 0x7fbe68064670 2023-11-06T20:56:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.660+0000 7fbe6d39a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbe680aafc0 0x7fbe681543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.660+0000 7fbe6d39a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbe680aafc0 0x7fbe681543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43744/0 (socket says 172.21.15.33:43744) 2023-11-06T20:56:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.660+0000 7fbe6d39a700 1 -- 172.21.15.33:0/1240949917 learned_addr learned my addr 172.21.15.33:0/1240949917 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.660+0000 7fbe6e39c700 1 --2- 172.21.15.33:0/1240949917 >> v2:172.21.15.69:3300/0 conn(0x7fbe68064670 0x7fbe68064b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.660+0000 7fbe6d39a700 1 -- 172.21.15.33:0/1240949917 >> v2:172.21.15.69:3300/0 conn(0x7fbe68064670 msgr2=0x7fbe68064b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.660+0000 7fbe6d39a700 1 --2- 172.21.15.33:0/1240949917 >> v2:172.21.15.69:3300/0 conn(0x7fbe68064670 0x7fbe68064b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.660+0000 7fbe6d39a700 1 -- 172.21.15.33:0/1240949917 >> v1:172.21.15.69:6789/0 conn(0x7fbe68154010 legacy=0x7fbe680c0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.660+0000 7fbe6d39a700 1 -- 172.21.15.33:0/1240949917 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe680c7720 con 0x7fbe680aafc0 2023-11-06T20:56:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.660+0000 7fbe6d39a700 1 --2- 172.21.15.33:0/1240949917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe680aafc0 0x7fbe681543f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fbe680aa900 tx=0x7fbe580090f0).ready entity=mon.0 client_cookie=fb0ffa48bfb587c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.661+0000 7fbe6cb99700 1 -- 172.21.15.33:0/1240949917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800b520 con 0x7fbe680aafc0 2023-11-06T20:56:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.661+0000 7fbe6cb99700 1 -- 172.21.15.33:0/1240949917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5800b680 con 0x7fbe680aafc0 2023-11-06T20:56:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.661+0000 7fbe6cb99700 1 -- 172.21.15.33:0/1240949917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800ba00 con 0x7fbe680aafc0 2023-11-06T20:56:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.661+0000 7fbe6fdff700 1 -- 172.21.15.33:0/1240949917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe680aafc0 msgr2=0x7fbe681543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.661+0000 7fbe6fdff700 1 --2- 172.21.15.33:0/1240949917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe680aafc0 0x7fbe681543f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fbe680aa900 tx=0x7fbe580090f0).stop 2023-11-06T20:56:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.661+0000 7fbe6fdff700 1 -- 172.21.15.33:0/1240949917 shutdown_connections 2023-11-06T20:56:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.661+0000 7fbe6fdff700 1 --2- 172.21.15.33:0/1240949917 >> v2:172.21.15.69:3300/0 conn(0x7fbe68064670 0x7fbe68064b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.661+0000 7fbe6fdff700 1 --2- 172.21.15.33:0/1240949917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe680aafc0 0x7fbe681543f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.661+0000 7fbe6fdff700 1 -- 172.21.15.33:0/1240949917 >> 172.21.15.33:0/1240949917 conn(0x7fbe680cb8e0 msgr2=0x7fbe680c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.661+0000 7fbe6fdff700 1 -- 172.21.15.33:0/1240949917 shutdown_connections 2023-11-06T20:56:58.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.662+0000 7fbe6fdff700 1 -- 172.21.15.33:0/1240949917 wait complete. 2023-11-06T20:56:58.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.663+0000 7fbe6fdff700 1 Processor -- start 2023-11-06T20:56:58.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.663+0000 7fbe6fdff700 1 -- start start 2023-11-06T20:56:58.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.664+0000 7fbe6fdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbe68064670 0x7fbe681505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.664+0000 7fbe6fdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe68154010 0x7fbe68150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.664+0000 7fbe6fdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe680b07e0 0x7fbe68151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.664+0000 7fbe6fdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe68151970 con 0x7fbe68154010 2023-11-06T20:56:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.664+0000 7fbe6fdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe68154aa0 con 0x7fbe680b07e0 2023-11-06T20:56:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.664+0000 7fbe6fdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe680adde0 con 0x7fbe68064670 2023-11-06T20:56:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.664+0000 7fbe6d39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe68154010 0x7fbe68150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.664+0000 7fbe6d39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe68154010 0x7fbe68150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43756/0 (socket says 172.21.15.33:43756) 2023-11-06T20:56:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.664+0000 7fbe6d39a700 1 -- 172.21.15.33:0/3378952760 learned_addr learned my addr 172.21.15.33:0/3378952760 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6e39c700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe680b07e0 0x7fbe68151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6db9b700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbe68064670 0x7fbe681505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6d39a700 1 -- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbe68064670 msgr2=0x7fbe681505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6d39a700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbe68064670 0x7fbe681505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6d39a700 1 -- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe680b07e0 msgr2=0x7fbe68151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6d39a700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe680b07e0 0x7fbe68151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6d39a700 1 -- 172.21.15.33:0/3378952760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58009370 con 0x7fbe68154010 2023-11-06T20:56:58.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6db9b700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbe68064670 0x7fbe681505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:58.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6e39c700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe680b07e0 0x7fbe68151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:56:58.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6d39a700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe68154010 0x7fbe68150f90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbe68062d70 tx=0x7fbe58017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:58.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe5dffb700 1 -- 172.21.15.33:0/3378952760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800b7f0 con 0x7fbe68154010 2023-11-06T20:56:58.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe5dffb700 1 -- 172.21.15.33:0/3378952760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5800b950 con 0x7fbe68154010 2023-11-06T20:56:58.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe5dffb700 1 -- 172.21.15.33:0/3378952760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5801c650 con 0x7fbe68154010 2023-11-06T20:56:58.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6fdff700 1 -- 172.21.15.33:0/3378952760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe680ae030 con 0x7fbe68154010 2023-11-06T20:56:58.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.665+0000 7fbe6fdff700 1 -- 172.21.15.33:0/3378952760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe680ae5f0 con 0x7fbe68154010 2023-11-06T20:56:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.666+0000 7fbe6fdff700 1 -- 172.21.15.33:0/3378952760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe68063f60 con 0x7fbe68154010 2023-11-06T20:56:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.667+0000 7fbe5dffb700 1 -- 172.21.15.33:0/3378952760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbe5801ca00 con 0x7fbe68154010 2023-11-06T20:56:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.668+0000 7fbe5dffb700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe4c05f240 0x7fbe4c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.668+0000 7fbe5dffb700 1 -- 172.21.15.33:0/3378952760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fbe5807be20 con 0x7fbe68154010 2023-11-06T20:56:58.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.669+0000 7fbe6db9b700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe4c05f240 0x7fbe4c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:58.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.669+0000 7fbe6db9b700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe4c05f240 0x7fbe4c061700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fbe64000c00 tx=0x7fbe64009040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:58.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.672+0000 7fbe5dffb700 1 -- 172.21.15.33:0/3378952760 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbe68063f60 con 0x7fbe68154010 2023-11-06T20:56:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.852+0000 7fbe6fdff700 1 -- 172.21.15.33:0/3378952760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbe68063f60 con 0x7fbe68154010 2023-11-06T20:56:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.853+0000 7fbe5dffb700 1 -- 172.21.15.33:0/3378952760 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v62) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbe68063f60 con 0x7fbe68154010 2023-11-06T20:56:58.852 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:56:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 -- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe4c05f240 msgr2=0x7fbe4c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe4c05f240 0x7fbe4c061700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fbe64000c00 tx=0x7fbe64009040).stop 2023-11-06T20:56:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 -- 172.21.15.33:0/3378952760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe68154010 msgr2=0x7fbe68150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe68154010 0x7fbe68150f90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbe68062d70 tx=0x7fbe58017000).stop 2023-11-06T20:56:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 -- 172.21.15.33:0/3378952760 shutdown_connections 2023-11-06T20:56:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbe4c05f240 0x7fbe4c061700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbe68064670 0x7fbe681505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe680b07e0 0x7fbe68151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 --2- 172.21.15.33:0/3378952760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe68154010 0x7fbe68150f90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 -- 172.21.15.33:0/3378952760 >> 172.21.15.33:0/3378952760 conn(0x7fbe680cb8e0 msgr2=0x7fbe680ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:58.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 -- 172.21.15.33:0/3378952760 shutdown_connections 2023-11-06T20:56:58.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:58.855+0000 7fbe6fdff700 1 -- 172.21.15.33:0/3378952760 wait complete. 2023-11-06T20:56:58.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:56:59.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.071+0000 7fd0c57ff700 1 Processor -- start 2023-11-06T20:56:59.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.071+0000 7fd0c57ff700 1 -- start start 2023-11-06T20:56:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.072+0000 7fd0c57ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd0c014faa0 0x7fd0c0064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.072+0000 7fd0c57ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00c3450 0x7fd0c00c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.072+0000 7fd0c57ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c0064270 con 0x7fd0c014fe80 2023-11-06T20:56:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.072+0000 7fd0c57ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c00c3dd0 con 0x7fd0c014faa0 2023-11-06T20:56:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.072+0000 7fd0c57ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00651b0 con 0x7fd0c00c3450 2023-11-06T20:56:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.072+0000 7fd0be7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd0c014fe80 0x7fd0c0152750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48192/0 (socket says 172.21.15.33:48192) 2023-11-06T20:56:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.072+0000 7fd0be7fc700 1 -- 172.21.15.33:0/3943956066 learned_addr learned my addr 172.21.15.33:0/3943956066 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:59.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0beffd700 1 --2- 172.21.15.33:0/3943956066 >> v2:172.21.15.69:3300/0 conn(0x7fd0c014faa0 0x7fd0c0064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bf7fe700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00c3450 0x7fd0c00c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640524611 0 0) 0x7fd0c0064270 con 0x7fd0c014fe80 2023-11-06T20:56:59.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0a80033f0 con 0x7fd0c014fe80 2023-11-06T20:56:59.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1092117727 0 0) 0x7fd0a80033f0 con 0x7fd0c014fe80 2023-11-06T20:56:59.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00c3450 msgr2=0x7fd0c00c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00c3450 0x7fd0c00c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 >> v2:172.21.15.69:3300/0 conn(0x7fd0c014faa0 msgr2=0x7fd0c0064cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 --2- 172.21.15.33:0/3943956066 >> v2:172.21.15.69:3300/0 conn(0x7fd0c014faa0 0x7fd0c0064cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c01531f0 con 0x7fd0c014fe80 2023-11-06T20:56:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fd0b4001470 con 0x7fd0c014fe80 2023-11-06T20:56:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 >> v1:172.21.15.33:6789/0 conn(0x7fd0c014fe80 legacy=0x7fd0c0152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0a80038e0 0x7fd0a8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0a8004690 0x7fd0a8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a8005000 0x7fd0a800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00c3dd0 con 0x7fd0a8004690 2023-11-06T20:56:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00651b0 con 0x7fd0a8005000 2023-11-06T20:56:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a800bdd0 con 0x7fd0a80038e0 2023-11-06T20:56:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.073+0000 7fd0bf7fe700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00c3450 0x7fd0c00c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:56:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0be7fc700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0a8004690 0x7fd0a8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0beffd700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0a80038e0 0x7fd0a8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0bf7fe700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a8005000 0x7fd0a800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0be7fc700 1 -- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0a80038e0 msgr2=0x7fd0a8003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0be7fc700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0a80038e0 0x7fd0a8003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0be7fc700 1 -- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a8005000 msgr2=0x7fd0a800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:59.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0be7fc700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a8005000 0x7fd0a800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0be7fc700 1 -- 172.21.15.33:0/3943956066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a80033f0 con 0x7fd0a8004690 2023-11-06T20:56:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0be7fc700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0a8004690 0x7fd0a8004b10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fd0b400c210 tx=0x7fd0b4001e90).ready entity=mon.0 client_cookie=9618faf9aba8dfc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4007a60 con 0x7fd0a8004690 2023-11-06T20:56:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b400caa0 con 0x7fd0a8004690 2023-11-06T20:56:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.074+0000 7fd0bdffb700 1 -- 172.21.15.33:0/3943956066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b400ce40 con 0x7fd0a8004690 2023-11-06T20:56:59.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.075+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3943956066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0a8004690 msgr2=0x7fd0a8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.075+0000 7fd0c57ff700 1 --2- 172.21.15.33:0/3943956066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0a8004690 0x7fd0a8004b10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fd0b400c210 tx=0x7fd0b4001e90).stop 2023-11-06T20:56:59.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.075+0000 7fd0bf7fe700 1 -- 172.21.15.33:0/3943956066 reap_dead start 2023-11-06T20:56:59.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.075+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3943956066 shutdown_connections 2023-11-06T20:56:59.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.075+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3943956066 >> 172.21.15.33:0/3943956066 conn(0x7fd0c0062710 msgr2=0x7fd0c0062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:59.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.075+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3943956066 shutdown_connections 2023-11-06T20:56:59.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.075+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3943956066 wait complete. 2023-11-06T20:56:59.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.077+0000 7fd0c57ff700 1 Processor -- start 2023-11-06T20:56:59.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.077+0000 7fd0c57ff700 1 -- start start 2023-11-06T20:56:59.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0c57ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a8004690 0x7fd0c00ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0c57ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00a6580 0x7fd0c0141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0c57ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c0141a40 0x7fd0c00c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0c57ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00a7630 con 0x7fd0c0141a40 2023-11-06T20:56:59.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0c57ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00a6a20 con 0x7fd0a8004690 2023-11-06T20:56:59.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0c57ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00a6d00 con 0x7fd0c00a6580 2023-11-06T20:56:59.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c0141a40 0x7fd0c00c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c0141a40 0x7fd0c00c49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43776/0 (socket says 172.21.15.33:43776) 2023-11-06T20:56:59.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0bf7fe700 1 -- 172.21.15.33:0/3519523627 learned_addr learned my addr 172.21.15.33:0/3519523627 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:59.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0be7fc700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00a6580 0x7fd0c0141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0beffd700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a8004690 0x7fd0c00ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0bf7fe700 1 -- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00a6580 msgr2=0x7fd0c0141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0bf7fe700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00a6580 0x7fd0c0141550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0bf7fe700 1 -- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a8004690 msgr2=0x7fd0c00ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0bf7fe700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a8004690 0x7fd0c00ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0bf7fe700 1 -- 172.21.15.33:0/3519523627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c00a6fe0 con 0x7fd0c0141a40 2023-11-06T20:56:59.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0be7fc700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00a6580 0x7fd0c0141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:59.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.078+0000 7fd0beffd700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a8004690 0x7fd0c00ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:59.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.079+0000 7fd0bf7fe700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c0141a40 0x7fd0c00c49c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fd0c00c0f70 tx=0x7fd0b800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:59.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.079+0000 7fd0a6ffd700 1 -- 172.21.15.33:0/3519523627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b801a070 con 0x7fd0c0141a40 2023-11-06T20:56:59.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.079+0000 7fd0a6ffd700 1 -- 172.21.15.33:0/3519523627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b800f040 con 0x7fd0c0141a40 2023-11-06T20:56:59.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.079+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3519523627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c00a72c0 con 0x7fd0c0141a40 2023-11-06T20:56:59.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.079+0000 7fd0a6ffd700 1 -- 172.21.15.33:0/3519523627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800e040 con 0x7fd0c0141a40 2023-11-06T20:56:59.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.079+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3519523627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c00c4eb0 con 0x7fd0c0141a40 2023-11-06T20:56:59.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.080+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3519523627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c00bd0a0 con 0x7fd0c0141a40 2023-11-06T20:56:59.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.081+0000 7fd0a6ffd700 1 -- 172.21.15.33:0/3519523627 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd0b8003020 con 0x7fd0c0141a40 2023-11-06T20:56:59.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.082+0000 7fd0a6ffd700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd0ac05f2a0 0x7fd0ac061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.082+0000 7fd0a6ffd700 1 -- 172.21.15.33:0/3519523627 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fd0b807a570 con 0x7fd0c0141a40 2023-11-06T20:56:59.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.082+0000 7fd0beffd700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd0ac05f2a0 0x7fd0ac061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.083+0000 7fd0beffd700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd0ac05f2a0 0x7fd0ac061760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd0b0009270 tx=0x7fd0b0008040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:59.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.086+0000 7fd0a6ffd700 1 -- 172.21.15.33:0/3519523627 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd0b804f380 con 0x7fd0c0141a40 2023-11-06T20:56:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.266+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3519523627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd0c01534d0 con 0x7fd0c0141a40 2023-11-06T20:56:59.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.266+0000 7fd0a6ffd700 1 -- 172.21.15.33:0/3519523627 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd0b804ef70 con 0x7fd0c0141a40 2023-11-06T20:56:59.266 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:56:59.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd0ac05f2a0 msgr2=0x7fd0ac061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd0ac05f2a0 0x7fd0ac061760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd0b0009270 tx=0x7fd0b0008040).stop 2023-11-06T20:56:59.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3519523627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c0141a40 msgr2=0x7fd0c00c49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c0141a40 0x7fd0c00c49c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fd0c00c0f70 tx=0x7fd0b800b040).stop 2023-11-06T20:56:59.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3519523627 shutdown_connections 2023-11-06T20:56:59.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd0ac05f2a0 0x7fd0ac061760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c0141a40 0x7fd0c00c49c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0a8004690 0x7fd0c00ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 --2- 172.21.15.33:0/3519523627 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd0c00a6580 0x7fd0c0141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3519523627 >> 172.21.15.33:0/3519523627 conn(0x7fd0c0062710 msgr2=0x7fd0c0068160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:59.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3519523627 shutdown_connections 2023-11-06T20:56:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.268+0000 7fd0c57ff700 1 -- 172.21.15.33:0/3519523627 wait complete. 2023-11-06T20:56:59.285 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T20:56:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.482+0000 7f88ec7ff700 1 Processor -- start 2023-11-06T20:56:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.482+0000 7f88ec7ff700 1 -- start start 2023-11-06T20:56:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.483+0000 7f88ec7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 0x7f88e40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.483+0000 7f88ec7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f88e40aa2d0 0x7f88e40c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.483+0000 7f88ec7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e41554a0 con 0x7f88e40c6240 2023-11-06T20:56:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.483+0000 7f88ec7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e4154960 con 0x7f88e40aa2d0 2023-11-06T20:56:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.483+0000 7f88ec7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4155930 con 0x7f88e40c1840 2023-11-06T20:56:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.484+0000 7f88e9d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f88e40aa2d0 0x7f88e40c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.484+0000 7f88ead9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f88e40c6240 0x7f88e4157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48198/0 (socket says 172.21.15.33:48198) 2023-11-06T20:56:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.484+0000 7f88ead9c700 1 -- 172.21.15.33:0/2726989620 learned_addr learned my addr 172.21.15.33:0/2726989620 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.484+0000 7f88ea59b700 1 --2- 172.21.15.33:0/2726989620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 0x7f88e40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.484+0000 7f88e9d9a700 1 -- 172.21.15.33:0/2726989620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 msgr2=0x7f88e40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.484+0000 7f88e9d9a700 1 --2- 172.21.15.33:0/2726989620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 0x7f88e40a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.484+0000 7f88e9d9a700 1 -- 172.21.15.33:0/2726989620 >> v1:172.21.15.33:6789/0 conn(0x7f88e40c6240 legacy=0x7f88e4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.484+0000 7f88e9d9a700 1 -- 172.21.15.33:0/2726989620 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e4154b70 con 0x7f88e40aa2d0 2023-11-06T20:56:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.484+0000 7f88e9599700 1 -- 172.21.15.33:0/2726989620 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012863044 0 0) 0x7f88e41554a0 con 0x7f88e40c6240 2023-11-06T20:56:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.484+0000 7f88e9d9a700 1 --2- 172.21.15.33:0/2726989620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 0x7f88e40c5d50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f88d4009760 tx=0x7f88d4009060).ready entity=mon.0 client_cookie=a48967b75fcd55dc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:59.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88e9599700 1 -- 172.21.15.33:0/2726989620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d400b580 con 0x7f88e40aa2d0 2023-11-06T20:56:59.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88e9599700 1 -- 172.21.15.33:0/2726989620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d400b6e0 con 0x7f88e40aa2d0 2023-11-06T20:56:59.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88e9599700 1 -- 172.21.15.33:0/2726989620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d400ba60 con 0x7f88e40aa2d0 2023-11-06T20:56:59.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88ec7ff700 1 -- 172.21.15.33:0/2726989620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 msgr2=0x7f88e40c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88ec7ff700 1 --2- 172.21.15.33:0/2726989620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 0x7f88e40c5d50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f88d4009760 tx=0x7f88d4009060).stop 2023-11-06T20:56:59.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88ec7ff700 1 -- 172.21.15.33:0/2726989620 shutdown_connections 2023-11-06T20:56:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88ec7ff700 1 --2- 172.21.15.33:0/2726989620 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 0x7f88e40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88ec7ff700 1 --2- 172.21.15.33:0/2726989620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 0x7f88e40c5d50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88ec7ff700 1 -- 172.21.15.33:0/2726989620 >> 172.21.15.33:0/2726989620 conn(0x7f88e405ec80 msgr2=0x7f88e40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88ec7ff700 1 -- 172.21.15.33:0/2726989620 shutdown_connections 2023-11-06T20:56:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.485+0000 7f88ec7ff700 1 -- 172.21.15.33:0/2726989620 wait complete. 2023-11-06T20:56:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.487+0000 7f88ec7ff700 1 Processor -- start 2023-11-06T20:56:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.487+0000 7f88ec7ff700 1 -- start start 2023-11-06T20:56:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.488+0000 7f88ec7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 0x7f88e4149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.488+0000 7f88ec7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 0x7f88e4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.488+0000 7f88ec7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e40c6240 0x7f88e4159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.488+0000 7f88ec7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4159a50 con 0x7f88e40aa2d0 2023-11-06T20:56:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.488+0000 7f88ec7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4155b80 con 0x7f88e40c6240 2023-11-06T20:56:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.488+0000 7f88ec7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e414a2e0 con 0x7f88e40c1840 2023-11-06T20:56:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.488+0000 7f88ea59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 0x7f88e4149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.488+0000 7f88ea59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 0x7f88e4149df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43792/0 (socket says 172.21.15.33:43792) 2023-11-06T20:56:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.488+0000 7f88ea59b700 1 -- 172.21.15.33:0/1855719472 learned_addr learned my addr 172.21.15.33:0/1855719472 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88e9d9a700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 0x7f88e4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88ea59b700 1 -- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 msgr2=0x7f88e4159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88ea59b700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 0x7f88e4159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88ea59b700 1 -- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e40c6240 msgr2=0x7f88e4159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88ea59b700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e40c6240 0x7f88e4159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88ea59b700 1 -- 172.21.15.33:0/1855719472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d40092a0 con 0x7f88e40aa2d0 2023-11-06T20:56:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88e9d9a700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 0x7f88e4159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:56:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88ea59b700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 0x7f88e4149df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f88e000d8c0 tx=0x7f88e0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88da7fc700 1 -- 172.21.15.33:0/1855719472 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e0009640 con 0x7f88e40aa2d0 2023-11-06T20:56:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88da7fc700 1 -- 172.21.15.33:0/1855719472 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e00097a0 con 0x7f88e40aa2d0 2023-11-06T20:56:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88da7fc700 1 -- 172.21.15.33:0/1855719472 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e0009b20 con 0x7f88e40aa2d0 2023-11-06T20:56:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88ec7ff700 1 -- 172.21.15.33:0/1855719472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e40c2220 con 0x7f88e40aa2d0 2023-11-06T20:56:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.489+0000 7f88ec7ff700 1 -- 172.21.15.33:0/1855719472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e40ad460 con 0x7f88e40aa2d0 2023-11-06T20:56:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.490+0000 7f88ec7ff700 1 -- 172.21.15.33:0/1855719472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e40c2500 con 0x7f88e40aa2d0 2023-11-06T20:56:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.491+0000 7f88da7fc700 1 -- 172.21.15.33:0/1855719472 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f88e0013480 con 0x7f88e40aa2d0 2023-11-06T20:56:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.492+0000 7f88da7fc700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f88c805f270 0x7f88c8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.492+0000 7f88da7fc700 1 -- 172.21.15.33:0/1855719472 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f88e0079fd0 con 0x7f88e40aa2d0 2023-11-06T20:56:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.492+0000 7f88e9d9a700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f88c805f270 0x7f88c8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.493+0000 7f88e9d9a700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f88c805f270 0x7f88c8061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f88e40cadc0 tx=0x7f88d4017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.496+0000 7f88da7fc700 1 -- 172.21.15.33:0/1855719472 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f88e004b350 con 0x7f88e40aa2d0 2023-11-06T20:56:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.677+0000 7f88ec7ff700 1 -- 172.21.15.33:0/1855719472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f88e406c6c0 con 0x7f88e40aa2d0 2023-11-06T20:56:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.677+0000 7f88da7fc700 1 -- 172.21.15.33:0/1855719472 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+143 (secure 0 0 0) 0x7f88e004e970 con 0x7f88e40aa2d0 2023-11-06T20:56:59.677 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:56:59.677 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T20:56:59.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:59.677 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T20:56:59.678 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T20:56:59.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:56:59.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:56:59.678 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T20:56:59.678 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T20:56:59.679 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:56:59.679 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T20:56:59.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 -- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f88c805f270 msgr2=0x7f88c8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f88c805f270 0x7f88c8061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f88e40cadc0 tx=0x7f88d4017000).stop 2023-11-06T20:56:59.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 -- 172.21.15.33:0/1855719472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 msgr2=0x7f88e4149df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 0x7f88e4149df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f88e000d8c0 tx=0x7f88e0008040).stop 2023-11-06T20:56:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 -- 172.21.15.33:0/1855719472 shutdown_connections 2023-11-06T20:56:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f88c805f270 0x7f88c8061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e40c6240 0x7f88e4159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:56:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88e40aa2d0 0x7f88e4149df0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 --2- 172.21.15.33:0/1855719472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88e40c1840 0x7f88e4159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 -- 172.21.15.33:0/1855719472 >> 172.21.15.33:0/1855719472 conn(0x7f88e405ec80 msgr2=0x7f88e4150f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.679+0000 7f88ec7ff700 1 -- 172.21.15.33:0/1855719472 shutdown_connections 2023-11-06T20:56:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.680+0000 7f88ec7ff700 1 -- 172.21.15.33:0/1855719472 wait complete. 2023-11-06T20:56:59.696 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T20:56:59.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T20:56:59.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.899+0000 7ff2327ff700 1 Processor -- start 2023-11-06T20:56:59.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.899+0000 7ff2327ff700 1 -- start start 2023-11-06T20:56:59.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.901+0000 7ff2327ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff22c0acfc0 0x7ff22c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.901+0000 7ff2327ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff22c0638b0 0x7ff22c0c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.901+0000 7ff2327ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22c1586a0 con 0x7ff22c1550d0 2023-11-06T20:56:59.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.901+0000 7ff2327ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c0c6590 con 0x7ff22c0acfc0 2023-11-06T20:56:59.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.901+0000 7ff2327ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c155580 con 0x7ff22c0638b0 2023-11-06T20:56:59.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.901+0000 7ff22bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff22c0acfc0 0x7ff22c0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.901+0000 7ff22b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff22c1550d0 0x7ff22c0631a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48208/0 (socket says 172.21.15.33:48208) 2023-11-06T20:56:59.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.901+0000 7ff22b7fe700 1 -- 172.21.15.33:0/229855001 learned_addr learned my addr 172.21.15.33:0/229855001 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:56:59.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.901+0000 7ff22bfff700 1 --2- 172.21.15.33:0/229855001 >> v2:172.21.15.33:3300/0 conn(0x7ff22c0acfc0 0x7ff22c0ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43808/0 (socket says 172.21.15.33:43808) 2023-11-06T20:56:59.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.901+0000 7ff230d9c700 1 --2- 172.21.15.33:0/229855001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff22c0638b0 0x7ff22c0c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.902+0000 7ff22bfff700 1 -- 172.21.15.33:0/229855001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff22c0638b0 msgr2=0x7ff22c0c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:56:59.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.902+0000 7ff22bfff700 1 --2- 172.21.15.33:0/229855001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff22c0638b0 0x7ff22c0c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.902+0000 7ff22bfff700 1 -- 172.21.15.33:0/229855001 >> v1:172.21.15.33:6789/0 conn(0x7ff22c1550d0 legacy=0x7ff22c0631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.902+0000 7ff22affd700 1 -- 172.21.15.33:0/229855001 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50514745 0 0) 0x7ff22c1586a0 con 0x7ff22c1550d0 2023-11-06T20:56:59.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.902+0000 7ff22bfff700 1 -- 172.21.15.33:0/229855001 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c1544c0 con 0x7ff22c0acfc0 2023-11-06T20:56:59.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.902+0000 7ff22bfff700 1 --2- 172.21.15.33:0/229855001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c0acfc0 0x7ff22c0ad3a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7ff21c009790 tx=0x7ff21c009060).ready entity=mon.0 client_cookie=98c3298a06ae0821 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:59.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.902+0000 7ff22affd700 1 -- 172.21.15.33:0/229855001 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00f510 con 0x7ff22c0acfc0 2023-11-06T20:56:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.902+0000 7ff22affd700 1 -- 172.21.15.33:0/229855001 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c00f670 con 0x7ff22c0acfc0 2023-11-06T20:56:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.902+0000 7ff22affd700 1 -- 172.21.15.33:0/229855001 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00f9f0 con 0x7ff22c0acfc0 2023-11-06T20:56:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.903+0000 7ff2327ff700 1 -- 172.21.15.33:0/229855001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c0acfc0 msgr2=0x7ff22c0ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.903+0000 7ff2327ff700 1 --2- 172.21.15.33:0/229855001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c0acfc0 0x7ff22c0ad3a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7ff21c009790 tx=0x7ff21c009060).stop 2023-11-06T20:56:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.903+0000 7ff2327ff700 1 -- 172.21.15.33:0/229855001 shutdown_connections 2023-11-06T20:56:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.903+0000 7ff2327ff700 1 --2- 172.21.15.33:0/229855001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff22c0638b0 0x7ff22c0c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.903+0000 7ff2327ff700 1 --2- 172.21.15.33:0/229855001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c0acfc0 0x7ff22c0ad3a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.903+0000 7ff2327ff700 1 -- 172.21.15.33:0/229855001 >> 172.21.15.33:0/229855001 conn(0x7ff22c05ec80 msgr2=0x7ff22c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:56:59.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.903+0000 7ff2327ff700 1 -- 172.21.15.33:0/229855001 shutdown_connections 2023-11-06T20:56:59.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.903+0000 7ff2327ff700 1 -- 172.21.15.33:0/229855001 wait complete. 2023-11-06T20:56:59.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.904+0000 7ff2327ff700 1 Processor -- start 2023-11-06T20:56:59.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.904+0000 7ff2327ff700 1 -- start start 2023-11-06T20:56:59.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.905+0000 7ff2327ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c0638b0 0x7ff22c1439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.905+0000 7ff2327ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff22c0acfc0 0x7ff22c0ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.905+0000 7ff2327ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c1550d0 0x7ff22c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.905+0000 7ff2327ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c143e90 con 0x7ff22c1550d0 2023-11-06T20:56:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.905+0000 7ff2327ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0c2200 con 0x7ff22c0638b0 2023-11-06T20:56:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.905+0000 7ff2327ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0c1eb0 con 0x7ff22c0acfc0 2023-11-06T20:56:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff230d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c1550d0 0x7ff22c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff230d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c1550d0 0x7ff22c156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43820/0 (socket says 172.21.15.33:43820) 2023-11-06T20:56:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff230d9c700 1 -- 172.21.15.33:0/570963282 learned_addr learned my addr 172.21.15.33:0/570963282 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:56:59.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff22bfff700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c0638b0 0x7ff22c1439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff22b7fe700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff22c0acfc0 0x7ff22c0ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff230d9c700 1 -- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff22c0acfc0 msgr2=0x7ff22c0ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff230d9c700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff22c0acfc0 0x7ff22c0ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff230d9c700 1 -- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c0638b0 msgr2=0x7ff22c1439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:56:59.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff230d9c700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c0638b0 0x7ff22c1439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:56:59.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff230d9c700 1 -- 172.21.15.33:0/570963282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0092a0 con 0x7ff22c1550d0 2023-11-06T20:56:59.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff230d9c700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c1550d0 0x7ff22c156840 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7ff224009ef0 tx=0x7ff22400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.906+0000 7ff213fff700 1 -- 172.21.15.33:0/570963282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22400cb70 con 0x7ff22c1550d0 2023-11-06T20:56:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.907+0000 7ff213fff700 1 -- 172.21.15.33:0/570963282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22400f040 con 0x7ff22c1550d0 2023-11-06T20:56:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.907+0000 7ff2327ff700 1 -- 172.21.15.33:0/570963282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c0b0420 con 0x7ff22c1550d0 2023-11-06T20:56:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.907+0000 7ff213fff700 1 -- 172.21.15.33:0/570963282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224013400 con 0x7ff22c1550d0 2023-11-06T20:56:59.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.907+0000 7ff2327ff700 1 -- 172.21.15.33:0/570963282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22c0b09e0 con 0x7ff22c1550d0 2023-11-06T20:56:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.908+0000 7ff2327ff700 1 -- 172.21.15.33:0/570963282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22c0cb7c0 con 0x7ff22c1550d0 2023-11-06T20:56:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.909+0000 7ff213fff700 1 -- 172.21.15.33:0/570963282 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff2240137b0 con 0x7ff22c1550d0 2023-11-06T20:56:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.909+0000 7ff213fff700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff21805f270 0x7ff218061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:56:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.909+0000 7ff213fff700 1 -- 172.21.15.33:0/570963282 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff22407a970 con 0x7ff22c1550d0 2023-11-06T20:56:59.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.910+0000 7ff22bfff700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff21805f270 0x7ff218061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:56:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.910+0000 7ff22bfff700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff21805f270 0x7ff218061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff21c009790 tx=0x7ff21c017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:56:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:56:59.913+0000 7ff213fff700 1 -- 172.21.15.33:0/570963282 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff22404bd50 con 0x7ff22c1550d0 2023-11-06T20:57:00.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.094+0000 7ff2327ff700 1 -- 172.21.15.33:0/570963282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7ff22c06c6c0 con 0x7ff22c1550d0 2023-11-06T20:57:00.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.199+0000 7ff213fff700 1 -- 172.21.15.33:0/570963282 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) v1 ==== 152+0+0 (secure 0 0 0) 0x7ff22404f370 con 0x7ff22c1550d0 2023-11-06T20:57:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.201+0000 7ff2327ff700 1 -- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff21805f270 msgr2=0x7ff218061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:00.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.201+0000 7ff2327ff700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff21805f270 0x7ff218061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff21c009790 tx=0x7ff21c017000).stop 2023-11-06T20:57:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.201+0000 7ff2327ff700 1 -- 172.21.15.33:0/570963282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c1550d0 msgr2=0x7ff22c156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.201+0000 7ff2327ff700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c1550d0 0x7ff22c156840 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7ff224009ef0 tx=0x7ff22400b040).stop 2023-11-06T20:57:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.202+0000 7ff2327ff700 1 -- 172.21.15.33:0/570963282 shutdown_connections 2023-11-06T20:57:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.202+0000 7ff2327ff700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff21805f270 0x7ff218061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:00.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.202+0000 7ff2327ff700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff22c1550d0 0x7ff22c156840 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.202+0000 7ff2327ff700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff22c0638b0 0x7ff22c1439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.202+0000 7ff2327ff700 1 --2- 172.21.15.33:0/570963282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff22c0acfc0 0x7ff22c0ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.202+0000 7ff2327ff700 1 -- 172.21.15.33:0/570963282 >> 172.21.15.33:0/570963282 conn(0x7ff22c05ec80 msgr2=0x7ff22c152a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:00.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.202+0000 7ff2327ff700 1 -- 172.21.15.33:0/570963282 shutdown_connections 2023-11-06T20:57:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.202+0000 7ff2327ff700 1 -- 172.21.15.33:0/570963282 wait complete. 2023-11-06T20:57:00.204 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T20:57:00.224 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T20:57:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.429+0000 7fd938bff700 1 Processor -- start 2023-11-06T20:57:00.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.429+0000 7fd938bff700 1 -- start start 2023-11-06T20:57:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.430+0000 7fd938bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9341546a0 0x7fd934064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.430+0000 7fd938bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd9340be2c0 0x7fd934066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd938bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9340641f0 con 0x7fd934066560 2023-11-06T20:57:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd938bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd934154a80 con 0x7fd9341546a0 2023-11-06T20:57:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd938bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9340638c0 con 0x7fd9340be2c0 2023-11-06T20:57:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd93259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9341546a0 0x7fd934064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd93259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9341546a0 0x7fd934064680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43822/0 (socket says 172.21.15.33:43822) 2023-11-06T20:57:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd93259c700 1 -- 172.21.15.33:0/3474062585 learned_addr learned my addr 172.21.15.33:0/3474062585 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd932d9d700 1 --2- 172.21.15.33:0/3474062585 >> v2:172.21.15.69:3300/0 conn(0x7fd9340be2c0 0x7fd934066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd93159a700 1 -- 172.21.15.33:0/3474062585 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882462822 0 0) 0x7fd9340641f0 con 0x7fd934066560 2023-11-06T20:57:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd93159a700 1 -- 172.21.15.33:0/3474062585 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd91c0033f0 con 0x7fd934066560 2023-11-06T20:57:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd93259c700 1 -- 172.21.15.33:0/3474062585 >> v2:172.21.15.69:3300/0 conn(0x7fd9340be2c0 msgr2=0x7fd934066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd93259c700 1 --2- 172.21.15.33:0/3474062585 >> v2:172.21.15.69:3300/0 conn(0x7fd9340be2c0 0x7fd934066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd93259c700 1 -- 172.21.15.33:0/3474062585 >> v1:172.21.15.33:6789/0 conn(0x7fd934066560 legacy=0x7fd9340bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.431+0000 7fd93259c700 1 -- 172.21.15.33:0/3474062585 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd934063ba0 con 0x7fd9341546a0 2023-11-06T20:57:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.432+0000 7fd93259c700 1 --2- 172.21.15.33:0/3474062585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 0x7fd934064680 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fd93406a450 tx=0x7fd9240090f0).ready entity=mon.0 client_cookie=733cae16dca1d619 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.432+0000 7fd93159a700 1 -- 172.21.15.33:0/3474062585 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92400f510 con 0x7fd9341546a0 2023-11-06T20:57:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.432+0000 7fd93159a700 1 -- 172.21.15.33:0/3474062585 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92400f670 con 0x7fd9341546a0 2023-11-06T20:57:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.432+0000 7fd93159a700 1 -- 172.21.15.33:0/3474062585 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92400f9f0 con 0x7fd9341546a0 2023-11-06T20:57:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.432+0000 7fd938bff700 1 -- 172.21.15.33:0/3474062585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 msgr2=0x7fd934064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.432+0000 7fd938bff700 1 --2- 172.21.15.33:0/3474062585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 0x7fd934064680 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fd93406a450 tx=0x7fd9240090f0).stop 2023-11-06T20:57:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.433+0000 7fd938bff700 1 -- 172.21.15.33:0/3474062585 shutdown_connections 2023-11-06T20:57:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.433+0000 7fd938bff700 1 --2- 172.21.15.33:0/3474062585 >> v2:172.21.15.69:3300/0 conn(0x7fd9340be2c0 0x7fd934066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.433+0000 7fd938bff700 1 --2- 172.21.15.33:0/3474062585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 0x7fd934064680 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.433+0000 7fd938bff700 1 -- 172.21.15.33:0/3474062585 >> 172.21.15.33:0/3474062585 conn(0x7fd9340c8c50 msgr2=0x7fd9340c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.433+0000 7fd938bff700 1 -- 172.21.15.33:0/3474062585 shutdown_connections 2023-11-06T20:57:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.433+0000 7fd938bff700 1 -- 172.21.15.33:0/3474062585 wait complete. 2023-11-06T20:57:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.435+0000 7fd938bff700 1 Processor -- start 2023-11-06T20:57:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.435+0000 7fd938bff700 1 -- start start 2023-11-06T20:57:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.436+0000 7fd938bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd934066560 0x7fd9340c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.436+0000 7fd938bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9340be2c0 0x7fd9340c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.436+0000 7fd938bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 0x7fd934164500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.436+0000 7fd938bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd934168480 con 0x7fd9341546a0 2023-11-06T20:57:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.436+0000 7fd938bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340c3d00 con 0x7fd9340be2c0 2023-11-06T20:57:00.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.436+0000 7fd938bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340c5e90 con 0x7fd934066560 2023-11-06T20:57:00.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.436+0000 7fd932d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 0x7fd934164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:00.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd932d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 0x7fd934164500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43828/0 (socket says 172.21.15.33:43828) 2023-11-06T20:57:00.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd932d9d700 1 -- 172.21.15.33:0/662674344 learned_addr learned my addr 172.21.15.33:0/662674344 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:00.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd93259c700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd934066560 0x7fd9340c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:00.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd931d9b700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9340be2c0 0x7fd9340c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:00.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd932d9d700 1 -- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd934066560 msgr2=0x7fd9340c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:00.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd932d9d700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd934066560 0x7fd9340c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:00.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd932d9d700 1 -- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9340be2c0 msgr2=0x7fd9340c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:00.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd932d9d700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9340be2c0 0x7fd9340c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:00.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd932d9d700 1 -- 172.21.15.33:0/662674344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd924009370 con 0x7fd9341546a0 2023-11-06T20:57:00.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd931d9b700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9340be2c0 0x7fd9340c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:57:00.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd932d9d700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 0x7fd934164500 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fd92c009f20 tx=0x7fd92c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:00.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd9227fc700 1 -- 172.21.15.33:0/662674344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c00cbd0 con 0x7fd9341546a0 2023-11-06T20:57:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.437+0000 7fd9227fc700 1 -- 172.21.15.33:0/662674344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c00f040 con 0x7fd9341546a0 2023-11-06T20:57:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.438+0000 7fd938bff700 1 -- 172.21.15.33:0/662674344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd934168940 con 0x7fd9341546a0 2023-11-06T20:57:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.438+0000 7fd9227fc700 1 -- 172.21.15.33:0/662674344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c013400 con 0x7fd9341546a0 2023-11-06T20:57:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.438+0000 7fd938bff700 1 -- 172.21.15.33:0/662674344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd934168710 con 0x7fd9341546a0 2023-11-06T20:57:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.439+0000 7fd938bff700 1 -- 172.21.15.33:0/662674344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd934066000 con 0x7fd9341546a0 2023-11-06T20:57:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.440+0000 7fd9227fc700 1 -- 172.21.15.33:0/662674344 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd92c0137b0 con 0x7fd9341546a0 2023-11-06T20:57:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.441+0000 7fd9227fc700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd91405f270 0x7fd914061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.441+0000 7fd9227fc700 1 -- 172.21.15.33:0/662674344 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5639+0+0 (secure 0 0 0) 0x7fd92c07ab30 con 0x7fd9341546a0 2023-11-06T20:57:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.442+0000 7fd93259c700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd91405f270 0x7fd914061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.442+0000 7fd93259c700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd91405f270 0x7fd914061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fd93406a450 tx=0x7fd924017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:00.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.446+0000 7fd9227fc700 1 -- 172.21.15.33:0/662674344 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd92c04bda0 con 0x7fd9341546a0 2023-11-06T20:57:00.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:00.626+0000 7fd938bff700 1 -- 172.21.15.33:0/662674344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd934151e20 con 0x7fd9341546a0 2023-11-06T20:57:01.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.208+0000 7fd9227fc700 1 -- 172.21.15.33:0/662674344 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd92c04f3c0 con 0x7fd9341546a0 2023-11-06T20:57:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.214+0000 7fd938bff700 1 -- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd91405f270 msgr2=0x7fd914061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.214+0000 7fd938bff700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd91405f270 0x7fd914061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fd93406a450 tx=0x7fd924017000).stop 2023-11-06T20:57:01.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.214+0000 7fd938bff700 1 -- 172.21.15.33:0/662674344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 msgr2=0x7fd934164500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.214+0000 7fd938bff700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 0x7fd934164500 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fd92c009f20 tx=0x7fd92c00b040).stop 2023-11-06T20:57:01.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.215+0000 7fd938bff700 1 -- 172.21.15.33:0/662674344 shutdown_connections 2023-11-06T20:57:01.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.215+0000 7fd938bff700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd91405f270 0x7fd914061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.215+0000 7fd938bff700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd934066560 0x7fd9340c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.215+0000 7fd938bff700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9341546a0 0x7fd934164500 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.215+0000 7fd938bff700 1 --2- 172.21.15.33:0/662674344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9340be2c0 0x7fd9340c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.215+0000 7fd938bff700 1 -- 172.21.15.33:0/662674344 >> 172.21.15.33:0/662674344 conn(0x7fd9340c8c50 msgr2=0x7fd934152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:01.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.216+0000 7fd938bff700 1 -- 172.21.15.33:0/662674344 shutdown_connections 2023-11-06T20:57:01.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.217+0000 7fd938bff700 1 -- 172.21.15.33:0/662674344 wait complete. 2023-11-06T20:57:01.220 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T20:57:01.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T20:57:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.453+0000 7f45489ff700 1 Processor -- start 2023-11-06T20:57:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.453+0000 7f45489ff700 1 -- start start 2023-11-06T20:57:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.454+0000 7f45489ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f45441544f0 0x7f45440c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.454+0000 7f45489ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45441556d0 con 0x7f4544064dd0 2023-11-06T20:57:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.454+0000 7f45489ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4544154ac0 con 0x7f45440c0c40 2023-11-06T20:57:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.454+0000 7f45489ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45441548d0 con 0x7f45441544f0 2023-11-06T20:57:01.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.455+0000 7f454259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f45441544f0 0x7f45440c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:01.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.455+0000 7f454259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f45441544f0 0x7f45440c0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43840/0 (socket says 172.21.15.33:43840) 2023-11-06T20:57:01.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.455+0000 7f4542d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4544064dd0 0x7f45440c57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48216/0 (socket says 172.21.15.33:48216) 2023-11-06T20:57:01.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.455+0000 7f454259c700 1 -- 172.21.15.33:0/1935139286 learned_addr learned my addr 172.21.15.33:0/1935139286 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.455+0000 7f454259c700 1 -- 172.21.15.33:0/1935139286 >> v1:172.21.15.69:6789/0 conn(0x7f45440c0c40 legacy=0x7f45440646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.455+0000 7f454259c700 1 -- 172.21.15.33:0/1935139286 >> v1:172.21.15.33:6789/0 conn(0x7f4544064dd0 legacy=0x7f45440c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.455+0000 7f454159a700 1 -- 172.21.15.33:0/1935139286 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274705375 0 0) 0x7f45441556d0 con 0x7f4544064dd0 2023-11-06T20:57:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.455+0000 7f454259c700 1 -- 172.21.15.33:0/1935139286 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4544154da0 con 0x7f45441544f0 2023-11-06T20:57:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.456+0000 7f454259c700 1 --2- 172.21.15.33:0/1935139286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45441544f0 0x7f45440c0750 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f4534006fd0 tx=0x7f45340090f0).ready entity=mon.0 client_cookie=cd410c201e054447 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.456+0000 7f454159a700 1 -- 172.21.15.33:0/1935139286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400b600 con 0x7f45441544f0 2023-11-06T20:57:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.456+0000 7f454159a700 1 -- 172.21.15.33:0/1935139286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453400b760 con 0x7f45441544f0 2023-11-06T20:57:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.456+0000 7f454159a700 1 -- 172.21.15.33:0/1935139286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400bae0 con 0x7f45441544f0 2023-11-06T20:57:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.456+0000 7f45489ff700 1 -- 172.21.15.33:0/1935139286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45441544f0 msgr2=0x7f45440c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.456+0000 7f45489ff700 1 --2- 172.21.15.33:0/1935139286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45441544f0 0x7f45440c0750 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f4534006fd0 tx=0x7f45340090f0).stop 2023-11-06T20:57:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.456+0000 7f45489ff700 1 -- 172.21.15.33:0/1935139286 shutdown_connections 2023-11-06T20:57:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.456+0000 7f45489ff700 1 --2- 172.21.15.33:0/1935139286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45441544f0 0x7f45440c0750 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.456+0000 7f45489ff700 1 -- 172.21.15.33:0/1935139286 >> 172.21.15.33:0/1935139286 conn(0x7f45440cb900 msgr2=0x7f45440c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.456+0000 7f45489ff700 1 -- 172.21.15.33:0/1935139286 shutdown_connections 2023-11-06T20:57:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.457+0000 7f45489ff700 1 -- 172.21.15.33:0/1935139286 wait complete. 2023-11-06T20:57:01.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.458+0000 7f45489ff700 1 Processor -- start 2023-11-06T20:57:01.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.458+0000 7f45489ff700 1 -- start start 2023-11-06T20:57:01.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.459+0000 7f45489ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4544064dd0 0x7f4544153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:01.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.459+0000 7f45489ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45440c0c40 0x7f45440c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:01.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.459+0000 7f45489ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45441544f0 0x7f45440b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:01.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.459+0000 7f45489ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4544153510 con 0x7f45440c0c40 2023-11-06T20:57:01.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.459+0000 7f45489ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45440af2a0 con 0x7f45441544f0 2023-11-06T20:57:01.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.459+0000 7f45489ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45440af580 con 0x7f4544064dd0 2023-11-06T20:57:01.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.459+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45440c0c40 0x7f45440c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:01.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.459+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45440c0c40 0x7f45440c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43856/0 (socket says 172.21.15.33:43856) 2023-11-06T20:57:01.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.459+0000 7f4541d9b700 1 -- 172.21.15.33:0/2387619655 learned_addr learned my addr 172.21.15.33:0/2387619655 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:01.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f454259c700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4544064dd0 0x7f4544153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:01.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f4542d9d700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45441544f0 0x7f45440b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:01.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f4541d9b700 1 -- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4544064dd0 msgr2=0x7f4544153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f4541d9b700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4544064dd0 0x7f4544153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f4541d9b700 1 -- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45441544f0 msgr2=0x7f45440b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f4541d9b700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45441544f0 0x7f45440b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f4541d9b700 1 -- 172.21.15.33:0/2387619655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4534009370 con 0x7f45440c0c40 2023-11-06T20:57:01.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f4542d9d700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45441544f0 0x7f45440b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:57:01.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f4541d9b700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45440c0c40 0x7f45440c1ee0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f453800c4c0 tx=0x7f4538007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f45327fc700 1 -- 172.21.15.33:0/2387619655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538007020 con 0x7f45440c0c40 2023-11-06T20:57:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f45327fc700 1 -- 172.21.15.33:0/2387619655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4538001e90 con 0x7f45440c0c40 2023-11-06T20:57:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f45327fc700 1 -- 172.21.15.33:0/2387619655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380073b0 con 0x7f45440c0c40 2023-11-06T20:57:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f45489ff700 1 -- 172.21.15.33:0/2387619655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45440af860 con 0x7f45440c0c40 2023-11-06T20:57:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.460+0000 7f45489ff700 1 -- 172.21.15.33:0/2387619655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4544164870 con 0x7f45440c0c40 2023-11-06T20:57:01.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.462+0000 7f45489ff700 1 -- 172.21.15.33:0/2387619655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4544155080 con 0x7f45440c0c40 2023-11-06T20:57:01.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.463+0000 7f45327fc700 1 -- 172.21.15.33:0/2387619655 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f45380068b0 con 0x7f45440c0c40 2023-11-06T20:57:01.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.464+0000 7f45327fc700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452405f270 0x7f4524061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:01.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.464+0000 7f45327fc700 1 -- 172.21.15.33:0/2387619655 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f4538079dc0 con 0x7f45440c0c40 2023-11-06T20:57:01.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.464+0000 7f454259c700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452405f270 0x7f4524061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:01.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.465+0000 7f454259c700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452405f270 0x7f4524061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4534009130 tx=0x7f4534017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:01.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.468+0000 7f45327fc700 1 -- 172.21.15.33:0/2387619655 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f453804aeb0 con 0x7f45440c0c40 2023-11-06T20:57:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.695+0000 7f45489ff700 1 -- 172.21.15.33:0/2387619655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f45440c5240 con 0x7f45440c0c40 2023-11-06T20:57:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.701+0000 7f45327fc700 1 -- 172.21.15.33:0/2387619655 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7f453804e4d0 con 0x7f45440c0c40 2023-11-06T20:57:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.705+0000 7f45489ff700 1 -- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452405f270 msgr2=0x7f4524061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.705+0000 7f45489ff700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452405f270 0x7f4524061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4534009130 tx=0x7f4534017000).stop 2023-11-06T20:57:01.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.705+0000 7f45489ff700 1 -- 172.21.15.33:0/2387619655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45440c0c40 msgr2=0x7f45440c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.705+0000 7f45489ff700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45440c0c40 0x7f45440c1ee0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f453800c4c0 tx=0x7f4538007e00).stop 2023-11-06T20:57:01.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.706+0000 7f45489ff700 1 -- 172.21.15.33:0/2387619655 shutdown_connections 2023-11-06T20:57:01.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.706+0000 7f45489ff700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f452405f270 0x7f4524061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.706+0000 7f45489ff700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4544064dd0 0x7f4544153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.706+0000 7f45489ff700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45441544f0 0x7f45440b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.706+0000 7f45489ff700 1 --2- 172.21.15.33:0/2387619655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45440c0c40 0x7f45440c1ee0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.706+0000 7f45489ff700 1 -- 172.21.15.33:0/2387619655 >> 172.21.15.33:0/2387619655 conn(0x7f45440cb900 msgr2=0x7f4544156da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:01.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.706+0000 7f45489ff700 1 -- 172.21.15.33:0/2387619655 shutdown_connections 2023-11-06T20:57:01.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.706+0000 7f45489ff700 1 -- 172.21.15.33:0/2387619655 wait complete. 2023-11-06T20:57:01.712 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 7 and data pool 8 2023-11-06T20:57:01.741 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:57:01.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.939+0000 7f90ddfff700 1 Processor -- start 2023-11-06T20:57:01.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.939+0000 7f90ddfff700 1 -- start start 2023-11-06T20:57:01.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.940+0000 7f90ddfff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f90d8153e60 0x7f90d80a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:01.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.941+0000 7f90ddfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d80c2ec0 con 0x7f90d8062710 2023-11-06T20:57:01.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.941+0000 7f90ddfff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d81546c0 con 0x7f90d8154240 2023-11-06T20:57:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.941+0000 7f90ddfff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d80a73d0 con 0x7f90d8153e60 2023-11-06T20:57:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.941+0000 7f90d7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f90d8062710 0x7f90d80c3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48230/0 (socket says 172.21.15.33:48230) 2023-11-06T20:57:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.941+0000 7f90d7fff700 1 -- 172.21.15.33:0/4210913363 learned_addr learned my addr 172.21.15.33:0/4210913363 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:57:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.941+0000 7f90d77fe700 1 --2- 172.21.15.33:0/4210913363 >> v2:172.21.15.69:3300/0 conn(0x7f90d8153e60 0x7f90d80a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.941+0000 7f90d67fc700 1 -- 172.21.15.33:0/4210913363 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478304482 0 0) 0x7f90d80c2ec0 con 0x7f90d8062710 2023-11-06T20:57:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.941+0000 7f90d67fc700 1 -- 172.21.15.33:0/4210913363 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c00033f0 con 0x7f90d8062710 2023-11-06T20:57:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.942+0000 7f90d77fe700 1 -- 172.21.15.33:0/4210913363 >> v1:172.21.15.69:6789/0 conn(0x7f90d8154240 legacy=0x7f90d8069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.942+0000 7f90d77fe700 1 -- 172.21.15.33:0/4210913363 >> v1:172.21.15.33:6789/0 conn(0x7f90d8062710 legacy=0x7f90d80c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.942+0000 7f90d77fe700 1 -- 172.21.15.33:0/4210913363 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d80c2590 con 0x7f90d8153e60 2023-11-06T20:57:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.942+0000 7f90d77fe700 1 --2- 172.21.15.33:0/4210913363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d8153e60 0x7f90d80a6d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f90c8009760 tx=0x7f90c8009060).ready entity=mon.1 client_cookie=943e164c1f5db992 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.944+0000 7f90d67fc700 1 -- 172.21.15.33:0/4210913363 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c80090d0 con 0x7f90d8153e60 2023-11-06T20:57:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.944+0000 7f90d67fc700 1 -- 172.21.15.33:0/4210913363 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c800f510 con 0x7f90d8153e60 2023-11-06T20:57:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.944+0000 7f90d67fc700 1 -- 172.21.15.33:0/4210913363 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800f890 con 0x7f90d8153e60 2023-11-06T20:57:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.944+0000 7f90ddfff700 1 -- 172.21.15.33:0/4210913363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d8153e60 msgr2=0x7f90d80a6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.944+0000 7f90ddfff700 1 --2- 172.21.15.33:0/4210913363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d8153e60 0x7f90d80a6d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f90c8009760 tx=0x7f90c8009060).stop 2023-11-06T20:57:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.944+0000 7f90ddfff700 1 -- 172.21.15.33:0/4210913363 shutdown_connections 2023-11-06T20:57:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.944+0000 7f90ddfff700 1 --2- 172.21.15.33:0/4210913363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d8153e60 0x7f90d80a6d40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.944+0000 7f90ddfff700 1 -- 172.21.15.33:0/4210913363 >> 172.21.15.33:0/4210913363 conn(0x7f90d80c7b50 msgr2=0x7f90d80c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.944+0000 7f90ddfff700 1 -- 172.21.15.33:0/4210913363 shutdown_connections 2023-11-06T20:57:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.944+0000 7f90ddfff700 1 -- 172.21.15.33:0/4210913363 wait complete. 2023-11-06T20:57:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.946+0000 7f90ddfff700 1 Processor -- start 2023-11-06T20:57:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.946+0000 7f90ddfff700 1 -- start start 2023-11-06T20:57:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90ddfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f90d8062710 0x7f90d80bed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:01.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90ddfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d8153e60 0x7f90d8148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:01.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90ddfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8154240 0x7f90d80acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:01.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90ddfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80a8b30 con 0x7f90d8154240 2023-11-06T20:57:01.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90ddfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8164ce0 con 0x7f90d8153e60 2023-11-06T20:57:01.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90ddfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8164fc0 con 0x7f90d8062710 2023-11-06T20:57:01.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8154240 0x7f90d80acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:01.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8154240 0x7f90d80acc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43868/0 (socket says 172.21.15.33:43868) 2023-11-06T20:57:01.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90d7fff700 1 -- 172.21.15.33:0/2401251671 learned_addr learned my addr 172.21.15.33:0/2401251671 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90d77fe700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f90d8062710 0x7f90d80bed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.947+0000 7f90d6ffd700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d8153e60 0x7f90d8148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90d7fff700 1 -- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f90d8062710 msgr2=0x7f90d80bed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90d7fff700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f90d8062710 0x7f90d80bed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90d7fff700 1 -- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d8153e60 msgr2=0x7f90d8148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90d7fff700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d8153e60 0x7f90d8148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90d7fff700 1 -- 172.21.15.33:0/2401251671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c80092a0 con 0x7f90d8154240 2023-11-06T20:57:01.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90d7fff700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8154240 0x7f90d80acc30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f90d0005ad0 tx=0x7f90d00015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:01.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90bf7fe700 1 -- 172.21.15.33:0/2401251671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0005d30 con 0x7f90d8154240 2023-11-06T20:57:01.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90bf7fe700 1 -- 172.21.15.33:0/2401251671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d0005e90 con 0x7f90d8154240 2023-11-06T20:57:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90ddfff700 1 -- 172.21.15.33:0/2401251671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d81652a0 con 0x7f90d8154240 2023-11-06T20:57:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90bf7fe700 1 -- 172.21.15.33:0/2401251671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0007320 con 0x7f90d8154240 2023-11-06T20:57:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.948+0000 7f90ddfff700 1 -- 172.21.15.33:0/2401251671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d80bf270 con 0x7f90d8154240 2023-11-06T20:57:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.949+0000 7f90ddfff700 1 -- 172.21.15.33:0/2401251671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d80a8ce0 con 0x7f90d8154240 2023-11-06T20:57:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.950+0000 7f90bf7fe700 1 -- 172.21.15.33:0/2401251671 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f90d0007710 con 0x7f90d8154240 2023-11-06T20:57:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.951+0000 7f90bf7fe700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90c405f270 0x7f90c4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.951+0000 7f90bf7fe700 1 -- 172.21.15.33:0/2401251671 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f90d007a2c0 con 0x7f90d8154240 2023-11-06T20:57:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.951+0000 7f90d77fe700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90c405f270 0x7f90c4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.952+0000 7f90d77fe700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90c405f270 0x7f90c4061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f90d80c2120 tx=0x7f90c8017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:01.955+0000 7f90bf7fe700 1 -- 172.21.15.33:0/2401251671 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f90d80a8ce0 con 0x7f90d8154240 2023-11-06T20:57:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.138+0000 7f90ddfff700 1 -- 172.21.15.33:0/2401251671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90d80a8c70 con 0x7f90d8154240 2023-11-06T20:57:02.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.139+0000 7f90bf7fe700 1 -- 172.21.15.33:0/2401251671 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f90d004b410 con 0x7f90d8154240 2023-11-06T20:57:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:57:02.139 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":65,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:57:01.696028+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:57:00.140567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T20:57:00.672828+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:57:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 -- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90c405f270 msgr2=0x7f90c4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90c405f270 0x7f90c4061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f90d80c2120 tx=0x7f90c8017000).stop 2023-11-06T20:57:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 -- 172.21.15.33:0/2401251671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8154240 msgr2=0x7f90d80acc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8154240 0x7f90d80acc30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f90d0005ad0 tx=0x7f90d00015b0).stop 2023-11-06T20:57:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 -- 172.21.15.33:0/2401251671 shutdown_connections 2023-11-06T20:57:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90c405f270 0x7f90c4061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f90d8062710 0x7f90d80bed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8154240 0x7f90d80acc30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 --2- 172.21.15.33:0/2401251671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d8153e60 0x7f90d8148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 -- 172.21.15.33:0/2401251671 >> 172.21.15.33:0/2401251671 conn(0x7f90d80c7b50 msgr2=0x7f90d80c4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:02.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 -- 172.21.15.33:0/2401251671 shutdown_connections 2023-11-06T20:57:02.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.142+0000 7f90ddfff700 1 -- 172.21.15.33:0/2401251671 wait complete. 2023-11-06T20:57:02.159 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:57:02.186 INFO:tasks.ceph.mds.b.smithi069.stderr:2023-11-06T20:57:02.186+0000 7fc690df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T20:57:02.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.357+0000 7fd38cdff700 1 Processor -- start 2023-11-06T20:57:02.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.357+0000 7fd38cdff700 1 -- start start 2023-11-06T20:57:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38cdff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd3880c1840 0x7fd3880a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38cdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 0x7fd3880c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38cdff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd3880c6240 0x7fd388157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38cdff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd388154960 con 0x7fd3880c1840 2023-11-06T20:57:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38cdff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3881554a0 con 0x7fd3880c6240 2023-11-06T20:57:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38cdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd388155930 con 0x7fd3880aa2d0 2023-11-06T20:57:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd3880c1840 0x7fd3880a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd3880c1840 0x7fd3880a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43870/0 (socket says 172.21.15.33:43870) 2023-11-06T20:57:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38659c700 1 -- 172.21.15.33:0/3953010903 learned_addr learned my addr 172.21.15.33:0/3953010903 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd385d9b700 1 --2- 172.21.15.33:0/3953010903 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 0x7fd3880c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38659c700 1 -- 172.21.15.33:0/3953010903 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 msgr2=0x7fd3880c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd386d9d700 1 --2- 172.21.15.33:0/3953010903 >> v2:172.21.15.69:3300/0 conn(0x7fd3880c6240 0x7fd388157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38659c700 1 --2- 172.21.15.33:0/3953010903 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 0x7fd3880c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38659c700 1 -- 172.21.15.33:0/3953010903 >> v2:172.21.15.69:3300/0 conn(0x7fd3880c6240 msgr2=0x7fd388157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38659c700 1 --2- 172.21.15.33:0/3953010903 >> v2:172.21.15.69:3300/0 conn(0x7fd3880c6240 0x7fd388157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd38659c700 1 -- 172.21.15.33:0/3953010903 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880c66b0 con 0x7fd3880c1840 2023-11-06T20:57:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd385d9b700 1 --2- 172.21.15.33:0/3953010903 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 0x7fd3880c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:57:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.359+0000 7fd386d9d700 1 --2- 172.21.15.33:0/3953010903 >> v2:172.21.15.69:3300/0 conn(0x7fd3880c6240 0x7fd388157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:57:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38659c700 1 --2- 172.21.15.33:0/3953010903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c1840 0x7fd3880a9de0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fd388084630 tx=0x7fd3780090f0).ready entity=mon.0 client_cookie=3916be48a3f319f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38559a700 1 -- 172.21.15.33:0/3953010903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800b600 con 0x7fd3880c1840 2023-11-06T20:57:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38559a700 1 -- 172.21.15.33:0/3953010903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37800b760 con 0x7fd3880c1840 2023-11-06T20:57:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38559a700 1 -- 172.21.15.33:0/3953010903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800bae0 con 0x7fd3880c1840 2023-11-06T20:57:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38cdff700 1 -- 172.21.15.33:0/3953010903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c1840 msgr2=0x7fd3880a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38cdff700 1 --2- 172.21.15.33:0/3953010903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c1840 0x7fd3880a9de0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fd388084630 tx=0x7fd3780090f0).stop 2023-11-06T20:57:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38cdff700 1 -- 172.21.15.33:0/3953010903 shutdown_connections 2023-11-06T20:57:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38cdff700 1 --2- 172.21.15.33:0/3953010903 >> v2:172.21.15.69:3300/0 conn(0x7fd3880c6240 0x7fd388157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38cdff700 1 --2- 172.21.15.33:0/3953010903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c1840 0x7fd3880a9de0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38cdff700 1 --2- 172.21.15.33:0/3953010903 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 0x7fd3880c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.360+0000 7fd38cdff700 1 -- 172.21.15.33:0/3953010903 >> 172.21.15.33:0/3953010903 conn(0x7fd38805ec80 msgr2=0x7fd3880cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.361+0000 7fd38cdff700 1 -- 172.21.15.33:0/3953010903 shutdown_connections 2023-11-06T20:57:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.361+0000 7fd38cdff700 1 -- 172.21.15.33:0/3953010903 wait complete. 2023-11-06T20:57:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.362+0000 7fd38cdff700 1 Processor -- start 2023-11-06T20:57:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.362+0000 7fd38cdff700 1 -- start start 2023-11-06T20:57:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.363+0000 7fd38cdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 0x7fd3880adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.363+0000 7fd38cdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3880c1840 0x7fd3880b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.363+0000 7fd38cdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c6240 0x7fd388164830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.363+0000 7fd38cdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881643a0 con 0x7fd3880c6240 2023-11-06T20:57:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.363+0000 7fd38cdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880ae290 con 0x7fd3880c1840 2023-11-06T20:57:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.363+0000 7fd38cdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880ae510 con 0x7fd3880aa2d0 2023-11-06T20:57:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.363+0000 7fd386d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c6240 0x7fd388164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.363+0000 7fd386d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c6240 0x7fd388164830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43880/0 (socket says 172.21.15.33:43880) 2023-11-06T20:57:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.363+0000 7fd386d9d700 1 -- 172.21.15.33:0/1530204484 learned_addr learned my addr 172.21.15.33:0/1530204484 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd38659c700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 0x7fd3880adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd385d9b700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3880c1840 0x7fd3880b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd386d9d700 1 -- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 msgr2=0x7fd3880adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd386d9d700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 0x7fd3880adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd386d9d700 1 -- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3880c1840 msgr2=0x7fd3880b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd386d9d700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3880c1840 0x7fd3880b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd386d9d700 1 -- 172.21.15.33:0/1530204484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378009370 con 0x7fd3880c6240 2023-11-06T20:57:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd386d9d700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c6240 0x7fd388164830 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fd38000a380 tx=0x7fd380007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd3767fc700 1 -- 172.21.15.33:0/1530204484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380016470 con 0x7fd3880c6240 2023-11-06T20:57:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd3767fc700 1 -- 172.21.15.33:0/1530204484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3800165d0 con 0x7fd3880c6240 2023-11-06T20:57:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd3767fc700 1 -- 172.21.15.33:0/1530204484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380016950 con 0x7fd3880c6240 2023-11-06T20:57:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.364+0000 7fd38cdff700 1 -- 172.21.15.33:0/1530204484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3880ae7f0 con 0x7fd3880c6240 2023-11-06T20:57:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.365+0000 7fd38cdff700 1 -- 172.21.15.33:0/1530204484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3880b0d50 con 0x7fd3880c6240 2023-11-06T20:57:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.366+0000 7fd38cdff700 1 -- 172.21.15.33:0/1530204484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd388164510 con 0x7fd3880c6240 2023-11-06T20:57:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.366+0000 7fd3767fc700 1 -- 172.21.15.33:0/1530204484 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd380013070 con 0x7fd3880c6240 2023-11-06T20:57:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.367+0000 7fd3767fc700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd36805f240 0x7fd368061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.367+0000 7fd3767fc700 1 -- 172.21.15.33:0/1530204484 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd38007b1c0 con 0x7fd3880c6240 2023-11-06T20:57:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.368+0000 7fd38659c700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd36805f240 0x7fd368061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.368+0000 7fd38659c700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd36805f240 0x7fd368061700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fd378009130 tx=0x7fd378017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.371+0000 7fd3767fc700 1 -- 172.21.15.33:0/1530204484 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd38004c250 con 0x7fd3880c6240 2023-11-06T20:57:02.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.553+0000 7fd38cdff700 1 -- 172.21.15.33:0/1530204484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd38806c6c0 con 0x7fd3880c6240 2023-11-06T20:57:02.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.554+0000 7fd3767fc700 1 -- 172.21.15.33:0/1530204484 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16989 (secure 0 0 0) 0x7fd38004f870 con 0x7fd3880c6240 2023-11-06T20:57:02.553 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:57:02.554 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":65,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:57:01.696028+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:57:00.140567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T20:57:00.672828+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:57:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 -- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd36805f240 msgr2=0x7fd368061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd36805f240 0x7fd368061700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fd378009130 tx=0x7fd378017000).stop 2023-11-06T20:57:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 -- 172.21.15.33:0/1530204484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c6240 msgr2=0x7fd388164830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c6240 0x7fd388164830 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fd38000a380 tx=0x7fd380007900).stop 2023-11-06T20:57:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 -- 172.21.15.33:0/1530204484 shutdown_connections 2023-11-06T20:57:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd36805f240 0x7fd368061700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd3880aa2d0 0x7fd3880adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3880c6240 0x7fd388164830 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 --2- 172.21.15.33:0/1530204484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3880c1840 0x7fd3880b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 -- 172.21.15.33:0/1530204484 >> 172.21.15.33:0/1530204484 conn(0x7fd38805ec80 msgr2=0x7fd3880ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:02.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 -- 172.21.15.33:0/1530204484 shutdown_connections 2023-11-06T20:57:02.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.556+0000 7fd38cdff700 1 -- 172.21.15.33:0/1530204484 wait complete. 2023-11-06T20:57:02.573 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T20:57:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.774+0000 7f16ad1ff700 1 Processor -- start 2023-11-06T20:57:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.775+0000 7f16ad1ff700 1 -- start start 2023-11-06T20:57:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.776+0000 7f16ad1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16a80c3b60 0x7f16a8064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.776+0000 7f16ad1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a8064270 con 0x7f16a814fe80 2023-11-06T20:57:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.776+0000 7f16ad1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a80651b0 con 0x7f16a814faa0 2023-11-06T20:57:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.776+0000 7f16ad1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a81531f0 con 0x7f16a80c3b60 2023-11-06T20:57:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.776+0000 7f16a759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16a80c3b60 0x7f16a8064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.776+0000 7f16a659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f16a814fe80 0x7f16a80c3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48242/0 (socket says 172.21.15.33:48242) 2023-11-06T20:57:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.776+0000 7f16a659c700 1 -- 172.21.15.33:0/2194440867 learned_addr learned my addr 172.21.15.33:0/2194440867 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:57:02.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.777+0000 7f16a759e700 1 -- 172.21.15.33:0/2194440867 >> v1:172.21.15.69:6789/0 conn(0x7f16a814faa0 legacy=0x7f16a8152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.777+0000 7f16a759e700 1 -- 172.21.15.33:0/2194440867 >> v1:172.21.15.33:6789/0 conn(0x7f16a814fe80 legacy=0x7f16a80c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.777+0000 7f16a759e700 1 -- 172.21.15.33:0/2194440867 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a81534d0 con 0x7f16a80c3b60 2023-11-06T20:57:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.777+0000 7f16a5d9b700 1 -- 172.21.15.33:0/2194440867 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108889844 0 0) 0x7f16a8064270 con 0x7f16a814fe80 2023-11-06T20:57:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.777+0000 7f16a759e700 1 --2- 172.21.15.33:0/2194440867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a80c3b60 0x7f16a8064cc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f16a0009e60 tx=0x7f16a0009780).ready entity=mon.0 client_cookie=cc8a4f91e6b61880 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.777+0000 7f16a5d9b700 1 -- 172.21.15.33:0/2194440867 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a000e070 con 0x7f16a80c3b60 2023-11-06T20:57:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.777+0000 7f16a5d9b700 1 -- 172.21.15.33:0/2194440867 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a000bc30 con 0x7f16a80c3b60 2023-11-06T20:57:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.777+0000 7f16a5d9b700 1 -- 172.21.15.33:0/2194440867 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0012410 con 0x7f16a80c3b60 2023-11-06T20:57:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.778+0000 7f16ad1ff700 1 -- 172.21.15.33:0/2194440867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a80c3b60 msgr2=0x7f16a8064cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.778+0000 7f16ad1ff700 1 --2- 172.21.15.33:0/2194440867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a80c3b60 0x7f16a8064cc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f16a0009e60 tx=0x7f16a0009780).stop 2023-11-06T20:57:02.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.778+0000 7f16ad1ff700 1 -- 172.21.15.33:0/2194440867 shutdown_connections 2023-11-06T20:57:02.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.778+0000 7f16ad1ff700 1 --2- 172.21.15.33:0/2194440867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a80c3b60 0x7f16a8064cc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.778+0000 7f16ad1ff700 1 -- 172.21.15.33:0/2194440867 >> 172.21.15.33:0/2194440867 conn(0x7f16a8062710 msgr2=0x7f16a8062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:02.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.778+0000 7f16ad1ff700 1 -- 172.21.15.33:0/2194440867 shutdown_connections 2023-11-06T20:57:02.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.778+0000 7f16ad1ff700 1 -- 172.21.15.33:0/2194440867 wait complete. 2023-11-06T20:57:02.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.779+0000 7f16ad1ff700 1 Processor -- start 2023-11-06T20:57:02.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.780+0000 7f16ad1ff700 1 -- start start 2023-11-06T20:57:02.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16ad1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16a80c3b60 0x7f16a81422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16ad1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a814faa0 0x7f16a80ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16ad1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16a814fe80 0x7f16a81549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16ad1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a80a9a10 con 0x7f16a814faa0 2023-11-06T20:57:02.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16ad1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a80aa590 con 0x7f16a814fe80 2023-11-06T20:57:02.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16ad1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a80a9c60 con 0x7f16a80c3b60 2023-11-06T20:57:02.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a814faa0 0x7f16a80ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a814faa0 0x7f16a80ac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43900/0 (socket says 172.21.15.33:43900) 2023-11-06T20:57:02.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a659c700 1 -- 172.21.15.33:0/76932567 learned_addr learned my addr 172.21.15.33:0/76932567 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:02.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a6d9d700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16a80c3b60 0x7f16a81422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a759e700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16a814fe80 0x7f16a81549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a659c700 1 -- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16a80c3b60 msgr2=0x7f16a81422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a659c700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16a80c3b60 0x7f16a81422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a659c700 1 -- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16a814fe80 msgr2=0x7f16a81549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:02.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a659c700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16a814fe80 0x7f16a81549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:02.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a659c700 1 -- 172.21.15.33:0/76932567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0009a20 con 0x7f16a814faa0 2023-11-06T20:57:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.781+0000 7f16a659c700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a814faa0 0x7f16a80ac020 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f169c00c460 tx=0x7f169c007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.782+0000 7f168effd700 1 -- 172.21.15.33:0/76932567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0072e0 con 0x7f16a814faa0 2023-11-06T20:57:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.782+0000 7f168effd700 1 -- 172.21.15.33:0/76932567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c007440 con 0x7f16a814faa0 2023-11-06T20:57:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.782+0000 7f168effd700 1 -- 172.21.15.33:0/76932567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0057a0 con 0x7f16a814faa0 2023-11-06T20:57:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.782+0000 7f16ad1ff700 1 -- 172.21.15.33:0/76932567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a80a9f40 con 0x7f16a814faa0 2023-11-06T20:57:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.782+0000 7f16ad1ff700 1 -- 172.21.15.33:0/76932567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a81427b0 con 0x7f16a814faa0 2023-11-06T20:57:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.783+0000 7f16ad1ff700 1 -- 172.21.15.33:0/76932567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a80aa700 con 0x7f16a814faa0 2023-11-06T20:57:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.784+0000 7f168effd700 1 -- 172.21.15.33:0/76932567 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f169c005b90 con 0x7f16a814faa0 2023-11-06T20:57:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.785+0000 7f168effd700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f169405f270 0x7f1694061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.785+0000 7f168effd700 1 -- 172.21.15.33:0/76932567 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f169c079bb0 con 0x7f16a814faa0 2023-11-06T20:57:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.785+0000 7f16a6d9d700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f169405f270 0x7f1694061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.786+0000 7f16a6d9d700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f169405f270 0x7f1694061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f16980036d0 tx=0x7f1698009040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:02.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:02.789+0000 7f168effd700 1 -- 172.21.15.33:0/76932567 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f169c04ac40 con 0x7f16a814faa0 2023-11-06T20:57:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.019+0000 7f16ad1ff700 1 -- 172.21.15.33:0/76932567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f16a80644c0 con 0x7f16a814faa0 2023-11-06T20:57:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.703+0000 7f168effd700 1 -- 172.21.15.33:0/76932567 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7f169c04e260 con 0x7f16a814faa0 2023-11-06T20:57:03.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 -- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f169405f270 msgr2=0x7f1694061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:03.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f169405f270 0x7f1694061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f16980036d0 tx=0x7f1698009040).stop 2023-11-06T20:57:03.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 -- 172.21.15.33:0/76932567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a814faa0 msgr2=0x7f16a80ac020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:03.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a814faa0 0x7f16a80ac020 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f169c00c460 tx=0x7f169c007e10).stop 2023-11-06T20:57:03.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 -- 172.21.15.33:0/76932567 shutdown_connections 2023-11-06T20:57:03.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f169405f270 0x7f1694061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:03.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16a80c3b60 0x7f16a81422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:03.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16a814fe80 0x7f16a81549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:03.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 --2- 172.21.15.33:0/76932567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16a814faa0 0x7f16a80ac020 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:03.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 -- 172.21.15.33:0/76932567 >> 172.21.15.33:0/76932567 conn(0x7f16a8062710 msgr2=0x7f16a81515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:03.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.706+0000 7f16ad1ff700 1 -- 172.21.15.33:0/76932567 shutdown_connections 2023-11-06T20:57:03.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.707+0000 7f16ad1ff700 1 -- 172.21.15.33:0/76932567 wait complete. 2023-11-06T20:57:03.726 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T20:57:03.727 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T20:57:03.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.932+0000 7ff9755ff700 1 Processor -- start 2023-11-06T20:57:03.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.932+0000 7ff9755ff700 1 -- start start 2023-11-06T20:57:03.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff9755ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff970154330 0x7ff9700c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:03.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff9755ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff9700c7ff0 0x7ff9700646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:03.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff9755ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9700c6b60 con 0x7ff970154710 2023-11-06T20:57:03.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff9755ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9700c7700 con 0x7ff970154330 2023-11-06T20:57:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff9755ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9700c5cd0 con 0x7ff9700c7ff0 2023-11-06T20:57:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff96effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff970154330 0x7ff9700c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff96effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff970154330 0x7ff9700c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43906/0 (socket says 172.21.15.33:43906) 2023-11-06T20:57:03.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff96effd700 1 -- 172.21.15.33:0/2280890483 learned_addr learned my addr 172.21.15.33:0/2280890483 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:03.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff96f7fe700 1 --2- 172.21.15.33:0/2280890483 >> v2:172.21.15.69:3300/0 conn(0x7ff9700c7ff0 0x7ff9700646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:03.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff96effd700 1 -- 172.21.15.33:0/2280890483 >> v2:172.21.15.69:3300/0 conn(0x7ff9700c7ff0 msgr2=0x7ff9700646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:03.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.934+0000 7ff96effd700 1 --2- 172.21.15.33:0/2280890483 >> v2:172.21.15.69:3300/0 conn(0x7ff9700c7ff0 0x7ff9700646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:03.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.935+0000 7ff96effd700 1 -- 172.21.15.33:0/2280890483 >> v1:172.21.15.69:6789/0 conn(0x7ff970154710 legacy=0x7ff9700c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:03.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.935+0000 7ff96effd700 1 -- 172.21.15.33:0/2280890483 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9700c5f50 con 0x7ff970154330 2023-11-06T20:57:03.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.935+0000 7ff96effd700 1 --2- 172.21.15.33:0/2280890483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 0x7ff9700c57e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7ff9700c1e60 tx=0x7ff9600090f0).ready entity=mon.0 client_cookie=25ad64e51ec10324 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:03.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.935+0000 7ff96dffb700 1 -- 172.21.15.33:0/2280890483 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000b520 con 0x7ff970154330 2023-11-06T20:57:03.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.935+0000 7ff96dffb700 1 -- 172.21.15.33:0/2280890483 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96000b680 con 0x7ff970154330 2023-11-06T20:57:03.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.935+0000 7ff96dffb700 1 -- 172.21.15.33:0/2280890483 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000ba00 con 0x7ff970154330 2023-11-06T20:57:03.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.935+0000 7ff9755ff700 1 -- 172.21.15.33:0/2280890483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 msgr2=0x7ff9700c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:03.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.935+0000 7ff9755ff700 1 --2- 172.21.15.33:0/2280890483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 0x7ff9700c57e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7ff9700c1e60 tx=0x7ff9600090f0).stop 2023-11-06T20:57:03.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.936+0000 7ff9755ff700 1 -- 172.21.15.33:0/2280890483 shutdown_connections 2023-11-06T20:57:03.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.936+0000 7ff9755ff700 1 --2- 172.21.15.33:0/2280890483 >> v2:172.21.15.69:3300/0 conn(0x7ff9700c7ff0 0x7ff9700646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:03.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.936+0000 7ff9755ff700 1 --2- 172.21.15.33:0/2280890483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 0x7ff9700c57e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:03.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.936+0000 7ff9755ff700 1 -- 172.21.15.33:0/2280890483 >> 172.21.15.33:0/2280890483 conn(0x7ff9700cb910 msgr2=0x7ff9700c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:03.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.936+0000 7ff9755ff700 1 -- 172.21.15.33:0/2280890483 shutdown_connections 2023-11-06T20:57:03.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.936+0000 7ff9755ff700 1 -- 172.21.15.33:0/2280890483 wait complete. 2023-11-06T20:57:03.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.938+0000 7ff9755ff700 1 Processor -- start 2023-11-06T20:57:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.938+0000 7ff9755ff700 1 -- start start 2023-11-06T20:57:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.939+0000 7ff9755ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9700c7ff0 0x7ff9700b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.939+0000 7ff9755ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 0x7ff970156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.939+0000 7ff9755ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff970154710 0x7ff9700646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.939+0000 7ff9755ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff970156630 con 0x7ff970154330 2023-11-06T20:57:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.939+0000 7ff9755ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff970155a20 con 0x7ff9700c7ff0 2023-11-06T20:57:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.939+0000 7ff9755ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff970155d00 con 0x7ff970154710 2023-11-06T20:57:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.939+0000 7ff96e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 0x7ff970156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff96e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 0x7ff970156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43918/0 (socket says 172.21.15.33:43918) 2023-11-06T20:57:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff96e7fc700 1 -- 172.21.15.33:0/3694370776 learned_addr learned my addr 172.21.15.33:0/3694370776 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff96effd700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9700c7ff0 0x7ff9700b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff96f7fe700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff970154710 0x7ff9700646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff96e7fc700 1 -- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff970154710 msgr2=0x7ff9700646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff96e7fc700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff970154710 0x7ff9700646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff96e7fc700 1 -- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9700c7ff0 msgr2=0x7ff9700b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff96e7fc700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9700c7ff0 0x7ff9700b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff96e7fc700 1 -- 172.21.15.33:0/3694370776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960009370 con 0x7ff970154330 2023-11-06T20:57:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff96e7fc700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 0x7ff970156ac0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7ff96400c440 tx=0x7ff964007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff956ffd700 1 -- 172.21.15.33:0/3694370776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96400ce40 con 0x7ff970154330 2023-11-06T20:57:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.940+0000 7ff956ffd700 1 -- 172.21.15.33:0/3694370776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff964007020 con 0x7ff970154330 2023-11-06T20:57:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.941+0000 7ff9755ff700 1 -- 172.21.15.33:0/3694370776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff970155fe0 con 0x7ff970154330 2023-11-06T20:57:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.941+0000 7ff956ffd700 1 -- 172.21.15.33:0/3694370776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9640073c0 con 0x7ff970154330 2023-11-06T20:57:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.941+0000 7ff9755ff700 1 -- 172.21.15.33:0/3694370776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff970156fb0 con 0x7ff970154330 2023-11-06T20:57:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.942+0000 7ff9755ff700 1 -- 172.21.15.33:0/3694370776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9701567a0 con 0x7ff970154330 2023-11-06T20:57:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.942+0000 7ff956ffd700 1 -- 172.21.15.33:0/3694370776 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff9640068b0 con 0x7ff970154330 2023-11-06T20:57:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.943+0000 7ff956ffd700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff95c05f270 0x7ff95c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.943+0000 7ff956ffd700 1 -- 172.21.15.33:0/3694370776 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff964079e20 con 0x7ff970154330 2023-11-06T20:57:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.944+0000 7ff96effd700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff95c05f270 0x7ff95c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.944+0000 7ff96effd700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff95c05f270 0x7ff95c061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff960009700 tx=0x7ff960017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:03.947+0000 7ff956ffd700 1 -- 172.21.15.33:0/3694370776 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff96404aeb0 con 0x7ff970154330 2023-11-06T20:57:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.169+0000 7ff9755ff700 1 -- 172.21.15.33:0/3694370776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff970150ee0 con 0x7ff970154330 2023-11-06T20:57:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.705+0000 7ff956ffd700 1 -- 172.21.15.33:0/3694370776 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff96404e4d0 con 0x7ff970154330 2023-11-06T20:57:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.707+0000 7ff9755ff700 1 -- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff95c05f270 msgr2=0x7ff95c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:04.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.707+0000 7ff9755ff700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff95c05f270 0x7ff95c061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff960009700 tx=0x7ff960017000).stop 2023-11-06T20:57:04.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.707+0000 7ff9755ff700 1 -- 172.21.15.33:0/3694370776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 msgr2=0x7ff970156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:04.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.707+0000 7ff9755ff700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 0x7ff970156ac0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7ff96400c440 tx=0x7ff964007dd0).stop 2023-11-06T20:57:04.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.707+0000 7ff9755ff700 1 -- 172.21.15.33:0/3694370776 shutdown_connections 2023-11-06T20:57:04.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.707+0000 7ff9755ff700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff95c05f270 0x7ff95c061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:04.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.707+0000 7ff9755ff700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff970154710 0x7ff9700646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:04.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.707+0000 7ff9755ff700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9700c7ff0 0x7ff9700b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:04.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.707+0000 7ff9755ff700 1 --2- 172.21.15.33:0/3694370776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff970154330 0x7ff970156ac0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:04.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.707+0000 7ff9755ff700 1 -- 172.21.15.33:0/3694370776 >> 172.21.15.33:0/3694370776 conn(0x7ff9700cb910 msgr2=0x7ff970153ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:04.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.708+0000 7ff9755ff700 1 -- 172.21.15.33:0/3694370776 shutdown_connections 2023-11-06T20:57:04.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.708+0000 7ff9755ff700 1 -- 172.21.15.33:0/3694370776 wait complete. 2023-11-06T20:57:04.727 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T20:57:04.728 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:57:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.938+0000 7f8bb7bff700 1 Processor -- start 2023-11-06T20:57:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.939+0000 7f8bb7bff700 1 -- start start 2023-11-06T20:57:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.940+0000 7f8bb7bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8bb0063140 0x7f8bb0063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.940+0000 7f8bb7bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8bb00c3680 0x7f8bb0154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.940+0000 7f8bb7bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb00c31f0 con 0x7f8bb005e1b0 2023-11-06T20:57:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.940+0000 7f8bb7bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb006a0f0 con 0x7f8bb0063140 2023-11-06T20:57:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.940+0000 7f8bb7bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb00c2920 con 0x7f8bb00c3680 2023-11-06T20:57:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.940+0000 7f8bb599b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8bb0063140 0x7f8bb0063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.940+0000 7f8bb599b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8bb0063140 0x7f8bb0063520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43932/0 (socket says 172.21.15.33:43932) 2023-11-06T20:57:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.940+0000 7f8bb599b700 1 -- 172.21.15.33:0/3609244968 learned_addr learned my addr 172.21.15.33:0/3609244968 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.940+0000 7f8bb619c700 1 --2- 172.21.15.33:0/3609244968 >> v2:172.21.15.69:3300/0 conn(0x7f8bb00c3680 0x7f8bb0154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.941+0000 7f8bb599b700 1 -- 172.21.15.33:0/3609244968 >> v2:172.21.15.69:3300/0 conn(0x7f8bb00c3680 msgr2=0x7f8bb0154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.941+0000 7f8bb599b700 1 --2- 172.21.15.33:0/3609244968 >> v2:172.21.15.69:3300/0 conn(0x7f8bb00c3680 0x7f8bb0154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.941+0000 7f8bb599b700 1 -- 172.21.15.33:0/3609244968 >> v1:172.21.15.69:6789/0 conn(0x7f8bb005e1b0 legacy=0x7f8bb00c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.941+0000 7f8bb599b700 1 -- 172.21.15.33:0/3609244968 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb005e640 con 0x7f8bb0063140 2023-11-06T20:57:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.941+0000 7f8bb619c700 1 --2- 172.21.15.33:0/3609244968 >> v2:172.21.15.69:3300/0 conn(0x7f8bb00c3680 0x7f8bb0154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:57:04.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.941+0000 7f8bb599b700 1 --2- 172.21.15.33:0/3609244968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb0063140 0x7f8bb0063520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f8bb00c3160 tx=0x7f8bac0090f0).ready entity=mon.0 client_cookie=3e6ace7c328a28d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:04.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.941+0000 7f8bb4999700 1 -- 172.21.15.33:0/3609244968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00b600 con 0x7f8bb0063140 2023-11-06T20:57:04.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.941+0000 7f8bb4999700 1 -- 172.21.15.33:0/3609244968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac00b760 con 0x7f8bb0063140 2023-11-06T20:57:04.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.941+0000 7f8bb4999700 1 -- 172.21.15.33:0/3609244968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00bae0 con 0x7f8bb0063140 2023-11-06T20:57:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.942+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3609244968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb0063140 msgr2=0x7f8bb0063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.942+0000 7f8bb7bff700 1 --2- 172.21.15.33:0/3609244968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb0063140 0x7f8bb0063520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f8bb00c3160 tx=0x7f8bac0090f0).stop 2023-11-06T20:57:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.942+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3609244968 shutdown_connections 2023-11-06T20:57:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.942+0000 7f8bb7bff700 1 --2- 172.21.15.33:0/3609244968 >> v2:172.21.15.69:3300/0 conn(0x7f8bb00c3680 0x7f8bb0154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:04.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.942+0000 7f8bb7bff700 1 --2- 172.21.15.33:0/3609244968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb0063140 0x7f8bb0063520 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:04.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.942+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3609244968 >> 172.21.15.33:0/3609244968 conn(0x7f8bb00c7d60 msgr2=0x7f8bb00c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:04.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.942+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3609244968 shutdown_connections 2023-11-06T20:57:04.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.942+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3609244968 wait complete. 2023-11-06T20:57:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.943+0000 7f8bb7bff700 1 Processor -- start 2023-11-06T20:57:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.943+0000 7f8bb7bff700 1 -- start start 2023-11-06T20:57:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.944+0000 7f8bb7bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8bb0063140 0x7f8bb0150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.944+0000 7f8bb7bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb00c3680 0x7f8bb0155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.944+0000 7f8bb7bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb0150c70 0x7f8bb0155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.944+0000 7f8bb7bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00a9010 con 0x7f8bb00c3680 2023-11-06T20:57:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.944+0000 7f8bb7bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb014aee0 con 0x7f8bb0150c70 2023-11-06T20:57:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.944+0000 7f8bb7bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb014d8c0 con 0x7f8bb0063140 2023-11-06T20:57:04.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb00c3680 0x7f8bb0155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:04.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb00c3680 0x7f8bb0155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43938/0 (socket says 172.21.15.33:43938) 2023-11-06T20:57:04.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb519a700 1 -- 172.21.15.33:0/3944406137 learned_addr learned my addr 172.21.15.33:0/3944406137 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:04.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb619c700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb0150c70 0x7f8bb0155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:04.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb599b700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8bb0063140 0x7f8bb0150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:04.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb519a700 1 -- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8bb0063140 msgr2=0x7f8bb0150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:04.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb519a700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8bb0063140 0x7f8bb0150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:04.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb519a700 1 -- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb0150c70 msgr2=0x7f8bb0155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb519a700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb0150c70 0x7f8bb0155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb519a700 1 -- 172.21.15.33:0/3944406137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac009370 con 0x7f8bb00c3680 2023-11-06T20:57:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb599b700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8bb0063140 0x7f8bb0150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:57:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.945+0000 7f8bb519a700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb00c3680 0x7f8bb0155080 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f8ba000c440 tx=0x7f8ba0007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.946+0000 7f8ba5ffb700 1 -- 172.21.15.33:0/3944406137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000ce40 con 0x7f8bb00c3680 2023-11-06T20:57:04.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.946+0000 7f8ba5ffb700 1 -- 172.21.15.33:0/3944406137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0007020 con 0x7f8bb00c3680 2023-11-06T20:57:04.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.946+0000 7f8ba5ffb700 1 -- 172.21.15.33:0/3944406137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00073c0 con 0x7f8bb00c3680 2023-11-06T20:57:04.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.946+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3944406137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb014dba0 con 0x7f8bb00c3680 2023-11-06T20:57:04.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.946+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3944406137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb014b410 con 0x7f8bb00c3680 2023-11-06T20:57:04.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.947+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3944406137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb0064420 con 0x7f8bb00c3680 2023-11-06T20:57:04.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.948+0000 7f8ba5ffb700 1 -- 172.21.15.33:0/3944406137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8ba00068b0 con 0x7f8bb00c3680 2023-11-06T20:57:04.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.949+0000 7f8ba5ffb700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b9405f270 0x7f8b94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:04.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.949+0000 7f8ba5ffb700 1 -- 172.21.15.33:0/3944406137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8ba0079ea0 con 0x7f8bb00c3680 2023-11-06T20:57:04.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.949+0000 7f8bb599b700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b9405f270 0x7f8b94061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:04.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.950+0000 7f8bb599b700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b9405f270 0x7f8b94061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8bac009130 tx=0x7f8bac017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:04.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:04.953+0000 7f8ba5ffb700 1 -- 172.21.15.33:0/3944406137 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8ba004af30 con 0x7f8bb00c3680 2023-11-06T20:57:05.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.180+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3944406137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bb014b630 con 0x7f8bb00c3680 2023-11-06T20:57:05.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.181+0000 7f8ba5ffb700 1 -- 172.21.15.33:0/3944406137 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4483 (secure 0 0 0) 0x7f8ba004e550 con 0x7f8bb00c3680 2023-11-06T20:57:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:57:05.181 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/455721224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":455721224},{"type":"v1","addr":"172.21.15.69:6836","nonce":455721224}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5632,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2050732931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2050732931},{"type":"v1","addr":"172.21.15.33:6834","nonce":2050732931}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1579729451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1579729451},{"type":"v1","addr":"172.21.15.33:6835","nonce":1579729451}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:57:01.696412+0000","modified":"2023-11-06T20:57:04.702227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/698812717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":698812717},{"type":"v1","addr":"172.21.15.69:6837","nonce":698812717}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-06T20:57:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b9405f270 msgr2=0x7f8b94061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b9405f270 0x7f8b94061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8bac009130 tx=0x7f8bac017000).stop 2023-11-06T20:57:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3944406137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb00c3680 msgr2=0x7f8bb0155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb00c3680 0x7f8bb0155080 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f8ba000c440 tx=0x7f8ba0007dd0).stop 2023-11-06T20:57:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3944406137 shutdown_connections 2023-11-06T20:57:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8b9405f270 0x7f8b94061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8bb0063140 0x7f8bb0150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb0150c70 0x7f8bb0155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 --2- 172.21.15.33:0/3944406137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bb00c3680 0x7f8bb0155080 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3944406137 >> 172.21.15.33:0/3944406137 conn(0x7f8bb00c7d60 msgr2=0x7f8bb0068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3944406137 shutdown_connections 2023-11-06T20:57:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.183+0000 7f8bb7bff700 1 -- 172.21.15.33:0/3944406137 wait complete. 2023-11-06T20:57:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 22 2023-11-06T20:57:05.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:57:05.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.404+0000 7f76249ff700 1 Processor -- start 2023-11-06T20:57:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.405+0000 7f76249ff700 1 -- start start 2023-11-06T20:57:05.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f76249ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f76201580e0 0x7f76201584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f76249ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76200bdbe0 0x7f76200684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f76249ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7620064010 con 0x7f76200a7910 2023-11-06T20:57:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f76249ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76200be430 con 0x7f76200bdbe0 2023-11-06T20:57:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f76249ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76200689b0 con 0x7f76201580e0 2023-11-06T20:57:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f761dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76200bdbe0 0x7f76200684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f761dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76200bdbe0 0x7f76200684c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43942/0 (socket says 172.21.15.33:43942) 2023-11-06T20:57:05.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f761dd9b700 1 -- 172.21.15.33:0/2402251094 learned_addr learned my addr 172.21.15.33:0/2402251094 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f761e59c700 1 --2- 172.21.15.33:0/2402251094 >> v2:172.21.15.69:3300/0 conn(0x7f76201580e0 0x7f76201584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f761dd9b700 1 -- 172.21.15.33:0/2402251094 >> v2:172.21.15.69:3300/0 conn(0x7f76201580e0 msgr2=0x7f76201584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f761dd9b700 1 --2- 172.21.15.33:0/2402251094 >> v2:172.21.15.69:3300/0 conn(0x7f76201580e0 0x7f76201584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f761dd9b700 1 -- 172.21.15.33:0/2402251094 >> v1:172.21.15.69:6789/0 conn(0x7f76200a7910 legacy=0x7f76200a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.407+0000 7f761dd9b700 1 -- 172.21.15.33:0/2402251094 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76200be150 con 0x7f76200bdbe0 2023-11-06T20:57:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.408+0000 7f761dd9b700 1 --2- 172.21.15.33:0/2402251094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76200bdbe0 0x7f76200684c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f762005f090 tx=0x7f76140090f0).ready entity=mon.0 client_cookie=34c8a964f0b05fa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.408+0000 7f761d59a700 1 -- 172.21.15.33:0/2402251094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400b5a0 con 0x7f76200bdbe0 2023-11-06T20:57:05.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.408+0000 7f761d59a700 1 -- 172.21.15.33:0/2402251094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761400b700 con 0x7f76200bdbe0 2023-11-06T20:57:05.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.408+0000 7f761d59a700 1 -- 172.21.15.33:0/2402251094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400ba80 con 0x7f76200bdbe0 2023-11-06T20:57:05.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.408+0000 7f76249ff700 1 -- 172.21.15.33:0/2402251094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76200bdbe0 msgr2=0x7f76200684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.408+0000 7f76249ff700 1 --2- 172.21.15.33:0/2402251094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76200bdbe0 0x7f76200684c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f762005f090 tx=0x7f76140090f0).stop 2023-11-06T20:57:05.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.408+0000 7f76249ff700 1 -- 172.21.15.33:0/2402251094 shutdown_connections 2023-11-06T20:57:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.408+0000 7f76249ff700 1 --2- 172.21.15.33:0/2402251094 >> v2:172.21.15.69:3300/0 conn(0x7f76201580e0 0x7f76201584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.408+0000 7f76249ff700 1 --2- 172.21.15.33:0/2402251094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76200bdbe0 0x7f76200684c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.408+0000 7f76249ff700 1 -- 172.21.15.33:0/2402251094 >> 172.21.15.33:0/2402251094 conn(0x7f762005ec80 msgr2=0x7f76200664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.409+0000 7f76249ff700 1 -- 172.21.15.33:0/2402251094 shutdown_connections 2023-11-06T20:57:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.409+0000 7f76249ff700 1 -- 172.21.15.33:0/2402251094 wait complete. 2023-11-06T20:57:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.410+0000 7f76249ff700 1 Processor -- start 2023-11-06T20:57:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.410+0000 7f76249ff700 1 -- start start 2023-11-06T20:57:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.411+0000 7f76249ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76200a7910 0x7f7620139b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.411+0000 7f76249ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76200bdbe0 0x7f762014ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.411+0000 7f76249ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76201580e0 0x7f7620155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.411+0000 7f76249ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762014d180 con 0x7f76201580e0 2023-11-06T20:57:05.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.411+0000 7f76249ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762013a040 con 0x7f76200bdbe0 2023-11-06T20:57:05.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.411+0000 7f76249ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762014c330 con 0x7f76200a7910 2023-11-06T20:57:05.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76201580e0 0x7f7620155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76201580e0 0x7f7620155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43952/0 (socket says 172.21.15.33:43952) 2023-11-06T20:57:05.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761ed9d700 1 -- 172.21.15.33:0/1258433582 learned_addr learned my addr 172.21.15.33:0/1258433582 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:05.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761dd9b700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76200bdbe0 0x7f762014ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761e59c700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76200a7910 0x7f7620139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761ed9d700 1 -- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76200a7910 msgr2=0x7f7620139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761ed9d700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76200a7910 0x7f7620139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761ed9d700 1 -- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76200bdbe0 msgr2=0x7f762014ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761ed9d700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76200bdbe0 0x7f762014ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761ed9d700 1 -- 172.21.15.33:0/1258433582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7614009370 con 0x7f76201580e0 2023-11-06T20:57:05.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f761ed9d700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76201580e0 0x7f7620155ff0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f761800c810 tx=0x7f7618002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:05.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f760e7fc700 1 -- 172.21.15.33:0/1258433582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618010510 con 0x7f76201580e0 2023-11-06T20:57:05.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f760e7fc700 1 -- 172.21.15.33:0/1258433582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618010690 con 0x7f76201580e0 2023-11-06T20:57:05.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f760e7fc700 1 -- 172.21.15.33:0/1258433582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618010a50 con 0x7f76201580e0 2023-11-06T20:57:05.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.412+0000 7f76249ff700 1 -- 172.21.15.33:0/1258433582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762014c610 con 0x7f76201580e0 2023-11-06T20:57:05.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.413+0000 7f76249ff700 1 -- 172.21.15.33:0/1258433582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762014cbd0 con 0x7f76201580e0 2023-11-06T20:57:05.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.414+0000 7f76249ff700 1 -- 172.21.15.33:0/1258433582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762014d2f0 con 0x7f76201580e0 2023-11-06T20:57:05.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.414+0000 7f760e7fc700 1 -- 172.21.15.33:0/1258433582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7618012440 con 0x7f76201580e0 2023-11-06T20:57:05.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.415+0000 7f760e7fc700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f760005f270 0x7f7600061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.415+0000 7f760e7fc700 1 -- 172.21.15.33:0/1258433582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7618079c80 con 0x7f76201580e0 2023-11-06T20:57:05.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.416+0000 7f761e59c700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f760005f270 0x7f7600061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.416+0000 7f761e59c700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f760005f270 0x7f7600061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f761000bb10 tx=0x7f7610008040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:05.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.419+0000 7f760e7fc700 1 -- 172.21.15.33:0/1258433582 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7618048af0 con 0x7f76201580e0 2023-11-06T20:57:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.598+0000 7f76249ff700 1 -- 172.21.15.33:0/1258433582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76200c7d30 con 0x7f76201580e0 2023-11-06T20:57:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.599+0000 7f760e7fc700 1 -- 172.21.15.33:0/1258433582 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f761804e850 con 0x7f76201580e0 2023-11-06T20:57:05.599 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:57:05.599 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":66,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:57:02.698589+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:57:00.140567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T20:57:00.672828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:57:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.601+0000 7f76249ff700 1 -- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f760005f270 msgr2=0x7f7600061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.601+0000 7f76249ff700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f760005f270 0x7f7600061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f761000bb10 tx=0x7f7610008040).stop 2023-11-06T20:57:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.601+0000 7f76249ff700 1 -- 172.21.15.33:0/1258433582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76201580e0 msgr2=0x7f7620155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.601+0000 7f76249ff700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76201580e0 0x7f7620155ff0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f761800c810 tx=0x7f7618002560).stop 2023-11-06T20:57:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.602+0000 7f76249ff700 1 -- 172.21.15.33:0/1258433582 shutdown_connections 2023-11-06T20:57:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.602+0000 7f76249ff700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f760005f270 0x7f7600061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.602+0000 7f76249ff700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76200a7910 0x7f7620139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.602+0000 7f76249ff700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76201580e0 0x7f7620155ff0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.602+0000 7f76249ff700 1 --2- 172.21.15.33:0/1258433582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76200bdbe0 0x7f762014ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.602+0000 7f76249ff700 1 -- 172.21.15.33:0/1258433582 >> 172.21.15.33:0/1258433582 conn(0x7f762005ec80 msgr2=0x7f7620064ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.602+0000 7f76249ff700 1 -- 172.21.15.33:0/1258433582 shutdown_connections 2023-11-06T20:57:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.602+0000 7f76249ff700 1 -- 172.21.15.33:0/1258433582 wait complete. 2023-11-06T20:57:05.619 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T20:57:05.620 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:57:05.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.816+0000 7efecc1ff700 1 Processor -- start 2023-11-06T20:57:05.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.816+0000 7efecc1ff700 1 -- start start 2023-11-06T20:57:05.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.817+0000 7efecc1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efec4154280 0x7efec4154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.817+0000 7efecc1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efec40c6dc0 0x7efec40c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.817+0000 7efecc1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec40c5db0 con 0x7efec40ad6e0 2023-11-06T20:57:05.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.817+0000 7efecc1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec40c6930 con 0x7efec4154280 2023-11-06T20:57:05.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.817+0000 7efecc1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec40aa630 con 0x7efec40c6dc0 2023-11-06T20:57:05.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.818+0000 7efec9f9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efec4154280 0x7efec4154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.818+0000 7efeca79c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efec40c6dc0 0x7efec40c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.818+0000 7efec9f9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efec4154280 0x7efec4154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:59766/0 (socket says 172.21.15.33:59766) 2023-11-06T20:57:05.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.818+0000 7efec9f9b700 1 -- 172.21.15.33:0/404560091 learned_addr learned my addr 172.21.15.33:0/404560091 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:05.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.818+0000 7efec979a700 1 --1- 172.21.15.33:0/404560091 >> v1:172.21.15.69:6789/0 conn(0x7efec40ad6e0 0x7efec40a9f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:38312/0 (socket says 172.21.15.33:38312) 2023-11-06T20:57:05.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.819+0000 7efec9f9b700 1 -- 172.21.15.33:0/404560091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efec40c6dc0 msgr2=0x7efec40c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:05.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.819+0000 7efec9f9b700 1 --2- 172.21.15.33:0/404560091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efec40c6dc0 0x7efec40c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.819+0000 7efec9f9b700 1 -- 172.21.15.33:0/404560091 >> v1:172.21.15.69:6789/0 conn(0x7efec40ad6e0 legacy=0x7efec40a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.819+0000 7efec9f9b700 1 -- 172.21.15.33:0/404560091 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec40c6000 con 0x7efec4154280 2023-11-06T20:57:05.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.819+0000 7efec8f99700 1 -- 172.21.15.33:0/404560091 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804235884 0 0) 0x7efec40c5db0 con 0x7efec40ad6e0 2023-11-06T20:57:05.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.819+0000 7efec9f9b700 1 --2- 172.21.15.33:0/404560091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec4154280 0x7efec4154660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7efec0006fd0 tx=0x7efec00090f0).ready entity=mon.1 client_cookie=cac6fac4efdaac6f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.819+0000 7efec8f99700 1 -- 172.21.15.33:0/404560091 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000b580 con 0x7efec4154280 2023-11-06T20:57:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.819+0000 7efec8f99700 1 -- 172.21.15.33:0/404560091 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec000b6e0 con 0x7efec4154280 2023-11-06T20:57:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.819+0000 7efec8f99700 1 -- 172.21.15.33:0/404560091 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000ba60 con 0x7efec4154280 2023-11-06T20:57:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.820+0000 7efecc1ff700 1 -- 172.21.15.33:0/404560091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec4154280 msgr2=0x7efec4154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.820+0000 7efecc1ff700 1 --2- 172.21.15.33:0/404560091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec4154280 0x7efec4154660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7efec0006fd0 tx=0x7efec00090f0).stop 2023-11-06T20:57:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.820+0000 7efecc1ff700 1 -- 172.21.15.33:0/404560091 shutdown_connections 2023-11-06T20:57:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.820+0000 7efecc1ff700 1 --2- 172.21.15.33:0/404560091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec4154280 0x7efec4154660 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.820+0000 7efecc1ff700 1 --2- 172.21.15.33:0/404560091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efec40c6dc0 0x7efec40c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.820+0000 7efecc1ff700 1 -- 172.21.15.33:0/404560091 >> 172.21.15.33:0/404560091 conn(0x7efec405e1b0 msgr2=0x7efec405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.820+0000 7efecc1ff700 1 -- 172.21.15.33:0/404560091 shutdown_connections 2023-11-06T20:57:05.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.820+0000 7efecc1ff700 1 -- 172.21.15.33:0/404560091 wait complete. 2023-11-06T20:57:05.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.822+0000 7efecc1ff700 1 Processor -- start 2023-11-06T20:57:05.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.822+0000 7efecc1ff700 1 -- start start 2023-11-06T20:57:05.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.823+0000 7efecc1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efec40ad6e0 0x7efec414acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.823+0000 7efecc1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec40c6dc0 0x7efec40c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.823+0000 7efecc1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec4154280 0x7efec40a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.823+0000 7efecc1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec40c9f30 con 0x7efec4154280 2023-11-06T20:57:05.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.823+0000 7efecc1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec414b1a0 con 0x7efec40c6dc0 2023-11-06T20:57:05.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.823+0000 7efecc1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec40c9320 con 0x7efec40ad6e0 2023-11-06T20:57:05.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efeca79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec4154280 0x7efec40a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efeca79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec4154280 0x7efec40a33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43962/0 (socket says 172.21.15.33:43962) 2023-11-06T20:57:05.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efeca79c700 1 -- 172.21.15.33:0/3378479545 learned_addr learned my addr 172.21.15.33:0/3378479545 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:05.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efec9f9b700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efec40ad6e0 0x7efec414acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efec979a700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec40c6dc0 0x7efec40c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efeca79c700 1 -- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efec40ad6e0 msgr2=0x7efec414acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efeca79c700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efec40ad6e0 0x7efec414acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efeca79c700 1 -- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec40c6dc0 msgr2=0x7efec40c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:05.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efeca79c700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec40c6dc0 0x7efec40c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:05.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efeca79c700 1 -- 172.21.15.33:0/3378479545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0009370 con 0x7efec4154280 2023-11-06T20:57:05.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.824+0000 7efeca79c700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec4154280 0x7efec40a33b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7efebc009e90 tx=0x7efebc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:05.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.825+0000 7efeb9ffb700 1 -- 172.21.15.33:0/3378479545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00caa0 con 0x7efec4154280 2023-11-06T20:57:05.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.825+0000 7efeb9ffb700 1 -- 172.21.15.33:0/3378479545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc00f040 con 0x7efec4154280 2023-11-06T20:57:05.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.825+0000 7efeb9ffb700 1 -- 172.21.15.33:0/3378479545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00ce10 con 0x7efec4154280 2023-11-06T20:57:05.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.825+0000 7efecc1ff700 1 -- 172.21.15.33:0/3378479545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec40c9600 con 0x7efec4154280 2023-11-06T20:57:05.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.825+0000 7efecc1ff700 1 -- 172.21.15.33:0/3378479545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec40c9bc0 con 0x7efec4154280 2023-11-06T20:57:05.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.826+0000 7efecc1ff700 1 -- 172.21.15.33:0/3378479545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec40ca0e0 con 0x7efec4154280 2023-11-06T20:57:05.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.827+0000 7efeb9ffb700 1 -- 172.21.15.33:0/3378479545 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7efebc013610 con 0x7efec4154280 2023-11-06T20:57:05.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.827+0000 7efeb9ffb700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efea805f270 0x7efea8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:05.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.828+0000 7efeb9ffb700 1 -- 172.21.15.33:0/3378479545 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efebc07acd0 con 0x7efec4154280 2023-11-06T20:57:05.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.828+0000 7efec9f9b700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efea805f270 0x7efea8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:05.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.828+0000 7efec9f9b700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efea805f270 0x7efea8061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7efec0009fd0 tx=0x7efec0017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:05.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:05.832+0000 7efeb9ffb700 1 -- 172.21.15.33:0/3378479545 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efec40ca0e0 con 0x7efec4154280 2023-11-06T20:57:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.011+0000 7efecc1ff700 1 -- 172.21.15.33:0/3378479545 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efec40c59b0 con 0x7efea805f270 2023-11-06T20:57:06.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.024+0000 7efeb9ffb700 1 -- 172.21.15.33:0/3378479545 <== mgr.5443 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+353136 (secure 0 0 0) 0x7efec40c59b0 con 0x7efea805f270 2023-11-06T20:57:06.028 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:57:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 -- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efea805f270 msgr2=0x7efea8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efea805f270 0x7efea8061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7efec0009fd0 tx=0x7efec0017000).stop 2023-11-06T20:57:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 -- 172.21.15.33:0/3378479545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec4154280 msgr2=0x7efec40a33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec4154280 0x7efec40a33b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7efebc009e90 tx=0x7efebc00b040).stop 2023-11-06T20:57:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 -- 172.21.15.33:0/3378479545 shutdown_connections 2023-11-06T20:57:06.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efea805f270 0x7efea8061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efec40ad6e0 0x7efec414acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec4154280 0x7efec40a33b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 --2- 172.21.15.33:0/3378479545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec40c6dc0 0x7efec40c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 -- 172.21.15.33:0/3378479545 >> 172.21.15.33:0/3378479545 conn(0x7efec405e1b0 msgr2=0x7efec40acb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:06.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 -- 172.21.15.33:0/3378479545 shutdown_connections 2023-11-06T20:57:06.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.029+0000 7efecc1ff700 1 -- 172.21.15.33:0/3378479545 wait complete. 2023-11-06T20:57:06.034 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:57:06.045 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T20:57:04.404423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":734,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":92,"acting":92,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411300,"kb_used_data":5860,"kb_used_omap":0,"kb_used_meta":2405376,"kb_avail":752563420,"statfs":{"total":773094113280,"available":770624942080,"internally_reserved":0,"allocated":6000640,"data_stored":2945166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2463105024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15005312,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-727,"num_write_kb":-15561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-34,"num_keys_recovered":-10,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.002781"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"65'2","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.726755+0000","last_change":"2023-11-06T20:57:01.233364+0000","last_active":"2023-11-06T20:57:01.726755+0000","last_peered":"2023-11-06T20:57:01.726755+0000","last_clean":"2023-11-06T20:57:01.726755+0000","last_became_active":"2023-11-06T20:57:01.232904+0000","last_became_peered":"2023-11-06T20:57:01.232904+0000","last_unstale":"2023-11-06T20:57:01.726755+0000","last_undegraded":"2023-11-06T20:57:01.726755+0000","last_fullsized":"2023-11-06T20:57:01.726755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.726441+0000","last_change":"2023-11-06T20:57:01.726441+0000","last_active":"2023-11-06T20:57:01.726441+0000","last_peered":"2023-11-06T20:57:01.726441+0000","last_clean":"2023-11-06T20:57:01.726441+0000","last_became_active":"2023-11-06T20:57:01.726141+0000","last_became_peered":"2023-11-06T20:57:01.726141+0000","last_unstale":"2023-11-06T20:57:01.726441+0000","last_undegraded":"2023-11-06T20:57:01.726441+0000","last_fullsized":"2023-11-06T20:57:01.726441+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.701376+0000","last_change":"2023-11-06T20:57:01.232855+0000","last_active":"2023-11-06T20:57:01.701376+0000","last_peered":"2023-11-06T20:57:01.701376+0000","last_clean":"2023-11-06T20:57:01.701376+0000","last_became_active":"2023-11-06T20:57:01.232551+0000","last_became_peered":"2023-11-06T20:57:01.232551+0000","last_unstale":"2023-11-06T20:57:01.701376+0000","last_undegraded":"2023-11-06T20:57:01.701376+0000","last_fullsized":"2023-11-06T20:57:01.701376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.240683+0000","last_change":"2023-11-06T20:57:01.231722+0000","last_active":"2023-11-06T20:57:01.240683+0000","last_peered":"2023-11-06T20:57:01.240683+0000","last_clean":"2023-11-06T20:57:01.240683+0000","last_became_active":"2023-11-06T20:57:01.228910+0000","last_became_peered":"2023-11-06T20:57:01.228910+0000","last_unstale":"2023-11-06T20:57:01.240683+0000","last_undegraded":"2023-11-06T20:57:01.240683+0000","last_fullsized":"2023-11-06T20:57:01.240683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.730548+0000","last_change":"2023-11-06T20:57:01.730548+0000","last_active":"2023-11-06T20:57:01.730548+0000","last_peered":"2023-11-06T20:57:01.730548+0000","last_clean":"2023-11-06T20:57:01.730548+0000","last_became_active":"2023-11-06T20:57:01.730280+0000","last_became_peered":"2023-11-06T20:57:01.730280+0000","last_unstale":"2023-11-06T20:57:01.730548+0000","last_undegraded":"2023-11-06T20:57:01.730548+0000","last_fullsized":"2023-11-06T20:57:01.730548+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.239646+0000","last_change":"2023-11-06T20:57:01.234844+0000","last_active":"2023-11-06T20:57:01.239646+0000","last_peered":"2023-11-06T20:57:01.239646+0000","last_clean":"2023-11-06T20:57:01.239646+0000","last_became_active":"2023-11-06T20:57:01.234546+0000","last_became_peered":"2023-11-06T20:57:01.234546+0000","last_unstale":"2023-11-06T20:57:01.239646+0000","last_undegraded":"2023-11-06T20:57:01.239646+0000","last_fullsized":"2023-11-06T20:57:01.239646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.239800+0000","last_change":"2023-11-06T20:57:01.232583+0000","last_active":"2023-11-06T20:57:01.239800+0000","last_peered":"2023-11-06T20:57:01.239800+0000","last_clean":"2023-11-06T20:57:01.239800+0000","last_became_active":"2023-11-06T20:57:01.232271+0000","last_became_peered":"2023-11-06T20:57:01.232271+0000","last_unstale":"2023-11-06T20:57:01.239800+0000","last_undegraded":"2023-11-06T20:57:01.239800+0000","last_fullsized":"2023-11-06T20:57:01.239800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.240360+0000","last_change":"2023-11-06T20:57:01.236757+0000","last_active":"2023-11-06T20:57:01.240360+0000","last_peered":"2023-11-06T20:57:01.240360+0000","last_clean":"2023-11-06T20:57:01.240360+0000","last_became_active":"2023-11-06T20:57:01.236499+0000","last_became_peered":"2023-11-06T20:57:01.236499+0000","last_unstale":"2023-11-06T20:57:01.240360+0000","last_undegraded":"2023-11-06T20:57:01.240360+0000","last_fullsized":"2023-11-06T20:57:01.240360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.739708+0000","last_change":"2023-11-06T20:57:01.738937+0000","last_active":"2023-11-06T20:57:01.739708+0000","last_peered":"2023-11-06T20:57:01.739708+0000","last_clean":"2023-11-06T20:57:01.739708+0000","last_became_active":"2023-11-06T20:57:01.738569+0000","last_became_peered":"2023-11-06T20:57:01.738569+0000","last_unstale":"2023-11-06T20:57:01.739708+0000","last_undegraded":"2023-11-06T20:57:01.739708+0000","last_fullsized":"2023-11-06T20:57:01.739708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"65'1","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.714192+0000","last_change":"2023-11-06T20:57:01.240329+0000","last_active":"2023-11-06T20:57:01.714192+0000","last_peered":"2023-11-06T20:57:01.714192+0000","last_clean":"2023-11-06T20:57:01.714192+0000","last_became_active":"2023-11-06T20:57:01.239949+0000","last_became_peered":"2023-11-06T20:57:01.239949+0000","last_unstale":"2023-11-06T20:57:01.714192+0000","last_undegraded":"2023-11-06T20:57:01.714192+0000","last_fullsized":"2023-11-06T20:57:01.714192+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.701945+0000","last_change":"2023-11-06T20:57:01.221081+0000","last_active":"2023-11-06T20:57:01.701945+0000","last_peered":"2023-11-06T20:57:01.701945+0000","last_clean":"2023-11-06T20:57:01.701945+0000","last_became_active":"2023-11-06T20:57:01.220690+0000","last_became_peered":"2023-11-06T20:57:01.220690+0000","last_unstale":"2023-11-06T20:57:01.701945+0000","last_undegraded":"2023-11-06T20:57:01.701945+0000","last_fullsized":"2023-11-06T20:57:01.701945+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.242400+0000","last_change":"2023-11-06T20:57:01.238822+0000","last_active":"2023-11-06T20:57:01.242400+0000","last_peered":"2023-11-06T20:57:01.242400+0000","last_clean":"2023-11-06T20:57:01.242400+0000","last_became_active":"2023-11-06T20:57:01.238052+0000","last_became_peered":"2023-11-06T20:57:01.238052+0000","last_unstale":"2023-11-06T20:57:01.242400+0000","last_undegraded":"2023-11-06T20:57:01.242400+0000","last_fullsized":"2023-11-06T20:57:01.242400+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.704058+0000","last_change":"2023-11-06T20:57:01.227291+0000","last_active":"2023-11-06T20:57:01.704058+0000","last_peered":"2023-11-06T20:57:01.704058+0000","last_clean":"2023-11-06T20:57:01.704058+0000","last_became_active":"2023-11-06T20:57:01.226986+0000","last_became_peered":"2023-11-06T20:57:01.226986+0000","last_unstale":"2023-11-06T20:57:01.704058+0000","last_undegraded":"2023-11-06T20:57:01.704058+0000","last_fullsized":"2023-11-06T20:57:01.704058+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.702629+0000","last_change":"2023-11-06T20:57:01.236406+0000","last_active":"2023-11-06T20:57:01.702629+0000","last_peered":"2023-11-06T20:57:01.702629+0000","last_clean":"2023-11-06T20:57:01.702629+0000","last_became_active":"2023-11-06T20:57:01.236108+0000","last_became_peered":"2023-11-06T20:57:01.236108+0000","last_unstale":"2023-11-06T20:57:01.702629+0000","last_undegraded":"2023-11-06T20:57:01.702629+0000","last_fullsized":"2023-11-06T20:57:01.702629+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.240648+0000","last_change":"2023-11-06T20:57:01.226867+0000","last_active":"2023-11-06T20:57:01.240648+0000","last_peered":"2023-11-06T20:57:01.240648+0000","last_clean":"2023-11-06T20:57:01.240648+0000","last_became_active":"2023-11-06T20:57:01.225791+0000","last_became_peered":"2023-11-06T20:57:01.225791+0000","last_unstale":"2023-11-06T20:57:01.240648+0000","last_undegraded":"2023-11-06T20:57:01.240648+0000","last_fullsized":"2023-11-06T20:57:01.240648+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.733001+0000","last_change":"2023-11-06T20:57:01.723581+0000","last_active":"2023-11-06T20:57:01.733001+0000","last_peered":"2023-11-06T20:57:01.733001+0000","last_clean":"2023-11-06T20:57:01.733001+0000","last_became_active":"2023-11-06T20:57:01.723176+0000","last_became_peered":"2023-11-06T20:57:01.723176+0000","last_unstale":"2023-11-06T20:57:01.733001+0000","last_undegraded":"2023-11-06T20:57:01.733001+0000","last_fullsized":"2023-11-06T20:57:01.733001+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.711614+0000","last_change":"2023-11-06T20:57:01.708125+0000","last_active":"2023-11-06T20:57:01.711614+0000","last_peered":"2023-11-06T20:57:01.711614+0000","last_clean":"2023-11-06T20:57:01.711614+0000","last_became_active":"2023-11-06T20:57:01.707567+0000","last_became_peered":"2023-11-06T20:57:01.707567+0000","last_unstale":"2023-11-06T20:57:01.711614+0000","last_undegraded":"2023-11-06T20:57:01.711614+0000","last_fullsized":"2023-11-06T20:57:01.711614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736089+0000","last_change":"2023-11-06T20:57:01.725908+0000","last_active":"2023-11-06T20:57:01.736089+0000","last_peered":"2023-11-06T20:57:01.736089+0000","last_clean":"2023-11-06T20:57:01.736089+0000","last_became_active":"2023-11-06T20:57:01.723634+0000","last_became_peered":"2023-11-06T20:57:01.723634+0000","last_unstale":"2023-11-06T20:57:01.736089+0000","last_undegraded":"2023-11-06T20:57:01.736089+0000","last_fullsized":"2023-11-06T20:57:01.736089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"65'2","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.720187+0000","last_change":"2023-11-06T20:57:01.229829+0000","last_active":"2023-11-06T20:57:01.720187+0000","last_peered":"2023-11-06T20:57:01.720187+0000","last_clean":"2023-11-06T20:57:01.720187+0000","last_became_active":"2023-11-06T20:57:01.229474+0000","last_became_peered":"2023-11-06T20:57:01.229474+0000","last_unstale":"2023-11-06T20:57:01.720187+0000","last_undegraded":"2023-11-06T20:57:01.720187+0000","last_fullsized":"2023-11-06T20:57:01.720187+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.739669+0000","last_change":"2023-11-06T20:57:01.739669+0000","last_active":"2023-11-06T20:57:01.739669+0000","last_peered":"2023-11-06T20:57:01.739669+0000","last_clean":"2023-11-06T20:57:01.739669+0000","last_became_active":"2023-11-06T20:57:01.739374+0000","last_became_peered":"2023-11-06T20:57:01.739374+0000","last_unstale":"2023-11-06T20:57:01.739669+0000","last_undegraded":"2023-11-06T20:57:01.739669+0000","last_fullsized":"2023-11-06T20:57:01.739669+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.719965+0000","last_change":"2023-11-06T20:57:01.713319+0000","last_active":"2023-11-06T20:57:01.719965+0000","last_peered":"2023-11-06T20:57:01.719965+0000","last_clean":"2023-11-06T20:57:01.719965+0000","last_became_active":"2023-11-06T20:57:01.713016+0000","last_became_peered":"2023-11-06T20:57:01.713016+0000","last_unstale":"2023-11-06T20:57:01.719965+0000","last_undegraded":"2023-11-06T20:57:01.719965+0000","last_fullsized":"2023-11-06T20:57:01.719965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.703237+0000","last_change":"2023-11-06T20:57:01.227061+0000","last_active":"2023-11-06T20:57:01.703237+0000","last_peered":"2023-11-06T20:57:01.703237+0000","last_clean":"2023-11-06T20:57:01.703237+0000","last_became_active":"2023-11-06T20:57:01.226778+0000","last_became_peered":"2023-11-06T20:57:01.226778+0000","last_unstale":"2023-11-06T20:57:01.703237+0000","last_undegraded":"2023-11-06T20:57:01.703237+0000","last_fullsized":"2023-11-06T20:57:01.703237+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.741768+0000","last_change":"2023-11-06T20:57:01.741768+0000","last_active":"2023-11-06T20:57:01.741768+0000","last_peered":"2023-11-06T20:57:01.741768+0000","last_clean":"2023-11-06T20:57:01.741768+0000","last_became_active":"2023-11-06T20:57:01.741289+0000","last_became_peered":"2023-11-06T20:57:01.741289+0000","last_unstale":"2023-11-06T20:57:01.741768+0000","last_undegraded":"2023-11-06T20:57:01.741768+0000","last_fullsized":"2023-11-06T20:57:01.741768+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.702041+0000","last_change":"2023-11-06T20:57:01.226336+0000","last_active":"2023-11-06T20:57:01.702041+0000","last_peered":"2023-11-06T20:57:01.702041+0000","last_clean":"2023-11-06T20:57:01.702041+0000","last_became_active":"2023-11-06T20:57:01.226022+0000","last_became_peered":"2023-11-06T20:57:01.226022+0000","last_unstale":"2023-11-06T20:57:01.702041+0000","last_undegraded":"2023-11-06T20:57:01.702041+0000","last_fullsized":"2023-11-06T20:57:01.702041+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"65'3","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736333+0000","last_change":"2023-11-06T20:57:01.235203+0000","last_active":"2023-11-06T20:57:01.736333+0000","last_peered":"2023-11-06T20:57:01.736333+0000","last_clean":"2023-11-06T20:57:01.736333+0000","last_became_active":"2023-11-06T20:57:01.234941+0000","last_became_peered":"2023-11-06T20:57:01.234941+0000","last_unstale":"2023-11-06T20:57:01.736333+0000","last_undegraded":"2023-11-06T20:57:01.736333+0000","last_fullsized":"2023-11-06T20:57:01.736333+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"65'2","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736596+0000","last_change":"2023-11-06T20:57:01.236941+0000","last_active":"2023-11-06T20:57:01.736596+0000","last_peered":"2023-11-06T20:57:01.736596+0000","last_clean":"2023-11-06T20:57:01.736596+0000","last_became_active":"2023-11-06T20:57:01.232758+0000","last_became_peered":"2023-11-06T20:57:01.232758+0000","last_unstale":"2023-11-06T20:57:01.736596+0000","last_undegraded":"2023-11-06T20:57:01.736596+0000","last_fullsized":"2023-11-06T20:57:01.736596+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736164+0000","last_change":"2023-11-06T20:57:01.730062+0000","last_active":"2023-11-06T20:57:01.736164+0000","last_peered":"2023-11-06T20:57:01.736164+0000","last_clean":"2023-11-06T20:57:01.736164+0000","last_became_active":"2023-11-06T20:57:01.729718+0000","last_became_peered":"2023-11-06T20:57:01.729718+0000","last_unstale":"2023-11-06T20:57:01.736164+0000","last_undegraded":"2023-11-06T20:57:01.736164+0000","last_fullsized":"2023-11-06T20:57:01.736164+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.719541+0000","last_change":"2023-11-06T20:57:01.719541+0000","last_active":"2023-11-06T20:57:01.719541+0000","last_peered":"2023-11-06T20:57:01.719541+0000","last_clean":"2023-11-06T20:57:01.719541+0000","last_became_active":"2023-11-06T20:57:01.719250+0000","last_became_peered":"2023-11-06T20:57:01.719250+0000","last_unstale":"2023-11-06T20:57:01.719541+0000","last_undegraded":"2023-11-06T20:57:01.719541+0000","last_fullsized":"2023-11-06T20:57:01.719541+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.239643+0000","last_change":"2023-11-06T20:57:01.232750+0000","last_active":"2023-11-06T20:57:01.239643+0000","last_peered":"2023-11-06T20:57:01.239643+0000","last_clean":"2023-11-06T20:57:01.239643+0000","last_became_active":"2023-11-06T20:57:01.232418+0000","last_became_peered":"2023-11-06T20:57:01.232418+0000","last_unstale":"2023-11-06T20:57:01.239643+0000","last_undegraded":"2023-11-06T20:57:01.239643+0000","last_fullsized":"2023-11-06T20:57:01.239643+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.240768+0000","last_change":"2023-11-06T20:57:01.235238+0000","last_active":"2023-11-06T20:57:01.240768+0000","last_peered":"2023-11-06T20:57:01.240768+0000","last_clean":"2023-11-06T20:57:01.240768+0000","last_became_active":"2023-11-06T20:57:01.234880+0000","last_became_peered":"2023-11-06T20:57:01.234880+0000","last_unstale":"2023-11-06T20:57:01.240768+0000","last_undegraded":"2023-11-06T20:57:01.240768+0000","last_fullsized":"2023-11-06T20:57:01.240768+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.738813+0000","last_change":"2023-11-06T20:57:01.738813+0000","last_active":"2023-11-06T20:57:01.738813+0000","last_peered":"2023-11-06T20:57:01.738813+0000","last_clean":"2023-11-06T20:57:01.738813+0000","last_became_active":"2023-11-06T20:57:01.738296+0000","last_became_peered":"2023-11-06T20:57:01.738296+0000","last_unstale":"2023-11-06T20:57:01.738813+0000","last_undegraded":"2023-11-06T20:57:01.738813+0000","last_fullsized":"2023-11-06T20:57:01.738813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.223310+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:57:01.223310+0000","last_peered":"2023-11-06T20:57:01.223310+0000","last_clean":"2023-11-06T20:57:01.223310+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:57:01.223310+0000","last_undegraded":"2023-11-06T20:57:01.223310+0000","last_fullsized":"2023-11-06T20:57:01.223310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.734944+0000","last_change":"2023-11-06T20:57:01.228436+0000","last_active":"2023-11-06T20:57:01.734944+0000","last_peered":"2023-11-06T20:57:01.734944+0000","last_clean":"2023-11-06T20:57:01.734944+0000","last_became_active":"2023-11-06T20:57:01.228153+0000","last_became_peered":"2023-11-06T20:57:01.228153+0000","last_unstale":"2023-11-06T20:57:01.734944+0000","last_undegraded":"2023-11-06T20:57:01.734944+0000","last_fullsized":"2023-11-06T20:57:01.734944+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.703321+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:57:01.703321+0000","last_peered":"2023-11-06T20:57:01.703321+0000","last_clean":"2023-11-06T20:57:01.703321+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:57:01.703321+0000","last_undegraded":"2023-11-06T20:57:01.703321+0000","last_fullsized":"2023-11-06T20:57:01.703321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-06T20:56:56.510840+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:56:56.510840+0000","last_peered":"2023-11-06T20:56:56.510840+0000","last_clean":"2023-11-06T20:56:56.510840+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:56:56.510840+0000","last_undegraded":"2023-11-06T20:56:56.510840+0000","last_fullsized":"2023-11-06T20:56:56.510840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.239145+0000","last_change":"2023-11-06T20:57:01.230367+0000","last_active":"2023-11-06T20:57:01.239145+0000","last_peered":"2023-11-06T20:57:01.239145+0000","last_clean":"2023-11-06T20:57:01.239145+0000","last_became_active":"2023-11-06T20:57:01.229895+0000","last_became_peered":"2023-11-06T20:57:01.229895+0000","last_unstale":"2023-11-06T20:57:01.239145+0000","last_undegraded":"2023-11-06T20:57:01.239145+0000","last_fullsized":"2023-11-06T20:57:01.239145+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.702522+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:57:01.702522+0000","last_peered":"2023-11-06T20:57:01.702522+0000","last_clean":"2023-11-06T20:57:01.702522+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:57:01.702522+0000","last_undegraded":"2023-11-06T20:57:01.702522+0000","last_fullsized":"2023-11-06T20:57:01.702522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-06T20:56:56.514454+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:56:56.514454+0000","last_peered":"2023-11-06T20:56:56.514454+0000","last_clean":"2023-11-06T20:56:56.514454+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:56:56.514454+0000","last_undegraded":"2023-11-06T20:56:56.514454+0000","last_fullsized":"2023-11-06T20:56:56.514454+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.738037+0000","last_change":"2023-11-06T20:57:01.738037+0000","last_active":"2023-11-06T20:57:01.738037+0000","last_peered":"2023-11-06T20:57:01.738037+0000","last_clean":"2023-11-06T20:57:01.738037+0000","last_became_active":"2023-11-06T20:57:01.737674+0000","last_became_peered":"2023-11-06T20:57:01.737674+0000","last_unstale":"2023-11-06T20:57:01.738037+0000","last_undegraded":"2023-11-06T20:57:01.738037+0000","last_fullsized":"2023-11-06T20:57:01.738037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.704156+0000","last_change":"2023-11-06T20:57:01.235925+0000","last_active":"2023-11-06T20:57:01.704156+0000","last_peered":"2023-11-06T20:57:01.704156+0000","last_clean":"2023-11-06T20:57:01.704156+0000","last_became_active":"2023-11-06T20:57:01.233217+0000","last_became_peered":"2023-11-06T20:57:01.233217+0000","last_unstale":"2023-11-06T20:57:01.704156+0000","last_undegraded":"2023-11-06T20:57:01.704156+0000","last_fullsized":"2023-11-06T20:57:01.704156+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.701391+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:57:01.701391+0000","last_peered":"2023-11-06T20:57:01.701391+0000","last_clean":"2023-11-06T20:57:01.701391+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:57:01.701391+0000","last_undegraded":"2023-11-06T20:57:01.701391+0000","last_fullsized":"2023-11-06T20:57:01.701391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.704160+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:57:01.704160+0000","last_peered":"2023-11-06T20:57:01.704160+0000","last_clean":"2023-11-06T20:57:01.704160+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:57:01.704160+0000","last_undegraded":"2023-11-06T20:57:01.704160+0000","last_fullsized":"2023-11-06T20:57:01.704160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.729799+0000","last_change":"2023-11-06T20:57:01.729799+0000","last_active":"2023-11-06T20:57:01.729799+0000","last_peered":"2023-11-06T20:57:01.729799+0000","last_clean":"2023-11-06T20:57:01.729799+0000","last_became_active":"2023-11-06T20:57:01.729468+0000","last_became_peered":"2023-11-06T20:57:01.729468+0000","last_unstale":"2023-11-06T20:57:01.729799+0000","last_undegraded":"2023-11-06T20:57:01.729799+0000","last_fullsized":"2023-11-06T20:57:01.729799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.245317+0000","last_change":"2023-11-06T20:57:01.241972+0000","last_active":"2023-11-06T20:57:01.245317+0000","last_peered":"2023-11-06T20:57:01.245317+0000","last_clean":"2023-11-06T20:57:01.245317+0000","last_became_active":"2023-11-06T20:57:01.240265+0000","last_became_peered":"2023-11-06T20:57:01.240265+0000","last_unstale":"2023-11-06T20:57:01.245317+0000","last_undegraded":"2023-11-06T20:57:01.245317+0000","last_fullsized":"2023-11-06T20:57:01.245317+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-06T20:56:56.836852+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:56:56.836852+0000","last_peered":"2023-11-06T20:56:56.836852+0000","last_clean":"2023-11-06T20:56:56.836852+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:56:56.836852+0000","last_undegraded":"2023-11-06T20:56:56.836852+0000","last_fullsized":"2023-11-06T20:56:56.836852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:01.198280+0000","last_change":"2023-11-06T20:57:01.198280+0000","last_active":"2023-11-06T20:57:01.198280+0000","last_peered":"2023-11-06T20:57:01.198280+0000","last_clean":"2023-11-06T20:57:01.198280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:01.198280+0000","last_undegraded":"2023-11-06T20:57:01.198280+0000","last_fullsized":"2023-11-06T20:57:01.198280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:57:00.190912+0000","last_change":"2023-11-06T20:57:00.190912+0000","last_active":"2023-11-06T20:57:00.190912+0000","last_peered":"2023-11-06T20:57:00.190912+0000","last_clean":"2023-11-06T20:57:00.190912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:57:00.190912+0000","last_undegraded":"2023-11-06T20:57:00.190912+0000","last_fullsized":"2023-11-06T20:57:00.190912+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.703882+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:57:01.703882+0000","last_peered":"2023-11-06T20:57:01.703882+0000","last_clean":"2023-11-06T20:57:01.703882+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:57:01.703882+0000","last_undegraded":"2023-11-06T20:57:01.703882+0000","last_fullsized":"2023-11-06T20:57:01.703882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":734,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":46,"acting":46,"num_store_stats":2},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301912,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069928,"statfs":{"total":96636764160,"available":96327606272,"internally_reserved":0,"allocated":737280,"data_stored":362185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.68200000000000005},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70799999999999996,"15min":0.68000000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67700000000000005}]},{"osd":1,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71199999999999997,"15min":0.745},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":0.98599999999999999,"15min":5.1749999999999998},"last":0.755},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.41399999999999998},"max":{"1min":0.85099999999999998,"5min":0.94399999999999995,"15min":1.133},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.51100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91600000000000004,"5min":0.996,"15min":1.165},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85399999999999998,"5min":0.92700000000000005,"15min":1.024},"last":0.81000000000000005}]},{"osd":3,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72199999999999998,"15min":0.76100000000000001},"min":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":0.872,"5min":1.0129999999999999,"15min":5.202},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.80600000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.49399999999999999},"max":{"1min":0.878,"5min":0.95699999999999996,"15min":5.165},"last":0.78900000000000003}]},{"osd":4,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.43099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.751,"5min":0.77400000000000002,"15min":1.083},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.61799999999999999},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.40200000000000002},"max":{"1min":0.74299999999999999,"5min":0.86799999999999999,"15min":0.92800000000000005},"last":0.66000000000000003}]},{"osd":5,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.621,"15min":0.66100000000000003},"min":{"1min":0.47799999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78700000000000003,"5min":0.93300000000000005,"15min":1.395},"last":0.745},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64700000000000002,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.45000000000000001,"15min":0.42699999999999999},"max":{"1min":0.80000000000000004,"5min":0.878,"15min":1.129},"last":0.76700000000000002}]},{"osd":6,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70199999999999996,"15min":0.73199999999999998},"min":{"1min":0.55200000000000005,"5min":0.505,"15min":0.435},"max":{"1min":0.79900000000000004,"5min":0.94499999999999995,"15min":1.24},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.51700000000000002,"5min":0.47899999999999998,"15min":0.443},"max":{"1min":0.88900000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301912,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069928,"statfs":{"total":96636764160,"available":96327606272,"internally_reserved":0,"allocated":737280,"data_stored":362185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70499999999999996,"15min":0.67300000000000004},"min":{"1min":0.505,"5min":0.45600000000000002,"15min":0.32800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.0720000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.66800000000000004},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.38800000000000001},"max":{"1min":0.90100000000000002,"5min":0.92500000000000004,"15min":0.92900000000000005},"last":0.78800000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71899999999999997,"15min":0.68100000000000005},"min":{"1min":0.54000000000000004,"5min":0.45000000000000001,"15min":0.42299999999999999},"max":{"1min":0.96199999999999997,"5min":0.97599999999999998,"15min":1.3799999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.69099999999999995},"min":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.39700000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.1659999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78500000000000003,"15min":0.74299999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.95699999999999996,"15min":1.321},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.496},"max":{"1min":0.88300000000000001,"5min":1.024,"15min":1.405},"last":0.52600000000000002}]},{"osd":3,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.71199999999999997},"min":{"1min":0.56200000000000006,"5min":0.48699999999999999,"15min":0.45900000000000002},"max":{"1min":0.91700000000000004,"5min":1.0580000000000001,"15min":1.5109999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73799999999999999,"15min":0.71699999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.48999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.55000000000000004}]},{"osd":4,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.61899999999999999},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.018},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.65900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.86499999999999999,"5min":0.875,"15min":2.0310000000000001},"last":0.745}]},{"osd":5,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.502},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.68799999999999994},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.96899999999999997,"15min":1.133},"last":0.65100000000000002}]},{"osd":7,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66900000000000004,"15min":0.69299999999999995},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.88400000000000001,"5min":1,"15min":1.0940000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72299999999999998,"15min":0.74399999999999999},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95499999999999996,"5min":0.98699999999999999,"15min":1.4319999999999999},"last":0.80100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301912,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069928,"statfs":{"total":96636764160,"available":96327606272,"internally_reserved":0,"allocated":737280,"data_stored":362185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46400000000000002,"15min":0.441},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.27500000000000002},"max":{"1min":0.63800000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52900000000000003,"15min":0.495},"min":{"1min":0.37,"5min":0.35299999999999998,"15min":0.318},"max":{"1min":0.72799999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.316}]},{"osd":2,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.497},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.311},"max":{"1min":0.67400000000000004,"5min":0.80800000000000005,"15min":0.86599999999999999},"last":0.501},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61399999999999999,"15min":0.58499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.71199999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59799999999999998,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.40100000000000002,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.54700000000000004},"min":{"1min":0.41299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73499999999999999,"5min":0.94099999999999995,"15min":1.29},"last":0.35199999999999998}]},{"osd":4,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79000000000000004,"15min":0.79600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.5},"max":{"1min":0.99299999999999999,"5min":1.171,"15min":1.171},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.81599999999999995},"min":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.57899999999999996},"max":{"1min":1.056,"5min":1.0980000000000001,"15min":1.1519999999999999},"last":0.80300000000000005}]},{"osd":5,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.64700000000000002,"5min":0.57599999999999996,"15min":0.54900000000000004},"max":{"1min":0.96799999999999997,"5min":1.0629999999999999,"15min":1.329},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.78400000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0229999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61099999999999999}]},{"osd":6,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81599999999999995,"15min":0.79900000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.501},"max":{"1min":1.038,"5min":1.123,"15min":1.135},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.82299999999999995},"min":{"1min":0.65500000000000003,"5min":0.623,"15min":0.54500000000000004},"max":{"1min":1.014,"5min":1.1539999999999999,"15min":1.3540000000000001},"last":0.65400000000000003}]},{"osd":7,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.86299999999999999,"15min":0.83599999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.54700000000000004},"max":{"1min":1.077,"5min":1.1419999999999999,"15min":1.405},"last":0.628},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84099999999999997,"15min":0.82199999999999995},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.55900000000000005},"max":{"1min":1.0669999999999999,"5min":1.105,"15min":1.1950000000000001},"last":0.66100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301912,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069928,"statfs":{"total":96636764160,"available":96327606272,"internally_reserved":0,"allocated":737280,"data_stored":362185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47299999999999998,"15min":0.45700000000000002},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29299999999999998},"max":{"1min":0.71599999999999997,"5min":0.77700000000000002,"15min":0.78600000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.498},"min":{"1min":0.39600000000000002,"5min":0.33900000000000002,"15min":0.311},"max":{"1min":0.70599999999999996,"5min":0.77600000000000002,"15min":0.97399999999999998},"last":0.33200000000000002}]},{"osd":2,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47299999999999998,"15min":0.46200000000000002},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71999999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55900000000000005,"15min":0.53700000000000003},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.36499999999999999},"max":{"1min":0.68000000000000005,"5min":0.89300000000000002,"15min":0.91000000000000003},"last":0.36299999999999999}]},{"osd":3,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.42699999999999999,"5min":0.40899999999999997,"15min":0.35999999999999999},"max":{"1min":0.69299999999999995,"5min":0.755,"15min":0.79700000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.56399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.38300000000000001},"max":{"1min":0.80100000000000005,"5min":0.89300000000000002,"15min":0.89800000000000002},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.60799999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1279999999999999,"5min":1.1779999999999999,"15min":1.3779999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81299999999999994,"15min":0.80800000000000005},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.53600000000000003},"max":{"1min":1.105,"5min":1.202,"15min":1.8380000000000001},"last":0.82499999999999996}]},{"osd":5,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81999999999999995,"15min":0.80600000000000005},"min":{"1min":0.66600000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98799999999999999,"5min":1.139,"15min":1.3169999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83799999999999997,"15min":0.82799999999999996},"min":{"1min":0.68400000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1459999999999999,"5min":1.2829999999999999,"15min":1.915},"last":0.70399999999999996}]},{"osd":6,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.54600000000000004},"max":{"1min":1.0449999999999999,"5min":1.111,"15min":1.1120000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.82199999999999995,"15min":0.81499999999999995},"min":{"1min":0.64400000000000002,"5min":0.621,"15min":0.56999999999999995},"max":{"1min":1,"5min":1.304,"15min":1.5860000000000001},"last":0.64800000000000002}]},{"osd":7,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.81999999999999995},"min":{"1min":0.66600000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0820000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83799999999999997,"15min":0.83399999999999996},"min":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.53200000000000003},"max":{"1min":1.089,"5min":1.236,"15min":1.6359999999999999},"last":0.74299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":737280,"data_stored":362185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.441,"15min":0.437},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.25900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.504,"15min":0.48499999999999999},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.26300000000000001},"max":{"1min":0.68500000000000005,"5min":0.79000000000000004,"15min":1.0269999999999999},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39500000000000002,"15min":0.30099999999999999},"max":{"1min":0.77100000000000002,"5min":0.97699999999999998,"15min":1.3240000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.39300000000000002},"max":{"1min":0.81299999999999994,"5min":1.0209999999999999,"15min":1.2},"last":0.57699999999999996}]},{"osd":3,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.5,"15min":0.495},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33400000000000002},"max":{"1min":0.65500000000000003,"5min":0.73299999999999998,"15min":1.55},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.755,"5min":0.877,"15min":2.242},"last":0.60899999999999999}]},{"osd":4,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.54300000000000004},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.371},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.50900000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.357},"last":0.754}]},{"osd":5,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76900000000000002,"15min":0.77600000000000002},"min":{"1min":0.55300000000000005,"5min":0.50900000000000001,"15min":0.45900000000000002},"max":{"1min":1.0880000000000001,"5min":1.0960000000000001,"15min":1.595},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.498},"max":{"1min":1.0409999999999999,"5min":1.1080000000000001,"15min":1.4099999999999999},"last":0.78400000000000003}]},{"osd":6,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80800000000000005,"15min":0.79600000000000004},"min":{"1min":0.58699999999999997,"5min":0.58599999999999997,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":1.0900000000000001,"15min":1.4239999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.52300000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.6339999999999999},"last":0.80500000000000005}]},{"osd":7,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82999999999999996,"15min":0.82299999999999995},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.46600000000000003},"max":{"1min":1.121,"5min":1.1319999999999999,"15min":1.534},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81899999999999995,"15min":0.81399999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.6930000000000001},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301964,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069876,"statfs":{"total":96636764160,"available":96327553024,"internally_reserved":0,"allocated":790528,"data_stored":389058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.45200000000000001,"15min":0.45300000000000001},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.26800000000000002},"max":{"1min":0.66800000000000004,"5min":0.76700000000000002,"15min":1.046},"last":0.313},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52500000000000002,"15min":0.503},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.90100000000000002},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48399999999999999,"15min":0.47699999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.313},"max":{"1min":0.64600000000000002,"5min":0.69699999999999995,"15min":0.753},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.38900000000000001,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.80900000000000005,"15min":1.177},"last":0.41899999999999998}]},{"osd":2,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.35299999999999998},"max":{"1min":0.68999999999999995,"5min":0.749,"15min":0.77300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.502,"5min":0.44900000000000001,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.88100000000000001,"15min":1.3120000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.82399999999999995,"15min":0.78100000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.51200000000000001},"max":{"1min":0.96099999999999997,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82199999999999995,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.47999999999999998},"max":{"1min":0.93300000000000005,"5min":1.2729999999999999,"15min":1.286},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.80000000000000004,"15min":0.76400000000000001},"min":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.51400000000000001},"max":{"1min":0.997,"5min":1.1779999999999999,"15min":1.228},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.77100000000000002},"min":{"1min":0.63,"5min":0.56499999999999995,"15min":0.49199999999999999},"max":{"1min":1.036,"5min":1.141,"15min":1.1439999999999999},"last":0.627}]},{"osd":6,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79500000000000004,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97999999999999998,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.82599999999999996,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.53700000000000003},"max":{"1min":0.97599999999999998,"5min":1.2130000000000001,"15min":1.268},"last":0.59299999999999997}]},{"osd":7,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.82499999999999996,"15min":0.79000000000000004},"min":{"1min":0.64700000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99099999999999999,"5min":1.302,"15min":1.302},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83099999999999996,"15min":0.79400000000000004},"min":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.47899999999999998},"max":{"1min":1.036,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.64100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301912,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069928,"statfs":{"total":96636764160,"available":96327606272,"internally_reserved":0,"allocated":737280,"data_stored":362185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.98499999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.89900000000000002,"15min":0.93400000000000005},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71399999999999997,"15min":0.69899999999999995},"min":{"1min":0.58999999999999997,"5min":0.47099999999999997,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.95699999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.51000000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86399999999999999,"5min":0.90200000000000002,"15min":1.0569999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73599999999999999,"15min":0.70799999999999996},"min":{"1min":0.58699999999999997,"5min":0.498,"15min":0.46200000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78200000000000003,"15min":0.77800000000000002},"min":{"1min":0.61699999999999999,"5min":0.58199999999999996,"15min":0.51900000000000002},"max":{"1min":0.92900000000000005,"5min":0.93400000000000005,"15min":1.028},"last":0.69399999999999995}]},{"osd":3,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70399999999999996,"15min":0.68899999999999995},"min":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.0169999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70099999999999996,"15min":0.68600000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.42799999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.629},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78200000000000003,"5min":1.0389999999999999,"15min":1.0880000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57899999999999996,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76400000000000001,"5min":0.877,"15min":1.3520000000000001},"last":0.59699999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.623},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.39900000000000002},"max":{"1min":0.76700000000000002,"5min":0.84799999999999998,"15min":1.077},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.64600000000000002,"15min":0.70199999999999996},"min":{"1min":0.47699999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.78700000000000003,"5min":0.93500000000000005,"15min":1.3999999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64200000000000002,"15min":0.68200000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83599999999999997,"5min":0.91200000000000003,"15min":1.5209999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65800000000000003,"15min":0.68000000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84699999999999998,"5min":0.93600000000000005,"15min":1.161},"last":0.68000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":382998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.37,"15min":0.36399999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0109999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":0.93500000000000005,"15min":0.96799999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.69799999999999995},"min":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.44700000000000001},"max":{"1min":0.92100000000000004,"5min":0.99399999999999999,"15min":1.0109999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.40300000000000002},"max":{"1min":0.98099999999999998,"5min":0.99199999999999999,"15min":1.044},"last":0.67400000000000004}]},{"osd":2,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.72999999999999998},"min":{"1min":0.50800000000000001,"5min":0.496,"15min":0.42899999999999999},"max":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":1.024},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72999999999999998,"15min":0.71399999999999997},"min":{"1min":0.60499999999999998,"5min":0.54600000000000004,"15min":0.45600000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.0329999999999999},"last":0.621}]},{"osd":3,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89000000000000001,"5min":0.90900000000000003,"15min":0.93899999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72499999999999998,"15min":0.71499999999999997},"min":{"1min":0.57799999999999996,"5min":0.47899999999999998,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":1.02},"last":0.52600000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.61799999999999999},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.83899999999999997,"15min":0.93700000000000006},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.98599999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61499999999999999,"15min":0.63700000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.97599999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.70499999999999996,"15min":0.72699999999999998},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.105},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63,"15min":0.67000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.0009999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.90100000000000002,"5min":0.90800000000000003,"15min":0.95599999999999996},"last":0.73399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:57:06.054 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:57:06.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.283+0000 7f16b0fff700 1 Processor -- start 2023-11-06T20:57:06.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.283+0000 7f16b0fff700 1 -- start start 2023-11-06T20:57:06.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.284+0000 7f16b0fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f16ac0c5d50 0x7f16ac0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.284+0000 7f16b0fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16ac0bfd70 0x7f16ac0c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.284+0000 7f16b0fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac1554a0 con 0x7f16ac0c1840 2023-11-06T20:57:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.284+0000 7f16b0fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ac154960 con 0x7f16ac0bfd70 2023-11-06T20:57:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.284+0000 7f16b0fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ac0c66c0 con 0x7f16ac0c5d50 2023-11-06T20:57:06.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.285+0000 7f16aad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16ac0bfd70 0x7f16ac0c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.285+0000 7f16aad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16ac0bfd70 0x7f16ac0c0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43976/0 (socket says 172.21.15.33:43976) 2023-11-06T20:57:06.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.285+0000 7f16aad9d700 1 -- 172.21.15.33:0/153466858 learned_addr learned my addr 172.21.15.33:0/153466858 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:06.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.285+0000 7f16aa59c700 1 --1- 172.21.15.33:0/153466858 >> v1:172.21.15.33:6789/0 conn(0x7f16ac0c1840 0x7f16ac0a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48250/0 (socket says 172.21.15.33:48250) 2023-11-06T20:57:06.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.285+0000 7f16a9d9b700 1 --2- 172.21.15.33:0/153466858 >> v2:172.21.15.69:3300/0 conn(0x7f16ac0c5d50 0x7f16ac0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.285+0000 7f16aad9d700 1 -- 172.21.15.33:0/153466858 >> v2:172.21.15.69:3300/0 conn(0x7f16ac0c5d50 msgr2=0x7f16ac0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.285+0000 7f16aad9d700 1 --2- 172.21.15.33:0/153466858 >> v2:172.21.15.69:3300/0 conn(0x7f16ac0c5d50 0x7f16ac0c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.285+0000 7f16aad9d700 1 -- 172.21.15.33:0/153466858 >> v1:172.21.15.33:6789/0 conn(0x7f16ac0c1840 legacy=0x7f16ac0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.285+0000 7f16aad9d700 1 -- 172.21.15.33:0/153466858 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac155930 con 0x7f16ac0bfd70 2023-11-06T20:57:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.285+0000 7f16aad9d700 1 --2- 172.21.15.33:0/153466858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 0x7f16ac0c0180 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f16ac0adb40 tx=0x7f16a4009840).ready entity=mon.0 client_cookie=1e869e1bb5d2690f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16a959a700 1 -- 172.21.15.33:0/153466858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a400e070 con 0x7f16ac0bfd70 2023-11-06T20:57:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16a959a700 1 -- 172.21.15.33:0/153466858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a400ba50 con 0x7f16ac0bfd70 2023-11-06T20:57:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16a959a700 1 -- 172.21.15.33:0/153466858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a400bdd0 con 0x7f16ac0bfd70 2023-11-06T20:57:06.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16b0fff700 1 -- 172.21.15.33:0/153466858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 msgr2=0x7f16ac0c0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16b0fff700 1 --2- 172.21.15.33:0/153466858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 0x7f16ac0c0180 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f16ac0adb40 tx=0x7f16a4009840).stop 2023-11-06T20:57:06.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16b0fff700 1 -- 172.21.15.33:0/153466858 shutdown_connections 2023-11-06T20:57:06.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16b0fff700 1 --2- 172.21.15.33:0/153466858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 0x7f16ac0c0180 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16b0fff700 1 --2- 172.21.15.33:0/153466858 >> v2:172.21.15.69:3300/0 conn(0x7f16ac0c5d50 0x7f16ac0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16b0fff700 1 -- 172.21.15.33:0/153466858 >> 172.21.15.33:0/153466858 conn(0x7f16ac05ec80 msgr2=0x7f16ac0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:06.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16b0fff700 1 -- 172.21.15.33:0/153466858 shutdown_connections 2023-11-06T20:57:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.286+0000 7f16b0fff700 1 -- 172.21.15.33:0/153466858 wait complete. 2023-11-06T20:57:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.288+0000 7f16b0fff700 1 Processor -- start 2023-11-06T20:57:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.288+0000 7f16b0fff700 1 -- start start 2023-11-06T20:57:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16b0fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 0x7f16ac0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16b0fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16ac0c1840 0x7f16ac0a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16b0fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ac0c5d50 0x7f16ac0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16b0fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0af660 con 0x7f16ac0bfd70 2023-11-06T20:57:06.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16b0fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0aea50 con 0x7f16ac0c5d50 2023-11-06T20:57:06.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16b0fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0aed30 con 0x7f16ac0c1840 2023-11-06T20:57:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16aa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 0x7f16ac0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16aa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 0x7f16ac0afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43978/0 (socket says 172.21.15.33:43978) 2023-11-06T20:57:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16aa59c700 1 -- 172.21.15.33:0/2376995758 learned_addr learned my addr 172.21.15.33:0/2376995758 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16a9d9b700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16ac0c1840 0x7f16ac0a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.289+0000 7f16aad9d700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ac0c5d50 0x7f16ac0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f16aa59c700 1 -- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16ac0c1840 msgr2=0x7f16ac0a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f16aa59c700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16ac0c1840 0x7f16ac0a9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f16aa59c700 1 -- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ac0c5d50 msgr2=0x7f16ac0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f16aa59c700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ac0c5d50 0x7f16ac0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f16aa59c700 1 -- 172.21.15.33:0/2376995758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a4009ae0 con 0x7f16ac0bfd70 2023-11-06T20:57:06.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f16aa59c700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 0x7f16ac0afaf0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f169c007fd0 tx=0x7f169c007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:06.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f169a7fc700 1 -- 172.21.15.33:0/2376995758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0072e0 con 0x7f16ac0bfd70 2023-11-06T20:57:06.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f169a7fc700 1 -- 172.21.15.33:0/2376995758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c007440 con 0x7f16ac0bfd70 2023-11-06T20:57:06.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f169a7fc700 1 -- 172.21.15.33:0/2376995758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0058f0 con 0x7f16ac0bfd70 2023-11-06T20:57:06.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f16b0fff700 1 -- 172.21.15.33:0/2376995758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac0af010 con 0x7f16ac0bfd70 2023-11-06T20:57:06.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.290+0000 7f16b0fff700 1 -- 172.21.15.33:0/2376995758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac0ae660 con 0x7f16ac0bfd70 2023-11-06T20:57:06.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.291+0000 7f16b0fff700 1 -- 172.21.15.33:0/2376995758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ac0af940 con 0x7f16ac0bfd70 2023-11-06T20:57:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.292+0000 7f169a7fc700 1 -- 172.21.15.33:0/2376995758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f169c005ce0 con 0x7f16ac0bfd70 2023-11-06T20:57:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.293+0000 7f169a7fc700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f168c05f270 0x7f168c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.294+0000 7f169a7fc700 1 -- 172.21.15.33:0/2376995758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f169c079c30 con 0x7f16ac0bfd70 2023-11-06T20:57:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.294+0000 7f16a9d9b700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f168c05f270 0x7f168c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.294+0000 7f16a9d9b700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f168c05f270 0x7f168c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f16a0006fd0 tx=0x7f16a0008040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.298+0000 7f169a7fc700 1 -- 172.21.15.33:0/2376995758 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f169c04acc0 con 0x7f16ac0bfd70 2023-11-06T20:57:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.474+0000 7f16b0fff700 1 -- 172.21.15.33:0/2376995758 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16ac0af7a0 con 0x7f168c05f270 2023-11-06T20:57:06.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.485+0000 7f169a7fc700 1 -- 172.21.15.33:0/2376995758 <== mgr.5443 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361278 (secure 0 0 0) 0x7f16a000f630 con 0x7f168c05f270 2023-11-06T20:57:06.490 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:57:06.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 -- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f168c05f270 msgr2=0x7f168c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f168c05f270 0x7f168c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f16a0006fd0 tx=0x7f16a0008040).stop 2023-11-06T20:57:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 -- 172.21.15.33:0/2376995758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 msgr2=0x7f16ac0afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 0x7f16ac0afaf0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f169c007fd0 tx=0x7f169c007df0).stop 2023-11-06T20:57:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 -- 172.21.15.33:0/2376995758 shutdown_connections 2023-11-06T20:57:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f168c05f270 0x7f168c061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ac0c5d50 0x7f16ac0ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16ac0bfd70 0x7f16ac0afaf0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 --2- 172.21.15.33:0/2376995758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16ac0c1840 0x7f16ac0a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 -- 172.21.15.33:0/2376995758 >> 172.21.15.33:0/2376995758 conn(0x7f16ac05ec80 msgr2=0x7f16ac152600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:06.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 -- 172.21.15.33:0/2376995758 shutdown_connections 2023-11-06T20:57:06.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.490+0000 7f16b0fff700 1 -- 172.21.15.33:0/2376995758 wait complete. 2023-11-06T20:57:06.496 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:57:06.506 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T20:57:06.405056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411600,"kb_used_data":6160,"kb_used_omap":0,"kb_used_meta":2405376,"kb_avail":752563120,"statfs":{"total":773094113280,"available":770624634880,"internally_reserved":0,"allocated":6307840,"data_stored":3089540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2463105024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15003740,"num_objects":-12,"num_object_clones":0,"num_object_copies":-24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-698,"num_write_kb":-15551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-34,"num_keys_recovered":-10,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.003415"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.725908+0000","last_change":"2023-11-06T20:57:01.718268+0000","last_active":"2023-11-06T20:57:01.725908+0000","last_peered":"2023-11-06T20:57:01.725908+0000","last_clean":"2023-11-06T20:57:01.725908+0000","last_became_active":"2023-11-06T20:57:01.717974+0000","last_became_peered":"2023-11-06T20:57:01.717974+0000","last_unstale":"2023-11-06T20:57:01.725908+0000","last_undegraded":"2023-11-06T20:57:01.725908+0000","last_fullsized":"2023-11-06T20:57:01.725908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3f","version":"65'3","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.733970+0000","last_change":"2023-11-06T20:57:01.242760+0000","last_active":"2023-11-06T20:57:01.733970+0000","last_peered":"2023-11-06T20:57:01.733970+0000","last_clean":"2023-11-06T20:57:01.733970+0000","last_became_active":"2023-11-06T20:57:01.242475+0000","last_became_peered":"2023-11-06T20:57:01.242475+0000","last_unstale":"2023-11-06T20:57:01.733970+0000","last_undegraded":"2023-11-06T20:57:01.733970+0000","last_fullsized":"2023-11-06T20:57:01.733970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.721076+0000","last_change":"2023-11-06T20:57:01.713631+0000","last_active":"2023-11-06T20:57:01.721076+0000","last_peered":"2023-11-06T20:57:01.721076+0000","last_clean":"2023-11-06T20:57:01.721076+0000","last_became_active":"2023-11-06T20:57:01.713382+0000","last_became_peered":"2023-11-06T20:57:01.713382+0000","last_unstale":"2023-11-06T20:57:01.721076+0000","last_undegraded":"2023-11-06T20:57:01.721076+0000","last_fullsized":"2023-11-06T20:57:01.721076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.715919+0000","last_change":"2023-11-06T20:57:01.239333+0000","last_active":"2023-11-06T20:57:01.715919+0000","last_peered":"2023-11-06T20:57:01.715919+0000","last_clean":"2023-11-06T20:57:01.715919+0000","last_became_active":"2023-11-06T20:57:01.238959+0000","last_became_peered":"2023-11-06T20:57:01.238959+0000","last_unstale":"2023-11-06T20:57:01.715919+0000","last_undegraded":"2023-11-06T20:57:01.715919+0000","last_fullsized":"2023-11-06T20:57:01.715919+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.728793+0000","last_change":"2023-11-06T20:57:01.728793+0000","last_active":"2023-11-06T20:57:01.728793+0000","last_peered":"2023-11-06T20:57:01.728793+0000","last_clean":"2023-11-06T20:57:01.728793+0000","last_became_active":"2023-11-06T20:57:01.727768+0000","last_became_peered":"2023-11-06T20:57:01.727768+0000","last_unstale":"2023-11-06T20:57:01.728793+0000","last_undegraded":"2023-11-06T20:57:01.728793+0000","last_fullsized":"2023-11-06T20:57:01.728793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.705553+0000","last_change":"2023-11-06T20:57:01.228995+0000","last_active":"2023-11-06T20:57:01.705553+0000","last_peered":"2023-11-06T20:57:01.705553+0000","last_clean":"2023-11-06T20:57:01.705553+0000","last_became_active":"2023-11-06T20:57:01.228722+0000","last_became_peered":"2023-11-06T20:57:01.228722+0000","last_unstale":"2023-11-06T20:57:01.705553+0000","last_undegraded":"2023-11-06T20:57:01.705553+0000","last_fullsized":"2023-11-06T20:57:01.705553+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.725407+0000","last_change":"2023-11-06T20:57:01.717587+0000","last_active":"2023-11-06T20:57:01.725407+0000","last_peered":"2023-11-06T20:57:01.725407+0000","last_clean":"2023-11-06T20:57:01.725407+0000","last_became_active":"2023-11-06T20:57:01.716771+0000","last_became_peered":"2023-11-06T20:57:01.716771+0000","last_unstale":"2023-11-06T20:57:01.725407+0000","last_undegraded":"2023-11-06T20:57:01.725407+0000","last_fullsized":"2023-11-06T20:57:01.725407+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3c","version":"65'2","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.726755+0000","last_change":"2023-11-06T20:57:01.233364+0000","last_active":"2023-11-06T20:57:01.726755+0000","last_peered":"2023-11-06T20:57:01.726755+0000","last_clean":"2023-11-06T20:57:01.726755+0000","last_became_active":"2023-11-06T20:57:01.232904+0000","last_became_peered":"2023-11-06T20:57:01.232904+0000","last_unstale":"2023-11-06T20:57:01.726755+0000","last_undegraded":"2023-11-06T20:57:01.726755+0000","last_fullsized":"2023-11-06T20:57:01.726755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.725863+0000","last_change":"2023-11-06T20:57:01.719853+0000","last_active":"2023-11-06T20:57:01.725863+0000","last_peered":"2023-11-06T20:57:01.725863+0000","last_clean":"2023-11-06T20:57:01.725863+0000","last_became_active":"2023-11-06T20:57:01.719412+0000","last_became_peered":"2023-11-06T20:57:01.719412+0000","last_unstale":"2023-11-06T20:57:01.725863+0000","last_undegraded":"2023-11-06T20:57:01.725863+0000","last_fullsized":"2023-11-06T20:57:01.725863+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3b","version":"65'1","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.711956+0000","last_change":"2023-11-06T20:57:01.234583+0000","last_active":"2023-11-06T20:57:01.711956+0000","last_peered":"2023-11-06T20:57:01.711956+0000","last_clean":"2023-11-06T20:57:01.711956+0000","last_became_active":"2023-11-06T20:57:01.234254+0000","last_became_peered":"2023-11-06T20:57:01.234254+0000","last_unstale":"2023-11-06T20:57:01.711956+0000","last_undegraded":"2023-11-06T20:57:01.711956+0000","last_fullsized":"2023-11-06T20:57:01.711956+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.718325+0000","last_change":"2023-11-06T20:57:01.718325+0000","last_active":"2023-11-06T20:57:01.718325+0000","last_peered":"2023-11-06T20:57:01.718325+0000","last_clean":"2023-11-06T20:57:01.718325+0000","last_became_active":"2023-11-06T20:57:01.716714+0000","last_became_peered":"2023-11-06T20:57:01.716714+0000","last_unstale":"2023-11-06T20:57:01.718325+0000","last_undegraded":"2023-11-06T20:57:01.718325+0000","last_fullsized":"2023-11-06T20:57:01.718325+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3a","version":"65'1","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.712884+0000","last_change":"2023-11-06T20:57:01.241940+0000","last_active":"2023-11-06T20:57:01.712884+0000","last_peered":"2023-11-06T20:57:01.712884+0000","last_clean":"2023-11-06T20:57:01.712884+0000","last_became_active":"2023-11-06T20:57:01.241593+0000","last_became_peered":"2023-11-06T20:57:01.241593+0000","last_unstale":"2023-11-06T20:57:01.712884+0000","last_undegraded":"2023-11-06T20:57:01.712884+0000","last_fullsized":"2023-11-06T20:57:01.712884+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.726441+0000","last_change":"2023-11-06T20:57:01.726441+0000","last_active":"2023-11-06T20:57:01.726441+0000","last_peered":"2023-11-06T20:57:01.726441+0000","last_clean":"2023-11-06T20:57:01.726441+0000","last_became_active":"2023-11-06T20:57:01.726141+0000","last_became_peered":"2023-11-06T20:57:01.726141+0000","last_unstale":"2023-11-06T20:57:01.726441+0000","last_undegraded":"2023-11-06T20:57:01.726441+0000","last_fullsized":"2023-11-06T20:57:01.726441+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.701376+0000","last_change":"2023-11-06T20:57:01.232855+0000","last_active":"2023-11-06T20:57:01.701376+0000","last_peered":"2023-11-06T20:57:01.701376+0000","last_clean":"2023-11-06T20:57:01.701376+0000","last_became_active":"2023-11-06T20:57:01.232551+0000","last_became_peered":"2023-11-06T20:57:01.232551+0000","last_unstale":"2023-11-06T20:57:01.701376+0000","last_undegraded":"2023-11-06T20:57:01.701376+0000","last_fullsized":"2023-11-06T20:57:01.701376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.741481+0000","last_change":"2023-11-06T20:57:01.738246+0000","last_active":"2023-11-06T20:57:01.741481+0000","last_peered":"2023-11-06T20:57:01.741481+0000","last_clean":"2023-11-06T20:57:01.741481+0000","last_became_active":"2023-11-06T20:57:01.736501+0000","last_became_peered":"2023-11-06T20:57:01.736501+0000","last_unstale":"2023-11-06T20:57:01.741481+0000","last_undegraded":"2023-11-06T20:57:01.741481+0000","last_fullsized":"2023-11-06T20:57:01.741481+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.240683+0000","last_change":"2023-11-06T20:57:01.231722+0000","last_active":"2023-11-06T20:57:01.240683+0000","last_peered":"2023-11-06T20:57:01.240683+0000","last_clean":"2023-11-06T20:57:01.240683+0000","last_became_active":"2023-11-06T20:57:01.228910+0000","last_became_peered":"2023-11-06T20:57:01.228910+0000","last_unstale":"2023-11-06T20:57:01.240683+0000","last_undegraded":"2023-11-06T20:57:01.240683+0000","last_fullsized":"2023-11-06T20:57:01.240683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.725929+0000","last_change":"2023-11-06T20:57:01.720278+0000","last_active":"2023-11-06T20:57:01.725929+0000","last_peered":"2023-11-06T20:57:01.725929+0000","last_clean":"2023-11-06T20:57:01.725929+0000","last_became_active":"2023-11-06T20:57:01.719395+0000","last_became_peered":"2023-11-06T20:57:01.719395+0000","last_unstale":"2023-11-06T20:57:01.725929+0000","last_undegraded":"2023-11-06T20:57:01.725929+0000","last_fullsized":"2023-11-06T20:57:01.725929+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.237878+0000","last_change":"2023-11-06T20:57:01.230343+0000","last_active":"2023-11-06T20:57:01.237878+0000","last_peered":"2023-11-06T20:57:01.237878+0000","last_clean":"2023-11-06T20:57:01.237878+0000","last_became_active":"2023-11-06T20:57:01.230008+0000","last_became_peered":"2023-11-06T20:57:01.230008+0000","last_unstale":"2023-11-06T20:57:01.237878+0000","last_undegraded":"2023-11-06T20:57:01.237878+0000","last_fullsized":"2023-11-06T20:57:01.237878+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.730548+0000","last_change":"2023-11-06T20:57:01.730548+0000","last_active":"2023-11-06T20:57:01.730548+0000","last_peered":"2023-11-06T20:57:01.730548+0000","last_clean":"2023-11-06T20:57:01.730548+0000","last_became_active":"2023-11-06T20:57:01.730280+0000","last_became_peered":"2023-11-06T20:57:01.730280+0000","last_unstale":"2023-11-06T20:57:01.730548+0000","last_undegraded":"2023-11-06T20:57:01.730548+0000","last_fullsized":"2023-11-06T20:57:01.730548+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.713611+0000","last_change":"2023-11-06T20:57:01.239247+0000","last_active":"2023-11-06T20:57:01.713611+0000","last_peered":"2023-11-06T20:57:01.713611+0000","last_clean":"2023-11-06T20:57:01.713611+0000","last_became_active":"2023-11-06T20:57:01.238971+0000","last_became_peered":"2023-11-06T20:57:01.238971+0000","last_unstale":"2023-11-06T20:57:01.713611+0000","last_undegraded":"2023-11-06T20:57:01.713611+0000","last_fullsized":"2023-11-06T20:57:01.713611+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.732982+0000","last_change":"2023-11-06T20:57:01.728658+0000","last_active":"2023-11-06T20:57:01.732982+0000","last_peered":"2023-11-06T20:57:01.732982+0000","last_clean":"2023-11-06T20:57:01.732982+0000","last_became_active":"2023-11-06T20:57:01.728285+0000","last_became_peered":"2023-11-06T20:57:01.728285+0000","last_unstale":"2023-11-06T20:57:01.732982+0000","last_undegraded":"2023-11-06T20:57:01.732982+0000","last_fullsized":"2023-11-06T20:57:01.732982+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.239646+0000","last_change":"2023-11-06T20:57:01.234844+0000","last_active":"2023-11-06T20:57:01.239646+0000","last_peered":"2023-11-06T20:57:01.239646+0000","last_clean":"2023-11-06T20:57:01.239646+0000","last_became_active":"2023-11-06T20:57:01.234546+0000","last_became_peered":"2023-11-06T20:57:01.234546+0000","last_unstale":"2023-11-06T20:57:01.239646+0000","last_undegraded":"2023-11-06T20:57:01.239646+0000","last_fullsized":"2023-11-06T20:57:01.239646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.725390+0000","last_change":"2023-11-06T20:57:01.721838+0000","last_active":"2023-11-06T20:57:01.725390+0000","last_peered":"2023-11-06T20:57:01.725390+0000","last_clean":"2023-11-06T20:57:01.725390+0000","last_became_active":"2023-11-06T20:57:01.721573+0000","last_became_peered":"2023-11-06T20:57:01.721573+0000","last_unstale":"2023-11-06T20:57:01.725390+0000","last_undegraded":"2023-11-06T20:57:01.725390+0000","last_fullsized":"2023-11-06T20:57:01.725390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.34","version":"65'4","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.738391+0000","last_change":"2023-11-06T20:57:01.245149+0000","last_active":"2023-11-06T20:57:01.738391+0000","last_peered":"2023-11-06T20:57:01.738391+0000","last_clean":"2023-11-06T20:57:01.738391+0000","last_became_active":"2023-11-06T20:57:01.244895+0000","last_became_peered":"2023-11-06T20:57:01.244895+0000","last_unstale":"2023-11-06T20:57:01.738391+0000","last_undegraded":"2023-11-06T20:57:01.738391+0000","last_fullsized":"2023-11-06T20:57:01.738391+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.732834+0000","last_change":"2023-11-06T20:57:01.725511+0000","last_active":"2023-11-06T20:57:01.732834+0000","last_peered":"2023-11-06T20:57:01.732834+0000","last_clean":"2023-11-06T20:57:01.732834+0000","last_became_active":"2023-11-06T20:57:01.725241+0000","last_became_peered":"2023-11-06T20:57:01.725241+0000","last_unstale":"2023-11-06T20:57:01.732834+0000","last_undegraded":"2023-11-06T20:57:01.732834+0000","last_fullsized":"2023-11-06T20:57:01.732834+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.33","version":"65'1","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.711468+0000","last_change":"2023-11-06T20:57:01.242481+0000","last_active":"2023-11-06T20:57:01.711468+0000","last_peered":"2023-11-06T20:57:01.711468+0000","last_clean":"2023-11-06T20:57:01.711468+0000","last_became_active":"2023-11-06T20:57:01.242123+0000","last_became_peered":"2023-11-06T20:57:01.242123+0000","last_unstale":"2023-11-06T20:57:01.711468+0000","last_undegraded":"2023-11-06T20:57:01.711468+0000","last_fullsized":"2023-11-06T20:57:01.711468+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.743710+0000","last_change":"2023-11-06T20:57:01.738693+0000","last_active":"2023-11-06T20:57:01.743710+0000","last_peered":"2023-11-06T20:57:01.743710+0000","last_clean":"2023-11-06T20:57:01.743710+0000","last_became_active":"2023-11-06T20:57:01.738419+0000","last_became_peered":"2023-11-06T20:57:01.738419+0000","last_unstale":"2023-11-06T20:57:01.743710+0000","last_undegraded":"2023-11-06T20:57:01.743710+0000","last_fullsized":"2023-11-06T20:57:01.743710+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.239800+0000","last_change":"2023-11-06T20:57:01.232583+0000","last_active":"2023-11-06T20:57:01.239800+0000","last_peered":"2023-11-06T20:57:01.239800+0000","last_clean":"2023-11-06T20:57:01.239800+0000","last_became_active":"2023-11-06T20:57:01.232271+0000","last_became_peered":"2023-11-06T20:57:01.232271+0000","last_unstale":"2023-11-06T20:57:01.239800+0000","last_undegraded":"2023-11-06T20:57:01.239800+0000","last_fullsized":"2023-11-06T20:57:01.239800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.737096+0000","last_change":"2023-11-06T20:57:01.732460+0000","last_active":"2023-11-06T20:57:01.737096+0000","last_peered":"2023-11-06T20:57:01.737096+0000","last_clean":"2023-11-06T20:57:01.737096+0000","last_became_active":"2023-11-06T20:57:01.731966+0000","last_became_peered":"2023-11-06T20:57:01.731966+0000","last_unstale":"2023-11-06T20:57:01.737096+0000","last_undegraded":"2023-11-06T20:57:01.737096+0000","last_fullsized":"2023-11-06T20:57:01.737096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.240360+0000","last_change":"2023-11-06T20:57:01.236757+0000","last_active":"2023-11-06T20:57:01.240360+0000","last_peered":"2023-11-06T20:57:01.240360+0000","last_clean":"2023-11-06T20:57:01.240360+0000","last_became_active":"2023-11-06T20:57:01.236499+0000","last_became_peered":"2023-11-06T20:57:01.236499+0000","last_unstale":"2023-11-06T20:57:01.240360+0000","last_undegraded":"2023-11-06T20:57:01.240360+0000","last_fullsized":"2023-11-06T20:57:01.240360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.718765+0000","last_change":"2023-11-06T20:57:01.718765+0000","last_active":"2023-11-06T20:57:01.718765+0000","last_peered":"2023-11-06T20:57:01.718765+0000","last_clean":"2023-11-06T20:57:01.718765+0000","last_became_active":"2023-11-06T20:57:01.718114+0000","last_became_peered":"2023-11-06T20:57:01.718114+0000","last_unstale":"2023-11-06T20:57:01.718765+0000","last_undegraded":"2023-11-06T20:57:01.718765+0000","last_fullsized":"2023-11-06T20:57:01.718765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.30","version":"65'4","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.733959+0000","last_change":"2023-11-06T20:57:01.242441+0000","last_active":"2023-11-06T20:57:01.733959+0000","last_peered":"2023-11-06T20:57:01.733959+0000","last_clean":"2023-11-06T20:57:01.733959+0000","last_became_active":"2023-11-06T20:57:01.242091+0000","last_became_peered":"2023-11-06T20:57:01.242091+0000","last_unstale":"2023-11-06T20:57:01.733959+0000","last_undegraded":"2023-11-06T20:57:01.733959+0000","last_fullsized":"2023-11-06T20:57:01.733959+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.730874+0000","last_change":"2023-11-06T20:57:01.726311+0000","last_active":"2023-11-06T20:57:01.730874+0000","last_peered":"2023-11-06T20:57:01.730874+0000","last_clean":"2023-11-06T20:57:01.730874+0000","last_became_active":"2023-11-06T20:57:01.723035+0000","last_became_peered":"2023-11-06T20:57:01.723035+0000","last_unstale":"2023-11-06T20:57:01.730874+0000","last_undegraded":"2023-11-06T20:57:01.730874+0000","last_fullsized":"2023-11-06T20:57:01.730874+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.716238+0000","last_change":"2023-11-06T20:57:01.235645+0000","last_active":"2023-11-06T20:57:01.716238+0000","last_peered":"2023-11-06T20:57:01.716238+0000","last_clean":"2023-11-06T20:57:01.716238+0000","last_became_active":"2023-11-06T20:57:01.235393+0000","last_became_peered":"2023-11-06T20:57:01.235393+0000","last_unstale":"2023-11-06T20:57:01.716238+0000","last_undegraded":"2023-11-06T20:57:01.716238+0000","last_fullsized":"2023-11-06T20:57:01.716238+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.743815+0000","last_change":"2023-11-06T20:57:01.738952+0000","last_active":"2023-11-06T20:57:01.743815+0000","last_peered":"2023-11-06T20:57:01.743815+0000","last_clean":"2023-11-06T20:57:01.743815+0000","last_became_active":"2023-11-06T20:57:01.737232+0000","last_became_peered":"2023-11-06T20:57:01.737232+0000","last_unstale":"2023-11-06T20:57:01.743815+0000","last_undegraded":"2023-11-06T20:57:01.743815+0000","last_fullsized":"2023-11-06T20:57:01.743815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.706334+0000","last_change":"2023-11-06T20:57:01.242985+0000","last_active":"2023-11-06T20:57:01.706334+0000","last_peered":"2023-11-06T20:57:01.706334+0000","last_clean":"2023-11-06T20:57:01.706334+0000","last_became_active":"2023-11-06T20:57:01.242548+0000","last_became_peered":"2023-11-06T20:57:01.242548+0000","last_unstale":"2023-11-06T20:57:01.706334+0000","last_undegraded":"2023-11-06T20:57:01.706334+0000","last_fullsized":"2023-11-06T20:57:01.706334+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.739708+0000","last_change":"2023-11-06T20:57:01.738937+0000","last_active":"2023-11-06T20:57:01.739708+0000","last_peered":"2023-11-06T20:57:01.739708+0000","last_clean":"2023-11-06T20:57:01.739708+0000","last_became_active":"2023-11-06T20:57:01.738569+0000","last_became_peered":"2023-11-06T20:57:01.738569+0000","last_unstale":"2023-11-06T20:57:01.739708+0000","last_undegraded":"2023-11-06T20:57:01.739708+0000","last_fullsized":"2023-11-06T20:57:01.739708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2d","version":"65'1","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.709333+0000","last_change":"2023-11-06T20:57:01.236097+0000","last_active":"2023-11-06T20:57:01.709333+0000","last_peered":"2023-11-06T20:57:01.709333+0000","last_clean":"2023-11-06T20:57:01.709333+0000","last_became_active":"2023-11-06T20:57:01.235792+0000","last_became_peered":"2023-11-06T20:57:01.235792+0000","last_unstale":"2023-11-06T20:57:01.709333+0000","last_undegraded":"2023-11-06T20:57:01.709333+0000","last_fullsized":"2023-11-06T20:57:01.709333+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.728044+0000","last_change":"2023-11-06T20:57:01.728044+0000","last_active":"2023-11-06T20:57:01.728044+0000","last_peered":"2023-11-06T20:57:01.728044+0000","last_clean":"2023-11-06T20:57:01.728044+0000","last_became_active":"2023-11-06T20:57:01.727712+0000","last_became_peered":"2023-11-06T20:57:01.727712+0000","last_unstale":"2023-11-06T20:57:01.728044+0000","last_undegraded":"2023-11-06T20:57:01.728044+0000","last_fullsized":"2023-11-06T20:57:01.728044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.711133+0000","last_change":"2023-11-06T20:57:01.238500+0000","last_active":"2023-11-06T20:57:01.711133+0000","last_peered":"2023-11-06T20:57:01.711133+0000","last_clean":"2023-11-06T20:57:01.711133+0000","last_became_active":"2023-11-06T20:57:01.238216+0000","last_became_peered":"2023-11-06T20:57:01.238216+0000","last_unstale":"2023-11-06T20:57:01.711133+0000","last_undegraded":"2023-11-06T20:57:01.711133+0000","last_fullsized":"2023-11-06T20:57:01.711133+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736163+0000","last_change":"2023-11-06T20:57:01.730308+0000","last_active":"2023-11-06T20:57:01.736163+0000","last_peered":"2023-11-06T20:57:01.736163+0000","last_clean":"2023-11-06T20:57:01.736163+0000","last_became_active":"2023-11-06T20:57:01.730046+0000","last_became_peered":"2023-11-06T20:57:01.730046+0000","last_unstale":"2023-11-06T20:57:01.736163+0000","last_undegraded":"2023-11-06T20:57:01.736163+0000","last_fullsized":"2023-11-06T20:57:01.736163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"65'1","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.714192+0000","last_change":"2023-11-06T20:57:01.240329+0000","last_active":"2023-11-06T20:57:01.714192+0000","last_peered":"2023-11-06T20:57:01.714192+0000","last_clean":"2023-11-06T20:57:01.714192+0000","last_became_active":"2023-11-06T20:57:01.239949+0000","last_became_peered":"2023-11-06T20:57:01.239949+0000","last_unstale":"2023-11-06T20:57:01.714192+0000","last_undegraded":"2023-11-06T20:57:01.714192+0000","last_fullsized":"2023-11-06T20:57:01.714192+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.725999+0000","last_change":"2023-11-06T20:57:01.722033+0000","last_active":"2023-11-06T20:57:01.725999+0000","last_peered":"2023-11-06T20:57:01.725999+0000","last_clean":"2023-11-06T20:57:01.725999+0000","last_became_active":"2023-11-06T20:57:01.721470+0000","last_became_peered":"2023-11-06T20:57:01.721470+0000","last_unstale":"2023-11-06T20:57:01.725999+0000","last_undegraded":"2023-11-06T20:57:01.725999+0000","last_fullsized":"2023-11-06T20:57:01.725999+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.709795+0000","last_change":"2023-11-06T20:57:01.246682+0000","last_active":"2023-11-06T20:57:01.709795+0000","last_peered":"2023-11-06T20:57:01.709795+0000","last_clean":"2023-11-06T20:57:01.709795+0000","last_became_active":"2023-11-06T20:57:01.246367+0000","last_became_peered":"2023-11-06T20:57:01.246367+0000","last_unstale":"2023-11-06T20:57:01.709795+0000","last_undegraded":"2023-11-06T20:57:01.709795+0000","last_fullsized":"2023-11-06T20:57:01.709795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.743952+0000","last_change":"2023-11-06T20:57:01.739111+0000","last_active":"2023-11-06T20:57:01.743952+0000","last_peered":"2023-11-06T20:57:01.743952+0000","last_clean":"2023-11-06T20:57:01.743952+0000","last_became_active":"2023-11-06T20:57:01.738689+0000","last_became_peered":"2023-11-06T20:57:01.738689+0000","last_unstale":"2023-11-06T20:57:01.743952+0000","last_undegraded":"2023-11-06T20:57:01.743952+0000","last_fullsized":"2023-11-06T20:57:01.743952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.701945+0000","last_change":"2023-11-06T20:57:01.221081+0000","last_active":"2023-11-06T20:57:01.701945+0000","last_peered":"2023-11-06T20:57:01.701945+0000","last_clean":"2023-11-06T20:57:01.701945+0000","last_became_active":"2023-11-06T20:57:01.220690+0000","last_became_peered":"2023-11-06T20:57:01.220690+0000","last_unstale":"2023-11-06T20:57:01.701945+0000","last_undegraded":"2023-11-06T20:57:01.701945+0000","last_fullsized":"2023-11-06T20:57:01.701945+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.729875+0000","last_change":"2023-11-06T20:57:01.729875+0000","last_active":"2023-11-06T20:57:01.729875+0000","last_peered":"2023-11-06T20:57:01.729875+0000","last_clean":"2023-11-06T20:57:01.729875+0000","last_became_active":"2023-11-06T20:57:01.729601+0000","last_became_peered":"2023-11-06T20:57:01.729601+0000","last_unstale":"2023-11-06T20:57:01.729875+0000","last_undegraded":"2023-11-06T20:57:01.729875+0000","last_fullsized":"2023-11-06T20:57:01.729875+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.710324+0000","last_change":"2023-11-06T20:57:01.241721+0000","last_active":"2023-11-06T20:57:01.710324+0000","last_peered":"2023-11-06T20:57:01.710324+0000","last_clean":"2023-11-06T20:57:01.710324+0000","last_became_active":"2023-11-06T20:57:01.241410+0000","last_became_peered":"2023-11-06T20:57:01.241410+0000","last_unstale":"2023-11-06T20:57:01.710324+0000","last_undegraded":"2023-11-06T20:57:01.710324+0000","last_fullsized":"2023-11-06T20:57:01.710324+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.737315+0000","last_change":"2023-11-06T20:57:01.733968+0000","last_active":"2023-11-06T20:57:01.737315+0000","last_peered":"2023-11-06T20:57:01.737315+0000","last_clean":"2023-11-06T20:57:01.737315+0000","last_became_active":"2023-11-06T20:57:01.733669+0000","last_became_peered":"2023-11-06T20:57:01.733669+0000","last_unstale":"2023-11-06T20:57:01.737315+0000","last_undegraded":"2023-11-06T20:57:01.737315+0000","last_fullsized":"2023-11-06T20:57:01.737315+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.241702+0000","last_change":"2023-11-06T20:57:01.239365+0000","last_active":"2023-11-06T20:57:01.241702+0000","last_peered":"2023-11-06T20:57:01.241702+0000","last_clean":"2023-11-06T20:57:01.241702+0000","last_became_active":"2023-11-06T20:57:01.237278+0000","last_became_peered":"2023-11-06T20:57:01.237278+0000","last_unstale":"2023-11-06T20:57:01.241702+0000","last_undegraded":"2023-11-06T20:57:01.241702+0000","last_fullsized":"2023-11-06T20:57:01.241702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.743795+0000","last_change":"2023-11-06T20:57:01.737950+0000","last_active":"2023-11-06T20:57:01.743795+0000","last_peered":"2023-11-06T20:57:01.743795+0000","last_clean":"2023-11-06T20:57:01.743795+0000","last_became_active":"2023-11-06T20:57:01.737601+0000","last_became_peered":"2023-11-06T20:57:01.737601+0000","last_unstale":"2023-11-06T20:57:01.743795+0000","last_undegraded":"2023-11-06T20:57:01.743795+0000","last_fullsized":"2023-11-06T20:57:01.743795+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.242400+0000","last_change":"2023-11-06T20:57:01.238822+0000","last_active":"2023-11-06T20:57:01.242400+0000","last_peered":"2023-11-06T20:57:01.242400+0000","last_clean":"2023-11-06T20:57:01.242400+0000","last_became_active":"2023-11-06T20:57:01.238052+0000","last_became_peered":"2023-11-06T20:57:01.238052+0000","last_unstale":"2023-11-06T20:57:01.242400+0000","last_undegraded":"2023-11-06T20:57:01.242400+0000","last_fullsized":"2023-11-06T20:57:01.242400+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.743863+0000","last_change":"2023-11-06T20:57:01.739612+0000","last_active":"2023-11-06T20:57:01.743863+0000","last_peered":"2023-11-06T20:57:01.743863+0000","last_clean":"2023-11-06T20:57:01.743863+0000","last_became_active":"2023-11-06T20:57:01.738849+0000","last_became_peered":"2023-11-06T20:57:01.738849+0000","last_unstale":"2023-11-06T20:57:01.743863+0000","last_undegraded":"2023-11-06T20:57:01.743863+0000","last_fullsized":"2023-11-06T20:57:01.743863+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.704058+0000","last_change":"2023-11-06T20:57:01.227291+0000","last_active":"2023-11-06T20:57:01.704058+0000","last_peered":"2023-11-06T20:57:01.704058+0000","last_clean":"2023-11-06T20:57:01.704058+0000","last_became_active":"2023-11-06T20:57:01.226986+0000","last_became_peered":"2023-11-06T20:57:01.226986+0000","last_unstale":"2023-11-06T20:57:01.704058+0000","last_undegraded":"2023-11-06T20:57:01.704058+0000","last_fullsized":"2023-11-06T20:57:01.704058+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.725080+0000","last_change":"2023-11-06T20:57:01.718787+0000","last_active":"2023-11-06T20:57:01.725080+0000","last_peered":"2023-11-06T20:57:01.725080+0000","last_clean":"2023-11-06T20:57:01.725080+0000","last_became_active":"2023-11-06T20:57:01.718524+0000","last_became_peered":"2023-11-06T20:57:01.718524+0000","last_unstale":"2023-11-06T20:57:01.725080+0000","last_undegraded":"2023-11-06T20:57:01.725080+0000","last_fullsized":"2023-11-06T20:57:01.725080+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.702629+0000","last_change":"2023-11-06T20:57:01.236406+0000","last_active":"2023-11-06T20:57:01.702629+0000","last_peered":"2023-11-06T20:57:01.702629+0000","last_clean":"2023-11-06T20:57:01.702629+0000","last_became_active":"2023-11-06T20:57:01.236108+0000","last_became_peered":"2023-11-06T20:57:01.236108+0000","last_unstale":"2023-11-06T20:57:01.702629+0000","last_undegraded":"2023-11-06T20:57:01.702629+0000","last_fullsized":"2023-11-06T20:57:01.702629+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736152+0000","last_change":"2023-11-06T20:57:01.730183+0000","last_active":"2023-11-06T20:57:01.736152+0000","last_peered":"2023-11-06T20:57:01.736152+0000","last_clean":"2023-11-06T20:57:01.736152+0000","last_became_active":"2023-11-06T20:57:01.729990+0000","last_became_peered":"2023-11-06T20:57:01.729990+0000","last_unstale":"2023-11-06T20:57:01.736152+0000","last_undegraded":"2023-11-06T20:57:01.736152+0000","last_fullsized":"2023-11-06T20:57:01.736152+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.232546+0000","last_change":"2023-11-06T20:57:01.228668+0000","last_active":"2023-11-06T20:57:01.232546+0000","last_peered":"2023-11-06T20:57:01.232546+0000","last_clean":"2023-11-06T20:57:01.232546+0000","last_became_active":"2023-11-06T20:57:01.228385+0000","last_became_peered":"2023-11-06T20:57:01.228385+0000","last_unstale":"2023-11-06T20:57:01.232546+0000","last_undegraded":"2023-11-06T20:57:01.232546+0000","last_fullsized":"2023-11-06T20:57:01.232546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.741466+0000","last_change":"2023-11-06T20:57:01.738126+0000","last_active":"2023-11-06T20:57:01.741466+0000","last_peered":"2023-11-06T20:57:01.741466+0000","last_clean":"2023-11-06T20:57:01.741466+0000","last_became_active":"2023-11-06T20:57:01.737095+0000","last_became_peered":"2023-11-06T20:57:01.737095+0000","last_unstale":"2023-11-06T20:57:01.741466+0000","last_undegraded":"2023-11-06T20:57:01.741466+0000","last_fullsized":"2023-11-06T20:57:01.741466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.711685+0000","last_change":"2023-11-06T20:57:01.232740+0000","last_active":"2023-11-06T20:57:01.711685+0000","last_peered":"2023-11-06T20:57:01.711685+0000","last_clean":"2023-11-06T20:57:01.711685+0000","last_became_active":"2023-11-06T20:57:01.232391+0000","last_became_peered":"2023-11-06T20:57:01.232391+0000","last_unstale":"2023-11-06T20:57:01.711685+0000","last_undegraded":"2023-11-06T20:57:01.711685+0000","last_fullsized":"2023-11-06T20:57:01.711685+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.733753+0000","last_change":"2023-11-06T20:57:01.730057+0000","last_active":"2023-11-06T20:57:01.733753+0000","last_peered":"2023-11-06T20:57:01.733753+0000","last_clean":"2023-11-06T20:57:01.733753+0000","last_became_active":"2023-11-06T20:57:01.729724+0000","last_became_peered":"2023-11-06T20:57:01.729724+0000","last_unstale":"2023-11-06T20:57:01.733753+0000","last_undegraded":"2023-11-06T20:57:01.733753+0000","last_fullsized":"2023-11-06T20:57:01.733753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.232115+0000","last_change":"2023-11-06T20:57:01.226561+0000","last_active":"2023-11-06T20:57:01.232115+0000","last_peered":"2023-11-06T20:57:01.232115+0000","last_clean":"2023-11-06T20:57:01.232115+0000","last_became_active":"2023-11-06T20:57:01.226202+0000","last_became_peered":"2023-11-06T20:57:01.226202+0000","last_unstale":"2023-11-06T20:57:01.232115+0000","last_undegraded":"2023-11-06T20:57:01.232115+0000","last_fullsized":"2023-11-06T20:57:01.232115+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.723359+0000","last_change":"2023-11-06T20:57:01.723359+0000","last_active":"2023-11-06T20:57:01.723359+0000","last_peered":"2023-11-06T20:57:01.723359+0000","last_clean":"2023-11-06T20:57:01.723359+0000","last_became_active":"2023-11-06T20:57:01.723081+0000","last_became_peered":"2023-11-06T20:57:01.723081+0000","last_unstale":"2023-11-06T20:57:01.723359+0000","last_undegraded":"2023-11-06T20:57:01.723359+0000","last_fullsized":"2023-11-06T20:57:01.723359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.240648+0000","last_change":"2023-11-06T20:57:01.226867+0000","last_active":"2023-11-06T20:57:01.240648+0000","last_peered":"2023-11-06T20:57:01.240648+0000","last_clean":"2023-11-06T20:57:01.240648+0000","last_became_active":"2023-11-06T20:57:01.225791+0000","last_became_peered":"2023-11-06T20:57:01.225791+0000","last_unstale":"2023-11-06T20:57:01.240648+0000","last_undegraded":"2023-11-06T20:57:01.240648+0000","last_fullsized":"2023-11-06T20:57:01.240648+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.733001+0000","last_change":"2023-11-06T20:57:01.723581+0000","last_active":"2023-11-06T20:57:01.733001+0000","last_peered":"2023-11-06T20:57:01.733001+0000","last_clean":"2023-11-06T20:57:01.733001+0000","last_became_active":"2023-11-06T20:57:01.723176+0000","last_became_peered":"2023-11-06T20:57:01.723176+0000","last_unstale":"2023-11-06T20:57:01.733001+0000","last_undegraded":"2023-11-06T20:57:01.733001+0000","last_fullsized":"2023-11-06T20:57:01.733001+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"65'1","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.713742+0000","last_change":"2023-11-06T20:57:01.234613+0000","last_active":"2023-11-06T20:57:01.713742+0000","last_peered":"2023-11-06T20:57:01.713742+0000","last_clean":"2023-11-06T20:57:01.713742+0000","last_became_active":"2023-11-06T20:57:01.234271+0000","last_became_peered":"2023-11-06T20:57:01.234271+0000","last_unstale":"2023-11-06T20:57:01.713742+0000","last_undegraded":"2023-11-06T20:57:01.713742+0000","last_fullsized":"2023-11-06T20:57:01.713742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.711614+0000","last_change":"2023-11-06T20:57:01.708125+0000","last_active":"2023-11-06T20:57:01.711614+0000","last_peered":"2023-11-06T20:57:01.711614+0000","last_clean":"2023-11-06T20:57:01.711614+0000","last_became_active":"2023-11-06T20:57:01.707567+0000","last_became_peered":"2023-11-06T20:57:01.707567+0000","last_unstale":"2023-11-06T20:57:01.711614+0000","last_undegraded":"2023-11-06T20:57:01.711614+0000","last_fullsized":"2023-11-06T20:57:01.711614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.740353+0000","last_change":"2023-11-06T20:57:01.250224+0000","last_active":"2023-11-06T20:57:01.740353+0000","last_peered":"2023-11-06T20:57:01.740353+0000","last_clean":"2023-11-06T20:57:01.740353+0000","last_became_active":"2023-11-06T20:57:01.249815+0000","last_became_peered":"2023-11-06T20:57:01.249815+0000","last_unstale":"2023-11-06T20:57:01.740353+0000","last_undegraded":"2023-11-06T20:57:01.740353+0000","last_fullsized":"2023-11-06T20:57:01.740353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736089+0000","last_change":"2023-11-06T20:57:01.725908+0000","last_active":"2023-11-06T20:57:01.736089+0000","last_peered":"2023-11-06T20:57:01.736089+0000","last_clean":"2023-11-06T20:57:01.736089+0000","last_became_active":"2023-11-06T20:57:01.723634+0000","last_became_peered":"2023-11-06T20:57:01.723634+0000","last_unstale":"2023-11-06T20:57:01.736089+0000","last_undegraded":"2023-11-06T20:57:01.736089+0000","last_fullsized":"2023-11-06T20:57:01.736089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"65'2","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.720187+0000","last_change":"2023-11-06T20:57:01.229829+0000","last_active":"2023-11-06T20:57:01.720187+0000","last_peered":"2023-11-06T20:57:01.720187+0000","last_clean":"2023-11-06T20:57:01.720187+0000","last_became_active":"2023-11-06T20:57:01.229474+0000","last_became_peered":"2023-11-06T20:57:01.229474+0000","last_unstale":"2023-11-06T20:57:01.720187+0000","last_undegraded":"2023-11-06T20:57:01.720187+0000","last_fullsized":"2023-11-06T20:57:01.720187+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.739669+0000","last_change":"2023-11-06T20:57:01.739669+0000","last_active":"2023-11-06T20:57:01.739669+0000","last_peered":"2023-11-06T20:57:01.739669+0000","last_clean":"2023-11-06T20:57:01.739669+0000","last_became_active":"2023-11-06T20:57:01.739374+0000","last_became_peered":"2023-11-06T20:57:01.739374+0000","last_unstale":"2023-11-06T20:57:01.739669+0000","last_undegraded":"2023-11-06T20:57:01.739669+0000","last_fullsized":"2023-11-06T20:57:01.739669+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.712809+0000","last_change":"2023-11-06T20:57:01.239403+0000","last_active":"2023-11-06T20:57:01.712809+0000","last_peered":"2023-11-06T20:57:01.712809+0000","last_clean":"2023-11-06T20:57:01.712809+0000","last_became_active":"2023-11-06T20:57:01.239044+0000","last_became_peered":"2023-11-06T20:57:01.239044+0000","last_unstale":"2023-11-06T20:57:01.712809+0000","last_undegraded":"2023-11-06T20:57:01.712809+0000","last_fullsized":"2023-11-06T20:57:01.712809+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.724591+0000","last_change":"2023-11-06T20:57:01.724591+0000","last_active":"2023-11-06T20:57:01.724591+0000","last_peered":"2023-11-06T20:57:01.724591+0000","last_clean":"2023-11-06T20:57:01.724591+0000","last_became_active":"2023-11-06T20:57:01.724263+0000","last_became_peered":"2023-11-06T20:57:01.724263+0000","last_unstale":"2023-11-06T20:57:01.724591+0000","last_undegraded":"2023-11-06T20:57:01.724591+0000","last_fullsized":"2023-11-06T20:57:01.724591+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.725710+0000","last_change":"2023-11-06T20:57:01.242318+0000","last_active":"2023-11-06T20:57:01.725710+0000","last_peered":"2023-11-06T20:57:01.725710+0000","last_clean":"2023-11-06T20:57:01.725710+0000","last_became_active":"2023-11-06T20:57:01.241262+0000","last_became_peered":"2023-11-06T20:57:01.241262+0000","last_unstale":"2023-11-06T20:57:01.725710+0000","last_undegraded":"2023-11-06T20:57:01.725710+0000","last_fullsized":"2023-11-06T20:57:01.725710+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.719965+0000","last_change":"2023-11-06T20:57:01.713319+0000","last_active":"2023-11-06T20:57:01.719965+0000","last_peered":"2023-11-06T20:57:01.719965+0000","last_clean":"2023-11-06T20:57:01.719965+0000","last_became_active":"2023-11-06T20:57:01.713016+0000","last_became_peered":"2023-11-06T20:57:01.713016+0000","last_unstale":"2023-11-06T20:57:01.719965+0000","last_undegraded":"2023-11-06T20:57:01.719965+0000","last_fullsized":"2023-11-06T20:57:01.719965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.703237+0000","last_change":"2023-11-06T20:57:01.227061+0000","last_active":"2023-11-06T20:57:01.703237+0000","last_peered":"2023-11-06T20:57:01.703237+0000","last_clean":"2023-11-06T20:57:01.703237+0000","last_became_active":"2023-11-06T20:57:01.226778+0000","last_became_peered":"2023-11-06T20:57:01.226778+0000","last_unstale":"2023-11-06T20:57:01.703237+0000","last_undegraded":"2023-11-06T20:57:01.703237+0000","last_fullsized":"2023-11-06T20:57:01.703237+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.741768+0000","last_change":"2023-11-06T20:57:01.741768+0000","last_active":"2023-11-06T20:57:01.741768+0000","last_peered":"2023-11-06T20:57:01.741768+0000","last_clean":"2023-11-06T20:57:01.741768+0000","last_became_active":"2023-11-06T20:57:01.741289+0000","last_became_peered":"2023-11-06T20:57:01.741289+0000","last_unstale":"2023-11-06T20:57:01.741768+0000","last_undegraded":"2023-11-06T20:57:01.741768+0000","last_fullsized":"2023-11-06T20:57:01.741768+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.702041+0000","last_change":"2023-11-06T20:57:01.226336+0000","last_active":"2023-11-06T20:57:01.702041+0000","last_peered":"2023-11-06T20:57:01.702041+0000","last_clean":"2023-11-06T20:57:01.702041+0000","last_became_active":"2023-11-06T20:57:01.226022+0000","last_became_peered":"2023-11-06T20:57:01.226022+0000","last_unstale":"2023-11-06T20:57:01.702041+0000","last_undegraded":"2023-11-06T20:57:01.702041+0000","last_fullsized":"2023-11-06T20:57:01.702041+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.721125+0000","last_change":"2023-11-06T20:57:01.715241+0000","last_active":"2023-11-06T20:57:01.721125+0000","last_peered":"2023-11-06T20:57:01.721125+0000","last_clean":"2023-11-06T20:57:01.721125+0000","last_became_active":"2023-11-06T20:57:01.714961+0000","last_became_peered":"2023-11-06T20:57:01.714961+0000","last_unstale":"2023-11-06T20:57:01.721125+0000","last_undegraded":"2023-11-06T20:57:01.721125+0000","last_fullsized":"2023-11-06T20:57:01.721125+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.710243+0000","last_change":"2023-11-06T20:57:01.233422+0000","last_active":"2023-11-06T20:57:01.710243+0000","last_peered":"2023-11-06T20:57:01.710243+0000","last_clean":"2023-11-06T20:57:01.710243+0000","last_became_active":"2023-11-06T20:57:01.233095+0000","last_became_peered":"2023-11-06T20:57:01.233095+0000","last_unstale":"2023-11-06T20:57:01.710243+0000","last_undegraded":"2023-11-06T20:57:01.710243+0000","last_fullsized":"2023-11-06T20:57:01.710243+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736864+0000","last_change":"2023-11-06T20:57:01.736864+0000","last_active":"2023-11-06T20:57:01.736864+0000","last_peered":"2023-11-06T20:57:01.736864+0000","last_clean":"2023-11-06T20:57:01.736864+0000","last_became_active":"2023-11-06T20:57:01.736589+0000","last_became_peered":"2023-11-06T20:57:01.736589+0000","last_unstale":"2023-11-06T20:57:01.736864+0000","last_undegraded":"2023-11-06T20:57:01.736864+0000","last_fullsized":"2023-11-06T20:57:01.736864+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.706638+0000","last_change":"2023-11-06T20:57:01.226119+0000","last_active":"2023-11-06T20:57:01.706638+0000","last_peered":"2023-11-06T20:57:01.706638+0000","last_clean":"2023-11-06T20:57:01.706638+0000","last_became_active":"2023-11-06T20:57:01.225602+0000","last_became_peered":"2023-11-06T20:57:01.225602+0000","last_unstale":"2023-11-06T20:57:01.706638+0000","last_undegraded":"2023-11-06T20:57:01.706638+0000","last_fullsized":"2023-11-06T20:57:01.706638+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.733696+0000","last_change":"2023-11-06T20:57:01.728111+0000","last_active":"2023-11-06T20:57:01.733696+0000","last_peered":"2023-11-06T20:57:01.733696+0000","last_clean":"2023-11-06T20:57:01.733696+0000","last_became_active":"2023-11-06T20:57:01.726519+0000","last_became_peered":"2023-11-06T20:57:01.726519+0000","last_unstale":"2023-11-06T20:57:01.733696+0000","last_undegraded":"2023-11-06T20:57:01.733696+0000","last_fullsized":"2023-11-06T20:57:01.733696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.706455+0000","last_change":"2023-11-06T20:57:01.242821+0000","last_active":"2023-11-06T20:57:01.706455+0000","last_peered":"2023-11-06T20:57:01.706455+0000","last_clean":"2023-11-06T20:57:01.706455+0000","last_became_active":"2023-11-06T20:57:01.241628+0000","last_became_peered":"2023-11-06T20:57:01.241628+0000","last_unstale":"2023-11-06T20:57:01.706455+0000","last_undegraded":"2023-11-06T20:57:01.706455+0000","last_fullsized":"2023-11-06T20:57:01.706455+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.721821+0000","last_change":"2023-11-06T20:57:01.721821+0000","last_active":"2023-11-06T20:57:01.721821+0000","last_peered":"2023-11-06T20:57:01.721821+0000","last_clean":"2023-11-06T20:57:01.721821+0000","last_became_active":"2023-11-06T20:57:01.721440+0000","last_became_peered":"2023-11-06T20:57:01.721440+0000","last_unstale":"2023-11-06T20:57:01.721821+0000","last_undegraded":"2023-11-06T20:57:01.721821+0000","last_fullsized":"2023-11-06T20:57:01.721821+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"65'3","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736333+0000","last_change":"2023-11-06T20:57:01.235203+0000","last_active":"2023-11-06T20:57:01.736333+0000","last_peered":"2023-11-06T20:57:01.736333+0000","last_clean":"2023-11-06T20:57:01.736333+0000","last_became_active":"2023-11-06T20:57:01.234941+0000","last_became_peered":"2023-11-06T20:57:01.234941+0000","last_unstale":"2023-11-06T20:57:01.736333+0000","last_undegraded":"2023-11-06T20:57:01.736333+0000","last_fullsized":"2023-11-06T20:57:01.736333+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.739281+0000","last_change":"2023-11-06T20:57:01.736992+0000","last_active":"2023-11-06T20:57:01.739281+0000","last_peered":"2023-11-06T20:57:01.739281+0000","last_clean":"2023-11-06T20:57:01.739281+0000","last_became_active":"2023-11-06T20:57:01.736697+0000","last_became_peered":"2023-11-06T20:57:01.736697+0000","last_unstale":"2023-11-06T20:57:01.739281+0000","last_undegraded":"2023-11-06T20:57:01.739281+0000","last_fullsized":"2023-11-06T20:57:01.739281+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"65'2","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736596+0000","last_change":"2023-11-06T20:57:01.236941+0000","last_active":"2023-11-06T20:57:01.736596+0000","last_peered":"2023-11-06T20:57:01.736596+0000","last_clean":"2023-11-06T20:57:01.736596+0000","last_became_active":"2023-11-06T20:57:01.232758+0000","last_became_peered":"2023-11-06T20:57:01.232758+0000","last_unstale":"2023-11-06T20:57:01.736596+0000","last_undegraded":"2023-11-06T20:57:01.736596+0000","last_fullsized":"2023-11-06T20:57:01.736596+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.736164+0000","last_change":"2023-11-06T20:57:01.730062+0000","last_active":"2023-11-06T20:57:01.736164+0000","last_peered":"2023-11-06T20:57:01.736164+0000","last_clean":"2023-11-06T20:57:01.736164+0000","last_became_active":"2023-11-06T20:57:01.729718+0000","last_became_peered":"2023-11-06T20:57:01.729718+0000","last_unstale":"2023-11-06T20:57:01.736164+0000","last_undegraded":"2023-11-06T20:57:01.736164+0000","last_fullsized":"2023-11-06T20:57:01.736164+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.727193+0000","last_change":"2023-11-06T20:57:01.245233+0000","last_active":"2023-11-06T20:57:01.727193+0000","last_peered":"2023-11-06T20:57:01.727193+0000","last_clean":"2023-11-06T20:57:01.727193+0000","last_became_active":"2023-11-06T20:57:01.244685+0000","last_became_peered":"2023-11-06T20:57:01.244685+0000","last_unstale":"2023-11-06T20:57:01.727193+0000","last_undegraded":"2023-11-06T20:57:01.727193+0000","last_fullsized":"2023-11-06T20:57:01.727193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.727158+0000","last_change":"2023-11-06T20:57:01.727158+0000","last_active":"2023-11-06T20:57:01.727158+0000","last_peered":"2023-11-06T20:57:01.727158+0000","last_clean":"2023-11-06T20:57:01.727158+0000","last_became_active":"2023-11-06T20:57:01.726867+0000","last_became_peered":"2023-11-06T20:57:01.726867+0000","last_unstale":"2023-11-06T20:57:01.727158+0000","last_undegraded":"2023-11-06T20:57:01.727158+0000","last_fullsized":"2023-11-06T20:57:01.727158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.705129+0000","last_change":"2023-11-06T20:57:01.238335+0000","last_active":"2023-11-06T20:57:01.705129+0000","last_peered":"2023-11-06T20:57:01.705129+0000","last_clean":"2023-11-06T20:57:01.705129+0000","last_became_active":"2023-11-06T20:57:01.238044+0000","last_became_peered":"2023-11-06T20:57:01.238044+0000","last_unstale":"2023-11-06T20:57:01.705129+0000","last_undegraded":"2023-11-06T20:57:01.705129+0000","last_fullsized":"2023-11-06T20:57:01.705129+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.733851+0000","last_change":"2023-11-06T20:57:01.731774+0000","last_active":"2023-11-06T20:57:01.733851+0000","last_peered":"2023-11-06T20:57:01.733851+0000","last_clean":"2023-11-06T20:57:01.733851+0000","last_became_active":"2023-11-06T20:57:01.731418+0000","last_became_peered":"2023-11-06T20:57:01.731418+0000","last_unstale":"2023-11-06T20:57:01.733851+0000","last_undegraded":"2023-11-06T20:57:01.733851+0000","last_fullsized":"2023-11-06T20:57:01.733851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.711043+0000","last_change":"2023-11-06T20:57:01.233057+0000","last_active":"2023-11-06T20:57:01.711043+0000","last_peered":"2023-11-06T20:57:01.711043+0000","last_clean":"2023-11-06T20:57:01.711043+0000","last_became_active":"2023-11-06T20:57:01.232743+0000","last_became_peered":"2023-11-06T20:57:01.232743+0000","last_unstale":"2023-11-06T20:57:01.711043+0000","last_undegraded":"2023-11-06T20:57:01.711043+0000","last_fullsized":"2023-11-06T20:57:01.711043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.738605+0000","last_change":"2023-11-06T20:57:01.733691+0000","last_active":"2023-11-06T20:57:01.738605+0000","last_peered":"2023-11-06T20:57:01.738605+0000","last_clean":"2023-11-06T20:57:01.738605+0000","last_became_active":"2023-11-06T20:57:01.733227+0000","last_became_peered":"2023-11-06T20:57:01.733227+0000","last_unstale":"2023-11-06T20:57:01.738605+0000","last_undegraded":"2023-11-06T20:57:01.738605+0000","last_fullsized":"2023-11-06T20:57:01.738605+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"65'4","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.737449+0000","last_change":"2023-11-06T20:57:01.239660+0000","last_active":"2023-11-06T20:57:01.737449+0000","last_peered":"2023-11-06T20:57:01.737449+0000","last_clean":"2023-11-06T20:57:01.737449+0000","last_became_active":"2023-11-06T20:57:01.239361+0000","last_became_peered":"2023-11-06T20:57:01.239361+0000","last_unstale":"2023-11-06T20:57:01.737449+0000","last_undegraded":"2023-11-06T20:57:01.737449+0000","last_fullsized":"2023-11-06T20:57:01.737449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.719541+0000","last_change":"2023-11-06T20:57:01.719541+0000","last_active":"2023-11-06T20:57:01.719541+0000","last_peered":"2023-11-06T20:57:01.719541+0000","last_clean":"2023-11-06T20:57:01.719541+0000","last_became_active":"2023-11-06T20:57:01.719250+0000","last_became_peered":"2023-11-06T20:57:01.719250+0000","last_unstale":"2023-11-06T20:57:01.719541+0000","last_undegraded":"2023-11-06T20:57:01.719541+0000","last_fullsized":"2023-11-06T20:57:01.719541+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.740624+0000","last_change":"2023-11-06T20:57:01.250155+0000","last_active":"2023-11-06T20:57:01.740624+0000","last_peered":"2023-11-06T20:57:01.740624+0000","last_clean":"2023-11-06T20:57:01.740624+0000","last_became_active":"2023-11-06T20:57:01.249854+0000","last_became_peered":"2023-11-06T20:57:01.249854+0000","last_unstale":"2023-11-06T20:57:01.740624+0000","last_undegraded":"2023-11-06T20:57:01.740624+0000","last_fullsized":"2023-11-06T20:57:01.740624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.739636+0000","last_change":"2023-11-06T20:57:01.739636+0000","last_active":"2023-11-06T20:57:01.739636+0000","last_peered":"2023-11-06T20:57:01.739636+0000","last_clean":"2023-11-06T20:57:01.739636+0000","last_became_active":"2023-11-06T20:57:01.739369+0000","last_became_peered":"2023-11-06T20:57:01.739369+0000","last_unstale":"2023-11-06T20:57:01.739636+0000","last_undegraded":"2023-11-06T20:57:01.739636+0000","last_fullsized":"2023-11-06T20:57:01.739636+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.726635+0000","last_change":"2023-11-06T20:57:01.246761+0000","last_active":"2023-11-06T20:57:01.726635+0000","last_peered":"2023-11-06T20:57:01.726635+0000","last_clean":"2023-11-06T20:57:01.726635+0000","last_became_active":"2023-11-06T20:57:01.246459+0000","last_became_peered":"2023-11-06T20:57:01.246459+0000","last_unstale":"2023-11-06T20:57:01.726635+0000","last_undegraded":"2023-11-06T20:57:01.726635+0000","last_fullsized":"2023-11-06T20:57:01.726635+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.729818+0000","last_change":"2023-11-06T20:57:01.726191+0000","last_active":"2023-11-06T20:57:01.729818+0000","last_peered":"2023-11-06T20:57:01.729818+0000","last_clean":"2023-11-06T20:57:01.729818+0000","last_became_active":"2023-11-06T20:57:01.725892+0000","last_became_peered":"2023-11-06T20:57:01.725892+0000","last_unstale":"2023-11-06T20:57:01.729818+0000","last_undegraded":"2023-11-06T20:57:01.729818+0000","last_fullsized":"2023-11-06T20:57:01.729818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.705634+0000","last_change":"2023-11-06T20:57:01.228976+0000","last_active":"2023-11-06T20:57:01.705634+0000","last_peered":"2023-11-06T20:57:01.705634+0000","last_clean":"2023-11-06T20:57:01.705634+0000","last_became_active":"2023-11-06T20:57:01.228700+0000","last_became_peered":"2023-11-06T20:57:01.228700+0000","last_unstale":"2023-11-06T20:57:01.705634+0000","last_undegraded":"2023-11-06T20:57:01.705634+0000","last_fullsized":"2023-11-06T20:57:01.705634+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.727312+0000","last_change":"2023-11-06T20:57:01.722322+0000","last_active":"2023-11-06T20:57:01.727312+0000","last_peered":"2023-11-06T20:57:01.727312+0000","last_clean":"2023-11-06T20:57:01.727312+0000","last_became_active":"2023-11-06T20:57:01.721956+0000","last_became_peered":"2023-11-06T20:57:01.721956+0000","last_unstale":"2023-11-06T20:57:01.727312+0000","last_undegraded":"2023-11-06T20:57:01.727312+0000","last_fullsized":"2023-11-06T20:57:01.727312+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.711591+0000","last_change":"2023-11-06T20:57:01.242471+0000","last_active":"2023-11-06T20:57:01.711591+0000","last_peered":"2023-11-06T20:57:01.711591+0000","last_clean":"2023-11-06T20:57:01.711591+0000","last_became_active":"2023-11-06T20:57:01.241383+0000","last_became_peered":"2023-11-06T20:57:01.241383+0000","last_unstale":"2023-11-06T20:57:01.711591+0000","last_undegraded":"2023-11-06T20:57:01.711591+0000","last_fullsized":"2023-11-06T20:57:01.711591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.725119+0000","last_change":"2023-11-06T20:57:01.720698+0000","last_active":"2023-11-06T20:57:01.725119+0000","last_peered":"2023-11-06T20:57:01.725119+0000","last_clean":"2023-11-06T20:57:01.725119+0000","last_became_active":"2023-11-06T20:57:01.717040+0000","last_became_peered":"2023-11-06T20:57:01.717040+0000","last_unstale":"2023-11-06T20:57:01.725119+0000","last_undegraded":"2023-11-06T20:57:01.725119+0000","last_fullsized":"2023-11-06T20:57:01.725119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"65'1","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.711575+0000","last_change":"2023-11-06T20:57:01.237562+0000","last_active":"2023-11-06T20:57:01.711575+0000","last_peered":"2023-11-06T20:57:01.711575+0000","last_clean":"2023-11-06T20:57:01.711575+0000","last_became_active":"2023-11-06T20:57:01.237280+0000","last_became_peered":"2023-11-06T20:57:01.237280+0000","last_unstale":"2023-11-06T20:57:01.711575+0000","last_undegraded":"2023-11-06T20:57:01.711575+0000","last_fullsized":"2023-11-06T20:57:01.711575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.722529+0000","last_change":"2023-11-06T20:57:01.717541+0000","last_active":"2023-11-06T20:57:01.722529+0000","last_peered":"2023-11-06T20:57:01.722529+0000","last_clean":"2023-11-06T20:57:01.722529+0000","last_became_active":"2023-11-06T20:57:01.716724+0000","last_became_peered":"2023-11-06T20:57:01.716724+0000","last_unstale":"2023-11-06T20:57:01.722529+0000","last_undegraded":"2023-11-06T20:57:01.722529+0000","last_fullsized":"2023-11-06T20:57:01.722529+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.721132+0000","last_change":"2023-11-06T20:57:01.243821+0000","last_active":"2023-11-06T20:57:01.721132+0000","last_peered":"2023-11-06T20:57:01.721132+0000","last_clean":"2023-11-06T20:57:01.721132+0000","last_became_active":"2023-11-06T20:57:01.242179+0000","last_became_peered":"2023-11-06T20:57:01.242179+0000","last_unstale":"2023-11-06T20:57:01.721132+0000","last_undegraded":"2023-11-06T20:57:01.721132+0000","last_fullsized":"2023-11-06T20:57:01.721132+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.718755+0000","last_change":"2023-11-06T20:57:01.718755+0000","last_active":"2023-11-06T20:57:01.718755+0000","last_peered":"2023-11-06T20:57:01.718755+0000","last_clean":"2023-11-06T20:57:01.718755+0000","last_became_active":"2023-11-06T20:57:01.718485+0000","last_became_peered":"2023-11-06T20:57:01.718485+0000","last_unstale":"2023-11-06T20:57:01.718755+0000","last_undegraded":"2023-11-06T20:57:01.718755+0000","last_fullsized":"2023-11-06T20:57:01.718755+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.239643+0000","last_change":"2023-11-06T20:57:01.232750+0000","last_active":"2023-11-06T20:57:01.239643+0000","last_peered":"2023-11-06T20:57:01.239643+0000","last_clean":"2023-11-06T20:57:01.239643+0000","last_became_active":"2023-11-06T20:57:01.232418+0000","last_became_peered":"2023-11-06T20:57:01.232418+0000","last_unstale":"2023-11-06T20:57:01.239643+0000","last_undegraded":"2023-11-06T20:57:01.239643+0000","last_fullsized":"2023-11-06T20:57:01.239643+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.732937+0000","last_change":"2023-11-06T20:57:01.728833+0000","last_active":"2023-11-06T20:57:01.732937+0000","last_peered":"2023-11-06T20:57:01.732937+0000","last_clean":"2023-11-06T20:57:01.732937+0000","last_became_active":"2023-11-06T20:57:01.728563+0000","last_became_peered":"2023-11-06T20:57:01.728563+0000","last_unstale":"2023-11-06T20:57:01.732937+0000","last_undegraded":"2023-11-06T20:57:01.732937+0000","last_fullsized":"2023-11-06T20:57:01.732937+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.240768+0000","last_change":"2023-11-06T20:57:01.235238+0000","last_active":"2023-11-06T20:57:01.240768+0000","last_peered":"2023-11-06T20:57:01.240768+0000","last_clean":"2023-11-06T20:57:01.240768+0000","last_became_active":"2023-11-06T20:57:01.234880+0000","last_became_peered":"2023-11-06T20:57:01.234880+0000","last_unstale":"2023-11-06T20:57:01.240768+0000","last_undegraded":"2023-11-06T20:57:01.240768+0000","last_fullsized":"2023-11-06T20:57:01.240768+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.738813+0000","last_change":"2023-11-06T20:57:01.738813+0000","last_active":"2023-11-06T20:57:01.738813+0000","last_peered":"2023-11-06T20:57:01.738813+0000","last_clean":"2023-11-06T20:57:01.738813+0000","last_became_active":"2023-11-06T20:57:01.738296+0000","last_became_peered":"2023-11-06T20:57:01.738296+0000","last_unstale":"2023-11-06T20:57:01.738813+0000","last_undegraded":"2023-11-06T20:57:01.738813+0000","last_fullsized":"2023-11-06T20:57:01.738813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.710946+0000","last_change":"2023-11-06T20:57:01.232748+0000","last_active":"2023-11-06T20:57:01.710946+0000","last_peered":"2023-11-06T20:57:01.710946+0000","last_clean":"2023-11-06T20:57:01.710946+0000","last_became_active":"2023-11-06T20:57:01.232368+0000","last_became_peered":"2023-11-06T20:57:01.232368+0000","last_unstale":"2023-11-06T20:57:01.710946+0000","last_undegraded":"2023-11-06T20:57:01.710946+0000","last_fullsized":"2023-11-06T20:57:01.710946+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.223310+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:57:01.223310+0000","last_peered":"2023-11-06T20:57:01.223310+0000","last_clean":"2023-11-06T20:57:01.223310+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:57:01.223310+0000","last_undegraded":"2023-11-06T20:57:01.223310+0000","last_fullsized":"2023-11-06T20:57:01.223310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.722559+0000","last_change":"2023-11-06T20:57:01.717644+0000","last_active":"2023-11-06T20:57:01.722559+0000","last_peered":"2023-11-06T20:57:01.722559+0000","last_clean":"2023-11-06T20:57:01.722559+0000","last_became_active":"2023-11-06T20:57:01.717265+0000","last_became_peered":"2023-11-06T20:57:01.717265+0000","last_unstale":"2023-11-06T20:57:01.722559+0000","last_undegraded":"2023-11-06T20:57:01.722559+0000","last_fullsized":"2023-11-06T20:57:01.722559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.734944+0000","last_change":"2023-11-06T20:57:01.228436+0000","last_active":"2023-11-06T20:57:01.734944+0000","last_peered":"2023-11-06T20:57:01.734944+0000","last_clean":"2023-11-06T20:57:01.734944+0000","last_became_active":"2023-11-06T20:57:01.228153+0000","last_became_peered":"2023-11-06T20:57:01.228153+0000","last_unstale":"2023-11-06T20:57:01.734944+0000","last_undegraded":"2023-11-06T20:57:01.734944+0000","last_fullsized":"2023-11-06T20:57:01.734944+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.703321+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:57:01.703321+0000","last_peered":"2023-11-06T20:57:01.703321+0000","last_clean":"2023-11-06T20:57:01.703321+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:57:01.703321+0000","last_undegraded":"2023-11-06T20:57:01.703321+0000","last_fullsized":"2023-11-06T20:57:01.703321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.737371+0000","last_change":"2023-11-06T20:57:01.733968+0000","last_active":"2023-11-06T20:57:01.737371+0000","last_peered":"2023-11-06T20:57:01.737371+0000","last_clean":"2023-11-06T20:57:01.737371+0000","last_became_active":"2023-11-06T20:57:01.733423+0000","last_became_peered":"2023-11-06T20:57:01.733423+0000","last_unstale":"2023-11-06T20:57:01.737371+0000","last_undegraded":"2023-11-06T20:57:01.737371+0000","last_fullsized":"2023-11-06T20:57:01.737371+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.711320+0000","last_change":"2023-11-06T20:57:01.230863+0000","last_active":"2023-11-06T20:57:01.711320+0000","last_peered":"2023-11-06T20:57:01.711320+0000","last_clean":"2023-11-06T20:57:01.711320+0000","last_became_active":"2023-11-06T20:57:01.230559+0000","last_became_peered":"2023-11-06T20:57:01.230559+0000","last_unstale":"2023-11-06T20:57:01.711320+0000","last_undegraded":"2023-11-06T20:57:01.711320+0000","last_fullsized":"2023-11-06T20:57:01.711320+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.707912+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:57:01.707912+0000","last_peered":"2023-11-06T20:57:01.707912+0000","last_clean":"2023-11-06T20:57:01.707912+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:57:01.707912+0000","last_undegraded":"2023-11-06T20:57:01.707912+0000","last_fullsized":"2023-11-06T20:57:01.707912+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.724707+0000","last_change":"2023-11-06T20:57:01.724707+0000","last_active":"2023-11-06T20:57:01.724707+0000","last_peered":"2023-11-06T20:57:01.724707+0000","last_clean":"2023-11-06T20:57:01.724707+0000","last_became_active":"2023-11-06T20:57:01.723899+0000","last_became_peered":"2023-11-06T20:57:01.723899+0000","last_unstale":"2023-11-06T20:57:01.724707+0000","last_undegraded":"2023-11-06T20:57:01.724707+0000","last_fullsized":"2023-11-06T20:57:01.724707+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.239145+0000","last_change":"2023-11-06T20:57:01.230367+0000","last_active":"2023-11-06T20:57:01.239145+0000","last_peered":"2023-11-06T20:57:01.239145+0000","last_clean":"2023-11-06T20:57:01.239145+0000","last_became_active":"2023-11-06T20:57:01.229895+0000","last_became_peered":"2023-11-06T20:57:01.229895+0000","last_unstale":"2023-11-06T20:57:01.239145+0000","last_undegraded":"2023-11-06T20:57:01.239145+0000","last_fullsized":"2023-11-06T20:57:01.239145+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.702522+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:57:01.702522+0000","last_peered":"2023-11-06T20:57:01.702522+0000","last_clean":"2023-11-06T20:57:01.702522+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:57:01.702522+0000","last_undegraded":"2023-11-06T20:57:01.702522+0000","last_fullsized":"2023-11-06T20:57:01.702522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.703769+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:57:01.703769+0000","last_peered":"2023-11-06T20:57:01.703769+0000","last_clean":"2023-11-06T20:57:01.703769+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:57:01.703769+0000","last_undegraded":"2023-11-06T20:57:01.703769+0000","last_fullsized":"2023-11-06T20:57:01.703769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.738037+0000","last_change":"2023-11-06T20:57:01.738037+0000","last_active":"2023-11-06T20:57:01.738037+0000","last_peered":"2023-11-06T20:57:01.738037+0000","last_clean":"2023-11-06T20:57:01.738037+0000","last_became_active":"2023-11-06T20:57:01.737674+0000","last_became_peered":"2023-11-06T20:57:01.737674+0000","last_unstale":"2023-11-06T20:57:01.738037+0000","last_undegraded":"2023-11-06T20:57:01.738037+0000","last_fullsized":"2023-11-06T20:57:01.738037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.704156+0000","last_change":"2023-11-06T20:57:01.235925+0000","last_active":"2023-11-06T20:57:01.704156+0000","last_peered":"2023-11-06T20:57:01.704156+0000","last_clean":"2023-11-06T20:57:01.704156+0000","last_became_active":"2023-11-06T20:57:01.233217+0000","last_became_peered":"2023-11-06T20:57:01.233217+0000","last_unstale":"2023-11-06T20:57:01.704156+0000","last_undegraded":"2023-11-06T20:57:01.704156+0000","last_fullsized":"2023-11-06T20:57:01.704156+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.701391+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:57:01.701391+0000","last_peered":"2023-11-06T20:57:01.701391+0000","last_clean":"2023-11-06T20:57:01.701391+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:57:01.701391+0000","last_undegraded":"2023-11-06T20:57:01.701391+0000","last_fullsized":"2023-11-06T20:57:01.701391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.728930+0000","last_change":"2023-11-06T20:57:01.723251+0000","last_active":"2023-11-06T20:57:01.728930+0000","last_peered":"2023-11-06T20:57:01.728930+0000","last_clean":"2023-11-06T20:57:01.728930+0000","last_became_active":"2023-11-06T20:57:01.721597+0000","last_became_peered":"2023-11-06T20:57:01.721597+0000","last_unstale":"2023-11-06T20:57:01.728930+0000","last_undegraded":"2023-11-06T20:57:01.728930+0000","last_fullsized":"2023-11-06T20:57:01.728930+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.703726+0000","last_change":"2023-11-06T20:57:01.238159+0000","last_active":"2023-11-06T20:57:01.703726+0000","last_peered":"2023-11-06T20:57:01.703726+0000","last_clean":"2023-11-06T20:57:01.703726+0000","last_became_active":"2023-11-06T20:57:01.237600+0000","last_became_peered":"2023-11-06T20:57:01.237600+0000","last_unstale":"2023-11-06T20:57:01.703726+0000","last_undegraded":"2023-11-06T20:57:01.703726+0000","last_fullsized":"2023-11-06T20:57:01.703726+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.704160+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:57:01.704160+0000","last_peered":"2023-11-06T20:57:01.704160+0000","last_clean":"2023-11-06T20:57:01.704160+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:57:01.704160+0000","last_undegraded":"2023-11-06T20:57:01.704160+0000","last_fullsized":"2023-11-06T20:57:01.704160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.729799+0000","last_change":"2023-11-06T20:57:01.729799+0000","last_active":"2023-11-06T20:57:01.729799+0000","last_peered":"2023-11-06T20:57:01.729799+0000","last_clean":"2023-11-06T20:57:01.729799+0000","last_became_active":"2023-11-06T20:57:01.729468+0000","last_became_peered":"2023-11-06T20:57:01.729468+0000","last_unstale":"2023-11-06T20:57:01.729799+0000","last_undegraded":"2023-11-06T20:57:01.729799+0000","last_fullsized":"2023-11-06T20:57:01.729799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.245317+0000","last_change":"2023-11-06T20:57:01.241972+0000","last_active":"2023-11-06T20:57:01.245317+0000","last_peered":"2023-11-06T20:57:01.245317+0000","last_clean":"2023-11-06T20:57:01.245317+0000","last_became_active":"2023-11-06T20:57:01.240265+0000","last_became_peered":"2023-11-06T20:57:01.240265+0000","last_unstale":"2023-11-06T20:57:01.245317+0000","last_undegraded":"2023-11-06T20:57:01.245317+0000","last_fullsized":"2023-11-06T20:57:01.245317+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.713690+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:57:01.713690+0000","last_peered":"2023-11-06T20:57:01.713690+0000","last_clean":"2023-11-06T20:57:01.713690+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:57:01.713690+0000","last_undegraded":"2023-11-06T20:57:01.713690+0000","last_fullsized":"2023-11-06T20:57:01.713690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.730900+0000","last_change":"2023-11-06T20:57:01.726476+0000","last_active":"2023-11-06T20:57:01.730900+0000","last_peered":"2023-11-06T20:57:01.730900+0000","last_clean":"2023-11-06T20:57:01.730900+0000","last_became_active":"2023-11-06T20:57:01.726137+0000","last_became_peered":"2023-11-06T20:57:01.726137+0000","last_unstale":"2023-11-06T20:57:01.730900+0000","last_undegraded":"2023-11-06T20:57:01.730900+0000","last_fullsized":"2023-11-06T20:57:01.730900+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:01.198280+0000","last_clean_scrub_stamp":"2023-11-06T20:57:01.198280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.721227+0000","last_change":"2023-11-06T20:57:01.241338+0000","last_active":"2023-11-06T20:57:01.721227+0000","last_peered":"2023-11-06T20:57:01.721227+0000","last_clean":"2023-11-06T20:57:01.721227+0000","last_became_active":"2023-11-06T20:57:01.241021+0000","last_became_peered":"2023-11-06T20:57:01.241021+0000","last_unstale":"2023-11-06T20:57:01.721227+0000","last_undegraded":"2023-11-06T20:57:01.721227+0000","last_fullsized":"2023-11-06T20:57:01.721227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:57:00.190912+0000","last_clean_scrub_stamp":"2023-11-06T20:57:00.190912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T20:57:01.703882+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:57:01.703882+0000","last_peered":"2023-11-06T20:57:01.703882+0000","last_clean":"2023-11-06T20:57:01.703882+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:57:01.703882+0000","last_undegraded":"2023-11-06T20:57:01.703882+0000","last_fullsized":"2023-11-06T20:57:01.703882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301972,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069868,"statfs":{"total":96636764160,"available":96327544832,"internally_reserved":0,"allocated":798720,"data_stored":389916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.68200000000000005},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70799999999999996,"15min":0.68000000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.89500000000000002}]},{"osd":1,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71199999999999997,"15min":0.745},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":0.98599999999999999,"15min":5.1749999999999998},"last":0.875},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.41399999999999998},"max":{"1min":0.85099999999999998,"5min":0.94399999999999995,"15min":1.133},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.51100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91600000000000004,"5min":0.996,"15min":1.165},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85399999999999998,"5min":0.92700000000000005,"15min":1.024},"last":0.82899999999999996}]},{"osd":3,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72199999999999998,"15min":0.76100000000000001},"min":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":0.872,"5min":1.0129999999999999,"15min":5.202},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.80600000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.49399999999999999},"max":{"1min":0.878,"5min":0.95699999999999996,"15min":5.165},"last":0.79200000000000004}]},{"osd":4,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.43099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.751,"5min":0.77400000000000002,"15min":1.083},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.61799999999999999},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.40200000000000002},"max":{"1min":0.74299999999999999,"5min":0.86799999999999999,"15min":0.92800000000000005},"last":0.53600000000000003}]},{"osd":5,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.621,"15min":0.66100000000000003},"min":{"1min":0.47799999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78700000000000003,"5min":0.93300000000000005,"15min":1.395},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64700000000000002,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.45000000000000001,"15min":0.42699999999999999},"max":{"1min":0.80000000000000004,"5min":0.878,"15min":1.129},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Nov 6 20:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70199999999999996,"15min":0.73199999999999998},"min":{"1min":0.55200000000000005,"5min":0.505,"15min":0.435},"max":{"1min":0.79900000000000004,"5min":0.94499999999999995,"15min":1.24},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.51700000000000002,"5min":0.47899999999999998,"15min":0.443},"max":{"1min":0.88900000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.81000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575006,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":382334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70499999999999996,"15min":0.67300000000000004},"min":{"1min":0.505,"5min":0.45600000000000002,"15min":0.32800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.0720000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.66800000000000004},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.38800000000000001},"max":{"1min":0.90100000000000002,"5min":0.92500000000000004,"15min":0.92900000000000005},"last":0.67200000000000004}]},{"osd":1,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71899999999999997,"15min":0.68100000000000005},"min":{"1min":0.54000000000000004,"5min":0.45000000000000001,"15min":0.42299999999999999},"max":{"1min":0.96199999999999997,"5min":0.97599999999999998,"15min":1.3799999999999999},"last":0.75},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.69099999999999995},"min":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.39700000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.1659999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78500000000000003,"15min":0.74299999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.95699999999999996,"15min":1.321},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.496},"max":{"1min":0.88300000000000001,"5min":1.024,"15min":1.405},"last":0.69299999999999995}]},{"osd":3,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.71199999999999997},"min":{"1min":0.56200000000000006,"5min":0.48699999999999999,"15min":0.45900000000000002},"max":{"1min":0.91700000000000004,"5min":1.0580000000000001,"15min":1.5109999999999999},"last":0.621},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73799999999999999,"15min":0.71699999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.48999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.61899999999999999},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.018},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.65900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.86499999999999999,"5min":0.875,"15min":2.0310000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.68799999999999994},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":0.96899999999999997,"15min":1.133},"last":0.76500000000000001}]},{"osd":7,"last update":"Mon Nov 6 20:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66900000000000004,"15min":0.69299999999999995},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.88400000000000001,"5min":1,"15min":1.0940000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72299999999999998,"15min":0.74399999999999999},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95499999999999996,"5min":0.98699999999999999,"15min":1.4319999999999999},"last":0.79900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575006,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":388968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46400000000000002,"15min":0.441},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.27500000000000002},"max":{"1min":0.63800000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52900000000000003,"15min":0.495},"min":{"1min":0.37,"5min":0.35299999999999998,"15min":0.318},"max":{"1min":0.72799999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.34499999999999997}]},{"osd":2,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.497},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.311},"max":{"1min":0.67400000000000004,"5min":0.80800000000000005,"15min":0.86599999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61399999999999999,"15min":0.58499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.71199999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59799999999999998,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.40100000000000002,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.54700000000000004},"min":{"1min":0.41299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73499999999999999,"5min":0.94099999999999995,"15min":1.29},"last":0.443}]},{"osd":4,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79000000000000004,"15min":0.79600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.5},"max":{"1min":0.99299999999999999,"5min":1.171,"15min":1.171},"last":0.746},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.81599999999999995},"min":{"1min":0.69799999999999995,"5min":0.64200000000000002,"15min":0.57899999999999996},"max":{"1min":1.056,"5min":1.0980000000000001,"15min":1.1519999999999999},"last":0.81899999999999995}]},{"osd":5,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.64700000000000002,"5min":0.57599999999999996,"15min":0.54900000000000004},"max":{"1min":0.96799999999999997,"5min":1.0629999999999999,"15min":1.329},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.78400000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0229999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81599999999999995,"15min":0.79900000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.501},"max":{"1min":1.038,"5min":1.123,"15min":1.135},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.82299999999999995},"min":{"1min":0.65500000000000003,"5min":0.623,"15min":0.54500000000000004},"max":{"1min":1.014,"5min":1.1539999999999999,"15min":1.3540000000000001},"last":0.78200000000000003}]},{"osd":7,"last update":"Mon Nov 6 20:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.86299999999999999,"15min":0.83599999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.54700000000000004},"max":{"1min":1.077,"5min":1.1419999999999999,"15min":1.405},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84099999999999997,"15min":0.82199999999999995},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.55900000000000005},"max":{"1min":1.0669999999999999,"5min":1.105,"15min":1.1950000000000001},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575006,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301956,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069884,"statfs":{"total":96636764160,"available":96327561216,"internally_reserved":0,"allocated":782336,"data_stored":382888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47299999999999998,"15min":0.45700000000000002},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29299999999999998},"max":{"1min":0.71599999999999997,"5min":0.77700000000000002,"15min":0.78600000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.498},"min":{"1min":0.39600000000000002,"5min":0.33900000000000002,"15min":0.311},"max":{"1min":0.70599999999999996,"5min":0.77600000000000002,"15min":0.97399999999999998},"last":0.33200000000000002}]},{"osd":2,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47299999999999998,"15min":0.46200000000000002},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71999999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55900000000000005,"15min":0.53700000000000003},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.36499999999999999},"max":{"1min":0.68000000000000005,"5min":0.89300000000000002,"15min":0.91000000000000003},"last":0.36299999999999999}]},{"osd":3,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.42699999999999999,"5min":0.40899999999999997,"15min":0.35999999999999999},"max":{"1min":0.69299999999999995,"5min":0.755,"15min":0.79700000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.56399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.38300000000000001},"max":{"1min":0.80100000000000005,"5min":0.89300000000000002,"15min":0.89800000000000002},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.60799999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1279999999999999,"5min":1.1779999999999999,"15min":1.3779999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81299999999999994,"15min":0.80800000000000005},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.53600000000000003},"max":{"1min":1.105,"5min":1.202,"15min":1.8380000000000001},"last":0.82499999999999996}]},{"osd":5,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81999999999999995,"15min":0.80600000000000005},"min":{"1min":0.66600000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98799999999999999,"5min":1.139,"15min":1.3169999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83799999999999997,"15min":0.82799999999999996},"min":{"1min":0.68400000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1459999999999999,"5min":1.2829999999999999,"15min":1.915},"last":0.70399999999999996}]},{"osd":6,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.54600000000000004},"max":{"1min":1.0449999999999999,"5min":1.111,"15min":1.1120000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.82199999999999995,"15min":0.81499999999999995},"min":{"1min":0.64400000000000002,"5min":0.621,"15min":0.56999999999999995},"max":{"1min":1,"5min":1.304,"15min":1.5860000000000001},"last":0.64800000000000002}]},{"osd":7,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.81999999999999995},"min":{"1min":0.66600000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0820000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83799999999999997,"15min":0.83399999999999996},"min":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.53200000000000003},"max":{"1min":1.089,"5min":1.236,"15min":1.6359999999999999},"last":0.74299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297876,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073964,"statfs":{"total":96636764160,"available":96331739136,"internally_reserved":0,"allocated":798720,"data_stored":390448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.441,"15min":0.437},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.25900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.504,"15min":0.48499999999999999},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.26300000000000001},"max":{"1min":0.68500000000000005,"5min":0.79000000000000004,"15min":1.0269999999999999},"last":0.44}]},{"osd":1,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39500000000000002,"15min":0.30099999999999999},"max":{"1min":0.77100000000000002,"5min":0.97699999999999998,"15min":1.3240000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.39300000000000002},"max":{"1min":0.81299999999999994,"5min":1.0209999999999999,"15min":1.2},"last":0.59599999999999997}]},{"osd":3,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.5,"15min":0.495},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33400000000000002},"max":{"1min":0.65500000000000003,"5min":0.73299999999999998,"15min":1.55},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.755,"5min":0.877,"15min":2.242},"last":0.55100000000000005}]},{"osd":4,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.54300000000000004},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.371},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.50900000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.357},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76900000000000002,"15min":0.77600000000000002},"min":{"1min":0.55300000000000005,"5min":0.50900000000000001,"15min":0.45900000000000002},"max":{"1min":1.0880000000000001,"5min":1.0960000000000001,"15min":1.595},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.498},"max":{"1min":1.0409999999999999,"5min":1.1080000000000001,"15min":1.4099999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80800000000000005,"15min":0.79600000000000004},"min":{"1min":0.58699999999999997,"5min":0.58599999999999997,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":1.0900000000000001,"15min":1.4239999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.52300000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.6339999999999999},"last":0.79600000000000004}]},{"osd":7,"last update":"Mon Nov 6 20:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82999999999999996,"15min":0.82299999999999995},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.46600000000000003},"max":{"1min":1.121,"5min":1.1319999999999999,"15min":1.534},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81899999999999995,"15min":0.81399999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.6930000000000001},"last":0.88700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301964,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069876,"statfs":{"total":96636764160,"available":96327553024,"internally_reserved":0,"allocated":790528,"data_stored":389058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.45200000000000001,"15min":0.45300000000000001},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.26800000000000002},"max":{"1min":0.66800000000000004,"5min":0.76700000000000002,"15min":1.046},"last":0.313},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52500000000000002,"15min":0.503},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.90100000000000002},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48399999999999999,"15min":0.47699999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.313},"max":{"1min":0.64600000000000002,"5min":0.69699999999999995,"15min":0.753},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.38900000000000001,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.80900000000000005,"15min":1.177},"last":0.41899999999999998}]},{"osd":2,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.35299999999999998},"max":{"1min":0.68999999999999995,"5min":0.749,"15min":0.77300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.502,"5min":0.44900000000000001,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.88100000000000001,"15min":1.3120000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.82399999999999995,"15min":0.78100000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.51200000000000001},"max":{"1min":0.96099999999999997,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82199999999999995,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.47999999999999998},"max":{"1min":0.93300000000000005,"5min":1.2729999999999999,"15min":1.286},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.80000000000000004,"15min":0.76400000000000001},"min":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.51400000000000001},"max":{"1min":0.997,"5min":1.1779999999999999,"15min":1.228},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.77100000000000002},"min":{"1min":0.63,"5min":0.56499999999999995,"15min":0.49199999999999999},"max":{"1min":1.036,"5min":1.141,"15min":1.1439999999999999},"last":0.627}]},{"osd":6,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79500000000000004,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97999999999999998,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.82599999999999996,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.53700000000000003},"max":{"1min":0.97599999999999998,"5min":1.2130000000000001,"15min":1.268},"last":0.59299999999999997}]},{"osd":7,"last update":"Mon Nov 6 20:56:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.82499999999999996,"15min":0.79000000000000004},"min":{"1min":0.64700000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99099999999999999,"5min":1.302,"15min":1.302},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83099999999999996,"15min":0.79400000000000004},"min":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.47899999999999998},"max":{"1min":1.036,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.64100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":382930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.98499999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.81499999999999995,"5min":0.89900000000000002,"15min":0.93400000000000005},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71399999999999997,"15min":0.69899999999999995},"min":{"1min":0.58999999999999997,"5min":0.47099999999999997,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.95699999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.51000000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86399999999999999,"5min":0.90200000000000002,"15min":1.0569999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73599999999999999,"15min":0.70799999999999996},"min":{"1min":0.58699999999999997,"5min":0.498,"15min":0.46200000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78200000000000003,"15min":0.77800000000000002},"min":{"1min":0.61699999999999999,"5min":0.58199999999999996,"15min":0.51900000000000002},"max":{"1min":0.92900000000000005,"5min":0.93400000000000005,"15min":1.028},"last":0.84399999999999997}]},{"osd":3,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70399999999999996,"15min":0.68899999999999995},"min":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.0169999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70099999999999996,"15min":0.68600000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.42799999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.71399999999999997}]},{"osd":5,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.629},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78200000000000003,"5min":1.0389999999999999,"15min":1.0880000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57899999999999996,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76400000000000001,"5min":0.877,"15min":1.3520000000000001},"last":0.70099999999999996}]},{"osd":6,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.623},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.39900000000000002},"max":{"1min":0.76700000000000002,"5min":0.84799999999999998,"15min":1.077},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.64600000000000002,"15min":0.70199999999999996},"min":{"1min":0.47699999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.78700000000000003,"5min":0.93500000000000005,"15min":1.3999999999999999},"last":0.88800000000000001}]},{"osd":7,"last update":"Mon Nov 6 20:56:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64200000000000002,"15min":0.68200000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83599999999999997,"5min":0.91200000000000003,"15min":1.5209999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65800000000000003,"15min":0.68000000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84699999999999998,"5min":0.93600000000000005,"15min":1.161},"last":0.81399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":382998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.37,"15min":0.36399999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0109999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":0.93500000000000005,"15min":0.96799999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.69799999999999995},"min":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.44700000000000001},"max":{"1min":0.92100000000000004,"5min":0.99399999999999999,"15min":1.0109999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.40300000000000002},"max":{"1min":0.98099999999999998,"5min":0.99199999999999999,"15min":1.044},"last":0.67400000000000004}]},{"osd":2,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.72999999999999998},"min":{"1min":0.50800000000000001,"5min":0.496,"15min":0.42899999999999999},"max":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":1.024},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72999999999999998,"15min":0.71399999999999997},"min":{"1min":0.60499999999999998,"5min":0.54600000000000004,"15min":0.45600000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.0329999999999999},"last":0.621}]},{"osd":3,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89000000000000001,"5min":0.90900000000000003,"15min":0.93899999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72499999999999998,"15min":0.71499999999999997},"min":{"1min":0.57799999999999996,"5min":0.47899999999999998,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":1.02},"last":0.52600000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.61799999999999999},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76100000000000001,"5min":0.83899999999999997,"15min":0.93700000000000006},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.98599999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61499999999999999,"15min":0.63700000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.97599999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.70499999999999996,"15min":0.72699999999999998},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.105},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Nov 6 20:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63,"15min":0.67000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.0009999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.90100000000000002,"5min":0.90800000000000003,"15min":0.95599999999999996},"last":0.73399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:57:06.514 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T20:57:06.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T20:57:06.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.752+0000 7f7e1ebff700 1 Processor -- start 2023-11-06T20:57:06.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.753+0000 7f7e1ebff700 1 -- start start 2023-11-06T20:57:06.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e1ebff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7e18154330 0x7f7e180c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e1ebff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7e18154710 0x7f7e180c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e1ebff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e180c7ff0 0x7f7e180646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e1ebff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e180c6b60 con 0x7f7e18154330 2023-11-06T20:57:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e1ebff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e180c5cd0 con 0x7f7e18154710 2023-11-06T20:57:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e1ebff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180c5f50 con 0x7f7e180c7ff0 2023-11-06T20:57:06.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e1c99b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7e18154330 0x7f7e180c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e1c99b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7e18154330 0x7f7e180c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43982/0 (socket says 172.21.15.33:43982) 2023-11-06T20:57:06.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e1c99b700 1 -- 172.21.15.33:0/4238319383 learned_addr learned my addr 172.21.15.33:0/4238319383 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:06.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e17fff700 1 --2- 172.21.15.33:0/4238319383 >> v2:172.21.15.69:3300/0 conn(0x7f7e18154710 0x7f7e180c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.754+0000 7f7e1d19c700 1 --2- 172.21.15.33:0/4238319383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e180c7ff0 0x7f7e180646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.755+0000 7f7e1c99b700 1 -- 172.21.15.33:0/4238319383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e180c7ff0 msgr2=0x7f7e180646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.755+0000 7f7e1c99b700 1 --2- 172.21.15.33:0/4238319383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e180c7ff0 0x7f7e180646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.755+0000 7f7e1c99b700 1 -- 172.21.15.33:0/4238319383 >> v2:172.21.15.69:3300/0 conn(0x7f7e18154710 msgr2=0x7f7e180c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.755+0000 7f7e1c99b700 1 --2- 172.21.15.33:0/4238319383 >> v2:172.21.15.69:3300/0 conn(0x7f7e18154710 0x7f7e180c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.755+0000 7f7e1c99b700 1 -- 172.21.15.33:0/4238319383 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e180c6230 con 0x7f7e18154330 2023-11-06T20:57:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.755+0000 7f7e17fff700 1 --2- 172.21.15.33:0/4238319383 >> v2:172.21.15.69:3300/0 conn(0x7f7e18154710 0x7f7e180c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:57:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.755+0000 7f7e1c99b700 1 --2- 172.21.15.33:0/4238319383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154330 0x7f7e180c57e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f7e180c8d10 tx=0x7f7e0c0090f0).ready entity=mon.0 client_cookie=928318266f431075 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.755+0000 7f7e177fe700 1 -- 172.21.15.33:0/4238319383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00b600 con 0x7f7e18154330 2023-11-06T20:57:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.755+0000 7f7e177fe700 1 -- 172.21.15.33:0/4238319383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0c00b760 con 0x7f7e18154330 2023-11-06T20:57:06.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.755+0000 7f7e177fe700 1 -- 172.21.15.33:0/4238319383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00bae0 con 0x7f7e18154330 2023-11-06T20:57:06.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.756+0000 7f7e1ebff700 1 -- 172.21.15.33:0/4238319383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154330 msgr2=0x7f7e180c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.756+0000 7f7e1ebff700 1 --2- 172.21.15.33:0/4238319383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154330 0x7f7e180c57e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f7e180c8d10 tx=0x7f7e0c0090f0).stop 2023-11-06T20:57:06.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.756+0000 7f7e1ebff700 1 -- 172.21.15.33:0/4238319383 shutdown_connections 2023-11-06T20:57:06.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.756+0000 7f7e1ebff700 1 --2- 172.21.15.33:0/4238319383 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e180c7ff0 0x7f7e180646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.756+0000 7f7e1ebff700 1 --2- 172.21.15.33:0/4238319383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154330 0x7f7e180c57e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.756+0000 7f7e1ebff700 1 --2- 172.21.15.33:0/4238319383 >> v2:172.21.15.69:3300/0 conn(0x7f7e18154710 0x7f7e180c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.756+0000 7f7e1ebff700 1 -- 172.21.15.33:0/4238319383 >> 172.21.15.33:0/4238319383 conn(0x7f7e180cb910 msgr2=0x7f7e180c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:06.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.756+0000 7f7e1ebff700 1 -- 172.21.15.33:0/4238319383 shutdown_connections 2023-11-06T20:57:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.756+0000 7f7e1ebff700 1 -- 172.21.15.33:0/4238319383 wait complete. 2023-11-06T20:57:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.757+0000 7f7e1ebff700 1 Processor -- start 2023-11-06T20:57:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.757+0000 7f7e1ebff700 1 -- start start 2023-11-06T20:57:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.758+0000 7f7e1ebff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e180c7ff0 0x7f7e18147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.758+0000 7f7e1ebff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e18154330 0x7f7e180addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.758+0000 7f7e1ebff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154710 0x7f7e180c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.758+0000 7f7e1ebff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180b0370 con 0x7f7e18154710 2023-11-06T20:57:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.758+0000 7f7e1ebff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180af2d0 con 0x7f7e180c7ff0 2023-11-06T20:57:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.758+0000 7f7e1ebff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180af5b0 con 0x7f7e18154330 2023-11-06T20:57:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1d19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154710 0x7f7e180c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1d19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154710 0x7f7e180c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43986/0 (socket says 172.21.15.33:43986) 2023-11-06T20:57:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1d19c700 1 -- 172.21.15.33:0/1480092277 learned_addr learned my addr 172.21.15.33:0/1480092277 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1c99b700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e180c7ff0 0x7f7e18147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e17fff700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e18154330 0x7f7e180addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1d19c700 1 -- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e18154330 msgr2=0x7f7e180addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1d19c700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e18154330 0x7f7e180addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1d19c700 1 -- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e180c7ff0 msgr2=0x7f7e18147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1d19c700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e180c7ff0 0x7f7e18147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1d19c700 1 -- 172.21.15.33:0/1480092277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c009370 con 0x7f7e18154710 2023-11-06T20:57:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1c99b700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e180c7ff0 0x7f7e18147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:57:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1d19c700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154710 0x7f7e180c9260 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f7e10007f20 tx=0x7f7e10009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e14ff9700 1 -- 172.21.15.33:0/1480092277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1000eca0 con 0x7f7e18154710 2023-11-06T20:57:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e14ff9700 1 -- 172.21.15.33:0/1480092277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1000ee00 con 0x7f7e18154710 2023-11-06T20:57:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.759+0000 7f7e1ebff700 1 -- 172.21.15.33:0/1480092277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e180af890 con 0x7f7e18154710 2023-11-06T20:57:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.760+0000 7f7e14ff9700 1 -- 172.21.15.33:0/1480092277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10014610 con 0x7f7e18154710 2023-11-06T20:57:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.760+0000 7f7e1ebff700 1 -- 172.21.15.33:0/1480092277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e180afe50 con 0x7f7e18154710 2023-11-06T20:57:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.761+0000 7f7e1ebff700 1 -- 172.21.15.33:0/1480092277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e180b0070 con 0x7f7e18154710 2023-11-06T20:57:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.762+0000 7f7e14ff9700 1 -- 172.21.15.33:0/1480092277 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7e10014980 con 0x7f7e18154710 2023-11-06T20:57:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.763+0000 7f7e14ff9700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7dfc05f270 0x7f7dfc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.763+0000 7f7e14ff9700 1 -- 172.21.15.33:0/1480092277 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7e1007a870 con 0x7f7e18154710 2023-11-06T20:57:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.763+0000 7f7e1c99b700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7dfc05f270 0x7f7dfc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.764+0000 7f7e1c99b700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7dfc05f270 0x7f7dfc061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7e0c009130 tx=0x7f7e0c017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:06.766+0000 7f7e14ff9700 1 -- 172.21.15.33:0/1480092277 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7e1004b9c0 con 0x7f7e18154710 2023-11-06T20:57:07.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.017+0000 7f7e1ebff700 1 -- 172.21.15.33:0/1480092277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7e18146760 con 0x7f7e18154710 2023-11-06T20:57:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.021+0000 7f7e14ff9700 1 -- 172.21.15.33:0/1480092277 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7e1004efe0 con 0x7f7e18154710 2023-11-06T20:57:07.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 -- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7dfc05f270 msgr2=0x7f7dfc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7dfc05f270 0x7f7dfc061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7e0c009130 tx=0x7f7e0c017000).stop 2023-11-06T20:57:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 -- 172.21.15.33:0/1480092277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154710 msgr2=0x7f7e180c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154710 0x7f7e180c9260 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f7e10007f20 tx=0x7f7e10009860).stop 2023-11-06T20:57:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 -- 172.21.15.33:0/1480092277 shutdown_connections 2023-11-06T20:57:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7dfc05f270 0x7f7dfc061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e18154710 0x7f7e180c9260 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e180c7ff0 0x7f7e18147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 --2- 172.21.15.33:0/1480092277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e18154330 0x7f7e180addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 -- 172.21.15.33:0/1480092277 >> 172.21.15.33:0/1480092277 conn(0x7f7e180cb910 msgr2=0x7f7e180aaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 -- 172.21.15.33:0/1480092277 shutdown_connections 2023-11-06T20:57:07.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.023+0000 7f7e1ebff700 1 -- 172.21.15.33:0/1480092277 wait complete. 2023-11-06T20:57:07.025 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T20:57:07.040 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:57:07.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.236+0000 7f9a385ff700 1 Processor -- start 2023-11-06T20:57:07.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.237+0000 7f9a385ff700 1 -- start start 2023-11-06T20:57:07.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.238+0000 7f9a385ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a301521a0 0x7f9a300c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:07.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.238+0000 7f9a385ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9a30155320 0x7f9a300c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.238+0000 7f9a385ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a30154e90 con 0x7f9a30151dc0 2023-11-06T20:57:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.238+0000 7f9a385ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a300639d0 con 0x7f9a30155320 2023-11-06T20:57:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.238+0000 7f9a385ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a30063cb0 con 0x7f9a301521a0 2023-11-06T20:57:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.238+0000 7f9a3639b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9a30151dc0 0x7f9a30063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48262/0 (socket says 172.21.15.33:48262) 2023-11-06T20:57:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.238+0000 7f9a3639b700 1 -- 172.21.15.33:0/3576133987 learned_addr learned my addr 172.21.15.33:0/3576133987 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:57:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.238+0000 7f9a36b9c700 1 --2- 172.21.15.33:0/3576133987 >> v2:172.21.15.33:3300/0 conn(0x7f9a30155320 0x7f9a300c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.238+0000 7f9a35b9a700 1 --2- 172.21.15.33:0/3576133987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a301521a0 0x7f9a300c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.239+0000 7f9a35399700 1 -- 172.21.15.33:0/3576133987 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391767454 0 0) 0x7f9a30154e90 con 0x7f9a30151dc0 2023-11-06T20:57:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.239+0000 7f9a35399700 1 -- 172.21.15.33:0/3576133987 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a1c0033f0 con 0x7f9a30151dc0 2023-11-06T20:57:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.239+0000 7f9a36b9c700 1 -- 172.21.15.33:0/3576133987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a301521a0 msgr2=0x7f9a300c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.239+0000 7f9a36b9c700 1 --2- 172.21.15.33:0/3576133987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a301521a0 0x7f9a300c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.239+0000 7f9a36b9c700 1 -- 172.21.15.33:0/3576133987 >> v1:172.21.15.33:6789/0 conn(0x7f9a30151dc0 legacy=0x7f9a30063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.239+0000 7f9a36b9c700 1 -- 172.21.15.33:0/3576133987 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a30063f90 con 0x7f9a30155320 2023-11-06T20:57:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.239+0000 7f9a36b9c700 1 --2- 172.21.15.33:0/3576133987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a30155320 0x7f9a300c7850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f9a28009e60 tx=0x7f9a28009780).ready entity=mon.0 client_cookie=b325d2bc23c7e34b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.239+0000 7f9a35399700 1 -- 172.21.15.33:0/3576133987 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2800e070 con 0x7f9a30155320 2023-11-06T20:57:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.239+0000 7f9a35399700 1 -- 172.21.15.33:0/3576133987 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2800bb60 con 0x7f9a30155320 2023-11-06T20:57:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.239+0000 7f9a35399700 1 -- 172.21.15.33:0/3576133987 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28012410 con 0x7f9a30155320 2023-11-06T20:57:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.240+0000 7f9a385ff700 1 -- 172.21.15.33:0/3576133987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a30155320 msgr2=0x7f9a300c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.240+0000 7f9a385ff700 1 --2- 172.21.15.33:0/3576133987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a30155320 0x7f9a300c7850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f9a28009e60 tx=0x7f9a28009780).stop 2023-11-06T20:57:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.240+0000 7f9a385ff700 1 -- 172.21.15.33:0/3576133987 shutdown_connections 2023-11-06T20:57:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.240+0000 7f9a385ff700 1 --2- 172.21.15.33:0/3576133987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a30155320 0x7f9a300c7850 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.240+0000 7f9a385ff700 1 --2- 172.21.15.33:0/3576133987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a301521a0 0x7f9a300c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.240+0000 7f9a385ff700 1 -- 172.21.15.33:0/3576133987 >> 172.21.15.33:0/3576133987 conn(0x7f9a3005e1b0 msgr2=0x7f9a3005e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.240+0000 7f9a385ff700 1 -- 172.21.15.33:0/3576133987 shutdown_connections 2023-11-06T20:57:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.240+0000 7f9a385ff700 1 -- 172.21.15.33:0/3576133987 wait complete. 2023-11-06T20:57:07.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.241+0000 7f9a385ff700 1 Processor -- start 2023-11-06T20:57:07.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.241+0000 7f9a385ff700 1 -- start start 2023-11-06T20:57:07.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.242+0000 7f9a385ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a30151dc0 0x7f9a300acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.242+0000 7f9a385ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a301521a0 0x7f9a301593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.242+0000 7f9a385ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a30155320 0x7f9a301598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.242+0000 7f9a385ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a30159d90 con 0x7f9a301521a0 2023-11-06T20:57:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.242+0000 7f9a385ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a300c2380 con 0x7f9a30151dc0 2023-11-06T20:57:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.242+0000 7f9a385ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a300c2660 con 0x7f9a30155320 2023-11-06T20:57:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a35b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a301521a0 0x7f9a301593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a35b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a301521a0 0x7f9a301593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44006/0 (socket says 172.21.15.33:44006) 2023-11-06T20:57:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a35b9a700 1 -- 172.21.15.33:0/4088144324 learned_addr learned my addr 172.21.15.33:0/4088144324 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a36b9c700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a30155320 0x7f9a301598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a3639b700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a30151dc0 0x7f9a300acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a35b9a700 1 -- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a30155320 msgr2=0x7f9a301598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a35b9a700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a30155320 0x7f9a301598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a35b9a700 1 -- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a30151dc0 msgr2=0x7f9a300acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a35b9a700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a30151dc0 0x7f9a300acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a35b9a700 1 -- 172.21.15.33:0/4088144324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a28009a20 con 0x7f9a301521a0 2023-11-06T20:57:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a3639b700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a30151dc0 0x7f9a300acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:57:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.243+0000 7f9a35b9a700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a301521a0 0x7f9a301593b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f9a2000d890 tx=0x7f9a20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.244+0000 7f9a267fc700 1 -- 172.21.15.33:0/4088144324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a200125f0 con 0x7f9a301521a0 2023-11-06T20:57:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.244+0000 7f9a267fc700 1 -- 172.21.15.33:0/4088144324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a20012750 con 0x7f9a301521a0 2023-11-06T20:57:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.244+0000 7f9a385ff700 1 -- 172.21.15.33:0/4088144324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a300c2940 con 0x7f9a301521a0 2023-11-06T20:57:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.244+0000 7f9a267fc700 1 -- 172.21.15.33:0/4088144324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20012ad0 con 0x7f9a301521a0 2023-11-06T20:57:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.244+0000 7f9a385ff700 1 -- 172.21.15.33:0/4088144324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a300c2f00 con 0x7f9a301521a0 2023-11-06T20:57:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.245+0000 7f9a385ff700 1 -- 172.21.15.33:0/4088144324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3006bc10 con 0x7f9a301521a0 2023-11-06T20:57:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.246+0000 7f9a267fc700 1 -- 172.21.15.33:0/4088144324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9a20013480 con 0x7f9a301521a0 2023-11-06T20:57:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.246+0000 7f9a267fc700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9a1405f270 0x7f9a14061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.247+0000 7f9a267fc700 1 -- 172.21.15.33:0/4088144324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9a2007a1e0 con 0x7f9a301521a0 2023-11-06T20:57:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.247+0000 7f9a3639b700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9a1405f270 0x7f9a14061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.248+0000 7f9a3639b700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9a1405f270 0x7f9a14061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f9a2c007fd0 tx=0x7f9a2c009040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.251+0000 7f9a267fc700 1 -- 172.21.15.33:0/4088144324 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9a2004b210 con 0x7f9a301521a0 2023-11-06T20:57:07.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.478+0000 7f9a385ff700 1 -- 172.21.15.33:0/4088144324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a300ad6a0 con 0x7f9a301521a0 2023-11-06T20:57:07.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.478+0000 7f9a267fc700 1 -- 172.21.15.33:0/4088144324 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4483 (secure 0 0 0) 0x7f9a2004e830 con 0x7f9a301521a0 2023-11-06T20:57:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:57:07.478 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/455721224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":455721224},{"type":"v1","addr":"172.21.15.69:6836","nonce":455721224}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5632,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2050732931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2050732931},{"type":"v1","addr":"172.21.15.33:6834","nonce":2050732931}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1579729451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1579729451},{"type":"v1","addr":"172.21.15.33:6835","nonce":1579729451}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:57:01.696412+0000","modified":"2023-11-06T20:57:04.702227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/698812717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":698812717},{"type":"v1","addr":"172.21.15.69:6837","nonce":698812717}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-06T20:57:07.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.480+0000 7f9a385ff700 1 -- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9a1405f270 msgr2=0x7f9a14061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:07.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.480+0000 7f9a385ff700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9a1405f270 0x7f9a14061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f9a2c007fd0 tx=0x7f9a2c009040).stop 2023-11-06T20:57:07.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.480+0000 7f9a385ff700 1 -- 172.21.15.33:0/4088144324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a301521a0 msgr2=0x7f9a301593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:07.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.480+0000 7f9a385ff700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a301521a0 0x7f9a301593b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f9a2000d890 tx=0x7f9a20008040).stop 2023-11-06T20:57:07.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.481+0000 7f9a385ff700 1 -- 172.21.15.33:0/4088144324 shutdown_connections 2023-11-06T20:57:07.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.481+0000 7f9a385ff700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9a1405f270 0x7f9a14061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.481+0000 7f9a385ff700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a30155320 0x7f9a301598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.481+0000 7f9a385ff700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a30151dc0 0x7f9a300acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.481+0000 7f9a385ff700 1 --2- 172.21.15.33:0/4088144324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a301521a0 0x7f9a301593b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:07.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.481+0000 7f9a385ff700 1 -- 172.21.15.33:0/4088144324 >> 172.21.15.33:0/4088144324 conn(0x7f9a3005e1b0 msgr2=0x7f9a300c91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:07.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.481+0000 7f9a385ff700 1 -- 172.21.15.33:0/4088144324 shutdown_connections 2023-11-06T20:57:07.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:07.481+0000 7f9a385ff700 1 -- 172.21.15.33:0/4088144324 wait complete. 2023-11-06T20:57:07.483 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 22 2023-11-06T20:57:07.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:57:07.685 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:57:07.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T20:57:01.696412+0000', 'modified': '2023-11-06T20:57:04.702227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5748}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5748': {'gid': 5748, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.69:6837/698812717', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 698812717}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 698812717}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T20:57:07.900 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T20:57:07.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-06T20:57:08.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.098+0000 7efdeddff700 1 Processor -- start 2023-11-06T20:57:08.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.098+0000 7efdeddff700 1 -- start start 2023-11-06T20:57:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.099+0000 7efdeddff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efde8151f30 0x7efde8068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efdeddff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde8068100 con 0x7efde80be2c0 2023-11-06T20:57:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efdeddff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde80658c0 con 0x7efde8152310 2023-11-06T20:57:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efdeddff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde8068a80 con 0x7efde8151f30 2023-11-06T20:57:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efde77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efde8151f30 0x7efde8068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efde77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efde8151f30 0x7efde8068590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51310/0 (socket says 172.21.15.33:51310) 2023-11-06T20:57:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efde7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efde80be2c0 0x7efde8154b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49530/0 (socket says 172.21.15.33:49530) 2023-11-06T20:57:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efde77fe700 1 -- 172.21.15.33:0/2364662215 learned_addr learned my addr 172.21.15.33:0/2364662215 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efde67fc700 1 -- 172.21.15.33:0/2364662215 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480467754 0 0) 0x7efde8068100 con 0x7efde80be2c0 2023-11-06T20:57:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efde67fc700 1 -- 172.21.15.33:0/2364662215 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd00033f0 con 0x7efde80be2c0 2023-11-06T20:57:08.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efde77fe700 1 -- 172.21.15.33:0/2364662215 >> v1:172.21.15.69:6789/0 conn(0x7efde8152310 legacy=0x7efde80bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efde77fe700 1 -- 172.21.15.33:0/2364662215 >> v1:172.21.15.33:6789/0 conn(0x7efde80be2c0 legacy=0x7efde8154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.100+0000 7efde77fe700 1 -- 172.21.15.33:0/2364662215 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde8155750 con 0x7efde8151f30 2023-11-06T20:57:08.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.101+0000 7efde77fe700 1 --2- 172.21.15.33:0/2364662215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde8151f30 0x7efde8068590 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7efdd8006fd0 tx=0x7efdd80090f0).ready entity=mon.0 client_cookie=5bf1b82bc51f25a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:08.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.101+0000 7efde67fc700 1 -- 172.21.15.33:0/2364662215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd800b580 con 0x7efde8151f30 2023-11-06T20:57:08.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.101+0000 7efde67fc700 1 -- 172.21.15.33:0/2364662215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd800b6e0 con 0x7efde8151f30 2023-11-06T20:57:08.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.101+0000 7efde67fc700 1 -- 172.21.15.33:0/2364662215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd800ba60 con 0x7efde8151f30 2023-11-06T20:57:08.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.101+0000 7efdeddff700 1 -- 172.21.15.33:0/2364662215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde8151f30 msgr2=0x7efde8068590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.101+0000 7efdeddff700 1 --2- 172.21.15.33:0/2364662215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde8151f30 0x7efde8068590 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7efdd8006fd0 tx=0x7efdd80090f0).stop 2023-11-06T20:57:08.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.101+0000 7efdeddff700 1 -- 172.21.15.33:0/2364662215 shutdown_connections 2023-11-06T20:57:08.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.101+0000 7efdeddff700 1 --2- 172.21.15.33:0/2364662215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde8151f30 0x7efde8068590 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.101+0000 7efdeddff700 1 -- 172.21.15.33:0/2364662215 >> 172.21.15.33:0/2364662215 conn(0x7efde80c56f0 msgr2=0x7efde80c5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:08.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.101+0000 7efdeddff700 1 -- 172.21.15.33:0/2364662215 shutdown_connections 2023-11-06T20:57:08.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.102+0000 7efdeddff700 1 -- 172.21.15.33:0/2364662215 wait complete. 2023-11-06T20:57:08.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.103+0000 7efdeddff700 1 Processor -- start 2023-11-06T20:57:08.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.103+0000 7efdeddff700 1 -- start start 2023-11-06T20:57:08.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.104+0000 7efdeddff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde80be2c0 0x7efde80ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.104+0000 7efdeddff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efde8151f30 0x7efde80c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.104+0000 7efdeddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde8152310 0x7efde80c1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.104+0000 7efdeddff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde80c2ab0 con 0x7efde80be2c0 2023-11-06T20:57:08.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.104+0000 7efdeddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde80ae920 con 0x7efde8152310 2023-11-06T20:57:08.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.104+0000 7efdeddff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde8153270 con 0x7efde8151f30 2023-11-06T20:57:08.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.104+0000 7efde77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde80be2c0 0x7efde80ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.104+0000 7efde77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde80be2c0 0x7efde80ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51312/0 (socket says 172.21.15.33:51312) 2023-11-06T20:57:08.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.104+0000 7efde77fe700 1 -- 172.21.15.33:0/577719110 learned_addr learned my addr 172.21.15.33:0/577719110 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:08.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.104+0000 7efde6ffd700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efde8151f30 0x7efde80c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efde7fff700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde8152310 0x7efde80c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efde77fe700 1 -- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efde8151f30 msgr2=0x7efde80c25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efde77fe700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efde8151f30 0x7efde80c25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efde77fe700 1 -- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde8152310 msgr2=0x7efde80c1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efde77fe700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde8152310 0x7efde80c1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efde77fe700 1 -- 172.21.15.33:0/577719110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd8009370 con 0x7efde80be2c0 2023-11-06T20:57:08.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efde6ffd700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efde8151f30 0x7efde80c25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:57:08.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efde7fff700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde8152310 0x7efde80c1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:57:08.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efde77fe700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde80be2c0 0x7efde80ae300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7efdd8009130 tx=0x7efdd8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:08.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efdcf7fe700 1 -- 172.21.15.33:0/577719110 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd800b850 con 0x7efde80be2c0 2023-11-06T20:57:08.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efdcf7fe700 1 -- 172.21.15.33:0/577719110 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd800b9b0 con 0x7efde80be2c0 2023-11-06T20:57:08.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efdcf7fe700 1 -- 172.21.15.33:0/577719110 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd801c650 con 0x7efde80be2c0 2023-11-06T20:57:08.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efdeddff700 1 -- 172.21.15.33:0/577719110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde8153550 con 0x7efde80be2c0 2023-11-06T20:57:08.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.105+0000 7efdeddff700 1 -- 172.21.15.33:0/577719110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde8153b10 con 0x7efde80be2c0 2023-11-06T20:57:08.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.107+0000 7efdcf7fe700 1 -- 172.21.15.33:0/577719110 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7efdd801ca00 con 0x7efde80be2c0 2023-11-06T20:57:08.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.108+0000 7efdcf7fe700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efdd405f270 0x7efdd4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.108+0000 7efdcf7fe700 1 -- 172.21.15.33:0/577719110 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efdd807c090 con 0x7efde80be2c0 2023-11-06T20:57:08.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.109+0000 7efde6ffd700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efdd405f270 0x7efdd4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.109+0000 7efde6ffd700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efdd405f270 0x7efdd4061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7efde8154b80 tx=0x7efddc009040).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:08.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.110+0000 7efe05038b80 1 Processor -- start 2023-11-06T20:57:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.110+0000 7efe05038b80 1 -- start start 2023-11-06T20:57:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.111+0000 7efe05038b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7805460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.111+0000 7efe05038b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.111+0000 7efe05038b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf78003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.111+0000 7efe05038b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf77412f0 con 0x561cf784dbe0 2023-11-06T20:57:08.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.111+0000 7efe05038b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf77567e0 con 0x561cf782d4c0 2023-11-06T20:57:08.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.111+0000 7efe05038b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cf7756920 con 0x561cf7805080 2023-11-06T20:57:08.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde77fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7805460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77ffef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51320/0 (socket says 172.21.15.33:51320) 2023-11-06T20:57:08.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde6ffd700 1 -- 172.21.15.33:0/4264533521 learned_addr learned my addr 172.21.15.33:0/4264533521 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf78003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde6ffd700 1 -- 172.21.15.33:0/4264533521 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 msgr2=0x561cf7805460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde6ffd700 1 --2- 172.21.15.33:0/4264533521 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7805460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde6ffd700 1 -- 172.21.15.33:0/4264533521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 msgr2=0x561cf78003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde6ffd700 1 --2- 172.21.15.33:0/4264533521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf78003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde6ffd700 1 -- 172.21.15.33:0/4264533521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cf774f720 con 0x561cf784dbe0 2023-11-06T20:57:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efde6ffd700 1 --2- 172.21.15.33:0/4264533521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77ffef0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7efddc018670 tx=0x7efddc003000).ready entity=mon.0 client_cookie=8bc230acc535bb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efdcc9fe700 1 -- 172.21.15.33:0/4264533521 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc018240 con 0x561cf784dbe0 2023-11-06T20:57:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efdcc9fe700 1 -- 172.21.15.33:0/4264533521 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc015410 con 0x561cf784dbe0 2023-11-06T20:57:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.112+0000 7efdcc9fe700 1 -- 172.21.15.33:0/4264533521 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc015790 con 0x561cf784dbe0 2023-11-06T20:57:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.113+0000 7efe05038b80 1 -- 172.21.15.33:0/4264533521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 msgr2=0x561cf77ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.113+0000 7efe05038b80 1 --2- 172.21.15.33:0/4264533521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77ffef0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7efddc018670 tx=0x7efddc003000).stop 2023-11-06T20:57:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.113+0000 7efe05038b80 1 -- 172.21.15.33:0/4264533521 shutdown_connections 2023-11-06T20:57:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.113+0000 7efe05038b80 1 --2- 172.21.15.33:0/4264533521 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7805460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.113+0000 7efe05038b80 1 --2- 172.21.15.33:0/4264533521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf78003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.113+0000 7efe05038b80 1 --2- 172.21.15.33:0/4264533521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77ffef0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.113+0000 7efe05038b80 1 -- 172.21.15.33:0/4264533521 >> 172.21.15.33:0/4264533521 conn(0x561cf766e940 msgr2=0x561cf784e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.113+0000 7efe05038b80 1 -- 172.21.15.33:0/4264533521 shutdown_connections 2023-11-06T20:57:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.113+0000 7efe05038b80 1 -- 172.21.15.33:0/4264533521 wait complete. 2023-11-06T20:57:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.114+0000 7efe05038b80 1 -- start start 2023-11-06T20:57:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.114+0000 7efe05038b80 1 Processor -- start 2023-11-06T20:57:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.115+0000 7efe05038b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf7805080 0x561cf7833bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.115+0000 7efe05038b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf7828f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.115+0000 7efe05038b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf784dbe0 0x561cf7800dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.115+0000 7efe05038b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7756ba0 con 0x561cf7805080 2023-11-06T20:57:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.115+0000 7efe05038b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7721e60 con 0x561cf782d4c0 2023-11-06T20:57:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.115+0000 7efe05038b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cf7859a10 con 0x561cf784dbe0 2023-11-06T20:57:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.115+0000 7efe05038b80 10 client.0 ll_register_callbacks cb 0x561cf76f6ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T20:57:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.115+0000 7efde77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf7805080 0x561cf7833bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.115+0000 7efde77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf7805080 0x561cf7833bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51336/0 (socket says 172.21.15.33:51336) 2023-11-06T20:57:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.115+0000 7efde77fe700 1 -- 172.21.15.33:0/3988923478 learned_addr learned my addr 172.21.15.33:0/3988923478 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efde6ffd700 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf7828f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efde7fff700 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf784dbe0 0x561cf7800dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efde77fe700 1 -- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf784dbe0 msgr2=0x561cf7800dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efde77fe700 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf784dbe0 0x561cf7800dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efde77fe700 1 -- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 msgr2=0x561cf7828f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efde77fe700 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf7828f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efde77fe700 1 -- 172.21.15.33:0/3988923478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc00f040 con 0x561cf7805080 2023-11-06T20:57:08.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efde6ffd700 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf7828f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:57:08.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efde77fe700 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf7805080 0x561cf7833bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7efdd801f000 tx=0x7efdd8027040).ready entity=mon.0 client_cookie=8ce1bad03bdd1b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:08.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efdc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T20:57:08.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efdc5ffb700 1 -- 172.21.15.33:0/3988923478 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd804d4f0 con 0x561cf7805080 2023-11-06T20:57:08.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efdc5ffb700 1 -- 172.21.15.33:0/3988923478 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efdd804d650 con 0x561cf7805080 2023-11-06T20:57:08.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efe05038b80 1 -- 172.21.15.33:0/3988923478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cf77f8e20 con 0x561cf7805080 2023-11-06T20:57:08.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efdc5ffb700 1 -- 172.21.15.33:0/3988923478 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdd80504e0 con 0x561cf7805080 2023-11-06T20:57:08.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efe05038b80 10 client.5898 fetch_fsmap learned FSMap version 22 2023-11-06T20:57:08.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.116+0000 7efe05038b80 1 -- 172.21.15.33:0/3988923478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x561cf774aac0 con 0x561cf7805080 2023-11-06T20:57:08.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.117+0000 7efdc5ffb700 1 -- 172.21.15.33:0/3988923478 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561cf77f8e20 con 0x561cf7805080 2023-11-06T20:57:08.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.117+0000 7efdc5ffb700 1 -- 172.21.15.33:0/3988923478 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efdd8056210 con 0x561cf7805080 2023-11-06T20:57:08.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.117+0000 7efe05038b80 10 client.5898 fetch_fsmap finished waiting for FSMap version 22 2023-11-06T20:57:08.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.117+0000 7efe05038b80 10 client.5898 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-06T20:57:08.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.117+0000 7efe05038b80 20 client.5898 populate_metadata read hostname 'smithi033' 2023-11-06T20:57:08.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.117+0000 7efe05038b80 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:6834/698812717,v1:172.21.15.69:6837/698812717] conn(0x561cf78266d0 0x561cf7831020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.117+0000 7efe05038b80 4 client.5898 mds_command: new command op to 5748 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T20:57:08.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.117+0000 7efe05038b80 1 -- 172.21.15.33:0/3988923478 --> [v2:172.21.15.69:6834/698812717,v1:172.21.15.69:6837/698812717] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cf75beef0 con 0x561cf78266d0 2023-11-06T20:57:08.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.117+0000 7efde6ffd700 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:6834/698812717,v1:172.21.15.69:6837/698812717] conn(0x561cf78266d0 0x561cf7831020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.118+0000 7efde6ffd700 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:6834/698812717,v1:172.21.15.69:6837/698812717] conn(0x561cf78266d0 0x561cf7831020 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4e6121b75c55fc8a server_cookie=6d54b899f4f5e2e8 in_seq=0 out_seq=0 2023-11-06T20:57:08.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.118+0000 7efdc5ffb700 10 client.5898 ms_handle_connect on v2:172.21.15.69:6834/698812717 2023-11-06T20:57:08.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efdc5ffb700 1 -- 172.21.15.33:0/3988923478 <== mds.0 v2:172.21.15.69:6834/698812717 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x561cf75beef0 con 0x561cf78266d0 2023-11-06T20:57:08.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efdc5ffb700 10 client.5898 handle_command_reply: tid=0 2023-11-06T20:57:08.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 client.5898 shutdown 2023-11-06T20:57:08.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 -- 172.21.15.33:0/3988923478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf7805080 msgr2=0x561cf7833bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf7805080 0x561cf7833bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7efdd801f000 tx=0x7efdd8027040).stop 2023-11-06T20:57:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 -- 172.21.15.33:0/3988923478 shutdown_connections 2023-11-06T20:57:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:6834/698812717,v1:172.21.15.69:6837/698812717] conn(0x561cf78266d0 0x561cf7831020 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf784dbe0 0x561cf7800dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf7805080 0x561cf7833bd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efdc5ffb700 0 client.5898 ms_handle_reset on v2:172.21.15.69:6834/698812717 2023-11-06T20:57:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 --2- 172.21.15.33:0/3988923478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf7828f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 -- 172.21.15.33:0/3988923478 >> 172.21.15.33:0/3988923478 conn(0x561cf766e940 msgr2=0x561cf7803d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 -- 172.21.15.33:0/3988923478 shutdown_connections 2023-11-06T20:57:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.119+0000 7efe05038b80 1 -- 172.21.15.33:0/3988923478 wait complete. 2023-11-06T20:57:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.120+0000 7efe05038b80 20 client.5898 trim_cache size 0 max 16384 2023-11-06T20:57:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.137+0000 7efe05038b80 1 Processor -- start 2023-11-06T20:57:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.137+0000 7efe05038b80 1 -- start start 2023-11-06T20:57:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efe05038b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7805a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efe05038b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf77ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efe05038b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efe05038b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf77fd0c0 con 0x561cf784dbe0 2023-11-06T20:57:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efe05038b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf784f4d0 con 0x561cf782d4c0 2023-11-06T20:57:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efe05038b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cf7807180 con 0x561cf7805080 2023-11-06T20:57:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efde7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efde7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77fef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51342/0 (socket says 172.21.15.33:51342) 2023-11-06T20:57:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efde77fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7805a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efde7fff700 1 -- 172.21.15.33:0/632029986 learned_addr learned my addr 172.21.15.33:0/632029986 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efde6ffd700 1 --2- 172.21.15.33:0/632029986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf77ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efde7fff700 1 -- 172.21.15.33:0/632029986 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 msgr2=0x561cf7805a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efde7fff700 1 --2- 172.21.15.33:0/632029986 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7805a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efde7fff700 1 -- 172.21.15.33:0/632029986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 msgr2=0x561cf77ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efde7fff700 1 --2- 172.21.15.33:0/632029986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf77ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.138+0000 7efde7fff700 1 -- 172.21.15.33:0/632029986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cf784f310 con 0x561cf784dbe0 2023-11-06T20:57:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efde7fff700 1 --2- 172.21.15.33:0/632029986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77fef60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x561cf784e040 tx=0x7efde0009040).ready entity=mon.0 client_cookie=87a33abbbc3f435f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efdcd1ff700 1 -- 172.21.15.33:0/632029986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0001930 con 0x561cf784dbe0 2023-11-06T20:57:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efdcd1ff700 1 -- 172.21.15.33:0/632029986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde0001a90 con 0x561cf784dbe0 2023-11-06T20:57:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efdcd1ff700 1 -- 172.21.15.33:0/632029986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0001e10 con 0x561cf784dbe0 2023-11-06T20:57:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efe05038b80 1 -- 172.21.15.33:0/632029986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 msgr2=0x561cf77fef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efe05038b80 1 --2- 172.21.15.33:0/632029986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77fef60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x561cf784e040 tx=0x7efde0009040).stop 2023-11-06T20:57:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efe05038b80 1 -- 172.21.15.33:0/632029986 shutdown_connections 2023-11-06T20:57:08.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efe05038b80 1 --2- 172.21.15.33:0/632029986 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7805a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efe05038b80 1 --2- 172.21.15.33:0/632029986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf784dbe0 0x561cf77fef60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efe05038b80 1 --2- 172.21.15.33:0/632029986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf782d4c0 0x561cf77ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efe05038b80 1 -- 172.21.15.33:0/632029986 >> 172.21.15.33:0/632029986 conn(0x561cf766e940 msgr2=0x561cf7832da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:08.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efe05038b80 1 -- 172.21.15.33:0/632029986 shutdown_connections 2023-11-06T20:57:08.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.139+0000 7efe05038b80 1 -- 172.21.15.33:0/632029986 wait complete. 2023-11-06T20:57:08.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.140+0000 7efe05038b80 1 -- start start 2023-11-06T20:57:08.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.140+0000 7efe05038b80 1 Processor -- start 2023-11-06T20:57:08.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efe05038b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7852920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efe05038b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf782d4c0 0x561cf7851870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efe05038b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf784dbe0 0x561cf7851d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efe05038b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7852250 con 0x561cf782d4c0 2023-11-06T20:57:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efe05038b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf78b0f70 con 0x561cf784dbe0 2023-11-06T20:57:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efe05038b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cf785bc10 con 0x561cf7805080 2023-11-06T20:57:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efe05038b80 10 client.0 ll_register_callbacks cb 0x561cf76f6ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T20:57:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efde6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf782d4c0 0x561cf7851870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efde6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf782d4c0 0x561cf7851870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51344/0 (socket says 172.21.15.33:51344) 2023-11-06T20:57:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efde6ffd700 1 -- 172.21.15.33:0/2457001905 learned_addr learned my addr 172.21.15.33:0/2457001905 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.142+0000 7efde7fff700 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf784dbe0 0x561cf7851d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efde77fe700 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7852920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efde6ffd700 1 -- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 msgr2=0x561cf7852920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:08.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efde6ffd700 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7852920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efde6ffd700 1 -- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf784dbe0 msgr2=0x561cf7851d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:08.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efde6ffd700 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf784dbe0 0x561cf7851d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efde6ffd700 1 -- 172.21.15.33:0/2457001905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde000f040 con 0x561cf782d4c0 2023-11-06T20:57:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efde6ffd700 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf782d4c0 0x561cf7851870 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7efddc018670 tx=0x7efddc016000).ready entity=mon.0 client_cookie=a1ef8e0764ef5651 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efdc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T20:57:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efdc5ffb700 1 -- 172.21.15.33:0/2457001905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc029040 con 0x561cf782d4c0 2023-11-06T20:57:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efdc5ffb700 1 -- 172.21.15.33:0/2457001905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efddc018240 con 0x561cf782d4c0 2023-11-06T20:57:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efe05038b80 1 -- 172.21.15.33:0/2457001905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efddc015410 con 0x561cf782d4c0 2023-11-06T20:57:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efdc5ffb700 1 -- 172.21.15.33:0/2457001905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7efddc02d280 con 0x561cf782d4c0 2023-11-06T20:57:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efe05038b80 10 client.5904 fetch_fsmap learned FSMap version 22 2023-11-06T20:57:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.143+0000 7efe05038b80 1 -- 172.21.15.33:0/2457001905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x561cf774a810 con 0x561cf782d4c0 2023-11-06T20:57:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.144+0000 7efdc5ffb700 1 -- 172.21.15.33:0/2457001905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc015410 con 0x561cf782d4c0 2023-11-06T20:57:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.144+0000 7efdc5ffb700 1 -- 172.21.15.33:0/2457001905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efddc020020 con 0x561cf782d4c0 2023-11-06T20:57:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.144+0000 7efe05038b80 10 client.5904 fetch_fsmap finished waiting for FSMap version 22 2023-11-06T20:57:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.144+0000 7efe05038b80 10 client.5904 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-06T20:57:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.144+0000 7efe05038b80 20 client.5904 populate_metadata read hostname 'smithi033' 2023-11-06T20:57:08.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.144+0000 7efe05038b80 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:6834/698812717,v1:172.21.15.69:6837/698812717] conn(0x561cf77ff390 0x561cf77ff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.144+0000 7efe05038b80 4 client.5904 mds_command: new command op to 5748 tid=0[{"prefix": "status"}] 2023-11-06T20:57:08.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.144+0000 7efe05038b80 1 -- 172.21.15.33:0/2457001905 --> [v2:172.21.15.69:6834/698812717,v1:172.21.15.69:6837/698812717] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561cf75f4040 con 0x561cf77ff390 2023-11-06T20:57:08.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.144+0000 7efde77fe700 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:6834/698812717,v1:172.21.15.69:6837/698812717] conn(0x561cf77ff390 0x561cf77ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efdc5ffb700 10 client.5904 ms_handle_connect on v2:172.21.15.69:6834/698812717 2023-11-06T20:57:08.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efde77fe700 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:6834/698812717,v1:172.21.15.69:6837/698812717] conn(0x561cf77ff390 0x561cf77ff770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c0365a22dfd90b1f server_cookie=13558fe0df5c0f86 in_seq=0 out_seq=0 2023-11-06T20:57:08.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efdc5ffb700 1 -- 172.21.15.33:0/2457001905 <== mds.0 v2:172.21.15.69:6834/698812717 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561cf75f4040 con 0x561cf77ff390 2023-11-06T20:57:08.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efdc5ffb700 10 client.5904 handle_command_reply: tid=0 2023-11-06T20:57:08.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 client.5904 shutdown 2023-11-06T20:57:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 -- 172.21.15.33:0/2457001905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf782d4c0 msgr2=0x561cf7851870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf782d4c0 0x561cf7851870 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7efddc018670 tx=0x7efddc016000).stop 2023-11-06T20:57:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 -- 172.21.15.33:0/2457001905 shutdown_connections 2023-11-06T20:57:08.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:6834/698812717,v1:172.21.15.69:6837/698812717] conn(0x561cf77ff390 0x561cf77ff770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x561cf7805080 0x561cf7852920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf784dbe0 0x561cf7851d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 --2- 172.21.15.33:0/2457001905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561cf782d4c0 0x561cf7851870 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efdc5ffb700 0 client.5904 ms_handle_reset on v2:172.21.15.69:6834/698812717 2023-11-06T20:57:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 -- 172.21.15.33:0/2457001905 >> 172.21.15.33:0/2457001905 conn(0x561cf766e940 msgr2=0x561cf7832cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 -- 172.21.15.33:0/2457001905 shutdown_connections 2023-11-06T20:57:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.145+0000 7efe05038b80 1 -- 172.21.15.33:0/2457001905 wait complete. 2023-11-06T20:57:08.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efe05038b80 20 client.5904 trim_cache size 0 max 16384 2023-11-06T20:57:08.164 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T20:57:08.164 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T20:57:08.164 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T20:57:08.164 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 5748, 2023-11-06T20:57:08.165 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T20:57:08.165 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T20:57:08.165 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T20:57:08.165 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.4462530170000001, 2023-11-06T20:57:08.165 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 22, 2023-11-06T20:57:08.166 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 66, 2023-11-06T20:57:08.166 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 66, 2023-11-06T20:57:08.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 11.648205122 2023-11-06T20:57:08.166 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T20:57:08.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 -- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efdd405f270 msgr2=0x7efdd4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efdd405f270 0x7efdd4061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7efde8154b80 tx=0x7efddc009040).stop 2023-11-06T20:57:08.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 -- 172.21.15.33:0/577719110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde80be2c0 msgr2=0x7efde80ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde80be2c0 0x7efde80ae300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7efdd8009130 tx=0x7efdd8017000).stop 2023-11-06T20:57:08.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 -- 172.21.15.33:0/577719110 shutdown_connections 2023-11-06T20:57:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efdd405f270 0x7efdd4061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde8152310 0x7efde80c1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde80be2c0 0x7efde80ae300 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 --2- 172.21.15.33:0/577719110 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efde8151f30 0x7efde80c25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 -- 172.21.15.33:0/577719110 >> 172.21.15.33:0/577719110 conn(0x7efde80c56f0 msgr2=0x7efde81516d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:08.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 -- 172.21.15.33:0/577719110 shutdown_connections 2023-11-06T20:57:08.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.146+0000 7efdeddff700 1 -- 172.21.15.33:0/577719110 wait complete. 2023-11-06T20:57:08.170 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T20:57:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.336+0000 7fd79d9ff700 1 Processor -- start 2023-11-06T20:57:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.336+0000 7fd79d9ff700 1 -- start start 2023-11-06T20:57:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.337+0000 7fd79d9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd7980c07b0 0x7fd7980c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.337+0000 7fd79d9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7980c5f80 con 0x7fd7981541a0 2023-11-06T20:57:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.337+0000 7fd79d9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7980aa740 con 0x7fd7980a9ee0 2023-11-06T20:57:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.337+0000 7fd79d9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7980c6800 con 0x7fd7980c07b0 2023-11-06T20:57:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd796ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd7981541a0 0x7fd7980c8220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49540/0 (socket says 172.21.15.33:49540) 2023-11-06T20:57:08.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd796ffd700 1 -- 172.21.15.33:0/3953449145 learned_addr learned my addr 172.21.15.33:0/3953449145 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:57:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd7977fe700 1 --2- 172.21.15.33:0/3953449145 >> v2:172.21.15.69:3300/0 conn(0x7fd7980c07b0 0x7fd7980c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd795ffb700 1 -- 172.21.15.33:0/3953449145 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449465823 0 0) 0x7fd7980c5f80 con 0x7fd7981541a0 2023-11-06T20:57:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd795ffb700 1 -- 172.21.15.33:0/3953449145 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7800033f0 con 0x7fd7981541a0 2023-11-06T20:57:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd795ffb700 1 -- 172.21.15.33:0/3953449145 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139959202 0 0) 0x7fd7980aa740 con 0x7fd7980a9ee0 2023-11-06T20:57:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd795ffb700 1 -- 172.21.15.33:0/3953449145 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7980c5f80 con 0x7fd7980a9ee0 2023-11-06T20:57:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd7977fe700 1 -- 172.21.15.33:0/3953449145 >> v1:172.21.15.69:6789/0 conn(0x7fd7980a9ee0 legacy=0x7fd7980c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd795ffb700 1 -- 172.21.15.33:0/3953449145 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4091363126 0 0) 0x7fd7800033f0 con 0x7fd7981541a0 2023-11-06T20:57:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd7977fe700 1 -- 172.21.15.33:0/3953449145 >> v1:172.21.15.33:6789/0 conn(0x7fd7981541a0 legacy=0x7fd7980c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.338+0000 7fd7977fe700 1 -- 172.21.15.33:0/3953449145 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7980c73a0 con 0x7fd7980c07b0 2023-11-06T20:57:08.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.339+0000 7fd7977fe700 1 --2- 172.21.15.33:0/3953449145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7980c07b0 0x7fd7980c0c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fd790009e60 tx=0x7fd790009780).ready entity=mon.1 client_cookie=f4210b2a7a47c656 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:08.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.339+0000 7fd795ffb700 1 -- 172.21.15.33:0/3953449145 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79000e070 con 0x7fd7980c07b0 2023-11-06T20:57:08.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.339+0000 7fd795ffb700 1 -- 172.21.15.33:0/3953449145 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79000b7a0 con 0x7fd7980c07b0 2023-11-06T20:57:08.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.339+0000 7fd795ffb700 1 -- 172.21.15.33:0/3953449145 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880015f0 con 0x7fd7980c07b0 2023-11-06T20:57:08.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.339+0000 7fd79d9ff700 1 -- 172.21.15.33:0/3953449145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7980c07b0 msgr2=0x7fd7980c0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.339+0000 7fd79d9ff700 1 --2- 172.21.15.33:0/3953449145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7980c07b0 0x7fd7980c0c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fd790009e60 tx=0x7fd790009780).stop 2023-11-06T20:57:08.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.339+0000 7fd79d9ff700 1 -- 172.21.15.33:0/3953449145 shutdown_connections 2023-11-06T20:57:08.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.339+0000 7fd79d9ff700 1 --2- 172.21.15.33:0/3953449145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7980c07b0 0x7fd7980c0c40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.339+0000 7fd79d9ff700 1 -- 172.21.15.33:0/3953449145 >> 172.21.15.33:0/3953449145 conn(0x7fd7980944e0 msgr2=0x7fd79806a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:08.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.340+0000 7fd79d9ff700 1 -- 172.21.15.33:0/3953449145 shutdown_connections 2023-11-06T20:57:08.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.340+0000 7fd79d9ff700 1 -- 172.21.15.33:0/3953449145 wait complete. 2023-11-06T20:57:08.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.341+0000 7fd79d9ff700 1 Processor -- start 2023-11-06T20:57:08.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.341+0000 7fd79d9ff700 1 -- start start 2023-11-06T20:57:08.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd79d9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7980a9ee0 0x7fd7980ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd79d9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7980c07b0 0x7fd7980abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd79d9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7981541a0 0x7fd798164a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd79d9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980ac220 con 0x7fd7981541a0 2023-11-06T20:57:08.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd79d9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980af000 con 0x7fd7980c07b0 2023-11-06T20:57:08.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd79d9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980af490 con 0x7fd7980a9ee0 2023-11-06T20:57:08.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd7977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7981541a0 0x7fd798164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd7977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7981541a0 0x7fd798164a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51350/0 (socket says 172.21.15.33:51350) 2023-11-06T20:57:08.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd7977fe700 1 -- 172.21.15.33:0/4127763736 learned_addr learned my addr 172.21.15.33:0/4127763736 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:08.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd7967fc700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7980c07b0 0x7fd7980abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.342+0000 7fd796ffd700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7980a9ee0 0x7fd7980ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.343+0000 7fd7977fe700 1 -- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7980a9ee0 msgr2=0x7fd7980ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.343+0000 7fd7977fe700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7980a9ee0 0x7fd7980ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.343+0000 7fd7977fe700 1 -- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7980c07b0 msgr2=0x7fd7980abd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.343+0000 7fd7977fe700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7980c07b0 0x7fd7980abd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.343+0000 7fd7977fe700 1 -- 172.21.15.33:0/4127763736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd790009a20 con 0x7fd7981541a0 2023-11-06T20:57:08.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.343+0000 7fd7977fe700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7981541a0 0x7fd798164a60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fd79000b610 tx=0x7fd790009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:08.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.343+0000 7fd77effd700 1 -- 172.21.15.33:0/4127763736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79001d440 con 0x7fd7981541a0 2023-11-06T20:57:08.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.343+0000 7fd77effd700 1 -- 172.21.15.33:0/4127763736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79001d5a0 con 0x7fd7981541a0 2023-11-06T20:57:08.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.343+0000 7fd77effd700 1 -- 172.21.15.33:0/4127763736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79001d940 con 0x7fd7981541a0 2023-11-06T20:57:08.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.343+0000 7fd79d9ff700 1 -- 172.21.15.33:0/4127763736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7980ae3f0 con 0x7fd7981541a0 2023-11-06T20:57:08.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.344+0000 7fd79d9ff700 1 -- 172.21.15.33:0/4127763736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7980ae9b0 con 0x7fd7981541a0 2023-11-06T20:57:08.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.344+0000 7fd79d9ff700 1 -- 172.21.15.33:0/4127763736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7980af190 con 0x7fd7981541a0 2023-11-06T20:57:08.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.346+0000 7fd77effd700 1 -- 172.21.15.33:0/4127763736 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd7900263b0 con 0x7fd7981541a0 2023-11-06T20:57:08.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.347+0000 7fd77effd700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd78405f270 0x7fd784061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:08.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.347+0000 7fd77effd700 1 -- 172.21.15.33:0/4127763736 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd79007c160 con 0x7fd7981541a0 2023-11-06T20:57:08.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.347+0000 7fd796ffd700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd78405f270 0x7fd784061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:08.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.347+0000 7fd796ffd700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd78405f270 0x7fd784061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd788001130 tx=0x7fd788002a30).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:08.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.351+0000 7fd77effd700 1 -- 172.21.15.33:0/4127763736 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd79004d190 con 0x7fd7981541a0 2023-11-06T20:57:08.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.580+0000 7fd79d9ff700 1 -- 172.21.15.33:0/4127763736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd7980475a0 con 0x7fd7981541a0 2023-11-06T20:57:08.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.580+0000 7fd77effd700 1 -- 172.21.15.33:0/4127763736 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd790022020 con 0x7fd7981541a0 2023-11-06T20:57:08.580 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T20:57:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.582+0000 7fd79d9ff700 1 -- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd78405f270 msgr2=0x7fd784061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.582+0000 7fd79d9ff700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd78405f270 0x7fd784061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd788001130 tx=0x7fd788002a30).stop 2023-11-06T20:57:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.582+0000 7fd79d9ff700 1 -- 172.21.15.33:0/4127763736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7981541a0 msgr2=0x7fd798164a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.582+0000 7fd79d9ff700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7981541a0 0x7fd798164a60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fd79000b610 tx=0x7fd790009780).stop 2023-11-06T20:57:08.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.583+0000 7fd79d9ff700 1 -- 172.21.15.33:0/4127763736 shutdown_connections 2023-11-06T20:57:08.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.583+0000 7fd79d9ff700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd78405f270 0x7fd784061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.583+0000 7fd79d9ff700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7980a9ee0 0x7fd7980ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.583+0000 7fd79d9ff700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7981541a0 0x7fd798164a60 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.583+0000 7fd79d9ff700 1 --2- 172.21.15.33:0/4127763736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7980c07b0 0x7fd7980abd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.583+0000 7fd79d9ff700 1 -- 172.21.15.33:0/4127763736 >> 172.21.15.33:0/4127763736 conn(0x7fd7980944e0 msgr2=0x7fd798063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.583+0000 7fd79d9ff700 1 -- 172.21.15.33:0/4127763736 shutdown_connections 2023-11-06T20:57:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:08.583+0000 7fd79d9ff700 1 -- 172.21.15.33:0/4127763736 wait complete. 2023-11-06T20:57:08.597 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T20:57:08.597 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T20:57:08.598 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T20:57:08.598 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T20:57:08.598 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T20:57:08.599 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T20:57:08.599 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T20:57:08.600 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T20:57:08.600 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T20:57:08.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:57:08.601 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T20:57:08.650 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:57:08.651 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:57:08.651 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:57:08.651 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:57:08.651 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T20:57:08.652 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:57:08.652 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:57:08.652 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:57:08.652 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T20:57:08.652 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:57:08.653 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:57:08.653 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T20:57:08.653 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T20:57:08.653 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:57:08.653 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T20:57:08.654 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T20:57:08.654 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41515sec preferred_lft 41515sec 2023-11-06T20:57:08.654 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T20:57:08.655 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:57:08.655 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:57:08.655 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:57:08.655 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T20:57:08.655 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T20:57:08.655 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T20:57:08.656 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T20:57:08.656 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:57:08.656 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T20:57:08.657 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:57:08.657 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T20:57:08.657 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T20:57:08.657 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T20:57:08.658 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:57:08.658 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T20:57:08.658 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T20:57:08.658 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T20:57:08.658 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:57:08.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:57:08.659 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T20:57:08.729 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T20:57:08.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:57:08.733 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T20:57:08.783 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T20:57:08.783 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T20:57:08.784 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T20:57:08.784 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T20:57:08.784 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T20:57:08.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:57:08.785 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:57:08.785 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:57:08.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T20:57:08.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T20:57:08.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T20:57:08.787 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:57:08.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:57:08.962 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T20:57:09.010 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T20:57:09.010 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T20:57:09.012 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T20:57:09.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:57:09.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T20:57:09.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:57:09.087 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T20:57:09.134 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T20:57:09.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:57:09.278 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:57:09.278 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:57:09.305 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:57:09.305 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:57:09.310 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T20:57:09.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:57:09.311 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T20:57:09.384 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T20:57:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.582+0000 7f87e59ff700 1 Processor -- start 2023-11-06T20:57:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.582+0000 7f87e59ff700 1 -- start start 2023-11-06T20:57:09.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.583+0000 7f87e59ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87e0154210 0x7f87e00a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:09.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.583+0000 7f87e59ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87e00bfd60 0x7f87e00c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:09.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.583+0000 7f87e59ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e00c06d0 con 0x7f87e00c9f00 2023-11-06T20:57:09.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.583+0000 7f87e59ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e00caaa0 con 0x7f87e0154210 2023-11-06T20:57:09.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.583+0000 7f87e59ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e005efc0 con 0x7f87e00bfd60 2023-11-06T20:57:09.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.584+0000 7f87deffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87e0154210 0x7f87e00a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:09.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.584+0000 7f87deffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87e0154210 0x7f87e00a7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51358/0 (socket says 172.21.15.33:51358) 2023-11-06T20:57:09.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.584+0000 7f87deffd700 1 -- 172.21.15.33:0/2302182597 learned_addr learned my addr 172.21.15.33:0/2302182597 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:09.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.584+0000 7f87de7fc700 1 --2- 172.21.15.33:0/2302182597 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87e00bfd60 0x7f87e00c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:09.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.584+0000 7f87deffd700 1 -- 172.21.15.33:0/2302182597 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87e00bfd60 msgr2=0x7f87e00c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:09.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.584+0000 7f87deffd700 1 --2- 172.21.15.33:0/2302182597 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87e00bfd60 0x7f87e00c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:09.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.584+0000 7f87deffd700 1 -- 172.21.15.33:0/2302182597 >> v1:172.21.15.69:6789/0 conn(0x7f87e00c9f00 legacy=0x7f87e00ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:09.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.584+0000 7f87deffd700 1 -- 172.21.15.33:0/2302182597 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e00c8ea0 con 0x7f87e0154210 2023-11-06T20:57:09.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.584+0000 7f87deffd700 1 --2- 172.21.15.33:0/2302182597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 0x7f87e00a7bd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f87d0006fd0 tx=0x7f87d00090f0).ready entity=mon.0 client_cookie=4edbeaa954aa37e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87ddffb700 1 -- 172.21.15.33:0/2302182597 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d000b600 con 0x7f87e0154210 2023-11-06T20:57:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87ddffb700 1 -- 172.21.15.33:0/2302182597 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d000b760 con 0x7f87e0154210 2023-11-06T20:57:09.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87ddffb700 1 -- 172.21.15.33:0/2302182597 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d000bae0 con 0x7f87e0154210 2023-11-06T20:57:09.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87e59ff700 1 -- 172.21.15.33:0/2302182597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 msgr2=0x7f87e00a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:09.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87e59ff700 1 --2- 172.21.15.33:0/2302182597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 0x7f87e00a7bd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f87d0006fd0 tx=0x7f87d00090f0).stop 2023-11-06T20:57:09.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87e59ff700 1 -- 172.21.15.33:0/2302182597 shutdown_connections 2023-11-06T20:57:09.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87e59ff700 1 --2- 172.21.15.33:0/2302182597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 0x7f87e00a7bd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:09.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87e59ff700 1 --2- 172.21.15.33:0/2302182597 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87e00bfd60 0x7f87e00c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:09.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87e59ff700 1 -- 172.21.15.33:0/2302182597 >> 172.21.15.33:0/2302182597 conn(0x7f87e005e990 msgr2=0x7f87e00cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:09.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87e59ff700 1 -- 172.21.15.33:0/2302182597 shutdown_connections 2023-11-06T20:57:09.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.585+0000 7f87e59ff700 1 -- 172.21.15.33:0/2302182597 wait complete. 2023-11-06T20:57:09.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.587+0000 7f87e59ff700 1 Processor -- start 2023-11-06T20:57:09.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.587+0000 7f87e59ff700 1 -- start start 2023-11-06T20:57:09.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87e59ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87e00bfd60 0x7f87e00c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:09.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87e59ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87e00c9f00 0x7f87e014e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:09.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87e59ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 0x7f87e0152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:09.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87e59ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e00ae060 con 0x7f87e0154210 2023-11-06T20:57:09.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87e59ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e014eb10 con 0x7f87e00bfd60 2023-11-06T20:57:09.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87e59ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e00ae300 con 0x7f87e00c9f00 2023-11-06T20:57:09.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 0x7f87e0152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:09.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 0x7f87e0152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51372/0 (socket says 172.21.15.33:51372) 2023-11-06T20:57:09.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87df7fe700 1 -- 172.21.15.33:0/2202158314 learned_addr learned my addr 172.21.15.33:0/2202158314 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:09.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87deffd700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87e00bfd60 0x7f87e00c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:09.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.588+0000 7f87de7fc700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87e00c9f00 0x7f87e014e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:09.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87df7fe700 1 -- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87e00c9f00 msgr2=0x7f87e014e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:09.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87df7fe700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87e00c9f00 0x7f87e014e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:09.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87df7fe700 1 -- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87e00bfd60 msgr2=0x7f87e00c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:09.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87df7fe700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87e00bfd60 0x7f87e00c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:09.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87df7fe700 1 -- 172.21.15.33:0/2202158314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0009370 con 0x7f87e0154210 2023-11-06T20:57:09.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87deffd700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87e00bfd60 0x7f87e00c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:57:09.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87df7fe700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 0x7f87e0152f90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f87d8002230 tx=0x7f87d80015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:09.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87c6ffd700 1 -- 172.21.15.33:0/2202158314 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8010510 con 0x7f87e0154210 2023-11-06T20:57:09.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87c6ffd700 1 -- 172.21.15.33:0/2202158314 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d8010670 con 0x7f87e0154210 2023-11-06T20:57:09.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87c6ffd700 1 -- 172.21.15.33:0/2202158314 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d80109f0 con 0x7f87e0154210 2023-11-06T20:57:09.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87e59ff700 1 -- 172.21.15.33:0/2202158314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e00ae5e0 con 0x7f87e0154210 2023-11-06T20:57:09.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.589+0000 7f87e59ff700 1 -- 172.21.15.33:0/2202158314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e00aeba0 con 0x7f87e0154210 2023-11-06T20:57:09.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.591+0000 7f87e59ff700 1 -- 172.21.15.33:0/2202158314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e0157c90 con 0x7f87e0154210 2023-11-06T20:57:09.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.592+0000 7f87c6ffd700 1 -- 172.21.15.33:0/2202158314 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f87d801b400 con 0x7f87e0154210 2023-11-06T20:57:09.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.592+0000 7f87c6ffd700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f87cc05f270 0x7f87cc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:09.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.592+0000 7f87c6ffd700 1 -- 172.21.15.33:0/2202158314 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f87d807a140 con 0x7f87e0154210 2023-11-06T20:57:09.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.593+0000 7f87deffd700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f87cc05f270 0x7f87cc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:09.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.593+0000 7f87deffd700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f87cc05f270 0x7f87cc061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f87d0009130 tx=0x7f87d0017000).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:09.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.597+0000 7f87c6ffd700 1 -- 172.21.15.33:0/2202158314 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f87e0157c90 con 0x7f87e0154210 2023-11-06T20:57:09.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.806+0000 7f87e59ff700 1 -- 172.21.15.33:0/2202158314 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f87e0157c90 con 0x7f87cc05f270 2023-11-06T20:57:09.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.808+0000 7f87c6ffd700 1 -- 172.21.15.33:0/2202158314 <== mgr.5443 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f87e0157c90 con 0x7f87cc05f270 2023-11-06T20:57:09.807 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T20:57:09.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:57:09.808 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T20:57:09.808 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:57:09.808 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T20:57:09.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 -- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f87cc05f270 msgr2=0x7f87cc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:09.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f87cc05f270 0x7f87cc061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f87d0009130 tx=0x7f87d0017000).stop 2023-11-06T20:57:09.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 -- 172.21.15.33:0/2202158314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 msgr2=0x7f87e0152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:09.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 0x7f87e0152f90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f87d8002230 tx=0x7f87d80015e0).stop 2023-11-06T20:57:09.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 -- 172.21.15.33:0/2202158314 shutdown_connections 2023-11-06T20:57:09.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f87cc05f270 0x7f87cc061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:09.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e0154210 0x7f87e0152f90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:09.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87e00bfd60 0x7f87e00c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:09.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 --2- 172.21.15.33:0/2202158314 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87e00c9f00 0x7f87e014e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:09.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 -- 172.21.15.33:0/2202158314 >> 172.21.15.33:0/2202158314 conn(0x7f87e005e990 msgr2=0x7f87e0064050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:09.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 -- 172.21.15.33:0/2202158314 shutdown_connections 2023-11-06T20:57:09.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:09.810+0000 7f87e59ff700 1 -- 172.21.15.33:0/2202158314 wait complete. 2023-11-06T20:57:09.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T20:57:10.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.025+0000 7fdd913ff700 1 Processor -- start 2023-11-06T20:57:10.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.025+0000 7fdd913ff700 1 -- start start 2023-11-06T20:57:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.026+0000 7fdd913ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 0x7fdd8c063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.026+0000 7fdd913ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdd8c0aa4f0 0x7fdd8c063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.027+0000 7fdd913ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8c158c00 con 0x7fdd8c157700 2023-11-06T20:57:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.027+0000 7fdd913ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd8c0c18f0 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.027+0000 7fdd913ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c063b10 con 0x7fdd8c0aaf50 2023-11-06T20:57:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.027+0000 7fdd8a7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 0x7fdd8c063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.027+0000 7fdd8b7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdd8c0aa4f0 0x7fdd8c063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.027+0000 7fdd8b7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdd8c0aa4f0 0x7fdd8c063620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:38230/0 (socket says 172.21.15.33:38230) 2023-11-06T20:57:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.027+0000 7fdd8a7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 0x7fdd8c063130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:53458/0 (socket says 172.21.15.33:53458) 2023-11-06T20:57:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.027+0000 7fdd8b7fe700 1 -- 172.21.15.33:0/717265997 learned_addr learned my addr 172.21.15.33:0/717265997 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.027+0000 7fdd8affd700 1 --1- 172.21.15.33:0/717265997 >> v1:172.21.15.69:6789/0 conn(0x7fdd8c157700 0x7fdd8c0c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:41574/0 (socket says 172.21.15.33:41574) 2023-11-06T20:57:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.028+0000 7fdd8b7fe700 1 -- 172.21.15.33:0/717265997 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 msgr2=0x7fdd8c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.028+0000 7fdd8b7fe700 1 --2- 172.21.15.33:0/717265997 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 0x7fdd8c063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.028+0000 7fdd8b7fe700 1 -- 172.21.15.33:0/717265997 >> v1:172.21.15.69:6789/0 conn(0x7fdd8c157700 legacy=0x7fdd8c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.028+0000 7fdd89ffb700 1 -- 172.21.15.33:0/717265997 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 416310198 0 0) 0x7fdd8c158c00 con 0x7fdd8c157700 2023-11-06T20:57:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.028+0000 7fdd8b7fe700 1 -- 172.21.15.33:0/717265997 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c0c7610 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.028+0000 7fdd8b7fe700 1 --2- 172.21.15.33:0/717265997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd8c0aa4f0 0x7fdd8c063620 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fdd84009f50 tx=0x7fdd84009840).ready entity=mon.1 client_cookie=6a8ab86bc87a071a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.028+0000 7fdd89ffb700 1 -- 172.21.15.33:0/717265997 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e070 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.028+0000 7fdd89ffb700 1 -- 172.21.15.33:0/717265997 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8400b9c0 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.028+0000 7fdd89ffb700 1 -- 172.21.15.33:0/717265997 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400bd40 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.029+0000 7fdd913ff700 1 -- 172.21.15.33:0/717265997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd8c0aa4f0 msgr2=0x7fdd8c063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.029+0000 7fdd913ff700 1 --2- 172.21.15.33:0/717265997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd8c0aa4f0 0x7fdd8c063620 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fdd84009f50 tx=0x7fdd84009840).stop 2023-11-06T20:57:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.029+0000 7fdd913ff700 1 -- 172.21.15.33:0/717265997 shutdown_connections 2023-11-06T20:57:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.029+0000 7fdd913ff700 1 --2- 172.21.15.33:0/717265997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd8c0aa4f0 0x7fdd8c063620 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.029+0000 7fdd913ff700 1 --2- 172.21.15.33:0/717265997 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 0x7fdd8c063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.029+0000 7fdd913ff700 1 -- 172.21.15.33:0/717265997 >> 172.21.15.33:0/717265997 conn(0x7fdd8c05ec80 msgr2=0x7fdd8c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.029+0000 7fdd913ff700 1 -- 172.21.15.33:0/717265997 shutdown_connections 2023-11-06T20:57:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.029+0000 7fdd913ff700 1 -- 172.21.15.33:0/717265997 wait complete. 2023-11-06T20:57:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.031+0000 7fdd913ff700 1 Processor -- start 2023-11-06T20:57:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.031+0000 7fdd913ff700 1 -- start start 2023-11-06T20:57:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.032+0000 7fdd913ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd8c0aa4f0 0x7fdd8c13b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.032+0000 7fdd913ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 0x7fdd8c0afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd913ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd8c157700 0x7fdd8c0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd913ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c0b0ef0 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd913ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c0c2ad0 con 0x7fdd8c157700 2023-11-06T20:57:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd913ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c0b0650 con 0x7fdd8c0aaf50 2023-11-06T20:57:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd8affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd8c0aa4f0 0x7fdd8c13b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd8affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd8c0aa4f0 0x7fdd8c13b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51376/0 (socket says 172.21.15.33:51376) 2023-11-06T20:57:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd8affd700 1 -- 172.21.15.33:0/4262111074 learned_addr learned my addr 172.21.15.33:0/4262111074 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd8a7fc700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 0x7fdd8c0afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd8b7fe700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd8c157700 0x7fdd8c0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd8affd700 1 -- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 msgr2=0x7fdd8c0afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd8affd700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 0x7fdd8c0afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd8affd700 1 -- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd8c157700 msgr2=0x7fdd8c0c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd8affd700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd8c157700 0x7fdd8c0c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.033+0000 7fdd8affd700 1 -- 172.21.15.33:0/4262111074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84009ae0 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.034+0000 7fdd8affd700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd8c0aa4f0 0x7fdd8c13b1b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fdd7c00c560 tx=0x7fdd7c007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.034+0000 7fdd72ffd700 1 -- 172.21.15.33:0/4262111074 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0072e0 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.034+0000 7fdd72ffd700 1 -- 172.21.15.33:0/4262111074 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c007440 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.034+0000 7fdd913ff700 1 -- 172.21.15.33:0/4262111074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8c0b0930 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.034+0000 7fdd72ffd700 1 -- 172.21.15.33:0/4262111074 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0058f0 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.034+0000 7fdd913ff700 1 -- 172.21.15.33:0/4262111074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8c13b6a0 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.035+0000 7fdd913ff700 1 -- 172.21.15.33:0/4262111074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8c06c6c0 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.036+0000 7fdd72ffd700 1 -- 172.21.15.33:0/4262111074 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdd7c005ce0 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.037+0000 7fdd72ffd700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd7805f270 0x7fdd78061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.037+0000 7fdd72ffd700 1 -- 172.21.15.33:0/4262111074 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdd7c079b90 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.037+0000 7fdd8a7fc700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd7805f270 0x7fdd78061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:10.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.038+0000 7fdd8a7fc700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd7805f270 0x7fdd78061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fdd8c0b10c0 tx=0x7fdd80009060).ready entity=mgr.5443 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:10.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.040+0000 7fdd72ffd700 1 -- 172.21.15.33:0/4262111074 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdd7c04ac20 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.270+0000 7fdd913ff700 1 -- 172.21.15.33:0/4262111074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fdd8c0bfe50 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.959+0000 7fdd72ffd700 1 -- 172.21.15.33:0/4262111074 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 15) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fdd7c077720 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.961+0000 7fdd72ffd700 1 -- 172.21.15.33:0/4262111074 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v15) v1 ==== 92+0+0 (secure 0 0 0) 0x7fdd7c076370 con 0x7fdd8c0aa4f0 2023-11-06T20:57:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 -- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd7805f270 msgr2=0x7fdd78061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd7805f270 0x7fdd78061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fdd8c0b10c0 tx=0x7fdd80009060).stop 2023-11-06T20:57:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 -- 172.21.15.33:0/4262111074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd8c0aa4f0 msgr2=0x7fdd8c13b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:10.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd8c0aa4f0 0x7fdd8c13b1b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fdd7c00c560 tx=0x7fdd7c007c90).stop 2023-11-06T20:57:10.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 -- 172.21.15.33:0/4262111074 shutdown_connections 2023-11-06T20:57:10.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdd7805f270 0x7fdd78061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:10.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd8c157700 0x7fdd8c0c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:10.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd8c0aa4f0 0x7fdd8c13b1b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:10.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 --2- 172.21.15.33:0/4262111074 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdd8c0aaf50 0x7fdd8c0afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:10.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 -- 172.21.15.33:0/4262111074 >> 172.21.15.33:0/4262111074 conn(0x7fdd8c05ec80 msgr2=0x7fdd8c0c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:10.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.964+0000 7fdd913ff700 1 -- 172.21.15.33:0/4262111074 shutdown_connections 2023-11-06T20:57:10.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:10.965+0000 7fdd913ff700 1 -- 172.21.15.33:0/4262111074 wait complete. 2023-11-06T20:57:10.984 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T20:57:11.173 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:11.174+0000 7f4ebfc2c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:57:11.179 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:11.180+0000 7f285b126680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:57:11.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.193+0000 7feb959ff700 1 Processor -- start 2023-11-06T20:57:11.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.193+0000 7feb959ff700 1 -- start start 2023-11-06T20:57:11.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.194+0000 7feb959ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb901546a0 0x7feb90064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.194+0000 7feb959ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb90066560 0x7feb900669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.194+0000 7feb959ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb900641f0 con 0x7feb900bdbb0 2023-11-06T20:57:11.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.194+0000 7feb959ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb90154a80 con 0x7feb901546a0 2023-11-06T20:57:11.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.195+0000 7feb959ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb900638c0 con 0x7feb90066560 2023-11-06T20:57:11.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.195+0000 7feb8effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb901546a0 0x7feb90064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:11.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.195+0000 7feb8f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7feb900bdbb0 0x7feb900c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49558/0 (socket says 172.21.15.33:49558) 2023-11-06T20:57:11.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.195+0000 7feb8f7fe700 1 -- 172.21.15.33:0/3629162719 learned_addr learned my addr 172.21.15.33:0/3629162719 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:57:11.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.195+0000 7feb8e7fc700 1 --2- 172.21.15.33:0/3629162719 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb90066560 0x7feb900669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:11.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.195+0000 7feb8effd700 1 -- 172.21.15.33:0/3629162719 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb90066560 msgr2=0x7feb900669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:11.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.195+0000 7feb8effd700 1 --2- 172.21.15.33:0/3629162719 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb90066560 0x7feb900669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.195+0000 7feb8effd700 1 -- 172.21.15.33:0/3629162719 >> v1:172.21.15.33:6789/0 conn(0x7feb900bdbb0 legacy=0x7feb900c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:11.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.195+0000 7feb8effd700 1 -- 172.21.15.33:0/3629162719 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90063ba0 con 0x7feb901546a0 2023-11-06T20:57:11.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.195+0000 7feb8dffb700 1 -- 172.21.15.33:0/3629162719 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274641608 0 0) 0x7feb900641f0 con 0x7feb900bdbb0 2023-11-06T20:57:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.196+0000 7feb8effd700 1 --2- 172.21.15.33:0/3629162719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb901546a0 0x7feb90064680 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7feb80009760 tx=0x7feb80009060).ready entity=mon.0 client_cookie=c55025300c35aac2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.196+0000 7feb8dffb700 1 -- 172.21.15.33:0/3629162719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8000f510 con 0x7feb901546a0 2023-11-06T20:57:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.196+0000 7feb8dffb700 1 -- 172.21.15.33:0/3629162719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8000f670 con 0x7feb901546a0 2023-11-06T20:57:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.196+0000 7feb8dffb700 1 -- 172.21.15.33:0/3629162719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8000f9f0 con 0x7feb901546a0 2023-11-06T20:57:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.196+0000 7feb959ff700 1 -- 172.21.15.33:0/3629162719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb901546a0 msgr2=0x7feb90064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:11.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.196+0000 7feb959ff700 1 --2- 172.21.15.33:0/3629162719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb901546a0 0x7feb90064680 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7feb80009760 tx=0x7feb80009060).stop 2023-11-06T20:57:11.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.196+0000 7feb959ff700 1 -- 172.21.15.33:0/3629162719 shutdown_connections 2023-11-06T20:57:11.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.196+0000 7feb959ff700 1 --2- 172.21.15.33:0/3629162719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb901546a0 0x7feb90064680 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.196+0000 7feb959ff700 1 --2- 172.21.15.33:0/3629162719 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb90066560 0x7feb900669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.197+0000 7feb959ff700 1 -- 172.21.15.33:0/3629162719 >> 172.21.15.33:0/3629162719 conn(0x7feb900c8c50 msgr2=0x7feb900c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:11.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.197+0000 7feb959ff700 1 -- 172.21.15.33:0/3629162719 shutdown_connections 2023-11-06T20:57:11.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.197+0000 7feb959ff700 1 -- 172.21.15.33:0/3629162719 wait complete. 2023-11-06T20:57:11.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.198+0000 7feb959ff700 1 Processor -- start 2023-11-06T20:57:11.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.198+0000 7feb959ff700 1 -- start start 2023-11-06T20:57:11.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.199+0000 7feb959ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb90066560 0x7feb90168460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.199+0000 7feb959ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb900bdbb0 0x7feb900c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.199+0000 7feb959ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb901546a0 0x7feb901523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.199+0000 7feb959ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9013f390 con 0x7feb900bdbb0 2023-11-06T20:57:11.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.199+0000 7feb959ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb90168950 con 0x7feb90066560 2023-11-06T20:57:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.199+0000 7feb959ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb90157a50 con 0x7feb901546a0 2023-11-06T20:57:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb900bdbb0 0x7feb900c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb900bdbb0 0x7feb900c2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51412/0 (socket says 172.21.15.33:51412) 2023-11-06T20:57:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8e7fc700 1 -- 172.21.15.33:0/3314330184 learned_addr learned my addr 172.21.15.33:0/3314330184 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8f7fe700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb901546a0 0x7feb901523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8effd700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb90066560 0x7feb90168460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8e7fc700 1 -- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb901546a0 msgr2=0x7feb901523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8e7fc700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb901546a0 0x7feb901523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8e7fc700 1 -- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb90066560 msgr2=0x7feb90168460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8e7fc700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb90066560 0x7feb90168460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8e7fc700 1 -- 172.21.15.33:0/3314330184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb800092a0 con 0x7feb900bdbb0 2023-11-06T20:57:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.200+0000 7feb8e7fc700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb900bdbb0 0x7feb900c2540 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7feb8400e930 tx=0x7feb84009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.201+0000 7feb76ffd700 1 -- 172.21.15.33:0/3314330184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84012600 con 0x7feb900bdbb0 2023-11-06T20:57:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.201+0000 7feb76ffd700 1 -- 172.21.15.33:0/3314330184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb84012760 con 0x7feb900bdbb0 2023-11-06T20:57:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.201+0000 7feb76ffd700 1 -- 172.21.15.33:0/3314330184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84012ae0 con 0x7feb900bdbb0 2023-11-06T20:57:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.201+0000 7feb959ff700 1 -- 172.21.15.33:0/3314330184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb901569b0 con 0x7feb900bdbb0 2023-11-06T20:57:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.201+0000 7feb959ff700 1 -- 172.21.15.33:0/3314330184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb90156f70 con 0x7feb900bdbb0 2023-11-06T20:57:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.202+0000 7feb959ff700 1 -- 172.21.15.33:0/3314330184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb90066000 con 0x7feb900bdbb0 2023-11-06T20:57:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.203+0000 7feb76ffd700 1 -- 172.21.15.33:0/3314330184 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) v1 ==== 75565+0+0 (secure 0 0 0) 0x7feb8401b400 con 0x7feb900bdbb0 2023-11-06T20:57:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.203+0000 7feb76ffd700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb7c05f2c0 0x7feb7c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.204+0000 7feb76ffd700 1 -- 172.21.15.33:0/3314330184 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feb8407a8d0 con 0x7feb900bdbb0 2023-11-06T20:57:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.204+0000 7feb8effd700 1 -- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb7c05f2c0 msgr2=0x7feb7c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:57:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.204+0000 7feb8effd700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb7c05f2c0 0x7feb7c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:57:11.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.208+0000 7feb76ffd700 1 -- 172.21.15.33:0/3314330184 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feb8404b900 con 0x7feb900bdbb0 2023-11-06T20:57:11.254 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:11.254+0000 7f4ebfc2c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:57:11.264 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:11.265+0000 7f285b126680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:57:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.393+0000 7feb959ff700 1 -- 172.21.15.33:0/3314330184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7feb90068690 con 0x7feb900bdbb0 2023-11-06T20:57:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.397+0000 7feb76ffd700 1 -- 172.21.15.33:0/3314330184 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v7)=0 v7) v1 ==== 138+0+0 (secure 0 0 0) 0x7feb8404ef20 con 0x7feb900bdbb0 2023-11-06T20:57:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.399+0000 7feb959ff700 1 -- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb7c05f2c0 msgr2=0x7feb7c061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:57:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.399+0000 7feb959ff700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb7c05f2c0 0x7feb7c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:57:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.399+0000 7feb959ff700 1 -- 172.21.15.33:0/3314330184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb900bdbb0 msgr2=0x7feb900c2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.400+0000 7feb959ff700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb900bdbb0 0x7feb900c2540 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7feb8400e930 tx=0x7feb84009060).stop 2023-11-06T20:57:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.400+0000 7feb959ff700 1 -- 172.21.15.33:0/3314330184 shutdown_connections 2023-11-06T20:57:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.400+0000 7feb959ff700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb7c05f2c0 0x7feb7c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:57:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.400+0000 7feb959ff700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb901546a0 0x7feb901523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.400+0000 7feb959ff700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb90066560 0x7feb90168460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.400+0000 7feb959ff700 1 --2- 172.21.15.33:0/3314330184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb900bdbb0 0x7feb900c2540 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.400+0000 7feb959ff700 1 -- 172.21.15.33:0/3314330184 >> 172.21.15.33:0/3314330184 conn(0x7feb900c8c50 msgr2=0x7feb900c3890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:11.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.400+0000 7feb959ff700 1 -- 172.21.15.33:0/3314330184 shutdown_connections 2023-11-06T20:57:11.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.400+0000 7feb959ff700 1 -- 172.21.15.33:0/3314330184 wait complete. 2023-11-06T20:57:11.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:11.418 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-06T20:57:11.479 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-11-06T20:57:11.584 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:11.585+0000 7f4ebfc2c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:57:11.631 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:11.631+0000 7f285b126680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:57:11.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.691+0000 7f2a2ddff700 1 Processor -- start 2023-11-06T20:57:11.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.691+0000 7f2a2ddff700 1 -- start start 2023-11-06T20:57:11.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.692+0000 7f2a2ddff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2813a960 0x7f2a28155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.692+0000 7f2a2ddff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a28066ba0 0x7f2a280c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.692+0000 7f2a2ddff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a28063950 con 0x7f2a280c9340 2023-11-06T20:57:11.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.692+0000 7f2a2ddff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a28064cc0 con 0x7f2a28066ba0 2023-11-06T20:57:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.692+0000 7f2a2ddff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a28155930 con 0x7f2a2813a960 2023-11-06T20:57:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.693+0000 7f2a27fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a28066ba0 0x7f2a280c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.693+0000 7f2a27fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a28066ba0 0x7f2a280c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51414/0 (socket says 172.21.15.33:51414) 2023-11-06T20:57:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.693+0000 7f2a27fff700 1 -- 172.21.15.33:0/976196165 learned_addr learned my addr 172.21.15.33:0/976196165 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.693+0000 7f2a277fe700 1 --2- 172.21.15.33:0/976196165 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2813a960 0x7f2a28155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:11.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.693+0000 7f2a27fff700 1 -- 172.21.15.33:0/976196165 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2813a960 msgr2=0x7f2a28155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.693+0000 7f2a27fff700 1 --2- 172.21.15.33:0/976196165 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2813a960 0x7f2a28155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.693+0000 7f2a27fff700 1 -- 172.21.15.33:0/976196165 >> v1:172.21.15.69:6789/0 conn(0x7f2a280c9340 legacy=0x7f2a28066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.693+0000 7f2a27fff700 1 -- 172.21.15.33:0/976196165 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a28063c30 con 0x7f2a28066ba0 2023-11-06T20:57:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.693+0000 7f2a267fc700 1 -- 172.21.15.33:0/976196165 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518203959 0 0) 0x7f2a28063950 con 0x7f2a280c9340 2023-11-06T20:57:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.693+0000 7f2a27fff700 1 --2- 172.21.15.33:0/976196165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a28066ba0 0x7f2a280c37b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f2a20009f20 tx=0x7f2a20009840).ready entity=mon.0 client_cookie=4bfc3f930b73ee08 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:11.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.694+0000 7f2a267fc700 1 -- 172.21.15.33:0/976196165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2000e070 con 0x7f2a28066ba0 2023-11-06T20:57:11.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.694+0000 7f2a267fc700 1 -- 172.21.15.33:0/976196165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2000b9d0 con 0x7f2a28066ba0 2023-11-06T20:57:11.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.694+0000 7f2a267fc700 1 -- 172.21.15.33:0/976196165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2000bd50 con 0x7f2a28066ba0 2023-11-06T20:57:11.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.694+0000 7f2a2ddff700 1 -- 172.21.15.33:0/976196165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a28066ba0 msgr2=0x7f2a280c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:11.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.694+0000 7f2a2ddff700 1 --2- 172.21.15.33:0/976196165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a28066ba0 0x7f2a280c37b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f2a20009f20 tx=0x7f2a20009840).stop 2023-11-06T20:57:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.694+0000 7f2a2ddff700 1 -- 172.21.15.33:0/976196165 shutdown_connections 2023-11-06T20:57:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.694+0000 7f2a2ddff700 1 --2- 172.21.15.33:0/976196165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a28066ba0 0x7f2a280c37b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.694+0000 7f2a2ddff700 1 --2- 172.21.15.33:0/976196165 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2813a960 0x7f2a28155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.694+0000 7f2a2ddff700 1 -- 172.21.15.33:0/976196165 >> 172.21.15.33:0/976196165 conn(0x7f2a2805ec70 msgr2=0x7f2a280c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.694+0000 7f2a2ddff700 1 -- 172.21.15.33:0/976196165 shutdown_connections 2023-11-06T20:57:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.695+0000 7f2a2ddff700 1 -- 172.21.15.33:0/976196165 wait complete. 2023-11-06T20:57:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.696+0000 7f2a2ddff700 1 Processor -- start 2023-11-06T20:57:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.696+0000 7f2a2ddff700 1 -- start start 2023-11-06T20:57:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a2ddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a28066ba0 0x7f2a2814bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a2ddff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a280c9340 0x7f2a280a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a2ddff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2813a960 0x7f2a280add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a2ddff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2814c0d0 con 0x7f2a280c9340 2023-11-06T20:57:11.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a2ddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280a7060 con 0x7f2a28066ba0 2023-11-06T20:57:11.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a2ddff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280a7340 con 0x7f2a2813a960 2023-11-06T20:57:11.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a26ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a280c9340 0x7f2a280a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a26ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a280c9340 0x7f2a280a6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51424/0 (socket says 172.21.15.33:51424) 2023-11-06T20:57:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a26ffd700 1 -- 172.21.15.33:0/3184761987 learned_addr learned my addr 172.21.15.33:0/3184761987 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a277fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a28066ba0 0x7f2a2814bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.697+0000 7f2a27fff700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2813a960 0x7f2a280add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a26ffd700 1 -- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2813a960 msgr2=0x7f2a280add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a26ffd700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2813a960 0x7f2a280add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a26ffd700 1 -- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a28066ba0 msgr2=0x7f2a2814bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a26ffd700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a28066ba0 0x7f2a2814bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a26ffd700 1 -- 172.21.15.33:0/3184761987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a20009ae0 con 0x7f2a280c9340 2023-11-06T20:57:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a26ffd700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a280c9340 0x7f2a280a6680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f2a1c001eb0 tx=0x7f2a1c007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00ce30 con 0x7f2a280c9340 2023-11-06T20:57:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c00f470 con 0x7f2a280c9340 2023-11-06T20:57:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00f830 con 0x7f2a280c9340 2023-11-06T20:57:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a2ddff700 1 -- 172.21.15.33:0/3184761987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a280a7620 con 0x7f2a280c9340 2023-11-06T20:57:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.698+0000 7f2a2ddff700 1 -- 172.21.15.33:0/3184761987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a280a7be0 con 0x7f2a280c9340 2023-11-06T20:57:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.700+0000 7f2a2ddff700 1 -- 172.21.15.33:0/3184761987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a280c2ae0 con 0x7f2a280c9340 2023-11-06T20:57:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.700+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f2a1c00fba0 con 0x7f2a280c9340 2023-11-06T20:57:11.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.701+0000 7f2a0f7fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:11.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.701+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2a1c0799c0 con 0x7f2a280c9340 2023-11-06T20:57:11.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.701+0000 7f2a277fe700 1 -- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 msgr2=0x7f2a14061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:57:11.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.701+0000 7f2a277fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:57:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.705+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2a1c04aa50 con 0x7f2a280c9340 2023-11-06T20:57:11.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.902+0000 7f2a277fe700 1 -- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 msgr2=0x7f2a14061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:57:11.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.902+0000 7f2a277fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:57:11.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:11.917+0000 7f2a2ddff700 1 -- 172.21.15.33:0/3184761987 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a28065f30 con 0x7f2a1405f2c0 2023-11-06T20:57:12.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:12.303+0000 7f2a277fe700 1 -- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 msgr2=0x7f2a14061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:57:12.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:12.303+0000 7f2a277fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T20:57:12.337 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:12.338+0000 7f4ebfc2c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:57:12.418 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:12.419+0000 7f285b126680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:57:12.421 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:12.421+0000 7f4ebfc2c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:57:12.506 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:12.506+0000 7f285b126680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:57:12.652 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:12.652+0000 7f4ebfc2c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:57:12.745 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:12.746+0000 7f285b126680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:57:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:13.104+0000 7f2a277fe700 1 -- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 msgr2=0x7f2a14061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:57:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:13.104+0000 7f2a277fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T20:57:13.162 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:13.162+0000 7f4ebfc2c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:57:13.277 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:13.278+0000 7f285b126680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:57:13.286 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:13.287+0000 7f4ebfc2c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:57:13.407 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:13.408+0000 7f285b126680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:57:13.470 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:13.470+0000 7f4ebfc2c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:57:13.566 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:13.566+0000 7f4ebfc2c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:57:13.597 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:13.597+0000 7f285b126680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:57:13.651 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:13.651+0000 7f4ebfc2c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:57:13.694 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:13.695+0000 7f285b126680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:57:13.782 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:13.783+0000 7f285b126680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:57:13.961 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:13.961+0000 7f4ebfc2c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:57:14.107 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:14.108+0000 7f285b126680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:57:14.184 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:14.185+0000 7f4ebfc2c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:57:14.336 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:14.336+0000 7f285b126680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:57:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:14.706+0000 7f2a277fe700 1 -- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 msgr2=0x7f2a14061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:57:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:14.706+0000 7f2a277fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T20:57:14.985 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:14.985+0000 7f4ebfc2c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:57:15.072 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:15.073+0000 7f4ebfc2c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:57:15.166 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:15.166+0000 7f4ebfc2c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:57:15.175 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:15.176+0000 7f285b126680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:57:15.266 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:15.266+0000 7f285b126680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:57:15.350 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:15.350+0000 7f4ebfc2c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:57:15.363 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:15.364+0000 7f285b126680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:57:15.439 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:15.439+0000 7f4ebfc2c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:57:15.555 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:15.556+0000 7f285b126680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:57:15.580 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:15.580+0000 7f4ebfc2c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:57:15.647 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:15.647+0000 7f285b126680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:57:15.710 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:15.710+0000 7f4ebfc2c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:57:15.793 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:15.794+0000 7f285b126680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:57:15.929 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:15.930+0000 7f285b126680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:57:16.056 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.057+0000 7f4ebfc2c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:57:16.145 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.145+0000 7f4ebfc2c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:57:16.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:16.160+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 16) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f2a1c077550 con 0x7f2a280c9340 2023-11-06T20:57:16.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:16.161+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 msgr2=0x7f2a14061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:57:16.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:16.161+0000 7f2a0f7fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:57:16.234 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.234+0000 7f4e34f46700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:57:16.234 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.234+0000 7f4e34f46700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:57:16.234 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.234+0000 7f4e34f46700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:57:16.235 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.234+0000 7f4e34f46700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:57:16.235 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.234+0000 7f4e34f46700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:57:16.235 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.234+0000 7f4e31f40700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:57:16.235 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.234+0000 7f4e31f40700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:57:16.235 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.234+0000 7f4e31f40700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:57:16.235 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.234+0000 7f4e31f40700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:57:16.236 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:57:16.234+0000 7f4e31f40700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:57:16.290 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:16.291+0000 7f285b126680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:57:16.382 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:57:16.383+0000 7f285b126680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:57:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.163+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 17) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f2a1c0778d0 con 0x7f2a280c9340 2023-11-06T20:57:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.163+0000 7f2a0f7fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.163+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a28065f30 con 0x7f2a1405f2c0 2023-11-06T20:57:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.164+0000 7f2a277fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.166+0000 7f2a277fe700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f2a180094c0 tx=0x7f2a18009060).ready entity=mgr.5794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:17.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.177+0000 7f2a0f7fe700 1 -- 172.21.15.33:0/3184761987 <== mgr.5794 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2a28065f30 con 0x7f2a1405f2c0 2023-11-06T20:57:17.176 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path snap_test_dir1 2023-11-06T20:57:17.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.179+0000 7f2a2ddff700 1 -- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 msgr2=0x7f2a14061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.179+0000 7f2a2ddff700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f2a180094c0 tx=0x7f2a18009060).stop 2023-11-06T20:57:17.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.179+0000 7f2a2ddff700 1 -- 172.21.15.33:0/3184761987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a280c9340 msgr2=0x7f2a280a6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.179+0000 7f2a2ddff700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a280c9340 0x7f2a280a6680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f2a1c001eb0 tx=0x7f2a1c007c40).stop 2023-11-06T20:57:17.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.179+0000 7f2a2ddff700 1 -- 172.21.15.33:0/3184761987 shutdown_connections 2023-11-06T20:57:17.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.179+0000 7f2a2ddff700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a1405f2c0 0x7f2a14061700 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.179+0000 7f2a2ddff700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a2813a960 0x7f2a280add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.179+0000 7f2a2ddff700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a28066ba0 0x7f2a2814bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.179+0000 7f2a2ddff700 1 --2- 172.21.15.33:0/3184761987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a280c9340 0x7f2a280a6680 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.180+0000 7f2a2ddff700 1 -- 172.21.15.33:0/3184761987 >> 172.21.15.33:0/3184761987 conn(0x7f2a2805ec70 msgr2=0x7f2a281546e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.180+0000 7f2a2ddff700 1 -- 172.21.15.33:0/3184761987 shutdown_connections 2023-11-06T20:57:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.180+0000 7f2a2ddff700 1 -- 172.21.15.33:0/3184761987 wait complete. 2023-11-06T20:57:17.200 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-06T20:57:17.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2023-11-06T20:57:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.411+0000 7f77955ff700 1 Processor -- start 2023-11-06T20:57:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.412+0000 7f77955ff700 1 -- start start 2023-11-06T20:57:17.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.413+0000 7f77955ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f77900c9340 0x7f7790155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.413+0000 7f77955ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7790063950 con 0x7f779013a960 2023-11-06T20:57:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.413+0000 7f77955ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7790063c30 con 0x7f7790066ba0 2023-11-06T20:57:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.413+0000 7f77955ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7790064cc0 con 0x7f77900c9340 2023-11-06T20:57:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.413+0000 7f778effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f779013a960 0x7f7790066490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49568/0 (socket says 172.21.15.33:49568) 2023-11-06T20:57:17.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.413+0000 7f778effd700 1 -- 172.21.15.33:0/2661649483 learned_addr learned my addr 172.21.15.33:0/2661649483 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:57:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778f7fe700 1 --2- 172.21.15.33:0/2661649483 >> v2:172.21.15.69:3300/0 conn(0x7f77900c9340 0x7f7790155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107892526 0 0) 0x7f7790063950 con 0x7f779013a960 2023-11-06T20:57:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77780033f0 con 0x7f779013a960 2023-11-06T20:57:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572827511 0 0) 0x7f77780033f0 con 0x7f779013a960 2023-11-06T20:57:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 >> v2:172.21.15.69:3300/0 conn(0x7f77900c9340 msgr2=0x7f7790155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 --2- 172.21.15.33:0/2661649483 >> v2:172.21.15.69:3300/0 conn(0x7f77900c9340 0x7f7790155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 >> v1:172.21.15.69:6789/0 conn(0x7f7790066ba0 legacy=0x7f77900c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779013ad40 con 0x7f779013a960 2023-11-06T20:57:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f7780001470 con 0x7f779013a960 2023-11-06T20:57:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 >> v1:172.21.15.33:6789/0 conn(0x7f779013a960 legacy=0x7f7790066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f7778003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7778004690 0x7f7778004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7778005000 0x7f777800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7790064cc0 con 0x7f77780038e0 2023-11-06T20:57:17.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777800bf60 con 0x7f7778005000 2023-11-06T20:57:17.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.414+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777800c0f0 con 0x7f7778004690 2023-11-06T20:57:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778effd700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f7778003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778f7fe700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7778005000 0x7f777800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778e7fc700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7778004690 0x7f7778004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778effd700 1 -- 172.21.15.33:0/2661649483 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7778004690 msgr2=0x7f7778004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:17.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778effd700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7778004690 0x7f7778004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778effd700 1 -- 172.21.15.33:0/2661649483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7778005000 msgr2=0x7f777800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:17.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778effd700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7778005000 0x7f777800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778effd700 1 -- 172.21.15.33:0/2661649483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77780033f0 con 0x7f77780038e0 2023-11-06T20:57:17.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778f7fe700 1 -- 172.21.15.33:0/2661649483 reap_dead start 2023-11-06T20:57:17.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778effd700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f7778003d00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f778000c190 tx=0x7f7780001e50).ready entity=mon.0 client_cookie=c3fb3e3e377d1170 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:17.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780007a20 con 0x7f77780038e0 2023-11-06T20:57:17.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778000ca40 con 0x7f77780038e0 2023-11-06T20:57:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.415+0000 7f778dffb700 1 -- 172.21.15.33:0/2661649483 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778000cdc0 con 0x7f77780038e0 2023-11-06T20:57:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.416+0000 7f77955ff700 1 -- 172.21.15.33:0/2661649483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 msgr2=0x7f7778003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.416+0000 7f77955ff700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f7778003d00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f778000c190 tx=0x7f7780001e50).stop 2023-11-06T20:57:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.416+0000 7f77955ff700 1 -- 172.21.15.33:0/2661649483 shutdown_connections 2023-11-06T20:57:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.416+0000 7f77955ff700 1 --2- 172.21.15.33:0/2661649483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f7778003d00 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.416+0000 7f77955ff700 1 -- 172.21.15.33:0/2661649483 >> 172.21.15.33:0/2661649483 conn(0x7f779005ec70 msgr2=0x7f77900c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.416+0000 7f77955ff700 1 -- 172.21.15.33:0/2661649483 shutdown_connections 2023-11-06T20:57:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.416+0000 7f77955ff700 1 -- 172.21.15.33:0/2661649483 wait complete. 2023-11-06T20:57:17.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.418+0000 7f77955ff700 1 Processor -- start 2023-11-06T20:57:17.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.418+0000 7f77955ff700 1 -- start start 2023-11-06T20:57:17.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f77955ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f77900a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f77955ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77900ac850 0x7f7790153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f77955ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77900a48e0 0x7f77900adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f77955ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7790155930 con 0x7f77780038e0 2023-11-06T20:57:17.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f77955ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77900a9760 con 0x7f77900a48e0 2023-11-06T20:57:17.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f77955ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77900ada10 con 0x7f77900ac850 2023-11-06T20:57:17.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f77900a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f77900a90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51436/0 (socket says 172.21.15.33:51436) 2023-11-06T20:57:17.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778effd700 1 -- 172.21.15.33:0/1420019217 learned_addr learned my addr 172.21.15.33:0/1420019217 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:17.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778effd700 1 -- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77900ac850 msgr2=0x7f7790153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778e7fc700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77900ac850 0x7f7790153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778f7fe700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77900a48e0 0x7f77900adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778effd700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77900ac850 0x7f7790153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778effd700 1 -- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77900a48e0 msgr2=0x7f77900adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778effd700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77900a48e0 0x7f77900adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778effd700 1 -- 172.21.15.33:0/1420019217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7790154150 con 0x7f77780038e0 2023-11-06T20:57:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.419+0000 7f778f7fe700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77900a48e0 0x7f77900adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:57:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.420+0000 7f778effd700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f77900a90d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f7780001940 tx=0x7f7780015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.420+0000 7f7776ffd700 1 -- 172.21.15.33:0/1420019217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77800073e0 con 0x7f77780038e0 2023-11-06T20:57:17.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.420+0000 7f7776ffd700 1 -- 172.21.15.33:0/1420019217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7780007540 con 0x7f77780038e0 2023-11-06T20:57:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.420+0000 7f7776ffd700 1 -- 172.21.15.33:0/1420019217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77800048e0 con 0x7f77780038e0 2023-11-06T20:57:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.420+0000 7f77955ff700 1 -- 172.21.15.33:0/1420019217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77900ad0e0 con 0x7f77780038e0 2023-11-06T20:57:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.420+0000 7f77955ff700 1 -- 172.21.15.33:0/1420019217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77900ad6a0 con 0x7f77780038e0 2023-11-06T20:57:17.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.421+0000 7f77955ff700 1 -- 172.21.15.33:0/1420019217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7790065f30 con 0x7f77780038e0 2023-11-06T20:57:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.422+0000 7f7776ffd700 1 -- 172.21.15.33:0/1420019217 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f778001c070 con 0x7f77780038e0 2023-11-06T20:57:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.423+0000 7f7776ffd700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f777c05f080 0x7f777c061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.423+0000 7f7776ffd700 1 -- 172.21.15.33:0/1420019217 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f778007a890 con 0x7f77780038e0 2023-11-06T20:57:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.424+0000 7f778e7fc700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f777c05f080 0x7f777c061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.424+0000 7f778e7fc700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f777c05f080 0x7f777c061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7784000c00 tx=0x7f7784009040).ready entity=mgr.5794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:17.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.428+0000 7f7776ffd700 1 -- 172.21.15.33:0/1420019217 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f778004bb20 con 0x7f77780038e0 2023-11-06T20:57:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.633+0000 7f77955ff700 1 -- 172.21.15.33:0/1420019217 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f77900634c0 con 0x7f777c05f080 2023-11-06T20:57:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.639+0000 7f7776ffd700 1 -- 172.21.15.33:0/1420019217 <== mgr.5794 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f77900634c0 con 0x7f777c05f080 2023-11-06T20:57:17.638 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path snap_test_dir1 2023-11-06T20:57:17.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 -- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f777c05f080 msgr2=0x7f777c061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f777c05f080 0x7f777c061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7784000c00 tx=0x7f7784009040).stop 2023-11-06T20:57:17.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 -- 172.21.15.33:0/1420019217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 msgr2=0x7f77900a90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f77900a90d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f7780001940 tx=0x7f7780015000).stop 2023-11-06T20:57:17.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 -- 172.21.15.33:0/1420019217 shutdown_connections 2023-11-06T20:57:17.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f777c05f080 0x7f777c061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77900a48e0 0x7f77900adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77780038e0 0x7f77900a90d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 --2- 172.21.15.33:0/1420019217 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77900ac850 0x7f7790153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 -- 172.21.15.33:0/1420019217 >> 172.21.15.33:0/1420019217 conn(0x7f779005ec70 msgr2=0x7f77900c7260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 -- 172.21.15.33:0/1420019217 shutdown_connections 2023-11-06T20:57:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.641+0000 7f77955ff700 1 -- 172.21.15.33:0/1420019217 wait complete. 2023-11-06T20:57:17.659 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-06T20:57:17.659 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-11-06-20_58_00 in ~43s... 2023-11-06T20:57:17.660 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-11-06-20_59_00 in ~103s... 2023-11-06T20:57:17.660 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-11-06T20:57:17.660 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-11-06T20:57:17.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.864+0000 7ff5f79ff700 1 Processor -- start 2023-11-06T20:57:17.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.865+0000 7ff5f79ff700 1 -- start start 2023-11-06T20:57:17.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.866+0000 7ff5f79ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5f00c8ee0 0x7ff5f00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.866+0000 7ff5f79ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff5f01542b0 0x7ff5f00c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.866+0000 7ff5f79ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f00c9f80 con 0x7ff5f0153ed0 2023-11-06T20:57:17.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.866+0000 7ff5f79ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f00c6000 con 0x7ff5f00c8ee0 2023-11-06T20:57:17.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.866+0000 7ff5f79ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f00c9370 con 0x7ff5f01542b0 2023-11-06T20:57:17.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.866+0000 7ff5f4f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5f00c8ee0 0x7ff5f00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.866+0000 7ff5f579b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff5f0153ed0 0x7ff5f0158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49576/0 (socket says 172.21.15.33:49576) 2023-11-06T20:57:17.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.866+0000 7ff5f579b700 1 -- 172.21.15.33:0/1315244186 learned_addr learned my addr 172.21.15.33:0/1315244186 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:57:17.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.867+0000 7ff5f5f9c700 1 --2- 172.21.15.33:0/1315244186 >> v2:172.21.15.69:3300/0 conn(0x7ff5f01542b0 0x7ff5f00c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.867+0000 7ff5f4f9a700 1 -- 172.21.15.33:0/1315244186 >> v2:172.21.15.69:3300/0 conn(0x7ff5f01542b0 msgr2=0x7ff5f00c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:57:17.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.867+0000 7ff5f4f9a700 1 --2- 172.21.15.33:0/1315244186 >> v2:172.21.15.69:3300/0 conn(0x7ff5f01542b0 0x7ff5f00c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.867+0000 7ff5f4f9a700 1 -- 172.21.15.33:0/1315244186 >> v1:172.21.15.33:6789/0 conn(0x7ff5f0153ed0 legacy=0x7ff5f0158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.867+0000 7ff5f4f9a700 1 -- 172.21.15.33:0/1315244186 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f00c9650 con 0x7ff5f00c8ee0 2023-11-06T20:57:17.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.867+0000 7ff5e7fff700 1 -- 172.21.15.33:0/1315244186 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34920311 0 0) 0x7ff5f00c9f80 con 0x7ff5f0153ed0 2023-11-06T20:57:17.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.867+0000 7ff5f4f9a700 1 --2- 172.21.15.33:0/1315244186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f00c8ee0 0x7ff5f00ca410 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7ff5f00c85d0 tx=0x7ff5e0009060).ready entity=mon.0 client_cookie=fe16dcef25afcc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:17.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.867+0000 7ff5e7fff700 1 -- 172.21.15.33:0/1315244186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000b580 con 0x7ff5f00c8ee0 2023-11-06T20:57:17.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.867+0000 7ff5e7fff700 1 -- 172.21.15.33:0/1315244186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e000b6e0 con 0x7ff5f00c8ee0 2023-11-06T20:57:17.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.867+0000 7ff5e7fff700 1 -- 172.21.15.33:0/1315244186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000ba60 con 0x7ff5f00c8ee0 2023-11-06T20:57:17.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.868+0000 7ff5f79ff700 1 -- 172.21.15.33:0/1315244186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f00c8ee0 msgr2=0x7ff5f00ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.868+0000 7ff5f79ff700 1 --2- 172.21.15.33:0/1315244186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f00c8ee0 0x7ff5f00ca410 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7ff5f00c85d0 tx=0x7ff5e0009060).stop 2023-11-06T20:57:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.868+0000 7ff5f79ff700 1 -- 172.21.15.33:0/1315244186 shutdown_connections 2023-11-06T20:57:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.868+0000 7ff5f79ff700 1 --2- 172.21.15.33:0/1315244186 >> v2:172.21.15.69:3300/0 conn(0x7ff5f01542b0 0x7ff5f00c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.868+0000 7ff5f79ff700 1 --2- 172.21.15.33:0/1315244186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f00c8ee0 0x7ff5f00ca410 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.868+0000 7ff5f79ff700 1 -- 172.21.15.33:0/1315244186 >> 172.21.15.33:0/1315244186 conn(0x7ff5f005e1b0 msgr2=0x7ff5f005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.868+0000 7ff5f79ff700 1 -- 172.21.15.33:0/1315244186 shutdown_connections 2023-11-06T20:57:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.868+0000 7ff5f79ff700 1 -- 172.21.15.33:0/1315244186 wait complete. 2023-11-06T20:57:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.869+0000 7ff5f79ff700 1 Processor -- start 2023-11-06T20:57:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.870+0000 7ff5f79ff700 1 -- start start 2023-11-06T20:57:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f79ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5f0153ed0 0x7ff5f01581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f79ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f01542b0 0x7ff5f00b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f79ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5f0142730 0x7ff5f00ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f79ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f014fef0 con 0x7ff5f01542b0 2023-11-06T20:57:17.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f79ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f01643a0 con 0x7ff5f0153ed0 2023-11-06T20:57:17.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f79ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f014f850 con 0x7ff5f0142730 2023-11-06T20:57:17.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f4f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f01542b0 0x7ff5f00b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f4f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f01542b0 0x7ff5f00b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51452/0 (socket says 172.21.15.33:51452) 2023-11-06T20:57:17.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f4f9a700 1 -- 172.21.15.33:0/2993683001 learned_addr learned my addr 172.21.15.33:0/2993683001 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:57:17.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f579b700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5f0153ed0 0x7ff5f01581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f5f9c700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5f0142730 0x7ff5f00ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f4f9a700 1 -- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5f0142730 msgr2=0x7ff5f00ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f4f9a700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5f0142730 0x7ff5f00ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f4f9a700 1 -- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5f0153ed0 msgr2=0x7ff5f01581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:17.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f4f9a700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5f0153ed0 0x7ff5f01581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:17.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f4f9a700 1 -- 172.21.15.33:0/2993683001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e00092a0 con 0x7ff5f01542b0 2023-11-06T20:57:17.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.871+0000 7ff5f5f9c700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5f0142730 0x7ff5f00ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:57:17.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.872+0000 7ff5f4f9a700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f01542b0 0x7ff5f00b0250 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7ff5e0009fd0 tx=0x7ff5e0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.872+0000 7ff5e57fa700 1 -- 172.21.15.33:0/2993683001 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000b850 con 0x7ff5f01542b0 2023-11-06T20:57:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.872+0000 7ff5e57fa700 1 -- 172.21.15.33:0/2993683001 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e000b9b0 con 0x7ff5f01542b0 2023-11-06T20:57:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.872+0000 7ff5e57fa700 1 -- 172.21.15.33:0/2993683001 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e001b660 con 0x7ff5f01542b0 2023-11-06T20:57:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.872+0000 7ff5f79ff700 1 -- 172.21.15.33:0/2993683001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f00b0a40 con 0x7ff5f01542b0 2023-11-06T20:57:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.872+0000 7ff5f79ff700 1 -- 172.21.15.33:0/2993683001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f014fb10 con 0x7ff5f01542b0 2023-11-06T20:57:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.873+0000 7ff5f79ff700 1 -- 172.21.15.33:0/2993683001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f0150030 con 0x7ff5f01542b0 2023-11-06T20:57:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.874+0000 7ff5e57fa700 1 -- 172.21.15.33:0/2993683001 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 75357+0+0 (secure 0 0 0) 0x7ff5e001ba90 con 0x7ff5f01542b0 2023-11-06T20:57:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.875+0000 7ff5e57fa700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff5d405f080 0x7ff5d4061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:57:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.875+0000 7ff5e57fa700 1 -- 172.21.15.33:0/2993683001 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6379+0+0 (secure 0 0 0) 0x7ff5e007bd50 con 0x7ff5f01542b0 2023-11-06T20:57:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.875+0000 7ff5f579b700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff5d405f080 0x7ff5d4061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:57:17.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.876+0000 7ff5f579b700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff5d405f080 0x7ff5d4061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7ff5ec0011e0 tx=0x7ff5ec007bb0).ready entity=mgr.5794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:57:17.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:17.879+0000 7ff5e57fa700 1 -- 172.21.15.33:0/2993683001 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff5f0150030 con 0x7ff5f01542b0 2023-11-06T20:57:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.083+0000 7ff5f79ff700 1 -- 172.21.15.33:0/2993683001 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5f0150030 con 0x7ff5d405f080 2023-11-06T20:57:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.086+0000 7ff5e57fa700 1 -- 172.21.15.33:0/2993683001 <== mgr.5794 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7ff5f0150030 con 0x7ff5d405f080 2023-11-06T20:57:18.086 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:57:18.086 INFO:teuthology.orchestra.run.smithi033.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-11-06T20:57:18.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 -- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff5d405f080 msgr2=0x7ff5d4061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff5d405f080 0x7ff5d4061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7ff5ec0011e0 tx=0x7ff5ec007bb0).stop 2023-11-06T20:57:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 -- 172.21.15.33:0/2993683001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f01542b0 msgr2=0x7ff5f00b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:57:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f01542b0 0x7ff5f00b0250 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7ff5e0009fd0 tx=0x7ff5e0017000).stop 2023-11-06T20:57:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 -- 172.21.15.33:0/2993683001 shutdown_connections 2023-11-06T20:57:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff5d405f080 0x7ff5d4061540 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5f0142730 0x7ff5f00ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5f0153ed0 0x7ff5f01581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 --2- 172.21.15.33:0/2993683001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5f01542b0 0x7ff5f00b0250 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:57:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 -- 172.21.15.33:0/2993683001 >> 172.21.15.33:0/2993683001 conn(0x7ff5f005e1b0 msgr2=0x7ff5f0152ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:57:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 -- 172.21.15.33:0/2993683001 shutdown_connections 2023-11-06T20:57:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:57:18.088+0000 7ff5f79ff700 1 -- 172.21.15.33:0/2993683001 wait complete. 2023-11-06T20:57:18.106 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-11-06T20:57:18.106 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-11-06T20:57:18.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:18.107 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:19.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:19.158 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:20.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:20.215 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:21.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:21.271 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:22.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:22.327 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:23.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:23.384 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:24.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:24.441 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:25.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:25.498 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:26.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:26.555 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:27.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:27.611 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:28.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:28.668 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:29.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:29.725 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:30.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:30.781 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:31.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:31.837 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:32.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:32.893 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:33.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:33.949 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:35.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:35.007 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:36.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:36.063 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:37.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:37.120 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:37.809 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:57:37.814 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:57:38.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:38.176 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:39.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:39.232 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:40.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:40.289 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:41.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:41.345 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:42.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:42.401 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:43.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:43.457 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:44.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:44.514 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:45.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:45.571 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:46.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:46.628 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:47.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:47.685 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:48.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:48.742 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:49.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:49.799 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:50.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:50.855 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:51.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:51.912 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:52.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:52.969 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:54.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:54.026 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:55.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:55.082 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:56.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:56.140 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:57.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:57.197 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:58.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:58.255 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:57:59.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:57:59.312 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:00.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:00.368 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:00.645 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:58:00.645+0000 7f4e246e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:58:00.646 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:58:00.645+0000 7f4e246e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:58:00.646 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:58:00.645+0000 7f4e246e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:58:00.646 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:58:00.645+0000 7f4e246e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:58:00.646 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:58:00.645+0000 7f4e246e5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:58:01.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:01.425 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:01.479 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:01.481 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-11-06-20_58_00_UTC'] 2023-11-06T20:58:01.482 DEBUG:tasks.cephfs.test_snap_schedules:exec=1699304237.6597662, now = 1699304281.48209, timo = 43 2023-11-06T20:58:02.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:02.483 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:02.537 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:03.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:03.540 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:03.594 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:04.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:04.596 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:04.651 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:05.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:05.653 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:05.707 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:06.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:06.709 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:06.763 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:07.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:07.765 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:07.818 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:08.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:58:08.177 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:58:08.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:08.821 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:08.875 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:09.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:09.878 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:09.932 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:10.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:10.934 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:10.988 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:11.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:11.991 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:12.046 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:13.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:13.048 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:13.104 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:14.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:14.108 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:14.163 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:15.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:15.165 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:15.219 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:16.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:16.221 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:16.281 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:17.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:17.283 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:17.337 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:18.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:18.339 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:18.393 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:19.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:19.396 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:19.450 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:20.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:20.453 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:20.507 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:21.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:21.509 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:21.564 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:22.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:22.567 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:22.620 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:23.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:23.622 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:23.676 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:24.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:24.679 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:24.733 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:25.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:25.736 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:25.790 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:26.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:26.792 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:26.846 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:27.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:27.849 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:27.906 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:28.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:28.908 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:28.962 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:29.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:29.965 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:30.019 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:31.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:31.022 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:31.076 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:32.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:32.078 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:32.131 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:33.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:33.134 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:33.190 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:34.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:34.193 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:34.246 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:35.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:35.249 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:35.304 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:36.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:36.307 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:37.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:37.363 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:37.418 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:38.301 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:58:38.310 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:58:38.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:38.420 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:39.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:39.475 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:39.530 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:40.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:40.532 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:40.586 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:41.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:41.589 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:41.643 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:42.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:42.646 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:42.700 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:43.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:43.703 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:43.757 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:44.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:44.760 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:44.814 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:45.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:45.817 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:45.870 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:46.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:46.873 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:46.927 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:47.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:47.931 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:47.986 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:48.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:48.988 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:49.047 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:50.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:50.050 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:50.104 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:51.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:51.107 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:51.161 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:52.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:52.163 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:53.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:53.220 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:53.276 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:54.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:54.278 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:54.333 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:55.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:55.336 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:55.390 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:56.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:56.393 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:56.448 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:57.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:57.451 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:57.505 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:58.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:58.507 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:58.562 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:58:59.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:58:59.565 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:58:59.619 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:59:00.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:59:00.621 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:59:00.675 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:59:00.676 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_59_00_UTC 2023-11-06T20:59:00.677 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-11-06-20_59_00_UTC'] 2023-11-06T20:59:00.677 DEBUG:tasks.cephfs.test_snap_schedules:exec=1699304237.6597662, now = 1699304340.6774907, timo = 103 2023-11-06T20:59:01.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-11-06T20:59:01.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.892+0000 7fb1ac9ff700 1 Processor -- start 2023-11-06T20:59:01.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.892+0000 7fb1ac9ff700 1 -- start start 2023-11-06T20:59:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.893+0000 7fb1ac9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a8153ed0 0x7fb1a80aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.893+0000 7fb1ac9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb1a80638d0 0x7fb1a8157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.894+0000 7fb1ac9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a80c02c0 con 0x7fb1a81542b0 2023-11-06T20:59:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.894+0000 7fb1ac9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a81581e0 con 0x7fb1a80638d0 2023-11-06T20:59:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.894+0000 7fb1ac9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a8158460 con 0x7fb1a8153ed0 2023-11-06T20:59:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.894+0000 7fb1a6d9d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb1a80638d0 0x7fb1a8157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.894+0000 7fb1a659c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a8153ed0 0x7fb1a80aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.894+0000 7fb1a6d9d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb1a80638d0 0x7fb1a8157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52002/0 (socket says 172.21.15.33:52002) 2023-11-06T20:59:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.894+0000 7fb1a6d9d700 1 -- 172.21.15.33:0/1943938114 learned_addr learned my addr 172.21.15.33:0/1943938114 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.894+0000 7fb1a5d9b700 1 --1- 172.21.15.33:0/1943938114 >> v1:172.21.15.69:6789/0 conn(0x7fb1a81542b0 0x7fb1a80631c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:48698/0 (socket says 172.21.15.33:48698) 2023-11-06T20:59:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.894+0000 7fb1a6d9d700 1 -- 172.21.15.33:0/1943938114 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a8153ed0 msgr2=0x7fb1a80aa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.894+0000 7fb1a6d9d700 1 --2- 172.21.15.33:0/1943938114 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a8153ed0 0x7fb1a80aa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.895+0000 7fb1a6d9d700 1 -- 172.21.15.33:0/1943938114 >> v1:172.21.15.69:6789/0 conn(0x7fb1a81542b0 legacy=0x7fb1a80631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.895+0000 7fb1a6d9d700 1 -- 172.21.15.33:0/1943938114 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8158740 con 0x7fb1a80638d0 2023-11-06T20:59:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.895+0000 7fb1a559a700 1 -- 172.21.15.33:0/1943938114 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054584133 0 0) 0x7fb1a80c02c0 con 0x7fb1a81542b0 2023-11-06T20:59:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.895+0000 7fb1a6d9d700 1 --2- 172.21.15.33:0/1943938114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a80638d0 0x7fb1a8157cf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fb1a0009f20 tx=0x7fb1a0009840).ready entity=mon.1 client_cookie=9c592acc52af7088 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:01.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.895+0000 7fb1a559a700 1 -- 172.21.15.33:0/1943938114 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000e070 con 0x7fb1a80638d0 2023-11-06T20:59:01.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.895+0000 7fb1a559a700 1 -- 172.21.15.33:0/1943938114 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a000b9d0 con 0x7fb1a80638d0 2023-11-06T20:59:01.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.895+0000 7fb1a559a700 1 -- 172.21.15.33:0/1943938114 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000bd50 con 0x7fb1a80638d0 2023-11-06T20:59:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.896+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1943938114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a80638d0 msgr2=0x7fb1a8157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.896+0000 7fb1ac9ff700 1 --2- 172.21.15.33:0/1943938114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a80638d0 0x7fb1a8157cf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fb1a0009f20 tx=0x7fb1a0009840).stop 2023-11-06T20:59:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.896+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1943938114 shutdown_connections 2023-11-06T20:59:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.896+0000 7fb1ac9ff700 1 --2- 172.21.15.33:0/1943938114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a80638d0 0x7fb1a8157cf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.896+0000 7fb1ac9ff700 1 --2- 172.21.15.33:0/1943938114 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a8153ed0 0x7fb1a80aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:01.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.896+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1943938114 >> 172.21.15.33:0/1943938114 conn(0x7fb1a805ec80 msgr2=0x7fb1a80cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:01.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.896+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1943938114 shutdown_connections 2023-11-06T20:59:01.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.896+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1943938114 wait complete. 2023-11-06T20:59:01.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.897+0000 7fb1ac9ff700 1 Processor -- start 2023-11-06T20:59:01.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.897+0000 7fb1ac9ff700 1 -- start start 2023-11-06T20:59:01.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.898+0000 7fb1ac9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a80638d0 0x7fb1a8146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:01.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.898+0000 7fb1ac9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a8153ed0 0x7fb1a80c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:01.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.898+0000 7fb1ac9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a81542b0 0x7fb1a80ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:01.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.898+0000 7fb1ac9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a8154c80 con 0x7fb1a80638d0 2023-11-06T20:59:01.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.898+0000 7fb1ac9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a8154f60 con 0x7fb1a81542b0 2023-11-06T20:59:01.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.898+0000 7fb1ac9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a8155240 con 0x7fb1a8153ed0 2023-11-06T20:59:01.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a80638d0 0x7fb1a8146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:01.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a80638d0 0x7fb1a8146180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50254/0 (socket says 172.21.15.33:50254) 2023-11-06T20:59:01.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a659c700 1 -- 172.21.15.33:0/1384089057 learned_addr learned my addr 172.21.15.33:0/1384089057 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:01.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a6d9d700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a81542b0 0x7fb1a80ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:01.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a5d9b700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a8153ed0 0x7fb1a80c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:01.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a659c700 1 -- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a8153ed0 msgr2=0x7fb1a80c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:01.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a659c700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a8153ed0 0x7fb1a80c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:01.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a659c700 1 -- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a81542b0 msgr2=0x7fb1a80ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:01.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a659c700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a81542b0 0x7fb1a80ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:01.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a659c700 1 -- 172.21.15.33:0/1384089057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a0009ae0 con 0x7fb1a80638d0 2023-11-06T20:59:01.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.899+0000 7fb1a659c700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a80638d0 0x7fb1a8146180 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fb19800e930 tx=0x7fb198009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:01.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.900+0000 7fb1967fc700 1 -- 172.21.15.33:0/1384089057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198012600 con 0x7fb1a80638d0 2023-11-06T20:59:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.900+0000 7fb1967fc700 1 -- 172.21.15.33:0/1384089057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb198012760 con 0x7fb1a80638d0 2023-11-06T20:59:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.900+0000 7fb1967fc700 1 -- 172.21.15.33:0/1384089057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198012ae0 con 0x7fb1a80638d0 2023-11-06T20:59:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.900+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1384089057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a8155520 con 0x7fb1a80638d0 2023-11-06T20:59:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.900+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1384089057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a80c48f0 con 0x7fb1a80638d0 2023-11-06T20:59:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.901+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1384089057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a8158a20 con 0x7fb1a80638d0 2023-11-06T20:59:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.902+0000 7fb1967fc700 1 -- 172.21.15.33:0/1384089057 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fb19801b400 con 0x7fb1a80638d0 2023-11-06T20:59:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.903+0000 7fb1967fc700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb18805f2c0 0x7fb188061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.903+0000 7fb1967fc700 1 -- 172.21.15.33:0/1384089057 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6553+0+0 (secure 0 0 0) 0x7fb19807ab40 con 0x7fb1a80638d0 2023-11-06T20:59:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.903+0000 7fb1a5d9b700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb18805f2c0 0x7fb188061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:01.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.904+0000 7fb1a5d9b700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb18805f2c0 0x7fb188061780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fb1a8159ac0 tx=0x7fb19c007bd0).ready entity=mgr.5794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:01.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:01.907+0000 7fb1967fc700 1 -- 172.21.15.33:0/1384089057 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb19804b9c0 con 0x7fb1a80638d0 2023-11-06T20:59:02.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.119+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1384089057 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1a8155b70 con 0x7fb18805f2c0 2023-11-06T20:59:02.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.124+0000 7fb1967fc700 1 -- 172.21.15.33:0/1384089057 <== mgr.5794 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb1a8155b70 con 0x7fb18805f2c0 2023-11-06T20:59:02.123 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path snap_test_dir1 2023-11-06T20:59:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb18805f2c0 msgr2=0x7fb188061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb18805f2c0 0x7fb188061780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fb1a8159ac0 tx=0x7fb19c007bd0).stop 2023-11-06T20:59:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1384089057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a80638d0 msgr2=0x7fb1a8146180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a80638d0 0x7fb1a8146180 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fb19800e930 tx=0x7fb198009060).stop 2023-11-06T20:59:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1384089057 shutdown_connections 2023-11-06T20:59:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb18805f2c0 0x7fb188061780 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a81542b0 0x7fb1a80ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:02.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a80638d0 0x7fb1a8146180 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:02.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 --2- 172.21.15.33:0/1384089057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a8153ed0 0x7fb1a80c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:02.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1384089057 >> 172.21.15.33:0/1384089057 conn(0x7fb1a805ec80 msgr2=0x7fb1a8158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:02.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1384089057 shutdown_connections 2023-11-06T20:59:02.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.126+0000 7fb1ac9ff700 1 -- 172.21.15.33:0/1384089057 wait complete. 2023-11-06T20:59:02.142 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-11-06T20:59:02.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:59:02.143 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T20:59:02.192 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:59:02.192 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-20_59_00_UTC 2023-11-06T20:59:02.193 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-20_58_00_UTC 2023-11-06T20:59:02.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:59:02.194 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-20_58_00_UTC) 2023-11-06T20:59:02.249 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-20_59_00_UTC 2023-11-06T20:59:02.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:59:02.250 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-20_59_00_UTC) 2023-11-06T20:59:02.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:59:02.305 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-06T20:59:02.356 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T20:59:02.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.555+0000 7f8113bff700 1 Processor -- start 2023-11-06T20:59:02.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.555+0000 7f8113bff700 1 -- start start 2023-11-06T20:59:02.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.556+0000 7f8113bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f810c063840 0x7f810c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.556+0000 7f8113bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f810c158c00 con 0x7f810c157700 2023-11-06T20:59:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.556+0000 7f8113bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f810c0c7610 con 0x7f810c0aaf50 2023-11-06T20:59:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.556+0000 7f8113bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0c18f0 con 0x7f810c063840 2023-11-06T20:59:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.556+0000 7f811199b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f810c157700 0x7f810c0c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44358/0 (socket says 172.21.15.33:44358) 2023-11-06T20:59:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.556+0000 7f811199b700 1 -- 172.21.15.33:0/1571968863 learned_addr learned my addr 172.21.15.33:0/1571968863 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:02.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f811219c700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f810c063840 0x7f810c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:02.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510813649 0 0) 0x7f810c158c00 con 0x7f810c157700 2023-11-06T20:59:02.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f80033f0 con 0x7f810c157700 2023-11-06T20:59:02.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300801425 0 0) 0x7f810c0c7610 con 0x7f810c0aaf50 2023-11-06T20:59:02.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f810c158c00 con 0x7f810c0aaf50 2023-11-06T20:59:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2950971921 0 0) 0x7f80f80033f0 con 0x7f810c157700 2023-11-06T20:59:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f810c063840 msgr2=0x7f810c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f810c063840 0x7f810c15f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 >> v1:172.21.15.69:6789/0 conn(0x7f810c0aaf50 legacy=0x7f810c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c157ff0 con 0x7f810c157700 2023-11-06T20:59:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f8108001470 con 0x7f810c157700 2023-11-06T20:59:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 >> v1:172.21.15.33:6789/0 conn(0x7f810c157700 legacy=0x7f810c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f810c063840 0x7f80f8003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80f8003fb0 0x7f80f8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f8004b80 0x7f80f8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:02.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0c18f0 con 0x7f80f8003fb0 2023-11-06T20:59:02.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f800b990 con 0x7f80f8004b80 2023-11-06T20:59:02.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.557+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f800bb50 con 0x7f810c063840 2023-11-06T20:59:02.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.558+0000 7f811119a700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80f8003fb0 0x7f80f8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:02.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.558+0000 7f811199b700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f810c063840 0x7f80f8003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:02.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.558+0000 7f811219c700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f8004b80 0x7f80f8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:02.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.558+0000 7f811119a700 1 -- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f810c063840 msgr2=0x7f80f8003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:02.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.558+0000 7f811119a700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f810c063840 0x7f80f8003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:02.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.558+0000 7f811119a700 1 -- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f8004b80 msgr2=0x7f80f8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:02.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.558+0000 7f811119a700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f8004b80 0x7f80f8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:02.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.558+0000 7f811119a700 1 -- 172.21.15.33:0/1571968863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f80033f0 con 0x7f80f8003fb0 2023-11-06T20:59:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.558+0000 7f811119a700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80f8003fb0 0x7f80f8004690 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f80fc001890 tx=0x7f80fc007bb0).ready entity=mon.0 client_cookie=dc226e6d407c38a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.559+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc007020 con 0x7f80f8003fb0 2023-11-06T20:59:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.559+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc007180 con 0x7f80f8003fb0 2023-11-06T20:59:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.559+0000 7f8110999700 1 -- 172.21.15.33:0/1571968863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc007500 con 0x7f80f8003fb0 2023-11-06T20:59:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.559+0000 7f8113bff700 1 -- 172.21.15.33:0/1571968863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80f8003fb0 msgr2=0x7f80f8004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.559+0000 7f8113bff700 1 --2- 172.21.15.33:0/1571968863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80f8003fb0 0x7f80f8004690 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f80fc001890 tx=0x7f80fc007bb0).stop 2023-11-06T20:59:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.559+0000 7f811219c700 1 -- 172.21.15.33:0/1571968863 reap_dead start 2023-11-06T20:59:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.559+0000 7f8113bff700 1 -- 172.21.15.33:0/1571968863 shutdown_connections 2023-11-06T20:59:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.559+0000 7f8113bff700 1 -- 172.21.15.33:0/1571968863 >> 172.21.15.33:0/1571968863 conn(0x7f810c05ec80 msgr2=0x7f810c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.559+0000 7f8113bff700 1 -- 172.21.15.33:0/1571968863 shutdown_connections 2023-11-06T20:59:02.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.559+0000 7f8113bff700 1 -- 172.21.15.33:0/1571968863 wait complete. 2023-11-06T20:59:02.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.561+0000 7f8113bff700 1 Processor -- start 2023-11-06T20:59:02.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.561+0000 7f8113bff700 1 -- start start 2023-11-06T20:59:02.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f8113bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80f8003fb0 0x7f810c14dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:02.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f8113bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f810c0b0aa0 0x7f810c14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:02.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f8113bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810c14d830 0x7f810c150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:02.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f8113bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0c2ba0 con 0x7f810c14d830 2023-11-06T20:59:02.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f8113bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0c1f90 con 0x7f810c0b0aa0 2023-11-06T20:59:02.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f8113bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0c2270 con 0x7f80f8003fb0 2023-11-06T20:59:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f811219c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810c14d830 0x7f810c150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f811219c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810c14d830 0x7f810c150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50274/0 (socket says 172.21.15.33:50274) 2023-11-06T20:59:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f811219c700 1 -- 172.21.15.33:0/3643129569 learned_addr learned my addr 172.21.15.33:0/3643129569 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f811199b700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80f8003fb0 0x7f810c14dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.562+0000 7f811119a700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f810c0b0aa0 0x7f810c14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f811219c700 1 -- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80f8003fb0 msgr2=0x7f810c14dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f811219c700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80f8003fb0 0x7f810c14dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f811219c700 1 -- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f810c0b0aa0 msgr2=0x7f810c14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f811219c700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f810c0b0aa0 0x7f810c14d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f811219c700 1 -- 172.21.15.33:0/3643129569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c0c2550 con 0x7f810c14d830 2023-11-06T20:59:02.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f811219c700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810c14d830 0x7f810c150630 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f810c064030 tx=0x7f810400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:02.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f8101ffb700 1 -- 172.21.15.33:0/3643129569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810400e040 con 0x7f810c14d830 2023-11-06T20:59:02.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f8101ffb700 1 -- 172.21.15.33:0/3643129569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810400f040 con 0x7f810c14d830 2023-11-06T20:59:02.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f8113bff700 1 -- 172.21.15.33:0/3643129569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810c0c2830 con 0x7f810c14d830 2023-11-06T20:59:02.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f8101ffb700 1 -- 172.21.15.33:0/3643129569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810400fd70 con 0x7f810c14d830 2023-11-06T20:59:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.563+0000 7f8113bff700 1 -- 172.21.15.33:0/3643129569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810c150b20 con 0x7f810c14d830 2023-11-06T20:59:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.564+0000 7f8113bff700 1 -- 172.21.15.33:0/3643129569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810c0c2d10 con 0x7f810c14d830 2023-11-06T20:59:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.565+0000 7f8101ffb700 1 -- 172.21.15.33:0/3643129569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f8104015660 con 0x7f810c14d830 2023-11-06T20:59:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.566+0000 7f8101ffb700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f80f005f2c0 0x7f80f0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.566+0000 7f8101ffb700 1 -- 172.21.15.33:0/3643129569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6553+0+0 (secure 0 0 0) 0x7f810407aff0 con 0x7f810c14d830 2023-11-06T20:59:02.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.567+0000 7f811199b700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f80f005f2c0 0x7f80f0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:02.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.567+0000 7f811199b700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f80f005f2c0 0x7f80f0061780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8108001200 tx=0x7f8108001df0).ready entity=mgr.5794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:02.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.570+0000 7f8101ffb700 1 -- 172.21.15.33:0/3643129569 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f81040a9af0 con 0x7f810c14d830 2023-11-06T20:59:02.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:02.795+0000 7f8113bff700 1 -- 172.21.15.33:0/3643129569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f810c06c6c0 con 0x7f810c14d830 2023-11-06T20:59:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.012+0000 7f8101ffb700 1 -- 172.21.15.33:0/3643129569 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f81040789f0 con 0x7f810c14d830 2023-11-06T20:59:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.013+0000 7f8101ffb700 1 -- 172.21.15.33:0/3643129569 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v21) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8104070710 con 0x7f810c14d830 2023-11-06T20:59:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.016+0000 7f8113bff700 1 -- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f80f005f2c0 msgr2=0x7f80f0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.016+0000 7f8113bff700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f80f005f2c0 0x7f80f0061780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8108001200 tx=0x7f8108001df0).stop 2023-11-06T20:59:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.016+0000 7f8113bff700 1 -- 172.21.15.33:0/3643129569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810c14d830 msgr2=0x7f810c150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.016+0000 7f8113bff700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810c14d830 0x7f810c150630 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f810c064030 tx=0x7f810400b040).stop 2023-11-06T20:59:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.016+0000 7f8113bff700 1 -- 172.21.15.33:0/3643129569 shutdown_connections 2023-11-06T20:59:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.016+0000 7f8113bff700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f80f005f2c0 0x7f80f0061780 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.016+0000 7f8113bff700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80f8003fb0 0x7f810c14dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.016+0000 7f8113bff700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810c14d830 0x7f810c150630 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.016+0000 7f8113bff700 1 --2- 172.21.15.33:0/3643129569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f810c0b0aa0 0x7f810c14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.016+0000 7f8113bff700 1 -- 172.21.15.33:0/3643129569 >> 172.21.15.33:0/3643129569 conn(0x7f810c05ec80 msgr2=0x7f810c14ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.017+0000 7f8113bff700 1 -- 172.21.15.33:0/3643129569 shutdown_connections 2023-11-06T20:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.017+0000 7f8113bff700 1 -- 172.21.15.33:0/3643129569 wait complete. 2023-11-06T20:59:03.043 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T20:59:03.044 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:59:03.142 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:59:03.233 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:03.233+0000 7fcc95f5a680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:59:03.236 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T20:59:03.237 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T20:59:03.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:59:03.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T20:59:03.242 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:03.242+0000 7f28aeca0680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:59:03.313 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:03.313+0000 7fcc95f5a680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:59:03.321 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:59:03.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:59:03.322 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T20:59:03.329 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:03.329+0000 7f28aeca0680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:59:03.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.586+0000 7fa6c8dff700 1 Processor -- start 2023-11-06T20:59:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.586+0000 7fa6c8dff700 1 -- start start 2023-11-06T20:59:03.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.588+0000 7fa6c8dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa6c41580e0 0x7fa6c41584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:03.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.588+0000 7fa6c8dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa6c40bdbe0 0x7fa6c40684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:03.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.588+0000 7fa6c8dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa6c40a7910 0x7fa6c40a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:03.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.588+0000 7fa6c8dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c40be430 con 0x7fa6c40bdbe0 2023-11-06T20:59:03.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.588+0000 7fa6c8dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c4064010 con 0x7fa6c40a7910 2023-11-06T20:59:03.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.588+0000 7fa6c8dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40a82b0 con 0x7fa6c41580e0 2023-11-06T20:59:03.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa6c40bdbe0 0x7fa6c40684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:03.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa6c40bdbe0 0x7fa6c40684c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50312/0 (socket says 172.21.15.33:50312) 2023-11-06T20:59:03.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c1d9b700 1 -- 172.21.15.33:0/2611704599 learned_addr learned my addr 172.21.15.33:0/2611704599 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:03.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c2d9d700 1 --2- 172.21.15.33:0/2611704599 >> v2:172.21.15.69:3300/0 conn(0x7fa6c40a7910 0x7fa6c40a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:03.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c259c700 1 --2- 172.21.15.33:0/2611704599 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa6c41580e0 0x7fa6c41584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:03.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c1d9b700 1 -- 172.21.15.33:0/2611704599 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa6c41580e0 msgr2=0x7fa6c41584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:03.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c1d9b700 1 --2- 172.21.15.33:0/2611704599 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa6c41580e0 0x7fa6c41584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c1d9b700 1 -- 172.21.15.33:0/2611704599 >> v2:172.21.15.69:3300/0 conn(0x7fa6c40a7910 msgr2=0x7fa6c40a7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:03.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c1d9b700 1 --2- 172.21.15.33:0/2611704599 >> v2:172.21.15.69:3300/0 conn(0x7fa6c40a7910 0x7fa6c40a7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c1d9b700 1 -- 172.21.15.33:0/2611704599 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40689b0 con 0x7fa6c40bdbe0 2023-11-06T20:59:03.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.589+0000 7fa6c2d9d700 1 --2- 172.21.15.33:0/2611704599 >> v2:172.21.15.69:3300/0 conn(0x7fa6c40a7910 0x7fa6c40a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:03.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c1d9b700 1 --2- 172.21.15.33:0/2611704599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c40bdbe0 0x7fa6c40684c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fa6c406a450 tx=0x7fa6b80090f0).ready entity=mon.0 client_cookie=2c610409f91cf9b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c159a700 1 -- 172.21.15.33:0/2611704599 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b800f510 con 0x7fa6c40bdbe0 2023-11-06T20:59:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c159a700 1 -- 172.21.15.33:0/2611704599 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b800f670 con 0x7fa6c40bdbe0 2023-11-06T20:59:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c159a700 1 -- 172.21.15.33:0/2611704599 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b800fa10 con 0x7fa6c40bdbe0 2023-11-06T20:59:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c8dff700 1 -- 172.21.15.33:0/2611704599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c40bdbe0 msgr2=0x7fa6c40684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c8dff700 1 --2- 172.21.15.33:0/2611704599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c40bdbe0 0x7fa6c40684c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fa6c406a450 tx=0x7fa6b80090f0).stop 2023-11-06T20:59:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c8dff700 1 -- 172.21.15.33:0/2611704599 shutdown_connections 2023-11-06T20:59:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c8dff700 1 --2- 172.21.15.33:0/2611704599 >> v2:172.21.15.69:3300/0 conn(0x7fa6c40a7910 0x7fa6c40a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c8dff700 1 --2- 172.21.15.33:0/2611704599 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa6c41580e0 0x7fa6c41584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c8dff700 1 --2- 172.21.15.33:0/2611704599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c40bdbe0 0x7fa6c40684c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.590+0000 7fa6c8dff700 1 -- 172.21.15.33:0/2611704599 >> 172.21.15.33:0/2611704599 conn(0x7fa6c405ec80 msgr2=0x7fa6c40664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.591+0000 7fa6c8dff700 1 -- 172.21.15.33:0/2611704599 shutdown_connections 2023-11-06T20:59:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.591+0000 7fa6c8dff700 1 -- 172.21.15.33:0/2611704599 wait complete. 2023-11-06T20:59:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.592+0000 7fa6c8dff700 1 Processor -- start 2023-11-06T20:59:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.592+0000 7fa6c8dff700 1 -- start start 2023-11-06T20:59:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c8dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c40a7910 0x7fa6c4137770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c8dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa6c40bdbe0 0x7fa6c40aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c8dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c41580e0 0x7fa6c40ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c8dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40aab30 con 0x7fa6c41580e0 2023-11-06T20:59:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c8dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40a9f80 con 0x7fa6c40a7910 2023-11-06T20:59:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c8dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40aa200 con 0x7fa6c40bdbe0 2023-11-06T20:59:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c40a7910 0x7fa6c4137770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c41580e0 0x7fa6c40ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c40a7910 0x7fa6c4137770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52042/0 (socket says 172.21.15.33:52042) 2023-11-06T20:59:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c259c700 1 -- 172.21.15.33:0/1490324683 learned_addr learned my addr 172.21.15.33:0/1490324683 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.593+0000 7fa6c1d9b700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa6c40bdbe0 0x7fa6c40aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.594+0000 7fa6c259c700 1 -- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa6c40bdbe0 msgr2=0x7fa6c40aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.594+0000 7fa6c259c700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa6c40bdbe0 0x7fa6c40aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.594+0000 7fa6c259c700 1 -- 172.21.15.33:0/1490324683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c41580e0 msgr2=0x7fa6c40ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.594+0000 7fa6c259c700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c41580e0 0x7fa6c40ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.594+0000 7fa6c259c700 1 -- 172.21.15.33:0/1490324683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc009a20 con 0x7fa6c40a7910 2023-11-06T20:59:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.594+0000 7fa6c259c700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c40a7910 0x7fa6c4137770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fa6b400e930 tx=0x7fa6b4009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.594+0000 7fa6b27fc700 1 -- 172.21.15.33:0/1490324683 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4009510 con 0x7fa6c40a7910 2023-11-06T20:59:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.594+0000 7fa6b27fc700 1 -- 172.21.15.33:0/1490324683 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b400c670 con 0x7fa6c40a7910 2023-11-06T20:59:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.594+0000 7fa6b27fc700 1 -- 172.21.15.33:0/1490324683 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400c9f0 con 0x7fa6c40a7910 2023-11-06T20:59:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.594+0000 7fa6c8dff700 1 -- 172.21.15.33:0/1490324683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b8009370 con 0x7fa6c40a7910 2023-11-06T20:59:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.595+0000 7fa6c8dff700 1 -- 172.21.15.33:0/1490324683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c40aa7c0 con 0x7fa6c40a7910 2023-11-06T20:59:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.596+0000 7fa6c8dff700 1 -- 172.21.15.33:0/1490324683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c40aae10 con 0x7fa6c40a7910 2023-11-06T20:59:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.597+0000 7fa6b27fc700 1 -- 172.21.15.33:0/1490324683 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fa6b401b400 con 0x7fa6c40a7910 2023-11-06T20:59:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.597+0000 7fa6b27fc700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa6a405f270 0x7fa6a4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.598+0000 7fa6b27fc700 1 -- 172.21.15.33:0/1490324683 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6553+0+0 (secure 0 0 0) 0x7fa6b407aa80 con 0x7fa6c40a7910 2023-11-06T20:59:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.598+0000 7fa6c1d9b700 1 -- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa6a405f270 msgr2=0x7fa6a4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.598+0000 7fa6c1d9b700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa6a405f270 0x7fa6a4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.603+0000 7fa6b27fc700 1 -- 172.21.15.33:0/1490324683 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa6b404b900 con 0x7fa6c40a7910 2023-11-06T20:59:03.642 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:03.642+0000 7fcc95f5a680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:59:03.710 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:03.711+0000 7f28aeca0680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:59:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.799+0000 7fa6c1d9b700 1 -- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa6a405f270 msgr2=0x7fa6a4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.799+0000 7fa6c1d9b700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa6a405f270 0x7fa6a4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.849+0000 7fa6c8dff700 1 -- 172.21.15.33:0/1490324683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6c40a5f10 con 0x7fa6c40a7910 2023-11-06T20:59:03.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.850+0000 7fa6b27fc700 1 -- 172.21.15.33:0/1490324683 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4483 (secure 0 0 0) 0x7fa6b404ef20 con 0x7fa6c40a7910 2023-11-06T20:59:03.849 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:03.849 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/455721224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":455721224},{"type":"v1","addr":"172.21.15.69:6836","nonce":455721224}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5632,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2050732931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2050732931},{"type":"v1","addr":"172.21.15.33:6834","nonce":2050732931}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1579729451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1579729451},{"type":"v1","addr":"172.21.15.33:6835","nonce":1579729451}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:57:01.696412+0000","modified":"2023-11-06T20:57:04.702227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/698812717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":698812717},{"type":"v1","addr":"172.21.15.69:6837","nonce":698812717}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-06T20:59:03.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 -- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa6a405f270 msgr2=0x7fa6a4061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:03.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa6a405f270 0x7fa6a4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:03.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 -- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c40a7910 msgr2=0x7fa6c4137770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:03.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c40a7910 0x7fa6c4137770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fa6b400e930 tx=0x7fa6b4009060).stop 2023-11-06T20:59:03.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 -- 172.21.15.33:0/1490324683 shutdown_connections 2023-11-06T20:59:03.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa6a405f270 0x7fa6a4061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:03.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c41580e0 0x7fa6c40ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6c40a7910 0x7fa6c4137770 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 --2- 172.21.15.33:0/1490324683 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa6c40bdbe0 0x7fa6c40aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:03.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 -- 172.21.15.33:0/1490324683 >> 172.21.15.33:0/1490324683 conn(0x7fa6c405ec80 msgr2=0x7fa6c4068290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:03.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 -- 172.21.15.33:0/1490324683 shutdown_connections 2023-11-06T20:59:03.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:03.852+0000 7fa6c8dff700 1 -- 172.21.15.33:0/1490324683 wait complete. 2023-11-06T20:59:03.854 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 22 2023-11-06T20:59:03.871 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.069+0000 7f1c9a7ff700 1 Processor -- start 2023-11-06T20:59:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.069+0000 7f1c9a7ff700 1 -- start start 2023-11-06T20:59:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.071+0000 7f1c9a7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c940c64c0 0x7f1c941544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.071+0000 7f1c9a7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c940c0880 con 0x7f1c94064ce0 2023-11-06T20:59:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.071+0000 7f1c9a7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c940c8260 con 0x7f1c94154110 2023-11-06T20:59:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.071+0000 7f1c9a7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940bfd40 con 0x7f1c940c64c0 2023-11-06T20:59:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.071+0000 7f1c937fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1c94064ce0 0x7f1c940c5db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44376/0 (socket says 172.21.15.33:44376) 2023-11-06T20:59:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.071+0000 7f1c937fe700 1 -- 172.21.15.33:0/1540203311 learned_addr learned my addr 172.21.15.33:0/1540203311 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.071+0000 7f1c98d9c700 1 --2- 172.21.15.33:0/1540203311 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c940c64c0 0x7f1c941544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.072+0000 7f1c92ffd700 1 -- 172.21.15.33:0/1540203311 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761577201 0 0) 0x7f1c940c0880 con 0x7f1c94064ce0 2023-11-06T20:59:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.072+0000 7f1c92ffd700 1 -- 172.21.15.33:0/1540203311 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c7c0033f0 con 0x7f1c94064ce0 2023-11-06T20:59:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.072+0000 7f1c92ffd700 1 -- 172.21.15.33:0/1540203311 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274777923 0 0) 0x7f1c940c8260 con 0x7f1c94154110 2023-11-06T20:59:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.072+0000 7f1c92ffd700 1 -- 172.21.15.33:0/1540203311 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c940c0880 con 0x7f1c94154110 2023-11-06T20:59:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.072+0000 7f1c98d9c700 1 -- 172.21.15.33:0/1540203311 >> v1:172.21.15.69:6789/0 conn(0x7f1c94154110 legacy=0x7f1c940645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.072+0000 7f1c98d9c700 1 -- 172.21.15.33:0/1540203311 >> v1:172.21.15.33:6789/0 conn(0x7f1c94064ce0 legacy=0x7f1c940c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.072+0000 7f1c98d9c700 1 -- 172.21.15.33:0/1540203311 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940bff50 con 0x7f1c940c64c0 2023-11-06T20:59:04.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.072+0000 7f1c98d9c700 1 --2- 172.21.15.33:0/1540203311 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c940c64c0 0x7f1c941544f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f1c8c009e90 tx=0x7f1c8c009780).ready entity=mon.2 client_cookie=91ae29793b938a99 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:04.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.072+0000 7f1c92ffd700 1 -- 172.21.15.33:0/1540203311 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c00e070 con 0x7f1c940c64c0 2023-11-06T20:59:04.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.072+0000 7f1c92ffd700 1 -- 172.21.15.33:0/1540203311 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c00bab0 con 0x7f1c940c64c0 2023-11-06T20:59:04.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.073+0000 7f1c92ffd700 1 -- 172.21.15.33:0/1540203311 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c00be30 con 0x7f1c940c64c0 2023-11-06T20:59:04.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.073+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/1540203311 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c940c64c0 msgr2=0x7f1c941544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.073+0000 7f1c9a7ff700 1 --2- 172.21.15.33:0/1540203311 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c940c64c0 0x7f1c941544f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f1c8c009e90 tx=0x7f1c8c009780).stop 2023-11-06T20:59:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.073+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/1540203311 shutdown_connections 2023-11-06T20:59:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.073+0000 7f1c9a7ff700 1 --2- 172.21.15.33:0/1540203311 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c940c64c0 0x7f1c941544f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.073+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/1540203311 >> 172.21.15.33:0/1540203311 conn(0x7f1c940cb8b0 msgr2=0x7f1c940c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.073+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/1540203311 shutdown_connections 2023-11-06T20:59:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.073+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/1540203311 wait complete. 2023-11-06T20:59:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.075+0000 7f1c9a7ff700 1 Processor -- start 2023-11-06T20:59:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.075+0000 7f1c9a7ff700 1 -- start start 2023-11-06T20:59:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c9a7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c94064ce0 0x7f1c940add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c9a7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c940c64c0 0x7f1c941598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c9a7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c94154110 0x7f1c9414ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c9a7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940c2b90 con 0x7f1c940c64c0 2023-11-06T20:59:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c9a7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94159dd0 con 0x7f1c94064ce0 2023-11-06T20:59:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c9a7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940ae3e0 con 0x7f1c94154110 2023-11-06T20:59:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c940c64c0 0x7f1c941598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c940c64c0 0x7f1c941598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50334/0 (socket says 172.21.15.33:50334) 2023-11-06T20:59:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c937fe700 1 -- 172.21.15.33:0/3693526133 learned_addr learned my addr 172.21.15.33:0/3693526133 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c93fff700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c94064ce0 0x7f1c940add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c98d9c700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c94154110 0x7f1c9414ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c937fe700 1 -- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c94154110 msgr2=0x7f1c9414ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c937fe700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c94154110 0x7f1c9414ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c937fe700 1 -- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c94064ce0 msgr2=0x7f1c940add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c937fe700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c94064ce0 0x7f1c940add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c937fe700 1 -- 172.21.15.33:0/3693526133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c009a20 con 0x7f1c940c64c0 2023-11-06T20:59:04.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.076+0000 7f1c98d9c700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c94154110 0x7f1c9414ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:04.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.077+0000 7f1c937fe700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c940c64c0 0x7f1c941598e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f1c8800c490 tx=0x7f1c88007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:04.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.077+0000 7f1c7bfff700 1 -- 172.21.15.33:0/3693526133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c880072e0 con 0x7f1c940c64c0 2023-11-06T20:59:04.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.077+0000 7f1c7bfff700 1 -- 172.21.15.33:0/3693526133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c88007440 con 0x7f1c940c64c0 2023-11-06T20:59:04.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.077+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/3693526133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c940ae6c0 con 0x7f1c940c64c0 2023-11-06T20:59:04.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.077+0000 7f1c7bfff700 1 -- 172.21.15.33:0/3693526133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c880057a0 con 0x7f1c940c64c0 2023-11-06T20:59:04.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.077+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/3693526133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c940c2820 con 0x7f1c940c64c0 2023-11-06T20:59:04.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.078+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/3693526133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c940c2d00 con 0x7f1c940c64c0 2023-11-06T20:59:04.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.079+0000 7f1c7bfff700 1 -- 172.21.15.33:0/3693526133 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f1c88005b90 con 0x7f1c940c64c0 2023-11-06T20:59:04.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.080+0000 7f1c7bfff700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c8005f270 0x7f1c80061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.080+0000 7f1c7bfff700 1 -- 172.21.15.33:0/3693526133 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6553+0+0 (secure 0 0 0) 0x7f1c88079e40 con 0x7f1c940c64c0 2023-11-06T20:59:04.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.080+0000 7f1c93fff700 1 -- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c8005f270 msgr2=0x7f1c80061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:04.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.080+0000 7f1c93fff700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c8005f270 0x7f1c80061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:04.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.088+0000 7f1c7bfff700 1 -- 172.21.15.33:0/3693526133 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1c8804ada0 con 0x7f1c940c64c0 2023-11-06T20:59:04.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.271+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/3693526133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c9406c6c0 con 0x7f1c940c64c0 2023-11-06T20:59:04.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.273+0000 7f1c7bfff700 1 -- 172.21.15.33:0/3693526133 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+17627 (secure 0 0 0) 0x7f1c8804e3c0 con 0x7f1c940c64c0 2023-11-06T20:59:04.273 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:04.273 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":69,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:58:17.249098+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:57:00.140567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T20:57:00.672828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.3","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.32","mappings":[{"from":5,"to":7}]},{"pgid":"7.3c","mappings":[{"from":5,"to":0}]},{"pgid":"8.31","mappings":[{"from":2,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/341956077":"2023-11-07T20:57:16.149070+0000","172.21.15.69:0/3137643476":"2023-11-07T20:57:16.149070+0000","172.21.15.69:0/2617344929":"2023-11-07T20:57:16.149070+0000","172.21.15.69:6833/114912":"2023-11-07T20:57:16.149070+0000","172.21.15.69:6832/114912":"2023-11-07T20:57:16.149070+0000","172.21.15.69:0/3896247200":"2023-11-07T20:57:16.149070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:04.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c8005f270 msgr2=0x7f1c80061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:04.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c8005f270 0x7f1c80061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:04.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/3693526133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c940c64c0 msgr2=0x7f1c941598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c940c64c0 0x7f1c941598e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f1c8800c490 tx=0x7f1c88007ca0).stop 2023-11-06T20:59:04.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/3693526133 shutdown_connections 2023-11-06T20:59:04.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c8005f270 0x7f1c80061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:04.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c94154110 0x7f1c9414ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c94064ce0 0x7f1c940add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 --2- 172.21.15.33:0/3693526133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c940c64c0 0x7f1c941598e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/3693526133 >> 172.21.15.33:0/3693526133 conn(0x7f1c940cb8b0 msgr2=0x7f1c940abc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:04.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/3693526133 shutdown_connections 2023-11-06T20:59:04.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.275+0000 7f1c9a7ff700 1 -- 172.21.15.33:0/3693526133 wait complete. 2023-11-06T20:59:04.293 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:57:00.140567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:57:00.672828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T20:59:04.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:59:04.385 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:04.385+0000 7fcc95f5a680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:59:04.468 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:04.468+0000 7fcc95f5a680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:59:04.517 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:04.518+0000 7f28aeca0680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:59:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.538+0000 7fce7c3ff700 1 Processor -- start 2023-11-06T20:59:04.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.538+0000 7fce7c3ff700 1 -- start start 2023-11-06T20:59:04.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.539+0000 7fce7c3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fce74154330 0x7fce740c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.539+0000 7fce7c3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce740c6b60 con 0x7fce74154710 2023-11-06T20:59:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.539+0000 7fce7c3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce740c7700 con 0x7fce740c7ff0 2023-11-06T20:59:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.539+0000 7fce7c3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce740c5cd0 con 0x7fce74154330 2023-11-06T20:59:04.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce7a19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fce74154330 0x7fce740c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce7999a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fce74154710 0x7fce740c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44388/0 (socket says 172.21.15.33:44388) 2023-11-06T20:59:04.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce7999a700 1 -- 172.21.15.33:0/2317288034 learned_addr learned my addr 172.21.15.33:0/2317288034 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:04.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331903392 0 0) 0x7fce740c6b60 con 0x7fce74154710 2023-11-06T20:59:04.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce600033f0 con 0x7fce74154710 2023-11-06T20:59:04.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458413489 0 0) 0x7fce740c7700 con 0x7fce740c7ff0 2023-11-06T20:59:04.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce740c6b60 con 0x7fce740c7ff0 2023-11-06T20:59:04.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1168722815 0 0) 0x7fce600033f0 con 0x7fce74154710 2023-11-06T20:59:04.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 >> v2:172.21.15.33:3300/0 conn(0x7fce74154330 msgr2=0x7fce740c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:04.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 --2- 172.21.15.33:0/2317288034 >> v2:172.21.15.33:3300/0 conn(0x7fce74154330 0x7fce740c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 >> v1:172.21.15.69:6789/0 conn(0x7fce740c7ff0 legacy=0x7fce740646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce740c5f50 con 0x7fce74154710 2023-11-06T20:59:04.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fce64001470 con 0x7fce74154710 2023-11-06T20:59:04.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 >> v1:172.21.15.33:6789/0 conn(0x7fce74154710 legacy=0x7fce740c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce60003ac0 0x7fce60003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.540+0000 7fce79199700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce60004690 0x7fce60004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce79199700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce60005000 0x7fce600140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce740c5cd0 con 0x7fce60003ac0 2023-11-06T20:59:04.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce600146c0 con 0x7fce60005000 2023-11-06T20:59:04.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce60014880 con 0x7fce60004690 2023-11-06T20:59:04.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce7999a700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce60004690 0x7fce60004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce7a19b700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce60003ac0 0x7fce60003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce7999a700 1 -- 172.21.15.33:0/2317288034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce60005000 msgr2=0x7fce600140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:04.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce7999a700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce60005000 0x7fce600140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:04.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce7999a700 1 -- 172.21.15.33:0/2317288034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce60003ac0 msgr2=0x7fce60003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:04.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce7999a700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce60003ac0 0x7fce60003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce7999a700 1 -- 172.21.15.33:0/2317288034 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600033f0 con 0x7fce60004690 2023-11-06T20:59:04.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce7a99c700 1 -- 172.21.15.33:0/2317288034 reap_dead start 2023-11-06T20:59:04.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.541+0000 7fce7a19b700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce60003ac0 0x7fce60003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:04.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.542+0000 7fce7999a700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce60004690 0x7fce60004b10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fce64007f50 tx=0x7fce64001d40).ready entity=mon.2 client_cookie=402985164b1c02c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:04.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.542+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6400c960 con 0x7fce60004690 2023-11-06T20:59:04.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.542+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6400cac0 con 0x7fce60004690 2023-11-06T20:59:04.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.542+0000 7fce79199700 1 -- 172.21.15.33:0/2317288034 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6400ce40 con 0x7fce60004690 2023-11-06T20:59:04.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.542+0000 7fce7c3ff700 1 -- 172.21.15.33:0/2317288034 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce60004690 msgr2=0x7fce60004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.542+0000 7fce7c3ff700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce60004690 0x7fce60004b10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fce64007f50 tx=0x7fce64001d40).stop 2023-11-06T20:59:04.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.542+0000 7fce7c3ff700 1 -- 172.21.15.33:0/2317288034 shutdown_connections 2023-11-06T20:59:04.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.542+0000 7fce7c3ff700 1 --2- 172.21.15.33:0/2317288034 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce60004690 0x7fce60004b10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.542+0000 7fce7c3ff700 1 -- 172.21.15.33:0/2317288034 >> 172.21.15.33:0/2317288034 conn(0x7fce740cb910 msgr2=0x7fce740c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:04.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.543+0000 7fce7c3ff700 1 -- 172.21.15.33:0/2317288034 shutdown_connections 2023-11-06T20:59:04.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.543+0000 7fce7c3ff700 1 -- 172.21.15.33:0/2317288034 wait complete. 2023-11-06T20:59:04.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.544+0000 7fce7c3ff700 1 Processor -- start 2023-11-06T20:59:04.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.544+0000 7fce7c3ff700 1 -- start start 2023-11-06T20:59:04.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.545+0000 7fce7c3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce60004690 0x7fce740c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.545+0000 7fce7c3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce740a9df0 0x7fce74148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.545+0000 7fce7c3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce740affc0 0x7fce740b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.545+0000 7fce7c3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce740afb30 con 0x7fce740a9df0 2023-11-06T20:59:04.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.545+0000 7fce7c3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7414e7c0 con 0x7fce60004690 2023-11-06T20:59:04.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.545+0000 7fce7c3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7414eaa0 con 0x7fce740affc0 2023-11-06T20:59:04.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.545+0000 7fce7999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce740a9df0 0x7fce74148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce740a9df0 0x7fce74148960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50352/0 (socket says 172.21.15.33:50352) 2023-11-06T20:59:04.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7a19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce60004690 0x7fce740c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7999a700 1 -- 172.21.15.33:0/462106874 learned_addr learned my addr 172.21.15.33:0/462106874 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:04.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7a99c700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce740affc0 0x7fce740b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7999a700 1 -- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce740affc0 msgr2=0x7fce740b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7999a700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce740affc0 0x7fce740b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7999a700 1 -- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce60004690 msgr2=0x7fce740c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7999a700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce60004690 0x7fce740c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7999a700 1 -- 172.21.15.33:0/462106874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce740af7c0 con 0x7fce740a9df0 2023-11-06T20:59:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7a19b700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce60004690 0x7fce740c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:04.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.546+0000 7fce7999a700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce740a9df0 0x7fce74148960 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fce64001120 tx=0x7fce64015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:04.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.547+0000 7fce6a7fc700 1 -- 172.21.15.33:0/462106874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640074f0 con 0x7fce740a9df0 2023-11-06T20:59:04.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.547+0000 7fce6a7fc700 1 -- 172.21.15.33:0/462106874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64007650 con 0x7fce740a9df0 2023-11-06T20:59:04.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.547+0000 7fce6a7fc700 1 -- 172.21.15.33:0/462106874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640048c0 con 0x7fce740a9df0 2023-11-06T20:59:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.547+0000 7fce7c3ff700 1 -- 172.21.15.33:0/462106874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce74148e50 con 0x7fce740a9df0 2023-11-06T20:59:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.547+0000 7fce7c3ff700 1 -- 172.21.15.33:0/462106874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce740c1ee0 con 0x7fce740a9df0 2023-11-06T20:59:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.548+0000 7fce7c3ff700 1 -- 172.21.15.33:0/462106874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce740af5e0 con 0x7fce740a9df0 2023-11-06T20:59:04.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.549+0000 7fce6a7fc700 1 -- 172.21.15.33:0/462106874 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fce64004c30 con 0x7fce740a9df0 2023-11-06T20:59:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.550+0000 7fce6a7fc700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce5805f270 0x7fce58061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.550+0000 7fce6a7fc700 1 -- 172.21.15.33:0/462106874 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6553+0+0 (secure 0 0 0) 0x7fce6407b430 con 0x7fce740a9df0 2023-11-06T20:59:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.550+0000 7fce7a19b700 1 -- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce5805f270 msgr2=0x7fce58061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.550+0000 7fce7a19b700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce5805f270 0x7fce58061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.554+0000 7fce6a7fc700 1 -- 172.21.15.33:0/462106874 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fce6404c3d0 con 0x7fce740a9df0 2023-11-06T20:59:04.608 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:04.609+0000 7f28aeca0680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:59:04.699 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:04.699+0000 7fcc95f5a680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:59:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.736+0000 7fce7c3ff700 1 -- 172.21.15.33:0/462106874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fce740af5e0 con 0x7fce740a9df0 2023-11-06T20:59:04.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.736+0000 7fce6a7fc700 1 -- 172.21.15.33:0/462106874 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v69) v1 ==== 110+0+10 (secure 0 0 0) 0x7fce64024020 con 0x7fce740a9df0 2023-11-06T20:59:04.737 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:59:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 -- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce5805f270 msgr2=0x7fce58061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce5805f270 0x7fce58061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 -- 172.21.15.33:0/462106874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce740a9df0 msgr2=0x7fce74148960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce740a9df0 0x7fce74148960 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fce64001120 tx=0x7fce64015000).stop 2023-11-06T20:59:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 -- 172.21.15.33:0/462106874 shutdown_connections 2023-11-06T20:59:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fce5805f270 0x7fce58061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fce740affc0 0x7fce740b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce60004690 0x7fce740c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 --2- 172.21.15.33:0/462106874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce740a9df0 0x7fce74148960 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 -- 172.21.15.33:0/462106874 >> 172.21.15.33:0/462106874 conn(0x7fce740cb910 msgr2=0x7fce74153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 -- 172.21.15.33:0/462106874 shutdown_connections 2023-11-06T20:59:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.739+0000 7fce7c3ff700 1 -- 172.21.15.33:0/462106874 wait complete. 2023-11-06T20:59:04.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:59:04.861 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:04.862+0000 7f28aeca0680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:59:04.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.968+0000 7fe075bff700 1 Processor -- start 2023-11-06T20:59:04.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.968+0000 7fe075bff700 1 -- start start 2023-11-06T20:59:04.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.970+0000 7fe075bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0700aafc0 0x7fe0701543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.970+0000 7fe075bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0700c6280 con 0x7fe070064670 2023-11-06T20:59:04.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.970+0000 7fe075bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0700c0d90 con 0x7fe070154010 2023-11-06T20:59:04.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.970+0000 7fe075bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700c5990 con 0x7fe0700aafc0 2023-11-06T20:59:04.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.970+0000 7fe06ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe070064670 0x7fe0700c6710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44390/0 (socket says 172.21.15.33:44390) 2023-11-06T20:59:04.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.970+0000 7fe06ffff700 1 -- 172.21.15.33:0/1856135500 learned_addr learned my addr 172.21.15.33:0/1856135500 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:04.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.970+0000 7fe06effd700 1 --2- 172.21.15.33:0/1856135500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0700aafc0 0x7fe0701543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.971+0000 7fe06e7fc700 1 -- 172.21.15.33:0/1856135500 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073040314 0 0) 0x7fe0700c6280 con 0x7fe070064670 2023-11-06T20:59:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.971+0000 7fe06e7fc700 1 -- 172.21.15.33:0/1856135500 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0580033f0 con 0x7fe070064670 2023-11-06T20:59:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.971+0000 7fe06e7fc700 1 -- 172.21.15.33:0/1856135500 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599836283 0 0) 0x7fe0700c0d90 con 0x7fe070154010 2023-11-06T20:59:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.971+0000 7fe06e7fc700 1 -- 172.21.15.33:0/1856135500 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0700c6280 con 0x7fe070154010 2023-11-06T20:59:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.971+0000 7fe06effd700 1 -- 172.21.15.33:0/1856135500 >> v1:172.21.15.69:6789/0 conn(0x7fe070154010 legacy=0x7fe0700c0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.971+0000 7fe06effd700 1 -- 172.21.15.33:0/1856135500 >> v1:172.21.15.33:6789/0 conn(0x7fe070064670 legacy=0x7fe0700c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.971+0000 7fe06effd700 1 -- 172.21.15.33:0/1856135500 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700c1000 con 0x7fe0700aafc0 2023-11-06T20:59:04.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.971+0000 7fe06effd700 1 --2- 172.21.15.33:0/1856135500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0700aafc0 0x7fe0701543f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fe064009790 tx=0x7fe064009060).ready entity=mon.2 client_cookie=f3da8a4951f36204 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:04.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.972+0000 7fe06e7fc700 1 -- 172.21.15.33:0/1856135500 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06400f510 con 0x7fe0700aafc0 2023-11-06T20:59:04.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.972+0000 7fe06e7fc700 1 -- 172.21.15.33:0/1856135500 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06400f670 con 0x7fe0700aafc0 2023-11-06T20:59:04.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.972+0000 7fe06e7fc700 1 -- 172.21.15.33:0/1856135500 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06400f9f0 con 0x7fe0700aafc0 2023-11-06T20:59:04.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.972+0000 7fe075bff700 1 -- 172.21.15.33:0/1856135500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0700aafc0 msgr2=0x7fe0701543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.972+0000 7fe075bff700 1 --2- 172.21.15.33:0/1856135500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0700aafc0 0x7fe0701543f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fe064009790 tx=0x7fe064009060).stop 2023-11-06T20:59:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.972+0000 7fe075bff700 1 -- 172.21.15.33:0/1856135500 shutdown_connections 2023-11-06T20:59:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.972+0000 7fe075bff700 1 --2- 172.21.15.33:0/1856135500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0700aafc0 0x7fe0701543f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.972+0000 7fe075bff700 1 -- 172.21.15.33:0/1856135500 >> 172.21.15.33:0/1856135500 conn(0x7fe0700cb8e0 msgr2=0x7fe0700c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.972+0000 7fe075bff700 1 -- 172.21.15.33:0/1856135500 shutdown_connections 2023-11-06T20:59:04.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.972+0000 7fe075bff700 1 -- 172.21.15.33:0/1856135500 wait complete. 2023-11-06T20:59:04.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.974+0000 7fe075bff700 1 Processor -- start 2023-11-06T20:59:04.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.975+0000 7fe075bff700 1 -- start start 2023-11-06T20:59:04.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe075bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe070064670 0x7fe0700aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe075bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070154010 0x7fe07013d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe075bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe070155330 0x7fe070155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe075bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07013d580 con 0x7fe070064670 2023-11-06T20:59:04.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe075bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe070156ce0 con 0x7fe070154010 2023-11-06T20:59:04.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe075bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701560d0 con 0x7fe070155330 2023-11-06T20:59:04.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe06f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe070064670 0x7fe0700aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe06f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe070064670 0x7fe0700aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50356/0 (socket says 172.21.15.33:50356) 2023-11-06T20:59:04.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe06f7fe700 1 -- 172.21.15.33:0/2011471896 learned_addr learned my addr 172.21.15.33:0/2011471896 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:04.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe06effd700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070154010 0x7fe07013d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.976+0000 7fe06ffff700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe070155330 0x7fe070155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:04.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe06effd700 1 -- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe070155330 msgr2=0x7fe070155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe06effd700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe070155330 0x7fe070155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe06effd700 1 -- 172.21.15.33:0/2011471896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe070064670 msgr2=0x7fe0700aa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:04.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe06effd700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe070064670 0x7fe0700aa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:04.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe06effd700 1 -- 172.21.15.33:0/2011471896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0640092a0 con 0x7fe070154010 2023-11-06T20:59:04.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe06f7fe700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe070064670 0x7fe0700aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:04.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe06effd700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070154010 0x7fe07013d090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fe064014000 tx=0x7fe064017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:04.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe0577fe700 1 -- 172.21.15.33:0/2011471896 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06400fe80 con 0x7fe070154010 2023-11-06T20:59:04.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe0577fe700 1 -- 172.21.15.33:0/2011471896 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06400f7e0 con 0x7fe070154010 2023-11-06T20:59:04.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe075bff700 1 -- 172.21.15.33:0/2011471896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0701563b0 con 0x7fe070154010 2023-11-06T20:59:04.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe0577fe700 1 -- 172.21.15.33:0/2011471896 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06401b640 con 0x7fe070154010 2023-11-06T20:59:04.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.977+0000 7fe075bff700 1 -- 172.21.15.33:0/2011471896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe070156970 con 0x7fe070154010 2023-11-06T20:59:04.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.979+0000 7fe075bff700 1 -- 172.21.15.33:0/2011471896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe070156e50 con 0x7fe070154010 2023-11-06T20:59:04.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.979+0000 7fe0577fe700 1 -- 172.21.15.33:0/2011471896 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fe06401b9b0 con 0x7fe070154010 2023-11-06T20:59:04.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.980+0000 7fe0577fe700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe05c05f270 0x7fe05c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:04.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.980+0000 7fe0577fe700 1 -- 172.21.15.33:0/2011471896 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6553+0+0 (secure 0 0 0) 0x7fe06407bef0 con 0x7fe070154010 2023-11-06T20:59:04.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.980+0000 7fe06f7fe700 1 -- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe05c05f270 msgr2=0x7fe05c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:04.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.980+0000 7fe06f7fe700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe05c05f270 0x7fe05c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:04.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:04.984+0000 7fe0577fe700 1 -- 172.21.15.33:0/2011471896 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe06404cdd0 con 0x7fe070154010 2023-11-06T20:59:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.168+0000 7fe075bff700 1 -- 172.21.15.33:0/2011471896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe070157d60 con 0x7fe070154010 2023-11-06T20:59:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.169+0000 7fe0577fe700 1 -- 172.21.15.33:0/2011471896 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v69) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe064028070 con 0x7fe070154010 2023-11-06T20:59:05.169 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:59:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 -- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe05c05f270 msgr2=0x7fe05c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe05c05f270 0x7fe05c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 -- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070154010 msgr2=0x7fe07013d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070154010 0x7fe07013d090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fe064014000 tx=0x7fe064017000).stop 2023-11-06T20:59:05.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 -- 172.21.15.33:0/2011471896 shutdown_connections 2023-11-06T20:59:05.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe05c05f270 0x7fe05c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:05.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe070155330 0x7fe070155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe070064670 0x7fe0700aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 --2- 172.21.15.33:0/2011471896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070154010 0x7fe07013d090 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 -- 172.21.15.33:0/2011471896 >> 172.21.15.33:0/2011471896 conn(0x7fe0700cb8e0 msgr2=0x7fe0700afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 -- 172.21.15.33:0/2011471896 shutdown_connections 2023-11-06T20:59:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.171+0000 7fe075bff700 1 -- 172.21.15.33:0/2011471896 wait complete. 2023-11-06T20:59:05.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T20:59:05.212 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:05.212+0000 7fcc95f5a680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:59:05.338 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:05.336+0000 7fcc95f5a680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:05.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.391+0000 7f0557fff700 1 Processor -- start 2023-11-06T20:59:05.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.391+0000 7f0557fff700 1 -- start start 2023-11-06T20:59:05.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f0557fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f05500c3590 0x7f05500c3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f0557fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0550154ff0 0x7f0550155480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f0557fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0550064fc0 con 0x7f0550064250 2023-11-06T20:59:05.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f0557fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05500652a0 con 0x7f0550154ff0 2023-11-06T20:59:05.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f0557fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05500bbaa0 con 0x7f05500c3590 2023-11-06T20:59:05.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f055559a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0550154ff0 0x7f0550155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:05.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f055559a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0550154ff0 0x7f0550155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50362/0 (socket says 172.21.15.33:50362) 2023-11-06T20:59:05.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f055559a700 1 -- 172.21.15.33:0/1224591588 learned_addr learned my addr 172.21.15.33:0/1224591588 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:05.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f055659c700 1 --1- 172.21.15.33:0/1224591588 >> v1:172.21.15.33:6789/0 conn(0x7f0550064250 0x7f05500646e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44394/0 (socket says 172.21.15.33:44394) 2023-11-06T20:59:05.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f0555d9b700 1 --2- 172.21.15.33:0/1224591588 >> v2:172.21.15.69:3300/0 conn(0x7f05500c3590 0x7f05500c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:05.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f055559a700 1 -- 172.21.15.33:0/1224591588 >> v2:172.21.15.69:3300/0 conn(0x7f05500c3590 msgr2=0x7f05500c3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:05.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f055559a700 1 --2- 172.21.15.33:0/1224591588 >> v2:172.21.15.69:3300/0 conn(0x7f05500c3590 0x7f05500c3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.393+0000 7f055559a700 1 -- 172.21.15.33:0/1224591588 >> v1:172.21.15.33:6789/0 conn(0x7f0550064250 legacy=0x7f05500646e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:05.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.394+0000 7f055559a700 1 -- 172.21.15.33:0/1224591588 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05500bbd80 con 0x7f0550154ff0 2023-11-06T20:59:05.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.394+0000 7f055559a700 1 --2- 172.21.15.33:0/1224591588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550154ff0 0x7f0550155480 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f0540006fd0 tx=0x7f05400090f0).ready entity=mon.0 client_cookie=36095e636a02edfd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:05.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.394+0000 7f0554d99700 1 -- 172.21.15.33:0/1224591588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000b520 con 0x7f0550154ff0 2023-11-06T20:59:05.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.394+0000 7f0554d99700 1 -- 172.21.15.33:0/1224591588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054000b680 con 0x7f0550154ff0 2023-11-06T20:59:05.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.394+0000 7f0554d99700 1 -- 172.21.15.33:0/1224591588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000ba00 con 0x7f0550154ff0 2023-11-06T20:59:05.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.394+0000 7f0557fff700 1 -- 172.21.15.33:0/1224591588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550154ff0 msgr2=0x7f0550155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:05.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.394+0000 7f0557fff700 1 --2- 172.21.15.33:0/1224591588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550154ff0 0x7f0550155480 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f0540006fd0 tx=0x7f05400090f0).stop 2023-11-06T20:59:05.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.395+0000 7f0557fff700 1 -- 172.21.15.33:0/1224591588 shutdown_connections 2023-11-06T20:59:05.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.395+0000 7f0557fff700 1 --2- 172.21.15.33:0/1224591588 >> v2:172.21.15.69:3300/0 conn(0x7f05500c3590 0x7f05500c3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.395+0000 7f0557fff700 1 --2- 172.21.15.33:0/1224591588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0550154ff0 0x7f0550155480 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.395+0000 7f0557fff700 1 -- 172.21.15.33:0/1224591588 >> 172.21.15.33:0/1224591588 conn(0x7f055005ec80 msgr2=0x7f0550062710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:05.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.395+0000 7f0557fff700 1 -- 172.21.15.33:0/1224591588 shutdown_connections 2023-11-06T20:59:05.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.395+0000 7f0557fff700 1 -- 172.21.15.33:0/1224591588 wait complete. 2023-11-06T20:59:05.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.396+0000 7f0557fff700 1 Processor -- start 2023-11-06T20:59:05.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.396+0000 7f0557fff700 1 -- start start 2023-11-06T20:59:05.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.397+0000 7f0557fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05500c3590 0x7f05500c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.397+0000 7f0557fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0550154ff0 0x7f05500c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.397+0000 7f0557fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05500a6480 0x7f05500c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.397+0000 7f0557fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05500a8af0 con 0x7f05500c3590 2023-11-06T20:59:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.397+0000 7f0557fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05500a9f40 con 0x7f0550154ff0 2023-11-06T20:59:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.397+0000 7f0557fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05501398d0 con 0x7f05500a6480 2023-11-06T20:59:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f0555d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05500c3590 0x7f05500c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:05.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f0555d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05500c3590 0x7f05500c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50372/0 (socket says 172.21.15.33:50372) 2023-11-06T20:59:05.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f0555d9b700 1 -- 172.21.15.33:0/1781537139 learned_addr learned my addr 172.21.15.33:0/1781537139 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:05.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f055559a700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0550154ff0 0x7f05500c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:05.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f055659c700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05500a6480 0x7f05500c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f0555d9b700 1 -- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05500a6480 msgr2=0x7f05500c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f0555d9b700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05500a6480 0x7f05500c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f0555d9b700 1 -- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0550154ff0 msgr2=0x7f05500c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f0555d9b700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0550154ff0 0x7f05500c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f0555d9b700 1 -- 172.21.15.33:0/1781537139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540009370 con 0x7f05500c3590 2023-11-06T20:59:05.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f055559a700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0550154ff0 0x7f05500c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:05.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.398+0000 7f0555d9b700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05500c3590 0x7f05500c7060 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f054c00e960 tx=0x7f054c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:05.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.399+0000 7f0545ffb700 1 -- 172.21.15.33:0/1781537139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c012600 con 0x7f05500c3590 2023-11-06T20:59:05.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.399+0000 7f0545ffb700 1 -- 172.21.15.33:0/1781537139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c012760 con 0x7f05500c3590 2023-11-06T20:59:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.399+0000 7f0557fff700 1 -- 172.21.15.33:0/1781537139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05500a8d40 con 0x7f05500c3590 2023-11-06T20:59:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.399+0000 7f0545ffb700 1 -- 172.21.15.33:0/1781537139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c012ae0 con 0x7f05500c3590 2023-11-06T20:59:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.399+0000 7f0557fff700 1 -- 172.21.15.33:0/1781537139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05500a9300 con 0x7f05500c3590 2023-11-06T20:59:05.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.401+0000 7f0545ffb700 1 -- 172.21.15.33:0/1781537139 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f054c01b400 con 0x7f05500c3590 2023-11-06T20:59:05.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.401+0000 7f0557fff700 1 -- 172.21.15.33:0/1781537139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0550065430 con 0x7f05500c3590 2023-11-06T20:59:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.401+0000 7f0545ffb700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f053405f270 0x7f0534061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.402+0000 7f0545ffb700 1 -- 172.21.15.33:0/1781537139 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6553+0+0 (secure 0 0 0) 0x7f054c07aa60 con 0x7f05500c3590 2023-11-06T20:59:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.402+0000 7f055559a700 1 -- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f053405f270 msgr2=0x7f0534061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.402+0000 7f055559a700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f053405f270 0x7f0534061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.406+0000 7f0545ffb700 1 -- 172.21.15.33:0/1781537139 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f054c04b8e0 con 0x7f05500c3590 2023-11-06T20:59:05.410 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:05.411+0000 7f28aeca0680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:59:05.520 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:05.520+0000 7fcc95f5a680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:05.555 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:05.556+0000 7f28aeca0680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.603+0000 7f055559a700 1 -- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f053405f270 msgr2=0x7f0534061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.603+0000 7f055559a700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f053405f270 0x7f0534061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:05.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.607+0000 7f0557fff700 1 -- 172.21.15.33:0/1781537139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05500c6050 con 0x7f05500c3590 2023-11-06T20:59:05.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.607+0000 7f0545ffb700 1 -- 172.21.15.33:0/1781537139 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f054c04ef00 con 0x7f05500c3590 2023-11-06T20:59:05.607 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:59:05.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 -- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f053405f270 msgr2=0x7f0534061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:05.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f053405f270 0x7f0534061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 -- 172.21.15.33:0/1781537139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05500c3590 msgr2=0x7f05500c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05500c3590 0x7f05500c7060 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f054c00e960 tx=0x7f054c009060).stop 2023-11-06T20:59:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 -- 172.21.15.33:0/1781537139 shutdown_connections 2023-11-06T20:59:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f053405f270 0x7f0534061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05500a6480 0x7f05500c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05500c3590 0x7f05500c7060 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 --2- 172.21.15.33:0/1781537139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0550154ff0 0x7f05500c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 -- 172.21.15.33:0/1781537139 >> 172.21.15.33:0/1781537139 conn(0x7f0550064250 msgr2=0x7f055015a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 -- 172.21.15.33:0/1781537139 shutdown_connections 2023-11-06T20:59:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.610+0000 7f0557fff700 1 -- 172.21.15.33:0/1781537139 wait complete. 2023-11-06T20:59:05.613 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:05.614+0000 7fcc95f5a680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:59:05.627 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T20:59:05.698 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:05.698+0000 7fcc95f5a680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:59:05.748 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:05.749+0000 7f28aeca0680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:05.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.830+0000 7f21689ff700 1 Processor -- start 2023-11-06T20:59:05.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.831+0000 7f21689ff700 1 -- start start 2023-11-06T20:59:05.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.832+0000 7f21689ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f216414e490 0x7f21640c17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.832+0000 7f21689ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f21640c1ce0 0x7f2164064550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.832+0000 7f21689ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21640c2120 con 0x7f216414e0b0 2023-11-06T20:59:05.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.832+0000 7f21689ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21640bcc90 con 0x7f21640c1ce0 2023-11-06T20:59:05.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.832+0000 7f21689ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21640bcf70 con 0x7f216414e490 2023-11-06T20:59:05.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.832+0000 7f2162d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f21640c1ce0 0x7f2164064550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:05.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.832+0000 7f2162d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f21640c1ce0 0x7f2164064550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50376/0 (socket says 172.21.15.33:50376) 2023-11-06T20:59:05.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.832+0000 7f2162d9d700 1 -- 172.21.15.33:0/4221648414 learned_addr learned my addr 172.21.15.33:0/4221648414 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:05.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.832+0000 7f2161d9b700 1 --2- 172.21.15.33:0/4221648414 >> v2:172.21.15.69:3300/0 conn(0x7f216414e490 0x7f21640c17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:05.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.833+0000 7f2161d9b700 1 -- 172.21.15.33:0/4221648414 >> v2:172.21.15.33:3300/0 conn(0x7f21640c1ce0 msgr2=0x7f2164064550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:05.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.833+0000 7f2161d9b700 1 --2- 172.21.15.33:0/4221648414 >> v2:172.21.15.33:3300/0 conn(0x7f21640c1ce0 0x7f2164064550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.833+0000 7f2161d9b700 1 -- 172.21.15.33:0/4221648414 >> v1:172.21.15.69:6789/0 conn(0x7f216414e0b0 legacy=0x7f21640bc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:05.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.833+0000 7f2161d9b700 1 -- 172.21.15.33:0/4221648414 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21640bd250 con 0x7f216414e490 2023-11-06T20:59:05.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.833+0000 7f216159a700 1 -- 172.21.15.33:0/4221648414 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253032600 0 0) 0x7f21640c2120 con 0x7f216414e0b0 2023-11-06T20:59:05.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.833+0000 7f2162d9d700 1 --2- 172.21.15.33:0/4221648414 >> v2:172.21.15.33:3300/0 conn(0x7f21640c1ce0 0x7f2164064550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:05.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.833+0000 7f2161d9b700 1 --2- 172.21.15.33:0/4221648414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f216414e490 0x7f21640c17f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f21640c3420 tx=0x7f2158009060).ready entity=mon.1 client_cookie=a8d6a674253c33e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:05.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.833+0000 7f216159a700 1 -- 172.21.15.33:0/4221648414 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215800f510 con 0x7f216414e490 2023-11-06T20:59:05.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.833+0000 7f216159a700 1 -- 172.21.15.33:0/4221648414 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215800f670 con 0x7f216414e490 2023-11-06T20:59:05.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.833+0000 7f216159a700 1 -- 172.21.15.33:0/4221648414 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215800f9f0 con 0x7f216414e490 2023-11-06T20:59:05.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.834+0000 7f21689ff700 1 -- 172.21.15.33:0/4221648414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f216414e490 msgr2=0x7f21640c17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:05.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.834+0000 7f21689ff700 1 --2- 172.21.15.33:0/4221648414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f216414e490 0x7f21640c17f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f21640c3420 tx=0x7f2158009060).stop 2023-11-06T20:59:05.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.834+0000 7f21689ff700 1 -- 172.21.15.33:0/4221648414 shutdown_connections 2023-11-06T20:59:05.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.834+0000 7f21689ff700 1 --2- 172.21.15.33:0/4221648414 >> v2:172.21.15.33:3300/0 conn(0x7f21640c1ce0 0x7f2164064550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.834+0000 7f21689ff700 1 --2- 172.21.15.33:0/4221648414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f216414e490 0x7f21640c17f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.834+0000 7f21689ff700 1 -- 172.21.15.33:0/4221648414 >> 172.21.15.33:0/4221648414 conn(0x7f2164062710 msgr2=0x7f2164062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.834+0000 7f21689ff700 1 -- 172.21.15.33:0/4221648414 shutdown_connections 2023-11-06T20:59:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.834+0000 7f21689ff700 1 -- 172.21.15.33:0/4221648414 wait complete. 2023-11-06T20:59:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.836+0000 7f21689ff700 1 Processor -- start 2023-11-06T20:59:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.836+0000 7f21689ff700 1 -- start start 2023-11-06T20:59:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f21689ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21640c1ce0 0x7f21640c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f21689ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f216414e0b0 0x7f216414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f21689ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f216414e490 0x7f21641476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f21689ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216412f160 con 0x7f216414e0b0 2023-11-06T20:59:05.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f21689ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21640aa9a0 con 0x7f21640c1ce0 2023-11-06T20:59:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f21689ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2164143a60 con 0x7f216414e490 2023-11-06T20:59:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f2161d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f216414e0b0 0x7f216414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f2161d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f216414e0b0 0x7f216414fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50382/0 (socket says 172.21.15.33:50382) 2023-11-06T20:59:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f2161d9b700 1 -- 172.21.15.33:0/1547350424 learned_addr learned my addr 172.21.15.33:0/1547350424 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f216259c700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21640c1ce0 0x7f21640c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:05.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f2162d9d700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f216414e490 0x7f21641476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:05.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f2161d9b700 1 -- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f216414e490 msgr2=0x7f21641476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:05.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f2161d9b700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f216414e490 0x7f21641476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f2161d9b700 1 -- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21640c1ce0 msgr2=0x7f21640c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:05.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f2161d9b700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21640c1ce0 0x7f21640c4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:05.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.837+0000 7f2161d9b700 1 -- 172.21.15.33:0/1547350424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c009ae0 con 0x7f216414e0b0 2023-11-06T20:59:05.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.838+0000 7f2161d9b700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f216414e0b0 0x7f216414fe30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f2158009dd0 tx=0x7f2158017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:05.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.838+0000 7f21527fc700 1 -- 172.21.15.33:0/1547350424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215800f7e0 con 0x7f216414e0b0 2023-11-06T20:59:05.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.838+0000 7f21527fc700 1 -- 172.21.15.33:0/1547350424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215800f940 con 0x7f216414e0b0 2023-11-06T20:59:05.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.838+0000 7f21527fc700 1 -- 172.21.15.33:0/1547350424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215801b620 con 0x7f216414e0b0 2023-11-06T20:59:05.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.838+0000 7f21689ff700 1 -- 172.21.15.33:0/1547350424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21580092a0 con 0x7f216414e0b0 2023-11-06T20:59:05.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.838+0000 7f21689ff700 1 -- 172.21.15.33:0/1547350424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21640aa070 con 0x7f216414e0b0 2023-11-06T20:59:05.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.840+0000 7f21689ff700 1 -- 172.21.15.33:0/1547350424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2164151ab0 con 0x7f216414e0b0 2023-11-06T20:59:05.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.840+0000 7f21527fc700 1 -- 172.21.15.33:0/1547350424 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f215801b9d0 con 0x7f216414e0b0 2023-11-06T20:59:05.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.841+0000 7f21527fc700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f214405f2a0 0x7f2144061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:05.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.841+0000 7f21527fc700 1 -- 172.21.15.33:0/1547350424 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6553+0+0 (secure 0 0 0) 0x7f215807be20 con 0x7f216414e0b0 2023-11-06T20:59:05.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.841+0000 7f216259c700 1 -- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f214405f2a0 msgr2=0x7f2144061760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:05.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.841+0000 7f216259c700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f214405f2a0 0x7f2144061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:05.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:05.845+0000 7f21527fc700 1 -- 172.21.15.33:0/1547350424 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f215804cd00 con 0x7f216414e0b0 2023-11-06T20:59:05.861 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:05.862+0000 7f28aeca0680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:59:05.964 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:05.965+0000 7f28aeca0680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:59:06.006 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:06.006+0000 7fcc95f5a680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:59:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.042+0000 7f216259c700 1 -- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f214405f2a0 msgr2=0x7f2144061760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.042+0000 7f216259c700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f214405f2a0 0x7f2144061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.054+0000 7f21689ff700 1 -- 172.21.15.33:0/1547350424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f21640bbd30 con 0x7f216414e0b0 2023-11-06T20:59:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.054+0000 7f21527fc700 1 -- 172.21.15.33:0/1547350424 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2158029020 con 0x7f216414e0b0 2023-11-06T20:59:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:59:06.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.057+0000 7f21689ff700 1 -- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f214405f2a0 msgr2=0x7f2144061760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:06.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.057+0000 7f21689ff700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f214405f2a0 0x7f2144061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:06.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.057+0000 7f21689ff700 1 -- 172.21.15.33:0/1547350424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f216414e0b0 msgr2=0x7f216414fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.057+0000 7f21689ff700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f216414e0b0 0x7f216414fe30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f2158009dd0 tx=0x7f2158017000).stop 2023-11-06T20:59:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.057+0000 7f21689ff700 1 -- 172.21.15.33:0/1547350424 shutdown_connections 2023-11-06T20:59:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.057+0000 7f21689ff700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f214405f2a0 0x7f2144061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.057+0000 7f21689ff700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f216414e490 0x7f21641476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.057+0000 7f21689ff700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21640c1ce0 0x7f21640c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.057+0000 7f21689ff700 1 --2- 172.21.15.33:0/1547350424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f216414e0b0 0x7f216414fe30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.057+0000 7f21689ff700 1 -- 172.21.15.33:0/1547350424 >> 172.21.15.33:0/1547350424 conn(0x7f2164062710 msgr2=0x7f216414cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:06.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.058+0000 7f21689ff700 1 -- 172.21.15.33:0/1547350424 shutdown_connections 2023-11-06T20:59:06.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.058+0000 7f21689ff700 1 -- 172.21.15.33:0/1547350424 wait complete. 2023-11-06T20:59:06.083 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:06.228 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:06.228+0000 7fcc95f5a680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:06.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.292+0000 7f28653ff700 1 Processor -- start 2023-11-06T20:59:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.292+0000 7f28653ff700 1 -- start start 2023-11-06T20:59:06.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.293+0000 7f28653ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2860154140 0x7f2860064c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.293+0000 7f28653ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f28600c38b0 0x7f2860158530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f28653ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28600a6390 con 0x7f28600c3400 2023-11-06T20:59:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f28653ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28600bc110 con 0x7f2860154140 2023-11-06T20:59:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f28653ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28600bc3f0 con 0x7f28600c38b0 2023-11-06T20:59:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f285effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2860154140 0x7f2860064c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f285e7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f28600c3400 0x7f2860157e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44406/0 (socket says 172.21.15.33:44406) 2023-11-06T20:59:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f285e7fc700 1 -- 172.21.15.33:0/1335461312 learned_addr learned my addr 172.21.15.33:0/1335461312 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:06.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f285f7fe700 1 --2- 172.21.15.33:0/1335461312 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f28600c38b0 0x7f2860158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:06.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f285dffb700 1 -- 172.21.15.33:0/1335461312 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721059219 0 0) 0x7f2854001110 con 0x7f28600c3400 2023-11-06T20:59:06.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f285effd700 1 -- 172.21.15.33:0/1335461312 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f28600c38b0 msgr2=0x7f2860158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:06.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f285effd700 1 --2- 172.21.15.33:0/1335461312 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f28600c38b0 0x7f2860158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f285effd700 1 -- 172.21.15.33:0/1335461312 >> v1:172.21.15.33:6789/0 conn(0x7f28600c3400 legacy=0x7f2860157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.294+0000 7f285effd700 1 -- 172.21.15.33:0/1335461312 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28600bc6d0 con 0x7f2860154140 2023-11-06T20:59:06.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.295+0000 7f285effd700 1 --2- 172.21.15.33:0/1335461312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860154140 0x7f2860064c80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f2850009760 tx=0x7f2850009060).ready entity=mon.0 client_cookie=331ec878877c5411 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:06.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.295+0000 7f285dffb700 1 -- 172.21.15.33:0/1335461312 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285000f510 con 0x7f2860154140 2023-11-06T20:59:06.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.295+0000 7f285dffb700 1 -- 172.21.15.33:0/1335461312 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285000f670 con 0x7f2860154140 2023-11-06T20:59:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.295+0000 7f285dffb700 1 -- 172.21.15.33:0/1335461312 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285000f9f0 con 0x7f2860154140 2023-11-06T20:59:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.295+0000 7f28653ff700 1 -- 172.21.15.33:0/1335461312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860154140 msgr2=0x7f2860064c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.295+0000 7f28653ff700 1 --2- 172.21.15.33:0/1335461312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860154140 0x7f2860064c80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f2850009760 tx=0x7f2850009060).stop 2023-11-06T20:59:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.296+0000 7f28653ff700 1 -- 172.21.15.33:0/1335461312 shutdown_connections 2023-11-06T20:59:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.296+0000 7f28653ff700 1 --2- 172.21.15.33:0/1335461312 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f28600c38b0 0x7f2860158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.296+0000 7f28653ff700 1 --2- 172.21.15.33:0/1335461312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2860154140 0x7f2860064c80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.296+0000 7f28653ff700 1 -- 172.21.15.33:0/1335461312 >> 172.21.15.33:0/1335461312 conn(0x7f2860062710 msgr2=0x7f2860062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.296+0000 7f28653ff700 1 -- 172.21.15.33:0/1335461312 shutdown_connections 2023-11-06T20:59:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.296+0000 7f28653ff700 1 -- 172.21.15.33:0/1335461312 wait complete. 2023-11-06T20:59:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.297+0000 7f28653ff700 1 Processor -- start 2023-11-06T20:59:06.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.297+0000 7f28653ff700 1 -- start start 2023-11-06T20:59:06.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.298+0000 7f28653ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28600c3400 0x7f286013e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.298+0000 7f28653ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28600c38b0 0x7f28600a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.298+0000 7f28653ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2860154140 0x7f28600abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.298+0000 7f28653ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28600a77b0 con 0x7f28600c3400 2023-11-06T20:59:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.298+0000 7f28653ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28600a9b80 con 0x7f28600c38b0 2023-11-06T20:59:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.298+0000 7f28653ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28600a6f10 con 0x7f2860154140 2023-11-06T20:59:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.298+0000 7f285effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28600c3400 0x7f286013e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.298+0000 7f285effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28600c3400 0x7f286013e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50386/0 (socket says 172.21.15.33:50386) 2023-11-06T20:59:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.298+0000 7f285effd700 1 -- 172.21.15.33:0/1381958436 learned_addr learned my addr 172.21.15.33:0/1381958436 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.299+0000 7f285f7fe700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2860154140 0x7f28600abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.299+0000 7f285e7fc700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28600c38b0 0x7f28600a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.299+0000 7f285effd700 1 -- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2860154140 msgr2=0x7f28600abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.299+0000 7f285effd700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2860154140 0x7f28600abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.299+0000 7f285effd700 1 -- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28600c38b0 msgr2=0x7f28600a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.299+0000 7f285effd700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28600c38b0 0x7f28600a6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.299+0000 7f285effd700 1 -- 172.21.15.33:0/1381958436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500092a0 con 0x7f28600c3400 2023-11-06T20:59:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.299+0000 7f285f7fe700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2860154140 0x7f28600abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.299+0000 7f285e7fc700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28600c38b0 0x7f28600a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.299+0000 7f285effd700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28600c3400 0x7f286013e5b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f2850009e00 tx=0x7f2850017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.300+0000 7f2846ffd700 1 -- 172.21.15.33:0/1381958436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285000fe80 con 0x7f28600c3400 2023-11-06T20:59:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.300+0000 7f2846ffd700 1 -- 172.21.15.33:0/1381958436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285000f7e0 con 0x7f28600c3400 2023-11-06T20:59:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.300+0000 7f28653ff700 1 -- 172.21.15.33:0/1381958436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28600a71f0 con 0x7f28600c3400 2023-11-06T20:59:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.300+0000 7f2846ffd700 1 -- 172.21.15.33:0/1381958436 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285001b670 con 0x7f28600c3400 2023-11-06T20:59:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.300+0000 7f28653ff700 1 -- 172.21.15.33:0/1381958436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286014b550 con 0x7f28600c3400 2023-11-06T20:59:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.301+0000 7f28653ff700 1 -- 172.21.15.33:0/1381958436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28600a9e60 con 0x7f28600c3400 2023-11-06T20:59:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.302+0000 7f2846ffd700 1 -- 172.21.15.33:0/1381958436 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f285001ba60 con 0x7f28600c3400 2023-11-06T20:59:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.303+0000 7f2846ffd700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f284c05f270 0x7f284c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.303+0000 7f2846ffd700 1 -- 172.21.15.33:0/1381958436 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6553+0+0 (secure 0 0 0) 0x7f285007bac0 con 0x7f28600c3400 2023-11-06T20:59:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.303+0000 7f285e7fc700 1 -- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f284c05f270 msgr2=0x7f284c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.303+0000 7f285e7fc700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f284c05f270 0x7f284c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.307+0000 7f2846ffd700 1 -- 172.21.15.33:0/1381958436 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f285004cb10 con 0x7f28600c3400 2023-11-06T20:59:06.320 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:06.320+0000 7f28aeca0680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:59:06.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.504+0000 7f285e7fc700 1 -- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f284c05f270 msgr2=0x7f284c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:06.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.504+0000 7f285e7fc700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f284c05f270 0x7f284c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:06.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.532+0000 7f28653ff700 1 -- 172.21.15.33:0/1381958436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28600aca10 con 0x7f28600c3400 2023-11-06T20:59:06.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.533+0000 7f2846ffd700 1 -- 172.21.15.33:0/1381958436 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4483 (secure 0 0 0) 0x7f2850050130 con 0x7f28600c3400 2023-11-06T20:59:06.532 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:06.532 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/455721224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":455721224},{"type":"v1","addr":"172.21.15.69:6836","nonce":455721224}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5632,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2050732931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2050732931},{"type":"v1","addr":"172.21.15.33:6834","nonce":2050732931}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1579729451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1579729451},{"type":"v1","addr":"172.21.15.33:6835","nonce":1579729451}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:57:01.696412+0000","modified":"2023-11-06T20:57:04.702227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/698812717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":698812717},{"type":"v1","addr":"172.21.15.69:6837","nonce":698812717}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-06T20:59:06.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 -- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f284c05f270 msgr2=0x7f284c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:06.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f284c05f270 0x7f284c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:06.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 -- 172.21.15.33:0/1381958436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28600c3400 msgr2=0x7f286013e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28600c3400 0x7f286013e5b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f2850009e00 tx=0x7f2850017000).stop 2023-11-06T20:59:06.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 -- 172.21.15.33:0/1381958436 shutdown_connections 2023-11-06T20:59:06.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f284c05f270 0x7f284c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:06.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2860154140 0x7f28600abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28600c3400 0x7f286013e5b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 --2- 172.21.15.33:0/1381958436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28600c38b0 0x7f28600a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 -- 172.21.15.33:0/1381958436 >> 172.21.15.33:0/1381958436 conn(0x7f2860062710 msgr2=0x7f2860068160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:06.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 -- 172.21.15.33:0/1381958436 shutdown_connections 2023-11-06T20:59:06.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.535+0000 7f28653ff700 1 -- 172.21.15.33:0/1381958436 wait complete. 2023-11-06T20:59:06.537 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 22 2023-11-06T20:59:06.554 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:06.559 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:06.560+0000 7f28aeca0680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:06.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.752+0000 7f0861dff700 1 Processor -- start 2023-11-06T20:59:06.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.752+0000 7f0861dff700 1 -- start start 2023-11-06T20:59:06.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.754+0000 7f0861dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f085c154280 0x7f085c154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.754+0000 7f0861dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c0c6dc0 0x7f085c0c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.754+0000 7f0861dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f085c0c5db0 con 0x7f085c0ad6e0 2023-11-06T20:59:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.754+0000 7f0861dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f085c0c6930 con 0x7f085c154280 2023-11-06T20:59:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.754+0000 7f0861dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0aa630 con 0x7f085c0c6dc0 2023-11-06T20:59:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.754+0000 7f085bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c0c6dc0 0x7f085c0c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:06.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.754+0000 7f085b7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f085c154280 0x7f085c154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:06.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.754+0000 7f085affd700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f085c0ad6e0 0x7f085c0a9f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:48744/0 (socket says 172.21.15.33:48744) 2023-11-06T20:59:06.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.754+0000 7f085affd700 1 -- 172.21.15.33:0/3405672523 learned_addr learned my addr 172.21.15.33:0/3405672523 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:06.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.755+0000 7f085bfff700 1 -- 172.21.15.33:0/3405672523 >> v2:172.21.15.69:3300/0 conn(0x7f085c154280 msgr2=0x7f085c154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:06.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.755+0000 7f085bfff700 1 --2- 172.21.15.33:0/3405672523 >> v2:172.21.15.69:3300/0 conn(0x7f085c154280 0x7f085c154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.755+0000 7f085a7fc700 1 -- 172.21.15.33:0/3405672523 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758847823 0 0) 0x7f085c0c5db0 con 0x7f085c0ad6e0 2023-11-06T20:59:06.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.755+0000 7f085bfff700 1 -- 172.21.15.33:0/3405672523 >> v1:172.21.15.69:6789/0 conn(0x7f085c0ad6e0 legacy=0x7f085c0a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.755+0000 7f085bfff700 1 -- 172.21.15.33:0/3405672523 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c0c6000 con 0x7f085c0c6dc0 2023-11-06T20:59:06.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.755+0000 7f085b7fe700 1 --2- 172.21.15.33:0/3405672523 >> v2:172.21.15.69:3300/0 conn(0x7f085c154280 0x7f085c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:06.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.755+0000 7f085bfff700 1 --2- 172.21.15.33:0/3405672523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c0c6dc0 0x7f085c0c7470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f0854009e60 tx=0x7f0854009780).ready entity=mon.2 client_cookie=b6ee5ca3da517c3d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:06.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.755+0000 7f085a7fc700 1 -- 172.21.15.33:0/3405672523 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085400e070 con 0x7f085c0c6dc0 2023-11-06T20:59:06.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.755+0000 7f085a7fc700 1 -- 172.21.15.33:0/3405672523 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085400bb60 con 0x7f085c0c6dc0 2023-11-06T20:59:06.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.755+0000 7f085a7fc700 1 -- 172.21.15.33:0/3405672523 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854012410 con 0x7f085c0c6dc0 2023-11-06T20:59:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.756+0000 7f0861dff700 1 -- 172.21.15.33:0/3405672523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c0c6dc0 msgr2=0x7f085c0c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.756+0000 7f0861dff700 1 --2- 172.21.15.33:0/3405672523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c0c6dc0 0x7f085c0c7470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f0854009e60 tx=0x7f0854009780).stop 2023-11-06T20:59:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.756+0000 7f0861dff700 1 -- 172.21.15.33:0/3405672523 shutdown_connections 2023-11-06T20:59:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.756+0000 7f0861dff700 1 --2- 172.21.15.33:0/3405672523 >> v2:172.21.15.69:3300/0 conn(0x7f085c154280 0x7f085c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.756+0000 7f0861dff700 1 --2- 172.21.15.33:0/3405672523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c0c6dc0 0x7f085c0c7470 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.756+0000 7f0861dff700 1 -- 172.21.15.33:0/3405672523 >> 172.21.15.33:0/3405672523 conn(0x7f085c05e1b0 msgr2=0x7f085c05e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.756+0000 7f0861dff700 1 -- 172.21.15.33:0/3405672523 shutdown_connections 2023-11-06T20:59:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.756+0000 7f0861dff700 1 -- 172.21.15.33:0/3405672523 wait complete. 2023-11-06T20:59:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.757+0000 7f0861dff700 1 Processor -- start 2023-11-06T20:59:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.757+0000 7f0861dff700 1 -- start start 2023-11-06T20:59:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.758+0000 7f0861dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f085c0ad6e0 0x7f085c14ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.758+0000 7f0861dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085c0c6dc0 0x7f085c0c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.758+0000 7f0861dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c154280 0x7f085c0c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.758+0000 7f0861dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0b0cb0 con 0x7f085c0ad6e0 2023-11-06T20:59:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.758+0000 7f0861dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c158030 con 0x7f085c0c6dc0 2023-11-06T20:59:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.758+0000 7f0861dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0c31e0 con 0x7f085c154280 2023-11-06T20:59:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f085c0ad6e0 0x7f085c14ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085c0c6dc0 0x7f085c0c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c154280 0x7f085c0c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f085c0ad6e0 0x7f085c14ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50392/0 (socket says 172.21.15.33:50392) 2023-11-06T20:59:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085b7fe700 1 -- 172.21.15.33:0/3266271232 learned_addr learned my addr 172.21.15.33:0/3266271232 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085bfff700 1 -- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085c0c6dc0 msgr2=0x7f085c0c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085bfff700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085c0c6dc0 0x7f085c0c2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085bfff700 1 -- 172.21.15.33:0/3266271232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f085c0ad6e0 msgr2=0x7f085c14ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085bfff700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f085c0ad6e0 0x7f085c14ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085bfff700 1 -- 172.21.15.33:0/3266271232 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c0092a0 con 0x7f085c154280 2023-11-06T20:59:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.759+0000 7f085bfff700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c154280 0x7f085c0c2cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f085c0c8910 tx=0x7f0854009780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.760+0000 7f08437fe700 1 -- 172.21.15.33:0/3266271232 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085401d630 con 0x7f085c154280 2023-11-06T20:59:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.760+0000 7f08437fe700 1 -- 172.21.15.33:0/3266271232 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085401d790 con 0x7f085c154280 2023-11-06T20:59:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.760+0000 7f08437fe700 1 -- 172.21.15.33:0/3266271232 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085401db50 con 0x7f085c154280 2023-11-06T20:59:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.760+0000 7f0861dff700 1 -- 172.21.15.33:0/3266271232 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0854009a20 con 0x7f085c154280 2023-11-06T20:59:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.760+0000 7f0861dff700 1 -- 172.21.15.33:0/3266271232 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c0b09d0 con 0x7f085c154280 2023-11-06T20:59:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.761+0000 7f0861dff700 1 -- 172.21.15.33:0/3266271232 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085c0aa410 con 0x7f085c154280 2023-11-06T20:59:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.762+0000 7f08437fe700 1 -- 172.21.15.33:0/3266271232 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f0854003040 con 0x7f085c154280 2023-11-06T20:59:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.763+0000 7f08437fe700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084805f270 0x7f0848061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.763+0000 7f08437fe700 1 -- 172.21.15.33:0/3266271232 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6553+0+0 (secure 0 0 0) 0x7f085407c660 con 0x7f085c154280 2023-11-06T20:59:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.763+0000 7f085b7fe700 1 -- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084805f270 msgr2=0x7f0848061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.763+0000 7f085b7fe700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084805f270 0x7f0848061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.772+0000 7f08437fe700 1 -- 172.21.15.33:0/3266271232 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f085404d540 con 0x7f085c154280 2023-11-06T20:59:06.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.956+0000 7f0861dff700 1 -- 172.21.15.33:0/3266271232 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f085c062ac0 con 0x7f085c154280 2023-11-06T20:59:06.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.958+0000 7f08437fe700 1 -- 172.21.15.33:0/3266271232 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+17627 (secure 0 0 0) 0x7f0854050b60 con 0x7f085c154280 2023-11-06T20:59:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:06.958 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":69,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:58:17.249098+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:57:00.140567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T20:57:00.672828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.3","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.32","mappings":[{"from":5,"to":7}]},{"pgid":"7.3c","mappings":[{"from":5,"to":0}]},{"pgid":"8.31","mappings":[{"from":2,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/341956077":"2023-11-07T20:57:16.149070+0000","172.21.15.69:0/3137643476":"2023-11-07T20:57:16.149070+0000","172.21.15.69:0/2617344929":"2023-11-07T20:57:16.149070+0000","172.21.15.69:6833/114912":"2023-11-07T20:57:16.149070+0000","172.21.15.69:6832/114912":"2023-11-07T20:57:16.149070+0000","172.21.15.69:0/3896247200":"2023-11-07T20:57:16.149070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:06.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 -- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084805f270 msgr2=0x7f0848061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:06.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084805f270 0x7f0848061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:06.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 -- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c154280 msgr2=0x7f085c0c2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:06.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c154280 0x7f085c0c2cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f085c0c8910 tx=0x7f0854009780).stop 2023-11-06T20:59:06.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 -- 172.21.15.33:0/3266271232 shutdown_connections 2023-11-06T20:59:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f084805f270 0x7f0848061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f085c154280 0x7f085c0c2cf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f085c0ad6e0 0x7f085c14ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 --2- 172.21.15.33:0/3266271232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f085c0c6dc0 0x7f085c0c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 -- 172.21.15.33:0/3266271232 >> 172.21.15.33:0/3266271232 conn(0x7f085c05e1b0 msgr2=0x7f085c150b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 -- 172.21.15.33:0/3266271232 shutdown_connections 2023-11-06T20:59:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:06.960+0000 7f0861dff700 1 -- 172.21.15.33:0/3266271232 wait complete. 2023-11-06T20:59:06.979 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T20:59:06.979 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:07.042 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:07.042+0000 7fcc95f5a680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:59:07.132 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:07.132+0000 7fcc95f5a680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:59:07.226 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:07.226+0000 7fcc95f5a680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:59:07.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.228+0000 7f9bb7bff700 1 Processor -- start 2023-11-06T20:59:07.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.229+0000 7f9bb7bff700 1 -- start start 2023-11-06T20:59:07.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.230+0000 7f9bb7bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb00c0c50 0x7f9bb00c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.230+0000 7f9bb7bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb0154e90 con 0x7f9bb01521a0 2023-11-06T20:59:07.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.230+0000 7f9bb7bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb00639d0 con 0x7f9bb0151dc0 2023-11-06T20:59:07.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.230+0000 7f9bb7bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb0155a30 con 0x7f9bb00c0c50 2023-11-06T20:59:07.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.230+0000 7f9bb519a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9bb01521a0 0x7f9bb0155320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44414/0 (socket says 172.21.15.33:44414) 2023-11-06T20:59:07.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.230+0000 7f9bb519a700 1 -- 172.21.15.33:0/2221308173 learned_addr learned my addr 172.21.15.33:0/2221308173 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:07.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.230+0000 7f9bb619c700 1 --2- 172.21.15.33:0/2221308173 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb00c0c50 0x7f9bb00c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:07.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb4999700 1 -- 172.21.15.33:0/2221308173 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738178377 0 0) 0x7f9bb00639d0 con 0x7f9bb0151dc0 2023-11-06T20:59:07.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb4999700 1 -- 172.21.15.33:0/2221308173 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b9c0033f0 con 0x7f9bb0151dc0 2023-11-06T20:59:07.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb4999700 1 -- 172.21.15.33:0/2221308173 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985680036 0 0) 0x7f9bb0154e90 con 0x7f9bb01521a0 2023-11-06T20:59:07.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb4999700 1 -- 172.21.15.33:0/2221308173 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb00639d0 con 0x7f9bb01521a0 2023-11-06T20:59:07.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb619c700 1 -- 172.21.15.33:0/2221308173 >> v1:172.21.15.69:6789/0 conn(0x7f9bb0151dc0 legacy=0x7f9bb0063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:07.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb619c700 1 -- 172.21.15.33:0/2221308173 >> v1:172.21.15.33:6789/0 conn(0x7f9bb01521a0 legacy=0x7f9bb0155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:07.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb619c700 1 -- 172.21.15.33:0/2221308173 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb0063cb0 con 0x7f9bb00c0c50 2023-11-06T20:59:07.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb619c700 1 --2- 172.21.15.33:0/2221308173 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb00c0c50 0x7f9bb00c7850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f9ba8009e90 tx=0x7f9ba8009780).ready entity=mon.2 client_cookie=6926889f9d0ed0b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:07.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb4999700 1 -- 172.21.15.33:0/2221308173 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba800e070 con 0x7f9bb00c0c50 2023-11-06T20:59:07.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb4999700 1 -- 172.21.15.33:0/2221308173 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba800bab0 con 0x7f9bb00c0c50 2023-11-06T20:59:07.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.231+0000 7f9bb4999700 1 -- 172.21.15.33:0/2221308173 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba800be30 con 0x7f9bb00c0c50 2023-11-06T20:59:07.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.232+0000 7f9bb7bff700 1 -- 172.21.15.33:0/2221308173 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb00c0c50 msgr2=0x7f9bb00c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:07.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.232+0000 7f9bb7bff700 1 --2- 172.21.15.33:0/2221308173 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb00c0c50 0x7f9bb00c7850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f9ba8009e90 tx=0x7f9ba8009780).stop 2023-11-06T20:59:07.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.232+0000 7f9bb7bff700 1 -- 172.21.15.33:0/2221308173 shutdown_connections 2023-11-06T20:59:07.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.232+0000 7f9bb7bff700 1 --2- 172.21.15.33:0/2221308173 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb00c0c50 0x7f9bb00c7850 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.232+0000 7f9bb7bff700 1 -- 172.21.15.33:0/2221308173 >> 172.21.15.33:0/2221308173 conn(0x7f9bb005e1b0 msgr2=0x7f9bb005e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:07.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.232+0000 7f9bb7bff700 1 -- 172.21.15.33:0/2221308173 shutdown_connections 2023-11-06T20:59:07.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.232+0000 7f9bb7bff700 1 -- 172.21.15.33:0/2221308173 wait complete. 2023-11-06T20:59:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.234+0000 7f9bb7bff700 1 Processor -- start 2023-11-06T20:59:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.234+0000 7f9bb7bff700 1 -- start start 2023-11-06T20:59:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb7bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb00c0c50 0x7f9bb0144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb7bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb0151dc0 0x7f9bb00c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb7bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb01521a0 0x7f9bb00c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb7bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb0164450 con 0x7f9bb0151dc0 2023-11-06T20:59:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb7bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb00c80b0 con 0x7f9bb00c0c50 2023-11-06T20:59:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb7bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb0150ca0 con 0x7f9bb01521a0 2023-11-06T20:59:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb0151dc0 0x7f9bb00c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb0151dc0 0x7f9bb00c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50408/0 (socket says 172.21.15.33:50408) 2023-11-06T20:59:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb519a700 1 -- 172.21.15.33:0/3193973138 learned_addr learned my addr 172.21.15.33:0/3193973138 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb619c700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb01521a0 0x7f9bb00c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.235+0000 7f9bb599b700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb00c0c50 0x7f9bb0144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9bb519a700 1 -- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb01521a0 msgr2=0x7f9bb00c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9bb519a700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb01521a0 0x7f9bb00c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9bb519a700 1 -- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb00c0c50 msgr2=0x7f9bb0144b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9bb519a700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb00c0c50 0x7f9bb0144b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9bb519a700 1 -- 172.21.15.33:0/3193973138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba8009a20 con 0x7f9bb0151dc0 2023-11-06T20:59:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9bb519a700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb0151dc0 0x7f9bb00c76d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f9ba000c5c0 tx=0x7f9ba0007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9ba5ffb700 1 -- 172.21.15.33:0/3193973138 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00072e0 con 0x7f9bb0151dc0 2023-11-06T20:59:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9ba5ffb700 1 -- 172.21.15.33:0/3193973138 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba0007440 con 0x7f9bb0151dc0 2023-11-06T20:59:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9bb7bff700 1 -- 172.21.15.33:0/3193973138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb00c82b0 con 0x7f9bb0151dc0 2023-11-06T20:59:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9ba5ffb700 1 -- 172.21.15.33:0/3193973138 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00057a0 con 0x7f9bb0151dc0 2023-11-06T20:59:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.236+0000 7f9bb7bff700 1 -- 172.21.15.33:0/3193973138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb0164630 con 0x7f9bb0151dc0 2023-11-06T20:59:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.237+0000 7f9bb7bff700 1 -- 172.21.15.33:0/3193973138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb00c0a30 con 0x7f9bb0151dc0 2023-11-06T20:59:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.238+0000 7f9ba5ffb700 1 -- 172.21.15.33:0/3193973138 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f9ba0005bd0 con 0x7f9bb0151dc0 2023-11-06T20:59:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.239+0000 7f9ba5ffb700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b9405f270 0x7f9b94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.240+0000 7f9ba5ffb700 1 -- 172.21.15.33:0/3193973138 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6609+0+0 (secure 0 0 0) 0x7f9ba0079a00 con 0x7f9bb0151dc0 2023-11-06T20:59:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.240+0000 7f9bb599b700 1 -- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b9405f270 msgr2=0x7f9b94061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:07.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.240+0000 7f9bb599b700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b9405f270 0x7f9b94061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:07.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.244+0000 7f9ba5ffb700 1 -- 172.21.15.33:0/3193973138 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9ba004a960 con 0x7f9bb0151dc0 2023-11-06T20:59:07.410 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:07.410+0000 7fcc95f5a680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:59:07.440 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:07.440+0000 7f28aeca0680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:59:07.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.441+0000 7f9bb599b700 1 -- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b9405f270 msgr2=0x7f9b94061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:07.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.441+0000 7f9bb599b700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b9405f270 0x7f9b94061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:07.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.482+0000 7f9bb7bff700 1 -- 172.21.15.33:0/3193973138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bb006ab40 con 0x7f9bb0151dc0 2023-11-06T20:59:07.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.483+0000 7f9ba5ffb700 1 -- 172.21.15.33:0/3193973138 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4483 (secure 0 0 0) 0x7f9ba0018020 con 0x7f9bb0151dc0 2023-11-06T20:59:07.482 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:07.483 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/455721224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":455721224},{"type":"v1","addr":"172.21.15.69:6836","nonce":455721224}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5632,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2050732931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2050732931},{"type":"v1","addr":"172.21.15.33:6834","nonce":2050732931}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1579729451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1579729451},{"type":"v1","addr":"172.21.15.33:6835","nonce":1579729451}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:57:01.696412+0000","modified":"2023-11-06T20:57:04.702227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/698812717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":698812717},{"type":"v1","addr":"172.21.15.69:6837","nonce":698812717}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-06T20:59:07.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 -- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b9405f270 msgr2=0x7f9b94061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:07.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b9405f270 0x7f9b94061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:07.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 -- 172.21.15.33:0/3193973138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb0151dc0 msgr2=0x7f9bb00c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:07.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb0151dc0 0x7f9bb00c76d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f9ba000c5c0 tx=0x7f9ba0007c80).stop 2023-11-06T20:59:07.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 -- 172.21.15.33:0/3193973138 shutdown_connections 2023-11-06T20:59:07.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9b9405f270 0x7f9b94061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:07.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9bb01521a0 0x7f9bb00c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb00c0c50 0x7f9bb0144b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 --2- 172.21.15.33:0/3193973138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb0151dc0 0x7f9bb00c76d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 -- 172.21.15.33:0/3193973138 >> 172.21.15.33:0/3193973138 conn(0x7f9bb005e1b0 msgr2=0x7f9bb00ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 -- 172.21.15.33:0/3193973138 shutdown_connections 2023-11-06T20:59:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.485+0000 7f9bb7bff700 1 -- 172.21.15.33:0/3193973138 wait complete. 2023-11-06T20:59:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 22 2023-11-06T20:59:07.502 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:07.504 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:07.505+0000 7fcc95f5a680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:59:07.535 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:07.536+0000 7f28aeca0680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:59:07.638 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:07.639+0000 7f28aeca0680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:59:07.645 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:07.645+0000 7fcc95f5a680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:59:07.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.701+0000 7efea3bff700 1 Processor -- start 2023-11-06T20:59:07.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.701+0000 7efea3bff700 1 -- start start 2023-11-06T20:59:07.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea3bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe9c14ae80 0x7efe9c0bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea3bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efe9c0bc5c0 0x7efe9c064400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea3bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe9c0648f0 0x7efe9c154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea3bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c14b260 con 0x7efe9c0648f0 2023-11-06T20:59:07.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea3bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c151cf0 con 0x7efe9c0bc5c0 2023-11-06T20:59:07.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea3bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0c1940 con 0x7efe9c14ae80 2023-11-06T20:59:07.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea219c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe9c0648f0 0x7efe9c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:07.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea219c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe9c0648f0 0x7efe9c154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50424/0 (socket says 172.21.15.33:50424) 2023-11-06T20:59:07.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea219c700 1 -- 172.21.15.33:0/2583844113 learned_addr learned my addr 172.21.15.33:0/2583844113 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:07.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea199b700 1 --2- 172.21.15.33:0/2583844113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe9c14ae80 0x7efe9c0bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:07.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.703+0000 7efea119a700 1 --2- 172.21.15.33:0/2583844113 >> v2:172.21.15.69:3300/0 conn(0x7efe9c0bc5c0 0x7efe9c064400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:07.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.704+0000 7efea219c700 1 -- 172.21.15.33:0/2583844113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe9c14ae80 msgr2=0x7efe9c0bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:07.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.704+0000 7efea219c700 1 --2- 172.21.15.33:0/2583844113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe9c14ae80 0x7efe9c0bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.704+0000 7efea219c700 1 -- 172.21.15.33:0/2583844113 >> v2:172.21.15.69:3300/0 conn(0x7efe9c0bc5c0 msgr2=0x7efe9c064400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:07.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.704+0000 7efea219c700 1 --2- 172.21.15.33:0/2583844113 >> v2:172.21.15.69:3300/0 conn(0x7efe9c0bc5c0 0x7efe9c064400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.704+0000 7efea219c700 1 -- 172.21.15.33:0/2583844113 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0bca40 con 0x7efe9c0648f0 2023-11-06T20:59:07.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.704+0000 7efea219c700 1 --2- 172.21.15.33:0/2583844113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 0x7efe9c154280 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7efe9c067f00 tx=0x7efe94009840).ready entity=mon.0 client_cookie=ef8f2f6d04ac445f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:07.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.704+0000 7efea0999700 1 -- 172.21.15.33:0/2583844113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9400e070 con 0x7efe9c0648f0 2023-11-06T20:59:07.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.704+0000 7efea0999700 1 -- 172.21.15.33:0/2583844113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9400b970 con 0x7efe9c0648f0 2023-11-06T20:59:07.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.704+0000 7efea0999700 1 -- 172.21.15.33:0/2583844113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9400bcf0 con 0x7efe9c0648f0 2023-11-06T20:59:07.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.705+0000 7efea3bff700 1 -- 172.21.15.33:0/2583844113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 msgr2=0x7efe9c154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:07.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.705+0000 7efea3bff700 1 --2- 172.21.15.33:0/2583844113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 0x7efe9c154280 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7efe9c067f00 tx=0x7efe94009840).stop 2023-11-06T20:59:07.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.705+0000 7efea3bff700 1 -- 172.21.15.33:0/2583844113 shutdown_connections 2023-11-06T20:59:07.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.705+0000 7efea3bff700 1 --2- 172.21.15.33:0/2583844113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 0x7efe9c154280 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.705+0000 7efea3bff700 1 --2- 172.21.15.33:0/2583844113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe9c14ae80 0x7efe9c0bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.705+0000 7efea3bff700 1 --2- 172.21.15.33:0/2583844113 >> v2:172.21.15.69:3300/0 conn(0x7efe9c0bc5c0 0x7efe9c064400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.705+0000 7efea3bff700 1 -- 172.21.15.33:0/2583844113 >> 172.21.15.33:0/2583844113 conn(0x7efe9c062710 msgr2=0x7efe9c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:07.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.705+0000 7efea3bff700 1 -- 172.21.15.33:0/2583844113 shutdown_connections 2023-11-06T20:59:07.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.705+0000 7efea3bff700 1 -- 172.21.15.33:0/2583844113 wait complete. 2023-11-06T20:59:07.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.707+0000 7efea3bff700 1 Processor -- start 2023-11-06T20:59:07.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.707+0000 7efea3bff700 1 -- start start 2023-11-06T20:59:07.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea3bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 0x7efe9c0a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea3bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe9c0bc5c0 0x7efe9c149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea3bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe9c14ae80 0x7efe9c1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea3bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c151e30 con 0x7efe9c0648f0 2023-11-06T20:59:07.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea3bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c152110 con 0x7efe9c14ae80 2023-11-06T20:59:07.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea3bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0aa1f0 con 0x7efe9c0bc5c0 2023-11-06T20:59:07.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 0x7efe9c0a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:07.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 0x7efe9c0a99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50430/0 (socket says 172.21.15.33:50430) 2023-11-06T20:59:07.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea199b700 1 -- 172.21.15.33:0/1415265657 learned_addr learned my addr 172.21.15.33:0/1415265657 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:07.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea219c700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe9c14ae80 0x7efe9c1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:07.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea119a700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe9c0bc5c0 0x7efe9c149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:07.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea199b700 1 -- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe9c0bc5c0 msgr2=0x7efe9c149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:07.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea199b700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe9c0bc5c0 0x7efe9c149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea199b700 1 -- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe9c14ae80 msgr2=0x7efe9c1525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:07.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.708+0000 7efea199b700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe9c14ae80 0x7efe9c1525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.709+0000 7efea199b700 1 -- 172.21.15.33:0/1415265657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94009ae0 con 0x7efe9c0648f0 2023-11-06T20:59:07.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.709+0000 7efea219c700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe9c14ae80 0x7efe9c1525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:07.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.709+0000 7efea199b700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 0x7efe9c0a99c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7efe9800e6c0 tx=0x7efe9800a200).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:07.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.709+0000 7efe91ffb700 1 -- 172.21.15.33:0/1415265657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9800a490 con 0x7efe9c0648f0 2023-11-06T20:59:07.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.709+0000 7efe91ffb700 1 -- 172.21.15.33:0/1415265657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe98011600 con 0x7efe9c0648f0 2023-11-06T20:59:07.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.709+0000 7efe91ffb700 1 -- 172.21.15.33:0/1415265657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98011980 con 0x7efe9c0648f0 2023-11-06T20:59:07.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.709+0000 7efea3bff700 1 -- 172.21.15.33:0/1415265657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c0aa4d0 con 0x7efe9c0648f0 2023-11-06T20:59:07.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.709+0000 7efea3bff700 1 -- 172.21.15.33:0/1415265657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c0a9eb0 con 0x7efe9c0648f0 2023-11-06T20:59:07.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.711+0000 7efea3bff700 1 -- 172.21.15.33:0/1415265657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c065160 con 0x7efe9c0648f0 2023-11-06T20:59:07.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.711+0000 7efe91ffb700 1 -- 172.21.15.33:0/1415265657 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7efe98011d30 con 0x7efe9c0648f0 2023-11-06T20:59:07.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.712+0000 7efe91ffb700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efe8005f270 0x7efe80061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:07.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.712+0000 7efe91ffb700 1 -- 172.21.15.33:0/1415265657 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6609+0+0 (secure 0 0 0) 0x7efe9807a6c0 con 0x7efe9c0648f0 2023-11-06T20:59:07.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.712+0000 7efea119a700 1 -- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efe8005f270 msgr2=0x7efe80061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:07.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.712+0000 7efea119a700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efe8005f270 0x7efe80061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:07.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.716+0000 7efe91ffb700 1 -- 172.21.15.33:0/1415265657 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efe9804ef90 con 0x7efe9c0648f0 2023-11-06T20:59:07.775 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:07.775+0000 7fcc95f5a680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:07.838 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:07.839+0000 7f28aeca0680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:59:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.913+0000 7efea119a700 1 -- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efe8005f270 msgr2=0x7efe80061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.913+0000 7efea119a700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efe8005f270 0x7efe80061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:07.931 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:07.932+0000 7f28aeca0680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:59:07.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.937+0000 7efea3bff700 1 -- 172.21.15.33:0/1415265657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe9c14a700 con 0x7efe9c0648f0 2023-11-06T20:59:07.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.938+0000 7efe91ffb700 1 -- 172.21.15.33:0/1415265657 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4483 (secure 0 0 0) 0x7efe98015090 con 0x7efe9c0648f0 2023-11-06T20:59:07.937 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:07.937 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/455721224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":455721224},{"type":"v1","addr":"172.21.15.69:6836","nonce":455721224}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5632,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2050732931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2050732931},{"type":"v1","addr":"172.21.15.33:6834","nonce":2050732931}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1579729451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1579729451},{"type":"v1","addr":"172.21.15.33:6835","nonce":1579729451}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:57:01.696412+0000","modified":"2023-11-06T20:57:04.702227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/698812717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":698812717},{"type":"v1","addr":"172.21.15.69:6837","nonce":698812717}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-06T20:59:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 -- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efe8005f270 msgr2=0x7efe80061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efe8005f270 0x7efe80061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 -- 172.21.15.33:0/1415265657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 msgr2=0x7efe9c0a99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 0x7efe9c0a99c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7efe9800e6c0 tx=0x7efe9800a200).stop 2023-11-06T20:59:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 -- 172.21.15.33:0/1415265657 shutdown_connections 2023-11-06T20:59:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efe8005f270 0x7efe80061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe9c14ae80 0x7efe9c1525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe9c0648f0 0x7efe9c0a99c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 --2- 172.21.15.33:0/1415265657 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe9c0bc5c0 0x7efe9c149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 -- 172.21.15.33:0/1415265657 >> 172.21.15.33:0/1415265657 conn(0x7efe9c062710 msgr2=0x7efe9c0c2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 -- 172.21.15.33:0/1415265657 shutdown_connections 2023-11-06T20:59:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:07.940+0000 7efea3bff700 1 -- 172.21.15.33:0/1415265657 wait complete. 2023-11-06T20:59:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 22 2023-11-06T20:59:07.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:08.087 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:08.088+0000 7f28aeca0680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:59:08.118 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.118+0000 7fcc95f5a680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:59:08.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.195+0000 7f6af31ff700 1 Processor -- start 2023-11-06T20:59:08.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.195+0000 7f6af31ff700 1 -- start start 2023-11-06T20:59:08.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.197+0000 7f6af31ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6aec063140 0x7f6aec063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:08.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.197+0000 7f6af31ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6aec0c3680 0x7f6aec154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.197+0000 7f6af31ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec0c31f0 con 0x7f6aec05e1b0 2023-11-06T20:59:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.197+0000 7f6af31ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec06a0f0 con 0x7f6aec063140 2023-11-06T20:59:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.197+0000 7f6af31ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec0c2920 con 0x7f6aec0c3680 2023-11-06T20:59:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.197+0000 7f6af0f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6aec063140 0x7f6aec063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.197+0000 7f6af0f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6aec063140 0x7f6aec063520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36010/0 (socket says 172.21.15.33:36010) 2023-11-06T20:59:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.197+0000 7f6af0f9b700 1 -- 172.21.15.33:0/1661340157 learned_addr learned my addr 172.21.15.33:0/1661340157 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:08.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.197+0000 7f6af179c700 1 --2- 172.21.15.33:0/1661340157 >> v2:172.21.15.69:3300/0 conn(0x7f6aec0c3680 0x7f6aec154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.198+0000 7f6af0f9b700 1 -- 172.21.15.33:0/1661340157 >> v2:172.21.15.69:3300/0 conn(0x7f6aec0c3680 msgr2=0x7f6aec154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.198+0000 7f6af0f9b700 1 --2- 172.21.15.33:0/1661340157 >> v2:172.21.15.69:3300/0 conn(0x7f6aec0c3680 0x7f6aec154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.198+0000 7f6af0f9b700 1 -- 172.21.15.33:0/1661340157 >> v1:172.21.15.69:6789/0 conn(0x7f6aec05e1b0 legacy=0x7f6aec0c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.198+0000 7f6af0f9b700 1 -- 172.21.15.33:0/1661340157 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec05e640 con 0x7f6aec063140 2023-11-06T20:59:08.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.198+0000 7f6aeb7fe700 1 -- 172.21.15.33:0/1661340157 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540724133 0 0) 0x7f6aec0c31f0 con 0x7f6aec05e1b0 2023-11-06T20:59:08.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.198+0000 7f6af0f9b700 1 --2- 172.21.15.33:0/1661340157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec063140 0x7f6aec063520 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f6aec0c3160 tx=0x7f6ae00090f0).ready entity=mon.0 client_cookie=73a178828d595c13 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:08.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.198+0000 7f6aeb7fe700 1 -- 172.21.15.33:0/1661340157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000b520 con 0x7f6aec063140 2023-11-06T20:59:08.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.198+0000 7f6aeb7fe700 1 -- 172.21.15.33:0/1661340157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae000b680 con 0x7f6aec063140 2023-11-06T20:59:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.198+0000 7f6aeb7fe700 1 -- 172.21.15.33:0/1661340157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000ba00 con 0x7f6aec063140 2023-11-06T20:59:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.199+0000 7f6af31ff700 1 -- 172.21.15.33:0/1661340157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec063140 msgr2=0x7f6aec063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.199+0000 7f6af31ff700 1 --2- 172.21.15.33:0/1661340157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec063140 0x7f6aec063520 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f6aec0c3160 tx=0x7f6ae00090f0).stop 2023-11-06T20:59:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.199+0000 7f6af31ff700 1 -- 172.21.15.33:0/1661340157 shutdown_connections 2023-11-06T20:59:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.199+0000 7f6af31ff700 1 --2- 172.21.15.33:0/1661340157 >> v2:172.21.15.69:3300/0 conn(0x7f6aec0c3680 0x7f6aec154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.199+0000 7f6af31ff700 1 --2- 172.21.15.33:0/1661340157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec063140 0x7f6aec063520 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.199+0000 7f6af31ff700 1 -- 172.21.15.33:0/1661340157 >> 172.21.15.33:0/1661340157 conn(0x7f6aec0c7d60 msgr2=0x7f6aec0c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.199+0000 7f6af31ff700 1 -- 172.21.15.33:0/1661340157 shutdown_connections 2023-11-06T20:59:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.199+0000 7f6af31ff700 1 -- 172.21.15.33:0/1661340157 wait complete. 2023-11-06T20:59:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.201+0000 7f6af31ff700 1 Processor -- start 2023-11-06T20:59:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.201+0000 7f6af31ff700 1 -- start start 2023-11-06T20:59:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6af31ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aec063140 0x7f6aec0a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6af31ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0c3680 0x7f6aec167a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6af31ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aec0c59a0 0x7f6aec167f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6af31ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0c5e70 con 0x7f6aec0c3680 2023-11-06T20:59:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6af31ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0c6150 con 0x7f6aec063140 2023-11-06T20:59:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6af31ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0c6430 con 0x7f6aec0c59a0 2023-11-06T20:59:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6aebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0c3680 0x7f6aec167a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6af0f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aec063140 0x7f6aec0a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6af0f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aec063140 0x7f6aec0a9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:48388/0 (socket says 172.21.15.33:48388) 2023-11-06T20:59:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6af179c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aec0c59a0 0x7f6aec167f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6af0f9b700 1 -- 172.21.15.33:0/154404416 learned_addr learned my addr 172.21.15.33:0/154404416 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6aebfff700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0c3680 0x7f6aec167a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36016/0 (socket says 172.21.15.33:36016) 2023-11-06T20:59:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6aebfff700 1 -- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aec0c59a0 msgr2=0x7f6aec167f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6aebfff700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aec0c59a0 0x7f6aec167f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6aebfff700 1 -- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aec063140 msgr2=0x7f6aec0a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6aebfff700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aec063140 0x7f6aec0a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.203+0000 7f6aebfff700 1 -- 172.21.15.33:0/154404416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae0009370 con 0x7f6aec0c3680 2023-11-06T20:59:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.204+0000 7f6aebfff700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0c3680 0x7f6aec167a40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f6ad800c460 tx=0x7f6ad8007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.204+0000 7f6ae8ff9700 1 -- 172.21.15.33:0/154404416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8007020 con 0x7f6aec0c3680 2023-11-06T20:59:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.204+0000 7f6ae8ff9700 1 -- 172.21.15.33:0/154404416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad80071a0 con 0x7f6aec0c3680 2023-11-06T20:59:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.204+0000 7f6af31ff700 1 -- 172.21.15.33:0/154404416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec0c6710 con 0x7f6aec0c3680 2023-11-06T20:59:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.204+0000 7f6ae8ff9700 1 -- 172.21.15.33:0/154404416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8007560 con 0x7f6aec0c3680 2023-11-06T20:59:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.204+0000 7f6af31ff700 1 -- 172.21.15.33:0/154404416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec14af40 con 0x7f6aec0c3680 2023-11-06T20:59:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.205+0000 7f6af31ff700 1 -- 172.21.15.33:0/154404416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec064420 con 0x7f6aec0c3680 2023-11-06T20:59:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.207+0000 7f6ae8ff9700 1 -- 172.21.15.33:0/154404416 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f6ad80068b0 con 0x7f6aec0c3680 2023-11-06T20:59:08.212 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.208+0000 7fcc95f5a680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:59:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.208+0000 7f6ae8ff9700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ad005f270 0x7f6ad0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.208+0000 7f6ae8ff9700 1 -- 172.21.15.33:0/154404416 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6609+0+0 (secure 0 0 0) 0x7f6ad8079730 con 0x7f6aec0c3680 2023-11-06T20:59:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.208+0000 7f6af0f9b700 1 -- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ad005f270 msgr2=0x7f6ad0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.208+0000 7f6af0f9b700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ad005f270 0x7f6ad0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.212+0000 7f6ae8ff9700 1 -- 172.21.15.33:0/154404416 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6ad8047af0 con 0x7f6aec0c3680 2023-11-06T20:59:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.222+0000 7f6ae8ff9700 1 -- 172.21.15.33:0/154404416 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 22) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f6ad8077190 con 0x7f6aec0c3680 2023-11-06T20:59:08.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.222+0000 7f6ae8ff9700 1 -- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ad005f270 msgr2=0x7f6ad0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:08.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.222+0000 7f6ae8ff9700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ad005f270 0x7f6ad0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:08.234 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:08.235+0000 7f28aeca0680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:08.291 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.291+0000 7fcc11252700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:08.291 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.291+0000 7fcc11252700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:08.291 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.291+0000 7fcc11252700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:08.292 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.291+0000 7fcc11252700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:08.292 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.291+0000 7fcc11252700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:08.292 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.291+0000 7fcc0da4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:08.292 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.291+0000 7fcc0da4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:08.292 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.291+0000 7fcc0da4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:08.292 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.291+0000 7fcc0da4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:08.293 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:08.291+0000 7fcc0da4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.405+0000 7f6af31ff700 1 -- 172.21.15.33:0/154404416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6aec168840 con 0x7f6aec0c3680 2023-11-06T20:59:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.406+0000 7f6ae8ff9700 1 -- 172.21.15.33:0/154404416 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17940 (secure 0 0 0) 0x7f6ad8070f00 con 0x7f6aec0c3680 2023-11-06T20:59:08.405 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:08.406 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":71,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:08.212295+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:57:00.140567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T20:57:00.672828+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.3","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.32","mappings":[{"from":5,"to":7}]},{"pgid":"7.3c","mappings":[{"from":5,"to":0}]},{"pgid":"8.31","mappings":[{"from":2,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3568213424":"2023-11-07T20:59:08.212253+0000","172.21.15.69:0/1044742912":"2023-11-07T20:59:08.212253+0000","172.21.15.69:0/1144852493":"2023-11-07T20:59:08.212253+0000","172.21.15.69:0/341956077":"2023-11-07T20:57:16.149070+0000","172.21.15.69:0/3170111821":"2023-11-07T20:59:08.212253+0000","172.21.15.69:0/3137643476":"2023-11-07T20:57:16.149070+0000","172.21.15.69:0/2617344929":"2023-11-07T20:57:16.149070+0000","172.21.15.69:6833/114912":"2023-11-07T20:57:16.149070+0000","172.21.15.69:6832/114912":"2023-11-07T20:57:16.149070+0000","172.21.15.69:0/3896247200":"2023-11-07T20:57:16.149070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.408+0000 7f6af31ff700 1 -- 172.21.15.33:0/154404416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0c3680 msgr2=0x7f6aec167a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.408+0000 7f6af31ff700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0c3680 0x7f6aec167a40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f6ad800c460 tx=0x7f6ad8007d40).stop 2023-11-06T20:59:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.408+0000 7f6af31ff700 1 -- 172.21.15.33:0/154404416 shutdown_connections 2023-11-06T20:59:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.408+0000 7f6af31ff700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ad005f270 0x7f6ad0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.408+0000 7f6af31ff700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aec0c59a0 0x7f6aec167f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.408+0000 7f6af31ff700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aec063140 0x7f6aec0a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.408+0000 7f6af31ff700 1 --2- 172.21.15.33:0/154404416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0c3680 0x7f6aec167a40 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.408+0000 7f6af31ff700 1 -- 172.21.15.33:0/154404416 >> 172.21.15.33:0/154404416 conn(0x7f6aec0c7d60 msgr2=0x7f6aec153420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.408+0000 7f6af31ff700 1 -- 172.21.15.33:0/154404416 shutdown_connections 2023-11-06T20:59:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.408+0000 7f6af31ff700 1 -- 172.21.15.33:0/154404416 wait complete. 2023-11-06T20:59:08.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:59:08.428 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:59:08.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T20:59:08.602 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:08.602+0000 7f28aeca0680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:59:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.672+0000 7fdb253ff700 1 Processor -- start 2023-11-06T20:59:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.672+0000 7fdb253ff700 1 -- start start 2023-11-06T20:59:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.673+0000 7fdb253ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb200c3b20 0x7fdb20156770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.673+0000 7fdb253ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb20155d20 con 0x7fdb2014ff50 2023-11-06T20:59:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.673+0000 7fdb253ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb201562e0 con 0x7fdb2014fb70 2023-11-06T20:59:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.673+0000 7fdb253ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb20156c60 con 0x7fdb200c3b20 2023-11-06T20:59:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.673+0000 7fdb1e7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdb2014ff50 0x7fdb200c3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52066/0 (socket says 172.21.15.33:52066) 2023-11-06T20:59:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.673+0000 7fdb1e7fc700 1 -- 172.21.15.33:0/1687442891 learned_addr learned my addr 172.21.15.33:0/1687442891 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.674+0000 7fdb1f7fe700 1 --2- 172.21.15.33:0/1687442891 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb200c3b20 0x7fdb20156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.674+0000 7fdb1dffb700 1 -- 172.21.15.33:0/1687442891 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288260279 0 0) 0x7fdb20155d20 con 0x7fdb2014ff50 2023-11-06T20:59:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.674+0000 7fdb1dffb700 1 -- 172.21.15.33:0/1687442891 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb080033f0 con 0x7fdb2014ff50 2023-11-06T20:59:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.674+0000 7fdb1dffb700 1 -- 172.21.15.33:0/1687442891 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868770272 0 0) 0x7fdb201562e0 con 0x7fdb2014fb70 2023-11-06T20:59:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.674+0000 7fdb1dffb700 1 -- 172.21.15.33:0/1687442891 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb20155d20 con 0x7fdb2014fb70 2023-11-06T20:59:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.674+0000 7fdb1f7fe700 1 -- 172.21.15.33:0/1687442891 >> v1:172.21.15.69:6789/0 conn(0x7fdb2014fb70 legacy=0x7fdb20152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.674+0000 7fdb1f7fe700 1 -- 172.21.15.33:0/1687442891 >> v1:172.21.15.33:6789/0 conn(0x7fdb2014ff50 legacy=0x7fdb200c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.674+0000 7fdb1f7fe700 1 -- 172.21.15.33:0/1687442891 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb20153280 con 0x7fdb200c3b20 2023-11-06T20:59:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.674+0000 7fdb1f7fe700 1 --2- 172.21.15.33:0/1687442891 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb200c3b20 0x7fdb20156770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fdb18009e90 tx=0x7fdb18009780).ready entity=mon.2 client_cookie=c68d7f1b4cf4eaf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.675+0000 7fdb1dffb700 1 -- 172.21.15.33:0/1687442891 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1800e070 con 0x7fdb200c3b20 2023-11-06T20:59:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.675+0000 7fdb1dffb700 1 -- 172.21.15.33:0/1687442891 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1800bab0 con 0x7fdb200c3b20 2023-11-06T20:59:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.675+0000 7fdb1dffb700 1 -- 172.21.15.33:0/1687442891 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1800be30 con 0x7fdb200c3b20 2023-11-06T20:59:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.675+0000 7fdb253ff700 1 -- 172.21.15.33:0/1687442891 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb200c3b20 msgr2=0x7fdb20156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.675+0000 7fdb253ff700 1 --2- 172.21.15.33:0/1687442891 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb200c3b20 0x7fdb20156770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fdb18009e90 tx=0x7fdb18009780).stop 2023-11-06T20:59:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.675+0000 7fdb253ff700 1 -- 172.21.15.33:0/1687442891 shutdown_connections 2023-11-06T20:59:08.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.675+0000 7fdb253ff700 1 --2- 172.21.15.33:0/1687442891 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb200c3b20 0x7fdb20156770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.675+0000 7fdb253ff700 1 -- 172.21.15.33:0/1687442891 >> 172.21.15.33:0/1687442891 conn(0x7fdb20062710 msgr2=0x7fdb20062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:08.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.675+0000 7fdb253ff700 1 -- 172.21.15.33:0/1687442891 shutdown_connections 2023-11-06T20:59:08.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.676+0000 7fdb253ff700 1 -- 172.21.15.33:0/1687442891 wait complete. 2023-11-06T20:59:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.677+0000 7fdb253ff700 1 Processor -- start 2023-11-06T20:59:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.677+0000 7fdb253ff700 1 -- start start 2023-11-06T20:59:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.678+0000 7fdb253ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb200c3b20 0x7fdb20165780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.678+0000 7fdb253ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb2014fb70 0x7fdb2014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.678+0000 7fdb253ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2014ff50 0x7fdb2014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.678+0000 7fdb253ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb20165c70 con 0x7fdb2014fb70 2023-11-06T20:59:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.678+0000 7fdb253ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2014f070 con 0x7fdb200c3b20 2023-11-06T20:59:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.678+0000 7fdb253ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2014d220 con 0x7fdb2014ff50 2023-11-06T20:59:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.678+0000 7fdb1e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb2014fb70 0x7fdb2014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb1e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb2014fb70 0x7fdb2014e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36032/0 (socket says 172.21.15.33:36032) 2023-11-06T20:59:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb1e7fc700 1 -- 172.21.15.33:0/1683026133 learned_addr learned my addr 172.21.15.33:0/1683026133 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb1f7fe700 1 --2- 172.21.15.33:0/1683026133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2014ff50 0x7fdb2014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb1effd700 1 --2- 172.21.15.33:0/1683026133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb200c3b20 0x7fdb20165780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb1e7fc700 1 -- 172.21.15.33:0/1683026133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2014ff50 msgr2=0x7fdb2014eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb1e7fc700 1 --2- 172.21.15.33:0/1683026133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2014ff50 0x7fdb2014eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb1e7fc700 1 -- 172.21.15.33:0/1683026133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb200c3b20 msgr2=0x7fdb20165780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb1e7fc700 1 --2- 172.21.15.33:0/1683026133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb200c3b20 0x7fdb20165780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb1e7fc700 1 -- 172.21.15.33:0/1683026133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18009a20 con 0x7fdb2014fb70 2023-11-06T20:59:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb1e7fc700 1 --2- 172.21.15.33:0/1683026133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb2014fb70 0x7fdb2014e5f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fdb1400c490 tx=0x7fdb14007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb06ffd700 1 -- 172.21.15.33:0/1683026133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb140070d0 con 0x7fdb2014fb70 2023-11-06T20:59:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb06ffd700 1 -- 172.21.15.33:0/1683026133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb14007230 con 0x7fdb2014fb70 2023-11-06T20:59:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb06ffd700 1 -- 172.21.15.33:0/1683026133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb140075b0 con 0x7fdb2014fb70 2023-11-06T20:59:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.679+0000 7fdb253ff700 1 -- 172.21.15.33:0/1683026133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2014d500 con 0x7fdb2014fb70 2023-11-06T20:59:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.680+0000 7fdb253ff700 1 -- 172.21.15.33:0/1683026133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2014dac0 con 0x7fdb2014fb70 2023-11-06T20:59:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.681+0000 7fdb253ff700 1 -- 172.21.15.33:0/1683026133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb200653d0 con 0x7fdb2014fb70 2023-11-06T20:59:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.681+0000 7fdb06ffd700 1 -- 172.21.15.33:0/1683026133 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fdb140059b0 con 0x7fdb2014fb70 2023-11-06T20:59:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.682+0000 7fdb06ffd700 1 -- 172.21.15.33:0/1683026133 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fdb14079960 con 0x7fdb2014fb70 2023-11-06T20:59:08.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.686+0000 7fdb06ffd700 1 -- 172.21.15.33:0/1683026133 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdb1404aa70 con 0x7fdb2014fb70 2023-11-06T20:59:08.708 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:08.709+0000 7f28aeca0680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:59:08.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:08.925+0000 7fdb253ff700 1 -- 172.21.15.33:0/1683026133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdb2014dcb0 con 0x7fdb2014fb70 2023-11-06T20:59:09.217 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T20:59:09.218+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T20:59:09.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.220+0000 7fdb06ffd700 1 -- 172.21.15.33:0/1683026133 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdb1404e090 con 0x7fdb2014fb70 2023-11-06T20:59:09.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.223+0000 7fdb253ff700 1 -- 172.21.15.33:0/1683026133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb2014fb70 msgr2=0x7fdb2014e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:09.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.223+0000 7fdb253ff700 1 --2- 172.21.15.33:0/1683026133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb2014fb70 0x7fdb2014e5f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fdb1400c490 tx=0x7fdb14007d20).stop 2023-11-06T20:59:09.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.223+0000 7fdb253ff700 1 -- 172.21.15.33:0/1683026133 shutdown_connections 2023-11-06T20:59:09.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.223+0000 7fdb253ff700 1 --2- 172.21.15.33:0/1683026133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdb2014ff50 0x7fdb2014eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:09.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.223+0000 7fdb253ff700 1 --2- 172.21.15.33:0/1683026133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb200c3b20 0x7fdb20165780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:09.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.223+0000 7fdb253ff700 1 --2- 172.21.15.33:0/1683026133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb2014fb70 0x7fdb2014e5f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:09.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.223+0000 7fdb253ff700 1 -- 172.21.15.33:0/1683026133 >> 172.21.15.33:0/1683026133 conn(0x7fdb20062710 msgr2=0x7fdb2014c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:09.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.223+0000 7fdb253ff700 1 -- 172.21.15.33:0/1683026133 shutdown_connections 2023-11-06T20:59:09.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.223+0000 7fdb253ff700 1 -- 172.21.15.33:0/1683026133 wait complete. 2023-11-06T20:59:09.224 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T20:59:09.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T20:59:09.286 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T20:59:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.446+0000 7fb272fff700 1 Processor -- start 2023-11-06T20:59:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.446+0000 7fb272fff700 1 -- start start 2023-11-06T20:59:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.447+0000 7fb272fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb26c0c5110 0x7fb26c066a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.447+0000 7fb272fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb26c158780 0x7fb26c0681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.447+0000 7fb272fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c0a8330 con 0x7fb26c066620 2023-11-06T20:59:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.447+0000 7fb272fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26c0be210 con 0x7fb26c158780 2023-11-06T20:59:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.447+0000 7fb272fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26c0bdc50 con 0x7fb26c0c5110 2023-11-06T20:59:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb27159c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb26c158780 0x7fb26c0681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb270d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb26c066620 0x7fb26c067ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52078/0 (socket says 172.21.15.33:52078) 2023-11-06T20:59:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb27159c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb26c158780 0x7fb26c0681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36046/0 (socket says 172.21.15.33:36046) 2023-11-06T20:59:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb270d9b700 1 -- 172.21.15.33:0/2135860773 learned_addr learned my addr 172.21.15.33:0/2135860773 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb26bfff700 1 --2- 172.21.15.33:0/2135860773 >> v2:172.21.15.69:3300/0 conn(0x7fb26c0c5110 0x7fb26c066a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb27159c700 1 -- 172.21.15.33:0/2135860773 >> v2:172.21.15.69:3300/0 conn(0x7fb26c0c5110 msgr2=0x7fb26c066a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb27159c700 1 --2- 172.21.15.33:0/2135860773 >> v2:172.21.15.69:3300/0 conn(0x7fb26c0c5110 0x7fb26c066a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb27159c700 1 -- 172.21.15.33:0/2135860773 >> v1:172.21.15.33:6789/0 conn(0x7fb26c066620 legacy=0x7fb26c067ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb27159c700 1 -- 172.21.15.33:0/2135860773 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c0bdf30 con 0x7fb26c158780 2023-11-06T20:59:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb26bfff700 1 --2- 172.21.15.33:0/2135860773 >> v2:172.21.15.69:3300/0 conn(0x7fb26c0c5110 0x7fb26c066a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.448+0000 7fb27159c700 1 --2- 172.21.15.33:0/2135860773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c158780 0x7fb26c0681e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fb26c0c9cc0 tx=0x7fb264009780).ready entity=mon.0 client_cookie=29c2bd0439045d4a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb26b7fe700 1 -- 172.21.15.33:0/2135860773 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26400e070 con 0x7fb26c158780 2023-11-06T20:59:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb26b7fe700 1 -- 172.21.15.33:0/2135860773 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26400bc30 con 0x7fb26c158780 2023-11-06T20:59:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb26b7fe700 1 -- 172.21.15.33:0/2135860773 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264012410 con 0x7fb26c158780 2023-11-06T20:59:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb272fff700 1 -- 172.21.15.33:0/2135860773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c158780 msgr2=0x7fb26c0681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb272fff700 1 --2- 172.21.15.33:0/2135860773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c158780 0x7fb26c0681e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fb26c0c9cc0 tx=0x7fb264009780).stop 2023-11-06T20:59:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb272fff700 1 -- 172.21.15.33:0/2135860773 shutdown_connections 2023-11-06T20:59:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb272fff700 1 --2- 172.21.15.33:0/2135860773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c158780 0x7fb26c0681e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb272fff700 1 --2- 172.21.15.33:0/2135860773 >> v2:172.21.15.69:3300/0 conn(0x7fb26c0c5110 0x7fb26c066a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb272fff700 1 -- 172.21.15.33:0/2135860773 >> 172.21.15.33:0/2135860773 conn(0x7fb26c0c8c50 msgr2=0x7fb26c0c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb272fff700 1 -- 172.21.15.33:0/2135860773 shutdown_connections 2023-11-06T20:59:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.449+0000 7fb272fff700 1 -- 172.21.15.33:0/2135860773 wait complete. 2023-11-06T20:59:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.451+0000 7fb272fff700 1 Processor -- start 2023-11-06T20:59:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.451+0000 7fb272fff700 1 -- start start 2023-11-06T20:59:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.452+0000 7fb272fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c066620 0x7fb26c14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.452+0000 7fb272fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb26c0c5110 0x7fb26c0a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.452+0000 7fb272fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb26c158780 0x7fb26c0a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.452+0000 7fb272fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c14dbf0 con 0x7fb26c066620 2023-11-06T20:59:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.452+0000 7fb272fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0a7680 con 0x7fb26c158780 2023-11-06T20:59:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.452+0000 7fb272fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c156960 con 0x7fb26c0c5110 2023-11-06T20:59:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.452+0000 7fb270d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c066620 0x7fb26c14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.452+0000 7fb270d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c066620 0x7fb26c14d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36048/0 (socket says 172.21.15.33:36048) 2023-11-06T20:59:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.452+0000 7fb270d9b700 1 -- 172.21.15.33:0/1434069530 learned_addr learned my addr 172.21.15.33:0/1434069530 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb27159c700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb26c158780 0x7fb26c0a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb26bfff700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb26c0c5110 0x7fb26c0a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb270d9b700 1 -- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb26c0c5110 msgr2=0x7fb26c0a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb270d9b700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb26c0c5110 0x7fb26c0a6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb270d9b700 1 -- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb26c158780 msgr2=0x7fb26c0a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb270d9b700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb26c158780 0x7fb26c0a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb270d9b700 1 -- 172.21.15.33:0/1434069530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb264009a20 con 0x7fb26c066620 2023-11-06T20:59:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb27159c700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb26c158780 0x7fb26c0a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb26bfff700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb26c0c5110 0x7fb26c0a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:09.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb270d9b700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c066620 0x7fb26c14d0a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fb26000c410 tx=0x7fb260007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:09.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb268ff9700 1 -- 172.21.15.33:0/1434069530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2600072e0 con 0x7fb26c066620 2023-11-06T20:59:09.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb268ff9700 1 -- 172.21.15.33:0/1434069530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb260007440 con 0x7fb26c066620 2023-11-06T20:59:09.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb272fff700 1 -- 172.21.15.33:0/1434069530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c0ae0e0 con 0x7fb26c066620 2023-11-06T20:59:09.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb268ff9700 1 -- 172.21.15.33:0/1434069530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2600057a0 con 0x7fb26c066620 2023-11-06T20:59:09.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.453+0000 7fb272fff700 1 -- 172.21.15.33:0/1434069530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c0ae6a0 con 0x7fb26c066620 2023-11-06T20:59:09.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.454+0000 7fb272fff700 1 -- 172.21.15.33:0/1434069530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c0aea40 con 0x7fb26c066620 2023-11-06T20:59:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.455+0000 7fb268ff9700 1 -- 172.21.15.33:0/1434069530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fb260005b10 con 0x7fb26c066620 2023-11-06T20:59:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.456+0000 7fb268ff9700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb25005f080 0x7fb250061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.456+0000 7fb268ff9700 1 -- 172.21.15.33:0/1434069530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6839+0+0 (secure 0 0 0) 0x7fb260079940 con 0x7fb26c066620 2023-11-06T20:59:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.456+0000 7fb26bfff700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb25005f080 0x7fb250061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.457+0000 7fb26bfff700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb25005f080 0x7fb250061540 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fb258006d80 tx=0x7fb258008040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.460+0000 7fb268ff9700 1 -- 172.21.15.33:0/1434069530 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb26004aa80 con 0x7fb26c066620 2023-11-06T20:59:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:09.684+0000 7fb272fff700 1 -- 172.21.15.33:0/1434069530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb26c068c00 con 0x7fb26c066620 2023-11-06T20:59:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.235+0000 7fb268ff9700 1 -- 172.21.15.33:0/1434069530 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb26004e0a0 con 0x7fb26c066620 2023-11-06T20:59:10.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 -- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb25005f080 msgr2=0x7fb250061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb25005f080 0x7fb250061540 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fb258006d80 tx=0x7fb258008040).stop 2023-11-06T20:59:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 -- 172.21.15.33:0/1434069530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c066620 msgr2=0x7fb26c14d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c066620 0x7fb26c14d0a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fb26000c410 tx=0x7fb260007e10).stop 2023-11-06T20:59:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 -- 172.21.15.33:0/1434069530 shutdown_connections 2023-11-06T20:59:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb25005f080 0x7fb250061540 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb26c158780 0x7fb26c0a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c066620 0x7fb26c14d0a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 --2- 172.21.15.33:0/1434069530 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb26c0c5110 0x7fb26c0a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 -- 172.21.15.33:0/1434069530 >> 172.21.15.33:0/1434069530 conn(0x7fb26c0c8c50 msgr2=0x7fb26c153a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 -- 172.21.15.33:0/1434069530 shutdown_connections 2023-11-06T20:59:10.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.238+0000 7fb272fff700 1 -- 172.21.15.33:0/1434069530 wait complete. 2023-11-06T20:59:10.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T20:59:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.457+0000 7f05329ff700 1 Processor -- start 2023-11-06T20:59:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.457+0000 7f05329ff700 1 -- start start 2023-11-06T20:59:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.459+0000 7f05329ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 0x7f052c0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.459+0000 7f05329ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f052c158720 0x7f052c063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.459+0000 7f05329ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c0c0e00 con 0x7f052c154340 2023-11-06T20:59:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.459+0000 7f05329ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c158bf0 con 0x7f052c158720 2023-11-06T20:59:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.459+0000 7f05329ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c158ed0 con 0x7f052c154720 2023-11-06T20:59:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.459+0000 7f052bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f052c154340 0x7f052c158010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52086/0 (socket says 172.21.15.33:52086) 2023-11-06T20:59:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.459+0000 7f052bfff700 1 -- 172.21.15.33:0/2409067938 learned_addr learned my addr 172.21.15.33:0/2409067938 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.459+0000 7f0530f9c700 1 --2- 172.21.15.33:0/2409067938 >> v2:172.21.15.69:3300/0 conn(0x7f052c158720 0x7f052c063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.459+0000 7f052b7fe700 1 --2- 172.21.15.33:0/2409067938 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 0x7f052c0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.460+0000 7f052affd700 1 -- 172.21.15.33:0/2409067938 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065730677 0 0) 0x7f052c0c0e00 con 0x7f052c154340 2023-11-06T20:59:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.460+0000 7f052affd700 1 -- 172.21.15.33:0/2409067938 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05140033f0 con 0x7f052c154340 2023-11-06T20:59:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.460+0000 7f0530f9c700 1 -- 172.21.15.33:0/2409067938 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 msgr2=0x7f052c0aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.460+0000 7f0530f9c700 1 --2- 172.21.15.33:0/2409067938 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 0x7f052c0aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.460+0000 7f0530f9c700 1 -- 172.21.15.33:0/2409067938 >> v1:172.21.15.33:6789/0 conn(0x7f052c154340 legacy=0x7f052c158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.460+0000 7f0530f9c700 1 -- 172.21.15.33:0/2409067938 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c0c5d80 con 0x7f052c158720 2023-11-06T20:59:10.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.460+0000 7f0530f9c700 1 --2- 172.21.15.33:0/2409067938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f052c158720 0x7f052c063200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f0524009e60 tx=0x7f0524009780).ready entity=mon.1 client_cookie=d6dab006834da241 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:10.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.460+0000 7f052affd700 1 -- 172.21.15.33:0/2409067938 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052400e070 con 0x7f052c158720 2023-11-06T20:59:10.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.460+0000 7f052affd700 1 -- 172.21.15.33:0/2409067938 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052400bab0 con 0x7f052c158720 2023-11-06T20:59:10.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.460+0000 7f052affd700 1 -- 172.21.15.33:0/2409067938 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052400be30 con 0x7f052c158720 2023-11-06T20:59:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.461+0000 7f05329ff700 1 -- 172.21.15.33:0/2409067938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f052c158720 msgr2=0x7f052c063200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.461+0000 7f05329ff700 1 --2- 172.21.15.33:0/2409067938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f052c158720 0x7f052c063200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f0524009e60 tx=0x7f0524009780).stop 2023-11-06T20:59:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.461+0000 7f05329ff700 1 -- 172.21.15.33:0/2409067938 shutdown_connections 2023-11-06T20:59:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.461+0000 7f05329ff700 1 --2- 172.21.15.33:0/2409067938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f052c158720 0x7f052c063200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.461+0000 7f05329ff700 1 --2- 172.21.15.33:0/2409067938 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 0x7f052c0aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.461+0000 7f05329ff700 1 -- 172.21.15.33:0/2409067938 >> 172.21.15.33:0/2409067938 conn(0x7f052c05ec80 msgr2=0x7f052c0cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.461+0000 7f05329ff700 1 -- 172.21.15.33:0/2409067938 shutdown_connections 2023-11-06T20:59:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.461+0000 7f05329ff700 1 -- 172.21.15.33:0/2409067938 wait complete. 2023-11-06T20:59:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.463+0000 7f05329ff700 1 Processor -- start 2023-11-06T20:59:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.463+0000 7f05329ff700 1 -- start start 2023-11-06T20:59:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f05329ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f052c154340 0x7f052c156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f05329ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 0x7f052c14ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f05329ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f052c158720 0x7f052c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f05329ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0ae9a0 con 0x7f052c158720 2023-11-06T20:59:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f05329ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c156af0 con 0x7f052c154340 2023-11-06T20:59:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f05329ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c156dd0 con 0x7f052c154720 2023-11-06T20:59:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f0530f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f052c158720 0x7f052c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f0530f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f052c158720 0x7f052c0b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36058/0 (socket says 172.21.15.33:36058) 2023-11-06T20:59:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f0530f9c700 1 -- 172.21.15.33:0/1988915849 learned_addr learned my addr 172.21.15.33:0/1988915849 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f052bfff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f052c154340 0x7f052c156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f052b7fe700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 0x7f052c14ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f0530f9c700 1 -- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 msgr2=0x7f052c14ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f0530f9c700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 0x7f052c14ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f0530f9c700 1 -- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f052c154340 msgr2=0x7f052c156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f0530f9c700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f052c154340 0x7f052c156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f0530f9c700 1 -- 172.21.15.33:0/1988915849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524009a20 con 0x7f052c158720 2023-11-06T20:59:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f052bfff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f052c154340 0x7f052c156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.464+0000 7f052b7fe700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 0x7f052c14ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.465+0000 7f0530f9c700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f052c158720 0x7f052c0b0530 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f052c065a00 tx=0x7f0524009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.465+0000 7f0513fff700 1 -- 172.21.15.33:0/1988915849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052401d630 con 0x7f052c158720 2023-11-06T20:59:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.465+0000 7f0513fff700 1 -- 172.21.15.33:0/1988915849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052401d7b0 con 0x7f052c158720 2023-11-06T20:59:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.465+0000 7f05329ff700 1 -- 172.21.15.33:0/1988915849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c1570b0 con 0x7f052c158720 2023-11-06T20:59:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.465+0000 7f0513fff700 1 -- 172.21.15.33:0/1988915849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052401db70 con 0x7f052c158720 2023-11-06T20:59:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.465+0000 7f05329ff700 1 -- 172.21.15.33:0/1988915849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c14d380 con 0x7f052c158720 2023-11-06T20:59:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.466+0000 7f05329ff700 1 -- 172.21.15.33:0/1988915849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c0c6060 con 0x7f052c158720 2023-11-06T20:59:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.467+0000 7f0513fff700 1 -- 172.21.15.33:0/1988915849 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f0524003040 con 0x7f052c158720 2023-11-06T20:59:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.467+0000 7f0513fff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f051805f030 0x7f05180614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.468+0000 7f0513fff700 1 -- 172.21.15.33:0/1988915849 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6839+0+0 (secure 0 0 0) 0x7f052407c120 con 0x7f052c158720 2023-11-06T20:59:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.468+0000 7f052bfff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f051805f030 0x7f05180614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.468+0000 7f052bfff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f051805f030 0x7f05180614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f051c005750 tx=0x7f051c009040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.472+0000 7f0513fff700 1 -- 172.21.15.33:0/1988915849 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f052404d1e0 con 0x7f052c158720 2023-11-06T20:59:10.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:10.651+0000 7f05329ff700 1 -- 172.21.15.33:0/1988915849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f052c0cbcf0 con 0x7f052c158720 2023-11-06T20:59:11.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.040+0000 7f0513fff700 1 -- 172.21.15.33:0/1988915849 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f052407e120 con 0x7f052c158720 2023-11-06T20:59:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.240+0000 7f0513fff700 1 -- 172.21.15.33:0/1988915849 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0524042950 con 0x7f052c158720 2023-11-06T20:59:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 -- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f051805f030 msgr2=0x7f05180614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f051805f030 0x7f05180614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f051c005750 tx=0x7f051c009040).stop 2023-11-06T20:59:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 -- 172.21.15.33:0/1988915849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f052c158720 msgr2=0x7f052c0b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f052c158720 0x7f052c0b0530 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f052c065a00 tx=0x7f0524009780).stop 2023-11-06T20:59:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 -- 172.21.15.33:0/1988915849 shutdown_connections 2023-11-06T20:59:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f051805f030 0x7f05180614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f052c158720 0x7f052c0b0530 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f052c154340 0x7f052c156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 --2- 172.21.15.33:0/1988915849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f052c154720 0x7f052c14ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 -- 172.21.15.33:0/1988915849 >> 172.21.15.33:0/1988915849 conn(0x7f052c05ec80 msgr2=0x7f052c0c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.243+0000 7f05329ff700 1 -- 172.21.15.33:0/1988915849 shutdown_connections 2023-11-06T20:59:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.244+0000 7f05329ff700 1 -- 172.21.15.33:0/1988915849 wait complete. 2023-11-06T20:59:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T20:59:11.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T20:59:11.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.469+0000 7fe7a7dff700 1 Processor -- start 2023-11-06T20:59:11.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.469+0000 7fe7a7dff700 1 -- start start 2023-11-06T20:59:11.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.471+0000 7fe7a7dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7a00638b0 0x7fe7a00ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:11.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.471+0000 7fe7a7dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe7a01550d0 0x7fe7a00c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:11.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.471+0000 7fe7a7dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a01586a0 con 0x7fe7a00acfc0 2023-11-06T20:59:11.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.471+0000 7fe7a7dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00c6590 con 0x7fe7a01550d0 2023-11-06T20:59:11.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.471+0000 7fe7a7dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a0155580 con 0x7fe7a00638b0 2023-11-06T20:59:11.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.471+0000 7fe7a639c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe7a01550d0 0x7fe7a00c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:11.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.471+0000 7fe7a639c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe7a01550d0 0x7fe7a00c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:48428/0 (socket says 172.21.15.33:48428) 2023-11-06T20:59:11.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.471+0000 7fe7a639c700 1 -- 172.21.15.33:0/3517771421 learned_addr learned my addr 172.21.15.33:0/3517771421 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:11.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.471+0000 7fe7a5b9b700 1 --1- 172.21.15.33:0/3517771421 >> v1:172.21.15.69:6789/0 conn(0x7fe7a00acfc0 0x7fe7a00631a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:41324/0 (socket says 172.21.15.33:41324) 2023-11-06T20:59:11.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.471+0000 7fe7a539a700 1 --2- 172.21.15.33:0/3517771421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7a00638b0 0x7fe7a00ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:11.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.472+0000 7fe7a639c700 1 -- 172.21.15.33:0/3517771421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7a00638b0 msgr2=0x7fe7a00ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:11.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.472+0000 7fe7a639c700 1 --2- 172.21.15.33:0/3517771421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7a00638b0 0x7fe7a00ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.472+0000 7fe7a639c700 1 -- 172.21.15.33:0/3517771421 >> v1:172.21.15.69:6789/0 conn(0x7fe7a00acfc0 legacy=0x7fe7a00631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:11.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.472+0000 7fe7a639c700 1 -- 172.21.15.33:0/3517771421 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a01544c0 con 0x7fe7a01550d0 2023-11-06T20:59:11.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.472+0000 7fe7a4b99700 1 -- 172.21.15.33:0/3517771421 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705656126 0 0) 0x7fe7a01586a0 con 0x7fe7a00acfc0 2023-11-06T20:59:11.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.472+0000 7fe7a639c700 1 --2- 172.21.15.33:0/3517771421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a01550d0 0x7fe7a00c5e20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fe798009f20 tx=0x7fe798009840).ready entity=mon.1 client_cookie=663d5d99f4a48da2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:11.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.472+0000 7fe7a4b99700 1 -- 172.21.15.33:0/3517771421 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79800e070 con 0x7fe7a01550d0 2023-11-06T20:59:11.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.472+0000 7fe7a4b99700 1 -- 172.21.15.33:0/3517771421 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79800b9d0 con 0x7fe7a01550d0 2023-11-06T20:59:11.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.472+0000 7fe7a4b99700 1 -- 172.21.15.33:0/3517771421 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79800bd50 con 0x7fe7a01550d0 2023-11-06T20:59:11.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.473+0000 7fe7a7dff700 1 -- 172.21.15.33:0/3517771421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a01550d0 msgr2=0x7fe7a00c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:11.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.473+0000 7fe7a7dff700 1 --2- 172.21.15.33:0/3517771421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a01550d0 0x7fe7a00c5e20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fe798009f20 tx=0x7fe798009840).stop 2023-11-06T20:59:11.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.473+0000 7fe7a7dff700 1 -- 172.21.15.33:0/3517771421 shutdown_connections 2023-11-06T20:59:11.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.473+0000 7fe7a7dff700 1 --2- 172.21.15.33:0/3517771421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a01550d0 0x7fe7a00c5e20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.473+0000 7fe7a7dff700 1 --2- 172.21.15.33:0/3517771421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7a00638b0 0x7fe7a00ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.473+0000 7fe7a7dff700 1 -- 172.21.15.33:0/3517771421 >> 172.21.15.33:0/3517771421 conn(0x7fe7a005ec80 msgr2=0x7fe7a00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:11.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.473+0000 7fe7a7dff700 1 -- 172.21.15.33:0/3517771421 shutdown_connections 2023-11-06T20:59:11.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.473+0000 7fe7a7dff700 1 -- 172.21.15.33:0/3517771421 wait complete. 2023-11-06T20:59:11.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.474+0000 7fe7a7dff700 1 Processor -- start 2023-11-06T20:59:11.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.475+0000 7fe7a7dff700 1 -- start start 2023-11-06T20:59:11.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.475+0000 7fe7a7dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7a00638b0 0x7fe7a00b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:11.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a7dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a00acfc0 0x7fe7a00ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:11.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a7dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a01550d0 0x7fe7a00aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:11.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a7dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a0156300 con 0x7fe7a01550d0 2023-11-06T20:59:11.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a7dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01556f0 con 0x7fe7a00acfc0 2023-11-06T20:59:11.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a7dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01559d0 con 0x7fe7a00638b0 2023-11-06T20:59:11.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a639c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a01550d0 0x7fe7a00aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:11.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a5b9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7a00638b0 0x7fe7a00b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:11.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a639c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a01550d0 0x7fe7a00aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36060/0 (socket says 172.21.15.33:36060) 2023-11-06T20:59:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a639c700 1 -- 172.21.15.33:0/4141083560 learned_addr learned my addr 172.21.15.33:0/4141083560 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a539a700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a00acfc0 0x7fe7a00ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a639c700 1 -- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7a00638b0 msgr2=0x7fe7a00b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a639c700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7a00638b0 0x7fe7a00b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a639c700 1 -- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a00acfc0 msgr2=0x7fe7a00ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a639c700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a00acfc0 0x7fe7a00ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.476+0000 7fe7a639c700 1 -- 172.21.15.33:0/4141083560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798009ae0 con 0x7fe7a01550d0 2023-11-06T20:59:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.477+0000 7fe7a639c700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a01550d0 0x7fe7a00aeb10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fe79800b610 tx=0x7fe798009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.477+0000 7fe795ffb700 1 -- 172.21.15.33:0/4141083560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7980092e0 con 0x7fe7a01550d0 2023-11-06T20:59:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.477+0000 7fe795ffb700 1 -- 172.21.15.33:0/4141083560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79801e470 con 0x7fe7a01550d0 2023-11-06T20:59:11.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.477+0000 7fe7a7dff700 1 -- 172.21.15.33:0/4141083560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a0155cb0 con 0x7fe7a01550d0 2023-11-06T20:59:11.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.477+0000 7fe795ffb700 1 -- 172.21.15.33:0/4141083560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79801e830 con 0x7fe7a01550d0 2023-11-06T20:59:11.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.477+0000 7fe7a7dff700 1 -- 172.21.15.33:0/4141083560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a00b0ca0 con 0x7fe7a01550d0 2023-11-06T20:59:11.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.478+0000 7fe7a7dff700 1 -- 172.21.15.33:0/4141083560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a0156470 con 0x7fe7a01550d0 2023-11-06T20:59:11.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.479+0000 7fe795ffb700 1 -- 172.21.15.33:0/4141083560 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe798003040 con 0x7fe7a01550d0 2023-11-06T20:59:11.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.480+0000 7fe795ffb700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe78405f270 0x7fe784061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:11.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.480+0000 7fe795ffb700 1 -- 172.21.15.33:0/4141083560 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6256+0+0 (secure 0 0 0) 0x7fe79807bec0 con 0x7fe7a01550d0 2023-11-06T20:59:11.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.480+0000 7fe7a5b9b700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe78405f270 0x7fe784061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:11.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.481+0000 7fe7a5b9b700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe78405f270 0x7fe784061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fe79c0011e0 tx=0x7fe79c007bd0).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:11.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.484+0000 7fe795ffb700 1 -- 172.21.15.33:0/4141083560 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe79804cf70 con 0x7fe7a01550d0 2023-11-06T20:59:11.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.664+0000 7fe7a7dff700 1 -- 172.21.15.33:0/4141083560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe7a006c6c0 con 0x7fe7a01550d0 2023-11-06T20:59:11.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.973+0000 7fe795ffb700 1 -- 172.21.15.33:0/4141083560 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe798023030 con 0x7fe7a01550d0 2023-11-06T20:59:11.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 -- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe78405f270 msgr2=0x7fe784061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:11.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe78405f270 0x7fe784061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fe79c0011e0 tx=0x7fe79c007bd0).stop 2023-11-06T20:59:11.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 -- 172.21.15.33:0/4141083560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a01550d0 msgr2=0x7fe7a00aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:11.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a01550d0 0x7fe7a00aeb10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fe79800b610 tx=0x7fe798009840).stop 2023-11-06T20:59:11.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 -- 172.21.15.33:0/4141083560 shutdown_connections 2023-11-06T20:59:11.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe78405f270 0x7fe784061730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7a00638b0 0x7fe7a00b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a01550d0 0x7fe7a00aeb10 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 --2- 172.21.15.33:0/4141083560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a00acfc0 0x7fe7a00ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:11.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 -- 172.21.15.33:0/4141083560 >> 172.21.15.33:0/4141083560 conn(0x7fe7a005ec80 msgr2=0x7fe7a00ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:11.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 -- 172.21.15.33:0/4141083560 shutdown_connections 2023-11-06T20:59:11.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:11.977+0000 7fe7a7dff700 1 -- 172.21.15.33:0/4141083560 wait complete. 2023-11-06T20:59:11.979 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T20:59:11.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T20:59:12.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.211+0000 7f5c233ff700 1 Processor -- start 2023-11-06T20:59:12.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.211+0000 7f5c233ff700 1 -- start start 2023-11-06T20:59:12.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.212+0000 7f5c233ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5c1c157700 0x7f5c1c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.212+0000 7f5c233ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5c1c0aaf50 0x7f5c1c063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.212+0000 7f5c233ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5c1c0aa4f0 0x7f5c1c063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.212+0000 7f5c233ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c0c18f0 con 0x7f5c1c0aa4f0 2023-11-06T20:59:12.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.212+0000 7f5c233ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c063b10 con 0x7f5c1c0aaf50 2023-11-06T20:59:12.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.212+0000 7f5c233ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c158c00 con 0x7f5c1c157700 2023-11-06T20:59:12.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.212+0000 7f5c2199c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5c1c0aa4f0 0x7f5c1c063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c2199c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5c1c0aa4f0 0x7f5c1c063620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36072/0 (socket says 172.21.15.33:36072) 2023-11-06T20:59:12.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c2199c700 1 -- 172.21.15.33:0/584120523 learned_addr learned my addr 172.21.15.33:0/584120523 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:12.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c2119b700 1 --2- 172.21.15.33:0/584120523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5c1c157700 0x7f5c1c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c2099a700 1 --2- 172.21.15.33:0/584120523 >> v2:172.21.15.69:3300/0 conn(0x7f5c1c0aaf50 0x7f5c1c063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c2199c700 1 -- 172.21.15.33:0/584120523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5c1c157700 msgr2=0x7f5c1c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c2199c700 1 --2- 172.21.15.33:0/584120523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5c1c157700 0x7f5c1c0c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c2199c700 1 -- 172.21.15.33:0/584120523 >> v2:172.21.15.69:3300/0 conn(0x7f5c1c0aaf50 msgr2=0x7f5c1c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c2199c700 1 --2- 172.21.15.33:0/584120523 >> v2:172.21.15.69:3300/0 conn(0x7f5c1c0aaf50 0x7f5c1c063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c2199c700 1 -- 172.21.15.33:0/584120523 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c157ff0 con 0x7f5c1c0aa4f0 2023-11-06T20:59:12.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c2199c700 1 --2- 172.21.15.33:0/584120523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c0aa4f0 0x7f5c1c063620 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f5c1c0c8960 tx=0x7f5c14009840).ready entity=mon.0 client_cookie=fdf4c4ab296872b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:12.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c13fff700 1 -- 172.21.15.33:0/584120523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400e070 con 0x7f5c1c0aa4f0 2023-11-06T20:59:12.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c13fff700 1 -- 172.21.15.33:0/584120523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1400b930 con 0x7f5c1c0aa4f0 2023-11-06T20:59:12.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.213+0000 7f5c13fff700 1 -- 172.21.15.33:0/584120523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400bcb0 con 0x7f5c1c0aa4f0 2023-11-06T20:59:12.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.214+0000 7f5c233ff700 1 -- 172.21.15.33:0/584120523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c0aa4f0 msgr2=0x7f5c1c063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.214+0000 7f5c233ff700 1 --2- 172.21.15.33:0/584120523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c0aa4f0 0x7f5c1c063620 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f5c1c0c8960 tx=0x7f5c14009840).stop 2023-11-06T20:59:12.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.214+0000 7f5c233ff700 1 -- 172.21.15.33:0/584120523 shutdown_connections 2023-11-06T20:59:12.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.214+0000 7f5c233ff700 1 --2- 172.21.15.33:0/584120523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c0aa4f0 0x7f5c1c063620 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.214+0000 7f5c233ff700 1 --2- 172.21.15.33:0/584120523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5c1c157700 0x7f5c1c0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.214+0000 7f5c233ff700 1 --2- 172.21.15.33:0/584120523 >> v2:172.21.15.69:3300/0 conn(0x7f5c1c0aaf50 0x7f5c1c063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.214+0000 7f5c233ff700 1 -- 172.21.15.33:0/584120523 >> 172.21.15.33:0/584120523 conn(0x7f5c1c05ec80 msgr2=0x7f5c1c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:12.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.214+0000 7f5c233ff700 1 -- 172.21.15.33:0/584120523 shutdown_connections 2023-11-06T20:59:12.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.214+0000 7f5c233ff700 1 -- 172.21.15.33:0/584120523 wait complete. 2023-11-06T20:59:12.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.216+0000 7f5c233ff700 1 Processor -- start 2023-11-06T20:59:12.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.216+0000 7f5c233ff700 1 -- start start 2023-11-06T20:59:12.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c233ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5c1c0aa4f0 0x7f5c1c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c233ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c1c0aaf50 0x7f5c1c155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c233ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c157700 0x7f5c1c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c233ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0ade00 con 0x7f5c1c157700 2023-11-06T20:59:12.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c233ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0aec60 con 0x7f5c1c0aaf50 2023-11-06T20:59:12.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c233ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0ae050 con 0x7f5c1c0aa4f0 2023-11-06T20:59:12.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c2199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c157700 0x7f5c1c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c2199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c157700 0x7f5c1c155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36086/0 (socket says 172.21.15.33:36086) 2023-11-06T20:59:12.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c2199c700 1 -- 172.21.15.33:0/3062511603 learned_addr learned my addr 172.21.15.33:0/3062511603 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:12.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c2099a700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c1c0aaf50 0x7f5c1c155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c2119b700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5c1c0aa4f0 0x7f5c1c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c2199c700 1 -- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5c1c0aa4f0 msgr2=0x7f5c1c1539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c2199c700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5c1c0aa4f0 0x7f5c1c1539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c2199c700 1 -- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c1c0aaf50 msgr2=0x7f5c1c155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c2199c700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c1c0aaf50 0x7f5c1c155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.217+0000 7f5c2199c700 1 -- 172.21.15.33:0/3062511603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c14009ae0 con 0x7f5c1c157700 2023-11-06T20:59:12.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.218+0000 7f5c2199c700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c157700 0x7f5c1c155f80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f5c1c05f0e0 tx=0x7f5c14009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:12.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.218+0000 7f5c117fa700 1 -- 172.21.15.33:0/3062511603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c140092e0 con 0x7f5c1c157700 2023-11-06T20:59:12.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.218+0000 7f5c117fa700 1 -- 172.21.15.33:0/3062511603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c14013c70 con 0x7f5c1c157700 2023-11-06T20:59:12.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.218+0000 7f5c233ff700 1 -- 172.21.15.33:0/3062511603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c0ae330 con 0x7f5c1c157700 2023-11-06T20:59:12.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.218+0000 7f5c117fa700 1 -- 172.21.15.33:0/3062511603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1401d460 con 0x7f5c1c157700 2023-11-06T20:59:12.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.218+0000 7f5c233ff700 1 -- 172.21.15.33:0/3062511603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c0ae8f0 con 0x7f5c1c157700 2023-11-06T20:59:12.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.219+0000 7f5c233ff700 1 -- 172.21.15.33:0/3062511603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c0aedd0 con 0x7f5c1c157700 2023-11-06T20:59:12.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.220+0000 7f5c117fa700 1 -- 172.21.15.33:0/3062511603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5c14003040 con 0x7f5c1c157700 2023-11-06T20:59:12.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.222+0000 7f5c117fa700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5c0005f270 0x7f5c00061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.222+0000 7f5c117fa700 1 -- 172.21.15.33:0/3062511603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f5c1407b460 con 0x7f5c1c157700 2023-11-06T20:59:12.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.222+0000 7f5c2119b700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5c0005f270 0x7f5c00061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.223+0000 7f5c2119b700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5c0005f270 0x7f5c00061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f5c18009c30 tx=0x7f5c18009060).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:12.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.226+0000 7f5c117fa700 1 -- 172.21.15.33:0/3062511603 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5c1404c6d0 con 0x7f5c1c157700 2023-11-06T20:59:12.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.406+0000 7f5c233ff700 1 -- 172.21.15.33:0/3062511603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f5c1c06c6c0 con 0x7f5c1c157700 2023-11-06T20:59:12.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.410+0000 7f5c117fa700 1 -- 172.21.15.33:0/3062511603 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v8) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5c140220b0 con 0x7f5c1c157700 2023-11-06T20:59:12.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.412+0000 7f5c233ff700 1 -- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5c0005f270 msgr2=0x7f5c00061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.412+0000 7f5c233ff700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5c0005f270 0x7f5c00061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f5c18009c30 tx=0x7f5c18009060).stop 2023-11-06T20:59:12.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.412+0000 7f5c233ff700 1 -- 172.21.15.33:0/3062511603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c157700 msgr2=0x7f5c1c155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.412+0000 7f5c233ff700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c157700 0x7f5c1c155f80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f5c1c05f0e0 tx=0x7f5c14009840).stop 2023-11-06T20:59:12.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.412+0000 7f5c233ff700 1 -- 172.21.15.33:0/3062511603 shutdown_connections 2023-11-06T20:59:12.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.412+0000 7f5c233ff700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5c0005f270 0x7f5c00061730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.412+0000 7f5c233ff700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5c1c0aa4f0 0x7f5c1c1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.412+0000 7f5c233ff700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c1c157700 0x7f5c1c155f80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.412+0000 7f5c233ff700 1 --2- 172.21.15.33:0/3062511603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c1c0aaf50 0x7f5c1c155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.412+0000 7f5c233ff700 1 -- 172.21.15.33:0/3062511603 >> 172.21.15.33:0/3062511603 conn(0x7f5c1c05ec80 msgr2=0x7f5c1c0ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:12.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.413+0000 7f5c233ff700 1 -- 172.21.15.33:0/3062511603 shutdown_connections 2023-11-06T20:59:12.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.413+0000 7f5c233ff700 1 -- 172.21.15.33:0/3062511603 wait complete. 2023-11-06T20:59:12.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T20:59:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.629+0000 7f6e345ff700 1 Processor -- start 2023-11-06T20:59:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.629+0000 7f6e345ff700 1 -- start start 2023-11-06T20:59:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.631+0000 7f6e345ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6e2c157700 0x7f6e2c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.631+0000 7f6e345ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aaf50 0x7f6e2c063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.631+0000 7f6e345ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c158c00 con 0x7f6e2c0aa4f0 2023-11-06T20:59:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.631+0000 7f6e345ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e2c0c18f0 con 0x7f6e2c157700 2023-11-06T20:59:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.631+0000 7f6e345ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0c6d80 con 0x7f6e2c0aaf50 2023-11-06T20:59:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.631+0000 7f6e3239b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6e2c157700 0x7f6e2c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.631+0000 7f6e32b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6e2c0aa4f0 0x7f6e2c0646d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52088/0 (socket says 172.21.15.33:52088) 2023-11-06T20:59:12.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.631+0000 7f6e32b9c700 1 -- 172.21.15.33:0/3261583319 learned_addr learned my addr 172.21.15.33:0/3261583319 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:12.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.631+0000 7f6e31b9a700 1 --2- 172.21.15.33:0/3261583319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aaf50 0x7f6e2c063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.631+0000 7f6e3239b700 1 -- 172.21.15.33:0/3261583319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aaf50 msgr2=0x7f6e2c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e3239b700 1 --2- 172.21.15.33:0/3261583319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aaf50 0x7f6e2c063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e3239b700 1 -- 172.21.15.33:0/3261583319 >> v1:172.21.15.33:6789/0 conn(0x7f6e2c0aa4f0 legacy=0x7f6e2c0646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e3239b700 1 -- 172.21.15.33:0/3261583319 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c157ff0 con 0x7f6e2c157700 2023-11-06T20:59:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e31b9a700 1 --2- 172.21.15.33:0/3261583319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aaf50 0x7f6e2c063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e31399700 1 -- 172.21.15.33:0/3261583319 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692216973 0 0) 0x7f6e2c158c00 con 0x7f6e2c0aa4f0 2023-11-06T20:59:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e3239b700 1 --2- 172.21.15.33:0/3261583319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 0x7f6e2c0c6f00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f6e28009790 tx=0x7f6e28009060).ready entity=mon.0 client_cookie=7a8181f3aef1f77 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e31399700 1 -- 172.21.15.33:0/3261583319 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800b580 con 0x7f6e2c157700 2023-11-06T20:59:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e31399700 1 -- 172.21.15.33:0/3261583319 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2800b6e0 con 0x7f6e2c157700 2023-11-06T20:59:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e31399700 1 -- 172.21.15.33:0/3261583319 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800ba60 con 0x7f6e2c157700 2023-11-06T20:59:12.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e345ff700 1 -- 172.21.15.33:0/3261583319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 msgr2=0x7f6e2c0c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.632+0000 7f6e345ff700 1 --2- 172.21.15.33:0/3261583319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 0x7f6e2c0c6f00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f6e28009790 tx=0x7f6e28009060).stop 2023-11-06T20:59:12.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.633+0000 7f6e345ff700 1 -- 172.21.15.33:0/3261583319 shutdown_connections 2023-11-06T20:59:12.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.633+0000 7f6e345ff700 1 --2- 172.21.15.33:0/3261583319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 0x7f6e2c0c6f00 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.633+0000 7f6e345ff700 1 --2- 172.21.15.33:0/3261583319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aaf50 0x7f6e2c063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.633+0000 7f6e345ff700 1 -- 172.21.15.33:0/3261583319 >> 172.21.15.33:0/3261583319 conn(0x7f6e2c05ec80 msgr2=0x7f6e2c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:12.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.633+0000 7f6e345ff700 1 -- 172.21.15.33:0/3261583319 shutdown_connections 2023-11-06T20:59:12.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.633+0000 7f6e345ff700 1 -- 172.21.15.33:0/3261583319 wait complete. 2023-11-06T20:59:12.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.634+0000 7f6e345ff700 1 Processor -- start 2023-11-06T20:59:12.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.634+0000 7f6e345ff700 1 -- start start 2023-11-06T20:59:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.635+0000 7f6e345ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aa4f0 0x7f6e2c150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.635+0000 7f6e345ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e2c0aaf50 0x7f6e2c0c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.635+0000 7f6e345ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 0x7f6e2c0c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.635+0000 7f6e345ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0c4a20 con 0x7f6e2c157700 2023-11-06T20:59:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.635+0000 7f6e345ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c1511b0 con 0x7f6e2c0aaf50 2023-11-06T20:59:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.635+0000 7f6e345ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c165380 con 0x7f6e2c0aa4f0 2023-11-06T20:59:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.635+0000 7f6e32b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 0x7f6e2c0c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.635+0000 7f6e32b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 0x7f6e2c0c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36100/0 (socket says 172.21.15.33:36100) 2023-11-06T20:59:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.635+0000 7f6e32b9c700 1 -- 172.21.15.33:0/1835974291 learned_addr learned my addr 172.21.15.33:0/1835974291 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e3239b700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aa4f0 0x7f6e2c150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e31b9a700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e2c0aaf50 0x7f6e2c0c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e32b9c700 1 -- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aa4f0 msgr2=0x7f6e2c150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e32b9c700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aa4f0 0x7f6e2c150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e32b9c700 1 -- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e2c0aaf50 msgr2=0x7f6e2c0c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e32b9c700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e2c0aaf50 0x7f6e2c0c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e32b9c700 1 -- 172.21.15.33:0/1835974291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e280092a0 con 0x7f6e2c157700 2023-11-06T20:59:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e32b9c700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 0x7f6e2c0c4530 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f6e2400c7e0 tx=0x7f6e240014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e227fc700 1 -- 172.21.15.33:0/1835974291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24005d30 con 0x7f6e2c157700 2023-11-06T20:59:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e227fc700 1 -- 172.21.15.33:0/1835974291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e24005e90 con 0x7f6e2c157700 2023-11-06T20:59:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e227fc700 1 -- 172.21.15.33:0/1835974291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24007300 con 0x7f6e2c157700 2023-11-06T20:59:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.636+0000 7f6e345ff700 1 -- 172.21.15.33:0/1835974291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2c155580 con 0x7f6e2c157700 2023-11-06T20:59:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.637+0000 7f6e345ff700 1 -- 172.21.15.33:0/1835974291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c155b40 con 0x7f6e2c157700 2023-11-06T20:59:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.638+0000 7f6e345ff700 1 -- 172.21.15.33:0/1835974291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2c155d30 con 0x7f6e2c157700 2023-11-06T20:59:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.638+0000 7f6e227fc700 1 -- 172.21.15.33:0/1835974291 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6e240076b0 con 0x7f6e2c157700 2023-11-06T20:59:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.639+0000 7f6e227fc700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6e1005f270 0x7f6e10061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.639+0000 7f6e227fc700 1 -- 172.21.15.33:0/1835974291 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f6e24079c30 con 0x7f6e2c157700 2023-11-06T20:59:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.640+0000 7f6e3239b700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6e1005f270 0x7f6e10061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:12.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.640+0000 7f6e3239b700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6e1005f270 0x7f6e10061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f6e280096e0 tx=0x7f6e28017000).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:12.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.643+0000 7f6e227fc700 1 -- 172.21.15.33:0/1835974291 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6e2404adf0 con 0x7f6e2c157700 2023-11-06T20:59:12.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.822+0000 7f6e345ff700 1 -- 172.21.15.33:0/1835974291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6e2c0bfe50 con 0x7f6e2c157700 2023-11-06T20:59:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.826+0000 7f6e227fc700 1 -- 172.21.15.33:0/1835974291 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6e2404e410 con 0x7f6e2c157700 2023-11-06T20:59:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 -- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6e1005f270 msgr2=0x7f6e10061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6e1005f270 0x7f6e10061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f6e280096e0 tx=0x7f6e28017000).stop 2023-11-06T20:59:12.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 -- 172.21.15.33:0/1835974291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 msgr2=0x7f6e2c0c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:12.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 0x7f6e2c0c4530 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f6e2400c7e0 tx=0x7f6e240014a0).stop 2023-11-06T20:59:12.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 -- 172.21.15.33:0/1835974291 shutdown_connections 2023-11-06T20:59:12.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6e1005f270 0x7f6e10061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6e2c0aa4f0 0x7f6e2c150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c157700 0x7f6e2c0c4530 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 --2- 172.21.15.33:0/1835974291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e2c0aaf50 0x7f6e2c0c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:12.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 -- 172.21.15.33:0/1835974291 >> 172.21.15.33:0/1835974291 conn(0x7f6e2c05ec80 msgr2=0x7f6e2c064aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:12.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 -- 172.21.15.33:0/1835974291 shutdown_connections 2023-11-06T20:59:12.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:12.829+0000 7f6e345ff700 1 -- 172.21.15.33:0/1835974291 wait complete. 2023-11-06T20:59:12.847 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-11-06T20:59:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.044+0000 7f9d6efff700 1 Processor -- start 2023-11-06T20:59:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.044+0000 7f9d6efff700 1 -- start start 2023-11-06T20:59:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.046+0000 7f9d6efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d6814c640 0x7f9d680bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.046+0000 7f9d6efff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d68153470 con 0x7f9d68152ff0 2023-11-06T20:59:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.046+0000 7f9d6efff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d680643a0 con 0x7f9d680c33f0 2023-11-06T20:59:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.046+0000 7f9d6efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680bd8f0 con 0x7f9d6814c640 2023-11-06T20:59:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.046+0000 7f9d67fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9d68152ff0 0x7f9d68065160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52104/0 (socket says 172.21.15.33:52104) 2023-11-06T20:59:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.046+0000 7f9d67fff700 1 -- 172.21.15.33:0/2618595402 learned_addr learned my addr 172.21.15.33:0/2618595402 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.046+0000 7f9d6cd9b700 1 --2- 172.21.15.33:0/2618595402 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d6814c640 0x7f9d680bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d677fe700 1 -- 172.21.15.33:0/2618595402 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996061064 0 0) 0x7f9d68153470 con 0x7f9d68152ff0 2023-11-06T20:59:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d677fe700 1 -- 172.21.15.33:0/2618595402 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d580033f0 con 0x7f9d68152ff0 2023-11-06T20:59:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d677fe700 1 -- 172.21.15.33:0/2618595402 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762991734 0 0) 0x7f9d680643a0 con 0x7f9d680c33f0 2023-11-06T20:59:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d677fe700 1 -- 172.21.15.33:0/2618595402 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d68153470 con 0x7f9d680c33f0 2023-11-06T20:59:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d6cd9b700 1 -- 172.21.15.33:0/2618595402 >> v1:172.21.15.69:6789/0 conn(0x7f9d680c33f0 legacy=0x7f9d680c13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d6cd9b700 1 -- 172.21.15.33:0/2618595402 >> v1:172.21.15.33:6789/0 conn(0x7f9d68152ff0 legacy=0x7f9d68065160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d6cd9b700 1 -- 172.21.15.33:0/2618595402 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d68064680 con 0x7f9d6814c640 2023-11-06T20:59:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d6cd9b700 1 --2- 172.21.15.33:0/2618595402 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d6814c640 0x7f9d680bb990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f9d5c009790 tx=0x7f9d5c009060).ready entity=mon.2 client_cookie=f5040f3fe6fdf5f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d677fe700 1 -- 172.21.15.33:0/2618595402 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c00f510 con 0x7f9d6814c640 2023-11-06T20:59:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d677fe700 1 -- 172.21.15.33:0/2618595402 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c00f670 con 0x7f9d6814c640 2023-11-06T20:59:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.047+0000 7f9d677fe700 1 -- 172.21.15.33:0/2618595402 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c00f9f0 con 0x7f9d6814c640 2023-11-06T20:59:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.048+0000 7f9d6efff700 1 -- 172.21.15.33:0/2618595402 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d6814c640 msgr2=0x7f9d680bb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.048+0000 7f9d6efff700 1 --2- 172.21.15.33:0/2618595402 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d6814c640 0x7f9d680bb990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f9d5c009790 tx=0x7f9d5c009060).stop 2023-11-06T20:59:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.048+0000 7f9d6efff700 1 -- 172.21.15.33:0/2618595402 shutdown_connections 2023-11-06T20:59:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.048+0000 7f9d6efff700 1 --2- 172.21.15.33:0/2618595402 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d6814c640 0x7f9d680bb990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.048+0000 7f9d6efff700 1 -- 172.21.15.33:0/2618595402 >> 172.21.15.33:0/2618595402 conn(0x7f9d68062710 msgr2=0x7f9d68062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.048+0000 7f9d6efff700 1 -- 172.21.15.33:0/2618595402 shutdown_connections 2023-11-06T20:59:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.048+0000 7f9d6efff700 1 -- 172.21.15.33:0/2618595402 wait complete. 2023-11-06T20:59:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.050+0000 7f9d6efff700 1 Processor -- start 2023-11-06T20:59:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.050+0000 7f9d6efff700 1 -- start start 2023-11-06T20:59:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d6efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d680c33f0 0x7f9d68069e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d6efff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d6814c640 0x7f9d680ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d6efff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68152ff0 0x7f9d6814ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d6efff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680bfe90 con 0x7f9d68152ff0 2023-11-06T20:59:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d6efff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680aa990 con 0x7f9d6814c640 2023-11-06T20:59:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d6efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680c0120 con 0x7f9d680c33f0 2023-11-06T20:59:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d6d59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68152ff0 0x7f9d6814ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:13.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d6d59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68152ff0 0x7f9d6814ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36116/0 (socket says 172.21.15.33:36116) 2023-11-06T20:59:13.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d6d59c700 1 -- 172.21.15.33:0/3968910657 learned_addr learned my addr 172.21.15.33:0/3968910657 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:13.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d6cd9b700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d680c33f0 0x7f9d68069e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:13.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.051+0000 7f9d67fff700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d6814c640 0x7f9d680ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:13.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d6d59c700 1 -- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d680c33f0 msgr2=0x7f9d68069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:13.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d6d59c700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d680c33f0 0x7f9d68069e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:13.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d6d59c700 1 -- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d6814c640 msgr2=0x7f9d680ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:13.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d6d59c700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d6814c640 0x7f9d680ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:13.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d6d59c700 1 -- 172.21.15.33:0/3968910657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c0092a0 con 0x7f9d68152ff0 2023-11-06T20:59:13.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d6d59c700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68152ff0 0x7f9d6814ab80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9d6000c710 tx=0x7f9d6000ca20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d64ff9700 1 -- 172.21.15.33:0/3968910657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000e4a0 con 0x7f9d68152ff0 2023-11-06T20:59:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d64ff9700 1 -- 172.21.15.33:0/3968910657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6000e600 con 0x7f9d68152ff0 2023-11-06T20:59:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d64ff9700 1 -- 172.21.15.33:0/3968910657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000e980 con 0x7f9d68152ff0 2023-11-06T20:59:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d6efff700 1 -- 172.21.15.33:0/3968910657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d680c0400 con 0x7f9d68152ff0 2023-11-06T20:59:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.052+0000 7f9d6efff700 1 -- 172.21.15.33:0/3968910657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d680ac690 con 0x7f9d68152ff0 2023-11-06T20:59:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.054+0000 7f9d6efff700 1 -- 172.21.15.33:0/3968910657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d680aab90 con 0x7f9d68152ff0 2023-11-06T20:59:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.054+0000 7f9d64ff9700 1 -- 172.21.15.33:0/3968910657 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9d60012400 con 0x7f9d68152ff0 2023-11-06T20:59:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.056+0000 7f9d64ff9700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4c05f270 0x7f9d4c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.056+0000 7f9d64ff9700 1 -- 172.21.15.33:0/3968910657 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f9d60079960 con 0x7f9d68152ff0 2023-11-06T20:59:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.056+0000 7f9d6cd9b700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4c05f270 0x7f9d4c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.057+0000 7f9d6cd9b700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4c05f270 0x7f9d4c061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f9d5c009fd0 tx=0x7f9d5c017000).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.059+0000 7f9d64ff9700 1 -- 172.21.15.33:0/3968910657 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9d60048af0 con 0x7f9d68152ff0 2023-11-06T20:59:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:13.314+0000 7f9d6efff700 1 -- 172.21.15.33:0/3968910657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f9d68064960 con 0x7f9d68152ff0 2023-11-06T20:59:14.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.049+0000 7f9d64ff9700 1 -- 172.21.15.33:0/3968910657 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v540) v1 ==== 154+0+0 (secure 0 0 0) 0x7f9d680aab90 con 0x7f9d68152ff0 2023-11-06T20:59:14.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 -- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4c05f270 msgr2=0x7f9d4c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:14.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4c05f270 0x7f9d4c061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f9d5c009fd0 tx=0x7f9d5c017000).stop 2023-11-06T20:59:14.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 -- 172.21.15.33:0/3968910657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68152ff0 msgr2=0x7f9d6814ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:14.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68152ff0 0x7f9d6814ab80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9d6000c710 tx=0x7f9d6000ca20).stop 2023-11-06T20:59:14.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 -- 172.21.15.33:0/3968910657 shutdown_connections 2023-11-06T20:59:14.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4c05f270 0x7f9d4c061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:14.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d680c33f0 0x7f9d68069e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:14.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68152ff0 0x7f9d6814ab80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:14.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 --2- 172.21.15.33:0/3968910657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d6814c640 0x7f9d680ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:14.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 -- 172.21.15.33:0/3968910657 >> 172.21.15.33:0/3968910657 conn(0x7f9d68062710 msgr2=0x7f9d680a8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:14.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 -- 172.21.15.33:0/3968910657 shutdown_connections 2023-11-06T20:59:14.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.052+0000 7f9d6efff700 1 -- 172.21.15.33:0/3968910657 wait complete. 2023-11-06T20:59:14.073 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-11-06T20:59:14.073 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-11-06T20:59:14.073 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-06T20:59:14.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-11-06T20:59:14.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.274+0000 7fb6d21ff700 1 Processor -- start 2023-11-06T20:59:14.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.275+0000 7fb6d21ff700 1 -- start start 2023-11-06T20:59:14.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.276+0000 7fb6d21ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb6cc14ebb0 0x7fb6cc063660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:14.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.276+0000 7fb6d21ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6cc158e20 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.276+0000 7fb6d21ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6cc1582a0 con 0x7fb6cc063280 2023-11-06T20:59:14.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.276+0000 7fb6d21ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6cc158580 con 0x7fb6cc14ebb0 2023-11-06T20:59:14.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.276+0000 7fb6cbfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb6cc0c05e0 0x7fb6cc0c0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52116/0 (socket says 172.21.15.33:52116) 2023-11-06T20:59:14.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.276+0000 7fb6cbfff700 1 -- 172.21.15.33:0/59925308 learned_addr learned my addr 172.21.15.33:0/59925308 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:14.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.276+0000 7fb6caffd700 1 --2- 172.21.15.33:0/59925308 >> v2:172.21.15.69:3300/0 conn(0x7fb6cc14ebb0 0x7fb6cc063660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:14.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.277+0000 7fb6ca7fc700 1 -- 172.21.15.33:0/59925308 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139201141 0 0) 0x7fb6cc158e20 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.277+0000 7fb6ca7fc700 1 -- 172.21.15.33:0/59925308 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6b40033f0 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.277+0000 7fb6caffd700 1 -- 172.21.15.33:0/59925308 >> v1:172.21.15.69:6789/0 conn(0x7fb6cc063280 legacy=0x7fb6cc14e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:14.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.277+0000 7fb6caffd700 1 -- 172.21.15.33:0/59925308 >> v1:172.21.15.33:6789/0 conn(0x7fb6cc0c05e0 legacy=0x7fb6cc0c0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:14.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.277+0000 7fb6caffd700 1 -- 172.21.15.33:0/59925308 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc158860 con 0x7fb6cc14ebb0 2023-11-06T20:59:14.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.277+0000 7fb6caffd700 1 --2- 172.21.15.33:0/59925308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6cc14ebb0 0x7fb6cc063660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fb6c0009760 tx=0x7fb6c0009060).ready entity=mon.1 client_cookie=68178b5fa321db1f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:14.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.277+0000 7fb6ca7fc700 1 -- 172.21.15.33:0/59925308 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000f510 con 0x7fb6cc14ebb0 2023-11-06T20:59:14.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.278+0000 7fb6ca7fc700 1 -- 172.21.15.33:0/59925308 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c000f670 con 0x7fb6cc14ebb0 2023-11-06T20:59:14.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.278+0000 7fb6ca7fc700 1 -- 172.21.15.33:0/59925308 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000f9f0 con 0x7fb6cc14ebb0 2023-11-06T20:59:14.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.278+0000 7fb6d21ff700 1 -- 172.21.15.33:0/59925308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6cc14ebb0 msgr2=0x7fb6cc063660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:14.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.278+0000 7fb6d21ff700 1 --2- 172.21.15.33:0/59925308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6cc14ebb0 0x7fb6cc063660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fb6c0009760 tx=0x7fb6c0009060).stop 2023-11-06T20:59:14.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.278+0000 7fb6d21ff700 1 -- 172.21.15.33:0/59925308 shutdown_connections 2023-11-06T20:59:14.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.278+0000 7fb6d21ff700 1 --2- 172.21.15.33:0/59925308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6cc14ebb0 0x7fb6cc063660 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:14.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.278+0000 7fb6d21ff700 1 -- 172.21.15.33:0/59925308 >> 172.21.15.33:0/59925308 conn(0x7fb6cc0cade0 msgr2=0x7fb6cc0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:14.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.278+0000 7fb6d21ff700 1 -- 172.21.15.33:0/59925308 shutdown_connections 2023-11-06T20:59:14.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.278+0000 7fb6d21ff700 1 -- 172.21.15.33:0/59925308 wait complete. 2023-11-06T20:59:14.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.280+0000 7fb6d21ff700 1 Processor -- start 2023-11-06T20:59:14.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.280+0000 7fb6d21ff700 1 -- start start 2023-11-06T20:59:14.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6d21ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6cc063280 0x7fb6cc0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:14.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6d21ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6cc0c05e0 0x7fb6cc0abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6d21ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6cc14ebb0 0x7fb6cc0ac250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6d21ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc0ac740 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6d21ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc0aecd0 con 0x7fb6cc063280 2023-11-06T20:59:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6d21ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc0a2120 con 0x7fb6cc14ebb0 2023-11-06T20:59:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6caffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6cc0c05e0 0x7fb6cc0abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6caffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6cc0c05e0 0x7fb6cc0abd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36124/0 (socket says 172.21.15.33:36124) 2023-11-06T20:59:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6caffd700 1 -- 172.21.15.33:0/2368509930 learned_addr learned my addr 172.21.15.33:0/2368509930 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6cb7fe700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6cc063280 0x7fb6cc0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.281+0000 7fb6cbfff700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6cc14ebb0 0x7fb6cc0ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6caffd700 1 -- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6cc14ebb0 msgr2=0x7fb6cc0ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:14.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6caffd700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6cc14ebb0 0x7fb6cc0ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:14.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6caffd700 1 -- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6cc063280 msgr2=0x7fb6cc0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:14.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6caffd700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6cc063280 0x7fb6cc0b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:14.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6caffd700 1 -- 172.21.15.33:0/2368509930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c00092a0 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6cbfff700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6cc14ebb0 0x7fb6cc0ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:14.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6caffd700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6cc0c05e0 0x7fb6cc0abd60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fb6cc0c88f0 tx=0x7fb6c0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6b37fe700 1 -- 172.21.15.33:0/2368509930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000f7e0 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6b37fe700 1 -- 172.21.15.33:0/2368509930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c000f940 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6b37fe700 1 -- 172.21.15.33:0/2368509930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c001b670 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6d21ff700 1 -- 172.21.15.33:0/2368509930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6cc0a2400 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.282+0000 7fb6d21ff700 1 -- 172.21.15.33:0/2368509930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6cc0a29c0 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.283+0000 7fb6d21ff700 1 -- 172.21.15.33:0/2368509930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6cc0a2b80 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.284+0000 7fb6b37fe700 1 -- 172.21.15.33:0/2368509930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb6c001ba20 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.285+0000 7fb6b37fe700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6b805f270 0x7fb6b8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:14.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.285+0000 7fb6b37fe700 1 -- 172.21.15.33:0/2368509930 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5813+0+0 (secure 0 0 0) 0x7fb6c007b960 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.285+0000 7fb6cb7fe700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6b805f270 0x7fb6b8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.286+0000 7fb6cb7fe700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6b805f270 0x7fb6b8061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb6bc003900 tx=0x7fb6bc007d40).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.289+0000 7fb6b37fe700 1 -- 172.21.15.33:0/2368509930 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb6c004cb20 con 0x7fb6cc0c05e0 2023-11-06T20:59:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:14.541+0000 7fb6d21ff700 1 -- 172.21.15.33:0/2368509930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fb6cc0a2d30 con 0x7fb6cc0c05e0 2023-11-06T20:59:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.055+0000 7fb6b37fe700 1 -- 172.21.15.33:0/2368509930 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v541) v1 ==== 159+0+0 (secure 0 0 0) 0x7fb6c007b030 con 0x7fb6cc0c05e0 2023-11-06T20:59:15.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 -- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6b805f270 msgr2=0x7fb6b8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6b805f270 0x7fb6b8061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb6bc003900 tx=0x7fb6bc007d40).stop 2023-11-06T20:59:15.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 -- 172.21.15.33:0/2368509930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6cc0c05e0 msgr2=0x7fb6cc0abd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6cc0c05e0 0x7fb6cc0abd60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fb6cc0c88f0 tx=0x7fb6c0017000).stop 2023-11-06T20:59:15.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 -- 172.21.15.33:0/2368509930 shutdown_connections 2023-11-06T20:59:15.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6b805f270 0x7fb6b8061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6cc14ebb0 0x7fb6cc0ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6cc063280 0x7fb6cc0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 --2- 172.21.15.33:0/2368509930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6cc0c05e0 0x7fb6cc0abd60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 -- 172.21.15.33:0/2368509930 >> 172.21.15.33:0/2368509930 conn(0x7fb6cc0cade0 msgr2=0x7fb6cc0c77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:15.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 -- 172.21.15.33:0/2368509930 shutdown_connections 2023-11-06T20:59:15.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.058+0000 7fb6d21ff700 1 -- 172.21.15.33:0/2368509930 wait complete. 2023-11-06T20:59:15.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T20:59:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.279+0000 7fb016fff700 1 Processor -- start 2023-11-06T20:59:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.279+0000 7fb016fff700 1 -- start start 2023-11-06T20:59:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.280+0000 7fb016fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb010154330 0x7fb0100c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.280+0000 7fb016fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb0100c7ff0 0x7fb0100646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.280+0000 7fb016fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0100c6b60 con 0x7fb010154710 2023-11-06T20:59:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.280+0000 7fb016fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0100c7700 con 0x7fb0100c7ff0 2023-11-06T20:59:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.280+0000 7fb016fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100c5cd0 con 0x7fb010154330 2023-11-06T20:59:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb010154710 0x7fb0100c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52118/0 (socket says 172.21.15.33:52118) 2023-11-06T20:59:15.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00ffff700 1 -- 172.21.15.33:0/2505090728 learned_addr learned my addr 172.21.15.33:0/2505090728 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:15.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb014d9b700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb010154330 0x7fb0100c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb01559c700 1 --2- 172.21.15.33:0/2505090728 >> v2:172.21.15.69:3300/0 conn(0x7fb0100c7ff0 0x7fb0100646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711264162 0 0) 0x7fb0100c6b60 con 0x7fb010154710 2023-11-06T20:59:15.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0000033f0 con 0x7fb010154710 2023-11-06T20:59:15.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 848535032 0 0) 0x7fb0000033f0 con 0x7fb010154710 2023-11-06T20:59:15.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb010154330 msgr2=0x7fb0100c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:15.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00f7fe700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb010154330 0x7fb0100c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 >> v2:172.21.15.69:3300/0 conn(0x7fb0100c7ff0 msgr2=0x7fb0100646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:15.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00f7fe700 1 --2- 172.21.15.33:0/2505090728 >> v2:172.21.15.69:3300/0 conn(0x7fb0100c7ff0 0x7fb0100646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100c5f50 con 0x7fb010154710 2023-11-06T20:59:15.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.281+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7faffc001470 con 0x7fb010154710 2023-11-06T20:59:15.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 >> v1:172.21.15.33:6789/0 conn(0x7fb010154710 legacy=0x7fb0100c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb00f7fe700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0000038e0 0x7fb000003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb00f7fe700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb010154330 0x7fb000004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb00f7fe700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb000004b80 0x7fb000005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100c7700 con 0x7fb0000038e0 2023-11-06T20:59:15.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100c5cd0 con 0x7fb000004b80 2023-11-06T20:59:15.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00000b8e0 con 0x7fb010154330 2023-11-06T20:59:15.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb014d9b700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0000038e0 0x7fb000003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb01559c700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb000004b80 0x7fb000005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb00ffff700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb010154330 0x7fb000004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb014d9b700 1 -- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb010154330 msgr2=0x7fb000004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:15.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb014d9b700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb010154330 0x7fb000004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb014d9b700 1 -- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb000004b80 msgr2=0x7fb000005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:15.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb014d9b700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb000004b80 0x7fb000005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.282+0000 7fb014d9b700 1 -- 172.21.15.33:0/2505090728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0000033f0 con 0x7fb0000038e0 2023-11-06T20:59:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb014d9b700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0000038e0 0x7fb000003d00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fb004009a80 tx=0x7fb004009060).ready entity=mon.0 client_cookie=d2cbe4eb4518174a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004009430 con 0x7fb0000038e0 2023-11-06T20:59:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00400e640 con 0x7fb0000038e0 2023-11-06T20:59:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb00f7fe700 1 -- 172.21.15.33:0/2505090728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00400e9c0 con 0x7fb0000038e0 2023-11-06T20:59:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb016fff700 1 -- 172.21.15.33:0/2505090728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0000038e0 msgr2=0x7fb000003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb016fff700 1 --2- 172.21.15.33:0/2505090728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0000038e0 0x7fb000003d00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fb004009a80 tx=0x7fb004009060).stop 2023-11-06T20:59:15.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb01559c700 1 -- 172.21.15.33:0/2505090728 reap_dead start 2023-11-06T20:59:15.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb016fff700 1 -- 172.21.15.33:0/2505090728 shutdown_connections 2023-11-06T20:59:15.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb016fff700 1 -- 172.21.15.33:0/2505090728 >> 172.21.15.33:0/2505090728 conn(0x7fb0100cb910 msgr2=0x7fb0100c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:15.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb016fff700 1 -- 172.21.15.33:0/2505090728 shutdown_connections 2023-11-06T20:59:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.283+0000 7fb016fff700 1 -- 172.21.15.33:0/2505090728 wait complete. 2023-11-06T20:59:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.285+0000 7fb016fff700 1 Processor -- start 2023-11-06T20:59:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.285+0000 7fb016fff700 1 -- start start 2023-11-06T20:59:15.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.286+0000 7fb016fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0000038e0 0x7fb0100addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.286+0000 7fb016fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb010165800 0x7fb010157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.286+0000 7fb016fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0100646c0 0x7fb010064b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.286+0000 7fb016fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb010065030 con 0x7fb010165800 2023-11-06T20:59:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.286+0000 7fb016fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb010165d50 con 0x7fb0000038e0 2023-11-06T20:59:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.286+0000 7fb016fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb010166030 con 0x7fb0100646c0 2023-11-06T20:59:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.286+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb010165800 0x7fb010157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.286+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb010165800 0x7fb010157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36128/0 (socket says 172.21.15.33:36128) 2023-11-06T20:59:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.286+0000 7fb00ffff700 1 -- 172.21.15.33:0/696107926 learned_addr learned my addr 172.21.15.33:0/696107926 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.286+0000 7fb014d9b700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0000038e0 0x7fb0100addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb01559c700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0100646c0 0x7fb010064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb00ffff700 1 -- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0100646c0 msgr2=0x7fb010064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb00ffff700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0100646c0 0x7fb010064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb00ffff700 1 -- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0000038e0 msgr2=0x7fb0100addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb00ffff700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0000038e0 0x7fb0100addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb00ffff700 1 -- 172.21.15.33:0/696107926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010166310 con 0x7fb010165800 2023-11-06T20:59:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb01559c700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0100646c0 0x7fb010064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb00ffff700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb010165800 0x7fb010157740 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7faffc001050 tx=0x7faffc001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb00cff9700 1 -- 172.21.15.33:0/696107926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc011590 con 0x7fb010165800 2023-11-06T20:59:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb00cff9700 1 -- 172.21.15.33:0/696107926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc0116f0 con 0x7fb010165800 2023-11-06T20:59:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb016fff700 1 -- 172.21.15.33:0/696107926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0101665f0 con 0x7fb010165800 2023-11-06T20:59:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb00cff9700 1 -- 172.21.15.33:0/696107926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc011a70 con 0x7fb010165800 2023-11-06T20:59:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.287+0000 7fb016fff700 1 -- 172.21.15.33:0/696107926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0100ae2c0 con 0x7fb010165800 2023-11-06T20:59:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.289+0000 7fb016fff700 1 -- 172.21.15.33:0/696107926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01014d8b0 con 0x7fb010165800 2023-11-06T20:59:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.289+0000 7fb00cff9700 1 -- 172.21.15.33:0/696107926 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7faffc01d3b0 con 0x7fb010165800 2023-11-06T20:59:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.291+0000 7fb00cff9700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faff405f270 0x7faff4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.291+0000 7fb00cff9700 1 -- 172.21.15.33:0/696107926 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5813+0+0 (secure 0 0 0) 0x7faffc079c40 con 0x7fb010165800 2023-11-06T20:59:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.291+0000 7fb014d9b700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faff405f270 0x7faff4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.292+0000 7fb014d9b700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faff405f270 0x7faff4061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb004006d50 tx=0x7fb00401a000).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:15.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.295+0000 7fb00cff9700 1 -- 172.21.15.33:0/696107926 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faffc04ae50 con 0x7fb010165800 2023-11-06T20:59:15.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.477+0000 7fb016fff700 1 -- 172.21.15.33:0/696107926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb0100830b0 con 0x7fb010165800 2023-11-06T20:59:15.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.483+0000 7fb00cff9700 1 -- 172.21.15.33:0/696107926 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7faffc019020 con 0x7fb010165800 2023-11-06T20:59:15.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 -- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faff405f270 msgr2=0x7faff4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faff405f270 0x7faff4061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb004006d50 tx=0x7fb00401a000).stop 2023-11-06T20:59:15.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 -- 172.21.15.33:0/696107926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb010165800 msgr2=0x7fb010157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb010165800 0x7fb010157740 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7faffc001050 tx=0x7faffc001e90).stop 2023-11-06T20:59:15.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 -- 172.21.15.33:0/696107926 shutdown_connections 2023-11-06T20:59:15.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faff405f270 0x7faff4061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb0100646c0 0x7fb010064b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0000038e0 0x7fb0100addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 --2- 172.21.15.33:0/696107926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb010165800 0x7fb010157740 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 -- 172.21.15.33:0/696107926 >> 172.21.15.33:0/696107926 conn(0x7fb0100cb910 msgr2=0x7fb0100aa270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:15.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 -- 172.21.15.33:0/696107926 shutdown_connections 2023-11-06T20:59:15.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.485+0000 7fb016fff700 1 -- 172.21.15.33:0/696107926 wait complete. 2023-11-06T20:59:15.503 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T20:59:15.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:59:15.592 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:59:15.693 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T20:59:15.694 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T20:59:15.694 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.890+0000 7f99415ff700 1 Processor -- start 2023-11-06T20:59:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.890+0000 7f99415ff700 1 -- start start 2023-11-06T20:59:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.891+0000 7f99415ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c137660 0x7f993c0c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.891+0000 7f99415ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c0c17e0 con 0x7f993c069410 2023-11-06T20:59:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.892+0000 7f99415ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c06a740 con 0x7f993c0c3600 2023-11-06T20:59:15.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.892+0000 7f99415ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c069f60 con 0x7f993c137660 2023-11-06T20:59:15.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.892+0000 7f993b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f993c069410 0x7f993c069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52124/0 (socket says 172.21.15.33:52124) 2023-11-06T20:59:15.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.892+0000 7f993b7fe700 1 -- 172.21.15.33:0/158198816 learned_addr learned my addr 172.21.15.33:0/158198816 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:15.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.892+0000 7f993a7fc700 1 --2- 172.21.15.33:0/158198816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c137660 0x7f993c0c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.892+0000 7f9939ffb700 1 -- 172.21.15.33:0/158198816 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865277170 0 0) 0x7f993c0c17e0 con 0x7f993c069410 2023-11-06T20:59:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.892+0000 7f9939ffb700 1 -- 172.21.15.33:0/158198816 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99240033f0 con 0x7f993c069410 2023-11-06T20:59:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.892+0000 7f9939ffb700 1 -- 172.21.15.33:0/158198816 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042573476 0 0) 0x7f993c06a740 con 0x7f993c0c3600 2023-11-06T20:59:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.892+0000 7f9939ffb700 1 -- 172.21.15.33:0/158198816 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f993c0c17e0 con 0x7f993c0c3600 2023-11-06T20:59:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.893+0000 7f993a7fc700 1 -- 172.21.15.33:0/158198816 >> v1:172.21.15.69:6789/0 conn(0x7f993c0c3600 legacy=0x7f993c136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.893+0000 7f993a7fc700 1 -- 172.21.15.33:0/158198816 >> v1:172.21.15.33:6789/0 conn(0x7f993c069410 legacy=0x7f993c069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.893+0000 7f9939ffb700 1 -- 172.21.15.33:0/158198816 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2704097024 0 0) 0x7f99240033f0 con 0x7f993c069410 2023-11-06T20:59:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.893+0000 7f993a7fc700 1 -- 172.21.15.33:0/158198816 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c06a180 con 0x7f993c137660 2023-11-06T20:59:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.893+0000 7f993a7fc700 1 --2- 172.21.15.33:0/158198816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c137660 0x7f993c0c39e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9930009790 tx=0x7f9930009060).ready entity=mon.2 client_cookie=fea417dd93a95782 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.893+0000 7f9939ffb700 1 -- 172.21.15.33:0/158198816 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340017f0 con 0x7f993c137660 2023-11-06T20:59:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.893+0000 7f9939ffb700 1 -- 172.21.15.33:0/158198816 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9930011400 con 0x7f993c137660 2023-11-06T20:59:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.893+0000 7f9939ffb700 1 -- 172.21.15.33:0/158198816 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930011780 con 0x7f993c137660 2023-11-06T20:59:15.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.894+0000 7f99415ff700 1 -- 172.21.15.33:0/158198816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c137660 msgr2=0x7f993c0c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.894+0000 7f99415ff700 1 --2- 172.21.15.33:0/158198816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c137660 0x7f993c0c39e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9930009790 tx=0x7f9930009060).stop 2023-11-06T20:59:15.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.894+0000 7f99415ff700 1 -- 172.21.15.33:0/158198816 shutdown_connections 2023-11-06T20:59:15.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.894+0000 7f99415ff700 1 --2- 172.21.15.33:0/158198816 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c137660 0x7f993c0c39e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.894+0000 7f99415ff700 1 -- 172.21.15.33:0/158198816 >> 172.21.15.33:0/158198816 conn(0x7f993c062710 msgr2=0x7f993c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:15.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.894+0000 7f99415ff700 1 -- 172.21.15.33:0/158198816 shutdown_connections 2023-11-06T20:59:15.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.894+0000 7f99415ff700 1 -- 172.21.15.33:0/158198816 wait complete. 2023-11-06T20:59:15.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.895+0000 7f99415ff700 1 Processor -- start 2023-11-06T20:59:15.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.895+0000 7f99415ff700 1 -- start start 2023-11-06T20:59:15.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.896+0000 7f99415ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f993c069410 0x7f993c164590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.896+0000 7f99415ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c0c3600 0x7f993c1674c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.896+0000 7f99415ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c137660 0x7f993c0ac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.896+0000 7f99415ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0a99c0 con 0x7f993c137660 2023-11-06T20:59:15.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.896+0000 7f99415ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0aa260 con 0x7f993c069410 2023-11-06T20:59:15.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.896+0000 7f99415ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0be900 con 0x7f993c0c3600 2023-11-06T20:59:15.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c137660 0x7f993c0ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c137660 0x7f993c0ac000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36142/0 (socket says 172.21.15.33:36142) 2023-11-06T20:59:15.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993a7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c0c3600 0x7f993c1674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993b7fe700 1 -- 172.21.15.33:0/1146981506 learned_addr learned my addr 172.21.15.33:0/1146981506 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:15.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993affd700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f993c069410 0x7f993c164590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993b7fe700 1 -- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c0c3600 msgr2=0x7f993c1674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993b7fe700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c0c3600 0x7f993c1674c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993b7fe700 1 -- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f993c069410 msgr2=0x7f993c164590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:15.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993b7fe700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f993c069410 0x7f993c164590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:15.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993b7fe700 1 -- 172.21.15.33:0/1146981506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300092a0 con 0x7f993c137660 2023-11-06T20:59:15.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f993b7fe700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c137660 0x7f993c0ac000 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f993400cbc0 tx=0x7f9934001da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f9922ffd700 1 -- 172.21.15.33:0/1146981506 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934004c40 con 0x7f993c137660 2023-11-06T20:59:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f9922ffd700 1 -- 172.21.15.33:0/1146981506 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9934004da0 con 0x7f993c137660 2023-11-06T20:59:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f9922ffd700 1 -- 172.21.15.33:0/1146981506 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340073a0 con 0x7f993c137660 2023-11-06T20:59:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.897+0000 7f99415ff700 1 -- 172.21.15.33:0/1146981506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c0a9c10 con 0x7f993c137660 2023-11-06T20:59:15.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.898+0000 7f99415ff700 1 -- 172.21.15.33:0/1146981506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c0ac4f0 con 0x7f993c137660 2023-11-06T20:59:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.899+0000 7f99415ff700 1 -- 172.21.15.33:0/1146981506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c0aa3f0 con 0x7f993c137660 2023-11-06T20:59:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.900+0000 7f9922ffd700 1 -- 172.21.15.33:0/1146981506 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9934007710 con 0x7f993c137660 2023-11-06T20:59:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.901+0000 7f9922ffd700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f992805f270 0x7f9928061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.901+0000 7f9922ffd700 1 -- 172.21.15.33:0/1146981506 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f9934079fe0 con 0x7f993c137660 2023-11-06T20:59:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.902+0000 7f993affd700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f992805f270 0x7f9928061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.902+0000 7f993affd700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f992805f270 0x7f9928061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f992c003930 tx=0x7f992c007bd0).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:15.905+0000 7f9922ffd700 1 -- 172.21.15.33:0/1146981506 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f993404b1e0 con 0x7f993c137660 2023-11-06T20:59:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.131+0000 7f99415ff700 1 -- 172.21.15.33:0/1146981506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f993c06a460 con 0x7f993c137660 2023-11-06T20:59:16.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.132+0000 7f9922ffd700 1 -- 172.21.15.33:0/1146981506 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f993404e800 con 0x7f993c137660 2023-11-06T20:59:16.132 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:16.133 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/455721224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":455721224},{"type":"v1","addr":"172.21.15.69:6836","nonce":455721224}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5632,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2050732931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2050732931},{"type":"v1","addr":"172.21.15.33:6834","nonce":2050732931}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1579729451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1579729451},{"type":"v1","addr":"172.21.15.33:6835","nonce":1579729451}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5986,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/847465550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":847465550},{"type":"v1","addr":"172.21.15.69:6837","nonce":847465550}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[]} 2023-11-06T20:59:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.133+0000 7f99415ff700 1 -- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f992805f270 msgr2=0x7f9928061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.133+0000 7f99415ff700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f992805f270 0x7f9928061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f992c003930 tx=0x7f992c007bd0).stop 2023-11-06T20:59:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.133+0000 7f99415ff700 1 -- 172.21.15.33:0/1146981506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c137660 msgr2=0x7f993c0ac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.133+0000 7f99415ff700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c137660 0x7f993c0ac000 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f993400cbc0 tx=0x7f9934001da0).stop 2023-11-06T20:59:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.134+0000 7f99415ff700 1 -- 172.21.15.33:0/1146981506 shutdown_connections 2023-11-06T20:59:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.134+0000 7f99415ff700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f992805f270 0x7f9928061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.134+0000 7f99415ff700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c137660 0x7f993c0ac000 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.134+0000 7f99415ff700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f993c069410 0x7f993c164590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.134+0000 7f99415ff700 1 --2- 172.21.15.33:0/1146981506 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f993c0c3600 0x7f993c1674c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.134+0000 7f99415ff700 1 -- 172.21.15.33:0/1146981506 >> 172.21.15.33:0/1146981506 conn(0x7f993c062710 msgr2=0x7f993c136470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.134+0000 7f99415ff700 1 -- 172.21.15.33:0/1146981506 shutdown_connections 2023-11-06T20:59:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.134+0000 7f99415ff700 1 -- 172.21.15.33:0/1146981506 wait complete. 2023-11-06T20:59:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2023-11-06T20:59:16.151 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T20:59:16.152 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T20:59:16.152 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T20:59:16.152 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:59:16.153 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T20:59:16.153 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T20:59:16.153 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T20:59:16.153 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:59:16.154 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T20:59:16.154 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T20:59:16.154 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T20:59:16.155 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:59:16.155 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T20:59:16.155 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T20:59:16.155 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T20:59:16.156 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:59:16.166 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T20:59:16.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T20:59:16.168 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T20:59:16.168 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T20:59:16.171 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T20:59:16.171 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T20:59:16.172 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T20:59:16.172 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T20:59:16.212 INFO:tasks.ceph.mds.c:Started 2023-11-06T20:59:16.212 DEBUG:teuthology.parallel:result is None 2023-11-06T20:59:16.215 INFO:tasks.ceph.mds.a:Started 2023-11-06T20:59:16.215 DEBUG:teuthology.parallel:result is None 2023-11-06T20:59:16.216 INFO:tasks.ceph.mds.b:Started 2023-11-06T20:59:16.216 DEBUG:teuthology.parallel:result is None 2023-11-06T20:59:16.218 INFO:tasks.ceph.mds.d:Started 2023-11-06T20:59:16.218 DEBUG:teuthology.parallel:result is None 2023-11-06T20:59:16.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T20:59:16.365 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T20:59:16.366 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T20:59:16.366 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T20:59:16.370 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T20:59:16.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.425+0000 7f551e9ff700 1 Processor -- start 2023-11-06T20:59:16.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.426+0000 7f551e9ff700 1 -- start start 2023-11-06T20:59:16.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.427+0000 7f551e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 0x7f55180c1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:16.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.427+0000 7f551e9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f551806a360 0x7f5518151810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:16.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.427+0000 7f551e9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5518155010 con 0x7f5518069f80 2023-11-06T20:59:16.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.427+0000 7f551e9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f551806a7f0 con 0x7f551806a360 2023-11-06T20:59:16.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.427+0000 7f551e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5518154440 con 0x7f55180c2100 2023-11-06T20:59:16.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f5517fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5518069f80 0x7f55180c19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52146/0 (socket says 172.21.15.33:52146) 2023-11-06T20:59:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f5517fff700 1 -- 172.21.15.33:0/998469618 learned_addr learned my addr 172.21.15.33:0/998469618 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f551cf9c700 1 --2- 172.21.15.33:0/998469618 >> v2:172.21.15.33:3300/0 conn(0x7f551806a360 0x7f5518151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f55177fe700 1 --2- 172.21.15.33:0/998469618 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 0x7f55180c1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f5516ffd700 1 -- 172.21.15.33:0/998469618 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628338599 0 0) 0x7f5518155010 con 0x7f5518069f80 2023-11-06T20:59:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f5516ffd700 1 -- 172.21.15.33:0/998469618 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55000033f0 con 0x7f5518069f80 2023-11-06T20:59:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f551cf9c700 1 -- 172.21.15.33:0/998469618 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 msgr2=0x7f55180c1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f551cf9c700 1 --2- 172.21.15.33:0/998469618 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 0x7f55180c1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f551cf9c700 1 -- 172.21.15.33:0/998469618 >> v1:172.21.15.33:6789/0 conn(0x7f5518069f80 legacy=0x7f55180c19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f551cf9c700 1 -- 172.21.15.33:0/998469618 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518151d00 con 0x7f551806a360 2023-11-06T20:59:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.428+0000 7f551cf9c700 1 --2- 172.21.15.33:0/998469618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f551806a360 0x7f5518151810 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f5510009e60 tx=0x7f5510009780).ready entity=mon.0 client_cookie=4aaee641166324fe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f5516ffd700 1 -- 172.21.15.33:0/998469618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551000e070 con 0x7f551806a360 2023-11-06T20:59:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f5516ffd700 1 -- 172.21.15.33:0/998469618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551000bab0 con 0x7f551806a360 2023-11-06T20:59:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f5516ffd700 1 -- 172.21.15.33:0/998469618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551000be30 con 0x7f551806a360 2023-11-06T20:59:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f551e9ff700 1 -- 172.21.15.33:0/998469618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f551806a360 msgr2=0x7f5518151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f551e9ff700 1 --2- 172.21.15.33:0/998469618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f551806a360 0x7f5518151810 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f5510009e60 tx=0x7f5510009780).stop 2023-11-06T20:59:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f551e9ff700 1 -- 172.21.15.33:0/998469618 shutdown_connections 2023-11-06T20:59:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f551e9ff700 1 --2- 172.21.15.33:0/998469618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f551806a360 0x7f5518151810 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f551e9ff700 1 --2- 172.21.15.33:0/998469618 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 0x7f55180c1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f551e9ff700 1 -- 172.21.15.33:0/998469618 >> 172.21.15.33:0/998469618 conn(0x7f5518062710 msgr2=0x7f5518062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f551e9ff700 1 -- 172.21.15.33:0/998469618 shutdown_connections 2023-11-06T20:59:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.429+0000 7f551e9ff700 1 -- 172.21.15.33:0/998469618 wait complete. 2023-11-06T20:59:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.431+0000 7f551e9ff700 1 Processor -- start 2023-11-06T20:59:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.431+0000 7f551e9ff700 1 -- start start 2023-11-06T20:59:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f551e9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5518069f80 0x7f5518151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f551e9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f551806a360 0x7f5518153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f551e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 0x7f55180bfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f551e9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55181645c0 con 0x7f5518069f80 2023-11-06T20:59:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f551e9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5518153fb0 con 0x7f551806a360 2023-11-06T20:59:16.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f551e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55180c0270 con 0x7f55180c2100 2023-11-06T20:59:16.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f5517fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5518069f80 0x7f5518151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:16.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f5517fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5518069f80 0x7f5518151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36192/0 (socket says 172.21.15.33:36192) 2023-11-06T20:59:16.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f5517fff700 1 -- 172.21.15.33:0/2456776660 learned_addr learned my addr 172.21.15.33:0/2456776660 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:16.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f55177fe700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f551806a360 0x7f5518153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:16.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.432+0000 7f551cf9c700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 0x7f55180bfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:16.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f5517fff700 1 -- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 msgr2=0x7f55180bfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:16.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f5517fff700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 0x7f55180bfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:16.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f5517fff700 1 -- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f551806a360 msgr2=0x7f5518153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:16.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f5517fff700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f551806a360 0x7f5518153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:16.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f5517fff700 1 -- 172.21.15.33:0/2456776660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510009a20 con 0x7f5518069f80 2023-11-06T20:59:16.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f551cf9c700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 0x7f55180bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:16.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f55177fe700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f551806a360 0x7f5518153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:16.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f5517fff700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5518069f80 0x7f5518151f80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f550800c590 tx=0x7f5508007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:16.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f54fffff700 1 -- 172.21.15.33:0/2456776660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508007e40 con 0x7f5518069f80 2023-11-06T20:59:16.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f54fffff700 1 -- 172.21.15.33:0/2456776660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55080072e0 con 0x7f5518069f80 2023-11-06T20:59:16.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f54fffff700 1 -- 172.21.15.33:0/2456776660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55080057a0 con 0x7f5518069f80 2023-11-06T20:59:16.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f551e9ff700 1 -- 172.21.15.33:0/2456776660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5518152610 con 0x7f5518069f80 2023-11-06T20:59:16.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.433+0000 7f551e9ff700 1 -- 172.21.15.33:0/2456776660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5518152bd0 con 0x7f5518069f80 2023-11-06T20:59:16.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.434+0000 7f551e9ff700 1 -- 172.21.15.33:0/2456776660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5518164b80 con 0x7f5518069f80 2023-11-06T20:59:16.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.435+0000 7f54fffff700 1 -- 172.21.15.33:0/2456776660 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5508005bd0 con 0x7f5518069f80 2023-11-06T20:59:16.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.436+0000 7f54fffff700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f550405f240 0x7f5504061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:16.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.436+0000 7f54fffff700 1 -- 172.21.15.33:0/2456776660 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5785+0+0 (secure 0 0 0) 0x7f5508079b00 con 0x7f5518069f80 2023-11-06T20:59:16.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.437+0000 7f55177fe700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f550405f240 0x7f5504061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:16.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.437+0000 7f55177fe700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f550405f240 0x7f5504061700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f5518069c70 tx=0x7f550c008040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:16.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.440+0000 7f54fffff700 1 -- 172.21.15.33:0/2456776660 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f550807be60 con 0x7f5518069f80 2023-11-06T20:59:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:16.637+0000 7f551e9ff700 1 -- 172.21.15.33:0/2456776660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f55181546e0 con 0x7f5518069f80 2023-11-06T20:59:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.401+0000 7f54fffff700 1 -- 172.21.15.33:0/2456776660 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7f550804ace0 con 0x7f5518069f80 2023-11-06T20:59:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.403+0000 7f551e9ff700 1 -- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f550405f240 msgr2=0x7f5504061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.403+0000 7f551e9ff700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f550405f240 0x7f5504061700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f5518069c70 tx=0x7f550c008040).stop 2023-11-06T20:59:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.403+0000 7f551e9ff700 1 -- 172.21.15.33:0/2456776660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5518069f80 msgr2=0x7f5518151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.403+0000 7f551e9ff700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5518069f80 0x7f5518151f80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f550800c590 tx=0x7f5508007c80).stop 2023-11-06T20:59:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.404+0000 7f551e9ff700 1 -- 172.21.15.33:0/2456776660 shutdown_connections 2023-11-06T20:59:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.404+0000 7f551e9ff700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f550405f240 0x7f5504061700 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.404+0000 7f551e9ff700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55180c2100 0x7f55180bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.404+0000 7f551e9ff700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5518069f80 0x7f5518151f80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.404+0000 7f551e9ff700 1 --2- 172.21.15.33:0/2456776660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f551806a360 0x7f5518153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.404+0000 7f551e9ff700 1 -- 172.21.15.33:0/2456776660 >> 172.21.15.33:0/2456776660 conn(0x7f5518062710 msgr2=0x7f55180650c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.404+0000 7f551e9ff700 1 -- 172.21.15.33:0/2456776660 shutdown_connections 2023-11-06T20:59:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.404+0000 7f551e9ff700 1 -- 172.21.15.33:0/2456776660 wait complete. 2023-11-06T20:59:17.407 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T20:59:17.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T20:59:17.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.625+0000 7f50df9ff700 1 Processor -- start 2023-11-06T20:59:17.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.625+0000 7f50df9ff700 1 -- start start 2023-11-06T20:59:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.626+0000 7f50df9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f50d80c1840 0x7f50d80a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.626+0000 7f50df9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50d80aa2d0 0x7f50d80c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.626+0000 7f50df9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d81554a0 con 0x7f50d80c6240 2023-11-06T20:59:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.627+0000 7f50df9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d8154960 con 0x7f50d80c1840 2023-11-06T20:59:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.627+0000 7f50df9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d8155930 con 0x7f50d80aa2d0 2023-11-06T20:59:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.627+0000 7f50ddf9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f50d80c6240 0x7f50d8157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52148/0 (socket says 172.21.15.33:52148) 2023-11-06T20:59:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.627+0000 7f50ddf9c700 1 -- 172.21.15.33:0/173891562 learned_addr learned my addr 172.21.15.33:0/173891562 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.627+0000 7f50dd79b700 1 --2- 172.21.15.33:0/173891562 >> v2:172.21.15.69:3300/0 conn(0x7f50d80c1840 0x7f50d80a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.627+0000 7f50dcf9a700 1 --2- 172.21.15.33:0/173891562 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50d80aa2d0 0x7f50d80c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.627+0000 7f50cffff700 1 -- 172.21.15.33:0/173891562 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077267729 0 0) 0x7f50d81554a0 con 0x7f50d80c6240 2023-11-06T20:59:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.627+0000 7f50cffff700 1 -- 172.21.15.33:0/173891562 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c40033f0 con 0x7f50d80c6240 2023-11-06T20:59:17.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.627+0000 7f50dd79b700 1 -- 172.21.15.33:0/173891562 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50d80aa2d0 msgr2=0x7f50d80c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:17.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.628+0000 7f50dd79b700 1 --2- 172.21.15.33:0/173891562 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50d80aa2d0 0x7f50d80c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.628+0000 7f50dd79b700 1 -- 172.21.15.33:0/173891562 >> v1:172.21.15.33:6789/0 conn(0x7f50d80c6240 legacy=0x7f50d8157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:17.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.628+0000 7f50dd79b700 1 -- 172.21.15.33:0/173891562 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80c66b0 con 0x7f50d80c1840 2023-11-06T20:59:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.628+0000 7f50dd79b700 1 --2- 172.21.15.33:0/173891562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50d80c1840 0x7f50d80a9de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f50d4009760 tx=0x7f50d4009060).ready entity=mon.1 client_cookie=8ccea902f43bbfd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.628+0000 7f50cffff700 1 -- 172.21.15.33:0/173891562 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d400d4a0 con 0x7f50d80c1840 2023-11-06T20:59:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.628+0000 7f50cffff700 1 -- 172.21.15.33:0/173891562 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d400d600 con 0x7f50d80c1840 2023-11-06T20:59:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.628+0000 7f50cffff700 1 -- 172.21.15.33:0/173891562 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d400d980 con 0x7f50d80c1840 2023-11-06T20:59:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.629+0000 7f50df9ff700 1 -- 172.21.15.33:0/173891562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50d80c1840 msgr2=0x7f50d80a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.629+0000 7f50df9ff700 1 --2- 172.21.15.33:0/173891562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50d80c1840 0x7f50d80a9de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f50d4009760 tx=0x7f50d4009060).stop 2023-11-06T20:59:17.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.629+0000 7f50df9ff700 1 -- 172.21.15.33:0/173891562 shutdown_connections 2023-11-06T20:59:17.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.629+0000 7f50df9ff700 1 --2- 172.21.15.33:0/173891562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50d80c1840 0x7f50d80a9de0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.629+0000 7f50df9ff700 1 --2- 172.21.15.33:0/173891562 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50d80aa2d0 0x7f50d80c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.629+0000 7f50df9ff700 1 -- 172.21.15.33:0/173891562 >> 172.21.15.33:0/173891562 conn(0x7f50d805ec80 msgr2=0x7f50d80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:17.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.629+0000 7f50df9ff700 1 -- 172.21.15.33:0/173891562 shutdown_connections 2023-11-06T20:59:17.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.629+0000 7f50df9ff700 1 -- 172.21.15.33:0/173891562 wait complete. 2023-11-06T20:59:17.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.630+0000 7f50df9ff700 1 Processor -- start 2023-11-06T20:59:17.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.631+0000 7f50df9ff700 1 -- start start 2023-11-06T20:59:17.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.631+0000 7f50df9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50d80aa2d0 0x7f50d814f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:17.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.631+0000 7f50df9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50d80c1840 0x7f50d8158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:17.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50df9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50d80c6240 0x7f50d8158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:17.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50df9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d8168610 con 0x7f50d80c1840 2023-11-06T20:59:17.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50df9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d8167a00 con 0x7f50d80c6240 2023-11-06T20:59:17.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50df9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d8167ce0 con 0x7f50d80aa2d0 2023-11-06T20:59:17.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50dcf9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50d80c1840 0x7f50d8158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:17.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50dcf9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50d80c1840 0x7f50d8158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36208/0 (socket says 172.21.15.33:36208) 2023-11-06T20:59:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50dcf9a700 1 -- 172.21.15.33:0/2621641501 learned_addr learned my addr 172.21.15.33:0/2621641501 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50dd79b700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50d80aa2d0 0x7f50d814f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50ddf9c700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50d80c6240 0x7f50d8158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50dcf9a700 1 -- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50d80aa2d0 msgr2=0x7f50d814f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:17.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50dcf9a700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50d80aa2d0 0x7f50d814f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50dcf9a700 1 -- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50d80c6240 msgr2=0x7f50d8158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:17.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50dcf9a700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50d80c6240 0x7f50d8158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50dcf9a700 1 -- 172.21.15.33:0/2621641501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d40092a0 con 0x7f50d80c1840 2023-11-06T20:59:17.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.632+0000 7f50dcf9a700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50d80c1840 0x7f50d8158580 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f50c800e900 tx=0x7f50c8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:17.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.633+0000 7f50cd7fa700 1 -- 172.21.15.33:0/2621641501 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8012600 con 0x7f50d80c1840 2023-11-06T20:59:17.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.633+0000 7f50cd7fa700 1 -- 172.21.15.33:0/2621641501 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8012760 con 0x7f50d80c1840 2023-11-06T20:59:17.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.633+0000 7f50cd7fa700 1 -- 172.21.15.33:0/2621641501 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8012ae0 con 0x7f50d80c1840 2023-11-06T20:59:17.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.633+0000 7f50df9ff700 1 -- 172.21.15.33:0/2621641501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d8167fc0 con 0x7f50d80c1840 2023-11-06T20:59:17.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.633+0000 7f50df9ff700 1 -- 172.21.15.33:0/2621641501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d8155d30 con 0x7f50d80c1840 2023-11-06T20:59:17.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.634+0000 7f50df9ff700 1 -- 172.21.15.33:0/2621641501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d806c6c0 con 0x7f50d80c1840 2023-11-06T20:59:17.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.635+0000 7f50cd7fa700 1 -- 172.21.15.33:0/2621641501 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f50c801b400 con 0x7f50d80c1840 2023-11-06T20:59:17.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.636+0000 7f50cd7fa700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f50bc05f270 0x7f50bc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:17.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.637+0000 7f50cd7fa700 1 -- 172.21.15.33:0/2621641501 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f50c807a530 con 0x7f50d80c1840 2023-11-06T20:59:17.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.637+0000 7f50dd79b700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f50bc05f270 0x7f50bc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:17.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.637+0000 7f50dd79b700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f50bc05f270 0x7f50bc061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f50d4006d30 tx=0x7f50d4016000).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:17.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.641+0000 7f50cd7fa700 1 -- 172.21.15.33:0/2621641501 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f50c804b8b0 con 0x7f50d80c1840 2023-11-06T20:59:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.892+0000 7f50df9ff700 1 -- 172.21.15.33:0/2621641501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f50d8063f20 con 0x7f50d80c1840 2023-11-06T20:59:17.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.892+0000 7f50cd7fa700 1 -- 172.21.15.33:0/2621641501 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f50c8016020 con 0x7f50d80c1840 2023-11-06T20:59:17.893 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:17.893 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T20:59:17.893 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T20:59:17.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.894 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T20:59:17.894 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T20:59:17.894 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.895 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:17.895 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:59:17.895 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:59:17.895 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.896 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T20:59:17.896 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T20:59:17.896 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.896 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:17.897 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:59:17.897 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:59:17.897 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.897 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.898 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T20:59:17.898 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T20:59:17.898 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.898 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:17.899 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:59:17.899 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:59:17.899 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.899 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.900 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T20:59:17.900 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T20:59:17.900 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.900 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:17.901 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:59:17.901 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:59:17.901 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.902 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T20:59:17.902 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T20:59:17.902 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.902 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:17.903 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:17.903 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.904 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T20:59:17.904 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T20:59:17.904 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.904 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:17.904 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:17.905 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.905 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T20:59:17.906 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T20:59:17.906 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.906 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:17.906 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:17.906 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.907 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T20:59:17.907 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T20:59:17.907 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.908 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:17.908 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:17.908 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.909 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T20:59:17.909 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T20:59:17.909 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.910 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:17.910 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:17.910 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.910 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.911 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T20:59:17.911 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T20:59:17.911 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.911 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:17.912 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:17.912 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.912 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T20:59:17.913 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T20:59:17.913 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.913 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:17.913 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:17.913 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.914 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T20:59:17.914 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T20:59:17.914 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.915 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:17.915 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:17.915 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.916 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T20:59:17.916 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T20:59:17.916 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.916 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:17.916 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-11-06T20:59:17.917 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T20:59:17.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.917 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T20:59:17.917 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T20:59:17.918 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.918 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:17.918 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:59:17.918 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:59:17.918 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:59:17.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.919 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T20:59:17.919 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T20:59:17.920 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.920 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:17.920 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:59:17.920 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:59:17.920 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:59:17.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.921 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T20:59:17.921 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T20:59:17.921 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.921 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:17.922 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:59:17.922 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:59:17.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:59:17.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.923 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T20:59:17.923 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T20:59:17.923 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.923 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:59:17.924 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:59:17.924 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:59:17.924 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.924 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.925 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T20:59:17.925 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T20:59:17.925 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.925 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:59:17.925 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T20:59:17.926 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:17.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:17.926 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T20:59:17.926 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T20:59:17.927 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:17.927 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:59:17.927 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T20:59:17.927 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:17.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:17.928 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T20:59:17.928 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T20:59:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.894+0000 7f50df9ff700 1 -- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f50bc05f270 msgr2=0x7f50bc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.894+0000 7f50df9ff700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f50bc05f270 0x7f50bc061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f50d4006d30 tx=0x7f50d4016000).stop 2023-11-06T20:59:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.894+0000 7f50df9ff700 1 -- 172.21.15.33:0/2621641501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50d80c1840 msgr2=0x7f50d8158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.894+0000 7f50df9ff700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50d80c1840 0x7f50d8158580 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f50c800e900 tx=0x7f50c8009060).stop 2023-11-06T20:59:17.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.894+0000 7f50df9ff700 1 -- 172.21.15.33:0/2621641501 shutdown_connections 2023-11-06T20:59:17.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.894+0000 7f50df9ff700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f50bc05f270 0x7f50bc061730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.894+0000 7f50df9ff700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f50d80aa2d0 0x7f50d814f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.894+0000 7f50df9ff700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50d80c6240 0x7f50d8158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.895+0000 7f50df9ff700 1 --2- 172.21.15.33:0/2621641501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50d80c1840 0x7f50d8158580 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:17.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.895+0000 7f50df9ff700 1 -- 172.21.15.33:0/2621641501 >> 172.21.15.33:0/2621641501 conn(0x7f50d805ec80 msgr2=0x7f50d81530a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:17.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.895+0000 7f50df9ff700 1 -- 172.21.15.33:0/2621641501 shutdown_connections 2023-11-06T20:59:17.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:17.895+0000 7f50df9ff700 1 -- 172.21.15.33:0/2621641501 wait complete. 2023-11-06T20:59:17.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:18.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.128+0000 7f3561dff700 1 Processor -- start 2023-11-06T20:59:18.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.129+0000 7f3561dff700 1 -- start start 2023-11-06T20:59:18.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.130+0000 7f3561dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f355c0c1840 0x7f355c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.130+0000 7f3561dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f355c0c6460 0x7f355c0bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.130+0000 7f3561dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c1554a0 con 0x7f355c0aa2d0 2023-11-06T20:59:18.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.130+0000 7f3561dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c154960 con 0x7f355c0c6460 2023-11-06T20:59:18.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.130+0000 7f3561dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c155930 con 0x7f355c0c1840 2023-11-06T20:59:18.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.130+0000 7f355bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f355c0c6460 0x7f355c0bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.130+0000 7f355bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f355c0c6460 0x7f355c0bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57162/0 (socket says 172.21.15.33:57162) 2023-11-06T20:59:18.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.130+0000 7f355bfff700 1 -- 172.21.15.33:0/4079917443 learned_addr learned my addr 172.21.15.33:0/4079917443 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:18.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.130+0000 7f355b7fe700 1 --2- 172.21.15.33:0/4079917443 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f355c0c1840 0x7f355c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.131+0000 7f355bfff700 1 -- 172.21.15.33:0/4079917443 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f355c0c1840 msgr2=0x7f355c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:18.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.131+0000 7f355bfff700 1 --2- 172.21.15.33:0/4079917443 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f355c0c1840 0x7f355c0a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.131+0000 7f355bfff700 1 -- 172.21.15.33:0/4079917443 >> v1:172.21.15.69:6789/0 conn(0x7f355c0aa2d0 legacy=0x7f355c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.131+0000 7f355bfff700 1 -- 172.21.15.33:0/4079917443 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c154b70 con 0x7f355c0c6460 2023-11-06T20:59:18.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.131+0000 7f355bfff700 1 --2- 172.21.15.33:0/4079917443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c6460 0x7f355c0bfd70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f3554009f20 tx=0x7f3554009840).ready entity=mon.0 client_cookie=8a58b64162ea8ce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:18.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.131+0000 7f355a7fc700 1 -- 172.21.15.33:0/4079917443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355400e070 con 0x7f355c0c6460 2023-11-06T20:59:18.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.131+0000 7f355a7fc700 1 -- 172.21.15.33:0/4079917443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355400ba80 con 0x7f355c0c6460 2023-11-06T20:59:18.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.131+0000 7f355a7fc700 1 -- 172.21.15.33:0/4079917443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355400be00 con 0x7f355c0c6460 2023-11-06T20:59:18.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.132+0000 7f3561dff700 1 -- 172.21.15.33:0/4079917443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c6460 msgr2=0x7f355c0bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.132+0000 7f3561dff700 1 --2- 172.21.15.33:0/4079917443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c6460 0x7f355c0bfd70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f3554009f20 tx=0x7f3554009840).stop 2023-11-06T20:59:18.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.132+0000 7f3561dff700 1 -- 172.21.15.33:0/4079917443 shutdown_connections 2023-11-06T20:59:18.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.132+0000 7f3561dff700 1 --2- 172.21.15.33:0/4079917443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c6460 0x7f355c0bfd70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.132+0000 7f3561dff700 1 --2- 172.21.15.33:0/4079917443 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f355c0c1840 0x7f355c0a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.132+0000 7f3561dff700 1 -- 172.21.15.33:0/4079917443 >> 172.21.15.33:0/4079917443 conn(0x7f355c05ec80 msgr2=0x7f355c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:18.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.132+0000 7f3561dff700 1 -- 172.21.15.33:0/4079917443 shutdown_connections 2023-11-06T20:59:18.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.132+0000 7f3561dff700 1 -- 172.21.15.33:0/4079917443 wait complete. 2023-11-06T20:59:18.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.134+0000 7f3561dff700 1 Processor -- start 2023-11-06T20:59:18.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.134+0000 7f3561dff700 1 -- start start 2023-11-06T20:59:18.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.135+0000 7f3561dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355c0aa2d0 0x7f355c0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.135+0000 7f3561dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c1840 0x7f355c150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.135+0000 7f3561dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f355c0c6460 0x7f355c0ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.135+0000 7f3561dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0b0db0 con 0x7f355c0c1840 2023-11-06T20:59:18.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.135+0000 7f3561dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0b0ad0 con 0x7f355c0aa2d0 2023-11-06T20:59:18.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.135+0000 7f3561dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0c3930 con 0x7f355c0c6460 2023-11-06T20:59:18.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c1840 0x7f355c150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c1840 0x7f355c150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57172/0 (socket says 172.21.15.33:57172) 2023-11-06T20:59:18.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355c0aa2d0 0x7f355c0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355affd700 1 -- 172.21.15.33:0/3726941922 learned_addr learned my addr 172.21.15.33:0/3726941922 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:18.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355bfff700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f355c0c6460 0x7f355c0ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355affd700 1 -- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f355c0c6460 msgr2=0x7f355c0ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355affd700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f355c0c6460 0x7f355c0ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355affd700 1 -- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355c0aa2d0 msgr2=0x7f355c0c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355affd700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355c0aa2d0 0x7f355c0c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355affd700 1 -- 172.21.15.33:0/3726941922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554009ae0 con 0x7f355c0c1840 2023-11-06T20:59:18.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.136+0000 7f355affd700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c1840 0x7f355c150960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f355000c440 tx=0x7f3550007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:18.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.137+0000 7f35437fe700 1 -- 172.21.15.33:0/3726941922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550007020 con 0x7f355c0c1840 2023-11-06T20:59:18.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.137+0000 7f35437fe700 1 -- 172.21.15.33:0/3726941922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3550007180 con 0x7f355c0c1840 2023-11-06T20:59:18.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.137+0000 7f35437fe700 1 -- 172.21.15.33:0/3726941922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550007520 con 0x7f355c0c1840 2023-11-06T20:59:18.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.137+0000 7f3561dff700 1 -- 172.21.15.33:0/3726941922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c0c3c10 con 0x7f355c0c1840 2023-11-06T20:59:18.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.137+0000 7f3561dff700 1 -- 172.21.15.33:0/3726941922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c0b07a0 con 0x7f355c0c1840 2023-11-06T20:59:18.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.138+0000 7f3561dff700 1 -- 172.21.15.33:0/3726941922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c06c6c0 con 0x7f355c0c1840 2023-11-06T20:59:18.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.139+0000 7f35437fe700 1 -- 172.21.15.33:0/3726941922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f35500068b0 con 0x7f355c0c1840 2023-11-06T20:59:18.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.139+0000 7f35437fe700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f354805f270 0x7f3548061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.140+0000 7f35437fe700 1 -- 172.21.15.33:0/3726941922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f3550079a50 con 0x7f355c0c1840 2023-11-06T20:59:18.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.140+0000 7f355b7fe700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f354805f270 0x7f3548061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.141+0000 7f355b7fe700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f354805f270 0x7f3548061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f354c00bbc0 tx=0x7f354c008040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:18.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.144+0000 7f35437fe700 1 -- 172.21.15.33:0/3726941922 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f355004ae30 con 0x7f355c0c1840 2023-11-06T20:59:18.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.325+0000 7f3561dff700 1 -- 172.21.15.33:0/3726941922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f355c0cb300 con 0x7f355c0c1840 2023-11-06T20:59:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.326+0000 7f35437fe700 1 -- 172.21.15.33:0/3726941922 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+13760 (secure 0 0 0) 0x7f355004e450 con 0x7f355c0c1840 2023-11-06T20:59:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:18.327 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":80,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:17.397477+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 -- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f354805f270 msgr2=0x7f3548061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f354805f270 0x7f3548061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f354c00bbc0 tx=0x7f354c008040).stop 2023-11-06T20:59:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 -- 172.21.15.33:0/3726941922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c1840 msgr2=0x7f355c150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c1840 0x7f355c150960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f355000c440 tx=0x7f3550007d40).stop 2023-11-06T20:59:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 -- 172.21.15.33:0/3726941922 shutdown_connections 2023-11-06T20:59:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f354805f270 0x7f3548061730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f355c0c6460 0x7f355c0ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f355c0aa2d0 0x7f355c0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 --2- 172.21.15.33:0/3726941922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f355c0c1840 0x7f355c150960 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 -- 172.21.15.33:0/3726941922 >> 172.21.15.33:0/3726941922 conn(0x7f355c05ec80 msgr2=0x7f355c0c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 -- 172.21.15.33:0/3726941922 shutdown_connections 2023-11-06T20:59:18.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.328+0000 7f3561dff700 1 -- 172.21.15.33:0/3726941922 wait complete. 2023-11-06T20:59:18.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T20:59:18.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:59:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.545+0000 7f6dc65ff700 1 Processor -- start 2023-11-06T20:59:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.546+0000 7f6dc65ff700 1 -- start start 2023-11-06T20:59:18.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.547+0000 7f6dc65ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6dc00636d0 0x7f6dc0158980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.547+0000 7f6dc65ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc014e2d0 0x7f6dc014e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.547+0000 7f6dc65ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc01584f0 con 0x7f6dc00632f0 2023-11-06T20:59:18.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.547+0000 7f6dc65ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc01559e0 con 0x7f6dc00636d0 2023-11-06T20:59:18.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.547+0000 7f6dc65ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0158e70 con 0x7f6dc014e2d0 2023-11-06T20:59:18.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dc4b9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc014e2d0 0x7f6dc014e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dbf7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6dc00636d0 0x7f6dc0158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dbffff700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f6dc00632f0 0x7f6dc01549d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:55808/0 (socket says 172.21.15.33:55808) 2023-11-06T20:59:18.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dbffff700 1 -- 172.21.15.33:0/3197641177 learned_addr learned my addr 172.21.15.33:0/3197641177 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:18.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dc4b9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc014e2d0 0x7f6dc014e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:51086/0 (socket says 172.21.15.33:51086) 2023-11-06T20:59:18.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dbeffd700 1 -- 172.21.15.33:0/3197641177 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957565962 0 0) 0x7f6dc01584f0 con 0x7f6dc00632f0 2023-11-06T20:59:18.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dbeffd700 1 -- 172.21.15.33:0/3197641177 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da80033f0 con 0x7f6dc00632f0 2023-11-06T20:59:18.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dc4b9c700 1 -- 172.21.15.33:0/3197641177 >> v2:172.21.15.69:3300/0 conn(0x7f6dc00636d0 msgr2=0x7f6dc0158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:18.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dc4b9c700 1 --2- 172.21.15.33:0/3197641177 >> v2:172.21.15.69:3300/0 conn(0x7f6dc00636d0 0x7f6dc0158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dc4b9c700 1 -- 172.21.15.33:0/3197641177 >> v1:172.21.15.69:6789/0 conn(0x7f6dc00632f0 legacy=0x7f6dc01549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dc4b9c700 1 -- 172.21.15.33:0/3197641177 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc0155420 con 0x7f6dc014e2d0 2023-11-06T20:59:18.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.548+0000 7f6dbf7fe700 1 --2- 172.21.15.33:0/3197641177 >> v2:172.21.15.69:3300/0 conn(0x7f6dc00636d0 0x7f6dc0158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:18.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dc4b9c700 1 --2- 172.21.15.33:0/3197641177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc014e2d0 0x7f6dc014e730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6db8009e60 tx=0x7f6db8009780).ready entity=mon.2 client_cookie=478ae45440fe4c43 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:18.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dbeffd700 1 -- 172.21.15.33:0/3197641177 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db800e070 con 0x7f6dc014e2d0 2023-11-06T20:59:18.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dbeffd700 1 -- 172.21.15.33:0/3197641177 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db800bab0 con 0x7f6dc014e2d0 2023-11-06T20:59:18.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dbeffd700 1 -- 172.21.15.33:0/3197641177 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db800be30 con 0x7f6dc014e2d0 2023-11-06T20:59:18.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3197641177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc014e2d0 msgr2=0x7f6dc014e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dc65ff700 1 --2- 172.21.15.33:0/3197641177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc014e2d0 0x7f6dc014e730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6db8009e60 tx=0x7f6db8009780).stop 2023-11-06T20:59:18.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3197641177 shutdown_connections 2023-11-06T20:59:18.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dc65ff700 1 --2- 172.21.15.33:0/3197641177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc014e2d0 0x7f6dc014e730 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dc65ff700 1 --2- 172.21.15.33:0/3197641177 >> v2:172.21.15.69:3300/0 conn(0x7f6dc00636d0 0x7f6dc0158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3197641177 >> 172.21.15.33:0/3197641177 conn(0x7f6dc00cb910 msgr2=0x7f6dc00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:18.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.549+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3197641177 shutdown_connections 2023-11-06T20:59:18.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.550+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3197641177 wait complete. 2023-11-06T20:59:18.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.551+0000 7f6dc65ff700 1 Processor -- start 2023-11-06T20:59:18.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.551+0000 7f6dc65ff700 1 -- start start 2023-11-06T20:59:18.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dc65ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc00632f0 0x7f6dc00c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dc65ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc00636d0 0x7f6dc00af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dc65ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc014e2d0 0x7f6dc0163e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dc65ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc00b0750 con 0x7f6dc00636d0 2023-11-06T20:59:18.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dc65ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc00c4870 con 0x7f6dc014e2d0 2023-11-06T20:59:18.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dc65ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc00c2320 con 0x7f6dc00632f0 2023-11-06T20:59:18.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dbf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc00636d0 0x7f6dc00af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dbf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc00636d0 0x7f6dc00af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57176/0 (socket says 172.21.15.33:57176) 2023-11-06T20:59:18.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dbf7fe700 1 -- 172.21.15.33:0/3793202801 learned_addr learned my addr 172.21.15.33:0/3793202801 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:18.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dc4b9c700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc014e2d0 0x7f6dc0163e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.552+0000 7f6dbffff700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc00632f0 0x7f6dc00c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6dbf7fe700 1 -- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc00632f0 msgr2=0x7f6dc00c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6dbf7fe700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc00632f0 0x7f6dc00c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6dbf7fe700 1 -- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc014e2d0 msgr2=0x7f6dc0163e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6dbf7fe700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc014e2d0 0x7f6dc0163e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6dbf7fe700 1 -- 172.21.15.33:0/3793202801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40092a0 con 0x7f6dc00636d0 2023-11-06T20:59:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6dbf7fe700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc00636d0 0x7f6dc00af770 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f6db400ea20 tx=0x7f6db4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6da7fff700 1 -- 172.21.15.33:0/3793202801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db400ee60 con 0x7f6dc00636d0 2023-11-06T20:59:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6da7fff700 1 -- 172.21.15.33:0/3793202801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db400c850 con 0x7f6dc00636d0 2023-11-06T20:59:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6da7fff700 1 -- 172.21.15.33:0/3793202801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db400cbd0 con 0x7f6dc00636d0 2023-11-06T20:59:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3793202801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db8009a20 con 0x7f6dc00636d0 2023-11-06T20:59:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.553+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3793202801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc00afc60 con 0x7f6dc00636d0 2023-11-06T20:59:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.555+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3793202801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc0157b30 con 0x7f6dc00636d0 2023-11-06T20:59:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.555+0000 7f6da7fff700 1 -- 172.21.15.33:0/3793202801 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6db401b400 con 0x7f6dc00636d0 2023-11-06T20:59:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.556+0000 7f6da7fff700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dac05f270 0x7f6dac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.556+0000 7f6da7fff700 1 -- 172.21.15.33:0/3793202801 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f6db407a550 con 0x7f6dc00636d0 2023-11-06T20:59:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.557+0000 7f6dbffff700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dac05f270 0x7f6dac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.557+0000 7f6dbffff700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dac05f270 0x7f6dac061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6db0007fd0 tx=0x7f6db0009040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:18.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.560+0000 7f6da7fff700 1 -- 172.21.15.33:0/3793202801 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6db404b990 con 0x7f6dc00636d0 2023-11-06T20:59:18.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.739+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3793202801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6dc00c2460 con 0x7f6dc00636d0 2023-11-06T20:59:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.739+0000 7f6da7fff700 1 -- 172.21.15.33:0/3793202801 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v80) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6db404efb0 con 0x7f6dc00636d0 2023-11-06T20:59:18.740 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:59:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dac05f270 msgr2=0x7f6dac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dac05f270 0x7f6dac061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6db0007fd0 tx=0x7f6db0009040).stop 2023-11-06T20:59:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3793202801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc00636d0 msgr2=0x7f6dc00af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc00636d0 0x7f6dc00af770 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f6db400ea20 tx=0x7f6db4009060).stop 2023-11-06T20:59:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3793202801 shutdown_connections 2023-11-06T20:59:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6dac05f270 0x7f6dac061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6dc00632f0 0x7f6dc00c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc014e2d0 0x7f6dc0163e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 --2- 172.21.15.33:0/3793202801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc00636d0 0x7f6dc00af770 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3793202801 >> 172.21.15.33:0/3793202801 conn(0x7f6dc00cb910 msgr2=0x7f6dc00ab130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3793202801 shutdown_connections 2023-11-06T20:59:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.741+0000 7f6dc65ff700 1 -- 172.21.15.33:0/3793202801 wait complete. 2023-11-06T20:59:18.759 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:59:18.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.957+0000 7f8b0a3ff700 1 Processor -- start 2023-11-06T20:59:18.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.957+0000 7f8b0a3ff700 1 -- start start 2023-11-06T20:59:18.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.959+0000 7f8b0a3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b041544f0 0x7f8b040c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.959+0000 7f8b0a3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8b040c0c40 0x7f8b040646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.959+0000 7f8b0a3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b041556d0 con 0x7f8b04064bb0 2023-11-06T20:59:18.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.959+0000 7f8b0a3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b041548d0 con 0x7f8b040c0c40 2023-11-06T20:59:18.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.959+0000 7f8b0a3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b04065030 con 0x7f8b041544f0 2023-11-06T20:59:18.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.959+0000 7f8b0899c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8b04064bb0 0x7f8b04157f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51754/0 (socket says 172.21.15.33:51754) 2023-11-06T20:59:18.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.959+0000 7f8b0899c700 1 -- 172.21.15.33:0/979956124 learned_addr learned my addr 172.21.15.33:0/979956124 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:18.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.959+0000 7f8b03fff700 1 --2- 172.21.15.33:0/979956124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b041544f0 0x7f8b040c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.960+0000 7f8b037fe700 1 --2- 172.21.15.33:0/979956124 >> v2:172.21.15.69:3300/0 conn(0x7f8b040c0c40 0x7f8b040646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.960+0000 7f8b02ffd700 1 -- 172.21.15.33:0/979956124 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270107956 0 0) 0x7f8b041556d0 con 0x7f8b04064bb0 2023-11-06T20:59:18.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.960+0000 7f8b02ffd700 1 -- 172.21.15.33:0/979956124 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aec0033f0 con 0x7f8b04064bb0 2023-11-06T20:59:18.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.960+0000 7f8b037fe700 1 -- 172.21.15.33:0/979956124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b041544f0 msgr2=0x7f8b040c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:18.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.960+0000 7f8b037fe700 1 --2- 172.21.15.33:0/979956124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b041544f0 0x7f8b040c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.960+0000 7f8b02ffd700 1 -- 172.21.15.33:0/979956124 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3727039150 0 0) 0x7f8aec0033f0 con 0x7f8b04064bb0 2023-11-06T20:59:18.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.960+0000 7f8b037fe700 1 -- 172.21.15.33:0/979956124 >> v1:172.21.15.33:6789/0 conn(0x7f8b04064bb0 legacy=0x7f8b04157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.960+0000 7f8b037fe700 1 -- 172.21.15.33:0/979956124 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b04154ac0 con 0x7f8b040c0c40 2023-11-06T20:59:18.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.960+0000 7f8b03fff700 1 --2- 172.21.15.33:0/979956124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b041544f0 0x7f8b040c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:18.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.960+0000 7f8b037fe700 1 --2- 172.21.15.33:0/979956124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b040c0c40 0x7f8b040646c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f8af8009760 tx=0x7f8af8009060).ready entity=mon.1 client_cookie=82732b60a47f3cbf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:18.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b02ffd700 1 -- 172.21.15.33:0/979956124 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc0017f0 con 0x7f8b040c0c40 2023-11-06T20:59:18.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b02ffd700 1 -- 172.21.15.33:0/979956124 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af8011400 con 0x7f8b040c0c40 2023-11-06T20:59:18.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b02ffd700 1 -- 172.21.15.33:0/979956124 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8011780 con 0x7f8b040c0c40 2023-11-06T20:59:18.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/979956124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b040c0c40 msgr2=0x7f8b040646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b0a3ff700 1 --2- 172.21.15.33:0/979956124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b040c0c40 0x7f8b040646c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f8af8009760 tx=0x7f8af8009060).stop 2023-11-06T20:59:18.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/979956124 shutdown_connections 2023-11-06T20:59:18.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b0a3ff700 1 --2- 172.21.15.33:0/979956124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b041544f0 0x7f8b040c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b0a3ff700 1 --2- 172.21.15.33:0/979956124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b040c0c40 0x7f8b040646c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/979956124 >> 172.21.15.33:0/979956124 conn(0x7f8b040cb900 msgr2=0x7f8b040c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:18.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/979956124 shutdown_connections 2023-11-06T20:59:18.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.961+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/979956124 wait complete. 2023-11-06T20:59:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.963+0000 7f8b0a3ff700 1 Processor -- start 2023-11-06T20:59:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.963+0000 7f8b0a3ff700 1 -- start start 2023-11-06T20:59:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0a3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b04064bb0 0x7f8b040aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0a3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b040c0c40 0x7f8b040c2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0a3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b041544f0 0x7f8b040c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0a3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b040c2a50 con 0x7f8b041544f0 2023-11-06T20:59:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0a3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b040b05b0 con 0x7f8b04064bb0 2023-11-06T20:59:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0a3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b040b0890 con 0x7f8b040c0c40 2023-11-06T20:59:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b041544f0 0x7f8b040c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b041544f0 0x7f8b040c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57178/0 (socket says 172.21.15.33:57178) 2023-11-06T20:59:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0899c700 1 -- 172.21.15.33:0/3523902057 learned_addr learned my addr 172.21.15.33:0/3523902057 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b03fff700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b04064bb0 0x7f8b040aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b037fe700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b040c0c40 0x7f8b040c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0899c700 1 -- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b040c0c40 msgr2=0x7f8b040c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0899c700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b040c0c40 0x7f8b040c2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0899c700 1 -- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b04064bb0 msgr2=0x7f8b040aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0899c700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b04064bb0 0x7f8b040aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:18.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.964+0000 7f8b0899c700 1 -- 172.21.15.33:0/3523902057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af40092a0 con 0x7f8b041544f0 2023-11-06T20:59:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.965+0000 7f8b0899c700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b041544f0 0x7f8b040c5810 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f8afc00cb40 tx=0x7f8afc001e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.965+0000 7f8aebfff700 1 -- 172.21.15.33:0/3523902057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc004c40 con 0x7f8b041544f0 2023-11-06T20:59:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.965+0000 7f8aebfff700 1 -- 172.21.15.33:0/3523902057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc004da0 con 0x7f8b041544f0 2023-11-06T20:59:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.965+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/3523902057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af80092a0 con 0x7f8b041544f0 2023-11-06T20:59:18.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.965+0000 7f8aebfff700 1 -- 172.21.15.33:0/3523902057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc007380 con 0x7f8b041544f0 2023-11-06T20:59:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.965+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/3523902057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b04167cd0 con 0x7f8b041544f0 2023-11-06T20:59:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.966+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/3523902057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b040b0db0 con 0x7f8b041544f0 2023-11-06T20:59:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.967+0000 7f8aebfff700 1 -- 172.21.15.33:0/3523902057 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8afc007770 con 0x7f8b041544f0 2023-11-06T20:59:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.968+0000 7f8aebfff700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af005f270 0x7f8af0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.968+0000 7f8aebfff700 1 -- 172.21.15.33:0/3523902057 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f8afc079f60 con 0x7f8b041544f0 2023-11-06T20:59:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.968+0000 7f8b03fff700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af005f270 0x7f8af0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.969+0000 7f8b03fff700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af005f270 0x7f8af0061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f8af4009d20 tx=0x7f8af4009060).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:18.972+0000 7f8aebfff700 1 -- 172.21.15.33:0/3523902057 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8afc04b340 con 0x7f8b041544f0 2023-11-06T20:59:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.156+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/3523902057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8b04154da0 con 0x7f8b041544f0 2023-11-06T20:59:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.157+0000 7f8aebfff700 1 -- 172.21.15.33:0/3523902057 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8afc04e960 con 0x7f8b041544f0 2023-11-06T20:59:19.158 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:59:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af005f270 msgr2=0x7f8af0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af005f270 0x7f8af0061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f8af4009d20 tx=0x7f8af4009060).stop 2023-11-06T20:59:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/3523902057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b041544f0 msgr2=0x7f8b040c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b041544f0 0x7f8b040c5810 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f8afc00cb40 tx=0x7f8afc001e20).stop 2023-11-06T20:59:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/3523902057 shutdown_connections 2023-11-06T20:59:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af005f270 0x7f8af0061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b041544f0 0x7f8b040c5810 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b04064bb0 0x7f8b040aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 --2- 172.21.15.33:0/3523902057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b040c0c40 0x7f8b040c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/3523902057 >> 172.21.15.33:0/3523902057 conn(0x7f8b040cb900 msgr2=0x7f8b040c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/3523902057 shutdown_connections 2023-11-06T20:59:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.159+0000 7f8b0a3ff700 1 -- 172.21.15.33:0/3523902057 wait complete. 2023-11-06T20:59:19.177 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T20:59:19.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.373+0000 7f261bfff700 1 Processor -- start 2023-11-06T20:59:19.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.373+0000 7f261bfff700 1 -- start start 2023-11-06T20:59:19.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.375+0000 7f261bfff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f26141580e0 0x7f26141584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.375+0000 7f261bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2614064010 con 0x7f26140a7910 2023-11-06T20:59:19.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.375+0000 7f261bfff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140be150 con 0x7f26140bdbe0 2023-11-06T20:59:19.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.375+0000 7f261bfff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26140be430 con 0x7f26141580e0 2023-11-06T20:59:19.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.375+0000 7f261a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f26140a7910 0x7f26140a7d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51758/0 (socket says 172.21.15.33:51758) 2023-11-06T20:59:19.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.375+0000 7f261a59c700 1 -- 172.21.15.33:0/2272850408 learned_addr learned my addr 172.21.15.33:0/2272850408 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:19.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.375+0000 7f2619d9b700 1 --2- 172.21.15.33:0/2272850408 >> v2:172.21.15.69:3300/0 conn(0x7f26141580e0 0x7f26141584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.375+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156909437 0 0) 0x7f2614064010 con 0x7f26140a7910 2023-11-06T20:59:19.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.375+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26000033f0 con 0x7f26140a7910 2023-11-06T20:59:19.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688636946 0 0) 0x7f26140be150 con 0x7f26140bdbe0 2023-11-06T20:59:19.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2614064010 con 0x7f26140bdbe0 2023-11-06T20:59:19.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 90351104 0 0) 0x7f26000033f0 con 0x7f26140a7910 2023-11-06T20:59:19.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 >> v2:172.21.15.69:3300/0 conn(0x7f26141580e0 msgr2=0x7f26141584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:19.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 --2- 172.21.15.33:0/2272850408 >> v2:172.21.15.69:3300/0 conn(0x7f26141580e0 0x7f26141584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 >> v1:172.21.15.69:6789/0 conn(0x7f26140bdbe0 legacy=0x7f26140684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140639c0 con 0x7f26140a7910 2023-11-06T20:59:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f260c0017f0 con 0x7f26140a7910 2023-11-06T20:59:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 >> v1:172.21.15.33:6789/0 conn(0x7f26140a7910 legacy=0x7f26140a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2600003a70 0x7f2600003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2600004690 0x7f2600004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2600005000 0x7f260000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140be430 con 0x7f2600004690 2023-11-06T20:59:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260000bf60 con 0x7f2600003a70 2023-11-06T20:59:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260000c0d0 con 0x7f2600005000 2023-11-06T20:59:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f261959a700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2600004690 0x7f2600004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f2619d9b700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2600003a70 0x7f2600003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.376+0000 7f261a59c700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2600005000 0x7f260000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f261959a700 1 -- 172.21.15.33:0/2272850408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2600005000 msgr2=0x7f260000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f261959a700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2600005000 0x7f260000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f261959a700 1 -- 172.21.15.33:0/2272850408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2600003a70 msgr2=0x7f2600003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f261959a700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2600003a70 0x7f2600003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f261959a700 1 -- 172.21.15.33:0/2272850408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26000033f0 con 0x7f2600004690 2023-11-06T20:59:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f2619d9b700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2600003a70 0x7f2600003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f261a59c700 1 -- 172.21.15.33:0/2272850408 reap_dead start 2023-11-06T20:59:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f261959a700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2600004690 0x7f2600004b10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f2604001890 tx=0x7f2604007bb0).ready entity=mon.0 client_cookie=5f6240535952bb13 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604007020 con 0x7f2600004690 2023-11-06T20:59:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2604007180 con 0x7f2600004690 2023-11-06T20:59:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.377+0000 7f2618d99700 1 -- 172.21.15.33:0/2272850408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604007520 con 0x7f2600004690 2023-11-06T20:59:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.378+0000 7f261bfff700 1 -- 172.21.15.33:0/2272850408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2600004690 msgr2=0x7f2600004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.378+0000 7f261bfff700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2600004690 0x7f2600004b10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f2604001890 tx=0x7f2604007bb0).stop 2023-11-06T20:59:19.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.378+0000 7f261bfff700 1 -- 172.21.15.33:0/2272850408 shutdown_connections 2023-11-06T20:59:19.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.378+0000 7f261bfff700 1 --2- 172.21.15.33:0/2272850408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2600004690 0x7f2600004b10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.378+0000 7f261bfff700 1 -- 172.21.15.33:0/2272850408 >> 172.21.15.33:0/2272850408 conn(0x7f261405ec80 msgr2=0x7f26140664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:19.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.378+0000 7f261bfff700 1 -- 172.21.15.33:0/2272850408 shutdown_connections 2023-11-06T20:59:19.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.378+0000 7f261bfff700 1 -- 172.21.15.33:0/2272850408 wait complete. 2023-11-06T20:59:19.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.379+0000 7f261bfff700 1 Processor -- start 2023-11-06T20:59:19.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.379+0000 7f261bfff700 1 -- start start 2023-11-06T20:59:19.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.380+0000 7f261bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2600004690 0x7f26140c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.380+0000 7f261bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26141399f0 0x7f26140c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.380+0000 7f261bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261414ca20 0x7f261414ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.380+0000 7f261bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140c5630 con 0x7f261414ca20 2023-11-06T20:59:19.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.380+0000 7f261bfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140aead0 con 0x7f26141399f0 2023-11-06T20:59:19.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.380+0000 7f261bfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26140c82e0 con 0x7f2600004690 2023-11-06T20:59:19.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f261a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261414ca20 0x7f261414ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f2619d9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2600004690 0x7f26140c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f261a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261414ca20 0x7f261414ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57202/0 (socket says 172.21.15.33:57202) 2023-11-06T20:59:19.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f261a59c700 1 -- 172.21.15.33:0/2182196352 learned_addr learned my addr 172.21.15.33:0/2182196352 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:19.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f261959a700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26141399f0 0x7f26140c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f261a59c700 1 -- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2600004690 msgr2=0x7f26140c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f261a59c700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2600004690 0x7f26140c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f261a59c700 1 -- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26141399f0 msgr2=0x7f26140c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f261a59c700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26141399f0 0x7f26140c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f261a59c700 1 -- 172.21.15.33:0/2182196352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140c60b0 con 0x7f261414ca20 2023-11-06T20:59:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f261a59c700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261414ca20 0x7f261414ce80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f261414ba20 tx=0x7f260c007f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f2609ffb700 1 -- 172.21.15.33:0/2182196352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00f8a0 con 0x7f261414ca20 2023-11-06T20:59:19.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.381+0000 7f2609ffb700 1 -- 172.21.15.33:0/2182196352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c00fa00 con 0x7f261414ca20 2023-11-06T20:59:19.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.382+0000 7f261bfff700 1 -- 172.21.15.33:0/2182196352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26140c5810 con 0x7f261414ca20 2023-11-06T20:59:19.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.382+0000 7f2609ffb700 1 -- 172.21.15.33:0/2182196352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00fda0 con 0x7f261414ca20 2023-11-06T20:59:19.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.382+0000 7f261bfff700 1 -- 172.21.15.33:0/2182196352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26140c5dd0 con 0x7f261414ca20 2023-11-06T20:59:19.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.383+0000 7f261bfff700 1 -- 172.21.15.33:0/2182196352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26140abca0 con 0x7f261414ca20 2023-11-06T20:59:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.383+0000 7f2609ffb700 1 -- 172.21.15.33:0/2182196352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f260c012560 con 0x7f261414ca20 2023-11-06T20:59:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.384+0000 7f2609ffb700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f25f805f270 0x7f25f8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.384+0000 7f2609ffb700 1 -- 172.21.15.33:0/2182196352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f260c079d50 con 0x7f261414ca20 2023-11-06T20:59:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.385+0000 7f2619d9b700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f25f805f270 0x7f25f8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.385+0000 7f2619d9b700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f25f805f270 0x7f25f8061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f26100095a0 tx=0x7f2610009060).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:19.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.388+0000 7f2609ffb700 1 -- 172.21.15.33:0/2182196352 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f26140abca0 con 0x7f261414ca20 2023-11-06T20:59:19.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.570+0000 7f261bfff700 1 -- 172.21.15.33:0/2182196352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f26140abca0 con 0x7f261414ca20 2023-11-06T20:59:19.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.570+0000 7f2609ffb700 1 -- 172.21.15.33:0/2182196352 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+143 (secure 0 0 0) 0x7f26140abca0 con 0x7f261414ca20 2023-11-06T20:59:19.571 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:19.571 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T20:59:19.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:19.572 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T20:59:19.572 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T20:59:19.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:19.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:19.573 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T20:59:19.573 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T20:59:19.573 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:19.573 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T20:59:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.572+0000 7f261bfff700 1 -- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f25f805f270 msgr2=0x7f25f8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.572+0000 7f261bfff700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f25f805f270 0x7f25f8061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f26100095a0 tx=0x7f2610009060).stop 2023-11-06T20:59:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.572+0000 7f261bfff700 1 -- 172.21.15.33:0/2182196352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261414ca20 msgr2=0x7f261414ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.572+0000 7f261bfff700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261414ca20 0x7f261414ce80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f261414ba20 tx=0x7f260c007f20).stop 2023-11-06T20:59:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.573+0000 7f261bfff700 1 -- 172.21.15.33:0/2182196352 shutdown_connections 2023-11-06T20:59:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.573+0000 7f261bfff700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f25f805f270 0x7f25f8061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.573+0000 7f261bfff700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2600004690 0x7f26140c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.573+0000 7f261bfff700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261414ca20 0x7f261414ce80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.573+0000 7f261bfff700 1 --2- 172.21.15.33:0/2182196352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26141399f0 0x7f26140c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.573+0000 7f261bfff700 1 -- 172.21.15.33:0/2182196352 >> 172.21.15.33:0/2182196352 conn(0x7f261405ec80 msgr2=0x7f261413a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.573+0000 7f261bfff700 1 -- 172.21.15.33:0/2182196352 shutdown_connections 2023-11-06T20:59:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.573+0000 7f261bfff700 1 -- 172.21.15.33:0/2182196352 wait complete. 2023-11-06T20:59:19.591 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T20:59:19.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T20:59:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.792+0000 7fea191ff700 1 Processor -- start 2023-11-06T20:59:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.792+0000 7fea191ff700 1 -- start start 2023-11-06T20:59:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea191ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fea140a8c10 0x7fea14153460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea191ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fea140a5de0 0x7fea140c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea191ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea140c3dd0 con 0x7fea140c3940 2023-11-06T20:59:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea191ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea14153950 con 0x7fea140a5de0 2023-11-06T20:59:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea191ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea140bd490 con 0x7fea140a8c10 2023-11-06T20:59:19.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea1259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fea140a5de0 0x7fea140c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea1259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fea140a5de0 0x7fea140c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57214/0 (socket says 172.21.15.33:57214) 2023-11-06T20:59:19.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea1259c700 1 -- 172.21.15.33:0/341481700 learned_addr learned my addr 172.21.15.33:0/341481700 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:19.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea12d9d700 1 --2- 172.21.15.33:0/341481700 >> v2:172.21.15.69:3300/0 conn(0x7fea140a8c10 0x7fea14153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea1259c700 1 -- 172.21.15.33:0/341481700 >> v2:172.21.15.69:3300/0 conn(0x7fea140a8c10 msgr2=0x7fea14153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:19.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea1259c700 1 --2- 172.21.15.33:0/341481700 >> v2:172.21.15.69:3300/0 conn(0x7fea140a8c10 0x7fea14153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea1259c700 1 -- 172.21.15.33:0/341481700 >> v1:172.21.15.69:6789/0 conn(0x7fea140c3940 legacy=0x7fea14155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.794+0000 7fea1259c700 1 -- 172.21.15.33:0/341481700 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea140bd770 con 0x7fea140a5de0 2023-11-06T20:59:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.795+0000 7fea1259c700 1 --2- 172.21.15.33:0/341481700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140a5de0 0x7fea140c3450 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fea1406be70 tx=0x7fea080090f0).ready entity=mon.0 client_cookie=b1d635f4858a79c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.795+0000 7fea11d9b700 1 -- 172.21.15.33:0/341481700 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0800b600 con 0x7fea140a5de0 2023-11-06T20:59:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.795+0000 7fea11d9b700 1 -- 172.21.15.33:0/341481700 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0800b760 con 0x7fea140a5de0 2023-11-06T20:59:19.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.795+0000 7fea11d9b700 1 -- 172.21.15.33:0/341481700 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0800bae0 con 0x7fea140a5de0 2023-11-06T20:59:19.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.795+0000 7fea191ff700 1 -- 172.21.15.33:0/341481700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140a5de0 msgr2=0x7fea140c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.795+0000 7fea191ff700 1 --2- 172.21.15.33:0/341481700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140a5de0 0x7fea140c3450 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fea1406be70 tx=0x7fea080090f0).stop 2023-11-06T20:59:19.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.795+0000 7fea191ff700 1 -- 172.21.15.33:0/341481700 shutdown_connections 2023-11-06T20:59:19.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.795+0000 7fea191ff700 1 --2- 172.21.15.33:0/341481700 >> v2:172.21.15.69:3300/0 conn(0x7fea140a8c10 0x7fea14153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.795+0000 7fea191ff700 1 --2- 172.21.15.33:0/341481700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140a5de0 0x7fea140c3450 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.795+0000 7fea191ff700 1 -- 172.21.15.33:0/341481700 >> 172.21.15.33:0/341481700 conn(0x7fea14062710 msgr2=0x7fea14062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:19.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.796+0000 7fea191ff700 1 -- 172.21.15.33:0/341481700 shutdown_connections 2023-11-06T20:59:19.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.796+0000 7fea191ff700 1 -- 172.21.15.33:0/341481700 wait complete. 2023-11-06T20:59:19.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.797+0000 7fea191ff700 1 Processor -- start 2023-11-06T20:59:19.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.797+0000 7fea191ff700 1 -- start start 2023-11-06T20:59:19.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.798+0000 7fea191ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea140a5de0 0x7fea1414b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.798+0000 7fea191ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea140a8c10 0x7fea140be190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.798+0000 7fea191ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140c3940 0x7fea140be680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.798+0000 7fea191ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1414b5c0 con 0x7fea140c3940 2023-11-06T20:59:19.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.798+0000 7fea191ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea140a99b0 con 0x7fea140a8c10 2023-11-06T20:59:19.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.798+0000 7fea191ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea140a9f70 con 0x7fea140a5de0 2023-11-06T20:59:19.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.798+0000 7fea1359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140c3940 0x7fea140be680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.798+0000 7fea12d9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea140a5de0 0x7fea1414b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea1259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea140a8c10 0x7fea140be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea1359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140c3940 0x7fea140be680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57226/0 (socket says 172.21.15.33:57226) 2023-11-06T20:59:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea1359e700 1 -- 172.21.15.33:0/422200353 learned_addr learned my addr 172.21.15.33:0/422200353 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea1359e700 1 -- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea140a5de0 msgr2=0x7fea1414b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea1359e700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea140a5de0 0x7fea1414b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea1359e700 1 -- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea140a8c10 msgr2=0x7fea140be190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:19.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea1359e700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea140a8c10 0x7fea140be190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:19.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea1359e700 1 -- 172.21.15.33:0/422200353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea08009370 con 0x7fea140c3940 2023-11-06T20:59:19.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea1359e700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140c3940 0x7fea140be680 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fea0c00c7f0 tx=0x7fea0c001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:19.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fe9faffd700 1 -- 172.21.15.33:0/422200353 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c00e4a0 con 0x7fea140c3940 2023-11-06T20:59:19.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fe9faffd700 1 -- 172.21.15.33:0/422200353 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c00e600 con 0x7fea140c3940 2023-11-06T20:59:19.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea191ff700 1 -- 172.21.15.33:0/422200353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea14154870 con 0x7fea140c3940 2023-11-06T20:59:19.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fe9faffd700 1 -- 172.21.15.33:0/422200353 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c00e980 con 0x7fea140c3940 2023-11-06T20:59:19.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.799+0000 7fea191ff700 1 -- 172.21.15.33:0/422200353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea140a9c00 con 0x7fea140c3940 2023-11-06T20:59:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.801+0000 7fea191ff700 1 -- 172.21.15.33:0/422200353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea140aa0b0 con 0x7fea140c3940 2023-11-06T20:59:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.801+0000 7fe9faffd700 1 -- 172.21.15.33:0/422200353 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fea0c00ed30 con 0x7fea140c3940 2023-11-06T20:59:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.802+0000 7fe9faffd700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fea0005f270 0x7fea00061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.802+0000 7fe9faffd700 1 -- 172.21.15.33:0/422200353 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fea0c079860 con 0x7fea140c3940 2023-11-06T20:59:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.803+0000 7fea12d9d700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fea0005f270 0x7fea00061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:19.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.803+0000 7fea12d9d700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fea0005f270 0x7fea00061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fea141567c0 tx=0x7fea04008040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:19.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.806+0000 7fe9faffd700 1 -- 172.21.15.33:0/422200353 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fea0c047af0 con 0x7fea140c3940 2023-11-06T20:59:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:19.988+0000 7fea191ff700 1 -- 172.21.15.33:0/422200353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fea140c2b50 con 0x7fea140c3940 2023-11-06T20:59:20.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.082+0000 7fe9faffd700 1 -- 172.21.15.33:0/422200353 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 152+0+0 (secure 0 0 0) 0x7fea0c04ad00 con 0x7fea140c3940 2023-11-06T20:59:20.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 -- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fea0005f270 msgr2=0x7fea00061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:20.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fea0005f270 0x7fea00061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fea141567c0 tx=0x7fea04008040).stop 2023-11-06T20:59:20.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 -- 172.21.15.33:0/422200353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140c3940 msgr2=0x7fea140be680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:20.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140c3940 0x7fea140be680 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fea0c00c7f0 tx=0x7fea0c001510).stop 2023-11-06T20:59:20.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 -- 172.21.15.33:0/422200353 shutdown_connections 2023-11-06T20:59:20.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fea0005f270 0x7fea00061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:20.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea140a5de0 0x7fea1414b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea140c3940 0x7fea140be680 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 --2- 172.21.15.33:0/422200353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea140a8c10 0x7fea140be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 -- 172.21.15.33:0/422200353 >> 172.21.15.33:0/422200353 conn(0x7fea14062710 msgr2=0x7fea1414c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.084+0000 7fea191ff700 1 -- 172.21.15.33:0/422200353 shutdown_connections 2023-11-06T20:59:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.085+0000 7fea191ff700 1 -- 172.21.15.33:0/422200353 wait complete. 2023-11-06T20:59:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T20:59:20.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T20:59:20.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.312+0000 7f5b329ff700 1 Processor -- start 2023-11-06T20:59:20.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.312+0000 7f5b329ff700 1 -- start start 2023-11-06T20:59:20.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.314+0000 7f5b329ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b2c154010 0x7f5b2c1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:20.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.314+0000 7f5b329ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5b2c0aafc0 0x7f5b2c0c0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:20.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.314+0000 7f5b329ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c0c6280 con 0x7f5b2c0c0b70 2023-11-06T20:59:20.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.314+0000 7f5b329ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2c0c7720 con 0x7f5b2c154010 2023-11-06T20:59:20.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.314+0000 7f5b329ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2c0c5990 con 0x7f5b2c0aafc0 2023-11-06T20:59:20.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.315+0000 7f5b2bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b2c154010 0x7f5b2c1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:20.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.315+0000 7f5b30f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5b2c0c0b70 0x7f5b2c064670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51760/0 (socket says 172.21.15.33:51760) 2023-11-06T20:59:20.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.315+0000 7f5b30f9c700 1 -- 172.21.15.33:0/3600938202 learned_addr learned my addr 172.21.15.33:0/3600938202 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:20.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.315+0000 7f5b2b7fe700 1 --2- 172.21.15.33:0/3600938202 >> v2:172.21.15.69:3300/0 conn(0x7f5b2c0aafc0 0x7f5b2c0c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.315+0000 7f5b2bfff700 1 -- 172.21.15.33:0/3600938202 >> v2:172.21.15.69:3300/0 conn(0x7f5b2c0aafc0 msgr2=0x7f5b2c0c0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.315+0000 7f5b2bfff700 1 --2- 172.21.15.33:0/3600938202 >> v2:172.21.15.69:3300/0 conn(0x7f5b2c0aafc0 0x7f5b2c0c0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.315+0000 7f5b2bfff700 1 -- 172.21.15.33:0/3600938202 >> v1:172.21.15.33:6789/0 conn(0x7f5b2c0c0b70 legacy=0x7f5b2c064670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.315+0000 7f5b2bfff700 1 -- 172.21.15.33:0/3600938202 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0c1000 con 0x7f5b2c154010 2023-11-06T20:59:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.315+0000 7f5b2affd700 1 -- 172.21.15.33:0/3600938202 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620267454 0 0) 0x7f5b2c0c6280 con 0x7f5b2c0c0b70 2023-11-06T20:59:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.315+0000 7f5b2bfff700 1 --2- 172.21.15.33:0/3600938202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c154010 0x7f5b2c1543f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f5b2c0c8d20 tx=0x7f5b1c009060).ready entity=mon.0 client_cookie=18f44411b5713e74 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b2affd700 1 -- 172.21.15.33:0/3600938202 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00b580 con 0x7f5b2c154010 2023-11-06T20:59:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b2affd700 1 -- 172.21.15.33:0/3600938202 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c00b6e0 con 0x7f5b2c154010 2023-11-06T20:59:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b2affd700 1 -- 172.21.15.33:0/3600938202 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00ba60 con 0x7f5b2c154010 2023-11-06T20:59:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b329ff700 1 -- 172.21.15.33:0/3600938202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c154010 msgr2=0x7f5b2c1543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:20.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b329ff700 1 --2- 172.21.15.33:0/3600938202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c154010 0x7f5b2c1543f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f5b2c0c8d20 tx=0x7f5b1c009060).stop 2023-11-06T20:59:20.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b329ff700 1 -- 172.21.15.33:0/3600938202 shutdown_connections 2023-11-06T20:59:20.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b329ff700 1 --2- 172.21.15.33:0/3600938202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c154010 0x7f5b2c1543f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:20.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b329ff700 1 --2- 172.21.15.33:0/3600938202 >> v2:172.21.15.69:3300/0 conn(0x7f5b2c0aafc0 0x7f5b2c0c0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:20.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b329ff700 1 -- 172.21.15.33:0/3600938202 >> 172.21.15.33:0/3600938202 conn(0x7f5b2c0cb8e0 msgr2=0x7f5b2c0c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:20.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b329ff700 1 -- 172.21.15.33:0/3600938202 shutdown_connections 2023-11-06T20:59:20.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.316+0000 7f5b329ff700 1 -- 172.21.15.33:0/3600938202 wait complete. 2023-11-06T20:59:20.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.318+0000 7f5b329ff700 1 Processor -- start 2023-11-06T20:59:20.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.318+0000 7f5b329ff700 1 -- start start 2023-11-06T20:59:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b329ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b2c154010 0x7f5b2c14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b329ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c0c3f30 0x7f5b2c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b329ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b2c158480 0x7f5b2c155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b329ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c1555b0 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b329ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c1549a0 con 0x7f5b2c158480 2023-11-06T20:59:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b329ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c154c80 con 0x7f5b2c154010 2023-11-06T20:59:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b2b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c0c3f30 0x7f5b2c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b2b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c0c3f30 0x7f5b2c157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57252/0 (socket says 172.21.15.33:57252) 2023-11-06T20:59:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b2b7fe700 1 -- 172.21.15.33:0/2948886020 learned_addr learned my addr 172.21.15.33:0/2948886020 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b2bfff700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b2c154010 0x7f5b2c14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.319+0000 7f5b30f9c700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b2c158480 0x7f5b2c155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b2b7fe700 1 -- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b2c154010 msgr2=0x7f5b2c14c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b2b7fe700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b2c154010 0x7f5b2c14c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b2b7fe700 1 -- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b2c158480 msgr2=0x7f5b2c155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b2b7fe700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b2c158480 0x7f5b2c155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b2b7fe700 1 -- 172.21.15.33:0/2948886020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c0092a0 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b2b7fe700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c0c3f30 0x7f5b2c157f90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f5b2000d890 tx=0x7f5b20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b13fff700 1 -- 172.21.15.33:0/2948886020 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20011590 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b13fff700 1 -- 172.21.15.33:0/2948886020 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b200116f0 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b329ff700 1 -- 172.21.15.33:0/2948886020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c154f60 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b13fff700 1 -- 172.21.15.33:0/2948886020 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20011a70 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.320+0000 7f5b329ff700 1 -- 172.21.15.33:0/2948886020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c155a40 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.321+0000 7f5b329ff700 1 -- 172.21.15.33:0/2948886020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c155760 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.322+0000 7f5b13fff700 1 -- 172.21.15.33:0/2948886020 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5b2001c460 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.323+0000 7f5b13fff700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b1805f270 0x7f5b18061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.323+0000 7f5b13fff700 1 -- 172.21.15.33:0/2948886020 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f5b20079f70 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.324+0000 7f5b2bfff700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b1805f270 0x7f5b18061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.324+0000 7f5b2bfff700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b1805f270 0x7f5b18061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f5b1c00b340 tx=0x7f5b1c017000).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.328+0000 7f5b13fff700 1 -- 172.21.15.33:0/2948886020 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5b2c155760 con 0x7f5b2c0c3f30 2023-11-06T20:59:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:20.513+0000 7f5b329ff700 1 -- 172.21.15.33:0/2948886020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5b2c1556f0 con 0x7f5b2c0c3f30 2023-11-06T20:59:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.090+0000 7f5b13fff700 1 -- 172.21.15.33:0/2948886020 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5b2004ebb0 con 0x7f5b2c0c3f30 2023-11-06T20:59:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.094+0000 7f5b329ff700 1 -- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b1805f270 msgr2=0x7f5b18061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.094+0000 7f5b329ff700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b1805f270 0x7f5b18061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f5b1c00b340 tx=0x7f5b1c017000).stop 2023-11-06T20:59:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.094+0000 7f5b329ff700 1 -- 172.21.15.33:0/2948886020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c0c3f30 msgr2=0x7f5b2c157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.094+0000 7f5b329ff700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c0c3f30 0x7f5b2c157f90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f5b2000d890 tx=0x7f5b20008040).stop 2023-11-06T20:59:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.094+0000 7f5b329ff700 1 -- 172.21.15.33:0/2948886020 shutdown_connections 2023-11-06T20:59:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.094+0000 7f5b329ff700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b1805f270 0x7f5b18061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.094+0000 7f5b329ff700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b2c154010 0x7f5b2c14c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.094+0000 7f5b329ff700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b2c158480 0x7f5b2c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.094+0000 7f5b329ff700 1 --2- 172.21.15.33:0/2948886020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b2c0c3f30 0x7f5b2c157f90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.094+0000 7f5b329ff700 1 -- 172.21.15.33:0/2948886020 >> 172.21.15.33:0/2948886020 conn(0x7f5b2c0cb8e0 msgr2=0x7f5b2c14d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:21.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.096+0000 7f5b329ff700 1 -- 172.21.15.33:0/2948886020 shutdown_connections 2023-11-06T20:59:21.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.096+0000 7f5b329ff700 1 -- 172.21.15.33:0/2948886020 wait complete. 2023-11-06T20:59:21.104 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T20:59:21.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T20:59:21.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.334+0000 7f233a1ff700 1 Processor -- start 2023-11-06T20:59:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.334+0000 7f233a1ff700 1 -- start start 2023-11-06T20:59:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.335+0000 7f233a1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2334154330 0x7f23340c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.335+0000 7f233a1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23340c6b60 con 0x7f23340c7ff0 2023-11-06T20:59:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.335+0000 7f233a1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23340c7700 con 0x7f2334154710 2023-11-06T20:59:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.335+0000 7f233a1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23340c5cd0 con 0x7f2334154330 2023-11-06T20:59:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.336+0000 7f23337fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2334154330 0x7f23340c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.336+0000 7f23337fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2334154330 0x7f23340c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57254/0 (socket says 172.21.15.33:57254) 2023-11-06T20:59:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.336+0000 7f23337fe700 1 -- 172.21.15.33:0/2288998953 learned_addr learned my addr 172.21.15.33:0/2288998953 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.336+0000 7f2333fff700 1 --1- 172.21.15.33:0/2288998953 >> v1:172.21.15.33:6789/0 conn(0x7f23340c7ff0 0x7f23340646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51770/0 (socket says 172.21.15.33:51770) 2023-11-06T20:59:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.336+0000 7f23337fe700 1 -- 172.21.15.33:0/2288998953 >> v1:172.21.15.69:6789/0 conn(0x7f2334154710 legacy=0x7f23340c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.336+0000 7f23337fe700 1 -- 172.21.15.33:0/2288998953 >> v1:172.21.15.33:6789/0 conn(0x7f23340c7ff0 legacy=0x7f23340646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.336+0000 7f23327fc700 1 -- 172.21.15.33:0/2288998953 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668344398 0 0) 0x7f23340c6b60 con 0x7f23340c7ff0 2023-11-06T20:59:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.336+0000 7f23337fe700 1 -- 172.21.15.33:0/2288998953 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340c5f50 con 0x7f2334154330 2023-11-06T20:59:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.336+0000 7f23337fe700 1 --2- 172.21.15.33:0/2288998953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154330 0x7f23340c57e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f2324006fd0 tx=0x7f23240090f0).ready entity=mon.0 client_cookie=2867b3613ec529c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.337+0000 7f23327fc700 1 -- 172.21.15.33:0/2288998953 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232400b600 con 0x7f2334154330 2023-11-06T20:59:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.337+0000 7f23327fc700 1 -- 172.21.15.33:0/2288998953 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232400b760 con 0x7f2334154330 2023-11-06T20:59:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.337+0000 7f23327fc700 1 -- 172.21.15.33:0/2288998953 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232400bae0 con 0x7f2334154330 2023-11-06T20:59:21.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.337+0000 7f233a1ff700 1 -- 172.21.15.33:0/2288998953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154330 msgr2=0x7f23340c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.337+0000 7f233a1ff700 1 --2- 172.21.15.33:0/2288998953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154330 0x7f23340c57e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f2324006fd0 tx=0x7f23240090f0).stop 2023-11-06T20:59:21.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.337+0000 7f233a1ff700 1 -- 172.21.15.33:0/2288998953 shutdown_connections 2023-11-06T20:59:21.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.337+0000 7f233a1ff700 1 --2- 172.21.15.33:0/2288998953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154330 0x7f23340c57e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.337+0000 7f233a1ff700 1 -- 172.21.15.33:0/2288998953 >> 172.21.15.33:0/2288998953 conn(0x7f23340cb910 msgr2=0x7f23340c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:21.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.337+0000 7f233a1ff700 1 -- 172.21.15.33:0/2288998953 shutdown_connections 2023-11-06T20:59:21.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.337+0000 7f233a1ff700 1 -- 172.21.15.33:0/2288998953 wait complete. 2023-11-06T20:59:21.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.339+0000 7f233a1ff700 1 Processor -- start 2023-11-06T20:59:21.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.339+0000 7f233a1ff700 1 -- start start 2023-11-06T20:59:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f233a1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23340c7ff0 0x7f2334151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f233a1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2334154330 0x7f23340ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f233a1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154710 0x7f2334156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f233a1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340af710 con 0x7f2334154710 2023-11-06T20:59:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f233a1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2334155040 con 0x7f2334154330 2023-11-06T20:59:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f233a1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23340af150 con 0x7f23340c7ff0 2023-11-06T20:59:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f2333fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154710 0x7f2334156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f2333fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154710 0x7f2334156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57264/0 (socket says 172.21.15.33:57264) 2023-11-06T20:59:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f2333fff700 1 -- 172.21.15.33:0/3880898538 learned_addr learned my addr 172.21.15.33:0/3880898538 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f23337fe700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23340c7ff0 0x7f2334151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.340+0000 7f2332ffd700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2334154330 0x7f23340ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f2333fff700 1 -- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23340c7ff0 msgr2=0x7f2334151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f2333fff700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23340c7ff0 0x7f2334151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f2333fff700 1 -- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2334154330 msgr2=0x7f23340ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f2333fff700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2334154330 0x7f23340ae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f2333fff700 1 -- 172.21.15.33:0/3880898538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2324009370 con 0x7f2334154710 2023-11-06T20:59:21.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f2332ffd700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2334154330 0x7f23340ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:21.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f2333fff700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154710 0x7f2334156bf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f232c00c730 tx=0x7f232c001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:21.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f231b7fe700 1 -- 172.21.15.33:0/3880898538 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c005c70 con 0x7f2334154710 2023-11-06T20:59:21.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f231b7fe700 1 -- 172.21.15.33:0/3880898538 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c005dd0 con 0x7f2334154710 2023-11-06T20:59:21.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f231b7fe700 1 -- 172.21.15.33:0/3880898538 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c007340 con 0x7f2334154710 2023-11-06T20:59:21.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f233a1ff700 1 -- 172.21.15.33:0/3880898538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23340af430 con 0x7f2334154710 2023-11-06T20:59:21.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.341+0000 7f233a1ff700 1 -- 172.21.15.33:0/3880898538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23341522d0 con 0x7f2334154710 2023-11-06T20:59:21.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.342+0000 7f233a1ff700 1 -- 172.21.15.33:0/3880898538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23341551f0 con 0x7f2334154710 2023-11-06T20:59:21.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.344+0000 7f231b7fe700 1 -- 172.21.15.33:0/3880898538 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f232c0076f0 con 0x7f2334154710 2023-11-06T20:59:21.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.344+0000 7f231b7fe700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f232005f270 0x7f2320061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.344+0000 7f231b7fe700 1 -- 172.21.15.33:0/3880898538 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f232c07a3b0 con 0x7f2334154710 2023-11-06T20:59:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.345+0000 7f23337fe700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f232005f270 0x7f2320061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.345+0000 7f23337fe700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f232005f270 0x7f2320061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f2324009130 tx=0x7f2324017000).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.349+0000 7f231b7fe700 1 -- 172.21.15.33:0/3880898538 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f23341551f0 con 0x7f2334154710 2023-11-06T20:59:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.573+0000 7f233a1ff700 1 -- 172.21.15.33:0/3880898538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f23340ad370 con 0x7f2334154710 2023-11-06T20:59:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.579+0000 7f231b7fe700 1 -- 172.21.15.33:0/3880898538 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v26) v1 ==== 173+0+0 (secure 0 0 0) 0x7f232c04eed0 con 0x7f2334154710 2023-11-06T20:59:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.584+0000 7f233a1ff700 1 -- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f232005f270 msgr2=0x7f2320061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.584+0000 7f233a1ff700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f232005f270 0x7f2320061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f2324009130 tx=0x7f2324017000).stop 2023-11-06T20:59:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.584+0000 7f233a1ff700 1 -- 172.21.15.33:0/3880898538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154710 msgr2=0x7f2334156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.584+0000 7f233a1ff700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154710 0x7f2334156bf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f232c00c730 tx=0x7f232c001520).stop 2023-11-06T20:59:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.584+0000 7f233a1ff700 1 -- 172.21.15.33:0/3880898538 shutdown_connections 2023-11-06T20:59:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.584+0000 7f233a1ff700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f232005f270 0x7f2320061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.584+0000 7f233a1ff700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23340c7ff0 0x7f2334151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.584+0000 7f233a1ff700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2334154710 0x7f2334156bf0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.584+0000 7f233a1ff700 1 --2- 172.21.15.33:0/3880898538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2334154330 0x7f23340ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.584+0000 7f233a1ff700 1 -- 172.21.15.33:0/3880898538 >> 172.21.15.33:0/3880898538 conn(0x7f23340cb910 msgr2=0x7f23340c9050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.586+0000 7f233a1ff700 1 -- 172.21.15.33:0/3880898538 shutdown_connections 2023-11-06T20:59:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.586+0000 7f233a1ff700 1 -- 172.21.15.33:0/3880898538 wait complete. 2023-11-06T20:59:21.600 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 9 and data pool 10 2023-11-06T20:59:21.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.824+0000 7fc51adff700 1 Processor -- start 2023-11-06T20:59:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.824+0000 7fc51adff700 1 -- start start 2023-11-06T20:59:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.825+0000 7fc51adff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc5140c5d50 0x7fc5140c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.825+0000 7fc51adff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc5140bfd70 0x7fc5140c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.825+0000 7fc51adff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5141554a0 con 0x7fc5140c1840 2023-11-06T20:59:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.825+0000 7fc51adff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc514154960 con 0x7fc5140bfd70 2023-11-06T20:59:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.825+0000 7fc51adff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5140c66c0 con 0x7fc5140c5d50 2023-11-06T20:59:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.826+0000 7fc51939c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc5140bfd70 0x7fc5140c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.826+0000 7fc51939c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc5140bfd70 0x7fc5140c0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57276/0 (socket says 172.21.15.33:57276) 2023-11-06T20:59:21.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.826+0000 7fc51939c700 1 -- 172.21.15.33:0/279470281 learned_addr learned my addr 172.21.15.33:0/279470281 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:21.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.826+0000 7fc513fff700 1 --2- 172.21.15.33:0/279470281 >> v2:172.21.15.69:3300/0 conn(0x7fc5140c5d50 0x7fc5140c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.826+0000 7fc51939c700 1 -- 172.21.15.33:0/279470281 >> v2:172.21.15.69:3300/0 conn(0x7fc5140c5d50 msgr2=0x7fc5140c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:21.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.826+0000 7fc51939c700 1 --2- 172.21.15.33:0/279470281 >> v2:172.21.15.69:3300/0 conn(0x7fc5140c5d50 0x7fc5140c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.826+0000 7fc51939c700 1 -- 172.21.15.33:0/279470281 >> v1:172.21.15.33:6789/0 conn(0x7fc5140c1840 legacy=0x7fc5140a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.826+0000 7fc51939c700 1 -- 172.21.15.33:0/279470281 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc514155930 con 0x7fc5140bfd70 2023-11-06T20:59:21.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.826+0000 7fc5137fe700 1 -- 172.21.15.33:0/279470281 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333136890 0 0) 0x7fc5141554a0 con 0x7fc5140c1840 2023-11-06T20:59:21.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.826+0000 7fc51939c700 1 --2- 172.21.15.33:0/279470281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140bfd70 0x7fc5140c0180 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fc5140adb40 tx=0x7fc50c009840).ready entity=mon.0 client_cookie=f44953eca3281493 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:21.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc5137fe700 1 -- 172.21.15.33:0/279470281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00e070 con 0x7fc5140bfd70 2023-11-06T20:59:21.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc5137fe700 1 -- 172.21.15.33:0/279470281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c00ba20 con 0x7fc5140bfd70 2023-11-06T20:59:21.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc5137fe700 1 -- 172.21.15.33:0/279470281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00bda0 con 0x7fc5140bfd70 2023-11-06T20:59:21.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc51adff700 1 -- 172.21.15.33:0/279470281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140bfd70 msgr2=0x7fc5140c0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc51adff700 1 --2- 172.21.15.33:0/279470281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140bfd70 0x7fc5140c0180 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fc5140adb40 tx=0x7fc50c009840).stop 2023-11-06T20:59:21.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc51adff700 1 -- 172.21.15.33:0/279470281 shutdown_connections 2023-11-06T20:59:21.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc51adff700 1 --2- 172.21.15.33:0/279470281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140bfd70 0x7fc5140c0180 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc51adff700 1 --2- 172.21.15.33:0/279470281 >> v2:172.21.15.69:3300/0 conn(0x7fc5140c5d50 0x7fc5140c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc51adff700 1 -- 172.21.15.33:0/279470281 >> 172.21.15.33:0/279470281 conn(0x7fc51405ec80 msgr2=0x7fc5140cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:21.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc51adff700 1 -- 172.21.15.33:0/279470281 shutdown_connections 2023-11-06T20:59:21.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.827+0000 7fc51adff700 1 -- 172.21.15.33:0/279470281 wait complete. 2023-11-06T20:59:21.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.829+0000 7fc51adff700 1 Processor -- start 2023-11-06T20:59:21.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.829+0000 7fc51adff700 1 -- start start 2023-11-06T20:59:21.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc51adff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc5140bfd70 0x7fc5140c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc51adff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5140c1840 0x7fc514149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc51adff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140c5d50 0x7fc514157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc51adff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140c2e80 con 0x7fc5140c5d50 2023-11-06T20:59:21.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc51adff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140c2880 con 0x7fc5140c1840 2023-11-06T20:59:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc51adff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140c4160 con 0x7fc5140bfd70 2023-11-06T20:59:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc51939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140c5d50 0x7fc514157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc51939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140c5d50 0x7fc514157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57278/0 (socket says 172.21.15.33:57278) 2023-11-06T20:59:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc51939c700 1 -- 172.21.15.33:0/2497283513 learned_addr learned my addr 172.21.15.33:0/2497283513 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc518b9b700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc5140bfd70 0x7fc5140c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.830+0000 7fc513fff700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5140c1840 0x7fc514149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc51939c700 1 -- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc5140bfd70 msgr2=0x7fc5140c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc51939c700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc5140bfd70 0x7fc5140c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc51939c700 1 -- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5140c1840 msgr2=0x7fc514149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:21.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc51939c700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5140c1840 0x7fc514149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:21.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc51939c700 1 -- 172.21.15.33:0/2497283513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c009ae0 con 0x7fc5140c5d50 2023-11-06T20:59:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc51939c700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140c5d50 0x7fc514157410 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fc51406bc90 tx=0x7fc514155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc510ff9700 1 -- 172.21.15.33:0/2497283513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0092e0 con 0x7fc5140c5d50 2023-11-06T20:59:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc510ff9700 1 -- 172.21.15.33:0/2497283513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c009500 con 0x7fc5140c5d50 2023-11-06T20:59:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc51adff700 1 -- 172.21.15.33:0/2497283513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5140c4420 con 0x7fc5140c5d50 2023-11-06T20:59:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc510ff9700 1 -- 172.21.15.33:0/2497283513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c01e6c0 con 0x7fc5140c5d50 2023-11-06T20:59:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.831+0000 7fc51adff700 1 -- 172.21.15.33:0/2497283513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5140c2b10 con 0x7fc5140c5d50 2023-11-06T20:59:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.832+0000 7fc51adff700 1 -- 172.21.15.33:0/2497283513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51406c6c0 con 0x7fc5140c5d50 2023-11-06T20:59:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.833+0000 7fc510ff9700 1 -- 172.21.15.33:0/2497283513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc50c003040 con 0x7fc5140c5d50 2023-11-06T20:59:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.835+0000 7fc510ff9700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4f805f270 0x7fc4f8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:21.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.835+0000 7fc510ff9700 1 -- 172.21.15.33:0/2497283513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc50c07b730 con 0x7fc5140c5d50 2023-11-06T20:59:21.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.835+0000 7fc518b9b700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4f805f270 0x7fc4f8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:21.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.836+0000 7fc518b9b700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4f805f270 0x7fc4f8061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fc508007fd0 tx=0x7fc508007bb0).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:21.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:21.839+0000 7fc510ff9700 1 -- 172.21.15.33:0/2497283513 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc50c04c930 con 0x7fc5140c5d50 2023-11-06T20:59:22.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.022+0000 7fc51adff700 1 -- 172.21.15.33:0/2497283513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc514063f20 con 0x7fc5140c5d50 2023-11-06T20:59:22.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.023+0000 7fc510ff9700 1 -- 172.21.15.33:0/2497283513 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16991 (secure 0 0 0) 0x7fc50c023020 con 0x7fc5140c5d50 2023-11-06T20:59:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:22.024 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":83,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:21.575491+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:20.035584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:20.561335+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:22.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 -- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4f805f270 msgr2=0x7fc4f8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4f805f270 0x7fc4f8061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fc508007fd0 tx=0x7fc508007bb0).stop 2023-11-06T20:59:22.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 -- 172.21.15.33:0/2497283513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140c5d50 msgr2=0x7fc514157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140c5d50 0x7fc514157410 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fc51406bc90 tx=0x7fc514155410).stop 2023-11-06T20:59:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 -- 172.21.15.33:0/2497283513 shutdown_connections 2023-11-06T20:59:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc4f805f270 0x7fc4f8061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc5140bfd70 0x7fc5140c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5140c5d50 0x7fc514157410 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 --2- 172.21.15.33:0/2497283513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5140c1840 0x7fc514149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 -- 172.21.15.33:0/2497283513 >> 172.21.15.33:0/2497283513 conn(0x7fc51405ec80 msgr2=0x7fc5140ca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:22.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 -- 172.21.15.33:0/2497283513 shutdown_connections 2023-11-06T20:59:22.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.025+0000 7fc51adff700 1 -- 172.21.15.33:0/2497283513 wait complete. 2023-11-06T20:59:22.044 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:22.065 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-11-06T20:59:22.065+0000 7f801d9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T20:59:22.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.242+0000 7f3f597ff700 1 Processor -- start 2023-11-06T20:59:22.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.243+0000 7f3f597ff700 1 -- start start 2023-11-06T20:59:22.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.244+0000 7f3f597ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3f540c1840 0x7f3f540a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.244+0000 7f3f597ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3f540aa2d0 0x7f3f540c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.244+0000 7f3f597ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c6240 0x7f3f54157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.244+0000 7f3f597ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f54154960 con 0x7f3f540aa2d0 2023-11-06T20:59:22.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.244+0000 7f3f597ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f541554a0 con 0x7f3f540c1840 2023-11-06T20:59:22.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.244+0000 7f3f597ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54155930 con 0x7f3f540c6240 2023-11-06T20:59:22.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.244+0000 7f3f527fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3f540aa2d0 0x7f3f540c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.244+0000 7f3f527fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3f540aa2d0 0x7f3f540c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57284/0 (socket says 172.21.15.33:57284) 2023-11-06T20:59:22.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.244+0000 7f3f527fc700 1 -- 172.21.15.33:0/1329630791 learned_addr learned my addr 172.21.15.33:0/1329630791 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:22.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f537fe700 1 --2- 172.21.15.33:0/1329630791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c6240 0x7f3f54157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f52ffd700 1 --2- 172.21.15.33:0/1329630791 >> v2:172.21.15.69:3300/0 conn(0x7f3f540c1840 0x7f3f540a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f527fc700 1 -- 172.21.15.33:0/1329630791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c6240 msgr2=0x7f3f54157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:22.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f527fc700 1 --2- 172.21.15.33:0/1329630791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c6240 0x7f3f54157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f527fc700 1 -- 172.21.15.33:0/1329630791 >> v2:172.21.15.69:3300/0 conn(0x7f3f540c1840 msgr2=0x7f3f540a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:22.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f527fc700 1 --2- 172.21.15.33:0/1329630791 >> v2:172.21.15.69:3300/0 conn(0x7f3f540c1840 0x7f3f540a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f527fc700 1 -- 172.21.15.33:0/1329630791 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540c6690 con 0x7f3f540aa2d0 2023-11-06T20:59:22.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f52ffd700 1 --2- 172.21.15.33:0/1329630791 >> v2:172.21.15.69:3300/0 conn(0x7f3f540c1840 0x7f3f540a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:22.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f537fe700 1 --2- 172.21.15.33:0/1329630791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c6240 0x7f3f54157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:22.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f527fc700 1 --2- 172.21.15.33:0/1329630791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540aa2d0 0x7f3f540c5d50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f3f540840b0 tx=0x7f3f480090f0).ready entity=mon.0 client_cookie=fe20bde736a8323a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:22.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f51ffb700 1 -- 172.21.15.33:0/1329630791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800b600 con 0x7f3f540aa2d0 2023-11-06T20:59:22.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f51ffb700 1 -- 172.21.15.33:0/1329630791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4800b760 con 0x7f3f540aa2d0 2023-11-06T20:59:22.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.245+0000 7f3f51ffb700 1 -- 172.21.15.33:0/1329630791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800bae0 con 0x7f3f540aa2d0 2023-11-06T20:59:22.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.246+0000 7f3f597ff700 1 -- 172.21.15.33:0/1329630791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540aa2d0 msgr2=0x7f3f540c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.246+0000 7f3f597ff700 1 --2- 172.21.15.33:0/1329630791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540aa2d0 0x7f3f540c5d50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f3f540840b0 tx=0x7f3f480090f0).stop 2023-11-06T20:59:22.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.246+0000 7f3f597ff700 1 -- 172.21.15.33:0/1329630791 shutdown_connections 2023-11-06T20:59:22.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.246+0000 7f3f597ff700 1 --2- 172.21.15.33:0/1329630791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c6240 0x7f3f54157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.246+0000 7f3f597ff700 1 --2- 172.21.15.33:0/1329630791 >> v2:172.21.15.69:3300/0 conn(0x7f3f540c1840 0x7f3f540a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.246+0000 7f3f597ff700 1 --2- 172.21.15.33:0/1329630791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540aa2d0 0x7f3f540c5d50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.246+0000 7f3f597ff700 1 -- 172.21.15.33:0/1329630791 >> 172.21.15.33:0/1329630791 conn(0x7f3f5405ec80 msgr2=0x7f3f540cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:22.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.246+0000 7f3f597ff700 1 -- 172.21.15.33:0/1329630791 shutdown_connections 2023-11-06T20:59:22.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.246+0000 7f3f597ff700 1 -- 172.21.15.33:0/1329630791 wait complete. 2023-11-06T20:59:22.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.247+0000 7f3f597ff700 1 Processor -- start 2023-11-06T20:59:22.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.247+0000 7f3f597ff700 1 -- start start 2023-11-06T20:59:22.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.248+0000 7f3f597ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f540aa2d0 0x7f3f540b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.248+0000 7f3f597ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c1840 0x7f3f5414f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.248+0000 7f3f597ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540c6240 0x7f3f540ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.248+0000 7f3f597ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540b07c0 con 0x7f3f540c6240 2023-11-06T20:59:22.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.248+0000 7f3f597ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54155aa0 con 0x7f3f540aa2d0 2023-11-06T20:59:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.248+0000 7f3f597ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5414fe20 con 0x7f3f540c1840 2023-11-06T20:59:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540c6240 0x7f3f540ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f527fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c1840 0x7f3f5414f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f52ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f540aa2d0 0x7f3f540b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540c6240 0x7f3f540ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57296/0 (socket says 172.21.15.33:57296) 2023-11-06T20:59:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f537fe700 1 -- 172.21.15.33:0/3699644559 learned_addr learned my addr 172.21.15.33:0/3699644559 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f527fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c1840 0x7f3f5414f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:51188/0 (socket says 172.21.15.33:51188) 2023-11-06T20:59:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f537fe700 1 -- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c1840 msgr2=0x7f3f5414f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f537fe700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c1840 0x7f3f5414f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f537fe700 1 -- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f540aa2d0 msgr2=0x7f3f540b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f537fe700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f540aa2d0 0x7f3f540b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f537fe700 1 -- 172.21.15.33:0/3699644559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f48009370 con 0x7f3f540c6240 2023-11-06T20:59:22.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.249+0000 7f3f537fe700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540c6240 0x7f3f540ae770 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f3f4c00a380 tx=0x7f3f4c007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:22.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.250+0000 7f3f3affd700 1 -- 172.21.15.33:0/3699644559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c016470 con 0x7f3f540c6240 2023-11-06T20:59:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.250+0000 7f3f3affd700 1 -- 172.21.15.33:0/3699644559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c0165d0 con 0x7f3f540c6240 2023-11-06T20:59:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.250+0000 7f3f597ff700 1 -- 172.21.15.33:0/3699644559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f54155cf0 con 0x7f3f540c6240 2023-11-06T20:59:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.250+0000 7f3f3affd700 1 -- 172.21.15.33:0/3699644559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c016950 con 0x7f3f540c6240 2023-11-06T20:59:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.250+0000 7f3f597ff700 1 -- 172.21.15.33:0/3699644559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f541562b0 con 0x7f3f540c6240 2023-11-06T20:59:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.251+0000 7f3f597ff700 1 -- 172.21.15.33:0/3699644559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5406c6c0 con 0x7f3f540c6240 2023-11-06T20:59:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.252+0000 7f3f3affd700 1 -- 172.21.15.33:0/3699644559 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3f4c013070 con 0x7f3f540c6240 2023-11-06T20:59:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.253+0000 7f3f3affd700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f4005f270 0x7f3f40061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.253+0000 7f3f3affd700 1 -- 172.21.15.33:0/3699644559 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3f4c07b0f0 con 0x7f3f540c6240 2023-11-06T20:59:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.253+0000 7f3f52ffd700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f4005f270 0x7f3f40061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.254+0000 7f3f52ffd700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f4005f270 0x7f3f40061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3f540c2820 tx=0x7f3f44008040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.257+0000 7f3f3affd700 1 -- 172.21.15.33:0/3699644559 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3f4c04c230 con 0x7f3f540c6240 2023-11-06T20:59:22.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.438+0000 7f3f597ff700 1 -- 172.21.15.33:0/3699644559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f54063f20 con 0x7f3f540c6240 2023-11-06T20:59:22.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.439+0000 7f3f3affd700 1 -- 172.21.15.33:0/3699644559 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16991 (secure 0 0 0) 0x7f3f4c04f850 con 0x7f3f540c6240 2023-11-06T20:59:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:22.440 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":83,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:21.575491+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:20.035584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:20.561335+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:22.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 -- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f4005f270 msgr2=0x7f3f40061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f4005f270 0x7f3f40061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3f540c2820 tx=0x7f3f44008040).stop 2023-11-06T20:59:22.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 -- 172.21.15.33:0/3699644559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540c6240 msgr2=0x7f3f540ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540c6240 0x7f3f540ae770 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f3f4c00a380 tx=0x7f3f4c007900).stop 2023-11-06T20:59:22.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 -- 172.21.15.33:0/3699644559 shutdown_connections 2023-11-06T20:59:22.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3f4005f270 0x7f3f40061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f540c6240 0x7f3f540ae770 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f540aa2d0 0x7f3f540b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 --2- 172.21.15.33:0/3699644559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f540c1840 0x7f3f5414f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 -- 172.21.15.33:0/3699644559 >> 172.21.15.33:0/3699644559 conn(0x7f3f5405ec80 msgr2=0x7f3f540c0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:22.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 -- 172.21.15.33:0/3699644559 shutdown_connections 2023-11-06T20:59:22.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.441+0000 7f3f597ff700 1 -- 172.21.15.33:0/3699644559 wait complete. 2023-11-06T20:59:22.459 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T20:59:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.699+0000 7fa2da3ff700 1 Processor -- start 2023-11-06T20:59:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.699+0000 7fa2da3ff700 1 -- start start 2023-11-06T20:59:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.701+0000 7fa2da3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d413dff0 0x7fa2d4157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.701+0000 7fa2da3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa2d40c4b00 0x7fa2d40c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.701+0000 7fa2da3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d4064bf0 con 0x7fa2d40c4720 2023-11-06T20:59:22.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.701+0000 7fa2da3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d405e490 con 0x7fa2d40c4b00 2023-11-06T20:59:22.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.701+0000 7fa2da3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4062e10 con 0x7fa2d413dff0 2023-11-06T20:59:22.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.701+0000 7fa2d899c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa2d40c4b00 0x7fa2d40c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.701+0000 7fa2d899c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa2d40c4b00 0x7fa2d40c2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57304/0 (socket says 172.21.15.33:57304) 2023-11-06T20:59:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.701+0000 7fa2d899c700 1 -- 172.21.15.33:0/3017890978 learned_addr learned my addr 172.21.15.33:0/3017890978 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.702+0000 7fa2d37fe700 1 --2- 172.21.15.33:0/3017890978 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d413dff0 0x7fa2d4157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.702+0000 7fa2d899c700 1 -- 172.21.15.33:0/3017890978 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d413dff0 msgr2=0x7fa2d4157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.702+0000 7fa2d899c700 1 --2- 172.21.15.33:0/3017890978 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d413dff0 0x7fa2d4157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.702+0000 7fa2d899c700 1 -- 172.21.15.33:0/3017890978 >> v1:172.21.15.69:6789/0 conn(0x7fa2d40c4720 legacy=0x7fa2d413d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.702+0000 7fa2d899c700 1 -- 172.21.15.33:0/3017890978 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d40630f0 con 0x7fa2d40c4b00 2023-11-06T20:59:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.702+0000 7fa2d37fe700 1 --2- 172.21.15.33:0/3017890978 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d413dff0 0x7fa2d4157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:22.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.702+0000 7fa2d899c700 1 --2- 172.21.15.33:0/3017890978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c4b00 0x7fa2d40c2860 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fa2cc009f20 tx=0x7fa2cc009840).ready entity=mon.0 client_cookie=be6516fe43ffb18e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:22.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.702+0000 7fa2d2ffd700 1 -- 172.21.15.33:0/3017890978 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00e070 con 0x7fa2d40c4b00 2023-11-06T20:59:22.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.702+0000 7fa2d2ffd700 1 -- 172.21.15.33:0/3017890978 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc00ba80 con 0x7fa2d40c4b00 2023-11-06T20:59:22.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.702+0000 7fa2d2ffd700 1 -- 172.21.15.33:0/3017890978 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00be00 con 0x7fa2d40c4b00 2023-11-06T20:59:22.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.703+0000 7fa2da3ff700 1 -- 172.21.15.33:0/3017890978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c4b00 msgr2=0x7fa2d40c2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.703+0000 7fa2da3ff700 1 --2- 172.21.15.33:0/3017890978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c4b00 0x7fa2d40c2860 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fa2cc009f20 tx=0x7fa2cc009840).stop 2023-11-06T20:59:22.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.703+0000 7fa2da3ff700 1 -- 172.21.15.33:0/3017890978 shutdown_connections 2023-11-06T20:59:22.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.703+0000 7fa2da3ff700 1 --2- 172.21.15.33:0/3017890978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c4b00 0x7fa2d40c2860 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.703+0000 7fa2da3ff700 1 --2- 172.21.15.33:0/3017890978 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d413dff0 0x7fa2d4157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.703+0000 7fa2da3ff700 1 -- 172.21.15.33:0/3017890978 >> 172.21.15.33:0/3017890978 conn(0x7fa2d40c7dc0 msgr2=0x7fa2d40c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:22.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.703+0000 7fa2da3ff700 1 -- 172.21.15.33:0/3017890978 shutdown_connections 2023-11-06T20:59:22.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.703+0000 7fa2da3ff700 1 -- 172.21.15.33:0/3017890978 wait complete. 2023-11-06T20:59:22.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.705+0000 7fa2da3ff700 1 Processor -- start 2023-11-06T20:59:22.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.705+0000 7fa2da3ff700 1 -- start start 2023-11-06T20:59:22.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2da3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40c4720 0x7fa2d4156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2da3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c4b00 0x7fa2d4152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2da3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d413dff0 0x7fa2d41679d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2da3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4135670 con 0x7fa2d413dff0 2023-11-06T20:59:22.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2da3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4135910 con 0x7fa2d40c4720 2023-11-06T20:59:22.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2da3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d41573c0 con 0x7fa2d40c4b00 2023-11-06T20:59:22.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d413dff0 0x7fa2d41679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d413dff0 0x7fa2d41679d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57320/0 (socket says 172.21.15.33:57320) 2023-11-06T20:59:22.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d899c700 1 -- 172.21.15.33:0/1104575322 learned_addr learned my addr 172.21.15.33:0/1104575322 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:22.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d899c700 1 -- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c4b00 msgr2=0x7fa2d4152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d37fe700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c4b00 0x7fa2d4152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d3fff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40c4720 0x7fa2d4156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d899c700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c4b00 0x7fa2d4152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d899c700 1 -- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40c4720 msgr2=0x7fa2d4156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:22.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d899c700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40c4720 0x7fa2d4156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:22.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d899c700 1 -- 172.21.15.33:0/1104575322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc009ae0 con 0x7fa2d413dff0 2023-11-06T20:59:22.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d37fe700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c4b00 0x7fa2d4152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:22.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.707+0000 7fa2d3fff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40c4720 0x7fa2d4156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:22.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.708+0000 7fa2d899c700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d413dff0 0x7fa2d41679d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fa2cc00ba50 tx=0x7fa2cc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:22.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.708+0000 7fa2bbfff700 1 -- 172.21.15.33:0/1104575322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc009500 con 0x7fa2d413dff0 2023-11-06T20:59:22.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.708+0000 7fa2bbfff700 1 -- 172.21.15.33:0/1104575322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc0032f0 con 0x7fa2d413dff0 2023-11-06T20:59:22.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.708+0000 7fa2da3ff700 1 -- 172.21.15.33:0/1104575322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d41576a0 con 0x7fa2d413dff0 2023-11-06T20:59:22.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.708+0000 7fa2bbfff700 1 -- 172.21.15.33:0/1104575322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc01d690 con 0x7fa2d413dff0 2023-11-06T20:59:22.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.708+0000 7fa2da3ff700 1 -- 172.21.15.33:0/1104575322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d4157060 con 0x7fa2d413dff0 2023-11-06T20:59:22.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.709+0000 7fa2da3ff700 1 -- 172.21.15.33:0/1104575322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d4135a80 con 0x7fa2d413dff0 2023-11-06T20:59:22.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.710+0000 7fa2bbfff700 1 -- 172.21.15.33:0/1104575322 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa2cc003040 con 0x7fa2d413dff0 2023-11-06T20:59:22.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.711+0000 7fa2bbfff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2c005f270 0x7fa2c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:22.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.711+0000 7fa2bbfff700 1 -- 172.21.15.33:0/1104575322 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa2cc07b860 con 0x7fa2d413dff0 2023-11-06T20:59:22.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.712+0000 7fa2d3fff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2c005f270 0x7fa2c0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:22.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.712+0000 7fa2d3fff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2c005f270 0x7fa2c0061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa2c4001670 tx=0x7fa2c4009040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:22.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.716+0000 7fa2bbfff700 1 -- 172.21.15.33:0/1104575322 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa2cc04c8f0 con 0x7fa2d413dff0 2023-11-06T20:59:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:22.942+0000 7fa2da3ff700 1 -- 172.21.15.33:0/1104575322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa2d40aa0d0 con 0x7fa2d413dff0 2023-11-06T20:59:23.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.583+0000 7fa2bbfff700 1 -- 172.21.15.33:0/1104575322 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa2cc022020 con 0x7fa2d413dff0 2023-11-06T20:59:23.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.585+0000 7fa2da3ff700 1 -- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2c005f270 msgr2=0x7fa2c0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:23.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.585+0000 7fa2da3ff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2c005f270 0x7fa2c0061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa2c4001670 tx=0x7fa2c4009040).stop 2023-11-06T20:59:23.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.585+0000 7fa2da3ff700 1 -- 172.21.15.33:0/1104575322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d413dff0 msgr2=0x7fa2d41679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:23.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.585+0000 7fa2da3ff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d413dff0 0x7fa2d41679d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fa2cc00ba50 tx=0x7fa2cc009840).stop 2023-11-06T20:59:23.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.586+0000 7fa2da3ff700 1 -- 172.21.15.33:0/1104575322 shutdown_connections 2023-11-06T20:59:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.586+0000 7fa2da3ff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2c005f270 0x7fa2c0061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.586+0000 7fa2da3ff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d413dff0 0x7fa2d41679d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.586+0000 7fa2da3ff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40c4720 0x7fa2d4156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.586+0000 7fa2da3ff700 1 --2- 172.21.15.33:0/1104575322 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c4b00 0x7fa2d4152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.586+0000 7fa2da3ff700 1 -- 172.21.15.33:0/1104575322 >> 172.21.15.33:0/1104575322 conn(0x7fa2d40c7dc0 msgr2=0x7fa2d4068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.586+0000 7fa2da3ff700 1 -- 172.21.15.33:0/1104575322 shutdown_connections 2023-11-06T20:59:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.586+0000 7fa2da3ff700 1 -- 172.21.15.33:0/1104575322 wait complete. 2023-11-06T20:59:23.605 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T20:59:23.606 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T20:59:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.806+0000 7fd70efff700 1 Processor -- start 2023-11-06T20:59:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.806+0000 7fd70efff700 1 -- start start 2023-11-06T20:59:23.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.807+0000 7fd70efff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd708154330 0x7fd7080c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:23.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.807+0000 7fd70efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 0x7fd7080c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:23.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.807+0000 7fd70efff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7080c6b60 con 0x7fd7080c7ff0 2023-11-06T20:59:23.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.807+0000 7fd70efff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7080c5cd0 con 0x7fd708154330 2023-11-06T20:59:23.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.807+0000 7fd70efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080c5f50 con 0x7fd708154710 2023-11-06T20:59:23.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.808+0000 7fd707fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 0x7fd7080c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:23.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.808+0000 7fd70cd9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd708154330 0x7fd7080c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:23.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.808+0000 7fd70d59c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fd7080c7ff0 0x7fd7080646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:55844/0 (socket says 172.21.15.33:55844) 2023-11-06T20:59:23.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.808+0000 7fd70d59c700 1 -- 172.21.15.33:0/738294744 learned_addr learned my addr 172.21.15.33:0/738294744 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:23.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.808+0000 7fd7077fe700 1 -- 172.21.15.33:0/738294744 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364780310 0 0) 0x7fd7080c6b60 con 0x7fd7080c7ff0 2023-11-06T20:59:23.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.808+0000 7fd7077fe700 1 -- 172.21.15.33:0/738294744 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6f80033f0 con 0x7fd7080c7ff0 2023-11-06T20:59:23.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.808+0000 7fd707fff700 1 -- 172.21.15.33:0/738294744 >> v2:172.21.15.69:3300/0 conn(0x7fd708154330 msgr2=0x7fd7080c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:23.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.808+0000 7fd707fff700 1 --2- 172.21.15.33:0/738294744 >> v2:172.21.15.69:3300/0 conn(0x7fd708154330 0x7fd7080c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:23.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.808+0000 7fd707fff700 1 -- 172.21.15.33:0/738294744 >> v1:172.21.15.69:6789/0 conn(0x7fd7080c7ff0 legacy=0x7fd7080646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:23.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.808+0000 7fd707fff700 1 -- 172.21.15.33:0/738294744 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7080c6230 con 0x7fd708154710 2023-11-06T20:59:23.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.809+0000 7fd707fff700 1 --2- 172.21.15.33:0/738294744 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 0x7fd7080c6ff0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd6f4009760 tx=0x7fd6f4009060).ready entity=mon.2 client_cookie=58355deb4e7e09a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:23.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.809+0000 7fd7077fe700 1 -- 172.21.15.33:0/738294744 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400d4a0 con 0x7fd708154710 2023-11-06T20:59:23.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.809+0000 7fd7077fe700 1 -- 172.21.15.33:0/738294744 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400d600 con 0x7fd708154710 2023-11-06T20:59:23.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.809+0000 7fd7077fe700 1 -- 172.21.15.33:0/738294744 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400d980 con 0x7fd708154710 2023-11-06T20:59:23.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.809+0000 7fd70efff700 1 -- 172.21.15.33:0/738294744 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 msgr2=0x7fd7080c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:23.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.809+0000 7fd70efff700 1 --2- 172.21.15.33:0/738294744 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 0x7fd7080c6ff0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd6f4009760 tx=0x7fd6f4009060).stop 2023-11-06T20:59:23.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.810+0000 7fd70efff700 1 -- 172.21.15.33:0/738294744 shutdown_connections 2023-11-06T20:59:23.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.810+0000 7fd70efff700 1 --2- 172.21.15.33:0/738294744 >> v2:172.21.15.69:3300/0 conn(0x7fd708154330 0x7fd7080c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:23.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.810+0000 7fd70efff700 1 --2- 172.21.15.33:0/738294744 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 0x7fd7080c6ff0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:23.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.810+0000 7fd70efff700 1 -- 172.21.15.33:0/738294744 >> 172.21.15.33:0/738294744 conn(0x7fd7080cb910 msgr2=0x7fd7080c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:23.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.810+0000 7fd70efff700 1 -- 172.21.15.33:0/738294744 shutdown_connections 2023-11-06T20:59:23.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.810+0000 7fd70efff700 1 -- 172.21.15.33:0/738294744 wait complete. 2023-11-06T20:59:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.811+0000 7fd70efff700 1 Processor -- start 2023-11-06T20:59:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.811+0000 7fd70efff700 1 -- start start 2023-11-06T20:59:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.812+0000 7fd70efff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7080c7ff0 0x7fd708139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.812+0000 7fd70efff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd708154330 0x7fd7080addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.812+0000 7fd70efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 0x7fd7080ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.812+0000 7fd70efff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080ae7b0 con 0x7fd7080c7ff0 2023-11-06T20:59:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.812+0000 7fd70efff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708157a20 con 0x7fd708154330 2023-11-06T20:59:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.812+0000 7fd70efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7081582c0 con 0x7fd708154710 2023-11-06T20:59:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.812+0000 7fd70cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7080c7ff0 0x7fd708139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd70cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7080c7ff0 0x7fd708139180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57324/0 (socket says 172.21.15.33:57324) 2023-11-06T20:59:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd70cd9b700 1 -- 172.21.15.33:0/1343857310 learned_addr learned my addr 172.21.15.33:0/1343857310 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd70d59c700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 0x7fd7080ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd707fff700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd708154330 0x7fd7080addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd70cd9b700 1 -- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 msgr2=0x7fd7080ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd70cd9b700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 0x7fd7080ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd70cd9b700 1 -- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd708154330 msgr2=0x7fd7080addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd70cd9b700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd708154330 0x7fd7080addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd70cd9b700 1 -- 172.21.15.33:0/1343857310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f40092a0 con 0x7fd7080c7ff0 2023-11-06T20:59:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd70cd9b700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7080c7ff0 0x7fd708139180 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fd6fc00e960 tx=0x7fd6fc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd704ff9700 1 -- 172.21.15.33:0/1343857310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc012600 con 0x7fd7080c7ff0 2023-11-06T20:59:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd704ff9700 1 -- 172.21.15.33:0/1343857310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc012760 con 0x7fd7080c7ff0 2023-11-06T20:59:23.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd70efff700 1 -- 172.21.15.33:0/1343857310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7080c4810 con 0x7fd7080c7ff0 2023-11-06T20:59:23.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.813+0000 7fd704ff9700 1 -- 172.21.15.33:0/1343857310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc012ae0 con 0x7fd7080c7ff0 2023-11-06T20:59:23.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.814+0000 7fd70efff700 1 -- 172.21.15.33:0/1343857310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd708157f50 con 0x7fd7080c7ff0 2023-11-06T20:59:23.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.815+0000 7fd70efff700 1 -- 172.21.15.33:0/1343857310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7081585a0 con 0x7fd7080c7ff0 2023-11-06T20:59:23.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.815+0000 7fd704ff9700 1 -- 172.21.15.33:0/1343857310 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd6fc01b400 con 0x7fd7080c7ff0 2023-11-06T20:59:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.817+0000 7fd704ff9700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd6ec05f270 0x7fd6ec061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.817+0000 7fd704ff9700 1 -- 172.21.15.33:0/1343857310 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd6fc07a7e0 con 0x7fd7080c7ff0 2023-11-06T20:59:23.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.818+0000 7fd707fff700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd6ec05f270 0x7fd6ec061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.818+0000 7fd707fff700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd6ec05f270 0x7fd6ec061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd708154b70 tx=0x7fd6f4016000).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:23.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:23.821+0000 7fd704ff9700 1 -- 172.21.15.33:0/1343857310 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd6fc04b8d0 con 0x7fd7080c7ff0 2023-11-06T20:59:24.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.048+0000 7fd70efff700 1 -- 172.21.15.33:0/1343857310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd7080c07a0 con 0x7fd7080c7ff0 2023-11-06T20:59:24.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.585+0000 7fd704ff9700 1 -- 172.21.15.33:0/1343857310 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd6fc04eef0 con 0x7fd7080c7ff0 2023-11-06T20:59:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 -- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd6ec05f270 msgr2=0x7fd6ec061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd6ec05f270 0x7fd6ec061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd708154b70 tx=0x7fd6f4016000).stop 2023-11-06T20:59:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 -- 172.21.15.33:0/1343857310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7080c7ff0 msgr2=0x7fd708139180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7080c7ff0 0x7fd708139180 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fd6fc00e960 tx=0x7fd6fc009060).stop 2023-11-06T20:59:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 -- 172.21.15.33:0/1343857310 shutdown_connections 2023-11-06T20:59:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd6ec05f270 0x7fd6ec061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd708154710 0x7fd7080ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7080c7ff0 0x7fd708139180 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 --2- 172.21.15.33:0/1343857310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd708154330 0x7fd7080addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:24.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 -- 172.21.15.33:0/1343857310 >> 172.21.15.33:0/1343857310 conn(0x7fd7080cb910 msgr2=0x7fd7080ca260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:24.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 -- 172.21.15.33:0/1343857310 shutdown_connections 2023-11-06T20:59:24.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.587+0000 7fd70efff700 1 -- 172.21.15.33:0/1343857310 wait complete. 2023-11-06T20:59:24.607 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T20:59:24.607 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:24.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.806+0000 7f472abff700 1 Processor -- start 2023-11-06T20:59:24.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.806+0000 7f472abff700 1 -- start start 2023-11-06T20:59:24.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.807+0000 7f472abff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47240c5d50 0x7f47240c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:24.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.807+0000 7f472abff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47240bfd70 0x7f47240c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:24.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.807+0000 7f472abff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47241554a0 con 0x7f47240c1840 2023-11-06T20:59:24.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.807+0000 7f472abff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4724154960 con 0x7f47240bfd70 2023-11-06T20:59:24.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.807+0000 7f472abff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47240c66c0 con 0x7f47240c5d50 2023-11-06T20:59:24.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.808+0000 7f472919c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47240bfd70 0x7f47240c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:24.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.808+0000 7f472919c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47240bfd70 0x7f47240c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57330/0 (socket says 172.21.15.33:57330) 2023-11-06T20:59:24.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.808+0000 7f472919c700 1 -- 172.21.15.33:0/1199763831 learned_addr learned my addr 172.21.15.33:0/1199763831 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:24.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.808+0000 7f4723fff700 1 --2- 172.21.15.33:0/1199763831 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47240c5d50 0x7f47240c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:24.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.808+0000 7f472919c700 1 -- 172.21.15.33:0/1199763831 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47240c5d50 msgr2=0x7f47240c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:24.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.808+0000 7f472919c700 1 --2- 172.21.15.33:0/1199763831 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47240c5d50 0x7f47240c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:24.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.808+0000 7f472919c700 1 -- 172.21.15.33:0/1199763831 >> v1:172.21.15.69:6789/0 conn(0x7f47240c1840 legacy=0x7f47240a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:24.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.808+0000 7f472919c700 1 -- 172.21.15.33:0/1199763831 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724155930 con 0x7f47240bfd70 2023-11-06T20:59:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.808+0000 7f472919c700 1 --2- 172.21.15.33:0/1199763831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 0x7f47240c01e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f471c009f20 tx=0x7f471c009840).ready entity=mon.0 client_cookie=11332c0d2855b60d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f47237fe700 1 -- 172.21.15.33:0/1199763831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c00e070 con 0x7f47240bfd70 2023-11-06T20:59:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f47237fe700 1 -- 172.21.15.33:0/1199763831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c00b9d0 con 0x7f47240bfd70 2023-11-06T20:59:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f47237fe700 1 -- 172.21.15.33:0/1199763831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c00bd50 con 0x7f47240bfd70 2023-11-06T20:59:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f472abff700 1 -- 172.21.15.33:0/1199763831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 msgr2=0x7f47240c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f472abff700 1 --2- 172.21.15.33:0/1199763831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 0x7f47240c01e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f471c009f20 tx=0x7f471c009840).stop 2023-11-06T20:59:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f472abff700 1 -- 172.21.15.33:0/1199763831 shutdown_connections 2023-11-06T20:59:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f472abff700 1 --2- 172.21.15.33:0/1199763831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 0x7f47240c01e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f472abff700 1 --2- 172.21.15.33:0/1199763831 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47240c5d50 0x7f47240c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f472abff700 1 -- 172.21.15.33:0/1199763831 >> 172.21.15.33:0/1199763831 conn(0x7f472405ec80 msgr2=0x7f47240cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f472abff700 1 -- 172.21.15.33:0/1199763831 shutdown_connections 2023-11-06T20:59:24.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.809+0000 7f472abff700 1 -- 172.21.15.33:0/1199763831 wait complete. 2023-11-06T20:59:24.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.811+0000 7f472abff700 1 Processor -- start 2023-11-06T20:59:24.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.812+0000 7f472abff700 1 -- start start 2023-11-06T20:59:24.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.813+0000 7f472abff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 0x7f4724149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:24.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.813+0000 7f472abff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47240c1840 0x7f472414f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:24.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.813+0000 7f472abff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47240c5d50 0x7f4724151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:24.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.813+0000 7f472abff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240c3710 con 0x7f47240bfd70 2023-11-06T20:59:24.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.813+0000 7f472abff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240c2c40 con 0x7f47240c1840 2023-11-06T20:59:24.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.813+0000 7f472abff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47240c2030 con 0x7f47240c5d50 2023-11-06T20:59:24.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.813+0000 7f472899b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 0x7f4724149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:24.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.813+0000 7f472899b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 0x7f4724149c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57340/0 (socket says 172.21.15.33:57340) 2023-11-06T20:59:24.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.813+0000 7f472899b700 1 -- 172.21.15.33:0/2749325606 learned_addr learned my addr 172.21.15.33:0/2749325606 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:24.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f4723fff700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47240c1840 0x7f472414f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:24.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f472919c700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47240c5d50 0x7f4724151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f472899b700 1 -- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47240c5d50 msgr2=0x7f4724151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f472899b700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47240c5d50 0x7f4724151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f472899b700 1 -- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47240c1840 msgr2=0x7f472414f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f472899b700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47240c1840 0x7f472414f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f472899b700 1 -- 172.21.15.33:0/2749325606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c009ae0 con 0x7f47240bfd70 2023-11-06T20:59:24.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f472899b700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 0x7f4724149c10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f471800c4c0 tx=0x7f4718007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:24.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f4720ff9700 1 -- 172.21.15.33:0/2749325606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718007020 con 0x7f47240bfd70 2023-11-06T20:59:24.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f4720ff9700 1 -- 172.21.15.33:0/2749325606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718001e90 con 0x7f47240bfd70 2023-11-06T20:59:24.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f4720ff9700 1 -- 172.21.15.33:0/2749325606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180073b0 con 0x7f47240bfd70 2023-11-06T20:59:24.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f472abff700 1 -- 172.21.15.33:0/2749325606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47240c2310 con 0x7f47240bfd70 2023-11-06T20:59:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.814+0000 7f472abff700 1 -- 172.21.15.33:0/2749325606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47240c28d0 con 0x7f47240bfd70 2023-11-06T20:59:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.815+0000 7f472abff700 1 -- 172.21.15.33:0/2749325606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472406c6c0 con 0x7f47240bfd70 2023-11-06T20:59:24.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.816+0000 7f4720ff9700 1 -- 172.21.15.33:0/2749325606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f47180068b0 con 0x7f47240bfd70 2023-11-06T20:59:24.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.817+0000 7f4720ff9700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f470805f270 0x7f4708061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:24.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.817+0000 7f4720ff9700 1 -- 172.21.15.33:0/2749325606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4718079e20 con 0x7f47240bfd70 2023-11-06T20:59:24.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.818+0000 7f4723fff700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f470805f270 0x7f4708061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:24.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.818+0000 7f4723fff700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f470805f270 0x7f4708061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f47240b0fe0 tx=0x7f4710008040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:24.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:24.821+0000 7f4720ff9700 1 -- 172.21.15.33:0/2749325606 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f471804aeb0 con 0x7f47240bfd70 2023-11-06T20:59:25.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.047+0000 7f472abff700 1 -- 172.21.15.33:0/2749325606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47240c2a60 con 0x7f47240bfd70 2023-11-06T20:59:25.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.047+0000 7f4720ff9700 1 -- 172.21.15.33:0/2749325606 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4490 (secure 0 0 0) 0x7f471804e4d0 con 0x7f47240bfd70 2023-11-06T20:59:25.049 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:25.049 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6055,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1867036074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1867036074},{"type":"v1","addr":"172.21.15.69:6836","nonce":1867036074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6058,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1920115024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1920115024},{"type":"v1","addr":"172.21.15.69:6837","nonce":1920115024}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2935600703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2935600703},{"type":"v1","addr":"172.21.15.33:6835","nonce":2935600703}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:21.575862+0000","modified":"2023-11-06T20:59:24.582786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/3385607254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3385607254},{"type":"v1","addr":"172.21.15.33:6833","nonce":3385607254}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-06T20:59:25.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 -- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f470805f270 msgr2=0x7f4708061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f470805f270 0x7f4708061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f47240b0fe0 tx=0x7f4710008040).stop 2023-11-06T20:59:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 -- 172.21.15.33:0/2749325606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 msgr2=0x7f4724149c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 0x7f4724149c10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f471800c4c0 tx=0x7f4718007e00).stop 2023-11-06T20:59:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 -- 172.21.15.33:0/2749325606 shutdown_connections 2023-11-06T20:59:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f470805f270 0x7f4708061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47240c5d50 0x7f4724151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47240bfd70 0x7f4724149c10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 --2- 172.21.15.33:0/2749325606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47240c1840 0x7f472414f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 -- 172.21.15.33:0/2749325606 >> 172.21.15.33:0/2749325606 conn(0x7f472405ec80 msgr2=0x7f47240ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 -- 172.21.15.33:0/2749325606 shutdown_connections 2023-11-06T20:59:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.049+0000 7f472abff700 1 -- 172.21.15.33:0/2749325606 wait complete. 2023-11-06T20:59:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2023-11-06T20:59:25.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.267+0000 7ff8c47ff700 1 Processor -- start 2023-11-06T20:59:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.267+0000 7ff8c47ff700 1 -- start start 2023-11-06T20:59:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.268+0000 7ff8c47ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8bc063280 0x7ff8bc063660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.268+0000 7ff8c47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc14e4a0 0x7ff8bc14e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.268+0000 7ff8c47ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8bc14ee10 con 0x7ff8bc0645d0 2023-11-06T20:59:25.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.268+0000 7ff8c47ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc158e20 con 0x7ff8bc063280 2023-11-06T20:59:25.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.268+0000 7ff8c47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc1582a0 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.268+0000 7ff8c259b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8bc063280 0x7ff8bc063660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.268+0000 7ff8c259b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8bc063280 0x7ff8bc063660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57356/0 (socket says 172.21.15.33:57356) 2023-11-06T20:59:25.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.268+0000 7ff8c259b700 1 -- 172.21.15.33:0/1965491294 learned_addr learned my addr 172.21.15.33:0/1965491294 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:25.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.269+0000 7ff8c1d9a700 1 --2- 172.21.15.33:0/1965491294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc14e4a0 0x7ff8bc14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.269+0000 7ff8c259b700 1 -- 172.21.15.33:0/1965491294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc14e4a0 msgr2=0x7ff8bc14e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:25.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.269+0000 7ff8c259b700 1 --2- 172.21.15.33:0/1965491294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc14e4a0 0x7ff8bc14e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.269+0000 7ff8c259b700 1 -- 172.21.15.33:0/1965491294 >> v1:172.21.15.69:6789/0 conn(0x7ff8bc0645d0 legacy=0x7ff8bc064a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:25.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.269+0000 7ff8c259b700 1 -- 172.21.15.33:0/1965491294 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc158580 con 0x7ff8bc063280 2023-11-06T20:59:25.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.269+0000 7ff8c259b700 1 --2- 172.21.15.33:0/1965491294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8bc063280 0x7ff8bc063660 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7ff8b8006fd0 tx=0x7ff8b80090f0).ready entity=mon.0 client_cookie=6cb35aedb36188e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:25.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.269+0000 7ff8c1599700 1 -- 172.21.15.33:0/1965491294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800b600 con 0x7ff8bc063280 2023-11-06T20:59:25.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.269+0000 7ff8c1599700 1 -- 172.21.15.33:0/1965491294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b800b760 con 0x7ff8bc063280 2023-11-06T20:59:25.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.269+0000 7ff8c1599700 1 -- 172.21.15.33:0/1965491294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800bae0 con 0x7ff8bc063280 2023-11-06T20:59:25.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.270+0000 7ff8c47ff700 1 -- 172.21.15.33:0/1965491294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8bc063280 msgr2=0x7ff8bc063660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.270+0000 7ff8c47ff700 1 --2- 172.21.15.33:0/1965491294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8bc063280 0x7ff8bc063660 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7ff8b8006fd0 tx=0x7ff8b80090f0).stop 2023-11-06T20:59:25.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.270+0000 7ff8c47ff700 1 -- 172.21.15.33:0/1965491294 shutdown_connections 2023-11-06T20:59:25.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.270+0000 7ff8c47ff700 1 --2- 172.21.15.33:0/1965491294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8bc063280 0x7ff8bc063660 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.270+0000 7ff8c47ff700 1 --2- 172.21.15.33:0/1965491294 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc14e4a0 0x7ff8bc14e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.270+0000 7ff8c47ff700 1 -- 172.21.15.33:0/1965491294 >> 172.21.15.33:0/1965491294 conn(0x7ff8bc0cade0 msgr2=0x7ff8bc0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:25.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.270+0000 7ff8c47ff700 1 -- 172.21.15.33:0/1965491294 shutdown_connections 2023-11-06T20:59:25.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.270+0000 7ff8c47ff700 1 -- 172.21.15.33:0/1965491294 wait complete. 2023-11-06T20:59:25.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.271+0000 7ff8c47ff700 1 Processor -- start 2023-11-06T20:59:25.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.271+0000 7ff8c47ff700 1 -- start start 2023-11-06T20:59:25.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.272+0000 7ff8c47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc063280 0x7ff8bc14d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.272+0000 7ff8c47ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc0645d0 0x7ff8bc0b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.272+0000 7ff8c47ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8bc14e4a0 0x7ff8bc0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.272+0000 7ff8c47ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0b0d60 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.272+0000 7ff8c47ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0adf30 con 0x7ff8bc0645d0 2023-11-06T20:59:25.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.272+0000 7ff8c47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0ae210 con 0x7ff8bc063280 2023-11-06T20:59:25.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c259b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc063280 0x7ff8bc14d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c2d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8bc14e4a0 0x7ff8bc0b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c259b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc063280 0x7ff8bc14d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:51256/0 (socket says 172.21.15.33:51256) 2023-11-06T20:59:25.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c259b700 1 -- 172.21.15.33:0/2496974239 learned_addr learned my addr 172.21.15.33:0/2496974239 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c1d9a700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc0645d0 0x7ff8bc0b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c2d9c700 1 -- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc063280 msgr2=0x7ff8bc14d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c2d9c700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc063280 0x7ff8bc14d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c2d9c700 1 -- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc0645d0 msgr2=0x7ff8bc0b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c2d9c700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc0645d0 0x7ff8bc0b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c2d9c700 1 -- 172.21.15.33:0/2496974239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8009370 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.273+0000 7ff8c2d9c700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8bc14e4a0 0x7ff8bc0b0870 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7ff8b400c810 tx=0x7ff8b40024b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.274+0000 7ff8b27fc700 1 -- 172.21.15.33:0/2496974239 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b400e4a0 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.274+0000 7ff8b27fc700 1 -- 172.21.15.33:0/2496974239 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b400e600 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.274+0000 7ff8c47ff700 1 -- 172.21.15.33:0/2496974239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc0ae4f0 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.274+0000 7ff8b27fc700 1 -- 172.21.15.33:0/2496974239 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b400e9c0 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.274+0000 7ff8c47ff700 1 -- 172.21.15.33:0/2496974239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc0aeab0 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.275+0000 7ff8c47ff700 1 -- 172.21.15.33:0/2496974239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc0aeca0 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.276+0000 7ff8b27fc700 1 -- 172.21.15.33:0/2496974239 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff8b40123f0 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.277+0000 7ff8b27fc700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff8a005f270 0x7ff8a0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.277+0000 7ff8b27fc700 1 -- 172.21.15.33:0/2496974239 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff8b4079d00 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.277+0000 7ff8c259b700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff8a005f270 0x7ff8a0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.278+0000 7ff8c259b700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff8a005f270 0x7ff8a0061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7ff8b800b3c0 tx=0x7ff8b8017000).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:25.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.280+0000 7ff8b27fc700 1 -- 172.21.15.33:0/2496974239 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff8b4047af0 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.461+0000 7ff8c47ff700 1 -- 172.21.15.33:0/2496974239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8bc06c6c0 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.462+0000 7ff8b27fc700 1 -- 172.21.15.33:0/2496974239 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16978 (secure 0 0 0) 0x7ff8b404ad30 con 0x7ff8bc14e4a0 2023-11-06T20:59:25.463 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:25.463 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:22.577744+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:20.035584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:20.561335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:25.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 -- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff8a005f270 msgr2=0x7ff8a0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff8a005f270 0x7ff8a0061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7ff8b800b3c0 tx=0x7ff8b8017000).stop 2023-11-06T20:59:25.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 -- 172.21.15.33:0/2496974239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8bc14e4a0 msgr2=0x7ff8bc0b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8bc14e4a0 0x7ff8bc0b0870 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7ff8b400c810 tx=0x7ff8b40024b0).stop 2023-11-06T20:59:25.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 -- 172.21.15.33:0/2496974239 shutdown_connections 2023-11-06T20:59:25.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff8a005f270 0x7ff8a0061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8bc063280 0x7ff8bc14d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8bc14e4a0 0x7ff8bc0b0870 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 --2- 172.21.15.33:0/2496974239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8bc0645d0 0x7ff8bc0b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 -- 172.21.15.33:0/2496974239 >> 172.21.15.33:0/2496974239 conn(0x7ff8bc0cade0 msgr2=0x7ff8bc0c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:25.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 -- 172.21.15.33:0/2496974239 shutdown_connections 2023-11-06T20:59:25.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.464+0000 7ff8c47ff700 1 -- 172.21.15.33:0/2496974239 wait complete. 2023-11-06T20:59:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T20:59:25.484 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:59:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.681+0000 7fa6775ff700 1 Processor -- start 2023-11-06T20:59:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.681+0000 7fa6775ff700 1 -- start start 2023-11-06T20:59:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.682+0000 7fa6775ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa6700c3600 0x7fa6700c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.682+0000 7fa6775ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670136f50 0x7fa670137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.683+0000 7fa6775ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700c17e0 con 0x7fa670069410 2023-11-06T20:59:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.683+0000 7fa6775ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa670069f60 con 0x7fa6700c3600 2023-11-06T20:59:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.683+0000 7fa6775ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67006a740 con 0x7fa670136f50 2023-11-06T20:59:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.683+0000 7fa675b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa670069410 0x7fa670069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51778/0 (socket says 172.21.15.33:51778) 2023-11-06T20:59:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.683+0000 7fa675b9c700 1 -- 172.21.15.33:0/3167809202 learned_addr learned my addr 172.21.15.33:0/3167809202 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.683+0000 7fa674b9a700 1 --2- 172.21.15.33:0/3167809202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670136f50 0x7fa670137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.683+0000 7fa67539b700 1 --2- 172.21.15.33:0/3167809202 >> v2:172.21.15.69:3300/0 conn(0x7fa6700c3600 0x7fa6700c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.683+0000 7fa667fff700 1 -- 172.21.15.33:0/3167809202 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510521968 0 0) 0x7fa6700c17e0 con 0x7fa670069410 2023-11-06T20:59:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.683+0000 7fa667fff700 1 -- 172.21.15.33:0/3167809202 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa65c0033f0 con 0x7fa670069410 2023-11-06T20:59:25.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.684+0000 7fa667fff700 1 -- 172.21.15.33:0/3167809202 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3057408165 0 0) 0x7fa65c0033f0 con 0x7fa670069410 2023-11-06T20:59:25.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.684+0000 7fa674b9a700 1 -- 172.21.15.33:0/3167809202 >> v2:172.21.15.69:3300/0 conn(0x7fa6700c3600 msgr2=0x7fa6700c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:25.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.684+0000 7fa674b9a700 1 --2- 172.21.15.33:0/3167809202 >> v2:172.21.15.69:3300/0 conn(0x7fa6700c3600 0x7fa6700c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.684+0000 7fa674b9a700 1 -- 172.21.15.33:0/3167809202 >> v1:172.21.15.33:6789/0 conn(0x7fa670069410 legacy=0x7fa670069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.684+0000 7fa674b9a700 1 -- 172.21.15.33:0/3167809202 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67006a180 con 0x7fa670136f50 2023-11-06T20:59:25.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.684+0000 7fa674b9a700 1 --2- 172.21.15.33:0/3167809202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670136f50 0x7fa670137400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fa660009760 tx=0x7fa660009060).ready entity=mon.2 client_cookie=1542327b84f821a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:25.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.684+0000 7fa667fff700 1 -- 172.21.15.33:0/3167809202 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680017f0 con 0x7fa670136f50 2023-11-06T20:59:25.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.684+0000 7fa667fff700 1 -- 172.21.15.33:0/3167809202 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa660011400 con 0x7fa670136f50 2023-11-06T20:59:25.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.684+0000 7fa667fff700 1 -- 172.21.15.33:0/3167809202 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa660011780 con 0x7fa670136f50 2023-11-06T20:59:25.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.685+0000 7fa6775ff700 1 -- 172.21.15.33:0/3167809202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670136f50 msgr2=0x7fa670137400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.685+0000 7fa6775ff700 1 --2- 172.21.15.33:0/3167809202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670136f50 0x7fa670137400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fa660009760 tx=0x7fa660009060).stop 2023-11-06T20:59:25.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.685+0000 7fa6775ff700 1 -- 172.21.15.33:0/3167809202 shutdown_connections 2023-11-06T20:59:25.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.685+0000 7fa6775ff700 1 --2- 172.21.15.33:0/3167809202 >> v2:172.21.15.69:3300/0 conn(0x7fa6700c3600 0x7fa6700c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.685+0000 7fa6775ff700 1 --2- 172.21.15.33:0/3167809202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670136f50 0x7fa670137400 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.685+0000 7fa6775ff700 1 -- 172.21.15.33:0/3167809202 >> 172.21.15.33:0/3167809202 conn(0x7fa670062710 msgr2=0x7fa670062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.685+0000 7fa6775ff700 1 -- 172.21.15.33:0/3167809202 shutdown_connections 2023-11-06T20:59:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.685+0000 7fa6775ff700 1 -- 172.21.15.33:0/3167809202 wait complete. 2023-11-06T20:59:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.686+0000 7fa6775ff700 1 Processor -- start 2023-11-06T20:59:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.686+0000 7fa6775ff700 1 -- start start 2023-11-06T20:59:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.687+0000 7fa6775ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670069410 0x7fa6701364b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.687+0000 7fa6775ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6700c3600 0x7fa670167820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.687+0000 7fa6775ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa670136f50 0x7fa670167d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.687+0000 7fa6775ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700ab400 con 0x7fa670136f50 2023-11-06T20:59:25.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.687+0000 7fa6775ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa670168350 con 0x7fa6700c3600 2023-11-06T20:59:25.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.687+0000 7fa6775ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa670168630 con 0x7fa670069410 2023-11-06T20:59:25.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa675b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa670136f50 0x7fa670167d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa675b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa670136f50 0x7fa670167d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57374/0 (socket says 172.21.15.33:57374) 2023-11-06T20:59:25.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa675b9c700 1 -- 172.21.15.33:0/3617991371 learned_addr learned my addr 172.21.15.33:0/3617991371 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa67539b700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670069410 0x7fa6701364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa674b9a700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6700c3600 0x7fa670167820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa675b9c700 1 -- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670069410 msgr2=0x7fa6701364b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa675b9c700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670069410 0x7fa6701364b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa675b9c700 1 -- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6700c3600 msgr2=0x7fa670167820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa675b9c700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6700c3600 0x7fa670167820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa675b9c700 1 -- 172.21.15.33:0/3617991371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6600092a0 con 0x7fa670136f50 2023-11-06T20:59:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa674b9a700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6700c3600 0x7fa670167820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.688+0000 7fa675b9c700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa670136f50 0x7fa670167d10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fa66800c980 tx=0x7fa668007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.689+0000 7fa6657fa700 1 -- 172.21.15.33:0/3617991371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668004c40 con 0x7fa670136f50 2023-11-06T20:59:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.689+0000 7fa6657fa700 1 -- 172.21.15.33:0/3617991371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa668004de0 con 0x7fa670136f50 2023-11-06T20:59:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.689+0000 7fa6775ff700 1 -- 172.21.15.33:0/3617991371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6700aadb0 con 0x7fa670136f50 2023-11-06T20:59:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.689+0000 7fa6657fa700 1 -- 172.21.15.33:0/3617991371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680073c0 con 0x7fa670136f50 2023-11-06T20:59:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.689+0000 7fa6775ff700 1 -- 172.21.15.33:0/3617991371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6700c09f0 con 0x7fa670136f50 2023-11-06T20:59:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.690+0000 7fa6775ff700 1 -- 172.21.15.33:0/3617991371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6700ab570 con 0x7fa670136f50 2023-11-06T20:59:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.691+0000 7fa6657fa700 1 -- 172.21.15.33:0/3617991371 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa668007770 con 0x7fa670136f50 2023-11-06T20:59:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.692+0000 7fa6657fa700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa65405f270 0x7fa654061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.692+0000 7fa6657fa700 1 -- 172.21.15.33:0/3617991371 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa66807a290 con 0x7fa670136f50 2023-11-06T20:59:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.692+0000 7fa67539b700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa65405f270 0x7fa654061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.693+0000 7fa67539b700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa65405f270 0x7fa654061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa66c009bc0 tx=0x7fa66c008040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.695+0000 7fa6657fa700 1 -- 172.21.15.33:0/3617991371 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa66804b320 con 0x7fa670136f50 2023-11-06T20:59:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.871+0000 7fa6775ff700 1 -- 172.21.15.33:0/3617991371 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa670065750 con 0x7fa65405f270 2023-11-06T20:59:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.884+0000 7fa6657fa700 1 -- 172.21.15.33:0/3617991371 <== mgr.5869 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+354282 (secure 0 0 0) 0x7fa670065750 con 0x7fa65405f270 2023-11-06T20:59:25.889 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 -- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa65405f270 msgr2=0x7fa654061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa65405f270 0x7fa654061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa66c009bc0 tx=0x7fa66c008040).stop 2023-11-06T20:59:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 -- 172.21.15.33:0/3617991371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa670136f50 msgr2=0x7fa670167d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa670136f50 0x7fa670167d10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fa66800c980 tx=0x7fa668007af0).stop 2023-11-06T20:59:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 -- 172.21.15.33:0/3617991371 shutdown_connections 2023-11-06T20:59:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa65405f270 0x7fa654061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa670069410 0x7fa6701364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa670136f50 0x7fa670167d10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 --2- 172.21.15.33:0/3617991371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6700c3600 0x7fa670167820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 -- 172.21.15.33:0/3617991371 >> 172.21.15.33:0/3617991371 conn(0x7fa670062710 msgr2=0x7fa670150910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 -- 172.21.15.33:0/3617991371 shutdown_connections 2023-11-06T20:59:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:25.888+0000 7fa6775ff700 1 -- 172.21.15.33:0/3617991371 wait complete. 2023-11-06T20:59:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:59:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T20:59:24.239749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":734,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":120,"acting":120,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2417264,"kb_used_data":7728,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752557456,"statfs":{"total":773094113280,"available":770618834944,"internally_reserved":0,"allocated":7913472,"data_stored":3939207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28930,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-82,"num_write_kb":-67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":-644,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.996445"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.619205+0000","last_change":"2023-11-06T20:59:21.610491+0000","last_active":"2023-11-06T20:59:21.619205+0000","last_peered":"2023-11-06T20:59:21.619205+0000","last_clean":"2023-11-06T20:59:21.619205+0000","last_became_active":"2023-11-06T20:59:21.609771+0000","last_became_peered":"2023-11-06T20:59:21.609771+0000","last_unstale":"2023-11-06T20:59:21.619205+0000","last_undegraded":"2023-11-06T20:59:21.619205+0000","last_fullsized":"2023-11-06T20:59:21.619205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.599917+0000","last_change":"2023-11-06T20:59:21.593772+0000","last_active":"2023-11-06T20:59:21.599917+0000","last_peered":"2023-11-06T20:59:21.599917+0000","last_clean":"2023-11-06T20:59:21.599917+0000","last_became_active":"2023-11-06T20:59:21.593235+0000","last_became_peered":"2023-11-06T20:59:21.593235+0000","last_unstale":"2023-11-06T20:59:21.599917+0000","last_undegraded":"2023-11-06T20:59:21.599917+0000","last_fullsized":"2023-11-06T20:59:21.599917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621411+0000","last_change":"2023-11-06T20:59:21.122869+0000","last_active":"2023-11-06T20:59:21.621411+0000","last_peered":"2023-11-06T20:59:21.621411+0000","last_clean":"2023-11-06T20:59:21.621411+0000","last_became_active":"2023-11-06T20:59:21.122587+0000","last_became_peered":"2023-11-06T20:59:21.122587+0000","last_unstale":"2023-11-06T20:59:21.621411+0000","last_undegraded":"2023-11-06T20:59:21.621411+0000","last_fullsized":"2023-11-06T20:59:21.621411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.618773+0000","last_change":"2023-11-06T20:59:21.608679+0000","last_active":"2023-11-06T20:59:21.618773+0000","last_peered":"2023-11-06T20:59:21.618773+0000","last_clean":"2023-11-06T20:59:21.618773+0000","last_became_active":"2023-11-06T20:59:21.608222+0000","last_became_peered":"2023-11-06T20:59:21.608222+0000","last_unstale":"2023-11-06T20:59:21.618773+0000","last_undegraded":"2023-11-06T20:59:21.618773+0000","last_fullsized":"2023-11-06T20:59:21.618773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.608277+0000","last_change":"2023-11-06T20:59:21.603532+0000","last_active":"2023-11-06T20:59:21.608277+0000","last_peered":"2023-11-06T20:59:21.608277+0000","last_clean":"2023-11-06T20:59:21.608277+0000","last_became_active":"2023-11-06T20:59:21.603207+0000","last_became_peered":"2023-11-06T20:59:21.603207+0000","last_unstale":"2023-11-06T20:59:21.608277+0000","last_undegraded":"2023-11-06T20:59:21.608277+0000","last_fullsized":"2023-11-06T20:59:21.608277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581964+0000","last_change":"2023-11-06T20:59:21.120942+0000","last_active":"2023-11-06T20:59:21.581964+0000","last_peered":"2023-11-06T20:59:21.581964+0000","last_clean":"2023-11-06T20:59:21.581964+0000","last_became_active":"2023-11-06T20:59:21.120654+0000","last_became_peered":"2023-11-06T20:59:21.120654+0000","last_unstale":"2023-11-06T20:59:21.581964+0000","last_undegraded":"2023-11-06T20:59:21.581964+0000","last_fullsized":"2023-11-06T20:59:21.581964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.618831+0000","last_change":"2023-11-06T20:59:21.614348+0000","last_active":"2023-11-06T20:59:21.618831+0000","last_peered":"2023-11-06T20:59:21.618831+0000","last_clean":"2023-11-06T20:59:21.618831+0000","last_became_active":"2023-11-06T20:59:21.614075+0000","last_became_peered":"2023-11-06T20:59:21.614075+0000","last_unstale":"2023-11-06T20:59:21.618831+0000","last_undegraded":"2023-11-06T20:59:21.618831+0000","last_fullsized":"2023-11-06T20:59:21.618831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.583446+0000","last_change":"2023-11-06T20:59:21.120782+0000","last_active":"2023-11-06T20:59:21.583446+0000","last_peered":"2023-11-06T20:59:21.583446+0000","last_clean":"2023-11-06T20:59:21.583446+0000","last_became_active":"2023-11-06T20:59:21.120434+0000","last_became_peered":"2023-11-06T20:59:21.120434+0000","last_unstale":"2023-11-06T20:59:21.583446+0000","last_undegraded":"2023-11-06T20:59:21.583446+0000","last_fullsized":"2023-11-06T20:59:21.583446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.602367+0000","last_change":"2023-11-06T20:59:21.116919+0000","last_active":"2023-11-06T20:59:21.602367+0000","last_peered":"2023-11-06T20:59:21.602367+0000","last_clean":"2023-11-06T20:59:21.602367+0000","last_became_active":"2023-11-06T20:59:21.116664+0000","last_became_peered":"2023-11-06T20:59:21.116664+0000","last_unstale":"2023-11-06T20:59:21.602367+0000","last_undegraded":"2023-11-06T20:59:21.602367+0000","last_fullsized":"2023-11-06T20:59:21.602367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604951+0000","last_change":"2023-11-06T20:59:21.600283+0000","last_active":"2023-11-06T20:59:21.604951+0000","last_peered":"2023-11-06T20:59:21.604951+0000","last_clean":"2023-11-06T20:59:21.604951+0000","last_became_active":"2023-11-06T20:59:21.599971+0000","last_became_peered":"2023-11-06T20:59:21.599971+0000","last_unstale":"2023-11-06T20:59:21.604951+0000","last_undegraded":"2023-11-06T20:59:21.604951+0000","last_fullsized":"2023-11-06T20:59:21.604951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.592674+0000","last_change":"2023-11-06T20:59:21.120032+0000","last_active":"2023-11-06T20:59:21.592674+0000","last_peered":"2023-11-06T20:59:21.592674+0000","last_clean":"2023-11-06T20:59:21.592674+0000","last_became_active":"2023-11-06T20:59:21.119692+0000","last_became_peered":"2023-11-06T20:59:21.119692+0000","last_unstale":"2023-11-06T20:59:21.592674+0000","last_undegraded":"2023-11-06T20:59:21.592674+0000","last_fullsized":"2023-11-06T20:59:21.592674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621106+0000","last_change":"2023-11-06T20:59:21.617605+0000","last_active":"2023-11-06T20:59:21.621106+0000","last_peered":"2023-11-06T20:59:21.621106+0000","last_clean":"2023-11-06T20:59:21.621106+0000","last_became_active":"2023-11-06T20:59:21.616791+0000","last_became_peered":"2023-11-06T20:59:21.616791+0000","last_unstale":"2023-11-06T20:59:21.621106+0000","last_undegraded":"2023-11-06T20:59:21.621106+0000","last_fullsized":"2023-11-06T20:59:21.621106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.600114+0000","last_change":"2023-11-06T20:59:21.593994+0000","last_active":"2023-11-06T20:59:21.600114+0000","last_peered":"2023-11-06T20:59:21.600114+0000","last_clean":"2023-11-06T20:59:21.600114+0000","last_became_active":"2023-11-06T20:59:21.593731+0000","last_became_peered":"2023-11-06T20:59:21.593731+0000","last_unstale":"2023-11-06T20:59:21.600114+0000","last_undegraded":"2023-11-06T20:59:21.600114+0000","last_fullsized":"2023-11-06T20:59:21.600114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.616567+0000","last_change":"2023-11-06T20:59:21.123486+0000","last_active":"2023-11-06T20:59:21.616567+0000","last_peered":"2023-11-06T20:59:21.616567+0000","last_clean":"2023-11-06T20:59:21.616567+0000","last_became_active":"2023-11-06T20:59:21.123191+0000","last_became_peered":"2023-11-06T20:59:21.123191+0000","last_unstale":"2023-11-06T20:59:21.616567+0000","last_undegraded":"2023-11-06T20:59:21.616567+0000","last_fullsized":"2023-11-06T20:59:21.616567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.605583+0000","last_change":"2023-11-06T20:59:21.601132+0000","last_active":"2023-11-06T20:59:21.605583+0000","last_peered":"2023-11-06T20:59:21.605583+0000","last_clean":"2023-11-06T20:59:21.605583+0000","last_became_active":"2023-11-06T20:59:21.600314+0000","last_became_peered":"2023-11-06T20:59:21.600314+0000","last_unstale":"2023-11-06T20:59:21.605583+0000","last_undegraded":"2023-11-06T20:59:21.605583+0000","last_fullsized":"2023-11-06T20:59:21.605583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.582068+0000","last_change":"2023-11-06T20:59:21.112822+0000","last_active":"2023-11-06T20:59:21.582068+0000","last_peered":"2023-11-06T20:59:21.582068+0000","last_clean":"2023-11-06T20:59:21.582068+0000","last_became_active":"2023-11-06T20:59:21.112477+0000","last_became_peered":"2023-11-06T20:59:21.112477+0000","last_unstale":"2023-11-06T20:59:21.582068+0000","last_undegraded":"2023-11-06T20:59:21.582068+0000","last_fullsized":"2023-11-06T20:59:21.582068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.610703+0000","last_change":"2023-11-06T20:59:21.605273+0000","last_active":"2023-11-06T20:59:21.610703+0000","last_peered":"2023-11-06T20:59:21.610703+0000","last_clean":"2023-11-06T20:59:21.610703+0000","last_became_active":"2023-11-06T20:59:21.604839+0000","last_became_peered":"2023-11-06T20:59:21.604839+0000","last_unstale":"2023-11-06T20:59:21.610703+0000","last_undegraded":"2023-11-06T20:59:21.610703+0000","last_fullsized":"2023-11-06T20:59:21.610703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621078+0000","last_change":"2023-11-06T20:59:21.616580+0000","last_active":"2023-11-06T20:59:21.621078+0000","last_peered":"2023-11-06T20:59:21.621078+0000","last_clean":"2023-11-06T20:59:21.621078+0000","last_became_active":"2023-11-06T20:59:21.616367+0000","last_became_peered":"2023-11-06T20:59:21.616367+0000","last_unstale":"2023-11-06T20:59:21.621078+0000","last_undegraded":"2023-11-06T20:59:21.621078+0000","last_fullsized":"2023-11-06T20:59:21.621078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.592513+0000","last_change":"2023-11-06T20:59:21.125545+0000","last_active":"2023-11-06T20:59:21.592513+0000","last_peered":"2023-11-06T20:59:21.592513+0000","last_clean":"2023-11-06T20:59:21.592513+0000","last_became_active":"2023-11-06T20:59:21.125208+0000","last_became_peered":"2023-11-06T20:59:21.125208+0000","last_unstale":"2023-11-06T20:59:21.592513+0000","last_undegraded":"2023-11-06T20:59:21.592513+0000","last_fullsized":"2023-11-06T20:59:21.592513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.608049+0000","last_change":"2023-11-06T20:59:21.600099+0000","last_active":"2023-11-06T20:59:21.608049+0000","last_peered":"2023-11-06T20:59:21.608049+0000","last_clean":"2023-11-06T20:59:21.608049+0000","last_became_active":"2023-11-06T20:59:21.598760+0000","last_became_peered":"2023-11-06T20:59:21.598760+0000","last_unstale":"2023-11-06T20:59:21.608049+0000","last_undegraded":"2023-11-06T20:59:21.608049+0000","last_fullsized":"2023-11-06T20:59:21.608049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621044+0000","last_change":"2023-11-06T20:59:21.613606+0000","last_active":"2023-11-06T20:59:21.621044+0000","last_peered":"2023-11-06T20:59:21.621044+0000","last_clean":"2023-11-06T20:59:21.621044+0000","last_became_active":"2023-11-06T20:59:21.607808+0000","last_became_peered":"2023-11-06T20:59:21.607808+0000","last_unstale":"2023-11-06T20:59:21.621044+0000","last_undegraded":"2023-11-06T20:59:21.621044+0000","last_fullsized":"2023-11-06T20:59:21.621044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584136+0000","last_change":"2023-11-06T20:59:21.116882+0000","last_active":"2023-11-06T20:59:21.584136+0000","last_peered":"2023-11-06T20:59:21.584136+0000","last_clean":"2023-11-06T20:59:21.584136+0000","last_became_active":"2023-11-06T20:59:21.116298+0000","last_became_peered":"2023-11-06T20:59:21.116298+0000","last_unstale":"2023-11-06T20:59:21.584136+0000","last_undegraded":"2023-11-06T20:59:21.584136+0000","last_fullsized":"2023-11-06T20:59:21.584136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.583977+0000","last_change":"2023-11-06T20:59:21.117270+0000","last_active":"2023-11-06T20:59:21.583977+0000","last_peered":"2023-11-06T20:59:21.583977+0000","last_clean":"2023-11-06T20:59:21.583977+0000","last_became_active":"2023-11-06T20:59:21.116899+0000","last_became_peered":"2023-11-06T20:59:21.116899+0000","last_unstale":"2023-11-06T20:59:21.583977+0000","last_undegraded":"2023-11-06T20:59:21.583977+0000","last_fullsized":"2023-11-06T20:59:21.583977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.599797+0000","last_change":"2023-11-06T20:59:21.596043+0000","last_active":"2023-11-06T20:59:21.599797+0000","last_peered":"2023-11-06T20:59:21.599797+0000","last_clean":"2023-11-06T20:59:21.599797+0000","last_became_active":"2023-11-06T20:59:21.595772+0000","last_became_peered":"2023-11-06T20:59:21.595772+0000","last_unstale":"2023-11-06T20:59:21.599797+0000","last_undegraded":"2023-11-06T20:59:21.599797+0000","last_fullsized":"2023-11-06T20:59:21.599797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581258+0000","last_change":"2023-11-06T20:59:21.124894+0000","last_active":"2023-11-06T20:59:21.581258+0000","last_peered":"2023-11-06T20:59:21.581258+0000","last_clean":"2023-11-06T20:59:21.581258+0000","last_became_active":"2023-11-06T20:59:21.124600+0000","last_became_peered":"2023-11-06T20:59:21.124600+0000","last_unstale":"2023-11-06T20:59:21.581258+0000","last_undegraded":"2023-11-06T20:59:21.581258+0000","last_fullsized":"2023-11-06T20:59:21.581258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.601633+0000","last_change":"2023-11-06T20:59:21.597521+0000","last_active":"2023-11-06T20:59:21.601633+0000","last_peered":"2023-11-06T20:59:21.601633+0000","last_clean":"2023-11-06T20:59:21.601633+0000","last_became_active":"2023-11-06T20:59:21.597250+0000","last_became_peered":"2023-11-06T20:59:21.597250+0000","last_unstale":"2023-11-06T20:59:21.601633+0000","last_undegraded":"2023-11-06T20:59:21.601633+0000","last_fullsized":"2023-11-06T20:59:21.601633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.599895+0000","last_change":"2023-11-06T20:59:21.593576+0000","last_active":"2023-11-06T20:59:21.599895+0000","last_peered":"2023-11-06T20:59:21.599895+0000","last_clean":"2023-11-06T20:59:21.599895+0000","last_became_active":"2023-11-06T20:59:21.593202+0000","last_became_peered":"2023-11-06T20:59:21.593202+0000","last_unstale":"2023-11-06T20:59:21.599895+0000","last_undegraded":"2023-11-06T20:59:21.599895+0000","last_fullsized":"2023-11-06T20:59:21.599895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584236+0000","last_change":"2023-11-06T20:59:21.120975+0000","last_active":"2023-11-06T20:59:21.584236+0000","last_peered":"2023-11-06T20:59:21.584236+0000","last_clean":"2023-11-06T20:59:21.584236+0000","last_became_active":"2023-11-06T20:59:21.120509+0000","last_became_peered":"2023-11-06T20:59:21.120509+0000","last_unstale":"2023-11-06T20:59:21.584236+0000","last_undegraded":"2023-11-06T20:59:21.584236+0000","last_fullsized":"2023-11-06T20:59:21.584236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584976+0000","last_change":"2023-11-06T20:59:21.109519+0000","last_active":"2023-11-06T20:59:21.584976+0000","last_peered":"2023-11-06T20:59:21.584976+0000","last_clean":"2023-11-06T20:59:21.584976+0000","last_became_active":"2023-11-06T20:59:21.109206+0000","last_became_peered":"2023-11-06T20:59:21.109206+0000","last_unstale":"2023-11-06T20:59:21.584976+0000","last_undegraded":"2023-11-06T20:59:21.584976+0000","last_fullsized":"2023-11-06T20:59:21.584976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.592632+0000","last_change":"2023-11-06T20:59:21.114744+0000","last_active":"2023-11-06T20:59:21.592632+0000","last_peered":"2023-11-06T20:59:21.592632+0000","last_clean":"2023-11-06T20:59:21.592632+0000","last_became_active":"2023-11-06T20:59:21.114559+0000","last_became_peered":"2023-11-06T20:59:21.114559+0000","last_unstale":"2023-11-06T20:59:21.592632+0000","last_undegraded":"2023-11-06T20:59:21.592632+0000","last_fullsized":"2023-11-06T20:59:21.592632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.614179+0000","last_change":"2023-11-06T20:59:21.608523+0000","last_active":"2023-11-06T20:59:21.614179+0000","last_peered":"2023-11-06T20:59:21.614179+0000","last_clean":"2023-11-06T20:59:21.614179+0000","last_became_active":"2023-11-06T20:59:21.607864+0000","last_became_peered":"2023-11-06T20:59:21.607864+0000","last_unstale":"2023-11-06T20:59:21.614179+0000","last_undegraded":"2023-11-06T20:59:21.614179+0000","last_fullsized":"2023-11-06T20:59:21.614179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"83'3","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621525+0000","last_change":"2023-11-06T20:59:21.108826+0000","last_active":"2023-11-06T20:59:21.621525+0000","last_peered":"2023-11-06T20:59:21.621525+0000","last_clean":"2023-11-06T20:59:21.621525+0000","last_became_active":"2023-11-06T20:59:21.108513+0000","last_became_peered":"2023-11-06T20:59:21.108513+0000","last_unstale":"2023-11-06T20:59:21.621525+0000","last_undegraded":"2023-11-06T20:59:21.621525+0000","last_fullsized":"2023-11-06T20:59:21.621525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604152+0000","last_change":"2023-11-06T20:59:21.594304+0000","last_active":"2023-11-06T20:59:21.604152+0000","last_peered":"2023-11-06T20:59:21.604152+0000","last_clean":"2023-11-06T20:59:21.604152+0000","last_became_active":"2023-11-06T20:59:21.594021+0000","last_became_peered":"2023-11-06T20:59:21.594021+0000","last_unstale":"2023-11-06T20:59:21.604152+0000","last_undegraded":"2023-11-06T20:59:21.604152+0000","last_fullsized":"2023-11-06T20:59:21.604152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611361+0000","last_change":"2023-11-06T20:59:21.120768+0000","last_active":"2023-11-06T20:59:21.611361+0000","last_peered":"2023-11-06T20:59:21.611361+0000","last_clean":"2023-11-06T20:59:21.611361+0000","last_became_active":"2023-11-06T20:59:21.120168+0000","last_became_peered":"2023-11-06T20:59:21.120168+0000","last_unstale":"2023-11-06T20:59:21.611361+0000","last_undegraded":"2023-11-06T20:59:21.611361+0000","last_fullsized":"2023-11-06T20:59:21.611361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.582062+0000","last_change":"2023-11-06T20:59:21.115393+0000","last_active":"2023-11-06T20:59:21.582062+0000","last_peered":"2023-11-06T20:59:21.582062+0000","last_clean":"2023-11-06T20:59:21.582062+0000","last_became_active":"2023-11-06T20:59:21.114673+0000","last_became_peered":"2023-11-06T20:59:21.114673+0000","last_unstale":"2023-11-06T20:59:21.582062+0000","last_undegraded":"2023-11-06T20:59:21.582062+0000","last_fullsized":"2023-11-06T20:59:21.582062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.595159+0000","last_change":"2023-11-06T20:59:21.590211+0000","last_active":"2023-11-06T20:59:21.595159+0000","last_peered":"2023-11-06T20:59:21.595159+0000","last_clean":"2023-11-06T20:59:21.595159+0000","last_became_active":"2023-11-06T20:59:21.589929+0000","last_became_peered":"2023-11-06T20:59:21.589929+0000","last_unstale":"2023-11-06T20:59:21.595159+0000","last_undegraded":"2023-11-06T20:59:21.595159+0000","last_fullsized":"2023-11-06T20:59:21.595159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584751+0000","last_change":"2023-11-06T20:59:21.117733+0000","last_active":"2023-11-06T20:59:21.584751+0000","last_peered":"2023-11-06T20:59:21.584751+0000","last_clean":"2023-11-06T20:59:21.584751+0000","last_became_active":"2023-11-06T20:59:21.117461+0000","last_became_peered":"2023-11-06T20:59:21.117461+0000","last_unstale":"2023-11-06T20:59:21.584751+0000","last_undegraded":"2023-11-06T20:59:21.584751+0000","last_fullsized":"2023-11-06T20:59:21.584751+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604116+0000","last_change":"2023-11-06T20:59:21.591047+0000","last_active":"2023-11-06T20:59:21.604116+0000","last_peered":"2023-11-06T20:59:21.604116+0000","last_clean":"2023-11-06T20:59:21.604116+0000","last_became_active":"2023-11-06T20:59:21.590754+0000","last_became_peered":"2023-11-06T20:59:21.590754+0000","last_unstale":"2023-11-06T20:59:21.604116+0000","last_undegraded":"2023-11-06T20:59:21.604116+0000","last_fullsized":"2023-11-06T20:59:21.604116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.594139+0000","last_change":"2023-11-06T20:59:21.114982+0000","last_active":"2023-11-06T20:59:21.594139+0000","last_peered":"2023-11-06T20:59:21.594139+0000","last_clean":"2023-11-06T20:59:21.594139+0000","last_became_active":"2023-11-06T20:59:21.114638+0000","last_became_peered":"2023-11-06T20:59:21.114638+0000","last_unstale":"2023-11-06T20:59:21.594139+0000","last_undegraded":"2023-11-06T20:59:21.594139+0000","last_fullsized":"2023-11-06T20:59:21.594139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.620982+0000","last_change":"2023-11-06T20:59:21.612049+0000","last_active":"2023-11-06T20:59:21.620982+0000","last_peered":"2023-11-06T20:59:21.620982+0000","last_clean":"2023-11-06T20:59:21.620982+0000","last_became_active":"2023-11-06T20:59:21.611703+0000","last_became_peered":"2023-11-06T20:59:21.611703+0000","last_unstale":"2023-11-06T20:59:21.620982+0000","last_undegraded":"2023-11-06T20:59:21.620982+0000","last_fullsized":"2023-11-06T20:59:21.620982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.615976+0000","last_change":"2023-11-06T20:59:21.612277+0000","last_active":"2023-11-06T20:59:21.615976+0000","last_peered":"2023-11-06T20:59:21.615976+0000","last_clean":"2023-11-06T20:59:21.615976+0000","last_became_active":"2023-11-06T20:59:21.611916+0000","last_became_peered":"2023-11-06T20:59:21.611916+0000","last_unstale":"2023-11-06T20:59:21.615976+0000","last_undegraded":"2023-11-06T20:59:21.615976+0000","last_fullsized":"2023-11-06T20:59:21.615976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.583550+0000","last_change":"2023-11-06T20:59:21.116311+0000","last_active":"2023-11-06T20:59:21.583550+0000","last_peered":"2023-11-06T20:59:21.583550+0000","last_clean":"2023-11-06T20:59:21.583550+0000","last_became_active":"2023-11-06T20:59:21.116024+0000","last_became_peered":"2023-11-06T20:59:21.116024+0000","last_unstale":"2023-11-06T20:59:21.583550+0000","last_undegraded":"2023-11-06T20:59:21.583550+0000","last_fullsized":"2023-11-06T20:59:21.583550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584453+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:59:21.584453+0000","last_peered":"2023-11-06T20:59:21.584453+0000","last_clean":"2023-11-06T20:59:21.584453+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:59:21.584453+0000","last_undegraded":"2023-11-06T20:59:21.584453+0000","last_fullsized":"2023-11-06T20:59:21.584453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584001+0000","last_change":"2023-11-06T20:59:21.116737+0000","last_active":"2023-11-06T20:59:21.584001+0000","last_peered":"2023-11-06T20:59:21.584001+0000","last_clean":"2023-11-06T20:59:21.584001+0000","last_became_active":"2023-11-06T20:59:21.116467+0000","last_became_peered":"2023-11-06T20:59:21.116467+0000","last_unstale":"2023-11-06T20:59:21.584001+0000","last_undegraded":"2023-11-06T20:59:21.584001+0000","last_fullsized":"2023-11-06T20:59:21.584001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581344+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:59:21.581344+0000","last_peered":"2023-11-06T20:59:21.581344+0000","last_clean":"2023-11-06T20:59:21.581344+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:59:21.581344+0000","last_undegraded":"2023-11-06T20:59:21.581344+0000","last_fullsized":"2023-11-06T20:59:21.581344+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.610617+0000","last_change":"2023-11-06T20:59:21.606804+0000","last_active":"2023-11-06T20:59:21.610617+0000","last_peered":"2023-11-06T20:59:21.610617+0000","last_clean":"2023-11-06T20:59:21.610617+0000","last_became_active":"2023-11-06T20:59:21.606553+0000","last_became_peered":"2023-11-06T20:59:21.606553+0000","last_unstale":"2023-11-06T20:59:21.610617+0000","last_undegraded":"2023-11-06T20:59:21.610617+0000","last_fullsized":"2023-11-06T20:59:21.610617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T20:59:16.399886+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:59:16.399886+0000","last_peered":"2023-11-06T20:59:16.399886+0000","last_clean":"2023-11-06T20:59:16.399886+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:59:16.399886+0000","last_undegraded":"2023-11-06T20:59:16.399886+0000","last_fullsized":"2023-11-06T20:59:16.399886+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:21.082925+0000","last_change":"2023-11-06T20:59:21.082925+0000","last_active":"2023-11-06T20:59:21.082925+0000","last_peered":"2023-11-06T20:59:21.082925+0000","last_clean":"2023-11-06T20:59:21.082925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:21.082925+0000","last_undegraded":"2023-11-06T20:59:21.082925+0000","last_fullsized":"2023-11-06T20:59:21.082925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.583654+0000","last_change":"2023-11-06T20:59:21.113701+0000","last_active":"2023-11-06T20:59:21.583654+0000","last_peered":"2023-11-06T20:59:21.583654+0000","last_clean":"2023-11-06T20:59:21.583654+0000","last_became_active":"2023-11-06T20:59:21.112239+0000","last_became_peered":"2023-11-06T20:59:21.112239+0000","last_unstale":"2023-11-06T20:59:21.583654+0000","last_undegraded":"2023-11-06T20:59:21.583654+0000","last_fullsized":"2023-11-06T20:59:21.583654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581400+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:59:21.581400+0000","last_peered":"2023-11-06T20:59:21.581400+0000","last_clean":"2023-11-06T20:59:21.581400+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:59:21.581400+0000","last_undegraded":"2023-11-06T20:59:21.581400+0000","last_fullsized":"2023-11-06T20:59:21.581400+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T20:59:16.406179+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:59:16.406179+0000","last_peered":"2023-11-06T20:59:16.406179+0000","last_clean":"2023-11-06T20:59:16.406179+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:59:16.406179+0000","last_undegraded":"2023-11-06T20:59:16.406179+0000","last_fullsized":"2023-11-06T20:59:16.406179+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.602079+0000","last_change":"2023-11-06T20:59:21.600370+0000","last_active":"2023-11-06T20:59:21.602079+0000","last_peered":"2023-11-06T20:59:21.602079+0000","last_clean":"2023-11-06T20:59:21.602079+0000","last_became_active":"2023-11-06T20:59:21.600069+0000","last_became_peered":"2023-11-06T20:59:21.600069+0000","last_unstale":"2023-11-06T20:59:21.602079+0000","last_undegraded":"2023-11-06T20:59:21.602079+0000","last_fullsized":"2023-11-06T20:59:21.602079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.598397+0000","last_change":"2023-11-06T20:59:21.105339+0000","last_active":"2023-11-06T20:59:21.598397+0000","last_peered":"2023-11-06T20:59:21.598397+0000","last_clean":"2023-11-06T20:59:21.598397+0000","last_became_active":"2023-11-06T20:59:21.104789+0000","last_became_peered":"2023-11-06T20:59:21.104789+0000","last_unstale":"2023-11-06T20:59:21.598397+0000","last_undegraded":"2023-11-06T20:59:21.598397+0000","last_fullsized":"2023-11-06T20:59:21.598397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581743+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:59:21.581743+0000","last_peered":"2023-11-06T20:59:21.581743+0000","last_clean":"2023-11-06T20:59:21.581743+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:59:21.581743+0000","last_undegraded":"2023-11-06T20:59:21.581743+0000","last_fullsized":"2023-11-06T20:59:21.581743+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.616646+0000","last_change":"2023-11-06T20:59:21.608676+0000","last_active":"2023-11-06T20:59:21.616646+0000","last_peered":"2023-11-06T20:59:21.616646+0000","last_clean":"2023-11-06T20:59:21.616646+0000","last_became_active":"2023-11-06T20:59:21.605765+0000","last_became_peered":"2023-11-06T20:59:21.605765+0000","last_unstale":"2023-11-06T20:59:21.616646+0000","last_undegraded":"2023-11-06T20:59:21.616646+0000","last_fullsized":"2023-11-06T20:59:21.616646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.582034+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:59:21.582034+0000","last_peered":"2023-11-06T20:59:21.582034+0000","last_clean":"2023-11-06T20:59:21.582034+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:59:21.582034+0000","last_undegraded":"2023-11-06T20:59:21.582034+0000","last_fullsized":"2023-11-06T20:59:21.582034+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604722+0000","last_change":"2023-11-06T20:59:21.599192+0000","last_active":"2023-11-06T20:59:21.604722+0000","last_peered":"2023-11-06T20:59:21.604722+0000","last_clean":"2023-11-06T20:59:21.604722+0000","last_became_active":"2023-11-06T20:59:21.598712+0000","last_became_peered":"2023-11-06T20:59:21.598712+0000","last_unstale":"2023-11-06T20:59:21.604722+0000","last_undegraded":"2023-11-06T20:59:21.604722+0000","last_fullsized":"2023-11-06T20:59:21.604722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581937+0000","last_change":"2023-11-06T20:59:21.120879+0000","last_active":"2023-11-06T20:59:21.581937+0000","last_peered":"2023-11-06T20:59:21.581937+0000","last_clean":"2023-11-06T20:59:21.581937+0000","last_became_active":"2023-11-06T20:59:21.120566+0000","last_became_peered":"2023-11-06T20:59:21.120566+0000","last_unstale":"2023-11-06T20:59:21.581937+0000","last_undegraded":"2023-11-06T20:59:21.581937+0000","last_fullsized":"2023-11-06T20:59:21.581937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T20:59:16.401781+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:59:16.401781+0000","last_peered":"2023-11-06T20:59:16.401781+0000","last_clean":"2023-11-06T20:59:16.401781+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:59:16.401781+0000","last_undegraded":"2023-11-06T20:59:16.401781+0000","last_fullsized":"2023-11-06T20:59:16.401781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604136+0000","last_change":"2023-11-06T20:59:21.592285+0000","last_active":"2023-11-06T20:59:21.604136+0000","last_peered":"2023-11-06T20:59:21.604136+0000","last_clean":"2023-11-06T20:59:21.604136+0000","last_became_active":"2023-11-06T20:59:21.591287+0000","last_became_peered":"2023-11-06T20:59:21.591287+0000","last_unstale":"2023-11-06T20:59:21.604136+0000","last_undegraded":"2023-11-06T20:59:21.604136+0000","last_fullsized":"2023-11-06T20:59:21.604136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T20:59:20.073854+0000","last_change":"2023-11-06T20:59:20.073854+0000","last_active":"2023-11-06T20:59:20.073854+0000","last_peered":"2023-11-06T20:59:20.073854+0000","last_clean":"2023-11-06T20:59:20.073854+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T20:59:20.073854+0000","last_undegraded":"2023-11-06T20:59:20.073854+0000","last_fullsized":"2023-11-06T20:59:20.073854+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.582175+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:59:21.582175+0000","last_peered":"2023-11-06T20:59:21.582175+0000","last_clean":"2023-11-06T20:59:21.582175+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:59:21.582175+0000","last_undegraded":"2023-11-06T20:59:21.582175+0000","last_fullsized":"2023-11-06T20:59:21.582175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":734,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":48,"acting":48,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575032,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":966656,"data_stored":482062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.61099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.95599999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.878,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57999999999999996}]},{"osd":1,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69099999999999995,"15min":0.71999999999999997},"min":{"1min":0.47899999999999998,"5min":0.439,"15min":0.40500000000000003},"max":{"1min":0.89400000000000002,"5min":0.98599999999999999,"15min":5.1749999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71699999999999997},"min":{"1min":0.57299999999999995,"5min":0.43099999999999999,"15min":0.41399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.133},"last":0.65600000000000003}]},{"osd":2,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72999999999999998,"15min":0.72799999999999998},"min":{"1min":0.59699999999999998,"5min":0.499,"15min":0.47799999999999998},"max":{"1min":0.96199999999999997,"5min":0.996,"15min":1.165},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.55400000000000005,"5min":0.442,"15min":0.442},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":1.024},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69999999999999996,"15min":0.73299999999999998},"min":{"1min":0.55500000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.872,"5min":1.0129999999999999,"15min":5.202},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.49399999999999999},"max":{"1min":0.91000000000000003,"5min":0.95699999999999996,"15min":5.165},"last":0.79800000000000004}]},{"osd":4,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":1.083},"last":0.504},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.48499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79700000000000004,"5min":0.86799999999999999,"15min":0.92800000000000005},"last":0.35899999999999999}]},{"osd":5,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63,"15min":0.64600000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41099999999999998},"max":{"1min":0.80600000000000005,"5min":0.93300000000000005,"15min":1.395},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.42699999999999999},"max":{"1min":0.84099999999999997,"5min":0.86199999999999999,"15min":1.129},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70899999999999996,"15min":0.72599999999999998},"min":{"1min":0.57499999999999996,"5min":0.505,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":0.94499999999999995,"15min":1.24},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.443},"max":{"1min":0.876,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575032,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":966656,"data_stored":482062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68400000000000005,"15min":0.67700000000000005},"min":{"1min":0.48199999999999998,"5min":0.33000000000000002,"15min":0.32800000000000001},"max":{"1min":0.92900000000000005,"5min":0.93999999999999995,"15min":1.0720000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69699999999999995,"15min":0.67900000000000005},"min":{"1min":0.52100000000000002,"5min":0.47599999999999998,"15min":0.38800000000000001},"max":{"1min":0.88500000000000001,"5min":1.216,"15min":1.216},"last":0.83399999999999996}]},{"osd":1,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.60499999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.877,"5min":0.97599999999999998,"15min":1.3799999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.54800000000000004,"5min":0.41899999999999998,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.98599999999999999,"15min":1.1659999999999999},"last":0.78800000000000003}]},{"osd":2,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.76700000000000002,"15min":0.752},"min":{"1min":0.63,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.321},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.75700000000000001,"15min":0.76000000000000001},"min":{"1min":0.65000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95099999999999996,"5min":1.05,"15min":1.405},"last":0.87}]},{"osd":3,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.59499999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.1459999999999999,"15min":1.5109999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.72199999999999998},"min":{"1min":0.57999999999999996,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":0.95999999999999996,"5min":1.002,"15min":1.002},"last":0.85199999999999998}]},{"osd":4,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.49099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.92200000000000004,"15min":1.018},"last":0.498},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63200000000000001,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.40699999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":2.0310000000000001},"last":0.51700000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90700000000000003,"5min":1.028,"15min":1.028},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.441,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85399999999999998,"5min":1.02,"15min":1.133},"last":0.54400000000000004}]},{"osd":7,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53300000000000003,"5min":0.40799999999999997,"15min":0.378},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.0940000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.53900000000000003,"5min":0.47699999999999998,"15min":0.46899999999999997},"max":{"1min":0.97099999999999997,"5min":0.98399999999999999,"15min":1.4319999999999999},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575032,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":966656,"data_stored":482062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.44900000000000001},"min":{"1min":0.34499999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77900000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.47799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.371}]},{"osd":2,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.86599999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.45900000000000002,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.504},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56999999999999995,"15min":0.55300000000000005},"min":{"1min":0.48199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73299999999999998,"5min":0.94099999999999995,"15min":1.29},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.61499999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.171},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82199999999999995,"15min":0.81899999999999995},"min":{"1min":0.76000000000000001,"5min":0.58199999999999996,"15min":0.57899999999999996},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.1519999999999999},"last":0.78300000000000003}]},{"osd":5,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.64600000000000002,"5min":0.57599999999999996,"15min":0.54900000000000004},"max":{"1min":1.042,"5min":1.042,"15min":1.329},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77100000000000002,"15min":0.78100000000000003},"min":{"1min":0.628,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.97599999999999998,"5min":1.028,"15min":1.1160000000000001},"last":0.85599999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.501},"max":{"1min":0.98599999999999999,"5min":1.0880000000000001,"15min":1.135},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.73199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.3540000000000001},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.64300000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.077,"15min":1.405},"last":0.752},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82199999999999995},"min":{"1min":0.66400000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0329999999999999,"5min":1.105,"15min":1.1950000000000001},"last":0.72299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575033,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":1019904,"data_stored":509737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.45700000000000002,"15min":0.46000000000000002},"min":{"1min":0.32100000000000001,"5min":0.308,"15min":0.29299999999999998},"max":{"1min":0.54700000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.503,"15min":0.501},"min":{"1min":0.42999999999999999,"5min":0.33200000000000002,"15min":0.311},"max":{"1min":0.61799999999999999,"5min":0.73299999999999998,"15min":0.97399999999999998},"last":0.41199999999999998}]},{"osd":2,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46200000000000002,"15min":0.46700000000000003},"min":{"1min":0.35499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.61699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63100000000000001,"5min":0.77500000000000002,"15min":0.91000000000000003},"last":0.45000000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.35999999999999999},"max":{"1min":0.71799999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.46800000000000003,"5min":0.39700000000000002,"15min":0.38300000000000001},"max":{"1min":0.628,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.57999999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97699999999999998,"5min":1.1279999999999999,"15min":1.3779999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79400000000000004,"15min":0.80800000000000005},"min":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.53600000000000003},"max":{"1min":0.89600000000000002,"5min":1.298,"15min":1.8380000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78900000000000003,"15min":0.80600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.97199999999999998,"5min":1.1120000000000001,"15min":1.3169999999999999},"last":1.069},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81499999999999995,"15min":0.82699999999999996},"min":{"1min":0.60899999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96299999999999997,"5min":1.2549999999999999,"15min":1.915},"last":1.006}]},{"osd":6,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78100000000000003,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.56499999999999995,"15min":0.54600000000000004},"max":{"1min":0.96399999999999997,"5min":1.048,"15min":1.1120000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.622,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97899999999999998,"5min":1.016,"15min":1.5860000000000001},"last":1.0309999999999999}]},{"osd":7,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.67600000000000005,"5min":0.60499999999999998,"15min":0.57199999999999995},"max":{"1min":1.01,"5min":1.0820000000000001,"15min":1.2490000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81299999999999994,"15min":0.83399999999999996},"min":{"1min":0.62,"5min":0.54600000000000004,"15min":0.53200000000000003},"max":{"1min":1.026,"5min":1.089,"15min":1.6359999999999999},"last":0.98799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575032,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":966656,"data_stored":482062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.44800000000000001,"15min":0.44500000000000001},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.25900000000000001},"max":{"1min":0.70299999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.434},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.496,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.72699999999999998,"5min":0.73099999999999998,"15min":1.0269999999999999},"last":0.5}]},{"osd":1,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.34599999999999997,"15min":0.30099999999999999},"max":{"1min":0.86099999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.60999999999999999,"15min":0.60299999999999998},"min":{"1min":0.499,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.96399999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.40400000000000003}]},{"osd":3,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.505},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.33400000000000002},"max":{"1min":0.65700000000000003,"5min":2.266,"15min":2.266},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.46800000000000003,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":1.732,"15min":2.242},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.371},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.51400000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.357},"last":0.78000000000000003}]},{"osd":5,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.45900000000000002},"max":{"1min":1.0209999999999999,"5min":1.657,"15min":1.657},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80800000000000005,"15min":0.80200000000000005},"min":{"1min":0.67900000000000005,"5min":0.58599999999999997,"15min":0.498},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.4099999999999999},"last":0.90900000000000003}]},{"osd":6,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.80300000000000005,"15min":0.80200000000000005},"min":{"1min":0.56000000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.4239999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.54900000000000004,"5min":0.54100000000000004,"15min":0.52300000000000002},"max":{"1min":1.075,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.85799999999999998}]},{"osd":7,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.65000000000000002,"5min":0.52800000000000002,"15min":0.46600000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.534},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83299999999999996,"15min":0.82399999999999995},"min":{"1min":0.66100000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.129,"5min":2.202,"15min":2.202},"last":0.90400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575034,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":1019904,"data_stored":508935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.45600000000000002},"min":{"1min":0.36199999999999999,"5min":0.28399999999999997,"15min":0.26800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":1.046},"last":0.372},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.376,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.51100000000000001}]},{"osd":1,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48599999999999999,"15min":0.48299999999999998},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.313},"max":{"1min":0.63900000000000001,"5min":0.64800000000000002,"15min":0.753},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.34499999999999997},"max":{"1min":0.69599999999999995,"5min":0.80900000000000005,"15min":1.177},"last":0.41999999999999998}]},{"osd":2,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.48199999999999998,"5min":0.379,"15min":0.35299999999999998},"max":{"1min":0.70899999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.45300000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":0.88100000000000001,"15min":1.3120000000000001},"last":0.501}]},{"osd":4,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.81799999999999995,"15min":0.80000000000000004},"min":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.51200000000000001},"max":{"1min":1.1080000000000001,"5min":1.212,"15min":1.2470000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.81699999999999995,"15min":0.79800000000000004},"min":{"1min":0.68000000000000005,"5min":0.55100000000000005,"15min":0.47999999999999998},"max":{"1min":1.1379999999999999,"5min":1.1819999999999999,"15min":1.286},"last":0.68799999999999994}]},{"osd":5,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77900000000000003,"15min":0.77500000000000002},"min":{"1min":0.59199999999999997,"5min":0.496,"15min":0.496},"max":{"1min":0.94799999999999995,"5min":1.1779999999999999,"15min":1.228},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.59599999999999997,"5min":0.54600000000000004,"15min":0.49199999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.1439999999999999},"last":0.621}]},{"osd":6,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79400000000000004,"15min":0.77400000000000002},"min":{"1min":0.61899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96799999999999997,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.80400000000000005},"min":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.97499999999999998,"5min":1.202,"15min":1.268},"last":0.81799999999999995}]},{"osd":7,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81200000000000006,"15min":0.80300000000000005},"min":{"1min":0.62,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0569999999999999,"5min":1.302,"15min":1.302},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.82199999999999995,"15min":0.81100000000000005},"min":{"1min":0.63800000000000001,"5min":0.56699999999999995,"15min":0.47899999999999998},"max":{"1min":1.0509999999999999,"5min":1.077,"15min":1.1739999999999999},"last":0.79400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575032,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":966656,"data_stored":482062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.69699999999999995}]},{"osd":1,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.495,"5min":0.44400000000000001,"15min":0.41399999999999998},"max":{"1min":1.107,"5min":1.645,"15min":1.645},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.58099999999999996,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":0.92000000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.72299999999999998,"5min":0.52900000000000003,"15min":0.51900000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.64400000000000002}]},{"osd":3,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.69199999999999995},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80000000000000004,"5min":0.95799999999999996,"15min":1.0169999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69499999999999995,"15min":0.69199999999999995},"min":{"1min":0.49099999999999999,"5min":0.46600000000000003,"15min":0.42799999999999999},"max":{"1min":0.93200000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60699999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59499999999999997,"15min":0.61199999999999999},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":1.0389999999999999,"15min":1.0880000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.45200000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.3520000000000001},"last":0.86599999999999999}]},{"osd":6,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.50600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84399999999999997,"5min":1.22,"15min":1.22},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67100000000000004,"15min":0.68400000000000005},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.44},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.3999999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.46300000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.034,"5min":1.034,"15min":1.5209999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.67800000000000005},"min":{"1min":0.495,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93999999999999995,"5min":1.177,"15min":1.177},"last":0.81299999999999994}]}]},{"osd":5,"up_from":13,"seq":55834575034,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302208,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069632,"statfs":{"total":96636764160,"available":96327303168,"internally_reserved":0,"allocated":1040384,"data_stored":510225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.42599999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84999999999999998,"5min":0.98599999999999999,"15min":1.0109999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.52300000000000002,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":0.96199999999999997,"15min":0.96799999999999997},"last":0.624}]},{"osd":1,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90700000000000003,"5min":0.99399999999999999,"15min":1.0109999999999999},"last":0.748},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.753,"15min":0.74199999999999999},"min":{"1min":0.66800000000000004,"5min":0.432,"15min":0.40300000000000002},"max":{"1min":0.93899999999999995,"5min":0.99199999999999999,"15min":1.044},"last":0.81299999999999994}]},{"osd":2,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.59299999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84799999999999998,"5min":0.96299999999999997,"15min":1.024},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.57199999999999995,"5min":0.46000000000000002,"15min":0.45600000000000002},"max":{"1min":0.86899999999999999,"5min":0.96999999999999997,"15min":1.0329999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69999999999999996,"15min":0.69699999999999995},"min":{"1min":0.53100000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.88900000000000001,"5min":0.90900000000000003,"15min":0.93899999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70699999999999996,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.91700000000000004,"5min":0.97599999999999998,"15min":1.02},"last":0.68799999999999994}]},{"osd":4,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71699999999999997,"5min":0.86399999999999999,"15min":0.93700000000000006},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.39400000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89800000000000002,"5min":0.97499999999999998,"15min":0.98599999999999999},"last":0.89800000000000002}]},{"osd":6,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.63200000000000001},"min":{"1min":0.45100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88600000000000001,"5min":0.96399999999999997,"15min":0.97599999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.71899999999999997},"min":{"1min":0.503,"5min":0.49099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":1.105},"last":0.85499999999999998}]},{"osd":7,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90600000000000003,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.434},"max":{"1min":0.89500000000000002,"5min":0.998,"15min":0.998},"last":0.84299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:59:25.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T20:59:26.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.143+0000 7f4132fff700 1 Processor -- start 2023-11-06T20:59:26.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.143+0000 7f4132fff700 1 -- start start 2023-11-06T20:59:26.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.144+0000 7f4132fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f412c0c5d50 0x7f412c0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.144+0000 7f4132fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f412c0bfd70 0x7f412c0c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.144+0000 7f4132fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c1554a0 con 0x7f412c0c1840 2023-11-06T20:59:26.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.144+0000 7f4132fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412c154960 con 0x7f412c0bfd70 2023-11-06T20:59:26.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.144+0000 7f4132fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412c0c66c0 con 0x7f412c0c5d50 2023-11-06T20:59:26.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.144+0000 7f413159c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f412c0bfd70 0x7f412c0c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.144+0000 7f4130d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f412c0c1840 0x7f412c0a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51788/0 (socket says 172.21.15.33:51788) 2023-11-06T20:59:26.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.144+0000 7f4130d9b700 1 -- 172.21.15.33:0/1229200931 learned_addr learned my addr 172.21.15.33:0/1229200931 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.145+0000 7f412bfff700 1 --2- 172.21.15.33:0/1229200931 >> v2:172.21.15.69:3300/0 conn(0x7f412c0c5d50 0x7f412c0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.145+0000 7f413159c700 1 -- 172.21.15.33:0/1229200931 >> v2:172.21.15.69:3300/0 conn(0x7f412c0c5d50 msgr2=0x7f412c0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.145+0000 7f413159c700 1 --2- 172.21.15.33:0/1229200931 >> v2:172.21.15.69:3300/0 conn(0x7f412c0c5d50 0x7f412c0c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.145+0000 7f413159c700 1 -- 172.21.15.33:0/1229200931 >> v1:172.21.15.33:6789/0 conn(0x7f412c0c1840 legacy=0x7f412c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.145+0000 7f413159c700 1 -- 172.21.15.33:0/1229200931 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c155930 con 0x7f412c0bfd70 2023-11-06T20:59:26.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.145+0000 7f412bfff700 1 --2- 172.21.15.33:0/1229200931 >> v2:172.21.15.69:3300/0 conn(0x7f412c0c5d50 0x7f412c0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:26.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.145+0000 7f413159c700 1 --2- 172.21.15.33:0/1229200931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 0x7f412c0c0180 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f412c0adb40 tx=0x7f4124009780).ready entity=mon.0 client_cookie=b0d024a89eef94c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:26.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.145+0000 7f412b7fe700 1 -- 172.21.15.33:0/1229200931 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412400e070 con 0x7f412c0bfd70 2023-11-06T20:59:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.145+0000 7f412b7fe700 1 -- 172.21.15.33:0/1229200931 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412400bc30 con 0x7f412c0bfd70 2023-11-06T20:59:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.145+0000 7f412b7fe700 1 -- 172.21.15.33:0/1229200931 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124012410 con 0x7f412c0bfd70 2023-11-06T20:59:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.146+0000 7f4132fff700 1 -- 172.21.15.33:0/1229200931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 msgr2=0x7f412c0c0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.146+0000 7f4132fff700 1 --2- 172.21.15.33:0/1229200931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 0x7f412c0c0180 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f412c0adb40 tx=0x7f4124009780).stop 2023-11-06T20:59:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.146+0000 7f4132fff700 1 -- 172.21.15.33:0/1229200931 shutdown_connections 2023-11-06T20:59:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.146+0000 7f4132fff700 1 --2- 172.21.15.33:0/1229200931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 0x7f412c0c0180 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.146+0000 7f4132fff700 1 --2- 172.21.15.33:0/1229200931 >> v2:172.21.15.69:3300/0 conn(0x7f412c0c5d50 0x7f412c0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.146+0000 7f4132fff700 1 -- 172.21.15.33:0/1229200931 >> 172.21.15.33:0/1229200931 conn(0x7f412c05ec80 msgr2=0x7f412c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.146+0000 7f4132fff700 1 -- 172.21.15.33:0/1229200931 shutdown_connections 2023-11-06T20:59:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.146+0000 7f4132fff700 1 -- 172.21.15.33:0/1229200931 wait complete. 2023-11-06T20:59:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.147+0000 7f4132fff700 1 Processor -- start 2023-11-06T20:59:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.148+0000 7f4132fff700 1 -- start start 2023-11-06T20:59:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.148+0000 7f4132fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 0x7f412c0b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.148+0000 7f4132fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f412c0c1840 0x7f412c167d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.148+0000 7f4132fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c0c5d50 0x7f412c1496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4132fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c168a60 con 0x7f412c0bfd70 2023-11-06T20:59:26.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4132fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c14b860 con 0x7f412c0c5d50 2023-11-06T20:59:26.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4132fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c14bb40 con 0x7f412c0c1840 2023-11-06T20:59:26.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4130d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 0x7f412c0b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4130d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 0x7f412c0b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57404/0 (socket says 172.21.15.33:57404) 2023-11-06T20:59:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4130d9b700 1 -- 172.21.15.33:0/1621531592 learned_addr learned my addr 172.21.15.33:0/1621531592 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f412bfff700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f412c0c1840 0x7f412c167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f413159c700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c0c5d50 0x7f412c1496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4130d9b700 1 -- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f412c0c1840 msgr2=0x7f412c167d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4130d9b700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f412c0c1840 0x7f412c167d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4130d9b700 1 -- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c0c5d50 msgr2=0x7f412c1496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4130d9b700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c0c5d50 0x7f412c1496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4130d9b700 1 -- 172.21.15.33:0/1621531592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4124009a20 con 0x7f412c0bfd70 2023-11-06T20:59:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4130d9b700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 0x7f412c0b0880 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f4120007fd0 tx=0x7f4120007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.149+0000 7f4128ff9700 1 -- 172.21.15.33:0/1621531592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41200072e0 con 0x7f412c0bfd70 2023-11-06T20:59:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.150+0000 7f4128ff9700 1 -- 172.21.15.33:0/1621531592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120007440 con 0x7f412c0bfd70 2023-11-06T20:59:26.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.150+0000 7f4132fff700 1 -- 172.21.15.33:0/1621531592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412c14be20 con 0x7f412c0bfd70 2023-11-06T20:59:26.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.150+0000 7f4128ff9700 1 -- 172.21.15.33:0/1621531592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41200057a0 con 0x7f412c0bfd70 2023-11-06T20:59:26.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.150+0000 7f4132fff700 1 -- 172.21.15.33:0/1621531592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412c0b0d70 con 0x7f412c0bfd70 2023-11-06T20:59:26.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.151+0000 7f4132fff700 1 -- 172.21.15.33:0/1621531592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412c14c210 con 0x7f412c0bfd70 2023-11-06T20:59:26.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.152+0000 7f4128ff9700 1 -- 172.21.15.33:0/1621531592 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4120005b90 con 0x7f412c0bfd70 2023-11-06T20:59:26.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.153+0000 7f4128ff9700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f411005f270 0x7f4110061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.153+0000 7f4128ff9700 1 -- 172.21.15.33:0/1621531592 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4120079bb0 con 0x7f412c0bfd70 2023-11-06T20:59:26.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.153+0000 7f412bfff700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f411005f270 0x7f4110061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.154+0000 7f412bfff700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f411005f270 0x7f4110061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f4118006d80 tx=0x7f4118008040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:26.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.157+0000 7f4128ff9700 1 -- 172.21.15.33:0/1621531592 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f412004ad00 con 0x7f412c0bfd70 2023-11-06T20:59:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.333+0000 7f4132fff700 1 -- 172.21.15.33:0/1621531592 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f412c14c010 con 0x7f411005f270 2023-11-06T20:59:26.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.344+0000 7f4128ff9700 1 -- 172.21.15.33:0/1621531592 <== mgr.5869 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361122 (secure 0 0 0) 0x7f412c14c010 con 0x7f411005f270 2023-11-06T20:59:26.349 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:26.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.348+0000 7f4132fff700 1 -- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f411005f270 msgr2=0x7f4110061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f411005f270 0x7f4110061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f4118006d80 tx=0x7f4118008040).stop 2023-11-06T20:59:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 -- 172.21.15.33:0/1621531592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 msgr2=0x7f412c0b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 0x7f412c0b0880 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f4120007fd0 tx=0x7f4120007df0).stop 2023-11-06T20:59:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 -- 172.21.15.33:0/1621531592 shutdown_connections 2023-11-06T20:59:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f411005f270 0x7f4110061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c0c5d50 0x7f412c1496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f412c0bfd70 0x7f412c0b0880 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 --2- 172.21.15.33:0/1621531592 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f412c0c1840 0x7f412c167d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 -- 172.21.15.33:0/1621531592 >> 172.21.15.33:0/1621531592 conn(0x7f412c05ec80 msgr2=0x7f412c150f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:26.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 -- 172.21.15.33:0/1621531592 shutdown_connections 2023-11-06T20:59:26.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.349+0000 7f4132fff700 1 -- 172.21.15.33:0/1621531592 wait complete. 2023-11-06T20:59:26.355 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T20:59:26.367 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T20:59:26.240348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":208,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2417512,"kb_used_data":7976,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752557208,"statfs":{"total":773094113280,"available":770618580992,"internally_reserved":0,"allocated":8167424,"data_stored":4061964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27358,"num_objects":-9,"num_object_clones":0,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":-644,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.997044"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.615914+0000","last_change":"2023-11-06T20:59:21.605888+0000","last_active":"2023-11-06T20:59:21.615914+0000","last_peered":"2023-11-06T20:59:21.615914+0000","last_clean":"2023-11-06T20:59:21.615914+0000","last_became_active":"2023-11-06T20:59:21.605540+0000","last_became_peered":"2023-11-06T20:59:21.605540+0000","last_unstale":"2023-11-06T20:59:21.615914+0000","last_undegraded":"2023-11-06T20:59:21.615914+0000","last_fullsized":"2023-11-06T20:59:21.615914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3f","version":"83'3","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.613298+0000","last_change":"2023-11-06T20:59:21.115109+0000","last_active":"2023-11-06T20:59:21.613298+0000","last_peered":"2023-11-06T20:59:21.613298+0000","last_clean":"2023-11-06T20:59:21.613298+0000","last_became_active":"2023-11-06T20:59:21.114828+0000","last_became_peered":"2023-11-06T20:59:21.114828+0000","last_unstale":"2023-11-06T20:59:21.613298+0000","last_undegraded":"2023-11-06T20:59:21.613298+0000","last_fullsized":"2023-11-06T20:59:21.613298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.619205+0000","last_change":"2023-11-06T20:59:21.610491+0000","last_active":"2023-11-06T20:59:21.619205+0000","last_peered":"2023-11-06T20:59:21.619205+0000","last_clean":"2023-11-06T20:59:21.619205+0000","last_became_active":"2023-11-06T20:59:21.609771+0000","last_became_peered":"2023-11-06T20:59:21.609771+0000","last_unstale":"2023-11-06T20:59:21.619205+0000","last_undegraded":"2023-11-06T20:59:21.619205+0000","last_fullsized":"2023-11-06T20:59:21.619205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.587268+0000","last_change":"2023-11-06T20:59:21.120516+0000","last_active":"2023-11-06T20:59:21.587268+0000","last_peered":"2023-11-06T20:59:21.587268+0000","last_clean":"2023-11-06T20:59:21.587268+0000","last_became_active":"2023-11-06T20:59:21.119700+0000","last_became_peered":"2023-11-06T20:59:21.119700+0000","last_unstale":"2023-11-06T20:59:21.587268+0000","last_undegraded":"2023-11-06T20:59:21.587268+0000","last_fullsized":"2023-11-06T20:59:21.587268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.599917+0000","last_change":"2023-11-06T20:59:21.593772+0000","last_active":"2023-11-06T20:59:21.599917+0000","last_peered":"2023-11-06T20:59:21.599917+0000","last_clean":"2023-11-06T20:59:21.599917+0000","last_became_active":"2023-11-06T20:59:21.593235+0000","last_became_peered":"2023-11-06T20:59:21.593235+0000","last_unstale":"2023-11-06T20:59:21.599917+0000","last_undegraded":"2023-11-06T20:59:21.599917+0000","last_fullsized":"2023-11-06T20:59:21.599917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.608194+0000","last_change":"2023-11-06T20:59:21.120544+0000","last_active":"2023-11-06T20:59:21.608194+0000","last_peered":"2023-11-06T20:59:21.608194+0000","last_clean":"2023-11-06T20:59:21.608194+0000","last_became_active":"2023-11-06T20:59:21.120255+0000","last_became_peered":"2023-11-06T20:59:21.120255+0000","last_unstale":"2023-11-06T20:59:21.608194+0000","last_undegraded":"2023-11-06T20:59:21.608194+0000","last_fullsized":"2023-11-06T20:59:21.608194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.613336+0000","last_change":"2023-11-06T20:59:21.609373+0000","last_active":"2023-11-06T20:59:21.613336+0000","last_peered":"2023-11-06T20:59:21.613336+0000","last_clean":"2023-11-06T20:59:21.613336+0000","last_became_active":"2023-11-06T20:59:21.609036+0000","last_became_peered":"2023-11-06T20:59:21.609036+0000","last_unstale":"2023-11-06T20:59:21.613336+0000","last_undegraded":"2023-11-06T20:59:21.613336+0000","last_fullsized":"2023-11-06T20:59:21.613336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3c","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621411+0000","last_change":"2023-11-06T20:59:21.122869+0000","last_active":"2023-11-06T20:59:21.621411+0000","last_peered":"2023-11-06T20:59:21.621411+0000","last_clean":"2023-11-06T20:59:21.621411+0000","last_became_active":"2023-11-06T20:59:21.122587+0000","last_became_peered":"2023-11-06T20:59:21.122587+0000","last_unstale":"2023-11-06T20:59:21.621411+0000","last_undegraded":"2023-11-06T20:59:21.621411+0000","last_fullsized":"2023-11-06T20:59:21.621411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.618773+0000","last_change":"2023-11-06T20:59:21.608679+0000","last_active":"2023-11-06T20:59:21.618773+0000","last_peered":"2023-11-06T20:59:21.618773+0000","last_clean":"2023-11-06T20:59:21.618773+0000","last_became_active":"2023-11-06T20:59:21.608222+0000","last_became_peered":"2023-11-06T20:59:21.608222+0000","last_unstale":"2023-11-06T20:59:21.618773+0000","last_undegraded":"2023-11-06T20:59:21.618773+0000","last_fullsized":"2023-11-06T20:59:21.618773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.619521+0000","last_change":"2023-11-06T20:59:21.112728+0000","last_active":"2023-11-06T20:59:21.619521+0000","last_peered":"2023-11-06T20:59:21.619521+0000","last_clean":"2023-11-06T20:59:21.619521+0000","last_became_active":"2023-11-06T20:59:21.112438+0000","last_became_peered":"2023-11-06T20:59:21.112438+0000","last_unstale":"2023-11-06T20:59:21.619521+0000","last_undegraded":"2023-11-06T20:59:21.619521+0000","last_fullsized":"2023-11-06T20:59:21.619521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611960+0000","last_change":"2023-11-06T20:59:21.606594+0000","last_active":"2023-11-06T20:59:21.611960+0000","last_peered":"2023-11-06T20:59:21.611960+0000","last_clean":"2023-11-06T20:59:21.611960+0000","last_became_active":"2023-11-06T20:59:21.606314+0000","last_became_peered":"2023-11-06T20:59:21.606314+0000","last_unstale":"2023-11-06T20:59:21.611960+0000","last_undegraded":"2023-11-06T20:59:21.611960+0000","last_fullsized":"2023-11-06T20:59:21.611960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611352+0000","last_change":"2023-11-06T20:59:21.123819+0000","last_active":"2023-11-06T20:59:21.611352+0000","last_peered":"2023-11-06T20:59:21.611352+0000","last_clean":"2023-11-06T20:59:21.611352+0000","last_became_active":"2023-11-06T20:59:21.123521+0000","last_became_peered":"2023-11-06T20:59:21.123521+0000","last_unstale":"2023-11-06T20:59:21.611352+0000","last_undegraded":"2023-11-06T20:59:21.611352+0000","last_fullsized":"2023-11-06T20:59:21.611352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.608277+0000","last_change":"2023-11-06T20:59:21.603532+0000","last_active":"2023-11-06T20:59:21.608277+0000","last_peered":"2023-11-06T20:59:21.608277+0000","last_clean":"2023-11-06T20:59:21.608277+0000","last_became_active":"2023-11-06T20:59:21.603207+0000","last_became_peered":"2023-11-06T20:59:21.603207+0000","last_unstale":"2023-11-06T20:59:21.608277+0000","last_undegraded":"2023-11-06T20:59:21.608277+0000","last_fullsized":"2023-11-06T20:59:21.608277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581964+0000","last_change":"2023-11-06T20:59:21.120942+0000","last_active":"2023-11-06T20:59:21.581964+0000","last_peered":"2023-11-06T20:59:21.581964+0000","last_clean":"2023-11-06T20:59:21.581964+0000","last_became_active":"2023-11-06T20:59:21.120654+0000","last_became_peered":"2023-11-06T20:59:21.120654+0000","last_unstale":"2023-11-06T20:59:21.581964+0000","last_undegraded":"2023-11-06T20:59:21.581964+0000","last_fullsized":"2023-11-06T20:59:21.581964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.618831+0000","last_change":"2023-11-06T20:59:21.614348+0000","last_active":"2023-11-06T20:59:21.618831+0000","last_peered":"2023-11-06T20:59:21.618831+0000","last_clean":"2023-11-06T20:59:21.618831+0000","last_became_active":"2023-11-06T20:59:21.614075+0000","last_became_peered":"2023-11-06T20:59:21.614075+0000","last_unstale":"2023-11-06T20:59:21.618831+0000","last_undegraded":"2023-11-06T20:59:21.618831+0000","last_fullsized":"2023-11-06T20:59:21.618831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.583446+0000","last_change":"2023-11-06T20:59:21.120782+0000","last_active":"2023-11-06T20:59:21.583446+0000","last_peered":"2023-11-06T20:59:21.583446+0000","last_clean":"2023-11-06T20:59:21.583446+0000","last_became_active":"2023-11-06T20:59:21.120434+0000","last_became_peered":"2023-11-06T20:59:21.120434+0000","last_unstale":"2023-11-06T20:59:21.583446+0000","last_undegraded":"2023-11-06T20:59:21.583446+0000","last_fullsized":"2023-11-06T20:59:21.583446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611049+0000","last_change":"2023-11-06T20:59:21.601939+0000","last_active":"2023-11-06T20:59:21.611049+0000","last_peered":"2023-11-06T20:59:21.611049+0000","last_clean":"2023-11-06T20:59:21.611049+0000","last_became_active":"2023-11-06T20:59:21.601672+0000","last_became_peered":"2023-11-06T20:59:21.601672+0000","last_unstale":"2023-11-06T20:59:21.611049+0000","last_undegraded":"2023-11-06T20:59:21.611049+0000","last_fullsized":"2023-11-06T20:59:21.611049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.588824+0000","last_change":"2023-11-06T20:59:21.118725+0000","last_active":"2023-11-06T20:59:21.588824+0000","last_peered":"2023-11-06T20:59:21.588824+0000","last_clean":"2023-11-06T20:59:21.588824+0000","last_became_active":"2023-11-06T20:59:21.118409+0000","last_became_peered":"2023-11-06T20:59:21.118409+0000","last_unstale":"2023-11-06T20:59:21.588824+0000","last_undegraded":"2023-11-06T20:59:21.588824+0000","last_fullsized":"2023-11-06T20:59:21.588824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.616099+0000","last_change":"2023-11-06T20:59:21.609560+0000","last_active":"2023-11-06T20:59:21.616099+0000","last_peered":"2023-11-06T20:59:21.616099+0000","last_clean":"2023-11-06T20:59:21.616099+0000","last_became_active":"2023-11-06T20:59:21.609247+0000","last_became_peered":"2023-11-06T20:59:21.609247+0000","last_unstale":"2023-11-06T20:59:21.616099+0000","last_undegraded":"2023-11-06T20:59:21.616099+0000","last_fullsized":"2023-11-06T20:59:21.616099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.598210+0000","last_change":"2023-11-06T20:59:21.123748+0000","last_active":"2023-11-06T20:59:21.598210+0000","last_peered":"2023-11-06T20:59:21.598210+0000","last_clean":"2023-11-06T20:59:21.598210+0000","last_became_active":"2023-11-06T20:59:21.123295+0000","last_became_peered":"2023-11-06T20:59:21.123295+0000","last_unstale":"2023-11-06T20:59:21.598210+0000","last_undegraded":"2023-11-06T20:59:21.598210+0000","last_fullsized":"2023-11-06T20:59:21.598210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.612206+0000","last_change":"2023-11-06T20:59:21.607444+0000","last_active":"2023-11-06T20:59:21.612206+0000","last_peered":"2023-11-06T20:59:21.612206+0000","last_clean":"2023-11-06T20:59:21.612206+0000","last_became_active":"2023-11-06T20:59:21.606576+0000","last_became_peered":"2023-11-06T20:59:21.606576+0000","last_unstale":"2023-11-06T20:59:21.612206+0000","last_undegraded":"2023-11-06T20:59:21.612206+0000","last_fullsized":"2023-11-06T20:59:21.612206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.589054+0000","last_change":"2023-11-06T20:59:21.124501+0000","last_active":"2023-11-06T20:59:21.589054+0000","last_peered":"2023-11-06T20:59:21.589054+0000","last_clean":"2023-11-06T20:59:21.589054+0000","last_became_active":"2023-11-06T20:59:21.124153+0000","last_became_peered":"2023-11-06T20:59:21.124153+0000","last_unstale":"2023-11-06T20:59:21.589054+0000","last_undegraded":"2023-11-06T20:59:21.589054+0000","last_fullsized":"2023-11-06T20:59:21.589054+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.614613+0000","last_change":"2023-11-06T20:59:21.602233+0000","last_active":"2023-11-06T20:59:21.614613+0000","last_peered":"2023-11-06T20:59:21.614613+0000","last_clean":"2023-11-06T20:59:21.614613+0000","last_became_active":"2023-11-06T20:59:21.601931+0000","last_became_peered":"2023-11-06T20:59:21.601931+0000","last_unstale":"2023-11-06T20:59:21.614613+0000","last_undegraded":"2023-11-06T20:59:21.614613+0000","last_fullsized":"2023-11-06T20:59:21.614613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.34","version":"83'4","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.623999+0000","last_change":"2023-11-06T20:59:21.116117+0000","last_active":"2023-11-06T20:59:21.623999+0000","last_peered":"2023-11-06T20:59:21.623999+0000","last_clean":"2023-11-06T20:59:21.623999+0000","last_became_active":"2023-11-06T20:59:21.115806+0000","last_became_peered":"2023-11-06T20:59:21.115806+0000","last_unstale":"2023-11-06T20:59:21.623999+0000","last_undegraded":"2023-11-06T20:59:21.623999+0000","last_fullsized":"2023-11-06T20:59:21.623999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.601736+0000","last_change":"2023-11-06T20:59:21.598954+0000","last_active":"2023-11-06T20:59:21.601736+0000","last_peered":"2023-11-06T20:59:21.601736+0000","last_clean":"2023-11-06T20:59:21.601736+0000","last_became_active":"2023-11-06T20:59:21.598605+0000","last_became_peered":"2023-11-06T20:59:21.598605+0000","last_unstale":"2023-11-06T20:59:21.601736+0000","last_undegraded":"2023-11-06T20:59:21.601736+0000","last_fullsized":"2023-11-06T20:59:21.601736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.33","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.602367+0000","last_change":"2023-11-06T20:59:21.116919+0000","last_active":"2023-11-06T20:59:21.602367+0000","last_peered":"2023-11-06T20:59:21.602367+0000","last_clean":"2023-11-06T20:59:21.602367+0000","last_became_active":"2023-11-06T20:59:21.116664+0000","last_became_peered":"2023-11-06T20:59:21.116664+0000","last_unstale":"2023-11-06T20:59:21.602367+0000","last_undegraded":"2023-11-06T20:59:21.602367+0000","last_fullsized":"2023-11-06T20:59:21.602367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.607633+0000","last_change":"2023-11-06T20:59:21.603550+0000","last_active":"2023-11-06T20:59:21.607633+0000","last_peered":"2023-11-06T20:59:21.607633+0000","last_clean":"2023-11-06T20:59:21.607633+0000","last_became_active":"2023-11-06T20:59:21.602050+0000","last_became_peered":"2023-11-06T20:59:21.602050+0000","last_unstale":"2023-11-06T20:59:21.607633+0000","last_undegraded":"2023-11-06T20:59:21.607633+0000","last_fullsized":"2023-11-06T20:59:21.607633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.602582+0000","last_change":"2023-11-06T20:59:21.115914+0000","last_active":"2023-11-06T20:59:21.602582+0000","last_peered":"2023-11-06T20:59:21.602582+0000","last_clean":"2023-11-06T20:59:21.602582+0000","last_became_active":"2023-11-06T20:59:21.113865+0000","last_became_peered":"2023-11-06T20:59:21.113865+0000","last_unstale":"2023-11-06T20:59:21.602582+0000","last_undegraded":"2023-11-06T20:59:21.602582+0000","last_fullsized":"2023-11-06T20:59:21.602582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604951+0000","last_change":"2023-11-06T20:59:21.600283+0000","last_active":"2023-11-06T20:59:21.604951+0000","last_peered":"2023-11-06T20:59:21.604951+0000","last_clean":"2023-11-06T20:59:21.604951+0000","last_became_active":"2023-11-06T20:59:21.599971+0000","last_became_peered":"2023-11-06T20:59:21.599971+0000","last_unstale":"2023-11-06T20:59:21.604951+0000","last_undegraded":"2023-11-06T20:59:21.604951+0000","last_fullsized":"2023-11-06T20:59:21.604951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.592674+0000","last_change":"2023-11-06T20:59:21.120032+0000","last_active":"2023-11-06T20:59:21.592674+0000","last_peered":"2023-11-06T20:59:21.592674+0000","last_clean":"2023-11-06T20:59:21.592674+0000","last_became_active":"2023-11-06T20:59:21.119692+0000","last_became_peered":"2023-11-06T20:59:21.119692+0000","last_unstale":"2023-11-06T20:59:21.592674+0000","last_undegraded":"2023-11-06T20:59:21.592674+0000","last_fullsized":"2023-11-06T20:59:21.592674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621106+0000","last_change":"2023-11-06T20:59:21.617605+0000","last_active":"2023-11-06T20:59:21.621106+0000","last_peered":"2023-11-06T20:59:21.621106+0000","last_clean":"2023-11-06T20:59:21.621106+0000","last_became_active":"2023-11-06T20:59:21.616791+0000","last_became_peered":"2023-11-06T20:59:21.616791+0000","last_unstale":"2023-11-06T20:59:21.621106+0000","last_undegraded":"2023-11-06T20:59:21.621106+0000","last_fullsized":"2023-11-06T20:59:21.621106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"83'4","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611091+0000","last_change":"2023-11-06T20:59:21.110306+0000","last_active":"2023-11-06T20:59:21.611091+0000","last_peered":"2023-11-06T20:59:21.611091+0000","last_clean":"2023-11-06T20:59:21.611091+0000","last_became_active":"2023-11-06T20:59:21.110024+0000","last_became_peered":"2023-11-06T20:59:21.110024+0000","last_unstale":"2023-11-06T20:59:21.611091+0000","last_undegraded":"2023-11-06T20:59:21.611091+0000","last_fullsized":"2023-11-06T20:59:21.611091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.618696+0000","last_change":"2023-11-06T20:59:21.614576+0000","last_active":"2023-11-06T20:59:21.618696+0000","last_peered":"2023-11-06T20:59:21.618696+0000","last_clean":"2023-11-06T20:59:21.618696+0000","last_became_active":"2023-11-06T20:59:21.614090+0000","last_became_peered":"2023-11-06T20:59:21.614090+0000","last_unstale":"2023-11-06T20:59:21.618696+0000","last_undegraded":"2023-11-06T20:59:21.618696+0000","last_fullsized":"2023-11-06T20:59:21.618696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.600174+0000","last_change":"2023-11-06T20:59:21.123219+0000","last_active":"2023-11-06T20:59:21.600174+0000","last_peered":"2023-11-06T20:59:21.600174+0000","last_clean":"2023-11-06T20:59:21.600174+0000","last_became_active":"2023-11-06T20:59:21.122930+0000","last_became_peered":"2023-11-06T20:59:21.122930+0000","last_unstale":"2023-11-06T20:59:21.600174+0000","last_undegraded":"2023-11-06T20:59:21.600174+0000","last_fullsized":"2023-11-06T20:59:21.600174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.616111+0000","last_change":"2023-11-06T20:59:21.609580+0000","last_active":"2023-11-06T20:59:21.616111+0000","last_peered":"2023-11-06T20:59:21.616111+0000","last_clean":"2023-11-06T20:59:21.616111+0000","last_became_active":"2023-11-06T20:59:21.609273+0000","last_became_peered":"2023-11-06T20:59:21.609273+0000","last_unstale":"2023-11-06T20:59:21.616111+0000","last_undegraded":"2023-11-06T20:59:21.616111+0000","last_fullsized":"2023-11-06T20:59:21.616111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.606937+0000","last_change":"2023-11-06T20:59:21.117925+0000","last_active":"2023-11-06T20:59:21.606937+0000","last_peered":"2023-11-06T20:59:21.606937+0000","last_clean":"2023-11-06T20:59:21.606937+0000","last_became_active":"2023-11-06T20:59:21.117459+0000","last_became_peered":"2023-11-06T20:59:21.117459+0000","last_unstale":"2023-11-06T20:59:21.606937+0000","last_undegraded":"2023-11-06T20:59:21.606937+0000","last_fullsized":"2023-11-06T20:59:21.606937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.615441+0000","last_change":"2023-11-06T20:59:21.608863+0000","last_active":"2023-11-06T20:59:21.615441+0000","last_peered":"2023-11-06T20:59:21.615441+0000","last_clean":"2023-11-06T20:59:21.615441+0000","last_became_active":"2023-11-06T20:59:21.608599+0000","last_became_peered":"2023-11-06T20:59:21.608599+0000","last_unstale":"2023-11-06T20:59:21.615441+0000","last_undegraded":"2023-11-06T20:59:21.615441+0000","last_fullsized":"2023-11-06T20:59:21.615441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.601311+0000","last_change":"2023-11-06T20:59:21.111916+0000","last_active":"2023-11-06T20:59:21.601311+0000","last_peered":"2023-11-06T20:59:21.601311+0000","last_clean":"2023-11-06T20:59:21.601311+0000","last_became_active":"2023-11-06T20:59:21.111558+0000","last_became_peered":"2023-11-06T20:59:21.111558+0000","last_unstale":"2023-11-06T20:59:21.601311+0000","last_undegraded":"2023-11-06T20:59:21.601311+0000","last_fullsized":"2023-11-06T20:59:21.601311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604452+0000","last_change":"2023-11-06T20:59:21.599682+0000","last_active":"2023-11-06T20:59:21.604452+0000","last_peered":"2023-11-06T20:59:21.604452+0000","last_clean":"2023-11-06T20:59:21.604452+0000","last_became_active":"2023-11-06T20:59:21.598615+0000","last_became_peered":"2023-11-06T20:59:21.598615+0000","last_unstale":"2023-11-06T20:59:21.604452+0000","last_undegraded":"2023-11-06T20:59:21.604452+0000","last_fullsized":"2023-11-06T20:59:21.604452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.590142+0000","last_change":"2023-11-06T20:59:21.123381+0000","last_active":"2023-11-06T20:59:21.590142+0000","last_peered":"2023-11-06T20:59:21.590142+0000","last_clean":"2023-11-06T20:59:21.590142+0000","last_became_active":"2023-11-06T20:59:21.123095+0000","last_became_peered":"2023-11-06T20:59:21.123095+0000","last_unstale":"2023-11-06T20:59:21.590142+0000","last_undegraded":"2023-11-06T20:59:21.590142+0000","last_fullsized":"2023-11-06T20:59:21.590142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.600114+0000","last_change":"2023-11-06T20:59:21.593994+0000","last_active":"2023-11-06T20:59:21.600114+0000","last_peered":"2023-11-06T20:59:21.600114+0000","last_clean":"2023-11-06T20:59:21.600114+0000","last_became_active":"2023-11-06T20:59:21.593731+0000","last_became_peered":"2023-11-06T20:59:21.593731+0000","last_unstale":"2023-11-06T20:59:21.600114+0000","last_undegraded":"2023-11-06T20:59:21.600114+0000","last_fullsized":"2023-11-06T20:59:21.600114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.616567+0000","last_change":"2023-11-06T20:59:21.123486+0000","last_active":"2023-11-06T20:59:21.616567+0000","last_peered":"2023-11-06T20:59:21.616567+0000","last_clean":"2023-11-06T20:59:21.616567+0000","last_became_active":"2023-11-06T20:59:21.123191+0000","last_became_peered":"2023-11-06T20:59:21.123191+0000","last_unstale":"2023-11-06T20:59:21.616567+0000","last_undegraded":"2023-11-06T20:59:21.616567+0000","last_fullsized":"2023-11-06T20:59:21.616567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.601162+0000","last_change":"2023-11-06T20:59:21.596219+0000","last_active":"2023-11-06T20:59:21.601162+0000","last_peered":"2023-11-06T20:59:21.601162+0000","last_clean":"2023-11-06T20:59:21.601162+0000","last_became_active":"2023-11-06T20:59:21.595921+0000","last_became_peered":"2023-11-06T20:59:21.595921+0000","last_unstale":"2023-11-06T20:59:21.601162+0000","last_undegraded":"2023-11-06T20:59:21.601162+0000","last_fullsized":"2023-11-06T20:59:21.601162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.587330+0000","last_change":"2023-11-06T20:59:21.119347+0000","last_active":"2023-11-06T20:59:21.587330+0000","last_peered":"2023-11-06T20:59:21.587330+0000","last_clean":"2023-11-06T20:59:21.587330+0000","last_became_active":"2023-11-06T20:59:21.118661+0000","last_became_peered":"2023-11-06T20:59:21.118661+0000","last_unstale":"2023-11-06T20:59:21.587330+0000","last_undegraded":"2023-11-06T20:59:21.587330+0000","last_fullsized":"2023-11-06T20:59:21.587330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.605583+0000","last_change":"2023-11-06T20:59:21.601132+0000","last_active":"2023-11-06T20:59:21.605583+0000","last_peered":"2023-11-06T20:59:21.605583+0000","last_clean":"2023-11-06T20:59:21.605583+0000","last_became_active":"2023-11-06T20:59:21.600314+0000","last_became_peered":"2023-11-06T20:59:21.600314+0000","last_unstale":"2023-11-06T20:59:21.605583+0000","last_undegraded":"2023-11-06T20:59:21.605583+0000","last_fullsized":"2023-11-06T20:59:21.605583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.582068+0000","last_change":"2023-11-06T20:59:21.112822+0000","last_active":"2023-11-06T20:59:21.582068+0000","last_peered":"2023-11-06T20:59:21.582068+0000","last_clean":"2023-11-06T20:59:21.582068+0000","last_became_active":"2023-11-06T20:59:21.112477+0000","last_became_peered":"2023-11-06T20:59:21.112477+0000","last_unstale":"2023-11-06T20:59:21.582068+0000","last_undegraded":"2023-11-06T20:59:21.582068+0000","last_fullsized":"2023-11-06T20:59:21.582068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.610703+0000","last_change":"2023-11-06T20:59:21.605273+0000","last_active":"2023-11-06T20:59:21.610703+0000","last_peered":"2023-11-06T20:59:21.610703+0000","last_clean":"2023-11-06T20:59:21.610703+0000","last_became_active":"2023-11-06T20:59:21.604839+0000","last_became_peered":"2023-11-06T20:59:21.604839+0000","last_unstale":"2023-11-06T20:59:21.610703+0000","last_undegraded":"2023-11-06T20:59:21.610703+0000","last_fullsized":"2023-11-06T20:59:21.610703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.586590+0000","last_change":"2023-11-06T20:59:21.118874+0000","last_active":"2023-11-06T20:59:21.586590+0000","last_peered":"2023-11-06T20:59:21.586590+0000","last_clean":"2023-11-06T20:59:21.586590+0000","last_became_active":"2023-11-06T20:59:21.118576+0000","last_became_peered":"2023-11-06T20:59:21.118576+0000","last_unstale":"2023-11-06T20:59:21.586590+0000","last_undegraded":"2023-11-06T20:59:21.586590+0000","last_fullsized":"2023-11-06T20:59:21.586590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.616248+0000","last_change":"2023-11-06T20:59:21.612078+0000","last_active":"2023-11-06T20:59:21.616248+0000","last_peered":"2023-11-06T20:59:21.616248+0000","last_clean":"2023-11-06T20:59:21.616248+0000","last_became_active":"2023-11-06T20:59:21.611760+0000","last_became_peered":"2023-11-06T20:59:21.611760+0000","last_unstale":"2023-11-06T20:59:21.616248+0000","last_undegraded":"2023-11-06T20:59:21.616248+0000","last_fullsized":"2023-11-06T20:59:21.616248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.586134+0000","last_change":"2023-11-06T20:59:21.120098+0000","last_active":"2023-11-06T20:59:21.586134+0000","last_peered":"2023-11-06T20:59:21.586134+0000","last_clean":"2023-11-06T20:59:21.586134+0000","last_became_active":"2023-11-06T20:59:21.119773+0000","last_became_peered":"2023-11-06T20:59:21.119773+0000","last_unstale":"2023-11-06T20:59:21.586134+0000","last_undegraded":"2023-11-06T20:59:21.586134+0000","last_fullsized":"2023-11-06T20:59:21.586134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621078+0000","last_change":"2023-11-06T20:59:21.616580+0000","last_active":"2023-11-06T20:59:21.621078+0000","last_peered":"2023-11-06T20:59:21.621078+0000","last_clean":"2023-11-06T20:59:21.621078+0000","last_became_active":"2023-11-06T20:59:21.616367+0000","last_became_peered":"2023-11-06T20:59:21.616367+0000","last_unstale":"2023-11-06T20:59:21.621078+0000","last_undegraded":"2023-11-06T20:59:21.621078+0000","last_fullsized":"2023-11-06T20:59:21.621078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.592513+0000","last_change":"2023-11-06T20:59:21.125545+0000","last_active":"2023-11-06T20:59:21.592513+0000","last_peered":"2023-11-06T20:59:21.592513+0000","last_clean":"2023-11-06T20:59:21.592513+0000","last_became_active":"2023-11-06T20:59:21.125208+0000","last_became_peered":"2023-11-06T20:59:21.125208+0000","last_unstale":"2023-11-06T20:59:21.592513+0000","last_undegraded":"2023-11-06T20:59:21.592513+0000","last_fullsized":"2023-11-06T20:59:21.592513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.608049+0000","last_change":"2023-11-06T20:59:21.600099+0000","last_active":"2023-11-06T20:59:21.608049+0000","last_peered":"2023-11-06T20:59:21.608049+0000","last_clean":"2023-11-06T20:59:21.608049+0000","last_became_active":"2023-11-06T20:59:21.598760+0000","last_became_peered":"2023-11-06T20:59:21.598760+0000","last_unstale":"2023-11-06T20:59:21.608049+0000","last_undegraded":"2023-11-06T20:59:21.608049+0000","last_fullsized":"2023-11-06T20:59:21.608049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.585728+0000","last_change":"2023-11-06T20:59:21.122853+0000","last_active":"2023-11-06T20:59:21.585728+0000","last_peered":"2023-11-06T20:59:21.585728+0000","last_clean":"2023-11-06T20:59:21.585728+0000","last_became_active":"2023-11-06T20:59:21.122531+0000","last_became_peered":"2023-11-06T20:59:21.122531+0000","last_unstale":"2023-11-06T20:59:21.585728+0000","last_undegraded":"2023-11-06T20:59:21.585728+0000","last_fullsized":"2023-11-06T20:59:21.585728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621044+0000","last_change":"2023-11-06T20:59:21.613606+0000","last_active":"2023-11-06T20:59:21.621044+0000","last_peered":"2023-11-06T20:59:21.621044+0000","last_clean":"2023-11-06T20:59:21.621044+0000","last_became_active":"2023-11-06T20:59:21.607808+0000","last_became_peered":"2023-11-06T20:59:21.607808+0000","last_unstale":"2023-11-06T20:59:21.621044+0000","last_undegraded":"2023-11-06T20:59:21.621044+0000","last_fullsized":"2023-11-06T20:59:21.621044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584136+0000","last_change":"2023-11-06T20:59:21.116882+0000","last_active":"2023-11-06T20:59:21.584136+0000","last_peered":"2023-11-06T20:59:21.584136+0000","last_clean":"2023-11-06T20:59:21.584136+0000","last_became_active":"2023-11-06T20:59:21.116298+0000","last_became_peered":"2023-11-06T20:59:21.116298+0000","last_unstale":"2023-11-06T20:59:21.584136+0000","last_undegraded":"2023-11-06T20:59:21.584136+0000","last_fullsized":"2023-11-06T20:59:21.584136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611146+0000","last_change":"2023-11-06T20:59:21.603256+0000","last_active":"2023-11-06T20:59:21.611146+0000","last_peered":"2023-11-06T20:59:21.611146+0000","last_clean":"2023-11-06T20:59:21.611146+0000","last_became_active":"2023-11-06T20:59:21.599751+0000","last_became_peered":"2023-11-06T20:59:21.599751+0000","last_unstale":"2023-11-06T20:59:21.611146+0000","last_undegraded":"2023-11-06T20:59:21.611146+0000","last_fullsized":"2023-11-06T20:59:21.611146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.583977+0000","last_change":"2023-11-06T20:59:21.117270+0000","last_active":"2023-11-06T20:59:21.583977+0000","last_peered":"2023-11-06T20:59:21.583977+0000","last_clean":"2023-11-06T20:59:21.583977+0000","last_became_active":"2023-11-06T20:59:21.116899+0000","last_became_peered":"2023-11-06T20:59:21.116899+0000","last_unstale":"2023-11-06T20:59:21.583977+0000","last_undegraded":"2023-11-06T20:59:21.583977+0000","last_fullsized":"2023-11-06T20:59:21.583977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.599797+0000","last_change":"2023-11-06T20:59:21.596043+0000","last_active":"2023-11-06T20:59:21.599797+0000","last_peered":"2023-11-06T20:59:21.599797+0000","last_clean":"2023-11-06T20:59:21.599797+0000","last_became_active":"2023-11-06T20:59:21.595772+0000","last_became_peered":"2023-11-06T20:59:21.595772+0000","last_unstale":"2023-11-06T20:59:21.599797+0000","last_undegraded":"2023-11-06T20:59:21.599797+0000","last_fullsized":"2023-11-06T20:59:21.599797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581258+0000","last_change":"2023-11-06T20:59:21.124894+0000","last_active":"2023-11-06T20:59:21.581258+0000","last_peered":"2023-11-06T20:59:21.581258+0000","last_clean":"2023-11-06T20:59:21.581258+0000","last_became_active":"2023-11-06T20:59:21.124600+0000","last_became_peered":"2023-11-06T20:59:21.124600+0000","last_unstale":"2023-11-06T20:59:21.581258+0000","last_undegraded":"2023-11-06T20:59:21.581258+0000","last_fullsized":"2023-11-06T20:59:21.581258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.617794+0000","last_change":"2023-11-06T20:59:21.610354+0000","last_active":"2023-11-06T20:59:21.617794+0000","last_peered":"2023-11-06T20:59:21.617794+0000","last_clean":"2023-11-06T20:59:21.617794+0000","last_became_active":"2023-11-06T20:59:21.610076+0000","last_became_peered":"2023-11-06T20:59:21.610076+0000","last_unstale":"2023-11-06T20:59:21.617794+0000","last_undegraded":"2023-11-06T20:59:21.617794+0000","last_fullsized":"2023-11-06T20:59:21.617794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.586797+0000","last_change":"2023-11-06T20:59:21.119857+0000","last_active":"2023-11-06T20:59:21.586797+0000","last_peered":"2023-11-06T20:59:21.586797+0000","last_clean":"2023-11-06T20:59:21.586797+0000","last_became_active":"2023-11-06T20:59:21.119497+0000","last_became_peered":"2023-11-06T20:59:21.119497+0000","last_unstale":"2023-11-06T20:59:21.586797+0000","last_undegraded":"2023-11-06T20:59:21.586797+0000","last_fullsized":"2023-11-06T20:59:21.586797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.618274+0000","last_change":"2023-11-06T20:59:21.611195+0000","last_active":"2023-11-06T20:59:21.618274+0000","last_peered":"2023-11-06T20:59:21.618274+0000","last_clean":"2023-11-06T20:59:21.618274+0000","last_became_active":"2023-11-06T20:59:21.609390+0000","last_became_peered":"2023-11-06T20:59:21.609390+0000","last_unstale":"2023-11-06T20:59:21.618274+0000","last_undegraded":"2023-11-06T20:59:21.618274+0000","last_fullsized":"2023-11-06T20:59:21.618274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584361+0000","last_change":"2023-11-06T20:59:21.118446+0000","last_active":"2023-11-06T20:59:21.584361+0000","last_peered":"2023-11-06T20:59:21.584361+0000","last_clean":"2023-11-06T20:59:21.584361+0000","last_became_active":"2023-11-06T20:59:21.118155+0000","last_became_peered":"2023-11-06T20:59:21.118155+0000","last_unstale":"2023-11-06T20:59:21.584361+0000","last_undegraded":"2023-11-06T20:59:21.584361+0000","last_fullsized":"2023-11-06T20:59:21.584361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.618984+0000","last_change":"2023-11-06T20:59:21.615815+0000","last_active":"2023-11-06T20:59:21.618984+0000","last_peered":"2023-11-06T20:59:21.618984+0000","last_clean":"2023-11-06T20:59:21.618984+0000","last_became_active":"2023-11-06T20:59:21.615498+0000","last_became_peered":"2023-11-06T20:59:21.615498+0000","last_unstale":"2023-11-06T20:59:21.618984+0000","last_undegraded":"2023-11-06T20:59:21.618984+0000","last_fullsized":"2023-11-06T20:59:21.618984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611698+0000","last_change":"2023-11-06T20:59:21.114630+0000","last_active":"2023-11-06T20:59:21.611698+0000","last_peered":"2023-11-06T20:59:21.611698+0000","last_clean":"2023-11-06T20:59:21.611698+0000","last_became_active":"2023-11-06T20:59:21.114310+0000","last_became_peered":"2023-11-06T20:59:21.114310+0000","last_unstale":"2023-11-06T20:59:21.611698+0000","last_undegraded":"2023-11-06T20:59:21.611698+0000","last_fullsized":"2023-11-06T20:59:21.611698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.601633+0000","last_change":"2023-11-06T20:59:21.597521+0000","last_active":"2023-11-06T20:59:21.601633+0000","last_peered":"2023-11-06T20:59:21.601633+0000","last_clean":"2023-11-06T20:59:21.601633+0000","last_became_active":"2023-11-06T20:59:21.597250+0000","last_became_peered":"2023-11-06T20:59:21.597250+0000","last_unstale":"2023-11-06T20:59:21.601633+0000","last_undegraded":"2023-11-06T20:59:21.601633+0000","last_fullsized":"2023-11-06T20:59:21.601633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611401+0000","last_change":"2023-11-06T20:59:21.123573+0000","last_active":"2023-11-06T20:59:21.611401+0000","last_peered":"2023-11-06T20:59:21.611401+0000","last_clean":"2023-11-06T20:59:21.611401+0000","last_became_active":"2023-11-06T20:59:21.122963+0000","last_became_peered":"2023-11-06T20:59:21.122963+0000","last_unstale":"2023-11-06T20:59:21.611401+0000","last_undegraded":"2023-11-06T20:59:21.611401+0000","last_fullsized":"2023-11-06T20:59:21.611401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.599895+0000","last_change":"2023-11-06T20:59:21.593576+0000","last_active":"2023-11-06T20:59:21.599895+0000","last_peered":"2023-11-06T20:59:21.599895+0000","last_clean":"2023-11-06T20:59:21.599895+0000","last_became_active":"2023-11-06T20:59:21.593202+0000","last_became_peered":"2023-11-06T20:59:21.593202+0000","last_unstale":"2023-11-06T20:59:21.599895+0000","last_undegraded":"2023-11-06T20:59:21.599895+0000","last_fullsized":"2023-11-06T20:59:21.599895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611098+0000","last_change":"2023-11-06T20:59:21.115729+0000","last_active":"2023-11-06T20:59:21.611098+0000","last_peered":"2023-11-06T20:59:21.611098+0000","last_clean":"2023-11-06T20:59:21.611098+0000","last_became_active":"2023-11-06T20:59:21.115401+0000","last_became_peered":"2023-11-06T20:59:21.115401+0000","last_unstale":"2023-11-06T20:59:21.611098+0000","last_undegraded":"2023-11-06T20:59:21.611098+0000","last_fullsized":"2023-11-06T20:59:21.611098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.615923+0000","last_change":"2023-11-06T20:59:21.607343+0000","last_active":"2023-11-06T20:59:21.615923+0000","last_peered":"2023-11-06T20:59:21.615923+0000","last_clean":"2023-11-06T20:59:21.615923+0000","last_became_active":"2023-11-06T20:59:21.607069+0000","last_became_peered":"2023-11-06T20:59:21.607069+0000","last_unstale":"2023-11-06T20:59:21.615923+0000","last_undegraded":"2023-11-06T20:59:21.615923+0000","last_fullsized":"2023-11-06T20:59:21.615923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584236+0000","last_change":"2023-11-06T20:59:21.120975+0000","last_active":"2023-11-06T20:59:21.584236+0000","last_peered":"2023-11-06T20:59:21.584236+0000","last_clean":"2023-11-06T20:59:21.584236+0000","last_became_active":"2023-11-06T20:59:21.120509+0000","last_became_peered":"2023-11-06T20:59:21.120509+0000","last_unstale":"2023-11-06T20:59:21.584236+0000","last_undegraded":"2023-11-06T20:59:21.584236+0000","last_fullsized":"2023-11-06T20:59:21.584236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.601545+0000","last_change":"2023-11-06T20:59:21.593390+0000","last_active":"2023-11-06T20:59:21.601545+0000","last_peered":"2023-11-06T20:59:21.601545+0000","last_clean":"2023-11-06T20:59:21.601545+0000","last_became_active":"2023-11-06T20:59:21.593070+0000","last_became_peered":"2023-11-06T20:59:21.593070+0000","last_unstale":"2023-11-06T20:59:21.601545+0000","last_undegraded":"2023-11-06T20:59:21.601545+0000","last_fullsized":"2023-11-06T20:59:21.601545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.587658+0000","last_change":"2023-11-06T20:59:21.118501+0000","last_active":"2023-11-06T20:59:21.587658+0000","last_peered":"2023-11-06T20:59:21.587658+0000","last_clean":"2023-11-06T20:59:21.587658+0000","last_became_active":"2023-11-06T20:59:21.118180+0000","last_became_peered":"2023-11-06T20:59:21.118180+0000","last_unstale":"2023-11-06T20:59:21.587658+0000","last_undegraded":"2023-11-06T20:59:21.587658+0000","last_fullsized":"2023-11-06T20:59:21.587658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621742+0000","last_change":"2023-11-06T20:59:21.617777+0000","last_active":"2023-11-06T20:59:21.621742+0000","last_peered":"2023-11-06T20:59:21.621742+0000","last_clean":"2023-11-06T20:59:21.621742+0000","last_became_active":"2023-11-06T20:59:21.617463+0000","last_became_peered":"2023-11-06T20:59:21.617463+0000","last_unstale":"2023-11-06T20:59:21.621742+0000","last_undegraded":"2023-11-06T20:59:21.621742+0000","last_fullsized":"2023-11-06T20:59:21.621742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584976+0000","last_change":"2023-11-06T20:59:21.109519+0000","last_active":"2023-11-06T20:59:21.584976+0000","last_peered":"2023-11-06T20:59:21.584976+0000","last_clean":"2023-11-06T20:59:21.584976+0000","last_became_active":"2023-11-06T20:59:21.109206+0000","last_became_peered":"2023-11-06T20:59:21.109206+0000","last_unstale":"2023-11-06T20:59:21.584976+0000","last_undegraded":"2023-11-06T20:59:21.584976+0000","last_fullsized":"2023-11-06T20:59:21.584976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.618439+0000","last_change":"2023-11-06T20:59:21.613001+0000","last_active":"2023-11-06T20:59:21.618439+0000","last_peered":"2023-11-06T20:59:21.618439+0000","last_clean":"2023-11-06T20:59:21.618439+0000","last_became_active":"2023-11-06T20:59:21.612714+0000","last_became_peered":"2023-11-06T20:59:21.612714+0000","last_unstale":"2023-11-06T20:59:21.618439+0000","last_undegraded":"2023-11-06T20:59:21.618439+0000","last_fullsized":"2023-11-06T20:59:21.618439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604171+0000","last_change":"2023-11-06T20:59:21.124482+0000","last_active":"2023-11-06T20:59:21.604171+0000","last_peered":"2023-11-06T20:59:21.604171+0000","last_clean":"2023-11-06T20:59:21.604171+0000","last_became_active":"2023-11-06T20:59:21.124232+0000","last_became_peered":"2023-11-06T20:59:21.124232+0000","last_unstale":"2023-11-06T20:59:21.604171+0000","last_undegraded":"2023-11-06T20:59:21.604171+0000","last_fullsized":"2023-11-06T20:59:21.604171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621465+0000","last_change":"2023-11-06T20:59:21.618203+0000","last_active":"2023-11-06T20:59:21.621465+0000","last_peered":"2023-11-06T20:59:21.621465+0000","last_clean":"2023-11-06T20:59:21.621465+0000","last_became_active":"2023-11-06T20:59:21.617930+0000","last_became_peered":"2023-11-06T20:59:21.617930+0000","last_unstale":"2023-11-06T20:59:21.621465+0000","last_undegraded":"2023-11-06T20:59:21.621465+0000","last_fullsized":"2023-11-06T20:59:21.621465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.602600+0000","last_change":"2023-11-06T20:59:21.114428+0000","last_active":"2023-11-06T20:59:21.602600+0000","last_peered":"2023-11-06T20:59:21.602600+0000","last_clean":"2023-11-06T20:59:21.602600+0000","last_became_active":"2023-11-06T20:59:21.111999+0000","last_became_peered":"2023-11-06T20:59:21.111999+0000","last_unstale":"2023-11-06T20:59:21.602600+0000","last_undegraded":"2023-11-06T20:59:21.602600+0000","last_fullsized":"2023-11-06T20:59:21.602600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611357+0000","last_change":"2023-11-06T20:59:21.606157+0000","last_active":"2023-11-06T20:59:21.611357+0000","last_peered":"2023-11-06T20:59:21.611357+0000","last_clean":"2023-11-06T20:59:21.611357+0000","last_became_active":"2023-11-06T20:59:21.604821+0000","last_became_peered":"2023-11-06T20:59:21.604821+0000","last_unstale":"2023-11-06T20:59:21.611357+0000","last_undegraded":"2023-11-06T20:59:21.611357+0000","last_fullsized":"2023-11-06T20:59:21.611357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.608913+0000","last_change":"2023-11-06T20:59:21.118890+0000","last_active":"2023-11-06T20:59:21.608913+0000","last_peered":"2023-11-06T20:59:21.608913+0000","last_clean":"2023-11-06T20:59:21.608913+0000","last_became_active":"2023-11-06T20:59:21.118413+0000","last_became_peered":"2023-11-06T20:59:21.118413+0000","last_unstale":"2023-11-06T20:59:21.608913+0000","last_undegraded":"2023-11-06T20:59:21.608913+0000","last_fullsized":"2023-11-06T20:59:21.608913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.602621+0000","last_change":"2023-11-06T20:59:21.596731+0000","last_active":"2023-11-06T20:59:21.602621+0000","last_peered":"2023-11-06T20:59:21.602621+0000","last_clean":"2023-11-06T20:59:21.602621+0000","last_became_active":"2023-11-06T20:59:21.596453+0000","last_became_peered":"2023-11-06T20:59:21.596453+0000","last_unstale":"2023-11-06T20:59:21.602621+0000","last_undegraded":"2023-11-06T20:59:21.602621+0000","last_fullsized":"2023-11-06T20:59:21.602621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.592632+0000","last_change":"2023-11-06T20:59:21.114744+0000","last_active":"2023-11-06T20:59:21.592632+0000","last_peered":"2023-11-06T20:59:21.592632+0000","last_clean":"2023-11-06T20:59:21.592632+0000","last_became_active":"2023-11-06T20:59:21.114559+0000","last_became_peered":"2023-11-06T20:59:21.114559+0000","last_unstale":"2023-11-06T20:59:21.592632+0000","last_undegraded":"2023-11-06T20:59:21.592632+0000","last_fullsized":"2023-11-06T20:59:21.592632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.614179+0000","last_change":"2023-11-06T20:59:21.608523+0000","last_active":"2023-11-06T20:59:21.614179+0000","last_peered":"2023-11-06T20:59:21.614179+0000","last_clean":"2023-11-06T20:59:21.614179+0000","last_became_active":"2023-11-06T20:59:21.607864+0000","last_became_peered":"2023-11-06T20:59:21.607864+0000","last_unstale":"2023-11-06T20:59:21.614179+0000","last_undegraded":"2023-11-06T20:59:21.614179+0000","last_fullsized":"2023-11-06T20:59:21.614179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"83'3","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621525+0000","last_change":"2023-11-06T20:59:21.108826+0000","last_active":"2023-11-06T20:59:21.621525+0000","last_peered":"2023-11-06T20:59:21.621525+0000","last_clean":"2023-11-06T20:59:21.621525+0000","last_became_active":"2023-11-06T20:59:21.108513+0000","last_became_peered":"2023-11-06T20:59:21.108513+0000","last_unstale":"2023-11-06T20:59:21.621525+0000","last_undegraded":"2023-11-06T20:59:21.621525+0000","last_fullsized":"2023-11-06T20:59:21.621525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604152+0000","last_change":"2023-11-06T20:59:21.594304+0000","last_active":"2023-11-06T20:59:21.604152+0000","last_peered":"2023-11-06T20:59:21.604152+0000","last_clean":"2023-11-06T20:59:21.604152+0000","last_became_active":"2023-11-06T20:59:21.594021+0000","last_became_peered":"2023-11-06T20:59:21.594021+0000","last_unstale":"2023-11-06T20:59:21.604152+0000","last_undegraded":"2023-11-06T20:59:21.604152+0000","last_fullsized":"2023-11-06T20:59:21.604152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611361+0000","last_change":"2023-11-06T20:59:21.120768+0000","last_active":"2023-11-06T20:59:21.611361+0000","last_peered":"2023-11-06T20:59:21.611361+0000","last_clean":"2023-11-06T20:59:21.611361+0000","last_became_active":"2023-11-06T20:59:21.120168+0000","last_became_peered":"2023-11-06T20:59:21.120168+0000","last_unstale":"2023-11-06T20:59:21.611361+0000","last_undegraded":"2023-11-06T20:59:21.611361+0000","last_fullsized":"2023-11-06T20:59:21.611361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.618497+0000","last_change":"2023-11-06T20:59:21.612988+0000","last_active":"2023-11-06T20:59:21.618497+0000","last_peered":"2023-11-06T20:59:21.618497+0000","last_clean":"2023-11-06T20:59:21.618497+0000","last_became_active":"2023-11-06T20:59:21.612698+0000","last_became_peered":"2023-11-06T20:59:21.612698+0000","last_unstale":"2023-11-06T20:59:21.618497+0000","last_undegraded":"2023-11-06T20:59:21.618497+0000","last_fullsized":"2023-11-06T20:59:21.618497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.622607+0000","last_change":"2023-11-06T20:59:21.119338+0000","last_active":"2023-11-06T20:59:21.622607+0000","last_peered":"2023-11-06T20:59:21.622607+0000","last_clean":"2023-11-06T20:59:21.622607+0000","last_became_active":"2023-11-06T20:59:21.119064+0000","last_became_peered":"2023-11-06T20:59:21.119064+0000","last_unstale":"2023-11-06T20:59:21.622607+0000","last_undegraded":"2023-11-06T20:59:21.622607+0000","last_fullsized":"2023-11-06T20:59:21.622607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604312+0000","last_change":"2023-11-06T20:59:21.601318+0000","last_active":"2023-11-06T20:59:21.604312+0000","last_peered":"2023-11-06T20:59:21.604312+0000","last_clean":"2023-11-06T20:59:21.604312+0000","last_became_active":"2023-11-06T20:59:21.600901+0000","last_became_peered":"2023-11-06T20:59:21.600901+0000","last_unstale":"2023-11-06T20:59:21.604312+0000","last_undegraded":"2023-11-06T20:59:21.604312+0000","last_fullsized":"2023-11-06T20:59:21.604312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.587535+0000","last_change":"2023-11-06T20:59:21.117068+0000","last_active":"2023-11-06T20:59:21.587535+0000","last_peered":"2023-11-06T20:59:21.587535+0000","last_clean":"2023-11-06T20:59:21.587535+0000","last_became_active":"2023-11-06T20:59:21.116791+0000","last_became_peered":"2023-11-06T20:59:21.116791+0000","last_unstale":"2023-11-06T20:59:21.587535+0000","last_undegraded":"2023-11-06T20:59:21.587535+0000","last_fullsized":"2023-11-06T20:59:21.587535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.599529+0000","last_change":"2023-11-06T20:59:21.597767+0000","last_active":"2023-11-06T20:59:21.599529+0000","last_peered":"2023-11-06T20:59:21.599529+0000","last_clean":"2023-11-06T20:59:21.599529+0000","last_became_active":"2023-11-06T20:59:21.597469+0000","last_became_peered":"2023-11-06T20:59:21.597469+0000","last_unstale":"2023-11-06T20:59:21.599529+0000","last_undegraded":"2023-11-06T20:59:21.599529+0000","last_fullsized":"2023-11-06T20:59:21.599529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.582062+0000","last_change":"2023-11-06T20:59:21.115393+0000","last_active":"2023-11-06T20:59:21.582062+0000","last_peered":"2023-11-06T20:59:21.582062+0000","last_clean":"2023-11-06T20:59:21.582062+0000","last_became_active":"2023-11-06T20:59:21.114673+0000","last_became_peered":"2023-11-06T20:59:21.114673+0000","last_unstale":"2023-11-06T20:59:21.582062+0000","last_undegraded":"2023-11-06T20:59:21.582062+0000","last_fullsized":"2023-11-06T20:59:21.582062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.611242+0000","last_change":"2023-11-06T20:59:21.603590+0000","last_active":"2023-11-06T20:59:21.611242+0000","last_peered":"2023-11-06T20:59:21.611242+0000","last_clean":"2023-11-06T20:59:21.611242+0000","last_became_active":"2023-11-06T20:59:21.603293+0000","last_became_peered":"2023-11-06T20:59:21.603293+0000","last_unstale":"2023-11-06T20:59:21.611242+0000","last_undegraded":"2023-11-06T20:59:21.611242+0000","last_fullsized":"2023-11-06T20:59:21.611242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"83'4","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.603856+0000","last_change":"2023-11-06T20:59:21.116727+0000","last_active":"2023-11-06T20:59:21.603856+0000","last_peered":"2023-11-06T20:59:21.603856+0000","last_clean":"2023-11-06T20:59:21.603856+0000","last_became_active":"2023-11-06T20:59:21.116446+0000","last_became_peered":"2023-11-06T20:59:21.116446+0000","last_unstale":"2023-11-06T20:59:21.603856+0000","last_undegraded":"2023-11-06T20:59:21.603856+0000","last_fullsized":"2023-11-06T20:59:21.603856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.602585+0000","last_change":"2023-11-06T20:59:21.598027+0000","last_active":"2023-11-06T20:59:21.602585+0000","last_peered":"2023-11-06T20:59:21.602585+0000","last_clean":"2023-11-06T20:59:21.602585+0000","last_became_active":"2023-11-06T20:59:21.597721+0000","last_became_peered":"2023-11-06T20:59:21.597721+0000","last_unstale":"2023-11-06T20:59:21.602585+0000","last_undegraded":"2023-11-06T20:59:21.602585+0000","last_fullsized":"2023-11-06T20:59:21.602585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.622855+0000","last_change":"2023-11-06T20:59:21.114930+0000","last_active":"2023-11-06T20:59:21.622855+0000","last_peered":"2023-11-06T20:59:21.622855+0000","last_clean":"2023-11-06T20:59:21.622855+0000","last_became_active":"2023-11-06T20:59:21.112906+0000","last_became_peered":"2023-11-06T20:59:21.112906+0000","last_unstale":"2023-11-06T20:59:21.622855+0000","last_undegraded":"2023-11-06T20:59:21.622855+0000","last_fullsized":"2023-11-06T20:59:21.622855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.613374+0000","last_change":"2023-11-06T20:59:21.609413+0000","last_active":"2023-11-06T20:59:21.613374+0000","last_peered":"2023-11-06T20:59:21.613374+0000","last_clean":"2023-11-06T20:59:21.613374+0000","last_became_active":"2023-11-06T20:59:21.609011+0000","last_became_peered":"2023-11-06T20:59:21.609011+0000","last_unstale":"2023-11-06T20:59:21.613374+0000","last_undegraded":"2023-11-06T20:59:21.613374+0000","last_fullsized":"2023-11-06T20:59:21.613374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.607136+0000","last_change":"2023-11-06T20:59:21.117747+0000","last_active":"2023-11-06T20:59:21.607136+0000","last_peered":"2023-11-06T20:59:21.607136+0000","last_clean":"2023-11-06T20:59:21.607136+0000","last_became_active":"2023-11-06T20:59:21.117001+0000","last_became_peered":"2023-11-06T20:59:21.117001+0000","last_unstale":"2023-11-06T20:59:21.607136+0000","last_undegraded":"2023-11-06T20:59:21.607136+0000","last_fullsized":"2023-11-06T20:59:21.607136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.595159+0000","last_change":"2023-11-06T20:59:21.590211+0000","last_active":"2023-11-06T20:59:21.595159+0000","last_peered":"2023-11-06T20:59:21.595159+0000","last_clean":"2023-11-06T20:59:21.595159+0000","last_became_active":"2023-11-06T20:59:21.589929+0000","last_became_peered":"2023-11-06T20:59:21.589929+0000","last_unstale":"2023-11-06T20:59:21.595159+0000","last_undegraded":"2023-11-06T20:59:21.595159+0000","last_fullsized":"2023-11-06T20:59:21.595159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584751+0000","last_change":"2023-11-06T20:59:21.117733+0000","last_active":"2023-11-06T20:59:21.584751+0000","last_peered":"2023-11-06T20:59:21.584751+0000","last_clean":"2023-11-06T20:59:21.584751+0000","last_became_active":"2023-11-06T20:59:21.117461+0000","last_became_peered":"2023-11-06T20:59:21.117461+0000","last_unstale":"2023-11-06T20:59:21.584751+0000","last_undegraded":"2023-11-06T20:59:21.584751+0000","last_fullsized":"2023-11-06T20:59:21.584751+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604116+0000","last_change":"2023-11-06T20:59:21.591047+0000","last_active":"2023-11-06T20:59:21.604116+0000","last_peered":"2023-11-06T20:59:21.604116+0000","last_clean":"2023-11-06T20:59:21.604116+0000","last_became_active":"2023-11-06T20:59:21.590754+0000","last_became_peered":"2023-11-06T20:59:21.590754+0000","last_unstale":"2023-11-06T20:59:21.604116+0000","last_undegraded":"2023-11-06T20:59:21.604116+0000","last_fullsized":"2023-11-06T20:59:21.604116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.585434+0000","last_change":"2023-11-06T20:59:21.110735+0000","last_active":"2023-11-06T20:59:21.585434+0000","last_peered":"2023-11-06T20:59:21.585434+0000","last_clean":"2023-11-06T20:59:21.585434+0000","last_became_active":"2023-11-06T20:59:21.110394+0000","last_became_peered":"2023-11-06T20:59:21.110394+0000","last_unstale":"2023-11-06T20:59:21.585434+0000","last_undegraded":"2023-11-06T20:59:21.585434+0000","last_fullsized":"2023-11-06T20:59:21.585434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.614226+0000","last_change":"2023-11-06T20:59:21.602954+0000","last_active":"2023-11-06T20:59:21.614226+0000","last_peered":"2023-11-06T20:59:21.614226+0000","last_clean":"2023-11-06T20:59:21.614226+0000","last_became_active":"2023-11-06T20:59:21.602721+0000","last_became_peered":"2023-11-06T20:59:21.602721+0000","last_unstale":"2023-11-06T20:59:21.614226+0000","last_undegraded":"2023-11-06T20:59:21.614226+0000","last_fullsized":"2023-11-06T20:59:21.614226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.594139+0000","last_change":"2023-11-06T20:59:21.114982+0000","last_active":"2023-11-06T20:59:21.594139+0000","last_peered":"2023-11-06T20:59:21.594139+0000","last_clean":"2023-11-06T20:59:21.594139+0000","last_became_active":"2023-11-06T20:59:21.114638+0000","last_became_peered":"2023-11-06T20:59:21.114638+0000","last_unstale":"2023-11-06T20:59:21.594139+0000","last_undegraded":"2023-11-06T20:59:21.594139+0000","last_fullsized":"2023-11-06T20:59:21.594139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.620982+0000","last_change":"2023-11-06T20:59:21.612049+0000","last_active":"2023-11-06T20:59:21.620982+0000","last_peered":"2023-11-06T20:59:21.620982+0000","last_clean":"2023-11-06T20:59:21.620982+0000","last_became_active":"2023-11-06T20:59:21.611703+0000","last_became_peered":"2023-11-06T20:59:21.611703+0000","last_unstale":"2023-11-06T20:59:21.620982+0000","last_undegraded":"2023-11-06T20:59:21.620982+0000","last_fullsized":"2023-11-06T20:59:21.620982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.594843+0000","last_change":"2023-11-06T20:59:21.119877+0000","last_active":"2023-11-06T20:59:21.594843+0000","last_peered":"2023-11-06T20:59:21.594843+0000","last_clean":"2023-11-06T20:59:21.594843+0000","last_became_active":"2023-11-06T20:59:21.119552+0000","last_became_peered":"2023-11-06T20:59:21.119552+0000","last_unstale":"2023-11-06T20:59:21.594843+0000","last_undegraded":"2023-11-06T20:59:21.594843+0000","last_fullsized":"2023-11-06T20:59:21.594843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.615976+0000","last_change":"2023-11-06T20:59:21.612277+0000","last_active":"2023-11-06T20:59:21.615976+0000","last_peered":"2023-11-06T20:59:21.615976+0000","last_clean":"2023-11-06T20:59:21.615976+0000","last_became_active":"2023-11-06T20:59:21.611916+0000","last_became_peered":"2023-11-06T20:59:21.611916+0000","last_unstale":"2023-11-06T20:59:21.615976+0000","last_undegraded":"2023-11-06T20:59:21.615976+0000","last_fullsized":"2023-11-06T20:59:21.615976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.586737+0000","last_change":"2023-11-06T20:59:21.114926+0000","last_active":"2023-11-06T20:59:21.586737+0000","last_peered":"2023-11-06T20:59:21.586737+0000","last_clean":"2023-11-06T20:59:21.586737+0000","last_became_active":"2023-11-06T20:59:21.114589+0000","last_became_peered":"2023-11-06T20:59:21.114589+0000","last_unstale":"2023-11-06T20:59:21.586737+0000","last_undegraded":"2023-11-06T20:59:21.586737+0000","last_fullsized":"2023-11-06T20:59:21.586737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.618704+0000","last_change":"2023-11-06T20:59:21.614370+0000","last_active":"2023-11-06T20:59:21.618704+0000","last_peered":"2023-11-06T20:59:21.618704+0000","last_clean":"2023-11-06T20:59:21.618704+0000","last_became_active":"2023-11-06T20:59:21.614026+0000","last_became_peered":"2023-11-06T20:59:21.614026+0000","last_unstale":"2023-11-06T20:59:21.618704+0000","last_undegraded":"2023-11-06T20:59:21.618704+0000","last_fullsized":"2023-11-06T20:59:21.618704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.583550+0000","last_change":"2023-11-06T20:59:21.116311+0000","last_active":"2023-11-06T20:59:21.583550+0000","last_peered":"2023-11-06T20:59:21.583550+0000","last_clean":"2023-11-06T20:59:21.583550+0000","last_became_active":"2023-11-06T20:59:21.116024+0000","last_became_peered":"2023-11-06T20:59:21.116024+0000","last_unstale":"2023-11-06T20:59:21.583550+0000","last_undegraded":"2023-11-06T20:59:21.583550+0000","last_fullsized":"2023-11-06T20:59:21.583550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.614190+0000","last_change":"2023-11-06T20:59:21.602496+0000","last_active":"2023-11-06T20:59:21.614190+0000","last_peered":"2023-11-06T20:59:21.614190+0000","last_clean":"2023-11-06T20:59:21.614190+0000","last_became_active":"2023-11-06T20:59:21.602153+0000","last_became_peered":"2023-11-06T20:59:21.602153+0000","last_unstale":"2023-11-06T20:59:21.614190+0000","last_undegraded":"2023-11-06T20:59:21.614190+0000","last_fullsized":"2023-11-06T20:59:21.614190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.621817+0000","last_change":"2023-11-06T20:59:21.115432+0000","last_active":"2023-11-06T20:59:21.621817+0000","last_peered":"2023-11-06T20:59:21.621817+0000","last_clean":"2023-11-06T20:59:21.621817+0000","last_became_active":"2023-11-06T20:59:21.115012+0000","last_became_peered":"2023-11-06T20:59:21.115012+0000","last_unstale":"2023-11-06T20:59:21.621817+0000","last_undegraded":"2023-11-06T20:59:21.621817+0000","last_fullsized":"2023-11-06T20:59:21.621817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584453+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:59:21.584453+0000","last_peered":"2023-11-06T20:59:21.584453+0000","last_clean":"2023-11-06T20:59:21.584453+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:59:21.584453+0000","last_undegraded":"2023-11-06T20:59:21.584453+0000","last_fullsized":"2023-11-06T20:59:21.584453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.616688+0000","last_change":"2023-11-06T20:59:21.606374+0000","last_active":"2023-11-06T20:59:21.616688+0000","last_peered":"2023-11-06T20:59:21.616688+0000","last_clean":"2023-11-06T20:59:21.616688+0000","last_became_active":"2023-11-06T20:59:21.605735+0000","last_became_peered":"2023-11-06T20:59:21.605735+0000","last_unstale":"2023-11-06T20:59:21.616688+0000","last_undegraded":"2023-11-06T20:59:21.616688+0000","last_fullsized":"2023-11-06T20:59:21.616688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.584001+0000","last_change":"2023-11-06T20:59:21.116737+0000","last_active":"2023-11-06T20:59:21.584001+0000","last_peered":"2023-11-06T20:59:21.584001+0000","last_clean":"2023-11-06T20:59:21.584001+0000","last_became_active":"2023-11-06T20:59:21.116467+0000","last_became_peered":"2023-11-06T20:59:21.116467+0000","last_unstale":"2023-11-06T20:59:21.584001+0000","last_undegraded":"2023-11-06T20:59:21.584001+0000","last_fullsized":"2023-11-06T20:59:21.584001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581344+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:59:21.581344+0000","last_peered":"2023-11-06T20:59:21.581344+0000","last_clean":"2023-11-06T20:59:21.581344+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:59:21.581344+0000","last_undegraded":"2023-11-06T20:59:21.581344+0000","last_fullsized":"2023-11-06T20:59:21.581344+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.610617+0000","last_change":"2023-11-06T20:59:21.606804+0000","last_active":"2023-11-06T20:59:21.610617+0000","last_peered":"2023-11-06T20:59:21.610617+0000","last_clean":"2023-11-06T20:59:21.610617+0000","last_became_active":"2023-11-06T20:59:21.606553+0000","last_became_peered":"2023-11-06T20:59:21.606553+0000","last_unstale":"2023-11-06T20:59:21.610617+0000","last_undegraded":"2023-11-06T20:59:21.610617+0000","last_fullsized":"2023-11-06T20:59:21.610617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.587625+0000","last_change":"2023-11-06T20:59:21.119277+0000","last_active":"2023-11-06T20:59:21.587625+0000","last_peered":"2023-11-06T20:59:21.587625+0000","last_clean":"2023-11-06T20:59:21.587625+0000","last_became_active":"2023-11-06T20:59:21.118613+0000","last_became_peered":"2023-11-06T20:59:21.118613+0000","last_unstale":"2023-11-06T20:59:21.587625+0000","last_undegraded":"2023-11-06T20:59:21.587625+0000","last_fullsized":"2023-11-06T20:59:21.587625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.586609+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:59:21.586609+0000","last_peered":"2023-11-06T20:59:21.586609+0000","last_clean":"2023-11-06T20:59:21.586609+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:59:21.586609+0000","last_undegraded":"2023-11-06T20:59:21.586609+0000","last_fullsized":"2023-11-06T20:59:21.586609+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.607564+0000","last_change":"2023-11-06T20:59:21.602805+0000","last_active":"2023-11-06T20:59:21.607564+0000","last_peered":"2023-11-06T20:59:21.607564+0000","last_clean":"2023-11-06T20:59:21.607564+0000","last_became_active":"2023-11-06T20:59:21.601930+0000","last_became_peered":"2023-11-06T20:59:21.601930+0000","last_unstale":"2023-11-06T20:59:21.607564+0000","last_undegraded":"2023-11-06T20:59:21.607564+0000","last_fullsized":"2023-11-06T20:59:21.607564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.583654+0000","last_change":"2023-11-06T20:59:21.113701+0000","last_active":"2023-11-06T20:59:21.583654+0000","last_peered":"2023-11-06T20:59:21.583654+0000","last_clean":"2023-11-06T20:59:21.583654+0000","last_became_active":"2023-11-06T20:59:21.112239+0000","last_became_peered":"2023-11-06T20:59:21.112239+0000","last_unstale":"2023-11-06T20:59:21.583654+0000","last_undegraded":"2023-11-06T20:59:21.583654+0000","last_fullsized":"2023-11-06T20:59:21.583654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581400+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:59:21.581400+0000","last_peered":"2023-11-06T20:59:21.581400+0000","last_clean":"2023-11-06T20:59:21.581400+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:59:21.581400+0000","last_undegraded":"2023-11-06T20:59:21.581400+0000","last_fullsized":"2023-11-06T20:59:21.581400+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.586760+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:59:21.586760+0000","last_peered":"2023-11-06T20:59:21.586760+0000","last_clean":"2023-11-06T20:59:21.586760+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:59:21.586760+0000","last_undegraded":"2023-11-06T20:59:21.586760+0000","last_fullsized":"2023-11-06T20:59:21.586760+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.602079+0000","last_change":"2023-11-06T20:59:21.600370+0000","last_active":"2023-11-06T20:59:21.602079+0000","last_peered":"2023-11-06T20:59:21.602079+0000","last_clean":"2023-11-06T20:59:21.602079+0000","last_became_active":"2023-11-06T20:59:21.600069+0000","last_became_peered":"2023-11-06T20:59:21.600069+0000","last_unstale":"2023-11-06T20:59:21.602079+0000","last_undegraded":"2023-11-06T20:59:21.602079+0000","last_fullsized":"2023-11-06T20:59:21.602079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.598397+0000","last_change":"2023-11-06T20:59:21.105339+0000","last_active":"2023-11-06T20:59:21.598397+0000","last_peered":"2023-11-06T20:59:21.598397+0000","last_clean":"2023-11-06T20:59:21.598397+0000","last_became_active":"2023-11-06T20:59:21.104789+0000","last_became_peered":"2023-11-06T20:59:21.104789+0000","last_unstale":"2023-11-06T20:59:21.598397+0000","last_undegraded":"2023-11-06T20:59:21.598397+0000","last_fullsized":"2023-11-06T20:59:21.598397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581743+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:59:21.581743+0000","last_peered":"2023-11-06T20:59:21.581743+0000","last_clean":"2023-11-06T20:59:21.581743+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:59:21.581743+0000","last_undegraded":"2023-11-06T20:59:21.581743+0000","last_fullsized":"2023-11-06T20:59:21.581743+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.616646+0000","last_change":"2023-11-06T20:59:21.608676+0000","last_active":"2023-11-06T20:59:21.616646+0000","last_peered":"2023-11-06T20:59:21.616646+0000","last_clean":"2023-11-06T20:59:21.616646+0000","last_became_active":"2023-11-06T20:59:21.605765+0000","last_became_peered":"2023-11-06T20:59:21.605765+0000","last_unstale":"2023-11-06T20:59:21.616646+0000","last_undegraded":"2023-11-06T20:59:21.616646+0000","last_fullsized":"2023-11-06T20:59:21.616646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.619310+0000","last_change":"2023-11-06T20:59:21.118110+0000","last_active":"2023-11-06T20:59:21.619310+0000","last_peered":"2023-11-06T20:59:21.619310+0000","last_clean":"2023-11-06T20:59:21.619310+0000","last_became_active":"2023-11-06T20:59:21.117866+0000","last_became_peered":"2023-11-06T20:59:21.117866+0000","last_unstale":"2023-11-06T20:59:21.619310+0000","last_undegraded":"2023-11-06T20:59:21.619310+0000","last_fullsized":"2023-11-06T20:59:21.619310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.582034+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:59:21.582034+0000","last_peered":"2023-11-06T20:59:21.582034+0000","last_clean":"2023-11-06T20:59:21.582034+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:59:21.582034+0000","last_undegraded":"2023-11-06T20:59:21.582034+0000","last_fullsized":"2023-11-06T20:59:21.582034+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604722+0000","last_change":"2023-11-06T20:59:21.599192+0000","last_active":"2023-11-06T20:59:21.604722+0000","last_peered":"2023-11-06T20:59:21.604722+0000","last_clean":"2023-11-06T20:59:21.604722+0000","last_became_active":"2023-11-06T20:59:21.598712+0000","last_became_peered":"2023-11-06T20:59:21.598712+0000","last_unstale":"2023-11-06T20:59:21.604722+0000","last_undegraded":"2023-11-06T20:59:21.604722+0000","last_fullsized":"2023-11-06T20:59:21.604722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.581937+0000","last_change":"2023-11-06T20:59:21.120879+0000","last_active":"2023-11-06T20:59:21.581937+0000","last_peered":"2023-11-06T20:59:21.581937+0000","last_clean":"2023-11-06T20:59:21.581937+0000","last_became_active":"2023-11-06T20:59:21.120566+0000","last_became_peered":"2023-11-06T20:59:21.120566+0000","last_unstale":"2023-11-06T20:59:21.581937+0000","last_undegraded":"2023-11-06T20:59:21.581937+0000","last_fullsized":"2023-11-06T20:59:21.581937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.589242+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:59:21.589242+0000","last_peered":"2023-11-06T20:59:21.589242+0000","last_clean":"2023-11-06T20:59:21.589242+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:59:21.589242+0000","last_undegraded":"2023-11-06T20:59:21.589242+0000","last_fullsized":"2023-11-06T20:59:21.589242+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.604136+0000","last_change":"2023-11-06T20:59:21.592285+0000","last_active":"2023-11-06T20:59:21.604136+0000","last_peered":"2023-11-06T20:59:21.604136+0000","last_clean":"2023-11-06T20:59:21.604136+0000","last_became_active":"2023-11-06T20:59:21.591287+0000","last_became_peered":"2023-11-06T20:59:21.591287+0000","last_unstale":"2023-11-06T20:59:21.604136+0000","last_undegraded":"2023-11-06T20:59:21.604136+0000","last_fullsized":"2023-11-06T20:59:21.604136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:21.082925+0000","last_clean_scrub_stamp":"2023-11-06T20:59:21.082925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.585247+0000","last_change":"2023-11-06T20:59:21.115796+0000","last_active":"2023-11-06T20:59:21.585247+0000","last_peered":"2023-11-06T20:59:21.585247+0000","last_clean":"2023-11-06T20:59:21.585247+0000","last_became_active":"2023-11-06T20:59:21.115493+0000","last_became_peered":"2023-11-06T20:59:21.115493+0000","last_unstale":"2023-11-06T20:59:21.585247+0000","last_undegraded":"2023-11-06T20:59:21.585247+0000","last_fullsized":"2023-11-06T20:59:21.585247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:20.073854+0000","last_clean_scrub_stamp":"2023-11-06T20:59:20.073854+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-06T20:59:21.582175+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:59:21.582175+0000","last_peered":"2023-11-06T20:59:21.582175+0000","last_clean":"2023-11-06T20:59:21.582175+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:59:21.582175+0000","last_undegraded":"2023-11-06T20:59:21.582175+0000","last_fullsized":"2023-11-06T20:59:21.582175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575033,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":1019904,"data_stored":508915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.61099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.95599999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.878,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.69099999999999995}]},{"osd":1,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69099999999999995,"15min":0.71999999999999997},"min":{"1min":0.47899999999999998,"5min":0.439,"15min":0.40500000000000003},"max":{"1min":0.89400000000000002,"5min":0.98599999999999999,"15min":5.1749999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71699999999999997},"min":{"1min":0.57299999999999995,"5min":0.43099999999999999,"15min":0.41399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.133},"last":0.621}]},{"osd":2,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72999999999999998,"15min":0.72799999999999998},"min":{"1min":0.59699999999999998,"5min":0.499,"15min":0.47799999999999998},"max":{"1min":0.96199999999999997,"5min":0.996,"15min":1.165},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.55400000000000005,"5min":0.442,"15min":0.442},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":1.024},"last":0.55300000000000005}]},{"osd":3,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69999999999999996,"15min":0.73299999999999998},"min":{"1min":0.55500000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.872,"5min":1.0129999999999999,"15min":5.202},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.49399999999999999},"max":{"1min":0.91000000000000003,"5min":0.95699999999999996,"15min":5.165},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":1.083},"last":0.497},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.48499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79700000000000004,"5min":0.86799999999999999,"15min":0.92800000000000005},"last":0.63900000000000001}]},{"osd":5,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63,"15min":0.64600000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41099999999999998},"max":{"1min":0.80600000000000005,"5min":0.93300000000000005,"15min":1.395},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.42699999999999999},"max":{"1min":0.84099999999999997,"5min":0.86199999999999999,"15min":1.129},"last":0.60399999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70899999999999996,"15min":0.72599999999999998},"min":{"1min":0.57499999999999996,"5min":0.505,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":0.94499999999999995,"15min":1.24},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.443},"max":{"1min":0.876,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.67800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575033,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302196,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069644,"statfs":{"total":96636764160,"available":96327315456,"internally_reserved":0,"allocated":1028096,"data_stored":510043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68400000000000005,"15min":0.67700000000000005},"min":{"1min":0.48199999999999998,"5min":0.33000000000000002,"15min":0.32800000000000001},"max":{"1min":0.92900000000000005,"5min":0.93999999999999995,"15min":1.0720000000000001},"last":0.749},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69699999999999995,"15min":0.67900000000000005},"min":{"1min":0.52100000000000002,"5min":0.47599999999999998,"15min":0.38800000000000001},"max":{"1min":0.88500000000000001,"5min":1.216,"15min":1.216},"last":0.68200000000000005}]},{"osd":1,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.60499999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.877,"5min":0.97599999999999998,"15min":1.3799999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.54800000000000004,"5min":0.41899999999999998,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.98599999999999999,"15min":1.1659999999999999},"last":0.69599999999999995}]},{"osd":2,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.76700000000000002,"15min":0.752},"min":{"1min":0.63,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.321},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.75700000000000001,"15min":0.76000000000000001},"min":{"1min":0.65000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95099999999999996,"5min":1.05,"15min":1.405},"last":0.85599999999999998}]},{"osd":3,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.59499999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.1459999999999999,"15min":1.5109999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.72199999999999998},"min":{"1min":0.57999999999999996,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":0.95999999999999996,"5min":1.002,"15min":1.002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.49099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.92200000000000004,"15min":1.018},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63200000000000001,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.40699999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":2.0310000000000001},"last":0.50600000000000001}]},{"osd":5,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90700000000000003,"5min":1.028,"15min":1.028},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.441,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85399999999999998,"5min":1.02,"15min":1.133},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53300000000000003,"5min":0.40799999999999997,"15min":0.378},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.0940000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.53900000000000003,"5min":0.47699999999999998,"15min":0.46899999999999997},"max":{"1min":0.97099999999999997,"5min":0.98399999999999999,"15min":1.4319999999999999},"last":0.83799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575033,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302180,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069660,"statfs":{"total":96636764160,"available":96327331840,"internally_reserved":0,"allocated":1011712,"data_stored":503067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.44900000000000001},"min":{"1min":0.34499999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77900000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.47799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45600000000000002}]},{"osd":2,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.75900000000000001,"5min":0.80800000000000005,"15min":0.86599999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.45900000000000002,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56999999999999995,"15min":0.55300000000000005},"min":{"1min":0.48199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73299999999999998,"5min":0.94099999999999995,"15min":1.29},"last":0.41099999999999998}]},{"osd":4,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.61499999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.171},"last":0.628},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82199999999999995,"15min":0.81899999999999995},"min":{"1min":0.76000000000000001,"5min":0.58199999999999996,"15min":0.57899999999999996},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.1519999999999999},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.64600000000000002,"5min":0.57599999999999996,"15min":0.54900000000000004},"max":{"1min":1.042,"5min":1.042,"15min":1.329},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77100000000000002,"15min":0.78100000000000003},"min":{"1min":0.628,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.97599999999999998,"5min":1.028,"15min":1.1160000000000001},"last":0.58199999999999996}]},{"osd":6,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.501},"max":{"1min":0.98599999999999999,"5min":1.0880000000000001,"15min":1.135},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.73199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.3540000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.64300000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.077,"15min":1.405},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82199999999999995},"min":{"1min":0.66400000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0329999999999999,"5min":1.105,"15min":1.1950000000000001},"last":0.66500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575033,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":1019904,"data_stored":509737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.45700000000000002,"15min":0.46000000000000002},"min":{"1min":0.32100000000000001,"5min":0.308,"15min":0.29299999999999998},"max":{"1min":0.54700000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.503,"15min":0.501},"min":{"1min":0.42999999999999999,"5min":0.33200000000000002,"15min":0.311},"max":{"1min":0.61799999999999999,"5min":0.73299999999999998,"15min":0.97399999999999998},"last":0.41199999999999998}]},{"osd":2,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46200000000000002,"15min":0.46700000000000003},"min":{"1min":0.35499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.61699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63100000000000001,"5min":0.77500000000000002,"15min":0.91000000000000003},"last":0.45000000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.35999999999999999},"max":{"1min":0.71799999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.46800000000000003,"5min":0.39700000000000002,"15min":0.38300000000000001},"max":{"1min":0.628,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.57999999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97699999999999998,"5min":1.1279999999999999,"15min":1.3779999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79400000000000004,"15min":0.80800000000000005},"min":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.53600000000000003},"max":{"1min":0.89600000000000002,"5min":1.298,"15min":1.8380000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78900000000000003,"15min":0.80600000000000005},"min":{"1min":0.64000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.97199999999999998,"5min":1.1120000000000001,"15min":1.3169999999999999},"last":1.069},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81499999999999995,"15min":0.82699999999999996},"min":{"1min":0.60899999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96299999999999997,"5min":1.2549999999999999,"15min":1.915},"last":1.006}]},{"osd":6,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78100000000000003,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.56499999999999995,"15min":0.54600000000000004},"max":{"1min":0.96399999999999997,"5min":1.048,"15min":1.1120000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.622,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97899999999999998,"5min":1.016,"15min":1.5860000000000001},"last":1.0309999999999999}]},{"osd":7,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.67600000000000005,"5min":0.60499999999999998,"15min":0.57199999999999995},"max":{"1min":1.01,"5min":1.0820000000000001,"15min":1.2490000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81299999999999994,"15min":0.83399999999999996},"min":{"1min":0.62,"5min":0.54600000000000004,"15min":0.53200000000000003},"max":{"1min":1.026,"5min":1.089,"15min":1.6359999999999999},"last":0.98799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575033,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302192,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069648,"statfs":{"total":96636764160,"available":96327319552,"internally_reserved":0,"allocated":1024000,"data_stored":508901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.44800000000000001,"15min":0.44500000000000001},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.25900000000000001},"max":{"1min":0.70299999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.496,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.72699999999999998,"5min":0.73099999999999998,"15min":1.0269999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.34599999999999997,"15min":0.30099999999999999},"max":{"1min":0.86099999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.5},{"interface":"front","average":{"1min":0.622,"5min":0.60999999999999999,"15min":0.60299999999999998},"min":{"1min":0.499,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.96399999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.54300000000000004}]},{"osd":3,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.505},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.33400000000000002},"max":{"1min":0.65700000000000003,"5min":2.266,"15min":2.266},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.46800000000000003,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":1.732,"15min":2.242},"last":0.47099999999999997}]},{"osd":4,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.371},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.51400000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.357},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.45900000000000002},"max":{"1min":1.0209999999999999,"5min":1.657,"15min":1.657},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80800000000000005,"15min":0.80200000000000005},"min":{"1min":0.67900000000000005,"5min":0.58599999999999997,"15min":0.498},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.4099999999999999},"last":0.64000000000000001}]},{"osd":6,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.80300000000000005,"15min":0.80200000000000005},"min":{"1min":0.56000000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.4239999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.54900000000000004,"5min":0.54100000000000004,"15min":0.52300000000000002},"max":{"1min":1.075,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Mon Nov 6 20:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.65000000000000002,"5min":0.52800000000000002,"15min":0.46600000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.534},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83299999999999996,"15min":0.82399999999999995},"min":{"1min":0.66100000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.129,"5min":2.202,"15min":2.202},"last":0.626}]}]},{"osd":3,"up_from":13,"seq":55834575034,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302188,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069652,"statfs":{"total":96636764160,"available":96327323648,"internally_reserved":0,"allocated":1019904,"data_stored":508935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.45600000000000002},"min":{"1min":0.36199999999999999,"5min":0.28399999999999997,"15min":0.26800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":1.046},"last":0.372},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.376,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.51100000000000001}]},{"osd":1,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48599999999999999,"15min":0.48299999999999998},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.313},"max":{"1min":0.63900000000000001,"5min":0.64800000000000002,"15min":0.753},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.34499999999999997},"max":{"1min":0.69599999999999995,"5min":0.80900000000000005,"15min":1.177},"last":0.41999999999999998}]},{"osd":2,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.48199999999999998,"5min":0.379,"15min":0.35299999999999998},"max":{"1min":0.70899999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.45300000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":0.88100000000000001,"15min":1.3120000000000001},"last":0.501}]},{"osd":4,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.81799999999999995,"15min":0.80000000000000004},"min":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.51200000000000001},"max":{"1min":1.1080000000000001,"5min":1.212,"15min":1.2470000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.81699999999999995,"15min":0.79800000000000004},"min":{"1min":0.68000000000000005,"5min":0.55100000000000005,"15min":0.47999999999999998},"max":{"1min":1.1379999999999999,"5min":1.1819999999999999,"15min":1.286},"last":0.68799999999999994}]},{"osd":5,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77900000000000003,"15min":0.77500000000000002},"min":{"1min":0.59199999999999997,"5min":0.496,"15min":0.496},"max":{"1min":0.94799999999999995,"5min":1.1779999999999999,"15min":1.228},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.59599999999999997,"5min":0.54600000000000004,"15min":0.49199999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.1439999999999999},"last":0.621}]},{"osd":6,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79400000000000004,"15min":0.77400000000000002},"min":{"1min":0.61899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96799999999999997,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.80400000000000005},"min":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.97499999999999998,"5min":1.202,"15min":1.268},"last":0.81799999999999995}]},{"osd":7,"last update":"Mon Nov 6 20:58:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81200000000000006,"15min":0.80300000000000005},"min":{"1min":0.62,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0569999999999999,"5min":1.302,"15min":1.302},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.82199999999999995,"15min":0.81100000000000005},"min":{"1min":0.63800000000000001,"5min":0.56699999999999995,"15min":0.47899999999999998},"max":{"1min":1.0509999999999999,"5min":1.077,"15min":1.1739999999999999},"last":0.79400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302172,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069668,"statfs":{"total":96636764160,"available":96327340032,"internally_reserved":0,"allocated":1003520,"data_stored":502141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.498},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.60599999999999998}]},{"osd":1,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.495,"5min":0.44400000000000001,"15min":0.41399999999999998},"max":{"1min":1.107,"5min":1.645,"15min":1.645},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.58099999999999996,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":0.92000000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.72299999999999998,"5min":0.52900000000000003,"15min":0.51900000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.72299999999999998}]},{"osd":3,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.69199999999999995},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80000000000000004,"5min":0.95799999999999996,"15min":1.0169999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69499999999999995,"15min":0.69199999999999995},"min":{"1min":0.49099999999999999,"5min":0.46600000000000003,"15min":0.42799999999999999},"max":{"1min":0.93200000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.58899999999999997}]},{"osd":5,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59499999999999997,"15min":0.61199999999999999},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":1.0389999999999999,"15min":1.0880000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.45200000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.3520000000000001},"last":0.47299999999999998}]},{"osd":6,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.50600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84399999999999997,"5min":1.22,"15min":1.22},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67100000000000004,"15min":0.68400000000000005},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.44},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.3999999999999999},"last":0.75}]},{"osd":7,"last update":"Mon Nov 6 20:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.46300000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.034,"5min":1.034,"15min":1.5209999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.67800000000000005},"min":{"1min":0.495,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93999999999999995,"5min":1.177,"15min":1.177},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575034,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302208,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069632,"statfs":{"total":96636764160,"available":96327303168,"internally_reserved":0,"allocated":1040384,"data_stored":510225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.42599999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84999999999999998,"5min":0.98599999999999999,"15min":1.0109999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.52300000000000002,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":0.96199999999999997,"15min":0.96799999999999997},"last":0.624}]},{"osd":1,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90700000000000003,"5min":0.99399999999999999,"15min":1.0109999999999999},"last":0.748},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.753,"15min":0.74199999999999999},"min":{"1min":0.66800000000000004,"5min":0.432,"15min":0.40300000000000002},"max":{"1min":0.93899999999999995,"5min":0.99199999999999999,"15min":1.044},"last":0.81299999999999994}]},{"osd":2,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.59299999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84799999999999998,"5min":0.96299999999999997,"15min":1.024},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.57199999999999995,"5min":0.46000000000000002,"15min":0.45600000000000002},"max":{"1min":0.86899999999999999,"5min":0.96999999999999997,"15min":1.0329999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69999999999999996,"15min":0.69699999999999995},"min":{"1min":0.53100000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.88900000000000001,"5min":0.90900000000000003,"15min":0.93899999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70699999999999996,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.91700000000000004,"5min":0.97599999999999998,"15min":1.02},"last":0.68799999999999994}]},{"osd":4,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71699999999999997,"5min":0.86399999999999999,"15min":0.93700000000000006},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.39400000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89800000000000002,"5min":0.97499999999999998,"15min":0.98599999999999999},"last":0.89800000000000002}]},{"osd":6,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.63200000000000001},"min":{"1min":0.45100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88600000000000001,"5min":0.96399999999999997,"15min":0.97599999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.71899999999999997},"min":{"1min":0.503,"5min":0.49099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":1.105},"last":0.85499999999999998}]},{"osd":7,"last update":"Mon Nov 6 20:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90600000000000003,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.434},"max":{"1min":0.89500000000000002,"5min":0.998,"15min":0.998},"last":0.84299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T20:59:26.376 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T20:59:26.377 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T20:59:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.612+0000 7f5dc7fff700 1 Processor -- start 2023-11-06T20:59:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.612+0000 7f5dc7fff700 1 -- start start 2023-11-06T20:59:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.613+0000 7f5dc7fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc014faa0 0x7f5dc0064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.613+0000 7f5dc7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc0064270 con 0x7f5dc00c3450 2023-11-06T20:59:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.613+0000 7f5dc7fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc00651b0 con 0x7f5dc014fe80 2023-11-06T20:59:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.613+0000 7f5dc7fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc01531f0 con 0x7f5dc014faa0 2023-11-06T20:59:26.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.613+0000 7f5dc659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5dc00c3450 0x7f5dc00bbb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51800/0 (socket says 172.21.15.33:51800) 2023-11-06T20:59:26.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.613+0000 7f5dc659c700 1 -- 172.21.15.33:0/711446415 learned_addr learned my addr 172.21.15.33:0/711446415 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:26.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.614+0000 7f5dc5d9b700 1 --2- 172.21.15.33:0/711446415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc014faa0 0x7f5dc0064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.614+0000 7f5dc4d99700 1 -- 172.21.15.33:0/711446415 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308150152 0 0) 0x7f5dc0064270 con 0x7f5dc00c3450 2023-11-06T20:59:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.614+0000 7f5dc4d99700 1 -- 172.21.15.33:0/711446415 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dac0033f0 con 0x7f5dc00c3450 2023-11-06T20:59:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.614+0000 7f5dc4d99700 1 -- 172.21.15.33:0/711446415 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075123474 0 0) 0x7f5dc00651b0 con 0x7f5dc014fe80 2023-11-06T20:59:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.614+0000 7f5dc4d99700 1 -- 172.21.15.33:0/711446415 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc0064270 con 0x7f5dc014fe80 2023-11-06T20:59:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.614+0000 7f5dc5d9b700 1 -- 172.21.15.33:0/711446415 >> v1:172.21.15.69:6789/0 conn(0x7f5dc014fe80 legacy=0x7f5dc0152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.614+0000 7f5dc4d99700 1 -- 172.21.15.33:0/711446415 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1092945750 0 0) 0x7f5dac0033f0 con 0x7f5dc00c3450 2023-11-06T20:59:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.614+0000 7f5dc5d9b700 1 -- 172.21.15.33:0/711446415 >> v1:172.21.15.33:6789/0 conn(0x7f5dc00c3450 legacy=0x7f5dc00bbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.614+0000 7f5dc5d9b700 1 -- 172.21.15.33:0/711446415 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc01534d0 con 0x7f5dc014faa0 2023-11-06T20:59:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.614+0000 7f5dc5d9b700 1 --2- 172.21.15.33:0/711446415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc014faa0 0x7f5dc0064cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f5dbc009790 tx=0x7f5dbc009060).ready entity=mon.2 client_cookie=6379b2d707b3656e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.615+0000 7f5dc4d99700 1 -- 172.21.15.33:0/711446415 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db80017f0 con 0x7f5dc014faa0 2023-11-06T20:59:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.615+0000 7f5dc4d99700 1 -- 172.21.15.33:0/711446415 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc011400 con 0x7f5dc014faa0 2023-11-06T20:59:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.615+0000 7f5dc4d99700 1 -- 172.21.15.33:0/711446415 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc011780 con 0x7f5dc014faa0 2023-11-06T20:59:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.615+0000 7f5dc7fff700 1 -- 172.21.15.33:0/711446415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc014faa0 msgr2=0x7f5dc0064cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.615+0000 7f5dc7fff700 1 --2- 172.21.15.33:0/711446415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc014faa0 0x7f5dc0064cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f5dbc009790 tx=0x7f5dbc009060).stop 2023-11-06T20:59:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.615+0000 7f5dc7fff700 1 -- 172.21.15.33:0/711446415 shutdown_connections 2023-11-06T20:59:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.615+0000 7f5dc7fff700 1 --2- 172.21.15.33:0/711446415 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc014faa0 0x7f5dc0064cc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.615+0000 7f5dc7fff700 1 -- 172.21.15.33:0/711446415 >> 172.21.15.33:0/711446415 conn(0x7f5dc0062710 msgr2=0x7f5dc0062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.616+0000 7f5dc7fff700 1 -- 172.21.15.33:0/711446415 shutdown_connections 2023-11-06T20:59:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.616+0000 7f5dc7fff700 1 -- 172.21.15.33:0/711446415 wait complete. 2023-11-06T20:59:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.617+0000 7f5dc7fff700 1 Processor -- start 2023-11-06T20:59:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.617+0000 7f5dc7fff700 1 -- start start 2023-11-06T20:59:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.618+0000 7f5dc7fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc00c3450 0x7f5dc0144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.618+0000 7f5dc7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc014faa0 0x7f5dc00c0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.618+0000 7f5dc7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc014fe80 0x7f5dc014ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.618+0000 7f5dc7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc00aba70 con 0x7f5dc014faa0 2023-11-06T20:59:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.618+0000 7f5dc7fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc00abd10 con 0x7f5dc014fe80 2023-11-06T20:59:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.618+0000 7f5dc7fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc00abff0 con 0x7f5dc00c3450 2023-11-06T20:59:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.618+0000 7f5dc559a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc014faa0 0x7f5dc00c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.618+0000 7f5dc559a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc014faa0 0x7f5dc00c0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57418/0 (socket says 172.21.15.33:57418) 2023-11-06T20:59:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.618+0000 7f5dc559a700 1 -- 172.21.15.33:0/1910698341 learned_addr learned my addr 172.21.15.33:0/1910698341 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5dc5d9b700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc00c3450 0x7f5dc0144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5dc659c700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc014fe80 0x7f5dc014ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5dc559a700 1 -- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc00c3450 msgr2=0x7f5dc0144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5dc559a700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc00c3450 0x7f5dc0144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5dc559a700 1 -- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc014fe80 msgr2=0x7f5dc014ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5dc559a700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc014fe80 0x7f5dc014ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5dc559a700 1 -- 172.21.15.33:0/1910698341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0092a0 con 0x7f5dc014faa0 2023-11-06T20:59:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5dc559a700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc014faa0 0x7f5dc00c0640 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f5db000c4a0 tx=0x7f5db0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5db5ffb700 1 -- 172.21.15.33:0/1910698341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db000f470 con 0x7f5dc014faa0 2023-11-06T20:59:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5dc7fff700 1 -- 172.21.15.33:0/1910698341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc00ac2d0 con 0x7f5dc014faa0 2023-11-06T20:59:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5db5ffb700 1 -- 172.21.15.33:0/1910698341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db000f5d0 con 0x7f5dc014faa0 2023-11-06T20:59:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5db5ffb700 1 -- 172.21.15.33:0/1910698341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db000f970 con 0x7f5dc014faa0 2023-11-06T20:59:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.619+0000 7f5dc7fff700 1 -- 172.21.15.33:0/1910698341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc01450c0 con 0x7f5dc014faa0 2023-11-06T20:59:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.621+0000 7f5dc7fff700 1 -- 172.21.15.33:0/1910698341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc00ac7f0 con 0x7f5dc014faa0 2023-11-06T20:59:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.621+0000 7f5db5ffb700 1 -- 172.21.15.33:0/1910698341 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5db000fce0 con 0x7f5dc014faa0 2023-11-06T20:59:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.622+0000 7f5db5ffb700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f2a0 0x7f5da4061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.622+0000 7f5db5ffb700 1 -- 172.21.15.33:0/1910698341 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5db0079b40 con 0x7f5dc014faa0 2023-11-06T20:59:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.623+0000 7f5dc5d9b700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f2a0 0x7f5da4061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.623+0000 7f5dc5d9b700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f2a0 0x7f5da4061760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f5dbc00a010 tx=0x7f5dbc016000).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.626+0000 7f5db5ffb700 1 -- 172.21.15.33:0/1910698341 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5dc00ac7f0 con 0x7f5dc014faa0 2023-11-06T20:59:26.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.874+0000 7f5dc7fff700 1 -- 172.21.15.33:0/1910698341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5dc00ac7f0 con 0x7f5dc014faa0 2023-11-06T20:59:26.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.880+0000 7f5db5ffb700 1 -- 172.21.15.33:0/1910698341 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5dc00ac7f0 con 0x7f5dc014faa0 2023-11-06T20:59:26.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 -- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f2a0 msgr2=0x7f5da4061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f2a0 0x7f5da4061760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f5dbc00a010 tx=0x7f5dbc016000).stop 2023-11-06T20:59:26.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 -- 172.21.15.33:0/1910698341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc014faa0 msgr2=0x7f5dc00c0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:26.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc014faa0 0x7f5dc00c0640 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f5db000c4a0 tx=0x7f5db0007c50).stop 2023-11-06T20:59:26.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 -- 172.21.15.33:0/1910698341 shutdown_connections 2023-11-06T20:59:26.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f2a0 0x7f5da4061760 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5dc00c3450 0x7f5dc0144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc014fe80 0x7f5dc014ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 --2- 172.21.15.33:0/1910698341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc014faa0 0x7f5dc00c0640 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:26.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 -- 172.21.15.33:0/1910698341 >> 172.21.15.33:0/1910698341 conn(0x7f5dc0062710 msgr2=0x7f5dc01514a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:26.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 -- 172.21.15.33:0/1910698341 shutdown_connections 2023-11-06T20:59:26.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:26.882+0000 7f5dc7fff700 1 -- 172.21.15.33:0/1910698341 wait complete. 2023-11-06T20:59:26.885 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T20:59:26.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.097+0000 7fd2b79ff700 1 Processor -- start 2023-11-06T20:59:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.097+0000 7fd2b79ff700 1 -- start start 2023-11-06T20:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.099+0000 7fd2b79ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c61d0 0x7fd2b00c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.099+0000 7fd2b79ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b01554a0 con 0x7fd2b00c1840 2023-11-06T20:59:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.099+0000 7fd2b79ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b0155930 con 0x7fd2b00c5d50 2023-11-06T20:59:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.099+0000 7fd2b79ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0154960 con 0x7fd2b00c61d0 2023-11-06T20:59:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.099+0000 7fd2b579b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd2b00c1840 0x7fd2b00a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51804/0 (socket says 172.21.15.33:51804) 2023-11-06T20:59:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.099+0000 7fd2b579b700 1 -- 172.21.15.33:0/1978436381 learned_addr learned my addr 172.21.15.33:0/1978436381 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.099+0000 7fd2b5f9c700 1 --2- 172.21.15.33:0/1978436381 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c61d0 0x7fd2b00c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2a7fff700 1 -- 172.21.15.33:0/1978436381 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593392226 0 0) 0x7fd2b01554a0 con 0x7fd2b00c1840 2023-11-06T20:59:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2a7fff700 1 -- 172.21.15.33:0/1978436381 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd29c0033f0 con 0x7fd2b00c1840 2023-11-06T20:59:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2a7fff700 1 -- 172.21.15.33:0/1978436381 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329848387 0 0) 0x7fd2b0155930 con 0x7fd2b00c5d50 2023-11-06T20:59:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2a7fff700 1 -- 172.21.15.33:0/1978436381 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2b01554a0 con 0x7fd2b00c5d50 2023-11-06T20:59:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2b5f9c700 1 -- 172.21.15.33:0/1978436381 >> v1:172.21.15.69:6789/0 conn(0x7fd2b00c5d50 legacy=0x7fd2b00bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2b5f9c700 1 -- 172.21.15.33:0/1978436381 >> v1:172.21.15.33:6789/0 conn(0x7fd2b00c1840 legacy=0x7fd2b00a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2b5f9c700 1 -- 172.21.15.33:0/1978436381 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00aa4f0 con 0x7fd2b00c61d0 2023-11-06T20:59:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2b5f9c700 1 --2- 172.21.15.33:0/1978436381 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c61d0 0x7fd2b00c0480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd2a8009e90 tx=0x7fd2a8009780).ready entity=mon.2 client_cookie=39232dc58498ce8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2a7fff700 1 -- 172.21.15.33:0/1978436381 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800e070 con 0x7fd2b00c61d0 2023-11-06T20:59:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2a7fff700 1 -- 172.21.15.33:0/1978436381 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a800bab0 con 0x7fd2b00c61d0 2023-11-06T20:59:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.100+0000 7fd2a7fff700 1 -- 172.21.15.33:0/1978436381 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800be30 con 0x7fd2b00c61d0 2023-11-06T20:59:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.101+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1978436381 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c61d0 msgr2=0x7fd2b00c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.101+0000 7fd2b79ff700 1 --2- 172.21.15.33:0/1978436381 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c61d0 0x7fd2b00c0480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd2a8009e90 tx=0x7fd2a8009780).stop 2023-11-06T20:59:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.101+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1978436381 shutdown_connections 2023-11-06T20:59:27.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.101+0000 7fd2b79ff700 1 --2- 172.21.15.33:0/1978436381 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c61d0 0x7fd2b00c0480 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.101+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1978436381 >> 172.21.15.33:0/1978436381 conn(0x7fd2b005ec80 msgr2=0x7fd2b00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:27.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.101+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1978436381 shutdown_connections 2023-11-06T20:59:27.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.101+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1978436381 wait complete. 2023-11-06T20:59:27.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.102+0000 7fd2b79ff700 1 Processor -- start 2023-11-06T20:59:27.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.103+0000 7fd2b79ff700 1 -- start start 2023-11-06T20:59:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.103+0000 7fd2b79ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2b00c1840 0x7fd2b0158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b79ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c5d50 0x7fd2b00abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b79ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2b00c61d0 0x7fd2b00ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b79ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0158a80 con 0x7fd2b00c61d0 2023-11-06T20:59:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b79ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b00ac7f0 con 0x7fd2b00c1840 2023-11-06T20:59:27.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b79ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0156990 con 0x7fd2b00c5d50 2023-11-06T20:59:27.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b5f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2b00c61d0 0x7fd2b00ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b5f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2b00c61d0 0x7fd2b00ac300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57434/0 (socket says 172.21.15.33:57434) 2023-11-06T20:59:27.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b5f9c700 1 -- 172.21.15.33:0/1233189440 learned_addr learned my addr 172.21.15.33:0/1233189440 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:27.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b4f9a700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c5d50 0x7fd2b00abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b579b700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2b00c1840 0x7fd2b0158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b5f9c700 1 -- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c5d50 msgr2=0x7fd2b00abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b5f9c700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c5d50 0x7fd2b00abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b5f9c700 1 -- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2b00c1840 msgr2=0x7fd2b0158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b5f9c700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2b00c1840 0x7fd2b0158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b5f9c700 1 -- 172.21.15.33:0/1233189440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8009a20 con 0x7fd2b00c61d0 2023-11-06T20:59:27.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b579b700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2b00c1840 0x7fd2b0158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:27.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.104+0000 7fd2b5f9c700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2b00c61d0 0x7fd2b00ac300 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fd2a8009de0 tx=0x7fd2b0155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:27.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.105+0000 7fd2a57fa700 1 -- 172.21.15.33:0/1233189440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a801d670 con 0x7fd2b00c61d0 2023-11-06T20:59:27.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.105+0000 7fd2a57fa700 1 -- 172.21.15.33:0/1233189440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a801d7d0 con 0x7fd2b00c61d0 2023-11-06T20:59:27.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.105+0000 7fd2a57fa700 1 -- 172.21.15.33:0/1233189440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a801db70 con 0x7fd2b00c61d0 2023-11-06T20:59:27.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.105+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1233189440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b00c1eb0 con 0x7fd2b00c61d0 2023-11-06T20:59:27.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.105+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1233189440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b00c2a30 con 0x7fd2b00c61d0 2023-11-06T20:59:27.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.106+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1233189440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b0158e90 con 0x7fd2b00c61d0 2023-11-06T20:59:27.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.107+0000 7fd2a57fa700 1 -- 172.21.15.33:0/1233189440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd2a8003040 con 0x7fd2b00c61d0 2023-11-06T20:59:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.108+0000 7fd2a57fa700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd29405f270 0x7fd294061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.108+0000 7fd2a57fa700 1 -- 172.21.15.33:0/1233189440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd2a807c700 con 0x7fd2b00c61d0 2023-11-06T20:59:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.108+0000 7fd2b579b700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd29405f270 0x7fd294061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.109+0000 7fd2b579b700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd29405f270 0x7fd294061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd2ac001680 tx=0x7fd2ac009040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:27.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.112+0000 7fd2a57fa700 1 -- 172.21.15.33:0/1233189440 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd2a804d850 con 0x7fd2b00c61d0 2023-11-06T20:59:27.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.332+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1233189440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2b00c31e0 con 0x7fd2b00c61d0 2023-11-06T20:59:27.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.333+0000 7fd2a57fa700 1 -- 172.21.15.33:0/1233189440 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4490 (secure 0 0 0) 0x7fd2a8050e70 con 0x7fd2b00c61d0 2023-11-06T20:59:27.334 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:27.334 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6055,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1867036074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1867036074},{"type":"v1","addr":"172.21.15.69:6836","nonce":1867036074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6058,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1920115024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1920115024},{"type":"v1","addr":"172.21.15.69:6837","nonce":1920115024}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2935600703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2935600703},{"type":"v1","addr":"172.21.15.33:6835","nonce":2935600703}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:21.575862+0000","modified":"2023-11-06T20:59:24.582786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/3385607254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3385607254},{"type":"v1","addr":"172.21.15.33:6833","nonce":3385607254}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-06T20:59:27.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd29405f270 msgr2=0x7fd294061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd29405f270 0x7fd294061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd2ac001680 tx=0x7fd2ac009040).stop 2023-11-06T20:59:27.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1233189440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2b00c61d0 msgr2=0x7fd2b00ac300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2b00c61d0 0x7fd2b00ac300 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fd2a8009de0 tx=0x7fd2b0155410).stop 2023-11-06T20:59:27.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1233189440 shutdown_connections 2023-11-06T20:59:27.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd29405f270 0x7fd294061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2b00c61d0 0x7fd2b00ac300 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2b00c1840 0x7fd2b0158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 --2- 172.21.15.33:0/1233189440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2b00c5d50 0x7fd2b00abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1233189440 >> 172.21.15.33:0/1233189440 conn(0x7fd2b005ec80 msgr2=0x7fd2b0159090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:27.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1233189440 shutdown_connections 2023-11-06T20:59:27.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.335+0000 7fd2b79ff700 1 -- 172.21.15.33:0/1233189440 wait complete. 2023-11-06T20:59:27.339 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2023-11-06T20:59:27.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T20:59:21.575862+0000', 'modified': '2023-11-06T20:59:24.582786+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6201}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6201': {'gid': 6201, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6833/3385607254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6832', 'nonce': 3385607254}, {'type': 'v1', 'addr': '172.21.15.33:6833', 'nonce': 3385607254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T20:59:27.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T20:59:27.756 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T20:59:27.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.955+0000 7f74c07ff700 1 Processor -- start 2023-11-06T20:59:27.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.955+0000 7f74c07ff700 1 -- start start 2023-11-06T20:59:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.956+0000 7f74c07ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f74b8154210 0x7f74b80a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.956+0000 7f74c07ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74b80ca610 0x7f74b80c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.956+0000 7f74c07ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b80c06d0 con 0x7f74b80bfd60 2023-11-06T20:59:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.956+0000 7f74c07ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b80caaa0 con 0x7f74b8154210 2023-11-06T20:59:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.956+0000 7f74c07ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b805efc0 con 0x7f74b80ca610 2023-11-06T20:59:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.956+0000 7f74bdd9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f74b80bfd60 0x7f74b80c9f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35466/0 (socket says 172.21.15.33:35466) 2023-11-06T20:59:27.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.956+0000 7f74bdd9a700 1 -- 172.21.15.33:0/1611960321 learned_addr learned my addr 172.21.15.33:0/1611960321 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bed9c700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74b80ca610 0x7f74b80c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74be59b700 1 --2- 172.21.15.33:0/1611960321 >> v2:172.21.15.69:3300/0 conn(0x7f74b8154210 0x7f74b80a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330068888 0 0) 0x7f74b80c06d0 con 0x7f74b80bfd60 2023-11-06T20:59:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a40033f0 con 0x7f74b80bfd60 2023-11-06T20:59:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1124648368 0 0) 0x7f74a40033f0 con 0x7f74b80bfd60 2023-11-06T20:59:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74b80ca610 msgr2=0x7f74b80c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:27.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74b80ca610 0x7f74b80c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 >> v2:172.21.15.69:3300/0 conn(0x7f74b8154210 msgr2=0x7f74b80a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:27.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 --2- 172.21.15.33:0/1611960321 >> v2:172.21.15.69:3300/0 conn(0x7f74b8154210 0x7f74b80a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b80c8ea0 con 0x7f74b80bfd60 2023-11-06T20:59:27.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f74a8001470 con 0x7f74b80bfd60 2023-11-06T20:59:27.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 >> v1:172.21.15.33:6789/0 conn(0x7f74b80bfd60 legacy=0x7f74b80c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a40038e0 0x7f74a4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a4004690 0x7f74a4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74b80ca610 0x7f74a4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80caaa0 con 0x7f74a4004690 2023-11-06T20:59:27.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b805efc0 con 0x7f74a40038e0 2023-11-06T20:59:27.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.957+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a400b8e0 con 0x7f74b80ca610 2023-11-06T20:59:27.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a4004690 0x7f74a4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bed9c700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74b80ca610 0x7f74a4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74be59b700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a40038e0 0x7f74a4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bdd9a700 1 -- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74b80ca610 msgr2=0x7f74a4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74b80ca610 0x7f74a4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bdd9a700 1 -- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a40038e0 msgr2=0x7f74a4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a40038e0 0x7f74a4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bdd9a700 1 -- 172.21.15.33:0/1611960321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a40033f0 con 0x7f74a4004690 2023-11-06T20:59:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a4004690 0x7f74a4004b10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f74a800c210 tx=0x7f74a8001e90).ready entity=mon.0 client_cookie=51f8a029ebd7321 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:27.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8007a60 con 0x7f74a4004690 2023-11-06T20:59:27.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a800cae0 con 0x7f74a4004690 2023-11-06T20:59:27.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.958+0000 7f74bd599700 1 -- 172.21.15.33:0/1611960321 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a800ce80 con 0x7f74a4004690 2023-11-06T20:59:27.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.959+0000 7f74c07ff700 1 -- 172.21.15.33:0/1611960321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a4004690 msgr2=0x7f74a4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.959+0000 7f74c07ff700 1 --2- 172.21.15.33:0/1611960321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74a4004690 0x7f74a4004b10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f74a800c210 tx=0x7f74a8001e90).stop 2023-11-06T20:59:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.959+0000 7f74bed9c700 1 -- 172.21.15.33:0/1611960321 reap_dead start 2023-11-06T20:59:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.959+0000 7f74c07ff700 1 -- 172.21.15.33:0/1611960321 shutdown_connections 2023-11-06T20:59:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.959+0000 7f74c07ff700 1 -- 172.21.15.33:0/1611960321 >> 172.21.15.33:0/1611960321 conn(0x7f74b805e990 msgr2=0x7f74b80cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.959+0000 7f74c07ff700 1 -- 172.21.15.33:0/1611960321 shutdown_connections 2023-11-06T20:59:27.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.959+0000 7f74c07ff700 1 -- 172.21.15.33:0/1611960321 wait complete. 2023-11-06T20:59:27.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.961+0000 7f74c07ff700 1 Processor -- start 2023-11-06T20:59:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.961+0000 7f74c07ff700 1 -- start start 2023-11-06T20:59:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74c07ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74a4004690 0x7f74b80ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74c07ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b8164570 0x7f74b8168260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74c07ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b8168750 0x7f74b80aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74c07ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80b0d10 con 0x7f74b8164570 2023-11-06T20:59:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74c07ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b81569a0 con 0x7f74b8168750 2023-11-06T20:59:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74c07ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b8168bd0 con 0x7f74a4004690 2023-11-06T20:59:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74bdd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b8164570 0x7f74b8168260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74bdd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b8164570 0x7f74b8168260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34810/0 (socket says 172.21.15.33:34810) 2023-11-06T20:59:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74bdd9a700 1 -- 172.21.15.33:0/2584718924 learned_addr learned my addr 172.21.15.33:0/2584718924 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74be59b700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74a4004690 0x7f74b80ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74bed9c700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b8168750 0x7f74b80aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74bdd9a700 1 -- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74a4004690 msgr2=0x7f74b80ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74a4004690 0x7f74b80ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74bdd9a700 1 -- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b8168750 msgr2=0x7f74b80aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b8168750 0x7f74b80aa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.962+0000 7f74bdd9a700 1 -- 172.21.15.33:0/2584718924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b80ad810 con 0x7f74b8164570 2023-11-06T20:59:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.963+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b8164570 0x7f74b8168260 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f74a80075f0 tx=0x7f74a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.963+0000 7f74ae7fc700 1 -- 172.21.15.33:0/2584718924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8001580 con 0x7f74b8164570 2023-11-06T20:59:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.963+0000 7f74ae7fc700 1 -- 172.21.15.33:0/2584718924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a80016e0 con 0x7f74b8164570 2023-11-06T20:59:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.963+0000 7f74c07ff700 1 -- 172.21.15.33:0/2584718924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b8156070 con 0x7f74b8164570 2023-11-06T20:59:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.963+0000 7f74ae7fc700 1 -- 172.21.15.33:0/2584718924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8004900 con 0x7f74b8164570 2023-11-06T20:59:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.963+0000 7f74c07ff700 1 -- 172.21.15.33:0/2584718924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b8156630 con 0x7f74b8164570 2023-11-06T20:59:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.965+0000 7f74ae7fc700 1 -- 172.21.15.33:0/2584718924 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f74a8004cb0 con 0x7f74b8164570 2023-11-06T20:59:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.966+0000 7f74ae7fc700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f749c05f270 0x7f749c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.966+0000 7f74ae7fc700 1 -- 172.21.15.33:0/2584718924 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f74a8019020 con 0x7f74b8164570 2023-11-06T20:59:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.966+0000 7f74be59b700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f749c05f270 0x7f749c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.967+0000 7f74be59b700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f749c05f270 0x7f749c061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f74b4006d80 tx=0x7f74b4009060).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.968+0000 7f74d7a3fb80 1 Processor -- start 2023-11-06T20:59:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.968+0000 7f74d7a3fb80 1 -- start start 2023-11-06T20:59:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 0x5609ce3cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce417be0 0x5609ce3c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3f74c0 0x5609ce3ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ce30b2f0 con 0x5609ce3f74c0 2023-11-06T20:59:27.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ce3207e0 con 0x5609ce417be0 2023-11-06T20:59:27.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609ce320920 con 0x5609ce3cf080 2023-11-06T20:59:27.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74bed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3f74c0 0x5609ce3ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74bed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3f74c0 0x5609ce3ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34820/0 (socket says 172.21.15.33:34820) 2023-11-06T20:59:27.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74bed9c700 1 -- 172.21.15.33:0/1218678310 learned_addr learned my addr 172.21.15.33:0/1218678310 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:27.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74be59b700 1 --2- 172.21.15.33:0/1218678310 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 0x5609ce3cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/1218678310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce417be0 0x5609ce3c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74bed9c700 1 -- 172.21.15.33:0/1218678310 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 msgr2=0x5609ce3cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:27.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.969+0000 7f74bed9c700 1 --2- 172.21.15.33:0/1218678310 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 0x5609ce3cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.970+0000 7f74bed9c700 1 -- 172.21.15.33:0/1218678310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce417be0 msgr2=0x5609ce3c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:27.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.970+0000 7f74bed9c700 1 --2- 172.21.15.33:0/1218678310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce417be0 0x5609ce3c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.970+0000 7f74bed9c700 1 -- 172.21.15.33:0/1218678310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609ce319720 con 0x5609ce3f74c0 2023-11-06T20:59:27.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.970+0000 7f74be59b700 1 --2- 172.21.15.33:0/1218678310 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 0x5609ce3cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:27.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.970+0000 7f74bed9c700 1 --2- 172.21.15.33:0/1218678310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3f74c0 0x5609ce3ca3e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f74b80c1ef0 tx=0x7f74b000b040).ready entity=mon.0 client_cookie=364b4a79b8e0b042 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:27.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.970+0000 7f749b7fe700 1 -- 172.21.15.33:0/1218678310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000e040 con 0x5609ce3f74c0 2023-11-06T20:59:27.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.970+0000 7f749b7fe700 1 -- 172.21.15.33:0/1218678310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b000f040 con 0x5609ce3f74c0 2023-11-06T20:59:27.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.970+0000 7f749b7fe700 1 -- 172.21.15.33:0/1218678310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b001e400 con 0x5609ce3f74c0 2023-11-06T20:59:27.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.971+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1218678310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3f74c0 msgr2=0x5609ce3ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.971+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/1218678310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3f74c0 0x5609ce3ca3e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f74b80c1ef0 tx=0x7f74b000b040).stop 2023-11-06T20:59:27.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.971+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1218678310 shutdown_connections 2023-11-06T20:59:27.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.971+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/1218678310 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 0x5609ce3cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.971+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/1218678310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3f74c0 0x5609ce3ca3e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.971+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/1218678310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce417be0 0x5609ce3c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.971+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1218678310 >> 172.21.15.33:0/1218678310 conn(0x5609ce238940 msgr2=0x5609ce418070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:27.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.971+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1218678310 shutdown_connections 2023-11-06T20:59:27.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.971+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1218678310 wait complete. 2023-11-06T20:59:27.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.971+0000 7f74d7a3fb80 1 -- start start 2023-11-06T20:59:27.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.972+0000 7f74d7a3fb80 1 Processor -- start 2023-11-06T20:59:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce4173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3f74c0 0x5609ce3fa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce417be0 0x5609ce4162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ce320ba0 con 0x5609ce3cf080 2023-11-06T20:59:27.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ce2ebe60 con 0x5609ce417be0 2023-11-06T20:59:27.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609ce424920 con 0x5609ce3f74c0 2023-11-06T20:59:27.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74d7a3fb80 10 client.0 ll_register_callbacks cb 0x5609ce2c0ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T20:59:27.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74be59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce4173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74be59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce4173f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34822/0 (socket says 172.21.15.33:34822) 2023-11-06T20:59:27.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74be59b700 1 -- 172.21.15.33:0/1649246062 learned_addr learned my addr 172.21.15.33:0/1649246062 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:27.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3f74c0 0x5609ce3fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74bed9c700 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce417be0 0x5609ce4162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74be59b700 1 -- 172.21.15.33:0/1649246062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3f74c0 msgr2=0x5609ce3fa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:27.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74be59b700 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3f74c0 0x5609ce3fa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74be59b700 1 -- 172.21.15.33:0/1649246062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce417be0 msgr2=0x5609ce4162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:27.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74be59b700 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce417be0 0x5609ce4162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.973+0000 7f74be59b700 1 -- 172.21.15.33:0/1649246062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b0009d90 con 0x5609ce3cf080 2023-11-06T20:59:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f74be59b700 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce4173f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f74b40157d0 tx=0x7f74b401a000).ready entity=mon.0 client_cookie=a46cad6da42be000 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f7498ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T20:59:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f7498ff9700 1 -- 172.21.15.33:0/1649246062 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4014870 con 0x5609ce3cf080 2023-11-06T20:59:27.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f7498ff9700 1 -- 172.21.15.33:0/1649246062 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f74b40216f0 con 0x5609ce3cf080 2023-11-06T20:59:27.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1649246062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609ce3c2e20 con 0x5609ce3cf080 2023-11-06T20:59:27.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f7498ff9700 1 -- 172.21.15.33:0/1649246062 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b40218b0 con 0x5609ce3cf080 2023-11-06T20:59:27.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f74d7a3fb80 10 client.6339 fetch_fsmap learned FSMap version 29 2023-11-06T20:59:27.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1649246062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5609ce314ac0 con 0x5609ce3cf080 2023-11-06T20:59:27.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f7498ff9700 1 -- 172.21.15.33:0/1649246062 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5609ce3c2e20 con 0x5609ce3cf080 2023-11-06T20:59:27.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f7498ff9700 1 -- 172.21.15.33:0/1649246062 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f74b4021a10 con 0x5609ce3cf080 2023-11-06T20:59:27.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f74d7a3fb80 10 client.6339 fetch_fsmap finished waiting for FSMap version 29 2023-11-06T20:59:27.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.974+0000 7f74d7a3fb80 10 client.6339 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T20:59:27.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.975+0000 7f74d7a3fb80 20 client.6339 populate_metadata read hostname 'smithi033' 2023-11-06T20:59:27.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.975+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.33:6832/3385607254,v1:172.21.15.33:6833/3385607254] conn(0x5609ce414b10 0x5609ce3f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:27.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.975+0000 7f74d7a3fb80 4 client.6339 mds_command: new command op to 6201 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T20:59:27.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.975+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1649246062 --> [v2:172.21.15.33:6832/3385607254,v1:172.21.15.33:6833/3385607254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609ce188ef0 con 0x5609ce414b10 2023-11-06T20:59:27.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.975+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.33:6832/3385607254,v1:172.21.15.33:6833/3385607254] conn(0x5609ce414b10 0x5609ce3f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:27.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.975+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.33:6832/3385607254,v1:172.21.15.33:6833/3385607254] conn(0x5609ce414b10 0x5609ce3f86e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e7eb984019068287 server_cookie=5450e0d8c27166d0 in_seq=0 out_seq=0 2023-11-06T20:59:27.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.975+0000 7f7498ff9700 10 client.6339 ms_handle_connect on v2:172.21.15.33:6832/3385607254 2023-11-06T20:59:27.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f7498ff9700 1 -- 172.21.15.33:0/1649246062 <== mds.0 v2:172.21.15.33:6832/3385607254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x5609ce188ef0 con 0x5609ce414b10 2023-11-06T20:59:27.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f7498ff9700 10 client.6339 handle_command_reply: tid=0 2023-11-06T20:59:27.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 client.6339 shutdown 2023-11-06T20:59:27.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1649246062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 msgr2=0x5609ce4173f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:27.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce4173f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f74b40157d0 tx=0x7f74b401a000).stop 2023-11-06T20:59:27.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1649246062 shutdown_connections 2023-11-06T20:59:27.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.33:6832/3385607254,v1:172.21.15.33:6833/3385607254] conn(0x5609ce414b10 0x5609ce3f86e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce417be0 0x5609ce4162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce4173f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/1649246062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3f74c0 0x5609ce3fa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:27.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f7498ff9700 0 client.6339 ms_handle_reset on v2:172.21.15.33:6832/3385607254 2023-11-06T20:59:27.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1649246062 >> 172.21.15.33:0/1649246062 conn(0x5609ce238940 msgr2=0x5609ce3ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:27.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1649246062 shutdown_connections 2023-11-06T20:59:27.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.976+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/1649246062 wait complete. 2023-11-06T20:59:27.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.977+0000 7f74d7a3fb80 20 client.6339 trim_cache size 0 max 16384 2023-11-06T20:59:28.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.994+0000 7f74d7a3fb80 1 Processor -- start 2023-11-06T20:59:28.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.994+0000 7f74d7a3fb80 1 -- start start 2023-11-06T20:59:28.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 0x5609ce411810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 0x5609ce3cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce417be0 0x5609ce4175a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ce3f7350 con 0x5609ce417be0 2023-11-06T20:59:28.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ce417a90 con 0x5609ce3f74c0 2023-11-06T20:59:28.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609ce3f9d70 con 0x5609ce3cf080 2023-11-06T20:59:28.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74bed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce417be0 0x5609ce4175a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74bed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce417be0 0x5609ce4175a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34828/0 (socket says 172.21.15.33:34828) 2023-11-06T20:59:28.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74bed9c700 1 -- 172.21.15.33:0/3746803206 learned_addr learned my addr 172.21.15.33:0/3746803206 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:28.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74be59b700 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 0x5609ce411810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 0x5609ce3cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74bed9c700 1 -- 172.21.15.33:0/3746803206 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 msgr2=0x5609ce411810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:28.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74bed9c700 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 0x5609ce411810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74bed9c700 1 -- 172.21.15.33:0/3746803206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 msgr2=0x5609ce3cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:28.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74bed9c700 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 0x5609ce3cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74bed9c700 1 -- 172.21.15.33:0/3746803206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609ce314810 con 0x5609ce417be0 2023-11-06T20:59:28.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 0x5609ce3cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:28.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.995+0000 7f74be59b700 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 0x5609ce411810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:28.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f74bed9c700 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce417be0 0x5609ce4175a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f74b80836e0 tx=0x7f74b0020000).ready entity=mon.0 client_cookie=c7044eb13b86aecb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:28.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f749bfff700 1 -- 172.21.15.33:0/3746803206 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000e040 con 0x5609ce417be0 2023-11-06T20:59:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f749bfff700 1 -- 172.21.15.33:0/3746803206 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b0014a40 con 0x5609ce417be0 2023-11-06T20:59:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f749bfff700 1 -- 172.21.15.33:0/3746803206 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b001e770 con 0x5609ce417be0 2023-11-06T20:59:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/3746803206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce417be0 msgr2=0x5609ce4175a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce417be0 0x5609ce4175a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f74b80836e0 tx=0x7f74b0020000).stop 2023-11-06T20:59:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/3746803206 shutdown_connections 2023-11-06T20:59:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce3cf080 0x5609ce411810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce417be0 0x5609ce4175a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/3746803206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 0x5609ce3cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.996+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/3746803206 >> 172.21.15.33:0/3746803206 conn(0x5609ce238940 msgr2=0x5609ce3caf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:28.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.997+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/3746803206 shutdown_connections 2023-11-06T20:59:28.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.997+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/3746803206 wait complete. 2023-11-06T20:59:28.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.997+0000 7f74d7a3fb80 1 -- start start 2023-11-06T20:59:28.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.998+0000 7f74d7a3fb80 1 Processor -- start 2023-11-06T20:59:28.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.999+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce410f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.999+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 0x5609ce4797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.999+0000 7f74d7a3fb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce417be0 0x5609ce479ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.999+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ce47a5b0 con 0x5609ce3cf080 2023-11-06T20:59:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.999+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609ce3f9eb0 con 0x5609ce3f74c0 2023-11-06T20:59:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.999+0000 7f74d7a3fb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609ce3fa100 con 0x5609ce417be0 2023-11-06T20:59:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:27.999+0000 7f74d7a3fb80 10 client.0 ll_register_callbacks cb 0x5609ce2c0ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T20:59:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74be59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce410f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74be59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce410f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34836/0 (socket says 172.21.15.33:34836) 2023-11-06T20:59:28.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74be59b700 1 -- 172.21.15.33:0/819858150 learned_addr learned my addr 172.21.15.33:0/819858150 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:28.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74bed9c700 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce417be0 0x5609ce479ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 0x5609ce4797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74be59b700 1 -- 172.21.15.33:0/819858150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce417be0 msgr2=0x5609ce479ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:28.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74be59b700 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce417be0 0x5609ce479ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74be59b700 1 -- 172.21.15.33:0/819858150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 msgr2=0x5609ce4797f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:28.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74be59b700 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 0x5609ce4797f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74be59b700 1 -- 172.21.15.33:0/819858150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609ce319720 con 0x5609ce3cf080 2023-11-06T20:59:28.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f74be59b700 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce410f90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f74b4006d30 tx=0x7f74b401e000).ready entity=mon.0 client_cookie=9f43bbba32a72252 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:28.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.000+0000 7f7498ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T20:59:28.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f7498ff9700 1 -- 172.21.15.33:0/819858150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4014870 con 0x5609ce3cf080 2023-11-06T20:59:28.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f7498ff9700 1 -- 172.21.15.33:0/819858150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f74b402a460 con 0x5609ce3cf080 2023-11-06T20:59:28.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/819858150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74b000f040 con 0x5609ce3cf080 2023-11-06T20:59:28.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f7498ff9700 1 -- 172.21.15.33:0/819858150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b402a870 con 0x5609ce3cf080 2023-11-06T20:59:28.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f74d7a3fb80 10 client.6345 fetch_fsmap learned FSMap version 29 2023-11-06T20:59:28.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/819858150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5609ce18e1d0 con 0x5609ce3cf080 2023-11-06T20:59:28.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f7498ff9700 1 -- 172.21.15.33:0/819858150 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b000f040 con 0x5609ce3cf080 2023-11-06T20:59:28.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f7498ff9700 1 -- 172.21.15.33:0/819858150 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f74b402a9d0 con 0x5609ce3cf080 2023-11-06T20:59:28.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f74d7a3fb80 10 client.6345 fetch_fsmap finished waiting for FSMap version 29 2023-11-06T20:59:28.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f74d7a3fb80 10 client.6345 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T20:59:28.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f74d7a3fb80 20 client.6345 populate_metadata read hostname 'smithi033' 2023-11-06T20:59:28.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.33:6832/3385607254,v1:172.21.15.33:6833/3385607254] conn(0x5609ce4150c0 0x5609ce4154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f74d7a3fb80 4 client.6345 mds_command: new command op to 6201 tid=0[{"prefix": "status"}] 2023-11-06T20:59:28.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.001+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/819858150 --> [v2:172.21.15.33:6832/3385607254,v1:172.21.15.33:6833/3385607254] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5609ce1be040 con 0x5609ce4150c0 2023-11-06T20:59:28.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.33:6832/3385607254,v1:172.21.15.33:6833/3385607254] conn(0x5609ce4150c0 0x5609ce4154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74bdd9a700 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.33:6832/3385607254,v1:172.21.15.33:6833/3385607254] conn(0x5609ce4150c0 0x5609ce4154a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=10b84e5da560d254 server_cookie=abe463bf8f12915d in_seq=0 out_seq=0 2023-11-06T20:59:28.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f7498ff9700 10 client.6345 ms_handle_connect on v2:172.21.15.33:6832/3385607254 2023-11-06T20:59:28.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f7498ff9700 1 -- 172.21.15.33:0/819858150 <== mds.0 v2:172.21.15.33:6832/3385607254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x5609ce1be040 con 0x5609ce4150c0 2023-11-06T20:59:28.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f7498ff9700 10 client.6345 handle_command_reply: tid=0 2023-11-06T20:59:28.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74d7a3fb80 1 client.6345 shutdown 2023-11-06T20:59:28.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/819858150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 msgr2=0x5609ce410f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce410f90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f74b4006d30 tx=0x7f74b401e000).stop 2023-11-06T20:59:28.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/819858150 shutdown_connections 2023-11-06T20:59:28.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5609ce417be0 0x5609ce479ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.33:6832/3385607254,v1:172.21.15.33:6833/3385607254] conn(0x5609ce4150c0 0x5609ce4154a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609ce3cf080 0x5609ce410f90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74d7a3fb80 1 --2- 172.21.15.33:0/819858150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5609ce3f74c0 0x5609ce4797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f7498ff9700 0 client.6345 ms_handle_reset on v2:172.21.15.33:6832/3385607254 2023-11-06T20:59:28.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.002+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/819858150 >> 172.21.15.33:0/819858150 conn(0x5609ce238940 msgr2=0x5609ce3cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:28.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/819858150 shutdown_connections 2023-11-06T20:59:28.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74d7a3fb80 1 -- 172.21.15.33:0/819858150 wait complete. 2023-11-06T20:59:28.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74d7a3fb80 20 client.6345 trim_cache size 0 max 16384 2023-11-06T20:59:28.020 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T20:59:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T20:59:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T20:59:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 6201, 2023-11-06T20:59:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T20:59:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T20:59:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T20:59:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.425412347, 2023-11-06T20:59:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 29, 2023-11-06T20:59:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 84, 2023-11-06T20:59:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 84, 2023-11-06T20:59:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 11.631445052 2023-11-06T20:59:28.022 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T20:59:28.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74c07ff700 1 -- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f749c05f270 msgr2=0x7f749c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74c07ff700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f749c05f270 0x7f749c061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f74b4006d80 tx=0x7f74b4009060).stop 2023-11-06T20:59:28.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74c07ff700 1 -- 172.21.15.33:0/2584718924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b8164570 msgr2=0x7f74b8168260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74c07ff700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b8164570 0x7f74b8168260 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f74a80075f0 tx=0x7f74a8015000).stop 2023-11-06T20:59:28.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74c07ff700 1 -- 172.21.15.33:0/2584718924 shutdown_connections 2023-11-06T20:59:28.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74c07ff700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f749c05f270 0x7f749c061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74c07ff700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f74a4004690 0x7f74b80ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74c07ff700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74b8168750 0x7f74b80aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74c07ff700 1 --2- 172.21.15.33:0/2584718924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b8164570 0x7f74b8168260 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.003+0000 7f74c07ff700 1 -- 172.21.15.33:0/2584718924 >> 172.21.15.33:0/2584718924 conn(0x7f74b805e990 msgr2=0x7f74b80c6640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:28.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.004+0000 7f74c07ff700 1 -- 172.21.15.33:0/2584718924 shutdown_connections 2023-11-06T20:59:28.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.004+0000 7f74c07ff700 1 -- 172.21.15.33:0/2584718924 wait complete. 2023-11-06T20:59:28.027 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T20:59:28.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.190+0000 7ff28bbff700 1 Processor -- start 2023-11-06T20:59:28.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.190+0000 7ff28bbff700 1 -- start start 2023-11-06T20:59:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.192+0000 7ff28bbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff2840c0e60 0x7ff2840a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.192+0000 7ff28bbff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2840a9820 con 0x7ff2840c9e30 2023-11-06T20:59:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.192+0000 7ff28bbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff284154700 con 0x7ff284154320 2023-11-06T20:59:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.192+0000 7ff28bbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2840a8c10 con 0x7ff2840c0e60 2023-11-06T20:59:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.192+0000 7ff28919a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff2840c9e30 0x7ff2840c0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35476/0 (socket says 172.21.15.33:35476) 2023-11-06T20:59:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.192+0000 7ff28919a700 1 -- 172.21.15.33:0/1990282219 learned_addr learned my addr 172.21.15.33:0/1990282219 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.192+0000 7ff28a19c700 1 --2- 172.21.15.33:0/1990282219 >> v2:172.21.15.69:3300/0 conn(0x7ff2840c0e60 0x7ff2840a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff288999700 1 -- 172.21.15.33:0/1990282219 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593261795 0 0) 0x7ff2840a9820 con 0x7ff2840c9e30 2023-11-06T20:59:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff288999700 1 -- 172.21.15.33:0/1990282219 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2700033f0 con 0x7ff2840c9e30 2023-11-06T20:59:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff288999700 1 -- 172.21.15.33:0/1990282219 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939895247 0 0) 0x7ff284154700 con 0x7ff284154320 2023-11-06T20:59:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff288999700 1 -- 172.21.15.33:0/1990282219 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2840a9820 con 0x7ff284154320 2023-11-06T20:59:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff28a19c700 1 -- 172.21.15.33:0/1990282219 >> v1:172.21.15.69:6789/0 conn(0x7ff284154320 legacy=0x7ff2840aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff28a19c700 1 -- 172.21.15.33:0/1990282219 >> v1:172.21.15.33:6789/0 conn(0x7ff2840c9e30 legacy=0x7ff2840c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff28a19c700 1 -- 172.21.15.33:0/1990282219 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840a8ef0 con 0x7ff2840c0e60 2023-11-06T20:59:28.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff28a19c700 1 --2- 172.21.15.33:0/1990282219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2840c0e60 0x7ff2840a8670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7ff27c009e60 tx=0x7ff27c009780).ready entity=mon.1 client_cookie=c7e1f84ee965915b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:28.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff288999700 1 -- 172.21.15.33:0/1990282219 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00e070 con 0x7ff2840c0e60 2023-11-06T20:59:28.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff288999700 1 -- 172.21.15.33:0/1990282219 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27c00bab0 con 0x7ff2840c0e60 2023-11-06T20:59:28.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.193+0000 7ff288999700 1 -- 172.21.15.33:0/1990282219 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00be30 con 0x7ff2840c0e60 2023-11-06T20:59:28.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.194+0000 7ff28bbff700 1 -- 172.21.15.33:0/1990282219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2840c0e60 msgr2=0x7ff2840a8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.194+0000 7ff28bbff700 1 --2- 172.21.15.33:0/1990282219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2840c0e60 0x7ff2840a8670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7ff27c009e60 tx=0x7ff27c009780).stop 2023-11-06T20:59:28.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.194+0000 7ff28bbff700 1 -- 172.21.15.33:0/1990282219 shutdown_connections 2023-11-06T20:59:28.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.194+0000 7ff28bbff700 1 --2- 172.21.15.33:0/1990282219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2840c0e60 0x7ff2840a8670 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.194+0000 7ff28bbff700 1 -- 172.21.15.33:0/1990282219 >> 172.21.15.33:0/1990282219 conn(0x7ff284094530 msgr2=0x7ff28406a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.194+0000 7ff28bbff700 1 -- 172.21.15.33:0/1990282219 shutdown_connections 2023-11-06T20:59:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.194+0000 7ff28bbff700 1 -- 172.21.15.33:0/1990282219 wait complete. 2023-11-06T20:59:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.196+0000 7ff28bbff700 1 Processor -- start 2023-11-06T20:59:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.196+0000 7ff28bbff700 1 -- start start 2023-11-06T20:59:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.197+0000 7ff28bbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff2840c0e60 0x7ff284158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28bbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2840c9e30 0x7ff2840c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28bbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff284154320 0x7ff2840b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28bbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff284158960 con 0x7ff284154320 2023-11-06T20:59:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28bbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840c2600 con 0x7ff2840c9e30 2023-11-06T20:59:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28bbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840ae140 con 0x7ff2840c0e60 2023-11-06T20:59:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28a19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff284154320 0x7ff2840b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28a19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff284154320 0x7ff2840b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34846/0 (socket says 172.21.15.33:34846) 2023-11-06T20:59:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28a19c700 1 -- 172.21.15.33:0/3796893202 learned_addr learned my addr 172.21.15.33:0/3796893202 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28999b700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff2840c0e60 0x7ff284158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28919a700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2840c9e30 0x7ff2840c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28a19c700 1 -- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff2840c0e60 msgr2=0x7ff284158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28a19c700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff2840c0e60 0x7ff284158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28a19c700 1 -- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2840c9e30 msgr2=0x7ff2840c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28a19c700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2840c9e30 0x7ff2840c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28a19c700 1 -- 172.21.15.33:0/3796893202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c009a20 con 0x7ff284154320 2023-11-06T20:59:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28999b700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff2840c0e60 0x7ff284158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28919a700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2840c9e30 0x7ff2840c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.198+0000 7ff28a19c700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff284154320 0x7ff2840b0480 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7ff27c00bf50 tx=0x7ff27c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.199+0000 7ff279ffb700 1 -- 172.21.15.33:0/3796893202 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c01d5a0 con 0x7ff284154320 2023-11-06T20:59:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.199+0000 7ff279ffb700 1 -- 172.21.15.33:0/3796893202 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27c01d700 con 0x7ff284154320 2023-11-06T20:59:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.199+0000 7ff279ffb700 1 -- 172.21.15.33:0/3796893202 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c01da80 con 0x7ff284154320 2023-11-06T20:59:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.199+0000 7ff28bbff700 1 -- 172.21.15.33:0/3796893202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2840aeb90 con 0x7ff284154320 2023-11-06T20:59:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.199+0000 7ff28bbff700 1 -- 172.21.15.33:0/3796893202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2840c1fb0 con 0x7ff284154320 2023-11-06T20:59:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.200+0000 7ff28bbff700 1 -- 172.21.15.33:0/3796893202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff284157bc0 con 0x7ff284154320 2023-11-06T20:59:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.201+0000 7ff279ffb700 1 -- 172.21.15.33:0/3796893202 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff27c003040 con 0x7ff284154320 2023-11-06T20:59:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.201+0000 7ff279ffb700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff26805f270 0x7ff268061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.201+0000 7ff279ffb700 1 -- 172.21.15.33:0/3796893202 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff27c07c5f0 con 0x7ff284154320 2023-11-06T20:59:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.202+0000 7ff28999b700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff26805f270 0x7ff268061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.203+0000 7ff28999b700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff26805f270 0x7ff268061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff280007fd0 tx=0x7ff280009040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.205+0000 7ff279ffb700 1 -- 172.21.15.33:0/3796893202 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff27c04d740 con 0x7ff284154320 2023-11-06T20:59:28.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.429+0000 7ff28bbff700 1 -- 172.21.15.33:0/3796893202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff2841574a0 con 0x7ff284154320 2023-11-06T20:59:28.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.429+0000 7ff279ffb700 1 -- 172.21.15.33:0/3796893202 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff27c050d60 con 0x7ff284154320 2023-11-06T20:59:28.430 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T20:59:28.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 -- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff26805f270 msgr2=0x7ff268061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff26805f270 0x7ff268061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff280007fd0 tx=0x7ff280009040).stop 2023-11-06T20:59:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 -- 172.21.15.33:0/3796893202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff284154320 msgr2=0x7ff2840b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff284154320 0x7ff2840b0480 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7ff27c00bf50 tx=0x7ff27c009780).stop 2023-11-06T20:59:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 -- 172.21.15.33:0/3796893202 shutdown_connections 2023-11-06T20:59:28.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff26805f270 0x7ff268061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff2840c0e60 0x7ff284158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff284154320 0x7ff2840b0480 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 --2- 172.21.15.33:0/3796893202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2840c9e30 0x7ff2840c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 -- 172.21.15.33:0/3796893202 >> 172.21.15.33:0/3796893202 conn(0x7ff284094530 msgr2=0x7ff2840c8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 -- 172.21.15.33:0/3796893202 shutdown_connections 2023-11-06T20:59:28.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:28.432+0000 7ff28bbff700 1 -- 172.21.15.33:0/3796893202 wait complete. 2023-11-06T20:59:28.447 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T20:59:28.448 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T20:59:28.448 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T20:59:28.449 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T20:59:28.449 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T20:59:28.450 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T20:59:28.450 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T20:59:28.451 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T20:59:28.451 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T20:59:28.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:59:28.453 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T20:59:28.502 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T20:59:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T20:59:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T20:59:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:59:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T20:59:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:59:28.504 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:59:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:59:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T20:59:28.504 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:59:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:59:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T20:59:28.505 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T20:59:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:59:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T20:59:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T20:59:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41376sec preferred_lft 41376sec 2023-11-06T20:59:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T20:59:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:59:28.506 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T20:59:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T20:59:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T20:59:28.507 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T20:59:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T20:59:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T20:59:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:59:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T20:59:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:59:28.508 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T20:59:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T20:59:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T20:59:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:59:28.509 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T20:59:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T20:59:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T20:59:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T20:59:28.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:59:28.510 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T20:59:28.579 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T20:59:28.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:59:28.583 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T20:59:28.633 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T20:59:28.633 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T20:59:28.634 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T20:59:28.634 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T20:59:28.634 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T20:59:28.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:59:28.636 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T20:59:28.636 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T20:59:28.636 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T20:59:28.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T20:59:28.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T20:59:28.637 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T20:59:28.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:59:28.812 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T20:59:28.859 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T20:59:28.860 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T20:59:28.861 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T20:59:28.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:59:28.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T20:59:28.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:59:28.935 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T20:59:28.982 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T20:59:28.983 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:59:29.125 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:59:29.126 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:59:29.151 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T20:59:29.152 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T20:59:29.157 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T20:59:29.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:59:29.158 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T20:59:29.230 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T20:59:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.428+0000 7f01e7fff700 1 Processor -- start 2023-11-06T20:59:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.428+0000 7f01e7fff700 1 -- start start 2023-11-06T20:59:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.429+0000 7f01e7fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f01e00c6d90 0x7f01e00c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.429+0000 7f01e7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01e00bfd60 0x7f01e00c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.429+0000 7f01e7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e00c7700 con 0x7f01e0154210 2023-11-06T20:59:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.429+0000 7f01e7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e00c06c0 con 0x7f01e00bfd60 2023-11-06T20:59:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.429+0000 7f01e7fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e005efc0 con 0x7f01e00c6d90 2023-11-06T20:59:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e5d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f01e0154210 0x7f01e00a7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35486/0 (socket says 172.21.15.33:35486) 2023-11-06T20:59:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e5d9b700 1 -- 172.21.15.33:0/2068326463 learned_addr learned my addr 172.21.15.33:0/2068326463 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e659c700 1 --2- 172.21.15.33:0/2068326463 >> v2:172.21.15.33:3300/0 conn(0x7f01e00bfd60 0x7f01e00c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e559a700 1 --2- 172.21.15.33:0/2068326463 >> v2:172.21.15.69:3300/0 conn(0x7f01e00c6d90 0x7f01e00c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e659c700 1 -- 172.21.15.33:0/2068326463 >> v2:172.21.15.69:3300/0 conn(0x7f01e00c6d90 msgr2=0x7f01e00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e659c700 1 --2- 172.21.15.33:0/2068326463 >> v2:172.21.15.69:3300/0 conn(0x7f01e00c6d90 0x7f01e00c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e659c700 1 -- 172.21.15.33:0/2068326463 >> v1:172.21.15.33:6789/0 conn(0x7f01e0154210 legacy=0x7f01e00a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e659c700 1 -- 172.21.15.33:0/2068326463 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e00c8ea0 con 0x7f01e00bfd60 2023-11-06T20:59:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e4d99700 1 -- 172.21.15.33:0/2068326463 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115387272 0 0) 0x7f01dc001110 con 0x7f01e0154210 2023-11-06T20:59:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e559a700 1 --2- 172.21.15.33:0/2068326463 >> v2:172.21.15.69:3300/0 conn(0x7f01e00c6d90 0x7f01e00c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.430+0000 7f01e659c700 1 --2- 172.21.15.33:0/2068326463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e00bfd60 0x7f01e00c01d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f01e00cbf10 tx=0x7f01d8009780).ready entity=mon.0 client_cookie=36ea9c91edf0b3d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e4d99700 1 -- 172.21.15.33:0/2068326463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d800e070 con 0x7f01e00bfd60 2023-11-06T20:59:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e4d99700 1 -- 172.21.15.33:0/2068326463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d800bc30 con 0x7f01e00bfd60 2023-11-06T20:59:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e4d99700 1 -- 172.21.15.33:0/2068326463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8012410 con 0x7f01e00bfd60 2023-11-06T20:59:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e7fff700 1 -- 172.21.15.33:0/2068326463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e00bfd60 msgr2=0x7f01e00c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e7fff700 1 --2- 172.21.15.33:0/2068326463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e00bfd60 0x7f01e00c01d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f01e00cbf10 tx=0x7f01d8009780).stop 2023-11-06T20:59:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e7fff700 1 -- 172.21.15.33:0/2068326463 shutdown_connections 2023-11-06T20:59:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e7fff700 1 --2- 172.21.15.33:0/2068326463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e00bfd60 0x7f01e00c01d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e7fff700 1 --2- 172.21.15.33:0/2068326463 >> v2:172.21.15.69:3300/0 conn(0x7f01e00c6d90 0x7f01e00c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e7fff700 1 -- 172.21.15.33:0/2068326463 >> 172.21.15.33:0/2068326463 conn(0x7f01e005e990 msgr2=0x7f01e00cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e7fff700 1 -- 172.21.15.33:0/2068326463 shutdown_connections 2023-11-06T20:59:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.431+0000 7f01e7fff700 1 -- 172.21.15.33:0/2068326463 wait complete. 2023-11-06T20:59:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.433+0000 7f01e7fff700 1 Processor -- start 2023-11-06T20:59:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.433+0000 7f01e7fff700 1 -- start start 2023-11-06T20:59:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.434+0000 7f01e7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01e00bfd60 0x7f01e014df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.434+0000 7f01e7fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01e00c6d90 0x7f01e00c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.434+0000 7f01e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0154210 0x7f01e00c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.434+0000 7f01e7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00c4180 con 0x7f01e0154210 2023-11-06T20:59:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.434+0000 7f01e7fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00c2e70 con 0x7f01e00bfd60 2023-11-06T20:59:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.434+0000 7f01e7fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00c3150 con 0x7f01e00c6d90 2023-11-06T20:59:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.434+0000 7f01e659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0154210 0x7f01e00c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.434+0000 7f01e659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0154210 0x7f01e00c28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34872/0 (socket says 172.21.15.33:34872) 2023-11-06T20:59:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.434+0000 7f01e659c700 1 -- 172.21.15.33:0/457011386 learned_addr learned my addr 172.21.15.33:0/457011386 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.434+0000 7f01e5d9b700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01e00bfd60 0x7f01e014df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01e559a700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01e00c6d90 0x7f01e00c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01e659c700 1 -- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01e00c6d90 msgr2=0x7f01e00c23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01e659c700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01e00c6d90 0x7f01e00c23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01e659c700 1 -- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01e00bfd60 msgr2=0x7f01e014df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01e659c700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01e00bfd60 0x7f01e014df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01e659c700 1 -- 172.21.15.33:0/457011386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8009a20 con 0x7f01e0154210 2023-11-06T20:59:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01e659c700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0154210 0x7f01e00c28c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f01e0083660 tx=0x7f01d8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01d5ffb700 1 -- 172.21.15.33:0/457011386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8012410 con 0x7f01e0154210 2023-11-06T20:59:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01d5ffb700 1 -- 172.21.15.33:0/457011386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d800b6f0 con 0x7f01e0154210 2023-11-06T20:59:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01e7fff700 1 -- 172.21.15.33:0/457011386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e00c3e10 con 0x7f01e0154210 2023-11-06T20:59:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01d5ffb700 1 -- 172.21.15.33:0/457011386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8011de0 con 0x7f01e0154210 2023-11-06T20:59:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.435+0000 7f01e7fff700 1 -- 172.21.15.33:0/457011386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e014e440 con 0x7f01e0154210 2023-11-06T20:59:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.436+0000 7f01e7fff700 1 -- 172.21.15.33:0/457011386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e00c3c70 con 0x7f01e0154210 2023-11-06T20:59:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.437+0000 7f01d5ffb700 1 -- 172.21.15.33:0/457011386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f01d8003040 con 0x7f01e0154210 2023-11-06T20:59:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.438+0000 7f01d5ffb700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f01c405f270 0x7f01c4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.438+0000 7f01d5ffb700 1 -- 172.21.15.33:0/457011386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f01d807c440 con 0x7f01e0154210 2023-11-06T20:59:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.439+0000 7f01e5d9b700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f01c405f270 0x7f01c4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.439+0000 7f01e5d9b700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f01c405f270 0x7f01c4061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f01dc00c7d0 tx=0x7f01dc009040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.442+0000 7f01d5ffb700 1 -- 172.21.15.33:0/457011386 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f01e00c3c70 con 0x7f01e0154210 2023-11-06T20:59:29.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.657+0000 7f01e7fff700 1 -- 172.21.15.33:0/457011386 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f01e00c3c70 con 0x7f01c405f270 2023-11-06T20:59:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.659+0000 7f01d5ffb700 1 -- 172.21.15.33:0/457011386 <== mgr.5869 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f01e00c3c70 con 0x7f01c405f270 2023-11-06T20:59:29.659 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T20:59:29.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:29.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T20:59:29.660 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:29.660 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T20:59:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 -- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f01c405f270 msgr2=0x7f01c4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:29.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f01c405f270 0x7f01c4061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f01dc00c7d0 tx=0x7f01dc009040).stop 2023-11-06T20:59:29.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 -- 172.21.15.33:0/457011386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0154210 msgr2=0x7f01e00c28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:29.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0154210 0x7f01e00c28c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f01e0083660 tx=0x7f01d8009780).stop 2023-11-06T20:59:29.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 -- 172.21.15.33:0/457011386 shutdown_connections 2023-11-06T20:59:29.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f01c405f270 0x7f01c4061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0154210 0x7f01e00c28c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01e00bfd60 0x7f01e014df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 --2- 172.21.15.33:0/457011386 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01e00c6d90 0x7f01e00c23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 -- 172.21.15.33:0/457011386 >> 172.21.15.33:0/457011386 conn(0x7f01e005e990 msgr2=0x7f01e0152170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 -- 172.21.15.33:0/457011386 shutdown_connections 2023-11-06T20:59:29.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.661+0000 7f01e7fff700 1 -- 172.21.15.33:0/457011386 wait complete. 2023-11-06T20:59:29.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T20:59:29.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.879+0000 7f822b3ff700 1 Processor -- start 2023-11-06T20:59:29.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.879+0000 7f822b3ff700 1 -- start start 2023-11-06T20:59:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.880+0000 7f822b3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8224066620 0x7f82240c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.880+0000 7f822b3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 0x7f8224158780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.880+0000 7f822b3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8224067ad0 0x7f8224067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.880+0000 7f822b3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82240be210 con 0x7f8224067ad0 2023-11-06T20:59:29.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.880+0000 7f822b3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82240a8330 con 0x7f8224066620 2023-11-06T20:59:29.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.880+0000 7f822b3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8224158c70 con 0x7f8224066a00 2023-11-06T20:59:29.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822999c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8224067ad0 0x7f8224067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822999c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8224067ad0 0x7f8224067f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34882/0 (socket says 172.21.15.33:34882) 2023-11-06T20:59:29.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822999c700 1 -- 172.21.15.33:0/530395 learned_addr learned my addr 172.21.15.33:0/530395 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:29.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822899a700 1 --2- 172.21.15.33:0/530395 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 0x7f8224158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822919b700 1 --2- 172.21.15.33:0/530395 >> v2:172.21.15.69:3300/0 conn(0x7f8224066620 0x7f82240c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822999c700 1 -- 172.21.15.33:0/530395 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 msgr2=0x7f8224158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:29.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822999c700 1 --2- 172.21.15.33:0/530395 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 0x7f8224158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822999c700 1 -- 172.21.15.33:0/530395 >> v2:172.21.15.69:3300/0 conn(0x7f8224066620 msgr2=0x7f82240c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:29.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822999c700 1 --2- 172.21.15.33:0/530395 >> v2:172.21.15.69:3300/0 conn(0x7f8224066620 0x7f82240c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822999c700 1 -- 172.21.15.33:0/530395 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82240bdc50 con 0x7f8224067ad0 2023-11-06T20:59:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.881+0000 7f822999c700 1 --2- 172.21.15.33:0/530395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224067ad0 0x7f8224067f60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f82240631e0 tx=0x7f821c009840).ready entity=mon.0 client_cookie=a2a4affd49ac0a27 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f821bfff700 1 -- 172.21.15.33:0/530395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c00e070 con 0x7f8224067ad0 2023-11-06T20:59:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f821bfff700 1 -- 172.21.15.33:0/530395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c00b970 con 0x7f8224067ad0 2023-11-06T20:59:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f821bfff700 1 -- 172.21.15.33:0/530395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c00bcf0 con 0x7f8224067ad0 2023-11-06T20:59:29.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f822b3ff700 1 -- 172.21.15.33:0/530395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224067ad0 msgr2=0x7f8224067f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f822b3ff700 1 --2- 172.21.15.33:0/530395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224067ad0 0x7f8224067f60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f82240631e0 tx=0x7f821c009840).stop 2023-11-06T20:59:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f822b3ff700 1 -- 172.21.15.33:0/530395 shutdown_connections 2023-11-06T20:59:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f822b3ff700 1 --2- 172.21.15.33:0/530395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224067ad0 0x7f8224067f60 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f822b3ff700 1 --2- 172.21.15.33:0/530395 >> v2:172.21.15.69:3300/0 conn(0x7f8224066620 0x7f82240c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f822b3ff700 1 --2- 172.21.15.33:0/530395 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 0x7f8224158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f822b3ff700 1 -- 172.21.15.33:0/530395 >> 172.21.15.33:0/530395 conn(0x7f82240c8c50 msgr2=0x7f82240c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f822b3ff700 1 -- 172.21.15.33:0/530395 shutdown_connections 2023-11-06T20:59:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.882+0000 7f822b3ff700 1 -- 172.21.15.33:0/530395 wait complete. 2023-11-06T20:59:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.884+0000 7f822b3ff700 1 Processor -- start 2023-11-06T20:59:29.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.885+0000 7f822b3ff700 1 -- start start 2023-11-06T20:59:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822b3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224066620 0x7f8224154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822b3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 0x7f82240c0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822b3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8224067ad0 0x7f8224157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822b3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8224155420 con 0x7f8224066620 2023-11-06T20:59:29.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822b3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82240ac370 con 0x7f8224067ad0 2023-11-06T20:59:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822b3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8224154900 con 0x7f8224066a00 2023-11-06T20:59:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224066620 0x7f8224154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224066620 0x7f8224154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34888/0 (socket says 172.21.15.33:34888) 2023-11-06T20:59:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822919b700 1 -- 172.21.15.33:0/3712596399 learned_addr learned my addr 172.21.15.33:0/3712596399 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:29.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822899a700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 0x7f82240c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.886+0000 7f822999c700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8224067ad0 0x7f8224157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f822919b700 1 -- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 msgr2=0x7f82240c0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f822919b700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 0x7f82240c0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f822919b700 1 -- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8224067ad0 msgr2=0x7f8224157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:29.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f822919b700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8224067ad0 0x7f8224157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f822919b700 1 -- 172.21.15.33:0/3712596399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c009ae0 con 0x7f8224066620 2023-11-06T20:59:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f822999c700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8224067ad0 0x7f8224157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f822899a700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 0x7f82240c0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f822919b700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224066620 0x7f8224154410 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f822000e930 tx=0x7f8220009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:29.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f82197fa700 1 -- 172.21.15.33:0/3712596399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822000c640 con 0x7f8224066620 2023-11-06T20:59:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f82197fa700 1 -- 172.21.15.33:0/3712596399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822000c7a0 con 0x7f8224066620 2023-11-06T20:59:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f822b3ff700 1 -- 172.21.15.33:0/3712596399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82240c0680 con 0x7f8224066620 2023-11-06T20:59:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f82197fa700 1 -- 172.21.15.33:0/3712596399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822000cb20 con 0x7f8224066620 2023-11-06T20:59:29.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.887+0000 7f822b3ff700 1 -- 172.21.15.33:0/3712596399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8224154c30 con 0x7f8224066620 2023-11-06T20:59:29.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.888+0000 7f822b3ff700 1 -- 172.21.15.33:0/3712596399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82240ac650 con 0x7f8224066620 2023-11-06T20:59:29.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.889+0000 7f82197fa700 1 -- 172.21.15.33:0/3712596399 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f822001b400 con 0x7f8224066620 2023-11-06T20:59:29.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.890+0000 7f82197fa700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f820805f240 0x7f8208061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:29.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.890+0000 7f82197fa700 1 -- 172.21.15.33:0/3712596399 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f822007a840 con 0x7f8224066620 2023-11-06T20:59:29.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.891+0000 7f822899a700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f820805f240 0x7f8208061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.892+0000 7f822899a700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f820805f240 0x7f8208061700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f82240c8560 tx=0x7f8214008040).ready entity=mgr.5869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:29.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:29.894+0000 7f82197fa700 1 -- 172.21.15.33:0/3712596399 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f822004b870 con 0x7f8224066620 2023-11-06T20:59:30.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.123+0000 7f822b3ff700 1 -- 172.21.15.33:0/3712596399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f82240bf950 con 0x7f8224066620 2023-11-06T20:59:30.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.610+0000 7f82197fa700 1 -- 172.21.15.33:0/3712596399 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 25) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f82200783d0 con 0x7f8224066620 2023-11-06T20:59:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.612+0000 7f82197fa700 1 -- 172.21.15.33:0/3712596399 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v25) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8220071d30 con 0x7f8224066620 2023-11-06T20:59:30.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 -- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f820805f240 msgr2=0x7f8208061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:30.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f820805f240 0x7f8208061700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f82240c8560 tx=0x7f8214008040).stop 2023-11-06T20:59:30.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 -- 172.21.15.33:0/3712596399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224066620 msgr2=0x7f8224154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:30.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224066620 0x7f8224154410 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f822000e930 tx=0x7f8220009060).stop 2023-11-06T20:59:30.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 -- 172.21.15.33:0/3712596399 shutdown_connections 2023-11-06T20:59:30.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f820805f240 0x7f8208061700 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8224067ad0 0x7f8224157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8224066620 0x7f8224154410 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 --2- 172.21.15.33:0/3712596399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8224066a00 0x7f82240c0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 -- 172.21.15.33:0/3712596399 >> 172.21.15.33:0/3712596399 conn(0x7f82240c8c50 msgr2=0x7f822414e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 -- 172.21.15.33:0/3712596399 shutdown_connections 2023-11-06T20:59:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.615+0000 7f822b3ff700 1 -- 172.21.15.33:0/3712596399 wait complete. 2023-11-06T20:59:30.635 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T20:59:30.827 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:30.826+0000 7f2a8f3ac680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:59:30.831 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:30.831+0000 7fb840582680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:59:30.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.841+0000 7f7a48dff700 1 Processor -- start 2023-11-06T20:59:30.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.841+0000 7f7a48dff700 1 -- start start 2023-11-06T20:59:30.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.842+0000 7f7a48dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a44154710 0x7f7a440c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:30.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.842+0000 7f7a48dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a440c7ff0 0x7f7a440646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:30.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.842+0000 7f7a48dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a440c6b60 con 0x7f7a44154330 2023-11-06T20:59:30.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.842+0000 7f7a48dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a440c7700 con 0x7f7a440c7ff0 2023-11-06T20:59:30.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.842+0000 7f7a48dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440c5cd0 con 0x7f7a44154710 2023-11-06T20:59:30.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a42d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a440c7ff0 0x7f7a440646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:30.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a4259c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f7a44154330 0x7f7a440c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:59974/0 (socket says 172.21.15.33:59974) 2023-11-06T20:59:30.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a4259c700 1 -- 172.21.15.33:0/990528424 learned_addr learned my addr 172.21.15.33:0/990528424 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:30.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a41d9b700 1 --2- 172.21.15.33:0/990528424 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a44154710 0x7f7a440c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:30.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a4159a700 1 -- 172.21.15.33:0/990528424 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419809279 0 0) 0x7f7a440c6b60 con 0x7f7a44154330 2023-11-06T20:59:30.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a42d9d700 1 -- 172.21.15.33:0/990528424 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a44154710 msgr2=0x7f7a440c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:30.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a42d9d700 1 --2- 172.21.15.33:0/990528424 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a44154710 0x7f7a440c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:30.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a42d9d700 1 -- 172.21.15.33:0/990528424 >> v1:172.21.15.69:6789/0 conn(0x7f7a44154330 legacy=0x7f7a440c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:30.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a42d9d700 1 -- 172.21.15.33:0/990528424 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a440c5f50 con 0x7f7a440c7ff0 2023-11-06T20:59:30.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a42d9d700 1 --2- 172.21.15.33:0/990528424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440c7ff0 0x7f7a440646c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f7a3c009e60 tx=0x7f7a3c009780).ready entity=mon.0 client_cookie=432714b668e66a16 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:30.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a4159a700 1 -- 172.21.15.33:0/990528424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00e070 con 0x7f7a440c7ff0 2023-11-06T20:59:30.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.843+0000 7f7a4159a700 1 -- 172.21.15.33:0/990528424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c00bb60 con 0x7f7a440c7ff0 2023-11-06T20:59:30.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.844+0000 7f7a4159a700 1 -- 172.21.15.33:0/990528424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c012410 con 0x7f7a440c7ff0 2023-11-06T20:59:30.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.844+0000 7f7a48dff700 1 -- 172.21.15.33:0/990528424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440c7ff0 msgr2=0x7f7a440646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:30.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.844+0000 7f7a48dff700 1 --2- 172.21.15.33:0/990528424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440c7ff0 0x7f7a440646c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f7a3c009e60 tx=0x7f7a3c009780).stop 2023-11-06T20:59:30.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.844+0000 7f7a48dff700 1 -- 172.21.15.33:0/990528424 shutdown_connections 2023-11-06T20:59:30.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.844+0000 7f7a48dff700 1 --2- 172.21.15.33:0/990528424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440c7ff0 0x7f7a440646c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:30.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.844+0000 7f7a48dff700 1 --2- 172.21.15.33:0/990528424 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a44154710 0x7f7a440c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:30.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.844+0000 7f7a48dff700 1 -- 172.21.15.33:0/990528424 >> 172.21.15.33:0/990528424 conn(0x7f7a440cb910 msgr2=0x7f7a440c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:30.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.844+0000 7f7a48dff700 1 -- 172.21.15.33:0/990528424 shutdown_connections 2023-11-06T20:59:30.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.844+0000 7f7a48dff700 1 -- 172.21.15.33:0/990528424 wait complete. 2023-11-06T20:59:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.847+0000 7f7a48dff700 1 Processor -- start 2023-11-06T20:59:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.847+0000 7f7a48dff700 1 -- start start 2023-11-06T20:59:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.848+0000 7f7a48dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a440c7ff0 0x7f7a440a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.848+0000 7f7a48dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a44154330 0x7f7a44152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:30.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.848+0000 7f7a48dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a44154710 0x7f7a44156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:30.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.848+0000 7f7a48dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440ae1e0 con 0x7f7a44154330 2023-11-06T20:59:30.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.848+0000 7f7a48dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a44157760 con 0x7f7a440c7ff0 2023-11-06T20:59:30.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a48dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a44157a40 con 0x7f7a44154710 2023-11-06T20:59:30.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a41d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a44154330 0x7f7a44152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:30.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a4259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a440c7ff0 0x7f7a440a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:30.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a41d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a44154330 0x7f7a44152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34916/0 (socket says 172.21.15.33:34916) 2023-11-06T20:59:30.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a41d9b700 1 -- 172.21.15.33:0/3635781122 learned_addr learned my addr 172.21.15.33:0/3635781122 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:30.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a42d9d700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a44154710 0x7f7a44156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:30.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a41d9b700 1 -- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a44154710 msgr2=0x7f7a44156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:30.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a41d9b700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a44154710 0x7f7a44156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:30.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a41d9b700 1 -- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a440c7ff0 msgr2=0x7f7a440a9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:30.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a41d9b700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a440c7ff0 0x7f7a440a9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:30.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a41d9b700 1 -- 172.21.15.33:0/3635781122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c009a20 con 0x7f7a44154330 2023-11-06T20:59:30.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.849+0000 7f7a41d9b700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a44154330 0x7f7a44152bc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f7a3800e900 tx=0x7f7a38009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:30.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.850+0000 7f7a327fc700 1 -- 172.21.15.33:0/3635781122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38012600 con 0x7f7a44154330 2023-11-06T20:59:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.850+0000 7f7a327fc700 1 -- 172.21.15.33:0/3635781122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a38012760 con 0x7f7a44154330 2023-11-06T20:59:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.850+0000 7f7a327fc700 1 -- 172.21.15.33:0/3635781122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38012ae0 con 0x7f7a44154330 2023-11-06T20:59:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.850+0000 7f7a48dff700 1 -- 172.21.15.33:0/3635781122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a44157d20 con 0x7f7a44154330 2023-11-06T20:59:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.850+0000 7f7a48dff700 1 -- 172.21.15.33:0/3635781122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a44157390 con 0x7f7a44154330 2023-11-06T20:59:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.851+0000 7f7a48dff700 1 -- 172.21.15.33:0/3635781122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a440ae350 con 0x7f7a44154330 2023-11-06T20:59:30.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.852+0000 7f7a327fc700 1 -- 172.21.15.33:0/3635781122 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f7a3801b400 con 0x7f7a44154330 2023-11-06T20:59:30.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.853+0000 7f7a327fc700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a2405f2c0 0x7f7a24061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:30.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.854+0000 7f7a327fc700 1 -- 172.21.15.33:0/3635781122 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7a3807a8e0 con 0x7f7a44154330 2023-11-06T20:59:30.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.854+0000 7f7a4259c700 1 -- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a2405f2c0 msgr2=0x7f7a24061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:30.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.854+0000 7f7a4259c700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a2405f2c0 0x7f7a24061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:30.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:30.857+0000 7f7a327fc700 1 -- 172.21.15.33:0/3635781122 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7a3804b910 con 0x7f7a44154330 2023-11-06T20:59:30.908 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:30.907+0000 7f2a8f3ac680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:59:30.917 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:30.917+0000 7fb840582680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:59:31.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.043+0000 7f7a48dff700 1 -- 172.21.15.33:0/3635781122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7a440652d0 con 0x7f7a44154330 2023-11-06T20:59:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.047+0000 7f7a327fc700 1 -- 172.21.15.33:0/3635781122 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7a38016070 con 0x7f7a44154330 2023-11-06T20:59:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 -- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a2405f2c0 msgr2=0x7f7a24061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a2405f2c0 0x7f7a24061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 -- 172.21.15.33:0/3635781122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a44154330 msgr2=0x7f7a44152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a44154330 0x7f7a44152bc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f7a3800e900 tx=0x7f7a38009060).stop 2023-11-06T20:59:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 -- 172.21.15.33:0/3635781122 shutdown_connections 2023-11-06T20:59:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a2405f2c0 0x7f7a24061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a44154710 0x7f7a44156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a440c7ff0 0x7f7a440a9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 --2- 172.21.15.33:0/3635781122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a44154330 0x7f7a44152bc0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 -- 172.21.15.33:0/3635781122 >> 172.21.15.33:0/3635781122 conn(0x7f7a440cb910 msgr2=0x7f7a440ab280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 -- 172.21.15.33:0/3635781122 shutdown_connections 2023-11-06T20:59:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.049+0000 7f7a48dff700 1 -- 172.21.15.33:0/3635781122 wait complete. 2023-11-06T20:59:31.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:59:31.068 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-06T20:59:31.127 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-11-06T20:59:31.237 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:31.237+0000 7f2a8f3ac680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:59:31.280 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:31.280+0000 7fb840582680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:59:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.335+0000 7f3ea6bff700 1 Processor -- start 2023-11-06T20:59:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.335+0000 7f3ea6bff700 1 -- start start 2023-11-06T20:59:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.336+0000 7f3ea6bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3ea0068c20 0x7f3ea0069000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:31.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.336+0000 7f3ea6bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ea005e3c0 0x7f3ea0063950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:31.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.336+0000 7f3ea6bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea00634c0 con 0x7f3ea0063e40 2023-11-06T20:59:31.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.336+0000 7f3ea6bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea00c4600 con 0x7f3ea0068c20 2023-11-06T20:59:31.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.336+0000 7f3ea6bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00628b0 con 0x7f3ea005e3c0 2023-11-06T20:59:31.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3ea519c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3ea0063e40 0x7f3ea00bc220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35490/0 (socket says 172.21.15.33:35490) 2023-11-06T20:59:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3ea519c700 1 -- 172.21.15.33:0/772604726 learned_addr learned my addr 172.21.15.33:0/772604726 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3ea499b700 1 --2- 172.21.15.33:0/772604726 >> v2:172.21.15.69:3300/0 conn(0x7f3ea0068c20 0x7f3ea0069000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/772604726 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ea005e3c0 0x7f3ea0063950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3e9f7fe700 1 -- 172.21.15.33:0/772604726 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913536439 0 0) 0x7f3ea00634c0 con 0x7f3ea0063e40 2023-11-06T20:59:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3e9f7fe700 1 -- 172.21.15.33:0/772604726 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e900033f0 con 0x7f3ea0063e40 2023-11-06T20:59:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3e9ffff700 1 -- 172.21.15.33:0/772604726 >> v2:172.21.15.69:3300/0 conn(0x7f3ea0068c20 msgr2=0x7f3ea0069000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/772604726 >> v2:172.21.15.69:3300/0 conn(0x7f3ea0068c20 0x7f3ea0069000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3e9ffff700 1 -- 172.21.15.33:0/772604726 >> v1:172.21.15.33:6789/0 conn(0x7f3ea0063e40 legacy=0x7f3ea00bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3e9ffff700 1 -- 172.21.15.33:0/772604726 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea0062b90 con 0x7f3ea005e3c0 2023-11-06T20:59:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.337+0000 7f3ea499b700 1 --2- 172.21.15.33:0/772604726 >> v2:172.21.15.69:3300/0 conn(0x7f3ea0068c20 0x7f3ea0069000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.338+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/772604726 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ea005e3c0 0x7f3ea0063950 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f3e8c009760 tx=0x7f3e8c009060).ready entity=mon.2 client_cookie=5d163f314ac5fee4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.338+0000 7f3e9f7fe700 1 -- 172.21.15.33:0/772604726 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c00d4a0 con 0x7f3ea005e3c0 2023-11-06T20:59:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.338+0000 7f3e9f7fe700 1 -- 172.21.15.33:0/772604726 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c00d600 con 0x7f3ea005e3c0 2023-11-06T20:59:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.338+0000 7f3e9f7fe700 1 -- 172.21.15.33:0/772604726 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c00d980 con 0x7f3ea005e3c0 2023-11-06T20:59:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.338+0000 7f3ea6bff700 1 -- 172.21.15.33:0/772604726 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ea005e3c0 msgr2=0x7f3ea0063950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.338+0000 7f3ea6bff700 1 --2- 172.21.15.33:0/772604726 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ea005e3c0 0x7f3ea0063950 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f3e8c009760 tx=0x7f3e8c009060).stop 2023-11-06T20:59:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.338+0000 7f3ea6bff700 1 -- 172.21.15.33:0/772604726 shutdown_connections 2023-11-06T20:59:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.338+0000 7f3ea6bff700 1 --2- 172.21.15.33:0/772604726 >> v2:172.21.15.69:3300/0 conn(0x7f3ea0068c20 0x7f3ea0069000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.338+0000 7f3ea6bff700 1 --2- 172.21.15.33:0/772604726 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ea005e3c0 0x7f3ea0063950 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.338+0000 7f3ea6bff700 1 -- 172.21.15.33:0/772604726 >> 172.21.15.33:0/772604726 conn(0x7f3ea00c7d10 msgr2=0x7f3ea00c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.339+0000 7f3ea6bff700 1 -- 172.21.15.33:0/772604726 shutdown_connections 2023-11-06T20:59:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.339+0000 7f3ea6bff700 1 -- 172.21.15.33:0/772604726 wait complete. 2023-11-06T20:59:31.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.340+0000 7f3ea6bff700 1 Processor -- start 2023-11-06T20:59:31.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.340+0000 7f3ea6bff700 1 -- start start 2023-11-06T20:59:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.341+0000 7f3ea6bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea005e3c0 0x7f3ea00a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea6bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ea0063e40 0x7f3ea00a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea6bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea0068c20 0x7f3ea00a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea6bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00a80a0 con 0x7f3ea005e3c0 2023-11-06T20:59:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea6bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00a5ec0 con 0x7f3ea0068c20 2023-11-06T20:59:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea6bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00a52b0 con 0x7f3ea0063e40 2023-11-06T20:59:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea499b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea005e3c0 0x7f3ea00a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea499b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea005e3c0 0x7f3ea00a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34932/0 (socket says 172.21.15.33:34932) 2023-11-06T20:59:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea499b700 1 -- 172.21.15.33:0/2877973526 learned_addr learned my addr 172.21.15.33:0/2877973526 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ea0063e40 0x7f3ea00a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea519c700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea0068c20 0x7f3ea00a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea499b700 1 -- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ea0063e40 msgr2=0x7f3ea00a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea499b700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ea0063e40 0x7f3ea00a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea499b700 1 -- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea0068c20 msgr2=0x7f3ea00a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:31.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea499b700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea0068c20 0x7f3ea00a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:31.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.342+0000 7f3ea499b700 1 -- 172.21.15.33:0/2877973526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940092a0 con 0x7f3ea005e3c0 2023-11-06T20:59:31.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.343+0000 7f3ea499b700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea005e3c0 0x7f3ea00a6350 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f3e9400e980 tx=0x7f3e94009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:31.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.343+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94012600 con 0x7f3ea005e3c0 2023-11-06T20:59:31.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.343+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e94012760 con 0x7f3ea005e3c0 2023-11-06T20:59:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.343+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94012ae0 con 0x7f3ea005e3c0 2023-11-06T20:59:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.343+0000 7f3ea6bff700 1 -- 172.21.15.33:0/2877973526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c0092a0 con 0x7f3ea005e3c0 2023-11-06T20:59:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.343+0000 7f3ea6bff700 1 -- 172.21.15.33:0/2877973526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea00a5870 con 0x7f3ea005e3c0 2023-11-06T20:59:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.345+0000 7f3ea6bff700 1 -- 172.21.15.33:0/2877973526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea00a61a0 con 0x7f3ea005e3c0 2023-11-06T20:59:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.345+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f3e9401b400 con 0x7f3ea005e3c0 2023-11-06T20:59:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.346+0000 7f3e9cff9700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 0x7f3e84061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.346+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3e9407a8c0 con 0x7f3ea005e3c0 2023-11-06T20:59:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.346+0000 7f3e9ffff700 1 -- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 msgr2=0x7f3e84061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.346+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 0x7f3e84061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.350+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e9404b8f0 con 0x7f3ea005e3c0 2023-11-06T20:59:31.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.547+0000 7f3e9ffff700 1 -- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 msgr2=0x7f3e84061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:31.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.547+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 0x7f3e84061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:31.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.561+0000 7f3ea6bff700 1 -- 172.21.15.33:0/2877973526 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ea00c1f10 con 0x7f3e8405f2c0 2023-11-06T20:59:31.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.948+0000 7f3e9ffff700 1 -- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 msgr2=0x7f3e84061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:31.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:31.948+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 0x7f3e84061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T20:59:31.979 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:31.979+0000 7f2a8f3ac680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:59:32.063 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:32.062+0000 7fb840582680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:59:32.063 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:32.063+0000 7f2a8f3ac680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:59:32.149 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:32.149+0000 7fb840582680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:59:32.295 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:32.295+0000 7f2a8f3ac680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:59:32.389 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:32.389+0000 7fb840582680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:59:32.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:32.749+0000 7f3e9ffff700 1 -- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 msgr2=0x7f3e84061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:32.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:32.749+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 0x7f3e84061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T20:59:32.805 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:32.805+0000 7f2a8f3ac680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:59:32.919 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:32.919+0000 7fb840582680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:59:32.929 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:32.929+0000 7f2a8f3ac680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:33.050 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:33.049+0000 7fb840582680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:33.113 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:33.113+0000 7f2a8f3ac680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:33.207 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:33.207+0000 7f2a8f3ac680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:59:33.240 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:33.240+0000 7fb840582680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:33.293 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:33.293+0000 7f2a8f3ac680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:59:33.338 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:33.338+0000 7fb840582680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:59:33.427 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:33.426+0000 7fb840582680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:59:33.603 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:33.604+0000 7f2a8f3ac680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:59:33.752 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:33.752+0000 7fb840582680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:59:33.827 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:33.828+0000 7f2a8f3ac680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:33.982 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:33.982+0000 7fb840582680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:34.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:34.350+0000 7f3e9ffff700 1 -- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 msgr2=0x7f3e84061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:34.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:34.350+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 0x7f3e84061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T20:59:34.631 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:34.631+0000 7f2a8f3ac680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:59:34.718 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:34.718+0000 7f2a8f3ac680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:59:34.812 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:34.812+0000 7f2a8f3ac680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:59:34.820 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:34.820+0000 7fb840582680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:59:34.911 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:34.911+0000 7fb840582680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:59:34.994 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:34.994+0000 7f2a8f3ac680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:59:35.009 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:35.009+0000 7fb840582680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:59:35.083 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.083+0000 7f2a8f3ac680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:59:35.201 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:35.201+0000 7fb840582680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:59:35.226 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.226+0000 7f2a8f3ac680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:59:35.294 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:35.293+0000 7fb840582680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:59:35.355 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.355+0000 7f2a8f3ac680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:35.441 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:35.441+0000 7fb840582680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:59:35.576 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:35.576+0000 7fb840582680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:35.700 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.700+0000 7f2a8f3ac680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:59:35.789 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.789+0000 7f2a8f3ac680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:59:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:35.799+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 26) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f3e94078450 con 0x7f3ea005e3c0 2023-11-06T20:59:35.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:35.799+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 msgr2=0x7f3e84061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:35.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:35.799+0000 7f3e9cff9700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 0x7f3e84061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:35.874 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.874+0000 7f2a01ec1700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:35.875 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.874+0000 7f2a01ec1700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:35.875 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.874+0000 7f2a01ec1700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:35.875 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.874+0000 7f2a01ec1700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:35.876 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.874+0000 7f2a01ec1700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:35.876 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.875+0000 7f2a04ec7700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:35.876 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.875+0000 7f2a04ec7700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:35.877 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.875+0000 7f2a04ec7700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:35.877 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.875+0000 7f2a04ec7700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:35.877 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:35.875+0000 7f2a04ec7700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:35.940 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:35.939+0000 7fb840582680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:59:36.033 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:36.033+0000 7fb840582680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:59:36.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:36.802+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 27) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f3e940787d0 con 0x7f3ea005e3c0 2023-11-06T20:59:36.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:36.803+0000 7f3e9cff9700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 0x7f3e84061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:36.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:36.803+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ea00c1f10 con 0x7f3e8405f2c0 2023-11-06T20:59:36.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:36.805+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 0x7f3e84061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:36.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:36.808+0000 7f3e9ffff700 1 --2- 172.21.15.33:0/2877973526 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3e8405f2c0 0x7f3e84061700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f3e8c009760 tx=0x7f3e8c016000).ready entity=mgr.6223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:36.811 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:36.812+0000 7f2a0c6d6700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-11-06T20:59:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:36.811+0000 7f3e9cff9700 1 -- 172.21.15.33:0/2877973526 <== mgr.6223 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f3ea00c1f10 con 0x7f3e8405f2c0 2023-11-06T20:59:36.814 INFO:teuthology.orchestra.run.smithi033.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-11-06T20:59:36.822 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-06T20:59:36.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:59:36.823 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-06T20:59:36.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T20:59:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.081+0000 7f168b5ff700 1 Processor -- start 2023-11-06T20:59:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.082+0000 7f168b5ff700 1 -- start start 2023-11-06T20:59:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.083+0000 7f168b5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1684157700 0x7f16840c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.083+0000 7f168b5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1684158c00 con 0x7f1684063840 2023-11-06T20:59:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.083+0000 7f168b5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1684157ff0 con 0x7f16840aaf50 2023-11-06T20:59:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.083+0000 7f168b5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16840c18f0 con 0x7f1684157700 2023-11-06T20:59:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.083+0000 7f168939b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1684157700 0x7f16840c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.083+0000 7f168939b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1684157700 0x7f16840c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34948/0 (socket says 172.21.15.33:34948) 2023-11-06T20:59:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.083+0000 7f168939b700 1 -- 172.21.15.33:0/42191114 learned_addr learned my addr 172.21.15.33:0/42191114 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.083+0000 7f1689b9c700 1 --1- 172.21.15.33:0/42191114 >> v1:172.21.15.33:6789/0 conn(0x7f1684063840 0x7f16840c5d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35506/0 (socket says 172.21.15.33:35506) 2023-11-06T20:59:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.084+0000 7f168939b700 1 -- 172.21.15.33:0/42191114 >> v1:172.21.15.69:6789/0 conn(0x7f16840aaf50 legacy=0x7f1684063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.084+0000 7f168939b700 1 -- 172.21.15.33:0/42191114 >> v1:172.21.15.33:6789/0 conn(0x7f1684063840 legacy=0x7f16840c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.084+0000 7f168939b700 1 -- 172.21.15.33:0/42191114 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16841582d0 con 0x7f1684157700 2023-11-06T20:59:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.084+0000 7f168939b700 1 --2- 172.21.15.33:0/42191114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1684157700 0x7f16840c6f00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f1680006fd0 tx=0x7f16800090f0).ready entity=mon.0 client_cookie=7813943a3a848bd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.084+0000 7f167bfff700 1 -- 172.21.15.33:0/42191114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168000b600 con 0x7f1684157700 2023-11-06T20:59:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.084+0000 7f167bfff700 1 -- 172.21.15.33:0/42191114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168000b760 con 0x7f1684157700 2023-11-06T20:59:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.084+0000 7f167bfff700 1 -- 172.21.15.33:0/42191114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168000bae0 con 0x7f1684157700 2023-11-06T20:59:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.085+0000 7f168b5ff700 1 -- 172.21.15.33:0/42191114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1684157700 msgr2=0x7f16840c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.085+0000 7f168b5ff700 1 --2- 172.21.15.33:0/42191114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1684157700 0x7f16840c6f00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f1680006fd0 tx=0x7f16800090f0).stop 2023-11-06T20:59:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.085+0000 7f168b5ff700 1 -- 172.21.15.33:0/42191114 shutdown_connections 2023-11-06T20:59:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.085+0000 7f168b5ff700 1 --2- 172.21.15.33:0/42191114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1684157700 0x7f16840c6f00 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.085+0000 7f168b5ff700 1 -- 172.21.15.33:0/42191114 >> 172.21.15.33:0/42191114 conn(0x7f168405ec80 msgr2=0x7f16840cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.085+0000 7f168b5ff700 1 -- 172.21.15.33:0/42191114 shutdown_connections 2023-11-06T20:59:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.085+0000 7f168b5ff700 1 -- 172.21.15.33:0/42191114 wait complete. 2023-11-06T20:59:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.086+0000 7f168b5ff700 1 Processor -- start 2023-11-06T20:59:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.086+0000 7f168b5ff700 1 -- start start 2023-11-06T20:59:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.087+0000 7f168b5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1684063840 0x7f16840adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.087+0000 7f168b5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16840aaf50 0x7f1684151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:37.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.087+0000 7f168b5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1684157700 0x7f1684154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:37.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.087+0000 7f168b5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16840b0ed0 con 0x7f16840aaf50 2023-11-06T20:59:37.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.087+0000 7f168b5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1684165c80 con 0x7f1684157700 2023-11-06T20:59:37.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.087+0000 7f168b5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1684165710 con 0x7f1684063840 2023-11-06T20:59:37.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1688b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16840aaf50 0x7f1684151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:37.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1688b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16840aaf50 0x7f1684151e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34958/0 (socket says 172.21.15.33:34958) 2023-11-06T20:59:37.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1688b9a700 1 -- 172.21.15.33:0/2541734640 learned_addr learned my addr 172.21.15.33:0/2541734640 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:37.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f168939b700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1684063840 0x7f16840adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:37.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1689b9c700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1684157700 0x7f1684154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:37.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1688b9a700 1 -- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1684063840 msgr2=0x7f16840adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:37.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1688b9a700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1684063840 0x7f16840adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:37.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1688b9a700 1 -- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1684157700 msgr2=0x7f1684154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:37.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1688b9a700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1684157700 0x7f1684154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:37.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1688b9a700 1 -- 172.21.15.33:0/2541734640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1680009370 con 0x7f16840aaf50 2023-11-06T20:59:37.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1689b9c700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1684157700 0x7f1684154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:37.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f1688b9a700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16840aaf50 0x7f1684151e60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f167400c420 tx=0x7f1674007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:37.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.088+0000 7f16797fa700 1 -- 172.21.15.33:0/2541734640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674007020 con 0x7f16840aaf50 2023-11-06T20:59:37.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.089+0000 7f16797fa700 1 -- 172.21.15.33:0/2541734640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1674007180 con 0x7f16840aaf50 2023-11-06T20:59:37.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.089+0000 7f168b5ff700 1 -- 172.21.15.33:0/2541734640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16840ae2d0 con 0x7f16840aaf50 2023-11-06T20:59:37.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.089+0000 7f16797fa700 1 -- 172.21.15.33:0/2541734640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674007540 con 0x7f16840aaf50 2023-11-06T20:59:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.089+0000 7f168b5ff700 1 -- 172.21.15.33:0/2541734640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1684153500 con 0x7f16840aaf50 2023-11-06T20:59:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.090+0000 7f168b5ff700 1 -- 172.21.15.33:0/2541734640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168406c6c0 con 0x7f16840aaf50 2023-11-06T20:59:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.091+0000 7f16797fa700 1 -- 172.21.15.33:0/2541734640 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f16740068b0 con 0x7f16840aaf50 2023-11-06T20:59:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.092+0000 7f16797fa700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f166805f080 0x7f1668061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.092+0000 7f16797fa700 1 -- 172.21.15.33:0/2541734640 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f16740795a0 con 0x7f16840aaf50 2023-11-06T20:59:37.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.092+0000 7f168939b700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f166805f080 0x7f1668061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:37.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.093+0000 7f168939b700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f166805f080 0x7f1668061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1680009130 tx=0x7f1680017000).ready entity=mgr.6223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:37.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.096+0000 7f16797fa700 1 -- 172.21.15.33:0/2541734640 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1674047af0 con 0x7f16840aaf50 2023-11-06T20:59:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.329+0000 7f168b5ff700 1 -- 172.21.15.33:0/2541734640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f16840bfe50 con 0x7f16840aaf50 2023-11-06T20:59:37.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.985+0000 7f16797fa700 1 -- 172.21.15.33:0/2541734640 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f1674077330 con 0x7f16840aaf50 2023-11-06T20:59:37.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.986+0000 7f16797fa700 1 -- 172.21.15.33:0/2541734640 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v28) v1 ==== 93+0+0 (secure 0 0 0) 0x7f1674074c20 con 0x7f16840aaf50 2023-11-06T20:59:37.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 -- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f166805f080 msgr2=0x7f1668061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:37.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f166805f080 0x7f1668061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1680009130 tx=0x7f1680017000).stop 2023-11-06T20:59:37.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 -- 172.21.15.33:0/2541734640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16840aaf50 msgr2=0x7f1684151e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:37.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16840aaf50 0x7f1684151e60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f167400c420 tx=0x7f1674007e00).stop 2023-11-06T20:59:37.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 -- 172.21.15.33:0/2541734640 shutdown_connections 2023-11-06T20:59:37.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f166805f080 0x7f1668061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:37.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1684063840 0x7f16840adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:37.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1684157700 0x7f1684154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:37.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 --2- 172.21.15.33:0/2541734640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16840aaf50 0x7f1684151e60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:37.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 -- 172.21.15.33:0/2541734640 >> 172.21.15.33:0/2541734640 conn(0x7f168405ec80 msgr2=0x7f16840c0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:37.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 -- 172.21.15.33:0/2541734640 shutdown_connections 2023-11-06T20:59:37.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:37.990+0000 7f168b5ff700 1 -- 172.21.15.33:0/2541734640 wait complete. 2023-11-06T20:59:38.012 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T20:59:38.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:59:38.124 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:59:38.205 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:38.205+0000 7f7045024680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:59:38.206 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:38.206+0000 7f40d8e46680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T20:59:38.209 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T20:59:38.210 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T20:59:38.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T20:59:38.211 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T20:59:38.291 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:38.287+0000 7f40d8e46680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:59:38.293 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:38.293+0000 7f7045024680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T20:59:38.294 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T20:59:38.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T20:59:38.295 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T20:59:38.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:38.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:59:38.541 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T20:59:38.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.544+0000 7f6d68bff700 1 Processor -- start 2023-11-06T20:59:38.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.544+0000 7f6d68bff700 1 -- start start 2023-11-06T20:59:38.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.545+0000 7f6d68bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d640c1840 0x7f6d640a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:38.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.545+0000 7f6d68bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640aa2d0 0x7f6d640c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:38.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.545+0000 7f6d68bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d641554a0 con 0x7f6d640c6240 2023-11-06T20:59:38.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.545+0000 7f6d68bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d64154960 con 0x7f6d640c1840 2023-11-06T20:59:38.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.545+0000 7f6d68bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d64155930 con 0x7f6d640aa2d0 2023-11-06T20:59:38.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.546+0000 7f6d6259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d640c1840 0x7f6d640a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:38.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.546+0000 7f6d62d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6d640c6240 0x7f6d64157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40604/0 (socket says 172.21.15.33:40604) 2023-11-06T20:59:38.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.546+0000 7f6d62d9d700 1 -- 172.21.15.33:0/3747433500 learned_addr learned my addr 172.21.15.33:0/3747433500 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:38.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.546+0000 7f6d61d9b700 1 --2- 172.21.15.33:0/3747433500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640aa2d0 0x7f6d640c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:38.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.546+0000 7f6d6259c700 1 -- 172.21.15.33:0/3747433500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640aa2d0 msgr2=0x7f6d640c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:38.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.546+0000 7f6d6259c700 1 --2- 172.21.15.33:0/3747433500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640aa2d0 0x7f6d640c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:38.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.546+0000 7f6d6259c700 1 -- 172.21.15.33:0/3747433500 >> v1:172.21.15.33:6789/0 conn(0x7f6d640c6240 legacy=0x7f6d64157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:38.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.546+0000 7f6d6259c700 1 -- 172.21.15.33:0/3747433500 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640c66b0 con 0x7f6d640c1840 2023-11-06T20:59:38.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.546+0000 7f6d6259c700 1 --2- 172.21.15.33:0/3747433500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d640c1840 0x7f6d640a9de0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f6d54009760 tx=0x7f6d54009060).ready entity=mon.0 client_cookie=38fe1ef5034d0f94 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:38.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.547+0000 7f6d6159a700 1 -- 172.21.15.33:0/3747433500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400f510 con 0x7f6d640c1840 2023-11-06T20:59:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.547+0000 7f6d6159a700 1 -- 172.21.15.33:0/3747433500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5400f670 con 0x7f6d640c1840 2023-11-06T20:59:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.547+0000 7f6d6159a700 1 -- 172.21.15.33:0/3747433500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400f9f0 con 0x7f6d640c1840 2023-11-06T20:59:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.547+0000 7f6d68bff700 1 -- 172.21.15.33:0/3747433500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d640c1840 msgr2=0x7f6d640a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.547+0000 7f6d68bff700 1 --2- 172.21.15.33:0/3747433500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d640c1840 0x7f6d640a9de0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f6d54009760 tx=0x7f6d54009060).stop 2023-11-06T20:59:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.547+0000 7f6d68bff700 1 -- 172.21.15.33:0/3747433500 shutdown_connections 2023-11-06T20:59:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.547+0000 7f6d68bff700 1 --2- 172.21.15.33:0/3747433500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d640c1840 0x7f6d640a9de0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.547+0000 7f6d68bff700 1 --2- 172.21.15.33:0/3747433500 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640aa2d0 0x7f6d640c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.547+0000 7f6d68bff700 1 -- 172.21.15.33:0/3747433500 >> 172.21.15.33:0/3747433500 conn(0x7f6d6405ec80 msgr2=0x7f6d640cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.547+0000 7f6d68bff700 1 -- 172.21.15.33:0/3747433500 shutdown_connections 2023-11-06T20:59:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.548+0000 7f6d68bff700 1 -- 172.21.15.33:0/3747433500 wait complete. 2023-11-06T20:59:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.550+0000 7f6d68bff700 1 Processor -- start 2023-11-06T20:59:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.550+0000 7f6d68bff700 1 -- start start 2023-11-06T20:59:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d68bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d640aa2d0 0x7f6d640c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d68bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640c1840 0x7f6d640c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d68bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d640c6240 0x7f6d640c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d68bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640c2890 con 0x7f6d640c6240 2023-11-06T20:59:38.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d68bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6414dcb0 con 0x7f6d640aa2d0 2023-11-06T20:59:38.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d68bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640c47d0 con 0x7f6d640c1840 2023-11-06T20:59:38.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d62d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d640c6240 0x7f6d640c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:38.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d61d9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640c1840 0x7f6d640c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:38.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d6259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d640aa2d0 0x7f6d640c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d61d9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640c1840 0x7f6d640c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:37052/0 (socket says 172.21.15.33:37052) 2023-11-06T20:59:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.552+0000 7f6d61d9b700 1 -- 172.21.15.33:0/2372319137 learned_addr learned my addr 172.21.15.33:0/2372319137 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.553+0000 7f6d6259c700 1 -- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640c1840 msgr2=0x7f6d640c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.553+0000 7f6d6259c700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640c1840 0x7f6d640c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.553+0000 7f6d6259c700 1 -- 172.21.15.33:0/2372319137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d640c6240 msgr2=0x7f6d640c23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.553+0000 7f6d6259c700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d640c6240 0x7f6d640c23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.553+0000 7f6d6259c700 1 -- 172.21.15.33:0/2372319137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d540092a0 con 0x7f6d640aa2d0 2023-11-06T20:59:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.553+0000 7f6d6259c700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d640aa2d0 0x7f6d640c3f10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f6d54014000 tx=0x7f6d54017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.553+0000 7f6d61d9b700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640c1840 0x7f6d640c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.553+0000 7f6d527fc700 1 -- 172.21.15.33:0/2372319137 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400f7e0 con 0x7f6d640aa2d0 2023-11-06T20:59:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.553+0000 7f6d527fc700 1 -- 172.21.15.33:0/2372319137 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5400f940 con 0x7f6d640aa2d0 2023-11-06T20:59:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.553+0000 7f6d68bff700 1 -- 172.21.15.33:0/2372319137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d640c4ab0 con 0x7f6d640aa2d0 2023-11-06T20:59:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.554+0000 7f6d527fc700 1 -- 172.21.15.33:0/2372319137 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5401b640 con 0x7f6d640aa2d0 2023-11-06T20:59:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.554+0000 7f6d68bff700 1 -- 172.21.15.33:0/2372319137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d640c5070 con 0x7f6d640aa2d0 2023-11-06T20:59:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.555+0000 7f6d68bff700 1 -- 172.21.15.33:0/2372319137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6406c6c0 con 0x7f6d640aa2d0 2023-11-06T20:59:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.556+0000 7f6d527fc700 1 -- 172.21.15.33:0/2372319137 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f6d5401ba30 con 0x7f6d640aa2d0 2023-11-06T20:59:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.557+0000 7f6d527fc700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d4405f300 0x7f6d440617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.557+0000 7f6d527fc700 1 -- 172.21.15.33:0/2372319137 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f6d5407bfe0 con 0x7f6d640aa2d0 2023-11-06T20:59:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.557+0000 7f6d61d9b700 1 -- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d4405f300 msgr2=0x7f6d440617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.557+0000 7f6d61d9b700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d4405f300 0x7f6d440617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:38.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.566+0000 7f6d527fc700 1 -- 172.21.15.33:0/2372319137 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6d5404ced0 con 0x7f6d640aa2d0 2023-11-06T20:59:38.624 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:38.624+0000 7f40d8e46680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:59:38.666 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:38.666+0000 7f7045024680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T20:59:38.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.758+0000 7f6d61d9b700 1 -- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d4405f300 msgr2=0x7f6d440617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.758+0000 7f6d61d9b700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d4405f300 0x7f6d440617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.796+0000 7f6d68bff700 1 -- 172.21.15.33:0/2372319137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d641652d0 con 0x7f6d640aa2d0 2023-11-06T20:59:38.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.797+0000 7f6d527fc700 1 -- 172.21.15.33:0/2372319137 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4490 (secure 0 0 0) 0x7f6d54006b30 con 0x7f6d640aa2d0 2023-11-06T20:59:38.798 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:38.798 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6055,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1867036074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1867036074},{"type":"v1","addr":"172.21.15.69:6836","nonce":1867036074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6058,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1920115024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1920115024},{"type":"v1","addr":"172.21.15.69:6837","nonce":1920115024}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2935600703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2935600703},{"type":"v1","addr":"172.21.15.33:6835","nonce":2935600703}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:21.575862+0000","modified":"2023-11-06T20:59:24.582786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/3385607254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3385607254},{"type":"v1","addr":"172.21.15.33:6833","nonce":3385607254}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-06T20:59:38.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 -- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d4405f300 msgr2=0x7f6d440617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d4405f300 0x7f6d440617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 -- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d640aa2d0 msgr2=0x7f6d640c3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d640aa2d0 0x7f6d640c3f10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f6d54014000 tx=0x7f6d54017000).stop 2023-11-06T20:59:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 -- 172.21.15.33:0/2372319137 shutdown_connections 2023-11-06T20:59:38.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d4405f300 0x7f6d440617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:38.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d640c6240 0x7f6d640c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:38.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d640aa2d0 0x7f6d640c3f10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:38.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 --2- 172.21.15.33:0/2372319137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d640c1840 0x7f6d640c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:38.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 -- 172.21.15.33:0/2372319137 >> 172.21.15.33:0/2372319137 conn(0x7f6d6405ec80 msgr2=0x7f6d64152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:38.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 -- 172.21.15.33:0/2372319137 shutdown_connections 2023-11-06T20:59:38.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:38.800+0000 7f6d68bff700 1 -- 172.21.15.33:0/2372319137 wait complete. 2023-11-06T20:59:38.803 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2023-11-06T20:59:38.820 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:39.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.019+0000 7f62b25ff700 1 Processor -- start 2023-11-06T20:59:39.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.019+0000 7f62b25ff700 1 -- start start 2023-11-06T20:59:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.021+0000 7f62b25ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 0x7f62ac154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.021+0000 7f62b25ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac0c5db0 con 0x7f62ac0c6dc0 2023-11-06T20:59:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.021+0000 7f62b25ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac0c6930 con 0x7f62ac154280 2023-11-06T20:59:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.021+0000 7f62b25ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0aa630 con 0x7f62ac0ad6e0 2023-11-06T20:59:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.021+0000 7f62b0b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f62ac0c6dc0 0x7f62ac0c7470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40614/0 (socket says 172.21.15.33:40614) 2023-11-06T20:59:39.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.021+0000 7f62b0b9c700 1 -- 172.21.15.33:0/2318603441 learned_addr learned my addr 172.21.15.33:0/2318603441 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:39.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.021+0000 7f62ab7fe700 1 --2- 172.21.15.33:0/2318603441 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 0x7f62ac154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.022+0000 7f62aaffd700 1 -- 172.21.15.33:0/2318603441 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451741798 0 0) 0x7f62ac0c5db0 con 0x7f62ac0c6dc0 2023-11-06T20:59:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.022+0000 7f62aaffd700 1 -- 172.21.15.33:0/2318603441 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62940033f0 con 0x7f62ac0c6dc0 2023-11-06T20:59:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.022+0000 7f62aaffd700 1 -- 172.21.15.33:0/2318603441 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940398595 0 0) 0x7f62ac0c6930 con 0x7f62ac154280 2023-11-06T20:59:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.022+0000 7f62aaffd700 1 -- 172.21.15.33:0/2318603441 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62ac0c5db0 con 0x7f62ac154280 2023-11-06T20:59:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.022+0000 7f62ab7fe700 1 -- 172.21.15.33:0/2318603441 >> v1:172.21.15.69:6789/0 conn(0x7f62ac154280 legacy=0x7f62ac0a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.022+0000 7f62ab7fe700 1 -- 172.21.15.33:0/2318603441 >> v1:172.21.15.33:6789/0 conn(0x7f62ac0c6dc0 legacy=0x7f62ac0c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.022+0000 7f62ab7fe700 1 -- 172.21.15.33:0/2318603441 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac0c6000 con 0x7f62ac0ad6e0 2023-11-06T20:59:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.022+0000 7f62ab7fe700 1 --2- 172.21.15.33:0/2318603441 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 0x7f62ac154660 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f62a0009790 tx=0x7f62a0009060).ready entity=mon.2 client_cookie=c656f1f3d2f4e6d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.023+0000 7f62aaffd700 1 -- 172.21.15.33:0/2318603441 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a000f510 con 0x7f62ac0ad6e0 2023-11-06T20:59:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.023+0000 7f62aaffd700 1 -- 172.21.15.33:0/2318603441 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a000f670 con 0x7f62ac0ad6e0 2023-11-06T20:59:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.023+0000 7f62aaffd700 1 -- 172.21.15.33:0/2318603441 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a000f9f0 con 0x7f62ac0ad6e0 2023-11-06T20:59:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.023+0000 7f62b25ff700 1 -- 172.21.15.33:0/2318603441 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 msgr2=0x7f62ac154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.023+0000 7f62b25ff700 1 --2- 172.21.15.33:0/2318603441 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 0x7f62ac154660 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f62a0009790 tx=0x7f62a0009060).stop 2023-11-06T20:59:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.023+0000 7f62b25ff700 1 -- 172.21.15.33:0/2318603441 shutdown_connections 2023-11-06T20:59:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.023+0000 7f62b25ff700 1 --2- 172.21.15.33:0/2318603441 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 0x7f62ac154660 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.023+0000 7f62b25ff700 1 -- 172.21.15.33:0/2318603441 >> 172.21.15.33:0/2318603441 conn(0x7f62ac05e1b0 msgr2=0x7f62ac05e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.023+0000 7f62b25ff700 1 -- 172.21.15.33:0/2318603441 shutdown_connections 2023-11-06T20:59:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.023+0000 7f62b25ff700 1 -- 172.21.15.33:0/2318603441 wait complete. 2023-11-06T20:59:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.025+0000 7f62b25ff700 1 Processor -- start 2023-11-06T20:59:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.025+0000 7f62b25ff700 1 -- start start 2023-11-06T20:59:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62b25ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 0x7f62ac0c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62b25ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac0c6dc0 0x7f62ac0cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62b25ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ac154280 0x7f62ac14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62b25ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0c3b90 con 0x7f62ac0c6dc0 2023-11-06T20:59:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62b25ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0c3e70 con 0x7f62ac154280 2023-11-06T20:59:39.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62b25ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0c4150 con 0x7f62ac0ad6e0 2023-11-06T20:59:39.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac0c6dc0 0x7f62ac0cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62b0b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ac154280 0x7f62ac14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62abfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 0x7f62ac0c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac0c6dc0 0x7f62ac0cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51500/0 (socket says 172.21.15.33:51500) 2023-11-06T20:59:39.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.026+0000 7f62ab7fe700 1 -- 172.21.15.33:0/1123701952 learned_addr learned my addr 172.21.15.33:0/1123701952 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:39.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f62b0b9c700 1 -- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 msgr2=0x7f62ac0c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f62b0b9c700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 0x7f62ac0c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f62b0b9c700 1 -- 172.21.15.33:0/1123701952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac0c6dc0 msgr2=0x7f62ac0cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f62b0b9c700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac0c6dc0 0x7f62ac0cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f62b0b9c700 1 -- 172.21.15.33:0/1123701952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a00092a0 con 0x7f62ac154280 2023-11-06T20:59:39.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f62ab7fe700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac0c6dc0 0x7f62ac0cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:39.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f62b0b9c700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ac154280 0x7f62ac14f000 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f62a4001460 tx=0x7f62a4007f30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:39.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f6293fff700 1 -- 172.21.15.33:0/1123701952 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4005d30 con 0x7f62ac154280 2023-11-06T20:59:39.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f62b25ff700 1 -- 172.21.15.33:0/1123701952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac0c4430 con 0x7f62ac154280 2023-11-06T20:59:39.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f6293fff700 1 -- 172.21.15.33:0/1123701952 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a40070d0 con 0x7f62ac154280 2023-11-06T20:59:39.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f6293fff700 1 -- 172.21.15.33:0/1123701952 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4007490 con 0x7f62ac154280 2023-11-06T20:59:39.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.027+0000 7f62b25ff700 1 -- 172.21.15.33:0/1123701952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac0c4c10 con 0x7f62ac154280 2023-11-06T20:59:39.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.029+0000 7f62b25ff700 1 -- 172.21.15.33:0/1123701952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ac0c4a80 con 0x7f62ac154280 2023-11-06T20:59:39.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.029+0000 7f6293fff700 1 -- 172.21.15.33:0/1123701952 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f62a4007840 con 0x7f62ac154280 2023-11-06T20:59:39.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.030+0000 7f6293fff700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f629805f300 0x7f62980617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.030+0000 7f6293fff700 1 -- 172.21.15.33:0/1123701952 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f62a4079fa0 con 0x7f62ac154280 2023-11-06T20:59:39.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.030+0000 7f62abfff700 1 -- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f629805f300 msgr2=0x7f62980617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:39.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.030+0000 7f62abfff700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f629805f300 0x7f62980617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:39.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.034+0000 7f6293fff700 1 -- 172.21.15.33:0/1123701952 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f62a404ae70 con 0x7f62ac154280 2023-11-06T20:59:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.231+0000 7f62abfff700 1 -- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f629805f300 msgr2=0x7f62980617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:39.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.231+0000 7f62abfff700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f629805f300 0x7f62980617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.240+0000 7f62b25ff700 1 -- 172.21.15.33:0/1123701952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62ac062ac0 con 0x7f62ac154280 2023-11-06T20:59:39.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.241+0000 7f6293fff700 1 -- 172.21.15.33:0/1123701952 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17346 (secure 0 0 0) 0x7f62a4018030 con 0x7f62ac154280 2023-11-06T20:59:39.242 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:39.243 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:35.792464+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:20.035584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:20.561335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3941224090":"2023-11-07T20:59:35.792423+0000","172.21.15.69:0/1477867129":"2023-11-07T20:59:35.792423+0000","172.21.15.69:6833/114912":"2023-11-07T20:59:35.792423+0000","172.21.15.69:6832/114912":"2023-11-07T20:59:35.792423+0000","172.21.15.69:0/616431469":"2023-11-07T20:59:35.792423+0000","172.21.15.69:0/1238590029":"2023-11-07T20:59:35.792423+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 -- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f629805f300 msgr2=0x7f62980617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:39.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f629805f300 0x7f62980617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:39.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 -- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ac154280 msgr2=0x7f62ac14f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ac154280 0x7f62ac14f000 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f62a4001460 tx=0x7f62a4007f30).stop 2023-11-06T20:59:39.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 -- 172.21.15.33:0/1123701952 shutdown_connections 2023-11-06T20:59:39.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f629805f300 0x7f62980617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:39.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62ac0ad6e0 0x7f62ac0c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ac154280 0x7f62ac14f000 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 --2- 172.21.15.33:0/1123701952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac0c6dc0 0x7f62ac0cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 -- 172.21.15.33:0/1123701952 >> 172.21.15.33:0/1123701952 conn(0x7f62ac05e1b0 msgr2=0x7f62ac153310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:39.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 -- 172.21.15.33:0/1123701952 shutdown_connections 2023-11-06T20:59:39.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.244+0000 7f62b25ff700 1 -- 172.21.15.33:0/1123701952 wait complete. 2023-11-06T20:59:39.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:59:20.035584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:59:20.561335+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T20:59:39.264 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:59:39.379 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:39.379+0000 7f40d8e46680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:59:39.463 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:39.463+0000 7f40d8e46680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:59:39.485 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:39.484+0000 7f7045024680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T20:59:39.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.498+0000 7fe85efff700 1 Processor -- start 2023-11-06T20:59:39.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.498+0000 7fe85efff700 1 -- start start 2023-11-06T20:59:39.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.499+0000 7fe85efff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe85814faa0 0x7fe858064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.499+0000 7fe85efff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe85814fe80 0x7fe858152750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.499+0000 7fe85efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe858152c40 0x7fe8580c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.499+0000 7fe85efff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe858064270 con 0x7fe85814fe80 2023-11-06T20:59:39.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.499+0000 7fe85efff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8580651b0 con 0x7fe85814faa0 2023-11-06T20:59:39.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.499+0000 7fe85efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8581531f0 con 0x7fe858152c40 2023-11-06T20:59:39.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe857fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe85814fe80 0x7fe858152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe85cd9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe85814faa0 0x7fe858064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe857fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe85814fe80 0x7fe858152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51502/0 (socket says 172.21.15.33:51502) 2023-11-06T20:59:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe857fff700 1 -- 172.21.15.33:0/1580135855 learned_addr learned my addr 172.21.15.33:0/1580135855 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe85d59c700 1 --2- 172.21.15.33:0/1580135855 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe858152c40 0x7fe8580c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe85cd9b700 1 -- 172.21.15.33:0/1580135855 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe858152c40 msgr2=0x7fe8580c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe85cd9b700 1 --2- 172.21.15.33:0/1580135855 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe858152c40 0x7fe8580c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe85cd9b700 1 -- 172.21.15.33:0/1580135855 >> v2:172.21.15.33:3300/0 conn(0x7fe85814fe80 msgr2=0x7fe858152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe85cd9b700 1 --2- 172.21.15.33:0/1580135855 >> v2:172.21.15.33:3300/0 conn(0x7fe85814fe80 0x7fe858152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe85cd9b700 1 -- 172.21.15.33:0/1580135855 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8581534d0 con 0x7fe85814faa0 2023-11-06T20:59:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.500+0000 7fe857fff700 1 --2- 172.21.15.33:0/1580135855 >> v2:172.21.15.33:3300/0 conn(0x7fe85814fe80 0x7fe858152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:39.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe85cd9b700 1 --2- 172.21.15.33:0/1580135855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe85814faa0 0x7fe858064cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fe8580676b0 tx=0x7fe84c009060).ready entity=mon.1 client_cookie=d3f1d540040ace64 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe8577fe700 1 -- 172.21.15.33:0/1580135855 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00d4a0 con 0x7fe85814faa0 2023-11-06T20:59:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe8577fe700 1 -- 172.21.15.33:0/1580135855 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c00d600 con 0x7fe85814faa0 2023-11-06T20:59:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe8577fe700 1 -- 172.21.15.33:0/1580135855 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00d980 con 0x7fe85814faa0 2023-11-06T20:59:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe85efff700 1 -- 172.21.15.33:0/1580135855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe85814faa0 msgr2=0x7fe858064cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe85efff700 1 --2- 172.21.15.33:0/1580135855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe85814faa0 0x7fe858064cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fe8580676b0 tx=0x7fe84c009060).stop 2023-11-06T20:59:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe85efff700 1 -- 172.21.15.33:0/1580135855 shutdown_connections 2023-11-06T20:59:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe85efff700 1 --2- 172.21.15.33:0/1580135855 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe858152c40 0x7fe8580c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe85efff700 1 --2- 172.21.15.33:0/1580135855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe85814faa0 0x7fe858064cc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe85efff700 1 --2- 172.21.15.33:0/1580135855 >> v2:172.21.15.33:3300/0 conn(0x7fe85814fe80 0x7fe858152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.501+0000 7fe85efff700 1 -- 172.21.15.33:0/1580135855 >> 172.21.15.33:0/1580135855 conn(0x7fe858062710 msgr2=0x7fe858062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.502+0000 7fe85efff700 1 -- 172.21.15.33:0/1580135855 shutdown_connections 2023-11-06T20:59:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.502+0000 7fe85efff700 1 -- 172.21.15.33:0/1580135855 wait complete. 2023-11-06T20:59:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.503+0000 7fe85efff700 1 Processor -- start 2023-11-06T20:59:39.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.503+0000 7fe85efff700 1 -- start start 2023-11-06T20:59:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.504+0000 7fe85efff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85814faa0 0x7fe8581365c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.504+0000 7fe85efff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe85814fe80 0x7fe8580aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.504+0000 7fe85efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe858152c40 0x7fe8580ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.504+0000 7fe85efff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580ab8a0 con 0x7fe85814faa0 2023-11-06T20:59:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.504+0000 7fe85efff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8581445e0 con 0x7fe85814fe80 2023-11-06T20:59:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.504+0000 7fe85efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8581439d0 con 0x7fe858152c40 2023-11-06T20:59:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.504+0000 7fe85cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85814faa0 0x7fe8581365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.504+0000 7fe85cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85814faa0 0x7fe8581365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51516/0 (socket says 172.21.15.33:51516) 2023-11-06T20:59:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe85cd9b700 1 -- 172.21.15.33:0/1159559674 learned_addr learned my addr 172.21.15.33:0/1159559674 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe857fff700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe85814fe80 0x7fe8580aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe85d59c700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe858152c40 0x7fe8580ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe857fff700 1 -- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe858152c40 msgr2=0x7fe8580ab3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe857fff700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe858152c40 0x7fe8580ab3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe857fff700 1 -- 172.21.15.33:0/1159559674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85814faa0 msgr2=0x7fe8581365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe857fff700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85814faa0 0x7fe8581365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe857fff700 1 -- 172.21.15.33:0/1159559674 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844009370 con 0x7fe85814fe80 2023-11-06T20:59:39.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe85cd9b700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85814faa0 0x7fe8581365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:39.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe857fff700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe85814fe80 0x7fe8580aaec0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fe844006fd0 tx=0x7fe8440090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:39.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe854ff9700 1 -- 172.21.15.33:0/1159559674 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400c6a0 con 0x7fe85814fe80 2023-11-06T20:59:39.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe854ff9700 1 -- 172.21.15.33:0/1159559674 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84400c800 con 0x7fe85814fe80 2023-11-06T20:59:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe85efff700 1 -- 172.21.15.33:0/1159559674 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c0092a0 con 0x7fe85814fe80 2023-11-06T20:59:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.505+0000 7fe854ff9700 1 -- 172.21.15.33:0/1159559674 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400cb80 con 0x7fe85814fe80 2023-11-06T20:59:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.506+0000 7fe85efff700 1 -- 172.21.15.33:0/1159559674 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe858143f90 con 0x7fe85814fe80 2023-11-06T20:59:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.507+0000 7fe85efff700 1 -- 172.21.15.33:0/1159559674 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe858144790 con 0x7fe85814fe80 2023-11-06T20:59:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.507+0000 7fe854ff9700 1 -- 172.21.15.33:0/1159559674 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fe84401b400 con 0x7fe85814fe80 2023-11-06T20:59:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.508+0000 7fe854ff9700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe83c05f300 0x7fe83c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.508+0000 7fe854ff9700 1 -- 172.21.15.33:0/1159559674 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fe84407aa50 con 0x7fe85814fe80 2023-11-06T20:59:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.509+0000 7fe85cd9b700 1 -- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe83c05f300 msgr2=0x7fe83c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.509+0000 7fe85cd9b700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe83c05f300 0x7fe83c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.512+0000 7fe854ff9700 1 -- 172.21.15.33:0/1159559674 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe84404b9c0 con 0x7fe85814fe80 2023-11-06T20:59:39.575 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:39.575+0000 7f7045024680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T20:59:39.695 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:39.695+0000 7f40d8e46680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:59:39.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.695+0000 7fe85efff700 1 -- 172.21.15.33:0/1159559674 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe858063c00 con 0x7fe85814fe80 2023-11-06T20:59:39.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.696+0000 7fe854ff9700 1 -- 172.21.15.33:0/1159559674 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v85) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe84404efe0 con 0x7fe85814fe80 2023-11-06T20:59:39.697 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:59:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 -- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe83c05f300 msgr2=0x7fe83c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe83c05f300 0x7fe83c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 -- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe85814fe80 msgr2=0x7fe8580aaec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe85814fe80 0x7fe8580aaec0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fe844006fd0 tx=0x7fe8440090f0).stop 2023-11-06T20:59:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 -- 172.21.15.33:0/1159559674 shutdown_connections 2023-11-06T20:59:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe83c05f300 0x7fe83c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe858152c40 0x7fe8580ab3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85814faa0 0x7fe8581365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 --2- 172.21.15.33:0/1159559674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe85814fe80 0x7fe8580aaec0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 -- 172.21.15.33:0/1159559674 >> 172.21.15.33:0/1159559674 conn(0x7fe858062710 msgr2=0x7fe8580c33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:39.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 -- 172.21.15.33:0/1159559674 shutdown_connections 2023-11-06T20:59:39.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.698+0000 7fe85efff700 1 -- 172.21.15.33:0/1159559674 wait complete. 2023-11-06T20:59:39.717 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:59:39.824 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:39.824+0000 7f7045024680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T20:59:39.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.917+0000 7f444b1ff700 1 Processor -- start 2023-11-06T20:59:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.917+0000 7f444b1ff700 1 -- start start 2023-11-06T20:59:39.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.918+0000 7f444b1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4444154330 0x7f44440c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.918+0000 7f444b1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f44440c7ff0 0x7f44440646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.918+0000 7f444b1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44440c6b60 con 0x7f4444154710 2023-11-06T20:59:39.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.918+0000 7f444b1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44440c7700 con 0x7f44440c7ff0 2023-11-06T20:59:39.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.918+0000 7f444b1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44440c5cd0 con 0x7f4444154330 2023-11-06T20:59:39.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.919+0000 7f4443fff700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f4444154710 0x7f44440c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:52148/0 (socket says 172.21.15.33:52148) 2023-11-06T20:59:39.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.919+0000 7f444979c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f44440c7ff0 0x7f44440646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.919+0000 7f4448f9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4444154330 0x7f44440c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.919+0000 7f4443fff700 1 -- 172.21.15.33:0/2583761751 learned_addr learned my addr 172.21.15.33:0/2583761751 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.919+0000 7f444979c700 1 -- 172.21.15.33:0/2583761751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4444154330 msgr2=0x7f44440c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.919+0000 7f444979c700 1 --2- 172.21.15.33:0/2583761751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4444154330 0x7f44440c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.919+0000 7f444979c700 1 -- 172.21.15.33:0/2583761751 >> v1:172.21.15.69:6789/0 conn(0x7f4444154710 legacy=0x7f44440c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.919+0000 7f444979c700 1 -- 172.21.15.33:0/2583761751 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44440c5f50 con 0x7f44440c7ff0 2023-11-06T20:59:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.919+0000 7f444979c700 1 --2- 172.21.15.33:0/2583761751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44440c7ff0 0x7f44440646c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f443c009e60 tx=0x7f443c009780).ready entity=mon.1 client_cookie=d7d8ed97e64ce3d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f44437fe700 1 -- 172.21.15.33:0/2583761751 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c00e070 con 0x7f44440c7ff0 2023-11-06T20:59:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f44437fe700 1 -- 172.21.15.33:0/2583761751 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c00bb60 con 0x7f44440c7ff0 2023-11-06T20:59:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f44437fe700 1 -- 172.21.15.33:0/2583761751 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c012410 con 0x7f44440c7ff0 2023-11-06T20:59:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f444b1ff700 1 -- 172.21.15.33:0/2583761751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44440c7ff0 msgr2=0x7f44440646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f444b1ff700 1 --2- 172.21.15.33:0/2583761751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44440c7ff0 0x7f44440646c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f443c009e60 tx=0x7f443c009780).stop 2023-11-06T20:59:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f444b1ff700 1 -- 172.21.15.33:0/2583761751 shutdown_connections 2023-11-06T20:59:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f444b1ff700 1 --2- 172.21.15.33:0/2583761751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44440c7ff0 0x7f44440646c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f444b1ff700 1 --2- 172.21.15.33:0/2583761751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4444154330 0x7f44440c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f444b1ff700 1 -- 172.21.15.33:0/2583761751 >> 172.21.15.33:0/2583761751 conn(0x7f44440cb910 msgr2=0x7f44440c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f444b1ff700 1 -- 172.21.15.33:0/2583761751 shutdown_connections 2023-11-06T20:59:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.920+0000 7f444b1ff700 1 -- 172.21.15.33:0/2583761751 wait complete. 2023-11-06T20:59:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.923+0000 7f444b1ff700 1 Processor -- start 2023-11-06T20:59:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.923+0000 7f444b1ff700 1 -- start start 2023-11-06T20:59:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f444b1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44440c7ff0 0x7f4444153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f444b1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444154330 0x7f44440aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f444b1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4444154710 0x7f4444158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f444b1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4444158080 con 0x7f4444154330 2023-11-06T20:59:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f444b1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4444158640 con 0x7f4444154710 2023-11-06T20:59:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f444b1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44441579d0 con 0x7f44440c7ff0 2023-11-06T20:59:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f4448f9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44440c7ff0 0x7f4444153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f444979c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4444154710 0x7f4444158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f4443fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444154330 0x7f44440aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f4448f9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44440c7ff0 0x7f4444153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:37102/0 (socket says 172.21.15.33:37102) 2023-11-06T20:59:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f444979c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4444154710 0x7f4444158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:58340/0 (socket says 172.21.15.33:58340) 2023-11-06T20:59:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.924+0000 7f4448f9b700 1 -- 172.21.15.33:0/11764745 learned_addr learned my addr 172.21.15.33:0/11764745 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f444979c700 1 -- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44440c7ff0 msgr2=0x7f4444153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f444979c700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44440c7ff0 0x7f4444153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f444979c700 1 -- 172.21.15.33:0/11764745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444154330 msgr2=0x7f44440aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:39.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f444979c700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444154330 0x7f44440aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:39.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f444979c700 1 -- 172.21.15.33:0/11764745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c009a20 con 0x7f4444154710 2023-11-06T20:59:39.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f4443fff700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444154330 0x7f44440aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:39.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f444979c700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4444154710 0x7f4444158ad0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f443c014000 tx=0x7f443c009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:39.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f4440ff9700 1 -- 172.21.15.33:0/11764745 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c012410 con 0x7f4444154710 2023-11-06T20:59:39.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f4440ff9700 1 -- 172.21.15.33:0/11764745 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c01d630 con 0x7f4444154710 2023-11-06T20:59:39.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f4440ff9700 1 -- 172.21.15.33:0/11764745 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c01d9f0 con 0x7f4444154710 2023-11-06T20:59:39.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.925+0000 7f444b1ff700 1 -- 172.21.15.33:0/11764745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4444157cb0 con 0x7f4444154710 2023-11-06T20:59:39.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.926+0000 7f444b1ff700 1 -- 172.21.15.33:0/11764745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4444153550 con 0x7f4444154710 2023-11-06T20:59:39.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.927+0000 7f444b1ff700 1 -- 172.21.15.33:0/11764745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44441587b0 con 0x7f4444154710 2023-11-06T20:59:39.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.927+0000 7f4440ff9700 1 -- 172.21.15.33:0/11764745 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f443c003040 con 0x7f4444154710 2023-11-06T20:59:39.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.928+0000 7f4440ff9700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f442805f300 0x7f44280617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:39.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.929+0000 7f4440ff9700 1 -- 172.21.15.33:0/11764745 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f443c07c450 con 0x7f4444154710 2023-11-06T20:59:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.929+0000 7f4448f9b700 1 -- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f442805f300 msgr2=0x7f44280617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.929+0000 7f4448f9b700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f442805f300 0x7f44280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:39.932+0000 7f4440ff9700 1 -- 172.21.15.33:0/11764745 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f443c04d290 con 0x7f4444154710 2023-11-06T20:59:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.115+0000 7f444b1ff700 1 -- 172.21.15.33:0/11764745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f44440830b0 con 0x7f4444154710 2023-11-06T20:59:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.116+0000 7f4440ff9700 1 -- 172.21.15.33:0/11764745 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v85) v1 ==== 92+0+10 (secure 0 0 0) 0x7f443c0508b0 con 0x7f4444154710 2023-11-06T20:59:40.117 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:59:40.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 -- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f442805f300 msgr2=0x7f44280617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:40.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f442805f300 0x7f44280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:40.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 -- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4444154710 msgr2=0x7f4444158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4444154710 0x7f4444158ad0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f443c014000 tx=0x7f443c009780).stop 2023-11-06T20:59:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 -- 172.21.15.33:0/11764745 shutdown_connections 2023-11-06T20:59:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f442805f300 0x7f44280617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44440c7ff0 0x7f4444153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4444154710 0x7f4444158ad0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 --2- 172.21.15.33:0/11764745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444154330 0x7f44440aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 -- 172.21.15.33:0/11764745 >> 172.21.15.33:0/11764745 conn(0x7f44440cb910 msgr2=0x7f44440ab820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.118+0000 7f444b1ff700 1 -- 172.21.15.33:0/11764745 shutdown_connections 2023-11-06T20:59:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.119+0000 7f444b1ff700 1 -- 172.21.15.33:0/11764745 wait complete. 2023-11-06T20:59:40.138 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T20:59:40.210 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:40.210+0000 7f40d8e46680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:59:40.335 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:40.335+0000 7f40d8e46680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:40.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.340+0000 7f4ebd1ff700 1 Processor -- start 2023-11-06T20:59:40.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.340+0000 7f4ebd1ff700 1 -- start start 2023-11-06T20:59:40.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.342+0000 7f4ebd1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80680c0 0x7f4eb80c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:40.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.342+0000 7f4ebd1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb80bbec0 con 0x7f4eb80bcef0 2023-11-06T20:59:40.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.342+0000 7f4ebd1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb80bc130 con 0x7f4eb8154250 2023-11-06T20:59:40.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.342+0000 7f4ebd1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80bdcd0 con 0x7f4eb80680c0 2023-11-06T20:59:40.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.342+0000 7f4eb759e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4eb80bcef0 0x7f4eb8157e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40628/0 (socket says 172.21.15.33:40628) 2023-11-06T20:59:40.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.342+0000 7f4eb759e700 1 -- 172.21.15.33:0/1728153522 learned_addr learned my addr 172.21.15.33:0/1728153522 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:40.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.342+0000 7f4eb659c700 1 --2- 172.21.15.33:0/1728153522 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80680c0 0x7f4eb80c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:40.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.343+0000 7f4eb5d9b700 1 -- 172.21.15.33:0/1728153522 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989181662 0 0) 0x7f4eb80bc130 con 0x7f4eb8154250 2023-11-06T20:59:40.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.343+0000 7f4eb5d9b700 1 -- 172.21.15.33:0/1728153522 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea00033f0 con 0x7f4eb8154250 2023-11-06T20:59:40.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.343+0000 7f4eb5d9b700 1 -- 172.21.15.33:0/1728153522 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112193935 0 0) 0x7f4eb80bbec0 con 0x7f4eb80bcef0 2023-11-06T20:59:40.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.343+0000 7f4eb659c700 1 -- 172.21.15.33:0/1728153522 >> v1:172.21.15.69:6789/0 conn(0x7f4eb8154250 legacy=0x7f4eb80bd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.343+0000 7f4eb659c700 1 -- 172.21.15.33:0/1728153522 >> v1:172.21.15.33:6789/0 conn(0x7f4eb80bcef0 legacy=0x7f4eb8157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.343+0000 7f4eb659c700 1 -- 172.21.15.33:0/1728153522 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb80bc410 con 0x7f4eb80680c0 2023-11-06T20:59:40.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.343+0000 7f4eb659c700 1 --2- 172.21.15.33:0/1728153522 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80680c0 0x7f4eb80c1820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f4eac009790 tx=0x7f4eac009060).ready entity=mon.2 client_cookie=3c16673e554f9480 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.343+0000 7f4eb5d9b700 1 -- 172.21.15.33:0/1728153522 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00f510 con 0x7f4eb80680c0 2023-11-06T20:59:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.343+0000 7f4eb5d9b700 1 -- 172.21.15.33:0/1728153522 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac00f670 con 0x7f4eb80680c0 2023-11-06T20:59:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.343+0000 7f4eb5d9b700 1 -- 172.21.15.33:0/1728153522 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00f9f0 con 0x7f4eb80680c0 2023-11-06T20:59:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.344+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/1728153522 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80680c0 msgr2=0x7f4eb80c1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.344+0000 7f4ebd1ff700 1 --2- 172.21.15.33:0/1728153522 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80680c0 0x7f4eb80c1820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f4eac009790 tx=0x7f4eac009060).stop 2023-11-06T20:59:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.344+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/1728153522 shutdown_connections 2023-11-06T20:59:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.344+0000 7f4ebd1ff700 1 --2- 172.21.15.33:0/1728153522 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80680c0 0x7f4eb80c1820 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.344+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/1728153522 >> 172.21.15.33:0/1728153522 conn(0x7f4eb80631e0 msgr2=0x7f4eb80c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.344+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/1728153522 shutdown_connections 2023-11-06T20:59:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.344+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/1728153522 wait complete. 2023-11-06T20:59:40.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.347+0000 7f4ebd1ff700 1 Processor -- start 2023-11-06T20:59:40.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.347+0000 7f4ebd1ff700 1 -- start start 2023-11-06T20:59:40.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.348+0000 7f4ebd1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80bcef0 0x7f4eb8145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:40.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.348+0000 7f4ebd1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4eb8154250 0x7f4eb814ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:40.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.348+0000 7f4ebd1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb814b010 0x7f4eb806a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:40.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4ebd1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80bf170 con 0x7f4eb814b010 2023-11-06T20:59:40.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4ebd1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80bffd0 con 0x7f4eb8154250 2023-11-06T20:59:40.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4ebd1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb814b450 con 0x7f4eb80bcef0 2023-11-06T20:59:40.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb814b010 0x7f4eb806a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:40.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4eb8154250 0x7f4eb814ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:40.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb6d9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80bcef0 0x7f4eb8145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:40.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb814b010 0x7f4eb806a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51542/0 (socket says 172.21.15.33:51542) 2023-11-06T20:59:40.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4eb8154250 0x7f4eb814ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:58350/0 (socket says 172.21.15.33:58350) 2023-11-06T20:59:40.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb759e700 1 -- 172.21.15.33:0/986149423 learned_addr learned my addr 172.21.15.33:0/986149423 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:40.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb659c700 1 -- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80bcef0 msgr2=0x7f4eb8145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb659c700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80bcef0 0x7f4eb8145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb659c700 1 -- 172.21.15.33:0/986149423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb814b010 msgr2=0x7f4eb806a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb659c700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb814b010 0x7f4eb806a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb659c700 1 -- 172.21.15.33:0/986149423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac0092a0 con 0x7f4eb8154250 2023-11-06T20:59:40.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb6d9d700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80bcef0 0x7f4eb8145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:40.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.349+0000 7f4eb759e700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb814b010 0x7f4eb806a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:40.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.350+0000 7f4eb659c700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4eb8154250 0x7f4eb814ab20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f4eac014000 tx=0x7f4eac017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:40.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.350+0000 7f4e9effd700 1 -- 172.21.15.33:0/986149423 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac000bc0 con 0x7f4eb8154250 2023-11-06T20:59:40.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.350+0000 7f4e9effd700 1 -- 172.21.15.33:0/986149423 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac00f7e0 con 0x7f4eb8154250 2023-11-06T20:59:40.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.350+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/986149423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb80c0460 con 0x7f4eb8154250 2023-11-06T20:59:40.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.350+0000 7f4e9effd700 1 -- 172.21.15.33:0/986149423 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac0236b0 con 0x7f4eb8154250 2023-11-06T20:59:40.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.350+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/986149423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb80bf6a0 con 0x7f4eb8154250 2023-11-06T20:59:40.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.352+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/986149423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb80c0140 con 0x7f4eb8154250 2023-11-06T20:59:40.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.352+0000 7f4e9effd700 1 -- 172.21.15.33:0/986149423 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f4eac023a20 con 0x7f4eb8154250 2023-11-06T20:59:40.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.354+0000 7f4e9effd700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ea405f300 0x7f4ea40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:40.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.354+0000 7f4e9effd700 1 -- 172.21.15.33:0/986149423 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f4eac07bcb0 con 0x7f4eb8154250 2023-11-06T20:59:40.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.354+0000 7f4eb6d9d700 1 -- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ea405f300 msgr2=0x7f4ea40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:40.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.354+0000 7f4eb6d9d700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ea405f300 0x7f4ea40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:40.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.357+0000 7f4e9effd700 1 -- 172.21.15.33:0/986149423 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4eac04cb80 con 0x7f4eb8154250 2023-11-06T20:59:40.399 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:40.399+0000 7f7045024680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T20:59:40.522 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:40.522+0000 7f40d8e46680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:40.542 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:40.541+0000 7f7045024680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:40.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.555+0000 7f4eb6d9d700 1 -- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ea405f300 msgr2=0x7f4ea40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:40.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.555+0000 7f4eb6d9d700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ea405f300 0x7f4ea40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:40.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.574+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/986149423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4eb80c1300 con 0x7f4eb8154250 2023-11-06T20:59:40.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.575+0000 7f4e9effd700 1 -- 172.21.15.33:0/986149423 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4eac01e0a0 con 0x7f4eb8154250 2023-11-06T20:59:40.576 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:59:40.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.577+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ea405f300 msgr2=0x7f4ea40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:40.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.577+0000 7f4ebd1ff700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ea405f300 0x7f4ea40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:40.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.577+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4eb8154250 msgr2=0x7f4eb814ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.577+0000 7f4ebd1ff700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4eb8154250 0x7f4eb814ab20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f4eac014000 tx=0x7f4eac017000).stop 2023-11-06T20:59:40.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.577+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/986149423 shutdown_connections 2023-11-06T20:59:40.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.577+0000 7f4ebd1ff700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ea405f300 0x7f4ea40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:40.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.577+0000 7f4ebd1ff700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4eb80bcef0 0x7f4eb8145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.577+0000 7f4ebd1ff700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb814b010 0x7f4eb806a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.577+0000 7f4ebd1ff700 1 --2- 172.21.15.33:0/986149423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4eb8154250 0x7f4eb814ab20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.577+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/986149423 >> 172.21.15.33:0/986149423 conn(0x7f4eb80631e0 msgr2=0x7f4eb814c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:40.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.578+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/986149423 shutdown_connections 2023-11-06T20:59:40.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.578+0000 7f4ebd1ff700 1 -- 172.21.15.33:0/986149423 wait complete. 2023-11-06T20:59:40.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T20:59:40.617 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:40.617+0000 7f40d8e46680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:59:40.702 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:40.702+0000 7f40d8e46680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:59:40.737 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:40.737+0000 7f7045024680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:40.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.802+0000 7fe75a7ff700 1 Processor -- start 2023-11-06T20:59:40.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.802+0000 7fe75a7ff700 1 -- start start 2023-11-06T20:59:40.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe75a7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe7540be2c0 0x7fe7540c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:40.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe75a7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7540641f0 con 0x7fe7541546a0 2023-11-06T20:59:40.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe75a7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7540638c0 con 0x7fe754064680 2023-11-06T20:59:40.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe75a7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe754154a80 con 0x7fe7540be2c0 2023-11-06T20:59:40.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe753fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe7541546a0 0x7fe754066560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40632/0 (socket says 172.21.15.33:40632) 2023-11-06T20:59:40.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe753fff700 1 -- 172.21.15.33:0/2792229332 learned_addr learned my addr 172.21.15.33:0/2792229332 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:40.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe758d9c700 1 --2- 172.21.15.33:0/2792229332 >> v2:172.21.15.69:3300/0 conn(0x7fe7540be2c0 0x7fe7540c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:40.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe752ffd700 1 -- 172.21.15.33:0/2792229332 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974271780 0 0) 0x7fe7540641f0 con 0x7fe7541546a0 2023-11-06T20:59:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe752ffd700 1 -- 172.21.15.33:0/2792229332 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe73c0033f0 con 0x7fe7541546a0 2023-11-06T20:59:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe752ffd700 1 -- 172.21.15.33:0/2792229332 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614041269 0 0) 0x7fe7540638c0 con 0x7fe754064680 2023-11-06T20:59:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe752ffd700 1 -- 172.21.15.33:0/2792229332 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7540641f0 con 0x7fe754064680 2023-11-06T20:59:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe758d9c700 1 -- 172.21.15.33:0/2792229332 >> v1:172.21.15.69:6789/0 conn(0x7fe754064680 legacy=0x7fe7540bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.805+0000 7fe758d9c700 1 -- 172.21.15.33:0/2792229332 >> v1:172.21.15.33:6789/0 conn(0x7fe7541546a0 legacy=0x7fe754066560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.806+0000 7fe758d9c700 1 -- 172.21.15.33:0/2792229332 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754063ba0 con 0x7fe7540be2c0 2023-11-06T20:59:40.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.806+0000 7fe758d9c700 1 --2- 172.21.15.33:0/2792229332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7540be2c0 0x7fe7540c35f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fe74c009e60 tx=0x7fe74c009780).ready entity=mon.1 client_cookie=a38ce163de11c636 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:40.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.806+0000 7fe752ffd700 1 -- 172.21.15.33:0/2792229332 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00e070 con 0x7fe7540be2c0 2023-11-06T20:59:40.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.806+0000 7fe752ffd700 1 -- 172.21.15.33:0/2792229332 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c00bab0 con 0x7fe7540be2c0 2023-11-06T20:59:40.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.806+0000 7fe752ffd700 1 -- 172.21.15.33:0/2792229332 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00be30 con 0x7fe7540be2c0 2023-11-06T20:59:40.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.807+0000 7fe75a7ff700 1 -- 172.21.15.33:0/2792229332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7540be2c0 msgr2=0x7fe7540c35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.807+0000 7fe75a7ff700 1 --2- 172.21.15.33:0/2792229332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7540be2c0 0x7fe7540c35f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fe74c009e60 tx=0x7fe74c009780).stop 2023-11-06T20:59:40.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.807+0000 7fe75a7ff700 1 -- 172.21.15.33:0/2792229332 shutdown_connections 2023-11-06T20:59:40.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.807+0000 7fe75a7ff700 1 --2- 172.21.15.33:0/2792229332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7540be2c0 0x7fe7540c35f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.807+0000 7fe75a7ff700 1 -- 172.21.15.33:0/2792229332 >> 172.21.15.33:0/2792229332 conn(0x7fe7540c8c50 msgr2=0x7fe7540c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:40.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.807+0000 7fe75a7ff700 1 -- 172.21.15.33:0/2792229332 shutdown_connections 2023-11-06T20:59:40.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.807+0000 7fe75a7ff700 1 -- 172.21.15.33:0/2792229332 wait complete. 2023-11-06T20:59:40.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.808+0000 7fe75a7ff700 1 Processor -- start 2023-11-06T20:59:40.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.809+0000 7fe75a7ff700 1 -- start start 2023-11-06T20:59:40.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.809+0000 7fe75a7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe754064680 0x7fe754146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:40.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.809+0000 7fe75a7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7540be2c0 0x7fe754155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:40.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe75a7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7541546a0 0x7fe754155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:40.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe75a7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540c1bf0 con 0x7fe7540be2c0 2023-11-06T20:59:40.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe75a7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540c1ed0 con 0x7fe7541546a0 2023-11-06T20:59:40.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe75a7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540c21b0 con 0x7fe754064680 2023-11-06T20:59:40.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe7537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7540be2c0 0x7fe754155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:40.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe753fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe754064680 0x7fe754146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:40.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe7537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7540be2c0 0x7fe754155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51548/0 (socket says 172.21.15.33:51548) 2023-11-06T20:59:40.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe7537fe700 1 -- 172.21.15.33:0/3777520701 learned_addr learned my addr 172.21.15.33:0/3777520701 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:40.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe758d9c700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7541546a0 0x7fe754155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:40.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe7537fe700 1 -- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe754064680 msgr2=0x7fe754146480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe7537fe700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe754064680 0x7fe754146480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe7537fe700 1 -- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7541546a0 msgr2=0x7fe754155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe7537fe700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7541546a0 0x7fe754155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe7537fe700 1 -- 172.21.15.33:0/3777520701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c009a20 con 0x7fe7540be2c0 2023-11-06T20:59:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe753fff700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe754064680 0x7fe754146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:40.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.810+0000 7fe758d9c700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7541546a0 0x7fe754155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:40.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.811+0000 7fe7537fe700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7540be2c0 0x7fe754155a90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fe74800c490 tx=0x7fe748007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:40.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.811+0000 7fe73bfff700 1 -- 172.21.15.33:0/3777520701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74800ce40 con 0x7fe7540be2c0 2023-11-06T20:59:40.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.811+0000 7fe73bfff700 1 -- 172.21.15.33:0/3777520701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74800f470 con 0x7fe7540be2c0 2023-11-06T20:59:40.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.811+0000 7fe75a7ff700 1 -- 172.21.15.33:0/3777520701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7540c2490 con 0x7fe7540be2c0 2023-11-06T20:59:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.811+0000 7fe73bfff700 1 -- 172.21.15.33:0/3777520701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74800f810 con 0x7fe7540be2c0 2023-11-06T20:59:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.811+0000 7fe75a7ff700 1 -- 172.21.15.33:0/3777520701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7540c1860 con 0x7fe7540be2c0 2023-11-06T20:59:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.812+0000 7fe75a7ff700 1 -- 172.21.15.33:0/3777520701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe754066000 con 0x7fe7540be2c0 2023-11-06T20:59:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.814+0000 7fe73bfff700 1 -- 172.21.15.33:0/3777520701 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fe74800fbc0 con 0x7fe7540be2c0 2023-11-06T20:59:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.815+0000 7fe73bfff700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe74005f300 0x7fe7400617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:40.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.815+0000 7fe73bfff700 1 -- 172.21.15.33:0/3777520701 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fe748079fd0 con 0x7fe7540be2c0 2023-11-06T20:59:40.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.815+0000 7fe753fff700 1 -- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe74005f300 msgr2=0x7fe7400617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:40.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.815+0000 7fe753fff700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe74005f300 0x7fe7400617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:40.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:40.818+0000 7fe73bfff700 1 -- 172.21.15.33:0/3777520701 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe74804af20 con 0x7fe7540be2c0 2023-11-06T20:59:40.840 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:40.839+0000 7f7045024680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T20:59:40.930 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:40.930+0000 7f7045024680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T20:59:41.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.002+0000 7fe75a7ff700 1 -- 172.21.15.33:0/3777520701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7540c29b0 con 0x7fe7540be2c0 2023-11-06T20:59:41.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.003+0000 7fe73bfff700 1 -- 172.21.15.33:0/3777520701 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe74804e540 con 0x7fe7540be2c0 2023-11-06T20:59:41.004 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T20:59:41.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 -- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe74005f300 msgr2=0x7fe7400617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:41.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe74005f300 0x7fe7400617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:41.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 -- 172.21.15.33:0/3777520701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7540be2c0 msgr2=0x7fe754155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7540be2c0 0x7fe754155a90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fe74800c490 tx=0x7fe748007c90).stop 2023-11-06T20:59:41.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 -- 172.21.15.33:0/3777520701 shutdown_connections 2023-11-06T20:59:41.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe74005f300 0x7fe7400617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:41.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe754064680 0x7fe754146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7541546a0 0x7fe754155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 --2- 172.21.15.33:0/3777520701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7540be2c0 0x7fe754155a90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 -- 172.21.15.33:0/3777520701 >> 172.21.15.33:0/3777520701 conn(0x7fe7540c8c50 msgr2=0x7fe7540c8450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:41.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 -- 172.21.15.33:0/3777520701 shutdown_connections 2023-11-06T20:59:41.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.005+0000 7fe75a7ff700 1 -- 172.21.15.33:0/3777520701 wait complete. 2023-11-06T20:59:41.014 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:41.014+0000 7f40d8e46680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:59:41.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.224+0000 7f624ebff700 1 Processor -- start 2023-11-06T20:59:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.224+0000 7f624ebff700 1 -- start start 2023-11-06T20:59:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.225+0000 7f624ebff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6248157720 0x7f6248158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.225+0000 7f624ebff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480a5080 0x7f62480c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.225+0000 7f624ebff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62480bbfb0 con 0x7f62480bccc0 2023-11-06T20:59:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.225+0000 7f624ebff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6248158d30 con 0x7f6248157720 2023-11-06T20:59:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.225+0000 7f624ebff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62480bdb90 con 0x7f62480a5080 2023-11-06T20:59:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f6247fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f62480bccc0 0x7f62480a4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40634/0 (socket says 172.21.15.33:40634) 2023-11-06T20:59:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f6247fff700 1 -- 172.21.15.33:0/574436578 learned_addr learned my addr 172.21.15.33:0/574436578 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f624d19c700 1 --2- 172.21.15.33:0/574436578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480a5080 0x7f62480c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f624c99b700 1 --2- 172.21.15.33:0/574436578 >> v2:172.21.15.69:3300/0 conn(0x7f6248157720 0x7f6248158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f624d19c700 1 -- 172.21.15.33:0/574436578 >> v2:172.21.15.69:3300/0 conn(0x7f6248157720 msgr2=0x7f6248158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f624d19c700 1 --2- 172.21.15.33:0/574436578 >> v2:172.21.15.69:3300/0 conn(0x7f6248157720 0x7f6248158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f62477fe700 1 -- 172.21.15.33:0/574436578 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498398009 0 0) 0x7f62480bbfb0 con 0x7f62480bccc0 2023-11-06T20:59:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f624d19c700 1 -- 172.21.15.33:0/574436578 >> v1:172.21.15.33:6789/0 conn(0x7f62480bccc0 legacy=0x7f62480a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f624d19c700 1 -- 172.21.15.33:0/574436578 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62480bc1e0 con 0x7f62480a5080 2023-11-06T20:59:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f624c99b700 1 --2- 172.21.15.33:0/574436578 >> v2:172.21.15.69:3300/0 conn(0x7f6248157720 0x7f6248158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.226+0000 7f624d19c700 1 --2- 172.21.15.33:0/574436578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480a5080 0x7f62480c1840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f6240009e60 tx=0x7f6240009780).ready entity=mon.2 client_cookie=db5c8ed145d9ee8f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f62477fe700 1 -- 172.21.15.33:0/574436578 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624000e070 con 0x7f62480a5080 2023-11-06T20:59:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f62477fe700 1 -- 172.21.15.33:0/574436578 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624000bb60 con 0x7f62480a5080 2023-11-06T20:59:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f62477fe700 1 -- 172.21.15.33:0/574436578 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240012410 con 0x7f62480a5080 2023-11-06T20:59:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f624ebff700 1 -- 172.21.15.33:0/574436578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480a5080 msgr2=0x7f62480c1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f624ebff700 1 --2- 172.21.15.33:0/574436578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480a5080 0x7f62480c1840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f6240009e60 tx=0x7f6240009780).stop 2023-11-06T20:59:41.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f624ebff700 1 -- 172.21.15.33:0/574436578 shutdown_connections 2023-11-06T20:59:41.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f624ebff700 1 --2- 172.21.15.33:0/574436578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480a5080 0x7f62480c1840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f624ebff700 1 --2- 172.21.15.33:0/574436578 >> v2:172.21.15.69:3300/0 conn(0x7f6248157720 0x7f6248158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f624ebff700 1 -- 172.21.15.33:0/574436578 >> 172.21.15.33:0/574436578 conn(0x7f62480631e0 msgr2=0x7f62480c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:41.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f624ebff700 1 -- 172.21.15.33:0/574436578 shutdown_connections 2023-11-06T20:59:41.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.227+0000 7f624ebff700 1 -- 172.21.15.33:0/574436578 wait complete. 2023-11-06T20:59:41.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.229+0000 7f624ebff700 1 Processor -- start 2023-11-06T20:59:41.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.229+0000 7f624ebff700 1 -- start start 2023-11-06T20:59:41.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.230+0000 7f624ebff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62480a5080 0x7f624806a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.230+0000 7f624ebff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480bccc0 0x7f624814d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.230+0000 7f624ebff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6248157720 0x7f62480c2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.230+0000 7f624ebff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6248155db0 con 0x7f62480a5080 2023-11-06T20:59:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.230+0000 7f624ebff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62480bedd0 con 0x7f6248157720 2023-11-06T20:59:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.230+0000 7f624ebff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62481551a0 con 0x7f62480bccc0 2023-11-06T20:59:41.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.230+0000 7f624c99b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62480a5080 0x7f624806a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:41.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.230+0000 7f6247fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480bccc0 0x7f624814d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:41.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6247fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480bccc0 0x7f624814d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:37142/0 (socket says 172.21.15.33:37142) 2023-11-06T20:59:41.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6247fff700 1 -- 172.21.15.33:0/143173605 learned_addr learned my addr 172.21.15.33:0/143173605 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:41.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f624d19c700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6248157720 0x7f62480c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:41.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6247fff700 1 -- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6248157720 msgr2=0x7f62480c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6247fff700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6248157720 0x7f62480c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6247fff700 1 -- 172.21.15.33:0/143173605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62480a5080 msgr2=0x7f624806a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6247fff700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62480a5080 0x7f624806a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6247fff700 1 -- 172.21.15.33:0/143173605 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c0092a0 con 0x7f62480bccc0 2023-11-06T20:59:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6247fff700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480bccc0 0x7f624814d800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f623400c4e0 tx=0x7f6234007c70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6244ff9700 1 -- 172.21.15.33:0/143173605 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623400ce90 con 0x7f62480bccc0 2023-11-06T20:59:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6244ff9700 1 -- 172.21.15.33:0/143173605 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62340072e0 con 0x7f62480bccc0 2023-11-06T20:59:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f6244ff9700 1 -- 172.21.15.33:0/143173605 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234007660 con 0x7f62480bccc0 2023-11-06T20:59:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.231+0000 7f624ebff700 1 -- 172.21.15.33:0/143173605 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6240009a20 con 0x7f62480bccc0 2023-11-06T20:59:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.232+0000 7f624ebff700 1 -- 172.21.15.33:0/143173605 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6248155a40 con 0x7f62480bccc0 2023-11-06T20:59:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.233+0000 7f624ebff700 1 -- 172.21.15.33:0/143173605 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62480bc4c0 con 0x7f62480bccc0 2023-11-06T20:59:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.233+0000 7f6244ff9700 1 -- 172.21.15.33:0/143173605 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f62340059b0 con 0x7f62480bccc0 2023-11-06T20:59:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.234+0000 7f6244ff9700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f622c05f300 0x7f622c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.235+0000 7f6244ff9700 1 -- 172.21.15.33:0/143173605 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f6234079d50 con 0x7f62480bccc0 2023-11-06T20:59:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.235+0000 7f624c99b700 1 -- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f622c05f300 msgr2=0x7f622c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.235+0000 7f624c99b700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f622c05f300 0x7f622c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:41.242 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:41.237+0000 7f40d8e46680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.239+0000 7f6244ff9700 1 -- 172.21.15.33:0/143173605 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f623404ac20 con 0x7f62480bccc0 2023-11-06T20:59:41.271 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:41.271+0000 7f7045024680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T20:59:41.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.436+0000 7f624c99b700 1 -- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f622c05f300 msgr2=0x7f622c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:41.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.436+0000 7f624c99b700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f622c05f300 0x7f622c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:41.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.469+0000 7f624ebff700 1 -- 172.21.15.33:0/143173605 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6248060690 con 0x7f62480bccc0 2023-11-06T20:59:41.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.470+0000 7f6244ff9700 1 -- 172.21.15.33:0/143173605 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4490 (secure 0 0 0) 0x7f623404e240 con 0x7f62480bccc0 2023-11-06T20:59:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:41.471 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6055,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1867036074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1867036074},{"type":"v1","addr":"172.21.15.69:6836","nonce":1867036074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6058,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1920115024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1920115024},{"type":"v1","addr":"172.21.15.69:6837","nonce":1920115024}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2935600703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2935600703},{"type":"v1","addr":"172.21.15.33:6835","nonce":2935600703}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:21.575862+0000","modified":"2023-11-06T20:59:24.582786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/3385607254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3385607254},{"type":"v1","addr":"172.21.15.33:6833","nonce":3385607254}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-06T20:59:41.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 -- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f622c05f300 msgr2=0x7f622c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:41.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f622c05f300 0x7f622c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:41.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 -- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480bccc0 msgr2=0x7f624814d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480bccc0 0x7f624814d800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f623400c4e0 tx=0x7f6234007c70).stop 2023-11-06T20:59:41.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 -- 172.21.15.33:0/143173605 shutdown_connections 2023-11-06T20:59:41.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f622c05f300 0x7f622c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:41.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6248157720 0x7f62480c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62480a5080 0x7f624806a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 --2- 172.21.15.33:0/143173605 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f62480bccc0 0x7f624814d800 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 -- 172.21.15.33:0/143173605 >> 172.21.15.33:0/143173605 conn(0x7f62480631e0 msgr2=0x7f62480c5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 -- 172.21.15.33:0/143173605 shutdown_connections 2023-11-06T20:59:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.472+0000 7f624ebff700 1 -- 172.21.15.33:0/143173605 wait complete. 2023-11-06T20:59:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2023-11-06T20:59:41.491 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:41.503 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:41.503+0000 7f7045024680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T20:59:41.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.688+0000 7fd128bff700 1 Processor -- start 2023-11-06T20:59:41.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.689+0000 7fd128bff700 1 -- start start 2023-11-06T20:59:41.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.690+0000 7fd128bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1240ad6e0 0x7fd124154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.690+0000 7fd128bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd1240c6dc0 0x7fd1240c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.690+0000 7fd128bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1240c5db0 con 0x7fd124154280 2023-11-06T20:59:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.690+0000 7fd128bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1240c6930 con 0x7fd1240c6dc0 2023-11-06T20:59:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.690+0000 7fd128bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240aa630 con 0x7fd1240ad6e0 2023-11-06T20:59:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.690+0000 7fd12259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd124154280 0x7fd1240a9f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40650/0 (socket says 172.21.15.33:40650) 2023-11-06T20:59:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.690+0000 7fd12259c700 1 -- 172.21.15.33:0/3818219122 learned_addr learned my addr 172.21.15.33:0/3818219122 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.690+0000 7fd121d9b700 1 --2- 172.21.15.33:0/3818219122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1240ad6e0 0x7fd124154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.690+0000 7fd122d9d700 1 --2- 172.21.15.33:0/3818219122 >> v2:172.21.15.69:3300/0 conn(0x7fd1240c6dc0 0x7fd1240c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:41.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.691+0000 7fd12159a700 1 -- 172.21.15.33:0/3818219122 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373208581 0 0) 0x7fd1240c5db0 con 0x7fd124154280 2023-11-06T20:59:41.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.691+0000 7fd12159a700 1 -- 172.21.15.33:0/3818219122 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd10c0033f0 con 0x7fd124154280 2023-11-06T20:59:41.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.691+0000 7fd121d9b700 1 -- 172.21.15.33:0/3818219122 >> v2:172.21.15.69:3300/0 conn(0x7fd1240c6dc0 msgr2=0x7fd1240c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:41.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.691+0000 7fd121d9b700 1 --2- 172.21.15.33:0/3818219122 >> v2:172.21.15.69:3300/0 conn(0x7fd1240c6dc0 0x7fd1240c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.691+0000 7fd121d9b700 1 -- 172.21.15.33:0/3818219122 >> v1:172.21.15.33:6789/0 conn(0x7fd124154280 legacy=0x7fd1240a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.691+0000 7fd121d9b700 1 -- 172.21.15.33:0/3818219122 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1240c6000 con 0x7fd1240ad6e0 2023-11-06T20:59:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.691+0000 7fd122d9d700 1 --2- 172.21.15.33:0/3818219122 >> v2:172.21.15.69:3300/0 conn(0x7fd1240c6dc0 0x7fd1240c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.691+0000 7fd121d9b700 1 --2- 172.21.15.33:0/3818219122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1240ad6e0 0x7fd124154660 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fd118009760 tx=0x7fd118009060).ready entity=mon.2 client_cookie=5c5ae26f3425e810 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.691+0000 7fd12159a700 1 -- 172.21.15.33:0/3818219122 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11800d4a0 con 0x7fd1240ad6e0 2023-11-06T20:59:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.692+0000 7fd12159a700 1 -- 172.21.15.33:0/3818219122 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11800d600 con 0x7fd1240ad6e0 2023-11-06T20:59:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.692+0000 7fd12159a700 1 -- 172.21.15.33:0/3818219122 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11800d980 con 0x7fd1240ad6e0 2023-11-06T20:59:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.692+0000 7fd128bff700 1 -- 172.21.15.33:0/3818219122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1240ad6e0 msgr2=0x7fd124154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.692+0000 7fd128bff700 1 --2- 172.21.15.33:0/3818219122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1240ad6e0 0x7fd124154660 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fd118009760 tx=0x7fd118009060).stop 2023-11-06T20:59:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.692+0000 7fd128bff700 1 -- 172.21.15.33:0/3818219122 shutdown_connections 2023-11-06T20:59:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.692+0000 7fd128bff700 1 --2- 172.21.15.33:0/3818219122 >> v2:172.21.15.69:3300/0 conn(0x7fd1240c6dc0 0x7fd1240c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.692+0000 7fd128bff700 1 --2- 172.21.15.33:0/3818219122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1240ad6e0 0x7fd124154660 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.692+0000 7fd128bff700 1 -- 172.21.15.33:0/3818219122 >> 172.21.15.33:0/3818219122 conn(0x7fd12405e1b0 msgr2=0x7fd12405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.692+0000 7fd128bff700 1 -- 172.21.15.33:0/3818219122 shutdown_connections 2023-11-06T20:59:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.692+0000 7fd128bff700 1 -- 172.21.15.33:0/3818219122 wait complete. 2023-11-06T20:59:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.694+0000 7fd128bff700 1 Processor -- start 2023-11-06T20:59:41.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.694+0000 7fd128bff700 1 -- start start 2023-11-06T20:59:41.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd128bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1240ad6e0 0x7fd12414fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd128bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1240c6dc0 0x7fd1240c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd128bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd124154280 0x7fd1240c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd128bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240c9b20 con 0x7fd1240ad6e0 2023-11-06T20:59:41.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd128bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240c9f00 con 0x7fd1240c6dc0 2023-11-06T20:59:41.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd128bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240c9280 con 0x7fd124154280 2023-11-06T20:59:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd12259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1240ad6e0 0x7fd12414fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd12259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1240ad6e0 0x7fd12414fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51572/0 (socket says 172.21.15.33:51572) 2023-11-06T20:59:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd12259c700 1 -- 172.21.15.33:0/1338378213 learned_addr learned my addr 172.21.15.33:0/1338378213 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd121d9b700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1240c6dc0 0x7fd1240c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.695+0000 7fd122d9d700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd124154280 0x7fd1240c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd12259c700 1 -- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd124154280 msgr2=0x7fd1240c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd12259c700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd124154280 0x7fd1240c38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd12259c700 1 -- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1240c6dc0 msgr2=0x7fd1240c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd12259c700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1240c6dc0 0x7fd1240c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd12259c700 1 -- 172.21.15.33:0/1338378213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c009a20 con 0x7fd1240ad6e0 2023-11-06T20:59:41.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd121d9b700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1240c6dc0 0x7fd1240c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:41.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd12259c700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1240ad6e0 0x7fd12414fbc0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fd11400c590 tx=0x7fd114007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:41.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd1127fc700 1 -- 172.21.15.33:0/1338378213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1140072e0 con 0x7fd1240ad6e0 2023-11-06T20:59:41.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd1127fc700 1 -- 172.21.15.33:0/1338378213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd114007440 con 0x7fd1240ad6e0 2023-11-06T20:59:41.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd1127fc700 1 -- 172.21.15.33:0/1338378213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1140057a0 con 0x7fd1240ad6e0 2023-11-06T20:59:41.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd128bff700 1 -- 172.21.15.33:0/1338378213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1180092a0 con 0x7fd1240ad6e0 2023-11-06T20:59:41.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.696+0000 7fd128bff700 1 -- 172.21.15.33:0/1338378213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1240c9840 con 0x7fd1240ad6e0 2023-11-06T20:59:41.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.697+0000 7fd128bff700 1 -- 172.21.15.33:0/1338378213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12406bc10 con 0x7fd1240ad6e0 2023-11-06T20:59:41.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.698+0000 7fd1127fc700 1 -- 172.21.15.33:0/1338378213 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fd114005bd0 con 0x7fd1240ad6e0 2023-11-06T20:59:41.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.699+0000 7fd1127fc700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10405f300 0x7fd1040617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:41.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.699+0000 7fd1127fc700 1 -- 172.21.15.33:0/1338378213 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fd114079d90 con 0x7fd1240ad6e0 2023-11-06T20:59:41.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.699+0000 7fd121d9b700 1 -- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10405f300 msgr2=0x7fd1040617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:41.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.700+0000 7fd121d9b700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10405f300 0x7fd1040617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:41.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.703+0000 7fd1127fc700 1 -- 172.21.15.33:0/1338378213 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd11404ad60 con 0x7fd1240ad6e0 2023-11-06T20:59:41.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.885+0000 7fd128bff700 1 -- 172.21.15.33:0/1338378213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1240c9cd0 con 0x7fd1240ad6e0 2023-11-06T20:59:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.886+0000 7fd1127fc700 1 -- 172.21.15.33:0/1338378213 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17346 (secure 0 0 0) 0x7fd11404e380 con 0x7fd1240ad6e0 2023-11-06T20:59:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:41.888 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:35.792464+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:20.035584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:20.561335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3941224090":"2023-11-07T20:59:35.792423+0000","172.21.15.69:0/1477867129":"2023-11-07T20:59:35.792423+0000","172.21.15.69:6833/114912":"2023-11-07T20:59:35.792423+0000","172.21.15.69:6832/114912":"2023-11-07T20:59:35.792423+0000","172.21.15.69:0/616431469":"2023-11-07T20:59:35.792423+0000","172.21.15.69:0/1238590029":"2023-11-07T20:59:35.792423+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 -- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10405f300 msgr2=0x7fd1040617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10405f300 0x7fd1040617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 -- 172.21.15.33:0/1338378213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1240ad6e0 msgr2=0x7fd12414fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1240ad6e0 0x7fd12414fbc0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fd11400c590 tx=0x7fd114007c80).stop 2023-11-06T20:59:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 -- 172.21.15.33:0/1338378213 shutdown_connections 2023-11-06T20:59:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10405f300 0x7fd1040617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd124154280 0x7fd1240c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1240ad6e0 0x7fd12414fbc0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 --2- 172.21.15.33:0/1338378213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1240c6dc0 0x7fd1240c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 -- 172.21.15.33:0/1338378213 >> 172.21.15.33:0/1338378213 conn(0x7fd12405e1b0 msgr2=0x7fd1241517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:41.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 -- 172.21.15.33:0/1338378213 shutdown_connections 2023-11-06T20:59:41.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:41.889+0000 7fd128bff700 1 -- 172.21.15.33:0/1338378213 wait complete. 2023-11-06T20:59:41.908 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T20:59:41.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:42.042 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:42.042+0000 7f40d8e46680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:59:42.128 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:42.128+0000 7f40d8e46680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:59:42.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.147+0000 7f7205fff700 1 Processor -- start 2023-11-06T20:59:42.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.147+0000 7f7205fff700 1 -- start start 2023-11-06T20:59:42.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f7205fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f72000c64c0 0x7f72001544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f7205fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72000c0880 con 0x7f7200064ce0 2023-11-06T20:59:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f7205fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72000c8260 con 0x7f7200154110 2023-11-06T20:59:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f7205fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72000bfd40 con 0x7f72000c64c0 2023-11-06T20:59:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f71feffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7200064ce0 0x7f72000c5db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40654/0 (socket says 172.21.15.33:40654) 2023-11-06T20:59:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f71feffd700 1 -- 172.21.15.33:0/2878818025 learned_addr learned my addr 172.21.15.33:0/2878818025 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f71fffff700 1 --2- 172.21.15.33:0/2878818025 >> v2:172.21.15.69:3300/0 conn(0x7f72000c64c0 0x7f72001544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:42.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f71fe7fc700 1 -- 172.21.15.33:0/2878818025 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263424467 0 0) 0x7f72000c8260 con 0x7f7200154110 2023-11-06T20:59:42.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f71fe7fc700 1 -- 172.21.15.33:0/2878818025 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e80033f0 con 0x7f7200154110 2023-11-06T20:59:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f71fe7fc700 1 -- 172.21.15.33:0/2878818025 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539714030 0 0) 0x7f72000c0880 con 0x7f7200064ce0 2023-11-06T20:59:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f71fe7fc700 1 -- 172.21.15.33:0/2878818025 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72000c8260 con 0x7f7200064ce0 2023-11-06T20:59:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f71fffff700 1 -- 172.21.15.33:0/2878818025 >> v1:172.21.15.69:6789/0 conn(0x7f7200154110 legacy=0x7f72000645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.149+0000 7f71fffff700 1 -- 172.21.15.33:0/2878818025 >> v1:172.21.15.33:6789/0 conn(0x7f7200064ce0 legacy=0x7f72000c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.150+0000 7f71fffff700 1 -- 172.21.15.33:0/2878818025 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000bff50 con 0x7f72000c64c0 2023-11-06T20:59:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.150+0000 7f71fffff700 1 --2- 172.21.15.33:0/2878818025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72000c64c0 0x7f72001544f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f71f8009e60 tx=0x7f71f8009780).ready entity=mon.1 client_cookie=eec9d7984faf22a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.150+0000 7f71fe7fc700 1 -- 172.21.15.33:0/2878818025 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f800e070 con 0x7f72000c64c0 2023-11-06T20:59:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.150+0000 7f71fe7fc700 1 -- 172.21.15.33:0/2878818025 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f800bab0 con 0x7f72000c64c0 2023-11-06T20:59:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.150+0000 7f71fe7fc700 1 -- 172.21.15.33:0/2878818025 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f800be30 con 0x7f72000c64c0 2023-11-06T20:59:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.151+0000 7f7205fff700 1 -- 172.21.15.33:0/2878818025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72000c64c0 msgr2=0x7f72001544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.151+0000 7f7205fff700 1 --2- 172.21.15.33:0/2878818025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72000c64c0 0x7f72001544f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f71f8009e60 tx=0x7f71f8009780).stop 2023-11-06T20:59:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.151+0000 7f7205fff700 1 -- 172.21.15.33:0/2878818025 shutdown_connections 2023-11-06T20:59:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.151+0000 7f7205fff700 1 --2- 172.21.15.33:0/2878818025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72000c64c0 0x7f72001544f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.151+0000 7f7205fff700 1 -- 172.21.15.33:0/2878818025 >> 172.21.15.33:0/2878818025 conn(0x7f72000cb8b0 msgr2=0x7f72000c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.151+0000 7f7205fff700 1 -- 172.21.15.33:0/2878818025 shutdown_connections 2023-11-06T20:59:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.151+0000 7f7205fff700 1 -- 172.21.15.33:0/2878818025 wait complete. 2023-11-06T20:59:42.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.153+0000 7f7205fff700 1 Processor -- start 2023-11-06T20:59:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.154+0000 7f7205fff700 1 -- start start 2023-11-06T20:59:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.155+0000 7f7205fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7200064ce0 0x7f72000a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.155+0000 7f7205fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f72000c64c0 0x7f7200150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.155+0000 7f7205fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7200154110 0x7f72001421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.155+0000 7f7205fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72001511d0 con 0x7f7200064ce0 2023-11-06T20:59:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.155+0000 7f7205fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7200151ff0 con 0x7f7200154110 2023-11-06T20:59:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.155+0000 7f7205fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72001513e0 con 0x7f72000c64c0 2023-11-06T20:59:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.155+0000 7f71ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7200064ce0 0x7f72000a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7200064ce0 0x7f72000a1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51584/0 (socket says 172.21.15.33:51584) 2023-11-06T20:59:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71ff7fe700 1 -- 172.21.15.33:0/3886480644 learned_addr learned my addr 172.21.15.33:0/3886480644 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71feffd700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f72000c64c0 0x7f7200150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71fffff700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7200154110 0x7f72001421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71ff7fe700 1 -- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f72000c64c0 msgr2=0x7f7200150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71ff7fe700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f72000c64c0 0x7f7200150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71ff7fe700 1 -- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7200154110 msgr2=0x7f72001421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71ff7fe700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7200154110 0x7f72001421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71ff7fe700 1 -- 172.21.15.33:0/3886480644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f8009a20 con 0x7f7200064ce0 2023-11-06T20:59:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71fffff700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7200154110 0x7f72001421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71ff7fe700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7200064ce0 0x7f72000a1c30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f71f000c410 tx=0x7f71f0007d30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.156+0000 7f71e77fe700 1 -- 172.21.15.33:0/3886480644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0012400 con 0x7f7200064ce0 2023-11-06T20:59:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.157+0000 7f71e77fe700 1 -- 172.21.15.33:0/3886480644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f0012560 con 0x7f7200064ce0 2023-11-06T20:59:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.157+0000 7f7205fff700 1 -- 172.21.15.33:0/3886480644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72001516c0 con 0x7f7200064ce0 2023-11-06T20:59:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.157+0000 7f71e77fe700 1 -- 172.21.15.33:0/3886480644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f00128e0 con 0x7f7200064ce0 2023-11-06T20:59:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.157+0000 7f7205fff700 1 -- 172.21.15.33:0/3886480644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7200151c80 con 0x7f7200064ce0 2023-11-06T20:59:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.158+0000 7f7205fff700 1 -- 172.21.15.33:0/3886480644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720006c6c0 con 0x7f7200064ce0 2023-11-06T20:59:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.159+0000 7f71e77fe700 1 -- 172.21.15.33:0/3886480644 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f71f0012c90 con 0x7f7200064ce0 2023-11-06T20:59:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.160+0000 7f71e77fe700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71ec05f300 0x7f71ec0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.160+0000 7f71e77fe700 1 -- 172.21.15.33:0/3886480644 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f71f0079930 con 0x7f7200064ce0 2023-11-06T20:59:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.160+0000 7f71feffd700 1 -- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71ec05f300 msgr2=0x7f71ec0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.160+0000 7f71feffd700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71ec05f300 0x7f71ec0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.164+0000 7f71e77fe700 1 -- 172.21.15.33:0/3886480644 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f71f0048af0 con 0x7f7200064ce0 2023-11-06T20:59:42.224 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:42.224+0000 7f40d8e46680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:59:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.361+0000 7f71feffd700 1 -- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71ec05f300 msgr2=0x7f71ec0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.361+0000 7f71feffd700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71ec05f300 0x7f71ec0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:42.375 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:42.375+0000 7f7045024680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T20:59:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.389+0000 7f7205fff700 1 -- 172.21.15.33:0/3886480644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72000c0230 con 0x7f7200064ce0 2023-11-06T20:59:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.390+0000 7f71e77fe700 1 -- 172.21.15.33:0/3886480644 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4490 (secure 0 0 0) 0x7f71f0017070 con 0x7f7200064ce0 2023-11-06T20:59:42.391 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:42.392 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6055,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1867036074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1867036074},{"type":"v1","addr":"172.21.15.69:6836","nonce":1867036074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6058,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1920115024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1920115024},{"type":"v1","addr":"172.21.15.69:6837","nonce":1920115024}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2935600703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2935600703},{"type":"v1","addr":"172.21.15.33:6835","nonce":2935600703}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:21.575862+0000","modified":"2023-11-06T20:59:24.582786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/3385607254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3385607254},{"type":"v1","addr":"172.21.15.33:6833","nonce":3385607254}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-06T20:59:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 -- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71ec05f300 msgr2=0x7f71ec0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71ec05f300 0x7f71ec0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 -- 172.21.15.33:0/3886480644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7200064ce0 msgr2=0x7f72000a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7200064ce0 0x7f72000a1c30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f71f000c410 tx=0x7f71f0007d30).stop 2023-11-06T20:59:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 -- 172.21.15.33:0/3886480644 shutdown_connections 2023-11-06T20:59:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71ec05f300 0x7f71ec0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7200154110 0x7f72001421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7200064ce0 0x7f72000a1c30 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 --2- 172.21.15.33:0/3886480644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f72000c64c0 0x7f7200150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 -- 172.21.15.33:0/3886480644 >> 172.21.15.33:0/3886480644 conn(0x7f72000cb8b0 msgr2=0x7f72000c8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 -- 172.21.15.33:0/3886480644 shutdown_connections 2023-11-06T20:59:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.392+0000 7f7205fff700 1 -- 172.21.15.33:0/3886480644 wait complete. 2023-11-06T20:59:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2023-11-06T20:59:42.408 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:42.408+0000 7f40d8e46680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:59:42.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:42.475 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:42.475+0000 7f7045024680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T20:59:42.497 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:42.497+0000 7f40d8e46680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:59:42.576 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:42.576+0000 7f7045024680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T20:59:42.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.613+0000 7f887fbff700 1 Processor -- start 2023-11-06T20:59:42.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.614+0000 7f887fbff700 1 -- start start 2023-11-06T20:59:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.615+0000 7f887fbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88781580e0 0x7f88781584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887fbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f88780bdbe0 0x7f88780684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887fbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f88780a7910 0x7f88780a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887fbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88780be430 con 0x7f88780a7910 2023-11-06T20:59:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887fbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8878064010 con 0x7f88780bdbe0 2023-11-06T20:59:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887fbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88780a82b0 con 0x7f88781580e0 2023-11-06T20:59:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887e19c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f88780a7910 0x7f88780a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887e19c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f88780a7910 0x7f88780a7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51592/0 (socket says 172.21.15.33:51592) 2023-11-06T20:59:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887e19c700 1 -- 172.21.15.33:0/3976477242 learned_addr learned my addr 172.21.15.33:0/3976477242 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887d99b700 1 --2- 172.21.15.33:0/3976477242 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88781580e0 0x7f88781584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887d19a700 1 --2- 172.21.15.33:0/3976477242 >> v2:172.21.15.69:3300/0 conn(0x7f88780bdbe0 0x7f88780684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887e19c700 1 -- 172.21.15.33:0/3976477242 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88781580e0 msgr2=0x7f88781584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887e19c700 1 --2- 172.21.15.33:0/3976477242 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88781580e0 0x7f88781584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887e19c700 1 -- 172.21.15.33:0/3976477242 >> v2:172.21.15.69:3300/0 conn(0x7f88780bdbe0 msgr2=0x7f88780684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887e19c700 1 --2- 172.21.15.33:0/3976477242 >> v2:172.21.15.69:3300/0 conn(0x7f88780bdbe0 0x7f88780684c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.616+0000 7f887e19c700 1 -- 172.21.15.33:0/3976477242 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88780689b0 con 0x7f88780a7910 2023-11-06T20:59:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887e19c700 1 --2- 172.21.15.33:0/3976477242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88780a7910 0x7f88780a7dc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f887805f0f0 tx=0x7f8870009840).ready entity=mon.0 client_cookie=b88827098357efb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887c999700 1 -- 172.21.15.33:0/3976477242 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887000e070 con 0x7f88780a7910 2023-11-06T20:59:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887c999700 1 -- 172.21.15.33:0/3976477242 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887000b970 con 0x7f88780a7910 2023-11-06T20:59:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887c999700 1 -- 172.21.15.33:0/3976477242 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887000bcf0 con 0x7f88780a7910 2023-11-06T20:59:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887fbff700 1 -- 172.21.15.33:0/3976477242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88780a7910 msgr2=0x7f88780a7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887fbff700 1 --2- 172.21.15.33:0/3976477242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88780a7910 0x7f88780a7dc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f887805f0f0 tx=0x7f8870009840).stop 2023-11-06T20:59:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887fbff700 1 -- 172.21.15.33:0/3976477242 shutdown_connections 2023-11-06T20:59:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887fbff700 1 --2- 172.21.15.33:0/3976477242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88780a7910 0x7f88780a7dc0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887fbff700 1 --2- 172.21.15.33:0/3976477242 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88781580e0 0x7f88781584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887fbff700 1 --2- 172.21.15.33:0/3976477242 >> v2:172.21.15.69:3300/0 conn(0x7f88780bdbe0 0x7f88780684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.617+0000 7f887fbff700 1 -- 172.21.15.33:0/3976477242 >> 172.21.15.33:0/3976477242 conn(0x7f887805ec80 msgr2=0x7f88780664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.618+0000 7f887fbff700 1 -- 172.21.15.33:0/3976477242 shutdown_connections 2023-11-06T20:59:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.618+0000 7f887fbff700 1 -- 172.21.15.33:0/3976477242 wait complete. 2023-11-06T20:59:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.620+0000 7f887fbff700 1 Processor -- start 2023-11-06T20:59:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.620+0000 7f887fbff700 1 -- start start 2023-11-06T20:59:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.621+0000 7f887fbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88780a7910 0x7f8878154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.621+0000 7f887fbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88780bdbe0 0x7f8878139960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.621+0000 7f887fbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88781580e0 0x7f887814e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.621+0000 7f887fbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887814ffe0 con 0x7f88781580e0 2023-11-06T20:59:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.621+0000 7f887fbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88780a87a0 con 0x7f88780a7910 2023-11-06T20:59:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.621+0000 7f887fbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88780ad480 con 0x7f88780bdbe0 2023-11-06T20:59:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887e19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88781580e0 0x7f887814e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887e19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88781580e0 0x7f887814e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51600/0 (socket says 172.21.15.33:51600) 2023-11-06T20:59:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887e19c700 1 -- 172.21.15.33:0/807048305 learned_addr learned my addr 172.21.15.33:0/807048305 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887d19a700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88780bdbe0 0x7f8878139960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887d99b700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88780a7910 0x7f8878154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887e19c700 1 -- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88780bdbe0 msgr2=0x7f8878139960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887e19c700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88780bdbe0 0x7f8878139960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887e19c700 1 -- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88780a7910 msgr2=0x7f8878154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887e19c700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88780a7910 0x7f8878154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887e19c700 1 -- 172.21.15.33:0/807048305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8870009ae0 con 0x7f88781580e0 2023-11-06T20:59:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887e19c700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88781580e0 0x7f887814e2a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f887000b610 tx=0x7f8870009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f886dffb700 1 -- 172.21.15.33:0/807048305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887000bcf0 con 0x7f88781580e0 2023-11-06T20:59:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f886dffb700 1 -- 172.21.15.33:0/807048305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887000b640 con 0x7f88781580e0 2023-11-06T20:59:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f887fbff700 1 -- 172.21.15.33:0/807048305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88780ac5d0 con 0x7f88781580e0 2023-11-06T20:59:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.622+0000 7f886dffb700 1 -- 172.21.15.33:0/807048305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887001d6b0 con 0x7f88781580e0 2023-11-06T20:59:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.623+0000 7f887fbff700 1 -- 172.21.15.33:0/807048305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88780acb50 con 0x7f88781580e0 2023-11-06T20:59:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.624+0000 7f887fbff700 1 -- 172.21.15.33:0/807048305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88780ad610 con 0x7f88781580e0 2023-11-06T20:59:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.625+0000 7f886dffb700 1 -- 172.21.15.33:0/807048305 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f8870003040 con 0x7f88781580e0 2023-11-06T20:59:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.626+0000 7f886dffb700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f885c05f300 0x7f885c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.626+0000 7f886dffb700 1 -- 172.21.15.33:0/807048305 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f887007b820 con 0x7f88781580e0 2023-11-06T20:59:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.626+0000 7f887d99b700 1 -- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f885c05f300 msgr2=0x7f885c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.626+0000 7f887d99b700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f885c05f300 0x7f885c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.630+0000 7f886dffb700 1 -- 172.21.15.33:0/807048305 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f887004c690 con 0x7f88781580e0 2023-11-06T20:59:42.640 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:42.640+0000 7f40d8e46680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:59:42.771 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:42.771+0000 7f40d8e46680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:42.776 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:42.776+0000 7f7045024680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T20:59:42.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.827+0000 7f887d99b700 1 -- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f885c05f300 msgr2=0x7f885c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:42.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.827+0000 7f887d99b700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f885c05f300 0x7f885c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T20:59:42.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.856+0000 7f887fbff700 1 -- 172.21.15.33:0/807048305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8878156e60 con 0x7f88781580e0 2023-11-06T20:59:42.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.857+0000 7f886dffb700 1 -- 172.21.15.33:0/807048305 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4490 (secure 0 0 0) 0x7f8870022020 con 0x7f88781580e0 2023-11-06T20:59:42.858 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:42.858 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6055,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1867036074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1867036074},{"type":"v1","addr":"172.21.15.69:6836","nonce":1867036074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6058,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1920115024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1920115024},{"type":"v1","addr":"172.21.15.69:6837","nonce":1920115024}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2935600703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2935600703},{"type":"v1","addr":"172.21.15.33:6835","nonce":2935600703}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:21.575862+0000","modified":"2023-11-06T20:59:24.582786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/3385607254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3385607254},{"type":"v1","addr":"172.21.15.33:6833","nonce":3385607254}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-06T20:59:42.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.859+0000 7f887fbff700 1 -- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f885c05f300 msgr2=0x7f885c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:42.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.859+0000 7f887fbff700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f885c05f300 0x7f885c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.859+0000 7f887fbff700 1 -- 172.21.15.33:0/807048305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88781580e0 msgr2=0x7f887814e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.859+0000 7f887fbff700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88781580e0 0x7f887814e2a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f887000b610 tx=0x7f8870009840).stop 2023-11-06T20:59:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.859+0000 7f887fbff700 1 -- 172.21.15.33:0/807048305 shutdown_connections 2023-11-06T20:59:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.859+0000 7f887fbff700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f885c05f300 0x7f885c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.859+0000 7f887fbff700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88781580e0 0x7f887814e2a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.859+0000 7f887fbff700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88780a7910 0x7f8878154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.859+0000 7f887fbff700 1 --2- 172.21.15.33:0/807048305 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88780bdbe0 0x7f8878139960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:42.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.859+0000 7f887fbff700 1 -- 172.21.15.33:0/807048305 >> 172.21.15.33:0/807048305 conn(0x7f887805ec80 msgr2=0x7f88780664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.860+0000 7f887fbff700 1 -- 172.21.15.33:0/807048305 shutdown_connections 2023-11-06T20:59:42.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:42.860+0000 7f887fbff700 1 -- 172.21.15.33:0/807048305 wait complete. 2023-11-06T20:59:42.864 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2023-11-06T20:59:42.870 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:42.870+0000 7f7045024680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T20:59:42.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:43.024 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:43.024+0000 7f7045024680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T20:59:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.076+0000 7f0d9efff700 1 Processor -- start 2023-11-06T20:59:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.076+0000 7f0d9efff700 1 -- start start 2023-11-06T20:59:43.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.077+0000 7f0d9efff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0d98153e60 0x7f0d980a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:43.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0d98154240 0x7f0d98069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:43.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9efff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d980c2ec0 con 0x7f0d98062710 2023-11-06T20:59:43.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9efff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d981546c0 con 0x7f0d98153e60 2023-11-06T20:59:43.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980c4360 con 0x7f0d98154240 2023-11-06T20:59:43.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9d59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0d98062710 0x7f0d98157df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40658/0 (socket says 172.21.15.33:40658) 2023-11-06T20:59:43.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9d59c700 1 -- 172.21.15.33:0/3887800365 learned_addr learned my addr 172.21.15.33:0/3887800365 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:43.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9cd9b700 1 --2- 172.21.15.33:0/3887800365 >> v2:172.21.15.69:3300/0 conn(0x7f0d98153e60 0x7f0d980a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:43.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d97fff700 1 --2- 172.21.15.33:0/3887800365 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0d98154240 0x7f0d98069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:43.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d977fe700 1 -- 172.21.15.33:0/3887800365 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058385846 0 0) 0x7f0d980c2ec0 con 0x7f0d98062710 2023-11-06T20:59:43.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d977fe700 1 -- 172.21.15.33:0/3887800365 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d880033f0 con 0x7f0d98062710 2023-11-06T20:59:43.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9cd9b700 1 -- 172.21.15.33:0/3887800365 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0d98154240 msgr2=0x7f0d98069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9cd9b700 1 --2- 172.21.15.33:0/3887800365 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0d98154240 0x7f0d98069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9cd9b700 1 -- 172.21.15.33:0/3887800365 >> v1:172.21.15.33:6789/0 conn(0x7f0d98062710 legacy=0x7f0d98157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.078+0000 7f0d9cd9b700 1 -- 172.21.15.33:0/3887800365 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d980a73d0 con 0x7f0d98153e60 2023-11-06T20:59:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.079+0000 7f0d9cd9b700 1 --2- 172.21.15.33:0/3887800365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d98153e60 0x7f0d980a6d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f0d8c009760 tx=0x7f0d8c009060).ready entity=mon.1 client_cookie=cbc1831dec0ba22a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.079+0000 7f0d977fe700 1 -- 172.21.15.33:0/3887800365 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00d4a0 con 0x7f0d98153e60 2023-11-06T20:59:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.079+0000 7f0d977fe700 1 -- 172.21.15.33:0/3887800365 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c00d600 con 0x7f0d98153e60 2023-11-06T20:59:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.079+0000 7f0d977fe700 1 -- 172.21.15.33:0/3887800365 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00d980 con 0x7f0d98153e60 2023-11-06T20:59:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.079+0000 7f0d9efff700 1 -- 172.21.15.33:0/3887800365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d98153e60 msgr2=0x7f0d980a6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.079+0000 7f0d9efff700 1 --2- 172.21.15.33:0/3887800365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d98153e60 0x7f0d980a6d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f0d8c009760 tx=0x7f0d8c009060).stop 2023-11-06T20:59:43.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.080+0000 7f0d9efff700 1 -- 172.21.15.33:0/3887800365 shutdown_connections 2023-11-06T20:59:43.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.080+0000 7f0d9efff700 1 --2- 172.21.15.33:0/3887800365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d98153e60 0x7f0d980a6d40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.080+0000 7f0d9efff700 1 --2- 172.21.15.33:0/3887800365 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0d98154240 0x7f0d98069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.080+0000 7f0d9efff700 1 -- 172.21.15.33:0/3887800365 >> 172.21.15.33:0/3887800365 conn(0x7f0d980c7b50 msgr2=0x7f0d980c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.080+0000 7f0d9efff700 1 -- 172.21.15.33:0/3887800365 shutdown_connections 2023-11-06T20:59:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.080+0000 7f0d9efff700 1 -- 172.21.15.33:0/3887800365 wait complete. 2023-11-06T20:59:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.081+0000 7f0d9efff700 1 Processor -- start 2023-11-06T20:59:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.081+0000 7f0d9efff700 1 -- start start 2023-11-06T20:59:43.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.082+0000 7f0d9efff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d98062710 0x7f0d980ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.082+0000 7f0d9efff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d98153e60 0x7f0d981531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.082+0000 7f0d9efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0d98154240 0x7f0d981536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.082+0000 7f0d9efff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98153b90 con 0x7f0d98153e60 2023-11-06T20:59:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.082+0000 7f0d9efff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980ad8d0 con 0x7f0d98062710 2023-11-06T20:59:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.082+0000 7f0d9efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980a8df0 con 0x7f0d98154240 2023-11-06T20:59:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d97fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d98153e60 0x7f0d981531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d97fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d98153e60 0x7f0d981531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51602/0 (socket says 172.21.15.33:51602) 2023-11-06T20:59:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d9cd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d98062710 0x7f0d980ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d97fff700 1 -- 172.21.15.33:0/2440734419 learned_addr learned my addr 172.21.15.33:0/2440734419 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d9d59c700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0d98154240 0x7f0d981536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d97fff700 1 -- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0d98154240 msgr2=0x7f0d981536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d97fff700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0d98154240 0x7f0d981536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d97fff700 1 -- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d98062710 msgr2=0x7f0d980ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d97fff700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d98062710 0x7f0d980ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d97fff700 1 -- 172.21.15.33:0/2440734419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c0092a0 con 0x7f0d98153e60 2023-11-06T20:59:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d9cd9b700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d98062710 0x7f0d980ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.083+0000 7f0d97fff700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d98153e60 0x7f0d981531b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f0d8400e900 tx=0x7f0d84009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.084+0000 7f0d94ff9700 1 -- 172.21.15.33:0/2440734419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84012600 con 0x7f0d98153e60 2023-11-06T20:59:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.084+0000 7f0d94ff9700 1 -- 172.21.15.33:0/2440734419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d84012760 con 0x7f0d98153e60 2023-11-06T20:59:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.084+0000 7f0d9efff700 1 -- 172.21.15.33:0/2440734419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d980ac130 con 0x7f0d98153e60 2023-11-06T20:59:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.084+0000 7f0d94ff9700 1 -- 172.21.15.33:0/2440734419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84012ae0 con 0x7f0d98153e60 2023-11-06T20:59:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.084+0000 7f0d9efff700 1 -- 172.21.15.33:0/2440734419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d980a87a0 con 0x7f0d98153e60 2023-11-06T20:59:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.085+0000 7f0d9efff700 1 -- 172.21.15.33:0/2440734419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d980a8600 con 0x7f0d98153e60 2023-11-06T20:59:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.085+0000 7f0d94ff9700 1 -- 172.21.15.33:0/2440734419 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f0d8401b400 con 0x7f0d98153e60 2023-11-06T20:59:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.087+0000 7f0d94ff9700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0d7c05f300 0x7f0d7c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.087+0000 7f0d94ff9700 1 -- 172.21.15.33:0/2440734419 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f0d8407a980 con 0x7f0d98153e60 2023-11-06T20:59:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.087+0000 7f0d9cd9b700 1 -- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0d7c05f300 msgr2=0x7f0d7c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T20:59:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.087+0000 7f0d9cd9b700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0d7c05f300 0x7f0d7c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T20:59:43.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.090+0000 7f0d94ff9700 1 -- 172.21.15.33:0/2440734419 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0d980a8600 con 0x7f0d98153e60 2023-11-06T20:59:43.115 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.115+0000 7f40d8e46680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:59:43.166 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:43.166+0000 7f7045024680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T20:59:43.205 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.205+0000 7f40d8e46680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:59:43.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.214+0000 7f0d94ff9700 1 -- 172.21.15.33:0/2440734419 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 29) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f0d84078350 con 0x7f0d98153e60 2023-11-06T20:59:43.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.214+0000 7f0d94ff9700 1 -- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0d7c05f300 msgr2=0x7f0d7c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T20:59:43.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.214+0000 7f0d94ff9700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0d7c05f300 0x7f0d7c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:43.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.279+0000 7f0d9efff700 1 -- 172.21.15.33:0/2440734419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d980a8600 con 0x7f0d98153e60 2023-11-06T20:59:43.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.280+0000 7f0d94ff9700 1 -- 172.21.15.33:0/2440734419 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+17655 (secure 0 0 0) 0x7f0d980a8600 con 0x7f0d98153e60 2023-11-06T20:59:43.281 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:43.281 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":86,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:43.208550+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:20.035584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:20.561335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2032684550":"2023-11-07T20:59:43.208506+0000","172.21.15.69:0/1238590029":"2023-11-07T20:59:35.792423+0000","172.21.15.69:0/955991039":"2023-11-07T20:59:43.208506+0000","172.21.15.69:0/3051105998":"2023-11-07T20:59:43.208506+0000","172.21.15.69:0/1532662811":"2023-11-07T20:59:43.208506+0000","172.21.15.69:0/616431469":"2023-11-07T20:59:35.792423+0000","172.21.15.69:6832/114912":"2023-11-07T20:59:35.792423+0000","172.21.15.69:6833/114912":"2023-11-07T20:59:35.792423+0000","172.21.15.69:0/1551837336":"2023-11-07T20:59:43.208506+0000","172.21.15.69:0/1477867129":"2023-11-07T20:59:35.792423+0000","172.21.15.69:0/3941224090":"2023-11-07T20:59:35.792423+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:43.282 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.281+0000 7f4054152700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:43.282 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.281+0000 7f4054152700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:43.282 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.281+0000 7f4054152700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:43.283 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.281+0000 7f4054152700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:43.283 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.281+0000 7f4054152700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:43.283 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.281+0000 7f405094b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:43.283 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.281+0000 7f405094b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:43.283 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.281+0000 7f405094b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:43.284 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.281+0000 7f405094b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:43.284 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T20:59:43.281+0000 7f405094b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T20:59:43.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.282+0000 7f0d9efff700 1 -- 172.21.15.33:0/2440734419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d98153e60 msgr2=0x7f0d981531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:43.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.282+0000 7f0d9efff700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d98153e60 0x7f0d981531b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f0d8400e900 tx=0x7f0d84009060).stop 2023-11-06T20:59:43.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.283+0000 7f0d9efff700 1 -- 172.21.15.33:0/2440734419 shutdown_connections 2023-11-06T20:59:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.283+0000 7f0d9efff700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0d7c05f300 0x7f0d7c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T20:59:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.283+0000 7f0d9efff700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0d98154240 0x7f0d981536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.283+0000 7f0d9efff700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d98062710 0x7f0d980ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.283+0000 7f0d9efff700 1 --2- 172.21.15.33:0/2440734419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d98153e60 0x7f0d981531b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.283+0000 7f0d9efff700 1 -- 172.21.15.33:0/2440734419 >> 172.21.15.33:0/2440734419 conn(0x7f0d980c7b50 msgr2=0x7f0d980c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.283+0000 7f0d9efff700 1 -- 172.21.15.33:0/2440734419 shutdown_connections 2023-11-06T20:59:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.283+0000 7f0d9efff700 1 -- 172.21.15.33:0/2440734419 wait complete. 2023-11-06T20:59:43.304 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T20:59:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.509+0000 7faf889ff700 1 Processor -- start 2023-11-06T20:59:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.509+0000 7faf889ff700 1 -- start start 2023-11-06T20:59:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.511+0000 7faf889ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c3990 0x7faf84069b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.511+0000 7faf889ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8406a470 con 0x7faf841373a0 2023-11-06T20:59:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.511+0000 7faf889ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8406a190 con 0x7faf840c1020 2023-11-06T20:59:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.511+0000 7faf889ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf840bbab0 con 0x7faf840c3990 2023-11-06T20:59:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.511+0000 7faf8259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faf841373a0 0x7faf840c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40664/0 (socket says 172.21.15.33:40664) 2023-11-06T20:59:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.511+0000 7faf8259c700 1 -- 172.21.15.33:0/2796988015 learned_addr learned my addr 172.21.15.33:0/2796988015 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.511+0000 7faf82d9d700 1 --2- 172.21.15.33:0/2796988015 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c3990 0x7faf84069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.512+0000 7faf8159a700 1 -- 172.21.15.33:0/2796988015 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633668048 0 0) 0x7faf8406a470 con 0x7faf841373a0 2023-11-06T20:59:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.512+0000 7faf8159a700 1 -- 172.21.15.33:0/2796988015 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf6c0033f0 con 0x7faf841373a0 2023-11-06T20:59:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.512+0000 7faf8159a700 1 -- 172.21.15.33:0/2796988015 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656438991 0 0) 0x7faf8406a190 con 0x7faf840c1020 2023-11-06T20:59:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.512+0000 7faf82d9d700 1 -- 172.21.15.33:0/2796988015 >> v1:172.21.15.69:6789/0 conn(0x7faf840c1020 legacy=0x7faf84069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.512+0000 7faf82d9d700 1 -- 172.21.15.33:0/2796988015 >> v1:172.21.15.33:6789/0 conn(0x7faf841373a0 legacy=0x7faf840c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.512+0000 7faf82d9d700 1 -- 172.21.15.33:0/2796988015 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf840bbd90 con 0x7faf840c3990 2023-11-06T20:59:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.512+0000 7faf82d9d700 1 --2- 172.21.15.33:0/2796988015 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c3990 0x7faf84069b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7faf7c009e90 tx=0x7faf7c009780).ready entity=mon.2 client_cookie=e9307b0836fc69a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.512+0000 7faf8159a700 1 -- 172.21.15.33:0/2796988015 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00e070 con 0x7faf840c3990 2023-11-06T20:59:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.512+0000 7faf8159a700 1 -- 172.21.15.33:0/2796988015 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c00bb60 con 0x7faf840c3990 2023-11-06T20:59:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.512+0000 7faf8159a700 1 -- 172.21.15.33:0/2796988015 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00bee0 con 0x7faf840c3990 2023-11-06T20:59:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.513+0000 7faf889ff700 1 -- 172.21.15.33:0/2796988015 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c3990 msgr2=0x7faf84069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.513+0000 7faf889ff700 1 --2- 172.21.15.33:0/2796988015 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c3990 0x7faf84069b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7faf7c009e90 tx=0x7faf7c009780).stop 2023-11-06T20:59:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.513+0000 7faf889ff700 1 -- 172.21.15.33:0/2796988015 shutdown_connections 2023-11-06T20:59:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.513+0000 7faf889ff700 1 --2- 172.21.15.33:0/2796988015 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c3990 0x7faf84069b50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.513+0000 7faf889ff700 1 -- 172.21.15.33:0/2796988015 >> 172.21.15.33:0/2796988015 conn(0x7faf84062710 msgr2=0x7faf84062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.513+0000 7faf889ff700 1 -- 172.21.15.33:0/2796988015 shutdown_connections 2023-11-06T20:59:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.513+0000 7faf889ff700 1 -- 172.21.15.33:0/2796988015 wait complete. 2023-11-06T20:59:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.515+0000 7faf889ff700 1 Processor -- start 2023-11-06T20:59:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.515+0000 7faf889ff700 1 -- start start 2023-11-06T20:59:43.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf889ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c1020 0x7faf840a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:43.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf889ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf840c3990 0x7faf841534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:43.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf889ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf841373a0 0x7faf84154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:43.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf889ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf84155090 con 0x7faf840c3990 2023-11-06T20:59:43.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf889ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf840a96a0 con 0x7faf841373a0 2023-11-06T20:59:43.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf889ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf84153010 con 0x7faf840c1020 2023-11-06T20:59:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf81d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf840c3990 0x7faf841534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf8259c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c1020 0x7faf840a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf82d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf841373a0 0x7faf84154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf8259c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c1020 0x7faf840a91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:37208/0 (socket says 172.21.15.33:37208) 2023-11-06T20:59:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf8259c700 1 -- 172.21.15.33:0/4258257062 learned_addr learned my addr 172.21.15.33:0/4258257062 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf81d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf840c3990 0x7faf841534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51608/0 (socket says 172.21.15.33:51608) 2023-11-06T20:59:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.516+0000 7faf82d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf841373a0 0x7faf84154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:58432/0 (socket says 172.21.15.33:58432) 2023-11-06T20:59:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf8259c700 1 -- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf841373a0 msgr2=0x7faf84154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf8259c700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf841373a0 0x7faf84154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf8259c700 1 -- 172.21.15.33:0/4258257062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf840c3990 msgr2=0x7faf841534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf8259c700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf840c3990 0x7faf841534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf8259c700 1 -- 172.21.15.33:0/4258257062 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c009a20 con 0x7faf840c1020 2023-11-06T20:59:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf8259c700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c1020 0x7faf840a91b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7faf7400c440 tx=0x7faf74007c70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf727fc700 1 -- 172.21.15.33:0/4258257062 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf740070d0 con 0x7faf840c1020 2023-11-06T20:59:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf727fc700 1 -- 172.21.15.33:0/4258257062 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf74007230 con 0x7faf840c1020 2023-11-06T20:59:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf727fc700 1 -- 172.21.15.33:0/4258257062 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf740075d0 con 0x7faf840c1020 2023-11-06T20:59:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf889ff700 1 -- 172.21.15.33:0/4258257062 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf84153990 con 0x7faf840c1020 2023-11-06T20:59:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.517+0000 7faf889ff700 1 -- 172.21.15.33:0/4258257062 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf841529c0 con 0x7faf840c1020 2023-11-06T20:59:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.518+0000 7faf889ff700 1 -- 172.21.15.33:0/4258257062 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf841532f0 con 0x7faf840c1020 2023-11-06T20:59:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.520+0000 7faf727fc700 1 -- 172.21.15.33:0/4258257062 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 29) v1 ==== 75270+0+0 (secure 0 0 0) 0x7faf740059b0 con 0x7faf840c1020 2023-11-06T20:59:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.520+0000 7faf727fc700 1 -- 172.21.15.33:0/4258257062 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6594+0+0 (secure 0 0 0) 0x7faf74079820 con 0x7faf840c1020 2023-11-06T20:59:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.525+0000 7faf727fc700 1 -- 172.21.15.33:0/4258257062 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf7404a9d0 con 0x7faf840c1020 2023-11-06T20:59:43.545 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:43.545+0000 7f7045024680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T20:59:43.642 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T20:59:43.642+0000 7f7045024680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T20:59:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:43.756+0000 7faf889ff700 1 -- 172.21.15.33:0/4258257062 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faf8414f0a0 con 0x7faf840c1020 2023-11-06T20:59:44.211 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T20:59:44.211+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T20:59:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.221+0000 7faf727fc700 1 -- 172.21.15.33:0/4258257062 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mgrmap(e 30) v1 ==== 75340+0+0 (secure 0 0 0) 0x7faf740774d0 con 0x7faf840c1020 2023-11-06T20:59:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.221+0000 7faf727fc700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf640302f0 0x7faf64067aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.222+0000 7faf81d9b700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf640302f0 0x7faf64067aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.223+0000 7faf81d9b700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf640302f0 0x7faf64067aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7faf841558a0 tx=0x7faf78009040).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.227+0000 7faf727fc700 1 -- 172.21.15.33:0/4258257062 <== mon.2 v2:172.21.15.69:3301/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7faf74072f70 con 0x7faf840c1020 2023-11-06T20:59:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 -- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf640302f0 msgr2=0x7faf64067aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf640302f0 0x7faf64067aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7faf841558a0 tx=0x7faf78009040).stop 2023-11-06T20:59:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 -- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c1020 msgr2=0x7faf840a91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c1020 0x7faf840a91b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7faf7400c440 tx=0x7faf74007c70).stop 2023-11-06T20:59:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 -- 172.21.15.33:0/4258257062 shutdown_connections 2023-11-06T20:59:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faf640302f0 0x7faf64067aa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf840c1020 0x7faf840a91b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf841373a0 0x7faf84154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 --2- 172.21.15.33:0/4258257062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf840c3990 0x7faf841534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 -- 172.21.15.33:0/4258257062 >> 172.21.15.33:0/4258257062 conn(0x7faf84062710 msgr2=0x7faf841350c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 -- 172.21.15.33:0/4258257062 shutdown_connections 2023-11-06T20:59:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.230+0000 7faf889ff700 1 -- 172.21.15.33:0/4258257062 wait complete. 2023-11-06T20:59:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T20:59:44.234 INFO:tasks.ceph.mds.c.smithi033.stderr: -2805> 2023-11-06T20:59:22.065+0000 7f801d9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T20:59:44.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T20:59:44.278 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T20:59:44.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.452+0000 7f1e6a3ff700 1 Processor -- start 2023-11-06T20:59:44.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.452+0000 7f1e6a3ff700 1 -- start start 2023-11-06T20:59:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.454+0000 7f1e6a3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aaf50 0x7f1e64063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.454+0000 7f1e6a3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1e640aa4f0 0x7f1e64063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.454+0000 7f1e6a3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e64158c00 con 0x7f1e64157700 2023-11-06T20:59:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.454+0000 7f1e6a3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e640c18f0 con 0x7f1e640aa4f0 2023-11-06T20:59:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.454+0000 7f1e6a3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e64063b10 con 0x7f1e640aaf50 2023-11-06T20:59:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.454+0000 7f1e63fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1e64157700 0x7f1e640c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40680/0 (socket says 172.21.15.33:40680) 2023-11-06T20:59:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.454+0000 7f1e63fff700 1 -- 172.21.15.33:0/277885330 learned_addr learned my addr 172.21.15.33:0/277885330 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.455+0000 7f1e637fe700 1 --2- 172.21.15.33:0/277885330 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aaf50 0x7f1e64063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.455+0000 7f1e6899c700 1 --2- 172.21.15.33:0/277885330 >> v2:172.21.15.69:3300/0 conn(0x7f1e640aa4f0 0x7f1e64063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.455+0000 7f1e62ffd700 1 -- 172.21.15.33:0/277885330 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730936160 0 0) 0x7f1e64158c00 con 0x7f1e64157700 2023-11-06T20:59:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.455+0000 7f1e62ffd700 1 -- 172.21.15.33:0/277885330 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e4c0033f0 con 0x7f1e64157700 2023-11-06T20:59:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.455+0000 7f1e6899c700 1 -- 172.21.15.33:0/277885330 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aaf50 msgr2=0x7f1e64063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.455+0000 7f1e6899c700 1 --2- 172.21.15.33:0/277885330 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aaf50 0x7f1e64063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.455+0000 7f1e6899c700 1 -- 172.21.15.33:0/277885330 >> v1:172.21.15.33:6789/0 conn(0x7f1e64157700 legacy=0x7f1e640c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.455+0000 7f1e6899c700 1 -- 172.21.15.33:0/277885330 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e640c7610 con 0x7f1e640aa4f0 2023-11-06T20:59:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.455+0000 7f1e6899c700 1 --2- 172.21.15.33:0/277885330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e640aa4f0 0x7f1e64063620 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f1e5c009e90 tx=0x7f1e5c009780).ready entity=mon.1 client_cookie=71c46725638fbe48 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:44.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e62ffd700 1 -- 172.21.15.33:0/277885330 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00e070 con 0x7f1e640aa4f0 2023-11-06T20:59:44.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e62ffd700 1 -- 172.21.15.33:0/277885330 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5c00ba70 con 0x7f1e640aa4f0 2023-11-06T20:59:44.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e62ffd700 1 -- 172.21.15.33:0/277885330 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00bdf0 con 0x7f1e640aa4f0 2023-11-06T20:59:44.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/277885330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e640aa4f0 msgr2=0x7f1e64063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:44.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e6a3ff700 1 --2- 172.21.15.33:0/277885330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e640aa4f0 0x7f1e64063620 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f1e5c009e90 tx=0x7f1e5c009780).stop 2023-11-06T20:59:44.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/277885330 shutdown_connections 2023-11-06T20:59:44.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e6a3ff700 1 --2- 172.21.15.33:0/277885330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e640aa4f0 0x7f1e64063620 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:44.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e6a3ff700 1 --2- 172.21.15.33:0/277885330 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aaf50 0x7f1e64063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:44.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/277885330 >> 172.21.15.33:0/277885330 conn(0x7f1e6405ec80 msgr2=0x7f1e640cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:44.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/277885330 shutdown_connections 2023-11-06T20:59:44.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.456+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/277885330 wait complete. 2023-11-06T20:59:44.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.458+0000 7f1e6a3ff700 1 Processor -- start 2023-11-06T20:59:44.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.458+0000 7f1e6a3ff700 1 -- start start 2023-11-06T20:59:44.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e6a3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aa4f0 0x7f1e641565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e6a3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640aaf50 0x7f1e640ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e6a3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e64157700 0x7f1e640c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e6a3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e64164380 con 0x7f1e640aaf50 2023-11-06T20:59:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e6a3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e641645b0 con 0x7f1e64157700 2023-11-06T20:59:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e6a3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e64164890 con 0x7f1e640aa4f0 2023-11-06T20:59:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e637fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640aaf50 0x7f1e640ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e637fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640aaf50 0x7f1e640ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51634/0 (socket says 172.21.15.33:51634) 2023-11-06T20:59:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e637fe700 1 -- 172.21.15.33:0/1137944124 learned_addr learned my addr 172.21.15.33:0/1137944124 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e63fff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aa4f0 0x7f1e641565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e6899c700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e64157700 0x7f1e640c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e637fe700 1 -- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aa4f0 msgr2=0x7f1e641565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e637fe700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aa4f0 0x7f1e641565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e637fe700 1 -- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e64157700 msgr2=0x7f1e640c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.459+0000 7f1e637fe700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e64157700 0x7f1e640c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.460+0000 7f1e637fe700 1 -- 172.21.15.33:0/1137944124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c009a20 con 0x7f1e640aaf50 2023-11-06T20:59:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.460+0000 7f1e63fff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aa4f0 0x7f1e641565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.460+0000 7f1e637fe700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640aaf50 0x7f1e640ae6c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f1e5800e930 tx=0x7f1e58009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.460+0000 7f1e4bfff700 1 -- 172.21.15.33:0/1137944124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800eed0 con 0x7f1e640aaf50 2023-11-06T20:59:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.460+0000 7f1e4bfff700 1 -- 172.21.15.33:0/1137944124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5800c850 con 0x7f1e640aaf50 2023-11-06T20:59:44.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.460+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/1137944124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e64164b70 con 0x7f1e640aaf50 2023-11-06T20:59:44.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.460+0000 7f1e4bfff700 1 -- 172.21.15.33:0/1137944124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800cbd0 con 0x7f1e640aaf50 2023-11-06T20:59:44.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.460+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/1137944124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e64165130 con 0x7f1e640aaf50 2023-11-06T20:59:44.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.461+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/1137944124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6406c6c0 con 0x7f1e640aaf50 2023-11-06T20:59:44.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.462+0000 7f1e4bfff700 1 -- 172.21.15.33:0/1137944124 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 30) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f1e5801b400 con 0x7f1e640aaf50 2023-11-06T20:59:44.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.463+0000 7f1e4bfff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e5005f030 0x7f1e500614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.463+0000 7f1e4bfff700 1 -- 172.21.15.33:0/1137944124 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6680+0+0 (secure 0 0 0) 0x7f1e5807a8a0 con 0x7f1e640aaf50 2023-11-06T20:59:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.463+0000 7f1e63fff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e5005f030 0x7f1e500614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.464+0000 7f1e63fff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e5005f030 0x7f1e500614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f1e54007fd0 tx=0x7f1e54009040).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.467+0000 7f1e4bfff700 1 -- 172.21.15.33:0/1137944124 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1e5804b9a0 con 0x7f1e640aaf50 2023-11-06T20:59:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:44.689+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/1137944124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1e640bfe50 con 0x7f1e640aaf50 2023-11-06T20:59:45.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.228+0000 7f1e4bfff700 1 -- 172.21.15.33:0/1137944124 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1e5804efc0 con 0x7f1e640aaf50 2023-11-06T20:59:45.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e5005f030 msgr2=0x7f1e500614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:45.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e5005f030 0x7f1e500614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f1e54007fd0 tx=0x7f1e54009040).stop 2023-11-06T20:59:45.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/1137944124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640aaf50 msgr2=0x7f1e640ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:45.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640aaf50 0x7f1e640ae6c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f1e5800e930 tx=0x7f1e58009060).stop 2023-11-06T20:59:45.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/1137944124 shutdown_connections 2023-11-06T20:59:45.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1e5005f030 0x7f1e500614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:45.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e640aa4f0 0x7f1e641565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:45.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e64157700 0x7f1e640c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:45.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 --2- 172.21.15.33:0/1137944124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640aaf50 0x7f1e640ae6c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:45.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/1137944124 >> 172.21.15.33:0/1137944124 conn(0x7f1e6405ec80 msgr2=0x7f1e6414ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:45.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/1137944124 shutdown_connections 2023-11-06T20:59:45.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.231+0000 7f1e6a3ff700 1 -- 172.21.15.33:0/1137944124 wait complete. 2023-11-06T20:59:45.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T20:59:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.491+0000 7fa8109ff700 1 Processor -- start 2023-11-06T20:59:45.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.491+0000 7fa8109ff700 1 -- start start 2023-11-06T20:59:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa8109ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa80c153ed0 0x7fa80c158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa8109ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa80c0c8ee0 0x7fa80c0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa8109ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa80c0c9f80 con 0x7fa80c1542b0 2023-11-06T20:59:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa8109ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80c0c6000 con 0x7fa80c153ed0 2023-11-06T20:59:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa8109ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80c0c9370 con 0x7fa80c0c8ee0 2023-11-06T20:59:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa80a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa80c153ed0 0x7fa80c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa80a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa80c153ed0 0x7fa80c158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51642/0 (socket says 172.21.15.33:51642) 2023-11-06T20:59:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa80a59c700 1 -- 172.21.15.33:0/1004532764 learned_addr learned my addr 172.21.15.33:0/1004532764 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa809d9b700 1 --2- 172.21.15.33:0/1004532764 >> v2:172.21.15.69:3300/0 conn(0x7fa80c0c8ee0 0x7fa80c0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa80a59c700 1 -- 172.21.15.33:0/1004532764 >> v2:172.21.15.69:3300/0 conn(0x7fa80c0c8ee0 msgr2=0x7fa80c0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.493+0000 7fa80a59c700 1 --2- 172.21.15.33:0/1004532764 >> v2:172.21.15.69:3300/0 conn(0x7fa80c0c8ee0 0x7fa80c0ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.494+0000 7fa80a59c700 1 -- 172.21.15.33:0/1004532764 >> v1:172.21.15.69:6789/0 conn(0x7fa80c1542b0 legacy=0x7fa80c0c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.494+0000 7fa80a59c700 1 -- 172.21.15.33:0/1004532764 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c0c9650 con 0x7fa80c153ed0 2023-11-06T20:59:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.494+0000 7fa80a59c700 1 --2- 172.21.15.33:0/1004532764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c153ed0 0x7fa80c158600 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fa80c0c8c90 tx=0x7fa7fc0090f0).ready entity=mon.0 client_cookie=fca9a2bcd75396f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.494+0000 7fa80959a700 1 -- 172.21.15.33:0/1004532764 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00b600 con 0x7fa80c153ed0 2023-11-06T20:59:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.494+0000 7fa80959a700 1 -- 172.21.15.33:0/1004532764 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc00b760 con 0x7fa80c153ed0 2023-11-06T20:59:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.494+0000 7fa80959a700 1 -- 172.21.15.33:0/1004532764 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00bae0 con 0x7fa80c153ed0 2023-11-06T20:59:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.494+0000 7fa8109ff700 1 -- 172.21.15.33:0/1004532764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c153ed0 msgr2=0x7fa80c158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.494+0000 7fa8109ff700 1 --2- 172.21.15.33:0/1004532764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c153ed0 0x7fa80c158600 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fa80c0c8c90 tx=0x7fa7fc0090f0).stop 2023-11-06T20:59:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.495+0000 7fa8109ff700 1 -- 172.21.15.33:0/1004532764 shutdown_connections 2023-11-06T20:59:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.495+0000 7fa8109ff700 1 --2- 172.21.15.33:0/1004532764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c153ed0 0x7fa80c158600 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.495+0000 7fa8109ff700 1 --2- 172.21.15.33:0/1004532764 >> v2:172.21.15.69:3300/0 conn(0x7fa80c0c8ee0 0x7fa80c0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.495+0000 7fa8109ff700 1 -- 172.21.15.33:0/1004532764 >> 172.21.15.33:0/1004532764 conn(0x7fa80c05e1b0 msgr2=0x7fa80c05e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.495+0000 7fa8109ff700 1 -- 172.21.15.33:0/1004532764 shutdown_connections 2023-11-06T20:59:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.495+0000 7fa8109ff700 1 -- 172.21.15.33:0/1004532764 wait complete. 2023-11-06T20:59:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.496+0000 7fa8109ff700 1 Processor -- start 2023-11-06T20:59:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.496+0000 7fa8109ff700 1 -- start start 2023-11-06T20:59:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.497+0000 7fa8109ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa80c153ed0 0x7fa80c163fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.497+0000 7fa8109ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c1542b0 0x7fa80c14ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.497+0000 7fa8109ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa80c1503f0 0x7fa80c0c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.497+0000 7fa8109ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c1648e0 con 0x7fa80c1542b0 2023-11-06T20:59:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.497+0000 7fa8109ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c150870 con 0x7fa80c153ed0 2023-11-06T20:59:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.497+0000 7fa8109ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c150b50 con 0x7fa80c1503f0 2023-11-06T20:59:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.497+0000 7fa809d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c1542b0 0x7fa80c14ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.497+0000 7fa809d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c1542b0 0x7fa80c14ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51648/0 (socket says 172.21.15.33:51648) 2023-11-06T20:59:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.497+0000 7fa809d9b700 1 -- 172.21.15.33:0/3143073051 learned_addr learned my addr 172.21.15.33:0/3143073051 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa80a59c700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa80c153ed0 0x7fa80c163fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa80ad9d700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa80c1503f0 0x7fa80c0c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa809d9b700 1 -- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa80c1503f0 msgr2=0x7fa80c0c2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa809d9b700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa80c1503f0 0x7fa80c0c2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa809d9b700 1 -- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa80c153ed0 msgr2=0x7fa80c163fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa809d9b700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa80c153ed0 0x7fa80c163fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa809d9b700 1 -- 172.21.15.33:0/3143073051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc009370 con 0x7fa80c1542b0 2023-11-06T20:59:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa80a59c700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa80c153ed0 0x7fa80c163fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa80ad9d700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa80c1503f0 0x7fa80c0c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa809d9b700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c1542b0 0x7fa80c14ff00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fa80000d890 tx=0x7fa800008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa7fa7fc700 1 -- 172.21.15.33:0/3143073051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8000125f0 con 0x7fa80c1542b0 2023-11-06T20:59:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa7fa7fc700 1 -- 172.21.15.33:0/3143073051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa800012750 con 0x7fa80c1542b0 2023-11-06T20:59:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa7fa7fc700 1 -- 172.21.15.33:0/3143073051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800012ad0 con 0x7fa80c1542b0 2023-11-06T20:59:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.498+0000 7fa8109ff700 1 -- 172.21.15.33:0/3143073051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c150e30 con 0x7fa80c1542b0 2023-11-06T20:59:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.499+0000 7fa8109ff700 1 -- 172.21.15.33:0/3143073051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c1644c0 con 0x7fa80c1542b0 2023-11-06T20:59:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.500+0000 7fa8109ff700 1 -- 172.21.15.33:0/3143073051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80c06c6c0 con 0x7fa80c1542b0 2023-11-06T20:59:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.500+0000 7fa7fa7fc700 1 -- 172.21.15.33:0/3143073051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 30) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fa800013480 con 0x7fa80c1542b0 2023-11-06T20:59:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.501+0000 7fa7fa7fc700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7ec05f030 0x7fa7ec0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.501+0000 7fa7fa7fc700 1 -- 172.21.15.33:0/3143073051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6680+0+0 (secure 0 0 0) 0x7fa80007a040 con 0x7fa80c1542b0 2023-11-06T20:59:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.501+0000 7fa80a59c700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7ec05f030 0x7fa7ec0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.502+0000 7fa80a59c700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7ec05f030 0x7fa7ec0614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa7fc009130 tx=0x7fa7fc017000).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.505+0000 7fa7fa7fc700 1 -- 172.21.15.33:0/3143073051 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa80004b2b0 con 0x7fa80c1542b0 2023-11-06T20:59:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:45.691+0000 7fa8109ff700 1 -- 172.21.15.33:0/3143073051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa80c0c90e0 con 0x7fa80c1542b0 2023-11-06T20:59:46.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.004+0000 7fa7fa7fc700 1 -- 172.21.15.33:0/3143073051 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa800077db0 con 0x7fa80c1542b0 2023-11-06T20:59:46.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.238+0000 7fa7fa7fc700 1 -- 172.21.15.33:0/3143073051 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa800071990 con 0x7fa80c1542b0 2023-11-06T20:59:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 -- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7ec05f030 msgr2=0x7fa7ec0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7ec05f030 0x7fa7ec0614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa7fc009130 tx=0x7fa7fc017000).stop 2023-11-06T20:59:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 -- 172.21.15.33:0/3143073051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c1542b0 msgr2=0x7fa80c14ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c1542b0 0x7fa80c14ff00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fa80000d890 tx=0x7fa800008040).stop 2023-11-06T20:59:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 -- 172.21.15.33:0/3143073051 shutdown_connections 2023-11-06T20:59:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7ec05f030 0x7fa7ec0614f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa80c1503f0 0x7fa80c0c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa80c153ed0 0x7fa80c163fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 --2- 172.21.15.33:0/3143073051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c1542b0 0x7fa80c14ff00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 -- 172.21.15.33:0/3143073051 >> 172.21.15.33:0/3143073051 conn(0x7fa80c05e1b0 msgr2=0x7fa80c0ac810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 -- 172.21.15.33:0/3143073051 shutdown_connections 2023-11-06T20:59:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.242+0000 7fa8109ff700 1 -- 172.21.15.33:0/3143073051 wait complete. 2023-11-06T20:59:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T20:59:46.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T20:59:46.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.471+0000 7ff3ce7ff700 1 Processor -- start 2023-11-06T20:59:46.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.471+0000 7ff3ce7ff700 1 -- start start 2023-11-06T20:59:46.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3ce7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3c80c8ee0 0x7ff3c80ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:46.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3ce7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff3c81542b0 0x7ff3c80c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:46.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3ce7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c80c9f80 con 0x7ff3c8153ed0 2023-11-06T20:59:46.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3ce7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c80c6000 con 0x7ff3c81542b0 2023-11-06T20:59:46.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3ce7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80c9370 con 0x7ff3c80c8ee0 2023-11-06T20:59:46.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3c7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff3c8153ed0 0x7ff3c8158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40690/0 (socket says 172.21.15.33:40690) 2023-11-06T20:59:46.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3c7fff700 1 -- 172.21.15.33:0/2380994710 learned_addr learned my addr 172.21.15.33:0/2380994710 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3c77fe700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3c80c8ee0 0x7ff3c80ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353959832 0 0) 0x7ff3c80c9f80 con 0x7ff3c8153ed0 2023-11-06T20:59:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3ccd9c700 1 --2- 172.21.15.33:0/2380994710 >> v2:172.21.15.69:3300/0 conn(0x7ff3c81542b0 0x7ff3c80c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.473+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b00033f0 con 0x7ff3c8153ed0 2023-11-06T20:59:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2470738954 0 0) 0x7ff3b00033f0 con 0x7ff3c8153ed0 2023-11-06T20:59:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3c80c8ee0 msgr2=0x7ff3c80ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3c80c8ee0 0x7ff3c80ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 >> v2:172.21.15.69:3300/0 conn(0x7ff3c81542b0 msgr2=0x7ff3c80c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 --2- 172.21.15.33:0/2380994710 >> v2:172.21.15.69:3300/0 conn(0x7ff3c81542b0 0x7ff3c80c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c80c9650 con 0x7ff3c8153ed0 2023-11-06T20:59:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7ff3b8001470 con 0x7ff3c8153ed0 2023-11-06T20:59:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 >> v1:172.21.15.33:6789/0 conn(0x7ff3c8153ed0 legacy=0x7ff3c8158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b00038e0 0x7ff3b0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3b0004690 0x7ff3b0004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b0004fc0 0x7ff3b000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80c6000 con 0x7ff3b00038e0 2023-11-06T20:59:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80c9370 con 0x7ff3b0004fc0 2023-11-06T20:59:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0004460 con 0x7ff3b0004690 2023-11-06T20:59:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c7fff700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b00038e0 0x7ff3b0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3c77fe700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3b0004690 0x7ff3b0004ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.474+0000 7ff3ccd9c700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b0004fc0 0x7ff3b000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3c7fff700 1 -- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3b0004690 msgr2=0x7ff3b0004ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3c7fff700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3b0004690 0x7ff3b0004ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3c7fff700 1 -- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b0004fc0 msgr2=0x7ff3b000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3c7fff700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b0004fc0 0x7ff3b000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3c7fff700 1 -- 172.21.15.33:0/2380994710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b00033f0 con 0x7ff3b00038e0 2023-11-06T20:59:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3c7fff700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b00038e0 0x7ff3b0003d00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7ff3b800c210 tx=0x7ff3b8001e90).ready entity=mon.0 client_cookie=d1015c821cbeaa74 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8007a60 con 0x7ff3b00038e0 2023-11-06T20:59:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b800cae0 con 0x7ff3b00038e0 2023-11-06T20:59:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3c6ffd700 1 -- 172.21.15.33:0/2380994710 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b800cea0 con 0x7ff3b00038e0 2023-11-06T20:59:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/2380994710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b00038e0 msgr2=0x7ff3b0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3ce7ff700 1 --2- 172.21.15.33:0/2380994710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b00038e0 0x7ff3b0003d00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7ff3b800c210 tx=0x7ff3b8001e90).stop 2023-11-06T20:59:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.475+0000 7ff3ccd9c700 1 -- 172.21.15.33:0/2380994710 reap_dead start 2023-11-06T20:59:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.476+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/2380994710 shutdown_connections 2023-11-06T20:59:46.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.476+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/2380994710 >> 172.21.15.33:0/2380994710 conn(0x7ff3c805e1b0 msgr2=0x7ff3c805e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:46.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.476+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/2380994710 shutdown_connections 2023-11-06T20:59:46.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.476+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/2380994710 wait complete. 2023-11-06T20:59:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.477+0000 7ff3ce7ff700 1 Processor -- start 2023-11-06T20:59:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.477+0000 7ff3ce7ff700 1 -- start start 2023-11-06T20:59:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.478+0000 7ff3ce7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3b00038e0 0x7ff3c80b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.478+0000 7ff3ce7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c814b730 0x7ff3c80ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.478+0000 7ff3ce7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3c8140780 0x7ff3c8152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.478+0000 7ff3ce7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80b0d70 con 0x7ff3c814b730 2023-11-06T20:59:46.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.478+0000 7ff3ce7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80ade70 con 0x7ff3c8140780 2023-11-06T20:59:46.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.478+0000 7ff3ce7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80ae150 con 0x7ff3b00038e0 2023-11-06T20:59:46.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.478+0000 7ff3c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c814b730 0x7ff3c80ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:46.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.478+0000 7ff3c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c814b730 0x7ff3c80ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51670/0 (socket says 172.21.15.33:51670) 2023-11-06T20:59:46.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.478+0000 7ff3c77fe700 1 -- 172.21.15.33:0/3613272588 learned_addr learned my addr 172.21.15.33:0/3613272588 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:46.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3c7fff700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3b00038e0 0x7ff3c80b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:46.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3ccd9c700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3c8140780 0x7ff3c8152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:46.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3c77fe700 1 -- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3b00038e0 msgr2=0x7ff3c80b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:46.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3c77fe700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3b00038e0 0x7ff3c80b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:46.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3c77fe700 1 -- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3c8140780 msgr2=0x7ff3c8152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:46.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3c77fe700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3c8140780 0x7ff3c8152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:46.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3c77fe700 1 -- 172.21.15.33:0/3613272588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c80ae430 con 0x7ff3c814b730 2023-11-06T20:59:46.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3ccd9c700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3c8140780 0x7ff3c8152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:46.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3c77fe700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c814b730 0x7ff3c80ae720 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7ff3bc0119f0 tx=0x7ff3bc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3affff700 1 -- 172.21.15.33:0/3613272588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00f940 con 0x7ff3c814b730 2023-11-06T20:59:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3affff700 1 -- 172.21.15.33:0/3613272588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc00faa0 con 0x7ff3c814b730 2023-11-06T20:59:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/3613272588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c8152530 con 0x7ff3c814b730 2023-11-06T20:59:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3affff700 1 -- 172.21.15.33:0/3613272588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00fe20 con 0x7ff3c814b730 2023-11-06T20:59:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.479+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/3613272588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c8140c00 con 0x7ff3c814b730 2023-11-06T20:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.481+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/3613272588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c8152750 con 0x7ff3c814b730 2023-11-06T20:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.481+0000 7ff3affff700 1 -- 172.21.15.33:0/3613272588 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff3bc014650 con 0x7ff3c814b730 2023-11-06T20:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.482+0000 7ff3affff700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3b405f270 0x7ff3b4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.482+0000 7ff3affff700 1 -- 172.21.15.33:0/3613272588 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6242+0+0 (secure 0 0 0) 0x7ff3bc07b140 con 0x7ff3c814b730 2023-11-06T20:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.483+0000 7ff3c7fff700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3b405f270 0x7ff3b4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.483+0000 7ff3c7fff700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3b405f270 0x7ff3b4061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff3b8001ed0 tx=0x7ff3b8015000).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.487+0000 7ff3affff700 1 -- 172.21.15.33:0/3613272588 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff3bc04c150 con 0x7ff3c814b730 2023-11-06T20:59:46.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:46.669+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/3613272588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff3c806c6c0 con 0x7ff3c814b730 2023-11-06T20:59:47.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.240+0000 7ff3affff700 1 -- 172.21.15.33:0/3613272588 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff3bc0180d0 con 0x7ff3c814b730 2023-11-06T20:59:47.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.243+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3b405f270 msgr2=0x7ff3b4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.243+0000 7ff3ce7ff700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3b405f270 0x7ff3b4061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff3b8001ed0 tx=0x7ff3b8015000).stop 2023-11-06T20:59:47.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.243+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/3613272588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c814b730 msgr2=0x7ff3c80ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.243+0000 7ff3ce7ff700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c814b730 0x7ff3c80ae720 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7ff3bc0119f0 tx=0x7ff3bc009060).stop 2023-11-06T20:59:47.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.244+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/3613272588 shutdown_connections 2023-11-06T20:59:47.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.244+0000 7ff3ce7ff700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3b405f270 0x7ff3b4061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.244+0000 7ff3ce7ff700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3b00038e0 0x7ff3c80b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.244+0000 7ff3ce7ff700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3c8140780 0x7ff3c8152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.244+0000 7ff3ce7ff700 1 --2- 172.21.15.33:0/3613272588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c814b730 0x7ff3c80ae720 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.244+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/3613272588 >> 172.21.15.33:0/3613272588 conn(0x7ff3c805e1b0 msgr2=0x7ff3c80ab070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.244+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/3613272588 shutdown_connections 2023-11-06T20:59:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.244+0000 7ff3ce7ff700 1 -- 172.21.15.33:0/3613272588 wait complete. 2023-11-06T20:59:47.247 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T20:59:47.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T20:59:47.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.474+0000 7f16d17ff700 1 Processor -- start 2023-11-06T20:59:47.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.474+0000 7f16d17ff700 1 -- start start 2023-11-06T20:59:47.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.476+0000 7f16d17ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f16cc157700 0x7f16cc0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.476+0000 7f16d17ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc0aaf50 0x7f16cc063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.476+0000 7f16d17ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16cc158c00 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.476+0000 7f16d17ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16cc0c18f0 con 0x7f16cc157700 2023-11-06T20:59:47.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.476+0000 7f16d17ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0c6d80 con 0x7f16cc0aaf50 2023-11-06T20:59:47.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.476+0000 7f16ca7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc0aaf50 0x7f16cc063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.476+0000 7f16caffd700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f16cc157700 0x7f16cc0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.476+0000 7f16cb7fe700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f16cc0aa4f0 0x7f16cc0646d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:52200/0 (socket says 172.21.15.33:52200) 2023-11-06T20:59:47.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.476+0000 7f16cb7fe700 1 -- 172.21.15.33:0/1461795484 learned_addr learned my addr 172.21.15.33:0/1461795484 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:47.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.477+0000 7f16caffd700 1 -- 172.21.15.33:0/1461795484 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc0aaf50 msgr2=0x7f16cc063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:47.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.477+0000 7f16caffd700 1 --2- 172.21.15.33:0/1461795484 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc0aaf50 0x7f16cc063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.477+0000 7f16caffd700 1 -- 172.21.15.33:0/1461795484 >> v1:172.21.15.69:6789/0 conn(0x7f16cc0aa4f0 legacy=0x7f16cc0646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.477+0000 7f16c9ffb700 1 -- 172.21.15.33:0/1461795484 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912397764 0 0) 0x7f16cc158c00 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.477+0000 7f16caffd700 1 -- 172.21.15.33:0/1461795484 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc157ff0 con 0x7f16cc157700 2023-11-06T20:59:47.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.477+0000 7f16caffd700 1 --2- 172.21.15.33:0/1461795484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16cc157700 0x7f16cc0c6f00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f16bc009790 tx=0x7f16bc009060).ready entity=mon.1 client_cookie=b8c1a3427e87036d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.477+0000 7f16c9ffb700 1 -- 172.21.15.33:0/1461795484 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00f510 con 0x7f16cc157700 2023-11-06T20:59:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.477+0000 7f16c9ffb700 1 -- 172.21.15.33:0/1461795484 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc00f670 con 0x7f16cc157700 2023-11-06T20:59:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.477+0000 7f16c9ffb700 1 -- 172.21.15.33:0/1461795484 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00f9f0 con 0x7f16cc157700 2023-11-06T20:59:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.478+0000 7f16d17ff700 1 -- 172.21.15.33:0/1461795484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16cc157700 msgr2=0x7f16cc0c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.478+0000 7f16d17ff700 1 --2- 172.21.15.33:0/1461795484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16cc157700 0x7f16cc0c6f00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f16bc009790 tx=0x7f16bc009060).stop 2023-11-06T20:59:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.478+0000 7f16d17ff700 1 -- 172.21.15.33:0/1461795484 shutdown_connections 2023-11-06T20:59:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.478+0000 7f16d17ff700 1 --2- 172.21.15.33:0/1461795484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16cc157700 0x7f16cc0c6f00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.478+0000 7f16d17ff700 1 --2- 172.21.15.33:0/1461795484 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc0aaf50 0x7f16cc063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.478+0000 7f16d17ff700 1 -- 172.21.15.33:0/1461795484 >> 172.21.15.33:0/1461795484 conn(0x7f16cc05ec80 msgr2=0x7f16cc0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.478+0000 7f16d17ff700 1 -- 172.21.15.33:0/1461795484 shutdown_connections 2023-11-06T20:59:47.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.478+0000 7f16d17ff700 1 -- 172.21.15.33:0/1461795484 wait complete. 2023-11-06T20:59:47.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.480+0000 7f16d17ff700 1 Processor -- start 2023-11-06T20:59:47.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.480+0000 7f16d17ff700 1 -- start start 2023-11-06T20:59:47.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16d17ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc0aa4f0 0x7f16cc165360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16d17ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16cc0aaf50 0x7f16cc0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16d17ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc157700 0x7f16cc14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16d17ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0aeec0 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16d17ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc165850 con 0x7f16cc0aaf50 2023-11-06T20:59:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16d17ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc1560c0 con 0x7f16cc157700 2023-11-06T20:59:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16caffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc0aa4f0 0x7f16cc165360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16caffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc0aa4f0 0x7f16cc165360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51678/0 (socket says 172.21.15.33:51678) 2023-11-06T20:59:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16caffd700 1 -- 172.21.15.33:0/198835572 learned_addr learned my addr 172.21.15.33:0/198835572 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16ca7fc700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16cc0aaf50 0x7f16cc0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16cb7fe700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc157700 0x7f16cc14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16caffd700 1 -- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc157700 msgr2=0x7f16cc14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16caffd700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc157700 0x7f16cc14de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16caffd700 1 -- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16cc0aaf50 msgr2=0x7f16cc0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16caffd700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16cc0aaf50 0x7f16cc0ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16caffd700 1 -- 172.21.15.33:0/198835572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc0092a0 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.481+0000 7f16cb7fe700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc157700 0x7f16cc14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.482+0000 7f16caffd700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc0aa4f0 0x7f16cc165360 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f16bc009e30 tx=0x7f16bc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.482+0000 7f16b2ffd700 1 -- 172.21.15.33:0/198835572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00f7e0 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.482+0000 7f16b2ffd700 1 -- 172.21.15.33:0/198835572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc00f940 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.482+0000 7f16b2ffd700 1 -- 172.21.15.33:0/198835572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc01b660 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.482+0000 7f16d17ff700 1 -- 172.21.15.33:0/198835572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc0ab980 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.482+0000 7f16d17ff700 1 -- 172.21.15.33:0/198835572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc0abc60 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.483+0000 7f16d17ff700 1 -- 172.21.15.33:0/198835572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16cc06c6c0 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.484+0000 7f16b2ffd700 1 -- 172.21.15.33:0/198835572 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f16bc01ba50 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.485+0000 7f16b2ffd700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16b805f270 0x7f16b8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.485+0000 7f16b2ffd700 1 -- 172.21.15.33:0/198835572 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5828+0+0 (secure 0 0 0) 0x7f16bc07bdb0 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.485+0000 7f16ca7fc700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16b805f270 0x7f16b8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.486+0000 7f16ca7fc700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16b805f270 0x7f16b8061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f16cc0ae540 tx=0x7f16c0009060).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:47.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.489+0000 7f16b2ffd700 1 -- 172.21.15.33:0/198835572 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f16bc080070 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.671+0000 7f16d17ff700 1 -- 172.21.15.33:0/198835572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f16cc0bfe50 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.675+0000 7f16b2ffd700 1 -- 172.21.15.33:0/198835572 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v12) v1 ==== 120+0+0 (secure 0 0 0) 0x7f16bc04cf60 con 0x7f16cc0aa4f0 2023-11-06T20:59:47.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 -- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16b805f270 msgr2=0x7f16b8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16b805f270 0x7f16b8061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f16cc0ae540 tx=0x7f16c0009060).stop 2023-11-06T20:59:47.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 -- 172.21.15.33:0/198835572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc0aa4f0 msgr2=0x7f16cc165360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc0aa4f0 0x7f16cc165360 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f16bc009e30 tx=0x7f16bc017000).stop 2023-11-06T20:59:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 -- 172.21.15.33:0/198835572 shutdown_connections 2023-11-06T20:59:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f16b805f270 0x7f16b8061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f16cc157700 0x7f16cc14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc0aa4f0 0x7f16cc165360 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 --2- 172.21.15.33:0/198835572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16cc0aaf50 0x7f16cc0ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 -- 172.21.15.33:0/198835572 >> 172.21.15.33:0/198835572 conn(0x7f16cc05ec80 msgr2=0x7f16cc151200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:47.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 -- 172.21.15.33:0/198835572 shutdown_connections 2023-11-06T20:59:47.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.677+0000 7f16d17ff700 1 -- 172.21.15.33:0/198835572 wait complete. 2023-11-06T20:59:47.695 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T20:59:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.893+0000 7fc3b6dff700 1 Processor -- start 2023-11-06T20:59:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.894+0000 7fc3b6dff700 1 -- start start 2023-11-06T20:59:47.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.895+0000 7fc3b6dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3b00a48e0 0x7fc3b0155280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.895+0000 7fc3b6dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3b0155770 0x7fc3b015e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.895+0000 7fc3b6dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b00a5030 con 0x7fc3b013aae0 2023-11-06T20:59:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.895+0000 7fc3b6dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b0065c80 con 0x7fc3b00a48e0 2023-11-06T20:59:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.895+0000 7fc3b6dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00bce10 con 0x7fc3b0155770 2023-11-06T20:59:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.895+0000 7fc3b4b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc3b013aae0 0x7fc3b0154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40698/0 (socket says 172.21.15.33:40698) 2023-11-06T20:59:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.895+0000 7fc3affff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3b00a48e0 0x7fc3b0155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.895+0000 7fc3b4b9b700 1 -- 172.21.15.33:0/2259392878 learned_addr learned my addr 172.21.15.33:0/2259392878 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:47.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.896+0000 7fc3b539c700 1 --2- 172.21.15.33:0/2259392878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3b0155770 0x7fc3b015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.896+0000 7fc3af7fe700 1 -- 172.21.15.33:0/2259392878 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2870149477 0 0) 0x7fc3b00a5030 con 0x7fc3b013aae0 2023-11-06T20:59:47.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.896+0000 7fc3affff700 1 -- 172.21.15.33:0/2259392878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3b0155770 msgr2=0x7fc3b015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:47.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.896+0000 7fc3affff700 1 --2- 172.21.15.33:0/2259392878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3b0155770 0x7fc3b015e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.896+0000 7fc3affff700 1 -- 172.21.15.33:0/2259392878 >> v1:172.21.15.33:6789/0 conn(0x7fc3b013aae0 legacy=0x7fc3b0154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.896+0000 7fc3affff700 1 -- 172.21.15.33:0/2259392878 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b00a4d60 con 0x7fc3b00a48e0 2023-11-06T20:59:47.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.896+0000 7fc3affff700 1 --2- 172.21.15.33:0/2259392878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 0x7fc3b0155280 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fc39c009760 tx=0x7fc39c009060).ready entity=mon.0 client_cookie=2655a2c3e70c0347 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:47.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.896+0000 7fc3af7fe700 1 -- 172.21.15.33:0/2259392878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c00b580 con 0x7fc3b00a48e0 2023-11-06T20:59:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.896+0000 7fc3af7fe700 1 -- 172.21.15.33:0/2259392878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c00b6e0 con 0x7fc3b00a48e0 2023-11-06T20:59:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.896+0000 7fc3af7fe700 1 -- 172.21.15.33:0/2259392878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c00ba60 con 0x7fc3b00a48e0 2023-11-06T20:59:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.897+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2259392878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 msgr2=0x7fc3b0155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.897+0000 7fc3b6dff700 1 --2- 172.21.15.33:0/2259392878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 0x7fc3b0155280 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fc39c009760 tx=0x7fc39c009060).stop 2023-11-06T20:59:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.897+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2259392878 shutdown_connections 2023-11-06T20:59:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.897+0000 7fc3b6dff700 1 --2- 172.21.15.33:0/2259392878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3b0155770 0x7fc3b015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.897+0000 7fc3b6dff700 1 --2- 172.21.15.33:0/2259392878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 0x7fc3b0155280 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.897+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2259392878 >> 172.21.15.33:0/2259392878 conn(0x7fc3b00c6ea0 msgr2=0x7fc3b00c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.897+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2259392878 shutdown_connections 2023-11-06T20:59:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.897+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2259392878 wait complete. 2023-11-06T20:59:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.898+0000 7fc3b6dff700 1 Processor -- start 2023-11-06T20:59:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.898+0000 7fc3b6dff700 1 -- start start 2023-11-06T20:59:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.899+0000 7fc3b6dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 0x7fc3b0138b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.899+0000 7fc3b6dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3b013aae0 0x7fc3b013d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.899+0000 7fc3b6dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3b0155770 0x7fc3b00a6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.899+0000 7fc3b6dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00a9030 con 0x7fc3b00a48e0 2023-11-06T20:59:47.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.899+0000 7fc3b6dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00a9310 con 0x7fc3b0155770 2023-11-06T20:59:47.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.899+0000 7fc3b6dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00a95f0 con 0x7fc3b013aae0 2023-11-06T20:59:47.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3b4b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 0x7fc3b0138b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3b4b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 0x7fc3b0138b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51702/0 (socket says 172.21.15.33:51702) 2023-11-06T20:59:47.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3b4b9b700 1 -- 172.21.15.33:0/2174802797 learned_addr learned my addr 172.21.15.33:0/2174802797 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:47.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3affff700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3b013aae0 0x7fc3b013d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3b539c700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3b0155770 0x7fc3b00a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3b4b9b700 1 -- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3b013aae0 msgr2=0x7fc3b013d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3b4b9b700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3b013aae0 0x7fc3b013d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3b4b9b700 1 -- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3b0155770 msgr2=0x7fc3b00a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3b4b9b700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3b0155770 0x7fc3b00a6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3b4b9b700 1 -- 172.21.15.33:0/2174802797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c0092a0 con 0x7fc3b00a48e0 2023-11-06T20:59:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3b4b9b700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 0x7fc3b0138b10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fc3a400c4e0 tx=0x7fc3a4007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3acff9700 1 -- 172.21.15.33:0/2174802797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40072e0 con 0x7fc3b00a48e0 2023-11-06T20:59:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.900+0000 7fc3acff9700 1 -- 172.21.15.33:0/2174802797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a4007440 con 0x7fc3b00a48e0 2023-11-06T20:59:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.901+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2174802797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b00a98d0 con 0x7fc3b00a48e0 2023-11-06T20:59:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.901+0000 7fc3acff9700 1 -- 172.21.15.33:0/2174802797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40057a0 con 0x7fc3b00a48e0 2023-11-06T20:59:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.901+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2174802797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b013c8f0 con 0x7fc3b00a48e0 2023-11-06T20:59:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.902+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2174802797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b00a9a60 con 0x7fc3b00a48e0 2023-11-06T20:59:47.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.903+0000 7fc3acff9700 1 -- 172.21.15.33:0/2174802797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc3a4005bd0 con 0x7fc3b00a48e0 2023-11-06T20:59:47.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.904+0000 7fc3acff9700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc39405f270 0x7fc394061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:47.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.904+0000 7fc3acff9700 1 -- 172.21.15.33:0/2174802797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5828+0+0 (secure 0 0 0) 0x7fc3a4079d40 con 0x7fc3b00a48e0 2023-11-06T20:59:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.904+0000 7fc3affff700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc39405f270 0x7fc394061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.905+0000 7fc3affff700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc39405f270 0x7fc394061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fc3b0167b30 tx=0x7fc39c017000).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:47.907+0000 7fc3acff9700 1 -- 172.21.15.33:0/2174802797 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc3a407d030 con 0x7fc3b00a48e0 2023-11-06T20:59:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.091+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2174802797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc3b00640e0 con 0x7fc3b00a48e0 2023-11-06T20:59:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.095+0000 7fc3acff9700 1 -- 172.21.15.33:0/2174802797 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc3a4018020 con 0x7fc3b00a48e0 2023-11-06T20:59:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc39405f270 msgr2=0x7fc394061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc39405f270 0x7fc394061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fc3b0167b30 tx=0x7fc39c017000).stop 2023-11-06T20:59:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2174802797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 msgr2=0x7fc3b0138b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 0x7fc3b0138b10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fc3a400c4e0 tx=0x7fc3a4007c70).stop 2023-11-06T20:59:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2174802797 shutdown_connections 2023-11-06T20:59:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc39405f270 0x7fc394061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3b0155770 0x7fc3b00a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3b00a48e0 0x7fc3b0138b10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 --2- 172.21.15.33:0/2174802797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3b013aae0 0x7fc3b013d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2174802797 >> 172.21.15.33:0/2174802797 conn(0x7fc3b00c6ea0 msgr2=0x7fc3b0154b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2174802797 shutdown_connections 2023-11-06T20:59:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.097+0000 7fc3b6dff700 1 -- 172.21.15.33:0/2174802797 wait complete. 2023-11-06T20:59:48.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-11-06T20:59:48.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.312+0000 7f0a50fff700 1 Processor -- start 2023-11-06T20:59:48.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.312+0000 7f0a50fff700 1 -- start start 2023-11-06T20:59:48.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.314+0000 7f0a50fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 0x7f0a4c0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:48.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.314+0000 7f0a50fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a4c0bfd70 0x7f0a4c0c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:48.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.314+0000 7f0a50fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4c1554a0 con 0x7f0a4c0c1840 2023-11-06T20:59:48.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.314+0000 7f0a50fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4c154960 con 0x7f0a4c0bfd70 2023-11-06T20:59:48.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.314+0000 7f0a50fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0c66c0 con 0x7f0a4c0c5d50 2023-11-06T20:59:48.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.315+0000 7f0a4ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a4c0bfd70 0x7f0a4c0c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:48.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.315+0000 7f0a4ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a4c0bfd70 0x7f0a4c0c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47010/0 (socket says 172.21.15.33:47010) 2023-11-06T20:59:48.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.315+0000 7f0a4ad9d700 1 -- 172.21.15.33:0/2021000621 learned_addr learned my addr 172.21.15.33:0/2021000621 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:48.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.315+0000 7f0a49d9b700 1 --2- 172.21.15.33:0/2021000621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 0x7f0a4c0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:48.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.315+0000 7f0a4ad9d700 1 -- 172.21.15.33:0/2021000621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 msgr2=0x7f0a4c0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:48.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.315+0000 7f0a4ad9d700 1 --2- 172.21.15.33:0/2021000621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 0x7f0a4c0c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:48.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.315+0000 7f0a4ad9d700 1 -- 172.21.15.33:0/2021000621 >> v1:172.21.15.69:6789/0 conn(0x7f0a4c0c1840 legacy=0x7f0a4c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:48.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.315+0000 7f0a4ad9d700 1 -- 172.21.15.33:0/2021000621 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c155930 con 0x7f0a4c0bfd70 2023-11-06T20:59:48.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.315+0000 7f0a4ad9d700 1 --2- 172.21.15.33:0/2021000621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0bfd70 0x7f0a4c0c01e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f0a44009f20 tx=0x7f0a44009840).ready entity=mon.0 client_cookie=e027c23ea3e7d9a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:48.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.315+0000 7f0a4959a700 1 -- 172.21.15.33:0/2021000621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400e070 con 0x7f0a4c0bfd70 2023-11-06T20:59:48.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.316+0000 7f0a4959a700 1 -- 172.21.15.33:0/2021000621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4400b9d0 con 0x7f0a4c0bfd70 2023-11-06T20:59:48.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.316+0000 7f0a4959a700 1 -- 172.21.15.33:0/2021000621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400bd50 con 0x7f0a4c0bfd70 2023-11-06T20:59:48.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.316+0000 7f0a50fff700 1 -- 172.21.15.33:0/2021000621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0bfd70 msgr2=0x7f0a4c0c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:48.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.316+0000 7f0a50fff700 1 --2- 172.21.15.33:0/2021000621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0bfd70 0x7f0a4c0c01e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f0a44009f20 tx=0x7f0a44009840).stop 2023-11-06T20:59:48.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.316+0000 7f0a50fff700 1 -- 172.21.15.33:0/2021000621 shutdown_connections 2023-11-06T20:59:48.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.316+0000 7f0a50fff700 1 --2- 172.21.15.33:0/2021000621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0bfd70 0x7f0a4c0c01e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:48.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.316+0000 7f0a50fff700 1 --2- 172.21.15.33:0/2021000621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 0x7f0a4c0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:48.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.316+0000 7f0a50fff700 1 -- 172.21.15.33:0/2021000621 >> 172.21.15.33:0/2021000621 conn(0x7f0a4c05ec80 msgr2=0x7f0a4c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:48.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.316+0000 7f0a50fff700 1 -- 172.21.15.33:0/2021000621 shutdown_connections 2023-11-06T20:59:48.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.316+0000 7f0a50fff700 1 -- 172.21.15.33:0/2021000621 wait complete. 2023-11-06T20:59:48.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.318+0000 7f0a50fff700 1 Processor -- start 2023-11-06T20:59:48.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.319+0000 7f0a50fff700 1 -- start start 2023-11-06T20:59:48.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a50fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c0bfd70 0x7f0a4c149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:48.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a50fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0c1840 0x7f0a4c14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a50fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 0x7f0a4c151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a50fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0c3710 con 0x7f0a4c0c1840 2023-11-06T20:59:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a50fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0c2c40 con 0x7f0a4c0bfd70 2023-11-06T20:59:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a50fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0c2030 con 0x7f0a4c0c5d50 2023-11-06T20:59:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a49d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0c1840 0x7f0a4c14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:48.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a49d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0c1840 0x7f0a4c14f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47020/0 (socket says 172.21.15.33:47020) 2023-11-06T20:59:48.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a49d9b700 1 -- 172.21.15.33:0/3883923307 learned_addr learned my addr 172.21.15.33:0/3883923307 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:48.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a4a59c700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c0bfd70 0x7f0a4c149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:48.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.320+0000 7f0a4ad9d700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 0x7f0a4c151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:48.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a49d9b700 1 -- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 msgr2=0x7f0a4c151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a49d9b700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 0x7f0a4c151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a49d9b700 1 -- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c0bfd70 msgr2=0x7f0a4c149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a49d9b700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c0bfd70 0x7f0a4c149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a49d9b700 1 -- 172.21.15.33:0/3883923307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a44009ae0 con 0x7f0a4c0c1840 2023-11-06T20:59:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a4ad9d700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 0x7f0a4c151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:48.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a49d9b700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0c1840 0x7f0a4c14f3f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f0a4000e900 tx=0x7f0a40009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:48.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a3a7fc700 1 -- 172.21.15.33:0/3883923307 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40012600 con 0x7f0a4c0c1840 2023-11-06T20:59:48.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a3a7fc700 1 -- 172.21.15.33:0/3883923307 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a40012760 con 0x7f0a4c0c1840 2023-11-06T20:59:48.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a50fff700 1 -- 172.21.15.33:0/3883923307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4c0c2310 con 0x7f0a4c0c1840 2023-11-06T20:59:48.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a3a7fc700 1 -- 172.21.15.33:0/3883923307 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40012ae0 con 0x7f0a4c0c1840 2023-11-06T20:59:48.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.321+0000 7f0a50fff700 1 -- 172.21.15.33:0/3883923307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4c0c28d0 con 0x7f0a4c0c1840 2023-11-06T20:59:48.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.322+0000 7f0a50fff700 1 -- 172.21.15.33:0/3883923307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4c0c2db0 con 0x7f0a4c0c1840 2023-11-06T20:59:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.323+0000 7f0a3a7fc700 1 -- 172.21.15.33:0/3883923307 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0a4001b400 con 0x7f0a4c0c1840 2023-11-06T20:59:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.324+0000 7f0a3a7fc700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0a2c05f270 0x7f0a2c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.324+0000 7f0a3a7fc700 1 -- 172.21.15.33:0/3883923307 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5828+0+0 (secure 0 0 0) 0x7f0a4007a700 con 0x7f0a4c0c1840 2023-11-06T20:59:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.325+0000 7f0a4a59c700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0a2c05f270 0x7f0a2c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.325+0000 7f0a4a59c700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0a2c05f270 0x7f0a2c061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0a3c003900 tx=0x7f0a3c007d40).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.328+0000 7f0a3a7fc700 1 -- 172.21.15.33:0/3883923307 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0a4004b850 con 0x7f0a4c0c1840 2023-11-06T20:59:48.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:48.581+0000 7f0a50fff700 1 -- 172.21.15.33:0/3883923307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f0a4c06c6c0 con 0x7f0a4c0c1840 2023-11-06T20:59:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.250+0000 7f0a3a7fc700 1 -- 172.21.15.33:0/3883923307 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v571) v1 ==== 156+0+0 (secure 0 0 0) 0x7f0a4004ee70 con 0x7f0a4c0c1840 2023-11-06T20:59:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 -- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0a2c05f270 msgr2=0x7f0a2c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:49.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0a2c05f270 0x7f0a2c061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0a3c003900 tx=0x7f0a3c007d40).stop 2023-11-06T20:59:49.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 -- 172.21.15.33:0/3883923307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0c1840 msgr2=0x7f0a4c14f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:49.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0c1840 0x7f0a4c14f3f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f0a4000e900 tx=0x7f0a40009060).stop 2023-11-06T20:59:49.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 -- 172.21.15.33:0/3883923307 shutdown_connections 2023-11-06T20:59:49.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0a2c05f270 0x7f0a2c061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:49.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a4c0c5d50 0x7f0a4c151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:49.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c0bfd70 0x7f0a4c149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:49.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 --2- 172.21.15.33:0/3883923307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4c0c1840 0x7f0a4c14f3f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:49.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 -- 172.21.15.33:0/3883923307 >> 172.21.15.33:0/3883923307 conn(0x7f0a4c05ec80 msgr2=0x7f0a4c0ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:49.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 -- 172.21.15.33:0/3883923307 shutdown_connections 2023-11-06T20:59:49.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.253+0000 7f0a50fff700 1 -- 172.21.15.33:0/3883923307 wait complete. 2023-11-06T20:59:49.272 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-11-06T20:59:49.273 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-11-06T20:59:49.273 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-06T20:59:49.273 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-11-06T20:59:49.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.473+0000 7f876c3ff700 1 Processor -- start 2023-11-06T20:59:49.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.473+0000 7f876c3ff700 1 -- start start 2023-11-06T20:59:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.474+0000 7f876c3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f87640c1840 0x7f87640a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.474+0000 7f876c3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87640c6460 0x7f87640bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.474+0000 7f876c3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87641554a0 con 0x7f87640aa2d0 2023-11-06T20:59:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.474+0000 7f876c3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8764154960 con 0x7f87640c6460 2023-11-06T20:59:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.474+0000 7f876c3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8764155930 con 0x7f87640c1840 2023-11-06T20:59:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.475+0000 7f876a99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87640c6460 0x7f87640bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.475+0000 7f876a99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87640c6460 0x7f87640bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47028/0 (socket says 172.21.15.33:47028) 2023-11-06T20:59:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.475+0000 7f876a99c700 1 -- 172.21.15.33:0/3165096501 learned_addr learned my addr 172.21.15.33:0/3165096501 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.475+0000 7f876a19b700 1 --2- 172.21.15.33:0/3165096501 >> v2:172.21.15.69:3300/0 conn(0x7f87640c1840 0x7f87640a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:49.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.475+0000 7f876a99c700 1 -- 172.21.15.33:0/3165096501 >> v2:172.21.15.69:3300/0 conn(0x7f87640c1840 msgr2=0x7f87640a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:49.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.475+0000 7f876a99c700 1 --2- 172.21.15.33:0/3165096501 >> v2:172.21.15.69:3300/0 conn(0x7f87640c1840 0x7f87640a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:49.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.475+0000 7f876a99c700 1 -- 172.21.15.33:0/3165096501 >> v1:172.21.15.69:6789/0 conn(0x7f87640aa2d0 legacy=0x7f87640c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:49.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.475+0000 7f8769199700 1 -- 172.21.15.33:0/3165096501 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405806709 0 0) 0x7f87641554a0 con 0x7f87640aa2d0 2023-11-06T20:59:49.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.475+0000 7f876a99c700 1 -- 172.21.15.33:0/3165096501 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764154b70 con 0x7f87640c6460 2023-11-06T20:59:49.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f876a99c700 1 --2- 172.21.15.33:0/3165096501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c6460 0x7f87640bfd70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f87640840b0 tx=0x7f875c009840).ready entity=mon.0 client_cookie=2085e4ec6ad994e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:49.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f8769199700 1 -- 172.21.15.33:0/3165096501 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00e070 con 0x7f87640c6460 2023-11-06T20:59:49.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f8769199700 1 -- 172.21.15.33:0/3165096501 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00ba20 con 0x7f87640c6460 2023-11-06T20:59:49.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f8769199700 1 -- 172.21.15.33:0/3165096501 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00bda0 con 0x7f87640c6460 2023-11-06T20:59:49.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f876c3ff700 1 -- 172.21.15.33:0/3165096501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c6460 msgr2=0x7f87640bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:49.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f876c3ff700 1 --2- 172.21.15.33:0/3165096501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c6460 0x7f87640bfd70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f87640840b0 tx=0x7f875c009840).stop 2023-11-06T20:59:49.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f876c3ff700 1 -- 172.21.15.33:0/3165096501 shutdown_connections 2023-11-06T20:59:49.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f876c3ff700 1 --2- 172.21.15.33:0/3165096501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640c6460 0x7f87640bfd70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f876c3ff700 1 --2- 172.21.15.33:0/3165096501 >> v2:172.21.15.69:3300/0 conn(0x7f87640c1840 0x7f87640a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f876c3ff700 1 -- 172.21.15.33:0/3165096501 >> 172.21.15.33:0/3165096501 conn(0x7f876405ec80 msgr2=0x7f87640cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.476+0000 7f876c3ff700 1 -- 172.21.15.33:0/3165096501 shutdown_connections 2023-11-06T20:59:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.477+0000 7f876c3ff700 1 -- 172.21.15.33:0/3165096501 wait complete. 2023-11-06T20:59:49.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.478+0000 7f876c3ff700 1 Processor -- start 2023-11-06T20:59:49.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.478+0000 7f876c3ff700 1 -- start start 2023-11-06T20:59:49.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876c3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640aa2d0 0x7f87640c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:49.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876c3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87640c1840 0x7f876414f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:49.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876c3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87640c6460 0x7f87640addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:49.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876c3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8764155ee0 con 0x7f87640aa2d0 2023-11-06T20:59:49.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876c3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c2690 con 0x7f87640c1840 2023-11-06T20:59:49.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876c3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c2970 con 0x7f87640c6460 2023-11-06T20:59:49.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876a19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640aa2d0 0x7f87640c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:49.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876a19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640aa2d0 0x7f87640c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47036/0 (socket says 172.21.15.33:47036) 2023-11-06T20:59:49.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876a19b700 1 -- 172.21.15.33:0/2849264461 learned_addr learned my addr 172.21.15.33:0/2849264461 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:49.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876999a700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87640c1840 0x7f876414f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:49.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.479+0000 7f876a99c700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87640c6460 0x7f87640addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:49.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f876a19b700 1 -- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87640c6460 msgr2=0x7f87640addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:49.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f876a19b700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87640c6460 0x7f87640addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:49.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f876a19b700 1 -- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87640c1840 msgr2=0x7f876414f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:49.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f876a19b700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87640c1840 0x7f876414f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:49.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f876a19b700 1 -- 172.21.15.33:0/2849264461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c009ae0 con 0x7f87640aa2d0 2023-11-06T20:59:49.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f876a19b700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640aa2d0 0x7f87640c1eb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f876000e950 tx=0x7f8760009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:49.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f875a7fc700 1 -- 172.21.15.33:0/2849264461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876000c850 con 0x7f87640aa2d0 2023-11-06T20:59:49.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f875a7fc700 1 -- 172.21.15.33:0/2849264461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876000c9b0 con 0x7f87640aa2d0 2023-11-06T20:59:49.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f876c3ff700 1 -- 172.21.15.33:0/2849264461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8764155b70 con 0x7f87640aa2d0 2023-11-06T20:59:49.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f875a7fc700 1 -- 172.21.15.33:0/2849264461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876000cd30 con 0x7f87640aa2d0 2023-11-06T20:59:49.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.480+0000 7f876c3ff700 1 -- 172.21.15.33:0/2849264461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87640c2c50 con 0x7f87640aa2d0 2023-11-06T20:59:49.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.481+0000 7f876c3ff700 1 -- 172.21.15.33:0/2849264461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876406ab10 con 0x7f87640aa2d0 2023-11-06T20:59:49.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.482+0000 7f875a7fc700 1 -- 172.21.15.33:0/2849264461 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8760012600 con 0x7f87640aa2d0 2023-11-06T20:59:49.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.483+0000 7f875a7fc700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f270 0x7f8748061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:49.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.483+0000 7f875a7fc700 1 -- 172.21.15.33:0/2849264461 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5828+0+0 (secure 0 0 0) 0x7f876007a8c0 con 0x7f87640aa2d0 2023-11-06T20:59:49.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.484+0000 7f876999a700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f270 0x7f8748061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:49.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.484+0000 7f876999a700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f270 0x7f8748061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f87640b08e0 tx=0x7f8754009040).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:49.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.487+0000 7f875a7fc700 1 -- 172.21.15.33:0/2849264461 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f876406ab10 con 0x7f87640aa2d0 2023-11-06T20:59:49.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:49.741+0000 7f876c3ff700 1 -- 172.21.15.33:0/2849264461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f876406ab10 con 0x7f87640aa2d0 2023-11-06T20:59:50.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.252+0000 7f875a7fc700 1 -- 172.21.15.33:0/2849264461 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v572) v1 ==== 169+0+0 (secure 0 0 0) 0x7f876406ab10 con 0x7f87640aa2d0 2023-11-06T20:59:50.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 -- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f270 msgr2=0x7f8748061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:50.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f270 0x7f8748061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f87640b08e0 tx=0x7f8754009040).stop 2023-11-06T20:59:50.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 -- 172.21.15.33:0/2849264461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640aa2d0 msgr2=0x7f87640c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:50.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640aa2d0 0x7f87640c1eb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f876000e950 tx=0x7f8760009060).stop 2023-11-06T20:59:50.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 -- 172.21.15.33:0/2849264461 shutdown_connections 2023-11-06T20:59:50.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f874805f270 0x7f8748061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87640c6460 0x7f87640addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87640aa2d0 0x7f87640c1eb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 --2- 172.21.15.33:0/2849264461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87640c1840 0x7f876414f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 -- 172.21.15.33:0/2849264461 >> 172.21.15.33:0/2849264461 conn(0x7f876405ec80 msgr2=0x7f8764152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:50.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 -- 172.21.15.33:0/2849264461 shutdown_connections 2023-11-06T20:59:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.255+0000 7f876c3ff700 1 -- 172.21.15.33:0/2849264461 wait complete. 2023-11-06T20:59:50.272 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T20:59:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.473+0000 7f55fb5ff700 1 Processor -- start 2023-11-06T20:59:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.473+0000 7f55fb5ff700 1 -- start start 2023-11-06T20:59:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.474+0000 7f55fb5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f4154330 0x7f55f40c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.474+0000 7f55fb5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f55f40c7ff0 0x7f55f40646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.474+0000 7f55fb5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f40c6b60 con 0x7f55f4154710 2023-11-06T20:59:50.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.474+0000 7f55fb5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f40c7700 con 0x7f55f40c7ff0 2023-11-06T20:59:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.474+0000 7f55fb5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40c5cd0 con 0x7f55f4154330 2023-11-06T20:59:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.475+0000 7f55f8b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f55f4154710 0x7f55f40c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35852/0 (socket says 172.21.15.33:35852) 2023-11-06T20:59:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.475+0000 7f55f8b9a700 1 -- 172.21.15.33:0/1516428754 learned_addr learned my addr 172.21.15.33:0/1516428754 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.475+0000 7f55f939b700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f4154330 0x7f55f40c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.475+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577999344 0 0) 0x7f55f40c6b60 con 0x7f55f4154710 2023-11-06T20:59:50.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.475+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55e00033f0 con 0x7f55f4154710 2023-11-06T20:59:50.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.475+0000 7f55f9b9c700 1 --2- 172.21.15.33:0/1516428754 >> v2:172.21.15.69:3300/0 conn(0x7f55f40c7ff0 0x7f55f40646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:50.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.475+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1647467652 0 0) 0x7f55e00033f0 con 0x7f55f4154710 2023-11-06T20:59:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.475+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f4154330 msgr2=0x7f55f40c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.475+0000 7f55ebfff700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f4154330 0x7f55f40c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.475+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 >> v2:172.21.15.69:3300/0 conn(0x7f55f40c7ff0 msgr2=0x7f55f40646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55ebfff700 1 --2- 172.21.15.33:0/1516428754 >> v2:172.21.15.69:3300/0 conn(0x7f55f40c7ff0 0x7f55f40646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40c5f50 con 0x7f55f4154710 2023-11-06T20:59:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f55e4001470 con 0x7f55f4154710 2023-11-06T20:59:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 >> v1:172.21.15.33:6789/0 conn(0x7f55f4154710 legacy=0x7f55f40c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55ebfff700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e00038e0 0x7f55e0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55ebfff700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55e0004690 0x7f55e0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:50.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55ebfff700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f4154330 0x7f55e0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:50.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40c7700 con 0x7f55e00038e0 2023-11-06T20:59:50.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40c5cd0 con 0x7f55e0004690 2023-11-06T20:59:50.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e000b8e0 con 0x7f55f4154330 2023-11-06T20:59:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55f939b700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e00038e0 0x7f55e0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55f9b9c700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f4154330 0x7f55e0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55f8b9a700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55e0004690 0x7f55e0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55f939b700 1 -- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f4154330 msgr2=0x7f55e0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55f939b700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f4154330 0x7f55e0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55f939b700 1 -- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55e0004690 msgr2=0x7f55e0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55f939b700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55e0004690 0x7f55e0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55f939b700 1 -- 172.21.15.33:0/1516428754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00033f0 con 0x7f55e00038e0 2023-11-06T20:59:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.476+0000 7f55f8b9a700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55e0004690 0x7f55e0004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55f939b700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e00038e0 0x7f55e0003d00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f55f0009a80 tx=0x7f55f0009060).ready entity=mon.0 client_cookie=a2f2dee26244c7a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0009430 con 0x7f55e00038e0 2023-11-06T20:59:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f000e640 con 0x7f55e00038e0 2023-11-06T20:59:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55ebfff700 1 -- 172.21.15.33:0/1516428754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f000e9c0 con 0x7f55e00038e0 2023-11-06T20:59:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55fb5ff700 1 -- 172.21.15.33:0/1516428754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e00038e0 msgr2=0x7f55e0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55fb5ff700 1 --2- 172.21.15.33:0/1516428754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e00038e0 0x7f55e0003d00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f55f0009a80 tx=0x7f55f0009060).stop 2023-11-06T20:59:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55f9b9c700 1 -- 172.21.15.33:0/1516428754 reap_dead start 2023-11-06T20:59:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55fb5ff700 1 -- 172.21.15.33:0/1516428754 shutdown_connections 2023-11-06T20:59:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55fb5ff700 1 -- 172.21.15.33:0/1516428754 >> 172.21.15.33:0/1516428754 conn(0x7f55f40cb910 msgr2=0x7f55f40c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55fb5ff700 1 -- 172.21.15.33:0/1516428754 shutdown_connections 2023-11-06T20:59:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.477+0000 7f55fb5ff700 1 -- 172.21.15.33:0/1516428754 wait complete. 2023-11-06T20:59:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.479+0000 7f55fb5ff700 1 Processor -- start 2023-11-06T20:59:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.479+0000 7f55fb5ff700 1 -- start start 2023-11-06T20:59:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.480+0000 7f55fb5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55e00038e0 0x7f55f4145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.480+0000 7f55fb5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4156b50 0x7f55f4156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.480+0000 7f55fb5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f40aed30 0x7f55f40af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.480+0000 7f55fb5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4156750 con 0x7f55f4156b50 2023-11-06T20:59:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.480+0000 7f55fb5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f41574c0 con 0x7f55e00038e0 2023-11-06T20:59:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.480+0000 7f55fb5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40adf70 con 0x7f55f40aed30 2023-11-06T20:59:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.480+0000 7f55f8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4156b50 0x7f55f4156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.480+0000 7f55f8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4156b50 0x7f55f4156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47052/0 (socket says 172.21.15.33:47052) 2023-11-06T20:59:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.480+0000 7f55f8b9a700 1 -- 172.21.15.33:0/3690130350 learned_addr learned my addr 172.21.15.33:0/3690130350 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55f939b700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55e00038e0 0x7f55f4145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55f9b9c700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f40aed30 0x7f55f40af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55f8b9a700 1 -- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f40aed30 msgr2=0x7f55f40af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55f8b9a700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f40aed30 0x7f55f40af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55f8b9a700 1 -- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55e00038e0 msgr2=0x7f55f4145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55f8b9a700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55e00038e0 0x7f55f4145540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55f8b9a700 1 -- 172.21.15.33:0/3690130350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40ae250 con 0x7f55f4156b50 2023-11-06T20:59:50.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55f8b9a700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4156b50 0x7f55f4156fd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f55e400e960 tx=0x7f55e40014e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:50.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55e97fa700 1 -- 172.21.15.33:0/3690130350 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e400c760 con 0x7f55f4156b50 2023-11-06T20:59:50.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55e97fa700 1 -- 172.21.15.33:0/3690130350 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e400c8c0 con 0x7f55f4156b50 2023-11-06T20:59:50.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55e97fa700 1 -- 172.21.15.33:0/3690130350 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e400cc40 con 0x7f55f4156b50 2023-11-06T20:59:50.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55fb5ff700 1 -- 172.21.15.33:0/3690130350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f40ae530 con 0x7f55f4156b50 2023-11-06T20:59:50.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.481+0000 7f55fb5ff700 1 -- 172.21.15.33:0/3690130350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f40af6a0 con 0x7f55f4156b50 2023-11-06T20:59:50.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.482+0000 7f55fb5ff700 1 -- 172.21.15.33:0/3690130350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f40830b0 con 0x7f55f4156b50 2023-11-06T20:59:50.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.483+0000 7f55e97fa700 1 -- 172.21.15.33:0/3690130350 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f55e4014640 con 0x7f55f4156b50 2023-11-06T20:59:50.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.485+0000 7f55e97fa700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f55d805f270 0x7f55d8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:50.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.485+0000 7f55e97fa700 1 -- 172.21.15.33:0/3690130350 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5828+0+0 (secure 0 0 0) 0x7f55e4079fe0 con 0x7f55f4156b50 2023-11-06T20:59:50.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.485+0000 7f55f939b700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f55d805f270 0x7f55d8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:50.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.486+0000 7f55f939b700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f55d805f270 0x7f55d8061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f55f0006d50 tx=0x7f55f001a000).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:50.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.489+0000 7f55e97fa700 1 -- 172.21.15.33:0/3690130350 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f55e4049af0 con 0x7f55f4156b50 2023-11-06T20:59:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.669+0000 7f55fb5ff700 1 -- 172.21.15.33:0/3690130350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f55f4062b00 con 0x7f55f4156b50 2023-11-06T20:59:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.673+0000 7f55e97fa700 1 -- 172.21.15.33:0/3690130350 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f55e404b1f0 con 0x7f55f4156b50 2023-11-06T20:59:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 -- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f55d805f270 msgr2=0x7f55d8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f55d805f270 0x7f55d8061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f55f0006d50 tx=0x7f55f001a000).stop 2023-11-06T20:59:50.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 -- 172.21.15.33:0/3690130350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4156b50 msgr2=0x7f55f4156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:50.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4156b50 0x7f55f4156fd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f55e400e960 tx=0x7f55e40014e0).stop 2023-11-06T20:59:50.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 -- 172.21.15.33:0/3690130350 shutdown_connections 2023-11-06T20:59:50.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f55d805f270 0x7f55d8061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55f40aed30 0x7f55f40af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55e00038e0 0x7f55f4145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 --2- 172.21.15.33:0/3690130350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4156b50 0x7f55f4156fd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 -- 172.21.15.33:0/3690130350 >> 172.21.15.33:0/3690130350 conn(0x7f55f40cb910 msgr2=0x7f55f4152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.675+0000 7f55fb5ff700 1 -- 172.21.15.33:0/3690130350 shutdown_connections 2023-11-06T20:59:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:50.676+0000 7f55fb5ff700 1 -- 172.21.15.33:0/3690130350 wait complete. 2023-11-06T20:59:50.692 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T20:59:50.693 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:59:50.783 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T20:59:50.876 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T20:59:50.877 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T20:59:50.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T20:59:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.074+0000 7f735a1ff700 1 Processor -- start 2023-11-06T20:59:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.075+0000 7f735a1ff700 1 -- start start 2023-11-06T20:59:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.076+0000 7f735a1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73540c4e60 0x7f7354066540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.076+0000 7f735a1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7354066a30 0x7f73540bdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.076+0000 7f735a1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73541553b0 con 0x7f73540c5240 2023-11-06T20:59:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.076+0000 7f735a1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73540c45c0 con 0x7f73540c4e60 2023-11-06T20:59:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.076+0000 7f735a1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7354154ac0 con 0x7f7354066a30 2023-11-06T20:59:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.076+0000 7f73537fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73540c4e60 0x7f7354066540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.076+0000 7f73537fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73540c4e60 0x7f7354066540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47062/0 (socket says 172.21.15.33:47062) 2023-11-06T20:59:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.076+0000 7f73537fe700 1 -- 172.21.15.33:0/2779307279 learned_addr learned my addr 172.21.15.33:0/2779307279 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.076+0000 7f7352ffd700 1 --2- 172.21.15.33:0/2779307279 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7354066a30 0x7f73540bdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.077+0000 7f73537fe700 1 -- 172.21.15.33:0/2779307279 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7354066a30 msgr2=0x7f73540bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:51.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.077+0000 7f73537fe700 1 --2- 172.21.15.33:0/2779307279 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7354066a30 0x7f73540bdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.077+0000 7f73537fe700 1 -- 172.21.15.33:0/2779307279 >> v1:172.21.15.69:6789/0 conn(0x7f73540c5240 legacy=0x7f7354067a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:51.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.077+0000 7f73537fe700 1 -- 172.21.15.33:0/2779307279 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7354154d60 con 0x7f73540c4e60 2023-11-06T20:59:51.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.077+0000 7f73537fe700 1 --2- 172.21.15.33:0/2779307279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c4e60 0x7f7354066540 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f7344006fd0 tx=0x7f73440090f0).ready entity=mon.0 client_cookie=7e1e3153e7fa07e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:51.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.077+0000 7f73527fc700 1 -- 172.21.15.33:0/2779307279 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734400b600 con 0x7f73540c4e60 2023-11-06T20:59:51.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.077+0000 7f73527fc700 1 -- 172.21.15.33:0/2779307279 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734400b760 con 0x7f73540c4e60 2023-11-06T20:59:51.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.077+0000 7f73527fc700 1 -- 172.21.15.33:0/2779307279 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734400bae0 con 0x7f73540c4e60 2023-11-06T20:59:51.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.078+0000 7f735a1ff700 1 -- 172.21.15.33:0/2779307279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c4e60 msgr2=0x7f7354066540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:51.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.078+0000 7f735a1ff700 1 --2- 172.21.15.33:0/2779307279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c4e60 0x7f7354066540 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f7344006fd0 tx=0x7f73440090f0).stop 2023-11-06T20:59:51.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.078+0000 7f735a1ff700 1 -- 172.21.15.33:0/2779307279 shutdown_connections 2023-11-06T20:59:51.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.078+0000 7f735a1ff700 1 --2- 172.21.15.33:0/2779307279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c4e60 0x7f7354066540 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.078+0000 7f735a1ff700 1 --2- 172.21.15.33:0/2779307279 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7354066a30 0x7f73540bdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.078+0000 7f735a1ff700 1 -- 172.21.15.33:0/2779307279 >> 172.21.15.33:0/2779307279 conn(0x7f73540c8c10 msgr2=0x7f73540c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:51.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.078+0000 7f735a1ff700 1 -- 172.21.15.33:0/2779307279 shutdown_connections 2023-11-06T20:59:51.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.078+0000 7f735a1ff700 1 -- 172.21.15.33:0/2779307279 wait complete. 2023-11-06T20:59:51.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.080+0000 7f735a1ff700 1 Processor -- start 2023-11-06T20:59:51.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.080+0000 7f735a1ff700 1 -- start start 2023-11-06T20:59:51.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.081+0000 7f735a1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7354066a30 0x7f7354067f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.081+0000 7f735a1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73540c4e60 0x7f7354068420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.081+0000 7f735a1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c5240 0x7f735413a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.081+0000 7f735a1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7354068910 con 0x7f73540c5240 2023-11-06T20:59:51.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.081+0000 7f735a1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73540c6ca0 con 0x7f73540c4e60 2023-11-06T20:59:51.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.081+0000 7f735a1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73540c6ef0 con 0x7f7354066a30 2023-11-06T20:59:51.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f7353fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c5240 0x7f735413a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f7353fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c5240 0x7f735413a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47066/0 (socket says 172.21.15.33:47066) 2023-11-06T20:59:51.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f7353fff700 1 -- 172.21.15.33:0/1109358842 learned_addr learned my addr 172.21.15.33:0/1109358842 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:51.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f73537fe700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7354066a30 0x7f7354067f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f7352ffd700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73540c4e60 0x7f7354068420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f7353fff700 1 -- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7354066a30 msgr2=0x7f7354067f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:51.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f7353fff700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7354066a30 0x7f7354067f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f7353fff700 1 -- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73540c4e60 msgr2=0x7f7354068420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:51.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f7353fff700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73540c4e60 0x7f7354068420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f7353fff700 1 -- 172.21.15.33:0/1109358842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7344009370 con 0x7f73540c5240 2023-11-06T20:59:51.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f7353fff700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c5240 0x7f735413a010 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f734c00c7c0 tx=0x7f734c001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:51.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.082+0000 7f733b7fe700 1 -- 172.21.15.33:0/1109358842 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c00e4a0 con 0x7f73540c5240 2023-11-06T20:59:51.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.083+0000 7f733b7fe700 1 -- 172.21.15.33:0/1109358842 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c00e600 con 0x7f73540c5240 2023-11-06T20:59:51.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.083+0000 7f733b7fe700 1 -- 172.21.15.33:0/1109358842 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c00e980 con 0x7f73540c5240 2023-11-06T20:59:51.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.083+0000 7f735a1ff700 1 -- 172.21.15.33:0/1109358842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73540c71d0 con 0x7f73540c5240 2023-11-06T20:59:51.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.083+0000 7f735a1ff700 1 -- 172.21.15.33:0/1109358842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73540c7790 con 0x7f73540c5240 2023-11-06T20:59:51.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.084+0000 7f735a1ff700 1 -- 172.21.15.33:0/1109358842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73540ac620 con 0x7f73540c5240 2023-11-06T20:59:51.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.084+0000 7f733b7fe700 1 -- 172.21.15.33:0/1109358842 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f734c0123f0 con 0x7f73540c5240 2023-11-06T20:59:51.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.085+0000 7f733b7fe700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f734005f270 0x7f7340061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.085+0000 7f733b7fe700 1 -- 172.21.15.33:0/1109358842 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5828+0+0 (secure 0 0 0) 0x7f734c079b00 con 0x7f73540c5240 2023-11-06T20:59:51.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.086+0000 7f73537fe700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f734005f270 0x7f7340061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.086+0000 7f73537fe700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f734005f270 0x7f7340061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f7344009130 tx=0x7f7344017000).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:51.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.089+0000 7f733b7fe700 1 -- 172.21.15.33:0/1109358842 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f734c047af0 con 0x7f73540c5240 2023-11-06T20:59:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.318+0000 7f735a1ff700 1 -- 172.21.15.33:0/1109358842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73540c7980 con 0x7f73540c5240 2023-11-06T20:59:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.319+0000 7f733b7fe700 1 -- 172.21.15.33:0/1109358842 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f734c07be60 con 0x7f73540c5240 2023-11-06T20:59:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:51.319 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":31,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6055,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1867036074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1867036074},{"type":"v1","addr":"172.21.15.69:6836","nonce":1867036074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6058,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1920115024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1920115024},{"type":"v1","addr":"172.21.15.69:6837","nonce":1920115024}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6061,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2935600703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2935600703},{"type":"v1","addr":"172.21.15.33:6835","nonce":2935600703}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6546,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/1650067455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1650067455},{"type":"v1","addr":"172.21.15.33:6833","nonce":1650067455}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[]} 2023-11-06T20:59:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 -- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f734005f270 msgr2=0x7f7340061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f734005f270 0x7f7340061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f7344009130 tx=0x7f7344017000).stop 2023-11-06T20:59:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 -- 172.21.15.33:0/1109358842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c5240 msgr2=0x7f735413a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:51.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c5240 0x7f735413a010 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f734c00c7c0 tx=0x7f734c001510).stop 2023-11-06T20:59:51.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 -- 172.21.15.33:0/1109358842 shutdown_connections 2023-11-06T20:59:51.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f734005f270 0x7f7340061730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7354066a30 0x7f7354067f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73540c5240 0x7f735413a010 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 --2- 172.21.15.33:0/1109358842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73540c4e60 0x7f7354068420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 -- 172.21.15.33:0/1109358842 >> 172.21.15.33:0/1109358842 conn(0x7f73540c8c10 msgr2=0x7f7354151770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:51.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.321+0000 7f735a1ff700 1 -- 172.21.15.33:0/1109358842 shutdown_connections 2023-11-06T20:59:51.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.322+0000 7f735a1ff700 1 -- 172.21.15.33:0/1109358842 wait complete. 2023-11-06T20:59:51.324 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 31 2023-11-06T20:59:51.339 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T20:59:51.339 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T20:59:51.339 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T20:59:51.340 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:59:51.340 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T20:59:51.340 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T20:59:51.341 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T20:59:51.341 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:59:51.341 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T20:59:51.341 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T20:59:51.342 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T20:59:51.342 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:59:51.342 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T20:59:51.343 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T20:59:51.343 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T20:59:51.343 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T20:59:51.354 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T20:59:51.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T20:59:51.356 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T20:59:51.356 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T20:59:51.358 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T20:59:51.359 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T20:59:51.360 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T20:59:51.360 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T20:59:51.400 INFO:tasks.ceph.mds.a:Started 2023-11-06T20:59:51.401 DEBUG:teuthology.parallel:result is None 2023-11-06T20:59:51.403 INFO:tasks.ceph.mds.b:Started 2023-11-06T20:59:51.404 INFO:tasks.ceph.mds.c:Started 2023-11-06T20:59:51.404 DEBUG:teuthology.parallel:result is None 2023-11-06T20:59:51.404 DEBUG:teuthology.parallel:result is None 2023-11-06T20:59:51.406 INFO:tasks.ceph.mds.d:Started 2023-11-06T20:59:51.406 DEBUG:teuthology.parallel:result is None 2023-11-06T20:59:51.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T20:59:51.552 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T20:59:51.552 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T20:59:51.555 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T20:59:51.555 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T20:59:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.616+0000 7f19cf5ff700 1 Processor -- start 2023-11-06T20:59:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.616+0000 7f19cf5ff700 1 -- start start 2023-11-06T20:59:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19cf5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f19c8154210 0x7f19c80a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19cf5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f19c80bfd60 0x7f19c80c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19cf5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c80c06d0 con 0x7f19c80c9f00 2023-11-06T20:59:51.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19cf5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c80caa80 con 0x7f19c80bfd60 2023-11-06T20:59:51.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19cf5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c805efc0 con 0x7f19c8154210 2023-11-06T20:59:51.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19ccb9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f19c80bfd60 0x7f19c80c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19ccb9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f19c80bfd60 0x7f19c80c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47076/0 (socket says 172.21.15.33:47076) 2023-11-06T20:59:51.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19ccb9a700 1 -- 172.21.15.33:0/2597104956 learned_addr learned my addr 172.21.15.33:0/2597104956 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:51.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19cd39b700 1 --2- 172.21.15.33:0/2597104956 >> v2:172.21.15.69:3300/0 conn(0x7f19c8154210 0x7f19c80a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19ccb9a700 1 -- 172.21.15.33:0/2597104956 >> v2:172.21.15.69:3300/0 conn(0x7f19c8154210 msgr2=0x7f19c80a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19ccb9a700 1 --2- 172.21.15.33:0/2597104956 >> v2:172.21.15.69:3300/0 conn(0x7f19c8154210 0x7f19c80a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19ccb9a700 1 -- 172.21.15.33:0/2597104956 >> v1:172.21.15.69:6789/0 conn(0x7f19c80c9f00 legacy=0x7f19c80ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.618+0000 7f19ccb9a700 1 -- 172.21.15.33:0/2597104956 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c80c8ea0 con 0x7f19c80bfd60 2023-11-06T20:59:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19ccb9a700 1 --2- 172.21.15.33:0/2597104956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c80bfd60 0x7f19c80c01e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f19c80adab0 tx=0x7f19b80090f0).ready entity=mon.0 client_cookie=836204a322b7126d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:51.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19bffff700 1 -- 172.21.15.33:0/2597104956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800b520 con 0x7f19c80bfd60 2023-11-06T20:59:51.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19bffff700 1 -- 172.21.15.33:0/2597104956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800b680 con 0x7f19c80bfd60 2023-11-06T20:59:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19bffff700 1 -- 172.21.15.33:0/2597104956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800ba00 con 0x7f19c80bfd60 2023-11-06T20:59:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19cf5ff700 1 -- 172.21.15.33:0/2597104956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c80bfd60 msgr2=0x7f19c80c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19cf5ff700 1 --2- 172.21.15.33:0/2597104956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c80bfd60 0x7f19c80c01e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f19c80adab0 tx=0x7f19b80090f0).stop 2023-11-06T20:59:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19cf5ff700 1 -- 172.21.15.33:0/2597104956 shutdown_connections 2023-11-06T20:59:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19cf5ff700 1 --2- 172.21.15.33:0/2597104956 >> v2:172.21.15.69:3300/0 conn(0x7f19c8154210 0x7f19c80a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19cf5ff700 1 --2- 172.21.15.33:0/2597104956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c80bfd60 0x7f19c80c01e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19cf5ff700 1 -- 172.21.15.33:0/2597104956 >> 172.21.15.33:0/2597104956 conn(0x7f19c805e990 msgr2=0x7f19c80cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:51.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.619+0000 7f19cf5ff700 1 -- 172.21.15.33:0/2597104956 shutdown_connections 2023-11-06T20:59:51.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.620+0000 7f19cf5ff700 1 -- 172.21.15.33:0/2597104956 wait complete. 2023-11-06T20:59:51.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.621+0000 7f19cf5ff700 1 Processor -- start 2023-11-06T20:59:51.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.621+0000 7f19cf5ff700 1 -- start start 2023-11-06T20:59:51.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cf5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c80bfd60 0x7f19c80c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cf5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f19c80c9f00 0x7f19c80ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cf5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c8154210 0x7f19c80abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cf5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c81403f0 con 0x7f19c8154210 2023-11-06T20:59:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cf5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80ac290 con 0x7f19c80bfd60 2023-11-06T20:59:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cf5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80ac490 con 0x7f19c80c9f00 2023-11-06T20:59:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cdb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c8154210 0x7f19c80abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cd39b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c80bfd60 0x7f19c80c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19ccb9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f19c80c9f00 0x7f19c80ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cd39b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c80bfd60 0x7f19c80c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:54448/0 (socket says 172.21.15.33:54448) 2023-11-06T20:59:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cd39b700 1 -- 172.21.15.33:0/1639518600 learned_addr learned my addr 172.21.15.33:0/1639518600 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.622+0000 7f19cdb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c8154210 0x7f19c80abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47086/0 (socket says 172.21.15.33:47086) 2023-11-06T20:59:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19ccb9a700 1 -- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c80bfd60 msgr2=0x7f19c80c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19ccb9a700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c80bfd60 0x7f19c80c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19ccb9a700 1 -- 172.21.15.33:0/1639518600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c8154210 msgr2=0x7f19c80abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:51.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19ccb9a700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c8154210 0x7f19c80abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:51.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19ccb9a700 1 -- 172.21.15.33:0/1639518600 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b8009370 con 0x7f19c80c9f00 2023-11-06T20:59:51.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19cd39b700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c80bfd60 0x7f19c80c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:51.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19cdb9c700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c8154210 0x7f19c80abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T20:59:51.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19ccb9a700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f19c80c9f00 0x7f19c80ab8b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f19b8014000 tx=0x7f19b8017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:51.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19bd7fa700 1 -- 172.21.15.33:0/1639518600 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800be90 con 0x7f19c80c9f00 2023-11-06T20:59:51.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19bd7fa700 1 -- 172.21.15.33:0/1639518600 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800b7f0 con 0x7f19c80c9f00 2023-11-06T20:59:51.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19cf5ff700 1 -- 172.21.15.33:0/1639518600 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c80ac770 con 0x7f19c80c9f00 2023-11-06T20:59:51.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.623+0000 7f19bd7fa700 1 -- 172.21.15.33:0/1639518600 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b801c670 con 0x7f19c80c9f00 2023-11-06T20:59:51.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.624+0000 7f19cf5ff700 1 -- 172.21.15.33:0/1639518600 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c80acd30 con 0x7f19c80c9f00 2023-11-06T20:59:51.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.625+0000 7f19cf5ff700 1 -- 172.21.15.33:0/1639518600 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c806c6c0 con 0x7f19c80c9f00 2023-11-06T20:59:51.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.625+0000 7f19bd7fa700 1 -- 172.21.15.33:0/1639518600 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f19b801ca20 con 0x7f19c80c9f00 2023-11-06T20:59:51.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.626+0000 7f19bd7fa700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f19ac05f270 0x7f19ac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:51.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.626+0000 7f19bd7fa700 1 -- 172.21.15.33:0/1639518600 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5828+0+0 (secure 0 0 0) 0x7f19b807bff0 con 0x7f19c80c9f00 2023-11-06T20:59:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.627+0000 7f19cd39b700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f19ac05f270 0x7f19ac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.627+0000 7f19cd39b700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f19ac05f270 0x7f19ac061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f19c4011310 tx=0x7f19c4009060).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.631+0000 7f19bd7fa700 1 -- 172.21.15.33:0/1639518600 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f19b804d1e0 con 0x7f19c80c9f00 2023-11-06T20:59:51.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:51.827+0000 7f19cf5ff700 1 -- 172.21.15.33:0/1639518600 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f19c805eda0 con 0x7f19c80c9f00 2023-11-06T20:59:52.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.680+0000 7f19bd7fa700 1 -- 172.21.15.33:0/1639518600 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v95) v1 ==== 97+0+0 (secure 0 0 0) 0x7f19b802a030 con 0x7f19c80c9f00 2023-11-06T20:59:52.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 -- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f19ac05f270 msgr2=0x7f19ac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:52.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f19ac05f270 0x7f19ac061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f19c4011310 tx=0x7f19c4009060).stop 2023-11-06T20:59:52.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 -- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f19c80c9f00 msgr2=0x7f19c80ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:52.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f19c80c9f00 0x7f19c80ab8b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f19b8014000 tx=0x7f19b8017000).stop 2023-11-06T20:59:52.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 -- 172.21.15.33:0/1639518600 shutdown_connections 2023-11-06T20:59:52.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f19ac05f270 0x7f19ac061730 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:52.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19c8154210 0x7f19c80abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:52.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19c80bfd60 0x7f19c80c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:52.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 --2- 172.21.15.33:0/1639518600 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f19c80c9f00 0x7f19c80ab8b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:52.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 -- 172.21.15.33:0/1639518600 >> 172.21.15.33:0/1639518600 conn(0x7f19c805e990 msgr2=0x7f19c80c5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:52.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 -- 172.21.15.33:0/1639518600 shutdown_connections 2023-11-06T20:59:52.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.683+0000 7f19cf5ff700 1 -- 172.21.15.33:0/1639518600 wait complete. 2023-11-06T20:59:52.685 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T20:59:52.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T20:59:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.906+0000 7f4a34fff700 1 Processor -- start 2023-11-06T20:59:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.906+0000 7f4a34fff700 1 -- start start 2023-11-06T20:59:52.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.907+0000 7f4a34fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300bdbe0 0x7f4a301584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:52.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.907+0000 7f4a34fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a300a7910 0x7f4a300a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:52.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.907+0000 7f4a34fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a30064010 con 0x7f4a301580e0 2023-11-06T20:59:52.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.907+0000 7f4a34fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a300be430 con 0x7f4a300a7910 2023-11-06T20:59:52.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.907+0000 7f4a34fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300be150 con 0x7f4a300bdbe0 2023-11-06T20:59:52.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.907+0000 7f4a2e59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4a301580e0 0x7f4a300684c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35880/0 (socket says 172.21.15.33:35880) 2023-11-06T20:59:52.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.907+0000 7f4a2e59c700 1 -- 172.21.15.33:0/552777958 learned_addr learned my addr 172.21.15.33:0/552777958 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:52.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.907+0000 7f4a2ed9d700 1 --2- 172.21.15.33:0/552777958 >> v2:172.21.15.33:3300/0 conn(0x7f4a300a7910 0x7f4a300a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:52.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2dd9b700 1 --2- 172.21.15.33:0/552777958 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300bdbe0 0x7f4a301584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:52.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2d59a700 1 -- 172.21.15.33:0/552777958 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020264098 0 0) 0x7f4a30064010 con 0x7f4a301580e0 2023-11-06T20:59:52.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2d59a700 1 -- 172.21.15.33:0/552777958 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a180033f0 con 0x7f4a301580e0 2023-11-06T20:59:52.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2ed9d700 1 -- 172.21.15.33:0/552777958 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300bdbe0 msgr2=0x7f4a301584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:52.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2ed9d700 1 --2- 172.21.15.33:0/552777958 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300bdbe0 0x7f4a301584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:52.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2ed9d700 1 -- 172.21.15.33:0/552777958 >> v1:172.21.15.33:6789/0 conn(0x7f4a301580e0 legacy=0x7f4a300684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:52.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2ed9d700 1 -- 172.21.15.33:0/552777958 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a300639c0 con 0x7f4a300a7910 2023-11-06T20:59:52.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2ed9d700 1 --2- 172.21.15.33:0/552777958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a300a7910 0x7f4a300a7d70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f4a28009e60 tx=0x7f4a28009780).ready entity=mon.0 client_cookie=fd0f03cfd54e5fd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:52.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2d59a700 1 -- 172.21.15.33:0/552777958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800e070 con 0x7f4a300a7910 2023-11-06T20:59:52.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2d59a700 1 -- 172.21.15.33:0/552777958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2800bab0 con 0x7f4a300a7910 2023-11-06T20:59:52.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.908+0000 7f4a2d59a700 1 -- 172.21.15.33:0/552777958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800be30 con 0x7f4a300a7910 2023-11-06T20:59:52.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.909+0000 7f4a34fff700 1 -- 172.21.15.33:0/552777958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a300a7910 msgr2=0x7f4a300a7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:52.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.909+0000 7f4a34fff700 1 --2- 172.21.15.33:0/552777958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a300a7910 0x7f4a300a7d70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f4a28009e60 tx=0x7f4a28009780).stop 2023-11-06T20:59:52.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.909+0000 7f4a34fff700 1 -- 172.21.15.33:0/552777958 shutdown_connections 2023-11-06T20:59:52.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.909+0000 7f4a34fff700 1 --2- 172.21.15.33:0/552777958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a300a7910 0x7f4a300a7d70 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:52.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.909+0000 7f4a34fff700 1 --2- 172.21.15.33:0/552777958 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300bdbe0 0x7f4a301584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:52.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.909+0000 7f4a34fff700 1 -- 172.21.15.33:0/552777958 >> 172.21.15.33:0/552777958 conn(0x7f4a3005ec80 msgr2=0x7f4a300664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:52.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.909+0000 7f4a34fff700 1 -- 172.21.15.33:0/552777958 shutdown_connections 2023-11-06T20:59:52.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.909+0000 7f4a34fff700 1 -- 172.21.15.33:0/552777958 wait complete. 2023-11-06T20:59:52.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.911+0000 7f4a34fff700 1 Processor -- start 2023-11-06T20:59:52.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.911+0000 7f4a34fff700 1 -- start start 2023-11-06T20:59:52.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a34fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300a7910 0x7f4a3013a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:52.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a34fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a300bdbe0 0x7f4a300a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:52.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a34fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a301580e0 0x7f4a300acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:52.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a34fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300a7260 con 0x7f4a301580e0 2023-11-06T20:59:52.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a34fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300a76f0 con 0x7f4a300bdbe0 2023-11-06T20:59:52.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a34fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300a6650 con 0x7f4a300a7910 2023-11-06T20:59:52.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a301580e0 0x7f4a300acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:52.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a301580e0 0x7f4a300acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47096/0 (socket says 172.21.15.33:47096) 2023-11-06T20:59:52.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2ed9d700 1 -- 172.21.15.33:0/1171864439 learned_addr learned my addr 172.21.15.33:0/1171864439 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:52.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2e59c700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300a7910 0x7f4a3013a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:52.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2ed9d700 1 -- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300a7910 msgr2=0x7f4a3013a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:52.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2dd9b700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a300bdbe0 0x7f4a300a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:52.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2ed9d700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300a7910 0x7f4a3013a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:52.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2ed9d700 1 -- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a300bdbe0 msgr2=0x7f4a300a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:52.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2ed9d700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a300bdbe0 0x7f4a300a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:52.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2ed9d700 1 -- 172.21.15.33:0/1171864439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28009a20 con 0x7f4a301580e0 2023-11-06T20:59:52.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2e59c700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300a7910 0x7f4a3013a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:52.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.913+0000 7f4a2dd9b700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a300bdbe0 0x7f4a300a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:52.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.914+0000 7f4a2ed9d700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a301580e0 0x7f4a300acff0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f4a30069bd0 tx=0x7f4a28009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:52.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.914+0000 7f4a1e7fc700 1 -- 172.21.15.33:0/1171864439 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2801d630 con 0x7f4a301580e0 2023-11-06T20:59:52.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.914+0000 7f4a1e7fc700 1 -- 172.21.15.33:0/1171864439 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2801d7b0 con 0x7f4a301580e0 2023-11-06T20:59:52.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.914+0000 7f4a1e7fc700 1 -- 172.21.15.33:0/1171864439 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2801db70 con 0x7f4a301580e0 2023-11-06T20:59:52.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.914+0000 7f4a34fff700 1 -- 172.21.15.33:0/1171864439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a300a6930 con 0x7f4a301580e0 2023-11-06T20:59:52.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.914+0000 7f4a34fff700 1 -- 172.21.15.33:0/1171864439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a300a6ef0 con 0x7f4a301580e0 2023-11-06T20:59:52.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.915+0000 7f4a34fff700 1 -- 172.21.15.33:0/1171864439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a300beef0 con 0x7f4a301580e0 2023-11-06T20:59:52.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.916+0000 7f4a1e7fc700 1 -- 172.21.15.33:0/1171864439 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4a28027460 con 0x7f4a301580e0 2023-11-06T20:59:52.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.917+0000 7f4a1e7fc700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a1005f270 0x7f4a10061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:52.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.917+0000 7f4a1e7fc700 1 -- 172.21.15.33:0/1171864439 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f4a2807bfb0 con 0x7f4a301580e0 2023-11-06T20:59:52.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.917+0000 7f4a2e59c700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a1005f270 0x7f4a10061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:52.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.918+0000 7f4a2e59c700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a1005f270 0x7f4a10061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4a20005750 tx=0x7f4a20009040).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:52.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:52.921+0000 7f4a1e7fc700 1 -- 172.21.15.33:0/1171864439 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4a2804d3e0 con 0x7f4a301580e0 2023-11-06T20:59:53.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.171+0000 7f4a34fff700 1 -- 172.21.15.33:0/1171864439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4a3013aca0 con 0x7f4a301580e0 2023-11-06T20:59:53.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.171+0000 7f4a1e7fc700 1 -- 172.21.15.33:0/1171864439 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4a28050a00 con 0x7f4a301580e0 2023-11-06T20:59:53.171 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:53.171 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T20:59:53.172 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T20:59:53.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.172 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T20:59:53.172 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T20:59:53.172 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.173 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:53.173 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:59:53.173 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:59:53.173 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.174 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T20:59:53.174 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T20:59:53.174 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.175 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:53.175 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:59:53.175 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:59:53.175 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.176 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T20:59:53.176 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T20:59:53.176 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.176 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:53.177 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:59:53.177 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:59:53.177 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.178 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.178 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T20:59:53.178 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T20:59:53.179 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.179 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:53.179 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:59:53.179 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:59:53.180 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.180 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T20:59:53.181 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T20:59:53.181 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.181 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:53.181 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:53.181 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.182 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.182 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T20:59:53.182 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T20:59:53.183 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.183 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:53.183 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:53.183 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.184 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T20:59:53.184 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T20:59:53.184 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T20:59:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T20:59:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T20:59:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T20:59:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T20:59:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T20:59:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T20:59:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T20:59:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T20:59:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T20:59:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T20:59:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T20:59:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T20:59:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T20:59:53.194 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-11-06T20:59:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T20:59:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T20:59:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T20:59:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:59:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:59:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:59:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.197 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T20:59:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T20:59:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:59:53.198 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:59:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:59:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.199 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T20:59:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T20:59:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T20:59:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T20:59:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T20:59:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T20:59:53.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.201 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T20:59:53.201 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T20:59:53.202 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.202 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:59:53.202 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T20:59:53.202 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T20:59:53.202 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.203 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.203 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T20:59:53.203 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T20:59:53.203 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.204 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:59:53.204 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T20:59:53.204 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:53.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:53.205 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T20:59:53.205 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T20:59:53.205 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T20:59:53.205 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T20:59:53.205 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T20:59:53.205 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T20:59:53.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:53.206 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T20:59:53.206 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T20:59:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.173+0000 7f4a34fff700 1 -- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a1005f270 msgr2=0x7f4a10061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.173+0000 7f4a34fff700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a1005f270 0x7f4a10061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4a20005750 tx=0x7f4a20009040).stop 2023-11-06T20:59:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.173+0000 7f4a34fff700 1 -- 172.21.15.33:0/1171864439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a301580e0 msgr2=0x7f4a300acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.173+0000 7f4a34fff700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a301580e0 0x7f4a300acff0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f4a30069bd0 tx=0x7f4a28009780).stop 2023-11-06T20:59:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.174+0000 7f4a34fff700 1 -- 172.21.15.33:0/1171864439 shutdown_connections 2023-11-06T20:59:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.174+0000 7f4a34fff700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4a1005f270 0x7f4a10061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.174+0000 7f4a34fff700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a300a7910 0x7f4a3013a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.174+0000 7f4a34fff700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a301580e0 0x7f4a300acff0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.174+0000 7f4a34fff700 1 --2- 172.21.15.33:0/1171864439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a300bdbe0 0x7f4a300a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.174+0000 7f4a34fff700 1 -- 172.21.15.33:0/1171864439 >> 172.21.15.33:0/1171864439 conn(0x7f4a3005ec80 msgr2=0x7f4a30157050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.174+0000 7f4a34fff700 1 -- 172.21.15.33:0/1171864439 shutdown_connections 2023-11-06T20:59:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.174+0000 7f4a34fff700 1 -- 172.21.15.33:0/1171864439 wait complete. 2023-11-06T20:59:53.210 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:53.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.408+0000 7f29addff700 1 Processor -- start 2023-11-06T20:59:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.408+0000 7f29addff700 1 -- start start 2023-11-06T20:59:53.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.409+0000 7f29addff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f29a80c1840 0x7f29a80a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.409+0000 7f29addff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f29a80c6460 0x7f29a80bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.409+0000 7f29addff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a81554a0 con 0x7f29a80aa2d0 2023-11-06T20:59:53.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.409+0000 7f29addff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a8154960 con 0x7f29a80c1840 2023-11-06T20:59:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.409+0000 7f29addff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a8155930 con 0x7f29a80c6460 2023-11-06T20:59:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.409+0000 7f29a77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f29a80c1840 0x7f29a80a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.409+0000 7f29a77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f29a80c1840 0x7f29a80a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47112/0 (socket says 172.21.15.33:47112) 2023-11-06T20:59:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.409+0000 7f29a77fe700 1 -- 172.21.15.33:0/2463018509 learned_addr learned my addr 172.21.15.33:0/2463018509 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.410+0000 7f29a77fe700 1 -- 172.21.15.33:0/2463018509 >> v2:172.21.15.69:3300/0 conn(0x7f29a80c6460 msgr2=0x7f29a80bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.410+0000 7f29a7fff700 1 --2- 172.21.15.33:0/2463018509 >> v2:172.21.15.69:3300/0 conn(0x7f29a80c6460 0x7f29a80bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.410+0000 7f29a77fe700 1 --2- 172.21.15.33:0/2463018509 >> v2:172.21.15.69:3300/0 conn(0x7f29a80c6460 0x7f29a80bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.410+0000 7f29a77fe700 1 -- 172.21.15.33:0/2463018509 >> v1:172.21.15.69:6789/0 conn(0x7f29a80aa2d0 legacy=0x7f29a80c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.410+0000 7f29a77fe700 1 -- 172.21.15.33:0/2463018509 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a8154b70 con 0x7f29a80c1840 2023-11-06T20:59:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.410+0000 7f29a7fff700 1 --2- 172.21.15.33:0/2463018509 >> v2:172.21.15.69:3300/0 conn(0x7f29a80c6460 0x7f29a80bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:53.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.410+0000 7f29a77fe700 1 --2- 172.21.15.33:0/2463018509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c1840 0x7f29a80a9de0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f29a80c55f0 tx=0x7f29980090f0).ready entity=mon.0 client_cookie=e2a2cfe9d378509a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:53.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.410+0000 7f29a67fc700 1 -- 172.21.15.33:0/2463018509 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299800b5a0 con 0x7f29a80c1840 2023-11-06T20:59:53.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.410+0000 7f29a67fc700 1 -- 172.21.15.33:0/2463018509 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299800b700 con 0x7f29a80c1840 2023-11-06T20:59:53.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.411+0000 7f29a67fc700 1 -- 172.21.15.33:0/2463018509 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299800ba80 con 0x7f29a80c1840 2023-11-06T20:59:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.411+0000 7f29addff700 1 -- 172.21.15.33:0/2463018509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c1840 msgr2=0x7f29a80a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.411+0000 7f29addff700 1 --2- 172.21.15.33:0/2463018509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c1840 0x7f29a80a9de0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f29a80c55f0 tx=0x7f29980090f0).stop 2023-11-06T20:59:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.411+0000 7f29addff700 1 -- 172.21.15.33:0/2463018509 shutdown_connections 2023-11-06T20:59:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.411+0000 7f29addff700 1 --2- 172.21.15.33:0/2463018509 >> v2:172.21.15.69:3300/0 conn(0x7f29a80c6460 0x7f29a80bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.411+0000 7f29addff700 1 --2- 172.21.15.33:0/2463018509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c1840 0x7f29a80a9de0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.411+0000 7f29addff700 1 -- 172.21.15.33:0/2463018509 >> 172.21.15.33:0/2463018509 conn(0x7f29a805ec80 msgr2=0x7f29a80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.411+0000 7f29addff700 1 -- 172.21.15.33:0/2463018509 shutdown_connections 2023-11-06T20:59:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.411+0000 7f29addff700 1 -- 172.21.15.33:0/2463018509 wait complete. 2023-11-06T20:59:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.413+0000 7f29addff700 1 Processor -- start 2023-11-06T20:59:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.413+0000 7f29addff700 1 -- start start 2023-11-06T20:59:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29addff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29a80aa2d0 0x7f29a80b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29addff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29a80c1840 0x7f29a8146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29addff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c6460 0x7f29a80c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29addff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80b0e90 con 0x7f29a80c6460 2023-11-06T20:59:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29addff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80c4be0 con 0x7f29a80aa2d0 2023-11-06T20:59:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29addff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a81470d0 con 0x7f29a80c1840 2023-11-06T20:59:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c6460 0x7f29a80c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c6460 0x7f29a80c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47128/0 (socket says 172.21.15.33:47128) 2023-11-06T20:59:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29a7fff700 1 -- 172.21.15.33:0/1045396836 learned_addr learned my addr 172.21.15.33:0/1045396836 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29a77fe700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29a80aa2d0 0x7f29a80b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29a6ffd700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29a80c1840 0x7f29a8146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29a7fff700 1 -- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29a80c1840 msgr2=0x7f29a8146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29a7fff700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29a80c1840 0x7f29a8146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29a7fff700 1 -- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29a80aa2d0 msgr2=0x7f29a80b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29a7fff700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29a80aa2d0 0x7f29a80b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.414+0000 7f29a7fff700 1 -- 172.21.15.33:0/1045396836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2998009370 con 0x7f29a80c6460 2023-11-06T20:59:53.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.415+0000 7f29a7fff700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c6460 0x7f29a80c37f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f29a000a380 tx=0x7f29a0007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:53.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.415+0000 7f298f7fe700 1 -- 172.21.15.33:0/1045396836 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000ae10 con 0x7f29a80c6460 2023-11-06T20:59:53.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.415+0000 7f298f7fe700 1 -- 172.21.15.33:0/1045396836 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a000d650 con 0x7f29a80c6460 2023-11-06T20:59:53.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.415+0000 7f298f7fe700 1 -- 172.21.15.33:0/1045396836 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000d9f0 con 0x7f29a80c6460 2023-11-06T20:59:53.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.415+0000 7f29addff700 1 -- 172.21.15.33:0/1045396836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a80c4060 con 0x7f29a80c6460 2023-11-06T20:59:53.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.415+0000 7f29addff700 1 -- 172.21.15.33:0/1045396836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a80c4620 con 0x7f29a80c6460 2023-11-06T20:59:53.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.416+0000 7f29addff700 1 -- 172.21.15.33:0/1045396836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a806c6c0 con 0x7f29a80c6460 2023-11-06T20:59:53.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.417+0000 7f298f7fe700 1 -- 172.21.15.33:0/1045396836 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f29a0013070 con 0x7f29a80c6460 2023-11-06T20:59:53.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.418+0000 7f298f7fe700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f299405f270 0x7f2994061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.418+0000 7f298f7fe700 1 -- 172.21.15.33:0/1045396836 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f29a007ab20 con 0x7f29a80c6460 2023-11-06T20:59:53.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.418+0000 7f29a77fe700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f299405f270 0x7f2994061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.419+0000 7f29a77fe700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f299405f270 0x7f2994061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f2998009130 tx=0x7f2998015000).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:53.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.422+0000 7f298f7fe700 1 -- 172.21.15.33:0/1045396836 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f29a004bf00 con 0x7f29a80c6460 2023-11-06T20:59:53.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.604+0000 7f29addff700 1 -- 172.21.15.33:0/1045396836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29a8063f20 con 0x7f29a80c6460 2023-11-06T20:59:53.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.605+0000 7f298f7fe700 1 -- 172.21.15.33:0/1045396836 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+13761 (secure 0 0 0) 0x7f29a0007970 con 0x7f29a80c6460 2023-11-06T20:59:53.604 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:53.604 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":95,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:52.674835+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 -- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f299405f270 msgr2=0x7f2994061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f299405f270 0x7f2994061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f2998009130 tx=0x7f2998015000).stop 2023-11-06T20:59:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 -- 172.21.15.33:0/1045396836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c6460 msgr2=0x7f29a80c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c6460 0x7f29a80c37f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f29a000a380 tx=0x7f29a0007900).stop 2023-11-06T20:59:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 -- 172.21.15.33:0/1045396836 shutdown_connections 2023-11-06T20:59:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f299405f270 0x7f2994061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a80c6460 0x7f29a80c37f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29a80aa2d0 0x7f29a80b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 --2- 172.21.15.33:0/1045396836 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29a80c1840 0x7f29a8146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 -- 172.21.15.33:0/1045396836 >> 172.21.15.33:0/1045396836 conn(0x7f29a805ec80 msgr2=0x7f29a81541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 -- 172.21.15.33:0/1045396836 shutdown_connections 2023-11-06T20:59:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.607+0000 7f29addff700 1 -- 172.21.15.33:0/1045396836 wait complete. 2023-11-06T20:59:53.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T20:59:53.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T20:59:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.825+0000 7f427c3ff700 1 Processor -- start 2023-11-06T20:59:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.825+0000 7f427c3ff700 1 -- start start 2023-11-06T20:59:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.826+0000 7f427c3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 0x7f42740c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.826+0000 7f427c3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42740c6b60 con 0x7f4274154330 2023-11-06T20:59:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.826+0000 7f427c3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42740c7700 con 0x7f42740c7ff0 2023-11-06T20:59:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.826+0000 7f427c3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42740c5cd0 con 0x7f4274154710 2023-11-06T20:59:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.826+0000 7f427a19b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4274154330 0x7f42740c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35892/0 (socket says 172.21.15.33:35892) 2023-11-06T20:59:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.826+0000 7f427a19b700 1 -- 172.21.15.33:0/3947511048 learned_addr learned my addr 172.21.15.33:0/3947511048 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.827+0000 7f427999a700 1 --2- 172.21.15.33:0/3947511048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 0x7f42740c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.827+0000 7f4279199700 1 -- 172.21.15.33:0/3947511048 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124796244 0 0) 0x7f42740c6b60 con 0x7f4274154330 2023-11-06T20:59:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.827+0000 7f4279199700 1 -- 172.21.15.33:0/3947511048 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42600033f0 con 0x7f4274154330 2023-11-06T20:59:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.827+0000 7f4279199700 1 -- 172.21.15.33:0/3947511048 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305314981 0 0) 0x7f42740c7700 con 0x7f42740c7ff0 2023-11-06T20:59:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.827+0000 7f4279199700 1 -- 172.21.15.33:0/3947511048 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42740c6b60 con 0x7f42740c7ff0 2023-11-06T20:59:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.827+0000 7f427999a700 1 -- 172.21.15.33:0/3947511048 >> v1:172.21.15.69:6789/0 conn(0x7f42740c7ff0 legacy=0x7f42740646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.827+0000 7f427999a700 1 -- 172.21.15.33:0/3947511048 >> v1:172.21.15.33:6789/0 conn(0x7f4274154330 legacy=0x7f42740c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.827+0000 7f4279199700 1 -- 172.21.15.33:0/3947511048 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3924272474 0 0) 0x7f42600033f0 con 0x7f4274154330 2023-11-06T20:59:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.827+0000 7f427999a700 1 -- 172.21.15.33:0/3947511048 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42740c5f50 con 0x7f4274154710 2023-11-06T20:59:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.827+0000 7f427999a700 1 --2- 172.21.15.33:0/3947511048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 0x7f42740c57e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f4264009790 tx=0x7f4264009060).ready entity=mon.2 client_cookie=4fc922f4fd08d242 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.828+0000 7f4279199700 1 -- 172.21.15.33:0/3947511048 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270001470 con 0x7f4274154710 2023-11-06T20:59:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.828+0000 7f4279199700 1 -- 172.21.15.33:0/3947511048 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4264011400 con 0x7f4274154710 2023-11-06T20:59:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.828+0000 7f4279199700 1 -- 172.21.15.33:0/3947511048 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264011780 con 0x7f4274154710 2023-11-06T20:59:53.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.828+0000 7f427c3ff700 1 -- 172.21.15.33:0/3947511048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 msgr2=0x7f42740c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.828+0000 7f427c3ff700 1 --2- 172.21.15.33:0/3947511048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 0x7f42740c57e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f4264009790 tx=0x7f4264009060).stop 2023-11-06T20:59:53.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.828+0000 7f427c3ff700 1 -- 172.21.15.33:0/3947511048 shutdown_connections 2023-11-06T20:59:53.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.828+0000 7f427c3ff700 1 --2- 172.21.15.33:0/3947511048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 0x7f42740c57e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.828+0000 7f427c3ff700 1 -- 172.21.15.33:0/3947511048 >> 172.21.15.33:0/3947511048 conn(0x7f42740cb910 msgr2=0x7f42740c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:53.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.829+0000 7f427c3ff700 1 -- 172.21.15.33:0/3947511048 shutdown_connections 2023-11-06T20:59:53.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.829+0000 7f427c3ff700 1 -- 172.21.15.33:0/3947511048 wait complete. 2023-11-06T20:59:53.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.830+0000 7f427c3ff700 1 Processor -- start 2023-11-06T20:59:53.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.830+0000 7f427c3ff700 1 -- start start 2023-11-06T20:59:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.831+0000 7f427c3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42740c7ff0 0x7f42740addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.831+0000 7f427c3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274154330 0x7f4274146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.831+0000 7f427c3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 0x7f427415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.831+0000 7f427c3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427415a000 con 0x7f4274154330 2023-11-06T20:59:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.831+0000 7f427c3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4274158f60 con 0x7f42740c7ff0 2023-11-06T20:59:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.831+0000 7f427c3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4274159240 con 0x7f4274154710 2023-11-06T20:59:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.831+0000 7f427999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274154330 0x7f4274146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.831+0000 7f427999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274154330 0x7f4274146940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47134/0 (socket says 172.21.15.33:47134) 2023-11-06T20:59:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.831+0000 7f427999a700 1 -- 172.21.15.33:0/579975188 learned_addr learned my addr 172.21.15.33:0/579975188 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f427a19b700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42740c7ff0 0x7f42740addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f427a99c700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 0x7f427415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f427999a700 1 -- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 msgr2=0x7f427415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f427999a700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 0x7f427415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f427999a700 1 -- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42740c7ff0 msgr2=0x7f42740addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:53.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f427999a700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42740c7ff0 0x7f42740addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:53.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f427999a700 1 -- 172.21.15.33:0/579975188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42640092a0 con 0x7f4274154330 2023-11-06T20:59:53.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f427a19b700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42740c7ff0 0x7f42740addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:53.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f427999a700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274154330 0x7f4274146940 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f42640096e0 tx=0x7f4264016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:53.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f426a7fc700 1 -- 172.21.15.33:0/579975188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264010920 con 0x7f4274154330 2023-11-06T20:59:53.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f426a7fc700 1 -- 172.21.15.33:0/579975188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4264010a80 con 0x7f4274154330 2023-11-06T20:59:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f426a7fc700 1 -- 172.21.15.33:0/579975188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264010e00 con 0x7f4274154330 2023-11-06T20:59:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.832+0000 7f427c3ff700 1 -- 172.21.15.33:0/579975188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4274159520 con 0x7f4274154330 2023-11-06T20:59:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.833+0000 7f427c3ff700 1 -- 172.21.15.33:0/579975188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4274159ae0 con 0x7f4274154330 2023-11-06T20:59:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.834+0000 7f427c3ff700 1 -- 172.21.15.33:0/579975188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4274159cd0 con 0x7f4274154330 2023-11-06T20:59:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.834+0000 7f426a7fc700 1 -- 172.21.15.33:0/579975188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4264018070 con 0x7f4274154330 2023-11-06T20:59:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.835+0000 7f426a7fc700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425805f270 0x7f4258061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.835+0000 7f426a7fc700 1 -- 172.21.15.33:0/579975188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f426407b660 con 0x7f4274154330 2023-11-06T20:59:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.836+0000 7f427a19b700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425805f270 0x7f4258061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.836+0000 7f427a19b700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425805f270 0x7f4258061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4270001ed0 tx=0x7f4270001940).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:53.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:53.839+0000 7f426a7fc700 1 -- 172.21.15.33:0/579975188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f426404ca40 con 0x7f4274154330 2023-11-06T20:59:54.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.020+0000 7f427c3ff700 1 -- 172.21.15.33:0/579975188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f42740830b0 con 0x7f4274154330 2023-11-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.021+0000 7f426a7fc700 1 -- 172.21.15.33:0/579975188 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v95) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4264021020 con 0x7f4274154330 2023-11-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 -- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425805f270 msgr2=0x7f4258061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425805f270 0x7f4258061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4270001ed0 tx=0x7f4270001940).stop 2023-11-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 -- 172.21.15.33:0/579975188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274154330 msgr2=0x7f4274146940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274154330 0x7f4274146940 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f42640096e0 tx=0x7f4264016000).stop 2023-11-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 -- 172.21.15.33:0/579975188 shutdown_connections 2023-11-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f425805f270 0x7f4258061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4274154710 0x7f427415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42740c7ff0 0x7f42740addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 --2- 172.21.15.33:0/579975188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274154330 0x7f4274146940 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 -- 172.21.15.33:0/579975188 >> 172.21.15.33:0/579975188 conn(0x7f42740cb910 msgr2=0x7f42740c0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 -- 172.21.15.33:0/579975188 shutdown_connections 2023-11-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.023+0000 7f427c3ff700 1 -- 172.21.15.33:0/579975188 wait complete. 2023-11-06T20:59:54.040 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T20:59:54.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.239+0000 7f4c529ff700 1 Processor -- start 2023-11-06T20:59:54.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.240+0000 7f4c529ff700 1 -- start start 2023-11-06T20:59:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.242+0000 7f4c529ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4c4c1546a0 0x7f4c4c064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.242+0000 7f4c529ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c4c0be2c0 0x7f4c4c066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.242+0000 7f4c529ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c0641f0 con 0x7f4c4c066560 2023-11-06T20:59:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.242+0000 7f4c529ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c154a80 con 0x7f4c4c1546a0 2023-11-06T20:59:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.242+0000 7f4c529ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0638c0 con 0x7f4c4c0be2c0 2023-11-06T20:59:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.242+0000 7f4c4b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4c4c066560 0x7f4c4c0bdbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35900/0 (socket says 172.21.15.33:35900) 2023-11-06T20:59:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.242+0000 7f4c4b7fe700 1 -- 172.21.15.33:0/205565573 learned_addr learned my addr 172.21.15.33:0/205565573 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.242+0000 7f4c4bfff700 1 --2- 172.21.15.33:0/205565573 >> v2:172.21.15.69:3300/0 conn(0x7f4c4c1546a0 0x7f4c4c064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.242+0000 7f4c50f9c700 1 --2- 172.21.15.33:0/205565573 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c4c0be2c0 0x7f4c4c066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.243+0000 7f4c4affd700 1 -- 172.21.15.33:0/205565573 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509353387 0 0) 0x7f4c4c0641f0 con 0x7f4c4c066560 2023-11-06T20:59:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.243+0000 7f4c4affd700 1 -- 172.21.15.33:0/205565573 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c340033f0 con 0x7f4c4c066560 2023-11-06T20:59:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.243+0000 7f4c4bfff700 1 -- 172.21.15.33:0/205565573 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c4c0be2c0 msgr2=0x7f4c4c066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.243+0000 7f4c4bfff700 1 --2- 172.21.15.33:0/205565573 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c4c0be2c0 0x7f4c4c066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.243+0000 7f4c4bfff700 1 -- 172.21.15.33:0/205565573 >> v1:172.21.15.33:6789/0 conn(0x7f4c4c066560 legacy=0x7f4c4c0bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.243+0000 7f4c4bfff700 1 -- 172.21.15.33:0/205565573 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c063ba0 con 0x7f4c4c1546a0 2023-11-06T20:59:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.243+0000 7f4c4bfff700 1 --2- 172.21.15.33:0/205565573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c4c1546a0 0x7f4c4c064680 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f4c3c009760 tx=0x7f4c3c009060).ready entity=mon.1 client_cookie=755682fa3009cc84 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.243+0000 7f4c4affd700 1 -- 172.21.15.33:0/205565573 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00d4a0 con 0x7f4c4c1546a0 2023-11-06T20:59:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.243+0000 7f4c4affd700 1 -- 172.21.15.33:0/205565573 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00d600 con 0x7f4c4c1546a0 2023-11-06T20:59:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.243+0000 7f4c4affd700 1 -- 172.21.15.33:0/205565573 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00d980 con 0x7f4c4c1546a0 2023-11-06T20:59:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.244+0000 7f4c529ff700 1 -- 172.21.15.33:0/205565573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c4c1546a0 msgr2=0x7f4c4c064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.244+0000 7f4c529ff700 1 --2- 172.21.15.33:0/205565573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c4c1546a0 0x7f4c4c064680 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f4c3c009760 tx=0x7f4c3c009060).stop 2023-11-06T20:59:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.244+0000 7f4c529ff700 1 -- 172.21.15.33:0/205565573 shutdown_connections 2023-11-06T20:59:54.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.244+0000 7f4c529ff700 1 --2- 172.21.15.33:0/205565573 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c4c0be2c0 0x7f4c4c066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.244+0000 7f4c529ff700 1 --2- 172.21.15.33:0/205565573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c4c1546a0 0x7f4c4c064680 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.244+0000 7f4c529ff700 1 -- 172.21.15.33:0/205565573 >> 172.21.15.33:0/205565573 conn(0x7f4c4c0c8c50 msgr2=0x7f4c4c0c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:54.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.244+0000 7f4c529ff700 1 -- 172.21.15.33:0/205565573 shutdown_connections 2023-11-06T20:59:54.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.244+0000 7f4c529ff700 1 -- 172.21.15.33:0/205565573 wait complete. 2023-11-06T20:59:54.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.246+0000 7f4c529ff700 1 Processor -- start 2023-11-06T20:59:54.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.246+0000 7f4c529ff700 1 -- start start 2023-11-06T20:59:54.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c529ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c4c066560 0x7f4c4c1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c529ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c4c0be2c0 0x7f4c4c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c529ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c1546a0 0x7f4c4c0aa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c529ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c1536d0 con 0x7f4c4c1546a0 2023-11-06T20:59:54.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c529ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c1576f0 con 0x7f4c4c066560 2023-11-06T20:59:54.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c529ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c156650 con 0x7f4c4c0be2c0 2023-11-06T20:59:54.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c50f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c1546a0 0x7f4c4c0aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c50f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c1546a0 0x7f4c4c0aa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47138/0 (socket says 172.21.15.33:47138) 2023-11-06T20:59:54.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c50f9c700 1 -- 172.21.15.33:0/565360077 learned_addr learned my addr 172.21.15.33:0/565360077 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c4bfff700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c4c066560 0x7f4c4c1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c4b7fe700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c4c0be2c0 0x7f4c4c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c50f9c700 1 -- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c4c0be2c0 msgr2=0x7f4c4c153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c50f9c700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c4c0be2c0 0x7f4c4c153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c50f9c700 1 -- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c4c066560 msgr2=0x7f4c4c1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c50f9c700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c4c066560 0x7f4c4c1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.247+0000 7f4c50f9c700 1 -- 172.21.15.33:0/565360077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0092a0 con 0x7f4c4c1546a0 2023-11-06T20:59:54.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.248+0000 7f4c50f9c700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c1546a0 0x7f4c4c0aa640 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f4c44009ec0 tx=0x7f4c4400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:54.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.248+0000 7f4c33fff700 1 -- 172.21.15.33:0/565360077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400cb70 con 0x7f4c4c1546a0 2023-11-06T20:59:54.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.248+0000 7f4c33fff700 1 -- 172.21.15.33:0/565360077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4400f040 con 0x7f4c4c1546a0 2023-11-06T20:59:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.248+0000 7f4c33fff700 1 -- 172.21.15.33:0/565360077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44013400 con 0x7f4c4c1546a0 2023-11-06T20:59:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.248+0000 7f4c529ff700 1 -- 172.21.15.33:0/565360077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c156930 con 0x7f4c4c1546a0 2023-11-06T20:59:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.248+0000 7f4c529ff700 1 -- 172.21.15.33:0/565360077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c156ef0 con 0x7f4c4c1546a0 2023-11-06T20:59:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.249+0000 7f4c529ff700 1 -- 172.21.15.33:0/565360077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c066000 con 0x7f4c4c1546a0 2023-11-06T20:59:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.250+0000 7f4c33fff700 1 -- 172.21.15.33:0/565360077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4c44013770 con 0x7f4c4c1546a0 2023-11-06T20:59:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.251+0000 7f4c33fff700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c3805f270 0x7f4c38061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.251+0000 7f4c33fff700 1 -- 172.21.15.33:0/565360077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f4c4407a880 con 0x7f4c4c1546a0 2023-11-06T20:59:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.251+0000 7f4c4bfff700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c3805f270 0x7f4c38061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.252+0000 7f4c4bfff700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c3805f270 0x7f4c38061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f4c3c00d260 tx=0x7f4c3c016000).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.255+0000 7f4c33fff700 1 -- 172.21.15.33:0/565360077 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4c4404bd20 con 0x7f4c4c1546a0 2023-11-06T20:59:54.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.441+0000 7f4c529ff700 1 -- 172.21.15.33:0/565360077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4c4c06be80 con 0x7f4c4c1546a0 2023-11-06T20:59:54.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.442+0000 7f4c33fff700 1 -- 172.21.15.33:0/565360077 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v95) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4c4404f340 con 0x7f4c4c1546a0 2023-11-06T20:59:54.442 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T20:59:54.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 -- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c3805f270 msgr2=0x7f4c38061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c3805f270 0x7f4c38061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f4c3c00d260 tx=0x7f4c3c016000).stop 2023-11-06T20:59:54.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 -- 172.21.15.33:0/565360077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c1546a0 msgr2=0x7f4c4c0aa640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c1546a0 0x7f4c4c0aa640 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f4c44009ec0 tx=0x7f4c4400b040).stop 2023-11-06T20:59:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 -- 172.21.15.33:0/565360077 shutdown_connections 2023-11-06T20:59:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4c3805f270 0x7f4c38061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c1546a0 0x7f4c4c0aa640 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c4c066560 0x7f4c4c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 --2- 172.21.15.33:0/565360077 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4c4c0be2c0 0x7f4c4c153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 -- 172.21.15.33:0/565360077 >> 172.21.15.33:0/565360077 conn(0x7f4c4c0c8c50 msgr2=0x7f4c4c067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:54.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 -- 172.21.15.33:0/565360077 shutdown_connections 2023-11-06T20:59:54.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.444+0000 7f4c529ff700 1 -- 172.21.15.33:0/565360077 wait complete. 2023-11-06T20:59:54.462 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T20:59:54.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.659+0000 7fe4a17ff700 1 Processor -- start 2023-11-06T20:59:54.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.659+0000 7fe4a17ff700 1 -- start start 2023-11-06T20:59:54.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.660+0000 7fe4a17ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49c0c1840 0x7fe49c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.660+0000 7fe4a17ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe49c0c6460 0x7fe49c0bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.660+0000 7fe4a17ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c1554a0 con 0x7fe49c0aa2d0 2023-11-06T20:59:54.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.660+0000 7fe4a17ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c154960 con 0x7fe49c0c6460 2023-11-06T20:59:54.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.660+0000 7fe4a17ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c155930 con 0x7fe49c0c1840 2023-11-06T20:59:54.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe49b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe49c0c6460 0x7fe49c0bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe49b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe49c0c6460 0x7fe49c0bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47154/0 (socket says 172.21.15.33:47154) 2023-11-06T20:59:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe49b7fe700 1 -- 172.21.15.33:0/2872128495 learned_addr learned my addr 172.21.15.33:0/2872128495 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe49a7fc700 1 --1- 172.21.15.33:0/2872128495 >> v1:172.21.15.33:6789/0 conn(0x7fe49c0aa2d0 0x7fe49c0c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35904/0 (socket says 172.21.15.33:35904) 2023-11-06T20:59:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe49affd700 1 --2- 172.21.15.33:0/2872128495 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49c0c1840 0x7fe49c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe49b7fe700 1 -- 172.21.15.33:0/2872128495 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49c0c1840 msgr2=0x7fe49c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe49b7fe700 1 --2- 172.21.15.33:0/2872128495 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49c0c1840 0x7fe49c0a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe49b7fe700 1 -- 172.21.15.33:0/2872128495 >> v1:172.21.15.33:6789/0 conn(0x7fe49c0aa2d0 legacy=0x7fe49c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe499ffb700 1 -- 172.21.15.33:0/2872128495 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288525866 0 0) 0x7fe49c1554a0 con 0x7fe49c0aa2d0 2023-11-06T20:59:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe49b7fe700 1 -- 172.21.15.33:0/2872128495 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c154b70 con 0x7fe49c0c6460 2023-11-06T20:59:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.661+0000 7fe49b7fe700 1 --2- 172.21.15.33:0/2872128495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c6460 0x7fe49c0bfd70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fe494009f20 tx=0x7fe494009840).ready entity=mon.0 client_cookie=3c5e269020f3d87e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:54.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe499ffb700 1 -- 172.21.15.33:0/2872128495 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400e070 con 0x7fe49c0c6460 2023-11-06T20:59:54.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe499ffb700 1 -- 172.21.15.33:0/2872128495 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49400b9d0 con 0x7fe49c0c6460 2023-11-06T20:59:54.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe499ffb700 1 -- 172.21.15.33:0/2872128495 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400bd50 con 0x7fe49c0c6460 2023-11-06T20:59:54.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe4a17ff700 1 -- 172.21.15.33:0/2872128495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c6460 msgr2=0x7fe49c0bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe4a17ff700 1 --2- 172.21.15.33:0/2872128495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c6460 0x7fe49c0bfd70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fe494009f20 tx=0x7fe494009840).stop 2023-11-06T20:59:54.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe4a17ff700 1 -- 172.21.15.33:0/2872128495 shutdown_connections 2023-11-06T20:59:54.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe4a17ff700 1 --2- 172.21.15.33:0/2872128495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c6460 0x7fe49c0bfd70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe4a17ff700 1 --2- 172.21.15.33:0/2872128495 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49c0c1840 0x7fe49c0a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe4a17ff700 1 -- 172.21.15.33:0/2872128495 >> 172.21.15.33:0/2872128495 conn(0x7fe49c05ec80 msgr2=0x7fe49c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:54.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe4a17ff700 1 -- 172.21.15.33:0/2872128495 shutdown_connections 2023-11-06T20:59:54.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.662+0000 7fe4a17ff700 1 -- 172.21.15.33:0/2872128495 wait complete. 2023-11-06T20:59:54.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.664+0000 7fe4a17ff700 1 Processor -- start 2023-11-06T20:59:54.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.664+0000 7fe4a17ff700 1 -- start start 2023-11-06T20:59:54.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe4a17ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49c0aa2d0 0x7fe49c144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe4a17ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c1840 0x7fe49c0c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe4a17ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe49c0c6460 0x7fe49c0c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe4a17ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c145040 con 0x7fe49c0c1840 2023-11-06T20:59:54.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe4a17ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0c26e0 con 0x7fe49c0c6460 2023-11-06T20:59:54.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe4a17ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0c29c0 con 0x7fe49c0aa2d0 2023-11-06T20:59:54.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe49a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c1840 0x7fe49c0c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe49a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c1840 0x7fe49c0c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47164/0 (socket says 172.21.15.33:47164) 2023-11-06T20:59:54.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe49a7fc700 1 -- 172.21.15.33:0/4109170596 learned_addr learned my addr 172.21.15.33:0/4109170596 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:54.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe49affd700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49c0aa2d0 0x7fe49c144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe49b7fe700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe49c0c6460 0x7fe49c0c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe49a7fc700 1 -- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49c0aa2d0 msgr2=0x7fe49c144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe49a7fc700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49c0aa2d0 0x7fe49c144b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe49a7fc700 1 -- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe49c0c6460 msgr2=0x7fe49c0c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe49a7fc700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe49c0c6460 0x7fe49c0c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.665+0000 7fe49a7fc700 1 -- 172.21.15.33:0/4109170596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494009ae0 con 0x7fe49c0c1840 2023-11-06T20:59:54.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.666+0000 7fe49a7fc700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c1840 0x7fe49c0c1eb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fe49000c440 tx=0x7fe490007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.666+0000 7fe482ffd700 1 -- 172.21.15.33:0/4109170596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4900070d0 con 0x7fe49c0c1840 2023-11-06T20:59:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.666+0000 7fe482ffd700 1 -- 172.21.15.33:0/4109170596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe490007230 con 0x7fe49c0c1840 2023-11-06T20:59:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.666+0000 7fe4a17ff700 1 -- 172.21.15.33:0/4109170596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c0c2ca0 con 0x7fe49c0c1840 2023-11-06T20:59:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.666+0000 7fe482ffd700 1 -- 172.21.15.33:0/4109170596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4900075b0 con 0x7fe49c0c1840 2023-11-06T20:59:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.666+0000 7fe4a17ff700 1 -- 172.21.15.33:0/4109170596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c0c23a0 con 0x7fe49c0c1840 2023-11-06T20:59:54.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.667+0000 7fe4a17ff700 1 -- 172.21.15.33:0/4109170596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c06c6c0 con 0x7fe49c0c1840 2023-11-06T20:59:54.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.668+0000 7fe482ffd700 1 -- 172.21.15.33:0/4109170596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe4900058f0 con 0x7fe49c0c1840 2023-11-06T20:59:54.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.669+0000 7fe482ffd700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe48805f270 0x7fe488061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:54.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.669+0000 7fe482ffd700 1 -- 172.21.15.33:0/4109170596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fe490079860 con 0x7fe49c0c1840 2023-11-06T20:59:54.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.669+0000 7fe49affd700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe48805f270 0x7fe488061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:54.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.670+0000 7fe49affd700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe48805f270 0x7fe488061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe49c1596c0 tx=0x7fe48c009060).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:54.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.673+0000 7fe482ffd700 1 -- 172.21.15.33:0/4109170596 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe49004ac40 con 0x7fe49c0c1840 2023-11-06T20:59:54.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.856+0000 7fe4a17ff700 1 -- 172.21.15.33:0/4109170596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe49c0c6ad0 con 0x7fe49c0c1840 2023-11-06T20:59:54.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.856+0000 7fe482ffd700 1 -- 172.21.15.33:0/4109170596 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v95) v1 ==== 84+0+143 (secure 0 0 0) 0x7fe49004e260 con 0x7fe49c0c1840 2023-11-06T20:59:54.856 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:54.857 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T20:59:54.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:54.857 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T20:59:54.857 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T20:59:54.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T20:59:54.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T20:59:54.858 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T20:59:54.858 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T20:59:54.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T20:59:54.858 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T20:59:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.858+0000 7fe4a17ff700 1 -- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe48805f270 msgr2=0x7fe488061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.858+0000 7fe4a17ff700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe48805f270 0x7fe488061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe49c1596c0 tx=0x7fe48c009060).stop 2023-11-06T20:59:54.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.858+0000 7fe4a17ff700 1 -- 172.21.15.33:0/4109170596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c1840 msgr2=0x7fe49c0c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:54.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.858+0000 7fe4a17ff700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c1840 0x7fe49c0c1eb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fe49000c440 tx=0x7fe490007bd0).stop 2023-11-06T20:59:54.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.858+0000 7fe4a17ff700 1 -- 172.21.15.33:0/4109170596 shutdown_connections 2023-11-06T20:59:54.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.858+0000 7fe4a17ff700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe48805f270 0x7fe488061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.858+0000 7fe4a17ff700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe49c0aa2d0 0x7fe49c144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.858+0000 7fe4a17ff700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe49c0c6460 0x7fe49c0c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.858+0000 7fe4a17ff700 1 --2- 172.21.15.33:0/4109170596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe49c0c1840 0x7fe49c0c1eb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:54.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.858+0000 7fe4a17ff700 1 -- 172.21.15.33:0/4109170596 >> 172.21.15.33:0/4109170596 conn(0x7fe49c05ec80 msgr2=0x7fe49c0bfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:54.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.859+0000 7fe4a17ff700 1 -- 172.21.15.33:0/4109170596 shutdown_connections 2023-11-06T20:59:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:54.859+0000 7fe4a17ff700 1 -- 172.21.15.33:0/4109170596 wait complete. 2023-11-06T20:59:54.875 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T20:59:54.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T20:59:55.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.077+0000 7fbf98dff700 1 Processor -- start 2023-11-06T20:59:55.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.077+0000 7fbf98dff700 1 -- start start 2023-11-06T20:59:55.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.079+0000 7fbf98dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbf940c8ee0 0x7fbf940ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:55.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.079+0000 7fbf98dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf940c9f80 con 0x7fbf94153ed0 2023-11-06T20:59:55.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.079+0000 7fbf98dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf940c6000 con 0x7fbf941542b0 2023-11-06T20:59:55.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.079+0000 7fbf98dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf940c9370 con 0x7fbf940c8ee0 2023-11-06T20:59:55.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.079+0000 7fbf9259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbf94153ed0 0x7fbf94158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35910/0 (socket says 172.21.15.33:35910) 2023-11-06T20:59:55.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.079+0000 7fbf9259c700 1 -- 172.21.15.33:0/4117980174 learned_addr learned my addr 172.21.15.33:0/4117980174 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:55.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.079+0000 7fbf91d9b700 1 --2- 172.21.15.33:0/4117980174 >> v2:172.21.15.69:3300/0 conn(0x7fbf940c8ee0 0x7fbf940ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:55.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.080+0000 7fbf9159a700 1 -- 172.21.15.33:0/4117980174 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010165477 0 0) 0x7fbf940c9f80 con 0x7fbf94153ed0 2023-11-06T20:59:55.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.080+0000 7fbf9159a700 1 -- 172.21.15.33:0/4117980174 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf7c0033f0 con 0x7fbf94153ed0 2023-11-06T20:59:55.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.080+0000 7fbf91d9b700 1 -- 172.21.15.33:0/4117980174 >> v1:172.21.15.69:6789/0 conn(0x7fbf941542b0 legacy=0x7fbf940c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:55.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.080+0000 7fbf9159a700 1 -- 172.21.15.33:0/4117980174 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1405981382 0 0) 0x7fbf7c0033f0 con 0x7fbf94153ed0 2023-11-06T20:59:55.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.080+0000 7fbf91d9b700 1 -- 172.21.15.33:0/4117980174 >> v1:172.21.15.33:6789/0 conn(0x7fbf94153ed0 legacy=0x7fbf94158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:55.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.080+0000 7fbf91d9b700 1 -- 172.21.15.33:0/4117980174 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf940c9650 con 0x7fbf940c8ee0 2023-11-06T20:59:55.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.080+0000 7fbf91d9b700 1 --2- 172.21.15.33:0/4117980174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf940c8ee0 0x7fbf940ca410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fbf88009760 tx=0x7fbf88009060).ready entity=mon.1 client_cookie=d6582cef947ef815 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:55.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.080+0000 7fbf9159a700 1 -- 172.21.15.33:0/4117980174 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84001470 con 0x7fbf940c8ee0 2023-11-06T20:59:55.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.080+0000 7fbf9159a700 1 -- 172.21.15.33:0/4117980174 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf88011400 con 0x7fbf940c8ee0 2023-11-06T20:59:55.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.080+0000 7fbf9159a700 1 -- 172.21.15.33:0/4117980174 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88011780 con 0x7fbf940c8ee0 2023-11-06T20:59:55.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.081+0000 7fbf98dff700 1 -- 172.21.15.33:0/4117980174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf940c8ee0 msgr2=0x7fbf940ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:55.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.081+0000 7fbf98dff700 1 --2- 172.21.15.33:0/4117980174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf940c8ee0 0x7fbf940ca410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fbf88009760 tx=0x7fbf88009060).stop 2023-11-06T20:59:55.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.081+0000 7fbf98dff700 1 -- 172.21.15.33:0/4117980174 shutdown_connections 2023-11-06T20:59:55.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.081+0000 7fbf98dff700 1 --2- 172.21.15.33:0/4117980174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf940c8ee0 0x7fbf940ca410 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:55.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.081+0000 7fbf98dff700 1 -- 172.21.15.33:0/4117980174 >> 172.21.15.33:0/4117980174 conn(0x7fbf9405e1b0 msgr2=0x7fbf9405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:55.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.081+0000 7fbf98dff700 1 -- 172.21.15.33:0/4117980174 shutdown_connections 2023-11-06T20:59:55.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.081+0000 7fbf98dff700 1 -- 172.21.15.33:0/4117980174 wait complete. 2023-11-06T20:59:55.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.083+0000 7fbf98dff700 1 Processor -- start 2023-11-06T20:59:55.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.083+0000 7fbf98dff700 1 -- start start 2023-11-06T20:59:55.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf98dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbf94153ed0 0x7fbf9414e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:55.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf98dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf941542b0 0x7fbf940ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:55.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf98dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf940aecd0 0x7fbf94155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf98dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf94153540 con 0x7fbf940aecd0 2023-11-06T20:59:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf98dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf940af150 con 0x7fbf941542b0 2023-11-06T20:59:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf98dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf940af430 con 0x7fbf94153ed0 2023-11-06T20:59:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf92d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf940aecd0 0x7fbf94155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf92d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf940aecd0 0x7fbf94155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47174/0 (socket says 172.21.15.33:47174) 2023-11-06T20:59:55.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf92d9d700 1 -- 172.21.15.33:0/216391466 learned_addr learned my addr 172.21.15.33:0/216391466 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:55.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf9259c700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbf94153ed0 0x7fbf9414e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:55.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf91d9b700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf941542b0 0x7fbf940ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:55.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf92d9d700 1 -- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbf94153ed0 msgr2=0x7fbf9414e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf92d9d700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbf94153ed0 0x7fbf9414e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf92d9d700 1 -- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf941542b0 msgr2=0x7fbf940ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf92d9d700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf941542b0 0x7fbf940ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf92d9d700 1 -- 172.21.15.33:0/216391466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf880092a0 con 0x7fbf940aecd0 2023-11-06T20:59:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.084+0000 7fbf91d9b700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf941542b0 0x7fbf940ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.085+0000 7fbf92d9d700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf940aecd0 0x7fbf94155c30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fbf8c00c7c0 tx=0x7fbf8c0014e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.085+0000 7fbf827fc700 1 -- 172.21.15.33:0/216391466 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c010510 con 0x7fbf940aecd0 2023-11-06T20:59:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.085+0000 7fbf827fc700 1 -- 172.21.15.33:0/216391466 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c010670 con 0x7fbf940aecd0 2023-11-06T20:59:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.085+0000 7fbf827fc700 1 -- 172.21.15.33:0/216391466 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c010a10 con 0x7fbf940aecd0 2023-11-06T20:59:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.085+0000 7fbf98dff700 1 -- 172.21.15.33:0/216391466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf94164270 con 0x7fbf940aecd0 2023-11-06T20:59:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.085+0000 7fbf98dff700 1 -- 172.21.15.33:0/216391466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf940af710 con 0x7fbf940aecd0 2023-11-06T20:59:55.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.086+0000 7fbf98dff700 1 -- 172.21.15.33:0/216391466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf941640d0 con 0x7fbf940aecd0 2023-11-06T20:59:55.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.087+0000 7fbf827fc700 1 -- 172.21.15.33:0/216391466 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbf8c01b3b0 con 0x7fbf940aecd0 2023-11-06T20:59:55.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.088+0000 7fbf827fc700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf7405f270 0x7fbf74061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:55.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.088+0000 7fbf827fc700 1 -- 172.21.15.33:0/216391466 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fbf8c079b00 con 0x7fbf940aecd0 2023-11-06T20:59:55.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.088+0000 7fbf9259c700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf7405f270 0x7fbf74061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:55.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.089+0000 7fbf9259c700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf7405f270 0x7fbf74061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fbf840011e0 tx=0x7fbf84001580).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:55.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.092+0000 7fbf827fc700 1 -- 172.21.15.33:0/216391466 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbf941640d0 con 0x7fbf940aecd0 2023-11-06T20:59:55.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:55.274+0000 7fbf98dff700 1 -- 172.21.15.33:0/216391466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fbf941640d0 con 0x7fbf940aecd0 2023-11-06T20:59:56.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.280+0000 7fbf827fc700 1 -- 172.21.15.33:0/216391466 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v96) v1 ==== 152+0+0 (secure 0 0 0) 0x7fbf941640d0 con 0x7fbf940aecd0 2023-11-06T20:59:56.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.285+0000 7fbf98dff700 1 -- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf7405f270 msgr2=0x7fbf74061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:56.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.285+0000 7fbf98dff700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf7405f270 0x7fbf74061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fbf840011e0 tx=0x7fbf84001580).stop 2023-11-06T20:59:56.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.285+0000 7fbf98dff700 1 -- 172.21.15.33:0/216391466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf940aecd0 msgr2=0x7fbf94155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:56.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.285+0000 7fbf98dff700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf940aecd0 0x7fbf94155c30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fbf8c00c7c0 tx=0x7fbf8c0014e0).stop 2023-11-06T20:59:56.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.286+0000 7fbf98dff700 1 -- 172.21.15.33:0/216391466 shutdown_connections 2023-11-06T20:59:56.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.286+0000 7fbf98dff700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbf7405f270 0x7fbf74061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.286+0000 7fbf98dff700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbf94153ed0 0x7fbf9414e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.286+0000 7fbf98dff700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf940aecd0 0x7fbf94155c30 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.286+0000 7fbf98dff700 1 --2- 172.21.15.33:0/216391466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf941542b0 0x7fbf940ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.286+0000 7fbf98dff700 1 -- 172.21.15.33:0/216391466 >> 172.21.15.33:0/216391466 conn(0x7fbf9405e1b0 msgr2=0x7fbf940ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.287+0000 7fbf98dff700 1 -- 172.21.15.33:0/216391466 shutdown_connections 2023-11-06T20:59:56.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.289+0000 7fbf98dff700 1 -- 172.21.15.33:0/216391466 wait complete. 2023-11-06T20:59:56.294 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T20:59:56.312 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T20:59:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.519+0000 7f02779ff700 1 Processor -- start 2023-11-06T20:59:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.520+0000 7f02779ff700 1 -- start start 2023-11-06T20:59:56.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.521+0000 7f02779ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0270154330 0x7f02700c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:56.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.521+0000 7f02779ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f02700c7ff0 0x7f02700646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:56.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.521+0000 7f02779ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02700c6b60 con 0x7f0270154710 2023-11-06T20:59:56.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.521+0000 7f02779ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02700c7700 con 0x7f02700c7ff0 2023-11-06T20:59:56.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.521+0000 7f02779ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02700c5cd0 con 0x7f0270154330 2023-11-06T20:59:56.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.521+0000 7f0275f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f02700c7ff0 0x7f02700646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:56.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.521+0000 7f0275f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f02700c7ff0 0x7f02700646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47180/0 (socket says 172.21.15.33:47180) 2023-11-06T20:59:56.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.521+0000 7f0275f9c700 1 -- 172.21.15.33:0/3865453613 learned_addr learned my addr 172.21.15.33:0/3865453613 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:56.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.521+0000 7f0274f9a700 1 --1- 172.21.15.33:0/3865453613 >> v1:172.21.15.33:6789/0 conn(0x7f0270154710 0x7f02700c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35924/0 (socket says 172.21.15.33:35924) 2023-11-06T20:59:56.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.521+0000 7f027579b700 1 --2- 172.21.15.33:0/3865453613 >> v2:172.21.15.69:3300/0 conn(0x7f0270154330 0x7f02700c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:56.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.522+0000 7f0267fff700 1 -- 172.21.15.33:0/3865453613 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349848598 0 0) 0x7f02700c6b60 con 0x7f0270154710 2023-11-06T20:59:56.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.522+0000 7f0267fff700 1 -- 172.21.15.33:0/3865453613 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f025c0033f0 con 0x7f0270154710 2023-11-06T20:59:56.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.522+0000 7f0275f9c700 1 -- 172.21.15.33:0/3865453613 >> v2:172.21.15.69:3300/0 conn(0x7f0270154330 msgr2=0x7f02700c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:56.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.522+0000 7f0275f9c700 1 --2- 172.21.15.33:0/3865453613 >> v2:172.21.15.69:3300/0 conn(0x7f0270154330 0x7f02700c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:56.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.522+0000 7f0275f9c700 1 -- 172.21.15.33:0/3865453613 >> v1:172.21.15.33:6789/0 conn(0x7f0270154710 legacy=0x7f02700c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:56.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.522+0000 7f0275f9c700 1 -- 172.21.15.33:0/3865453613 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02700c5f50 con 0x7f02700c7ff0 2023-11-06T20:59:56.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.522+0000 7f0275f9c700 1 --2- 172.21.15.33:0/3865453613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02700c7ff0 0x7f02700646c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f02700605d0 tx=0x7f0268009840).ready entity=mon.0 client_cookie=75aa3aa59f396547 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.522+0000 7f0267fff700 1 -- 172.21.15.33:0/3865453613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800e070 con 0x7f02700c7ff0 2023-11-06T20:59:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.522+0000 7f0267fff700 1 -- 172.21.15.33:0/3865453613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026800b970 con 0x7f02700c7ff0 2023-11-06T20:59:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.522+0000 7f0267fff700 1 -- 172.21.15.33:0/3865453613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800bcf0 con 0x7f02700c7ff0 2023-11-06T20:59:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.523+0000 7f02779ff700 1 -- 172.21.15.33:0/3865453613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02700c7ff0 msgr2=0x7f02700646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.523+0000 7f02779ff700 1 --2- 172.21.15.33:0/3865453613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02700c7ff0 0x7f02700646c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f02700605d0 tx=0x7f0268009840).stop 2023-11-06T20:59:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.523+0000 7f02779ff700 1 -- 172.21.15.33:0/3865453613 shutdown_connections 2023-11-06T20:59:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.523+0000 7f02779ff700 1 --2- 172.21.15.33:0/3865453613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02700c7ff0 0x7f02700646c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.523+0000 7f02779ff700 1 --2- 172.21.15.33:0/3865453613 >> v2:172.21.15.69:3300/0 conn(0x7f0270154330 0x7f02700c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.523+0000 7f02779ff700 1 -- 172.21.15.33:0/3865453613 >> 172.21.15.33:0/3865453613 conn(0x7f02700cb910 msgr2=0x7f02700c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.523+0000 7f02779ff700 1 -- 172.21.15.33:0/3865453613 shutdown_connections 2023-11-06T20:59:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.523+0000 7f02779ff700 1 -- 172.21.15.33:0/3865453613 wait complete. 2023-11-06T20:59:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.524+0000 7f02779ff700 1 Processor -- start 2023-11-06T20:59:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.525+0000 7f02779ff700 1 -- start start 2023-11-06T20:59:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.525+0000 7f02779ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02700c7ff0 0x7f0270153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f02779ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0270154330 0x7f0270164620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f02779ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0270154710 0x7f0270155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f02779ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0270154d40 con 0x7f0270154330 2023-11-06T20:59:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f02779ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0270155020 con 0x7f0270154710 2023-11-06T20:59:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f02779ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0270155300 con 0x7f02700c7ff0 2023-11-06T20:59:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0274f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0270154330 0x7f0270164620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0274f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0270154330 0x7f0270164620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47186/0 (socket says 172.21.15.33:47186) 2023-11-06T20:59:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0274f9a700 1 -- 172.21.15.33:0/3416936661 learned_addr learned my addr 172.21.15.33:0/3416936661 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f027579b700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02700c7ff0 0x7f0270153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:56.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0275f9c700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0270154710 0x7f0270155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:56.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0274f9a700 1 -- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02700c7ff0 msgr2=0x7f0270153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:56.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0274f9a700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02700c7ff0 0x7f0270153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:56.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0274f9a700 1 -- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0270154710 msgr2=0x7f0270155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:56.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0274f9a700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0270154710 0x7f0270155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0274f9a700 1 -- 172.21.15.33:0/3416936661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0268009ae0 con 0x7f0270154330 2023-11-06T20:59:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0275f9c700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0270154710 0x7f0270155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f027579b700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02700c7ff0 0x7f0270153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.526+0000 7f0274f9a700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0270154330 0x7f0270164620 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f026000c420 tx=0x7f0260007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.527+0000 7f02657fa700 1 -- 172.21.15.33:0/3416936661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02600072e0 con 0x7f0270154330 2023-11-06T20:59:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.527+0000 7f02657fa700 1 -- 172.21.15.33:0/3416936661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0260007440 con 0x7f0270154330 2023-11-06T20:59:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.527+0000 7f02779ff700 1 -- 172.21.15.33:0/3416936661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02701555e0 con 0x7f0270154330 2023-11-06T20:59:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.527+0000 7f02657fa700 1 -- 172.21.15.33:0/3416936661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260005930 con 0x7f0270154330 2023-11-06T20:59:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.527+0000 7f02779ff700 1 -- 172.21.15.33:0/3416936661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0270153f40 con 0x7f0270154330 2023-11-06T20:59:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.528+0000 7f02779ff700 1 -- 172.21.15.33:0/3416936661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02700830b0 con 0x7f0270154330 2023-11-06T20:59:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.529+0000 7f02657fa700 1 -- 172.21.15.33:0/3416936661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0260005ca0 con 0x7f0270154330 2023-11-06T20:59:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.530+0000 7f02657fa700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f025405f270 0x7f0254061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.530+0000 7f02657fa700 1 -- 172.21.15.33:0/3416936661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f0260079810 con 0x7f0270154330 2023-11-06T20:59:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.531+0000 7f027579b700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f025405f270 0x7f0254061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.531+0000 7f027579b700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f025405f270 0x7f0254061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f026c009c00 tx=0x7f026c008040).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.534+0000 7f02657fa700 1 -- 172.21.15.33:0/3416936661 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f026004abf0 con 0x7f0270154330 2023-11-06T20:59:56.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:56.719+0000 7f02779ff700 1 -- 172.21.15.33:0/3416936661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f02700652d0 con 0x7f0270154330 2023-11-06T20:59:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.285+0000 7f02657fa700 1 -- 172.21.15.33:0/3416936661 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v97) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0260007d50 con 0x7f0270154330 2023-11-06T20:59:57.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.289+0000 7f02779ff700 1 -- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f025405f270 msgr2=0x7f0254061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:57.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.289+0000 7f02779ff700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f025405f270 0x7f0254061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f026c009c00 tx=0x7f026c008040).stop 2023-11-06T20:59:57.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.289+0000 7f02779ff700 1 -- 172.21.15.33:0/3416936661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0270154330 msgr2=0x7f0270164620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:57.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.289+0000 7f02779ff700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0270154330 0x7f0270164620 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f026000c420 tx=0x7f0260007c10).stop 2023-11-06T20:59:57.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.290+0000 7f02779ff700 1 -- 172.21.15.33:0/3416936661 shutdown_connections 2023-11-06T20:59:57.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.290+0000 7f02779ff700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f025405f270 0x7f0254061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.290+0000 7f02779ff700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f02700c7ff0 0x7f0270153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.290+0000 7f02779ff700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0270154710 0x7f0270155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.290+0000 7f02779ff700 1 --2- 172.21.15.33:0/3416936661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0270154330 0x7f0270164620 secure :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0x7f026000c420 tx=0x7f0260007c10).stop 2023-11-06T20:59:57.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.290+0000 7f02779ff700 1 -- 172.21.15.33:0/3416936661 >> 172.21.15.33:0/3416936661 conn(0x7f02700cb910 msgr2=0x7f02700646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:57.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.291+0000 7f02779ff700 1 -- 172.21.15.33:0/3416936661 shutdown_connections 2023-11-06T20:59:57.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.291+0000 7f02779ff700 1 -- 172.21.15.33:0/3416936661 wait complete. 2023-11-06T20:59:57.296 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T20:59:57.333 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T20:59:57.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.533+0000 7f9c9e7ff700 1 Processor -- start 2023-11-06T20:59:57.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.533+0000 7f9c9e7ff700 1 -- start start 2023-11-06T20:59:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.534+0000 7f9c9e7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c98154330 0x7f9c980c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9e7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9c980c7ff0 0x7f9c980646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9e7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c980c6b60 con 0x7f9c98154710 2023-11-06T20:59:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9e7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c980c7700 con 0x7f9c980c7ff0 2023-11-06T20:59:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9e7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980c5cd0 con 0x7f9c98154330 2023-11-06T20:59:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9cd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9c980c7ff0 0x7f9c980646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:57.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9cd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9c980c7ff0 0x7f9c980646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47192/0 (socket says 172.21.15.33:47192) 2023-11-06T20:59:57.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9cd9c700 1 -- 172.21.15.33:0/1539725380 learned_addr learned my addr 172.21.15.33:0/1539725380 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:57.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c97fff700 1 --2- 172.21.15.33:0/1539725380 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c98154330 0x7f9c980c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:57.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9cd9c700 1 -- 172.21.15.33:0/1539725380 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c98154330 msgr2=0x7f9c980c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:57.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9cd9c700 1 --2- 172.21.15.33:0/1539725380 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c98154330 0x7f9c980c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9cd9c700 1 -- 172.21.15.33:0/1539725380 >> v1:172.21.15.33:6789/0 conn(0x7f9c98154710 legacy=0x7f9c980c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c96ffd700 1 -- 172.21.15.33:0/1539725380 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133167199 0 0) 0x7f9c980c6b60 con 0x7f9c98154710 2023-11-06T20:59:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.535+0000 7f9c9cd9c700 1 -- 172.21.15.33:0/1539725380 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c980c5f50 con 0x7f9c980c7ff0 2023-11-06T20:59:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.536+0000 7f9c9cd9c700 1 --2- 172.21.15.33:0/1539725380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c980c7ff0 0x7f9c980646c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f9c90009f20 tx=0x7f9c90009840).ready entity=mon.0 client_cookie=532900510504e650 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:57.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.536+0000 7f9c96ffd700 1 -- 172.21.15.33:0/1539725380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000e070 con 0x7f9c980c7ff0 2023-11-06T20:59:57.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.536+0000 7f9c96ffd700 1 -- 172.21.15.33:0/1539725380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9000b9d0 con 0x7f9c980c7ff0 2023-11-06T20:59:57.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.536+0000 7f9c96ffd700 1 -- 172.21.15.33:0/1539725380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000bd50 con 0x7f9c980c7ff0 2023-11-06T20:59:57.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.536+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/1539725380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c980c7ff0 msgr2=0x7f9c980646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:57.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.536+0000 7f9c9e7ff700 1 --2- 172.21.15.33:0/1539725380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c980c7ff0 0x7f9c980646c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f9c90009f20 tx=0x7f9c90009840).stop 2023-11-06T20:59:57.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.537+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/1539725380 shutdown_connections 2023-11-06T20:59:57.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.537+0000 7f9c9e7ff700 1 --2- 172.21.15.33:0/1539725380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c980c7ff0 0x7f9c980646c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.537+0000 7f9c9e7ff700 1 --2- 172.21.15.33:0/1539725380 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c98154330 0x7f9c980c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.537+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/1539725380 >> 172.21.15.33:0/1539725380 conn(0x7f9c980cb910 msgr2=0x7f9c980c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:57.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.537+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/1539725380 shutdown_connections 2023-11-06T20:59:57.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.537+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/1539725380 wait complete. 2023-11-06T20:59:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.538+0000 7f9c9e7ff700 1 Processor -- start 2023-11-06T20:59:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.538+0000 7f9c9e7ff700 1 -- start start 2023-11-06T20:59:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.539+0000 7f9c9e7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c980c7ff0 0x7f9c98164530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.539+0000 7f9c9e7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98154330 0x7f9c98153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.539+0000 7f9c9e7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c98154710 0x7f9c980c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:57.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.539+0000 7f9c9e7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980aee00 con 0x7f9c98154330 2023-11-06T20:59:57.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.539+0000 7f9c9e7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98164a20 con 0x7f9c98154710 2023-11-06T20:59:57.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.539+0000 7f9c9e7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980af050 con 0x7f9c980c7ff0 2023-11-06T20:59:57.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98154330 0x7f9c98153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98154330 0x7f9c98153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47202/0 (socket says 172.21.15.33:47202) 2023-11-06T20:59:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c977fe700 1 -- 172.21.15.33:0/2723643769 learned_addr learned my addr 172.21.15.33:0/2723643769 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c97fff700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c980c7ff0 0x7f9c98164530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c9cd9c700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c98154710 0x7f9c980c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c977fe700 1 -- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c980c7ff0 msgr2=0x7f9c98164530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c977fe700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c980c7ff0 0x7f9c98164530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c977fe700 1 -- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c98154710 msgr2=0x7f9c980c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c977fe700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c98154710 0x7f9c980c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c977fe700 1 -- 172.21.15.33:0/2723643769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c90009ae0 con 0x7f9c98154330 2023-11-06T20:59:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c977fe700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98154330 0x7f9c98153a70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f9c8c00c4c0 tx=0x7f9c8c007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c7ffff700 1 -- 172.21.15.33:0/2723643769 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c007020 con 0x7f9c98154330 2023-11-06T20:59:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c7ffff700 1 -- 172.21.15.33:0/2723643769 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c007180 con 0x7f9c98154330 2023-11-06T20:59:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/2723643769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c980af330 con 0x7f9c98154330 2023-11-06T20:59:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.540+0000 7f9c7ffff700 1 -- 172.21.15.33:0/2723643769 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c007540 con 0x7f9c98154330 2023-11-06T20:59:57.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.541+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/2723643769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c980af970 con 0x7f9c98154330 2023-11-06T20:59:57.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.542+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/2723643769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c980830b0 con 0x7f9c98154330 2023-11-06T20:59:57.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.543+0000 7f9c7ffff700 1 -- 172.21.15.33:0/2723643769 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9c8c0068b0 con 0x7f9c98154330 2023-11-06T20:59:57.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.544+0000 7f9c7ffff700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9c8405f270 0x7f9c84061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:57.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.544+0000 7f9c7ffff700 1 -- 172.21.15.33:0/2723643769 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f9c8c079af0 con 0x7f9c98154330 2023-11-06T20:59:57.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.544+0000 7f9c97fff700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9c8405f270 0x7f9c84061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:57.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.545+0000 7f9c97fff700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9c8405f270 0x7f9c84061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9c88009c30 tx=0x7f9c88009060).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:57.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.548+0000 7f9c7ffff700 1 -- 172.21.15.33:0/2723643769 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9c8c04aca0 con 0x7f9c98154330 2023-11-06T20:59:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.773+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/2723643769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9c980c4b20 con 0x7f9c98154330 2023-11-06T20:59:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.779+0000 7f9c7ffff700 1 -- 172.21.15.33:0/2723643769 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v33) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9c8c001e60 con 0x7f9c98154330 2023-11-06T20:59:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.782+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9c8405f270 msgr2=0x7f9c84061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.782+0000 7f9c9e7ff700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9c8405f270 0x7f9c84061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9c88009c30 tx=0x7f9c88009060).stop 2023-11-06T20:59:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.782+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/2723643769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98154330 msgr2=0x7f9c98153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.782+0000 7f9c9e7ff700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98154330 0x7f9c98153a70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f9c8c00c4c0 tx=0x7f9c8c007da0).stop 2023-11-06T20:59:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.783+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/2723643769 shutdown_connections 2023-11-06T20:59:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.783+0000 7f9c9e7ff700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9c8405f270 0x7f9c84061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.783+0000 7f9c9e7ff700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c980c7ff0 0x7f9c98164530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.783+0000 7f9c9e7ff700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c98154710 0x7f9c980c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.783+0000 7f9c9e7ff700 1 --2- 172.21.15.33:0/2723643769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98154330 0x7f9c98153a70 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:57.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.783+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/2723643769 >> 172.21.15.33:0/2723643769 conn(0x7f9c980cb910 msgr2=0x7f9c980ac4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.785+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/2723643769 shutdown_connections 2023-11-06T20:59:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:57.785+0000 7f9c9e7ff700 1 -- 172.21.15.33:0/2723643769 wait complete. 2023-11-06T20:59:57.790 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 11 and data pool 12 2023-11-06T20:59:57.820 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:58.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.017+0000 7f1c7d5ff700 1 Processor -- start 2023-11-06T20:59:58.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.017+0000 7f1c7d5ff700 1 -- start start 2023-11-06T20:59:58.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.018+0000 7f1c7d5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1c7805e1b0 0x7f1c78063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.018+0000 7f1c7d5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c780c3680 0x7f1c78154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.018+0000 7f1c7d5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c780c31f0 con 0x7f1c78063140 2023-11-06T20:59:58.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.018+0000 7f1c7d5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7806a0f0 con 0x7f1c7805e1b0 2023-11-06T20:59:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.018+0000 7f1c7d5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780c2920 con 0x7f1c780c3680 2023-11-06T20:59:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.018+0000 7f1c767fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1c7805e1b0 0x7f1c78063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.018+0000 7f1c76ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1c78063140 0x7f1c780c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38820/0 (socket says 172.21.15.33:38820) 2023-11-06T20:59:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c76ffd700 1 -- 172.21.15.33:0/822969896 learned_addr learned my addr 172.21.15.33:0/822969896 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c777fe700 1 --2- 172.21.15.33:0/822969896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c780c3680 0x7f1c78154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c767fc700 1 -- 172.21.15.33:0/822969896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c780c3680 msgr2=0x7f1c78154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c767fc700 1 --2- 172.21.15.33:0/822969896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c780c3680 0x7f1c78154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c767fc700 1 -- 172.21.15.33:0/822969896 >> v1:172.21.15.33:6789/0 conn(0x7f1c78063140 legacy=0x7f1c780c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c767fc700 1 -- 172.21.15.33:0/822969896 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7805e640 con 0x7f1c7805e1b0 2023-11-06T20:59:58.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c75ffb700 1 -- 172.21.15.33:0/822969896 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611371523 0 0) 0x7f1c780c31f0 con 0x7f1c78063140 2023-11-06T20:59:58.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c767fc700 1 --2- 172.21.15.33:0/822969896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c7805e1b0 0x7f1c78063520 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f1c6c009760 tx=0x7f1c6c009060).ready entity=mon.0 client_cookie=838e329d4b034ade server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:58.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c75ffb700 1 -- 172.21.15.33:0/822969896 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00b580 con 0x7f1c7805e1b0 2023-11-06T20:59:58.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c75ffb700 1 -- 172.21.15.33:0/822969896 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c00b6e0 con 0x7f1c7805e1b0 2023-11-06T20:59:58.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.019+0000 7f1c75ffb700 1 -- 172.21.15.33:0/822969896 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00ba60 con 0x7f1c7805e1b0 2023-11-06T20:59:58.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.020+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/822969896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c7805e1b0 msgr2=0x7f1c78063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.020+0000 7f1c7d5ff700 1 --2- 172.21.15.33:0/822969896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c7805e1b0 0x7f1c78063520 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f1c6c009760 tx=0x7f1c6c009060).stop 2023-11-06T20:59:58.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.020+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/822969896 shutdown_connections 2023-11-06T20:59:58.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.020+0000 7f1c7d5ff700 1 --2- 172.21.15.33:0/822969896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c780c3680 0x7f1c78154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.020+0000 7f1c7d5ff700 1 --2- 172.21.15.33:0/822969896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c7805e1b0 0x7f1c78063520 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.020+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/822969896 >> 172.21.15.33:0/822969896 conn(0x7f1c780c7d60 msgr2=0x7f1c780c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:58.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.020+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/822969896 shutdown_connections 2023-11-06T20:59:58.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.020+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/822969896 wait complete. 2023-11-06T20:59:58.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.021+0000 7f1c7d5ff700 1 Processor -- start 2023-11-06T20:59:58.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.022+0000 7f1c7d5ff700 1 -- start start 2023-11-06T20:59:58.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.022+0000 7f1c7d5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78063140 0x7f1c78156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c7d5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c780c3680 0x7f1c78164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c7d5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c78155290 0x7f1c78165030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c7d5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7814e660 con 0x7f1c78155290 2023-11-06T20:59:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c7d5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c781574a0 con 0x7f1c78063140 2023-11-06T20:59:58.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c7d5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c78157780 con 0x7f1c780c3680 2023-11-06T20:59:58.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c78155290 0x7f1c78165030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c78155290 0x7f1c78165030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59750/0 (socket says 172.21.15.33:59750) 2023-11-06T20:59:58.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c777fe700 1 -- 172.21.15.33:0/1959970408 learned_addr learned my addr 172.21.15.33:0/1959970408 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:58.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c767fc700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c780c3680 0x7f1c78164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c76ffd700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78063140 0x7f1c78156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c777fe700 1 -- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c780c3680 msgr2=0x7f1c78164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c777fe700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c780c3680 0x7f1c78164b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c777fe700 1 -- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78063140 msgr2=0x7f1c78156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c777fe700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78063140 0x7f1c78156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c777fe700 1 -- 172.21.15.33:0/1959970408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c0092a0 con 0x7f1c78155290 2023-11-06T20:59:58.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c76ffd700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78063140 0x7f1c78156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T20:59:58.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.023+0000 7f1c777fe700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c78155290 0x7f1c78165030 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f1c70009fa0 tx=0x7f1c70009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:58.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.024+0000 7f1c5effd700 1 -- 172.21.15.33:0/1959970408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000ec70 con 0x7f1c78155290 2023-11-06T20:59:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.024+0000 7f1c5effd700 1 -- 172.21.15.33:0/1959970408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7000edd0 con 0x7f1c78155290 2023-11-06T20:59:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.024+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/1959970408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c78157a60 con 0x7f1c78155290 2023-11-06T20:59:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.024+0000 7f1c5effd700 1 -- 172.21.15.33:0/1959970408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70014610 con 0x7f1c78155290 2023-11-06T20:59:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.024+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/1959970408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c78156f40 con 0x7f1c78155290 2023-11-06T20:59:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.025+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/1959970408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c78064420 con 0x7f1c78155290 2023-11-06T20:59:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.026+0000 7f1c5effd700 1 -- 172.21.15.33:0/1959970408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1c7001d3b0 con 0x7f1c78155290 2023-11-06T20:59:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.027+0000 7f1c5effd700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c6405f270 0x7f1c64061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.027+0000 7f1c5effd700 1 -- 172.21.15.33:0/1959970408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1c7007a9e0 con 0x7f1c78155290 2023-11-06T20:59:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.027+0000 7f1c76ffd700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c6405f270 0x7f1c64061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.028+0000 7f1c76ffd700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c6405f270 0x7f1c64061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1c68007fd0 tx=0x7f1c68009040).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:58.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.031+0000 7f1c5effd700 1 -- 172.21.15.33:0/1959970408 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1c7004ba10 con 0x7f1c78155290 2023-11-06T20:59:58.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.213+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/1959970408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c7806bfd0 con 0x7f1c78155290 2023-11-06T20:59:58.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.214+0000 7f1c5effd700 1 -- 172.21.15.33:0/1959970408 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16992 (secure 0 0 0) 0x7f1c7004f030 con 0x7f1c78155290 2023-11-06T20:59:58.214 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:58.214 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":98,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:57.774377+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:55.320441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:56.766396+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:58.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c6405f270 msgr2=0x7f1c64061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c6405f270 0x7f1c64061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1c68007fd0 tx=0x7f1c68009040).stop 2023-11-06T20:59:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/1959970408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c78155290 msgr2=0x7f1c78165030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c78155290 0x7f1c78165030 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f1c70009fa0 tx=0x7f1c70009900).stop 2023-11-06T20:59:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/1959970408 shutdown_connections 2023-11-06T20:59:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c6405f270 0x7f1c64061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c78155290 0x7f1c78165030 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78063140 0x7f1c78156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 --2- 172.21.15.33:0/1959970408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c780c3680 0x7f1c78164b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/1959970408 >> 172.21.15.33:0/1959970408 conn(0x7f1c780c7d60 msgr2=0x7f1c780c6410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:58.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/1959970408 shutdown_connections 2023-11-06T20:59:58.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.216+0000 7f1c7d5ff700 1 -- 172.21.15.33:0/1959970408 wait complete. 2023-11-06T20:59:58.233 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T20:59:58.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.431+0000 7f60fe1ff700 1 Processor -- start 2023-11-06T20:59:58.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.431+0000 7f60fe1ff700 1 -- start start 2023-11-06T20:59:58.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.433+0000 7f60fe1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f60f81580e0 0x7f60f81584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.433+0000 7f60fe1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f60f80bdbe0 0x7f60f80684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.433+0000 7f60fe1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f8064010 con 0x7f60f80a7910 2023-11-06T20:59:58.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.433+0000 7f60fe1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f80be430 con 0x7f60f81580e0 2023-11-06T20:59:58.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.433+0000 7f60fe1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f80689b0 con 0x7f60f80bdbe0 2023-11-06T20:59:58.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.433+0000 7f60f77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f60f81580e0 0x7f60f81584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.433+0000 7f60f7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f60f80a7910 0x7f60f80a7d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38828/0 (socket says 172.21.15.33:38828) 2023-11-06T20:59:58.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.433+0000 7f60f7fff700 1 -- 172.21.15.33:0/607652846 learned_addr learned my addr 172.21.15.33:0/607652846 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.433+0000 7f60f6ffd700 1 --2- 172.21.15.33:0/607652846 >> v2:172.21.15.69:3300/0 conn(0x7f60f80bdbe0 0x7f60f80684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.434+0000 7f60f67fc700 1 -- 172.21.15.33:0/607652846 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401861120 0 0) 0x7f60f8064010 con 0x7f60f80a7910 2023-11-06T20:59:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.434+0000 7f60f77fe700 1 -- 172.21.15.33:0/607652846 >> v2:172.21.15.69:3300/0 conn(0x7f60f80bdbe0 msgr2=0x7f60f80684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T20:59:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.434+0000 7f60f77fe700 1 --2- 172.21.15.33:0/607652846 >> v2:172.21.15.69:3300/0 conn(0x7f60f80bdbe0 0x7f60f80684c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.434+0000 7f60f77fe700 1 -- 172.21.15.33:0/607652846 >> v1:172.21.15.33:6789/0 conn(0x7f60f80a7910 legacy=0x7f60f80a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.434+0000 7f60f77fe700 1 -- 172.21.15.33:0/607652846 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f80be150 con 0x7f60f81580e0 2023-11-06T20:59:58.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.435+0000 7f60f77fe700 1 --2- 172.21.15.33:0/607652846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 0x7f60f81584c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f60f80bfd70 tx=0x7f60e8009060).ready entity=mon.0 client_cookie=74f218633077b777 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:58.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.435+0000 7f60f67fc700 1 -- 172.21.15.33:0/607652846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e800b580 con 0x7f60f81580e0 2023-11-06T20:59:58.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.435+0000 7f60f67fc700 1 -- 172.21.15.33:0/607652846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e800b6e0 con 0x7f60f81580e0 2023-11-06T20:59:58.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.435+0000 7f60f67fc700 1 -- 172.21.15.33:0/607652846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e800ba60 con 0x7f60f81580e0 2023-11-06T20:59:58.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.436+0000 7f60fe1ff700 1 -- 172.21.15.33:0/607652846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 msgr2=0x7f60f81584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.436+0000 7f60fe1ff700 1 --2- 172.21.15.33:0/607652846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 0x7f60f81584c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f60f80bfd70 tx=0x7f60e8009060).stop 2023-11-06T20:59:58.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.436+0000 7f60fe1ff700 1 -- 172.21.15.33:0/607652846 shutdown_connections 2023-11-06T20:59:58.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.436+0000 7f60fe1ff700 1 --2- 172.21.15.33:0/607652846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 0x7f60f81584c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.436+0000 7f60fe1ff700 1 --2- 172.21.15.33:0/607652846 >> v2:172.21.15.69:3300/0 conn(0x7f60f80bdbe0 0x7f60f80684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.436+0000 7f60fe1ff700 1 -- 172.21.15.33:0/607652846 >> 172.21.15.33:0/607652846 conn(0x7f60f805ec80 msgr2=0x7f60f80664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:58.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.436+0000 7f60fe1ff700 1 -- 172.21.15.33:0/607652846 shutdown_connections 2023-11-06T20:59:58.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.436+0000 7f60fe1ff700 1 -- 172.21.15.33:0/607652846 wait complete. 2023-11-06T20:59:58.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.437+0000 7f60fe1ff700 1 Processor -- start 2023-11-06T20:59:58.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.438+0000 7f60fe1ff700 1 -- start start 2023-11-06T20:59:58.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60fe1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60f80a7910 0x7f60f81398b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60fe1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60f80bdbe0 0x7f60f80acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60fe1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 0x7f60f80ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60fe1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f814f6d0 con 0x7f60f81580e0 2023-11-06T20:59:58.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60fe1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f8139da0 con 0x7f60f80a7910 2023-11-06T20:59:58.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60fe1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f80ae5c0 con 0x7f60f80bdbe0 2023-11-06T20:59:58.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 0x7f60f80ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 0x7f60f80ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59762/0 (socket says 172.21.15.33:59762) 2023-11-06T20:59:58.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f7fff700 1 -- 172.21.15.33:0/4141434431 learned_addr learned my addr 172.21.15.33:0/4141434431 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:58.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f77fe700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60f80a7910 0x7f60f81398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f6ffd700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60f80bdbe0 0x7f60f80acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f7fff700 1 -- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60f80bdbe0 msgr2=0x7f60f80acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f7fff700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60f80bdbe0 0x7f60f80acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f7fff700 1 -- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60f80a7910 msgr2=0x7f60f81398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f7fff700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60f80a7910 0x7f60f81398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f7fff700 1 -- 172.21.15.33:0/4141434431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e80092a0 con 0x7f60f81580e0 2023-11-06T20:59:58.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.439+0000 7f60f6ffd700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60f80bdbe0 0x7f60f80acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:58.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.440+0000 7f60f7fff700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 0x7f60f80ad0d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f60f000c7e0 tx=0x7f60f0007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:58.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.440+0000 7f60df7fe700 1 -- 172.21.15.33:0/4141434431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0005d30 con 0x7f60f81580e0 2023-11-06T20:59:58.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.440+0000 7f60df7fe700 1 -- 172.21.15.33:0/4141434431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f0005e90 con 0x7f60f81580e0 2023-11-06T20:59:58.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.440+0000 7f60fe1ff700 1 -- 172.21.15.33:0/4141434431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f80add20 con 0x7f60f81580e0 2023-11-06T20:59:58.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.440+0000 7f60df7fe700 1 -- 172.21.15.33:0/4141434431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0007320 con 0x7f60f81580e0 2023-11-06T20:59:58.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.440+0000 7f60fe1ff700 1 -- 172.21.15.33:0/4141434431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f80ae2e0 con 0x7f60f81580e0 2023-11-06T20:59:58.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.441+0000 7f60fe1ff700 1 -- 172.21.15.33:0/4141434431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f8066bf0 con 0x7f60f81580e0 2023-11-06T20:59:58.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.442+0000 7f60df7fe700 1 -- 172.21.15.33:0/4141434431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f60f0007710 con 0x7f60f81580e0 2023-11-06T20:59:58.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.443+0000 7f60df7fe700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f60e405f240 0x7f60e4061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.443+0000 7f60df7fe700 1 -- 172.21.15.33:0/4141434431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f60f007a530 con 0x7f60f81580e0 2023-11-06T20:59:58.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.443+0000 7f60f77fe700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f60e405f240 0x7f60e4061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.444+0000 7f60f77fe700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f60e405f240 0x7f60e4061700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f60e800b340 tx=0x7f60e8017000).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:58.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.447+0000 7f60df7fe700 1 -- 172.21.15.33:0/4141434431 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f60f004b5c0 con 0x7f60f81580e0 2023-11-06T20:59:58.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.627+0000 7f60fe1ff700 1 -- 172.21.15.33:0/4141434431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60f80c86e0 con 0x7f60f81580e0 2023-11-06T20:59:58.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.628+0000 7f60df7fe700 1 -- 172.21.15.33:0/4141434431 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16992 (secure 0 0 0) 0x7f60f004ebe0 con 0x7f60f81580e0 2023-11-06T20:59:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T20:59:58.628 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":98,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:57.774377+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:55.320441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:56.766396+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T20:59:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 -- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f60e405f240 msgr2=0x7f60e4061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f60e405f240 0x7f60e4061700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f60e800b340 tx=0x7f60e8017000).stop 2023-11-06T20:59:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 -- 172.21.15.33:0/4141434431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 msgr2=0x7f60f80ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 0x7f60f80ad0d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f60f000c7e0 tx=0x7f60f0007f90).stop 2023-11-06T20:59:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 -- 172.21.15.33:0/4141434431 shutdown_connections 2023-11-06T20:59:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f60e405f240 0x7f60e4061700 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f81580e0 0x7f60f80ad0d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60f80a7910 0x7f60f81398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 --2- 172.21.15.33:0/4141434431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60f80bdbe0 0x7f60f80acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 -- 172.21.15.33:0/4141434431 >> 172.21.15.33:0/4141434431 conn(0x7f60f805ec80 msgr2=0x7f60f8068290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:58.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 -- 172.21.15.33:0/4141434431 shutdown_connections 2023-11-06T20:59:58.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.630+0000 7f60fe1ff700 1 -- 172.21.15.33:0/4141434431 wait complete. 2023-11-06T20:59:58.648 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T20:59:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.847+0000 7f2bd93ff700 1 Processor -- start 2023-11-06T20:59:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.847+0000 7f2bd93ff700 1 -- start start 2023-11-06T20:59:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.849+0000 7f2bd93ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154710 0x7f2bd40c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.849+0000 7f2bd93ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd40c6b60 con 0x7f2bd4154330 2023-11-06T20:59:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.849+0000 7f2bd93ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd40c7700 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.849+0000 7f2bd93ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40c5cd0 con 0x7f2bd4154710 2023-11-06T20:59:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.849+0000 7f2bd2ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2bd4154330 0x7f2bd40c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38842/0 (socket says 172.21.15.33:38842) 2023-11-06T20:59:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.849+0000 7f2bd2ffd700 1 -- 172.21.15.33:0/2433793749 learned_addr learned my addr 172.21.15.33:0/2433793749 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T20:59:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.849+0000 7f2bd27fc700 1 --2- 172.21.15.33:0/2433793749 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154710 0x7f2bd40c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.850+0000 7f2bd1ffb700 1 -- 172.21.15.33:0/2433793749 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160307978 0 0) 0x7f2bd40c6b60 con 0x7f2bd4154330 2023-11-06T20:59:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.850+0000 7f2bd1ffb700 1 -- 172.21.15.33:0/2433793749 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bbc0033f0 con 0x7f2bd4154330 2023-11-06T20:59:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.850+0000 7f2bd1ffb700 1 -- 172.21.15.33:0/2433793749 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157314921 0 0) 0x7f2bd40c7700 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.850+0000 7f2bd1ffb700 1 -- 172.21.15.33:0/2433793749 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd40c6b60 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.850+0000 7f2bd27fc700 1 -- 172.21.15.33:0/2433793749 >> v1:172.21.15.69:6789/0 conn(0x7f2bd40c7ff0 legacy=0x7f2bd40646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.850+0000 7f2bd27fc700 1 -- 172.21.15.33:0/2433793749 >> v1:172.21.15.33:6789/0 conn(0x7f2bd4154330 legacy=0x7f2bd40c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.850+0000 7f2bd1ffb700 1 -- 172.21.15.33:0/2433793749 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1788218562 0 0) 0x7f2bbc0033f0 con 0x7f2bd4154330 2023-11-06T20:59:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.850+0000 7f2bd27fc700 1 -- 172.21.15.33:0/2433793749 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd40c5f50 con 0x7f2bd4154710 2023-11-06T20:59:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.850+0000 7f2bd27fc700 1 --2- 172.21.15.33:0/2433793749 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154710 0x7f2bd40c57e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f2bc8009790 tx=0x7f2bc8009060).ready entity=mon.2 client_cookie=cf78eb39d9561212 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.850+0000 7f2bd1ffb700 1 -- 172.21.15.33:0/2433793749 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4001470 con 0x7f2bd4154710 2023-11-06T20:59:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.851+0000 7f2bd1ffb700 1 -- 172.21.15.33:0/2433793749 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc8011400 con 0x7f2bd4154710 2023-11-06T20:59:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.851+0000 7f2bd1ffb700 1 -- 172.21.15.33:0/2433793749 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8011780 con 0x7f2bd4154710 2023-11-06T20:59:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.851+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2433793749 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154710 msgr2=0x7f2bd40c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.851+0000 7f2bd93ff700 1 --2- 172.21.15.33:0/2433793749 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154710 0x7f2bd40c57e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f2bc8009790 tx=0x7f2bc8009060).stop 2023-11-06T20:59:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.851+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2433793749 shutdown_connections 2023-11-06T20:59:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.851+0000 7f2bd93ff700 1 --2- 172.21.15.33:0/2433793749 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154710 0x7f2bd40c57e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.851+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2433793749 >> 172.21.15.33:0/2433793749 conn(0x7f2bd40cb910 msgr2=0x7f2bd40c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.851+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2433793749 shutdown_connections 2023-11-06T20:59:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.851+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2433793749 wait complete. 2023-11-06T20:59:58.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.853+0000 7f2bd93ff700 1 Processor -- start 2023-11-06T20:59:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.853+0000 7f2bd93ff700 1 -- start start 2023-11-06T20:59:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd93ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd40c7ff0 0x7f2bd4168450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd93ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154330 0x7f2bd41524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd93ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4154710 0x7f2bd40c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd93ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40a4bd0 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd93ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd4168940 con 0x7f2bd4154710 2023-11-06T20:59:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd93ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd41648e0 con 0x7f2bd4154330 2023-11-06T20:59:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd40c7ff0 0x7f2bd4168450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd40c7ff0 0x7f2bd4168450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59772/0 (socket says 172.21.15.33:59772) 2023-11-06T20:59:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd2ffd700 1 -- 172.21.15.33:0/2868546162 learned_addr learned my addr 172.21.15.33:0/2868546162 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T20:59:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd27fc700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154330 0x7f2bd41524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd37fe700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4154710 0x7f2bd40c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd2ffd700 1 -- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154330 msgr2=0x7f2bd41524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd2ffd700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154330 0x7f2bd41524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd2ffd700 1 -- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4154710 msgr2=0x7f2bd40c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd2ffd700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4154710 0x7f2bd40c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd2ffd700 1 -- 172.21.15.33:0/2868546162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80092a0 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.854+0000 7f2bd37fe700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4154710 0x7f2bd40c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T20:59:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.855+0000 7f2bd2ffd700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd40c7ff0 0x7f2bd4168450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f2bc400c7c0 tx=0x7f2bc4001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:58.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.855+0000 7f2bbaffd700 1 -- 172.21.15.33:0/2868546162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4007170 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.855+0000 7f2bbaffd700 1 -- 172.21.15.33:0/2868546162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc40072f0 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.855+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2868546162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd4164bc0 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.855+0000 7f2bbaffd700 1 -- 172.21.15.33:0/2868546162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40048c0 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.855+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2868546162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd4165180 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.856+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2868546162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd40830b0 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.857+0000 7f2bbaffd700 1 -- 172.21.15.33:0/2868546162 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2bc4004c70 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.858+0000 7f2bbaffd700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2bc005f270 0x7f2bc0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T20:59:58.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.858+0000 7f2bbaffd700 1 -- 172.21.15.33:0/2868546162 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2bc4079f30 con 0x7f2bd40c7ff0 2023-11-06T20:59:58.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.858+0000 7f2bd27fc700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2bc005f270 0x7f2bc0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T20:59:58.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.859+0000 7f2bd27fc700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2bc005f270 0x7f2bc0061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f2bd4154b70 tx=0x7f2bc8016000).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T20:59:58.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:58.862+0000 7f2bbaffd700 1 -- 172.21.15.33:0/2868546162 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2bc404b080 con 0x7f2bd40c7ff0 2023-11-06T20:59:59.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.088+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2868546162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2bd4165310 con 0x7f2bd40c7ff0 2023-11-06T20:59:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.792+0000 7f2bbaffd700 1 -- 172.21.15.33:0/2868546162 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2bc4001c70 con 0x7f2bd40c7ff0 2023-11-06T20:59:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2bc005f270 msgr2=0x7f2bc0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2bc005f270 0x7f2bc0061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f2bd4154b70 tx=0x7f2bc8016000).stop 2023-11-06T20:59:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2868546162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd40c7ff0 msgr2=0x7f2bd4168450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T20:59:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd40c7ff0 0x7f2bd4168450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f2bc400c7c0 tx=0x7f2bc4001b50).stop 2023-11-06T20:59:59.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2868546162 shutdown_connections 2023-11-06T20:59:59.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2bc005f270 0x7f2bc0061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:59.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4154710 0x7f2bd40c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:59.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd40c7ff0 0x7f2bd4168450 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:59.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 --2- 172.21.15.33:0/2868546162 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bd4154330 0x7f2bd41524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T20:59:59.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2868546162 >> 172.21.15.33:0/2868546162 conn(0x7f2bd40cb910 msgr2=0x7f2bd40aaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T20:59:59.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2868546162 shutdown_connections 2023-11-06T20:59:59.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T20:59:59.795+0000 7f2bd93ff700 1 -- 172.21.15.33:0/2868546162 wait complete. 2023-11-06T20:59:59.815 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T20:59:59.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T21:00:00.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.023+0000 7f55749ff700 1 Processor -- start 2023-11-06T21:00:00.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.023+0000 7f55749ff700 1 -- start start 2023-11-06T21:00:00.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.024+0000 7f55749ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5570154330 0x7f55700c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.024+0000 7f55749ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5570154710 0x7f55700c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.024+0000 7f55749ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55700c6b60 con 0x7f55700c7ff0 2023-11-06T21:00:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.024+0000 7f55749ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55700c5cd0 con 0x7f5570154330 2023-11-06T21:00:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.024+0000 7f55749ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55700c5f50 con 0x7f5570154710 2023-11-06T21:00:00.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.025+0000 7f556e59c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5570154330 0x7f55700c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:00.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.025+0000 7f556dd9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5570154710 0x7f55700c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.025+0000 7f556e59c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5570154330 0x7f55700c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:46946/0 (socket says 172.21.15.33:46946) 2023-11-06T21:00:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.025+0000 7f556e59c700 1 -- 172.21.15.33:0/3236672078 learned_addr learned my addr 172.21.15.33:0/3236672078 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.025+0000 7f556ed9d700 1 --1- 172.21.15.33:0/3236672078 >> v1:172.21.15.69:6789/0 conn(0x7f55700c7ff0 0x7f55700646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:53092/0 (socket says 172.21.15.33:53092) 2023-11-06T21:00:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.025+0000 7f556e59c700 1 -- 172.21.15.33:0/3236672078 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5570154710 msgr2=0x7f55700c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.025+0000 7f556e59c700 1 --2- 172.21.15.33:0/3236672078 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5570154710 0x7f55700c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.025+0000 7f556e59c700 1 -- 172.21.15.33:0/3236672078 >> v1:172.21.15.69:6789/0 conn(0x7f55700c7ff0 legacy=0x7f55700646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.025+0000 7f556e59c700 1 -- 172.21.15.33:0/3236672078 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55700c6230 con 0x7f5570154330 2023-11-06T21:00:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f556e59c700 1 --2- 172.21.15.33:0/3236672078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5570154330 0x7f55700c57e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f5560006fd0 tx=0x7f55600090f0).ready entity=mon.1 client_cookie=952fd13d17e71734 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f556d59a700 1 -- 172.21.15.33:0/3236672078 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556000b580 con 0x7f5570154330 2023-11-06T21:00:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f556d59a700 1 -- 172.21.15.33:0/3236672078 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556000b6e0 con 0x7f5570154330 2023-11-06T21:00:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f556d59a700 1 -- 172.21.15.33:0/3236672078 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556000ba60 con 0x7f5570154330 2023-11-06T21:00:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f55749ff700 1 -- 172.21.15.33:0/3236672078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5570154330 msgr2=0x7f55700c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:00.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f55749ff700 1 --2- 172.21.15.33:0/3236672078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5570154330 0x7f55700c57e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f5560006fd0 tx=0x7f55600090f0).stop 2023-11-06T21:00:00.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f55749ff700 1 -- 172.21.15.33:0/3236672078 shutdown_connections 2023-11-06T21:00:00.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f55749ff700 1 --2- 172.21.15.33:0/3236672078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5570154330 0x7f55700c57e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:00.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f55749ff700 1 --2- 172.21.15.33:0/3236672078 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5570154710 0x7f55700c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:00.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f55749ff700 1 -- 172.21.15.33:0/3236672078 >> 172.21.15.33:0/3236672078 conn(0x7f55700cb910 msgr2=0x7f55700c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:00.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.026+0000 7f55749ff700 1 -- 172.21.15.33:0/3236672078 shutdown_connections 2023-11-06T21:00:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.027+0000 7f55749ff700 1 -- 172.21.15.33:0/3236672078 wait complete. 2023-11-06T21:00:00.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.028+0000 7f55749ff700 1 Processor -- start 2023-11-06T21:00:00.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.028+0000 7f55749ff700 1 -- start start 2023-11-06T21:00:00.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.029+0000 7f55749ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55700c7ff0 0x7f5570150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:00.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.029+0000 7f55749ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5570154330 0x7f557015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:00.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.029+0000 7f55749ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5570154710 0x7f5570154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:00.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.029+0000 7f55749ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5570159580 con 0x7f55700c7ff0 2023-11-06T21:00:00.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.029+0000 7f55749ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5570151220 con 0x7f5570154710 2023-11-06T21:00:00.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.029+0000 7f55749ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5570158fc0 con 0x7f5570154330 2023-11-06T21:00:00.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.029+0000 7f556e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55700c7ff0 0x7f5570150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:00.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55700c7ff0 0x7f5570150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59788/0 (socket says 172.21.15.33:59788) 2023-11-06T21:00:00.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556e59c700 1 -- 172.21.15.33:0/578948390 learned_addr learned my addr 172.21.15.33:0/578948390 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:00.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556ed9d700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5570154710 0x7f5570154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:00.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556dd9b700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5570154330 0x7f557015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:00.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556e59c700 1 -- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5570154330 msgr2=0x7f557015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:00.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556e59c700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5570154330 0x7f557015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556e59c700 1 -- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5570154710 msgr2=0x7f5570154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556e59c700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5570154710 0x7f5570154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556e59c700 1 -- 172.21.15.33:0/578948390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5560009370 con 0x7f55700c7ff0 2023-11-06T21:00:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556ed9d700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5570154710 0x7f5570154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.030+0000 7f556e59c700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55700c7ff0 0x7f5570150d30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f5560009130 tx=0x7f5560017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.031+0000 7f555e7fc700 1 -- 172.21.15.33:0/578948390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560000bc0 con 0x7f55700c7ff0 2023-11-06T21:00:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.031+0000 7f555e7fc700 1 -- 172.21.15.33:0/578948390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556000b850 con 0x7f55700c7ff0 2023-11-06T21:00:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.031+0000 7f555e7fc700 1 -- 172.21.15.33:0/578948390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560000d20 con 0x7f55700c7ff0 2023-11-06T21:00:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.031+0000 7f55749ff700 1 -- 172.21.15.33:0/578948390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55701592a0 con 0x7f55700c7ff0 2023-11-06T21:00:00.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.031+0000 7f55749ff700 1 -- 172.21.15.33:0/578948390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55700afd30 con 0x7f55700c7ff0 2023-11-06T21:00:00.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.032+0000 7f55749ff700 1 -- 172.21.15.33:0/578948390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55700af8b0 con 0x7f55700c7ff0 2023-11-06T21:00:00.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.032+0000 7f555e7fc700 1 -- 172.21.15.33:0/578948390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f556001c900 con 0x7f55700c7ff0 2023-11-06T21:00:00.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.033+0000 7f555e7fc700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f555005f270 0x7f5550061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:00.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.034+0000 7f555e7fc700 1 -- 172.21.15.33:0/578948390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f556007c140 con 0x7f55700c7ff0 2023-11-06T21:00:00.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.034+0000 7f556dd9b700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f555005f270 0x7f5550061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:00.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.035+0000 7f556dd9b700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f555005f270 0x7f5550061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f5570158a10 tx=0x7f5564009060).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.037+0000 7f555e7fc700 1 -- 172.21.15.33:0/578948390 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f55700af8b0 con 0x7f55700c7ff0 2023-11-06T21:00:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.265+0000 7f55749ff700 1 -- 172.21.15.33:0/578948390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f55700af8b0 con 0x7f55700c7ff0 2023-11-06T21:00:00.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.798+0000 7f555e7fc700 1 -- 172.21.15.33:0/578948390 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f55700af8b0 con 0x7f55700c7ff0 2023-11-06T21:00:00.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.800+0000 7f55749ff700 1 -- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f555005f270 msgr2=0x7f5550061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:00.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.800+0000 7f55749ff700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f555005f270 0x7f5550061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f5570158a10 tx=0x7f5564009060).stop 2023-11-06T21:00:00.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.800+0000 7f55749ff700 1 -- 172.21.15.33:0/578948390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55700c7ff0 msgr2=0x7f5570150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:00.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.800+0000 7f55749ff700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55700c7ff0 0x7f5570150d30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f5560009130 tx=0x7f5560017000).stop 2023-11-06T21:00:00.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.800+0000 7f55749ff700 1 -- 172.21.15.33:0/578948390 shutdown_connections 2023-11-06T21:00:00.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.800+0000 7f55749ff700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f555005f270 0x7f5550061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:00.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.800+0000 7f55749ff700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5570154710 0x7f5570154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.800+0000 7f55749ff700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55700c7ff0 0x7f5570150d30 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.800+0000 7f55749ff700 1 --2- 172.21.15.33:0/578948390 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5570154330 0x7f557015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.800+0000 7f55749ff700 1 -- 172.21.15.33:0/578948390 >> 172.21.15.33:0/578948390 conn(0x7f55700cb910 msgr2=0x7f55700646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.801+0000 7f55749ff700 1 -- 172.21.15.33:0/578948390 shutdown_connections 2023-11-06T21:00:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:00.801+0000 7f55749ff700 1 -- 172.21.15.33:0/578948390 wait complete. 2023-11-06T21:00:00.819 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T21:00:00.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:00:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.025+0000 7f47c9bff700 1 Processor -- start 2023-11-06T21:00:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.025+0000 7f47c9bff700 1 -- start start 2023-11-06T21:00:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c9bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c41580e0 0x7f47c41584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c9bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f47c40bdbe0 0x7f47c40684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c9bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47c40a7910 0x7f47c40a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c9bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c40be430 con 0x7f47c40a7910 2023-11-06T21:00:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c9bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c4064010 con 0x7f47c40bdbe0 2023-11-06T21:00:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c9bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40a82b0 con 0x7f47c41580e0 2023-11-06T21:00:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47c40a7910 0x7f47c40a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47c40a7910 0x7f47c40a7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59792/0 (socket says 172.21.15.33:59792) 2023-11-06T21:00:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c3fff700 1 -- 172.21.15.33:0/1976511291 learned_addr learned my addr 172.21.15.33:0/1976511291 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c2ffd700 1 --2- 172.21.15.33:0/1976511291 >> v2:172.21.15.69:3300/0 conn(0x7f47c40bdbe0 0x7f47c40684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c37fe700 1 --2- 172.21.15.33:0/1976511291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c41580e0 0x7f47c41584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c3fff700 1 -- 172.21.15.33:0/1976511291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c41580e0 msgr2=0x7f47c41584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:01.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c3fff700 1 --2- 172.21.15.33:0/1976511291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c41580e0 0x7f47c41584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c3fff700 1 -- 172.21.15.33:0/1976511291 >> v2:172.21.15.69:3300/0 conn(0x7f47c40bdbe0 msgr2=0x7f47c40684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:01.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c3fff700 1 --2- 172.21.15.33:0/1976511291 >> v2:172.21.15.69:3300/0 conn(0x7f47c40bdbe0 0x7f47c40684c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c3fff700 1 -- 172.21.15.33:0/1976511291 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40689b0 con 0x7f47c40a7910 2023-11-06T21:00:01.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c37fe700 1 --2- 172.21.15.33:0/1976511291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c41580e0 0x7f47c41584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:01.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.027+0000 7f47c2ffd700 1 --2- 172.21.15.33:0/1976511291 >> v2:172.21.15.69:3300/0 conn(0x7f47c40bdbe0 0x7f47c40684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:01.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c3fff700 1 --2- 172.21.15.33:0/1976511291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40a7910 0x7f47c40a7dc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f47c405f0f0 tx=0x7f47bc009840).ready entity=mon.0 client_cookie=c498e63812048e56 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:01.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c27fc700 1 -- 172.21.15.33:0/1976511291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00e070 con 0x7f47c40a7910 2023-11-06T21:00:01.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c27fc700 1 -- 172.21.15.33:0/1976511291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc00ba50 con 0x7f47c40a7910 2023-11-06T21:00:01.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c27fc700 1 -- 172.21.15.33:0/1976511291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00bdd0 con 0x7f47c40a7910 2023-11-06T21:00:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c9bff700 1 -- 172.21.15.33:0/1976511291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40a7910 msgr2=0x7f47c40a7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c9bff700 1 --2- 172.21.15.33:0/1976511291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40a7910 0x7f47c40a7dc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f47c405f0f0 tx=0x7f47bc009840).stop 2023-11-06T21:00:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c9bff700 1 -- 172.21.15.33:0/1976511291 shutdown_connections 2023-11-06T21:00:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c9bff700 1 --2- 172.21.15.33:0/1976511291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40a7910 0x7f47c40a7dc0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c9bff700 1 --2- 172.21.15.33:0/1976511291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c41580e0 0x7f47c41584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c9bff700 1 --2- 172.21.15.33:0/1976511291 >> v2:172.21.15.69:3300/0 conn(0x7f47c40bdbe0 0x7f47c40684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c9bff700 1 -- 172.21.15.33:0/1976511291 >> 172.21.15.33:0/1976511291 conn(0x7f47c405ec80 msgr2=0x7f47c40664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:01.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.028+0000 7f47c9bff700 1 -- 172.21.15.33:0/1976511291 shutdown_connections 2023-11-06T21:00:01.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.029+0000 7f47c9bff700 1 -- 172.21.15.33:0/1976511291 wait complete. 2023-11-06T21:00:01.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.030+0000 7f47c9bff700 1 Processor -- start 2023-11-06T21:00:01.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.030+0000 7f47c9bff700 1 -- start start 2023-11-06T21:00:01.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c9bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47c40a7910 0x7f47c4152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c9bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40bdbe0 0x7f47c40c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c9bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c41580e0 0x7f47c414e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c9bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4153db0 con 0x7f47c41580e0 2023-11-06T21:00:01.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c9bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40c4180 con 0x7f47c40a7910 2023-11-06T21:00:01.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c9bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40c4450 con 0x7f47c40bdbe0 2023-11-06T21:00:01.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c41580e0 0x7f47c414e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c41580e0 0x7f47c414e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59806/0 (socket says 172.21.15.33:59806) 2023-11-06T21:00:01.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c3fff700 1 -- 172.21.15.33:0/1221786819 learned_addr learned my addr 172.21.15.33:0/1221786819 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:01.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c2ffd700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40bdbe0 0x7f47c40c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.031+0000 7f47c37fe700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47c40a7910 0x7f47c4152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47c3fff700 1 -- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40bdbe0 msgr2=0x7f47c40c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47c3fff700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40bdbe0 0x7f47c40c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47c3fff700 1 -- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47c40a7910 msgr2=0x7f47c4152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47c3fff700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47c40a7910 0x7f47c4152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47c3fff700 1 -- 172.21.15.33:0/1221786819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc009ae0 con 0x7f47c41580e0 2023-11-06T21:00:01.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47c3fff700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c41580e0 0x7f47c414e7b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f47bc009fd0 tx=0x7f47bc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:01.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47ab7fe700 1 -- 172.21.15.33:0/1221786819 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00bdd0 con 0x7f47c41580e0 2023-11-06T21:00:01.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47ab7fe700 1 -- 172.21.15.33:0/1221786819 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc00b6c0 con 0x7f47c41580e0 2023-11-06T21:00:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47c9bff700 1 -- 172.21.15.33:0/1221786819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c40c4730 con 0x7f47c41580e0 2023-11-06T21:00:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47ab7fe700 1 -- 172.21.15.33:0/1221786819 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc01d6b0 con 0x7f47c41580e0 2023-11-06T21:00:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.032+0000 7f47c9bff700 1 -- 172.21.15.33:0/1221786819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c40c4cf0 con 0x7f47c41580e0 2023-11-06T21:00:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.034+0000 7f47c9bff700 1 -- 172.21.15.33:0/1221786819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c4153ef0 con 0x7f47c41580e0 2023-11-06T21:00:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.034+0000 7f47ab7fe700 1 -- 172.21.15.33:0/1221786819 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f47bc003040 con 0x7f47c41580e0 2023-11-06T21:00:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.035+0000 7f47ab7fe700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47b005f270 0x7f47b0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.035+0000 7f47ab7fe700 1 -- 172.21.15.33:0/1221786819 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f47bc07b7a0 con 0x7f47c41580e0 2023-11-06T21:00:01.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.036+0000 7f47c37fe700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47b005f270 0x7f47b0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.036+0000 7f47c37fe700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47b005f270 0x7f47b0061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f47b400ab40 tx=0x7f47b4008040).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:01.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.039+0000 7f47ab7fe700 1 -- 172.21.15.33:0/1221786819 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f47bc04c8f0 con 0x7f47c41580e0 2023-11-06T21:00:01.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.262+0000 7f47c9bff700 1 -- 172.21.15.33:0/1221786819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47c40664e0 con 0x7f47c41580e0 2023-11-06T21:00:01.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.263+0000 7f47ab7fe700 1 -- 172.21.15.33:0/1221786819 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f47bc022020 con 0x7f47c41580e0 2023-11-06T21:00:01.263 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:01.263 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1797026087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1797026087},{"type":"v1","addr":"172.21.15.69:6836","nonce":1797026087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4199570083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4199570083},{"type":"v1","addr":"172.21.15.33:6834","nonce":4199570083}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2429309034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2429309034},{"type":"v1","addr":"172.21.15.33:6835","nonce":2429309034}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:57.774874+0000","modified":"2023-11-06T21:00:00.791809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6621":{"gid":6621,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/1260218648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1260218648},{"type":"v1","addr":"172.21.15.69:6837","nonce":1260218648}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-06T21:00:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 -- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47b005f270 msgr2=0x7f47b0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47b005f270 0x7f47b0061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f47b400ab40 tx=0x7f47b4008040).stop 2023-11-06T21:00:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 -- 172.21.15.33:0/1221786819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c41580e0 msgr2=0x7f47c414e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c41580e0 0x7f47c414e7b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f47bc009fd0 tx=0x7f47bc009840).stop 2023-11-06T21:00:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 -- 172.21.15.33:0/1221786819 shutdown_connections 2023-11-06T21:00:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47b005f270 0x7f47b0061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c41580e0 0x7f47c414e7b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47c40a7910 0x7f47c4152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 --2- 172.21.15.33:0/1221786819 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40bdbe0 0x7f47c40c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 -- 172.21.15.33:0/1221786819 >> 172.21.15.33:0/1221786819 conn(0x7f47c405ec80 msgr2=0x7f47c4068290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:01.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.265+0000 7f47c9bff700 1 -- 172.21.15.33:0/1221786819 shutdown_connections 2023-11-06T21:00:01.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.266+0000 7f47c9bff700 1 -- 172.21.15.33:0/1221786819 wait complete. 2023-11-06T21:00:01.268 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 37 2023-11-06T21:00:01.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:00:01.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.479+0000 7f3956bff700 1 Processor -- start 2023-11-06T21:00:01.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.479+0000 7f3956bff700 1 -- start start 2023-11-06T21:00:01.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.480+0000 7f3956bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3950154110 0x7f39501544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.480+0000 7f3956bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3950064ce0 0x7f39500c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f3956bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39500c0880 con 0x7f39500aade0 2023-11-06T21:00:01.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f3956bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39500bfd40 con 0x7f3950064ce0 2023-11-06T21:00:01.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f3956bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39500c8260 con 0x7f3950154110 2023-11-06T21:00:01.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f394ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f39500aade0 0x7f39500645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38856/0 (socket says 172.21.15.33:38856) 2023-11-06T21:00:01.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f394ffff700 1 -- 172.21.15.33:0/4044683167 learned_addr learned my addr 172.21.15.33:0/4044683167 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:01.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f395519c700 1 --2- 172.21.15.33:0/4044683167 >> v2:172.21.15.33:3300/0 conn(0x7f3950064ce0 0x7f39500c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f395499b700 1 --2- 172.21.15.33:0/4044683167 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3950154110 0x7f39501544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f394f7fe700 1 -- 172.21.15.33:0/4044683167 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737059756 0 0) 0x7f39500c0880 con 0x7f39500aade0 2023-11-06T21:00:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f394f7fe700 1 -- 172.21.15.33:0/4044683167 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39400033f0 con 0x7f39500aade0 2023-11-06T21:00:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f395519c700 1 -- 172.21.15.33:0/4044683167 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3950154110 msgr2=0x7f39501544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f395519c700 1 --2- 172.21.15.33:0/4044683167 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3950154110 0x7f39501544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f395519c700 1 -- 172.21.15.33:0/4044683167 >> v1:172.21.15.33:6789/0 conn(0x7f39500aade0 legacy=0x7f39500645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.481+0000 7f395519c700 1 -- 172.21.15.33:0/4044683167 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500bff50 con 0x7f3950064ce0 2023-11-06T21:00:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.482+0000 7f395519c700 1 --2- 172.21.15.33:0/4044683167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3950064ce0 0x7f39500c5db0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f3948009e60 tx=0x7f3948009780).ready entity=mon.0 client_cookie=1ddb3b5250babda0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.482+0000 7f394f7fe700 1 -- 172.21.15.33:0/4044683167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800e070 con 0x7f3950064ce0 2023-11-06T21:00:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.482+0000 7f394f7fe700 1 -- 172.21.15.33:0/4044683167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394800bab0 con 0x7f3950064ce0 2023-11-06T21:00:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.482+0000 7f394f7fe700 1 -- 172.21.15.33:0/4044683167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800be30 con 0x7f3950064ce0 2023-11-06T21:00:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.482+0000 7f3956bff700 1 -- 172.21.15.33:0/4044683167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3950064ce0 msgr2=0x7f39500c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.482+0000 7f3956bff700 1 --2- 172.21.15.33:0/4044683167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3950064ce0 0x7f39500c5db0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f3948009e60 tx=0x7f3948009780).stop 2023-11-06T21:00:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.482+0000 7f3956bff700 1 -- 172.21.15.33:0/4044683167 shutdown_connections 2023-11-06T21:00:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.482+0000 7f3956bff700 1 --2- 172.21.15.33:0/4044683167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3950064ce0 0x7f39500c5db0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.482+0000 7f3956bff700 1 --2- 172.21.15.33:0/4044683167 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3950154110 0x7f39501544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.482+0000 7f3956bff700 1 -- 172.21.15.33:0/4044683167 >> 172.21.15.33:0/4044683167 conn(0x7f39500cb8b0 msgr2=0x7f39500c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.483+0000 7f3956bff700 1 -- 172.21.15.33:0/4044683167 shutdown_connections 2023-11-06T21:00:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.483+0000 7f3956bff700 1 -- 172.21.15.33:0/4044683167 wait complete. 2023-11-06T21:00:01.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.484+0000 7f3956bff700 1 Processor -- start 2023-11-06T21:00:01.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.484+0000 7f3956bff700 1 -- start start 2023-11-06T21:00:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.485+0000 7f3956bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3950064ce0 0x7f395014fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.485+0000 7f3956bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39500aade0 0x7f39500b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.485+0000 7f3956bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3950154110 0x7f39500b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.485+0000 7f3956bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500af520 con 0x7f39500aade0 2023-11-06T21:00:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.485+0000 7f3956bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500b0a50 con 0x7f3950154110 2023-11-06T21:00:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.485+0000 7f3956bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39500c2160 con 0x7f3950064ce0 2023-11-06T21:00:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.485+0000 7f394ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39500aade0 0x7f39500b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.485+0000 7f394ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39500aade0 0x7f39500b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59828/0 (socket says 172.21.15.33:59828) 2023-11-06T21:00:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.485+0000 7f394ffff700 1 -- 172.21.15.33:0/619817751 learned_addr learned my addr 172.21.15.33:0/619817751 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f395519c700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3950154110 0x7f39500b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f395499b700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3950064ce0 0x7f395014fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f394ffff700 1 -- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3950064ce0 msgr2=0x7f395014fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f394ffff700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3950064ce0 0x7f395014fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f394ffff700 1 -- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3950154110 msgr2=0x7f39500b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f394ffff700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3950154110 0x7f39500b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f394ffff700 1 -- 172.21.15.33:0/619817751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3948009a20 con 0x7f39500aade0 2023-11-06T21:00:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f394ffff700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39500aade0 0x7f39500b0070 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f393c0010e0 tx=0x7f393c007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f394cff9700 1 -- 172.21.15.33:0/619817751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c0070d0 con 0x7f39500aade0 2023-11-06T21:00:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f394cff9700 1 -- 172.21.15.33:0/619817751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c007230 con 0x7f39500aade0 2023-11-06T21:00:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f3956bff700 1 -- 172.21.15.33:0/619817751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39500aec80 con 0x7f39500aade0 2023-11-06T21:00:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.486+0000 7f394cff9700 1 -- 172.21.15.33:0/619817751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c0075d0 con 0x7f39500aade0 2023-11-06T21:00:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.487+0000 7f3956bff700 1 -- 172.21.15.33:0/619817751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39500af240 con 0x7f39500aade0 2023-11-06T21:00:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.488+0000 7f3956bff700 1 -- 172.21.15.33:0/619817751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39500c1f30 con 0x7f39500aade0 2023-11-06T21:00:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.488+0000 7f394cff9700 1 -- 172.21.15.33:0/619817751 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f393c0059b0 con 0x7f39500aade0 2023-11-06T21:00:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.490+0000 7f394cff9700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f393405f270 0x7f3934061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.490+0000 7f394cff9700 1 -- 172.21.15.33:0/619817751 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f393c079b20 con 0x7f39500aade0 2023-11-06T21:00:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.490+0000 7f395499b700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f393405f270 0x7f3934061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.491+0000 7f395499b700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f393405f270 0x7f3934061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3950167430 tx=0x7f3944009060).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.493+0000 7f394cff9700 1 -- 172.21.15.33:0/619817751 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f39500c1f30 con 0x7f39500aade0 2023-11-06T21:00:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.675+0000 7f3956bff700 1 -- 172.21.15.33:0/619817751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39500c1f30 con 0x7f39500aade0 2023-11-06T21:00:01.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.676+0000 7f394cff9700 1 -- 172.21.15.33:0/619817751 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16979 (secure 0 0 0) 0x7f39500c1f30 con 0x7f39500aade0 2023-11-06T21:00:01.677 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:01.677 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":99,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T20:59:58.777412+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:55.320441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:56.766396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:00:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.678+0000 7f3956bff700 1 -- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f393405f270 msgr2=0x7f3934061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.678+0000 7f3956bff700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f393405f270 0x7f3934061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3950167430 tx=0x7f3944009060).stop 2023-11-06T21:00:01.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.678+0000 7f3956bff700 1 -- 172.21.15.33:0/619817751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39500aade0 msgr2=0x7f39500b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.678+0000 7f3956bff700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39500aade0 0x7f39500b0070 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f393c0010e0 tx=0x7f393c007c30).stop 2023-11-06T21:00:01.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.678+0000 7f3956bff700 1 -- 172.21.15.33:0/619817751 shutdown_connections 2023-11-06T21:00:01.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.678+0000 7f3956bff700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f393405f270 0x7f3934061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.678+0000 7f3956bff700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3950064ce0 0x7f395014fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.678+0000 7f3956bff700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3950154110 0x7f39500b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.678+0000 7f3956bff700 1 --2- 172.21.15.33:0/619817751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39500aade0 0x7f39500b0070 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.678+0000 7f3956bff700 1 -- 172.21.15.33:0/619817751 >> 172.21.15.33:0/619817751 conn(0x7f39500cb8b0 msgr2=0x7f39500c8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:01.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.679+0000 7f3956bff700 1 -- 172.21.15.33:0/619817751 shutdown_connections 2023-11-06T21:00:01.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.679+0000 7f3956bff700 1 -- 172.21.15.33:0/619817751 wait complete. 2023-11-06T21:00:01.696 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T21:00:01.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T21:00:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.894+0000 7f173b3ff700 1 Processor -- start 2023-11-06T21:00:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.894+0000 7f173b3ff700 1 -- start start 2023-11-06T21:00:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.895+0000 7f173b3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17340c3600 0x7f17340c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.895+0000 7f173b3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1734069410 0x7f1734069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.895+0000 7f173b3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17340c17e0 con 0x7f1734136f50 2023-11-06T21:00:01.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.895+0000 7f173b3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173406a740 con 0x7f1734069410 2023-11-06T21:00:01.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.895+0000 7f173b3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173406a180 con 0x7f17340c3600 2023-11-06T21:00:01.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.895+0000 7f173899a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1734136f50 0x7f17341373d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38868/0 (socket says 172.21.15.33:38868) 2023-11-06T21:00:01.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.895+0000 7f173899a700 1 -- 172.21.15.33:0/9433860 learned_addr learned my addr 172.21.15.33:0/9433860 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:01.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f173919b700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17340c3600 0x7f17340c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f173999c700 1 --2- 172.21.15.33:0/9433860 >> v2:172.21.15.69:3300/0 conn(0x7f1734069410 0x7f1734069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643097118 0 0) 0x7f17340c17e0 con 0x7f1734136f50 2023-11-06T21:00:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17200033f0 con 0x7f1734136f50 2023-11-06T21:00:01.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 337600953 0 0) 0x7f17200033f0 con 0x7f1734136f50 2023-11-06T21:00:01.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17340c3600 msgr2=0x7f17340c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:01.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17340c3600 0x7f17340c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 >> v2:172.21.15.69:3300/0 conn(0x7f1734069410 msgr2=0x7f1734069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:01.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 --2- 172.21.15.33:0/9433860 >> v2:172.21.15.69:3300/0 conn(0x7f1734069410 0x7f1734069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173406a460 con 0x7f1734136f50 2023-11-06T21:00:01.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f1724001470 con 0x7f1734136f50 2023-11-06T21:00:01.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 >> v1:172.21.15.33:6789/0 conn(0x7f1734136f50 legacy=0x7f17341373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 0x7f1720003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1720004690 0x7f1720004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1720005000 0x7f172000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173406a740 con 0x7f17200038e0 2023-11-06T21:00:01.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173406a180 con 0x7f1720004690 2023-11-06T21:00:01.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172000bdd0 con 0x7f1720005000 2023-11-06T21:00:01.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.896+0000 7f173919b700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17340c3600 0x7f17340c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:01.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.897+0000 7f173919b700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 0x7f1720003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.897+0000 7f173899a700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1720004690 0x7f1720004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.897+0000 7f173999c700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1720005000 0x7f172000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.897+0000 7f173919b700 1 -- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1720005000 msgr2=0x7f172000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:01.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.897+0000 7f173919b700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1720005000 0x7f172000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.897+0000 7f173919b700 1 -- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1720004690 msgr2=0x7f1720004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:01.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.897+0000 7f173919b700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1720004690 0x7f1720004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.897+0000 7f173919b700 1 -- 172.21.15.33:0/9433860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17200033f0 con 0x7f17200038e0 2023-11-06T21:00:01.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.897+0000 7f173999c700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1720005000 0x7f172000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:01.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.897+0000 7f173919b700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 0x7f1720003d00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f17340c3020 tx=0x7f1730009060).ready entity=mon.0 client_cookie=414be32eb531c084 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:01.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.898+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17300094d0 con 0x7f17200038e0 2023-11-06T21:00:01.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.898+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173000e640 con 0x7f17200038e0 2023-11-06T21:00:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.898+0000 7f172bfff700 1 -- 172.21.15.33:0/9433860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000ea00 con 0x7f17200038e0 2023-11-06T21:00:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.898+0000 7f173b3ff700 1 -- 172.21.15.33:0/9433860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 msgr2=0x7f1720003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.898+0000 7f173b3ff700 1 --2- 172.21.15.33:0/9433860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 0x7f1720003d00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f17340c3020 tx=0x7f1730009060).stop 2023-11-06T21:00:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.898+0000 7f173999c700 1 -- 172.21.15.33:0/9433860 reap_dead start 2023-11-06T21:00:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.898+0000 7f173b3ff700 1 -- 172.21.15.33:0/9433860 shutdown_connections 2023-11-06T21:00:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.898+0000 7f173b3ff700 1 -- 172.21.15.33:0/9433860 >> 172.21.15.33:0/9433860 conn(0x7f1734062710 msgr2=0x7f1734062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.898+0000 7f173b3ff700 1 -- 172.21.15.33:0/9433860 shutdown_connections 2023-11-06T21:00:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.898+0000 7f173b3ff700 1 -- 172.21.15.33:0/9433860 wait complete. 2023-11-06T21:00:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.900+0000 7f173b3ff700 1 Processor -- start 2023-11-06T21:00:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.900+0000 7f173b3ff700 1 -- start start 2023-11-06T21:00:01.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.901+0000 7f173b3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 0x7f17340c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.901+0000 7f173b3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17340bded0 0x7f173414da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.901+0000 7f173b3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17340bffb0 0x7f17340c0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.901+0000 7f173b3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173415fe50 con 0x7f17200038e0 2023-11-06T21:00:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.901+0000 7f173b3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17340c0920 con 0x7f17340bded0 2023-11-06T21:00:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.901+0000 7f173b3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17340c6870 con 0x7f17340bffb0 2023-11-06T21:00:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.901+0000 7f173919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 0x7f17340c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.901+0000 7f173919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 0x7f17340c62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59838/0 (socket says 172.21.15.33:59838) 2023-11-06T21:00:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.901+0000 7f173919b700 1 -- 172.21.15.33:0/2821003664 learned_addr learned my addr 172.21.15.33:0/2821003664 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f173919b700 1 -- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17340bffb0 msgr2=0x7f17340c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f173919b700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17340bffb0 0x7f17340c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f173899a700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17340bded0 0x7f173414da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f173919b700 1 -- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17340bded0 msgr2=0x7f173414da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f173919b700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17340bded0 0x7f173414da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f173919b700 1 -- 172.21.15.33:0/2821003664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17340c6b50 con 0x7f17200038e0 2023-11-06T21:00:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f173899a700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17340bded0 0x7f173414da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f173919b700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 0x7f17340c62b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f1720005600 tx=0x7f1730019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f17297fa700 1 -- 172.21.15.33:0/2821003664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730009430 con 0x7f17200038e0 2023-11-06T21:00:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f17297fa700 1 -- 172.21.15.33:0/2821003664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173000ed70 con 0x7f17200038e0 2023-11-06T21:00:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f17297fa700 1 -- 172.21.15.33:0/2821003664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000eee0 con 0x7f17200038e0 2023-11-06T21:00:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f173b3ff700 1 -- 172.21.15.33:0/2821003664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17340c6e30 con 0x7f17200038e0 2023-11-06T21:00:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.902+0000 7f173b3ff700 1 -- 172.21.15.33:0/2821003664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173414df70 con 0x7f17200038e0 2023-11-06T21:00:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.903+0000 7f173b3ff700 1 -- 172.21.15.33:0/2821003664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17340c1470 con 0x7f17200038e0 2023-11-06T21:00:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.904+0000 7f17297fa700 1 -- 172.21.15.33:0/2821003664 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f173001c980 con 0x7f17200038e0 2023-11-06T21:00:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.905+0000 7f17297fa700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f171805f270 0x7f1718061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.905+0000 7f17297fa700 1 -- 172.21.15.33:0/2821003664 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1730000e40 con 0x7f17200038e0 2023-11-06T21:00:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.906+0000 7f173899a700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f171805f270 0x7f1718061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.906+0000 7f173899a700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f171805f270 0x7f1718061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f1724000e10 tx=0x7f1724009040).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:01.909+0000 7f17297fa700 1 -- 172.21.15.33:0/2821003664 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f173004d4e0 con 0x7f17200038e0 2023-11-06T21:00:02.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.089+0000 7f173b3ff700 1 -- 172.21.15.33:0/2821003664 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f173415ff90 con 0x7f171805f270 2023-11-06T21:00:02.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.103+0000 7f17297fa700 1 -- 172.21.15.33:0/2821003664 <== mgr.6280 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361224 (secure 0 0 0) 0x7f17240046b0 con 0x7f171805f270 2023-11-06T21:00:02.108 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.108+0000 7f173b3ff700 1 -- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f171805f270 msgr2=0x7f1718061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.108+0000 7f173b3ff700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f171805f270 0x7f1718061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f1724000e10 tx=0x7f1724009040).stop 2023-11-06T21:00:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.108+0000 7f173b3ff700 1 -- 172.21.15.33:0/2821003664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 msgr2=0x7f17340c62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.108+0000 7f173b3ff700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 0x7f17340c62b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f1720005600 tx=0x7f1730019000).stop 2023-11-06T21:00:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.108+0000 7f173b3ff700 1 -- 172.21.15.33:0/2821003664 shutdown_connections 2023-11-06T21:00:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.108+0000 7f173b3ff700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f171805f270 0x7f1718061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.108+0000 7f173b3ff700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17340bffb0 0x7f17340c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.108+0000 7f173b3ff700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17200038e0 0x7f17340c62b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.108+0000 7f173b3ff700 1 --2- 172.21.15.33:0/2821003664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17340bded0 0x7f173414da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.108+0000 7f173b3ff700 1 -- 172.21.15.33:0/2821003664 >> 172.21.15.33:0/2821003664 conn(0x7f1734062710 msgr2=0x7f17340bba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.109+0000 7f173b3ff700 1 -- 172.21.15.33:0/2821003664 shutdown_connections 2023-11-06T21:00:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.109+0000 7f173b3ff700 1 -- 172.21.15.33:0/2821003664 wait complete. 2023-11-06T21:00:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T21:00:02.125 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T21:00:01.232346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2418944,"kb_used_data":9408,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752555776,"statfs":{"total":773094113280,"available":770617114624,"internally_reserved":0,"allocated":9633792,"data_stored":4807580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11235,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-64,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013750"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.807797+0000","last_change":"2023-11-06T20:59:57.807797+0000","last_active":"2023-11-06T20:59:57.807797+0000","last_peered":"2023-11-06T20:59:57.807797+0000","last_clean":"2023-11-06T20:59:57.807797+0000","last_became_active":"2023-11-06T20:59:57.807529+0000","last_became_peered":"2023-11-06T20:59:57.807529+0000","last_unstale":"2023-11-06T20:59:57.807797+0000","last_undegraded":"2023-11-06T20:59:57.807797+0000","last_fullsized":"2023-11-06T20:59:57.807797+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3f","version":"98'3","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.816456+0000","last_change":"2023-11-06T20:59:57.306863+0000","last_active":"2023-11-06T20:59:57.816456+0000","last_peered":"2023-11-06T20:59:57.816456+0000","last_clean":"2023-11-06T20:59:57.816456+0000","last_became_active":"2023-11-06T20:59:57.306589+0000","last_became_peered":"2023-11-06T20:59:57.306589+0000","last_unstale":"2023-11-06T20:59:57.816456+0000","last_undegraded":"2023-11-06T20:59:57.816456+0000","last_fullsized":"2023-11-06T20:59:57.816456+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.816368+0000","last_change":"2023-11-06T20:59:57.814029+0000","last_active":"2023-11-06T20:59:57.816368+0000","last_peered":"2023-11-06T20:59:57.816368+0000","last_clean":"2023-11-06T20:59:57.816368+0000","last_became_active":"2023-11-06T20:59:57.813769+0000","last_became_peered":"2023-11-06T20:59:57.813769+0000","last_unstale":"2023-11-06T20:59:57.816368+0000","last_undegraded":"2023-11-06T20:59:57.816368+0000","last_fullsized":"2023-11-06T20:59:57.816368+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.790650+0000","last_change":"2023-11-06T20:59:57.320386+0000","last_active":"2023-11-06T20:59:57.790650+0000","last_peered":"2023-11-06T20:59:57.790650+0000","last_clean":"2023-11-06T20:59:57.790650+0000","last_became_active":"2023-11-06T20:59:57.320050+0000","last_became_peered":"2023-11-06T20:59:57.320050+0000","last_unstale":"2023-11-06T20:59:57.790650+0000","last_undegraded":"2023-11-06T20:59:57.790650+0000","last_fullsized":"2023-11-06T20:59:57.790650+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.820366+0000","last_change":"2023-11-06T20:59:57.819182+0000","last_active":"2023-11-06T20:59:57.820366+0000","last_peered":"2023-11-06T20:59:57.820366+0000","last_clean":"2023-11-06T20:59:57.820366+0000","last_became_active":"2023-11-06T20:59:57.818866+0000","last_became_peered":"2023-11-06T20:59:57.818866+0000","last_unstale":"2023-11-06T20:59:57.820366+0000","last_undegraded":"2023-11-06T20:59:57.820366+0000","last_fullsized":"2023-11-06T20:59:57.820366+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.791490+0000","last_change":"2023-11-06T20:59:57.322470+0000","last_active":"2023-11-06T20:59:57.791490+0000","last_peered":"2023-11-06T20:59:57.791490+0000","last_clean":"2023-11-06T20:59:57.791490+0000","last_became_active":"2023-11-06T20:59:57.321828+0000","last_became_peered":"2023-11-06T20:59:57.321828+0000","last_unstale":"2023-11-06T20:59:57.791490+0000","last_undegraded":"2023-11-06T20:59:57.791490+0000","last_fullsized":"2023-11-06T20:59:57.791490+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.817395+0000","last_change":"2023-11-06T20:59:57.808655+0000","last_active":"2023-11-06T20:59:57.817395+0000","last_peered":"2023-11-06T20:59:57.817395+0000","last_clean":"2023-11-06T20:59:57.817395+0000","last_became_active":"2023-11-06T20:59:57.807192+0000","last_became_peered":"2023-11-06T20:59:57.807192+0000","last_unstale":"2023-11-06T20:59:57.817395+0000","last_undegraded":"2023-11-06T20:59:57.817395+0000","last_fullsized":"2023-11-06T20:59:57.817395+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819889+0000","last_change":"2023-11-06T20:59:57.322230+0000","last_active":"2023-11-06T20:59:57.819889+0000","last_peered":"2023-11-06T20:59:57.819889+0000","last_clean":"2023-11-06T20:59:57.819889+0000","last_became_active":"2023-11-06T20:59:57.321840+0000","last_became_peered":"2023-11-06T20:59:57.321840+0000","last_unstale":"2023-11-06T20:59:57.819889+0000","last_undegraded":"2023-11-06T20:59:57.819889+0000","last_fullsized":"2023-11-06T20:59:57.819889+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806946+0000","last_change":"2023-11-06T20:59:57.801348+0000","last_active":"2023-11-06T20:59:57.806946+0000","last_peered":"2023-11-06T20:59:57.806946+0000","last_clean":"2023-11-06T20:59:57.806946+0000","last_became_active":"2023-11-06T20:59:57.799253+0000","last_became_peered":"2023-11-06T20:59:57.799253+0000","last_unstale":"2023-11-06T20:59:57.806946+0000","last_undegraded":"2023-11-06T20:59:57.806946+0000","last_fullsized":"2023-11-06T20:59:57.806946+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.818466+0000","last_change":"2023-11-06T20:59:57.304170+0000","last_active":"2023-11-06T20:59:57.818466+0000","last_peered":"2023-11-06T20:59:57.818466+0000","last_clean":"2023-11-06T20:59:57.818466+0000","last_became_active":"2023-11-06T20:59:57.303898+0000","last_became_peered":"2023-11-06T20:59:57.303898+0000","last_unstale":"2023-11-06T20:59:57.818466+0000","last_undegraded":"2023-11-06T20:59:57.818466+0000","last_fullsized":"2023-11-06T20:59:57.818466+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805363+0000","last_change":"2023-11-06T20:59:57.805363+0000","last_active":"2023-11-06T20:59:57.805363+0000","last_peered":"2023-11-06T20:59:57.805363+0000","last_clean":"2023-11-06T20:59:57.805363+0000","last_became_active":"2023-11-06T20:59:57.805059+0000","last_became_peered":"2023-11-06T20:59:57.805059+0000","last_unstale":"2023-11-06T20:59:57.805363+0000","last_undegraded":"2023-11-06T20:59:57.805363+0000","last_fullsized":"2023-11-06T20:59:57.805363+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3a","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.817043+0000","last_change":"2023-11-06T20:59:57.313767+0000","last_active":"2023-11-06T20:59:57.817043+0000","last_peered":"2023-11-06T20:59:57.817043+0000","last_clean":"2023-11-06T20:59:57.817043+0000","last_became_active":"2023-11-06T20:59:57.313496+0000","last_became_peered":"2023-11-06T20:59:57.313496+0000","last_unstale":"2023-11-06T20:59:57.817043+0000","last_undegraded":"2023-11-06T20:59:57.817043+0000","last_fullsized":"2023-11-06T20:59:57.817043+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.815394+0000","last_change":"2023-11-06T20:59:57.815394+0000","last_active":"2023-11-06T20:59:57.815394+0000","last_peered":"2023-11-06T20:59:57.815394+0000","last_clean":"2023-11-06T20:59:57.815394+0000","last_became_active":"2023-11-06T20:59:57.814883+0000","last_became_peered":"2023-11-06T20:59:57.814883+0000","last_unstale":"2023-11-06T20:59:57.815394+0000","last_undegraded":"2023-11-06T20:59:57.815394+0000","last_fullsized":"2023-11-06T20:59:57.815394+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.320786+0000","last_change":"2023-11-06T20:59:57.315997+0000","last_active":"2023-11-06T20:59:57.320786+0000","last_peered":"2023-11-06T20:59:57.320786+0000","last_clean":"2023-11-06T20:59:57.320786+0000","last_became_active":"2023-11-06T20:59:57.315722+0000","last_became_peered":"2023-11-06T20:59:57.315722+0000","last_unstale":"2023-11-06T20:59:57.320786+0000","last_undegraded":"2023-11-06T20:59:57.320786+0000","last_fullsized":"2023-11-06T20:59:57.320786+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.795557+0000","last_change":"2023-11-06T20:59:57.795557+0000","last_active":"2023-11-06T20:59:57.795557+0000","last_peered":"2023-11-06T20:59:57.795557+0000","last_clean":"2023-11-06T20:59:57.795557+0000","last_became_active":"2023-11-06T20:59:57.795070+0000","last_became_peered":"2023-11-06T20:59:57.795070+0000","last_unstale":"2023-11-06T20:59:57.795557+0000","last_undegraded":"2023-11-06T20:59:57.795557+0000","last_fullsized":"2023-11-06T20:59:57.795557+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.309319+0000","last_change":"2023-11-06T20:59:57.303023+0000","last_active":"2023-11-06T20:59:57.309319+0000","last_peered":"2023-11-06T20:59:57.309319+0000","last_clean":"2023-11-06T20:59:57.309319+0000","last_became_active":"2023-11-06T20:59:57.302679+0000","last_became_peered":"2023-11-06T20:59:57.302679+0000","last_unstale":"2023-11-06T20:59:57.309319+0000","last_undegraded":"2023-11-06T20:59:57.309319+0000","last_fullsized":"2023-11-06T20:59:57.309319+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.812485+0000","last_change":"2023-11-06T20:59:57.806679+0000","last_active":"2023-11-06T20:59:57.812485+0000","last_peered":"2023-11-06T20:59:57.812485+0000","last_clean":"2023-11-06T20:59:57.812485+0000","last_became_active":"2023-11-06T20:59:57.805039+0000","last_became_peered":"2023-11-06T20:59:57.805039+0000","last_unstale":"2023-11-06T20:59:57.812485+0000","last_undegraded":"2023-11-06T20:59:57.812485+0000","last_fullsized":"2023-11-06T20:59:57.812485+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.796920+0000","last_change":"2023-11-06T20:59:57.317756+0000","last_active":"2023-11-06T20:59:57.796920+0000","last_peered":"2023-11-06T20:59:57.796920+0000","last_clean":"2023-11-06T20:59:57.796920+0000","last_became_active":"2023-11-06T20:59:57.315938+0000","last_became_peered":"2023-11-06T20:59:57.315938+0000","last_unstale":"2023-11-06T20:59:57.796920+0000","last_undegraded":"2023-11-06T20:59:57.796920+0000","last_fullsized":"2023-11-06T20:59:57.796920+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819737+0000","last_change":"2023-11-06T20:59:57.818428+0000","last_active":"2023-11-06T20:59:57.819737+0000","last_peered":"2023-11-06T20:59:57.819737+0000","last_clean":"2023-11-06T20:59:57.819737+0000","last_became_active":"2023-11-06T20:59:57.818199+0000","last_became_peered":"2023-11-06T20:59:57.818199+0000","last_unstale":"2023-11-06T20:59:57.819737+0000","last_undegraded":"2023-11-06T20:59:57.819737+0000","last_fullsized":"2023-11-06T20:59:57.819737+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.326166+0000","last_change":"2023-11-06T20:59:57.319047+0000","last_active":"2023-11-06T20:59:57.326166+0000","last_peered":"2023-11-06T20:59:57.326166+0000","last_clean":"2023-11-06T20:59:57.326166+0000","last_became_active":"2023-11-06T20:59:57.318681+0000","last_became_peered":"2023-11-06T20:59:57.318681+0000","last_unstale":"2023-11-06T20:59:57.326166+0000","last_undegraded":"2023-11-06T20:59:57.326166+0000","last_fullsized":"2023-11-06T20:59:57.326166+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.817951+0000","last_change":"2023-11-06T20:59:57.817951+0000","last_active":"2023-11-06T20:59:57.817951+0000","last_peered":"2023-11-06T20:59:57.817951+0000","last_clean":"2023-11-06T20:59:57.817951+0000","last_became_active":"2023-11-06T20:59:57.814597+0000","last_became_peered":"2023-11-06T20:59:57.814597+0000","last_unstale":"2023-11-06T20:59:57.817951+0000","last_undegraded":"2023-11-06T20:59:57.817951+0000","last_fullsized":"2023-11-06T20:59:57.817951+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.309497+0000","last_change":"2023-11-06T20:59:57.303890+0000","last_active":"2023-11-06T20:59:57.309497+0000","last_peered":"2023-11-06T20:59:57.309497+0000","last_clean":"2023-11-06T20:59:57.309497+0000","last_became_active":"2023-11-06T20:59:57.303617+0000","last_became_peered":"2023-11-06T20:59:57.303617+0000","last_unstale":"2023-11-06T20:59:57.309497+0000","last_undegraded":"2023-11-06T20:59:57.309497+0000","last_fullsized":"2023-11-06T20:59:57.309497+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.798874+0000","last_change":"2023-11-06T20:59:57.798874+0000","last_active":"2023-11-06T20:59:57.798874+0000","last_peered":"2023-11-06T20:59:57.798874+0000","last_clean":"2023-11-06T20:59:57.798874+0000","last_became_active":"2023-11-06T20:59:57.798094+0000","last_became_peered":"2023-11-06T20:59:57.798094+0000","last_unstale":"2023-11-06T20:59:57.798874+0000","last_undegraded":"2023-11-06T20:59:57.798874+0000","last_fullsized":"2023-11-06T20:59:57.798874+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.34","version":"98'4","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.820776+0000","last_change":"2023-11-06T20:59:57.308642+0000","last_active":"2023-11-06T20:59:57.820776+0000","last_peered":"2023-11-06T20:59:57.820776+0000","last_clean":"2023-11-06T20:59:57.820776+0000","last_became_active":"2023-11-06T20:59:57.308280+0000","last_became_peered":"2023-11-06T20:59:57.308280+0000","last_unstale":"2023-11-06T20:59:57.820776+0000","last_undegraded":"2023-11-06T20:59:57.820776+0000","last_fullsized":"2023-11-06T20:59:57.820776+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.795643+0000","last_change":"2023-11-06T20:59:57.791607+0000","last_active":"2023-11-06T20:59:57.795643+0000","last_peered":"2023-11-06T20:59:57.795643+0000","last_clean":"2023-11-06T20:59:57.795643+0000","last_became_active":"2023-11-06T20:59:57.791324+0000","last_became_peered":"2023-11-06T20:59:57.791324+0000","last_unstale":"2023-11-06T20:59:57.795643+0000","last_undegraded":"2023-11-06T20:59:57.795643+0000","last_fullsized":"2023-11-06T20:59:57.795643+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.33","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.817462+0000","last_change":"2023-11-06T20:59:57.320439+0000","last_active":"2023-11-06T20:59:57.817462+0000","last_peered":"2023-11-06T20:59:57.817462+0000","last_clean":"2023-11-06T20:59:57.817462+0000","last_became_active":"2023-11-06T20:59:57.320151+0000","last_became_peered":"2023-11-06T20:59:57.320151+0000","last_unstale":"2023-11-06T20:59:57.817462+0000","last_undegraded":"2023-11-06T20:59:57.817462+0000","last_fullsized":"2023-11-06T20:59:57.817462+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.821693+0000","last_change":"2023-11-06T20:59:57.821693+0000","last_active":"2023-11-06T20:59:57.821693+0000","last_peered":"2023-11-06T20:59:57.821693+0000","last_clean":"2023-11-06T20:59:57.821693+0000","last_became_active":"2023-11-06T20:59:57.821093+0000","last_became_peered":"2023-11-06T20:59:57.821093+0000","last_unstale":"2023-11-06T20:59:57.821693+0000","last_undegraded":"2023-11-06T20:59:57.821693+0000","last_fullsized":"2023-11-06T20:59:57.821693+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.319759+0000","last_change":"2023-11-06T20:59:57.315519+0000","last_active":"2023-11-06T20:59:57.319759+0000","last_peered":"2023-11-06T20:59:57.319759+0000","last_clean":"2023-11-06T20:59:57.319759+0000","last_became_active":"2023-11-06T20:59:57.314968+0000","last_became_peered":"2023-11-06T20:59:57.314968+0000","last_unstale":"2023-11-06T20:59:57.319759+0000","last_undegraded":"2023-11-06T20:59:57.319759+0000","last_fullsized":"2023-11-06T20:59:57.319759+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.804259+0000","last_change":"2023-11-06T20:59:57.804259+0000","last_active":"2023-11-06T20:59:57.804259+0000","last_peered":"2023-11-06T20:59:57.804259+0000","last_clean":"2023-11-06T20:59:57.804259+0000","last_became_active":"2023-11-06T20:59:57.803976+0000","last_became_peered":"2023-11-06T20:59:57.803976+0000","last_unstale":"2023-11-06T20:59:57.804259+0000","last_undegraded":"2023-11-06T20:59:57.804259+0000","last_fullsized":"2023-11-06T20:59:57.804259+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.319890+0000","last_change":"2023-11-06T20:59:57.314831+0000","last_active":"2023-11-06T20:59:57.319890+0000","last_peered":"2023-11-06T20:59:57.319890+0000","last_clean":"2023-11-06T20:59:57.319890+0000","last_became_active":"2023-11-06T20:59:57.314515+0000","last_became_peered":"2023-11-06T20:59:57.314515+0000","last_unstale":"2023-11-06T20:59:57.319890+0000","last_undegraded":"2023-11-06T20:59:57.319890+0000","last_fullsized":"2023-11-06T20:59:57.319890+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.807118+0000","last_change":"2023-11-06T20:59:57.801898+0000","last_active":"2023-11-06T20:59:57.807118+0000","last_peered":"2023-11-06T20:59:57.807118+0000","last_clean":"2023-11-06T20:59:57.807118+0000","last_became_active":"2023-11-06T20:59:57.801250+0000","last_became_peered":"2023-11-06T20:59:57.801250+0000","last_unstale":"2023-11-06T20:59:57.807118+0000","last_undegraded":"2023-11-06T20:59:57.807118+0000","last_fullsized":"2023-11-06T20:59:57.807118+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"98'4","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.816729+0000","last_change":"2023-11-06T20:59:57.298724+0000","last_active":"2023-11-06T20:59:57.816729+0000","last_peered":"2023-11-06T20:59:57.816729+0000","last_clean":"2023-11-06T20:59:57.816729+0000","last_became_active":"2023-11-06T20:59:57.298410+0000","last_became_peered":"2023-11-06T20:59:57.298410+0000","last_unstale":"2023-11-06T20:59:57.816729+0000","last_undegraded":"2023-11-06T20:59:57.816729+0000","last_fullsized":"2023-11-06T20:59:57.816729+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.800997+0000","last_change":"2023-11-06T20:59:57.800997+0000","last_active":"2023-11-06T20:59:57.800997+0000","last_peered":"2023-11-06T20:59:57.800997+0000","last_clean":"2023-11-06T20:59:57.800997+0000","last_became_active":"2023-11-06T20:59:57.800682+0000","last_became_peered":"2023-11-06T20:59:57.800682+0000","last_unstale":"2023-11-06T20:59:57.800997+0000","last_undegraded":"2023-11-06T20:59:57.800997+0000","last_fullsized":"2023-11-06T20:59:57.800997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785489+0000","last_change":"2023-11-06T20:59:57.310678+0000","last_active":"2023-11-06T20:59:57.785489+0000","last_peered":"2023-11-06T20:59:57.785489+0000","last_clean":"2023-11-06T20:59:57.785489+0000","last_became_active":"2023-11-06T20:59:57.310377+0000","last_became_peered":"2023-11-06T20:59:57.310377+0000","last_unstale":"2023-11-06T20:59:57.785489+0000","last_undegraded":"2023-11-06T20:59:57.785489+0000","last_fullsized":"2023-11-06T20:59:57.785489+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.791236+0000","last_change":"2023-11-06T20:59:57.791236+0000","last_active":"2023-11-06T20:59:57.791236+0000","last_peered":"2023-11-06T20:59:57.791236+0000","last_clean":"2023-11-06T20:59:57.791236+0000","last_became_active":"2023-11-06T20:59:57.790456+0000","last_became_peered":"2023-11-06T20:59:57.790456+0000","last_unstale":"2023-11-06T20:59:57.791236+0000","last_undegraded":"2023-11-06T20:59:57.791236+0000","last_fullsized":"2023-11-06T20:59:57.791236+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.784405+0000","last_change":"2023-11-06T20:59:57.308539+0000","last_active":"2023-11-06T20:59:57.784405+0000","last_peered":"2023-11-06T20:59:57.784405+0000","last_clean":"2023-11-06T20:59:57.784405+0000","last_became_active":"2023-11-06T20:59:57.308272+0000","last_became_peered":"2023-11-06T20:59:57.308272+0000","last_unstale":"2023-11-06T20:59:57.784405+0000","last_undegraded":"2023-11-06T20:59:57.784405+0000","last_fullsized":"2023-11-06T20:59:57.784405+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.803389+0000","last_change":"2023-11-06T20:59:57.803389+0000","last_active":"2023-11-06T20:59:57.803389+0000","last_peered":"2023-11-06T20:59:57.803389+0000","last_clean":"2023-11-06T20:59:57.803389+0000","last_became_active":"2023-11-06T20:59:57.803091+0000","last_became_peered":"2023-11-06T20:59:57.803091+0000","last_unstale":"2023-11-06T20:59:57.803389+0000","last_undegraded":"2023-11-06T20:59:57.803389+0000","last_fullsized":"2023-11-06T20:59:57.803389+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2d","version":"98'1","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.781589+0000","last_change":"2023-11-06T20:59:57.312619+0000","last_active":"2023-11-06T20:59:57.781589+0000","last_peered":"2023-11-06T20:59:57.781589+0000","last_clean":"2023-11-06T20:59:57.781589+0000","last_became_active":"2023-11-06T20:59:57.310604+0000","last_became_peered":"2023-11-06T20:59:57.310604+0000","last_unstale":"2023-11-06T20:59:57.781589+0000","last_undegraded":"2023-11-06T20:59:57.781589+0000","last_fullsized":"2023-11-06T20:59:57.781589+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.821937+0000","last_change":"2023-11-06T20:59:57.821243+0000","last_active":"2023-11-06T20:59:57.821937+0000","last_peered":"2023-11-06T20:59:57.821937+0000","last_clean":"2023-11-06T20:59:57.821937+0000","last_became_active":"2023-11-06T20:59:57.820958+0000","last_became_peered":"2023-11-06T20:59:57.820958+0000","last_unstale":"2023-11-06T20:59:57.821937+0000","last_undegraded":"2023-11-06T20:59:57.821937+0000","last_fullsized":"2023-11-06T20:59:57.821937+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785610+0000","last_change":"2023-11-06T20:59:57.311760+0000","last_active":"2023-11-06T20:59:57.785610+0000","last_peered":"2023-11-06T20:59:57.785610+0000","last_clean":"2023-11-06T20:59:57.785610+0000","last_became_active":"2023-11-06T20:59:57.311495+0000","last_became_peered":"2023-11-06T20:59:57.311495+0000","last_unstale":"2023-11-06T20:59:57.785610+0000","last_undegraded":"2023-11-06T20:59:57.785610+0000","last_fullsized":"2023-11-06T20:59:57.785610+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806345+0000","last_change":"2023-11-06T20:59:57.795623+0000","last_active":"2023-11-06T20:59:57.806345+0000","last_peered":"2023-11-06T20:59:57.806345+0000","last_clean":"2023-11-06T20:59:57.806345+0000","last_became_active":"2023-11-06T20:59:57.794889+0000","last_became_peered":"2023-11-06T20:59:57.794889+0000","last_unstale":"2023-11-06T20:59:57.806345+0000","last_undegraded":"2023-11-06T20:59:57.806345+0000","last_fullsized":"2023-11-06T20:59:57.806345+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819739+0000","last_change":"2023-11-06T20:59:57.315291+0000","last_active":"2023-11-06T20:59:57.819739+0000","last_peered":"2023-11-06T20:59:57.819739+0000","last_clean":"2023-11-06T20:59:57.819739+0000","last_became_active":"2023-11-06T20:59:57.315002+0000","last_became_peered":"2023-11-06T20:59:57.315002+0000","last_unstale":"2023-11-06T20:59:57.819739+0000","last_undegraded":"2023-11-06T20:59:57.819739+0000","last_fullsized":"2023-11-06T20:59:57.819739+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.810834+0000","last_change":"2023-11-06T20:59:57.810834+0000","last_active":"2023-11-06T20:59:57.810834+0000","last_peered":"2023-11-06T20:59:57.810834+0000","last_clean":"2023-11-06T20:59:57.810834+0000","last_became_active":"2023-11-06T20:59:57.810553+0000","last_became_peered":"2023-11-06T20:59:57.810553+0000","last_unstale":"2023-11-06T20:59:57.810834+0000","last_undegraded":"2023-11-06T20:59:57.810834+0000","last_fullsized":"2023-11-06T20:59:57.810834+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785206+0000","last_change":"2023-11-06T20:59:57.317922+0000","last_active":"2023-11-06T20:59:57.785206+0000","last_peered":"2023-11-06T20:59:57.785206+0000","last_clean":"2023-11-06T20:59:57.785206+0000","last_became_active":"2023-11-06T20:59:57.317651+0000","last_became_peered":"2023-11-06T20:59:57.317651+0000","last_unstale":"2023-11-06T20:59:57.785206+0000","last_undegraded":"2023-11-06T20:59:57.785206+0000","last_fullsized":"2023-11-06T20:59:57.785206+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.797407+0000","last_change":"2023-11-06T20:59:57.797407+0000","last_active":"2023-11-06T20:59:57.797407+0000","last_peered":"2023-11-06T20:59:57.797407+0000","last_clean":"2023-11-06T20:59:57.797407+0000","last_became_active":"2023-11-06T20:59:57.796981+0000","last_became_peered":"2023-11-06T20:59:57.796981+0000","last_unstale":"2023-11-06T20:59:57.797407+0000","last_undegraded":"2023-11-06T20:59:57.797407+0000","last_fullsized":"2023-11-06T20:59:57.797407+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.320745+0000","last_change":"2023-11-06T20:59:57.313179+0000","last_active":"2023-11-06T20:59:57.320745+0000","last_peered":"2023-11-06T20:59:57.320745+0000","last_clean":"2023-11-06T20:59:57.320745+0000","last_became_active":"2023-11-06T20:59:57.312900+0000","last_became_peered":"2023-11-06T20:59:57.312900+0000","last_unstale":"2023-11-06T20:59:57.320745+0000","last_undegraded":"2023-11-06T20:59:57.320745+0000","last_fullsized":"2023-11-06T20:59:57.320745+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806033+0000","last_change":"2023-11-06T20:59:57.806033+0000","last_active":"2023-11-06T20:59:57.806033+0000","last_peered":"2023-11-06T20:59:57.806033+0000","last_clean":"2023-11-06T20:59:57.806033+0000","last_became_active":"2023-11-06T20:59:57.804271+0000","last_became_peered":"2023-11-06T20:59:57.804271+0000","last_unstale":"2023-11-06T20:59:57.806033+0000","last_undegraded":"2023-11-06T20:59:57.806033+0000","last_fullsized":"2023-11-06T20:59:57.806033+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.317691+0000","last_change":"2023-11-06T20:59:57.312337+0000","last_active":"2023-11-06T20:59:57.317691+0000","last_peered":"2023-11-06T20:59:57.317691+0000","last_clean":"2023-11-06T20:59:57.317691+0000","last_became_active":"2023-11-06T20:59:57.312044+0000","last_became_peered":"2023-11-06T20:59:57.312044+0000","last_unstale":"2023-11-06T20:59:57.317691+0000","last_undegraded":"2023-11-06T20:59:57.317691+0000","last_fullsized":"2023-11-06T20:59:57.317691+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.802382+0000","last_change":"2023-11-06T20:59:57.795993+0000","last_active":"2023-11-06T20:59:57.802382+0000","last_peered":"2023-11-06T20:59:57.802382+0000","last_clean":"2023-11-06T20:59:57.802382+0000","last_became_active":"2023-11-06T20:59:57.795513+0000","last_became_peered":"2023-11-06T20:59:57.795513+0000","last_unstale":"2023-11-06T20:59:57.802382+0000","last_undegraded":"2023-11-06T20:59:57.802382+0000","last_fullsized":"2023-11-06T20:59:57.802382+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.326335+0000","last_change":"2023-11-06T20:59:57.321522+0000","last_active":"2023-11-06T20:59:57.326335+0000","last_peered":"2023-11-06T20:59:57.326335+0000","last_clean":"2023-11-06T20:59:57.326335+0000","last_became_active":"2023-11-06T20:59:57.321077+0000","last_became_peered":"2023-11-06T20:59:57.321077+0000","last_unstale":"2023-11-06T20:59:57.326335+0000","last_undegraded":"2023-11-06T20:59:57.326335+0000","last_fullsized":"2023-11-06T20:59:57.326335+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.801568+0000","last_change":"2023-11-06T20:59:57.801568+0000","last_active":"2023-11-06T20:59:57.801568+0000","last_peered":"2023-11-06T20:59:57.801568+0000","last_clean":"2023-11-06T20:59:57.801568+0000","last_became_active":"2023-11-06T20:59:57.801290+0000","last_became_peered":"2023-11-06T20:59:57.801290+0000","last_unstale":"2023-11-06T20:59:57.801568+0000","last_undegraded":"2023-11-06T20:59:57.801568+0000","last_fullsized":"2023-11-06T20:59:57.801568+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.318060+0000","last_change":"2023-11-06T20:59:57.313674+0000","last_active":"2023-11-06T20:59:57.318060+0000","last_peered":"2023-11-06T20:59:57.318060+0000","last_clean":"2023-11-06T20:59:57.318060+0000","last_became_active":"2023-11-06T20:59:57.313401+0000","last_became_peered":"2023-11-06T20:59:57.313401+0000","last_unstale":"2023-11-06T20:59:57.318060+0000","last_undegraded":"2023-11-06T20:59:57.318060+0000","last_fullsized":"2023-11-06T20:59:57.318060+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.799278+0000","last_change":"2023-11-06T20:59:57.799278+0000","last_active":"2023-11-06T20:59:57.799278+0000","last_peered":"2023-11-06T20:59:57.799278+0000","last_clean":"2023-11-06T20:59:57.799278+0000","last_became_active":"2023-11-06T20:59:57.797784+0000","last_became_peered":"2023-11-06T20:59:57.797784+0000","last_unstale":"2023-11-06T20:59:57.799278+0000","last_undegraded":"2023-11-06T20:59:57.799278+0000","last_fullsized":"2023-11-06T20:59:57.799278+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.783823+0000","last_change":"2023-11-06T20:59:57.312090+0000","last_active":"2023-11-06T20:59:57.783823+0000","last_peered":"2023-11-06T20:59:57.783823+0000","last_clean":"2023-11-06T20:59:57.783823+0000","last_became_active":"2023-11-06T20:59:57.311780+0000","last_became_peered":"2023-11-06T20:59:57.311780+0000","last_unstale":"2023-11-06T20:59:57.783823+0000","last_undegraded":"2023-11-06T20:59:57.783823+0000","last_fullsized":"2023-11-06T20:59:57.783823+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805610+0000","last_change":"2023-11-06T20:59:57.793640+0000","last_active":"2023-11-06T20:59:57.805610+0000","last_peered":"2023-11-06T20:59:57.805610+0000","last_clean":"2023-11-06T20:59:57.805610+0000","last_became_active":"2023-11-06T20:59:57.793179+0000","last_became_peered":"2023-11-06T20:59:57.793179+0000","last_unstale":"2023-11-06T20:59:57.805610+0000","last_undegraded":"2023-11-06T20:59:57.805610+0000","last_fullsized":"2023-11-06T20:59:57.805610+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.790365+0000","last_change":"2023-11-06T20:59:57.317021+0000","last_active":"2023-11-06T20:59:57.790365+0000","last_peered":"2023-11-06T20:59:57.790365+0000","last_clean":"2023-11-06T20:59:57.790365+0000","last_became_active":"2023-11-06T20:59:57.316296+0000","last_became_peered":"2023-11-06T20:59:57.316296+0000","last_unstale":"2023-11-06T20:59:57.790365+0000","last_undegraded":"2023-11-06T20:59:57.790365+0000","last_fullsized":"2023-11-06T20:59:57.790365+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.810402+0000","last_change":"2023-11-06T20:59:57.807802+0000","last_active":"2023-11-06T20:59:57.810402+0000","last_peered":"2023-11-06T20:59:57.810402+0000","last_clean":"2023-11-06T20:59:57.810402+0000","last_became_active":"2023-11-06T20:59:57.806949+0000","last_became_peered":"2023-11-06T20:59:57.806949+0000","last_unstale":"2023-11-06T20:59:57.810402+0000","last_undegraded":"2023-11-06T20:59:57.810402+0000","last_fullsized":"2023-11-06T20:59:57.810402+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.786352+0000","last_change":"2023-11-06T20:59:57.318055+0000","last_active":"2023-11-06T20:59:57.786352+0000","last_peered":"2023-11-06T20:59:57.786352+0000","last_clean":"2023-11-06T20:59:57.786352+0000","last_became_active":"2023-11-06T20:59:57.317452+0000","last_became_peered":"2023-11-06T20:59:57.317452+0000","last_unstale":"2023-11-06T20:59:57.786352+0000","last_undegraded":"2023-11-06T20:59:57.786352+0000","last_fullsized":"2023-11-06T20:59:57.786352+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.803144+0000","last_change":"2023-11-06T20:59:57.797464+0000","last_active":"2023-11-06T20:59:57.803144+0000","last_peered":"2023-11-06T20:59:57.803144+0000","last_clean":"2023-11-06T20:59:57.803144+0000","last_became_active":"2023-11-06T20:59:57.795198+0000","last_became_peered":"2023-11-06T20:59:57.795198+0000","last_unstale":"2023-11-06T20:59:57.803144+0000","last_undegraded":"2023-11-06T20:59:57.803144+0000","last_fullsized":"2023-11-06T20:59:57.803144+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.314735+0000","last_change":"2023-11-06T20:59:57.309710+0000","last_active":"2023-11-06T20:59:57.314735+0000","last_peered":"2023-11-06T20:59:57.314735+0000","last_clean":"2023-11-06T20:59:57.314735+0000","last_became_active":"2023-11-06T20:59:57.309538+0000","last_became_peered":"2023-11-06T20:59:57.309538+0000","last_unstale":"2023-11-06T20:59:57.314735+0000","last_undegraded":"2023-11-06T20:59:57.314735+0000","last_fullsized":"2023-11-06T20:59:57.314735+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.816352+0000","last_change":"2023-11-06T20:59:57.813446+0000","last_active":"2023-11-06T20:59:57.816352+0000","last_peered":"2023-11-06T20:59:57.816352+0000","last_clean":"2023-11-06T20:59:57.816352+0000","last_became_active":"2023-11-06T20:59:57.813080+0000","last_became_peered":"2023-11-06T20:59:57.813080+0000","last_unstale":"2023-11-06T20:59:57.816352+0000","last_undegraded":"2023-11-06T20:59:57.816352+0000","last_fullsized":"2023-11-06T20:59:57.816352+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.788633+0000","last_change":"2023-11-06T20:59:57.317280+0000","last_active":"2023-11-06T20:59:57.788633+0000","last_peered":"2023-11-06T20:59:57.788633+0000","last_clean":"2023-11-06T20:59:57.788633+0000","last_became_active":"2023-11-06T20:59:57.317021+0000","last_became_peered":"2023-11-06T20:59:57.317021+0000","last_unstale":"2023-11-06T20:59:57.788633+0000","last_undegraded":"2023-11-06T20:59:57.788633+0000","last_fullsized":"2023-11-06T20:59:57.788633+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805958+0000","last_change":"2023-11-06T20:59:57.799180+0000","last_active":"2023-11-06T20:59:57.805958+0000","last_peered":"2023-11-06T20:59:57.805958+0000","last_clean":"2023-11-06T20:59:57.805958+0000","last_became_active":"2023-11-06T20:59:57.798759+0000","last_became_peered":"2023-11-06T20:59:57.798759+0000","last_unstale":"2023-11-06T20:59:57.805958+0000","last_undegraded":"2023-11-06T20:59:57.805958+0000","last_fullsized":"2023-11-06T20:59:57.805958+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.788202+0000","last_change":"2023-11-06T20:59:57.305796+0000","last_active":"2023-11-06T20:59:57.788202+0000","last_peered":"2023-11-06T20:59:57.788202+0000","last_clean":"2023-11-06T20:59:57.788202+0000","last_became_active":"2023-11-06T20:59:57.304989+0000","last_became_peered":"2023-11-06T20:59:57.304989+0000","last_unstale":"2023-11-06T20:59:57.788202+0000","last_undegraded":"2023-11-06T20:59:57.788202+0000","last_fullsized":"2023-11-06T20:59:57.788202+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.809550+0000","last_change":"2023-11-06T20:59:57.805526+0000","last_active":"2023-11-06T20:59:57.809550+0000","last_peered":"2023-11-06T20:59:57.809550+0000","last_clean":"2023-11-06T20:59:57.809550+0000","last_became_active":"2023-11-06T20:59:57.805148+0000","last_became_peered":"2023-11-06T20:59:57.805148+0000","last_unstale":"2023-11-06T20:59:57.809550+0000","last_undegraded":"2023-11-06T20:59:57.809550+0000","last_fullsized":"2023-11-06T20:59:57.809550+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.811669+0000","last_change":"2023-11-06T20:59:57.319917+0000","last_active":"2023-11-06T20:59:57.811669+0000","last_peered":"2023-11-06T20:59:57.811669+0000","last_clean":"2023-11-06T20:59:57.811669+0000","last_became_active":"2023-11-06T20:59:57.318879+0000","last_became_peered":"2023-11-06T20:59:57.318879+0000","last_unstale":"2023-11-06T20:59:57.811669+0000","last_undegraded":"2023-11-06T20:59:57.811669+0000","last_fullsized":"2023-11-06T20:59:57.811669+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.811315+0000","last_change":"2023-11-06T20:59:57.798478+0000","last_active":"2023-11-06T20:59:57.811315+0000","last_peered":"2023-11-06T20:59:57.811315+0000","last_clean":"2023-11-06T20:59:57.811315+0000","last_became_active":"2023-11-06T20:59:57.798157+0000","last_became_peered":"2023-11-06T20:59:57.798157+0000","last_unstale":"2023-11-06T20:59:57.811315+0000","last_undegraded":"2023-11-06T20:59:57.811315+0000","last_fullsized":"2023-11-06T20:59:57.811315+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"98'1","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.781609+0000","last_change":"2023-11-06T20:59:57.298404+0000","last_active":"2023-11-06T20:59:57.781609+0000","last_peered":"2023-11-06T20:59:57.781609+0000","last_clean":"2023-11-06T20:59:57.781609+0000","last_became_active":"2023-11-06T20:59:57.298073+0000","last_became_peered":"2023-11-06T20:59:57.298073+0000","last_unstale":"2023-11-06T20:59:57.781609+0000","last_undegraded":"2023-11-06T20:59:57.781609+0000","last_fullsized":"2023-11-06T20:59:57.781609+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.804631+0000","last_change":"2023-11-06T20:59:57.797490+0000","last_active":"2023-11-06T20:59:57.804631+0000","last_peered":"2023-11-06T20:59:57.804631+0000","last_clean":"2023-11-06T20:59:57.804631+0000","last_became_active":"2023-11-06T20:59:57.797179+0000","last_became_peered":"2023-11-06T20:59:57.797179+0000","last_unstale":"2023-11-06T20:59:57.804631+0000","last_undegraded":"2023-11-06T20:59:57.804631+0000","last_fullsized":"2023-11-06T20:59:57.804631+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"98'2","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.820980+0000","last_change":"2023-11-06T20:59:57.320542+0000","last_active":"2023-11-06T20:59:57.820980+0000","last_peered":"2023-11-06T20:59:57.820980+0000","last_clean":"2023-11-06T20:59:57.820980+0000","last_became_active":"2023-11-06T20:59:57.320214+0000","last_became_peered":"2023-11-06T20:59:57.320214+0000","last_unstale":"2023-11-06T20:59:57.820980+0000","last_undegraded":"2023-11-06T20:59:57.820980+0000","last_fullsized":"2023-11-06T20:59:57.820980+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805039+0000","last_change":"2023-11-06T20:59:57.799950+0000","last_active":"2023-11-06T20:59:57.805039+0000","last_peered":"2023-11-06T20:59:57.805039+0000","last_clean":"2023-11-06T20:59:57.805039+0000","last_became_active":"2023-11-06T20:59:57.799648+0000","last_became_peered":"2023-11-06T20:59:57.799648+0000","last_unstale":"2023-11-06T20:59:57.805039+0000","last_undegraded":"2023-11-06T20:59:57.805039+0000","last_fullsized":"2023-11-06T20:59:57.805039+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785377+0000","last_change":"2023-11-06T20:59:57.312011+0000","last_active":"2023-11-06T20:59:57.785377+0000","last_peered":"2023-11-06T20:59:57.785377+0000","last_clean":"2023-11-06T20:59:57.785377+0000","last_became_active":"2023-11-06T20:59:57.311521+0000","last_became_peered":"2023-11-06T20:59:57.311521+0000","last_unstale":"2023-11-06T20:59:57.785377+0000","last_undegraded":"2023-11-06T20:59:57.785377+0000","last_fullsized":"2023-11-06T20:59:57.785377+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806366+0000","last_change":"2023-11-06T20:59:57.795865+0000","last_active":"2023-11-06T20:59:57.806366+0000","last_peered":"2023-11-06T20:59:57.806366+0000","last_clean":"2023-11-06T20:59:57.806366+0000","last_became_active":"2023-11-06T20:59:57.795242+0000","last_became_peered":"2023-11-06T20:59:57.795242+0000","last_unstale":"2023-11-06T20:59:57.806366+0000","last_undegraded":"2023-11-06T20:59:57.806366+0000","last_fullsized":"2023-11-06T20:59:57.806366+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785261+0000","last_change":"2023-11-06T20:59:57.309685+0000","last_active":"2023-11-06T20:59:57.785261+0000","last_peered":"2023-11-06T20:59:57.785261+0000","last_clean":"2023-11-06T20:59:57.785261+0000","last_became_active":"2023-11-06T20:59:57.308807+0000","last_became_peered":"2023-11-06T20:59:57.308807+0000","last_unstale":"2023-11-06T20:59:57.785261+0000","last_undegraded":"2023-11-06T20:59:57.785261+0000","last_fullsized":"2023-11-06T20:59:57.785261+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.798182+0000","last_change":"2023-11-06T20:59:57.798182+0000","last_active":"2023-11-06T20:59:57.798182+0000","last_peered":"2023-11-06T20:59:57.798182+0000","last_clean":"2023-11-06T20:59:57.798182+0000","last_became_active":"2023-11-06T20:59:57.797862+0000","last_became_peered":"2023-11-06T20:59:57.797862+0000","last_unstale":"2023-11-06T20:59:57.798182+0000","last_undegraded":"2023-11-06T20:59:57.798182+0000","last_fullsized":"2023-11-06T20:59:57.798182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.319323+0000","last_change":"2023-11-06T20:59:57.315832+0000","last_active":"2023-11-06T20:59:57.319323+0000","last_peered":"2023-11-06T20:59:57.319323+0000","last_clean":"2023-11-06T20:59:57.319323+0000","last_became_active":"2023-11-06T20:59:57.315524+0000","last_became_peered":"2023-11-06T20:59:57.315524+0000","last_unstale":"2023-11-06T20:59:57.319323+0000","last_undegraded":"2023-11-06T20:59:57.319323+0000","last_fullsized":"2023-11-06T20:59:57.319323+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805497+0000","last_change":"2023-11-06T20:59:57.801297+0000","last_active":"2023-11-06T20:59:57.805497+0000","last_peered":"2023-11-06T20:59:57.805497+0000","last_clean":"2023-11-06T20:59:57.805497+0000","last_became_active":"2023-11-06T20:59:57.800996+0000","last_became_peered":"2023-11-06T20:59:57.800996+0000","last_unstale":"2023-11-06T20:59:57.805497+0000","last_undegraded":"2023-11-06T20:59:57.805497+0000","last_fullsized":"2023-11-06T20:59:57.805497+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.314053+0000","last_change":"2023-11-06T20:59:57.303473+0000","last_active":"2023-11-06T20:59:57.314053+0000","last_peered":"2023-11-06T20:59:57.314053+0000","last_clean":"2023-11-06T20:59:57.314053+0000","last_became_active":"2023-11-06T20:59:57.303125+0000","last_became_peered":"2023-11-06T20:59:57.303125+0000","last_unstale":"2023-11-06T20:59:57.314053+0000","last_undegraded":"2023-11-06T20:59:57.314053+0000","last_fullsized":"2023-11-06T20:59:57.314053+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.800150+0000","last_change":"2023-11-06T20:59:57.800150+0000","last_active":"2023-11-06T20:59:57.800150+0000","last_peered":"2023-11-06T20:59:57.800150+0000","last_clean":"2023-11-06T20:59:57.800150+0000","last_became_active":"2023-11-06T20:59:57.799731+0000","last_became_peered":"2023-11-06T20:59:57.799731+0000","last_unstale":"2023-11-06T20:59:57.800150+0000","last_undegraded":"2023-11-06T20:59:57.800150+0000","last_fullsized":"2023-11-06T20:59:57.800150+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.324932+0000","last_change":"2023-11-06T20:59:57.323634+0000","last_active":"2023-11-06T20:59:57.324932+0000","last_peered":"2023-11-06T20:59:57.324932+0000","last_clean":"2023-11-06T20:59:57.324932+0000","last_became_active":"2023-11-06T20:59:57.323435+0000","last_became_peered":"2023-11-06T20:59:57.323435+0000","last_unstale":"2023-11-06T20:59:57.324932+0000","last_undegraded":"2023-11-06T20:59:57.324932+0000","last_fullsized":"2023-11-06T20:59:57.324932+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.797257+0000","last_change":"2023-11-06T20:59:57.797257+0000","last_active":"2023-11-06T20:59:57.797257+0000","last_peered":"2023-11-06T20:59:57.797257+0000","last_clean":"2023-11-06T20:59:57.797257+0000","last_became_active":"2023-11-06T20:59:57.794748+0000","last_became_peered":"2023-11-06T20:59:57.794748+0000","last_unstale":"2023-11-06T20:59:57.797257+0000","last_undegraded":"2023-11-06T20:59:57.797257+0000","last_fullsized":"2023-11-06T20:59:57.797257+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.791100+0000","last_change":"2023-11-06T20:59:57.320281+0000","last_active":"2023-11-06T20:59:57.791100+0000","last_peered":"2023-11-06T20:59:57.791100+0000","last_clean":"2023-11-06T20:59:57.791100+0000","last_became_active":"2023-11-06T20:59:57.319958+0000","last_became_peered":"2023-11-06T20:59:57.319958+0000","last_unstale":"2023-11-06T20:59:57.791100+0000","last_undegraded":"2023-11-06T20:59:57.791100+0000","last_fullsized":"2023-11-06T20:59:57.791100+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.800905+0000","last_change":"2023-11-06T20:59:57.791592+0000","last_active":"2023-11-06T20:59:57.800905+0000","last_peered":"2023-11-06T20:59:57.800905+0000","last_clean":"2023-11-06T20:59:57.800905+0000","last_became_active":"2023-11-06T20:59:57.791292+0000","last_became_peered":"2023-11-06T20:59:57.791292+0000","last_unstale":"2023-11-06T20:59:57.800905+0000","last_undegraded":"2023-11-06T20:59:57.800905+0000","last_fullsized":"2023-11-06T20:59:57.800905+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.319658+0000","last_change":"2023-11-06T20:59:57.314975+0000","last_active":"2023-11-06T20:59:57.319658+0000","last_peered":"2023-11-06T20:59:57.319658+0000","last_clean":"2023-11-06T20:59:57.319658+0000","last_became_active":"2023-11-06T20:59:57.311193+0000","last_became_peered":"2023-11-06T20:59:57.311193+0000","last_unstale":"2023-11-06T20:59:57.319658+0000","last_undegraded":"2023-11-06T20:59:57.319658+0000","last_fullsized":"2023-11-06T20:59:57.319658+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.802599+0000","last_change":"2023-11-06T20:59:57.802599+0000","last_active":"2023-11-06T20:59:57.802599+0000","last_peered":"2023-11-06T20:59:57.802599+0000","last_clean":"2023-11-06T20:59:57.802599+0000","last_became_active":"2023-11-06T20:59:57.802210+0000","last_became_peered":"2023-11-06T20:59:57.802210+0000","last_unstale":"2023-11-06T20:59:57.802599+0000","last_undegraded":"2023-11-06T20:59:57.802599+0000","last_fullsized":"2023-11-06T20:59:57.802599+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"98'3","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.813980+0000","last_change":"2023-11-06T20:59:57.311675+0000","last_active":"2023-11-06T20:59:57.813980+0000","last_peered":"2023-11-06T20:59:57.813980+0000","last_clean":"2023-11-06T20:59:57.813980+0000","last_became_active":"2023-11-06T20:59:57.311341+0000","last_became_peered":"2023-11-06T20:59:57.311341+0000","last_unstale":"2023-11-06T20:59:57.813980+0000","last_undegraded":"2023-11-06T20:59:57.813980+0000","last_fullsized":"2023-11-06T20:59:57.813980+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.820356+0000","last_change":"2023-11-06T20:59:57.819029+0000","last_active":"2023-11-06T20:59:57.820356+0000","last_peered":"2023-11-06T20:59:57.820356+0000","last_clean":"2023-11-06T20:59:57.820356+0000","last_became_active":"2023-11-06T20:59:57.818667+0000","last_became_peered":"2023-11-06T20:59:57.818667+0000","last_unstale":"2023-11-06T20:59:57.820356+0000","last_undegraded":"2023-11-06T20:59:57.820356+0000","last_fullsized":"2023-11-06T20:59:57.820356+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"98'2","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.816742+0000","last_change":"2023-11-06T20:59:57.314129+0000","last_active":"2023-11-06T20:59:57.816742+0000","last_peered":"2023-11-06T20:59:57.816742+0000","last_clean":"2023-11-06T20:59:57.816742+0000","last_became_active":"2023-11-06T20:59:57.313590+0000","last_became_peered":"2023-11-06T20:59:57.313590+0000","last_unstale":"2023-11-06T20:59:57.816742+0000","last_undegraded":"2023-11-06T20:59:57.816742+0000","last_fullsized":"2023-11-06T20:59:57.816742+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806972+0000","last_change":"2023-11-06T20:59:57.797473+0000","last_active":"2023-11-06T20:59:57.806972+0000","last_peered":"2023-11-06T20:59:57.806972+0000","last_clean":"2023-11-06T20:59:57.806972+0000","last_became_active":"2023-11-06T20:59:57.797206+0000","last_became_peered":"2023-11-06T20:59:57.797206+0000","last_unstale":"2023-11-06T20:59:57.806972+0000","last_undegraded":"2023-11-06T20:59:57.806972+0000","last_fullsized":"2023-11-06T20:59:57.806972+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"98'1","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.783641+0000","last_change":"2023-11-06T20:59:57.320406+0000","last_active":"2023-11-06T20:59:57.783641+0000","last_peered":"2023-11-06T20:59:57.783641+0000","last_clean":"2023-11-06T20:59:57.783641+0000","last_became_active":"2023-11-06T20:59:57.319539+0000","last_became_peered":"2023-11-06T20:59:57.319539+0000","last_unstale":"2023-11-06T20:59:57.783641+0000","last_undegraded":"2023-11-06T20:59:57.783641+0000","last_fullsized":"2023-11-06T20:59:57.783641+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.798445+0000","last_change":"2023-11-06T20:59:57.798445+0000","last_active":"2023-11-06T20:59:57.798445+0000","last_peered":"2023-11-06T20:59:57.798445+0000","last_clean":"2023-11-06T20:59:57.798445+0000","last_became_active":"2023-11-06T20:59:57.797706+0000","last_became_peered":"2023-11-06T20:59:57.797706+0000","last_unstale":"2023-11-06T20:59:57.798445+0000","last_undegraded":"2023-11-06T20:59:57.798445+0000","last_fullsized":"2023-11-06T20:59:57.798445+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785085+0000","last_change":"2023-11-06T20:59:57.317838+0000","last_active":"2023-11-06T20:59:57.785085+0000","last_peered":"2023-11-06T20:59:57.785085+0000","last_clean":"2023-11-06T20:59:57.785085+0000","last_became_active":"2023-11-06T20:59:57.316851+0000","last_became_peered":"2023-11-06T20:59:57.316851+0000","last_unstale":"2023-11-06T20:59:57.785085+0000","last_undegraded":"2023-11-06T20:59:57.785085+0000","last_fullsized":"2023-11-06T20:59:57.785085+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.803856+0000","last_change":"2023-11-06T20:59:57.795416+0000","last_active":"2023-11-06T20:59:57.803856+0000","last_peered":"2023-11-06T20:59:57.803856+0000","last_clean":"2023-11-06T20:59:57.803856+0000","last_became_active":"2023-11-06T20:59:57.795066+0000","last_became_peered":"2023-11-06T20:59:57.795066+0000","last_unstale":"2023-11-06T20:59:57.803856+0000","last_undegraded":"2023-11-06T20:59:57.803856+0000","last_fullsized":"2023-11-06T20:59:57.803856+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.787611+0000","last_change":"2023-11-06T20:59:57.319453+0000","last_active":"2023-11-06T20:59:57.787611+0000","last_peered":"2023-11-06T20:59:57.787611+0000","last_clean":"2023-11-06T20:59:57.787611+0000","last_became_active":"2023-11-06T20:59:57.318820+0000","last_became_peered":"2023-11-06T20:59:57.318820+0000","last_unstale":"2023-11-06T20:59:57.787611+0000","last_undegraded":"2023-11-06T20:59:57.787611+0000","last_fullsized":"2023-11-06T20:59:57.787611+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806244+0000","last_change":"2023-11-06T20:59:57.806244+0000","last_active":"2023-11-06T20:59:57.806244+0000","last_peered":"2023-11-06T20:59:57.806244+0000","last_clean":"2023-11-06T20:59:57.806244+0000","last_became_active":"2023-11-06T20:59:57.805915+0000","last_became_peered":"2023-11-06T20:59:57.805915+0000","last_unstale":"2023-11-06T20:59:57.806244+0000","last_undegraded":"2023-11-06T20:59:57.806244+0000","last_fullsized":"2023-11-06T20:59:57.806244+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"98'4","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.818562+0000","last_change":"2023-11-06T20:59:57.308497+0000","last_active":"2023-11-06T20:59:57.818562+0000","last_peered":"2023-11-06T20:59:57.818562+0000","last_clean":"2023-11-06T20:59:57.818562+0000","last_became_active":"2023-11-06T20:59:57.307686+0000","last_became_peered":"2023-11-06T20:59:57.307686+0000","last_unstale":"2023-11-06T20:59:57.818562+0000","last_undegraded":"2023-11-06T20:59:57.818562+0000","last_fullsized":"2023-11-06T20:59:57.818562+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.802363+0000","last_change":"2023-11-06T20:59:57.795803+0000","last_active":"2023-11-06T20:59:57.802363+0000","last_peered":"2023-11-06T20:59:57.802363+0000","last_clean":"2023-11-06T20:59:57.802363+0000","last_became_active":"2023-11-06T20:59:57.795109+0000","last_became_peered":"2023-11-06T20:59:57.795109+0000","last_unstale":"2023-11-06T20:59:57.802363+0000","last_undegraded":"2023-11-06T20:59:57.802363+0000","last_fullsized":"2023-11-06T20:59:57.802363+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"98'2","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.815514+0000","last_change":"2023-11-06T20:59:57.319967+0000","last_active":"2023-11-06T20:59:57.815514+0000","last_peered":"2023-11-06T20:59:57.815514+0000","last_clean":"2023-11-06T20:59:57.815514+0000","last_became_active":"2023-11-06T20:59:57.319682+0000","last_became_peered":"2023-11-06T20:59:57.319682+0000","last_unstale":"2023-11-06T20:59:57.815514+0000","last_undegraded":"2023-11-06T20:59:57.815514+0000","last_fullsized":"2023-11-06T20:59:57.815514+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.799334+0000","last_change":"2023-11-06T20:59:57.799334+0000","last_active":"2023-11-06T20:59:57.799334+0000","last_peered":"2023-11-06T20:59:57.799334+0000","last_clean":"2023-11-06T20:59:57.799334+0000","last_became_active":"2023-11-06T20:59:57.798970+0000","last_became_peered":"2023-11-06T20:59:57.798970+0000","last_unstale":"2023-11-06T20:59:57.799334+0000","last_undegraded":"2023-11-06T20:59:57.799334+0000","last_fullsized":"2023-11-06T20:59:57.799334+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.784519+0000","last_change":"2023-11-06T20:59:57.311249+0000","last_active":"2023-11-06T20:59:57.784519+0000","last_peered":"2023-11-06T20:59:57.784519+0000","last_clean":"2023-11-06T20:59:57.784519+0000","last_became_active":"2023-11-06T20:59:57.310494+0000","last_became_peered":"2023-11-06T20:59:57.310494+0000","last_unstale":"2023-11-06T20:59:57.784519+0000","last_undegraded":"2023-11-06T20:59:57.784519+0000","last_fullsized":"2023-11-06T20:59:57.784519+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.804651+0000","last_change":"2023-11-06T20:59:57.797850+0000","last_active":"2023-11-06T20:59:57.804651+0000","last_peered":"2023-11-06T20:59:57.804651+0000","last_clean":"2023-11-06T20:59:57.804651+0000","last_became_active":"2023-11-06T20:59:57.797541+0000","last_became_peered":"2023-11-06T20:59:57.797541+0000","last_unstale":"2023-11-06T20:59:57.804651+0000","last_undegraded":"2023-11-06T20:59:57.804651+0000","last_fullsized":"2023-11-06T20:59:57.804651+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.323507+0000","last_change":"2023-11-06T20:59:57.318333+0000","last_active":"2023-11-06T20:59:57.323507+0000","last_peered":"2023-11-06T20:59:57.323507+0000","last_clean":"2023-11-06T20:59:57.323507+0000","last_became_active":"2023-11-06T20:59:57.318074+0000","last_became_peered":"2023-11-06T20:59:57.318074+0000","last_unstale":"2023-11-06T20:59:57.323507+0000","last_undegraded":"2023-11-06T20:59:57.323507+0000","last_fullsized":"2023-11-06T20:59:57.323507+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.815578+0000","last_change":"2023-11-06T20:59:57.815578+0000","last_active":"2023-11-06T20:59:57.815578+0000","last_peered":"2023-11-06T20:59:57.815578+0000","last_clean":"2023-11-06T20:59:57.815578+0000","last_became_active":"2023-11-06T20:59:57.815299+0000","last_became_peered":"2023-11-06T20:59:57.815299+0000","last_unstale":"2023-11-06T20:59:57.815578+0000","last_undegraded":"2023-11-06T20:59:57.815578+0000","last_fullsized":"2023-11-06T20:59:57.815578+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.790461+0000","last_change":"2023-11-06T20:59:57.312716+0000","last_active":"2023-11-06T20:59:57.790461+0000","last_peered":"2023-11-06T20:59:57.790461+0000","last_clean":"2023-11-06T20:59:57.790461+0000","last_became_active":"2023-11-06T20:59:57.312396+0000","last_became_peered":"2023-11-06T20:59:57.312396+0000","last_unstale":"2023-11-06T20:59:57.790461+0000","last_undegraded":"2023-11-06T20:59:57.790461+0000","last_fullsized":"2023-11-06T20:59:57.790461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805535+0000","last_change":"2023-11-06T20:59:57.800920+0000","last_active":"2023-11-06T20:59:57.805535+0000","last_peered":"2023-11-06T20:59:57.805535+0000","last_clean":"2023-11-06T20:59:57.805535+0000","last_became_active":"2023-11-06T20:59:57.800611+0000","last_became_peered":"2023-11-06T20:59:57.800611+0000","last_unstale":"2023-11-06T20:59:57.805535+0000","last_undegraded":"2023-11-06T20:59:57.805535+0000","last_fullsized":"2023-11-06T20:59:57.805535+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.301905+0000","last_change":"2023-11-06T20:59:57.299084+0000","last_active":"2023-11-06T20:59:57.301905+0000","last_peered":"2023-11-06T20:59:57.301905+0000","last_clean":"2023-11-06T20:59:57.301905+0000","last_became_active":"2023-11-06T20:59:57.298811+0000","last_became_peered":"2023-11-06T20:59:57.298811+0000","last_unstale":"2023-11-06T20:59:57.301905+0000","last_undegraded":"2023-11-06T20:59:57.301905+0000","last_fullsized":"2023-11-06T20:59:57.301905+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.807366+0000","last_change":"2023-11-06T20:59:57.803388+0000","last_active":"2023-11-06T20:59:57.807366+0000","last_peered":"2023-11-06T20:59:57.807366+0000","last_clean":"2023-11-06T20:59:57.807366+0000","last_became_active":"2023-11-06T20:59:57.803107+0000","last_became_peered":"2023-11-06T20:59:57.803107+0000","last_unstale":"2023-11-06T20:59:57.807366+0000","last_undegraded":"2023-11-06T20:59:57.807366+0000","last_fullsized":"2023-11-06T20:59:57.807366+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.783942+0000","last_change":"2023-11-06T20:59:57.310736+0000","last_active":"2023-11-06T20:59:57.783942+0000","last_peered":"2023-11-06T20:59:57.783942+0000","last_clean":"2023-11-06T20:59:57.783942+0000","last_became_active":"2023-11-06T20:59:57.310474+0000","last_became_peered":"2023-11-06T20:59:57.310474+0000","last_unstale":"2023-11-06T20:59:57.783942+0000","last_undegraded":"2023-11-06T20:59:57.783942+0000","last_fullsized":"2023-11-06T20:59:57.783942+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.814071+0000","last_change":"2023-11-06T20:59:57.814071+0000","last_active":"2023-11-06T20:59:57.814071+0000","last_peered":"2023-11-06T20:59:57.814071+0000","last_clean":"2023-11-06T20:59:57.814071+0000","last_became_active":"2023-11-06T20:59:57.813715+0000","last_became_peered":"2023-11-06T20:59:57.813715+0000","last_unstale":"2023-11-06T20:59:57.814071+0000","last_undegraded":"2023-11-06T20:59:57.814071+0000","last_fullsized":"2023-11-06T20:59:57.814071+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.788540+0000","last_change":"2023-11-06T20:59:57.304789+0000","last_active":"2023-11-06T20:59:57.788540+0000","last_peered":"2023-11-06T20:59:57.788540+0000","last_clean":"2023-11-06T20:59:57.788540+0000","last_became_active":"2023-11-06T20:59:57.304393+0000","last_became_peered":"2023-11-06T20:59:57.304393+0000","last_unstale":"2023-11-06T20:59:57.788540+0000","last_undegraded":"2023-11-06T20:59:57.788540+0000","last_fullsized":"2023-11-06T20:59:57.788540+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.812492+0000","last_change":"2023-11-06T20:59:57.808207+0000","last_active":"2023-11-06T20:59:57.812492+0000","last_peered":"2023-11-06T20:59:57.812492+0000","last_clean":"2023-11-06T20:59:57.812492+0000","last_became_active":"2023-11-06T20:59:57.807348+0000","last_became_peered":"2023-11-06T20:59:57.807348+0000","last_unstale":"2023-11-06T20:59:57.812492+0000","last_undegraded":"2023-11-06T20:59:57.812492+0000","last_fullsized":"2023-11-06T20:59:57.812492+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.319544+0000","last_change":"2023-11-06T20:59:57.313834+0000","last_active":"2023-11-06T20:59:57.319544+0000","last_peered":"2023-11-06T20:59:57.319544+0000","last_clean":"2023-11-06T20:59:57.319544+0000","last_became_active":"2023-11-06T20:59:57.313575+0000","last_became_peered":"2023-11-06T20:59:57.313575+0000","last_unstale":"2023-11-06T20:59:57.319544+0000","last_undegraded":"2023-11-06T20:59:57.319544+0000","last_fullsized":"2023-11-06T20:59:57.319544+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.814348+0000","last_change":"2023-11-06T20:59:57.814348+0000","last_active":"2023-11-06T20:59:57.814348+0000","last_peered":"2023-11-06T20:59:57.814348+0000","last_clean":"2023-11-06T20:59:57.814348+0000","last_became_active":"2023-11-06T20:59:57.814112+0000","last_became_peered":"2023-11-06T20:59:57.814112+0000","last_unstale":"2023-11-06T20:59:57.814348+0000","last_undegraded":"2023-11-06T20:59:57.814348+0000","last_fullsized":"2023-11-06T20:59:57.814348+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.791594+0000","last_change":"2023-11-06T20:59:57.317448+0000","last_active":"2023-11-06T20:59:57.791594+0000","last_peered":"2023-11-06T20:59:57.791594+0000","last_clean":"2023-11-06T20:59:57.791594+0000","last_became_active":"2023-11-06T20:59:57.316629+0000","last_became_peered":"2023-11-06T20:59:57.316629+0000","last_unstale":"2023-11-06T20:59:57.791594+0000","last_undegraded":"2023-11-06T20:59:57.791594+0000","last_fullsized":"2023-11-06T20:59:57.791594+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785695+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:59:57.785695+0000","last_peered":"2023-11-06T20:59:57.785695+0000","last_clean":"2023-11-06T20:59:57.785695+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:59:57.785695+0000","last_undegraded":"2023-11-06T20:59:57.785695+0000","last_fullsized":"2023-11-06T20:59:57.785695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819962+0000","last_change":"2023-11-06T20:59:57.811834+0000","last_active":"2023-11-06T20:59:57.819962+0000","last_peered":"2023-11-06T20:59:57.819962+0000","last_clean":"2023-11-06T20:59:57.819962+0000","last_became_active":"2023-11-06T20:59:57.811563+0000","last_became_peered":"2023-11-06T20:59:57.811563+0000","last_unstale":"2023-11-06T20:59:57.819962+0000","last_undegraded":"2023-11-06T20:59:57.819962+0000","last_fullsized":"2023-11-06T20:59:57.819962+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"98'1","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.788723+0000","last_change":"2023-11-06T20:59:57.316476+0000","last_active":"2023-11-06T20:59:57.788723+0000","last_peered":"2023-11-06T20:59:57.788723+0000","last_clean":"2023-11-06T20:59:57.788723+0000","last_became_active":"2023-11-06T20:59:57.315964+0000","last_became_peered":"2023-11-06T20:59:57.315964+0000","last_unstale":"2023-11-06T20:59:57.788723+0000","last_undegraded":"2023-11-06T20:59:57.788723+0000","last_fullsized":"2023-11-06T20:59:57.788723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.286986+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:59:57.286986+0000","last_peered":"2023-11-06T20:59:57.286986+0000","last_clean":"2023-11-06T20:59:57.286986+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:59:57.286986+0000","last_undegraded":"2023-11-06T20:59:57.286986+0000","last_fullsized":"2023-11-06T20:59:57.286986+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.797054+0000","last_change":"2023-11-06T20:59:57.797054+0000","last_active":"2023-11-06T20:59:57.797054+0000","last_peered":"2023-11-06T20:59:57.797054+0000","last_clean":"2023-11-06T20:59:57.797054+0000","last_became_active":"2023-11-06T20:59:57.796476+0000","last_became_peered":"2023-11-06T20:59:57.796476+0000","last_unstale":"2023-11-06T20:59:57.797054+0000","last_undegraded":"2023-11-06T20:59:57.797054+0000","last_fullsized":"2023-11-06T20:59:57.797054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.314743+0000","last_change":"2023-11-06T20:59:57.303556+0000","last_active":"2023-11-06T20:59:57.314743+0000","last_peered":"2023-11-06T20:59:57.314743+0000","last_clean":"2023-11-06T20:59:57.314743+0000","last_became_active":"2023-11-06T20:59:57.303267+0000","last_became_peered":"2023-11-06T20:59:57.303267+0000","last_unstale":"2023-11-06T20:59:57.314743+0000","last_undegraded":"2023-11-06T20:59:57.314743+0000","last_fullsized":"2023-11-06T20:59:57.314743+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.787693+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:59:57.787693+0000","last_peered":"2023-11-06T20:59:57.787693+0000","last_clean":"2023-11-06T20:59:57.787693+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:59:57.787693+0000","last_undegraded":"2023-11-06T20:59:57.787693+0000","last_fullsized":"2023-11-06T20:59:57.787693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.794237+0000","last_change":"2023-11-06T20:59:57.794237+0000","last_active":"2023-11-06T20:59:57.794237+0000","last_peered":"2023-11-06T20:59:57.794237+0000","last_clean":"2023-11-06T20:59:57.794237+0000","last_became_active":"2023-11-06T20:59:57.793983+0000","last_became_peered":"2023-11-06T20:59:57.793983+0000","last_unstale":"2023-11-06T20:59:57.794237+0000","last_undegraded":"2023-11-06T20:59:57.794237+0000","last_fullsized":"2023-11-06T20:59:57.794237+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.783825+0000","last_change":"2023-11-06T20:59:57.295140+0000","last_active":"2023-11-06T20:59:57.783825+0000","last_peered":"2023-11-06T20:59:57.783825+0000","last_clean":"2023-11-06T20:59:57.783825+0000","last_became_active":"2023-11-06T20:59:57.294744+0000","last_became_peered":"2023-11-06T20:59:57.294744+0000","last_unstale":"2023-11-06T20:59:57.783825+0000","last_undegraded":"2023-11-06T20:59:57.783825+0000","last_fullsized":"2023-11-06T20:59:57.783825+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.286620+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:59:57.286620+0000","last_peered":"2023-11-06T20:59:57.286620+0000","last_clean":"2023-11-06T20:59:57.286620+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:59:57.286620+0000","last_undegraded":"2023-11-06T20:59:57.286620+0000","last_fullsized":"2023-11-06T20:59:57.286620+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.300712+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:59:57.300712+0000","last_peered":"2023-11-06T20:59:57.300712+0000","last_clean":"2023-11-06T20:59:57.300712+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:59:57.300712+0000","last_undegraded":"2023-11-06T20:59:57.300712+0000","last_fullsized":"2023-11-06T20:59:57.300712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.809483+0000","last_change":"2023-11-06T20:59:57.804489+0000","last_active":"2023-11-06T20:59:57.809483+0000","last_peered":"2023-11-06T20:59:57.809483+0000","last_clean":"2023-11-06T20:59:57.809483+0000","last_became_active":"2023-11-06T20:59:57.804234+0000","last_became_peered":"2023-11-06T20:59:57.804234+0000","last_unstale":"2023-11-06T20:59:57.809483+0000","last_undegraded":"2023-11-06T20:59:57.809483+0000","last_fullsized":"2023-11-06T20:59:57.809483+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.318638+0000","last_change":"2023-11-06T20:59:57.312336+0000","last_active":"2023-11-06T20:59:57.318638+0000","last_peered":"2023-11-06T20:59:57.318638+0000","last_clean":"2023-11-06T20:59:57.318638+0000","last_became_active":"2023-11-06T20:59:57.311957+0000","last_became_peered":"2023-11-06T20:59:57.311957+0000","last_unstale":"2023-11-06T20:59:57.318638+0000","last_undegraded":"2023-11-06T20:59:57.318638+0000","last_fullsized":"2023-11-06T20:59:57.318638+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.287092+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:59:57.287092+0000","last_peered":"2023-11-06T20:59:57.287092+0000","last_clean":"2023-11-06T20:59:57.287092+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:59:57.287092+0000","last_undegraded":"2023-11-06T20:59:57.287092+0000","last_fullsized":"2023-11-06T20:59:57.287092+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819677+0000","last_change":"2023-11-06T20:59:57.817627+0000","last_active":"2023-11-06T20:59:57.819677+0000","last_peered":"2023-11-06T20:59:57.819677+0000","last_clean":"2023-11-06T20:59:57.819677+0000","last_became_active":"2023-11-06T20:59:57.817354+0000","last_became_peered":"2023-11-06T20:59:57.817354+0000","last_unstale":"2023-11-06T20:59:57.819677+0000","last_undegraded":"2023-11-06T20:59:57.819677+0000","last_fullsized":"2023-11-06T20:59:57.819677+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.790630+0000","last_change":"2023-11-06T20:59:57.316591+0000","last_active":"2023-11-06T20:59:57.790630+0000","last_peered":"2023-11-06T20:59:57.790630+0000","last_clean":"2023-11-06T20:59:57.790630+0000","last_became_active":"2023-11-06T20:59:57.316273+0000","last_became_peered":"2023-11-06T20:59:57.316273+0000","last_unstale":"2023-11-06T20:59:57.790630+0000","last_undegraded":"2023-11-06T20:59:57.790630+0000","last_fullsized":"2023-11-06T20:59:57.790630+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.288394+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:59:57.288394+0000","last_peered":"2023-11-06T20:59:57.288394+0000","last_clean":"2023-11-06T20:59:57.288394+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:59:57.288394+0000","last_undegraded":"2023-11-06T20:59:57.288394+0000","last_fullsized":"2023-11-06T20:59:57.288394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.809403+0000","last_change":"2023-11-06T20:59:57.803221+0000","last_active":"2023-11-06T20:59:57.809403+0000","last_peered":"2023-11-06T20:59:57.809403+0000","last_clean":"2023-11-06T20:59:57.809403+0000","last_became_active":"2023-11-06T20:59:57.802937+0000","last_became_peered":"2023-11-06T20:59:57.802937+0000","last_unstale":"2023-11-06T20:59:57.809403+0000","last_undegraded":"2023-11-06T20:59:57.809403+0000","last_fullsized":"2023-11-06T20:59:57.809403+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.817814+0000","last_change":"2023-11-06T20:59:57.308516+0000","last_active":"2023-11-06T20:59:57.817814+0000","last_peered":"2023-11-06T20:59:57.817814+0000","last_clean":"2023-11-06T20:59:57.817814+0000","last_became_active":"2023-11-06T20:59:57.308234+0000","last_became_peered":"2023-11-06T20:59:57.308234+0000","last_unstale":"2023-11-06T20:59:57.817814+0000","last_undegraded":"2023-11-06T20:59:57.817814+0000","last_fullsized":"2023-11-06T20:59:57.817814+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":78,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.303006+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:59:57.303006+0000","last_peered":"2023-11-06T20:59:57.303006+0000","last_clean":"2023-11-06T20:59:57.303006+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:59:57.303006+0000","last_undegraded":"2023-11-06T20:59:57.303006+0000","last_fullsized":"2023-11-06T20:59:57.303006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.793078+0000","last_change":"2023-11-06T20:59:57.793078+0000","last_active":"2023-11-06T20:59:57.793078+0000","last_peered":"2023-11-06T20:59:57.793078+0000","last_clean":"2023-11-06T20:59:57.793078+0000","last_became_active":"2023-11-06T20:59:57.792772+0000","last_became_peered":"2023-11-06T20:59:57.792772+0000","last_unstale":"2023-11-06T20:59:57.793078+0000","last_undegraded":"2023-11-06T20:59:57.793078+0000","last_fullsized":"2023-11-06T20:59:57.793078+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819254+0000","last_change":"2023-11-06T20:59:57.301686+0000","last_active":"2023-11-06T20:59:57.819254+0000","last_peered":"2023-11-06T20:59:57.819254+0000","last_clean":"2023-11-06T20:59:57.819254+0000","last_became_active":"2023-11-06T20:59:57.301412+0000","last_became_peered":"2023-11-06T20:59:57.301412+0000","last_unstale":"2023-11-06T20:59:57.819254+0000","last_undegraded":"2023-11-06T20:59:57.819254+0000","last_fullsized":"2023-11-06T20:59:57.819254+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.288014+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:59:57.288014+0000","last_peered":"2023-11-06T20:59:57.288014+0000","last_clean":"2023-11-06T20:59:57.288014+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:59:57.288014+0000","last_undegraded":"2023-11-06T20:59:57.288014+0000","last_fullsized":"2023-11-06T20:59:57.288014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575039,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":1196032,"data_stored":600371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.68500000000000005,"15min":0.69299999999999995},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.91300000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66900000000000004,"15min":0.69099999999999995},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.46800000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67000000000000004,"15min":0.70499999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40500000000000003},"max":{"1min":0.84899999999999998,"5min":0.95199999999999996,"15min":5.1749999999999998},"last":0.44},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.71499999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":1.133},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.70499999999999996,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.83199999999999996,"5min":0.996,"15min":1.165},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":1.024},"last":0.53000000000000003}]},{"osd":3,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89400000000000002,"5min":1.0129999999999999,"15min":5.202},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73099999999999998,"15min":0.76100000000000001},"min":{"1min":0.55100000000000005,"5min":0.51000000000000001,"15min":0.49399999999999999},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":5.165},"last":0.60399999999999998}]},{"osd":4,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.81399999999999995,"15min":1.083},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.86799999999999999,"15min":0.92800000000000005},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41099999999999998},"max":{"1min":0.85099999999999998,"5min":0.93300000000000005,"15min":1.395},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.129},"last":0.61299999999999999}]},{"osd":6,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.435},"max":{"1min":0.86599999999999999,"5min":0.94499999999999995,"15min":1.24},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.54500000000000004,"5min":0.47399999999999998,"15min":0.443},"max":{"1min":0.80200000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575039,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302372,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069468,"statfs":{"total":96636764160,"available":96327135232,"internally_reserved":0,"allocated":1208320,"data_stored":601423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.33000000000000002,"15min":0.32800000000000001},"max":{"1min":0.87,"5min":0.93999999999999995,"15min":1.0720000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.47899999999999998,"5min":0.47599999999999998,"15min":0.38800000000000001},"max":{"1min":0.86799999999999999,"5min":1.216,"15min":1.216},"last":0.86299999999999999}]},{"osd":1,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.69799999999999995},"min":{"1min":0.49199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91700000000000004,"5min":0.97599999999999998,"15min":1.3799999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.41899999999999998,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":0.98599999999999999,"15min":1.1659999999999999},"last":0.84099999999999997}]},{"osd":2,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76300000000000001,"15min":0.75800000000000001},"min":{"1min":0.59699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96699999999999997,"5min":1.0049999999999999,"15min":1.321},"last":0.746},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.60899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92400000000000004,"5min":1.05,"15min":1.405},"last":0.91300000000000003}]},{"osd":3,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71199999999999997,"15min":0.71699999999999997},"min":{"1min":0.59499999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.1459999999999999,"15min":1.5109999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.60699999999999998,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":0.93200000000000005,"5min":1.002,"15min":1.002},"last":0.82699999999999996}]},{"osd":4,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.018},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.63,"15min":0.64100000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":2.0310000000000001},"last":0.93300000000000005}]},{"osd":5,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.47999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79900000000000004,"5min":1.028,"15min":1.028},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.133},"last":0.97099999999999997}]},{"osd":7,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.47999999999999998,"5min":0.40799999999999997,"15min":0.378},"max":{"1min":0.85599999999999998,"5min":1.002,"15min":1.0940000000000001},"last":0.997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.47699999999999998,"15min":0.46899999999999997},"max":{"1min":0.94799999999999995,"5min":0.97799999999999998,"15min":1.4319999999999999},"last":0.98699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575039,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302372,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069468,"statfs":{"total":96636764160,"available":96327135232,"internally_reserved":0,"allocated":1208320,"data_stored":601049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46999999999999997,"15min":0.45300000000000001},"min":{"1min":0.29199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.75700000000000001,"5min":0.77900000000000003,"15min":0.85199999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51200000000000001,"15min":0.505},"min":{"1min":0.35299999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72499999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":1.8360000000000001}]},{"osd":2,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51100000000000001,"15min":0.504},"min":{"1min":0.33800000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.71499999999999997,"5min":0.80800000000000005,"15min":0.86599999999999999},"last":2.6930000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.59899999999999998},"min":{"1min":0.45300000000000001,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":4.431}]},{"osd":3,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.35899999999999999},"max":{"1min":0.876,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":4.5380000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85399999999999998,"5min":0.94099999999999995,"15min":1.29},"last":4.3550000000000004}]},{"osd":4,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77400000000000002,"15min":0.79000000000000004},"min":{"1min":0.51400000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95899999999999996,"5min":1.083,"15min":1.171},"last":2.5870000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"max":{"1min":1.0369999999999999,"5min":1.0720000000000001,"15min":1.1519999999999999},"last":1.883}]},{"osd":5,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.008,"5min":1.042,"15min":1.329},"last":4.399},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.78400000000000003},"min":{"1min":0.56699999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0109999999999999,"5min":1.028,"15min":1.1160000000000001},"last":2.649}]},{"osd":6,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80600000000000005,"15min":0.80700000000000005},"min":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.501},"max":{"1min":1.0009999999999999,"5min":1.0880000000000001,"15min":1.135},"last":3.024},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80800000000000005,"15min":0.82199999999999995},"min":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94099999999999995,"5min":1.1040000000000001,"15min":1.3540000000000001},"last":3.0609999999999999}]},{"osd":7,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.83699999999999997},"min":{"1min":0.58699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98699999999999999,"5min":1.077,"15min":1.405},"last":2.7869999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.64100000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.012,"5min":1.105,"15min":1.1950000000000001},"last":5.8230000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575039,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302364,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069476,"statfs":{"total":96636764160,"available":96327143424,"internally_reserved":0,"allocated":1200128,"data_stored":600393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.34300000000000003,"5min":0.308,"15min":0.29299999999999998},"max":{"1min":0.69999999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.311},"max":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.97399999999999998},"last":0.499}]},{"osd":2,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46300000000000002,"15min":0.46800000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60599999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53300000000000003,"15min":0.54600000000000004},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.65100000000000002,"5min":0.77500000000000002,"15min":0.91000000000000003},"last":0.59099999999999997}]},{"osd":3,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.38300000000000001},"max":{"1min":0.75,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.52500000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.80700000000000005},"min":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.3779999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.153,"5min":1.298,"15min":1.8380000000000001},"last":0.84199999999999997}]},{"osd":5,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78600000000000003,"15min":0.80800000000000005},"min":{"1min":0.63500000000000001,"5min":0.51000000000000001,"15min":0.48999999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.3169999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.55500000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2010000000000001,"5min":1.2549999999999999,"15min":1.915},"last":0.93000000000000005}]},{"osd":6,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77400000000000002,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.54600000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78400000000000003,"15min":0.81299999999999994},"min":{"1min":0.53900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.5860000000000001},"last":0.879}]},{"osd":7,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"max":{"1min":1.171,"5min":1.171,"15min":1.2490000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81399999999999995,"15min":0.83399999999999996},"min":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.6359999999999999},"last":0.98399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575039,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":1196032,"data_stored":600371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.25900000000000001},"max":{"1min":0.54400000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.36299999999999999,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.64300000000000002,"5min":0.72699999999999998,"15min":1.0269999999999999},"last":0.34799999999999998}]},{"osd":1,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61399999999999999,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73199999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52100000000000002,"15min":0.50600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85899999999999999,"5min":2.266,"15min":2.266},"last":0.374},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60499999999999998,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.78200000000000003,"5min":1.732,"15min":2.242},"last":0.57699999999999996}]},{"osd":4,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.55900000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1,"5min":1.0980000000000001,"15min":1.371},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.75,"15min":0.77700000000000002},"min":{"1min":0.51900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0429999999999999,"5min":1.087,"15min":1.357},"last":0.52700000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76700000000000002,"15min":0.78000000000000003},"min":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.45900000000000002},"max":{"1min":1.085,"5min":1.657,"15min":1.657},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.498},"max":{"1min":0.95299999999999996,"5min":1.1479999999999999,"15min":1.4099999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78700000000000003,"15min":0.80300000000000005},"min":{"1min":0.54900000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.026,"5min":1.1499999999999999,"15min":1.4239999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81399999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.47499999999999998}]},{"osd":7,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.54900000000000004,"5min":0.52800000000000002,"15min":0.46600000000000003},"max":{"1min":0.98699999999999999,"5min":1.1859999999999999,"15min":1.534},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82799999999999996,"15min":0.82599999999999996},"min":{"1min":0.58799999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0369999999999999,"5min":2.202,"15min":2.202},"last":0.54100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575040,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302380,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069460,"statfs":{"total":96636764160,"available":96327127040,"internally_reserved":0,"allocated":1216512,"data_stored":601673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44400000000000001,"15min":0.45600000000000002},"min":{"1min":0.30199999999999999,"5min":0.28399999999999997,"15min":0.26800000000000002},"max":{"1min":0.56899999999999995,"5min":0.71399999999999997,"15min":1.046},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.52700000000000002,"15min":0.51200000000000001},"min":{"1min":0.39400000000000002,"5min":0.34599999999999997,"15min":0.32600000000000001},"max":{"1min":0.68700000000000006,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46600000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48799999999999999,"15min":0.48499999999999999},"min":{"1min":0.39200000000000002,"5min":0.318,"15min":0.313},"max":{"1min":0.57399999999999995,"5min":0.64600000000000002,"15min":0.753},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54600000000000004,"15min":0.55900000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34499999999999997},"max":{"1min":0.70099999999999996,"5min":0.80900000000000005,"15min":1.177},"last":0.40200000000000002}]},{"osd":2,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.437,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73399999999999999,"5min":0.88100000000000001,"15min":1.3120000000000001},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81699999999999995,"15min":0.81000000000000005},"min":{"1min":0.64500000000000002,"5min":0.57899999999999996,"15min":0.51200000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.2470000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.47999999999999998},"max":{"1min":1.276,"5min":1.276,"15min":1.286},"last":0.66800000000000004}]},{"osd":5,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.1519999999999999,"5min":1.1779999999999999,"15min":1.228},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.60399999999999998,"5min":0.54600000000000004,"15min":0.49199999999999999},"max":{"1min":1.101,"5min":1.137,"15min":1.1439999999999999},"last":0.65800000000000003}]},{"osd":6,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.55900000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.747},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80100000000000005,"15min":0.81000000000000005},"min":{"1min":0.622,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.054,"5min":1.202,"15min":1.268},"last":0.71699999999999997}]},{"osd":7,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0740000000000001,"5min":1.302,"15min":1.302},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.60499999999999998,"5min":0.56899999999999995,"15min":0.47899999999999998},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.61799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575039,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302368,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069472,"statfs":{"total":96636764160,"available":96327139328,"internally_reserved":0,"allocated":1204224,"data_stored":601285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66700000000000004,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94699999999999995,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.501},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.67600000000000005},"min":{"1min":0.58099999999999996,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83099999999999996,"5min":1,"15min":1},"last":0.53500000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.645,"15min":1.645},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.54800000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.625}]},{"osd":2,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70999999999999996,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":0.92800000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92300000000000004,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.73899999999999999}]},{"osd":3,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68200000000000005,"15min":0.69499999999999995},"min":{"1min":0.45200000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90300000000000002,"5min":0.95799999999999996,"15min":1.0169999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.42799999999999999},"max":{"1min":0.89700000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83099999999999996,"5min":1.0389999999999999,"15min":1.0880000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.40799999999999997},"max":{"1min":0.86599999999999999,"5min":0.93999999999999995,"15min":1.3520000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.81699999999999995,"5min":1.22,"15min":1.22},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.51300000000000001,"5min":0.44900000000000001,"15min":0.44},"max":{"1min":0.95899999999999996,"5min":1.0349999999999999,"15min":1.3999999999999999},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.66200000000000003},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.89600000000000002,"5min":1.034,"15min":1.5209999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.67600000000000005},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89800000000000002,"5min":1.177,"15min":1.177},"last":0.72199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575040,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302368,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069472,"statfs":{"total":96636764160,"available":96327139328,"internally_reserved":0,"allocated":1204224,"data_stored":601015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.66400000000000003},"min":{"1min":0.40100000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.82499999999999996,"5min":0.98599999999999999,"15min":1.0109999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.92000000000000004,"5min":0.96199999999999997,"15min":0.96799999999999997},"last":0.72299999999999998}]},{"osd":1,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70599999999999996,"15min":0.70699999999999996},"min":{"1min":0.54300000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":1.0109999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.747},"min":{"1min":0.48199999999999998,"5min":0.432,"15min":0.40300000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.044},"last":0.79100000000000004}]},{"osd":2,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.503,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.024},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.59799999999999998,"5min":0.46000000000000002,"15min":0.45600000000000002},"max":{"1min":0.96399999999999997,"5min":0.96999999999999997,"15min":1.0329999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.629},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.92000000000000004,"5min":0.97599999999999998,"15min":1.02},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.45300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.86399999999999999,"15min":0.93700000000000006},"last":0.496},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.68600000000000005},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89800000000000002,"5min":0.97499999999999998,"15min":0.98599999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.96399999999999997,"15min":0.97599999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":0.88,"5min":0.96399999999999997,"15min":1.105},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.998,"15min":0.998},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T21:00:02.133 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T21:00:02.265 INFO:tasks.ceph.mds.b.smithi069.stderr:2023-11-06T21:00:02.265+0000 7f42623f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T21:00:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.362+0000 7f77ec3ff700 1 Processor -- start 2023-11-06T21:00:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.362+0000 7f77ec3ff700 1 -- start start 2023-11-06T21:00:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.363+0000 7f77ec3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f77e4137660 0x7f77e40c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.363+0000 7f77ec3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77e4069410 0x7f77e4069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.363+0000 7f77ec3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e40c17e0 con 0x7f77e40c3600 2023-11-06T21:00:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.363+0000 7f77ec3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e406a740 con 0x7f77e4137660 2023-11-06T21:00:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.363+0000 7f77ec3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e406a180 con 0x7f77e4069410 2023-11-06T21:00:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.363+0000 7f77ea19b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f77e40c3600 0x7f77e4136f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38870/0 (socket says 172.21.15.33:38870) 2023-11-06T21:00:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.364+0000 7f77ea19b700 1 -- 172.21.15.33:0/3807860433 learned_addr learned my addr 172.21.15.33:0/3807860433 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.364+0000 7f77e999a700 1 --2- 172.21.15.33:0/3807860433 >> v2:172.21.15.69:3300/0 conn(0x7f77e4137660 0x7f77e40c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.364+0000 7f77ea99c700 1 --2- 172.21.15.33:0/3807860433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77e4069410 0x7f77e4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.364+0000 7f77e9199700 1 -- 172.21.15.33:0/3807860433 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175422128 0 0) 0x7f77e40c17e0 con 0x7f77e40c3600 2023-11-06T21:00:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.364+0000 7f77e9199700 1 -- 172.21.15.33:0/3807860433 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77d00033f0 con 0x7f77e40c3600 2023-11-06T21:00:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.364+0000 7f77e999a700 1 -- 172.21.15.33:0/3807860433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77e4069410 msgr2=0x7f77e4069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.364+0000 7f77e999a700 1 --2- 172.21.15.33:0/3807860433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77e4069410 0x7f77e4069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.364+0000 7f77e999a700 1 -- 172.21.15.33:0/3807860433 >> v1:172.21.15.33:6789/0 conn(0x7f77e40c3600 legacy=0x7f77e4136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.364+0000 7f77e999a700 1 -- 172.21.15.33:0/3807860433 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e406a460 con 0x7f77e4137660 2023-11-06T21:00:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.365+0000 7f77e999a700 1 --2- 172.21.15.33:0/3807860433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77e4137660 0x7f77e40c39e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f77d4009760 tx=0x7f77d4009060).ready entity=mon.1 client_cookie=26afd8202a6d843b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.365+0000 7f77e9199700 1 -- 172.21.15.33:0/3807860433 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d400d4a0 con 0x7f77e4137660 2023-11-06T21:00:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.365+0000 7f77e9199700 1 -- 172.21.15.33:0/3807860433 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d400d600 con 0x7f77e4137660 2023-11-06T21:00:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.365+0000 7f77e9199700 1 -- 172.21.15.33:0/3807860433 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d400d980 con 0x7f77e4137660 2023-11-06T21:00:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.365+0000 7f77ec3ff700 1 -- 172.21.15.33:0/3807860433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77e4137660 msgr2=0x7f77e40c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.365+0000 7f77ec3ff700 1 --2- 172.21.15.33:0/3807860433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77e4137660 0x7f77e40c39e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f77d4009760 tx=0x7f77d4009060).stop 2023-11-06T21:00:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.366+0000 7f77ec3ff700 1 -- 172.21.15.33:0/3807860433 shutdown_connections 2023-11-06T21:00:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.366+0000 7f77ec3ff700 1 --2- 172.21.15.33:0/3807860433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77e4069410 0x7f77e4069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.366+0000 7f77ec3ff700 1 --2- 172.21.15.33:0/3807860433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77e4137660 0x7f77e40c39e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.366+0000 7f77ec3ff700 1 -- 172.21.15.33:0/3807860433 >> 172.21.15.33:0/3807860433 conn(0x7f77e4062710 msgr2=0x7f77e4062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.366+0000 7f77ec3ff700 1 -- 172.21.15.33:0/3807860433 shutdown_connections 2023-11-06T21:00:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.366+0000 7f77ec3ff700 1 -- 172.21.15.33:0/3807860433 wait complete. 2023-11-06T21:00:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.367+0000 7f77ec3ff700 1 Processor -- start 2023-11-06T21:00:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.367+0000 7f77ec3ff700 1 -- start start 2023-11-06T21:00:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.368+0000 7f77ec3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77e4069410 0x7f77e40c0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.368+0000 7f77ec3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e40c3600 0x7f77e41331a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.368+0000 7f77ec3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77e4137660 0x7f77e4133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.368+0000 7f77ec3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4135320 con 0x7f77e40c3600 2023-11-06T21:00:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.368+0000 7f77ec3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4135600 con 0x7f77e4137660 2023-11-06T21:00:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.368+0000 7f77ec3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e41358e0 con 0x7f77e4069410 2023-11-06T21:00:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.368+0000 7f77e999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e40c3600 0x7f77e41331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.368+0000 7f77e999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e40c3600 0x7f77e41331a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59842/0 (socket says 172.21.15.33:59842) 2023-11-06T21:00:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.368+0000 7f77e999a700 1 -- 172.21.15.33:0/2279438959 learned_addr learned my addr 172.21.15.33:0/2279438959 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77ea19b700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77e4069410 0x7f77e40c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77ea99c700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77e4137660 0x7f77e4133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77e999a700 1 -- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77e4069410 msgr2=0x7f77e40c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77e999a700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77e4069410 0x7f77e40c0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77e999a700 1 -- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77e4137660 msgr2=0x7f77e4133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77e999a700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77e4137660 0x7f77e4133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77e999a700 1 -- 172.21.15.33:0/2279438959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d40092a0 con 0x7f77e40c3600 2023-11-06T21:00:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77ea99c700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77e4137660 0x7f77e4133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77e999a700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e40c3600 0x7f77e41331a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f77d4009d50 tx=0x7f77d4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77da7fc700 1 -- 172.21.15.33:0/2279438959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4000cb0 con 0x7f77e40c3600 2023-11-06T21:00:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77da7fc700 1 -- 172.21.15.33:0/2279438959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d400d770 con 0x7f77e40c3600 2023-11-06T21:00:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77da7fc700 1 -- 172.21.15.33:0/2279438959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d401d430 con 0x7f77e40c3600 2023-11-06T21:00:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.369+0000 7f77ec3ff700 1 -- 172.21.15.33:0/2279438959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e4135bc0 con 0x7f77e40c3600 2023-11-06T21:00:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.370+0000 7f77ec3ff700 1 -- 172.21.15.33:0/2279438959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e4134fd0 con 0x7f77e40c3600 2023-11-06T21:00:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.371+0000 7f77ec3ff700 1 -- 172.21.15.33:0/2279438959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e40acb50 con 0x7f77e40c3600 2023-11-06T21:00:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.371+0000 7f77da7fc700 1 -- 172.21.15.33:0/2279438959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f77d401d7e0 con 0x7f77e40c3600 2023-11-06T21:00:02.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.372+0000 7f77da7fc700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f77c805f270 0x7f77c8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.372+0000 7f77da7fc700 1 -- 172.21.15.33:0/2279438959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f77d407be20 con 0x7f77e40c3600 2023-11-06T21:00:02.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.373+0000 7f77ea19b700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f77c805f270 0x7f77c8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.373+0000 7f77ea19b700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f77c805f270 0x7f77c8061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f77e0005750 tx=0x7f77e0009040).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:02.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.377+0000 7f77da7fc700 1 -- 172.21.15.33:0/2279438959 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f77e40acb50 con 0x7f77e40c3600 2023-11-06T21:00:02.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.551+0000 7f77ec3ff700 1 -- 172.21.15.33:0/2279438959 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77e40acb50 con 0x7f77c805f270 2023-11-06T21:00:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.562+0000 7f77da7fc700 1 -- 172.21.15.33:0/2279438959 <== mgr.6280 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361224 (secure 0 0 0) 0x7f77e40acb50 con 0x7f77c805f270 2023-11-06T21:00:02.567 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:02.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 -- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f77c805f270 msgr2=0x7f77c8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f77c805f270 0x7f77c8061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f77e0005750 tx=0x7f77e0009040).stop 2023-11-06T21:00:02.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 -- 172.21.15.33:0/2279438959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e40c3600 msgr2=0x7f77e41331a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e40c3600 0x7f77e41331a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f77d4009d50 tx=0x7f77d4016000).stop 2023-11-06T21:00:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 -- 172.21.15.33:0/2279438959 shutdown_connections 2023-11-06T21:00:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f77c805f270 0x7f77c8061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f77e4069410 0x7f77e40c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77e4137660 0x7f77e4133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 --2- 172.21.15.33:0/2279438959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e40c3600 0x7f77e41331a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 -- 172.21.15.33:0/2279438959 >> 172.21.15.33:0/2279438959 conn(0x7f77e4062710 msgr2=0x7f77e414fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 -- 172.21.15.33:0/2279438959 shutdown_connections 2023-11-06T21:00:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.567+0000 7f77ec3ff700 1 -- 172.21.15.33:0/2279438959 wait complete. 2023-11-06T21:00:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T21:00:02.584 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T21:00:01.232346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2418944,"kb_used_data":9408,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752555776,"statfs":{"total":773094113280,"available":770617114624,"internally_reserved":0,"allocated":9633792,"data_stored":4807580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11235,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-64,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013750"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.807797+0000","last_change":"2023-11-06T20:59:57.807797+0000","last_active":"2023-11-06T20:59:57.807797+0000","last_peered":"2023-11-06T20:59:57.807797+0000","last_clean":"2023-11-06T20:59:57.807797+0000","last_became_active":"2023-11-06T20:59:57.807529+0000","last_became_peered":"2023-11-06T20:59:57.807529+0000","last_unstale":"2023-11-06T20:59:57.807797+0000","last_undegraded":"2023-11-06T20:59:57.807797+0000","last_fullsized":"2023-11-06T20:59:57.807797+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3f","version":"98'3","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.816456+0000","last_change":"2023-11-06T20:59:57.306863+0000","last_active":"2023-11-06T20:59:57.816456+0000","last_peered":"2023-11-06T20:59:57.816456+0000","last_clean":"2023-11-06T20:59:57.816456+0000","last_became_active":"2023-11-06T20:59:57.306589+0000","last_became_peered":"2023-11-06T20:59:57.306589+0000","last_unstale":"2023-11-06T20:59:57.816456+0000","last_undegraded":"2023-11-06T20:59:57.816456+0000","last_fullsized":"2023-11-06T20:59:57.816456+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.816368+0000","last_change":"2023-11-06T20:59:57.814029+0000","last_active":"2023-11-06T20:59:57.816368+0000","last_peered":"2023-11-06T20:59:57.816368+0000","last_clean":"2023-11-06T20:59:57.816368+0000","last_became_active":"2023-11-06T20:59:57.813769+0000","last_became_peered":"2023-11-06T20:59:57.813769+0000","last_unstale":"2023-11-06T20:59:57.816368+0000","last_undegraded":"2023-11-06T20:59:57.816368+0000","last_fullsized":"2023-11-06T20:59:57.816368+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.790650+0000","last_change":"2023-11-06T20:59:57.320386+0000","last_active":"2023-11-06T20:59:57.790650+0000","last_peered":"2023-11-06T20:59:57.790650+0000","last_clean":"2023-11-06T20:59:57.790650+0000","last_became_active":"2023-11-06T20:59:57.320050+0000","last_became_peered":"2023-11-06T20:59:57.320050+0000","last_unstale":"2023-11-06T20:59:57.790650+0000","last_undegraded":"2023-11-06T20:59:57.790650+0000","last_fullsized":"2023-11-06T20:59:57.790650+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.820366+0000","last_change":"2023-11-06T20:59:57.819182+0000","last_active":"2023-11-06T20:59:57.820366+0000","last_peered":"2023-11-06T20:59:57.820366+0000","last_clean":"2023-11-06T20:59:57.820366+0000","last_became_active":"2023-11-06T20:59:57.818866+0000","last_became_peered":"2023-11-06T20:59:57.818866+0000","last_unstale":"2023-11-06T20:59:57.820366+0000","last_undegraded":"2023-11-06T20:59:57.820366+0000","last_fullsized":"2023-11-06T20:59:57.820366+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.791490+0000","last_change":"2023-11-06T20:59:57.322470+0000","last_active":"2023-11-06T20:59:57.791490+0000","last_peered":"2023-11-06T20:59:57.791490+0000","last_clean":"2023-11-06T20:59:57.791490+0000","last_became_active":"2023-11-06T20:59:57.321828+0000","last_became_peered":"2023-11-06T20:59:57.321828+0000","last_unstale":"2023-11-06T20:59:57.791490+0000","last_undegraded":"2023-11-06T20:59:57.791490+0000","last_fullsized":"2023-11-06T20:59:57.791490+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.817395+0000","last_change":"2023-11-06T20:59:57.808655+0000","last_active":"2023-11-06T20:59:57.817395+0000","last_peered":"2023-11-06T20:59:57.817395+0000","last_clean":"2023-11-06T20:59:57.817395+0000","last_became_active":"2023-11-06T20:59:57.807192+0000","last_became_peered":"2023-11-06T20:59:57.807192+0000","last_unstale":"2023-11-06T20:59:57.817395+0000","last_undegraded":"2023-11-06T20:59:57.817395+0000","last_fullsized":"2023-11-06T20:59:57.817395+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819889+0000","last_change":"2023-11-06T20:59:57.322230+0000","last_active":"2023-11-06T20:59:57.819889+0000","last_peered":"2023-11-06T20:59:57.819889+0000","last_clean":"2023-11-06T20:59:57.819889+0000","last_became_active":"2023-11-06T20:59:57.321840+0000","last_became_peered":"2023-11-06T20:59:57.321840+0000","last_unstale":"2023-11-06T20:59:57.819889+0000","last_undegraded":"2023-11-06T20:59:57.819889+0000","last_fullsized":"2023-11-06T20:59:57.819889+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806946+0000","last_change":"2023-11-06T20:59:57.801348+0000","last_active":"2023-11-06T20:59:57.806946+0000","last_peered":"2023-11-06T20:59:57.806946+0000","last_clean":"2023-11-06T20:59:57.806946+0000","last_became_active":"2023-11-06T20:59:57.799253+0000","last_became_peered":"2023-11-06T20:59:57.799253+0000","last_unstale":"2023-11-06T20:59:57.806946+0000","last_undegraded":"2023-11-06T20:59:57.806946+0000","last_fullsized":"2023-11-06T20:59:57.806946+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.818466+0000","last_change":"2023-11-06T20:59:57.304170+0000","last_active":"2023-11-06T20:59:57.818466+0000","last_peered":"2023-11-06T20:59:57.818466+0000","last_clean":"2023-11-06T20:59:57.818466+0000","last_became_active":"2023-11-06T20:59:57.303898+0000","last_became_peered":"2023-11-06T20:59:57.303898+0000","last_unstale":"2023-11-06T20:59:57.818466+0000","last_undegraded":"2023-11-06T20:59:57.818466+0000","last_fullsized":"2023-11-06T20:59:57.818466+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805363+0000","last_change":"2023-11-06T20:59:57.805363+0000","last_active":"2023-11-06T20:59:57.805363+0000","last_peered":"2023-11-06T20:59:57.805363+0000","last_clean":"2023-11-06T20:59:57.805363+0000","last_became_active":"2023-11-06T20:59:57.805059+0000","last_became_peered":"2023-11-06T20:59:57.805059+0000","last_unstale":"2023-11-06T20:59:57.805363+0000","last_undegraded":"2023-11-06T20:59:57.805363+0000","last_fullsized":"2023-11-06T20:59:57.805363+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3a","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.817043+0000","last_change":"2023-11-06T20:59:57.313767+0000","last_active":"2023-11-06T20:59:57.817043+0000","last_peered":"2023-11-06T20:59:57.817043+0000","last_clean":"2023-11-06T20:59:57.817043+0000","last_became_active":"2023-11-06T20:59:57.313496+0000","last_became_peered":"2023-11-06T20:59:57.313496+0000","last_unstale":"2023-11-06T20:59:57.817043+0000","last_undegraded":"2023-11-06T20:59:57.817043+0000","last_fullsized":"2023-11-06T20:59:57.817043+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.815394+0000","last_change":"2023-11-06T20:59:57.815394+0000","last_active":"2023-11-06T20:59:57.815394+0000","last_peered":"2023-11-06T20:59:57.815394+0000","last_clean":"2023-11-06T20:59:57.815394+0000","last_became_active":"2023-11-06T20:59:57.814883+0000","last_became_peered":"2023-11-06T20:59:57.814883+0000","last_unstale":"2023-11-06T20:59:57.815394+0000","last_undegraded":"2023-11-06T20:59:57.815394+0000","last_fullsized":"2023-11-06T20:59:57.815394+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.320786+0000","last_change":"2023-11-06T20:59:57.315997+0000","last_active":"2023-11-06T20:59:57.320786+0000","last_peered":"2023-11-06T20:59:57.320786+0000","last_clean":"2023-11-06T20:59:57.320786+0000","last_became_active":"2023-11-06T20:59:57.315722+0000","last_became_peered":"2023-11-06T20:59:57.315722+0000","last_unstale":"2023-11-06T20:59:57.320786+0000","last_undegraded":"2023-11-06T20:59:57.320786+0000","last_fullsized":"2023-11-06T20:59:57.320786+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.795557+0000","last_change":"2023-11-06T20:59:57.795557+0000","last_active":"2023-11-06T20:59:57.795557+0000","last_peered":"2023-11-06T20:59:57.795557+0000","last_clean":"2023-11-06T20:59:57.795557+0000","last_became_active":"2023-11-06T20:59:57.795070+0000","last_became_peered":"2023-11-06T20:59:57.795070+0000","last_unstale":"2023-11-06T20:59:57.795557+0000","last_undegraded":"2023-11-06T20:59:57.795557+0000","last_fullsized":"2023-11-06T20:59:57.795557+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.309319+0000","last_change":"2023-11-06T20:59:57.303023+0000","last_active":"2023-11-06T20:59:57.309319+0000","last_peered":"2023-11-06T20:59:57.309319+0000","last_clean":"2023-11-06T20:59:57.309319+0000","last_became_active":"2023-11-06T20:59:57.302679+0000","last_became_peered":"2023-11-06T20:59:57.302679+0000","last_unstale":"2023-11-06T20:59:57.309319+0000","last_undegraded":"2023-11-06T20:59:57.309319+0000","last_fullsized":"2023-11-06T20:59:57.309319+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.812485+0000","last_change":"2023-11-06T20:59:57.806679+0000","last_active":"2023-11-06T20:59:57.812485+0000","last_peered":"2023-11-06T20:59:57.812485+0000","last_clean":"2023-11-06T20:59:57.812485+0000","last_became_active":"2023-11-06T20:59:57.805039+0000","last_became_peered":"2023-11-06T20:59:57.805039+0000","last_unstale":"2023-11-06T20:59:57.812485+0000","last_undegraded":"2023-11-06T20:59:57.812485+0000","last_fullsized":"2023-11-06T20:59:57.812485+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.796920+0000","last_change":"2023-11-06T20:59:57.317756+0000","last_active":"2023-11-06T20:59:57.796920+0000","last_peered":"2023-11-06T20:59:57.796920+0000","last_clean":"2023-11-06T20:59:57.796920+0000","last_became_active":"2023-11-06T20:59:57.315938+0000","last_became_peered":"2023-11-06T20:59:57.315938+0000","last_unstale":"2023-11-06T20:59:57.796920+0000","last_undegraded":"2023-11-06T20:59:57.796920+0000","last_fullsized":"2023-11-06T20:59:57.796920+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819737+0000","last_change":"2023-11-06T20:59:57.818428+0000","last_active":"2023-11-06T20:59:57.819737+0000","last_peered":"2023-11-06T20:59:57.819737+0000","last_clean":"2023-11-06T20:59:57.819737+0000","last_became_active":"2023-11-06T20:59:57.818199+0000","last_became_peered":"2023-11-06T20:59:57.818199+0000","last_unstale":"2023-11-06T20:59:57.819737+0000","last_undegraded":"2023-11-06T20:59:57.819737+0000","last_fullsized":"2023-11-06T20:59:57.819737+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.326166+0000","last_change":"2023-11-06T20:59:57.319047+0000","last_active":"2023-11-06T20:59:57.326166+0000","last_peered":"2023-11-06T20:59:57.326166+0000","last_clean":"2023-11-06T20:59:57.326166+0000","last_became_active":"2023-11-06T20:59:57.318681+0000","last_became_peered":"2023-11-06T20:59:57.318681+0000","last_unstale":"2023-11-06T20:59:57.326166+0000","last_undegraded":"2023-11-06T20:59:57.326166+0000","last_fullsized":"2023-11-06T20:59:57.326166+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.817951+0000","last_change":"2023-11-06T20:59:57.817951+0000","last_active":"2023-11-06T20:59:57.817951+0000","last_peered":"2023-11-06T20:59:57.817951+0000","last_clean":"2023-11-06T20:59:57.817951+0000","last_became_active":"2023-11-06T20:59:57.814597+0000","last_became_peered":"2023-11-06T20:59:57.814597+0000","last_unstale":"2023-11-06T20:59:57.817951+0000","last_undegraded":"2023-11-06T20:59:57.817951+0000","last_fullsized":"2023-11-06T20:59:57.817951+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.309497+0000","last_change":"2023-11-06T20:59:57.303890+0000","last_active":"2023-11-06T20:59:57.309497+0000","last_peered":"2023-11-06T20:59:57.309497+0000","last_clean":"2023-11-06T20:59:57.309497+0000","last_became_active":"2023-11-06T20:59:57.303617+0000","last_became_peered":"2023-11-06T20:59:57.303617+0000","last_unstale":"2023-11-06T20:59:57.309497+0000","last_undegraded":"2023-11-06T20:59:57.309497+0000","last_fullsized":"2023-11-06T20:59:57.309497+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.798874+0000","last_change":"2023-11-06T20:59:57.798874+0000","last_active":"2023-11-06T20:59:57.798874+0000","last_peered":"2023-11-06T20:59:57.798874+0000","last_clean":"2023-11-06T20:59:57.798874+0000","last_became_active":"2023-11-06T20:59:57.798094+0000","last_became_peered":"2023-11-06T20:59:57.798094+0000","last_unstale":"2023-11-06T20:59:57.798874+0000","last_undegraded":"2023-11-06T20:59:57.798874+0000","last_fullsized":"2023-11-06T20:59:57.798874+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.34","version":"98'4","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.820776+0000","last_change":"2023-11-06T20:59:57.308642+0000","last_active":"2023-11-06T20:59:57.820776+0000","last_peered":"2023-11-06T20:59:57.820776+0000","last_clean":"2023-11-06T20:59:57.820776+0000","last_became_active":"2023-11-06T20:59:57.308280+0000","last_became_peered":"2023-11-06T20:59:57.308280+0000","last_unstale":"2023-11-06T20:59:57.820776+0000","last_undegraded":"2023-11-06T20:59:57.820776+0000","last_fullsized":"2023-11-06T20:59:57.820776+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.795643+0000","last_change":"2023-11-06T20:59:57.791607+0000","last_active":"2023-11-06T20:59:57.795643+0000","last_peered":"2023-11-06T20:59:57.795643+0000","last_clean":"2023-11-06T20:59:57.795643+0000","last_became_active":"2023-11-06T20:59:57.791324+0000","last_became_peered":"2023-11-06T20:59:57.791324+0000","last_unstale":"2023-11-06T20:59:57.795643+0000","last_undegraded":"2023-11-06T20:59:57.795643+0000","last_fullsized":"2023-11-06T20:59:57.795643+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.33","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.817462+0000","last_change":"2023-11-06T20:59:57.320439+0000","last_active":"2023-11-06T20:59:57.817462+0000","last_peered":"2023-11-06T20:59:57.817462+0000","last_clean":"2023-11-06T20:59:57.817462+0000","last_became_active":"2023-11-06T20:59:57.320151+0000","last_became_peered":"2023-11-06T20:59:57.320151+0000","last_unstale":"2023-11-06T20:59:57.817462+0000","last_undegraded":"2023-11-06T20:59:57.817462+0000","last_fullsized":"2023-11-06T20:59:57.817462+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.821693+0000","last_change":"2023-11-06T20:59:57.821693+0000","last_active":"2023-11-06T20:59:57.821693+0000","last_peered":"2023-11-06T20:59:57.821693+0000","last_clean":"2023-11-06T20:59:57.821693+0000","last_became_active":"2023-11-06T20:59:57.821093+0000","last_became_peered":"2023-11-06T20:59:57.821093+0000","last_unstale":"2023-11-06T20:59:57.821693+0000","last_undegraded":"2023-11-06T20:59:57.821693+0000","last_fullsized":"2023-11-06T20:59:57.821693+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.319759+0000","last_change":"2023-11-06T20:59:57.315519+0000","last_active":"2023-11-06T20:59:57.319759+0000","last_peered":"2023-11-06T20:59:57.319759+0000","last_clean":"2023-11-06T20:59:57.319759+0000","last_became_active":"2023-11-06T20:59:57.314968+0000","last_became_peered":"2023-11-06T20:59:57.314968+0000","last_unstale":"2023-11-06T20:59:57.319759+0000","last_undegraded":"2023-11-06T20:59:57.319759+0000","last_fullsized":"2023-11-06T20:59:57.319759+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.804259+0000","last_change":"2023-11-06T20:59:57.804259+0000","last_active":"2023-11-06T20:59:57.804259+0000","last_peered":"2023-11-06T20:59:57.804259+0000","last_clean":"2023-11-06T20:59:57.804259+0000","last_became_active":"2023-11-06T20:59:57.803976+0000","last_became_peered":"2023-11-06T20:59:57.803976+0000","last_unstale":"2023-11-06T20:59:57.804259+0000","last_undegraded":"2023-11-06T20:59:57.804259+0000","last_fullsized":"2023-11-06T20:59:57.804259+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.319890+0000","last_change":"2023-11-06T20:59:57.314831+0000","last_active":"2023-11-06T20:59:57.319890+0000","last_peered":"2023-11-06T20:59:57.319890+0000","last_clean":"2023-11-06T20:59:57.319890+0000","last_became_active":"2023-11-06T20:59:57.314515+0000","last_became_peered":"2023-11-06T20:59:57.314515+0000","last_unstale":"2023-11-06T20:59:57.319890+0000","last_undegraded":"2023-11-06T20:59:57.319890+0000","last_fullsized":"2023-11-06T20:59:57.319890+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.807118+0000","last_change":"2023-11-06T20:59:57.801898+0000","last_active":"2023-11-06T20:59:57.807118+0000","last_peered":"2023-11-06T20:59:57.807118+0000","last_clean":"2023-11-06T20:59:57.807118+0000","last_became_active":"2023-11-06T20:59:57.801250+0000","last_became_peered":"2023-11-06T20:59:57.801250+0000","last_unstale":"2023-11-06T20:59:57.807118+0000","last_undegraded":"2023-11-06T20:59:57.807118+0000","last_fullsized":"2023-11-06T20:59:57.807118+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"98'4","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.816729+0000","last_change":"2023-11-06T20:59:57.298724+0000","last_active":"2023-11-06T20:59:57.816729+0000","last_peered":"2023-11-06T20:59:57.816729+0000","last_clean":"2023-11-06T20:59:57.816729+0000","last_became_active":"2023-11-06T20:59:57.298410+0000","last_became_peered":"2023-11-06T20:59:57.298410+0000","last_unstale":"2023-11-06T20:59:57.816729+0000","last_undegraded":"2023-11-06T20:59:57.816729+0000","last_fullsized":"2023-11-06T20:59:57.816729+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.800997+0000","last_change":"2023-11-06T20:59:57.800997+0000","last_active":"2023-11-06T20:59:57.800997+0000","last_peered":"2023-11-06T20:59:57.800997+0000","last_clean":"2023-11-06T20:59:57.800997+0000","last_became_active":"2023-11-06T20:59:57.800682+0000","last_became_peered":"2023-11-06T20:59:57.800682+0000","last_unstale":"2023-11-06T20:59:57.800997+0000","last_undegraded":"2023-11-06T20:59:57.800997+0000","last_fullsized":"2023-11-06T20:59:57.800997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785489+0000","last_change":"2023-11-06T20:59:57.310678+0000","last_active":"2023-11-06T20:59:57.785489+0000","last_peered":"2023-11-06T20:59:57.785489+0000","last_clean":"2023-11-06T20:59:57.785489+0000","last_became_active":"2023-11-06T20:59:57.310377+0000","last_became_peered":"2023-11-06T20:59:57.310377+0000","last_unstale":"2023-11-06T20:59:57.785489+0000","last_undegraded":"2023-11-06T20:59:57.785489+0000","last_fullsized":"2023-11-06T20:59:57.785489+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.791236+0000","last_change":"2023-11-06T20:59:57.791236+0000","last_active":"2023-11-06T20:59:57.791236+0000","last_peered":"2023-11-06T20:59:57.791236+0000","last_clean":"2023-11-06T20:59:57.791236+0000","last_became_active":"2023-11-06T20:59:57.790456+0000","last_became_peered":"2023-11-06T20:59:57.790456+0000","last_unstale":"2023-11-06T20:59:57.791236+0000","last_undegraded":"2023-11-06T20:59:57.791236+0000","last_fullsized":"2023-11-06T20:59:57.791236+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.784405+0000","last_change":"2023-11-06T20:59:57.308539+0000","last_active":"2023-11-06T20:59:57.784405+0000","last_peered":"2023-11-06T20:59:57.784405+0000","last_clean":"2023-11-06T20:59:57.784405+0000","last_became_active":"2023-11-06T20:59:57.308272+0000","last_became_peered":"2023-11-06T20:59:57.308272+0000","last_unstale":"2023-11-06T20:59:57.784405+0000","last_undegraded":"2023-11-06T20:59:57.784405+0000","last_fullsized":"2023-11-06T20:59:57.784405+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.803389+0000","last_change":"2023-11-06T20:59:57.803389+0000","last_active":"2023-11-06T20:59:57.803389+0000","last_peered":"2023-11-06T20:59:57.803389+0000","last_clean":"2023-11-06T20:59:57.803389+0000","last_became_active":"2023-11-06T20:59:57.803091+0000","last_became_peered":"2023-11-06T20:59:57.803091+0000","last_unstale":"2023-11-06T20:59:57.803389+0000","last_undegraded":"2023-11-06T20:59:57.803389+0000","last_fullsized":"2023-11-06T20:59:57.803389+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2d","version":"98'1","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.781589+0000","last_change":"2023-11-06T20:59:57.312619+0000","last_active":"2023-11-06T20:59:57.781589+0000","last_peered":"2023-11-06T20:59:57.781589+0000","last_clean":"2023-11-06T20:59:57.781589+0000","last_became_active":"2023-11-06T20:59:57.310604+0000","last_became_peered":"2023-11-06T20:59:57.310604+0000","last_unstale":"2023-11-06T20:59:57.781589+0000","last_undegraded":"2023-11-06T20:59:57.781589+0000","last_fullsized":"2023-11-06T20:59:57.781589+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.821937+0000","last_change":"2023-11-06T20:59:57.821243+0000","last_active":"2023-11-06T20:59:57.821937+0000","last_peered":"2023-11-06T20:59:57.821937+0000","last_clean":"2023-11-06T20:59:57.821937+0000","last_became_active":"2023-11-06T20:59:57.820958+0000","last_became_peered":"2023-11-06T20:59:57.820958+0000","last_unstale":"2023-11-06T20:59:57.821937+0000","last_undegraded":"2023-11-06T20:59:57.821937+0000","last_fullsized":"2023-11-06T20:59:57.821937+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785610+0000","last_change":"2023-11-06T20:59:57.311760+0000","last_active":"2023-11-06T20:59:57.785610+0000","last_peered":"2023-11-06T20:59:57.785610+0000","last_clean":"2023-11-06T20:59:57.785610+0000","last_became_active":"2023-11-06T20:59:57.311495+0000","last_became_peered":"2023-11-06T20:59:57.311495+0000","last_unstale":"2023-11-06T20:59:57.785610+0000","last_undegraded":"2023-11-06T20:59:57.785610+0000","last_fullsized":"2023-11-06T20:59:57.785610+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806345+0000","last_change":"2023-11-06T20:59:57.795623+0000","last_active":"2023-11-06T20:59:57.806345+0000","last_peered":"2023-11-06T20:59:57.806345+0000","last_clean":"2023-11-06T20:59:57.806345+0000","last_became_active":"2023-11-06T20:59:57.794889+0000","last_became_peered":"2023-11-06T20:59:57.794889+0000","last_unstale":"2023-11-06T20:59:57.806345+0000","last_undegraded":"2023-11-06T20:59:57.806345+0000","last_fullsized":"2023-11-06T20:59:57.806345+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819739+0000","last_change":"2023-11-06T20:59:57.315291+0000","last_active":"2023-11-06T20:59:57.819739+0000","last_peered":"2023-11-06T20:59:57.819739+0000","last_clean":"2023-11-06T20:59:57.819739+0000","last_became_active":"2023-11-06T20:59:57.315002+0000","last_became_peered":"2023-11-06T20:59:57.315002+0000","last_unstale":"2023-11-06T20:59:57.819739+0000","last_undegraded":"2023-11-06T20:59:57.819739+0000","last_fullsized":"2023-11-06T20:59:57.819739+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.810834+0000","last_change":"2023-11-06T20:59:57.810834+0000","last_active":"2023-11-06T20:59:57.810834+0000","last_peered":"2023-11-06T20:59:57.810834+0000","last_clean":"2023-11-06T20:59:57.810834+0000","last_became_active":"2023-11-06T20:59:57.810553+0000","last_became_peered":"2023-11-06T20:59:57.810553+0000","last_unstale":"2023-11-06T20:59:57.810834+0000","last_undegraded":"2023-11-06T20:59:57.810834+0000","last_fullsized":"2023-11-06T20:59:57.810834+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785206+0000","last_change":"2023-11-06T20:59:57.317922+0000","last_active":"2023-11-06T20:59:57.785206+0000","last_peered":"2023-11-06T20:59:57.785206+0000","last_clean":"2023-11-06T20:59:57.785206+0000","last_became_active":"2023-11-06T20:59:57.317651+0000","last_became_peered":"2023-11-06T20:59:57.317651+0000","last_unstale":"2023-11-06T20:59:57.785206+0000","last_undegraded":"2023-11-06T20:59:57.785206+0000","last_fullsized":"2023-11-06T20:59:57.785206+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.797407+0000","last_change":"2023-11-06T20:59:57.797407+0000","last_active":"2023-11-06T20:59:57.797407+0000","last_peered":"2023-11-06T20:59:57.797407+0000","last_clean":"2023-11-06T20:59:57.797407+0000","last_became_active":"2023-11-06T20:59:57.796981+0000","last_became_peered":"2023-11-06T20:59:57.796981+0000","last_unstale":"2023-11-06T20:59:57.797407+0000","last_undegraded":"2023-11-06T20:59:57.797407+0000","last_fullsized":"2023-11-06T20:59:57.797407+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.320745+0000","last_change":"2023-11-06T20:59:57.313179+0000","last_active":"2023-11-06T20:59:57.320745+0000","last_peered":"2023-11-06T20:59:57.320745+0000","last_clean":"2023-11-06T20:59:57.320745+0000","last_became_active":"2023-11-06T20:59:57.312900+0000","last_became_peered":"2023-11-06T20:59:57.312900+0000","last_unstale":"2023-11-06T20:59:57.320745+0000","last_undegraded":"2023-11-06T20:59:57.320745+0000","last_fullsized":"2023-11-06T20:59:57.320745+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806033+0000","last_change":"2023-11-06T20:59:57.806033+0000","last_active":"2023-11-06T20:59:57.806033+0000","last_peered":"2023-11-06T20:59:57.806033+0000","last_clean":"2023-11-06T20:59:57.806033+0000","last_became_active":"2023-11-06T20:59:57.804271+0000","last_became_peered":"2023-11-06T20:59:57.804271+0000","last_unstale":"2023-11-06T20:59:57.806033+0000","last_undegraded":"2023-11-06T20:59:57.806033+0000","last_fullsized":"2023-11-06T20:59:57.806033+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.317691+0000","last_change":"2023-11-06T20:59:57.312337+0000","last_active":"2023-11-06T20:59:57.317691+0000","last_peered":"2023-11-06T20:59:57.317691+0000","last_clean":"2023-11-06T20:59:57.317691+0000","last_became_active":"2023-11-06T20:59:57.312044+0000","last_became_peered":"2023-11-06T20:59:57.312044+0000","last_unstale":"2023-11-06T20:59:57.317691+0000","last_undegraded":"2023-11-06T20:59:57.317691+0000","last_fullsized":"2023-11-06T20:59:57.317691+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.802382+0000","last_change":"2023-11-06T20:59:57.795993+0000","last_active":"2023-11-06T20:59:57.802382+0000","last_peered":"2023-11-06T20:59:57.802382+0000","last_clean":"2023-11-06T20:59:57.802382+0000","last_became_active":"2023-11-06T20:59:57.795513+0000","last_became_peered":"2023-11-06T20:59:57.795513+0000","last_unstale":"2023-11-06T20:59:57.802382+0000","last_undegraded":"2023-11-06T20:59:57.802382+0000","last_fullsized":"2023-11-06T20:59:57.802382+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.326335+0000","last_change":"2023-11-06T20:59:57.321522+0000","last_active":"2023-11-06T20:59:57.326335+0000","last_peered":"2023-11-06T20:59:57.326335+0000","last_clean":"2023-11-06T20:59:57.326335+0000","last_became_active":"2023-11-06T20:59:57.321077+0000","last_became_peered":"2023-11-06T20:59:57.321077+0000","last_unstale":"2023-11-06T20:59:57.326335+0000","last_undegraded":"2023-11-06T20:59:57.326335+0000","last_fullsized":"2023-11-06T20:59:57.326335+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.801568+0000","last_change":"2023-11-06T20:59:57.801568+0000","last_active":"2023-11-06T20:59:57.801568+0000","last_peered":"2023-11-06T20:59:57.801568+0000","last_clean":"2023-11-06T20:59:57.801568+0000","last_became_active":"2023-11-06T20:59:57.801290+0000","last_became_peered":"2023-11-06T20:59:57.801290+0000","last_unstale":"2023-11-06T20:59:57.801568+0000","last_undegraded":"2023-11-06T20:59:57.801568+0000","last_fullsized":"2023-11-06T20:59:57.801568+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.318060+0000","last_change":"2023-11-06T20:59:57.313674+0000","last_active":"2023-11-06T20:59:57.318060+0000","last_peered":"2023-11-06T20:59:57.318060+0000","last_clean":"2023-11-06T20:59:57.318060+0000","last_became_active":"2023-11-06T20:59:57.313401+0000","last_became_peered":"2023-11-06T20:59:57.313401+0000","last_unstale":"2023-11-06T20:59:57.318060+0000","last_undegraded":"2023-11-06T20:59:57.318060+0000","last_fullsized":"2023-11-06T20:59:57.318060+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.799278+0000","last_change":"2023-11-06T20:59:57.799278+0000","last_active":"2023-11-06T20:59:57.799278+0000","last_peered":"2023-11-06T20:59:57.799278+0000","last_clean":"2023-11-06T20:59:57.799278+0000","last_became_active":"2023-11-06T20:59:57.797784+0000","last_became_peered":"2023-11-06T20:59:57.797784+0000","last_unstale":"2023-11-06T20:59:57.799278+0000","last_undegraded":"2023-11-06T20:59:57.799278+0000","last_fullsized":"2023-11-06T20:59:57.799278+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.783823+0000","last_change":"2023-11-06T20:59:57.312090+0000","last_active":"2023-11-06T20:59:57.783823+0000","last_peered":"2023-11-06T20:59:57.783823+0000","last_clean":"2023-11-06T20:59:57.783823+0000","last_became_active":"2023-11-06T20:59:57.311780+0000","last_became_peered":"2023-11-06T20:59:57.311780+0000","last_unstale":"2023-11-06T20:59:57.783823+0000","last_undegraded":"2023-11-06T20:59:57.783823+0000","last_fullsized":"2023-11-06T20:59:57.783823+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805610+0000","last_change":"2023-11-06T20:59:57.793640+0000","last_active":"2023-11-06T20:59:57.805610+0000","last_peered":"2023-11-06T20:59:57.805610+0000","last_clean":"2023-11-06T20:59:57.805610+0000","last_became_active":"2023-11-06T20:59:57.793179+0000","last_became_peered":"2023-11-06T20:59:57.793179+0000","last_unstale":"2023-11-06T20:59:57.805610+0000","last_undegraded":"2023-11-06T20:59:57.805610+0000","last_fullsized":"2023-11-06T20:59:57.805610+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.790365+0000","last_change":"2023-11-06T20:59:57.317021+0000","last_active":"2023-11-06T20:59:57.790365+0000","last_peered":"2023-11-06T20:59:57.790365+0000","last_clean":"2023-11-06T20:59:57.790365+0000","last_became_active":"2023-11-06T20:59:57.316296+0000","last_became_peered":"2023-11-06T20:59:57.316296+0000","last_unstale":"2023-11-06T20:59:57.790365+0000","last_undegraded":"2023-11-06T20:59:57.790365+0000","last_fullsized":"2023-11-06T20:59:57.790365+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.810402+0000","last_change":"2023-11-06T20:59:57.807802+0000","last_active":"2023-11-06T20:59:57.810402+0000","last_peered":"2023-11-06T20:59:57.810402+0000","last_clean":"2023-11-06T20:59:57.810402+0000","last_became_active":"2023-11-06T20:59:57.806949+0000","last_became_peered":"2023-11-06T20:59:57.806949+0000","last_unstale":"2023-11-06T20:59:57.810402+0000","last_undegraded":"2023-11-06T20:59:57.810402+0000","last_fullsized":"2023-11-06T20:59:57.810402+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.786352+0000","last_change":"2023-11-06T20:59:57.318055+0000","last_active":"2023-11-06T20:59:57.786352+0000","last_peered":"2023-11-06T20:59:57.786352+0000","last_clean":"2023-11-06T20:59:57.786352+0000","last_became_active":"2023-11-06T20:59:57.317452+0000","last_became_peered":"2023-11-06T20:59:57.317452+0000","last_unstale":"2023-11-06T20:59:57.786352+0000","last_undegraded":"2023-11-06T20:59:57.786352+0000","last_fullsized":"2023-11-06T20:59:57.786352+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.803144+0000","last_change":"2023-11-06T20:59:57.797464+0000","last_active":"2023-11-06T20:59:57.803144+0000","last_peered":"2023-11-06T20:59:57.803144+0000","last_clean":"2023-11-06T20:59:57.803144+0000","last_became_active":"2023-11-06T20:59:57.795198+0000","last_became_peered":"2023-11-06T20:59:57.795198+0000","last_unstale":"2023-11-06T20:59:57.803144+0000","last_undegraded":"2023-11-06T20:59:57.803144+0000","last_fullsized":"2023-11-06T20:59:57.803144+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.314735+0000","last_change":"2023-11-06T20:59:57.309710+0000","last_active":"2023-11-06T20:59:57.314735+0000","last_peered":"2023-11-06T20:59:57.314735+0000","last_clean":"2023-11-06T20:59:57.314735+0000","last_became_active":"2023-11-06T20:59:57.309538+0000","last_became_peered":"2023-11-06T20:59:57.309538+0000","last_unstale":"2023-11-06T20:59:57.314735+0000","last_undegraded":"2023-11-06T20:59:57.314735+0000","last_fullsized":"2023-11-06T20:59:57.314735+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.816352+0000","last_change":"2023-11-06T20:59:57.813446+0000","last_active":"2023-11-06T20:59:57.816352+0000","last_peered":"2023-11-06T20:59:57.816352+0000","last_clean":"2023-11-06T20:59:57.816352+0000","last_became_active":"2023-11-06T20:59:57.813080+0000","last_became_peered":"2023-11-06T20:59:57.813080+0000","last_unstale":"2023-11-06T20:59:57.816352+0000","last_undegraded":"2023-11-06T20:59:57.816352+0000","last_fullsized":"2023-11-06T20:59:57.816352+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.788633+0000","last_change":"2023-11-06T20:59:57.317280+0000","last_active":"2023-11-06T20:59:57.788633+0000","last_peered":"2023-11-06T20:59:57.788633+0000","last_clean":"2023-11-06T20:59:57.788633+0000","last_became_active":"2023-11-06T20:59:57.317021+0000","last_became_peered":"2023-11-06T20:59:57.317021+0000","last_unstale":"2023-11-06T20:59:57.788633+0000","last_undegraded":"2023-11-06T20:59:57.788633+0000","last_fullsized":"2023-11-06T20:59:57.788633+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805958+0000","last_change":"2023-11-06T20:59:57.799180+0000","last_active":"2023-11-06T20:59:57.805958+0000","last_peered":"2023-11-06T20:59:57.805958+0000","last_clean":"2023-11-06T20:59:57.805958+0000","last_became_active":"2023-11-06T20:59:57.798759+0000","last_became_peered":"2023-11-06T20:59:57.798759+0000","last_unstale":"2023-11-06T20:59:57.805958+0000","last_undegraded":"2023-11-06T20:59:57.805958+0000","last_fullsized":"2023-11-06T20:59:57.805958+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.788202+0000","last_change":"2023-11-06T20:59:57.305796+0000","last_active":"2023-11-06T20:59:57.788202+0000","last_peered":"2023-11-06T20:59:57.788202+0000","last_clean":"2023-11-06T20:59:57.788202+0000","last_became_active":"2023-11-06T20:59:57.304989+0000","last_became_peered":"2023-11-06T20:59:57.304989+0000","last_unstale":"2023-11-06T20:59:57.788202+0000","last_undegraded":"2023-11-06T20:59:57.788202+0000","last_fullsized":"2023-11-06T20:59:57.788202+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.809550+0000","last_change":"2023-11-06T20:59:57.805526+0000","last_active":"2023-11-06T20:59:57.809550+0000","last_peered":"2023-11-06T20:59:57.809550+0000","last_clean":"2023-11-06T20:59:57.809550+0000","last_became_active":"2023-11-06T20:59:57.805148+0000","last_became_peered":"2023-11-06T20:59:57.805148+0000","last_unstale":"2023-11-06T20:59:57.809550+0000","last_undegraded":"2023-11-06T20:59:57.809550+0000","last_fullsized":"2023-11-06T20:59:57.809550+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.811669+0000","last_change":"2023-11-06T20:59:57.319917+0000","last_active":"2023-11-06T20:59:57.811669+0000","last_peered":"2023-11-06T20:59:57.811669+0000","last_clean":"2023-11-06T20:59:57.811669+0000","last_became_active":"2023-11-06T20:59:57.318879+0000","last_became_peered":"2023-11-06T20:59:57.318879+0000","last_unstale":"2023-11-06T20:59:57.811669+0000","last_undegraded":"2023-11-06T20:59:57.811669+0000","last_fullsized":"2023-11-06T20:59:57.811669+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.811315+0000","last_change":"2023-11-06T20:59:57.798478+0000","last_active":"2023-11-06T20:59:57.811315+0000","last_peered":"2023-11-06T20:59:57.811315+0000","last_clean":"2023-11-06T20:59:57.811315+0000","last_became_active":"2023-11-06T20:59:57.798157+0000","last_became_peered":"2023-11-06T20:59:57.798157+0000","last_unstale":"2023-11-06T20:59:57.811315+0000","last_undegraded":"2023-11-06T20:59:57.811315+0000","last_fullsized":"2023-11-06T20:59:57.811315+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"98'1","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.781609+0000","last_change":"2023-11-06T20:59:57.298404+0000","last_active":"2023-11-06T20:59:57.781609+0000","last_peered":"2023-11-06T20:59:57.781609+0000","last_clean":"2023-11-06T20:59:57.781609+0000","last_became_active":"2023-11-06T20:59:57.298073+0000","last_became_peered":"2023-11-06T20:59:57.298073+0000","last_unstale":"2023-11-06T20:59:57.781609+0000","last_undegraded":"2023-11-06T20:59:57.781609+0000","last_fullsized":"2023-11-06T20:59:57.781609+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.804631+0000","last_change":"2023-11-06T20:59:57.797490+0000","last_active":"2023-11-06T20:59:57.804631+0000","last_peered":"2023-11-06T20:59:57.804631+0000","last_clean":"2023-11-06T20:59:57.804631+0000","last_became_active":"2023-11-06T20:59:57.797179+0000","last_became_peered":"2023-11-06T20:59:57.797179+0000","last_unstale":"2023-11-06T20:59:57.804631+0000","last_undegraded":"2023-11-06T20:59:57.804631+0000","last_fullsized":"2023-11-06T20:59:57.804631+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"98'2","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.820980+0000","last_change":"2023-11-06T20:59:57.320542+0000","last_active":"2023-11-06T20:59:57.820980+0000","last_peered":"2023-11-06T20:59:57.820980+0000","last_clean":"2023-11-06T20:59:57.820980+0000","last_became_active":"2023-11-06T20:59:57.320214+0000","last_became_peered":"2023-11-06T20:59:57.320214+0000","last_unstale":"2023-11-06T20:59:57.820980+0000","last_undegraded":"2023-11-06T20:59:57.820980+0000","last_fullsized":"2023-11-06T20:59:57.820980+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805039+0000","last_change":"2023-11-06T20:59:57.799950+0000","last_active":"2023-11-06T20:59:57.805039+0000","last_peered":"2023-11-06T20:59:57.805039+0000","last_clean":"2023-11-06T20:59:57.805039+0000","last_became_active":"2023-11-06T20:59:57.799648+0000","last_became_peered":"2023-11-06T20:59:57.799648+0000","last_unstale":"2023-11-06T20:59:57.805039+0000","last_undegraded":"2023-11-06T20:59:57.805039+0000","last_fullsized":"2023-11-06T20:59:57.805039+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785377+0000","last_change":"2023-11-06T20:59:57.312011+0000","last_active":"2023-11-06T20:59:57.785377+0000","last_peered":"2023-11-06T20:59:57.785377+0000","last_clean":"2023-11-06T20:59:57.785377+0000","last_became_active":"2023-11-06T20:59:57.311521+0000","last_became_peered":"2023-11-06T20:59:57.311521+0000","last_unstale":"2023-11-06T20:59:57.785377+0000","last_undegraded":"2023-11-06T20:59:57.785377+0000","last_fullsized":"2023-11-06T20:59:57.785377+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806366+0000","last_change":"2023-11-06T20:59:57.795865+0000","last_active":"2023-11-06T20:59:57.806366+0000","last_peered":"2023-11-06T20:59:57.806366+0000","last_clean":"2023-11-06T20:59:57.806366+0000","last_became_active":"2023-11-06T20:59:57.795242+0000","last_became_peered":"2023-11-06T20:59:57.795242+0000","last_unstale":"2023-11-06T20:59:57.806366+0000","last_undegraded":"2023-11-06T20:59:57.806366+0000","last_fullsized":"2023-11-06T20:59:57.806366+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785261+0000","last_change":"2023-11-06T20:59:57.309685+0000","last_active":"2023-11-06T20:59:57.785261+0000","last_peered":"2023-11-06T20:59:57.785261+0000","last_clean":"2023-11-06T20:59:57.785261+0000","last_became_active":"2023-11-06T20:59:57.308807+0000","last_became_peered":"2023-11-06T20:59:57.308807+0000","last_unstale":"2023-11-06T20:59:57.785261+0000","last_undegraded":"2023-11-06T20:59:57.785261+0000","last_fullsized":"2023-11-06T20:59:57.785261+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.798182+0000","last_change":"2023-11-06T20:59:57.798182+0000","last_active":"2023-11-06T20:59:57.798182+0000","last_peered":"2023-11-06T20:59:57.798182+0000","last_clean":"2023-11-06T20:59:57.798182+0000","last_became_active":"2023-11-06T20:59:57.797862+0000","last_became_peered":"2023-11-06T20:59:57.797862+0000","last_unstale":"2023-11-06T20:59:57.798182+0000","last_undegraded":"2023-11-06T20:59:57.798182+0000","last_fullsized":"2023-11-06T20:59:57.798182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.319323+0000","last_change":"2023-11-06T20:59:57.315832+0000","last_active":"2023-11-06T20:59:57.319323+0000","last_peered":"2023-11-06T20:59:57.319323+0000","last_clean":"2023-11-06T20:59:57.319323+0000","last_became_active":"2023-11-06T20:59:57.315524+0000","last_became_peered":"2023-11-06T20:59:57.315524+0000","last_unstale":"2023-11-06T20:59:57.319323+0000","last_undegraded":"2023-11-06T20:59:57.319323+0000","last_fullsized":"2023-11-06T20:59:57.319323+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805497+0000","last_change":"2023-11-06T20:59:57.801297+0000","last_active":"2023-11-06T20:59:57.805497+0000","last_peered":"2023-11-06T20:59:57.805497+0000","last_clean":"2023-11-06T20:59:57.805497+0000","last_became_active":"2023-11-06T20:59:57.800996+0000","last_became_peered":"2023-11-06T20:59:57.800996+0000","last_unstale":"2023-11-06T20:59:57.805497+0000","last_undegraded":"2023-11-06T20:59:57.805497+0000","last_fullsized":"2023-11-06T20:59:57.805497+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.314053+0000","last_change":"2023-11-06T20:59:57.303473+0000","last_active":"2023-11-06T20:59:57.314053+0000","last_peered":"2023-11-06T20:59:57.314053+0000","last_clean":"2023-11-06T20:59:57.314053+0000","last_became_active":"2023-11-06T20:59:57.303125+0000","last_became_peered":"2023-11-06T20:59:57.303125+0000","last_unstale":"2023-11-06T20:59:57.314053+0000","last_undegraded":"2023-11-06T20:59:57.314053+0000","last_fullsized":"2023-11-06T20:59:57.314053+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.800150+0000","last_change":"2023-11-06T20:59:57.800150+0000","last_active":"2023-11-06T20:59:57.800150+0000","last_peered":"2023-11-06T20:59:57.800150+0000","last_clean":"2023-11-06T20:59:57.800150+0000","last_became_active":"2023-11-06T20:59:57.799731+0000","last_became_peered":"2023-11-06T20:59:57.799731+0000","last_unstale":"2023-11-06T20:59:57.800150+0000","last_undegraded":"2023-11-06T20:59:57.800150+0000","last_fullsized":"2023-11-06T20:59:57.800150+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.324932+0000","last_change":"2023-11-06T20:59:57.323634+0000","last_active":"2023-11-06T20:59:57.324932+0000","last_peered":"2023-11-06T20:59:57.324932+0000","last_clean":"2023-11-06T20:59:57.324932+0000","last_became_active":"2023-11-06T20:59:57.323435+0000","last_became_peered":"2023-11-06T20:59:57.323435+0000","last_unstale":"2023-11-06T20:59:57.324932+0000","last_undegraded":"2023-11-06T20:59:57.324932+0000","last_fullsized":"2023-11-06T20:59:57.324932+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.797257+0000","last_change":"2023-11-06T20:59:57.797257+0000","last_active":"2023-11-06T20:59:57.797257+0000","last_peered":"2023-11-06T20:59:57.797257+0000","last_clean":"2023-11-06T20:59:57.797257+0000","last_became_active":"2023-11-06T20:59:57.794748+0000","last_became_peered":"2023-11-06T20:59:57.794748+0000","last_unstale":"2023-11-06T20:59:57.797257+0000","last_undegraded":"2023-11-06T20:59:57.797257+0000","last_fullsized":"2023-11-06T20:59:57.797257+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.791100+0000","last_change":"2023-11-06T20:59:57.320281+0000","last_active":"2023-11-06T20:59:57.791100+0000","last_peered":"2023-11-06T20:59:57.791100+0000","last_clean":"2023-11-06T20:59:57.791100+0000","last_became_active":"2023-11-06T20:59:57.319958+0000","last_became_peered":"2023-11-06T20:59:57.319958+0000","last_unstale":"2023-11-06T20:59:57.791100+0000","last_undegraded":"2023-11-06T20:59:57.791100+0000","last_fullsized":"2023-11-06T20:59:57.791100+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.800905+0000","last_change":"2023-11-06T20:59:57.791592+0000","last_active":"2023-11-06T20:59:57.800905+0000","last_peered":"2023-11-06T20:59:57.800905+0000","last_clean":"2023-11-06T20:59:57.800905+0000","last_became_active":"2023-11-06T20:59:57.791292+0000","last_became_peered":"2023-11-06T20:59:57.791292+0000","last_unstale":"2023-11-06T20:59:57.800905+0000","last_undegraded":"2023-11-06T20:59:57.800905+0000","last_fullsized":"2023-11-06T20:59:57.800905+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.319658+0000","last_change":"2023-11-06T20:59:57.314975+0000","last_active":"2023-11-06T20:59:57.319658+0000","last_peered":"2023-11-06T20:59:57.319658+0000","last_clean":"2023-11-06T20:59:57.319658+0000","last_became_active":"2023-11-06T20:59:57.311193+0000","last_became_peered":"2023-11-06T20:59:57.311193+0000","last_unstale":"2023-11-06T20:59:57.319658+0000","last_undegraded":"2023-11-06T20:59:57.319658+0000","last_fullsized":"2023-11-06T20:59:57.319658+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.802599+0000","last_change":"2023-11-06T20:59:57.802599+0000","last_active":"2023-11-06T20:59:57.802599+0000","last_peered":"2023-11-06T20:59:57.802599+0000","last_clean":"2023-11-06T20:59:57.802599+0000","last_became_active":"2023-11-06T20:59:57.802210+0000","last_became_peered":"2023-11-06T20:59:57.802210+0000","last_unstale":"2023-11-06T20:59:57.802599+0000","last_undegraded":"2023-11-06T20:59:57.802599+0000","last_fullsized":"2023-11-06T20:59:57.802599+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"98'3","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.813980+0000","last_change":"2023-11-06T20:59:57.311675+0000","last_active":"2023-11-06T20:59:57.813980+0000","last_peered":"2023-11-06T20:59:57.813980+0000","last_clean":"2023-11-06T20:59:57.813980+0000","last_became_active":"2023-11-06T20:59:57.311341+0000","last_became_peered":"2023-11-06T20:59:57.311341+0000","last_unstale":"2023-11-06T20:59:57.813980+0000","last_undegraded":"2023-11-06T20:59:57.813980+0000","last_fullsized":"2023-11-06T20:59:57.813980+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.820356+0000","last_change":"2023-11-06T20:59:57.819029+0000","last_active":"2023-11-06T20:59:57.820356+0000","last_peered":"2023-11-06T20:59:57.820356+0000","last_clean":"2023-11-06T20:59:57.820356+0000","last_became_active":"2023-11-06T20:59:57.818667+0000","last_became_peered":"2023-11-06T20:59:57.818667+0000","last_unstale":"2023-11-06T20:59:57.820356+0000","last_undegraded":"2023-11-06T20:59:57.820356+0000","last_fullsized":"2023-11-06T20:59:57.820356+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"98'2","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.816742+0000","last_change":"2023-11-06T20:59:57.314129+0000","last_active":"2023-11-06T20:59:57.816742+0000","last_peered":"2023-11-06T20:59:57.816742+0000","last_clean":"2023-11-06T20:59:57.816742+0000","last_became_active":"2023-11-06T20:59:57.313590+0000","last_became_peered":"2023-11-06T20:59:57.313590+0000","last_unstale":"2023-11-06T20:59:57.816742+0000","last_undegraded":"2023-11-06T20:59:57.816742+0000","last_fullsized":"2023-11-06T20:59:57.816742+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806972+0000","last_change":"2023-11-06T20:59:57.797473+0000","last_active":"2023-11-06T20:59:57.806972+0000","last_peered":"2023-11-06T20:59:57.806972+0000","last_clean":"2023-11-06T20:59:57.806972+0000","last_became_active":"2023-11-06T20:59:57.797206+0000","last_became_peered":"2023-11-06T20:59:57.797206+0000","last_unstale":"2023-11-06T20:59:57.806972+0000","last_undegraded":"2023-11-06T20:59:57.806972+0000","last_fullsized":"2023-11-06T20:59:57.806972+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"98'1","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.783641+0000","last_change":"2023-11-06T20:59:57.320406+0000","last_active":"2023-11-06T20:59:57.783641+0000","last_peered":"2023-11-06T20:59:57.783641+0000","last_clean":"2023-11-06T20:59:57.783641+0000","last_became_active":"2023-11-06T20:59:57.319539+0000","last_became_peered":"2023-11-06T20:59:57.319539+0000","last_unstale":"2023-11-06T20:59:57.783641+0000","last_undegraded":"2023-11-06T20:59:57.783641+0000","last_fullsized":"2023-11-06T20:59:57.783641+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.798445+0000","last_change":"2023-11-06T20:59:57.798445+0000","last_active":"2023-11-06T20:59:57.798445+0000","last_peered":"2023-11-06T20:59:57.798445+0000","last_clean":"2023-11-06T20:59:57.798445+0000","last_became_active":"2023-11-06T20:59:57.797706+0000","last_became_peered":"2023-11-06T20:59:57.797706+0000","last_unstale":"2023-11-06T20:59:57.798445+0000","last_undegraded":"2023-11-06T20:59:57.798445+0000","last_fullsized":"2023-11-06T20:59:57.798445+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785085+0000","last_change":"2023-11-06T20:59:57.317838+0000","last_active":"2023-11-06T20:59:57.785085+0000","last_peered":"2023-11-06T20:59:57.785085+0000","last_clean":"2023-11-06T20:59:57.785085+0000","last_became_active":"2023-11-06T20:59:57.316851+0000","last_became_peered":"2023-11-06T20:59:57.316851+0000","last_unstale":"2023-11-06T20:59:57.785085+0000","last_undegraded":"2023-11-06T20:59:57.785085+0000","last_fullsized":"2023-11-06T20:59:57.785085+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.803856+0000","last_change":"2023-11-06T20:59:57.795416+0000","last_active":"2023-11-06T20:59:57.803856+0000","last_peered":"2023-11-06T20:59:57.803856+0000","last_clean":"2023-11-06T20:59:57.803856+0000","last_became_active":"2023-11-06T20:59:57.795066+0000","last_became_peered":"2023-11-06T20:59:57.795066+0000","last_unstale":"2023-11-06T20:59:57.803856+0000","last_undegraded":"2023-11-06T20:59:57.803856+0000","last_fullsized":"2023-11-06T20:59:57.803856+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.787611+0000","last_change":"2023-11-06T20:59:57.319453+0000","last_active":"2023-11-06T20:59:57.787611+0000","last_peered":"2023-11-06T20:59:57.787611+0000","last_clean":"2023-11-06T20:59:57.787611+0000","last_became_active":"2023-11-06T20:59:57.318820+0000","last_became_peered":"2023-11-06T20:59:57.318820+0000","last_unstale":"2023-11-06T20:59:57.787611+0000","last_undegraded":"2023-11-06T20:59:57.787611+0000","last_fullsized":"2023-11-06T20:59:57.787611+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.806244+0000","last_change":"2023-11-06T20:59:57.806244+0000","last_active":"2023-11-06T20:59:57.806244+0000","last_peered":"2023-11-06T20:59:57.806244+0000","last_clean":"2023-11-06T20:59:57.806244+0000","last_became_active":"2023-11-06T20:59:57.805915+0000","last_became_peered":"2023-11-06T20:59:57.805915+0000","last_unstale":"2023-11-06T20:59:57.806244+0000","last_undegraded":"2023-11-06T20:59:57.806244+0000","last_fullsized":"2023-11-06T20:59:57.806244+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"98'4","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.818562+0000","last_change":"2023-11-06T20:59:57.308497+0000","last_active":"2023-11-06T20:59:57.818562+0000","last_peered":"2023-11-06T20:59:57.818562+0000","last_clean":"2023-11-06T20:59:57.818562+0000","last_became_active":"2023-11-06T20:59:57.307686+0000","last_became_peered":"2023-11-06T20:59:57.307686+0000","last_unstale":"2023-11-06T20:59:57.818562+0000","last_undegraded":"2023-11-06T20:59:57.818562+0000","last_fullsized":"2023-11-06T20:59:57.818562+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.802363+0000","last_change":"2023-11-06T20:59:57.795803+0000","last_active":"2023-11-06T20:59:57.802363+0000","last_peered":"2023-11-06T20:59:57.802363+0000","last_clean":"2023-11-06T20:59:57.802363+0000","last_became_active":"2023-11-06T20:59:57.795109+0000","last_became_peered":"2023-11-06T20:59:57.795109+0000","last_unstale":"2023-11-06T20:59:57.802363+0000","last_undegraded":"2023-11-06T20:59:57.802363+0000","last_fullsized":"2023-11-06T20:59:57.802363+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"98'2","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.815514+0000","last_change":"2023-11-06T20:59:57.319967+0000","last_active":"2023-11-06T20:59:57.815514+0000","last_peered":"2023-11-06T20:59:57.815514+0000","last_clean":"2023-11-06T20:59:57.815514+0000","last_became_active":"2023-11-06T20:59:57.319682+0000","last_became_peered":"2023-11-06T20:59:57.319682+0000","last_unstale":"2023-11-06T20:59:57.815514+0000","last_undegraded":"2023-11-06T20:59:57.815514+0000","last_fullsized":"2023-11-06T20:59:57.815514+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.799334+0000","last_change":"2023-11-06T20:59:57.799334+0000","last_active":"2023-11-06T20:59:57.799334+0000","last_peered":"2023-11-06T20:59:57.799334+0000","last_clean":"2023-11-06T20:59:57.799334+0000","last_became_active":"2023-11-06T20:59:57.798970+0000","last_became_peered":"2023-11-06T20:59:57.798970+0000","last_unstale":"2023-11-06T20:59:57.799334+0000","last_undegraded":"2023-11-06T20:59:57.799334+0000","last_fullsized":"2023-11-06T20:59:57.799334+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.784519+0000","last_change":"2023-11-06T20:59:57.311249+0000","last_active":"2023-11-06T20:59:57.784519+0000","last_peered":"2023-11-06T20:59:57.784519+0000","last_clean":"2023-11-06T20:59:57.784519+0000","last_became_active":"2023-11-06T20:59:57.310494+0000","last_became_peered":"2023-11-06T20:59:57.310494+0000","last_unstale":"2023-11-06T20:59:57.784519+0000","last_undegraded":"2023-11-06T20:59:57.784519+0000","last_fullsized":"2023-11-06T20:59:57.784519+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.804651+0000","last_change":"2023-11-06T20:59:57.797850+0000","last_active":"2023-11-06T20:59:57.804651+0000","last_peered":"2023-11-06T20:59:57.804651+0000","last_clean":"2023-11-06T20:59:57.804651+0000","last_became_active":"2023-11-06T20:59:57.797541+0000","last_became_peered":"2023-11-06T20:59:57.797541+0000","last_unstale":"2023-11-06T20:59:57.804651+0000","last_undegraded":"2023-11-06T20:59:57.804651+0000","last_fullsized":"2023-11-06T20:59:57.804651+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.323507+0000","last_change":"2023-11-06T20:59:57.318333+0000","last_active":"2023-11-06T20:59:57.323507+0000","last_peered":"2023-11-06T20:59:57.323507+0000","last_clean":"2023-11-06T20:59:57.323507+0000","last_became_active":"2023-11-06T20:59:57.318074+0000","last_became_peered":"2023-11-06T20:59:57.318074+0000","last_unstale":"2023-11-06T20:59:57.323507+0000","last_undegraded":"2023-11-06T20:59:57.323507+0000","last_fullsized":"2023-11-06T20:59:57.323507+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.815578+0000","last_change":"2023-11-06T20:59:57.815578+0000","last_active":"2023-11-06T20:59:57.815578+0000","last_peered":"2023-11-06T20:59:57.815578+0000","last_clean":"2023-11-06T20:59:57.815578+0000","last_became_active":"2023-11-06T20:59:57.815299+0000","last_became_peered":"2023-11-06T20:59:57.815299+0000","last_unstale":"2023-11-06T20:59:57.815578+0000","last_undegraded":"2023-11-06T20:59:57.815578+0000","last_fullsized":"2023-11-06T20:59:57.815578+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.790461+0000","last_change":"2023-11-06T20:59:57.312716+0000","last_active":"2023-11-06T20:59:57.790461+0000","last_peered":"2023-11-06T20:59:57.790461+0000","last_clean":"2023-11-06T20:59:57.790461+0000","last_became_active":"2023-11-06T20:59:57.312396+0000","last_became_peered":"2023-11-06T20:59:57.312396+0000","last_unstale":"2023-11-06T20:59:57.790461+0000","last_undegraded":"2023-11-06T20:59:57.790461+0000","last_fullsized":"2023-11-06T20:59:57.790461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.805535+0000","last_change":"2023-11-06T20:59:57.800920+0000","last_active":"2023-11-06T20:59:57.805535+0000","last_peered":"2023-11-06T20:59:57.805535+0000","last_clean":"2023-11-06T20:59:57.805535+0000","last_became_active":"2023-11-06T20:59:57.800611+0000","last_became_peered":"2023-11-06T20:59:57.800611+0000","last_unstale":"2023-11-06T20:59:57.805535+0000","last_undegraded":"2023-11-06T20:59:57.805535+0000","last_fullsized":"2023-11-06T20:59:57.805535+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.301905+0000","last_change":"2023-11-06T20:59:57.299084+0000","last_active":"2023-11-06T20:59:57.301905+0000","last_peered":"2023-11-06T20:59:57.301905+0000","last_clean":"2023-11-06T20:59:57.301905+0000","last_became_active":"2023-11-06T20:59:57.298811+0000","last_became_peered":"2023-11-06T20:59:57.298811+0000","last_unstale":"2023-11-06T20:59:57.301905+0000","last_undegraded":"2023-11-06T20:59:57.301905+0000","last_fullsized":"2023-11-06T20:59:57.301905+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.807366+0000","last_change":"2023-11-06T20:59:57.803388+0000","last_active":"2023-11-06T20:59:57.807366+0000","last_peered":"2023-11-06T20:59:57.807366+0000","last_clean":"2023-11-06T20:59:57.807366+0000","last_became_active":"2023-11-06T20:59:57.803107+0000","last_became_peered":"2023-11-06T20:59:57.803107+0000","last_unstale":"2023-11-06T20:59:57.807366+0000","last_undegraded":"2023-11-06T20:59:57.807366+0000","last_fullsized":"2023-11-06T20:59:57.807366+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.783942+0000","last_change":"2023-11-06T20:59:57.310736+0000","last_active":"2023-11-06T20:59:57.783942+0000","last_peered":"2023-11-06T20:59:57.783942+0000","last_clean":"2023-11-06T20:59:57.783942+0000","last_became_active":"2023-11-06T20:59:57.310474+0000","last_became_peered":"2023-11-06T20:59:57.310474+0000","last_unstale":"2023-11-06T20:59:57.783942+0000","last_undegraded":"2023-11-06T20:59:57.783942+0000","last_fullsized":"2023-11-06T20:59:57.783942+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.814071+0000","last_change":"2023-11-06T20:59:57.814071+0000","last_active":"2023-11-06T20:59:57.814071+0000","last_peered":"2023-11-06T20:59:57.814071+0000","last_clean":"2023-11-06T20:59:57.814071+0000","last_became_active":"2023-11-06T20:59:57.813715+0000","last_became_peered":"2023-11-06T20:59:57.813715+0000","last_unstale":"2023-11-06T20:59:57.814071+0000","last_undegraded":"2023-11-06T20:59:57.814071+0000","last_fullsized":"2023-11-06T20:59:57.814071+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.788540+0000","last_change":"2023-11-06T20:59:57.304789+0000","last_active":"2023-11-06T20:59:57.788540+0000","last_peered":"2023-11-06T20:59:57.788540+0000","last_clean":"2023-11-06T20:59:57.788540+0000","last_became_active":"2023-11-06T20:59:57.304393+0000","last_became_peered":"2023-11-06T20:59:57.304393+0000","last_unstale":"2023-11-06T20:59:57.788540+0000","last_undegraded":"2023-11-06T20:59:57.788540+0000","last_fullsized":"2023-11-06T20:59:57.788540+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.812492+0000","last_change":"2023-11-06T20:59:57.808207+0000","last_active":"2023-11-06T20:59:57.812492+0000","last_peered":"2023-11-06T20:59:57.812492+0000","last_clean":"2023-11-06T20:59:57.812492+0000","last_became_active":"2023-11-06T20:59:57.807348+0000","last_became_peered":"2023-11-06T20:59:57.807348+0000","last_unstale":"2023-11-06T20:59:57.812492+0000","last_undegraded":"2023-11-06T20:59:57.812492+0000","last_fullsized":"2023-11-06T20:59:57.812492+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.319544+0000","last_change":"2023-11-06T20:59:57.313834+0000","last_active":"2023-11-06T20:59:57.319544+0000","last_peered":"2023-11-06T20:59:57.319544+0000","last_clean":"2023-11-06T20:59:57.319544+0000","last_became_active":"2023-11-06T20:59:57.313575+0000","last_became_peered":"2023-11-06T20:59:57.313575+0000","last_unstale":"2023-11-06T20:59:57.319544+0000","last_undegraded":"2023-11-06T20:59:57.319544+0000","last_fullsized":"2023-11-06T20:59:57.319544+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.814348+0000","last_change":"2023-11-06T20:59:57.814348+0000","last_active":"2023-11-06T20:59:57.814348+0000","last_peered":"2023-11-06T20:59:57.814348+0000","last_clean":"2023-11-06T20:59:57.814348+0000","last_became_active":"2023-11-06T20:59:57.814112+0000","last_became_peered":"2023-11-06T20:59:57.814112+0000","last_unstale":"2023-11-06T20:59:57.814348+0000","last_undegraded":"2023-11-06T20:59:57.814348+0000","last_fullsized":"2023-11-06T20:59:57.814348+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.791594+0000","last_change":"2023-11-06T20:59:57.317448+0000","last_active":"2023-11-06T20:59:57.791594+0000","last_peered":"2023-11-06T20:59:57.791594+0000","last_clean":"2023-11-06T20:59:57.791594+0000","last_became_active":"2023-11-06T20:59:57.316629+0000","last_became_peered":"2023-11-06T20:59:57.316629+0000","last_unstale":"2023-11-06T20:59:57.791594+0000","last_undegraded":"2023-11-06T20:59:57.791594+0000","last_fullsized":"2023-11-06T20:59:57.791594+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.785695+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T20:59:57.785695+0000","last_peered":"2023-11-06T20:59:57.785695+0000","last_clean":"2023-11-06T20:59:57.785695+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T20:59:57.785695+0000","last_undegraded":"2023-11-06T20:59:57.785695+0000","last_fullsized":"2023-11-06T20:59:57.785695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819962+0000","last_change":"2023-11-06T20:59:57.811834+0000","last_active":"2023-11-06T20:59:57.819962+0000","last_peered":"2023-11-06T20:59:57.819962+0000","last_clean":"2023-11-06T20:59:57.819962+0000","last_became_active":"2023-11-06T20:59:57.811563+0000","last_became_peered":"2023-11-06T20:59:57.811563+0000","last_unstale":"2023-11-06T20:59:57.819962+0000","last_undegraded":"2023-11-06T20:59:57.819962+0000","last_fullsized":"2023-11-06T20:59:57.819962+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"98'1","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.788723+0000","last_change":"2023-11-06T20:59:57.316476+0000","last_active":"2023-11-06T20:59:57.788723+0000","last_peered":"2023-11-06T20:59:57.788723+0000","last_clean":"2023-11-06T20:59:57.788723+0000","last_became_active":"2023-11-06T20:59:57.315964+0000","last_became_peered":"2023-11-06T20:59:57.315964+0000","last_unstale":"2023-11-06T20:59:57.788723+0000","last_undegraded":"2023-11-06T20:59:57.788723+0000","last_fullsized":"2023-11-06T20:59:57.788723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.286986+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T20:59:57.286986+0000","last_peered":"2023-11-06T20:59:57.286986+0000","last_clean":"2023-11-06T20:59:57.286986+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T20:59:57.286986+0000","last_undegraded":"2023-11-06T20:59:57.286986+0000","last_fullsized":"2023-11-06T20:59:57.286986+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.797054+0000","last_change":"2023-11-06T20:59:57.797054+0000","last_active":"2023-11-06T20:59:57.797054+0000","last_peered":"2023-11-06T20:59:57.797054+0000","last_clean":"2023-11-06T20:59:57.797054+0000","last_became_active":"2023-11-06T20:59:57.796476+0000","last_became_peered":"2023-11-06T20:59:57.796476+0000","last_unstale":"2023-11-06T20:59:57.797054+0000","last_undegraded":"2023-11-06T20:59:57.797054+0000","last_fullsized":"2023-11-06T20:59:57.797054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.314743+0000","last_change":"2023-11-06T20:59:57.303556+0000","last_active":"2023-11-06T20:59:57.314743+0000","last_peered":"2023-11-06T20:59:57.314743+0000","last_clean":"2023-11-06T20:59:57.314743+0000","last_became_active":"2023-11-06T20:59:57.303267+0000","last_became_peered":"2023-11-06T20:59:57.303267+0000","last_unstale":"2023-11-06T20:59:57.314743+0000","last_undegraded":"2023-11-06T20:59:57.314743+0000","last_fullsized":"2023-11-06T20:59:57.314743+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.787693+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T20:59:57.787693+0000","last_peered":"2023-11-06T20:59:57.787693+0000","last_clean":"2023-11-06T20:59:57.787693+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T20:59:57.787693+0000","last_undegraded":"2023-11-06T20:59:57.787693+0000","last_fullsized":"2023-11-06T20:59:57.787693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.794237+0000","last_change":"2023-11-06T20:59:57.794237+0000","last_active":"2023-11-06T20:59:57.794237+0000","last_peered":"2023-11-06T20:59:57.794237+0000","last_clean":"2023-11-06T20:59:57.794237+0000","last_became_active":"2023-11-06T20:59:57.793983+0000","last_became_peered":"2023-11-06T20:59:57.793983+0000","last_unstale":"2023-11-06T20:59:57.794237+0000","last_undegraded":"2023-11-06T20:59:57.794237+0000","last_fullsized":"2023-11-06T20:59:57.794237+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.783825+0000","last_change":"2023-11-06T20:59:57.295140+0000","last_active":"2023-11-06T20:59:57.783825+0000","last_peered":"2023-11-06T20:59:57.783825+0000","last_clean":"2023-11-06T20:59:57.783825+0000","last_became_active":"2023-11-06T20:59:57.294744+0000","last_became_peered":"2023-11-06T20:59:57.294744+0000","last_unstale":"2023-11-06T20:59:57.783825+0000","last_undegraded":"2023-11-06T20:59:57.783825+0000","last_fullsized":"2023-11-06T20:59:57.783825+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.286620+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T20:59:57.286620+0000","last_peered":"2023-11-06T20:59:57.286620+0000","last_clean":"2023-11-06T20:59:57.286620+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T20:59:57.286620+0000","last_undegraded":"2023-11-06T20:59:57.286620+0000","last_fullsized":"2023-11-06T20:59:57.286620+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.300712+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T20:59:57.300712+0000","last_peered":"2023-11-06T20:59:57.300712+0000","last_clean":"2023-11-06T20:59:57.300712+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T20:59:57.300712+0000","last_undegraded":"2023-11-06T20:59:57.300712+0000","last_fullsized":"2023-11-06T20:59:57.300712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.809483+0000","last_change":"2023-11-06T20:59:57.804489+0000","last_active":"2023-11-06T20:59:57.809483+0000","last_peered":"2023-11-06T20:59:57.809483+0000","last_clean":"2023-11-06T20:59:57.809483+0000","last_became_active":"2023-11-06T20:59:57.804234+0000","last_became_peered":"2023-11-06T20:59:57.804234+0000","last_unstale":"2023-11-06T20:59:57.809483+0000","last_undegraded":"2023-11-06T20:59:57.809483+0000","last_fullsized":"2023-11-06T20:59:57.809483+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.318638+0000","last_change":"2023-11-06T20:59:57.312336+0000","last_active":"2023-11-06T20:59:57.318638+0000","last_peered":"2023-11-06T20:59:57.318638+0000","last_clean":"2023-11-06T20:59:57.318638+0000","last_became_active":"2023-11-06T20:59:57.311957+0000","last_became_peered":"2023-11-06T20:59:57.311957+0000","last_unstale":"2023-11-06T20:59:57.318638+0000","last_undegraded":"2023-11-06T20:59:57.318638+0000","last_fullsized":"2023-11-06T20:59:57.318638+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.287092+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T20:59:57.287092+0000","last_peered":"2023-11-06T20:59:57.287092+0000","last_clean":"2023-11-06T20:59:57.287092+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T20:59:57.287092+0000","last_undegraded":"2023-11-06T20:59:57.287092+0000","last_fullsized":"2023-11-06T20:59:57.287092+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819677+0000","last_change":"2023-11-06T20:59:57.817627+0000","last_active":"2023-11-06T20:59:57.819677+0000","last_peered":"2023-11-06T20:59:57.819677+0000","last_clean":"2023-11-06T20:59:57.819677+0000","last_became_active":"2023-11-06T20:59:57.817354+0000","last_became_peered":"2023-11-06T20:59:57.817354+0000","last_unstale":"2023-11-06T20:59:57.819677+0000","last_undegraded":"2023-11-06T20:59:57.819677+0000","last_fullsized":"2023-11-06T20:59:57.819677+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.790630+0000","last_change":"2023-11-06T20:59:57.316591+0000","last_active":"2023-11-06T20:59:57.790630+0000","last_peered":"2023-11-06T20:59:57.790630+0000","last_clean":"2023-11-06T20:59:57.790630+0000","last_became_active":"2023-11-06T20:59:57.316273+0000","last_became_peered":"2023-11-06T20:59:57.316273+0000","last_unstale":"2023-11-06T20:59:57.790630+0000","last_undegraded":"2023-11-06T20:59:57.790630+0000","last_fullsized":"2023-11-06T20:59:57.790630+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.288394+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T20:59:57.288394+0000","last_peered":"2023-11-06T20:59:57.288394+0000","last_clean":"2023-11-06T20:59:57.288394+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T20:59:57.288394+0000","last_undegraded":"2023-11-06T20:59:57.288394+0000","last_fullsized":"2023-11-06T20:59:57.288394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.809403+0000","last_change":"2023-11-06T20:59:57.803221+0000","last_active":"2023-11-06T20:59:57.809403+0000","last_peered":"2023-11-06T20:59:57.809403+0000","last_clean":"2023-11-06T20:59:57.809403+0000","last_became_active":"2023-11-06T20:59:57.802937+0000","last_became_peered":"2023-11-06T20:59:57.802937+0000","last_unstale":"2023-11-06T20:59:57.809403+0000","last_undegraded":"2023-11-06T20:59:57.809403+0000","last_fullsized":"2023-11-06T20:59:57.809403+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.817814+0000","last_change":"2023-11-06T20:59:57.308516+0000","last_active":"2023-11-06T20:59:57.817814+0000","last_peered":"2023-11-06T20:59:57.817814+0000","last_clean":"2023-11-06T20:59:57.817814+0000","last_became_active":"2023-11-06T20:59:57.308234+0000","last_became_peered":"2023-11-06T20:59:57.308234+0000","last_unstale":"2023-11-06T20:59:57.817814+0000","last_undegraded":"2023-11-06T20:59:57.817814+0000","last_fullsized":"2023-11-06T20:59:57.817814+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":78,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.303006+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T20:59:57.303006+0000","last_peered":"2023-11-06T20:59:57.303006+0000","last_clean":"2023-11-06T20:59:57.303006+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T20:59:57.303006+0000","last_undegraded":"2023-11-06T20:59:57.303006+0000","last_fullsized":"2023-11-06T20:59:57.303006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.793078+0000","last_change":"2023-11-06T20:59:57.793078+0000","last_active":"2023-11-06T20:59:57.793078+0000","last_peered":"2023-11-06T20:59:57.793078+0000","last_clean":"2023-11-06T20:59:57.793078+0000","last_became_active":"2023-11-06T20:59:57.792772+0000","last_became_peered":"2023-11-06T20:59:57.792772+0000","last_unstale":"2023-11-06T20:59:57.793078+0000","last_undegraded":"2023-11-06T20:59:57.793078+0000","last_fullsized":"2023-11-06T20:59:57.793078+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:57.278563+0000","last_clean_scrub_stamp":"2023-11-06T20:59:57.278563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.819254+0000","last_change":"2023-11-06T20:59:57.301686+0000","last_active":"2023-11-06T20:59:57.819254+0000","last_peered":"2023-11-06T20:59:57.819254+0000","last_clean":"2023-11-06T20:59:57.819254+0000","last_became_active":"2023-11-06T20:59:57.301412+0000","last_became_peered":"2023-11-06T20:59:57.301412+0000","last_unstale":"2023-11-06T20:59:57.819254+0000","last_undegraded":"2023-11-06T20:59:57.819254+0000","last_fullsized":"2023-11-06T20:59:57.819254+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:59:56.267965+0000","last_clean_scrub_stamp":"2023-11-06T20:59:56.267965+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T20:59:57.288014+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T20:59:57.288014+0000","last_peered":"2023-11-06T20:59:57.288014+0000","last_clean":"2023-11-06T20:59:57.288014+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T20:59:57.288014+0000","last_undegraded":"2023-11-06T20:59:57.288014+0000","last_fullsized":"2023-11-06T20:59:57.288014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575039,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":1196032,"data_stored":600371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.68500000000000005,"15min":0.69299999999999995},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.91300000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66900000000000004,"15min":0.69099999999999995},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.46800000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67000000000000004,"15min":0.70499999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40500000000000003},"max":{"1min":0.84899999999999998,"5min":0.95199999999999996,"15min":5.1749999999999998},"last":0.44},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.71499999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":1.133},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.70499999999999996,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.83199999999999996,"5min":0.996,"15min":1.165},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":1.024},"last":0.53000000000000003}]},{"osd":3,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89400000000000002,"5min":1.0129999999999999,"15min":5.202},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73099999999999998,"15min":0.76100000000000001},"min":{"1min":0.55100000000000005,"5min":0.51000000000000001,"15min":0.49399999999999999},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":5.165},"last":0.60399999999999998}]},{"osd":4,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.81399999999999995,"15min":1.083},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.86799999999999999,"15min":0.92800000000000005},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41099999999999998},"max":{"1min":0.85099999999999998,"5min":0.93300000000000005,"15min":1.395},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.129},"last":0.61299999999999999}]},{"osd":6,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.435},"max":{"1min":0.86599999999999999,"5min":0.94499999999999995,"15min":1.24},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.54500000000000004,"5min":0.47399999999999998,"15min":0.443},"max":{"1min":0.80200000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575039,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302372,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069468,"statfs":{"total":96636764160,"available":96327135232,"internally_reserved":0,"allocated":1208320,"data_stored":601423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.33000000000000002,"15min":0.32800000000000001},"max":{"1min":0.87,"5min":0.93999999999999995,"15min":1.0720000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.47899999999999998,"5min":0.47599999999999998,"15min":0.38800000000000001},"max":{"1min":0.86799999999999999,"5min":1.216,"15min":1.216},"last":0.86299999999999999}]},{"osd":1,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.69799999999999995},"min":{"1min":0.49199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91700000000000004,"5min":0.97599999999999998,"15min":1.3799999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.41899999999999998,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":0.98599999999999999,"15min":1.1659999999999999},"last":0.84099999999999997}]},{"osd":2,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76300000000000001,"15min":0.75800000000000001},"min":{"1min":0.59699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96699999999999997,"5min":1.0049999999999999,"15min":1.321},"last":0.746},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.60899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92400000000000004,"5min":1.05,"15min":1.405},"last":0.91300000000000003}]},{"osd":3,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71199999999999997,"15min":0.71699999999999997},"min":{"1min":0.59499999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.1459999999999999,"15min":1.5109999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.60699999999999998,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":0.93200000000000005,"5min":1.002,"15min":1.002},"last":0.82699999999999996}]},{"osd":4,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.018},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.63,"15min":0.64100000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":2.0310000000000001},"last":0.93300000000000005}]},{"osd":5,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.47999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79900000000000004,"5min":1.028,"15min":1.028},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.133},"last":0.97099999999999997}]},{"osd":7,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.47999999999999998,"5min":0.40799999999999997,"15min":0.378},"max":{"1min":0.85599999999999998,"5min":1.002,"15min":1.0940000000000001},"last":0.997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.47699999999999998,"15min":0.46899999999999997},"max":{"1min":0.94799999999999995,"5min":0.97799999999999998,"15min":1.4319999999999999},"last":0.98699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575039,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302372,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069468,"statfs":{"total":96636764160,"available":96327135232,"internally_reserved":0,"allocated":1208320,"data_stored":601049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46999999999999997,"15min":0.45300000000000001},"min":{"1min":0.29199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.75700000000000001,"5min":0.77900000000000003,"15min":0.85199999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51200000000000001,"15min":0.505},"min":{"1min":0.35299999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72499999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":1.8360000000000001}]},{"osd":2,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51100000000000001,"15min":0.504},"min":{"1min":0.33800000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.71499999999999997,"5min":0.80800000000000005,"15min":0.86599999999999999},"last":2.6930000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.59899999999999998},"min":{"1min":0.45300000000000001,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":4.431}]},{"osd":3,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.35899999999999999},"max":{"1min":0.876,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":4.5380000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85399999999999998,"5min":0.94099999999999995,"15min":1.29},"last":4.3550000000000004}]},{"osd":4,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77400000000000002,"15min":0.79000000000000004},"min":{"1min":0.51400000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95899999999999996,"5min":1.083,"15min":1.171},"last":2.5870000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"max":{"1min":1.0369999999999999,"5min":1.0720000000000001,"15min":1.1519999999999999},"last":1.883}]},{"osd":5,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.008,"5min":1.042,"15min":1.329},"last":4.399},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.78400000000000003},"min":{"1min":0.56699999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0109999999999999,"5min":1.028,"15min":1.1160000000000001},"last":2.649}]},{"osd":6,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80600000000000005,"15min":0.80700000000000005},"min":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.501},"max":{"1min":1.0009999999999999,"5min":1.0880000000000001,"15min":1.135},"last":3.024},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80800000000000005,"15min":0.82199999999999995},"min":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94099999999999995,"5min":1.1040000000000001,"15min":1.3540000000000001},"last":3.0609999999999999}]},{"osd":7,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.83699999999999997},"min":{"1min":0.58699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98699999999999999,"5min":1.077,"15min":1.405},"last":2.7869999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.64100000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.012,"5min":1.105,"15min":1.1950000000000001},"last":5.8230000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575039,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302364,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069476,"statfs":{"total":96636764160,"available":96327143424,"internally_reserved":0,"allocated":1200128,"data_stored":600393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.34300000000000003,"5min":0.308,"15min":0.29299999999999998},"max":{"1min":0.69999999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.311},"max":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.97399999999999998},"last":0.499}]},{"osd":2,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46300000000000002,"15min":0.46800000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60599999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53300000000000003,"15min":0.54600000000000004},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.65100000000000002,"5min":0.77500000000000002,"15min":0.91000000000000003},"last":0.59099999999999997}]},{"osd":3,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.38300000000000001},"max":{"1min":0.75,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.52500000000000002}]},{"osd":4,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.80700000000000005},"min":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.3779999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.153,"5min":1.298,"15min":1.8380000000000001},"last":0.84199999999999997}]},{"osd":5,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78600000000000003,"15min":0.80800000000000005},"min":{"1min":0.63500000000000001,"5min":0.51000000000000001,"15min":0.48999999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.3169999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.55500000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2010000000000001,"5min":1.2549999999999999,"15min":1.915},"last":0.93000000000000005}]},{"osd":6,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77400000000000002,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.54600000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78400000000000003,"15min":0.81299999999999994},"min":{"1min":0.53900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.5860000000000001},"last":0.879}]},{"osd":7,"last update":"Mon Nov 6 20:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"max":{"1min":1.171,"5min":1.171,"15min":1.2490000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81399999999999995,"15min":0.83399999999999996},"min":{"1min":0.65400000000000003,"5min":0.54600000000000004,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.6359999999999999},"last":0.98399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575039,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":1196032,"data_stored":600371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.25900000000000001},"max":{"1min":0.54400000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.36299999999999999,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.64300000000000002,"5min":0.72699999999999998,"15min":1.0269999999999999},"last":0.34799999999999998}]},{"osd":1,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.30099999999999999},"max":{"1min":0.69599999999999995,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61399999999999999,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73199999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52100000000000002,"15min":0.50600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85899999999999999,"5min":2.266,"15min":2.266},"last":0.374},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60499999999999998,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.78200000000000003,"5min":1.732,"15min":2.242},"last":0.57699999999999996}]},{"osd":4,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.55900000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1,"5min":1.0980000000000001,"15min":1.371},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.75,"15min":0.77700000000000002},"min":{"1min":0.51900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0429999999999999,"5min":1.087,"15min":1.357},"last":0.52700000000000002}]},{"osd":5,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76700000000000002,"15min":0.78000000000000003},"min":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.45900000000000002},"max":{"1min":1.085,"5min":1.657,"15min":1.657},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.498},"max":{"1min":0.95299999999999996,"5min":1.1479999999999999,"15min":1.4099999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78700000000000003,"15min":0.80300000000000005},"min":{"1min":0.54900000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.026,"5min":1.1499999999999999,"15min":1.4239999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81399999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.47499999999999998}]},{"osd":7,"last update":"Mon Nov 6 20:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.54900000000000004,"5min":0.52800000000000002,"15min":0.46600000000000003},"max":{"1min":0.98699999999999999,"5min":1.1859999999999999,"15min":1.534},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82799999999999996,"15min":0.82599999999999996},"min":{"1min":0.58799999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0369999999999999,"5min":2.202,"15min":2.202},"last":0.54100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575040,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302380,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069460,"statfs":{"total":96636764160,"available":96327127040,"internally_reserved":0,"allocated":1216512,"data_stored":601673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44400000000000001,"15min":0.45600000000000002},"min":{"1min":0.30199999999999999,"5min":0.28399999999999997,"15min":0.26800000000000002},"max":{"1min":0.56899999999999995,"5min":0.71399999999999997,"15min":1.046},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.52700000000000002,"15min":0.51200000000000001},"min":{"1min":0.39400000000000002,"5min":0.34599999999999997,"15min":0.32600000000000001},"max":{"1min":0.68700000000000006,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46600000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48799999999999999,"15min":0.48499999999999999},"min":{"1min":0.39200000000000002,"5min":0.318,"15min":0.313},"max":{"1min":0.57399999999999995,"5min":0.64600000000000002,"15min":0.753},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54600000000000004,"15min":0.55900000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34499999999999997},"max":{"1min":0.70099999999999996,"5min":0.80900000000000005,"15min":1.177},"last":0.40200000000000002}]},{"osd":2,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.437,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73399999999999999,"5min":0.88100000000000001,"15min":1.3120000000000001},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81699999999999995,"15min":0.81000000000000005},"min":{"1min":0.64500000000000002,"5min":0.57899999999999996,"15min":0.51200000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.2470000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.47999999999999998},"max":{"1min":1.276,"5min":1.276,"15min":1.286},"last":0.66800000000000004}]},{"osd":5,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.1519999999999999,"5min":1.1779999999999999,"15min":1.228},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.60399999999999998,"5min":0.54600000000000004,"15min":0.49199999999999999},"max":{"1min":1.101,"5min":1.137,"15min":1.1439999999999999},"last":0.65800000000000003}]},{"osd":6,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.55900000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.747},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80100000000000005,"15min":0.81000000000000005},"min":{"1min":0.622,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.054,"5min":1.202,"15min":1.268},"last":0.71699999999999997}]},{"osd":7,"last update":"Mon Nov 6 20:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0740000000000001,"5min":1.302,"15min":1.302},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.60499999999999998,"5min":0.56899999999999995,"15min":0.47899999999999998},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.61799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575039,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302368,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069472,"statfs":{"total":96636764160,"available":96327139328,"internally_reserved":0,"allocated":1204224,"data_stored":601285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66700000000000004,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94699999999999995,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.501},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.67600000000000005},"min":{"1min":0.58099999999999996,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83099999999999996,"5min":1,"15min":1},"last":0.53500000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.645,"15min":1.645},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.54800000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.625}]},{"osd":2,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70999999999999996,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":0.92800000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92300000000000004,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.73899999999999999}]},{"osd":3,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68200000000000005,"15min":0.69499999999999995},"min":{"1min":0.45200000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90300000000000002,"5min":0.95799999999999996,"15min":1.0169999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.42799999999999999},"max":{"1min":0.89700000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83099999999999996,"5min":1.0389999999999999,"15min":1.0880000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.40799999999999997},"max":{"1min":0.86599999999999999,"5min":0.93999999999999995,"15min":1.3520000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.81699999999999995,"5min":1.22,"15min":1.22},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.51300000000000001,"5min":0.44900000000000001,"15min":0.44},"max":{"1min":0.95899999999999996,"5min":1.0349999999999999,"15min":1.3999999999999999},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Nov 6 20:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.66200000000000003},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.89600000000000002,"5min":1.034,"15min":1.5209999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.67600000000000005},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89800000000000002,"5min":1.177,"15min":1.177},"last":0.72199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575040,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302368,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069472,"statfs":{"total":96636764160,"available":96327139328,"internally_reserved":0,"allocated":1204224,"data_stored":601015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.66400000000000003},"min":{"1min":0.40100000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.82499999999999996,"5min":0.98599999999999999,"15min":1.0109999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.92000000000000004,"5min":0.96199999999999997,"15min":0.96799999999999997},"last":0.72299999999999998}]},{"osd":1,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70599999999999996,"15min":0.70699999999999996},"min":{"1min":0.54300000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":1.0109999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.747},"min":{"1min":0.48199999999999998,"5min":0.432,"15min":0.40300000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.044},"last":0.79100000000000004}]},{"osd":2,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.503,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.024},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.59799999999999998,"5min":0.46000000000000002,"15min":0.45600000000000002},"max":{"1min":0.96399999999999997,"5min":0.96999999999999997,"15min":1.0329999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.629},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.92000000000000004,"5min":0.97599999999999998,"15min":1.02},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.45300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.86399999999999999,"15min":0.93700000000000006},"last":0.496},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.68600000000000005},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89800000000000002,"5min":0.97499999999999998,"15min":0.98599999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.96399999999999997,"15min":0.97599999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":0.88,"5min":0.96399999999999997,"15min":1.105},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.998,"15min":0.998},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T21:00:02.593 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T21:00:02.594 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T21:00:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.827+0000 7ffa633ff700 1 Processor -- start 2023-11-06T21:00:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.827+0000 7ffa633ff700 1 -- start start 2023-11-06T21:00:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa633ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ffa5c064b40 0x7ffa5c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa633ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5c1556d0 con 0x7ffa5c0646c0 2023-11-06T21:00:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa633ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5c064fd0 con 0x7ffa5c1544f0 2023-11-06T21:00:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa633ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5c1548d0 con 0x7ffa5c064b40 2023-11-06T21:00:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa6099a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffa5c0646c0 0x7ffa5c0c57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38884/0 (socket says 172.21.15.33:38884) 2023-11-06T21:00:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa6099a700 1 -- 172.21.15.33:0/187769837 learned_addr learned my addr 172.21.15.33:0/187769837 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa6199c700 1 --2- 172.21.15.33:0/187769837 >> v2:172.21.15.69:3300/0 conn(0x7ffa5c064b40 0x7ffa5c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa53fff700 1 -- 172.21.15.33:0/187769837 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815681921 0 0) 0x7ffa5c1556d0 con 0x7ffa5c0646c0 2023-11-06T21:00:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa53fff700 1 -- 172.21.15.33:0/187769837 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa480033f0 con 0x7ffa5c0646c0 2023-11-06T21:00:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa53fff700 1 -- 172.21.15.33:0/187769837 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1434812660 0 0) 0x7ffa5c064fd0 con 0x7ffa5c1544f0 2023-11-06T21:00:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.829+0000 7ffa53fff700 1 -- 172.21.15.33:0/187769837 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa5c1556d0 con 0x7ffa5c1544f0 2023-11-06T21:00:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.830+0000 7ffa6199c700 1 -- 172.21.15.33:0/187769837 >> v1:172.21.15.69:6789/0 conn(0x7ffa5c1544f0 legacy=0x7ffa5c0c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.830+0000 7ffa6199c700 1 -- 172.21.15.33:0/187769837 >> v1:172.21.15.33:6789/0 conn(0x7ffa5c0646c0 legacy=0x7ffa5c0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.830+0000 7ffa6199c700 1 -- 172.21.15.33:0/187769837 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c154ac0 con 0x7ffa5c064b40 2023-11-06T21:00:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.830+0000 7ffa6199c700 1 --2- 172.21.15.33:0/187769837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa5c064b40 0x7ffa5c0c5ef0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7ffa54009e60 tx=0x7ffa54009780).ready entity=mon.1 client_cookie=107f20e9d2979437 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.830+0000 7ffa53fff700 1 -- 172.21.15.33:0/187769837 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400e070 con 0x7ffa5c064b40 2023-11-06T21:00:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.830+0000 7ffa53fff700 1 -- 172.21.15.33:0/187769837 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5400bab0 con 0x7ffa5c064b40 2023-11-06T21:00:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.830+0000 7ffa53fff700 1 -- 172.21.15.33:0/187769837 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400be30 con 0x7ffa5c064b40 2023-11-06T21:00:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.831+0000 7ffa633ff700 1 -- 172.21.15.33:0/187769837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa5c064b40 msgr2=0x7ffa5c0c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.831+0000 7ffa633ff700 1 --2- 172.21.15.33:0/187769837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa5c064b40 0x7ffa5c0c5ef0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7ffa54009e60 tx=0x7ffa54009780).stop 2023-11-06T21:00:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.831+0000 7ffa633ff700 1 -- 172.21.15.33:0/187769837 shutdown_connections 2023-11-06T21:00:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.831+0000 7ffa633ff700 1 --2- 172.21.15.33:0/187769837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa5c064b40 0x7ffa5c0c5ef0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.831+0000 7ffa633ff700 1 -- 172.21.15.33:0/187769837 >> 172.21.15.33:0/187769837 conn(0x7ffa5c0cb900 msgr2=0x7ffa5c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.831+0000 7ffa633ff700 1 -- 172.21.15.33:0/187769837 shutdown_connections 2023-11-06T21:00:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.831+0000 7ffa633ff700 1 -- 172.21.15.33:0/187769837 wait complete. 2023-11-06T21:00:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.832+0000 7ffa633ff700 1 Processor -- start 2023-11-06T21:00:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.832+0000 7ffa633ff700 1 -- start start 2023-11-06T21:00:02.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.833+0000 7ffa633ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa5c0646c0 0x7ffa5c167b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.833+0000 7ffa633ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa5c064b40 0x7ffa5c1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.833+0000 7ffa633ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c1544f0 0x7ffa5c143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.833+0000 7ffa633ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0aeda0 con 0x7ffa5c1544f0 2023-11-06T21:00:02.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.833+0000 7ffa633ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c168380 con 0x7ffa5c0646c0 2023-11-06T21:00:02.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.833+0000 7ffa633ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c168660 con 0x7ffa5c064b40 2023-11-06T21:00:02.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.833+0000 7ffa6199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c1544f0 0x7ffa5c143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa6199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c1544f0 0x7ffa5c143660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59856/0 (socket says 172.21.15.33:59856) 2023-11-06T21:00:02.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa6199c700 1 -- 172.21.15.33:0/1302275675 learned_addr learned my addr 172.21.15.33:0/1302275675 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:02.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa6119b700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa5c0646c0 0x7ffa5c167b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa6099a700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa5c064b40 0x7ffa5c1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa6199c700 1 -- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa5c064b40 msgr2=0x7ffa5c1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa6199c700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa5c064b40 0x7ffa5c1539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa6199c700 1 -- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa5c0646c0 msgr2=0x7ffa5c167b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:02.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa6199c700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa5c0646c0 0x7ffa5c167b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa6199c700 1 -- 172.21.15.33:0/1302275675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54009a20 con 0x7ffa5c1544f0 2023-11-06T21:00:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa6199c700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c1544f0 0x7ffa5c143660 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7ffa5400b610 tx=0x7ffa54009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.834+0000 7ffa517fa700 1 -- 172.21.15.33:0/1302275675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54012e40 con 0x7ffa5c1544f0 2023-11-06T21:00:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.835+0000 7ffa517fa700 1 -- 172.21.15.33:0/1302275675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5401d630 con 0x7ffa5c1544f0 2023-11-06T21:00:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.835+0000 7ffa633ff700 1 -- 172.21.15.33:0/1302275675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c168940 con 0x7ffa5c1544f0 2023-11-06T21:00:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.835+0000 7ffa517fa700 1 -- 172.21.15.33:0/1302275675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5401d9b0 con 0x7ffa5c1544f0 2023-11-06T21:00:02.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.835+0000 7ffa633ff700 1 -- 172.21.15.33:0/1302275675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c168040 con 0x7ffa5c1544f0 2023-11-06T21:00:02.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.836+0000 7ffa633ff700 1 -- 172.21.15.33:0/1302275675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c0af230 con 0x7ffa5c1544f0 2023-11-06T21:00:02.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.836+0000 7ffa517fa700 1 -- 172.21.15.33:0/1302275675 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ffa5401dd20 con 0x7ffa5c1544f0 2023-11-06T21:00:02.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.838+0000 7ffa517fa700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa4005f270 0x7ffa40061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:02.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.838+0000 7ffa517fa700 1 -- 172.21.15.33:0/1302275675 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ffa5407c1c0 con 0x7ffa5c1544f0 2023-11-06T21:00:02.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.838+0000 7ffa6119b700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa4005f270 0x7ffa40061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:02.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.839+0000 7ffa6119b700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa4005f270 0x7ffa40061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ffa58003890 tx=0x7ffa58007bd0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:02.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:02.841+0000 7ffa517fa700 1 -- 172.21.15.33:0/1302275675 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffa5c0af230 con 0x7ffa5c1544f0 2023-11-06T21:00:03.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.088+0000 7ffa633ff700 1 -- 172.21.15.33:0/1302275675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ffa5c0af230 con 0x7ffa5c1544f0 2023-11-06T21:00:03.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.092+0000 7ffa517fa700 1 -- 172.21.15.33:0/1302275675 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7ffa5c0af230 con 0x7ffa5c1544f0 2023-11-06T21:00:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 -- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa4005f270 msgr2=0x7ffa40061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa4005f270 0x7ffa40061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ffa58003890 tx=0x7ffa58007bd0).stop 2023-11-06T21:00:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 -- 172.21.15.33:0/1302275675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c1544f0 msgr2=0x7ffa5c143660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c1544f0 0x7ffa5c143660 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7ffa5400b610 tx=0x7ffa54009780).stop 2023-11-06T21:00:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 -- 172.21.15.33:0/1302275675 shutdown_connections 2023-11-06T21:00:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa4005f270 0x7ffa40061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c1544f0 0x7ffa5c143660 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa5c0646c0 0x7ffa5c167b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 --2- 172.21.15.33:0/1302275675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa5c064b40 0x7ffa5c1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 -- 172.21.15.33:0/1302275675 >> 172.21.15.33:0/1302275675 conn(0x7ffa5c0cb900 msgr2=0x7ffa5c0ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:03.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 -- 172.21.15.33:0/1302275675 shutdown_connections 2023-11-06T21:00:03.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.094+0000 7ffa633ff700 1 -- 172.21.15.33:0/1302275675 wait complete. 2023-11-06T21:00:03.097 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T21:00:03.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:00:03.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.309+0000 7f4f095ff700 1 Processor -- start 2023-11-06T21:00:03.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.310+0000 7f4f095ff700 1 -- start start 2023-11-06T21:00:03.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.311+0000 7f4f095ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4f040bb990 0x7f4f04152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:03.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.311+0000 7f4f095ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f041534e0 con 0x7f4f0414c640 2023-11-06T21:00:03.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.311+0000 7f4f095ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f040643a0 con 0x7f4f040c33f0 2023-11-06T21:00:03.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.311+0000 7f4f095ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f040bd8f0 con 0x7f4f040bb990 2023-11-06T21:00:03.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.311+0000 7f4f027fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4f040bb990 0x7f4f04152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:03.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.311+0000 7f4f027fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4f040bb990 0x7f4f04152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59870/0 (socket says 172.21.15.33:59870) 2023-11-06T21:00:03.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.311+0000 7f4f027fc700 1 -- 172.21.15.33:0/4155700448 learned_addr learned my addr 172.21.15.33:0/4155700448 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:03.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.311+0000 7f4f02ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4f0414c640 0x7f4f04065160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38886/0 (socket says 172.21.15.33:38886) 2023-11-06T21:00:03.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.312+0000 7f4f027fc700 1 -- 172.21.15.33:0/4155700448 >> v1:172.21.15.69:6789/0 conn(0x7f4f040c33f0 legacy=0x7f4f040c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:03.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.312+0000 7f4f027fc700 1 -- 172.21.15.33:0/4155700448 >> v1:172.21.15.33:6789/0 conn(0x7f4f0414c640 legacy=0x7f4f04065160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:03.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.312+0000 7f4f027fc700 1 -- 172.21.15.33:0/4155700448 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f04064680 con 0x7f4f040bb990 2023-11-06T21:00:03.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.312+0000 7f4f027fc700 1 --2- 172.21.15.33:0/4155700448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f040bb990 0x7f4f04152ff0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f4ef8006fd0 tx=0x7f4ef80090f0).ready entity=mon.0 client_cookie=9bf0d65d51523b07 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:03.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.312+0000 7f4f01ffb700 1 -- 172.21.15.33:0/4155700448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800b600 con 0x7f4f040bb990 2023-11-06T21:00:03.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.312+0000 7f4f01ffb700 1 -- 172.21.15.33:0/4155700448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef800b760 con 0x7f4f040bb990 2023-11-06T21:00:03.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.312+0000 7f4f01ffb700 1 -- 172.21.15.33:0/4155700448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800bae0 con 0x7f4f040bb990 2023-11-06T21:00:03.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.313+0000 7f4f095ff700 1 -- 172.21.15.33:0/4155700448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f040bb990 msgr2=0x7f4f04152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:03.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.313+0000 7f4f095ff700 1 --2- 172.21.15.33:0/4155700448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f040bb990 0x7f4f04152ff0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f4ef8006fd0 tx=0x7f4ef80090f0).stop 2023-11-06T21:00:03.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.313+0000 7f4f095ff700 1 -- 172.21.15.33:0/4155700448 shutdown_connections 2023-11-06T21:00:03.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.313+0000 7f4f095ff700 1 --2- 172.21.15.33:0/4155700448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f040bb990 0x7f4f04152ff0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.313+0000 7f4f095ff700 1 -- 172.21.15.33:0/4155700448 >> 172.21.15.33:0/4155700448 conn(0x7f4f04062710 msgr2=0x7f4f04062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:03.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.313+0000 7f4f095ff700 1 -- 172.21.15.33:0/4155700448 shutdown_connections 2023-11-06T21:00:03.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.313+0000 7f4f095ff700 1 -- 172.21.15.33:0/4155700448 wait complete. 2023-11-06T21:00:03.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.314+0000 7f4f095ff700 1 Processor -- start 2023-11-06T21:00:03.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.314+0000 7f4f095ff700 1 -- start start 2023-11-06T21:00:03.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.315+0000 7f4f095ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4f040bb990 0x7f4f040a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:03.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.315+0000 7f4f095ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f040c33f0 0x7f4f0413b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:03.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.315+0000 7f4f095ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f0414c640 0x7f4f04068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.315+0000 7f4f095ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040c65d0 con 0x7f4f0414c640 2023-11-06T21:00:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.315+0000 7f4f095ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0413ba70 con 0x7f4f040c33f0 2023-11-06T21:00:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.315+0000 7f4f095ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040c5950 con 0x7f4f040bb990 2023-11-06T21:00:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f0414c640 0x7f4f04068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:03.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f0414c640 0x7f4f04068fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59880/0 (socket says 172.21.15.33:59880) 2023-11-06T21:00:03.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f037fe700 1 -- 172.21.15.33:0/996654319 learned_addr learned my addr 172.21.15.33:0/996654319 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:03.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f02ffd700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4f040bb990 0x7f4f040a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:03.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f027fc700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f040c33f0 0x7f4f0413b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:03.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f037fe700 1 -- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4f040bb990 msgr2=0x7f4f040a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:03.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f037fe700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4f040bb990 0x7f4f040a99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f037fe700 1 -- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f040c33f0 msgr2=0x7f4f0413b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f037fe700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f040c33f0 0x7f4f0413b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f037fe700 1 -- 172.21.15.33:0/996654319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8009370 con 0x7f4f0414c640 2023-11-06T21:00:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4f037fe700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f0414c640 0x7f4f04068fa0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f4efc00b7e0 tx=0x7f4efc009310).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.316+0000 7f4eeaffd700 1 -- 172.21.15.33:0/996654319 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc001500 con 0x7f4f0414c640 2023-11-06T21:00:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.317+0000 7f4eeaffd700 1 -- 172.21.15.33:0/996654319 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc0016a0 con 0x7f4f0414c640 2023-11-06T21:00:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.317+0000 7f4f095ff700 1 -- 172.21.15.33:0/996654319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f040c5c30 con 0x7f4f0414c640 2023-11-06T21:00:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.317+0000 7f4eeaffd700 1 -- 172.21.15.33:0/996654319 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc001a60 con 0x7f4f0414c640 2023-11-06T21:00:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.317+0000 7f4f095ff700 1 -- 172.21.15.33:0/996654319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f040c61f0 con 0x7f4f0414c640 2023-11-06T21:00:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.318+0000 7f4f095ff700 1 -- 172.21.15.33:0/996654319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f040c6710 con 0x7f4f0414c640 2023-11-06T21:00:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.319+0000 7f4eeaffd700 1 -- 172.21.15.33:0/996654319 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4efc011400 con 0x7f4f0414c640 2023-11-06T21:00:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.319+0000 7f4eeaffd700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ef005f270 0x7f4ef0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.319+0000 7f4eeaffd700 1 -- 172.21.15.33:0/996654319 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4efc079d30 con 0x7f4f0414c640 2023-11-06T21:00:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.320+0000 7f4f02ffd700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ef005f270 0x7f4ef0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.321+0000 7f4f02ffd700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ef005f270 0x7f4ef0061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f4ef4007fd0 tx=0x7f4ef4007d30).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.323+0000 7f4eeaffd700 1 -- 172.21.15.33:0/996654319 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4efc047af0 con 0x7f4f0414c640 2023-11-06T21:00:03.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.549+0000 7f4f095ff700 1 -- 172.21.15.33:0/996654319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f04064960 con 0x7f4f0414c640 2023-11-06T21:00:03.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.550+0000 7f4eeaffd700 1 -- 172.21.15.33:0/996654319 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f4f040c6710 con 0x7f4f0414c640 2023-11-06T21:00:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:03.550 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1797026087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1797026087},{"type":"v1","addr":"172.21.15.69:6836","nonce":1797026087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4199570083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4199570083},{"type":"v1","addr":"172.21.15.33:6834","nonce":4199570083}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2429309034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2429309034},{"type":"v1","addr":"172.21.15.33:6835","nonce":2429309034}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:57.774874+0000","modified":"2023-11-06T21:00:00.791809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6621":{"gid":6621,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/1260218648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1260218648},{"type":"v1","addr":"172.21.15.69:6837","nonce":1260218648}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-06T21:00:03.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 -- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ef005f270 msgr2=0x7f4ef0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:03.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ef005f270 0x7f4ef0061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f4ef4007fd0 tx=0x7f4ef4007d30).stop 2023-11-06T21:00:03.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 -- 172.21.15.33:0/996654319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f0414c640 msgr2=0x7f4f04068fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:03.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f0414c640 0x7f4f04068fa0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f4efc00b7e0 tx=0x7f4efc009310).stop 2023-11-06T21:00:03.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 -- 172.21.15.33:0/996654319 shutdown_connections 2023-11-06T21:00:03.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ef005f270 0x7f4ef0061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4f040bb990 0x7f4f040a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f0414c640 0x7f4f04068fa0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 --2- 172.21.15.33:0/996654319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f040c33f0 0x7f4f0413b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:03.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 -- 172.21.15.33:0/996654319 >> 172.21.15.33:0/996654319 conn(0x7f4f04062710 msgr2=0x7f4f0414a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:03.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 -- 172.21.15.33:0/996654319 shutdown_connections 2023-11-06T21:00:03.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:03.552+0000 7f4f095ff700 1 -- 172.21.15.33:0/996654319 wait complete. 2023-11-06T21:00:03.554 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 37 2023-11-06T21:00:03.972 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T20:59:57.774874+0000', 'modified': '2023-11-06T21:00:00.791809+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6621}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6621': {'gid': 6621, 'name': 'b', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.69:6837/1260218648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6835', 'nonce': 1260218648}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 1260218648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T21:00:03.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T21:00:03.973 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-06T21:00:04.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.174+0000 7f05c8fff700 1 Processor -- start 2023-11-06T21:00:04.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.174+0000 7f05c8fff700 1 -- start start 2023-11-06T21:00:04.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.176+0000 7f05c8fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05c41544a0 0x7f05c40c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.176+0000 7f05c8fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c41553d0 con 0x7f05c40c6180 2023-11-06T21:00:04.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.176+0000 7f05c8fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c40c6710 con 0x7f05c40637f0 2023-11-06T21:00:04.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.176+0000 7f05c8fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4154880 con 0x7f05c41544a0 2023-11-06T21:00:04.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.176+0000 7f05c1d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f05c40c6180 0x7f05c40630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38892/0 (socket says 172.21.15.33:38892) 2023-11-06T21:00:04.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.176+0000 7f05c1d9b700 1 -- 172.21.15.33:0/451911549 learned_addr learned my addr 172.21.15.33:0/451911549 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:04.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.176+0000 7f05c259c700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05c41544a0 0x7f05c40c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.176+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150042389 0 0) 0x7f05c41553d0 con 0x7f05c40c6180 2023-11-06T21:00:04.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.176+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ac0033f0 con 0x7f05c40c6180 2023-11-06T21:00:04.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379394595 0 0) 0x7f05c40c6710 con 0x7f05c40637f0 2023-11-06T21:00:04.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c41553d0 con 0x7f05c40637f0 2023-11-06T21:00:04.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1663749115 0 0) 0x7f05ac0033f0 con 0x7f05c40c6180 2023-11-06T21:00:04.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05c41544a0 msgr2=0x7f05c40c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05c41544a0 0x7f05c40c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 >> v1:172.21.15.69:6789/0 conn(0x7f05c40637f0 legacy=0x7f05c40bfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c4154aa0 con 0x7f05c40c6180 2023-11-06T21:00:04.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f05b8001470 con 0x7f05c40c6180 2023-11-06T21:00:04.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 >> v1:172.21.15.33:6789/0 conn(0x7f05c40c6180 legacy=0x7f05c40630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05ac003aa0 0x7f05ac003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05ac004690 0x7f05ac004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ac005000 0x7f05ac00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4154880 con 0x7f05ac004690 2023-11-06T21:00:04.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac00bdd0 con 0x7f05ac005000 2023-11-06T21:00:04.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac00bf90 con 0x7f05ac003aa0 2023-11-06T21:00:04.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c259c700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05c41544a0 0x7f05c40c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:04.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05ac004690 0x7f05ac004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c2d9d700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ac005000 0x7f05ac00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.177+0000 7f05c259c700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05ac003aa0 0x7f05ac003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.178+0000 7f05c1d9b700 1 -- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05ac003aa0 msgr2=0x7f05ac003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.178+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05ac003aa0 0x7f05ac003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.178+0000 7f05c1d9b700 1 -- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ac005000 msgr2=0x7f05ac00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.178+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ac005000 0x7f05ac00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.178+0000 7f05c1d9b700 1 -- 172.21.15.33:0/451911549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac0033f0 con 0x7f05ac004690 2023-11-06T21:00:04.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.178+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05ac004690 0x7f05ac004b10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f05b800c160 tx=0x7f05b8001e10).ready entity=mon.0 client_cookie=c29d088a9cc6afa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:04.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.178+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800c9d0 con 0x7f05ac004690 2023-11-06T21:00:04.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.178+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b800cb30 con 0x7f05ac004690 2023-11-06T21:00:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.178+0000 7f05c159a700 1 -- 172.21.15.33:0/451911549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b80070d0 con 0x7f05ac004690 2023-11-06T21:00:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.179+0000 7f05c8fff700 1 -- 172.21.15.33:0/451911549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05ac004690 msgr2=0x7f05ac004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.179+0000 7f05c8fff700 1 --2- 172.21.15.33:0/451911549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05ac004690 0x7f05ac004b10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f05b800c160 tx=0x7f05b8001e10).stop 2023-11-06T21:00:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.179+0000 7f05c2d9d700 1 -- 172.21.15.33:0/451911549 reap_dead start 2023-11-06T21:00:04.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.179+0000 7f05c8fff700 1 -- 172.21.15.33:0/451911549 shutdown_connections 2023-11-06T21:00:04.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.179+0000 7f05c8fff700 1 -- 172.21.15.33:0/451911549 >> 172.21.15.33:0/451911549 conn(0x7f05c405ec80 msgr2=0x7f05c40cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:04.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.179+0000 7f05c8fff700 1 -- 172.21.15.33:0/451911549 shutdown_connections 2023-11-06T21:00:04.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.179+0000 7f05c8fff700 1 -- 172.21.15.33:0/451911549 wait complete. 2023-11-06T21:00:04.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.180+0000 7f05c8fff700 1 Processor -- start 2023-11-06T21:00:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.181+0000 7f05c8fff700 1 -- start start 2023-11-06T21:00:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.181+0000 7f05c8fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05ac004690 0x7f05c4155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.181+0000 7f05c8fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4146c50 0x7f05c4157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.181+0000 7f05c8fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c40c9b20 0x7f05c40c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.181+0000 7f05c8fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40c2860 con 0x7f05c4146c50 2023-11-06T21:00:04.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.181+0000 7f05c8fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4156830 con 0x7f05c40c9b20 2023-11-06T21:00:04.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.181+0000 7f05c8fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40c3090 con 0x7f05ac004690 2023-11-06T21:00:04.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4146c50 0x7f05c4157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4146c50 0x7f05c4157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59894/0 (socket says 172.21.15.33:59894) 2023-11-06T21:00:04.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c1d9b700 1 -- 172.21.15.33:0/1782832824 learned_addr learned my addr 172.21.15.33:0/1782832824 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:04.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c259c700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05ac004690 0x7f05c4155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c2d9d700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c40c9b20 0x7f05c40c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c1d9b700 1 -- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05ac004690 msgr2=0x7f05c4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05ac004690 0x7f05c4155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c1d9b700 1 -- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c40c9b20 msgr2=0x7f05c40c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c40c9b20 0x7f05c40c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c1d9b700 1 -- 172.21.15.33:0/1782832824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40ca490 con 0x7f05c4146c50 2023-11-06T21:00:04.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.182+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4146c50 0x7f05c4157dc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f05b800ca80 tx=0x7f05b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:04.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.183+0000 7f05b27fc700 1 -- 172.21.15.33:0/1782832824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b80046b0 con 0x7f05c4146c50 2023-11-06T21:00:04.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.183+0000 7f05b27fc700 1 -- 172.21.15.33:0/1782832824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b8004810 con 0x7f05c4146c50 2023-11-06T21:00:04.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.183+0000 7f05b27fc700 1 -- 172.21.15.33:0/1782832824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8004bd0 con 0x7f05c4146c50 2023-11-06T21:00:04.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.183+0000 7f05c8fff700 1 -- 172.21.15.33:0/1782832824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c4156550 con 0x7f05c4146c50 2023-11-06T21:00:04.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.183+0000 7f05c8fff700 1 -- 172.21.15.33:0/1782832824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c40c2d20 con 0x7f05c4146c50 2023-11-06T21:00:04.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.185+0000 7f05b27fc700 1 -- 172.21.15.33:0/1782832824 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f05b8020450 con 0x7f05c4146c50 2023-11-06T21:00:04.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.185+0000 7f05b27fc700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f05a405f270 0x7f05a4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.185+0000 7f05b27fc700 1 -- 172.21.15.33:0/1782832824 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f05b807b100 con 0x7f05c4146c50 2023-11-06T21:00:04.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.186+0000 7f05c259c700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f05a405f270 0x7f05a4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.187+0000 7f05c259c700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f05a405f270 0x7f05a4061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f05c4083270 tx=0x7f05b4009060).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:04.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.188+0000 7f05e02abb80 1 Processor -- start 2023-11-06T21:00:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.188+0000 7f05e02abb80 1 -- start start 2023-11-06T21:00:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 0x557dff483460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff47def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff47e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05e02abb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff3bf2f0 con 0x557dff4ab4c0 2023-11-06T21:00:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05e02abb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff3d47e0 con 0x557dff483080 2023-11-06T21:00:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05e02abb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x557dff3d4920 con 0x557dff4cbbe0 2023-11-06T21:00:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff47e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff47e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59898/0 (socket says 172.21.15.33:59898) 2023-11-06T21:00:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05c2d9d700 1 -- 172.21.15.33:0/4115472085 learned_addr learned my addr 172.21.15.33:0/4115472085 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:04.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05c259c700 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 0x557dff483460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.189+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff47def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05c1d9b700 1 -- 172.21.15.33:0/4115472085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 msgr2=0x557dff483460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 0x557dff483460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05c1d9b700 1 -- 172.21.15.33:0/4115472085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 msgr2=0x557dff47e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff47e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05c1d9b700 1 -- 172.21.15.33:0/4115472085 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dff3cd720 con 0x557dff4cbbe0 2023-11-06T21:00:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05c259c700 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 0x557dff483460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05c2d9d700 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff47e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff47def0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f05b8079450 tx=0x7f05b800f000).ready entity=mon.2 client_cookie=88c9c1764853a5f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05a37fe700 1 -- 172.21.15.33:0/4115472085 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b804fb60 con 0x557dff4cbbe0 2023-11-06T21:00:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05a37fe700 1 -- 172.21.15.33:0/4115472085 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b804f930 con 0x557dff4cbbe0 2023-11-06T21:00:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.190+0000 7f05a37fe700 1 -- 172.21.15.33:0/4115472085 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b804f750 con 0x557dff4cbbe0 2023-11-06T21:00:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.191+0000 7f05e02abb80 1 -- 172.21.15.33:0/4115472085 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 msgr2=0x557dff47def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.191+0000 7f05e02abb80 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff47def0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f05b8079450 tx=0x7f05b800f000).stop 2023-11-06T21:00:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.191+0000 7f05e02abb80 1 -- 172.21.15.33:0/4115472085 shutdown_connections 2023-11-06T21:00:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.191+0000 7f05e02abb80 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff47e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.191+0000 7f05e02abb80 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 0x557dff483460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.191+0000 7f05e02abb80 1 --2- 172.21.15.33:0/4115472085 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff47def0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.191+0000 7f05e02abb80 1 -- 172.21.15.33:0/4115472085 >> 172.21.15.33:0/4115472085 conn(0x557dff2ec940 msgr2=0x557dff4cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.191+0000 7f05e02abb80 1 -- 172.21.15.33:0/4115472085 shutdown_connections 2023-11-06T21:00:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.191+0000 7f05e02abb80 1 -- 172.21.15.33:0/4115472085 wait complete. 2023-11-06T21:00:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.192+0000 7f05e02abb80 1 -- start start 2023-11-06T21:00:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.192+0000 7f05e02abb80 1 Processor -- start 2023-11-06T21:00:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 0x557dff4a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff47ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05e02abb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff3d4ba0 con 0x557dff4ab4c0 2023-11-06T21:00:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05e02abb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff39fe60 con 0x557dff483080 2023-11-06T21:00:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05e02abb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x557dff4c8b10 con 0x557dff4cbbe0 2023-11-06T21:00:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05e02abb80 10 client.0 ll_register_callbacks cb 0x557dff374ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T21:00:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59902/0 (socket says 172.21.15.33:59902) 2023-11-06T21:00:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05c1d9b700 1 -- 172.21.15.33:0/938472567 learned_addr learned my addr 172.21.15.33:0/938472567 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.193+0000 7f05c2d9d700 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff47ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05c1d9b700 1 -- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 msgr2=0x557dff47ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05c259c700 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 0x557dff4a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff47ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05c1d9b700 1 -- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 msgr2=0x557dff4a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 0x557dff4a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05c1d9b700 1 -- 172.21.15.33:0/938472567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b40092a0 con 0x557dff4ab4c0 2023-11-06T21:00:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05c2d9d700 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff47ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05c259c700 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 0x557dff4a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c7d70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f05b804faa0 tx=0x7f05b802e000).ready entity=mon.0 client_cookie=1d2d13808002c439 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T21:00:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05a0ff9700 1 -- 172.21.15.33:0/938472567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8055530 con 0x557dff4ab4c0 2023-11-06T21:00:04.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05a0ff9700 1 -- 172.21.15.33:0/938472567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f05b8001650 con 0x557dff4ab4c0 2023-11-06T21:00:04.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05e02abb80 1 -- 172.21.15.33:0/938472567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557dff476e20 con 0x557dff4ab4c0 2023-11-06T21:00:04.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05a0ff9700 1 -- 172.21.15.33:0/938472567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b80552f0 con 0x557dff4ab4c0 2023-11-06T21:00:04.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05e02abb80 10 client.6768 fetch_fsmap learned FSMap version 37 2023-11-06T21:00:04.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05e02abb80 1 -- 172.21.15.33:0/938472567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x7f05bc00f040 con 0x557dff4ab4c0 2023-11-06T21:00:04.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05a0ff9700 1 -- 172.21.15.33:0/938472567 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557dff476e20 con 0x557dff4ab4c0 2023-11-06T21:00:04.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.194+0000 7f05a0ff9700 1 -- 172.21.15.33:0/938472567 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f05b8032040 con 0x557dff4ab4c0 2023-11-06T21:00:04.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.195+0000 7f05e02abb80 10 client.6768 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T21:00:04.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.195+0000 7f05e02abb80 10 client.6768 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-06T21:00:04.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.195+0000 7f05e02abb80 20 client.6768 populate_metadata read hostname 'smithi033' 2023-11-06T21:00:04.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.195+0000 7f05e02abb80 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:6835/1260218648,v1:172.21.15.69:6837/1260218648] conn(0x557dff4b07f0 0x557dff4b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.195+0000 7f05e02abb80 4 client.6768 mds_command: new command op to 6621 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T21:00:04.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.195+0000 7f05e02abb80 1 -- 172.21.15.33:0/938472567 --> [v2:172.21.15.69:6835/1260218648,v1:172.21.15.69:6837/1260218648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557dff23cef0 con 0x557dff4b07f0 2023-11-06T21:00:04.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.195+0000 7f05c259c700 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:6835/1260218648,v1:172.21.15.69:6837/1260218648] conn(0x557dff4b07f0 0x557dff4b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.196+0000 7f05a0ff9700 10 client.6768 ms_handle_connect on v2:172.21.15.69:6835/1260218648 2023-11-06T21:00:04.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.196+0000 7f05c259c700 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:6835/1260218648,v1:172.21.15.69:6837/1260218648] conn(0x557dff4b07f0 0x557dff4b0bd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=5c8692f21c19f71b server_cookie=35a98350c13322c2 in_seq=0 out_seq=0 2023-11-06T21:00:04.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.196+0000 7f05a0ff9700 1 -- 172.21.15.33:0/938472567 <== mds.0 v2:172.21.15.69:6835/1260218648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x557dff23cef0 con 0x557dff4b07f0 2023-11-06T21:00:04.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.196+0000 7f05a0ff9700 10 client.6768 handle_command_reply: tid=0 2023-11-06T21:00:04.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 client.6768 shutdown 2023-11-06T21:00:04.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 -- 172.21.15.33:0/938472567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 msgr2=0x557dff4c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c7d70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f05b804faa0 tx=0x7f05b802e000).stop 2023-11-06T21:00:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 -- 172.21.15.33:0/938472567 shutdown_connections 2023-11-06T21:00:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:6835/1260218648,v1:172.21.15.69:6837/1260218648] conn(0x557dff4b07f0 0x557dff4b0bd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff47ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff483080 0x557dff4a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 --2- 172.21.15.33:0/938472567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c7d70 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05a0ff9700 0 client.6768 ms_handle_reset on v2:172.21.15.69:6835/1260218648 2023-11-06T21:00:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 -- 172.21.15.33:0/938472567 >> 172.21.15.33:0/938472567 conn(0x557dff2ec940 msgr2=0x557dff4cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 -- 172.21.15.33:0/938472567 shutdown_connections 2023-11-06T21:00:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 1 -- 172.21.15.33:0/938472567 wait complete. 2023-11-06T21:00:04.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.197+0000 7f05e02abb80 20 client.6768 trim_cache size 0 max 16384 2023-11-06T21:00:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.214+0000 7f05e02abb80 1 Processor -- start 2023-11-06T21:00:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.214+0000 7f05e02abb80 1 -- start start 2023-11-06T21:00:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.215+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff483080 0x557dff4ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.215+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4ab4c0 0x557dff4cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.215+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff4a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.215+0000 7f05e02abb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff47d530 con 0x557dff483080 2023-11-06T21:00:04.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.215+0000 7f05e02abb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff4794c0 con 0x557dff4ab4c0 2023-11-06T21:00:04.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.215+0000 7f05e02abb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x557dff4cb330 con 0x557dff4cbbe0 2023-11-06T21:00:04.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.215+0000 7f05c259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff483080 0x557dff4ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.215+0000 7f05c259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff483080 0x557dff4ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59910/0 (socket says 172.21.15.33:59910) 2023-11-06T21:00:04.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.215+0000 7f05c259c700 1 -- 172.21.15.33:0/3872303185 learned_addr learned my addr 172.21.15.33:0/3872303185 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:04.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05c2d9d700 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff4a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05c259c700 1 -- 172.21.15.33:0/3872303185 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 msgr2=0x557dff4a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4ab4c0 0x557dff4cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05c259c700 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff4a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05c259c700 1 -- 172.21.15.33:0/3872303185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4ab4c0 msgr2=0x557dff4cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05c259c700 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4ab4c0 0x557dff4cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05c259c700 1 -- 172.21.15.33:0/3872303185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40c2d20 con 0x557dff483080 2023-11-06T21:00:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05c2d9d700 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff4a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4ab4c0 0x557dff4cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05c259c700 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff483080 0x557dff4ca950 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f05c40718e0 tx=0x7f05b4019000).ready entity=mon.0 client_cookie=5f36e1cc9b6a9486 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05a3fff700 1 -- 172.21.15.33:0/3872303185 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4024070 con 0x557dff483080 2023-11-06T21:00:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05a3fff700 1 -- 172.21.15.33:0/3872303185 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b4025040 con 0x557dff483080 2023-11-06T21:00:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.216+0000 7f05a3fff700 1 -- 172.21.15.33:0/3872303185 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4028280 con 0x557dff483080 2023-11-06T21:00:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.217+0000 7f05e02abb80 1 -- 172.21.15.33:0/3872303185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff483080 msgr2=0x557dff4ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.217+0000 7f05e02abb80 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff483080 0x557dff4ca950 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f05c40718e0 tx=0x7f05b4019000).stop 2023-11-06T21:00:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.217+0000 7f05e02abb80 1 -- 172.21.15.33:0/3872303185 shutdown_connections 2023-11-06T21:00:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.217+0000 7f05e02abb80 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff4cbbe0 0x557dff4a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.217+0000 7f05e02abb80 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff483080 0x557dff4ca950 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.217+0000 7f05e02abb80 1 --2- 172.21.15.33:0/3872303185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4ab4c0 0x557dff4cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.217+0000 7f05e02abb80 1 -- 172.21.15.33:0/3872303185 >> 172.21.15.33:0/3872303185 conn(0x557dff2ec940 msgr2=0x557dff482470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:04.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.217+0000 7f05e02abb80 1 -- 172.21.15.33:0/3872303185 shutdown_connections 2023-11-06T21:00:04.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.217+0000 7f05e02abb80 1 -- 172.21.15.33:0/3872303185 wait complete. 2023-11-06T21:00:04.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.217+0000 7f05e02abb80 1 -- start start 2023-11-06T21:00:04.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.218+0000 7f05e02abb80 1 Processor -- start 2023-11-06T21:00:04.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff483080 0x557dff4c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05e02abb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4cbbe0 0x557dff4b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05e02abb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff4c4fd0 con 0x557dff4ab4c0 2023-11-06T21:00:04.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05e02abb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff47dff0 con 0x557dff4cbbe0 2023-11-06T21:00:04.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05e02abb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x557dff52d600 con 0x557dff483080 2023-11-06T21:00:04.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05e02abb80 10 client.0 ll_register_callbacks cb 0x557dff374ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T21:00:04.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59916/0 (socket says 172.21.15.33:59916) 2023-11-06T21:00:04.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c1d9b700 1 -- 172.21.15.33:0/1495705878 learned_addr learned my addr 172.21.15.33:0/1495705878 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c259c700 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff483080 0x557dff4c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c2d9d700 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4cbbe0 0x557dff4b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c1d9b700 1 -- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff483080 msgr2=0x557dff4c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff483080 0x557dff4c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c1d9b700 1 -- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4cbbe0 msgr2=0x557dff4b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4cbbe0 0x557dff4b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c1d9b700 1 -- 172.21.15.33:0/1495705878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b4014a30 con 0x557dff4ab4c0 2023-11-06T21:00:04.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.219+0000 7f05c2d9d700 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4cbbe0 0x557dff4b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:04.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05c1d9b700 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c4ae0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f05b8000dd0 tx=0x7f05b8033000).ready entity=mon.0 client_cookie=453c8304fa8abc10 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:04.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T21:00:04.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05a0ff9700 1 -- 172.21.15.33:0/1495705878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b80552f0 con 0x557dff4ab4c0 2023-11-06T21:00:04.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05a0ff9700 1 -- 172.21.15.33:0/1495705878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f05b804fd90 con 0x557dff4ab4c0 2023-11-06T21:00:04.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05e02abb80 1 -- 172.21.15.33:0/1495705878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05b804f930 con 0x557dff4ab4c0 2023-11-06T21:00:04.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05a0ff9700 1 -- 172.21.15.33:0/1495705878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b8055530 con 0x557dff4ab4c0 2023-11-06T21:00:04.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05e02abb80 10 client.6774 fetch_fsmap learned FSMap version 37 2023-11-06T21:00:04.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05e02abb80 1 -- 172.21.15.33:0/1495705878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x557dff3c8ac0 con 0x557dff4ab4c0 2023-11-06T21:00:04.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05a0ff9700 1 -- 172.21.15.33:0/1495705878 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b804f930 con 0x557dff4ab4c0 2023-11-06T21:00:04.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05a0ff9700 1 -- 172.21.15.33:0/1495705878 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f05b8079010 con 0x557dff4ab4c0 2023-11-06T21:00:04.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05e02abb80 10 client.6774 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T21:00:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05e02abb80 10 client.6774 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-06T21:00:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05e02abb80 20 client.6774 populate_metadata read hostname 'smithi033' 2023-11-06T21:00:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05e02abb80 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:6835/1260218648,v1:172.21.15.69:6837/1260218648] conn(0x557dff52a230 0x557dff4b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05e02abb80 4 client.6774 mds_command: new command op to 6621 tid=0[{"prefix": "status"}] 2023-11-06T21:00:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.220+0000 7f05e02abb80 1 -- 172.21.15.33:0/1495705878 --> [v2:172.21.15.69:6835/1260218648,v1:172.21.15.69:6837/1260218648] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557dff272040 con 0x557dff52a230 2023-11-06T21:00:04.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.221+0000 7f05c259c700 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:6835/1260218648,v1:172.21.15.69:6837/1260218648] conn(0x557dff52a230 0x557dff4b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.221+0000 7f05c259c700 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:6835/1260218648,v1:172.21.15.69:6837/1260218648] conn(0x557dff52a230 0x557dff4b1430 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6f3291b5052f77a0 server_cookie=a9158bcae6c6044f in_seq=0 out_seq=0 2023-11-06T21:00:04.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.221+0000 7f05a0ff9700 10 client.6774 ms_handle_connect on v2:172.21.15.69:6835/1260218648 2023-11-06T21:00:04.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05a0ff9700 1 -- 172.21.15.33:0/1495705878 <== mds.0 v2:172.21.15.69:6835/1260218648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557dff272040 con 0x557dff52a230 2023-11-06T21:00:04.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05a0ff9700 10 client.6774 handle_command_reply: tid=0 2023-11-06T21:00:04.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 client.6774 shutdown 2023-11-06T21:00:04.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 -- 172.21.15.33:0/1495705878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 msgr2=0x557dff4c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c4ae0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f05b8000dd0 tx=0x7f05b8033000).stop 2023-11-06T21:00:04.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 -- 172.21.15.33:0/1495705878 shutdown_connections 2023-11-06T21:00:04.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:6835/1260218648,v1:172.21.15.69:6837/1260218648] conn(0x557dff52a230 0x557dff4b1430 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x557dff483080 0x557dff4c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557dff4cbbe0 0x557dff4b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 --2- 172.21.15.33:0/1495705878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557dff4ab4c0 0x557dff4c4ae0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05a0ff9700 0 client.6774 ms_handle_reset on v2:172.21.15.69:6835/1260218648 2023-11-06T21:00:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 -- 172.21.15.33:0/1495705878 >> 172.21.15.33:0/1495705878 conn(0x557dff2ec940 msgr2=0x557dff4ca610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 -- 172.21.15.33:0/1495705878 shutdown_connections 2023-11-06T21:00:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.222+0000 7f05e02abb80 1 -- 172.21.15.33:0/1495705878 wait complete. 2023-11-06T21:00:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05e02abb80 20 client.6774 trim_cache size 0 max 16384 2023-11-06T21:00:04.245 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T21:00:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T21:00:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T21:00:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 6621, 2023-11-06T21:00:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T21:00:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T21:00:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T21:00:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.4445860770000003, 2023-11-06T21:00:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 37, 2023-11-06T21:00:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 99, 2023-11-06T21:00:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 99, 2023-11-06T21:00:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.660259981999999 2023-11-06T21:00:04.248 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T21:00:04.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 -- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f05a405f270 msgr2=0x7f05a4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f05a405f270 0x7f05a4061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f05c4083270 tx=0x7f05b4009060).stop 2023-11-06T21:00:04.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 -- 172.21.15.33:0/1782832824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4146c50 msgr2=0x7f05c4157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4146c50 0x7f05c4157dc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f05b800ca80 tx=0x7f05b8015000).stop 2023-11-06T21:00:04.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 -- 172.21.15.33:0/1782832824 shutdown_connections 2023-11-06T21:00:04.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f05a405f270 0x7f05a4061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05ac004690 0x7f05c4155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c40c9b20 0x7f05c40c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 --2- 172.21.15.33:0/1782832824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4146c50 0x7f05c4157dc0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 -- 172.21.15.33:0/1782832824 >> 172.21.15.33:0/1782832824 conn(0x7f05c405ec80 msgr2=0x7f05c40afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:04.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 -- 172.21.15.33:0/1782832824 shutdown_connections 2023-11-06T21:00:04.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.223+0000 7f05c8fff700 1 -- 172.21.15.33:0/1782832824 wait complete. 2023-11-06T21:00:04.252 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T21:00:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.417+0000 7ffa4e3ff700 1 Processor -- start 2023-11-06T21:00:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.417+0000 7ffa4e3ff700 1 -- start start 2023-11-06T21:00:04.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.419+0000 7ffa4e3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa481541a0 0x7ffa480c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.419+0000 7ffa4e3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 0x7ffa480c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.419+0000 7ffa4e3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa480c5f80 con 0x7ffa480c07b0 2023-11-06T21:00:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.419+0000 7ffa4e3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa480aa740 con 0x7ffa481541a0 2023-11-06T21:00:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.419+0000 7ffa4e3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480c6800 con 0x7ffa480a9ee0 2023-11-06T21:00:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.419+0000 7ffa47fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa481541a0 0x7ffa480c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.419+0000 7ffa4c99c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffa480c07b0 0x7ffa480a7c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38904/0 (socket says 172.21.15.33:38904) 2023-11-06T21:00:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.419+0000 7ffa4c99c700 1 -- 172.21.15.33:0/2138850850 learned_addr learned my addr 172.21.15.33:0/2138850850 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa477fe700 1 --2- 172.21.15.33:0/2138850850 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 0x7ffa480c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa47fff700 1 -- 172.21.15.33:0/2138850850 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 msgr2=0x7ffa480c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa47fff700 1 --2- 172.21.15.33:0/2138850850 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 0x7ffa480c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa47fff700 1 -- 172.21.15.33:0/2138850850 >> v1:172.21.15.33:6789/0 conn(0x7ffa480c07b0 legacy=0x7ffa480a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa47fff700 1 -- 172.21.15.33:0/2138850850 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480aa460 con 0x7ffa481541a0 2023-11-06T21:00:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa46ffd700 1 -- 172.21.15.33:0/2138850850 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948860610 0 0) 0x7ffa480c5f80 con 0x7ffa480c07b0 2023-11-06T21:00:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa477fe700 1 --2- 172.21.15.33:0/2138850850 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 0x7ffa480c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa47fff700 1 --2- 172.21.15.33:0/2138850850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa481541a0 0x7ffa480c8220 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7ffa38006fd0 tx=0x7ffa38006f40).ready entity=mon.0 client_cookie=4906d66e1bb5974d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa46ffd700 1 -- 172.21.15.33:0/2138850850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38005880 con 0x7ffa481541a0 2023-11-06T21:00:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa46ffd700 1 -- 172.21.15.33:0/2138850850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa380059e0 con 0x7ffa481541a0 2023-11-06T21:00:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.420+0000 7ffa46ffd700 1 -- 172.21.15.33:0/2138850850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38005d60 con 0x7ffa481541a0 2023-11-06T21:00:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.421+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/2138850850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa481541a0 msgr2=0x7ffa480c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.421+0000 7ffa4e3ff700 1 --2- 172.21.15.33:0/2138850850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa481541a0 0x7ffa480c8220 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7ffa38006fd0 tx=0x7ffa38006f40).stop 2023-11-06T21:00:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.421+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/2138850850 shutdown_connections 2023-11-06T21:00:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.421+0000 7ffa4e3ff700 1 --2- 172.21.15.33:0/2138850850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa481541a0 0x7ffa480c8220 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.421+0000 7ffa4e3ff700 1 --2- 172.21.15.33:0/2138850850 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 0x7ffa480c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.421+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/2138850850 >> 172.21.15.33:0/2138850850 conn(0x7ffa480944e0 msgr2=0x7ffa4806a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.421+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/2138850850 shutdown_connections 2023-11-06T21:00:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.421+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/2138850850 wait complete. 2023-11-06T21:00:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.422+0000 7ffa4e3ff700 1 Processor -- start 2023-11-06T21:00:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.423+0000 7ffa4e3ff700 1 -- start start 2023-11-06T21:00:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa4e3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 0x7ffa480ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa4e3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa480c07b0 0x7ffa480c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa4e3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa481541a0 0x7ffa4814c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa4e3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480ace60 con 0x7ffa480c07b0 2023-11-06T21:00:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa4e3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480ac250 con 0x7ffa481541a0 2023-11-06T21:00:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa4e3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480ac530 con 0x7ffa480a9ee0 2023-11-06T21:00:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa480c07b0 0x7ffa480c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa480c07b0 0x7ffa480c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59932/0 (socket says 172.21.15.33:59932) 2023-11-06T21:00:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa477fe700 1 -- 172.21.15.33:0/708051438 learned_addr learned my addr 172.21.15.33:0/708051438 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa47fff700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 0x7ffa480ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa4c99c700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa481541a0 0x7ffa4814c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa477fe700 1 -- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 msgr2=0x7ffa480ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa477fe700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 0x7ffa480ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa477fe700 1 -- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa481541a0 msgr2=0x7ffa4814c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa477fe700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa481541a0 0x7ffa4814c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.424+0000 7ffa477fe700 1 -- 172.21.15.33:0/708051438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa38009250 con 0x7ffa480c07b0 2023-11-06T21:00:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.425+0000 7ffa477fe700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa480c07b0 0x7ffa480c45e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7ffa3c00d860 tx=0x7ffa3c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.425+0000 7ffa2ffff700 1 -- 172.21.15.33:0/708051438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c011590 con 0x7ffa480c07b0 2023-11-06T21:00:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.425+0000 7ffa2ffff700 1 -- 172.21.15.33:0/708051438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c0116f0 con 0x7ffa480c07b0 2023-11-06T21:00:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.425+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/708051438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa480ac810 con 0x7ffa480c07b0 2023-11-06T21:00:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.425+0000 7ffa2ffff700 1 -- 172.21.15.33:0/708051438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c011a70 con 0x7ffa480c07b0 2023-11-06T21:00:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.425+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/708051438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa480c4ad0 con 0x7ffa480c07b0 2023-11-06T21:00:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.426+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/708051438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa480ca7d0 con 0x7ffa480c07b0 2023-11-06T21:00:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.427+0000 7ffa2ffff700 1 -- 172.21.15.33:0/708051438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ffa3c012420 con 0x7ffa480c07b0 2023-11-06T21:00:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.428+0000 7ffa2ffff700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa3405f270 0x7ffa34061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.428+0000 7ffa2ffff700 1 -- 172.21.15.33:0/708051438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ffa3c079fe0 con 0x7ffa480c07b0 2023-11-06T21:00:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.428+0000 7ffa47fff700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa3405f270 0x7ffa34061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.429+0000 7ffa47fff700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa3405f270 0x7ffa34061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ffa38006f80 tx=0x7ffa38016000).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.432+0000 7ffa2ffff700 1 -- 172.21.15.33:0/708051438 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffa3c04b050 con 0x7ffa480c07b0 2023-11-06T21:00:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.655+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/708051438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffa480475a0 con 0x7ffa480c07b0 2023-11-06T21:00:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.656+0000 7ffa2ffff700 1 -- 172.21.15.33:0/708051438 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffa3c04e670 con 0x7ffa480c07b0 2023-11-06T21:00:04.656 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T21:00:04.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa3405f270 msgr2=0x7ffa34061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa3405f270 0x7ffa34061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ffa38006f80 tx=0x7ffa38016000).stop 2023-11-06T21:00:04.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/708051438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa480c07b0 msgr2=0x7ffa480c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:04.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa480c07b0 0x7ffa480c45e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7ffa3c00d860 tx=0x7ffa3c008040).stop 2023-11-06T21:00:04.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/708051438 shutdown_connections 2023-11-06T21:00:04.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ffa3405f270 0x7ffa34061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffa480a9ee0 0x7ffa480ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa481541a0 0x7ffa4814c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 --2- 172.21.15.33:0/708051438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa480c07b0 0x7ffa480c45e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/708051438 >> 172.21.15.33:0/708051438 conn(0x7ffa480944e0 msgr2=0x7ffa48064730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/708051438 shutdown_connections 2023-11-06T21:00:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:04.658+0000 7ffa4e3ff700 1 -- 172.21.15.33:0/708051438 wait complete. 2023-11-06T21:00:04.672 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T21:00:04.673 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T21:00:04.673 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T21:00:04.673 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T21:00:04.674 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T21:00:04.674 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T21:00:04.674 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T21:00:04.675 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T21:00:04.675 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T21:00:04.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:04.676 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T21:00:04.724 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T21:00:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T21:00:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T21:00:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T21:00:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:04.727 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:00:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:00:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T21:00:04.727 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:00:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:00:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T21:00:04.728 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T21:00:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:00:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T21:00:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T21:00:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41339sec preferred_lft 41339sec 2023-11-06T21:00:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T21:00:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:04.729 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:00:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:00:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T21:00:04.730 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T21:00:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T21:00:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T21:00:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T21:00:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:04.731 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T21:00:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T21:00:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T21:00:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:04.732 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T21:00:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T21:00:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T21:00:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:04.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:04.733 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T21:00:04.802 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T21:00:04.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:04.806 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T21:00:04.856 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T21:00:04.856 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T21:00:04.857 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T21:00:04.857 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T21:00:04.857 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T21:00:04.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:04.859 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T21:00:04.859 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T21:00:04.859 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T21:00:04.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T21:00:04.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T21:00:04.860 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T21:00:05.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:05.037 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T21:00:05.084 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T21:00:05.085 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T21:00:05.086 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T21:00:05.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:05.086 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T21:00:05.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:05.159 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T21:00:05.207 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T21:00:05.208 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:00:05.309 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:00:05.310 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T21:00:05.337 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:00:05.337 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T21:00:05.343 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T21:00:05.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:05.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T21:00:05.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T21:00:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.614+0000 7f08b8dff700 1 Processor -- start 2023-11-06T21:00:05.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.615+0000 7f08b8dff700 1 -- start start 2023-11-06T21:00:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.616+0000 7f08b8dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b40c0470 0x7f08b40c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.616+0000 7f08b8dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b40c7700 con 0x7f08b4154210 2023-11-06T21:00:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.616+0000 7f08b8dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b405efc0 con 0x7f08b40c6d90 2023-11-06T21:00:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.616+0000 7f08b8dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b40c8ea0 con 0x7f08b40c0470 2023-11-06T21:00:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.616+0000 7f08b259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f08b4154210 0x7f08b40a7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38912/0 (socket says 172.21.15.33:38912) 2023-11-06T21:00:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.616+0000 7f08b259c700 1 -- 172.21.15.33:0/3937492927 learned_addr learned my addr 172.21.15.33:0/3937492927 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.616+0000 7f08b2d9d700 1 --2- 172.21.15.33:0/3937492927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b40c0470 0x7f08b40c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.616+0000 7f08b159a700 1 -- 172.21.15.33:0/3937492927 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770778843 0 0) 0x7f08a4001110 con 0x7f08b4154210 2023-11-06T21:00:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b159a700 1 -- 172.21.15.33:0/3937492927 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f089c0033f0 con 0x7f08b4154210 2023-11-06T21:00:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b159a700 1 -- 172.21.15.33:0/3937492927 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152694725 0 0) 0x7f08b405efc0 con 0x7f08b40c6d90 2023-11-06T21:00:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b159a700 1 -- 172.21.15.33:0/3937492927 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08a4001110 con 0x7f08b40c6d90 2023-11-06T21:00:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b159a700 1 -- 172.21.15.33:0/3937492927 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3349480561 0 0) 0x7f089c0033f0 con 0x7f08b4154210 2023-11-06T21:00:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b2d9d700 1 -- 172.21.15.33:0/3937492927 >> v1:172.21.15.69:6789/0 conn(0x7f08b40c6d90 legacy=0x7f08b40bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b2d9d700 1 -- 172.21.15.33:0/3937492927 >> v1:172.21.15.33:6789/0 conn(0x7f08b4154210 legacy=0x7f08b40a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b2d9d700 1 -- 172.21.15.33:0/3937492927 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b40c9140 con 0x7f08b40c0470 2023-11-06T21:00:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b2d9d700 1 --2- 172.21.15.33:0/3937492927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b40c0470 0x7f08b40c7210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f08ac009e90 tx=0x7f08ac009780).ready entity=mon.2 client_cookie=fd7e6fc9f9c4848c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b159a700 1 -- 172.21.15.33:0/3937492927 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac00e070 con 0x7f08b40c0470 2023-11-06T21:00:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b159a700 1 -- 172.21.15.33:0/3937492927 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac00b7a0 con 0x7f08b40c0470 2023-11-06T21:00:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.617+0000 7f08b159a700 1 -- 172.21.15.33:0/3937492927 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4001600 con 0x7f08b40c0470 2023-11-06T21:00:05.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.618+0000 7f08b8dff700 1 -- 172.21.15.33:0/3937492927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b40c0470 msgr2=0x7f08b40c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:05.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.618+0000 7f08b8dff700 1 --2- 172.21.15.33:0/3937492927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b40c0470 0x7f08b40c7210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f08ac009e90 tx=0x7f08ac009780).stop 2023-11-06T21:00:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.618+0000 7f08b8dff700 1 -- 172.21.15.33:0/3937492927 shutdown_connections 2023-11-06T21:00:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.618+0000 7f08b8dff700 1 --2- 172.21.15.33:0/3937492927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b40c0470 0x7f08b40c7210 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.618+0000 7f08b8dff700 1 -- 172.21.15.33:0/3937492927 >> 172.21.15.33:0/3937492927 conn(0x7f08b405e990 msgr2=0x7f08b40cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.618+0000 7f08b8dff700 1 -- 172.21.15.33:0/3937492927 shutdown_connections 2023-11-06T21:00:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.618+0000 7f08b8dff700 1 -- 172.21.15.33:0/3937492927 wait complete. 2023-11-06T21:00:05.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.620+0000 7f08b8dff700 1 Processor -- start 2023-11-06T21:00:05.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.620+0000 7f08b8dff700 1 -- start start 2023-11-06T21:00:05.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b8dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08b40c0470 0x7f08b4145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:05.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b8dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b40c6d90 0x7f08b40ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:05.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b8dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b4154210 0x7f08b4167da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b8dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b40ae180 con 0x7f08b40c6d90 2023-11-06T21:00:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b8dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b41464e0 con 0x7f08b40c0470 2023-11-06T21:00:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b8dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b40ad140 con 0x7f08b4154210 2023-11-06T21:00:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b40c6d90 0x7f08b40ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:05.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b40c6d90 0x7f08b40ac6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59942/0 (socket says 172.21.15.33:59942) 2023-11-06T21:00:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b1d9b700 1 -- 172.21.15.33:0/885664391 learned_addr learned my addr 172.21.15.33:0/885664391 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b2d9d700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b4154210 0x7f08b4167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b259c700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08b40c0470 0x7f08b4145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b1d9b700 1 -- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b4154210 msgr2=0x7f08b4167da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b1d9b700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b4154210 0x7f08b4167da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b1d9b700 1 -- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08b40c0470 msgr2=0x7f08b4145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b1d9b700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08b40c0470 0x7f08b4145ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b1d9b700 1 -- 172.21.15.33:0/885664391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac009a20 con 0x7f08b40c6d90 2023-11-06T21:00:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b2d9d700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b4154210 0x7f08b4167da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.621+0000 7f08b259c700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08b40c0470 0x7f08b4145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:05.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.622+0000 7f08b1d9b700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b40c6d90 0x7f08b40ac6e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f08a800c4f0 tx=0x7f08a8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.622+0000 7f08a27fc700 1 -- 172.21.15.33:0/885664391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a800f470 con 0x7f08b40c6d90 2023-11-06T21:00:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.622+0000 7f08a27fc700 1 -- 172.21.15.33:0/885664391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a800f5d0 con 0x7f08b40c6d90 2023-11-06T21:00:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.622+0000 7f08b8dff700 1 -- 172.21.15.33:0/885664391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b40ae3d0 con 0x7f08b40c6d90 2023-11-06T21:00:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.622+0000 7f08a27fc700 1 -- 172.21.15.33:0/885664391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a800f990 con 0x7f08b40c6d90 2023-11-06T21:00:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.622+0000 7f08b8dff700 1 -- 172.21.15.33:0/885664391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b40ae990 con 0x7f08b40c6d90 2023-11-06T21:00:05.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.623+0000 7f08b8dff700 1 -- 172.21.15.33:0/885664391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b406c6c0 con 0x7f08b40c6d90 2023-11-06T21:00:05.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.624+0000 7f08a27fc700 1 -- 172.21.15.33:0/885664391 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f08a801b3b0 con 0x7f08b40c6d90 2023-11-06T21:00:05.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.625+0000 7f08a27fc700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f089405f270 0x7f0894061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:05.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.625+0000 7f08a27fc700 1 -- 172.21.15.33:0/885664391 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f08a8079ad0 con 0x7f08b40c6d90 2023-11-06T21:00:05.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.625+0000 7f08b259c700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f089405f270 0x7f0894061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:05.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.626+0000 7f08b259c700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f089405f270 0x7f0894061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f08a40017f0 tx=0x7f08a4009040).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:05.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.629+0000 7f08a27fc700 1 -- 172.21.15.33:0/885664391 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f08a804ab00 con 0x7f08b40c6d90 2023-11-06T21:00:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.838+0000 7f08b8dff700 1 -- 172.21.15.33:0/885664391 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f08b40c9420 con 0x7f089405f270 2023-11-06T21:00:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.840+0000 7f08a27fc700 1 -- 172.21.15.33:0/885664391 <== mgr.6280 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f08b40c9420 con 0x7f089405f270 2023-11-06T21:00:05.840 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T21:00:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T21:00:05.841 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:05.841 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T21:00:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 -- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f089405f270 msgr2=0x7f0894061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f089405f270 0x7f0894061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f08a40017f0 tx=0x7f08a4009040).stop 2023-11-06T21:00:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 -- 172.21.15.33:0/885664391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b40c6d90 msgr2=0x7f08b40ac6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b40c6d90 0x7f08b40ac6e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f08a800c4f0 tx=0x7f08a8007c10).stop 2023-11-06T21:00:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 -- 172.21.15.33:0/885664391 shutdown_connections 2023-11-06T21:00:05.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f089405f270 0x7f0894061730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:05.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08b4154210 0x7f08b4167da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:05.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08b40c0470 0x7f08b4145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:05.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 --2- 172.21.15.33:0/885664391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b40c6d90 0x7f08b40ac6e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:05.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 -- 172.21.15.33:0/885664391 >> 172.21.15.33:0/885664391 conn(0x7f08b405e990 msgr2=0x7f08b40bfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:05.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 -- 172.21.15.33:0/885664391 shutdown_connections 2023-11-06T21:00:05.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:05.843+0000 7f08b8dff700 1 -- 172.21.15.33:0/885664391 wait complete. 2023-11-06T21:00:05.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T21:00:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.059+0000 7f85a77ff700 1 Processor -- start 2023-11-06T21:00:06.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.059+0000 7f85a77ff700 1 -- start start 2023-11-06T21:00:06.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.060+0000 7f85a77ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a01547d0 0x7f85a015f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:06.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.060+0000 7f85a77ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a0063c00 con 0x7f85a0155410 2023-11-06T21:00:06.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f85a77ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a0064090 con 0x7f85a0155030 2023-11-06T21:00:06.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f85a77ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0062ff0 con 0x7f85a01547d0 2023-11-06T21:00:06.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f85a4d9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f85a0155410 0x7f85a00644f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38920/0 (socket says 172.21.15.33:38920) 2023-11-06T21:00:06.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f85a4d9a700 1 -- 172.21.15.33:0/3793440849 learned_addr learned my addr 172.21.15.33:0/3793440849 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:06.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f85a5d9c700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a01547d0 0x7f85a015f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:06.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421742878 0 0) 0x7f85a0063c00 con 0x7f85a0155410 2023-11-06T21:00:06.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f858c0033f0 con 0x7f85a0155410 2023-11-06T21:00:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635404354 0 0) 0x7f85a0064090 con 0x7f85a0155030 2023-11-06T21:00:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a0063c00 con 0x7f85a0155030 2023-11-06T21:00:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 762123841 0 0) 0x7f858c0033f0 con 0x7f85a0155410 2023-11-06T21:00:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a01547d0 msgr2=0x7f85a015f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f8597fff700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a01547d0 0x7f85a015f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:06.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.061+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 >> v1:172.21.15.69:6789/0 conn(0x7f85a0155030 legacy=0x7f85a0062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:06.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00632d0 con 0x7f85a0155410 2023-11-06T21:00:06.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f8590001470 con 0x7f85a0155410 2023-11-06T21:00:06.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f85a5d9c700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a01547d0 0x7f85a015f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:06.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 >> v1:172.21.15.33:6789/0 conn(0x7f85a0155410 legacy=0x7f85a00644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:06.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f8597fff700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c003aa0 0x7f858c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:06.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f8597fff700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a01547d0 0x7f858c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:06.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f8597fff700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f858c004b80 0x7f858c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:06.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0062ff0 con 0x7f858c004b80 2023-11-06T21:00:06.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c00b8e0 con 0x7f858c003aa0 2023-11-06T21:00:06.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c00ba70 con 0x7f85a01547d0 2023-11-06T21:00:06.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f85a5d9c700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f858c004b80 0x7f858c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:06.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f85a559b700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c003aa0 0x7f858c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:06.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f85a4d9a700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a01547d0 0x7f858c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:06.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f85a5d9c700 1 -- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a01547d0 msgr2=0x7f858c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:06.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f85a5d9c700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a01547d0 0x7f858c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:06.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f85a5d9c700 1 -- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c003aa0 msgr2=0x7f858c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:06.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f85a5d9c700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c003aa0 0x7f858c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:06.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.062+0000 7f85a5d9c700 1 -- 172.21.15.33:0/3793440849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8598009a20 con 0x7f858c004b80 2023-11-06T21:00:06.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.063+0000 7f85a5d9c700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f858c004b80 0x7f858c005000 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f85a0068810 tx=0x7f859800b040).ready entity=mon.0 client_cookie=f86d9c60e6b8a800 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:06.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.063+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598009c50 con 0x7f858c004b80 2023-11-06T21:00:06.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.063+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859800f040 con 0x7f858c004b80 2023-11-06T21:00:06.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.063+0000 7f8597fff700 1 -- 172.21.15.33:0/3793440849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859800ee50 con 0x7f858c004b80 2023-11-06T21:00:06.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.063+0000 7f85a77ff700 1 -- 172.21.15.33:0/3793440849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f858c004b80 msgr2=0x7f858c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:06.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.063+0000 7f85a77ff700 1 --2- 172.21.15.33:0/3793440849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f858c004b80 0x7f858c005000 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f85a0068810 tx=0x7f859800b040).stop 2023-11-06T21:00:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.063+0000 7f85a5d9c700 1 -- 172.21.15.33:0/3793440849 reap_dead start 2023-11-06T21:00:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.063+0000 7f85a77ff700 1 -- 172.21.15.33:0/3793440849 shutdown_connections 2023-11-06T21:00:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.063+0000 7f85a77ff700 1 -- 172.21.15.33:0/3793440849 >> 172.21.15.33:0/3793440849 conn(0x7f85a00c3d30 msgr2=0x7f85a00c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.063+0000 7f85a77ff700 1 -- 172.21.15.33:0/3793440849 shutdown_connections 2023-11-06T21:00:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.064+0000 7f85a77ff700 1 -- 172.21.15.33:0/3793440849 wait complete. 2023-11-06T21:00:06.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.065+0000 7f85a77ff700 1 Processor -- start 2023-11-06T21:00:06.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.065+0000 7f85a77ff700 1 -- start start 2023-11-06T21:00:06.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.066+0000 7f85a77ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c004b80 0x7f85a0135f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:06.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.066+0000 7f85a77ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a00a8d20 0x7f85a00a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:06.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.066+0000 7f85a77ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a00bfb90 0x7f85a00c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:06.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.066+0000 7f85a77ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00bf0e0 con 0x7f85a00bfb90 2023-11-06T21:00:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.066+0000 7f85a77ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00a9690 con 0x7f858c004b80 2023-11-06T21:00:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.066+0000 7f85a77ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00c0500 con 0x7f85a00a8d20 2023-11-06T21:00:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.066+0000 7f85a5d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a00bfb90 0x7f85a00c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.066+0000 7f85a5d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a00bfb90 0x7f85a00c0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59954/0 (socket says 172.21.15.33:59954) 2023-11-06T21:00:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.066+0000 7f85a5d9c700 1 -- 172.21.15.33:0/1818403758 learned_addr learned my addr 172.21.15.33:0/1818403758 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.066+0000 7f85a559b700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c004b80 0x7f85a0135f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85a4d9a700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a00a8d20 0x7f85a00a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85a5d9c700 1 -- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a00a8d20 msgr2=0x7f85a00a91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85a5d9c700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a00a8d20 0x7f85a00a91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85a5d9c700 1 -- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c004b80 msgr2=0x7f85a0135f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85a5d9c700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c004b80 0x7f85a0135f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85a5d9c700 1 -- 172.21.15.33:0/1818403758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00be4d0 con 0x7f85a00bfb90 2023-11-06T21:00:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85a5d9c700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a00bfb90 0x7f85a00c0010 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f858c0045e0 tx=0x7f859801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85957fa700 1 -- 172.21.15.33:0/1818403758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598022030 con 0x7f85a00bfb90 2023-11-06T21:00:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85957fa700 1 -- 172.21.15.33:0/1818403758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8598009c50 con 0x7f85a00bfb90 2023-11-06T21:00:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85a77ff700 1 -- 172.21.15.33:0/1818403758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a00be7b0 con 0x7f85a00bfb90 2023-11-06T21:00:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85957fa700 1 -- 172.21.15.33:0/1818403758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859801e870 con 0x7f85a00bfb90 2023-11-06T21:00:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.067+0000 7f85a77ff700 1 -- 172.21.15.33:0/1818403758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a00bed70 con 0x7f85a00bfb90 2023-11-06T21:00:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.068+0000 7f85a77ff700 1 -- 172.21.15.33:0/1818403758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a00bef00 con 0x7f85a00bfb90 2023-11-06T21:00:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.069+0000 7f85957fa700 1 -- 172.21.15.33:0/1818403758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8598025070 con 0x7f85a00bfb90 2023-11-06T21:00:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.071+0000 7f85957fa700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f858405f270 0x7f8584061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.071+0000 7f85957fa700 1 -- 172.21.15.33:0/1818403758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8598023070 con 0x7f85a00bfb90 2023-11-06T21:00:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.071+0000 7f85a559b700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f858405f270 0x7f8584061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.072+0000 7f85a559b700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f858405f270 0x7f8584061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f859c007fd0 tx=0x7f859c007bb0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.075+0000 7f85957fa700 1 -- 172.21.15.33:0/1818403758 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f859804d260 con 0x7f85a00bfb90 2023-11-06T21:00:06.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:06.305+0000 7f85a77ff700 1 -- 172.21.15.33:0/1818403758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f85a00bf220 con 0x7f85a00bfb90 2023-11-06T21:00:07.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.003+0000 7f85957fa700 1 -- 172.21.15.33:0/1818403758 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 32) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f8598003040 con 0x7f85a00bfb90 2023-11-06T21:00:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.005+0000 7f85957fa700 1 -- 172.21.15.33:0/1818403758 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v32) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8598042a50 con 0x7f85a00bfb90 2023-11-06T21:00:07.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 -- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f858405f270 msgr2=0x7f8584061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f858405f270 0x7f8584061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f859c007fd0 tx=0x7f859c007bb0).stop 2023-11-06T21:00:07.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 -- 172.21.15.33:0/1818403758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a00bfb90 msgr2=0x7f85a00c0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a00bfb90 0x7f85a00c0010 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f858c0045e0 tx=0x7f859801a000).stop 2023-11-06T21:00:07.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 -- 172.21.15.33:0/1818403758 shutdown_connections 2023-11-06T21:00:07.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f858405f270 0x7f8584061730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85a00bfb90 0x7f85a00c0010 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c004b80 0x7f85a0135f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 --2- 172.21.15.33:0/1818403758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85a00a8d20 0x7f85a00a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 -- 172.21.15.33:0/1818403758 >> 172.21.15.33:0/1818403758 conn(0x7f85a00c3d30 msgr2=0x7f85a00c4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 -- 172.21.15.33:0/1818403758 shutdown_connections 2023-11-06T21:00:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.008+0000 7f85a77ff700 1 -- 172.21.15.33:0/1818403758 wait complete. 2023-11-06T21:00:07.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T21:00:07.217 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:07.218+0000 7fd11a549680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:00:07.227 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:07.228+0000 7f7c79ac5680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:00:07.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.236+0000 7f8db35ff700 1 Processor -- start 2023-11-06T21:00:07.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.237+0000 7f8db35ff700 1 -- start start 2023-11-06T21:00:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.238+0000 7f8db35ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8dac1544f0 0x7f8dac0c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.238+0000 7f8db35ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8dac0c0c40 0x7f8dac0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.238+0000 7f8db35ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dac1556d0 con 0x7f8dac064bb0 2023-11-06T21:00:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.238+0000 7f8db35ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dac1548d0 con 0x7f8dac1544f0 2023-11-06T21:00:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.238+0000 7f8db35ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dac065040 con 0x7f8dac0c0c40 2023-11-06T21:00:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.238+0000 7f8db1b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8dac064bb0 0x7f8dac157f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38946/0 (socket says 172.21.15.33:38946) 2023-11-06T21:00:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.238+0000 7f8db1b9c700 1 -- 172.21.15.33:0/2669707562 learned_addr learned my addr 172.21.15.33:0/2669707562 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.238+0000 7f8db139b700 1 --2- 172.21.15.33:0/2669707562 >> v2:172.21.15.33:3300/0 conn(0x7f8dac1544f0 0x7f8dac0c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.238+0000 7f8db0b9a700 1 --2- 172.21.15.33:0/2669707562 >> v2:172.21.15.69:3300/0 conn(0x7f8dac0c0c40 0x7f8dac0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.239+0000 7f8da3fff700 1 -- 172.21.15.33:0/2669707562 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267903221 0 0) 0x7f8dac1556d0 con 0x7f8dac064bb0 2023-11-06T21:00:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.239+0000 7f8da3fff700 1 -- 172.21.15.33:0/2669707562 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d980033f0 con 0x7f8dac064bb0 2023-11-06T21:00:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.239+0000 7f8db139b700 1 -- 172.21.15.33:0/2669707562 >> v2:172.21.15.69:3300/0 conn(0x7f8dac0c0c40 msgr2=0x7f8dac0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.239+0000 7f8db139b700 1 --2- 172.21.15.33:0/2669707562 >> v2:172.21.15.69:3300/0 conn(0x7f8dac0c0c40 0x7f8dac0646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.239+0000 7f8db139b700 1 -- 172.21.15.33:0/2669707562 >> v1:172.21.15.33:6789/0 conn(0x7f8dac064bb0 legacy=0x7f8dac157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.239+0000 7f8db139b700 1 -- 172.21.15.33:0/2669707562 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac154ac0 con 0x7f8dac1544f0 2023-11-06T21:00:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.239+0000 7f8db139b700 1 --2- 172.21.15.33:0/2669707562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac1544f0 0x7f8dac0c0750 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f8dac0adab0 tx=0x7f8da8009060).ready entity=mon.0 client_cookie=3f21df0414ea88dc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.239+0000 7f8da3fff700 1 -- 172.21.15.33:0/2669707562 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800d4a0 con 0x7f8dac1544f0 2023-11-06T21:00:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.239+0000 7f8da3fff700 1 -- 172.21.15.33:0/2669707562 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da800d600 con 0x7f8dac1544f0 2023-11-06T21:00:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.239+0000 7f8da3fff700 1 -- 172.21.15.33:0/2669707562 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800d980 con 0x7f8dac1544f0 2023-11-06T21:00:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.240+0000 7f8db35ff700 1 -- 172.21.15.33:0/2669707562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac1544f0 msgr2=0x7f8dac0c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.240+0000 7f8db35ff700 1 --2- 172.21.15.33:0/2669707562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac1544f0 0x7f8dac0c0750 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f8dac0adab0 tx=0x7f8da8009060).stop 2023-11-06T21:00:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.240+0000 7f8db35ff700 1 -- 172.21.15.33:0/2669707562 shutdown_connections 2023-11-06T21:00:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.240+0000 7f8db35ff700 1 --2- 172.21.15.33:0/2669707562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac1544f0 0x7f8dac0c0750 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.240+0000 7f8db35ff700 1 --2- 172.21.15.33:0/2669707562 >> v2:172.21.15.69:3300/0 conn(0x7f8dac0c0c40 0x7f8dac0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.240+0000 7f8db35ff700 1 -- 172.21.15.33:0/2669707562 >> 172.21.15.33:0/2669707562 conn(0x7f8dac0cb900 msgr2=0x7f8dac0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.240+0000 7f8db35ff700 1 -- 172.21.15.33:0/2669707562 shutdown_connections 2023-11-06T21:00:07.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.240+0000 7f8db35ff700 1 -- 172.21.15.33:0/2669707562 wait complete. 2023-11-06T21:00:07.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.242+0000 7f8db35ff700 1 Processor -- start 2023-11-06T21:00:07.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.242+0000 7f8db35ff700 1 -- start start 2023-11-06T21:00:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db35ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac064bb0 0x7f8dac0c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db35ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dac0c0c40 0x7f8dac0af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db35ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dac1544f0 0x7f8dac1439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db35ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0afa40 con 0x7f8dac064bb0 2023-11-06T21:00:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db35ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0c37b0 con 0x7f8dac1544f0 2023-11-06T21:00:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db35ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0afba0 con 0x7f8dac0c0c40 2023-11-06T21:00:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db139b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac064bb0 0x7f8dac0c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db139b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac064bb0 0x7f8dac0c2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59994/0 (socket says 172.21.15.33:59994) 2023-11-06T21:00:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db139b700 1 -- 172.21.15.33:0/2662596482 learned_addr learned my addr 172.21.15.33:0/2662596482 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db1b9c700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dac1544f0 0x7f8dac1439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.243+0000 7f8db0b9a700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dac0c0c40 0x7f8dac0af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8db139b700 1 -- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dac0c0c40 msgr2=0x7f8dac0af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8db139b700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dac0c0c40 0x7f8dac0af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8db139b700 1 -- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dac1544f0 msgr2=0x7f8dac1439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8db139b700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dac1544f0 0x7f8dac1439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8db139b700 1 -- 172.21.15.33:0/2662596482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80092a0 con 0x7f8dac064bb0 2023-11-06T21:00:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8db139b700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac064bb0 0x7f8dac0c2230 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f8dac0843a0 tx=0x7f8da8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8da17fa700 1 -- 172.21.15.33:0/2662596482 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800dd40 con 0x7f8dac064bb0 2023-11-06T21:00:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8da17fa700 1 -- 172.21.15.33:0/2662596482 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da800d770 con 0x7f8dac064bb0 2023-11-06T21:00:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8da17fa700 1 -- 172.21.15.33:0/2662596482 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800dea0 con 0x7f8dac064bb0 2023-11-06T21:00:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8db35ff700 1 -- 172.21.15.33:0/2662596482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac0b06c0 con 0x7f8dac064bb0 2023-11-06T21:00:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.244+0000 7f8db35ff700 1 -- 172.21.15.33:0/2662596482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac0afe10 con 0x7f8dac064bb0 2023-11-06T21:00:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.246+0000 7f8db35ff700 1 -- 172.21.15.33:0/2662596482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac154da0 con 0x7f8dac064bb0 2023-11-06T21:00:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.246+0000 7f8da17fa700 1 -- 172.21.15.33:0/2662596482 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 32) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f8da801d8b0 con 0x7f8dac064bb0 2023-11-06T21:00:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.247+0000 7f8da17fa700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d9005f2c0 0x7f8d90061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.248+0000 7f8da17fa700 1 -- 172.21.15.33:0/2662596482 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8da807bde0 con 0x7f8dac064bb0 2023-11-06T21:00:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.248+0000 7f8db0b9a700 1 -- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d9005f2c0 msgr2=0x7f8d90061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.248+0000 7f8db0b9a700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d9005f2c0 0x7f8d90061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.252+0000 7f8da17fa700 1 -- 172.21.15.33:0/2662596482 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8da804ce70 con 0x7f8dac064bb0 2023-11-06T21:00:07.297 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:07.297+0000 7fd11a549680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:00:07.312 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:07.313+0000 7f7c79ac5680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:00:07.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.436+0000 7f8db35ff700 1 -- 172.21.15.33:0/2662596482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f8dac0aa700 con 0x7f8dac064bb0 2023-11-06T21:00:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.440+0000 7f8da17fa700 1 -- 172.21.15.33:0/2662596482 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v15)=0 v15) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8da8050490 con 0x7f8dac064bb0 2023-11-06T21:00:07.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 -- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d9005f2c0 msgr2=0x7f8d90061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:07.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d9005f2c0 0x7f8d90061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:07.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 -- 172.21.15.33:0/2662596482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac064bb0 msgr2=0x7f8dac0c2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac064bb0 0x7f8dac0c2230 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f8dac0843a0 tx=0x7f8da8016000).stop 2023-11-06T21:00:07.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 -- 172.21.15.33:0/2662596482 shutdown_connections 2023-11-06T21:00:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8d9005f2c0 0x7f8d90061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dac1544f0 0x7f8dac1439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac064bb0 0x7f8dac0c2230 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 --2- 172.21.15.33:0/2662596482 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dac0c0c40 0x7f8dac0af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 -- 172.21.15.33:0/2662596482 >> 172.21.15.33:0/2662596482 conn(0x7f8dac0cb900 msgr2=0x7f8dac156ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:07.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 -- 172.21.15.33:0/2662596482 shutdown_connections 2023-11-06T21:00:07.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.442+0000 7f8db35ff700 1 -- 172.21.15.33:0/2662596482 wait complete. 2023-11-06T21:00:07.460 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-11-06T21:00:07.627 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:07.627+0000 7fd11a549680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:00:07.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.672+0000 7f37c13ff700 1 Processor -- start 2023-11-06T21:00:07.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.672+0000 7f37c13ff700 1 -- start start 2023-11-06T21:00:07.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.673+0000 7f37c13ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f37bc153ed0 0x7f37bc158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.673+0000 7f37c13ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37bc0c8ee0 0x7f37bc0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.673+0000 7f37c13ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37bc0c9f80 con 0x7f37bc1542b0 2023-11-06T21:00:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.673+0000 7f37c13ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37bc0c6000 con 0x7f37bc153ed0 2023-11-06T21:00:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.673+0000 7f37c13ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0c9370 con 0x7f37bc0c8ee0 2023-11-06T21:00:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37baffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f37bc153ed0 0x7f37bc158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37bb7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f37bc1542b0 0x7f37bc0c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38950/0 (socket says 172.21.15.33:38950) 2023-11-06T21:00:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37bb7fe700 1 -- 172.21.15.33:0/823599174 learned_addr learned my addr 172.21.15.33:0/823599174 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37ba7fc700 1 --2- 172.21.15.33:0/823599174 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37bc0c8ee0 0x7f37bc0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37baffd700 1 -- 172.21.15.33:0/823599174 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37bc0c8ee0 msgr2=0x7f37bc0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37baffd700 1 --2- 172.21.15.33:0/823599174 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37bc0c8ee0 0x7f37bc0ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37baffd700 1 -- 172.21.15.33:0/823599174 >> v1:172.21.15.33:6789/0 conn(0x7f37bc1542b0 legacy=0x7f37bc0c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37baffd700 1 -- 172.21.15.33:0/823599174 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc0c9650 con 0x7f37bc153ed0 2023-11-06T21:00:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37b9ffb700 1 -- 172.21.15.33:0/823599174 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533135608 0 0) 0x7f37bc0c9f80 con 0x7f37bc1542b0 2023-11-06T21:00:07.676 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f7c79ac5680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:00:07.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37baffd700 1 --2- 172.21.15.33:0/823599174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc153ed0 0x7f37bc158600 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f37ac009760 tx=0x7f37ac009060).ready entity=mon.0 client_cookie=17ad4ae5856b70c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:07.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37b9ffb700 1 -- 172.21.15.33:0/823599174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00f510 con 0x7f37bc153ed0 2023-11-06T21:00:07.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37b9ffb700 1 -- 172.21.15.33:0/823599174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00f670 con 0x7f37bc153ed0 2023-11-06T21:00:07.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.674+0000 7f37b9ffb700 1 -- 172.21.15.33:0/823599174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00f9f0 con 0x7f37bc153ed0 2023-11-06T21:00:07.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.675+0000 7f37c13ff700 1 -- 172.21.15.33:0/823599174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc153ed0 msgr2=0x7f37bc158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.675+0000 7f37c13ff700 1 --2- 172.21.15.33:0/823599174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc153ed0 0x7f37bc158600 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f37ac009760 tx=0x7f37ac009060).stop 2023-11-06T21:00:07.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.675+0000 7f37c13ff700 1 -- 172.21.15.33:0/823599174 shutdown_connections 2023-11-06T21:00:07.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.675+0000 7f37c13ff700 1 --2- 172.21.15.33:0/823599174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc153ed0 0x7f37bc158600 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.675+0000 7f37c13ff700 1 --2- 172.21.15.33:0/823599174 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37bc0c8ee0 0x7f37bc0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.675+0000 7f37c13ff700 1 -- 172.21.15.33:0/823599174 >> 172.21.15.33:0/823599174 conn(0x7f37bc05e1b0 msgr2=0x7f37bc05e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:07.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.675+0000 7f37c13ff700 1 -- 172.21.15.33:0/823599174 shutdown_connections 2023-11-06T21:00:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.676+0000 7f37c13ff700 1 -- 172.21.15.33:0/823599174 wait complete. 2023-11-06T21:00:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.677+0000 7f37c13ff700 1 Processor -- start 2023-11-06T21:00:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.677+0000 7f37c13ff700 1 -- start start 2023-11-06T21:00:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37c13ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37bc153ed0 0x7f37bc14da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37c13ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37bc1542b0 0x7f37bc14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37c13ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc150300 0x7f37bc151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37c13ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc151390 con 0x7f37bc150300 2023-11-06T21:00:07.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37c13ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc150780 con 0x7f37bc153ed0 2023-11-06T21:00:07.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37c13ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc150a60 con 0x7f37bc1542b0 2023-11-06T21:00:07.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc150300 0x7f37bc151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:07.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc150300 0x7f37bc151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60006/0 (socket says 172.21.15.33:60006) 2023-11-06T21:00:07.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37bb7fe700 1 -- 172.21.15.33:0/970247341 learned_addr learned my addr 172.21.15.33:0/970247341 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:07.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37ba7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37bc1542b0 0x7f37bc14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:07.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.678+0000 7f37baffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37bc153ed0 0x7f37bc14da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:07.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37bb7fe700 1 -- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37bc1542b0 msgr2=0x7f37bc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37bb7fe700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37bc1542b0 0x7f37bc14fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37bb7fe700 1 -- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37bc153ed0 msgr2=0x7f37bc14da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:07.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37bb7fe700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37bc153ed0 0x7f37bc14da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:07.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37bb7fe700 1 -- 172.21.15.33:0/970247341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0092a0 con 0x7f37bc150300 2023-11-06T21:00:07.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37bb7fe700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37bc150300 0x7f37bc151860 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f37b400c7e0 tx=0x7f37b4007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:07.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4005d30 con 0x7f37bc150300 2023-11-06T21:00:07.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b4005e90 con 0x7f37bc150300 2023-11-06T21:00:07.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37c13ff700 1 -- 172.21.15.33:0/970247341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc150d40 con 0x7f37bc150300 2023-11-06T21:00:07.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4007320 con 0x7f37bc150300 2023-11-06T21:00:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.679+0000 7f37c13ff700 1 -- 172.21.15.33:0/970247341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc14df40 con 0x7f37bc150300 2023-11-06T21:00:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.681+0000 7f37c13ff700 1 -- 172.21.15.33:0/970247341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37bc06c6c0 con 0x7f37bc150300 2023-11-06T21:00:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.681+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 32) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f37b4007710 con 0x7f37bc150300 2023-11-06T21:00:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.683+0000 7f37a2ffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 0x7f37a8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.683+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f37b407a400 con 0x7f37bc150300 2023-11-06T21:00:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.683+0000 7f37baffd700 1 -- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 msgr2=0x7f37a8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.683+0000 7f37baffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 0x7f37a8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.687+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f37b404b430 con 0x7f37bc150300 2023-11-06T21:00:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.884+0000 7f37baffd700 1 -- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 msgr2=0x7f37a8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.884+0000 7f37baffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 0x7f37a8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:00:07.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:07.895+0000 7f37c13ff700 1 -- 172.21.15.33:0/970247341 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f37bc0c9930 con 0x7f37a805f2c0 2023-11-06T21:00:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:08.285+0000 7f37baffd700 1 -- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 msgr2=0x7f37a8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:08.285+0000 7f37baffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 0x7f37a8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T21:00:08.372 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:08.372+0000 7fd11a549680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:00:08.455 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:08.455+0000 7fd11a549680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:00:08.456 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:08.457+0000 7f7c79ac5680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:00:08.542 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:08.543+0000 7f7c79ac5680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:00:08.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:00:08.681 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:00:08.686 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:08.685+0000 7fd11a549680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:00:08.786 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:08.787+0000 7f7c79ac5680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:00:09.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:09.086+0000 7f37baffd700 1 -- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 msgr2=0x7f37a8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:09.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:09.086+0000 7f37baffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 0x7f37a8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T21:00:09.208 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:09.208+0000 7fd11a549680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:00:09.318 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:09.319+0000 7f7c79ac5680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:00:09.333 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:09.333+0000 7fd11a549680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:09.448 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:09.449+0000 7f7c79ac5680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:09.516 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:09.517+0000 7fd11a549680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:09.611 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:09.611+0000 7fd11a549680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:00:09.640 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:09.639+0000 7f7c79ac5680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:09.696 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:09.696+0000 7fd11a549680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:00:09.738 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:09.738+0000 7f7c79ac5680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:00:09.827 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:09.827+0000 7f7c79ac5680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:00:10.003 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:10.003+0000 7fd11a549680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:00:10.153 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:10.153+0000 7f7c79ac5680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:00:10.225 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:10.225+0000 7fd11a549680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:10.381 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:10.381+0000 7f7c79ac5680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:10.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:10.687+0000 7f37baffd700 1 -- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 msgr2=0x7f37a8061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:10.687+0000 7f37baffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 0x7f37a8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T21:00:11.024 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:11.023+0000 7fd11a549680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:00:11.111 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:11.110+0000 7fd11a549680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:00:11.205 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:11.204+0000 7fd11a549680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:00:11.219 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:11.219+0000 7f7c79ac5680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:00:11.310 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:11.309+0000 7f7c79ac5680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:00:11.388 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:11.387+0000 7fd11a549680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:00:11.407 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:11.407+0000 7f7c79ac5680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:00:11.476 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:11.475+0000 7fd11a549680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:00:11.599 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:11.599+0000 7f7c79ac5680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:00:11.618 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:11.618+0000 7fd11a549680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:00:11.691 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:11.691+0000 7f7c79ac5680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:00:11.747 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:11.748+0000 7fd11a549680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:11.839 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:11.838+0000 7f7c79ac5680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:00:11.974 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:11.974+0000 7f7c79ac5680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:12.093 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.093+0000 7fd11a549680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:00:12.182 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.182+0000 7fd11a549680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:00:12.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:12.194+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 33) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f37b401adb0 con 0x7f37bc150300 2023-11-06T21:00:12.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:12.194+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 msgr2=0x7f37a8061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:12.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:12.194+0000 7f37a2ffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 0x7f37a8061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:12.269 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.269+0000 7fd08f045700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:12.270 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.269+0000 7fd08f045700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:12.270 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.269+0000 7fd08f045700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:12.270 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.269+0000 7fd08f045700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:12.270 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.269+0000 7fd08f045700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:12.270 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.269+0000 7fd08b83e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:12.271 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.269+0000 7fd08b83e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:12.271 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.269+0000 7fd08b83e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:12.271 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.269+0000 7fd08b83e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:12.271 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:12.269+0000 7fd08b83e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:12.338 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:12.338+0000 7f7c79ac5680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:00:12.431 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:12.431+0000 7f7c79ac5680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:00:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.197+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 34) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f37b4078310 con 0x7f37bc150300 2023-11-06T21:00:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.197+0000 7f37a2ffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 0x7f37a8061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.197+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f37bc0c9930 con 0x7f37a805f2c0 2023-11-06T21:00:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.198+0000 7f37baffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 0x7f37a8061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:13.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.202+0000 7f37baffd700 1 --2- 172.21.15.33:0/970247341 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f37a805f2c0 0x7f37a8061700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f37ac009fd0 tx=0x7f37ac017000).ready entity=mgr.6598 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:13.205 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:13.205+0000 7fd097055700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-11-06T21:00:13.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.206+0000 7f37a2ffd700 1 -- 172.21.15.33:0/970247341 <== mgr.6598 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f37bc0c9930 con 0x7f37a805f2c0 2023-11-06T21:00:13.208 INFO:teuthology.orchestra.run.smithi033.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-11-06T21:00:13.217 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-06T21:00:13.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T21:00:13.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.465+0000 7f5dbd7ff700 1 Processor -- start 2023-11-06T21:00:13.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.465+0000 7f5dbd7ff700 1 -- start start 2023-11-06T21:00:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.466+0000 7f5dbd7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5db80c72c0 0x7f5db81553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.466+0000 7f5dbd7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5db80637f0 0x7f5db80c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.466+0000 7f5dbd7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db8152470 con 0x7f5db8157fa0 2023-11-06T21:00:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.466+0000 7f5dbd7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db8154990 con 0x7f5db80637f0 2023-11-06T21:00:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.466+0000 7f5dbd7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80c76a0 con 0x7f5db80c72c0 2023-11-06T21:00:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.466+0000 7f5db77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5db80637f0 0x7f5db80c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5db80637f0 0x7f5db80c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35912/0 (socket says 172.21.15.33:35912) 2023-11-06T21:00:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db77fe700 1 -- 172.21.15.33:0/3030247558 learned_addr learned my addr 172.21.15.33:0/3030247558 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db6ffd700 1 --2- 172.21.15.33:0/3030247558 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5db80c72c0 0x7f5db81553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db77fe700 1 -- 172.21.15.33:0/3030247558 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5db80c72c0 msgr2=0x7f5db81553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db77fe700 1 --2- 172.21.15.33:0/3030247558 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5db80c72c0 0x7f5db81553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db77fe700 1 -- 172.21.15.33:0/3030247558 >> v1:172.21.15.69:6789/0 conn(0x7f5db8157fa0 legacy=0x7f5db80630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db77fe700 1 -- 172.21.15.33:0/3030247558 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8158910 con 0x7f5db80637f0 2023-11-06T21:00:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db77fe700 1 --2- 172.21.15.33:0/3030247558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db80637f0 0x7f5db80c5be0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f5db0009f50 tx=0x7f5db0009840).ready entity=mon.0 client_cookie=d31aaadbab26dd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db5ffb700 1 -- 172.21.15.33:0/3030247558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db000e070 con 0x7f5db80637f0 2023-11-06T21:00:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db5ffb700 1 -- 172.21.15.33:0/3030247558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db000ba70 con 0x7f5db80637f0 2023-11-06T21:00:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.467+0000 7f5db5ffb700 1 -- 172.21.15.33:0/3030247558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db000bdf0 con 0x7f5db80637f0 2023-11-06T21:00:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.468+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/3030247558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db80637f0 msgr2=0x7f5db80c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.468+0000 7f5dbd7ff700 1 --2- 172.21.15.33:0/3030247558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db80637f0 0x7f5db80c5be0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f5db0009f50 tx=0x7f5db0009840).stop 2023-11-06T21:00:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.468+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/3030247558 shutdown_connections 2023-11-06T21:00:13.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.468+0000 7f5dbd7ff700 1 --2- 172.21.15.33:0/3030247558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db80637f0 0x7f5db80c5be0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.468+0000 7f5dbd7ff700 1 --2- 172.21.15.33:0/3030247558 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5db80c72c0 0x7f5db81553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.468+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/3030247558 >> 172.21.15.33:0/3030247558 conn(0x7f5db805ec20 msgr2=0x7f5db80cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.468+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/3030247558 shutdown_connections 2023-11-06T21:00:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.468+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/3030247558 wait complete. 2023-11-06T21:00:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.469+0000 7f5dbd7ff700 1 Processor -- start 2023-11-06T21:00:13.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.470+0000 7f5dbd7ff700 1 -- start start 2023-11-06T21:00:13.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.470+0000 7f5dbd7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db80637f0 0x7f5db80ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:13.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5dbd7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5db80c72c0 0x7f5db81511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:13.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5dbd7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8157fa0 0x7f5db80c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:13.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5dbd7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80aece0 con 0x7f5db8157fa0 2023-11-06T21:00:13.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5dbd7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db81516b0 con 0x7f5db80637f0 2023-11-06T21:00:13.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5dbd7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db8168430 con 0x7f5db80c72c0 2023-11-06T21:00:13.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5db77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8157fa0 0x7f5db80c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:13.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5db67fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5db80c72c0 0x7f5db81511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:13.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5db6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db80637f0 0x7f5db80ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:13.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5db77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8157fa0 0x7f5db80c4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35928/0 (socket says 172.21.15.33:35928) 2023-11-06T21:00:13.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5db77fe700 1 -- 172.21.15.33:0/223478705 learned_addr learned my addr 172.21.15.33:0/223478705 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:13.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5db77fe700 1 -- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5db80c72c0 msgr2=0x7f5db81511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:13.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5db77fe700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5db80c72c0 0x7f5db81511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:13.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5db77fe700 1 -- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db80637f0 msgr2=0x7f5db80ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5db77fe700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db80637f0 0x7f5db80ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.471+0000 7f5db77fe700 1 -- 172.21.15.33:0/223478705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db0009ae0 con 0x7f5db8157fa0 2023-11-06T21:00:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.472+0000 7f5db77fe700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8157fa0 0x7f5db80c4840 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f5db000b770 tx=0x7f5db0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.472+0000 7f5d9effd700 1 -- 172.21.15.33:0/223478705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0009500 con 0x7f5db8157fa0 2023-11-06T21:00:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.472+0000 7f5d9effd700 1 -- 172.21.15.33:0/223478705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db0007170 con 0x7f5db8157fa0 2023-11-06T21:00:13.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.472+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/223478705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db8168850 con 0x7f5db8157fa0 2023-11-06T21:00:13.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.472+0000 7f5d9effd700 1 -- 172.21.15.33:0/223478705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db001e6a0 con 0x7f5db8157fa0 2023-11-06T21:00:13.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.472+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/223478705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db8167d70 con 0x7f5db8157fa0 2023-11-06T21:00:13.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.473+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/223478705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db80c4150 con 0x7f5db8157fa0 2023-11-06T21:00:13.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.474+0000 7f5d9effd700 1 -- 172.21.15.33:0/223478705 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f5db0003040 con 0x7f5db8157fa0 2023-11-06T21:00:13.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.475+0000 7f5d9effd700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f080 0x7f5da4061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:13.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.475+0000 7f5d9effd700 1 -- 172.21.15.33:0/223478705 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f5db007b630 con 0x7f5db8157fa0 2023-11-06T21:00:13.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.475+0000 7f5db6ffd700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f080 0x7f5da4061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:13.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.476+0000 7f5db6ffd700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f080 0x7f5da4061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f5da800bbc0 tx=0x7f5da8008040).ready entity=mgr.6598 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:13.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.479+0000 7f5d9effd700 1 -- 172.21.15.33:0/223478705 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5db00504b0 con 0x7f5db8157fa0 2023-11-06T21:00:13.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:13.709+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/223478705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f5db8062b80 con 0x7f5db8157fa0 2023-11-06T21:00:14.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.204+0000 7f5d9effd700 1 -- 172.21.15.33:0/223478705 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f5db00794d0 con 0x7f5db8157fa0 2023-11-06T21:00:14.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.206+0000 7f5d9effd700 1 -- 172.21.15.33:0/223478705 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v35) v1 ==== 93+0+0 (secure 0 0 0) 0x7f5db0042a40 con 0x7f5db8157fa0 2023-11-06T21:00:14.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f080 msgr2=0x7f5da4061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:14.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f080 0x7f5da4061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f5da800bbc0 tx=0x7f5da8008040).stop 2023-11-06T21:00:14.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/223478705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8157fa0 msgr2=0x7f5db80c4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:14.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8157fa0 0x7f5db80c4840 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f5db000b770 tx=0x7f5db0009840).stop 2023-11-06T21:00:14.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/223478705 shutdown_connections 2023-11-06T21:00:14.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5da405f080 0x7f5da4061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:14.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5db8157fa0 0x7f5db80c4840 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:14.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db80637f0 0x7f5db80ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:14.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 --2- 172.21.15.33:0/223478705 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5db80c72c0 0x7f5db81511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:14.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/223478705 >> 172.21.15.33:0/223478705 conn(0x7f5db805ec20 msgr2=0x7f5db8151920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:14.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/223478705 shutdown_connections 2023-11-06T21:00:14.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.209+0000 7f5dbd7ff700 1 -- 172.21.15.33:0/223478705 wait complete. 2023-11-06T21:00:14.230 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T21:00:14.230 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:00:14.345 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:00:14.424 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:14.424+0000 7faa81349680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:00:14.431 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:14.431+0000 7f7773972680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:00:14.435 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T21:00:14.436 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T21:00:14.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:00:14.437 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T21:00:14.508 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:14.508+0000 7faa81349680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:00:14.511 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:14.511+0000 7f7773972680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:00:14.518 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T21:00:14.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:14.519 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T21:00:14.567 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:00:14.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.766+0000 7fb4737ff700 1 Processor -- start 2023-11-06T21:00:14.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.766+0000 7fb4737ff700 1 -- start start 2023-11-06T21:00:14.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.767+0000 7fb4737ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb46c0acda0 0x7fb46c0638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:14.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.767+0000 7fb4737ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0c4490 0x7fb46c0c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:14.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.767+0000 7fb4737ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c0c3380 con 0x7fb46c152250 2023-11-06T21:00:14.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.767+0000 7fb4737ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c0c20d0 con 0x7fb46c0acda0 2023-11-06T21:00:14.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.767+0000 7fb4737ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c05e520 con 0x7fb46c0c4490 2023-11-06T21:00:14.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.767+0000 7fb47159b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb46c152250 0x7fb46c0ac690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54890/0 (socket says 172.21.15.33:54890) 2023-11-06T21:00:14.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.767+0000 7fb47159b700 1 -- 172.21.15.33:0/2154712756 learned_addr learned my addr 172.21.15.33:0/2154712756 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:14.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.768+0000 7fb470d9a700 1 --2- 172.21.15.33:0/2154712756 >> v2:172.21.15.69:3300/0 conn(0x7fb46c0acda0 0x7fb46c0638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:14.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.768+0000 7fb471d9c700 1 --2- 172.21.15.33:0/2154712756 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0c4490 0x7fb46c0c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.768+0000 7fb463fff700 1 -- 172.21.15.33:0/2154712756 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129770628 0 0) 0x7fb46c0c3380 con 0x7fb46c152250 2023-11-06T21:00:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.768+0000 7fb463fff700 1 -- 172.21.15.33:0/2154712756 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4580033f0 con 0x7fb46c152250 2023-11-06T21:00:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.768+0000 7fb470d9a700 1 -- 172.21.15.33:0/2154712756 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0c4490 msgr2=0x7fb46c0c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:14.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.768+0000 7fb470d9a700 1 --2- 172.21.15.33:0/2154712756 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0c4490 0x7fb46c0c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:14.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.768+0000 7fb470d9a700 1 -- 172.21.15.33:0/2154712756 >> v1:172.21.15.33:6789/0 conn(0x7fb46c152250 legacy=0x7fb46c0ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:14.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.768+0000 7fb470d9a700 1 -- 172.21.15.33:0/2154712756 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c05e800 con 0x7fb46c0acda0 2023-11-06T21:00:14.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.768+0000 7fb471d9c700 1 --2- 172.21.15.33:0/2154712756 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0c4490 0x7fb46c0c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:14.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.768+0000 7fb470d9a700 1 --2- 172.21.15.33:0/2154712756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46c0acda0 0x7fb46c0638b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fb45c009760 tx=0x7fb45c009060).ready entity=mon.1 client_cookie=3a463c29c917a137 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:14.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb463fff700 1 -- 172.21.15.33:0/2154712756 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00d4a0 con 0x7fb46c0acda0 2023-11-06T21:00:14.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb463fff700 1 -- 172.21.15.33:0/2154712756 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c00d600 con 0x7fb46c0acda0 2023-11-06T21:00:14.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb463fff700 1 -- 172.21.15.33:0/2154712756 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00d980 con 0x7fb46c0acda0 2023-11-06T21:00:14.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb4737ff700 1 -- 172.21.15.33:0/2154712756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46c0acda0 msgr2=0x7fb46c0638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:14.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb4737ff700 1 --2- 172.21.15.33:0/2154712756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46c0acda0 0x7fb46c0638b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fb45c009760 tx=0x7fb45c009060).stop 2023-11-06T21:00:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb4737ff700 1 -- 172.21.15.33:0/2154712756 shutdown_connections 2023-11-06T21:00:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb4737ff700 1 --2- 172.21.15.33:0/2154712756 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0c4490 0x7fb46c0c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb4737ff700 1 --2- 172.21.15.33:0/2154712756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46c0acda0 0x7fb46c0638b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb4737ff700 1 -- 172.21.15.33:0/2154712756 >> 172.21.15.33:0/2154712756 conn(0x7fb46c0c7d60 msgr2=0x7fb46c0c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb4737ff700 1 -- 172.21.15.33:0/2154712756 shutdown_connections 2023-11-06T21:00:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.769+0000 7fb4737ff700 1 -- 172.21.15.33:0/2154712756 wait complete. 2023-11-06T21:00:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.771+0000 7fb4737ff700 1 Processor -- start 2023-11-06T21:00:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.771+0000 7fb4737ff700 1 -- start start 2023-11-06T21:00:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb4737ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0acda0 0x7fb46c0a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb4737ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46c0c4490 0x7fb46c0a8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb4737ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb46c152250 0x7fb46c163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb4737ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0a5d60 con 0x7fb46c152250 2023-11-06T21:00:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb4737ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0a6040 con 0x7fb46c0c4490 2023-11-06T21:00:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb4737ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0a6320 con 0x7fb46c0acda0 2023-11-06T21:00:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb471d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb46c152250 0x7fb46c163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb470d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46c0c4490 0x7fb46c0a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:14.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb47159b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0acda0 0x7fb46c0a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:14.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb470d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46c0c4490 0x7fb46c0a8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:43102/0 (socket says 172.21.15.33:43102) 2023-11-06T21:00:14.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.772+0000 7fb470d9a700 1 -- 172.21.15.33:0/3557636323 learned_addr learned my addr 172.21.15.33:0/3557636323 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:14.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb471d9c700 1 -- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0acda0 msgr2=0x7fb46c0a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:14.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb471d9c700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0acda0 0x7fb46c0a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:14.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb471d9c700 1 -- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46c0c4490 msgr2=0x7fb46c0a8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:14.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb471d9c700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46c0c4490 0x7fb46c0a8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:14.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb471d9c700 1 -- 172.21.15.33:0/3557636323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb464009a20 con 0x7fb46c152250 2023-11-06T21:00:14.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb47159b700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0acda0 0x7fb46c0a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:14.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb471d9c700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb46c152250 0x7fb46c163ed0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7fb464009f10 tx=0x7fb46400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:14.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb4617fa700 1 -- 172.21.15.33:0/3557636323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46400cb70 con 0x7fb46c152250 2023-11-06T21:00:14.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb4617fa700 1 -- 172.21.15.33:0/3557636323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46400f040 con 0x7fb46c152250 2023-11-06T21:00:14.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb4737ff700 1 -- 172.21.15.33:0/3557636323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c0092a0 con 0x7fb46c152250 2023-11-06T21:00:14.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb4617fa700 1 -- 172.21.15.33:0/3557636323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb464013400 con 0x7fb46c152250 2023-11-06T21:00:14.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.773+0000 7fb4737ff700 1 -- 172.21.15.33:0/3557636323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c0a68e0 con 0x7fb46c152250 2023-11-06T21:00:14.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.775+0000 7fb4737ff700 1 -- 172.21.15.33:0/3557636323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c064420 con 0x7fb46c152250 2023-11-06T21:00:14.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.775+0000 7fb4617fa700 1 -- 172.21.15.33:0/3557636323 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fb4640137f0 con 0x7fb46c152250 2023-11-06T21:00:14.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.777+0000 7fb4617fa700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb45005f300 0x7fb4500617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:14.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.777+0000 7fb4617fa700 1 -- 172.21.15.33:0/3557636323 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fb46407aea0 con 0x7fb46c152250 2023-11-06T21:00:14.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.777+0000 7fb47159b700 1 -- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb45005f300 msgr2=0x7fb4500617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:14.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.777+0000 7fb47159b700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb45005f300 0x7fb4500617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:14.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.780+0000 7fb4617fa700 1 -- 172.21.15.33:0/3557636323 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb46404bd60 con 0x7fb46c152250 2023-11-06T21:00:14.838 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:14.838+0000 7f7773972680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:00:14.867 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:14.867+0000 7faa81349680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:00:14.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.978+0000 7fb47159b700 1 -- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb45005f300 msgr2=0x7fb4500617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:14.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:14.978+0000 7fb47159b700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb45005f300 0x7fb4500617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:00:15.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.007+0000 7fb4737ff700 1 -- 172.21.15.33:0/3557636323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb46c0c7000 con 0x7fb46c152250 2023-11-06T21:00:15.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.008+0000 7fb4617fa700 1 -- 172.21.15.33:0/3557636323 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4491 (secure 0 0 0) 0x7fb46404f380 con 0x7fb46c152250 2023-11-06T21:00:15.009 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:15.009 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1797026087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1797026087},{"type":"v1","addr":"172.21.15.69:6836","nonce":1797026087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4199570083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4199570083},{"type":"v1","addr":"172.21.15.33:6834","nonce":4199570083}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2429309034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2429309034},{"type":"v1","addr":"172.21.15.33:6835","nonce":2429309034}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:57.774874+0000","modified":"2023-11-06T21:00:00.791809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6621":{"gid":6621,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/1260218648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1260218648},{"type":"v1","addr":"172.21.15.69:6837","nonce":1260218648}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-06T21:00:15.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.010+0000 7fb4737ff700 1 -- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb45005f300 msgr2=0x7fb4500617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:15.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.010+0000 7fb4737ff700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb45005f300 0x7fb4500617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.010+0000 7fb4737ff700 1 -- 172.21.15.33:0/3557636323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb46c152250 msgr2=0x7fb46c163ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.010+0000 7fb4737ff700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb46c152250 0x7fb46c163ed0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7fb464009f10 tx=0x7fb46400b040).stop 2023-11-06T21:00:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.011+0000 7fb4737ff700 1 -- 172.21.15.33:0/3557636323 shutdown_connections 2023-11-06T21:00:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.011+0000 7fb4737ff700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb45005f300 0x7fb4500617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.011+0000 7fb4737ff700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb46c0acda0 0x7fb46c0a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.011+0000 7fb4737ff700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb46c152250 0x7fb46c163ed0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.011+0000 7fb4737ff700 1 --2- 172.21.15.33:0/3557636323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46c0c4490 0x7fb46c0a8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.011+0000 7fb4737ff700 1 -- 172.21.15.33:0/3557636323 >> 172.21.15.33:0/3557636323 conn(0x7fb46c0c7d60 msgr2=0x7fb46c0632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:15.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.011+0000 7fb4737ff700 1 -- 172.21.15.33:0/3557636323 shutdown_connections 2023-11-06T21:00:15.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.011+0000 7fb4737ff700 1 -- 172.21.15.33:0/3557636323 wait complete. 2023-11-06T21:00:15.014 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 37 2023-11-06T21:00:15.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:00:15.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.228+0000 7f9ac7fff700 1 Processor -- start 2023-11-06T21:00:15.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.228+0000 7f9ac7fff700 1 -- start start 2023-11-06T21:00:15.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.230+0000 7f9ac7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ac00c38b0 0x7f9ac00a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.230+0000 7f9ac7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac00a6390 con 0x7f9ac00c3400 2023-11-06T21:00:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.230+0000 7f9ac7fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac00bc110 con 0x7f9ac0154140 2023-11-06T21:00:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.230+0000 7f9ac7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac00bc3f0 con 0x7f9ac00c38b0 2023-11-06T21:00:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.230+0000 7f9ac559a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9ac00c3400 0x7f9ac00a48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54902/0 (socket says 172.21.15.33:54902) 2023-11-06T21:00:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.230+0000 7f9ac559a700 1 -- 172.21.15.33:0/2025261678 learned_addr learned my addr 172.21.15.33:0/2025261678 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.230+0000 7f9ac659c700 1 --2- 172.21.15.33:0/2025261678 >> v2:172.21.15.33:3300/0 conn(0x7f9ac00c38b0 0x7f9ac00a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.231+0000 7f9ac659c700 1 -- 172.21.15.33:0/2025261678 >> v1:172.21.15.69:6789/0 conn(0x7f9ac0154140 legacy=0x7f9ac0064c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.231+0000 7f9ac659c700 1 -- 172.21.15.33:0/2025261678 >> v1:172.21.15.33:6789/0 conn(0x7f9ac00c3400 legacy=0x7f9ac00a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.231+0000 7f9ac659c700 1 -- 172.21.15.33:0/2025261678 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac00bc6d0 con 0x7f9ac00c38b0 2023-11-06T21:00:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.231+0000 7f9ac659c700 1 --2- 172.21.15.33:0/2025261678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac00c38b0 0x7f9ac00a4ff0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f9ab8009e60 tx=0x7f9ab8009780).ready entity=mon.0 client_cookie=5a1037cdd53fee87 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.231+0000 7f9ac4d99700 1 -- 172.21.15.33:0/2025261678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800e070 con 0x7f9ac00c38b0 2023-11-06T21:00:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.231+0000 7f9ac4d99700 1 -- 172.21.15.33:0/2025261678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab800bc30 con 0x7f9ac00c38b0 2023-11-06T21:00:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.231+0000 7f9ac4d99700 1 -- 172.21.15.33:0/2025261678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8012410 con 0x7f9ac00c38b0 2023-11-06T21:00:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.231+0000 7f9ac7fff700 1 -- 172.21.15.33:0/2025261678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac00c38b0 msgr2=0x7f9ac00a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.231+0000 7f9ac7fff700 1 --2- 172.21.15.33:0/2025261678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac00c38b0 0x7f9ac00a4ff0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f9ab8009e60 tx=0x7f9ab8009780).stop 2023-11-06T21:00:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.232+0000 7f9ac7fff700 1 -- 172.21.15.33:0/2025261678 shutdown_connections 2023-11-06T21:00:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.232+0000 7f9ac7fff700 1 --2- 172.21.15.33:0/2025261678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac00c38b0 0x7f9ac00a4ff0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.232+0000 7f9ac7fff700 1 -- 172.21.15.33:0/2025261678 >> 172.21.15.33:0/2025261678 conn(0x7f9ac0062710 msgr2=0x7f9ac0062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.232+0000 7f9ac7fff700 1 -- 172.21.15.33:0/2025261678 shutdown_connections 2023-11-06T21:00:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.232+0000 7f9ac7fff700 1 -- 172.21.15.33:0/2025261678 wait complete. 2023-11-06T21:00:15.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.233+0000 7f9ac7fff700 1 Processor -- start 2023-11-06T21:00:15.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.233+0000 7f9ac7fff700 1 -- start start 2023-11-06T21:00:15.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.234+0000 7f9ac7fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac00c3400 0x7f9ac00a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.234+0000 7f9ac7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ac00c38b0 0x7f9ac0164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.234+0000 7f9ac7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac0154140 0x7f9ac0164b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.234+0000 7f9ac7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac00be040 con 0x7f9ac0154140 2023-11-06T21:00:15.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.234+0000 7f9ac7fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac00a7d60 con 0x7f9ac00c38b0 2023-11-06T21:00:15.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.234+0000 7f9ac7fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac00a8040 con 0x7f9ac00c3400 2023-11-06T21:00:15.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac0154140 0x7f9ac0164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:15.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac0154140 0x7f9ac0164b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35978/0 (socket says 172.21.15.33:35978) 2023-11-06T21:00:15.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac659c700 1 -- 172.21.15.33:0/956073437 learned_addr learned my addr 172.21.15.33:0/956073437 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:15.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac5d9b700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac00c3400 0x7f9ac00a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:15.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac559a700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ac00c38b0 0x7f9ac0164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:15.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac659c700 1 -- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac00c3400 msgr2=0x7f9ac00a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac659c700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac00c3400 0x7f9ac00a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac659c700 1 -- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ac00c38b0 msgr2=0x7f9ac0164680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac659c700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ac00c38b0 0x7f9ac0164680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac659c700 1 -- 172.21.15.33:0/956073437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab8009a20 con 0x7f9ac0154140 2023-11-06T21:00:15.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.235+0000 7f9ac659c700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac0154140 0x7f9ac0164b70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f9ab800b6c0 tx=0x7f9ab8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:15.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.236+0000 7f9ab5ffb700 1 -- 172.21.15.33:0/956073437 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8006420 con 0x7f9ac0154140 2023-11-06T21:00:15.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.236+0000 7f9ab5ffb700 1 -- 172.21.15.33:0/956073437 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab8006580 con 0x7f9ac0154140 2023-11-06T21:00:15.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.236+0000 7f9ac7fff700 1 -- 172.21.15.33:0/956073437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac00a8320 con 0x7f9ac0154140 2023-11-06T21:00:15.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.236+0000 7f9ab5ffb700 1 -- 172.21.15.33:0/956073437 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8006920 con 0x7f9ac0154140 2023-11-06T21:00:15.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.236+0000 7f9ac7fff700 1 -- 172.21.15.33:0/956073437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac00a7940 con 0x7f9ac0154140 2023-11-06T21:00:15.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.237+0000 7f9ac7fff700 1 -- 172.21.15.33:0/956073437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac00a7b30 con 0x7f9ac0154140 2023-11-06T21:00:15.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.238+0000 7f9ab5ffb700 1 -- 172.21.15.33:0/956073437 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f9ab801d630 con 0x7f9ac0154140 2023-11-06T21:00:15.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.239+0000 7f9ab5ffb700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa405f300 0x7f9aa40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.239+0000 7f9ab5ffb700 1 -- 172.21.15.33:0/956073437 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f9ab807cac0 con 0x7f9ac0154140 2023-11-06T21:00:15.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.239+0000 7f9ac5d9b700 1 -- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa405f300 msgr2=0x7f9aa40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:15.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.239+0000 7f9ac5d9b700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa405f300 0x7f9aa40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:15.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.243+0000 7f9ab5ffb700 1 -- 172.21.15.33:0/956073437 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9ab804d980 con 0x7f9ac0154140 2023-11-06T21:00:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.425+0000 7f9ac7fff700 1 -- 172.21.15.33:0/956073437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ac00c54f0 con 0x7f9ac0154140 2023-11-06T21:00:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.426+0000 7f9ab5ffb700 1 -- 172.21.15.33:0/956073437 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+17349 (secure 0 0 0) 0x7f9ab8050fa0 con 0x7f9ac0154140 2023-11-06T21:00:15.427 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:15.427 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":100,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:00:12.186197+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:55.320441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:56.766396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1148377580":"2023-11-07T21:00:12.186151+0000","172.21.15.69:6833/114912":"2023-11-07T21:00:12.186151+0000","172.21.15.69:6832/114912":"2023-11-07T21:00:12.186151+0000","172.21.15.69:0/4025560788":"2023-11-07T21:00:12.186151+0000","172.21.15.69:0/1979880988":"2023-11-07T21:00:12.186151+0000","172.21.15.69:0/3973297670":"2023-11-07T21:00:12.186151+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:00:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.428+0000 7f9ac7fff700 1 -- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa405f300 msgr2=0x7f9aa40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.428+0000 7f9ac7fff700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa405f300 0x7f9aa40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.428+0000 7f9ac7fff700 1 -- 172.21.15.33:0/956073437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac0154140 msgr2=0x7f9ac0164b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.428+0000 7f9ac7fff700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac0154140 0x7f9ac0164b70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f9ab800b6c0 tx=0x7f9ab8009780).stop 2023-11-06T21:00:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.428+0000 7f9ac7fff700 1 -- 172.21.15.33:0/956073437 shutdown_connections 2023-11-06T21:00:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.428+0000 7f9ac7fff700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa405f300 0x7f9aa40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.428+0000 7f9ac7fff700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac00c3400 0x7f9ac00a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.428+0000 7f9ac7fff700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac0154140 0x7f9ac0164b70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.428+0000 7f9ac7fff700 1 --2- 172.21.15.33:0/956073437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ac00c38b0 0x7f9ac0164680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.428+0000 7f9ac7fff700 1 -- 172.21.15.33:0/956073437 >> 172.21.15.33:0/956073437 conn(0x7f9ac0062710 msgr2=0x7f9ac00c6140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.429+0000 7f9ac7fff700 1 -- 172.21.15.33:0/956073437 shutdown_connections 2023-11-06T21:00:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.429+0000 7f9ac7fff700 1 -- 172.21.15.33:0/956073437 wait complete. 2023-11-06T21:00:15.447 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T20:59:55.320441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T20:59:56.766396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T21:00:15.448 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T21:00:15.586 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:15.586+0000 7f7773972680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:00:15.668 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:15.667+0000 7faa81349680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:00:15.669 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:15.669+0000 7f7773972680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:00:15.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.679+0000 7f1b239ff700 1 Processor -- start 2023-11-06T21:00:15.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.680+0000 7f1b239ff700 1 -- start start 2023-11-06T21:00:15.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.681+0000 7f1b239ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1b1c0c4720 0x7f1b1c157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.681+0000 7f1b239ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 0x7f1b1c0c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.681+0000 7f1b239ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c064bf0 con 0x7f1b1c062710 2023-11-06T21:00:15.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.681+0000 7f1b239ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c0c4f60 con 0x7f1b1c0c4720 2023-11-06T21:00:15.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.681+0000 7f1b239ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c05e490 con 0x7f1b1c0c4b00 2023-11-06T21:00:15.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.682+0000 7f1b20f9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 0x7f1b1c0c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:15.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.682+0000 7f1b21f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1b1c062710 0x7f1b1c13d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54914/0 (socket says 172.21.15.33:54914) 2023-11-06T21:00:15.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.682+0000 7f1b2179b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1b1c0c4720 0x7f1b1c157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:15.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.682+0000 7f1b21f9c700 1 -- 172.21.15.33:0/2994451335 learned_addr learned my addr 172.21.15.33:0/2994451335 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:15.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.682+0000 7f1b2179b700 1 -- 172.21.15.33:0/2994451335 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 msgr2=0x7f1b1c0c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:15.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.682+0000 7f1b2179b700 1 --2- 172.21.15.33:0/2994451335 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 0x7f1b1c0c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.682+0000 7f1b2179b700 1 -- 172.21.15.33:0/2994451335 >> v1:172.21.15.33:6789/0 conn(0x7f1b1c062710 legacy=0x7f1b1c13d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.682+0000 7f1b13fff700 1 -- 172.21.15.33:0/2994451335 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457209274 0 0) 0x7f1b1c064bf0 con 0x7f1b1c062710 2023-11-06T21:00:15.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.682+0000 7f1b2179b700 1 -- 172.21.15.33:0/2994451335 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c062ba0 con 0x7f1b1c0c4720 2023-11-06T21:00:15.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.682+0000 7f1b20f9a700 1 --2- 172.21.15.33:0/2994451335 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 0x7f1b1c0c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:15.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.683+0000 7f1b2179b700 1 --2- 172.21.15.33:0/2994451335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b1c0c4720 0x7f1b1c157e20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f1b18009760 tx=0x7f1b18009060).ready entity=mon.1 client_cookie=662d5d623fe7d561 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:15.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.683+0000 7f1b13fff700 1 -- 172.21.15.33:0/2994451335 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800f510 con 0x7f1b1c0c4720 2023-11-06T21:00:15.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.683+0000 7f1b13fff700 1 -- 172.21.15.33:0/2994451335 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1800f670 con 0x7f1b1c0c4720 2023-11-06T21:00:15.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.683+0000 7f1b13fff700 1 -- 172.21.15.33:0/2994451335 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800f9f0 con 0x7f1b1c0c4720 2023-11-06T21:00:15.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.683+0000 7f1b239ff700 1 -- 172.21.15.33:0/2994451335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b1c0c4720 msgr2=0x7f1b1c157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.683+0000 7f1b239ff700 1 --2- 172.21.15.33:0/2994451335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b1c0c4720 0x7f1b1c157e20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f1b18009760 tx=0x7f1b18009060).stop 2023-11-06T21:00:15.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.683+0000 7f1b239ff700 1 -- 172.21.15.33:0/2994451335 shutdown_connections 2023-11-06T21:00:15.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.683+0000 7f1b239ff700 1 --2- 172.21.15.33:0/2994451335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b1c0c4720 0x7f1b1c157e20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.683+0000 7f1b239ff700 1 --2- 172.21.15.33:0/2994451335 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 0x7f1b1c0c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.683+0000 7f1b239ff700 1 -- 172.21.15.33:0/2994451335 >> 172.21.15.33:0/2994451335 conn(0x7f1b1c0c7dc0 msgr2=0x7f1b1c0c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:15.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.684+0000 7f1b239ff700 1 -- 172.21.15.33:0/2994451335 shutdown_connections 2023-11-06T21:00:15.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.684+0000 7f1b239ff700 1 -- 172.21.15.33:0/2994451335 wait complete. 2023-11-06T21:00:15.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.685+0000 7f1b239ff700 1 Processor -- start 2023-11-06T21:00:15.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.685+0000 7f1b239ff700 1 -- start start 2023-11-06T21:00:15.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b239ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b1c062710 0x7f1b1c156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b239ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b1c0c4720 0x7f1b1c1346a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b239ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 0x7f1b1c153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b239ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c135a90 con 0x7f1b1c062710 2023-11-06T21:00:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b239ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c134e80 con 0x7f1b1c0c4720 2023-11-06T21:00:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b239ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c135160 con 0x7f1b1c0c4b00 2023-11-06T21:00:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b2179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b1c062710 0x7f1b1c156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b20f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b1c0c4720 0x7f1b1c1346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b2179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b1c062710 0x7f1b1c156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35994/0 (socket says 172.21.15.33:35994) 2023-11-06T21:00:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b2179b700 1 -- 172.21.15.33:0/2724752460 learned_addr learned my addr 172.21.15.33:0/2724752460 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.686+0000 7f1b21f9c700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 0x7f1b1c153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b2179b700 1 -- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 msgr2=0x7f1b1c153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b2179b700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 0x7f1b1c153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b2179b700 1 -- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b1c0c4720 msgr2=0x7f1b1c1346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b2179b700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b1c0c4720 0x7f1b1c1346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b2179b700 1 -- 172.21.15.33:0/2724752460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c0092a0 con 0x7f1b1c062710 2023-11-06T21:00:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b20f9a700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b1c0c4720 0x7f1b1c1346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b2179b700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b1c062710 0x7f1b1c156540 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f1b1c068270 tx=0x7f1b18017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b117fa700 1 -- 172.21.15.33:0/2724752460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800f7e0 con 0x7f1b1c062710 2023-11-06T21:00:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b117fa700 1 -- 172.21.15.33:0/2724752460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1800f940 con 0x7f1b1c062710 2023-11-06T21:00:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b117fa700 1 -- 172.21.15.33:0/2724752460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1801b660 con 0x7f1b1c062710 2023-11-06T21:00:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.687+0000 7f1b239ff700 1 -- 172.21.15.33:0/2724752460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b180092a0 con 0x7f1b1c062710 2023-11-06T21:00:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.688+0000 7f1b239ff700 1 -- 172.21.15.33:0/2724752460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c135720 con 0x7f1b1c062710 2023-11-06T21:00:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.689+0000 7f1b239ff700 1 -- 172.21.15.33:0/2724752460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c0aa0d0 con 0x7f1b1c062710 2023-11-06T21:00:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.690+0000 7f1b117fa700 1 -- 172.21.15.33:0/2724752460 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f1b1801ba50 con 0x7f1b1c062710 2023-11-06T21:00:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.691+0000 7f1b117fa700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b0005f300 0x7f1b000617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.691+0000 7f1b117fa700 1 -- 172.21.15.33:0/2724752460 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f1b1807bf40 con 0x7f1b1c062710 2023-11-06T21:00:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.691+0000 7f1b20f9a700 1 -- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b0005f300 msgr2=0x7f1b000617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.691+0000 7f1b20f9a700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b0005f300 0x7f1b000617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.694+0000 7f1b117fa700 1 -- 172.21.15.33:0/2724752460 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1b1804cf10 con 0x7f1b1c062710 2023-11-06T21:00:15.759 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:15.759+0000 7faa81349680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:00:15.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.876+0000 7f1b239ff700 1 -- 172.21.15.33:0/2724752460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1b1c069c20 con 0x7f1b1c062710 2023-11-06T21:00:15.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.877+0000 7f1b117fa700 1 -- 172.21.15.33:0/2724752460 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v100) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1b18006b30 con 0x7f1b1c062710 2023-11-06T21:00:15.878 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T21:00:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 -- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b0005f300 msgr2=0x7f1b000617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b0005f300 0x7f1b000617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 -- 172.21.15.33:0/2724752460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b1c062710 msgr2=0x7f1b1c156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b1c062710 0x7f1b1c156540 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f1b1c068270 tx=0x7f1b18017000).stop 2023-11-06T21:00:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 -- 172.21.15.33:0/2724752460 shutdown_connections 2023-11-06T21:00:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b0005f300 0x7f1b000617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b1c0c4b00 0x7f1b1c153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b1c062710 0x7f1b1c156540 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 --2- 172.21.15.33:0/2724752460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b1c0c4720 0x7f1b1c1346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 -- 172.21.15.33:0/2724752460 >> 172.21.15.33:0/2724752460 conn(0x7f1b1c0c7dc0 msgr2=0x7f1b1c0acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 -- 172.21.15.33:0/2724752460 shutdown_connections 2023-11-06T21:00:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:15.879+0000 7f1b239ff700 1 -- 172.21.15.33:0/2724752460 wait complete. 2023-11-06T21:00:15.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T21:00:15.899 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:15.900+0000 7f7773972680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:00:16.008 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:16.008+0000 7faa81349680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:00:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.102+0000 7fe2d2fff700 1 Processor -- start 2023-11-06T21:00:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.102+0000 7fe2d2fff700 1 -- start start 2023-11-06T21:00:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.103+0000 7fe2d2fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe2cc1544f0 0x7fe2cc0c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.103+0000 7fe2d2fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2cc0c0c40 0x7fe2cc0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.103+0000 7fe2d2fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2cc1556d0 con 0x7fe2cc064bb0 2023-11-06T21:00:16.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.103+0000 7fe2d2fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2cc1548d0 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.103+0000 7fe2d2fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc065030 con 0x7fe2cc1544f0 2023-11-06T21:00:16.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.104+0000 7fe2cbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2cc0c0c40 0x7fe2cc0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.104+0000 7fe2cbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2cc0c0c40 0x7fe2cc0646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36000/0 (socket says 172.21.15.33:36000) 2023-11-06T21:00:16.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.104+0000 7fe2d0d9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe2cc1544f0 0x7fe2cc0c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.104+0000 7fe2cbfff700 1 -- 172.21.15.33:0/707509001 learned_addr learned my addr 172.21.15.33:0/707509001 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:16.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.104+0000 7fe2cbfff700 1 -- 172.21.15.33:0/707509001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe2cc1544f0 msgr2=0x7fe2cc0c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:16.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.104+0000 7fe2cbfff700 1 --2- 172.21.15.33:0/707509001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe2cc1544f0 0x7fe2cc0c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.104+0000 7fe2cbfff700 1 -- 172.21.15.33:0/707509001 >> v1:172.21.15.33:6789/0 conn(0x7fe2cc064bb0 legacy=0x7fe2cc157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.104+0000 7fe2cbfff700 1 -- 172.21.15.33:0/707509001 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc154ac0 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.104+0000 7fe2cbfff700 1 --2- 172.21.15.33:0/707509001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 0x7fe2cc0646c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fe2b8006fd0 tx=0x7fe2b80090f0).ready entity=mon.0 client_cookie=50c5ae6c817045 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:16.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.105+0000 7fe2cb7fe700 1 -- 172.21.15.33:0/707509001 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b800b580 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.105+0000 7fe2cb7fe700 1 -- 172.21.15.33:0/707509001 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b800b6e0 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.105+0000 7fe2cb7fe700 1 -- 172.21.15.33:0/707509001 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b800ba60 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.105+0000 7fe2d2fff700 1 -- 172.21.15.33:0/707509001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 msgr2=0x7fe2cc0646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.105+0000 7fe2d2fff700 1 --2- 172.21.15.33:0/707509001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 0x7fe2cc0646c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fe2b8006fd0 tx=0x7fe2b80090f0).stop 2023-11-06T21:00:16.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.105+0000 7fe2d2fff700 1 -- 172.21.15.33:0/707509001 shutdown_connections 2023-11-06T21:00:16.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.105+0000 7fe2d2fff700 1 --2- 172.21.15.33:0/707509001 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe2cc1544f0 0x7fe2cc0c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.105+0000 7fe2d2fff700 1 --2- 172.21.15.33:0/707509001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 0x7fe2cc0646c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.105+0000 7fe2d2fff700 1 -- 172.21.15.33:0/707509001 >> 172.21.15.33:0/707509001 conn(0x7fe2cc0cb900 msgr2=0x7fe2cc0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:16.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.105+0000 7fe2d2fff700 1 -- 172.21.15.33:0/707509001 shutdown_connections 2023-11-06T21:00:16.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.106+0000 7fe2d2fff700 1 -- 172.21.15.33:0/707509001 wait complete. 2023-11-06T21:00:16.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.108+0000 7fe2d2fff700 1 Processor -- start 2023-11-06T21:00:16.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.108+0000 7fe2d2fff700 1 -- start start 2023-11-06T21:00:16.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2d2fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2cc064bb0 0x7fe2cc0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2d2fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 0x7fe2cc0a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2d2fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe2cc1544f0 0x7fe2cc148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2d2fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0aefc0 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2d2fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0a2bd0 con 0x7fe2cc064bb0 2023-11-06T21:00:16.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2d2fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0ae3b0 con 0x7fe2cc1544f0 2023-11-06T21:00:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 0x7fe2cc0a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 0x7fe2cc0a26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36004/0 (socket says 172.21.15.33:36004) 2023-11-06T21:00:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2cbfff700 1 -- 172.21.15.33:0/2315022968 learned_addr learned my addr 172.21.15.33:0/2315022968 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2d159c700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe2cc1544f0 0x7fe2cc148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.110+0000 7fe2d0d9b700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2cc064bb0 0x7fe2cc0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2cbfff700 1 -- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe2cc1544f0 msgr2=0x7fe2cc148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2cbfff700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe2cc1544f0 0x7fe2cc148c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2cbfff700 1 -- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2cc064bb0 msgr2=0x7fe2cc0addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2cbfff700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2cc064bb0 0x7fe2cc0addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2cbfff700 1 -- 172.21.15.33:0/2315022968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b8009370 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2cbfff700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 0x7fe2cc0a26e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7fe2b8006df0 tx=0x7fe2b8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2c8ff9700 1 -- 172.21.15.33:0/2315022968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8000bc0 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2c8ff9700 1 -- 172.21.15.33:0/2315022968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b800b850 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2c8ff9700 1 -- 172.21.15.33:0/2315022968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8000d20 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2d2fff700 1 -- 172.21.15.33:0/2315022968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2cc0ae690 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.111+0000 7fe2d2fff700 1 -- 172.21.15.33:0/2315022968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2cc0aec50 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.113+0000 7fe2d2fff700 1 -- 172.21.15.33:0/2315022968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2cc0af130 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.113+0000 7fe2c8ff9700 1 -- 172.21.15.33:0/2315022968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fe2b801c8c0 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.114+0000 7fe2c8ff9700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe2b005f300 0x7fe2b00617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.114+0000 7fe2c8ff9700 1 -- 172.21.15.33:0/2315022968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fe2b807c210 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.114+0000 7fe2d0d9b700 1 -- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe2b005f300 msgr2=0x7fe2b00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:16.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.114+0000 7fe2d0d9b700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe2b005f300 0x7fe2b00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:16.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.118+0000 7fe2c8ff9700 1 -- 172.21.15.33:0/2315022968 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe2b804d0e0 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.297+0000 7fe2d2fff700 1 -- 172.21.15.33:0/2315022968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe2cc0c8760 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.297+0000 7fe2c8ff9700 1 -- 172.21.15.33:0/2315022968 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v100) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe2b802a0a0 con 0x7fe2cc0c0c40 2023-11-06T21:00:16.298 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T21:00:16.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.299+0000 7fe2d2fff700 1 -- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe2b005f300 msgr2=0x7fe2b00617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:16.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.299+0000 7fe2d2fff700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe2b005f300 0x7fe2b00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:16.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.299+0000 7fe2d2fff700 1 -- 172.21.15.33:0/2315022968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 msgr2=0x7fe2cc0a26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.299+0000 7fe2d2fff700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 0x7fe2cc0a26e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7fe2b8006df0 tx=0x7fe2b8017000).stop 2023-11-06T21:00:16.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.300+0000 7fe2d2fff700 1 -- 172.21.15.33:0/2315022968 shutdown_connections 2023-11-06T21:00:16.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.300+0000 7fe2d2fff700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe2b005f300 0x7fe2b00617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:16.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.300+0000 7fe2d2fff700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe2cc1544f0 0x7fe2cc148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.300+0000 7fe2d2fff700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2cc064bb0 0x7fe2cc0addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.300+0000 7fe2d2fff700 1 --2- 172.21.15.33:0/2315022968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc0c0c40 0x7fe2cc0a26e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.300+0000 7fe2d2fff700 1 -- 172.21.15.33:0/2315022968 >> 172.21.15.33:0/2315022968 conn(0x7fe2cc0cb900 msgr2=0x7fe2cc0c6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:16.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.300+0000 7fe2d2fff700 1 -- 172.21.15.33:0/2315022968 shutdown_connections 2023-11-06T21:00:16.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.300+0000 7fe2d2fff700 1 -- 172.21.15.33:0/2315022968 wait complete. 2023-11-06T21:00:16.319 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T21:00:16.408 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:16.408+0000 7f7773972680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:00:16.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.519+0000 7f6f0c1ff700 1 Processor -- start 2023-11-06T21:00:16.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.520+0000 7f6f0c1ff700 1 -- start start 2023-11-06T21:00:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.521+0000 7f6f0c1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f040c3b60 0x7f6f041520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.521+0000 7f6f0c1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f041525e0 con 0x7f6f041342f0 2023-11-06T21:00:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.521+0000 7f6f0c1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f04154f90 con 0x7f6f04133f10 2023-11-06T21:00:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.521+0000 7f6f0c1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0414bc20 con 0x7f6f040c3b60 2023-11-06T21:00:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.521+0000 7f6f0979a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f040c3b60 0x7f6f041520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.521+0000 7f6f0979a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f040c3b60 0x7f6f041520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36012/0 (socket says 172.21.15.33:36012) 2023-11-06T21:00:16.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.521+0000 7f6f0a79c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6f041342f0 0x7f6f04069530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54938/0 (socket says 172.21.15.33:54938) 2023-11-06T21:00:16.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.521+0000 7f6f0979a700 1 -- 172.21.15.33:0/3653046078 learned_addr learned my addr 172.21.15.33:0/3653046078 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.522+0000 7f6f0979a700 1 -- 172.21.15.33:0/3653046078 >> v1:172.21.15.69:6789/0 conn(0x7f6f04133f10 legacy=0x7f6f040c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.522+0000 7f6f0979a700 1 -- 172.21.15.33:0/3653046078 >> v1:172.21.15.33:6789/0 conn(0x7f6f041342f0 legacy=0x7f6f04069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.522+0000 7f6f0979a700 1 -- 172.21.15.33:0/3653046078 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f04155270 con 0x7f6f040c3b60 2023-11-06T21:00:16.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.522+0000 7f6f08f99700 1 -- 172.21.15.33:0/3653046078 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777389446 0 0) 0x7f6f041525e0 con 0x7f6f041342f0 2023-11-06T21:00:16.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.522+0000 7f6f0979a700 1 --2- 172.21.15.33:0/3653046078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f040c3b60 0x7f6f041520f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f6ef4006fd0 tx=0x7f6ef40090f0).ready entity=mon.0 client_cookie=f80a79820f387624 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:16.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.522+0000 7f6f08f99700 1 -- 172.21.15.33:0/3653046078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400b600 con 0x7f6f040c3b60 2023-11-06T21:00:16.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.522+0000 7f6f08f99700 1 -- 172.21.15.33:0/3653046078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef400b760 con 0x7f6f040c3b60 2023-11-06T21:00:16.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.522+0000 7f6f08f99700 1 -- 172.21.15.33:0/3653046078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400bae0 con 0x7f6f040c3b60 2023-11-06T21:00:16.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.523+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/3653046078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f040c3b60 msgr2=0x7f6f041520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.523+0000 7f6f0c1ff700 1 --2- 172.21.15.33:0/3653046078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f040c3b60 0x7f6f041520f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f6ef4006fd0 tx=0x7f6ef40090f0).stop 2023-11-06T21:00:16.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.523+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/3653046078 shutdown_connections 2023-11-06T21:00:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.523+0000 7f6f0c1ff700 1 --2- 172.21.15.33:0/3653046078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f040c3b60 0x7f6f041520f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.523+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/3653046078 >> 172.21.15.33:0/3653046078 conn(0x7f6f04062710 msgr2=0x7f6f04062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.523+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/3653046078 shutdown_connections 2023-11-06T21:00:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.523+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/3653046078 wait complete. 2023-11-06T21:00:16.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.525+0000 7f6f0c1ff700 1 Processor -- start 2023-11-06T21:00:16.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.525+0000 7f6f0c1ff700 1 -- start start 2023-11-06T21:00:16.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f0c1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f040c3b60 0x7f6f04136180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f0c1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f04133f10 0x7f6f040ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f0c1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f041342f0 0x7f6f04131470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f0c1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04135d60 con 0x7f6f04133f10 2023-11-06T21:00:16.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f0c1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04135170 con 0x7f6f040c3b60 2023-11-06T21:00:16.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f0c1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f041353c0 con 0x7f6f041342f0 2023-11-06T21:00:16.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f09f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f040c3b60 0x7f6f04136180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f0979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f04133f10 0x7f6f040ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f09f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f040c3b60 0x7f6f04136180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:43158/0 (socket says 172.21.15.33:43158) 2023-11-06T21:00:16.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f09f9b700 1 -- 172.21.15.33:0/266097019 learned_addr learned my addr 172.21.15.33:0/266097019 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:16.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.526+0000 7f6f0a79c700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f041342f0 0x7f6f04131470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6f09f9b700 1 -- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f041342f0 msgr2=0x7f6f04131470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6f09f9b700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f041342f0 0x7f6f04131470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6f09f9b700 1 -- 172.21.15.33:0/266097019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f04133f10 msgr2=0x7f6f040ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6f09f9b700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f04133f10 0x7f6f040ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6f09f9b700 1 -- 172.21.15.33:0/266097019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4009370 con 0x7f6f040c3b60 2023-11-06T21:00:16.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6f0979a700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f04133f10 0x7f6f040ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:16.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6f09f9b700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f040c3b60 0x7f6f04136180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f6f0000c4c0 tx=0x7f6f00007e00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:16.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6ef9ffb700 1 -- 172.21.15.33:0/266097019 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00007020 con 0x7f6f040c3b60 2023-11-06T21:00:16.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6ef9ffb700 1 -- 172.21.15.33:0/266097019 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f00001e90 con 0x7f6f040c3b60 2023-11-06T21:00:16.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6ef9ffb700 1 -- 172.21.15.33:0/266097019 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f000073b0 con 0x7f6f040c3b60 2023-11-06T21:00:16.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/266097019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f041356a0 con 0x7f6f040c3b60 2023-11-06T21:00:16.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.527+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/266097019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f040ab850 con 0x7f6f040c3b60 2023-11-06T21:00:16.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.528+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/266097019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f04135fd0 con 0x7f6f040c3b60 2023-11-06T21:00:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.529+0000 7f6ef9ffb700 1 -- 172.21.15.33:0/266097019 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f6f000068b0 con 0x7f6f040c3b60 2023-11-06T21:00:16.535 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:16.533+0000 7f7773972680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.530+0000 7f6ef9ffb700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ee805f330 0x7f6ee80617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.530+0000 7f6ef9ffb700 1 -- 172.21.15.33:0/266097019 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f6f00079f40 con 0x7f6f040c3b60 2023-11-06T21:00:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.530+0000 7f6f0979a700 1 -- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ee805f330 msgr2=0x7f6ee80617f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.530+0000 7f6f0979a700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ee805f330 0x7f6ee80617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:16.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.534+0000 7f6ef9ffb700 1 -- 172.21.15.33:0/266097019 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6f0004ae10 con 0x7f6f040c3b60 2023-11-06T21:00:16.565 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:16.564+0000 7faa81349680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:00:16.698 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:16.698+0000 7faa81349680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.716+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/266097019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f040659f0 con 0x7f6f040c3b60 2023-11-06T21:00:16.718 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:16.718+0000 7f7773972680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:16.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.717+0000 7f6ef9ffb700 1 -- 172.21.15.33:0/266097019 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f0004e430 con 0x7f6f040c3b60 2023-11-06T21:00:16.719 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T21:00:16.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ee805f330 msgr2=0x7f6ee80617f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:16.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ee805f330 0x7f6ee80617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:16.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f040c3b60 msgr2=0x7f6f04136180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f040c3b60 0x7f6f04136180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f6f0000c4c0 tx=0x7f6f00007e00).stop 2023-11-06T21:00:16.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/266097019 shutdown_connections 2023-11-06T21:00:16.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6ee805f330 0x7f6ee80617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:16.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f041342f0 0x7f6f04131470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f040c3b60 0x7f6f04136180 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 --2- 172.21.15.33:0/266097019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f04133f10 0x7f6f040ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/266097019 >> 172.21.15.33:0/266097019 conn(0x7f6f04062710 msgr2=0x7f6f04133410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:16.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/266097019 shutdown_connections 2023-11-06T21:00:16.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.719+0000 7f6f0c1ff700 1 -- 172.21.15.33:0/266097019 wait complete. 2023-11-06T21:00:16.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T21:00:16.813 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:16.812+0000 7f7773972680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:00:16.894 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:16.893+0000 7faa81349680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:16.897 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:16.897+0000 7f7773972680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:00:16.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.942+0000 7f26243ff700 1 Processor -- start 2023-11-06T21:00:16.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.942+0000 7f26243ff700 1 -- start start 2023-11-06T21:00:16.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.944+0000 7f26243ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f261c151f20 0x7f261c0ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.944+0000 7f26243ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f261c0c2d50 0x7f261c152300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.944+0000 7f26243ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261c0c3cb0 con 0x7f261c0c4560 2023-11-06T21:00:16.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.944+0000 7f26243ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261c0c31d0 con 0x7f261c0c2d50 2023-11-06T21:00:16.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.944+0000 7f26243ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261c062940 con 0x7f261c151f20 2023-11-06T21:00:16.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.944+0000 7f262199a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f261c0c2d50 0x7f261c152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.944+0000 7f262299c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f261c0c4560 0x7f261c064bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54950/0 (socket says 172.21.15.33:54950) 2023-11-06T21:00:16.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.944+0000 7f262299c700 1 -- 172.21.15.33:0/3830452850 learned_addr learned my addr 172.21.15.33:0/3830452850 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:16.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.944+0000 7f262219b700 1 --2- 172.21.15.33:0/3830452850 >> v2:172.21.15.69:3300/0 conn(0x7f261c151f20 0x7f261c0ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.945+0000 7f262199a700 1 -- 172.21.15.33:0/3830452850 >> v2:172.21.15.69:3300/0 conn(0x7f261c151f20 msgr2=0x7f261c0ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:16.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.945+0000 7f262199a700 1 --2- 172.21.15.33:0/3830452850 >> v2:172.21.15.69:3300/0 conn(0x7f261c151f20 0x7f261c0ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.945+0000 7f262199a700 1 -- 172.21.15.33:0/3830452850 >> v1:172.21.15.33:6789/0 conn(0x7f261c0c4560 legacy=0x7f261c064bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.945+0000 7f262199a700 1 -- 172.21.15.33:0/3830452850 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c05e860 con 0x7f261c0c2d50 2023-11-06T21:00:16.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.945+0000 7f2621199700 1 -- 172.21.15.33:0/3830452850 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808131850 0 0) 0x7f2614001490 con 0x7f261c0c4560 2023-11-06T21:00:16.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.945+0000 7f262199a700 1 --2- 172.21.15.33:0/3830452850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c0c2d50 0x7f261c152300 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f261c0c5130 tx=0x7f260c009060).ready entity=mon.0 client_cookie=da8267f7e4e96650 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:16.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.945+0000 7f2621199700 1 -- 172.21.15.33:0/3830452850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00f510 con 0x7f261c0c2d50 2023-11-06T21:00:16.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.945+0000 7f2621199700 1 -- 172.21.15.33:0/3830452850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c00f670 con 0x7f261c0c2d50 2023-11-06T21:00:16.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.945+0000 7f2621199700 1 -- 172.21.15.33:0/3830452850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00f9f0 con 0x7f261c0c2d50 2023-11-06T21:00:16.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.946+0000 7f26243ff700 1 -- 172.21.15.33:0/3830452850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c0c2d50 msgr2=0x7f261c152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.946+0000 7f26243ff700 1 --2- 172.21.15.33:0/3830452850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c0c2d50 0x7f261c152300 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f261c0c5130 tx=0x7f260c009060).stop 2023-11-06T21:00:16.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.946+0000 7f26243ff700 1 -- 172.21.15.33:0/3830452850 shutdown_connections 2023-11-06T21:00:16.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.946+0000 7f26243ff700 1 --2- 172.21.15.33:0/3830452850 >> v2:172.21.15.69:3300/0 conn(0x7f261c151f20 0x7f261c0ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.946+0000 7f26243ff700 1 --2- 172.21.15.33:0/3830452850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c0c2d50 0x7f261c152300 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.946+0000 7f26243ff700 1 -- 172.21.15.33:0/3830452850 >> 172.21.15.33:0/3830452850 conn(0x7f261c0c7dc0 msgr2=0x7f261c0c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:16.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.946+0000 7f26243ff700 1 -- 172.21.15.33:0/3830452850 shutdown_connections 2023-11-06T21:00:16.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.946+0000 7f26243ff700 1 -- 172.21.15.33:0/3830452850 wait complete. 2023-11-06T21:00:16.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.947+0000 7f26243ff700 1 Processor -- start 2023-11-06T21:00:16.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.948+0000 7f26243ff700 1 -- start start 2023-11-06T21:00:16.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f26243ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c0c2d50 0x7f261c14d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f26243ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f261c0c4560 0x7f261c0c0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f26243ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c151f20 0x7f261c0c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f26243ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c0c1240 con 0x7f261c151f20 2023-11-06T21:00:16.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f26243ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c0aa5d0 con 0x7f261c0c2d50 2023-11-06T21:00:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f26243ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c1655f0 con 0x7f261c0c4560 2023-11-06T21:00:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f262299c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c151f20 0x7f261c0c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f262299c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c151f20 0x7f261c0c0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36026/0 (socket says 172.21.15.33:36026) 2023-11-06T21:00:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f262299c700 1 -- 172.21.15.33:0/3313682057 learned_addr learned my addr 172.21.15.33:0/3313682057 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:16.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f262219b700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c0c2d50 0x7f261c14d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f262199a700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f261c0c4560 0x7f261c0c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:16.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f262219b700 1 -- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f261c0c4560 msgr2=0x7f261c0c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f262219b700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f261c0c4560 0x7f261c0c0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f262219b700 1 -- 172.21.15.33:0/3313682057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c151f20 msgr2=0x7f261c0c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:16.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f262219b700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c151f20 0x7f261c0c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:16.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.949+0000 7f262219b700 1 -- 172.21.15.33:0/3313682057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c0092a0 con 0x7f261c0c2d50 2023-11-06T21:00:16.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.950+0000 7f262219b700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c0c2d50 0x7f261c14d670 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f261800e720 tx=0x7f2618009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:16.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.950+0000 7f26127fc700 1 -- 172.21.15.33:0/3313682057 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618012600 con 0x7f261c0c2d50 2023-11-06T21:00:16.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.950+0000 7f26127fc700 1 -- 172.21.15.33:0/3313682057 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2618012760 con 0x7f261c0c2d50 2023-11-06T21:00:16.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.950+0000 7f26243ff700 1 -- 172.21.15.33:0/3313682057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261c0aaac0 con 0x7f261c0c2d50 2023-11-06T21:00:16.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.950+0000 7f26127fc700 1 -- 172.21.15.33:0/3313682057 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618012ae0 con 0x7f261c0c2d50 2023-11-06T21:00:16.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.950+0000 7f26243ff700 1 -- 172.21.15.33:0/3313682057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261c0ab100 con 0x7f261c0c2d50 2023-11-06T21:00:16.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.951+0000 7f26243ff700 1 -- 172.21.15.33:0/3313682057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261c068740 con 0x7f261c0c2d50 2023-11-06T21:00:16.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.952+0000 7f26127fc700 1 -- 172.21.15.33:0/3313682057 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f261801b400 con 0x7f261c0c2d50 2023-11-06T21:00:16.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.952+0000 7f26127fc700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f260005f300 0x7f26000617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:16.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.953+0000 7f26127fc700 1 -- 172.21.15.33:0/3313682057 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f261807a870 con 0x7f261c0c2d50 2023-11-06T21:00:16.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.953+0000 7f262199a700 1 -- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f260005f300 msgr2=0x7f26000617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:16.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.953+0000 7f262199a700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f260005f300 0x7f26000617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:16.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:16.957+0000 7f26127fc700 1 -- 172.21.15.33:0/3313682057 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f261804b6e0 con 0x7f261c0c2d50 2023-11-06T21:00:17.000 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:16.999+0000 7faa81349680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:00:17.091 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:17.091+0000 7faa81349680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:00:17.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.139+0000 7f26243ff700 1 -- 172.21.15.33:0/3313682057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f261c0c5c10 con 0x7f261c0c2d50 2023-11-06T21:00:17.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.140+0000 7f26127fc700 1 -- 172.21.15.33:0/3313682057 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f261804ed00 con 0x7f261c0c2d50 2023-11-06T21:00:17.141 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T21:00:17.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 -- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f260005f300 msgr2=0x7f26000617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:17.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f260005f300 0x7f26000617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:17.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 -- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c0c2d50 msgr2=0x7f261c14d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:17.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c0c2d50 0x7f261c14d670 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f261800e720 tx=0x7f2618009060).stop 2023-11-06T21:00:17.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 -- 172.21.15.33:0/3313682057 shutdown_connections 2023-11-06T21:00:17.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f260005f300 0x7f26000617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:17.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c151f20 0x7f261c0c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c0c2d50 0x7f261c14d670 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 --2- 172.21.15.33:0/3313682057 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f261c0c4560 0x7f261c0c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 -- 172.21.15.33:0/3313682057 >> 172.21.15.33:0/3313682057 conn(0x7f261c0c7dc0 msgr2=0x7f261c0c5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:17.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 -- 172.21.15.33:0/3313682057 shutdown_connections 2023-11-06T21:00:17.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.142+0000 7f26243ff700 1 -- 172.21.15.33:0/3313682057 wait complete. 2023-11-06T21:00:17.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:00:17.208 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:17.208+0000 7f7773972680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:00:17.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.360+0000 7f47cc1ff700 1 Processor -- start 2023-11-06T21:00:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.360+0000 7f47cc1ff700 1 -- start start 2023-11-06T21:00:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.361+0000 7f47cc1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c1840 0x7f47c40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.361+0000 7f47cc1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f47c40c6460 0x7f47c40bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.361+0000 7f47cc1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c41554a0 con 0x7f47c40aa2d0 2023-11-06T21:00:17.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.361+0000 7f47cc1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c4154960 con 0x7f47c40c6460 2023-11-06T21:00:17.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.361+0000 7f47cc1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4155930 con 0x7f47c40c1840 2023-11-06T21:00:17.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.362+0000 7f47c9f9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c1840 0x7f47c40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.362+0000 7f47ca79c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f47c40c6460 0x7f47c40bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.362+0000 7f47c9f9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c1840 0x7f47c40a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:53740/0 (socket says 172.21.15.33:53740) 2023-11-06T21:00:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.362+0000 7f47c9f9b700 1 -- 172.21.15.33:0/3167476126 learned_addr learned my addr 172.21.15.33:0/3167476126 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.362+0000 7f47c9f9b700 1 -- 172.21.15.33:0/3167476126 >> v2:172.21.15.69:3300/0 conn(0x7f47c40c6460 msgr2=0x7f47c40bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.362+0000 7f47c9f9b700 1 --2- 172.21.15.33:0/3167476126 >> v2:172.21.15.69:3300/0 conn(0x7f47c40c6460 0x7f47c40bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.362+0000 7f47c9f9b700 1 -- 172.21.15.33:0/3167476126 >> v1:172.21.15.69:6789/0 conn(0x7f47c40aa2d0 legacy=0x7f47c40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.362+0000 7f47c9f9b700 1 -- 172.21.15.33:0/3167476126 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4154b70 con 0x7f47c40c1840 2023-11-06T21:00:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.362+0000 7f47ca79c700 1 --2- 172.21.15.33:0/3167476126 >> v2:172.21.15.69:3300/0 conn(0x7f47c40c6460 0x7f47c40bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.363+0000 7f47c9f9b700 1 --2- 172.21.15.33:0/3167476126 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c1840 0x7f47c40a9de0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f47c0006fd0 tx=0x7f47c00090f0).ready entity=mon.2 client_cookie=e7e7dcc9af04aec2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.363+0000 7f47c8f99700 1 -- 172.21.15.33:0/3167476126 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000b580 con 0x7f47c40c1840 2023-11-06T21:00:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.363+0000 7f47c8f99700 1 -- 172.21.15.33:0/3167476126 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c000b6e0 con 0x7f47c40c1840 2023-11-06T21:00:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.363+0000 7f47c8f99700 1 -- 172.21.15.33:0/3167476126 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000ba60 con 0x7f47c40c1840 2023-11-06T21:00:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.363+0000 7f47cc1ff700 1 -- 172.21.15.33:0/3167476126 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c1840 msgr2=0x7f47c40a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.363+0000 7f47cc1ff700 1 --2- 172.21.15.33:0/3167476126 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c1840 0x7f47c40a9de0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f47c0006fd0 tx=0x7f47c00090f0).stop 2023-11-06T21:00:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.363+0000 7f47cc1ff700 1 -- 172.21.15.33:0/3167476126 shutdown_connections 2023-11-06T21:00:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.363+0000 7f47cc1ff700 1 --2- 172.21.15.33:0/3167476126 >> v2:172.21.15.69:3300/0 conn(0x7f47c40c6460 0x7f47c40bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.363+0000 7f47cc1ff700 1 --2- 172.21.15.33:0/3167476126 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c1840 0x7f47c40a9de0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.363+0000 7f47cc1ff700 1 -- 172.21.15.33:0/3167476126 >> 172.21.15.33:0/3167476126 conn(0x7f47c405ec80 msgr2=0x7f47c40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.364+0000 7f47cc1ff700 1 -- 172.21.15.33:0/3167476126 shutdown_connections 2023-11-06T21:00:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.364+0000 7f47cc1ff700 1 -- 172.21.15.33:0/3167476126 wait complete. 2023-11-06T21:00:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.365+0000 7f47cc1ff700 1 Processor -- start 2023-11-06T21:00:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.365+0000 7f47cc1ff700 1 -- start start 2023-11-06T21:00:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.366+0000 7f47cc1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47c40aa2d0 0x7f47c40c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.366+0000 7f47cc1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40c1840 0x7f47c4166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.366+0000 7f47cc1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c6460 0x7f47c4166fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.366+0000 7f47cc1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40aae40 con 0x7f47c40c1840 2023-11-06T21:00:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.366+0000 7f47cc1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40ab6e0 con 0x7f47c40aa2d0 2023-11-06T21:00:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.366+0000 7f47cc1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41674c0 con 0x7f47c40c6460 2023-11-06T21:00:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.366+0000 7f47c979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40c1840 0x7f47c4166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47c979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40c1840 0x7f47c4166ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36028/0 (socket says 172.21.15.33:36028) 2023-11-06T21:00:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47c979a700 1 -- 172.21.15.33:0/853035979 learned_addr learned my addr 172.21.15.33:0/853035979 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47ca79c700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c6460 0x7f47c4166fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47c9f9b700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47c40aa2d0 0x7f47c40c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47c979a700 1 -- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c6460 msgr2=0x7f47c4166fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47c979a700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c6460 0x7f47c4166fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47c979a700 1 -- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47c40aa2d0 msgr2=0x7f47c40c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47c979a700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47c40aa2d0 0x7f47c40c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47c979a700 1 -- 172.21.15.33:0/853035979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc009a20 con 0x7f47c40c1840 2023-11-06T21:00:17.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47c979a700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40c1840 0x7f47c4166ae0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f47b400c440 tx=0x7f47b4007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:17.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.367+0000 7f47b9ffb700 1 -- 172.21.15.33:0/853035979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b400ce40 con 0x7f47c40c1840 2023-11-06T21:00:17.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.368+0000 7f47b9ffb700 1 -- 172.21.15.33:0/853035979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b4007020 con 0x7f47c40c1840 2023-11-06T21:00:17.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.368+0000 7f47cc1ff700 1 -- 172.21.15.33:0/853035979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c0009370 con 0x7f47c40c1840 2023-11-06T21:00:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.368+0000 7f47b9ffb700 1 -- 172.21.15.33:0/853035979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b40073c0 con 0x7f47c40c1840 2023-11-06T21:00:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.368+0000 7f47cc1ff700 1 -- 172.21.15.33:0/853035979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c40c4b60 con 0x7f47c40c1840 2023-11-06T21:00:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.369+0000 7f47cc1ff700 1 -- 172.21.15.33:0/853035979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c40ab850 con 0x7f47c40c1840 2023-11-06T21:00:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.369+0000 7f47b9ffb700 1 -- 172.21.15.33:0/853035979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f47b40068b0 con 0x7f47c40c1840 2023-11-06T21:00:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.370+0000 7f47b9ffb700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47a805f300 0x7f47a80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.370+0000 7f47b9ffb700 1 -- 172.21.15.33:0/853035979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f47b4079f20 con 0x7f47c40c1840 2023-11-06T21:00:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.370+0000 7f47c9f9b700 1 -- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47a805f300 msgr2=0x7f47a80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.370+0000 7f47c9f9b700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47a805f300 0x7f47a80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.374+0000 7f47b9ffb700 1 -- 172.21.15.33:0/853035979 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f47b404aef0 con 0x7f47c40c1840 2023-11-06T21:00:17.432 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:17.432+0000 7f7773972680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:17.443 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:17.443+0000 7faa81349680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:00:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.571+0000 7f47c9f9b700 1 -- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47a805f300 msgr2=0x7f47a80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.571+0000 7f47c9f9b700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47a805f300 0x7f47a80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:00:17.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.615+0000 7f47cc1ff700 1 -- 172.21.15.33:0/853035979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47c406c6c0 con 0x7f47c40c1840 2023-11-06T21:00:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.616+0000 7f47b9ffb700 1 -- 172.21.15.33:0/853035979 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f47b404e510 con 0x7f47c40c1840 2023-11-06T21:00:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:17.617 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1797026087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1797026087},{"type":"v1","addr":"172.21.15.69:6836","nonce":1797026087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4199570083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4199570083},{"type":"v1","addr":"172.21.15.33:6834","nonce":4199570083}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2429309034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2429309034},{"type":"v1","addr":"172.21.15.33:6835","nonce":2429309034}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:57.774874+0000","modified":"2023-11-06T21:00:00.791809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6621":{"gid":6621,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/1260218648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1260218648},{"type":"v1","addr":"172.21.15.69:6837","nonce":1260218648}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-06T21:00:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 -- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47a805f300 msgr2=0x7f47a80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47a805f300 0x7f47a80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 -- 172.21.15.33:0/853035979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40c1840 msgr2=0x7f47c4166ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40c1840 0x7f47c4166ae0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f47b400c440 tx=0x7f47b4007d40).stop 2023-11-06T21:00:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 -- 172.21.15.33:0/853035979 shutdown_connections 2023-11-06T21:00:17.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f47a805f300 0x7f47a80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:17.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f47c40c6460 0x7f47c4166fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47c40aa2d0 0x7f47c40c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 --2- 172.21.15.33:0/853035979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c40c1840 0x7f47c4166ae0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 -- 172.21.15.33:0/853035979 >> 172.21.15.33:0/853035979 conn(0x7f47c405ec80 msgr2=0x7f47c41528d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 -- 172.21.15.33:0/853035979 shutdown_connections 2023-11-06T21:00:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.618+0000 7f47cc1ff700 1 -- 172.21.15.33:0/853035979 wait complete. 2023-11-06T21:00:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 37 2023-11-06T21:00:17.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:00:17.677 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:17.677+0000 7faa81349680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.835+0000 7f0bcafff700 1 Processor -- start 2023-11-06T21:00:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.836+0000 7f0bcafff700 1 -- start start 2023-11-06T21:00:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.837+0000 7f0bcafff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0bc4063140 0x7f0bc4063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.837+0000 7f0bcafff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0bc405e1b0 0x7f0bc40c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.837+0000 7f0bcafff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc40c31f0 con 0x7f0bc40c4490 2023-11-06T21:00:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.837+0000 7f0bcafff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc406a0f0 con 0x7f0bc4063140 2023-11-06T21:00:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.837+0000 7f0bcafff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc40c2920 con 0x7f0bc405e1b0 2023-11-06T21:00:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.837+0000 7f0bc8d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0bc4063140 0x7f0bc4063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.837+0000 7f0bc8d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0bc4063140 0x7f0bc4063520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36030/0 (socket says 172.21.15.33:36030) 2023-11-06T21:00:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.837+0000 7f0bc8d9b700 1 -- 172.21.15.33:0/2640265067 learned_addr learned my addr 172.21.15.33:0/2640265067 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.837+0000 7f0bc3fff700 1 --2- 172.21.15.33:0/2640265067 >> v2:172.21.15.69:3300/0 conn(0x7f0bc405e1b0 0x7f0bc40c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.838+0000 7f0bc8d9b700 1 -- 172.21.15.33:0/2640265067 >> v2:172.21.15.69:3300/0 conn(0x7f0bc405e1b0 msgr2=0x7f0bc40c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.838+0000 7f0bc8d9b700 1 --2- 172.21.15.33:0/2640265067 >> v2:172.21.15.69:3300/0 conn(0x7f0bc405e1b0 0x7f0bc40c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.838+0000 7f0bc8d9b700 1 -- 172.21.15.33:0/2640265067 >> v1:172.21.15.33:6789/0 conn(0x7f0bc40c4490 legacy=0x7f0bc4154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.838+0000 7f0bc8d9b700 1 -- 172.21.15.33:0/2640265067 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc40c3b70 con 0x7f0bc4063140 2023-11-06T21:00:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.838+0000 7f0bc37fe700 1 -- 172.21.15.33:0/2640265067 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925020753 0 0) 0x7f0bc40c31f0 con 0x7f0bc40c4490 2023-11-06T21:00:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.838+0000 7f0bc8d9b700 1 --2- 172.21.15.33:0/2640265067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 0x7f0bc4063520 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f0bc40c1890 tx=0x7f0bb80090f0).ready entity=mon.0 client_cookie=4fa5cf58df09d306 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.838+0000 7f0bc37fe700 1 -- 172.21.15.33:0/2640265067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800b520 con 0x7f0bc4063140 2023-11-06T21:00:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.838+0000 7f0bc37fe700 1 -- 172.21.15.33:0/2640265067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb800b680 con 0x7f0bc4063140 2023-11-06T21:00:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.838+0000 7f0bc37fe700 1 -- 172.21.15.33:0/2640265067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800ba00 con 0x7f0bc4063140 2023-11-06T21:00:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.839+0000 7f0bcafff700 1 -- 172.21.15.33:0/2640265067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 msgr2=0x7f0bc4063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.839+0000 7f0bcafff700 1 --2- 172.21.15.33:0/2640265067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 0x7f0bc4063520 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f0bc40c1890 tx=0x7f0bb80090f0).stop 2023-11-06T21:00:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.839+0000 7f0bcafff700 1 -- 172.21.15.33:0/2640265067 shutdown_connections 2023-11-06T21:00:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.839+0000 7f0bcafff700 1 --2- 172.21.15.33:0/2640265067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 0x7f0bc4063520 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.839+0000 7f0bcafff700 1 --2- 172.21.15.33:0/2640265067 >> v2:172.21.15.69:3300/0 conn(0x7f0bc405e1b0 0x7f0bc40c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.839+0000 7f0bcafff700 1 -- 172.21.15.33:0/2640265067 >> 172.21.15.33:0/2640265067 conn(0x7f0bc40c7d60 msgr2=0x7f0bc40c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:17.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.839+0000 7f0bcafff700 1 -- 172.21.15.33:0/2640265067 shutdown_connections 2023-11-06T21:00:17.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.839+0000 7f0bcafff700 1 -- 172.21.15.33:0/2640265067 wait complete. 2023-11-06T21:00:17.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.841+0000 7f0bcafff700 1 Processor -- start 2023-11-06T21:00:17.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.841+0000 7f0bcafff700 1 -- start start 2023-11-06T21:00:17.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bcafff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 0x7f0bc4154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bcafff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bc40c4490 0x7f0bc414fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bcafff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc4150280 0x7f0bc41642d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bcafff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc4157f00 con 0x7f0bc4063140 2023-11-06T21:00:17.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bcafff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc4155e40 con 0x7f0bc4150280 2023-11-06T21:00:17.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bcafff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc4155b10 con 0x7f0bc40c4490 2023-11-06T21:00:17.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bc8d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 0x7f0bc4154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:17.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bc8d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 0x7f0bc4154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36032/0 (socket says 172.21.15.33:36032) 2023-11-06T21:00:17.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bc8d9b700 1 -- 172.21.15.33:0/112423953 learned_addr learned my addr 172.21.15.33:0/112423953 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bc3fff700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bc40c4490 0x7f0bc414fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bc959c700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc4150280 0x7f0bc41642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bc8d9b700 1 -- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bc40c4490 msgr2=0x7f0bc414fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bc8d9b700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bc40c4490 0x7f0bc414fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bc8d9b700 1 -- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc4150280 msgr2=0x7f0bc41642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:17.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bc8d9b700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc4150280 0x7f0bc41642d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:17.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.842+0000 7f0bc8d9b700 1 -- 172.21.15.33:0/112423953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8009370 con 0x7f0bc4063140 2023-11-06T21:00:17.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.843+0000 7f0bc8d9b700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 0x7f0bc4154830 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f0bc405ebe0 tx=0x7f0bb8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:17.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.843+0000 7f0bc0ff9700 1 -- 172.21.15.33:0/112423953 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8000bc0 con 0x7f0bc4063140 2023-11-06T21:00:17.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.843+0000 7f0bc0ff9700 1 -- 172.21.15.33:0/112423953 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb800b7f0 con 0x7f0bc4063140 2023-11-06T21:00:17.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.843+0000 7f0bcafff700 1 -- 172.21.15.33:0/112423953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc4157940 con 0x7f0bc4063140 2023-11-06T21:00:17.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.843+0000 7f0bc0ff9700 1 -- 172.21.15.33:0/112423953 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8000d20 con 0x7f0bc4063140 2023-11-06T21:00:17.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.843+0000 7f0bcafff700 1 -- 172.21.15.33:0/112423953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc4154d20 con 0x7f0bc4063140 2023-11-06T21:00:17.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.844+0000 7f0bcafff700 1 -- 172.21.15.33:0/112423953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc4155970 con 0x7f0bc4063140 2023-11-06T21:00:17.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.845+0000 7f0bc0ff9700 1 -- 172.21.15.33:0/112423953 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f0bb801c880 con 0x7f0bc4063140 2023-11-06T21:00:17.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.847+0000 7f0bc0ff9700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0ba805f300 0x7f0ba80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:17.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.847+0000 7f0bc0ff9700 1 -- 172.21.15.33:0/112423953 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f0bb807c250 con 0x7f0bc4063140 2023-11-06T21:00:17.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.847+0000 7f0bc3fff700 1 -- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0ba805f300 msgr2=0x7f0ba80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:17.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.847+0000 7f0bc3fff700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0ba805f300 0x7f0ba80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:17.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:17.851+0000 7f0bc0ff9700 1 -- 172.21.15.33:0/112423953 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0bc4155970 con 0x7f0bc4063140 2023-11-06T21:00:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.032+0000 7f0bcafff700 1 -- 172.21.15.33:0/112423953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bc4155970 con 0x7f0bc4063140 2023-11-06T21:00:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.033+0000 7f0bc0ff9700 1 -- 172.21.15.33:0/112423953 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+17349 (secure 0 0 0) 0x7f0bc4155970 con 0x7f0bc4063140 2023-11-06T21:00:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:18.035 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":100,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:00:12.186197+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:55.320441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:56.766396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1148377580":"2023-11-07T21:00:12.186151+0000","172.21.15.69:6833/114912":"2023-11-07T21:00:12.186151+0000","172.21.15.69:6832/114912":"2023-11-07T21:00:12.186151+0000","172.21.15.69:0/4025560788":"2023-11-07T21:00:12.186151+0000","172.21.15.69:0/1979880988":"2023-11-07T21:00:12.186151+0000","172.21.15.69:0/3973297670":"2023-11-07T21:00:12.186151+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:00:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 -- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0ba805f300 msgr2=0x7f0ba80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0ba805f300 0x7f0ba80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 -- 172.21.15.33:0/112423953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 msgr2=0x7f0bc4154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 0x7f0bc4154830 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f0bc405ebe0 tx=0x7f0bb8017000).stop 2023-11-06T21:00:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 -- 172.21.15.33:0/112423953 shutdown_connections 2023-11-06T21:00:18.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0ba805f300 0x7f0ba80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:18.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc4150280 0x7f0bc41642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc4063140 0x7f0bc4154830 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 --2- 172.21.15.33:0/112423953 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bc40c4490 0x7f0bc414fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 -- 172.21.15.33:0/112423953 >> 172.21.15.33:0/112423953 conn(0x7f0bc40c7d60 msgr2=0x7f0bc4151a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 -- 172.21.15.33:0/112423953 shutdown_connections 2023-11-06T21:00:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.036+0000 7f0bcafff700 1 -- 172.21.15.33:0/112423953 wait complete. 2023-11-06T21:00:18.057 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T21:00:18.058 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:00:18.236 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:18.237+0000 7f7773972680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:00:18.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.292+0000 7f1d0e9ff700 1 Processor -- start 2023-11-06T21:00:18.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.293+0000 7f1d0e9ff700 1 -- start start 2023-11-06T21:00:18.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.294+0000 7f1d0e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 0x7f1d080c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.294+0000 7f1d0e9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1d08155320 0x7f1d080c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.294+0000 7f1d0e9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d08154e90 con 0x7f1d081521a0 2023-11-06T21:00:18.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.294+0000 7f1d0e9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080639d0 con 0x7f1d08155320 2023-11-06T21:00:18.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.294+0000 7f1d0e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d08063cb0 con 0x7f1d08151dc0 2023-11-06T21:00:18.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.294+0000 7f1d077fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1d081521a0 0x7f1d08063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38510/0 (socket says 172.21.15.33:38510) 2023-11-06T21:00:18.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.294+0000 7f1d077fe700 1 -- 172.21.15.33:0/2946058480 learned_addr learned my addr 172.21.15.33:0/2946058480 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:18.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.295+0000 7f1d0cf9c700 1 --2- 172.21.15.33:0/2946058480 >> v2:172.21.15.33:3300/0 conn(0x7f1d08155320 0x7f1d080c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:18.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.295+0000 7f1d07fff700 1 --2- 172.21.15.33:0/2946058480 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 0x7f1d080c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:18.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.295+0000 7f1d06ffd700 1 -- 172.21.15.33:0/2946058480 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439910645 0 0) 0x7f1d08154e90 con 0x7f1d081521a0 2023-11-06T21:00:18.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.295+0000 7f1d06ffd700 1 -- 172.21.15.33:0/2946058480 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf00033f0 con 0x7f1d081521a0 2023-11-06T21:00:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.295+0000 7f1d0cf9c700 1 -- 172.21.15.33:0/2946058480 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 msgr2=0x7f1d080c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.295+0000 7f1d0cf9c700 1 --2- 172.21.15.33:0/2946058480 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 0x7f1d080c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.295+0000 7f1d0cf9c700 1 -- 172.21.15.33:0/2946058480 >> v1:172.21.15.33:6789/0 conn(0x7f1d081521a0 legacy=0x7f1d08063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.295+0000 7f1d0cf9c700 1 -- 172.21.15.33:0/2946058480 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d08063f90 con 0x7f1d08155320 2023-11-06T21:00:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.295+0000 7f1d0cf9c700 1 --2- 172.21.15.33:0/2946058480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d08155320 0x7f1d080c7850 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f1d00009e60 tx=0x7f1d00009780).ready entity=mon.0 client_cookie=b1a87bc9120e6686 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d06ffd700 1 -- 172.21.15.33:0/2946058480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000e070 con 0x7f1d08155320 2023-11-06T21:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d06ffd700 1 -- 172.21.15.33:0/2946058480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0000bab0 con 0x7f1d08155320 2023-11-06T21:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d06ffd700 1 -- 172.21.15.33:0/2946058480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000be30 con 0x7f1d08155320 2023-11-06T21:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/2946058480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d08155320 msgr2=0x7f1d080c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d0e9ff700 1 --2- 172.21.15.33:0/2946058480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d08155320 0x7f1d080c7850 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f1d00009e60 tx=0x7f1d00009780).stop 2023-11-06T21:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/2946058480 shutdown_connections 2023-11-06T21:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d0e9ff700 1 --2- 172.21.15.33:0/2946058480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d08155320 0x7f1d080c7850 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d0e9ff700 1 --2- 172.21.15.33:0/2946058480 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 0x7f1d080c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/2946058480 >> 172.21.15.33:0/2946058480 conn(0x7f1d0805e1b0 msgr2=0x7f1d0805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/2946058480 shutdown_connections 2023-11-06T21:00:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.296+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/2946058480 wait complete. 2023-11-06T21:00:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.298+0000 7f1d0e9ff700 1 Processor -- start 2023-11-06T21:00:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.299+0000 7f1d0e9ff700 1 -- start start 2023-11-06T21:00:18.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d0e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 0x7f1d08150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d0e9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d081521a0 0x7f1d080c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d0e9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d08155320 0x7f1d080c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d0e9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080ae910 con 0x7f1d081521a0 2023-11-06T21:00:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d0e9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080c6570 con 0x7f1d08155320 2023-11-06T21:00:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d0e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080c4ba0 con 0x7f1d08151dc0 2023-11-06T21:00:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d081521a0 0x7f1d080c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d07fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 0x7f1d08150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d081521a0 0x7f1d080c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43192/0 (socket says 172.21.15.33:43192) 2023-11-06T21:00:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d077fe700 1 -- 172.21.15.33:0/142992764 learned_addr learned my addr 172.21.15.33:0/142992764 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:18.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.300+0000 7f1d0cf9c700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d08155320 0x7f1d080c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:18.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1d077fe700 1 -- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 msgr2=0x7f1d08150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:18.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1d077fe700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 0x7f1d08150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1d077fe700 1 -- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d08155320 msgr2=0x7f1d080c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:18.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1d077fe700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d08155320 0x7f1d080c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1d077fe700 1 -- 172.21.15.33:0/142992764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d00009a20 con 0x7f1d081521a0 2023-11-06T21:00:18.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1d07fff700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 0x7f1d08150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:18.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1d077fe700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d081521a0 0x7f1d080c2180 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f1cfc00c4c0 tx=0x7f1cfc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:18.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1ceffff700 1 -- 172.21.15.33:0/142992764 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00ce30 con 0x7f1d081521a0 2023-11-06T21:00:18.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1ceffff700 1 -- 172.21.15.33:0/142992764 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc0072e0 con 0x7f1d081521a0 2023-11-06T21:00:18.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/142992764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d080c6800 con 0x7f1d081521a0 2023-11-06T21:00:18.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.301+0000 7f1ceffff700 1 -- 172.21.15.33:0/142992764 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0057a0 con 0x7f1d081521a0 2023-11-06T21:00:18.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.302+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/142992764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d080c6dc0 con 0x7f1d081521a0 2023-11-06T21:00:18.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.303+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/142992764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d080c6fb0 con 0x7f1d081521a0 2023-11-06T21:00:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.304+0000 7f1ceffff700 1 -- 172.21.15.33:0/142992764 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f1cfc005b10 con 0x7f1d081521a0 2023-11-06T21:00:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.304+0000 7f1ceffff700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1cf405f300 0x7f1cf40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.304+0000 7f1ceffff700 1 -- 172.21.15.33:0/142992764 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f1cfc079bf0 con 0x7f1d081521a0 2023-11-06T21:00:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.304+0000 7f1d07fff700 1 -- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1cf405f300 msgr2=0x7f1cf40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.305+0000 7f1d07fff700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1cf405f300 0x7f1cf40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:18.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.308+0000 7f1ceffff700 1 -- 172.21.15.33:0/142992764 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1cfc04aac0 con 0x7f1d081521a0 2023-11-06T21:00:18.324 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:18.325+0000 7f7773972680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:00:18.418 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:18.418+0000 7f7773972680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:00:18.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.506+0000 7f1d07fff700 1 -- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1cf405f300 msgr2=0x7f1cf40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:18.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.506+0000 7f1d07fff700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1cf405f300 0x7f1cf40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:00:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.535+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/142992764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d08062ac0 con 0x7f1d081521a0 2023-11-06T21:00:18.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.536+0000 7f1ceffff700 1 -- 172.21.15.33:0/142992764 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f1cfc018070 con 0x7f1d081521a0 2023-11-06T21:00:18.537 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:18.537 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1797026087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1797026087},{"type":"v1","addr":"172.21.15.69:6836","nonce":1797026087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4199570083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4199570083},{"type":"v1","addr":"172.21.15.33:6834","nonce":4199570083}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2429309034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2429309034},{"type":"v1","addr":"172.21.15.33:6835","nonce":2429309034}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:57.774874+0000","modified":"2023-11-06T21:00:00.791809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6621":{"gid":6621,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/1260218648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1260218648},{"type":"v1","addr":"172.21.15.69:6837","nonce":1260218648}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-06T21:00:18.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1cf405f300 msgr2=0x7f1cf40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:18.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1cf405f300 0x7f1cf40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:18.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/142992764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d081521a0 msgr2=0x7f1d080c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d081521a0 0x7f1d080c2180 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f1cfc00c4c0 tx=0x7f1cfc007c10).stop 2023-11-06T21:00:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/142992764 shutdown_connections 2023-11-06T21:00:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1cf405f300 0x7f1cf40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1d08151dc0 0x7f1d08150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d08155320 0x7f1d080c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 --2- 172.21.15.33:0/142992764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d081521a0 0x7f1d080c2180 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/142992764 >> 172.21.15.33:0/142992764 conn(0x7f1d0805e1b0 msgr2=0x7f1d08152640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/142992764 shutdown_connections 2023-11-06T21:00:18.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.538+0000 7f1d0e9ff700 1 -- 172.21.15.33:0/142992764 wait complete. 2023-11-06T21:00:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 37 2023-11-06T21:00:18.552 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:18.552+0000 7faa81349680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:00:18.558 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:00:18.603 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:18.603+0000 7f7773972680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:00:18.648 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:18.647+0000 7faa81349680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:00:18.691 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:18.692+0000 7f7773972680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:00:18.750 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:18.750+0000 7faa81349680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:00:18.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.755+0000 7fcf165ff700 1 Processor -- start 2023-11-06T21:00:18.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.755+0000 7fcf165ff700 1 -- start start 2023-11-06T21:00:18.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.757+0000 7fcf165ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcf100c1840 0x7fcf100a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.757+0000 7fcf165ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcf100aa2d0 0x7fcf100c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.757+0000 7fcf165ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf101554a0 con 0x7fcf100c6240 2023-11-06T21:00:18.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.757+0000 7fcf165ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf10154960 con 0x7fcf100aa2d0 2023-11-06T21:00:18.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.757+0000 7fcf165ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf10155930 con 0x7fcf100c1840 2023-11-06T21:00:18.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.757+0000 7fcf0f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcf100aa2d0 0x7fcf100c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:18.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.757+0000 7fcf0ffff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcf100c1840 0x7fcf100a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:18.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.757+0000 7fcf0f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcf100aa2d0 0x7fcf100c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43208/0 (socket says 172.21.15.33:43208) 2023-11-06T21:00:18.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.758+0000 7fcf0f7fe700 1 -- 172.21.15.33:0/3218277293 learned_addr learned my addr 172.21.15.33:0/3218277293 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:18.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.758+0000 7fcf0ffff700 1 -- 172.21.15.33:0/3218277293 >> v2:172.21.15.33:3300/0 conn(0x7fcf100aa2d0 msgr2=0x7fcf100c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:18.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.758+0000 7fcf0ffff700 1 --2- 172.21.15.33:0/3218277293 >> v2:172.21.15.33:3300/0 conn(0x7fcf100aa2d0 0x7fcf100c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.758+0000 7fcf0ffff700 1 -- 172.21.15.33:0/3218277293 >> v1:172.21.15.69:6789/0 conn(0x7fcf100c6240 legacy=0x7fcf10157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:18.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.758+0000 7fcf0ffff700 1 -- 172.21.15.33:0/3218277293 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10154b70 con 0x7fcf100c1840 2023-11-06T21:00:18.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.758+0000 7fcf0f7fe700 1 --2- 172.21.15.33:0/3218277293 >> v2:172.21.15.33:3300/0 conn(0x7fcf100aa2d0 0x7fcf100c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:18.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.758+0000 7fcf0effd700 1 -- 172.21.15.33:0/3218277293 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683197571 0 0) 0x7fcf101554a0 con 0x7fcf100c6240 2023-11-06T21:00:18.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.758+0000 7fcf0ffff700 1 --2- 172.21.15.33:0/3218277293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf100c1840 0x7fcf100a9de0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fcf100840b0 tx=0x7fcf00009060).ready entity=mon.1 client_cookie=22d18695486fcd15 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:18.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.758+0000 7fcf0effd700 1 -- 172.21.15.33:0/3218277293 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000f510 con 0x7fcf100c1840 2023-11-06T21:00:18.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.759+0000 7fcf0effd700 1 -- 172.21.15.33:0/3218277293 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0000f670 con 0x7fcf100c1840 2023-11-06T21:00:18.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.759+0000 7fcf0effd700 1 -- 172.21.15.33:0/3218277293 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000f9f0 con 0x7fcf100c1840 2023-11-06T21:00:18.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.759+0000 7fcf165ff700 1 -- 172.21.15.33:0/3218277293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf100c1840 msgr2=0x7fcf100a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:18.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.759+0000 7fcf165ff700 1 --2- 172.21.15.33:0/3218277293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf100c1840 0x7fcf100a9de0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fcf100840b0 tx=0x7fcf00009060).stop 2023-11-06T21:00:18.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.759+0000 7fcf165ff700 1 -- 172.21.15.33:0/3218277293 shutdown_connections 2023-11-06T21:00:18.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.759+0000 7fcf165ff700 1 --2- 172.21.15.33:0/3218277293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf100c1840 0x7fcf100a9de0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.759+0000 7fcf165ff700 1 --2- 172.21.15.33:0/3218277293 >> v2:172.21.15.33:3300/0 conn(0x7fcf100aa2d0 0x7fcf100c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.759+0000 7fcf165ff700 1 -- 172.21.15.33:0/3218277293 >> 172.21.15.33:0/3218277293 conn(0x7fcf1005ec80 msgr2=0x7fcf100cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:18.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.759+0000 7fcf165ff700 1 -- 172.21.15.33:0/3218277293 shutdown_connections 2023-11-06T21:00:18.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.759+0000 7fcf165ff700 1 -- 172.21.15.33:0/3218277293 wait complete. 2023-11-06T21:00:18.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.761+0000 7fcf165ff700 1 Processor -- start 2023-11-06T21:00:18.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.761+0000 7fcf165ff700 1 -- start start 2023-11-06T21:00:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf165ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf100aa2d0 0x7fcf10167470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf165ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf100c1840 0x7fcf101647a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf165ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf100c6240 0x7fcf100c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf165ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100c2bf0 con 0x7fcf100c6240 2023-11-06T21:00:18.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf165ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100c1fe0 con 0x7fcf100aa2d0 2023-11-06T21:00:18.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf165ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100c22c0 con 0x7fcf100c1840 2023-11-06T21:00:18.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf14b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf100c6240 0x7fcf100c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:18.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf0ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf100aa2d0 0x7fcf10167470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:18.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf14b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf100c6240 0x7fcf100c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43220/0 (socket says 172.21.15.33:43220) 2023-11-06T21:00:18.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf14b9c700 1 -- 172.21.15.33:0/1626685950 learned_addr learned my addr 172.21.15.33:0/1626685950 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:18.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.762+0000 7fcf0f7fe700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf100c1840 0x7fcf101647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:18.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcf0ffff700 1 -- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf100c1840 msgr2=0x7fcf101647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:18.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcf0ffff700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf100c1840 0x7fcf101647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcf0ffff700 1 -- 172.21.15.33:0/1626685950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf100c6240 msgr2=0x7fcf100c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:18.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcf0ffff700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf100c6240 0x7fcf100c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:18.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcf0ffff700 1 -- 172.21.15.33:0/1626685950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04009370 con 0x7fcf100aa2d0 2023-11-06T21:00:18.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcf14b9c700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf100c6240 0x7fcf100c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:18.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcf0ffff700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf100aa2d0 0x7fcf10167470 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fcf00014000 tx=0x7fcf00017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:18.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcef7fff700 1 -- 172.21.15.33:0/1626685950 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000fe80 con 0x7fcf100aa2d0 2023-11-06T21:00:18.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcef7fff700 1 -- 172.21.15.33:0/1626685950 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0000f7e0 con 0x7fcf100aa2d0 2023-11-06T21:00:18.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcef7fff700 1 -- 172.21.15.33:0/1626685950 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0001b640 con 0x7fcf100aa2d0 2023-11-06T21:00:18.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcf165ff700 1 -- 172.21.15.33:0/1626685950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf000092a0 con 0x7fcf100aa2d0 2023-11-06T21:00:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.763+0000 7fcf165ff700 1 -- 172.21.15.33:0/1626685950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf100c2880 con 0x7fcf100aa2d0 2023-11-06T21:00:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.764+0000 7fcf165ff700 1 -- 172.21.15.33:0/1626685950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf100c2ed0 con 0x7fcf100aa2d0 2023-11-06T21:00:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.765+0000 7fcef7fff700 1 -- 172.21.15.33:0/1626685950 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fcf0001b9f0 con 0x7fcf100aa2d0 2023-11-06T21:00:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.766+0000 7fcef7fff700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcefc05f300 0x7fcefc0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.766+0000 7fcef7fff700 1 -- 172.21.15.33:0/1626685950 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fcf0007bc60 con 0x7fcf100aa2d0 2023-11-06T21:00:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.766+0000 7fcf0f7fe700 1 -- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcefc05f300 msgr2=0x7fcefc0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.766+0000 7fcf0f7fe700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcefc05f300 0x7fcefc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.770+0000 7fcef7fff700 1 -- 172.21.15.33:0/1626685950 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcf0004cc30 con 0x7fcf100aa2d0 2023-11-06T21:00:18.833 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:18.833+0000 7f7773972680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:00:18.950 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:18.949+0000 7faa81349680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:00:18.962 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:18.963+0000 7f7773972680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:18.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.967+0000 7fcf0f7fe700 1 -- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcefc05f300 msgr2=0x7fcefc0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:18.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:18.967+0000 7fcf0f7fe700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcefc05f300 0x7fcefc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:00:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.000+0000 7fcf165ff700 1 -- 172.21.15.33:0/1626685950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf1006c6c0 con 0x7fcf100aa2d0 2023-11-06T21:00:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.001+0000 7fcef7fff700 1 -- 172.21.15.33:0/1626685950 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4491 (secure 0 0 0) 0x7fcf00029020 con 0x7fcf100aa2d0 2023-11-06T21:00:19.002 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:19.002 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1797026087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1797026087},{"type":"v1","addr":"172.21.15.69:6836","nonce":1797026087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4199570083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4199570083},{"type":"v1","addr":"172.21.15.33:6834","nonce":4199570083}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2429309034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2429309034},{"type":"v1","addr":"172.21.15.33:6835","nonce":2429309034}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T20:59:57.774874+0000","modified":"2023-11-06T21:00:00.791809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6621":{"gid":6621,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/1260218648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1260218648},{"type":"v1","addr":"172.21.15.69:6837","nonce":1260218648}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-06T21:00:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 -- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcefc05f300 msgr2=0x7fcefc0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcefc05f300 0x7fcefc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 -- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf100aa2d0 msgr2=0x7fcf10167470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf100aa2d0 0x7fcf10167470 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fcf00014000 tx=0x7fcf00017000).stop 2023-11-06T21:00:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 -- 172.21.15.33:0/1626685950 shutdown_connections 2023-11-06T21:00:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcefc05f300 0x7fcefc0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf100c6240 0x7fcf100c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf100aa2d0 0x7fcf10167470 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 --2- 172.21.15.33:0/1626685950 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf100c1840 0x7fcf101647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 -- 172.21.15.33:0/1626685950 >> 172.21.15.33:0/1626685950 conn(0x7fcf1005ec80 msgr2=0x7fcf10152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:19.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.003+0000 7fcf165ff700 1 -- 172.21.15.33:0/1626685950 shutdown_connections 2023-11-06T21:00:19.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.004+0000 7fcf165ff700 1 -- 172.21.15.33:0/1626685950 wait complete. 2023-11-06T21:00:19.008 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 37 2023-11-06T21:00:19.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:00:19.048 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:19.048+0000 7faa81349680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:00:19.206 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:19.206+0000 7faa81349680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:00:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.220+0000 7f71b03ff700 1 Processor -- start 2023-11-06T21:00:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.221+0000 7f71b03ff700 1 -- start start 2023-11-06T21:00:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.222+0000 7f71b03ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a805e1b0 0x7f71a8063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.222+0000 7f71b03ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a80c31f0 con 0x7f71a80c3680 2023-11-06T21:00:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.222+0000 7f71b03ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a80c2920 con 0x7f71a8063140 2023-11-06T21:00:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.222+0000 7f71b03ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a806a0f0 con 0x7f71a805e1b0 2023-11-06T21:00:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.222+0000 7f71ad99a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a805e1b0 0x7f71a8063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.222+0000 7f71ae99c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f71a80c3680 0x7f71a8154050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38518/0 (socket says 172.21.15.33:38518) 2023-11-06T21:00:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.222+0000 7f71ae19b700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f71a8063140 0x7f71a80c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:53524/0 (socket says 172.21.15.33:53524) 2023-11-06T21:00:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.222+0000 7f71ae99c700 1 -- 172.21.15.33:0/2704001150 learned_addr learned my addr 172.21.15.33:0/2704001150 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.223+0000 7f71ad199700 1 -- 172.21.15.33:0/2704001150 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476984953 0 0) 0x7f71a80c2920 con 0x7f71a8063140 2023-11-06T21:00:19.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.223+0000 7f71ad99a700 1 -- 172.21.15.33:0/2704001150 >> v1:172.21.15.69:6789/0 conn(0x7f71a8063140 legacy=0x7f71a80c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:19.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.223+0000 7f71ad99a700 1 -- 172.21.15.33:0/2704001150 >> v1:172.21.15.33:6789/0 conn(0x7f71a80c3680 legacy=0x7f71a8154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:19.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.223+0000 7f71ad99a700 1 -- 172.21.15.33:0/2704001150 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a805e640 con 0x7f71a805e1b0 2023-11-06T21:00:19.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.223+0000 7f71ad99a700 1 --2- 172.21.15.33:0/2704001150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a805e1b0 0x7f71a8063520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f7198009790 tx=0x7f7198009060).ready entity=mon.2 client_cookie=4c33b6e67d8c26e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.223+0000 7f71ad199700 1 -- 172.21.15.33:0/2704001150 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719800b5a0 con 0x7f71a805e1b0 2023-11-06T21:00:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.223+0000 7f71ad199700 1 -- 172.21.15.33:0/2704001150 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719800b700 con 0x7f71a805e1b0 2023-11-06T21:00:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.223+0000 7f71ad199700 1 -- 172.21.15.33:0/2704001150 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719800ba80 con 0x7f71a805e1b0 2023-11-06T21:00:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.224+0000 7f71b03ff700 1 -- 172.21.15.33:0/2704001150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a805e1b0 msgr2=0x7f71a8063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.224+0000 7f71b03ff700 1 --2- 172.21.15.33:0/2704001150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a805e1b0 0x7f71a8063520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f7198009790 tx=0x7f7198009060).stop 2023-11-06T21:00:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.224+0000 7f71b03ff700 1 -- 172.21.15.33:0/2704001150 shutdown_connections 2023-11-06T21:00:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.224+0000 7f71b03ff700 1 --2- 172.21.15.33:0/2704001150 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a805e1b0 0x7f71a8063520 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.224+0000 7f71b03ff700 1 -- 172.21.15.33:0/2704001150 >> 172.21.15.33:0/2704001150 conn(0x7f71a80c7d60 msgr2=0x7f71a80c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.224+0000 7f71b03ff700 1 -- 172.21.15.33:0/2704001150 shutdown_connections 2023-11-06T21:00:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.224+0000 7f71b03ff700 1 -- 172.21.15.33:0/2704001150 wait complete. 2023-11-06T21:00:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.225+0000 7f71b03ff700 1 Processor -- start 2023-11-06T21:00:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.225+0000 7f71b03ff700 1 -- start start 2023-11-06T21:00:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.226+0000 7f71b03ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a8063140 0x7f71a8167400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.226+0000 7f71b03ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a80c3680 0x7f71a814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.226+0000 7f71b03ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a814dfb0 0x7f71a8153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.226+0000 7f71b03ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a814e430 con 0x7f71a814dfb0 2023-11-06T21:00:19.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.226+0000 7f71b03ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a814f220 con 0x7f71a80c3680 2023-11-06T21:00:19.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.226+0000 7f71b03ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a8158bf0 con 0x7f71a8063140 2023-11-06T21:00:19.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ae99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a814dfb0 0x7f71a8153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:19.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ae19b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a8063140 0x7f71a8167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:19.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ad99a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a80c3680 0x7f71a814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:19.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ae99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a814dfb0 0x7f71a8153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43228/0 (socket says 172.21.15.33:43228) 2023-11-06T21:00:19.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ae99c700 1 -- 172.21.15.33:0/3447650026 learned_addr learned my addr 172.21.15.33:0/3447650026 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:19.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ae19b700 1 -- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a80c3680 msgr2=0x7f71a814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ae19b700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a80c3680 0x7f71a814dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ae19b700 1 -- 172.21.15.33:0/3447650026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a814dfb0 msgr2=0x7f71a8153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ae19b700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a814dfb0 0x7f71a8153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ae19b700 1 -- 172.21.15.33:0/3447650026 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71980092a0 con 0x7f71a8063140 2023-11-06T21:00:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.227+0000 7f71ae19b700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a8063140 0x7f71a8167400 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f71a400c410 tx=0x7f71a4007bd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.228+0000 7f719e7fc700 1 -- 172.21.15.33:0/3447650026 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40070d0 con 0x7f71a8063140 2023-11-06T21:00:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.228+0000 7f719e7fc700 1 -- 172.21.15.33:0/3447650026 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a4007230 con 0x7f71a8063140 2023-11-06T21:00:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.228+0000 7f71b03ff700 1 -- 172.21.15.33:0/3447650026 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a814e610 con 0x7f71a8063140 2023-11-06T21:00:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.228+0000 7f719e7fc700 1 -- 172.21.15.33:0/3447650026 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40056e0 con 0x7f71a8063140 2023-11-06T21:00:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.228+0000 7f71b03ff700 1 -- 172.21.15.33:0/3447650026 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a814ebd0 con 0x7f71a8063140 2023-11-06T21:00:19.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.229+0000 7f71b03ff700 1 -- 172.21.15.33:0/3447650026 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a8064420 con 0x7f71a8063140 2023-11-06T21:00:19.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.230+0000 7f719e7fc700 1 -- 172.21.15.33:0/3447650026 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f71a4005ad0 con 0x7f71a8063140 2023-11-06T21:00:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.230+0000 7f719e7fc700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f718c05f2d0 0x7f718c061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.230+0000 7f719e7fc700 1 -- 172.21.15.33:0/3447650026 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f71a4079ca0 con 0x7f71a8063140 2023-11-06T21:00:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.231+0000 7f71ad99a700 1 -- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f718c05f2d0 msgr2=0x7f718c061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:19.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.231+0000 7f71ad99a700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f718c05f2d0 0x7f718c061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:19.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.235+0000 7f719e7fc700 1 -- 172.21.15.33:0/3447650026 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f71a404ab70 con 0x7f71a8063140 2023-11-06T21:00:19.309 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.309+0000 7f7773972680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:00:19.351 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:19.350+0000 7faa81349680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:19.399 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.399+0000 7f7773972680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:00:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.412+0000 7f719e7fc700 1 -- 172.21.15.33:0/3447650026 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mgrmap(e 36) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f71a4077670 con 0x7f71a8063140 2023-11-06T21:00:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.412+0000 7f719e7fc700 1 -- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f718c05f2d0 msgr2=0x7f718c061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.412+0000 7f719e7fc700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f718c05f2d0 0x7f718c061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:19.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.423+0000 7f71b03ff700 1 -- 172.21.15.33:0/3447650026 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71a806bfd0 con 0x7f71a8063140 2023-11-06T21:00:19.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.424+0000 7f719e7fc700 1 -- 172.21.15.33:0/3447650026 <== mon.2 v2:172.21.15.69:3301/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+17658 (secure 0 0 0) 0x7f71a4072240 con 0x7f71a8063140 2023-11-06T21:00:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:19.425 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":101,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:00:19.403128+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T20:59:55.320441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T20:59:56.766396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1885447676":"2023-11-07T21:00:19.403093+0000","172.21.15.69:0/3206076845":"2023-11-07T21:00:19.403093+0000","172.21.15.69:0/748793228":"2023-11-07T21:00:19.403093+0000","172.21.15.69:0/3130297818":"2023-11-07T21:00:19.403093+0000","172.21.15.69:0/1148377580":"2023-11-07T21:00:12.186151+0000","172.21.15.69:6833/114912":"2023-11-07T21:00:12.186151+0000","172.21.15.69:6832/114912":"2023-11-07T21:00:12.186151+0000","172.21.15.69:0/4025560788":"2023-11-07T21:00:12.186151+0000","172.21.15.69:0/2637518809":"2023-11-07T21:00:19.403093+0000","172.21.15.69:0/1979880988":"2023-11-07T21:00:12.186151+0000","172.21.15.69:0/3973297670":"2023-11-07T21:00:12.186151+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:00:19.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.426+0000 7f71b03ff700 1 -- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a8063140 msgr2=0x7f71a8167400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:19.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.426+0000 7f71b03ff700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a8063140 0x7f71a8167400 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f71a400c410 tx=0x7f71a4007bd0).stop 2023-11-06T21:00:19.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.426+0000 7f71b03ff700 1 -- 172.21.15.33:0/3447650026 shutdown_connections 2023-11-06T21:00:19.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.426+0000 7f71b03ff700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f718c05f2d0 0x7f718c061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:19.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.426+0000 7f71b03ff700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71a8063140 0x7f71a8167400 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.426+0000 7f71b03ff700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a814dfb0 0x7f71a8153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.426+0000 7f71b03ff700 1 --2- 172.21.15.33:0/3447650026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a80c3680 0x7f71a814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.426+0000 7f71b03ff700 1 -- 172.21.15.33:0/3447650026 >> 172.21.15.33:0/3447650026 conn(0x7f71a80c7d60 msgr2=0x7f71a80c0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:19.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.426+0000 7f71b03ff700 1 -- 172.21.15.33:0/3447650026 shutdown_connections 2023-11-06T21:00:19.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.426+0000 7f71b03ff700 1 -- 172.21.15.33:0/3447650026 wait complete. 2023-11-06T21:00:19.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T21:00:19.475 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.475+0000 7f76ee451700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:19.475 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.475+0000 7f76ee451700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:19.475 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.475+0000 7f76ee451700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:19.476 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.475+0000 7f76ee451700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:19.476 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.475+0000 7f76ee451700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:19.476 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.476+0000 7f76eb44b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:19.477 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.476+0000 7f76eb44b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:19.477 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.476+0000 7f76eb44b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:19.477 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.476+0000 7f76eb44b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:19.477 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:19.476+0000 7f76eb44b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.688+0000 7ff81c3ff700 1 Processor -- start 2023-11-06T21:00:19.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.688+0000 7ff81c3ff700 1 -- start start 2023-11-06T21:00:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.689+0000 7ff81c3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8140c3530 0x7ff8140c3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.689+0000 7ff81c3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8140c2900 0x7ff8140bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.689+0000 7ff81c3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff814153d00 0x7ff8141541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.689+0000 7ff81c3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8140c2470 con 0x7ff8140c3530 2023-11-06T21:00:19.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.689+0000 7ff81c3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8140c15e0 con 0x7ff814153d00 2023-11-06T21:00:19.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81c3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140c1860 con 0x7ff8140c2900 2023-11-06T21:00:19.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81a19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8140c3530 0x7ff8140c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81a19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8140c3530 0x7ff8140c3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43236/0 (socket says 172.21.15.33:43236) 2023-11-06T21:00:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81a19b700 1 -- 172.21.15.33:0/994809048 learned_addr learned my addr 172.21.15.33:0/994809048 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81999a700 1 --2- 172.21.15.33:0/994809048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8140c2900 0x7ff8140bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81a99c700 1 --2- 172.21.15.33:0/994809048 >> v2:172.21.15.69:3300/0 conn(0x7ff814153d00 0x7ff8141541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81a19b700 1 -- 172.21.15.33:0/994809048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8140c2900 msgr2=0x7ff8140bc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81a19b700 1 --2- 172.21.15.33:0/994809048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8140c2900 0x7ff8140bc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81a19b700 1 -- 172.21.15.33:0/994809048 >> v2:172.21.15.69:3300/0 conn(0x7ff814153d00 msgr2=0x7ff8141541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:19.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81a19b700 1 --2- 172.21.15.33:0/994809048 >> v2:172.21.15.69:3300/0 conn(0x7ff814153d00 0x7ff8141541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.690+0000 7ff81a19b700 1 -- 172.21.15.33:0/994809048 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8140c1b40 con 0x7ff8140c3530 2023-11-06T21:00:19.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.691+0000 7ff81a19b700 1 --2- 172.21.15.33:0/994809048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8140c3530 0x7ff8140c3910 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7ff8140bcce0 tx=0x7ff8100090f0).ready entity=mon.0 client_cookie=242cf31d7567076f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:19.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.691+0000 7ff819199700 1 -- 172.21.15.33:0/994809048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000f510 con 0x7ff8140c3530 2023-11-06T21:00:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.691+0000 7ff819199700 1 -- 172.21.15.33:0/994809048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81000f670 con 0x7ff8140c3530 2023-11-06T21:00:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.691+0000 7ff819199700 1 -- 172.21.15.33:0/994809048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000f9f0 con 0x7ff8140c3530 2023-11-06T21:00:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.691+0000 7ff81c3ff700 1 -- 172.21.15.33:0/994809048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8140c3530 msgr2=0x7ff8140c3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.691+0000 7ff81c3ff700 1 --2- 172.21.15.33:0/994809048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8140c3530 0x7ff8140c3910 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7ff8140bcce0 tx=0x7ff8100090f0).stop 2023-11-06T21:00:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.692+0000 7ff81c3ff700 1 -- 172.21.15.33:0/994809048 shutdown_connections 2023-11-06T21:00:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.692+0000 7ff81c3ff700 1 --2- 172.21.15.33:0/994809048 >> v2:172.21.15.69:3300/0 conn(0x7ff814153d00 0x7ff8141541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.692+0000 7ff81c3ff700 1 --2- 172.21.15.33:0/994809048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8140c3530 0x7ff8140c3910 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.692+0000 7ff81c3ff700 1 --2- 172.21.15.33:0/994809048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8140c2900 0x7ff8140bc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.692+0000 7ff81c3ff700 1 -- 172.21.15.33:0/994809048 >> 172.21.15.33:0/994809048 conn(0x7ff814062710 msgr2=0x7ff814062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.692+0000 7ff81c3ff700 1 -- 172.21.15.33:0/994809048 shutdown_connections 2023-11-06T21:00:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.692+0000 7ff81c3ff700 1 -- 172.21.15.33:0/994809048 wait complete. 2023-11-06T21:00:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.693+0000 7ff81c3ff700 1 Processor -- start 2023-11-06T21:00:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.693+0000 7ff81c3ff700 1 -- start start 2023-11-06T21:00:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.694+0000 7ff81c3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8140c2900 0x7ff8141688c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.694+0000 7ff81c3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8140c3530 0x7ff8140aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.694+0000 7ff81c3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff814153d00 0x7ff8140a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:19.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.694+0000 7ff81c3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140aaf50 con 0x7ff814153d00 2023-11-06T21:00:19.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.694+0000 7ff81c3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140abad0 con 0x7ff8140c2900 2023-11-06T21:00:19.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.694+0000 7ff81c3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140ab1a0 con 0x7ff8140c3530 2023-11-06T21:00:19.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff814153d00 0x7ff8140a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81999a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8140c3530 0x7ff8140aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8140c2900 0x7ff8141688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff814153d00 0x7ff8140a8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43240/0 (socket says 172.21.15.33:43240) 2023-11-06T21:00:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a99c700 1 -- 172.21.15.33:0/3705190198 learned_addr learned my addr 172.21.15.33:0/3705190198 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a19b700 1 -- 172.21.15.33:0/3705190198 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8140c3530 msgr2=0x7ff8140aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a19b700 1 --2- 172.21.15.33:0/3705190198 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8140c3530 0x7ff8140aa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a19b700 1 -- 172.21.15.33:0/3705190198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff814153d00 msgr2=0x7ff8140a8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a19b700 1 --2- 172.21.15.33:0/3705190198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff814153d00 0x7ff8140a8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a19b700 1 -- 172.21.15.33:0/3705190198 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff810009370 con 0x7ff8140c2900 2023-11-06T21:00:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a99c700 1 --2- 172.21.15.33:0/3705190198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff814153d00 0x7ff8140a8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:19.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81a19b700 1 --2- 172.21.15.33:0/3705190198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8140c2900 0x7ff8141688c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7ff810014000 tx=0x7ff810017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:19.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff80a7fc700 1 -- 172.21.15.33:0/3705190198 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810006a40 con 0x7ff8140c2900 2023-11-06T21:00:19.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff80a7fc700 1 -- 172.21.15.33:0/3705190198 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff810011b40 con 0x7ff8140c2900 2023-11-06T21:00:19.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff80a7fc700 1 -- 172.21.15.33:0/3705190198 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81001c420 con 0x7ff8140c2900 2023-11-06T21:00:19.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.695+0000 7ff81c3ff700 1 -- 172.21.15.33:0/3705190198 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8140ab480 con 0x7ff8140c2900 2023-11-06T21:00:19.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.696+0000 7ff81c3ff700 1 -- 172.21.15.33:0/3705190198 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8140aab80 con 0x7ff8140c2900 2023-11-06T21:00:19.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.697+0000 7ff81c3ff700 1 -- 172.21.15.33:0/3705190198 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8140c1e20 con 0x7ff8140c2900 2023-11-06T21:00:19.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.697+0000 7ff80a7fc700 1 -- 172.21.15.33:0/3705190198 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 36) v1 ==== 75270+0+0 (secure 0 0 0) 0x7ff81001c810 con 0x7ff8140c2900 2023-11-06T21:00:19.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.698+0000 7ff80a7fc700 1 -- 172.21.15.33:0/3705190198 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6594+0+0 (secure 0 0 0) 0x7ff81007b7c0 con 0x7ff8140c2900 2023-11-06T21:00:19.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.702+0000 7ff80a7fc700 1 -- 172.21.15.33:0/3705190198 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff81004c970 con 0x7ff8140c2900 2023-11-06T21:00:19.722 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:19.722+0000 7faa81349680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:00:19.819 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:19.819+0000 7faa81349680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:00:19.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:19.930+0000 7ff81c3ff700 1 -- 172.21.15.33:0/3705190198 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff814065900 con 0x7ff8140c2900 2023-11-06T21:00:20.404 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T21:00:20.404+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T21:00:20.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.413+0000 7ff80a7fc700 1 -- 172.21.15.33:0/3705190198 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff81004ff90 con 0x7ff8140c2900 2023-11-06T21:00:20.415 INFO:tasks.ceph.mds.b.smithi069.stderr: -1402> 2023-11-06T21:00:02.265+0000 7f42623f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T21:00:20.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.416+0000 7ff81c3ff700 1 -- 172.21.15.33:0/3705190198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8140c2900 msgr2=0x7ff8141688c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:20.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.416+0000 7ff81c3ff700 1 --2- 172.21.15.33:0/3705190198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8140c2900 0x7ff8141688c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7ff810014000 tx=0x7ff810017000).stop 2023-11-06T21:00:20.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.416+0000 7ff81c3ff700 1 -- 172.21.15.33:0/3705190198 shutdown_connections 2023-11-06T21:00:20.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.416+0000 7ff81c3ff700 1 --2- 172.21.15.33:0/3705190198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff814153d00 0x7ff8140a8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:20.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.416+0000 7ff81c3ff700 1 --2- 172.21.15.33:0/3705190198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8140c2900 0x7ff8141688c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:20.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.416+0000 7ff81c3ff700 1 --2- 172.21.15.33:0/3705190198 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff8140c3530 0x7ff8140aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:20.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.416+0000 7ff81c3ff700 1 -- 172.21.15.33:0/3705190198 >> 172.21.15.33:0/3705190198 conn(0x7ff814062710 msgr2=0x7ff8140c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:20.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.416+0000 7ff81c3ff700 1 -- 172.21.15.33:0/3705190198 shutdown_connections 2023-11-06T21:00:20.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.416+0000 7ff81c3ff700 1 -- 172.21.15.33:0/3705190198 wait complete. 2023-11-06T21:00:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T21:00:20.438 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T21:00:20.458 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T21:00:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.637+0000 7f43553ff700 1 Processor -- start 2023-11-06T21:00:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.637+0000 7f43553ff700 1 -- start start 2023-11-06T21:00:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.638+0000 7f43553ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43500aaf50 0x7f4350063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.638+0000 7f43553ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f43500aa4f0 0x7f4350063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:20.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.638+0000 7f43553ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4350158c00 con 0x7f4350157700 2023-11-06T21:00:20.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.638+0000 7f43553ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43500c18f0 con 0x7f43500aaf50 2023-11-06T21:00:20.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.638+0000 7f43553ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4350063b10 con 0x7f43500aa4f0 2023-11-06T21:00:20.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.638+0000 7f434e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43500aaf50 0x7f4350063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:20.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.638+0000 7f434e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43500aaf50 0x7f4350063130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43250/0 (socket says 172.21.15.33:43250) 2023-11-06T21:00:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.638+0000 7f434e7fc700 1 -- 172.21.15.33:0/1822480145 learned_addr learned my addr 172.21.15.33:0/1822480145 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.639+0000 7f434f7fe700 1 --2- 172.21.15.33:0/1822480145 >> v2:172.21.15.69:3300/0 conn(0x7f43500aa4f0 0x7f4350063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.639+0000 7f434e7fc700 1 -- 172.21.15.33:0/1822480145 >> v2:172.21.15.69:3300/0 conn(0x7f43500aa4f0 msgr2=0x7f4350063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.639+0000 7f434e7fc700 1 --2- 172.21.15.33:0/1822480145 >> v2:172.21.15.69:3300/0 conn(0x7f43500aa4f0 0x7f4350063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.639+0000 7f434e7fc700 1 -- 172.21.15.33:0/1822480145 >> v1:172.21.15.69:6789/0 conn(0x7f4350157700 legacy=0x7f43500c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.639+0000 7f434e7fc700 1 -- 172.21.15.33:0/1822480145 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500c7610 con 0x7f43500aaf50 2023-11-06T21:00:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.639+0000 7f434dffb700 1 -- 172.21.15.33:0/1822480145 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941001024 0 0) 0x7f4350158c00 con 0x7f4350157700 2023-11-06T21:00:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.639+0000 7f434e7fc700 1 --2- 172.21.15.33:0/1822480145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 0x7f4350063130 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f43500c5400 tx=0x7f43440090f0).ready entity=mon.0 client_cookie=d1adb82201740765 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:20.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.639+0000 7f434dffb700 1 -- 172.21.15.33:0/1822480145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434400b520 con 0x7f43500aaf50 2023-11-06T21:00:20.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.639+0000 7f434dffb700 1 -- 172.21.15.33:0/1822480145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434400b680 con 0x7f43500aaf50 2023-11-06T21:00:20.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.639+0000 7f434dffb700 1 -- 172.21.15.33:0/1822480145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434400ba00 con 0x7f43500aaf50 2023-11-06T21:00:20.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.640+0000 7f43553ff700 1 -- 172.21.15.33:0/1822480145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 msgr2=0x7f4350063130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:20.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.640+0000 7f43553ff700 1 --2- 172.21.15.33:0/1822480145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 0x7f4350063130 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f43500c5400 tx=0x7f43440090f0).stop 2023-11-06T21:00:20.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.640+0000 7f43553ff700 1 -- 172.21.15.33:0/1822480145 shutdown_connections 2023-11-06T21:00:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.640+0000 7f43553ff700 1 --2- 172.21.15.33:0/1822480145 >> v2:172.21.15.69:3300/0 conn(0x7f43500aa4f0 0x7f4350063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.640+0000 7f43553ff700 1 --2- 172.21.15.33:0/1822480145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 0x7f4350063130 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.640+0000 7f43553ff700 1 -- 172.21.15.33:0/1822480145 >> 172.21.15.33:0/1822480145 conn(0x7f435005ec80 msgr2=0x7f43500cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.640+0000 7f43553ff700 1 -- 172.21.15.33:0/1822480145 shutdown_connections 2023-11-06T21:00:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.640+0000 7f43553ff700 1 -- 172.21.15.33:0/1822480145 wait complete. 2023-11-06T21:00:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.641+0000 7f43553ff700 1 Processor -- start 2023-11-06T21:00:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.642+0000 7f43553ff700 1 -- start start 2023-11-06T21:00:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f43553ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43500aa4f0 0x7f4350154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f43553ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 0x7f43501561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f43553ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4350157700 0x7f4350150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f43553ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350155580 con 0x7f43500aaf50 2023-11-06T21:00:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f43553ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43501566a0 con 0x7f43500aa4f0 2023-11-06T21:00:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f43553ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4350154970 con 0x7f4350157700 2023-11-06T21:00:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 0x7f43501561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 0x7f43501561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43262/0 (socket says 172.21.15.33:43262) 2023-11-06T21:00:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434e7fc700 1 -- 172.21.15.33:0/2922314917 learned_addr learned my addr 172.21.15.33:0/2922314917 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434effd700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43500aa4f0 0x7f4350154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434f7fe700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4350157700 0x7f4350150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434e7fc700 1 -- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4350157700 msgr2=0x7f4350150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434e7fc700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4350157700 0x7f4350150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434e7fc700 1 -- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43500aa4f0 msgr2=0x7f4350154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434e7fc700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43500aa4f0 0x7f4350154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434e7fc700 1 -- 172.21.15.33:0/2922314917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4344009370 con 0x7f43500aaf50 2023-11-06T21:00:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.643+0000 7f434e7fc700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 0x7f43501561b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f4344009130 tx=0x7f4344017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.644+0000 7f4336ffd700 1 -- 172.21.15.33:0/2922314917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344000bc0 con 0x7f43500aaf50 2023-11-06T21:00:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.644+0000 7f4336ffd700 1 -- 172.21.15.33:0/2922314917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434400b7f0 con 0x7f43500aaf50 2023-11-06T21:00:20.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.644+0000 7f4336ffd700 1 -- 172.21.15.33:0/2922314917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344000d20 con 0x7f43500aaf50 2023-11-06T21:00:20.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.644+0000 7f43553ff700 1 -- 172.21.15.33:0/2922314917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4350154c50 con 0x7f43500aaf50 2023-11-06T21:00:20.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.644+0000 7f43553ff700 1 -- 172.21.15.33:0/2922314917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4350155210 con 0x7f43500aaf50 2023-11-06T21:00:20.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.645+0000 7f43553ff700 1 -- 172.21.15.33:0/2922314917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435006c6c0 con 0x7f43500aaf50 2023-11-06T21:00:20.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.646+0000 7f4336ffd700 1 -- 172.21.15.33:0/2922314917 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 37) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f434401c8c0 con 0x7f43500aaf50 2023-11-06T21:00:20.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.647+0000 7f4336ffd700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f433c05f030 0x7f433c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.647+0000 7f4336ffd700 1 -- 172.21.15.33:0/2922314917 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6680+0+0 (secure 0 0 0) 0x7f434407bd70 con 0x7f43500aaf50 2023-11-06T21:00:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.647+0000 7f434effd700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f433c05f030 0x7f433c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.648+0000 7f434effd700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f433c05f030 0x7f433c0614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f4340007fd0 tx=0x7f4340007bb0).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.651+0000 7f4336ffd700 1 -- 172.21.15.33:0/2922314917 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f434404cfe0 con 0x7f43500aaf50 2023-11-06T21:00:20.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:20.875+0000 7f43553ff700 1 -- 172.21.15.33:0/2922314917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4350062b60 con 0x7f43500aaf50 2023-11-06T21:00:21.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.421+0000 7f4336ffd700 1 -- 172.21.15.33:0/2922314917 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7f434402a020 con 0x7f43500aaf50 2023-11-06T21:00:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 -- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f433c05f030 msgr2=0x7f433c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f433c05f030 0x7f433c0614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f4340007fd0 tx=0x7f4340007bb0).stop 2023-11-06T21:00:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 -- 172.21.15.33:0/2922314917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 msgr2=0x7f43501561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 0x7f43501561b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f4344009130 tx=0x7f4344017000).stop 2023-11-06T21:00:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 -- 172.21.15.33:0/2922314917 shutdown_connections 2023-11-06T21:00:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f433c05f030 0x7f433c0614f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4350157700 0x7f4350150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43500aa4f0 0x7f4350154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 --2- 172.21.15.33:0/2922314917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43500aaf50 0x7f43501561b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 -- 172.21.15.33:0/2922314917 >> 172.21.15.33:0/2922314917 conn(0x7f435005ec80 msgr2=0x7f4350151290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 -- 172.21.15.33:0/2922314917 shutdown_connections 2023-11-06T21:00:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.424+0000 7f43553ff700 1 -- 172.21.15.33:0/2922314917 wait complete. 2023-11-06T21:00:21.444 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T21:00:21.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.647+0000 7f1eccbff700 1 Processor -- start 2023-11-06T21:00:21.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.647+0000 7f1eccbff700 1 -- start start 2023-11-06T21:00:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.648+0000 7f1eccbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1ec81510e0 0x7f1ec80a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.648+0000 7f1eccbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ec80bc090 0x7f1ec815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.648+0000 7f1eccbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec8154c60 con 0x7f1ec81514c0 2023-11-06T21:00:21.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.648+0000 7f1eccbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec8068f50 con 0x7f1ec80bc090 2023-11-06T21:00:21.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.648+0000 7f1eccbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec8151dc0 con 0x7f1ec81510e0 2023-11-06T21:00:21.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.649+0000 7f1ec6d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ec80bc090 0x7f1ec815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:21.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.649+0000 7f1ec6d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ec80bc090 0x7f1ec815ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43266/0 (socket says 172.21.15.33:43266) 2023-11-06T21:00:21.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.649+0000 7f1ec6d9d700 1 -- 172.21.15.33:0/3065745669 learned_addr learned my addr 172.21.15.33:0/3065745669 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:21.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.649+0000 7f1ec659c700 1 --2- 172.21.15.33:0/3065745669 >> v2:172.21.15.69:3300/0 conn(0x7f1ec81510e0 0x7f1ec80a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.649+0000 7f1ec6d9d700 1 -- 172.21.15.33:0/3065745669 >> v2:172.21.15.69:3300/0 conn(0x7f1ec81510e0 msgr2=0x7f1ec80a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.649+0000 7f1ec6d9d700 1 --2- 172.21.15.33:0/3065745669 >> v2:172.21.15.69:3300/0 conn(0x7f1ec81510e0 0x7f1ec80a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.649+0000 7f1ec6d9d700 1 -- 172.21.15.33:0/3065745669 >> v1:172.21.15.69:6789/0 conn(0x7f1ec81514c0 legacy=0x7f1ec80bb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.649+0000 7f1ec6d9d700 1 -- 172.21.15.33:0/3065745669 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec81520a0 con 0x7f1ec80bc090 2023-11-06T21:00:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.649+0000 7f1ec6d9d700 1 --2- 172.21.15.33:0/3065745669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ec80bc090 0x7f1ec815ae90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f1ec806be70 tx=0x7f1ec0009840).ready entity=mon.0 client_cookie=d74a1e5328f4ebf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.649+0000 7f1ec559a700 1 -- 172.21.15.33:0/3065745669 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000e070 con 0x7f1ec80bc090 2023-11-06T21:00:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.650+0000 7f1ec559a700 1 -- 172.21.15.33:0/3065745669 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec000ba20 con 0x7f1ec80bc090 2023-11-06T21:00:21.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.650+0000 7f1ec559a700 1 -- 172.21.15.33:0/3065745669 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000bda0 con 0x7f1ec80bc090 2023-11-06T21:00:21.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.650+0000 7f1eccbff700 1 -- 172.21.15.33:0/3065745669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ec80bc090 msgr2=0x7f1ec815ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:21.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.650+0000 7f1eccbff700 1 --2- 172.21.15.33:0/3065745669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ec80bc090 0x7f1ec815ae90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f1ec806be70 tx=0x7f1ec0009840).stop 2023-11-06T21:00:21.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.650+0000 7f1eccbff700 1 -- 172.21.15.33:0/3065745669 shutdown_connections 2023-11-06T21:00:21.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.650+0000 7f1eccbff700 1 --2- 172.21.15.33:0/3065745669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ec80bc090 0x7f1ec815ae90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.650+0000 7f1eccbff700 1 --2- 172.21.15.33:0/3065745669 >> v2:172.21.15.69:3300/0 conn(0x7f1ec81510e0 0x7f1ec80a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.650+0000 7f1eccbff700 1 -- 172.21.15.33:0/3065745669 >> 172.21.15.33:0/3065745669 conn(0x7f1ec8062710 msgr2=0x7f1ec8062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.650+0000 7f1eccbff700 1 -- 172.21.15.33:0/3065745669 shutdown_connections 2023-11-06T21:00:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.650+0000 7f1eccbff700 1 -- 172.21.15.33:0/3065745669 wait complete. 2023-11-06T21:00:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.652+0000 7f1eccbff700 1 Processor -- start 2023-11-06T21:00:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.652+0000 7f1eccbff700 1 -- start start 2023-11-06T21:00:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1eccbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ec80bc090 0x7f1ec8150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1eccbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ec81510e0 0x7f1ec80bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1eccbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ec81514c0 0x7f1ec81680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1eccbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80bed60 con 0x7f1ec81510e0 2023-11-06T21:00:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1eccbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80bf6e0 con 0x7f1ec80bc090 2023-11-06T21:00:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1eccbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80be150 con 0x7f1ec81514c0 2023-11-06T21:00:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1ec5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ec81510e0 0x7f1ec80bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1ec659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ec80bc090 0x7f1ec8150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1ec659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ec80bc090 0x7f1ec8150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:59226/0 (socket says 172.21.15.33:59226) 2023-11-06T21:00:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1ec659c700 1 -- 172.21.15.33:0/1462681490 learned_addr learned my addr 172.21.15.33:0/1462681490 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.653+0000 7f1ec6d9d700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ec81514c0 0x7f1ec81680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1ec659c700 1 -- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ec81514c0 msgr2=0x7f1ec81680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1ec659c700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ec81514c0 0x7f1ec81680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1ec659c700 1 -- 172.21.15.33:0/1462681490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ec81510e0 msgr2=0x7f1ec80bf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1ec659c700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ec81510e0 0x7f1ec80bf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1ec659c700 1 -- 172.21.15.33:0/1462681490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec0009ae0 con 0x7f1ec80bc090 2023-11-06T21:00:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1ec5d9b700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ec81510e0 0x7f1ec80bf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1ec659c700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ec80bc090 0x7f1ec8150920 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f1eb800e930 tx=0x7f1eb8009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1eb67fc700 1 -- 172.21.15.33:0/1462681490 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8012600 con 0x7f1ec80bc090 2023-11-06T21:00:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1eb67fc700 1 -- 172.21.15.33:0/1462681490 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb8012760 con 0x7f1ec80bc090 2023-11-06T21:00:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1eccbff700 1 -- 172.21.15.33:0/1462681490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec80be430 con 0x7f1ec80bc090 2023-11-06T21:00:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1eb67fc700 1 -- 172.21.15.33:0/1462681490 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8012ae0 con 0x7f1ec80bc090 2023-11-06T21:00:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.654+0000 7f1eccbff700 1 -- 172.21.15.33:0/1462681490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec80be9f0 con 0x7f1ec80bc090 2023-11-06T21:00:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.655+0000 7f1eccbff700 1 -- 172.21.15.33:0/1462681490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec80bf040 con 0x7f1ec80bc090 2023-11-06T21:00:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.656+0000 7f1eb67fc700 1 -- 172.21.15.33:0/1462681490 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 37) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f1eb801b400 con 0x7f1ec80bc090 2023-11-06T21:00:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.657+0000 7f1eb67fc700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1ea805f000 0x7f1ea80614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.657+0000 7f1eb67fc700 1 -- 172.21.15.33:0/1462681490 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6680+0+0 (secure 0 0 0) 0x7f1eb807a5c0 con 0x7f1ec80bc090 2023-11-06T21:00:21.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.657+0000 7f1ec5d9b700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1ea805f000 0x7f1ea80614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:21.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.658+0000 7f1ec5d9b700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1ea805f000 0x7f1ea80614c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f1ec814be70 tx=0x7f1ebc009040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:21.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.662+0000 7f1eb67fc700 1 -- 172.21.15.33:0/1462681490 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1eb804b7d0 con 0x7f1ec80bc090 2023-11-06T21:00:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:21.846+0000 7f1eccbff700 1 -- 172.21.15.33:0/1462681490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ec8152380 con 0x7f1ec80bc090 2023-11-06T21:00:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.225+0000 7f1eb67fc700 1 -- 172.21.15.33:0/1462681490 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1eb8078330 con 0x7f1ec80bc090 2023-11-06T21:00:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.425+0000 7f1eb67fc700 1 -- 172.21.15.33:0/1462681490 <== mon.1 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1eb8071dc0 con 0x7f1ec80bc090 2023-11-06T21:00:22.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.429+0000 7f1eccbff700 1 -- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1ea805f000 msgr2=0x7f1ea80614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:22.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.429+0000 7f1eccbff700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1ea805f000 0x7f1ea80614c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f1ec814be70 tx=0x7f1ebc009040).stop 2023-11-06T21:00:22.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.429+0000 7f1eccbff700 1 -- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ec80bc090 msgr2=0x7f1ec8150920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.429+0000 7f1eccbff700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ec80bc090 0x7f1ec8150920 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f1eb800e930 tx=0x7f1eb8009060).stop 2023-11-06T21:00:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.429+0000 7f1eccbff700 1 -- 172.21.15.33:0/1462681490 shutdown_connections 2023-11-06T21:00:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.429+0000 7f1eccbff700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1ea805f000 0x7f1ea80614c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.429+0000 7f1eccbff700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ec81514c0 0x7f1ec81680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.429+0000 7f1eccbff700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ec80bc090 0x7f1ec8150920 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.429+0000 7f1eccbff700 1 --2- 172.21.15.33:0/1462681490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ec81510e0 0x7f1ec80bf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.429+0000 7f1eccbff700 1 -- 172.21.15.33:0/1462681490 >> 172.21.15.33:0/1462681490 conn(0x7f1ec8062710 msgr2=0x7f1ec80c3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.430+0000 7f1eccbff700 1 -- 172.21.15.33:0/1462681490 shutdown_connections 2023-11-06T21:00:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.430+0000 7f1eccbff700 1 -- 172.21.15.33:0/1462681490 wait complete. 2023-11-06T21:00:22.433 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T21:00:22.452 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T21:00:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.654+0000 7f73bd1ff700 1 Processor -- start 2023-11-06T21:00:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.654+0000 7f73bd1ff700 1 -- start start 2023-11-06T21:00:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.655+0000 7f73bd1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f73b80c8ee0 0x7f73b80ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.655+0000 7f73bd1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b80c9f80 con 0x7f73b8153ed0 2023-11-06T21:00:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.655+0000 7f73bd1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b80c6000 con 0x7f73b81542b0 2023-11-06T21:00:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.655+0000 7f73bd1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b80c9370 con 0x7f73b80c8ee0 2023-11-06T21:00:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.656+0000 7f73b6d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f73b8153ed0 0x7f73b8158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38534/0 (socket says 172.21.15.33:38534) 2023-11-06T21:00:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.656+0000 7f73b6d9d700 1 -- 172.21.15.33:0/922001122 learned_addr learned my addr 172.21.15.33:0/922001122 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.656+0000 7f73b659c700 1 --2- 172.21.15.33:0/922001122 >> v2:172.21.15.69:3300/0 conn(0x7f73b80c8ee0 0x7f73b80ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.656+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419859808 0 0) 0x7f73b80c9f80 con 0x7f73b8153ed0 2023-11-06T21:00:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.656+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73a00033f0 con 0x7f73b8153ed0 2023-11-06T21:00:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 884877824 0 0) 0x7f73a00033f0 con 0x7f73b8153ed0 2023-11-06T21:00:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 >> v2:172.21.15.69:3300/0 conn(0x7f73b80c8ee0 msgr2=0x7f73b80ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 --2- 172.21.15.33:0/922001122 >> v2:172.21.15.69:3300/0 conn(0x7f73b80c8ee0 0x7f73b80ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 >> v1:172.21.15.69:6789/0 conn(0x7f73b81542b0 legacy=0x7f73b80c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b80c9650 con 0x7f73b8153ed0 2023-11-06T21:00:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f73a8001470 con 0x7f73b8153ed0 2023-11-06T21:00:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 >> v1:172.21.15.33:6789/0 conn(0x7f73b8153ed0 legacy=0x7f73b8158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a00038e0 0x7f73a0003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f73a0004690 0x7f73a0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73a0005000 0x7f73a000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80c9370 con 0x7f73a00038e0 2023-11-06T21:00:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a000beb0 con 0x7f73a0005000 2023-11-06T21:00:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a000c070 con 0x7f73a0004690 2023-11-06T21:00:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b6d9d700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a00038e0 0x7f73a0003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b659c700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f73a0004690 0x7f73a0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b759e700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73a0005000 0x7f73a000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b6d9d700 1 -- 172.21.15.33:0/922001122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f73a0004690 msgr2=0x7f73a0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b6d9d700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f73a0004690 0x7f73a0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b6d9d700 1 -- 172.21.15.33:0/922001122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73a0005000 msgr2=0x7f73a000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b6d9d700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73a0005000 0x7f73a000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b6d9d700 1 -- 172.21.15.33:0/922001122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a00033f0 con 0x7f73a00038e0 2023-11-06T21:00:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.657+0000 7f73b759e700 1 -- 172.21.15.33:0/922001122 reap_dead start 2023-11-06T21:00:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.658+0000 7f73b6d9d700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a00038e0 0x7f73a0003cc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f73a800c110 tx=0x7f73a8001df0).ready entity=mon.0 client_cookie=1e4776985258b24b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.658+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a800ca40 con 0x7f73a00038e0 2023-11-06T21:00:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.658+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a800cba0 con 0x7f73a00038e0 2023-11-06T21:00:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.658+0000 7f73b5d9b700 1 -- 172.21.15.33:0/922001122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a80070d0 con 0x7f73a00038e0 2023-11-06T21:00:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.658+0000 7f73bd1ff700 1 -- 172.21.15.33:0/922001122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a00038e0 msgr2=0x7f73a0003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.658+0000 7f73bd1ff700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a00038e0 0x7f73a0003cc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f73a800c110 tx=0x7f73a8001df0).stop 2023-11-06T21:00:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.658+0000 7f73bd1ff700 1 -- 172.21.15.33:0/922001122 shutdown_connections 2023-11-06T21:00:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.658+0000 7f73bd1ff700 1 --2- 172.21.15.33:0/922001122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a00038e0 0x7f73a0003cc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.658+0000 7f73bd1ff700 1 -- 172.21.15.33:0/922001122 >> 172.21.15.33:0/922001122 conn(0x7f73b805e1b0 msgr2=0x7f73b805e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.659+0000 7f73bd1ff700 1 -- 172.21.15.33:0/922001122 shutdown_connections 2023-11-06T21:00:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.659+0000 7f73bd1ff700 1 -- 172.21.15.33:0/922001122 wait complete. 2023-11-06T21:00:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.660+0000 7f73bd1ff700 1 Processor -- start 2023-11-06T21:00:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.660+0000 7f73bd1ff700 1 -- start start 2023-11-06T21:00:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73bd1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f73a00038e0 0x7f73b814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73bd1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73b80c2790 0x7f73b8165480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73bd1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80addd0 0x7f73b8152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73bd1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b8164a90 con 0x7f73b80addd0 2023-11-06T21:00:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73bd1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b814f070 con 0x7f73b80c2790 2023-11-06T21:00:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73bd1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80aebe0 con 0x7f73a00038e0 2023-11-06T21:00:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73b759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80addd0 0x7f73b8152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73b759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80addd0 0x7f73b8152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43288/0 (socket says 172.21.15.33:43288) 2023-11-06T21:00:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73b759e700 1 -- 172.21.15.33:0/2785155124 learned_addr learned my addr 172.21.15.33:0/2785155124 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73b659c700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73b80c2790 0x7f73b8165480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.661+0000 7f73b6d9d700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f73a00038e0 0x7f73b814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f73b759e700 1 -- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f73a00038e0 msgr2=0x7f73b814eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f73b759e700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f73a00038e0 0x7f73b814eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f73b759e700 1 -- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73b80c2790 msgr2=0x7f73b8165480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f73b759e700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73b80c2790 0x7f73b8165480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f73b759e700 1 -- 172.21.15.33:0/2785155124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b80ae620 con 0x7f73b80addd0 2023-11-06T21:00:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f73b659c700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73b80c2790 0x7f73b8165480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f73b759e700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80addd0 0x7f73b8152ce0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f73b00013d0 tx=0x7f73b0007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f739effd700 1 -- 172.21.15.33:0/2785155124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0004100 con 0x7f73b80addd0 2023-11-06T21:00:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f739effd700 1 -- 172.21.15.33:0/2785155124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b0006a20 con 0x7f73b80addd0 2023-11-06T21:00:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f739effd700 1 -- 172.21.15.33:0/2785155124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0006dc0 con 0x7f73b80addd0 2023-11-06T21:00:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f73bd1ff700 1 -- 172.21.15.33:0/2785155124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b80ae900 con 0x7f73b80addd0 2023-11-06T21:00:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.662+0000 7f73bd1ff700 1 -- 172.21.15.33:0/2785155124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b8164f60 con 0x7f73b80addd0 2023-11-06T21:00:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.664+0000 7f73bd1ff700 1 -- 172.21.15.33:0/2785155124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b806c6c0 con 0x7f73b80addd0 2023-11-06T21:00:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.664+0000 7f739effd700 1 -- 172.21.15.33:0/2785155124 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f73b000c7b0 con 0x7f73b80addd0 2023-11-06T21:00:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.665+0000 7f739effd700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f73a405f270 0x7f73a4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.665+0000 7f739effd700 1 -- 172.21.15.33:0/2785155124 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6242+0+0 (secure 0 0 0) 0x7f73b0079c50 con 0x7f73b80addd0 2023-11-06T21:00:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.666+0000 7f73b6d9d700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f73a405f270 0x7f73a4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.666+0000 7f73b6d9d700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f73a405f270 0x7f73a4061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f73a80010e0 tx=0x7f73a8015000).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.669+0000 7f739effd700 1 -- 172.21.15.33:0/2785155124 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f73b0049af0 con 0x7f73b80addd0 2023-11-06T21:00:22.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:22.853+0000 7f73bd1ff700 1 -- 172.21.15.33:0/2785155124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73b80c9930 con 0x7f73b80addd0 2023-11-06T21:00:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.431+0000 7f739effd700 1 -- 172.21.15.33:0/2785155124 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) v1 ==== 169+0+0 (secure 0 0 0) 0x7f73b004e750 con 0x7f73b80addd0 2023-11-06T21:00:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.435+0000 7f73bd1ff700 1 -- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f73a405f270 msgr2=0x7f73a4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.435+0000 7f73bd1ff700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f73a405f270 0x7f73a4061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f73a80010e0 tx=0x7f73a8015000).stop 2023-11-06T21:00:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.435+0000 7f73bd1ff700 1 -- 172.21.15.33:0/2785155124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80addd0 msgr2=0x7f73b8152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.435+0000 7f73bd1ff700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80addd0 0x7f73b8152ce0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f73b00013d0 tx=0x7f73b0007f30).stop 2023-11-06T21:00:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.436+0000 7f73bd1ff700 1 -- 172.21.15.33:0/2785155124 shutdown_connections 2023-11-06T21:00:23.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.436+0000 7f73bd1ff700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f73a405f270 0x7f73a4061730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.436+0000 7f73bd1ff700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f73a00038e0 0x7f73b814eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.436+0000 7f73bd1ff700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80addd0 0x7f73b8152ce0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.436+0000 7f73bd1ff700 1 --2- 172.21.15.33:0/2785155124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73b80c2790 0x7f73b8165480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.436+0000 7f73bd1ff700 1 -- 172.21.15.33:0/2785155124 >> 172.21.15.33:0/2785155124 conn(0x7f73b805e1b0 msgr2=0x7f73b80cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.437+0000 7f73bd1ff700 1 -- 172.21.15.33:0/2785155124 shutdown_connections 2023-11-06T21:00:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.438+0000 7f73bd1ff700 1 -- 172.21.15.33:0/2785155124 wait complete. 2023-11-06T21:00:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T21:00:23.462 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T21:00:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.662+0000 7f931d3ff700 1 Processor -- start 2023-11-06T21:00:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.662+0000 7f931d3ff700 1 -- start start 2023-11-06T21:00:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.663+0000 7f931d3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9318157700 0x7f93180c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.663+0000 7f931d3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9318158c00 con 0x7f9318063840 2023-11-06T21:00:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.663+0000 7f931d3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9318157ff0 con 0x7f93180aaf50 2023-11-06T21:00:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.663+0000 7f931d3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93180c18f0 con 0x7f9318157700 2023-11-06T21:00:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f93177fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9318063840 0x7f93180c5d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38544/0 (socket says 172.21.15.33:38544) 2023-11-06T21:00:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f93177fe700 1 -- 172.21.15.33:0/2236104061 learned_addr learned my addr 172.21.15.33:0/2236104061 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9316ffd700 1 --2- 172.21.15.33:0/2236104061 >> v2:172.21.15.69:3300/0 conn(0x7f9318157700 0x7f93180c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352992108 0 0) 0x7f9318158c00 con 0x7f9318063840 2023-11-06T21:00:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93000033f0 con 0x7f9318063840 2023-11-06T21:00:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687882405 0 0) 0x7f9318157ff0 con 0x7f93180aaf50 2023-11-06T21:00:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9318158c00 con 0x7f93180aaf50 2023-11-06T21:00:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1542565727 0 0) 0x7f93000033f0 con 0x7f9318063840 2023-11-06T21:00:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 >> v2:172.21.15.69:3300/0 conn(0x7f9318157700 msgr2=0x7f93180c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9315ffb700 1 --2- 172.21.15.33:0/2236104061 >> v2:172.21.15.69:3300/0 conn(0x7f9318157700 0x7f93180c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 >> v1:172.21.15.69:6789/0 conn(0x7f93180aaf50 legacy=0x7f9318063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93181582d0 con 0x7f9318063840 2023-11-06T21:00:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.664+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f93100017f0 con 0x7f9318063840 2023-11-06T21:00:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 >> v1:172.21.15.33:6789/0 conn(0x7f9318063840 legacy=0x7f93180c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f9315ffb700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9300003a70 0x7f9300003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:23.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f9315ffb700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9300004690 0x7f9300004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f9315ffb700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9300005000 0x7f930000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180c18f0 con 0x7f9300004690 2023-11-06T21:00:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930000be80 con 0x7f9300005000 2023-11-06T21:00:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930000c040 con 0x7f9300003a70 2023-11-06T21:00:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f93167fc700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9300004690 0x7f9300004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:23.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f9316ffd700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9300003a70 0x7f9300003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:23.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f93177fe700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9300005000 0x7f930000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:23.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f93167fc700 1 -- 172.21.15.33:0/2236104061 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9300003a70 msgr2=0x7f9300003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:23.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f93167fc700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9300003a70 0x7f9300003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f93167fc700 1 -- 172.21.15.33:0/2236104061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9300005000 msgr2=0x7f930000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:23.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f93167fc700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9300005000 0x7f930000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f93167fc700 1 -- 172.21.15.33:0/2236104061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93000033f0 con 0x7f9300004690 2023-11-06T21:00:23.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.665+0000 7f93177fe700 1 -- 172.21.15.33:0/2236104061 reap_dead start 2023-11-06T21:00:23.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f93167fc700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9300004690 0x7f9300004b10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f930c001860 tx=0x7f930c007bb0).ready entity=mon.0 client_cookie=16c70968ef59cc5e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:23.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c007020 con 0x7f9300004690 2023-11-06T21:00:23.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c007180 con 0x7f9300004690 2023-11-06T21:00:23.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f9315ffb700 1 -- 172.21.15.33:0/2236104061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c007500 con 0x7f9300004690 2023-11-06T21:00:23.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f931d3ff700 1 -- 172.21.15.33:0/2236104061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9300004690 msgr2=0x7f9300004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:23.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f931d3ff700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9300004690 0x7f9300004b10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f930c001860 tx=0x7f930c007bb0).stop 2023-11-06T21:00:23.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f931d3ff700 1 -- 172.21.15.33:0/2236104061 shutdown_connections 2023-11-06T21:00:23.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f931d3ff700 1 --2- 172.21.15.33:0/2236104061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9300004690 0x7f9300004b10 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f931d3ff700 1 -- 172.21.15.33:0/2236104061 >> 172.21.15.33:0/2236104061 conn(0x7f931805ec80 msgr2=0x7f93180cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:23.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f931d3ff700 1 -- 172.21.15.33:0/2236104061 shutdown_connections 2023-11-06T21:00:23.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.666+0000 7f931d3ff700 1 -- 172.21.15.33:0/2236104061 wait complete. 2023-11-06T21:00:23.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.668+0000 7f931d3ff700 1 Processor -- start 2023-11-06T21:00:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.668+0000 7f931d3ff700 1 -- start start 2023-11-06T21:00:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.669+0000 7f931d3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9300004690 0x7f93181657e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.669+0000 7f931d3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9318167060 0x7f93181674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.669+0000 7f931d3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93180b0260 0x7f93180b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.669+0000 7f931d3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93181679d0 con 0x7f9318167060 2023-11-06T21:00:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.669+0000 7f931d3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180b0e80 con 0x7f93180b0260 2023-11-06T21:00:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.669+0000 7f931d3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93180b0bb0 con 0x7f9300004690 2023-11-06T21:00:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.669+0000 7f93167fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9318167060 0x7f93181674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.669+0000 7f93167fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9318167060 0x7f93181674e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43306/0 (socket says 172.21.15.33:43306) 2023-11-06T21:00:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.669+0000 7f93167fc700 1 -- 172.21.15.33:0/347610368 learned_addr learned my addr 172.21.15.33:0/347610368 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:23.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f93177fe700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93180b0260 0x7f93180b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f9316ffd700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9300004690 0x7f93181657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f93167fc700 1 -- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9300004690 msgr2=0x7f93181657e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f93167fc700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9300004690 0x7f93181657e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f93167fc700 1 -- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93180b0260 msgr2=0x7f93180b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f93167fc700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93180b0260 0x7f93180b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f93167fc700 1 -- 172.21.15.33:0/347610368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180ab480 con 0x7f9318167060 2023-11-06T21:00:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f9316ffd700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9300004690 0x7f93181657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f93177fe700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93180b0260 0x7f93180b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f93167fc700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9318167060 0x7f93181674e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f930c001e50 tx=0x7f930c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f92feffd700 1 -- 172.21.15.33:0/347610368 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c012610 con 0x7f9318167060 2023-11-06T21:00:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f92feffd700 1 -- 172.21.15.33:0/347610368 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c012770 con 0x7f9318167060 2023-11-06T21:00:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f92feffd700 1 -- 172.21.15.33:0/347610368 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c012900 con 0x7f9318167060 2023-11-06T21:00:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f931d3ff700 1 -- 172.21.15.33:0/347610368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93180ab760 con 0x7f9318167060 2023-11-06T21:00:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.670+0000 7f931d3ff700 1 -- 172.21.15.33:0/347610368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93180abd20 con 0x7f9318167060 2023-11-06T21:00:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.671+0000 7f931d3ff700 1 -- 172.21.15.33:0/347610368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93180abee0 con 0x7f9318167060 2023-11-06T21:00:23.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.672+0000 7f92feffd700 1 -- 172.21.15.33:0/347610368 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f930c018020 con 0x7f9318167060 2023-11-06T21:00:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.673+0000 7f92feffd700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f930405f270 0x7f9304061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.673+0000 7f92feffd700 1 -- 172.21.15.33:0/347610368 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5828+0+0 (secure 0 0 0) 0x7f930c07a8c0 con 0x7f9318167060 2023-11-06T21:00:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.673+0000 7f9316ffd700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f930405f270 0x7f9304061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.674+0000 7f9316ffd700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f930405f270 0x7f9304061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9308006c40 tx=0x7f9308008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.677+0000 7f92feffd700 1 -- 172.21.15.33:0/347610368 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f930c04bb40 con 0x7f9318167060 2023-11-06T21:00:23.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.861+0000 7f931d3ff700 1 -- 172.21.15.33:0/347610368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f93180ac090 con 0x7f9318167060 2023-11-06T21:00:23.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.865+0000 7f92feffd700 1 -- 172.21.15.33:0/347610368 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v16) v1 ==== 120+0+0 (secure 0 0 0) 0x7f930c022020 con 0x7f9318167060 2023-11-06T21:00:23.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.867+0000 7f931d3ff700 1 -- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f930405f270 msgr2=0x7f9304061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:23.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.867+0000 7f931d3ff700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f930405f270 0x7f9304061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9308006c40 tx=0x7f9308008040).stop 2023-11-06T21:00:23.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.867+0000 7f931d3ff700 1 -- 172.21.15.33:0/347610368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9318167060 msgr2=0x7f93181674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:23.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.867+0000 7f931d3ff700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9318167060 0x7f93181674e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f930c001e50 tx=0x7f930c015000).stop 2023-11-06T21:00:23.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.867+0000 7f931d3ff700 1 -- 172.21.15.33:0/347610368 shutdown_connections 2023-11-06T21:00:23.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.867+0000 7f931d3ff700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f930405f270 0x7f9304061730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.867+0000 7f931d3ff700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9300004690 0x7f93181657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.867+0000 7f931d3ff700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93180b0260 0x7f93180b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.867+0000 7f931d3ff700 1 --2- 172.21.15.33:0/347610368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9318167060 0x7f93181674e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:23.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.867+0000 7f931d3ff700 1 -- 172.21.15.33:0/347610368 >> 172.21.15.33:0/347610368 conn(0x7f931805ec80 msgr2=0x7f93180c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:23.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.868+0000 7f931d3ff700 1 -- 172.21.15.33:0/347610368 shutdown_connections 2023-11-06T21:00:23.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:23.868+0000 7f931d3ff700 1 -- 172.21.15.33:0/347610368 wait complete. 2023-11-06T21:00:23.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T21:00:24.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.083+0000 7f416a5ff700 1 Processor -- start 2023-11-06T21:00:24.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.084+0000 7f416a5ff700 1 -- start start 2023-11-06T21:00:24.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.085+0000 7f416a5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4164155030 0x7f4164062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.085+0000 7f416a5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f41641547d0 0x7f416415e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.085+0000 7f416a5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4164063c00 con 0x7f4164155410 2023-11-06T21:00:24.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.085+0000 7f416a5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4164064090 con 0x7f41641547d0 2023-11-06T21:00:24.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.085+0000 7f416a5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4164062ff0 con 0x7f4164155030 2023-11-06T21:00:24.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f41637fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4164155410 0x7f41640644f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38546/0 (socket says 172.21.15.33:38546) 2023-11-06T21:00:24.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f41637fe700 1 -- 172.21.15.33:0/1537143862 learned_addr learned my addr 172.21.15.33:0/1537143862 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:24.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4168b9c700 1 --2- 172.21.15.33:0/1537143862 >> v2:172.21.15.69:3300/0 conn(0x7f41641547d0 0x7f416415e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4163fff700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4164155030 0x7f4164062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686059300 0 0) 0x7f4164063c00 con 0x7f4164155410 2023-11-06T21:00:24.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f414c0033f0 con 0x7f4164155410 2023-11-06T21:00:24.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3456130388 0 0) 0x7f414c0033f0 con 0x7f4164155410 2023-11-06T21:00:24.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4164155030 msgr2=0x7f4164062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:24.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4164155030 0x7f4164062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 >> v2:172.21.15.69:3300/0 conn(0x7f41641547d0 msgr2=0x7f416415e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:24.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 --2- 172.21.15.33:0/1537143862 >> v2:172.21.15.69:3300/0 conn(0x7f41641547d0 0x7f416415e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41640632d0 con 0x7f4164155410 2023-11-06T21:00:24.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f4158001470 con 0x7f4164155410 2023-11-06T21:00:24.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 >> v1:172.21.15.33:6789/0 conn(0x7f4164155410 legacy=0x7f41640644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:24.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.086+0000 7f4162ffd700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 0x7f414c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4162ffd700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f414c004690 0x7f414c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4162ffd700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4164155030 0x7f414c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4164064090 con 0x7f414c0038e0 2023-11-06T21:00:24.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4164062ff0 con 0x7f414c004690 2023-11-06T21:00:24.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c00b8e0 con 0x7f4164155030 2023-11-06T21:00:24.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4168b9c700 1 --2- 172.21.15.33:0/1537143862 >> v2:172.21.15.69:3300/0 conn(0x7f41641547d0 0x7f416415e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:24.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4163fff700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 0x7f414c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f41637fe700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f414c004690 0x7f414c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4168b9c700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4164155030 0x7f414c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4163fff700 1 -- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4164155030 msgr2=0x7f414c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:24.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4163fff700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4164155030 0x7f414c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4163fff700 1 -- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f414c004690 msgr2=0x7f414c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:24.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4163fff700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f414c004690 0x7f414c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4163fff700 1 -- 172.21.15.33:0/1537143862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c0033f0 con 0x7f414c0038e0 2023-11-06T21:00:24.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4168b9c700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4164155030 0x7f414c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:24.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.087+0000 7f4163fff700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 0x7f414c003d00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f4154009a80 tx=0x7f4154009060).ready entity=mon.0 client_cookie=e689639e71b4a5ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:24.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.088+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154009430 con 0x7f414c0038e0 2023-11-06T21:00:24.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.088+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415400e640 con 0x7f414c0038e0 2023-11-06T21:00:24.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.088+0000 7f4162ffd700 1 -- 172.21.15.33:0/1537143862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415400e9c0 con 0x7f414c0038e0 2023-11-06T21:00:24.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.088+0000 7f416a5ff700 1 -- 172.21.15.33:0/1537143862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 msgr2=0x7f414c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:24.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.088+0000 7f416a5ff700 1 --2- 172.21.15.33:0/1537143862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 0x7f414c003d00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f4154009a80 tx=0x7f4154009060).stop 2023-11-06T21:00:24.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.088+0000 7f4168b9c700 1 -- 172.21.15.33:0/1537143862 reap_dead start 2023-11-06T21:00:24.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.088+0000 7f416a5ff700 1 -- 172.21.15.33:0/1537143862 shutdown_connections 2023-11-06T21:00:24.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.088+0000 7f416a5ff700 1 -- 172.21.15.33:0/1537143862 >> 172.21.15.33:0/1537143862 conn(0x7f41640c3d30 msgr2=0x7f41640c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:24.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.088+0000 7f416a5ff700 1 -- 172.21.15.33:0/1537143862 shutdown_connections 2023-11-06T21:00:24.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.088+0000 7f416a5ff700 1 -- 172.21.15.33:0/1537143862 wait complete. 2023-11-06T21:00:24.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.090+0000 7f416a5ff700 1 Processor -- start 2023-11-06T21:00:24.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.091+0000 7f416a5ff700 1 -- start start 2023-11-06T21:00:24.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.092+0000 7f416a5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 0x7f4164163e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.092+0000 7f416a5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41640a9af0 0x7f4164167c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.092+0000 7f416a5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416413d420 0x7f4164168110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.092+0000 7f416a5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416413c760 con 0x7f414c0038e0 2023-11-06T21:00:24.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.092+0000 7f416a5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41640a74c0 con 0x7f416413d420 2023-11-06T21:00:24.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.092+0000 7f416a5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416413cf90 con 0x7f41640a9af0 2023-11-06T21:00:24.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.092+0000 7f4163fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 0x7f4164163e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.092+0000 7f4163fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 0x7f4164163e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43322/0 (socket says 172.21.15.33:43322) 2023-11-06T21:00:24.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.092+0000 7f4163fff700 1 -- 172.21.15.33:0/2901298460 learned_addr learned my addr 172.21.15.33:0/2901298460 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:24.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f41637fe700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41640a9af0 0x7f4164167c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f4168b9c700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416413d420 0x7f4164168110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f4163fff700 1 -- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41640a9af0 msgr2=0x7f4164167c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:24.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f4163fff700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41640a9af0 0x7f4164167c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f4163fff700 1 -- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416413d420 msgr2=0x7f4164168110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:24.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f4163fff700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416413d420 0x7f4164168110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f4163fff700 1 -- 172.21.15.33:0/2901298460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41640a71e0 con 0x7f414c0038e0 2023-11-06T21:00:24.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f4163fff700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 0x7f4164163e50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f415400e4b0 tx=0x7f415401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:24.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f414bfff700 1 -- 172.21.15.33:0/2901298460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154000cb0 con 0x7f414c0038e0 2023-11-06T21:00:24.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f414bfff700 1 -- 172.21.15.33:0/2901298460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415400e7b0 con 0x7f414c0038e0 2023-11-06T21:00:24.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f414bfff700 1 -- 172.21.15.33:0/2901298460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154026690 con 0x7f414c0038e0 2023-11-06T21:00:24.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f416a5ff700 1 -- 172.21.15.33:0/2901298460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416413c940 con 0x7f414c0038e0 2023-11-06T21:00:24.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.093+0000 7f416a5ff700 1 -- 172.21.15.33:0/2901298460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4164164340 con 0x7f414c0038e0 2023-11-06T21:00:24.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.094+0000 7f416a5ff700 1 -- 172.21.15.33:0/2901298460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416413d270 con 0x7f414c0038e0 2023-11-06T21:00:24.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.095+0000 7f414bfff700 1 -- 172.21.15.33:0/2901298460 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f415402f440 con 0x7f414c0038e0 2023-11-06T21:00:24.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.096+0000 7f414bfff700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f415005f270 0x7f4150061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.096+0000 7f414bfff700 1 -- 172.21.15.33:0/2901298460 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5828+0+0 (secure 0 0 0) 0x7f415407bff0 con 0x7f414c0038e0 2023-11-06T21:00:24.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.097+0000 7f41637fe700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f415005f270 0x7f4150061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.097+0000 7f41637fe700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f415005f270 0x7f4150061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4158001620 tx=0x7f4158009040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.101+0000 7f414bfff700 1 -- 172.21.15.33:0/2901298460 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f41540aaaf0 con 0x7f414c0038e0 2023-11-06T21:00:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.283+0000 7f416a5ff700 1 -- 172.21.15.33:0/2901298460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f41640635b0 con 0x7f414c0038e0 2023-11-06T21:00:24.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.287+0000 7f414bfff700 1 -- 172.21.15.33:0/2901298460 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4154016020 con 0x7f414c0038e0 2023-11-06T21:00:24.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 -- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f415005f270 msgr2=0x7f4150061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:24.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f415005f270 0x7f4150061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4158001620 tx=0x7f4158009040).stop 2023-11-06T21:00:24.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 -- 172.21.15.33:0/2901298460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 msgr2=0x7f4164163e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:24.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 0x7f4164163e50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f415400e4b0 tx=0x7f415401a000).stop 2023-11-06T21:00:24.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 -- 172.21.15.33:0/2901298460 shutdown_connections 2023-11-06T21:00:24.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f415005f270 0x7f4150061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f416413d420 0x7f4164168110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414c0038e0 0x7f4164163e50 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 --2- 172.21.15.33:0/2901298460 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41640a9af0 0x7f4164167c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 -- 172.21.15.33:0/2901298460 >> 172.21.15.33:0/2901298460 conn(0x7f41640c3d30 msgr2=0x7f4164064420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:24.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 -- 172.21.15.33:0/2901298460 shutdown_connections 2023-11-06T21:00:24.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.289+0000 7f416a5ff700 1 -- 172.21.15.33:0/2901298460 wait complete. 2023-11-06T21:00:24.307 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-11-06T21:00:24.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.503+0000 7faf16fff700 1 Processor -- start 2023-11-06T21:00:24.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.503+0000 7faf16fff700 1 -- start start 2023-11-06T21:00:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.504+0000 7faf16fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf100c5d50 0x7faf100c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.504+0000 7faf16fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf100bfd70 0x7faf100c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.504+0000 7faf16fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf101554a0 con 0x7faf100c1840 2023-11-06T21:00:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.504+0000 7faf16fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf10154960 con 0x7faf100c5d50 2023-11-06T21:00:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.504+0000 7faf16fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf100c66c0 con 0x7faf100bfd70 2023-11-06T21:00:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.504+0000 7faf0ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf100c5d50 0x7faf100c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf100c5d50 0x7faf100c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43336/0 (socket says 172.21.15.33:43336) 2023-11-06T21:00:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0ffff700 1 -- 172.21.15.33:0/1049442611 learned_addr learned my addr 172.21.15.33:0/1049442611 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf1559c700 1 --2- 172.21.15.33:0/1049442611 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf100bfd70 0x7faf100c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0ffff700 1 -- 172.21.15.33:0/1049442611 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf100bfd70 msgr2=0x7faf100c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0ffff700 1 --2- 172.21.15.33:0/1049442611 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf100bfd70 0x7faf100c01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0ffff700 1 -- 172.21.15.33:0/1049442611 >> v1:172.21.15.69:6789/0 conn(0x7faf100c1840 legacy=0x7faf100a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0ffff700 1 -- 172.21.15.33:0/1049442611 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf10155930 con 0x7faf100c5d50 2023-11-06T21:00:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0f7fe700 1 -- 172.21.15.33:0/1049442611 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903688591 0 0) 0x7faf101554a0 con 0x7faf100c1840 2023-11-06T21:00:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf1559c700 1 --2- 172.21.15.33:0/1049442611 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf100bfd70 0x7faf100c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0ffff700 1 --2- 172.21.15.33:0/1049442611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 0x7faf100c61d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7faefc006fd0 tx=0x7faefc0090f0).ready entity=mon.0 client_cookie=9d1e1704507c6182 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0f7fe700 1 -- 172.21.15.33:0/1049442611 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00b600 con 0x7faf100c5d50 2023-11-06T21:00:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0f7fe700 1 -- 172.21.15.33:0/1049442611 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc00b760 con 0x7faf100c5d50 2023-11-06T21:00:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.505+0000 7faf0f7fe700 1 -- 172.21.15.33:0/1049442611 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00bae0 con 0x7faf100c5d50 2023-11-06T21:00:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.506+0000 7faf16fff700 1 -- 172.21.15.33:0/1049442611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 msgr2=0x7faf100c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.506+0000 7faf16fff700 1 --2- 172.21.15.33:0/1049442611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 0x7faf100c61d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7faefc006fd0 tx=0x7faefc0090f0).stop 2023-11-06T21:00:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.506+0000 7faf16fff700 1 -- 172.21.15.33:0/1049442611 shutdown_connections 2023-11-06T21:00:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.506+0000 7faf16fff700 1 --2- 172.21.15.33:0/1049442611 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf100bfd70 0x7faf100c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.506+0000 7faf16fff700 1 --2- 172.21.15.33:0/1049442611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 0x7faf100c61d0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.506+0000 7faf16fff700 1 -- 172.21.15.33:0/1049442611 >> 172.21.15.33:0/1049442611 conn(0x7faf1005ec80 msgr2=0x7faf100cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.506+0000 7faf16fff700 1 -- 172.21.15.33:0/1049442611 shutdown_connections 2023-11-06T21:00:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.506+0000 7faf16fff700 1 -- 172.21.15.33:0/1049442611 wait complete. 2023-11-06T21:00:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.508+0000 7faf16fff700 1 Processor -- start 2023-11-06T21:00:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.508+0000 7faf16fff700 1 -- start start 2023-11-06T21:00:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf16fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf100bfd70 0x7faf10150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf16fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf100c1840 0x7faf100ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf16fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 0x7faf100aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf16fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf100afaf0 con 0x7faf100c5d50 2023-11-06T21:00:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf16fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf10151200 con 0x7faf100c1840 2023-11-06T21:00:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf16fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf100af250 con 0x7faf100bfd70 2023-11-06T21:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf1559c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 0x7faf100aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf1559c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 0x7faf100aec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43344/0 (socket says 172.21.15.33:43344) 2023-11-06T21:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf1559c700 1 -- 172.21.15.33:0/975487153 learned_addr learned my addr 172.21.15.33:0/975487153 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf0ffff700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf100c1840 0x7faf100ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf1559c700 1 -- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf100bfd70 msgr2=0x7faf10150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf1559c700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf100bfd70 0x7faf10150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf1559c700 1 -- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf100c1840 msgr2=0x7faf100ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf1559c700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf100c1840 0x7faf100ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf1559c700 1 -- 172.21.15.33:0/975487153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc009370 con 0x7faf100c5d50 2023-11-06T21:00:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.509+0000 7faf0ffff700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf100c1840 0x7faf100ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.510+0000 7faf1559c700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 0x7faf100aec80 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7faf0800d180 tx=0x7faf08009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.510+0000 7faf0cff9700 1 -- 172.21.15.33:0/975487153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800de20 con 0x7faf100c5d50 2023-11-06T21:00:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.510+0000 7faf0cff9700 1 -- 172.21.15.33:0/975487153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0800f040 con 0x7faf100c5d50 2023-11-06T21:00:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.510+0000 7faf0cff9700 1 -- 172.21.15.33:0/975487153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08013600 con 0x7faf100c5d50 2023-11-06T21:00:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.510+0000 7faf16fff700 1 -- 172.21.15.33:0/975487153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf100af530 con 0x7faf100c5d50 2023-11-06T21:00:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.510+0000 7faf16fff700 1 -- 172.21.15.33:0/975487153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf100c2100 con 0x7faf100c5d50 2023-11-06T21:00:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.511+0000 7faf16fff700 1 -- 172.21.15.33:0/975487153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf100c1f60 con 0x7faf100c5d50 2023-11-06T21:00:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.512+0000 7faf0cff9700 1 -- 172.21.15.33:0/975487153 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7faf080139f0 con 0x7faf100c5d50 2023-11-06T21:00:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.513+0000 7faf0cff9700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faef405f270 0x7faef4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.513+0000 7faf0cff9700 1 -- 172.21.15.33:0/975487153 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5828+0+0 (secure 0 0 0) 0x7faf0807ab10 con 0x7faf100c5d50 2023-11-06T21:00:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.513+0000 7faf14d9b700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faef405f270 0x7faef4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.514+0000 7faf14d9b700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faef405f270 0x7faef4061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7faf04001550 tx=0x7faf04007810).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.517+0000 7faf0cff9700 1 -- 172.21.15.33:0/975487153 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf100c1f60 con 0x7faf100c5d50 2023-11-06T21:00:24.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:24.763+0000 7faf16fff700 1 -- 172.21.15.33:0/975487153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7faf100c1f60 con 0x7faf100c5d50 2023-11-06T21:00:25.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.439+0000 7faf0cff9700 1 -- 172.21.15.33:0/975487153 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v601) v1 ==== 166+0+0 (secure 0 0 0) 0x7faf100c1f60 con 0x7faf100c5d50 2023-11-06T21:00:25.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.441+0000 7faf16fff700 1 -- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faef405f270 msgr2=0x7faef4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:25.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.441+0000 7faf16fff700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faef405f270 0x7faef4061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7faf04001550 tx=0x7faf04007810).stop 2023-11-06T21:00:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.441+0000 7faf16fff700 1 -- 172.21.15.33:0/975487153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 msgr2=0x7faf100aec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.441+0000 7faf16fff700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 0x7faf100aec80 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7faf0800d180 tx=0x7faf08009a50).stop 2023-11-06T21:00:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.441+0000 7faf16fff700 1 -- 172.21.15.33:0/975487153 shutdown_connections 2023-11-06T21:00:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.441+0000 7faf16fff700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faef405f270 0x7faef4061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.441+0000 7faf16fff700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf100bfd70 0x7faf10150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:25.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.441+0000 7faf16fff700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf100c5d50 0x7faf100aec80 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:25.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.441+0000 7faf16fff700 1 --2- 172.21.15.33:0/975487153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf100c1840 0x7faf100ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:25.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.441+0000 7faf16fff700 1 -- 172.21.15.33:0/975487153 >> 172.21.15.33:0/975487153 conn(0x7faf1005ec80 msgr2=0x7faf100c92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:25.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.442+0000 7faf16fff700 1 -- 172.21.15.33:0/975487153 shutdown_connections 2023-11-06T21:00:25.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.442+0000 7faf16fff700 1 -- 172.21.15.33:0/975487153 wait complete. 2023-11-06T21:00:25.461 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-11-06T21:00:25.462 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-11-06T21:00:25.462 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-06T21:00:25.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-11-06T21:00:25.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.661+0000 7fa2dafff700 1 Processor -- start 2023-11-06T21:00:25.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.662+0000 7fa2dafff700 1 -- start start 2023-11-06T21:00:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2dafff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa2d40c1840 0x7fa2d40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2dafff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa2d40aa2d0 0x7fa2d40c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2dafff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c6240 0x7fa2d4157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2dafff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d4154960 con 0x7fa2d40aa2d0 2023-11-06T21:00:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2dafff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d41554a0 con 0x7fa2d40c1840 2023-11-06T21:00:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2dafff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4155930 con 0x7fa2d40c6240 2023-11-06T21:00:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2d3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa2d40aa2d0 0x7fa2d40c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2d3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa2d40aa2d0 0x7fa2d40c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43346/0 (socket says 172.21.15.33:43346) 2023-11-06T21:00:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2d3fff700 1 -- 172.21.15.33:0/4189723791 learned_addr learned my addr 172.21.15.33:0/4189723791 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2d959c700 1 --2- 172.21.15.33:0/4189723791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c6240 0x7fa2d4157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.663+0000 7fa2d8d9b700 1 --2- 172.21.15.33:0/4189723791 >> v2:172.21.15.69:3300/0 conn(0x7fa2d40c1840 0x7fa2d40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.664+0000 7fa2d3fff700 1 -- 172.21.15.33:0/4189723791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c6240 msgr2=0x7fa2d4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.664+0000 7fa2d3fff700 1 --2- 172.21.15.33:0/4189723791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c6240 0x7fa2d4157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.664+0000 7fa2d3fff700 1 -- 172.21.15.33:0/4189723791 >> v2:172.21.15.69:3300/0 conn(0x7fa2d40c1840 msgr2=0x7fa2d40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.664+0000 7fa2d3fff700 1 --2- 172.21.15.33:0/4189723791 >> v2:172.21.15.69:3300/0 conn(0x7fa2d40c1840 0x7fa2d40a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.664+0000 7fa2d3fff700 1 -- 172.21.15.33:0/4189723791 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d40c6690 con 0x7fa2d40aa2d0 2023-11-06T21:00:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.664+0000 7fa2d8d9b700 1 --2- 172.21.15.33:0/4189723791 >> v2:172.21.15.69:3300/0 conn(0x7fa2d40c1840 0x7fa2d40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.664+0000 7fa2d3fff700 1 --2- 172.21.15.33:0/4189723791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40aa2d0 0x7fa2d40c5d50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fa2d40840b0 tx=0x7fa2c00090f0).ready entity=mon.0 client_cookie=6571e03698a8638e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.664+0000 7fa2d37fe700 1 -- 172.21.15.33:0/4189723791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000b520 con 0x7fa2d40aa2d0 2023-11-06T21:00:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.664+0000 7fa2d37fe700 1 -- 172.21.15.33:0/4189723791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c000b680 con 0x7fa2d40aa2d0 2023-11-06T21:00:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.664+0000 7fa2d37fe700 1 -- 172.21.15.33:0/4189723791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000ba00 con 0x7fa2d40aa2d0 2023-11-06T21:00:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.665+0000 7fa2dafff700 1 -- 172.21.15.33:0/4189723791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40aa2d0 msgr2=0x7fa2d40c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.665+0000 7fa2dafff700 1 --2- 172.21.15.33:0/4189723791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40aa2d0 0x7fa2d40c5d50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fa2d40840b0 tx=0x7fa2c00090f0).stop 2023-11-06T21:00:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.665+0000 7fa2dafff700 1 -- 172.21.15.33:0/4189723791 shutdown_connections 2023-11-06T21:00:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.665+0000 7fa2dafff700 1 --2- 172.21.15.33:0/4189723791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c6240 0x7fa2d4157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.665+0000 7fa2dafff700 1 --2- 172.21.15.33:0/4189723791 >> v2:172.21.15.69:3300/0 conn(0x7fa2d40c1840 0x7fa2d40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.665+0000 7fa2dafff700 1 --2- 172.21.15.33:0/4189723791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40aa2d0 0x7fa2d40c5d50 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.665+0000 7fa2dafff700 1 -- 172.21.15.33:0/4189723791 >> 172.21.15.33:0/4189723791 conn(0x7fa2d405ec80 msgr2=0x7fa2d40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.665+0000 7fa2dafff700 1 -- 172.21.15.33:0/4189723791 shutdown_connections 2023-11-06T21:00:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.665+0000 7fa2dafff700 1 -- 172.21.15.33:0/4189723791 wait complete. 2023-11-06T21:00:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.666+0000 7fa2dafff700 1 Processor -- start 2023-11-06T21:00:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.666+0000 7fa2dafff700 1 -- start start 2023-11-06T21:00:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.667+0000 7fa2dafff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40aa2d0 0x7fa2d4151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.667+0000 7fa2dafff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c1840 0x7fa2d40aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:25.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.667+0000 7fa2dafff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c6240 0x7fa2d4143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:25.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.667+0000 7fa2dafff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40af5c0 con 0x7fa2d40c6240 2023-11-06T21:00:25.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.667+0000 7fa2dafff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40af8a0 con 0x7fa2d40aa2d0 2023-11-06T21:00:25.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.667+0000 7fa2dafff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40afb80 con 0x7fa2d40c1840 2023-11-06T21:00:25.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d959c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c6240 0x7fa2d4143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:25.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d959c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c6240 0x7fa2d4143b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43362/0 (socket says 172.21.15.33:43362) 2023-11-06T21:00:25.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d959c700 1 -- 172.21.15.33:0/2017533374 learned_addr learned my addr 172.21.15.33:0/2017533374 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:25.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d8d9b700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40aa2d0 0x7fa2d4151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:25.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d3fff700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c1840 0x7fa2d40aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:25.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d959c700 1 -- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c1840 msgr2=0x7fa2d40aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d959c700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c1840 0x7fa2d40aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d959c700 1 -- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40aa2d0 msgr2=0x7fa2d4151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d959c700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40aa2d0 0x7fa2d4151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d959c700 1 -- 172.21.15.33:0/2017533374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c0009370 con 0x7fa2d40c6240 2023-11-06T21:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.668+0000 7fa2d959c700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c6240 0x7fa2d4143b10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7fa2cc009e90 tx=0x7fa2cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.669+0000 7fa2d0ff9700 1 -- 172.21.15.33:0/2017533374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00cc20 con 0x7fa2d40c6240 2023-11-06T21:00:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.669+0000 7fa2d0ff9700 1 -- 172.21.15.33:0/2017533374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc00f040 con 0x7fa2d40c6240 2023-11-06T21:00:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.669+0000 7fa2dafff700 1 -- 172.21.15.33:0/2017533374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d40afe60 con 0x7fa2d40c6240 2023-11-06T21:00:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.669+0000 7fa2d0ff9700 1 -- 172.21.15.33:0/2017533374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc013400 con 0x7fa2d40c6240 2023-11-06T21:00:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.669+0000 7fa2dafff700 1 -- 172.21.15.33:0/2017533374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d40af280 con 0x7fa2d40c6240 2023-11-06T21:00:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.670+0000 7fa2dafff700 1 -- 172.21.15.33:0/2017533374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d40c58b0 con 0x7fa2d40c6240 2023-11-06T21:00:25.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.671+0000 7fa2d0ff9700 1 -- 172.21.15.33:0/2017533374 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa2cc0137b0 con 0x7fa2d40c6240 2023-11-06T21:00:25.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.672+0000 7fa2d0ff9700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2b805f270 0x7fa2b8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:25.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.672+0000 7fa2d0ff9700 1 -- 172.21.15.33:0/2017533374 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5828+0+0 (secure 0 0 0) 0x7fa2cc07abf0 con 0x7fa2d40c6240 2023-11-06T21:00:25.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.672+0000 7fa2d8d9b700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2b805f270 0x7fa2b8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:25.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.673+0000 7fa2d8d9b700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2b805f270 0x7fa2b8061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa2d4156bd0 tx=0x7fa2c8008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:25.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.675+0000 7fa2d0ff9700 1 -- 172.21.15.33:0/2017533374 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa2cc04bda0 con 0x7fa2d40c6240 2023-11-06T21:00:25.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:25.925+0000 7fa2dafff700 1 -- 172.21.15.33:0/2017533374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fa2d406c6c0 con 0x7fa2d40c6240 2023-11-06T21:00:26.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.442+0000 7fa2d0ff9700 1 -- 172.21.15.33:0/2017533374 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v602) v1 ==== 186+0+0 (secure 0 0 0) 0x7fa2cc04f3c0 con 0x7fa2d40c6240 2023-11-06T21:00:26.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.444+0000 7fa2dafff700 1 -- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2b805f270 msgr2=0x7fa2b8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:26.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.444+0000 7fa2dafff700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2b805f270 0x7fa2b8061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa2d4156bd0 tx=0x7fa2c8008040).stop 2023-11-06T21:00:26.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.444+0000 7fa2dafff700 1 -- 172.21.15.33:0/2017533374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c6240 msgr2=0x7fa2d4143b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.444+0000 7fa2dafff700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c6240 0x7fa2d4143b10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7fa2cc009e90 tx=0x7fa2cc00b040).stop 2023-11-06T21:00:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.445+0000 7fa2dafff700 1 -- 172.21.15.33:0/2017533374 shutdown_connections 2023-11-06T21:00:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.445+0000 7fa2dafff700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa2b805f270 0x7fa2b8061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.445+0000 7fa2dafff700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d40c6240 0x7fa2d4143b10 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.445+0000 7fa2dafff700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d40aa2d0 0x7fa2d4151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.445+0000 7fa2dafff700 1 --2- 172.21.15.33:0/2017533374 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2d40c1840 0x7fa2d40aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.445+0000 7fa2dafff700 1 -- 172.21.15.33:0/2017533374 >> 172.21.15.33:0/2017533374 conn(0x7fa2d405ec80 msgr2=0x7fa2d40ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.445+0000 7fa2dafff700 1 -- 172.21.15.33:0/2017533374 shutdown_connections 2023-11-06T21:00:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.445+0000 7fa2dafff700 1 -- 172.21.15.33:0/2017533374 wait complete. 2023-11-06T21:00:26.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T21:00:26.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.668+0000 7efd8e9ff700 1 Processor -- start 2023-11-06T21:00:26.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.668+0000 7efd8e9ff700 1 -- start start 2023-11-06T21:00:26.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.670+0000 7efd8e9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd881544f0 0x7efd880c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:26.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.670+0000 7efd8e9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd881556d0 con 0x7efd88064dd0 2023-11-06T21:00:26.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.670+0000 7efd8e9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd88154ac0 con 0x7efd880c0c40 2023-11-06T21:00:26.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.670+0000 7efd8e9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd881548d0 con 0x7efd881544f0 2023-11-06T21:00:26.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.670+0000 7efd87fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd881544f0 0x7efd880c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:26.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.670+0000 7efd8cf9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efd88064dd0 0x7efd880c57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38556/0 (socket says 172.21.15.33:38556) 2023-11-06T21:00:26.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.670+0000 7efd8cf9c700 1 -- 172.21.15.33:0/1117413135 learned_addr learned my addr 172.21.15.33:0/1117413135 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:26.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.670+0000 7efd87fff700 1 -- 172.21.15.33:0/1117413135 >> v1:172.21.15.69:6789/0 conn(0x7efd880c0c40 legacy=0x7efd880646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:26.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.670+0000 7efd87fff700 1 -- 172.21.15.33:0/1117413135 >> v1:172.21.15.33:6789/0 conn(0x7efd88064dd0 legacy=0x7efd880c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:26.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.671+0000 7efd87fff700 1 -- 172.21.15.33:0/1117413135 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd88154da0 con 0x7efd881544f0 2023-11-06T21:00:26.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.671+0000 7efd86ffd700 1 -- 172.21.15.33:0/1117413135 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283383475 0 0) 0x7efd881556d0 con 0x7efd88064dd0 2023-11-06T21:00:26.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.671+0000 7efd87fff700 1 --2- 172.21.15.33:0/1117413135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd881544f0 0x7efd880c0750 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7efd78009760 tx=0x7efd78009060).ready entity=mon.0 client_cookie=24cc77f0c47b6610 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:26.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.671+0000 7efd86ffd700 1 -- 172.21.15.33:0/1117413135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7800b580 con 0x7efd881544f0 2023-11-06T21:00:26.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.671+0000 7efd86ffd700 1 -- 172.21.15.33:0/1117413135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7800b6e0 con 0x7efd881544f0 2023-11-06T21:00:26.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.671+0000 7efd86ffd700 1 -- 172.21.15.33:0/1117413135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7800ba60 con 0x7efd881544f0 2023-11-06T21:00:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.671+0000 7efd8e9ff700 1 -- 172.21.15.33:0/1117413135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd881544f0 msgr2=0x7efd880c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.671+0000 7efd8e9ff700 1 --2- 172.21.15.33:0/1117413135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd881544f0 0x7efd880c0750 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7efd78009760 tx=0x7efd78009060).stop 2023-11-06T21:00:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.672+0000 7efd8e9ff700 1 -- 172.21.15.33:0/1117413135 shutdown_connections 2023-11-06T21:00:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.672+0000 7efd8e9ff700 1 --2- 172.21.15.33:0/1117413135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd881544f0 0x7efd880c0750 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.672+0000 7efd8e9ff700 1 -- 172.21.15.33:0/1117413135 >> 172.21.15.33:0/1117413135 conn(0x7efd880cb900 msgr2=0x7efd880c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:26.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.672+0000 7efd8e9ff700 1 -- 172.21.15.33:0/1117413135 shutdown_connections 2023-11-06T21:00:26.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.672+0000 7efd8e9ff700 1 -- 172.21.15.33:0/1117413135 wait complete. 2023-11-06T21:00:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.673+0000 7efd8e9ff700 1 Processor -- start 2023-11-06T21:00:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.673+0000 7efd8e9ff700 1 -- start start 2023-11-06T21:00:26.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.674+0000 7efd8e9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88064dd0 0x7efd88152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:26.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.674+0000 7efd8e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd880c0c40 0x7efd880a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:26.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.674+0000 7efd8e9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd881544f0 0x7efd881561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:26.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.674+0000 7efd8e9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd880c2640 con 0x7efd88064dd0 2023-11-06T21:00:26.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.674+0000 7efd8e9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd880b0e00 con 0x7efd881544f0 2023-11-06T21:00:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.674+0000 7efd8e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880b0b20 con 0x7efd880c0c40 2023-11-06T21:00:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.674+0000 7efd87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88064dd0 0x7efd88152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.674+0000 7efd87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88064dd0 0x7efd88152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43384/0 (socket says 172.21.15.33:43384) 2023-11-06T21:00:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.674+0000 7efd87fff700 1 -- 172.21.15.33:0/2830732736 learned_addr learned my addr 172.21.15.33:0/2830732736 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd8cf9c700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd881544f0 0x7efd881561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd877fe700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd880c0c40 0x7efd880a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd87fff700 1 -- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd880c0c40 msgr2=0x7efd880a93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd87fff700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd880c0c40 0x7efd880a93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd87fff700 1 -- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd881544f0 msgr2=0x7efd881561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd87fff700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd881544f0 0x7efd881561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd87fff700 1 -- 172.21.15.33:0/2830732736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd780092a0 con 0x7efd88064dd0 2023-11-06T21:00:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd87fff700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88064dd0 0x7efd88152450 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7efd7800b340 tx=0x7efd78017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd6ffff700 1 -- 172.21.15.33:0/2830732736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7800ba60 con 0x7efd88064dd0 2023-11-06T21:00:26.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd6ffff700 1 -- 172.21.15.33:0/2830732736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd78000c90 con 0x7efd88064dd0 2023-11-06T21:00:26.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd6ffff700 1 -- 172.21.15.33:0/2830732736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7801b670 con 0x7efd88064dd0 2023-11-06T21:00:26.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.675+0000 7efd8e9ff700 1 -- 172.21.15.33:0/2830732736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd880c1ff0 con 0x7efd88064dd0 2023-11-06T21:00:26.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.676+0000 7efd8e9ff700 1 -- 172.21.15.33:0/2830732736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd88152940 con 0x7efd88064dd0 2023-11-06T21:00:26.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.677+0000 7efd6dffb700 1 -- 172.21.15.33:0/2830732736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd60005120 con 0x7efd88064dd0 2023-11-06T21:00:26.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.677+0000 7efd6ffff700 1 -- 172.21.15.33:0/2830732736 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7efd7801ba60 con 0x7efd88064dd0 2023-11-06T21:00:26.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.678+0000 7efd6ffff700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd7405f240 0x7efd74061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:26.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.678+0000 7efd6ffff700 1 -- 172.21.15.33:0/2830732736 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5828+0+0 (secure 0 0 0) 0x7efd7807bb70 con 0x7efd88064dd0 2023-11-06T21:00:26.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.679+0000 7efd877fe700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd7405f240 0x7efd74061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:26.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.679+0000 7efd877fe700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd7405f240 0x7efd74061700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7efd880cacc0 tx=0x7efd7c009040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:26.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.682+0000 7efd6ffff700 1 -- 172.21.15.33:0/2830732736 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efd7804cd40 con 0x7efd88064dd0 2023-11-06T21:00:26.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.864+0000 7efd6dffb700 1 -- 172.21.15.33:0/2830732736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efd60005ac0 con 0x7efd88064dd0 2023-11-06T21:00:26.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.870+0000 7efd6ffff700 1 -- 172.21.15.33:0/2830732736 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7efd78028020 con 0x7efd88064dd0 2023-11-06T21:00:26.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.872+0000 7efd6dffb700 1 -- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd7405f240 msgr2=0x7efd74061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:26.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.872+0000 7efd6dffb700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd7405f240 0x7efd74061700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7efd880cacc0 tx=0x7efd7c009040).stop 2023-11-06T21:00:26.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.872+0000 7efd6dffb700 1 -- 172.21.15.33:0/2830732736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88064dd0 msgr2=0x7efd88152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:26.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.872+0000 7efd6dffb700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88064dd0 0x7efd88152450 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7efd7800b340 tx=0x7efd78017000).stop 2023-11-06T21:00:26.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.873+0000 7efd6dffb700 1 -- 172.21.15.33:0/2830732736 shutdown_connections 2023-11-06T21:00:26.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.873+0000 7efd6dffb700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efd7405f240 0x7efd74061700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.873+0000 7efd6dffb700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd881544f0 0x7efd881561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.873+0000 7efd6dffb700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd88064dd0 0x7efd88152450 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.873+0000 7efd6dffb700 1 --2- 172.21.15.33:0/2830732736 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd880c0c40 0x7efd880a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:26.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.873+0000 7efd6dffb700 1 -- 172.21.15.33:0/2830732736 >> 172.21.15.33:0/2830732736 conn(0x7efd880cb900 msgr2=0x7efd88157040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:26.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.873+0000 7efd6dffb700 1 -- 172.21.15.33:0/2830732736 shutdown_connections 2023-11-06T21:00:26.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:26.873+0000 7efd6dffb700 1 -- 172.21.15.33:0/2830732736 wait complete. 2023-11-06T21:00:26.891 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T21:00:26.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:00:26.983 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:00:27.085 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T21:00:27.086 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T21:00:27.086 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:00:27.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.282+0000 7f342c3ff700 1 Processor -- start 2023-11-06T21:00:27.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.282+0000 7f342c3ff700 1 -- start start 2023-11-06T21:00:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.284+0000 7f342c3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f34240a6d40 0x7f3424154240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.284+0000 7f342c3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34240c2ec0 con 0x7f3424062710 2023-11-06T21:00:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.284+0000 7f342c3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34240a73d0 con 0x7f3424153e60 2023-11-06T21:00:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.284+0000 7f342c3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3424154730 con 0x7f34240a6d40 2023-11-06T21:00:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.284+0000 7f342a99c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3424062710 0x7f34240c3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38568/0 (socket says 172.21.15.33:38568) 2023-11-06T21:00:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.284+0000 7f342a99c700 1 -- 172.21.15.33:0/3679973869 learned_addr learned my addr 172.21.15.33:0/3679973869 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:27.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.284+0000 7f342999a700 1 --2- 172.21.15.33:0/3679973869 >> v2:172.21.15.69:3300/0 conn(0x7f34240a6d40 0x7f3424154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.284+0000 7f3429199700 1 -- 172.21.15.33:0/3679973869 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815773825 0 0) 0x7f34240c2ec0 con 0x7f3424062710 2023-11-06T21:00:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f3429199700 1 -- 172.21.15.33:0/3679973869 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34100033f0 con 0x7f3424062710 2023-11-06T21:00:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f3429199700 1 -- 172.21.15.33:0/3679973869 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996581907 0 0) 0x7f34240a73d0 con 0x7f3424153e60 2023-11-06T21:00:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f3429199700 1 -- 172.21.15.33:0/3679973869 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34240c2ec0 con 0x7f3424153e60 2023-11-06T21:00:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f3429199700 1 -- 172.21.15.33:0/3679973869 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2712839098 0 0) 0x7f34100033f0 con 0x7f3424062710 2023-11-06T21:00:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f342999a700 1 -- 172.21.15.33:0/3679973869 >> v1:172.21.15.69:6789/0 conn(0x7f3424153e60 legacy=0x7f3424069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f342999a700 1 -- 172.21.15.33:0/3679973869 >> v1:172.21.15.33:6789/0 conn(0x7f3424062710 legacy=0x7f34240c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f342999a700 1 -- 172.21.15.33:0/3679973869 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34240c2590 con 0x7f34240a6d40 2023-11-06T21:00:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f342999a700 1 --2- 172.21.15.33:0/3679973869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34240a6d40 0x7f3424154240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f3414009760 tx=0x7f3414009060).ready entity=mon.1 client_cookie=f72cc18bd9fe6625 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f3429199700 1 -- 172.21.15.33:0/3679973869 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c0017f0 con 0x7f34240a6d40 2023-11-06T21:00:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f3429199700 1 -- 172.21.15.33:0/3679973869 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34140090d0 con 0x7f34240a6d40 2023-11-06T21:00:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.285+0000 7f3429199700 1 -- 172.21.15.33:0/3679973869 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414011610 con 0x7f34240a6d40 2023-11-06T21:00:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.286+0000 7f342c3ff700 1 -- 172.21.15.33:0/3679973869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34240a6d40 msgr2=0x7f3424154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.286+0000 7f342c3ff700 1 --2- 172.21.15.33:0/3679973869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34240a6d40 0x7f3424154240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f3414009760 tx=0x7f3414009060).stop 2023-11-06T21:00:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.286+0000 7f342c3ff700 1 -- 172.21.15.33:0/3679973869 shutdown_connections 2023-11-06T21:00:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.286+0000 7f342c3ff700 1 --2- 172.21.15.33:0/3679973869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34240a6d40 0x7f3424154240 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.286+0000 7f342c3ff700 1 -- 172.21.15.33:0/3679973869 >> 172.21.15.33:0/3679973869 conn(0x7f34240c7b50 msgr2=0x7f34240c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.286+0000 7f342c3ff700 1 -- 172.21.15.33:0/3679973869 shutdown_connections 2023-11-06T21:00:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.286+0000 7f342c3ff700 1 -- 172.21.15.33:0/3679973869 wait complete. 2023-11-06T21:00:27.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.288+0000 7f342c3ff700 1 Processor -- start 2023-11-06T21:00:27.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.288+0000 7f342c3ff700 1 -- start start 2023-11-06T21:00:27.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342c3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3424062710 0x7f34240ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342c3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34240a6d40 0x7f34241532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342c3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3424153e60 0x7f34241537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342c3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3424153c90 con 0x7f3424062710 2023-11-06T21:00:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342c3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3424165770 con 0x7f3424153e60 2023-11-06T21:00:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342c3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240ad1f0 con 0x7f34240a6d40 2023-11-06T21:00:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342a19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3424062710 0x7f34240ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:27.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342a19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3424062710 0x7f34240ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43394/0 (socket says 172.21.15.33:43394) 2023-11-06T21:00:27.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342a19b700 1 -- 172.21.15.33:0/2591435641 learned_addr learned my addr 172.21.15.33:0/2591435641 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:27.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342a19b700 1 -- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34240a6d40 msgr2=0x7f34241532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342a19b700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34240a6d40 0x7f34241532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:27.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342a99c700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3424153e60 0x7f34241537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:27.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342a19b700 1 -- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3424153e60 msgr2=0x7f34241537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342a19b700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3424153e60 0x7f34241537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:27.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342a19b700 1 -- 172.21.15.33:0/2591435641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34140092a0 con 0x7f3424062710 2023-11-06T21:00:27.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.290+0000 7f342a99c700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3424153e60 0x7f34241537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:27.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.291+0000 7f342a19b700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3424062710 0x7f34240ac8c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f342000c440 tx=0x7f3420007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:27.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.291+0000 7f341a7fc700 1 -- 172.21.15.33:0/2591435641 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000f470 con 0x7f3424062710 2023-11-06T21:00:27.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.291+0000 7f341a7fc700 1 -- 172.21.15.33:0/2591435641 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342000f5d0 con 0x7f3424062710 2023-11-06T21:00:27.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.291+0000 7f342c3ff700 1 -- 172.21.15.33:0/2591435641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34240ad4d0 con 0x7f3424062710 2023-11-06T21:00:27.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.291+0000 7f341a7fc700 1 -- 172.21.15.33:0/2591435641 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000f990 con 0x7f3424062710 2023-11-06T21:00:27.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.291+0000 7f342c3ff700 1 -- 172.21.15.33:0/2591435641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34240ada90 con 0x7f3424062710 2023-11-06T21:00:27.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.292+0000 7f342c3ff700 1 -- 172.21.15.33:0/2591435641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34240a60e0 con 0x7f3424062710 2023-11-06T21:00:27.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.293+0000 7f341a7fc700 1 -- 172.21.15.33:0/2591435641 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f342000fd40 con 0x7f3424062710 2023-11-06T21:00:27.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.294+0000 7f341a7fc700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f340805f270 0x7f3408061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.294+0000 7f341a7fc700 1 -- 172.21.15.33:0/2591435641 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5828+0+0 (secure 0 0 0) 0x7f3420079970 con 0x7f3424062710 2023-11-06T21:00:27.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.294+0000 7f342999a700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f340805f270 0x7f3408061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:27.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.295+0000 7f342999a700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f340805f270 0x7f3408061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f34240c1d70 tx=0x7f3414016000).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:27.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.298+0000 7f341a7fc700 1 -- 172.21.15.33:0/2591435641 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f342004ab20 con 0x7f3424062710 2023-11-06T21:00:27.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.526+0000 7f342c3ff700 1 -- 172.21.15.33:0/2591435641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f342406a3b0 con 0x7f3424062710 2023-11-06T21:00:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.527+0000 7f341a7fc700 1 -- 172.21.15.33:0/2591435641 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f342406a3b0 con 0x7f3424062710 2023-11-06T21:00:27.528 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:27.528 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1797026087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1797026087},{"type":"v1","addr":"172.21.15.69:6836","nonce":1797026087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4199570083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4199570083},{"type":"v1","addr":"172.21.15.33:6834","nonce":4199570083}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2429309034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2429309034},{"type":"v1","addr":"172.21.15.33:6835","nonce":2429309034}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/13898668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":13898668},{"type":"v1","addr":"172.21.15.69:6837","nonce":13898668}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[]} 2023-11-06T21:00:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 -- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f340805f270 msgr2=0x7f3408061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f340805f270 0x7f3408061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f34240c1d70 tx=0x7f3414016000).stop 2023-11-06T21:00:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 -- 172.21.15.33:0/2591435641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3424062710 msgr2=0x7f34240ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3424062710 0x7f34240ac8c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f342000c440 tx=0x7f3420007c90).stop 2023-11-06T21:00:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 -- 172.21.15.33:0/2591435641 shutdown_connections 2023-11-06T21:00:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f340805f270 0x7f3408061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3424153e60 0x7f34241537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3424062710 0x7f34240ac8c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 --2- 172.21.15.33:0/2591435641 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34240a6d40 0x7f34241532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 -- 172.21.15.33:0/2591435641 >> 172.21.15.33:0/2591435641 conn(0x7f34240c7b50 msgr2=0x7f34240670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 -- 172.21.15.33:0/2591435641 shutdown_connections 2023-11-06T21:00:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.529+0000 7f342c3ff700 1 -- 172.21.15.33:0/2591435641 wait complete. 2023-11-06T21:00:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 39 2023-11-06T21:00:27.547 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T21:00:27.547 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T21:00:27.548 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T21:00:27.548 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:00:27.548 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T21:00:27.548 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T21:00:27.549 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T21:00:27.549 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:00:27.549 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T21:00:27.550 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T21:00:27.550 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T21:00:27.551 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:00:27.551 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T21:00:27.551 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T21:00:27.552 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T21:00:27.552 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:00:27.562 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T21:00:27.562 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T21:00:27.563 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T21:00:27.564 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T21:00:27.567 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T21:00:27.567 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T21:00:27.568 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T21:00:27.569 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T21:00:27.607 INFO:tasks.ceph.mds.a:Started 2023-11-06T21:00:27.607 DEBUG:teuthology.parallel:result is None 2023-11-06T21:00:27.610 INFO:tasks.ceph.mds.c:Started 2023-11-06T21:00:27.610 DEBUG:teuthology.parallel:result is None 2023-11-06T21:00:27.613 INFO:tasks.ceph.mds.b:Started 2023-11-06T21:00:27.613 DEBUG:teuthology.parallel:result is None 2023-11-06T21:00:27.615 INFO:tasks.ceph.mds.d:Started 2023-11-06T21:00:27.616 DEBUG:teuthology.parallel:result is None 2023-11-06T21:00:27.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T21:00:27.758 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T21:00:27.759 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T21:00:27.760 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T21:00:27.761 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T21:00:27.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.827+0000 7fedbcf9d700 1 Processor -- start 2023-11-06T21:00:27.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.827+0000 7fedbcf9d700 1 -- start start 2023-11-06T21:00:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.828+0000 7fedbcf9d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fedb81542c0 0x7fedb80644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.828+0000 7fedbcf9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fedb80c6260 0x7fedb8064990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.828+0000 7fedbcf9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb80c0680 con 0x7fedb80aade0 2023-11-06T21:00:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.828+0000 7fedbcf9d700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb8064e80 con 0x7fedb81542c0 2023-11-06T21:00:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.828+0000 7fedbcf9d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80c1b20 con 0x7fedb80c6260 2023-11-06T21:00:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.828+0000 7fedb6ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fedb80aade0 0x7fedb80c5b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38592/0 (socket says 172.21.15.33:38592) 2023-11-06T21:00:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.828+0000 7fedb6ffd700 1 -- 172.21.15.33:0/1373434463 learned_addr learned my addr 172.21.15.33:0/1373434463 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.828+0000 7fedb7fff700 1 --2- 172.21.15.33:0/1373434463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fedb80c6260 0x7fedb8064990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.828+0000 7fedb77fe700 1 --2- 172.21.15.33:0/1373434463 >> v2:172.21.15.69:3300/0 conn(0x7fedb81542c0 0x7fedb80644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:27.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.829+0000 7fedb67fc700 1 -- 172.21.15.33:0/1373434463 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792438448 0 0) 0x7fedb80c0680 con 0x7fedb80aade0 2023-11-06T21:00:27.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.829+0000 7fedb67fc700 1 -- 172.21.15.33:0/1373434463 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feda00033f0 con 0x7fedb80aade0 2023-11-06T21:00:27.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.829+0000 7fedb77fe700 1 -- 172.21.15.33:0/1373434463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fedb80c6260 msgr2=0x7fedb8064990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.829+0000 7fedb77fe700 1 --2- 172.21.15.33:0/1373434463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fedb80c6260 0x7fedb8064990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.829+0000 7fedb77fe700 1 -- 172.21.15.33:0/1373434463 >> v1:172.21.15.33:6789/0 conn(0x7fedb80aade0 legacy=0x7fedb80c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.829+0000 7fedb77fe700 1 -- 172.21.15.33:0/1373434463 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb80bfd90 con 0x7fedb81542c0 2023-11-06T21:00:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.829+0000 7fedb77fe700 1 --2- 172.21.15.33:0/1373434463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb81542c0 0x7fedb80644a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7feda8009760 tx=0x7feda8009060).ready entity=mon.1 client_cookie=e7b0ab3260afd23e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.829+0000 7fedb67fc700 1 -- 172.21.15.33:0/1373434463 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800d4a0 con 0x7fedb81542c0 2023-11-06T21:00:27.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.829+0000 7fedb67fc700 1 -- 172.21.15.33:0/1373434463 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda800d600 con 0x7fedb81542c0 2023-11-06T21:00:27.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.829+0000 7fedb67fc700 1 -- 172.21.15.33:0/1373434463 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800d980 con 0x7fedb81542c0 2023-11-06T21:00:27.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.830+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1373434463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb81542c0 msgr2=0x7fedb80644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.830+0000 7fedbcf9d700 1 --2- 172.21.15.33:0/1373434463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb81542c0 0x7fedb80644a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7feda8009760 tx=0x7feda8009060).stop 2023-11-06T21:00:27.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.830+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1373434463 shutdown_connections 2023-11-06T21:00:27.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.830+0000 7fedbcf9d700 1 --2- 172.21.15.33:0/1373434463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fedb80c6260 0x7fedb8064990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:27.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.830+0000 7fedbcf9d700 1 --2- 172.21.15.33:0/1373434463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb81542c0 0x7fedb80644a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:27.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.830+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1373434463 >> 172.21.15.33:0/1373434463 conn(0x7fedb80cb760 msgr2=0x7fedb80c7730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:27.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.830+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1373434463 shutdown_connections 2023-11-06T21:00:27.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.830+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1373434463 wait complete. 2023-11-06T21:00:27.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.831+0000 7fedbcf9d700 1 Processor -- start 2023-11-06T21:00:27.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.832+0000 7fedbcf9d700 1 -- start start 2023-11-06T21:00:27.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedbcf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb80aade0 0x7fedb8152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedbcf9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fedb80c6260 0x7fedb814da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedbcf9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb81542c0 0x7fedb80ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedbcf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80aff50 con 0x7fedb80aade0 2023-11-06T21:00:27.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedbcf9d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb8152b00 con 0x7fedb81542c0 2023-11-06T21:00:27.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedbcf9d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb814df20 con 0x7fedb80c6260 2023-11-06T21:00:27.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb80aade0 0x7fedb8152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:27.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb80aade0 0x7fedb8152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43428/0 (socket says 172.21.15.33:43428) 2023-11-06T21:00:27.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb77fe700 1 -- 172.21.15.33:0/1488640091 learned_addr learned my addr 172.21.15.33:0/1488640091 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:27.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb6ffd700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fedb80c6260 0x7fedb814da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:27.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb7fff700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb81542c0 0x7fedb80ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb77fe700 1 -- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fedb80c6260 msgr2=0x7fedb814da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb77fe700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fedb80c6260 0x7fedb814da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb77fe700 1 -- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb81542c0 msgr2=0x7fedb80ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb77fe700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb81542c0 0x7fedb80ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb77fe700 1 -- 172.21.15.33:0/1488640091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda80092a0 con 0x7fedb80aade0 2023-11-06T21:00:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb7fff700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb81542c0 0x7fedb80ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.833+0000 7fedb77fe700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb80aade0 0x7fedb8152610 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7fedb80841d0 tx=0x7feda8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.834+0000 7fed9f7fe700 1 -- 172.21.15.33:0/1488640091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8000c90 con 0x7fedb80aade0 2023-11-06T21:00:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.834+0000 7fed9f7fe700 1 -- 172.21.15.33:0/1488640091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda800d770 con 0x7fedb80aade0 2023-11-06T21:00:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.834+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1488640091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb80af3d0 con 0x7fedb80aade0 2023-11-06T21:00:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.834+0000 7fed9f7fe700 1 -- 172.21.15.33:0/1488640091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda801d430 con 0x7fedb80aade0 2023-11-06T21:00:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.834+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1488640091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb80af990 con 0x7fedb80aade0 2023-11-06T21:00:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.835+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1488640091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb806ad80 con 0x7fedb80aade0 2023-11-06T21:00:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.836+0000 7fed9f7fe700 1 -- 172.21.15.33:0/1488640091 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7feda801d7e0 con 0x7fedb80aade0 2023-11-06T21:00:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.837+0000 7fed9f7fe700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feda405f240 0x7feda4061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.837+0000 7fed9f7fe700 1 -- 172.21.15.33:0/1488640091 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5828+0+0 (secure 0 0 0) 0x7feda807bb50 con 0x7fedb80aade0 2023-11-06T21:00:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.837+0000 7fedb6ffd700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feda405f240 0x7feda4061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.837+0000 7fedb6ffd700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feda405f240 0x7feda4061700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fedb8154050 tx=0x7fedac009040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:27.841+0000 7fed9f7fe700 1 -- 172.21.15.33:0/1488640091 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feda804cca0 con 0x7fedb80aade0 2023-11-06T21:00:28.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.038+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1488640091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fedb80475a0 con 0x7fedb80aade0 2023-11-06T21:00:28.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.996+0000 7fed9f7fe700 1 -- 172.21.15.33:0/1488640091 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v110) v1 ==== 97+0+0 (secure 0 0 0) 0x7feda80502c0 con 0x7fedb80aade0 2023-11-06T21:00:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.998+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feda405f240 msgr2=0x7feda4061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.998+0000 7fedbcf9d700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feda405f240 0x7feda4061700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fedb8154050 tx=0x7fedac009040).stop 2023-11-06T21:00:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.998+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1488640091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb80aade0 msgr2=0x7fedb8152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.998+0000 7fedbcf9d700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb80aade0 0x7fedb8152610 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7fedb80841d0 tx=0x7feda8016000).stop 2023-11-06T21:00:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.999+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1488640091 shutdown_connections 2023-11-06T21:00:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.999+0000 7fedbcf9d700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feda405f240 0x7feda4061700 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.999+0000 7fedbcf9d700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb81542c0 0x7fedb80ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.999+0000 7fedbcf9d700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb80aade0 0x7fedb8152610 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.999+0000 7fedbcf9d700 1 --2- 172.21.15.33:0/1488640091 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fedb80c6260 0x7fedb814da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.999+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1488640091 >> 172.21.15.33:0/1488640091 conn(0x7fedb80cb760 msgr2=0x7fedb80ab2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:29.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.999+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1488640091 shutdown_connections 2023-11-06T21:00:29.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:28.999+0000 7fedbcf9d700 1 -- 172.21.15.33:0/1488640091 wait complete. 2023-11-06T21:00:29.002 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T21:00:29.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T21:00:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.220+0000 7f42029ff700 1 Processor -- start 2023-11-06T21:00:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.220+0000 7f42029ff700 1 -- start start 2023-11-06T21:00:29.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.221+0000 7f42029ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f41fc0c1840 0x7f41fc0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.221+0000 7f42029ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc0c6460 0x7f41fc0bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.221+0000 7f42029ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41fc1554a0 con 0x7f41fc0aa2d0 2023-11-06T21:00:29.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.221+0000 7f42029ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41fc154960 con 0x7f41fc0c1840 2023-11-06T21:00:29.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.221+0000 7f42029ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc155930 con 0x7f41fc0c6460 2023-11-06T21:00:29.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41fb7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f41fc0aa2d0 0x7f41fc0c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48280/0 (socket says 172.21.15.33:48280) 2023-11-06T21:00:29.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41fb7fe700 1 -- 172.21.15.33:0/3981345637 learned_addr learned my addr 172.21.15.33:0/3981345637 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41fbfff700 1 --2- 172.21.15.33:0/3981345637 >> v2:172.21.15.69:3300/0 conn(0x7f41fc0c1840 0x7f41fc0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f4200f9c700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc0c6460 0x7f41fc0bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563330765 0 0) 0x7f41fc1554a0 con 0x7f41fc0aa2d0 2023-11-06T21:00:29.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e40033f0 con 0x7f41fc0aa2d0 2023-11-06T21:00:29.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1744148228 0 0) 0x7f41e40033f0 con 0x7f41fc0aa2d0 2023-11-06T21:00:29.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc0c6460 msgr2=0x7f41fc0bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:29.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41faffd700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc0c6460 0x7f41fc0bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 >> v2:172.21.15.69:3300/0 conn(0x7f41fc0c1840 msgr2=0x7f41fc0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41faffd700 1 --2- 172.21.15.33:0/3981345637 >> v2:172.21.15.69:3300/0 conn(0x7f41fc0c1840 0x7f41fc0a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc154b70 con 0x7f41fc0aa2d0 2023-11-06T21:00:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.222+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f41f0001470 con 0x7f41fc0aa2d0 2023-11-06T21:00:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 >> v1:172.21.15.33:6789/0 conn(0x7f41fc0aa2d0 legacy=0x7f41fc0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41faffd700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41e40038e0 0x7f41e4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41faffd700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4004690 0x7f41e4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41faffd700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc0c6460 0x7f41e4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc154960 con 0x7f41e4004690 2023-11-06T21:00:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc155930 con 0x7f41e40038e0 2023-11-06T21:00:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e400b8e0 con 0x7f41fc0c6460 2023-11-06T21:00:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41fb7fe700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4004690 0x7f41e4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41fbfff700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41e40038e0 0x7f41e4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f4200f9c700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc0c6460 0x7f41e4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41fb7fe700 1 -- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc0c6460 msgr2=0x7f41e4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41fb7fe700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc0c6460 0x7f41e4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41fb7fe700 1 -- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41e40038e0 msgr2=0x7f41e4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41fb7fe700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41e40038e0 0x7f41e4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41fb7fe700 1 -- 172.21.15.33:0/3981345637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e40033f0 con 0x7f41e4004690 2023-11-06T21:00:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.223+0000 7f41fb7fe700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4004690 0x7f41e4004b10 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f41f000c1e0 tx=0x7f41fc155410).ready entity=mon.0 client_cookie=7c058a4aaf602224 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.224+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0007a60 con 0x7f41e4004690 2023-11-06T21:00:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.224+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f000cab0 con 0x7f41e4004690 2023-11-06T21:00:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.224+0000 7f41faffd700 1 -- 172.21.15.33:0/3981345637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f000ce30 con 0x7f41e4004690 2023-11-06T21:00:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.224+0000 7f42029ff700 1 -- 172.21.15.33:0/3981345637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4004690 msgr2=0x7f41e4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.224+0000 7f42029ff700 1 --2- 172.21.15.33:0/3981345637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4004690 0x7f41e4004b10 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f41f000c1e0 tx=0x7f41fc155410).stop 2023-11-06T21:00:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.224+0000 7f4200f9c700 1 -- 172.21.15.33:0/3981345637 reap_dead start 2023-11-06T21:00:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.224+0000 7f42029ff700 1 -- 172.21.15.33:0/3981345637 shutdown_connections 2023-11-06T21:00:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.224+0000 7f42029ff700 1 -- 172.21.15.33:0/3981345637 >> 172.21.15.33:0/3981345637 conn(0x7f41fc05ec80 msgr2=0x7f41fc0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.224+0000 7f42029ff700 1 -- 172.21.15.33:0/3981345637 shutdown_connections 2023-11-06T21:00:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.224+0000 7f42029ff700 1 -- 172.21.15.33:0/3981345637 wait complete. 2023-11-06T21:00:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.226+0000 7f42029ff700 1 Processor -- start 2023-11-06T21:00:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.226+0000 7f42029ff700 1 -- start start 2023-11-06T21:00:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.227+0000 7f42029ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41e4004690 0x7f41fc0c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.227+0000 7f42029ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41fc1586e0 0x7f41fc140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.227+0000 7f42029ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc14edc0 0x7f41fc152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.227+0000 7f42029ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0c4d30 con 0x7f41fc1586e0 2023-11-06T21:00:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.227+0000 7f42029ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc158250 con 0x7f41e4004690 2023-11-06T21:00:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.227+0000 7f42029ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc140fd0 con 0x7f41fc14edc0 2023-11-06T21:00:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.227+0000 7f41fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41fc1586e0 0x7f41fc140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.227+0000 7f41fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41fc1586e0 0x7f41fc140ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36580/0 (socket says 172.21.15.33:36580) 2023-11-06T21:00:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.227+0000 7f41fb7fe700 1 -- 172.21.15.33:0/4177597008 learned_addr learned my addr 172.21.15.33:0/4177597008 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41fbfff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41e4004690 0x7f41fc0c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f4200f9c700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc14edc0 0x7f41fc152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41fb7fe700 1 -- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc14edc0 msgr2=0x7f41fc152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41fb7fe700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc14edc0 0x7f41fc152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41fb7fe700 1 -- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41e4004690 msgr2=0x7f41fc0c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41fb7fe700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41e4004690 0x7f41fc0c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41fb7fe700 1 -- 172.21.15.33:0/4177597008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc157640 con 0x7f41fc1586e0 2023-11-06T21:00:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41fbfff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41e4004690 0x7f41fc0c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41fb7fe700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41fc1586e0 0x7f41fc140ae0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f41f0007b90 tx=0x7f41f0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41e3fff700 1 -- 172.21.15.33:0/4177597008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0001710 con 0x7f41fc1586e0 2023-11-06T21:00:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41e3fff700 1 -- 172.21.15.33:0/4177597008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f00046b0 con 0x7f41fc1586e0 2023-11-06T21:00:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f41e3fff700 1 -- 172.21.15.33:0/4177597008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0004a70 con 0x7f41fc1586e0 2023-11-06T21:00:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f42029ff700 1 -- 172.21.15.33:0/4177597008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc157920 con 0x7f41fc1586e0 2023-11-06T21:00:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.228+0000 7f42029ff700 1 -- 172.21.15.33:0/4177597008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc157ee0 con 0x7f41fc1586e0 2023-11-06T21:00:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.229+0000 7f42029ff700 1 -- 172.21.15.33:0/4177597008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41fc06c6c0 con 0x7f41fc1586e0 2023-11-06T21:00:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.230+0000 7f41e3fff700 1 -- 172.21.15.33:0/4177597008 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f41f0020450 con 0x7f41fc1586e0 2023-11-06T21:00:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.231+0000 7f41e3fff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f41e805f270 0x7f41e8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.231+0000 7f41e3fff700 1 -- 172.21.15.33:0/4177597008 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f41f007ad60 con 0x7f41fc1586e0 2023-11-06T21:00:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.231+0000 7f41fbfff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f41e805f270 0x7f41e8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.232+0000 7f41fbfff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f41e805f270 0x7f41e8061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f41fc153230 tx=0x7f41ec008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.235+0000 7f41e3fff700 1 -- 172.21.15.33:0/4177597008 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f41f004c0e0 con 0x7f41fc1586e0 2023-11-06T21:00:29.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.486+0000 7f42029ff700 1 -- 172.21.15.33:0/4177597008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f41fc158390 con 0x7f41fc1586e0 2023-11-06T21:00:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.486+0000 7f41e3fff700 1 -- 172.21.15.33:0/4177597008 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f41f0025020 con 0x7f41fc1586e0 2023-11-06T21:00:29.487 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:29.487 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T21:00:29.488 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T21:00:29.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.488 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T21:00:29.488 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T21:00:29.488 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.489 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:00:29.489 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:00:29.489 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:00:29.489 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.490 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T21:00:29.490 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T21:00:29.490 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.490 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:00:29.491 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:00:29.491 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:00:29.491 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.491 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.492 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T21:00:29.492 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T21:00:29.492 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.492 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:00:29.493 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:00:29.493 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:00:29.493 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.494 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T21:00:29.494 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T21:00:29.494 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.494 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:00:29.494 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:00:29.495 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:00:29.495 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.495 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.496 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T21:00:29.496 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T21:00:29.496 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.496 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:00:29.496 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:00:29.496 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.497 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.497 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T21:00:29.497 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T21:00:29.498 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.498 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:00:29.498 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:00:29.498 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.499 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T21:00:29.499 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T21:00:29.499 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.499 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:00:29.500 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:00:29.500 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.500 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.501 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T21:00:29.501 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T21:00:29.501 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.501 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:00:29.502 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:00:29.502 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.502 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.503 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T21:00:29.503 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T21:00:29.503 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.503 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:00:29.503 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:00:29.503 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.504 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.504 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T21:00:29.504 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T21:00:29.505 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:00:29.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:00:29.505 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.505 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.506 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T21:00:29.506 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T21:00:29.506 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:00:29.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:00:29.507 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.508 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T21:00:29.508 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T21:00:29.508 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:00:29.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:00:29.509 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.509 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.509 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T21:00:29.510 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T21:00:29.510 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:00:29.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-11-06T21:00:29.510 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T21:00:29.511 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.511 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T21:00:29.511 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T21:00:29.512 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:00:29.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:00:29.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:00:29.512 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:00:29.513 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.514 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T21:00:29.514 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T21:00:29.514 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:00:29.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:00:29.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:00:29.515 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:00:29.515 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.516 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T21:00:29.516 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T21:00:29.516 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:00:29.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:00:29.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:00:29.517 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:00:29.517 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.518 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T21:00:29.518 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T21:00:29.518 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:00:29.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:00:29.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:00:29.519 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.519 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.520 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T21:00:29.520 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T21:00:29.520 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:00:29.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T21:00:29.521 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.521 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:29.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:29.521 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T21:00:29.522 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T21:00:29.522 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:00:29.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:00:29.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T21:00:29.522 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:00:29.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:29.523 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T21:00:29.523 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T21:00:29.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 -- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f41e805f270 msgr2=0x7f41e8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f41e805f270 0x7f41e8061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f41fc153230 tx=0x7f41ec008040).stop 2023-11-06T21:00:29.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 -- 172.21.15.33:0/4177597008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41fc1586e0 msgr2=0x7f41fc140ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41fc1586e0 0x7f41fc140ae0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f41f0007b90 tx=0x7f41f0015000).stop 2023-11-06T21:00:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 -- 172.21.15.33:0/4177597008 shutdown_connections 2023-11-06T21:00:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f41e805f270 0x7f41e8061730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41fc14edc0 0x7f41fc152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41e4004690 0x7f41fc0c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 --2- 172.21.15.33:0/4177597008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41fc1586e0 0x7f41fc140ae0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 -- 172.21.15.33:0/4177597008 >> 172.21.15.33:0/4177597008 conn(0x7f41fc05ec80 msgr2=0x7f41fc0ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:29.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 -- 172.21.15.33:0/4177597008 shutdown_connections 2023-11-06T21:00:29.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.488+0000 7f42029ff700 1 -- 172.21.15.33:0/4177597008 wait complete. 2023-11-06T21:00:29.528 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:00:29.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.723+0000 7fd72e9ff700 1 Processor -- start 2023-11-06T21:00:29.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.723+0000 7fd72e9ff700 1 -- start start 2023-11-06T21:00:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.725+0000 7fd72e9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7280c1840 0x7fd7280a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.725+0000 7fd72e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7280aa2d0 0x7fd7280c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.725+0000 7fd72e9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd7280c6240 0x7fd728157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.725+0000 7fd72e9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd728154960 con 0x7fd7280c1840 2023-11-06T21:00:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.725+0000 7fd72e9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7281554a0 con 0x7fd7280c6240 2023-11-06T21:00:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.725+0000 7fd72e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd728155930 con 0x7fd7280aa2d0 2023-11-06T21:00:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.725+0000 7fd727fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7280c1840 0x7fd7280a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.725+0000 7fd727fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7280c1840 0x7fd7280a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36592/0 (socket says 172.21.15.33:36592) 2023-11-06T21:00:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.725+0000 7fd727fff700 1 -- 172.21.15.33:0/3962801113 learned_addr learned my addr 172.21.15.33:0/3962801113 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd7277fe700 1 --2- 172.21.15.33:0/3962801113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7280aa2d0 0x7fd7280c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd72cf9c700 1 --2- 172.21.15.33:0/3962801113 >> v2:172.21.15.69:3300/0 conn(0x7fd7280c6240 0x7fd728157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd727fff700 1 -- 172.21.15.33:0/3962801113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7280aa2d0 msgr2=0x7fd7280c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:29.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd727fff700 1 --2- 172.21.15.33:0/3962801113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7280aa2d0 0x7fd7280c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd727fff700 1 -- 172.21.15.33:0/3962801113 >> v2:172.21.15.69:3300/0 conn(0x7fd7280c6240 msgr2=0x7fd728157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:29.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd727fff700 1 --2- 172.21.15.33:0/3962801113 >> v2:172.21.15.69:3300/0 conn(0x7fd7280c6240 0x7fd728157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd727fff700 1 -- 172.21.15.33:0/3962801113 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280c66b0 con 0x7fd7280c1840 2023-11-06T21:00:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd727fff700 1 --2- 172.21.15.33:0/3962801113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280c1840 0x7fd7280a9de0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7fd728084630 tx=0x7fd7180090f0).ready entity=mon.0 client_cookie=7d8561e409a7c0e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd726ffd700 1 -- 172.21.15.33:0/3962801113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800b600 con 0x7fd7280c1840 2023-11-06T21:00:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd726ffd700 1 -- 172.21.15.33:0/3962801113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71800b760 con 0x7fd7280c1840 2023-11-06T21:00:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.726+0000 7fd726ffd700 1 -- 172.21.15.33:0/3962801113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800bae0 con 0x7fd7280c1840 2023-11-06T21:00:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.727+0000 7fd72e9ff700 1 -- 172.21.15.33:0/3962801113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280c1840 msgr2=0x7fd7280a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.727+0000 7fd72e9ff700 1 --2- 172.21.15.33:0/3962801113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280c1840 0x7fd7280a9de0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7fd728084630 tx=0x7fd7180090f0).stop 2023-11-06T21:00:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.727+0000 7fd72e9ff700 1 -- 172.21.15.33:0/3962801113 shutdown_connections 2023-11-06T21:00:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.727+0000 7fd72e9ff700 1 --2- 172.21.15.33:0/3962801113 >> v2:172.21.15.69:3300/0 conn(0x7fd7280c6240 0x7fd728157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.727+0000 7fd72e9ff700 1 --2- 172.21.15.33:0/3962801113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280c1840 0x7fd7280a9de0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.727+0000 7fd72e9ff700 1 --2- 172.21.15.33:0/3962801113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7280aa2d0 0x7fd7280c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.727+0000 7fd72e9ff700 1 -- 172.21.15.33:0/3962801113 >> 172.21.15.33:0/3962801113 conn(0x7fd72805ec80 msgr2=0x7fd7280cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.727+0000 7fd72e9ff700 1 -- 172.21.15.33:0/3962801113 shutdown_connections 2023-11-06T21:00:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.727+0000 7fd72e9ff700 1 -- 172.21.15.33:0/3962801113 wait complete. 2023-11-06T21:00:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.728+0000 7fd72e9ff700 1 Processor -- start 2023-11-06T21:00:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.729+0000 7fd72e9ff700 1 -- start start 2023-11-06T21:00:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.729+0000 7fd72e9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280aa2d0 0x7fd728157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.729+0000 7fd72e9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7280c1840 0x7fd7280b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.729+0000 7fd72e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7280c6240 0x7fd728149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd72e9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280ae320 con 0x7fd7280aa2d0 2023-11-06T21:00:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd72e9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd728158190 con 0x7fd7280c1840 2023-11-06T21:00:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd72e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd728158470 con 0x7fd7280c6240 2023-11-06T21:00:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd727fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280aa2d0 0x7fd728157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd727fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280aa2d0 0x7fd728157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36596/0 (socket says 172.21.15.33:36596) 2023-11-06T21:00:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd727fff700 1 -- 172.21.15.33:0/1633762591 learned_addr learned my addr 172.21.15.33:0/1633762591 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd7277fe700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7280c1840 0x7fd7280b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd72cf9c700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7280c6240 0x7fd728149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd727fff700 1 -- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7280c6240 msgr2=0x7fd728149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd727fff700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7280c6240 0x7fd728149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd727fff700 1 -- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7280c1840 msgr2=0x7fd7280b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd727fff700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7280c1840 0x7fd7280b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd727fff700 1 -- 172.21.15.33:0/1633762591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd718009370 con 0x7fd7280aa2d0 2023-11-06T21:00:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.730+0000 7fd727fff700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280aa2d0 0x7fd728157b60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7fd718009130 tx=0x7fd718017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.731+0000 7fd70ffff700 1 -- 172.21.15.33:0/1633762591 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800b8d0 con 0x7fd7280aa2d0 2023-11-06T21:00:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.731+0000 7fd70ffff700 1 -- 172.21.15.33:0/1633762591 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71800ba30 con 0x7fd7280aa2d0 2023-11-06T21:00:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.731+0000 7fd70ffff700 1 -- 172.21.15.33:0/1633762591 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71801c670 con 0x7fd7280aa2d0 2023-11-06T21:00:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.731+0000 7fd72e9ff700 1 -- 172.21.15.33:0/1633762591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd728158750 con 0x7fd7280aa2d0 2023-11-06T21:00:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.731+0000 7fd72e9ff700 1 -- 172.21.15.33:0/1633762591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7280b0db0 con 0x7fd7280aa2d0 2023-11-06T21:00:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.732+0000 7fd72e9ff700 1 -- 172.21.15.33:0/1633762591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72806c6c0 con 0x7fd7280aa2d0 2023-11-06T21:00:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.733+0000 7fd70ffff700 1 -- 172.21.15.33:0/1633762591 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd71801ca20 con 0x7fd7280aa2d0 2023-11-06T21:00:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.734+0000 7fd70ffff700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd71405f240 0x7fd714061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.734+0000 7fd70ffff700 1 -- 172.21.15.33:0/1633762591 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fd71807c160 con 0x7fd7280aa2d0 2023-11-06T21:00:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.734+0000 7fd7277fe700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd71405f240 0x7fd714061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.735+0000 7fd7277fe700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd71405f240 0x7fd714061700 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fd7280c5290 tx=0x7fd71c008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.738+0000 7fd70ffff700 1 -- 172.21.15.33:0/1633762591 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd71804d540 con 0x7fd7280aa2d0 2023-11-06T21:00:29.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.921+0000 7fd72e9ff700 1 -- 172.21.15.33:0/1633762591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd728063f20 con 0x7fd7280aa2d0 2023-11-06T21:00:29.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.922+0000 7fd70ffff700 1 -- 172.21.15.33:0/1633762591 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+13763 (secure 0 0 0) 0x7fd71802a090 con 0x7fd7280aa2d0 2023-11-06T21:00:29.923 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:29.923 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":110,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:00:28.992516+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:00:29.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.924+0000 7fd72e9ff700 1 -- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd71405f240 msgr2=0x7fd714061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.924+0000 7fd72e9ff700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd71405f240 0x7fd714061700 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fd7280c5290 tx=0x7fd71c008040).stop 2023-11-06T21:00:29.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.924+0000 7fd72e9ff700 1 -- 172.21.15.33:0/1633762591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280aa2d0 msgr2=0x7fd728157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:29.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.924+0000 7fd72e9ff700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280aa2d0 0x7fd728157b60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7fd718009130 tx=0x7fd718017000).stop 2023-11-06T21:00:29.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.924+0000 7fd72e9ff700 1 -- 172.21.15.33:0/1633762591 shutdown_connections 2023-11-06T21:00:29.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.924+0000 7fd72e9ff700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd71405f240 0x7fd714061700 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.924+0000 7fd72e9ff700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd7280c6240 0x7fd728149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.924+0000 7fd72e9ff700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7280aa2d0 0x7fd728157b60 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.924+0000 7fd72e9ff700 1 --2- 172.21.15.33:0/1633762591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7280c1840 0x7fd7280b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:29.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.924+0000 7fd72e9ff700 1 -- 172.21.15.33:0/1633762591 >> 172.21.15.33:0/1633762591 conn(0x7fd72805ec80 msgr2=0x7fd728152b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:29.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.925+0000 7fd72e9ff700 1 -- 172.21.15.33:0/1633762591 shutdown_connections 2023-11-06T21:00:29.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:29.925+0000 7fd72e9ff700 1 -- 172.21.15.33:0/1633762591 wait complete. 2023-11-06T21:00:29.943 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T21:00:29.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T21:00:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.143+0000 7fe4e25ff700 1 Processor -- start 2023-11-06T21:00:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.143+0000 7fe4e25ff700 1 -- start start 2023-11-06T21:00:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.145+0000 7fe4e25ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4dc1546a0 0x7fe4dc064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.145+0000 7fe4e25ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4dc0641f0 con 0x7fe4dc0be2c0 2023-11-06T21:00:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.145+0000 7fe4e25ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4dc0638c0 con 0x7fe4dc066560 2023-11-06T21:00:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.145+0000 7fe4e25ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4dc154a80 con 0x7fe4dc1546a0 2023-11-06T21:00:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.145+0000 7fe4dbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4dc1546a0 0x7fe4dc064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.145+0000 7fe4dbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4dc1546a0 0x7fe4dc064680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36600/0 (socket says 172.21.15.33:36600) 2023-11-06T21:00:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.145+0000 7fe4dbfff700 1 -- 172.21.15.33:0/4152019552 learned_addr learned my addr 172.21.15.33:0/4152019552 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.145+0000 7fe4e0b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe4dc0be2c0 0x7fe4dc0c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48282/0 (socket says 172.21.15.33:48282) 2023-11-06T21:00:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.146+0000 7fe4dbfff700 1 -- 172.21.15.33:0/4152019552 >> v1:172.21.15.69:6789/0 conn(0x7fe4dc066560 legacy=0x7fe4dc0bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.146+0000 7fe4dbfff700 1 -- 172.21.15.33:0/4152019552 >> v1:172.21.15.33:6789/0 conn(0x7fe4dc0be2c0 legacy=0x7fe4dc0c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.146+0000 7fe4dbfff700 1 -- 172.21.15.33:0/4152019552 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc063ba0 con 0x7fe4dc1546a0 2023-11-06T21:00:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.146+0000 7fe4dbfff700 1 --2- 172.21.15.33:0/4152019552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 0x7fe4dc064680 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fe4cc006fd0 tx=0x7fe4cc0090f0).ready entity=mon.0 client_cookie=eb56ab6fe0cba1e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.146+0000 7fe4daffd700 1 -- 172.21.15.33:0/4152019552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00b600 con 0x7fe4dc1546a0 2023-11-06T21:00:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.146+0000 7fe4daffd700 1 -- 172.21.15.33:0/4152019552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc00b760 con 0x7fe4dc1546a0 2023-11-06T21:00:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.146+0000 7fe4daffd700 1 -- 172.21.15.33:0/4152019552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00bae0 con 0x7fe4dc1546a0 2023-11-06T21:00:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.147+0000 7fe4e25ff700 1 -- 172.21.15.33:0/4152019552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 msgr2=0x7fe4dc064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.147+0000 7fe4e25ff700 1 --2- 172.21.15.33:0/4152019552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 0x7fe4dc064680 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fe4cc006fd0 tx=0x7fe4cc0090f0).stop 2023-11-06T21:00:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.147+0000 7fe4e25ff700 1 -- 172.21.15.33:0/4152019552 shutdown_connections 2023-11-06T21:00:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.147+0000 7fe4e25ff700 1 --2- 172.21.15.33:0/4152019552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 0x7fe4dc064680 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.147+0000 7fe4e25ff700 1 -- 172.21.15.33:0/4152019552 >> 172.21.15.33:0/4152019552 conn(0x7fe4dc0c8c50 msgr2=0x7fe4dc0c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.147+0000 7fe4e25ff700 1 -- 172.21.15.33:0/4152019552 shutdown_connections 2023-11-06T21:00:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.147+0000 7fe4e25ff700 1 -- 172.21.15.33:0/4152019552 wait complete. 2023-11-06T21:00:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.148+0000 7fe4e25ff700 1 Processor -- start 2023-11-06T21:00:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.149+0000 7fe4e25ff700 1 -- start start 2023-11-06T21:00:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.149+0000 7fe4e25ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4dc066560 0x7fe4dc0ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e25ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4dc0be2c0 0x7fe4dc0c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e25ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 0x7fe4dc0a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e25ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc0aafc0 con 0x7fe4dc1546a0 2023-11-06T21:00:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e25ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc0aab30 con 0x7fe4dc0be2c0 2023-11-06T21:00:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e25ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc0aa200 con 0x7fe4dc066560 2023-11-06T21:00:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e0b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 0x7fe4dc0a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e0b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 0x7fe4dc0a8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36606/0 (socket says 172.21.15.33:36606) 2023-11-06T21:00:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e0b9c700 1 -- 172.21.15.33:0/2848886661 learned_addr learned my addr 172.21.15.33:0/2848886661 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4dbfff700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4dc066560 0x7fe4dc0ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4db7fe700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4dc0be2c0 0x7fe4dc0c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e0b9c700 1 -- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4dc066560 msgr2=0x7fe4dc0ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e0b9c700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4dc066560 0x7fe4dc0ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e0b9c700 1 -- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4dc0be2c0 msgr2=0x7fe4dc0c06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e0b9c700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4dc0be2c0 0x7fe4dc0c06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e0b9c700 1 -- 172.21.15.33:0/2848886661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc009370 con 0x7fe4dc1546a0 2023-11-06T21:00:30.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.150+0000 7fe4e0b9c700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 0x7fe4dc0a8c10 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7fe4d400c7c0 tx=0x7fe4d4001560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:30.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.151+0000 7fe4c3fff700 1 -- 172.21.15.33:0/2848886661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4005c70 con 0x7fe4dc1546a0 2023-11-06T21:00:30.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.151+0000 7fe4c3fff700 1 -- 172.21.15.33:0/2848886661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d4005dd0 con 0x7fe4dc1546a0 2023-11-06T21:00:30.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.151+0000 7fe4c3fff700 1 -- 172.21.15.33:0/2848886661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4007320 con 0x7fe4dc1546a0 2023-11-06T21:00:30.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.151+0000 7fe4e25ff700 1 -- 172.21.15.33:0/2848886661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc0aa4e0 con 0x7fe4dc1546a0 2023-11-06T21:00:30.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.151+0000 7fe4e25ff700 1 -- 172.21.15.33:0/2848886661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc0c0bc0 con 0x7fe4dc1546a0 2023-11-06T21:00:30.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.152+0000 7fe4e25ff700 1 -- 172.21.15.33:0/2848886661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4dc066000 con 0x7fe4dc1546a0 2023-11-06T21:00:30.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.153+0000 7fe4c3fff700 1 -- 172.21.15.33:0/2848886661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe4d40076d0 con 0x7fe4dc1546a0 2023-11-06T21:00:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.154+0000 7fe4c3fff700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe4c805f270 0x7fe4c8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.154+0000 7fe4c3fff700 1 -- 172.21.15.33:0/2848886661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fe4d407a010 con 0x7fe4dc1546a0 2023-11-06T21:00:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.154+0000 7fe4dbfff700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe4c805f270 0x7fe4c8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.155+0000 7fe4dbfff700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe4c805f270 0x7fe4c8061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fe4cc009130 tx=0x7fe4cc017000).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:30.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.158+0000 7fe4c3fff700 1 -- 172.21.15.33:0/2848886661 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe4d404b4b0 con 0x7fe4dc1546a0 2023-11-06T21:00:30.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.335+0000 7fe4e25ff700 1 -- 172.21.15.33:0/2848886661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe4dc0c35b0 con 0x7fe4dc1546a0 2023-11-06T21:00:30.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.336+0000 7fe4c3fff700 1 -- 172.21.15.33:0/2848886661 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v110) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe4d404ead0 con 0x7fe4dc1546a0 2023-11-06T21:00:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T21:00:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 -- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe4c805f270 msgr2=0x7fe4c8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe4c805f270 0x7fe4c8061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fe4cc009130 tx=0x7fe4cc017000).stop 2023-11-06T21:00:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 -- 172.21.15.33:0/2848886661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 msgr2=0x7fe4dc0a8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 0x7fe4dc0a8c10 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7fe4d400c7c0 tx=0x7fe4d4001560).stop 2023-11-06T21:00:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 -- 172.21.15.33:0/2848886661 shutdown_connections 2023-11-06T21:00:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe4c805f270 0x7fe4c8061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4dc066560 0x7fe4dc0ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4dc1546a0 0x7fe4dc0a8c10 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 --2- 172.21.15.33:0/2848886661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4dc0be2c0 0x7fe4dc0c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 -- 172.21.15.33:0/2848886661 >> 172.21.15.33:0/2848886661 conn(0x7fe4dc0c8c50 msgr2=0x7fe4dc0682d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 -- 172.21.15.33:0/2848886661 shutdown_connections 2023-11-06T21:00:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.338+0000 7fe4e25ff700 1 -- 172.21.15.33:0/2848886661 wait complete. 2023-11-06T21:00:30.356 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T21:00:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.554+0000 7fcfcd9ff700 1 Processor -- start 2023-11-06T21:00:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.555+0000 7fcfcd9ff700 1 -- start start 2023-11-06T21:00:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.556+0000 7fcfcd9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 0x7fcfc80c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.556+0000 7fcfcd9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc80c6b60 con 0x7fcfc8154710 2023-11-06T21:00:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.556+0000 7fcfcd9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc80c7700 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.556+0000 7fcfcd9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc80c5cd0 con 0x7fcfc8154330 2023-11-06T21:00:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.556+0000 7fcfc67fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcfc8154710 0x7fcfc80c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48294/0 (socket says 172.21.15.33:48294) 2023-11-06T21:00:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.556+0000 7fcfc67fc700 1 -- 172.21.15.33:0/1260857619 learned_addr learned my addr 172.21.15.33:0/1260857619 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.556+0000 7fcfc6ffd700 1 --2- 172.21.15.33:0/1260857619 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 0x7fcfc80c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc5ffb700 1 -- 172.21.15.33:0/1260857619 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021134385 0 0) 0x7fcfc80c6b60 con 0x7fcfc8154710 2023-11-06T21:00:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc5ffb700 1 -- 172.21.15.33:0/1260857619 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb00033f0 con 0x7fcfc8154710 2023-11-06T21:00:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc5ffb700 1 -- 172.21.15.33:0/1260857619 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906512589 0 0) 0x7fcfc80c7700 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc5ffb700 1 -- 172.21.15.33:0/1260857619 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc80c6b60 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc6ffd700 1 -- 172.21.15.33:0/1260857619 >> v1:172.21.15.69:6789/0 conn(0x7fcfc80c7ff0 legacy=0x7fcfc80646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc6ffd700 1 -- 172.21.15.33:0/1260857619 >> v1:172.21.15.33:6789/0 conn(0x7fcfc8154710 legacy=0x7fcfc80c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc6ffd700 1 -- 172.21.15.33:0/1260857619 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80c5f50 con 0x7fcfc8154330 2023-11-06T21:00:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc6ffd700 1 --2- 172.21.15.33:0/1260857619 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 0x7fcfc80c57e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fcfb8009790 tx=0x7fcfb8009060).ready entity=mon.2 client_cookie=627140f668fc5654 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc5ffb700 1 -- 172.21.15.33:0/1260857619 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800f510 con 0x7fcfc8154330 2023-11-06T21:00:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc5ffb700 1 -- 172.21.15.33:0/1260857619 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb800f670 con 0x7fcfc8154330 2023-11-06T21:00:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.557+0000 7fcfc5ffb700 1 -- 172.21.15.33:0/1260857619 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800f9f0 con 0x7fcfc8154330 2023-11-06T21:00:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.558+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/1260857619 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 msgr2=0x7fcfc80c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.558+0000 7fcfcd9ff700 1 --2- 172.21.15.33:0/1260857619 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 0x7fcfc80c57e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fcfb8009790 tx=0x7fcfb8009060).stop 2023-11-06T21:00:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.558+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/1260857619 shutdown_connections 2023-11-06T21:00:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.558+0000 7fcfcd9ff700 1 --2- 172.21.15.33:0/1260857619 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 0x7fcfc80c57e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.558+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/1260857619 >> 172.21.15.33:0/1260857619 conn(0x7fcfc80cb910 msgr2=0x7fcfc80c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.558+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/1260857619 shutdown_connections 2023-11-06T21:00:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.558+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/1260857619 wait complete. 2023-11-06T21:00:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.559+0000 7fcfcd9ff700 1 Processor -- start 2023-11-06T21:00:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.560+0000 7fcfcd9ff700 1 -- start start 2023-11-06T21:00:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfcd9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80c7ff0 0x7fcfc81564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfcd9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 0x7fcfc8154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfcd9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfc8154710 0x7fcfc81656c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfcd9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc8155090 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfcd9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc8165bb0 con 0x7fcfc8154710 2023-11-06T21:00:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfcd9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc80ae340 con 0x7fcfc8154330 2023-11-06T21:00:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80c7ff0 0x7fcfc81564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80c7ff0 0x7fcfc81564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36618/0 (socket says 172.21.15.33:36618) 2023-11-06T21:00:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc6ffd700 1 -- 172.21.15.33:0/578693148 learned_addr learned my addr 172.21.15.33:0/578693148 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc77fe700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfc8154710 0x7fcfc81656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc67fc700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 0x7fcfc8154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc6ffd700 1 -- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 msgr2=0x7fcfc8154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc6ffd700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 0x7fcfc8154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc6ffd700 1 -- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfc8154710 msgr2=0x7fcfc81656c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc6ffd700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfc8154710 0x7fcfc81656c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc6ffd700 1 -- 172.21.15.33:0/578693148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80092a0 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.561+0000 7fcfc6ffd700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80c7ff0 0x7fcfc81564b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7fcfc806c170 tx=0x7fcfb8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.562+0000 7fcfaeffd700 1 -- 172.21.15.33:0/578693148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800f7e0 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.562+0000 7fcfaeffd700 1 -- 172.21.15.33:0/578693148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb800f940 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.562+0000 7fcfaeffd700 1 -- 172.21.15.33:0/578693148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb801b690 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.562+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/578693148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc8155b20 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.562+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/578693148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc8155560 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.563+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/578693148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc80830b0 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.564+0000 7fcfaeffd700 1 -- 172.21.15.33:0/578693148 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcfb801ba00 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.565+0000 7fcfaeffd700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfb405f270 0x7fcfb4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.565+0000 7fcfaeffd700 1 -- 172.21.15.33:0/578693148 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fcfb807b820 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.565+0000 7fcfc67fc700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfb405f270 0x7fcfb4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.566+0000 7fcfc67fc700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfb405f270 0x7fcfb4061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcfc8162780 tx=0x7fcfbc009040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.569+0000 7fcfaeffd700 1 -- 172.21.15.33:0/578693148 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcfb804ccc0 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.746+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/578693148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcfc80652d0 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.746+0000 7fcfaeffd700 1 -- 172.21.15.33:0/578693148 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v110) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcfb801bcb0 con 0x7fcfc80c7ff0 2023-11-06T21:00:30.748 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T21:00:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.748+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfb405f270 msgr2=0x7fcfb4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfb405f270 0x7fcfb4061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcfc8162780 tx=0x7fcfbc009040).stop 2023-11-06T21:00:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/578693148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80c7ff0 msgr2=0x7fcfc81564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80c7ff0 0x7fcfc81564b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7fcfc806c170 tx=0x7fcfb8017000).stop 2023-11-06T21:00:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/578693148 shutdown_connections 2023-11-06T21:00:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfb405f270 0x7fcfb4061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfc8154710 0x7fcfc81656c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80c7ff0 0x7fcfc81564b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 --2- 172.21.15.33:0/578693148 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc8154330 0x7fcfc8154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/578693148 >> 172.21.15.33:0/578693148 conn(0x7fcfc80cb910 msgr2=0x7fcfc80aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/578693148 shutdown_connections 2023-11-06T21:00:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.749+0000 7fcfcd9ff700 1 -- 172.21.15.33:0/578693148 wait complete. 2023-11-06T21:00:30.767 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T21:00:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.962+0000 7f7242dff700 1 Processor -- start 2023-11-06T21:00:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.962+0000 7f7242dff700 1 -- start start 2023-11-06T21:00:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7242dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f723c0c1840 0x7f723c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7242dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f723c0c6460 0x7f723c0bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7242dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723c1554a0 con 0x7f723c0aa2d0 2023-11-06T21:00:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7242dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723c154960 con 0x7f723c0c1840 2023-11-06T21:00:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7242dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723c155930 con 0x7f723c0c6460 2023-11-06T21:00:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7240b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f723c0c1840 0x7f723c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7240b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f723c0c1840 0x7f723c0a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36632/0 (socket says 172.21.15.33:36632) 2023-11-06T21:00:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7240b9b700 1 -- 172.21.15.33:0/681019815 learned_addr learned my addr 172.21.15.33:0/681019815 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f724139c700 1 --2- 172.21.15.33:0/681019815 >> v2:172.21.15.69:3300/0 conn(0x7f723c0c6460 0x7f723c0bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7240b9b700 1 -- 172.21.15.33:0/681019815 >> v2:172.21.15.69:3300/0 conn(0x7f723c0c6460 msgr2=0x7f723c0bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7240b9b700 1 --2- 172.21.15.33:0/681019815 >> v2:172.21.15.69:3300/0 conn(0x7f723c0c6460 0x7f723c0bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7240b9b700 1 -- 172.21.15.33:0/681019815 >> v1:172.21.15.69:6789/0 conn(0x7f723c0aa2d0 legacy=0x7f723c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.964+0000 7f7240b9b700 1 -- 172.21.15.33:0/681019815 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c154b70 con 0x7f723c0c1840 2023-11-06T21:00:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f7240b9b700 1 --2- 172.21.15.33:0/681019815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0c1840 0x7f723c0a9de0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f723c0c55f0 tx=0x7f72300090f0).ready entity=mon.0 client_cookie=aab07cf4a6f7f873 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f723b7fe700 1 -- 172.21.15.33:0/681019815 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723000b5a0 con 0x7f723c0c1840 2023-11-06T21:00:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f723b7fe700 1 -- 172.21.15.33:0/681019815 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723000b700 con 0x7f723c0c1840 2023-11-06T21:00:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f723b7fe700 1 -- 172.21.15.33:0/681019815 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723000ba80 con 0x7f723c0c1840 2023-11-06T21:00:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f7242dff700 1 -- 172.21.15.33:0/681019815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0c1840 msgr2=0x7f723c0a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f7242dff700 1 --2- 172.21.15.33:0/681019815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0c1840 0x7f723c0a9de0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f723c0c55f0 tx=0x7f72300090f0).stop 2023-11-06T21:00:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f7242dff700 1 -- 172.21.15.33:0/681019815 shutdown_connections 2023-11-06T21:00:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f7242dff700 1 --2- 172.21.15.33:0/681019815 >> v2:172.21.15.69:3300/0 conn(0x7f723c0c6460 0x7f723c0bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f7242dff700 1 --2- 172.21.15.33:0/681019815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0c1840 0x7f723c0a9de0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f7242dff700 1 -- 172.21.15.33:0/681019815 >> 172.21.15.33:0/681019815 conn(0x7f723c05ec80 msgr2=0x7f723c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f7242dff700 1 -- 172.21.15.33:0/681019815 shutdown_connections 2023-11-06T21:00:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.965+0000 7f7242dff700 1 -- 172.21.15.33:0/681019815 wait complete. 2023-11-06T21:00:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.967+0000 7f7242dff700 1 Processor -- start 2023-11-06T21:00:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.967+0000 7f7242dff700 1 -- start start 2023-11-06T21:00:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f7242dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0aa2d0 0x7f723c14cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f7242dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f723c0c1840 0x7f723c0ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f7242dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f723c0c6460 0x7f723c0ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f7242dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c0aea00 con 0x7f723c0aa2d0 2023-11-06T21:00:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f7242dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c14d0b0 con 0x7f723c0c6460 2023-11-06T21:00:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f7242dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723c0c3e20 con 0x7f723c0c1840 2023-11-06T21:00:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f7240b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0aa2d0 0x7f723c14cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f7240b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0aa2d0 0x7f723c14cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36636/0 (socket says 172.21.15.33:36636) 2023-11-06T21:00:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f7240b9b700 1 -- 172.21.15.33:0/2151252997 learned_addr learned my addr 172.21.15.33:0/2151252997 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f724139c700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f723c0c6460 0x7f723c0ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.968+0000 7f723bfff700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f723c0c1840 0x7f723c0ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7240b9b700 1 -- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f723c0c1840 msgr2=0x7f723c0ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7240b9b700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f723c0c1840 0x7f723c0ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7240b9b700 1 -- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f723c0c6460 msgr2=0x7f723c0ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7240b9b700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f723c0c6460 0x7f723c0ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7240b9b700 1 -- 172.21.15.33:0/2151252997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7230009370 con 0x7f723c0aa2d0 2023-11-06T21:00:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f723bfff700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f723c0c1840 0x7f723c0ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7240b9b700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0aa2d0 0x7f723c14cbc0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f723c06bc90 tx=0x7f7230015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7238ff9700 1 -- 172.21.15.33:0/2151252997 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723000b870 con 0x7f723c0aa2d0 2023-11-06T21:00:30.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7238ff9700 1 -- 172.21.15.33:0/2151252997 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723000b9d0 con 0x7f723c0aa2d0 2023-11-06T21:00:30.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7242dff700 1 -- 172.21.15.33:0/2151252997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723c0c4100 con 0x7f723c0aa2d0 2023-11-06T21:00:30.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7238ff9700 1 -- 172.21.15.33:0/2151252997 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723001c630 con 0x7f723c0aa2d0 2023-11-06T21:00:30.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.969+0000 7f7242dff700 1 -- 172.21.15.33:0/2151252997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723c0c46c0 con 0x7f723c0aa2d0 2023-11-06T21:00:30.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.971+0000 7f7242dff700 1 -- 172.21.15.33:0/2151252997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723c0c4a10 con 0x7f723c0aa2d0 2023-11-06T21:00:30.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.971+0000 7f7238ff9700 1 -- 172.21.15.33:0/2151252997 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f723001c9e0 con 0x7f723c0aa2d0 2023-11-06T21:00:30.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.973+0000 7f7238ff9700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f722005f270 0x7f7220061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:30.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.973+0000 7f7238ff9700 1 -- 172.21.15.33:0/2151252997 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f723007bcc0 con 0x7f723c0aa2d0 2023-11-06T21:00:30.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.973+0000 7f723bfff700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f722005f270 0x7f7220061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:30.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.974+0000 7f723bfff700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f722005f270 0x7f7220061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f723c154270 tx=0x7f7228008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:30.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:30.976+0000 7f7238ff9700 1 -- 172.21.15.33:0/2151252997 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f723004d160 con 0x7f723c0aa2d0 2023-11-06T21:00:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.158+0000 7f7242dff700 1 -- 172.21.15.33:0/2151252997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f723c06c6c0 con 0x7f723c0aa2d0 2023-11-06T21:00:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.158+0000 7f7238ff9700 1 -- 172.21.15.33:0/2151252997 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) v1 ==== 84+0+143 (secure 0 0 0) 0x7f7230020070 con 0x7f723c0aa2d0 2023-11-06T21:00:31.159 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:31.159 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T21:00:31.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:31.160 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T21:00:31.160 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T21:00:31.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:00:31.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:31.161 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T21:00:31.161 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T21:00:31.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:31.161 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T21:00:31.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 -- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f722005f270 msgr2=0x7f7220061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:31.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f722005f270 0x7f7220061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f723c154270 tx=0x7f7228008040).stop 2023-11-06T21:00:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 -- 172.21.15.33:0/2151252997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0aa2d0 msgr2=0x7f723c14cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0aa2d0 0x7f723c14cbc0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f723c06bc90 tx=0x7f7230015000).stop 2023-11-06T21:00:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 -- 172.21.15.33:0/2151252997 shutdown_connections 2023-11-06T21:00:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f722005f270 0x7f7220061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f723c0c6460 0x7f723c0ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f723c0aa2d0 0x7f723c14cbc0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 --2- 172.21.15.33:0/2151252997 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f723c0c1840 0x7f723c0ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 -- 172.21.15.33:0/2151252997 >> 172.21.15.33:0/2151252997 conn(0x7f723c05ec80 msgr2=0x7f723c14d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 -- 172.21.15.33:0/2151252997 shutdown_connections 2023-11-06T21:00:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.160+0000 7f7242dff700 1 -- 172.21.15.33:0/2151252997 wait complete. 2023-11-06T21:00:31.178 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T21:00:31.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T21:00:31.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.379+0000 7fb1ed1ff700 1 Processor -- start 2023-11-06T21:00:31.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.379+0000 7fb1ed1ff700 1 -- start start 2023-11-06T21:00:31.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1ed1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e80c8ee0 0x7fb1e80ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:31.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1ed1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb1e81542b0 0x7fb1e80c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:31.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1ed1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e80c9f80 con 0x7fb1e8153ed0 2023-11-06T21:00:31.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1ed1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e80c6000 con 0x7fb1e81542b0 2023-11-06T21:00:31.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1ed1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e80c9370 con 0x7fb1e80c8ee0 2023-11-06T21:00:31.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1e759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb1e81542b0 0x7fb1e80c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:31.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1e759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb1e81542b0 0x7fb1e80c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36642/0 (socket says 172.21.15.33:36642) 2023-11-06T21:00:31.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1e759e700 1 -- 172.21.15.33:0/1561412305 learned_addr learned my addr 172.21.15.33:0/1561412305 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:31.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1e659c700 1 --2- 172.21.15.33:0/1561412305 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e80c8ee0 0x7fb1e80ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:31.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1e759e700 1 -- 172.21.15.33:0/1561412305 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e80c8ee0 msgr2=0x7fb1e80ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:31.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.381+0000 7fb1e759e700 1 --2- 172.21.15.33:0/1561412305 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e80c8ee0 0x7fb1e80ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:31.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1e759e700 1 -- 172.21.15.33:0/1561412305 >> v1:172.21.15.69:6789/0 conn(0x7fb1e8153ed0 legacy=0x7fb1e8158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:31.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1e759e700 1 -- 172.21.15.33:0/1561412305 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e80c9650 con 0x7fb1e81542b0 2023-11-06T21:00:31.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1e759e700 1 --2- 172.21.15.33:0/1561412305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e81542b0 0x7fb1e80c76f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7fb1e0009f20 tx=0x7fb1e0009840).ready entity=mon.0 client_cookie=cd5678249a7ccb2a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:31.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1e5d9b700 1 -- 172.21.15.33:0/1561412305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000e070 con 0x7fb1e81542b0 2023-11-06T21:00:31.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1e5d9b700 1 -- 172.21.15.33:0/1561412305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e000b9d0 con 0x7fb1e81542b0 2023-11-06T21:00:31.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1e5d9b700 1 -- 172.21.15.33:0/1561412305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000bd50 con 0x7fb1e81542b0 2023-11-06T21:00:31.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/1561412305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e81542b0 msgr2=0x7fb1e80c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:31.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1ed1ff700 1 --2- 172.21.15.33:0/1561412305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e81542b0 0x7fb1e80c76f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7fb1e0009f20 tx=0x7fb1e0009840).stop 2023-11-06T21:00:31.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/1561412305 shutdown_connections 2023-11-06T21:00:31.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1ed1ff700 1 --2- 172.21.15.33:0/1561412305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e81542b0 0x7fb1e80c76f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:31.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1ed1ff700 1 --2- 172.21.15.33:0/1561412305 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e80c8ee0 0x7fb1e80ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:31.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.382+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/1561412305 >> 172.21.15.33:0/1561412305 conn(0x7fb1e805e1b0 msgr2=0x7fb1e805e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:31.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.383+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/1561412305 shutdown_connections 2023-11-06T21:00:31.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.383+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/1561412305 wait complete. 2023-11-06T21:00:31.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.384+0000 7fb1ed1ff700 1 Processor -- start 2023-11-06T21:00:31.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.384+0000 7fb1ed1ff700 1 -- start start 2023-11-06T21:00:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.385+0000 7fb1ed1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e8153ed0 0x7fb1e81418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.385+0000 7fb1ed1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1e81542b0 0x7fb1e81579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.385+0000 7fb1ed1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e8157ed0 0x7fb1e80ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.385+0000 7fb1ed1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e81643f0 con 0x7fb1e8153ed0 2023-11-06T21:00:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.385+0000 7fb1ed1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e81586a0 con 0x7fb1e81542b0 2023-11-06T21:00:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.385+0000 7fb1ed1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e80af5f0 con 0x7fb1e8157ed0 2023-11-06T21:00:31.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.385+0000 7fb1e6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e8153ed0 0x7fb1e81418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:31.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.385+0000 7fb1e6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e8153ed0 0x7fb1e81418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36648/0 (socket says 172.21.15.33:36648) 2023-11-06T21:00:31.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.385+0000 7fb1e6d9d700 1 -- 172.21.15.33:0/4243310227 learned_addr learned my addr 172.21.15.33:0/4243310227 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:31.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.385+0000 7fb1e659c700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1e81542b0 0x7fb1e81579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:31.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1e759e700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e8157ed0 0x7fb1e80ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1e6d9d700 1 -- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e8157ed0 msgr2=0x7fb1e80ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1e6d9d700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e8157ed0 0x7fb1e80ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1e6d9d700 1 -- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1e81542b0 msgr2=0x7fb1e81579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1e6d9d700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1e81542b0 0x7fb1e81579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1e6d9d700 1 -- 172.21.15.33:0/4243310227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e0009ae0 con 0x7fb1e8153ed0 2023-11-06T21:00:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1e759e700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e8157ed0 0x7fb1e80ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1e6d9d700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e8153ed0 0x7fb1e81418f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7fb1d800c4f0 tx=0x7fb1d8007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1ceffd700 1 -- 172.21.15.33:0/4243310227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8007020 con 0x7fb1e8153ed0 2023-11-06T21:00:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1ceffd700 1 -- 172.21.15.33:0/4243310227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d8001e90 con 0x7fb1e8153ed0 2023-11-06T21:00:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1ceffd700 1 -- 172.21.15.33:0/4243310227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80073b0 con 0x7fb1e8153ed0 2023-11-06T21:00:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/4243310227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e8141de0 con 0x7fb1e8153ed0 2023-11-06T21:00:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.386+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/4243310227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e80aefa0 con 0x7fb1e8153ed0 2023-11-06T21:00:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.388+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/4243310227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e806c6c0 con 0x7fb1e8153ed0 2023-11-06T21:00:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.388+0000 7fb1ceffd700 1 -- 172.21.15.33:0/4243310227 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb1d80068b0 con 0x7fb1e8153ed0 2023-11-06T21:00:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.389+0000 7fb1ceffd700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb1d405f270 0x7fb1d4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.389+0000 7fb1ceffd700 1 -- 172.21.15.33:0/4243310227 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fb1d8079930 con 0x7fb1e8153ed0 2023-11-06T21:00:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.389+0000 7fb1e659c700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb1d405f270 0x7fb1d4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.390+0000 7fb1e659c700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb1d405f270 0x7fb1d4061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb1e80b0ff0 tx=0x7fb1dc009060).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.393+0000 7fb1ceffd700 1 -- 172.21.15.33:0/4243310227 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb1d804add0 con 0x7fb1e8153ed0 2023-11-06T21:00:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:31.575+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/4243310227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fb1e80c90e0 con 0x7fb1e8153ed0 2023-11-06T21:00:32.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.467+0000 7fb1ceffd700 1 -- 172.21.15.33:0/4243310227 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) v1 ==== 152+0+0 (secure 0 0 0) 0x7fb1d804e3f0 con 0x7fb1e8153ed0 2023-11-06T21:00:32.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.471+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb1d405f270 msgr2=0x7fb1d4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:32.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.471+0000 7fb1ed1ff700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb1d405f270 0x7fb1d4061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb1e80b0ff0 tx=0x7fb1dc009060).stop 2023-11-06T21:00:32.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.471+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/4243310227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e8153ed0 msgr2=0x7fb1e81418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:32.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.471+0000 7fb1ed1ff700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e8153ed0 0x7fb1e81418f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7fb1d800c4f0 tx=0x7fb1d8007e00).stop 2023-11-06T21:00:32.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.472+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/4243310227 shutdown_connections 2023-11-06T21:00:32.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.472+0000 7fb1ed1ff700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb1d405f270 0x7fb1d4061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:32.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.472+0000 7fb1ed1ff700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1e8157ed0 0x7fb1e80ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:32.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.472+0000 7fb1ed1ff700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e8153ed0 0x7fb1e81418f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:32.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.472+0000 7fb1ed1ff700 1 --2- 172.21.15.33:0/4243310227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1e81542b0 0x7fb1e81579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:32.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.472+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/4243310227 >> 172.21.15.33:0/4243310227 conn(0x7fb1e805e1b0 msgr2=0x7fb1e80ab5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:32.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.473+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/4243310227 shutdown_connections 2023-11-06T21:00:32.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.473+0000 7fb1ed1ff700 1 -- 172.21.15.33:0/4243310227 wait complete. 2023-11-06T21:00:32.480 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T21:00:32.502 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T21:00:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.708+0000 7f32689ff700 1 Processor -- start 2023-11-06T21:00:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.708+0000 7f32689ff700 1 -- start start 2023-11-06T21:00:32.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.709+0000 7f32689ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3264154330 0x7f32640c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:32.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.709+0000 7f32689ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3264154710 0x7f32640c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:32.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.709+0000 7f32689ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32640c6b60 con 0x7f32640c7ff0 2023-11-06T21:00:32.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.709+0000 7f32689ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32640c5cd0 con 0x7f3264154710 2023-11-06T21:00:32.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.709+0000 7f32689ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32640c5f50 con 0x7f3264154330 2023-11-06T21:00:32.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.709+0000 7f3261d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3264154710 0x7f32640c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:32.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.709+0000 7f3261d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3264154710 0x7f32640c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36650/0 (socket says 172.21.15.33:36650) 2023-11-06T21:00:32.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.709+0000 7f3261d9b700 1 -- 172.21.15.33:0/3316594597 learned_addr learned my addr 172.21.15.33:0/3316594597 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:32.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.710+0000 7f326259c700 1 --2- 172.21.15.33:0/3316594597 >> v2:172.21.15.69:3300/0 conn(0x7f3264154330 0x7f32640c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:32.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.710+0000 7f3261d9b700 1 -- 172.21.15.33:0/3316594597 >> v2:172.21.15.69:3300/0 conn(0x7f3264154330 msgr2=0x7f32640c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:32.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.710+0000 7f3261d9b700 1 --2- 172.21.15.33:0/3316594597 >> v2:172.21.15.69:3300/0 conn(0x7f3264154330 0x7f32640c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:32.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.710+0000 7f3261d9b700 1 -- 172.21.15.33:0/3316594597 >> v1:172.21.15.69:6789/0 conn(0x7f32640c7ff0 legacy=0x7f32640646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:32.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.710+0000 7f3261d9b700 1 -- 172.21.15.33:0/3316594597 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32640c6230 con 0x7f3264154710 2023-11-06T21:00:32.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.710+0000 7f3261d9b700 1 --2- 172.21.15.33:0/3316594597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3264154710 0x7f32640c6ff0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f32640605d0 tx=0x7f32580090f0).ready entity=mon.0 client_cookie=d9c9dc4f4019cd5e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:32.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.710+0000 7f326159a700 1 -- 172.21.15.33:0/3316594597 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800b5a0 con 0x7f3264154710 2023-11-06T21:00:32.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.710+0000 7f326159a700 1 -- 172.21.15.33:0/3316594597 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325800b700 con 0x7f3264154710 2023-11-06T21:00:32.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.710+0000 7f326159a700 1 -- 172.21.15.33:0/3316594597 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800ba80 con 0x7f3264154710 2023-11-06T21:00:32.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.711+0000 7f32689ff700 1 -- 172.21.15.33:0/3316594597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3264154710 msgr2=0x7f32640c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:32.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.711+0000 7f32689ff700 1 --2- 172.21.15.33:0/3316594597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3264154710 0x7f32640c6ff0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f32640605d0 tx=0x7f32580090f0).stop 2023-11-06T21:00:32.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.711+0000 7f32689ff700 1 -- 172.21.15.33:0/3316594597 shutdown_connections 2023-11-06T21:00:32.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.711+0000 7f32689ff700 1 --2- 172.21.15.33:0/3316594597 >> v2:172.21.15.69:3300/0 conn(0x7f3264154330 0x7f32640c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:32.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.711+0000 7f32689ff700 1 --2- 172.21.15.33:0/3316594597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3264154710 0x7f32640c6ff0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:32.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.711+0000 7f32689ff700 1 -- 172.21.15.33:0/3316594597 >> 172.21.15.33:0/3316594597 conn(0x7f32640cb910 msgr2=0x7f32640c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:32.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.711+0000 7f32689ff700 1 -- 172.21.15.33:0/3316594597 shutdown_connections 2023-11-06T21:00:32.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.711+0000 7f32689ff700 1 -- 172.21.15.33:0/3316594597 wait complete. 2023-11-06T21:00:32.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.712+0000 7f32689ff700 1 Processor -- start 2023-11-06T21:00:32.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.713+0000 7f32689ff700 1 -- start start 2023-11-06T21:00:32.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.713+0000 7f32689ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640c7ff0 0x7f3264151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:32.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.713+0000 7f32689ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264154330 0x7f3264156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:32.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.713+0000 7f32689ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3264154710 0x7f3264164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:32.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.713+0000 7f32689ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32640ade70 con 0x7f32640c7ff0 2023-11-06T21:00:32.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.713+0000 7f32689ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32640ae110 con 0x7f3264154330 2023-11-06T21:00:32.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.713+0000 7f32689ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32640ae3f0 con 0x7f3264154710 2023-11-06T21:00:32.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f326259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640c7ff0 0x7f3264151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:32.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f326259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640c7ff0 0x7f3264151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36652/0 (socket says 172.21.15.33:36652) 2023-11-06T21:00:32.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f326259c700 1 -- 172.21.15.33:0/1599516347 learned_addr learned my addr 172.21.15.33:0/1599516347 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:32.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f3261d9b700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264154330 0x7f3264156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f3262d9d700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3264154710 0x7f3264164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f326259c700 1 -- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3264154710 msgr2=0x7f3264164bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f326259c700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3264154710 0x7f3264164bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f326259c700 1 -- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264154330 msgr2=0x7f3264156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f326259c700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264154330 0x7f3264156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f326259c700 1 -- 172.21.15.33:0/1599516347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3258009370 con 0x7f32640c7ff0 2023-11-06T21:00:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f3262d9d700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3264154710 0x7f3264164bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.714+0000 7f326259c700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640c7ff0 0x7f3264151cc0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f325400d890 tx=0x7f3254008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.715+0000 7f32527fc700 1 -- 172.21.15.33:0/1599516347 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540125f0 con 0x7f32640c7ff0 2023-11-06T21:00:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.715+0000 7f32527fc700 1 -- 172.21.15.33:0/1599516347 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3254012750 con 0x7f32640c7ff0 2023-11-06T21:00:32.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.715+0000 7f32527fc700 1 -- 172.21.15.33:0/1599516347 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254012ad0 con 0x7f32640c7ff0 2023-11-06T21:00:32.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.715+0000 7f32689ff700 1 -- 172.21.15.33:0/1599516347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32640ae6d0 con 0x7f32640c7ff0 2023-11-06T21:00:32.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.715+0000 7f32689ff700 1 -- 172.21.15.33:0/1599516347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32640b09e0 con 0x7f32640c7ff0 2023-11-06T21:00:32.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.716+0000 7f32689ff700 1 -- 172.21.15.33:0/1599516347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32640aeb70 con 0x7f32640c7ff0 2023-11-06T21:00:32.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.717+0000 7f32527fc700 1 -- 172.21.15.33:0/1599516347 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3254013480 con 0x7f32640c7ff0 2023-11-06T21:00:32.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.719+0000 7f32527fc700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f324405f270 0x7f3244061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:32.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.719+0000 7f32527fc700 1 -- 172.21.15.33:0/1599516347 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f3254079f90 con 0x7f32640c7ff0 2023-11-06T21:00:32.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.719+0000 7f3261d9b700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f324405f270 0x7f3244061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:32.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.720+0000 7f3261d9b700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f324405f270 0x7f3244061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f3264154b70 tx=0x7f3258015000).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:32.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.724+0000 7f32527fc700 1 -- 172.21.15.33:0/1599516347 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f325404b1a0 con 0x7f32640c7ff0 2023-11-06T21:00:32.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:32.903+0000 7f32689ff700 1 -- 172.21.15.33:0/1599516347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f32640b0bd0 con 0x7f32640c7ff0 2023-11-06T21:00:33.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.474+0000 7f32527fc700 1 -- 172.21.15.33:0/1599516347 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) v1 ==== 170+0+0 (secure 0 0 0) 0x7f325404e7c0 con 0x7f32640c7ff0 2023-11-06T21:00:33.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.478+0000 7f32689ff700 1 -- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f324405f270 msgr2=0x7f3244061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:33.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.478+0000 7f32689ff700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f324405f270 0x7f3244061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f3264154b70 tx=0x7f3258015000).stop 2023-11-06T21:00:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.478+0000 7f32689ff700 1 -- 172.21.15.33:0/1599516347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640c7ff0 msgr2=0x7f3264151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.478+0000 7f32689ff700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640c7ff0 0x7f3264151cc0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f325400d890 tx=0x7f3254008040).stop 2023-11-06T21:00:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.478+0000 7f32689ff700 1 -- 172.21.15.33:0/1599516347 shutdown_connections 2023-11-06T21:00:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.478+0000 7f32689ff700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f324405f270 0x7f3244061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.478+0000 7f32689ff700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3264154710 0x7f3264164bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.478+0000 7f32689ff700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640c7ff0 0x7f3264151cc0 secure :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0x7f325400d890 tx=0x7f3254008040).stop 2023-11-06T21:00:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.478+0000 7f32689ff700 1 --2- 172.21.15.33:0/1599516347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3264154330 0x7f3264156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.478+0000 7f32689ff700 1 -- 172.21.15.33:0/1599516347 >> 172.21.15.33:0/1599516347 conn(0x7f32640cb910 msgr2=0x7f3264152c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.479+0000 7f32689ff700 1 -- 172.21.15.33:0/1599516347 shutdown_connections 2023-11-06T21:00:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.479+0000 7f32689ff700 1 -- 172.21.15.33:0/1599516347 wait complete. 2023-11-06T21:00:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T21:00:33.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T21:00:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.721+0000 7f9d633ff700 1 Processor -- start 2023-11-06T21:00:33.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.721+0000 7f9d633ff700 1 -- start start 2023-11-06T21:00:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.722+0000 7f9d633ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9d5c154330 0x7f9d5c0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.722+0000 7f9d633ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9d5c154710 0x7f9d5c0c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.722+0000 7f9d633ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c0c7ff0 0x7f9d5c0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.722+0000 7f9d633ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5c0c6b60 con 0x7f9d5c154330 2023-11-06T21:00:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.722+0000 7f9d633ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5c0c5cd0 con 0x7f9d5c154710 2023-11-06T21:00:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.722+0000 7f9d633ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c0c5f50 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.722+0000 7f9d6119b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9d5c154330 0x7f9d5c0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d6119b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9d5c154330 0x7f9d5c0c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36668/0 (socket says 172.21.15.33:36668) 2023-11-06T21:00:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d6119b700 1 -- 172.21.15.33:0/2291162218 learned_addr learned my addr 172.21.15.33:0/2291162218 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d6099a700 1 --2- 172.21.15.33:0/2291162218 >> v2:172.21.15.69:3300/0 conn(0x7f9d5c154710 0x7f9d5c0c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d6199c700 1 --2- 172.21.15.33:0/2291162218 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c0c7ff0 0x7f9d5c0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:33.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d6119b700 1 -- 172.21.15.33:0/2291162218 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c0c7ff0 msgr2=0x7f9d5c0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:33.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d6119b700 1 --2- 172.21.15.33:0/2291162218 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c0c7ff0 0x7f9d5c0646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d6119b700 1 -- 172.21.15.33:0/2291162218 >> v2:172.21.15.69:3300/0 conn(0x7f9d5c154710 msgr2=0x7f9d5c0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:33.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d6119b700 1 --2- 172.21.15.33:0/2291162218 >> v2:172.21.15.69:3300/0 conn(0x7f9d5c154710 0x7f9d5c0c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d6119b700 1 -- 172.21.15.33:0/2291162218 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c0c6230 con 0x7f9d5c154330 2023-11-06T21:00:33.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d6119b700 1 --2- 172.21.15.33:0/2291162218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c154330 0x7f9d5c0c57e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f9d5c0c8d10 tx=0x7f9d580090f0).ready entity=mon.0 client_cookie=1a4b8aa541c920f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:33.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d53fff700 1 -- 172.21.15.33:0/2291162218 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800b600 con 0x7f9d5c154330 2023-11-06T21:00:33.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d53fff700 1 -- 172.21.15.33:0/2291162218 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5800b760 con 0x7f9d5c154330 2023-11-06T21:00:33.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.723+0000 7f9d53fff700 1 -- 172.21.15.33:0/2291162218 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800bae0 con 0x7f9d5c154330 2023-11-06T21:00:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.724+0000 7f9d633ff700 1 -- 172.21.15.33:0/2291162218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c154330 msgr2=0x7f9d5c0c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.724+0000 7f9d633ff700 1 --2- 172.21.15.33:0/2291162218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c154330 0x7f9d5c0c57e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f9d5c0c8d10 tx=0x7f9d580090f0).stop 2023-11-06T21:00:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.724+0000 7f9d633ff700 1 -- 172.21.15.33:0/2291162218 shutdown_connections 2023-11-06T21:00:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.724+0000 7f9d633ff700 1 --2- 172.21.15.33:0/2291162218 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c0c7ff0 0x7f9d5c0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.724+0000 7f9d633ff700 1 --2- 172.21.15.33:0/2291162218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c154330 0x7f9d5c0c57e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.724+0000 7f9d633ff700 1 --2- 172.21.15.33:0/2291162218 >> v2:172.21.15.69:3300/0 conn(0x7f9d5c154710 0x7f9d5c0c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.724+0000 7f9d633ff700 1 -- 172.21.15.33:0/2291162218 >> 172.21.15.33:0/2291162218 conn(0x7f9d5c0cb910 msgr2=0x7f9d5c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.724+0000 7f9d633ff700 1 -- 172.21.15.33:0/2291162218 shutdown_connections 2023-11-06T21:00:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.724+0000 7f9d633ff700 1 -- 172.21.15.33:0/2291162218 wait complete. 2023-11-06T21:00:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.726+0000 7f9d633ff700 1 Processor -- start 2023-11-06T21:00:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.726+0000 7f9d633ff700 1 -- start start 2023-11-06T21:00:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d633ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c0c7ff0 0x7f9d5c0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d633ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c154330 0x7f9d5c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d633ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d5c154710 0x7f9d5c149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d633ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c0aefa0 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d633ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c0ae390 con 0x7f9d5c154710 2023-11-06T21:00:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d633ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c0ae670 con 0x7f9d5c154330 2023-11-06T21:00:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c0c7ff0 0x7f9d5c0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c0c7ff0 0x7f9d5c0addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36682/0 (socket says 172.21.15.33:36682) 2023-11-06T21:00:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6119b700 1 -- 172.21.15.33:0/3005070559 learned_addr learned my addr 172.21.15.33:0/3005070559 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6099a700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c154330 0x7f9d5c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6199c700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d5c154710 0x7f9d5c149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6119b700 1 -- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c154330 msgr2=0x7f9d5c1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6119b700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c154330 0x7f9d5c1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6119b700 1 -- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d5c154710 msgr2=0x7f9d5c149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6119b700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d5c154710 0x7f9d5c149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6119b700 1 -- 172.21.15.33:0/3005070559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d58009370 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6099a700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c154330 0x7f9d5c1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.727+0000 7f9d6199c700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d5c154710 0x7f9d5c149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:33.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.728+0000 7f9d6119b700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c0c7ff0 0x7f9d5c0addd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f9d58009130 tx=0x7f9d58017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:33.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.728+0000 7f9d517fa700 1 -- 172.21.15.33:0/3005070559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800b8d0 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.728+0000 7f9d517fa700 1 -- 172.21.15.33:0/3005070559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5800ba50 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.728+0000 7f9d517fa700 1 -- 172.21.15.33:0/3005070559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5801c670 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.728+0000 7f9d633ff700 1 -- 172.21.15.33:0/3005070559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5c0ae950 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.728+0000 7f9d633ff700 1 -- 172.21.15.33:0/3005070559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c157cb0 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.729+0000 7f9d633ff700 1 -- 172.21.15.33:0/3005070559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5c0af280 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.730+0000 7f9d517fa700 1 -- 172.21.15.33:0/3005070559 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9d5801c9e0 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.731+0000 7f9d517fa700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4005f270 0x7f9d40061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.731+0000 7f9d517fa700 1 -- 172.21.15.33:0/3005070559 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f9d5807c1b0 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.732+0000 7f9d6099a700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4005f270 0x7f9d40061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:33.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.732+0000 7f9d6099a700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4005f270 0x7f9d40061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9d4c006fd0 tx=0x7f9d4c008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:33.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.736+0000 7f9d517fa700 1 -- 172.21.15.33:0/3005070559 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9d58050cd0 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.960+0000 7f9d633ff700 1 -- 172.21.15.33:0/3005070559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9d5c0830b0 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.968+0000 7f9d517fa700 1 -- 172.21.15.33:0/3005070559 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v41) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9d5802a0c0 con 0x7f9d5c0c7ff0 2023-11-06T21:00:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.974+0000 7f9d633ff700 1 -- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4005f270 msgr2=0x7f9d40061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:33.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.974+0000 7f9d633ff700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4005f270 0x7f9d40061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9d4c006fd0 tx=0x7f9d4c008040).stop 2023-11-06T21:00:33.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.974+0000 7f9d633ff700 1 -- 172.21.15.33:0/3005070559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c0c7ff0 msgr2=0x7f9d5c0addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:33.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.974+0000 7f9d633ff700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c0c7ff0 0x7f9d5c0addd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f9d58009130 tx=0x7f9d58017000).stop 2023-11-06T21:00:33.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.975+0000 7f9d633ff700 1 -- 172.21.15.33:0/3005070559 shutdown_connections 2023-11-06T21:00:33.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.975+0000 7f9d633ff700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9d4005f270 0x7f9d40061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.975+0000 7f9d633ff700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d5c154710 0x7f9d5c149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.975+0000 7f9d633ff700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d5c0c7ff0 0x7f9d5c0addd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.975+0000 7f9d633ff700 1 --2- 172.21.15.33:0/3005070559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9d5c154330 0x7f9d5c1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:33.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.975+0000 7f9d633ff700 1 -- 172.21.15.33:0/3005070559 >> 172.21.15.33:0/3005070559 conn(0x7f9d5c0cb910 msgr2=0x7f9d5c0aa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:33.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.975+0000 7f9d633ff700 1 -- 172.21.15.33:0/3005070559 shutdown_connections 2023-11-06T21:00:33.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:33.975+0000 7f9d633ff700 1 -- 172.21.15.33:0/3005070559 wait complete. 2023-11-06T21:00:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 13 and data pool 14 2023-11-06T21:00:34.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:00:34.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.200+0000 7f2f0dfff700 1 Processor -- start 2023-11-06T21:00:34.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.200+0000 7f2f0dfff700 1 -- start start 2023-11-06T21:00:34.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.202+0000 7f2f0dfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080be280 0x7f2f0815f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.202+0000 7f2f0dfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f081553b0 con 0x7f2f080c5240 2023-11-06T21:00:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.202+0000 7f2f0dfff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f080c45c0 con 0x7f2f080c4e60 2023-11-06T21:00:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.202+0000 7f2f0dfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f08066c50 con 0x7f2f080be280 2023-11-06T21:00:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.202+0000 7f2f06ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2f080c5240 0x7f2f080bdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48304/0 (socket says 172.21.15.33:48304) 2023-11-06T21:00:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.202+0000 7f2f06ffd700 1 -- 172.21.15.33:0/2674086139 learned_addr learned my addr 172.21.15.33:0/2674086139 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.202+0000 7f2f07fff700 1 --2- 172.21.15.33:0/2674086139 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080be280 0x7f2f0815f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.202+0000 7f2f067fc700 1 -- 172.21.15.33:0/2674086139 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590612064 0 0) 0x7f2f081553b0 con 0x7f2f080c5240 2023-11-06T21:00:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.202+0000 7f2f067fc700 1 -- 172.21.15.33:0/2674086139 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ef00033f0 con 0x7f2f080c5240 2023-11-06T21:00:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.203+0000 7f2f067fc700 1 -- 172.21.15.33:0/2674086139 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586825855 0 0) 0x7f2f080c45c0 con 0x7f2f080c4e60 2023-11-06T21:00:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.203+0000 7f2f067fc700 1 -- 172.21.15.33:0/2674086139 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f081553b0 con 0x7f2f080c4e60 2023-11-06T21:00:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.203+0000 7f2f07fff700 1 -- 172.21.15.33:0/2674086139 >> v1:172.21.15.69:6789/0 conn(0x7f2f080c4e60 legacy=0x7f2f08066540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.203+0000 7f2f07fff700 1 -- 172.21.15.33:0/2674086139 >> v1:172.21.15.33:6789/0 conn(0x7f2f080c5240 legacy=0x7f2f080bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.203+0000 7f2f067fc700 1 -- 172.21.15.33:0/2674086139 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3402508289 0 0) 0x7f2ef00033f0 con 0x7f2f080c5240 2023-11-06T21:00:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.203+0000 7f2f07fff700 1 -- 172.21.15.33:0/2674086139 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08154ac0 con 0x7f2f080be280 2023-11-06T21:00:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.203+0000 7f2f07fff700 1 --2- 172.21.15.33:0/2674086139 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080be280 0x7f2f0815f110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f2f00009e90 tx=0x7f2f00009780).ready entity=mon.2 client_cookie=d8e0f8184fffaf89 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.203+0000 7f2f067fc700 1 -- 172.21.15.33:0/2674086139 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000e070 con 0x7f2f080be280 2023-11-06T21:00:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.203+0000 7f2f067fc700 1 -- 172.21.15.33:0/2674086139 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0000b7a0 con 0x7f2f080be280 2023-11-06T21:00:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.203+0000 7f2f067fc700 1 -- 172.21.15.33:0/2674086139 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc001470 con 0x7f2f080be280 2023-11-06T21:00:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.204+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2674086139 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080be280 msgr2=0x7f2f0815f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.204+0000 7f2f0dfff700 1 --2- 172.21.15.33:0/2674086139 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080be280 0x7f2f0815f110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f2f00009e90 tx=0x7f2f00009780).stop 2023-11-06T21:00:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.204+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2674086139 shutdown_connections 2023-11-06T21:00:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.204+0000 7f2f0dfff700 1 --2- 172.21.15.33:0/2674086139 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080be280 0x7f2f0815f110 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.204+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2674086139 >> 172.21.15.33:0/2674086139 conn(0x7f2f080c8c10 msgr2=0x7f2f080c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.204+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2674086139 shutdown_connections 2023-11-06T21:00:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.204+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2674086139 wait complete. 2023-11-06T21:00:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.205+0000 7f2f0dfff700 1 Processor -- start 2023-11-06T21:00:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.206+0000 7f2f0dfff700 1 -- start start 2023-11-06T21:00:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.206+0000 7f2f0dfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f080be280 0x7f2f08152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.206+0000 7f2f0dfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080c4e60 0x7f2f080aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.206+0000 7f2f0dfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f080c5240 0x7f2f080aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.206+0000 7f2f0dfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080a9100 con 0x7f2f080be280 2023-11-06T21:00:34.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.206+0000 7f2f0dfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080ac1c0 con 0x7f2f080c5240 2023-11-06T21:00:34.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.206+0000 7f2f0dfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080ac4a0 con 0x7f2f080c4e60 2023-11-06T21:00:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f080be280 0x7f2f08152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f080be280 0x7f2f08152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36686/0 (socket says 172.21.15.33:36686) 2023-11-06T21:00:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f077fe700 1 -- 172.21.15.33:0/2319957053 learned_addr learned my addr 172.21.15.33:0/2319957053 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f06ffd700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080c4e60 0x7f2f080aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f07fff700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f080c5240 0x7f2f080aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f077fe700 1 -- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080c4e60 msgr2=0x7f2f080aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f077fe700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080c4e60 0x7f2f080aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f077fe700 1 -- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f080c5240 msgr2=0x7f2f080aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f077fe700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f080c5240 0x7f2f080aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f077fe700 1 -- 172.21.15.33:0/2319957053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f00009a20 con 0x7f2f080be280 2023-11-06T21:00:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.207+0000 7f2f077fe700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f080be280 0x7f2f08152c00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f2ef800c490 tx=0x7f2ef8007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.208+0000 7f2eef7fe700 1 -- 172.21.15.33:0/2319957053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800ce40 con 0x7f2f080be280 2023-11-06T21:00:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.208+0000 7f2eef7fe700 1 -- 172.21.15.33:0/2319957053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef800f470 con 0x7f2f080be280 2023-11-06T21:00:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.208+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2319957053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f080ac780 con 0x7f2f080be280 2023-11-06T21:00:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.208+0000 7f2eef7fe700 1 -- 172.21.15.33:0/2319957053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800f830 con 0x7f2f080be280 2023-11-06T21:00:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.208+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2319957053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f08139f20 con 0x7f2f080be280 2023-11-06T21:00:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.209+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2319957053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f080acca0 con 0x7f2f080be280 2023-11-06T21:00:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.210+0000 7f2eef7fe700 1 -- 172.21.15.33:0/2319957053 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2ef800fc20 con 0x7f2f080be280 2023-11-06T21:00:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.211+0000 7f2eef7fe700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2ef405f270 0x7f2ef4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.211+0000 7f2eef7fe700 1 -- 172.21.15.33:0/2319957053 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2ef8079800 con 0x7f2f080be280 2023-11-06T21:00:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.211+0000 7f2f06ffd700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2ef405f270 0x7f2ef4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.212+0000 7f2f06ffd700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2ef405f270 0x7f2ef4061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2efc0011e0 tx=0x7f2efc001620).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:34.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.215+0000 7f2eef7fe700 1 -- 172.21.15.33:0/2319957053 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2ef8048af0 con 0x7f2f080be280 2023-11-06T21:00:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.395+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2319957053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f08154d60 con 0x7f2f080be280 2023-11-06T21:00:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.396+0000 7f2eef7fe700 1 -- 172.21.15.33:0/2319957053 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17004 (secure 0 0 0) 0x7f2ef804a940 con 0x7f2f080be280 2023-11-06T21:00:34.397 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:34.397 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":113,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:00:33.962582+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:00:31.623152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T21:00:32.950772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:00:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2ef405f270 msgr2=0x7f2ef4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2ef405f270 0x7f2ef4061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2efc0011e0 tx=0x7f2efc001620).stop 2023-11-06T21:00:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2319957053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f080be280 msgr2=0x7f2f08152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f080be280 0x7f2f08152c00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f2ef800c490 tx=0x7f2ef8007c40).stop 2023-11-06T21:00:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2319957053 shutdown_connections 2023-11-06T21:00:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2ef405f270 0x7f2ef4061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f080c5240 0x7f2f080aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f080be280 0x7f2f08152c00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 --2- 172.21.15.33:0/2319957053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2f080c4e60 0x7f2f080aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2319957053 >> 172.21.15.33:0/2319957053 conn(0x7f2f080c8c10 msgr2=0x7f2f08067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2319957053 shutdown_connections 2023-11-06T21:00:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.398+0000 7f2f0dfff700 1 -- 172.21.15.33:0/2319957053 wait complete. 2023-11-06T21:00:34.416 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:00:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.614+0000 7f26389ff700 1 Processor -- start 2023-11-06T21:00:34.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.614+0000 7f26389ff700 1 -- start start 2023-11-06T21:00:34.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.615+0000 7f26389ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26340c1840 0x7f26340a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.615+0000 7f26389ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f26340aa2d0 0x7f26340c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.615+0000 7f26389ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26341554a0 con 0x7f26340c6240 2023-11-06T21:00:34.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.615+0000 7f26389ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2634154960 con 0x7f26340aa2d0 2023-11-06T21:00:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.615+0000 7f26389ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2634155930 con 0x7f26340c1840 2023-11-06T21:00:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f2632d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f26340c6240 0x7f2634157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48306/0 (socket says 172.21.15.33:48306) 2023-11-06T21:00:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f2632d9d700 1 -- 172.21.15.33:0/1980254472 learned_addr learned my addr 172.21.15.33:0/1980254472 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f2631d9b700 1 --2- 172.21.15.33:0/1980254472 >> v2:172.21.15.69:3300/0 conn(0x7f26340aa2d0 0x7f26340c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f263259c700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26340c1840 0x7f26340a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404172948 0 0) 0x7f26341554a0 con 0x7f26340c6240 2023-11-06T21:00:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f261c0033f0 con 0x7f26340c6240 2023-11-06T21:00:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3312079213 0 0) 0x7f261c0033f0 con 0x7f26340c6240 2023-11-06T21:00:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26340c1840 msgr2=0x7f26340a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f263159a700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26340c1840 0x7f26340a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 >> v2:172.21.15.69:3300/0 conn(0x7f26340aa2d0 msgr2=0x7f26340c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f263159a700 1 --2- 172.21.15.33:0/1980254472 >> v2:172.21.15.69:3300/0 conn(0x7f26340aa2d0 0x7f26340c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2634154b70 con 0x7f26340c6240 2023-11-06T21:00:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.616+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f262c0017f0 con 0x7f26340c6240 2023-11-06T21:00:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 >> v1:172.21.15.33:6789/0 conn(0x7f26340c6240 legacy=0x7f2634157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f263159a700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26340c1840 0x7f261c0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f263159a700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c003e10 0x7f261c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f263159a700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c004b80 0x7f261c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2634154960 con 0x7f261c003e10 2023-11-06T21:00:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2634155930 con 0x7f261c004b80 2023-11-06T21:00:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c00b8e0 con 0x7f26340c1840 2023-11-06T21:00:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f2631d9b700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c003e10 0x7f261c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f263259c700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26340c1840 0x7f261c0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f2632d9d700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c004b80 0x7f261c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f2631d9b700 1 -- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26340c1840 msgr2=0x7f261c0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f2631d9b700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26340c1840 0x7f261c0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f2631d9b700 1 -- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c004b80 msgr2=0x7f261c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f2631d9b700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c004b80 0x7f261c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f2631d9b700 1 -- 172.21.15.33:0/1980254472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c0033f0 con 0x7f261c003e10 2023-11-06T21:00:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.617+0000 7f2632d9d700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f261c004b80 0x7f261c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.618+0000 7f2631d9b700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c003e10 0x7f261c004690 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f26340adb40 tx=0x7f2628009060).ready entity=mon.0 client_cookie=babd890877ebc2b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.618+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628009e90 con 0x7f261c003e10 2023-11-06T21:00:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.618+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628009430 con 0x7f261c003e10 2023-11-06T21:00:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.618+0000 7f263159a700 1 -- 172.21.15.33:0/1980254472 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800e820 con 0x7f261c003e10 2023-11-06T21:00:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.618+0000 7f26389ff700 1 -- 172.21.15.33:0/1980254472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c003e10 msgr2=0x7f261c004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.618+0000 7f26389ff700 1 --2- 172.21.15.33:0/1980254472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f261c003e10 0x7f261c004690 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f26340adb40 tx=0x7f2628009060).stop 2023-11-06T21:00:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.618+0000 7f2632d9d700 1 -- 172.21.15.33:0/1980254472 reap_dead start 2023-11-06T21:00:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.618+0000 7f26389ff700 1 -- 172.21.15.33:0/1980254472 shutdown_connections 2023-11-06T21:00:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.618+0000 7f26389ff700 1 -- 172.21.15.33:0/1980254472 >> 172.21.15.33:0/1980254472 conn(0x7f263405ec80 msgr2=0x7f26340cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.618+0000 7f26389ff700 1 -- 172.21.15.33:0/1980254472 shutdown_connections 2023-11-06T21:00:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.619+0000 7f26389ff700 1 -- 172.21.15.33:0/1980254472 wait complete. 2023-11-06T21:00:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.621+0000 7f26389ff700 1 Processor -- start 2023-11-06T21:00:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.621+0000 7f26389ff700 1 -- start start 2023-11-06T21:00:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f26389ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f261c003e10 0x7f2634151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f26389ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2634155a70 0x7f26340aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f26389ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f263414ef90 0x7f26340aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f26389ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26340af3e0 con 0x7f263414ef90 2023-11-06T21:00:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f26389ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26340c3a00 con 0x7f2634155a70 2023-11-06T21:00:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f26389ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26340c2df0 con 0x7f261c003e10 2023-11-06T21:00:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f2632d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f263414ef90 0x7f26340aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f2632d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f263414ef90 0x7f26340aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36702/0 (socket says 172.21.15.33:36702) 2023-11-06T21:00:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f2632d9d700 1 -- 172.21.15.33:0/1787790372 learned_addr learned my addr 172.21.15.33:0/1787790372 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f263259c700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f261c003e10 0x7f2634151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.622+0000 7f2631d9b700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2634155a70 0x7f26340aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f2632d9d700 1 -- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f261c003e10 msgr2=0x7f2634151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f2632d9d700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f261c003e10 0x7f2634151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f2632d9d700 1 -- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2634155a70 msgr2=0x7f26340aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f2632d9d700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2634155a70 0x7f26340aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f2632d9d700 1 -- 172.21.15.33:0/1787790372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26340c30d0 con 0x7f263414ef90 2023-11-06T21:00:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f2632d9d700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f263414ef90 0x7f26340aeef0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f262c000f80 tx=0x7f2634155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f26227fc700 1 -- 172.21.15.33:0/1787790372 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c00ee70 con 0x7f263414ef90 2023-11-06T21:00:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f26227fc700 1 -- 172.21.15.33:0/1787790372 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c00c7b0 con 0x7f263414ef90 2023-11-06T21:00:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f26227fc700 1 -- 172.21.15.33:0/1787790372 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c00cb70 con 0x7f263414ef90 2023-11-06T21:00:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f26389ff700 1 -- 172.21.15.33:0/1787790372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26340c33b0 con 0x7f263414ef90 2023-11-06T21:00:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.623+0000 7f26389ff700 1 -- 172.21.15.33:0/1787790372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26340b0cf0 con 0x7f263414ef90 2023-11-06T21:00:34.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.624+0000 7f26389ff700 1 -- 172.21.15.33:0/1787790372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26340c3b90 con 0x7f263414ef90 2023-11-06T21:00:34.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.625+0000 7f26227fc700 1 -- 172.21.15.33:0/1787790372 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f262c014430 con 0x7f263414ef90 2023-11-06T21:00:34.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.626+0000 7f26227fc700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f261405f2a0 0x7f2614061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:34.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.626+0000 7f26227fc700 1 -- 172.21.15.33:0/1787790372 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f262c07a350 con 0x7f263414ef90 2023-11-06T21:00:34.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.626+0000 7f263259c700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f261405f2a0 0x7f2614061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.627+0000 7f263259c700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f261405f2a0 0x7f2614061760 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2624006d80 tx=0x7f2624009060).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.630+0000 7f26227fc700 1 -- 172.21.15.33:0/1787790372 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f262c04b380 con 0x7f263414ef90 2023-11-06T21:00:34.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.811+0000 7f26389ff700 1 -- 172.21.15.33:0/1787790372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f263406c6c0 con 0x7f263414ef90 2023-11-06T21:00:34.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.812+0000 7f26227fc700 1 -- 172.21.15.33:0/1787790372 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17004 (secure 0 0 0) 0x7f262c04e9a0 con 0x7f263414ef90 2023-11-06T21:00:34.812 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:34.813 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":113,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:00:33.962582+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:00:31.623152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T21:00:32.950772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:00:34.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 -- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f261405f2a0 msgr2=0x7f2614061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f261405f2a0 0x7f2614061760 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2624006d80 tx=0x7f2624009060).stop 2023-11-06T21:00:34.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 -- 172.21.15.33:0/1787790372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f263414ef90 msgr2=0x7f26340aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:34.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f263414ef90 0x7f26340aeef0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f262c000f80 tx=0x7f2634155410).stop 2023-11-06T21:00:34.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 -- 172.21.15.33:0/1787790372 shutdown_connections 2023-11-06T21:00:34.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f261405f2a0 0x7f2614061760 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f261c003e10 0x7f2634151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f263414ef90 0x7f26340aeef0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 --2- 172.21.15.33:0/1787790372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2634155a70 0x7f26340aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:34.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 -- 172.21.15.33:0/1787790372 >> 172.21.15.33:0/1787790372 conn(0x7f263405ec80 msgr2=0x7f263414f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:34.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 -- 172.21.15.33:0/1787790372 shutdown_connections 2023-11-06T21:00:34.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:34.814+0000 7f26389ff700 1 -- 172.21.15.33:0/1787790372 wait complete. 2023-11-06T21:00:34.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T21:00:35.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.034+0000 7feb5ebff700 1 Processor -- start 2023-11-06T21:00:35.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.034+0000 7feb5ebff700 1 -- start start 2023-11-06T21:00:35.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.035+0000 7feb5ebff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 0x7feb58157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:35.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.035+0000 7feb5ebff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb580c4b00 0x7feb580c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:35.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.035+0000 7feb5ebff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7feb58062710 0x7feb5813d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:35.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.035+0000 7feb5ebff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb580c4f60 con 0x7feb580c4b00 2023-11-06T21:00:35.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.035+0000 7feb5ebff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5805e490 con 0x7feb58062710 2023-11-06T21:00:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.035+0000 7feb5ebff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb58064bf0 con 0x7feb580c4720 2023-11-06T21:00:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.035+0000 7feb57fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb580c4b00 0x7feb580c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb57fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb580c4b00 0x7feb580c2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36706/0 (socket says 172.21.15.33:36706) 2023-11-06T21:00:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb57fff700 1 -- 172.21.15.33:0/3832132369 learned_addr learned my addr 172.21.15.33:0/3832132369 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb5c99b700 1 --2- 172.21.15.33:0/3832132369 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 0x7feb58157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:35.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb5d19c700 1 --2- 172.21.15.33:0/3832132369 >> v2:172.21.15.69:3300/0 conn(0x7feb58062710 0x7feb5813d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:35.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb57fff700 1 -- 172.21.15.33:0/3832132369 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 msgr2=0x7feb58157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:35.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb57fff700 1 --2- 172.21.15.33:0/3832132369 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 0x7feb58157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb57fff700 1 -- 172.21.15.33:0/3832132369 >> v2:172.21.15.69:3300/0 conn(0x7feb58062710 msgr2=0x7feb5813d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:35.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb57fff700 1 --2- 172.21.15.33:0/3832132369 >> v2:172.21.15.69:3300/0 conn(0x7feb58062710 0x7feb5813d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb57fff700 1 -- 172.21.15.33:0/3832132369 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb58062bc0 con 0x7feb580c4b00 2023-11-06T21:00:35.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb5c99b700 1 --2- 172.21.15.33:0/3832132369 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 0x7feb58157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:35.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb5d19c700 1 --2- 172.21.15.33:0/3832132369 >> v2:172.21.15.69:3300/0 conn(0x7feb58062710 0x7feb5813d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:35.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb57fff700 1 --2- 172.21.15.33:0/3832132369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 0x7feb580c2860 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7feb580c2370 tx=0x7feb440090f0).ready entity=mon.0 client_cookie=c10f44e21eace5d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:35.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb577fe700 1 -- 172.21.15.33:0/3832132369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400b600 con 0x7feb580c4b00 2023-11-06T21:00:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb577fe700 1 -- 172.21.15.33:0/3832132369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4400b760 con 0x7feb580c4b00 2023-11-06T21:00:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.036+0000 7feb577fe700 1 -- 172.21.15.33:0/3832132369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400bae0 con 0x7feb580c4b00 2023-11-06T21:00:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.037+0000 7feb5ebff700 1 -- 172.21.15.33:0/3832132369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 msgr2=0x7feb580c2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.037+0000 7feb5ebff700 1 --2- 172.21.15.33:0/3832132369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 0x7feb580c2860 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7feb580c2370 tx=0x7feb440090f0).stop 2023-11-06T21:00:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.037+0000 7feb5ebff700 1 -- 172.21.15.33:0/3832132369 shutdown_connections 2023-11-06T21:00:35.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.037+0000 7feb5ebff700 1 --2- 172.21.15.33:0/3832132369 >> v2:172.21.15.69:3300/0 conn(0x7feb58062710 0x7feb5813d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.037+0000 7feb5ebff700 1 --2- 172.21.15.33:0/3832132369 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 0x7feb58157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.037+0000 7feb5ebff700 1 --2- 172.21.15.33:0/3832132369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 0x7feb580c2860 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.037+0000 7feb5ebff700 1 -- 172.21.15.33:0/3832132369 >> 172.21.15.33:0/3832132369 conn(0x7feb580c7dc0 msgr2=0x7feb580c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:35.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.037+0000 7feb5ebff700 1 -- 172.21.15.33:0/3832132369 shutdown_connections 2023-11-06T21:00:35.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.037+0000 7feb5ebff700 1 -- 172.21.15.33:0/3832132369 wait complete. 2023-11-06T21:00:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.039+0000 7feb5ebff700 1 Processor -- start 2023-11-06T21:00:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.039+0000 7feb5ebff700 1 -- start start 2023-11-06T21:00:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb5ebff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb58062710 0x7feb58153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb5ebff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 0x7feb5813be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb5ebff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 0x7feb5813c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb5ebff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5813c800 con 0x7feb580c4b00 2023-11-06T21:00:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb5ebff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb581660b0 con 0x7feb58062710 2023-11-06T21:00:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb5ebff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb581540f0 con 0x7feb580c4720 2023-11-06T21:00:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb5d19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 0x7feb5813c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb5d19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 0x7feb5813c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36714/0 (socket says 172.21.15.33:36714) 2023-11-06T21:00:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb5d19c700 1 -- 172.21.15.33:0/606826924 learned_addr learned my addr 172.21.15.33:0/606826924 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb5c99b700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb58062710 0x7feb58153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:35.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.040+0000 7feb57fff700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 0x7feb5813be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:35.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb5d19c700 1 -- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 msgr2=0x7feb5813be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:35.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb5d19c700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 0x7feb5813be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb5d19c700 1 -- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb58062710 msgr2=0x7feb58153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:35.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb5d19c700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb58062710 0x7feb58153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb5d19c700 1 -- 172.21.15.33:0/606826924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb44009370 con 0x7feb580c4b00 2023-11-06T21:00:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb5c99b700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb58062710 0x7feb58153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb5d19c700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 0x7feb5813c310 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7feb5000cef0 tx=0x7feb50007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb54ff9700 1 -- 172.21.15.33:0/606826924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5000add0 con 0x7feb580c4b00 2023-11-06T21:00:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb54ff9700 1 -- 172.21.15.33:0/606826924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb50016400 con 0x7feb580c4b00 2023-11-06T21:00:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb54ff9700 1 -- 172.21.15.33:0/606826924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50016780 con 0x7feb580c4b00 2023-11-06T21:00:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb5ebff700 1 -- 172.21.15.33:0/606826924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb58165780 con 0x7feb580c4b00 2023-11-06T21:00:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.041+0000 7feb5ebff700 1 -- 172.21.15.33:0/606826924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb58165d40 con 0x7feb580c4b00 2023-11-06T21:00:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.042+0000 7feb5ebff700 1 -- 172.21.15.33:0/606826924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb580aa0d0 con 0x7feb580c4b00 2023-11-06T21:00:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.043+0000 7feb54ff9700 1 -- 172.21.15.33:0/606826924 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7feb500203f0 con 0x7feb580c4b00 2023-11-06T21:00:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.044+0000 7feb54ff9700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb3c05f270 0x7feb3c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:35.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.044+0000 7feb54ff9700 1 -- 172.21.15.33:0/606826924 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feb5007ae40 con 0x7feb580c4b00 2023-11-06T21:00:35.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.045+0000 7feb5c99b700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb3c05f270 0x7feb3c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:35.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.045+0000 7feb5c99b700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb3c05f270 0x7feb3c061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7feb4c00aa90 tx=0x7feb4c008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:35.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.048+0000 7feb54ff9700 1 -- 172.21.15.33:0/606826924 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feb5004be70 con 0x7feb580c4b00 2023-11-06T21:00:35.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.279+0000 7feb5ebff700 1 -- 172.21.15.33:0/606826924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feb58069c20 con 0x7feb580c4b00 2023-11-06T21:00:35.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.976+0000 7feb54ff9700 1 -- 172.21.15.33:0/606826924 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) v1 ==== 118+0+0 (secure 0 0 0) 0x7feb5004f490 con 0x7feb580c4b00 2023-11-06T21:00:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 -- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb3c05f270 msgr2=0x7feb3c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb3c05f270 0x7feb3c061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7feb4c00aa90 tx=0x7feb4c008040).stop 2023-11-06T21:00:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 -- 172.21.15.33:0/606826924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 msgr2=0x7feb5813c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 0x7feb5813c310 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7feb5000cef0 tx=0x7feb50007860).stop 2023-11-06T21:00:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 -- 172.21.15.33:0/606826924 shutdown_connections 2023-11-06T21:00:35.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7feb3c05f270 0x7feb3c061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c4b00 0x7feb5813c310 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb58062710 0x7feb58153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 --2- 172.21.15.33:0/606826924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feb580c4720 0x7feb5813be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:35.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 -- 172.21.15.33:0/606826924 >> 172.21.15.33:0/606826924 conn(0x7feb580c7dc0 msgr2=0x7feb581381d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:35.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 -- 172.21.15.33:0/606826924 shutdown_connections 2023-11-06T21:00:35.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:35.979+0000 7feb5ebff700 1 -- 172.21.15.33:0/606826924 wait complete. 2023-11-06T21:00:36.002 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T21:00:36.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T21:00:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.207+0000 7f7d489ff700 1 Processor -- start 2023-11-06T21:00:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.207+0000 7f7d489ff700 1 -- start start 2023-11-06T21:00:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.208+0000 7f7d489ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7d44154330 0x7f7d440c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.208+0000 7f7d489ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7d44154710 0x7f7d440c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.208+0000 7f7d489ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d440c6b60 con 0x7f7d440c7ff0 2023-11-06T21:00:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.208+0000 7f7d489ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d440c5cd0 con 0x7f7d44154710 2023-11-06T21:00:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.208+0000 7f7d489ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d440c5f50 con 0x7f7d44154330 2023-11-06T21:00:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.209+0000 7f7d41d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7d44154710 0x7f7d440c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.209+0000 7f7d42d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7d440c7ff0 0x7f7d440646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48312/0 (socket says 172.21.15.33:48312) 2023-11-06T21:00:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.209+0000 7f7d42d9d700 1 -- 172.21.15.33:0/3614134411 learned_addr learned my addr 172.21.15.33:0/3614134411 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.209+0000 7f7d4259c700 1 --2- 172.21.15.33:0/3614134411 >> v2:172.21.15.69:3300/0 conn(0x7f7d44154330 0x7f7d440c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.209+0000 7f7d41d9b700 1 -- 172.21.15.33:0/3614134411 >> v2:172.21.15.69:3300/0 conn(0x7f7d44154330 msgr2=0x7f7d440c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.209+0000 7f7d41d9b700 1 --2- 172.21.15.33:0/3614134411 >> v2:172.21.15.69:3300/0 conn(0x7f7d44154330 0x7f7d440c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.209+0000 7f7d41d9b700 1 -- 172.21.15.33:0/3614134411 >> v1:172.21.15.33:6789/0 conn(0x7f7d440c7ff0 legacy=0x7f7d440646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.209+0000 7f7d41d9b700 1 -- 172.21.15.33:0/3614134411 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d440c6230 con 0x7f7d44154710 2023-11-06T21:00:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.209+0000 7f7d4159a700 1 -- 172.21.15.33:0/3614134411 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408390110 0 0) 0x7f7d440c6b60 con 0x7f7d440c7ff0 2023-11-06T21:00:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.209+0000 7f7d41d9b700 1 --2- 172.21.15.33:0/3614134411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 0x7f7d440c6ff0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f7d440605d0 tx=0x7f7d38009060).ready entity=mon.0 client_cookie=34b1d00938dbb899 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d4159a700 1 -- 172.21.15.33:0/3614134411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800b580 con 0x7f7d44154710 2023-11-06T21:00:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d4159a700 1 -- 172.21.15.33:0/3614134411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800b6e0 con 0x7f7d44154710 2023-11-06T21:00:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d4159a700 1 -- 172.21.15.33:0/3614134411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800ba60 con 0x7f7d44154710 2023-11-06T21:00:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d489ff700 1 -- 172.21.15.33:0/3614134411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 msgr2=0x7f7d440c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d489ff700 1 --2- 172.21.15.33:0/3614134411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 0x7f7d440c6ff0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f7d440605d0 tx=0x7f7d38009060).stop 2023-11-06T21:00:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d489ff700 1 -- 172.21.15.33:0/3614134411 shutdown_connections 2023-11-06T21:00:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d489ff700 1 --2- 172.21.15.33:0/3614134411 >> v2:172.21.15.69:3300/0 conn(0x7f7d44154330 0x7f7d440c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d489ff700 1 --2- 172.21.15.33:0/3614134411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 0x7f7d440c6ff0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d489ff700 1 -- 172.21.15.33:0/3614134411 >> 172.21.15.33:0/3614134411 conn(0x7f7d440cb910 msgr2=0x7f7d440c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d489ff700 1 -- 172.21.15.33:0/3614134411 shutdown_connections 2023-11-06T21:00:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.210+0000 7f7d489ff700 1 -- 172.21.15.33:0/3614134411 wait complete. 2023-11-06T21:00:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.212+0000 7f7d489ff700 1 Processor -- start 2023-11-06T21:00:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.212+0000 7f7d489ff700 1 -- start start 2023-11-06T21:00:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d489ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7d440c7ff0 0x7f7d440c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d489ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d44154330 0x7f7d4414d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d489ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 0x7f7d440aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d489ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440afad0 con 0x7f7d44154710 2023-11-06T21:00:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d489ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440c3b30 con 0x7f7d44154330 2023-11-06T21:00:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d489ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4414d950 con 0x7f7d440c7ff0 2023-11-06T21:00:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d42d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 0x7f7d440aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d42d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 0x7f7d440aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36736/0 (socket says 172.21.15.33:36736) 2023-11-06T21:00:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d42d9d700 1 -- 172.21.15.33:0/1172911989 learned_addr learned my addr 172.21.15.33:0/1172911989 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d4259c700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7d440c7ff0 0x7f7d440c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.213+0000 7f7d41d9b700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d44154330 0x7f7d4414d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d42d9d700 1 -- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7d440c7ff0 msgr2=0x7f7d440c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d42d9d700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7d440c7ff0 0x7f7d440c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d42d9d700 1 -- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d44154330 msgr2=0x7f7d4414d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d42d9d700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d44154330 0x7f7d4414d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:36.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d42d9d700 1 -- 172.21.15.33:0/1172911989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d380092a0 con 0x7f7d44154710 2023-11-06T21:00:36.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d42d9d700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 0x7f7d440aee00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f7d3c00c810 tx=0x7f7d3c0014d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:36.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d327fc700 1 -- 172.21.15.33:0/1172911989 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c005d30 con 0x7f7d44154710 2023-11-06T21:00:36.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d327fc700 1 -- 172.21.15.33:0/1172911989 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c005e90 con 0x7f7d44154710 2023-11-06T21:00:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d489ff700 1 -- 172.21.15.33:0/1172911989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d440c3570 con 0x7f7d44154710 2023-11-06T21:00:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d327fc700 1 -- 172.21.15.33:0/1172911989 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0072e0 con 0x7f7d44154710 2023-11-06T21:00:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.214+0000 7f7d489ff700 1 -- 172.21.15.33:0/1172911989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d440af760 con 0x7f7d44154710 2023-11-06T21:00:36.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.216+0000 7f7d489ff700 1 -- 172.21.15.33:0/1172911989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d440afc40 con 0x7f7d44154710 2023-11-06T21:00:36.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.216+0000 7f7d327fc700 1 -- 172.21.15.33:0/1172911989 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7d3c007710 con 0x7f7d44154710 2023-11-06T21:00:36.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.218+0000 7f7d327fc700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7d2405f270 0x7f7d24061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:36.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.218+0000 7f7d327fc700 1 -- 172.21.15.33:0/1172911989 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7d3c079dd0 con 0x7f7d44154710 2023-11-06T21:00:36.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.218+0000 7f7d4259c700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7d2405f270 0x7f7d24061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:36.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.219+0000 7f7d4259c700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7d2405f270 0x7f7d24061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7d3400bbc0 tx=0x7f7d34008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:36.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.222+0000 7f7d327fc700 1 -- 172.21.15.33:0/1172911989 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7d3c04af20 con 0x7f7d44154710 2023-11-06T21:00:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.449+0000 7f7d489ff700 1 -- 172.21.15.33:0/1172911989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7d440652d0 con 0x7f7d44154710 2023-11-06T21:00:36.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.978+0000 7f7d327fc700 1 -- 172.21.15.33:0/1172911989 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7d3c018030 con 0x7f7d44154710 2023-11-06T21:00:36.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.981+0000 7f7d489ff700 1 -- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7d2405f270 msgr2=0x7f7d24061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:36.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.981+0000 7f7d489ff700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7d2405f270 0x7f7d24061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7d3400bbc0 tx=0x7f7d34008040).stop 2023-11-06T21:00:36.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.981+0000 7f7d489ff700 1 -- 172.21.15.33:0/1172911989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 msgr2=0x7f7d440aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:36.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.981+0000 7f7d489ff700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 0x7f7d440aee00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f7d3c00c810 tx=0x7f7d3c0014d0).stop 2023-11-06T21:00:36.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.982+0000 7f7d489ff700 1 -- 172.21.15.33:0/1172911989 shutdown_connections 2023-11-06T21:00:36.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.982+0000 7f7d489ff700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7d2405f270 0x7f7d24061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:36.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.982+0000 7f7d489ff700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7d440c7ff0 0x7f7d440c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:36.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.982+0000 7f7d489ff700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d44154710 0x7f7d440aee00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:36.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.982+0000 7f7d489ff700 1 --2- 172.21.15.33:0/1172911989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d44154330 0x7f7d4414d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:36.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.982+0000 7f7d489ff700 1 -- 172.21.15.33:0/1172911989 >> 172.21.15.33:0/1172911989 conn(0x7f7d440cb910 msgr2=0x7f7d440c9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:36.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.982+0000 7f7d489ff700 1 -- 172.21.15.33:0/1172911989 shutdown_connections 2023-11-06T21:00:36.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:36.982+0000 7f7d489ff700 1 -- 172.21.15.33:0/1172911989 wait complete. 2023-11-06T21:00:37.006 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T21:00:37.006 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:00:37.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.206+0000 7f39e9dff700 1 Processor -- start 2023-11-06T21:00:37.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.206+0000 7f39e9dff700 1 -- start start 2023-11-06T21:00:37.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.208+0000 7f39e9dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f39e4154110 0x7f39e41544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.208+0000 7f39e9dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 0x7f39e40c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.208+0000 7f39e9dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e40c0880 con 0x7f39e40aade0 2023-11-06T21:00:37.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.208+0000 7f39e9dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e40bfd40 con 0x7f39e4154110 2023-11-06T21:00:37.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.208+0000 7f39e9dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e40c8260 con 0x7f39e4064ce0 2023-11-06T21:00:37.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.208+0000 7f39e3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 0x7f39e40c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.208+0000 7f39e2ffd700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f39e40aade0 0x7f39e40645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:49422/0 (socket says 172.21.15.33:49422) 2023-11-06T21:00:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.208+0000 7f39e2ffd700 1 -- 172.21.15.33:0/3393693681 learned_addr learned my addr 172.21.15.33:0/3393693681 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.208+0000 7f39e37fe700 1 --2- 172.21.15.33:0/3393693681 >> v2:172.21.15.69:3300/0 conn(0x7f39e4154110 0x7f39e41544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.209+0000 7f39e27fc700 1 -- 172.21.15.33:0/3393693681 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411100082 0 0) 0x7f39e40c0880 con 0x7f39e40aade0 2023-11-06T21:00:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.209+0000 7f39e27fc700 1 -- 172.21.15.33:0/3393693681 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39cc0033f0 con 0x7f39e40aade0 2023-11-06T21:00:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.209+0000 7f39e3fff700 1 -- 172.21.15.33:0/3393693681 >> v2:172.21.15.69:3300/0 conn(0x7f39e4154110 msgr2=0x7f39e41544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:37.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.209+0000 7f39e3fff700 1 --2- 172.21.15.33:0/3393693681 >> v2:172.21.15.69:3300/0 conn(0x7f39e4154110 0x7f39e41544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.209+0000 7f39e3fff700 1 -- 172.21.15.33:0/3393693681 >> v1:172.21.15.69:6789/0 conn(0x7f39e40aade0 legacy=0x7f39e40645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.209+0000 7f39e3fff700 1 -- 172.21.15.33:0/3393693681 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e40bff50 con 0x7f39e4064ce0 2023-11-06T21:00:37.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.209+0000 7f39e3fff700 1 --2- 172.21.15.33:0/3393693681 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 0x7f39e40c5db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f39dc009e60 tx=0x7f39dc009780).ready entity=mon.2 client_cookie=47475be39ea749f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:37.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.209+0000 7f39e27fc700 1 -- 172.21.15.33:0/3393693681 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00e070 con 0x7f39e4064ce0 2023-11-06T21:00:37.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.209+0000 7f39e27fc700 1 -- 172.21.15.33:0/3393693681 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc00bab0 con 0x7f39e4064ce0 2023-11-06T21:00:37.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.209+0000 7f39e27fc700 1 -- 172.21.15.33:0/3393693681 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00be30 con 0x7f39e4064ce0 2023-11-06T21:00:37.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.210+0000 7f39e9dff700 1 -- 172.21.15.33:0/3393693681 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 msgr2=0x7f39e40c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.210+0000 7f39e9dff700 1 --2- 172.21.15.33:0/3393693681 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 0x7f39e40c5db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f39dc009e60 tx=0x7f39dc009780).stop 2023-11-06T21:00:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.210+0000 7f39e9dff700 1 -- 172.21.15.33:0/3393693681 shutdown_connections 2023-11-06T21:00:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.210+0000 7f39e9dff700 1 --2- 172.21.15.33:0/3393693681 >> v2:172.21.15.69:3300/0 conn(0x7f39e4154110 0x7f39e41544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.210+0000 7f39e9dff700 1 --2- 172.21.15.33:0/3393693681 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 0x7f39e40c5db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.210+0000 7f39e9dff700 1 -- 172.21.15.33:0/3393693681 >> 172.21.15.33:0/3393693681 conn(0x7f39e40cb8b0 msgr2=0x7f39e40c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.210+0000 7f39e9dff700 1 -- 172.21.15.33:0/3393693681 shutdown_connections 2023-11-06T21:00:37.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.210+0000 7f39e9dff700 1 -- 172.21.15.33:0/3393693681 wait complete. 2023-11-06T21:00:37.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.211+0000 7f39e9dff700 1 Processor -- start 2023-11-06T21:00:37.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.212+0000 7f39e9dff700 1 -- start start 2023-11-06T21:00:37.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e9dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 0x7f39e414fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e9dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e40aade0 0x7f39e40ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e9dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39e4154110 0x7f39e40aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e9dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e4156550 con 0x7f39e4154110 2023-11-06T21:00:37.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e9dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e4156240 con 0x7f39e40aade0 2023-11-06T21:00:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e9dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e40af1f0 con 0x7f39e4064ce0 2023-11-06T21:00:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39e4154110 0x7f39e40aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39e4154110 0x7f39e40aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36752/0 (socket says 172.21.15.33:36752) 2023-11-06T21:00:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e37fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 0x7f39e414fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e3fff700 1 -- 172.21.15.33:0/1470551282 learned_addr learned my addr 172.21.15.33:0/1470551282 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.213+0000 7f39e2ffd700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e40aade0 0x7f39e40ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39e3fff700 1 -- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 msgr2=0x7f39e414fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39e3fff700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 0x7f39e414fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39e3fff700 1 -- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e40aade0 msgr2=0x7f39e40ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39e3fff700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e40aade0 0x7f39e40ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39e3fff700 1 -- 172.21.15.33:0/1470551282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc009a20 con 0x7f39e4154110 2023-11-06T21:00:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39e3fff700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39e4154110 0x7f39e40aec50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f39e40bfe80 tx=0x7f39dc009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39cb7fe700 1 -- 172.21.15.33:0/1470551282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc013e40 con 0x7f39e4154110 2023-11-06T21:00:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39cb7fe700 1 -- 172.21.15.33:0/1470551282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc01d630 con 0x7f39e4154110 2023-11-06T21:00:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39cb7fe700 1 -- 172.21.15.33:0/1470551282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc01d9f0 con 0x7f39e4154110 2023-11-06T21:00:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39e9dff700 1 -- 172.21.15.33:0/1470551282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e40af4d0 con 0x7f39e4154110 2023-11-06T21:00:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.214+0000 7f39e9dff700 1 -- 172.21.15.33:0/1470551282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e4154bd0 con 0x7f39e4154110 2023-11-06T21:00:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.215+0000 7f39e9dff700 1 -- 172.21.15.33:0/1470551282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e40af9f0 con 0x7f39e4154110 2023-11-06T21:00:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.216+0000 7f39cb7fe700 1 -- 172.21.15.33:0/1470551282 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f39dc003040 con 0x7f39e4154110 2023-11-06T21:00:37.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.217+0000 7f39cb7fe700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39d005f270 0x7f39d0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.217+0000 7f39cb7fe700 1 -- 172.21.15.33:0/1470551282 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f39dc07c140 con 0x7f39e4154110 2023-11-06T21:00:37.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.218+0000 7f39e37fe700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39d005f270 0x7f39d0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.219+0000 7f39e37fe700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39d005f270 0x7f39d0061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f39d4009b90 tx=0x7f39d4009060).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:37.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.222+0000 7f39cb7fe700 1 -- 172.21.15.33:0/1470551282 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f39e40af9f0 con 0x7f39e4154110 2023-11-06T21:00:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.447+0000 7f39e9dff700 1 -- 172.21.15.33:0/1470551282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39e40af9f0 con 0x7f39e4154110 2023-11-06T21:00:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.447+0000 7f39cb7fe700 1 -- 172.21.15.33:0/1470551282 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4485 (secure 0 0 0) 0x7f39e40af9f0 con 0x7f39e4154110 2023-11-06T21:00:37.448 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:37.448 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":45,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2108889615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2108889615},{"type":"v1","addr":"172.21.15.33:6834","nonce":2108889615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/836821298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":836821298},{"type":"v1","addr":"172.21.15.69:6836","nonce":836821298}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6838,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2007995447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2007995447},{"type":"v1","addr":"172.21.15.33:6835","nonce":2007995447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:00:33.962914+0000","modified":"2023-11-06T21:00:36.976166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6844":{"gid":6844,"name":"d","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/396991049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":396991049},{"type":"v1","addr":"172.21.15.69:6837","nonce":396991049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-06T21:00:37.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.449+0000 7f39e9dff700 1 -- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39d005f270 msgr2=0x7f39d0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.449+0000 7f39e9dff700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39d005f270 0x7f39d0061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f39d4009b90 tx=0x7f39d4009060).stop 2023-11-06T21:00:37.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.449+0000 7f39e9dff700 1 -- 172.21.15.33:0/1470551282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39e4154110 msgr2=0x7f39e40aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.449+0000 7f39e9dff700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39e4154110 0x7f39e40aec50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f39e40bfe80 tx=0x7f39dc009780).stop 2023-11-06T21:00:37.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.449+0000 7f39e9dff700 1 -- 172.21.15.33:0/1470551282 shutdown_connections 2023-11-06T21:00:37.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.449+0000 7f39e9dff700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39d005f270 0x7f39d0061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.449+0000 7f39e9dff700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39e4064ce0 0x7f39e414fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.449+0000 7f39e9dff700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39e4154110 0x7f39e40aec50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.449+0000 7f39e9dff700 1 --2- 172.21.15.33:0/1470551282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e40aade0 0x7f39e40ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.449+0000 7f39e9dff700 1 -- 172.21.15.33:0/1470551282 >> 172.21.15.33:0/1470551282 conn(0x7f39e40cb8b0 msgr2=0x7f39e40c8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.450+0000 7f39e9dff700 1 -- 172.21.15.33:0/1470551282 shutdown_connections 2023-11-06T21:00:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.450+0000 7f39e9dff700 1 -- 172.21.15.33:0/1470551282 wait complete. 2023-11-06T21:00:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 45 2023-11-06T21:00:37.468 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:00:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.663+0000 7fd4133ff700 1 Processor -- start 2023-11-06T21:00:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.663+0000 7fd4133ff700 1 -- start start 2023-11-06T21:00:37.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.664+0000 7fd4133ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0c1840 0x7fd40c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.664+0000 7fd4133ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd40c0c6460 0x7fd40c0bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.665+0000 7fd4133ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c1554a0 con 0x7fd40c0aa2d0 2023-11-06T21:00:37.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.665+0000 7fd4133ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40c154960 con 0x7fd40c0c6460 2023-11-06T21:00:37.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.665+0000 7fd4133ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c155930 con 0x7fd40c0c1840 2023-11-06T21:00:37.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.665+0000 7fd41099a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd40c0aa2d0 0x7fd40c0c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48314/0 (socket says 172.21.15.33:48314) 2023-11-06T21:00:37.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.665+0000 7fd41099a700 1 -- 172.21.15.33:0/2927806924 learned_addr learned my addr 172.21.15.33:0/2927806924 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:37.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.665+0000 7fd41199c700 1 --2- 172.21.15.33:0/2927806924 >> v2:172.21.15.69:3300/0 conn(0x7fd40c0c6460 0x7fd40c0bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.665+0000 7fd41119b700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0c1840 0x7fd40c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.665+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025152490 0 0) 0x7fd40c1554a0 con 0x7fd40c0aa2d0 2023-11-06T21:00:37.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.665+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f80033f0 con 0x7fd40c0aa2d0 2023-11-06T21:00:37.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.665+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3970711304 0 0) 0x7fd3f80033f0 con 0x7fd40c0aa2d0 2023-11-06T21:00:37.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0c1840 msgr2=0x7fd40c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:37.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0c1840 0x7fd40c0a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 >> v2:172.21.15.69:3300/0 conn(0x7fd40c0c6460 msgr2=0x7fd40c0bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:37.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 --2- 172.21.15.33:0/2927806924 >> v2:172.21.15.69:3300/0 conn(0x7fd40c0c6460 0x7fd40c0bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c154b70 con 0x7fd40c0aa2d0 2023-11-06T21:00:37.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fd3fc001470 con 0x7fd40c0aa2d0 2023-11-06T21:00:37.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 >> v1:172.21.15.33:6789/0 conn(0x7fd40c0aa2d0 legacy=0x7fd40c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0c1840 0x7fd3f80038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f8003e10 0x7fd3f8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 0x7fd3f8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c154960 con 0x7fd3f8004b80 2023-11-06T21:00:37.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c155930 con 0x7fd3f8003e10 2023-11-06T21:00:37.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f800b8e0 con 0x7fd40c0c1840 2023-11-06T21:00:37.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd41199c700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 0x7fd3f8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd41099a700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f8003e10 0x7fd3f8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd41119b700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0c1840 0x7fd3f80038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd41199c700 1 -- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0c1840 msgr2=0x7fd3f80038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:37.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.666+0000 7fd41199c700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0c1840 0x7fd3f80038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd41199c700 1 -- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f8003e10 msgr2=0x7fd3f8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:37.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd41199c700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f8003e10 0x7fd3f8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd41199c700 1 -- 172.21.15.33:0/2927806924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f80033f0 con 0x7fd3f8004b80 2023-11-06T21:00:37.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd41099a700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f8003e10 0x7fd3f8004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:37.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd41199c700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 0x7fd3f8005000 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fd40400e1f0 tx=0x7fd40400b040).ready entity=mon.0 client_cookie=ac82cb594be010de server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:37.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404009ac0 con 0x7fd3f8004b80 2023-11-06T21:00:37.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40400f040 con 0x7fd3f8004b80 2023-11-06T21:00:37.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd403fff700 1 -- 172.21.15.33:0/2927806924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400ecd0 con 0x7fd3f8004b80 2023-11-06T21:00:37.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd4133ff700 1 -- 172.21.15.33:0/2927806924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 msgr2=0x7fd3f8005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd4133ff700 1 --2- 172.21.15.33:0/2927806924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 0x7fd3f8005000 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fd40400e1f0 tx=0x7fd40400b040).stop 2023-11-06T21:00:37.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.667+0000 7fd41199c700 1 -- 172.21.15.33:0/2927806924 reap_dead start 2023-11-06T21:00:37.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.668+0000 7fd4133ff700 1 -- 172.21.15.33:0/2927806924 shutdown_connections 2023-11-06T21:00:37.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.668+0000 7fd4133ff700 1 -- 172.21.15.33:0/2927806924 >> 172.21.15.33:0/2927806924 conn(0x7fd40c05ec80 msgr2=0x7fd40c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:37.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.668+0000 7fd4133ff700 1 -- 172.21.15.33:0/2927806924 shutdown_connections 2023-11-06T21:00:37.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.668+0000 7fd4133ff700 1 -- 172.21.15.33:0/2927806924 wait complete. 2023-11-06T21:00:37.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.670+0000 7fd4133ff700 1 Processor -- start 2023-11-06T21:00:37.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.670+0000 7fd4133ff700 1 -- start start 2023-11-06T21:00:37.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.671+0000 7fd4133ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 0x7fd40c0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.671+0000 7fd4133ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0ac8c0 0x7fd40c0acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.671+0000 7fd4133ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd40c155d30 0x7fd40c1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.671+0000 7fd4133ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c157700 con 0x7fd3f8004b80 2023-11-06T21:00:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.671+0000 7fd4133ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0ad230 con 0x7fd40c155d30 2023-11-06T21:00:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.671+0000 7fd4133ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0b0b20 con 0x7fd40c0ac8c0 2023-11-06T21:00:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 0x7fd40c0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 0x7fd40c0b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36764/0 (socket says 172.21.15.33:36764) 2023-11-06T21:00:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41119b700 1 -- 172.21.15.33:0/94884435 learned_addr learned my addr 172.21.15.33:0/94884435 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41199c700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd40c155d30 0x7fd40c1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41119b700 1 -- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0ac8c0 msgr2=0x7fd40c0acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41119b700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0ac8c0 0x7fd40c0acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:37.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41119b700 1 -- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd40c155d30 msgr2=0x7fd40c1561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41119b700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd40c155d30 0x7fd40c1561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41119b700 1 -- 172.21.15.33:0/94884435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c157b90 con 0x7fd3f8004b80 2023-11-06T21:00:37.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41199c700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd40c155d30 0x7fd40c1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd41119b700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 0x7fd40c0b0630 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7fd4080119c0 tx=0x7fd408009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd4017fa700 1 -- 172.21.15.33:0/94884435 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40800f940 con 0x7fd3f8004b80 2023-11-06T21:00:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd4017fa700 1 -- 172.21.15.33:0/94884435 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40800faa0 con 0x7fd3f8004b80 2023-11-06T21:00:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd4017fa700 1 -- 172.21.15.33:0/94884435 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40800fe20 con 0x7fd3f8004b80 2023-11-06T21:00:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.672+0000 7fd4133ff700 1 -- 172.21.15.33:0/94884435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c156af0 con 0x7fd3f8004b80 2023-11-06T21:00:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.673+0000 7fd4133ff700 1 -- 172.21.15.33:0/94884435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c1570b0 con 0x7fd3f8004b80 2023-11-06T21:00:37.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.674+0000 7fd4133ff700 1 -- 172.21.15.33:0/94884435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c1579e0 con 0x7fd3f8004b80 2023-11-06T21:00:37.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.674+0000 7fd4017fa700 1 -- 172.21.15.33:0/94884435 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd408014650 con 0x7fd3f8004b80 2023-11-06T21:00:37.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.676+0000 7fd4017fa700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd3f005f240 0x7fd3f0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:37.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.676+0000 7fd4017fa700 1 -- 172.21.15.33:0/94884435 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd40807b0a0 con 0x7fd3f8004b80 2023-11-06T21:00:37.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.676+0000 7fd41099a700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd3f005f240 0x7fd3f0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.677+0000 7fd41099a700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd3f005f240 0x7fd3f0061700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd3fc007ba0 tx=0x7fd40c155410).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.679+0000 7fd4017fa700 1 -- 172.21.15.33:0/94884435 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd40804c130 con 0x7fd3f8004b80 2023-11-06T21:00:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.863+0000 7fd4133ff700 1 -- 172.21.15.33:0/94884435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd40c06c6c0 con 0x7fd3f8004b80 2023-11-06T21:00:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.864+0000 7fd4017fa700 1 -- 172.21.15.33:0/94884435 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16991 (secure 0 0 0) 0x7fd408018020 con 0x7fd3f8004b80 2023-11-06T21:00:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:37.865 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":114,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:00:34.965430+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:00:31.623152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T21:00:32.950772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:00:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.866+0000 7fd4133ff700 1 -- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd3f005f240 msgr2=0x7fd3f0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.866+0000 7fd4133ff700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd3f005f240 0x7fd3f0061700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd3fc007ba0 tx=0x7fd40c155410).stop 2023-11-06T21:00:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.866+0000 7fd4133ff700 1 -- 172.21.15.33:0/94884435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 msgr2=0x7fd40c0b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.866+0000 7fd4133ff700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 0x7fd40c0b0630 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7fd4080119c0 tx=0x7fd408009060).stop 2023-11-06T21:00:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.866+0000 7fd4133ff700 1 -- 172.21.15.33:0/94884435 shutdown_connections 2023-11-06T21:00:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.866+0000 7fd4133ff700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd3f005f240 0x7fd3f0061700 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.866+0000 7fd4133ff700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd40c155d30 0x7fd40c1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.866+0000 7fd4133ff700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f8004b80 0x7fd40c0b0630 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.866+0000 7fd4133ff700 1 --2- 172.21.15.33:0/94884435 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd40c0ac8c0 0x7fd40c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.866+0000 7fd4133ff700 1 -- 172.21.15.33:0/94884435 >> 172.21.15.33:0/94884435 conn(0x7fd40c05ec80 msgr2=0x7fd40c153af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.867+0000 7fd4133ff700 1 -- 172.21.15.33:0/94884435 shutdown_connections 2023-11-06T21:00:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:37.867+0000 7fd4133ff700 1 -- 172.21.15.33:0/94884435 wait complete. 2023-11-06T21:00:37.885 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T21:00:37.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T21:00:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.082+0000 7faaa83ff700 1 Processor -- start 2023-11-06T21:00:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.082+0000 7faaa83ff700 1 -- start start 2023-11-06T21:00:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.083+0000 7faaa83ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaa014ae80 0x7faaa00bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.083+0000 7faaa83ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa00c1940 con 0x7faaa00bc5c0 2023-11-06T21:00:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.083+0000 7faaa83ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa00bca40 con 0x7faaa0064b10 2023-11-06T21:00:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.083+0000 7faaa83ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa014b260 con 0x7faaa014ae80 2023-11-06T21:00:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.083+0000 7faaa599a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faaa00bc5c0 0x7faaa0064400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46814/0 (socket says 172.21.15.33:46814) 2023-11-06T21:00:38.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.083+0000 7faaa599a700 1 -- 172.21.15.33:0/1638993133 learned_addr learned my addr 172.21.15.33:0/1638993133 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:38.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa619b700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaa014ae80 0x7faaa00bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605232488 0 0) 0x7faaa00c1940 con 0x7faaa00bc5c0 2023-11-06T21:00:38.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa8c0033f0 con 0x7faaa00bc5c0 2023-11-06T21:00:38.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599345993 0 0) 0x7faaa00bca40 con 0x7faaa0064b10 2023-11-06T21:00:38.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa00c1940 con 0x7faaa0064b10 2023-11-06T21:00:38.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2562728148 0 0) 0x7faa8c0033f0 con 0x7faaa00bc5c0 2023-11-06T21:00:38.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaa014ae80 msgr2=0x7faaa00bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:38.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaa014ae80 0x7faaa00bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 >> v1:172.21.15.69:6789/0 conn(0x7faaa0064b10 legacy=0x7faaa00c1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0065160 con 0x7faaa00bc5c0 2023-11-06T21:00:38.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7faa90001470 con 0x7faaa00bc5c0 2023-11-06T21:00:38.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 >> v1:172.21.15.33:6789/0 conn(0x7faaa00bc5c0 legacy=0x7faaa0064400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c003aa0 0x7faa8c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaa014ae80 0x7faa8c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa8c004b80 0x7faa8c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa014b260 con 0x7faa8c003aa0 2023-11-06T21:00:38.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c00b910 con 0x7faa8c004b80 2023-11-06T21:00:38.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.084+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c00bad0 con 0x7faaa014ae80 2023-11-06T21:00:38.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa619b700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c003aa0 0x7faa8c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa699c700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa8c004b80 0x7faa8c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa599a700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaa014ae80 0x7faa8c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa619b700 1 -- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaa014ae80 msgr2=0x7faa8c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa619b700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaa014ae80 0x7faa8c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa619b700 1 -- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa8c004b80 msgr2=0x7faa8c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa619b700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa8c004b80 0x7faa8c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa619b700 1 -- 172.21.15.33:0/1638993133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c0033f0 con 0x7faa8c003aa0 2023-11-06T21:00:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa599a700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaa014ae80 0x7faa8c004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:38.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa619b700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c003aa0 0x7faa8c003ea0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7faa9c00ab30 tx=0x7faa9c00a200).ready entity=mon.0 client_cookie=931672a9012c0245 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:38.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00e6e0 con 0x7faa8c003aa0 2023-11-06T21:00:38.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9c00e840 con 0x7faa8c003aa0 2023-11-06T21:00:38.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.085+0000 7faaa5199700 1 -- 172.21.15.33:0/1638993133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00e9b0 con 0x7faa8c003aa0 2023-11-06T21:00:38.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.086+0000 7faaa83ff700 1 -- 172.21.15.33:0/1638993133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c003aa0 msgr2=0x7faa8c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.086+0000 7faaa83ff700 1 --2- 172.21.15.33:0/1638993133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c003aa0 0x7faa8c003ea0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7faa9c00ab30 tx=0x7faa9c00a200).stop 2023-11-06T21:00:38.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.086+0000 7faaa699c700 1 -- 172.21.15.33:0/1638993133 reap_dead start 2023-11-06T21:00:38.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.086+0000 7faaa83ff700 1 -- 172.21.15.33:0/1638993133 shutdown_connections 2023-11-06T21:00:38.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.086+0000 7faaa83ff700 1 -- 172.21.15.33:0/1638993133 >> 172.21.15.33:0/1638993133 conn(0x7faaa0062710 msgr2=0x7faaa0062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:38.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.086+0000 7faaa83ff700 1 -- 172.21.15.33:0/1638993133 shutdown_connections 2023-11-06T21:00:38.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.086+0000 7faaa83ff700 1 -- 172.21.15.33:0/1638993133 wait complete. 2023-11-06T21:00:38.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.088+0000 7faaa83ff700 1 Processor -- start 2023-11-06T21:00:38.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.088+0000 7faaa83ff700 1 -- start start 2023-11-06T21:00:38.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.089+0000 7faaa83ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faa8c003aa0 0x7faaa0148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.089+0000 7faaa83ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0147ab0 0x7faaa0145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa83ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaa00691a0 0x7faaa0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa83ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa00be780 con 0x7faaa0147ab0 2023-11-06T21:00:38.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa83ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa0069b10 con 0x7faaa00691a0 2023-11-06T21:00:38.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa83ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa015f9d0 con 0x7faa8c003aa0 2023-11-06T21:00:38.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0147ab0 0x7faaa0145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0147ab0 0x7faaa0145880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41032/0 (socket says 172.21.15.33:41032) 2023-11-06T21:00:38.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa599a700 1 -- 172.21.15.33:0/3748403580 learned_addr learned my addr 172.21.15.33:0/3748403580 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:38.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa619b700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faa8c003aa0 0x7faaa0148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa699c700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaa00691a0 0x7faaa0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa599a700 1 -- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faa8c003aa0 msgr2=0x7faaa0148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa599a700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faa8c003aa0 0x7faaa0148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa599a700 1 -- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaa00691a0 msgr2=0x7faaa0069620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa599a700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaa00691a0 0x7faaa0069620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa599a700 1 -- 172.21.15.33:0/3748403580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0147f30 con 0x7faaa0147ab0 2023-11-06T21:00:38.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa619b700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faa8c003aa0 0x7faaa0148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:38.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa699c700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaa00691a0 0x7faaa0069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:38.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.090+0000 7faaa599a700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0147ab0 0x7faaa0145880 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7faa9000d7e0 tx=0x7faa90009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:38.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.091+0000 7faa967fc700 1 -- 172.21.15.33:0/3748403580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa90007170 con 0x7faaa0147ab0 2023-11-06T21:00:38.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.091+0000 7faa967fc700 1 -- 172.21.15.33:0/3748403580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa900072d0 con 0x7faaa0147ab0 2023-11-06T21:00:38.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.091+0000 7faa967fc700 1 -- 172.21.15.33:0/3748403580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa900048c0 con 0x7faaa0147ab0 2023-11-06T21:00:38.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.091+0000 7faaa83ff700 1 -- 172.21.15.33:0/3748403580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa00bdb70 con 0x7faaa0147ab0 2023-11-06T21:00:38.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.091+0000 7faaa83ff700 1 -- 172.21.15.33:0/3748403580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa00be130 con 0x7faaa0147ab0 2023-11-06T21:00:38.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.092+0000 7faaa83ff700 1 -- 172.21.15.33:0/3748403580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa005f0b0 con 0x7faaa0147ab0 2023-11-06T21:00:38.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.093+0000 7faa967fc700 1 -- 172.21.15.33:0/3748403580 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7faa90004c30 con 0x7faaa0147ab0 2023-11-06T21:00:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.094+0000 7faa967fc700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faa8405f270 0x7faa84061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.094+0000 7faa967fc700 1 -- 172.21.15.33:0/3748403580 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7faa9007a620 con 0x7faaa0147ab0 2023-11-06T21:00:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.094+0000 7faaa619b700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faa8405f270 0x7faa84061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.095+0000 7faaa619b700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faa8405f270 0x7faa84061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7faa9c00a240 tx=0x7faa9c019000).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.098+0000 7faa967fc700 1 -- 172.21.15.33:0/3748403580 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faa9004b6b0 con 0x7faaa0147ab0 2023-11-06T21:00:38.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.274+0000 7faaa83ff700 1 -- 172.21.15.33:0/3748403580 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faaa006bf60 con 0x7faa8405f270 2023-11-06T21:00:38.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.287+0000 7faa967fc700 1 -- 172.21.15.33:0/3748403580 <== mgr.6050 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361711 (secure 0 0 0) 0x7faaa006bf60 con 0x7faa8405f270 2023-11-06T21:00:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 -- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faa8405f270 msgr2=0x7faa84061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faa8405f270 0x7faa84061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7faa9c00a240 tx=0x7faa9c019000).stop 2023-11-06T21:00:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 -- 172.21.15.33:0/3748403580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0147ab0 msgr2=0x7faaa0145880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0147ab0 0x7faaa0145880 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7faa9000d7e0 tx=0x7faa90009040).stop 2023-11-06T21:00:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 -- 172.21.15.33:0/3748403580 shutdown_connections 2023-11-06T21:00:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faa8405f270 0x7faa84061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faa8c003aa0 0x7faaa0148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaa00691a0 0x7faaa0069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 --2- 172.21.15.33:0/3748403580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0147ab0 0x7faaa0145880 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 -- 172.21.15.33:0/3748403580 >> 172.21.15.33:0/3748403580 conn(0x7faaa0062710 msgr2=0x7faaa00c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 -- 172.21.15.33:0/3748403580 shutdown_connections 2023-11-06T21:00:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.292+0000 7faaa83ff700 1 -- 172.21.15.33:0/3748403580 wait complete. 2023-11-06T21:00:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T21:00:38.310 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T21:00:37.429266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2420460,"kb_used_data":10924,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752554260,"statfs":{"total":773094113280,"available":770615562240,"internally_reserved":0,"allocated":11186176,"data_stored":5600124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3334,"num_objects":-6,"num_object_clones":0,"num_object_copies":-12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-21,"num_write_kb":-16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.993244"},"pg_stats":[{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001167+0000","last_change":"2023-11-06T21:00:33.988209+0000","last_active":"2023-11-06T21:00:34.001167+0000","last_peered":"2023-11-06T21:00:34.001167+0000","last_clean":"2023-11-06T21:00:34.001167+0000","last_became_active":"2023-11-06T21:00:33.987851+0000","last_became_peered":"2023-11-06T21:00:33.987851+0000","last_unstale":"2023-11-06T21:00:34.001167+0000","last_undegraded":"2023-11-06T21:00:34.001167+0000","last_fullsized":"2023-11-06T21:00:34.001167+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3f","version":"113'3","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.996669+0000","last_change":"2023-11-06T21:00:33.511915+0000","last_active":"2023-11-06T21:00:33.996669+0000","last_peered":"2023-11-06T21:00:33.996669+0000","last_clean":"2023-11-06T21:00:33.996669+0000","last_became_active":"2023-11-06T21:00:33.511617+0000","last_became_peered":"2023-11-06T21:00:33.511617+0000","last_unstale":"2023-11-06T21:00:33.996669+0000","last_undegraded":"2023-11-06T21:00:33.996669+0000","last_fullsized":"2023-11-06T21:00:33.996669+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.992607+0000","last_change":"2023-11-06T21:00:33.984787+0000","last_active":"2023-11-06T21:00:33.992607+0000","last_peered":"2023-11-06T21:00:33.992607+0000","last_clean":"2023-11-06T21:00:33.992607+0000","last_became_active":"2023-11-06T21:00:33.984489+0000","last_became_peered":"2023-11-06T21:00:33.984489+0000","last_unstale":"2023-11-06T21:00:33.992607+0000","last_undegraded":"2023-11-06T21:00:33.992607+0000","last_fullsized":"2023-11-06T21:00:33.992607+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.991788+0000","last_change":"2023-11-06T21:00:33.493526+0000","last_active":"2023-11-06T21:00:33.991788+0000","last_peered":"2023-11-06T21:00:33.991788+0000","last_clean":"2023-11-06T21:00:33.991788+0000","last_became_active":"2023-11-06T21:00:33.493206+0000","last_became_peered":"2023-11-06T21:00:33.493206+0000","last_unstale":"2023-11-06T21:00:33.991788+0000","last_undegraded":"2023-11-06T21:00:33.991788+0000","last_fullsized":"2023-11-06T21:00:33.991788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989675+0000","last_change":"2023-11-06T21:00:33.984929+0000","last_active":"2023-11-06T21:00:33.989675+0000","last_peered":"2023-11-06T21:00:33.989675+0000","last_clean":"2023-11-06T21:00:33.989675+0000","last_became_active":"2023-11-06T21:00:33.984078+0000","last_became_peered":"2023-11-06T21:00:33.984078+0000","last_unstale":"2023-11-06T21:00:33.989675+0000","last_undegraded":"2023-11-06T21:00:33.989675+0000","last_fullsized":"2023-11-06T21:00:33.989675+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.983970+0000","last_change":"2023-11-06T21:00:33.506948+0000","last_active":"2023-11-06T21:00:33.983970+0000","last_peered":"2023-11-06T21:00:33.983970+0000","last_clean":"2023-11-06T21:00:33.983970+0000","last_became_active":"2023-11-06T21:00:33.506698+0000","last_became_peered":"2023-11-06T21:00:33.506698+0000","last_unstale":"2023-11-06T21:00:33.983970+0000","last_undegraded":"2023-11-06T21:00:33.983970+0000","last_fullsized":"2023-11-06T21:00:33.983970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.007818+0000","last_change":"2023-11-06T21:00:34.000384+0000","last_active":"2023-11-06T21:00:34.007818+0000","last_peered":"2023-11-06T21:00:34.007818+0000","last_clean":"2023-11-06T21:00:34.007818+0000","last_became_active":"2023-11-06T21:00:34.000039+0000","last_became_peered":"2023-11-06T21:00:34.000039+0000","last_unstale":"2023-11-06T21:00:34.007818+0000","last_undegraded":"2023-11-06T21:00:34.007818+0000","last_fullsized":"2023-11-06T21:00:34.007818+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3c","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002814+0000","last_change":"2023-11-06T21:00:33.500541+0000","last_active":"2023-11-06T21:00:34.002814+0000","last_peered":"2023-11-06T21:00:34.002814+0000","last_clean":"2023-11-06T21:00:34.002814+0000","last_became_active":"2023-11-06T21:00:33.500275+0000","last_became_peered":"2023-11-06T21:00:33.500275+0000","last_unstale":"2023-11-06T21:00:34.002814+0000","last_undegraded":"2023-11-06T21:00:34.002814+0000","last_fullsized":"2023-11-06T21:00:34.002814+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004280+0000","last_change":"2023-11-06T21:00:33.997018+0000","last_active":"2023-11-06T21:00:34.004280+0000","last_peered":"2023-11-06T21:00:34.004280+0000","last_clean":"2023-11-06T21:00:34.004280+0000","last_became_active":"2023-11-06T21:00:33.996115+0000","last_became_peered":"2023-11-06T21:00:33.996115+0000","last_unstale":"2023-11-06T21:00:34.004280+0000","last_undegraded":"2023-11-06T21:00:34.004280+0000","last_fullsized":"2023-11-06T21:00:34.004280+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3b","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003969+0000","last_change":"2023-11-06T21:00:33.491787+0000","last_active":"2023-11-06T21:00:34.003969+0000","last_peered":"2023-11-06T21:00:34.003969+0000","last_clean":"2023-11-06T21:00:34.003969+0000","last_became_active":"2023-11-06T21:00:33.491502+0000","last_became_peered":"2023-11-06T21:00:33.491502+0000","last_unstale":"2023-11-06T21:00:34.003969+0000","last_undegraded":"2023-11-06T21:00:34.003969+0000","last_fullsized":"2023-11-06T21:00:34.003969+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.983634+0000","last_change":"2023-11-06T21:00:33.979851+0000","last_active":"2023-11-06T21:00:33.983634+0000","last_peered":"2023-11-06T21:00:33.983634+0000","last_clean":"2023-11-06T21:00:33.983634+0000","last_became_active":"2023-11-06T21:00:33.979522+0000","last_became_peered":"2023-11-06T21:00:33.979522+0000","last_unstale":"2023-11-06T21:00:33.983634+0000","last_undegraded":"2023-11-06T21:00:33.983634+0000","last_fullsized":"2023-11-06T21:00:33.983634+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.995907+0000","last_change":"2023-11-06T21:00:33.497866+0000","last_active":"2023-11-06T21:00:33.995907+0000","last_peered":"2023-11-06T21:00:33.995907+0000","last_clean":"2023-11-06T21:00:33.995907+0000","last_became_active":"2023-11-06T21:00:33.497436+0000","last_became_peered":"2023-11-06T21:00:33.497436+0000","last_unstale":"2023-11-06T21:00:33.995907+0000","last_undegraded":"2023-11-06T21:00:33.995907+0000","last_fullsized":"2023-11-06T21:00:33.995907+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006526+0000","last_change":"2023-11-06T21:00:33.999689+0000","last_active":"2023-11-06T21:00:34.006526+0000","last_peered":"2023-11-06T21:00:34.006526+0000","last_clean":"2023-11-06T21:00:34.006526+0000","last_became_active":"2023-11-06T21:00:33.999408+0000","last_became_peered":"2023-11-06T21:00:33.999408+0000","last_unstale":"2023-11-06T21:00:34.006526+0000","last_undegraded":"2023-11-06T21:00:34.006526+0000","last_fullsized":"2023-11-06T21:00:34.006526+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.992594+0000","last_change":"2023-11-06T21:00:33.502350+0000","last_active":"2023-11-06T21:00:33.992594+0000","last_peered":"2023-11-06T21:00:33.992594+0000","last_clean":"2023-11-06T21:00:33.992594+0000","last_became_active":"2023-11-06T21:00:33.500784+0000","last_became_peered":"2023-11-06T21:00:33.500784+0000","last_unstale":"2023-11-06T21:00:33.992594+0000","last_undegraded":"2023-11-06T21:00:33.992594+0000","last_fullsized":"2023-11-06T21:00:33.992594+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001734+0000","last_change":"2023-11-06T21:00:33.995983+0000","last_active":"2023-11-06T21:00:34.001734+0000","last_peered":"2023-11-06T21:00:34.001734+0000","last_clean":"2023-11-06T21:00:34.001734+0000","last_became_active":"2023-11-06T21:00:33.995765+0000","last_became_peered":"2023-11-06T21:00:33.995765+0000","last_unstale":"2023-11-06T21:00:34.001734+0000","last_undegraded":"2023-11-06T21:00:34.001734+0000","last_fullsized":"2023-11-06T21:00:34.001734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.988395+0000","last_change":"2023-11-06T21:00:33.504202+0000","last_active":"2023-11-06T21:00:33.988395+0000","last_peered":"2023-11-06T21:00:33.988395+0000","last_clean":"2023-11-06T21:00:33.988395+0000","last_became_active":"2023-11-06T21:00:33.503854+0000","last_became_peered":"2023-11-06T21:00:33.503854+0000","last_unstale":"2023-11-06T21:00:33.988395+0000","last_undegraded":"2023-11-06T21:00:33.988395+0000","last_fullsized":"2023-11-06T21:00:33.988395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003532+0000","last_change":"2023-11-06T21:00:34.000628+0000","last_active":"2023-11-06T21:00:34.003532+0000","last_peered":"2023-11-06T21:00:34.003532+0000","last_clean":"2023-11-06T21:00:34.003532+0000","last_became_active":"2023-11-06T21:00:33.997694+0000","last_became_peered":"2023-11-06T21:00:33.997694+0000","last_unstale":"2023-11-06T21:00:34.003532+0000","last_undegraded":"2023-11-06T21:00:34.003532+0000","last_fullsized":"2023-11-06T21:00:34.003532+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.972482+0000","last_change":"2023-11-06T21:00:33.507854+0000","last_active":"2023-11-06T21:00:33.972482+0000","last_peered":"2023-11-06T21:00:33.972482+0000","last_clean":"2023-11-06T21:00:33.972482+0000","last_became_active":"2023-11-06T21:00:33.505595+0000","last_became_peered":"2023-11-06T21:00:33.505595+0000","last_unstale":"2023-11-06T21:00:33.972482+0000","last_undegraded":"2023-11-06T21:00:33.972482+0000","last_fullsized":"2023-11-06T21:00:33.972482+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003838+0000","last_change":"2023-11-06T21:00:33.990282+0000","last_active":"2023-11-06T21:00:34.003838+0000","last_peered":"2023-11-06T21:00:34.003838+0000","last_clean":"2023-11-06T21:00:34.003838+0000","last_became_active":"2023-11-06T21:00:33.990006+0000","last_became_peered":"2023-11-06T21:00:33.990006+0000","last_unstale":"2023-11-06T21:00:34.003838+0000","last_undegraded":"2023-11-06T21:00:34.003838+0000","last_fullsized":"2023-11-06T21:00:34.003838+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.985731+0000","last_change":"2023-11-06T21:00:33.501033+0000","last_active":"2023-11-06T21:00:33.985731+0000","last_peered":"2023-11-06T21:00:33.985731+0000","last_clean":"2023-11-06T21:00:33.985731+0000","last_became_active":"2023-11-06T21:00:33.496809+0000","last_became_peered":"2023-11-06T21:00:33.496809+0000","last_unstale":"2023-11-06T21:00:33.985731+0000","last_undegraded":"2023-11-06T21:00:33.985731+0000","last_fullsized":"2023-11-06T21:00:33.985731+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003537+0000","last_change":"2023-11-06T21:00:34.001011+0000","last_active":"2023-11-06T21:00:34.003537+0000","last_peered":"2023-11-06T21:00:34.003537+0000","last_clean":"2023-11-06T21:00:34.003537+0000","last_became_active":"2023-11-06T21:00:34.000705+0000","last_became_peered":"2023-11-06T21:00:34.000705+0000","last_unstale":"2023-11-06T21:00:34.003537+0000","last_undegraded":"2023-11-06T21:00:34.003537+0000","last_fullsized":"2023-11-06T21:00:34.003537+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981928+0000","last_change":"2023-11-06T21:00:33.500469+0000","last_active":"2023-11-06T21:00:33.981928+0000","last_peered":"2023-11-06T21:00:33.981928+0000","last_clean":"2023-11-06T21:00:33.981928+0000","last_became_active":"2023-11-06T21:00:33.500127+0000","last_became_peered":"2023-11-06T21:00:33.500127+0000","last_unstale":"2023-11-06T21:00:33.981928+0000","last_undegraded":"2023-11-06T21:00:33.981928+0000","last_fullsized":"2023-11-06T21:00:33.981928+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003903+0000","last_change":"2023-11-06T21:00:33.993716+0000","last_active":"2023-11-06T21:00:34.003903+0000","last_peered":"2023-11-06T21:00:34.003903+0000","last_clean":"2023-11-06T21:00:34.003903+0000","last_became_active":"2023-11-06T21:00:33.989757+0000","last_became_peered":"2023-11-06T21:00:33.989757+0000","last_unstale":"2023-11-06T21:00:34.003903+0000","last_undegraded":"2023-11-06T21:00:34.003903+0000","last_fullsized":"2023-11-06T21:00:34.003903+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.34","version":"113'4","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008142+0000","last_change":"2023-11-06T21:00:33.500855+0000","last_active":"2023-11-06T21:00:34.008142+0000","last_peered":"2023-11-06T21:00:34.008142+0000","last_clean":"2023-11-06T21:00:34.008142+0000","last_became_active":"2023-11-06T21:00:33.500233+0000","last_became_peered":"2023-11-06T21:00:33.500233+0000","last_unstale":"2023-11-06T21:00:34.008142+0000","last_undegraded":"2023-11-06T21:00:34.008142+0000","last_fullsized":"2023-11-06T21:00:34.008142+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008835+0000","last_change":"2023-11-06T21:00:34.004623+0000","last_active":"2023-11-06T21:00:34.008835+0000","last_peered":"2023-11-06T21:00:34.008835+0000","last_clean":"2023-11-06T21:00:34.008835+0000","last_became_active":"2023-11-06T21:00:34.004086+0000","last_became_peered":"2023-11-06T21:00:34.004086+0000","last_unstale":"2023-11-06T21:00:34.008835+0000","last_undegraded":"2023-11-06T21:00:34.008835+0000","last_fullsized":"2023-11-06T21:00:34.008835+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.33","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003847+0000","last_change":"2023-11-06T21:00:33.504508+0000","last_active":"2023-11-06T21:00:34.003847+0000","last_peered":"2023-11-06T21:00:34.003847+0000","last_clean":"2023-11-06T21:00:34.003847+0000","last_became_active":"2023-11-06T21:00:33.504187+0000","last_became_peered":"2023-11-06T21:00:33.504187+0000","last_unstale":"2023-11-06T21:00:34.003847+0000","last_undegraded":"2023-11-06T21:00:34.003847+0000","last_fullsized":"2023-11-06T21:00:34.003847+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.000543+0000","last_change":"2023-11-06T21:00:33.997037+0000","last_active":"2023-11-06T21:00:34.000543+0000","last_peered":"2023-11-06T21:00:34.000543+0000","last_clean":"2023-11-06T21:00:34.000543+0000","last_became_active":"2023-11-06T21:00:33.996731+0000","last_became_peered":"2023-11-06T21:00:33.996731+0000","last_unstale":"2023-11-06T21:00:34.000543+0000","last_undegraded":"2023-11-06T21:00:34.000543+0000","last_fullsized":"2023-11-06T21:00:34.000543+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973314+0000","last_change":"2023-11-06T21:00:33.502669+0000","last_active":"2023-11-06T21:00:33.973314+0000","last_peered":"2023-11-06T21:00:33.973314+0000","last_clean":"2023-11-06T21:00:33.973314+0000","last_became_active":"2023-11-06T21:00:33.502367+0000","last_became_peered":"2023-11-06T21:00:33.502367+0000","last_unstale":"2023-11-06T21:00:33.973314+0000","last_undegraded":"2023-11-06T21:00:33.973314+0000","last_fullsized":"2023-11-06T21:00:33.973314+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006311+0000","last_change":"2023-11-06T21:00:33.995410+0000","last_active":"2023-11-06T21:00:34.006311+0000","last_peered":"2023-11-06T21:00:34.006311+0000","last_clean":"2023-11-06T21:00:34.006311+0000","last_became_active":"2023-11-06T21:00:33.995082+0000","last_became_peered":"2023-11-06T21:00:33.995082+0000","last_unstale":"2023-11-06T21:00:34.006311+0000","last_undegraded":"2023-11-06T21:00:34.006311+0000","last_fullsized":"2023-11-06T21:00:34.006311+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.992722+0000","last_change":"2023-11-06T21:00:33.497168+0000","last_active":"2023-11-06T21:00:33.992722+0000","last_peered":"2023-11-06T21:00:33.992722+0000","last_clean":"2023-11-06T21:00:33.992722+0000","last_became_active":"2023-11-06T21:00:33.495184+0000","last_became_peered":"2023-11-06T21:00:33.495184+0000","last_unstale":"2023-11-06T21:00:33.992722+0000","last_undegraded":"2023-11-06T21:00:33.992722+0000","last_fullsized":"2023-11-06T21:00:33.992722+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002193+0000","last_change":"2023-11-06T21:00:33.990185+0000","last_active":"2023-11-06T21:00:34.002193+0000","last_peered":"2023-11-06T21:00:34.002193+0000","last_clean":"2023-11-06T21:00:34.002193+0000","last_became_active":"2023-11-06T21:00:33.989838+0000","last_became_peered":"2023-11-06T21:00:33.989838+0000","last_unstale":"2023-11-06T21:00:34.002193+0000","last_undegraded":"2023-11-06T21:00:34.002193+0000","last_fullsized":"2023-11-06T21:00:34.002193+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.30","version":"113'4","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004597+0000","last_change":"2023-11-06T21:00:33.504781+0000","last_active":"2023-11-06T21:00:34.004597+0000","last_peered":"2023-11-06T21:00:34.004597+0000","last_clean":"2023-11-06T21:00:34.004597+0000","last_became_active":"2023-11-06T21:00:33.504455+0000","last_became_peered":"2023-11-06T21:00:33.504455+0000","last_unstale":"2023-11-06T21:00:34.004597+0000","last_undegraded":"2023-11-06T21:00:34.004597+0000","last_fullsized":"2023-11-06T21:00:34.004597+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008468+0000","last_change":"2023-11-06T21:00:34.003399+0000","last_active":"2023-11-06T21:00:34.008468+0000","last_peered":"2023-11-06T21:00:34.008468+0000","last_clean":"2023-11-06T21:00:34.008468+0000","last_became_active":"2023-11-06T21:00:34.002952+0000","last_became_peered":"2023-11-06T21:00:34.002952+0000","last_unstale":"2023-11-06T21:00:34.008468+0000","last_undegraded":"2023-11-06T21:00:34.008468+0000","last_fullsized":"2023-11-06T21:00:34.008468+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.991927+0000","last_change":"2023-11-06T21:00:33.500695+0000","last_active":"2023-11-06T21:00:33.991927+0000","last_peered":"2023-11-06T21:00:33.991927+0000","last_clean":"2023-11-06T21:00:33.991927+0000","last_became_active":"2023-11-06T21:00:33.500323+0000","last_became_peered":"2023-11-06T21:00:33.500323+0000","last_unstale":"2023-11-06T21:00:33.991927+0000","last_undegraded":"2023-11-06T21:00:33.991927+0000","last_fullsized":"2023-11-06T21:00:33.991927+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002209+0000","last_change":"2023-11-06T21:00:33.995121+0000","last_active":"2023-11-06T21:00:34.002209+0000","last_peered":"2023-11-06T21:00:34.002209+0000","last_clean":"2023-11-06T21:00:34.002209+0000","last_became_active":"2023-11-06T21:00:33.994566+0000","last_became_peered":"2023-11-06T21:00:33.994566+0000","last_unstale":"2023-11-06T21:00:34.002209+0000","last_undegraded":"2023-11-06T21:00:34.002209+0000","last_fullsized":"2023-11-06T21:00:34.002209+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.980998+0000","last_change":"2023-11-06T21:00:33.509086+0000","last_active":"2023-11-06T21:00:33.980998+0000","last_peered":"2023-11-06T21:00:33.980998+0000","last_clean":"2023-11-06T21:00:33.980998+0000","last_became_active":"2023-11-06T21:00:33.508718+0000","last_became_peered":"2023-11-06T21:00:33.508718+0000","last_unstale":"2023-11-06T21:00:33.980998+0000","last_undegraded":"2023-11-06T21:00:33.980998+0000","last_fullsized":"2023-11-06T21:00:33.980998+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.000880+0000","last_change":"2023-11-06T21:00:33.996484+0000","last_active":"2023-11-06T21:00:34.000880+0000","last_peered":"2023-11-06T21:00:34.000880+0000","last_clean":"2023-11-06T21:00:34.000880+0000","last_became_active":"2023-11-06T21:00:33.996140+0000","last_became_peered":"2023-11-06T21:00:33.996140+0000","last_unstale":"2023-11-06T21:00:34.000880+0000","last_undegraded":"2023-11-06T21:00:34.000880+0000","last_fullsized":"2023-11-06T21:00:34.000880+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2d","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003046+0000","last_change":"2023-11-06T21:00:33.507896+0000","last_active":"2023-11-06T21:00:34.003046+0000","last_peered":"2023-11-06T21:00:34.003046+0000","last_clean":"2023-11-06T21:00:34.003046+0000","last_became_active":"2023-11-06T21:00:33.507593+0000","last_became_peered":"2023-11-06T21:00:33.507593+0000","last_unstale":"2023-11-06T21:00:34.003046+0000","last_undegraded":"2023-11-06T21:00:34.003046+0000","last_fullsized":"2023-11-06T21:00:34.003046+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008744+0000","last_change":"2023-11-06T21:00:34.004324+0000","last_active":"2023-11-06T21:00:34.008744+0000","last_peered":"2023-11-06T21:00:34.008744+0000","last_clean":"2023-11-06T21:00:34.008744+0000","last_became_active":"2023-11-06T21:00:34.003957+0000","last_became_peered":"2023-11-06T21:00:34.003957+0000","last_unstale":"2023-11-06T21:00:34.008744+0000","last_undegraded":"2023-11-06T21:00:34.008744+0000","last_fullsized":"2023-11-06T21:00:34.008744+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.975569+0000","last_change":"2023-11-06T21:00:33.504626+0000","last_active":"2023-11-06T21:00:33.975569+0000","last_peered":"2023-11-06T21:00:33.975569+0000","last_clean":"2023-11-06T21:00:33.975569+0000","last_became_active":"2023-11-06T21:00:33.502414+0000","last_became_peered":"2023-11-06T21:00:33.502414+0000","last_unstale":"2023-11-06T21:00:33.975569+0000","last_undegraded":"2023-11-06T21:00:33.975569+0000","last_fullsized":"2023-11-06T21:00:33.975569+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004059+0000","last_change":"2023-11-06T21:00:33.991346+0000","last_active":"2023-11-06T21:00:34.004059+0000","last_peered":"2023-11-06T21:00:34.004059+0000","last_clean":"2023-11-06T21:00:34.004059+0000","last_became_active":"2023-11-06T21:00:33.989703+0000","last_became_peered":"2023-11-06T21:00:33.989703+0000","last_unstale":"2023-11-06T21:00:34.004059+0000","last_undegraded":"2023-11-06T21:00:34.004059+0000","last_fullsized":"2023-11-06T21:00:34.004059+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2b","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002099+0000","last_change":"2023-11-06T21:00:33.504535+0000","last_active":"2023-11-06T21:00:34.002099+0000","last_peered":"2023-11-06T21:00:34.002099+0000","last_clean":"2023-11-06T21:00:34.002099+0000","last_became_active":"2023-11-06T21:00:33.504217+0000","last_became_peered":"2023-11-06T21:00:33.504217+0000","last_unstale":"2023-11-06T21:00:34.002099+0000","last_undegraded":"2023-11-06T21:00:34.002099+0000","last_fullsized":"2023-11-06T21:00:34.002099+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989760+0000","last_change":"2023-11-06T21:00:33.986263+0000","last_active":"2023-11-06T21:00:33.989760+0000","last_peered":"2023-11-06T21:00:33.989760+0000","last_clean":"2023-11-06T21:00:33.989760+0000","last_became_active":"2023-11-06T21:00:33.985933+0000","last_became_peered":"2023-11-06T21:00:33.985933+0000","last_unstale":"2023-11-06T21:00:33.989760+0000","last_undegraded":"2023-11-06T21:00:33.989760+0000","last_fullsized":"2023-11-06T21:00:33.989760+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973209+0000","last_change":"2023-11-06T21:00:33.506301+0000","last_active":"2023-11-06T21:00:33.973209+0000","last_peered":"2023-11-06T21:00:33.973209+0000","last_clean":"2023-11-06T21:00:33.973209+0000","last_became_active":"2023-11-06T21:00:33.506006+0000","last_became_peered":"2023-11-06T21:00:33.506006+0000","last_unstale":"2023-11-06T21:00:33.973209+0000","last_undegraded":"2023-11-06T21:00:33.973209+0000","last_fullsized":"2023-11-06T21:00:33.973209+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.999602+0000","last_change":"2023-11-06T21:00:33.990242+0000","last_active":"2023-11-06T21:00:33.999602+0000","last_peered":"2023-11-06T21:00:33.999602+0000","last_clean":"2023-11-06T21:00:33.999602+0000","last_became_active":"2023-11-06T21:00:33.989932+0000","last_became_peered":"2023-11-06T21:00:33.989932+0000","last_unstale":"2023-11-06T21:00:33.999602+0000","last_undegraded":"2023-11-06T21:00:33.999602+0000","last_fullsized":"2023-11-06T21:00:33.999602+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989012+0000","last_change":"2023-11-06T21:00:33.508959+0000","last_active":"2023-11-06T21:00:33.989012+0000","last_peered":"2023-11-06T21:00:33.989012+0000","last_clean":"2023-11-06T21:00:33.989012+0000","last_became_active":"2023-11-06T21:00:33.508590+0000","last_became_peered":"2023-11-06T21:00:33.508590+0000","last_unstale":"2023-11-06T21:00:33.989012+0000","last_undegraded":"2023-11-06T21:00:33.989012+0000","last_fullsized":"2023-11-06T21:00:33.989012+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001752+0000","last_change":"2023-11-06T21:00:33.997337+0000","last_active":"2023-11-06T21:00:34.001752+0000","last_peered":"2023-11-06T21:00:34.001752+0000","last_clean":"2023-11-06T21:00:34.001752+0000","last_became_active":"2023-11-06T21:00:33.996737+0000","last_became_peered":"2023-11-06T21:00:33.996737+0000","last_unstale":"2023-11-06T21:00:34.001752+0000","last_undegraded":"2023-11-06T21:00:34.001752+0000","last_fullsized":"2023-11-06T21:00:34.001752+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981086+0000","last_change":"2023-11-06T21:00:33.504796+0000","last_active":"2023-11-06T21:00:33.981086+0000","last_peered":"2023-11-06T21:00:33.981086+0000","last_clean":"2023-11-06T21:00:33.981086+0000","last_became_active":"2023-11-06T21:00:33.503331+0000","last_became_peered":"2023-11-06T21:00:33.503331+0000","last_unstale":"2023-11-06T21:00:33.981086+0000","last_undegraded":"2023-11-06T21:00:33.981086+0000","last_fullsized":"2023-11-06T21:00:33.981086+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.994997+0000","last_change":"2023-11-06T21:00:33.983077+0000","last_active":"2023-11-06T21:00:33.994997+0000","last_peered":"2023-11-06T21:00:33.994997+0000","last_clean":"2023-11-06T21:00:33.994997+0000","last_became_active":"2023-11-06T21:00:33.982804+0000","last_became_peered":"2023-11-06T21:00:33.982804+0000","last_unstale":"2023-11-06T21:00:33.994997+0000","last_undegraded":"2023-11-06T21:00:33.994997+0000","last_fullsized":"2023-11-06T21:00:33.994997+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981175+0000","last_change":"2023-11-06T21:00:33.513913+0000","last_active":"2023-11-06T21:00:33.981175+0000","last_peered":"2023-11-06T21:00:33.981175+0000","last_clean":"2023-11-06T21:00:33.981175+0000","last_became_active":"2023-11-06T21:00:33.513590+0000","last_became_peered":"2023-11-06T21:00:33.513590+0000","last_unstale":"2023-11-06T21:00:33.981175+0000","last_undegraded":"2023-11-06T21:00:33.981175+0000","last_fullsized":"2023-11-06T21:00:33.981175+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.000512+0000","last_change":"2023-11-06T21:00:33.997347+0000","last_active":"2023-11-06T21:00:34.000512+0000","last_peered":"2023-11-06T21:00:34.000512+0000","last_clean":"2023-11-06T21:00:34.000512+0000","last_became_active":"2023-11-06T21:00:33.996154+0000","last_became_peered":"2023-11-06T21:00:33.996154+0000","last_unstale":"2023-11-06T21:00:34.000512+0000","last_undegraded":"2023-11-06T21:00:34.000512+0000","last_fullsized":"2023-11-06T21:00:34.000512+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974900+0000","last_change":"2023-11-06T21:00:33.493162+0000","last_active":"2023-11-06T21:00:33.974900+0000","last_peered":"2023-11-06T21:00:33.974900+0000","last_clean":"2023-11-06T21:00:33.974900+0000","last_became_active":"2023-11-06T21:00:33.490940+0000","last_became_peered":"2023-11-06T21:00:33.490940+0000","last_unstale":"2023-11-06T21:00:33.974900+0000","last_undegraded":"2023-11-06T21:00:33.974900+0000","last_fullsized":"2023-11-06T21:00:33.974900+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003932+0000","last_change":"2023-11-06T21:00:33.998602+0000","last_active":"2023-11-06T21:00:34.003932+0000","last_peered":"2023-11-06T21:00:34.003932+0000","last_clean":"2023-11-06T21:00:34.003932+0000","last_became_active":"2023-11-06T21:00:33.995194+0000","last_became_peered":"2023-11-06T21:00:33.995194+0000","last_unstale":"2023-11-06T21:00:34.003932+0000","last_undegraded":"2023-11-06T21:00:34.003932+0000","last_fullsized":"2023-11-06T21:00:34.003932+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974604+0000","last_change":"2023-11-06T21:00:33.494908+0000","last_active":"2023-11-06T21:00:33.974604+0000","last_peered":"2023-11-06T21:00:33.974604+0000","last_clean":"2023-11-06T21:00:33.974604+0000","last_became_active":"2023-11-06T21:00:33.494624+0000","last_became_peered":"2023-11-06T21:00:33.494624+0000","last_unstale":"2023-11-06T21:00:33.974604+0000","last_undegraded":"2023-11-06T21:00:33.974604+0000","last_fullsized":"2023-11-06T21:00:33.974604+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001486+0000","last_change":"2023-11-06T21:00:33.996894+0000","last_active":"2023-11-06T21:00:34.001486+0000","last_peered":"2023-11-06T21:00:34.001486+0000","last_clean":"2023-11-06T21:00:34.001486+0000","last_became_active":"2023-11-06T21:00:33.996323+0000","last_became_peered":"2023-11-06T21:00:33.996323+0000","last_unstale":"2023-11-06T21:00:34.001486+0000","last_undegraded":"2023-11-06T21:00:34.001486+0000","last_fullsized":"2023-11-06T21:00:34.001486+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971779+0000","last_change":"2023-11-06T21:00:33.507514+0000","last_active":"2023-11-06T21:00:33.971779+0000","last_peered":"2023-11-06T21:00:33.971779+0000","last_clean":"2023-11-06T21:00:33.971779+0000","last_became_active":"2023-11-06T21:00:33.507243+0000","last_became_peered":"2023-11-06T21:00:33.507243+0000","last_unstale":"2023-11-06T21:00:33.971779+0000","last_undegraded":"2023-11-06T21:00:33.971779+0000","last_fullsized":"2023-11-06T21:00:33.971779+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008852+0000","last_change":"2023-11-06T21:00:34.004845+0000","last_active":"2023-11-06T21:00:34.008852+0000","last_peered":"2023-11-06T21:00:34.008852+0000","last_clean":"2023-11-06T21:00:34.008852+0000","last_became_active":"2023-11-06T21:00:34.004567+0000","last_became_peered":"2023-11-06T21:00:34.004567+0000","last_unstale":"2023-11-06T21:00:34.008852+0000","last_undegraded":"2023-11-06T21:00:34.008852+0000","last_fullsized":"2023-11-06T21:00:34.008852+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989125+0000","last_change":"2023-11-06T21:00:33.501114+0000","last_active":"2023-11-06T21:00:33.989125+0000","last_peered":"2023-11-06T21:00:33.989125+0000","last_clean":"2023-11-06T21:00:33.989125+0000","last_became_active":"2023-11-06T21:00:33.500796+0000","last_became_peered":"2023-11-06T21:00:33.500796+0000","last_unstale":"2023-11-06T21:00:33.989125+0000","last_undegraded":"2023-11-06T21:00:33.989125+0000","last_fullsized":"2023-11-06T21:00:33.989125+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.999943+0000","last_change":"2023-11-06T21:00:33.993642+0000","last_active":"2023-11-06T21:00:33.999943+0000","last_peered":"2023-11-06T21:00:33.999943+0000","last_clean":"2023-11-06T21:00:33.999943+0000","last_became_active":"2023-11-06T21:00:33.993165+0000","last_became_peered":"2023-11-06T21:00:33.993165+0000","last_unstale":"2023-11-06T21:00:33.999943+0000","last_undegraded":"2023-11-06T21:00:33.999943+0000","last_fullsized":"2023-11-06T21:00:33.999943+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973100+0000","last_change":"2023-11-06T21:00:33.508157+0000","last_active":"2023-11-06T21:00:33.973100+0000","last_peered":"2023-11-06T21:00:33.973100+0000","last_clean":"2023-11-06T21:00:33.973100+0000","last_became_active":"2023-11-06T21:00:33.507854+0000","last_became_peered":"2023-11-06T21:00:33.507854+0000","last_unstale":"2023-11-06T21:00:33.973100+0000","last_undegraded":"2023-11-06T21:00:33.973100+0000","last_fullsized":"2023-11-06T21:00:33.973100+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989604+0000","last_change":"2023-11-06T21:00:33.984857+0000","last_active":"2023-11-06T21:00:33.989604+0000","last_peered":"2023-11-06T21:00:33.989604+0000","last_clean":"2023-11-06T21:00:33.989604+0000","last_became_active":"2023-11-06T21:00:33.984584+0000","last_became_peered":"2023-11-06T21:00:33.984584+0000","last_unstale":"2023-11-06T21:00:33.989604+0000","last_undegraded":"2023-11-06T21:00:33.989604+0000","last_fullsized":"2023-11-06T21:00:33.989604+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.980915+0000","last_change":"2023-11-06T21:00:33.494846+0000","last_active":"2023-11-06T21:00:33.980915+0000","last_peered":"2023-11-06T21:00:33.980915+0000","last_clean":"2023-11-06T21:00:33.980915+0000","last_became_active":"2023-11-06T21:00:33.494551+0000","last_became_peered":"2023-11-06T21:00:33.494551+0000","last_unstale":"2023-11-06T21:00:33.980915+0000","last_undegraded":"2023-11-06T21:00:33.980915+0000","last_fullsized":"2023-11-06T21:00:33.980915+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001713+0000","last_change":"2023-11-06T21:00:33.995501+0000","last_active":"2023-11-06T21:00:34.001713+0000","last_peered":"2023-11-06T21:00:34.001713+0000","last_clean":"2023-11-06T21:00:34.001713+0000","last_became_active":"2023-11-06T21:00:33.992247+0000","last_became_peered":"2023-11-06T21:00:33.992247+0000","last_unstale":"2023-11-06T21:00:34.001713+0000","last_undegraded":"2023-11-06T21:00:34.001713+0000","last_fullsized":"2023-11-06T21:00:34.001713+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981167+0000","last_change":"2023-11-06T21:00:33.493627+0000","last_active":"2023-11-06T21:00:33.981167+0000","last_peered":"2023-11-06T21:00:33.981167+0000","last_clean":"2023-11-06T21:00:33.981167+0000","last_became_active":"2023-11-06T21:00:33.493323+0000","last_became_peered":"2023-11-06T21:00:33.493323+0000","last_unstale":"2023-11-06T21:00:33.981167+0000","last_undegraded":"2023-11-06T21:00:33.981167+0000","last_fullsized":"2023-11-06T21:00:33.981167+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.007749+0000","last_change":"2023-11-06T21:00:33.999025+0000","last_active":"2023-11-06T21:00:34.007749+0000","last_peered":"2023-11-06T21:00:34.007749+0000","last_clean":"2023-11-06T21:00:34.007749+0000","last_became_active":"2023-11-06T21:00:33.998592+0000","last_became_peered":"2023-11-06T21:00:33.998592+0000","last_unstale":"2023-11-06T21:00:34.007749+0000","last_undegraded":"2023-11-06T21:00:34.007749+0000","last_fullsized":"2023-11-06T21:00:34.007749+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001335+0000","last_change":"2023-11-06T21:00:33.501241+0000","last_active":"2023-11-06T21:00:34.001335+0000","last_peered":"2023-11-06T21:00:34.001335+0000","last_clean":"2023-11-06T21:00:34.001335+0000","last_became_active":"2023-11-06T21:00:33.500900+0000","last_became_peered":"2023-11-06T21:00:33.500900+0000","last_unstale":"2023-11-06T21:00:34.001335+0000","last_undegraded":"2023-11-06T21:00:34.001335+0000","last_fullsized":"2023-11-06T21:00:34.001335+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008600+0000","last_change":"2023-11-06T21:00:34.003507+0000","last_active":"2023-11-06T21:00:34.008600+0000","last_peered":"2023-11-06T21:00:34.008600+0000","last_clean":"2023-11-06T21:00:34.008600+0000","last_became_active":"2023-11-06T21:00:34.003233+0000","last_became_peered":"2023-11-06T21:00:34.003233+0000","last_unstale":"2023-11-06T21:00:34.008600+0000","last_undegraded":"2023-11-06T21:00:34.008600+0000","last_fullsized":"2023-11-06T21:00:34.008600+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006314+0000","last_change":"2023-11-06T21:00:33.509163+0000","last_active":"2023-11-06T21:00:34.006314+0000","last_peered":"2023-11-06T21:00:34.006314+0000","last_clean":"2023-11-06T21:00:34.006314+0000","last_became_active":"2023-11-06T21:00:33.508745+0000","last_became_peered":"2023-11-06T21:00:33.508745+0000","last_unstale":"2023-11-06T21:00:34.006314+0000","last_undegraded":"2023-11-06T21:00:34.006314+0000","last_fullsized":"2023-11-06T21:00:34.006314+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004164+0000","last_change":"2023-11-06T21:00:33.998100+0000","last_active":"2023-11-06T21:00:34.004164+0000","last_peered":"2023-11-06T21:00:34.004164+0000","last_clean":"2023-11-06T21:00:34.004164+0000","last_became_active":"2023-11-06T21:00:33.996793+0000","last_became_peered":"2023-11-06T21:00:33.996793+0000","last_unstale":"2023-11-06T21:00:34.004164+0000","last_undegraded":"2023-11-06T21:00:34.004164+0000","last_fullsized":"2023-11-06T21:00:34.004164+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.000122+0000","last_change":"2023-11-06T21:00:33.504531+0000","last_active":"2023-11-06T21:00:34.000122+0000","last_peered":"2023-11-06T21:00:34.000122+0000","last_clean":"2023-11-06T21:00:34.000122+0000","last_became_active":"2023-11-06T21:00:33.504164+0000","last_became_peered":"2023-11-06T21:00:33.504164+0000","last_unstale":"2023-11-06T21:00:34.000122+0000","last_undegraded":"2023-11-06T21:00:34.000122+0000","last_fullsized":"2023-11-06T21:00:34.000122+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004474+0000","last_change":"2023-11-06T21:00:34.001550+0000","last_active":"2023-11-06T21:00:34.004474+0000","last_peered":"2023-11-06T21:00:34.004474+0000","last_clean":"2023-11-06T21:00:34.004474+0000","last_became_active":"2023-11-06T21:00:34.001233+0000","last_became_peered":"2023-11-06T21:00:34.001233+0000","last_unstale":"2023-11-06T21:00:34.004474+0000","last_undegraded":"2023-11-06T21:00:34.004474+0000","last_fullsized":"2023-11-06T21:00:34.004474+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.988863+0000","last_change":"2023-11-06T21:00:33.498742+0000","last_active":"2023-11-06T21:00:33.988863+0000","last_peered":"2023-11-06T21:00:33.988863+0000","last_clean":"2023-11-06T21:00:33.988863+0000","last_became_active":"2023-11-06T21:00:33.498013+0000","last_became_peered":"2023-11-06T21:00:33.498013+0000","last_unstale":"2023-11-06T21:00:33.988863+0000","last_undegraded":"2023-11-06T21:00:33.988863+0000","last_fullsized":"2023-11-06T21:00:33.988863+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002489+0000","last_change":"2023-11-06T21:00:33.990825+0000","last_active":"2023-11-06T21:00:34.002489+0000","last_peered":"2023-11-06T21:00:34.002489+0000","last_clean":"2023-11-06T21:00:34.002489+0000","last_became_active":"2023-11-06T21:00:33.987540+0000","last_became_peered":"2023-11-06T21:00:33.987540+0000","last_unstale":"2023-11-06T21:00:34.002489+0000","last_undegraded":"2023-11-06T21:00:34.002489+0000","last_fullsized":"2023-11-06T21:00:34.002489+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981916+0000","last_change":"2023-11-06T21:00:33.500170+0000","last_active":"2023-11-06T21:00:33.981916+0000","last_peered":"2023-11-06T21:00:33.981916+0000","last_clean":"2023-11-06T21:00:33.981916+0000","last_became_active":"2023-11-06T21:00:33.499851+0000","last_became_peered":"2023-11-06T21:00:33.499851+0000","last_unstale":"2023-11-06T21:00:33.981916+0000","last_undegraded":"2023-11-06T21:00:33.981916+0000","last_fullsized":"2023-11-06T21:00:33.981916+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.983613+0000","last_change":"2023-11-06T21:00:33.979672+0000","last_active":"2023-11-06T21:00:33.983613+0000","last_peered":"2023-11-06T21:00:33.983613+0000","last_clean":"2023-11-06T21:00:33.983613+0000","last_became_active":"2023-11-06T21:00:33.979373+0000","last_became_peered":"2023-11-06T21:00:33.979373+0000","last_unstale":"2023-11-06T21:00:33.983613+0000","last_undegraded":"2023-11-06T21:00:33.983613+0000","last_fullsized":"2023-11-06T21:00:33.983613+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.988188+0000","last_change":"2023-11-06T21:00:33.506600+0000","last_active":"2023-11-06T21:00:33.988188+0000","last_peered":"2023-11-06T21:00:33.988188+0000","last_clean":"2023-11-06T21:00:33.988188+0000","last_became_active":"2023-11-06T21:00:33.506285+0000","last_became_peered":"2023-11-06T21:00:33.506285+0000","last_unstale":"2023-11-06T21:00:33.988188+0000","last_undegraded":"2023-11-06T21:00:33.988188+0000","last_fullsized":"2023-11-06T21:00:33.988188+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003933+0000","last_change":"2023-11-06T21:00:33.999110+0000","last_active":"2023-11-06T21:00:34.003933+0000","last_peered":"2023-11-06T21:00:34.003933+0000","last_clean":"2023-11-06T21:00:34.003933+0000","last_became_active":"2023-11-06T21:00:33.998611+0000","last_became_peered":"2023-11-06T21:00:33.998611+0000","last_unstale":"2023-11-06T21:00:34.003933+0000","last_undegraded":"2023-11-06T21:00:34.003933+0000","last_fullsized":"2023-11-06T21:00:34.003933+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974256+0000","last_change":"2023-11-06T21:00:33.508266+0000","last_active":"2023-11-06T21:00:33.974256+0000","last_peered":"2023-11-06T21:00:33.974256+0000","last_clean":"2023-11-06T21:00:33.974256+0000","last_became_active":"2023-11-06T21:00:33.507866+0000","last_became_peered":"2023-11-06T21:00:33.507866+0000","last_unstale":"2023-11-06T21:00:33.974256+0000","last_undegraded":"2023-11-06T21:00:33.974256+0000","last_fullsized":"2023-11-06T21:00:33.974256+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.998336+0000","last_change":"2023-11-06T21:00:33.989497+0000","last_active":"2023-11-06T21:00:33.998336+0000","last_peered":"2023-11-06T21:00:33.998336+0000","last_clean":"2023-11-06T21:00:33.998336+0000","last_became_active":"2023-11-06T21:00:33.988697+0000","last_became_peered":"2023-11-06T21:00:33.988697+0000","last_unstale":"2023-11-06T21:00:33.998336+0000","last_undegraded":"2023-11-06T21:00:33.998336+0000","last_fullsized":"2023-11-06T21:00:33.998336+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971644+0000","last_change":"2023-11-06T21:00:33.502195+0000","last_active":"2023-11-06T21:00:33.971644+0000","last_peered":"2023-11-06T21:00:33.971644+0000","last_clean":"2023-11-06T21:00:33.971644+0000","last_became_active":"2023-11-06T21:00:33.501125+0000","last_became_peered":"2023-11-06T21:00:33.501125+0000","last_unstale":"2023-11-06T21:00:33.971644+0000","last_undegraded":"2023-11-06T21:00:33.971644+0000","last_fullsized":"2023-11-06T21:00:33.971644+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.998410+0000","last_change":"2023-11-06T21:00:33.993666+0000","last_active":"2023-11-06T21:00:33.998410+0000","last_peered":"2023-11-06T21:00:33.998410+0000","last_clean":"2023-11-06T21:00:33.998410+0000","last_became_active":"2023-11-06T21:00:33.990418+0000","last_became_peered":"2023-11-06T21:00:33.990418+0000","last_unstale":"2023-11-06T21:00:33.998410+0000","last_undegraded":"2023-11-06T21:00:33.998410+0000","last_fullsized":"2023-11-06T21:00:33.998410+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.972534+0000","last_change":"2023-11-06T21:00:33.505514+0000","last_active":"2023-11-06T21:00:33.972534+0000","last_peered":"2023-11-06T21:00:33.972534+0000","last_clean":"2023-11-06T21:00:33.972534+0000","last_became_active":"2023-11-06T21:00:33.505168+0000","last_became_peered":"2023-11-06T21:00:33.505168+0000","last_unstale":"2023-11-06T21:00:33.972534+0000","last_undegraded":"2023-11-06T21:00:33.972534+0000","last_fullsized":"2023-11-06T21:00:33.972534+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003858+0000","last_change":"2023-11-06T21:00:33.991115+0000","last_active":"2023-11-06T21:00:34.003858+0000","last_peered":"2023-11-06T21:00:34.003858+0000","last_clean":"2023-11-06T21:00:34.003858+0000","last_became_active":"2023-11-06T21:00:33.990842+0000","last_became_peered":"2023-11-06T21:00:33.990842+0000","last_unstale":"2023-11-06T21:00:34.003858+0000","last_undegraded":"2023-11-06T21:00:34.003858+0000","last_fullsized":"2023-11-06T21:00:34.003858+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973823+0000","last_change":"2023-11-06T21:00:33.510936+0000","last_active":"2023-11-06T21:00:33.973823+0000","last_peered":"2023-11-06T21:00:33.973823+0000","last_clean":"2023-11-06T21:00:33.973823+0000","last_became_active":"2023-11-06T21:00:33.510597+0000","last_became_peered":"2023-11-06T21:00:33.510597+0000","last_unstale":"2023-11-06T21:00:33.973823+0000","last_undegraded":"2023-11-06T21:00:33.973823+0000","last_fullsized":"2023-11-06T21:00:33.973823+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.007822+0000","last_change":"2023-11-06T21:00:33.998999+0000","last_active":"2023-11-06T21:00:34.007822+0000","last_peered":"2023-11-06T21:00:34.007822+0000","last_clean":"2023-11-06T21:00:34.007822+0000","last_became_active":"2023-11-06T21:00:33.998625+0000","last_became_peered":"2023-11-06T21:00:33.998625+0000","last_unstale":"2023-11-06T21:00:34.007822+0000","last_undegraded":"2023-11-06T21:00:34.007822+0000","last_fullsized":"2023-11-06T21:00:34.007822+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"113'3","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.007569+0000","last_change":"2023-11-06T21:00:33.508112+0000","last_active":"2023-11-06T21:00:34.007569+0000","last_peered":"2023-11-06T21:00:34.007569+0000","last_clean":"2023-11-06T21:00:34.007569+0000","last_became_active":"2023-11-06T21:00:33.507601+0000","last_became_peered":"2023-11-06T21:00:33.507601+0000","last_unstale":"2023-11-06T21:00:34.007569+0000","last_undegraded":"2023-11-06T21:00:34.007569+0000","last_fullsized":"2023-11-06T21:00:34.007569+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.009041+0000","last_change":"2023-11-06T21:00:34.005671+0000","last_active":"2023-11-06T21:00:34.009041+0000","last_peered":"2023-11-06T21:00:34.009041+0000","last_clean":"2023-11-06T21:00:34.009041+0000","last_became_active":"2023-11-06T21:00:34.005392+0000","last_became_peered":"2023-11-06T21:00:34.005392+0000","last_unstale":"2023-11-06T21:00:34.009041+0000","last_undegraded":"2023-11-06T21:00:34.009041+0000","last_fullsized":"2023-11-06T21:00:34.009041+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004731+0000","last_change":"2023-11-06T21:00:33.512547+0000","last_active":"2023-11-06T21:00:34.004731+0000","last_peered":"2023-11-06T21:00:34.004731+0000","last_clean":"2023-11-06T21:00:34.004731+0000","last_became_active":"2023-11-06T21:00:33.512284+0000","last_became_peered":"2023-11-06T21:00:33.512284+0000","last_unstale":"2023-11-06T21:00:34.004731+0000","last_undegraded":"2023-11-06T21:00:34.004731+0000","last_fullsized":"2023-11-06T21:00:34.004731+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006085+0000","last_change":"2023-11-06T21:00:33.994039+0000","last_active":"2023-11-06T21:00:34.006085+0000","last_peered":"2023-11-06T21:00:34.006085+0000","last_clean":"2023-11-06T21:00:34.006085+0000","last_became_active":"2023-11-06T21:00:33.993609+0000","last_became_peered":"2023-11-06T21:00:33.993609+0000","last_unstale":"2023-11-06T21:00:34.006085+0000","last_undegraded":"2023-11-06T21:00:34.006085+0000","last_fullsized":"2023-11-06T21:00:34.006085+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.000581+0000","last_change":"2023-11-06T21:00:33.507648+0000","last_active":"2023-11-06T21:00:34.000581+0000","last_peered":"2023-11-06T21:00:34.000581+0000","last_clean":"2023-11-06T21:00:34.000581+0000","last_became_active":"2023-11-06T21:00:33.505304+0000","last_became_peered":"2023-11-06T21:00:33.505304+0000","last_unstale":"2023-11-06T21:00:34.000581+0000","last_undegraded":"2023-11-06T21:00:34.000581+0000","last_fullsized":"2023-11-06T21:00:34.000581+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989476+0000","last_change":"2023-11-06T21:00:33.984386+0000","last_active":"2023-11-06T21:00:33.989476+0000","last_peered":"2023-11-06T21:00:33.989476+0000","last_clean":"2023-11-06T21:00:33.989476+0000","last_became_active":"2023-11-06T21:00:33.984017+0000","last_became_peered":"2023-11-06T21:00:33.984017+0000","last_unstale":"2023-11-06T21:00:33.989476+0000","last_undegraded":"2023-11-06T21:00:33.989476+0000","last_fullsized":"2023-11-06T21:00:33.989476+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.980656+0000","last_change":"2023-11-06T21:00:33.516349+0000","last_active":"2023-11-06T21:00:33.980656+0000","last_peered":"2023-11-06T21:00:33.980656+0000","last_clean":"2023-11-06T21:00:33.980656+0000","last_became_active":"2023-11-06T21:00:33.516078+0000","last_became_peered":"2023-11-06T21:00:33.516078+0000","last_unstale":"2023-11-06T21:00:33.980656+0000","last_undegraded":"2023-11-06T21:00:33.980656+0000","last_fullsized":"2023-11-06T21:00:33.980656+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.005250+0000","last_change":"2023-11-06T21:00:33.992182+0000","last_active":"2023-11-06T21:00:34.005250+0000","last_peered":"2023-11-06T21:00:34.005250+0000","last_clean":"2023-11-06T21:00:34.005250+0000","last_became_active":"2023-11-06T21:00:33.991869+0000","last_became_peered":"2023-11-06T21:00:33.991869+0000","last_unstale":"2023-11-06T21:00:34.005250+0000","last_undegraded":"2023-11-06T21:00:34.005250+0000","last_fullsized":"2023-11-06T21:00:34.005250+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981799+0000","last_change":"2023-11-06T21:00:33.503527+0000","last_active":"2023-11-06T21:00:33.981799+0000","last_peered":"2023-11-06T21:00:33.981799+0000","last_clean":"2023-11-06T21:00:33.981799+0000","last_became_active":"2023-11-06T21:00:33.503227+0000","last_became_peered":"2023-11-06T21:00:33.503227+0000","last_unstale":"2023-11-06T21:00:33.981799+0000","last_undegraded":"2023-11-06T21:00:33.981799+0000","last_fullsized":"2023-11-06T21:00:33.981799+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.995333+0000","last_change":"2023-11-06T21:00:33.985818+0000","last_active":"2023-11-06T21:00:33.995333+0000","last_peered":"2023-11-06T21:00:33.995333+0000","last_clean":"2023-11-06T21:00:33.995333+0000","last_became_active":"2023-11-06T21:00:33.982772+0000","last_became_peered":"2023-11-06T21:00:33.982772+0000","last_unstale":"2023-11-06T21:00:33.995333+0000","last_undegraded":"2023-11-06T21:00:33.995333+0000","last_fullsized":"2023-11-06T21:00:33.995333+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"113'4","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.998821+0000","last_change":"2023-11-06T21:00:33.502715+0000","last_active":"2023-11-06T21:00:33.998821+0000","last_peered":"2023-11-06T21:00:33.998821+0000","last_clean":"2023-11-06T21:00:33.998821+0000","last_became_active":"2023-11-06T21:00:33.501512+0000","last_became_peered":"2023-11-06T21:00:33.501512+0000","last_unstale":"2023-11-06T21:00:33.998821+0000","last_undegraded":"2023-11-06T21:00:33.998821+0000","last_fullsized":"2023-11-06T21:00:33.998821+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.994959+0000","last_change":"2023-11-06T21:00:33.987754+0000","last_active":"2023-11-06T21:00:33.994959+0000","last_peered":"2023-11-06T21:00:33.994959+0000","last_clean":"2023-11-06T21:00:33.994959+0000","last_became_active":"2023-11-06T21:00:33.987482+0000","last_became_peered":"2023-11-06T21:00:33.987482+0000","last_unstale":"2023-11-06T21:00:33.994959+0000","last_undegraded":"2023-11-06T21:00:33.994959+0000","last_fullsized":"2023-11-06T21:00:33.994959+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006579+0000","last_change":"2023-11-06T21:00:33.502509+0000","last_active":"2023-11-06T21:00:34.006579+0000","last_peered":"2023-11-06T21:00:34.006579+0000","last_clean":"2023-11-06T21:00:34.006579+0000","last_became_active":"2023-11-06T21:00:33.502005+0000","last_became_peered":"2023-11-06T21:00:33.502005+0000","last_unstale":"2023-11-06T21:00:34.006579+0000","last_undegraded":"2023-11-06T21:00:34.006579+0000","last_fullsized":"2023-11-06T21:00:34.006579+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002333+0000","last_change":"2023-11-06T21:00:33.994110+0000","last_active":"2023-11-06T21:00:34.002333+0000","last_peered":"2023-11-06T21:00:34.002333+0000","last_clean":"2023-11-06T21:00:34.002333+0000","last_became_active":"2023-11-06T21:00:33.993060+0000","last_became_peered":"2023-11-06T21:00:33.993060+0000","last_unstale":"2023-11-06T21:00:34.002333+0000","last_undegraded":"2023-11-06T21:00:34.002333+0000","last_fullsized":"2023-11-06T21:00:34.002333+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974970+0000","last_change":"2023-11-06T21:00:33.512460+0000","last_active":"2023-11-06T21:00:33.974970+0000","last_peered":"2023-11-06T21:00:33.974970+0000","last_clean":"2023-11-06T21:00:33.974970+0000","last_became_active":"2023-11-06T21:00:33.510887+0000","last_became_peered":"2023-11-06T21:00:33.510887+0000","last_unstale":"2023-11-06T21:00:33.974970+0000","last_undegraded":"2023-11-06T21:00:33.974970+0000","last_fullsized":"2023-11-06T21:00:33.974970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002433+0000","last_change":"2023-11-06T21:00:33.997711+0000","last_active":"2023-11-06T21:00:34.002433+0000","last_peered":"2023-11-06T21:00:34.002433+0000","last_clean":"2023-11-06T21:00:34.002433+0000","last_became_active":"2023-11-06T21:00:33.996708+0000","last_became_peered":"2023-11-06T21:00:33.996708+0000","last_unstale":"2023-11-06T21:00:34.002433+0000","last_undegraded":"2023-11-06T21:00:34.002433+0000","last_fullsized":"2023-11-06T21:00:34.002433+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.984085+0000","last_change":"2023-11-06T21:00:33.500512+0000","last_active":"2023-11-06T21:00:33.984085+0000","last_peered":"2023-11-06T21:00:33.984085+0000","last_clean":"2023-11-06T21:00:33.984085+0000","last_became_active":"2023-11-06T21:00:33.499912+0000","last_became_peered":"2023-11-06T21:00:33.499912+0000","last_unstale":"2023-11-06T21:00:33.984085+0000","last_undegraded":"2023-11-06T21:00:33.984085+0000","last_fullsized":"2023-11-06T21:00:33.984085+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001439+0000","last_change":"2023-11-06T21:00:33.995924+0000","last_active":"2023-11-06T21:00:34.001439+0000","last_peered":"2023-11-06T21:00:34.001439+0000","last_clean":"2023-11-06T21:00:34.001439+0000","last_became_active":"2023-11-06T21:00:33.995177+0000","last_became_peered":"2023-11-06T21:00:33.995177+0000","last_unstale":"2023-11-06T21:00:34.001439+0000","last_undegraded":"2023-11-06T21:00:34.001439+0000","last_fullsized":"2023-11-06T21:00:34.001439+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974478+0000","last_change":"2023-11-06T21:00:33.492925+0000","last_active":"2023-11-06T21:00:33.974478+0000","last_peered":"2023-11-06T21:00:33.974478+0000","last_clean":"2023-11-06T21:00:33.974478+0000","last_became_active":"2023-11-06T21:00:33.490246+0000","last_became_peered":"2023-11-06T21:00:33.490246+0000","last_unstale":"2023-11-06T21:00:33.974478+0000","last_undegraded":"2023-11-06T21:00:33.974478+0000","last_fullsized":"2023-11-06T21:00:33.974478+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008816+0000","last_change":"2023-11-06T21:00:34.004445+0000","last_active":"2023-11-06T21:00:34.008816+0000","last_peered":"2023-11-06T21:00:34.008816+0000","last_clean":"2023-11-06T21:00:34.008816+0000","last_became_active":"2023-11-06T21:00:34.003867+0000","last_became_peered":"2023-11-06T21:00:34.003867+0000","last_unstale":"2023-11-06T21:00:34.008816+0000","last_undegraded":"2023-11-06T21:00:34.008816+0000","last_fullsized":"2023-11-06T21:00:34.008816+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981176+0000","last_change":"2023-11-06T21:00:33.500328+0000","last_active":"2023-11-06T21:00:33.981176+0000","last_peered":"2023-11-06T21:00:33.981176+0000","last_clean":"2023-11-06T21:00:33.981176+0000","last_became_active":"2023-11-06T21:00:33.499803+0000","last_became_peered":"2023-11-06T21:00:33.499803+0000","last_unstale":"2023-11-06T21:00:33.981176+0000","last_undegraded":"2023-11-06T21:00:33.981176+0000","last_fullsized":"2023-11-06T21:00:33.981176+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.999843+0000","last_change":"2023-11-06T21:00:33.991829+0000","last_active":"2023-11-06T21:00:33.999843+0000","last_peered":"2023-11-06T21:00:33.999843+0000","last_clean":"2023-11-06T21:00:33.999843+0000","last_became_active":"2023-11-06T21:00:33.991531+0000","last_became_peered":"2023-11-06T21:00:33.991531+0000","last_unstale":"2023-11-06T21:00:33.999843+0000","last_undegraded":"2023-11-06T21:00:33.999843+0000","last_fullsized":"2023-11-06T21:00:33.999843+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971505+0000","last_change":"2023-11-06T21:00:33.508090+0000","last_active":"2023-11-06T21:00:33.971505+0000","last_peered":"2023-11-06T21:00:33.971505+0000","last_clean":"2023-11-06T21:00:33.971505+0000","last_became_active":"2023-11-06T21:00:33.507822+0000","last_became_peered":"2023-11-06T21:00:33.507822+0000","last_unstale":"2023-11-06T21:00:33.971505+0000","last_undegraded":"2023-11-06T21:00:33.971505+0000","last_fullsized":"2023-11-06T21:00:33.971505+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002540+0000","last_change":"2023-11-06T21:00:33.995778+0000","last_active":"2023-11-06T21:00:34.002540+0000","last_peered":"2023-11-06T21:00:34.002540+0000","last_clean":"2023-11-06T21:00:34.002540+0000","last_became_active":"2023-11-06T21:00:33.995457+0000","last_became_peered":"2023-11-06T21:00:33.995457+0000","last_unstale":"2023-11-06T21:00:34.002540+0000","last_undegraded":"2023-11-06T21:00:34.002540+0000","last_fullsized":"2023-11-06T21:00:34.002540+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971856+0000","last_change":"2023-11-06T21:00:33.496482+0000","last_active":"2023-11-06T21:00:33.971856+0000","last_peered":"2023-11-06T21:00:33.971856+0000","last_clean":"2023-11-06T21:00:33.971856+0000","last_became_active":"2023-11-06T21:00:33.496181+0000","last_became_peered":"2023-11-06T21:00:33.496181+0000","last_unstale":"2023-11-06T21:00:33.971856+0000","last_undegraded":"2023-11-06T21:00:33.971856+0000","last_fullsized":"2023-11-06T21:00:33.971856+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.005683+0000","last_change":"2023-11-06T21:00:33.999635+0000","last_active":"2023-11-06T21:00:34.005683+0000","last_peered":"2023-11-06T21:00:34.005683+0000","last_clean":"2023-11-06T21:00:34.005683+0000","last_became_active":"2023-11-06T21:00:33.999360+0000","last_became_peered":"2023-11-06T21:00:33.999360+0000","last_unstale":"2023-11-06T21:00:34.005683+0000","last_undegraded":"2023-11-06T21:00:34.005683+0000","last_fullsized":"2023-11-06T21:00:34.005683+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971755+0000","last_change":"2023-11-06T21:00:33.493974+0000","last_active":"2023-11-06T21:00:33.971755+0000","last_peered":"2023-11-06T21:00:33.971755+0000","last_clean":"2023-11-06T21:00:33.971755+0000","last_became_active":"2023-11-06T21:00:33.493635+0000","last_became_peered":"2023-11-06T21:00:33.493635+0000","last_unstale":"2023-11-06T21:00:33.971755+0000","last_undegraded":"2023-11-06T21:00:33.971755+0000","last_fullsized":"2023-11-06T21:00:33.971755+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006063+0000","last_change":"2023-11-06T21:00:34.000967+0000","last_active":"2023-11-06T21:00:34.006063+0000","last_peered":"2023-11-06T21:00:34.006063+0000","last_clean":"2023-11-06T21:00:34.006063+0000","last_became_active":"2023-11-06T21:00:34.000699+0000","last_became_peered":"2023-11-06T21:00:34.000699+0000","last_unstale":"2023-11-06T21:00:34.006063+0000","last_undegraded":"2023-11-06T21:00:34.006063+0000","last_fullsized":"2023-11-06T21:00:34.006063+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.005962+0000","last_change":"2023-11-06T21:00:33.507971+0000","last_active":"2023-11-06T21:00:34.005962+0000","last_peered":"2023-11-06T21:00:34.005962+0000","last_clean":"2023-11-06T21:00:34.005962+0000","last_became_active":"2023-11-06T21:00:33.507726+0000","last_became_peered":"2023-11-06T21:00:33.507726+0000","last_unstale":"2023-11-06T21:00:34.005962+0000","last_undegraded":"2023-11-06T21:00:34.005962+0000","last_fullsized":"2023-11-06T21:00:34.005962+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989166+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T21:00:33.989166+0000","last_peered":"2023-11-06T21:00:33.989166+0000","last_clean":"2023-11-06T21:00:33.989166+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T21:00:33.989166+0000","last_undegraded":"2023-11-06T21:00:33.989166+0000","last_fullsized":"2023-11-06T21:00:33.989166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.997521+0000","last_change":"2023-11-06T21:00:33.992978+0000","last_active":"2023-11-06T21:00:33.997521+0000","last_peered":"2023-11-06T21:00:33.997521+0000","last_clean":"2023-11-06T21:00:33.997521+0000","last_became_active":"2023-11-06T21:00:33.991485+0000","last_became_peered":"2023-11-06T21:00:33.991485+0000","last_unstale":"2023-11-06T21:00:33.997521+0000","last_undegraded":"2023-11-06T21:00:33.997521+0000","last_fullsized":"2023-11-06T21:00:33.997521+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.992549+0000","last_change":"2023-11-06T21:00:33.515504+0000","last_active":"2023-11-06T21:00:33.992549+0000","last_peered":"2023-11-06T21:00:33.992549+0000","last_clean":"2023-11-06T21:00:33.992549+0000","last_became_active":"2023-11-06T21:00:33.510340+0000","last_became_peered":"2023-11-06T21:00:33.510340+0000","last_unstale":"2023-11-06T21:00:33.992549+0000","last_undegraded":"2023-11-06T21:00:33.992549+0000","last_fullsized":"2023-11-06T21:00:33.992549+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973060+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T21:00:33.973060+0000","last_peered":"2023-11-06T21:00:33.973060+0000","last_clean":"2023-11-06T21:00:33.973060+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T21:00:33.973060+0000","last_undegraded":"2023-11-06T21:00:33.973060+0000","last_fullsized":"2023-11-06T21:00:33.973060+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004205+0000","last_change":"2023-11-06T21:00:34.000561+0000","last_active":"2023-11-06T21:00:34.004205+0000","last_peered":"2023-11-06T21:00:34.004205+0000","last_clean":"2023-11-06T21:00:34.004205+0000","last_became_active":"2023-11-06T21:00:34.000132+0000","last_became_peered":"2023-11-06T21:00:34.000132+0000","last_unstale":"2023-11-06T21:00:34.004205+0000","last_undegraded":"2023-11-06T21:00:34.004205+0000","last_fullsized":"2023-11-06T21:00:34.004205+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006842+0000","last_change":"2023-11-06T21:00:33.515388+0000","last_active":"2023-11-06T21:00:34.006842+0000","last_peered":"2023-11-06T21:00:34.006842+0000","last_clean":"2023-11-06T21:00:34.006842+0000","last_became_active":"2023-11-06T21:00:33.510267+0000","last_became_peered":"2023-11-06T21:00:33.510267+0000","last_unstale":"2023-11-06T21:00:34.006842+0000","last_undegraded":"2023-11-06T21:00:34.006842+0000","last_fullsized":"2023-11-06T21:00:34.006842+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":103,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974347+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T21:00:33.974347+0000","last_peered":"2023-11-06T21:00:33.974347+0000","last_clean":"2023-11-06T21:00:33.974347+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T21:00:33.974347+0000","last_undegraded":"2023-11-06T21:00:33.974347+0000","last_fullsized":"2023-11-06T21:00:33.974347+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001464+0000","last_change":"2023-11-06T21:00:33.996139+0000","last_active":"2023-11-06T21:00:34.001464+0000","last_peered":"2023-11-06T21:00:34.001464+0000","last_clean":"2023-11-06T21:00:34.001464+0000","last_became_active":"2023-11-06T21:00:33.995634+0000","last_became_peered":"2023-11-06T21:00:33.995634+0000","last_unstale":"2023-11-06T21:00:34.001464+0000","last_undegraded":"2023-11-06T21:00:34.001464+0000","last_fullsized":"2023-11-06T21:00:34.001464+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981812+0000","last_change":"2023-11-06T21:00:33.512541+0000","last_active":"2023-11-06T21:00:33.981812+0000","last_peered":"2023-11-06T21:00:33.981812+0000","last_clean":"2023-11-06T21:00:33.981812+0000","last_became_active":"2023-11-06T21:00:33.512236+0000","last_became_peered":"2023-11-06T21:00:33.512236+0000","last_unstale":"2023-11-06T21:00:33.981812+0000","last_undegraded":"2023-11-06T21:00:33.981812+0000","last_fullsized":"2023-11-06T21:00:33.981812+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.972209+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T21:00:33.972209+0000","last_peered":"2023-11-06T21:00:33.972209+0000","last_clean":"2023-11-06T21:00:33.972209+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T21:00:33.972209+0000","last_undegraded":"2023-11-06T21:00:33.972209+0000","last_fullsized":"2023-11-06T21:00:33.972209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971840+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T21:00:33.971840+0000","last_peered":"2023-11-06T21:00:33.971840+0000","last_clean":"2023-11-06T21:00:33.971840+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T21:00:33.971840+0000","last_undegraded":"2023-11-06T21:00:33.971840+0000","last_fullsized":"2023-11-06T21:00:33.971840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002574+0000","last_change":"2023-11-06T21:00:33.998649+0000","last_active":"2023-11-06T21:00:34.002574+0000","last_peered":"2023-11-06T21:00:34.002574+0000","last_clean":"2023-11-06T21:00:34.002574+0000","last_became_active":"2023-11-06T21:00:33.998308+0000","last_became_peered":"2023-11-06T21:00:33.998308+0000","last_unstale":"2023-11-06T21:00:34.002574+0000","last_undegraded":"2023-11-06T21:00:34.002574+0000","last_fullsized":"2023-11-06T21:00:34.002574+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004440+0000","last_change":"2023-11-06T21:00:33.505117+0000","last_active":"2023-11-06T21:00:34.004440+0000","last_peered":"2023-11-06T21:00:34.004440+0000","last_clean":"2023-11-06T21:00:34.004440+0000","last_became_active":"2023-11-06T21:00:33.504876+0000","last_became_peered":"2023-11-06T21:00:33.504876+0000","last_unstale":"2023-11-06T21:00:34.004440+0000","last_undegraded":"2023-11-06T21:00:34.004440+0000","last_fullsized":"2023-11-06T21:00:34.004440+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973667+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T21:00:33.973667+0000","last_peered":"2023-11-06T21:00:33.973667+0000","last_clean":"2023-11-06T21:00:33.973667+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T21:00:33.973667+0000","last_undegraded":"2023-11-06T21:00:33.973667+0000","last_fullsized":"2023-11-06T21:00:33.973667+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006593+0000","last_change":"2023-11-06T21:00:33.997726+0000","last_active":"2023-11-06T21:00:34.006593+0000","last_peered":"2023-11-06T21:00:34.006593+0000","last_clean":"2023-11-06T21:00:34.006593+0000","last_became_active":"2023-11-06T21:00:33.997083+0000","last_became_peered":"2023-11-06T21:00:33.997083+0000","last_unstale":"2023-11-06T21:00:34.006593+0000","last_undegraded":"2023-11-06T21:00:34.006593+0000","last_fullsized":"2023-11-06T21:00:34.006593+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.992522+0000","last_change":"2023-11-06T21:00:33.499548+0000","last_active":"2023-11-06T21:00:33.992522+0000","last_peered":"2023-11-06T21:00:33.992522+0000","last_clean":"2023-11-06T21:00:33.992522+0000","last_became_active":"2023-11-06T21:00:33.499278+0000","last_became_peered":"2023-11-06T21:00:33.499278+0000","last_unstale":"2023-11-06T21:00:33.992522+0000","last_undegraded":"2023-11-06T21:00:33.992522+0000","last_fullsized":"2023-11-06T21:00:33.992522+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974767+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T21:00:33.974767+0000","last_peered":"2023-11-06T21:00:33.974767+0000","last_clean":"2023-11-06T21:00:33.974767+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T21:00:33.974767+0000","last_undegraded":"2023-11-06T21:00:33.974767+0000","last_fullsized":"2023-11-06T21:00:33.974767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.999906+0000","last_change":"2023-11-06T21:00:33.991904+0000","last_active":"2023-11-06T21:00:33.999906+0000","last_peered":"2023-11-06T21:00:33.999906+0000","last_clean":"2023-11-06T21:00:33.999906+0000","last_became_active":"2023-11-06T21:00:33.991641+0000","last_became_peered":"2023-11-06T21:00:33.991641+0000","last_unstale":"2023-11-06T21:00:33.999906+0000","last_undegraded":"2023-11-06T21:00:33.999906+0000","last_fullsized":"2023-11-06T21:00:33.999906+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989108+0000","last_change":"2023-11-06T21:00:33.493526+0000","last_active":"2023-11-06T21:00:33.989108+0000","last_peered":"2023-11-06T21:00:33.989108+0000","last_clean":"2023-11-06T21:00:33.989108+0000","last_became_active":"2023-11-06T21:00:33.493180+0000","last_became_peered":"2023-11-06T21:00:33.493180+0000","last_unstale":"2023-11-06T21:00:33.989108+0000","last_undegraded":"2023-11-06T21:00:33.989108+0000","last_fullsized":"2023-11-06T21:00:33.989108+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":92,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.991880+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T21:00:33.991880+0000","last_peered":"2023-11-06T21:00:33.991880+0000","last_clean":"2023-11-06T21:00:33.991880+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T21:00:33.991880+0000","last_undegraded":"2023-11-06T21:00:33.991880+0000","last_fullsized":"2023-11-06T21:00:33.991880+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008907+0000","last_change":"2023-11-06T21:00:34.006403+0000","last_active":"2023-11-06T21:00:34.008907+0000","last_peered":"2023-11-06T21:00:34.008907+0000","last_clean":"2023-11-06T21:00:34.008907+0000","last_became_active":"2023-11-06T21:00:34.006121+0000","last_became_peered":"2023-11-06T21:00:34.006121+0000","last_unstale":"2023-11-06T21:00:34.008907+0000","last_undegraded":"2023-11-06T21:00:34.008907+0000","last_fullsized":"2023-11-06T21:00:34.008907+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.990567+0000","last_change":"2023-11-06T21:00:33.501196+0000","last_active":"2023-11-06T21:00:33.990567+0000","last_peered":"2023-11-06T21:00:33.990567+0000","last_clean":"2023-11-06T21:00:33.990567+0000","last_became_active":"2023-11-06T21:00:33.500874+0000","last_became_peered":"2023-11-06T21:00:33.500874+0000","last_unstale":"2023-11-06T21:00:33.990567+0000","last_undegraded":"2023-11-06T21:00:33.990567+0000","last_fullsized":"2023-11-06T21:00:33.990567+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973257+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T21:00:33.973257+0000","last_peered":"2023-11-06T21:00:33.973257+0000","last_clean":"2023-11-06T21:00:33.973257+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T21:00:33.973257+0000","last_undegraded":"2023-11-06T21:00:33.973257+0000","last_fullsized":"2023-11-06T21:00:33.973257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302568,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069272,"statfs":{"total":96636764160,"available":96326934528,"internally_reserved":0,"allocated":1409024,"data_stored":706267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.68500000000000005,"15min":0.69299999999999995},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.91300000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66900000000000004,"15min":0.69099999999999995},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67000000000000004,"15min":0.70499999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40500000000000003},"max":{"1min":0.84899999999999998,"5min":0.95199999999999996,"15min":5.1749999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.71499999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":1.133},"last":0.59899999999999998}]},{"osd":2,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.70499999999999996,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.83199999999999996,"5min":0.996,"15min":1.165},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":1.024},"last":0.42699999999999999}]},{"osd":3,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89400000000000002,"5min":1.0129999999999999,"15min":5.202},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73099999999999998,"15min":0.76100000000000001},"min":{"1min":0.55100000000000005,"5min":0.51000000000000001,"15min":0.49399999999999999},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":5.165},"last":0.505}]},{"osd":4,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.81399999999999995,"15min":1.083},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.86799999999999999,"15min":0.92800000000000005},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41099999999999998},"max":{"1min":0.85099999999999998,"5min":0.93300000000000005,"15min":1.395},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.129},"last":0.57099999999999995}]},{"osd":6,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.435},"max":{"1min":0.86599999999999999,"5min":0.94499999999999995,"15min":1.24},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.54500000000000004,"5min":0.47399999999999998,"15min":0.443},"max":{"1min":0.80200000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.61299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1404928,"data_stored":699369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.65700000000000003,"15min":0.68100000000000005},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.32800000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.0640000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.38800000000000001},"max":{"1min":0.86899999999999999,"5min":1.216,"15min":1.216},"last":0.46800000000000003}]},{"osd":1,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.67400000000000004,"15min":0.69499999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80900000000000005,"5min":0.96199999999999997,"15min":0.98399999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84099999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.73499999999999999,"15min":0.75600000000000001},"min":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85499999999999998,"5min":1.0049999999999999,"15min":1.044},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73899999999999999,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91300000000000003,"5min":1.05,"15min":1.05},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.47099999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82399999999999995,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71499999999999997,"15min":0.72599999999999998},"min":{"1min":0.53100000000000003,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":0.82699999999999996,"5min":1.002,"15min":1.002},"last":0.56299999999999994}]},{"osd":4,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.627,"15min":0.60899999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":1.012},"last":0.53100000000000003}]},{"osd":5,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88700000000000001,"5min":1.028,"15min":1.028},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69799999999999995,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.45400000000000001,"5min":0.40799999999999997,"15min":0.378},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302556,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069284,"statfs":{"total":96636764160,"available":96326946816,"internally_reserved":0,"allocated":1396736,"data_stored":699189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46100000000000002,"15min":0.45400000000000001},"min":{"1min":0.32500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":0.77900000000000003,"15min":0.85199999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.51100000000000001},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.441}]},{"osd":2,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.434},{"interface":"front","average":{"1min":0.751,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.42899999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":4.431,"5min":4.431,"15min":4.431},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.60899999999999999,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.5380000000000003,"5min":4.5380000000000003,"15min":4.5380000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77200000000000002,"15min":0.79100000000000004},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":0.80700000000000005}]},{"osd":5,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.79200000000000004,"15min":0.79000000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":4.399,"5min":4.399,"15min":4.399},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.54800000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.69599999999999995}]},{"osd":6,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.81000000000000005,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.501},"max":{"1min":3.024,"5min":3.024,"15min":3.024},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.0609999999999999,"5min":3.0609999999999999,"15min":3.0609999999999999},"last":0.81699999999999995}]},{"osd":7,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.58199999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.85799999999999998,"15min":0.84299999999999997},"min":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":5.8230000000000004,"5min":5.8230000000000004,"15min":5.8230000000000004},"last":0.86399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575045,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302552,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069288,"statfs":{"total":96636764160,"available":96326950912,"internally_reserved":0,"allocated":1392640,"data_stored":699493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44600000000000001,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":0.73899999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.505,"15min":0.503},"min":{"1min":0.33700000000000002,"5min":0.33200000000000002,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":0.72899999999999998,"15min":0.80700000000000005},"last":0.502}]},{"osd":2,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45100000000000001,"15min":0.46800000000000003},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.82099999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.54800000000000004},"min":{"1min":0.38400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68300000000000005,"5min":0.77500000000000002,"15min":0.91000000000000003},"last":0.57199999999999995}]},{"osd":3,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54600000000000004,"15min":0.56999999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.747,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55100000000000005}]},{"osd":4,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.77200000000000002,"15min":0.80400000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":0.95099999999999996,"5min":1.1299999999999999,"15min":1.1779999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77700000000000002,"15min":0.80200000000000005},"min":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98899999999999999,"5min":1.298,"15min":1.298},"last":0.88}]},{"osd":5,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77300000000000002,"15min":0.80600000000000005},"min":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.48999999999999999},"max":{"1min":0.96699999999999997,"5min":1.069,"15min":1.2010000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.79800000000000004,"15min":0.81999999999999995},"min":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.51400000000000001},"max":{"1min":0.94799999999999995,"5min":1.2549999999999999,"15min":1.2829999999999999},"last":0.94799999999999995}]},{"osd":6,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76100000000000001,"15min":0.79000000000000004},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.54600000000000004},"max":{"1min":0.91600000000000004,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77700000000000002,"15min":0.81000000000000005},"min":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95999999999999996,"5min":1.056,"15min":1.304},"last":0.95999999999999996}]},{"osd":7,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78700000000000003,"15min":0.81699999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.97699999999999998,"5min":1.171,"15min":1.2490000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80900000000000005,"15min":0.83199999999999996},"min":{"1min":0.63400000000000001,"5min":0.54600000000000004,"15min":0.53200000000000003},"max":{"1min":0.997,"5min":1.1899999999999999,"15min":1.236},"last":0.997}]}]},{"osd":2,"up_from":13,"seq":55834575045,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302556,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069284,"statfs":{"total":96636764160,"available":96326946816,"internally_reserved":0,"allocated":1396736,"data_stored":699177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.44800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65700000000000003,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.433},{"interface":"front","average":{"1min":0.434,"5min":0.47899999999999998,"15min":0.48999999999999999},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58599999999999997,"5min":0.72699999999999998,"15min":0.81699999999999995},"last":0.69499999999999995}]},{"osd":1,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.755,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65300000000000002,"5min":2.266,"15min":2.266},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":1.732,"15min":1.732},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76800000000000002,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.995,"5min":1.0980000000000001,"15min":1.1890000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.74099999999999999,"15min":0.77600000000000002},"min":{"1min":0.46700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0269999999999999,"5min":1.087,"15min":1.357},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.755,"15min":0.77600000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.45900000000000002},"max":{"1min":1.0089999999999999,"5min":1.657,"15min":1.657},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.1919999999999999},"last":0.77100000000000002}]},{"osd":6,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78000000000000003,"15min":0.80400000000000005},"min":{"1min":0.495,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.113,"5min":1.1499999999999999,"15min":1.2969999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.099,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.752}]},{"osd":7,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79900000000000004,"15min":0.82699999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.46600000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.448},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82199999999999995,"15min":0.82699999999999996},"min":{"1min":0.54100000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.107,"5min":2.202,"15min":2.202},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575046,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1404928,"data_stored":699357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.439,"15min":0.45500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.26800000000000002},"max":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.82599999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.32600000000000001},"max":{"1min":0.81999999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.81999999999999995}]},{"osd":1,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.57399999999999995,"5min":0.64600000000000002,"15min":0.753},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.56100000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.86699999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.55300000000000005},"min":{"1min":0.35599999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.80100000000000005,"15min":0.91900000000000004},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.80100000000000005,"15min":0.81499999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.51200000000000001},"max":{"1min":1.008,"5min":1.234,"15min":1.2470000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.47999999999999998},"max":{"1min":1.0269999999999999,"5min":1.276,"15min":1.286},"last":0.89400000000000002}]},{"osd":5,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.75600000000000001,"15min":0.78600000000000003},"min":{"1min":0.52200000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0209999999999999,"5min":1.1519999999999999,"15min":1.1779999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.79300000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96699999999999997,"5min":1.137,"15min":1.1439999999999999},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.755,"15min":0.78000000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90000000000000002,"5min":1.323,"15min":1.323},"last":0.754},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78100000000000003,"15min":0.81399999999999995},"min":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98099999999999998,"5min":1.054,"15min":1.2130000000000001},"last":0.78900000000000003}]},{"osd":7,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77600000000000002,"15min":0.81299999999999994},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":1.0920000000000001,"15min":1.302},"last":0.877},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.79900000000000004,"15min":0.82299999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.55200000000000005},"max":{"1min":0.98299999999999998,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302548,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069292,"statfs":{"total":96636764160,"available":96326955008,"internally_reserved":0,"allocated":1388544,"data_stored":698601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65300000000000002,"15min":0.69199999999999995},"min":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83799999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.64600000000000002,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83499999999999996,"5min":1,"15min":1},"last":0.58199999999999996}]},{"osd":1,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.67100000000000004,"15min":0.70299999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":0.81299999999999994,"5min":1.645,"15min":1.645},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.69499999999999995,"15min":0.71599999999999997},"min":{"1min":0.46100000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.753,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69299999999999995,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86699999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.60199999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.86199999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.79600000000000004}]},{"osd":3,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66600000000000004,"15min":0.69399999999999995},"min":{"1min":0.48099999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77900000000000003,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67500000000000004,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.436},"max":{"1min":0.85499999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83999999999999997,"5min":0.88200000000000001,"15min":1.0389999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.60799999999999998},"min":{"1min":0.41699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.91000000000000003,"5min":1.22,"15min":1.22},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69199999999999995,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.44900000000000001,"15min":0.44},"max":{"1min":0.84299999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.92700000000000005,"5min":1.034,"15min":1.034},"last":0.747},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.67700000000000005},"min":{"1min":0.47299999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95199999999999996,"5min":1.177,"15min":1.177},"last":0.77900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575046,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302552,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069288,"statfs":{"total":96636764160,"available":96326950912,"internally_reserved":0,"allocated":1392640,"data_stored":698671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.66400000000000003},"min":{"1min":0.40100000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.82499999999999996,"5min":0.98599999999999999,"15min":1.0109999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.92000000000000004,"5min":0.96199999999999997,"15min":0.96799999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70599999999999996,"15min":0.70699999999999996},"min":{"1min":0.54300000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":1.0109999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.747},"min":{"1min":0.48199999999999998,"5min":0.432,"15min":0.40300000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.044},"last":0.45000000000000001}]},{"osd":2,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.503,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.024},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.59799999999999998,"5min":0.46000000000000002,"15min":0.45600000000000002},"max":{"1min":0.96399999999999997,"5min":0.96999999999999997,"15min":1.0329999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.92000000000000004,"5min":0.97599999999999998,"15min":1.02},"last":0.60299999999999998}]},{"osd":4,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.45300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.86399999999999999,"15min":0.93700000000000006},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.68600000000000005},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89800000000000002,"5min":0.97499999999999998,"15min":0.98599999999999999},"last":0.745}]},{"osd":6,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.96399999999999997,"15min":0.97599999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":0.88,"5min":0.96399999999999997,"15min":1.105},"last":0.78900000000000003}]},{"osd":7,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.998,"15min":0.998},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T21:00:38.317 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T21:00:38.456 INFO:tasks.ceph.mds.d.smithi069.stderr:2023-11-06T21:00:38.456+0000 7fa5801f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T21:00:38.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.545+0000 7f14b57ff700 1 Processor -- start 2023-11-06T21:00:38.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.545+0000 7f14b57ff700 1 -- start start 2023-11-06T21:00:38.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14b57ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f14b01521a0 0x7f14b00c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14b57ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14b0155320 0x7f14b00c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14b57ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b0154e90 con 0x7f14b0151dc0 2023-11-06T21:00:38.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14b57ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b00639d0 con 0x7f14b0155320 2023-11-06T21:00:38.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14b57ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b0063cb0 con 0x7f14b01521a0 2023-11-06T21:00:38.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14af7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14b0155320 0x7f14b00c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14af7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14b0155320 0x7f14b00c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41042/0 (socket says 172.21.15.33:41042) 2023-11-06T21:00:38.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14af7fe700 1 -- 172.21.15.33:0/909586471 learned_addr learned my addr 172.21.15.33:0/909586471 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:38.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14ae7fc700 1 --2- 172.21.15.33:0/909586471 >> v2:172.21.15.69:3300/0 conn(0x7f14b01521a0 0x7f14b00c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14af7fe700 1 -- 172.21.15.33:0/909586471 >> v2:172.21.15.69:3300/0 conn(0x7f14b01521a0 msgr2=0x7f14b00c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14af7fe700 1 --2- 172.21.15.33:0/909586471 >> v2:172.21.15.69:3300/0 conn(0x7f14b01521a0 0x7f14b00c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.547+0000 7f14af7fe700 1 -- 172.21.15.33:0/909586471 >> v1:172.21.15.69:6789/0 conn(0x7f14b0151dc0 legacy=0x7f14b0063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.548+0000 7f14af7fe700 1 -- 172.21.15.33:0/909586471 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b0063f90 con 0x7f14b0155320 2023-11-06T21:00:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.548+0000 7f14ae7fc700 1 --2- 172.21.15.33:0/909586471 >> v2:172.21.15.69:3300/0 conn(0x7f14b01521a0 0x7f14b00c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.548+0000 7f14af7fe700 1 --2- 172.21.15.33:0/909586471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0155320 0x7f14b00c7850 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f14b00c0680 tx=0x7f14a8009840).ready entity=mon.0 client_cookie=3986b3438eac1a51 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.548+0000 7f14adffb700 1 -- 172.21.15.33:0/909586471 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800e070 con 0x7f14b0155320 2023-11-06T21:00:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.548+0000 7f14adffb700 1 -- 172.21.15.33:0/909586471 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a800ba50 con 0x7f14b0155320 2023-11-06T21:00:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.548+0000 7f14adffb700 1 -- 172.21.15.33:0/909586471 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800bdd0 con 0x7f14b0155320 2023-11-06T21:00:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.548+0000 7f14b57ff700 1 -- 172.21.15.33:0/909586471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0155320 msgr2=0x7f14b00c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.548+0000 7f14b57ff700 1 --2- 172.21.15.33:0/909586471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0155320 0x7f14b00c7850 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f14b00c0680 tx=0x7f14a8009840).stop 2023-11-06T21:00:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.549+0000 7f14b57ff700 1 -- 172.21.15.33:0/909586471 shutdown_connections 2023-11-06T21:00:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.549+0000 7f14b57ff700 1 --2- 172.21.15.33:0/909586471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0155320 0x7f14b00c7850 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.549+0000 7f14b57ff700 1 --2- 172.21.15.33:0/909586471 >> v2:172.21.15.69:3300/0 conn(0x7f14b01521a0 0x7f14b00c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.549+0000 7f14b57ff700 1 -- 172.21.15.33:0/909586471 >> 172.21.15.33:0/909586471 conn(0x7f14b005e1b0 msgr2=0x7f14b005e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.549+0000 7f14b57ff700 1 -- 172.21.15.33:0/909586471 shutdown_connections 2023-11-06T21:00:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.549+0000 7f14b57ff700 1 -- 172.21.15.33:0/909586471 wait complete. 2023-11-06T21:00:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.550+0000 7f14b57ff700 1 Processor -- start 2023-11-06T21:00:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.550+0000 7f14b57ff700 1 -- start start 2023-11-06T21:00:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.551+0000 7f14b57ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0151dc0 0x7f14b00c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.551+0000 7f14b57ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14b01521a0 0x7f14b00addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.551+0000 7f14b57ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14b0155320 0x7f14b00ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.551+0000 7f14b57ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b014f1f0 con 0x7f14b0151dc0 2023-11-06T21:00:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.551+0000 7f14b57ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00ae790 con 0x7f14b0155320 2023-11-06T21:00:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.551+0000 7f14b57ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00ae9c0 con 0x7f14b01521a0 2023-11-06T21:00:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0151dc0 0x7f14b00c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0151dc0 0x7f14b00c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41048/0 (socket says 172.21.15.33:41048) 2023-11-06T21:00:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14aeffd700 1 -- 172.21.15.33:0/4188809268 learned_addr learned my addr 172.21.15.33:0/4188809268 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14ae7fc700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14b01521a0 0x7f14b00addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14af7fe700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14b0155320 0x7f14b00ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14aeffd700 1 -- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14b01521a0 msgr2=0x7f14b00addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14aeffd700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14b01521a0 0x7f14b00addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14aeffd700 1 -- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14b0155320 msgr2=0x7f14b00ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14aeffd700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14b0155320 0x7f14b00ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14aeffd700 1 -- 172.21.15.33:0/4188809268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a8009ae0 con 0x7f14b0151dc0 2023-11-06T21:00:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14ae7fc700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14b01521a0 0x7f14b00addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14aeffd700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0151dc0 0x7f14b00c42d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f14a000c4c0 tx=0x7f14a0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f1496ffd700 1 -- 172.21.15.33:0/4188809268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0007020 con 0x7f14b0151dc0 2023-11-06T21:00:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f1496ffd700 1 -- 172.21.15.33:0/4188809268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a0001e90 con 0x7f14b0151dc0 2023-11-06T21:00:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f1496ffd700 1 -- 172.21.15.33:0/4188809268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a00073b0 con 0x7f14b0151dc0 2023-11-06T21:00:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.552+0000 7f14b57ff700 1 -- 172.21.15.33:0/4188809268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b00ab440 con 0x7f14b0151dc0 2023-11-06T21:00:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.553+0000 7f14b57ff700 1 -- 172.21.15.33:0/4188809268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b00aba00 con 0x7f14b0151dc0 2023-11-06T21:00:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.554+0000 7f14b57ff700 1 -- 172.21.15.33:0/4188809268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b00c0a30 con 0x7f14b0151dc0 2023-11-06T21:00:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.554+0000 7f1496ffd700 1 -- 172.21.15.33:0/4188809268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f14a00068b0 con 0x7f14b0151dc0 2023-11-06T21:00:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.556+0000 7f1496ffd700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f149c05f270 0x7f149c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.556+0000 7f1496ffd700 1 -- 172.21.15.33:0/4188809268 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f14a0079dc0 con 0x7f14b0151dc0 2023-11-06T21:00:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.556+0000 7f14ae7fc700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f149c05f270 0x7f149c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:38.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.557+0000 7f14ae7fc700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f149c05f270 0x7f149c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f14b0167b70 tx=0x7f14a4008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:38.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.560+0000 7f1496ffd700 1 -- 172.21.15.33:0/4188809268 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f14a004af10 con 0x7f14b0151dc0 2023-11-06T21:00:38.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.739+0000 7f14b57ff700 1 -- 172.21.15.33:0/4188809268 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14b006ab40 con 0x7f149c05f270 2023-11-06T21:00:38.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.750+0000 7f1496ffd700 1 -- 172.21.15.33:0/4188809268 <== mgr.6050 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361711 (secure 0 0 0) 0x7f14b006ab40 con 0x7f149c05f270 2023-11-06T21:00:38.755 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:38.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 -- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f149c05f270 msgr2=0x7f149c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f149c05f270 0x7f149c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f14b0167b70 tx=0x7f14a4008040).stop 2023-11-06T21:00:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 -- 172.21.15.33:0/4188809268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0151dc0 msgr2=0x7f14b00c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0151dc0 0x7f14b00c42d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f14a000c4c0 tx=0x7f14a0007e00).stop 2023-11-06T21:00:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 -- 172.21.15.33:0/4188809268 shutdown_connections 2023-11-06T21:00:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f149c05f270 0x7f149c061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14b0155320 0x7f14b00ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b0151dc0 0x7f14b00c42d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 --2- 172.21.15.33:0/4188809268 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f14b01521a0 0x7f14b00addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 -- 172.21.15.33:0/4188809268 >> 172.21.15.33:0/4188809268 conn(0x7f14b005e1b0 msgr2=0x7f14b0152640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 -- 172.21.15.33:0/4188809268 shutdown_connections 2023-11-06T21:00:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:38.755+0000 7f14b57ff700 1 -- 172.21.15.33:0/4188809268 wait complete. 2023-11-06T21:00:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T21:00:38.773 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T21:00:37.429266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2420460,"kb_used_data":10924,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752554260,"statfs":{"total":773094113280,"available":770615562240,"internally_reserved":0,"allocated":11186176,"data_stored":5600124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3334,"num_objects":-6,"num_object_clones":0,"num_object_copies":-12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-21,"num_write_kb":-16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.993244"},"pg_stats":[{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001167+0000","last_change":"2023-11-06T21:00:33.988209+0000","last_active":"2023-11-06T21:00:34.001167+0000","last_peered":"2023-11-06T21:00:34.001167+0000","last_clean":"2023-11-06T21:00:34.001167+0000","last_became_active":"2023-11-06T21:00:33.987851+0000","last_became_peered":"2023-11-06T21:00:33.987851+0000","last_unstale":"2023-11-06T21:00:34.001167+0000","last_undegraded":"2023-11-06T21:00:34.001167+0000","last_fullsized":"2023-11-06T21:00:34.001167+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3f","version":"113'3","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.996669+0000","last_change":"2023-11-06T21:00:33.511915+0000","last_active":"2023-11-06T21:00:33.996669+0000","last_peered":"2023-11-06T21:00:33.996669+0000","last_clean":"2023-11-06T21:00:33.996669+0000","last_became_active":"2023-11-06T21:00:33.511617+0000","last_became_peered":"2023-11-06T21:00:33.511617+0000","last_unstale":"2023-11-06T21:00:33.996669+0000","last_undegraded":"2023-11-06T21:00:33.996669+0000","last_fullsized":"2023-11-06T21:00:33.996669+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.992607+0000","last_change":"2023-11-06T21:00:33.984787+0000","last_active":"2023-11-06T21:00:33.992607+0000","last_peered":"2023-11-06T21:00:33.992607+0000","last_clean":"2023-11-06T21:00:33.992607+0000","last_became_active":"2023-11-06T21:00:33.984489+0000","last_became_peered":"2023-11-06T21:00:33.984489+0000","last_unstale":"2023-11-06T21:00:33.992607+0000","last_undegraded":"2023-11-06T21:00:33.992607+0000","last_fullsized":"2023-11-06T21:00:33.992607+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.991788+0000","last_change":"2023-11-06T21:00:33.493526+0000","last_active":"2023-11-06T21:00:33.991788+0000","last_peered":"2023-11-06T21:00:33.991788+0000","last_clean":"2023-11-06T21:00:33.991788+0000","last_became_active":"2023-11-06T21:00:33.493206+0000","last_became_peered":"2023-11-06T21:00:33.493206+0000","last_unstale":"2023-11-06T21:00:33.991788+0000","last_undegraded":"2023-11-06T21:00:33.991788+0000","last_fullsized":"2023-11-06T21:00:33.991788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989675+0000","last_change":"2023-11-06T21:00:33.984929+0000","last_active":"2023-11-06T21:00:33.989675+0000","last_peered":"2023-11-06T21:00:33.989675+0000","last_clean":"2023-11-06T21:00:33.989675+0000","last_became_active":"2023-11-06T21:00:33.984078+0000","last_became_peered":"2023-11-06T21:00:33.984078+0000","last_unstale":"2023-11-06T21:00:33.989675+0000","last_undegraded":"2023-11-06T21:00:33.989675+0000","last_fullsized":"2023-11-06T21:00:33.989675+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.983970+0000","last_change":"2023-11-06T21:00:33.506948+0000","last_active":"2023-11-06T21:00:33.983970+0000","last_peered":"2023-11-06T21:00:33.983970+0000","last_clean":"2023-11-06T21:00:33.983970+0000","last_became_active":"2023-11-06T21:00:33.506698+0000","last_became_peered":"2023-11-06T21:00:33.506698+0000","last_unstale":"2023-11-06T21:00:33.983970+0000","last_undegraded":"2023-11-06T21:00:33.983970+0000","last_fullsized":"2023-11-06T21:00:33.983970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.007818+0000","last_change":"2023-11-06T21:00:34.000384+0000","last_active":"2023-11-06T21:00:34.007818+0000","last_peered":"2023-11-06T21:00:34.007818+0000","last_clean":"2023-11-06T21:00:34.007818+0000","last_became_active":"2023-11-06T21:00:34.000039+0000","last_became_peered":"2023-11-06T21:00:34.000039+0000","last_unstale":"2023-11-06T21:00:34.007818+0000","last_undegraded":"2023-11-06T21:00:34.007818+0000","last_fullsized":"2023-11-06T21:00:34.007818+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3c","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002814+0000","last_change":"2023-11-06T21:00:33.500541+0000","last_active":"2023-11-06T21:00:34.002814+0000","last_peered":"2023-11-06T21:00:34.002814+0000","last_clean":"2023-11-06T21:00:34.002814+0000","last_became_active":"2023-11-06T21:00:33.500275+0000","last_became_peered":"2023-11-06T21:00:33.500275+0000","last_unstale":"2023-11-06T21:00:34.002814+0000","last_undegraded":"2023-11-06T21:00:34.002814+0000","last_fullsized":"2023-11-06T21:00:34.002814+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004280+0000","last_change":"2023-11-06T21:00:33.997018+0000","last_active":"2023-11-06T21:00:34.004280+0000","last_peered":"2023-11-06T21:00:34.004280+0000","last_clean":"2023-11-06T21:00:34.004280+0000","last_became_active":"2023-11-06T21:00:33.996115+0000","last_became_peered":"2023-11-06T21:00:33.996115+0000","last_unstale":"2023-11-06T21:00:34.004280+0000","last_undegraded":"2023-11-06T21:00:34.004280+0000","last_fullsized":"2023-11-06T21:00:34.004280+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3b","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003969+0000","last_change":"2023-11-06T21:00:33.491787+0000","last_active":"2023-11-06T21:00:34.003969+0000","last_peered":"2023-11-06T21:00:34.003969+0000","last_clean":"2023-11-06T21:00:34.003969+0000","last_became_active":"2023-11-06T21:00:33.491502+0000","last_became_peered":"2023-11-06T21:00:33.491502+0000","last_unstale":"2023-11-06T21:00:34.003969+0000","last_undegraded":"2023-11-06T21:00:34.003969+0000","last_fullsized":"2023-11-06T21:00:34.003969+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.983634+0000","last_change":"2023-11-06T21:00:33.979851+0000","last_active":"2023-11-06T21:00:33.983634+0000","last_peered":"2023-11-06T21:00:33.983634+0000","last_clean":"2023-11-06T21:00:33.983634+0000","last_became_active":"2023-11-06T21:00:33.979522+0000","last_became_peered":"2023-11-06T21:00:33.979522+0000","last_unstale":"2023-11-06T21:00:33.983634+0000","last_undegraded":"2023-11-06T21:00:33.983634+0000","last_fullsized":"2023-11-06T21:00:33.983634+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.995907+0000","last_change":"2023-11-06T21:00:33.497866+0000","last_active":"2023-11-06T21:00:33.995907+0000","last_peered":"2023-11-06T21:00:33.995907+0000","last_clean":"2023-11-06T21:00:33.995907+0000","last_became_active":"2023-11-06T21:00:33.497436+0000","last_became_peered":"2023-11-06T21:00:33.497436+0000","last_unstale":"2023-11-06T21:00:33.995907+0000","last_undegraded":"2023-11-06T21:00:33.995907+0000","last_fullsized":"2023-11-06T21:00:33.995907+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006526+0000","last_change":"2023-11-06T21:00:33.999689+0000","last_active":"2023-11-06T21:00:34.006526+0000","last_peered":"2023-11-06T21:00:34.006526+0000","last_clean":"2023-11-06T21:00:34.006526+0000","last_became_active":"2023-11-06T21:00:33.999408+0000","last_became_peered":"2023-11-06T21:00:33.999408+0000","last_unstale":"2023-11-06T21:00:34.006526+0000","last_undegraded":"2023-11-06T21:00:34.006526+0000","last_fullsized":"2023-11-06T21:00:34.006526+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.992594+0000","last_change":"2023-11-06T21:00:33.502350+0000","last_active":"2023-11-06T21:00:33.992594+0000","last_peered":"2023-11-06T21:00:33.992594+0000","last_clean":"2023-11-06T21:00:33.992594+0000","last_became_active":"2023-11-06T21:00:33.500784+0000","last_became_peered":"2023-11-06T21:00:33.500784+0000","last_unstale":"2023-11-06T21:00:33.992594+0000","last_undegraded":"2023-11-06T21:00:33.992594+0000","last_fullsized":"2023-11-06T21:00:33.992594+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001734+0000","last_change":"2023-11-06T21:00:33.995983+0000","last_active":"2023-11-06T21:00:34.001734+0000","last_peered":"2023-11-06T21:00:34.001734+0000","last_clean":"2023-11-06T21:00:34.001734+0000","last_became_active":"2023-11-06T21:00:33.995765+0000","last_became_peered":"2023-11-06T21:00:33.995765+0000","last_unstale":"2023-11-06T21:00:34.001734+0000","last_undegraded":"2023-11-06T21:00:34.001734+0000","last_fullsized":"2023-11-06T21:00:34.001734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.988395+0000","last_change":"2023-11-06T21:00:33.504202+0000","last_active":"2023-11-06T21:00:33.988395+0000","last_peered":"2023-11-06T21:00:33.988395+0000","last_clean":"2023-11-06T21:00:33.988395+0000","last_became_active":"2023-11-06T21:00:33.503854+0000","last_became_peered":"2023-11-06T21:00:33.503854+0000","last_unstale":"2023-11-06T21:00:33.988395+0000","last_undegraded":"2023-11-06T21:00:33.988395+0000","last_fullsized":"2023-11-06T21:00:33.988395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003532+0000","last_change":"2023-11-06T21:00:34.000628+0000","last_active":"2023-11-06T21:00:34.003532+0000","last_peered":"2023-11-06T21:00:34.003532+0000","last_clean":"2023-11-06T21:00:34.003532+0000","last_became_active":"2023-11-06T21:00:33.997694+0000","last_became_peered":"2023-11-06T21:00:33.997694+0000","last_unstale":"2023-11-06T21:00:34.003532+0000","last_undegraded":"2023-11-06T21:00:34.003532+0000","last_fullsized":"2023-11-06T21:00:34.003532+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.972482+0000","last_change":"2023-11-06T21:00:33.507854+0000","last_active":"2023-11-06T21:00:33.972482+0000","last_peered":"2023-11-06T21:00:33.972482+0000","last_clean":"2023-11-06T21:00:33.972482+0000","last_became_active":"2023-11-06T21:00:33.505595+0000","last_became_peered":"2023-11-06T21:00:33.505595+0000","last_unstale":"2023-11-06T21:00:33.972482+0000","last_undegraded":"2023-11-06T21:00:33.972482+0000","last_fullsized":"2023-11-06T21:00:33.972482+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003838+0000","last_change":"2023-11-06T21:00:33.990282+0000","last_active":"2023-11-06T21:00:34.003838+0000","last_peered":"2023-11-06T21:00:34.003838+0000","last_clean":"2023-11-06T21:00:34.003838+0000","last_became_active":"2023-11-06T21:00:33.990006+0000","last_became_peered":"2023-11-06T21:00:33.990006+0000","last_unstale":"2023-11-06T21:00:34.003838+0000","last_undegraded":"2023-11-06T21:00:34.003838+0000","last_fullsized":"2023-11-06T21:00:34.003838+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.985731+0000","last_change":"2023-11-06T21:00:33.501033+0000","last_active":"2023-11-06T21:00:33.985731+0000","last_peered":"2023-11-06T21:00:33.985731+0000","last_clean":"2023-11-06T21:00:33.985731+0000","last_became_active":"2023-11-06T21:00:33.496809+0000","last_became_peered":"2023-11-06T21:00:33.496809+0000","last_unstale":"2023-11-06T21:00:33.985731+0000","last_undegraded":"2023-11-06T21:00:33.985731+0000","last_fullsized":"2023-11-06T21:00:33.985731+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003537+0000","last_change":"2023-11-06T21:00:34.001011+0000","last_active":"2023-11-06T21:00:34.003537+0000","last_peered":"2023-11-06T21:00:34.003537+0000","last_clean":"2023-11-06T21:00:34.003537+0000","last_became_active":"2023-11-06T21:00:34.000705+0000","last_became_peered":"2023-11-06T21:00:34.000705+0000","last_unstale":"2023-11-06T21:00:34.003537+0000","last_undegraded":"2023-11-06T21:00:34.003537+0000","last_fullsized":"2023-11-06T21:00:34.003537+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981928+0000","last_change":"2023-11-06T21:00:33.500469+0000","last_active":"2023-11-06T21:00:33.981928+0000","last_peered":"2023-11-06T21:00:33.981928+0000","last_clean":"2023-11-06T21:00:33.981928+0000","last_became_active":"2023-11-06T21:00:33.500127+0000","last_became_peered":"2023-11-06T21:00:33.500127+0000","last_unstale":"2023-11-06T21:00:33.981928+0000","last_undegraded":"2023-11-06T21:00:33.981928+0000","last_fullsized":"2023-11-06T21:00:33.981928+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003903+0000","last_change":"2023-11-06T21:00:33.993716+0000","last_active":"2023-11-06T21:00:34.003903+0000","last_peered":"2023-11-06T21:00:34.003903+0000","last_clean":"2023-11-06T21:00:34.003903+0000","last_became_active":"2023-11-06T21:00:33.989757+0000","last_became_peered":"2023-11-06T21:00:33.989757+0000","last_unstale":"2023-11-06T21:00:34.003903+0000","last_undegraded":"2023-11-06T21:00:34.003903+0000","last_fullsized":"2023-11-06T21:00:34.003903+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.34","version":"113'4","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008142+0000","last_change":"2023-11-06T21:00:33.500855+0000","last_active":"2023-11-06T21:00:34.008142+0000","last_peered":"2023-11-06T21:00:34.008142+0000","last_clean":"2023-11-06T21:00:34.008142+0000","last_became_active":"2023-11-06T21:00:33.500233+0000","last_became_peered":"2023-11-06T21:00:33.500233+0000","last_unstale":"2023-11-06T21:00:34.008142+0000","last_undegraded":"2023-11-06T21:00:34.008142+0000","last_fullsized":"2023-11-06T21:00:34.008142+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008835+0000","last_change":"2023-11-06T21:00:34.004623+0000","last_active":"2023-11-06T21:00:34.008835+0000","last_peered":"2023-11-06T21:00:34.008835+0000","last_clean":"2023-11-06T21:00:34.008835+0000","last_became_active":"2023-11-06T21:00:34.004086+0000","last_became_peered":"2023-11-06T21:00:34.004086+0000","last_unstale":"2023-11-06T21:00:34.008835+0000","last_undegraded":"2023-11-06T21:00:34.008835+0000","last_fullsized":"2023-11-06T21:00:34.008835+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.33","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003847+0000","last_change":"2023-11-06T21:00:33.504508+0000","last_active":"2023-11-06T21:00:34.003847+0000","last_peered":"2023-11-06T21:00:34.003847+0000","last_clean":"2023-11-06T21:00:34.003847+0000","last_became_active":"2023-11-06T21:00:33.504187+0000","last_became_peered":"2023-11-06T21:00:33.504187+0000","last_unstale":"2023-11-06T21:00:34.003847+0000","last_undegraded":"2023-11-06T21:00:34.003847+0000","last_fullsized":"2023-11-06T21:00:34.003847+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.000543+0000","last_change":"2023-11-06T21:00:33.997037+0000","last_active":"2023-11-06T21:00:34.000543+0000","last_peered":"2023-11-06T21:00:34.000543+0000","last_clean":"2023-11-06T21:00:34.000543+0000","last_became_active":"2023-11-06T21:00:33.996731+0000","last_became_peered":"2023-11-06T21:00:33.996731+0000","last_unstale":"2023-11-06T21:00:34.000543+0000","last_undegraded":"2023-11-06T21:00:34.000543+0000","last_fullsized":"2023-11-06T21:00:34.000543+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973314+0000","last_change":"2023-11-06T21:00:33.502669+0000","last_active":"2023-11-06T21:00:33.973314+0000","last_peered":"2023-11-06T21:00:33.973314+0000","last_clean":"2023-11-06T21:00:33.973314+0000","last_became_active":"2023-11-06T21:00:33.502367+0000","last_became_peered":"2023-11-06T21:00:33.502367+0000","last_unstale":"2023-11-06T21:00:33.973314+0000","last_undegraded":"2023-11-06T21:00:33.973314+0000","last_fullsized":"2023-11-06T21:00:33.973314+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006311+0000","last_change":"2023-11-06T21:00:33.995410+0000","last_active":"2023-11-06T21:00:34.006311+0000","last_peered":"2023-11-06T21:00:34.006311+0000","last_clean":"2023-11-06T21:00:34.006311+0000","last_became_active":"2023-11-06T21:00:33.995082+0000","last_became_peered":"2023-11-06T21:00:33.995082+0000","last_unstale":"2023-11-06T21:00:34.006311+0000","last_undegraded":"2023-11-06T21:00:34.006311+0000","last_fullsized":"2023-11-06T21:00:34.006311+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.992722+0000","last_change":"2023-11-06T21:00:33.497168+0000","last_active":"2023-11-06T21:00:33.992722+0000","last_peered":"2023-11-06T21:00:33.992722+0000","last_clean":"2023-11-06T21:00:33.992722+0000","last_became_active":"2023-11-06T21:00:33.495184+0000","last_became_peered":"2023-11-06T21:00:33.495184+0000","last_unstale":"2023-11-06T21:00:33.992722+0000","last_undegraded":"2023-11-06T21:00:33.992722+0000","last_fullsized":"2023-11-06T21:00:33.992722+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002193+0000","last_change":"2023-11-06T21:00:33.990185+0000","last_active":"2023-11-06T21:00:34.002193+0000","last_peered":"2023-11-06T21:00:34.002193+0000","last_clean":"2023-11-06T21:00:34.002193+0000","last_became_active":"2023-11-06T21:00:33.989838+0000","last_became_peered":"2023-11-06T21:00:33.989838+0000","last_unstale":"2023-11-06T21:00:34.002193+0000","last_undegraded":"2023-11-06T21:00:34.002193+0000","last_fullsized":"2023-11-06T21:00:34.002193+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.30","version":"113'4","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004597+0000","last_change":"2023-11-06T21:00:33.504781+0000","last_active":"2023-11-06T21:00:34.004597+0000","last_peered":"2023-11-06T21:00:34.004597+0000","last_clean":"2023-11-06T21:00:34.004597+0000","last_became_active":"2023-11-06T21:00:33.504455+0000","last_became_peered":"2023-11-06T21:00:33.504455+0000","last_unstale":"2023-11-06T21:00:34.004597+0000","last_undegraded":"2023-11-06T21:00:34.004597+0000","last_fullsized":"2023-11-06T21:00:34.004597+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008468+0000","last_change":"2023-11-06T21:00:34.003399+0000","last_active":"2023-11-06T21:00:34.008468+0000","last_peered":"2023-11-06T21:00:34.008468+0000","last_clean":"2023-11-06T21:00:34.008468+0000","last_became_active":"2023-11-06T21:00:34.002952+0000","last_became_peered":"2023-11-06T21:00:34.002952+0000","last_unstale":"2023-11-06T21:00:34.008468+0000","last_undegraded":"2023-11-06T21:00:34.008468+0000","last_fullsized":"2023-11-06T21:00:34.008468+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.991927+0000","last_change":"2023-11-06T21:00:33.500695+0000","last_active":"2023-11-06T21:00:33.991927+0000","last_peered":"2023-11-06T21:00:33.991927+0000","last_clean":"2023-11-06T21:00:33.991927+0000","last_became_active":"2023-11-06T21:00:33.500323+0000","last_became_peered":"2023-11-06T21:00:33.500323+0000","last_unstale":"2023-11-06T21:00:33.991927+0000","last_undegraded":"2023-11-06T21:00:33.991927+0000","last_fullsized":"2023-11-06T21:00:33.991927+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002209+0000","last_change":"2023-11-06T21:00:33.995121+0000","last_active":"2023-11-06T21:00:34.002209+0000","last_peered":"2023-11-06T21:00:34.002209+0000","last_clean":"2023-11-06T21:00:34.002209+0000","last_became_active":"2023-11-06T21:00:33.994566+0000","last_became_peered":"2023-11-06T21:00:33.994566+0000","last_unstale":"2023-11-06T21:00:34.002209+0000","last_undegraded":"2023-11-06T21:00:34.002209+0000","last_fullsized":"2023-11-06T21:00:34.002209+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.980998+0000","last_change":"2023-11-06T21:00:33.509086+0000","last_active":"2023-11-06T21:00:33.980998+0000","last_peered":"2023-11-06T21:00:33.980998+0000","last_clean":"2023-11-06T21:00:33.980998+0000","last_became_active":"2023-11-06T21:00:33.508718+0000","last_became_peered":"2023-11-06T21:00:33.508718+0000","last_unstale":"2023-11-06T21:00:33.980998+0000","last_undegraded":"2023-11-06T21:00:33.980998+0000","last_fullsized":"2023-11-06T21:00:33.980998+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.000880+0000","last_change":"2023-11-06T21:00:33.996484+0000","last_active":"2023-11-06T21:00:34.000880+0000","last_peered":"2023-11-06T21:00:34.000880+0000","last_clean":"2023-11-06T21:00:34.000880+0000","last_became_active":"2023-11-06T21:00:33.996140+0000","last_became_peered":"2023-11-06T21:00:33.996140+0000","last_unstale":"2023-11-06T21:00:34.000880+0000","last_undegraded":"2023-11-06T21:00:34.000880+0000","last_fullsized":"2023-11-06T21:00:34.000880+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2d","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003046+0000","last_change":"2023-11-06T21:00:33.507896+0000","last_active":"2023-11-06T21:00:34.003046+0000","last_peered":"2023-11-06T21:00:34.003046+0000","last_clean":"2023-11-06T21:00:34.003046+0000","last_became_active":"2023-11-06T21:00:33.507593+0000","last_became_peered":"2023-11-06T21:00:33.507593+0000","last_unstale":"2023-11-06T21:00:34.003046+0000","last_undegraded":"2023-11-06T21:00:34.003046+0000","last_fullsized":"2023-11-06T21:00:34.003046+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008744+0000","last_change":"2023-11-06T21:00:34.004324+0000","last_active":"2023-11-06T21:00:34.008744+0000","last_peered":"2023-11-06T21:00:34.008744+0000","last_clean":"2023-11-06T21:00:34.008744+0000","last_became_active":"2023-11-06T21:00:34.003957+0000","last_became_peered":"2023-11-06T21:00:34.003957+0000","last_unstale":"2023-11-06T21:00:34.008744+0000","last_undegraded":"2023-11-06T21:00:34.008744+0000","last_fullsized":"2023-11-06T21:00:34.008744+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.975569+0000","last_change":"2023-11-06T21:00:33.504626+0000","last_active":"2023-11-06T21:00:33.975569+0000","last_peered":"2023-11-06T21:00:33.975569+0000","last_clean":"2023-11-06T21:00:33.975569+0000","last_became_active":"2023-11-06T21:00:33.502414+0000","last_became_peered":"2023-11-06T21:00:33.502414+0000","last_unstale":"2023-11-06T21:00:33.975569+0000","last_undegraded":"2023-11-06T21:00:33.975569+0000","last_fullsized":"2023-11-06T21:00:33.975569+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004059+0000","last_change":"2023-11-06T21:00:33.991346+0000","last_active":"2023-11-06T21:00:34.004059+0000","last_peered":"2023-11-06T21:00:34.004059+0000","last_clean":"2023-11-06T21:00:34.004059+0000","last_became_active":"2023-11-06T21:00:33.989703+0000","last_became_peered":"2023-11-06T21:00:33.989703+0000","last_unstale":"2023-11-06T21:00:34.004059+0000","last_undegraded":"2023-11-06T21:00:34.004059+0000","last_fullsized":"2023-11-06T21:00:34.004059+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2b","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002099+0000","last_change":"2023-11-06T21:00:33.504535+0000","last_active":"2023-11-06T21:00:34.002099+0000","last_peered":"2023-11-06T21:00:34.002099+0000","last_clean":"2023-11-06T21:00:34.002099+0000","last_became_active":"2023-11-06T21:00:33.504217+0000","last_became_peered":"2023-11-06T21:00:33.504217+0000","last_unstale":"2023-11-06T21:00:34.002099+0000","last_undegraded":"2023-11-06T21:00:34.002099+0000","last_fullsized":"2023-11-06T21:00:34.002099+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989760+0000","last_change":"2023-11-06T21:00:33.986263+0000","last_active":"2023-11-06T21:00:33.989760+0000","last_peered":"2023-11-06T21:00:33.989760+0000","last_clean":"2023-11-06T21:00:33.989760+0000","last_became_active":"2023-11-06T21:00:33.985933+0000","last_became_peered":"2023-11-06T21:00:33.985933+0000","last_unstale":"2023-11-06T21:00:33.989760+0000","last_undegraded":"2023-11-06T21:00:33.989760+0000","last_fullsized":"2023-11-06T21:00:33.989760+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973209+0000","last_change":"2023-11-06T21:00:33.506301+0000","last_active":"2023-11-06T21:00:33.973209+0000","last_peered":"2023-11-06T21:00:33.973209+0000","last_clean":"2023-11-06T21:00:33.973209+0000","last_became_active":"2023-11-06T21:00:33.506006+0000","last_became_peered":"2023-11-06T21:00:33.506006+0000","last_unstale":"2023-11-06T21:00:33.973209+0000","last_undegraded":"2023-11-06T21:00:33.973209+0000","last_fullsized":"2023-11-06T21:00:33.973209+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.999602+0000","last_change":"2023-11-06T21:00:33.990242+0000","last_active":"2023-11-06T21:00:33.999602+0000","last_peered":"2023-11-06T21:00:33.999602+0000","last_clean":"2023-11-06T21:00:33.999602+0000","last_became_active":"2023-11-06T21:00:33.989932+0000","last_became_peered":"2023-11-06T21:00:33.989932+0000","last_unstale":"2023-11-06T21:00:33.999602+0000","last_undegraded":"2023-11-06T21:00:33.999602+0000","last_fullsized":"2023-11-06T21:00:33.999602+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989012+0000","last_change":"2023-11-06T21:00:33.508959+0000","last_active":"2023-11-06T21:00:33.989012+0000","last_peered":"2023-11-06T21:00:33.989012+0000","last_clean":"2023-11-06T21:00:33.989012+0000","last_became_active":"2023-11-06T21:00:33.508590+0000","last_became_peered":"2023-11-06T21:00:33.508590+0000","last_unstale":"2023-11-06T21:00:33.989012+0000","last_undegraded":"2023-11-06T21:00:33.989012+0000","last_fullsized":"2023-11-06T21:00:33.989012+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001752+0000","last_change":"2023-11-06T21:00:33.997337+0000","last_active":"2023-11-06T21:00:34.001752+0000","last_peered":"2023-11-06T21:00:34.001752+0000","last_clean":"2023-11-06T21:00:34.001752+0000","last_became_active":"2023-11-06T21:00:33.996737+0000","last_became_peered":"2023-11-06T21:00:33.996737+0000","last_unstale":"2023-11-06T21:00:34.001752+0000","last_undegraded":"2023-11-06T21:00:34.001752+0000","last_fullsized":"2023-11-06T21:00:34.001752+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981086+0000","last_change":"2023-11-06T21:00:33.504796+0000","last_active":"2023-11-06T21:00:33.981086+0000","last_peered":"2023-11-06T21:00:33.981086+0000","last_clean":"2023-11-06T21:00:33.981086+0000","last_became_active":"2023-11-06T21:00:33.503331+0000","last_became_peered":"2023-11-06T21:00:33.503331+0000","last_unstale":"2023-11-06T21:00:33.981086+0000","last_undegraded":"2023-11-06T21:00:33.981086+0000","last_fullsized":"2023-11-06T21:00:33.981086+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.994997+0000","last_change":"2023-11-06T21:00:33.983077+0000","last_active":"2023-11-06T21:00:33.994997+0000","last_peered":"2023-11-06T21:00:33.994997+0000","last_clean":"2023-11-06T21:00:33.994997+0000","last_became_active":"2023-11-06T21:00:33.982804+0000","last_became_peered":"2023-11-06T21:00:33.982804+0000","last_unstale":"2023-11-06T21:00:33.994997+0000","last_undegraded":"2023-11-06T21:00:33.994997+0000","last_fullsized":"2023-11-06T21:00:33.994997+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981175+0000","last_change":"2023-11-06T21:00:33.513913+0000","last_active":"2023-11-06T21:00:33.981175+0000","last_peered":"2023-11-06T21:00:33.981175+0000","last_clean":"2023-11-06T21:00:33.981175+0000","last_became_active":"2023-11-06T21:00:33.513590+0000","last_became_peered":"2023-11-06T21:00:33.513590+0000","last_unstale":"2023-11-06T21:00:33.981175+0000","last_undegraded":"2023-11-06T21:00:33.981175+0000","last_fullsized":"2023-11-06T21:00:33.981175+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.000512+0000","last_change":"2023-11-06T21:00:33.997347+0000","last_active":"2023-11-06T21:00:34.000512+0000","last_peered":"2023-11-06T21:00:34.000512+0000","last_clean":"2023-11-06T21:00:34.000512+0000","last_became_active":"2023-11-06T21:00:33.996154+0000","last_became_peered":"2023-11-06T21:00:33.996154+0000","last_unstale":"2023-11-06T21:00:34.000512+0000","last_undegraded":"2023-11-06T21:00:34.000512+0000","last_fullsized":"2023-11-06T21:00:34.000512+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974900+0000","last_change":"2023-11-06T21:00:33.493162+0000","last_active":"2023-11-06T21:00:33.974900+0000","last_peered":"2023-11-06T21:00:33.974900+0000","last_clean":"2023-11-06T21:00:33.974900+0000","last_became_active":"2023-11-06T21:00:33.490940+0000","last_became_peered":"2023-11-06T21:00:33.490940+0000","last_unstale":"2023-11-06T21:00:33.974900+0000","last_undegraded":"2023-11-06T21:00:33.974900+0000","last_fullsized":"2023-11-06T21:00:33.974900+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003932+0000","last_change":"2023-11-06T21:00:33.998602+0000","last_active":"2023-11-06T21:00:34.003932+0000","last_peered":"2023-11-06T21:00:34.003932+0000","last_clean":"2023-11-06T21:00:34.003932+0000","last_became_active":"2023-11-06T21:00:33.995194+0000","last_became_peered":"2023-11-06T21:00:33.995194+0000","last_unstale":"2023-11-06T21:00:34.003932+0000","last_undegraded":"2023-11-06T21:00:34.003932+0000","last_fullsized":"2023-11-06T21:00:34.003932+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974604+0000","last_change":"2023-11-06T21:00:33.494908+0000","last_active":"2023-11-06T21:00:33.974604+0000","last_peered":"2023-11-06T21:00:33.974604+0000","last_clean":"2023-11-06T21:00:33.974604+0000","last_became_active":"2023-11-06T21:00:33.494624+0000","last_became_peered":"2023-11-06T21:00:33.494624+0000","last_unstale":"2023-11-06T21:00:33.974604+0000","last_undegraded":"2023-11-06T21:00:33.974604+0000","last_fullsized":"2023-11-06T21:00:33.974604+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001486+0000","last_change":"2023-11-06T21:00:33.996894+0000","last_active":"2023-11-06T21:00:34.001486+0000","last_peered":"2023-11-06T21:00:34.001486+0000","last_clean":"2023-11-06T21:00:34.001486+0000","last_became_active":"2023-11-06T21:00:33.996323+0000","last_became_peered":"2023-11-06T21:00:33.996323+0000","last_unstale":"2023-11-06T21:00:34.001486+0000","last_undegraded":"2023-11-06T21:00:34.001486+0000","last_fullsized":"2023-11-06T21:00:34.001486+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971779+0000","last_change":"2023-11-06T21:00:33.507514+0000","last_active":"2023-11-06T21:00:33.971779+0000","last_peered":"2023-11-06T21:00:33.971779+0000","last_clean":"2023-11-06T21:00:33.971779+0000","last_became_active":"2023-11-06T21:00:33.507243+0000","last_became_peered":"2023-11-06T21:00:33.507243+0000","last_unstale":"2023-11-06T21:00:33.971779+0000","last_undegraded":"2023-11-06T21:00:33.971779+0000","last_fullsized":"2023-11-06T21:00:33.971779+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008852+0000","last_change":"2023-11-06T21:00:34.004845+0000","last_active":"2023-11-06T21:00:34.008852+0000","last_peered":"2023-11-06T21:00:34.008852+0000","last_clean":"2023-11-06T21:00:34.008852+0000","last_became_active":"2023-11-06T21:00:34.004567+0000","last_became_peered":"2023-11-06T21:00:34.004567+0000","last_unstale":"2023-11-06T21:00:34.008852+0000","last_undegraded":"2023-11-06T21:00:34.008852+0000","last_fullsized":"2023-11-06T21:00:34.008852+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989125+0000","last_change":"2023-11-06T21:00:33.501114+0000","last_active":"2023-11-06T21:00:33.989125+0000","last_peered":"2023-11-06T21:00:33.989125+0000","last_clean":"2023-11-06T21:00:33.989125+0000","last_became_active":"2023-11-06T21:00:33.500796+0000","last_became_peered":"2023-11-06T21:00:33.500796+0000","last_unstale":"2023-11-06T21:00:33.989125+0000","last_undegraded":"2023-11-06T21:00:33.989125+0000","last_fullsized":"2023-11-06T21:00:33.989125+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.999943+0000","last_change":"2023-11-06T21:00:33.993642+0000","last_active":"2023-11-06T21:00:33.999943+0000","last_peered":"2023-11-06T21:00:33.999943+0000","last_clean":"2023-11-06T21:00:33.999943+0000","last_became_active":"2023-11-06T21:00:33.993165+0000","last_became_peered":"2023-11-06T21:00:33.993165+0000","last_unstale":"2023-11-06T21:00:33.999943+0000","last_undegraded":"2023-11-06T21:00:33.999943+0000","last_fullsized":"2023-11-06T21:00:33.999943+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973100+0000","last_change":"2023-11-06T21:00:33.508157+0000","last_active":"2023-11-06T21:00:33.973100+0000","last_peered":"2023-11-06T21:00:33.973100+0000","last_clean":"2023-11-06T21:00:33.973100+0000","last_became_active":"2023-11-06T21:00:33.507854+0000","last_became_peered":"2023-11-06T21:00:33.507854+0000","last_unstale":"2023-11-06T21:00:33.973100+0000","last_undegraded":"2023-11-06T21:00:33.973100+0000","last_fullsized":"2023-11-06T21:00:33.973100+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989604+0000","last_change":"2023-11-06T21:00:33.984857+0000","last_active":"2023-11-06T21:00:33.989604+0000","last_peered":"2023-11-06T21:00:33.989604+0000","last_clean":"2023-11-06T21:00:33.989604+0000","last_became_active":"2023-11-06T21:00:33.984584+0000","last_became_peered":"2023-11-06T21:00:33.984584+0000","last_unstale":"2023-11-06T21:00:33.989604+0000","last_undegraded":"2023-11-06T21:00:33.989604+0000","last_fullsized":"2023-11-06T21:00:33.989604+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.980915+0000","last_change":"2023-11-06T21:00:33.494846+0000","last_active":"2023-11-06T21:00:33.980915+0000","last_peered":"2023-11-06T21:00:33.980915+0000","last_clean":"2023-11-06T21:00:33.980915+0000","last_became_active":"2023-11-06T21:00:33.494551+0000","last_became_peered":"2023-11-06T21:00:33.494551+0000","last_unstale":"2023-11-06T21:00:33.980915+0000","last_undegraded":"2023-11-06T21:00:33.980915+0000","last_fullsized":"2023-11-06T21:00:33.980915+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001713+0000","last_change":"2023-11-06T21:00:33.995501+0000","last_active":"2023-11-06T21:00:34.001713+0000","last_peered":"2023-11-06T21:00:34.001713+0000","last_clean":"2023-11-06T21:00:34.001713+0000","last_became_active":"2023-11-06T21:00:33.992247+0000","last_became_peered":"2023-11-06T21:00:33.992247+0000","last_unstale":"2023-11-06T21:00:34.001713+0000","last_undegraded":"2023-11-06T21:00:34.001713+0000","last_fullsized":"2023-11-06T21:00:34.001713+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981167+0000","last_change":"2023-11-06T21:00:33.493627+0000","last_active":"2023-11-06T21:00:33.981167+0000","last_peered":"2023-11-06T21:00:33.981167+0000","last_clean":"2023-11-06T21:00:33.981167+0000","last_became_active":"2023-11-06T21:00:33.493323+0000","last_became_peered":"2023-11-06T21:00:33.493323+0000","last_unstale":"2023-11-06T21:00:33.981167+0000","last_undegraded":"2023-11-06T21:00:33.981167+0000","last_fullsized":"2023-11-06T21:00:33.981167+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.007749+0000","last_change":"2023-11-06T21:00:33.999025+0000","last_active":"2023-11-06T21:00:34.007749+0000","last_peered":"2023-11-06T21:00:34.007749+0000","last_clean":"2023-11-06T21:00:34.007749+0000","last_became_active":"2023-11-06T21:00:33.998592+0000","last_became_peered":"2023-11-06T21:00:33.998592+0000","last_unstale":"2023-11-06T21:00:34.007749+0000","last_undegraded":"2023-11-06T21:00:34.007749+0000","last_fullsized":"2023-11-06T21:00:34.007749+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001335+0000","last_change":"2023-11-06T21:00:33.501241+0000","last_active":"2023-11-06T21:00:34.001335+0000","last_peered":"2023-11-06T21:00:34.001335+0000","last_clean":"2023-11-06T21:00:34.001335+0000","last_became_active":"2023-11-06T21:00:33.500900+0000","last_became_peered":"2023-11-06T21:00:33.500900+0000","last_unstale":"2023-11-06T21:00:34.001335+0000","last_undegraded":"2023-11-06T21:00:34.001335+0000","last_fullsized":"2023-11-06T21:00:34.001335+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008600+0000","last_change":"2023-11-06T21:00:34.003507+0000","last_active":"2023-11-06T21:00:34.008600+0000","last_peered":"2023-11-06T21:00:34.008600+0000","last_clean":"2023-11-06T21:00:34.008600+0000","last_became_active":"2023-11-06T21:00:34.003233+0000","last_became_peered":"2023-11-06T21:00:34.003233+0000","last_unstale":"2023-11-06T21:00:34.008600+0000","last_undegraded":"2023-11-06T21:00:34.008600+0000","last_fullsized":"2023-11-06T21:00:34.008600+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006314+0000","last_change":"2023-11-06T21:00:33.509163+0000","last_active":"2023-11-06T21:00:34.006314+0000","last_peered":"2023-11-06T21:00:34.006314+0000","last_clean":"2023-11-06T21:00:34.006314+0000","last_became_active":"2023-11-06T21:00:33.508745+0000","last_became_peered":"2023-11-06T21:00:33.508745+0000","last_unstale":"2023-11-06T21:00:34.006314+0000","last_undegraded":"2023-11-06T21:00:34.006314+0000","last_fullsized":"2023-11-06T21:00:34.006314+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004164+0000","last_change":"2023-11-06T21:00:33.998100+0000","last_active":"2023-11-06T21:00:34.004164+0000","last_peered":"2023-11-06T21:00:34.004164+0000","last_clean":"2023-11-06T21:00:34.004164+0000","last_became_active":"2023-11-06T21:00:33.996793+0000","last_became_peered":"2023-11-06T21:00:33.996793+0000","last_unstale":"2023-11-06T21:00:34.004164+0000","last_undegraded":"2023-11-06T21:00:34.004164+0000","last_fullsized":"2023-11-06T21:00:34.004164+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.000122+0000","last_change":"2023-11-06T21:00:33.504531+0000","last_active":"2023-11-06T21:00:34.000122+0000","last_peered":"2023-11-06T21:00:34.000122+0000","last_clean":"2023-11-06T21:00:34.000122+0000","last_became_active":"2023-11-06T21:00:33.504164+0000","last_became_peered":"2023-11-06T21:00:33.504164+0000","last_unstale":"2023-11-06T21:00:34.000122+0000","last_undegraded":"2023-11-06T21:00:34.000122+0000","last_fullsized":"2023-11-06T21:00:34.000122+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004474+0000","last_change":"2023-11-06T21:00:34.001550+0000","last_active":"2023-11-06T21:00:34.004474+0000","last_peered":"2023-11-06T21:00:34.004474+0000","last_clean":"2023-11-06T21:00:34.004474+0000","last_became_active":"2023-11-06T21:00:34.001233+0000","last_became_peered":"2023-11-06T21:00:34.001233+0000","last_unstale":"2023-11-06T21:00:34.004474+0000","last_undegraded":"2023-11-06T21:00:34.004474+0000","last_fullsized":"2023-11-06T21:00:34.004474+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.988863+0000","last_change":"2023-11-06T21:00:33.498742+0000","last_active":"2023-11-06T21:00:33.988863+0000","last_peered":"2023-11-06T21:00:33.988863+0000","last_clean":"2023-11-06T21:00:33.988863+0000","last_became_active":"2023-11-06T21:00:33.498013+0000","last_became_peered":"2023-11-06T21:00:33.498013+0000","last_unstale":"2023-11-06T21:00:33.988863+0000","last_undegraded":"2023-11-06T21:00:33.988863+0000","last_fullsized":"2023-11-06T21:00:33.988863+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002489+0000","last_change":"2023-11-06T21:00:33.990825+0000","last_active":"2023-11-06T21:00:34.002489+0000","last_peered":"2023-11-06T21:00:34.002489+0000","last_clean":"2023-11-06T21:00:34.002489+0000","last_became_active":"2023-11-06T21:00:33.987540+0000","last_became_peered":"2023-11-06T21:00:33.987540+0000","last_unstale":"2023-11-06T21:00:34.002489+0000","last_undegraded":"2023-11-06T21:00:34.002489+0000","last_fullsized":"2023-11-06T21:00:34.002489+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981916+0000","last_change":"2023-11-06T21:00:33.500170+0000","last_active":"2023-11-06T21:00:33.981916+0000","last_peered":"2023-11-06T21:00:33.981916+0000","last_clean":"2023-11-06T21:00:33.981916+0000","last_became_active":"2023-11-06T21:00:33.499851+0000","last_became_peered":"2023-11-06T21:00:33.499851+0000","last_unstale":"2023-11-06T21:00:33.981916+0000","last_undegraded":"2023-11-06T21:00:33.981916+0000","last_fullsized":"2023-11-06T21:00:33.981916+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.983613+0000","last_change":"2023-11-06T21:00:33.979672+0000","last_active":"2023-11-06T21:00:33.983613+0000","last_peered":"2023-11-06T21:00:33.983613+0000","last_clean":"2023-11-06T21:00:33.983613+0000","last_became_active":"2023-11-06T21:00:33.979373+0000","last_became_peered":"2023-11-06T21:00:33.979373+0000","last_unstale":"2023-11-06T21:00:33.983613+0000","last_undegraded":"2023-11-06T21:00:33.983613+0000","last_fullsized":"2023-11-06T21:00:33.983613+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.988188+0000","last_change":"2023-11-06T21:00:33.506600+0000","last_active":"2023-11-06T21:00:33.988188+0000","last_peered":"2023-11-06T21:00:33.988188+0000","last_clean":"2023-11-06T21:00:33.988188+0000","last_became_active":"2023-11-06T21:00:33.506285+0000","last_became_peered":"2023-11-06T21:00:33.506285+0000","last_unstale":"2023-11-06T21:00:33.988188+0000","last_undegraded":"2023-11-06T21:00:33.988188+0000","last_fullsized":"2023-11-06T21:00:33.988188+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003933+0000","last_change":"2023-11-06T21:00:33.999110+0000","last_active":"2023-11-06T21:00:34.003933+0000","last_peered":"2023-11-06T21:00:34.003933+0000","last_clean":"2023-11-06T21:00:34.003933+0000","last_became_active":"2023-11-06T21:00:33.998611+0000","last_became_peered":"2023-11-06T21:00:33.998611+0000","last_unstale":"2023-11-06T21:00:34.003933+0000","last_undegraded":"2023-11-06T21:00:34.003933+0000","last_fullsized":"2023-11-06T21:00:34.003933+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974256+0000","last_change":"2023-11-06T21:00:33.508266+0000","last_active":"2023-11-06T21:00:33.974256+0000","last_peered":"2023-11-06T21:00:33.974256+0000","last_clean":"2023-11-06T21:00:33.974256+0000","last_became_active":"2023-11-06T21:00:33.507866+0000","last_became_peered":"2023-11-06T21:00:33.507866+0000","last_unstale":"2023-11-06T21:00:33.974256+0000","last_undegraded":"2023-11-06T21:00:33.974256+0000","last_fullsized":"2023-11-06T21:00:33.974256+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.998336+0000","last_change":"2023-11-06T21:00:33.989497+0000","last_active":"2023-11-06T21:00:33.998336+0000","last_peered":"2023-11-06T21:00:33.998336+0000","last_clean":"2023-11-06T21:00:33.998336+0000","last_became_active":"2023-11-06T21:00:33.988697+0000","last_became_peered":"2023-11-06T21:00:33.988697+0000","last_unstale":"2023-11-06T21:00:33.998336+0000","last_undegraded":"2023-11-06T21:00:33.998336+0000","last_fullsized":"2023-11-06T21:00:33.998336+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971644+0000","last_change":"2023-11-06T21:00:33.502195+0000","last_active":"2023-11-06T21:00:33.971644+0000","last_peered":"2023-11-06T21:00:33.971644+0000","last_clean":"2023-11-06T21:00:33.971644+0000","last_became_active":"2023-11-06T21:00:33.501125+0000","last_became_peered":"2023-11-06T21:00:33.501125+0000","last_unstale":"2023-11-06T21:00:33.971644+0000","last_undegraded":"2023-11-06T21:00:33.971644+0000","last_fullsized":"2023-11-06T21:00:33.971644+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.998410+0000","last_change":"2023-11-06T21:00:33.993666+0000","last_active":"2023-11-06T21:00:33.998410+0000","last_peered":"2023-11-06T21:00:33.998410+0000","last_clean":"2023-11-06T21:00:33.998410+0000","last_became_active":"2023-11-06T21:00:33.990418+0000","last_became_peered":"2023-11-06T21:00:33.990418+0000","last_unstale":"2023-11-06T21:00:33.998410+0000","last_undegraded":"2023-11-06T21:00:33.998410+0000","last_fullsized":"2023-11-06T21:00:33.998410+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.972534+0000","last_change":"2023-11-06T21:00:33.505514+0000","last_active":"2023-11-06T21:00:33.972534+0000","last_peered":"2023-11-06T21:00:33.972534+0000","last_clean":"2023-11-06T21:00:33.972534+0000","last_became_active":"2023-11-06T21:00:33.505168+0000","last_became_peered":"2023-11-06T21:00:33.505168+0000","last_unstale":"2023-11-06T21:00:33.972534+0000","last_undegraded":"2023-11-06T21:00:33.972534+0000","last_fullsized":"2023-11-06T21:00:33.972534+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.003858+0000","last_change":"2023-11-06T21:00:33.991115+0000","last_active":"2023-11-06T21:00:34.003858+0000","last_peered":"2023-11-06T21:00:34.003858+0000","last_clean":"2023-11-06T21:00:34.003858+0000","last_became_active":"2023-11-06T21:00:33.990842+0000","last_became_peered":"2023-11-06T21:00:33.990842+0000","last_unstale":"2023-11-06T21:00:34.003858+0000","last_undegraded":"2023-11-06T21:00:34.003858+0000","last_fullsized":"2023-11-06T21:00:34.003858+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973823+0000","last_change":"2023-11-06T21:00:33.510936+0000","last_active":"2023-11-06T21:00:33.973823+0000","last_peered":"2023-11-06T21:00:33.973823+0000","last_clean":"2023-11-06T21:00:33.973823+0000","last_became_active":"2023-11-06T21:00:33.510597+0000","last_became_peered":"2023-11-06T21:00:33.510597+0000","last_unstale":"2023-11-06T21:00:33.973823+0000","last_undegraded":"2023-11-06T21:00:33.973823+0000","last_fullsized":"2023-11-06T21:00:33.973823+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.007822+0000","last_change":"2023-11-06T21:00:33.998999+0000","last_active":"2023-11-06T21:00:34.007822+0000","last_peered":"2023-11-06T21:00:34.007822+0000","last_clean":"2023-11-06T21:00:34.007822+0000","last_became_active":"2023-11-06T21:00:33.998625+0000","last_became_peered":"2023-11-06T21:00:33.998625+0000","last_unstale":"2023-11-06T21:00:34.007822+0000","last_undegraded":"2023-11-06T21:00:34.007822+0000","last_fullsized":"2023-11-06T21:00:34.007822+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"113'3","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.007569+0000","last_change":"2023-11-06T21:00:33.508112+0000","last_active":"2023-11-06T21:00:34.007569+0000","last_peered":"2023-11-06T21:00:34.007569+0000","last_clean":"2023-11-06T21:00:34.007569+0000","last_became_active":"2023-11-06T21:00:33.507601+0000","last_became_peered":"2023-11-06T21:00:33.507601+0000","last_unstale":"2023-11-06T21:00:34.007569+0000","last_undegraded":"2023-11-06T21:00:34.007569+0000","last_fullsized":"2023-11-06T21:00:34.007569+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.009041+0000","last_change":"2023-11-06T21:00:34.005671+0000","last_active":"2023-11-06T21:00:34.009041+0000","last_peered":"2023-11-06T21:00:34.009041+0000","last_clean":"2023-11-06T21:00:34.009041+0000","last_became_active":"2023-11-06T21:00:34.005392+0000","last_became_peered":"2023-11-06T21:00:34.005392+0000","last_unstale":"2023-11-06T21:00:34.009041+0000","last_undegraded":"2023-11-06T21:00:34.009041+0000","last_fullsized":"2023-11-06T21:00:34.009041+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004731+0000","last_change":"2023-11-06T21:00:33.512547+0000","last_active":"2023-11-06T21:00:34.004731+0000","last_peered":"2023-11-06T21:00:34.004731+0000","last_clean":"2023-11-06T21:00:34.004731+0000","last_became_active":"2023-11-06T21:00:33.512284+0000","last_became_peered":"2023-11-06T21:00:33.512284+0000","last_unstale":"2023-11-06T21:00:34.004731+0000","last_undegraded":"2023-11-06T21:00:34.004731+0000","last_fullsized":"2023-11-06T21:00:34.004731+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006085+0000","last_change":"2023-11-06T21:00:33.994039+0000","last_active":"2023-11-06T21:00:34.006085+0000","last_peered":"2023-11-06T21:00:34.006085+0000","last_clean":"2023-11-06T21:00:34.006085+0000","last_became_active":"2023-11-06T21:00:33.993609+0000","last_became_peered":"2023-11-06T21:00:33.993609+0000","last_unstale":"2023-11-06T21:00:34.006085+0000","last_undegraded":"2023-11-06T21:00:34.006085+0000","last_fullsized":"2023-11-06T21:00:34.006085+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.000581+0000","last_change":"2023-11-06T21:00:33.507648+0000","last_active":"2023-11-06T21:00:34.000581+0000","last_peered":"2023-11-06T21:00:34.000581+0000","last_clean":"2023-11-06T21:00:34.000581+0000","last_became_active":"2023-11-06T21:00:33.505304+0000","last_became_peered":"2023-11-06T21:00:33.505304+0000","last_unstale":"2023-11-06T21:00:34.000581+0000","last_undegraded":"2023-11-06T21:00:34.000581+0000","last_fullsized":"2023-11-06T21:00:34.000581+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989476+0000","last_change":"2023-11-06T21:00:33.984386+0000","last_active":"2023-11-06T21:00:33.989476+0000","last_peered":"2023-11-06T21:00:33.989476+0000","last_clean":"2023-11-06T21:00:33.989476+0000","last_became_active":"2023-11-06T21:00:33.984017+0000","last_became_peered":"2023-11-06T21:00:33.984017+0000","last_unstale":"2023-11-06T21:00:33.989476+0000","last_undegraded":"2023-11-06T21:00:33.989476+0000","last_fullsized":"2023-11-06T21:00:33.989476+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.980656+0000","last_change":"2023-11-06T21:00:33.516349+0000","last_active":"2023-11-06T21:00:33.980656+0000","last_peered":"2023-11-06T21:00:33.980656+0000","last_clean":"2023-11-06T21:00:33.980656+0000","last_became_active":"2023-11-06T21:00:33.516078+0000","last_became_peered":"2023-11-06T21:00:33.516078+0000","last_unstale":"2023-11-06T21:00:33.980656+0000","last_undegraded":"2023-11-06T21:00:33.980656+0000","last_fullsized":"2023-11-06T21:00:33.980656+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.005250+0000","last_change":"2023-11-06T21:00:33.992182+0000","last_active":"2023-11-06T21:00:34.005250+0000","last_peered":"2023-11-06T21:00:34.005250+0000","last_clean":"2023-11-06T21:00:34.005250+0000","last_became_active":"2023-11-06T21:00:33.991869+0000","last_became_peered":"2023-11-06T21:00:33.991869+0000","last_unstale":"2023-11-06T21:00:34.005250+0000","last_undegraded":"2023-11-06T21:00:34.005250+0000","last_fullsized":"2023-11-06T21:00:34.005250+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981799+0000","last_change":"2023-11-06T21:00:33.503527+0000","last_active":"2023-11-06T21:00:33.981799+0000","last_peered":"2023-11-06T21:00:33.981799+0000","last_clean":"2023-11-06T21:00:33.981799+0000","last_became_active":"2023-11-06T21:00:33.503227+0000","last_became_peered":"2023-11-06T21:00:33.503227+0000","last_unstale":"2023-11-06T21:00:33.981799+0000","last_undegraded":"2023-11-06T21:00:33.981799+0000","last_fullsized":"2023-11-06T21:00:33.981799+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.995333+0000","last_change":"2023-11-06T21:00:33.985818+0000","last_active":"2023-11-06T21:00:33.995333+0000","last_peered":"2023-11-06T21:00:33.995333+0000","last_clean":"2023-11-06T21:00:33.995333+0000","last_became_active":"2023-11-06T21:00:33.982772+0000","last_became_peered":"2023-11-06T21:00:33.982772+0000","last_unstale":"2023-11-06T21:00:33.995333+0000","last_undegraded":"2023-11-06T21:00:33.995333+0000","last_fullsized":"2023-11-06T21:00:33.995333+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"113'4","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.998821+0000","last_change":"2023-11-06T21:00:33.502715+0000","last_active":"2023-11-06T21:00:33.998821+0000","last_peered":"2023-11-06T21:00:33.998821+0000","last_clean":"2023-11-06T21:00:33.998821+0000","last_became_active":"2023-11-06T21:00:33.501512+0000","last_became_peered":"2023-11-06T21:00:33.501512+0000","last_unstale":"2023-11-06T21:00:33.998821+0000","last_undegraded":"2023-11-06T21:00:33.998821+0000","last_fullsized":"2023-11-06T21:00:33.998821+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.994959+0000","last_change":"2023-11-06T21:00:33.987754+0000","last_active":"2023-11-06T21:00:33.994959+0000","last_peered":"2023-11-06T21:00:33.994959+0000","last_clean":"2023-11-06T21:00:33.994959+0000","last_became_active":"2023-11-06T21:00:33.987482+0000","last_became_peered":"2023-11-06T21:00:33.987482+0000","last_unstale":"2023-11-06T21:00:33.994959+0000","last_undegraded":"2023-11-06T21:00:33.994959+0000","last_fullsized":"2023-11-06T21:00:33.994959+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006579+0000","last_change":"2023-11-06T21:00:33.502509+0000","last_active":"2023-11-06T21:00:34.006579+0000","last_peered":"2023-11-06T21:00:34.006579+0000","last_clean":"2023-11-06T21:00:34.006579+0000","last_became_active":"2023-11-06T21:00:33.502005+0000","last_became_peered":"2023-11-06T21:00:33.502005+0000","last_unstale":"2023-11-06T21:00:34.006579+0000","last_undegraded":"2023-11-06T21:00:34.006579+0000","last_fullsized":"2023-11-06T21:00:34.006579+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002333+0000","last_change":"2023-11-06T21:00:33.994110+0000","last_active":"2023-11-06T21:00:34.002333+0000","last_peered":"2023-11-06T21:00:34.002333+0000","last_clean":"2023-11-06T21:00:34.002333+0000","last_became_active":"2023-11-06T21:00:33.993060+0000","last_became_peered":"2023-11-06T21:00:33.993060+0000","last_unstale":"2023-11-06T21:00:34.002333+0000","last_undegraded":"2023-11-06T21:00:34.002333+0000","last_fullsized":"2023-11-06T21:00:34.002333+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974970+0000","last_change":"2023-11-06T21:00:33.512460+0000","last_active":"2023-11-06T21:00:33.974970+0000","last_peered":"2023-11-06T21:00:33.974970+0000","last_clean":"2023-11-06T21:00:33.974970+0000","last_became_active":"2023-11-06T21:00:33.510887+0000","last_became_peered":"2023-11-06T21:00:33.510887+0000","last_unstale":"2023-11-06T21:00:33.974970+0000","last_undegraded":"2023-11-06T21:00:33.974970+0000","last_fullsized":"2023-11-06T21:00:33.974970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002433+0000","last_change":"2023-11-06T21:00:33.997711+0000","last_active":"2023-11-06T21:00:34.002433+0000","last_peered":"2023-11-06T21:00:34.002433+0000","last_clean":"2023-11-06T21:00:34.002433+0000","last_became_active":"2023-11-06T21:00:33.996708+0000","last_became_peered":"2023-11-06T21:00:33.996708+0000","last_unstale":"2023-11-06T21:00:34.002433+0000","last_undegraded":"2023-11-06T21:00:34.002433+0000","last_fullsized":"2023-11-06T21:00:34.002433+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.984085+0000","last_change":"2023-11-06T21:00:33.500512+0000","last_active":"2023-11-06T21:00:33.984085+0000","last_peered":"2023-11-06T21:00:33.984085+0000","last_clean":"2023-11-06T21:00:33.984085+0000","last_became_active":"2023-11-06T21:00:33.499912+0000","last_became_peered":"2023-11-06T21:00:33.499912+0000","last_unstale":"2023-11-06T21:00:33.984085+0000","last_undegraded":"2023-11-06T21:00:33.984085+0000","last_fullsized":"2023-11-06T21:00:33.984085+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001439+0000","last_change":"2023-11-06T21:00:33.995924+0000","last_active":"2023-11-06T21:00:34.001439+0000","last_peered":"2023-11-06T21:00:34.001439+0000","last_clean":"2023-11-06T21:00:34.001439+0000","last_became_active":"2023-11-06T21:00:33.995177+0000","last_became_peered":"2023-11-06T21:00:33.995177+0000","last_unstale":"2023-11-06T21:00:34.001439+0000","last_undegraded":"2023-11-06T21:00:34.001439+0000","last_fullsized":"2023-11-06T21:00:34.001439+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974478+0000","last_change":"2023-11-06T21:00:33.492925+0000","last_active":"2023-11-06T21:00:33.974478+0000","last_peered":"2023-11-06T21:00:33.974478+0000","last_clean":"2023-11-06T21:00:33.974478+0000","last_became_active":"2023-11-06T21:00:33.490246+0000","last_became_peered":"2023-11-06T21:00:33.490246+0000","last_unstale":"2023-11-06T21:00:33.974478+0000","last_undegraded":"2023-11-06T21:00:33.974478+0000","last_fullsized":"2023-11-06T21:00:33.974478+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008816+0000","last_change":"2023-11-06T21:00:34.004445+0000","last_active":"2023-11-06T21:00:34.008816+0000","last_peered":"2023-11-06T21:00:34.008816+0000","last_clean":"2023-11-06T21:00:34.008816+0000","last_became_active":"2023-11-06T21:00:34.003867+0000","last_became_peered":"2023-11-06T21:00:34.003867+0000","last_unstale":"2023-11-06T21:00:34.008816+0000","last_undegraded":"2023-11-06T21:00:34.008816+0000","last_fullsized":"2023-11-06T21:00:34.008816+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981176+0000","last_change":"2023-11-06T21:00:33.500328+0000","last_active":"2023-11-06T21:00:33.981176+0000","last_peered":"2023-11-06T21:00:33.981176+0000","last_clean":"2023-11-06T21:00:33.981176+0000","last_became_active":"2023-11-06T21:00:33.499803+0000","last_became_peered":"2023-11-06T21:00:33.499803+0000","last_unstale":"2023-11-06T21:00:33.981176+0000","last_undegraded":"2023-11-06T21:00:33.981176+0000","last_fullsized":"2023-11-06T21:00:33.981176+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.999843+0000","last_change":"2023-11-06T21:00:33.991829+0000","last_active":"2023-11-06T21:00:33.999843+0000","last_peered":"2023-11-06T21:00:33.999843+0000","last_clean":"2023-11-06T21:00:33.999843+0000","last_became_active":"2023-11-06T21:00:33.991531+0000","last_became_peered":"2023-11-06T21:00:33.991531+0000","last_unstale":"2023-11-06T21:00:33.999843+0000","last_undegraded":"2023-11-06T21:00:33.999843+0000","last_fullsized":"2023-11-06T21:00:33.999843+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971505+0000","last_change":"2023-11-06T21:00:33.508090+0000","last_active":"2023-11-06T21:00:33.971505+0000","last_peered":"2023-11-06T21:00:33.971505+0000","last_clean":"2023-11-06T21:00:33.971505+0000","last_became_active":"2023-11-06T21:00:33.507822+0000","last_became_peered":"2023-11-06T21:00:33.507822+0000","last_unstale":"2023-11-06T21:00:33.971505+0000","last_undegraded":"2023-11-06T21:00:33.971505+0000","last_fullsized":"2023-11-06T21:00:33.971505+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002540+0000","last_change":"2023-11-06T21:00:33.995778+0000","last_active":"2023-11-06T21:00:34.002540+0000","last_peered":"2023-11-06T21:00:34.002540+0000","last_clean":"2023-11-06T21:00:34.002540+0000","last_became_active":"2023-11-06T21:00:33.995457+0000","last_became_peered":"2023-11-06T21:00:33.995457+0000","last_unstale":"2023-11-06T21:00:34.002540+0000","last_undegraded":"2023-11-06T21:00:34.002540+0000","last_fullsized":"2023-11-06T21:00:34.002540+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971856+0000","last_change":"2023-11-06T21:00:33.496482+0000","last_active":"2023-11-06T21:00:33.971856+0000","last_peered":"2023-11-06T21:00:33.971856+0000","last_clean":"2023-11-06T21:00:33.971856+0000","last_became_active":"2023-11-06T21:00:33.496181+0000","last_became_peered":"2023-11-06T21:00:33.496181+0000","last_unstale":"2023-11-06T21:00:33.971856+0000","last_undegraded":"2023-11-06T21:00:33.971856+0000","last_fullsized":"2023-11-06T21:00:33.971856+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.005683+0000","last_change":"2023-11-06T21:00:33.999635+0000","last_active":"2023-11-06T21:00:34.005683+0000","last_peered":"2023-11-06T21:00:34.005683+0000","last_clean":"2023-11-06T21:00:34.005683+0000","last_became_active":"2023-11-06T21:00:33.999360+0000","last_became_peered":"2023-11-06T21:00:33.999360+0000","last_unstale":"2023-11-06T21:00:34.005683+0000","last_undegraded":"2023-11-06T21:00:34.005683+0000","last_fullsized":"2023-11-06T21:00:34.005683+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971755+0000","last_change":"2023-11-06T21:00:33.493974+0000","last_active":"2023-11-06T21:00:33.971755+0000","last_peered":"2023-11-06T21:00:33.971755+0000","last_clean":"2023-11-06T21:00:33.971755+0000","last_became_active":"2023-11-06T21:00:33.493635+0000","last_became_peered":"2023-11-06T21:00:33.493635+0000","last_unstale":"2023-11-06T21:00:33.971755+0000","last_undegraded":"2023-11-06T21:00:33.971755+0000","last_fullsized":"2023-11-06T21:00:33.971755+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006063+0000","last_change":"2023-11-06T21:00:34.000967+0000","last_active":"2023-11-06T21:00:34.006063+0000","last_peered":"2023-11-06T21:00:34.006063+0000","last_clean":"2023-11-06T21:00:34.006063+0000","last_became_active":"2023-11-06T21:00:34.000699+0000","last_became_peered":"2023-11-06T21:00:34.000699+0000","last_unstale":"2023-11-06T21:00:34.006063+0000","last_undegraded":"2023-11-06T21:00:34.006063+0000","last_fullsized":"2023-11-06T21:00:34.006063+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.005962+0000","last_change":"2023-11-06T21:00:33.507971+0000","last_active":"2023-11-06T21:00:34.005962+0000","last_peered":"2023-11-06T21:00:34.005962+0000","last_clean":"2023-11-06T21:00:34.005962+0000","last_became_active":"2023-11-06T21:00:33.507726+0000","last_became_peered":"2023-11-06T21:00:33.507726+0000","last_unstale":"2023-11-06T21:00:34.005962+0000","last_undegraded":"2023-11-06T21:00:34.005962+0000","last_fullsized":"2023-11-06T21:00:34.005962+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989166+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T21:00:33.989166+0000","last_peered":"2023-11-06T21:00:33.989166+0000","last_clean":"2023-11-06T21:00:33.989166+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T21:00:33.989166+0000","last_undegraded":"2023-11-06T21:00:33.989166+0000","last_fullsized":"2023-11-06T21:00:33.989166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.997521+0000","last_change":"2023-11-06T21:00:33.992978+0000","last_active":"2023-11-06T21:00:33.997521+0000","last_peered":"2023-11-06T21:00:33.997521+0000","last_clean":"2023-11-06T21:00:33.997521+0000","last_became_active":"2023-11-06T21:00:33.991485+0000","last_became_peered":"2023-11-06T21:00:33.991485+0000","last_unstale":"2023-11-06T21:00:33.997521+0000","last_undegraded":"2023-11-06T21:00:33.997521+0000","last_fullsized":"2023-11-06T21:00:33.997521+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.992549+0000","last_change":"2023-11-06T21:00:33.515504+0000","last_active":"2023-11-06T21:00:33.992549+0000","last_peered":"2023-11-06T21:00:33.992549+0000","last_clean":"2023-11-06T21:00:33.992549+0000","last_became_active":"2023-11-06T21:00:33.510340+0000","last_became_peered":"2023-11-06T21:00:33.510340+0000","last_unstale":"2023-11-06T21:00:33.992549+0000","last_undegraded":"2023-11-06T21:00:33.992549+0000","last_fullsized":"2023-11-06T21:00:33.992549+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973060+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T21:00:33.973060+0000","last_peered":"2023-11-06T21:00:33.973060+0000","last_clean":"2023-11-06T21:00:33.973060+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T21:00:33.973060+0000","last_undegraded":"2023-11-06T21:00:33.973060+0000","last_fullsized":"2023-11-06T21:00:33.973060+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004205+0000","last_change":"2023-11-06T21:00:34.000561+0000","last_active":"2023-11-06T21:00:34.004205+0000","last_peered":"2023-11-06T21:00:34.004205+0000","last_clean":"2023-11-06T21:00:34.004205+0000","last_became_active":"2023-11-06T21:00:34.000132+0000","last_became_peered":"2023-11-06T21:00:34.000132+0000","last_unstale":"2023-11-06T21:00:34.004205+0000","last_undegraded":"2023-11-06T21:00:34.004205+0000","last_fullsized":"2023-11-06T21:00:34.004205+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006842+0000","last_change":"2023-11-06T21:00:33.515388+0000","last_active":"2023-11-06T21:00:34.006842+0000","last_peered":"2023-11-06T21:00:34.006842+0000","last_clean":"2023-11-06T21:00:34.006842+0000","last_became_active":"2023-11-06T21:00:33.510267+0000","last_became_peered":"2023-11-06T21:00:33.510267+0000","last_unstale":"2023-11-06T21:00:34.006842+0000","last_undegraded":"2023-11-06T21:00:34.006842+0000","last_fullsized":"2023-11-06T21:00:34.006842+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":103,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974347+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T21:00:33.974347+0000","last_peered":"2023-11-06T21:00:33.974347+0000","last_clean":"2023-11-06T21:00:33.974347+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T21:00:33.974347+0000","last_undegraded":"2023-11-06T21:00:33.974347+0000","last_fullsized":"2023-11-06T21:00:33.974347+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.001464+0000","last_change":"2023-11-06T21:00:33.996139+0000","last_active":"2023-11-06T21:00:34.001464+0000","last_peered":"2023-11-06T21:00:34.001464+0000","last_clean":"2023-11-06T21:00:34.001464+0000","last_became_active":"2023-11-06T21:00:33.995634+0000","last_became_peered":"2023-11-06T21:00:33.995634+0000","last_unstale":"2023-11-06T21:00:34.001464+0000","last_undegraded":"2023-11-06T21:00:34.001464+0000","last_fullsized":"2023-11-06T21:00:34.001464+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.981812+0000","last_change":"2023-11-06T21:00:33.512541+0000","last_active":"2023-11-06T21:00:33.981812+0000","last_peered":"2023-11-06T21:00:33.981812+0000","last_clean":"2023-11-06T21:00:33.981812+0000","last_became_active":"2023-11-06T21:00:33.512236+0000","last_became_peered":"2023-11-06T21:00:33.512236+0000","last_unstale":"2023-11-06T21:00:33.981812+0000","last_undegraded":"2023-11-06T21:00:33.981812+0000","last_fullsized":"2023-11-06T21:00:33.981812+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.972209+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T21:00:33.972209+0000","last_peered":"2023-11-06T21:00:33.972209+0000","last_clean":"2023-11-06T21:00:33.972209+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T21:00:33.972209+0000","last_undegraded":"2023-11-06T21:00:33.972209+0000","last_fullsized":"2023-11-06T21:00:33.972209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.971840+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T21:00:33.971840+0000","last_peered":"2023-11-06T21:00:33.971840+0000","last_clean":"2023-11-06T21:00:33.971840+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T21:00:33.971840+0000","last_undegraded":"2023-11-06T21:00:33.971840+0000","last_fullsized":"2023-11-06T21:00:33.971840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.002574+0000","last_change":"2023-11-06T21:00:33.998649+0000","last_active":"2023-11-06T21:00:34.002574+0000","last_peered":"2023-11-06T21:00:34.002574+0000","last_clean":"2023-11-06T21:00:34.002574+0000","last_became_active":"2023-11-06T21:00:33.998308+0000","last_became_peered":"2023-11-06T21:00:33.998308+0000","last_unstale":"2023-11-06T21:00:34.002574+0000","last_undegraded":"2023-11-06T21:00:34.002574+0000","last_fullsized":"2023-11-06T21:00:34.002574+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.004440+0000","last_change":"2023-11-06T21:00:33.505117+0000","last_active":"2023-11-06T21:00:34.004440+0000","last_peered":"2023-11-06T21:00:34.004440+0000","last_clean":"2023-11-06T21:00:34.004440+0000","last_became_active":"2023-11-06T21:00:33.504876+0000","last_became_peered":"2023-11-06T21:00:33.504876+0000","last_unstale":"2023-11-06T21:00:34.004440+0000","last_undegraded":"2023-11-06T21:00:34.004440+0000","last_fullsized":"2023-11-06T21:00:34.004440+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973667+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T21:00:33.973667+0000","last_peered":"2023-11-06T21:00:33.973667+0000","last_clean":"2023-11-06T21:00:33.973667+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T21:00:33.973667+0000","last_undegraded":"2023-11-06T21:00:33.973667+0000","last_fullsized":"2023-11-06T21:00:33.973667+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.006593+0000","last_change":"2023-11-06T21:00:33.997726+0000","last_active":"2023-11-06T21:00:34.006593+0000","last_peered":"2023-11-06T21:00:34.006593+0000","last_clean":"2023-11-06T21:00:34.006593+0000","last_became_active":"2023-11-06T21:00:33.997083+0000","last_became_peered":"2023-11-06T21:00:33.997083+0000","last_unstale":"2023-11-06T21:00:34.006593+0000","last_undegraded":"2023-11-06T21:00:34.006593+0000","last_fullsized":"2023-11-06T21:00:34.006593+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.992522+0000","last_change":"2023-11-06T21:00:33.499548+0000","last_active":"2023-11-06T21:00:33.992522+0000","last_peered":"2023-11-06T21:00:33.992522+0000","last_clean":"2023-11-06T21:00:33.992522+0000","last_became_active":"2023-11-06T21:00:33.499278+0000","last_became_peered":"2023-11-06T21:00:33.499278+0000","last_unstale":"2023-11-06T21:00:33.992522+0000","last_undegraded":"2023-11-06T21:00:33.992522+0000","last_fullsized":"2023-11-06T21:00:33.992522+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.974767+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T21:00:33.974767+0000","last_peered":"2023-11-06T21:00:33.974767+0000","last_clean":"2023-11-06T21:00:33.974767+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T21:00:33.974767+0000","last_undegraded":"2023-11-06T21:00:33.974767+0000","last_fullsized":"2023-11-06T21:00:33.974767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.999906+0000","last_change":"2023-11-06T21:00:33.991904+0000","last_active":"2023-11-06T21:00:33.999906+0000","last_peered":"2023-11-06T21:00:33.999906+0000","last_clean":"2023-11-06T21:00:33.999906+0000","last_became_active":"2023-11-06T21:00:33.991641+0000","last_became_peered":"2023-11-06T21:00:33.991641+0000","last_unstale":"2023-11-06T21:00:33.999906+0000","last_undegraded":"2023-11-06T21:00:33.999906+0000","last_fullsized":"2023-11-06T21:00:33.999906+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.989108+0000","last_change":"2023-11-06T21:00:33.493526+0000","last_active":"2023-11-06T21:00:33.989108+0000","last_peered":"2023-11-06T21:00:33.989108+0000","last_clean":"2023-11-06T21:00:33.989108+0000","last_became_active":"2023-11-06T21:00:33.493180+0000","last_became_peered":"2023-11-06T21:00:33.493180+0000","last_unstale":"2023-11-06T21:00:33.989108+0000","last_undegraded":"2023-11-06T21:00:33.989108+0000","last_fullsized":"2023-11-06T21:00:33.989108+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":92,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.991880+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T21:00:33.991880+0000","last_peered":"2023-11-06T21:00:33.991880+0000","last_clean":"2023-11-06T21:00:33.991880+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T21:00:33.991880+0000","last_undegraded":"2023-11-06T21:00:33.991880+0000","last_fullsized":"2023-11-06T21:00:33.991880+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:34.008907+0000","last_change":"2023-11-06T21:00:34.006403+0000","last_active":"2023-11-06T21:00:34.008907+0000","last_peered":"2023-11-06T21:00:34.008907+0000","last_clean":"2023-11-06T21:00:34.008907+0000","last_became_active":"2023-11-06T21:00:34.006121+0000","last_became_peered":"2023-11-06T21:00:34.006121+0000","last_unstale":"2023-11-06T21:00:34.008907+0000","last_undegraded":"2023-11-06T21:00:34.008907+0000","last_fullsized":"2023-11-06T21:00:34.008907+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:33.466673+0000","last_clean_scrub_stamp":"2023-11-06T21:00:33.466673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.990567+0000","last_change":"2023-11-06T21:00:33.501196+0000","last_active":"2023-11-06T21:00:33.990567+0000","last_peered":"2023-11-06T21:00:33.990567+0000","last_clean":"2023-11-06T21:00:33.990567+0000","last_became_active":"2023-11-06T21:00:33.500874+0000","last_became_peered":"2023-11-06T21:00:33.500874+0000","last_unstale":"2023-11-06T21:00:33.990567+0000","last_undegraded":"2023-11-06T21:00:33.990567+0000","last_fullsized":"2023-11-06T21:00:33.990567+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:00:32.459651+0000","last_clean_scrub_stamp":"2023-11-06T21:00:32.459651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-11-06T21:00:33.973257+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T21:00:33.973257+0000","last_peered":"2023-11-06T21:00:33.973257+0000","last_clean":"2023-11-06T21:00:33.973257+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T21:00:33.973257+0000","last_undegraded":"2023-11-06T21:00:33.973257+0000","last_fullsized":"2023-11-06T21:00:33.973257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302568,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069272,"statfs":{"total":96636764160,"available":96326934528,"internally_reserved":0,"allocated":1409024,"data_stored":706267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.68500000000000005,"15min":0.69299999999999995},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.91300000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66900000000000004,"15min":0.69099999999999995},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67000000000000004,"15min":0.70499999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40500000000000003},"max":{"1min":0.84899999999999998,"5min":0.95199999999999996,"15min":5.1749999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69199999999999995,"15min":0.71499999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":1.133},"last":0.59899999999999998}]},{"osd":2,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.70499999999999996,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.83199999999999996,"5min":0.996,"15min":1.165},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":1.024},"last":0.42699999999999999}]},{"osd":3,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89400000000000002,"5min":1.0129999999999999,"15min":5.202},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73099999999999998,"15min":0.76100000000000001},"min":{"1min":0.55100000000000005,"5min":0.51000000000000001,"15min":0.49399999999999999},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":5.165},"last":0.505}]},{"osd":4,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.81399999999999995,"15min":1.083},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.86799999999999999,"15min":0.92800000000000005},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41099999999999998},"max":{"1min":0.85099999999999998,"5min":0.93300000000000005,"15min":1.395},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.129},"last":0.57099999999999995}]},{"osd":6,"last update":"Mon Nov 6 20:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.435},"max":{"1min":0.86599999999999999,"5min":0.94499999999999995,"15min":1.24},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.54500000000000004,"5min":0.47399999999999998,"15min":0.443},"max":{"1min":0.80200000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.61299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1404928,"data_stored":699369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.65700000000000003,"15min":0.68100000000000005},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.32800000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.0640000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.38800000000000001},"max":{"1min":0.86899999999999999,"5min":1.216,"15min":1.216},"last":0.46800000000000003}]},{"osd":1,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.67400000000000004,"15min":0.69499999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80900000000000005,"5min":0.96199999999999997,"15min":0.98399999999999999},"last":0.625},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84099999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.73499999999999999,"15min":0.75600000000000001},"min":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85499999999999998,"5min":1.0049999999999999,"15min":1.044},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73899999999999999,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91300000000000003,"5min":1.05,"15min":1.05},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.47099999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82399999999999995,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71499999999999997,"15min":0.72599999999999998},"min":{"1min":0.53100000000000003,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":0.82699999999999996,"5min":1.002,"15min":1.002},"last":0.56299999999999994}]},{"osd":4,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.627,"15min":0.60899999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":1.012},"last":0.53100000000000003}]},{"osd":5,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88700000000000001,"5min":1.028,"15min":1.028},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69799999999999995,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Nov 6 21:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.45400000000000001,"5min":0.40799999999999997,"15min":0.378},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302556,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069284,"statfs":{"total":96636764160,"available":96326946816,"internally_reserved":0,"allocated":1396736,"data_stored":699189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46100000000000002,"15min":0.45400000000000001},"min":{"1min":0.32500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":0.77900000000000003,"15min":0.85199999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.51100000000000001},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.441}]},{"osd":2,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.434},{"interface":"front","average":{"1min":0.751,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.42899999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":4.431,"5min":4.431,"15min":4.431},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.60899999999999999,"15min":0.59199999999999997},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.5380000000000003,"5min":4.5380000000000003,"15min":4.5380000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77200000000000002,"15min":0.79100000000000004},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":0.80700000000000005}]},{"osd":5,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.79200000000000004,"15min":0.79000000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":4.399,"5min":4.399,"15min":4.399},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.54800000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.69599999999999995}]},{"osd":6,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.81000000000000005,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.501},"max":{"1min":3.024,"5min":3.024,"15min":3.024},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.0609999999999999,"5min":3.0609999999999999,"15min":3.0609999999999999},"last":0.81699999999999995}]},{"osd":7,"last update":"Mon Nov 6 21:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.58199999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.85799999999999998,"15min":0.84299999999999997},"min":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":5.8230000000000004,"5min":5.8230000000000004,"15min":5.8230000000000004},"last":0.86399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575045,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302552,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069288,"statfs":{"total":96636764160,"available":96326950912,"internally_reserved":0,"allocated":1392640,"data_stored":699493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44600000000000001,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":0.73899999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.505,"15min":0.503},"min":{"1min":0.33700000000000002,"5min":0.33200000000000002,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":0.72899999999999998,"15min":0.80700000000000005},"last":0.502}]},{"osd":2,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45100000000000001,"15min":0.46800000000000003},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.82099999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.54800000000000004},"min":{"1min":0.38400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68300000000000005,"5min":0.77500000000000002,"15min":0.91000000000000003},"last":0.57199999999999995}]},{"osd":3,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.54600000000000004,"15min":0.56999999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.747,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55100000000000005}]},{"osd":4,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.77200000000000002,"15min":0.80400000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":0.95099999999999996,"5min":1.1299999999999999,"15min":1.1779999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77700000000000002,"15min":0.80200000000000005},"min":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98899999999999999,"5min":1.298,"15min":1.298},"last":0.88}]},{"osd":5,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77300000000000002,"15min":0.80600000000000005},"min":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.48999999999999999},"max":{"1min":0.96699999999999997,"5min":1.069,"15min":1.2010000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.79800000000000004,"15min":0.81999999999999995},"min":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.51400000000000001},"max":{"1min":0.94799999999999995,"5min":1.2549999999999999,"15min":1.2829999999999999},"last":0.94799999999999995}]},{"osd":6,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76100000000000001,"15min":0.79000000000000004},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.54600000000000004},"max":{"1min":0.91600000000000004,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77700000000000002,"15min":0.81000000000000005},"min":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95999999999999996,"5min":1.056,"15min":1.304},"last":0.95999999999999996}]},{"osd":7,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78700000000000003,"15min":0.81699999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.97699999999999998,"5min":1.171,"15min":1.2490000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80900000000000005,"15min":0.83199999999999996},"min":{"1min":0.63400000000000001,"5min":0.54600000000000004,"15min":0.53200000000000003},"max":{"1min":0.997,"5min":1.1899999999999999,"15min":1.236},"last":0.997}]}]},{"osd":2,"up_from":13,"seq":55834575045,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302556,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069284,"statfs":{"total":96636764160,"available":96326946816,"internally_reserved":0,"allocated":1396736,"data_stored":699177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.44800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65700000000000003,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.433},{"interface":"front","average":{"1min":0.434,"5min":0.47899999999999998,"15min":0.48999999999999999},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58599999999999997,"5min":0.72699999999999998,"15min":0.81699999999999995},"last":0.69499999999999995}]},{"osd":1,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.755,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65300000000000002,"5min":2.266,"15min":2.266},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":1.732,"15min":1.732},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76800000000000002,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.995,"5min":1.0980000000000001,"15min":1.1890000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.74099999999999999,"15min":0.77600000000000002},"min":{"1min":0.46700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0269999999999999,"5min":1.087,"15min":1.357},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.755,"15min":0.77600000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.45900000000000002},"max":{"1min":1.0089999999999999,"5min":1.657,"15min":1.657},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.1919999999999999},"last":0.77100000000000002}]},{"osd":6,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78000000000000003,"15min":0.80400000000000005},"min":{"1min":0.495,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.113,"5min":1.1499999999999999,"15min":1.2969999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.099,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.752}]},{"osd":7,"last update":"Mon Nov 6 21:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79900000000000004,"15min":0.82699999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.46600000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.448},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82199999999999995,"15min":0.82699999999999996},"min":{"1min":0.54100000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.107,"5min":2.202,"15min":2.202},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575046,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1404928,"data_stored":699357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.439,"15min":0.45500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.26800000000000002},"max":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.82599999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.32600000000000001},"max":{"1min":0.81999999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.81999999999999995}]},{"osd":1,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.57399999999999995,"5min":0.64600000000000002,"15min":0.753},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.56100000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.86699999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.55300000000000005},"min":{"1min":0.35599999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.80100000000000005,"15min":0.91900000000000004},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.80100000000000005,"15min":0.81499999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.51200000000000001},"max":{"1min":1.008,"5min":1.234,"15min":1.2470000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.47999999999999998},"max":{"1min":1.0269999999999999,"5min":1.276,"15min":1.286},"last":0.89400000000000002}]},{"osd":5,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.75600000000000001,"15min":0.78600000000000003},"min":{"1min":0.52200000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0209999999999999,"5min":1.1519999999999999,"15min":1.1779999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.79300000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96699999999999997,"5min":1.137,"15min":1.1439999999999999},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.755,"15min":0.78000000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90000000000000002,"5min":1.323,"15min":1.323},"last":0.754},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78100000000000003,"15min":0.81399999999999995},"min":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98099999999999998,"5min":1.054,"15min":1.2130000000000001},"last":0.78900000000000003}]},{"osd":7,"last update":"Mon Nov 6 21:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77600000000000002,"15min":0.81299999999999994},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":1.0920000000000001,"15min":1.302},"last":0.877},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.79900000000000004,"15min":0.82299999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.55200000000000005},"max":{"1min":0.98299999999999998,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302548,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069292,"statfs":{"total":96636764160,"available":96326955008,"internally_reserved":0,"allocated":1388544,"data_stored":698601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.65300000000000002,"15min":0.69199999999999995},"min":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83799999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.64600000000000002,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83499999999999996,"5min":1,"15min":1},"last":0.58199999999999996}]},{"osd":1,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.67100000000000004,"15min":0.70299999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":0.81299999999999994,"5min":1.645,"15min":1.645},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.69499999999999995,"15min":0.71599999999999997},"min":{"1min":0.46100000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.753,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.69299999999999995,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86699999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.60199999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.86199999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.79600000000000004}]},{"osd":3,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66600000000000004,"15min":0.69399999999999995},"min":{"1min":0.48099999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77900000000000003,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67500000000000004,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.436},"max":{"1min":0.85499999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83999999999999997,"5min":0.88200000000000001,"15min":1.0389999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.60799999999999998},"min":{"1min":0.41699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.91000000000000003,"5min":1.22,"15min":1.22},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69199999999999995,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.44900000000000001,"15min":0.44},"max":{"1min":0.84299999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Nov 6 21:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.92700000000000005,"5min":1.034,"15min":1.034},"last":0.747},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.67700000000000005},"min":{"1min":0.47299999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95199999999999996,"5min":1.177,"15min":1.177},"last":0.77900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575046,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302552,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069288,"statfs":{"total":96636764160,"available":96326950912,"internally_reserved":0,"allocated":1392640,"data_stored":698671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.66400000000000003},"min":{"1min":0.40100000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.82499999999999996,"5min":0.98599999999999999,"15min":1.0109999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.92000000000000004,"5min":0.96199999999999997,"15min":0.96799999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70599999999999996,"15min":0.70699999999999996},"min":{"1min":0.54300000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":1.0109999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.747},"min":{"1min":0.48199999999999998,"5min":0.432,"15min":0.40300000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.044},"last":0.45000000000000001}]},{"osd":2,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.503,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.024},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.59799999999999998,"5min":0.46000000000000002,"15min":0.45600000000000002},"max":{"1min":0.96399999999999997,"5min":0.96999999999999997,"15min":1.0329999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45100000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.54600000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.92000000000000004,"5min":0.97599999999999998,"15min":1.02},"last":0.60299999999999998}]},{"osd":4,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.45300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.86399999999999999,"15min":0.93700000000000006},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.68600000000000005},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89800000000000002,"5min":0.97499999999999998,"15min":0.98599999999999999},"last":0.745}]},{"osd":6,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.96399999999999997,"15min":0.97599999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.43099999999999999},"max":{"1min":0.88,"5min":0.96399999999999997,"15min":1.105},"last":0.78900000000000003}]},{"osd":7,"last update":"Mon Nov 6 20:59:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.998,"15min":0.998},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T21:00:38.780 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T21:00:38.780 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T21:00:38.798 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:00:38.822 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:00:39.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.019+0000 7f61ef1ff700 1 Processor -- start 2023-11-06T21:00:39.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.019+0000 7f61ef1ff700 1 -- start start 2023-11-06T21:00:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.020+0000 7f61ef1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8154010 0x7f61e81543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.020+0000 7f61ef1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f61e8064670 0x7f61e8064b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.020+0000 7f61ef1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e8065010 con 0x7f61e80aafc0 2023-11-06T21:00:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.020+0000 7f61ef1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e80c6280 con 0x7f61e8064670 2023-11-06T21:00:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.020+0000 7f61ef1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80c0d90 con 0x7f61e8154010 2023-11-06T21:00:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61e7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f61e80aafc0 0x7f61e80c0680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46816/0 (socket says 172.21.15.33:46816) 2023-11-06T21:00:39.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61e7fff700 1 -- 172.21.15.33:0/1831158025 learned_addr learned my addr 172.21.15.33:0/1831158025 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:39.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61ecf9b700 1 --2- 172.21.15.33:0/1831158025 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8154010 0x7f61e81543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61ed79c700 1 --2- 172.21.15.33:0/1831158025 >> v2:172.21.15.69:3300/0 conn(0x7f61e8064670 0x7f61e8064b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61e77fe700 1 -- 172.21.15.33:0/1831158025 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686543978 0 0) 0x7f61d4001110 con 0x7f61e80aafc0 2023-11-06T21:00:39.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61e77fe700 1 -- 172.21.15.33:0/1831158025 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d80033f0 con 0x7f61e80aafc0 2023-11-06T21:00:39.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61e77fe700 1 -- 172.21.15.33:0/1831158025 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 362992285 0 0) 0x7f61d80033f0 con 0x7f61e80aafc0 2023-11-06T21:00:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61ecf9b700 1 -- 172.21.15.33:0/1831158025 >> v2:172.21.15.69:3300/0 conn(0x7f61e8064670 msgr2=0x7f61e8064b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61ecf9b700 1 --2- 172.21.15.33:0/1831158025 >> v2:172.21.15.69:3300/0 conn(0x7f61e8064670 0x7f61e8064b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61ecf9b700 1 -- 172.21.15.33:0/1831158025 >> v1:172.21.15.33:6789/0 conn(0x7f61e80aafc0 legacy=0x7f61e80c0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.021+0000 7f61ecf9b700 1 -- 172.21.15.33:0/1831158025 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80c7720 con 0x7f61e8154010 2023-11-06T21:00:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.022+0000 7f61ecf9b700 1 --2- 172.21.15.33:0/1831158025 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8154010 0x7f61e81543f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f61dc009760 tx=0x7f61dc009060).ready entity=mon.2 client_cookie=7594e9b330a4293e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.022+0000 7f61e77fe700 1 -- 172.21.15.33:0/1831158025 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4001600 con 0x7f61e8154010 2023-11-06T21:00:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.022+0000 7f61e77fe700 1 -- 172.21.15.33:0/1831158025 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc011400 con 0x7f61e8154010 2023-11-06T21:00:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.022+0000 7f61e77fe700 1 -- 172.21.15.33:0/1831158025 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc011780 con 0x7f61e8154010 2023-11-06T21:00:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.022+0000 7f61ef1ff700 1 -- 172.21.15.33:0/1831158025 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8154010 msgr2=0x7f61e81543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.022+0000 7f61ef1ff700 1 --2- 172.21.15.33:0/1831158025 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8154010 0x7f61e81543f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f61dc009760 tx=0x7f61dc009060).stop 2023-11-06T21:00:39.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.022+0000 7f61ef1ff700 1 -- 172.21.15.33:0/1831158025 shutdown_connections 2023-11-06T21:00:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.022+0000 7f61ef1ff700 1 --2- 172.21.15.33:0/1831158025 >> v2:172.21.15.69:3300/0 conn(0x7f61e8064670 0x7f61e8064b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.022+0000 7f61ef1ff700 1 --2- 172.21.15.33:0/1831158025 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8154010 0x7f61e81543f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.022+0000 7f61ef1ff700 1 -- 172.21.15.33:0/1831158025 >> 172.21.15.33:0/1831158025 conn(0x7f61e80cb8e0 msgr2=0x7f61e80c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.023+0000 7f61ef1ff700 1 -- 172.21.15.33:0/1831158025 shutdown_connections 2023-11-06T21:00:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.023+0000 7f61ef1ff700 1 -- 172.21.15.33:0/1831158025 wait complete. 2023-11-06T21:00:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.024+0000 7f61ef1ff700 1 Processor -- start 2023-11-06T21:00:39.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.024+0000 7f61ef1ff700 1 -- start start 2023-11-06T21:00:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.025+0000 7f61ef1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8064670 0x7f61e814e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.025+0000 7f61ef1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e8154010 0x7f61e80c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.025+0000 7f61ef1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61e8150aa0 0x7f61e814dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.025+0000 7f61ef1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80c4d50 con 0x7f61e8154010 2023-11-06T21:00:39.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.025+0000 7f61ef1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80c44e0 con 0x7f61e8150aa0 2023-11-06T21:00:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.025+0000 7f61ef1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80c4790 con 0x7f61e8064670 2023-11-06T21:00:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e8154010 0x7f61e80c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e8154010 0x7f61e80c3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41064/0 (socket says 172.21.15.33:41064) 2023-11-06T21:00:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e7fff700 1 -- 172.21.15.33:0/3883793224 learned_addr learned my addr 172.21.15.33:0/3883793224 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61ecf9b700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8064670 0x7f61e814e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61ed79c700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61e8150aa0 0x7f61e814dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e7fff700 1 -- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8064670 msgr2=0x7f61e814e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e7fff700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8064670 0x7f61e814e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e7fff700 1 -- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61e8150aa0 msgr2=0x7f61e814dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e7fff700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61e8150aa0 0x7f61e814dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e7fff700 1 -- 172.21.15.33:0/3883793224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc0092a0 con 0x7f61e8154010 2023-11-06T21:00:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e7fff700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e8154010 0x7f61e80c3ff0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f61d400c7c0 tx=0x7f61d4001d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e4ff9700 1 -- 172.21.15.33:0/3883793224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d40072e0 con 0x7f61e8154010 2023-11-06T21:00:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.026+0000 7f61e4ff9700 1 -- 172.21.15.33:0/3883793224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d4007440 con 0x7f61e8154010 2023-11-06T21:00:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.027+0000 7f61ef1ff700 1 -- 172.21.15.33:0/3883793224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e80c4a70 con 0x7f61e8154010 2023-11-06T21:00:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.027+0000 7f61e4ff9700 1 -- 172.21.15.33:0/3883793224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4005780 con 0x7f61e8154010 2023-11-06T21:00:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.027+0000 7f61ef1ff700 1 -- 172.21.15.33:0/3883793224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e8150580 con 0x7f61e8154010 2023-11-06T21:00:39.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.028+0000 7f61ef1ff700 1 -- 172.21.15.33:0/3883793224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e81507a0 con 0x7f61e8154010 2023-11-06T21:00:39.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.029+0000 7f61e4ff9700 1 -- 172.21.15.33:0/3883793224 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f61d4005af0 con 0x7f61e8154010 2023-11-06T21:00:39.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.030+0000 7f61e4ff9700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61cc05f270 0x7f61cc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.031+0000 7f61e4ff9700 1 -- 172.21.15.33:0/3883793224 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f61d4079ab0 con 0x7f61e8154010 2023-11-06T21:00:39.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.031+0000 7f61ecf9b700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61cc05f270 0x7f61cc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.031+0000 7f61ecf9b700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61cc05f270 0x7f61cc061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f61e8153ac0 tx=0x7f61dc016000).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:39.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.034+0000 7f61e4ff9700 1 -- 172.21.15.33:0/3883793224 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f61d404ac00 con 0x7f61e8154010 2023-11-06T21:00:39.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.284+0000 7f61ef1ff700 1 -- 172.21.15.33:0/3883793224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f61e8065010 con 0x7f61e8154010 2023-11-06T21:00:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.289+0000 7f61e4ff9700 1 -- 172.21.15.33:0/3883793224 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f61e8065010 con 0x7f61e8154010 2023-11-06T21:00:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 -- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61cc05f270 msgr2=0x7f61cc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61cc05f270 0x7f61cc061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f61e8153ac0 tx=0x7f61dc016000).stop 2023-11-06T21:00:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 -- 172.21.15.33:0/3883793224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e8154010 msgr2=0x7f61e80c3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e8154010 0x7f61e80c3ff0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f61d400c7c0 tx=0x7f61d4001d80).stop 2023-11-06T21:00:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 -- 172.21.15.33:0/3883793224 shutdown_connections 2023-11-06T21:00:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61cc05f270 0x7f61cc061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61e8064670 0x7f61e814e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61e8150aa0 0x7f61e814dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 --2- 172.21.15.33:0/3883793224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e8154010 0x7f61e80c3ff0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 -- 172.21.15.33:0/3883793224 >> 172.21.15.33:0/3883793224 conn(0x7f61e80cb8e0 msgr2=0x7f61e814f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 -- 172.21.15.33:0/3883793224 shutdown_connections 2023-11-06T21:00:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.291+0000 7f61ef1ff700 1 -- 172.21.15.33:0/3883793224 wait complete. 2023-11-06T21:00:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T21:00:39.309 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:00:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.505+0000 7f640a5ff700 1 Processor -- start 2023-11-06T21:00:39.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.505+0000 7f640a5ff700 1 -- start start 2023-11-06T21:00:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.506+0000 7f640a5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f64040c1840 0x7f64040a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.506+0000 7f640a5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64041554a0 con 0x7f64040aa2d0 2023-11-06T21:00:39.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.506+0000 7f640a5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6404155930 con 0x7f64040c6460 2023-11-06T21:00:39.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.506+0000 7f640a5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6404154960 con 0x7f64040c1840 2023-11-06T21:00:39.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.507+0000 7f64037fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f64040aa2d0 0x7f64040c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46822/0 (socket says 172.21.15.33:46822) 2023-11-06T21:00:39.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.507+0000 7f6403fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f64040c1840 0x7f64040a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.507+0000 7f64037fe700 1 -- 172.21.15.33:0/3855681003 learned_addr learned my addr 172.21.15.33:0/3855681003 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.507+0000 7f6403fff700 1 -- 172.21.15.33:0/3855681003 >> v1:172.21.15.69:6789/0 conn(0x7f64040c6460 legacy=0x7f64040bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.507+0000 7f6403fff700 1 -- 172.21.15.33:0/3855681003 >> v1:172.21.15.33:6789/0 conn(0x7f64040aa2d0 legacy=0x7f64040c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.507+0000 7f6403fff700 1 -- 172.21.15.33:0/3855681003 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6404154b70 con 0x7f64040c1840 2023-11-06T21:00:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.507+0000 7f6402ffd700 1 -- 172.21.15.33:0/3855681003 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278875792 0 0) 0x7f64041554a0 con 0x7f64040aa2d0 2023-11-06T21:00:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.507+0000 7f6403fff700 1 --2- 172.21.15.33:0/3855681003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c1840 0x7f64040a9de0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f63f4009760 tx=0x7f63f4009060).ready entity=mon.0 client_cookie=9dde6e9a74e775a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.508+0000 7f6402ffd700 1 -- 172.21.15.33:0/3855681003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f400b580 con 0x7f64040c1840 2023-11-06T21:00:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.508+0000 7f6402ffd700 1 -- 172.21.15.33:0/3855681003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f400b6e0 con 0x7f64040c1840 2023-11-06T21:00:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.508+0000 7f6402ffd700 1 -- 172.21.15.33:0/3855681003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f400ba60 con 0x7f64040c1840 2023-11-06T21:00:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.508+0000 7f640a5ff700 1 -- 172.21.15.33:0/3855681003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c1840 msgr2=0x7f64040a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.508+0000 7f640a5ff700 1 --2- 172.21.15.33:0/3855681003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c1840 0x7f64040a9de0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f63f4009760 tx=0x7f63f4009060).stop 2023-11-06T21:00:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.508+0000 7f640a5ff700 1 -- 172.21.15.33:0/3855681003 shutdown_connections 2023-11-06T21:00:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.508+0000 7f640a5ff700 1 --2- 172.21.15.33:0/3855681003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c1840 0x7f64040a9de0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.508+0000 7f640a5ff700 1 -- 172.21.15.33:0/3855681003 >> 172.21.15.33:0/3855681003 conn(0x7f640405ec80 msgr2=0x7f64040cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.508+0000 7f640a5ff700 1 -- 172.21.15.33:0/3855681003 shutdown_connections 2023-11-06T21:00:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.508+0000 7f640a5ff700 1 -- 172.21.15.33:0/3855681003 wait complete. 2023-11-06T21:00:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.510+0000 7f640a5ff700 1 Processor -- start 2023-11-06T21:00:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.510+0000 7f640a5ff700 1 -- start start 2023-11-06T21:00:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f640a5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64040aa2d0 0x7f64040aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f640a5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64040c1840 0x7f640414e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f640a5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c6460 0x7f64040c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f640a5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640414ea30 con 0x7f64040c6460 2023-11-06T21:00:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f640a5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64040af670 con 0x7f64040aa2d0 2023-11-06T21:00:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f640a5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64040af950 con 0x7f64040c1840 2023-11-06T21:00:39.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f6408b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c6460 0x7f64040c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f6408b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c6460 0x7f64040c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41096/0 (socket says 172.21.15.33:41096) 2023-11-06T21:00:39.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f6408b9c700 1 -- 172.21.15.33:0/3836728531 learned_addr learned my addr 172.21.15.33:0/3836728531 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:39.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f6403fff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64040aa2d0 0x7f64040aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.511+0000 7f64037fe700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64040c1840 0x7f640414e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f6408b9c700 1 -- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64040c1840 msgr2=0x7f640414e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f6408b9c700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64040c1840 0x7f640414e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f6408b9c700 1 -- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64040aa2d0 msgr2=0x7f64040aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f6408b9c700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64040aa2d0 0x7f64040aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f6408b9c700 1 -- 172.21.15.33:0/3836728531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f40092a0 con 0x7f64040c6460 2023-11-06T21:00:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f6403fff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64040aa2d0 0x7f64040aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f6408b9c700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c6460 0x7f64040c8de0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f63fc00c7c0 tx=0x7f63fc001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f63ebfff700 1 -- 172.21.15.33:0/3836728531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc010510 con 0x7f64040c6460 2023-11-06T21:00:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f63ebfff700 1 -- 172.21.15.33:0/3836728531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc010670 con 0x7f64040c6460 2023-11-06T21:00:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f63ebfff700 1 -- 172.21.15.33:0/3836728531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0109f0 con 0x7f64040c6460 2023-11-06T21:00:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f640a5ff700 1 -- 172.21.15.33:0/3836728531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64040afc30 con 0x7f64040c6460 2023-11-06T21:00:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.512+0000 7f640a5ff700 1 -- 172.21.15.33:0/3836728531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64040b01f0 con 0x7f64040c6460 2023-11-06T21:00:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.513+0000 7f640a5ff700 1 -- 172.21.15.33:0/3836728531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640406c6c0 con 0x7f64040c6460 2023-11-06T21:00:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.514+0000 7f63ebfff700 1 -- 172.21.15.33:0/3836728531 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f63fc01b400 con 0x7f64040c6460 2023-11-06T21:00:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.515+0000 7f63ebfff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f63f005f240 0x7f63f0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.515+0000 7f63ebfff700 1 -- 172.21.15.33:0/3836728531 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f63fc07a220 con 0x7f64040c6460 2023-11-06T21:00:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.515+0000 7f6403fff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f63f005f240 0x7f63f0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.516+0000 7f6403fff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f63f005f240 0x7f63f0061700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f63f400b340 tx=0x7f63f4017000).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:39.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.519+0000 7f63ebfff700 1 -- 172.21.15.33:0/3836728531 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f63fc04b250 con 0x7f64040c6460 2023-11-06T21:00:39.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.748+0000 7f640a5ff700 1 -- 172.21.15.33:0/3836728531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6404063f20 con 0x7f64040c6460 2023-11-06T21:00:39.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.748+0000 7f63ebfff700 1 -- 172.21.15.33:0/3836728531 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4485 (secure 0 0 0) 0x7f63fc04e870 con 0x7f64040c6460 2023-11-06T21:00:39.750 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:39.750 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":45,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2108889615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2108889615},{"type":"v1","addr":"172.21.15.33:6834","nonce":2108889615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/836821298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":836821298},{"type":"v1","addr":"172.21.15.69:6836","nonce":836821298}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6838,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2007995447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2007995447},{"type":"v1","addr":"172.21.15.33:6835","nonce":2007995447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:00:33.962914+0000","modified":"2023-11-06T21:00:36.976166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6844":{"gid":6844,"name":"d","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/396991049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":396991049},{"type":"v1","addr":"172.21.15.69:6837","nonce":396991049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-06T21:00:39.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.750+0000 7f640a5ff700 1 -- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f63f005f240 msgr2=0x7f63f0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.750+0000 7f640a5ff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f63f005f240 0x7f63f0061700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f63f400b340 tx=0x7f63f4017000).stop 2023-11-06T21:00:39.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.750+0000 7f640a5ff700 1 -- 172.21.15.33:0/3836728531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c6460 msgr2=0x7f64040c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:39.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.750+0000 7f640a5ff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c6460 0x7f64040c8de0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f63fc00c7c0 tx=0x7f63fc001510).stop 2023-11-06T21:00:39.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.751+0000 7f640a5ff700 1 -- 172.21.15.33:0/3836728531 shutdown_connections 2023-11-06T21:00:39.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.751+0000 7f640a5ff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f63f005f240 0x7f63f0061700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.751+0000 7f640a5ff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64040c6460 0x7f64040c8de0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.751+0000 7f640a5ff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64040aa2d0 0x7f64040aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.751+0000 7f640a5ff700 1 --2- 172.21.15.33:0/3836728531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f64040c1840 0x7f640414e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:39.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.751+0000 7f640a5ff700 1 -- 172.21.15.33:0/3836728531 >> 172.21.15.33:0/3836728531 conn(0x7f640405ec80 msgr2=0x7f64041599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:39.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.751+0000 7f640a5ff700 1 -- 172.21.15.33:0/3836728531 shutdown_connections 2023-11-06T21:00:39.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:39.751+0000 7f640a5ff700 1 -- 172.21.15.33:0/3836728531 wait complete. 2023-11-06T21:00:39.754 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 45 2023-11-06T21:00:40.171 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T21:00:33.962914+0000', 'modified': '2023-11-06T21:00:36.976166+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6844}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6844': {'gid': 6844, 'name': 'd', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.69:6837/396991049', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6835', 'nonce': 396991049}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 396991049}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T21:00:40.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T21:00:40.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-06T21:00:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.373+0000 7f75d67ff700 1 Processor -- start 2023-11-06T21:00:40.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.373+0000 7f75d67ff700 1 -- start start 2023-11-06T21:00:40.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d67ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d00c6d90 0x7f75d00c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d67ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f75d00bfd60 0x7f75d00c01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d67ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d00c7700 con 0x7f75d0154210 2023-11-06T21:00:40.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d67ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d00c06e0 con 0x7f75d00bfd60 2023-11-06T21:00:40.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d67ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d005efc0 con 0x7f75d00c6d90 2023-11-06T21:00:40.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d4d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f75d00bfd60 0x7f75d00c01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d4d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f75d00bfd60 0x7f75d00c01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41100/0 (socket says 172.21.15.33:41100) 2023-11-06T21:00:40.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d4d9c700 1 -- 172.21.15.33:0/1519844399 learned_addr learned my addr 172.21.15.33:0/1519844399 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:40.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/1519844399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d00c6d90 0x7f75d00c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d4d9c700 1 -- 172.21.15.33:0/1519844399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d00c6d90 msgr2=0x7f75d00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d4d9c700 1 --2- 172.21.15.33:0/1519844399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d00c6d90 0x7f75d00c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d4d9c700 1 -- 172.21.15.33:0/1519844399 >> v1:172.21.15.69:6789/0 conn(0x7f75d0154210 legacy=0x7f75d00a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.375+0000 7f75d4d9c700 1 -- 172.21.15.33:0/1519844399 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d00c8ea0 con 0x7f75d00bfd60 2023-11-06T21:00:40.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.376+0000 7f75d4d9c700 1 --2- 172.21.15.33:0/1519844399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 0x7f75d00c01f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f75c8009f20 tx=0x7f75c8009840).ready entity=mon.0 client_cookie=d5069afe753c7411 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.376+0000 7f75ceffd700 1 -- 172.21.15.33:0/1519844399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c800e070 con 0x7f75d00bfd60 2023-11-06T21:00:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.376+0000 7f75ceffd700 1 -- 172.21.15.33:0/1519844399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c800ba80 con 0x7f75d00bfd60 2023-11-06T21:00:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.376+0000 7f75ceffd700 1 -- 172.21.15.33:0/1519844399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c800be00 con 0x7f75d00bfd60 2023-11-06T21:00:40.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.376+0000 7f75d67ff700 1 -- 172.21.15.33:0/1519844399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 msgr2=0x7f75d00c01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.376+0000 7f75d67ff700 1 --2- 172.21.15.33:0/1519844399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 0x7f75d00c01f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f75c8009f20 tx=0x7f75c8009840).stop 2023-11-06T21:00:40.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.376+0000 7f75d67ff700 1 -- 172.21.15.33:0/1519844399 shutdown_connections 2023-11-06T21:00:40.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.376+0000 7f75d67ff700 1 --2- 172.21.15.33:0/1519844399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 0x7f75d00c01f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.376+0000 7f75d67ff700 1 --2- 172.21.15.33:0/1519844399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d00c6d90 0x7f75d00c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.376+0000 7f75d67ff700 1 -- 172.21.15.33:0/1519844399 >> 172.21.15.33:0/1519844399 conn(0x7f75d005e990 msgr2=0x7f75d00cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:40.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.377+0000 7f75d67ff700 1 -- 172.21.15.33:0/1519844399 shutdown_connections 2023-11-06T21:00:40.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.377+0000 7f75d67ff700 1 -- 172.21.15.33:0/1519844399 wait complete. 2023-11-06T21:00:40.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.378+0000 7f75d67ff700 1 Processor -- start 2023-11-06T21:00:40.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.378+0000 7f75d67ff700 1 -- start start 2023-11-06T21:00:40.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.379+0000 7f75d67ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 0x7f75d014d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.379+0000 7f75d67ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75d00c6d90 0x7f75d0155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.379+0000 7f75d67ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d0154210 0x7f75d00645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.379+0000 7f75d67ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d00c2930 con 0x7f75d00bfd60 2023-11-06T21:00:40.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.379+0000 7f75d67ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d00c51f0 con 0x7f75d00c6d90 2023-11-06T21:00:40.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.379+0000 7f75d67ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d00c2000 con 0x7f75d0154210 2023-11-06T21:00:40.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.379+0000 7f75cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 0x7f75d014d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.379+0000 7f75d4d9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d0154210 0x7f75d00645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 0x7f75d014d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41112/0 (socket says 172.21.15.33:41112) 2023-11-06T21:00:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75d4d9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d0154210 0x7f75d00645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:34560/0 (socket says 172.21.15.33:34560) 2023-11-06T21:00:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75cffff700 1 -- 172.21.15.33:0/604518783 learned_addr learned my addr 172.21.15.33:0/604518783 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75d00c6d90 0x7f75d0155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75cffff700 1 -- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d0154210 msgr2=0x7f75d00645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75cffff700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d0154210 0x7f75d00645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75cffff700 1 -- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75d00c6d90 msgr2=0x7f75d0155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75cffff700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75d00c6d90 0x7f75d0155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75cffff700 1 -- 172.21.15.33:0/604518783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8009ae0 con 0x7f75d00bfd60 2023-11-06T21:00:40.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75cffff700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 0x7f75d014d920 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f75c000c440 tx=0x7f75c0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:40.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75b7fff700 1 -- 172.21.15.33:0/604518783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0007020 con 0x7f75d00bfd60 2023-11-06T21:00:40.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75b7fff700 1 -- 172.21.15.33:0/604518783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c0007180 con 0x7f75d00bfd60 2023-11-06T21:00:40.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75b7fff700 1 -- 172.21.15.33:0/604518783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0007520 con 0x7f75d00bfd60 2023-11-06T21:00:40.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.380+0000 7f75d67ff700 1 -- 172.21.15.33:0/604518783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d00c22e0 con 0x7f75d00bfd60 2023-11-06T21:00:40.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.381+0000 7f75d67ff700 1 -- 172.21.15.33:0/604518783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d014de10 con 0x7f75d00bfd60 2023-11-06T21:00:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.383+0000 7f75b7fff700 1 -- 172.21.15.33:0/604518783 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f75c00068b0 con 0x7f75d00bfd60 2023-11-06T21:00:40.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.384+0000 7f75b7fff700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75bc05f270 0x7f75bc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.384+0000 7f75b7fff700 1 -- 172.21.15.33:0/604518783 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f75c00795b0 con 0x7f75d00bfd60 2023-11-06T21:00:40.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.384+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75bc05f270 0x7f75bc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.385+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75bc05f270 0x7f75bc061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f75d00b05f0 tx=0x7f75c4008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:40.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.386+0000 7f75edb3eb80 1 Processor -- start 2023-11-06T21:00:40.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.386+0000 7f75edb3eb80 1 -- start start 2023-11-06T21:00:40.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.387+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 0x558a05c36460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.387+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c7ebe0 0x558a05c30ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.387+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c5e4c0 0x558a05c313e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.387+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a05b722f0 con 0x558a05c7ebe0 2023-11-06T21:00:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.387+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a05b877e0 con 0x558a05c36080 2023-11-06T21:00:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.387+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a05b87920 con 0x558a05c5e4c0 2023-11-06T21:00:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c7ebe0 0x558a05c30ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 0x558a05c36460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c7ebe0 0x558a05c30ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41116/0 (socket says 172.21.15.33:41116) 2023-11-06T21:00:40.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 0x558a05c36460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:48786/0 (socket says 172.21.15.33:48786) 2023-11-06T21:00:40.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cf7fe700 1 -- 172.21.15.33:0/1060913779 learned_addr learned my addr 172.21.15.33:0/1060913779 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:40.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75d4d9c700 1 --2- 172.21.15.33:0/1060913779 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c5e4c0 0x558a05c313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cffff700 1 -- 172.21.15.33:0/1060913779 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c5e4c0 msgr2=0x558a05c313e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cffff700 1 --2- 172.21.15.33:0/1060913779 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c5e4c0 0x558a05c313e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cffff700 1 -- 172.21.15.33:0/1060913779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c7ebe0 msgr2=0x558a05c30ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cffff700 1 --2- 172.21.15.33:0/1060913779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c7ebe0 0x558a05c30ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cffff700 1 -- 172.21.15.33:0/1060913779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a05b80720 con 0x558a05c36080 2023-11-06T21:00:40.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75cffff700 1 --2- 172.21.15.33:0/1060913779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 0x558a05c36460 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f75c000c410 tx=0x7f75c000f000).ready entity=mon.1 client_cookie=460ab27ab86d955a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:40.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75b57fa700 1 -- 172.21.15.33:0/1060913779 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c004e320 con 0x558a05c36080 2023-11-06T21:00:40.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75b57fa700 1 -- 172.21.15.33:0/1060913779 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c004c440 con 0x558a05c36080 2023-11-06T21:00:40.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.388+0000 7f75b57fa700 1 -- 172.21.15.33:0/1060913779 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c004aa10 con 0x558a05c36080 2023-11-06T21:00:40.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.389+0000 7f75edb3eb80 1 -- 172.21.15.33:0/1060913779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 msgr2=0x558a05c36460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.389+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/1060913779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 0x558a05c36460 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f75c000c410 tx=0x7f75c000f000).stop 2023-11-06T21:00:40.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.389+0000 7f75edb3eb80 1 -- 172.21.15.33:0/1060913779 shutdown_connections 2023-11-06T21:00:40.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.389+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/1060913779 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c5e4c0 0x558a05c313e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.389+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/1060913779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 0x558a05c36460 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.389+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/1060913779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c7ebe0 0x558a05c30ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.389+0000 7f75edb3eb80 1 -- 172.21.15.33:0/1060913779 >> 172.21.15.33:0/1060913779 conn(0x558a05a9f940 msgr2=0x558a05c7f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:40.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.389+0000 7f75edb3eb80 1 -- 172.21.15.33:0/1060913779 shutdown_connections 2023-11-06T21:00:40.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.389+0000 7f75edb3eb80 1 -- 172.21.15.33:0/1060913779 wait complete. 2023-11-06T21:00:40.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.390+0000 7f75edb3eb80 1 -- start start 2023-11-06T21:00:40.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.390+0000 7f75edb3eb80 1 Processor -- start 2023-11-06T21:00:40.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.392+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c36080 0x558a05c788c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.392+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c5e4c0 0x558a05c5eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.392+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 0x558a05c60870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.392+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a05b87ba0 con 0x558a05c36080 2023-11-06T21:00:40.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.392+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a05b52e60 con 0x558a05c5e4c0 2023-11-06T21:00:40.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.392+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a05c7dbe0 con 0x558a05c7ebe0 2023-11-06T21:00:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.392+0000 7f75edb3eb80 10 client.0 ll_register_callbacks cb 0x558a05b27ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T21:00:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.392+0000 7f75cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c36080 0x558a05c788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.392+0000 7f75cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c36080 0x558a05c788c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41120/0 (socket says 172.21.15.33:41120) 2023-11-06T21:00:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.392+0000 7f75cffff700 1 -- 172.21.15.33:0/946962920 learned_addr learned my addr 172.21.15.33:0/946962920 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75d4d9c700 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 0x558a05c60870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c5e4c0 0x558a05c5eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75cffff700 1 -- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 msgr2=0x558a05c60870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75cffff700 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 0x558a05c60870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75cffff700 1 -- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c5e4c0 msgr2=0x558a05c5eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75cffff700 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c5e4c0 0x558a05c5eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75cffff700 1 -- 172.21.15.33:0/946962920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d014de10 con 0x558a05c36080 2023-11-06T21:00:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c5e4c0 0x558a05c5eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75d4d9c700 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 0x558a05c60870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75cffff700 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c36080 0x558a05c788c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f75c000ca30 tx=0x7f75c0053000).ready entity=mon.0 client_cookie=709a8d8423192fe7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75ae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T21:00:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75ae7fc700 1 -- 172.21.15.33:0/946962920 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c005b070 con 0x558a05c36080 2023-11-06T21:00:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75ae7fc700 1 -- 172.21.15.33:0/946962920 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f75c0006df0 con 0x558a05c36080 2023-11-06T21:00:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75edb3eb80 1 -- 172.21.15.33:0/946962920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a05c29e20 con 0x558a05c36080 2023-11-06T21:00:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75ae7fc700 1 -- 172.21.15.33:0/946962920 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75c004c770 con 0x558a05c36080 2023-11-06T21:00:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75edb3eb80 10 client.7206 fetch_fsmap learned FSMap version 45 2023-11-06T21:00:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.393+0000 7f75edb3eb80 1 -- 172.21.15.33:0/946962920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x558a05b7bac0 con 0x558a05c36080 2023-11-06T21:00:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.394+0000 7f75ae7fc700 1 -- 172.21.15.33:0/946962920 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a05c29e20 con 0x558a05c36080 2023-11-06T21:00:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.394+0000 7f75ae7fc700 1 -- 172.21.15.33:0/946962920 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f75c004a0f0 con 0x558a05c36080 2023-11-06T21:00:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.394+0000 7f75edb3eb80 10 client.7206 fetch_fsmap finished waiting for FSMap version 45 2023-11-06T21:00:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.394+0000 7f75edb3eb80 10 client.7206 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T21:00:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.394+0000 7f75edb3eb80 20 client.7206 populate_metadata read hostname 'smithi033' 2023-11-06T21:00:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.394+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:6835/396991049,v1:172.21.15.69:6837/396991049] conn(0x558a05c61250 0x558a05c63420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.394+0000 7f75edb3eb80 4 client.7206 mds_command: new command op to 6844 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T21:00:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.394+0000 7f75edb3eb80 1 -- 172.21.15.33:0/946962920 --> [v2:172.21.15.69:6835/396991049,v1:172.21.15.69:6837/396991049] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a059efef0 con 0x558a05c61250 2023-11-06T21:00:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.394+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:6835/396991049,v1:172.21.15.69:6837/396991049] conn(0x558a05c61250 0x558a05c63420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.395+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:6835/396991049,v1:172.21.15.69:6837/396991049] conn(0x558a05c61250 0x558a05c63420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7aa344c8ff668324 server_cookie=33ebcbe52b7fd7c0 in_seq=0 out_seq=0 2023-11-06T21:00:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.395+0000 7f75ae7fc700 10 client.7206 ms_handle_connect on v2:172.21.15.69:6835/396991049 2023-11-06T21:00:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.395+0000 7f75ae7fc700 1 -- 172.21.15.33:0/946962920 <== mds.0 v2:172.21.15.69:6835/396991049 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x558a059efef0 con 0x558a05c61250 2023-11-06T21:00:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.395+0000 7f75ae7fc700 10 client.7206 handle_command_reply: tid=0 2023-11-06T21:00:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 client.7206 shutdown 2023-11-06T21:00:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 -- 172.21.15.33:0/946962920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c36080 msgr2=0x558a05c788c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c36080 0x558a05c788c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f75c000ca30 tx=0x7f75c0053000).stop 2023-11-06T21:00:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 -- 172.21.15.33:0/946962920 shutdown_connections 2023-11-06T21:00:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:6835/396991049,v1:172.21.15.69:6837/396991049] conn(0x558a05c61250 0x558a05c63420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 0x558a05c60870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c36080 0x558a05c788c0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/946962920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c5e4c0 0x558a05c5eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 -- 172.21.15.33:0/946962920 >> 172.21.15.33:0/946962920 conn(0x558a05a9f940 msgr2=0x558a05c32200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75ae7fc700 0 client.7206 ms_handle_reset on v2:172.21.15.69:6835/396991049 2023-11-06T21:00:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 -- 172.21.15.33:0/946962920 shutdown_connections 2023-11-06T21:00:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 1 -- 172.21.15.33:0/946962920 wait complete. 2023-11-06T21:00:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.396+0000 7f75edb3eb80 20 client.7206 trim_cache size 0 max 16384 2023-11-06T21:00:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.414+0000 7f75edb3eb80 1 Processor -- start 2023-11-06T21:00:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.414+0000 7f75edb3eb80 1 -- start start 2023-11-06T21:00:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c36080 0x558a05c352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05c36990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c7ebe0 0x558a05c7b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a05c2e130 con 0x558a05c5e4c0 2023-11-06T21:00:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a05c33bc0 con 0x558a05c7ebe0 2023-11-06T21:00:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a05c604e0 con 0x558a05c36080 2023-11-06T21:00:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05c36990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05c36990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41132/0 (socket says 172.21.15.33:41132) 2023-11-06T21:00:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75cf7fe700 1 -- 172.21.15.33:0/967399054 learned_addr learned my addr 172.21.15.33:0/967399054 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75cffff700 1 --2- 172.21.15.33:0/967399054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c36080 0x558a05c352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.415+0000 7f75d4d9c700 1 --2- 172.21.15.33:0/967399054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c7ebe0 0x558a05c7b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.416+0000 7f75cf7fe700 1 -- 172.21.15.33:0/967399054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c36080 msgr2=0x558a05c352e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.416+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/967399054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c36080 0x558a05c352e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.416+0000 7f75cf7fe700 1 -- 172.21.15.33:0/967399054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c7ebe0 msgr2=0x558a05c7b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.416+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/967399054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c7ebe0 0x558a05c7b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.416+0000 7f75cf7fe700 1 -- 172.21.15.33:0/967399054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a05b7b810 con 0x558a05c5e4c0 2023-11-06T21:00:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.416+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/967399054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05c36990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x558a05c5f460 tx=0x7f75c4015000).ready entity=mon.0 client_cookie=4dadce77e4e66cf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.416+0000 7f75b5ffb700 1 -- 172.21.15.33:0/967399054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4010e60 con 0x558a05c5e4c0 2023-11-06T21:00:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.416+0000 7f75b5ffb700 1 -- 172.21.15.33:0/967399054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c4026a60 con 0x558a05c5e4c0 2023-11-06T21:00:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.416+0000 7f75b5ffb700 1 -- 172.21.15.33:0/967399054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4026de0 con 0x558a05c5e4c0 2023-11-06T21:00:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.417+0000 7f75edb3eb80 1 -- 172.21.15.33:0/967399054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 msgr2=0x558a05c36990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.417+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/967399054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05c36990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x558a05c5f460 tx=0x7f75c4015000).stop 2023-11-06T21:00:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.417+0000 7f75edb3eb80 1 -- 172.21.15.33:0/967399054 shutdown_connections 2023-11-06T21:00:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.417+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/967399054 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c36080 0x558a05c352e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.417+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/967399054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c7ebe0 0x558a05c7b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.417+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/967399054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05c36990 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.417+0000 7f75edb3eb80 1 -- 172.21.15.33:0/967399054 >> 172.21.15.33:0/967399054 conn(0x558a05a9f940 msgr2=0x558a05c5f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.417+0000 7f75edb3eb80 1 -- 172.21.15.33:0/967399054 shutdown_connections 2023-11-06T21:00:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.417+0000 7f75edb3eb80 1 -- 172.21.15.33:0/967399054 wait complete. 2023-11-06T21:00:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.417+0000 7f75edb3eb80 1 -- start start 2023-11-06T21:00:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.418+0000 7f75edb3eb80 1 Processor -- start 2023-11-06T21:00:40.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 0x558a05cdff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05ce0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75edb3eb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 0x558a05c34cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a05c8bec0 con 0x558a05c5e4c0 2023-11-06T21:00:40.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a05c8d1b0 con 0x558a05c36080 2023-11-06T21:00:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75edb3eb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a05c9c0a0 con 0x558a05c7ebe0 2023-11-06T21:00:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75edb3eb80 10 client.0 ll_register_callbacks cb 0x558a05b27ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T21:00:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05ce0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05ce0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41146/0 (socket says 172.21.15.33:41146) 2023-11-06T21:00:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75cf7fe700 1 -- 172.21.15.33:0/3435121494 learned_addr learned my addr 172.21.15.33:0/3435121494 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75cffff700 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 0x558a05cdff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75d4d9c700 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 0x558a05c34cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75cf7fe700 1 -- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 msgr2=0x558a05c34cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 0x558a05c34cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75cf7fe700 1 -- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 msgr2=0x558a05cdff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 0x558a05cdff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.419+0000 7f75cf7fe700 1 -- 172.21.15.33:0/3435121494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c400e040 con 0x558a05c5e4c0 2023-11-06T21:00:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75cf7fe700 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05ce0450 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x558a05c60140 tx=0x7f75c4017000).ready entity=mon.0 client_cookie=317b616470814a9b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75ae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T21:00:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75ae7fc700 1 -- 172.21.15.33:0/3435121494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4026de0 con 0x558a05c5e4c0 2023-11-06T21:00:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75ae7fc700 1 -- 172.21.15.33:0/3435121494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f75c4014680 con 0x558a05c5e4c0 2023-11-06T21:00:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75edb3eb80 1 -- 172.21.15.33:0/3435121494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75c004c440 con 0x558a05c5e4c0 2023-11-06T21:00:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75ae7fc700 1 -- 172.21.15.33:0/3435121494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75c4014a50 con 0x558a05c5e4c0 2023-11-06T21:00:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75edb3eb80 10 client.7212 fetch_fsmap learned FSMap version 45 2023-11-06T21:00:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75edb3eb80 1 -- 172.21.15.33:0/3435121494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x558a059f51d0 con 0x558a05c5e4c0 2023-11-06T21:00:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75ae7fc700 1 -- 172.21.15.33:0/3435121494 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c004c440 con 0x558a05c5e4c0 2023-11-06T21:00:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75ae7fc700 1 -- 172.21.15.33:0/3435121494 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f75c4014bb0 con 0x558a05c5e4c0 2023-11-06T21:00:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75edb3eb80 10 client.7212 fetch_fsmap finished waiting for FSMap version 45 2023-11-06T21:00:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75edb3eb80 10 client.7212 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T21:00:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75edb3eb80 20 client.7212 populate_metadata read hostname 'smithi033' 2023-11-06T21:00:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:6835/396991049,v1:172.21.15.69:6837/396991049] conn(0x558a05c9a8c0 0x558a05c9aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75edb3eb80 4 client.7212 mds_command: new command op to 6844 tid=0[{"prefix": "status"}] 2023-11-06T21:00:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.420+0000 7f75edb3eb80 1 -- 172.21.15.33:0/3435121494 --> [v2:172.21.15.69:6835/396991049,v1:172.21.15.69:6837/396991049] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558a05a25040 con 0x558a05c9a8c0 2023-11-06T21:00:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.421+0000 7f75cffff700 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:6835/396991049,v1:172.21.15.69:6837/396991049] conn(0x558a05c9a8c0 0x558a05c9aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.421+0000 7f75cffff700 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:6835/396991049,v1:172.21.15.69:6837/396991049] conn(0x558a05c9a8c0 0x558a05c9aca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b309eed1d617438b server_cookie=9883b8187d4a2891 in_seq=0 out_seq=0 2023-11-06T21:00:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.421+0000 7f75ae7fc700 10 client.7212 ms_handle_connect on v2:172.21.15.69:6835/396991049 2023-11-06T21:00:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75ae7fc700 1 -- 172.21.15.33:0/3435121494 <== mds.0 v2:172.21.15.69:6835/396991049 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x558a05a25040 con 0x558a05c9a8c0 2023-11-06T21:00:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75ae7fc700 10 client.7212 handle_command_reply: tid=0 2023-11-06T21:00:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 client.7212 shutdown 2023-11-06T21:00:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 -- 172.21.15.33:0/3435121494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 msgr2=0x558a05ce0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05ce0450 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x558a05c60140 tx=0x7f75c4017000).stop 2023-11-06T21:00:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 -- 172.21.15.33:0/3435121494 shutdown_connections 2023-11-06T21:00:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:6835/396991049,v1:172.21.15.69:6837/396991049] conn(0x558a05c9a8c0 0x558a05c9aca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x558a05c7ebe0 0x558a05c34cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a05c36080 0x558a05cdff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75ae7fc700 0 client.7212 ms_handle_reset on v2:172.21.15.69:6835/396991049 2023-11-06T21:00:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 --2- 172.21.15.33:0/3435121494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a05c5e4c0 0x558a05ce0450 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 -- 172.21.15.33:0/3435121494 >> 172.21.15.33:0/3435121494 conn(0x558a05a9f940 msgr2=0x558a05c61ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 -- 172.21.15.33:0/3435121494 shutdown_connections 2023-11-06T21:00:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 1 -- 172.21.15.33:0/3435121494 wait complete. 2023-11-06T21:00:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.422+0000 7f75edb3eb80 20 client.7212 trim_cache size 0 max 16384 2023-11-06T21:00:40.438 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T21:00:40.438 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T21:00:40.438 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T21:00:40.438 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 6844, 2023-11-06T21:00:40.439 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T21:00:40.439 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T21:00:40.439 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T21:00:40.439 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.4567803479999997, 2023-11-06T21:00:40.439 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 45, 2023-11-06T21:00:40.440 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 114, 2023-11-06T21:00:40.440 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 114, 2023-11-06T21:00:40.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.656680399000001 2023-11-06T21:00:40.440 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T21:00:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 -- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75bc05f270 msgr2=0x7f75bc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75bc05f270 0x7f75bc061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f75d00b05f0 tx=0x7f75c4008040).stop 2023-11-06T21:00:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 -- 172.21.15.33:0/604518783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 msgr2=0x7f75d014d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 0x7f75d014d920 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f75c000c440 tx=0x7f75c0007e00).stop 2023-11-06T21:00:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 -- 172.21.15.33:0/604518783 shutdown_connections 2023-11-06T21:00:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f75bc05f270 0x7f75bc061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75d0154210 0x7f75d00645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d00bfd60 0x7f75d014d920 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 --2- 172.21.15.33:0/604518783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75d00c6d90 0x7f75d0155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 -- 172.21.15.33:0/604518783 >> 172.21.15.33:0/604518783 conn(0x7f75d005e990 msgr2=0x7f75d00cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:40.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 -- 172.21.15.33:0/604518783 shutdown_connections 2023-11-06T21:00:40.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.423+0000 7f75d67ff700 1 -- 172.21.15.33:0/604518783 wait complete. 2023-11-06T21:00:40.444 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T21:00:40.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.609+0000 7f78aafff700 1 Processor -- start 2023-11-06T21:00:40.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.610+0000 7f78aafff700 1 -- start start 2023-11-06T21:00:40.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.611+0000 7f78aafff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f78a40a9ee0 0x7f78a40c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.611+0000 7f78aafff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a40c5f80 con 0x7f78a40c07b0 2023-11-06T21:00:40.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.611+0000 7f78aafff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a40aa740 con 0x7f78a41541a0 2023-11-06T21:00:40.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.611+0000 7f78aafff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a40c6800 con 0x7f78a40a9ee0 2023-11-06T21:00:40.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.611+0000 7f78a959c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f78a40c07b0 0x7f78a40a7c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46834/0 (socket says 172.21.15.33:46834) 2023-11-06T21:00:40.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.611+0000 7f78a959c700 1 -- 172.21.15.33:0/1726793608 learned_addr learned my addr 172.21.15.33:0/1726793608 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:40.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a3fff700 1 --2- 172.21.15.33:0/1726793608 >> v2:172.21.15.69:3300/0 conn(0x7f78a40a9ee0 0x7f78a40c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918956932 0 0) 0x7f78a40c5f80 con 0x7f78a40c07b0 2023-11-06T21:00:40.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78940033f0 con 0x7f78a40c07b0 2023-11-06T21:00:40.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3242270863 0 0) 0x7f78940033f0 con 0x7f78a40c07b0 2023-11-06T21:00:40.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 >> v2:172.21.15.69:3300/0 conn(0x7f78a40a9ee0 msgr2=0x7f78a40c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 --2- 172.21.15.33:0/1726793608 >> v2:172.21.15.69:3300/0 conn(0x7f78a40a9ee0 0x7f78a40c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 >> v1:172.21.15.69:6789/0 conn(0x7f78a41541a0 legacy=0x7f78a40c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a40aa460 con 0x7f78a40c07b0 2023-11-06T21:00:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f789c0017f0 con 0x7f78a40c07b0 2023-11-06T21:00:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 >> v1:172.21.15.33:6789/0 conn(0x7f78a40c07b0 legacy=0x7f78a40a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78940038e0 0x7f7894003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7894004690 0x7f7894004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7894005000 0x7f789400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40c6800 con 0x7f78940038e0 2023-11-06T21:00:40.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789400bf60 con 0x7f7894004690 2023-11-06T21:00:40.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789400c0f0 con 0x7f7894005000 2023-11-06T21:00:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.612+0000 7f78a8d9b700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78940038e0 0x7f7894003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a3fff700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7894004690 0x7f7894004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a959c700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7894005000 0x7f789400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a8d9b700 1 -- 172.21.15.33:0/1726793608 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7894005000 msgr2=0x7f789400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a8d9b700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7894005000 0x7f789400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a8d9b700 1 -- 172.21.15.33:0/1726793608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7894004690 msgr2=0x7f7894004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:40.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a8d9b700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7894004690 0x7f7894004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a8d9b700 1 -- 172.21.15.33:0/1726793608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940033f0 con 0x7f78940038e0 2023-11-06T21:00:40.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a959c700 1 -- 172.21.15.33:0/1726793608 reap_dead start 2023-11-06T21:00:40.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a8d9b700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78940038e0 0x7f7894003d00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f7898001780 tx=0x7f7898007c00).ready entity=mon.0 client_cookie=9864051d3b89fae3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:40.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898001ea0 con 0x7f78940038e0 2023-11-06T21:00:40.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7898007020 con 0x7f78940038e0 2023-11-06T21:00:40.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.613+0000 7f78a37fe700 1 -- 172.21.15.33:0/1726793608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980073c0 con 0x7f78940038e0 2023-11-06T21:00:40.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.614+0000 7f78aafff700 1 -- 172.21.15.33:0/1726793608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78940038e0 msgr2=0x7f7894003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.614+0000 7f78aafff700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78940038e0 0x7f7894003d00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f7898001780 tx=0x7f7898007c00).stop 2023-11-06T21:00:40.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.614+0000 7f78aafff700 1 -- 172.21.15.33:0/1726793608 shutdown_connections 2023-11-06T21:00:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.614+0000 7f78aafff700 1 --2- 172.21.15.33:0/1726793608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78940038e0 0x7f7894003d00 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.614+0000 7f78aafff700 1 -- 172.21.15.33:0/1726793608 >> 172.21.15.33:0/1726793608 conn(0x7f78a40944e0 msgr2=0x7f78a406a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.614+0000 7f78aafff700 1 -- 172.21.15.33:0/1726793608 shutdown_connections 2023-11-06T21:00:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.614+0000 7f78aafff700 1 -- 172.21.15.33:0/1726793608 wait complete. 2023-11-06T21:00:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.615+0000 7f78aafff700 1 Processor -- start 2023-11-06T21:00:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.616+0000 7f78aafff700 1 -- start start 2023-11-06T21:00:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.616+0000 7f78aafff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78940038e0 0x7f78a40c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.616+0000 7f78aafff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78a40cb990 0x7f78a40c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.616+0000 7f78aafff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a40ac790 0x7f78a40acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.616+0000 7f78aafff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40c3a70 con 0x7f78a40ac790 2023-11-06T21:00:40.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78aafff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a414f310 con 0x7f78940038e0 2023-11-06T21:00:40.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78aafff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40cbe10 con 0x7f78a40cb990 2023-11-06T21:00:40.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a959c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a40ac790 0x7f78a40acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a959c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a40ac790 0x7f78a40acc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41160/0 (socket says 172.21.15.33:41160) 2023-11-06T21:00:40.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a959c700 1 -- 172.21.15.33:0/55734215 learned_addr learned my addr 172.21.15.33:0/55734215 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:40.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a8d9b700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78940038e0 0x7f78a40c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a3fff700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78a40cb990 0x7f78a40c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a959c700 1 -- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78a40cb990 msgr2=0x7f78a40c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a959c700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78a40cb990 0x7f78a40c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a959c700 1 -- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78940038e0 msgr2=0x7f78a40c28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a959c700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78940038e0 0x7f78a40c28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a959c700 1 -- 172.21.15.33:0/55734215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a40ad100 con 0x7f78a40ac790 2023-11-06T21:00:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a8d9b700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78940038e0 0x7f78a40c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a3fff700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78a40cb990 0x7f78a40c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.617+0000 7f78a959c700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a40ac790 0x7f78a40acc10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f789c000f80 tx=0x7f789c007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.618+0000 7f78a0ff9700 1 -- 172.21.15.33:0/55734215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c00f970 con 0x7f78a40ac790 2023-11-06T21:00:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.618+0000 7f78a0ff9700 1 -- 172.21.15.33:0/55734215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c00fad0 con 0x7f78a40ac790 2023-11-06T21:00:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.618+0000 7f78a0ff9700 1 -- 172.21.15.33:0/55734215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c00fe50 con 0x7f78a40ac790 2023-11-06T21:00:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.618+0000 7f78aafff700 1 -- 172.21.15.33:0/55734215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a40c9850 con 0x7f78a40ac790 2023-11-06T21:00:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.618+0000 7f78aafff700 1 -- 172.21.15.33:0/55734215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a40c2ef0 con 0x7f78a40ac790 2023-11-06T21:00:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.619+0000 7f78aafff700 1 -- 172.21.15.33:0/55734215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a40475a0 con 0x7f78a40ac790 2023-11-06T21:00:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.620+0000 7f78a0ff9700 1 -- 172.21.15.33:0/55734215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f789c015620 con 0x7f78a40ac790 2023-11-06T21:00:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.621+0000 7f78a0ff9700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f788805f270 0x7f7888061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.621+0000 7f78a0ff9700 1 -- 172.21.15.33:0/55734215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f789c079b30 con 0x7f78a40ac790 2023-11-06T21:00:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.622+0000 7f78a8d9b700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f788805f270 0x7f7888061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.622+0000 7f78a8d9b700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f788805f270 0x7f7888061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7898001780 tx=0x7f7898015000).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.625+0000 7f78a0ff9700 1 -- 172.21.15.33:0/55734215 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f789c04ac20 con 0x7f78a40ac790 2023-11-06T21:00:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.848+0000 7f78aafff700 1 -- 172.21.15.33:0/55734215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f78a40c99e0 con 0x7f78a40ac790 2023-11-06T21:00:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.848+0000 7f78a0ff9700 1 -- 172.21.15.33:0/55734215 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) v1 ==== 53+0+73 (secure 0 0 0) 0x7f789c01d070 con 0x7f78a40ac790 2023-11-06T21:00:40.849 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T21:00:40.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 -- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f788805f270 msgr2=0x7f7888061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f788805f270 0x7f7888061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7898001780 tx=0x7f7898015000).stop 2023-11-06T21:00:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 -- 172.21.15.33:0/55734215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a40ac790 msgr2=0x7f78a40acc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a40ac790 0x7f78a40acc10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f789c000f80 tx=0x7f789c007d80).stop 2023-11-06T21:00:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 -- 172.21.15.33:0/55734215 shutdown_connections 2023-11-06T21:00:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f788805f270 0x7f7888061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a40ac790 0x7f78a40acc10 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78940038e0 0x7f78a40c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 --2- 172.21.15.33:0/55734215 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78a40cb990 0x7f78a40c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 -- 172.21.15.33:0/55734215 >> 172.21.15.33:0/55734215 conn(0x7f78a40944e0 msgr2=0x7f78a40644b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 -- 172.21.15.33:0/55734215 shutdown_connections 2023-11-06T21:00:40.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:40.850+0000 7f78aafff700 1 -- 172.21.15.33:0/55734215 wait complete. 2023-11-06T21:00:40.865 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T21:00:40.866 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T21:00:40.866 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T21:00:40.866 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T21:00:40.867 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T21:00:40.867 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T21:00:40.867 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T21:00:40.868 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T21:00:40.868 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T21:00:40.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:40.869 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T21:00:40.917 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T21:00:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T21:00:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T21:00:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T21:00:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:40.919 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:00:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:00:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T21:00:40.919 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:00:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:00:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T21:00:40.920 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T21:00:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:00:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T21:00:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T21:00:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41303sec preferred_lft 41303sec 2023-11-06T21:00:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T21:00:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:40.921 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:00:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:00:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T21:00:40.922 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T21:00:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T21:00:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T21:00:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T21:00:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:40.923 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T21:00:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T21:00:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T21:00:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:40.924 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T21:00:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T21:00:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T21:00:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:00:40.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:40.925 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T21:00:40.994 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T21:00:40.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:40.997 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T21:00:41.048 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T21:00:41.048 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T21:00:41.048 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T21:00:41.049 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T21:00:41.049 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T21:00:41.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:41.051 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T21:00:41.051 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T21:00:41.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T21:00:41.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T21:00:41.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T21:00:41.052 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T21:00:41.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:41.228 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T21:00:41.276 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T21:00:41.276 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T21:00:41.277 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T21:00:41.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:41.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T21:00:41.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:41.352 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T21:00:41.399 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T21:00:41.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:00:41.502 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:00:41.502 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T21:00:41.530 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:00:41.530 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T21:00:41.536 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T21:00:41.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:00:41.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T21:00:41.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T21:00:41.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.807+0000 7f29be7ff700 1 Processor -- start 2023-11-06T21:00:41.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.807+0000 7f29be7ff700 1 -- start start 2023-11-06T21:00:41.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29be7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f29b8154440 0x7f29b80bd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:41.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29be7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f29b80bcea0 0x7f29b8064a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:41.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29be7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b80686d0 con 0x7f29b8064620 2023-11-06T21:00:41.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29be7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b80bd310 con 0x7f29b80bcea0 2023-11-06T21:00:41.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29be7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b80a48e0 con 0x7f29b8154440 2023-11-06T21:00:41.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29bcd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f29b80bcea0 0x7f29b8064a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:41.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29bcd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f29b80bcea0 0x7f29b8064a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41162/0 (socket says 172.21.15.33:41162) 2023-11-06T21:00:41.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29bcd9c700 1 -- 172.21.15.33:0/347771552 learned_addr learned my addr 172.21.15.33:0/347771552 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:41.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29b7fff700 1 --2- 172.21.15.33:0/347771552 >> v2:172.21.15.69:3300/0 conn(0x7f29b8154440 0x7f29b80bd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:41.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29bcd9c700 1 -- 172.21.15.33:0/347771552 >> v2:172.21.15.69:3300/0 conn(0x7f29b8154440 msgr2=0x7f29b80bd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:41.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29bcd9c700 1 --2- 172.21.15.33:0/347771552 >> v2:172.21.15.69:3300/0 conn(0x7f29b8154440 0x7f29b80bd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:41.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29bcd9c700 1 -- 172.21.15.33:0/347771552 >> v1:172.21.15.69:6789/0 conn(0x7f29b8064620 legacy=0x7f29b80a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:41.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.809+0000 7f29bcd9c700 1 -- 172.21.15.33:0/347771552 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8064f70 con 0x7f29b80bcea0 2023-11-06T21:00:41.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29bcd9c700 1 --2- 172.21.15.33:0/347771552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 0x7f29b8064a80 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f29b80636f0 tx=0x7f29b0009840).ready entity=mon.0 client_cookie=ed82b3b589c198ba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:41.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29b6ffd700 1 -- 172.21.15.33:0/347771552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000e070 con 0x7f29b80bcea0 2023-11-06T21:00:41.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29b6ffd700 1 -- 172.21.15.33:0/347771552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b000ba10 con 0x7f29b80bcea0 2023-11-06T21:00:41.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29b6ffd700 1 -- 172.21.15.33:0/347771552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000bd90 con 0x7f29b80bcea0 2023-11-06T21:00:41.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29be7ff700 1 -- 172.21.15.33:0/347771552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 msgr2=0x7f29b8064a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:41.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29be7ff700 1 --2- 172.21.15.33:0/347771552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 0x7f29b8064a80 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f29b80636f0 tx=0x7f29b0009840).stop 2023-11-06T21:00:41.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29be7ff700 1 -- 172.21.15.33:0/347771552 shutdown_connections 2023-11-06T21:00:41.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29be7ff700 1 --2- 172.21.15.33:0/347771552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 0x7f29b8064a80 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:41.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29be7ff700 1 --2- 172.21.15.33:0/347771552 >> v2:172.21.15.69:3300/0 conn(0x7f29b8154440 0x7f29b80bd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:41.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29be7ff700 1 -- 172.21.15.33:0/347771552 >> 172.21.15.33:0/347771552 conn(0x7f29b80631a0 msgr2=0x7f29b80c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:41.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.810+0000 7f29be7ff700 1 -- 172.21.15.33:0/347771552 shutdown_connections 2023-11-06T21:00:41.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.811+0000 7f29be7ff700 1 -- 172.21.15.33:0/347771552 wait complete. 2023-11-06T21:00:41.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.812+0000 7f29be7ff700 1 Processor -- start 2023-11-06T21:00:41.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.812+0000 7f29be7ff700 1 -- start start 2023-11-06T21:00:41.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29be7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29b8064620 0x7f29b814eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:41.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29be7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 0x7f29b80a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:41.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29be7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8154440 0x7f29b80c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:41.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29be7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80aae00 con 0x7f29b80bcea0 2023-11-06T21:00:41.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29be7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80a7650 con 0x7f29b8154440 2023-11-06T21:00:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29be7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80aab20 con 0x7f29b8064620 2023-11-06T21:00:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 0x7f29b80a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 0x7f29b80a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41164/0 (socket says 172.21.15.33:41164) 2023-11-06T21:00:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29b77fe700 1 -- 172.21.15.33:0/92714464 learned_addr learned my addr 172.21.15.33:0/92714464 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29bcd9c700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8154440 0x7f29b80c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:41.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.813+0000 7f29b7fff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29b8064620 0x7f29b814eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:41.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f29b77fe700 1 -- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29b8064620 msgr2=0x7f29b814eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:41.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f29b77fe700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29b8064620 0x7f29b814eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:41.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f29b77fe700 1 -- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8154440 msgr2=0x7f29b80c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:41.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f29b77fe700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8154440 0x7f29b80c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:41.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f29b77fe700 1 -- 172.21.15.33:0/92714464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0009ae0 con 0x7f29b80bcea0 2023-11-06T21:00:41.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f29b7fff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29b8064620 0x7f29b814eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:00:41.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f29b77fe700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 0x7f29b80a7ae0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f29ac00b460 tx=0x7f29ac006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:41.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f299ffff700 1 -- 172.21.15.33:0/92714464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00be60 con 0x7f29b80bcea0 2023-11-06T21:00:41.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f299ffff700 1 -- 172.21.15.33:0/92714464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac012410 con 0x7f29b80bcea0 2023-11-06T21:00:41.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f299ffff700 1 -- 172.21.15.33:0/92714464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0127b0 con 0x7f29b80bcea0 2023-11-06T21:00:41.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f29be7ff700 1 -- 172.21.15.33:0/92714464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b80a7000 con 0x7f29b80bcea0 2023-11-06T21:00:41.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.814+0000 7f29be7ff700 1 -- 172.21.15.33:0/92714464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b814eff0 con 0x7f29b80bcea0 2023-11-06T21:00:41.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.816+0000 7f29be7ff700 1 -- 172.21.15.33:0/92714464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b80a6e60 con 0x7f29b80bcea0 2023-11-06T21:00:41.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.816+0000 7f299ffff700 1 -- 172.21.15.33:0/92714464 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f29ac012b60 con 0x7f29b80bcea0 2023-11-06T21:00:41.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.817+0000 7f299ffff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a405f270 0x7f29a4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:41.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.817+0000 7f299ffff700 1 -- 172.21.15.33:0/92714464 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f29ac07a110 con 0x7f29b80bcea0 2023-11-06T21:00:41.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.818+0000 7f29b7fff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a405f270 0x7f29a4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:41.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.818+0000 7f29b7fff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a405f270 0x7f29a4061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f29a800bb60 tx=0x7f29a8008040).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:41.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:41.821+0000 7f299ffff700 1 -- 172.21.15.33:0/92714464 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f29b80a6e60 con 0x7f29b80bcea0 2023-11-06T21:00:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.033+0000 7f29be7ff700 1 -- 172.21.15.33:0/92714464 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f29b80a6e60 con 0x7f29a405f270 2023-11-06T21:00:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.035+0000 7f299ffff700 1 -- 172.21.15.33:0/92714464 <== mgr.6050 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f29b80a6e60 con 0x7f29a405f270 2023-11-06T21:00:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T21:00:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:00:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T21:00:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:00:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T21:00:42.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.037+0000 7f29be7ff700 1 -- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a405f270 msgr2=0x7f29a4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:42.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.037+0000 7f29be7ff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a405f270 0x7f29a4061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f29a800bb60 tx=0x7f29a8008040).stop 2023-11-06T21:00:42.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.037+0000 7f29be7ff700 1 -- 172.21.15.33:0/92714464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 msgr2=0x7f29b80a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:42.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.037+0000 7f29be7ff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 0x7f29b80a7ae0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f29ac00b460 tx=0x7f29ac006f20).stop 2023-11-06T21:00:42.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.038+0000 7f29be7ff700 1 -- 172.21.15.33:0/92714464 shutdown_connections 2023-11-06T21:00:42.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.038+0000 7f29be7ff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29a405f270 0x7f29a4061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:42.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.038+0000 7f29be7ff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29b8064620 0x7f29b814eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:42.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.038+0000 7f29be7ff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29b8154440 0x7f29b80c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:42.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.038+0000 7f29be7ff700 1 --2- 172.21.15.33:0/92714464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b80bcea0 0x7f29b80a7ae0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:42.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.038+0000 7f29be7ff700 1 -- 172.21.15.33:0/92714464 >> 172.21.15.33:0/92714464 conn(0x7f29b80631a0 msgr2=0x7f29b80c1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:42.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.038+0000 7f29be7ff700 1 -- 172.21.15.33:0/92714464 shutdown_connections 2023-11-06T21:00:42.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.038+0000 7f29be7ff700 1 -- 172.21.15.33:0/92714464 wait complete. 2023-11-06T21:00:42.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T21:00:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.253+0000 7f0f917ff700 1 Processor -- start 2023-11-06T21:00:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.253+0000 7f0f917ff700 1 -- start start 2023-11-06T21:00:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.255+0000 7f0f917ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0f8c063840 0x7f0f8c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.255+0000 7f0f917ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c158c00 con 0x7f0f8c157700 2023-11-06T21:00:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.255+0000 7f0f917ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c0c7610 con 0x7f0f8c0aaf50 2023-11-06T21:00:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.255+0000 7f0f917ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c0c18f0 con 0x7f0f8c063840 2023-11-06T21:00:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.255+0000 7f0f8affd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0f8c157700 0x7f0f8c0c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46846/0 (socket says 172.21.15.33:46846) 2023-11-06T21:00:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.255+0000 7f0f8affd700 1 -- 172.21.15.33:0/3689231272 learned_addr learned my addr 172.21.15.33:0/3689231272 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.255+0000 7f0f8b7fe700 1 --2- 172.21.15.33:0/3689231272 >> v2:172.21.15.69:3300/0 conn(0x7f0f8c063840 0x7f0f8c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.255+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561117563 0 0) 0x7f0f8c158c00 con 0x7f0f8c157700 2023-11-06T21:00:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.255+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f740033f0 con 0x7f0f8c157700 2023-11-06T21:00:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 682863853 0 0) 0x7f0f740033f0 con 0x7f0f8c157700 2023-11-06T21:00:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 >> v2:172.21.15.69:3300/0 conn(0x7f0f8c063840 msgr2=0x7f0f8c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 --2- 172.21.15.33:0/3689231272 >> v2:172.21.15.69:3300/0 conn(0x7f0f8c063840 0x7f0f8c15f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 >> v1:172.21.15.69:6789/0 conn(0x7f0f8c0aaf50 legacy=0x7f0f8c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c157ff0 con 0x7f0f8c157700 2023-11-06T21:00:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f0f7c001470 con 0x7f0f8c157700 2023-11-06T21:00:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 >> v1:172.21.15.33:6789/0 conn(0x7f0f8c157700 legacy=0x7f0f8c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740038e0 0x7f0f74003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f74004690 0x7f0f74004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f74005000 0x7f0f7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0c18f0 con 0x7f0f740038e0 2023-11-06T21:00:42.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7400beb0 con 0x7f0f74004690 2023-11-06T21:00:42.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7400c070 con 0x7f0f74005000 2023-11-06T21:00:42.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f8b7fe700 1 --2- 172.21.15.33:0/3689231272 >> v2:172.21.15.69:3300/0 conn(0x7f0f8c063840 0x7f0f8c15f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:00:42.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f8affd700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740038e0 0x7f0f74003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:42.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f8a7fc700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f74004690 0x7f0f74004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:42.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.256+0000 7f0f8b7fe700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f74005000 0x7f0f7400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:42.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f8affd700 1 -- 172.21.15.33:0/3689231272 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f74005000 msgr2=0x7f0f7400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:42.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f8affd700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f74005000 0x7f0f7400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:42.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f8affd700 1 -- 172.21.15.33:0/3689231272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f74004690 msgr2=0x7f0f74004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:42.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f8affd700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f74004690 0x7f0f74004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f8affd700 1 -- 172.21.15.33:0/3689231272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f740033f0 con 0x7f0f740038e0 2023-11-06T21:00:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f8b7fe700 1 -- 172.21.15.33:0/3689231272 reap_dead start 2023-11-06T21:00:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f8affd700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740038e0 0x7f0f74003d00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f0f7c00c190 tx=0x7f0f7c001e50).ready entity=mon.0 client_cookie=bc47a45815d19967 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c007a20 con 0x7f0f740038e0 2023-11-06T21:00:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c00cac0 con 0x7f0f740038e0 2023-11-06T21:00:42.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f89ffb700 1 -- 172.21.15.33:0/3689231272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00ce40 con 0x7f0f740038e0 2023-11-06T21:00:42.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f917ff700 1 -- 172.21.15.33:0/3689231272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740038e0 msgr2=0x7f0f74003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:42.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.257+0000 7f0f917ff700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740038e0 0x7f0f74003d00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f0f7c00c190 tx=0x7f0f7c001e50).stop 2023-11-06T21:00:42.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.258+0000 7f0f917ff700 1 -- 172.21.15.33:0/3689231272 shutdown_connections 2023-11-06T21:00:42.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.258+0000 7f0f917ff700 1 --2- 172.21.15.33:0/3689231272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740038e0 0x7f0f74003d00 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:42.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.258+0000 7f0f917ff700 1 -- 172.21.15.33:0/3689231272 >> 172.21.15.33:0/3689231272 conn(0x7f0f8c05ec80 msgr2=0x7f0f8c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:42.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.258+0000 7f0f917ff700 1 -- 172.21.15.33:0/3689231272 shutdown_connections 2023-11-06T21:00:42.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.258+0000 7f0f917ff700 1 -- 172.21.15.33:0/3689231272 wait complete. 2023-11-06T21:00:42.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.259+0000 7f0f917ff700 1 Processor -- start 2023-11-06T21:00:42.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.259+0000 7f0f917ff700 1 -- start start 2023-11-06T21:00:42.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.260+0000 7f0f917ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f740038e0 0x7f0f8c14eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:42.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.260+0000 7f0f917ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f8c0adde0 0x7f0f8c155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:42.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.260+0000 7f0f917ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f8c1561c0 0x7f0f8c0af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:42.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.260+0000 7f0f917ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0aee80 con 0x7f0f8c0adde0 2023-11-06T21:00:42.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.260+0000 7f0f917ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0ae270 con 0x7f0f8c1561c0 2023-11-06T21:00:42.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.260+0000 7f0f917ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0ae550 con 0x7f0f740038e0 2023-11-06T21:00:42.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f8c0adde0 0x7f0f8c155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:42.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f8c0adde0 0x7f0f8c155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41188/0 (socket says 172.21.15.33:41188) 2023-11-06T21:00:42.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8a7fc700 1 -- 172.21.15.33:0/1427510339 learned_addr learned my addr 172.21.15.33:0/1427510339 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:42.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8affd700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f740038e0 0x7f0f8c14eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:42.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8b7fe700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f8c1561c0 0x7f0f8c0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:42.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8a7fc700 1 -- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f740038e0 msgr2=0x7f0f8c14eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:42.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8a7fc700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f740038e0 0x7f0f8c14eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:42.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8a7fc700 1 -- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f8c1561c0 msgr2=0x7f0f8c0af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:42.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8a7fc700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f8c1561c0 0x7f0f8c0af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:42.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8a7fc700 1 -- 172.21.15.33:0/1427510339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c0ae830 con 0x7f0f8c0adde0 2023-11-06T21:00:42.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f8a7fc700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f8c0adde0 0x7f0f8c155cd0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f0f80002670 tx=0x7f0f80007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:42.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f72ffd700 1 -- 172.21.15.33:0/1427510339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80007240 con 0x7f0f8c0adde0 2023-11-06T21:00:42.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f72ffd700 1 -- 172.21.15.33:0/1427510339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f800073a0 con 0x7f0f8c0adde0 2023-11-06T21:00:42.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f72ffd700 1 -- 172.21.15.33:0/1427510339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f800066a0 con 0x7f0f8c0adde0 2023-11-06T21:00:42.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.261+0000 7f0f917ff700 1 -- 172.21.15.33:0/1427510339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c0aeb10 con 0x7f0f8c0adde0 2023-11-06T21:00:42.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.262+0000 7f0f917ff700 1 -- 172.21.15.33:0/1427510339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c0af800 con 0x7f0f8c0adde0 2023-11-06T21:00:42.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.263+0000 7f0f917ff700 1 -- 172.21.15.33:0/1427510339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c0aeca0 con 0x7f0f8c0adde0 2023-11-06T21:00:42.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.263+0000 7f0f72ffd700 1 -- 172.21.15.33:0/1427510339 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 38) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0f80006a50 con 0x7f0f8c0adde0 2023-11-06T21:00:42.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.264+0000 7f0f72ffd700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0f7805f270 0x7f0f78061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:42.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.264+0000 7f0f72ffd700 1 -- 172.21.15.33:0/1427510339 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0f8007a560 con 0x7f0f8c0adde0 2023-11-06T21:00:42.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.264+0000 7f0f8affd700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0f7805f270 0x7f0f78061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:42.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.265+0000 7f0f8affd700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0f7805f270 0x7f0f78061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0f7c001ed0 tx=0x7f0f7c015000).ready entity=mgr.6050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:42.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.269+0000 7f0f72ffd700 1 -- 172.21.15.33:0/1427510339 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0f8004b630 con 0x7f0f8c0adde0 2023-11-06T21:00:42.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:42.503+0000 7f0f917ff700 1 -- 172.21.15.33:0/1427510339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f0f8c0aefc0 con 0x7f0f8c0adde0 2023-11-06T21:00:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.303+0000 7f0f72ffd700 1 -- 172.21.15.33:0/1427510339 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 39) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f0f80006d00 con 0x7f0f8c0adde0 2023-11-06T21:00:43.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.305+0000 7f0f72ffd700 1 -- 172.21.15.33:0/1427510339 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v39) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0f80073f30 con 0x7f0f8c0adde0 2023-11-06T21:00:43.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 -- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0f7805f270 msgr2=0x7f0f78061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0f7805f270 0x7f0f78061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0f7c001ed0 tx=0x7f0f7c015000).stop 2023-11-06T21:00:43.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 -- 172.21.15.33:0/1427510339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f8c0adde0 msgr2=0x7f0f8c155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f8c0adde0 0x7f0f8c155cd0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f0f80002670 tx=0x7f0f80007bb0).stop 2023-11-06T21:00:43.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 -- 172.21.15.33:0/1427510339 shutdown_connections 2023-11-06T21:00:43.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0f7805f270 0x7f0f78061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f740038e0 0x7f0f8c14eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f8c1561c0 0x7f0f8c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 --2- 172.21.15.33:0/1427510339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f8c0adde0 0x7f0f8c155cd0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 -- 172.21.15.33:0/1427510339 >> 172.21.15.33:0/1427510339 conn(0x7f0f8c05ec80 msgr2=0x7f0f8c0ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:43.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 -- 172.21.15.33:0/1427510339 shutdown_connections 2023-11-06T21:00:43.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.308+0000 7f0f917ff700 1 -- 172.21.15.33:0/1427510339 wait complete. 2023-11-06T21:00:43.330 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T21:00:43.520 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:43.520+0000 7f36046ed680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:00:43.526 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:43.526+0000 7f086fd66680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:00:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.540+0000 7fd12f9ff700 1 Processor -- start 2023-11-06T21:00:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.540+0000 7fd12f9ff700 1 -- start start 2023-11-06T21:00:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.541+0000 7fd12f9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd12814fe80 0x7fd128064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.541+0000 7fd12f9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd1280c3450 0x7fd1280c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.541+0000 7fd12f9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd128064270 con 0x7fd12814faa0 2023-11-06T21:00:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.541+0000 7fd12f9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1280c3dd0 con 0x7fd12814fe80 2023-11-06T21:00:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.541+0000 7fd12f9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1280651b0 con 0x7fd1280c3450 2023-11-06T21:00:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.542+0000 7fd12cf9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd12814fe80 0x7fd128064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.542+0000 7fd12d79b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd12814faa0 0x7fd128152750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46868/0 (socket says 172.21.15.33:46868) 2023-11-06T21:00:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.542+0000 7fd12d79b700 1 -- 172.21.15.33:0/2038278338 learned_addr learned my addr 172.21.15.33:0/2038278338 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.542+0000 7fd12df9c700 1 --2- 172.21.15.33:0/2038278338 >> v2:172.21.15.69:3300/0 conn(0x7fd1280c3450 0x7fd1280c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.542+0000 7fd12cf9a700 1 -- 172.21.15.33:0/2038278338 >> v2:172.21.15.69:3300/0 conn(0x7fd1280c3450 msgr2=0x7fd1280c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.542+0000 7fd12cf9a700 1 --2- 172.21.15.33:0/2038278338 >> v2:172.21.15.69:3300/0 conn(0x7fd1280c3450 0x7fd1280c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.542+0000 7fd12cf9a700 1 -- 172.21.15.33:0/2038278338 >> v1:172.21.15.33:6789/0 conn(0x7fd12814faa0 legacy=0x7fd128152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.542+0000 7fd12cf9a700 1 -- 172.21.15.33:0/2038278338 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1281531f0 con 0x7fd12814fe80 2023-11-06T21:00:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.542+0000 7fd11ffff700 1 -- 172.21.15.33:0/2038278338 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080797572 0 0) 0x7fd128064270 con 0x7fd12814faa0 2023-11-06T21:00:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.543+0000 7fd12cf9a700 1 --2- 172.21.15.33:0/2038278338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd12814fe80 0x7fd128064cc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7fd1280653a0 tx=0x7fd118009060).ready entity=mon.0 client_cookie=fa3e4cf637683594 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:43.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.543+0000 7fd11ffff700 1 -- 172.21.15.33:0/2038278338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11800f510 con 0x7fd12814fe80 2023-11-06T21:00:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.543+0000 7fd11ffff700 1 -- 172.21.15.33:0/2038278338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11800f670 con 0x7fd12814fe80 2023-11-06T21:00:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.543+0000 7fd11ffff700 1 -- 172.21.15.33:0/2038278338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11800f9f0 con 0x7fd12814fe80 2023-11-06T21:00:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.543+0000 7fd12f9ff700 1 -- 172.21.15.33:0/2038278338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd12814fe80 msgr2=0x7fd128064cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.543+0000 7fd12f9ff700 1 --2- 172.21.15.33:0/2038278338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd12814fe80 0x7fd128064cc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7fd1280653a0 tx=0x7fd118009060).stop 2023-11-06T21:00:43.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.543+0000 7fd12f9ff700 1 -- 172.21.15.33:0/2038278338 shutdown_connections 2023-11-06T21:00:43.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.543+0000 7fd12f9ff700 1 --2- 172.21.15.33:0/2038278338 >> v2:172.21.15.69:3300/0 conn(0x7fd1280c3450 0x7fd1280c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.543+0000 7fd12f9ff700 1 --2- 172.21.15.33:0/2038278338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd12814fe80 0x7fd128064cc0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.543+0000 7fd12f9ff700 1 -- 172.21.15.33:0/2038278338 >> 172.21.15.33:0/2038278338 conn(0x7fd128062710 msgr2=0x7fd128062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:43.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.544+0000 7fd12f9ff700 1 -- 172.21.15.33:0/2038278338 shutdown_connections 2023-11-06T21:00:43.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.544+0000 7fd12f9ff700 1 -- 172.21.15.33:0/2038278338 wait complete. 2023-11-06T21:00:43.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.545+0000 7fd12f9ff700 1 Processor -- start 2023-11-06T21:00:43.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.545+0000 7fd12f9ff700 1 -- start start 2023-11-06T21:00:43.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.546+0000 7fd12f9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1280c3450 0x7fd1280c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.546+0000 7fd12f9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd12814faa0 0x7fd1280c1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.546+0000 7fd12f9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd12814fe80 0x7fd1280c1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.546+0000 7fd12f9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280c2860 con 0x7fd12814fe80 2023-11-06T21:00:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.546+0000 7fd12f9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280c1a00 con 0x7fd1280c3450 2023-11-06T21:00:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.546+0000 7fd12f9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280c1c50 con 0x7fd12814faa0 2023-11-06T21:00:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12d79b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1280c3450 0x7fd1280c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12df9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd12814fe80 0x7fd1280c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12d79b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1280c3450 0x7fd1280c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:48918/0 (socket says 172.21.15.33:48918) 2023-11-06T21:00:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12d79b700 1 -- 172.21.15.33:0/3857817361 learned_addr learned my addr 172.21.15.33:0/3857817361 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12cf9a700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd12814faa0 0x7fd1280c1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12d79b700 1 -- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd12814faa0 msgr2=0x7fd1280c1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12d79b700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd12814faa0 0x7fd1280c1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12d79b700 1 -- 172.21.15.33:0/3857817361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd12814fe80 msgr2=0x7fd1280c1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12d79b700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd12814fe80 0x7fd1280c1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12d79b700 1 -- 172.21.15.33:0/3857817361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1180092a0 con 0x7fd1280c3450 2023-11-06T21:00:43.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.547+0000 7fd12d79b700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1280c3450 0x7fd1280c7230 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fd12400c4e0 tx=0x7fd124007c70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:43.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.548+0000 7fd11d7fa700 1 -- 172.21.15.33:0/3857817361 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12400ce90 con 0x7fd1280c3450 2023-11-06T21:00:43.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.548+0000 7fd11d7fa700 1 -- 172.21.15.33:0/3857817361 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1240072e0 con 0x7fd1280c3450 2023-11-06T21:00:43.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.548+0000 7fd11d7fa700 1 -- 172.21.15.33:0/3857817361 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124007660 con 0x7fd1280c3450 2023-11-06T21:00:43.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.548+0000 7fd12f9ff700 1 -- 172.21.15.33:0/3857817361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1280c1f30 con 0x7fd1280c3450 2023-11-06T21:00:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.548+0000 7fd12f9ff700 1 -- 172.21.15.33:0/3857817361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1280c24f0 con 0x7fd1280c3450 2023-11-06T21:00:43.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.549+0000 7fd12f9ff700 1 -- 172.21.15.33:0/3857817361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1281534d0 con 0x7fd1280c3450 2023-11-06T21:00:43.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.550+0000 7fd11d7fa700 1 -- 172.21.15.33:0/3857817361 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 39) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fd1240059b0 con 0x7fd1280c3450 2023-11-06T21:00:43.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.551+0000 7fd11d7fa700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10c05f2c0 0x7fd10c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.551+0000 7fd11d7fa700 1 -- 172.21.15.33:0/3857817361 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd124079b80 con 0x7fd1280c3450 2023-11-06T21:00:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.551+0000 7fd12cf9a700 1 -- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10c05f2c0 msgr2=0x7fd10c061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.551+0000 7fd12cf9a700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10c05f2c0 0x7fd10c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.555+0000 7fd11d7fa700 1 -- 172.21.15.33:0/3857817361 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd12404ac10 con 0x7fd1280c3450 2023-11-06T21:00:43.599 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:43.599+0000 7f36046ed680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:00:43.611 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:43.610+0000 7f086fd66680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:00:43.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.738+0000 7fd12f9ff700 1 -- 172.21.15.33:0/3857817361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fd128063c00 con 0x7fd1280c3450 2023-11-06T21:00:43.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.744+0000 7fd11d7fa700 1 -- 172.21.15.33:0/3857817361 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v19)=0 v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd124079e60 con 0x7fd1280c3450 2023-11-06T21:00:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 -- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10c05f2c0 msgr2=0x7fd10c061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10c05f2c0 0x7fd10c061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 -- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1280c3450 msgr2=0x7fd1280c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1280c3450 0x7fd1280c7230 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fd12400c4e0 tx=0x7fd124007c70).stop 2023-11-06T21:00:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 -- 172.21.15.33:0/3857817361 shutdown_connections 2023-11-06T21:00:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd10c05f2c0 0x7fd10c061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd12814fe80 0x7fd1280c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1280c3450 0x7fd1280c7230 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 --2- 172.21.15.33:0/3857817361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd12814faa0 0x7fd1280c1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 -- 172.21.15.33:0/3857817361 >> 172.21.15.33:0/3857817361 conn(0x7fd128062710 msgr2=0x7fd1281523b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 -- 172.21.15.33:0/3857817361 shutdown_connections 2023-11-06T21:00:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.746+0000 7fd12f9ff700 1 -- 172.21.15.33:0/3857817361 wait complete. 2023-11-06T21:00:43.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /bad-path 1M 2023-11-06T21:00:43.928 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:43.928+0000 7f36046ed680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:00:43.967 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:43.967+0000 7f086fd66680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:00:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.969+0000 7f57acfff700 1 Processor -- start 2023-11-06T21:00:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.970+0000 7f57acfff700 1 -- start start 2023-11-06T21:00:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.971+0000 7f57acfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a81542b0 0x7f57a80c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.971+0000 7f57acfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a80c9f80 con 0x7f57a8153ed0 2023-11-06T21:00:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.971+0000 7f57acfff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a80cab20 con 0x7f57a80c8ee0 2023-11-06T21:00:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.971+0000 7f57acfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80c6000 con 0x7f57a81542b0 2023-11-06T21:00:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.971+0000 7f57a659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f57a8153ed0 0x7f57a8158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46876/0 (socket says 172.21.15.33:46876) 2023-11-06T21:00:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.971+0000 7f57a659c700 1 -- 172.21.15.33:0/3949517759 learned_addr learned my addr 172.21.15.33:0/3949517759 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.971+0000 7f57a6d9d700 1 --2- 172.21.15.33:0/3949517759 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a81542b0 0x7f57a80c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:43.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.972+0000 7f57a559a700 1 -- 172.21.15.33:0/3949517759 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426746050 0 0) 0x7f57a80c9f80 con 0x7f57a8153ed0 2023-11-06T21:00:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.972+0000 7f57a559a700 1 -- 172.21.15.33:0/3949517759 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57900033f0 con 0x7f57a8153ed0 2023-11-06T21:00:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.972+0000 7f57a559a700 1 -- 172.21.15.33:0/3949517759 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801665445 0 0) 0x7f57a80cab20 con 0x7f57a80c8ee0 2023-11-06T21:00:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.972+0000 7f57a559a700 1 -- 172.21.15.33:0/3949517759 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a80c9f80 con 0x7f57a80c8ee0 2023-11-06T21:00:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.972+0000 7f57a6d9d700 1 -- 172.21.15.33:0/3949517759 >> v1:172.21.15.69:6789/0 conn(0x7f57a80c8ee0 legacy=0x7f57a80ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.972+0000 7f57a6d9d700 1 -- 172.21.15.33:0/3949517759 >> v1:172.21.15.33:6789/0 conn(0x7f57a8153ed0 legacy=0x7f57a8158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.972+0000 7f57a6d9d700 1 -- 172.21.15.33:0/3949517759 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80c9370 con 0x7f57a81542b0 2023-11-06T21:00:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.972+0000 7f57a6d9d700 1 --2- 172.21.15.33:0/3949517759 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a81542b0 0x7f57a80c76f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f57a0009e90 tx=0x7f57a0009780).ready entity=mon.2 client_cookie=212895e15a6e1b59 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.973+0000 7f57a559a700 1 -- 172.21.15.33:0/3949517759 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a000e070 con 0x7f57a81542b0 2023-11-06T21:00:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.973+0000 7f57a559a700 1 -- 172.21.15.33:0/3949517759 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a000bab0 con 0x7f57a81542b0 2023-11-06T21:00:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.973+0000 7f57a559a700 1 -- 172.21.15.33:0/3949517759 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a000be30 con 0x7f57a81542b0 2023-11-06T21:00:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.973+0000 7f57acfff700 1 -- 172.21.15.33:0/3949517759 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a81542b0 msgr2=0x7f57a80c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.973+0000 7f57acfff700 1 --2- 172.21.15.33:0/3949517759 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a81542b0 0x7f57a80c76f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f57a0009e90 tx=0x7f57a0009780).stop 2023-11-06T21:00:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.973+0000 7f57acfff700 1 -- 172.21.15.33:0/3949517759 shutdown_connections 2023-11-06T21:00:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.973+0000 7f57acfff700 1 --2- 172.21.15.33:0/3949517759 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a81542b0 0x7f57a80c76f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.973+0000 7f57acfff700 1 -- 172.21.15.33:0/3949517759 >> 172.21.15.33:0/3949517759 conn(0x7f57a805e1b0 msgr2=0x7f57a805e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:43.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.973+0000 7f57acfff700 1 -- 172.21.15.33:0/3949517759 shutdown_connections 2023-11-06T21:00:43.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.973+0000 7f57acfff700 1 -- 172.21.15.33:0/3949517759 wait complete. 2023-11-06T21:00:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.975+0000 7f57acfff700 1 Processor -- start 2023-11-06T21:00:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.975+0000 7f57acfff700 1 -- start start 2023-11-06T21:00:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57acfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a8153ed0 0x7f57a8164750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57acfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57a81542b0 0x7f57a8168700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57acfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a8150cc0 0x7f57a814e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57acfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8168bf0 con 0x7f57a8150cc0 2023-11-06T21:00:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57acfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8152bf0 con 0x7f57a81542b0 2023-11-06T21:00:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57acfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a814f600 con 0x7f57a8153ed0 2023-11-06T21:00:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57a6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a8150cc0 0x7f57a814e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57a659c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a8153ed0 0x7f57a8164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57a659c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a8153ed0 0x7f57a8164750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:34650/0 (socket says 172.21.15.33:34650) 2023-11-06T21:00:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57a6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a8150cc0 0x7f57a814e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41228/0 (socket says 172.21.15.33:41228) 2023-11-06T21:00:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57a5d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57a81542b0 0x7f57a8168700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.976+0000 7f57a659c700 1 -- 172.21.15.33:0/1057725999 learned_addr learned my addr 172.21.15.33:0/1057725999 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57a659c700 1 -- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57a81542b0 msgr2=0x7f57a8168700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57a659c700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57a81542b0 0x7f57a8168700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57a659c700 1 -- 172.21.15.33:0/1057725999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a8150cc0 msgr2=0x7f57a814e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57a659c700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a8150cc0 0x7f57a814e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57a659c700 1 -- 172.21.15.33:0/1057725999 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a0009a20 con 0x7f57a8153ed0 2023-11-06T21:00:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57a659c700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a8153ed0 0x7f57a8164750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f579800c560 tx=0x7f5798007d00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579800ce70 con 0x7f57a8153ed0 2023-11-06T21:00:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57980072e0 con 0x7f57a8153ed0 2023-11-06T21:00:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57acfff700 1 -- 172.21.15.33:0/1057725999 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a80af7e0 con 0x7f57a8153ed0 2023-11-06T21:00:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798007660 con 0x7f57a8153ed0 2023-11-06T21:00:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.977+0000 7f57acfff700 1 -- 172.21.15.33:0/1057725999 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a8165040 con 0x7f57a8153ed0 2023-11-06T21:00:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.979+0000 7f57acfff700 1 -- 172.21.15.33:0/1057725999 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a806c6c0 con 0x7f57a8153ed0 2023-11-06T21:00:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.979+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 39) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f57980059b0 con 0x7f57a8153ed0 2023-11-06T21:00:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.980+0000 7f57967fc700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f5788061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.980+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5798079ab0 con 0x7f57a8153ed0 2023-11-06T21:00:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.980+0000 7f57a5d9b700 1 -- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 msgr2=0x7f5788061750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.980+0000 7f57a5d9b700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f5788061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:00:43.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:43.985+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f579804ab40 con 0x7f57a8153ed0 2023-11-06T21:00:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:44.181+0000 7f57a5d9b700 1 -- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 msgr2=0x7f5788061750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:44.181+0000 7f57a5d9b700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f5788061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:00:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:44.193+0000 7f57acfff700 1 -- 172.21.15.33:0/1057725999 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f57a806abf0 con 0x7f578805f290 2023-11-06T21:00:44.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:44.582+0000 7f57a5d9b700 1 -- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 msgr2=0x7f5788061750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:44.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:44.582+0000 7f57a5d9b700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f5788061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T21:00:44.675 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:44.675+0000 7f36046ed680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:00:44.754 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:44.754+0000 7f086fd66680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:00:44.758 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:44.758+0000 7f36046ed680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:00:44.841 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:44.841+0000 7f086fd66680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:00:44.990 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:44.990+0000 7f36046ed680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:00:45.081 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:45.081+0000 7f086fd66680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:00:45.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:45.383+0000 7f57a5d9b700 1 -- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 msgr2=0x7f5788061750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:45.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:45.383+0000 7f57a5d9b700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f5788061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T21:00:45.504 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:45.504+0000 7f36046ed680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:00:45.616 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:45.616+0000 7f086fd66680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:00:45.629 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:45.629+0000 7f36046ed680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:45.747 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:45.746+0000 7f086fd66680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:45.813 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:45.813+0000 7f36046ed680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:45.908 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:45.908+0000 7f36046ed680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:00:45.938 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:45.938+0000 7f086fd66680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:45.993 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:45.993+0000 7f36046ed680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:00:46.037 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:46.038+0000 7f086fd66680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:00:46.126 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:46.127+0000 7f086fd66680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:00:46.305 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:46.303+0000 7f36046ed680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:00:46.451 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:46.452+0000 7f086fd66680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:00:46.526 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:46.526+0000 7f36046ed680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:46.681 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:46.682+0000 7f086fd66680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:00:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:46.986+0000 7f57a5d9b700 1 -- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 msgr2=0x7f5788061750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:00:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:46.986+0000 7f57a5d9b700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f5788061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T21:00:47.333 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:47.333+0000 7f36046ed680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:00:47.420 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:47.420+0000 7f36046ed680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:00:47.515 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:47.515+0000 7f36046ed680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:00:47.527 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:47.528+0000 7f086fd66680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:00:47.619 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:47.619+0000 7f086fd66680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:00:47.699 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:47.699+0000 7f36046ed680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:00:47.717 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:47.718+0000 7f086fd66680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:00:47.788 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:47.788+0000 7f36046ed680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:00:47.911 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:47.912+0000 7f086fd66680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:00:47.929 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:47.929+0000 7f36046ed680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:00:48.005 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:48.005+0000 7f086fd66680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:00:48.059 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.060+0000 7f36046ed680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:48.153 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:48.154+0000 7f086fd66680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:00:48.291 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:48.291+0000 7f086fd66680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:00:48.406 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.406+0000 7f36046ed680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:00:48.497 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.497+0000 7f36046ed680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:00:48.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:48.511+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mgrmap(e 40) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f5798077640 con 0x7f57a8153ed0 2023-11-06T21:00:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:48.511+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 msgr2=0x7f5788061750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:00:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:48.511+0000 7f57967fc700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f5788061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:00:48.583 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.583+0000 7f357a207700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:48.583 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.583+0000 7f357a207700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:48.583 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.583+0000 7f357a207700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:48.583 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.583+0000 7f357a207700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:48.583 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.583+0000 7f357a207700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:48.584 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.584+0000 7f35761ff700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:48.584 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.584+0000 7f35761ff700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:48.584 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.584+0000 7f35761ff700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:48.585 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.584+0000 7f35761ff700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:48.585 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:00:48.584+0000 7f35761ff700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:00:48.655 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:48.656+0000 7f086fd66680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:00:48.750 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:00:48.750+0000 7f086fd66680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:00:49.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.519+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 <== mon.2 v2:172.21.15.69:3301/0 8 ==== mgrmap(e 41) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f57980779c0 con 0x7f57a8153ed0 2023-11-06T21:00:49.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.519+0000 7f57967fc700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f57880616d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:49.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.519+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f57a806abf0 con 0x7f578805f290 2023-11-06T21:00:49.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.523+0000 7f57a5d9b700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f57880616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:49.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.524+0000 7f57a5d9b700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f57880616d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f57a8168e00 tx=0x7f579c007bd0).ready entity=mgr.6311 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.532+0000 7f57967fc700 1 -- 172.21.15.33:0/1057725999 <== mgr.6311 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f57a806abf0 con 0x7f578805f290 2023-11-06T21:00:49.532 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /bad-path 2023-11-06T21:00:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 -- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 msgr2=0x7f57880616d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f57880616d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f57a8168e00 tx=0x7f579c007bd0).stop 2023-11-06T21:00:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 -- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a8153ed0 msgr2=0x7f57a8164750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a8153ed0 0x7f57a8164750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f579800c560 tx=0x7f5798007d00).stop 2023-11-06T21:00:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 -- 172.21.15.33:0/1057725999 shutdown_connections 2023-11-06T21:00:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f578805f290 0x7f57880616d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57a8153ed0 0x7f57a8164750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a8150cc0 0x7f57a814e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 --2- 172.21.15.33:0/1057725999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57a81542b0 0x7f57a8168700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 -- 172.21.15.33:0/1057725999 >> 172.21.15.33:0/1057725999 conn(0x7f57a805e1b0 msgr2=0x7f57a80c7580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 -- 172.21.15.33:0/1057725999 shutdown_connections 2023-11-06T21:00:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.535+0000 7f57acfff700 1 -- 172.21.15.33:0/1057725999 wait complete. 2023-11-06T21:00:49.555 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-11-06T21:00:49.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-11-06T21:00:49.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.762+0000 7fb7463ff700 1 Processor -- start 2023-11-06T21:00:49.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.762+0000 7fb7463ff700 1 -- start start 2023-11-06T21:00:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.764+0000 7fb7463ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb740154340 0x7fb7400aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.764+0000 7fb7463ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb740158720 0x7fb740063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.764+0000 7fb7463ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400c0e00 con 0x7fb740154720 2023-11-06T21:00:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.764+0000 7fb7463ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb740158bf0 con 0x7fb740154340 2023-11-06T21:00:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.764+0000 7fb7463ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb740158ed0 con 0x7fb740158720 2023-11-06T21:00:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.764+0000 7fb73ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb740154340 0x7fb7400aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.764+0000 7fb73f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb740154720 0x7fb740158010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57670/0 (socket says 172.21.15.33:57670) 2023-11-06T21:00:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.764+0000 7fb73f7fe700 1 -- 172.21.15.33:0/3268191102 learned_addr learned my addr 172.21.15.33:0/3268191102 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:00:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb73ffff700 1 -- 172.21.15.33:0/3268191102 >> v2:172.21.15.69:3300/0 conn(0x7fb740158720 msgr2=0x7fb740063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:00:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb74499c700 1 --2- 172.21.15.33:0/3268191102 >> v2:172.21.15.69:3300/0 conn(0x7fb740158720 0x7fb740063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:49.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb73ffff700 1 --2- 172.21.15.33:0/3268191102 >> v2:172.21.15.69:3300/0 conn(0x7fb740158720 0x7fb740063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb73effd700 1 -- 172.21.15.33:0/3268191102 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437287941 0 0) 0x7fb7400c0e00 con 0x7fb740154720 2023-11-06T21:00:49.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb73ffff700 1 -- 172.21.15.33:0/3268191102 >> v1:172.21.15.33:6789/0 conn(0x7fb740154720 legacy=0x7fb740158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:49.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb73ffff700 1 -- 172.21.15.33:0/3268191102 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400c5d80 con 0x7fb740154340 2023-11-06T21:00:49.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb74499c700 1 --2- 172.21.15.33:0/3268191102 >> v2:172.21.15.69:3300/0 conn(0x7fb740158720 0x7fb740063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:00:49.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb73ffff700 1 --2- 172.21.15.33:0/3268191102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 0x7fb7400aae40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7fb7400adba0 tx=0x7fb730009060).ready entity=mon.0 client_cookie=b9faa27d039bbb17 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:49.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb73effd700 1 -- 172.21.15.33:0/3268191102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73000b580 con 0x7fb740154340 2023-11-06T21:00:49.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb73effd700 1 -- 172.21.15.33:0/3268191102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73000b6e0 con 0x7fb740154340 2023-11-06T21:00:49.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.765+0000 7fb73effd700 1 -- 172.21.15.33:0/3268191102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73000ba60 con 0x7fb740154340 2023-11-06T21:00:49.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.766+0000 7fb7463ff700 1 -- 172.21.15.33:0/3268191102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 msgr2=0x7fb7400aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:49.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.766+0000 7fb7463ff700 1 --2- 172.21.15.33:0/3268191102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 0x7fb7400aae40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7fb7400adba0 tx=0x7fb730009060).stop 2023-11-06T21:00:49.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.766+0000 7fb7463ff700 1 -- 172.21.15.33:0/3268191102 shutdown_connections 2023-11-06T21:00:49.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.766+0000 7fb7463ff700 1 --2- 172.21.15.33:0/3268191102 >> v2:172.21.15.69:3300/0 conn(0x7fb740158720 0x7fb740063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.766+0000 7fb7463ff700 1 --2- 172.21.15.33:0/3268191102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 0x7fb7400aae40 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.766+0000 7fb7463ff700 1 -- 172.21.15.33:0/3268191102 >> 172.21.15.33:0/3268191102 conn(0x7fb74005ec80 msgr2=0x7fb7400cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:49.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.766+0000 7fb7463ff700 1 -- 172.21.15.33:0/3268191102 shutdown_connections 2023-11-06T21:00:49.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.766+0000 7fb7463ff700 1 -- 172.21.15.33:0/3268191102 wait complete. 2023-11-06T21:00:49.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.768+0000 7fb7463ff700 1 Processor -- start 2023-11-06T21:00:49.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.768+0000 7fb7463ff700 1 -- start start 2023-11-06T21:00:49.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.769+0000 7fb7463ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 0x7fb7400b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:49.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb7463ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb740154720 0x7fb7400c2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:49.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb7463ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb740158720 0x7fb7400c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:49.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb7463ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400c3690 con 0x7fb740154340 2023-11-06T21:00:49.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb7463ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400c2ef0 con 0x7fb740154720 2023-11-06T21:00:49.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb7463ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb740156140 con 0x7fb740158720 2023-11-06T21:00:49.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 0x7fb7400b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:49.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 0x7fb7400b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42668/0 (socket says 172.21.15.33:42668) 2023-11-06T21:00:49.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb73ffff700 1 -- 172.21.15.33:0/189018876 learned_addr learned my addr 172.21.15.33:0/189018876 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:00:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb73f7fe700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb740154720 0x7fb7400c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb74499c700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb740158720 0x7fb7400c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb73ffff700 1 -- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb740158720 msgr2=0x7fb7400c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb73ffff700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb740158720 0x7fb7400c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb73ffff700 1 -- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb740154720 msgr2=0x7fb7400c2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb73ffff700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb740154720 0x7fb7400c2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb73ffff700 1 -- 172.21.15.33:0/189018876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300092a0 con 0x7fb740154340 2023-11-06T21:00:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.770+0000 7fb73ffff700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 0x7fb7400b0780 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fb740065a00 tx=0x7fb730017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.771+0000 7fb727fff700 1 -- 172.21.15.33:0/189018876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73000b850 con 0x7fb740154340 2023-11-06T21:00:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.771+0000 7fb727fff700 1 -- 172.21.15.33:0/189018876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73000b9b0 con 0x7fb740154340 2023-11-06T21:00:49.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.771+0000 7fb727fff700 1 -- 172.21.15.33:0/189018876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73001b620 con 0x7fb740154340 2023-11-06T21:00:49.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.771+0000 7fb7463ff700 1 -- 172.21.15.33:0/189018876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7401566a0 con 0x7fb740154340 2023-11-06T21:00:49.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.771+0000 7fb7463ff700 1 -- 172.21.15.33:0/189018876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7400c30d0 con 0x7fb740154340 2023-11-06T21:00:49.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.772+0000 7fb7463ff700 1 -- 172.21.15.33:0/189018876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7400c6060 con 0x7fb740154340 2023-11-06T21:00:49.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.773+0000 7fb727fff700 1 -- 172.21.15.33:0/189018876 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fb73001b990 con 0x7fb740154340 2023-11-06T21:00:49.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.774+0000 7fb727fff700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72c05f080 0x7fb72c061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:00:49.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.774+0000 7fb727fff700 1 -- 172.21.15.33:0/189018876 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fb73007bd20 con 0x7fb740154340 2023-11-06T21:00:49.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.775+0000 7fb73f7fe700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72c05f080 0x7fb72c061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:00:49.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.775+0000 7fb73f7fe700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72c05f080 0x7fb72c061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fb7400cb650 tx=0x7fb734009040).ready entity=mgr.6311 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:00:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.779+0000 7fb727fff700 1 -- 172.21.15.33:0/189018876 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb73004d0c0 con 0x7fb740154340 2023-11-06T21:00:49.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.990+0000 7fb7463ff700 1 -- 172.21.15.33:0/189018876 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb740062ba0 con 0x7fb72c05f080 2023-11-06T21:00:49.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.993+0000 7fb727fff700 1 -- 172.21.15.33:0/189018876 <== mgr.6311 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7fb740062ba0 con 0x7fb72c05f080 2023-11-06T21:00:49.993 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:00:49.993 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T21:00:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-11-06T21:00:49.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 -- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72c05f080 msgr2=0x7fb72c061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:49.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72c05f080 0x7fb72c061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fb7400cb650 tx=0x7fb734009040).stop 2023-11-06T21:00:49.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 -- 172.21.15.33:0/189018876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 msgr2=0x7fb7400b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:00:49.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 0x7fb7400b0780 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fb740065a00 tx=0x7fb730017000).stop 2023-11-06T21:00:49.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 -- 172.21.15.33:0/189018876 shutdown_connections 2023-11-06T21:00:49.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb72c05f080 0x7fb72c061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb740158720 0x7fb7400c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb740154340 0x7fb7400b0780 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 --2- 172.21.15.33:0/189018876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb740154720 0x7fb7400c2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:00:49.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 -- 172.21.15.33:0/189018876 >> 172.21.15.33:0/189018876 conn(0x7fb74005ec80 msgr2=0x7fb74014de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:00:49.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 -- 172.21.15.33:0/189018876 shutdown_connections 2023-11-06T21:00:49.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:00:49.996+0000 7fb7463ff700 1 -- 172.21.15.33:0/189018876 wait complete. 2023-11-06T21:00:50.013 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T21:00:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-11-06T21:01:00.538 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:00.538+0000 7f35699e6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:00.538 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:00.538+0000 7f35699e6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:00.539 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:00.538+0000 7f35699e6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:00.539 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:00.538+0000 7f35699e6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:00.539 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:00.538+0000 7f35699e6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:08.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:01:08.943 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:01:39.059 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:01:39.065 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:01:50.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-11-06T21:01:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.231+0000 7fd1987ff700 1 Processor -- start 2023-11-06T21:01:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.231+0000 7fd1987ff700 1 -- start start 2023-11-06T21:01:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.233+0000 7fd1987ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 0x7fd1900a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.233+0000 7fd1987ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd190154c60 con 0x7fd190151940 2023-11-06T21:01:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.233+0000 7fd1987ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd190068f50 con 0x7fd1901514c0 2023-11-06T21:01:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.233+0000 7fd1987ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd190151dc0 con 0x7fd1901510e0 2023-11-06T21:01:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.233+0000 7fd196d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd190151940 0x7fd19015ae90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56360/0 (socket says 172.21.15.33:56360) 2023-11-06T21:01:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.233+0000 7fd196d9c700 1 -- 172.21.15.33:0/882403793 learned_addr learned my addr 172.21.15.33:0/882403793 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.233+0000 7fd19659b700 1 --2- 172.21.15.33:0/882403793 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 0x7fd1900a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.233+0000 7fd195599700 1 -- 172.21.15.33:0/882403793 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257214456 0 0) 0x7fd190154c60 con 0x7fd190151940 2023-11-06T21:01:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.233+0000 7fd195599700 1 -- 172.21.15.33:0/882403793 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd17c0033f0 con 0x7fd190151940 2023-11-06T21:01:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.234+0000 7fd195599700 1 -- 172.21.15.33:0/882403793 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1710410717 0 0) 0x7fd190068f50 con 0x7fd1901514c0 2023-11-06T21:01:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.234+0000 7fd195599700 1 -- 172.21.15.33:0/882403793 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd190154c60 con 0x7fd1901514c0 2023-11-06T21:01:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.234+0000 7fd19659b700 1 -- 172.21.15.33:0/882403793 >> v1:172.21.15.69:6789/0 conn(0x7fd1901514c0 legacy=0x7fd1900bb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.234+0000 7fd19659b700 1 -- 172.21.15.33:0/882403793 >> v1:172.21.15.33:6789/0 conn(0x7fd190151940 legacy=0x7fd19015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.234+0000 7fd19659b700 1 -- 172.21.15.33:0/882403793 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1901520a0 con 0x7fd1901510e0 2023-11-06T21:01:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.234+0000 7fd19659b700 1 --2- 172.21.15.33:0/882403793 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 0x7fd1900a5d00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fd18c009790 tx=0x7fd18c009060).ready entity=mon.2 client_cookie=a58608ccef42293c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.234+0000 7fd195599700 1 -- 172.21.15.33:0/882403793 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00f510 con 0x7fd1901510e0 2023-11-06T21:01:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.234+0000 7fd195599700 1 -- 172.21.15.33:0/882403793 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c00f670 con 0x7fd1901510e0 2023-11-06T21:01:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.234+0000 7fd195599700 1 -- 172.21.15.33:0/882403793 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00f9f0 con 0x7fd1901510e0 2023-11-06T21:01:50.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.235+0000 7fd1987ff700 1 -- 172.21.15.33:0/882403793 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 msgr2=0x7fd1900a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.235+0000 7fd1987ff700 1 --2- 172.21.15.33:0/882403793 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 0x7fd1900a5d00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fd18c009790 tx=0x7fd18c009060).stop 2023-11-06T21:01:50.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.235+0000 7fd1987ff700 1 -- 172.21.15.33:0/882403793 shutdown_connections 2023-11-06T21:01:50.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.235+0000 7fd1987ff700 1 --2- 172.21.15.33:0/882403793 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 0x7fd1900a5d00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.235+0000 7fd1987ff700 1 -- 172.21.15.33:0/882403793 >> 172.21.15.33:0/882403793 conn(0x7fd190062710 msgr2=0x7fd190062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.235+0000 7fd1987ff700 1 -- 172.21.15.33:0/882403793 shutdown_connections 2023-11-06T21:01:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.235+0000 7fd1987ff700 1 -- 172.21.15.33:0/882403793 wait complete. 2023-11-06T21:01:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.237+0000 7fd1987ff700 1 Processor -- start 2023-11-06T21:01:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.237+0000 7fd1987ff700 1 -- start start 2023-11-06T21:01:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd1987ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 0x7fd190159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd1987ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1901514c0 0x7fd19014f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd1987ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd190151940 0x7fd19014d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd1987ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19014fef0 con 0x7fd1901514c0 2023-11-06T21:01:50.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd1987ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19014f6b0 con 0x7fd190151940 2023-11-06T21:01:50.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd1987ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19014f930 con 0x7fd1901510e0 2023-11-06T21:01:50.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd195d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1901514c0 0x7fd19014f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd195d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1901514c0 0x7fd19014f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54838/0 (socket says 172.21.15.33:54838) 2023-11-06T21:01:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd195d9a700 1 -- 172.21.15.33:0/3731059348 learned_addr learned my addr 172.21.15.33:0/3731059348 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd19659b700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 0x7fd190159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd196d9c700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd190151940 0x7fd19014d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.238+0000 7fd195d9a700 1 -- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 msgr2=0x7fd190159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.239+0000 7fd195d9a700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 0x7fd190159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.239+0000 7fd195d9a700 1 -- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd190151940 msgr2=0x7fd19014d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.239+0000 7fd195d9a700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd190151940 0x7fd19014d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.239+0000 7fd195d9a700 1 -- 172.21.15.33:0/3731059348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0092a0 con 0x7fd1901514c0 2023-11-06T21:01:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.239+0000 7fd195d9a700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1901514c0 0x7fd19014f1c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fd18000c4c0 tx=0x7fd180007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:50.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.239+0000 7fd1867fc700 1 -- 172.21.15.33:0/3731059348 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000ce10 con 0x7fd1901514c0 2023-11-06T21:01:50.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.239+0000 7fd1867fc700 1 -- 172.21.15.33:0/3731059348 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18000f470 con 0x7fd1901514c0 2023-11-06T21:01:50.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.239+0000 7fd1867fc700 1 -- 172.21.15.33:0/3731059348 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000f830 con 0x7fd1901514c0 2023-11-06T21:01:50.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.239+0000 7fd1987ff700 1 -- 172.21.15.33:0/3731059348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19014fc10 con 0x7fd1901514c0 2023-11-06T21:01:50.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.239+0000 7fd1987ff700 1 -- 172.21.15.33:0/3731059348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1900c1db0 con 0x7fd1901514c0 2023-11-06T21:01:50.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.240+0000 7fd1987ff700 1 -- 172.21.15.33:0/3731059348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1900c1930 con 0x7fd1901514c0 2023-11-06T21:01:50.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.241+0000 7fd1867fc700 1 -- 172.21.15.33:0/3731059348 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 43) v1 ==== 75679+0+0 (secure 0 0 0) 0x7fd18000fc20 con 0x7fd1901514c0 2023-11-06T21:01:50.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.242+0000 7fd1867fc700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd17405f3b0 0x7fd174061870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.242+0000 7fd1867fc700 1 -- 172.21.15.33:0/3731059348 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fd180079b90 con 0x7fd1901514c0 2023-11-06T21:01:50.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.243+0000 7fd19659b700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd17405f3b0 0x7fd174061870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.243+0000 7fd19659b700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd17405f3b0 0x7fd174061870 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd18c009fd0 tx=0x7fd18c017000).ready entity=mgr.6311 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.246+0000 7fd1867fc700 1 -- 172.21.15.33:0/3731059348 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd180048af0 con 0x7fd1901514c0 2023-11-06T21:01:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.455+0000 7fd1987ff700 1 -- 172.21.15.33:0/3731059348 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd190152380 con 0x7fd17405f3b0 2023-11-06T21:01:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.458+0000 7fd1867fc700 1 -- 172.21.15.33:0/3731059348 <== mgr.6311 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7fd190152380 con 0x7fd17405f3b0 2023-11-06T21:01:50.457 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:01:50.458 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T21:00:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-11-06T21:01:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 -- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd17405f3b0 msgr2=0x7fd174061870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd17405f3b0 0x7fd174061870 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd18c009fd0 tx=0x7fd18c017000).stop 2023-11-06T21:01:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 -- 172.21.15.33:0/3731059348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1901514c0 msgr2=0x7fd19014f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1901514c0 0x7fd19014f1c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fd18000c4c0 tx=0x7fd180007c10).stop 2023-11-06T21:01:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 -- 172.21.15.33:0/3731059348 shutdown_connections 2023-11-06T21:01:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd17405f3b0 0x7fd174061870 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1901510e0 0x7fd190159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd190151940 0x7fd19014d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 --2- 172.21.15.33:0/3731059348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1901514c0 0x7fd19014f1c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 -- 172.21.15.33:0/3731059348 >> 172.21.15.33:0/3731059348 conn(0x7fd190062710 msgr2=0x7fd1901532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 -- 172.21.15.33:0/3731059348 shutdown_connections 2023-11-06T21:01:50.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.460+0000 7fd1987ff700 1 -- 172.21.15.33:0/3731059348 wait complete. 2023-11-06T21:01:50.477 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T21:00:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-11-06T21:01:50.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /bad-path 2023-11-06T21:01:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.678+0000 7f4ed65ff700 1 Processor -- start 2023-11-06T21:01:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.678+0000 7f4ed65ff700 1 -- start start 2023-11-06T21:01:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.680+0000 7f4ed65ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ed01510e0 0x7f4ed00a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.680+0000 7f4ed65ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ed01514c0 0x7f4ed00bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.680+0000 7f4ed65ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed0154c60 con 0x7f4ed0151940 2023-11-06T21:01:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.680+0000 7f4ed65ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed0068f50 con 0x7f4ed01514c0 2023-11-06T21:01:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.680+0000 7f4ed65ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0151dc0 con 0x7f4ed01510e0 2023-11-06T21:01:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.680+0000 7f4ecf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ed01514c0 0x7f4ed00bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.680+0000 7f4ecf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ed01514c0 0x7f4ed00bb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54852/0 (socket says 172.21.15.33:54852) 2023-11-06T21:01:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.680+0000 7f4ed4b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4ed0151940 0x7f4ed0069430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56368/0 (socket says 172.21.15.33:56368) 2023-11-06T21:01:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.680+0000 7f4ecf7fe700 1 -- 172.21.15.33:0/3531696892 learned_addr learned my addr 172.21.15.33:0/3531696892 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:50.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.681+0000 7f4ecffff700 1 --2- 172.21.15.33:0/3531696892 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ed01510e0 0x7f4ed00a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.681+0000 7f4ecf7fe700 1 -- 172.21.15.33:0/3531696892 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ed01510e0 msgr2=0x7f4ed00a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:01:50.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.681+0000 7f4ecf7fe700 1 --2- 172.21.15.33:0/3531696892 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ed01510e0 0x7f4ed00a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.681+0000 7f4ecf7fe700 1 -- 172.21.15.33:0/3531696892 >> v1:172.21.15.33:6789/0 conn(0x7f4ed0151940 legacy=0x7f4ed0069430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.681+0000 7f4eceffd700 1 -- 172.21.15.33:0/3531696892 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33219782 0 0) 0x7f4ed0154c60 con 0x7f4ed0151940 2023-11-06T21:01:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.681+0000 7f4ecf7fe700 1 -- 172.21.15.33:0/3531696892 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed01520a0 con 0x7f4ed01514c0 2023-11-06T21:01:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.681+0000 7f4ecf7fe700 1 --2- 172.21.15.33:0/3531696892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01514c0 0x7f4ed00bb980 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f4ec4006fd0 tx=0x7f4ec40090f0).ready entity=mon.0 client_cookie=c2cb2ec147ab9f16 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.681+0000 7f4eceffd700 1 -- 172.21.15.33:0/3531696892 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400b600 con 0x7f4ed01514c0 2023-11-06T21:01:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.681+0000 7f4eceffd700 1 -- 172.21.15.33:0/3531696892 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec400b760 con 0x7f4ed01514c0 2023-11-06T21:01:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.681+0000 7f4eceffd700 1 -- 172.21.15.33:0/3531696892 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400bae0 con 0x7f4ed01514c0 2023-11-06T21:01:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.682+0000 7f4ed65ff700 1 -- 172.21.15.33:0/3531696892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01514c0 msgr2=0x7f4ed00bb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.682+0000 7f4ed65ff700 1 --2- 172.21.15.33:0/3531696892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01514c0 0x7f4ed00bb980 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f4ec4006fd0 tx=0x7f4ec40090f0).stop 2023-11-06T21:01:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.682+0000 7f4ed65ff700 1 -- 172.21.15.33:0/3531696892 shutdown_connections 2023-11-06T21:01:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.682+0000 7f4ed65ff700 1 --2- 172.21.15.33:0/3531696892 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ed01510e0 0x7f4ed00a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.682+0000 7f4ed65ff700 1 --2- 172.21.15.33:0/3531696892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01514c0 0x7f4ed00bb980 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.682+0000 7f4ed65ff700 1 -- 172.21.15.33:0/3531696892 >> 172.21.15.33:0/3531696892 conn(0x7f4ed0062710 msgr2=0x7f4ed0062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:50.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.682+0000 7f4ed65ff700 1 -- 172.21.15.33:0/3531696892 shutdown_connections 2023-11-06T21:01:50.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.682+0000 7f4ed65ff700 1 -- 172.21.15.33:0/3531696892 wait complete. 2023-11-06T21:01:50.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.683+0000 7f4ed65ff700 1 Processor -- start 2023-11-06T21:01:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.684+0000 7f4ed65ff700 1 -- start start 2023-11-06T21:01:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.684+0000 7f4ed65ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01510e0 0x7f4ed0165210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.684+0000 7f4ed65ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ed01514c0 0x7f4ed0162da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ed65ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed0151940 0x7f4ed01634f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ed65ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0165700 con 0x7f4ed01510e0 2023-11-06T21:01:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ed65ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0163290 con 0x7f4ed0151940 2023-11-06T21:01:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ed65ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed01639e0 con 0x7f4ed01514c0 2023-11-06T21:01:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ecffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01510e0 0x7f4ed0165210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ecffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01510e0 0x7f4ed0165210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54854/0 (socket says 172.21.15.33:54854) 2023-11-06T21:01:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ecffff700 1 -- 172.21.15.33:0/4192824766 learned_addr learned my addr 172.21.15.33:0/4192824766 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ecf7fe700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ed01514c0 0x7f4ed0162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ed4b9c700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed0151940 0x7f4ed01634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ecffff700 1 -- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ed01514c0 msgr2=0x7f4ed0162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ecffff700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ed01514c0 0x7f4ed0162da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ecffff700 1 -- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed0151940 msgr2=0x7f4ed01634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ecffff700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed0151940 0x7f4ed01634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ecffff700 1 -- 172.21.15.33:0/4192824766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4009370 con 0x7f4ed01510e0 2023-11-06T21:01:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ed4b9c700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed0151940 0x7f4ed01634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:01:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.685+0000 7f4ecffff700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01510e0 0x7f4ed0165210 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f4ec000d890 tx=0x7f4ec0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:50.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.686+0000 7f4eb7fff700 1 -- 172.21.15.33:0/4192824766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00125f0 con 0x7f4ed01510e0 2023-11-06T21:01:50.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.686+0000 7f4eb7fff700 1 -- 172.21.15.33:0/4192824766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec0012750 con 0x7f4ed01510e0 2023-11-06T21:01:50.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.686+0000 7f4ed65ff700 1 -- 172.21.15.33:0/4192824766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed00a8540 con 0x7f4ed01510e0 2023-11-06T21:01:50.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.686+0000 7f4eb7fff700 1 -- 172.21.15.33:0/4192824766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0012ad0 con 0x7f4ed01510e0 2023-11-06T21:01:50.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.686+0000 7f4ed65ff700 1 -- 172.21.15.33:0/4192824766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed00be4e0 con 0x7f4ed01510e0 2023-11-06T21:01:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.687+0000 7f4ed65ff700 1 -- 172.21.15.33:0/4192824766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed0152380 con 0x7f4ed01510e0 2023-11-06T21:01:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.688+0000 7f4eb7fff700 1 -- 172.21.15.33:0/4192824766 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 43) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f4ec0013480 con 0x7f4ed01510e0 2023-11-06T21:01:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.688+0000 7f4eb7fff700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ebc05f3e0 0x7f4ebc0618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.688+0000 7f4eb7fff700 1 -- 172.21.15.33:0/4192824766 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f4ec007a540 con 0x7f4ed01510e0 2023-11-06T21:01:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.689+0000 7f4ecf7fe700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ebc05f3e0 0x7f4ebc0618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:50.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.690+0000 7f4ecf7fe700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ebc05f3e0 0x7f4ebc0618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4ec4006fd0 tx=0x7f4ec4017000).ready entity=mgr.6311 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:50.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.692+0000 7f4eb7fff700 1 -- 172.21.15.33:0/4192824766 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4ec004b290 con 0x7f4ed01510e0 2023-11-06T21:01:50.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.900+0000 7f4ed65ff700 1 -- 172.21.15.33:0/4192824766 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ed0150190 con 0x7f4ebc05f3e0 2023-11-06T21:01:50.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.906+0000 7f4eb7fff700 1 -- 172.21.15.33:0/4192824766 <== mgr.6311 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f4ed0150190 con 0x7f4ebc05f3e0 2023-11-06T21:01:50.905 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /bad-path 2023-11-06T21:01:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 -- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ebc05f3e0 msgr2=0x7f4ebc0618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ebc05f3e0 0x7f4ebc0618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4ec4006fd0 tx=0x7f4ec4017000).stop 2023-11-06T21:01:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 -- 172.21.15.33:0/4192824766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01510e0 msgr2=0x7f4ed0165210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01510e0 0x7f4ed0165210 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f4ec000d890 tx=0x7f4ec0008040).stop 2023-11-06T21:01:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 -- 172.21.15.33:0/4192824766 shutdown_connections 2023-11-06T21:01:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4ebc05f3e0 0x7f4ebc0618a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed0151940 0x7f4ed01634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed01510e0 0x7f4ed0165210 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 --2- 172.21.15.33:0/4192824766 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4ed01514c0 0x7f4ed0162da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 -- 172.21.15.33:0/4192824766 >> 172.21.15.33:0/4192824766 conn(0x7f4ed0062710 msgr2=0x7f4ed00c3500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 -- 172.21.15.33:0/4192824766 shutdown_connections 2023-11-06T21:01:50.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:50.908+0000 7f4ed65ff700 1 -- 172.21.15.33:0/4192824766 wait complete. 2023-11-06T21:01:50.925 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-11-06T21:01:50.925 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T21:01:51.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.124+0000 7f16181ff700 1 Processor -- start 2023-11-06T21:01:51.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.124+0000 7f16181ff700 1 -- start start 2023-11-06T21:01:51.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.125+0000 7f16181ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1610066620 0x7f16100c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.125+0000 7f16181ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1610066a00 0x7f1610158780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.125+0000 7f16181ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16100a8330 con 0x7f1610067ad0 2023-11-06T21:01:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.125+0000 7f16181ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16100be210 con 0x7f1610066a00 2023-11-06T21:01:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.125+0000 7f16181ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1610158c70 con 0x7f1610066620 2023-11-06T21:01:51.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.125+0000 7f161579a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1610066a00 0x7f1610158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f161679c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1610067ad0 0x7f1610067f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56376/0 (socket says 172.21.15.33:56376) 2023-11-06T21:01:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f161679c700 1 -- 172.21.15.33:0/2186647575 learned_addr learned my addr 172.21.15.33:0/2186647575 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f161579a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1610066a00 0x7f1610158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54868/0 (socket says 172.21.15.33:54868) 2023-11-06T21:01:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f1615f9b700 1 --2- 172.21.15.33:0/2186647575 >> v2:172.21.15.69:3300/0 conn(0x7f1610066620 0x7f16100c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f161579a700 1 -- 172.21.15.33:0/2186647575 >> v2:172.21.15.69:3300/0 conn(0x7f1610066620 msgr2=0x7f16100c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:01:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f161579a700 1 --2- 172.21.15.33:0/2186647575 >> v2:172.21.15.69:3300/0 conn(0x7f1610066620 0x7f16100c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f161579a700 1 -- 172.21.15.33:0/2186647575 >> v1:172.21.15.33:6789/0 conn(0x7f1610067ad0 legacy=0x7f1610067f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f161579a700 1 -- 172.21.15.33:0/2186647575 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16100bdc50 con 0x7f1610066a00 2023-11-06T21:01:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f1614f99700 1 -- 172.21.15.33:0/2186647575 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298772907 0 0) 0x7f16100a8330 con 0x7f1610067ad0 2023-11-06T21:01:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f161579a700 1 --2- 172.21.15.33:0/2186647575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066a00 0x7f1610158780 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f16100631e0 tx=0x7f1600009060).ready entity=mon.0 client_cookie=5f0318a40c82b51c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f1614f99700 1 -- 172.21.15.33:0/2186647575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160000b580 con 0x7f1610066a00 2023-11-06T21:01:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f1614f99700 1 -- 172.21.15.33:0/2186647575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160000b6e0 con 0x7f1610066a00 2023-11-06T21:01:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.126+0000 7f1614f99700 1 -- 172.21.15.33:0/2186647575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160000ba60 con 0x7f1610066a00 2023-11-06T21:01:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.127+0000 7f16181ff700 1 -- 172.21.15.33:0/2186647575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066a00 msgr2=0x7f1610158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.127+0000 7f16181ff700 1 --2- 172.21.15.33:0/2186647575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066a00 0x7f1610158780 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f16100631e0 tx=0x7f1600009060).stop 2023-11-06T21:01:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.127+0000 7f16181ff700 1 -- 172.21.15.33:0/2186647575 shutdown_connections 2023-11-06T21:01:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.127+0000 7f16181ff700 1 --2- 172.21.15.33:0/2186647575 >> v2:172.21.15.69:3300/0 conn(0x7f1610066620 0x7f16100c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.127+0000 7f16181ff700 1 --2- 172.21.15.33:0/2186647575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066a00 0x7f1610158780 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.127+0000 7f16181ff700 1 -- 172.21.15.33:0/2186647575 >> 172.21.15.33:0/2186647575 conn(0x7f16100c8c50 msgr2=0x7f16100c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:51.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.127+0000 7f16181ff700 1 -- 172.21.15.33:0/2186647575 shutdown_connections 2023-11-06T21:01:51.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.127+0000 7f16181ff700 1 -- 172.21.15.33:0/2186647575 wait complete. 2023-11-06T21:01:51.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.129+0000 7f16181ff700 1 Processor -- start 2023-11-06T21:01:51.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.129+0000 7f16181ff700 1 -- start start 2023-11-06T21:01:51.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f16181ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066620 0x7f16100a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:51.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f16181ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1610066a00 0x7f16100a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:51.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f16181ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1610067ad0 0x7f16100ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f16181ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1610143000 con 0x7f1610066620 2023-11-06T21:01:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f16181ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100ae610 con 0x7f1610067ad0 2023-11-06T21:01:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f16181ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16100a5450 con 0x7f1610066a00 2023-11-06T21:01:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f1615f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066620 0x7f16100a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f1615f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066620 0x7f16100a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54884/0 (socket says 172.21.15.33:54884) 2023-11-06T21:01:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f1615f9b700 1 -- 172.21.15.33:0/2460212742 learned_addr learned my addr 172.21.15.33:0/2460212742 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f161679c700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1610067ad0 0x7f16100ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f161579a700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1610066a00 0x7f16100a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f1615f9b700 1 -- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1610066a00 msgr2=0x7f16100a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f1615f9b700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1610066a00 0x7f16100a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f1615f9b700 1 -- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1610067ad0 msgr2=0x7f16100ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f1615f9b700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1610067ad0 0x7f16100ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f1615f9b700 1 -- 172.21.15.33:0/2460212742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16000092a0 con 0x7f1610066620 2023-11-06T21:01:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.130+0000 7f1615f9b700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066620 0x7f16100a48e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f160c00d890 tx=0x7f160c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:51.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.131+0000 7f1605ffb700 1 -- 172.21.15.33:0/2460212742 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c0125f0 con 0x7f1610066620 2023-11-06T21:01:51.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.131+0000 7f1605ffb700 1 -- 172.21.15.33:0/2460212742 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c012750 con 0x7f1610066620 2023-11-06T21:01:51.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.131+0000 7f16181ff700 1 -- 172.21.15.33:0/2460212742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16100a5730 con 0x7f1610066620 2023-11-06T21:01:51.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.131+0000 7f1605ffb700 1 -- 172.21.15.33:0/2460212742 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c012ad0 con 0x7f1610066620 2023-11-06T21:01:51.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.131+0000 7f16181ff700 1 -- 172.21.15.33:0/2460212742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16100a5cf0 con 0x7f1610066620 2023-11-06T21:01:51.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.132+0000 7f16181ff700 1 -- 172.21.15.33:0/2460212742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1610068c00 con 0x7f1610066620 2023-11-06T21:01:51.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.133+0000 7f1605ffb700 1 -- 172.21.15.33:0/2460212742 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 43) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f160c013480 con 0x7f1610066620 2023-11-06T21:01:51.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.134+0000 7f1605ffb700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f15f405f3b0 0x7f15f4061870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:51.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.134+0000 7f1605ffb700 1 -- 172.21.15.33:0/2460212742 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f160c07a6c0 con 0x7f1610066620 2023-11-06T21:01:51.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.134+0000 7f161579a700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f15f405f3b0 0x7f15f4061870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.135+0000 7f161579a700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f15f405f3b0 0x7f15f4061870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f1600009fd0 tx=0x7f1600017000).ready entity=mgr.6311 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.138+0000 7f1605ffb700 1 -- 172.21.15.33:0/2460212742 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f160c04b390 con 0x7f1610066620 2023-11-06T21:01:51.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:51.368+0000 7f16181ff700 1 -- 172.21.15.33:0/2460212742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f16100639f0 con 0x7f1610066620 2023-11-06T21:01:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.191+0000 7f1605ffb700 1 -- 172.21.15.33:0/2460212742 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f160c077ef0 con 0x7f1610066620 2023-11-06T21:01:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.193+0000 7f1605ffb700 1 -- 172.21.15.33:0/2460212742 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v44) v1 ==== 93+0+0 (secure 0 0 0) 0x7f160c072c50 con 0x7f1610066620 2023-11-06T21:01:52.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.197+0000 7f16181ff700 1 -- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f15f405f3b0 msgr2=0x7f15f4061870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:52.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.197+0000 7f16181ff700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f15f405f3b0 0x7f15f4061870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f1600009fd0 tx=0x7f1600017000).stop 2023-11-06T21:01:52.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.197+0000 7f16181ff700 1 -- 172.21.15.33:0/2460212742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066620 msgr2=0x7f16100a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:52.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.197+0000 7f16181ff700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066620 0x7f16100a48e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f160c00d890 tx=0x7f160c008040).stop 2023-11-06T21:01:52.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.197+0000 7f16181ff700 1 -- 172.21.15.33:0/2460212742 shutdown_connections 2023-11-06T21:01:52.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.197+0000 7f16181ff700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f15f405f3b0 0x7f15f4061870 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:52.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.197+0000 7f16181ff700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1610067ad0 0x7f16100ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:52.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.197+0000 7f16181ff700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1610066620 0x7f16100a48e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:52.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.197+0000 7f16181ff700 1 --2- 172.21.15.33:0/2460212742 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1610066a00 0x7f16100a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:52.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.197+0000 7f16181ff700 1 -- 172.21.15.33:0/2460212742 >> 172.21.15.33:0/2460212742 conn(0x7f16100c8c50 msgr2=0x7f1610144680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:52.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.198+0000 7f16181ff700 1 -- 172.21.15.33:0/2460212742 shutdown_connections 2023-11-06T21:01:52.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.198+0000 7f16181ff700 1 -- 172.21.15.33:0/2460212742 wait complete. 2023-11-06T21:01:52.229 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T21:01:52.230 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:01:52.329 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:01:52.419 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T21:01:52.419 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T21:01:52.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:01:52.420 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T21:01:52.421 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:52.421+0000 7f34994a2680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:01:52.426 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:52.427+0000 7fae27539680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:01:52.501 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T21:01:52.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:01:52.503 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T21:01:52.504 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:52.501+0000 7f34994a2680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:01:52.510 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:52.511+0000 7fae27539680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:01:52.559 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:01:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.757+0000 7f99799ff700 1 Processor -- start 2023-11-06T21:01:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.757+0000 7f99799ff700 1 -- start start 2023-11-06T21:01:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.759+0000 7f99799ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9974154760 0x7f9974063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.759+0000 7f99799ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740c31f0 con 0x7f997405e1b0 2023-11-06T21:01:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.759+0000 7f99799ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740c2920 con 0x7f9974063140 2023-11-06T21:01:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.759+0000 7f99799ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997406a0f0 con 0x7f9974154760 2023-11-06T21:01:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.759+0000 7f99727fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f997405e1b0 0x7f9974154050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56398/0 (socket says 172.21.15.33:56398) 2023-11-06T21:01:52.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.759+0000 7f99727fc700 1 -- 172.21.15.33:0/3336448877 learned_addr learned my addr 172.21.15.33:0/3336448877 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:52.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.759+0000 7f99737fe700 1 --2- 172.21.15.33:0/3336448877 >> v2:172.21.15.69:3300/0 conn(0x7f9974154760 0x7f9974063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:52.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f9971ffb700 1 -- 172.21.15.33:0/3336448877 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857235787 0 0) 0x7f99740c31f0 con 0x7f997405e1b0 2023-11-06T21:01:52.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f9971ffb700 1 -- 172.21.15.33:0/3336448877 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f995c0033f0 con 0x7f997405e1b0 2023-11-06T21:01:52.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f9971ffb700 1 -- 172.21.15.33:0/3336448877 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047068837 0 0) 0x7f99740c2920 con 0x7f9974063140 2023-11-06T21:01:52.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f9971ffb700 1 -- 172.21.15.33:0/3336448877 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99740c31f0 con 0x7f9974063140 2023-11-06T21:01:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f99737fe700 1 -- 172.21.15.33:0/3336448877 >> v1:172.21.15.69:6789/0 conn(0x7f9974063140 legacy=0x7f99740c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f99737fe700 1 -- 172.21.15.33:0/3336448877 >> v1:172.21.15.33:6789/0 conn(0x7f997405e1b0 legacy=0x7f9974154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f99737fe700 1 -- 172.21.15.33:0/3336448877 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997405e640 con 0x7f9974154760 2023-11-06T21:01:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f99737fe700 1 --2- 172.21.15.33:0/3336448877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974154760 0x7f9974063520 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f996c009e60 tx=0x7f996c009780).ready entity=mon.1 client_cookie=8f8a26a1afb62df6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f9971ffb700 1 -- 172.21.15.33:0/3336448877 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00e070 con 0x7f9974154760 2023-11-06T21:01:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f9971ffb700 1 -- 172.21.15.33:0/3336448877 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00bab0 con 0x7f9974154760 2023-11-06T21:01:52.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.760+0000 7f9971ffb700 1 -- 172.21.15.33:0/3336448877 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00be30 con 0x7f9974154760 2023-11-06T21:01:52.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.761+0000 7f99799ff700 1 -- 172.21.15.33:0/3336448877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974154760 msgr2=0x7f9974063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:52.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.761+0000 7f99799ff700 1 --2- 172.21.15.33:0/3336448877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974154760 0x7f9974063520 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f996c009e60 tx=0x7f996c009780).stop 2023-11-06T21:01:52.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.761+0000 7f99799ff700 1 -- 172.21.15.33:0/3336448877 shutdown_connections 2023-11-06T21:01:52.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.761+0000 7f99799ff700 1 --2- 172.21.15.33:0/3336448877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974154760 0x7f9974063520 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.761+0000 7f99799ff700 1 -- 172.21.15.33:0/3336448877 >> 172.21.15.33:0/3336448877 conn(0x7f99740c7d60 msgr2=0x7f99740c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.761+0000 7f99799ff700 1 -- 172.21.15.33:0/3336448877 shutdown_connections 2023-11-06T21:01:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.761+0000 7f99799ff700 1 -- 172.21.15.33:0/3336448877 wait complete. 2023-11-06T21:01:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.763+0000 7f99799ff700 1 Processor -- start 2023-11-06T21:01:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.763+0000 7f99799ff700 1 -- start start 2023-11-06T21:01:52.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f99799ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9974063140 0x7f99740a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:52.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f99799ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974154760 0x7f99741501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:52.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f99799ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9974164c20 0x7f99741650a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:52.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f99799ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974164790 con 0x7f9974164c20 2023-11-06T21:01:52.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f99799ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974150690 con 0x7f9974154760 2023-11-06T21:01:52.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f99799ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9974164430 con 0x7f9974063140 2023-11-06T21:01:52.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f99737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9974164c20 0x7f99741650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:52.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f9972ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9974063140 0x7f99740a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:52.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f99727fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974154760 0x7f99741501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f9972ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9974063140 0x7f99740a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:48878/0 (socket says 172.21.15.33:48878) 2023-11-06T21:01:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.764+0000 7f9972ffd700 1 -- 172.21.15.33:0/818584780 learned_addr learned my addr 172.21.15.33:0/818584780 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f99727fc700 1 -- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9974063140 msgr2=0x7f99740a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f99727fc700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9974063140 0x7f99740a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f99727fc700 1 -- 172.21.15.33:0/818584780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9974164c20 msgr2=0x7f99741650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f99727fc700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9974164c20 0x7f99741650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f99727fc700 1 -- 172.21.15.33:0/818584780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c009a20 con 0x7f9974154760 2023-11-06T21:01:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f99727fc700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974154760 0x7f99741501a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f997405f2f0 tx=0x7f9968007bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f995affd700 1 -- 172.21.15.33:0/818584780 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99680070d0 con 0x7f9974154760 2023-11-06T21:01:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f995affd700 1 -- 172.21.15.33:0/818584780 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9968007230 con 0x7f9974154760 2023-11-06T21:01:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f995affd700 1 -- 172.21.15.33:0/818584780 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99680075b0 con 0x7f9974154760 2023-11-06T21:01:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f99799ff700 1 -- 172.21.15.33:0/818584780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997415b050 con 0x7f9974154760 2023-11-06T21:01:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.765+0000 7f99799ff700 1 -- 172.21.15.33:0/818584780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99741687e0 con 0x7f9974154760 2023-11-06T21:01:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.766+0000 7f99799ff700 1 -- 172.21.15.33:0/818584780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9974164940 con 0x7f9974154760 2023-11-06T21:01:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.767+0000 7f995affd700 1 -- 172.21.15.33:0/818584780 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f99680059b0 con 0x7f9974154760 2023-11-06T21:01:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.768+0000 7f995affd700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f996005f360 0x7f9960061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.768+0000 7f995affd700 1 -- 172.21.15.33:0/818584780 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f9968079e10 con 0x7f9974154760 2023-11-06T21:01:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.768+0000 7f9972ffd700 1 -- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f996005f360 msgr2=0x7f9960061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.768+0000 7f9972ffd700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f996005f360 0x7f9960061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.772+0000 7f995affd700 1 -- 172.21.15.33:0/818584780 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9974164940 con 0x7f9974154760 2023-11-06T21:01:52.828 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:52.828+0000 7f34994a2680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:01:52.863 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:52.864+0000 7fae27539680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:01:52.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.969+0000 7f9972ffd700 1 -- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f996005f360 msgr2=0x7f9960061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:52.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:52.969+0000 7f9972ffd700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f996005f360 0x7f9960061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:01:53.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.001+0000 7f99799ff700 1 -- 172.21.15.33:0/818584780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9974164940 con 0x7f9974154760 2023-11-06T21:01:53.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.002+0000 7f995affd700 1 -- 172.21.15.33:0/818584780 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4485 (secure 0 0 0) 0x7f9974164940 con 0x7f9974154760 2023-11-06T21:01:53.002 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:01:53.002 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":45,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2108889615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2108889615},{"type":"v1","addr":"172.21.15.33:6834","nonce":2108889615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/836821298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":836821298},{"type":"v1","addr":"172.21.15.69:6836","nonce":836821298}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6838,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2007995447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2007995447},{"type":"v1","addr":"172.21.15.33:6835","nonce":2007995447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:00:33.962914+0000","modified":"2023-11-06T21:00:36.976166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6844":{"gid":6844,"name":"d","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/396991049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":396991049},{"type":"v1","addr":"172.21.15.69:6837","nonce":396991049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-06T21:01:53.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.005+0000 7f99799ff700 1 -- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f996005f360 msgr2=0x7f9960061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:53.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.005+0000 7f99799ff700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f996005f360 0x7f9960061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:53.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.005+0000 7f99799ff700 1 -- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974154760 msgr2=0x7f99741501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.005+0000 7f99799ff700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974154760 0x7f99741501a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f997405f2f0 tx=0x7f9968007bf0).stop 2023-11-06T21:01:53.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.005+0000 7f99799ff700 1 -- 172.21.15.33:0/818584780 shutdown_connections 2023-11-06T21:01:53.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.006+0000 7f99799ff700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f996005f360 0x7f9960061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:53.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.006+0000 7f99799ff700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9974063140 0x7f99740a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.006+0000 7f99799ff700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9974164c20 0x7f99741650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.006+0000 7f99799ff700 1 --2- 172.21.15.33:0/818584780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974154760 0x7f99741501a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.006+0000 7f99799ff700 1 -- 172.21.15.33:0/818584780 >> 172.21.15.33:0/818584780 conn(0x7f99740c7d60 msgr2=0x7f99740ac490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:53.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.006+0000 7f99799ff700 1 -- 172.21.15.33:0/818584780 shutdown_connections 2023-11-06T21:01:53.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.006+0000 7f99799ff700 1 -- 172.21.15.33:0/818584780 wait complete. 2023-11-06T21:01:53.008 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 45 2023-11-06T21:01:53.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:01:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.241+0000 7f18813ff700 1 Processor -- start 2023-11-06T21:01:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.242+0000 7f18813ff700 1 -- start start 2023-11-06T21:01:53.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.244+0000 7f18813ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f187c14ae80 0x7f187c0bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.244+0000 7f18813ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f187c064b10 0x7f187c0c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.244+0000 7f18813ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187c0bca70 con 0x7f187c0bc5c0 2023-11-06T21:01:53.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.244+0000 7f18813ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187c064f80 con 0x7f187c064b10 2023-11-06T21:01:53.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.244+0000 7f18813ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187c14b260 con 0x7f187c14ae80 2023-11-06T21:01:53.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.244+0000 7f187b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f187c064b10 0x7f187c0c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:53.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.244+0000 7f187b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f187c064b10 0x7f187c0c1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54916/0 (socket says 172.21.15.33:54916) 2023-11-06T21:01:53.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.244+0000 7f187b7fe700 1 -- 172.21.15.33:0/370202824 learned_addr learned my addr 172.21.15.33:0/370202824 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:53.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.244+0000 7f187affd700 1 --2- 172.21.15.33:0/370202824 >> v2:172.21.15.69:3300/0 conn(0x7f187c14ae80 0x7f187c0bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:53.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f1879ffb700 1 -- 172.21.15.33:0/370202824 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616199478 0 0) 0x7f1870001110 con 0x7f187c0bc5c0 2023-11-06T21:01:53.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f1879ffb700 1 -- 172.21.15.33:0/370202824 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18640033f0 con 0x7f187c0bc5c0 2023-11-06T21:01:53.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f187b7fe700 1 -- 172.21.15.33:0/370202824 >> v2:172.21.15.69:3300/0 conn(0x7f187c14ae80 msgr2=0x7f187c0bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:01:53.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f187b7fe700 1 --2- 172.21.15.33:0/370202824 >> v2:172.21.15.69:3300/0 conn(0x7f187c14ae80 0x7f187c0bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f187b7fe700 1 -- 172.21.15.33:0/370202824 >> v1:172.21.15.69:6789/0 conn(0x7f187c0bc5c0 legacy=0x7f187c064400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f187b7fe700 1 -- 172.21.15.33:0/370202824 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0c1940 con 0x7f187c064b10 2023-11-06T21:01:53.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f187affd700 1 --2- 172.21.15.33:0/370202824 >> v2:172.21.15.69:3300/0 conn(0x7f187c14ae80 0x7f187c0bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:01:53.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f187b7fe700 1 --2- 172.21.15.33:0/370202824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c064b10 0x7f187c0c1dd0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f187c067d10 tx=0x7f1874009840).ready entity=mon.0 client_cookie=42df8739b6d0ce50 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:53.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f1879ffb700 1 -- 172.21.15.33:0/370202824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187400e070 con 0x7f187c064b10 2023-11-06T21:01:53.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f1879ffb700 1 -- 172.21.15.33:0/370202824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187400b970 con 0x7f187c064b10 2023-11-06T21:01:53.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.245+0000 7f1879ffb700 1 -- 172.21.15.33:0/370202824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187400bcf0 con 0x7f187c064b10 2023-11-06T21:01:53.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.246+0000 7f18813ff700 1 -- 172.21.15.33:0/370202824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c064b10 msgr2=0x7f187c0c1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.246+0000 7f18813ff700 1 --2- 172.21.15.33:0/370202824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c064b10 0x7f187c0c1dd0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f187c067d10 tx=0x7f1874009840).stop 2023-11-06T21:01:53.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.246+0000 7f18813ff700 1 -- 172.21.15.33:0/370202824 shutdown_connections 2023-11-06T21:01:53.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.246+0000 7f18813ff700 1 --2- 172.21.15.33:0/370202824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c064b10 0x7f187c0c1dd0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.246+0000 7f18813ff700 1 --2- 172.21.15.33:0/370202824 >> v2:172.21.15.69:3300/0 conn(0x7f187c14ae80 0x7f187c0bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.246+0000 7f18813ff700 1 -- 172.21.15.33:0/370202824 >> 172.21.15.33:0/370202824 conn(0x7f187c062710 msgr2=0x7f187c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:53.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.246+0000 7f18813ff700 1 -- 172.21.15.33:0/370202824 shutdown_connections 2023-11-06T21:01:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.246+0000 7f18813ff700 1 -- 172.21.15.33:0/370202824 wait complete. 2023-11-06T21:01:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.248+0000 7f18813ff700 1 Processor -- start 2023-11-06T21:01:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.248+0000 7f18813ff700 1 -- start start 2023-11-06T21:01:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f18813ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c064b10 0x7f187c1497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f18813ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c0bc5c0 0x7f187c069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f18813ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f187c14ae80 0x7f187c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f18813ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c06a310 con 0x7f187c0bc5c0 2023-11-06T21:01:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f18813ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c14ce30 con 0x7f187c064b10 2023-11-06T21:01:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f18813ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c14c220 con 0x7f187c14ae80 2023-11-06T21:01:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f187a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c0bc5c0 0x7f187c069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f187affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c064b10 0x7f187c1497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f187a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c0bc5c0 0x7f187c069930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54920/0 (socket says 172.21.15.33:54920) 2023-11-06T21:01:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f187a7fc700 1 -- 172.21.15.33:0/608085758 learned_addr learned my addr 172.21.15.33:0/608085758 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.249+0000 7f187b7fe700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f187c14ae80 0x7f187c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f187affd700 1 -- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f187c14ae80 msgr2=0x7f187c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f187affd700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f187c14ae80 0x7f187c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f187affd700 1 -- 172.21.15.33:0/608085758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c0bc5c0 msgr2=0x7f187c069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f187affd700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c0bc5c0 0x7f187c069930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f187affd700 1 -- 172.21.15.33:0/608085758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c0092a0 con 0x7f187c064b10 2023-11-06T21:01:53.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f187a7fc700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c0bc5c0 0x7f187c069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:01:53.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f187affd700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c064b10 0x7f187c1497d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f186c00e880 tx=0x7f186c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f1862ffd700 1 -- 172.21.15.33:0/608085758 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c012600 con 0x7f187c064b10 2023-11-06T21:01:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f1862ffd700 1 -- 172.21.15.33:0/608085758 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186c012760 con 0x7f187c064b10 2023-11-06T21:01:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f18813ff700 1 -- 172.21.15.33:0/608085758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1874009ae0 con 0x7f187c064b10 2023-11-06T21:01:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f1862ffd700 1 -- 172.21.15.33:0/608085758 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c012ae0 con 0x7f187c064b10 2023-11-06T21:01:53.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.250+0000 7f18813ff700 1 -- 172.21.15.33:0/608085758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c14c860 con 0x7f187c064b10 2023-11-06T21:01:53.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.252+0000 7f18813ff700 1 -- 172.21.15.33:0/608085758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187c14d110 con 0x7f187c064b10 2023-11-06T21:01:53.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.252+0000 7f1862ffd700 1 -- 172.21.15.33:0/608085758 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f186c01b400 con 0x7f187c064b10 2023-11-06T21:01:53.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.253+0000 7f1862ffd700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f186805f390 0x7f1868061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.253+0000 7f1862ffd700 1 -- 172.21.15.33:0/608085758 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f186c07abb0 con 0x7f187c064b10 2023-11-06T21:01:53.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.253+0000 7f187a7fc700 1 -- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f186805f390 msgr2=0x7f1868061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:53.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.253+0000 7f187a7fc700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f186805f390 0x7f1868061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.257+0000 7f1862ffd700 1 -- 172.21.15.33:0/608085758 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f186c04b880 con 0x7f187c064b10 2023-11-06T21:01:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.440+0000 7f18813ff700 1 -- 172.21.15.33:0/608085758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f187c14e470 con 0x7f187c064b10 2023-11-06T21:01:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.442+0000 7f1862ffd700 1 -- 172.21.15.33:0/608085758 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+17792 (secure 0 0 0) 0x7f186c016070 con 0x7f187c064b10 2023-11-06T21:01:53.442 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:01:53.442 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":117,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:01:50.172076+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:00:31.623152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T21:00:32.950772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.11","mappings":[{"from":1,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.16","mappings":[{"from":4,"to":3}]},{"pgid":"14.25","mappings":[{"from":7,"to":3}]},{"pgid":"14.27","mappings":[{"from":1,"to":3}]},{"pgid":"14.35","mappings":[{"from":7,"to":0}]},{"pgid":"14.3b","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2951161837":"2023-11-07T21:00:48.500569+0000","172.21.15.69:6832/114912":"2023-11-07T21:00:48.500569+0000","172.21.15.69:0/443852199":"2023-11-07T21:00:48.500569+0000","172.21.15.69:6833/114912":"2023-11-07T21:00:48.500569+0000","172.21.15.69:0/398102620":"2023-11-07T21:00:48.500569+0000","172.21.15.69:0/4081113363":"2023-11-07T21:00:48.500569+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:01:53.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 -- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f186805f390 msgr2=0x7f1868061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f186805f390 0x7f1868061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 -- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c064b10 msgr2=0x7f187c1497d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c064b10 0x7f187c1497d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f186c00e880 tx=0x7f186c009060).stop 2023-11-06T21:01:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 -- 172.21.15.33:0/608085758 shutdown_connections 2023-11-06T21:01:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f186805f390 0x7f1868061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f187c14ae80 0x7f187c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c064b10 0x7f187c1497d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 --2- 172.21.15.33:0/608085758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c0bc5c0 0x7f187c069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 -- 172.21.15.33:0/608085758 >> 172.21.15.33:0/608085758 conn(0x7f187c062710 msgr2=0x7f187c0c4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 -- 172.21.15.33:0/608085758 shutdown_connections 2023-11-06T21:01:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.444+0000 7f18813ff700 1 -- 172.21.15.33:0/608085758 wait complete. 2023-11-06T21:01:53.462 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T21:00:31.623152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T21:00:32.950772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T21:01:53.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T21:01:53.579 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:53.579+0000 7f34994a2680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:01:53.663 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:53.663+0000 7f34994a2680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:01:53.680 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:53.681+0000 7fae27539680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:01:53.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.698+0000 7f3788fff700 1 Processor -- start 2023-11-06T21:01:53.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.699+0000 7f3788fff700 1 -- start start 2023-11-06T21:01:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.700+0000 7f3788fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f378413dff0 0x7f3784157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.700+0000 7f3788fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3784064bf0 con 0x7f37840c4b00 2023-11-06T21:01:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.700+0000 7f3788fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3784062e10 con 0x7f37840c4720 2023-11-06T21:01:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.700+0000 7f3788fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378405e490 con 0x7f378413dff0 2023-11-06T21:01:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.700+0000 7f3781d9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f378413dff0 0x7f3784157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.700+0000 7f3782d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f37840c4b00 0x7f3784065080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56410/0 (socket says 172.21.15.33:56410) 2023-11-06T21:01:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.700+0000 7f3782d9d700 1 -- 172.21.15.33:0/2591850273 learned_addr learned my addr 172.21.15.33:0/2591850273 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.700+0000 7f378259c700 1 --1- 172.21.15.33:0/2591850273 >> v1:172.21.15.69:6789/0 conn(0x7f37840c4720 0x7f378413d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:57008/0 (socket says 172.21.15.33:57008) 2023-11-06T21:01:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f378159a700 1 -- 172.21.15.33:0/2591850273 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865783301 0 0) 0x7f3784062e10 con 0x7f37840c4720 2023-11-06T21:01:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f378159a700 1 -- 172.21.15.33:0/2591850273 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f376c0033f0 con 0x7f37840c4720 2023-11-06T21:01:53.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f378159a700 1 -- 172.21.15.33:0/2591850273 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065470265 0 0) 0x7f3784064bf0 con 0x7f37840c4b00 2023-11-06T21:01:53.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f378159a700 1 -- 172.21.15.33:0/2591850273 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3784062e10 con 0x7f37840c4b00 2023-11-06T21:01:53.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f3781d9b700 1 -- 172.21.15.33:0/2591850273 >> v1:172.21.15.69:6789/0 conn(0x7f37840c4720 legacy=0x7f378413d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f3781d9b700 1 -- 172.21.15.33:0/2591850273 >> v1:172.21.15.33:6789/0 conn(0x7f37840c4b00 legacy=0x7f3784065080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f3781d9b700 1 -- 172.21.15.33:0/2591850273 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37840630f0 con 0x7f378413dff0 2023-11-06T21:01:53.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f3781d9b700 1 --2- 172.21.15.33:0/2591850273 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f378413dff0 0x7f3784157e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f3778009790 tx=0x7f3778009060).ready entity=mon.2 client_cookie=19c8d0103684f25f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:53.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f378159a700 1 -- 172.21.15.33:0/2591850273 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377800f510 con 0x7f378413dff0 2023-11-06T21:01:53.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f378159a700 1 -- 172.21.15.33:0/2591850273 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377800f670 con 0x7f378413dff0 2023-11-06T21:01:53.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.701+0000 7f378159a700 1 -- 172.21.15.33:0/2591850273 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377800f9f0 con 0x7f378413dff0 2023-11-06T21:01:53.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.702+0000 7f3788fff700 1 -- 172.21.15.33:0/2591850273 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f378413dff0 msgr2=0x7f3784157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.702+0000 7f3788fff700 1 --2- 172.21.15.33:0/2591850273 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f378413dff0 0x7f3784157e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f3778009790 tx=0x7f3778009060).stop 2023-11-06T21:01:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.702+0000 7f3788fff700 1 -- 172.21.15.33:0/2591850273 shutdown_connections 2023-11-06T21:01:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.702+0000 7f3788fff700 1 --2- 172.21.15.33:0/2591850273 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f378413dff0 0x7f3784157e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.702+0000 7f3788fff700 1 -- 172.21.15.33:0/2591850273 >> 172.21.15.33:0/2591850273 conn(0x7f37840c7dc0 msgr2=0x7f37840c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.702+0000 7f3788fff700 1 -- 172.21.15.33:0/2591850273 shutdown_connections 2023-11-06T21:01:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.702+0000 7f3788fff700 1 -- 172.21.15.33:0/2591850273 wait complete. 2023-11-06T21:01:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.704+0000 7f3788fff700 1 Processor -- start 2023-11-06T21:01:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.704+0000 7f3788fff700 1 -- start start 2023-11-06T21:01:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f3788fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37840c4720 0x7f3784153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f3788fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37840c4b00 0x7f37841346a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f3788fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f378413dff0 0x7f37841569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f3788fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784135cc0 con 0x7f37840c4720 2023-11-06T21:01:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f3788fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37841350b0 con 0x7f378413dff0 2023-11-06T21:01:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f3788fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3784135390 con 0x7f37840c4b00 2023-11-06T21:01:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f378259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37840c4720 0x7f3784153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f378259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37840c4720 0x7f3784153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54922/0 (socket says 172.21.15.33:54922) 2023-11-06T21:01:53.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f378259c700 1 -- 172.21.15.33:0/167112668 learned_addr learned my addr 172.21.15.33:0/167112668 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:53.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f3781d9b700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37840c4b00 0x7f37841346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:53.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f3782d9d700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f378413dff0 0x7f37841569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:53.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f378259c700 1 -- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37840c4b00 msgr2=0x7f37841346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f378259c700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37840c4b00 0x7f37841346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f378259c700 1 -- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f378413dff0 msgr2=0x7f37841569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f378259c700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f378413dff0 0x7f37841569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f378259c700 1 -- 172.21.15.33:0/167112668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37780092a0 con 0x7f37840c4720 2023-11-06T21:01:53.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.705+0000 7f3782d9d700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f378413dff0 0x7f37841569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:01:53.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.706+0000 7f378259c700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37840c4720 0x7f3784153d50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f377400c440 tx=0x7f3774007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:53.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.706+0000 7f37727fc700 1 -- 172.21.15.33:0/167112668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377400ce40 con 0x7f37840c4720 2023-11-06T21:01:53.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.706+0000 7f37727fc700 1 -- 172.21.15.33:0/167112668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37740072e0 con 0x7f37840c4720 2023-11-06T21:01:53.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.706+0000 7f37727fc700 1 -- 172.21.15.33:0/167112668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37740056e0 con 0x7f37840c4720 2023-11-06T21:01:53.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.706+0000 7f3788fff700 1 -- 172.21.15.33:0/167112668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3784135670 con 0x7f37840c4720 2023-11-06T21:01:53.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.706+0000 7f3788fff700 1 -- 172.21.15.33:0/167112668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3784154240 con 0x7f37840c4720 2023-11-06T21:01:53.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.708+0000 7f3788fff700 1 -- 172.21.15.33:0/167112668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3784062b50 con 0x7f37840c4720 2023-11-06T21:01:53.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.709+0000 7f37727fc700 1 -- 172.21.15.33:0/167112668 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f3774005b10 con 0x7f37840c4720 2023-11-06T21:01:53.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.710+0000 7f37727fc700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f376405f390 0x7f3764061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:53.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.710+0000 7f37727fc700 1 -- 172.21.15.33:0/167112668 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f3774079e20 con 0x7f37840c4720 2023-11-06T21:01:53.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.710+0000 7f3781d9b700 1 -- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f376405f390 msgr2=0x7f3764061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.710+0000 7f3781d9b700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f376405f390 0x7f3764061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.714+0000 7f37727fc700 1 -- 172.21.15.33:0/167112668 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f377404ab50 con 0x7f37840c4720 2023-11-06T21:01:53.770 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:53.771+0000 7fae27539680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:01:53.895 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:53.896+0000 7f34994a2680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:01:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.897+0000 7f3788fff700 1 -- 172.21.15.33:0/167112668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f37840aa0d0 con 0x7f37840c4720 2023-11-06T21:01:53.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.898+0000 7f37727fc700 1 -- 172.21.15.33:0/167112668 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v117) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3774018030 con 0x7f37840c4720 2023-11-06T21:01:53.897 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T21:01:53.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 -- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f376405f390 msgr2=0x7f3764061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:53.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f376405f390 0x7f3764061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:53.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 -- 172.21.15.33:0/167112668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37840c4720 msgr2=0x7f3784153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:53.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37840c4720 0x7f3784153d50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f377400c440 tx=0x7f3774007c60).stop 2023-11-06T21:01:53.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 -- 172.21.15.33:0/167112668 shutdown_connections 2023-11-06T21:01:53.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f376405f390 0x7f3764061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:53.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f378413dff0 0x7f37841569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37840c4720 0x7f3784153d50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 --2- 172.21.15.33:0/167112668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f37840c4b00 0x7f37841346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:53.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 -- 172.21.15.33:0/167112668 >> 172.21.15.33:0/167112668 conn(0x7f37840c7dc0 msgr2=0x7f37840c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:53.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 -- 172.21.15.33:0/167112668 shutdown_connections 2023-11-06T21:01:53.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:53.900+0000 7f3788fff700 1 -- 172.21.15.33:0/167112668 wait complete. 2023-11-06T21:01:53.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T21:01:54.017 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:54.017+0000 7fae27539680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:01:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.121+0000 7f38b37ff700 1 Processor -- start 2023-11-06T21:01:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.122+0000 7f38b37ff700 1 -- start start 2023-11-06T21:01:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.123+0000 7f38b37ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38ac0aadf0 0x7f38ac068350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.123+0000 7f38b37ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ac152940 con 0x7f38ac0ab1d0 2023-11-06T21:01:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.123+0000 7f38b37ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ac153790 con 0x7f38ac0be2c0 2023-11-06T21:01:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.123+0000 7f38b37ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ac152b80 con 0x7f38ac0aadf0 2023-11-06T21:01:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.123+0000 7f38b159b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38ac0aadf0 0x7f38ac068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.123+0000 7f38b159b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38ac0aadf0 0x7f38ac068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54938/0 (socket says 172.21.15.33:54938) 2023-11-06T21:01:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.123+0000 7f38b159b700 1 -- 172.21.15.33:0/2399803771 learned_addr learned my addr 172.21.15.33:0/2399803771 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.123+0000 7f38b0d9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f38ac0ab1d0 0x7f38ac0bdbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56424/0 (socket says 172.21.15.33:56424) 2023-11-06T21:01:54.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.124+0000 7f38b159b700 1 -- 172.21.15.33:0/2399803771 >> v1:172.21.15.69:6789/0 conn(0x7f38ac0be2c0 legacy=0x7f38ac0c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.124+0000 7f38b159b700 1 -- 172.21.15.33:0/2399803771 >> v1:172.21.15.33:6789/0 conn(0x7f38ac0ab1d0 legacy=0x7f38ac0bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.124+0000 7f38b159b700 1 -- 172.21.15.33:0/2399803771 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac152e60 con 0x7f38ac0aadf0 2023-11-06T21:01:54.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.124+0000 7f38a3fff700 1 -- 172.21.15.33:0/2399803771 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193078472 0 0) 0x7f389c001110 con 0x7f38ac0ab1d0 2023-11-06T21:01:54.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.124+0000 7f38b159b700 1 --2- 172.21.15.33:0/2399803771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 0x7f38ac068350 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f38a8006fd0 tx=0x7f38a80090f0).ready entity=mon.0 client_cookie=3289a8f88d42b878 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:54.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.124+0000 7f38a3fff700 1 -- 172.21.15.33:0/2399803771 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a800b600 con 0x7f38ac0aadf0 2023-11-06T21:01:54.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.124+0000 7f38a3fff700 1 -- 172.21.15.33:0/2399803771 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a800b760 con 0x7f38ac0aadf0 2023-11-06T21:01:54.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.124+0000 7f38a3fff700 1 -- 172.21.15.33:0/2399803771 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a800bae0 con 0x7f38ac0aadf0 2023-11-06T21:01:54.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.125+0000 7f38b37ff700 1 -- 172.21.15.33:0/2399803771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 msgr2=0x7f38ac068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.125+0000 7f38b37ff700 1 --2- 172.21.15.33:0/2399803771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 0x7f38ac068350 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f38a8006fd0 tx=0x7f38a80090f0).stop 2023-11-06T21:01:54.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.125+0000 7f38b37ff700 1 -- 172.21.15.33:0/2399803771 shutdown_connections 2023-11-06T21:01:54.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.125+0000 7f38b37ff700 1 --2- 172.21.15.33:0/2399803771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 0x7f38ac068350 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.125+0000 7f38b37ff700 1 -- 172.21.15.33:0/2399803771 >> 172.21.15.33:0/2399803771 conn(0x7f38ac0c8c50 msgr2=0x7f38ac0c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:54.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.125+0000 7f38b37ff700 1 -- 172.21.15.33:0/2399803771 shutdown_connections 2023-11-06T21:01:54.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.125+0000 7f38b37ff700 1 -- 172.21.15.33:0/2399803771 wait complete. 2023-11-06T21:01:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.127+0000 7f38b37ff700 1 Processor -- start 2023-11-06T21:01:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.127+0000 7f38b37ff700 1 -- start start 2023-11-06T21:01:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b37ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 0x7f38ac0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b37ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38ac0ab1d0 0x7f38ac0a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b37ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38ac0be2c0 0x7f38ac064e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b37ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac0ae560 con 0x7f38ac0aadf0 2023-11-06T21:01:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b37ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac0ad950 con 0x7f38ac0be2c0 2023-11-06T21:01:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b37ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac0adc30 con 0x7f38ac0ab1d0 2023-11-06T21:01:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b159b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 0x7f38ac0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b159b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 0x7f38ac0ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54950/0 (socket says 172.21.15.33:54950) 2023-11-06T21:01:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b159b700 1 -- 172.21.15.33:0/2796181514 learned_addr learned my addr 172.21.15.33:0/2796181514 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b0d9a700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38ac0ab1d0 0x7f38ac0a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b1d9c700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38ac0be2c0 0x7f38ac064e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b159b700 1 -- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38ac0ab1d0 msgr2=0x7f38ac0a90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b159b700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38ac0ab1d0 0x7f38ac0a90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b159b700 1 -- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38ac0be2c0 msgr2=0x7f38ac064e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b159b700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38ac0be2c0 0x7f38ac064e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b159b700 1 -- 172.21.15.33:0/2796181514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a8009370 con 0x7f38ac0aadf0 2023-11-06T21:01:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.129+0000 7f38b1d9c700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38ac0be2c0 0x7f38ac064e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:01:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.130+0000 7f38b159b700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 0x7f38ac0ad0e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f38a8009130 tx=0x7f38a8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.130+0000 7f38a17fa700 1 -- 172.21.15.33:0/2796181514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a800b8d0 con 0x7f38ac0aadf0 2023-11-06T21:01:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.130+0000 7f38a17fa700 1 -- 172.21.15.33:0/2796181514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a800ba30 con 0x7f38ac0aadf0 2023-11-06T21:01:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.130+0000 7f38a17fa700 1 -- 172.21.15.33:0/2796181514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a801c670 con 0x7f38ac0aadf0 2023-11-06T21:01:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.130+0000 7f38b37ff700 1 -- 172.21.15.33:0/2796181514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac0adf10 con 0x7f38ac0aadf0 2023-11-06T21:01:54.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.130+0000 7f38b37ff700 1 -- 172.21.15.33:0/2796181514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ac0ad5d0 con 0x7f38ac0aadf0 2023-11-06T21:01:54.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.131+0000 7f38b37ff700 1 -- 172.21.15.33:0/2796181514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ac0676c0 con 0x7f38ac0aadf0 2023-11-06T21:01:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.132+0000 7f38a17fa700 1 -- 172.21.15.33:0/2796181514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f38a801c9e0 con 0x7f38ac0aadf0 2023-11-06T21:01:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.133+0000 7f38a17fa700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f389005f390 0x7f3890061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.134+0000 7f38a17fa700 1 -- 172.21.15.33:0/2796181514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f38a807c680 con 0x7f38ac0aadf0 2023-11-06T21:01:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.134+0000 7f38b0d9a700 1 -- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f389005f390 msgr2=0x7f3890061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.134+0000 7f38b0d9a700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f389005f390 0x7f3890061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.137+0000 7f38a17fa700 1 -- 172.21.15.33:0/2796181514 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f38a804d3b0 con 0x7f38ac0aadf0 2023-11-06T21:01:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.322+0000 7f38b37ff700 1 -- 172.21.15.33:0/2796181514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f38ac066000 con 0x7f38ac0aadf0 2023-11-06T21:01:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.322+0000 7f38a17fa700 1 -- 172.21.15.33:0/2796181514 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v117) v1 ==== 92+0+10 (secure 0 0 0) 0x7f38a802a020 con 0x7f38ac0aadf0 2023-11-06T21:01:54.323 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T21:01:54.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 -- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f389005f390 msgr2=0x7f3890061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:54.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f389005f390 0x7f3890061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:54.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 -- 172.21.15.33:0/2796181514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 msgr2=0x7f38ac0ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 0x7f38ac0ad0e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f38a8009130 tx=0x7f38a8017000).stop 2023-11-06T21:01:54.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 -- 172.21.15.33:0/2796181514 shutdown_connections 2023-11-06T21:01:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f389005f390 0x7f3890061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38ac0be2c0 0x7f38ac064e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ac0aadf0 0x7f38ac0ad0e0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 --2- 172.21.15.33:0/2796181514 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38ac0ab1d0 0x7f38ac0a90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 -- 172.21.15.33:0/2796181514 >> 172.21.15.33:0/2796181514 conn(0x7f38ac0c8c50 msgr2=0x7f38ac0c5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:54.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 -- 172.21.15.33:0/2796181514 shutdown_connections 2023-11-06T21:01:54.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.325+0000 7f38b37ff700 1 -- 172.21.15.33:0/2796181514 wait complete. 2023-11-06T21:01:54.342 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T21:01:54.416 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:54.416+0000 7f34994a2680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:01:54.540 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:54.540+0000 7f34994a2680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:01:54.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.543+0000 7f0cc69ff700 1 Processor -- start 2023-11-06T21:01:54.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.543+0000 7f0cc69ff700 1 -- start start 2023-11-06T21:01:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.545+0000 7f0cc69ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0cc01544f0 0x7f0cc00c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.546+0000 7f0cc69ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc01556d0 con 0x7f0cc0064dd0 2023-11-06T21:01:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.546+0000 7f0cc69ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc0154ac0 con 0x7f0cc00c0c40 2023-11-06T21:01:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.546+0000 7f0cc69ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc01548d0 con 0x7f0cc01544f0 2023-11-06T21:01:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.546+0000 7f0cbffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0cc01544f0 0x7f0cc00c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.546+0000 7f0cc4f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0cc0064dd0 0x7f0cc00c57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56432/0 (socket says 172.21.15.33:56432) 2023-11-06T21:01:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.546+0000 7f0cc4f9c700 1 -- 172.21.15.33:0/1251492067 learned_addr learned my addr 172.21.15.33:0/1251492067 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.546+0000 7f0cbffff700 1 -- 172.21.15.33:0/1251492067 >> v1:172.21.15.69:6789/0 conn(0x7f0cc00c0c40 legacy=0x7f0cc00646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.546+0000 7f0cbffff700 1 -- 172.21.15.33:0/1251492067 >> v1:172.21.15.33:6789/0 conn(0x7f0cc0064dd0 legacy=0x7f0cc00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.546+0000 7f0cbffff700 1 -- 172.21.15.33:0/1251492067 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc0154da0 con 0x7f0cc01544f0 2023-11-06T21:01:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.546+0000 7f0cbeffd700 1 -- 172.21.15.33:0/1251492067 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938217647 0 0) 0x7f0cc01556d0 con 0x7f0cc0064dd0 2023-11-06T21:01:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.547+0000 7f0cbffff700 1 --2- 172.21.15.33:0/1251492067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc01544f0 0x7f0cc00c0750 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f0cb0009760 tx=0x7f0cb0009060).ready entity=mon.0 client_cookie=5f6372c9f67df4d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.547+0000 7f0cbeffd700 1 -- 172.21.15.33:0/1251492067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb000b580 con 0x7f0cc01544f0 2023-11-06T21:01:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.547+0000 7f0cbeffd700 1 -- 172.21.15.33:0/1251492067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb000b6e0 con 0x7f0cc01544f0 2023-11-06T21:01:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.547+0000 7f0cbeffd700 1 -- 172.21.15.33:0/1251492067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb000ba60 con 0x7f0cc01544f0 2023-11-06T21:01:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.547+0000 7f0cc69ff700 1 -- 172.21.15.33:0/1251492067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc01544f0 msgr2=0x7f0cc00c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.547+0000 7f0cc69ff700 1 --2- 172.21.15.33:0/1251492067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc01544f0 0x7f0cc00c0750 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f0cb0009760 tx=0x7f0cb0009060).stop 2023-11-06T21:01:54.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.548+0000 7f0cc69ff700 1 -- 172.21.15.33:0/1251492067 shutdown_connections 2023-11-06T21:01:54.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.548+0000 7f0cc69ff700 1 --2- 172.21.15.33:0/1251492067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc01544f0 0x7f0cc00c0750 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.548+0000 7f0cc69ff700 1 -- 172.21.15.33:0/1251492067 >> 172.21.15.33:0/1251492067 conn(0x7f0cc00cb900 msgr2=0x7f0cc00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:54.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.548+0000 7f0cc69ff700 1 -- 172.21.15.33:0/1251492067 shutdown_connections 2023-11-06T21:01:54.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.548+0000 7f0cc69ff700 1 -- 172.21.15.33:0/1251492067 wait complete. 2023-11-06T21:01:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.549+0000 7f0cc69ff700 1 Processor -- start 2023-11-06T21:01:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.549+0000 7f0cc69ff700 1 -- start start 2023-11-06T21:01:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.550+0000 7f0cc69ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc0064dd0 0x7f0cc0152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.550+0000 7f0cc69ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cc00c0c40 0x7f0cc00a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.550+0000 7f0cc69ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cc01544f0 0x7f0cc01561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.550+0000 7f0cc69ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00c2640 con 0x7f0cc0064dd0 2023-11-06T21:01:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.550+0000 7f0cc69ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00b0e00 con 0x7f0cc01544f0 2023-11-06T21:01:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.550+0000 7f0cc69ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00b0b20 con 0x7f0cc00c0c40 2023-11-06T21:01:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cbffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc0064dd0 0x7f0cc0152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cbffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc0064dd0 0x7f0cc0152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54966/0 (socket says 172.21.15.33:54966) 2023-11-06T21:01:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cbffff700 1 -- 172.21.15.33:0/2525404873 learned_addr learned my addr 172.21.15.33:0/2525404873 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cc4f9c700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cc01544f0 0x7f0cc01561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cbf7fe700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cc00c0c40 0x7f0cc00a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cbffff700 1 -- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cc00c0c40 msgr2=0x7f0cc00a93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cbffff700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cc00c0c40 0x7f0cc00a93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cbffff700 1 -- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cc01544f0 msgr2=0x7f0cc01561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cbffff700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cc01544f0 0x7f0cc01561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cbffff700 1 -- 172.21.15.33:0/2525404873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb00092a0 con 0x7f0cc0064dd0 2023-11-06T21:01:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0cbffff700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc0064dd0 0x7f0cc0152450 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f0cb000b340 tx=0x7f0cb0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.551+0000 7f0ca7fff700 1 -- 172.21.15.33:0/2525404873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb000b850 con 0x7f0cc0064dd0 2023-11-06T21:01:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.552+0000 7f0ca7fff700 1 -- 172.21.15.33:0/2525404873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb000b9d0 con 0x7f0cc0064dd0 2023-11-06T21:01:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.552+0000 7f0cc69ff700 1 -- 172.21.15.33:0/2525404873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc00c1ff0 con 0x7f0cc0064dd0 2023-11-06T21:01:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.552+0000 7f0ca7fff700 1 -- 172.21.15.33:0/2525404873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb001b690 con 0x7f0cc0064dd0 2023-11-06T21:01:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.552+0000 7f0cc69ff700 1 -- 172.21.15.33:0/2525404873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc0152940 con 0x7f0cc0064dd0 2023-11-06T21:01:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.553+0000 7f0cc69ff700 1 -- 172.21.15.33:0/2525404873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc0155080 con 0x7f0cc0064dd0 2023-11-06T21:01:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.554+0000 7f0ca7fff700 1 -- 172.21.15.33:0/2525404873 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f0cb001ba80 con 0x7f0cc0064dd0 2023-11-06T21:01:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.554+0000 7f0ca7fff700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cac05f3b0 0x7f0cac061870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.555+0000 7f0ca7fff700 1 -- 172.21.15.33:0/2525404873 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f0cb007be10 con 0x7f0cc0064dd0 2023-11-06T21:01:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.555+0000 7f0cbf7fe700 1 -- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cac05f3b0 msgr2=0x7f0cac061870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:54.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.555+0000 7f0cbf7fe700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cac05f3b0 0x7f0cac061870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:54.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.559+0000 7f0ca7fff700 1 -- 172.21.15.33:0/2525404873 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0cb004ccc0 con 0x7f0cc0064dd0 2023-11-06T21:01:54.568 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:54.568+0000 7fae27539680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:01:54.699 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:54.700+0000 7fae27539680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:01:54.726 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:54.726+0000 7f34994a2680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:01:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.741+0000 7f0cc69ff700 1 -- 172.21.15.33:0/2525404873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0cc00c8760 con 0x7f0cc0064dd0 2023-11-06T21:01:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.741+0000 7f0ca7fff700 1 -- 172.21.15.33:0/2525404873 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0cb00280a0 con 0x7f0cc0064dd0 2023-11-06T21:01:54.741 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T21:01:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 -- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cac05f3b0 msgr2=0x7f0cac061870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cac05f3b0 0x7f0cac061870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 -- 172.21.15.33:0/2525404873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc0064dd0 msgr2=0x7f0cc0152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc0064dd0 0x7f0cc0152450 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f0cb000b340 tx=0x7f0cb0017000).stop 2023-11-06T21:01:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 -- 172.21.15.33:0/2525404873 shutdown_connections 2023-11-06T21:01:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0cac05f3b0 0x7f0cac061870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cc01544f0 0x7f0cc01561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc0064dd0 0x7f0cc0152450 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 --2- 172.21.15.33:0/2525404873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0cc00c0c40 0x7f0cc00a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 -- 172.21.15.33:0/2525404873 >> 172.21.15.33:0/2525404873 conn(0x7f0cc00cb900 msgr2=0x7f0cc0157040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 -- 172.21.15.33:0/2525404873 shutdown_connections 2023-11-06T21:01:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.743+0000 7f0cc69ff700 1 -- 172.21.15.33:0/2525404873 wait complete. 2023-11-06T21:01:54.760 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T21:01:54.820 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:54.820+0000 7f34994a2680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:01:54.899 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:54.899+0000 7fae27539680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:01:54.905 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:54.906+0000 7f34994a2680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:01:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.971+0000 7f1ba5dff700 1 Processor -- start 2023-11-06T21:01:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.971+0000 7f1ba5dff700 1 -- start start 2023-11-06T21:01:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.972+0000 7f1ba5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154330 0x7f1ba00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.972+0000 7f1ba5dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1ba0154710 0x7f1ba00c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.972+0000 7f1ba5dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba00c6b60 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.972+0000 7f1ba5dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba00c5cd0 con 0x7f1ba0154710 2023-11-06T21:01:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.972+0000 7f1ba5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00c5f50 con 0x7f1ba0154330 2023-11-06T21:01:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1ba00c7ff0 0x7f1ba00646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56448/0 (socket says 172.21.15.33:56448) 2023-11-06T21:01:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9ffff700 1 -- 172.21.15.33:0/2475956851 learned_addr learned my addr 172.21.15.33:0/2475956851 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9effd700 1 --2- 172.21.15.33:0/2475956851 >> v2:172.21.15.69:3300/0 conn(0x7f1ba0154710 0x7f1ba00c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9f7fe700 1 --2- 172.21.15.33:0/2475956851 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154330 0x7f1ba00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9e7fc700 1 -- 172.21.15.33:0/2475956851 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307954857 0 0) 0x7f1ba00c6b60 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9e7fc700 1 -- 172.21.15.33:0/2475956851 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b880033f0 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9effd700 1 -- 172.21.15.33:0/2475956851 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154330 msgr2=0x7f1ba00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:01:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9effd700 1 --2- 172.21.15.33:0/2475956851 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154330 0x7f1ba00c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9effd700 1 -- 172.21.15.33:0/2475956851 >> v1:172.21.15.33:6789/0 conn(0x7f1ba00c7ff0 legacy=0x7f1ba00646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9effd700 1 -- 172.21.15.33:0/2475956851 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba00c6230 con 0x7f1ba0154710 2023-11-06T21:01:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.973+0000 7f1b9effd700 1 --2- 172.21.15.33:0/2475956851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ba0154710 0x7f1ba00c6ff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f1b94009760 tx=0x7f1b94009060).ready entity=mon.1 client_cookie=fff355910124093d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.974+0000 7f1b9e7fc700 1 -- 172.21.15.33:0/2475956851 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400d4a0 con 0x7f1ba0154710 2023-11-06T21:01:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.974+0000 7f1b9e7fc700 1 -- 172.21.15.33:0/2475956851 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9400d600 con 0x7f1ba0154710 2023-11-06T21:01:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.974+0000 7f1b9e7fc700 1 -- 172.21.15.33:0/2475956851 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400d980 con 0x7f1ba0154710 2023-11-06T21:01:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.974+0000 7f1ba5dff700 1 -- 172.21.15.33:0/2475956851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ba0154710 msgr2=0x7f1ba00c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.974+0000 7f1ba5dff700 1 --2- 172.21.15.33:0/2475956851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ba0154710 0x7f1ba00c6ff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f1b94009760 tx=0x7f1b94009060).stop 2023-11-06T21:01:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.974+0000 7f1ba5dff700 1 -- 172.21.15.33:0/2475956851 shutdown_connections 2023-11-06T21:01:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.974+0000 7f1ba5dff700 1 --2- 172.21.15.33:0/2475956851 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154330 0x7f1ba00c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.974+0000 7f1ba5dff700 1 --2- 172.21.15.33:0/2475956851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ba0154710 0x7f1ba00c6ff0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.974+0000 7f1ba5dff700 1 -- 172.21.15.33:0/2475956851 >> 172.21.15.33:0/2475956851 conn(0x7f1ba00cb910 msgr2=0x7f1ba00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:54.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.974+0000 7f1ba5dff700 1 -- 172.21.15.33:0/2475956851 shutdown_connections 2023-11-06T21:01:54.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.975+0000 7f1ba5dff700 1 -- 172.21.15.33:0/2475956851 wait complete. 2023-11-06T21:01:54.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.976+0000 7f1ba5dff700 1 Processor -- start 2023-11-06T21:01:54.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.976+0000 7f1ba5dff700 1 -- start start 2023-11-06T21:01:54.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.977+0000 7f1ba5dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ba00c7ff0 0x7f1ba00c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.977+0000 7f1ba5dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ba0154330 0x7f1ba00c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.977+0000 7f1ba5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154710 0x7f1ba00a2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.977+0000 7f1ba5dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00c48d0 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.977+0000 7f1ba5dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00a2490 con 0x7f1ba0154330 2023-11-06T21:01:54.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.977+0000 7f1ba5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00b0810 con 0x7f1ba0154710 2023-11-06T21:01:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.977+0000 7f1b9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ba00c7ff0 0x7f1ba00c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.977+0000 7f1b9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ba00c7ff0 0x7f1ba00c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54974/0 (socket says 172.21.15.33:54974) 2023-11-06T21:01:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.977+0000 7f1b9f7fe700 1 -- 172.21.15.33:0/3403335654 learned_addr learned my addr 172.21.15.33:0/3403335654 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.977+0000 7f1b9effd700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ba0154330 0x7f1ba00c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b9ffff700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154710 0x7f1ba00a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b9f7fe700 1 -- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154710 msgr2=0x7f1ba00a2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b9f7fe700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154710 0x7f1ba00a2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b9f7fe700 1 -- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ba0154330 msgr2=0x7f1ba00c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b9f7fe700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ba0154330 0x7f1ba00c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b9f7fe700 1 -- 172.21.15.33:0/3403335654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940092a0 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b9ffff700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154710 0x7f1ba00a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:01:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b9f7fe700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ba00c7ff0 0x7f1ba00c3ef0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f1b9000e930 tx=0x7f1b90009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b877fe700 1 -- 172.21.15.33:0/3403335654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000c670 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b877fe700 1 -- 172.21.15.33:0/3403335654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9000c7d0 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1b877fe700 1 -- 172.21.15.33:0/3403335654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000cb50 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1ba5dff700 1 -- 172.21.15.33:0/3403335654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba00a1b60 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.978+0000 7f1ba5dff700 1 -- 172.21.15.33:0/3403335654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba00a2120 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.980+0000 7f1ba5dff700 1 -- 172.21.15.33:0/3403335654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba00a2770 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.980+0000 7f1b877fe700 1 -- 172.21.15.33:0/3403335654 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f1b9001b400 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.981+0000 7f1b877fe700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b8c05f390 0x7f1b8c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.981+0000 7f1b877fe700 1 -- 172.21.15.33:0/3403335654 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f1b9007adb0 con 0x7f1ba00c7ff0 2023-11-06T21:01:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.981+0000 7f1b9effd700 1 -- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b8c05f390 msgr2=0x7f1b8c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:54.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.981+0000 7f1b9effd700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b8c05f390 0x7f1b8c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:54.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:54.985+0000 7f1b877fe700 1 -- 172.21.15.33:0/3403335654 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1b9007f070 con 0x7f1ba00c7ff0 2023-11-06T21:01:55.000 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:55.001+0000 7fae27539680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:01:55.089 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:55.090+0000 7fae27539680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:01:55.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.168+0000 7f1ba5dff700 1 -- 172.21.15.33:0/3403335654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ba00652d0 con 0x7f1ba00c7ff0 2023-11-06T21:01:55.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.168+0000 7f1b877fe700 1 -- 172.21.15.33:0/3403335654 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b9004bb30 con 0x7f1ba00c7ff0 2023-11-06T21:01:55.168 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T21:01:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 -- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b8c05f390 msgr2=0x7f1b8c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b8c05f390 0x7f1b8c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 -- 172.21.15.33:0/3403335654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ba00c7ff0 msgr2=0x7f1ba00c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ba00c7ff0 0x7f1ba00c3ef0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f1b9000e930 tx=0x7f1b90009060).stop 2023-11-06T21:01:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 -- 172.21.15.33:0/3403335654 shutdown_connections 2023-11-06T21:01:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b8c05f390 0x7f1b8c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ba0154710 0x7f1ba00a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ba00c7ff0 0x7f1ba00c3ef0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 --2- 172.21.15.33:0/3403335654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ba0154330 0x7f1ba00c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 -- 172.21.15.33:0/3403335654 >> 172.21.15.33:0/3403335654 conn(0x7f1ba00cb910 msgr2=0x7f1ba0063760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 -- 172.21.15.33:0/3403335654 shutdown_connections 2023-11-06T21:01:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.171+0000 7f1ba5dff700 1 -- 172.21.15.33:0/3403335654 wait complete. 2023-11-06T21:01:55.190 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:01:55.220 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:55.220+0000 7f34994a2680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:01:55.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.389+0000 7fcd4f3ff700 1 Processor -- start 2023-11-06T21:01:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.389+0000 7fcd4f3ff700 1 -- start start 2023-11-06T21:01:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.390+0000 7fcd4f3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48063140 0x7fcd48063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.390+0000 7fcd4f3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd480c31f0 con 0x7fcd4805e1b0 2023-11-06T21:01:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.390+0000 7fcd4f3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd480c2920 con 0x7fcd480c3680 2023-11-06T21:01:55.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.390+0000 7fcd4f3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4806a0f0 con 0x7fcd48063140 2023-11-06T21:01:55.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.391+0000 7fcd4c99a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcd4805e1b0 0x7fcd480c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56456/0 (socket says 172.21.15.33:56456) 2023-11-06T21:01:55.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.391+0000 7fcd4c99a700 1 -- 172.21.15.33:0/3969963770 learned_addr learned my addr 172.21.15.33:0/3969963770 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:55.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.391+0000 7fcd4d19b700 1 --2- 172.21.15.33:0/3969963770 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48063140 0x7fcd48063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:55.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.391+0000 7fcd3ffff700 1 -- 172.21.15.33:0/3969963770 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714220597 0 0) 0x7fcd480c31f0 con 0x7fcd4805e1b0 2023-11-06T21:01:55.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.391+0000 7fcd3ffff700 1 -- 172.21.15.33:0/3969963770 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd340033f0 con 0x7fcd4805e1b0 2023-11-06T21:01:55.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.391+0000 7fcd3ffff700 1 -- 172.21.15.33:0/3969963770 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104841013 0 0) 0x7fcd480c2920 con 0x7fcd480c3680 2023-11-06T21:01:55.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.391+0000 7fcd3ffff700 1 -- 172.21.15.33:0/3969963770 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd480c31f0 con 0x7fcd480c3680 2023-11-06T21:01:55.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.391+0000 7fcd4d19b700 1 -- 172.21.15.33:0/3969963770 >> v1:172.21.15.69:6789/0 conn(0x7fcd480c3680 legacy=0x7fcd48154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.391+0000 7fcd4d19b700 1 -- 172.21.15.33:0/3969963770 >> v1:172.21.15.33:6789/0 conn(0x7fcd4805e1b0 legacy=0x7fcd480c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.391+0000 7fcd4d19b700 1 -- 172.21.15.33:0/3969963770 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4805e640 con 0x7fcd48063140 2023-11-06T21:01:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.392+0000 7fcd4d19b700 1 --2- 172.21.15.33:0/3969963770 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48063140 0x7fcd48063520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fcd44009790 tx=0x7fcd44009060).ready entity=mon.2 client_cookie=ad8197ca78253f33 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.392+0000 7fcd3ffff700 1 -- 172.21.15.33:0/3969963770 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400f510 con 0x7fcd48063140 2023-11-06T21:01:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.392+0000 7fcd3ffff700 1 -- 172.21.15.33:0/3969963770 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4400f670 con 0x7fcd48063140 2023-11-06T21:01:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.392+0000 7fcd3ffff700 1 -- 172.21.15.33:0/3969963770 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400f9f0 con 0x7fcd48063140 2023-11-06T21:01:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.393+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/3969963770 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48063140 msgr2=0x7fcd48063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.393+0000 7fcd4f3ff700 1 --2- 172.21.15.33:0/3969963770 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48063140 0x7fcd48063520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fcd44009790 tx=0x7fcd44009060).stop 2023-11-06T21:01:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.393+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/3969963770 shutdown_connections 2023-11-06T21:01:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.393+0000 7fcd4f3ff700 1 --2- 172.21.15.33:0/3969963770 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48063140 0x7fcd48063520 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.393+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/3969963770 >> 172.21.15.33:0/3969963770 conn(0x7fcd480c7d60 msgr2=0x7fcd480c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.393+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/3969963770 shutdown_connections 2023-11-06T21:01:55.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.393+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/3969963770 wait complete. 2023-11-06T21:01:55.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.394+0000 7fcd4f3ff700 1 Processor -- start 2023-11-06T21:01:55.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.395+0000 7fcd4f3ff700 1 -- start start 2023-11-06T21:01:55.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.395+0000 7fcd4f3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd48063140 0x7fcd48142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:55.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4f3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd480c3680 0x7fcd480ab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4f3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48143110 0x7fcd480abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4f3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd480abfe0 con 0x7fcd48063140 2023-11-06T21:01:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4f3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd48155530 con 0x7fcd480c3680 2023-11-06T21:01:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4f3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd48154920 con 0x7fcd48143110 2023-11-06T21:01:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4d19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd48063140 0x7fcd48142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4d19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd48063140 0x7fcd48142c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54988/0 (socket says 172.21.15.33:54988) 2023-11-06T21:01:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4d19b700 1 -- 172.21.15.33:0/2623796053 learned_addr learned my addr 172.21.15.33:0/2623796053 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4c99a700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd480c3680 0x7fcd480ab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4d99c700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48143110 0x7fcd480abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4d19b700 1 -- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48143110 msgr2=0x7fcd480abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4d19b700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48143110 0x7fcd480abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4d19b700 1 -- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd480c3680 msgr2=0x7fcd480ab600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4d19b700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd480c3680 0x7fcd480ab600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.396+0000 7fcd4d19b700 1 -- 172.21.15.33:0/2623796053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440092a0 con 0x7fcd48063140 2023-11-06T21:01:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.397+0000 7fcd4d19b700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd48063140 0x7fcd48142c20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7fcd4805ebe0 tx=0x7fcd44017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.397+0000 7fcd3d7fa700 1 -- 172.21.15.33:0/2623796053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4401a4d0 con 0x7fcd48063140 2023-11-06T21:01:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.397+0000 7fcd3d7fa700 1 -- 172.21.15.33:0/2623796053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4401e070 con 0x7fcd48063140 2023-11-06T21:01:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.397+0000 7fcd3d7fa700 1 -- 172.21.15.33:0/2623796053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4401a880 con 0x7fcd48063140 2023-11-06T21:01:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.397+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/2623796053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd48154c00 con 0x7fcd48063140 2023-11-06T21:01:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.397+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/2623796053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd481551c0 con 0x7fcd48063140 2023-11-06T21:01:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.398+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/2623796053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd48064420 con 0x7fcd48063140 2023-11-06T21:01:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.399+0000 7fcd3d7fa700 1 -- 172.21.15.33:0/2623796053 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fcd44020030 con 0x7fcd48063140 2023-11-06T21:01:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.400+0000 7fcd3d7fa700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd2c05f390 0x7fcd2c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.400+0000 7fcd3d7fa700 1 -- 172.21.15.33:0/2623796053 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fcd4407b760 con 0x7fcd48063140 2023-11-06T21:01:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.400+0000 7fcd4c99a700 1 -- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd2c05f390 msgr2=0x7fcd2c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.400+0000 7fcd4c99a700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd2c05f390 0x7fcd2c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.404+0000 7fcd3d7fa700 1 -- 172.21.15.33:0/2623796053 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcd4404c5e0 con 0x7fcd48063140 2023-11-06T21:01:55.431 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:55.432+0000 7fae27539680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:01:55.444 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:55.444+0000 7f34994a2680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:01:55.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.601+0000 7fcd4c99a700 1 -- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd2c05f390 msgr2=0x7fcd2c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.601+0000 7fcd4c99a700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd2c05f390 0x7fcd2c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:01:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.627+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/2623796053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd4806bfd0 con 0x7fcd48063140 2023-11-06T21:01:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.627+0000 7fcd3d7fa700 1 -- 172.21.15.33:0/2623796053 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4485 (secure 0 0 0) 0x7fcd4404fc00 con 0x7fcd48063140 2023-11-06T21:01:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:01:55.627 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":45,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2108889615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2108889615},{"type":"v1","addr":"172.21.15.33:6834","nonce":2108889615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/836821298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":836821298},{"type":"v1","addr":"172.21.15.69:6836","nonce":836821298}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6838,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2007995447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2007995447},{"type":"v1","addr":"172.21.15.33:6835","nonce":2007995447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:00:33.962914+0000","modified":"2023-11-06T21:00:36.976166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6844":{"gid":6844,"name":"d","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/396991049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":396991049},{"type":"v1","addr":"172.21.15.69:6837","nonce":396991049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-06T21:01:55.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.629+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd2c05f390 msgr2=0x7fcd2c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.629+0000 7fcd4f3ff700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd2c05f390 0x7fcd2c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.629+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/2623796053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd48063140 msgr2=0x7fcd48142c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.629+0000 7fcd4f3ff700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd48063140 0x7fcd48142c20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7fcd4805ebe0 tx=0x7fcd44017000).stop 2023-11-06T21:01:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.630+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/2623796053 shutdown_connections 2023-11-06T21:01:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.630+0000 7fcd4f3ff700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd2c05f390 0x7fcd2c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.630+0000 7fcd4f3ff700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcd48143110 0x7fcd480abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.630+0000 7fcd4f3ff700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd48063140 0x7fcd48142c20 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.630+0000 7fcd4f3ff700 1 --2- 172.21.15.33:0/2623796053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd480c3680 0x7fcd480ab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.630+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/2623796053 >> 172.21.15.33:0/2623796053 conn(0x7fcd480c7d60 msgr2=0x7fcd4814f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:55.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.630+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/2623796053 shutdown_connections 2023-11-06T21:01:55.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.630+0000 7fcd4f3ff700 1 -- 172.21.15.33:0/2623796053 wait complete. 2023-11-06T21:01:55.632 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 45 2023-11-06T21:01:55.648 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:01:55.662 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:55.663+0000 7fae27539680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:01:55.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.846+0000 7f8cccdff700 1 Processor -- start 2023-11-06T21:01:55.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.846+0000 7f8cccdff700 1 -- start start 2023-11-06T21:01:55.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.848+0000 7f8cccdff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8cc80bccc0 0x7f8cc80a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:55.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.848+0000 7f8cccdff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc80bbfb0 con 0x7f8cc8157720 2023-11-06T21:01:55.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.848+0000 7f8cccdff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc80a52d0 con 0x7f8cc80a4e60 2023-11-06T21:01:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.848+0000 7f8cccdff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc80bdb90 con 0x7f8cc80bccc0 2023-11-06T21:01:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.848+0000 7f8cc659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8cc8157720 0x7f8cc8158840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56458/0 (socket says 172.21.15.33:56458) 2023-11-06T21:01:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.848+0000 7f8cc659c700 1 -- 172.21.15.33:0/2496624683 learned_addr learned my addr 172.21.15.33:0/2496624683 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.848+0000 7f8cc5d9b700 1 --2- 172.21.15.33:0/2496624683 >> v2:172.21.15.69:3300/0 conn(0x7f8cc80bccc0 0x7f8cc80a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.848+0000 7f8cc559a700 1 -- 172.21.15.33:0/2496624683 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509552813 0 0) 0x7f8cc80bbfb0 con 0x7f8cc8157720 2023-11-06T21:01:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.848+0000 7f8cc559a700 1 -- 172.21.15.33:0/2496624683 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb00033f0 con 0x7f8cc8157720 2023-11-06T21:01:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.849+0000 7f8cc5d9b700 1 -- 172.21.15.33:0/2496624683 >> v1:172.21.15.69:6789/0 conn(0x7f8cc80a4e60 legacy=0x7f8cc80c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.849+0000 7f8cc5d9b700 1 -- 172.21.15.33:0/2496624683 >> v1:172.21.15.33:6789/0 conn(0x7f8cc8157720 legacy=0x7f8cc8158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.849+0000 7f8cc5d9b700 1 -- 172.21.15.33:0/2496624683 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc80bc1e0 con 0x7f8cc80bccc0 2023-11-06T21:01:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.849+0000 7f8cc5d9b700 1 --2- 172.21.15.33:0/2496624683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc80bccc0 0x7f8cc80a4970 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f8cbc009760 tx=0x7f8cbc009060).ready entity=mon.1 client_cookie=73cf33f9017f3003 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.849+0000 7f8cc559a700 1 -- 172.21.15.33:0/2496624683 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc00f510 con 0x7f8cc80bccc0 2023-11-06T21:01:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.849+0000 7f8cc559a700 1 -- 172.21.15.33:0/2496624683 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc00f670 con 0x7f8cc80bccc0 2023-11-06T21:01:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.849+0000 7f8cc559a700 1 -- 172.21.15.33:0/2496624683 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc00f9f0 con 0x7f8cc80bccc0 2023-11-06T21:01:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.850+0000 7f8cccdff700 1 -- 172.21.15.33:0/2496624683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc80bccc0 msgr2=0x7f8cc80a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.850+0000 7f8cccdff700 1 --2- 172.21.15.33:0/2496624683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc80bccc0 0x7f8cc80a4970 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f8cbc009760 tx=0x7f8cbc009060).stop 2023-11-06T21:01:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.850+0000 7f8cccdff700 1 -- 172.21.15.33:0/2496624683 shutdown_connections 2023-11-06T21:01:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.850+0000 7f8cccdff700 1 --2- 172.21.15.33:0/2496624683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc80bccc0 0x7f8cc80a4970 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.850+0000 7f8cccdff700 1 -- 172.21.15.33:0/2496624683 >> 172.21.15.33:0/2496624683 conn(0x7f8cc80631e0 msgr2=0x7f8cc80c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.850+0000 7f8cccdff700 1 -- 172.21.15.33:0/2496624683 shutdown_connections 2023-11-06T21:01:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.850+0000 7f8cccdff700 1 -- 172.21.15.33:0/2496624683 wait complete. 2023-11-06T21:01:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.851+0000 7f8cccdff700 1 Processor -- start 2023-11-06T21:01:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.851+0000 7f8cccdff700 1 -- start start 2023-11-06T21:01:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.852+0000 7f8cccdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc80a4e60 0x7f8cc80ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.852+0000 7f8cccdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cc80bccc0 0x7f8cc80aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.852+0000 7f8cccdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8157720 0x7f8cc80bdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.852+0000 7f8cccdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc814c4f0 con 0x7f8cc80a4e60 2023-11-06T21:01:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.852+0000 7f8cccdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc81482a0 con 0x7f8cc8157720 2023-11-06T21:01:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.852+0000 7f8cccdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80ab090 con 0x7f8cc80bccc0 2023-11-06T21:01:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc80a4e60 0x7f8cc80ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc80a4e60 0x7f8cc80ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54992/0 (socket says 172.21.15.33:54992) 2023-11-06T21:01:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc659c700 1 -- 172.21.15.33:0/4146842109 learned_addr learned my addr 172.21.15.33:0/4146842109 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc5d9b700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cc80bccc0 0x7f8cc80aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc6d9d700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8157720 0x7f8cc80bdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc659c700 1 -- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cc80bccc0 msgr2=0x7f8cc80aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc659c700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cc80bccc0 0x7f8cc80aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc659c700 1 -- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8157720 msgr2=0x7f8cc80bdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:55.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc659c700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8157720 0x7f8cc80bdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:55.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc659c700 1 -- 172.21.15.33:0/4146842109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc0092a0 con 0x7f8cc80a4e60 2023-11-06T21:01:55.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.853+0000 7f8cc659c700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc80a4e60 0x7f8cc80ac8c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f8cb800c590 tx=0x7f8cb8007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:55.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.854+0000 7f8cb67fc700 1 -- 172.21.15.33:0/4146842109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800ce80 con 0x7f8cc80a4e60 2023-11-06T21:01:55.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.854+0000 7f8cb67fc700 1 -- 172.21.15.33:0/4146842109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb8007e40 con 0x7f8cc80a4e60 2023-11-06T21:01:55.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.854+0000 7f8cb67fc700 1 -- 172.21.15.33:0/4146842109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb80074f0 con 0x7f8cc80a4e60 2023-11-06T21:01:55.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.854+0000 7f8cccdff700 1 -- 172.21.15.33:0/4146842109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc814c710 con 0x7f8cc80a4e60 2023-11-06T21:01:55.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.854+0000 7f8cccdff700 1 -- 172.21.15.33:0/4146842109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc80aa290 con 0x7f8cc80a4e60 2023-11-06T21:01:55.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.855+0000 7f8cccdff700 1 -- 172.21.15.33:0/4146842109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc80c1220 con 0x7f8cc80a4e60 2023-11-06T21:01:55.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.856+0000 7f8cb67fc700 1 -- 172.21.15.33:0/4146842109 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f8cb80059b0 con 0x7f8cc80a4e60 2023-11-06T21:01:55.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.857+0000 7f8cb67fc700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8ca805f390 0x7f8ca8061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:55.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.857+0000 7f8cb67fc700 1 -- 172.21.15.33:0/4146842109 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f8cb8079ed0 con 0x7f8cc80a4e60 2023-11-06T21:01:55.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.857+0000 7f8cc5d9b700 1 -- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8ca805f390 msgr2=0x7f8ca8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:55.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.857+0000 7f8cc5d9b700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8ca805f390 0x7f8ca8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:55.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:55.861+0000 7f8cb67fc700 1 -- 172.21.15.33:0/4146842109 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8cb804ac00 con 0x7f8cc80a4e60 2023-11-06T21:01:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.047+0000 7f8cccdff700 1 -- 172.21.15.33:0/4146842109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cc8157040 con 0x7f8cc80a4e60 2023-11-06T21:01:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.048+0000 7f8cb67fc700 1 -- 172.21.15.33:0/4146842109 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+17792 (secure 0 0 0) 0x7f8cb804e220 con 0x7f8cc80a4e60 2023-11-06T21:01:56.048 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:01:56.048 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":117,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:01:50.172076+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:00:31.623152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T21:00:32.950772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.11","mappings":[{"from":1,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.16","mappings":[{"from":4,"to":3}]},{"pgid":"14.25","mappings":[{"from":7,"to":3}]},{"pgid":"14.27","mappings":[{"from":1,"to":3}]},{"pgid":"14.35","mappings":[{"from":7,"to":0}]},{"pgid":"14.3b","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2951161837":"2023-11-07T21:00:48.500569+0000","172.21.15.69:6832/114912":"2023-11-07T21:00:48.500569+0000","172.21.15.69:0/443852199":"2023-11-07T21:00:48.500569+0000","172.21.15.69:6833/114912":"2023-11-07T21:00:48.500569+0000","172.21.15.69:0/398102620":"2023-11-07T21:00:48.500569+0000","172.21.15.69:0/4081113363":"2023-11-07T21:00:48.500569+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:01:56.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 -- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8ca805f390 msgr2=0x7f8ca8061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:56.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8ca805f390 0x7f8ca8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:56.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 -- 172.21.15.33:0/4146842109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc80a4e60 msgr2=0x7f8cc80ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc80a4e60 0x7f8cc80ac8c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f8cb800c590 tx=0x7f8cb8007c80).stop 2023-11-06T21:01:56.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 -- 172.21.15.33:0/4146842109 shutdown_connections 2023-11-06T21:01:56.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8ca805f390 0x7f8ca8061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:56.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8157720 0x7f8cc80bdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc80a4e60 0x7f8cc80ac8c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 --2- 172.21.15.33:0/4146842109 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cc80bccc0 0x7f8cc80aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 -- 172.21.15.33:0/4146842109 >> 172.21.15.33:0/4146842109 conn(0x7f8cc80631e0 msgr2=0x7f8cc80c61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:56.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 -- 172.21.15.33:0/4146842109 shutdown_connections 2023-11-06T21:01:56.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.050+0000 7f8cccdff700 1 -- 172.21.15.33:0/4146842109 wait complete. 2023-11-06T21:01:56.068 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T21:01:56.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:01:56.253 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:56.253+0000 7f34994a2680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:01:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.308+0000 7f6c16bff700 1 Processor -- start 2023-11-06T21:01:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.308+0000 7f6c16bff700 1 -- start start 2023-11-06T21:01:56.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.310+0000 7f6c16bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c1014ae80 0x7f6c100bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.310+0000 7f6c16bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c100c1940 con 0x7f6c100bc5c0 2023-11-06T21:01:56.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.310+0000 7f6c16bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c100bca40 con 0x7f6c10064b10 2023-11-06T21:01:56.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.310+0000 7f6c16bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1014b260 con 0x7f6c1014ae80 2023-11-06T21:01:56.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.311+0000 7f6c0ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6c100bc5c0 0x7f6c10064400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56472/0 (socket says 172.21.15.33:56472) 2023-11-06T21:01:56.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.311+0000 7f6c0ffff700 1 -- 172.21.15.33:0/1601553507 learned_addr learned my addr 172.21.15.33:0/1601553507 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:56.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.311+0000 7f6c1499b700 1 --2- 172.21.15.33:0/1601553507 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c1014ae80 0x7f6c100bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.311+0000 7f6c0f7fe700 1 -- 172.21.15.33:0/1601553507 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582871918 0 0) 0x7f6c100c1940 con 0x7f6c100bc5c0 2023-11-06T21:01:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.311+0000 7f6c0f7fe700 1 -- 172.21.15.33:0/1601553507 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c000033f0 con 0x7f6c100bc5c0 2023-11-06T21:01:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.311+0000 7f6c0f7fe700 1 -- 172.21.15.33:0/1601553507 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188297928 0 0) 0x7f6c100bca40 con 0x7f6c10064b10 2023-11-06T21:01:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.311+0000 7f6c0f7fe700 1 -- 172.21.15.33:0/1601553507 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c100c1940 con 0x7f6c10064b10 2023-11-06T21:01:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.311+0000 7f6c1499b700 1 -- 172.21.15.33:0/1601553507 >> v1:172.21.15.69:6789/0 conn(0x7f6c10064b10 legacy=0x7f6c100c1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.311+0000 7f6c1499b700 1 -- 172.21.15.33:0/1601553507 >> v1:172.21.15.33:6789/0 conn(0x7f6c100bc5c0 legacy=0x7f6c10064400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.311+0000 7f6c1499b700 1 -- 172.21.15.33:0/1601553507 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c10065160 con 0x7f6c1014ae80 2023-11-06T21:01:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.312+0000 7f6c1499b700 1 --2- 172.21.15.33:0/1601553507 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c1014ae80 0x7f6c100bc0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f6c04009790 tx=0x7f6c04009060).ready entity=mon.2 client_cookie=900fa3f2256c9944 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.312+0000 7f6c0f7fe700 1 -- 172.21.15.33:0/1601553507 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400f510 con 0x7f6c1014ae80 2023-11-06T21:01:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.312+0000 7f6c0f7fe700 1 -- 172.21.15.33:0/1601553507 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0400f670 con 0x7f6c1014ae80 2023-11-06T21:01:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.312+0000 7f6c0f7fe700 1 -- 172.21.15.33:0/1601553507 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400f9f0 con 0x7f6c1014ae80 2023-11-06T21:01:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.313+0000 7f6c16bff700 1 -- 172.21.15.33:0/1601553507 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c1014ae80 msgr2=0x7f6c100bc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.313+0000 7f6c16bff700 1 --2- 172.21.15.33:0/1601553507 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c1014ae80 0x7f6c100bc0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f6c04009790 tx=0x7f6c04009060).stop 2023-11-06T21:01:56.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.313+0000 7f6c16bff700 1 -- 172.21.15.33:0/1601553507 shutdown_connections 2023-11-06T21:01:56.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.313+0000 7f6c16bff700 1 --2- 172.21.15.33:0/1601553507 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c1014ae80 0x7f6c100bc0d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.313+0000 7f6c16bff700 1 -- 172.21.15.33:0/1601553507 >> 172.21.15.33:0/1601553507 conn(0x7f6c10062710 msgr2=0x7f6c10062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:56.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.313+0000 7f6c16bff700 1 -- 172.21.15.33:0/1601553507 shutdown_connections 2023-11-06T21:01:56.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.313+0000 7f6c16bff700 1 -- 172.21.15.33:0/1601553507 wait complete. 2023-11-06T21:01:56.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.315+0000 7f6c16bff700 1 Processor -- start 2023-11-06T21:01:56.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.316+0000 7f6c16bff700 1 -- start start 2023-11-06T21:01:56.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.317+0000 7f6c16bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c10064b10 0x7f6c10154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.317+0000 7f6c16bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c100bc5c0 0x7f6c1014d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.317+0000 7f6c16bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1014ae80 0x7f6c1014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.317+0000 7f6c16bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1014df60 con 0x7f6c1014ae80 2023-11-06T21:01:56.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.317+0000 7f6c16bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10152770 con 0x7f6c10064b10 2023-11-06T21:01:56.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.317+0000 7f6c16bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10154d60 con 0x7f6c100bc5c0 2023-11-06T21:01:56.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.317+0000 7f6c1519c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1014ae80 0x7f6c1014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:56.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.317+0000 7f6c1499b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c10064b10 0x7f6c10154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:56.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.318+0000 7f6c0ffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c100bc5c0 0x7f6c1014d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:56.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.318+0000 7f6c1519c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1014ae80 0x7f6c1014da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55004/0 (socket says 172.21.15.33:55004) 2023-11-06T21:01:56.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.318+0000 7f6c1519c700 1 -- 172.21.15.33:0/2060282895 learned_addr learned my addr 172.21.15.33:0/2060282895 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:56.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.318+0000 7f6c1499b700 1 -- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c100bc5c0 msgr2=0x7f6c1014d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.318+0000 7f6c1499b700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c100bc5c0 0x7f6c1014d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.318+0000 7f6c1499b700 1 -- 172.21.15.33:0/2060282895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1014ae80 msgr2=0x7f6c1014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.318+0000 7f6c1499b700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1014ae80 0x7f6c1014da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.318+0000 7f6c1499b700 1 -- 172.21.15.33:0/2060282895 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c040092a0 con 0x7f6c10064b10 2023-11-06T21:01:56.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.318+0000 7f6c1499b700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c10064b10 0x7f6c10154870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f6c10067f00 tx=0x7f6c04017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:56.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.318+0000 7f6c0cff9700 1 -- 172.21.15.33:0/2060282895 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400fe80 con 0x7f6c10064b10 2023-11-06T21:01:56.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.319+0000 7f6c0cff9700 1 -- 172.21.15.33:0/2060282895 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0400f7e0 con 0x7f6c10064b10 2023-11-06T21:01:56.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.319+0000 7f6c0cff9700 1 -- 172.21.15.33:0/2060282895 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0401b640 con 0x7f6c10064b10 2023-11-06T21:01:56.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.319+0000 7f6c16bff700 1 -- 172.21.15.33:0/2060282895 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1014c3e0 con 0x7f6c10064b10 2023-11-06T21:01:56.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.319+0000 7f6c16bff700 1 -- 172.21.15.33:0/2060282895 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1014c9a0 con 0x7f6c10064b10 2023-11-06T21:01:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.320+0000 7f6c16bff700 1 -- 172.21.15.33:0/2060282895 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1005f0b0 con 0x7f6c10064b10 2023-11-06T21:01:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.321+0000 7f6c0cff9700 1 -- 172.21.15.33:0/2060282895 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f6c0401b9f0 con 0x7f6c10064b10 2023-11-06T21:01:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.322+0000 7f6c0cff9700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6bf405f390 0x7f6bf4061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.322+0000 7f6c0cff9700 1 -- 172.21.15.33:0/2060282895 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f6c0407bf20 con 0x7f6c10064b10 2023-11-06T21:01:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.322+0000 7f6c0ffff700 1 -- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6bf405f390 msgr2=0x7f6bf4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.322+0000 7f6c0ffff700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6bf405f390 0x7f6bf4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.330+0000 7f6c0cff9700 1 -- 172.21.15.33:0/2060282895 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6c0404cc90 con 0x7f6c10064b10 2023-11-06T21:01:56.343 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:56.343+0000 7f34994a2680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:01:56.436 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:56.436+0000 7f34994a2680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:01:56.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.523+0000 7f6c0ffff700 1 -- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6bf405f390 msgr2=0x7f6bf4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:56.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.523+0000 7f6c0ffff700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6bf405f390 0x7f6bf4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:01:56.523 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:56.523+0000 7fae27539680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:01:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.557+0000 7f6c16bff700 1 -- 172.21.15.33:0/2060282895 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c101528b0 con 0x7f6c10064b10 2023-11-06T21:01:56.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.558+0000 7f6c0cff9700 1 -- 172.21.15.33:0/2060282895 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4485 (secure 0 0 0) 0x7f6c04029020 con 0x7f6c10064b10 2023-11-06T21:01:56.558 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:01:56.559 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":45,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2108889615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2108889615},{"type":"v1","addr":"172.21.15.33:6834","nonce":2108889615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/836821298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":836821298},{"type":"v1","addr":"172.21.15.69:6836","nonce":836821298}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6838,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2007995447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2007995447},{"type":"v1","addr":"172.21.15.33:6835","nonce":2007995447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:00:33.962914+0000","modified":"2023-11-06T21:00:36.976166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6844":{"gid":6844,"name":"d","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/396991049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":396991049},{"type":"v1","addr":"172.21.15.69:6837","nonce":396991049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-06T21:01:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 -- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6bf405f390 msgr2=0x7f6bf4061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6bf405f390 0x7f6bf4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 -- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c10064b10 msgr2=0x7f6c10154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c10064b10 0x7f6c10154870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f6c10067f00 tx=0x7f6c04017000).stop 2023-11-06T21:01:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 -- 172.21.15.33:0/2060282895 shutdown_connections 2023-11-06T21:01:56.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6bf405f390 0x7f6bf4061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:56.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1014ae80 0x7f6c1014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c10064b10 0x7f6c10154870 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 --2- 172.21.15.33:0/2060282895 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6c100bc5c0 0x7f6c1014d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 -- 172.21.15.33:0/2060282895 >> 172.21.15.33:0/2060282895 conn(0x7f6c10062710 msgr2=0x7f6c101511f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:56.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 -- 172.21.15.33:0/2060282895 shutdown_connections 2023-11-06T21:01:56.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.561+0000 7f6c16bff700 1 -- 172.21.15.33:0/2060282895 wait complete. 2023-11-06T21:01:56.563 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 45 2023-11-06T21:01:56.579 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:01:56.621 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:56.621+0000 7f34994a2680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:01:56.622 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:56.622+0000 7fae27539680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:01:56.710 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:56.710+0000 7f34994a2680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:01:56.724 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:56.724+0000 7fae27539680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:01:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.782+0000 7fb6a25ff700 1 Processor -- start 2023-11-06T21:01:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.782+0000 7fb6a25ff700 1 -- start start 2023-11-06T21:01:56.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.784+0000 7fb6a25ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb69c064580 0x7fb69c064960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.784+0000 7fb6a25ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb69c0c3490 0x7fb69c0a7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.784+0000 7fb6a25ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69c0c2bc0 con 0x7fb69c0c3050 2023-11-06T21:01:56.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.784+0000 7fb6a25ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c0bbe00 con 0x7fb69c0c3490 2023-11-06T21:01:56.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.784+0000 7fb6a25ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c152620 con 0x7fb69c064580 2023-11-06T21:01:56.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.784+0000 7fb6a0b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb69c0c3490 0x7fb69c0a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:56.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.784+0000 7fb6a0b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb69c0c3490 0x7fb69c0a7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55018/0 (socket says 172.21.15.33:55018) 2023-11-06T21:01:56.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.784+0000 7fb69b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb69c0c3050 0x7fb69c0a6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56476/0 (socket says 172.21.15.33:56476) 2023-11-06T21:01:56.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.784+0000 7fb6a0b9c700 1 -- 172.21.15.33:0/1776109047 learned_addr learned my addr 172.21.15.33:0/1776109047 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:56.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.784+0000 7fb69bfff700 1 --2- 172.21.15.33:0/1776109047 >> v2:172.21.15.69:3300/0 conn(0x7fb69c064580 0x7fb69c064960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:56.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.785+0000 7fb6a0b9c700 1 -- 172.21.15.33:0/1776109047 >> v2:172.21.15.69:3300/0 conn(0x7fb69c064580 msgr2=0x7fb69c064960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:01:56.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.785+0000 7fb6a0b9c700 1 --2- 172.21.15.33:0/1776109047 >> v2:172.21.15.69:3300/0 conn(0x7fb69c064580 0x7fb69c064960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.785+0000 7fb6a0b9c700 1 -- 172.21.15.33:0/1776109047 >> v1:172.21.15.33:6789/0 conn(0x7fb69c0c3050 legacy=0x7fb69c0a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.785+0000 7fb6a0b9c700 1 -- 172.21.15.33:0/1776109047 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c0bbfe0 con 0x7fb69c0c3490 2023-11-06T21:01:56.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.785+0000 7fb69affd700 1 -- 172.21.15.33:0/1776109047 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378217554 0 0) 0x7fb69c0c2bc0 con 0x7fb69c0c3050 2023-11-06T21:01:56.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.785+0000 7fb6a0b9c700 1 --2- 172.21.15.33:0/1776109047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c0c3490 0x7fb69c0a7160 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7fb69c0684b0 tx=0x7fb694009840).ready entity=mon.0 client_cookie=a40cac0de7b759dd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.785+0000 7fb69affd700 1 -- 172.21.15.33:0/1776109047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69400e070 con 0x7fb69c0c3490 2023-11-06T21:01:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.785+0000 7fb69affd700 1 -- 172.21.15.33:0/1776109047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69400ba20 con 0x7fb69c0c3490 2023-11-06T21:01:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.785+0000 7fb69affd700 1 -- 172.21.15.33:0/1776109047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69400bda0 con 0x7fb69c0c3490 2023-11-06T21:01:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.786+0000 7fb6a25ff700 1 -- 172.21.15.33:0/1776109047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c0c3490 msgr2=0x7fb69c0a7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.786+0000 7fb6a25ff700 1 --2- 172.21.15.33:0/1776109047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c0c3490 0x7fb69c0a7160 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7fb69c0684b0 tx=0x7fb694009840).stop 2023-11-06T21:01:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.786+0000 7fb6a25ff700 1 -- 172.21.15.33:0/1776109047 shutdown_connections 2023-11-06T21:01:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.786+0000 7fb6a25ff700 1 --2- 172.21.15.33:0/1776109047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c0c3490 0x7fb69c0a7160 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.786+0000 7fb6a25ff700 1 --2- 172.21.15.33:0/1776109047 >> v2:172.21.15.69:3300/0 conn(0x7fb69c064580 0x7fb69c064960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.786+0000 7fb6a25ff700 1 -- 172.21.15.33:0/1776109047 >> 172.21.15.33:0/1776109047 conn(0x7fb69c0631e0 msgr2=0x7fb69c0c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.786+0000 7fb6a25ff700 1 -- 172.21.15.33:0/1776109047 shutdown_connections 2023-11-06T21:01:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.786+0000 7fb6a25ff700 1 -- 172.21.15.33:0/1776109047 wait complete. 2023-11-06T21:01:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.788+0000 7fb6a25ff700 1 Processor -- start 2023-11-06T21:01:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.788+0000 7fb6a25ff700 1 -- start start 2023-11-06T21:01:56.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb6a25ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c064580 0x7fb69c0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb6a25ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb69c0c3050 0x7fb69c164960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb6a25ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69c0c3490 0x7fb69c14b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb6a25ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0c5f90 con 0x7fb69c064580 2023-11-06T21:01:56.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb6a25ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0c5380 con 0x7fb69c0c3490 2023-11-06T21:01:56.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb6a25ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0c5660 con 0x7fb69c0c3050 2023-11-06T21:01:56.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb69bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c064580 0x7fb69c0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:56.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb69bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c064580 0x7fb69c0c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55024/0 (socket says 172.21.15.33:55024) 2023-11-06T21:01:56.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb69bfff700 1 -- 172.21.15.33:0/955914664 learned_addr learned my addr 172.21.15.33:0/955914664 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:56.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb69b7fe700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb69c0c3050 0x7fb69c164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:56.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.789+0000 7fb6a0b9c700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69c0c3490 0x7fb69c14b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:56.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb69bfff700 1 -- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb69c0c3050 msgr2=0x7fb69c164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb69bfff700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb69c0c3050 0x7fb69c164960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb69bfff700 1 -- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69c0c3490 msgr2=0x7fb69c14b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb69bfff700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69c0c3490 0x7fb69c14b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb69bfff700 1 -- 172.21.15.33:0/955914664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb694009ae0 con 0x7fb69c064580 2023-11-06T21:01:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb69bfff700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c064580 0x7fb69c0c4d80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fb68c00e930 tx=0x7fb68c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb683fff700 1 -- 172.21.15.33:0/955914664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c012600 con 0x7fb69c064580 2023-11-06T21:01:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb683fff700 1 -- 172.21.15.33:0/955914664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c012760 con 0x7fb69c064580 2023-11-06T21:01:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb6a25ff700 1 -- 172.21.15.33:0/955914664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c0c5940 con 0x7fb69c064580 2023-11-06T21:01:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb683fff700 1 -- 172.21.15.33:0/955914664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c012ae0 con 0x7fb69c064580 2023-11-06T21:01:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.790+0000 7fb6a25ff700 1 -- 172.21.15.33:0/955914664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c164e50 con 0x7fb69c064580 2023-11-06T21:01:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.791+0000 7fb6a25ff700 1 -- 172.21.15.33:0/955914664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c0c6270 con 0x7fb69c064580 2023-11-06T21:01:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.792+0000 7fb683fff700 1 -- 172.21.15.33:0/955914664 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fb68c01b400 con 0x7fb69c064580 2023-11-06T21:01:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.793+0000 7fb683fff700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb68805f390 0x7fb688061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.793+0000 7fb683fff700 1 -- 172.21.15.33:0/955914664 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fb68c07ac10 con 0x7fb69c064580 2023-11-06T21:01:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.793+0000 7fb69b7fe700 1 -- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb68805f390 msgr2=0x7fb688061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.793+0000 7fb69b7fe700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb68805f390 0x7fb688061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.797+0000 7fb683fff700 1 -- 172.21.15.33:0/955914664 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb68c04b980 con 0x7fb69c064580 2023-11-06T21:01:56.850 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:56.850+0000 7f34994a2680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:01:56.921 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:56.922+0000 7fae27539680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:01:56.980 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:56.980+0000 7f34994a2680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:01:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.994+0000 7fb69b7fe700 1 -- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb68805f390 msgr2=0x7fb688061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:56.994+0000 7fb69b7fe700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb68805f390 0x7fb688061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:01:57.030 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:57.030+0000 7fae27539680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:01:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.065+0000 7fb6a25ff700 1 -- 172.21.15.33:0/955914664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb69c0bc2c0 con 0x7fb69c064580 2023-11-06T21:01:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.066+0000 7fb683fff700 1 -- 172.21.15.33:0/955914664 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4485 (secure 0 0 0) 0x7fb68c04efa0 con 0x7fb69c064580 2023-11-06T21:01:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:01:57.066 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":45,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2108889615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2108889615},{"type":"v1","addr":"172.21.15.33:6834","nonce":2108889615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/836821298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":836821298},{"type":"v1","addr":"172.21.15.69:6836","nonce":836821298}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6838,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2007995447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2007995447},{"type":"v1","addr":"172.21.15.33:6835","nonce":2007995447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:00:33.962914+0000","modified":"2023-11-06T21:00:36.976166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6844":{"gid":6844,"name":"d","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/396991049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":396991049},{"type":"v1","addr":"172.21.15.69:6837","nonce":396991049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-06T21:01:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 -- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb68805f390 msgr2=0x7fb688061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb68805f390 0x7fb688061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 -- 172.21.15.33:0/955914664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c064580 msgr2=0x7fb69c0c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c064580 0x7fb69c0c4d80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fb68c00e930 tx=0x7fb68c009060).stop 2023-11-06T21:01:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 -- 172.21.15.33:0/955914664 shutdown_connections 2023-11-06T21:01:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb68805f390 0x7fb688061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69c0c3490 0x7fb69c14b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb69c064580 0x7fb69c0c4d80 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 --2- 172.21.15.33:0/955914664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb69c0c3050 0x7fb69c164960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 -- 172.21.15.33:0/955914664 >> 172.21.15.33:0/955914664 conn(0x7fb69c0631e0 msgr2=0x7fb69c1511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:57.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 -- 172.21.15.33:0/955914664 shutdown_connections 2023-11-06T21:01:57.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.069+0000 7fb6a25ff700 1 -- 172.21.15.33:0/955914664 wait complete. 2023-11-06T21:01:57.072 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 45 2023-11-06T21:01:57.095 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:01:57.204 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:57.205+0000 7fae27539680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:01:57.327 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.327+0000 7f34994a2680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:01:57.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.335+0000 7f818ddff700 1 Processor -- start 2023-11-06T21:01:57.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.336+0000 7f818ddff700 1 -- start start 2023-11-06T21:01:57.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.337+0000 7f818ddff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8188138530 0x7f8188066540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:57.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.337+0000 7f818ddff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188068470 0x7f8188066a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:57.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.337+0000 7f818ddff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81880688e0 con 0x7f8188138910 2023-11-06T21:01:57.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.337+0000 7f818ddff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8188066f20 con 0x7f8188138530 2023-11-06T21:01:57.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.337+0000 7f818ddff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81880c35b0 con 0x7f8188068470 2023-11-06T21:01:57.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.337+0000 7f8186ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8188138910 0x7f8188067d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56484/0 (socket says 172.21.15.33:56484) 2023-11-06T21:01:57.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.337+0000 7f8186ffd700 1 -- 172.21.15.33:0/2034928338 learned_addr learned my addr 172.21.15.33:0/2034928338 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:57.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.337+0000 7f8187fff700 1 --2- 172.21.15.33:0/2034928338 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188068470 0x7f8188066a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:57.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.337+0000 7f81877fe700 1 --2- 172.21.15.33:0/2034928338 >> v2:172.21.15.69:3300/0 conn(0x7f8188138530 0x7f8188066540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:57.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.338+0000 7f81867fc700 1 -- 172.21.15.33:0/2034928338 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413993737 0 0) 0x7f81880688e0 con 0x7f8188138910 2023-11-06T21:01:57.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.338+0000 7f81867fc700 1 -- 172.21.15.33:0/2034928338 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81700033f0 con 0x7f8188138910 2023-11-06T21:01:57.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.338+0000 7f8187fff700 1 -- 172.21.15.33:0/2034928338 >> v2:172.21.15.69:3300/0 conn(0x7f8188138530 msgr2=0x7f8188066540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:01:57.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.338+0000 7f8187fff700 1 --2- 172.21.15.33:0/2034928338 >> v2:172.21.15.69:3300/0 conn(0x7f8188138530 0x7f8188066540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.338+0000 7f8187fff700 1 -- 172.21.15.33:0/2034928338 >> v1:172.21.15.33:6789/0 conn(0x7f8188138910 legacy=0x7f8188067d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.338+0000 7f8187fff700 1 -- 172.21.15.33:0/2034928338 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880c4130 con 0x7f8188068470 2023-11-06T21:01:57.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.338+0000 7f81877fe700 1 --2- 172.21.15.33:0/2034928338 >> v2:172.21.15.69:3300/0 conn(0x7f8188138530 0x7f8188066540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:01:57.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.338+0000 7f8187fff700 1 --2- 172.21.15.33:0/2034928338 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188068470 0x7f8188066a30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8180009e60 tx=0x7f8180009780).ready entity=mon.2 client_cookie=65eb164fc260d244 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:57.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.338+0000 7f81867fc700 1 -- 172.21.15.33:0/2034928338 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000e070 con 0x7f8188068470 2023-11-06T21:01:57.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.339+0000 7f81867fc700 1 -- 172.21.15.33:0/2034928338 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818000bab0 con 0x7f8188068470 2023-11-06T21:01:57.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.339+0000 7f81867fc700 1 -- 172.21.15.33:0/2034928338 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000be30 con 0x7f8188068470 2023-11-06T21:01:57.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.339+0000 7f818ddff700 1 -- 172.21.15.33:0/2034928338 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188068470 msgr2=0x7f8188066a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.339+0000 7f818ddff700 1 --2- 172.21.15.33:0/2034928338 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188068470 0x7f8188066a30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8180009e60 tx=0x7f8180009780).stop 2023-11-06T21:01:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.339+0000 7f818ddff700 1 -- 172.21.15.33:0/2034928338 shutdown_connections 2023-11-06T21:01:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.339+0000 7f818ddff700 1 --2- 172.21.15.33:0/2034928338 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188068470 0x7f8188066a30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.339+0000 7f818ddff700 1 --2- 172.21.15.33:0/2034928338 >> v2:172.21.15.69:3300/0 conn(0x7f8188138530 0x7f8188066540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.339+0000 7f818ddff700 1 -- 172.21.15.33:0/2034928338 >> 172.21.15.33:0/2034928338 conn(0x7f81880c8c10 msgr2=0x7f81880c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.339+0000 7f818ddff700 1 -- 172.21.15.33:0/2034928338 shutdown_connections 2023-11-06T21:01:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.339+0000 7f818ddff700 1 -- 172.21.15.33:0/2034928338 wait complete. 2023-11-06T21:01:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.341+0000 7f818ddff700 1 Processor -- start 2023-11-06T21:01:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.341+0000 7f818ddff700 1 -- start start 2023-11-06T21:01:57.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f818ddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8188068470 0x7f81880c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:57.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f818ddff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188138530 0x7f81880acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:57.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f818ddff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188138910 0x7f8188151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:57.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f818ddff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81880ae0a0 con 0x7f8188138910 2023-11-06T21:01:57.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f818ddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81880ad490 con 0x7f8188068470 2023-11-06T21:01:57.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f818ddff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81880ad770 con 0x7f8188138530 2023-11-06T21:01:57.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f8187fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188138910 0x7f8188151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:57.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f81877fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8188068470 0x7f81880c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:57.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f8187fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188138910 0x7f8188151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55028/0 (socket says 172.21.15.33:55028) 2023-11-06T21:01:57.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f81877fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8188068470 0x7f81880c26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:35268/0 (socket says 172.21.15.33:35268) 2023-11-06T21:01:57.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f8187fff700 1 -- 172.21.15.33:0/2553478798 learned_addr learned my addr 172.21.15.33:0/2553478798 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:57.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.342+0000 7f8186ffd700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188138530 0x7f81880acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:57.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f8187fff700 1 -- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188138530 msgr2=0x7f81880acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f8187fff700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188138530 0x7f81880acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f8187fff700 1 -- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8188068470 msgr2=0x7f81880c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f8187fff700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8188068470 0x7f81880c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f8187fff700 1 -- 172.21.15.33:0/2553478798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81780092a0 con 0x7f8188138910 2023-11-06T21:01:57.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f81877fe700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8188068470 0x7f81880c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:01:57.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f8187fff700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188138910 0x7f8188151ba0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f818000b610 tx=0x7f8180009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:57.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f816f7fe700 1 -- 172.21.15.33:0/2553478798 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180013e70 con 0x7f8188138910 2023-11-06T21:01:57.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f816f7fe700 1 -- 172.21.15.33:0/2553478798 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818001d630 con 0x7f8188138910 2023-11-06T21:01:57.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f818ddff700 1 -- 172.21.15.33:0/2553478798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8180009a20 con 0x7f8188138910 2023-11-06T21:01:57.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f816f7fe700 1 -- 172.21.15.33:0/2553478798 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818001d9f0 con 0x7f8188138910 2023-11-06T21:01:57.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.343+0000 7f818ddff700 1 -- 172.21.15.33:0/2553478798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81880add30 con 0x7f8188138910 2023-11-06T21:01:57.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.344+0000 7f818ddff700 1 -- 172.21.15.33:0/2553478798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81880ae210 con 0x7f8188138910 2023-11-06T21:01:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.345+0000 7f816f7fe700 1 -- 172.21.15.33:0/2553478798 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f8180003040 con 0x7f8188138910 2023-11-06T21:01:57.353 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:57.346+0000 7fae27539680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:01:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.346+0000 7f816f7fe700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f817405f390 0x7f8174061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.346+0000 7f816f7fe700 1 -- 172.21.15.33:0/2553478798 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f818007c690 con 0x7f8188138910 2023-11-06T21:01:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.347+0000 7f81877fe700 1 -- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f817405f390 msgr2=0x7f8174061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:01:57.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.347+0000 7f81877fe700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f817405f390 0x7f8174061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:01:57.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.351+0000 7f816f7fe700 1 -- 172.21.15.33:0/2553478798 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f818004d2b0 con 0x7f8188138910 2023-11-06T21:01:57.417 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.417+0000 7f34994a2680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:01:57.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.426+0000 7f816f7fe700 1 -- 172.21.15.33:0/2553478798 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 45) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f8180079db0 con 0x7f8188138910 2023-11-06T21:01:57.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.427+0000 7f816f7fe700 1 -- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f817405f390 msgr2=0x7f8174061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:01:57.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.427+0000 7f816f7fe700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f817405f390 0x7f8174061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:57.493 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.493+0000 7f340ff89700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:57.493 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.493+0000 7f340ff89700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:57.493 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.493+0000 7f340ff89700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:57.494 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.493+0000 7f340ff89700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:57.494 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.493+0000 7f340ff89700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:57.494 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.493+0000 7f3413f91700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:57.495 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.493+0000 7f3413f91700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:57.495 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.493+0000 7f3413f91700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:57.495 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.493+0000 7f3413f91700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:57.495 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:01:57.493+0000 7f3413f91700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:01:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.545+0000 7f818ddff700 1 -- 172.21.15.33:0/2553478798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81880c3800 con 0x7f8188138910 2023-11-06T21:01:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.546+0000 7f816f7fe700 1 -- 172.21.15.33:0/2553478798 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+18162 (secure 0 0 0) 0x7f8180055680 con 0x7f8188138910 2023-11-06T21:01:57.546 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:01:57.547 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":118,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:01:57.420982+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:00:31.623152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T21:00:32.950772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.11","mappings":[{"from":1,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.16","mappings":[{"from":4,"to":3}]},{"pgid":"14.25","mappings":[{"from":7,"to":3}]},{"pgid":"14.27","mappings":[{"from":1,"to":3}]},{"pgid":"14.35","mappings":[{"from":7,"to":0}]},{"pgid":"14.3b","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3384612804":"2023-11-07T21:01:57.420937+0000","172.21.15.69:0/143271597":"2023-11-07T21:01:57.420937+0000","172.21.15.69:0/230091947":"2023-11-07T21:01:57.420937+0000","172.21.15.69:0/3664782963":"2023-11-07T21:01:57.420937+0000","172.21.15.69:0/3993108777":"2023-11-07T21:01:57.420937+0000","172.21.15.69:0/2951161837":"2023-11-07T21:00:48.500569+0000","172.21.15.69:6832/114912":"2023-11-07T21:00:48.500569+0000","172.21.15.69:0/443852199":"2023-11-07T21:00:48.500569+0000","172.21.15.69:6833/114912":"2023-11-07T21:00:48.500569+0000","172.21.15.69:0/398102620":"2023-11-07T21:00:48.500569+0000","172.21.15.69:0/4251593465":"2023-11-07T21:01:57.420937+0000","172.21.15.69:0/4081113363":"2023-11-07T21:00:48.500569+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:01:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.549+0000 7f818ddff700 1 -- 172.21.15.33:0/2553478798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188138910 msgr2=0x7f8188151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.549+0000 7f818ddff700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188138910 0x7f8188151ba0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f818000b610 tx=0x7f8180009780).stop 2023-11-06T21:01:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.549+0000 7f818ddff700 1 -- 172.21.15.33:0/2553478798 shutdown_connections 2023-11-06T21:01:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.549+0000 7f818ddff700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f817405f390 0x7f8174061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:01:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.549+0000 7f818ddff700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188138910 0x7f8188151ba0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.549+0000 7f818ddff700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8188068470 0x7f81880c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.549+0000 7f818ddff700 1 --2- 172.21.15.33:0/2553478798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8188138530 0x7f81880acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.549+0000 7f818ddff700 1 -- 172.21.15.33:0/2553478798 >> 172.21.15.33:0/2553478798 conn(0x7f81880c8c10 msgr2=0x7f81880a9510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:57.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.549+0000 7f818ddff700 1 -- 172.21.15.33:0/2553478798 shutdown_connections 2023-11-06T21:01:57.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.549+0000 7f818ddff700 1 -- 172.21.15.33:0/2553478798 wait complete. 2023-11-06T21:01:57.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T21:01:57.716 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:57.717+0000 7fae27539680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:01:57.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.770+0000 7ff0583ff700 1 Processor -- start 2023-11-06T21:01:57.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.770+0000 7ff0583ff700 1 -- start start 2023-11-06T21:01:57.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.772+0000 7ff0583ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff05014fb70 0x7ff050156770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:57.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.772+0000 7ff0583ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff050155d20 con 0x7ff0500c3410 2023-11-06T21:01:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.772+0000 7ff0583ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0501562e0 con 0x7ff05014ff50 2023-11-06T21:01:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.772+0000 7ff0583ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff050156c60 con 0x7ff05014fb70 2023-11-06T21:01:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.772+0000 7ff05619b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff05014fb70 0x7ff050156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.772+0000 7ff05699c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff0500c3410 0x7ff0500c3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56488/0 (socket says 172.21.15.33:56488) 2023-11-06T21:01:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.772+0000 7ff05699c700 1 -- 172.21.15.33:0/2411091133 learned_addr learned my addr 172.21.15.33:0/2411091133 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.772+0000 7ff05619b700 1 -- 172.21.15.33:0/2411091133 >> v1:172.21.15.69:6789/0 conn(0x7ff05014ff50 legacy=0x7ff050152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.772+0000 7ff05619b700 1 -- 172.21.15.33:0/2411091133 >> v1:172.21.15.33:6789/0 conn(0x7ff0500c3410 legacy=0x7ff0500c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.772+0000 7ff05619b700 1 -- 172.21.15.33:0/2411091133 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff050153280 con 0x7ff05014fb70 2023-11-06T21:01:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.773+0000 7ff05619b700 1 --2- 172.21.15.33:0/2411091133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05014fb70 0x7ff050156770 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7ff04c009760 tx=0x7ff04c009060).ready entity=mon.0 client_cookie=b357e779d2e0231 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.773+0000 7ff055199700 1 -- 172.21.15.33:0/2411091133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00b580 con 0x7ff05014fb70 2023-11-06T21:01:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.773+0000 7ff055199700 1 -- 172.21.15.33:0/2411091133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c00b6e0 con 0x7ff05014fb70 2023-11-06T21:01:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.773+0000 7ff055199700 1 -- 172.21.15.33:0/2411091133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00ba60 con 0x7ff05014fb70 2023-11-06T21:01:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.773+0000 7ff0583ff700 1 -- 172.21.15.33:0/2411091133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05014fb70 msgr2=0x7ff050156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.773+0000 7ff0583ff700 1 --2- 172.21.15.33:0/2411091133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05014fb70 0x7ff050156770 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7ff04c009760 tx=0x7ff04c009060).stop 2023-11-06T21:01:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.774+0000 7ff0583ff700 1 -- 172.21.15.33:0/2411091133 shutdown_connections 2023-11-06T21:01:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.774+0000 7ff0583ff700 1 --2- 172.21.15.33:0/2411091133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05014fb70 0x7ff050156770 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.774+0000 7ff0583ff700 1 -- 172.21.15.33:0/2411091133 >> 172.21.15.33:0/2411091133 conn(0x7ff050062710 msgr2=0x7ff050062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.774+0000 7ff0583ff700 1 -- 172.21.15.33:0/2411091133 shutdown_connections 2023-11-06T21:01:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.774+0000 7ff0583ff700 1 -- 172.21.15.33:0/2411091133 wait complete. 2023-11-06T21:01:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.775+0000 7ff0583ff700 1 Processor -- start 2023-11-06T21:01:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.775+0000 7ff0583ff700 1 -- start start 2023-11-06T21:01:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.776+0000 7ff0583ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0500c3410 0x7ff05014bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.776+0000 7ff0583ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff05014fb70 0x7ff05014c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.776+0000 7ff0583ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff05014ff50 0x7ff0500bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.776+0000 7ff0583ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff050165cc0 con 0x7ff0500c3410 2023-11-06T21:01:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.776+0000 7ff0583ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500c27b0 con 0x7ff05014ff50 2023-11-06T21:01:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.776+0000 7ff0583ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500c1f10 con 0x7ff05014fb70 2023-11-06T21:01:57.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0500c3410 0x7ff05014bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:57.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05699c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff05014ff50 0x7ff0500bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:57.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0500c3410 0x7ff05014bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55038/0 (socket says 172.21.15.33:55038) 2023-11-06T21:01:57.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05619b700 1 -- 172.21.15.33:0/745628088 learned_addr learned my addr 172.21.15.33:0/745628088 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:57.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05599a700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff05014fb70 0x7ff05014c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:57.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05619b700 1 -- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff05014fb70 msgr2=0x7ff05014c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05619b700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff05014fb70 0x7ff05014c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05619b700 1 -- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff05014ff50 msgr2=0x7ff0500bbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:57.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05619b700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff05014ff50 0x7ff0500bbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:57.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05619b700 1 -- 172.21.15.33:0/745628088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0092a0 con 0x7ff0500c3410 2023-11-06T21:01:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05699c700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff05014ff50 0x7ff0500bbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:01:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.777+0000 7ff05619b700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0500c3410 0x7ff05014bed0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7ff04c009eb0 tx=0x7ff04c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.778+0000 7ff0467fc700 1 -- 172.21.15.33:0/745628088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00ba60 con 0x7ff0500c3410 2023-11-06T21:01:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.778+0000 7ff0467fc700 1 -- 172.21.15.33:0/745628088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c000c90 con 0x7ff0500c3410 2023-11-06T21:01:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.778+0000 7ff0467fc700 1 -- 172.21.15.33:0/745628088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c01b620 con 0x7ff0500c3410 2023-11-06T21:01:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.778+0000 7ff0583ff700 1 -- 172.21.15.33:0/745628088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0500c21f0 con 0x7ff0500c3410 2023-11-06T21:01:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.778+0000 7ff0583ff700 1 -- 172.21.15.33:0/745628088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0500bc080 con 0x7ff0500c3410 2023-11-06T21:01:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.779+0000 7ff0583ff700 1 -- 172.21.15.33:0/745628088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff050153560 con 0x7ff0500c3410 2023-11-06T21:01:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.780+0000 7ff0467fc700 1 -- 172.21.15.33:0/745628088 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 45) v1 ==== 75270+0+0 (secure 0 0 0) 0x7ff04c01b9d0 con 0x7ff0500c3410 2023-11-06T21:01:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.781+0000 7ff0467fc700 1 -- 172.21.15.33:0/745628088 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6898+0+0 (secure 0 0 0) 0x7ff04c07bfe0 con 0x7ff0500c3410 2023-11-06T21:01:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:57.785+0000 7ff0467fc700 1 -- 172.21.15.33:0/745628088 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff04c04d060 con 0x7ff0500c3410 2023-11-06T21:01:57.814 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:01:57.815+0000 7fae27539680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:01:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.014+0000 7ff0583ff700 1 -- 172.21.15.33:0/745628088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff050065670 con 0x7ff0500c3410 2023-11-06T21:01:58.424 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T21:01:58.424+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T21:01:58.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.428+0000 7ff0467fc700 1 -- 172.21.15.33:0/745628088 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 46) v1 ==== 75340+0+0 (secure 0 0 0) 0x7ff04c079b60 con 0x7ff0500c3410 2023-11-06T21:01:58.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.429+0000 7ff0467fc700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0340302f0 0x7ff034067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:58.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.429+0000 7ff05599a700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0340302f0 0x7ff034067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:58.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.430+0000 7ff05599a700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0340302f0 0x7ff034067f70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7ff0500bdfb0 tx=0x7ff040007d40).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:58.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.442+0000 7ff0467fc700 1 -- 172.21.15.33:0/745628088 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff04c055a50 con 0x7ff0500c3410 2023-11-06T21:01:58.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 -- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0340302f0 msgr2=0x7ff034067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:58.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0340302f0 0x7ff034067f70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7ff0500bdfb0 tx=0x7ff040007d40).stop 2023-11-06T21:01:58.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 -- 172.21.15.33:0/745628088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0500c3410 msgr2=0x7ff05014bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:58.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0500c3410 0x7ff05014bed0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7ff04c009eb0 tx=0x7ff04c017000).stop 2023-11-06T21:01:58.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 -- 172.21.15.33:0/745628088 shutdown_connections 2023-11-06T21:01:58.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff0340302f0 0x7ff034067f70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:58.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff05014ff50 0x7ff0500bbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:58.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0500c3410 0x7ff05014bed0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:58.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 --2- 172.21.15.33:0/745628088 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff05014fb70 0x7ff05014c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:58.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 -- 172.21.15.33:0/745628088 >> 172.21.15.33:0/745628088 conn(0x7ff050062710 msgr2=0x7ff050153c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:58.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 -- 172.21.15.33:0/745628088 shutdown_connections 2023-11-06T21:01:58.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.445+0000 7ff0583ff700 1 -- 172.21.15.33:0/745628088 wait complete. 2023-11-06T21:01:58.448 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T21:01:58.453 INFO:tasks.ceph.mds.d.smithi069.stderr: -6227> 2023-11-06T21:00:38.456+0000 7fa5801f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T21:01:58.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T21:01:58.496 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T21:01:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.663+0000 7f7f533ff700 1 Processor -- start 2023-11-06T21:01:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.663+0000 7f7f533ff700 1 -- start start 2023-11-06T21:01:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.665+0000 7f7f533ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f4c063840 0x7f7f4c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.665+0000 7f7f533ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4c158c00 con 0x7f7f4c0aaf50 2023-11-06T21:01:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.665+0000 7f7f533ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4c0c7610 con 0x7f7f4c157700 2023-11-06T21:01:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.665+0000 7f7f533ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4c0c18f0 con 0x7f7f4c063840 2023-11-06T21:01:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.665+0000 7f7f5099a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7f4c0aaf50 0x7f7f4c063130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50214/0 (socket says 172.21.15.33:50214) 2023-11-06T21:01:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.665+0000 7f7f5099a700 1 -- 172.21.15.33:0/4088350276 learned_addr learned my addr 172.21.15.33:0/4088350276 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.665+0000 7f7f5199c700 1 --2- 172.21.15.33:0/4088350276 >> v2:172.21.15.33:3300/0 conn(0x7f7f4c063840 0x7f7f4c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:58.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.666+0000 7f7f43fff700 1 -- 172.21.15.33:0/4088350276 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427405983 0 0) 0x7f7f4c0c7610 con 0x7f7f4c157700 2023-11-06T21:01:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.666+0000 7f7f43fff700 1 -- 172.21.15.33:0/4088350276 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f380033f0 con 0x7f7f4c157700 2023-11-06T21:01:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.666+0000 7f7f5199c700 1 -- 172.21.15.33:0/4088350276 >> v1:172.21.15.69:6789/0 conn(0x7f7f4c157700 legacy=0x7f7f4c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.666+0000 7f7f5199c700 1 -- 172.21.15.33:0/4088350276 >> v1:172.21.15.33:6789/0 conn(0x7f7f4c0aaf50 legacy=0x7f7f4c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.666+0000 7f7f43fff700 1 -- 172.21.15.33:0/4088350276 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260729644 0 0) 0x7f7f4c158c00 con 0x7f7f4c0aaf50 2023-11-06T21:01:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.666+0000 7f7f5199c700 1 -- 172.21.15.33:0/4088350276 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c157ff0 con 0x7f7f4c063840 2023-11-06T21:01:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.666+0000 7f7f5199c700 1 --2- 172.21.15.33:0/4088350276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 0x7f7f4c15f0c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f7f44009e60 tx=0x7f7f44009780).ready entity=mon.0 client_cookie=b206d4eb74f0d260 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.666+0000 7f7f43fff700 1 -- 172.21.15.33:0/4088350276 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400e070 con 0x7f7f4c063840 2023-11-06T21:01:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.666+0000 7f7f43fff700 1 -- 172.21.15.33:0/4088350276 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4400bb60 con 0x7f7f4c063840 2023-11-06T21:01:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.666+0000 7f7f43fff700 1 -- 172.21.15.33:0/4088350276 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400bee0 con 0x7f7f4c063840 2023-11-06T21:01:58.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.667+0000 7f7f533ff700 1 -- 172.21.15.33:0/4088350276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 msgr2=0x7f7f4c15f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:58.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.667+0000 7f7f533ff700 1 --2- 172.21.15.33:0/4088350276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 0x7f7f4c15f0c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f7f44009e60 tx=0x7f7f44009780).stop 2023-11-06T21:01:58.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.667+0000 7f7f533ff700 1 -- 172.21.15.33:0/4088350276 shutdown_connections 2023-11-06T21:01:58.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.667+0000 7f7f533ff700 1 --2- 172.21.15.33:0/4088350276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 0x7f7f4c15f0c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:58.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.667+0000 7f7f533ff700 1 -- 172.21.15.33:0/4088350276 >> 172.21.15.33:0/4088350276 conn(0x7f7f4c05ec80 msgr2=0x7f7f4c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:58.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.667+0000 7f7f533ff700 1 -- 172.21.15.33:0/4088350276 shutdown_connections 2023-11-06T21:01:58.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.667+0000 7f7f533ff700 1 -- 172.21.15.33:0/4088350276 wait complete. 2023-11-06T21:01:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.669+0000 7f7f533ff700 1 Processor -- start 2023-11-06T21:01:58.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.669+0000 7f7f533ff700 1 -- start start 2023-11-06T21:01:58.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.670+0000 7f7f533ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 0x7f7f4c151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:58.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.670+0000 7f7f533ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f4c0aaf50 0x7f7f4c0c3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:58.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.670+0000 7f7f533ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f4c157700 0x7f7f4c0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:58.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.670+0000 7f7f533ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c0c2de0 con 0x7f7f4c063840 2023-11-06T21:01:58.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.670+0000 7f7f533ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1522c0 con 0x7f7f4c157700 2023-11-06T21:01:58.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.670+0000 7f7f533ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c0c1ef0 con 0x7f7f4c0aaf50 2023-11-06T21:01:58.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 0x7f7f4c151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:58.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 0x7f7f4c151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50068/0 (socket says 172.21.15.33:50068) 2023-11-06T21:01:58.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5119b700 1 -- 172.21.15.33:0/2692720661 learned_addr learned my addr 172.21.15.33:0/2692720661 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:58.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5099a700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f4c0aaf50 0x7f7f4c0c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:58.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5199c700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f4c157700 0x7f7f4c0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:58.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5119b700 1 -- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f4c0aaf50 msgr2=0x7f7f4c0c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5119b700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f4c0aaf50 0x7f7f4c0c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5119b700 1 -- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f4c157700 msgr2=0x7f7f4c0aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5119b700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f4c157700 0x7f7f4c0aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5119b700 1 -- 172.21.15.33:0/2692720661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f44009a20 con 0x7f7f4c063840 2023-11-06T21:01:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.671+0000 7f7f5119b700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 0x7f7f4c151dd0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f7f4800c440 tx=0x7f7f48007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:58.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.672+0000 7f7f417fa700 1 -- 172.21.15.33:0/2692720661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800f470 con 0x7f7f4c063840 2023-11-06T21:01:58.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.672+0000 7f7f417fa700 1 -- 172.21.15.33:0/2692720661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4800f5d0 con 0x7f7f4c063840 2023-11-06T21:01:58.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.672+0000 7f7f533ff700 1 -- 172.21.15.33:0/2692720661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c0c21d0 con 0x7f7f4c063840 2023-11-06T21:01:58.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.672+0000 7f7f417fa700 1 -- 172.21.15.33:0/2692720661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800f990 con 0x7f7f4c063840 2023-11-06T21:01:58.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.672+0000 7f7f533ff700 1 -- 172.21.15.33:0/2692720661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4c0c2790 con 0x7f7f4c063840 2023-11-06T21:01:58.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.673+0000 7f7f533ff700 1 -- 172.21.15.33:0/2692720661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4c0c30c0 con 0x7f7f4c063840 2023-11-06T21:01:58.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.674+0000 7f7f417fa700 1 -- 172.21.15.33:0/2692720661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 46) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f7f4800fd40 con 0x7f7f4c063840 2023-11-06T21:01:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.675+0000 7f7f417fa700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7f3005f030 0x7f7f300614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.675+0000 7f7f417fa700 1 -- 172.21.15.33:0/2692720661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6984+0+0 (secure 0 0 0) 0x7f7f480799e0 con 0x7f7f4c063840 2023-11-06T21:01:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.675+0000 7f7f5099a700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7f3005f030 0x7f7f300614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.676+0000 7f7f5099a700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7f3005f030 0x7f7f300614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f7f4c0c9900 tx=0x7f7f3c009040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:58.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.678+0000 7f7f417fa700 1 -- 172.21.15.33:0/2692720661 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7f4804aa10 con 0x7f7f4c063840 2023-11-06T21:01:58.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:58.900+0000 7f7f533ff700 1 -- 172.21.15.33:0/2692720661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7f4c06c6c0 con 0x7f7f4c063840 2023-11-06T21:01:59.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.451+0000 7f7f417fa700 1 -- 172.21.15.33:0/2692720661 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7f4804e030 con 0x7f7f4c063840 2023-11-06T21:01:59.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.453+0000 7f7f533ff700 1 -- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7f3005f030 msgr2=0x7f7f300614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:59.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.453+0000 7f7f533ff700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7f3005f030 0x7f7f300614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f7f4c0c9900 tx=0x7f7f3c009040).stop 2023-11-06T21:01:59.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.453+0000 7f7f533ff700 1 -- 172.21.15.33:0/2692720661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 msgr2=0x7f7f4c151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:59.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.453+0000 7f7f533ff700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 0x7f7f4c151dd0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f7f4800c440 tx=0x7f7f48007c90).stop 2023-11-06T21:01:59.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.454+0000 7f7f533ff700 1 -- 172.21.15.33:0/2692720661 shutdown_connections 2023-11-06T21:01:59.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.454+0000 7f7f533ff700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7f3005f030 0x7f7f300614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:59.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.454+0000 7f7f533ff700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f4c157700 0x7f7f4c0aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:59.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.454+0000 7f7f533ff700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f4c063840 0x7f7f4c151dd0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:59.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.454+0000 7f7f533ff700 1 --2- 172.21.15.33:0/2692720661 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f4c0aaf50 0x7f7f4c0c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:59.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.454+0000 7f7f533ff700 1 -- 172.21.15.33:0/2692720661 >> 172.21.15.33:0/2692720661 conn(0x7f7f4c05ec80 msgr2=0x7f7f4c0acf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.454+0000 7f7f533ff700 1 -- 172.21.15.33:0/2692720661 shutdown_connections 2023-11-06T21:01:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.454+0000 7f7f533ff700 1 -- 172.21.15.33:0/2692720661 wait complete. 2023-11-06T21:01:59.472 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T21:01:59.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.674+0000 7f41073ff700 1 Processor -- start 2023-11-06T21:01:59.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.674+0000 7f41073ff700 1 -- start start 2023-11-06T21:01:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.675+0000 7f41073ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f41001514c0 0x7f41000bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:59.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.675+0000 7f41073ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4100154c60 con 0x7f41001510e0 2023-11-06T21:01:59.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.675+0000 7f41073ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4100068f50 con 0x7f41000bbe70 2023-11-06T21:01:59.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.675+0000 7f41073ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41000bc2c0 con 0x7f41001514c0 2023-11-06T21:01:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.676+0000 7f410519b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f41001510e0 0x7f41000a5d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50230/0 (socket says 172.21.15.33:50230) 2023-11-06T21:01:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.676+0000 7f410519b700 1 -- 172.21.15.33:0/372168073 learned_addr learned my addr 172.21.15.33:0/372168073 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:01:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.676+0000 7f410499a700 1 --2- 172.21.15.33:0/372168073 >> v2:172.21.15.69:3300/0 conn(0x7f41001514c0 0x7f41000bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.676+0000 7f40f7fff700 1 -- 172.21.15.33:0/372168073 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060534418 0 0) 0x7f4100154c60 con 0x7f41001510e0 2023-11-06T21:01:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.676+0000 7f40f7fff700 1 -- 172.21.15.33:0/372168073 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40ec005600 con 0x7f41001510e0 2023-11-06T21:01:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.676+0000 7f410499a700 1 -- 172.21.15.33:0/372168073 >> v1:172.21.15.69:6789/0 conn(0x7f41000bbe70 legacy=0x7f410015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.676+0000 7f40f7fff700 1 -- 172.21.15.33:0/372168073 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476987195 0 0) 0x7f4100068f50 con 0x7f41000bbe70 2023-11-06T21:01:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.676+0000 7f410499a700 1 -- 172.21.15.33:0/372168073 >> v1:172.21.15.33:6789/0 conn(0x7f41001510e0 legacy=0x7f41000a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.676+0000 7f410499a700 1 -- 172.21.15.33:0/372168073 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100151dc0 con 0x7f41001514c0 2023-11-06T21:01:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.677+0000 7f410499a700 1 --2- 172.21.15.33:0/372168073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41001514c0 0x7f41000bb980 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f40f0009760 tx=0x7f40f0009060).ready entity=mon.1 client_cookie=5da26ec22fa2f25e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.677+0000 7f40f7fff700 1 -- 172.21.15.33:0/372168073 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f000f510 con 0x7f41001514c0 2023-11-06T21:01:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.677+0000 7f40f7fff700 1 -- 172.21.15.33:0/372168073 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f000f670 con 0x7f41001514c0 2023-11-06T21:01:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.677+0000 7f40f7fff700 1 -- 172.21.15.33:0/372168073 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f000f9f0 con 0x7f41001514c0 2023-11-06T21:01:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.677+0000 7f41073ff700 1 -- 172.21.15.33:0/372168073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41001514c0 msgr2=0x7f41000bb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.677+0000 7f41073ff700 1 --2- 172.21.15.33:0/372168073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41001514c0 0x7f41000bb980 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f40f0009760 tx=0x7f40f0009060).stop 2023-11-06T21:01:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.677+0000 7f41073ff700 1 -- 172.21.15.33:0/372168073 shutdown_connections 2023-11-06T21:01:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.677+0000 7f41073ff700 1 --2- 172.21.15.33:0/372168073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41001514c0 0x7f41000bb980 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.677+0000 7f41073ff700 1 -- 172.21.15.33:0/372168073 >> 172.21.15.33:0/372168073 conn(0x7f4100062710 msgr2=0x7f4100062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:01:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.678+0000 7f41073ff700 1 -- 172.21.15.33:0/372168073 shutdown_connections 2023-11-06T21:01:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.678+0000 7f41073ff700 1 -- 172.21.15.33:0/372168073 wait complete. 2023-11-06T21:01:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.679+0000 7f41073ff700 1 Processor -- start 2023-11-06T21:01:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.679+0000 7f41073ff700 1 -- start start 2023-11-06T21:01:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f41073ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000bbe70 0x7f410014e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f41073ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41001510e0 0x7f4100164a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f41073ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41001514c0 0x7f41000a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f41073ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41000bf110 con 0x7f41000bbe70 2023-11-06T21:01:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f41073ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410014ec30 con 0x7f41001514c0 2023-11-06T21:01:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f41073ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41000bede0 con 0x7f41001510e0 2023-11-06T21:01:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f410519b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000bbe70 0x7f410014e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:59.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f410519b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000bbe70 0x7f410014e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50072/0 (socket says 172.21.15.33:50072) 2023-11-06T21:01:59.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f410519b700 1 -- 172.21.15.33:0/4100451991 learned_addr learned my addr 172.21.15.33:0/4100451991 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:01:59.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f410499a700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41001510e0 0x7f4100164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:59.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.680+0000 7f410599c700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41001514c0 0x7f41000a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:59.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f410519b700 1 -- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41001510e0 msgr2=0x7f4100164a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:59.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f410519b700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41001510e0 0x7f4100164a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:59.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f410519b700 1 -- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41001514c0 msgr2=0x7f41000a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:01:59.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f410519b700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41001514c0 0x7f41000a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:01:59.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f410519b700 1 -- 172.21.15.33:0/4100451991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f00092a0 con 0x7f41000bbe70 2023-11-06T21:01:59.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f410519b700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000bbe70 0x7f410014e740 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f40fc00c490 tx=0x7f40fc007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:59.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f40f57fa700 1 -- 172.21.15.33:0/4100451991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc00ce00 con 0x7f41000bbe70 2023-11-06T21:01:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f40f57fa700 1 -- 172.21.15.33:0/4100451991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40fc0072e0 con 0x7f41000bbe70 2023-11-06T21:01:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f40f57fa700 1 -- 172.21.15.33:0/4100451991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0057a0 con 0x7f41000bbe70 2023-11-06T21:01:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f41073ff700 1 -- 172.21.15.33:0/4100451991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41001653b0 con 0x7f41000bbe70 2023-11-06T21:01:59.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.681+0000 7f41073ff700 1 -- 172.21.15.33:0/4100451991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4100165970 con 0x7f41000bbe70 2023-11-06T21:01:59.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.682+0000 7f41073ff700 1 -- 172.21.15.33:0/4100451991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41001520a0 con 0x7f41000bbe70 2023-11-06T21:01:59.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.683+0000 7f40f57fa700 1 -- 172.21.15.33:0/4100451991 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 46) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f40fc005b50 con 0x7f41000bbe70 2023-11-06T21:01:59.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.684+0000 7f40f57fa700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f40e405f030 0x7f40e40614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:01:59.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.684+0000 7f40f57fa700 1 -- 172.21.15.33:0/4100451991 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6984+0+0 (secure 0 0 0) 0x7f40fc079a30 con 0x7f41000bbe70 2023-11-06T21:01:59.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.685+0000 7f410499a700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f40e405f030 0x7f40e40614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:01:59.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.685+0000 7f410499a700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f40e405f030 0x7f40e40614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f41000c61d0 tx=0x7f40f0017000).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:01:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.688+0000 7f40f57fa700 1 -- 172.21.15.33:0/4100451991 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f40fc04aa60 con 0x7f41000bbe70 2023-11-06T21:01:59.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:01:59.871+0000 7f41073ff700 1 -- 172.21.15.33:0/4100451991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4100065570 con 0x7f41000bbe70 2023-11-06T21:02:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.218+0000 7f40f57fa700 1 -- 172.21.15.33:0/4100451991 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f40fc077560 con 0x7f41000bbe70 2023-11-06T21:02:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.455+0000 7f40f57fa700 1 -- 172.21.15.33:0/4100451991 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 181+0+0 (secure 0 0 0) 0x7f40fc01da20 con 0x7f41000bbe70 2023-11-06T21:02:00.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.459+0000 7f41073ff700 1 -- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f40e405f030 msgr2=0x7f40e40614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:00.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.459+0000 7f41073ff700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f40e405f030 0x7f40e40614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f41000c61d0 tx=0x7f40f0017000).stop 2023-11-06T21:02:00.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.459+0000 7f41073ff700 1 -- 172.21.15.33:0/4100451991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000bbe70 msgr2=0x7f410014e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:00.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.459+0000 7f41073ff700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000bbe70 0x7f410014e740 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f40fc00c490 tx=0x7f40fc007c70).stop 2023-11-06T21:02:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.460+0000 7f41073ff700 1 -- 172.21.15.33:0/4100451991 shutdown_connections 2023-11-06T21:02:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.460+0000 7f41073ff700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f40e405f030 0x7f40e40614f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.460+0000 7f41073ff700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41001514c0 0x7f41000a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.460+0000 7f41073ff700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000bbe70 0x7f410014e740 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.460+0000 7f41073ff700 1 --2- 172.21.15.33:0/4100451991 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f41001510e0 0x7f4100164a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.460+0000 7f41073ff700 1 -- 172.21.15.33:0/4100451991 >> 172.21.15.33:0/4100451991 conn(0x7f4100062710 msgr2=0x7f410014fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.461+0000 7f41073ff700 1 -- 172.21.15.33:0/4100451991 shutdown_connections 2023-11-06T21:02:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.461+0000 7f41073ff700 1 -- 172.21.15.33:0/4100451991 wait complete. 2023-11-06T21:02:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T21:02:00.482 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T21:02:00.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.687+0000 7f99b1bff700 1 Processor -- start 2023-11-06T21:02:00.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.688+0000 7f99b1bff700 1 -- start start 2023-11-06T21:02:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.689+0000 7f99b1bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f99ac153ed0 0x7f99ac158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.689+0000 7f99b1bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99ac1542b0 0x7f99ac0c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.689+0000 7f99b1bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac0c9f80 con 0x7f99ac0c8ee0 2023-11-06T21:02:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.689+0000 7f99b1bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac0cab20 con 0x7f99ac1542b0 2023-11-06T21:02:00.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.689+0000 7f99b1bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac0c6000 con 0x7f99ac153ed0 2023-11-06T21:02:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.689+0000 7f99abfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99ac1542b0 0x7f99ac0c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.689+0000 7f99abfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99ac1542b0 0x7f99ac0c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50082/0 (socket says 172.21.15.33:50082) 2023-11-06T21:02:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.689+0000 7f99abfff700 1 -- 172.21.15.33:0/3572349067 learned_addr learned my addr 172.21.15.33:0/3572349067 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.689+0000 7f99ab7fe700 1 --2- 172.21.15.33:0/3572349067 >> v2:172.21.15.69:3300/0 conn(0x7f99ac153ed0 0x7f99ac158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.690+0000 7f99abfff700 1 -- 172.21.15.33:0/3572349067 >> v2:172.21.15.69:3300/0 conn(0x7f99ac153ed0 msgr2=0x7f99ac158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.690+0000 7f99abfff700 1 --2- 172.21.15.33:0/3572349067 >> v2:172.21.15.69:3300/0 conn(0x7f99ac153ed0 0x7f99ac158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.690+0000 7f99abfff700 1 -- 172.21.15.33:0/3572349067 >> v1:172.21.15.69:6789/0 conn(0x7f99ac0c8ee0 legacy=0x7f99ac0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.690+0000 7f99abfff700 1 -- 172.21.15.33:0/3572349067 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0c9370 con 0x7f99ac1542b0 2023-11-06T21:02:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.690+0000 7f99abfff700 1 --2- 172.21.15.33:0/3572349067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac1542b0 0x7f99ac0c76f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f99ac06bdb0 tx=0x7f99a4009840).ready entity=mon.0 client_cookie=fe75d883dbdff94e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.690+0000 7f99aa7fc700 1 -- 172.21.15.33:0/3572349067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a400e070 con 0x7f99ac1542b0 2023-11-06T21:02:00.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.690+0000 7f99aa7fc700 1 -- 172.21.15.33:0/3572349067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a400ba50 con 0x7f99ac1542b0 2023-11-06T21:02:00.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.690+0000 7f99aa7fc700 1 -- 172.21.15.33:0/3572349067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a400bdd0 con 0x7f99ac1542b0 2023-11-06T21:02:00.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.691+0000 7f99b1bff700 1 -- 172.21.15.33:0/3572349067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac1542b0 msgr2=0x7f99ac0c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:00.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.691+0000 7f99b1bff700 1 --2- 172.21.15.33:0/3572349067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac1542b0 0x7f99ac0c76f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f99ac06bdb0 tx=0x7f99a4009840).stop 2023-11-06T21:02:00.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.691+0000 7f99b1bff700 1 -- 172.21.15.33:0/3572349067 shutdown_connections 2023-11-06T21:02:00.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.691+0000 7f99b1bff700 1 --2- 172.21.15.33:0/3572349067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac1542b0 0x7f99ac0c76f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:00.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.691+0000 7f99b1bff700 1 --2- 172.21.15.33:0/3572349067 >> v2:172.21.15.69:3300/0 conn(0x7f99ac153ed0 0x7f99ac158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:00.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.691+0000 7f99b1bff700 1 -- 172.21.15.33:0/3572349067 >> 172.21.15.33:0/3572349067 conn(0x7f99ac05e1b0 msgr2=0x7f99ac05e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:00.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.691+0000 7f99b1bff700 1 -- 172.21.15.33:0/3572349067 shutdown_connections 2023-11-06T21:02:00.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.691+0000 7f99b1bff700 1 -- 172.21.15.33:0/3572349067 wait complete. 2023-11-06T21:02:00.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.692+0000 7f99b1bff700 1 Processor -- start 2023-11-06T21:02:00.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.693+0000 7f99b1bff700 1 -- start start 2023-11-06T21:02:00.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.693+0000 7f99b1bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac153ed0 0x7f99ac13c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.693+0000 7f99b1bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99ac1542b0 0x7f99ac0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.693+0000 7f99b1bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99ac0c2490 0x7f99ac152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.693+0000 7f99b1bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0b0d80 con 0x7f99ac153ed0 2023-11-06T21:02:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.693+0000 7f99b1bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0c2a00 con 0x7f99ac1542b0 2023-11-06T21:02:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.693+0000 7f99b1bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0c2ce0 con 0x7f99ac0c2490 2023-11-06T21:02:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac153ed0 0x7f99ac13c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac153ed0 0x7f99ac13c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50094/0 (socket says 172.21.15.33:50094) 2023-11-06T21:02:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99ab7fe700 1 -- 172.21.15.33:0/1447352841 learned_addr learned my addr 172.21.15.33:0/1447352841 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99aaffd700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99ac1542b0 0x7f99ac0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99abfff700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99ac0c2490 0x7f99ac152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:00.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99ab7fe700 1 -- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99ac0c2490 msgr2=0x7f99ac152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:00.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99ab7fe700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99ac0c2490 0x7f99ac152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:00.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99ab7fe700 1 -- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99ac1542b0 msgr2=0x7f99ac0b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:00.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99ab7fe700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99ac1542b0 0x7f99ac0b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:00.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99ab7fe700 1 -- 172.21.15.33:0/1447352841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a4009ae0 con 0x7f99ac153ed0 2023-11-06T21:02:00.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.694+0000 7f99ab7fe700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac153ed0 0x7f99ac13c5a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f999c00d890 tx=0x7f999c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:00.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.695+0000 7f99937fe700 1 -- 172.21.15.33:0/1447352841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c0125f0 con 0x7f99ac153ed0 2023-11-06T21:02:00.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.695+0000 7f99937fe700 1 -- 172.21.15.33:0/1447352841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c012750 con 0x7f99ac153ed0 2023-11-06T21:02:00.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.695+0000 7f99937fe700 1 -- 172.21.15.33:0/1447352841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c012ad0 con 0x7f99ac153ed0 2023-11-06T21:02:00.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.695+0000 7f99b1bff700 1 -- 172.21.15.33:0/1447352841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac0c2fc0 con 0x7f99ac153ed0 2023-11-06T21:02:00.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.695+0000 7f99b1bff700 1 -- 172.21.15.33:0/1447352841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac0c3580 con 0x7f99ac153ed0 2023-11-06T21:02:00.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.696+0000 7f99b1bff700 1 -- 172.21.15.33:0/1447352841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac06c6c0 con 0x7f99ac153ed0 2023-11-06T21:02:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.697+0000 7f99937fe700 1 -- 172.21.15.33:0/1447352841 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f999c013480 con 0x7f99ac153ed0 2023-11-06T21:02:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.697+0000 7f99937fe700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f999805f270 0x7f9998061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.698+0000 7f99937fe700 1 -- 172.21.15.33:0/1447352841 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f999c07a460 con 0x7f99ac153ed0 2023-11-06T21:02:00.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.698+0000 7f99aaffd700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f999805f270 0x7f9998061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:00.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.699+0000 7f99aaffd700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f999805f270 0x7f9998061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f99ac1536d0 tx=0x7f99a0009040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:00.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.702+0000 7f99937fe700 1 -- 172.21.15.33:0/1447352841 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f999c04b2e0 con 0x7f99ac153ed0 2023-11-06T21:02:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:00.883+0000 7f99b1bff700 1 -- 172.21.15.33:0/1447352841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99ac0c9650 con 0x7f99ac153ed0 2023-11-06T21:02:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.459+0000 7f99937fe700 1 -- 172.21.15.33:0/1447352841 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) v1 ==== 169+0+0 (secure 0 0 0) 0x7f999c04e900 con 0x7f99ac153ed0 2023-11-06T21:02:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.461+0000 7f99b1bff700 1 -- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f999805f270 msgr2=0x7f9998061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:01.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.461+0000 7f99b1bff700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f999805f270 0x7f9998061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f99ac1536d0 tx=0x7f99a0009040).stop 2023-11-06T21:02:01.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.461+0000 7f99b1bff700 1 -- 172.21.15.33:0/1447352841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac153ed0 msgr2=0x7f99ac13c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:01.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.461+0000 7f99b1bff700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac153ed0 0x7f99ac13c5a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f999c00d890 tx=0x7f999c008040).stop 2023-11-06T21:02:01.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.462+0000 7f99b1bff700 1 -- 172.21.15.33:0/1447352841 shutdown_connections 2023-11-06T21:02:01.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.462+0000 7f99b1bff700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f999805f270 0x7f9998061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.462+0000 7f99b1bff700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99ac0c2490 0x7f99ac152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.462+0000 7f99b1bff700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99ac153ed0 0x7f99ac13c5a0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.462+0000 7f99b1bff700 1 --2- 172.21.15.33:0/1447352841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99ac1542b0 0x7f99ac0b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.462+0000 7f99b1bff700 1 -- 172.21.15.33:0/1447352841 >> 172.21.15.33:0/1447352841 conn(0x7f99ac05e1b0 msgr2=0x7f99ac0aabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.462+0000 7f99b1bff700 1 -- 172.21.15.33:0/1447352841 shutdown_connections 2023-11-06T21:02:01.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.462+0000 7f99b1bff700 1 -- 172.21.15.33:0/1447352841 wait complete. 2023-11-06T21:02:01.464 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T21:02:01.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T21:02:01.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.681+0000 7f47a13ff700 1 Processor -- start 2023-11-06T21:02:01.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.681+0000 7f47a13ff700 1 -- start start 2023-11-06T21:02:01.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.683+0000 7f47a13ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f479c0aaf50 0x7f479c063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:01.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.683+0000 7f47a13ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 0x7f479c063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:01.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.683+0000 7f47a13ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479c158c00 con 0x7f479c157700 2023-11-06T21:02:01.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.683+0000 7f47a13ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479c0c18f0 con 0x7f479c0aaf50 2023-11-06T21:02:01.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.683+0000 7f47a13ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c063b10 con 0x7f479c0aa4f0 2023-11-06T21:02:01.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.683+0000 7f479a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f479c0aaf50 0x7f479c063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.683+0000 7f479a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f479c0aaf50 0x7f479c063130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50106/0 (socket says 172.21.15.33:50106) 2023-11-06T21:02:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.683+0000 7f479a7fc700 1 -- 172.21.15.33:0/3536780927 learned_addr learned my addr 172.21.15.33:0/3536780927 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.683+0000 7f479b7fe700 1 --2- 172.21.15.33:0/3536780927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 0x7f479c063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.684+0000 7f479a7fc700 1 -- 172.21.15.33:0/3536780927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 msgr2=0x7f479c063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.684+0000 7f479a7fc700 1 --2- 172.21.15.33:0/3536780927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 0x7f479c063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.684+0000 7f479a7fc700 1 -- 172.21.15.33:0/3536780927 >> v1:172.21.15.69:6789/0 conn(0x7f479c157700 legacy=0x7f479c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:01.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.684+0000 7f479a7fc700 1 -- 172.21.15.33:0/3536780927 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0c7610 con 0x7f479c0aaf50 2023-11-06T21:02:01.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.684+0000 7f479a7fc700 1 --2- 172.21.15.33:0/3536780927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c0aaf50 0x7f479c063130 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f4790006fd0 tx=0x7f47900090f0).ready entity=mon.0 client_cookie=26180d54d1c6c24c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:01.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.684+0000 7f4799ffb700 1 -- 172.21.15.33:0/3536780927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479000b580 con 0x7f479c0aaf50 2023-11-06T21:02:01.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.684+0000 7f4799ffb700 1 -- 172.21.15.33:0/3536780927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479000b6e0 con 0x7f479c0aaf50 2023-11-06T21:02:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.684+0000 7f4799ffb700 1 -- 172.21.15.33:0/3536780927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479000ba60 con 0x7f479c0aaf50 2023-11-06T21:02:01.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.685+0000 7f47a13ff700 1 -- 172.21.15.33:0/3536780927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c0aaf50 msgr2=0x7f479c063130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:01.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.685+0000 7f47a13ff700 1 --2- 172.21.15.33:0/3536780927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c0aaf50 0x7f479c063130 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f4790006fd0 tx=0x7f47900090f0).stop 2023-11-06T21:02:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.685+0000 7f47a13ff700 1 -- 172.21.15.33:0/3536780927 shutdown_connections 2023-11-06T21:02:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.685+0000 7f47a13ff700 1 --2- 172.21.15.33:0/3536780927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 0x7f479c063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.685+0000 7f47a13ff700 1 --2- 172.21.15.33:0/3536780927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c0aaf50 0x7f479c063130 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.685+0000 7f47a13ff700 1 -- 172.21.15.33:0/3536780927 >> 172.21.15.33:0/3536780927 conn(0x7f479c05ec80 msgr2=0x7f479c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.685+0000 7f47a13ff700 1 -- 172.21.15.33:0/3536780927 shutdown_connections 2023-11-06T21:02:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.685+0000 7f47a13ff700 1 -- 172.21.15.33:0/3536780927 wait complete. 2023-11-06T21:02:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.686+0000 7f47a13ff700 1 Processor -- start 2023-11-06T21:02:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.686+0000 7f47a13ff700 1 -- start start 2023-11-06T21:02:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.687+0000 7f47a13ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 0x7f479c0afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.687+0000 7f47a13ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f479c0aaf50 0x7f479c0c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.687+0000 7f47a13ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c157700 0x7f479c149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.687+0000 7f47a13ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0b01e0 con 0x7f479c157700 2023-11-06T21:02:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.687+0000 7f47a13ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c1568e0 con 0x7f479c0aaf50 2023-11-06T21:02:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.687+0000 7f47a13ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0ade00 con 0x7f479c0aa4f0 2023-11-06T21:02:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c157700 0x7f479c149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c157700 0x7f479c149130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50116/0 (socket says 172.21.15.33:50116) 2023-11-06T21:02:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479b7fe700 1 -- 172.21.15.33:0/3317869678 learned_addr learned my addr 172.21.15.33:0/3317869678 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479b7fe700 1 -- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 msgr2=0x7f479c0afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479a7fc700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f479c0aaf50 0x7f479c0c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479affd700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 0x7f479c0afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479b7fe700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 0x7f479c0afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479b7fe700 1 -- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f479c0aaf50 msgr2=0x7f479c0c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479b7fe700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f479c0aaf50 0x7f479c0c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479b7fe700 1 -- 172.21.15.33:0/3317869678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4790009370 con 0x7f479c157700 2023-11-06T21:02:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479a7fc700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f479c0aaf50 0x7f479c0c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:01.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479affd700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 0x7f479c0afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:01.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.688+0000 7f479b7fe700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c157700 0x7f479c149130 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f4794009ef0 tx=0x7f479400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:01.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.689+0000 7f4782ffd700 1 -- 172.21.15.33:0/3317869678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479400cc50 con 0x7f479c157700 2023-11-06T21:02:01.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.689+0000 7f4782ffd700 1 -- 172.21.15.33:0/3317869678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479400f040 con 0x7f479c157700 2023-11-06T21:02:01.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.689+0000 7f4782ffd700 1 -- 172.21.15.33:0/3317869678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794013400 con 0x7f479c157700 2023-11-06T21:02:01.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.689+0000 7f47a13ff700 1 -- 172.21.15.33:0/3317869678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479c0ae3c0 con 0x7f479c157700 2023-11-06T21:02:01.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.689+0000 7f47a13ff700 1 -- 172.21.15.33:0/3317869678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479c156e10 con 0x7f479c157700 2023-11-06T21:02:01.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.690+0000 7f47a13ff700 1 -- 172.21.15.33:0/3317869678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479c0ae550 con 0x7f479c157700 2023-11-06T21:02:01.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.691+0000 7f4782ffd700 1 -- 172.21.15.33:0/3317869678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4794013770 con 0x7f479c157700 2023-11-06T21:02:01.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.692+0000 7f4782ffd700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f478805f270 0x7f4788061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:01.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.692+0000 7f4782ffd700 1 -- 172.21.15.33:0/3317869678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5871+0+0 (secure 0 0 0) 0x7f479407ad30 con 0x7f479c157700 2023-11-06T21:02:01.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.692+0000 7f479affd700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f478805f270 0x7f4788061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:01.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.693+0000 7f479affd700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f478805f270 0x7f4788061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f478c0015e0 tx=0x7f478c009040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:01.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.696+0000 7f4782ffd700 1 -- 172.21.15.33:0/3317869678 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f479404bec0 con 0x7f479c157700 2023-11-06T21:02:01.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.889+0000 7f47a13ff700 1 -- 172.21.15.33:0/3317869678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f479c06c6c0 con 0x7f479c157700 2023-11-06T21:02:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.893+0000 7f4782ffd700 1 -- 172.21.15.33:0/3317869678 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v20) v1 ==== 120+0+0 (secure 0 0 0) 0x7f479404f4e0 con 0x7f479c157700 2023-11-06T21:02:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 -- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f478805f270 msgr2=0x7f4788061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:01.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f478805f270 0x7f4788061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f478c0015e0 tx=0x7f478c009040).stop 2023-11-06T21:02:01.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 -- 172.21.15.33:0/3317869678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c157700 msgr2=0x7f479c149130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:01.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c157700 0x7f479c149130 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f4794009ef0 tx=0x7f479400b040).stop 2023-11-06T21:02:01.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 -- 172.21.15.33:0/3317869678 shutdown_connections 2023-11-06T21:02:01.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f478805f270 0x7f4788061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f479c0aa4f0 0x7f479c0afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f479c157700 0x7f479c149130 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 --2- 172.21.15.33:0/3317869678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f479c0aaf50 0x7f479c0c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 -- 172.21.15.33:0/3317869678 >> 172.21.15.33:0/3317869678 conn(0x7f479c05ec80 msgr2=0x7f479c0ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 -- 172.21.15.33:0/3317869678 shutdown_connections 2023-11-06T21:02:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:01.896+0000 7f47a13ff700 1 -- 172.21.15.33:0/3317869678 wait complete. 2023-11-06T21:02:01.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T21:02:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.112+0000 7f8a431ff700 1 Processor -- start 2023-11-06T21:02:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.112+0000 7f8a431ff700 1 -- start start 2023-11-06T21:02:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.114+0000 7f8a431ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8a3c066620 0x7f8a3c0c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.114+0000 7f8a431ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8a3c158780 0x7f8a3c0681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.114+0000 7f8a431ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3c0a8330 con 0x7f8a3c066a00 2023-11-06T21:02:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.114+0000 7f8a431ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3c0be210 con 0x7f8a3c158780 2023-11-06T21:02:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.114+0000 7f8a431ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0bdc50 con 0x7f8a3c066620 2023-11-06T21:02:02.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.114+0000 7f8a3bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8a3c066a00 0x7f8a3c067ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50238/0 (socket says 172.21.15.33:50238) 2023-11-06T21:02:02.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.114+0000 7f8a3bfff700 1 -- 172.21.15.33:0/4150883166 learned_addr learned my addr 172.21.15.33:0/4150883166 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:02.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.114+0000 7f8a4179c700 1 --2- 172.21.15.33:0/4150883166 >> v2:172.21.15.69:3300/0 conn(0x7f8a3c158780 0x7f8a3c0681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.114+0000 7f8a40f9b700 1 --2- 172.21.15.33:0/4150883166 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8a3c066620 0x7f8a3c0c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.114+0000 7f8a3b7fe700 1 -- 172.21.15.33:0/4150883166 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253465710 0 0) 0x7f8a3c0a8330 con 0x7f8a3c066a00 2023-11-06T21:02:02.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.115+0000 7f8a3b7fe700 1 -- 172.21.15.33:0/4150883166 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a2c0033f0 con 0x7f8a3c066a00 2023-11-06T21:02:02.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.115+0000 7f8a4179c700 1 -- 172.21.15.33:0/4150883166 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8a3c066620 msgr2=0x7f8a3c0c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:02.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.115+0000 7f8a4179c700 1 --2- 172.21.15.33:0/4150883166 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8a3c066620 0x7f8a3c0c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.115+0000 7f8a4179c700 1 -- 172.21.15.33:0/4150883166 >> v1:172.21.15.33:6789/0 conn(0x7f8a3c066a00 legacy=0x7f8a3c067ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:02.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.115+0000 7f8a4179c700 1 -- 172.21.15.33:0/4150883166 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0bdf30 con 0x7f8a3c158780 2023-11-06T21:02:02.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.115+0000 7f8a4179c700 1 --2- 172.21.15.33:0/4150883166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a3c158780 0x7f8a3c0681e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f8a34009e60 tx=0x7f8a34009780).ready entity=mon.1 client_cookie=9028986671bae08b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:02.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.115+0000 7f8a3b7fe700 1 -- 172.21.15.33:0/4150883166 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400e070 con 0x7f8a3c158780 2023-11-06T21:02:02.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.115+0000 7f8a3b7fe700 1 -- 172.21.15.33:0/4150883166 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3400bab0 con 0x7f8a3c158780 2023-11-06T21:02:02.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.115+0000 7f8a3b7fe700 1 -- 172.21.15.33:0/4150883166 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400be30 con 0x7f8a3c158780 2023-11-06T21:02:02.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.116+0000 7f8a431ff700 1 -- 172.21.15.33:0/4150883166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a3c158780 msgr2=0x7f8a3c0681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:02.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.116+0000 7f8a431ff700 1 --2- 172.21.15.33:0/4150883166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a3c158780 0x7f8a3c0681e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f8a34009e60 tx=0x7f8a34009780).stop 2023-11-06T21:02:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.116+0000 7f8a431ff700 1 -- 172.21.15.33:0/4150883166 shutdown_connections 2023-11-06T21:02:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.116+0000 7f8a431ff700 1 --2- 172.21.15.33:0/4150883166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a3c158780 0x7f8a3c0681e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.116+0000 7f8a431ff700 1 --2- 172.21.15.33:0/4150883166 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8a3c066620 0x7f8a3c0c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.116+0000 7f8a431ff700 1 -- 172.21.15.33:0/4150883166 >> 172.21.15.33:0/4150883166 conn(0x7f8a3c0c8c50 msgr2=0x7f8a3c0c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.116+0000 7f8a431ff700 1 -- 172.21.15.33:0/4150883166 shutdown_connections 2023-11-06T21:02:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.116+0000 7f8a431ff700 1 -- 172.21.15.33:0/4150883166 wait complete. 2023-11-06T21:02:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.118+0000 7f8a431ff700 1 Processor -- start 2023-11-06T21:02:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.118+0000 7f8a431ff700 1 -- start start 2023-11-06T21:02:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a431ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a3c066620 0x7f8a3c0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a431ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8a3c066a00 0x7f8a3c14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a431ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a3c158780 0x7f8a3c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a431ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c14c020 con 0x7f8a3c158780 2023-11-06T21:02:02.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a431ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0a7330 con 0x7f8a3c066620 2023-11-06T21:02:02.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a431ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0a5cb0 con 0x7f8a3c066a00 2023-11-06T21:02:02.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a4179c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a3c158780 0x7f8a3c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a4179c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a3c158780 0x7f8a3c0a57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50126/0 (socket says 172.21.15.33:50126) 2023-11-06T21:02:02.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a4179c700 1 -- 172.21.15.33:0/1950689798 learned_addr learned my addr 172.21.15.33:0/1950689798 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:02.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a40f9b700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a3c066620 0x7f8a3c0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.119+0000 7f8a3bfff700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8a3c066a00 0x7f8a3c14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a4179c700 1 -- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8a3c066a00 msgr2=0x7f8a3c14bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a4179c700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8a3c066a00 0x7f8a3c14bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a4179c700 1 -- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a3c066620 msgr2=0x7f8a3c0a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a4179c700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a3c066620 0x7f8a3c0a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a4179c700 1 -- 172.21.15.33:0/1950689798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a20 con 0x7f8a3c158780 2023-11-06T21:02:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a40f9b700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a3c066620 0x7f8a3c0a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:02:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a4179c700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a3c158780 0x7f8a3c0a57c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f8a3c069600 tx=0x7f8a34009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a38ff9700 1 -- 172.21.15.33:0/1950689798 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3401d630 con 0x7f8a3c158780 2023-11-06T21:02:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a38ff9700 1 -- 172.21.15.33:0/1950689798 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3401d790 con 0x7f8a3c158780 2023-11-06T21:02:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a431ff700 1 -- 172.21.15.33:0/1950689798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c0a6720 con 0x7f8a3c158780 2023-11-06T21:02:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a38ff9700 1 -- 172.21.15.33:0/1950689798 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3401db30 con 0x7f8a3c158780 2023-11-06T21:02:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.120+0000 7f8a431ff700 1 -- 172.21.15.33:0/1950689798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c0a6ce0 con 0x7f8a3c158780 2023-11-06T21:02:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.121+0000 7f8a431ff700 1 -- 172.21.15.33:0/1950689798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a3c0a74c0 con 0x7f8a3c158780 2023-11-06T21:02:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.122+0000 7f8a38ff9700 1 -- 172.21.15.33:0/1950689798 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8a34003040 con 0x7f8a3c158780 2023-11-06T21:02:02.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.123+0000 7f8a38ff9700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8a2005f270 0x7f8a20061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.123+0000 7f8a38ff9700 1 -- 172.21.15.33:0/1950689798 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5871+0+0 (secure 0 0 0) 0x7f8a3407c320 con 0x7f8a3c158780 2023-11-06T21:02:02.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.124+0000 7f8a40f9b700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8a2005f270 0x7f8a20061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.124+0000 7f8a40f9b700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8a2005f270 0x7f8a20061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f8a30009c00 tx=0x7f8a30008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:02.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.127+0000 7f8a38ff9700 1 -- 172.21.15.33:0/1950689798 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8a3404d4b0 con 0x7f8a3c158780 2023-11-06T21:02:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.308+0000 7f8a431ff700 1 -- 172.21.15.33:0/1950689798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8a3c0c53e0 con 0x7f8a3c158780 2023-11-06T21:02:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.311+0000 7f8a38ff9700 1 -- 172.21.15.33:0/1950689798 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8a34050ad0 con 0x7f8a3c158780 2023-11-06T21:02:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 -- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8a2005f270 msgr2=0x7f8a20061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8a2005f270 0x7f8a20061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f8a30009c00 tx=0x7f8a30008040).stop 2023-11-06T21:02:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 -- 172.21.15.33:0/1950689798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a3c158780 msgr2=0x7f8a3c0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a3c158780 0x7f8a3c0a57c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f8a3c069600 tx=0x7f8a34009780).stop 2023-11-06T21:02:02.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 -- 172.21.15.33:0/1950689798 shutdown_connections 2023-11-06T21:02:02.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8a2005f270 0x7f8a20061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a3c158780 0x7f8a3c0a57c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a3c066620 0x7f8a3c0a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 --2- 172.21.15.33:0/1950689798 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8a3c066a00 0x7f8a3c14bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 -- 172.21.15.33:0/1950689798 >> 172.21.15.33:0/1950689798 conn(0x7f8a3c0c8c50 msgr2=0x7f8a3c0c7120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 -- 172.21.15.33:0/1950689798 shutdown_connections 2023-11-06T21:02:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.313+0000 7f8a431ff700 1 -- 172.21.15.33:0/1950689798 wait complete. 2023-11-06T21:02:02.330 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-11-06T21:02:02.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.528+0000 7fcdbb3ff700 1 Processor -- start 2023-11-06T21:02:02.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.528+0000 7fcdbb3ff700 1 -- start start 2023-11-06T21:02:02.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdbb3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcdb40c3600 0x7fcdb40c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdbb3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcdb4136f50 0x7fcdb4137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdbb3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcdb4069410 0x7fcdb4069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdbb3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb40c17e0 con 0x7fcdb4136f50 2023-11-06T21:02:02.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdbb3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb406a740 con 0x7fcdb40c3600 2023-11-06T21:02:02.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdbb3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb406a180 con 0x7fcdb4069410 2023-11-06T21:02:02.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdb899a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcdb4136f50 0x7fcdb4137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdb899a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcdb4136f50 0x7fcdb4137400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50142/0 (socket says 172.21.15.33:50142) 2023-11-06T21:02:02.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdb899a700 1 -- 172.21.15.33:0/729083102 learned_addr learned my addr 172.21.15.33:0/729083102 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:02.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdb919b700 1 --2- 172.21.15.33:0/729083102 >> v2:172.21.15.69:3300/0 conn(0x7fcdb40c3600 0x7fcdb40c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.530+0000 7fcdb999c700 1 --2- 172.21.15.33:0/729083102 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcdb4069410 0x7fcdb4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.531+0000 7fcdb899a700 1 -- 172.21.15.33:0/729083102 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcdb4069410 msgr2=0x7fcdb4069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:02.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.531+0000 7fcdb899a700 1 --2- 172.21.15.33:0/729083102 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcdb4069410 0x7fcdb4069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.531+0000 7fcdb899a700 1 -- 172.21.15.33:0/729083102 >> v2:172.21.15.69:3300/0 conn(0x7fcdb40c3600 msgr2=0x7fcdb40c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:02.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.531+0000 7fcdb899a700 1 --2- 172.21.15.33:0/729083102 >> v2:172.21.15.69:3300/0 conn(0x7fcdb40c3600 0x7fcdb40c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.531+0000 7fcdb899a700 1 -- 172.21.15.33:0/729083102 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb406a460 con 0x7fcdb4136f50 2023-11-06T21:02:02.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.531+0000 7fcdb919b700 1 --2- 172.21.15.33:0/729083102 >> v2:172.21.15.69:3300/0 conn(0x7fcdb40c3600 0x7fcdb40c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:02:02.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.531+0000 7fcdb899a700 1 --2- 172.21.15.33:0/729083102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 0x7fcdb4137400 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7fcdb40c7aa0 tx=0x7fcda40090f0).ready entity=mon.0 client_cookie=98f4fd91949528ff server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:02.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.531+0000 7fcdabfff700 1 -- 172.21.15.33:0/729083102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400b600 con 0x7fcdb4136f50 2023-11-06T21:02:02.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.531+0000 7fcdabfff700 1 -- 172.21.15.33:0/729083102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda400b760 con 0x7fcdb4136f50 2023-11-06T21:02:02.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.531+0000 7fcdabfff700 1 -- 172.21.15.33:0/729083102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400bae0 con 0x7fcdb4136f50 2023-11-06T21:02:02.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.532+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/729083102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 msgr2=0x7fcdb4137400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:02.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.532+0000 7fcdbb3ff700 1 --2- 172.21.15.33:0/729083102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 0x7fcdb4137400 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7fcdb40c7aa0 tx=0x7fcda40090f0).stop 2023-11-06T21:02:02.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.532+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/729083102 shutdown_connections 2023-11-06T21:02:02.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.532+0000 7fcdbb3ff700 1 --2- 172.21.15.33:0/729083102 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcdb4069410 0x7fcdb4069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.532+0000 7fcdbb3ff700 1 --2- 172.21.15.33:0/729083102 >> v2:172.21.15.69:3300/0 conn(0x7fcdb40c3600 0x7fcdb40c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.532+0000 7fcdbb3ff700 1 --2- 172.21.15.33:0/729083102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 0x7fcdb4137400 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.532+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/729083102 >> 172.21.15.33:0/729083102 conn(0x7fcdb4062710 msgr2=0x7fcdb4062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:02.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.532+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/729083102 shutdown_connections 2023-11-06T21:02:02.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.532+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/729083102 wait complete. 2023-11-06T21:02:02.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.534+0000 7fcdbb3ff700 1 Processor -- start 2023-11-06T21:02:02.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.534+0000 7fcdbb3ff700 1 -- start start 2023-11-06T21:02:02.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.535+0000 7fcdbb3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdb4069410 0x7fcdb4150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdbb3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcdb40c3600 0x7fcdb414e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdbb3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 0x7fcdb41649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdbb3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4150880 con 0x7fcdb4136f50 2023-11-06T21:02:02.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdbb3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb414ee60 con 0x7fcdb4069410 2023-11-06T21:02:02.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdbb3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb40bdad0 con 0x7fcdb40c3600 2023-11-06T21:02:02.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdb999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 0x7fcdb41649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdb999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 0x7fcdb41649f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50148/0 (socket says 172.21.15.33:50148) 2023-11-06T21:02:02.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdb999c700 1 -- 172.21.15.33:0/3516742971 learned_addr learned my addr 172.21.15.33:0/3516742971 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:02.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdb899a700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcdb40c3600 0x7fcdb414e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdb919b700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdb4069410 0x7fcdb4150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdb999c700 1 -- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcdb40c3600 msgr2=0x7fcdb414e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:02.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdb999c700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcdb40c3600 0x7fcdb414e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdb999c700 1 -- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdb4069410 msgr2=0x7fcdb4150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:02.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdb999c700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdb4069410 0x7fcdb4150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:02.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.536+0000 7fcdb999c700 1 -- 172.21.15.33:0/3516742971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda4009370 con 0x7fcdb4136f50 2023-11-06T21:02:02.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.537+0000 7fcdb999c700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 0x7fcdb41649f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7fcdac009fa0 tx=0x7fcdac009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:02.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.537+0000 7fcda97fa700 1 -- 172.21.15.33:0/3516742971 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ebc0 con 0x7fcdb4136f50 2023-11-06T21:02:02.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.537+0000 7fcda97fa700 1 -- 172.21.15.33:0/3516742971 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac00ed20 con 0x7fcdb4136f50 2023-11-06T21:02:02.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.537+0000 7fcda97fa700 1 -- 172.21.15.33:0/3516742971 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac014610 con 0x7fcdb4136f50 2023-11-06T21:02:02.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.537+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/3516742971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb414f070 con 0x7fcdb4136f50 2023-11-06T21:02:02.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.537+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/3516742971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb40be000 con 0x7fcdb4136f50 2023-11-06T21:02:02.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.538+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/3516742971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb40acb50 con 0x7fcdb4136f50 2023-11-06T21:02:02.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.539+0000 7fcda97fa700 1 -- 172.21.15.33:0/3516742971 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcdac01d3b0 con 0x7fcdb4136f50 2023-11-06T21:02:02.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.540+0000 7fcda97fa700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd9805f270 0x7fcd98061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:02.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.540+0000 7fcda97fa700 1 -- 172.21.15.33:0/3516742971 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5871+0+0 (secure 0 0 0) 0x7fcdac07a800 con 0x7fcdb4136f50 2023-11-06T21:02:02.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.541+0000 7fcdb919b700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd9805f270 0x7fcd98061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:02.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.541+0000 7fcdb919b700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd9805f270 0x7fcd98061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fcdb0006d80 tx=0x7fcdb0008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:02.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.544+0000 7fcda97fa700 1 -- 172.21.15.33:0/3516742971 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcdb40acb50 con 0x7fcdb4136f50 2023-11-06T21:02:02.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:02.794+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/3516742971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fcdb40acb50 con 0x7fcdb4136f50 2023-11-06T21:02:03.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.464+0000 7fcda97fa700 1 -- 172.21.15.33:0/3516742971 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v665) v1 ==== 183+0+0 (secure 0 0 0) 0x7fcdb40acb50 con 0x7fcdb4136f50 2023-11-06T21:02:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd9805f270 msgr2=0x7fcd98061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd9805f270 0x7fcd98061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fcdb0006d80 tx=0x7fcdb0008040).stop 2023-11-06T21:02:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/3516742971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 msgr2=0x7fcdb41649f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 0x7fcdb41649f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7fcdac009fa0 tx=0x7fcdac009900).stop 2023-11-06T21:02:03.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/3516742971 shutdown_connections 2023-11-06T21:02:03.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcd9805f270 0x7fcd98061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:03.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4136f50 0x7fcdb41649f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:03.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdb4069410 0x7fcdb4150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:03.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 --2- 172.21.15.33:0/3516742971 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcdb40c3600 0x7fcdb414e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:03.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/3516742971 >> 172.21.15.33:0/3516742971 conn(0x7fcdb4062710 msgr2=0x7fcdb4136360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:03.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/3516742971 shutdown_connections 2023-11-06T21:02:03.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.467+0000 7fcdbb3ff700 1 -- 172.21.15.33:0/3516742971 wait complete. 2023-11-06T21:02:03.489 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-11-06T21:02:03.489 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-11-06T21:02:03.489 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-06T21:02:03.490 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-11-06T21:02:03.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.689+0000 7fda71dff700 1 Processor -- start 2023-11-06T21:02:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.690+0000 7fda71dff700 1 -- start start 2023-11-06T21:02:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.691+0000 7fda71dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fda6c133ce0 0x7fda6c1340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.691+0000 7fda71dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fda6c0c6120 0x7fda6c0bc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:03.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.691+0000 7fda71dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c0bc860 con 0x7fda6c151480 2023-11-06T21:02:03.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.691+0000 7fda71dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6c06a730 con 0x7fda6c0c6120 2023-11-06T21:02:03.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.691+0000 7fda71dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c0bcd30 con 0x7fda6c133ce0 2023-11-06T21:02:03.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.691+0000 7fda6bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fda6c0c6120 0x7fda6c0bc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:03.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.691+0000 7fda6bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fda6c0c6120 0x7fda6c0bc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50158/0 (socket says 172.21.15.33:50158) 2023-11-06T21:02:03.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.691+0000 7fda6bfff700 1 -- 172.21.15.33:0/2734927470 learned_addr learned my addr 172.21.15.33:0/2734927470 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:03.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.691+0000 7fda6b7fe700 1 --2- 172.21.15.33:0/2734927470 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fda6c133ce0 0x7fda6c1340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:03.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.692+0000 7fda6bfff700 1 -- 172.21.15.33:0/2734927470 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fda6c133ce0 msgr2=0x7fda6c1340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:03.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.692+0000 7fda6bfff700 1 --2- 172.21.15.33:0/2734927470 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fda6c133ce0 0x7fda6c1340c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:03.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.692+0000 7fda6bfff700 1 -- 172.21.15.33:0/2734927470 >> v1:172.21.15.33:6789/0 conn(0x7fda6c151480 legacy=0x7fda6c0c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:03.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.692+0000 7fda6bfff700 1 -- 172.21.15.33:0/2734927470 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c1519c0 con 0x7fda6c0c6120 2023-11-06T21:02:03.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.692+0000 7fda6bfff700 1 --2- 172.21.15.33:0/2734927470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c0c6120 0x7fda6c0bc370 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7fda64009f20 tx=0x7fda64009840).ready entity=mon.0 client_cookie=321ca95d26f44295 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:03.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.692+0000 7fda6a7fc700 1 -- 172.21.15.33:0/2734927470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400e070 con 0x7fda6c0c6120 2023-11-06T21:02:03.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.692+0000 7fda6a7fc700 1 -- 172.21.15.33:0/2734927470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6400b9d0 con 0x7fda6c0c6120 2023-11-06T21:02:03.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.692+0000 7fda6a7fc700 1 -- 172.21.15.33:0/2734927470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400bd50 con 0x7fda6c0c6120 2023-11-06T21:02:03.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.692+0000 7fda71dff700 1 -- 172.21.15.33:0/2734927470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c0c6120 msgr2=0x7fda6c0bc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:03.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.692+0000 7fda71dff700 1 --2- 172.21.15.33:0/2734927470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c0c6120 0x7fda6c0bc370 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7fda64009f20 tx=0x7fda64009840).stop 2023-11-06T21:02:03.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.693+0000 7fda71dff700 1 -- 172.21.15.33:0/2734927470 shutdown_connections 2023-11-06T21:02:03.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.693+0000 7fda71dff700 1 --2- 172.21.15.33:0/2734927470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c0c6120 0x7fda6c0bc370 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:03.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.693+0000 7fda71dff700 1 --2- 172.21.15.33:0/2734927470 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fda6c133ce0 0x7fda6c1340c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:03.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.693+0000 7fda71dff700 1 -- 172.21.15.33:0/2734927470 >> 172.21.15.33:0/2734927470 conn(0x7fda6c0631a0 msgr2=0x7fda6c0c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:03.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.693+0000 7fda71dff700 1 -- 172.21.15.33:0/2734927470 shutdown_connections 2023-11-06T21:02:03.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.693+0000 7fda71dff700 1 -- 172.21.15.33:0/2734927470 wait complete. 2023-11-06T21:02:03.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.694+0000 7fda71dff700 1 Processor -- start 2023-11-06T21:02:03.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.694+0000 7fda71dff700 1 -- start start 2023-11-06T21:02:03.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.695+0000 7fda71dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda6c0c6120 0x7fda6c149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:03.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.695+0000 7fda71dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fda6c133ce0 0x7fda6c0a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:03.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.695+0000 7fda71dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c151480 0x7fda6c0aa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:03.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.695+0000 7fda71dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c154c20 con 0x7fda6c151480 2023-11-06T21:02:03.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.695+0000 7fda71dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c1542f0 con 0x7fda6c0c6120 2023-11-06T21:02:03.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.695+0000 7fda71dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c1545d0 con 0x7fda6c133ce0 2023-11-06T21:02:03.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.695+0000 7fda6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c151480 0x7fda6c0aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:03.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.695+0000 7fda6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c151480 0x7fda6c0aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50172/0 (socket says 172.21.15.33:50172) 2023-11-06T21:02:03.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda6bfff700 1 -- 172.21.15.33:0/1750880445 learned_addr learned my addr 172.21.15.33:0/1750880445 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:03.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda6b7fe700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda6c0c6120 0x7fda6c149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:03.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda6affd700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fda6c133ce0 0x7fda6c0a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:03.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda6bfff700 1 -- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fda6c133ce0 msgr2=0x7fda6c0a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:03.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda6bfff700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fda6c133ce0 0x7fda6c0a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:03.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda6bfff700 1 -- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda6c0c6120 msgr2=0x7fda6c149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda6bfff700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda6c0c6120 0x7fda6c149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda6bfff700 1 -- 172.21.15.33:0/1750880445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda64009ae0 con 0x7fda6c151480 2023-11-06T21:02:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda6b7fe700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda6c0c6120 0x7fda6c149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:02:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda6bfff700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c151480 0x7fda6c0aa420 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7fda6c0635b0 tx=0x7fda64009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda537fe700 1 -- 172.21.15.33:0/1750880445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400bd50 con 0x7fda6c151480 2023-11-06T21:02:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda537fe700 1 -- 172.21.15.33:0/1750880445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda640092e0 con 0x7fda6c151480 2023-11-06T21:02:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda71dff700 1 -- 172.21.15.33:0/1750880445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c1548b0 con 0x7fda6c151480 2023-11-06T21:02:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.696+0000 7fda537fe700 1 -- 172.21.15.33:0/1750880445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6401d690 con 0x7fda6c151480 2023-11-06T21:02:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.697+0000 7fda71dff700 1 -- 172.21.15.33:0/1750880445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c14a2d0 con 0x7fda6c151480 2023-11-06T21:02:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.697+0000 7fda71dff700 1 -- 172.21.15.33:0/1750880445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c154d60 con 0x7fda6c151480 2023-11-06T21:02:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.698+0000 7fda537fe700 1 -- 172.21.15.33:0/1750880445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fda64003040 con 0x7fda6c151480 2023-11-06T21:02:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.699+0000 7fda537fe700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fda5805f270 0x7fda58061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.699+0000 7fda537fe700 1 -- 172.21.15.33:0/1750880445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5871+0+0 (secure 0 0 0) 0x7fda6407b6e0 con 0x7fda6c151480 2023-11-06T21:02:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.699+0000 7fda6b7fe700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fda5805f270 0x7fda58061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.700+0000 7fda6b7fe700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fda5805f270 0x7fda58061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fda5c009c00 tx=0x7fda5c008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.703+0000 7fda537fe700 1 -- 172.21.15.33:0/1750880445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fda6404c920 con 0x7fda6c151480 2023-11-06T21:02:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:03.954+0000 7fda71dff700 1 -- 172.21.15.33:0/1750880445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fda6c0c75f0 con 0x7fda6c151480 2023-11-06T21:02:04.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.466+0000 7fda537fe700 1 -- 172.21.15.33:0/1750880445 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v666) v1 ==== 151+0+0 (secure 0 0 0) 0x7fda64022020 con 0x7fda6c151480 2023-11-06T21:02:04.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 -- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fda5805f270 msgr2=0x7fda58061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:04.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fda5805f270 0x7fda58061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fda5c009c00 tx=0x7fda5c008040).stop 2023-11-06T21:02:04.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 -- 172.21.15.33:0/1750880445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c151480 msgr2=0x7fda6c0aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:04.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c151480 0x7fda6c0aa420 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7fda6c0635b0 tx=0x7fda64009840).stop 2023-11-06T21:02:04.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 -- 172.21.15.33:0/1750880445 shutdown_connections 2023-11-06T21:02:04.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fda5805f270 0x7fda58061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda6c151480 0x7fda6c0aa420 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda6c0c6120 0x7fda6c149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 --2- 172.21.15.33:0/1750880445 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fda6c133ce0 0x7fda6c0a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 -- 172.21.15.33:0/1750880445 >> 172.21.15.33:0/1750880445 conn(0x7fda6c0631a0 msgr2=0x7fda6c14ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:04.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 -- 172.21.15.33:0/1750880445 shutdown_connections 2023-11-06T21:02:04.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.469+0000 7fda71dff700 1 -- 172.21.15.33:0/1750880445 wait complete. 2023-11-06T21:02:04.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T21:02:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.689+0000 7f93983ff700 1 Processor -- start 2023-11-06T21:02:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.690+0000 7f93983ff700 1 -- start start 2023-11-06T21:02:04.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.691+0000 7f93983ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9390154330 0x7f93900c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.691+0000 7f93983ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 0x7f93900646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.691+0000 7f93983ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93900c6b60 con 0x7f9390154710 2023-11-06T21:02:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.691+0000 7f93983ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93900c7700 con 0x7f9390154330 2023-11-06T21:02:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.691+0000 7f93983ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93900c5cd0 con 0x7f93900c7ff0 2023-11-06T21:02:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939699c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 0x7f93900646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939619b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9390154330 0x7f93900c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939699c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 0x7f93900646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:34258/0 (socket says 172.21.15.33:34258) 2023-11-06T21:02:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939619b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9390154330 0x7f93900c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:56924/0 (socket says 172.21.15.33:56924) 2023-11-06T21:02:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939699c700 1 -- 172.21.15.33:0/2492661056 learned_addr learned my addr 172.21.15.33:0/2492661056 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939619b700 1 -- 172.21.15.33:0/2492661056 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 msgr2=0x7f93900646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939619b700 1 --2- 172.21.15.33:0/2492661056 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 0x7f93900646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939619b700 1 -- 172.21.15.33:0/2492661056 >> v1:172.21.15.69:6789/0 conn(0x7f9390154710 legacy=0x7f93900c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939619b700 1 -- 172.21.15.33:0/2492661056 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93900c5f50 con 0x7f9390154330 2023-11-06T21:02:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939699c700 1 --2- 172.21.15.33:0/2492661056 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 0x7f93900646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:02:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.692+0000 7f939619b700 1 --2- 172.21.15.33:0/2492661056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9390154330 0x7f93900c57e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f938c009760 tx=0x7f938c009060).ready entity=mon.1 client_cookie=798a40de612a60bb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f9395199700 1 -- 172.21.15.33:0/2492661056 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c00f510 con 0x7f9390154330 2023-11-06T21:02:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f9395199700 1 -- 172.21.15.33:0/2492661056 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c00f670 con 0x7f9390154330 2023-11-06T21:02:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f9395199700 1 -- 172.21.15.33:0/2492661056 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c00f9f0 con 0x7f9390154330 2023-11-06T21:02:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f93983ff700 1 -- 172.21.15.33:0/2492661056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9390154330 msgr2=0x7f93900c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f93983ff700 1 --2- 172.21.15.33:0/2492661056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9390154330 0x7f93900c57e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f938c009760 tx=0x7f938c009060).stop 2023-11-06T21:02:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f93983ff700 1 -- 172.21.15.33:0/2492661056 shutdown_connections 2023-11-06T21:02:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f93983ff700 1 --2- 172.21.15.33:0/2492661056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9390154330 0x7f93900c57e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f93983ff700 1 --2- 172.21.15.33:0/2492661056 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 0x7f93900646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f93983ff700 1 -- 172.21.15.33:0/2492661056 >> 172.21.15.33:0/2492661056 conn(0x7f93900cb910 msgr2=0x7f93900c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:04.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f93983ff700 1 -- 172.21.15.33:0/2492661056 shutdown_connections 2023-11-06T21:02:04.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.693+0000 7f93983ff700 1 -- 172.21.15.33:0/2492661056 wait complete. 2023-11-06T21:02:04.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.695+0000 7f93983ff700 1 Processor -- start 2023-11-06T21:02:04.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.695+0000 7f93983ff700 1 -- start start 2023-11-06T21:02:04.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f93983ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 0x7f9390164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:04.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f93983ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9390154330 0x7f9390168b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:04.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f93983ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9390154710 0x7f93900c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:04.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f93983ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93900a25a0 con 0x7f9390154710 2023-11-06T21:02:04.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f93983ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9390165e30 con 0x7f9390154330 2023-11-06T21:02:04.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f93983ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9390169060 con 0x7f93900c7ff0 2023-11-06T21:02:04.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f939699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9390154710 0x7f93900c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:04.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f939699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9390154710 0x7f93900c40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50186/0 (socket says 172.21.15.33:50186) 2023-11-06T21:02:04.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f939699c700 1 -- 172.21.15.33:0/64925355 learned_addr learned my addr 172.21.15.33:0/64925355 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:04.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f939599a700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9390154330 0x7f9390168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:04.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.696+0000 7f939619b700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 0x7f9390164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:04.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f939699c700 1 -- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 msgr2=0x7f9390164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:04.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f939699c700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 0x7f9390164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f939699c700 1 -- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9390154330 msgr2=0x7f9390168b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f939699c700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9390154330 0x7f9390168b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f939699c700 1 -- 172.21.15.33:0/64925355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9388009ae0 con 0x7f9390154710 2023-11-06T21:02:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f939699c700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9390154710 0x7f93900c40b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f938800c0c0 tx=0x7f938800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f93867fc700 1 -- 172.21.15.33:0/64925355 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93880092e0 con 0x7f9390154710 2023-11-06T21:02:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f93867fc700 1 -- 172.21.15.33:0/64925355 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938800f040 con 0x7f9390154710 2023-11-06T21:02:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f93867fc700 1 -- 172.21.15.33:0/64925355 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938800cc00 con 0x7f9390154710 2023-11-06T21:02:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f93983ff700 1 -- 172.21.15.33:0/64925355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c0092a0 con 0x7f9390154710 2023-11-06T21:02:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.697+0000 7f93983ff700 1 -- 172.21.15.33:0/64925355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9390165500 con 0x7f9390154710 2023-11-06T21:02:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.699+0000 7f93983ff700 1 -- 172.21.15.33:0/64925355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9390165fe0 con 0x7f9390154710 2023-11-06T21:02:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.699+0000 7f93867fc700 1 -- 172.21.15.33:0/64925355 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9388007400 con 0x7f9390154710 2023-11-06T21:02:04.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.700+0000 7f93867fc700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f937405f270 0x7f9374061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:04.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.700+0000 7f93867fc700 1 -- 172.21.15.33:0/64925355 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5871+0+0 (secure 0 0 0) 0x7f938807a220 con 0x7f9390154710 2023-11-06T21:02:04.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.700+0000 7f939619b700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f937405f270 0x7f9374061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:04.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.701+0000 7f939619b700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f937405f270 0x7f9374061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f938c009fd0 tx=0x7f938c017000).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:04.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.704+0000 7f93867fc700 1 -- 172.21.15.33:0/64925355 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f938804aaf0 con 0x7f9390154710 2023-11-06T21:02:04.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.886+0000 7f93983ff700 1 -- 172.21.15.33:0/64925355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f93900c4c30 con 0x7f9390154710 2023-11-06T21:02:04.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.890+0000 7f93867fc700 1 -- 172.21.15.33:0/64925355 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9390165fe0 con 0x7f9390154710 2023-11-06T21:02:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 -- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f937405f270 msgr2=0x7f9374061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f937405f270 0x7f9374061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f938c009fd0 tx=0x7f938c017000).stop 2023-11-06T21:02:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 -- 172.21.15.33:0/64925355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9390154710 msgr2=0x7f93900c40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9390154710 0x7f93900c40b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f938800c0c0 tx=0x7f938800b040).stop 2023-11-06T21:02:04.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 -- 172.21.15.33:0/64925355 shutdown_connections 2023-11-06T21:02:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f937405f270 0x7f9374061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f93900c7ff0 0x7f9390164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9390154710 0x7f93900c40b0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 --2- 172.21.15.33:0/64925355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9390154330 0x7f9390168b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 -- 172.21.15.33:0/64925355 >> 172.21.15.33:0/64925355 conn(0x7f93900cb910 msgr2=0x7f93900c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 -- 172.21.15.33:0/64925355 shutdown_connections 2023-11-06T21:02:04.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:04.892+0000 7f93983ff700 1 -- 172.21.15.33:0/64925355 wait complete. 2023-11-06T21:02:04.910 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T21:02:04.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:02:04.999 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:02:05.099 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T21:02:05.099 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T21:02:05.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:02:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.295+0000 7f5b5f5ff700 1 Processor -- start 2023-11-06T21:02:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.295+0000 7f5b5f5ff700 1 -- start start 2023-11-06T21:02:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.296+0000 7f5b5f5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b581580e0 0x7f5b581584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5f5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b580a7910 0x7f5b580a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5f5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b58064010 con 0x7f5b580bdbe0 2023-11-06T21:02:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5f5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b580be430 con 0x7f5b581580e0 2023-11-06T21:02:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5f5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580a8290 con 0x7f5b580a7910 2023-11-06T21:02:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5d39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b581580e0 0x7f5b581584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5d39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b581580e0 0x7f5b581584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50190/0 (socket says 172.21.15.33:50190) 2023-11-06T21:02:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5d39b700 1 -- 172.21.15.33:0/863781880 learned_addr learned my addr 172.21.15.33:0/863781880 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5db9c700 1 --2- 172.21.15.33:0/863781880 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b580a7910 0x7f5b580a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5d39b700 1 -- 172.21.15.33:0/863781880 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b580a7910 msgr2=0x7f5b580a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5d39b700 1 --2- 172.21.15.33:0/863781880 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b580a7910 0x7f5b580a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5d39b700 1 -- 172.21.15.33:0/863781880 >> v1:172.21.15.69:6789/0 conn(0x7f5b580bdbe0 legacy=0x7f5b580684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5d39b700 1 -- 172.21.15.33:0/863781880 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b580be150 con 0x7f5b581580e0 2023-11-06T21:02:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5db9c700 1 --2- 172.21.15.33:0/863781880 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b580a7910 0x7f5b580a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:02:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.297+0000 7f5b5d39b700 1 --2- 172.21.15.33:0/863781880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b581580e0 0x7f5b581584c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f5b54006fd0 tx=0x7f5b540090f0).ready entity=mon.0 client_cookie=1c7fda71a9bdd9d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b4ffff700 1 -- 172.21.15.33:0/863781880 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5400b600 con 0x7f5b581580e0 2023-11-06T21:02:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b4ffff700 1 -- 172.21.15.33:0/863781880 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5400b760 con 0x7f5b581580e0 2023-11-06T21:02:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b4ffff700 1 -- 172.21.15.33:0/863781880 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5400bae0 con 0x7f5b581580e0 2023-11-06T21:02:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/863781880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b581580e0 msgr2=0x7f5b581584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b5f5ff700 1 --2- 172.21.15.33:0/863781880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b581580e0 0x7f5b581584c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f5b54006fd0 tx=0x7f5b540090f0).stop 2023-11-06T21:02:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/863781880 shutdown_connections 2023-11-06T21:02:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b5f5ff700 1 --2- 172.21.15.33:0/863781880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b581580e0 0x7f5b581584c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b5f5ff700 1 --2- 172.21.15.33:0/863781880 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b580a7910 0x7f5b580a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/863781880 >> 172.21.15.33:0/863781880 conn(0x7f5b5805ec80 msgr2=0x7f5b580664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/863781880 shutdown_connections 2023-11-06T21:02:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.298+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/863781880 wait complete. 2023-11-06T21:02:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.300+0000 7f5b5f5ff700 1 Processor -- start 2023-11-06T21:02:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.300+0000 7f5b5f5ff700 1 -- start start 2023-11-06T21:02:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5f5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b580a7910 0x7f5b58154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5f5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580bdbe0 0x7f5b580c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5f5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b581580e0 0x7f5b580add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5f5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b58152570 con 0x7f5b580bdbe0 2023-11-06T21:02:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5f5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580c44b0 con 0x7f5b580a7910 2023-11-06T21:02:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5f5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b58155430 con 0x7f5b581580e0 2023-11-06T21:02:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5cb9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580bdbe0 0x7f5b580c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5d39b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b580a7910 0x7f5b58154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5cb9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580bdbe0 0x7f5b580c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50202/0 (socket says 172.21.15.33:50202) 2023-11-06T21:02:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5cb9a700 1 -- 172.21.15.33:0/2646603709 learned_addr learned my addr 172.21.15.33:0/2646603709 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.301+0000 7f5b5db9c700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b581580e0 0x7f5b580add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b5cb9a700 1 -- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b581580e0 msgr2=0x7f5b580add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b5cb9a700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b581580e0 0x7f5b580add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b5cb9a700 1 -- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b580a7910 msgr2=0x7f5b58154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b5cb9a700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b580a7910 0x7f5b58154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b5cb9a700 1 -- 172.21.15.33:0/2646603709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b54009370 con 0x7f5b580bdbe0 2023-11-06T21:02:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b5d39b700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b580a7910 0x7f5b58154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:02:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b5cb9a700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580bdbe0 0x7f5b580c5c40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f5b4800c4c0 tx=0x7f5b48007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b4d7fa700 1 -- 172.21.15.33:0/2646603709 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800ce70 con 0x7f5b580bdbe0 2023-11-06T21:02:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b4d7fa700 1 -- 172.21.15.33:0/2646603709 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b48001e90 con 0x7f5b580bdbe0 2023-11-06T21:02:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/2646603709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b580c41d0 con 0x7f5b580bdbe0 2023-11-06T21:02:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b4d7fa700 1 -- 172.21.15.33:0/2646603709 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48007270 con 0x7f5b580bdbe0 2023-11-06T21:02:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.302+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/2646603709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b580c5440 con 0x7f5b580bdbe0 2023-11-06T21:02:05.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.304+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/2646603709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b580c5a90 con 0x7f5b580bdbe0 2023-11-06T21:02:05.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.304+0000 7f5b4d7fa700 1 -- 172.21.15.33:0/2646603709 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5b480066a0 con 0x7f5b580bdbe0 2023-11-06T21:02:05.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.305+0000 7f5b4d7fa700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b3c05f270 0x7f5b3c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.306+0000 7f5b4d7fa700 1 -- 172.21.15.33:0/2646603709 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5871+0+0 (secure 0 0 0) 0x7f5b48079ba0 con 0x7f5b580bdbe0 2023-11-06T21:02:05.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.306+0000 7f5b5d39b700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b3c05f270 0x7f5b3c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.306+0000 7f5b5d39b700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b3c05f270 0x7f5b3c061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f5b5400b3c0 tx=0x7f5b54017000).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:05.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.309+0000 7f5b4d7fa700 1 -- 172.21.15.33:0/2646603709 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5b4804ad30 con 0x7f5b580bdbe0 2023-11-06T21:02:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.533+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/2646603709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b58064f50 con 0x7f5b580bdbe0 2023-11-06T21:02:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.534+0000 7f5b4d7fa700 1 -- 172.21.15.33:0/2646603709 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f5b4804e350 con 0x7f5b580bdbe0 2023-11-06T21:02:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:05.535 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6176,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2108889615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2108889615},{"type":"v1","addr":"172.21.15.33:6834","nonce":2108889615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/836821298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":836821298},{"type":"v1","addr":"172.21.15.69:6836","nonce":836821298}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2546625927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2546625927},{"type":"v1","addr":"172.21.15.69:6837","nonce":2546625927}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6838,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2007995447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2007995447},{"type":"v1","addr":"172.21.15.33:6835","nonce":2007995447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[]} 2023-11-06T21:02:05.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b3c05f270 msgr2=0x7f5b3c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:05.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b3c05f270 0x7f5b3c061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f5b5400b3c0 tx=0x7f5b54017000).stop 2023-11-06T21:02:05.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/2646603709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580bdbe0 msgr2=0x7f5b580c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:05.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580bdbe0 0x7f5b580c5c40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f5b4800c4c0 tx=0x7f5b48007e00).stop 2023-11-06T21:02:05.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/2646603709 shutdown_connections 2023-11-06T21:02:05.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5b3c05f270 0x7f5b3c061730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b581580e0 0x7f5b580add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b580a7910 0x7f5b58154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 --2- 172.21.15.33:0/2646603709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580bdbe0 0x7f5b580c5c40 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/2646603709 >> 172.21.15.33:0/2646603709 conn(0x7f5b5805ec80 msgr2=0x7f5b58155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:05.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/2646603709 shutdown_connections 2023-11-06T21:02:05.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.536+0000 7f5b5f5ff700 1 -- 172.21.15.33:0/2646603709 wait complete. 2023-11-06T21:02:05.540 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-11-06T21:02:05.555 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T21:02:05.555 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T21:02:05.556 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T21:02:05.556 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:02:05.556 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T21:02:05.556 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T21:02:05.557 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T21:02:05.557 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:02:05.557 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T21:02:05.557 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T21:02:05.558 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T21:02:05.558 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:02:05.558 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T21:02:05.559 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T21:02:05.559 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T21:02:05.559 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:02:05.569 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T21:02:05.570 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T21:02:05.571 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T21:02:05.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T21:02:05.573 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T21:02:05.573 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T21:02:05.574 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T21:02:05.575 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T21:02:05.615 INFO:tasks.ceph.mds.a:Started 2023-11-06T21:02:05.616 DEBUG:teuthology.parallel:result is None 2023-11-06T21:02:05.619 INFO:tasks.ceph.mds.b:Started 2023-11-06T21:02:05.619 DEBUG:teuthology.parallel:result is None 2023-11-06T21:02:05.620 INFO:tasks.ceph.mds.c:Started 2023-11-06T21:02:05.620 DEBUG:teuthology.parallel:result is None 2023-11-06T21:02:05.622 INFO:tasks.ceph.mds.d:Started 2023-11-06T21:02:05.623 DEBUG:teuthology.parallel:result is None 2023-11-06T21:02:05.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T21:02:05.764 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T21:02:05.765 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T21:02:05.765 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T21:02:05.766 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T21:02:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.837+0000 7f9ceddff700 1 Processor -- start 2023-11-06T21:02:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.837+0000 7f9ceddff700 1 -- start start 2023-11-06T21:02:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ceddff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9ce813ce50 0x7f9ce813d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ceddff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ce80c4560 0x7f9ce80c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ceddff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ce8157fe0 0x7f9ce8158430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ceddff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce8158d50 con 0x7f9ce8157fe0 2023-11-06T21:02:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ceddff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce80c4ed0 con 0x7f9ce813ce50 2023-11-06T21:02:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ceddff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce8062840 con 0x7f9ce80c4560 2023-11-06T21:02:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ce7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ce8157fe0 0x7f9ce8158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ce7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ce8157fe0 0x7f9ce8158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50250/0 (socket says 172.21.15.33:50250) 2023-11-06T21:02:05.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ce7fff700 1 -- 172.21.15.33:0/3715885832 learned_addr learned my addr 172.21.15.33:0/3715885832 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:05.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ce77fe700 1 --2- 172.21.15.33:0/3715885832 >> v2:172.21.15.69:3300/0 conn(0x7f9ce813ce50 0x7f9ce813d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.839+0000 7f9ce6ffd700 1 --2- 172.21.15.33:0/3715885832 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ce80c4560 0x7f9ce80c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.840+0000 7f9ce7fff700 1 -- 172.21.15.33:0/3715885832 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ce80c4560 msgr2=0x7f9ce80c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.840+0000 7f9ce7fff700 1 --2- 172.21.15.33:0/3715885832 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ce80c4560 0x7f9ce80c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.840+0000 7f9ce7fff700 1 -- 172.21.15.33:0/3715885832 >> v2:172.21.15.69:3300/0 conn(0x7f9ce813ce50 msgr2=0x7f9ce813d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.840+0000 7f9ce7fff700 1 --2- 172.21.15.33:0/3715885832 >> v2:172.21.15.69:3300/0 conn(0x7f9ce813ce50 0x7f9ce813d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.840+0000 7f9ce7fff700 1 -- 172.21.15.33:0/3715885832 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce8062b20 con 0x7f9ce8157fe0 2023-11-06T21:02:05.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.840+0000 7f9ce7fff700 1 --2- 172.21.15.33:0/3715885832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce8157fe0 0x7f9ce8158430 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f9ce806c2d0 tx=0x7f9ce0009840).ready entity=mon.0 client_cookie=433ac78503180701 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:05.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.840+0000 7f9ce67fc700 1 -- 172.21.15.33:0/3715885832 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000e070 con 0x7f9ce8157fe0 2023-11-06T21:02:05.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.840+0000 7f9ce67fc700 1 -- 172.21.15.33:0/3715885832 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce000b970 con 0x7f9ce8157fe0 2023-11-06T21:02:05.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.840+0000 7f9ce67fc700 1 -- 172.21.15.33:0/3715885832 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000bcf0 con 0x7f9ce8157fe0 2023-11-06T21:02:05.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.841+0000 7f9ceddff700 1 -- 172.21.15.33:0/3715885832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce8157fe0 msgr2=0x7f9ce8158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:05.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.841+0000 7f9ceddff700 1 --2- 172.21.15.33:0/3715885832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce8157fe0 0x7f9ce8158430 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f9ce806c2d0 tx=0x7f9ce0009840).stop 2023-11-06T21:02:05.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.841+0000 7f9ceddff700 1 -- 172.21.15.33:0/3715885832 shutdown_connections 2023-11-06T21:02:05.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.841+0000 7f9ceddff700 1 --2- 172.21.15.33:0/3715885832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce8157fe0 0x7f9ce8158430 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.841+0000 7f9ceddff700 1 --2- 172.21.15.33:0/3715885832 >> v2:172.21.15.69:3300/0 conn(0x7f9ce813ce50 0x7f9ce813d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.841+0000 7f9ceddff700 1 --2- 172.21.15.33:0/3715885832 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ce80c4560 0x7f9ce80c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.841+0000 7f9ceddff700 1 -- 172.21.15.33:0/3715885832 >> 172.21.15.33:0/3715885832 conn(0x7f9ce80c7dc0 msgr2=0x7f9ce80c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:05.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.841+0000 7f9ceddff700 1 -- 172.21.15.33:0/3715885832 shutdown_connections 2023-11-06T21:02:05.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.841+0000 7f9ceddff700 1 -- 172.21.15.33:0/3715885832 wait complete. 2023-11-06T21:02:05.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.843+0000 7f9ceddff700 1 Processor -- start 2023-11-06T21:02:05.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.843+0000 7f9ceddff700 1 -- start start 2023-11-06T21:02:05.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ceddff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ce80c4560 0x7f9ce80a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ceddff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce813ce50 0x7f9ce80ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ceddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ce8157fe0 0x7f9ce80ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ceddff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80abb20 con 0x7f9ce813ce50 2023-11-06T21:02:05.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ceddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80abdf0 con 0x7f9ce8157fe0 2023-11-06T21:02:05.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ceddff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80ac0d0 con 0x7f9ce80c4560 2023-11-06T21:02:05.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ce6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce813ce50 0x7f9ce80ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ce6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce813ce50 0x7f9ce80ace90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50262/0 (socket says 172.21.15.33:50262) 2023-11-06T21:02:05.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ce6ffd700 1 -- 172.21.15.33:0/2708095515 learned_addr learned my addr 172.21.15.33:0/2708095515 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:05.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ce77fe700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ce80c4560 0x7f9ce80a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ce7fff700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ce8157fe0 0x7f9ce80ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ce6ffd700 1 -- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ce80c4560 msgr2=0x7f9ce80a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:05.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ce6ffd700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ce80c4560 0x7f9ce80a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ce6ffd700 1 -- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ce8157fe0 msgr2=0x7f9ce80ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:05.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ce6ffd700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ce8157fe0 0x7f9ce80ab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:05.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.844+0000 7f9ce6ffd700 1 -- 172.21.15.33:0/2708095515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce0009ae0 con 0x7f9ce813ce50 2023-11-06T21:02:05.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.845+0000 7f9ce6ffd700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce813ce50 0x7f9ce80ace90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f9cdc00e900 tx=0x7f9cdc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:05.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.845+0000 7f9ccf7fe700 1 -- 172.21.15.33:0/2708095515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc009390 con 0x7f9ce813ce50 2023-11-06T21:02:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.845+0000 7f9ccf7fe700 1 -- 172.21.15.33:0/2708095515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc012600 con 0x7f9ce813ce50 2023-11-06T21:02:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.845+0000 7f9ceddff700 1 -- 172.21.15.33:0/2708095515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce80ac3b0 con 0x7f9ce813ce50 2023-11-06T21:02:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.845+0000 7f9ccf7fe700 1 -- 172.21.15.33:0/2708095515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc012980 con 0x7f9ce813ce50 2023-11-06T21:02:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.845+0000 7f9ceddff700 1 -- 172.21.15.33:0/2708095515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce80ac970 con 0x7f9ce813ce50 2023-11-06T21:02:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.846+0000 7f9ceddff700 1 -- 172.21.15.33:0/2708095515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce8062e00 con 0x7f9ce813ce50 2023-11-06T21:02:05.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.847+0000 7f9ccf7fe700 1 -- 172.21.15.33:0/2708095515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9cdc012d30 con 0x7f9ce813ce50 2023-11-06T21:02:05.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.847+0000 7f9ccf7fe700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cd405f2a0 0x7f9cd4061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:05.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.848+0000 7f9ccf7fe700 1 -- 172.21.15.33:0/2708095515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5871+0+0 (secure 0 0 0) 0x7f9cdc07a580 con 0x7f9ce813ce50 2023-11-06T21:02:05.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.848+0000 7f9ce77fe700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cd405f2a0 0x7f9cd4061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:05.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.848+0000 7f9ce77fe700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cd405f2a0 0x7f9cd4061760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9cd8009c30 tx=0x7f9cd8009060).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:05.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:05.852+0000 7f9ccf7fe700 1 -- 172.21.15.33:0/2708095515 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9cdc04b710 con 0x7f9ce813ce50 2023-11-06T21:02:06.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.045+0000 7f9ceddff700 1 -- 172.21.15.33:0/2708095515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9ce8069be0 con 0x7f9ce813ce50 2023-11-06T21:02:06.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.894+0000 7f9ccf7fe700 1 -- 172.21.15.33:0/2708095515 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v127) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9cdc04ed30 con 0x7f9ce813ce50 2023-11-06T21:02:06.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 -- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cd405f2a0 msgr2=0x7f9cd4061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:06.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cd405f2a0 0x7f9cd4061760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9cd8009c30 tx=0x7f9cd8009060).stop 2023-11-06T21:02:06.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 -- 172.21.15.33:0/2708095515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce813ce50 msgr2=0x7f9ce80ace90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:06.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce813ce50 0x7f9ce80ace90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f9cdc00e900 tx=0x7f9cdc009060).stop 2023-11-06T21:02:06.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 -- 172.21.15.33:0/2708095515 shutdown_connections 2023-11-06T21:02:06.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cd405f2a0 0x7f9cd4061760 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:06.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ce80c4560 0x7f9ce80a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:06.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ce8157fe0 0x7f9ce80ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:06.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 --2- 172.21.15.33:0/2708095515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ce813ce50 0x7f9ce80ace90 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:06.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 -- 172.21.15.33:0/2708095515 >> 172.21.15.33:0/2708095515 conn(0x7f9ce80c7dc0 msgr2=0x7f9ce8064bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:06.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 -- 172.21.15.33:0/2708095515 shutdown_connections 2023-11-06T21:02:06.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:06.897+0000 7f9ceddff700 1 -- 172.21.15.33:0/2708095515 wait complete. 2023-11-06T21:02:06.900 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T21:02:06.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T21:02:07.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.119+0000 7ff9e51ff700 1 Processor -- start 2023-11-06T21:02:07.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.120+0000 7ff9e51ff700 1 -- start start 2023-11-06T21:02:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.121+0000 7ff9e51ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff9e00c1840 0x7ff9e00a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.121+0000 7ff9e51ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9e00aa2d0 0x7ff9e00c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.121+0000 7ff9e51ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e01554a0 con 0x7ff9e00c6240 2023-11-06T21:02:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.121+0000 7ff9e51ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e0154960 con 0x7ff9e00aa2d0 2023-11-06T21:02:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.121+0000 7ff9e51ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e0155930 con 0x7ff9e00c1840 2023-11-06T21:02:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.121+0000 7ff9de59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9e00aa2d0 0x7ff9e00c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.121+0000 7ff9df59e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff9e00c6240 0x7ff9e0157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50260/0 (socket says 172.21.15.33:50260) 2023-11-06T21:02:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.121+0000 7ff9de59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9e00aa2d0 0x7ff9e00c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50278/0 (socket says 172.21.15.33:50278) 2023-11-06T21:02:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.121+0000 7ff9df59e700 1 -- 172.21.15.33:0/1431924120 learned_addr learned my addr 172.21.15.33:0/1431924120 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.121+0000 7ff9ded9d700 1 --2- 172.21.15.33:0/1431924120 >> v2:172.21.15.69:3300/0 conn(0x7ff9e00c1840 0x7ff9e00a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.122+0000 7ff9de59c700 1 -- 172.21.15.33:0/1431924120 >> v2:172.21.15.69:3300/0 conn(0x7ff9e00c1840 msgr2=0x7ff9e00a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:07.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.122+0000 7ff9de59c700 1 --2- 172.21.15.33:0/1431924120 >> v2:172.21.15.69:3300/0 conn(0x7ff9e00c1840 0x7ff9e00a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.122+0000 7ff9de59c700 1 -- 172.21.15.33:0/1431924120 >> v1:172.21.15.33:6789/0 conn(0x7ff9e00c6240 legacy=0x7ff9e0157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.122+0000 7ff9de59c700 1 -- 172.21.15.33:0/1431924120 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0154b70 con 0x7ff9e00aa2d0 2023-11-06T21:02:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.122+0000 7ff9ddd9b700 1 -- 172.21.15.33:0/1431924120 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237761844 0 0) 0x7ff9e01554a0 con 0x7ff9e00c6240 2023-11-06T21:02:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.122+0000 7ff9de59c700 1 --2- 172.21.15.33:0/1431924120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00aa2d0 0x7ff9e00c5d50 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7ff9e00840b0 tx=0x7ff9d4009060).ready entity=mon.0 client_cookie=df114d3555862744 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.122+0000 7ff9ddd9b700 1 -- 172.21.15.33:0/1431924120 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400b580 con 0x7ff9e00aa2d0 2023-11-06T21:02:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.122+0000 7ff9ddd9b700 1 -- 172.21.15.33:0/1431924120 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d400b6e0 con 0x7ff9e00aa2d0 2023-11-06T21:02:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.122+0000 7ff9ddd9b700 1 -- 172.21.15.33:0/1431924120 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400ba60 con 0x7ff9e00aa2d0 2023-11-06T21:02:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.123+0000 7ff9e51ff700 1 -- 172.21.15.33:0/1431924120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00aa2d0 msgr2=0x7ff9e00c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.123+0000 7ff9e51ff700 1 --2- 172.21.15.33:0/1431924120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00aa2d0 0x7ff9e00c5d50 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7ff9e00840b0 tx=0x7ff9d4009060).stop 2023-11-06T21:02:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.123+0000 7ff9e51ff700 1 -- 172.21.15.33:0/1431924120 shutdown_connections 2023-11-06T21:02:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.123+0000 7ff9e51ff700 1 --2- 172.21.15.33:0/1431924120 >> v2:172.21.15.69:3300/0 conn(0x7ff9e00c1840 0x7ff9e00a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.123+0000 7ff9e51ff700 1 --2- 172.21.15.33:0/1431924120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00aa2d0 0x7ff9e00c5d50 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.123+0000 7ff9e51ff700 1 -- 172.21.15.33:0/1431924120 >> 172.21.15.33:0/1431924120 conn(0x7ff9e005ec80 msgr2=0x7ff9e00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.123+0000 7ff9e51ff700 1 -- 172.21.15.33:0/1431924120 shutdown_connections 2023-11-06T21:02:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.123+0000 7ff9e51ff700 1 -- 172.21.15.33:0/1431924120 wait complete. 2023-11-06T21:02:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.124+0000 7ff9e51ff700 1 Processor -- start 2023-11-06T21:02:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.124+0000 7ff9e51ff700 1 -- start start 2023-11-06T21:02:07.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.125+0000 7ff9e51ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9e00aa2d0 0x7ff9e00c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.125+0000 7ff9e51ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00c1840 0x7ff9e00afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.125+0000 7ff9e51ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00c6240 0x7ff9e0144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.125+0000 7ff9e51ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00adda0 con 0x7ff9e00c1840 2023-11-06T21:02:07.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.125+0000 7ff9e51ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c36c0 con 0x7ff9e00c6240 2023-11-06T21:02:07.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.125+0000 7ff9e51ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c39a0 con 0x7ff9e00aa2d0 2023-11-06T21:02:07.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9de59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00c1840 0x7ff9e00afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9de59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00c1840 0x7ff9e00afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50284/0 (socket says 172.21.15.33:50284) 2023-11-06T21:02:07.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9de59c700 1 -- 172.21.15.33:0/3446159196 learned_addr learned my addr 172.21.15.33:0/3446159196 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:07.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9ded9d700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9e00aa2d0 0x7ff9e00c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9df59e700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00c6240 0x7ff9e0144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9de59c700 1 -- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9e00aa2d0 msgr2=0x7ff9e00c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9de59c700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9e00aa2d0 0x7ff9e00c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9de59c700 1 -- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00c6240 msgr2=0x7ff9e0144a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9de59c700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00c6240 0x7ff9e0144a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9de59c700 1 -- 172.21.15.33:0/3446159196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d40092a0 con 0x7ff9e00c1840 2023-11-06T21:02:07.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9de59c700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00c1840 0x7ff9e00afd20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7ff9d4009fd0 tx=0x7ff9d4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:07.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9c6ffd700 1 -- 172.21.15.33:0/3446159196 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400b850 con 0x7ff9e00c1840 2023-11-06T21:02:07.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9c6ffd700 1 -- 172.21.15.33:0/3446159196 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d400b9b0 con 0x7ff9e00c1840 2023-11-06T21:02:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9c6ffd700 1 -- 172.21.15.33:0/3446159196 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d401b690 con 0x7ff9e00c1840 2023-11-06T21:02:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.126+0000 7ff9e51ff700 1 -- 172.21.15.33:0/3446159196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e00c3c80 con 0x7ff9e00c1840 2023-11-06T21:02:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.127+0000 7ff9e51ff700 1 -- 172.21.15.33:0/3446159196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e00b0210 con 0x7ff9e00c1840 2023-11-06T21:02:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.128+0000 7ff9e51ff700 1 -- 172.21.15.33:0/3446159196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e006c6c0 con 0x7ff9e00c1840 2023-11-06T21:02:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.129+0000 7ff9c6ffd700 1 -- 172.21.15.33:0/3446159196 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff9d401bac0 con 0x7ff9e00c1840 2023-11-06T21:02:07.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.130+0000 7ff9c6ffd700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff9cc05f270 0x7ff9cc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.130+0000 7ff9c6ffd700 1 -- 172.21.15.33:0/3446159196 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff9d407b930 con 0x7ff9e00c1840 2023-11-06T21:02:07.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.130+0000 7ff9ded9d700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff9cc05f270 0x7ff9cc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.131+0000 7ff9ded9d700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff9cc05f270 0x7ff9cc061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7ff9d000bbc0 tx=0x7ff9d0008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:07.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.134+0000 7ff9c6ffd700 1 -- 172.21.15.33:0/3446159196 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff9d404cd10 con 0x7ff9e00c1840 2023-11-06T21:02:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.380+0000 7ff9e51ff700 1 -- 172.21.15.33:0/3446159196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff9e0063f20 con 0x7ff9e00c1840 2023-11-06T21:02:07.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.380+0000 7ff9c6ffd700 1 -- 172.21.15.33:0/3446159196 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff9d4050330 con 0x7ff9e00c1840 2023-11-06T21:02:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:07.381 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T21:02:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T21:02:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T21:02:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T21:02:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:02:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:02:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:02:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T21:02:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T21:02:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:02:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:02:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:02:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T21:02:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T21:02:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:02:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:02:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:02:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T21:02:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T21:02:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:02:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:02:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:02:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T21:02:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T21:02:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:02:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:02:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T21:02:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T21:02:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:02:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:02:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T21:02:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T21:02:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:02:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:02:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T21:02:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T21:02:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:02:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:02:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T21:02:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T21:02:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:02:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:02:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T21:02:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T21:02:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:02:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:02:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T21:02:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T21:02:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:02:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:02:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T21:02:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T21:02:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:02:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:02:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T21:02:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T21:02:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:02:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-11-06T21:02:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T21:02:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T21:02:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T21:02:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:02:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:02:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:02:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:02:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T21:02:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T21:02:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:02:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:02:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:02:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:02:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T21:02:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T21:02:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:02:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:02:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:02:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:02:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T21:02:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T21:02:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:02:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:02:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:02:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T21:02:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T21:02:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:02:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T21:02:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T21:02:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T21:02:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:02:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:02:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T21:02:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:02:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T21:02:07.418 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T21:02:07.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 -- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff9cc05f270 msgr2=0x7ff9cc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff9cc05f270 0x7ff9cc061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7ff9d000bbc0 tx=0x7ff9d0008040).stop 2023-11-06T21:02:07.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 -- 172.21.15.33:0/3446159196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00c1840 msgr2=0x7ff9e00afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00c1840 0x7ff9e00afd20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7ff9d4009fd0 tx=0x7ff9d4017000).stop 2023-11-06T21:02:07.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 -- 172.21.15.33:0/3446159196 shutdown_connections 2023-11-06T21:02:07.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff9cc05f270 0x7ff9cc061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9e00aa2d0 0x7ff9e00c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00c6240 0x7ff9e0144a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 --2- 172.21.15.33:0/3446159196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9e00c1840 0x7ff9e00afd20 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 -- 172.21.15.33:0/3446159196 >> 172.21.15.33:0/3446159196 conn(0x7ff9e005ec80 msgr2=0x7ff9e00ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:07.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 -- 172.21.15.33:0/3446159196 shutdown_connections 2023-11-06T21:02:07.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.382+0000 7ff9e51ff700 1 -- 172.21.15.33:0/3446159196 wait complete. 2023-11-06T21:02:07.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:02:07.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.619+0000 7f6f069ff700 1 Processor -- start 2023-11-06T21:02:07.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.619+0000 7f6f069ff700 1 -- start start 2023-11-06T21:02:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6f069ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6f000c61d0 0x7f6f000c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6f069ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f001554a0 con 0x7f6f000c5d50 2023-11-06T21:02:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6f069ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f00155930 con 0x7f6f000c1840 2023-11-06T21:02:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6f069ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f00154960 con 0x7f6f000c61d0 2023-11-06T21:02:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6eff7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6f000c5d50 0x7f6f000bfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50264/0 (socket says 172.21.15.33:50264) 2023-11-06T21:02:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6eff7fe700 1 -- 172.21.15.33:0/3523071077 learned_addr learned my addr 172.21.15.33:0/3523071077 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6f04f9c700 1 --2- 172.21.15.33:0/3523071077 >> v2:172.21.15.69:3300/0 conn(0x7f6f000c61d0 0x7f6f000c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6efeffd700 1 -- 172.21.15.33:0/3523071077 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504556558 0 0) 0x7f6f001554a0 con 0x7f6f000c5d50 2023-11-06T21:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6efeffd700 1 -- 172.21.15.33:0/3523071077 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee80033f0 con 0x7f6f000c5d50 2023-11-06T21:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6f04f9c700 1 -- 172.21.15.33:0/3523071077 >> v1:172.21.15.69:6789/0 conn(0x7f6f000c1840 legacy=0x7f6f000a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.621+0000 7f6efeffd700 1 -- 172.21.15.33:0/3523071077 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655001042 0 0) 0x7f6f00155930 con 0x7f6f000c1840 2023-11-06T21:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.622+0000 7f6f04f9c700 1 -- 172.21.15.33:0/3523071077 >> v1:172.21.15.33:6789/0 conn(0x7f6f000c5d50 legacy=0x7f6f000bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.622+0000 7f6f04f9c700 1 -- 172.21.15.33:0/3523071077 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f000aa4f0 con 0x7f6f000c61d0 2023-11-06T21:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.622+0000 7f6f04f9c700 1 --2- 172.21.15.33:0/3523071077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f000c61d0 0x7f6f000c0480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f6ef8009e60 tx=0x7f6ef8009780).ready entity=mon.1 client_cookie=c7bb37fa89c6bb46 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.622+0000 7f6efeffd700 1 -- 172.21.15.33:0/3523071077 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800e070 con 0x7f6f000c61d0 2023-11-06T21:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.622+0000 7f6efeffd700 1 -- 172.21.15.33:0/3523071077 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef800bb60 con 0x7f6f000c61d0 2023-11-06T21:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.622+0000 7f6efeffd700 1 -- 172.21.15.33:0/3523071077 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800bee0 con 0x7f6f000c61d0 2023-11-06T21:02:07.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.622+0000 7f6f069ff700 1 -- 172.21.15.33:0/3523071077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f000c61d0 msgr2=0x7f6f000c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.622+0000 7f6f069ff700 1 --2- 172.21.15.33:0/3523071077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f000c61d0 0x7f6f000c0480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f6ef8009e60 tx=0x7f6ef8009780).stop 2023-11-06T21:02:07.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.623+0000 7f6f069ff700 1 -- 172.21.15.33:0/3523071077 shutdown_connections 2023-11-06T21:02:07.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.623+0000 7f6f069ff700 1 --2- 172.21.15.33:0/3523071077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f000c61d0 0x7f6f000c0480 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.623+0000 7f6f069ff700 1 -- 172.21.15.33:0/3523071077 >> 172.21.15.33:0/3523071077 conn(0x7f6f0005ec80 msgr2=0x7f6f000cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:07.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.623+0000 7f6f069ff700 1 -- 172.21.15.33:0/3523071077 shutdown_connections 2023-11-06T21:02:07.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.623+0000 7f6f069ff700 1 -- 172.21.15.33:0/3523071077 wait complete. 2023-11-06T21:02:07.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.624+0000 7f6f069ff700 1 Processor -- start 2023-11-06T21:02:07.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.624+0000 7f6f069ff700 1 -- start start 2023-11-06T21:02:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.625+0000 7f6f069ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f000c1840 0x7f6f00149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.625+0000 7f6f069ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f000c5d50 0x7f6f00148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.625+0000 7f6f069ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f000c61d0 0x7f6f000ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.625+0000 7f6f069ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0014d0d0 con 0x7f6f000c61d0 2023-11-06T21:02:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.625+0000 7f6f069ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f001499c0 con 0x7f6f000c5d50 2023-11-06T21:02:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.625+0000 7f6f069ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0014ca80 con 0x7f6f000c1840 2023-11-06T21:02:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6f04f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f000c61d0 0x7f6f000ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6f04f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f000c61d0 0x7f6f000ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50296/0 (socket says 172.21.15.33:50296) 2023-11-06T21:02:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6f04f9c700 1 -- 172.21.15.33:0/320589048 learned_addr learned my addr 172.21.15.33:0/320589048 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6eff7fe700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f000c5d50 0x7f6f00148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6efffff700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f000c1840 0x7f6f00149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6f04f9c700 1 -- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f000c1840 msgr2=0x7f6f00149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6f04f9c700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f000c1840 0x7f6f00149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6f04f9c700 1 -- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f000c5d50 msgr2=0x7f6f00148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6f04f9c700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f000c5d50 0x7f6f00148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6f04f9c700 1 -- 172.21.15.33:0/320589048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef8009a20 con 0x7f6f000c61d0 2023-11-06T21:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6f04f9c700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f000c61d0 0x7f6f000ace50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f6ef8009920 tx=0x7f6f00155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6ee7fff700 1 -- 172.21.15.33:0/320589048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef801d630 con 0x7f6f000c61d0 2023-11-06T21:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6ee7fff700 1 -- 172.21.15.33:0/320589048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef801d790 con 0x7f6f000c61d0 2023-11-06T21:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6ee7fff700 1 -- 172.21.15.33:0/320589048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef801db30 con 0x7f6f000c61d0 2023-11-06T21:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.626+0000 7f6f069ff700 1 -- 172.21.15.33:0/320589048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0014cd60 con 0x7f6f000c61d0 2023-11-06T21:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.627+0000 7f6f069ff700 1 -- 172.21.15.33:0/320589048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0014d560 con 0x7f6f000c61d0 2023-11-06T21:02:07.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.628+0000 7f6f069ff700 1 -- 172.21.15.33:0/320589048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0014d240 con 0x7f6f000c61d0 2023-11-06T21:02:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.628+0000 7f6ee7fff700 1 -- 172.21.15.33:0/320589048 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6ef8003040 con 0x7f6f000c61d0 2023-11-06T21:02:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.629+0000 7f6ee7fff700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6eec05f270 0x7f6eec061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.629+0000 7f6ee7fff700 1 -- 172.21.15.33:0/320589048 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f6ef807bf70 con 0x7f6f000c61d0 2023-11-06T21:02:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.630+0000 7f6efffff700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6eec05f270 0x7f6eec061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.630+0000 7f6efffff700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6eec05f270 0x7f6eec061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f6ef00035f0 tx=0x7f6ef0009040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:07.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.633+0000 7f6ee7fff700 1 -- 172.21.15.33:0/320589048 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6ef80aaaf0 con 0x7f6f000c61d0 2023-11-06T21:02:07.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.814+0000 7f6f069ff700 1 -- 172.21.15.33:0/320589048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f0006c6c0 con 0x7f6f000c61d0 2023-11-06T21:02:07.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.815+0000 7f6ee7fff700 1 -- 172.21.15.33:0/320589048 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+13771 (secure 0 0 0) 0x7f6ef804d410 con 0x7f6f000c61d0 2023-11-06T21:02:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:07.816 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":127,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:02:06.890964+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:02:07.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 -- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6eec05f270 msgr2=0x7f6eec061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6eec05f270 0x7f6eec061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f6ef00035f0 tx=0x7f6ef0009040).stop 2023-11-06T21:02:07.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 -- 172.21.15.33:0/320589048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f000c61d0 msgr2=0x7f6f000ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:07.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f000c61d0 0x7f6f000ace50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f6ef8009920 tx=0x7f6f00155410).stop 2023-11-06T21:02:07.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 -- 172.21.15.33:0/320589048 shutdown_connections 2023-11-06T21:02:07.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6eec05f270 0x7f6eec061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f000c1840 0x7f6f00149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f000c61d0 0x7f6f000ace50 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 --2- 172.21.15.33:0/320589048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f000c5d50 0x7f6f00148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:07.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 -- 172.21.15.33:0/320589048 >> 172.21.15.33:0/320589048 conn(0x7f6f0005ec80 msgr2=0x7f6f00150dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:07.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 -- 172.21.15.33:0/320589048 shutdown_connections 2023-11-06T21:02:07.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:07.817+0000 7f6f069ff700 1 -- 172.21.15.33:0/320589048 wait complete. 2023-11-06T21:02:07.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T21:02:07.837 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T21:02:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.035+0000 7f6617bff700 1 Processor -- start 2023-11-06T21:02:08.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.036+0000 7f6617bff700 1 -- start start 2023-11-06T21:02:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.037+0000 7f6617bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66101546a0 0x7f6610064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.037+0000 7f6617bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66100641f0 con 0x7f66100be2c0 2023-11-06T21:02:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.037+0000 7f6617bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66100638c0 con 0x7f6610066560 2023-11-06T21:02:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.037+0000 7f6617bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6610154a80 con 0x7f66101546a0 2023-11-06T21:02:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.037+0000 7f661599b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66101546a0 0x7f6610064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.037+0000 7f661619c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f66100be2c0 0x7f66100c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42098/0 (socket says 172.21.15.33:42098) 2023-11-06T21:02:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.037+0000 7f661599b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66101546a0 0x7f6610064680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36974/0 (socket says 172.21.15.33:36974) 2023-11-06T21:02:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.037+0000 7f661619c700 1 -- 172.21.15.33:0/837454982 learned_addr learned my addr 172.21.15.33:0/837454982 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.038+0000 7f661599b700 1 -- 172.21.15.33:0/837454982 >> v1:172.21.15.69:6789/0 conn(0x7f6610066560 legacy=0x7f66100bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.038+0000 7f661599b700 1 -- 172.21.15.33:0/837454982 >> v1:172.21.15.33:6789/0 conn(0x7f66100be2c0 legacy=0x7f66100c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.038+0000 7f661599b700 1 -- 172.21.15.33:0/837454982 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610063ba0 con 0x7f66101546a0 2023-11-06T21:02:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.038+0000 7f6614999700 1 -- 172.21.15.33:0/837454982 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434419302 0 0) 0x7f66100641f0 con 0x7f66100be2c0 2023-11-06T21:02:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.038+0000 7f661599b700 1 --2- 172.21.15.33:0/837454982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66101546a0 0x7f6610064680 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f660c009760 tx=0x7f660c009060).ready entity=mon.0 client_cookie=b320daca9777a2f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.038+0000 7f6614999700 1 -- 172.21.15.33:0/837454982 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00b580 con 0x7f66101546a0 2023-11-06T21:02:08.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.038+0000 7f6614999700 1 -- 172.21.15.33:0/837454982 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c00b6e0 con 0x7f66101546a0 2023-11-06T21:02:08.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.038+0000 7f6614999700 1 -- 172.21.15.33:0/837454982 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00ba60 con 0x7f66101546a0 2023-11-06T21:02:08.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.039+0000 7f6617bff700 1 -- 172.21.15.33:0/837454982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66101546a0 msgr2=0x7f6610064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.039+0000 7f6617bff700 1 --2- 172.21.15.33:0/837454982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66101546a0 0x7f6610064680 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f660c009760 tx=0x7f660c009060).stop 2023-11-06T21:02:08.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.039+0000 7f6617bff700 1 -- 172.21.15.33:0/837454982 shutdown_connections 2023-11-06T21:02:08.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.039+0000 7f6617bff700 1 --2- 172.21.15.33:0/837454982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66101546a0 0x7f6610064680 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.039+0000 7f6617bff700 1 -- 172.21.15.33:0/837454982 >> 172.21.15.33:0/837454982 conn(0x7f66100c8c50 msgr2=0x7f66100c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:08.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.039+0000 7f6617bff700 1 -- 172.21.15.33:0/837454982 shutdown_connections 2023-11-06T21:02:08.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.039+0000 7f6617bff700 1 -- 172.21.15.33:0/837454982 wait complete. 2023-11-06T21:02:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.041+0000 7f6617bff700 1 Processor -- start 2023-11-06T21:02:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.041+0000 7f6617bff700 1 -- start start 2023-11-06T21:02:08.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f6617bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6610066560 0x7f66100c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f6617bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100be2c0 0x7f66100ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f6617bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66101546a0 0x7f66101508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f6617bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66100c1880 con 0x7f66100be2c0 2023-11-06T21:02:08.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f6617bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66100c0f50 con 0x7f66101546a0 2023-11-06T21:02:08.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f6617bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66100c1230 con 0x7f6610066560 2023-11-06T21:02:08.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f661519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100be2c0 0x7f66100ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f661519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100be2c0 0x7f66100ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36982/0 (socket says 172.21.15.33:36982) 2023-11-06T21:02:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f661519a700 1 -- 172.21.15.33:0/496347439 learned_addr learned my addr 172.21.15.33:0/496347439 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f661599b700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6610066560 0x7f66100c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f661619c700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66101546a0 0x7f66101508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f661519a700 1 -- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6610066560 msgr2=0x7f66100c1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f661519a700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6610066560 0x7f66100c1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f661519a700 1 -- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66101546a0 msgr2=0x7f66101508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f661519a700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66101546a0 0x7f66101508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.042+0000 7f661519a700 1 -- 172.21.15.33:0/496347439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0092a0 con 0x7f66100be2c0 2023-11-06T21:02:08.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.043+0000 7f661519a700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100be2c0 0x7f66100ae2a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f660000c4c0 tx=0x7f6600007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:08.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.043+0000 7f6605ffb700 1 -- 172.21.15.33:0/496347439 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660000ce70 con 0x7f66100be2c0 2023-11-06T21:02:08.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.043+0000 7f6605ffb700 1 -- 172.21.15.33:0/496347439 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6600001e90 con 0x7f66100be2c0 2023-11-06T21:02:08.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.043+0000 7f6605ffb700 1 -- 172.21.15.33:0/496347439 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600007270 con 0x7f66100be2c0 2023-11-06T21:02:08.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.043+0000 7f6617bff700 1 -- 172.21.15.33:0/496347439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66100c1510 con 0x7f66100be2c0 2023-11-06T21:02:08.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.043+0000 7f6617bff700 1 -- 172.21.15.33:0/496347439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66100ae790 con 0x7f66100be2c0 2023-11-06T21:02:08.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.044+0000 7f6617bff700 1 -- 172.21.15.33:0/496347439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66100bf240 con 0x7f66100be2c0 2023-11-06T21:02:08.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.045+0000 7f6605ffb700 1 -- 172.21.15.33:0/496347439 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f66000066a0 con 0x7f66100be2c0 2023-11-06T21:02:08.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.046+0000 7f6605ffb700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f65f405f270 0x7f65f4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.046+0000 7f6605ffb700 1 -- 172.21.15.33:0/496347439 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f6600079930 con 0x7f66100be2c0 2023-11-06T21:02:08.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.046+0000 7f661599b700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f65f405f270 0x7f65f4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.047+0000 7f661599b700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f65f405f270 0x7f65f4061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f660c006e00 tx=0x7f660c017000).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.049+0000 7f6605ffb700 1 -- 172.21.15.33:0/496347439 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f660004acb0 con 0x7f66100be2c0 2023-11-06T21:02:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.230+0000 7f6617bff700 1 -- 172.21.15.33:0/496347439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6610064580 con 0x7f66100be2c0 2023-11-06T21:02:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.231+0000 7f6605ffb700 1 -- 172.21.15.33:0/496347439 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v127) v1 ==== 110+0+10 (secure 0 0 0) 0x7f660004e2d0 con 0x7f66100be2c0 2023-11-06T21:02:08.232 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T21:02:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 -- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f65f405f270 msgr2=0x7f65f4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f65f405f270 0x7f65f4061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f660c006e00 tx=0x7f660c017000).stop 2023-11-06T21:02:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 -- 172.21.15.33:0/496347439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100be2c0 msgr2=0x7f66100ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100be2c0 0x7f66100ae2a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f660000c4c0 tx=0x7f6600007e00).stop 2023-11-06T21:02:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 -- 172.21.15.33:0/496347439 shutdown_connections 2023-11-06T21:02:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f65f405f270 0x7f65f4061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6610066560 0x7f66100c1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66101546a0 0x7f66101508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 --2- 172.21.15.33:0/496347439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100be2c0 0x7f66100ae2a0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 -- 172.21.15.33:0/496347439 >> 172.21.15.33:0/496347439 conn(0x7f66100c8c50 msgr2=0x7f66100c3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 -- 172.21.15.33:0/496347439 shutdown_connections 2023-11-06T21:02:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.233+0000 7f6617bff700 1 -- 172.21.15.33:0/496347439 wait complete. 2023-11-06T21:02:08.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T21:02:08.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.450+0000 7f8b0dbff700 1 Processor -- start 2023-11-06T21:02:08.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.450+0000 7f8b0dbff700 1 -- start start 2023-11-06T21:02:08.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.451+0000 7f8b0dbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 0x7f8b08157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b0dbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b080c4b00 0x7f8b080c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b0dbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b08064bf0 con 0x7f8b08062710 2023-11-06T21:02:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b0dbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b080c4f60 con 0x7f8b080c4b00 2023-11-06T21:02:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b0dbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0805e490 con 0x7f8b080c4720 2023-11-06T21:02:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b06ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b080c4b00 0x7f8b080c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b06ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b080c4b00 0x7f8b080c2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36988/0 (socket says 172.21.15.33:36988) 2023-11-06T21:02:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b06ffd700 1 -- 172.21.15.33:0/657717114 learned_addr learned my addr 172.21.15.33:0/657717114 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b077fe700 1 --2- 172.21.15.33:0/657717114 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 0x7f8b08157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b06ffd700 1 -- 172.21.15.33:0/657717114 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 msgr2=0x7f8b08157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b06ffd700 1 --2- 172.21.15.33:0/657717114 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 0x7f8b08157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b06ffd700 1 -- 172.21.15.33:0/657717114 >> v1:172.21.15.69:6789/0 conn(0x7f8b08062710 legacy=0x7f8b0813d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b06ffd700 1 -- 172.21.15.33:0/657717114 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b08062bc0 con 0x7f8b080c4b00 2023-11-06T21:02:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.452+0000 7f8b077fe700 1 --2- 172.21.15.33:0/657717114 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 0x7f8b08157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b06ffd700 1 --2- 172.21.15.33:0/657717114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 0x7f8b080c2860 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f8afc006fd0 tx=0x7f8afc0090f0).ready entity=mon.0 client_cookie=3b5ab0139051a051 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b067fc700 1 -- 172.21.15.33:0/657717114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00b600 con 0x7f8b080c4b00 2023-11-06T21:02:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b067fc700 1 -- 172.21.15.33:0/657717114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc00b760 con 0x7f8b080c4b00 2023-11-06T21:02:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b067fc700 1 -- 172.21.15.33:0/657717114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00bae0 con 0x7f8b080c4b00 2023-11-06T21:02:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b0dbff700 1 -- 172.21.15.33:0/657717114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 msgr2=0x7f8b080c2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b0dbff700 1 --2- 172.21.15.33:0/657717114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 0x7f8b080c2860 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f8afc006fd0 tx=0x7f8afc0090f0).stop 2023-11-06T21:02:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b0dbff700 1 -- 172.21.15.33:0/657717114 shutdown_connections 2023-11-06T21:02:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b0dbff700 1 --2- 172.21.15.33:0/657717114 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 0x7f8b08157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b0dbff700 1 --2- 172.21.15.33:0/657717114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 0x7f8b080c2860 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b0dbff700 1 -- 172.21.15.33:0/657717114 >> 172.21.15.33:0/657717114 conn(0x7f8b080c7dc0 msgr2=0x7f8b080c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.453+0000 7f8b0dbff700 1 -- 172.21.15.33:0/657717114 shutdown_connections 2023-11-06T21:02:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.454+0000 7f8b0dbff700 1 -- 172.21.15.33:0/657717114 wait complete. 2023-11-06T21:02:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.455+0000 7f8b0dbff700 1 Processor -- start 2023-11-06T21:02:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.455+0000 7f8b0dbff700 1 -- start start 2023-11-06T21:02:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.456+0000 7f8b0dbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b08062710 0x7f8b08137c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.456+0000 7f8b0dbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 0x7f8b081380f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.456+0000 7f8b0dbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 0x7f8b08139a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.456+0000 7f8b0dbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b08135b90 con 0x7f8b080c4b00 2023-11-06T21:02:08.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.456+0000 7f8b0dbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b081385e0 con 0x7f8b08062710 2023-11-06T21:02:08.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.456+0000 7f8b0dbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b080acea0 con 0x7f8b080c4720 2023-11-06T21:02:08.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.456+0000 7f8b07fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 0x7f8b08139a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.456+0000 7f8b07fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 0x7f8b08139a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36990/0 (socket says 172.21.15.33:36990) 2023-11-06T21:02:08.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.456+0000 7f8b07fff700 1 -- 172.21.15.33:0/555517275 learned_addr learned my addr 172.21.15.33:0/555517275 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8b077fe700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b08062710 0x7f8b08137c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8b06ffd700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 0x7f8b081380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8b07fff700 1 -- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 msgr2=0x7f8b081380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8b07fff700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 0x7f8b081380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8b07fff700 1 -- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b08062710 msgr2=0x7f8b08137c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8b07fff700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b08062710 0x7f8b08137c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8b07fff700 1 -- 172.21.15.33:0/555517275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc009370 con 0x7f8b080c4b00 2023-11-06T21:02:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8b07fff700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 0x7f8b08139a70 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f8b0000b810 tx=0x7f8b00009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8aef7fe700 1 -- 172.21.15.33:0/555517275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000d500 con 0x7f8b080c4b00 2023-11-06T21:02:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8aef7fe700 1 -- 172.21.15.33:0/555517275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0000d660 con 0x7f8b080c4b00 2023-11-06T21:02:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8b0dbff700 1 -- 172.21.15.33:0/555517275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b08135260 con 0x7f8b080c4b00 2023-11-06T21:02:08.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8aef7fe700 1 -- 172.21.15.33:0/555517275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000da20 con 0x7f8b080c4b00 2023-11-06T21:02:08.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.457+0000 7f8b0dbff700 1 -- 172.21.15.33:0/555517275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b08135820 con 0x7f8b080c4b00 2023-11-06T21:02:08.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.459+0000 7f8b0dbff700 1 -- 172.21.15.33:0/555517275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b08151b90 con 0x7f8b080c4b00 2023-11-06T21:02:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.459+0000 7f8aef7fe700 1 -- 172.21.15.33:0/555517275 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8b00011460 con 0x7f8b080c4b00 2023-11-06T21:02:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.460+0000 7f8aef7fe700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af405f270 0x7f8af4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.460+0000 7f8aef7fe700 1 -- 172.21.15.33:0/555517275 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f8b00079a90 con 0x7f8b080c4b00 2023-11-06T21:02:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.461+0000 7f8b077fe700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af405f270 0x7f8af4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.462+0000 7f8b077fe700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af405f270 0x7f8af4061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f8af800abd0 tx=0x7f8af8008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.464+0000 7f8aef7fe700 1 -- 172.21.15.33:0/555517275 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8b00047af0 con 0x7f8b080c4b00 2023-11-06T21:02:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.647+0000 7f8b0dbff700 1 -- 172.21.15.33:0/555517275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8b080aa0d0 con 0x7f8b080c4b00 2023-11-06T21:02:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.647+0000 7f8aef7fe700 1 -- 172.21.15.33:0/555517275 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v127) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8b0004af80 con 0x7f8b080c4b00 2023-11-06T21:02:08.648 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T21:02:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.649+0000 7f8b0dbff700 1 -- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af405f270 msgr2=0x7f8af4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.649+0000 7f8b0dbff700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af405f270 0x7f8af4061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f8af800abd0 tx=0x7f8af8008040).stop 2023-11-06T21:02:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.649+0000 7f8b0dbff700 1 -- 172.21.15.33:0/555517275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 msgr2=0x7f8b08139a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.649+0000 7f8b0dbff700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 0x7f8b08139a70 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f8b0000b810 tx=0x7f8b00009290).stop 2023-11-06T21:02:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.650+0000 7f8b0dbff700 1 -- 172.21.15.33:0/555517275 shutdown_connections 2023-11-06T21:02:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.650+0000 7f8b0dbff700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8af405f270 0x7f8af4061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.650+0000 7f8b0dbff700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b080c4b00 0x7f8b08139a70 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.650+0000 7f8b0dbff700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b08062710 0x7f8b08137c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.650+0000 7f8b0dbff700 1 --2- 172.21.15.33:0/555517275 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b080c4720 0x7f8b081380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.650+0000 7f8b0dbff700 1 -- 172.21.15.33:0/555517275 >> 172.21.15.33:0/555517275 conn(0x7f8b080c7dc0 msgr2=0x7f8b080c2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.650+0000 7f8b0dbff700 1 -- 172.21.15.33:0/555517275 shutdown_connections 2023-11-06T21:02:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.650+0000 7f8b0dbff700 1 -- 172.21.15.33:0/555517275 wait complete. 2023-11-06T21:02:08.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T21:02:08.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.864+0000 7f9536dff700 1 Processor -- start 2023-11-06T21:02:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.864+0000 7f9536dff700 1 -- start start 2023-11-06T21:02:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.865+0000 7f9536dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f95301521a0 0x7f95300c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.865+0000 7f9536dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9530154e90 con 0x7f9530151dc0 2023-11-06T21:02:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.865+0000 7f9536dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95300639d0 con 0x7f9530155320 2023-11-06T21:02:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.865+0000 7f9536dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9530063cb0 con 0x7f95301521a0 2023-11-06T21:02:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.865+0000 7f9534b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9530151dc0 0x7f9530063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42104/0 (socket says 172.21.15.33:42104) 2023-11-06T21:02:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f9534b9b700 1 -- 172.21.15.33:0/3190422457 learned_addr learned my addr 172.21.15.33:0/3190422457 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f952ffff700 1 --2- 172.21.15.33:0/3190422457 >> v2:172.21.15.69:3300/0 conn(0x7f95301521a0 0x7f95300c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f952f7fe700 1 -- 172.21.15.33:0/3190422457 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097862686 0 0) 0x7f9530154e90 con 0x7f9530151dc0 2023-11-06T21:02:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f952f7fe700 1 -- 172.21.15.33:0/3190422457 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95200033f0 con 0x7f9530151dc0 2023-11-06T21:02:08.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f952f7fe700 1 -- 172.21.15.33:0/3190422457 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317818851 0 0) 0x7f95300639d0 con 0x7f9530155320 2023-11-06T21:02:08.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f952f7fe700 1 -- 172.21.15.33:0/3190422457 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9530154e90 con 0x7f9530155320 2023-11-06T21:02:08.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f952ffff700 1 -- 172.21.15.33:0/3190422457 >> v1:172.21.15.69:6789/0 conn(0x7f9530155320 legacy=0x7f95300c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f952ffff700 1 -- 172.21.15.33:0/3190422457 >> v1:172.21.15.33:6789/0 conn(0x7f9530151dc0 legacy=0x7f9530063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f952f7fe700 1 -- 172.21.15.33:0/3190422457 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3736262429 0 0) 0x7f95200033f0 con 0x7f9530151dc0 2023-11-06T21:02:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f952ffff700 1 -- 172.21.15.33:0/3190422457 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9530063f90 con 0x7f95301521a0 2023-11-06T21:02:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.866+0000 7f952ffff700 1 --2- 172.21.15.33:0/3190422457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95301521a0 0x7f95300c0c50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f951c009760 tx=0x7f951c009060).ready entity=mon.1 client_cookie=d5289f4b67451976 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.867+0000 7f952f7fe700 1 -- 172.21.15.33:0/3190422457 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524001470 con 0x7f95301521a0 2023-11-06T21:02:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.867+0000 7f952f7fe700 1 -- 172.21.15.33:0/3190422457 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c011400 con 0x7f95301521a0 2023-11-06T21:02:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.867+0000 7f952f7fe700 1 -- 172.21.15.33:0/3190422457 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c011780 con 0x7f95301521a0 2023-11-06T21:02:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.867+0000 7f9536dff700 1 -- 172.21.15.33:0/3190422457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95301521a0 msgr2=0x7f95300c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.867+0000 7f9536dff700 1 --2- 172.21.15.33:0/3190422457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95301521a0 0x7f95300c0c50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f951c009760 tx=0x7f951c009060).stop 2023-11-06T21:02:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.867+0000 7f9536dff700 1 -- 172.21.15.33:0/3190422457 shutdown_connections 2023-11-06T21:02:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.867+0000 7f9536dff700 1 --2- 172.21.15.33:0/3190422457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95301521a0 0x7f95300c0c50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.867+0000 7f9536dff700 1 -- 172.21.15.33:0/3190422457 >> 172.21.15.33:0/3190422457 conn(0x7f953005e1b0 msgr2=0x7f953005e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.867+0000 7f9536dff700 1 -- 172.21.15.33:0/3190422457 shutdown_connections 2023-11-06T21:02:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.867+0000 7f9536dff700 1 -- 172.21.15.33:0/3190422457 wait complete. 2023-11-06T21:02:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.869+0000 7f9536dff700 1 Processor -- start 2023-11-06T21:02:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.870+0000 7f9536dff700 1 -- start start 2023-11-06T21:02:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f9536dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9530151dc0 0x7f9530141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f9536dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f95301521a0 0x7f95300c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f9536dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9530155320 0x7f9530152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f9536dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9530142090 con 0x7f9530155320 2023-11-06T21:02:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f9536dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95300c6a90 con 0x7f9530151dc0 2023-11-06T21:02:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f9536dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95300ae790 con 0x7f95301521a0 2023-11-06T21:02:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f953539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9530155320 0x7f9530152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f953539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9530155320 0x7f9530152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37002/0 (socket says 172.21.15.33:37002) 2023-11-06T21:02:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f953539c700 1 -- 172.21.15.33:0/711885146 learned_addr learned my addr 172.21.15.33:0/711885146 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f9534b9b700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9530151dc0 0x7f9530141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.871+0000 7f952ffff700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f95301521a0 0x7f95300c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f953539c700 1 -- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f95301521a0 msgr2=0x7f95300c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f953539c700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f95301521a0 0x7f95300c3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f953539c700 1 -- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9530151dc0 msgr2=0x7f9530141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f953539c700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9530151dc0 0x7f9530141ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f953539c700 1 -- 172.21.15.33:0/711885146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c0092a0 con 0x7f9530155320 2023-11-06T21:02:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f953539c700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9530155320 0x7f9530152660 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f952800c7c0 tx=0x7f9528007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f952cff9700 1 -- 172.21.15.33:0/711885146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528002240 con 0x7f9530155320 2023-11-06T21:02:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f952cff9700 1 -- 172.21.15.33:0/711885146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95280023a0 con 0x7f9530155320 2023-11-06T21:02:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f952cff9700 1 -- 172.21.15.33:0/711885146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528001820 con 0x7f9530155320 2023-11-06T21:02:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f9536dff700 1 -- 172.21.15.33:0/711885146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95300c5ba0 con 0x7f9530155320 2023-11-06T21:02:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.872+0000 7f9536dff700 1 -- 172.21.15.33:0/711885146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95300c6160 con 0x7f9530155320 2023-11-06T21:02:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.873+0000 7f9536dff700 1 -- 172.21.15.33:0/711885146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95300c0a30 con 0x7f9530155320 2023-11-06T21:02:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.874+0000 7f952cff9700 1 -- 172.21.15.33:0/711885146 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9528001bd0 con 0x7f9530155320 2023-11-06T21:02:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.875+0000 7f952cff9700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f951405f270 0x7f9514061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.875+0000 7f952cff9700 1 -- 172.21.15.33:0/711885146 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f9528079f40 con 0x7f9530155320 2023-11-06T21:02:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.876+0000 7f9534b9b700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f951405f270 0x7f9514061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.876+0000 7f9534b9b700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f951405f270 0x7f9514061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9524001380 tx=0x7f9524001620).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:08.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:08.879+0000 7f952cff9700 1 -- 172.21.15.33:0/711885146 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f952804b3e0 con 0x7f9530155320 2023-11-06T21:02:09.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.063+0000 7f9536dff700 1 -- 172.21.15.33:0/711885146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9530062ac0 con 0x7f9530155320 2023-11-06T21:02:09.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.064+0000 7f952cff9700 1 -- 172.21.15.33:0/711885146 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v127) v1 ==== 84+0+143 (secure 0 0 0) 0x7f952804ea00 con 0x7f9530155320 2023-11-06T21:02:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T21:02:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T21:02:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T21:02:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:02:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T21:02:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T21:02:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T21:02:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 -- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f951405f270 msgr2=0x7f9514061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f951405f270 0x7f9514061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9524001380 tx=0x7f9524001620).stop 2023-11-06T21:02:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 -- 172.21.15.33:0/711885146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9530155320 msgr2=0x7f9530152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9530155320 0x7f9530152660 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f952800c7c0 tx=0x7f9528007f50).stop 2023-11-06T21:02:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 -- 172.21.15.33:0/711885146 shutdown_connections 2023-11-06T21:02:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f951405f270 0x7f9514061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9530155320 0x7f9530152660 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9530151dc0 0x7f9530141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 --2- 172.21.15.33:0/711885146 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f95301521a0 0x7f95300c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 -- 172.21.15.33:0/711885146 >> 172.21.15.33:0/711885146 conn(0x7f953005e1b0 msgr2=0x7f9530157100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 -- 172.21.15.33:0/711885146 shutdown_connections 2023-11-06T21:02:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.066+0000 7f9536dff700 1 -- 172.21.15.33:0/711885146 wait complete. 2023-11-06T21:02:09.084 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T21:02:09.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T21:02:09.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:02:09.194 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:02:09.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.288+0000 7f6074dff700 1 Processor -- start 2023-11-06T21:02:09.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.288+0000 7f6074dff700 1 -- start start 2023-11-06T21:02:09.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.289+0000 7f6074dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60700c8ee0 0x7f60700ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:09.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.289+0000 7f6074dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f60701542b0 0x7f60700c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.289+0000 7f6074dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60700c9f80 con 0x7f6070153ed0 2023-11-06T21:02:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.289+0000 7f6074dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60700c6000 con 0x7f60701542b0 2023-11-06T21:02:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.289+0000 7f6074dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60700c9370 con 0x7f60700c8ee0 2023-11-06T21:02:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.289+0000 7f606e59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6070153ed0 0x7f6070158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42118/0 (socket says 172.21.15.33:42118) 2023-11-06T21:02:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.289+0000 7f606e59c700 1 -- 172.21.15.33:0/2518703263 learned_addr learned my addr 172.21.15.33:0/2518703263 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.290+0000 7f606ed9d700 1 --2- 172.21.15.33:0/2518703263 >> v2:172.21.15.69:3300/0 conn(0x7f60701542b0 0x7f60700c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.290+0000 7f606dd9b700 1 --2- 172.21.15.33:0/2518703263 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60700c8ee0 0x7f60700ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.290+0000 7f606d59a700 1 -- 172.21.15.33:0/2518703263 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481902818 0 0) 0x7f60700c9f80 con 0x7f6070153ed0 2023-11-06T21:02:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.290+0000 7f606d59a700 1 -- 172.21.15.33:0/2518703263 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60580033f0 con 0x7f6070153ed0 2023-11-06T21:02:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.290+0000 7f606dd9b700 1 -- 172.21.15.33:0/2518703263 >> v2:172.21.15.69:3300/0 conn(0x7f60701542b0 msgr2=0x7f60700c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:09.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.290+0000 7f606dd9b700 1 --2- 172.21.15.33:0/2518703263 >> v2:172.21.15.69:3300/0 conn(0x7f60701542b0 0x7f60700c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:09.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.290+0000 7f606dd9b700 1 -- 172.21.15.33:0/2518703263 >> v1:172.21.15.33:6789/0 conn(0x7f6070153ed0 legacy=0x7f6070158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:09.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.290+0000 7f606dd9b700 1 -- 172.21.15.33:0/2518703263 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60700c9650 con 0x7f60700c8ee0 2023-11-06T21:02:09.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.290+0000 7f606ed9d700 1 --2- 172.21.15.33:0/2518703263 >> v2:172.21.15.69:3300/0 conn(0x7f60701542b0 0x7f60700c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:02:09.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.290+0000 7f606dd9b700 1 --2- 172.21.15.33:0/2518703263 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60700c8ee0 0x7f60700ca410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f6064009760 tx=0x7f6064009060).ready entity=mon.2 client_cookie=761a00fff2090b63 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:09.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f606d59a700 1 -- 172.21.15.33:0/2518703263 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400d4a0 con 0x7f60700c8ee0 2023-11-06T21:02:09.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f606d59a700 1 -- 172.21.15.33:0/2518703263 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606400d600 con 0x7f60700c8ee0 2023-11-06T21:02:09.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f606d59a700 1 -- 172.21.15.33:0/2518703263 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400d980 con 0x7f60700c8ee0 2023-11-06T21:02:09.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f6074dff700 1 -- 172.21.15.33:0/2518703263 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60700c8ee0 msgr2=0x7f60700ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:09.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f6074dff700 1 --2- 172.21.15.33:0/2518703263 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60700c8ee0 0x7f60700ca410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f6064009760 tx=0x7f6064009060).stop 2023-11-06T21:02:09.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f6074dff700 1 -- 172.21.15.33:0/2518703263 shutdown_connections 2023-11-06T21:02:09.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f6074dff700 1 --2- 172.21.15.33:0/2518703263 >> v2:172.21.15.69:3300/0 conn(0x7f60701542b0 0x7f60700c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:09.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f6074dff700 1 --2- 172.21.15.33:0/2518703263 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60700c8ee0 0x7f60700ca410 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:09.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f6074dff700 1 -- 172.21.15.33:0/2518703263 >> 172.21.15.33:0/2518703263 conn(0x7f607005e1b0 msgr2=0x7f607005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:09.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f6074dff700 1 -- 172.21.15.33:0/2518703263 shutdown_connections 2023-11-06T21:02:09.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.291+0000 7f6074dff700 1 -- 172.21.15.33:0/2518703263 wait complete. 2023-11-06T21:02:09.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.293+0000 7f6074dff700 1 Processor -- start 2023-11-06T21:02:09.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.293+0000 7f6074dff700 1 -- start start 2023-11-06T21:02:09.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.294+0000 7f6074dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6070153ed0 0x7f60700ad630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:09.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.294+0000 7f6074dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60701542b0 0x7f607014d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:09.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.294+0000 7f6074dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60700ae7e0 0x7f60700cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.294+0000 7f6074dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607014d820 con 0x7f60700ae7e0 2023-11-06T21:02:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.294+0000 7f6074dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60700afbb0 con 0x7f6070153ed0 2023-11-06T21:02:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.294+0000 7f6074dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60700aefa0 con 0x7f60701542b0 2023-11-06T21:02:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.294+0000 7f606ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60700ae7e0 0x7f60700cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.294+0000 7f606ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60700ae7e0 0x7f60700cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37016/0 (socket says 172.21.15.33:37016) 2023-11-06T21:02:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.294+0000 7f606ed9d700 1 -- 172.21.15.33:0/2787471857 learned_addr learned my addr 172.21.15.33:0/2787471857 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:09.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.294+0000 7f606e59c700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6070153ed0 0x7f60700ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:09.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f606dd9b700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60701542b0 0x7f607014d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:09.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f606ed9d700 1 -- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60701542b0 msgr2=0x7f607014d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:09.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f606ed9d700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60701542b0 0x7f607014d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:09.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f606ed9d700 1 -- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6070153ed0 msgr2=0x7f60700ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:09.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f606ed9d700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6070153ed0 0x7f60700ad630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:09.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f606ed9d700 1 -- 172.21.15.33:0/2787471857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6068009a20 con 0x7f60700ae7e0 2023-11-06T21:02:09.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f606dd9b700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60701542b0 0x7f607014d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:02:09.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f606ed9d700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60700ae7e0 0x7f60700cae00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f6068009f20 tx=0x7f606800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:09.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f605e7fc700 1 -- 172.21.15.33:0/2787471857 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606800cbd0 con 0x7f60700ae7e0 2023-11-06T21:02:09.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f605e7fc700 1 -- 172.21.15.33:0/2787471857 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606800f040 con 0x7f60700ae7e0 2023-11-06T21:02:09.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f6074dff700 1 -- 172.21.15.33:0/2787471857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60640092a0 con 0x7f60700ae7e0 2023-11-06T21:02:09.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f605e7fc700 1 -- 172.21.15.33:0/2787471857 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068013400 con 0x7f60700ae7e0 2023-11-06T21:02:09.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.295+0000 7f6074dff700 1 -- 172.21.15.33:0/2787471857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60700af560 con 0x7f60700ae7e0 2023-11-06T21:02:09.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.297+0000 7f6074dff700 1 -- 172.21.15.33:0/2787471857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607006c6c0 con 0x7f60700ae7e0 2023-11-06T21:02:09.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.297+0000 7f605e7fc700 1 -- 172.21.15.33:0/2787471857 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f60680137f0 con 0x7f60700ae7e0 2023-11-06T21:02:09.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.298+0000 7f605e7fc700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f605005f270 0x7f6050061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:09.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.298+0000 7f605e7fc700 1 -- 172.21.15.33:0/2787471857 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f606807a960 con 0x7f60700ae7e0 2023-11-06T21:02:09.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.298+0000 7f606e59c700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f605005f270 0x7f6050061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:09.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.299+0000 7f606e59c700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f605005f270 0x7f6050061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6060005750 tx=0x7f6060009040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:09.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.302+0000 7f605e7fc700 1 -- 172.21.15.33:0/2787471857 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f606804be00 con 0x7f60700ae7e0 2023-11-06T21:02:09.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:09.485+0000 7f6074dff700 1 -- 172.21.15.33:0/2787471857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f60700c90e0 con 0x7f60700ae7e0 2023-11-06T21:02:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.498+0000 7f605e7fc700 1 -- 172.21.15.33:0/2787471857 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v128) v1 ==== 152+0+0 (secure 0 0 0) 0x7f606804f420 con 0x7f60700ae7e0 2023-11-06T21:02:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.504+0000 7f6074dff700 1 -- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f605005f270 msgr2=0x7f6050061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.504+0000 7f6074dff700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f605005f270 0x7f6050061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6060005750 tx=0x7f6060009040).stop 2023-11-06T21:02:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.504+0000 7f6074dff700 1 -- 172.21.15.33:0/2787471857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60700ae7e0 msgr2=0x7f60700cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.504+0000 7f6074dff700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60700ae7e0 0x7f60700cae00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f6068009f20 tx=0x7f606800b040).stop 2023-11-06T21:02:10.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.504+0000 7f6074dff700 1 -- 172.21.15.33:0/2787471857 shutdown_connections 2023-11-06T21:02:10.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.504+0000 7f6074dff700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f605005f270 0x7f6050061730 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0x7f6060005750 tx=0x7f6060009040).stop 2023-11-06T21:02:10.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.504+0000 7f6074dff700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60700ae7e0 0x7f60700cae00 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:10.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.504+0000 7f6074dff700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6070153ed0 0x7f60700ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:10.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.504+0000 7f6074dff700 1 --2- 172.21.15.33:0/2787471857 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f60701542b0 0x7f607014d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:10.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.504+0000 7f6074dff700 1 -- 172.21.15.33:0/2787471857 >> 172.21.15.33:0/2787471857 conn(0x7f607005e1b0 msgr2=0x7f60700ac200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:10.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.505+0000 7f6074dff700 1 -- 172.21.15.33:0/2787471857 shutdown_connections 2023-11-06T21:02:10.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.505+0000 7f6074dff700 1 -- 172.21.15.33:0/2787471857 wait complete. 2023-11-06T21:02:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T21:02:10.528 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T21:02:10.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.734+0000 7f0fe59ff700 1 Processor -- start 2023-11-06T21:02:10.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.734+0000 7f0fe59ff700 1 -- start start 2023-11-06T21:02:10.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fe59ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0fe0154330 0x7f0fe00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:10.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fe59ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0fe00c7ff0 0x7f0fe00646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:10.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fe59ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe00c6b60 con 0x7f0fe0154710 2023-11-06T21:02:10.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fe59ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe00c7700 con 0x7f0fe00c7ff0 2023-11-06T21:02:10.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fe59ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe00c5cd0 con 0x7f0fe0154330 2023-11-06T21:02:10.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fdf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0fe00c7ff0 0x7f0fe00646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:10.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fdf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0fe00c7ff0 0x7f0fe00646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37026/0 (socket says 172.21.15.33:37026) 2023-11-06T21:02:10.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fdf7fe700 1 -- 172.21.15.33:0/126669259 learned_addr learned my addr 172.21.15.33:0/126669259 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:10.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fdeffd700 1 --2- 172.21.15.33:0/126669259 >> v2:172.21.15.69:3300/0 conn(0x7f0fe0154330 0x7f0fe00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:10.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fdf7fe700 1 -- 172.21.15.33:0/126669259 >> v2:172.21.15.69:3300/0 conn(0x7f0fe0154330 msgr2=0x7f0fe00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:10.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fdf7fe700 1 --2- 172.21.15.33:0/126669259 >> v2:172.21.15.69:3300/0 conn(0x7f0fe0154330 0x7f0fe00c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:10.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fdf7fe700 1 -- 172.21.15.33:0/126669259 >> v1:172.21.15.69:6789/0 conn(0x7f0fe0154710 legacy=0x7f0fe00c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:10.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.736+0000 7f0fdf7fe700 1 -- 172.21.15.33:0/126669259 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe00c5f50 con 0x7f0fe00c7ff0 2023-11-06T21:02:10.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.737+0000 7f0fdf7fe700 1 --2- 172.21.15.33:0/126669259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe00c7ff0 0x7f0fe00646c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f0fe00605d0 tx=0x7f0fd8009840).ready entity=mon.0 client_cookie=ecd7bd255c2aa26b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:10.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.737+0000 7f0fddffb700 1 -- 172.21.15.33:0/126669259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800e070 con 0x7f0fe00c7ff0 2023-11-06T21:02:10.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.737+0000 7f0fddffb700 1 -- 172.21.15.33:0/126669259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd800ba50 con 0x7f0fe00c7ff0 2023-11-06T21:02:10.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.737+0000 7f0fddffb700 1 -- 172.21.15.33:0/126669259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800bdd0 con 0x7f0fe00c7ff0 2023-11-06T21:02:10.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.737+0000 7f0fe59ff700 1 -- 172.21.15.33:0/126669259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe00c7ff0 msgr2=0x7f0fe00646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:10.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.737+0000 7f0fe59ff700 1 --2- 172.21.15.33:0/126669259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe00c7ff0 0x7f0fe00646c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f0fe00605d0 tx=0x7f0fd8009840).stop 2023-11-06T21:02:10.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.737+0000 7f0fe59ff700 1 -- 172.21.15.33:0/126669259 shutdown_connections 2023-11-06T21:02:10.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.737+0000 7f0fe59ff700 1 --2- 172.21.15.33:0/126669259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe00c7ff0 0x7f0fe00646c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:10.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.737+0000 7f0fe59ff700 1 --2- 172.21.15.33:0/126669259 >> v2:172.21.15.69:3300/0 conn(0x7f0fe0154330 0x7f0fe00c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:10.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.737+0000 7f0fe59ff700 1 -- 172.21.15.33:0/126669259 >> 172.21.15.33:0/126669259 conn(0x7f0fe00cb910 msgr2=0x7f0fe00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:10.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.738+0000 7f0fe59ff700 1 -- 172.21.15.33:0/126669259 shutdown_connections 2023-11-06T21:02:10.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.738+0000 7f0fe59ff700 1 -- 172.21.15.33:0/126669259 wait complete. 2023-11-06T21:02:10.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.739+0000 7f0fe59ff700 1 Processor -- start 2023-11-06T21:02:10.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.739+0000 7f0fe59ff700 1 -- start start 2023-11-06T21:02:10.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.740+0000 7f0fe59ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fe00c7ff0 0x7f0fe00a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:10.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.740+0000 7f0fe59ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0154330 0x7f0fe0155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:10.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.740+0000 7f0fe59ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fe0154710 0x7f0fe00b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:10.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.740+0000 7f0fe59ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe0157e30 con 0x7f0fe0154330 2023-11-06T21:02:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.740+0000 7f0fe59ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00ac7c0 con 0x7f0fe0154710 2023-11-06T21:02:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.740+0000 7f0fe59ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00afd00 con 0x7f0fe00c7ff0 2023-11-06T21:02:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.740+0000 7f0fde7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0154330 0x7f0fe0155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.740+0000 7f0fde7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0154330 0x7f0fe0155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37038/0 (socket says 172.21.15.33:37038) 2023-11-06T21:02:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.740+0000 7f0fde7fc700 1 -- 172.21.15.33:0/3753136707 learned_addr learned my addr 172.21.15.33:0/3753136707 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:10.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fdf7fe700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fe0154710 0x7f0fe00b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:10.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fdeffd700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fe00c7ff0 0x7f0fe00a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:10.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fde7fc700 1 -- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fe00c7ff0 msgr2=0x7f0fe00a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:10.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fde7fc700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fe00c7ff0 0x7f0fe00a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:10.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fde7fc700 1 -- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fe0154710 msgr2=0x7f0fe00b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:10.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fde7fc700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fe0154710 0x7f0fe00b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:10.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fde7fc700 1 -- 172.21.15.33:0/3753136707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd8009ae0 con 0x7f0fe0154330 2023-11-06T21:02:10.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fde7fc700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0154330 0x7f0fe0155a40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f0fd400c440 tx=0x7f0fd4007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:10.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fc6ffd700 1 -- 172.21.15.33:0/3753136707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4007020 con 0x7f0fe0154330 2023-11-06T21:02:10.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fc6ffd700 1 -- 172.21.15.33:0/3753136707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd4007180 con 0x7f0fe0154330 2023-11-06T21:02:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fe59ff700 1 -- 172.21.15.33:0/3753136707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe01578b0 con 0x7f0fe0154330 2023-11-06T21:02:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.741+0000 7f0fc6ffd700 1 -- 172.21.15.33:0/3753136707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4007520 con 0x7f0fe0154330 2023-11-06T21:02:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.742+0000 7f0fe59ff700 1 -- 172.21.15.33:0/3753136707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe00af990 con 0x7f0fe0154330 2023-11-06T21:02:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.743+0000 7f0fe59ff700 1 -- 172.21.15.33:0/3753136707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe00af7f0 con 0x7f0fe0154330 2023-11-06T21:02:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.744+0000 7f0fc6ffd700 1 -- 172.21.15.33:0/3753136707 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0fd40068b0 con 0x7f0fe0154330 2023-11-06T21:02:10.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.745+0000 7f0fc6ffd700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fcc05f270 0x7f0fcc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:10.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.745+0000 7f0fc6ffd700 1 -- 172.21.15.33:0/3753136707 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f0fd4079f20 con 0x7f0fe0154330 2023-11-06T21:02:10.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.746+0000 7f0fdeffd700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fcc05f270 0x7f0fcc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:10.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.747+0000 7f0fdeffd700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fcc05f270 0x7f0fcc061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f0fd000bbc0 tx=0x7f0fd0008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:10.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.749+0000 7f0fc6ffd700 1 -- 172.21.15.33:0/3753136707 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0fd404b310 con 0x7f0fe0154330 2023-11-06T21:02:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:10.930+0000 7f0fe59ff700 1 -- 172.21.15.33:0/3753136707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0fe00af7f0 con 0x7f0fe0154330 2023-11-06T21:02:11.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.503+0000 7f0fc6ffd700 1 -- 172.21.15.33:0/3753136707 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v129) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0fe00af7f0 con 0x7f0fe0154330 2023-11-06T21:02:11.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.508+0000 7f0fe59ff700 1 -- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fcc05f270 msgr2=0x7f0fcc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:11.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.509+0000 7f0fe59ff700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fcc05f270 0x7f0fcc061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f0fd000bbc0 tx=0x7f0fd0008040).stop 2023-11-06T21:02:11.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.509+0000 7f0fe59ff700 1 -- 172.21.15.33:0/3753136707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0154330 msgr2=0x7f0fe0155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:11.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.509+0000 7f0fe59ff700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0154330 0x7f0fe0155a40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f0fd400c440 tx=0x7f0fd4007dd0).stop 2023-11-06T21:02:11.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.509+0000 7f0fe59ff700 1 -- 172.21.15.33:0/3753136707 shutdown_connections 2023-11-06T21:02:11.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.509+0000 7f0fe59ff700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fcc05f270 0x7f0fcc061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:11.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.509+0000 7f0fe59ff700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fe00c7ff0 0x7f0fe00a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:11.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.509+0000 7f0fe59ff700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fe0154710 0x7f0fe00b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:11.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.509+0000 7f0fe59ff700 1 --2- 172.21.15.33:0/3753136707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0154330 0x7f0fe0155a40 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:11.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.509+0000 7f0fe59ff700 1 -- 172.21.15.33:0/3753136707 >> 172.21.15.33:0/3753136707 conn(0x7f0fe00cb910 msgr2=0x7f0fe00c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:11.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.512+0000 7f0fe59ff700 1 -- 172.21.15.33:0/3753136707 shutdown_connections 2023-11-06T21:02:11.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.513+0000 7f0fe59ff700 1 -- 172.21.15.33:0/3753136707 wait complete. 2023-11-06T21:02:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T21:02:11.549 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T21:02:11.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.750+0000 7f3505dff700 1 Processor -- start 2023-11-06T21:02:11.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.751+0000 7f3505dff700 1 -- start start 2023-11-06T21:02:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.752+0000 7f3505dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f35000632f0 0x7f35000636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.752+0000 7f3505dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 0x7f350014e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.752+0000 7f3505dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35001584f0 con 0x7f3500158980 2023-11-06T21:02:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.752+0000 7f3505dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35001559e0 con 0x7f35000632f0 2023-11-06T21:02:11.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.752+0000 7f3505dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3500158e10 con 0x7f350014e2d0 2023-11-06T21:02:11.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.752+0000 7f34feffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3500158980 0x7f35001549d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42126/0 (socket says 172.21.15.33:42126) 2023-11-06T21:02:11.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.752+0000 7f34feffd700 1 -- 172.21.15.33:0/560837666 learned_addr learned my addr 172.21.15.33:0/560837666 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:11.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.752+0000 7f34ff7fe700 1 --2- 172.21.15.33:0/560837666 >> v2:172.21.15.69:3300/0 conn(0x7f35000632f0 0x7f35000636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:11.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.752+0000 7f34fffff700 1 --2- 172.21.15.33:0/560837666 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 0x7f350014e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34fe7fc700 1 -- 172.21.15.33:0/560837666 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140790752 0 0) 0x7f35001584f0 con 0x7f3500158980 2023-11-06T21:02:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34fe7fc700 1 -- 172.21.15.33:0/560837666 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e80033f0 con 0x7f3500158980 2023-11-06T21:02:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34fffff700 1 -- 172.21.15.33:0/560837666 >> v2:172.21.15.69:3300/0 conn(0x7f35000632f0 msgr2=0x7f35000636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34fffff700 1 --2- 172.21.15.33:0/560837666 >> v2:172.21.15.69:3300/0 conn(0x7f35000632f0 0x7f35000636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34fffff700 1 -- 172.21.15.33:0/560837666 >> v1:172.21.15.33:6789/0 conn(0x7f3500158980 legacy=0x7f35001549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34fffff700 1 -- 172.21.15.33:0/560837666 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500155420 con 0x7f350014e2d0 2023-11-06T21:02:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34ff7fe700 1 --2- 172.21.15.33:0/560837666 >> v2:172.21.15.69:3300/0 conn(0x7f35000632f0 0x7f35000636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:02:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34fffff700 1 --2- 172.21.15.33:0/560837666 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 0x7f350014e730 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f34f8009e60 tx=0x7f34f8009780).ready entity=mon.2 client_cookie=d47842bbf4ece8c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:11.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34fe7fc700 1 -- 172.21.15.33:0/560837666 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f800e070 con 0x7f350014e2d0 2023-11-06T21:02:11.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34fe7fc700 1 -- 172.21.15.33:0/560837666 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f800bab0 con 0x7f350014e2d0 2023-11-06T21:02:11.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.753+0000 7f34fe7fc700 1 -- 172.21.15.33:0/560837666 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f800be30 con 0x7f350014e2d0 2023-11-06T21:02:11.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.754+0000 7f3505dff700 1 -- 172.21.15.33:0/560837666 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 msgr2=0x7f350014e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:11.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.754+0000 7f3505dff700 1 --2- 172.21.15.33:0/560837666 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 0x7f350014e730 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f34f8009e60 tx=0x7f34f8009780).stop 2023-11-06T21:02:11.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.754+0000 7f3505dff700 1 -- 172.21.15.33:0/560837666 shutdown_connections 2023-11-06T21:02:11.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.754+0000 7f3505dff700 1 --2- 172.21.15.33:0/560837666 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 0x7f350014e730 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:11.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.754+0000 7f3505dff700 1 --2- 172.21.15.33:0/560837666 >> v2:172.21.15.69:3300/0 conn(0x7f35000632f0 0x7f35000636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:11.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.754+0000 7f3505dff700 1 -- 172.21.15.33:0/560837666 >> 172.21.15.33:0/560837666 conn(0x7f35000cb910 msgr2=0x7f35000c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:11.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.754+0000 7f3505dff700 1 -- 172.21.15.33:0/560837666 shutdown_connections 2023-11-06T21:02:11.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.754+0000 7f3505dff700 1 -- 172.21.15.33:0/560837666 wait complete. 2023-11-06T21:02:11.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.756+0000 7f3505dff700 1 Processor -- start 2023-11-06T21:02:11.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.756+0000 7f3505dff700 1 -- start start 2023-11-06T21:02:11.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f3505dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35000632f0 0x7f35000ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:11.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f3505dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 0x7f350013b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:11.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f3505dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3500158980 0x7f3500155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:11.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f3505dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000c1ee0 con 0x7f3500158980 2023-11-06T21:02:11.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f3505dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3500156550 con 0x7f35000632f0 2023-11-06T21:02:11.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f3505dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3500156830 con 0x7f350014e2d0 2023-11-06T21:02:11.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f34fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3500158980 0x7f3500155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:11.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f34ff7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35000632f0 0x7f35000ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:11.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f34fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3500158980 0x7f3500155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37048/0 (socket says 172.21.15.33:37048) 2023-11-06T21:02:11.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f34fffff700 1 -- 172.21.15.33:0/3877989950 learned_addr learned my addr 172.21.15.33:0/3877989950 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:11.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f34ff7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35000632f0 0x7f35000ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:37794/0 (socket says 172.21.15.33:37794) 2023-11-06T21:02:11.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.757+0000 7f34feffd700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 0x7f350013b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:11.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f34fffff700 1 -- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 msgr2=0x7f350013b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:11.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f34fffff700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 0x7f350013b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:11.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f34fffff700 1 -- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35000632f0 msgr2=0x7f35000ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:11.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f34fffff700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35000632f0 0x7f35000ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:11.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f34fffff700 1 -- 172.21.15.33:0/3877989950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00092a0 con 0x7f3500158980 2023-11-06T21:02:11.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f34ff7fe700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35000632f0 0x7f35000ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:02:11.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f34fffff700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3500158980 0x7f3500155b20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f34f8009460 tx=0x7f34f8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:11.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f34e77fe700 1 -- 172.21.15.33:0/3877989950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8013e60 con 0x7f3500158980 2023-11-06T21:02:11.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f34e77fe700 1 -- 172.21.15.33:0/3877989950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f801d630 con 0x7f3500158980 2023-11-06T21:02:11.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f34e77fe700 1 -- 172.21.15.33:0/3877989950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f801d9d0 con 0x7f3500158980 2023-11-06T21:02:11.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f3505dff700 1 -- 172.21.15.33:0/3877989950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f8009a20 con 0x7f3500158980 2023-11-06T21:02:11.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.758+0000 7f3505dff700 1 -- 172.21.15.33:0/3877989950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35000c20f0 con 0x7f3500158980 2023-11-06T21:02:11.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.760+0000 7f3505dff700 1 -- 172.21.15.33:0/3877989950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35000c2310 con 0x7f3500158980 2023-11-06T21:02:11.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.761+0000 7f34e77fe700 1 -- 172.21.15.33:0/3877989950 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f34f8003040 con 0x7f3500158980 2023-11-06T21:02:11.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.762+0000 7f34e77fe700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f34ec05f270 0x7f34ec061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:11.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.762+0000 7f34e77fe700 1 -- 172.21.15.33:0/3877989950 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f34f807d060 con 0x7f3500158980 2023-11-06T21:02:11.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.762+0000 7f34ff7fe700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f34ec05f270 0x7f34ec061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:11.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.763+0000 7f34ff7fe700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f34ec05f270 0x7f34ec061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f34f00114c0 tx=0x7f34f0009060).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:11.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.765+0000 7f34e77fe700 1 -- 172.21.15.33:0/3877989950 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f34f804e1d0 con 0x7f3500158980 2023-11-06T21:02:11.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.989+0000 7f3505dff700 1 -- 172.21.15.33:0/3877989950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3500155700 con 0x7f3500158980 2023-11-06T21:02:11.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.994+0000 7f34e77fe700 1 -- 172.21.15.33:0/3877989950 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7f34f80517f0 con 0x7f3500158980 2023-11-06T21:02:11.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.998+0000 7f3505dff700 1 -- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f34ec05f270 msgr2=0x7f34ec061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:11.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.998+0000 7f3505dff700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f34ec05f270 0x7f34ec061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f34f00114c0 tx=0x7f34f0009060).stop 2023-11-06T21:02:11.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.998+0000 7f3505dff700 1 -- 172.21.15.33:0/3877989950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3500158980 msgr2=0x7f3500155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.998+0000 7f3505dff700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3500158980 0x7f3500155b20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f34f8009460 tx=0x7f34f8009780).stop 2023-11-06T21:02:12.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.999+0000 7f3505dff700 1 -- 172.21.15.33:0/3877989950 shutdown_connections 2023-11-06T21:02:12.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.999+0000 7f3505dff700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f34ec05f270 0x7f34ec061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.999+0000 7f3505dff700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3500158980 0x7f3500155b20 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.999+0000 7f3505dff700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35000632f0 0x7f35000ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.999+0000 7f3505dff700 1 --2- 172.21.15.33:0/3877989950 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f350014e2d0 0x7f350013b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.999+0000 7f3505dff700 1 -- 172.21.15.33:0/3877989950 >> 172.21.15.33:0/3877989950 conn(0x7f35000cb910 msgr2=0x7f35000c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:12.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.999+0000 7f3505dff700 1 -- 172.21.15.33:0/3877989950 shutdown_connections 2023-11-06T21:02:12.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:11.999+0000 7f3505dff700 1 -- 172.21.15.33:0/3877989950 wait complete. 2023-11-06T21:02:12.003 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 15 and data pool 16 2023-11-06T21:02:12.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:02:12.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.232+0000 7fc11a5ff700 1 Processor -- start 2023-11-06T21:02:12.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.232+0000 7fc11a5ff700 1 -- start start 2023-11-06T21:02:12.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.234+0000 7fc11a5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc114154110 0x7fc1141544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.234+0000 7fc11a5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1140aade0 0x7fc1140645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.234+0000 7fc11a5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc114064f70 con 0x7fc114064ac0 2023-11-06T21:02:12.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.234+0000 7fc11a5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1140bfd40 con 0x7fc1140aade0 2023-11-06T21:02:12.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.234+0000 7fc11a5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1140c0880 con 0x7fc114154110 2023-11-06T21:02:12.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.234+0000 7fc1137fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1140aade0 0x7fc1140645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc1137fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1140aade0 0x7fc1140645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37054/0 (socket says 172.21.15.33:37054) 2023-11-06T21:02:12.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc1137fe700 1 -- 172.21.15.33:0/2623902615 learned_addr learned my addr 172.21.15.33:0/2623902615 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:12.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc118b9c700 1 --1- 172.21.15.33:0/2623902615 >> v1:172.21.15.33:6789/0 conn(0x7fc114064ac0 0x7fc1140c5db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42138/0 (socket says 172.21.15.33:42138) 2023-11-06T21:02:12.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc113fff700 1 --2- 172.21.15.33:0/2623902615 >> v2:172.21.15.69:3300/0 conn(0x7fc114154110 0x7fc1141544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc1137fe700 1 -- 172.21.15.33:0/2623902615 >> v2:172.21.15.69:3300/0 conn(0x7fc114154110 msgr2=0x7fc1141544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:12.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc1137fe700 1 --2- 172.21.15.33:0/2623902615 >> v2:172.21.15.69:3300/0 conn(0x7fc114154110 0x7fc1141544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc1137fe700 1 -- 172.21.15.33:0/2623902615 >> v1:172.21.15.33:6789/0 conn(0x7fc114064ac0 legacy=0x7fc1140c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc1137fe700 1 -- 172.21.15.33:0/2623902615 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1140c8260 con 0x7fc1140aade0 2023-11-06T21:02:12.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc112ffd700 1 -- 172.21.15.33:0/2623902615 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258533335 0 0) 0x7fc10c001490 con 0x7fc114064ac0 2023-11-06T21:02:12.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc1137fe700 1 --2- 172.21.15.33:0/2623902615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1140aade0 0x7fc1140645d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fc11406c130 tx=0x7fc1080090f0).ready entity=mon.0 client_cookie=156fdde670e36fc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:12.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.235+0000 7fc112ffd700 1 -- 172.21.15.33:0/2623902615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800b520 con 0x7fc1140aade0 2023-11-06T21:02:12.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.236+0000 7fc112ffd700 1 -- 172.21.15.33:0/2623902615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10800b680 con 0x7fc1140aade0 2023-11-06T21:02:12.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.236+0000 7fc112ffd700 1 -- 172.21.15.33:0/2623902615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800ba00 con 0x7fc1140aade0 2023-11-06T21:02:12.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.236+0000 7fc11a5ff700 1 -- 172.21.15.33:0/2623902615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1140aade0 msgr2=0x7fc1140645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.236+0000 7fc11a5ff700 1 --2- 172.21.15.33:0/2623902615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1140aade0 0x7fc1140645d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fc11406c130 tx=0x7fc1080090f0).stop 2023-11-06T21:02:12.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.236+0000 7fc11a5ff700 1 -- 172.21.15.33:0/2623902615 shutdown_connections 2023-11-06T21:02:12.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.236+0000 7fc11a5ff700 1 --2- 172.21.15.33:0/2623902615 >> v2:172.21.15.69:3300/0 conn(0x7fc114154110 0x7fc1141544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.236+0000 7fc11a5ff700 1 --2- 172.21.15.33:0/2623902615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1140aade0 0x7fc1140645d0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.236+0000 7fc11a5ff700 1 -- 172.21.15.33:0/2623902615 >> 172.21.15.33:0/2623902615 conn(0x7fc1140cb8b0 msgr2=0x7fc1140c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:12.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.236+0000 7fc11a5ff700 1 -- 172.21.15.33:0/2623902615 shutdown_connections 2023-11-06T21:02:12.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.236+0000 7fc11a5ff700 1 -- 172.21.15.33:0/2623902615 wait complete. 2023-11-06T21:02:12.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.238+0000 7fc11a5ff700 1 Processor -- start 2023-11-06T21:02:12.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.239+0000 7fc11a5ff700 1 -- start start 2023-11-06T21:02:12.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc11a5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114064ac0 0x7fc114159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc11a5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1140aade0 0x7fc1140c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc11a5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc114154110 0x7fc114165c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc11a5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc114164a00 con 0x7fc114064ac0 2023-11-06T21:02:12.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc11a5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc114164cb0 con 0x7fc1140aade0 2023-11-06T21:02:12.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc11a5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1141572f0 con 0x7fc114154110 2023-11-06T21:02:12.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc113fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114064ac0 0x7fc114159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc113fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114064ac0 0x7fc114159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37058/0 (socket says 172.21.15.33:37058) 2023-11-06T21:02:12.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc113fff700 1 -- 172.21.15.33:0/1654899548 learned_addr learned my addr 172.21.15.33:0/1654899548 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:12.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc1137fe700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1140aade0 0x7fc1140c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc118b9c700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc114154110 0x7fc114165c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc113fff700 1 -- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc114154110 msgr2=0x7fc114165c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.240+0000 7fc113fff700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc114154110 0x7fc114165c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.241+0000 7fc113fff700 1 -- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1140aade0 msgr2=0x7fc1140c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.241+0000 7fc113fff700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1140aade0 0x7fc1140c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.241+0000 7fc113fff700 1 -- 172.21.15.33:0/1654899548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc108009370 con 0x7fc114064ac0 2023-11-06T21:02:12.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.241+0000 7fc113fff700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114064ac0 0x7fc114159660 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7fc10400e930 tx=0x7fc104009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:12.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.241+0000 7fc0fbfff700 1 -- 172.21.15.33:0/1654899548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104012600 con 0x7fc114064ac0 2023-11-06T21:02:12.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.241+0000 7fc0fbfff700 1 -- 172.21.15.33:0/1654899548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc104012760 con 0x7fc114064ac0 2023-11-06T21:02:12.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.241+0000 7fc0fbfff700 1 -- 172.21.15.33:0/1654899548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104012ae0 con 0x7fc114064ac0 2023-11-06T21:02:12.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.241+0000 7fc11a5ff700 1 -- 172.21.15.33:0/1654899548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1141575d0 con 0x7fc114064ac0 2023-11-06T21:02:12.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.241+0000 7fc11a5ff700 1 -- 172.21.15.33:0/1654899548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc114159b50 con 0x7fc114064ac0 2023-11-06T21:02:12.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.242+0000 7fc11a5ff700 1 -- 172.21.15.33:0/1654899548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc114157af0 con 0x7fc114064ac0 2023-11-06T21:02:12.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.243+0000 7fc0fbfff700 1 -- 172.21.15.33:0/1654899548 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc10401b400 con 0x7fc114064ac0 2023-11-06T21:02:12.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.244+0000 7fc0fbfff700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc10005f270 0x7fc100061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.244+0000 7fc0fbfff700 1 -- 172.21.15.33:0/1654899548 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc10407a6b0 con 0x7fc114064ac0 2023-11-06T21:02:12.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.244+0000 7fc1137fe700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc10005f270 0x7fc100061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.245+0000 7fc1137fe700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc10005f270 0x7fc100061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc108009130 tx=0x7fc108017000).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:12.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.248+0000 7fc0fbfff700 1 -- 172.21.15.33:0/1654899548 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc114157af0 con 0x7fc114064ac0 2023-11-06T21:02:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.429+0000 7fc11a5ff700 1 -- 172.21.15.33:0/1654899548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc114157af0 con 0x7fc114064ac0 2023-11-06T21:02:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.431+0000 7fc0fbfff700 1 -- 172.21.15.33:0/1654899548 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+17004 (secure 0 0 0) 0x7fc114157af0 con 0x7fc114064ac0 2023-11-06T21:02:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:12.432 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":130,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:02:11.990898+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:02:09.533597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T21:02:10.978394+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:02:12.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 -- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc10005f270 msgr2=0x7fc100061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc10005f270 0x7fc100061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc108009130 tx=0x7fc108017000).stop 2023-11-06T21:02:12.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 -- 172.21.15.33:0/1654899548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114064ac0 msgr2=0x7fc114159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114064ac0 0x7fc114159660 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7fc10400e930 tx=0x7fc104009060).stop 2023-11-06T21:02:12.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 -- 172.21.15.33:0/1654899548 shutdown_connections 2023-11-06T21:02:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc10005f270 0x7fc100061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc114154110 0x7fc114165c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114064ac0 0x7fc114159660 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 --2- 172.21.15.33:0/1654899548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1140aade0 0x7fc1140c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 -- 172.21.15.33:0/1654899548 >> 172.21.15.33:0/1654899548 conn(0x7fc1140cb8b0 msgr2=0x7fc114153170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 -- 172.21.15.33:0/1654899548 shutdown_connections 2023-11-06T21:02:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.433+0000 7fc11a5ff700 1 -- 172.21.15.33:0/1654899548 wait complete. 2023-11-06T21:02:12.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:02:12.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.647+0000 7ff409bff700 1 Processor -- start 2023-11-06T21:02:12.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.647+0000 7ff409bff700 1 -- start start 2023-11-06T21:02:12.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.648+0000 7ff409bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4041521a0 0x7ff4040c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.648+0000 7ff409bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff404155320 0x7ff4040c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.648+0000 7ff409bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff404154e90 con 0x7ff404151dc0 2023-11-06T21:02:12.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.648+0000 7ff409bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4040639d0 con 0x7ff4041521a0 2023-11-06T21:02:12.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.648+0000 7ff409bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff404063cb0 con 0x7ff404155320 2023-11-06T21:02:12.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff4037fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff404151dc0 0x7ff404063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42150/0 (socket says 172.21.15.33:42150) 2023-11-06T21:02:12.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff4037fe700 1 -- 172.21.15.33:0/227321341 learned_addr learned my addr 172.21.15.33:0/227321341 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:12.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff402ffd700 1 --2- 172.21.15.33:0/227321341 >> v2:172.21.15.33:3300/0 conn(0x7ff4041521a0 0x7ff4040c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff4027fc700 1 -- 172.21.15.33:0/227321341 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100285292 0 0) 0x7ff404154e90 con 0x7ff404151dc0 2023-11-06T21:02:12.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff403fff700 1 --2- 172.21.15.33:0/227321341 >> v2:172.21.15.69:3300/0 conn(0x7ff404155320 0x7ff4040c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff4027fc700 1 -- 172.21.15.33:0/227321341 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3ec0033f0 con 0x7ff404151dc0 2023-11-06T21:02:12.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff402ffd700 1 -- 172.21.15.33:0/227321341 >> v2:172.21.15.69:3300/0 conn(0x7ff404155320 msgr2=0x7ff4040c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:12.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff402ffd700 1 --2- 172.21.15.33:0/227321341 >> v2:172.21.15.69:3300/0 conn(0x7ff404155320 0x7ff4040c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff402ffd700 1 -- 172.21.15.33:0/227321341 >> v1:172.21.15.33:6789/0 conn(0x7ff404151dc0 legacy=0x7ff404063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff402ffd700 1 -- 172.21.15.33:0/227321341 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff404063f90 con 0x7ff4041521a0 2023-11-06T21:02:12.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.649+0000 7ff403fff700 1 --2- 172.21.15.33:0/227321341 >> v2:172.21.15.69:3300/0 conn(0x7ff404155320 0x7ff4040c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:12.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff402ffd700 1 --2- 172.21.15.33:0/227321341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 0x7ff4040c0c50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7ff404154a10 tx=0x7ff3f8009060).ready entity=mon.0 client_cookie=4e36193f4a9021ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:12.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff4027fc700 1 -- 172.21.15.33:0/227321341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800f510 con 0x7ff4041521a0 2023-11-06T21:02:12.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff4027fc700 1 -- 172.21.15.33:0/227321341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f800f670 con 0x7ff4041521a0 2023-11-06T21:02:12.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff4027fc700 1 -- 172.21.15.33:0/227321341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800f9f0 con 0x7ff4041521a0 2023-11-06T21:02:12.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff409bff700 1 -- 172.21.15.33:0/227321341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 msgr2=0x7ff4040c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff409bff700 1 --2- 172.21.15.33:0/227321341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 0x7ff4040c0c50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7ff404154a10 tx=0x7ff3f8009060).stop 2023-11-06T21:02:12.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff409bff700 1 -- 172.21.15.33:0/227321341 shutdown_connections 2023-11-06T21:02:12.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff409bff700 1 --2- 172.21.15.33:0/227321341 >> v2:172.21.15.69:3300/0 conn(0x7ff404155320 0x7ff4040c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff409bff700 1 --2- 172.21.15.33:0/227321341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 0x7ff4040c0c50 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff409bff700 1 -- 172.21.15.33:0/227321341 >> 172.21.15.33:0/227321341 conn(0x7ff40405e1b0 msgr2=0x7ff40405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:12.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff409bff700 1 -- 172.21.15.33:0/227321341 shutdown_connections 2023-11-06T21:02:12.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.650+0000 7ff409bff700 1 -- 172.21.15.33:0/227321341 wait complete. 2023-11-06T21:02:12.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.652+0000 7ff409bff700 1 Processor -- start 2023-11-06T21:02:12.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.652+0000 7ff409bff700 1 -- start start 2023-11-06T21:02:12.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff409bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff404151dc0 0x7ff4040c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff409bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 0x7ff4041507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff409bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff404155320 0x7ff4040c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff409bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404168060 con 0x7ff4041521a0 2023-11-06T21:02:12.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff409bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404168340 con 0x7ff404155320 2023-11-06T21:02:12.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff409bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff404168620 con 0x7ff404151dc0 2023-11-06T21:02:12.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff402ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 0x7ff4041507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff402ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 0x7ff4041507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37078/0 (socket says 172.21.15.33:37078) 2023-11-06T21:02:12.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff402ffd700 1 -- 172.21.15.33:0/1137387896 learned_addr learned my addr 172.21.15.33:0/1137387896 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:12.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff4037fe700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff404151dc0 0x7ff4040c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.653+0000 7ff403fff700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff404155320 0x7ff4040c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff402ffd700 1 -- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff404151dc0 msgr2=0x7ff4040c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff402ffd700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff404151dc0 0x7ff4040c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff402ffd700 1 -- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff404155320 msgr2=0x7ff4040c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff402ffd700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff404155320 0x7ff4040c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff402ffd700 1 -- 172.21.15.33:0/1137387896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f80092a0 con 0x7ff4041521a0 2023-11-06T21:02:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff402ffd700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 0x7ff4041507f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7ff3f8009dd0 tx=0x7ff3f8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:12.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff3eb7fe700 1 -- 172.21.15.33:0/1137387896 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800f9f0 con 0x7ff4041521a0 2023-11-06T21:02:12.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff3eb7fe700 1 -- 172.21.15.33:0/1137387896 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f8000c90 con 0x7ff4041521a0 2023-11-06T21:02:12.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff3eb7fe700 1 -- 172.21.15.33:0/1137387896 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f801b660 con 0x7ff4041521a0 2023-11-06T21:02:12.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff409bff700 1 -- 172.21.15.33:0/1137387896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff404168900 con 0x7ff4041521a0 2023-11-06T21:02:12.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.654+0000 7ff409bff700 1 -- 172.21.15.33:0/1137387896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff404150db0 con 0x7ff4041521a0 2023-11-06T21:02:12.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.655+0000 7ff409bff700 1 -- 172.21.15.33:0/1137387896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4040c0a30 con 0x7ff4041521a0 2023-11-06T21:02:12.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.656+0000 7ff3eb7fe700 1 -- 172.21.15.33:0/1137387896 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff3f801b9d0 con 0x7ff4041521a0 2023-11-06T21:02:12.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.657+0000 7ff3eb7fe700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3f005f270 0x7ff3f0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:12.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.657+0000 7ff3eb7fe700 1 -- 172.21.15.33:0/1137387896 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff3f807bf50 con 0x7ff4041521a0 2023-11-06T21:02:12.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.657+0000 7ff4037fe700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3f005f270 0x7ff3f0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:12.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.658+0000 7ff4037fe700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3f005f270 0x7ff3f0061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff3f4005750 tx=0x7ff3f4009040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:12.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.661+0000 7ff3eb7fe700 1 -- 172.21.15.33:0/1137387896 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff3f804cfe0 con 0x7ff4041521a0 2023-11-06T21:02:12.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.842+0000 7ff409bff700 1 -- 172.21.15.33:0/1137387896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff404062ac0 con 0x7ff4041521a0 2023-11-06T21:02:12.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.843+0000 7ff3eb7fe700 1 -- 172.21.15.33:0/1137387896 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+17004 (secure 0 0 0) 0x7ff3f8028070 con 0x7ff4041521a0 2023-11-06T21:02:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:12.844 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":130,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:02:11.990898+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:02:09.533597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T21:02:10.978394+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:02:12.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 -- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3f005f270 msgr2=0x7ff3f0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3f005f270 0x7ff3f0061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff3f4005750 tx=0x7ff3f4009040).stop 2023-11-06T21:02:12.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 -- 172.21.15.33:0/1137387896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 msgr2=0x7ff4041507f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 0x7ff4041507f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7ff3f8009dd0 tx=0x7ff3f8017000).stop 2023-11-06T21:02:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 -- 172.21.15.33:0/1137387896 shutdown_connections 2023-11-06T21:02:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff3f005f270 0x7ff3f0061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff404151dc0 0x7ff4040c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff404155320 0x7ff4040c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 --2- 172.21.15.33:0/1137387896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4041521a0 0x7ff4041507f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:12.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 -- 172.21.15.33:0/1137387896 >> 172.21.15.33:0/1137387896 conn(0x7ff40405e1b0 msgr2=0x7ff404152640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:12.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 -- 172.21.15.33:0/1137387896 shutdown_connections 2023-11-06T21:02:12.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:12.845+0000 7ff409bff700 1 -- 172.21.15.33:0/1137387896 wait complete. 2023-11-06T21:02:12.864 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T21:02:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.065+0000 7f53e69ff700 1 Processor -- start 2023-11-06T21:02:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.065+0000 7f53e69ff700 1 -- start start 2023-11-06T21:02:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.067+0000 7f53e69ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53e0154010 0x7f53e01543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.067+0000 7f53e69ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f53e0064670 0x7f53e0064b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.067+0000 7f53e69ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e0065010 con 0x7f53e00aafc0 2023-11-06T21:02:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.067+0000 7f53e69ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00c6280 con 0x7f53e0154010 2023-11-06T21:02:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.067+0000 7f53e69ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00c0d90 con 0x7f53e0064670 2023-11-06T21:02:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.067+0000 7f53dffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53e0154010 0x7f53e01543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.067+0000 7f53dffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53e0154010 0x7f53e01543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37082/0 (socket says 172.21.15.33:37082) 2023-11-06T21:02:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.067+0000 7f53df7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f53e00aafc0 0x7f53e00c0680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42154/0 (socket says 172.21.15.33:42154) 2023-11-06T21:02:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.067+0000 7f53dffff700 1 -- 172.21.15.33:0/3418074631 learned_addr learned my addr 172.21.15.33:0/3418074631 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.068+0000 7f53e4f9c700 1 --2- 172.21.15.33:0/3418074631 >> v2:172.21.15.69:3300/0 conn(0x7f53e0064670 0x7f53e0064b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.068+0000 7f53dffff700 1 -- 172.21.15.33:0/3418074631 >> v2:172.21.15.69:3300/0 conn(0x7f53e0064670 msgr2=0x7f53e0064b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.068+0000 7f53dffff700 1 --2- 172.21.15.33:0/3418074631 >> v2:172.21.15.69:3300/0 conn(0x7f53e0064670 0x7f53e0064b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.068+0000 7f53dffff700 1 -- 172.21.15.33:0/3418074631 >> v1:172.21.15.33:6789/0 conn(0x7f53e00aafc0 legacy=0x7f53e00c0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.068+0000 7f53dffff700 1 -- 172.21.15.33:0/3418074631 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00c7720 con 0x7f53e0154010 2023-11-06T21:02:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.068+0000 7f53deffd700 1 -- 172.21.15.33:0/3418074631 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798060968 0 0) 0x7f53d4001110 con 0x7f53e00aafc0 2023-11-06T21:02:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.068+0000 7f53dffff700 1 --2- 172.21.15.33:0/3418074631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0154010 0x7f53e01543f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f53e00aa900 tx=0x7f53d00090f0).ready entity=mon.0 client_cookie=2e4a985a8d870c71 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.068+0000 7f53deffd700 1 -- 172.21.15.33:0/3418074631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000b600 con 0x7f53e0154010 2023-11-06T21:02:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.068+0000 7f53deffd700 1 -- 172.21.15.33:0/3418074631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d000b760 con 0x7f53e0154010 2023-11-06T21:02:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.068+0000 7f53deffd700 1 -- 172.21.15.33:0/3418074631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000bae0 con 0x7f53e0154010 2023-11-06T21:02:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.069+0000 7f53e69ff700 1 -- 172.21.15.33:0/3418074631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0154010 msgr2=0x7f53e01543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.069+0000 7f53e69ff700 1 --2- 172.21.15.33:0/3418074631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0154010 0x7f53e01543f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f53e00aa900 tx=0x7f53d00090f0).stop 2023-11-06T21:02:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.069+0000 7f53e69ff700 1 -- 172.21.15.33:0/3418074631 shutdown_connections 2023-11-06T21:02:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.069+0000 7f53e69ff700 1 --2- 172.21.15.33:0/3418074631 >> v2:172.21.15.69:3300/0 conn(0x7f53e0064670 0x7f53e0064b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.069+0000 7f53e69ff700 1 --2- 172.21.15.33:0/3418074631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0154010 0x7f53e01543f0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.069+0000 7f53e69ff700 1 -- 172.21.15.33:0/3418074631 >> 172.21.15.33:0/3418074631 conn(0x7f53e00cb8e0 msgr2=0x7f53e00c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.069+0000 7f53e69ff700 1 -- 172.21.15.33:0/3418074631 shutdown_connections 2023-11-06T21:02:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.069+0000 7f53e69ff700 1 -- 172.21.15.33:0/3418074631 wait complete. 2023-11-06T21:02:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.071+0000 7f53e69ff700 1 Processor -- start 2023-11-06T21:02:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.071+0000 7f53e69ff700 1 -- start start 2023-11-06T21:02:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53e69ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0064670 0x7f53e0154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53e69ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53e0154010 0x7f53e014cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53e69ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53e0158450 0x7f53e0164470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53e69ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0155d20 con 0x7f53e0064670 2023-11-06T21:02:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53e69ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0155110 con 0x7f53e0158450 2023-11-06T21:02:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53e69ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01553f0 con 0x7f53e0154010 2023-11-06T21:02:13.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53dffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0064670 0x7f53e0154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:13.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53dffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0064670 0x7f53e0154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37092/0 (socket says 172.21.15.33:37092) 2023-11-06T21:02:13.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53dffff700 1 -- 172.21.15.33:0/1763090565 learned_addr learned my addr 172.21.15.33:0/1763090565 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:13.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53df7fe700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53e0154010 0x7f53e014cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:13.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53e4f9c700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53e0158450 0x7f53e0164470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53dffff700 1 -- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53e0154010 msgr2=0x7f53e014cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53dffff700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53e0154010 0x7f53e014cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53dffff700 1 -- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53e0158450 msgr2=0x7f53e0164470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53dffff700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53e0158450 0x7f53e0164470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53dffff700 1 -- 172.21.15.33:0/1763090565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0009370 con 0x7f53e0064670 2023-11-06T21:02:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.072+0000 7f53e4f9c700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53e0158450 0x7f53e0164470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:02:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.073+0000 7f53dffff700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0064670 0x7f53e0154900 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f53d0009130 tx=0x7f53d0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.073+0000 7f53c7fff700 1 -- 172.21.15.33:0/1763090565 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000b8d0 con 0x7f53e0064670 2023-11-06T21:02:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.073+0000 7f53c7fff700 1 -- 172.21.15.33:0/1763090565 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d000ba30 con 0x7f53e0064670 2023-11-06T21:02:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.073+0000 7f53c7fff700 1 -- 172.21.15.33:0/1763090565 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d001b660 con 0x7f53e0064670 2023-11-06T21:02:13.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.073+0000 7f53e69ff700 1 -- 172.21.15.33:0/1763090565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e01556d0 con 0x7f53e0064670 2023-11-06T21:02:13.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.073+0000 7f53e69ff700 1 -- 172.21.15.33:0/1763090565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e0154df0 con 0x7f53e0064670 2023-11-06T21:02:13.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.074+0000 7f53e69ff700 1 -- 172.21.15.33:0/1763090565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e0065870 con 0x7f53e0064670 2023-11-06T21:02:13.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.075+0000 7f53c7fff700 1 -- 172.21.15.33:0/1763090565 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f53d001ba10 con 0x7f53e0064670 2023-11-06T21:02:13.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.076+0000 7f53c7fff700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53cc05f270 0x7f53cc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:13.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.076+0000 7f53c7fff700 1 -- 172.21.15.33:0/1763090565 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f53d007be40 con 0x7f53e0064670 2023-11-06T21:02:13.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.076+0000 7f53df7fe700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53cc05f270 0x7f53cc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:13.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.077+0000 7f53df7fe700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53cc05f270 0x7f53cc061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f53e00c4bd0 tx=0x7f53d4007a10).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:13.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.080+0000 7f53c7fff700 1 -- 172.21.15.33:0/1763090565 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f53d004ced0 con 0x7f53e0064670 2023-11-06T21:02:13.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:13.303+0000 7f53e69ff700 1 -- 172.21.15.33:0/1763090565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f53e014f580 con 0x7f53e0064670 2023-11-06T21:02:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.032+0000 7f53c7fff700 1 -- 172.21.15.33:0/1763090565 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f53d0029070 con 0x7f53e0064670 2023-11-06T21:02:14.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 -- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53cc05f270 msgr2=0x7f53cc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:14.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53cc05f270 0x7f53cc061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f53e00c4bd0 tx=0x7f53d4007a10).stop 2023-11-06T21:02:14.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 -- 172.21.15.33:0/1763090565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0064670 msgr2=0x7f53e0154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:14.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0064670 0x7f53e0154900 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f53d0009130 tx=0x7f53d0017000).stop 2023-11-06T21:02:14.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 -- 172.21.15.33:0/1763090565 shutdown_connections 2023-11-06T21:02:14.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f53cc05f270 0x7f53cc061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:14.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53e0158450 0x7f53e0164470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:14.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e0064670 0x7f53e0154900 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:14.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 --2- 172.21.15.33:0/1763090565 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f53e0154010 0x7f53e014cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:14.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 -- 172.21.15.33:0/1763090565 >> 172.21.15.33:0/1763090565 conn(0x7f53e00cb8e0 msgr2=0x7f53e0063000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:14.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 -- 172.21.15.33:0/1763090565 shutdown_connections 2023-11-06T21:02:14.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.035+0000 7f53e69ff700 1 -- 172.21.15.33:0/1763090565 wait complete. 2023-11-06T21:02:14.055 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T21:02:14.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T21:02:14.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.257+0000 7fbee3bff700 1 Processor -- start 2023-11-06T21:02:14.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.257+0000 7fbee3bff700 1 -- start start 2023-11-06T21:02:14.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.258+0000 7fbee3bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbedc0632f0 0x7fbedc0636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:14.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.258+0000 7fbee3bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbedc14e2d0 0x7fbedc14e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:14.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.258+0000 7fbee3bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc1584f0 con 0x7fbedc158980 2023-11-06T21:02:14.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.258+0000 7fbee3bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc1559e0 con 0x7fbedc14e2d0 2023-11-06T21:02:14.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.258+0000 7fbee3bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc155420 con 0x7fbedc0632f0 2023-11-06T21:02:14.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee219c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbedc14e2d0 0x7fbedc14e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:14.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee219c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbedc14e2d0 0x7fbedc14e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37096/0 (socket says 172.21.15.33:37096) 2023-11-06T21:02:14.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee219c700 1 -- 172.21.15.33:0/2395854676 learned_addr learned my addr 172.21.15.33:0/2395854676 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:14.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee199b700 1 --2- 172.21.15.33:0/2395854676 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbedc0632f0 0x7fbedc0636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:14.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee219c700 1 -- 172.21.15.33:0/2395854676 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbedc0632f0 msgr2=0x7fbedc0636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:14.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee219c700 1 --2- 172.21.15.33:0/2395854676 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbedc0632f0 0x7fbedc0636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:14.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee219c700 1 -- 172.21.15.33:0/2395854676 >> v1:172.21.15.69:6789/0 conn(0x7fbedc158980 legacy=0x7fbedc1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:14.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee219c700 1 -- 172.21.15.33:0/2395854676 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc155700 con 0x7fbedc14e2d0 2023-11-06T21:02:14.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee219c700 1 --2- 172.21.15.33:0/2395854676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc14e2d0 0x7fbedc14e750 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7fbed4009f20 tx=0x7fbed4009840).ready entity=mon.0 client_cookie=a985e44804878fe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:14.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee0999700 1 -- 172.21.15.33:0/2395854676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed400e070 con 0x7fbedc14e2d0 2023-11-06T21:02:14.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee0999700 1 -- 172.21.15.33:0/2395854676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed400ba80 con 0x7fbedc14e2d0 2023-11-06T21:02:14.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.259+0000 7fbee0999700 1 -- 172.21.15.33:0/2395854676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed400be00 con 0x7fbedc14e2d0 2023-11-06T21:02:14.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.260+0000 7fbee3bff700 1 -- 172.21.15.33:0/2395854676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc14e2d0 msgr2=0x7fbedc14e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:14.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.260+0000 7fbee3bff700 1 --2- 172.21.15.33:0/2395854676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc14e2d0 0x7fbedc14e750 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7fbed4009f20 tx=0x7fbed4009840).stop 2023-11-06T21:02:14.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.260+0000 7fbee3bff700 1 -- 172.21.15.33:0/2395854676 shutdown_connections 2023-11-06T21:02:14.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.260+0000 7fbee3bff700 1 --2- 172.21.15.33:0/2395854676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc14e2d0 0x7fbedc14e750 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:14.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.260+0000 7fbee3bff700 1 --2- 172.21.15.33:0/2395854676 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbedc0632f0 0x7fbedc0636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:14.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.260+0000 7fbee3bff700 1 -- 172.21.15.33:0/2395854676 >> 172.21.15.33:0/2395854676 conn(0x7fbedc0cb910 msgr2=0x7fbedc0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:14.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.260+0000 7fbee3bff700 1 -- 172.21.15.33:0/2395854676 shutdown_connections 2023-11-06T21:02:14.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.260+0000 7fbee3bff700 1 -- 172.21.15.33:0/2395854676 wait complete. 2023-11-06T21:02:14.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.262+0000 7fbee3bff700 1 Processor -- start 2023-11-06T21:02:14.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.262+0000 7fbee3bff700 1 -- start start 2023-11-06T21:02:14.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee3bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbedc0632f0 0x7fbedc14db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:14.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee3bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbedc14e2d0 0x7fbedc164650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:14.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee3bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc158980 0x7fbedc164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:14.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee3bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0af260 con 0x7fbedc158980 2023-11-06T21:02:14.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee3bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0ae650 con 0x7fbedc0632f0 2023-11-06T21:02:14.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee3bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0ae930 con 0x7fbedc14e2d0 2023-11-06T21:02:14.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee219c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc158980 0x7fbedc164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:14.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee219c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc158980 0x7fbedc164b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37100/0 (socket says 172.21.15.33:37100) 2023-11-06T21:02:14.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee219c700 1 -- 172.21.15.33:0/1842165179 learned_addr learned my addr 172.21.15.33:0/1842165179 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:14.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee199b700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbedc0632f0 0x7fbedc14db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:14.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.263+0000 7fbee119a700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbedc14e2d0 0x7fbedc164650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:14.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbee219c700 1 -- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbedc14e2d0 msgr2=0x7fbedc164650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:14.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbee219c700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbedc14e2d0 0x7fbedc164650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:14.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbee219c700 1 -- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbedc0632f0 msgr2=0x7fbedc14db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:14.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbee219c700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbedc0632f0 0x7fbedc14db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:14.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbee219c700 1 -- 172.21.15.33:0/1842165179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed4009ae0 con 0x7fbedc158980 2023-11-06T21:02:14.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbee219c700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc158980 0x7fbedc164b40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7fbedc083800 tx=0x7fbedc0c15e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:14.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbed1ffb700 1 -- 172.21.15.33:0/1842165179 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4013ea0 con 0x7fbedc158980 2023-11-06T21:02:14.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbed1ffb700 1 -- 172.21.15.33:0/1842165179 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed40092e0 con 0x7fbedc158980 2023-11-06T21:02:14.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbee3bff700 1 -- 172.21.15.33:0/1842165179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc0aec10 con 0x7fbedc158980 2023-11-06T21:02:14.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbed1ffb700 1 -- 172.21.15.33:0/1842165179 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed401e6c0 con 0x7fbedc158980 2023-11-06T21:02:14.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.264+0000 7fbee3bff700 1 -- 172.21.15.33:0/1842165179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc0af6f0 con 0x7fbedc158980 2023-11-06T21:02:14.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.265+0000 7fbee3bff700 1 -- 172.21.15.33:0/1842165179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc158180 con 0x7fbedc158980 2023-11-06T21:02:14.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.266+0000 7fbed1ffb700 1 -- 172.21.15.33:0/1842165179 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbed4003040 con 0x7fbedc158980 2023-11-06T21:02:14.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.267+0000 7fbed1ffb700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbec005f270 0x7fbec0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:14.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.267+0000 7fbed1ffb700 1 -- 172.21.15.33:0/1842165179 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbed407b9b0 con 0x7fbedc158980 2023-11-06T21:02:14.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.268+0000 7fbee199b700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbec005f270 0x7fbec0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:14.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.268+0000 7fbee199b700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbec005f270 0x7fbec0061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fbed800bbc0 tx=0x7fbed8008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:14.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.272+0000 7fbed1ffb700 1 -- 172.21.15.33:0/1842165179 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbed404caf0 con 0x7fbedc158980 2023-11-06T21:02:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:14.501+0000 7fbee3bff700 1 -- 172.21.15.33:0/1842165179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbedc06ab40 con 0x7fbedc158980 2023-11-06T21:02:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.034+0000 7fbed1ffb700 1 -- 172.21.15.33:0/1842165179 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbed4023030 con 0x7fbedc158980 2023-11-06T21:02:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 -- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbec005f270 msgr2=0x7fbec0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbec005f270 0x7fbec0061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fbed800bbc0 tx=0x7fbed8008040).stop 2023-11-06T21:02:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 -- 172.21.15.33:0/1842165179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc158980 msgr2=0x7fbedc164b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc158980 0x7fbedc164b40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7fbedc083800 tx=0x7fbedc0c15e0).stop 2023-11-06T21:02:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 -- 172.21.15.33:0/1842165179 shutdown_connections 2023-11-06T21:02:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbec005f270 0x7fbec0061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbedc158980 0x7fbedc164b40 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbedc0632f0 0x7fbedc14db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 --2- 172.21.15.33:0/1842165179 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbedc14e2d0 0x7fbedc164650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 -- 172.21.15.33:0/1842165179 >> 172.21.15.33:0/1842165179 conn(0x7fbedc0cb910 msgr2=0x7fbedc0c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:15.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 -- 172.21.15.33:0/1842165179 shutdown_connections 2023-11-06T21:02:15.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.037+0000 7fbee3bff700 1 -- 172.21.15.33:0/1842165179 wait complete. 2023-11-06T21:02:15.058 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T21:02:15.059 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:02:15.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.257+0000 7f11737ff700 1 Processor -- start 2023-11-06T21:02:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.257+0000 7f11737ff700 1 -- start start 2023-11-06T21:02:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.258+0000 7f11737ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f116c0c1840 0x7f116c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.258+0000 7f11737ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f116c0aa2d0 0x7f116c0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.258+0000 7f11737ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116c1554a0 con 0x7f116c0c6240 2023-11-06T21:02:15.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.258+0000 7f11737ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c154960 con 0x7f116c0c1840 2023-11-06T21:02:15.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.258+0000 7f11737ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c155930 con 0x7f116c0aa2d0 2023-11-06T21:02:15.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.258+0000 7f117159b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f116c0c1840 0x7f116c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.259+0000 7f117159b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f116c0c1840 0x7f116c0a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37106/0 (socket says 172.21.15.33:37106) 2023-11-06T21:02:15.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.259+0000 7f117159b700 1 -- 172.21.15.33:0/1591694158 learned_addr learned my addr 172.21.15.33:0/1591694158 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:15.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.259+0000 7f1171d9c700 1 --1- 172.21.15.33:0/1591694158 >> v1:172.21.15.33:6789/0 conn(0x7f116c0c6240 0x7f116c157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42162/0 (socket says 172.21.15.33:42162) 2023-11-06T21:02:15.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.259+0000 7f1170d9a700 1 --2- 172.21.15.33:0/1591694158 >> v2:172.21.15.69:3300/0 conn(0x7f116c0aa2d0 0x7f116c0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.259+0000 7f117159b700 1 -- 172.21.15.33:0/1591694158 >> v2:172.21.15.69:3300/0 conn(0x7f116c0aa2d0 msgr2=0x7f116c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:15.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.259+0000 7f117159b700 1 --2- 172.21.15.33:0/1591694158 >> v2:172.21.15.69:3300/0 conn(0x7f116c0aa2d0 0x7f116c0c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.259+0000 7f117159b700 1 -- 172.21.15.33:0/1591694158 >> v1:172.21.15.33:6789/0 conn(0x7f116c0c6240 legacy=0x7f116c157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.259+0000 7f117159b700 1 -- 172.21.15.33:0/1591694158 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c154b70 con 0x7f116c0c1840 2023-11-06T21:02:15.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.259+0000 7f117159b700 1 --2- 172.21.15.33:0/1591694158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0c1840 0x7f116c0a9de0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f116c0adb40 tx=0x7f11680090f0).ready entity=mon.0 client_cookie=357fe629e9c0c051 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:15.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.259+0000 7f1163fff700 1 -- 172.21.15.33:0/1591694158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800b520 con 0x7f116c0c1840 2023-11-06T21:02:15.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.260+0000 7f1163fff700 1 -- 172.21.15.33:0/1591694158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116800b680 con 0x7f116c0c1840 2023-11-06T21:02:15.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.260+0000 7f1163fff700 1 -- 172.21.15.33:0/1591694158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800ba00 con 0x7f116c0c1840 2023-11-06T21:02:15.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.260+0000 7f11737ff700 1 -- 172.21.15.33:0/1591694158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0c1840 msgr2=0x7f116c0a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.260+0000 7f11737ff700 1 --2- 172.21.15.33:0/1591694158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0c1840 0x7f116c0a9de0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f116c0adb40 tx=0x7f11680090f0).stop 2023-11-06T21:02:15.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.260+0000 7f11737ff700 1 -- 172.21.15.33:0/1591694158 shutdown_connections 2023-11-06T21:02:15.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.260+0000 7f11737ff700 1 --2- 172.21.15.33:0/1591694158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0c1840 0x7f116c0a9de0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.260+0000 7f11737ff700 1 --2- 172.21.15.33:0/1591694158 >> v2:172.21.15.69:3300/0 conn(0x7f116c0aa2d0 0x7f116c0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.260+0000 7f11737ff700 1 -- 172.21.15.33:0/1591694158 >> 172.21.15.33:0/1591694158 conn(0x7f116c05ec80 msgr2=0x7f116c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.260+0000 7f11737ff700 1 -- 172.21.15.33:0/1591694158 shutdown_connections 2023-11-06T21:02:15.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.260+0000 7f11737ff700 1 -- 172.21.15.33:0/1591694158 wait complete. 2023-11-06T21:02:15.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.262+0000 7f11737ff700 1 Processor -- start 2023-11-06T21:02:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.262+0000 7f11737ff700 1 -- start start 2023-11-06T21:02:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f11737ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0aa2d0 0x7f116c157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f11737ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f116c0c1840 0x7f116c1689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f11737ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f116c0c6240 0x7f116c14eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f11737ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c168eb0 con 0x7f116c0aa2d0 2023-11-06T21:02:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f11737ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c1652f0 con 0x7f116c0c6240 2023-11-06T21:02:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f11737ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c14f040 con 0x7f116c0c1840 2023-11-06T21:02:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f117159b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0aa2d0 0x7f116c157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f117159b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0aa2d0 0x7f116c157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37110/0 (socket says 172.21.15.33:37110) 2023-11-06T21:02:15.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f117159b700 1 -- 172.21.15.33:0/1360169180 learned_addr learned my addr 172.21.15.33:0/1360169180 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:15.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f1171d9c700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f116c0c6240 0x7f116c14eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f1170d9a700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f116c0c1840 0x7f116c1689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f117159b700 1 -- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f116c0c1840 msgr2=0x7f116c1689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f117159b700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f116c0c1840 0x7f116c1689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f117159b700 1 -- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f116c0c6240 msgr2=0x7f116c14eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f117159b700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f116c0c6240 0x7f116c14eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f117159b700 1 -- 172.21.15.33:0/1360169180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168009370 con 0x7f116c0aa2d0 2023-11-06T21:02:15.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.263+0000 7f1170d9a700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f116c0c1840 0x7f116c1689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:15.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.264+0000 7f117159b700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0aa2d0 0x7f116c157300 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f1168009130 tx=0x7f1168017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:15.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.264+0000 7f11617fa700 1 -- 172.21.15.33:0/1360169180 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800be90 con 0x7f116c0aa2d0 2023-11-06T21:02:15.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.264+0000 7f11617fa700 1 -- 172.21.15.33:0/1360169180 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116800b7f0 con 0x7f116c0aa2d0 2023-11-06T21:02:15.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.264+0000 7f11737ff700 1 -- 172.21.15.33:0/1360169180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116c1649e0 con 0x7f116c0aa2d0 2023-11-06T21:02:15.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.264+0000 7f11617fa700 1 -- 172.21.15.33:0/1360169180 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116801c670 con 0x7f116c0aa2d0 2023-11-06T21:02:15.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.264+0000 7f11737ff700 1 -- 172.21.15.33:0/1360169180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c164ca0 con 0x7f116c0aa2d0 2023-11-06T21:02:15.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.265+0000 7f11737ff700 1 -- 172.21.15.33:0/1360169180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116c06c6c0 con 0x7f116c0aa2d0 2023-11-06T21:02:15.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.266+0000 7f11617fa700 1 -- 172.21.15.33:0/1360169180 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f116801c9e0 con 0x7f116c0aa2d0 2023-11-06T21:02:15.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.267+0000 7f11617fa700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f115005f270 0x7f1150061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.267+0000 7f11617fa700 1 -- 172.21.15.33:0/1360169180 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f116807bfc0 con 0x7f116c0aa2d0 2023-11-06T21:02:15.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.268+0000 7f1170d9a700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f115005f270 0x7f1150061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.268+0000 7f1170d9a700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f115005f270 0x7f1150061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f115c006d80 tx=0x7f115c008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:15.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.271+0000 7f11617fa700 1 -- 172.21.15.33:0/1360169180 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f116804d110 con 0x7f116c0aa2d0 2023-11-06T21:02:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.496+0000 7f11737ff700 1 -- 172.21.15.33:0/1360169180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f116c154ff0 con 0x7f116c0aa2d0 2023-11-06T21:02:15.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.496+0000 7f11617fa700 1 -- 172.21.15.33:0/1360169180 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f116802a070 con 0x7f116c0aa2d0 2023-11-06T21:02:15.497 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:15.497 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":53,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2500558031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2500558031},{"type":"v1","addr":"172.21.15.69:6837","nonce":2500558031}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1012474146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1012474146},{"type":"v1","addr":"172.21.15.69:6836","nonce":1012474146}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4020289695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4020289695},{"type":"v1","addr":"172.21.15.33:6834","nonce":4020289695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:02:11.991421+0000","modified":"2023-11-06T21:02:15.032423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1130278640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1130278640},{"type":"v1","addr":"172.21.15.33:6835","nonce":1130278640}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-06T21:02:15.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.498+0000 7f11737ff700 1 -- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f115005f270 msgr2=0x7f1150061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.498+0000 7f11737ff700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f115005f270 0x7f1150061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f115c006d80 tx=0x7f115c008040).stop 2023-11-06T21:02:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.498+0000 7f11737ff700 1 -- 172.21.15.33:0/1360169180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0aa2d0 msgr2=0x7f116c157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.498+0000 7f11737ff700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0aa2d0 0x7f116c157300 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f1168009130 tx=0x7f1168017000).stop 2023-11-06T21:02:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.499+0000 7f11737ff700 1 -- 172.21.15.33:0/1360169180 shutdown_connections 2023-11-06T21:02:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.499+0000 7f11737ff700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f115005f270 0x7f1150061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.499+0000 7f11737ff700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f116c0c6240 0x7f116c14eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.499+0000 7f11737ff700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f116c0aa2d0 0x7f116c157300 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.499+0000 7f11737ff700 1 --2- 172.21.15.33:0/1360169180 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f116c0c1840 0x7f116c1689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.499+0000 7f11737ff700 1 -- 172.21.15.33:0/1360169180 >> 172.21.15.33:0/1360169180 conn(0x7f116c05ec80 msgr2=0x7f116c14f910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:15.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.499+0000 7f11737ff700 1 -- 172.21.15.33:0/1360169180 shutdown_connections 2023-11-06T21:02:15.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.499+0000 7f11737ff700 1 -- 172.21.15.33:0/1360169180 wait complete. 2023-11-06T21:02:15.502 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 53 2023-11-06T21:02:15.517 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:02:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.711+0000 7f7a963ff700 1 Processor -- start 2023-11-06T21:02:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.712+0000 7f7a963ff700 1 -- start start 2023-11-06T21:02:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.713+0000 7f7a963ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a900c1840 0x7f7a900a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.713+0000 7f7a963ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a900aa2d0 0x7f7a900c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.713+0000 7f7a963ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a901554a0 con 0x7f7a900c6240 2023-11-06T21:02:15.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.713+0000 7f7a963ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a90154960 con 0x7f7a900aa2d0 2023-11-06T21:02:15.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.713+0000 7f7a963ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a90155930 con 0x7f7a900c1840 2023-11-06T21:02:15.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.713+0000 7f7a8f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a900aa2d0 0x7f7a900c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a900aa2d0 0x7f7a900c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37114/0 (socket says 172.21.15.33:37114) 2023-11-06T21:02:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8f7fe700 1 -- 172.21.15.33:0/4057576996 learned_addr learned my addr 172.21.15.33:0/4057576996 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8ffff700 1 --2- 172.21.15.33:0/4057576996 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a900c1840 0x7f7a900a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8f7fe700 1 -- 172.21.15.33:0/4057576996 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a900c1840 msgr2=0x7f7a900a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8f7fe700 1 --2- 172.21.15.33:0/4057576996 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a900c1840 0x7f7a900a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8f7fe700 1 -- 172.21.15.33:0/4057576996 >> v1:172.21.15.69:6789/0 conn(0x7f7a900c6240 legacy=0x7f7a90157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8f7fe700 1 -- 172.21.15.33:0/4057576996 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90154b70 con 0x7f7a900aa2d0 2023-11-06T21:02:15.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8f7fe700 1 --2- 172.21.15.33:0/4057576996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900aa2d0 0x7f7a900c5d50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f7a84006fd0 tx=0x7f7a840090f0).ready entity=mon.0 client_cookie=1865f7fe3b6145bf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:15.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8effd700 1 -- 172.21.15.33:0/4057576996 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400b580 con 0x7f7a900aa2d0 2023-11-06T21:02:15.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8effd700 1 -- 172.21.15.33:0/4057576996 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8400b6e0 con 0x7f7a900aa2d0 2023-11-06T21:02:15.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.714+0000 7f7a8effd700 1 -- 172.21.15.33:0/4057576996 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400ba60 con 0x7f7a900aa2d0 2023-11-06T21:02:15.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.715+0000 7f7a963ff700 1 -- 172.21.15.33:0/4057576996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900aa2d0 msgr2=0x7f7a900c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.715+0000 7f7a963ff700 1 --2- 172.21.15.33:0/4057576996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900aa2d0 0x7f7a900c5d50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f7a84006fd0 tx=0x7f7a840090f0).stop 2023-11-06T21:02:15.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.715+0000 7f7a963ff700 1 -- 172.21.15.33:0/4057576996 shutdown_connections 2023-11-06T21:02:15.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.715+0000 7f7a963ff700 1 --2- 172.21.15.33:0/4057576996 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a900c1840 0x7f7a900a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.715+0000 7f7a963ff700 1 --2- 172.21.15.33:0/4057576996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900aa2d0 0x7f7a900c5d50 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.715+0000 7f7a963ff700 1 -- 172.21.15.33:0/4057576996 >> 172.21.15.33:0/4057576996 conn(0x7f7a9005ec80 msgr2=0x7f7a900cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:15.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.715+0000 7f7a963ff700 1 -- 172.21.15.33:0/4057576996 shutdown_connections 2023-11-06T21:02:15.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.715+0000 7f7a963ff700 1 -- 172.21.15.33:0/4057576996 wait complete. 2023-11-06T21:02:15.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.717+0000 7f7a963ff700 1 Processor -- start 2023-11-06T21:02:15.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.717+0000 7f7a963ff700 1 -- start start 2023-11-06T21:02:15.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a963ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a900aa2d0 0x7f7a901571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a963ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900c1840 0x7f7a900bff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a963ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a900c6240 0x7f7a900af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a963ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a900b0700 con 0x7f7a900c1840 2023-11-06T21:02:15.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a963ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a900afe60 con 0x7f7a900c6240 2023-11-06T21:02:15.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a963ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a900b0140 con 0x7f7a900aa2d0 2023-11-06T21:02:15.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a8f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900c1840 0x7f7a900bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a8f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900c1840 0x7f7a900bff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37116/0 (socket says 172.21.15.33:37116) 2023-11-06T21:02:15.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a8f7fe700 1 -- 172.21.15.33:0/224407315 learned_addr learned my addr 172.21.15.33:0/224407315 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:15.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a8ffff700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a900aa2d0 0x7f7a901571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a9499c700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a900c6240 0x7f7a900af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a8f7fe700 1 -- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a900aa2d0 msgr2=0x7f7a901571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a8f7fe700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a900aa2d0 0x7f7a901571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a8f7fe700 1 -- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a900c6240 msgr2=0x7f7a900af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a8f7fe700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a900c6240 0x7f7a900af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.718+0000 7f7a8f7fe700 1 -- 172.21.15.33:0/224407315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84009370 con 0x7f7a900c1840 2023-11-06T21:02:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.719+0000 7f7a8f7fe700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900c1840 0x7f7a900bff30 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f7a84009fd0 tx=0x7f7a84017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.719+0000 7f7a77fff700 1 -- 172.21.15.33:0/224407315 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84006a40 con 0x7f7a900c1840 2023-11-06T21:02:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.719+0000 7f7a77fff700 1 -- 172.21.15.33:0/224407315 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a84000bc0 con 0x7f7a900c1840 2023-11-06T21:02:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.719+0000 7f7a963ff700 1 -- 172.21.15.33:0/224407315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a900b0420 con 0x7f7a900c1840 2023-11-06T21:02:15.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.719+0000 7f7a77fff700 1 -- 172.21.15.33:0/224407315 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8401c420 con 0x7f7a900c1840 2023-11-06T21:02:15.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.719+0000 7f7a963ff700 1 -- 172.21.15.33:0/224407315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a90167e30 con 0x7f7a900c1840 2023-11-06T21:02:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.720+0000 7f7a963ff700 1 -- 172.21.15.33:0/224407315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9006c6c0 con 0x7f7a900c1840 2023-11-06T21:02:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.721+0000 7f7a77fff700 1 -- 172.21.15.33:0/224407315 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7a8401c810 con 0x7f7a900c1840 2023-11-06T21:02:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.722+0000 7f7a77fff700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a7c05f270 0x7f7a7c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.722+0000 7f7a77fff700 1 -- 172.21.15.33:0/224407315 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7a8407bff0 con 0x7f7a900c1840 2023-11-06T21:02:15.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.722+0000 7f7a8ffff700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a7c05f270 0x7f7a7c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:15.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.723+0000 7f7a8ffff700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a7c05f270 0x7f7a7c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7a80009c30 tx=0x7f7a80009060).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:15.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.726+0000 7f7a77fff700 1 -- 172.21.15.33:0/224407315 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7a8404d0e0 con 0x7f7a900c1840 2023-11-06T21:02:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.908+0000 7f7a963ff700 1 -- 172.21.15.33:0/224407315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a90063f20 con 0x7f7a900c1840 2023-11-06T21:02:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.909+0000 7f7a77fff700 1 -- 172.21.15.33:0/224407315 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16991 (secure 0 0 0) 0x7f7a8402a070 con 0x7f7a900c1840 2023-11-06T21:02:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:15.910 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":131,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:02:12.993590+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:02:09.533597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T21:02:10.978394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:02:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 -- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a7c05f270 msgr2=0x7f7a7c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a7c05f270 0x7f7a7c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7a80009c30 tx=0x7f7a80009060).stop 2023-11-06T21:02:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 -- 172.21.15.33:0/224407315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900c1840 msgr2=0x7f7a900bff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900c1840 0x7f7a900bff30 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f7a84009fd0 tx=0x7f7a84017000).stop 2023-11-06T21:02:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 -- 172.21.15.33:0/224407315 shutdown_connections 2023-11-06T21:02:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7a7c05f270 0x7f7a7c061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7a900aa2d0 0x7f7a901571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a900c6240 0x7f7a900af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 --2- 172.21.15.33:0/224407315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a900c1840 0x7f7a900bff30 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 -- 172.21.15.33:0/224407315 >> 172.21.15.33:0/224407315 conn(0x7f7a9005ec80 msgr2=0x7f7a90152800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 -- 172.21.15.33:0/224407315 shutdown_connections 2023-11-06T21:02:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:15.911+0000 7f7a963ff700 1 -- 172.21.15.33:0/224407315 wait complete. 2023-11-06T21:02:15.929 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T21:02:15.930 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T21:02:16.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.129+0000 7fd788fff700 1 Processor -- start 2023-11-06T21:02:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.129+0000 7fd788fff700 1 -- start start 2023-11-06T21:02:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.130+0000 7fd788fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7840c0c50 0x7fd7840c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.130+0000 7fd788fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd784154e90 con 0x7fd7841521a0 2023-11-06T21:02:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.130+0000 7fd788fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7840639d0 con 0x7fd784151dc0 2023-11-06T21:02:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.130+0000 7fd788fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd784155a30 con 0x7fd7840c0c50 2023-11-06T21:02:16.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.130+0000 7fd782d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7840c0c50 0x7fd7840c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.130+0000 7fd782d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7840c0c50 0x7fd7840c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37124/0 (socket says 172.21.15.33:37124) 2023-11-06T21:02:16.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.130+0000 7fd782d9d700 1 -- 172.21.15.33:0/2553982987 learned_addr learned my addr 172.21.15.33:0/2553982987 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:16.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.130+0000 7fd781d9b700 1 --1- 172.21.15.33:0/2553982987 >> v1:172.21.15.33:6789/0 conn(0x7fd7841521a0 0x7fd784155320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42164/0 (socket says 172.21.15.33:42164) 2023-11-06T21:02:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.131+0000 7fd782d9d700 1 -- 172.21.15.33:0/2553982987 >> v1:172.21.15.69:6789/0 conn(0x7fd784151dc0 legacy=0x7fd784063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.131+0000 7fd782d9d700 1 -- 172.21.15.33:0/2553982987 >> v1:172.21.15.33:6789/0 conn(0x7fd7841521a0 legacy=0x7fd784155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.131+0000 7fd782d9d700 1 -- 172.21.15.33:0/2553982987 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd784063cb0 con 0x7fd7840c0c50 2023-11-06T21:02:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.131+0000 7fd782d9d700 1 --2- 172.21.15.33:0/2553982987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 0x7fd7840c7850 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7fd77c009f20 tx=0x7fd77c009840).ready entity=mon.0 client_cookie=e333bfd2a821e589 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.131+0000 7fd78159a700 1 -- 172.21.15.33:0/2553982987 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00e070 con 0x7fd7840c0c50 2023-11-06T21:02:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.131+0000 7fd78159a700 1 -- 172.21.15.33:0/2553982987 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77c00ba80 con 0x7fd7840c0c50 2023-11-06T21:02:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.131+0000 7fd78159a700 1 -- 172.21.15.33:0/2553982987 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00be00 con 0x7fd7840c0c50 2023-11-06T21:02:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.132+0000 7fd788fff700 1 -- 172.21.15.33:0/2553982987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 msgr2=0x7fd7840c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.132+0000 7fd788fff700 1 --2- 172.21.15.33:0/2553982987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 0x7fd7840c7850 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7fd77c009f20 tx=0x7fd77c009840).stop 2023-11-06T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.132+0000 7fd788fff700 1 -- 172.21.15.33:0/2553982987 shutdown_connections 2023-11-06T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.132+0000 7fd788fff700 1 --2- 172.21.15.33:0/2553982987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 0x7fd7840c7850 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.132+0000 7fd788fff700 1 -- 172.21.15.33:0/2553982987 >> 172.21.15.33:0/2553982987 conn(0x7fd78405e1b0 msgr2=0x7fd78405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.132+0000 7fd788fff700 1 -- 172.21.15.33:0/2553982987 shutdown_connections 2023-11-06T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.132+0000 7fd788fff700 1 -- 172.21.15.33:0/2553982987 wait complete. 2023-11-06T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.133+0000 7fd788fff700 1 Processor -- start 2023-11-06T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.134+0000 7fd788fff700 1 -- start start 2023-11-06T21:02:16.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.134+0000 7fd788fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 0x7fd7841688e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.134+0000 7fd788fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd784151dc0 0x7fd7840af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.134+0000 7fd788fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7841521a0 0x7fd7840addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.134+0000 7fd788fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd784150860 con 0x7fd7840c0c50 2023-11-06T21:02:16.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.134+0000 7fd788fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd784168450 con 0x7fd7841521a0 2023-11-06T21:02:16.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.134+0000 7fd788fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840aec00 con 0x7fd784151dc0 2023-11-06T21:02:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd78259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 0x7fd7841688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd78259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 0x7fd7841688e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37134/0 (socket says 172.21.15.33:37134) 2023-11-06T21:02:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd78259c700 1 -- 172.21.15.33:0/1086524488 learned_addr learned my addr 172.21.15.33:0/1086524488 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd781d9b700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd784151dc0 0x7fd7840af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd782d9d700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7841521a0 0x7fd7840addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd78259c700 1 -- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd784151dc0 msgr2=0x7fd7840af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd78259c700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd784151dc0 0x7fd7840af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd78259c700 1 -- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7841521a0 msgr2=0x7fd7840addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd78259c700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7841521a0 0x7fd7840addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd78259c700 1 -- 172.21.15.33:0/1086524488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c009ae0 con 0x7fd7840c0c50 2023-11-06T21:02:16.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.135+0000 7fd78259c700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 0x7fd7841688e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7fd77400c4c0 tx=0x7fd774007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:16.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.136+0000 7fd7727fc700 1 -- 172.21.15.33:0/1086524488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77400ce70 con 0x7fd7840c0c50 2023-11-06T21:02:16.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.136+0000 7fd7727fc700 1 -- 172.21.15.33:0/1086524488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd774001e90 con 0x7fd7840c0c50 2023-11-06T21:02:16.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.136+0000 7fd788fff700 1 -- 172.21.15.33:0/1086524488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7840ae920 con 0x7fd7840c0c50 2023-11-06T21:02:16.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.136+0000 7fd7727fc700 1 -- 172.21.15.33:0/1086524488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774007250 con 0x7fd7840c0c50 2023-11-06T21:02:16.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.136+0000 7fd788fff700 1 -- 172.21.15.33:0/1086524488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd784167eb0 con 0x7fd7840c0c50 2023-11-06T21:02:16.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.137+0000 7fd788fff700 1 -- 172.21.15.33:0/1086524488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd784150db0 con 0x7fd7840c0c50 2023-11-06T21:02:16.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.138+0000 7fd7727fc700 1 -- 172.21.15.33:0/1086524488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd7740066a0 con 0x7fd7840c0c50 2023-11-06T21:02:16.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.138+0000 7fd7727fc700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd76405f240 0x7fd764061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.139+0000 7fd7727fc700 1 -- 172.21.15.33:0/1086524488 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd774079c20 con 0x7fd7840c0c50 2023-11-06T21:02:16.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.139+0000 7fd781d9b700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd76405f240 0x7fd764061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.140+0000 7fd781d9b700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd76405f240 0x7fd764061700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd7840b0cb0 tx=0x7fd778007d30).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:16.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.142+0000 7fd7727fc700 1 -- 172.21.15.33:0/1086524488 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd77404ad70 con 0x7fd7840c0c50 2023-11-06T21:02:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.321+0000 7fd788fff700 1 -- 172.21.15.33:0/1086524488 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd784168730 con 0x7fd76405f240 2023-11-06T21:02:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.334+0000 7fd7727fc700 1 -- 172.21.15.33:0/1086524488 <== mgr.6377 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361873 (secure 0 0 0) 0x7fd784168730 con 0x7fd76405f240 2023-11-06T21:02:16.340 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:16.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 -- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd76405f240 msgr2=0x7fd764061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd76405f240 0x7fd764061700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd7840b0cb0 tx=0x7fd778007d30).stop 2023-11-06T21:02:16.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 -- 172.21.15.33:0/1086524488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 msgr2=0x7fd7841688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 0x7fd7841688e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7fd77400c4c0 tx=0x7fd774007e00).stop 2023-11-06T21:02:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 -- 172.21.15.33:0/1086524488 shutdown_connections 2023-11-06T21:02:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd76405f240 0x7fd764061700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7841521a0 0x7fd7840addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7840c0c50 0x7fd7841688e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 --2- 172.21.15.33:0/1086524488 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd784151dc0 0x7fd7840af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 -- 172.21.15.33:0/1086524488 >> 172.21.15.33:0/1086524488 conn(0x7fd78405e1b0 msgr2=0x7fd78405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:16.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 -- 172.21.15.33:0/1086524488 shutdown_connections 2023-11-06T21:02:16.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.339+0000 7fd788fff700 1 -- 172.21.15.33:0/1086524488 wait complete. 2023-11-06T21:02:16.348 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T21:02:16.358 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T21:02:15.445566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422156,"kb_used_data":12620,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752552564,"statfs":{"total":773094113280,"available":770613825536,"internally_reserved":0,"allocated":12922880,"data_stored":6513980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6815,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-61,"num_write_kb":-29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015470"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.021932+0000","last_change":"2023-11-06T21:02:12.013786+0000","last_active":"2023-11-06T21:02:12.021932+0000","last_peered":"2023-11-06T21:02:12.021932+0000","last_clean":"2023-11-06T21:02:12.021932+0000","last_became_active":"2023-11-06T21:02:12.013498+0000","last_became_peered":"2023-11-06T21:02:12.013498+0000","last_unstale":"2023-11-06T21:02:12.021932+0000","last_undegraded":"2023-11-06T21:02:12.021932+0000","last_fullsized":"2023-11-06T21:02:12.021932+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.3f","version":"130'3","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.026295+0000","last_change":"2023-11-06T21:02:11.527254+0000","last_active":"2023-11-06T21:02:12.026295+0000","last_peered":"2023-11-06T21:02:12.026295+0000","last_clean":"2023-11-06T21:02:12.026295+0000","last_became_active":"2023-11-06T21:02:11.526778+0000","last_became_peered":"2023-11-06T21:02:11.526778+0000","last_unstale":"2023-11-06T21:02:12.026295+0000","last_undegraded":"2023-11-06T21:02:12.026295+0000","last_fullsized":"2023-11-06T21:02:12.026295+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038118+0000","last_change":"2023-11-06T21:02:12.038118+0000","last_active":"2023-11-06T21:02:12.038118+0000","last_peered":"2023-11-06T21:02:12.038118+0000","last_clean":"2023-11-06T21:02:12.038118+0000","last_became_active":"2023-11-06T21:02:12.037856+0000","last_became_peered":"2023-11-06T21:02:12.037856+0000","last_unstale":"2023-11-06T21:02:12.038118+0000","last_undegraded":"2023-11-06T21:02:12.038118+0000","last_fullsized":"2023-11-06T21:02:12.038118+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.531440+0000","last_change":"2023-11-06T21:02:11.520733+0000","last_active":"2023-11-06T21:02:11.531440+0000","last_peered":"2023-11-06T21:02:11.531440+0000","last_clean":"2023-11-06T21:02:11.531440+0000","last_became_active":"2023-11-06T21:02:11.520446+0000","last_became_peered":"2023-11-06T21:02:11.520446+0000","last_unstale":"2023-11-06T21:02:11.531440+0000","last_undegraded":"2023-11-06T21:02:11.531440+0000","last_fullsized":"2023-11-06T21:02:11.531440+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.013962+0000","last_change":"2023-11-06T21:02:12.013962+0000","last_active":"2023-11-06T21:02:12.013962+0000","last_peered":"2023-11-06T21:02:12.013962+0000","last_clean":"2023-11-06T21:02:12.013962+0000","last_became_active":"2023-11-06T21:02:12.013683+0000","last_became_peered":"2023-11-06T21:02:12.013683+0000","last_unstale":"2023-11-06T21:02:12.013962+0000","last_undegraded":"2023-11-06T21:02:12.013962+0000","last_fullsized":"2023-11-06T21:02:12.013962+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.542922+0000","last_change":"2023-11-06T21:02:11.541728+0000","last_active":"2023-11-06T21:02:11.542922+0000","last_peered":"2023-11-06T21:02:11.542922+0000","last_clean":"2023-11-06T21:02:11.542922+0000","last_became_active":"2023-11-06T21:02:11.541443+0000","last_became_peered":"2023-11-06T21:02:11.541443+0000","last_unstale":"2023-11-06T21:02:11.542922+0000","last_undegraded":"2023-11-06T21:02:11.542922+0000","last_fullsized":"2023-11-06T21:02:11.542922+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.034854+0000","last_change":"2023-11-06T21:02:12.034854+0000","last_active":"2023-11-06T21:02:12.034854+0000","last_peered":"2023-11-06T21:02:12.034854+0000","last_clean":"2023-11-06T21:02:12.034854+0000","last_became_active":"2023-11-06T21:02:12.034572+0000","last_became_peered":"2023-11-06T21:02:12.034572+0000","last_unstale":"2023-11-06T21:02:12.034854+0000","last_undegraded":"2023-11-06T21:02:12.034854+0000","last_fullsized":"2023-11-06T21:02:12.034854+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3c","version":"130'2","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.027348+0000","last_change":"2023-11-06T21:02:11.535851+0000","last_active":"2023-11-06T21:02:12.027348+0000","last_peered":"2023-11-06T21:02:12.027348+0000","last_clean":"2023-11-06T21:02:12.027348+0000","last_became_active":"2023-11-06T21:02:11.535529+0000","last_became_peered":"2023-11-06T21:02:11.535529+0000","last_unstale":"2023-11-06T21:02:12.027348+0000","last_undegraded":"2023-11-06T21:02:12.027348+0000","last_fullsized":"2023-11-06T21:02:12.027348+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.035158+0000","last_change":"2023-11-06T21:02:12.035158+0000","last_active":"2023-11-06T21:02:12.035158+0000","last_peered":"2023-11-06T21:02:12.035158+0000","last_clean":"2023-11-06T21:02:12.035158+0000","last_became_active":"2023-11-06T21:02:12.034883+0000","last_became_peered":"2023-11-06T21:02:12.034883+0000","last_unstale":"2023-11-06T21:02:12.035158+0000","last_undegraded":"2023-11-06T21:02:12.035158+0000","last_fullsized":"2023-11-06T21:02:12.035158+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3b","version":"130'1","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.999531+0000","last_change":"2023-11-06T21:02:11.522959+0000","last_active":"2023-11-06T21:02:11.999531+0000","last_peered":"2023-11-06T21:02:11.999531+0000","last_clean":"2023-11-06T21:02:11.999531+0000","last_became_active":"2023-11-06T21:02:11.522509+0000","last_became_peered":"2023-11-06T21:02:11.522509+0000","last_unstale":"2023-11-06T21:02:11.999531+0000","last_undegraded":"2023-11-06T21:02:11.999531+0000","last_fullsized":"2023-11-06T21:02:11.999531+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.026628+0000","last_change":"2023-11-06T21:02:12.020160+0000","last_active":"2023-11-06T21:02:12.026628+0000","last_peered":"2023-11-06T21:02:12.026628+0000","last_clean":"2023-11-06T21:02:12.026628+0000","last_became_active":"2023-11-06T21:02:12.018793+0000","last_became_peered":"2023-11-06T21:02:12.018793+0000","last_unstale":"2023-11-06T21:02:12.026628+0000","last_undegraded":"2023-11-06T21:02:12.026628+0000","last_fullsized":"2023-11-06T21:02:12.026628+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3a","version":"130'1","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.999304+0000","last_change":"2023-11-06T21:02:11.542300+0000","last_active":"2023-11-06T21:02:11.999304+0000","last_peered":"2023-11-06T21:02:11.999304+0000","last_clean":"2023-11-06T21:02:11.999304+0000","last_became_active":"2023-11-06T21:02:11.541995+0000","last_became_peered":"2023-11-06T21:02:11.541995+0000","last_unstale":"2023-11-06T21:02:11.999304+0000","last_undegraded":"2023-11-06T21:02:11.999304+0000","last_fullsized":"2023-11-06T21:02:11.999304+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.032349+0000","last_change":"2023-11-06T21:02:12.025288+0000","last_active":"2023-11-06T21:02:12.032349+0000","last_peered":"2023-11-06T21:02:12.032349+0000","last_clean":"2023-11-06T21:02:12.032349+0000","last_became_active":"2023-11-06T21:02:12.024628+0000","last_became_peered":"2023-11-06T21:02:12.024628+0000","last_unstale":"2023-11-06T21:02:12.032349+0000","last_undegraded":"2023-11-06T21:02:12.032349+0000","last_fullsized":"2023-11-06T21:02:12.032349+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.533775+0000","last_change":"2023-11-06T21:02:11.530168+0000","last_active":"2023-11-06T21:02:11.533775+0000","last_peered":"2023-11-06T21:02:11.533775+0000","last_clean":"2023-11-06T21:02:11.533775+0000","last_became_active":"2023-11-06T21:02:11.529750+0000","last_became_peered":"2023-11-06T21:02:11.529750+0000","last_unstale":"2023-11-06T21:02:11.533775+0000","last_undegraded":"2023-11-06T21:02:11.533775+0000","last_fullsized":"2023-11-06T21:02:11.533775+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.032607+0000","last_change":"2023-11-06T21:02:12.028903+0000","last_active":"2023-11-06T21:02:12.032607+0000","last_peered":"2023-11-06T21:02:12.032607+0000","last_clean":"2023-11-06T21:02:12.032607+0000","last_became_active":"2023-11-06T21:02:12.025751+0000","last_became_peered":"2023-11-06T21:02:12.025751+0000","last_unstale":"2023-11-06T21:02:12.032607+0000","last_undegraded":"2023-11-06T21:02:12.032607+0000","last_fullsized":"2023-11-06T21:02:12.032607+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.538714+0000","last_change":"2023-11-06T21:02:11.535205+0000","last_active":"2023-11-06T21:02:11.538714+0000","last_peered":"2023-11-06T21:02:11.538714+0000","last_clean":"2023-11-06T21:02:11.538714+0000","last_became_active":"2023-11-06T21:02:11.534788+0000","last_became_peered":"2023-11-06T21:02:11.534788+0000","last_unstale":"2023-11-06T21:02:11.538714+0000","last_undegraded":"2023-11-06T21:02:11.538714+0000","last_fullsized":"2023-11-06T21:02:11.538714+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.022239+0000","last_change":"2023-11-06T21:02:12.022239+0000","last_active":"2023-11-06T21:02:12.022239+0000","last_peered":"2023-11-06T21:02:12.022239+0000","last_clean":"2023-11-06T21:02:12.022239+0000","last_became_active":"2023-11-06T21:02:12.020806+0000","last_became_peered":"2023-11-06T21:02:12.020806+0000","last_unstale":"2023-11-06T21:02:12.022239+0000","last_undegraded":"2023-11-06T21:02:12.022239+0000","last_fullsized":"2023-11-06T21:02:12.022239+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.542403+0000","last_change":"2023-11-06T21:02:11.537853+0000","last_active":"2023-11-06T21:02:11.542403+0000","last_peered":"2023-11-06T21:02:11.542403+0000","last_clean":"2023-11-06T21:02:11.542403+0000","last_became_active":"2023-11-06T21:02:11.537277+0000","last_became_peered":"2023-11-06T21:02:11.537277+0000","last_unstale":"2023-11-06T21:02:11.542403+0000","last_undegraded":"2023-11-06T21:02:11.542403+0000","last_fullsized":"2023-11-06T21:02:11.542403+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.030838+0000","last_change":"2023-11-06T21:02:12.030838+0000","last_active":"2023-11-06T21:02:12.030838+0000","last_peered":"2023-11-06T21:02:12.030838+0000","last_clean":"2023-11-06T21:02:12.030838+0000","last_became_active":"2023-11-06T21:02:12.030461+0000","last_became_peered":"2023-11-06T21:02:12.030461+0000","last_unstale":"2023-11-06T21:02:12.030838+0000","last_undegraded":"2023-11-06T21:02:12.030838+0000","last_fullsized":"2023-11-06T21:02:12.030838+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.533818+0000","last_change":"2023-11-06T21:02:11.527819+0000","last_active":"2023-11-06T21:02:11.533818+0000","last_peered":"2023-11-06T21:02:11.533818+0000","last_clean":"2023-11-06T21:02:11.533818+0000","last_became_active":"2023-11-06T21:02:11.527546+0000","last_became_peered":"2023-11-06T21:02:11.527546+0000","last_unstale":"2023-11-06T21:02:11.533818+0000","last_undegraded":"2023-11-06T21:02:11.533818+0000","last_fullsized":"2023-11-06T21:02:11.533818+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.037251+0000","last_change":"2023-11-06T21:02:12.034884+0000","last_active":"2023-11-06T21:02:12.037251+0000","last_peered":"2023-11-06T21:02:12.037251+0000","last_clean":"2023-11-06T21:02:12.037251+0000","last_became_active":"2023-11-06T21:02:12.034027+0000","last_became_peered":"2023-11-06T21:02:12.034027+0000","last_unstale":"2023-11-06T21:02:12.037251+0000","last_undegraded":"2023-11-06T21:02:12.037251+0000","last_fullsized":"2023-11-06T21:02:12.037251+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.538680+0000","last_change":"2023-11-06T21:02:11.536631+0000","last_active":"2023-11-06T21:02:11.538680+0000","last_peered":"2023-11-06T21:02:11.538680+0000","last_clean":"2023-11-06T21:02:11.538680+0000","last_became_active":"2023-11-06T21:02:11.536091+0000","last_became_peered":"2023-11-06T21:02:11.536091+0000","last_unstale":"2023-11-06T21:02:11.538680+0000","last_undegraded":"2023-11-06T21:02:11.538680+0000","last_fullsized":"2023-11-06T21:02:11.538680+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.016079+0000","last_change":"2023-11-06T21:02:12.014195+0000","last_active":"2023-11-06T21:02:12.016079+0000","last_peered":"2023-11-06T21:02:12.016079+0000","last_clean":"2023-11-06T21:02:12.016079+0000","last_became_active":"2023-11-06T21:02:12.012965+0000","last_became_peered":"2023-11-06T21:02:12.012965+0000","last_unstale":"2023-11-06T21:02:12.016079+0000","last_undegraded":"2023-11-06T21:02:12.016079+0000","last_fullsized":"2023-11-06T21:02:12.016079+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.34","version":"130'4","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.036697+0000","last_change":"2023-11-06T21:02:11.531141+0000","last_active":"2023-11-06T21:02:12.036697+0000","last_peered":"2023-11-06T21:02:12.036697+0000","last_clean":"2023-11-06T21:02:12.036697+0000","last_became_active":"2023-11-06T21:02:11.530885+0000","last_became_peered":"2023-11-06T21:02:11.530885+0000","last_unstale":"2023-11-06T21:02:12.036697+0000","last_undegraded":"2023-11-06T21:02:12.036697+0000","last_fullsized":"2023-11-06T21:02:12.036697+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.032775+0000","last_change":"2023-11-06T21:02:12.027108+0000","last_active":"2023-11-06T21:02:12.032775+0000","last_peered":"2023-11-06T21:02:12.032775+0000","last_clean":"2023-11-06T21:02:12.032775+0000","last_became_active":"2023-11-06T21:02:12.022110+0000","last_became_peered":"2023-11-06T21:02:12.022110+0000","last_unstale":"2023-11-06T21:02:12.032775+0000","last_undegraded":"2023-11-06T21:02:12.032775+0000","last_fullsized":"2023-11-06T21:02:12.032775+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.039387+0000","last_change":"2023-11-06T21:02:11.528106+0000","last_active":"2023-11-06T21:02:12.039387+0000","last_peered":"2023-11-06T21:02:12.039387+0000","last_clean":"2023-11-06T21:02:12.039387+0000","last_became_active":"2023-11-06T21:02:11.527781+0000","last_became_peered":"2023-11-06T21:02:11.527781+0000","last_unstale":"2023-11-06T21:02:12.039387+0000","last_undegraded":"2023-11-06T21:02:12.039387+0000","last_fullsized":"2023-11-06T21:02:12.039387+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038158+0000","last_change":"2023-11-06T21:02:12.038158+0000","last_active":"2023-11-06T21:02:12.038158+0000","last_peered":"2023-11-06T21:02:12.038158+0000","last_clean":"2023-11-06T21:02:12.038158+0000","last_became_active":"2023-11-06T21:02:12.037737+0000","last_became_peered":"2023-11-06T21:02:12.037737+0000","last_unstale":"2023-11-06T21:02:12.038158+0000","last_undegraded":"2023-11-06T21:02:12.038158+0000","last_fullsized":"2023-11-06T21:02:12.038158+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.530651+0000","last_change":"2023-11-06T21:02:11.519433+0000","last_active":"2023-11-06T21:02:11.530651+0000","last_peered":"2023-11-06T21:02:11.530651+0000","last_clean":"2023-11-06T21:02:11.530651+0000","last_became_active":"2023-11-06T21:02:11.517627+0000","last_became_peered":"2023-11-06T21:02:11.517627+0000","last_unstale":"2023-11-06T21:02:11.530651+0000","last_undegraded":"2023-11-06T21:02:11.530651+0000","last_fullsized":"2023-11-06T21:02:11.530651+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.022378+0000","last_change":"2023-11-06T21:02:12.022378+0000","last_active":"2023-11-06T21:02:12.022378+0000","last_peered":"2023-11-06T21:02:12.022378+0000","last_clean":"2023-11-06T21:02:12.022378+0000","last_became_active":"2023-11-06T21:02:12.022081+0000","last_became_peered":"2023-11-06T21:02:12.022081+0000","last_unstale":"2023-11-06T21:02:12.022378+0000","last_undegraded":"2023-11-06T21:02:12.022378+0000","last_fullsized":"2023-11-06T21:02:12.022378+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.532638+0000","last_change":"2023-11-06T21:02:11.528866+0000","last_active":"2023-11-06T21:02:11.532638+0000","last_peered":"2023-11-06T21:02:11.532638+0000","last_clean":"2023-11-06T21:02:11.532638+0000","last_became_active":"2023-11-06T21:02:11.528564+0000","last_became_peered":"2023-11-06T21:02:11.528564+0000","last_unstale":"2023-11-06T21:02:11.532638+0000","last_undegraded":"2023-11-06T21:02:11.532638+0000","last_fullsized":"2023-11-06T21:02:11.532638+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038100+0000","last_change":"2023-11-06T21:02:12.035815+0000","last_active":"2023-11-06T21:02:12.038100+0000","last_peered":"2023-11-06T21:02:12.038100+0000","last_clean":"2023-11-06T21:02:12.038100+0000","last_became_active":"2023-11-06T21:02:12.035518+0000","last_became_peered":"2023-11-06T21:02:12.035518+0000","last_unstale":"2023-11-06T21:02:12.038100+0000","last_undegraded":"2023-11-06T21:02:12.038100+0000","last_fullsized":"2023-11-06T21:02:12.038100+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.30","version":"130'4","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.015309+0000","last_change":"2023-11-06T21:02:11.537869+0000","last_active":"2023-11-06T21:02:12.015309+0000","last_peered":"2023-11-06T21:02:12.015309+0000","last_clean":"2023-11-06T21:02:12.015309+0000","last_became_active":"2023-11-06T21:02:11.537564+0000","last_became_peered":"2023-11-06T21:02:11.537564+0000","last_unstale":"2023-11-06T21:02:12.015309+0000","last_undegraded":"2023-11-06T21:02:12.015309+0000","last_fullsized":"2023-11-06T21:02:12.015309+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.026589+0000","last_change":"2023-11-06T21:02:12.023044+0000","last_active":"2023-11-06T21:02:12.026589+0000","last_peered":"2023-11-06T21:02:12.026589+0000","last_clean":"2023-11-06T21:02:12.026589+0000","last_became_active":"2023-11-06T21:02:12.022833+0000","last_became_peered":"2023-11-06T21:02:12.022833+0000","last_unstale":"2023-11-06T21:02:12.026589+0000","last_undegraded":"2023-11-06T21:02:12.026589+0000","last_fullsized":"2023-11-06T21:02:12.026589+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.001267+0000","last_change":"2023-11-06T21:02:11.529293+0000","last_active":"2023-11-06T21:02:12.001267+0000","last_peered":"2023-11-06T21:02:12.001267+0000","last_clean":"2023-11-06T21:02:12.001267+0000","last_became_active":"2023-11-06T21:02:11.528969+0000","last_became_peered":"2023-11-06T21:02:11.528969+0000","last_unstale":"2023-11-06T21:02:12.001267+0000","last_undegraded":"2023-11-06T21:02:12.001267+0000","last_fullsized":"2023-11-06T21:02:12.001267+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.028394+0000","last_change":"2023-11-06T21:02:12.028394+0000","last_active":"2023-11-06T21:02:12.028394+0000","last_peered":"2023-11-06T21:02:12.028394+0000","last_clean":"2023-11-06T21:02:12.028394+0000","last_became_active":"2023-11-06T21:02:12.027216+0000","last_became_peered":"2023-11-06T21:02:12.027216+0000","last_unstale":"2023-11-06T21:02:12.028394+0000","last_undegraded":"2023-11-06T21:02:12.028394+0000","last_fullsized":"2023-11-06T21:02:12.028394+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.539228+0000","last_change":"2023-11-06T21:02:11.534000+0000","last_active":"2023-11-06T21:02:11.539228+0000","last_peered":"2023-11-06T21:02:11.539228+0000","last_clean":"2023-11-06T21:02:11.539228+0000","last_became_active":"2023-11-06T21:02:11.533617+0000","last_became_peered":"2023-11-06T21:02:11.533617+0000","last_unstale":"2023-11-06T21:02:11.539228+0000","last_undegraded":"2023-11-06T21:02:11.539228+0000","last_fullsized":"2023-11-06T21:02:11.539228+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.031893+0000","last_change":"2023-11-06T21:02:12.031893+0000","last_active":"2023-11-06T21:02:12.031893+0000","last_peered":"2023-11-06T21:02:12.031893+0000","last_clean":"2023-11-06T21:02:12.031893+0000","last_became_active":"2023-11-06T21:02:12.031566+0000","last_became_peered":"2023-11-06T21:02:12.031566+0000","last_unstale":"2023-11-06T21:02:12.031893+0000","last_undegraded":"2023-11-06T21:02:12.031893+0000","last_fullsized":"2023-11-06T21:02:12.031893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2d","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.015230+0000","last_change":"2023-11-06T21:02:11.533505+0000","last_active":"2023-11-06T21:02:12.015230+0000","last_peered":"2023-11-06T21:02:12.015230+0000","last_clean":"2023-11-06T21:02:12.015230+0000","last_became_active":"2023-11-06T21:02:11.533239+0000","last_became_peered":"2023-11-06T21:02:11.533239+0000","last_unstale":"2023-11-06T21:02:12.015230+0000","last_undegraded":"2023-11-06T21:02:12.015230+0000","last_fullsized":"2023-11-06T21:02:12.015230+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.030130+0000","last_change":"2023-11-06T21:02:12.030130+0000","last_active":"2023-11-06T21:02:12.030130+0000","last_peered":"2023-11-06T21:02:12.030130+0000","last_clean":"2023-11-06T21:02:12.030130+0000","last_became_active":"2023-11-06T21:02:12.029573+0000","last_became_peered":"2023-11-06T21:02:12.029573+0000","last_unstale":"2023-11-06T21:02:12.030130+0000","last_undegraded":"2023-11-06T21:02:12.030130+0000","last_fullsized":"2023-11-06T21:02:12.030130+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.005813+0000","last_change":"2023-11-06T21:02:11.527607+0000","last_active":"2023-11-06T21:02:12.005813+0000","last_peered":"2023-11-06T21:02:12.005813+0000","last_clean":"2023-11-06T21:02:12.005813+0000","last_became_active":"2023-11-06T21:02:11.527149+0000","last_became_peered":"2023-11-06T21:02:11.527149+0000","last_unstale":"2023-11-06T21:02:12.005813+0000","last_undegraded":"2023-11-06T21:02:12.005813+0000","last_fullsized":"2023-11-06T21:02:12.005813+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.037845+0000","last_change":"2023-11-06T21:02:12.037845+0000","last_active":"2023-11-06T21:02:12.037845+0000","last_peered":"2023-11-06T21:02:12.037845+0000","last_clean":"2023-11-06T21:02:12.037845+0000","last_became_active":"2023-11-06T21:02:12.037570+0000","last_became_peered":"2023-11-06T21:02:12.037570+0000","last_unstale":"2023-11-06T21:02:12.037845+0000","last_undegraded":"2023-11-06T21:02:12.037845+0000","last_fullsized":"2023-11-06T21:02:12.037845+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2b","version":"130'1","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.010266+0000","last_change":"2023-11-06T21:02:11.538265+0000","last_active":"2023-11-06T21:02:12.010266+0000","last_peered":"2023-11-06T21:02:12.010266+0000","last_clean":"2023-11-06T21:02:12.010266+0000","last_became_active":"2023-11-06T21:02:11.537809+0000","last_became_peered":"2023-11-06T21:02:11.537809+0000","last_unstale":"2023-11-06T21:02:12.010266+0000","last_undegraded":"2023-11-06T21:02:12.010266+0000","last_fullsized":"2023-11-06T21:02:12.010266+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.014645+0000","last_change":"2023-11-06T21:02:12.014645+0000","last_active":"2023-11-06T21:02:12.014645+0000","last_peered":"2023-11-06T21:02:12.014645+0000","last_clean":"2023-11-06T21:02:12.014645+0000","last_became_active":"2023-11-06T21:02:12.014391+0000","last_became_peered":"2023-11-06T21:02:12.014391+0000","last_unstale":"2023-11-06T21:02:12.014645+0000","last_undegraded":"2023-11-06T21:02:12.014645+0000","last_fullsized":"2023-11-06T21:02:12.014645+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.540384+0000","last_change":"2023-11-06T21:02:11.538253+0000","last_active":"2023-11-06T21:02:11.540384+0000","last_peered":"2023-11-06T21:02:11.540384+0000","last_clean":"2023-11-06T21:02:11.540384+0000","last_became_active":"2023-11-06T21:02:11.537682+0000","last_became_peered":"2023-11-06T21:02:11.537682+0000","last_unstale":"2023-11-06T21:02:11.540384+0000","last_undegraded":"2023-11-06T21:02:11.540384+0000","last_fullsized":"2023-11-06T21:02:11.540384+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.009419+0000","last_change":"2023-11-06T21:02:12.009419+0000","last_active":"2023-11-06T21:02:12.009419+0000","last_peered":"2023-11-06T21:02:12.009419+0000","last_clean":"2023-11-06T21:02:12.009419+0000","last_became_active":"2023-11-06T21:02:12.009140+0000","last_became_peered":"2023-11-06T21:02:12.009140+0000","last_unstale":"2023-11-06T21:02:12.009419+0000","last_undegraded":"2023-11-06T21:02:12.009419+0000","last_fullsized":"2023-11-06T21:02:12.009419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.526336+0000","last_change":"2023-11-06T21:02:11.516613+0000","last_active":"2023-11-06T21:02:11.526336+0000","last_peered":"2023-11-06T21:02:11.526336+0000","last_clean":"2023-11-06T21:02:11.526336+0000","last_became_active":"2023-11-06T21:02:11.515887+0000","last_became_peered":"2023-11-06T21:02:11.515887+0000","last_unstale":"2023-11-06T21:02:11.526336+0000","last_undegraded":"2023-11-06T21:02:11.526336+0000","last_fullsized":"2023-11-06T21:02:11.526336+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.023525+0000","last_change":"2023-11-06T21:02:12.023525+0000","last_active":"2023-11-06T21:02:12.023525+0000","last_peered":"2023-11-06T21:02:12.023525+0000","last_clean":"2023-11-06T21:02:12.023525+0000","last_became_active":"2023-11-06T21:02:12.023224+0000","last_became_peered":"2023-11-06T21:02:12.023224+0000","last_unstale":"2023-11-06T21:02:12.023525+0000","last_undegraded":"2023-11-06T21:02:12.023525+0000","last_fullsized":"2023-11-06T21:02:12.023525+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.536585+0000","last_change":"2023-11-06T21:02:11.532170+0000","last_active":"2023-11-06T21:02:11.536585+0000","last_peered":"2023-11-06T21:02:11.536585+0000","last_clean":"2023-11-06T21:02:11.536585+0000","last_became_active":"2023-11-06T21:02:11.531800+0000","last_became_peered":"2023-11-06T21:02:11.531800+0000","last_unstale":"2023-11-06T21:02:11.536585+0000","last_undegraded":"2023-11-06T21:02:11.536585+0000","last_fullsized":"2023-11-06T21:02:11.536585+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.034523+0000","last_change":"2023-11-06T21:02:12.034523+0000","last_active":"2023-11-06T21:02:12.034523+0000","last_peered":"2023-11-06T21:02:12.034523+0000","last_clean":"2023-11-06T21:02:12.034523+0000","last_became_active":"2023-11-06T21:02:12.034127+0000","last_became_peered":"2023-11-06T21:02:12.034127+0000","last_unstale":"2023-11-06T21:02:12.034523+0000","last_undegraded":"2023-11-06T21:02:12.034523+0000","last_fullsized":"2023-11-06T21:02:12.034523+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.543347+0000","last_change":"2023-11-06T21:02:11.539499+0000","last_active":"2023-11-06T21:02:11.543347+0000","last_peered":"2023-11-06T21:02:11.543347+0000","last_clean":"2023-11-06T21:02:11.543347+0000","last_became_active":"2023-11-06T21:02:11.539158+0000","last_became_peered":"2023-11-06T21:02:11.539158+0000","last_unstale":"2023-11-06T21:02:11.543347+0000","last_undegraded":"2023-11-06T21:02:11.543347+0000","last_fullsized":"2023-11-06T21:02:11.543347+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.039351+0000","last_change":"2023-11-06T21:02:12.039351+0000","last_active":"2023-11-06T21:02:12.039351+0000","last_peered":"2023-11-06T21:02:12.039351+0000","last_clean":"2023-11-06T21:02:12.039351+0000","last_became_active":"2023-11-06T21:02:12.038547+0000","last_became_peered":"2023-11-06T21:02:12.038547+0000","last_unstale":"2023-11-06T21:02:12.039351+0000","last_undegraded":"2023-11-06T21:02:12.039351+0000","last_fullsized":"2023-11-06T21:02:12.039351+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.006096+0000","last_change":"2023-11-06T21:02:11.530207+0000","last_active":"2023-11-06T21:02:12.006096+0000","last_peered":"2023-11-06T21:02:12.006096+0000","last_clean":"2023-11-06T21:02:12.006096+0000","last_became_active":"2023-11-06T21:02:11.529882+0000","last_became_peered":"2023-11-06T21:02:11.529882+0000","last_unstale":"2023-11-06T21:02:12.006096+0000","last_undegraded":"2023-11-06T21:02:12.006096+0000","last_fullsized":"2023-11-06T21:02:12.006096+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.034792+0000","last_change":"2023-11-06T21:02:12.034792+0000","last_active":"2023-11-06T21:02:12.034792+0000","last_peered":"2023-11-06T21:02:12.034792+0000","last_clean":"2023-11-06T21:02:12.034792+0000","last_became_active":"2023-11-06T21:02:12.034090+0000","last_became_peered":"2023-11-06T21:02:12.034090+0000","last_unstale":"2023-11-06T21:02:12.034792+0000","last_undegraded":"2023-11-06T21:02:12.034792+0000","last_fullsized":"2023-11-06T21:02:12.034792+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.528865+0000","last_change":"2023-11-06T21:02:11.523951+0000","last_active":"2023-11-06T21:02:11.528865+0000","last_peered":"2023-11-06T21:02:11.528865+0000","last_clean":"2023-11-06T21:02:11.528865+0000","last_became_active":"2023-11-06T21:02:11.523622+0000","last_became_peered":"2023-11-06T21:02:11.523622+0000","last_unstale":"2023-11-06T21:02:11.528865+0000","last_undegraded":"2023-11-06T21:02:11.528865+0000","last_fullsized":"2023-11-06T21:02:11.528865+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.021177+0000","last_change":"2023-11-06T21:02:12.021177+0000","last_active":"2023-11-06T21:02:12.021177+0000","last_peered":"2023-11-06T21:02:12.021177+0000","last_clean":"2023-11-06T21:02:12.021177+0000","last_became_active":"2023-11-06T21:02:12.020435+0000","last_became_peered":"2023-11-06T21:02:12.020435+0000","last_unstale":"2023-11-06T21:02:12.021177+0000","last_undegraded":"2023-11-06T21:02:12.021177+0000","last_fullsized":"2023-11-06T21:02:12.021177+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.536678+0000","last_change":"2023-11-06T21:02:11.532667+0000","last_active":"2023-11-06T21:02:11.536678+0000","last_peered":"2023-11-06T21:02:11.536678+0000","last_clean":"2023-11-06T21:02:11.536678+0000","last_became_active":"2023-11-06T21:02:11.532393+0000","last_became_peered":"2023-11-06T21:02:11.532393+0000","last_unstale":"2023-11-06T21:02:11.536678+0000","last_undegraded":"2023-11-06T21:02:11.536678+0000","last_fullsized":"2023-11-06T21:02:11.536678+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.018844+0000","last_change":"2023-11-06T21:02:12.018844+0000","last_active":"2023-11-06T21:02:12.018844+0000","last_peered":"2023-11-06T21:02:12.018844+0000","last_clean":"2023-11-06T21:02:12.018844+0000","last_became_active":"2023-11-06T21:02:12.018553+0000","last_became_peered":"2023-11-06T21:02:12.018553+0000","last_unstale":"2023-11-06T21:02:12.018844+0000","last_undegraded":"2023-11-06T21:02:12.018844+0000","last_fullsized":"2023-11-06T21:02:12.018844+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.005665+0000","last_change":"2023-11-06T21:02:11.531958+0000","last_active":"2023-11-06T21:02:12.005665+0000","last_peered":"2023-11-06T21:02:12.005665+0000","last_clean":"2023-11-06T21:02:12.005665+0000","last_became_active":"2023-11-06T21:02:11.530284+0000","last_became_peered":"2023-11-06T21:02:11.530284+0000","last_unstale":"2023-11-06T21:02:12.005665+0000","last_undegraded":"2023-11-06T21:02:12.005665+0000","last_fullsized":"2023-11-06T21:02:12.005665+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038190+0000","last_change":"2023-11-06T21:02:12.036172+0000","last_active":"2023-11-06T21:02:12.038190+0000","last_peered":"2023-11-06T21:02:12.038190+0000","last_clean":"2023-11-06T21:02:12.038190+0000","last_became_active":"2023-11-06T21:02:12.035812+0000","last_became_peered":"2023-11-06T21:02:12.035812+0000","last_unstale":"2023-11-06T21:02:12.038190+0000","last_undegraded":"2023-11-06T21:02:12.038190+0000","last_fullsized":"2023-11-06T21:02:12.038190+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.005386+0000","last_change":"2023-11-06T21:02:11.536801+0000","last_active":"2023-11-06T21:02:12.005386+0000","last_peered":"2023-11-06T21:02:12.005386+0000","last_clean":"2023-11-06T21:02:12.005386+0000","last_became_active":"2023-11-06T21:02:11.535891+0000","last_became_peered":"2023-11-06T21:02:11.535891+0000","last_unstale":"2023-11-06T21:02:12.005386+0000","last_undegraded":"2023-11-06T21:02:12.005386+0000","last_fullsized":"2023-11-06T21:02:12.005386+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.036551+0000","last_change":"2023-11-06T21:02:12.036551+0000","last_active":"2023-11-06T21:02:12.036551+0000","last_peered":"2023-11-06T21:02:12.036551+0000","last_clean":"2023-11-06T21:02:12.036551+0000","last_became_active":"2023-11-06T21:02:12.035367+0000","last_became_peered":"2023-11-06T21:02:12.035367+0000","last_unstale":"2023-11-06T21:02:12.036551+0000","last_undegraded":"2023-11-06T21:02:12.036551+0000","last_fullsized":"2023-11-06T21:02:12.036551+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.538331+0000","last_change":"2023-11-06T21:02:11.528800+0000","last_active":"2023-11-06T21:02:11.538331+0000","last_peered":"2023-11-06T21:02:11.538331+0000","last_clean":"2023-11-06T21:02:11.538331+0000","last_became_active":"2023-11-06T21:02:11.528497+0000","last_became_peered":"2023-11-06T21:02:11.528497+0000","last_unstale":"2023-11-06T21:02:11.538331+0000","last_undegraded":"2023-11-06T21:02:11.538331+0000","last_fullsized":"2023-11-06T21:02:11.538331+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.018260+0000","last_change":"2023-11-06T21:02:12.018260+0000","last_active":"2023-11-06T21:02:12.018260+0000","last_peered":"2023-11-06T21:02:12.018260+0000","last_clean":"2023-11-06T21:02:12.018260+0000","last_became_active":"2023-11-06T21:02:12.017954+0000","last_became_peered":"2023-11-06T21:02:12.017954+0000","last_unstale":"2023-11-06T21:02:12.018260+0000","last_undegraded":"2023-11-06T21:02:12.018260+0000","last_fullsized":"2023-11-06T21:02:12.018260+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.534137+0000","last_change":"2023-11-06T21:02:11.532852+0000","last_active":"2023-11-06T21:02:11.534137+0000","last_peered":"2023-11-06T21:02:11.534137+0000","last_clean":"2023-11-06T21:02:11.534137+0000","last_became_active":"2023-11-06T21:02:11.532562+0000","last_became_peered":"2023-11-06T21:02:11.532562+0000","last_unstale":"2023-11-06T21:02:11.534137+0000","last_undegraded":"2023-11-06T21:02:11.534137+0000","last_fullsized":"2023-11-06T21:02:11.534137+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.037725+0000","last_change":"2023-11-06T21:02:12.032349+0000","last_active":"2023-11-06T21:02:12.037725+0000","last_peered":"2023-11-06T21:02:12.037725+0000","last_clean":"2023-11-06T21:02:12.037725+0000","last_became_active":"2023-11-06T21:02:12.031899+0000","last_became_peered":"2023-11-06T21:02:12.031899+0000","last_unstale":"2023-11-06T21:02:12.037725+0000","last_undegraded":"2023-11-06T21:02:12.037725+0000","last_fullsized":"2023-11-06T21:02:12.037725+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.021300+0000","last_change":"2023-11-06T21:02:11.530621+0000","last_active":"2023-11-06T21:02:12.021300+0000","last_peered":"2023-11-06T21:02:12.021300+0000","last_clean":"2023-11-06T21:02:12.021300+0000","last_became_active":"2023-11-06T21:02:11.530132+0000","last_became_peered":"2023-11-06T21:02:11.530132+0000","last_unstale":"2023-11-06T21:02:12.021300+0000","last_undegraded":"2023-11-06T21:02:12.021300+0000","last_fullsized":"2023-11-06T21:02:12.021300+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.022898+0000","last_change":"2023-11-06T21:02:12.022898+0000","last_active":"2023-11-06T21:02:12.022898+0000","last_peered":"2023-11-06T21:02:12.022898+0000","last_clean":"2023-11-06T21:02:12.022898+0000","last_became_active":"2023-11-06T21:02:12.022599+0000","last_became_peered":"2023-11-06T21:02:12.022599+0000","last_unstale":"2023-11-06T21:02:12.022898+0000","last_undegraded":"2023-11-06T21:02:12.022898+0000","last_fullsized":"2023-11-06T21:02:12.022898+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.026025+0000","last_change":"2023-11-06T21:02:11.537902+0000","last_active":"2023-11-06T21:02:12.026025+0000","last_peered":"2023-11-06T21:02:12.026025+0000","last_clean":"2023-11-06T21:02:12.026025+0000","last_became_active":"2023-11-06T21:02:11.537376+0000","last_became_peered":"2023-11-06T21:02:11.537376+0000","last_unstale":"2023-11-06T21:02:12.026025+0000","last_undegraded":"2023-11-06T21:02:12.026025+0000","last_fullsized":"2023-11-06T21:02:12.026025+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.016242+0000","last_change":"2023-11-06T21:02:12.016242+0000","last_active":"2023-11-06T21:02:12.016242+0000","last_peered":"2023-11-06T21:02:12.016242+0000","last_clean":"2023-11-06T21:02:12.016242+0000","last_became_active":"2023-11-06T21:02:12.015969+0000","last_became_peered":"2023-11-06T21:02:12.015969+0000","last_unstale":"2023-11-06T21:02:12.016242+0000","last_undegraded":"2023-11-06T21:02:12.016242+0000","last_fullsized":"2023-11-06T21:02:12.016242+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"130'2","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.021153+0000","last_change":"2023-11-06T21:02:11.527008+0000","last_active":"2023-11-06T21:02:12.021153+0000","last_peered":"2023-11-06T21:02:12.021153+0000","last_clean":"2023-11-06T21:02:12.021153+0000","last_became_active":"2023-11-06T21:02:11.526660+0000","last_became_peered":"2023-11-06T21:02:11.526660+0000","last_unstale":"2023-11-06T21:02:12.021153+0000","last_undegraded":"2023-11-06T21:02:12.021153+0000","last_fullsized":"2023-11-06T21:02:12.021153+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.035039+0000","last_change":"2023-11-06T21:02:12.035039+0000","last_active":"2023-11-06T21:02:12.035039+0000","last_peered":"2023-11-06T21:02:12.035039+0000","last_clean":"2023-11-06T21:02:12.035039+0000","last_became_active":"2023-11-06T21:02:12.034756+0000","last_became_peered":"2023-11-06T21:02:12.034756+0000","last_unstale":"2023-11-06T21:02:12.035039+0000","last_undegraded":"2023-11-06T21:02:12.035039+0000","last_fullsized":"2023-11-06T21:02:12.035039+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.538061+0000","last_change":"2023-11-06T21:02:11.533277+0000","last_active":"2023-11-06T21:02:11.538061+0000","last_peered":"2023-11-06T21:02:11.538061+0000","last_clean":"2023-11-06T21:02:11.538061+0000","last_became_active":"2023-11-06T21:02:11.530115+0000","last_became_peered":"2023-11-06T21:02:11.530115+0000","last_unstale":"2023-11-06T21:02:11.538061+0000","last_undegraded":"2023-11-06T21:02:11.538061+0000","last_fullsized":"2023-11-06T21:02:11.538061+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.020506+0000","last_change":"2023-11-06T21:02:12.020506+0000","last_active":"2023-11-06T21:02:12.020506+0000","last_peered":"2023-11-06T21:02:12.020506+0000","last_clean":"2023-11-06T21:02:12.020506+0000","last_became_active":"2023-11-06T21:02:12.019777+0000","last_became_peered":"2023-11-06T21:02:12.019777+0000","last_unstale":"2023-11-06T21:02:12.020506+0000","last_undegraded":"2023-11-06T21:02:12.020506+0000","last_fullsized":"2023-11-06T21:02:12.020506+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.537110+0000","last_change":"2023-11-06T21:02:11.535322+0000","last_active":"2023-11-06T21:02:11.537110+0000","last_peered":"2023-11-06T21:02:11.537110+0000","last_clean":"2023-11-06T21:02:11.537110+0000","last_became_active":"2023-11-06T21:02:11.534968+0000","last_became_peered":"2023-11-06T21:02:11.534968+0000","last_unstale":"2023-11-06T21:02:11.537110+0000","last_undegraded":"2023-11-06T21:02:11.537110+0000","last_fullsized":"2023-11-06T21:02:11.537110+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.029999+0000","last_change":"2023-11-06T21:02:12.029999+0000","last_active":"2023-11-06T21:02:12.029999+0000","last_peered":"2023-11-06T21:02:12.029999+0000","last_clean":"2023-11-06T21:02:12.029999+0000","last_became_active":"2023-11-06T21:02:12.029491+0000","last_became_peered":"2023-11-06T21:02:12.029491+0000","last_unstale":"2023-11-06T21:02:12.029999+0000","last_undegraded":"2023-11-06T21:02:12.029999+0000","last_fullsized":"2023-11-06T21:02:12.029999+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.008714+0000","last_change":"2023-11-06T21:02:11.530065+0000","last_active":"2023-11-06T21:02:12.008714+0000","last_peered":"2023-11-06T21:02:12.008714+0000","last_clean":"2023-11-06T21:02:12.008714+0000","last_became_active":"2023-11-06T21:02:11.529637+0000","last_became_peered":"2023-11-06T21:02:11.529637+0000","last_unstale":"2023-11-06T21:02:12.008714+0000","last_undegraded":"2023-11-06T21:02:12.008714+0000","last_fullsized":"2023-11-06T21:02:12.008714+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.007184+0000","last_change":"2023-11-06T21:02:12.007184+0000","last_active":"2023-11-06T21:02:12.007184+0000","last_peered":"2023-11-06T21:02:12.007184+0000","last_clean":"2023-11-06T21:02:12.007184+0000","last_became_active":"2023-11-06T21:02:12.006905+0000","last_became_peered":"2023-11-06T21:02:12.006905+0000","last_unstale":"2023-11-06T21:02:12.007184+0000","last_undegraded":"2023-11-06T21:02:12.007184+0000","last_fullsized":"2023-11-06T21:02:12.007184+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.011335+0000","last_change":"2023-11-06T21:02:11.530858+0000","last_active":"2023-11-06T21:02:12.011335+0000","last_peered":"2023-11-06T21:02:12.011335+0000","last_clean":"2023-11-06T21:02:12.011335+0000","last_became_active":"2023-11-06T21:02:11.529225+0000","last_became_peered":"2023-11-06T21:02:11.529225+0000","last_unstale":"2023-11-06T21:02:12.011335+0000","last_undegraded":"2023-11-06T21:02:12.011335+0000","last_fullsized":"2023-11-06T21:02:12.011335+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.024709+0000","last_change":"2023-11-06T21:02:12.024709+0000","last_active":"2023-11-06T21:02:12.024709+0000","last_peered":"2023-11-06T21:02:12.024709+0000","last_clean":"2023-11-06T21:02:12.024709+0000","last_became_active":"2023-11-06T21:02:12.023925+0000","last_became_peered":"2023-11-06T21:02:12.023925+0000","last_unstale":"2023-11-06T21:02:12.024709+0000","last_undegraded":"2023-11-06T21:02:12.024709+0000","last_fullsized":"2023-11-06T21:02:12.024709+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.005061+0000","last_change":"2023-11-06T21:02:11.525499+0000","last_active":"2023-11-06T21:02:12.005061+0000","last_peered":"2023-11-06T21:02:12.005061+0000","last_clean":"2023-11-06T21:02:12.005061+0000","last_became_active":"2023-11-06T21:02:11.525223+0000","last_became_peered":"2023-11-06T21:02:11.525223+0000","last_unstale":"2023-11-06T21:02:12.005061+0000","last_undegraded":"2023-11-06T21:02:12.005061+0000","last_fullsized":"2023-11-06T21:02:12.005061+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.030460+0000","last_change":"2023-11-06T21:02:12.030460+0000","last_active":"2023-11-06T21:02:12.030460+0000","last_peered":"2023-11-06T21:02:12.030460+0000","last_clean":"2023-11-06T21:02:12.030460+0000","last_became_active":"2023-11-06T21:02:12.029757+0000","last_became_peered":"2023-11-06T21:02:12.029757+0000","last_unstale":"2023-11-06T21:02:12.030460+0000","last_undegraded":"2023-11-06T21:02:12.030460+0000","last_fullsized":"2023-11-06T21:02:12.030460+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.536635+0000","last_change":"2023-11-06T21:02:11.532331+0000","last_active":"2023-11-06T21:02:11.536635+0000","last_peered":"2023-11-06T21:02:11.536635+0000","last_clean":"2023-11-06T21:02:11.536635+0000","last_became_active":"2023-11-06T21:02:11.531841+0000","last_became_peered":"2023-11-06T21:02:11.531841+0000","last_unstale":"2023-11-06T21:02:11.536635+0000","last_undegraded":"2023-11-06T21:02:11.536635+0000","last_fullsized":"2023-11-06T21:02:11.536635+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.019769+0000","last_change":"2023-11-06T21:02:12.019769+0000","last_active":"2023-11-06T21:02:12.019769+0000","last_peered":"2023-11-06T21:02:12.019769+0000","last_clean":"2023-11-06T21:02:12.019769+0000","last_became_active":"2023-11-06T21:02:12.018406+0000","last_became_peered":"2023-11-06T21:02:12.018406+0000","last_unstale":"2023-11-06T21:02:12.019769+0000","last_undegraded":"2023-11-06T21:02:12.019769+0000","last_fullsized":"2023-11-06T21:02:12.019769+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.540662+0000","last_change":"2023-11-06T21:02:11.536915+0000","last_active":"2023-11-06T21:02:11.540662+0000","last_peered":"2023-11-06T21:02:11.540662+0000","last_clean":"2023-11-06T21:02:11.540662+0000","last_became_active":"2023-11-06T21:02:11.536633+0000","last_became_peered":"2023-11-06T21:02:11.536633+0000","last_unstale":"2023-11-06T21:02:11.540662+0000","last_undegraded":"2023-11-06T21:02:11.540662+0000","last_fullsized":"2023-11-06T21:02:11.540662+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.019602+0000","last_change":"2023-11-06T21:02:12.019602+0000","last_active":"2023-11-06T21:02:12.019602+0000","last_peered":"2023-11-06T21:02:12.019602+0000","last_clean":"2023-11-06T21:02:12.019602+0000","last_became_active":"2023-11-06T21:02:12.019247+0000","last_became_peered":"2023-11-06T21:02:12.019247+0000","last_unstale":"2023-11-06T21:02:12.019602+0000","last_undegraded":"2023-11-06T21:02:12.019602+0000","last_fullsized":"2023-11-06T21:02:12.019602+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"130'3","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.033827+0000","last_change":"2023-11-06T21:02:11.538292+0000","last_active":"2023-11-06T21:02:12.033827+0000","last_peered":"2023-11-06T21:02:12.033827+0000","last_clean":"2023-11-06T21:02:12.033827+0000","last_became_active":"2023-11-06T21:02:11.537241+0000","last_became_peered":"2023-11-06T21:02:11.537241+0000","last_unstale":"2023-11-06T21:02:12.033827+0000","last_undegraded":"2023-11-06T21:02:12.033827+0000","last_fullsized":"2023-11-06T21:02:12.033827+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.032794+0000","last_change":"2023-11-06T21:02:12.027737+0000","last_active":"2023-11-06T21:02:12.032794+0000","last_peered":"2023-11-06T21:02:12.032794+0000","last_clean":"2023-11-06T21:02:12.032794+0000","last_became_active":"2023-11-06T21:02:12.027205+0000","last_became_peered":"2023-11-06T21:02:12.027205+0000","last_unstale":"2023-11-06T21:02:12.032794+0000","last_undegraded":"2023-11-06T21:02:12.032794+0000","last_fullsized":"2023-11-06T21:02:12.032794+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"130'2","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.016018+0000","last_change":"2023-11-06T21:02:11.519327+0000","last_active":"2023-11-06T21:02:12.016018+0000","last_peered":"2023-11-06T21:02:12.016018+0000","last_clean":"2023-11-06T21:02:12.016018+0000","last_became_active":"2023-11-06T21:02:11.518908+0000","last_became_peered":"2023-11-06T21:02:11.518908+0000","last_unstale":"2023-11-06T21:02:12.016018+0000","last_undegraded":"2023-11-06T21:02:12.016018+0000","last_fullsized":"2023-11-06T21:02:12.016018+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.022968+0000","last_change":"2023-11-06T21:02:12.022968+0000","last_active":"2023-11-06T21:02:12.022968+0000","last_peered":"2023-11-06T21:02:12.022968+0000","last_clean":"2023-11-06T21:02:12.022968+0000","last_became_active":"2023-11-06T21:02:12.022697+0000","last_became_peered":"2023-11-06T21:02:12.022697+0000","last_unstale":"2023-11-06T21:02:12.022968+0000","last_undegraded":"2023-11-06T21:02:12.022968+0000","last_fullsized":"2023-11-06T21:02:12.022968+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"130'1","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.010405+0000","last_change":"2023-11-06T21:02:11.537213+0000","last_active":"2023-11-06T21:02:12.010405+0000","last_peered":"2023-11-06T21:02:12.010405+0000","last_clean":"2023-11-06T21:02:12.010405+0000","last_became_active":"2023-11-06T21:02:11.536949+0000","last_became_peered":"2023-11-06T21:02:11.536949+0000","last_unstale":"2023-11-06T21:02:12.010405+0000","last_undegraded":"2023-11-06T21:02:12.010405+0000","last_fullsized":"2023-11-06T21:02:12.010405+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.016075+0000","last_change":"2023-11-06T21:02:12.016075+0000","last_active":"2023-11-06T21:02:12.016075+0000","last_peered":"2023-11-06T21:02:12.016075+0000","last_clean":"2023-11-06T21:02:12.016075+0000","last_became_active":"2023-11-06T21:02:12.015841+0000","last_became_peered":"2023-11-06T21:02:12.015841+0000","last_unstale":"2023-11-06T21:02:12.016075+0000","last_undegraded":"2023-11-06T21:02:12.016075+0000","last_fullsized":"2023-11-06T21:02:12.016075+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.532820+0000","last_change":"2023-11-06T21:02:11.527072+0000","last_active":"2023-11-06T21:02:11.532820+0000","last_peered":"2023-11-06T21:02:11.532820+0000","last_clean":"2023-11-06T21:02:11.532820+0000","last_became_active":"2023-11-06T21:02:11.526763+0000","last_became_peered":"2023-11-06T21:02:11.526763+0000","last_unstale":"2023-11-06T21:02:11.532820+0000","last_undegraded":"2023-11-06T21:02:11.532820+0000","last_fullsized":"2023-11-06T21:02:11.532820+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038104+0000","last_change":"2023-11-06T21:02:12.035523+0000","last_active":"2023-11-06T21:02:12.038104+0000","last_peered":"2023-11-06T21:02:12.038104+0000","last_clean":"2023-11-06T21:02:12.038104+0000","last_became_active":"2023-11-06T21:02:12.035218+0000","last_became_peered":"2023-11-06T21:02:12.035218+0000","last_unstale":"2023-11-06T21:02:12.038104+0000","last_undegraded":"2023-11-06T21:02:12.038104+0000","last_fullsized":"2023-11-06T21:02:12.038104+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.538351+0000","last_change":"2023-11-06T21:02:11.533087+0000","last_active":"2023-11-06T21:02:11.538351+0000","last_peered":"2023-11-06T21:02:11.538351+0000","last_clean":"2023-11-06T21:02:11.538351+0000","last_became_active":"2023-11-06T21:02:11.532327+0000","last_became_peered":"2023-11-06T21:02:11.532327+0000","last_unstale":"2023-11-06T21:02:11.538351+0000","last_undegraded":"2023-11-06T21:02:11.538351+0000","last_fullsized":"2023-11-06T21:02:11.538351+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038230+0000","last_change":"2023-11-06T21:02:12.035967+0000","last_active":"2023-11-06T21:02:12.038230+0000","last_peered":"2023-11-06T21:02:12.038230+0000","last_clean":"2023-11-06T21:02:12.038230+0000","last_became_active":"2023-11-06T21:02:12.035631+0000","last_became_peered":"2023-11-06T21:02:12.035631+0000","last_unstale":"2023-11-06T21:02:12.038230+0000","last_undegraded":"2023-11-06T21:02:12.038230+0000","last_fullsized":"2023-11-06T21:02:12.038230+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"130'4","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.029646+0000","last_change":"2023-11-06T21:02:11.521162+0000","last_active":"2023-11-06T21:02:12.029646+0000","last_peered":"2023-11-06T21:02:12.029646+0000","last_clean":"2023-11-06T21:02:12.029646+0000","last_became_active":"2023-11-06T21:02:11.520878+0000","last_became_peered":"2023-11-06T21:02:11.520878+0000","last_unstale":"2023-11-06T21:02:12.029646+0000","last_undegraded":"2023-11-06T21:02:12.029646+0000","last_fullsized":"2023-11-06T21:02:12.029646+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.021963+0000","last_change":"2023-11-06T21:02:12.016162+0000","last_active":"2023-11-06T21:02:12.021963+0000","last_peered":"2023-11-06T21:02:12.021963+0000","last_clean":"2023-11-06T21:02:12.021963+0000","last_became_active":"2023-11-06T21:02:12.015419+0000","last_became_peered":"2023-11-06T21:02:12.015419+0000","last_unstale":"2023-11-06T21:02:12.021963+0000","last_undegraded":"2023-11-06T21:02:12.021963+0000","last_fullsized":"2023-11-06T21:02:12.021963+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.013754+0000","last_change":"2023-11-06T21:02:11.531067+0000","last_active":"2023-11-06T21:02:12.013754+0000","last_peered":"2023-11-06T21:02:12.013754+0000","last_clean":"2023-11-06T21:02:12.013754+0000","last_became_active":"2023-11-06T21:02:11.530776+0000","last_became_peered":"2023-11-06T21:02:11.530776+0000","last_unstale":"2023-11-06T21:02:12.013754+0000","last_undegraded":"2023-11-06T21:02:12.013754+0000","last_fullsized":"2023-11-06T21:02:12.013754+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.009852+0000","last_change":"2023-11-06T21:02:12.009852+0000","last_active":"2023-11-06T21:02:12.009852+0000","last_peered":"2023-11-06T21:02:12.009852+0000","last_clean":"2023-11-06T21:02:12.009852+0000","last_became_active":"2023-11-06T21:02:12.009609+0000","last_became_peered":"2023-11-06T21:02:12.009609+0000","last_unstale":"2023-11-06T21:02:12.009852+0000","last_undegraded":"2023-11-06T21:02:12.009852+0000","last_fullsized":"2023-11-06T21:02:12.009852+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.001599+0000","last_change":"2023-11-06T21:02:11.534019+0000","last_active":"2023-11-06T21:02:12.001599+0000","last_peered":"2023-11-06T21:02:12.001599+0000","last_clean":"2023-11-06T21:02:12.001599+0000","last_became_active":"2023-11-06T21:02:11.533757+0000","last_became_peered":"2023-11-06T21:02:11.533757+0000","last_unstale":"2023-11-06T21:02:12.001599+0000","last_undegraded":"2023-11-06T21:02:12.001599+0000","last_fullsized":"2023-11-06T21:02:12.001599+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.034460+0000","last_change":"2023-11-06T21:02:12.034460+0000","last_active":"2023-11-06T21:02:12.034460+0000","last_peered":"2023-11-06T21:02:12.034460+0000","last_clean":"2023-11-06T21:02:12.034460+0000","last_became_active":"2023-11-06T21:02:12.034069+0000","last_became_peered":"2023-11-06T21:02:12.034069+0000","last_unstale":"2023-11-06T21:02:12.034460+0000","last_undegraded":"2023-11-06T21:02:12.034460+0000","last_fullsized":"2023-11-06T21:02:12.034460+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.536828+0000","last_change":"2023-11-06T21:02:11.531661+0000","last_active":"2023-11-06T21:02:11.536828+0000","last_peered":"2023-11-06T21:02:11.536828+0000","last_clean":"2023-11-06T21:02:11.536828+0000","last_became_active":"2023-11-06T21:02:11.531274+0000","last_became_peered":"2023-11-06T21:02:11.531274+0000","last_unstale":"2023-11-06T21:02:11.536828+0000","last_undegraded":"2023-11-06T21:02:11.536828+0000","last_fullsized":"2023-11-06T21:02:11.536828+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.030856+0000","last_change":"2023-11-06T21:02:12.026002+0000","last_active":"2023-11-06T21:02:12.030856+0000","last_peered":"2023-11-06T21:02:12.030856+0000","last_clean":"2023-11-06T21:02:12.030856+0000","last_became_active":"2023-11-06T21:02:12.025435+0000","last_became_peered":"2023-11-06T21:02:12.025435+0000","last_unstale":"2023-11-06T21:02:12.030856+0000","last_undegraded":"2023-11-06T21:02:12.030856+0000","last_fullsized":"2023-11-06T21:02:12.030856+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.542962+0000","last_change":"2023-11-06T21:02:11.541952+0000","last_active":"2023-11-06T21:02:11.542962+0000","last_peered":"2023-11-06T21:02:11.542962+0000","last_clean":"2023-11-06T21:02:11.542962+0000","last_became_active":"2023-11-06T21:02:11.541458+0000","last_became_peered":"2023-11-06T21:02:11.541458+0000","last_unstale":"2023-11-06T21:02:11.542962+0000","last_undegraded":"2023-11-06T21:02:11.542962+0000","last_fullsized":"2023-11-06T21:02:11.542962+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.025068+0000","last_change":"2023-11-06T21:02:12.025068+0000","last_active":"2023-11-06T21:02:12.025068+0000","last_peered":"2023-11-06T21:02:12.025068+0000","last_clean":"2023-11-06T21:02:12.025068+0000","last_became_active":"2023-11-06T21:02:12.023625+0000","last_became_peered":"2023-11-06T21:02:12.023625+0000","last_unstale":"2023-11-06T21:02:12.025068+0000","last_undegraded":"2023-11-06T21:02:12.025068+0000","last_fullsized":"2023-11-06T21:02:12.025068+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.543835+0000","last_change":"2023-11-06T21:02:11.541344+0000","last_active":"2023-11-06T21:02:11.543835+0000","last_peered":"2023-11-06T21:02:11.543835+0000","last_clean":"2023-11-06T21:02:11.543835+0000","last_became_active":"2023-11-06T21:02:11.541032+0000","last_became_peered":"2023-11-06T21:02:11.541032+0000","last_unstale":"2023-11-06T21:02:11.543835+0000","last_undegraded":"2023-11-06T21:02:11.543835+0000","last_fullsized":"2023-11-06T21:02:11.543835+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.024277+0000","last_change":"2023-11-06T21:02:12.024277+0000","last_active":"2023-11-06T21:02:12.024277+0000","last_peered":"2023-11-06T21:02:12.024277+0000","last_clean":"2023-11-06T21:02:12.024277+0000","last_became_active":"2023-11-06T21:02:12.023944+0000","last_became_peered":"2023-11-06T21:02:12.023944+0000","last_unstale":"2023-11-06T21:02:12.024277+0000","last_undegraded":"2023-11-06T21:02:12.024277+0000","last_fullsized":"2023-11-06T21:02:12.024277+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.999797+0000","last_change":"2023-11-06T21:02:11.529689+0000","last_active":"2023-11-06T21:02:11.999797+0000","last_peered":"2023-11-06T21:02:11.999797+0000","last_clean":"2023-11-06T21:02:11.999797+0000","last_became_active":"2023-11-06T21:02:11.529428+0000","last_became_peered":"2023-11-06T21:02:11.529428+0000","last_unstale":"2023-11-06T21:02:11.999797+0000","last_undegraded":"2023-11-06T21:02:11.999797+0000","last_fullsized":"2023-11-06T21:02:11.999797+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.034592+0000","last_change":"2023-11-06T21:02:12.034592+0000","last_active":"2023-11-06T21:02:12.034592+0000","last_peered":"2023-11-06T21:02:12.034592+0000","last_clean":"2023-11-06T21:02:12.034592+0000","last_became_active":"2023-11-06T21:02:12.034304+0000","last_became_peered":"2023-11-06T21:02:12.034304+0000","last_unstale":"2023-11-06T21:02:12.034592+0000","last_undegraded":"2023-11-06T21:02:12.034592+0000","last_fullsized":"2023-11-06T21:02:12.034592+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.011475+0000","last_change":"2023-11-06T21:02:11.520242+0000","last_active":"2023-11-06T21:02:12.011475+0000","last_peered":"2023-11-06T21:02:12.011475+0000","last_clean":"2023-11-06T21:02:12.011475+0000","last_became_active":"2023-11-06T21:02:11.519979+0000","last_became_peered":"2023-11-06T21:02:11.519979+0000","last_unstale":"2023-11-06T21:02:12.011475+0000","last_undegraded":"2023-11-06T21:02:12.011475+0000","last_fullsized":"2023-11-06T21:02:12.011475+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.036366+0000","last_change":"2023-11-06T21:02:12.036366+0000","last_active":"2023-11-06T21:02:12.036366+0000","last_peered":"2023-11-06T21:02:12.036366+0000","last_clean":"2023-11-06T21:02:12.036366+0000","last_became_active":"2023-11-06T21:02:12.036090+0000","last_became_peered":"2023-11-06T21:02:12.036090+0000","last_unstale":"2023-11-06T21:02:12.036366+0000","last_undegraded":"2023-11-06T21:02:12.036366+0000","last_fullsized":"2023-11-06T21:02:12.036366+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.000232+0000","last_change":"2023-11-06T21:02:11.527732+0000","last_active":"2023-11-06T21:02:12.000232+0000","last_peered":"2023-11-06T21:02:12.000232+0000","last_clean":"2023-11-06T21:02:12.000232+0000","last_became_active":"2023-11-06T21:02:11.527411+0000","last_became_peered":"2023-11-06T21:02:11.527411+0000","last_unstale":"2023-11-06T21:02:12.000232+0000","last_undegraded":"2023-11-06T21:02:12.000232+0000","last_fullsized":"2023-11-06T21:02:12.000232+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.032837+0000","last_change":"2023-11-06T21:02:12.026488+0000","last_active":"2023-11-06T21:02:12.032837+0000","last_peered":"2023-11-06T21:02:12.032837+0000","last_clean":"2023-11-06T21:02:12.032837+0000","last_became_active":"2023-11-06T21:02:12.026203+0000","last_became_peered":"2023-11-06T21:02:12.026203+0000","last_unstale":"2023-11-06T21:02:12.032837+0000","last_undegraded":"2023-11-06T21:02:12.032837+0000","last_fullsized":"2023-11-06T21:02:12.032837+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.532656+0000","last_change":"2023-11-06T21:02:11.526228+0000","last_active":"2023-11-06T21:02:11.532656+0000","last_peered":"2023-11-06T21:02:11.532656+0000","last_clean":"2023-11-06T21:02:11.532656+0000","last_became_active":"2023-11-06T21:02:11.525933+0000","last_became_peered":"2023-11-06T21:02:11.525933+0000","last_unstale":"2023-11-06T21:02:11.532656+0000","last_undegraded":"2023-11-06T21:02:11.532656+0000","last_fullsized":"2023-11-06T21:02:11.532656+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.999615+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T21:02:11.999615+0000","last_peered":"2023-11-06T21:02:11.999615+0000","last_clean":"2023-11-06T21:02:11.999615+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T21:02:11.999615+0000","last_undegraded":"2023-11-06T21:02:11.999615+0000","last_fullsized":"2023-11-06T21:02:11.999615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.022833+0000","last_change":"2023-11-06T21:02:12.022833+0000","last_active":"2023-11-06T21:02:12.022833+0000","last_peered":"2023-11-06T21:02:12.022833+0000","last_clean":"2023-11-06T21:02:12.022833+0000","last_became_active":"2023-11-06T21:02:12.022564+0000","last_became_peered":"2023-11-06T21:02:12.022564+0000","last_unstale":"2023-11-06T21:02:12.022833+0000","last_undegraded":"2023-11-06T21:02:12.022833+0000","last_fullsized":"2023-11-06T21:02:12.022833+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"130'1","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.020850+0000","last_change":"2023-11-06T21:02:11.541111+0000","last_active":"2023-11-06T21:02:12.020850+0000","last_peered":"2023-11-06T21:02:12.020850+0000","last_clean":"2023-11-06T21:02:12.020850+0000","last_became_active":"2023-11-06T21:02:11.540505+0000","last_became_peered":"2023-11-06T21:02:11.540505+0000","last_unstale":"2023-11-06T21:02:12.020850+0000","last_undegraded":"2023-11-06T21:02:12.020850+0000","last_fullsized":"2023-11-06T21:02:12.020850+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":116,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.513519+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T21:02:11.513519+0000","last_peered":"2023-11-06T21:02:11.513519+0000","last_clean":"2023-11-06T21:02:11.513519+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T21:02:11.513519+0000","last_undegraded":"2023-11-06T21:02:11.513519+0000","last_fullsized":"2023-11-06T21:02:11.513519+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.037362+0000","last_change":"2023-11-06T21:02:12.035276+0000","last_active":"2023-11-06T21:02:12.037362+0000","last_peered":"2023-11-06T21:02:12.037362+0000","last_clean":"2023-11-06T21:02:12.037362+0000","last_became_active":"2023-11-06T21:02:12.034464+0000","last_became_peered":"2023-11-06T21:02:12.034464+0000","last_unstale":"2023-11-06T21:02:12.037362+0000","last_undegraded":"2023-11-06T21:02:12.037362+0000","last_fullsized":"2023-11-06T21:02:12.037362+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.544261+0000","last_change":"2023-11-06T21:02:11.541489+0000","last_active":"2023-11-06T21:02:11.544261+0000","last_peered":"2023-11-06T21:02:11.544261+0000","last_clean":"2023-11-06T21:02:11.544261+0000","last_became_active":"2023-11-06T21:02:11.541189+0000","last_became_peered":"2023-11-06T21:02:11.541189+0000","last_unstale":"2023-11-06T21:02:11.544261+0000","last_undegraded":"2023-11-06T21:02:11.544261+0000","last_fullsized":"2023-11-06T21:02:11.544261+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.005163+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T21:02:12.005163+0000","last_peered":"2023-11-06T21:02:12.005163+0000","last_clean":"2023-11-06T21:02:12.005163+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T21:02:12.005163+0000","last_undegraded":"2023-11-06T21:02:12.005163+0000","last_fullsized":"2023-11-06T21:02:12.005163+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.039402+0000","last_change":"2023-11-06T21:02:12.039402+0000","last_active":"2023-11-06T21:02:12.039402+0000","last_peered":"2023-11-06T21:02:12.039402+0000","last_clean":"2023-11-06T21:02:12.039402+0000","last_became_active":"2023-11-06T21:02:12.039126+0000","last_became_peered":"2023-11-06T21:02:12.039126+0000","last_unstale":"2023-11-06T21:02:12.039402+0000","last_undegraded":"2023-11-06T21:02:12.039402+0000","last_fullsized":"2023-11-06T21:02:12.039402+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"130'2","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.028466+0000","last_change":"2023-11-06T21:02:11.538811+0000","last_active":"2023-11-06T21:02:12.028466+0000","last_peered":"2023-11-06T21:02:12.028466+0000","last_clean":"2023-11-06T21:02:12.028466+0000","last_became_active":"2023-11-06T21:02:11.538537+0000","last_became_peered":"2023-11-06T21:02:11.538537+0000","last_unstale":"2023-11-06T21:02:12.028466+0000","last_undegraded":"2023-11-06T21:02:12.028466+0000","last_fullsized":"2023-11-06T21:02:12.028466+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":116,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.509888+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T21:02:11.509888+0000","last_peered":"2023-11-06T21:02:11.509888+0000","last_clean":"2023-11-06T21:02:11.509888+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T21:02:11.509888+0000","last_undegraded":"2023-11-06T21:02:11.509888+0000","last_fullsized":"2023-11-06T21:02:11.509888+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":111,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.004844+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T21:02:12.004844+0000","last_peered":"2023-11-06T21:02:12.004844+0000","last_clean":"2023-11-06T21:02:12.004844+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T21:02:12.004844+0000","last_undegraded":"2023-11-06T21:02:12.004844+0000","last_fullsized":"2023-11-06T21:02:12.004844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.040755+0000","last_change":"2023-11-06T21:02:12.033454+0000","last_active":"2023-11-06T21:02:12.040755+0000","last_peered":"2023-11-06T21:02:12.040755+0000","last_clean":"2023-11-06T21:02:12.040755+0000","last_became_active":"2023-11-06T21:02:12.032937+0000","last_became_peered":"2023-11-06T21:02:12.032937+0000","last_unstale":"2023-11-06T21:02:12.040755+0000","last_undegraded":"2023-11-06T21:02:12.040755+0000","last_fullsized":"2023-11-06T21:02:12.040755+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.004505+0000","last_change":"2023-11-06T21:02:11.535993+0000","last_active":"2023-11-06T21:02:12.004505+0000","last_peered":"2023-11-06T21:02:12.004505+0000","last_clean":"2023-11-06T21:02:12.004505+0000","last_became_active":"2023-11-06T21:02:11.534095+0000","last_became_peered":"2023-11-06T21:02:11.534095+0000","last_unstale":"2023-11-06T21:02:12.004505+0000","last_undegraded":"2023-11-06T21:02:12.004505+0000","last_fullsized":"2023-11-06T21:02:12.004505+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":116,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.508951+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T21:02:11.508951+0000","last_peered":"2023-11-06T21:02:11.508951+0000","last_clean":"2023-11-06T21:02:11.508951+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T21:02:11.508951+0000","last_undegraded":"2023-11-06T21:02:11.508951+0000","last_fullsized":"2023-11-06T21:02:11.508951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.024897+0000","last_change":"2023-11-06T21:02:12.024897+0000","last_active":"2023-11-06T21:02:12.024897+0000","last_peered":"2023-11-06T21:02:12.024897+0000","last_clean":"2023-11-06T21:02:12.024897+0000","last_became_active":"2023-11-06T21:02:12.024348+0000","last_became_peered":"2023-11-06T21:02:12.024348+0000","last_unstale":"2023-11-06T21:02:12.024897+0000","last_undegraded":"2023-11-06T21:02:12.024897+0000","last_fullsized":"2023-11-06T21:02:12.024897+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.525396+0000","last_change":"2023-11-06T21:02:11.518624+0000","last_active":"2023-11-06T21:02:11.525396+0000","last_peered":"2023-11-06T21:02:11.525396+0000","last_clean":"2023-11-06T21:02:11.525396+0000","last_became_active":"2023-11-06T21:02:11.518352+0000","last_became_peered":"2023-11-06T21:02:11.518352+0000","last_unstale":"2023-11-06T21:02:11.525396+0000","last_undegraded":"2023-11-06T21:02:11.525396+0000","last_fullsized":"2023-11-06T21:02:11.525396+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":116,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.510715+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T21:02:11.510715+0000","last_peered":"2023-11-06T21:02:11.510715+0000","last_clean":"2023-11-06T21:02:11.510715+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T21:02:11.510715+0000","last_undegraded":"2023-11-06T21:02:11.510715+0000","last_fullsized":"2023-11-06T21:02:11.510715+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.026637+0000","last_change":"2023-11-06T21:02:12.026637+0000","last_active":"2023-11-06T21:02:12.026637+0000","last_peered":"2023-11-06T21:02:12.026637+0000","last_clean":"2023-11-06T21:02:12.026637+0000","last_became_active":"2023-11-06T21:02:12.024774+0000","last_became_peered":"2023-11-06T21:02:12.024774+0000","last_unstale":"2023-11-06T21:02:12.026637+0000","last_undegraded":"2023-11-06T21:02:12.026637+0000","last_fullsized":"2023-11-06T21:02:12.026637+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.013156+0000","last_change":"2023-11-06T21:02:11.541045+0000","last_active":"2023-11-06T21:02:12.013156+0000","last_peered":"2023-11-06T21:02:12.013156+0000","last_clean":"2023-11-06T21:02:12.013156+0000","last_became_active":"2023-11-06T21:02:11.540738+0000","last_became_peered":"2023-11-06T21:02:11.540738+0000","last_unstale":"2023-11-06T21:02:12.013156+0000","last_undegraded":"2023-11-06T21:02:12.013156+0000","last_fullsized":"2023-11-06T21:02:12.013156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.526226+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T21:02:11.526226+0000","last_peered":"2023-11-06T21:02:11.526226+0000","last_clean":"2023-11-06T21:02:11.526226+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T21:02:11.526226+0000","last_undegraded":"2023-11-06T21:02:11.526226+0000","last_fullsized":"2023-11-06T21:02:11.526226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.024370+0000","last_change":"2023-11-06T21:02:12.023205+0000","last_active":"2023-11-06T21:02:12.024370+0000","last_peered":"2023-11-06T21:02:12.024370+0000","last_clean":"2023-11-06T21:02:12.024370+0000","last_became_active":"2023-11-06T21:02:12.021768+0000","last_became_peered":"2023-11-06T21:02:12.021768+0000","last_unstale":"2023-11-06T21:02:12.024370+0000","last_undegraded":"2023-11-06T21:02:12.024370+0000","last_fullsized":"2023-11-06T21:02:12.024370+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.999691+0000","last_change":"2023-11-06T21:02:11.536063+0000","last_active":"2023-11-06T21:02:11.999691+0000","last_peered":"2023-11-06T21:02:11.999691+0000","last_clean":"2023-11-06T21:02:11.999691+0000","last_became_active":"2023-11-06T21:02:11.535724+0000","last_became_peered":"2023-11-06T21:02:11.535724+0000","last_unstale":"2023-11-06T21:02:11.999691+0000","last_undegraded":"2023-11-06T21:02:11.999691+0000","last_fullsized":"2023-11-06T21:02:11.999691+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":116,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.509814+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T21:02:11.509814+0000","last_peered":"2023-11-06T21:02:11.509814+0000","last_clean":"2023-11-06T21:02:11.509814+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T21:02:11.509814+0000","last_undegraded":"2023-11-06T21:02:11.509814+0000","last_fullsized":"2023-11-06T21:02:11.509814+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1617920,"data_stored":819537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66300000000000003,"15min":0.69799999999999995},"min":{"1min":0.52300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.69899999999999995},"min":{"1min":0.51600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91300000000000003,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.69099999999999995}]},{"osd":1,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65200000000000002,"15min":0.69499999999999995},"min":{"1min":0.51700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":0.90600000000000003,"15min":0.98599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68400000000000005,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0049999999999999,"5min":1.0600000000000001,"15min":1.111},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.72399999999999998},"min":{"1min":0.57999999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.875,"5min":1.0209999999999999,"15min":1.165},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66500000000000004,"15min":0.71699999999999997},"min":{"1min":0.49299999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95799999999999996,"5min":0.97599999999999998,"15min":1.024},"last":0.628}]},{"osd":3,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65900000000000003,"15min":0.69999999999999996},"min":{"1min":0.497,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.77800000000000002,"5min":0.89900000000000002,"15min":1.0129999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72299999999999998,"15min":0.747},"min":{"1min":0.54900000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.274},"last":0.61599999999999999}]},{"osd":4,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66700000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82999999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.65300000000000002,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.74199999999999999}]},{"osd":6,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89800000000000002,"5min":1.0680000000000001,"15min":1.1379999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.86799999999999999,"5min":0.96999999999999997,"15min":1.0549999999999999},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575063,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302780,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069060,"statfs":{"total":96636764160,"available":96326717440,"internally_reserved":0,"allocated":1626112,"data_stored":814993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.92200000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66400000000000003,"15min":0.68600000000000005},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93700000000000006,"5min":1.216,"15min":1.216},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65500000000000003,"15min":0.69199999999999995},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90100000000000002,"5min":0.91700000000000004,"15min":0.98399999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.69599999999999995},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.879,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.78000000000000003}]},{"osd":2,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72299999999999998,"15min":0.754},"min":{"1min":0.49299999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91000000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73299999999999998,"15min":0.76200000000000001},"min":{"1min":0.55300000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96399999999999997,"5min":1.05,"15min":1.05},"last":0.68300000000000005}]},{"osd":3,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67800000000000005,"15min":0.70999999999999996},"min":{"1min":0.443,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88500000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70399999999999996,"15min":0.72399999999999998},"min":{"1min":0.50800000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.95099999999999996,"5min":1.002,"15min":1.002},"last":0.58599999999999997}]},{"osd":4,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.60899999999999999},"min":{"1min":0.48799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80600000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.873,"5min":0.93300000000000005,"15min":1.012},"last":0.56000000000000005}]},{"osd":5,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.50700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":1.028,"15min":1.028},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.996,"5min":1.04,"15min":1.04},"last":0.748}]},{"osd":7,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.53400000000000003,"5min":0.45400000000000001,"15min":0.378},"max":{"1min":0.92900000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.56899999999999995,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575063,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302768,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069072,"statfs":{"total":96636764160,"available":96326729728,"internally_reserved":0,"allocated":1613824,"data_stored":813747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.35899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71399999999999997,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68999999999999995,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63400000000000001,"15min":0.61699999999999999},"min":{"1min":0.441,"5min":0.42899999999999999,"15min":0.41099999999999998},"max":{"1min":0.78500000000000003,"5min":4.431,"15min":4.431},"last":0.51700000000000002}]},{"osd":3,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61099999999999999,"15min":0.59499999999999997},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75900000000000001,"5min":4.5380000000000003,"15min":4.5380000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58499999999999996,"15min":0.56999999999999995},"min":{"1min":0.42299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71899999999999997,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.53300000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0389999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.82099999999999995},"min":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.95699999999999996,"5min":1.883,"15min":1.883},"last":0.746}]},{"osd":5,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.78100000000000003,"15min":0.78600000000000003},"min":{"1min":0.53500000000000003,"5min":0.504,"15min":0.504},"max":{"1min":0.91500000000000004,"5min":4.399,"15min":4.399},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.755,"15min":0.78400000000000003},"min":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.091,"5min":2.649,"15min":2.649},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.96899999999999997,"5min":3.024,"15min":3.024},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79900000000000004,"15min":0.82599999999999996},"min":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96699999999999997,"5min":3.0609999999999999,"15min":3.0609999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.57899999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0249999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.84799999999999998,"15min":0.84199999999999997},"min":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.032,"5min":5.8230000000000004,"15min":5.8230000000000004},"last":0.79200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575063,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302760,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069080,"statfs":{"total":96636764160,"available":96326737920,"internally_reserved":0,"allocated":1605632,"data_stored":812833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.45000000000000001,"15min":0.46899999999999997},"min":{"1min":0.34000000000000002,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":0.63900000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.34300000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.34499999999999997}]},{"osd":2,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.71299999999999997,"15min":0.82099999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.55000000000000004},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73799999999999999,"5min":0.77500000000000002,"15min":0.91000000000000003},"last":0.40100000000000002}]},{"osd":3,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53100000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.503},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.746,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.438}]},{"osd":4,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.60099999999999998,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":1.071,"5min":1.1299999999999999,"15min":1.1779999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.61899999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0149999999999999,"5min":1.298,"15min":1.298},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77400000000000002,"15min":0.80800000000000005},"min":{"1min":0.55500000000000005,"5min":0.51000000000000001,"15min":0.48999999999999999},"max":{"1min":0.97899999999999998,"5min":1.069,"15min":1.2010000000000001},"last":0.752},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.80400000000000005,"15min":0.82299999999999995},"min":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.51400000000000001},"max":{"1min":1.0580000000000001,"5min":1.2549999999999999,"15min":1.2829999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76800000000000002,"15min":0.79400000000000004},"min":{"1min":0.60999999999999999,"5min":0.56499999999999995,"15min":0.56299999999999994},"max":{"1min":1.0940000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78400000000000003,"15min":0.81200000000000006},"min":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.304},"last":0.71999999999999997}]},{"osd":7,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79000000000000004,"15min":0.82099999999999995},"min":{"1min":0.66100000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.1379999999999999,"5min":1.171,"15min":1.2490000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81699999999999995,"15min":0.83599999999999997},"min":{"1min":0.60099999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.097,"5min":1.1899999999999999,"15min":1.236},"last":0.79400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575063,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302768,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069072,"statfs":{"total":96636764160,"available":96326729728,"internally_reserved":0,"allocated":1613824,"data_stored":812945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.308,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.376},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.48499999999999999,"15min":0.49399999999999999},"min":{"1min":0.32900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.81699999999999995},"last":0.45500000000000002}]},{"osd":1,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85899999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84199999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.745,"5min":2.266,"15min":2.266},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.42699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78700000000000003,"5min":1.732,"15min":1.732},"last":0.42399999999999999}]},{"osd":4,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.63800000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.1890000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.752,"15min":0.77700000000000002},"min":{"1min":0.57599999999999996,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0149999999999999,"5min":1.087,"15min":1.357},"last":0.66400000000000003}]},{"osd":5,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.46300000000000002},"max":{"1min":1.1619999999999999,"5min":1.657,"15min":1.657},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79800000000000004,"15min":0.80900000000000005},"min":{"1min":0.59499999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0129999999999999,"5min":1.159,"15min":1.159},"last":0.61099999999999999}]},{"osd":6,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.78600000000000003,"15min":0.80700000000000005},"min":{"1min":0.64600000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.107,"5min":1.1499999999999999,"15min":1.2969999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80300000000000005,"15min":0.81399999999999995},"min":{"1min":0.59899999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.095,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.80800000000000005,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1819999999999999,"5min":1.1870000000000001,"15min":1.448},"last":0.623},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82599999999999996,"15min":0.83099999999999996},"min":{"1min":0.58799999999999997,"5min":0.54100000000000004,"15min":0.52100000000000002},"max":{"1min":1.093,"5min":2.202,"15min":2.202},"last":0.67600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575064,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1617920,"data_stored":814031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44400000000000001,"15min":0.45800000000000002},"min":{"1min":0.33400000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64800000000000002,"5min":0.71399999999999997,"15min":0.82599999999999996},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.32600000000000001},"max":{"1min":0.68400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48199999999999998,"15min":0.48799999999999999},"min":{"1min":0.33500000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.753},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.56200000000000006},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.79900000000000004,"15min":0.82999999999999996},"last":0.441}]},{"osd":2,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.46899999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73799999999999999,"5min":0.80100000000000005,"15min":0.91000000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1990000000000001,"5min":1.234,"15min":1.2470000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80000000000000004,"15min":0.81399999999999995},"min":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.47999999999999998},"max":{"1min":1.222,"5min":1.276,"15min":1.286},"last":0.92800000000000005}]},{"osd":5,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76300000000000001,"15min":0.78600000000000003},"min":{"1min":0.56299999999999994,"5min":0.496,"15min":0.496},"max":{"1min":0.94199999999999995,"5min":1.1519999999999999,"15min":1.1779999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77300000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.86199999999999999}]},{"osd":6,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.77000000000000002,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1539999999999999,"5min":1.323,"15min":1.323},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.79600000000000004,"15min":0.81799999999999995},"min":{"1min":0.66100000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.044,"5min":1.054,"15min":1.2130000000000001},"last":0.89100000000000001}]},{"osd":7,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78600000000000003,"15min":0.81599999999999995},"min":{"1min":0.58099999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.0780000000000001,"5min":1.0920000000000001,"15min":1.302},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.81200000000000006,"15min":0.82899999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.876}]}]},{"osd":4,"up_from":13,"seq":55834575063,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302764,"kb_used_data":1572,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069076,"statfs":{"total":96636764160,"available":96326733824,"internally_reserved":0,"allocated":1609728,"data_stored":812867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64000000000000001,"15min":0.68899999999999995},"min":{"1min":0.48399999999999999,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.63700000000000001,"15min":0.67400000000000004},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78500000000000003,"5min":1,"15min":1},"last":0.61799999999999999}]},{"osd":1,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65800000000000003,"15min":0.69999999999999996},"min":{"1min":0.47899999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80600000000000005,"5min":1.645,"15min":1.645},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68999999999999995,"15min":0.71299999999999997},"min":{"1min":0.45600000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83599999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73399999999999999}]},{"osd":2,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67600000000000005,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81599999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.748,"15min":0.77500000000000002},"min":{"1min":0.625,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.81499999999999995,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65400000000000003,"15min":0.69199999999999995},"min":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81100000000000005,"5min":0.90300000000000002,"15min":0.95799999999999996},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66900000000000004,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89300000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71399999999999997}]},{"osd":5,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77900000000000003,"5min":0.88200000000000001,"15min":1.0389999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.625,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.40799999999999997},"max":{"1min":0.80200000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.437}]},{"osd":6,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64000000000000001,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.93100000000000005,"5min":1.22,"15min":1.22},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70899999999999996,"15min":0.66900000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83699999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68100000000000005,"15min":0.66000000000000003},"min":{"1min":0.49399999999999999,"5min":0.45600000000000002,"15min":0.44},"max":{"1min":0.83799999999999997,"5min":1.034,"15min":1.034},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.67800000000000005},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.45400000000000001},"max":{"1min":0.86699999999999999,"5min":1.177,"15min":1.177},"last":0.56699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575064,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1617920,"data_stored":813027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.67200000000000004},"min":{"1min":0.45800000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.90300000000000002,"5min":0.94099999999999995,"15min":1.0109999999999999},"last":0.629},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64200000000000002,"15min":0.67700000000000005},"min":{"1min":0.45400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88600000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66200000000000003}]},{"osd":1,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90900000000000003,"5min":0.92300000000000004,"15min":0.99399999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72999999999999998,"15min":0.753},"min":{"1min":0.54500000000000004,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.034},"last":0.72999999999999998}]},{"osd":2,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69199999999999995,"15min":0.73699999999999999},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93899999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.748},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46000000000000002,"15min":0.45600000000000002},"max":{"1min":0.92800000000000005,"5min":0.98099999999999998,"15min":1.0089999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.70299999999999996},"min":{"1min":0.46800000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.95399999999999996,"5min":0.998,"15min":0.998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.54400000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.94199999999999995,"5min":1.175,"15min":1.175},"last":0.79300000000000004}]},{"osd":4,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80800000000000005,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.78700000000000003}]},{"osd":6,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64700000000000002,"15min":0.626},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0009999999999999,"5min":1.214,"15min":1.214},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70799999999999996,"15min":0.71199999999999997},"min":{"1min":0.48899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95499999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.64900000000000002},"min":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.40600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68400000000000005,"15min":0.68000000000000005},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97299999999999998,"5min":1.258,"15min":1.258},"last":0.70099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T21:02:16.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T21:02:16.481 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-11-06T21:02:16.480+0000 7fb66fdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T21:02:16.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.592+0000 7f530d9ff700 1 Processor -- start 2023-11-06T21:02:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.592+0000 7f530d9ff700 1 -- start start 2023-11-06T21:02:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.593+0000 7f530d9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5308151dc0 0x7f53080c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.593+0000 7f530d9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5308155320 0x7f53080c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.593+0000 7f530d9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5308154e90 con 0x7f53081521a0 2023-11-06T21:02:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.593+0000 7f530d9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53080639d0 con 0x7f5308155320 2023-11-06T21:02:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.593+0000 7f530d9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5308063cb0 con 0x7f5308151dc0 2023-11-06T21:02:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.593+0000 7f53077fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5308155320 0x7f53080c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.593+0000 7f53077fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5308155320 0x7f53080c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37142/0 (socket says 172.21.15.33:37142) 2023-11-06T21:02:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.593+0000 7f53077fe700 1 -- 172.21.15.33:0/84049341 learned_addr learned my addr 172.21.15.33:0/84049341 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.593+0000 7f5306ffd700 1 --2- 172.21.15.33:0/84049341 >> v2:172.21.15.69:3300/0 conn(0x7f5308151dc0 0x7f53080c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.594+0000 7f53077fe700 1 -- 172.21.15.33:0/84049341 >> v2:172.21.15.69:3300/0 conn(0x7f5308151dc0 msgr2=0x7f53080c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.594+0000 7f53077fe700 1 --2- 172.21.15.33:0/84049341 >> v2:172.21.15.69:3300/0 conn(0x7f5308151dc0 0x7f53080c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.594+0000 7f53077fe700 1 -- 172.21.15.33:0/84049341 >> v1:172.21.15.69:6789/0 conn(0x7f53081521a0 legacy=0x7f5308063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.594+0000 7f53077fe700 1 -- 172.21.15.33:0/84049341 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5308063f90 con 0x7f5308155320 2023-11-06T21:02:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.594+0000 7f53077fe700 1 --2- 172.21.15.33:0/84049341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5308155320 0x7f53080c7850 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7f53080651b0 tx=0x7f5300009840).ready entity=mon.0 client_cookie=2baa2b6cb36b0b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.594+0000 7f5305ffb700 1 -- 172.21.15.33:0/84049341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000e070 con 0x7f5308155320 2023-11-06T21:02:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.594+0000 7f5305ffb700 1 -- 172.21.15.33:0/84049341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530000ba50 con 0x7f5308155320 2023-11-06T21:02:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.594+0000 7f5305ffb700 1 -- 172.21.15.33:0/84049341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000bdd0 con 0x7f5308155320 2023-11-06T21:02:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.595+0000 7f530d9ff700 1 -- 172.21.15.33:0/84049341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5308155320 msgr2=0x7f53080c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.595+0000 7f530d9ff700 1 --2- 172.21.15.33:0/84049341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5308155320 0x7f53080c7850 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7f53080651b0 tx=0x7f5300009840).stop 2023-11-06T21:02:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.595+0000 7f530d9ff700 1 -- 172.21.15.33:0/84049341 shutdown_connections 2023-11-06T21:02:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.595+0000 7f530d9ff700 1 --2- 172.21.15.33:0/84049341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5308155320 0x7f53080c7850 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.595+0000 7f530d9ff700 1 --2- 172.21.15.33:0/84049341 >> v2:172.21.15.69:3300/0 conn(0x7f5308151dc0 0x7f53080c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.595+0000 7f530d9ff700 1 -- 172.21.15.33:0/84049341 >> 172.21.15.33:0/84049341 conn(0x7f530805e1b0 msgr2=0x7f530805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.595+0000 7f530d9ff700 1 -- 172.21.15.33:0/84049341 shutdown_connections 2023-11-06T21:02:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.595+0000 7f530d9ff700 1 -- 172.21.15.33:0/84049341 wait complete. 2023-11-06T21:02:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.596+0000 7f530d9ff700 1 Processor -- start 2023-11-06T21:02:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.597+0000 7f530d9ff700 1 -- start start 2023-11-06T21:02:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.597+0000 7f530d9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5308151dc0 0x7f530813a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f530d9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53081521a0 0x7f53080ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f530d9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5308155320 0x7f53080ab510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f530d9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080aba00 con 0x7f53081521a0 2023-11-06T21:02:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f530d9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080c5d30 con 0x7f5308155320 2023-11-06T21:02:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f530d9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530814fc40 con 0x7f5308151dc0 2023-11-06T21:02:16.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f53067fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53081521a0 0x7f53080ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f5306ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5308151dc0 0x7f530813a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f53067fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53081521a0 0x7f53080ab020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37154/0 (socket says 172.21.15.33:37154) 2023-11-06T21:02:16.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f53067fc700 1 -- 172.21.15.33:0/1783553878 learned_addr learned my addr 172.21.15.33:0/1783553878 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:16.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f53077fe700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5308155320 0x7f53080ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f53067fc700 1 -- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5308151dc0 msgr2=0x7f530813a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f53067fc700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5308151dc0 0x7f530813a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f53067fc700 1 -- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5308155320 msgr2=0x7f53080ab510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f53067fc700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5308155320 0x7f53080ab510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.598+0000 7f53067fc700 1 -- 172.21.15.33:0/1783553878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5300009ae0 con 0x7f53081521a0 2023-11-06T21:02:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.599+0000 7f53067fc700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53081521a0 0x7f53080ab020 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f52fc00c4c0 tx=0x7f52fc007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.599+0000 7f52eeffd700 1 -- 172.21.15.33:0/1783553878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00ce70 con 0x7f53081521a0 2023-11-06T21:02:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.599+0000 7f52eeffd700 1 -- 172.21.15.33:0/1783553878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc001e90 con 0x7f53081521a0 2023-11-06T21:02:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.599+0000 7f52eeffd700 1 -- 172.21.15.33:0/1783553878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc007250 con 0x7f53081521a0 2023-11-06T21:02:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.599+0000 7f530d9ff700 1 -- 172.21.15.33:0/1783553878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530814ff20 con 0x7f53081521a0 2023-11-06T21:02:16.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.599+0000 7f530d9ff700 1 -- 172.21.15.33:0/1783553878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53080c5ff0 con 0x7f53081521a0 2023-11-06T21:02:16.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.600+0000 7f530d9ff700 1 -- 172.21.15.33:0/1783553878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53080c6210 con 0x7f53081521a0 2023-11-06T21:02:16.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.601+0000 7f52eeffd700 1 -- 172.21.15.33:0/1783553878 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f52fc0066a0 con 0x7f53081521a0 2023-11-06T21:02:16.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.602+0000 7f52eeffd700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f52f405f270 0x7f52f4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:16.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.602+0000 7f52eeffd700 1 -- 172.21.15.33:0/1783553878 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f52fc079d00 con 0x7f53081521a0 2023-11-06T21:02:16.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.602+0000 7f5306ffd700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f52f405f270 0x7f52f4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:16.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.603+0000 7f5306ffd700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f52f405f270 0x7f52f4061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f52f800bbc0 tx=0x7f52f8008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:16.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.605+0000 7f52eeffd700 1 -- 172.21.15.33:0/1783553878 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f52fc04ad90 con 0x7f53081521a0 2023-11-06T21:02:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.783+0000 7f530d9ff700 1 -- 172.21.15.33:0/1783553878 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5308062ac0 con 0x7f52f405f270 2023-11-06T21:02:16.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.794+0000 7f52eeffd700 1 -- 172.21.15.33:0/1783553878 <== mgr.6377 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361873 (secure 0 0 0) 0x7f5308062ac0 con 0x7f52f405f270 2023-11-06T21:02:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 -- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f52f405f270 msgr2=0x7f52f4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f52f405f270 0x7f52f4061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f52f800bbc0 tx=0x7f52f8008040).stop 2023-11-06T21:02:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 -- 172.21.15.33:0/1783553878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53081521a0 msgr2=0x7f53080ab020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53081521a0 0x7f53080ab020 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f52fc00c4c0 tx=0x7f52fc007e00).stop 2023-11-06T21:02:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 -- 172.21.15.33:0/1783553878 shutdown_connections 2023-11-06T21:02:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f52f405f270 0x7f52f4061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5308151dc0 0x7f530813a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5308155320 0x7f53080ab510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 --2- 172.21.15.33:0/1783553878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53081521a0 0x7f53080ab020 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:16.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 -- 172.21.15.33:0/1783553878 >> 172.21.15.33:0/1783553878 conn(0x7f530805e1b0 msgr2=0x7f5308150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:16.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 -- 172.21.15.33:0/1783553878 shutdown_connections 2023-11-06T21:02:16.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:16.798+0000 7f530d9ff700 1 -- 172.21.15.33:0/1783553878 wait complete. 2023-11-06T21:02:16.804 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T21:02:16.816 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T21:02:15.445566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422156,"kb_used_data":12620,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752552564,"statfs":{"total":773094113280,"available":770613825536,"internally_reserved":0,"allocated":12922880,"data_stored":6513980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6815,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-61,"num_write_kb":-29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015470"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.021932+0000","last_change":"2023-11-06T21:02:12.013786+0000","last_active":"2023-11-06T21:02:12.021932+0000","last_peered":"2023-11-06T21:02:12.021932+0000","last_clean":"2023-11-06T21:02:12.021932+0000","last_became_active":"2023-11-06T21:02:12.013498+0000","last_became_peered":"2023-11-06T21:02:12.013498+0000","last_unstale":"2023-11-06T21:02:12.021932+0000","last_undegraded":"2023-11-06T21:02:12.021932+0000","last_fullsized":"2023-11-06T21:02:12.021932+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.3f","version":"130'3","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.026295+0000","last_change":"2023-11-06T21:02:11.527254+0000","last_active":"2023-11-06T21:02:12.026295+0000","last_peered":"2023-11-06T21:02:12.026295+0000","last_clean":"2023-11-06T21:02:12.026295+0000","last_became_active":"2023-11-06T21:02:11.526778+0000","last_became_peered":"2023-11-06T21:02:11.526778+0000","last_unstale":"2023-11-06T21:02:12.026295+0000","last_undegraded":"2023-11-06T21:02:12.026295+0000","last_fullsized":"2023-11-06T21:02:12.026295+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038118+0000","last_change":"2023-11-06T21:02:12.038118+0000","last_active":"2023-11-06T21:02:12.038118+0000","last_peered":"2023-11-06T21:02:12.038118+0000","last_clean":"2023-11-06T21:02:12.038118+0000","last_became_active":"2023-11-06T21:02:12.037856+0000","last_became_peered":"2023-11-06T21:02:12.037856+0000","last_unstale":"2023-11-06T21:02:12.038118+0000","last_undegraded":"2023-11-06T21:02:12.038118+0000","last_fullsized":"2023-11-06T21:02:12.038118+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.531440+0000","last_change":"2023-11-06T21:02:11.520733+0000","last_active":"2023-11-06T21:02:11.531440+0000","last_peered":"2023-11-06T21:02:11.531440+0000","last_clean":"2023-11-06T21:02:11.531440+0000","last_became_active":"2023-11-06T21:02:11.520446+0000","last_became_peered":"2023-11-06T21:02:11.520446+0000","last_unstale":"2023-11-06T21:02:11.531440+0000","last_undegraded":"2023-11-06T21:02:11.531440+0000","last_fullsized":"2023-11-06T21:02:11.531440+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.013962+0000","last_change":"2023-11-06T21:02:12.013962+0000","last_active":"2023-11-06T21:02:12.013962+0000","last_peered":"2023-11-06T21:02:12.013962+0000","last_clean":"2023-11-06T21:02:12.013962+0000","last_became_active":"2023-11-06T21:02:12.013683+0000","last_became_peered":"2023-11-06T21:02:12.013683+0000","last_unstale":"2023-11-06T21:02:12.013962+0000","last_undegraded":"2023-11-06T21:02:12.013962+0000","last_fullsized":"2023-11-06T21:02:12.013962+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.542922+0000","last_change":"2023-11-06T21:02:11.541728+0000","last_active":"2023-11-06T21:02:11.542922+0000","last_peered":"2023-11-06T21:02:11.542922+0000","last_clean":"2023-11-06T21:02:11.542922+0000","last_became_active":"2023-11-06T21:02:11.541443+0000","last_became_peered":"2023-11-06T21:02:11.541443+0000","last_unstale":"2023-11-06T21:02:11.542922+0000","last_undegraded":"2023-11-06T21:02:11.542922+0000","last_fullsized":"2023-11-06T21:02:11.542922+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.034854+0000","last_change":"2023-11-06T21:02:12.034854+0000","last_active":"2023-11-06T21:02:12.034854+0000","last_peered":"2023-11-06T21:02:12.034854+0000","last_clean":"2023-11-06T21:02:12.034854+0000","last_became_active":"2023-11-06T21:02:12.034572+0000","last_became_peered":"2023-11-06T21:02:12.034572+0000","last_unstale":"2023-11-06T21:02:12.034854+0000","last_undegraded":"2023-11-06T21:02:12.034854+0000","last_fullsized":"2023-11-06T21:02:12.034854+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3c","version":"130'2","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.027348+0000","last_change":"2023-11-06T21:02:11.535851+0000","last_active":"2023-11-06T21:02:12.027348+0000","last_peered":"2023-11-06T21:02:12.027348+0000","last_clean":"2023-11-06T21:02:12.027348+0000","last_became_active":"2023-11-06T21:02:11.535529+0000","last_became_peered":"2023-11-06T21:02:11.535529+0000","last_unstale":"2023-11-06T21:02:12.027348+0000","last_undegraded":"2023-11-06T21:02:12.027348+0000","last_fullsized":"2023-11-06T21:02:12.027348+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.035158+0000","last_change":"2023-11-06T21:02:12.035158+0000","last_active":"2023-11-06T21:02:12.035158+0000","last_peered":"2023-11-06T21:02:12.035158+0000","last_clean":"2023-11-06T21:02:12.035158+0000","last_became_active":"2023-11-06T21:02:12.034883+0000","last_became_peered":"2023-11-06T21:02:12.034883+0000","last_unstale":"2023-11-06T21:02:12.035158+0000","last_undegraded":"2023-11-06T21:02:12.035158+0000","last_fullsized":"2023-11-06T21:02:12.035158+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3b","version":"130'1","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.999531+0000","last_change":"2023-11-06T21:02:11.522959+0000","last_active":"2023-11-06T21:02:11.999531+0000","last_peered":"2023-11-06T21:02:11.999531+0000","last_clean":"2023-11-06T21:02:11.999531+0000","last_became_active":"2023-11-06T21:02:11.522509+0000","last_became_peered":"2023-11-06T21:02:11.522509+0000","last_unstale":"2023-11-06T21:02:11.999531+0000","last_undegraded":"2023-11-06T21:02:11.999531+0000","last_fullsized":"2023-11-06T21:02:11.999531+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.026628+0000","last_change":"2023-11-06T21:02:12.020160+0000","last_active":"2023-11-06T21:02:12.026628+0000","last_peered":"2023-11-06T21:02:12.026628+0000","last_clean":"2023-11-06T21:02:12.026628+0000","last_became_active":"2023-11-06T21:02:12.018793+0000","last_became_peered":"2023-11-06T21:02:12.018793+0000","last_unstale":"2023-11-06T21:02:12.026628+0000","last_undegraded":"2023-11-06T21:02:12.026628+0000","last_fullsized":"2023-11-06T21:02:12.026628+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3a","version":"130'1","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.999304+0000","last_change":"2023-11-06T21:02:11.542300+0000","last_active":"2023-11-06T21:02:11.999304+0000","last_peered":"2023-11-06T21:02:11.999304+0000","last_clean":"2023-11-06T21:02:11.999304+0000","last_became_active":"2023-11-06T21:02:11.541995+0000","last_became_peered":"2023-11-06T21:02:11.541995+0000","last_unstale":"2023-11-06T21:02:11.999304+0000","last_undegraded":"2023-11-06T21:02:11.999304+0000","last_fullsized":"2023-11-06T21:02:11.999304+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.032349+0000","last_change":"2023-11-06T21:02:12.025288+0000","last_active":"2023-11-06T21:02:12.032349+0000","last_peered":"2023-11-06T21:02:12.032349+0000","last_clean":"2023-11-06T21:02:12.032349+0000","last_became_active":"2023-11-06T21:02:12.024628+0000","last_became_peered":"2023-11-06T21:02:12.024628+0000","last_unstale":"2023-11-06T21:02:12.032349+0000","last_undegraded":"2023-11-06T21:02:12.032349+0000","last_fullsized":"2023-11-06T21:02:12.032349+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.533775+0000","last_change":"2023-11-06T21:02:11.530168+0000","last_active":"2023-11-06T21:02:11.533775+0000","last_peered":"2023-11-06T21:02:11.533775+0000","last_clean":"2023-11-06T21:02:11.533775+0000","last_became_active":"2023-11-06T21:02:11.529750+0000","last_became_peered":"2023-11-06T21:02:11.529750+0000","last_unstale":"2023-11-06T21:02:11.533775+0000","last_undegraded":"2023-11-06T21:02:11.533775+0000","last_fullsized":"2023-11-06T21:02:11.533775+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.032607+0000","last_change":"2023-11-06T21:02:12.028903+0000","last_active":"2023-11-06T21:02:12.032607+0000","last_peered":"2023-11-06T21:02:12.032607+0000","last_clean":"2023-11-06T21:02:12.032607+0000","last_became_active":"2023-11-06T21:02:12.025751+0000","last_became_peered":"2023-11-06T21:02:12.025751+0000","last_unstale":"2023-11-06T21:02:12.032607+0000","last_undegraded":"2023-11-06T21:02:12.032607+0000","last_fullsized":"2023-11-06T21:02:12.032607+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.538714+0000","last_change":"2023-11-06T21:02:11.535205+0000","last_active":"2023-11-06T21:02:11.538714+0000","last_peered":"2023-11-06T21:02:11.538714+0000","last_clean":"2023-11-06T21:02:11.538714+0000","last_became_active":"2023-11-06T21:02:11.534788+0000","last_became_peered":"2023-11-06T21:02:11.534788+0000","last_unstale":"2023-11-06T21:02:11.538714+0000","last_undegraded":"2023-11-06T21:02:11.538714+0000","last_fullsized":"2023-11-06T21:02:11.538714+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.022239+0000","last_change":"2023-11-06T21:02:12.022239+0000","last_active":"2023-11-06T21:02:12.022239+0000","last_peered":"2023-11-06T21:02:12.022239+0000","last_clean":"2023-11-06T21:02:12.022239+0000","last_became_active":"2023-11-06T21:02:12.020806+0000","last_became_peered":"2023-11-06T21:02:12.020806+0000","last_unstale":"2023-11-06T21:02:12.022239+0000","last_undegraded":"2023-11-06T21:02:12.022239+0000","last_fullsized":"2023-11-06T21:02:12.022239+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.542403+0000","last_change":"2023-11-06T21:02:11.537853+0000","last_active":"2023-11-06T21:02:11.542403+0000","last_peered":"2023-11-06T21:02:11.542403+0000","last_clean":"2023-11-06T21:02:11.542403+0000","last_became_active":"2023-11-06T21:02:11.537277+0000","last_became_peered":"2023-11-06T21:02:11.537277+0000","last_unstale":"2023-11-06T21:02:11.542403+0000","last_undegraded":"2023-11-06T21:02:11.542403+0000","last_fullsized":"2023-11-06T21:02:11.542403+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.030838+0000","last_change":"2023-11-06T21:02:12.030838+0000","last_active":"2023-11-06T21:02:12.030838+0000","last_peered":"2023-11-06T21:02:12.030838+0000","last_clean":"2023-11-06T21:02:12.030838+0000","last_became_active":"2023-11-06T21:02:12.030461+0000","last_became_peered":"2023-11-06T21:02:12.030461+0000","last_unstale":"2023-11-06T21:02:12.030838+0000","last_undegraded":"2023-11-06T21:02:12.030838+0000","last_fullsized":"2023-11-06T21:02:12.030838+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.533818+0000","last_change":"2023-11-06T21:02:11.527819+0000","last_active":"2023-11-06T21:02:11.533818+0000","last_peered":"2023-11-06T21:02:11.533818+0000","last_clean":"2023-11-06T21:02:11.533818+0000","last_became_active":"2023-11-06T21:02:11.527546+0000","last_became_peered":"2023-11-06T21:02:11.527546+0000","last_unstale":"2023-11-06T21:02:11.533818+0000","last_undegraded":"2023-11-06T21:02:11.533818+0000","last_fullsized":"2023-11-06T21:02:11.533818+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.037251+0000","last_change":"2023-11-06T21:02:12.034884+0000","last_active":"2023-11-06T21:02:12.037251+0000","last_peered":"2023-11-06T21:02:12.037251+0000","last_clean":"2023-11-06T21:02:12.037251+0000","last_became_active":"2023-11-06T21:02:12.034027+0000","last_became_peered":"2023-11-06T21:02:12.034027+0000","last_unstale":"2023-11-06T21:02:12.037251+0000","last_undegraded":"2023-11-06T21:02:12.037251+0000","last_fullsized":"2023-11-06T21:02:12.037251+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.538680+0000","last_change":"2023-11-06T21:02:11.536631+0000","last_active":"2023-11-06T21:02:11.538680+0000","last_peered":"2023-11-06T21:02:11.538680+0000","last_clean":"2023-11-06T21:02:11.538680+0000","last_became_active":"2023-11-06T21:02:11.536091+0000","last_became_peered":"2023-11-06T21:02:11.536091+0000","last_unstale":"2023-11-06T21:02:11.538680+0000","last_undegraded":"2023-11-06T21:02:11.538680+0000","last_fullsized":"2023-11-06T21:02:11.538680+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.016079+0000","last_change":"2023-11-06T21:02:12.014195+0000","last_active":"2023-11-06T21:02:12.016079+0000","last_peered":"2023-11-06T21:02:12.016079+0000","last_clean":"2023-11-06T21:02:12.016079+0000","last_became_active":"2023-11-06T21:02:12.012965+0000","last_became_peered":"2023-11-06T21:02:12.012965+0000","last_unstale":"2023-11-06T21:02:12.016079+0000","last_undegraded":"2023-11-06T21:02:12.016079+0000","last_fullsized":"2023-11-06T21:02:12.016079+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.34","version":"130'4","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.036697+0000","last_change":"2023-11-06T21:02:11.531141+0000","last_active":"2023-11-06T21:02:12.036697+0000","last_peered":"2023-11-06T21:02:12.036697+0000","last_clean":"2023-11-06T21:02:12.036697+0000","last_became_active":"2023-11-06T21:02:11.530885+0000","last_became_peered":"2023-11-06T21:02:11.530885+0000","last_unstale":"2023-11-06T21:02:12.036697+0000","last_undegraded":"2023-11-06T21:02:12.036697+0000","last_fullsized":"2023-11-06T21:02:12.036697+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.032775+0000","last_change":"2023-11-06T21:02:12.027108+0000","last_active":"2023-11-06T21:02:12.032775+0000","last_peered":"2023-11-06T21:02:12.032775+0000","last_clean":"2023-11-06T21:02:12.032775+0000","last_became_active":"2023-11-06T21:02:12.022110+0000","last_became_peered":"2023-11-06T21:02:12.022110+0000","last_unstale":"2023-11-06T21:02:12.032775+0000","last_undegraded":"2023-11-06T21:02:12.032775+0000","last_fullsized":"2023-11-06T21:02:12.032775+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.039387+0000","last_change":"2023-11-06T21:02:11.528106+0000","last_active":"2023-11-06T21:02:12.039387+0000","last_peered":"2023-11-06T21:02:12.039387+0000","last_clean":"2023-11-06T21:02:12.039387+0000","last_became_active":"2023-11-06T21:02:11.527781+0000","last_became_peered":"2023-11-06T21:02:11.527781+0000","last_unstale":"2023-11-06T21:02:12.039387+0000","last_undegraded":"2023-11-06T21:02:12.039387+0000","last_fullsized":"2023-11-06T21:02:12.039387+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038158+0000","last_change":"2023-11-06T21:02:12.038158+0000","last_active":"2023-11-06T21:02:12.038158+0000","last_peered":"2023-11-06T21:02:12.038158+0000","last_clean":"2023-11-06T21:02:12.038158+0000","last_became_active":"2023-11-06T21:02:12.037737+0000","last_became_peered":"2023-11-06T21:02:12.037737+0000","last_unstale":"2023-11-06T21:02:12.038158+0000","last_undegraded":"2023-11-06T21:02:12.038158+0000","last_fullsized":"2023-11-06T21:02:12.038158+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.530651+0000","last_change":"2023-11-06T21:02:11.519433+0000","last_active":"2023-11-06T21:02:11.530651+0000","last_peered":"2023-11-06T21:02:11.530651+0000","last_clean":"2023-11-06T21:02:11.530651+0000","last_became_active":"2023-11-06T21:02:11.517627+0000","last_became_peered":"2023-11-06T21:02:11.517627+0000","last_unstale":"2023-11-06T21:02:11.530651+0000","last_undegraded":"2023-11-06T21:02:11.530651+0000","last_fullsized":"2023-11-06T21:02:11.530651+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.022378+0000","last_change":"2023-11-06T21:02:12.022378+0000","last_active":"2023-11-06T21:02:12.022378+0000","last_peered":"2023-11-06T21:02:12.022378+0000","last_clean":"2023-11-06T21:02:12.022378+0000","last_became_active":"2023-11-06T21:02:12.022081+0000","last_became_peered":"2023-11-06T21:02:12.022081+0000","last_unstale":"2023-11-06T21:02:12.022378+0000","last_undegraded":"2023-11-06T21:02:12.022378+0000","last_fullsized":"2023-11-06T21:02:12.022378+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.532638+0000","last_change":"2023-11-06T21:02:11.528866+0000","last_active":"2023-11-06T21:02:11.532638+0000","last_peered":"2023-11-06T21:02:11.532638+0000","last_clean":"2023-11-06T21:02:11.532638+0000","last_became_active":"2023-11-06T21:02:11.528564+0000","last_became_peered":"2023-11-06T21:02:11.528564+0000","last_unstale":"2023-11-06T21:02:11.532638+0000","last_undegraded":"2023-11-06T21:02:11.532638+0000","last_fullsized":"2023-11-06T21:02:11.532638+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038100+0000","last_change":"2023-11-06T21:02:12.035815+0000","last_active":"2023-11-06T21:02:12.038100+0000","last_peered":"2023-11-06T21:02:12.038100+0000","last_clean":"2023-11-06T21:02:12.038100+0000","last_became_active":"2023-11-06T21:02:12.035518+0000","last_became_peered":"2023-11-06T21:02:12.035518+0000","last_unstale":"2023-11-06T21:02:12.038100+0000","last_undegraded":"2023-11-06T21:02:12.038100+0000","last_fullsized":"2023-11-06T21:02:12.038100+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.30","version":"130'4","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.015309+0000","last_change":"2023-11-06T21:02:11.537869+0000","last_active":"2023-11-06T21:02:12.015309+0000","last_peered":"2023-11-06T21:02:12.015309+0000","last_clean":"2023-11-06T21:02:12.015309+0000","last_became_active":"2023-11-06T21:02:11.537564+0000","last_became_peered":"2023-11-06T21:02:11.537564+0000","last_unstale":"2023-11-06T21:02:12.015309+0000","last_undegraded":"2023-11-06T21:02:12.015309+0000","last_fullsized":"2023-11-06T21:02:12.015309+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.026589+0000","last_change":"2023-11-06T21:02:12.023044+0000","last_active":"2023-11-06T21:02:12.026589+0000","last_peered":"2023-11-06T21:02:12.026589+0000","last_clean":"2023-11-06T21:02:12.026589+0000","last_became_active":"2023-11-06T21:02:12.022833+0000","last_became_peered":"2023-11-06T21:02:12.022833+0000","last_unstale":"2023-11-06T21:02:12.026589+0000","last_undegraded":"2023-11-06T21:02:12.026589+0000","last_fullsized":"2023-11-06T21:02:12.026589+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.001267+0000","last_change":"2023-11-06T21:02:11.529293+0000","last_active":"2023-11-06T21:02:12.001267+0000","last_peered":"2023-11-06T21:02:12.001267+0000","last_clean":"2023-11-06T21:02:12.001267+0000","last_became_active":"2023-11-06T21:02:11.528969+0000","last_became_peered":"2023-11-06T21:02:11.528969+0000","last_unstale":"2023-11-06T21:02:12.001267+0000","last_undegraded":"2023-11-06T21:02:12.001267+0000","last_fullsized":"2023-11-06T21:02:12.001267+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.028394+0000","last_change":"2023-11-06T21:02:12.028394+0000","last_active":"2023-11-06T21:02:12.028394+0000","last_peered":"2023-11-06T21:02:12.028394+0000","last_clean":"2023-11-06T21:02:12.028394+0000","last_became_active":"2023-11-06T21:02:12.027216+0000","last_became_peered":"2023-11-06T21:02:12.027216+0000","last_unstale":"2023-11-06T21:02:12.028394+0000","last_undegraded":"2023-11-06T21:02:12.028394+0000","last_fullsized":"2023-11-06T21:02:12.028394+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.539228+0000","last_change":"2023-11-06T21:02:11.534000+0000","last_active":"2023-11-06T21:02:11.539228+0000","last_peered":"2023-11-06T21:02:11.539228+0000","last_clean":"2023-11-06T21:02:11.539228+0000","last_became_active":"2023-11-06T21:02:11.533617+0000","last_became_peered":"2023-11-06T21:02:11.533617+0000","last_unstale":"2023-11-06T21:02:11.539228+0000","last_undegraded":"2023-11-06T21:02:11.539228+0000","last_fullsized":"2023-11-06T21:02:11.539228+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.031893+0000","last_change":"2023-11-06T21:02:12.031893+0000","last_active":"2023-11-06T21:02:12.031893+0000","last_peered":"2023-11-06T21:02:12.031893+0000","last_clean":"2023-11-06T21:02:12.031893+0000","last_became_active":"2023-11-06T21:02:12.031566+0000","last_became_peered":"2023-11-06T21:02:12.031566+0000","last_unstale":"2023-11-06T21:02:12.031893+0000","last_undegraded":"2023-11-06T21:02:12.031893+0000","last_fullsized":"2023-11-06T21:02:12.031893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2d","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.015230+0000","last_change":"2023-11-06T21:02:11.533505+0000","last_active":"2023-11-06T21:02:12.015230+0000","last_peered":"2023-11-06T21:02:12.015230+0000","last_clean":"2023-11-06T21:02:12.015230+0000","last_became_active":"2023-11-06T21:02:11.533239+0000","last_became_peered":"2023-11-06T21:02:11.533239+0000","last_unstale":"2023-11-06T21:02:12.015230+0000","last_undegraded":"2023-11-06T21:02:12.015230+0000","last_fullsized":"2023-11-06T21:02:12.015230+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.030130+0000","last_change":"2023-11-06T21:02:12.030130+0000","last_active":"2023-11-06T21:02:12.030130+0000","last_peered":"2023-11-06T21:02:12.030130+0000","last_clean":"2023-11-06T21:02:12.030130+0000","last_became_active":"2023-11-06T21:02:12.029573+0000","last_became_peered":"2023-11-06T21:02:12.029573+0000","last_unstale":"2023-11-06T21:02:12.030130+0000","last_undegraded":"2023-11-06T21:02:12.030130+0000","last_fullsized":"2023-11-06T21:02:12.030130+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.005813+0000","last_change":"2023-11-06T21:02:11.527607+0000","last_active":"2023-11-06T21:02:12.005813+0000","last_peered":"2023-11-06T21:02:12.005813+0000","last_clean":"2023-11-06T21:02:12.005813+0000","last_became_active":"2023-11-06T21:02:11.527149+0000","last_became_peered":"2023-11-06T21:02:11.527149+0000","last_unstale":"2023-11-06T21:02:12.005813+0000","last_undegraded":"2023-11-06T21:02:12.005813+0000","last_fullsized":"2023-11-06T21:02:12.005813+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.037845+0000","last_change":"2023-11-06T21:02:12.037845+0000","last_active":"2023-11-06T21:02:12.037845+0000","last_peered":"2023-11-06T21:02:12.037845+0000","last_clean":"2023-11-06T21:02:12.037845+0000","last_became_active":"2023-11-06T21:02:12.037570+0000","last_became_peered":"2023-11-06T21:02:12.037570+0000","last_unstale":"2023-11-06T21:02:12.037845+0000","last_undegraded":"2023-11-06T21:02:12.037845+0000","last_fullsized":"2023-11-06T21:02:12.037845+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2b","version":"130'1","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.010266+0000","last_change":"2023-11-06T21:02:11.538265+0000","last_active":"2023-11-06T21:02:12.010266+0000","last_peered":"2023-11-06T21:02:12.010266+0000","last_clean":"2023-11-06T21:02:12.010266+0000","last_became_active":"2023-11-06T21:02:11.537809+0000","last_became_peered":"2023-11-06T21:02:11.537809+0000","last_unstale":"2023-11-06T21:02:12.010266+0000","last_undegraded":"2023-11-06T21:02:12.010266+0000","last_fullsized":"2023-11-06T21:02:12.010266+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.014645+0000","last_change":"2023-11-06T21:02:12.014645+0000","last_active":"2023-11-06T21:02:12.014645+0000","last_peered":"2023-11-06T21:02:12.014645+0000","last_clean":"2023-11-06T21:02:12.014645+0000","last_became_active":"2023-11-06T21:02:12.014391+0000","last_became_peered":"2023-11-06T21:02:12.014391+0000","last_unstale":"2023-11-06T21:02:12.014645+0000","last_undegraded":"2023-11-06T21:02:12.014645+0000","last_fullsized":"2023-11-06T21:02:12.014645+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.540384+0000","last_change":"2023-11-06T21:02:11.538253+0000","last_active":"2023-11-06T21:02:11.540384+0000","last_peered":"2023-11-06T21:02:11.540384+0000","last_clean":"2023-11-06T21:02:11.540384+0000","last_became_active":"2023-11-06T21:02:11.537682+0000","last_became_peered":"2023-11-06T21:02:11.537682+0000","last_unstale":"2023-11-06T21:02:11.540384+0000","last_undegraded":"2023-11-06T21:02:11.540384+0000","last_fullsized":"2023-11-06T21:02:11.540384+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.009419+0000","last_change":"2023-11-06T21:02:12.009419+0000","last_active":"2023-11-06T21:02:12.009419+0000","last_peered":"2023-11-06T21:02:12.009419+0000","last_clean":"2023-11-06T21:02:12.009419+0000","last_became_active":"2023-11-06T21:02:12.009140+0000","last_became_peered":"2023-11-06T21:02:12.009140+0000","last_unstale":"2023-11-06T21:02:12.009419+0000","last_undegraded":"2023-11-06T21:02:12.009419+0000","last_fullsized":"2023-11-06T21:02:12.009419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.526336+0000","last_change":"2023-11-06T21:02:11.516613+0000","last_active":"2023-11-06T21:02:11.526336+0000","last_peered":"2023-11-06T21:02:11.526336+0000","last_clean":"2023-11-06T21:02:11.526336+0000","last_became_active":"2023-11-06T21:02:11.515887+0000","last_became_peered":"2023-11-06T21:02:11.515887+0000","last_unstale":"2023-11-06T21:02:11.526336+0000","last_undegraded":"2023-11-06T21:02:11.526336+0000","last_fullsized":"2023-11-06T21:02:11.526336+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.023525+0000","last_change":"2023-11-06T21:02:12.023525+0000","last_active":"2023-11-06T21:02:12.023525+0000","last_peered":"2023-11-06T21:02:12.023525+0000","last_clean":"2023-11-06T21:02:12.023525+0000","last_became_active":"2023-11-06T21:02:12.023224+0000","last_became_peered":"2023-11-06T21:02:12.023224+0000","last_unstale":"2023-11-06T21:02:12.023525+0000","last_undegraded":"2023-11-06T21:02:12.023525+0000","last_fullsized":"2023-11-06T21:02:12.023525+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.536585+0000","last_change":"2023-11-06T21:02:11.532170+0000","last_active":"2023-11-06T21:02:11.536585+0000","last_peered":"2023-11-06T21:02:11.536585+0000","last_clean":"2023-11-06T21:02:11.536585+0000","last_became_active":"2023-11-06T21:02:11.531800+0000","last_became_peered":"2023-11-06T21:02:11.531800+0000","last_unstale":"2023-11-06T21:02:11.536585+0000","last_undegraded":"2023-11-06T21:02:11.536585+0000","last_fullsized":"2023-11-06T21:02:11.536585+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.034523+0000","last_change":"2023-11-06T21:02:12.034523+0000","last_active":"2023-11-06T21:02:12.034523+0000","last_peered":"2023-11-06T21:02:12.034523+0000","last_clean":"2023-11-06T21:02:12.034523+0000","last_became_active":"2023-11-06T21:02:12.034127+0000","last_became_peered":"2023-11-06T21:02:12.034127+0000","last_unstale":"2023-11-06T21:02:12.034523+0000","last_undegraded":"2023-11-06T21:02:12.034523+0000","last_fullsized":"2023-11-06T21:02:12.034523+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.543347+0000","last_change":"2023-11-06T21:02:11.539499+0000","last_active":"2023-11-06T21:02:11.543347+0000","last_peered":"2023-11-06T21:02:11.543347+0000","last_clean":"2023-11-06T21:02:11.543347+0000","last_became_active":"2023-11-06T21:02:11.539158+0000","last_became_peered":"2023-11-06T21:02:11.539158+0000","last_unstale":"2023-11-06T21:02:11.543347+0000","last_undegraded":"2023-11-06T21:02:11.543347+0000","last_fullsized":"2023-11-06T21:02:11.543347+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.039351+0000","last_change":"2023-11-06T21:02:12.039351+0000","last_active":"2023-11-06T21:02:12.039351+0000","last_peered":"2023-11-06T21:02:12.039351+0000","last_clean":"2023-11-06T21:02:12.039351+0000","last_became_active":"2023-11-06T21:02:12.038547+0000","last_became_peered":"2023-11-06T21:02:12.038547+0000","last_unstale":"2023-11-06T21:02:12.039351+0000","last_undegraded":"2023-11-06T21:02:12.039351+0000","last_fullsized":"2023-11-06T21:02:12.039351+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.006096+0000","last_change":"2023-11-06T21:02:11.530207+0000","last_active":"2023-11-06T21:02:12.006096+0000","last_peered":"2023-11-06T21:02:12.006096+0000","last_clean":"2023-11-06T21:02:12.006096+0000","last_became_active":"2023-11-06T21:02:11.529882+0000","last_became_peered":"2023-11-06T21:02:11.529882+0000","last_unstale":"2023-11-06T21:02:12.006096+0000","last_undegraded":"2023-11-06T21:02:12.006096+0000","last_fullsized":"2023-11-06T21:02:12.006096+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.034792+0000","last_change":"2023-11-06T21:02:12.034792+0000","last_active":"2023-11-06T21:02:12.034792+0000","last_peered":"2023-11-06T21:02:12.034792+0000","last_clean":"2023-11-06T21:02:12.034792+0000","last_became_active":"2023-11-06T21:02:12.034090+0000","last_became_peered":"2023-11-06T21:02:12.034090+0000","last_unstale":"2023-11-06T21:02:12.034792+0000","last_undegraded":"2023-11-06T21:02:12.034792+0000","last_fullsized":"2023-11-06T21:02:12.034792+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.528865+0000","last_change":"2023-11-06T21:02:11.523951+0000","last_active":"2023-11-06T21:02:11.528865+0000","last_peered":"2023-11-06T21:02:11.528865+0000","last_clean":"2023-11-06T21:02:11.528865+0000","last_became_active":"2023-11-06T21:02:11.523622+0000","last_became_peered":"2023-11-06T21:02:11.523622+0000","last_unstale":"2023-11-06T21:02:11.528865+0000","last_undegraded":"2023-11-06T21:02:11.528865+0000","last_fullsized":"2023-11-06T21:02:11.528865+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.021177+0000","last_change":"2023-11-06T21:02:12.021177+0000","last_active":"2023-11-06T21:02:12.021177+0000","last_peered":"2023-11-06T21:02:12.021177+0000","last_clean":"2023-11-06T21:02:12.021177+0000","last_became_active":"2023-11-06T21:02:12.020435+0000","last_became_peered":"2023-11-06T21:02:12.020435+0000","last_unstale":"2023-11-06T21:02:12.021177+0000","last_undegraded":"2023-11-06T21:02:12.021177+0000","last_fullsized":"2023-11-06T21:02:12.021177+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.536678+0000","last_change":"2023-11-06T21:02:11.532667+0000","last_active":"2023-11-06T21:02:11.536678+0000","last_peered":"2023-11-06T21:02:11.536678+0000","last_clean":"2023-11-06T21:02:11.536678+0000","last_became_active":"2023-11-06T21:02:11.532393+0000","last_became_peered":"2023-11-06T21:02:11.532393+0000","last_unstale":"2023-11-06T21:02:11.536678+0000","last_undegraded":"2023-11-06T21:02:11.536678+0000","last_fullsized":"2023-11-06T21:02:11.536678+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.018844+0000","last_change":"2023-11-06T21:02:12.018844+0000","last_active":"2023-11-06T21:02:12.018844+0000","last_peered":"2023-11-06T21:02:12.018844+0000","last_clean":"2023-11-06T21:02:12.018844+0000","last_became_active":"2023-11-06T21:02:12.018553+0000","last_became_peered":"2023-11-06T21:02:12.018553+0000","last_unstale":"2023-11-06T21:02:12.018844+0000","last_undegraded":"2023-11-06T21:02:12.018844+0000","last_fullsized":"2023-11-06T21:02:12.018844+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.005665+0000","last_change":"2023-11-06T21:02:11.531958+0000","last_active":"2023-11-06T21:02:12.005665+0000","last_peered":"2023-11-06T21:02:12.005665+0000","last_clean":"2023-11-06T21:02:12.005665+0000","last_became_active":"2023-11-06T21:02:11.530284+0000","last_became_peered":"2023-11-06T21:02:11.530284+0000","last_unstale":"2023-11-06T21:02:12.005665+0000","last_undegraded":"2023-11-06T21:02:12.005665+0000","last_fullsized":"2023-11-06T21:02:12.005665+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038190+0000","last_change":"2023-11-06T21:02:12.036172+0000","last_active":"2023-11-06T21:02:12.038190+0000","last_peered":"2023-11-06T21:02:12.038190+0000","last_clean":"2023-11-06T21:02:12.038190+0000","last_became_active":"2023-11-06T21:02:12.035812+0000","last_became_peered":"2023-11-06T21:02:12.035812+0000","last_unstale":"2023-11-06T21:02:12.038190+0000","last_undegraded":"2023-11-06T21:02:12.038190+0000","last_fullsized":"2023-11-06T21:02:12.038190+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.005386+0000","last_change":"2023-11-06T21:02:11.536801+0000","last_active":"2023-11-06T21:02:12.005386+0000","last_peered":"2023-11-06T21:02:12.005386+0000","last_clean":"2023-11-06T21:02:12.005386+0000","last_became_active":"2023-11-06T21:02:11.535891+0000","last_became_peered":"2023-11-06T21:02:11.535891+0000","last_unstale":"2023-11-06T21:02:12.005386+0000","last_undegraded":"2023-11-06T21:02:12.005386+0000","last_fullsized":"2023-11-06T21:02:12.005386+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.036551+0000","last_change":"2023-11-06T21:02:12.036551+0000","last_active":"2023-11-06T21:02:12.036551+0000","last_peered":"2023-11-06T21:02:12.036551+0000","last_clean":"2023-11-06T21:02:12.036551+0000","last_became_active":"2023-11-06T21:02:12.035367+0000","last_became_peered":"2023-11-06T21:02:12.035367+0000","last_unstale":"2023-11-06T21:02:12.036551+0000","last_undegraded":"2023-11-06T21:02:12.036551+0000","last_fullsized":"2023-11-06T21:02:12.036551+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.538331+0000","last_change":"2023-11-06T21:02:11.528800+0000","last_active":"2023-11-06T21:02:11.538331+0000","last_peered":"2023-11-06T21:02:11.538331+0000","last_clean":"2023-11-06T21:02:11.538331+0000","last_became_active":"2023-11-06T21:02:11.528497+0000","last_became_peered":"2023-11-06T21:02:11.528497+0000","last_unstale":"2023-11-06T21:02:11.538331+0000","last_undegraded":"2023-11-06T21:02:11.538331+0000","last_fullsized":"2023-11-06T21:02:11.538331+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.018260+0000","last_change":"2023-11-06T21:02:12.018260+0000","last_active":"2023-11-06T21:02:12.018260+0000","last_peered":"2023-11-06T21:02:12.018260+0000","last_clean":"2023-11-06T21:02:12.018260+0000","last_became_active":"2023-11-06T21:02:12.017954+0000","last_became_peered":"2023-11-06T21:02:12.017954+0000","last_unstale":"2023-11-06T21:02:12.018260+0000","last_undegraded":"2023-11-06T21:02:12.018260+0000","last_fullsized":"2023-11-06T21:02:12.018260+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.534137+0000","last_change":"2023-11-06T21:02:11.532852+0000","last_active":"2023-11-06T21:02:11.534137+0000","last_peered":"2023-11-06T21:02:11.534137+0000","last_clean":"2023-11-06T21:02:11.534137+0000","last_became_active":"2023-11-06T21:02:11.532562+0000","last_became_peered":"2023-11-06T21:02:11.532562+0000","last_unstale":"2023-11-06T21:02:11.534137+0000","last_undegraded":"2023-11-06T21:02:11.534137+0000","last_fullsized":"2023-11-06T21:02:11.534137+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.037725+0000","last_change":"2023-11-06T21:02:12.032349+0000","last_active":"2023-11-06T21:02:12.037725+0000","last_peered":"2023-11-06T21:02:12.037725+0000","last_clean":"2023-11-06T21:02:12.037725+0000","last_became_active":"2023-11-06T21:02:12.031899+0000","last_became_peered":"2023-11-06T21:02:12.031899+0000","last_unstale":"2023-11-06T21:02:12.037725+0000","last_undegraded":"2023-11-06T21:02:12.037725+0000","last_fullsized":"2023-11-06T21:02:12.037725+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.021300+0000","last_change":"2023-11-06T21:02:11.530621+0000","last_active":"2023-11-06T21:02:12.021300+0000","last_peered":"2023-11-06T21:02:12.021300+0000","last_clean":"2023-11-06T21:02:12.021300+0000","last_became_active":"2023-11-06T21:02:11.530132+0000","last_became_peered":"2023-11-06T21:02:11.530132+0000","last_unstale":"2023-11-06T21:02:12.021300+0000","last_undegraded":"2023-11-06T21:02:12.021300+0000","last_fullsized":"2023-11-06T21:02:12.021300+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.022898+0000","last_change":"2023-11-06T21:02:12.022898+0000","last_active":"2023-11-06T21:02:12.022898+0000","last_peered":"2023-11-06T21:02:12.022898+0000","last_clean":"2023-11-06T21:02:12.022898+0000","last_became_active":"2023-11-06T21:02:12.022599+0000","last_became_peered":"2023-11-06T21:02:12.022599+0000","last_unstale":"2023-11-06T21:02:12.022898+0000","last_undegraded":"2023-11-06T21:02:12.022898+0000","last_fullsized":"2023-11-06T21:02:12.022898+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.026025+0000","last_change":"2023-11-06T21:02:11.537902+0000","last_active":"2023-11-06T21:02:12.026025+0000","last_peered":"2023-11-06T21:02:12.026025+0000","last_clean":"2023-11-06T21:02:12.026025+0000","last_became_active":"2023-11-06T21:02:11.537376+0000","last_became_peered":"2023-11-06T21:02:11.537376+0000","last_unstale":"2023-11-06T21:02:12.026025+0000","last_undegraded":"2023-11-06T21:02:12.026025+0000","last_fullsized":"2023-11-06T21:02:12.026025+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.016242+0000","last_change":"2023-11-06T21:02:12.016242+0000","last_active":"2023-11-06T21:02:12.016242+0000","last_peered":"2023-11-06T21:02:12.016242+0000","last_clean":"2023-11-06T21:02:12.016242+0000","last_became_active":"2023-11-06T21:02:12.015969+0000","last_became_peered":"2023-11-06T21:02:12.015969+0000","last_unstale":"2023-11-06T21:02:12.016242+0000","last_undegraded":"2023-11-06T21:02:12.016242+0000","last_fullsized":"2023-11-06T21:02:12.016242+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"130'2","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.021153+0000","last_change":"2023-11-06T21:02:11.527008+0000","last_active":"2023-11-06T21:02:12.021153+0000","last_peered":"2023-11-06T21:02:12.021153+0000","last_clean":"2023-11-06T21:02:12.021153+0000","last_became_active":"2023-11-06T21:02:11.526660+0000","last_became_peered":"2023-11-06T21:02:11.526660+0000","last_unstale":"2023-11-06T21:02:12.021153+0000","last_undegraded":"2023-11-06T21:02:12.021153+0000","last_fullsized":"2023-11-06T21:02:12.021153+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.035039+0000","last_change":"2023-11-06T21:02:12.035039+0000","last_active":"2023-11-06T21:02:12.035039+0000","last_peered":"2023-11-06T21:02:12.035039+0000","last_clean":"2023-11-06T21:02:12.035039+0000","last_became_active":"2023-11-06T21:02:12.034756+0000","last_became_peered":"2023-11-06T21:02:12.034756+0000","last_unstale":"2023-11-06T21:02:12.035039+0000","last_undegraded":"2023-11-06T21:02:12.035039+0000","last_fullsized":"2023-11-06T21:02:12.035039+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.538061+0000","last_change":"2023-11-06T21:02:11.533277+0000","last_active":"2023-11-06T21:02:11.538061+0000","last_peered":"2023-11-06T21:02:11.538061+0000","last_clean":"2023-11-06T21:02:11.538061+0000","last_became_active":"2023-11-06T21:02:11.530115+0000","last_became_peered":"2023-11-06T21:02:11.530115+0000","last_unstale":"2023-11-06T21:02:11.538061+0000","last_undegraded":"2023-11-06T21:02:11.538061+0000","last_fullsized":"2023-11-06T21:02:11.538061+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.020506+0000","last_change":"2023-11-06T21:02:12.020506+0000","last_active":"2023-11-06T21:02:12.020506+0000","last_peered":"2023-11-06T21:02:12.020506+0000","last_clean":"2023-11-06T21:02:12.020506+0000","last_became_active":"2023-11-06T21:02:12.019777+0000","last_became_peered":"2023-11-06T21:02:12.019777+0000","last_unstale":"2023-11-06T21:02:12.020506+0000","last_undegraded":"2023-11-06T21:02:12.020506+0000","last_fullsized":"2023-11-06T21:02:12.020506+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.537110+0000","last_change":"2023-11-06T21:02:11.535322+0000","last_active":"2023-11-06T21:02:11.537110+0000","last_peered":"2023-11-06T21:02:11.537110+0000","last_clean":"2023-11-06T21:02:11.537110+0000","last_became_active":"2023-11-06T21:02:11.534968+0000","last_became_peered":"2023-11-06T21:02:11.534968+0000","last_unstale":"2023-11-06T21:02:11.537110+0000","last_undegraded":"2023-11-06T21:02:11.537110+0000","last_fullsized":"2023-11-06T21:02:11.537110+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.029999+0000","last_change":"2023-11-06T21:02:12.029999+0000","last_active":"2023-11-06T21:02:12.029999+0000","last_peered":"2023-11-06T21:02:12.029999+0000","last_clean":"2023-11-06T21:02:12.029999+0000","last_became_active":"2023-11-06T21:02:12.029491+0000","last_became_peered":"2023-11-06T21:02:12.029491+0000","last_unstale":"2023-11-06T21:02:12.029999+0000","last_undegraded":"2023-11-06T21:02:12.029999+0000","last_fullsized":"2023-11-06T21:02:12.029999+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.008714+0000","last_change":"2023-11-06T21:02:11.530065+0000","last_active":"2023-11-06T21:02:12.008714+0000","last_peered":"2023-11-06T21:02:12.008714+0000","last_clean":"2023-11-06T21:02:12.008714+0000","last_became_active":"2023-11-06T21:02:11.529637+0000","last_became_peered":"2023-11-06T21:02:11.529637+0000","last_unstale":"2023-11-06T21:02:12.008714+0000","last_undegraded":"2023-11-06T21:02:12.008714+0000","last_fullsized":"2023-11-06T21:02:12.008714+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.007184+0000","last_change":"2023-11-06T21:02:12.007184+0000","last_active":"2023-11-06T21:02:12.007184+0000","last_peered":"2023-11-06T21:02:12.007184+0000","last_clean":"2023-11-06T21:02:12.007184+0000","last_became_active":"2023-11-06T21:02:12.006905+0000","last_became_peered":"2023-11-06T21:02:12.006905+0000","last_unstale":"2023-11-06T21:02:12.007184+0000","last_undegraded":"2023-11-06T21:02:12.007184+0000","last_fullsized":"2023-11-06T21:02:12.007184+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.011335+0000","last_change":"2023-11-06T21:02:11.530858+0000","last_active":"2023-11-06T21:02:12.011335+0000","last_peered":"2023-11-06T21:02:12.011335+0000","last_clean":"2023-11-06T21:02:12.011335+0000","last_became_active":"2023-11-06T21:02:11.529225+0000","last_became_peered":"2023-11-06T21:02:11.529225+0000","last_unstale":"2023-11-06T21:02:12.011335+0000","last_undegraded":"2023-11-06T21:02:12.011335+0000","last_fullsized":"2023-11-06T21:02:12.011335+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.024709+0000","last_change":"2023-11-06T21:02:12.024709+0000","last_active":"2023-11-06T21:02:12.024709+0000","last_peered":"2023-11-06T21:02:12.024709+0000","last_clean":"2023-11-06T21:02:12.024709+0000","last_became_active":"2023-11-06T21:02:12.023925+0000","last_became_peered":"2023-11-06T21:02:12.023925+0000","last_unstale":"2023-11-06T21:02:12.024709+0000","last_undegraded":"2023-11-06T21:02:12.024709+0000","last_fullsized":"2023-11-06T21:02:12.024709+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.005061+0000","last_change":"2023-11-06T21:02:11.525499+0000","last_active":"2023-11-06T21:02:12.005061+0000","last_peered":"2023-11-06T21:02:12.005061+0000","last_clean":"2023-11-06T21:02:12.005061+0000","last_became_active":"2023-11-06T21:02:11.525223+0000","last_became_peered":"2023-11-06T21:02:11.525223+0000","last_unstale":"2023-11-06T21:02:12.005061+0000","last_undegraded":"2023-11-06T21:02:12.005061+0000","last_fullsized":"2023-11-06T21:02:12.005061+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.030460+0000","last_change":"2023-11-06T21:02:12.030460+0000","last_active":"2023-11-06T21:02:12.030460+0000","last_peered":"2023-11-06T21:02:12.030460+0000","last_clean":"2023-11-06T21:02:12.030460+0000","last_became_active":"2023-11-06T21:02:12.029757+0000","last_became_peered":"2023-11-06T21:02:12.029757+0000","last_unstale":"2023-11-06T21:02:12.030460+0000","last_undegraded":"2023-11-06T21:02:12.030460+0000","last_fullsized":"2023-11-06T21:02:12.030460+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.536635+0000","last_change":"2023-11-06T21:02:11.532331+0000","last_active":"2023-11-06T21:02:11.536635+0000","last_peered":"2023-11-06T21:02:11.536635+0000","last_clean":"2023-11-06T21:02:11.536635+0000","last_became_active":"2023-11-06T21:02:11.531841+0000","last_became_peered":"2023-11-06T21:02:11.531841+0000","last_unstale":"2023-11-06T21:02:11.536635+0000","last_undegraded":"2023-11-06T21:02:11.536635+0000","last_fullsized":"2023-11-06T21:02:11.536635+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.019769+0000","last_change":"2023-11-06T21:02:12.019769+0000","last_active":"2023-11-06T21:02:12.019769+0000","last_peered":"2023-11-06T21:02:12.019769+0000","last_clean":"2023-11-06T21:02:12.019769+0000","last_became_active":"2023-11-06T21:02:12.018406+0000","last_became_peered":"2023-11-06T21:02:12.018406+0000","last_unstale":"2023-11-06T21:02:12.019769+0000","last_undegraded":"2023-11-06T21:02:12.019769+0000","last_fullsized":"2023-11-06T21:02:12.019769+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.540662+0000","last_change":"2023-11-06T21:02:11.536915+0000","last_active":"2023-11-06T21:02:11.540662+0000","last_peered":"2023-11-06T21:02:11.540662+0000","last_clean":"2023-11-06T21:02:11.540662+0000","last_became_active":"2023-11-06T21:02:11.536633+0000","last_became_peered":"2023-11-06T21:02:11.536633+0000","last_unstale":"2023-11-06T21:02:11.540662+0000","last_undegraded":"2023-11-06T21:02:11.540662+0000","last_fullsized":"2023-11-06T21:02:11.540662+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.019602+0000","last_change":"2023-11-06T21:02:12.019602+0000","last_active":"2023-11-06T21:02:12.019602+0000","last_peered":"2023-11-06T21:02:12.019602+0000","last_clean":"2023-11-06T21:02:12.019602+0000","last_became_active":"2023-11-06T21:02:12.019247+0000","last_became_peered":"2023-11-06T21:02:12.019247+0000","last_unstale":"2023-11-06T21:02:12.019602+0000","last_undegraded":"2023-11-06T21:02:12.019602+0000","last_fullsized":"2023-11-06T21:02:12.019602+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"130'3","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.033827+0000","last_change":"2023-11-06T21:02:11.538292+0000","last_active":"2023-11-06T21:02:12.033827+0000","last_peered":"2023-11-06T21:02:12.033827+0000","last_clean":"2023-11-06T21:02:12.033827+0000","last_became_active":"2023-11-06T21:02:11.537241+0000","last_became_peered":"2023-11-06T21:02:11.537241+0000","last_unstale":"2023-11-06T21:02:12.033827+0000","last_undegraded":"2023-11-06T21:02:12.033827+0000","last_fullsized":"2023-11-06T21:02:12.033827+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.032794+0000","last_change":"2023-11-06T21:02:12.027737+0000","last_active":"2023-11-06T21:02:12.032794+0000","last_peered":"2023-11-06T21:02:12.032794+0000","last_clean":"2023-11-06T21:02:12.032794+0000","last_became_active":"2023-11-06T21:02:12.027205+0000","last_became_peered":"2023-11-06T21:02:12.027205+0000","last_unstale":"2023-11-06T21:02:12.032794+0000","last_undegraded":"2023-11-06T21:02:12.032794+0000","last_fullsized":"2023-11-06T21:02:12.032794+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"130'2","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.016018+0000","last_change":"2023-11-06T21:02:11.519327+0000","last_active":"2023-11-06T21:02:12.016018+0000","last_peered":"2023-11-06T21:02:12.016018+0000","last_clean":"2023-11-06T21:02:12.016018+0000","last_became_active":"2023-11-06T21:02:11.518908+0000","last_became_peered":"2023-11-06T21:02:11.518908+0000","last_unstale":"2023-11-06T21:02:12.016018+0000","last_undegraded":"2023-11-06T21:02:12.016018+0000","last_fullsized":"2023-11-06T21:02:12.016018+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.022968+0000","last_change":"2023-11-06T21:02:12.022968+0000","last_active":"2023-11-06T21:02:12.022968+0000","last_peered":"2023-11-06T21:02:12.022968+0000","last_clean":"2023-11-06T21:02:12.022968+0000","last_became_active":"2023-11-06T21:02:12.022697+0000","last_became_peered":"2023-11-06T21:02:12.022697+0000","last_unstale":"2023-11-06T21:02:12.022968+0000","last_undegraded":"2023-11-06T21:02:12.022968+0000","last_fullsized":"2023-11-06T21:02:12.022968+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"130'1","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.010405+0000","last_change":"2023-11-06T21:02:11.537213+0000","last_active":"2023-11-06T21:02:12.010405+0000","last_peered":"2023-11-06T21:02:12.010405+0000","last_clean":"2023-11-06T21:02:12.010405+0000","last_became_active":"2023-11-06T21:02:11.536949+0000","last_became_peered":"2023-11-06T21:02:11.536949+0000","last_unstale":"2023-11-06T21:02:12.010405+0000","last_undegraded":"2023-11-06T21:02:12.010405+0000","last_fullsized":"2023-11-06T21:02:12.010405+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.016075+0000","last_change":"2023-11-06T21:02:12.016075+0000","last_active":"2023-11-06T21:02:12.016075+0000","last_peered":"2023-11-06T21:02:12.016075+0000","last_clean":"2023-11-06T21:02:12.016075+0000","last_became_active":"2023-11-06T21:02:12.015841+0000","last_became_peered":"2023-11-06T21:02:12.015841+0000","last_unstale":"2023-11-06T21:02:12.016075+0000","last_undegraded":"2023-11-06T21:02:12.016075+0000","last_fullsized":"2023-11-06T21:02:12.016075+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.532820+0000","last_change":"2023-11-06T21:02:11.527072+0000","last_active":"2023-11-06T21:02:11.532820+0000","last_peered":"2023-11-06T21:02:11.532820+0000","last_clean":"2023-11-06T21:02:11.532820+0000","last_became_active":"2023-11-06T21:02:11.526763+0000","last_became_peered":"2023-11-06T21:02:11.526763+0000","last_unstale":"2023-11-06T21:02:11.532820+0000","last_undegraded":"2023-11-06T21:02:11.532820+0000","last_fullsized":"2023-11-06T21:02:11.532820+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038104+0000","last_change":"2023-11-06T21:02:12.035523+0000","last_active":"2023-11-06T21:02:12.038104+0000","last_peered":"2023-11-06T21:02:12.038104+0000","last_clean":"2023-11-06T21:02:12.038104+0000","last_became_active":"2023-11-06T21:02:12.035218+0000","last_became_peered":"2023-11-06T21:02:12.035218+0000","last_unstale":"2023-11-06T21:02:12.038104+0000","last_undegraded":"2023-11-06T21:02:12.038104+0000","last_fullsized":"2023-11-06T21:02:12.038104+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.538351+0000","last_change":"2023-11-06T21:02:11.533087+0000","last_active":"2023-11-06T21:02:11.538351+0000","last_peered":"2023-11-06T21:02:11.538351+0000","last_clean":"2023-11-06T21:02:11.538351+0000","last_became_active":"2023-11-06T21:02:11.532327+0000","last_became_peered":"2023-11-06T21:02:11.532327+0000","last_unstale":"2023-11-06T21:02:11.538351+0000","last_undegraded":"2023-11-06T21:02:11.538351+0000","last_fullsized":"2023-11-06T21:02:11.538351+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.038230+0000","last_change":"2023-11-06T21:02:12.035967+0000","last_active":"2023-11-06T21:02:12.038230+0000","last_peered":"2023-11-06T21:02:12.038230+0000","last_clean":"2023-11-06T21:02:12.038230+0000","last_became_active":"2023-11-06T21:02:12.035631+0000","last_became_peered":"2023-11-06T21:02:12.035631+0000","last_unstale":"2023-11-06T21:02:12.038230+0000","last_undegraded":"2023-11-06T21:02:12.038230+0000","last_fullsized":"2023-11-06T21:02:12.038230+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"130'4","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.029646+0000","last_change":"2023-11-06T21:02:11.521162+0000","last_active":"2023-11-06T21:02:12.029646+0000","last_peered":"2023-11-06T21:02:12.029646+0000","last_clean":"2023-11-06T21:02:12.029646+0000","last_became_active":"2023-11-06T21:02:11.520878+0000","last_became_peered":"2023-11-06T21:02:11.520878+0000","last_unstale":"2023-11-06T21:02:12.029646+0000","last_undegraded":"2023-11-06T21:02:12.029646+0000","last_fullsized":"2023-11-06T21:02:12.029646+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.021963+0000","last_change":"2023-11-06T21:02:12.016162+0000","last_active":"2023-11-06T21:02:12.021963+0000","last_peered":"2023-11-06T21:02:12.021963+0000","last_clean":"2023-11-06T21:02:12.021963+0000","last_became_active":"2023-11-06T21:02:12.015419+0000","last_became_peered":"2023-11-06T21:02:12.015419+0000","last_unstale":"2023-11-06T21:02:12.021963+0000","last_undegraded":"2023-11-06T21:02:12.021963+0000","last_fullsized":"2023-11-06T21:02:12.021963+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.013754+0000","last_change":"2023-11-06T21:02:11.531067+0000","last_active":"2023-11-06T21:02:12.013754+0000","last_peered":"2023-11-06T21:02:12.013754+0000","last_clean":"2023-11-06T21:02:12.013754+0000","last_became_active":"2023-11-06T21:02:11.530776+0000","last_became_peered":"2023-11-06T21:02:11.530776+0000","last_unstale":"2023-11-06T21:02:12.013754+0000","last_undegraded":"2023-11-06T21:02:12.013754+0000","last_fullsized":"2023-11-06T21:02:12.013754+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.009852+0000","last_change":"2023-11-06T21:02:12.009852+0000","last_active":"2023-11-06T21:02:12.009852+0000","last_peered":"2023-11-06T21:02:12.009852+0000","last_clean":"2023-11-06T21:02:12.009852+0000","last_became_active":"2023-11-06T21:02:12.009609+0000","last_became_peered":"2023-11-06T21:02:12.009609+0000","last_unstale":"2023-11-06T21:02:12.009852+0000","last_undegraded":"2023-11-06T21:02:12.009852+0000","last_fullsized":"2023-11-06T21:02:12.009852+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.001599+0000","last_change":"2023-11-06T21:02:11.534019+0000","last_active":"2023-11-06T21:02:12.001599+0000","last_peered":"2023-11-06T21:02:12.001599+0000","last_clean":"2023-11-06T21:02:12.001599+0000","last_became_active":"2023-11-06T21:02:11.533757+0000","last_became_peered":"2023-11-06T21:02:11.533757+0000","last_unstale":"2023-11-06T21:02:12.001599+0000","last_undegraded":"2023-11-06T21:02:12.001599+0000","last_fullsized":"2023-11-06T21:02:12.001599+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.034460+0000","last_change":"2023-11-06T21:02:12.034460+0000","last_active":"2023-11-06T21:02:12.034460+0000","last_peered":"2023-11-06T21:02:12.034460+0000","last_clean":"2023-11-06T21:02:12.034460+0000","last_became_active":"2023-11-06T21:02:12.034069+0000","last_became_peered":"2023-11-06T21:02:12.034069+0000","last_unstale":"2023-11-06T21:02:12.034460+0000","last_undegraded":"2023-11-06T21:02:12.034460+0000","last_fullsized":"2023-11-06T21:02:12.034460+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.536828+0000","last_change":"2023-11-06T21:02:11.531661+0000","last_active":"2023-11-06T21:02:11.536828+0000","last_peered":"2023-11-06T21:02:11.536828+0000","last_clean":"2023-11-06T21:02:11.536828+0000","last_became_active":"2023-11-06T21:02:11.531274+0000","last_became_peered":"2023-11-06T21:02:11.531274+0000","last_unstale":"2023-11-06T21:02:11.536828+0000","last_undegraded":"2023-11-06T21:02:11.536828+0000","last_fullsized":"2023-11-06T21:02:11.536828+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.030856+0000","last_change":"2023-11-06T21:02:12.026002+0000","last_active":"2023-11-06T21:02:12.030856+0000","last_peered":"2023-11-06T21:02:12.030856+0000","last_clean":"2023-11-06T21:02:12.030856+0000","last_became_active":"2023-11-06T21:02:12.025435+0000","last_became_peered":"2023-11-06T21:02:12.025435+0000","last_unstale":"2023-11-06T21:02:12.030856+0000","last_undegraded":"2023-11-06T21:02:12.030856+0000","last_fullsized":"2023-11-06T21:02:12.030856+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.542962+0000","last_change":"2023-11-06T21:02:11.541952+0000","last_active":"2023-11-06T21:02:11.542962+0000","last_peered":"2023-11-06T21:02:11.542962+0000","last_clean":"2023-11-06T21:02:11.542962+0000","last_became_active":"2023-11-06T21:02:11.541458+0000","last_became_peered":"2023-11-06T21:02:11.541458+0000","last_unstale":"2023-11-06T21:02:11.542962+0000","last_undegraded":"2023-11-06T21:02:11.542962+0000","last_fullsized":"2023-11-06T21:02:11.542962+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.025068+0000","last_change":"2023-11-06T21:02:12.025068+0000","last_active":"2023-11-06T21:02:12.025068+0000","last_peered":"2023-11-06T21:02:12.025068+0000","last_clean":"2023-11-06T21:02:12.025068+0000","last_became_active":"2023-11-06T21:02:12.023625+0000","last_became_peered":"2023-11-06T21:02:12.023625+0000","last_unstale":"2023-11-06T21:02:12.025068+0000","last_undegraded":"2023-11-06T21:02:12.025068+0000","last_fullsized":"2023-11-06T21:02:12.025068+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.543835+0000","last_change":"2023-11-06T21:02:11.541344+0000","last_active":"2023-11-06T21:02:11.543835+0000","last_peered":"2023-11-06T21:02:11.543835+0000","last_clean":"2023-11-06T21:02:11.543835+0000","last_became_active":"2023-11-06T21:02:11.541032+0000","last_became_peered":"2023-11-06T21:02:11.541032+0000","last_unstale":"2023-11-06T21:02:11.543835+0000","last_undegraded":"2023-11-06T21:02:11.543835+0000","last_fullsized":"2023-11-06T21:02:11.543835+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.024277+0000","last_change":"2023-11-06T21:02:12.024277+0000","last_active":"2023-11-06T21:02:12.024277+0000","last_peered":"2023-11-06T21:02:12.024277+0000","last_clean":"2023-11-06T21:02:12.024277+0000","last_became_active":"2023-11-06T21:02:12.023944+0000","last_became_peered":"2023-11-06T21:02:12.023944+0000","last_unstale":"2023-11-06T21:02:12.024277+0000","last_undegraded":"2023-11-06T21:02:12.024277+0000","last_fullsized":"2023-11-06T21:02:12.024277+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.999797+0000","last_change":"2023-11-06T21:02:11.529689+0000","last_active":"2023-11-06T21:02:11.999797+0000","last_peered":"2023-11-06T21:02:11.999797+0000","last_clean":"2023-11-06T21:02:11.999797+0000","last_became_active":"2023-11-06T21:02:11.529428+0000","last_became_peered":"2023-11-06T21:02:11.529428+0000","last_unstale":"2023-11-06T21:02:11.999797+0000","last_undegraded":"2023-11-06T21:02:11.999797+0000","last_fullsized":"2023-11-06T21:02:11.999797+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.034592+0000","last_change":"2023-11-06T21:02:12.034592+0000","last_active":"2023-11-06T21:02:12.034592+0000","last_peered":"2023-11-06T21:02:12.034592+0000","last_clean":"2023-11-06T21:02:12.034592+0000","last_became_active":"2023-11-06T21:02:12.034304+0000","last_became_peered":"2023-11-06T21:02:12.034304+0000","last_unstale":"2023-11-06T21:02:12.034592+0000","last_undegraded":"2023-11-06T21:02:12.034592+0000","last_fullsized":"2023-11-06T21:02:12.034592+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.011475+0000","last_change":"2023-11-06T21:02:11.520242+0000","last_active":"2023-11-06T21:02:12.011475+0000","last_peered":"2023-11-06T21:02:12.011475+0000","last_clean":"2023-11-06T21:02:12.011475+0000","last_became_active":"2023-11-06T21:02:11.519979+0000","last_became_peered":"2023-11-06T21:02:11.519979+0000","last_unstale":"2023-11-06T21:02:12.011475+0000","last_undegraded":"2023-11-06T21:02:12.011475+0000","last_fullsized":"2023-11-06T21:02:12.011475+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.036366+0000","last_change":"2023-11-06T21:02:12.036366+0000","last_active":"2023-11-06T21:02:12.036366+0000","last_peered":"2023-11-06T21:02:12.036366+0000","last_clean":"2023-11-06T21:02:12.036366+0000","last_became_active":"2023-11-06T21:02:12.036090+0000","last_became_peered":"2023-11-06T21:02:12.036090+0000","last_unstale":"2023-11-06T21:02:12.036366+0000","last_undegraded":"2023-11-06T21:02:12.036366+0000","last_fullsized":"2023-11-06T21:02:12.036366+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.000232+0000","last_change":"2023-11-06T21:02:11.527732+0000","last_active":"2023-11-06T21:02:12.000232+0000","last_peered":"2023-11-06T21:02:12.000232+0000","last_clean":"2023-11-06T21:02:12.000232+0000","last_became_active":"2023-11-06T21:02:11.527411+0000","last_became_peered":"2023-11-06T21:02:11.527411+0000","last_unstale":"2023-11-06T21:02:12.000232+0000","last_undegraded":"2023-11-06T21:02:12.000232+0000","last_fullsized":"2023-11-06T21:02:12.000232+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.032837+0000","last_change":"2023-11-06T21:02:12.026488+0000","last_active":"2023-11-06T21:02:12.032837+0000","last_peered":"2023-11-06T21:02:12.032837+0000","last_clean":"2023-11-06T21:02:12.032837+0000","last_became_active":"2023-11-06T21:02:12.026203+0000","last_became_peered":"2023-11-06T21:02:12.026203+0000","last_unstale":"2023-11-06T21:02:12.032837+0000","last_undegraded":"2023-11-06T21:02:12.032837+0000","last_fullsized":"2023-11-06T21:02:12.032837+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.532656+0000","last_change":"2023-11-06T21:02:11.526228+0000","last_active":"2023-11-06T21:02:11.532656+0000","last_peered":"2023-11-06T21:02:11.532656+0000","last_clean":"2023-11-06T21:02:11.532656+0000","last_became_active":"2023-11-06T21:02:11.525933+0000","last_became_peered":"2023-11-06T21:02:11.525933+0000","last_unstale":"2023-11-06T21:02:11.532656+0000","last_undegraded":"2023-11-06T21:02:11.532656+0000","last_fullsized":"2023-11-06T21:02:11.532656+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.999615+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T21:02:11.999615+0000","last_peered":"2023-11-06T21:02:11.999615+0000","last_clean":"2023-11-06T21:02:11.999615+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T21:02:11.999615+0000","last_undegraded":"2023-11-06T21:02:11.999615+0000","last_fullsized":"2023-11-06T21:02:11.999615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.022833+0000","last_change":"2023-11-06T21:02:12.022833+0000","last_active":"2023-11-06T21:02:12.022833+0000","last_peered":"2023-11-06T21:02:12.022833+0000","last_clean":"2023-11-06T21:02:12.022833+0000","last_became_active":"2023-11-06T21:02:12.022564+0000","last_became_peered":"2023-11-06T21:02:12.022564+0000","last_unstale":"2023-11-06T21:02:12.022833+0000","last_undegraded":"2023-11-06T21:02:12.022833+0000","last_fullsized":"2023-11-06T21:02:12.022833+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"130'1","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.020850+0000","last_change":"2023-11-06T21:02:11.541111+0000","last_active":"2023-11-06T21:02:12.020850+0000","last_peered":"2023-11-06T21:02:12.020850+0000","last_clean":"2023-11-06T21:02:12.020850+0000","last_became_active":"2023-11-06T21:02:11.540505+0000","last_became_peered":"2023-11-06T21:02:11.540505+0000","last_unstale":"2023-11-06T21:02:12.020850+0000","last_undegraded":"2023-11-06T21:02:12.020850+0000","last_fullsized":"2023-11-06T21:02:12.020850+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":116,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.513519+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T21:02:11.513519+0000","last_peered":"2023-11-06T21:02:11.513519+0000","last_clean":"2023-11-06T21:02:11.513519+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T21:02:11.513519+0000","last_undegraded":"2023-11-06T21:02:11.513519+0000","last_fullsized":"2023-11-06T21:02:11.513519+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.037362+0000","last_change":"2023-11-06T21:02:12.035276+0000","last_active":"2023-11-06T21:02:12.037362+0000","last_peered":"2023-11-06T21:02:12.037362+0000","last_clean":"2023-11-06T21:02:12.037362+0000","last_became_active":"2023-11-06T21:02:12.034464+0000","last_became_peered":"2023-11-06T21:02:12.034464+0000","last_unstale":"2023-11-06T21:02:12.037362+0000","last_undegraded":"2023-11-06T21:02:12.037362+0000","last_fullsized":"2023-11-06T21:02:12.037362+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.544261+0000","last_change":"2023-11-06T21:02:11.541489+0000","last_active":"2023-11-06T21:02:11.544261+0000","last_peered":"2023-11-06T21:02:11.544261+0000","last_clean":"2023-11-06T21:02:11.544261+0000","last_became_active":"2023-11-06T21:02:11.541189+0000","last_became_peered":"2023-11-06T21:02:11.541189+0000","last_unstale":"2023-11-06T21:02:11.544261+0000","last_undegraded":"2023-11-06T21:02:11.544261+0000","last_fullsized":"2023-11-06T21:02:11.544261+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.005163+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T21:02:12.005163+0000","last_peered":"2023-11-06T21:02:12.005163+0000","last_clean":"2023-11-06T21:02:12.005163+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T21:02:12.005163+0000","last_undegraded":"2023-11-06T21:02:12.005163+0000","last_fullsized":"2023-11-06T21:02:12.005163+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.039402+0000","last_change":"2023-11-06T21:02:12.039402+0000","last_active":"2023-11-06T21:02:12.039402+0000","last_peered":"2023-11-06T21:02:12.039402+0000","last_clean":"2023-11-06T21:02:12.039402+0000","last_became_active":"2023-11-06T21:02:12.039126+0000","last_became_peered":"2023-11-06T21:02:12.039126+0000","last_unstale":"2023-11-06T21:02:12.039402+0000","last_undegraded":"2023-11-06T21:02:12.039402+0000","last_fullsized":"2023-11-06T21:02:12.039402+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"130'2","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.028466+0000","last_change":"2023-11-06T21:02:11.538811+0000","last_active":"2023-11-06T21:02:12.028466+0000","last_peered":"2023-11-06T21:02:12.028466+0000","last_clean":"2023-11-06T21:02:12.028466+0000","last_became_active":"2023-11-06T21:02:11.538537+0000","last_became_peered":"2023-11-06T21:02:11.538537+0000","last_unstale":"2023-11-06T21:02:12.028466+0000","last_undegraded":"2023-11-06T21:02:12.028466+0000","last_fullsized":"2023-11-06T21:02:12.028466+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":116,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.509888+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T21:02:11.509888+0000","last_peered":"2023-11-06T21:02:11.509888+0000","last_clean":"2023-11-06T21:02:11.509888+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T21:02:11.509888+0000","last_undegraded":"2023-11-06T21:02:11.509888+0000","last_fullsized":"2023-11-06T21:02:11.509888+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":111,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.004844+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T21:02:12.004844+0000","last_peered":"2023-11-06T21:02:12.004844+0000","last_clean":"2023-11-06T21:02:12.004844+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T21:02:12.004844+0000","last_undegraded":"2023-11-06T21:02:12.004844+0000","last_fullsized":"2023-11-06T21:02:12.004844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.040755+0000","last_change":"2023-11-06T21:02:12.033454+0000","last_active":"2023-11-06T21:02:12.040755+0000","last_peered":"2023-11-06T21:02:12.040755+0000","last_clean":"2023-11-06T21:02:12.040755+0000","last_became_active":"2023-11-06T21:02:12.032937+0000","last_became_peered":"2023-11-06T21:02:12.032937+0000","last_unstale":"2023-11-06T21:02:12.040755+0000","last_undegraded":"2023-11-06T21:02:12.040755+0000","last_fullsized":"2023-11-06T21:02:12.040755+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.004505+0000","last_change":"2023-11-06T21:02:11.535993+0000","last_active":"2023-11-06T21:02:12.004505+0000","last_peered":"2023-11-06T21:02:12.004505+0000","last_clean":"2023-11-06T21:02:12.004505+0000","last_became_active":"2023-11-06T21:02:11.534095+0000","last_became_peered":"2023-11-06T21:02:11.534095+0000","last_unstale":"2023-11-06T21:02:12.004505+0000","last_undegraded":"2023-11-06T21:02:12.004505+0000","last_fullsized":"2023-11-06T21:02:12.004505+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":116,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.508951+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T21:02:11.508951+0000","last_peered":"2023-11-06T21:02:11.508951+0000","last_clean":"2023-11-06T21:02:11.508951+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T21:02:11.508951+0000","last_undegraded":"2023-11-06T21:02:11.508951+0000","last_fullsized":"2023-11-06T21:02:11.508951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.024897+0000","last_change":"2023-11-06T21:02:12.024897+0000","last_active":"2023-11-06T21:02:12.024897+0000","last_peered":"2023-11-06T21:02:12.024897+0000","last_clean":"2023-11-06T21:02:12.024897+0000","last_became_active":"2023-11-06T21:02:12.024348+0000","last_became_peered":"2023-11-06T21:02:12.024348+0000","last_unstale":"2023-11-06T21:02:12.024897+0000","last_undegraded":"2023-11-06T21:02:12.024897+0000","last_fullsized":"2023-11-06T21:02:12.024897+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.525396+0000","last_change":"2023-11-06T21:02:11.518624+0000","last_active":"2023-11-06T21:02:11.525396+0000","last_peered":"2023-11-06T21:02:11.525396+0000","last_clean":"2023-11-06T21:02:11.525396+0000","last_became_active":"2023-11-06T21:02:11.518352+0000","last_became_peered":"2023-11-06T21:02:11.518352+0000","last_unstale":"2023-11-06T21:02:11.525396+0000","last_undegraded":"2023-11-06T21:02:11.525396+0000","last_fullsized":"2023-11-06T21:02:11.525396+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":116,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.510715+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T21:02:11.510715+0000","last_peered":"2023-11-06T21:02:11.510715+0000","last_clean":"2023-11-06T21:02:11.510715+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T21:02:11.510715+0000","last_undegraded":"2023-11-06T21:02:11.510715+0000","last_fullsized":"2023-11-06T21:02:11.510715+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.026637+0000","last_change":"2023-11-06T21:02:12.026637+0000","last_active":"2023-11-06T21:02:12.026637+0000","last_peered":"2023-11-06T21:02:12.026637+0000","last_clean":"2023-11-06T21:02:12.026637+0000","last_became_active":"2023-11-06T21:02:12.024774+0000","last_became_peered":"2023-11-06T21:02:12.024774+0000","last_unstale":"2023-11-06T21:02:12.026637+0000","last_undegraded":"2023-11-06T21:02:12.026637+0000","last_fullsized":"2023-11-06T21:02:12.026637+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.013156+0000","last_change":"2023-11-06T21:02:11.541045+0000","last_active":"2023-11-06T21:02:12.013156+0000","last_peered":"2023-11-06T21:02:12.013156+0000","last_clean":"2023-11-06T21:02:12.013156+0000","last_became_active":"2023-11-06T21:02:11.540738+0000","last_became_peered":"2023-11-06T21:02:11.540738+0000","last_unstale":"2023-11-06T21:02:12.013156+0000","last_undegraded":"2023-11-06T21:02:12.013156+0000","last_fullsized":"2023-11-06T21:02:12.013156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.526226+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T21:02:11.526226+0000","last_peered":"2023-11-06T21:02:11.526226+0000","last_clean":"2023-11-06T21:02:11.526226+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T21:02:11.526226+0000","last_undegraded":"2023-11-06T21:02:11.526226+0000","last_fullsized":"2023-11-06T21:02:11.526226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:12.024370+0000","last_change":"2023-11-06T21:02:12.023205+0000","last_active":"2023-11-06T21:02:12.024370+0000","last_peered":"2023-11-06T21:02:12.024370+0000","last_clean":"2023-11-06T21:02:12.024370+0000","last_became_active":"2023-11-06T21:02:12.021768+0000","last_became_peered":"2023-11-06T21:02:12.021768+0000","last_unstale":"2023-11-06T21:02:12.024370+0000","last_undegraded":"2023-11-06T21:02:12.024370+0000","last_fullsized":"2023-11-06T21:02:12.024370+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:11.495483+0000","last_clean_scrub_stamp":"2023-11-06T21:02:11.495483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.999691+0000","last_change":"2023-11-06T21:02:11.536063+0000","last_active":"2023-11-06T21:02:11.999691+0000","last_peered":"2023-11-06T21:02:11.999691+0000","last_clean":"2023-11-06T21:02:11.999691+0000","last_became_active":"2023-11-06T21:02:11.535724+0000","last_became_peered":"2023-11-06T21:02:11.535724+0000","last_unstale":"2023-11-06T21:02:11.999691+0000","last_undegraded":"2023-11-06T21:02:11.999691+0000","last_fullsized":"2023-11-06T21:02:11.999691+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:02:10.486428+0000","last_clean_scrub_stamp":"2023-11-06T21:02:10.486428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":116,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-06T21:02:11.509814+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T21:02:11.509814+0000","last_peered":"2023-11-06T21:02:11.509814+0000","last_clean":"2023-11-06T21:02:11.509814+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T21:02:11.509814+0000","last_undegraded":"2023-11-06T21:02:11.509814+0000","last_fullsized":"2023-11-06T21:02:11.509814+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1617920,"data_stored":819537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66300000000000003,"15min":0.69799999999999995},"min":{"1min":0.52300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66100000000000003,"15min":0.69899999999999995},"min":{"1min":0.51600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91300000000000003,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.69099999999999995}]},{"osd":1,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65200000000000002,"15min":0.69499999999999995},"min":{"1min":0.51700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":0.90600000000000003,"15min":0.98599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68400000000000005,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0049999999999999,"5min":1.0600000000000001,"15min":1.111},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.72399999999999998},"min":{"1min":0.57999999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.875,"5min":1.0209999999999999,"15min":1.165},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66500000000000004,"15min":0.71699999999999997},"min":{"1min":0.49299999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95799999999999996,"5min":0.97599999999999998,"15min":1.024},"last":0.628}]},{"osd":3,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65900000000000003,"15min":0.69999999999999996},"min":{"1min":0.497,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.77800000000000002,"5min":0.89900000000000002,"15min":1.0129999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72299999999999998,"15min":0.747},"min":{"1min":0.54900000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.274},"last":0.61599999999999999}]},{"osd":4,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66700000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82999999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.65300000000000002,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.74199999999999999}]},{"osd":6,"last update":"Mon Nov 6 21:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89800000000000002,"5min":1.0680000000000001,"15min":1.1379999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.86799999999999999,"5min":0.96999999999999997,"15min":1.0549999999999999},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575063,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302780,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069060,"statfs":{"total":96636764160,"available":96326717440,"internally_reserved":0,"allocated":1626112,"data_stored":814993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.92200000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66400000000000003,"15min":0.68600000000000005},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93700000000000006,"5min":1.216,"15min":1.216},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65500000000000003,"15min":0.69199999999999995},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90100000000000002,"5min":0.91700000000000004,"15min":0.98399999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.69599999999999995},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.879,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.78000000000000003}]},{"osd":2,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72299999999999998,"15min":0.754},"min":{"1min":0.49299999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91000000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73299999999999998,"15min":0.76200000000000001},"min":{"1min":0.55300000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96399999999999997,"5min":1.05,"15min":1.05},"last":0.68300000000000005}]},{"osd":3,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67800000000000005,"15min":0.70999999999999996},"min":{"1min":0.443,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88500000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70399999999999996,"15min":0.72399999999999998},"min":{"1min":0.50800000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.95099999999999996,"5min":1.002,"15min":1.002},"last":0.58599999999999997}]},{"osd":4,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.626,"15min":0.60899999999999999},"min":{"1min":0.48799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80600000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.873,"5min":0.93300000000000005,"15min":1.012},"last":0.56000000000000005}]},{"osd":5,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.50700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":1.028,"15min":1.028},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.996,"5min":1.04,"15min":1.04},"last":0.748}]},{"osd":7,"last update":"Mon Nov 6 21:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.53400000000000003,"5min":0.45400000000000001,"15min":0.378},"max":{"1min":0.92900000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.56899999999999995,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575063,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302768,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069072,"statfs":{"total":96636764160,"available":96326729728,"internally_reserved":0,"allocated":1613824,"data_stored":813747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.35899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71399999999999997,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.39100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68999999999999995,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63400000000000001,"15min":0.61699999999999999},"min":{"1min":0.441,"5min":0.42899999999999999,"15min":0.41099999999999998},"max":{"1min":0.78500000000000003,"5min":4.431,"15min":4.431},"last":0.51700000000000002}]},{"osd":3,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61099999999999999,"15min":0.59499999999999997},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75900000000000001,"5min":4.5380000000000003,"15min":4.5380000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58499999999999996,"15min":0.56999999999999995},"min":{"1min":0.42299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71899999999999997,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.53300000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0389999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.82099999999999995},"min":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.95699999999999996,"5min":1.883,"15min":1.883},"last":0.746}]},{"osd":5,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.78100000000000003,"15min":0.78600000000000003},"min":{"1min":0.53500000000000003,"5min":0.504,"15min":0.504},"max":{"1min":0.91500000000000004,"5min":4.399,"15min":4.399},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.755,"15min":0.78400000000000003},"min":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.091,"5min":2.649,"15min":2.649},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.96899999999999997,"5min":3.024,"15min":3.024},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79900000000000004,"15min":0.82599999999999996},"min":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96699999999999997,"5min":3.0609999999999999,"15min":3.0609999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Mon Nov 6 21:01:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.57899999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0249999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.84799999999999998,"15min":0.84199999999999997},"min":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.032,"5min":5.8230000000000004,"15min":5.8230000000000004},"last":0.79200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575063,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302760,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069080,"statfs":{"total":96636764160,"available":96326737920,"internally_reserved":0,"allocated":1605632,"data_stored":812833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.45000000000000001,"15min":0.46899999999999997},"min":{"1min":0.34000000000000002,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":0.63900000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.34300000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.34499999999999997}]},{"osd":2,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.71299999999999997,"15min":0.82099999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.55000000000000004},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73799999999999999,"5min":0.77500000000000002,"15min":0.91000000000000003},"last":0.40100000000000002}]},{"osd":3,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53100000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.503},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.746,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.438}]},{"osd":4,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.60099999999999998,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":1.071,"5min":1.1299999999999999,"15min":1.1779999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.61899999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0149999999999999,"5min":1.298,"15min":1.298},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77400000000000002,"15min":0.80800000000000005},"min":{"1min":0.55500000000000005,"5min":0.51000000000000001,"15min":0.48999999999999999},"max":{"1min":0.97899999999999998,"5min":1.069,"15min":1.2010000000000001},"last":0.752},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.80400000000000005,"15min":0.82299999999999995},"min":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.51400000000000001},"max":{"1min":1.0580000000000001,"5min":1.2549999999999999,"15min":1.2829999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76800000000000002,"15min":0.79400000000000004},"min":{"1min":0.60999999999999999,"5min":0.56499999999999995,"15min":0.56299999999999994},"max":{"1min":1.0940000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78400000000000003,"15min":0.81200000000000006},"min":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.304},"last":0.71999999999999997}]},{"osd":7,"last update":"Mon Nov 6 21:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79000000000000004,"15min":0.82099999999999995},"min":{"1min":0.66100000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.1379999999999999,"5min":1.171,"15min":1.2490000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81699999999999995,"15min":0.83599999999999997},"min":{"1min":0.60099999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.097,"5min":1.1899999999999999,"15min":1.236},"last":0.79400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575063,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302768,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069072,"statfs":{"total":96636764160,"available":96326729728,"internally_reserved":0,"allocated":1613824,"data_stored":812945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.308,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.376},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.48499999999999999,"15min":0.49399999999999999},"min":{"1min":0.32900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.81699999999999995},"last":0.45500000000000002}]},{"osd":1,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85899999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84199999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.745,"5min":2.266,"15min":2.266},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.42699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78700000000000003,"5min":1.732,"15min":1.732},"last":0.42399999999999999}]},{"osd":4,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.63800000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.1890000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.752,"15min":0.77700000000000002},"min":{"1min":0.57599999999999996,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0149999999999999,"5min":1.087,"15min":1.357},"last":0.66400000000000003}]},{"osd":5,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.46300000000000002},"max":{"1min":1.1619999999999999,"5min":1.657,"15min":1.657},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79800000000000004,"15min":0.80900000000000005},"min":{"1min":0.59499999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0129999999999999,"5min":1.159,"15min":1.159},"last":0.61099999999999999}]},{"osd":6,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.78600000000000003,"15min":0.80700000000000005},"min":{"1min":0.64600000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.107,"5min":1.1499999999999999,"15min":1.2969999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80300000000000005,"15min":0.81399999999999995},"min":{"1min":0.59899999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.095,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Mon Nov 6 21:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.80800000000000005,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1819999999999999,"5min":1.1870000000000001,"15min":1.448},"last":0.623},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82599999999999996,"15min":0.83099999999999996},"min":{"1min":0.58799999999999997,"5min":0.54100000000000004,"15min":0.52100000000000002},"max":{"1min":1.093,"5min":2.202,"15min":2.202},"last":0.67600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575064,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1617920,"data_stored":814031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44400000000000001,"15min":0.45800000000000002},"min":{"1min":0.33400000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64800000000000002,"5min":0.71399999999999997,"15min":0.82599999999999996},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.50900000000000001,"15min":0.51600000000000001},"min":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.32600000000000001},"max":{"1min":0.68400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48199999999999998,"15min":0.48799999999999999},"min":{"1min":0.33500000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.753},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.56200000000000006},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.79900000000000004,"15min":0.82999999999999996},"last":0.441}]},{"osd":2,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.46899999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73799999999999999,"5min":0.80100000000000005,"15min":0.91000000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1990000000000001,"5min":1.234,"15min":1.2470000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80000000000000004,"15min":0.81399999999999995},"min":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.47999999999999998},"max":{"1min":1.222,"5min":1.276,"15min":1.286},"last":0.92800000000000005}]},{"osd":5,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76300000000000001,"15min":0.78600000000000003},"min":{"1min":0.56299999999999994,"5min":0.496,"15min":0.496},"max":{"1min":0.94199999999999995,"5min":1.1519999999999999,"15min":1.1779999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77300000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.86199999999999999}]},{"osd":6,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.77000000000000002,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1539999999999999,"5min":1.323,"15min":1.323},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.79600000000000004,"15min":0.81799999999999995},"min":{"1min":0.66100000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.044,"5min":1.054,"15min":1.2130000000000001},"last":0.89100000000000001}]},{"osd":7,"last update":"Mon Nov 6 21:01:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78600000000000003,"15min":0.81599999999999995},"min":{"1min":0.58099999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.0780000000000001,"5min":1.0920000000000001,"15min":1.302},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.81200000000000006,"15min":0.82899999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.876}]}]},{"osd":4,"up_from":13,"seq":55834575063,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302764,"kb_used_data":1572,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069076,"statfs":{"total":96636764160,"available":96326733824,"internally_reserved":0,"allocated":1609728,"data_stored":812867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64000000000000001,"15min":0.68899999999999995},"min":{"1min":0.48399999999999999,"5min":0.39500000000000002,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.63700000000000001,"15min":0.67400000000000004},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78500000000000003,"5min":1,"15min":1},"last":0.61799999999999999}]},{"osd":1,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65800000000000003,"15min":0.69999999999999996},"min":{"1min":0.47899999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80600000000000005,"5min":1.645,"15min":1.645},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68999999999999995,"15min":0.71299999999999997},"min":{"1min":0.45600000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83599999999999997,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73399999999999999}]},{"osd":2,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67600000000000005,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81599999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.748,"15min":0.77500000000000002},"min":{"1min":0.625,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.81499999999999995,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65400000000000003,"15min":0.69199999999999995},"min":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81100000000000005,"5min":0.90300000000000002,"15min":0.95799999999999996},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66900000000000004,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89300000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71399999999999997}]},{"osd":5,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77900000000000003,"5min":0.88200000000000001,"15min":1.0389999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.625,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.40799999999999997},"max":{"1min":0.80200000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.437}]},{"osd":6,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64000000000000001,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.93100000000000005,"5min":1.22,"15min":1.22},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70899999999999996,"15min":0.66900000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83699999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Nov 6 21:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68100000000000005,"15min":0.66000000000000003},"min":{"1min":0.49399999999999999,"5min":0.45600000000000002,"15min":0.44},"max":{"1min":0.83799999999999997,"5min":1.034,"15min":1.034},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.67800000000000005},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.45400000000000001},"max":{"1min":0.86699999999999999,"5min":1.177,"15min":1.177},"last":0.56699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575064,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1617920,"data_stored":813027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.67200000000000004},"min":{"1min":0.45800000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.90300000000000002,"5min":0.94099999999999995,"15min":1.0109999999999999},"last":0.629},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64200000000000002,"15min":0.67700000000000005},"min":{"1min":0.45400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88600000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66200000000000003}]},{"osd":1,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90900000000000003,"5min":0.92300000000000004,"15min":0.99399999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72999999999999998,"15min":0.753},"min":{"1min":0.54500000000000004,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.034},"last":0.72999999999999998}]},{"osd":2,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69199999999999995,"15min":0.73699999999999999},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93899999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.748},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46000000000000002,"15min":0.45600000000000002},"max":{"1min":0.92800000000000005,"5min":0.98099999999999998,"15min":1.0089999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.70299999999999996},"min":{"1min":0.46800000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.95399999999999996,"5min":0.998,"15min":0.998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.54400000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.94199999999999995,"5min":1.175,"15min":1.175},"last":0.79300000000000004}]},{"osd":4,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80800000000000005,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.78700000000000003}]},{"osd":6,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64700000000000002,"15min":0.626},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0009999999999999,"5min":1.214,"15min":1.214},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70799999999999996,"15min":0.71199999999999997},"min":{"1min":0.48899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95499999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Mon Nov 6 21:01:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.64900000000000002},"min":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.40600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68400000000000005,"15min":0.68000000000000005},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97299999999999998,"5min":1.258,"15min":1.258},"last":0.70099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T21:02:16.823 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T21:02:16.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T21:02:17.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.060+0000 7fdea3fff700 1 Processor -- start 2023-11-06T21:02:17.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.060+0000 7fdea3fff700 1 -- start start 2023-11-06T21:02:17.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.061+0000 7fdea3fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fde9c14e800 0x7fde9c0c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.061+0000 7fdea3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde9c0c7b30 0x7fde9c1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.061+0000 7fdea3fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c0ad980 con 0x7fde9c14e420 2023-11-06T21:02:17.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.061+0000 7fdea3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c158880 con 0x7fde9c0c7b30 2023-11-06T21:02:17.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.061+0000 7fdea3fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c157c90 con 0x7fde9c14e800 2023-11-06T21:02:17.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.061+0000 7fdea259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde9c0c7b30 0x7fde9c1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.061+0000 7fdea259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde9c0c7b30 0x7fde9c1546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37168/0 (socket says 172.21.15.33:37168) 2023-11-06T21:02:17.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.061+0000 7fdea259c700 1 -- 172.21.15.33:0/1472765852 learned_addr learned my addr 172.21.15.33:0/1472765852 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:17.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.062+0000 7fdea159a700 1 --2- 172.21.15.33:0/1472765852 >> v2:172.21.15.69:3300/0 conn(0x7fde9c14e800 0x7fde9c0c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.062+0000 7fdea259c700 1 -- 172.21.15.33:0/1472765852 >> v2:172.21.15.69:3300/0 conn(0x7fde9c14e800 msgr2=0x7fde9c0c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:17.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.062+0000 7fdea259c700 1 --2- 172.21.15.33:0/1472765852 >> v2:172.21.15.69:3300/0 conn(0x7fde9c14e800 0x7fde9c0c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.062+0000 7fdea259c700 1 -- 172.21.15.33:0/1472765852 >> v1:172.21.15.69:6789/0 conn(0x7fde9c14e420 legacy=0x7fde9c0c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.062+0000 7fdea259c700 1 -- 172.21.15.33:0/1472765852 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c157f50 con 0x7fde9c0c7b30 2023-11-06T21:02:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.062+0000 7fdea259c700 1 --2- 172.21.15.33:0/1472765852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 0x7fde9c1546c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fde9c0c8c00 tx=0x7fde94009840).ready entity=mon.0 client_cookie=5fe63c7d89835f64 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:17.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.062+0000 7fdea0d99700 1 -- 172.21.15.33:0/1472765852 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400e070 con 0x7fde9c0c7b30 2023-11-06T21:02:17.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.062+0000 7fdea0d99700 1 -- 172.21.15.33:0/1472765852 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9400ba50 con 0x7fde9c0c7b30 2023-11-06T21:02:17.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.062+0000 7fdea0d99700 1 -- 172.21.15.33:0/1472765852 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400bdd0 con 0x7fde9c0c7b30 2023-11-06T21:02:17.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.063+0000 7fdea3fff700 1 -- 172.21.15.33:0/1472765852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 msgr2=0x7fde9c1546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.063+0000 7fdea3fff700 1 --2- 172.21.15.33:0/1472765852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 0x7fde9c1546c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fde9c0c8c00 tx=0x7fde94009840).stop 2023-11-06T21:02:17.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.063+0000 7fdea3fff700 1 -- 172.21.15.33:0/1472765852 shutdown_connections 2023-11-06T21:02:17.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.063+0000 7fdea3fff700 1 --2- 172.21.15.33:0/1472765852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 0x7fde9c1546c0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.063+0000 7fdea3fff700 1 --2- 172.21.15.33:0/1472765852 >> v2:172.21.15.69:3300/0 conn(0x7fde9c14e800 0x7fde9c0c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.063+0000 7fdea3fff700 1 -- 172.21.15.33:0/1472765852 >> 172.21.15.33:0/1472765852 conn(0x7fde9c05e1b0 msgr2=0x7fde9c05e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:17.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.063+0000 7fdea3fff700 1 -- 172.21.15.33:0/1472765852 shutdown_connections 2023-11-06T21:02:17.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.063+0000 7fdea3fff700 1 -- 172.21.15.33:0/1472765852 wait complete. 2023-11-06T21:02:17.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.065+0000 7fdea3fff700 1 Processor -- start 2023-11-06T21:02:17.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.065+0000 7fdea3fff700 1 -- start start 2023-11-06T21:02:17.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 0x7fde9c0c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde9c14e420 0x7fde9c14b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fde9c14e800 0x7fde9c0c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0c3af0 con 0x7fde9c0c7b30 2023-11-06T21:02:17.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea3fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0c3dd0 con 0x7fde9c14e420 2023-11-06T21:02:17.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea3fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0c40b0 con 0x7fde9c14e800 2023-11-06T21:02:17.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 0x7fde9c0c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 0x7fde9c0c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37174/0 (socket says 172.21.15.33:37174) 2023-11-06T21:02:17.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea1d9b700 1 -- 172.21.15.33:0/1828553680 learned_addr learned my addr 172.21.15.33:0/1828553680 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:17.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea159a700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde9c14e420 0x7fde9c14b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea259c700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fde9c14e800 0x7fde9c0c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea1d9b700 1 -- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fde9c14e800 msgr2=0x7fde9c0c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea1d9b700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fde9c14e800 0x7fde9c0c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea1d9b700 1 -- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde9c14e420 msgr2=0x7fde9c14b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea1d9b700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde9c14e420 0x7fde9c14b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.066+0000 7fdea1d9b700 1 -- 172.21.15.33:0/1828553680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde94009ae0 con 0x7fde9c0c7b30 2023-11-06T21:02:17.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.067+0000 7fdea1d9b700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 0x7fde9c0c32a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7fde9800c4c0 tx=0x7fde98007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:17.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.067+0000 7fde91ffb700 1 -- 172.21.15.33:0/1828553680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9800ce70 con 0x7fde9c0c7b30 2023-11-06T21:02:17.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.067+0000 7fde91ffb700 1 -- 172.21.15.33:0/1828553680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde98001e90 con 0x7fde9c0c7b30 2023-11-06T21:02:17.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.067+0000 7fdea3fff700 1 -- 172.21.15.33:0/1828553680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c0c4390 con 0x7fde9c0c7b30 2023-11-06T21:02:17.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.067+0000 7fde91ffb700 1 -- 172.21.15.33:0/1828553680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98007250 con 0x7fde9c0c7b30 2023-11-06T21:02:17.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.067+0000 7fdea3fff700 1 -- 172.21.15.33:0/1828553680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c0c3790 con 0x7fde9c0c7b30 2023-11-06T21:02:17.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.068+0000 7fdea3fff700 1 -- 172.21.15.33:0/1828553680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c0aec70 con 0x7fde9c0c7b30 2023-11-06T21:02:17.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.069+0000 7fde91ffb700 1 -- 172.21.15.33:0/1828553680 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fde980066a0 con 0x7fde9c0c7b30 2023-11-06T21:02:17.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.070+0000 7fde91ffb700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fde8005f270 0x7fde80061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.070+0000 7fde91ffb700 1 -- 172.21.15.33:0/1828553680 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fde98079c40 con 0x7fde9c0c7b30 2023-11-06T21:02:17.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.070+0000 7fdea159a700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fde8005f270 0x7fde80061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.071+0000 7fdea159a700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fde8005f270 0x7fde80061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fde9c0c1cb0 tx=0x7fde8c008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:17.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.074+0000 7fde91ffb700 1 -- 172.21.15.33:0/1828553680 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fde9807f020 con 0x7fde9c0c7b30 2023-11-06T21:02:17.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.318+0000 7fdea3fff700 1 -- 172.21.15.33:0/1828553680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fde9c06c6c0 con 0x7fde9c0c7b30 2023-11-06T21:02:17.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.322+0000 7fde91ffb700 1 -- 172.21.15.33:0/1828553680 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fde9804e7c0 con 0x7fde9c0c7b30 2023-11-06T21:02:17.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 -- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fde8005f270 msgr2=0x7fde80061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fde8005f270 0x7fde80061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fde9c0c1cb0 tx=0x7fde8c008040).stop 2023-11-06T21:02:17.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 -- 172.21.15.33:0/1828553680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 msgr2=0x7fde9c0c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 0x7fde9c0c32a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7fde9800c4c0 tx=0x7fde98007e00).stop 2023-11-06T21:02:17.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 -- 172.21.15.33:0/1828553680 shutdown_connections 2023-11-06T21:02:17.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fde8005f270 0x7fde80061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fde9c14e800 0x7fde9c0c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c0c7b30 0x7fde9c0c32a0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 --2- 172.21.15.33:0/1828553680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde9c14e420 0x7fde9c14b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 -- 172.21.15.33:0/1828553680 >> 172.21.15.33:0/1828553680 conn(0x7fde9c05e1b0 msgr2=0x7fde9c0ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 -- 172.21.15.33:0/1828553680 shutdown_connections 2023-11-06T21:02:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.324+0000 7fdea3fff700 1 -- 172.21.15.33:0/1828553680 wait complete. 2023-11-06T21:02:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T21:02:17.342 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:02:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.538+0000 7f8661dff700 1 Processor -- start 2023-11-06T21:02:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.538+0000 7f8661dff700 1 -- start start 2023-11-06T21:02:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.539+0000 7f8661dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c05e1b0 0x7f865c063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.539+0000 7f8661dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f865c0c3680 0x7f865c154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.540+0000 7f8661dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c0c31f0 con 0x7f865c063140 2023-11-06T21:02:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.540+0000 7f8661dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c06a0f0 con 0x7f865c0c3680 2023-11-06T21:02:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.540+0000 7f8661dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0c2920 con 0x7f865c05e1b0 2023-11-06T21:02:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.540+0000 7f865b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f865c063140 0x7f865c0c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42174/0 (socket says 172.21.15.33:42174) 2023-11-06T21:02:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.540+0000 7f865b7fe700 1 -- 172.21.15.33:0/1162800531 learned_addr learned my addr 172.21.15.33:0/1162800531 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.540+0000 7f865bfff700 1 --2- 172.21.15.33:0/1162800531 >> v2:172.21.15.69:3300/0 conn(0x7f865c0c3680 0x7f865c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.540+0000 7f865affd700 1 --2- 172.21.15.33:0/1162800531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c05e1b0 0x7f865c063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.540+0000 7f865a7fc700 1 -- 172.21.15.33:0/1162800531 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673113746 0 0) 0x7f865c0c31f0 con 0x7f865c063140 2023-11-06T21:02:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.540+0000 7f865a7fc700 1 -- 172.21.15.33:0/1162800531 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86440033f0 con 0x7f865c063140 2023-11-06T21:02:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.541+0000 7f865bfff700 1 -- 172.21.15.33:0/1162800531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c05e1b0 msgr2=0x7f865c063520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.541+0000 7f865bfff700 1 --2- 172.21.15.33:0/1162800531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c05e1b0 0x7f865c063520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.541+0000 7f865bfff700 1 -- 172.21.15.33:0/1162800531 >> v1:172.21.15.33:6789/0 conn(0x7f865c063140 legacy=0x7f865c0c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.541+0000 7f865bfff700 1 -- 172.21.15.33:0/1162800531 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c05e640 con 0x7f865c0c3680 2023-11-06T21:02:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.541+0000 7f865affd700 1 --2- 172.21.15.33:0/1162800531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c05e1b0 0x7f865c063520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:02:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.541+0000 7f865bfff700 1 --2- 172.21.15.33:0/1162800531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f865c0c3680 0x7f865c154050 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f8654009e60 tx=0x7f8654009780).ready entity=mon.1 client_cookie=abc48466369134dd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.541+0000 7f865a7fc700 1 -- 172.21.15.33:0/1162800531 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400e070 con 0x7f865c0c3680 2023-11-06T21:02:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.541+0000 7f865a7fc700 1 -- 172.21.15.33:0/1162800531 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865400bab0 con 0x7f865c0c3680 2023-11-06T21:02:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.541+0000 7f865a7fc700 1 -- 172.21.15.33:0/1162800531 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400be30 con 0x7f865c0c3680 2023-11-06T21:02:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.542+0000 7f8661dff700 1 -- 172.21.15.33:0/1162800531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f865c0c3680 msgr2=0x7f865c154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.542+0000 7f8661dff700 1 --2- 172.21.15.33:0/1162800531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f865c0c3680 0x7f865c154050 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f8654009e60 tx=0x7f8654009780).stop 2023-11-06T21:02:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.542+0000 7f8661dff700 1 -- 172.21.15.33:0/1162800531 shutdown_connections 2023-11-06T21:02:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.542+0000 7f8661dff700 1 --2- 172.21.15.33:0/1162800531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f865c0c3680 0x7f865c154050 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.542+0000 7f8661dff700 1 --2- 172.21.15.33:0/1162800531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c05e1b0 0x7f865c063520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.542+0000 7f8661dff700 1 -- 172.21.15.33:0/1162800531 >> 172.21.15.33:0/1162800531 conn(0x7f865c0c7d60 msgr2=0x7f865c0c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.542+0000 7f8661dff700 1 -- 172.21.15.33:0/1162800531 shutdown_connections 2023-11-06T21:02:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.542+0000 7f8661dff700 1 -- 172.21.15.33:0/1162800531 wait complete. 2023-11-06T21:02:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.543+0000 7f8661dff700 1 Processor -- start 2023-11-06T21:02:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.543+0000 7f8661dff700 1 -- start start 2023-11-06T21:02:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.544+0000 7f8661dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f865c063140 0x7f865c164640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.544+0000 7f8661dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f865c0c3680 0x7f865c141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.544+0000 7f8661dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c142250 0x7f865c1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.544+0000 7f8661dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1553a0 con 0x7f865c0c3680 2023-11-06T21:02:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.544+0000 7f8661dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c14f660 con 0x7f865c063140 2023-11-06T21:02:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.544+0000 7f8661dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1581e0 con 0x7f865c142250 2023-11-06T21:02:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f865c0c3680 0x7f865c141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f865c063140 0x7f865c164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f865c0c3680 0x7f865c141d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37186/0 (socket says 172.21.15.33:37186) 2023-11-06T21:02:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865affd700 1 -- 172.21.15.33:0/255991650 learned_addr learned my addr 172.21.15.33:0/255991650 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865bfff700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c142250 0x7f865c1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865affd700 1 -- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c142250 msgr2=0x7f865c1547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865affd700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c142250 0x7f865c1547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865affd700 1 -- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f865c063140 msgr2=0x7f865c164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865affd700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f865c063140 0x7f865c164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865affd700 1 -- 172.21.15.33:0/255991650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500092a0 con 0x7f865c0c3680 2023-11-06T21:02:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.545+0000 7f865b7fe700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f865c063140 0x7f865c164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:02:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.546+0000 7f865affd700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f865c0c3680 0x7f865c141d60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f865000e930 tx=0x7f8650009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.546+0000 7f86437fe700 1 -- 172.21.15.33:0/255991650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650012600 con 0x7f865c0c3680 2023-11-06T21:02:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.546+0000 7f86437fe700 1 -- 172.21.15.33:0/255991650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8650012760 con 0x7f865c0c3680 2023-11-06T21:02:17.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.546+0000 7f86437fe700 1 -- 172.21.15.33:0/255991650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650012ae0 con 0x7f865c0c3680 2023-11-06T21:02:17.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.546+0000 7f8661dff700 1 -- 172.21.15.33:0/255991650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8654009a20 con 0x7f865c0c3680 2023-11-06T21:02:17.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.546+0000 7f8661dff700 1 -- 172.21.15.33:0/255991650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c158820 con 0x7f865c0c3680 2023-11-06T21:02:17.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.547+0000 7f8661dff700 1 -- 172.21.15.33:0/255991650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c064420 con 0x7f865c0c3680 2023-11-06T21:02:17.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.548+0000 7f86437fe700 1 -- 172.21.15.33:0/255991650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f865001b400 con 0x7f865c0c3680 2023-11-06T21:02:17.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.549+0000 7f86437fe700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f864805f270 0x7f8648061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:17.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.549+0000 7f86437fe700 1 -- 172.21.15.33:0/255991650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f865007a7f0 con 0x7f865c0c3680 2023-11-06T21:02:17.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.549+0000 7f865b7fe700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f864805f270 0x7f8648061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:17.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.550+0000 7f865b7fe700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f864805f270 0x7f8648061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f864c005750 tx=0x7f864c009040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.553+0000 7f86437fe700 1 -- 172.21.15.33:0/255991650 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f865004b8e0 con 0x7f865c0c3680 2023-11-06T21:02:17.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.775+0000 7f8661dff700 1 -- 172.21.15.33:0/255991650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f865c06bfd0 con 0x7f865c0c3680 2023-11-06T21:02:17.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.775+0000 7f86437fe700 1 -- 172.21.15.33:0/255991650 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f8650016020 con 0x7f865c0c3680 2023-11-06T21:02:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:17.776 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":53,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2500558031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2500558031},{"type":"v1","addr":"172.21.15.69:6837","nonce":2500558031}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1012474146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1012474146},{"type":"v1","addr":"172.21.15.69:6836","nonce":1012474146}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4020289695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4020289695},{"type":"v1","addr":"172.21.15.33:6834","nonce":4020289695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:02:11.991421+0000","modified":"2023-11-06T21:02:15.032423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1130278640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1130278640},{"type":"v1","addr":"172.21.15.33:6835","nonce":1130278640}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-06T21:02:17.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 -- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f864805f270 msgr2=0x7f8648061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f864805f270 0x7f8648061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f864c005750 tx=0x7f864c009040).stop 2023-11-06T21:02:17.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 -- 172.21.15.33:0/255991650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f865c0c3680 msgr2=0x7f865c141d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:17.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f865c0c3680 0x7f865c141d60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f865000e930 tx=0x7f8650009060).stop 2023-11-06T21:02:17.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 -- 172.21.15.33:0/255991650 shutdown_connections 2023-11-06T21:02:17.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f864805f270 0x7f8648061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f865c142250 0x7f865c1547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f865c063140 0x7f865c164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 --2- 172.21.15.33:0/255991650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f865c0c3680 0x7f865c141d60 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:17.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 -- 172.21.15.33:0/255991650 >> 172.21.15.33:0/255991650 conn(0x7f865c0c7d60 msgr2=0x7f865c151570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:17.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 -- 172.21.15.33:0/255991650 shutdown_connections 2023-11-06T21:02:17.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:17.777+0000 7f8661dff700 1 -- 172.21.15.33:0/255991650 wait complete. 2023-11-06T21:02:17.781 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 53 2023-11-06T21:02:18.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T21:02:11.991421+0000', 'modified': '2023-11-06T21:02:15.032423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7527}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7527': {'gid': 7527, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/1130278640', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6832', 'nonce': 1130278640}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 1130278640}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T21:02:18.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T21:02:18.199 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T21:02:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.397+0000 7f2c3d9ff700 1 Processor -- start 2023-11-06T21:02:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.397+0000 7f2c3d9ff700 1 -- start start 2023-11-06T21:02:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.399+0000 7f2c3d9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 0x7f2c38063560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.399+0000 7f2c3d9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2c380bfd50 0x7f2c380c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.399+0000 7f2c3d9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c381553d0 con 0x7f2c381544a0 2023-11-06T21:02:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.399+0000 7f2c3d9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c38154880 con 0x7f2c380bfd50 2023-11-06T21:02:18.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.399+0000 7f2c3d9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c38063a50 con 0x7f2c380630e0 2023-11-06T21:02:18.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.399+0000 7f2c377fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2c380bfd50 0x7f2c380c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.399+0000 7f2c367fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 0x7f2c38063560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.399+0000 7f2c367fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 0x7f2c38063560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:56908/0 (socket says 172.21.15.33:56908) 2023-11-06T21:02:18.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.399+0000 7f2c367fc700 1 -- 172.21.15.33:0/560621709 learned_addr learned my addr 172.21.15.33:0/560621709 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:18.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.400+0000 7f2c377fe700 1 -- 172.21.15.33:0/560621709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 msgr2=0x7f2c38063560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:18.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.400+0000 7f2c377fe700 1 --2- 172.21.15.33:0/560621709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 0x7f2c38063560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.400+0000 7f2c377fe700 1 -- 172.21.15.33:0/560621709 >> v1:172.21.15.69:6789/0 conn(0x7f2c381544a0 legacy=0x7f2c380c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.400+0000 7f2c377fe700 1 -- 172.21.15.33:0/560621709 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c380c6710 con 0x7f2c380bfd50 2023-11-06T21:02:18.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.400+0000 7f2c35ffb700 1 -- 172.21.15.33:0/560621709 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454957588 0 0) 0x7f2c381553d0 con 0x7f2c381544a0 2023-11-06T21:02:18.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.400+0000 7f2c367fc700 1 --2- 172.21.15.33:0/560621709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 0x7f2c38063560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:02:18.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.400+0000 7f2c377fe700 1 --2- 172.21.15.33:0/560621709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c380bfd50 0x7f2c380c01e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f2c30009e60 tx=0x7f2c30009780).ready entity=mon.1 client_cookie=fbb7cd54d137ade8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:18.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.400+0000 7f2c35ffb700 1 -- 172.21.15.33:0/560621709 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3000e070 con 0x7f2c380bfd50 2023-11-06T21:02:18.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.400+0000 7f2c35ffb700 1 -- 172.21.15.33:0/560621709 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3000bb60 con 0x7f2c380bfd50 2023-11-06T21:02:18.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.400+0000 7f2c35ffb700 1 -- 172.21.15.33:0/560621709 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30012410 con 0x7f2c380bfd50 2023-11-06T21:02:18.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.401+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/560621709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c380bfd50 msgr2=0x7f2c380c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.401+0000 7f2c3d9ff700 1 --2- 172.21.15.33:0/560621709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c380bfd50 0x7f2c380c01e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f2c30009e60 tx=0x7f2c30009780).stop 2023-11-06T21:02:18.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.401+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/560621709 shutdown_connections 2023-11-06T21:02:18.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.401+0000 7f2c3d9ff700 1 --2- 172.21.15.33:0/560621709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c380bfd50 0x7f2c380c01e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.401+0000 7f2c3d9ff700 1 --2- 172.21.15.33:0/560621709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 0x7f2c38063560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.401+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/560621709 >> 172.21.15.33:0/560621709 conn(0x7f2c3805ec80 msgr2=0x7f2c380cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.401+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/560621709 shutdown_connections 2023-11-06T21:02:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.401+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/560621709 wait complete. 2023-11-06T21:02:18.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.403+0000 7f2c3d9ff700 1 Processor -- start 2023-11-06T21:02:18.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.403+0000 7f2c3d9ff700 1 -- start start 2023-11-06T21:02:18.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c3d9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 0x7f2c38164390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c3d9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c380bfd50 0x7f2c380a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c3d9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c381544a0 0x7f2c380a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c3d9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380a47c0 con 0x7f2c380bfd50 2023-11-06T21:02:18.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c3d9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380afb00 con 0x7f2c381544a0 2023-11-06T21:02:18.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c3d9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380aeef0 con 0x7f2c380630e0 2023-11-06T21:02:18.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c367fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c380bfd50 0x7f2c380a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c367fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c380bfd50 0x7f2c380a3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59922/0 (socket says 172.21.15.33:59922) 2023-11-06T21:02:18.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c367fc700 1 -- 172.21.15.33:0/3322725254 learned_addr learned my addr 172.21.15.33:0/3322725254 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:18.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 0x7f2c38164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c377fe700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c381544a0 0x7f2c380a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c367fc700 1 -- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 msgr2=0x7f2c38164390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c367fc700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 0x7f2c38164390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c367fc700 1 -- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c381544a0 msgr2=0x7f2c380a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c367fc700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c381544a0 0x7f2c380a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c367fc700 1 -- 172.21.15.33:0/3322725254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c009370 con 0x7f2c380bfd50 2023-11-06T21:02:18.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.404+0000 7f2c377fe700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c381544a0 0x7f2c380a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:02:18.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.405+0000 7f2c367fc700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c380bfd50 0x7f2c380a3de0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f2c2c006d00 tx=0x7f2c2c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:18.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.405+0000 7f2c1effd700 1 -- 172.21.15.33:0/3322725254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00c6a0 con 0x7f2c380bfd50 2023-11-06T21:02:18.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.405+0000 7f2c1effd700 1 -- 172.21.15.33:0/3322725254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2c00c800 con 0x7f2c380bfd50 2023-11-06T21:02:18.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.405+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/3322725254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c30009a20 con 0x7f2c380bfd50 2023-11-06T21:02:18.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.405+0000 7f2c1effd700 1 -- 172.21.15.33:0/3322725254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00cb80 con 0x7f2c380bfd50 2023-11-06T21:02:18.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.405+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/3322725254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c380af4b0 con 0x7f2c380bfd50 2023-11-06T21:02:18.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.408+0000 7f2c1effd700 1 -- 172.21.15.33:0/3322725254 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2c2c01b400 con 0x7f2c380bfd50 2023-11-06T21:02:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.409+0000 7f2c1effd700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c2405f270 0x7f2c24061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.409+0000 7f2c1effd700 1 -- 172.21.15.33:0/3322725254 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2c2c07a8b0 con 0x7f2c380bfd50 2023-11-06T21:02:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.409+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c2405f270 0x7f2c24061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.410+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c2405f270 0x7f2c24061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2c28007fd0 tx=0x7f2c28007bb0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.410+0000 7f2c54dc7b80 1 Processor -- start 2023-11-06T21:02:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.411+0000 7f2c54dc7b80 1 -- start start 2023-11-06T21:02:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.411+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac9127e080 0x55ac9127e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.411+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 0x55ac91278ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.411+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912793e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.411+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac911ba2f0 con 0x55ac912a64c0 2023-11-06T21:02:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.411+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac911cf7e0 con 0x55ac9127e080 2023-11-06T21:02:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac911cf920 con 0x55ac912c6be0 2023-11-06T21:02:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912793e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59928/0 (socket says 172.21.15.33:59928) 2023-11-06T21:02:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c377fe700 1 -- 172.21.15.33:0/3435480718 learned_addr learned my addr 172.21.15.33:0/3435480718 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/3435480718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac9127e080 0x55ac9127e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c367fc700 1 --2- 172.21.15.33:0/3435480718 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 0x55ac91278ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c377fe700 1 -- 172.21.15.33:0/3435480718 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 msgr2=0x55ac91278ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c377fe700 1 --2- 172.21.15.33:0/3435480718 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 0x55ac91278ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c377fe700 1 -- 172.21.15.33:0/3435480718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac9127e080 msgr2=0x55ac9127e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c377fe700 1 --2- 172.21.15.33:0/3435480718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac9127e080 0x55ac9127e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c377fe700 1 -- 172.21.15.33:0/3435480718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac911c8720 con 0x55ac912a64c0 2023-11-06T21:02:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c367fc700 1 --2- 172.21.15.33:0/3435480718 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 0x55ac91278ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.412+0000 7f2c377fe700 1 --2- 172.21.15.33:0/3435480718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912793e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f2c3000b6c0 tx=0x7f2c30009780).ready entity=mon.0 client_cookie=59b42f792262483c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c17fff700 1 -- 172.21.15.33:0/3435480718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3000bda0 con 0x55ac912a64c0 2023-11-06T21:02:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c17fff700 1 -- 172.21.15.33:0/3435480718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c30012780 con 0x55ac912a64c0 2023-11-06T21:02:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c17fff700 1 -- 172.21.15.33:0/3435480718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30012b00 con 0x55ac912a64c0 2023-11-06T21:02:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/3435480718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 msgr2=0x55ac912793e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/3435480718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912793e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f2c3000b6c0 tx=0x7f2c30009780).stop 2023-11-06T21:02:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/3435480718 shutdown_connections 2023-11-06T21:02:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/3435480718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912793e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/3435480718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac9127e080 0x55ac9127e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/3435480718 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 0x55ac91278ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/3435480718 >> 172.21.15.33:0/3435480718 conn(0x55ac910e7940 msgr2=0x55ac912c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.413+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/3435480718 shutdown_connections 2023-11-06T21:02:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.414+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/3435480718 wait complete. 2023-11-06T21:02:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.414+0000 7f2c54dc7b80 1 -- start start 2023-11-06T21:02:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.414+0000 7f2c54dc7b80 1 Processor -- start 2023-11-06T21:02:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.415+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 0x55ac912a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912c6be0 0x55ac912c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac911cfba0 con 0x55ac912a64c0 2023-11-06T21:02:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9119ae60 con 0x55ac912c6be0 2023-11-06T21:02:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac912c3e00 con 0x55ac9127e080 2023-11-06T21:02:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c54dc7b80 10 client.0 ll_register_callbacks cb 0x55ac9116fce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T21:02:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c367fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c367fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59934/0 (socket says 172.21.15.33:59934) 2023-11-06T21:02:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c367fc700 1 -- 172.21.15.33:0/2760766668 learned_addr learned my addr 172.21.15.33:0/2760766668 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c367fc700 1 -- 172.21.15.33:0/2760766668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 msgr2=0x55ac912a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 0x55ac912a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c377fe700 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912c6be0 0x55ac912c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c367fc700 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 0x55ac912a2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c367fc700 1 -- 172.21.15.33:0/2760766668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912c6be0 msgr2=0x55ac912c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:18.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c367fc700 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912c6be0 0x55ac912c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c367fc700 1 -- 172.21.15.33:0/2760766668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c380c6710 con 0x55ac912a64c0 2023-11-06T21:02:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 0x55ac912a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c377fe700 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912c6be0 0x55ac912c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.416+0000 7f2c367fc700 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912aa020 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f2c2c078c60 tx=0x7f2c2c011000).ready entity=mon.0 client_cookie=e1d9b9d6330bf3a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c157fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T21:02:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c157fa700 1 -- 172.21.15.33:0/2760766668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c04dad0 con 0x55ac912a64c0 2023-11-06T21:02:18.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c157fa700 1 -- 172.21.15.33:0/2760766668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2c2c05a060 con 0x55ac912a64c0 2023-11-06T21:02:18.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2760766668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac91271e20 con 0x55ac912a64c0 2023-11-06T21:02:18.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c157fa700 1 -- 172.21.15.33:0/2760766668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2c04ed60 con 0x55ac912a64c0 2023-11-06T21:02:18.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c54dc7b80 10 client.7674 fetch_fsmap learned FSMap version 53 2023-11-06T21:02:18.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2760766668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55ac911c3ac0 con 0x55ac912a64c0 2023-11-06T21:02:18.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c157fa700 1 -- 172.21.15.33:0/2760766668 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac91271e20 con 0x55ac912a64c0 2023-11-06T21:02:18.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c157fa700 1 -- 172.21.15.33:0/2760766668 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 53) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2c2c04b450 con 0x55ac912a64c0 2023-11-06T21:02:18.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c54dc7b80 10 client.7674 fetch_fsmap finished waiting for FSMap version 53 2023-11-06T21:02:18.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c54dc7b80 10 client.7674 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T21:02:18.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c54dc7b80 20 client.7674 populate_metadata read hostname 'smithi033' 2023-11-06T21:02:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.33:6832/1130278640,v1:172.21.15.33:6835/1130278640] conn(0x55ac912c6190 0x55ac912d64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c54dc7b80 4 client.7674 mds_command: new command op to 7527 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T21:02:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.417+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2760766668 --> [v2:172.21.15.33:6832/1130278640,v1:172.21.15.33:6835/1130278640] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac91037ef0 con 0x55ac912c6190 2023-11-06T21:02:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.418+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.33:6832/1130278640,v1:172.21.15.33:6835/1130278640] conn(0x55ac912c6190 0x55ac912d64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.418+0000 7f2c157fa700 10 client.7674 ms_handle_connect on v2:172.21.15.33:6832/1130278640 2023-11-06T21:02:18.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.418+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.33:6832/1130278640,v1:172.21.15.33:6835/1130278640] conn(0x55ac912c6190 0x55ac912d64d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=99c5e87d392fd60e server_cookie=4fbb24314f9f2331 in_seq=0 out_seq=0 2023-11-06T21:02:18.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c157fa700 1 -- 172.21.15.33:0/2760766668 <== mds.0 v2:172.21.15.33:6832/1130278640 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55ac91037ef0 con 0x55ac912c6190 2023-11-06T21:02:18.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c157fa700 10 client.7674 handle_command_reply: tid=0 2023-11-06T21:02:18.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c54dc7b80 1 client.7674 shutdown 2023-11-06T21:02:18.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2760766668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 msgr2=0x55ac912aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912aa020 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f2c2c078c60 tx=0x7f2c2c011000).stop 2023-11-06T21:02:18.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2760766668 shutdown_connections 2023-11-06T21:02:18.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 0x55ac912a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.33:6832/1130278640,v1:172.21.15.33:6835/1130278640] conn(0x55ac912c6190 0x55ac912d64d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912c6be0 0x55ac912c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c157fa700 0 client.7674 ms_handle_reset on v2:172.21.15.33:6832/1130278640 2023-11-06T21:02:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2760766668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912a64c0 0x55ac912aa020 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2760766668 >> 172.21.15.33:0/2760766668 conn(0x55ac910e7940 msgr2=0x55ac9127bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.419+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2760766668 shutdown_connections 2023-11-06T21:02:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.420+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2760766668 wait complete. 2023-11-06T21:02:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.420+0000 7f2c54dc7b80 20 client.7674 trim_cache size 0 max 16384 2023-11-06T21:02:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.437+0000 7f2c54dc7b80 1 Processor -- start 2023-11-06T21:02:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.437+0000 7f2c54dc7b80 1 -- start start 2023-11-06T21:02:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.438+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac9127e080 0x55ac912ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.438+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912a64c0 0x55ac912aba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.438+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 0x55ac9127bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.438+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac91280370 con 0x55ac9127e080 2023-11-06T21:02:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.438+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac912a6b90 con 0x55ac912a64c0 2023-11-06T21:02:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.438+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac9127a940 con 0x55ac912c6be0 2023-11-06T21:02:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c36ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac9127e080 0x55ac912ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c36ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac9127e080 0x55ac912ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59940/0 (socket says 172.21.15.33:59940) 2023-11-06T21:02:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c36ffd700 1 -- 172.21.15.33:0/741673199 learned_addr learned my addr 172.21.15.33:0/741673199 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c367fc700 1 --2- 172.21.15.33:0/741673199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912a64c0 0x55ac912aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c377fe700 1 --2- 172.21.15.33:0/741673199 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 0x55ac9127bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c36ffd700 1 -- 172.21.15.33:0/741673199 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 msgr2=0x55ac9127bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/741673199 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 0x55ac9127bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c36ffd700 1 -- 172.21.15.33:0/741673199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912a64c0 msgr2=0x55ac912aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/741673199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912a64c0 0x55ac912aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c36ffd700 1 -- 172.21.15.33:0/741673199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac911c3810 con 0x55ac9127e080 2023-11-06T21:02:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/741673199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac9127e080 0x55ac912ac230 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f2c280078e0 tx=0x7f2c28013000).ready entity=mon.0 client_cookie=9b4d05bbce354730 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c1cff9700 1 -- 172.21.15.33:0/741673199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28007030 con 0x55ac9127e080 2023-11-06T21:02:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c1cff9700 1 -- 172.21.15.33:0/741673199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c28007190 con 0x55ac9127e080 2023-11-06T21:02:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.439+0000 7f2c1cff9700 1 -- 172.21.15.33:0/741673199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28007300 con 0x55ac9127e080 2023-11-06T21:02:18.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.440+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/741673199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac9127e080 msgr2=0x55ac912ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.440+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/741673199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac9127e080 0x55ac912ac230 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f2c280078e0 tx=0x7f2c28013000).stop 2023-11-06T21:02:18.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.440+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/741673199 shutdown_connections 2023-11-06T21:02:18.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.440+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/741673199 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac912c6be0 0x55ac9127bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.440+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/741673199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac9127e080 0x55ac912ac230 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.440+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/741673199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912a64c0 0x55ac912aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.440+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/741673199 >> 172.21.15.33:0/741673199 conn(0x55ac910e7940 msgr2=0x55ac912a2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:18.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.440+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/741673199 shutdown_connections 2023-11-06T21:02:18.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.440+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/741673199 wait complete. 2023-11-06T21:02:18.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.441+0000 7f2c54dc7b80 1 -- start start 2023-11-06T21:02:18.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.441+0000 7f2c54dc7b80 1 Processor -- start 2023-11-06T21:02:18.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.442+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 0x55ac9127bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.442+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912a64c0 0x55ac9127c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.442+0000 7f2c54dc7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912c6be0 0x55ac912bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.442+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac912a93f0 con 0x55ac912c6be0 2023-11-06T21:02:18.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.442+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac912c0510 con 0x55ac912a64c0 2023-11-06T21:02:18.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.442+0000 7f2c54dc7b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac912e4d40 con 0x55ac9127e080 2023-11-06T21:02:18.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.442+0000 7f2c54dc7b80 10 client.0 ll_register_callbacks cb 0x55ac9116fce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T21:02:18.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.442+0000 7f2c377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912c6be0 0x55ac912bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912c6be0 0x55ac912bf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59946/0 (socket says 172.21.15.33:59946) 2023-11-06T21:02:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c377fe700 1 -- 172.21.15.33:0/2637843403 learned_addr learned my addr 172.21.15.33:0/2637843403 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 0x55ac9127bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c367fc700 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912a64c0 0x55ac9127c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c377fe700 1 -- 172.21.15.33:0/2637843403 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 msgr2=0x55ac9127bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c377fe700 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 0x55ac9127bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c377fe700 1 -- 172.21.15.33:0/2637843403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912a64c0 msgr2=0x55ac9127c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c377fe700 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912a64c0 0x55ac9127c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c377fe700 1 -- 172.21.15.33:0/2637843403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2801c040 con 0x55ac912c6be0 2023-11-06T21:02:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c377fe700 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912c6be0 0x55ac912bf5f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f2c300097c0 tx=0x7f2c3001f000).ready entity=mon.0 client_cookie=b366ac25e5c7fa32 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c157fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T21:02:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c157fa700 1 -- 172.21.15.33:0/2637843403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30012bc0 con 0x55ac912c6be0 2023-11-06T21:02:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c157fa700 1 -- 172.21.15.33:0/2637843403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2c30032ca0 con 0x55ac912c6be0 2023-11-06T21:02:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.443+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2637843403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c30012780 con 0x55ac912c6be0 2023-11-06T21:02:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c157fa700 1 -- 172.21.15.33:0/2637843403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c30032e60 con 0x55ac912c6be0 2023-11-06T21:02:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c54dc7b80 10 client.7680 fetch_fsmap learned FSMap version 53 2023-11-06T21:02:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2637843403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55ac9103d1d0 con 0x55ac912c6be0 2023-11-06T21:02:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c157fa700 1 -- 172.21.15.33:0/2637843403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c30012780 con 0x55ac912c6be0 2023-11-06T21:02:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c157fa700 1 -- 172.21.15.33:0/2637843403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 53) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2c30026820 con 0x55ac912c6be0 2023-11-06T21:02:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c54dc7b80 10 client.7680 fetch_fsmap finished waiting for FSMap version 53 2023-11-06T21:02:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c54dc7b80 10 client.7680 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T21:02:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c54dc7b80 20 client.7680 populate_metadata read hostname 'smithi033' 2023-11-06T21:02:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.33:6832/1130278640,v1:172.21.15.33:6835/1130278640] conn(0x55ac9127e6b0 0x55ac912bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c54dc7b80 4 client.7680 mds_command: new command op to 7527 tid=0[{"prefix": "status"}] 2023-11-06T21:02:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2637843403 --> [v2:172.21.15.33:6832/1130278640,v1:172.21.15.33:6835/1130278640] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ac9106d040 con 0x55ac9127e6b0 2023-11-06T21:02:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.444+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.33:6832/1130278640,v1:172.21.15.33:6835/1130278640] conn(0x55ac9127e6b0 0x55ac912bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c36ffd700 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.33:6832/1130278640,v1:172.21.15.33:6835/1130278640] conn(0x55ac9127e6b0 0x55ac912bfae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ae5a93453f8da0fa server_cookie=a3f15d4be685916 in_seq=0 out_seq=0 2023-11-06T21:02:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c157fa700 10 client.7680 ms_handle_connect on v2:172.21.15.33:6832/1130278640 2023-11-06T21:02:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c157fa700 1 -- 172.21.15.33:0/2637843403 <== mds.0 v2:172.21.15.33:6832/1130278640 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ac9106d040 con 0x55ac9127e6b0 2023-11-06T21:02:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c157fa700 10 client.7680 handle_command_reply: tid=0 2023-11-06T21:02:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c54dc7b80 1 client.7680 shutdown 2023-11-06T21:02:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2637843403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912c6be0 msgr2=0x55ac912bf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912c6be0 0x55ac912bf5f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f2c300097c0 tx=0x7f2c3001f000).stop 2023-11-06T21:02:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2637843403 shutdown_connections 2023-11-06T21:02:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55ac9127e080 0x55ac9127bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.33:6832/1130278640,v1:172.21.15.33:6835/1130278640] conn(0x55ac9127e6b0 0x55ac912bfae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac912c6be0 0x55ac912bf5f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c54dc7b80 1 --2- 172.21.15.33:0/2637843403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac912a64c0 0x55ac9127c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c157fa700 0 client.7680 ms_handle_reset on v2:172.21.15.33:6832/1130278640 2023-11-06T21:02:18.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2637843403 >> 172.21.15.33:0/2637843403 conn(0x55ac910e7940 msgr2=0x55ac912a1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:18.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.445+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2637843403 shutdown_connections 2023-11-06T21:02:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c54dc7b80 1 -- 172.21.15.33:0/2637843403 wait complete. 2023-11-06T21:02:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c54dc7b80 20 client.7680 trim_cache size 0 max 16384 2023-11-06T21:02:18.460 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T21:02:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T21:02:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T21:02:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 7527, 2023-11-06T21:02:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T21:02:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T21:02:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T21:02:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.4525426149999996, 2023-11-06T21:02:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 53, 2023-11-06T21:02:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 131, 2023-11-06T21:02:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 131, 2023-11-06T21:02:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.676129173 2023-11-06T21:02:18.463 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T21:02:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c2405f270 msgr2=0x7f2c24061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c2405f270 0x7f2c24061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2c28007fd0 tx=0x7f2c28007bb0).stop 2023-11-06T21:02:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/3322725254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c380bfd50 msgr2=0x7f2c380a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c380bfd50 0x7f2c380a3de0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f2c2c006d00 tx=0x7f2c2c0090f0).stop 2023-11-06T21:02:18.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/3322725254 shutdown_connections 2023-11-06T21:02:18.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c2405f270 0x7f2c24061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c380630e0 0x7f2c38164390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c381544a0 0x7f2c380a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 --2- 172.21.15.33:0/3322725254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c380bfd50 0x7f2c380a3de0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/3322725254 >> 172.21.15.33:0/3322725254 conn(0x7f2c3805ec80 msgr2=0x7f2c380c8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/3322725254 shutdown_connections 2023-11-06T21:02:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.446+0000 7f2c3d9ff700 1 -- 172.21.15.33:0/3322725254 wait complete. 2023-11-06T21:02:18.470 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T21:02:18.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.634+0000 7f25741ff700 1 Processor -- start 2023-11-06T21:02:18.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.634+0000 7f25741ff700 1 -- start start 2023-11-06T21:02:18.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.635+0000 7f25741ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f256c1541a0 0x7f256c0c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.635+0000 7f25741ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f256c0c07b0 0x7f256c0c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.635+0000 7f25741ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256c0c5f80 con 0x7f256c0a9ee0 2023-11-06T21:02:18.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.635+0000 7f25741ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c0c1150 con 0x7f256c0c07b0 2023-11-06T21:02:18.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.635+0000 7f25741ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0aa740 con 0x7f256c1541a0 2023-11-06T21:02:18.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.636+0000 7f257279c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f256c0c07b0 0x7f256c0c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.636+0000 7f257279c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f256c0c07b0 0x7f256c0c0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59952/0 (socket says 172.21.15.33:59952) 2023-11-06T21:02:18.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.636+0000 7f257279c700 1 -- 172.21.15.33:0/4006375560 learned_addr learned my addr 172.21.15.33:0/4006375560 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:18.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.636+0000 7f2571f9b700 1 --2- 172.21.15.33:0/4006375560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f256c1541a0 0x7f256c0c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.636+0000 7f257279c700 1 -- 172.21.15.33:0/4006375560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f256c1541a0 msgr2=0x7f256c0c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:18.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.636+0000 7f257279c700 1 --2- 172.21.15.33:0/4006375560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f256c1541a0 0x7f256c0c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.636+0000 7f257279c700 1 -- 172.21.15.33:0/4006375560 >> v1:172.21.15.69:6789/0 conn(0x7f256c0a9ee0 legacy=0x7f256c0c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.636+0000 7f257279c700 1 -- 172.21.15.33:0/4006375560 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c0c6800 con 0x7f256c0c07b0 2023-11-06T21:02:18.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.636+0000 7f257279c700 1 --2- 172.21.15.33:0/4006375560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0c07b0 0x7f256c0c0c60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f2564009f20 tx=0x7f2564009840).ready entity=mon.0 client_cookie=e3f5c895aab437ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:18.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f2570f99700 1 -- 172.21.15.33:0/4006375560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256400e070 con 0x7f256c0c07b0 2023-11-06T21:02:18.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f2570f99700 1 -- 172.21.15.33:0/4006375560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256400b9d0 con 0x7f256c0c07b0 2023-11-06T21:02:18.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f2570f99700 1 -- 172.21.15.33:0/4006375560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256400bd50 con 0x7f256c0c07b0 2023-11-06T21:02:18.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f25741ff700 1 -- 172.21.15.33:0/4006375560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0c07b0 msgr2=0x7f256c0c0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f25741ff700 1 --2- 172.21.15.33:0/4006375560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0c07b0 0x7f256c0c0c60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f2564009f20 tx=0x7f2564009840).stop 2023-11-06T21:02:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f25741ff700 1 -- 172.21.15.33:0/4006375560 shutdown_connections 2023-11-06T21:02:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f25741ff700 1 --2- 172.21.15.33:0/4006375560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0c07b0 0x7f256c0c0c60 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f25741ff700 1 --2- 172.21.15.33:0/4006375560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f256c1541a0 0x7f256c0c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f25741ff700 1 -- 172.21.15.33:0/4006375560 >> 172.21.15.33:0/4006375560 conn(0x7f256c0944e0 msgr2=0x7f256c06a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f25741ff700 1 -- 172.21.15.33:0/4006375560 shutdown_connections 2023-11-06T21:02:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.637+0000 7f25741ff700 1 -- 172.21.15.33:0/4006375560 wait complete. 2023-11-06T21:02:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.639+0000 7f25741ff700 1 Processor -- start 2023-11-06T21:02:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.639+0000 7f25741ff700 1 -- start start 2023-11-06T21:02:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f25741ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0a9ee0 0x7f256c0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f25741ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f256c0c07b0 0x7f256c14ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f25741ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f256c1541a0 0x7f256c142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f25741ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c14f1d0 con 0x7f256c0a9ee0 2023-11-06T21:02:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f25741ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0cb720 con 0x7f256c1541a0 2023-11-06T21:02:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f25741ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0cba00 con 0x7f256c0c07b0 2023-11-06T21:02:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f2571f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0a9ee0 0x7f256c0cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f2571f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0a9ee0 0x7f256c0cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59966/0 (socket says 172.21.15.33:59966) 2023-11-06T21:02:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f2571f9b700 1 -- 172.21.15.33:0/3303969530 learned_addr learned my addr 172.21.15.33:0/3303969530 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f257179a700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f256c0c07b0 0x7f256c14ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.640+0000 7f257279c700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f256c1541a0 0x7f256c142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f2571f9b700 1 -- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f256c0c07b0 msgr2=0x7f256c14ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f2571f9b700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f256c0c07b0 0x7f256c14ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f2571f9b700 1 -- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f256c1541a0 msgr2=0x7f256c142860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f2571f9b700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f256c1541a0 0x7f256c142860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f2571f9b700 1 -- 172.21.15.33:0/3303969530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2564009ae0 con 0x7f256c0a9ee0 2023-11-06T21:02:18.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f257279c700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f256c1541a0 0x7f256c142860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:02:18.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f2571f9b700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0a9ee0 0x7f256c0cb090 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f256800e8d0 tx=0x7f2568006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:18.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f2561ffb700 1 -- 172.21.15.33:0/3303969530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568009450 con 0x7f256c0a9ee0 2023-11-06T21:02:18.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f2561ffb700 1 -- 172.21.15.33:0/3303969530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2568012600 con 0x7f256c0a9ee0 2023-11-06T21:02:18.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f25741ff700 1 -- 172.21.15.33:0/3303969530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c0cbce0 con 0x7f256c0a9ee0 2023-11-06T21:02:18.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f2561ffb700 1 -- 172.21.15.33:0/3303969530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568012980 con 0x7f256c0a9ee0 2023-11-06T21:02:18.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.641+0000 7f25741ff700 1 -- 172.21.15.33:0/3303969530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c0aadc0 con 0x7f256c0a9ee0 2023-11-06T21:02:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.642+0000 7f25741ff700 1 -- 172.21.15.33:0/3303969530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c0475a0 con 0x7f256c0a9ee0 2023-11-06T21:02:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.643+0000 7f2561ffb700 1 -- 172.21.15.33:0/3303969530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2568012cf0 con 0x7f256c0a9ee0 2023-11-06T21:02:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.644+0000 7f2561ffb700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f255005f270 0x7f2550061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.644+0000 7f2561ffb700 1 -- 172.21.15.33:0/3303969530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f256807a720 con 0x7f256c0a9ee0 2023-11-06T21:02:18.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.644+0000 7f257179a700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f255005f270 0x7f2550061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:18.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.645+0000 7f257179a700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f255005f270 0x7f2550061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f256c0aa9d0 tx=0x7f255c007d20).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.648+0000 7f2561ffb700 1 -- 172.21.15.33:0/3303969530 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f256804b870 con 0x7f256c0a9ee0 2023-11-06T21:02:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.871+0000 7f25741ff700 1 -- 172.21.15.33:0/3303969530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f256c0ab130 con 0x7f256c0a9ee0 2023-11-06T21:02:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.872+0000 7f2561ffb700 1 -- 172.21.15.33:0/3303969530 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2568016070 con 0x7f256c0a9ee0 2023-11-06T21:02:18.874 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T21:02:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 -- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f255005f270 msgr2=0x7f2550061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f255005f270 0x7f2550061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f256c0aa9d0 tx=0x7f255c007d20).stop 2023-11-06T21:02:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 -- 172.21.15.33:0/3303969530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0a9ee0 msgr2=0x7f256c0cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0a9ee0 0x7f256c0cb090 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f256800e8d0 tx=0x7f2568006f40).stop 2023-11-06T21:02:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 -- 172.21.15.33:0/3303969530 shutdown_connections 2023-11-06T21:02:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f255005f270 0x7f2550061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f256c1541a0 0x7f256c142860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c0a9ee0 0x7f256c0cb090 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 --2- 172.21.15.33:0/3303969530 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f256c0c07b0 0x7f256c14ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 -- 172.21.15.33:0/3303969530 >> 172.21.15.33:0/3303969530 conn(0x7f256c0944e0 msgr2=0x7f256c0a90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 -- 172.21.15.33:0/3303969530 shutdown_connections 2023-11-06T21:02:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:18.874+0000 7f25741ff700 1 -- 172.21.15.33:0/3303969530 wait complete. 2023-11-06T21:02:18.889 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T21:02:18.889 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T21:02:18.890 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T21:02:18.890 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T21:02:18.890 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T21:02:18.891 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T21:02:18.891 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T21:02:18.891 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T21:02:18.892 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T21:02:18.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:02:18.893 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T21:02:18.942 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T21:02:18.942 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T21:02:18.943 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T21:02:18.943 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:02:18.943 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T21:02:18.943 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:02:18.943 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:02:18.944 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:02:18.944 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T21:02:18.944 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:02:18.944 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:02:18.944 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T21:02:18.945 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T21:02:18.945 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:02:18.945 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T21:02:18.945 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T21:02:18.945 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41205sec preferred_lft 41205sec 2023-11-06T21:02:18.946 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T21:02:18.946 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:02:18.946 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:02:18.946 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:02:18.947 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T21:02:18.947 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T21:02:18.947 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T21:02:18.947 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T21:02:18.947 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:02:18.948 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T21:02:18.948 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:02:18.948 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T21:02:18.948 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T21:02:18.949 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T21:02:18.949 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:02:18.949 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T21:02:18.950 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T21:02:18.950 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T21:02:18.950 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:02:18.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:02:18.952 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T21:02:19.021 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T21:02:19.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:02:19.024 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T21:02:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T21:02:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T21:02:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T21:02:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T21:02:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T21:02:19.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:02:19.081 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T21:02:19.081 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T21:02:19.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T21:02:19.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T21:02:19.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T21:02:19.083 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T21:02:19.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:02:19.258 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T21:02:19.305 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T21:02:19.306 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T21:02:19.307 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T21:02:19.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:02:19.308 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T21:02:19.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:02:19.381 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T21:02:19.430 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T21:02:19.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:02:19.532 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:02:19.533 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T21:02:19.560 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:02:19.560 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T21:02:19.565 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T21:02:19.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:02:19.566 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T21:02:19.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T21:02:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.835+0000 7fa2167ff700 1 Processor -- start 2023-11-06T21:02:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.835+0000 7fa2167ff700 1 -- start start 2023-11-06T21:02:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.837+0000 7fa2167ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa210154440 0x7fa2100bd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.837+0000 7fa2167ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa210064620 0x7fa210064a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.837+0000 7fa2167ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2100686d0 con 0x7fa2100a5610 2023-11-06T21:02:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.837+0000 7fa2167ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2100a48e0 con 0x7fa210064620 2023-11-06T21:02:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.837+0000 7fa2167ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa210064f70 con 0x7fa210154440 2023-11-06T21:02:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.837+0000 7fa20f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa210064620 0x7fa210064a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.837+0000 7fa214d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa2100a5610 0x7fa2100a5ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42510/0 (socket says 172.21.15.33:42510) 2023-11-06T21:02:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.837+0000 7fa20f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa210064620 0x7fa210064a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59974/0 (socket says 172.21.15.33:59974) 2023-11-06T21:02:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.837+0000 7fa214d9c700 1 -- 172.21.15.33:0/3857722349 learned_addr learned my addr 172.21.15.33:0/3857722349 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.837+0000 7fa20ffff700 1 --2- 172.21.15.33:0/3857722349 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa210154440 0x7fa2100bd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.838+0000 7fa20f7fe700 1 -- 172.21.15.33:0/3857722349 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa210154440 msgr2=0x7fa2100bd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.838+0000 7fa20f7fe700 1 --2- 172.21.15.33:0/3857722349 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa210154440 0x7fa2100bd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.838+0000 7fa20f7fe700 1 -- 172.21.15.33:0/3857722349 >> v1:172.21.15.33:6789/0 conn(0x7fa2100a5610 legacy=0x7fa2100a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.838+0000 7fa20f7fe700 1 -- 172.21.15.33:0/3857722349 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa210154820 con 0x7fa210064620 2023-11-06T21:02:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.838+0000 7fa20f7fe700 1 --2- 172.21.15.33:0/3857722349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210064620 0x7fa210064a80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fa204009760 tx=0x7fa204009060).ready entity=mon.0 client_cookie=c43babc5c229115d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.838+0000 7fa20effd700 1 -- 172.21.15.33:0/3857722349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20400f510 con 0x7fa210064620 2023-11-06T21:02:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.838+0000 7fa20effd700 1 -- 172.21.15.33:0/3857722349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20400f670 con 0x7fa210064620 2023-11-06T21:02:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.838+0000 7fa20effd700 1 -- 172.21.15.33:0/3857722349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20400f9f0 con 0x7fa210064620 2023-11-06T21:02:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.839+0000 7fa2167ff700 1 -- 172.21.15.33:0/3857722349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210064620 msgr2=0x7fa210064a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.839+0000 7fa2167ff700 1 --2- 172.21.15.33:0/3857722349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210064620 0x7fa210064a80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fa204009760 tx=0x7fa204009060).stop 2023-11-06T21:02:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.839+0000 7fa2167ff700 1 -- 172.21.15.33:0/3857722349 shutdown_connections 2023-11-06T21:02:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.839+0000 7fa2167ff700 1 --2- 172.21.15.33:0/3857722349 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa210154440 0x7fa2100bd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.839+0000 7fa2167ff700 1 --2- 172.21.15.33:0/3857722349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210064620 0x7fa210064a80 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.839+0000 7fa2167ff700 1 -- 172.21.15.33:0/3857722349 >> 172.21.15.33:0/3857722349 conn(0x7fa2100631a0 msgr2=0x7fa2100c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.839+0000 7fa2167ff700 1 -- 172.21.15.33:0/3857722349 shutdown_connections 2023-11-06T21:02:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.839+0000 7fa2167ff700 1 -- 172.21.15.33:0/3857722349 wait complete. 2023-11-06T21:02:19.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.840+0000 7fa2167ff700 1 Processor -- start 2023-11-06T21:02:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.840+0000 7fa2167ff700 1 -- start start 2023-11-06T21:02:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.841+0000 7fa2167ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa210064620 0x7fa2101401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.841+0000 7fa2167ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2100a5610 0x7fa2100a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.841+0000 7fa2167ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210154440 0x7fa2100a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.841+0000 7fa2167ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100a8aa0 con 0x7fa210154440 2023-11-06T21:02:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.841+0000 7fa2167ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100a74d0 con 0x7fa210064620 2023-11-06T21:02:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.841+0000 7fa2167ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21013d5c0 con 0x7fa2100a5610 2023-11-06T21:02:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa214d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210154440 0x7fa2100a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa214d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210154440 0x7fa2100a85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59988/0 (socket says 172.21.15.33:59988) 2023-11-06T21:02:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa214d9c700 1 -- 172.21.15.33:0/356242128 learned_addr learned my addr 172.21.15.33:0/356242128 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa20ffff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa210064620 0x7fa2101401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa20f7fe700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2100a5610 0x7fa2100a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa214d9c700 1 -- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2100a5610 msgr2=0x7fa2100a80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa214d9c700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2100a5610 0x7fa2100a80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa214d9c700 1 -- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa210064620 msgr2=0x7fa2101401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:19.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa214d9c700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa210064620 0x7fa2101401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:19.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa214d9c700 1 -- 172.21.15.33:0/356242128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2040092a0 con 0x7fa210154440 2023-11-06T21:02:19.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa20ffff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa210064620 0x7fa2101401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:19.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa20f7fe700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2100a5610 0x7fa2100a80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:19.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa214d9c700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210154440 0x7fa2100a85b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7fa20800c7c0 tx=0x7fa208001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.842+0000 7fa1f7fff700 1 -- 172.21.15.33:0/356242128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208005d30 con 0x7fa210154440 2023-11-06T21:02:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.843+0000 7fa1f7fff700 1 -- 172.21.15.33:0/356242128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208005e90 con 0x7fa210154440 2023-11-06T21:02:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.843+0000 7fa1f7fff700 1 -- 172.21.15.33:0/356242128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208007320 con 0x7fa210154440 2023-11-06T21:02:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.843+0000 7fa2167ff700 1 -- 172.21.15.33:0/356242128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2100a9890 con 0x7fa210154440 2023-11-06T21:02:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.843+0000 7fa2167ff700 1 -- 172.21.15.33:0/356242128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2100a8f60 con 0x7fa210154440 2023-11-06T21:02:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.844+0000 7fa2167ff700 1 -- 172.21.15.33:0/356242128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2100bdc50 con 0x7fa210154440 2023-11-06T21:02:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.845+0000 7fa1f7fff700 1 -- 172.21.15.33:0/356242128 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa208007710 con 0x7fa210154440 2023-11-06T21:02:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.846+0000 7fa1f7fff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1fc05f270 0x7fa1fc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.846+0000 7fa1f7fff700 1 -- 172.21.15.33:0/356242128 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa20807a3c0 con 0x7fa210154440 2023-11-06T21:02:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.846+0000 7fa20ffff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1fc05f270 0x7fa1fc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.847+0000 7fa20ffff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1fc05f270 0x7fa1fc061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa200006d80 tx=0x7fa200008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:19.850+0000 7fa1f7fff700 1 -- 172.21.15.33:0/356242128 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa20804b510 con 0x7fa210154440 2023-11-06T21:02:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.061+0000 7fa2167ff700 1 -- 172.21.15.33:0/356242128 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2100c4320 con 0x7fa1fc05f270 2023-11-06T21:02:20.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.063+0000 7fa1f7fff700 1 -- 172.21.15.33:0/356242128 <== mgr.6377 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa2100c4320 con 0x7fa1fc05f270 2023-11-06T21:02:20.064 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T21:02:20.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:02:20.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T21:02:20.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:02:20.065 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T21:02:20.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 -- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1fc05f270 msgr2=0x7fa1fc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1fc05f270 0x7fa1fc061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa200006d80 tx=0x7fa200008040).stop 2023-11-06T21:02:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 -- 172.21.15.33:0/356242128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210154440 msgr2=0x7fa2100a85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210154440 0x7fa2100a85b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7fa20800c7c0 tx=0x7fa208001510).stop 2023-11-06T21:02:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 -- 172.21.15.33:0/356242128 shutdown_connections 2023-11-06T21:02:20.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa1fc05f270 0x7fa1fc061730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:20.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa210154440 0x7fa2100a85b0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:20.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa210064620 0x7fa2101401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:20.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 --2- 172.21.15.33:0/356242128 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa2100a5610 0x7fa2100a80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:20.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 -- 172.21.15.33:0/356242128 >> 172.21.15.33:0/356242128 conn(0x7fa2100631a0 msgr2=0x7fa210068860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:20.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 -- 172.21.15.33:0/356242128 shutdown_connections 2023-11-06T21:02:20.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.066+0000 7fa2167ff700 1 -- 172.21.15.33:0/356242128 wait complete. 2023-11-06T21:02:20.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T21:02:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.281+0000 7f33651ff700 1 Processor -- start 2023-11-06T21:02:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.281+0000 7f33651ff700 1 -- start start 2023-11-06T21:02:20.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.283+0000 7f33651ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3360152850 0x7f3360065ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:20.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.283+0000 7f33651ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3360152c30 0x7f3360153040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:20.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.283+0000 7f33651ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33600bc7c0 con 0x7f3360063f20 2023-11-06T21:02:20.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.283+0000 7f33651ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33600a5d20 con 0x7f3360152c30 2023-11-06T21:02:20.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.283+0000 7f33651ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33600a6000 con 0x7f3360152850 2023-11-06T21:02:20.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.283+0000 7f335f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3360152c30 0x7f3360153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.283+0000 7f335e59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3360063f20 0x7f33600bcc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42520/0 (socket says 172.21.15.33:42520) 2023-11-06T21:02:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.283+0000 7f335f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3360152c30 0x7f3360153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60000/0 (socket says 172.21.15.33:60000) 2023-11-06T21:02:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.283+0000 7f335e59c700 1 -- 172.21.15.33:0/335475225 learned_addr learned my addr 172.21.15.33:0/335475225 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:02:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.283+0000 7f335ed9d700 1 --2- 172.21.15.33:0/335475225 >> v2:172.21.15.69:3300/0 conn(0x7f3360152850 0x7f3360065ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.284+0000 7f335dd9b700 1 -- 172.21.15.33:0/335475225 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588076550 0 0) 0x7f33600bc7c0 con 0x7f3360063f20 2023-11-06T21:02:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.284+0000 7f335dd9b700 1 -- 172.21.15.33:0/335475225 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33480033f0 con 0x7f3360063f20 2023-11-06T21:02:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.284+0000 7f335f59e700 1 -- 172.21.15.33:0/335475225 >> v2:172.21.15.69:3300/0 conn(0x7f3360152850 msgr2=0x7f3360065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.284+0000 7f335f59e700 1 --2- 172.21.15.33:0/335475225 >> v2:172.21.15.69:3300/0 conn(0x7f3360152850 0x7f3360065ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.284+0000 7f335f59e700 1 -- 172.21.15.33:0/335475225 >> v1:172.21.15.33:6789/0 conn(0x7f3360063f20 legacy=0x7f33600bcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.284+0000 7f335f59e700 1 -- 172.21.15.33:0/335475225 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33600a62e0 con 0x7f3360152c30 2023-11-06T21:02:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.284+0000 7f335f59e700 1 --2- 172.21.15.33:0/335475225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 0x7f3360153040 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f3360153960 tx=0x7f3358009780).ready entity=mon.0 client_cookie=101a84a2398c613 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.284+0000 7f335dd9b700 1 -- 172.21.15.33:0/335475225 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335800e070 con 0x7f3360152c30 2023-11-06T21:02:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.284+0000 7f335dd9b700 1 -- 172.21.15.33:0/335475225 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335800bab0 con 0x7f3360152c30 2023-11-06T21:02:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.284+0000 7f335dd9b700 1 -- 172.21.15.33:0/335475225 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335800be30 con 0x7f3360152c30 2023-11-06T21:02:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.285+0000 7f33651ff700 1 -- 172.21.15.33:0/335475225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 msgr2=0x7f3360153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.285+0000 7f33651ff700 1 --2- 172.21.15.33:0/335475225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 0x7f3360153040 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f3360153960 tx=0x7f3358009780).stop 2023-11-06T21:02:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.285+0000 7f33651ff700 1 -- 172.21.15.33:0/335475225 shutdown_connections 2023-11-06T21:02:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.285+0000 7f33651ff700 1 --2- 172.21.15.33:0/335475225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 0x7f3360153040 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.285+0000 7f33651ff700 1 --2- 172.21.15.33:0/335475225 >> v2:172.21.15.69:3300/0 conn(0x7f3360152850 0x7f3360065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.285+0000 7f33651ff700 1 -- 172.21.15.33:0/335475225 >> 172.21.15.33:0/335475225 conn(0x7f33600c6a40 msgr2=0x7f33600c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.285+0000 7f33651ff700 1 -- 172.21.15.33:0/335475225 shutdown_connections 2023-11-06T21:02:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.285+0000 7f33651ff700 1 -- 172.21.15.33:0/335475225 wait complete. 2023-11-06T21:02:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.287+0000 7f33651ff700 1 Processor -- start 2023-11-06T21:02:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.287+0000 7f33651ff700 1 -- start start 2023-11-06T21:02:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.288+0000 7f33651ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3360063f20 0x7f33600aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.288+0000 7f33651ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3360152850 0x7f3360069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:20.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.288+0000 7f33651ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 0x7f3360151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:20.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.288+0000 7f33651ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3360069b40 con 0x7f3360152c30 2023-11-06T21:02:20.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.288+0000 7f33651ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3360151af0 con 0x7f3360152850 2023-11-06T21:02:20.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.288+0000 7f33651ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336006a310 con 0x7f3360063f20 2023-11-06T21:02:20.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 0x7f3360151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:20.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 0x7f3360151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60006/0 (socket says 172.21.15.33:60006) 2023-11-06T21:02:20.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335f59e700 1 -- 172.21.15.33:0/1543978355 learned_addr learned my addr 172.21.15.33:0/1543978355 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:20.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335e59c700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3360152850 0x7f3360069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:20.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335ed9d700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3360063f20 0x7f33600aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:20.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335f59e700 1 -- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3360063f20 msgr2=0x7f33600aa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335f59e700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3360063f20 0x7f33600aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335f59e700 1 -- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3360152850 msgr2=0x7f3360069650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335f59e700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3360152850 0x7f3360069650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335f59e700 1 -- 172.21.15.33:0/1543978355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3358009a20 con 0x7f3360152c30 2023-11-06T21:02:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335e59c700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3360152850 0x7f3360069650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:02:20.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f335f59e700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 0x7f3360151600 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f3358013d40 tx=0x7f33601503c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:20.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f3346ffd700 1 -- 172.21.15.33:0/1543978355 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335800b640 con 0x7f3360152c30 2023-11-06T21:02:20.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f3346ffd700 1 -- 172.21.15.33:0/1543978355 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335801d630 con 0x7f3360152c30 2023-11-06T21:02:20.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f33651ff700 1 -- 172.21.15.33:0/1543978355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33600a9e70 con 0x7f3360152c30 2023-11-06T21:02:20.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.289+0000 7f3346ffd700 1 -- 172.21.15.33:0/1543978355 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335801d9b0 con 0x7f3360152c30 2023-11-06T21:02:20.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.290+0000 7f33651ff700 1 -- 172.21.15.33:0/1543978355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336006a030 con 0x7f3360152c30 2023-11-06T21:02:20.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.291+0000 7f33651ff700 1 -- 172.21.15.33:0/1543978355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33600aa000 con 0x7f3360152c30 2023-11-06T21:02:20.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.291+0000 7f3346ffd700 1 -- 172.21.15.33:0/1543978355 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 47) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3358003040 con 0x7f3360152c30 2023-11-06T21:02:20.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.292+0000 7f3346ffd700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f334c05f270 0x7f334c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:20.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.293+0000 7f3346ffd700 1 -- 172.21.15.33:0/1543978355 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f335807c0c0 con 0x7f3360152c30 2023-11-06T21:02:20.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.293+0000 7f335ed9d700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f334c05f270 0x7f334c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:20.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.293+0000 7f335ed9d700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f334c05f270 0x7f334c061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3350009270 tx=0x7f3350008040).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:20.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.296+0000 7f3346ffd700 1 -- 172.21.15.33:0/1543978355 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f335804d150 con 0x7f3360152c30 2023-11-06T21:02:20.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:20.523+0000 7f33651ff700 1 -- 172.21.15.33:0/1543978355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f33600bc450 con 0x7f3360152c30 2023-11-06T21:02:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.060+0000 7f3346ffd700 1 -- 172.21.15.33:0/1543978355 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 48) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f3358079c50 con 0x7f3360152c30 2023-11-06T21:02:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.062+0000 7f3346ffd700 1 -- 172.21.15.33:0/1543978355 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v48) v1 ==== 92+0+0 (secure 0 0 0) 0x7f33580430a0 con 0x7f3360152c30 2023-11-06T21:02:21.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 -- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f334c05f270 msgr2=0x7f334c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:21.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f334c05f270 0x7f334c061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3350009270 tx=0x7f3350008040).stop 2023-11-06T21:02:21.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 -- 172.21.15.33:0/1543978355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 msgr2=0x7f3360151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:21.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 0x7f3360151600 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f3358013d40 tx=0x7f33601503c0).stop 2023-11-06T21:02:21.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 -- 172.21.15.33:0/1543978355 shutdown_connections 2023-11-06T21:02:21.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f334c05f270 0x7f334c061730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3360063f20 0x7f33600aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3360152c30 0x7f3360151600 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 --2- 172.21.15.33:0/1543978355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3360152850 0x7f3360069650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 -- 172.21.15.33:0/1543978355 >> 172.21.15.33:0/1543978355 conn(0x7f33600c6a40 msgr2=0x7f3360150430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:21.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 -- 172.21.15.33:0/1543978355 shutdown_connections 2023-11-06T21:02:21.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.065+0000 7f33651ff700 1 -- 172.21.15.33:0/1543978355 wait complete. 2023-11-06T21:02:21.087 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T21:02:21.278 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:21.278+0000 7ffaf3c71680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:02:21.289 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:21.288+0000 7f4185fd5680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:02:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.292+0000 7f663bbff700 1 Processor -- start 2023-11-06T21:02:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.293+0000 7f663bbff700 1 -- start start 2023-11-06T21:02:21.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663bbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154330 0x7f66340c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663bbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6634154710 0x7f66340c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663bbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f66340c7ff0 0x7f66340646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663bbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66340c6b60 con 0x7f6634154710 2023-11-06T21:02:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663bbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66340c5cd0 con 0x7f66340c7ff0 2023-11-06T21:02:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663bbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66340c5f50 con 0x7f6634154330 2023-11-06T21:02:21.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6634154710 0x7f66340c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6634154710 0x7f66340c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60028/0 (socket says 172.21.15.33:60028) 2023-11-06T21:02:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663919a700 1 -- 172.21.15.33:0/1749267254 learned_addr learned my addr 172.21.15.33:0/1749267254 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663a19c700 1 --2- 172.21.15.33:0/1749267254 >> v2:172.21.15.69:3300/0 conn(0x7f66340c7ff0 0x7f66340646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.294+0000 7f663999b700 1 --2- 172.21.15.33:0/1749267254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154330 0x7f66340c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f663919a700 1 -- 172.21.15.33:0/1749267254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154330 msgr2=0x7f66340c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:21.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f663919a700 1 --2- 172.21.15.33:0/1749267254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154330 0x7f66340c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f663919a700 1 -- 172.21.15.33:0/1749267254 >> v2:172.21.15.69:3300/0 conn(0x7f66340c7ff0 msgr2=0x7f66340646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f663919a700 1 --2- 172.21.15.33:0/1749267254 >> v2:172.21.15.69:3300/0 conn(0x7f66340c7ff0 0x7f66340646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f663919a700 1 -- 172.21.15.33:0/1749267254 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66340c6230 con 0x7f6634154710 2023-11-06T21:02:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f663999b700 1 --2- 172.21.15.33:0/1749267254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154330 0x7f66340c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:02:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f663a19c700 1 --2- 172.21.15.33:0/1749267254 >> v2:172.21.15.69:3300/0 conn(0x7f66340c7ff0 0x7f66340646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:02:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f663919a700 1 --2- 172.21.15.33:0/1749267254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154710 0x7f66340c6ff0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f66340c1e90 tx=0x7f66240090f0).ready entity=mon.0 client_cookie=b48cd9f7c443b014 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f6638999700 1 -- 172.21.15.33:0/1749267254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662400f510 con 0x7f6634154710 2023-11-06T21:02:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f6638999700 1 -- 172.21.15.33:0/1749267254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662400f670 con 0x7f6634154710 2023-11-06T21:02:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.295+0000 7f6638999700 1 -- 172.21.15.33:0/1749267254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662400f9f0 con 0x7f6634154710 2023-11-06T21:02:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.296+0000 7f663bbff700 1 -- 172.21.15.33:0/1749267254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154710 msgr2=0x7f66340c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.296+0000 7f663bbff700 1 --2- 172.21.15.33:0/1749267254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154710 0x7f66340c6ff0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f66340c1e90 tx=0x7f66240090f0).stop 2023-11-06T21:02:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.296+0000 7f663bbff700 1 -- 172.21.15.33:0/1749267254 shutdown_connections 2023-11-06T21:02:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.296+0000 7f663bbff700 1 --2- 172.21.15.33:0/1749267254 >> v2:172.21.15.69:3300/0 conn(0x7f66340c7ff0 0x7f66340646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.296+0000 7f663bbff700 1 --2- 172.21.15.33:0/1749267254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154330 0x7f66340c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.296+0000 7f663bbff700 1 --2- 172.21.15.33:0/1749267254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154710 0x7f66340c6ff0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.296+0000 7f663bbff700 1 -- 172.21.15.33:0/1749267254 >> 172.21.15.33:0/1749267254 conn(0x7f66340cb910 msgr2=0x7f66340c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.296+0000 7f663bbff700 1 -- 172.21.15.33:0/1749267254 shutdown_connections 2023-11-06T21:02:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.296+0000 7f663bbff700 1 -- 172.21.15.33:0/1749267254 wait complete. 2023-11-06T21:02:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.298+0000 7f663bbff700 1 Processor -- start 2023-11-06T21:02:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.298+0000 7f663bbff700 1 -- start start 2023-11-06T21:02:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663bbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66340c7ff0 0x7f663414e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663bbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154330 0x7f66340c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663bbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154710 0x7f663414c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663bbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663414f620 con 0x7f6634154330 2023-11-06T21:02:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663bbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663414ea10 con 0x7f66340c7ff0 2023-11-06T21:02:21.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663bbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663414ecf0 con 0x7f6634154710 2023-11-06T21:02:21.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154330 0x7f66340c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154330 0x7f66340c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60042/0 (socket says 172.21.15.33:60042) 2023-11-06T21:02:21.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663919a700 1 -- 172.21.15.33:0/3286206391 learned_addr learned my addr 172.21.15.33:0/3286206391 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:21.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663999b700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66340c7ff0 0x7f663414e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.299+0000 7f663a19c700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154710 0x7f663414c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.300+0000 7f663919a700 1 -- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154710 msgr2=0x7f663414c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:21.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.300+0000 7f663919a700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154710 0x7f663414c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.300+0000 7f663919a700 1 -- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66340c7ff0 msgr2=0x7f663414e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:21.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.300+0000 7f663919a700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66340c7ff0 0x7f663414e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.300+0000 7f663919a700 1 -- 172.21.15.33:0/3286206391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c009a20 con 0x7f6634154330 2023-11-06T21:02:21.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.300+0000 7f663919a700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154330 0x7f66340c1ee0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f6624011000 tx=0x7f6624017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:21.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.300+0000 7f6629ffb700 1 -- 172.21.15.33:0/3286206391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662400f7e0 con 0x7f6634154330 2023-11-06T21:02:21.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.300+0000 7f6629ffb700 1 -- 172.21.15.33:0/3286206391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662400f940 con 0x7f6634154330 2023-11-06T21:02:21.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.300+0000 7f6629ffb700 1 -- 172.21.15.33:0/3286206391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662401c670 con 0x7f6634154330 2023-11-06T21:02:21.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.300+0000 7f663bbff700 1 -- 172.21.15.33:0/3286206391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66300092a0 con 0x7f6634154330 2023-11-06T21:02:21.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.301+0000 7f663bbff700 1 -- 172.21.15.33:0/3286206391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6624009370 con 0x7f6634154330 2023-11-06T21:02:21.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.302+0000 7f663bbff700 1 -- 172.21.15.33:0/3286206391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66340c67e0 con 0x7f6634154330 2023-11-06T21:02:21.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.303+0000 7f6629ffb700 1 -- 172.21.15.33:0/3286206391 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 48) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f662401ca60 con 0x7f6634154330 2023-11-06T21:02:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.303+0000 7f6629ffb700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f661805f2c0 0x7f6618061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.303+0000 7f6629ffb700 1 -- 172.21.15.33:0/3286206391 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f662407b360 con 0x7f6634154330 2023-11-06T21:02:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.304+0000 7f663999b700 1 -- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f661805f2c0 msgr2=0x7f6618061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:02:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.304+0000 7f663999b700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f661805f2c0 0x7f6618061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:02:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.308+0000 7f6629ffb700 1 -- 172.21.15.33:0/3286206391 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f662404c560 con 0x7f6634154330 2023-11-06T21:02:21.358 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:21.358+0000 7ffaf3c71680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:02:21.375 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:21.374+0000 7f4185fd5680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:02:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.491+0000 7f663bbff700 1 -- 172.21.15.33:0/3286206391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f66340830b0 con 0x7f6634154330 2023-11-06T21:02:21.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.497+0000 7f6629ffb700 1 -- 172.21.15.33:0/3286206391 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f662404fb80 con 0x7f6634154330 2023-11-06T21:02:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.499+0000 7f663bbff700 1 -- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f661805f2c0 msgr2=0x7f6618061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:02:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.499+0000 7f663bbff700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f661805f2c0 0x7f6618061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:02:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.499+0000 7f663bbff700 1 -- 172.21.15.33:0/3286206391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154330 msgr2=0x7f66340c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.499+0000 7f663bbff700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154330 0x7f66340c1ee0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f6624011000 tx=0x7f6624017000).stop 2023-11-06T21:02:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.499+0000 7f663bbff700 1 -- 172.21.15.33:0/3286206391 shutdown_connections 2023-11-06T21:02:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.499+0000 7f663bbff700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f661805f2c0 0x7f6618061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:02:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.499+0000 7f663bbff700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6634154710 0x7f663414c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.499+0000 7f663bbff700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66340c7ff0 0x7f663414e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.500+0000 7f663bbff700 1 --2- 172.21.15.33:0/3286206391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6634154330 0x7f66340c1ee0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.500+0000 7f663bbff700 1 -- 172.21.15.33:0/3286206391 >> 172.21.15.33:0/3286206391 conn(0x7f66340cb910 msgr2=0x7f66340646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.500+0000 7f663bbff700 1 -- 172.21.15.33:0/3286206391 shutdown_connections 2023-11-06T21:02:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.500+0000 7f663bbff700 1 -- 172.21.15.33:0/3286206391 wait complete. 2023-11-06T21:02:21.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:21.519 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-06T21:02:21.571 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-11-06T21:02:21.689 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:21.689+0000 7ffaf3c71680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:02:21.727 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:21.726+0000 7f4185fd5680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:02:21.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.774+0000 7f27c83ff700 1 Processor -- start 2023-11-06T21:02:21.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.775+0000 7f27c83ff700 1 -- start start 2023-11-06T21:02:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.776+0000 7f27c83ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0154720 0x7f27c00aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.776+0000 7f27c83ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f27c0158720 0x7f27c0063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.776+0000 7f27c83ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c00c0e00 con 0x7f27c0154340 2023-11-06T21:02:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.776+0000 7f27c83ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c0158bf0 con 0x7f27c0158720 2023-11-06T21:02:21.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.776+0000 7f27c83ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c0158ed0 con 0x7f27c0154720 2023-11-06T21:02:21.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.776+0000 7f27c599a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0154720 0x7f27c00aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.776+0000 7f27c699c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f27c0158720 0x7f27c0063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.776+0000 7f27c599a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0154720 0x7f27c00aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:57058/0 (socket says 172.21.15.33:57058) 2023-11-06T21:02:21.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.776+0000 7f27c599a700 1 -- 172.21.15.33:0/2934666251 learned_addr learned my addr 172.21.15.33:0/2934666251 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.777+0000 7f27c5199700 1 -- 172.21.15.33:0/2934666251 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236318049 0 0) 0x7f27c00c0e00 con 0x7f27c0154340 2023-11-06T21:02:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.777+0000 7f27c5199700 1 -- 172.21.15.33:0/2934666251 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ac0033f0 con 0x7f27c0154340 2023-11-06T21:02:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.777+0000 7f27c599a700 1 -- 172.21.15.33:0/2934666251 >> v2:172.21.15.69:3300/0 conn(0x7f27c0158720 msgr2=0x7f27c0063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.777+0000 7f27c599a700 1 --2- 172.21.15.33:0/2934666251 >> v2:172.21.15.69:3300/0 conn(0x7f27c0158720 0x7f27c0063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.777+0000 7f27c599a700 1 -- 172.21.15.33:0/2934666251 >> v1:172.21.15.69:6789/0 conn(0x7f27c0154340 legacy=0x7f27c0158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:21.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.777+0000 7f27c599a700 1 -- 172.21.15.33:0/2934666251 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c00c5d80 con 0x7f27c0154720 2023-11-06T21:02:21.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.777+0000 7f27c599a700 1 --2- 172.21.15.33:0/2934666251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0154720 0x7f27c00aae40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f27b0006fd0 tx=0x7f27b00090f0).ready entity=mon.2 client_cookie=d8223e387fc05405 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:21.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.777+0000 7f27c5199700 1 -- 172.21.15.33:0/2934666251 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b000f510 con 0x7f27c0154720 2023-11-06T21:02:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.777+0000 7f27c5199700 1 -- 172.21.15.33:0/2934666251 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b000f670 con 0x7f27c0154720 2023-11-06T21:02:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.777+0000 7f27c5199700 1 -- 172.21.15.33:0/2934666251 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b000f9f0 con 0x7f27c0154720 2023-11-06T21:02:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.778+0000 7f27c83ff700 1 -- 172.21.15.33:0/2934666251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0154720 msgr2=0x7f27c00aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.778+0000 7f27c83ff700 1 --2- 172.21.15.33:0/2934666251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0154720 0x7f27c00aae40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f27b0006fd0 tx=0x7f27b00090f0).stop 2023-11-06T21:02:21.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.778+0000 7f27c83ff700 1 -- 172.21.15.33:0/2934666251 shutdown_connections 2023-11-06T21:02:21.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.778+0000 7f27c83ff700 1 --2- 172.21.15.33:0/2934666251 >> v2:172.21.15.69:3300/0 conn(0x7f27c0158720 0x7f27c0063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.778+0000 7f27c83ff700 1 --2- 172.21.15.33:0/2934666251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0154720 0x7f27c00aae40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.778+0000 7f27c83ff700 1 -- 172.21.15.33:0/2934666251 >> 172.21.15.33:0/2934666251 conn(0x7f27c005ec80 msgr2=0x7f27c00cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:21.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.778+0000 7f27c83ff700 1 -- 172.21.15.33:0/2934666251 shutdown_connections 2023-11-06T21:02:21.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.778+0000 7f27c83ff700 1 -- 172.21.15.33:0/2934666251 wait complete. 2023-11-06T21:02:21.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.779+0000 7f27c83ff700 1 Processor -- start 2023-11-06T21:02:21.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.780+0000 7f27c83ff700 1 -- start start 2023-11-06T21:02:21.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c83ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27c0154340 0x7f27c0167d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c83ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c0154720 0x7f27c0155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c83ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0158720 0x7f27c00ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c83ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c00ade70 con 0x7f27c0154720 2023-11-06T21:02:21.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c83ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c00c8d20 con 0x7f27c0154340 2023-11-06T21:02:21.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c83ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c0168320 con 0x7f27c0158720 2023-11-06T21:02:21.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c0154720 0x7f27c0155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c0154720 0x7f27c0155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60056/0 (socket says 172.21.15.33:60056) 2023-11-06T21:02:21.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c599a700 1 -- 172.21.15.33:0/3935280959 learned_addr learned my addr 172.21.15.33:0/3935280959 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:21.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c619b700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27c0154340 0x7f27c0167d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c699c700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0158720 0x7f27c00ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:21.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c599a700 1 -- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0158720 msgr2=0x7f27c00ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:21.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c599a700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0158720 0x7f27c00ac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c599a700 1 -- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27c0154340 msgr2=0x7f27c0167d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:21.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c599a700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27c0154340 0x7f27c0167d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:21.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.781+0000 7f27c599a700 1 -- 172.21.15.33:0/3935280959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0009370 con 0x7f27c0154720 2023-11-06T21:02:21.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.782+0000 7f27c599a700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c0154720 0x7f27c0155500 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7f27b0011000 tx=0x7f27b0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:21.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.782+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b000fe80 con 0x7f27c0154720 2023-11-06T21:02:21.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.782+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b000f7e0 con 0x7f27c0154720 2023-11-06T21:02:21.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.782+0000 7f27c83ff700 1 -- 172.21.15.33:0/3935280959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c0168600 con 0x7f27c0154720 2023-11-06T21:02:21.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.782+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b001c630 con 0x7f27c0154720 2023-11-06T21:02:21.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.782+0000 7f27c83ff700 1 -- 172.21.15.33:0/3935280959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c0168bc0 con 0x7f27c0154720 2023-11-06T21:02:21.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.783+0000 7f27c83ff700 1 -- 172.21.15.33:0/3935280959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c00c8e90 con 0x7f27c0154720 2023-11-06T21:02:21.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.784+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 48) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f27b001ca20 con 0x7f27c0154720 2023-11-06T21:02:21.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.785+0000 7f27b67fc700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:21.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.785+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f27b007bff0 con 0x7f27c0154720 2023-11-06T21:02:21.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.785+0000 7f27c619b700 1 -- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 msgr2=0x7f27a4061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:02:21.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.785+0000 7f27c619b700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:02:21.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.789+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f27b004d080 con 0x7f27c0154720 2023-11-06T21:02:21.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.986+0000 7f27c619b700 1 -- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 msgr2=0x7f27a4061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:02:21.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.986+0000 7f27c619b700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:02:21.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:21.995+0000 7f27c83ff700 1 -- 172.21.15.33:0/3935280959 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f27c00cbcf0 con 0x7f27a405f2c0 2023-11-06T21:02:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:22.387+0000 7f27c619b700 1 -- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 msgr2=0x7f27a4061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:02:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:22.387+0000 7f27c619b700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T21:02:22.436 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:22.436+0000 7ffaf3c71680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:02:22.514 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:22.513+0000 7f4185fd5680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:02:22.520 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:22.520+0000 7ffaf3c71680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:02:22.600 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:22.600+0000 7f4185fd5680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:02:22.750 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:22.750+0000 7ffaf3c71680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:02:22.839 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:22.838+0000 7f4185fd5680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:02:23.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:23.188+0000 7f27c619b700 1 -- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 msgr2=0x7f27a4061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:02:23.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:23.188+0000 7f27c619b700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T21:02:23.260 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:23.260+0000 7ffaf3c71680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:02:23.373 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:23.372+0000 7f4185fd5680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:02:23.386 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:23.386+0000 7ffaf3c71680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:02:23.504 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:23.504+0000 7f4185fd5680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:02:23.574 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:23.571+0000 7ffaf3c71680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:02:23.664 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:23.664+0000 7ffaf3c71680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:02:23.695 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:23.694+0000 7f4185fd5680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:02:23.751 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:23.751+0000 7ffaf3c71680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:02:23.792 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:23.792+0000 7f4185fd5680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:02:23.881 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:23.881+0000 7f4185fd5680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:02:24.061 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:24.061+0000 7ffaf3c71680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:02:24.208 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:24.208+0000 7f4185fd5680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:02:24.282 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:24.282+0000 7ffaf3c71680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:02:24.436 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:24.436+0000 7f4185fd5680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:02:24.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:24.790+0000 7f27c619b700 1 -- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 msgr2=0x7f27a4061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:02:24.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:24.790+0000 7f27c619b700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T21:02:25.085 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:25.085+0000 7ffaf3c71680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:02:25.172 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:25.172+0000 7ffaf3c71680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:02:25.267 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:25.267+0000 7ffaf3c71680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:02:25.279 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:25.278+0000 7f4185fd5680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:02:25.370 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:25.369+0000 7f4185fd5680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:02:25.450 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:25.450+0000 7ffaf3c71680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:02:25.470 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:25.470+0000 7f4185fd5680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:02:25.539 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:25.539+0000 7ffaf3c71680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:02:25.664 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:25.663+0000 7f4185fd5680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:02:25.680 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:25.680+0000 7ffaf3c71680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:02:25.757 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:25.757+0000 7f4185fd5680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:02:25.810 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:25.810+0000 7ffaf3c71680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:02:25.906 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:25.906+0000 7f4185fd5680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:02:26.043 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:26.043+0000 7f4185fd5680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:02:26.156 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.156+0000 7ffaf3c71680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:02:26.247 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.247+0000 7ffaf3c71680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:02:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:26.258+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 49) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f27b0079b80 con 0x7f27c0154720 2023-11-06T21:02:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:26.258+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 msgr2=0x7f27a4061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:02:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:26.258+0000 7f27b67fc700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:02:26.332 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.332+0000 7ffa69747700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:02:26.333 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.332+0000 7ffa69747700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:02:26.333 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.332+0000 7ffa69747700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:02:26.333 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.332+0000 7ffa69747700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:02:26.333 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.332+0000 7ffa69747700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:02:26.334 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.333+0000 7ffa66741700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:02:26.334 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.333+0000 7ffa66741700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:02:26.334 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.333+0000 7ffa66741700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:02:26.335 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.333+0000 7ffa66741700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:02:26.335 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:02:26.333+0000 7ffa66741700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:02:26.406 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:26.406+0000 7f4185fd5680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:02:26.500 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:02:26.499+0000 7f4185fd5680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:02:27.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.263+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 50) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f27b0079f00 con 0x7f27c0154720 2023-11-06T21:02:27.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.264+0000 7f27b67fc700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:27.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.264+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f27c00cbcf0 con 0x7f27a405f2c0 2023-11-06T21:02:27.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.266+0000 7f27c619b700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:27.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.270+0000 7f27c619b700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f27c00c52f0 tx=0x7f27bc007bd0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:27.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.279+0000 7f27b67fc700 1 -- 172.21.15.33:0/3935280959 <== mgr.7435 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f27c00cbcf0 con 0x7f27a405f2c0 2023-11-06T21:02:27.280 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path snap_test_dir1 2023-11-06T21:02:27.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.281+0000 7f27c83ff700 1 -- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 msgr2=0x7f27a4061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:27.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.281+0000 7f27c83ff700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f27c00c52f0 tx=0x7f27bc007bd0).stop 2023-11-06T21:02:27.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.281+0000 7f27c83ff700 1 -- 172.21.15.33:0/3935280959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c0154720 msgr2=0x7f27c0155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:27.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.281+0000 7f27c83ff700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c0154720 0x7f27c0155500 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7f27b0011000 tx=0x7f27b0017000).stop 2023-11-06T21:02:27.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.281+0000 7f27c83ff700 1 -- 172.21.15.33:0/3935280959 shutdown_connections 2023-11-06T21:02:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.282+0000 7f27c83ff700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f27a405f2c0 0x7f27a4061700 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.282+0000 7f27c83ff700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f27c0158720 0x7f27c00ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.282+0000 7f27c83ff700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27c0154340 0x7f27c0167d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.282+0000 7f27c83ff700 1 --2- 172.21.15.33:0/3935280959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c0154720 0x7f27c0155500 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.282+0000 7f27c83ff700 1 -- 172.21.15.33:0/3935280959 >> 172.21.15.33:0/3935280959 conn(0x7f27c005ec80 msgr2=0x7f27c014ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.282+0000 7f27c83ff700 1 -- 172.21.15.33:0/3935280959 shutdown_connections 2023-11-06T21:02:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.282+0000 7f27c83ff700 1 -- 172.21.15.33:0/3935280959 wait complete. 2023-11-06T21:02:27.303 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-06T21:02:27.303 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-11-06-21_03_00 in ~33s... 2023-11-06T21:02:27.303 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-11-06T21:02:27.304 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-11-06T21:02:27.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.507+0000 7f1c557ff700 1 Processor -- start 2023-11-06T21:02:27.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.507+0000 7f1c557ff700 1 -- start start 2023-11-06T21:02:27.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.508+0000 7f1c557ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 0x7f1c500a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:27.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.508+0000 7f1c557ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1c501514c0 0x7f1c500bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:27.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.508+0000 7f1c557ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c50154c60 con 0x7f1c50151940 2023-11-06T21:02:27.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.508+0000 7f1c557ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c50068f50 con 0x7f1c501514c0 2023-11-06T21:02:27.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.508+0000 7f1c557ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c50151dc0 con 0x7f1c501510e0 2023-11-06T21:02:27.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.509+0000 7f1c4effd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 0x7f1c500a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.509+0000 7f1c4e7fc700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1c501514c0 0x7f1c500bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.509+0000 7f1c4effd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 0x7f1c500a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:57076/0 (socket says 172.21.15.33:57076) 2023-11-06T21:02:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.509+0000 7f1c4effd700 1 -- 172.21.15.33:0/3103664686 learned_addr learned my addr 172.21.15.33:0/3103664686 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.509+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/3103664686 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 msgr2=0x7f1c500a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:02:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.509+0000 7f1c4e7fc700 1 --2- 172.21.15.33:0/3103664686 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 0x7f1c500a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.509+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/3103664686 >> v1:172.21.15.69:6789/0 conn(0x7f1c50151940 legacy=0x7f1c50069430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.509+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/3103664686 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c501520a0 con 0x7f1c501514c0 2023-11-06T21:02:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.509+0000 7f1c4dffb700 1 -- 172.21.15.33:0/3103664686 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179231524 0 0) 0x7f1c50154c60 con 0x7f1c50151940 2023-11-06T21:02:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.510+0000 7f1c4e7fc700 1 --2- 172.21.15.33:0/3103664686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501514c0 0x7f1c500bb980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f1c44009760 tx=0x7f1c44009060).ready entity=mon.1 client_cookie=451bdf10bd3539c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.510+0000 7f1c4dffb700 1 -- 172.21.15.33:0/3103664686 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400f510 con 0x7f1c501514c0 2023-11-06T21:02:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.510+0000 7f1c4dffb700 1 -- 172.21.15.33:0/3103664686 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4400f670 con 0x7f1c501514c0 2023-11-06T21:02:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.510+0000 7f1c4dffb700 1 -- 172.21.15.33:0/3103664686 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400f9f0 con 0x7f1c501514c0 2023-11-06T21:02:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.510+0000 7f1c557ff700 1 -- 172.21.15.33:0/3103664686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501514c0 msgr2=0x7f1c500bb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.510+0000 7f1c557ff700 1 --2- 172.21.15.33:0/3103664686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501514c0 0x7f1c500bb980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f1c44009760 tx=0x7f1c44009060).stop 2023-11-06T21:02:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.511+0000 7f1c557ff700 1 -- 172.21.15.33:0/3103664686 shutdown_connections 2023-11-06T21:02:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.511+0000 7f1c557ff700 1 --2- 172.21.15.33:0/3103664686 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 0x7f1c500a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.511+0000 7f1c557ff700 1 --2- 172.21.15.33:0/3103664686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501514c0 0x7f1c500bb980 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.511+0000 7f1c557ff700 1 -- 172.21.15.33:0/3103664686 >> 172.21.15.33:0/3103664686 conn(0x7f1c50062710 msgr2=0x7f1c50062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.511+0000 7f1c557ff700 1 -- 172.21.15.33:0/3103664686 shutdown_connections 2023-11-06T21:02:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.511+0000 7f1c557ff700 1 -- 172.21.15.33:0/3103664686 wait complete. 2023-11-06T21:02:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.512+0000 7f1c557ff700 1 Processor -- start 2023-11-06T21:02:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.512+0000 7f1c557ff700 1 -- start start 2023-11-06T21:02:27.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.513+0000 7f1c557ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 0x7f1c500ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.513+0000 7f1c557ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c501514c0 0x7f1c501687e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.513+0000 7f1c557ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c50151940 0x7f1c501532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.513+0000 7f1c557ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c50168cd0 con 0x7f1c501514c0 2023-11-06T21:02:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.513+0000 7f1c557ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c500a8e30 con 0x7f1c50151940 2023-11-06T21:02:27.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.513+0000 7f1c557ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c500ab4e0 con 0x7f1c501510e0 2023-11-06T21:02:27.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.513+0000 7f1c4e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c501514c0 0x7f1c501687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:27.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.514+0000 7f1c4e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c501514c0 0x7f1c501687e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60058/0 (socket says 172.21.15.33:60058) 2023-11-06T21:02:27.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.514+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/3830167373 learned_addr learned my addr 172.21.15.33:0/3830167373 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:02:27.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.514+0000 7f1c4effd700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 0x7f1c500ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:27.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.514+0000 7f1c4f7fe700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c50151940 0x7f1c501532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:27.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.514+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 msgr2=0x7f1c500ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:27.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.514+0000 7f1c4e7fc700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 0x7f1c500ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.514+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c50151940 msgr2=0x7f1c501532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:27.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.514+0000 7f1c4e7fc700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c50151940 0x7f1c501532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.514+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/3830167373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c440092a0 con 0x7f1c501514c0 2023-11-06T21:02:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.514+0000 7f1c4e7fc700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c501514c0 0x7f1c501687e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f1c500c4530 tx=0x7f1c44017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.515+0000 7f1c36ffd700 1 -- 172.21.15.33:0/3830167373 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400fe80 con 0x7f1c501514c0 2023-11-06T21:02:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.515+0000 7f1c36ffd700 1 -- 172.21.15.33:0/3830167373 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4400f7e0 con 0x7f1c501514c0 2023-11-06T21:02:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.515+0000 7f1c557ff700 1 -- 172.21.15.33:0/3830167373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c500ab7c0 con 0x7f1c501514c0 2023-11-06T21:02:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.515+0000 7f1c36ffd700 1 -- 172.21.15.33:0/3830167373 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4401b640 con 0x7f1c501514c0 2023-11-06T21:02:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.515+0000 7f1c557ff700 1 -- 172.21.15.33:0/3830167373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c500abd80 con 0x7f1c501514c0 2023-11-06T21:02:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.516+0000 7f1c557ff700 1 -- 172.21.15.33:0/3830167373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c50152380 con 0x7f1c501514c0 2023-11-06T21:02:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.516+0000 7f1c36ffd700 1 -- 172.21.15.33:0/3830167373 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 50) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f1c4401ba70 con 0x7f1c501514c0 2023-11-06T21:02:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.517+0000 7f1c36ffd700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c3c05f080 0x7f1c3c061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:02:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.517+0000 7f1c36ffd700 1 -- 172.21.15.33:0/3830167373 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f1c4407bb90 con 0x7f1c501514c0 2023-11-06T21:02:27.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.518+0000 7f1c4effd700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c3c05f080 0x7f1c3c061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:02:27.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.518+0000 7f1c4effd700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c3c05f080 0x7f1c3c061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1c40006bc0 tx=0x7f1c400090f0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:02:27.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.522+0000 7f1c36ffd700 1 -- 172.21.15.33:0/3830167373 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1c4404cfe0 con 0x7f1c501514c0 2023-11-06T21:02:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.731+0000 7f1c557ff700 1 -- 172.21.15.33:0/3830167373 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c50065570 con 0x7f1c3c05f080 2023-11-06T21:02:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.736+0000 7f1c36ffd700 1 -- 172.21.15.33:0/3830167373 <== mgr.7435 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f1c50065570 con 0x7f1c3c05f080 2023-11-06T21:02:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:02:27.737 INFO:teuthology.orchestra.run.smithi033.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-11-06T21:02:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 -- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c3c05f080 msgr2=0x7f1c3c061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c3c05f080 0x7f1c3c061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1c40006bc0 tx=0x7f1c400090f0).stop 2023-11-06T21:02:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 -- 172.21.15.33:0/3830167373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c501514c0 msgr2=0x7f1c501687e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:02:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c501514c0 0x7f1c501687e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f1c500c4530 tx=0x7f1c44017000).stop 2023-11-06T21:02:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 -- 172.21.15.33:0/3830167373 shutdown_connections 2023-11-06T21:02:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1c3c05f080 0x7f1c3c061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1c501510e0 0x7f1c500ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c50151940 0x7f1c501532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 --2- 172.21.15.33:0/3830167373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c501514c0 0x7f1c501687e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:02:27.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 -- 172.21.15.33:0/3830167373 >> 172.21.15.33:0/3830167373 conn(0x7f1c50062710 msgr2=0x7f1c50064560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:02:27.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 -- 172.21.15.33:0/3830167373 shutdown_connections 2023-11-06T21:02:27.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:02:27.738+0000 7f1c557ff700 1 -- 172.21.15.33:0/3830167373 wait complete. 2023-11-06T21:02:27.756 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-11-06T21:02:27.757 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-11-06T21:02:27.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:27.758 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:28.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:28.809 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:29.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:29.865 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:30.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:30.922 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:31.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:31.978 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:33.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:33.035 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:34.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:34.091 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:35.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:35.148 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:36.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:36.205 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:37.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:37.262 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:38.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:38.319 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:39.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:39.374 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:39.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:02:39.425 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:02:40.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:40.430 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:41.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:41.487 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:42.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:42.543 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:43.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:43.599 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:44.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:44.655 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:45.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:45.711 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:46.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:46.767 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:47.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:47.822 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:48.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:48.879 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:49.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:49.938 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:50.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:50.994 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:52.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:52.049 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:53.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:53.104 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:54.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:54.161 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:55.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:55.217 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:56.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:56.274 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:57.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:57.330 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:58.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:58.386 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:02:59.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:02:59.443 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:03:00.286 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:00.286+0000 7ffa58f26700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:00.292 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:00.286+0000 7ffa58f26700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:00.292 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:00.286+0000 7ffa58f26700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:00.292 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:00.286+0000 7ffa58f26700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:00.292 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:00.286+0000 7ffa58f26700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:00.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:03:00.499 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:03:00.552 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_03_00_UTC 2023-11-06T21:03:00.553 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-11-06-21_03_00_UTC'] 2023-11-06T21:03:00.554 DEBUG:tasks.cephfs.test_snap_schedules:exec=1699304547.3034828, now = 1699304580.554263, timo = 33 2023-11-06T21:03:01.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-11-06T21:03:01.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.766+0000 7f20477ff700 1 Processor -- start 2023-11-06T21:03:01.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.766+0000 7f20477ff700 1 -- start start 2023-11-06T21:03:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.767+0000 7f20477ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2040153ed0 0x7f2040158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.767+0000 7f20477ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20400c8ee0 0x7f20400ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.767+0000 7f20477ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20400c9f80 con 0x7f20401542b0 2023-11-06T21:03:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.767+0000 7f20477ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20400c6000 con 0x7f2040153ed0 2023-11-06T21:03:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.767+0000 7f20477ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20400c9370 con 0x7f20400c8ee0 2023-11-06T21:03:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.768+0000 7f204559b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2040153ed0 0x7f2040158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.768+0000 7f2044d9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20400c8ee0 0x7f20400ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.768+0000 7f2044d9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20400c8ee0 0x7f20400ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:33946/0 (socket says 172.21.15.33:33946) 2023-11-06T21:03:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.768+0000 7f204559b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2040153ed0 0x7f2040158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:56700/0 (socket says 172.21.15.33:56700) 2023-11-06T21:03:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.768+0000 7f2044d9a700 1 -- 172.21.15.33:0/3866464205 learned_addr learned my addr 172.21.15.33:0/3866464205 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.769+0000 7f2044d9a700 1 -- 172.21.15.33:0/3866464205 >> v2:172.21.15.69:3300/0 conn(0x7f2040153ed0 msgr2=0x7f2040158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.769+0000 7f2044d9a700 1 --2- 172.21.15.33:0/3866464205 >> v2:172.21.15.69:3300/0 conn(0x7f2040153ed0 0x7f2040158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.769+0000 7f2044d9a700 1 -- 172.21.15.33:0/3866464205 >> v1:172.21.15.69:6789/0 conn(0x7f20401542b0 legacy=0x7f20400c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.769+0000 7f2044d9a700 1 -- 172.21.15.33:0/3866464205 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20400c9650 con 0x7f20400c8ee0 2023-11-06T21:03:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.769+0000 7f204559b700 1 --2- 172.21.15.33:0/3866464205 >> v2:172.21.15.69:3300/0 conn(0x7f2040153ed0 0x7f2040158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.769+0000 7f2037fff700 1 -- 172.21.15.33:0/3866464205 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230382229 0 0) 0x7f20400c9f80 con 0x7f20401542b0 2023-11-06T21:03:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.769+0000 7f2044d9a700 1 --2- 172.21.15.33:0/3866464205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20400c8ee0 0x7f20400ca410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f2030006fd0 tx=0x7f20300090f0).ready entity=mon.2 client_cookie=c9429336be96d02f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.769+0000 7f2037fff700 1 -- 172.21.15.33:0/3866464205 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203000b580 con 0x7f20400c8ee0 2023-11-06T21:03:01.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.769+0000 7f2037fff700 1 -- 172.21.15.33:0/3866464205 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203000b6e0 con 0x7f20400c8ee0 2023-11-06T21:03:01.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.769+0000 7f2037fff700 1 -- 172.21.15.33:0/3866464205 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203000ba60 con 0x7f20400c8ee0 2023-11-06T21:03:01.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.770+0000 7f20477ff700 1 -- 172.21.15.33:0/3866464205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20400c8ee0 msgr2=0x7f20400ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:01.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.770+0000 7f20477ff700 1 --2- 172.21.15.33:0/3866464205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20400c8ee0 0x7f20400ca410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f2030006fd0 tx=0x7f20300090f0).stop 2023-11-06T21:03:01.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.770+0000 7f20477ff700 1 -- 172.21.15.33:0/3866464205 shutdown_connections 2023-11-06T21:03:01.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.770+0000 7f20477ff700 1 --2- 172.21.15.33:0/3866464205 >> v2:172.21.15.69:3300/0 conn(0x7f2040153ed0 0x7f2040158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:01.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.770+0000 7f20477ff700 1 --2- 172.21.15.33:0/3866464205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20400c8ee0 0x7f20400ca410 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:01.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.770+0000 7f20477ff700 1 -- 172.21.15.33:0/3866464205 >> 172.21.15.33:0/3866464205 conn(0x7f204005e1b0 msgr2=0x7f204005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:01.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.770+0000 7f20477ff700 1 -- 172.21.15.33:0/3866464205 shutdown_connections 2023-11-06T21:03:01.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.770+0000 7f20477ff700 1 -- 172.21.15.33:0/3866464205 wait complete. 2023-11-06T21:03:01.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.772+0000 7f20477ff700 1 Processor -- start 2023-11-06T21:03:01.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.772+0000 7f20477ff700 1 -- start start 2023-11-06T21:03:01.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.773+0000 7f20477ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2040153ed0 0x7f204014fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:01.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.773+0000 7f20477ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20401542b0 0x7f20400aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:01.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.773+0000 7f20477ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20400af450 0x7f20401669e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:01.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.773+0000 7f20477ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2040165940 con 0x7f20400af450 2023-11-06T21:03:01.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.773+0000 7f20477ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2040165c20 con 0x7f2040153ed0 2023-11-06T21:03:01.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.773+0000 7f20477ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2040165f00 con 0x7f20401542b0 2023-11-06T21:03:01.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2045d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20400af450 0x7f20401669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:01.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2045d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20400af450 0x7f20401669e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33122/0 (socket says 172.21.15.33:33122) 2023-11-06T21:03:01.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2045d9c700 1 -- 172.21.15.33:0/3559845583 learned_addr learned my addr 172.21.15.33:0/3559845583 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:01.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2044d9a700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20401542b0 0x7f20400aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:01.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f204559b700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2040153ed0 0x7f204014fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:01.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2045d9c700 1 -- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20401542b0 msgr2=0x7f20400aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:01.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2045d9c700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20401542b0 0x7f20400aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2045d9c700 1 -- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2040153ed0 msgr2=0x7f204014fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2045d9c700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2040153ed0 0x7f204014fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2045d9c700 1 -- 172.21.15.33:0/3559845583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0092a0 con 0x7f20400af450 2023-11-06T21:03:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f204559b700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2040153ed0 0x7f204014fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:01.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2044d9a700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20401542b0 0x7f20400aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.774+0000 7f2045d9c700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20400af450 0x7f20401669e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f203800c810 tx=0x7f2038001500).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.775+0000 7f20357fa700 1 -- 172.21.15.33:0/3559845583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038010510 con 0x7f20400af450 2023-11-06T21:03:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.775+0000 7f20357fa700 1 -- 172.21.15.33:0/3559845583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2038010670 con 0x7f20400af450 2023-11-06T21:03:01.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.775+0000 7f20477ff700 1 -- 172.21.15.33:0/3559845583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2030009370 con 0x7f20400af450 2023-11-06T21:03:01.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.775+0000 7f20357fa700 1 -- 172.21.15.33:0/3559845583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038010a10 con 0x7f20400af450 2023-11-06T21:03:01.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.775+0000 7f20477ff700 1 -- 172.21.15.33:0/3559845583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20401664c0 con 0x7f20400af450 2023-11-06T21:03:01.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.776+0000 7f20477ff700 1 -- 172.21.15.33:0/3559845583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204006c6c0 con 0x7f20400af450 2023-11-06T21:03:01.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.777+0000 7f20357fa700 1 -- 172.21.15.33:0/3559845583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 51) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f2038012440 con 0x7f20400af450 2023-11-06T21:03:01.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.778+0000 7f20357fa700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202405f320 0x7f20240617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:01.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.778+0000 7f20357fa700 1 -- 172.21.15.33:0/3559845583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f203807a0c0 con 0x7f20400af450 2023-11-06T21:03:01.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.778+0000 7f204559b700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202405f320 0x7f20240617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:01.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.779+0000 7f204559b700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202405f320 0x7f20240617e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f203c009270 tx=0x7f203c008040).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:01.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.782+0000 7f20357fa700 1 -- 172.21.15.33:0/3559845583 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2038048af0 con 0x7f20400af450 2023-11-06T21:03:01.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.986+0000 7f20477ff700 1 -- 172.21.15.33:0/3559845583 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f20401666b0 con 0x7f202405f320 2023-11-06T21:03:01.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.991+0000 7f20357fa700 1 -- 172.21.15.33:0/3559845583 <== mgr.7435 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f20401666b0 con 0x7f202405f320 2023-11-06T21:03:01.991 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path snap_test_dir1 2023-11-06T21:03:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 -- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202405f320 msgr2=0x7f20240617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202405f320 0x7f20240617e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f203c009270 tx=0x7f203c008040).stop 2023-11-06T21:03:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 -- 172.21.15.33:0/3559845583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20400af450 msgr2=0x7f20401669e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20400af450 0x7f20401669e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f203800c810 tx=0x7f2038001500).stop 2023-11-06T21:03:01.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 -- 172.21.15.33:0/3559845583 shutdown_connections 2023-11-06T21:03:01.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202405f320 0x7f20240617e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:01.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20400af450 0x7f20401669e0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:01.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2040153ed0 0x7f204014fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:01.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 --2- 172.21.15.33:0/3559845583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20401542b0 0x7f20400aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:01.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 -- 172.21.15.33:0/3559845583 >> 172.21.15.33:0/3559845583 conn(0x7f204005e1b0 msgr2=0x7f204014e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 -- 172.21.15.33:0/3559845583 shutdown_connections 2023-11-06T21:03:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:01.993+0000 7f20477ff700 1 -- 172.21.15.33:0/3559845583 wait complete. 2023-11-06T21:03:02.011 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-11-06T21:03:02.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:03:02.012 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:03:02.060 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_03_00_UTC 2023-11-06T21:03:02.061 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_03_00_UTC 2023-11-06T21:03:02.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:03:02.062 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_03_00_UTC) 2023-11-06T21:03:02.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:03:02.116 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-06T21:03:02.166 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.364+0000 7f870e1ff700 1 Processor -- start 2023-11-06T21:03:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.364+0000 7f870e1ff700 1 -- start start 2023-11-06T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.365+0000 7f870e1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8708152850 0x7f8708065ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.365+0000 7f870e1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8708063f20 0x7f87080bcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.365+0000 7f870e1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87080bc7c0 con 0x7f8708152c30 2023-11-06T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.365+0000 7f870e1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87080a5d20 con 0x7f8708063f20 2023-11-06T21:03:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.365+0000 7f870e1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87080a6000 con 0x7f8708152850 2023-11-06T21:03:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.366+0000 7f8706ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8708063f20 0x7f87080bcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.366+0000 7f8706ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8708063f20 0x7f87080bcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33130/0 (socket says 172.21.15.33:33130) 2023-11-06T21:03:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.366+0000 7f8706ffd700 1 -- 172.21.15.33:0/2520018541 learned_addr learned my addr 172.21.15.33:0/2520018541 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.366+0000 7f87077fe700 1 --2- 172.21.15.33:0/2520018541 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8708152850 0x7f8708065ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.366+0000 7f8706ffd700 1 -- 172.21.15.33:0/2520018541 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8708152850 msgr2=0x7f8708065ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.366+0000 7f8706ffd700 1 --2- 172.21.15.33:0/2520018541 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8708152850 0x7f8708065ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.366+0000 7f8706ffd700 1 -- 172.21.15.33:0/2520018541 >> v1:172.21.15.69:6789/0 conn(0x7f8708152c30 legacy=0x7f87081530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.366+0000 7f8706ffd700 1 -- 172.21.15.33:0/2520018541 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87080a62e0 con 0x7f8708063f20 2023-11-06T21:03:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.366+0000 7f8706ffd700 1 --2- 172.21.15.33:0/2520018541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708063f20 0x7f87080bcc50 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f86fc006fd0 tx=0x7f86fc0090f0).ready entity=mon.0 client_cookie=c769553b0b133117 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f87067fc700 1 -- 172.21.15.33:0/2520018541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00b600 con 0x7f8708063f20 2023-11-06T21:03:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f87067fc700 1 -- 172.21.15.33:0/2520018541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc00b760 con 0x7f8708063f20 2023-11-06T21:03:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f87067fc700 1 -- 172.21.15.33:0/2520018541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00bb00 con 0x7f8708063f20 2023-11-06T21:03:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f870e1ff700 1 -- 172.21.15.33:0/2520018541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708063f20 msgr2=0x7f87080bcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f870e1ff700 1 --2- 172.21.15.33:0/2520018541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708063f20 0x7f87080bcc50 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f86fc006fd0 tx=0x7f86fc0090f0).stop 2023-11-06T21:03:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f870e1ff700 1 -- 172.21.15.33:0/2520018541 shutdown_connections 2023-11-06T21:03:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f870e1ff700 1 --2- 172.21.15.33:0/2520018541 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8708152850 0x7f8708065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f870e1ff700 1 --2- 172.21.15.33:0/2520018541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708063f20 0x7f87080bcc50 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f870e1ff700 1 -- 172.21.15.33:0/2520018541 >> 172.21.15.33:0/2520018541 conn(0x7f87080c6a40 msgr2=0x7f87080c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f870e1ff700 1 -- 172.21.15.33:0/2520018541 shutdown_connections 2023-11-06T21:03:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.367+0000 7f870e1ff700 1 -- 172.21.15.33:0/2520018541 wait complete. 2023-11-06T21:03:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.369+0000 7f870e1ff700 1 Processor -- start 2023-11-06T21:03:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.369+0000 7f870e1ff700 1 -- start start 2023-11-06T21:03:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f870e1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8708063f20 0x7f8708168920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f870e1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708152850 0x7f87080a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f870e1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708152c30 0x7f870814eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f870e1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87081562e0 con 0x7f8708152850 2023-11-06T21:03:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f870e1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8708155b70 con 0x7f8708152c30 2023-11-06T21:03:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f870e1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8708168490 con 0x7f8708063f20 2023-11-06T21:03:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f8706ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708152850 0x7f87080a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f8706ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708152850 0x7f87080a6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33142/0 (socket says 172.21.15.33:33142) 2023-11-06T21:03:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f8706ffd700 1 -- 172.21.15.33:0/688322839 learned_addr learned my addr 172.21.15.33:0/688322839 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f87077fe700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8708063f20 0x7f8708168920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f8707fff700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708152c30 0x7f870814eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f8706ffd700 1 -- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8708063f20 msgr2=0x7f8708168920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f8706ffd700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8708063f20 0x7f8708168920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f8706ffd700 1 -- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708152c30 msgr2=0x7f870814eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f8706ffd700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708152c30 0x7f870814eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f8706ffd700 1 -- 172.21.15.33:0/688322839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc009370 con 0x7f8708152850 2023-11-06T21:03:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.370+0000 7f8707fff700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708152c30 0x7f870814eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.371+0000 7f8706ffd700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708152850 0x7f87080a6f30 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f86fc011000 tx=0x7f86fc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.371+0000 7f86ef7fe700 1 -- 172.21.15.33:0/688322839 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc006d90 con 0x7f8708152850 2023-11-06T21:03:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.371+0000 7f86ef7fe700 1 -- 172.21.15.33:0/688322839 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc00bec0 con 0x7f8708152850 2023-11-06T21:03:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.371+0000 7f86ef7fe700 1 -- 172.21.15.33:0/688322839 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc006930 con 0x7f8708152850 2023-11-06T21:03:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.371+0000 7f870e1ff700 1 -- 172.21.15.33:0/688322839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87080a7420 con 0x7f8708152850 2023-11-06T21:03:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.371+0000 7f870e1ff700 1 -- 172.21.15.33:0/688322839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8708167e40 con 0x7f8708152850 2023-11-06T21:03:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.372+0000 7f870e1ff700 1 -- 172.21.15.33:0/688322839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8708168600 con 0x7f8708152850 2023-11-06T21:03:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.373+0000 7f86ef7fe700 1 -- 172.21.15.33:0/688322839 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 51) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f86fc021070 con 0x7f8708152850 2023-11-06T21:03:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.373+0000 7f86ef7fe700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f86f405f350 0x7f86f4061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.374+0000 7f86ef7fe700 1 -- 172.21.15.33:0/688322839 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f86fc07c0f0 con 0x7f8708152850 2023-11-06T21:03:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.374+0000 7f87077fe700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f86f405f350 0x7f86f4061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.374+0000 7f87077fe700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f86f405f350 0x7f86f4061810 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f87080aca30 tx=0x7f86f8008040).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.378+0000 7f86ef7fe700 1 -- 172.21.15.33:0/688322839 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f86fc04cfc0 con 0x7f8708152850 2023-11-06T21:03:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:02.603+0000 7f870e1ff700 1 -- 172.21.15.33:0/688322839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f87080bc450 con 0x7f8708152850 2023-11-06T21:03:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.102+0000 7f86ef7fe700 1 -- 172.21.15.33:0/688322839 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f86fc079ac0 con 0x7f8708152850 2023-11-06T21:03:03.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.103+0000 7f86ef7fe700 1 -- 172.21.15.33:0/688322839 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v52) v1 ==== 93+0+0 (secure 0 0 0) 0x7f86fc0424a0 con 0x7f8708152850 2023-11-06T21:03:03.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 -- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f86f405f350 msgr2=0x7f86f4061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:03.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f86f405f350 0x7f86f4061810 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f87080aca30 tx=0x7f86f8008040).stop 2023-11-06T21:03:03.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 -- 172.21.15.33:0/688322839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708152850 msgr2=0x7f87080a6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:03.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708152850 0x7f87080a6f30 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f86fc011000 tx=0x7f86fc017000).stop 2023-11-06T21:03:03.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 -- 172.21.15.33:0/688322839 shutdown_connections 2023-11-06T21:03:03.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f86f405f350 0x7f86f4061810 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8708063f20 0x7f8708168920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8708152c30 0x7f870814eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 --2- 172.21.15.33:0/688322839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708152850 0x7f87080a6f30 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 -- 172.21.15.33:0/688322839 >> 172.21.15.33:0/688322839 conn(0x7f87080c6a40 msgr2=0x7f87080c6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:03.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 -- 172.21.15.33:0/688322839 shutdown_connections 2023-11-06T21:03:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.107+0000 7f870e1ff700 1 -- 172.21.15.33:0/688322839 wait complete. 2023-11-06T21:03:03.128 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T21:03:03.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:03:03.229 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:03:03.319 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T21:03:03.319 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T21:03:03.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:03:03.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T21:03:03.324 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:03.324+0000 7f63222b3680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:03:03.338 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:03.338+0000 7fc95cf60680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:03:03.404 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:03.404+0000 7f63222b3680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:03:03.408 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T21:03:03.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:03:03.409 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T21:03:03.429 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:03.428+0000 7fc95cf60680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:03:03.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:03:03.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.674+0000 7fc3dfbff700 1 Processor -- start 2023-11-06T21:03:03.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.674+0000 7fc3dfbff700 1 -- start start 2023-11-06T21:03:03.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dfbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3d8151dc0 0x7fc3d80c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:03.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dfbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc3d81521a0 0x7fc3d8063110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:03.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dfbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d8154e90 con 0x7fc3d8155320 2023-11-06T21:03:03.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dfbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d80639d0 con 0x7fc3d8151dc0 2023-11-06T21:03:03.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dfbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d8063cb0 con 0x7fc3d81521a0 2023-11-06T21:03:03.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dd99b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3d8151dc0 0x7fc3d80c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:03.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dd99b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3d8151dc0 0x7fc3d80c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33166/0 (socket says 172.21.15.33:33166) 2023-11-06T21:03:03.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dd99b700 1 -- 172.21.15.33:0/1038042572 learned_addr learned my addr 172.21.15.33:0/1038042572 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:03.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dd19a700 1 --2- 172.21.15.33:0/1038042572 >> v2:172.21.15.69:3300/0 conn(0x7fc3d81521a0 0x7fc3d8063110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:03.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dd99b700 1 -- 172.21.15.33:0/1038042572 >> v2:172.21.15.69:3300/0 conn(0x7fc3d81521a0 msgr2=0x7fc3d8063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:03.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dd99b700 1 --2- 172.21.15.33:0/1038042572 >> v2:172.21.15.69:3300/0 conn(0x7fc3d81521a0 0x7fc3d8063110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dd99b700 1 -- 172.21.15.33:0/1038042572 >> v1:172.21.15.69:6789/0 conn(0x7fc3d8155320 legacy=0x7fc3d80c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:03.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.676+0000 7fc3dd99b700 1 -- 172.21.15.33:0/1038042572 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8063f90 con 0x7fc3d8151dc0 2023-11-06T21:03:03.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.677+0000 7fc3dd99b700 1 --2- 172.21.15.33:0/1038042572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d8151dc0 0x7fc3d80c0c50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fc3d80c0680 tx=0x7fc3d40090f0).ready entity=mon.0 client_cookie=d1264a580d1fec76 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:03.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.677+0000 7fc3dc999700 1 -- 172.21.15.33:0/1038042572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d400b520 con 0x7fc3d8151dc0 2023-11-06T21:03:03.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.677+0000 7fc3dc999700 1 -- 172.21.15.33:0/1038042572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d400b680 con 0x7fc3d8151dc0 2023-11-06T21:03:03.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.677+0000 7fc3dc999700 1 -- 172.21.15.33:0/1038042572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d400ba00 con 0x7fc3d8151dc0 2023-11-06T21:03:03.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.677+0000 7fc3dfbff700 1 -- 172.21.15.33:0/1038042572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d8151dc0 msgr2=0x7fc3d80c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:03.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.677+0000 7fc3dfbff700 1 --2- 172.21.15.33:0/1038042572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d8151dc0 0x7fc3d80c0c50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fc3d80c0680 tx=0x7fc3d40090f0).stop 2023-11-06T21:03:03.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.678+0000 7fc3dfbff700 1 -- 172.21.15.33:0/1038042572 shutdown_connections 2023-11-06T21:03:03.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.678+0000 7fc3dfbff700 1 --2- 172.21.15.33:0/1038042572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d8151dc0 0x7fc3d80c0c50 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.678+0000 7fc3dfbff700 1 --2- 172.21.15.33:0/1038042572 >> v2:172.21.15.69:3300/0 conn(0x7fc3d81521a0 0x7fc3d8063110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.678+0000 7fc3dfbff700 1 -- 172.21.15.33:0/1038042572 >> 172.21.15.33:0/1038042572 conn(0x7fc3d805e1b0 msgr2=0x7fc3d805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:03.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.678+0000 7fc3dfbff700 1 -- 172.21.15.33:0/1038042572 shutdown_connections 2023-11-06T21:03:03.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.678+0000 7fc3dfbff700 1 -- 172.21.15.33:0/1038042572 wait complete. 2023-11-06T21:03:03.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.679+0000 7fc3dfbff700 1 Processor -- start 2023-11-06T21:03:03.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.679+0000 7fc3dfbff700 1 -- start start 2023-11-06T21:03:03.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.680+0000 7fc3dfbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8151dc0 0x7fc3d80c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:03.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.680+0000 7fc3dfbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3d81521a0 0x7fc3d80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:03.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.680+0000 7fc3dfbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d8155320 0x7fc3d8150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:03.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.680+0000 7fc3dfbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80c3b50 con 0x7fc3d8155320 2023-11-06T21:03:03.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.680+0000 7fc3dfbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80c3e60 con 0x7fc3d8151dc0 2023-11-06T21:03:03.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.680+0000 7fc3dfbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80c2550 con 0x7fc3d81521a0 2023-11-06T21:03:03.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3dd19a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3d81521a0 0x7fc3d80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:03.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3dd99b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8151dc0 0x7fc3d80c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:03.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3dd19a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3d81521a0 0x7fc3d80c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:33996/0 (socket says 172.21.15.33:33996) 2023-11-06T21:03:03.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3dd19a700 1 -- 172.21.15.33:0/3307604034 learned_addr learned my addr 172.21.15.33:0/3307604034 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:03.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3de19c700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d8155320 0x7fc3d8150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:03.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3dd19a700 1 -- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8151dc0 msgr2=0x7fc3d80c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:03.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3dd19a700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8151dc0 0x7fc3d80c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3dd19a700 1 -- 172.21.15.33:0/3307604034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d8155320 msgr2=0x7fc3d8150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3dd19a700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d8155320 0x7fc3d8150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3dd19a700 1 -- 172.21.15.33:0/3307604034 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d4009370 con 0x7fc3d81521a0 2023-11-06T21:03:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.681+0000 7fc3dd19a700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3d81521a0 0x7fc3d80c3660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fc3c800e950 tx=0x7fc3c8009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.682+0000 7fc3cdffb700 1 -- 172.21.15.33:0/3307604034 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8009530 con 0x7fc3d81521a0 2023-11-06T21:03:03.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.682+0000 7fc3cdffb700 1 -- 172.21.15.33:0/3307604034 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c8012600 con 0x7fc3d81521a0 2023-11-06T21:03:03.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.682+0000 7fc3cdffb700 1 -- 172.21.15.33:0/3307604034 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8012980 con 0x7fc3d81521a0 2023-11-06T21:03:03.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.682+0000 7fc3dfbff700 1 -- 172.21.15.33:0/3307604034 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d80c2830 con 0x7fc3d81521a0 2023-11-06T21:03:03.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.682+0000 7fc3dfbff700 1 -- 172.21.15.33:0/3307604034 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d80c2df0 con 0x7fc3d81521a0 2023-11-06T21:03:03.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.683+0000 7fc3dfbff700 1 -- 172.21.15.33:0/3307604034 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d8154b20 con 0x7fc3d81521a0 2023-11-06T21:03:03.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.683+0000 7fc3cdffb700 1 -- 172.21.15.33:0/3307604034 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fc3c8012d30 con 0x7fc3d81521a0 2023-11-06T21:03:03.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.684+0000 7fc3cdffb700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc3bc05f390 0x7fc3bc061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:03.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.684+0000 7fc3cdffb700 1 -- 172.21.15.33:0/3307604034 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fc3c807abd0 con 0x7fc3d81521a0 2023-11-06T21:03:03.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.684+0000 7fc3dd99b700 1 -- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc3bc05f390 msgr2=0x7fc3bc061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:03.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.685+0000 7fc3dd99b700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc3bc05f390 0x7fc3bc061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:03.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.689+0000 7fc3cdffb700 1 -- 172.21.15.33:0/3307604034 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc3c804b9e0 con 0x7fc3d81521a0 2023-11-06T21:03:03.733 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:03.733+0000 7f63222b3680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:03:03.789 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:03.789+0000 7fc95cf60680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:03:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.886+0000 7fc3dd99b700 1 -- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc3bc05f390 msgr2=0x7fc3bc061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.886+0000 7fc3dd99b700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc3bc05f390 0x7fc3bc061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:03:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.914+0000 7fc3dfbff700 1 -- 172.21.15.33:0/3307604034 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3d8062ac0 con 0x7fc3d81521a0 2023-11-06T21:03:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.915+0000 7fc3cdffb700 1 -- 172.21.15.33:0/3307604034 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4491 (secure 0 0 0) 0x7fc3c804f000 con 0x7fc3d81521a0 2023-11-06T21:03:03.916 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:03.916 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":53,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2500558031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2500558031},{"type":"v1","addr":"172.21.15.69:6837","nonce":2500558031}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1012474146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1012474146},{"type":"v1","addr":"172.21.15.69:6836","nonce":1012474146}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4020289695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4020289695},{"type":"v1","addr":"172.21.15.33:6834","nonce":4020289695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:02:11.991421+0000","modified":"2023-11-06T21:02:15.032423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1130278640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1130278640},{"type":"v1","addr":"172.21.15.33:6835","nonce":1130278640}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-06T21:03:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 -- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc3bc05f390 msgr2=0x7fc3bc061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc3bc05f390 0x7fc3bc061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:03.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 -- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3d81521a0 msgr2=0x7fc3d80c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:03.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3d81521a0 0x7fc3d80c3660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fc3c800e950 tx=0x7fc3c8009060).stop 2023-11-06T21:03:03.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 -- 172.21.15.33:0/3307604034 shutdown_connections 2023-11-06T21:03:03.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc3bc05f390 0x7fc3bc061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:03.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d8155320 0x7fc3d8150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3d8151dc0 0x7fc3d80c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 --2- 172.21.15.33:0/3307604034 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3d81521a0 0x7fc3d80c3660 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 -- 172.21.15.33:0/3307604034 >> 172.21.15.33:0/3307604034 conn(0x7fc3d805e1b0 msgr2=0x7fc3d80ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 -- 172.21.15.33:0/3307604034 shutdown_connections 2023-11-06T21:03:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:03.917+0000 7fc3dfbff700 1 -- 172.21.15.33:0/3307604034 wait complete. 2023-11-06T21:03:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 53 2023-11-06T21:03:03.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.144+0000 7f6a8efff700 1 Processor -- start 2023-11-06T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.145+0000 7f6a8efff700 1 -- start start 2023-11-06T21:03:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.146+0000 7f6a8efff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a8814c640 0x7f6a880bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.146+0000 7f6a8efff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6a880c33f0 0x7f6a880c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.146+0000 7f6a8efff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a88153470 con 0x7f6a88152ff0 2023-11-06T21:03:04.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.146+0000 7f6a8efff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a880bd8f0 con 0x7f6a8814c640 2023-11-06T21:03:04.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.146+0000 7f6a8efff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a880643a0 con 0x7f6a880c33f0 2023-11-06T21:03:04.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.146+0000 7f6a8cd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a8814c640 0x7f6a880bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:04.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.146+0000 7f6a8cd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a8814c640 0x7f6a880bb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33184/0 (socket says 172.21.15.33:33184) 2023-11-06T21:03:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.146+0000 7f6a8cd9b700 1 -- 172.21.15.33:0/3543681024 learned_addr learned my addr 172.21.15.33:0/3543681024 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.146+0000 7f6a8d59c700 1 --2- 172.21.15.33:0/3543681024 >> v2:172.21.15.69:3300/0 conn(0x7f6a880c33f0 0x7f6a880c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.147+0000 7f6a8cd9b700 1 -- 172.21.15.33:0/3543681024 >> v2:172.21.15.69:3300/0 conn(0x7f6a880c33f0 msgr2=0x7f6a880c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.147+0000 7f6a8cd9b700 1 --2- 172.21.15.33:0/3543681024 >> v2:172.21.15.69:3300/0 conn(0x7f6a880c33f0 0x7f6a880c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.147+0000 7f6a8cd9b700 1 -- 172.21.15.33:0/3543681024 >> v1:172.21.15.69:6789/0 conn(0x7f6a88152ff0 legacy=0x7f6a88065160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.147+0000 7f6a8cd9b700 1 -- 172.21.15.33:0/3543681024 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a88064680 con 0x7f6a8814c640 2023-11-06T21:03:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.147+0000 7f6a8cd9b700 1 --2- 172.21.15.33:0/3543681024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8814c640 0x7f6a880bb990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f6a880c0fc0 tx=0x7f6a7c0090f0).ready entity=mon.0 client_cookie=98119c32f807a745 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:04.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.147+0000 7f6a877fe700 1 -- 172.21.15.33:0/3543681024 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00b520 con 0x7f6a8814c640 2023-11-06T21:03:04.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.147+0000 7f6a877fe700 1 -- 172.21.15.33:0/3543681024 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c00b680 con 0x7f6a8814c640 2023-11-06T21:03:04.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.147+0000 7f6a877fe700 1 -- 172.21.15.33:0/3543681024 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00ba00 con 0x7f6a8814c640 2023-11-06T21:03:04.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.148+0000 7f6a8efff700 1 -- 172.21.15.33:0/3543681024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8814c640 msgr2=0x7f6a880bb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:04.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.148+0000 7f6a8efff700 1 --2- 172.21.15.33:0/3543681024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8814c640 0x7f6a880bb990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f6a880c0fc0 tx=0x7f6a7c0090f0).stop 2023-11-06T21:03:04.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.148+0000 7f6a8efff700 1 -- 172.21.15.33:0/3543681024 shutdown_connections 2023-11-06T21:03:04.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.148+0000 7f6a8efff700 1 --2- 172.21.15.33:0/3543681024 >> v2:172.21.15.69:3300/0 conn(0x7f6a880c33f0 0x7f6a880c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.148+0000 7f6a8efff700 1 --2- 172.21.15.33:0/3543681024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8814c640 0x7f6a880bb990 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.148+0000 7f6a8efff700 1 -- 172.21.15.33:0/3543681024 >> 172.21.15.33:0/3543681024 conn(0x7f6a88062710 msgr2=0x7f6a88062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.148+0000 7f6a8efff700 1 -- 172.21.15.33:0/3543681024 shutdown_connections 2023-11-06T21:03:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.148+0000 7f6a8efff700 1 -- 172.21.15.33:0/3543681024 wait complete. 2023-11-06T21:03:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.150+0000 7f6a8efff700 1 Processor -- start 2023-11-06T21:03:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.150+0000 7f6a8efff700 1 -- start start 2023-11-06T21:03:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8efff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a880c33f0 0x7f6a8814f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6a8814c640 0x7f6a880c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8efff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a88152ff0 0x7f6a8814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8efff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a88166670 con 0x7f6a880c33f0 2023-11-06T21:03:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8efff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a88165a60 con 0x7f6a88152ff0 2023-11-06T21:03:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a88165d40 con 0x7f6a8814c640 2023-11-06T21:03:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a880c33f0 0x7f6a8814f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a880c33f0 0x7f6a8814f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33196/0 (socket says 172.21.15.33:33196) 2023-11-06T21:03:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8cd9b700 1 -- 172.21.15.33:0/3878914080 learned_addr learned my addr 172.21.15.33:0/3878914080 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8d59c700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a88152ff0 0x7f6a8814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a87fff700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6a8814c640 0x7f6a880c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:04.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8cd9b700 1 -- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6a8814c640 msgr2=0x7f6a880c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:04.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8cd9b700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6a8814c640 0x7f6a880c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8cd9b700 1 -- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a88152ff0 msgr2=0x7f6a8814a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:04.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.151+0000 7f6a8cd9b700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a88152ff0 0x7f6a8814a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.152+0000 7f6a8cd9b700 1 -- 172.21.15.33:0/3878914080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c009370 con 0x7f6a880c33f0 2023-11-06T21:03:04.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.152+0000 7f6a87fff700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6a8814c640 0x7f6a880c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:04.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.152+0000 7f6a8cd9b700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a880c33f0 0x7f6a8814f360 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f6a7c009130 tx=0x7f6a7c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:04.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.152+0000 7f6a84ff9700 1 -- 172.21.15.33:0/3878914080 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c000bc0 con 0x7f6a880c33f0 2023-11-06T21:03:04.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.152+0000 7f6a84ff9700 1 -- 172.21.15.33:0/3878914080 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c00b7f0 con 0x7f6a880c33f0 2023-11-06T21:03:04.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.152+0000 7f6a8efff700 1 -- 172.21.15.33:0/3878914080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a88166020 con 0x7f6a880c33f0 2023-11-06T21:03:04.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.152+0000 7f6a84ff9700 1 -- 172.21.15.33:0/3878914080 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c000d20 con 0x7f6a880c33f0 2023-11-06T21:03:04.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.152+0000 7f6a8efff700 1 -- 172.21.15.33:0/3878914080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a880a9110 con 0x7f6a880c33f0 2023-11-06T21:03:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.153+0000 7f6a8efff700 1 -- 172.21.15.33:0/3878914080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a88064960 con 0x7f6a880c33f0 2023-11-06T21:03:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.154+0000 7f6a84ff9700 1 -- 172.21.15.33:0/3878914080 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f6a7c01c8c0 con 0x7f6a880c33f0 2023-11-06T21:03:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.155+0000 7f6a84ff9700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a6c05f390 0x7f6a6c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.155+0000 7f6a84ff9700 1 -- 172.21.15.33:0/3878914080 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f6a7c07c370 con 0x7f6a880c33f0 2023-11-06T21:03:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.155+0000 7f6a87fff700 1 -- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a6c05f390 msgr2=0x7f6a6c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.155+0000 7f6a87fff700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a6c05f390 0x7f6a6c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.159+0000 7f6a84ff9700 1 -- 172.21.15.33:0/3878914080 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6a7c04d180 con 0x7f6a880c33f0 2023-11-06T21:03:04.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.341+0000 7f6a8efff700 1 -- 172.21.15.33:0/3878914080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a8805f0b0 con 0x7f6a880c33f0 2023-11-06T21:03:04.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.342+0000 7f6a84ff9700 1 -- 172.21.15.33:0/3878914080 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+17359 (secure 0 0 0) 0x7f6a7c02a020 con 0x7f6a880c33f0 2023-11-06T21:03:04.343 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:04.343 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":132,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:02:26.250736+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:02:09.533597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T21:02:10.978394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6832/114912":"2023-11-07T21:02:26.250703+0000","172.21.15.69:6833/114912":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/3344790742":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/874235539":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/2223677285":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/3213553706":"2023-11-07T21:02:26.250703+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:03:04.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.344+0000 7f6a8efff700 1 -- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a6c05f390 msgr2=0x7f6a6c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:04.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.344+0000 7f6a8efff700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a6c05f390 0x7f6a6c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:04.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.345+0000 7f6a8efff700 1 -- 172.21.15.33:0/3878914080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a880c33f0 msgr2=0x7f6a8814f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:04.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.345+0000 7f6a8efff700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a880c33f0 0x7f6a8814f360 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f6a7c009130 tx=0x7f6a7c017000).stop 2023-11-06T21:03:04.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.345+0000 7f6a8efff700 1 -- 172.21.15.33:0/3878914080 shutdown_connections 2023-11-06T21:03:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.345+0000 7f6a8efff700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a6c05f390 0x7f6a6c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.345+0000 7f6a8efff700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a88152ff0 0x7f6a8814a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.345+0000 7f6a8efff700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a880c33f0 0x7f6a8814f360 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.345+0000 7f6a8efff700 1 --2- 172.21.15.33:0/3878914080 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6a8814c640 0x7f6a880c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.345+0000 7f6a8efff700 1 -- 172.21.15.33:0/3878914080 >> 172.21.15.33:0/3878914080 conn(0x7f6a88062710 msgr2=0x7f6a880c4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:04.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.345+0000 7f6a8efff700 1 -- 172.21.15.33:0/3878914080 shutdown_connections 2023-11-06T21:03:04.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.345+0000 7f6a8efff700 1 -- 172.21.15.33:0/3878914080 wait complete. 2023-11-06T21:03:04.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T21:02:09.533597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T21:02:10.978394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T21:03:04.365 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T21:03:04.477 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:04.476+0000 7f63222b3680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:03:04.559 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:04.559+0000 7f63222b3680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:03:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.598+0000 7fd27dfff700 1 Processor -- start 2023-11-06T21:03:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.599+0000 7fd27dfff700 1 -- start start 2023-11-06T21:03:04.600 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:04.599+0000 7fc95cf60680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:03:04.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.600+0000 7fd27dfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd278154290 0x7fd2780a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.600+0000 7fd27dfff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd278064cf0 0x7fd2780c0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.600+0000 7fd27dfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2780c0820 con 0x7fd2780aa310 2023-11-06T21:03:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.600+0000 7fd27dfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2780bfc50 con 0x7fd278154290 2023-11-06T21:03:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.600+0000 7fd27dfff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2780bfef0 con 0x7fd278064cf0 2023-11-06T21:03:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.600+0000 7fd2777fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd278154290 0x7fd2780a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.600+0000 7fd276ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd2780aa310 0x7fd2780645e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44462/0 (socket says 172.21.15.33:44462) 2023-11-06T21:03:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.601+0000 7fd276ffd700 1 -- 172.21.15.33:0/1599689552 learned_addr learned my addr 172.21.15.33:0/1599689552 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.601+0000 7fd277fff700 1 --2- 172.21.15.33:0/1599689552 >> v2:172.21.15.69:3300/0 conn(0x7fd278064cf0 0x7fd2780c0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.601+0000 7fd277fff700 1 -- 172.21.15.33:0/1599689552 >> v2:172.21.15.33:3300/0 conn(0x7fd278154290 msgr2=0x7fd2780a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.601+0000 7fd277fff700 1 --2- 172.21.15.33:0/1599689552 >> v2:172.21.15.33:3300/0 conn(0x7fd278154290 0x7fd2780a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.601+0000 7fd277fff700 1 -- 172.21.15.33:0/1599689552 >> v1:172.21.15.33:6789/0 conn(0x7fd2780aa310 legacy=0x7fd2780645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.601+0000 7fd277fff700 1 -- 172.21.15.33:0/1599689552 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780c01d0 con 0x7fd278064cf0 2023-11-06T21:03:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.601+0000 7fd2777fe700 1 --2- 172.21.15.33:0/1599689552 >> v2:172.21.15.33:3300/0 conn(0x7fd278154290 0x7fd2780a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.601+0000 7fd277fff700 1 --2- 172.21.15.33:0/1599689552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd278064cf0 0x7fd2780c0cb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fd27805fef0 tx=0x7fd270009780).ready entity=mon.1 client_cookie=585dbabe01a89629 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.602+0000 7fd2767fc700 1 -- 172.21.15.33:0/1599689552 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27000e070 con 0x7fd278064cf0 2023-11-06T21:03:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.602+0000 7fd2767fc700 1 -- 172.21.15.33:0/1599689552 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27000bb60 con 0x7fd278064cf0 2023-11-06T21:03:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.602+0000 7fd2767fc700 1 -- 172.21.15.33:0/1599689552 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270012410 con 0x7fd278064cf0 2023-11-06T21:03:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.602+0000 7fd27dfff700 1 -- 172.21.15.33:0/1599689552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd278064cf0 msgr2=0x7fd2780c0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.602+0000 7fd27dfff700 1 --2- 172.21.15.33:0/1599689552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd278064cf0 0x7fd2780c0cb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fd27805fef0 tx=0x7fd270009780).stop 2023-11-06T21:03:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.602+0000 7fd27dfff700 1 -- 172.21.15.33:0/1599689552 shutdown_connections 2023-11-06T21:03:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.602+0000 7fd27dfff700 1 --2- 172.21.15.33:0/1599689552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd278064cf0 0x7fd2780c0cb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.602+0000 7fd27dfff700 1 --2- 172.21.15.33:0/1599689552 >> v2:172.21.15.33:3300/0 conn(0x7fd278154290 0x7fd2780a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.602+0000 7fd27dfff700 1 -- 172.21.15.33:0/1599689552 >> 172.21.15.33:0/1599689552 conn(0x7fd2780cb800 msgr2=0x7fd2780c7790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:04.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.602+0000 7fd27dfff700 1 -- 172.21.15.33:0/1599689552 shutdown_connections 2023-11-06T21:03:04.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.603+0000 7fd27dfff700 1 -- 172.21.15.33:0/1599689552 wait complete. 2023-11-06T21:03:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.604+0000 7fd27dfff700 1 Processor -- start 2023-11-06T21:03:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.604+0000 7fd27dfff700 1 -- start start 2023-11-06T21:03:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.605+0000 7fd27dfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278064cf0 0x7fd2781534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.605+0000 7fd27dfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2780aa310 0x7fd27814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.605+0000 7fd27dfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd278154290 0x7fd2780afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.605+0000 7fd27dfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd278156270 con 0x7fd278064cf0 2023-11-06T21:03:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.605+0000 7fd27dfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd278156550 con 0x7fd2780aa310 2023-11-06T21:03:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.605+0000 7fd27dfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd278156830 con 0x7fd278154290 2023-11-06T21:03:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.605+0000 7fd2777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278064cf0 0x7fd2781534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd2777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278064cf0 0x7fd2781534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33204/0 (socket says 172.21.15.33:33204) 2023-11-06T21:03:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd2777fe700 1 -- 172.21.15.33:0/2231943790 learned_addr learned my addr 172.21.15.33:0/2231943790 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd276ffd700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2780aa310 0x7fd27814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd277fff700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd278154290 0x7fd2780afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd2777fe700 1 -- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd278154290 msgr2=0x7fd2780afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd2777fe700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd278154290 0x7fd2780afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd2777fe700 1 -- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2780aa310 msgr2=0x7fd27814e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:04.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd2777fe700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2780aa310 0x7fd27814e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd2777fe700 1 -- 172.21.15.33:0/2231943790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2680092a0 con 0x7fd278064cf0 2023-11-06T21:03:04.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd277fff700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd278154290 0x7fd2780afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:04.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.606+0000 7fd2777fe700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278064cf0 0x7fd2781534a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7fd26800ea20 tx=0x7fd268009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:04.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.607+0000 7fd25f7fe700 1 -- 172.21.15.33:0/2231943790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800c850 con 0x7fd278064cf0 2023-11-06T21:03:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.607+0000 7fd25f7fe700 1 -- 172.21.15.33:0/2231943790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26800c9b0 con 0x7fd278064cf0 2023-11-06T21:03:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.607+0000 7fd27dfff700 1 -- 172.21.15.33:0/2231943790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd270009a20 con 0x7fd278064cf0 2023-11-06T21:03:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.607+0000 7fd25f7fe700 1 -- 172.21.15.33:0/2231943790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800cd30 con 0x7fd278064cf0 2023-11-06T21:03:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.607+0000 7fd27dfff700 1 -- 172.21.15.33:0/2231943790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd278156df0 con 0x7fd278064cf0 2023-11-06T21:03:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.608+0000 7fd27dfff700 1 -- 172.21.15.33:0/2231943790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd278060830 con 0x7fd278064cf0 2023-11-06T21:03:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.609+0000 7fd25f7fe700 1 -- 172.21.15.33:0/2231943790 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fd268012600 con 0x7fd278064cf0 2023-11-06T21:03:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.610+0000 7fd25f7fe700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd26405f3c0 0x7fd264061880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.610+0000 7fd25f7fe700 1 -- 172.21.15.33:0/2231943790 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fd26807ace0 con 0x7fd278064cf0 2023-11-06T21:03:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.610+0000 7fd276ffd700 1 -- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd26405f3c0 msgr2=0x7fd264061880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.610+0000 7fd276ffd700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd26405f3c0 0x7fd264061880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.614+0000 7fd25f7fe700 1 -- 172.21.15.33:0/2231943790 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd26804baf0 con 0x7fd278064cf0 2023-11-06T21:03:04.690 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:04.689+0000 7fc95cf60680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:03:04.791 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:04.791+0000 7f63222b3680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:03:04.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.795+0000 7fd27dfff700 1 -- 172.21.15.33:0/2231943790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd2780475a0 con 0x7fd278064cf0 2023-11-06T21:03:04.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.795+0000 7fd25f7fe700 1 -- 172.21.15.33:0/2231943790 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v132) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd2780475a0 con 0x7fd278064cf0 2023-11-06T21:03:04.796 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T21:03:04.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 -- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd26405f3c0 msgr2=0x7fd264061880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:04.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd26405f3c0 0x7fd264061880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:04.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 -- 172.21.15.33:0/2231943790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278064cf0 msgr2=0x7fd2781534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:04.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278064cf0 0x7fd2781534a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7fd26800ea20 tx=0x7fd268009060).stop 2023-11-06T21:03:04.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 -- 172.21.15.33:0/2231943790 shutdown_connections 2023-11-06T21:03:04.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd26405f3c0 0x7fd264061880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:04.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd278154290 0x7fd2780afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd278064cf0 0x7fd2781534a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 --2- 172.21.15.33:0/2231943790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2780aa310 0x7fd27814e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:04.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 -- 172.21.15.33:0/2231943790 >> 172.21.15.33:0/2231943790 conn(0x7fd2780cb800 msgr2=0x7fd2780ca0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:04.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 -- 172.21.15.33:0/2231943790 shutdown_connections 2023-11-06T21:03:04.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:04.798+0000 7fd27dfff700 1 -- 172.21.15.33:0/2231943790 wait complete. 2023-11-06T21:03:04.817 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T21:03:04.943 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:04.942+0000 7fc95cf60680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:03:05.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.025+0000 7faafe1ff700 1 Processor -- start 2023-11-06T21:03:05.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.025+0000 7faafe1ff700 1 -- start start 2023-11-06T21:03:05.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.026+0000 7faafe1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faaf81546a0 0x7faaf8064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.026+0000 7faafe1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7faaf8066560 0x7faaf8066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.026+0000 7faafe1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf80641f0 con 0x7faaf80bdbb0 2023-11-06T21:03:05.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.026+0000 7faafe1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf8154a80 con 0x7faaf81546a0 2023-11-06T21:03:05.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.026+0000 7faafe1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf80638c0 con 0x7faaf8066560 2023-11-06T21:03:05.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.027+0000 7faaf77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faaf81546a0 0x7faaf8064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.027+0000 7faaf77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faaf81546a0 0x7faaf8064680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33210/0 (socket says 172.21.15.33:33210) 2023-11-06T21:03:05.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.027+0000 7faaf77fe700 1 -- 172.21.15.33:0/4069807021 learned_addr learned my addr 172.21.15.33:0/4069807021 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:05.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.027+0000 7faaf6ffd700 1 --2- 172.21.15.33:0/4069807021 >> v2:172.21.15.69:3300/0 conn(0x7faaf8066560 0x7faaf8066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.027+0000 7faaf77fe700 1 -- 172.21.15.33:0/4069807021 >> v2:172.21.15.69:3300/0 conn(0x7faaf8066560 msgr2=0x7faaf8066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.027+0000 7faaf77fe700 1 --2- 172.21.15.33:0/4069807021 >> v2:172.21.15.69:3300/0 conn(0x7faaf8066560 0x7faaf8066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.027+0000 7faaf77fe700 1 -- 172.21.15.33:0/4069807021 >> v1:172.21.15.69:6789/0 conn(0x7faaf80bdbb0 legacy=0x7faaf80c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.027+0000 7faaf77fe700 1 -- 172.21.15.33:0/4069807021 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8063ba0 con 0x7faaf81546a0 2023-11-06T21:03:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.027+0000 7faaf77fe700 1 --2- 172.21.15.33:0/4069807021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf81546a0 0x7faaf8064680 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7faaf80c3180 tx=0x7faae80090f0).ready entity=mon.0 client_cookie=a13ddf75c53e6b0c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.028+0000 7faaf67fc700 1 -- 172.21.15.33:0/4069807021 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae800b520 con 0x7faaf81546a0 2023-11-06T21:03:05.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.028+0000 7faaf67fc700 1 -- 172.21.15.33:0/4069807021 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae800b680 con 0x7faaf81546a0 2023-11-06T21:03:05.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.028+0000 7faaf67fc700 1 -- 172.21.15.33:0/4069807021 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae800ba00 con 0x7faaf81546a0 2023-11-06T21:03:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.028+0000 7faafe1ff700 1 -- 172.21.15.33:0/4069807021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf81546a0 msgr2=0x7faaf8064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.028+0000 7faafe1ff700 1 --2- 172.21.15.33:0/4069807021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf81546a0 0x7faaf8064680 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7faaf80c3180 tx=0x7faae80090f0).stop 2023-11-06T21:03:05.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.028+0000 7faafe1ff700 1 -- 172.21.15.33:0/4069807021 shutdown_connections 2023-11-06T21:03:05.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.028+0000 7faafe1ff700 1 --2- 172.21.15.33:0/4069807021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf81546a0 0x7faaf8064680 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.028+0000 7faafe1ff700 1 --2- 172.21.15.33:0/4069807021 >> v2:172.21.15.69:3300/0 conn(0x7faaf8066560 0x7faaf8066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.028+0000 7faafe1ff700 1 -- 172.21.15.33:0/4069807021 >> 172.21.15.33:0/4069807021 conn(0x7faaf80c8c50 msgr2=0x7faaf80c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:05.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.029+0000 7faafe1ff700 1 -- 172.21.15.33:0/4069807021 shutdown_connections 2023-11-06T21:03:05.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.029+0000 7faafe1ff700 1 -- 172.21.15.33:0/4069807021 wait complete. 2023-11-06T21:03:05.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.031+0000 7faafe1ff700 1 Processor -- start 2023-11-06T21:03:05.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.031+0000 7faafe1ff700 1 -- start start 2023-11-06T21:03:05.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.032+0000 7faafe1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf8066560 0x7faaf80acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.032+0000 7faafe1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaf80bdbb0 0x7faaf80ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.032+0000 7faafe1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaf81546a0 0x7faaf8156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.032+0000 7faafe1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80ae610 con 0x7faaf8066560 2023-11-06T21:03:05.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.032+0000 7faafe1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80ada00 con 0x7faaf80bdbb0 2023-11-06T21:03:05.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.032+0000 7faafe1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80adce0 con 0x7faaf81546a0 2023-11-06T21:03:05.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.032+0000 7faaf77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf8066560 0x7faaf80acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.032+0000 7faaf77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf8066560 0x7faaf80acc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33220/0 (socket says 172.21.15.33:33220) 2023-11-06T21:03:05.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.032+0000 7faaf77fe700 1 -- 172.21.15.33:0/755682469 learned_addr learned my addr 172.21.15.33:0/755682469 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:05.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.033+0000 7faaf6ffd700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaf80bdbb0 0x7faaf80ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.033+0000 7faaf7fff700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaf81546a0 0x7faaf8156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.033+0000 7faaf77fe700 1 -- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaf81546a0 msgr2=0x7faaf8156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.033+0000 7faaf77fe700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaf81546a0 0x7faaf8156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.033+0000 7faaf77fe700 1 -- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaf80bdbb0 msgr2=0x7faaf80ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.033+0000 7faaf77fe700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaf80bdbb0 0x7faaf80ad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.033+0000 7faaf77fe700 1 -- 172.21.15.33:0/755682469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae8009370 con 0x7faaf8066560 2023-11-06T21:03:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.033+0000 7faaf77fe700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf8066560 0x7faaf80acc90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7faae8009130 tx=0x7faae8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.033+0000 7faadf7fe700 1 -- 172.21.15.33:0/755682469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8000bc0 con 0x7faaf8066560 2023-11-06T21:03:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.034+0000 7faadf7fe700 1 -- 172.21.15.33:0/755682469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae800b7f0 con 0x7faaf8066560 2023-11-06T21:03:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.034+0000 7faadf7fe700 1 -- 172.21.15.33:0/755682469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8000d20 con 0x7faaf8066560 2023-11-06T21:03:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.034+0000 7faafe1ff700 1 -- 172.21.15.33:0/755682469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf80adfc0 con 0x7faaf8066560 2023-11-06T21:03:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.034+0000 7faafe1ff700 1 -- 172.21.15.33:0/755682469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf80ad670 con 0x7faaf8066560 2023-11-06T21:03:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.035+0000 7faafe1ff700 1 -- 172.21.15.33:0/755682469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf8066000 con 0x7faaf8066560 2023-11-06T21:03:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.036+0000 7faadf7fe700 1 -- 172.21.15.33:0/755682469 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7faae801b8b0 con 0x7faaf8066560 2023-11-06T21:03:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.037+0000 7faadf7fe700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faae405f390 0x7faae4061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.037+0000 7faadf7fe700 1 -- 172.21.15.33:0/755682469 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6379+0+0 (secure 0 0 0) 0x7faae807bff0 con 0x7faaf8066560 2023-11-06T21:03:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.037+0000 7faaf6ffd700 1 -- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faae405f390 msgr2=0x7faae4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.037+0000 7faaf6ffd700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faae405f390 0x7faae4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.042+0000 7faadf7fe700 1 -- 172.21.15.33:0/755682469 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faae804ce60 con 0x7faaf8066560 2023-11-06T21:03:05.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.223+0000 7faafe1ff700 1 -- 172.21.15.33:0/755682469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faaf806a7b0 con 0x7faaf8066560 2023-11-06T21:03:05.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.224+0000 7faadf7fe700 1 -- 172.21.15.33:0/755682469 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7faae801f020 con 0x7faaf8066560 2023-11-06T21:03:05.225 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T21:03:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.226+0000 7faafe1ff700 1 -- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faae405f390 msgr2=0x7faae4061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.226+0000 7faafe1ff700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faae405f390 0x7faae4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.226+0000 7faafe1ff700 1 -- 172.21.15.33:0/755682469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf8066560 msgr2=0x7faaf80acc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.226+0000 7faafe1ff700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf8066560 0x7faaf80acc90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7faae8009130 tx=0x7faae8017000).stop 2023-11-06T21:03:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.226+0000 7faafe1ff700 1 -- 172.21.15.33:0/755682469 shutdown_connections 2023-11-06T21:03:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.226+0000 7faafe1ff700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7faae405f390 0x7faae4061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:05.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.226+0000 7faafe1ff700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faaf81546a0 0x7faaf8156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.226+0000 7faafe1ff700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf8066560 0x7faaf80acc90 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.226+0000 7faafe1ff700 1 --2- 172.21.15.33:0/755682469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faaf80bdbb0 0x7faaf80ad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.226+0000 7faafe1ff700 1 -- 172.21.15.33:0/755682469 >> 172.21.15.33:0/755682469 conn(0x7faaf80c8c50 msgr2=0x7faaf8152670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:05.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.227+0000 7faafe1ff700 1 -- 172.21.15.33:0/755682469 shutdown_connections 2023-11-06T21:03:05.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.227+0000 7faafe1ff700 1 -- 172.21.15.33:0/755682469 wait complete. 2023-11-06T21:03:05.246 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T21:03:05.305 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:05.305+0000 7f63222b3680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:03:05.429 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:05.429+0000 7f63222b3680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:03:05.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.447+0000 7f61d7bff700 1 Processor -- start 2023-11-06T21:03:05.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.447+0000 7f61d7bff700 1 -- start start 2023-11-06T21:03:05.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.449+0000 7f61d7bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f61d01546a0 0x7f61d0064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.449+0000 7f61d7bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61d0066560 0x7f61d0066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.449+0000 7f61d7bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d00bdbb0 0x7f61d00be040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.449+0000 7f61d7bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d0154a80 con 0x7f61d0066560 2023-11-06T21:03:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.449+0000 7f61d7bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d00641f0 con 0x7f61d01546a0 2023-11-06T21:03:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.449+0000 7f61d7bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00be530 con 0x7f61d00bdbb0 2023-11-06T21:03:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d519a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61d0066560 0x7f61d0066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d519a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61d0066560 0x7f61d0066a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33234/0 (socket says 172.21.15.33:33234) 2023-11-06T21:03:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d519a700 1 -- 172.21.15.33:0/1223102961 learned_addr learned my addr 172.21.15.33:0/1223102961 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:05.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d619c700 1 --2- 172.21.15.33:0/1223102961 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d00bdbb0 0x7f61d00be040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d599b700 1 --2- 172.21.15.33:0/1223102961 >> v2:172.21.15.69:3300/0 conn(0x7f61d01546a0 0x7f61d0064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d519a700 1 -- 172.21.15.33:0/1223102961 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d00bdbb0 msgr2=0x7f61d00be040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:05.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d519a700 1 --2- 172.21.15.33:0/1223102961 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d00bdbb0 0x7f61d00be040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d519a700 1 -- 172.21.15.33:0/1223102961 >> v2:172.21.15.69:3300/0 conn(0x7f61d01546a0 msgr2=0x7f61d0064680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d519a700 1 --2- 172.21.15.33:0/1223102961 >> v2:172.21.15.69:3300/0 conn(0x7f61d01546a0 0x7f61d0064680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d519a700 1 -- 172.21.15.33:0/1223102961 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00638c0 con 0x7f61d0066560 2023-11-06T21:03:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.450+0000 7f61d519a700 1 --2- 172.21.15.33:0/1223102961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d0066560 0x7f61d0066a10 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f61d00c68d0 tx=0x7f61c00090f0).ready entity=mon.0 client_cookie=7d5df59f51db6772 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d4999700 1 -- 172.21.15.33:0/1223102961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c000f510 con 0x7f61d0066560 2023-11-06T21:03:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d4999700 1 -- 172.21.15.33:0/1223102961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c000f670 con 0x7f61d0066560 2023-11-06T21:03:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d4999700 1 -- 172.21.15.33:0/1223102961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c000fa10 con 0x7f61d0066560 2023-11-06T21:03:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d7bff700 1 -- 172.21.15.33:0/1223102961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d0066560 msgr2=0x7f61d0066a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d7bff700 1 --2- 172.21.15.33:0/1223102961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d0066560 0x7f61d0066a10 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f61d00c68d0 tx=0x7f61c00090f0).stop 2023-11-06T21:03:05.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d7bff700 1 -- 172.21.15.33:0/1223102961 shutdown_connections 2023-11-06T21:03:05.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d7bff700 1 --2- 172.21.15.33:0/1223102961 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d00bdbb0 0x7f61d00be040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d7bff700 1 --2- 172.21.15.33:0/1223102961 >> v2:172.21.15.69:3300/0 conn(0x7f61d01546a0 0x7f61d0064680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d7bff700 1 --2- 172.21.15.33:0/1223102961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d0066560 0x7f61d0066a10 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d7bff700 1 -- 172.21.15.33:0/1223102961 >> 172.21.15.33:0/1223102961 conn(0x7f61d00c8c50 msgr2=0x7f61d00c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:05.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d7bff700 1 -- 172.21.15.33:0/1223102961 shutdown_connections 2023-11-06T21:03:05.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.451+0000 7f61d7bff700 1 -- 172.21.15.33:0/1223102961 wait complete. 2023-11-06T21:03:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.453+0000 7f61d7bff700 1 Processor -- start 2023-11-06T21:03:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.453+0000 7f61d7bff700 1 -- start start 2023-11-06T21:03:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d7bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d0066560 0x7f61d0156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d7bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d00bdbb0 0x7f61d0067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d7bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d01546a0 0x7f61d0067fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d7bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0068490 con 0x7f61d01546a0 2023-11-06T21:03:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d7bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00abda0 con 0x7f61d00bdbb0 2023-11-06T21:03:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d7bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0157b40 con 0x7f61d0066560 2023-11-06T21:03:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d619c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d01546a0 0x7f61d0067fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d599b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d0066560 0x7f61d0156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d619c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d01546a0 0x7f61d0067fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33250/0 (socket says 172.21.15.33:33250) 2023-11-06T21:03:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d619c700 1 -- 172.21.15.33:0/590432973 learned_addr learned my addr 172.21.15.33:0/590432973 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.454+0000 7f61d519a700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d00bdbb0 0x7f61d0067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61d619c700 1 -- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d0066560 msgr2=0x7f61d0156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61d619c700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d0066560 0x7f61d0156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61d619c700 1 -- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d00bdbb0 msgr2=0x7f61d0067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61d619c700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d00bdbb0 0x7f61d0067ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61d619c700 1 -- 172.21.15.33:0/590432973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c0009370 con 0x7f61d01546a0 2023-11-06T21:03:05.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61d599b700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d0066560 0x7f61d0156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:05.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61d619c700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d01546a0 0x7f61d0067fa0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f61c8009e90 tx=0x7f61c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:05.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61c5ffb700 1 -- 172.21.15.33:0/590432973 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800caa0 con 0x7f61d01546a0 2023-11-06T21:03:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61c5ffb700 1 -- 172.21.15.33:0/590432973 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c800f040 con 0x7f61d01546a0 2023-11-06T21:03:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61d7bff700 1 -- 172.21.15.33:0/590432973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d01572a0 con 0x7f61d01546a0 2023-11-06T21:03:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61c5ffb700 1 -- 172.21.15.33:0/590432973 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800ce10 con 0x7f61d01546a0 2023-11-06T21:03:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.455+0000 7f61d7bff700 1 -- 172.21.15.33:0/590432973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d0157860 con 0x7f61d01546a0 2023-11-06T21:03:05.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.456+0000 7f61d7bff700 1 -- 172.21.15.33:0/590432973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d00abf10 con 0x7f61d01546a0 2023-11-06T21:03:05.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.457+0000 7f61c5ffb700 1 -- 172.21.15.33:0/590432973 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f61c8013610 con 0x7f61d01546a0 2023-11-06T21:03:05.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.458+0000 7f61c5ffb700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61b405f390 0x7f61b4061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.458+0000 7f61c5ffb700 1 -- 172.21.15.33:0/590432973 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f61c807afc0 con 0x7f61d01546a0 2023-11-06T21:03:05.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.458+0000 7f61d599b700 1 -- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61b405f390 msgr2=0x7f61b4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.458+0000 7f61d599b700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61b405f390 0x7f61b4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:05.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.462+0000 7f61c5ffb700 1 -- 172.21.15.33:0/590432973 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f61d00abf10 con 0x7f61d01546a0 2023-11-06T21:03:05.503 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:05.502+0000 7fc95cf60680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:03:05.615 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:05.614+0000 7f63222b3680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:03:05.636 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:05.636+0000 7fc95cf60680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:03:05.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.644+0000 7f61d7bff700 1 -- 172.21.15.33:0/590432973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61d00abf10 con 0x7f61d01546a0 2023-11-06T21:03:05.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.644+0000 7f61c5ffb700 1 -- 172.21.15.33:0/590432973 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61d00abf10 con 0x7f61d01546a0 2023-11-06T21:03:05.646 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T21:03:05.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 -- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61b405f390 msgr2=0x7f61b4061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:05.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61b405f390 0x7f61b4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:05.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 -- 172.21.15.33:0/590432973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d01546a0 msgr2=0x7f61d0067fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d01546a0 0x7f61d0067fa0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f61c8009e90 tx=0x7f61c800b040).stop 2023-11-06T21:03:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 -- 172.21.15.33:0/590432973 shutdown_connections 2023-11-06T21:03:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f61b405f390 0x7f61b4061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f61d0066560 0x7f61d0156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61d01546a0 0x7f61d0067fa0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 --2- 172.21.15.33:0/590432973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d00bdbb0 0x7f61d0067ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 -- 172.21.15.33:0/590432973 >> 172.21.15.33:0/590432973 conn(0x7f61d00c8c50 msgr2=0x7f61d00c6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 -- 172.21.15.33:0/590432973 shutdown_connections 2023-11-06T21:03:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.647+0000 7f61d7bff700 1 -- 172.21.15.33:0/590432973 wait complete. 2023-11-06T21:03:05.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T21:03:05.709 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:05.709+0000 7f63222b3680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:03:05.795 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:05.795+0000 7f63222b3680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:03:05.839 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:05.838+0000 7fc95cf60680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:03:05.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.870+0000 7fd148fff700 1 Processor -- start 2023-11-06T21:03:05.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.870+0000 7fd148fff700 1 -- start start 2023-11-06T21:03:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.872+0000 7fd148fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd144158980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.872+0000 7fd148fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1441584f0 con 0x7fd1440632f0 2023-11-06T21:03:05.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.872+0000 7fd148fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd144158e70 con 0x7fd1440636d0 2023-11-06T21:03:05.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.872+0000 7fd148fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1441559e0 con 0x7fd14414e9e0 2023-11-06T21:03:05.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.872+0000 7fd14259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd1440632f0 0x7fd1441549d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44478/0 (socket says 172.21.15.33:44478) 2023-11-06T21:03:05.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.872+0000 7fd14259c700 1 -- 172.21.15.33:0/965340380 learned_addr learned my addr 172.21.15.33:0/965340380 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:05.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.872+0000 7fd142d9d700 1 --2- 172.21.15.33:0/965340380 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd144158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd14159a700 1 -- 172.21.15.33:0/965340380 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492229475 0 0) 0x7fd1441584f0 con 0x7fd1440632f0 2023-11-06T21:03:05.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd14159a700 1 -- 172.21.15.33:0/965340380 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd12c0033f0 con 0x7fd1440632f0 2023-11-06T21:03:05.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd14159a700 1 -- 172.21.15.33:0/965340380 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704427690 0 0) 0x7fd144158e70 con 0x7fd1440636d0 2023-11-06T21:03:05.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd14159a700 1 -- 172.21.15.33:0/965340380 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1441584f0 con 0x7fd1440636d0 2023-11-06T21:03:05.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd142d9d700 1 -- 172.21.15.33:0/965340380 >> v1:172.21.15.69:6789/0 conn(0x7fd1440636d0 legacy=0x7fd14414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd142d9d700 1 -- 172.21.15.33:0/965340380 >> v1:172.21.15.33:6789/0 conn(0x7fd1440632f0 legacy=0x7fd1441549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd142d9d700 1 -- 172.21.15.33:0/965340380 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144155420 con 0x7fd14414e9e0 2023-11-06T21:03:05.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd142d9d700 1 --2- 172.21.15.33:0/965340380 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd144158980 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fd13c009e90 tx=0x7fd13c009780).ready entity=mon.2 client_cookie=ede09755306003b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:05.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd14159a700 1 -- 172.21.15.33:0/965340380 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00e070 con 0x7fd14414e9e0 2023-11-06T21:03:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd14159a700 1 -- 172.21.15.33:0/965340380 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c00bab0 con 0x7fd14414e9e0 2023-11-06T21:03:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.873+0000 7fd14159a700 1 -- 172.21.15.33:0/965340380 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00be30 con 0x7fd14414e9e0 2023-11-06T21:03:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.874+0000 7fd148fff700 1 -- 172.21.15.33:0/965340380 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 msgr2=0x7fd144158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.874+0000 7fd148fff700 1 --2- 172.21.15.33:0/965340380 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd144158980 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fd13c009e90 tx=0x7fd13c009780).stop 2023-11-06T21:03:05.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.874+0000 7fd148fff700 1 -- 172.21.15.33:0/965340380 shutdown_connections 2023-11-06T21:03:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.874+0000 7fd148fff700 1 --2- 172.21.15.33:0/965340380 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd144158980 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.874+0000 7fd148fff700 1 -- 172.21.15.33:0/965340380 >> 172.21.15.33:0/965340380 conn(0x7fd1440cb910 msgr2=0x7fd1440c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.874+0000 7fd148fff700 1 -- 172.21.15.33:0/965340380 shutdown_connections 2023-11-06T21:03:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.874+0000 7fd148fff700 1 -- 172.21.15.33:0/965340380 wait complete. 2023-11-06T21:03:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.877+0000 7fd148fff700 1 Processor -- start 2023-11-06T21:03:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.877+0000 7fd148fff700 1 -- start start 2023-11-06T21:03:05.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.878+0000 7fd148fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1440632f0 0x7fd1441642f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.878+0000 7fd148fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1440636d0 0x7fd1440c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.878+0000 7fd148fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd1440b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.878+0000 7fd148fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd144155bb0 con 0x7fd1440636d0 2023-11-06T21:03:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.878+0000 7fd148fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd144164910 con 0x7fd1440632f0 2023-11-06T21:03:05.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.878+0000 7fd148fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd144164bf0 con 0x7fd14414e9e0 2023-11-06T21:03:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd14259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1440632f0 0x7fd1441642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd141d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1440636d0 0x7fd1440c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd142d9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd1440b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:05.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd14259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1440632f0 0x7fd1441642f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:56840/0 (socket says 172.21.15.33:56840) 2023-11-06T21:03:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd142d9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd1440b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:34028/0 (socket says 172.21.15.33:34028) 2023-11-06T21:03:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd141d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1440636d0 0x7fd1440c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33254/0 (socket says 172.21.15.33:33254) 2023-11-06T21:03:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd14259c700 1 -- 172.21.15.33:0/1894609750 learned_addr learned my addr 172.21.15.33:0/1894609750 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd142d9d700 1 -- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1440632f0 msgr2=0x7fd1441642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd142d9d700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1440632f0 0x7fd1441642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd142d9d700 1 -- 172.21.15.33:0/1894609750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1440636d0 msgr2=0x7fd1440c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd142d9d700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1440636d0 0x7fd1440c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd142d9d700 1 -- 172.21.15.33:0/1894609750 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c009a20 con 0x7fd14414e9e0 2023-11-06T21:03:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.879+0000 7fd142d9d700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd1440b0540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fd13c014000 tx=0x7fd13c009780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.880+0000 7fd1327fc700 1 -- 172.21.15.33:0/1894609750 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c01d630 con 0x7fd14414e9e0 2023-11-06T21:03:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.880+0000 7fd1327fc700 1 -- 172.21.15.33:0/1894609750 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c01d790 con 0x7fd14414e9e0 2023-11-06T21:03:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.880+0000 7fd1327fc700 1 -- 172.21.15.33:0/1894609750 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c01db50 con 0x7fd14414e9e0 2023-11-06T21:03:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.880+0000 7fd148fff700 1 -- 172.21.15.33:0/1894609750 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd144164ed0 con 0x7fd14414e9e0 2023-11-06T21:03:05.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.880+0000 7fd148fff700 1 -- 172.21.15.33:0/1894609750 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1440c5120 con 0x7fd14414e9e0 2023-11-06T21:03:05.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.881+0000 7fd148fff700 1 -- 172.21.15.33:0/1894609750 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd144155d20 con 0x7fd14414e9e0 2023-11-06T21:03:05.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.882+0000 7fd1327fc700 1 -- 172.21.15.33:0/1894609750 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fd13c003040 con 0x7fd14414e9e0 2023-11-06T21:03:05.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.883+0000 7fd1327fc700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd12405f390 0x7fd124061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:05.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.883+0000 7fd1327fc700 1 -- 172.21.15.33:0/1894609750 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fd13c07c6d0 con 0x7fd14414e9e0 2023-11-06T21:03:05.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.883+0000 7fd14259c700 1 -- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd12405f390 msgr2=0x7fd124061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:05.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.883+0000 7fd14259c700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd12405f390 0x7fd124061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:05.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:05.891+0000 7fd1327fc700 1 -- 172.21.15.33:0/1894609750 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd13c04d3d0 con 0x7fd14414e9e0 2023-11-06T21:03:05.944 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:05.944+0000 7fc95cf60680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:03:06.035 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:06.035+0000 7fc95cf60680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:03:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.071+0000 7fd148fff700 1 -- 172.21.15.33:0/1894609750 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd144062ac0 con 0x7fd14414e9e0 2023-11-06T21:03:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.072+0000 7fd1327fc700 1 -- 172.21.15.33:0/1894609750 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd13c0509f0 con 0x7fd14414e9e0 2023-11-06T21:03:06.074 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T21:03:06.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 -- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd12405f390 msgr2=0x7fd124061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:06.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd12405f390 0x7fd124061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:06.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 -- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 msgr2=0x7fd1440b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd1440b0540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fd13c014000 tx=0x7fd13c009780).stop 2023-11-06T21:03:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 -- 172.21.15.33:0/1894609750 shutdown_connections 2023-11-06T21:03:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd12405f390 0x7fd124061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd14414e9e0 0x7fd1440b0540 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1440632f0 0x7fd1441642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 --2- 172.21.15.33:0/1894609750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1440636d0 0x7fd1440c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 -- 172.21.15.33:0/1894609750 >> 172.21.15.33:0/1894609750 conn(0x7fd1440cb910 msgr2=0x7fd1440c6900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.074+0000 7fd148fff700 1 -- 172.21.15.33:0/1894609750 shutdown_connections 2023-11-06T21:03:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.075+0000 7fd148fff700 1 -- 172.21.15.33:0/1894609750 wait complete. 2023-11-06T21:03:06.093 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:03:06.112 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:06.112+0000 7f63222b3680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:03:06.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.296+0000 7efc1a1ff700 1 Processor -- start 2023-11-06T21:03:06.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.296+0000 7efc1a1ff700 1 -- start start 2023-11-06T21:03:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.297+0000 7efc1a1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc140c1840 0x7efc140a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.297+0000 7efc1a1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140aa2d0 0x7efc140c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.297+0000 7efc1a1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc141554a0 con 0x7efc140c6240 2023-11-06T21:03:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.297+0000 7efc1a1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc14154960 con 0x7efc140c1840 2023-11-06T21:03:06.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.297+0000 7efc1a1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc14155930 con 0x7efc140aa2d0 2023-11-06T21:03:06.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.298+0000 7efc137fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc140c1840 0x7efc140a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:06.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.298+0000 7efc12ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140aa2d0 0x7efc140c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:06.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.298+0000 7efc12ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140aa2d0 0x7efc140c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:34038/0 (socket says 172.21.15.33:34038) 2023-11-06T21:03:06.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.298+0000 7efc12ffd700 1 -- 172.21.15.33:0/710529523 learned_addr learned my addr 172.21.15.33:0/710529523 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:06.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.298+0000 7efc13fff700 1 --1- 172.21.15.33:0/710529523 >> v1:172.21.15.33:6789/0 conn(0x7efc140c6240 0x7efc14157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44486/0 (socket says 172.21.15.33:44486) 2023-11-06T21:03:06.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.298+0000 7efc12ffd700 1 -- 172.21.15.33:0/710529523 >> v2:172.21.15.33:3300/0 conn(0x7efc140c1840 msgr2=0x7efc140a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:06.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.298+0000 7efc12ffd700 1 --2- 172.21.15.33:0/710529523 >> v2:172.21.15.33:3300/0 conn(0x7efc140c1840 0x7efc140a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.298+0000 7efc12ffd700 1 -- 172.21.15.33:0/710529523 >> v1:172.21.15.33:6789/0 conn(0x7efc140c6240 legacy=0x7efc14157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.298+0000 7efc12ffd700 1 -- 172.21.15.33:0/710529523 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc140c66b0 con 0x7efc140aa2d0 2023-11-06T21:03:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.298+0000 7efc12ffd700 1 --2- 172.21.15.33:0/710529523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140aa2d0 0x7efc140c5d50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7efc08006fd0 tx=0x7efc080090f0).ready entity=mon.2 client_cookie=28b7fcc7eb8a16e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.299+0000 7efc127fc700 1 -- 172.21.15.33:0/710529523 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0800b580 con 0x7efc140aa2d0 2023-11-06T21:03:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.299+0000 7efc127fc700 1 -- 172.21.15.33:0/710529523 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0800b6e0 con 0x7efc140aa2d0 2023-11-06T21:03:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.299+0000 7efc127fc700 1 -- 172.21.15.33:0/710529523 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0800ba60 con 0x7efc140aa2d0 2023-11-06T21:03:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.299+0000 7efc1a1ff700 1 -- 172.21.15.33:0/710529523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140aa2d0 msgr2=0x7efc140c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.299+0000 7efc1a1ff700 1 --2- 172.21.15.33:0/710529523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140aa2d0 0x7efc140c5d50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7efc08006fd0 tx=0x7efc080090f0).stop 2023-11-06T21:03:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.299+0000 7efc1a1ff700 1 -- 172.21.15.33:0/710529523 shutdown_connections 2023-11-06T21:03:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.299+0000 7efc1a1ff700 1 --2- 172.21.15.33:0/710529523 >> v2:172.21.15.33:3300/0 conn(0x7efc140c1840 0x7efc140a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.299+0000 7efc1a1ff700 1 --2- 172.21.15.33:0/710529523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140aa2d0 0x7efc140c5d50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.299+0000 7efc1a1ff700 1 -- 172.21.15.33:0/710529523 >> 172.21.15.33:0/710529523 conn(0x7efc1405ec80 msgr2=0x7efc140cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.300+0000 7efc1a1ff700 1 -- 172.21.15.33:0/710529523 shutdown_connections 2023-11-06T21:03:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.300+0000 7efc1a1ff700 1 -- 172.21.15.33:0/710529523 wait complete. 2023-11-06T21:03:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.302+0000 7efc1a1ff700 1 Processor -- start 2023-11-06T21:03:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.302+0000 7efc1a1ff700 1 -- start start 2023-11-06T21:03:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc1a1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc140aa2d0 0x7efc140c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc1a1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140c1840 0x7efc141408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc1a1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc140c6240 0x7efc14152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc1a1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140c3780 con 0x7efc140aa2d0 2023-11-06T21:03:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc1a1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140c4520 con 0x7efc140c6240 2023-11-06T21:03:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc1a1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc140c3bf0 con 0x7efc140c1840 2023-11-06T21:03:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc140aa2d0 0x7efc140c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc140aa2d0 0x7efc140c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33266/0 (socket says 172.21.15.33:33266) 2023-11-06T21:03:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc137fe700 1 -- 172.21.15.33:0/714315523 learned_addr learned my addr 172.21.15.33:0/714315523 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc12ffd700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140c1840 0x7efc141408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc13fff700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc140c6240 0x7efc14152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc137fe700 1 -- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140c1840 msgr2=0x7efc141408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc137fe700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140c1840 0x7efc141408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc137fe700 1 -- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc140c6240 msgr2=0x7efc14152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc137fe700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc140c6240 0x7efc14152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.303+0000 7efc137fe700 1 -- 172.21.15.33:0/714315523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc08009370 con 0x7efc140aa2d0 2023-11-06T21:03:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.304+0000 7efc137fe700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc140aa2d0 0x7efc140c49b0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7efc0400e930 tx=0x7efc04009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.304+0000 7efbfb7fe700 1 -- 172.21.15.33:0/714315523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0400c680 con 0x7efc140aa2d0 2023-11-06T21:03:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.304+0000 7efbfb7fe700 1 -- 172.21.15.33:0/714315523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0400c7e0 con 0x7efc140aa2d0 2023-11-06T21:03:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.304+0000 7efc1a1ff700 1 -- 172.21.15.33:0/714315523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc140c3ed0 con 0x7efc140aa2d0 2023-11-06T21:03:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.304+0000 7efbfb7fe700 1 -- 172.21.15.33:0/714315523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0400cb60 con 0x7efc140aa2d0 2023-11-06T21:03:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.304+0000 7efc1a1ff700 1 -- 172.21.15.33:0/714315523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc14140da0 con 0x7efc140aa2d0 2023-11-06T21:03:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.305+0000 7efbf97fa700 1 -- 172.21.15.33:0/714315523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc140c4800 con 0x7efc140aa2d0 2023-11-06T21:03:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.306+0000 7efbfb7fe700 1 -- 172.21.15.33:0/714315523 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7efc0401b400 con 0x7efc140aa2d0 2023-11-06T21:03:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.307+0000 7efbfb7fe700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0005f3c0 0x7efc00061880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.307+0000 7efbfb7fe700 1 -- 172.21.15.33:0/714315523 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6435+0+0 (secure 0 0 0) 0x7efc0407ac30 con 0x7efc140aa2d0 2023-11-06T21:03:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.307+0000 7efc12ffd700 1 -- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0005f3c0 msgr2=0x7efc00061880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.307+0000 7efc12ffd700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0005f3c0 0x7efc00061880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.311+0000 7efbfb7fe700 1 -- 172.21.15.33:0/714315523 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efc0404bb70 con 0x7efc140aa2d0 2023-11-06T21:03:06.342 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:06.342+0000 7f63222b3680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:03:06.381 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:06.381+0000 7fc95cf60680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:03:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.508+0000 7efc12ffd700 1 -- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0005f3c0 msgr2=0x7efc00061880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:06.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.508+0000 7efc12ffd700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0005f3c0 0x7efc00061880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:03:06.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.539+0000 7efbf97fa700 1 -- 172.21.15.33:0/714315523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc14140ff0 con 0x7efc140aa2d0 2023-11-06T21:03:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.539+0000 7efbfb7fe700 1 -- 172.21.15.33:0/714315523 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4491 (secure 0 0 0) 0x7efc0404f190 con 0x7efc140aa2d0 2023-11-06T21:03:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:06.540 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":53,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2500558031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2500558031},{"type":"v1","addr":"172.21.15.69:6837","nonce":2500558031}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1012474146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1012474146},{"type":"v1","addr":"172.21.15.69:6836","nonce":1012474146}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4020289695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4020289695},{"type":"v1","addr":"172.21.15.33:6834","nonce":4020289695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:02:11.991421+0000","modified":"2023-11-06T21:02:15.032423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1130278640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1130278640},{"type":"v1","addr":"172.21.15.33:6835","nonce":1130278640}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-06T21:03:06.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 -- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0005f3c0 msgr2=0x7efc00061880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:06.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0005f3c0 0x7efc00061880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:06.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 -- 172.21.15.33:0/714315523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc140aa2d0 msgr2=0x7efc140c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc140aa2d0 0x7efc140c49b0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7efc0400e930 tx=0x7efc04009060).stop 2023-11-06T21:03:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 -- 172.21.15.33:0/714315523 shutdown_connections 2023-11-06T21:03:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0005f3c0 0x7efc00061880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc140c6240 0x7efc14152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc140aa2d0 0x7efc140c49b0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 --2- 172.21.15.33:0/714315523 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc140c1840 0x7efc141408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 -- 172.21.15.33:0/714315523 >> 172.21.15.33:0/714315523 conn(0x7efc1405ec80 msgr2=0x7efc1414f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:06.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 -- 172.21.15.33:0/714315523 shutdown_connections 2023-11-06T21:03:06.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.542+0000 7efbf97fa700 1 -- 172.21.15.33:0/714315523 wait complete. 2023-11-06T21:03:06.546 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 53 2023-11-06T21:03:06.562 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:03:06.615 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:06.615+0000 7fc95cf60680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:03:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.763+0000 7f588a7ff700 1 Processor -- start 2023-11-06T21:03:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.763+0000 7f588a7ff700 1 -- start start 2023-11-06T21:03:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.764+0000 7f588a7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5884151dc0 0x7f58840c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.764+0000 7f588a7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f58841521a0 0x7f5884063110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f588a7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5884154e90 con 0x7f5884155320 2023-11-06T21:03:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f588a7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58840639d0 con 0x7f58841521a0 2023-11-06T21:03:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f588a7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5884063cb0 con 0x7f5884151dc0 2023-11-06T21:03:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f5883fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5884151dc0 0x7f58840c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f58837fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f58841521a0 0x7f5884063110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f5883fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5884151dc0 0x7f58840c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:34054/0 (socket says 172.21.15.33:34054) 2023-11-06T21:03:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f5883fff700 1 -- 172.21.15.33:0/3269206455 learned_addr learned my addr 172.21.15.33:0/3269206455 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f5883fff700 1 -- 172.21.15.33:0/3269206455 >> v2:172.21.15.69:3300/0 conn(0x7f58841521a0 msgr2=0x7f5884063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f5883fff700 1 --2- 172.21.15.33:0/3269206455 >> v2:172.21.15.69:3300/0 conn(0x7f58841521a0 0x7f5884063110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f5883fff700 1 -- 172.21.15.33:0/3269206455 >> v1:172.21.15.69:6789/0 conn(0x7f5884155320 legacy=0x7f58840c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.765+0000 7f5883fff700 1 -- 172.21.15.33:0/3269206455 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884063f90 con 0x7f5884151dc0 2023-11-06T21:03:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.766+0000 7f5883fff700 1 --2- 172.21.15.33:0/3269206455 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5884151dc0 0x7f58840c0c50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f5874006fd0 tx=0x7f58740090f0).ready entity=mon.2 client_cookie=e74cfa56c05e5fe5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.766+0000 7f5882ffd700 1 -- 172.21.15.33:0/3269206455 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587400b580 con 0x7f5884151dc0 2023-11-06T21:03:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.766+0000 7f5882ffd700 1 -- 172.21.15.33:0/3269206455 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587400b6e0 con 0x7f5884151dc0 2023-11-06T21:03:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.766+0000 7f5882ffd700 1 -- 172.21.15.33:0/3269206455 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587400ba60 con 0x7f5884151dc0 2023-11-06T21:03:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.766+0000 7f588a7ff700 1 -- 172.21.15.33:0/3269206455 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5884151dc0 msgr2=0x7f58840c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.766+0000 7f588a7ff700 1 --2- 172.21.15.33:0/3269206455 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5884151dc0 0x7f58840c0c50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f5874006fd0 tx=0x7f58740090f0).stop 2023-11-06T21:03:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.766+0000 7f588a7ff700 1 -- 172.21.15.33:0/3269206455 shutdown_connections 2023-11-06T21:03:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.766+0000 7f588a7ff700 1 --2- 172.21.15.33:0/3269206455 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5884151dc0 0x7f58840c0c50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.766+0000 7f588a7ff700 1 --2- 172.21.15.33:0/3269206455 >> v2:172.21.15.69:3300/0 conn(0x7f58841521a0 0x7f5884063110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.766+0000 7f588a7ff700 1 -- 172.21.15.33:0/3269206455 >> 172.21.15.33:0/3269206455 conn(0x7f588405e1b0 msgr2=0x7f588405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.767+0000 7f588a7ff700 1 -- 172.21.15.33:0/3269206455 shutdown_connections 2023-11-06T21:03:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.767+0000 7f588a7ff700 1 -- 172.21.15.33:0/3269206455 wait complete. 2023-11-06T21:03:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.768+0000 7f588a7ff700 1 Processor -- start 2023-11-06T21:03:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.768+0000 7f588a7ff700 1 -- start start 2023-11-06T21:03:06.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.769+0000 7f588a7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5884151dc0 0x7f588414cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.769+0000 7f588a7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58841521a0 0x7f58840addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.769+0000 7f588a7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5884155320 0x7f58840ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.769+0000 7f588a7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5884164d80 con 0x7f5884155320 2023-11-06T21:03:06.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.769+0000 7f588a7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840af6a0 con 0x7f5884151dc0 2023-11-06T21:03:06.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.769+0000 7f588a7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58840ae900 con 0x7f58841521a0 2023-11-06T21:03:06.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.769+0000 7f58837fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58841521a0 0x7f58840addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:06.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.769+0000 7f5883fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5884151dc0 0x7f588414cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:06.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.769+0000 7f58837fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58841521a0 0x7f58840addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:34056/0 (socket says 172.21.15.33:34056) 2023-11-06T21:03:06.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.769+0000 7f58837fe700 1 -- 172.21.15.33:0/77535571 learned_addr learned my addr 172.21.15.33:0/77535571 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:06.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.770+0000 7f5888d9c700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5884155320 0x7f58840ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:06.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.770+0000 7f58837fe700 1 -- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5884151dc0 msgr2=0x7f588414cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.770+0000 7f58837fe700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5884151dc0 0x7f588414cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.770+0000 7f58837fe700 1 -- 172.21.15.33:0/77535571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5884155320 msgr2=0x7f58840ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.770+0000 7f58837fe700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5884155320 0x7f58840ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.770+0000 7f58837fe700 1 -- 172.21.15.33:0/77535571 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5874009370 con 0x7f58841521a0 2023-11-06T21:03:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.770+0000 7f58837fe700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58841521a0 0x7f58840addb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f587800e930 tx=0x7f5878009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.770+0000 7f586bfff700 1 -- 172.21.15.33:0/77535571 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878009510 con 0x7f58841521a0 2023-11-06T21:03:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.771+0000 7f586bfff700 1 -- 172.21.15.33:0/77535571 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5878012600 con 0x7f58841521a0 2023-11-06T21:03:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.771+0000 7f586bfff700 1 -- 172.21.15.33:0/77535571 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878012980 con 0x7f58841521a0 2023-11-06T21:03:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.771+0000 7f588a7ff700 1 -- 172.21.15.33:0/77535571 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58840aebe0 con 0x7f58841521a0 2023-11-06T21:03:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.771+0000 7f588a7ff700 1 -- 172.21.15.33:0/77535571 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58840c4d90 con 0x7f58841521a0 2023-11-06T21:03:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.772+0000 7f588a7ff700 1 -- 172.21.15.33:0/77535571 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58840aee20 con 0x7f58841521a0 2023-11-06T21:03:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.773+0000 7f586bfff700 1 -- 172.21.15.33:0/77535571 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f587801b400 con 0x7f58841521a0 2023-11-06T21:03:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.773+0000 7f586bfff700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f587005f390 0x7f5870061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.773+0000 7f586bfff700 1 -- 172.21.15.33:0/77535571 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6435+0+0 (secure 0 0 0) 0x7f587807ab00 con 0x7f58841521a0 2023-11-06T21:03:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.774+0000 7f5883fff700 1 -- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f587005f390 msgr2=0x7f5870061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.774+0000 7f5883fff700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f587005f390 0x7f5870061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.777+0000 7f586bfff700 1 -- 172.21.15.33:0/77535571 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f58840aee20 con 0x7f58841521a0 2023-11-06T21:03:06.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.962+0000 7f588a7ff700 1 -- 172.21.15.33:0/77535571 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58840aee20 con 0x7f58841521a0 2023-11-06T21:03:06.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.963+0000 7f586bfff700 1 -- 172.21.15.33:0/77535571 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17470 (secure 0 0 0) 0x7f58840aee20 con 0x7f58841521a0 2023-11-06T21:03:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:06.965 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":133,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:03:06.102987+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:02:09.533597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T21:02:10.978394+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6832/114912":"2023-11-07T21:02:26.250703+0000","172.21.15.69:6833/114912":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/3344790742":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/874235539":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/2223677285":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/3213553706":"2023-11-07T21:02:26.250703+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:03:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 -- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f587005f390 msgr2=0x7f5870061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f587005f390 0x7f5870061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 -- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58841521a0 msgr2=0x7f58840addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58841521a0 0x7f58840addb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f587800e930 tx=0x7f5878009060).stop 2023-11-06T21:03:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 -- 172.21.15.33:0/77535571 shutdown_connections 2023-11-06T21:03:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f587005f390 0x7f5870061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5884155320 0x7f58840ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5884151dc0 0x7f588414cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 --2- 172.21.15.33:0/77535571 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58841521a0 0x7f58840addb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 -- 172.21.15.33:0/77535571 >> 172.21.15.33:0/77535571 conn(0x7f588405e1b0 msgr2=0x7f58840ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:06.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 -- 172.21.15.33:0/77535571 shutdown_connections 2023-11-06T21:03:06.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:06.966+0000 7f588a7ff700 1 -- 172.21.15.33:0/77535571 wait complete. 2023-11-06T21:03:06.985 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T21:03:06.985 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:03:07.145 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:07.145+0000 7f63222b3680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:03:07.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.186+0000 7f79e33ff700 1 Processor -- start 2023-11-06T21:03:07.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.186+0000 7f79e33ff700 1 -- start start 2023-11-06T21:03:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.187+0000 7f79e33ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f79dc137660 0x7f79dc0c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.187+0000 7f79e33ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc069410 0x7f79dc069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.187+0000 7f79e33ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79dc0c17e0 con 0x7f79dc0c3600 2023-11-06T21:03:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.187+0000 7f79e33ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc06a740 con 0x7f79dc137660 2023-11-06T21:03:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.187+0000 7f79e33ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc06a180 con 0x7f79dc069410 2023-11-06T21:03:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79e119b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f79dc0c3600 0x7f79dc136f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44498/0 (socket says 172.21.15.33:44498) 2023-11-06T21:03:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79e119b700 1 -- 172.21.15.33:0/1386585423 learned_addr learned my addr 172.21.15.33:0/1386585423 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79e199c700 1 --2- 172.21.15.33:0/1386585423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc069410 0x7f79dc069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79e099a700 1 --2- 172.21.15.33:0/1386585423 >> v2:172.21.15.69:3300/0 conn(0x7f79dc137660 0x7f79dc0c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:07.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79d3fff700 1 -- 172.21.15.33:0/1386585423 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693152763 0 0) 0x7f79dc0c17e0 con 0x7f79dc0c3600 2023-11-06T21:03:07.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79d3fff700 1 -- 172.21.15.33:0/1386585423 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c80033f0 con 0x7f79dc0c3600 2023-11-06T21:03:07.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79e199c700 1 -- 172.21.15.33:0/1386585423 >> v2:172.21.15.69:3300/0 conn(0x7f79dc137660 msgr2=0x7f79dc0c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:07.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79e199c700 1 --2- 172.21.15.33:0/1386585423 >> v2:172.21.15.69:3300/0 conn(0x7f79dc137660 0x7f79dc0c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79e199c700 1 -- 172.21.15.33:0/1386585423 >> v1:172.21.15.33:6789/0 conn(0x7f79dc0c3600 legacy=0x7f79dc136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79e199c700 1 -- 172.21.15.33:0/1386585423 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc06a460 con 0x7f79dc069410 2023-11-06T21:03:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.188+0000 7f79e099a700 1 --2- 172.21.15.33:0/1386585423 >> v2:172.21.15.69:3300/0 conn(0x7f79dc137660 0x7f79dc0c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.189+0000 7f79e199c700 1 --2- 172.21.15.33:0/1386585423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc069410 0x7f79dc069880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f79d4009e60 tx=0x7f79d4009780).ready entity=mon.2 client_cookie=a0ac68b22b6c0b74 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.189+0000 7f79d3fff700 1 -- 172.21.15.33:0/1386585423 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400e070 con 0x7f79dc069410 2023-11-06T21:03:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.189+0000 7f79d3fff700 1 -- 172.21.15.33:0/1386585423 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d400bab0 con 0x7f79dc069410 2023-11-06T21:03:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.189+0000 7f79d3fff700 1 -- 172.21.15.33:0/1386585423 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400be30 con 0x7f79dc069410 2023-11-06T21:03:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.189+0000 7f79e33ff700 1 -- 172.21.15.33:0/1386585423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc069410 msgr2=0x7f79dc069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.189+0000 7f79e33ff700 1 --2- 172.21.15.33:0/1386585423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc069410 0x7f79dc069880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f79d4009e60 tx=0x7f79d4009780).stop 2023-11-06T21:03:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.189+0000 7f79e33ff700 1 -- 172.21.15.33:0/1386585423 shutdown_connections 2023-11-06T21:03:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.189+0000 7f79e33ff700 1 --2- 172.21.15.33:0/1386585423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc069410 0x7f79dc069880 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.189+0000 7f79e33ff700 1 --2- 172.21.15.33:0/1386585423 >> v2:172.21.15.69:3300/0 conn(0x7f79dc137660 0x7f79dc0c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.189+0000 7f79e33ff700 1 -- 172.21.15.33:0/1386585423 >> 172.21.15.33:0/1386585423 conn(0x7f79dc062710 msgr2=0x7f79dc062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.190+0000 7f79e33ff700 1 -- 172.21.15.33:0/1386585423 shutdown_connections 2023-11-06T21:03:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.190+0000 7f79e33ff700 1 -- 172.21.15.33:0/1386585423 wait complete. 2023-11-06T21:03:07.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.191+0000 7f79e33ff700 1 Processor -- start 2023-11-06T21:03:07.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.191+0000 7f79e33ff700 1 -- start start 2023-11-06T21:03:07.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.192+0000 7f79e33ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79dc069410 0x7f79dc0aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.192+0000 7f79e33ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc0c3600 0x7f79dc136240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.192+0000 7f79e33ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc137660 0x7f79dc155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.192+0000 7f79e33ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0be6c0 con 0x7f79dc137660 2023-11-06T21:03:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.192+0000 7f79e33ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0abdb0 con 0x7f79dc069410 2023-11-06T21:03:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.192+0000 7f79e33ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0ab290 con 0x7f79dc0c3600 2023-11-06T21:03:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc137660 0x7f79dc155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e099a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc0c3600 0x7f79dc136240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e119b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79dc069410 0x7f79dc0aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc137660 0x7f79dc155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33292/0 (socket says 172.21.15.33:33292) 2023-11-06T21:03:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e199c700 1 -- 172.21.15.33:0/1639001404 learned_addr learned my addr 172.21.15.33:0/1639001404 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e099a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc0c3600 0x7f79dc136240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:34076/0 (socket says 172.21.15.33:34076) 2023-11-06T21:03:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e199c700 1 -- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc0c3600 msgr2=0x7f79dc136240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e199c700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc0c3600 0x7f79dc136240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e199c700 1 -- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79dc069410 msgr2=0x7f79dc0aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:07.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e199c700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79dc069410 0x7f79dc0aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e199c700 1 -- 172.21.15.33:0/1639001404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc0092a0 con 0x7f79dc137660 2023-11-06T21:03:07.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.193+0000 7f79e199c700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc137660 0x7f79dc155bb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f79d400b610 tx=0x7f79d4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:07.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.194+0000 7f79d17fa700 1 -- 172.21.15.33:0/1639001404 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4013e70 con 0x7f79dc137660 2023-11-06T21:03:07.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.194+0000 7f79d17fa700 1 -- 172.21.15.33:0/1639001404 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d401d630 con 0x7f79dc137660 2023-11-06T21:03:07.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.194+0000 7f79d17fa700 1 -- 172.21.15.33:0/1639001404 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d401d9f0 con 0x7f79dc137660 2023-11-06T21:03:07.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.194+0000 7f79e33ff700 1 -- 172.21.15.33:0/1639001404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d4009a20 con 0x7f79dc137660 2023-11-06T21:03:07.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.194+0000 7f79e33ff700 1 -- 172.21.15.33:0/1639001404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc0abad0 con 0x7f79dc137660 2023-11-06T21:03:07.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.195+0000 7f79e33ff700 1 -- 172.21.15.33:0/1639001404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc0c1470 con 0x7f79dc137660 2023-11-06T21:03:07.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.197+0000 7f79d17fa700 1 -- 172.21.15.33:0/1639001404 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f79d4003040 con 0x7f79dc137660 2023-11-06T21:03:07.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.198+0000 7f79d17fa700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f79c005f390 0x7f79c0061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.198+0000 7f79d17fa700 1 -- 172.21.15.33:0/1639001404 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6435+0+0 (secure 0 0 0) 0x7f79d407c560 con 0x7f79dc137660 2023-11-06T21:03:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.198+0000 7f79e119b700 1 -- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f79c005f390 msgr2=0x7f79c0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.198+0000 7f79e119b700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f79c005f390 0x7f79c0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.202+0000 7f79d17fa700 1 -- 172.21.15.33:0/1639001404 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f79d404d2e0 con 0x7f79dc137660 2023-11-06T21:03:07.234 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:07.234+0000 7f63222b3680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:03:07.327 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:07.327+0000 7f63222b3680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:03:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.399+0000 7f79e119b700 1 -- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f79c005f390 msgr2=0x7f79c0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:07.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.399+0000 7f79e119b700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f79c005f390 0x7f79c0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:03:07.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.428+0000 7f79e33ff700 1 -- 172.21.15.33:0/1639001404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79dc065750 con 0x7f79dc137660 2023-11-06T21:03:07.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.429+0000 7f79d17fa700 1 -- 172.21.15.33:0/1639001404 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f79d4050900 con 0x7f79dc137660 2023-11-06T21:03:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:07.430 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":53,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2500558031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2500558031},{"type":"v1","addr":"172.21.15.69:6837","nonce":2500558031}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1012474146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1012474146},{"type":"v1","addr":"172.21.15.69:6836","nonce":1012474146}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4020289695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4020289695},{"type":"v1","addr":"172.21.15.33:6834","nonce":4020289695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:02:11.991421+0000","modified":"2023-11-06T21:02:15.032423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1130278640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1130278640},{"type":"v1","addr":"172.21.15.33:6835","nonce":1130278640}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-06T21:03:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 -- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f79c005f390 msgr2=0x7f79c0061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f79c005f390 0x7f79c0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 -- 172.21.15.33:0/1639001404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc137660 msgr2=0x7f79dc155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc137660 0x7f79dc155bb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f79d400b610 tx=0x7f79d4009780).stop 2023-11-06T21:03:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 -- 172.21.15.33:0/1639001404 shutdown_connections 2023-11-06T21:03:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f79c005f390 0x7f79c0061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc137660 0x7f79dc155bb0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79dc069410 0x7f79dc0aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 --2- 172.21.15.33:0/1639001404 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79dc0c3600 0x7f79dc136240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 -- 172.21.15.33:0/1639001404 >> 172.21.15.33:0/1639001404 conn(0x7f79dc062710 msgr2=0x7f79dc0bd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:07.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 -- 172.21.15.33:0/1639001404 shutdown_connections 2023-11-06T21:03:07.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.431+0000 7f79e33ff700 1 -- 172.21.15.33:0/1639001404 wait complete. 2023-11-06T21:03:07.434 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 53 2023-11-06T21:03:07.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:03:07.492 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:07.491+0000 7fc95cf60680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:03:07.511 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:07.511+0000 7f63222b3680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:03:07.592 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:07.591+0000 7fc95cf60680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:03:07.604 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:07.604+0000 7f63222b3680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:03:07.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.658+0000 7fc8a39ff700 1 Processor -- start 2023-11-06T21:03:07.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.658+0000 7fc8a39ff700 1 -- start start 2023-11-06T21:03:07.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.659+0000 7fc8a39ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc89c1521a0 0x7fc89c0c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.659+0000 7fc8a39ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 0x7fc89c0c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.659+0000 7fc8a39ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c154e90 con 0x7fc89c151dc0 2023-11-06T21:03:07.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.659+0000 7fc8a39ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c0639d0 con 0x7fc89c1521a0 2023-11-06T21:03:07.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.659+0000 7fc8a39ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c063cb0 con 0x7fc89c155320 2023-11-06T21:03:07.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.659+0000 7fc8a179b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc89c151dc0 0x7fc89c063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44500/0 (socket says 172.21.15.33:44500) 2023-11-06T21:03:07.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.659+0000 7fc8a179b700 1 -- 172.21.15.33:0/932292238 learned_addr learned my addr 172.21.15.33:0/932292238 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:07.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.660+0000 7fc8a0f9a700 1 --2- 172.21.15.33:0/932292238 >> v2:172.21.15.69:3300/0 conn(0x7fc89c1521a0 0x7fc89c0c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:07.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.660+0000 7fc8a1f9c700 1 --2- 172.21.15.33:0/932292238 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 0x7fc89c0c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:07.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.660+0000 7fc893fff700 1 -- 172.21.15.33:0/932292238 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709121759 0 0) 0x7fc89c154e90 con 0x7fc89c151dc0 2023-11-06T21:03:07.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.660+0000 7fc893fff700 1 -- 172.21.15.33:0/932292238 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8880033f0 con 0x7fc89c151dc0 2023-11-06T21:03:07.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.660+0000 7fc8a0f9a700 1 -- 172.21.15.33:0/932292238 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 msgr2=0x7fc89c0c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:07.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.660+0000 7fc8a0f9a700 1 --2- 172.21.15.33:0/932292238 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 0x7fc89c0c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.660+0000 7fc8a0f9a700 1 -- 172.21.15.33:0/932292238 >> v1:172.21.15.33:6789/0 conn(0x7fc89c151dc0 legacy=0x7fc89c063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:07.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.660+0000 7fc8a0f9a700 1 -- 172.21.15.33:0/932292238 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c063f90 con 0x7fc89c1521a0 2023-11-06T21:03:07.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.660+0000 7fc8a1f9c700 1 --2- 172.21.15.33:0/932292238 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 0x7fc89c0c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:07.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.660+0000 7fc8a0f9a700 1 --2- 172.21.15.33:0/932292238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc89c1521a0 0x7fc89c0c0c50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fc88c009760 tx=0x7fc88c009060).ready entity=mon.1 client_cookie=c417d3a96036037b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:07.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.661+0000 7fc893fff700 1 -- 172.21.15.33:0/932292238 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c00d4a0 con 0x7fc89c1521a0 2023-11-06T21:03:07.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.661+0000 7fc893fff700 1 -- 172.21.15.33:0/932292238 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c00d600 con 0x7fc89c1521a0 2023-11-06T21:03:07.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.661+0000 7fc893fff700 1 -- 172.21.15.33:0/932292238 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c00d980 con 0x7fc89c1521a0 2023-11-06T21:03:07.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.661+0000 7fc8a39ff700 1 -- 172.21.15.33:0/932292238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc89c1521a0 msgr2=0x7fc89c0c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:07.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.661+0000 7fc8a39ff700 1 --2- 172.21.15.33:0/932292238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc89c1521a0 0x7fc89c0c0c50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fc88c009760 tx=0x7fc88c009060).stop 2023-11-06T21:03:07.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.661+0000 7fc8a39ff700 1 -- 172.21.15.33:0/932292238 shutdown_connections 2023-11-06T21:03:07.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.661+0000 7fc8a39ff700 1 --2- 172.21.15.33:0/932292238 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 0x7fc89c0c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.661+0000 7fc8a39ff700 1 --2- 172.21.15.33:0/932292238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc89c1521a0 0x7fc89c0c0c50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.661+0000 7fc8a39ff700 1 -- 172.21.15.33:0/932292238 >> 172.21.15.33:0/932292238 conn(0x7fc89c05e1b0 msgr2=0x7fc89c05e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:07.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.662+0000 7fc8a39ff700 1 -- 172.21.15.33:0/932292238 shutdown_connections 2023-11-06T21:03:07.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.662+0000 7fc8a39ff700 1 -- 172.21.15.33:0/932292238 wait complete. 2023-11-06T21:03:07.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.663+0000 7fc8a39ff700 1 Processor -- start 2023-11-06T21:03:07.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.663+0000 7fc8a39ff700 1 -- start start 2023-11-06T21:03:07.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.664+0000 7fc8a39ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89c151dc0 0x7fc89c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.664+0000 7fc8a39ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc89c1521a0 0x7fc89c150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.664+0000 7fc8a39ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 0x7fc89c0c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.664+0000 7fc8a39ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0b0ae0 con 0x7fc89c151dc0 2023-11-06T21:03:07.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.664+0000 7fc8a39ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c151240 con 0x7fc89c1521a0 2023-11-06T21:03:07.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.664+0000 7fc8a39ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0afef0 con 0x7fc89c155320 2023-11-06T21:03:07.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.664+0000 7fc8a179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89c151dc0 0x7fc89c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:07.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89c151dc0 0x7fc89c150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33300/0 (socket says 172.21.15.33:33300) 2023-11-06T21:03:07.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a0f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc89c1521a0 0x7fc89c150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:07.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a179b700 1 -- 172.21.15.33:0/3403077276 learned_addr learned my addr 172.21.15.33:0/3403077276 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:07.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a1f9c700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 0x7fc89c0c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:07.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a1f9c700 1 -- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc89c1521a0 msgr2=0x7fc89c150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:07.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a1f9c700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc89c1521a0 0x7fc89c150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a1f9c700 1 -- 172.21.15.33:0/3403077276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89c151dc0 msgr2=0x7fc89c150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:07.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a1f9c700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89c151dc0 0x7fc89c150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a1f9c700 1 -- 172.21.15.33:0/3403077276 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc894009a20 con 0x7fc89c155320 2023-11-06T21:03:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a179b700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89c151dc0 0x7fc89c150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8a1f9c700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 0x7fc89c0c31e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fc894009f10 tx=0x7fc89400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.665+0000 7fc8917fa700 1 -- 172.21.15.33:0/3403077276 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400cb10 con 0x7fc89c155320 2023-11-06T21:03:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.666+0000 7fc8917fa700 1 -- 172.21.15.33:0/3403077276 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89400f040 con 0x7fc89c155320 2023-11-06T21:03:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.666+0000 7fc8917fa700 1 -- 172.21.15.33:0/3403077276 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400ce80 con 0x7fc89c155320 2023-11-06T21:03:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.666+0000 7fc8a39ff700 1 -- 172.21.15.33:0/3403077276 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c0092a0 con 0x7fc89c155320 2023-11-06T21:03:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.666+0000 7fc8a39ff700 1 -- 172.21.15.33:0/3403077276 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c0b0700 con 0x7fc89c155320 2023-11-06T21:03:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.667+0000 7fc8a39ff700 1 -- 172.21.15.33:0/3403077276 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c0b0c20 con 0x7fc89c155320 2023-11-06T21:03:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.667+0000 7fc8917fa700 1 -- 172.21.15.33:0/3403077276 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fc894013610 con 0x7fc89c155320 2023-11-06T21:03:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.669+0000 7fc8917fa700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc88005f390 0x7fc880061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.669+0000 7fc8917fa700 1 -- 172.21.15.33:0/3403077276 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6435+0+0 (secure 0 0 0) 0x7fc89407b0c0 con 0x7fc89c155320 2023-11-06T21:03:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.669+0000 7fc8a179b700 1 -- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc88005f390 msgr2=0x7fc880061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.669+0000 7fc8a179b700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc88005f390 0x7fc880061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:07.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.673+0000 7fc8917fa700 1 -- 172.21.15.33:0/3403077276 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc89c0b0c20 con 0x7fc89c155320 2023-11-06T21:03:07.695 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:07.694+0000 7fc95cf60680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:03:07.747 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:07.747+0000 7f63222b3680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:03:07.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.870+0000 7fc8a179b700 1 -- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc88005f390 msgr2=0x7fc880061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:07.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.870+0000 7fc8a179b700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc88005f390 0x7fc880061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:03:07.877 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:07.877+0000 7f63222b3680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:03:07.893 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:07.892+0000 7fc95cf60680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:03:07.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.898+0000 7fc8a39ff700 1 -- 172.21.15.33:0/3403077276 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc89c0b0c20 con 0x7fc89c155320 2023-11-06T21:03:07.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.899+0000 7fc8917fa700 1 -- 172.21.15.33:0/3403077276 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4491 (secure 0 0 0) 0x7fc89c0b0c20 con 0x7fc89c155320 2023-11-06T21:03:07.900 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:07.900 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":53,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2500558031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2500558031},{"type":"v1","addr":"172.21.15.69:6837","nonce":2500558031}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1012474146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1012474146},{"type":"v1","addr":"172.21.15.69:6836","nonce":1012474146}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4020289695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4020289695},{"type":"v1","addr":"172.21.15.33:6834","nonce":4020289695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:02:11.991421+0000","modified":"2023-11-06T21:02:15.032423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1130278640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1130278640},{"type":"v1","addr":"172.21.15.33:6835","nonce":1130278640}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-06T21:03:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 -- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc88005f390 msgr2=0x7fc880061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc88005f390 0x7fc880061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 -- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 msgr2=0x7fc89c0c31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 0x7fc89c0c31e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fc894009f10 tx=0x7fc89400b040).stop 2023-11-06T21:03:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 -- 172.21.15.33:0/3403077276 shutdown_connections 2023-11-06T21:03:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc88005f390 0x7fc880061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc89c155320 0x7fc89c0c31e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89c151dc0 0x7fc89c150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 --2- 172.21.15.33:0/3403077276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc89c1521a0 0x7fc89c150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 -- 172.21.15.33:0/3403077276 >> 172.21.15.33:0/3403077276 conn(0x7fc89c05e1b0 msgr2=0x7fc89c0c8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 -- 172.21.15.33:0/3403077276 shutdown_connections 2023-11-06T21:03:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:07.901+0000 7fc8a39ff700 1 -- 172.21.15.33:0/3403077276 wait complete. 2023-11-06T21:03:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 53 2023-11-06T21:03:07.920 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:03:07.993 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:07.992+0000 7fc95cf60680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:03:08.154 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:08.153+0000 7fc95cf60680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:03:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.166+0000 7f3b5edff700 1 Processor -- start 2023-11-06T21:03:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.166+0000 7f3b5edff700 1 -- start start 2023-11-06T21:03:08.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.167+0000 7f3b5edff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3b58154280 0x7f3b58154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:08.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.167+0000 7f3b5edff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3b580ad6e0 0x7f3b580c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.167+0000 7f3b5edff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b580c5db0 con 0x7f3b580a9f20 2023-11-06T21:03:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.167+0000 7f3b5edff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b580c6930 con 0x7f3b580ad6e0 2023-11-06T21:03:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.167+0000 7f3b5edff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b580c6000 con 0x7f3b58154280 2023-11-06T21:03:08.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.168+0000 7f3b57fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3b580ad6e0 0x7f3b580c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:08.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.168+0000 7f3b57fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3b580ad6e0 0x7f3b580c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50654/0 (socket says 172.21.15.33:50654) 2023-11-06T21:03:08.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.168+0000 7f3b57fff700 1 -- 172.21.15.33:0/522510744 learned_addr learned my addr 172.21.15.33:0/522510744 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:08.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.168+0000 7f3b5cb9b700 1 --2- 172.21.15.33:0/522510744 >> v2:172.21.15.69:3300/0 conn(0x7f3b58154280 0x7f3b58154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.168+0000 7f3b57fff700 1 -- 172.21.15.33:0/522510744 >> v2:172.21.15.69:3300/0 conn(0x7f3b58154280 msgr2=0x7f3b58154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.168+0000 7f3b57fff700 1 --2- 172.21.15.33:0/522510744 >> v2:172.21.15.69:3300/0 conn(0x7f3b58154280 0x7f3b58154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.168+0000 7f3b57fff700 1 -- 172.21.15.33:0/522510744 >> v1:172.21.15.69:6789/0 conn(0x7f3b580a9f20 legacy=0x7f3b580c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:08.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.168+0000 7f3b57fff700 1 -- 172.21.15.33:0/522510744 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b580c62e0 con 0x7f3b580ad6e0 2023-11-06T21:03:08.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.169+0000 7f3b57fff700 1 --2- 172.21.15.33:0/522510744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b580ad6e0 0x7f3b580c6dc0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f3b58065150 tx=0x7f3b440090f0).ready entity=mon.0 client_cookie=ea5e469cfd6682ec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:08.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.169+0000 7f3b577fe700 1 -- 172.21.15.33:0/522510744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4400b520 con 0x7f3b580ad6e0 2023-11-06T21:03:08.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.169+0000 7f3b577fe700 1 -- 172.21.15.33:0/522510744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4400b680 con 0x7f3b580ad6e0 2023-11-06T21:03:08.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.169+0000 7f3b577fe700 1 -- 172.21.15.33:0/522510744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4400ba00 con 0x7f3b580ad6e0 2023-11-06T21:03:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.169+0000 7f3b5edff700 1 -- 172.21.15.33:0/522510744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b580ad6e0 msgr2=0x7f3b580c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.169+0000 7f3b5edff700 1 --2- 172.21.15.33:0/522510744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b580ad6e0 0x7f3b580c6dc0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f3b58065150 tx=0x7f3b440090f0).stop 2023-11-06T21:03:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.169+0000 7f3b5edff700 1 -- 172.21.15.33:0/522510744 shutdown_connections 2023-11-06T21:03:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.169+0000 7f3b5edff700 1 --2- 172.21.15.33:0/522510744 >> v2:172.21.15.69:3300/0 conn(0x7f3b58154280 0x7f3b58154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.169+0000 7f3b5edff700 1 --2- 172.21.15.33:0/522510744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b580ad6e0 0x7f3b580c6dc0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.169+0000 7f3b5edff700 1 -- 172.21.15.33:0/522510744 >> 172.21.15.33:0/522510744 conn(0x7f3b5805e1b0 msgr2=0x7f3b5805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.170+0000 7f3b5edff700 1 -- 172.21.15.33:0/522510744 shutdown_connections 2023-11-06T21:03:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.170+0000 7f3b5edff700 1 -- 172.21.15.33:0/522510744 wait complete. 2023-11-06T21:03:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.172+0000 7f3b5edff700 1 Processor -- start 2023-11-06T21:03:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.172+0000 7f3b5edff700 1 -- start start 2023-11-06T21:03:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.173+0000 7f3b5edff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b580a9f20 0x7f3b58149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.173+0000 7f3b5edff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b580ad6e0 0x7f3b58164760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.173+0000 7f3b5edff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3b58154280 0x7f3b580a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.173+0000 7f3b5edff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580c2ed0 con 0x7f3b580ad6e0 2023-11-06T21:03:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.173+0000 7f3b5edff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580c3cc0 con 0x7f3b580a9f20 2023-11-06T21:03:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.173+0000 7f3b5edff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580c3120 con 0x7f3b58154280 2023-11-06T21:03:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.173+0000 7f3b5cb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b580a9f20 0x7f3b58149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.173+0000 7f3b57fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b580ad6e0 0x7f3b58164760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.173+0000 7f3b5cb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b580a9f20 0x7f3b58149b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:48150/0 (socket says 172.21.15.33:48150) 2023-11-06T21:03:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.173+0000 7f3b5cb9b700 1 -- 172.21.15.33:0/301326008 learned_addr learned my addr 172.21.15.33:0/301326008 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b5d39c700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3b58154280 0x7f3b580a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b57fff700 1 -- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3b58154280 msgr2=0x7f3b580a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b57fff700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3b58154280 0x7f3b580a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b57fff700 1 -- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b580a9f20 msgr2=0x7f3b58149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b57fff700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b580a9f20 0x7f3b58149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b57fff700 1 -- 172.21.15.33:0/301326008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b44009370 con 0x7f3b580ad6e0 2023-11-06T21:03:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b5cb9b700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b580a9f20 0x7f3b58149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b57fff700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b580ad6e0 0x7f3b58164760 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f3b44014000 tx=0x7f3b44017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b54ff9700 1 -- 172.21.15.33:0/301326008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44006d90 con 0x7f3b580ad6e0 2023-11-06T21:03:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b54ff9700 1 -- 172.21.15.33:0/301326008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b44000bc0 con 0x7f3b580ad6e0 2023-11-06T21:03:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b54ff9700 1 -- 172.21.15.33:0/301326008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4400ba00 con 0x7f3b580ad6e0 2023-11-06T21:03:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.174+0000 7f3b5edff700 1 -- 172.21.15.33:0/301326008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b580c3390 con 0x7f3b580ad6e0 2023-11-06T21:03:08.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.175+0000 7f3b5edff700 1 -- 172.21.15.33:0/301326008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b580c3950 con 0x7f3b580ad6e0 2023-11-06T21:03:08.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.176+0000 7f3b5edff700 1 -- 172.21.15.33:0/301326008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b580c5100 con 0x7f3b580ad6e0 2023-11-06T21:03:08.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.176+0000 7f3b54ff9700 1 -- 172.21.15.33:0/301326008 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 52) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f3b4401c630 con 0x7f3b580ad6e0 2023-11-06T21:03:08.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.178+0000 7f3b54ff9700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3b3c05f390 0x7f3b3c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:08.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.178+0000 7f3b54ff9700 1 -- 172.21.15.33:0/301326008 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6435+0+0 (secure 0 0 0) 0x7f3b4407c370 con 0x7f3b580ad6e0 2023-11-06T21:03:08.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.179+0000 7f3b5cb9b700 1 -- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3b3c05f390 msgr2=0x7f3b3c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.179+0000 7f3b5cb9b700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3b3c05f390 0x7f3b3c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.181+0000 7f3b54ff9700 1 -- 172.21.15.33:0/301326008 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3b4404d0e0 con 0x7f3b580ad6e0 2023-11-06T21:03:08.227 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.227+0000 7f63222b3680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:03:08.309 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:08.309+0000 7fc95cf60680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:03:08.316 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.316+0000 7f63222b3680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:03:08.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.328+0000 7f3b54ff9700 1 -- 172.21.15.33:0/301326008 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 53) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f3b44079c40 con 0x7f3b580ad6e0 2023-11-06T21:03:08.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.329+0000 7f3b54ff9700 1 -- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3b3c05f390 msgr2=0x7f3b3c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:08.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.329+0000 7f3b54ff9700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3b3c05f390 0x7f3b3c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:08.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.385+0000 7f3b5edff700 1 -- 172.21.15.33:0/301326008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b580c6cc0 con 0x7f3b580ad6e0 2023-11-06T21:03:08.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.386+0000 7f3b54ff9700 1 -- 172.21.15.33:0/301326008 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17797 (secure 0 0 0) 0x7f3b4402a090 con 0x7f3b580ad6e0 2023-11-06T21:03:08.387 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:08.387 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":134,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:03:08.319899+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:02:09.533597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T21:02:10.978394+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2847685094":"2023-11-07T21:03:08.319849+0000","172.21.15.69:0/3427712961":"2023-11-07T21:03:08.319849+0000","172.21.15.69:0/3890222999":"2023-11-07T21:03:08.319849+0000","172.21.15.69:0/522077326":"2023-11-07T21:03:08.319849+0000","172.21.15.69:0/3589002288":"2023-11-07T21:03:08.319849+0000","172.21.15.69:6832/114912":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/1353409815":"2023-11-07T21:03:08.319849+0000","172.21.15.69:6833/114912":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/3344790742":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/874235539":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/2223677285":"2023-11-07T21:02:26.250703+0000","172.21.15.69:0/3213553706":"2023-11-07T21:02:26.250703+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:03:08.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.388+0000 7f3b5edff700 1 -- 172.21.15.33:0/301326008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b580ad6e0 msgr2=0x7f3b58164760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:08.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.388+0000 7f3b5edff700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b580ad6e0 0x7f3b58164760 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f3b44014000 tx=0x7f3b44017000).stop 2023-11-06T21:03:08.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.388+0000 7f3b5edff700 1 -- 172.21.15.33:0/301326008 shutdown_connections 2023-11-06T21:03:08.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.388+0000 7f3b5edff700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3b3c05f390 0x7f3b3c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:08.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.388+0000 7f3b5edff700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3b58154280 0x7f3b580a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.388+0000 7f3b5edff700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b580a9f20 0x7f3b58149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.388+0000 7f3b5edff700 1 --2- 172.21.15.33:0/301326008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b580ad6e0 0x7f3b58164760 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.388+0000 7f3b5edff700 1 -- 172.21.15.33:0/301326008 >> 172.21.15.33:0/301326008 conn(0x7f3b5805e1b0 msgr2=0x7f3b580ab7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:08.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.388+0000 7f3b5edff700 1 -- 172.21.15.33:0/301326008 shutdown_connections 2023-11-06T21:03:08.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.388+0000 7f3b5edff700 1 -- 172.21.15.33:0/301326008 wait complete. 2023-11-06T21:03:08.393 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.393+0000 7f6299d4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:08.394 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.393+0000 7f6299d4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:08.394 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.393+0000 7f6299d4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:08.394 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.393+0000 7f6299d4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:08.394 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.393+0000 7f6299d4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:08.394 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.393+0000 7f629dd53700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:08.395 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.393+0000 7f629dd53700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:08.395 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.393+0000 7f629dd53700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:08.395 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.393+0000 7f629dd53700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:08.395 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:08.393+0000 7f629dd53700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:08.409 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T21:03:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.652+0000 7fd4c2bff700 1 Processor -- start 2023-11-06T21:03:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.652+0000 7fd4c2bff700 1 -- start start 2023-11-06T21:03:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.653+0000 7fd4c2bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4bc1521f0 0x7fd4bc0bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.653+0000 7fd4c2bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4bc0bc2d0 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.653+0000 7fd4c2bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4bc0bddb0 con 0x7fd4bc0c7590 2023-11-06T21:03:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.653+0000 7fd4c2bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4bc0c37a0 con 0x7fd4bc1521f0 2023-11-06T21:03:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.654+0000 7fd4c099b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4bc1521f0 0x7fd4bc0bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.654+0000 7fd4bbfff700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fd4bc0c7590 0x7fd4bc0c3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:42534/0 (socket says 172.21.15.33:42534) 2023-11-06T21:03:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.654+0000 7fd4c119c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd4bc0a8fa0 0x7fd4bc0a93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50258/0 (socket says 172.21.15.33:50258) 2023-11-06T21:03:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.654+0000 7fd4bbfff700 1 -- 172.21.15.33:0/3087752514 learned_addr learned my addr 172.21.15.33:0/3087752514 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.654+0000 7fd4c099b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4bc1521f0 0x7fd4bc0bbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50678/0 (socket says 172.21.15.33:50678) 2023-11-06T21:03:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.654+0000 7fd4c099b700 1 -- 172.21.15.33:0/3087752514 >> v1:172.21.15.69:6789/0 conn(0x7fd4bc0c7590 legacy=0x7fd4bc0c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.654+0000 7fd4c099b700 1 -- 172.21.15.33:0/3087752514 >> v1:172.21.15.33:6789/0 conn(0x7fd4bc0a8fa0 legacy=0x7fd4bc0a93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.654+0000 7fd4bb7fe700 1 -- 172.21.15.33:0/3087752514 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322543206 0 0) 0x7fd4bc0bddb0 con 0x7fd4bc0c7590 2023-11-06T21:03:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.654+0000 7fd4c099b700 1 -- 172.21.15.33:0/3087752514 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc1525d0 con 0x7fd4bc1521f0 2023-11-06T21:03:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.654+0000 7fd4c099b700 1 --2- 172.21.15.33:0/3087752514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1521f0 0x7fd4bc0bbde0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7fd4b0009760 tx=0x7fd4b0009060).ready entity=mon.0 client_cookie=55c7c93a07ba375a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.655+0000 7fd4bb7fe700 1 -- 172.21.15.33:0/3087752514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00090d0 con 0x7fd4bc1521f0 2023-11-06T21:03:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.655+0000 7fd4bb7fe700 1 -- 172.21.15.33:0/3087752514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b000f510 con 0x7fd4bc1521f0 2023-11-06T21:03:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.655+0000 7fd4bb7fe700 1 -- 172.21.15.33:0/3087752514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000f890 con 0x7fd4bc1521f0 2023-11-06T21:03:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.655+0000 7fd4c2bff700 1 -- 172.21.15.33:0/3087752514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1521f0 msgr2=0x7fd4bc0bbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.655+0000 7fd4c2bff700 1 --2- 172.21.15.33:0/3087752514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1521f0 0x7fd4bc0bbde0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7fd4b0009760 tx=0x7fd4b0009060).stop 2023-11-06T21:03:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.656+0000 7fd4c2bff700 1 -- 172.21.15.33:0/3087752514 shutdown_connections 2023-11-06T21:03:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.656+0000 7fd4c2bff700 1 --2- 172.21.15.33:0/3087752514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1521f0 0x7fd4bc0bbde0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.656+0000 7fd4c2bff700 1 -- 172.21.15.33:0/3087752514 >> 172.21.15.33:0/3087752514 conn(0x7fd4bc0631a0 msgr2=0x7fd4bc0c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.656+0000 7fd4c2bff700 1 -- 172.21.15.33:0/3087752514 shutdown_connections 2023-11-06T21:03:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.656+0000 7fd4c2bff700 1 -- 172.21.15.33:0/3087752514 wait complete. 2023-11-06T21:03:08.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.657+0000 7fd4c2bff700 1 Processor -- start 2023-11-06T21:03:08.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.657+0000 7fd4c2bff700 1 -- start start 2023-11-06T21:03:08.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.658+0000 7fd4c2bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4bc0a8fa0 0x7fd4bc14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:08.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.658+0000 7fd4c2bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4bc0c7590 0x7fd4bc154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:08.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.658+0000 7fd4c2bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1521f0 0x7fd4bc154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:08.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.658+0000 7fd4c2bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc153190 con 0x7fd4bc1521f0 2023-11-06T21:03:08.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.658+0000 7fd4c2bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc1555d0 con 0x7fd4bc0c7590 2023-11-06T21:03:08.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.658+0000 7fd4c2bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc152860 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c099b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4bc0a8fa0 0x7fd4bc14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:08.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c119c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1521f0 0x7fd4bc154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:08.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4bbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4bc0c7590 0x7fd4bc154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:08.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c099b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4bc0a8fa0 0x7fd4bc14deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:33232/0 (socket says 172.21.15.33:33232) 2023-11-06T21:03:08.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c099b700 1 -- 172.21.15.33:0/1906611801 learned_addr learned my addr 172.21.15.33:0/1906611801 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:08.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c099b700 1 -- 172.21.15.33:0/1906611801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4bc0c7590 msgr2=0x7fd4bc154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:08.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c099b700 1 --2- 172.21.15.33:0/1906611801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4bc0c7590 0x7fd4bc154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c099b700 1 -- 172.21.15.33:0/1906611801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1521f0 msgr2=0x7fd4bc154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c099b700 1 --2- 172.21.15.33:0/1906611801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1521f0 0x7fd4bc154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c099b700 1 -- 172.21.15.33:0/1906611801 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00092a0 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c119c700 1 --2- 172.21.15.33:0/1906611801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1521f0 0x7fd4bc154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.659+0000 7fd4c099b700 1 --2- 172.21.15.33:0/1906611801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4bc0a8fa0 0x7fd4bc14deb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fd4b0009eb0 tx=0x7fd4b0017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.660+0000 7fd4b8ff9700 1 -- 172.21.15.33:0/1906611801 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000fea0 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.660+0000 7fd4b8ff9700 1 -- 172.21.15.33:0/1906611801 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b0006910 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.660+0000 7fd4b8ff9700 1 -- 172.21.15.33:0/1906611801 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000f890 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.660+0000 7fd4c2bff700 1 -- 172.21.15.33:0/1906611801 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4bc152b40 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.660+0000 7fd4c2bff700 1 -- 172.21.15.33:0/1906611801 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4bc0c6c20 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.661+0000 7fd4c2bff700 1 -- 172.21.15.33:0/1906611801 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4bc0c22d0 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.662+0000 7fd4b8ff9700 1 -- 172.21.15.33:0/1906611801 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 53) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fd4b001b620 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.663+0000 7fd4b8ff9700 1 -- 172.21.15.33:0/1906611801 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fd4b007b9a0 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.667+0000 7fd4b8ff9700 1 -- 172.21.15.33:0/1906611801 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd4b004cb10 con 0x7fd4bc0a8fa0 2023-11-06T21:03:08.683 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:08.683+0000 7fc95cf60680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:03:08.780 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:08.779+0000 7fc95cf60680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:03:08.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:08.890+0000 7fd4c2bff700 1 -- 172.21.15.33:0/1906611801 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd4bc1529a0 con 0x7fd4bc0a8fa0 2023-11-06T21:03:09.322 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T21:03:09.321+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T21:03:09.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.325+0000 7fd4b8ff9700 1 -- 172.21.15.33:0/1906611801 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd4b0029090 con 0x7fd4bc0a8fa0 2023-11-06T21:03:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.328+0000 7fd4c2bff700 1 -- 172.21.15.33:0/1906611801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4bc0a8fa0 msgr2=0x7fd4bc14deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.328+0000 7fd4c2bff700 1 --2- 172.21.15.33:0/1906611801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4bc0a8fa0 0x7fd4bc14deb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fd4b0009eb0 tx=0x7fd4b0017000).stop 2023-11-06T21:03:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.328+0000 7fd4c2bff700 1 -- 172.21.15.33:0/1906611801 shutdown_connections 2023-11-06T21:03:09.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.328+0000 7fd4c2bff700 1 --2- 172.21.15.33:0/1906611801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4bc0a8fa0 0x7fd4bc14deb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:09.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.328+0000 7fd4c2bff700 1 --2- 172.21.15.33:0/1906611801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1521f0 0x7fd4bc154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:09.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.328+0000 7fd4c2bff700 1 --2- 172.21.15.33:0/1906611801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4bc0c7590 0x7fd4bc154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:09.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.328+0000 7fd4c2bff700 1 -- 172.21.15.33:0/1906611801 >> 172.21.15.33:0/1906611801 conn(0x7fd4bc0631a0 msgr2=0x7fd4bc0c1710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:09.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.328+0000 7fd4c2bff700 1 -- 172.21.15.33:0/1906611801 shutdown_connections 2023-11-06T21:03:09.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.329+0000 7fd4c2bff700 1 -- 172.21.15.33:0/1906611801 wait complete. 2023-11-06T21:03:09.331 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T21:03:09.345 INFO:tasks.ceph.mds.c.smithi033.stderr: -6899> 2023-11-06T21:02:16.480+0000 7fb66fdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T21:03:09.360 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T21:03:09.397 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T21:03:09.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:03:09.538 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:03:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.561+0000 7f94aedff700 1 Processor -- start 2023-11-06T21:03:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.562+0000 7f94aedff700 1 -- start start 2023-11-06T21:03:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.563+0000 7f94aedff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a8157700 0x7f94a80c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.563+0000 7f94aedff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f94a8063840 0x7f94a80c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:09.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94aedff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a8158c00 con 0x7f94a80aaf50 2023-11-06T21:03:09.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94aedff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a80c18f0 con 0x7f94a8063840 2023-11-06T21:03:09.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94aedff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a8157ff0 con 0x7f94a8157700 2023-11-06T21:03:09.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94a7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f94a80aaf50 0x7f94a8063130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50262/0 (socket says 172.21.15.33:50262) 2023-11-06T21:03:09.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94a7fff700 1 -- 172.21.15.33:0/2387606115 learned_addr learned my addr 172.21.15.33:0/2387606115 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:09.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94acb9b700 1 --2- 172.21.15.33:0/2387606115 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a8157700 0x7f94a80c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:09.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94ad39c700 1 --2- 172.21.15.33:0/2387606115 >> v2:172.21.15.69:3300/0 conn(0x7f94a8063840 0x7f94a80c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:09.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94a77fe700 1 -- 172.21.15.33:0/2387606115 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237466768 0 0) 0x7f94a8158c00 con 0x7f94a80aaf50 2023-11-06T21:03:09.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94a77fe700 1 -- 172.21.15.33:0/2387606115 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94980033f0 con 0x7f94a80aaf50 2023-11-06T21:03:09.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94ad39c700 1 -- 172.21.15.33:0/2387606115 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a8157700 msgr2=0x7f94a80c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:09.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94ad39c700 1 --2- 172.21.15.33:0/2387606115 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a8157700 0x7f94a80c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:09.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94ad39c700 1 -- 172.21.15.33:0/2387606115 >> v1:172.21.15.33:6789/0 conn(0x7f94a80aaf50 legacy=0x7f94a8063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:09.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94ad39c700 1 -- 172.21.15.33:0/2387606115 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a81582d0 con 0x7f94a8063840 2023-11-06T21:03:09.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.564+0000 7f94acb9b700 1 --2- 172.21.15.33:0/2387606115 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a8157700 0x7f94a80c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:09.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.565+0000 7f94ad39c700 1 --2- 172.21.15.33:0/2387606115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a8063840 0x7f94a80c5d70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f94a0009e60 tx=0x7f94a0009780).ready entity=mon.1 client_cookie=a7b2d73f7dda7010 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:09.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.565+0000 7f94a77fe700 1 -- 172.21.15.33:0/2387606115 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000e070 con 0x7f94a8063840 2023-11-06T21:03:09.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.565+0000 7f94a77fe700 1 -- 172.21.15.33:0/2387606115 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a000bab0 con 0x7f94a8063840 2023-11-06T21:03:09.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.565+0000 7f94a77fe700 1 -- 172.21.15.33:0/2387606115 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000be30 con 0x7f94a8063840 2023-11-06T21:03:09.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.565+0000 7f94aedff700 1 -- 172.21.15.33:0/2387606115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a8063840 msgr2=0x7f94a80c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:09.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.565+0000 7f94aedff700 1 --2- 172.21.15.33:0/2387606115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a8063840 0x7f94a80c5d70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f94a0009e60 tx=0x7f94a0009780).stop 2023-11-06T21:03:09.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.566+0000 7f94aedff700 1 -- 172.21.15.33:0/2387606115 shutdown_connections 2023-11-06T21:03:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.566+0000 7f94aedff700 1 --2- 172.21.15.33:0/2387606115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a8063840 0x7f94a80c5d70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.566+0000 7f94aedff700 1 --2- 172.21.15.33:0/2387606115 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a8157700 0x7f94a80c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.566+0000 7f94aedff700 1 -- 172.21.15.33:0/2387606115 >> 172.21.15.33:0/2387606115 conn(0x7f94a805ec80 msgr2=0x7f94a80cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.566+0000 7f94aedff700 1 -- 172.21.15.33:0/2387606115 shutdown_connections 2023-11-06T21:03:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.566+0000 7f94aedff700 1 -- 172.21.15.33:0/2387606115 wait complete. 2023-11-06T21:03:09.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.567+0000 7f94aedff700 1 Processor -- start 2023-11-06T21:03:09.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.567+0000 7f94aedff700 1 -- start start 2023-11-06T21:03:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.568+0000 7f94aedff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a8063840 0x7f94a80c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.568+0000 7f94aedff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a80aaf50 0x7f94a814aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.568+0000 7f94aedff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a8157700 0x7f94a8154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.568+0000 7f94aedff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80ab350 con 0x7f94a8063840 2023-11-06T21:03:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.568+0000 7f94aedff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80abed0 con 0x7f94a8157700 2023-11-06T21:03:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.568+0000 7f94aedff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80ab5a0 con 0x7f94a80aaf50 2023-11-06T21:03:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94acb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a8063840 0x7f94a80c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94acb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a8063840 0x7f94a80c28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50700/0 (socket says 172.21.15.33:50700) 2023-11-06T21:03:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94acb9b700 1 -- 172.21.15.33:0/1047430885 learned_addr learned my addr 172.21.15.33:0/1047430885 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94ad39c700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a8157700 0x7f94a8154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94a7fff700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a80aaf50 0x7f94a814aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94acb9b700 1 -- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a80aaf50 msgr2=0x7f94a814aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94acb9b700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a80aaf50 0x7f94a814aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94acb9b700 1 -- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a8157700 msgr2=0x7f94a8154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94acb9b700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a8157700 0x7f94a8154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94acb9b700 1 -- 172.21.15.33:0/1047430885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0092a0 con 0x7f94a8063840 2023-11-06T21:03:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94acb9b700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a8063840 0x7f94a80c28c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f949c00ea20 tx=0x7f949c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94a4ff9700 1 -- 172.21.15.33:0/1047430885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c00ee60 con 0x7f94a8063840 2023-11-06T21:03:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94a4ff9700 1 -- 172.21.15.33:0/1047430885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c009600 con 0x7f94a8063840 2023-11-06T21:03:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.569+0000 7f94a4ff9700 1 -- 172.21.15.33:0/1047430885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c00ca60 con 0x7f94a8063840 2023-11-06T21:03:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.570+0000 7f94aedff700 1 -- 172.21.15.33:0/1047430885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a0009a20 con 0x7f94a8063840 2023-11-06T21:03:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.570+0000 7f94aedff700 1 -- 172.21.15.33:0/1047430885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a80abb60 con 0x7f94a8063840 2023-11-06T21:03:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.571+0000 7f94aedff700 1 -- 172.21.15.33:0/1047430885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a806c6c0 con 0x7f94a8063840 2023-11-06T21:03:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.572+0000 7f94a4ff9700 1 -- 172.21.15.33:0/1047430885 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 54) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f949c01b400 con 0x7f94a8063840 2023-11-06T21:03:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.573+0000 7f94a4ff9700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f948c05f030 0x7f948c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.573+0000 7f94a4ff9700 1 -- 172.21.15.33:0/1047430885 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6751+0+0 (secure 0 0 0) 0x7f949c07a710 con 0x7f94a8063840 2023-11-06T21:03:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.573+0000 7f94a7fff700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f948c05f030 0x7f948c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.574+0000 7f94a7fff700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f948c05f030 0x7f948c0614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f94a8168c90 tx=0x7f9494009040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.577+0000 7f94a4ff9700 1 -- 172.21.15.33:0/1047430885 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f949c04f200 con 0x7f94a8063840 2023-11-06T21:03:09.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:09.802+0000 7f94aedff700 1 -- 172.21.15.33:0/1047430885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f94a8062b60 con 0x7f94a8063840 2023-11-06T21:03:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.341+0000 7f94a4ff9700 1 -- 172.21.15.33:0/1047430885 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f949c0160a0 con 0x7f94a8063840 2023-11-06T21:03:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 -- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f948c05f030 msgr2=0x7f948c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f948c05f030 0x7f948c0614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f94a8168c90 tx=0x7f9494009040).stop 2023-11-06T21:03:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 -- 172.21.15.33:0/1047430885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a8063840 msgr2=0x7f94a80c28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a8063840 0x7f94a80c28c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f949c00ea20 tx=0x7f949c009060).stop 2023-11-06T21:03:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 -- 172.21.15.33:0/1047430885 shutdown_connections 2023-11-06T21:03:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f948c05f030 0x7f948c0614f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94a8157700 0x7f94a8154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a8063840 0x7f94a80c28c0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 --2- 172.21.15.33:0/1047430885 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f94a80aaf50 0x7f94a814aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 -- 172.21.15.33:0/1047430885 >> 172.21.15.33:0/1047430885 conn(0x7f94a805ec80 msgr2=0x7f94a81594a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.344+0000 7f94aedff700 1 -- 172.21.15.33:0/1047430885 shutdown_connections 2023-11-06T21:03:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.345+0000 7f94aedff700 1 -- 172.21.15.33:0/1047430885 wait complete. 2023-11-06T21:03:10.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T21:03:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.565+0000 7facd0fff700 1 Processor -- start 2023-11-06T21:03:10.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.565+0000 7facd0fff700 1 -- start start 2023-11-06T21:03:10.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.566+0000 7facd0fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faccc154340 0x7faccc0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:10.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.566+0000 7facd0fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faccc158720 0x7faccc063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:10.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.566+0000 7facd0fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc0c0e00 con 0x7faccc154720 2023-11-06T21:03:10.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.566+0000 7facd0fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc158bf0 con 0x7faccc154340 2023-11-06T21:03:10.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.566+0000 7facd0fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc158ed0 con 0x7faccc158720 2023-11-06T21:03:10.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.567+0000 7facca59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faccc154340 0x7faccc0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:10.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.567+0000 7facca59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faccc154340 0x7faccc0aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50714/0 (socket says 172.21.15.33:50714) 2023-11-06T21:03:10.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.567+0000 7facca59c700 1 -- 172.21.15.33:0/1438216873 learned_addr learned my addr 172.21.15.33:0/1438216873 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:10.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.567+0000 7faccad9d700 1 --2- 172.21.15.33:0/1438216873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faccc158720 0x7faccc063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.567+0000 7facca59c700 1 -- 172.21.15.33:0/1438216873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faccc158720 msgr2=0x7faccc063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.567+0000 7facca59c700 1 --2- 172.21.15.33:0/1438216873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faccc158720 0x7faccc063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.567+0000 7facca59c700 1 -- 172.21.15.33:0/1438216873 >> v1:172.21.15.33:6789/0 conn(0x7faccc154720 legacy=0x7faccc158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.567+0000 7facca59c700 1 -- 172.21.15.33:0/1438216873 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc0c5d80 con 0x7faccc154340 2023-11-06T21:03:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.567+0000 7facca59c700 1 --2- 172.21.15.33:0/1438216873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc154340 0x7faccc0aae40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7facbc006fd0 tx=0x7facbc0090f0).ready entity=mon.0 client_cookie=a6f38392debab849 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:10.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facc959a700 1 -- 172.21.15.33:0/1438216873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00b600 con 0x7faccc154340 2023-11-06T21:03:10.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facc959a700 1 -- 172.21.15.33:0/1438216873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc00b760 con 0x7faccc154340 2023-11-06T21:03:10.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facc959a700 1 -- 172.21.15.33:0/1438216873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00bae0 con 0x7faccc154340 2023-11-06T21:03:10.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facd0fff700 1 -- 172.21.15.33:0/1438216873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc154340 msgr2=0x7faccc0aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:10.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facd0fff700 1 --2- 172.21.15.33:0/1438216873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc154340 0x7faccc0aae40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7facbc006fd0 tx=0x7facbc0090f0).stop 2023-11-06T21:03:10.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facd0fff700 1 -- 172.21.15.33:0/1438216873 shutdown_connections 2023-11-06T21:03:10.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facd0fff700 1 --2- 172.21.15.33:0/1438216873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faccc158720 0x7faccc063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:10.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facd0fff700 1 --2- 172.21.15.33:0/1438216873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc154340 0x7faccc0aae40 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:10.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facd0fff700 1 -- 172.21.15.33:0/1438216873 >> 172.21.15.33:0/1438216873 conn(0x7faccc05ec80 msgr2=0x7faccc0cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:10.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facd0fff700 1 -- 172.21.15.33:0/1438216873 shutdown_connections 2023-11-06T21:03:10.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.568+0000 7facd0fff700 1 -- 172.21.15.33:0/1438216873 wait complete. 2023-11-06T21:03:10.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.570+0000 7facd0fff700 1 Processor -- start 2023-11-06T21:03:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.570+0000 7facd0fff700 1 -- start start 2023-11-06T21:03:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7facd0fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faccc154340 0x7faccc1657f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7facd0fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faccc154720 0x7faccc152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7facd0fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc158720 0x7faccc0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7facd0fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c92f0 con 0x7faccc158720 2023-11-06T21:03:10.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7facd0fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0adf30 con 0x7faccc154720 2023-11-06T21:03:10.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7facd0fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0ae210 con 0x7faccc154340 2023-11-06T21:03:10.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7faccad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc158720 0x7faccc0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:10.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7faccad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc158720 0x7faccc0ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50730/0 (socket says 172.21.15.33:50730) 2023-11-06T21:03:10.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7faccad9d700 1 -- 172.21.15.33:0/4080447124 learned_addr learned my addr 172.21.15.33:0/4080447124 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7facca59c700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faccc154340 0x7faccc1657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.571+0000 7facc9d9b700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faccc154720 0x7faccc152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7faccad9d700 1 -- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faccc154340 msgr2=0x7faccc1657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7faccad9d700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faccc154340 0x7faccc1657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7faccad9d700 1 -- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faccc154720 msgr2=0x7faccc152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:10.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7faccad9d700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faccc154720 0x7faccc152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:10.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7faccad9d700 1 -- 172.21.15.33:0/4080447124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc009370 con 0x7faccc158720 2023-11-06T21:03:10.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7faccad9d700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc158720 0x7faccc0ae810 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7facc4007f20 tx=0x7facc4009800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:10.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7facba7fc700 1 -- 172.21.15.33:0/4080447124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc400eb90 con 0x7faccc158720 2023-11-06T21:03:10.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7facba7fc700 1 -- 172.21.15.33:0/4080447124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc400ecf0 con 0x7faccc158720 2023-11-06T21:03:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7facd0fff700 1 -- 172.21.15.33:0/4080447124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc0c9540 con 0x7faccc158720 2023-11-06T21:03:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7facba7fc700 1 -- 172.21.15.33:0/4080447124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4014610 con 0x7faccc158720 2023-11-06T21:03:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.572+0000 7facd0fff700 1 -- 172.21.15.33:0/4080447124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc165ce0 con 0x7faccc158720 2023-11-06T21:03:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.573+0000 7facd0fff700 1 -- 172.21.15.33:0/4080447124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc0c6060 con 0x7faccc158720 2023-11-06T21:03:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.574+0000 7facba7fc700 1 -- 172.21.15.33:0/4080447124 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 54) v1 ==== 75340+0+0 (secure 0 0 0) 0x7facc40149c0 con 0x7faccc158720 2023-11-06T21:03:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.575+0000 7facba7fc700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facac05f030 0x7facac0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.575+0000 7facba7fc700 1 -- 172.21.15.33:0/4080447124 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6751+0+0 (secure 0 0 0) 0x7facc407a5f0 con 0x7faccc158720 2023-11-06T21:03:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.575+0000 7facca59c700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facac05f030 0x7facac0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.576+0000 7facca59c700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facac05f030 0x7facac0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7facbc009130 tx=0x7facbc017000).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:10.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.579+0000 7facba7fc700 1 -- 172.21.15.33:0/4080447124 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7facc404f140 con 0x7faccc158720 2023-11-06T21:03:10.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:10.762+0000 7facd0fff700 1 -- 172.21.15.33:0/4080447124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faccc062ba0 con 0x7faccc158720 2023-11-06T21:03:11.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.123+0000 7facba7fc700 1 -- 172.21.15.33:0/4080447124 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7facc4078210 con 0x7faccc158720 2023-11-06T21:03:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.348+0000 7facba7fc700 1 -- 172.21.15.33:0/4080447124 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v137) v1 ==== 181+0+0 (secure 0 0 0) 0x7facc40726a0 con 0x7faccc158720 2023-11-06T21:03:11.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.350+0000 7facd0fff700 1 -- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facac05f030 msgr2=0x7facac0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.350+0000 7facd0fff700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facac05f030 0x7facac0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7facbc009130 tx=0x7facbc017000).stop 2023-11-06T21:03:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.350+0000 7facd0fff700 1 -- 172.21.15.33:0/4080447124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc158720 msgr2=0x7faccc0ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.350+0000 7facd0fff700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc158720 0x7faccc0ae810 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7facc4007f20 tx=0x7facc4009800).stop 2023-11-06T21:03:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.351+0000 7facd0fff700 1 -- 172.21.15.33:0/4080447124 shutdown_connections 2023-11-06T21:03:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.351+0000 7facd0fff700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7facac05f030 0x7facac0614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.351+0000 7facd0fff700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faccc154340 0x7faccc1657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.351+0000 7facd0fff700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faccc158720 0x7faccc0ae810 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.351+0000 7facd0fff700 1 --2- 172.21.15.33:0/4080447124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faccc154720 0x7faccc152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.351+0000 7facd0fff700 1 -- 172.21.15.33:0/4080447124 >> 172.21.15.33:0/4080447124 conn(0x7faccc05ec80 msgr2=0x7faccc0c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.351+0000 7facd0fff700 1 -- 172.21.15.33:0/4080447124 shutdown_connections 2023-11-06T21:03:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.351+0000 7facd0fff700 1 -- 172.21.15.33:0/4080447124 wait complete. 2023-11-06T21:03:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T21:03:11.371 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T21:03:11.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.574+0000 7fc84c5ff700 1 Processor -- start 2023-11-06T21:03:11.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.574+0000 7fc84c5ff700 1 -- start start 2023-11-06T21:03:11.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.575+0000 7fc84c5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc844153ed0 0x7fc844158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:11.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.575+0000 7fc84c5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc8441542b0 0x7fc8440c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:11.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.575+0000 7fc84c5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8440c9f80 con 0x7fc8440c8ee0 2023-11-06T21:03:11.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.575+0000 7fc84c5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8440cab20 con 0x7fc844153ed0 2023-11-06T21:03:11.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.575+0000 7fc84c5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8440c6000 con 0x7fc8441542b0 2023-11-06T21:03:11.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.575+0000 7fc84a39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc844153ed0 0x7fc844158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:11.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.575+0000 7fc84a39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc844153ed0 0x7fc844158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50744/0 (socket says 172.21.15.33:50744) 2023-11-06T21:03:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.575+0000 7fc84a39b700 1 -- 172.21.15.33:0/1131840719 learned_addr learned my addr 172.21.15.33:0/1131840719 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.576+0000 7fc84ab9c700 1 --2- 172.21.15.33:0/1131840719 >> v2:172.21.15.69:3300/0 conn(0x7fc8441542b0 0x7fc8440c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.576+0000 7fc84a39b700 1 -- 172.21.15.33:0/1131840719 >> v2:172.21.15.69:3300/0 conn(0x7fc8441542b0 msgr2=0x7fc8440c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.576+0000 7fc84a39b700 1 --2- 172.21.15.33:0/1131840719 >> v2:172.21.15.69:3300/0 conn(0x7fc8441542b0 0x7fc8440c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.576+0000 7fc84a39b700 1 -- 172.21.15.33:0/1131840719 >> v1:172.21.15.33:6789/0 conn(0x7fc8440c8ee0 legacy=0x7fc8440ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:11.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.576+0000 7fc84a39b700 1 -- 172.21.15.33:0/1131840719 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8440c9370 con 0x7fc844153ed0 2023-11-06T21:03:11.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.576+0000 7fc84a39b700 1 --2- 172.21.15.33:0/1131840719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 0x7fc844158600 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7fc8440c85d0 tx=0x7fc8400090f0).ready entity=mon.0 client_cookie=30a78f4fcec61255 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:11.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.576+0000 7fc849399700 1 -- 172.21.15.33:0/1131840719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000b600 con 0x7fc844153ed0 2023-11-06T21:03:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.576+0000 7fc849399700 1 -- 172.21.15.33:0/1131840719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84000b760 con 0x7fc844153ed0 2023-11-06T21:03:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.576+0000 7fc849399700 1 -- 172.21.15.33:0/1131840719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000bae0 con 0x7fc844153ed0 2023-11-06T21:03:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.577+0000 7fc84c5ff700 1 -- 172.21.15.33:0/1131840719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 msgr2=0x7fc844158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.577+0000 7fc84c5ff700 1 --2- 172.21.15.33:0/1131840719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 0x7fc844158600 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7fc8440c85d0 tx=0x7fc8400090f0).stop 2023-11-06T21:03:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.577+0000 7fc84c5ff700 1 -- 172.21.15.33:0/1131840719 shutdown_connections 2023-11-06T21:03:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.577+0000 7fc84c5ff700 1 --2- 172.21.15.33:0/1131840719 >> v2:172.21.15.69:3300/0 conn(0x7fc8441542b0 0x7fc8440c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.577+0000 7fc84c5ff700 1 --2- 172.21.15.33:0/1131840719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 0x7fc844158600 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.577+0000 7fc84c5ff700 1 -- 172.21.15.33:0/1131840719 >> 172.21.15.33:0/1131840719 conn(0x7fc84405e1b0 msgr2=0x7fc84405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.577+0000 7fc84c5ff700 1 -- 172.21.15.33:0/1131840719 shutdown_connections 2023-11-06T21:03:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.577+0000 7fc84c5ff700 1 -- 172.21.15.33:0/1131840719 wait complete. 2023-11-06T21:03:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.579+0000 7fc84c5ff700 1 Processor -- start 2023-11-06T21:03:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.579+0000 7fc84c5ff700 1 -- start start 2023-11-06T21:03:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.580+0000 7fc84c5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 0x7fc8440ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84c5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc8441542b0 0x7fc8440b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84c5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc844148b00 0x7fc8440c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84c5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8440b0820 con 0x7fc844153ed0 2023-11-06T21:03:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84c5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8440addd0 con 0x7fc844148b00 2023-11-06T21:03:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84c5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8440ae670 con 0x7fc8441542b0 2023-11-06T21:03:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84a39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 0x7fc8440ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84a39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 0x7fc8440ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50752/0 (socket says 172.21.15.33:50752) 2023-11-06T21:03:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84a39b700 1 -- 172.21.15.33:0/3526688761 learned_addr learned my addr 172.21.15.33:0/3526688761 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84ab9c700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc844148b00 0x7fc8440c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc849b9a700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc8441542b0 0x7fc8440b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84a39b700 1 -- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc8441542b0 msgr2=0x7fc8440b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84a39b700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc8441542b0 0x7fc8440b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84a39b700 1 -- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc844148b00 msgr2=0x7fc8440c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84a39b700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc844148b00 0x7fc8440c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84a39b700 1 -- 172.21.15.33:0/3526688761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc840009370 con 0x7fc844153ed0 2023-11-06T21:03:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.581+0000 7fc84a39b700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 0x7fc8440ac170 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7fc840009130 tx=0x7fc840017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.582+0000 7fc83a7fc700 1 -- 172.21.15.33:0/3526688761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000b8d0 con 0x7fc844153ed0 2023-11-06T21:03:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.582+0000 7fc83a7fc700 1 -- 172.21.15.33:0/3526688761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84000ba30 con 0x7fc844153ed0 2023-11-06T21:03:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.582+0000 7fc83a7fc700 1 -- 172.21.15.33:0/3526688761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84001c670 con 0x7fc844153ed0 2023-11-06T21:03:11.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.582+0000 7fc84c5ff700 1 -- 172.21.15.33:0/3526688761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc844148f80 con 0x7fc844153ed0 2023-11-06T21:03:11.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.582+0000 7fc84c5ff700 1 -- 172.21.15.33:0/3526688761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8440ae020 con 0x7fc844153ed0 2023-11-06T21:03:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.583+0000 7fc84c5ff700 1 -- 172.21.15.33:0/3526688761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84406c6c0 con 0x7fc844153ed0 2023-11-06T21:03:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.584+0000 7fc83a7fc700 1 -- 172.21.15.33:0/3526688761 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc84001ca20 con 0x7fc844153ed0 2023-11-06T21:03:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.585+0000 7fc83a7fc700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc82805f270 0x7fc828061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.585+0000 7fc83a7fc700 1 -- 172.21.15.33:0/3526688761 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6313+0+0 (secure 0 0 0) 0x7fc84007c1c0 con 0x7fc844153ed0 2023-11-06T21:03:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.585+0000 7fc849b9a700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc82805f270 0x7fc828061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.586+0000 7fc849b9a700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc82805f270 0x7fc828061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc844167b40 tx=0x7fc834009040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.589+0000 7fc83a7fc700 1 -- 172.21.15.33:0/3526688761 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc84004d250 con 0x7fc844153ed0 2023-11-06T21:03:11.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:11.773+0000 7fc84c5ff700 1 -- 172.21.15.33:0/3526688761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8440c9650 con 0x7fc844153ed0 2023-11-06T21:03:12.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.035+0000 7fc83a7fc700 1 -- 172.21.15.33:0/3526688761 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v138) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc84002a020 con 0x7fc844153ed0 2023-11-06T21:03:12.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 -- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc82805f270 msgr2=0x7fc828061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc82805f270 0x7fc828061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc844167b40 tx=0x7fc834009040).stop 2023-11-06T21:03:12.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 -- 172.21.15.33:0/3526688761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 msgr2=0x7fc8440ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 0x7fc8440ac170 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7fc840009130 tx=0x7fc840017000).stop 2023-11-06T21:03:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 -- 172.21.15.33:0/3526688761 shutdown_connections 2023-11-06T21:03:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc82805f270 0x7fc828061730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc844148b00 0x7fc8440c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc844153ed0 0x7fc8440ac170 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 --2- 172.21.15.33:0/3526688761 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc8441542b0 0x7fc8440b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 -- 172.21.15.33:0/3526688761 >> 172.21.15.33:0/3526688761 conn(0x7fc84405e1b0 msgr2=0x7fc8440c7680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 -- 172.21.15.33:0/3526688761 shutdown_connections 2023-11-06T21:03:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.038+0000 7fc84c5ff700 1 -- 172.21.15.33:0/3526688761 wait complete. 2023-11-06T21:03:12.042 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T21:03:12.067 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T21:03:12.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.270+0000 7efe085ff700 1 Processor -- start 2023-11-06T21:03:12.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.270+0000 7efe085ff700 1 -- start start 2023-11-06T21:03:12.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.271+0000 7efe085ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe00069430 0x7efe000c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.271+0000 7efe085ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe000bcce0 con 0x7efe000bd170 2023-11-06T21:03:12.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.271+0000 7efe085ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe000a6100 con 0x7efe000a5b90 2023-11-06T21:03:12.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.271+0000 7efe085ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000c1720 con 0x7efe00069430 2023-11-06T21:03:12.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.272+0000 7efe05b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efe000bd170 0x7efe00069810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50280/0 (socket says 172.21.15.33:50280) 2023-11-06T21:03:12.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.272+0000 7efe05b9a700 1 -- 172.21.15.33:0/753438664 learned_addr learned my addr 172.21.15.33:0/753438664 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:12.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.272+0000 7efe0639b700 1 --2- 172.21.15.33:0/753438664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe00069430 0x7efe000c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.272+0000 7efe05399700 1 -- 172.21.15.33:0/753438664 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509215411 0 0) 0x7efe000bcce0 con 0x7efe000bd170 2023-11-06T21:03:12.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.272+0000 7efe05399700 1 -- 172.21.15.33:0/753438664 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdec0033f0 con 0x7efe000bd170 2023-11-06T21:03:12.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.272+0000 7efe05399700 1 -- 172.21.15.33:0/753438664 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997894618 0 0) 0x7efe000a6100 con 0x7efe000a5b90 2023-11-06T21:03:12.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.272+0000 7efe05399700 1 -- 172.21.15.33:0/753438664 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe000bcce0 con 0x7efe000a5b90 2023-11-06T21:03:12.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.272+0000 7efe0639b700 1 -- 172.21.15.33:0/753438664 >> v1:172.21.15.69:6789/0 conn(0x7efe000a5b90 legacy=0x7efe0015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.272+0000 7efe0639b700 1 -- 172.21.15.33:0/753438664 >> v1:172.21.15.33:6789/0 conn(0x7efe000bd170 legacy=0x7efe00069810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.272+0000 7efe0639b700 1 -- 172.21.15.33:0/753438664 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000bc690 con 0x7efe00069430 2023-11-06T21:03:12.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe0639b700 1 --2- 172.21.15.33:0/753438664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe00069430 0x7efe000c2d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7efdfc009790 tx=0x7efdfc009060).ready entity=mon.2 client_cookie=7369f2ba13d71665 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:12.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe05399700 1 -- 172.21.15.33:0/753438664 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00f510 con 0x7efe00069430 2023-11-06T21:03:12.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe05399700 1 -- 172.21.15.33:0/753438664 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc00f670 con 0x7efe00069430 2023-11-06T21:03:12.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe05399700 1 -- 172.21.15.33:0/753438664 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00f9f0 con 0x7efe00069430 2023-11-06T21:03:12.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe085ff700 1 -- 172.21.15.33:0/753438664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe00069430 msgr2=0x7efe000c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe085ff700 1 --2- 172.21.15.33:0/753438664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe00069430 0x7efe000c2d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7efdfc009790 tx=0x7efdfc009060).stop 2023-11-06T21:03:12.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe085ff700 1 -- 172.21.15.33:0/753438664 shutdown_connections 2023-11-06T21:03:12.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe085ff700 1 --2- 172.21.15.33:0/753438664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe00069430 0x7efe000c2d80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe085ff700 1 -- 172.21.15.33:0/753438664 >> 172.21.15.33:0/753438664 conn(0x7efe000c6a40 msgr2=0x7efe000c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:12.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe085ff700 1 -- 172.21.15.33:0/753438664 shutdown_connections 2023-11-06T21:03:12.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.273+0000 7efe085ff700 1 -- 172.21.15.33:0/753438664 wait complete. 2023-11-06T21:03:12.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.275+0000 7efe085ff700 1 Processor -- start 2023-11-06T21:03:12.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.275+0000 7efe085ff700 1 -- start start 2023-11-06T21:03:12.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.276+0000 7efe085ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe00069430 0x7efe00151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.276+0000 7efe085ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000a5b90 0x7efe00146ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.276+0000 7efe085ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe000bd170 0x7efe00150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.276+0000 7efe085ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe00152160 con 0x7efe000a5b90 2023-11-06T21:03:12.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.276+0000 7efe085ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe00153720 con 0x7efe00069430 2023-11-06T21:03:12.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.276+0000 7efe085ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe00165430 con 0x7efe000bd170 2023-11-06T21:03:12.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.276+0000 7efe05b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000a5b90 0x7efe00146ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe0639b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe00069430 0x7efe00151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe05b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000a5b90 0x7efe00146ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50762/0 (socket says 172.21.15.33:50762) 2023-11-06T21:03:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe05b9a700 1 -- 172.21.15.33:0/1525379247 learned_addr learned my addr 172.21.15.33:0/1525379247 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe06b9c700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe000bd170 0x7efe00150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe05b9a700 1 -- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe000bd170 msgr2=0x7efe00150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe05b9a700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe000bd170 0x7efe00150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe05b9a700 1 -- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe00069430 msgr2=0x7efe00151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe05b9a700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe00069430 0x7efe00151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe05b9a700 1 -- 172.21.15.33:0/1525379247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0092a0 con 0x7efe000a5b90 2023-11-06T21:03:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe06b9c700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe000bd170 0x7efe00150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe0639b700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe00069430 0x7efe00151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe05b9a700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000a5b90 0x7efe00146ea0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7efdf000c410 tx=0x7efdf0007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efdf67fc700 1 -- 172.21.15.33:0/1525379247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00070d0 con 0x7efe000a5b90 2023-11-06T21:03:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efdf67fc700 1 -- 172.21.15.33:0/1525379247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf0007230 con 0x7efe000a5b90 2023-11-06T21:03:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efe085ff700 1 -- 172.21.15.33:0/1525379247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe00165710 con 0x7efe000a5b90 2023-11-06T21:03:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.277+0000 7efdf67fc700 1 -- 172.21.15.33:0/1525379247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00075b0 con 0x7efe000a5b90 2023-11-06T21:03:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.278+0000 7efe085ff700 1 -- 172.21.15.33:0/1525379247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe00165cd0 con 0x7efe000a5b90 2023-11-06T21:03:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.279+0000 7efe085ff700 1 -- 172.21.15.33:0/1525379247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe001538b0 con 0x7efe000a5b90 2023-11-06T21:03:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.279+0000 7efdf67fc700 1 -- 172.21.15.33:0/1525379247 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7efdf00059b0 con 0x7efe000a5b90 2023-11-06T21:03:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.280+0000 7efdf67fc700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde405f2a0 0x7efde4061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.280+0000 7efdf67fc700 1 -- 172.21.15.33:0/1525379247 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5899+0+0 (secure 0 0 0) 0x7efdf00799b0 con 0x7efe000a5b90 2023-11-06T21:03:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.281+0000 7efe0639b700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde405f2a0 0x7efde4061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.281+0000 7efe0639b700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde405f2a0 0x7efde4061760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7efdfc009fd0 tx=0x7efdfc017000).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:12.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.284+0000 7efdf67fc700 1 -- 172.21.15.33:0/1525379247 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efdf004abe0 con 0x7efe000a5b90 2023-11-06T21:03:12.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.481+0000 7efe085ff700 1 -- 172.21.15.33:0/1525379247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7efe00063d00 con 0x7efe000a5b90 2023-11-06T21:03:12.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.485+0000 7efdf67fc700 1 -- 172.21.15.33:0/1525379247 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v24) v1 ==== 120+0+0 (secure 0 0 0) 0x7efdf001b340 con 0x7efe000a5b90 2023-11-06T21:03:12.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 -- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde405f2a0 msgr2=0x7efde4061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde405f2a0 0x7efde4061760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7efdfc009fd0 tx=0x7efdfc017000).stop 2023-11-06T21:03:12.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 -- 172.21.15.33:0/1525379247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000a5b90 msgr2=0x7efe00146ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000a5b90 0x7efe00146ea0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7efdf000c410 tx=0x7efdf0007c70).stop 2023-11-06T21:03:12.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 -- 172.21.15.33:0/1525379247 shutdown_connections 2023-11-06T21:03:12.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efde405f2a0 0x7efde4061760 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efe000bd170 0x7efe00150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe00069430 0x7efe00151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 --2- 172.21.15.33:0/1525379247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000a5b90 0x7efe00146ea0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 -- 172.21.15.33:0/1525379247 >> 172.21.15.33:0/1525379247 conn(0x7efe000c6a40 msgr2=0x7efe0014b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:12.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 -- 172.21.15.33:0/1525379247 shutdown_connections 2023-11-06T21:03:12.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.487+0000 7efe085ff700 1 -- 172.21.15.33:0/1525379247 wait complete. 2023-11-06T21:03:12.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T21:03:12.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.702+0000 7ff89dbff700 1 Processor -- start 2023-11-06T21:03:12.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.702+0000 7ff89dbff700 1 -- start start 2023-11-06T21:03:12.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.703+0000 7ff89dbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8980c72c0 0x7ff8981553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.703+0000 7ff89dbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 0x7ff8980630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.703+0000 7ff89dbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff898158420 0x7ff8980635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.703+0000 7ff89dbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff898063ac0 con 0x7ff8980c72c0 2023-11-06T21:03:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.703+0000 7ff89dbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff898152470 con 0x7ff898158420 2023-11-06T21:03:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.703+0000 7ff89dbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff898154990 con 0x7ff898157fa0 2023-11-06T21:03:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff8977fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8980c72c0 0x7ff8981553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff8977fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8980c72c0 0x7ff8981553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50778/0 (socket says 172.21.15.33:50778) 2023-11-06T21:03:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff8977fe700 1 -- 172.21.15.33:0/2538136691 learned_addr learned my addr 172.21.15.33:0/2538136691 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff897fff700 1 --2- 172.21.15.33:0/2538136691 >> v2:172.21.15.69:3300/0 conn(0x7ff898158420 0x7ff8980635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff896ffd700 1 --2- 172.21.15.33:0/2538136691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 0x7ff8980630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff8977fe700 1 -- 172.21.15.33:0/2538136691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 msgr2=0x7ff8980630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff8977fe700 1 --2- 172.21.15.33:0/2538136691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 0x7ff8980630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff8977fe700 1 -- 172.21.15.33:0/2538136691 >> v2:172.21.15.69:3300/0 conn(0x7ff898158420 msgr2=0x7ff8980635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff8977fe700 1 --2- 172.21.15.33:0/2538136691 >> v2:172.21.15.69:3300/0 conn(0x7ff898158420 0x7ff8980635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff8977fe700 1 -- 172.21.15.33:0/2538136691 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8980c76a0 con 0x7ff8980c72c0 2023-11-06T21:03:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff896ffd700 1 --2- 172.21.15.33:0/2538136691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 0x7ff8980630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.704+0000 7ff8977fe700 1 --2- 172.21.15.33:0/2538136691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 0x7ff8981553e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7ff898157f00 tx=0x7ff8880090f0).ready entity=mon.0 client_cookie=c5f20ff81627e138 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff8967fc700 1 -- 172.21.15.33:0/2538136691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88800b520 con 0x7ff8980c72c0 2023-11-06T21:03:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff8967fc700 1 -- 172.21.15.33:0/2538136691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88800b680 con 0x7ff8980c72c0 2023-11-06T21:03:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff8967fc700 1 -- 172.21.15.33:0/2538136691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88800ba00 con 0x7ff8980c72c0 2023-11-06T21:03:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff89dbff700 1 -- 172.21.15.33:0/2538136691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 msgr2=0x7ff8981553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff89dbff700 1 --2- 172.21.15.33:0/2538136691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 0x7ff8981553e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7ff898157f00 tx=0x7ff8880090f0).stop 2023-11-06T21:03:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff89dbff700 1 -- 172.21.15.33:0/2538136691 shutdown_connections 2023-11-06T21:03:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff89dbff700 1 --2- 172.21.15.33:0/2538136691 >> v2:172.21.15.69:3300/0 conn(0x7ff898158420 0x7ff8980635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff89dbff700 1 --2- 172.21.15.33:0/2538136691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 0x7ff8981553e0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff89dbff700 1 --2- 172.21.15.33:0/2538136691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 0x7ff8980630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff89dbff700 1 -- 172.21.15.33:0/2538136691 >> 172.21.15.33:0/2538136691 conn(0x7ff89805ec20 msgr2=0x7ff8980cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff89dbff700 1 -- 172.21.15.33:0/2538136691 shutdown_connections 2023-11-06T21:03:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.705+0000 7ff89dbff700 1 -- 172.21.15.33:0/2538136691 wait complete. 2023-11-06T21:03:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.707+0000 7ff89dbff700 1 Processor -- start 2023-11-06T21:03:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.707+0000 7ff89dbff700 1 -- start start 2023-11-06T21:03:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff89dbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 0x7ff8980c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff89dbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 0x7ff8980c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff89dbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff898158420 0x7ff8981458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff89dbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980c2390 con 0x7ff8980c72c0 2023-11-06T21:03:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff89dbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980affa0 con 0x7ff898158420 2023-11-06T21:03:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff89dbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980ae6e0 con 0x7ff898157fa0 2023-11-06T21:03:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff8977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 0x7ff8980c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff8977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 0x7ff8980c1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50784/0 (socket says 172.21.15.33:50784) 2023-11-06T21:03:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff8977fe700 1 -- 172.21.15.33:0/2529939705 learned_addr learned my addr 172.21.15.33:0/2529939705 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff896ffd700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 0x7ff8980c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff897fff700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff898158420 0x7ff8981458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff8977fe700 1 -- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 msgr2=0x7ff8980c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff8977fe700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 0x7ff8980c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff8977fe700 1 -- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff898158420 msgr2=0x7ff8981458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff8977fe700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff898158420 0x7ff8981458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.708+0000 7ff8977fe700 1 -- 172.21.15.33:0/2529939705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff888009370 con 0x7ff8980c72c0 2023-11-06T21:03:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.709+0000 7ff8977fe700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 0x7ff8980c1ea0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7ff888009130 tx=0x7ff888017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.709+0000 7ff87f7fe700 1 -- 172.21.15.33:0/2529939705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88800be90 con 0x7ff8980c72c0 2023-11-06T21:03:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.709+0000 7ff87f7fe700 1 -- 172.21.15.33:0/2529939705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88800b7f0 con 0x7ff8980c72c0 2023-11-06T21:03:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.709+0000 7ff89dbff700 1 -- 172.21.15.33:0/2529939705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8980c9180 con 0x7ff8980c72c0 2023-11-06T21:03:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.709+0000 7ff87f7fe700 1 -- 172.21.15.33:0/2529939705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88801b660 con 0x7ff8980c72c0 2023-11-06T21:03:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.709+0000 7ff89dbff700 1 -- 172.21.15.33:0/2529939705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8980ae090 con 0x7ff8980c72c0 2023-11-06T21:03:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.710+0000 7ff89dbff700 1 -- 172.21.15.33:0/2529939705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff898158910 con 0x7ff8980c72c0 2023-11-06T21:03:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.711+0000 7ff87f7fe700 1 -- 172.21.15.33:0/2529939705 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff88801ba50 con 0x7ff8980c72c0 2023-11-06T21:03:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.712+0000 7ff87f7fe700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f270 0x7ff884061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.712+0000 7ff87f7fe700 1 -- 172.21.15.33:0/2529939705 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5899+0+0 (secure 0 0 0) 0x7ff88807bca0 con 0x7ff8980c72c0 2023-11-06T21:03:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.712+0000 7ff896ffd700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f270 0x7ff884061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.713+0000 7ff896ffd700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f270 0x7ff884061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff8980ca7b0 tx=0x7ff88c008040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.716+0000 7ff87f7fe700 1 -- 172.21.15.33:0/2529939705 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff88804ce10 con 0x7ff8980c72c0 2023-11-06T21:03:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.896+0000 7ff89dbff700 1 -- 172.21.15.33:0/2529939705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff898062b80 con 0x7ff8980c72c0 2023-11-06T21:03:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.900+0000 7ff87f7fe700 1 -- 172.21.15.33:0/2529939705 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff888029020 con 0x7ff8980c72c0 2023-11-06T21:03:12.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.902+0000 7ff89dbff700 1 -- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f270 msgr2=0x7ff884061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.902+0000 7ff89dbff700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f270 0x7ff884061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff8980ca7b0 tx=0x7ff88c008040).stop 2023-11-06T21:03:12.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.902+0000 7ff89dbff700 1 -- 172.21.15.33:0/2529939705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 msgr2=0x7ff8980c1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:12.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.902+0000 7ff89dbff700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 0x7ff8980c1ea0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7ff888009130 tx=0x7ff888017000).stop 2023-11-06T21:03:12.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.902+0000 7ff89dbff700 1 -- 172.21.15.33:0/2529939705 shutdown_connections 2023-11-06T21:03:12.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.902+0000 7ff89dbff700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff88405f270 0x7ff884061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.902+0000 7ff89dbff700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff898158420 0x7ff8981458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.902+0000 7ff89dbff700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8980c72c0 0x7ff8980c1ea0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.902+0000 7ff89dbff700 1 --2- 172.21.15.33:0/2529939705 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff898157fa0 0x7ff8980c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.902+0000 7ff89dbff700 1 -- 172.21.15.33:0/2529939705 >> 172.21.15.33:0/2529939705 conn(0x7ff89805ec20 msgr2=0x7ff8980ac7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.903+0000 7ff89dbff700 1 -- 172.21.15.33:0/2529939705 shutdown_connections 2023-11-06T21:03:12.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:12.903+0000 7ff89dbff700 1 -- 172.21.15.33:0/2529939705 wait complete. 2023-11-06T21:03:12.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-11-06T21:03:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.117+0000 7fa0b37ff700 1 Processor -- start 2023-11-06T21:03:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.117+0000 7fa0b37ff700 1 -- start start 2023-11-06T21:03:13.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.119+0000 7fa0b37ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 0x7fa0ac0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.119+0000 7fa0b37ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa0ac0c6460 0x7fa0ac0bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.119+0000 7fa0b37ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ac1554a0 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.119+0000 7fa0b37ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ac154960 con 0x7fa0ac0c6460 2023-11-06T21:03:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.119+0000 7fa0b37ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac155930 con 0x7fa0ac0c1840 2023-11-06T21:03:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.119+0000 7fa0b1d9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa0ac0c6460 0x7fa0ac0bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.119+0000 7fa0b159b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 0x7fa0ac0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.119+0000 7fa0b0d9a700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fa0ac0aa2d0 0x7fa0ac0c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:42554/0 (socket says 172.21.15.33:42554) 2023-11-06T21:03:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.119+0000 7fa0b0d9a700 1 -- 172.21.15.33:0/2472475093 learned_addr learned my addr 172.21.15.33:0/2472475093 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:13.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.120+0000 7fa0b159b700 1 -- 172.21.15.33:0/2472475093 >> v2:172.21.15.69:3300/0 conn(0x7fa0ac0c6460 msgr2=0x7fa0ac0bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:13.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.120+0000 7fa0b159b700 1 --2- 172.21.15.33:0/2472475093 >> v2:172.21.15.69:3300/0 conn(0x7fa0ac0c6460 0x7fa0ac0bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:13.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.120+0000 7fa0b159b700 1 -- 172.21.15.33:0/2472475093 >> v1:172.21.15.69:6789/0 conn(0x7fa0ac0aa2d0 legacy=0x7fa0ac0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:13.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.120+0000 7fa0b159b700 1 -- 172.21.15.33:0/2472475093 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac154b70 con 0x7fa0ac0c1840 2023-11-06T21:03:13.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.120+0000 7fa0a3fff700 1 -- 172.21.15.33:0/2472475093 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422027945 0 0) 0x7fa0ac1554a0 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.120+0000 7fa0b1d9c700 1 --2- 172.21.15.33:0/2472475093 >> v2:172.21.15.69:3300/0 conn(0x7fa0ac0c6460 0x7fa0ac0bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:13.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.120+0000 7fa0b159b700 1 --2- 172.21.15.33:0/2472475093 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 0x7fa0ac0a9de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fa0a8009760 tx=0x7fa0a8009060).ready entity=mon.2 client_cookie=6c425376bbe37b0b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:13.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.120+0000 7fa0a3fff700 1 -- 172.21.15.33:0/2472475093 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a800f510 con 0x7fa0ac0c1840 2023-11-06T21:03:13.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.120+0000 7fa0a3fff700 1 -- 172.21.15.33:0/2472475093 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a800f670 con 0x7fa0ac0c1840 2023-11-06T21:03:13.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.120+0000 7fa0a3fff700 1 -- 172.21.15.33:0/2472475093 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a800f9f0 con 0x7fa0ac0c1840 2023-11-06T21:03:13.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.121+0000 7fa0b37ff700 1 -- 172.21.15.33:0/2472475093 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 msgr2=0x7fa0ac0a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:13.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.121+0000 7fa0b37ff700 1 --2- 172.21.15.33:0/2472475093 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 0x7fa0ac0a9de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fa0a8009760 tx=0x7fa0a8009060).stop 2023-11-06T21:03:13.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.121+0000 7fa0b37ff700 1 -- 172.21.15.33:0/2472475093 shutdown_connections 2023-11-06T21:03:13.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.121+0000 7fa0b37ff700 1 --2- 172.21.15.33:0/2472475093 >> v2:172.21.15.69:3300/0 conn(0x7fa0ac0c6460 0x7fa0ac0bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:13.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.121+0000 7fa0b37ff700 1 --2- 172.21.15.33:0/2472475093 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 0x7fa0ac0a9de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:13.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.121+0000 7fa0b37ff700 1 -- 172.21.15.33:0/2472475093 >> 172.21.15.33:0/2472475093 conn(0x7fa0ac05ec80 msgr2=0x7fa0ac0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.121+0000 7fa0b37ff700 1 -- 172.21.15.33:0/2472475093 shutdown_connections 2023-11-06T21:03:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.121+0000 7fa0b37ff700 1 -- 172.21.15.33:0/2472475093 wait complete. 2023-11-06T21:03:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.122+0000 7fa0b37ff700 1 Processor -- start 2023-11-06T21:03:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.123+0000 7fa0b37ff700 1 -- start start 2023-11-06T21:03:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b37ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0ac0aa2d0 0x7fa0ac149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b37ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 0x7fa0ac0ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b37ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0ac0c6460 0x7fa0ac0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b37ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac14b840 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b37ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0c3d00 con 0x7fa0ac0c6460 2023-11-06T21:03:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b37ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0c3fe0 con 0x7fa0ac0c1840 2023-11-06T21:03:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b159b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0ac0aa2d0 0x7fa0ac149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:13.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b159b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0ac0aa2d0 0x7fa0ac149f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50794/0 (socket says 172.21.15.33:50794) 2023-11-06T21:03:13.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b159b700 1 -- 172.21.15.33:0/1723294860 learned_addr learned my addr 172.21.15.33:0/1723294860 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:13.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b0d9a700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 0x7fa0ac0ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:13.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b1d9c700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0ac0c6460 0x7fa0ac0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:13.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b159b700 1 -- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 msgr2=0x7fa0ac0ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:13.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b159b700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 0x7fa0ac0ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:13.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b159b700 1 -- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0ac0c6460 msgr2=0x7fa0ac0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:13.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b159b700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0ac0c6460 0x7fa0ac0adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:13.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.124+0000 7fa0b159b700 1 -- 172.21.15.33:0/1723294860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a4009a20 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.125+0000 7fa0b159b700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0ac0aa2d0 0x7fa0ac149f40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7fa0ac154aa0 tx=0x7fa0a8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.125+0000 7fa0a17fa700 1 -- 172.21.15.33:0/1723294860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a801a4d0 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.125+0000 7fa0a17fa700 1 -- 172.21.15.33:0/1723294860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a801e070 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.125+0000 7fa0b37ff700 1 -- 172.21.15.33:0/1723294860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a80092a0 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.125+0000 7fa0a17fa700 1 -- 172.21.15.33:0/1723294860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a801a880 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.125+0000 7fa0b37ff700 1 -- 172.21.15.33:0/1723294860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac0c45a0 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.126+0000 7fa0b37ff700 1 -- 172.21.15.33:0/1723294860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ac14bb00 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.127+0000 7fa0a17fa700 1 -- 172.21.15.33:0/1723294860 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa0a8020030 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.128+0000 7fa0a17fa700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa09005f270 0x7fa090061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.128+0000 7fa0a17fa700 1 -- 172.21.15.33:0/1723294860 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5899+0+0 (secure 0 0 0) 0x7fa0a807b350 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.128+0000 7fa0b0d9a700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa09005f270 0x7fa090061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.129+0000 7fa0b0d9a700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa09005f270 0x7fa090061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa0ac150fb0 tx=0x7fa09c007bd0).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.132+0000 7fa0a17fa700 1 -- 172.21.15.33:0/1723294860 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa0a804c4c0 con 0x7fa0ac0aa2d0 2023-11-06T21:03:13.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:13.388+0000 7fa0b37ff700 1 -- 172.21.15.33:0/1723294860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fa0ac06c6c0 con 0x7fa0ac0aa2d0 2023-11-06T21:03:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.161+0000 7fa0a17fa700 1 -- 172.21.15.33:0/1723294860 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v714) v1 ==== 148+0+0 (secure 0 0 0) 0x7fa0a804fae0 con 0x7fa0ac0aa2d0 2023-11-06T21:03:14.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 -- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa09005f270 msgr2=0x7fa090061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:14.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa09005f270 0x7fa090061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa0ac150fb0 tx=0x7fa09c007bd0).stop 2023-11-06T21:03:14.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 -- 172.21.15.33:0/1723294860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0ac0aa2d0 msgr2=0x7fa0ac149f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:14.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0ac0aa2d0 0x7fa0ac149f40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7fa0ac154aa0 tx=0x7fa0a8017000).stop 2023-11-06T21:03:14.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 -- 172.21.15.33:0/1723294860 shutdown_connections 2023-11-06T21:03:14.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa09005f270 0x7fa090061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0ac0c6460 0x7fa0ac0adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0ac0aa2d0 0x7fa0ac149f40 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 --2- 172.21.15.33:0/1723294860 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa0ac0c1840 0x7fa0ac0ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 -- 172.21.15.33:0/1723294860 >> 172.21.15.33:0/1723294860 conn(0x7fa0ac05ec80 msgr2=0x7fa0ac063140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 -- 172.21.15.33:0/1723294860 shutdown_connections 2023-11-06T21:03:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.164+0000 7fa0b37ff700 1 -- 172.21.15.33:0/1723294860 wait complete. 2023-11-06T21:03:14.185 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-11-06T21:03:14.186 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-11-06T21:03:14.186 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-06T21:03:14.187 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-11-06T21:03:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.387+0000 7f800e9ff700 1 Processor -- start 2023-11-06T21:03:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.387+0000 7f800e9ff700 1 -- start start 2023-11-06T21:03:14.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.389+0000 7f800e9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80080c1840 0x7f80080a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.389+0000 7f800e9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f80080aa2d0 0x7f80080c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.389+0000 7f800e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80080c6240 0x7f8008157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.389+0000 7f800e9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8008154960 con 0x7f80080c1840 2023-11-06T21:03:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.389+0000 7f800e9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80081554a0 con 0x7f80080aa2d0 2023-11-06T21:03:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.389+0000 7f800e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8008155930 con 0x7f80080c6240 2023-11-06T21:03:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.389+0000 7f8007fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80080c1840 0x7f80080a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.389+0000 7f8007fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80080c1840 0x7f80080a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50798/0 (socket says 172.21.15.33:50798) 2023-11-06T21:03:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.389+0000 7f8007fff700 1 -- 172.21.15.33:0/1203487485 learned_addr learned my addr 172.21.15.33:0/1203487485 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f800cf9c700 1 --2- 172.21.15.33:0/1203487485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80080c6240 0x7f8008157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f80077fe700 1 --2- 172.21.15.33:0/1203487485 >> v2:172.21.15.69:3300/0 conn(0x7f80080aa2d0 0x7f80080c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f8007fff700 1 -- 172.21.15.33:0/1203487485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80080c6240 msgr2=0x7f8008157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f8007fff700 1 --2- 172.21.15.33:0/1203487485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80080c6240 0x7f8008157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f8007fff700 1 -- 172.21.15.33:0/1203487485 >> v2:172.21.15.69:3300/0 conn(0x7f80080aa2d0 msgr2=0x7f80080c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f8007fff700 1 --2- 172.21.15.33:0/1203487485 >> v2:172.21.15.69:3300/0 conn(0x7f80080aa2d0 0x7f80080c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f8007fff700 1 -- 172.21.15.33:0/1203487485 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080c6690 con 0x7f80080c1840 2023-11-06T21:03:14.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f8007fff700 1 --2- 172.21.15.33:0/1203487485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c1840 0x7f80080a9de0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f80080adb40 tx=0x7f7ff80090f0).ready entity=mon.0 client_cookie=fbae1c2058bc9731 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f8006ffd700 1 -- 172.21.15.33:0/1203487485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800b600 con 0x7f80080c1840 2023-11-06T21:03:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f8006ffd700 1 -- 172.21.15.33:0/1203487485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff800b760 con 0x7f80080c1840 2023-11-06T21:03:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.390+0000 7f8006ffd700 1 -- 172.21.15.33:0/1203487485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800bae0 con 0x7f80080c1840 2023-11-06T21:03:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.391+0000 7f800e9ff700 1 -- 172.21.15.33:0/1203487485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c1840 msgr2=0x7f80080a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.391+0000 7f800e9ff700 1 --2- 172.21.15.33:0/1203487485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c1840 0x7f80080a9de0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f80080adb40 tx=0x7f7ff80090f0).stop 2023-11-06T21:03:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.391+0000 7f800e9ff700 1 -- 172.21.15.33:0/1203487485 shutdown_connections 2023-11-06T21:03:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.391+0000 7f800e9ff700 1 --2- 172.21.15.33:0/1203487485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80080c6240 0x7f8008157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.391+0000 7f800e9ff700 1 --2- 172.21.15.33:0/1203487485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c1840 0x7f80080a9de0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.391+0000 7f800e9ff700 1 --2- 172.21.15.33:0/1203487485 >> v2:172.21.15.69:3300/0 conn(0x7f80080aa2d0 0x7f80080c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.391+0000 7f800e9ff700 1 -- 172.21.15.33:0/1203487485 >> 172.21.15.33:0/1203487485 conn(0x7f800805ec80 msgr2=0x7f80080cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.391+0000 7f800e9ff700 1 -- 172.21.15.33:0/1203487485 shutdown_connections 2023-11-06T21:03:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.391+0000 7f800e9ff700 1 -- 172.21.15.33:0/1203487485 wait complete. 2023-11-06T21:03:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.393+0000 7f800e9ff700 1 Processor -- start 2023-11-06T21:03:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.393+0000 7f800e9ff700 1 -- start start 2023-11-06T21:03:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800e9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80080aa2d0 0x7f80081517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80080c1840 0x7f80080c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800e9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c6240 0x7f80080c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800e9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80081560d0 con 0x7f80080c6240 2023-11-06T21:03:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800e9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080c2970 con 0x7f80080aa2d0 2023-11-06T21:03:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80080c2c50 con 0x7f80080c1840 2023-11-06T21:03:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800cf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c6240 0x7f80080c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800cf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c6240 0x7f80080c23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50810/0 (socket says 172.21.15.33:50810) 2023-11-06T21:03:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800cf9c700 1 -- 172.21.15.33:0/130282079 learned_addr learned my addr 172.21.15.33:0/130282079 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f8007fff700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80080aa2d0 0x7f80081517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f80077fe700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80080c1840 0x7f80080c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800cf9c700 1 -- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80080c1840 msgr2=0x7f80080c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800cf9c700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80080c1840 0x7f80080c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800cf9c700 1 -- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80080aa2d0 msgr2=0x7f80081517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800cf9c700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80080aa2d0 0x7f80081517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.394+0000 7f800cf9c700 1 -- 172.21.15.33:0/130282079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff8009370 con 0x7f80080c6240 2023-11-06T21:03:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.395+0000 7f800cf9c700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c6240 0x7f80080c23a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f8000009fa0 tx=0x7f8000009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.395+0000 7f7feffff700 1 -- 172.21.15.33:0/130282079 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800000ebc0 con 0x7f80080c6240 2023-11-06T21:03:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.395+0000 7f7feffff700 1 -- 172.21.15.33:0/130282079 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800000ed20 con 0x7f80080c6240 2023-11-06T21:03:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.395+0000 7f7feffff700 1 -- 172.21.15.33:0/130282079 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000014610 con 0x7f80080c6240 2023-11-06T21:03:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.395+0000 7f800e9ff700 1 -- 172.21.15.33:0/130282079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80080c2f30 con 0x7f80080c6240 2023-11-06T21:03:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.395+0000 7f800e9ff700 1 -- 172.21.15.33:0/130282079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80080aca20 con 0x7f80080c6240 2023-11-06T21:03:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.396+0000 7f800e9ff700 1 -- 172.21.15.33:0/130282079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8008156590 con 0x7f80080c6240 2023-11-06T21:03:14.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.397+0000 7f7feffff700 1 -- 172.21.15.33:0/130282079 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f800001d3b0 con 0x7f80080c6240 2023-11-06T21:03:14.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.398+0000 7f7feffff700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ff405f240 0x7f7ff4061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.398+0000 7f7feffff700 1 -- 172.21.15.33:0/130282079 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5899+0+0 (secure 0 0 0) 0x7f800007a820 con 0x7f80080c6240 2023-11-06T21:03:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.398+0000 7f8007fff700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ff405f240 0x7f7ff4061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.399+0000 7f8007fff700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ff405f240 0x7f7ff4061700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f7ff8009130 tx=0x7f7ff8017000).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.402+0000 7f7feffff700 1 -- 172.21.15.33:0/130282079 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8008156590 con 0x7f80080c6240 2023-11-06T21:03:14.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:14.652+0000 7f800e9ff700 1 -- 172.21.15.33:0/130282079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f8008156590 con 0x7f80080c6240 2023-11-06T21:03:15.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.166+0000 7f7feffff700 1 -- 172.21.15.33:0/130282079 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v715) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8008156590 con 0x7f80080c6240 2023-11-06T21:03:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.168+0000 7f800e9ff700 1 -- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ff405f240 msgr2=0x7f7ff4061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.168+0000 7f800e9ff700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ff405f240 0x7f7ff4061700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f7ff8009130 tx=0x7f7ff8017000).stop 2023-11-06T21:03:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.168+0000 7f800e9ff700 1 -- 172.21.15.33:0/130282079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c6240 msgr2=0x7f80080c23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.168+0000 7f800e9ff700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c6240 0x7f80080c23a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f8000009fa0 tx=0x7f8000009900).stop 2023-11-06T21:03:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.169+0000 7f800e9ff700 1 -- 172.21.15.33:0/130282079 shutdown_connections 2023-11-06T21:03:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.169+0000 7f800e9ff700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ff405f240 0x7f7ff4061700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.169+0000 7f800e9ff700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80080c6240 0x7f80080c23a0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.169+0000 7f800e9ff700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80080aa2d0 0x7f80081517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.169+0000 7f800e9ff700 1 --2- 172.21.15.33:0/130282079 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80080c1840 0x7f80080c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.169+0000 7f800e9ff700 1 -- 172.21.15.33:0/130282079 >> 172.21.15.33:0/130282079 conn(0x7f800805ec80 msgr2=0x7f80080ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.169+0000 7f800e9ff700 1 -- 172.21.15.33:0/130282079 shutdown_connections 2023-11-06T21:03:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.169+0000 7f800e9ff700 1 -- 172.21.15.33:0/130282079 wait complete. 2023-11-06T21:03:15.190 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T21:03:15.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.393+0000 7f0fd53ff700 1 Processor -- start 2023-11-06T21:03:15.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.393+0000 7f0fd53ff700 1 -- start start 2023-11-06T21:03:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.394+0000 7f0fd53ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0fd0064dd0 0x7f0fd00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.394+0000 7f0fd53ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd00c6b60 con 0x7f0fd0154330 2023-11-06T21:03:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.394+0000 7f0fd53ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd00c7700 con 0x7f0fd0154710 2023-11-06T21:03:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.394+0000 7f0fd53ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd00c5cd0 con 0x7f0fd0064dd0 2023-11-06T21:03:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.395+0000 7f0fcf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0fd0064dd0 0x7f0fd00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.395+0000 7f0fceffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0fd0154330 0x7f0fd00c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50282/0 (socket says 172.21.15.33:50282) 2023-11-06T21:03:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.395+0000 7f0fceffd700 1 -- 172.21.15.33:0/4037590574 learned_addr learned my addr 172.21.15.33:0/4037590574 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.395+0000 7f0fcf7fe700 1 -- 172.21.15.33:0/4037590574 >> v1:172.21.15.69:6789/0 conn(0x7f0fd0154710 legacy=0x7f0fd00646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.395+0000 7f0fcf7fe700 1 -- 172.21.15.33:0/4037590574 >> v1:172.21.15.33:6789/0 conn(0x7f0fd0154330 legacy=0x7f0fd00c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.395+0000 7f0fcf7fe700 1 -- 172.21.15.33:0/4037590574 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00c5f50 con 0x7f0fd0064dd0 2023-11-06T21:03:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.395+0000 7f0fcdffb700 1 -- 172.21.15.33:0/4037590574 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830986737 0 0) 0x7f0fd00c6b60 con 0x7f0fd0154330 2023-11-06T21:03:15.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.395+0000 7f0fcf7fe700 1 --2- 172.21.15.33:0/4037590574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0064dd0 0x7f0fd00c57e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f0fc8009e60 tx=0x7f0fc8009780).ready entity=mon.0 client_cookie=5bb9edef2881ba6b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:15.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.396+0000 7f0fcdffb700 1 -- 172.21.15.33:0/4037590574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800e070 con 0x7f0fd0064dd0 2023-11-06T21:03:15.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.396+0000 7f0fcdffb700 1 -- 172.21.15.33:0/4037590574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc800bc30 con 0x7f0fd0064dd0 2023-11-06T21:03:15.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.396+0000 7f0fcdffb700 1 -- 172.21.15.33:0/4037590574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8012410 con 0x7f0fd0064dd0 2023-11-06T21:03:15.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.396+0000 7f0fd53ff700 1 -- 172.21.15.33:0/4037590574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0064dd0 msgr2=0x7f0fd00c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.396+0000 7f0fd53ff700 1 --2- 172.21.15.33:0/4037590574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0064dd0 0x7f0fd00c57e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f0fc8009e60 tx=0x7f0fc8009780).stop 2023-11-06T21:03:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.396+0000 7f0fd53ff700 1 -- 172.21.15.33:0/4037590574 shutdown_connections 2023-11-06T21:03:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.396+0000 7f0fd53ff700 1 --2- 172.21.15.33:0/4037590574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0064dd0 0x7f0fd00c57e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.396+0000 7f0fd53ff700 1 -- 172.21.15.33:0/4037590574 >> 172.21.15.33:0/4037590574 conn(0x7f0fd00cb910 msgr2=0x7f0fd00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.396+0000 7f0fd53ff700 1 -- 172.21.15.33:0/4037590574 shutdown_connections 2023-11-06T21:03:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.396+0000 7f0fd53ff700 1 -- 172.21.15.33:0/4037590574 wait complete. 2023-11-06T21:03:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.398+0000 7f0fd53ff700 1 Processor -- start 2023-11-06T21:03:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.398+0000 7f0fd53ff700 1 -- start start 2023-11-06T21:03:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fd53ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd0064dd0 0x7f0fd0146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fd53ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0154330 0x7f0fd00aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fd53ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd0154710 0x7f0fd0155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fd53ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0157af0 con 0x7f0fd0154330 2023-11-06T21:03:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fd53ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0157290 con 0x7f0fd0154710 2023-11-06T21:03:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fd53ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0147100 con 0x7f0fd0064dd0 2023-11-06T21:03:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fce7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0154330 0x7f0fd00aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fce7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0154330 0x7f0fd00aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50822/0 (socket says 172.21.15.33:50822) 2023-11-06T21:03:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fce7fc700 1 -- 172.21.15.33:0/3155113029 learned_addr learned my addr 172.21.15.33:0/3155113029 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fcf7fe700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd0154710 0x7f0fd0155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fceffd700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd0064dd0 0x7f0fd0146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:15.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fce7fc700 1 -- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd0064dd0 msgr2=0x7f0fd0146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:15.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fce7fc700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd0064dd0 0x7f0fd0146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fce7fc700 1 -- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd0154710 msgr2=0x7f0fd0155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:15.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fce7fc700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd0154710 0x7f0fd0155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fce7fc700 1 -- 172.21.15.33:0/3155113029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8009a20 con 0x7f0fd0154330 2023-11-06T21:03:15.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.399+0000 7f0fceffd700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd0064dd0 0x7f0fd0146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:15.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.400+0000 7f0fce7fc700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0154330 0x7f0fd00aeb60 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7f0fc400c440 tx=0x7f0fc4007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:15.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.400+0000 7f0fb6ffd700 1 -- 172.21.15.33:0/3155113029 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc400ce40 con 0x7f0fd0154330 2023-11-06T21:03:15.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.400+0000 7f0fb6ffd700 1 -- 172.21.15.33:0/3155113029 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc4007020 con 0x7f0fd0154330 2023-11-06T21:03:15.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.400+0000 7f0fb6ffd700 1 -- 172.21.15.33:0/3155113029 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40073c0 con 0x7f0fd0154330 2023-11-06T21:03:15.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.400+0000 7f0fd53ff700 1 -- 172.21.15.33:0/3155113029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd0165450 con 0x7f0fd0154330 2023-11-06T21:03:15.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.400+0000 7f0fd53ff700 1 -- 172.21.15.33:0/3155113029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd0165980 con 0x7f0fd0154330 2023-11-06T21:03:15.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.401+0000 7f0fd53ff700 1 -- 172.21.15.33:0/3155113029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd00c0cd0 con 0x7f0fd0154330 2023-11-06T21:03:15.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.402+0000 7f0fb6ffd700 1 -- 172.21.15.33:0/3155113029 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0fc40068b0 con 0x7f0fd0154330 2023-11-06T21:03:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.403+0000 7f0fb6ffd700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fbc05f270 0x7f0fbc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.403+0000 7f0fb6ffd700 1 -- 172.21.15.33:0/3155113029 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5899+0+0 (secure 0 0 0) 0x7f0fc4079d00 con 0x7f0fd0154330 2023-11-06T21:03:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.403+0000 7f0fceffd700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fbc05f270 0x7f0fbc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.404+0000 7f0fceffd700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fbc05f270 0x7f0fbc061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0fc00011e0 tx=0x7f0fc0009040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.407+0000 7f0fb6ffd700 1 -- 172.21.15.33:0/3155113029 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0fc404ae70 con 0x7f0fd0154330 2023-11-06T21:03:15.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.587+0000 7f0fd53ff700 1 -- 172.21.15.33:0/3155113029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0fd00830b0 con 0x7f0fd0154330 2023-11-06T21:03:15.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.591+0000 7f0fb6ffd700 1 -- 172.21.15.33:0/3155113029 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0fc404e490 con 0x7f0fd0154330 2023-11-06T21:03:15.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.593+0000 7f0fd53ff700 1 -- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fbc05f270 msgr2=0x7f0fbc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:15.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.593+0000 7f0fd53ff700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fbc05f270 0x7f0fbc061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0fc00011e0 tx=0x7f0fc0009040).stop 2023-11-06T21:03:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.593+0000 7f0fd53ff700 1 -- 172.21.15.33:0/3155113029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0154330 msgr2=0x7f0fd00aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.593+0000 7f0fd53ff700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0154330 0x7f0fd00aeb60 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7f0fc400c440 tx=0x7f0fc4007dd0).stop 2023-11-06T21:03:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.593+0000 7f0fd53ff700 1 -- 172.21.15.33:0/3155113029 shutdown_connections 2023-11-06T21:03:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.593+0000 7f0fd53ff700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0fbc05f270 0x7f0fbc061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.593+0000 7f0fd53ff700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0fd0064dd0 0x7f0fd0146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.593+0000 7f0fd53ff700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fd0154710 0x7f0fd0155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.593+0000 7f0fd53ff700 1 --2- 172.21.15.33:0/3155113029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd0154330 0x7f0fd00aeb60 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:15.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.593+0000 7f0fd53ff700 1 -- 172.21.15.33:0/3155113029 >> 172.21.15.33:0/3155113029 conn(0x7f0fd00cb910 msgr2=0x7f0fd00a9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:15.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.594+0000 7f0fd53ff700 1 -- 172.21.15.33:0/3155113029 shutdown_connections 2023-11-06T21:03:15.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.594+0000 7f0fd53ff700 1 -- 172.21.15.33:0/3155113029 wait complete. 2023-11-06T21:03:15.611 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T21:03:15.612 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:03:15.701 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:03:15.803 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T21:03:15.804 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T21:03:15.804 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:03:16.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:15.999+0000 7fc31bdff700 1 Processor -- start 2023-11-06T21:03:16.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.000+0000 7fc31bdff700 1 -- start start 2023-11-06T21:03:16.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc31bdff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc314154280 0x7fc314154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc31bdff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc3140ad6e0 0x7fc3140c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc31bdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140a9f20 0x7fc3140c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc31bdff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3140c5db0 con 0x7fc314154280 2023-11-06T21:03:16.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc31bdff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3140c6930 con 0x7fc3140ad6e0 2023-11-06T21:03:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc31bdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3140c6000 con 0x7fc3140a9f20 2023-11-06T21:03:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc319b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc314154280 0x7fc314154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc319b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc314154280 0x7fc314154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50832/0 (socket says 172.21.15.33:50832) 2023-11-06T21:03:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc319b9b700 1 -- 172.21.15.33:0/3225147894 learned_addr learned my addr 172.21.15.33:0/3225147894 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc31a39c700 1 --2- 172.21.15.33:0/3225147894 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140a9f20 0x7fc3140c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.001+0000 7fc31939a700 1 --2- 172.21.15.33:0/3225147894 >> v2:172.21.15.69:3300/0 conn(0x7fc3140ad6e0 0x7fc3140c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.002+0000 7fc319b9b700 1 -- 172.21.15.33:0/3225147894 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140a9f20 msgr2=0x7fc3140c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:16.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.002+0000 7fc319b9b700 1 --2- 172.21.15.33:0/3225147894 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140a9f20 0x7fc3140c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.002+0000 7fc319b9b700 1 -- 172.21.15.33:0/3225147894 >> v2:172.21.15.69:3300/0 conn(0x7fc3140ad6e0 msgr2=0x7fc3140c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.002+0000 7fc319b9b700 1 --2- 172.21.15.33:0/3225147894 >> v2:172.21.15.69:3300/0 conn(0x7fc3140ad6e0 0x7fc3140c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.002+0000 7fc319b9b700 1 -- 172.21.15.33:0/3225147894 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3140c62e0 con 0x7fc314154280 2023-11-06T21:03:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.002+0000 7fc31939a700 1 --2- 172.21.15.33:0/3225147894 >> v2:172.21.15.69:3300/0 conn(0x7fc3140ad6e0 0x7fc3140c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.002+0000 7fc319b9b700 1 --2- 172.21.15.33:0/3225147894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc314154280 0x7fc314154660 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7fc3140c8910 tx=0x7fc3100090f0).ready entity=mon.0 client_cookie=fba6a6e7059a13a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.002+0000 7fc318b99700 1 -- 172.21.15.33:0/3225147894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31000b520 con 0x7fc314154280 2023-11-06T21:03:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.002+0000 7fc318b99700 1 -- 172.21.15.33:0/3225147894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31000b680 con 0x7fc314154280 2023-11-06T21:03:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.002+0000 7fc318b99700 1 -- 172.21.15.33:0/3225147894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31000ba00 con 0x7fc314154280 2023-11-06T21:03:16.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.003+0000 7fc31bdff700 1 -- 172.21.15.33:0/3225147894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc314154280 msgr2=0x7fc314154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:16.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.003+0000 7fc31bdff700 1 --2- 172.21.15.33:0/3225147894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc314154280 0x7fc314154660 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7fc3140c8910 tx=0x7fc3100090f0).stop 2023-11-06T21:03:16.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.003+0000 7fc31bdff700 1 -- 172.21.15.33:0/3225147894 shutdown_connections 2023-11-06T21:03:16.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.003+0000 7fc31bdff700 1 --2- 172.21.15.33:0/3225147894 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140a9f20 0x7fc3140c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.003+0000 7fc31bdff700 1 --2- 172.21.15.33:0/3225147894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc314154280 0x7fc314154660 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.003+0000 7fc31bdff700 1 --2- 172.21.15.33:0/3225147894 >> v2:172.21.15.69:3300/0 conn(0x7fc3140ad6e0 0x7fc3140c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.003+0000 7fc31bdff700 1 -- 172.21.15.33:0/3225147894 >> 172.21.15.33:0/3225147894 conn(0x7fc31405e1b0 msgr2=0x7fc31405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:16.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.003+0000 7fc31bdff700 1 -- 172.21.15.33:0/3225147894 shutdown_connections 2023-11-06T21:03:16.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.003+0000 7fc31bdff700 1 -- 172.21.15.33:0/3225147894 wait complete. 2023-11-06T21:03:16.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.004+0000 7fc31bdff700 1 Processor -- start 2023-11-06T21:03:16.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.005+0000 7fc31bdff700 1 -- start start 2023-11-06T21:03:16.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.005+0000 7fc31bdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3140a9f20 0x7fc31414ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.005+0000 7fc31bdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140ad6e0 0x7fc3140afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.005+0000 7fc31bdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc314154280 0x7fc3140b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.005+0000 7fc31bdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3140b06f0 con 0x7fc3140a9f20 2023-11-06T21:03:16.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.005+0000 7fc31bdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31414bad0 con 0x7fc314154280 2023-11-06T21:03:16.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.005+0000 7fc31bdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3140c1ec0 con 0x7fc3140ad6e0 2023-11-06T21:03:16.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc319b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3140a9f20 0x7fc31414ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc319b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3140a9f20 0x7fc31414ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50834/0 (socket says 172.21.15.33:50834) 2023-11-06T21:03:16.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc319b9b700 1 -- 172.21.15.33:0/2153249449 learned_addr learned my addr 172.21.15.33:0/2153249449 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:16.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc31939a700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140ad6e0 0x7fc3140afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc31a39c700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc314154280 0x7fc3140b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc319b9b700 1 -- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140ad6e0 msgr2=0x7fc3140afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:16.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc319b9b700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140ad6e0 0x7fc3140afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc319b9b700 1 -- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc314154280 msgr2=0x7fc3140b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:16.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc319b9b700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc314154280 0x7fc3140b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc319b9b700 1 -- 172.21.15.33:0/2153249449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc310009370 con 0x7fc3140a9f20 2023-11-06T21:03:16.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.006+0000 7fc319b9b700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3140a9f20 0x7fc31414ac30 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7fc3140841f0 tx=0x7fc310017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:16.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.007+0000 7fc309ffb700 1 -- 172.21.15.33:0/2153249449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31000be90 con 0x7fc3140a9f20 2023-11-06T21:03:16.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.007+0000 7fc309ffb700 1 -- 172.21.15.33:0/2153249449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31000b7f0 con 0x7fc3140a9f20 2023-11-06T21:03:16.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.007+0000 7fc31bdff700 1 -- 172.21.15.33:0/2153249449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3140c2160 con 0x7fc3140a9f20 2023-11-06T21:03:16.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.007+0000 7fc309ffb700 1 -- 172.21.15.33:0/2153249449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31001b620 con 0x7fc3140a9f20 2023-11-06T21:03:16.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.007+0000 7fc31bdff700 1 -- 172.21.15.33:0/2153249449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3140c2720 con 0x7fc3140a9f20 2023-11-06T21:03:16.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.008+0000 7fc31bdff700 1 -- 172.21.15.33:0/2153249449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3140c65c0 con 0x7fc3140a9f20 2023-11-06T21:03:16.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.009+0000 7fc309ffb700 1 -- 172.21.15.33:0/2153249449 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc31001ba10 con 0x7fc3140a9f20 2023-11-06T21:03:16.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.010+0000 7fc309ffb700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc2f805f270 0x7fc2f8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.010+0000 7fc309ffb700 1 -- 172.21.15.33:0/2153249449 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5899+0+0 (secure 0 0 0) 0x7fc31007b1c0 con 0x7fc3140a9f20 2023-11-06T21:03:16.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.010+0000 7fc31939a700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc2f805f270 0x7fc2f8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.011+0000 7fc31939a700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc2f805f270 0x7fc2f8061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc31414ea90 tx=0x7fc304008040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:16.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.014+0000 7fc309ffb700 1 -- 172.21.15.33:0/2153249449 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc31004c3e0 con 0x7fc3140a9f20 2023-11-06T21:03:16.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.240+0000 7fc31bdff700 1 -- 172.21.15.33:0/2153249449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc31406ab40 con 0x7fc3140a9f20 2023-11-06T21:03:16.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.240+0000 7fc309ffb700 1 -- 172.21.15.33:0/2153249449 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fc310021090 con 0x7fc3140a9f20 2023-11-06T21:03:16.242 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:16.242 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2500558031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2500558031},{"type":"v1","addr":"172.21.15.69:6837","nonce":2500558031}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1012474146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1012474146},{"type":"v1","addr":"172.21.15.69:6836","nonce":1012474146}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4020289695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4020289695},{"type":"v1","addr":"172.21.15.33:6834","nonce":4020289695}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/4197440054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4197440054},{"type":"v1","addr":"172.21.15.33:6835","nonce":4197440054}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[]} 2023-11-06T21:03:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 -- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc2f805f270 msgr2=0x7fc2f8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc2f805f270 0x7fc2f8061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc31414ea90 tx=0x7fc304008040).stop 2023-11-06T21:03:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 -- 172.21.15.33:0/2153249449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3140a9f20 msgr2=0x7fc31414ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3140a9f20 0x7fc31414ac30 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7fc3140841f0 tx=0x7fc310017000).stop 2023-11-06T21:03:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 -- 172.21.15.33:0/2153249449 shutdown_connections 2023-11-06T21:03:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc2f805f270 0x7fc2f8061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc314154280 0x7fc3140b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3140a9f20 0x7fc31414ac30 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 --2- 172.21.15.33:0/2153249449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140ad6e0 0x7fc3140afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 -- 172.21.15.33:0/2153249449 >> 172.21.15.33:0/2153249449 conn(0x7fc31405e1b0 msgr2=0x7fc3141535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:16.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 -- 172.21.15.33:0/2153249449 shutdown_connections 2023-11-06T21:03:16.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.243+0000 7fc31bdff700 1 -- 172.21.15.33:0/2153249449 wait complete. 2023-11-06T21:03:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 55 2023-11-06T21:03:16.261 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T21:03:16.262 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T21:03:16.262 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T21:03:16.263 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:03:16.263 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T21:03:16.263 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T21:03:16.264 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T21:03:16.264 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:03:16.264 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T21:03:16.264 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T21:03:16.265 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T21:03:16.265 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:03:16.265 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T21:03:16.266 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T21:03:16.266 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T21:03:16.266 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:03:16.280 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T21:03:16.281 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T21:03:16.282 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T21:03:16.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T21:03:16.285 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T21:03:16.286 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T21:03:16.287 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T21:03:16.288 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T21:03:16.326 INFO:tasks.ceph.mds.c:Started 2023-11-06T21:03:16.327 DEBUG:teuthology.parallel:result is None 2023-11-06T21:03:16.329 INFO:tasks.ceph.mds.a:Started 2023-11-06T21:03:16.329 DEBUG:teuthology.parallel:result is None 2023-11-06T21:03:16.331 INFO:tasks.ceph.mds.d:Started 2023-11-06T21:03:16.331 DEBUG:teuthology.parallel:result is None 2023-11-06T21:03:16.332 INFO:tasks.ceph.mds.b:Started 2023-11-06T21:03:16.333 DEBUG:teuthology.parallel:result is None 2023-11-06T21:03:16.333 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T21:03:16.475 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T21:03:16.475 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T21:03:16.476 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T21:03:16.477 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T21:03:16.478 INFO:tasks.ceph.mds.d.smithi069.stderr:2023-11-06T21:03:16.478+0000 7fd97b9fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-06T21:03:16.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.552+0000 7fcfc5dff700 1 Processor -- start 2023-11-06T21:03:16.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.552+0000 7fcfc5dff700 1 -- start start 2023-11-06T21:03:16.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.553+0000 7fcfc5dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcfc00c8500 0x7fcfc0062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.553+0000 7fcfc5dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcfc0062c00 0x7fcfc0065380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.553+0000 7fcfc5dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc0069c80 con 0x7fcfc01542d0 2023-11-06T21:03:16.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.553+0000 7fcfc5dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc0069f60 con 0x7fcfc0062c00 2023-11-06T21:03:16.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.553+0000 7fcfc5dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc00bcb10 con 0x7fcfc00c8500 2023-11-06T21:03:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.553+0000 7fcfbf7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcfc01542d0 0x7fcfc00a6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50284/0 (socket says 172.21.15.33:50284) 2023-11-06T21:03:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.553+0000 7fcfbf7fe700 1 -- 172.21.15.33:0/173691639 learned_addr learned my addr 172.21.15.33:0/173691639 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.554+0000 7fcfbffff700 1 --2- 172.21.15.33:0/173691639 >> v2:172.21.15.33:3300/0 conn(0x7fcfc0062c00 0x7fcfc0065380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.554+0000 7fcfbeffd700 1 --2- 172.21.15.33:0/173691639 >> v2:172.21.15.69:3300/0 conn(0x7fcfc00c8500 0x7fcfc0062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.554+0000 7fcfbffff700 1 -- 172.21.15.33:0/173691639 >> v2:172.21.15.69:3300/0 conn(0x7fcfc00c8500 msgr2=0x7fcfc0062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.554+0000 7fcfbffff700 1 --2- 172.21.15.33:0/173691639 >> v2:172.21.15.69:3300/0 conn(0x7fcfc00c8500 0x7fcfc0062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.554+0000 7fcfbffff700 1 -- 172.21.15.33:0/173691639 >> v1:172.21.15.33:6789/0 conn(0x7fcfc01542d0 legacy=0x7fcfc00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.554+0000 7fcfbffff700 1 -- 172.21.15.33:0/173691639 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00bcdf0 con 0x7fcfc0062c00 2023-11-06T21:03:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.554+0000 7fcfbeffd700 1 --2- 172.21.15.33:0/173691639 >> v2:172.21.15.69:3300/0 conn(0x7fcfc00c8500 0x7fcfc0062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.554+0000 7fcfbffff700 1 --2- 172.21.15.33:0/173691639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0062c00 0x7fcfc0065380 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7fcfc00637d0 tx=0x7fcfb8009780).ready entity=mon.0 client_cookie=c0af0e48c6ec9097 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:16.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.554+0000 7fcfbe7fc700 1 -- 172.21.15.33:0/173691639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800e070 con 0x7fcfc0062c00 2023-11-06T21:03:16.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.555+0000 7fcfbe7fc700 1 -- 172.21.15.33:0/173691639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb800bb60 con 0x7fcfc0062c00 2023-11-06T21:03:16.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.555+0000 7fcfbe7fc700 1 -- 172.21.15.33:0/173691639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8012410 con 0x7fcfc0062c00 2023-11-06T21:03:16.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.555+0000 7fcfc5dff700 1 -- 172.21.15.33:0/173691639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0062c00 msgr2=0x7fcfc0065380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:16.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.555+0000 7fcfc5dff700 1 --2- 172.21.15.33:0/173691639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0062c00 0x7fcfc0065380 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7fcfc00637d0 tx=0x7fcfb8009780).stop 2023-11-06T21:03:16.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.555+0000 7fcfc5dff700 1 -- 172.21.15.33:0/173691639 shutdown_connections 2023-11-06T21:03:16.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.555+0000 7fcfc5dff700 1 --2- 172.21.15.33:0/173691639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0062c00 0x7fcfc0065380 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.555+0000 7fcfc5dff700 1 --2- 172.21.15.33:0/173691639 >> v2:172.21.15.69:3300/0 conn(0x7fcfc00c8500 0x7fcfc0062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.555+0000 7fcfc5dff700 1 -- 172.21.15.33:0/173691639 >> 172.21.15.33:0/173691639 conn(0x7fcfc00642a0 msgr2=0x7fcfc00c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:16.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.555+0000 7fcfc5dff700 1 -- 172.21.15.33:0/173691639 shutdown_connections 2023-11-06T21:03:16.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.555+0000 7fcfc5dff700 1 -- 172.21.15.33:0/173691639 wait complete. 2023-11-06T21:03:16.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.557+0000 7fcfc5dff700 1 Processor -- start 2023-11-06T21:03:16.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.557+0000 7fcfc5dff700 1 -- start start 2023-11-06T21:03:16.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfc5dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfc0062c00 0x7fcfc0163cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfc5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc00c8500 0x7fcfc00a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfc5dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc01542d0 0x7fcfc0152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfc5dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc01686d0 con 0x7fcfc01542d0 2023-11-06T21:03:16.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfc5dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc014d4b0 con 0x7fcfc0062c00 2023-11-06T21:03:16.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfc5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc0167ac0 con 0x7fcfc00c8500 2023-11-06T21:03:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfbffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc01542d0 0x7fcfc0152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfbffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc01542d0 0x7fcfc0152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50888/0 (socket says 172.21.15.33:50888) 2023-11-06T21:03:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfbffff700 1 -- 172.21.15.33:0/2776965183 learned_addr learned my addr 172.21.15.33:0/2776965183 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfbf7fe700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfc0062c00 0x7fcfc0163cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfbeffd700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc00c8500 0x7fcfc00a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.558+0000 7fcfbffff700 1 -- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc00c8500 msgr2=0x7fcfc00a7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.559+0000 7fcfbffff700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc00c8500 0x7fcfc00a7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.559+0000 7fcfbffff700 1 -- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfc0062c00 msgr2=0x7fcfc0163cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.559+0000 7fcfbffff700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfc0062c00 0x7fcfc0163cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.559+0000 7fcfbffff700 1 -- 172.21.15.33:0/2776965183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40092a0 con 0x7fcfc01542d0 2023-11-06T21:03:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.559+0000 7fcfbffff700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc01542d0 0x7fcfc0152960 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7fcfb800b6c0 tx=0x7fcfb8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.559+0000 7fcfa77fe700 1 -- 172.21.15.33:0/2776965183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800b6f0 con 0x7fcfc01542d0 2023-11-06T21:03:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.559+0000 7fcfa77fe700 1 -- 172.21.15.33:0/2776965183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb801d630 con 0x7fcfc01542d0 2023-11-06T21:03:16.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.559+0000 7fcfa77fe700 1 -- 172.21.15.33:0/2776965183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb801d9b0 con 0x7fcfc01542d0 2023-11-06T21:03:16.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.559+0000 7fcfc5dff700 1 -- 172.21.15.33:0/2776965183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb8009a20 con 0x7fcfc01542d0 2023-11-06T21:03:16.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.559+0000 7fcfc5dff700 1 -- 172.21.15.33:0/2776965183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc0168080 con 0x7fcfc01542d0 2023-11-06T21:03:16.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.560+0000 7fcfc5dff700 1 -- 172.21.15.33:0/2776965183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc0168840 con 0x7fcfc01542d0 2023-11-06T21:03:16.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.561+0000 7fcfa77fe700 1 -- 172.21.15.33:0/2776965183 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcfb8026410 con 0x7fcfc01542d0 2023-11-06T21:03:16.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.562+0000 7fcfa77fe700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfac05f270 0x7fcfac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:16.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.562+0000 7fcfa77fe700 1 -- 172.21.15.33:0/2776965183 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5871+0+0 (secure 0 0 0) 0x7fcfb807c1d0 con 0x7fcfc01542d0 2023-11-06T21:03:16.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.562+0000 7fcfbf7fe700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfac05f270 0x7fcfac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:16.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.563+0000 7fcfbf7fe700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfac05f270 0x7fcfac061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fcfb0007fd0 tx=0x7fcfb0000bc0).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:16.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.566+0000 7fcfa77fe700 1 -- 172.21.15.33:0/2776965183 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcfb804d300 con 0x7fcfc01542d0 2023-11-06T21:03:16.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:16.761+0000 7fcfc5dff700 1 -- 172.21.15.33:0/2776965183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcfc0150630 con 0x7fcfc01542d0 2023-11-06T21:03:17.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.514+0000 7fcfa77fe700 1 -- 172.21.15.33:0/2776965183 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v143) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcfb8050920 con 0x7fcfc01542d0 2023-11-06T21:03:17.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.516+0000 7fcfc5dff700 1 -- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfac05f270 msgr2=0x7fcfac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.516+0000 7fcfc5dff700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfac05f270 0x7fcfac061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fcfb0007fd0 tx=0x7fcfb0000bc0).stop 2023-11-06T21:03:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.517+0000 7fcfc5dff700 1 -- 172.21.15.33:0/2776965183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc01542d0 msgr2=0x7fcfc0152960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.517+0000 7fcfc5dff700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc01542d0 0x7fcfc0152960 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7fcfb800b6c0 tx=0x7fcfb8009780).stop 2023-11-06T21:03:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.517+0000 7fcfc5dff700 1 -- 172.21.15.33:0/2776965183 shutdown_connections 2023-11-06T21:03:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.517+0000 7fcfc5dff700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcfac05f270 0x7fcfac061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.517+0000 7fcfc5dff700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc01542d0 0x7fcfc0152960 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.517+0000 7fcfc5dff700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfc0062c00 0x7fcfc0163cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.517+0000 7fcfc5dff700 1 --2- 172.21.15.33:0/2776965183 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcfc00c8500 0x7fcfc00a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:17.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.517+0000 7fcfc5dff700 1 -- 172.21.15.33:0/2776965183 >> 172.21.15.33:0/2776965183 conn(0x7fcfc00642a0 msgr2=0x7fcfc00a69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:17.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.517+0000 7fcfc5dff700 1 -- 172.21.15.33:0/2776965183 shutdown_connections 2023-11-06T21:03:17.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.517+0000 7fcfc5dff700 1 -- 172.21.15.33:0/2776965183 wait complete. 2023-11-06T21:03:17.520 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T21:03:17.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T21:03:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.739+0000 7fafcd5ff700 1 Processor -- start 2023-11-06T21:03:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.739+0000 7fafcd5ff700 1 -- start start 2023-11-06T21:03:17.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.740+0000 7fafcd5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fafc80c5d50 0x7fafc80c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:17.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafcd5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fafc80bfd70 0x7fafc80c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:17.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafcd5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc81554a0 con 0x7fafc80c1840 2023-11-06T21:03:17.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafcd5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc8154960 con 0x7fafc80bfd70 2023-11-06T21:03:17.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafcd5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc80c66c0 con 0x7fafc80c5d50 2023-11-06T21:03:17.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafc77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fafc80bfd70 0x7fafc80c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:17.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafc77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fafc80bfd70 0x7fafc80c0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50902/0 (socket says 172.21.15.33:50902) 2023-11-06T21:03:17.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafc77fe700 1 -- 172.21.15.33:0/2737341520 learned_addr learned my addr 172.21.15.33:0/2737341520 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:17.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafc67fc700 1 --2- 172.21.15.33:0/2737341520 >> v2:172.21.15.69:3300/0 conn(0x7fafc80c5d50 0x7fafc80c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:17.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafc77fe700 1 -- 172.21.15.33:0/2737341520 >> v2:172.21.15.69:3300/0 conn(0x7fafc80c5d50 msgr2=0x7fafc80c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:17.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafc77fe700 1 --2- 172.21.15.33:0/2737341520 >> v2:172.21.15.69:3300/0 conn(0x7fafc80c5d50 0x7fafc80c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:17.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafc77fe700 1 -- 172.21.15.33:0/2737341520 >> v1:172.21.15.69:6789/0 conn(0x7fafc80c1840 legacy=0x7fafc80a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:17.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafc77fe700 1 -- 172.21.15.33:0/2737341520 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc8155930 con 0x7fafc80bfd70 2023-11-06T21:03:17.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafc67fc700 1 --2- 172.21.15.33:0/2737341520 >> v2:172.21.15.69:3300/0 conn(0x7fafc80c5d50 0x7fafc80c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:17.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.741+0000 7fafc77fe700 1 --2- 172.21.15.33:0/2737341520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80bfd70 0x7fafc80c0180 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7fafc80adb40 tx=0x7fafc0009840).ready entity=mon.0 client_cookie=874c98834e323a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:17.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafc5ffb700 1 -- 172.21.15.33:0/2737341520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000e070 con 0x7fafc80bfd70 2023-11-06T21:03:17.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafc5ffb700 1 -- 172.21.15.33:0/2737341520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc000ba10 con 0x7fafc80bfd70 2023-11-06T21:03:17.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafc5ffb700 1 -- 172.21.15.33:0/2737341520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000bd90 con 0x7fafc80bfd70 2023-11-06T21:03:17.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2737341520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80bfd70 msgr2=0x7fafc80c0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:17.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafcd5ff700 1 --2- 172.21.15.33:0/2737341520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80bfd70 0x7fafc80c0180 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7fafc80adb40 tx=0x7fafc0009840).stop 2023-11-06T21:03:17.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2737341520 shutdown_connections 2023-11-06T21:03:17.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafcd5ff700 1 --2- 172.21.15.33:0/2737341520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80bfd70 0x7fafc80c0180 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:17.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafcd5ff700 1 --2- 172.21.15.33:0/2737341520 >> v2:172.21.15.69:3300/0 conn(0x7fafc80c5d50 0x7fafc80c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:17.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2737341520 >> 172.21.15.33:0/2737341520 conn(0x7fafc805ec80 msgr2=0x7fafc80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:17.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2737341520 shutdown_connections 2023-11-06T21:03:17.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.742+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2737341520 wait complete. 2023-11-06T21:03:17.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.744+0000 7fafcd5ff700 1 Processor -- start 2023-11-06T21:03:17.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.744+0000 7fafcd5ff700 1 -- start start 2023-11-06T21:03:17.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafcd5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fafc80bfd70 0x7fafc80c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafcd5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80c1840 0x7fafc814cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafcd5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafc80c5d50 0x7fafc815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafcd5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8158370 con 0x7fafc80c1840 2023-11-06T21:03:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafcd5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc80c2030 con 0x7fafc80c5d50 2023-11-06T21:03:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafcd5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc80c22f0 con 0x7fafc80bfd70 2023-11-06T21:03:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafc67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80c1840 0x7fafc814cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafc67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80c1840 0x7fafc814cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50912/0 (socket says 172.21.15.33:50912) 2023-11-06T21:03:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafc67fc700 1 -- 172.21.15.33:0/2480388084 learned_addr learned my addr 172.21.15.33:0/2480388084 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafc77fe700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafc80c5d50 0x7fafc815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.745+0000 7fafc6ffd700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fafc80bfd70 0x7fafc80c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafc67fc700 1 -- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fafc80bfd70 msgr2=0x7fafc80c28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafc67fc700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fafc80bfd70 0x7fafc80c28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafc67fc700 1 -- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafc80c5d50 msgr2=0x7fafc815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafc67fc700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafc80c5d50 0x7fafc815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafc67fc700 1 -- 172.21.15.33:0/2480388084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0009ae0 con 0x7fafc80c1840 2023-11-06T21:03:17.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafc6ffd700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fafc80bfd70 0x7fafc80c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:17.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafc67fc700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80c1840 0x7fafc814cb10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7fafbc00c880 tx=0x7fafbc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:17.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafaeffd700 1 -- 172.21.15.33:0/2480388084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc010580 con 0x7fafc80c1840 2023-11-06T21:03:17.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafaeffd700 1 -- 172.21.15.33:0/2480388084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc0106e0 con 0x7fafc80c1840 2023-11-06T21:03:17.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2480388084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc80c25d0 con 0x7fafc80c1840 2023-11-06T21:03:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafaeffd700 1 -- 172.21.15.33:0/2480388084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc010a60 con 0x7fafc80c1840 2023-11-06T21:03:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.746+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2480388084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc80adda0 con 0x7fafc80c1840 2023-11-06T21:03:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.747+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2480388084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc80c5210 con 0x7fafc80c1840 2023-11-06T21:03:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.748+0000 7fafaeffd700 1 -- 172.21.15.33:0/2480388084 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fafbc01b460 con 0x7fafc80c1840 2023-11-06T21:03:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.749+0000 7fafaeffd700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fafb405f270 0x7fafb4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.749+0000 7fafaeffd700 1 -- 172.21.15.33:0/2480388084 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fafbc07a080 con 0x7fafc80c1840 2023-11-06T21:03:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.749+0000 7fafc6ffd700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fafb405f270 0x7fafb4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.750+0000 7fafc6ffd700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fafb405f270 0x7fafb4061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fafb80026f0 tx=0x7fafb8009040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:17.753+0000 7fafaeffd700 1 -- 172.21.15.33:0/2480388084 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fafbc04b4c0 con 0x7fafc80c1840 2023-11-06T21:03:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.001+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2480388084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fafc806c6c0 con 0x7fafc80c1840 2023-11-06T21:03:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.001+0000 7fafaeffd700 1 -- 172.21.15.33:0/2480388084 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fafbc04eae0 con 0x7fafc80c1840 2023-11-06T21:03:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:18.003 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T21:03:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T21:03:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T21:03:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T21:03:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:03:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:03:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:03:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T21:03:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T21:03:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:03:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:03:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:03:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T21:03:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T21:03:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:03:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:03:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:03:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T21:03:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T21:03:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:03:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:03:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:03:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T21:03:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T21:03:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:03:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:03:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T21:03:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T21:03:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:03:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:03:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T21:03:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T21:03:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:03:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:03:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T21:03:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T21:03:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:03:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:03:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T21:03:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T21:03:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:03:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:03:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T21:03:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T21:03:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:03:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:03:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T21:03:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T21:03:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:03:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:03:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T21:03:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T21:03:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:03:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:03:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T21:03:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T21:03:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:03:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-11-06T21:03:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T21:03:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T21:03:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T21:03:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:03:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:03:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:03:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:03:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T21:03:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T21:03:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:03:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:03:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:03:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:03:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T21:03:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T21:03:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:03:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:03:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:03:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:03:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T21:03:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T21:03:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:03:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:03:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:03:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T21:03:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T21:03:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:03:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T21:03:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T21:03:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T21:03:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:03:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:03:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T21:03:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:03:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T21:03:18.042 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T21:03:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.003+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fafb405f270 msgr2=0x7fafb4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.003+0000 7fafcd5ff700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fafb405f270 0x7fafb4061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fafb80026f0 tx=0x7fafb8009040).stop 2023-11-06T21:03:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.003+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2480388084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80c1840 msgr2=0x7fafc814cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.003+0000 7fafcd5ff700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80c1840 0x7fafc814cb10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7fafbc00c880 tx=0x7fafbc008040).stop 2023-11-06T21:03:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.003+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2480388084 shutdown_connections 2023-11-06T21:03:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.003+0000 7fafcd5ff700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fafb405f270 0x7fafb4061730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.003+0000 7fafcd5ff700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fafc80bfd70 0x7fafc80c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.004+0000 7fafcd5ff700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafc80c5d50 0x7fafc815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.004+0000 7fafcd5ff700 1 --2- 172.21.15.33:0/2480388084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80c1840 0x7fafc814cb10 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.004+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2480388084 >> 172.21.15.33:0/2480388084 conn(0x7fafc805ec80 msgr2=0x7fafc80ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.004+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2480388084 shutdown_connections 2023-11-06T21:03:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.004+0000 7fafcd5ff700 1 -- 172.21.15.33:0/2480388084 wait complete. 2023-11-06T21:03:18.046 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:03:18.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.242+0000 7f81905ff700 1 Processor -- start 2023-11-06T21:03:18.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.242+0000 7f81905ff700 1 -- start start 2023-11-06T21:03:18.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f81905ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f81880bdbe0 0x7f81881584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f81905ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81880a7910 0x7f81880a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f81905ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8188064010 con 0x7f81881580e0 2023-11-06T21:03:18.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f81905ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81880be430 con 0x7f81880a7910 2023-11-06T21:03:18.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f81905ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81880be150 con 0x7f81880bdbe0 2023-11-06T21:03:18.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f818eb9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81880a7910 0x7f81880a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f818eb9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81880a7910 0x7f81880a7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49966/0 (socket says 172.21.15.33:49966) 2023-11-06T21:03:18.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f818eb9c700 1 -- 172.21.15.33:0/1266058266 learned_addr learned my addr 172.21.15.33:0/1266058266 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f818e39b700 1 --1- 172.21.15.33:0/1266058266 >> v1:172.21.15.33:6789/0 conn(0x7f81881580e0 0x7f81880684c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36508/0 (socket says 172.21.15.33:36508) 2023-11-06T21:03:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f818eb9c700 1 -- 172.21.15.33:0/1266058266 >> v2:172.21.15.69:3300/0 conn(0x7f81880bdbe0 msgr2=0x7f81881584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f818db9a700 1 --2- 172.21.15.33:0/1266058266 >> v2:172.21.15.69:3300/0 conn(0x7f81880bdbe0 0x7f81881584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f818eb9c700 1 --2- 172.21.15.33:0/1266058266 >> v2:172.21.15.69:3300/0 conn(0x7f81880bdbe0 0x7f81881584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f818eb9c700 1 -- 172.21.15.33:0/1266058266 >> v1:172.21.15.33:6789/0 conn(0x7f81881580e0 legacy=0x7f81880684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f818eb9c700 1 -- 172.21.15.33:0/1266058266 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880639c0 con 0x7f81880a7910 2023-11-06T21:03:18.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.244+0000 7f818db9a700 1 --2- 172.21.15.33:0/1266058266 >> v2:172.21.15.69:3300/0 conn(0x7f81880bdbe0 0x7f81881584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:18.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.245+0000 7f818eb9c700 1 --2- 172.21.15.33:0/1266058266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880a7910 0x7f81880a7d60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f81880bfd70 tx=0x7f8180009840).ready entity=mon.0 client_cookie=3f5a2950f33528b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:18.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.245+0000 7f818d399700 1 -- 172.21.15.33:0/1266058266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000e070 con 0x7f81880a7910 2023-11-06T21:03:18.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.245+0000 7f818d399700 1 -- 172.21.15.33:0/1266058266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818000ba50 con 0x7f81880a7910 2023-11-06T21:03:18.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.245+0000 7f818d399700 1 -- 172.21.15.33:0/1266058266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000bdd0 con 0x7f81880a7910 2023-11-06T21:03:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.245+0000 7f81905ff700 1 -- 172.21.15.33:0/1266058266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880a7910 msgr2=0x7f81880a7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.245+0000 7f81905ff700 1 --2- 172.21.15.33:0/1266058266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880a7910 0x7f81880a7d60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f81880bfd70 tx=0x7f8180009840).stop 2023-11-06T21:03:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.245+0000 7f81905ff700 1 -- 172.21.15.33:0/1266058266 shutdown_connections 2023-11-06T21:03:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.245+0000 7f81905ff700 1 --2- 172.21.15.33:0/1266058266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880a7910 0x7f81880a7d60 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.245+0000 7f81905ff700 1 --2- 172.21.15.33:0/1266058266 >> v2:172.21.15.69:3300/0 conn(0x7f81880bdbe0 0x7f81881584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.245+0000 7f81905ff700 1 -- 172.21.15.33:0/1266058266 >> 172.21.15.33:0/1266058266 conn(0x7f818805ec80 msgr2=0x7f81880664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.246+0000 7f81905ff700 1 -- 172.21.15.33:0/1266058266 shutdown_connections 2023-11-06T21:03:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.246+0000 7f81905ff700 1 -- 172.21.15.33:0/1266058266 wait complete. 2023-11-06T21:03:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.248+0000 7f81905ff700 1 Processor -- start 2023-11-06T21:03:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.248+0000 7f81905ff700 1 -- start start 2023-11-06T21:03:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.249+0000 7f81905ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81880a7910 0x7f8188132980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.249+0000 7f81905ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880bdbe0 0x7f81880c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.249+0000 7f81905ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81881580e0 0x7f81880c2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.249+0000 7f81905ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81880c2930 con 0x7f81880bdbe0 2023-11-06T21:03:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.249+0000 7f81905ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81880ae4b0 con 0x7f81881580e0 2023-11-06T21:03:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.249+0000 7f81905ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81880ad8a0 con 0x7f81880a7910 2023-11-06T21:03:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.249+0000 7f818db9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880bdbe0 0x7f81880c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f818db9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880bdbe0 0x7f81880c1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49974/0 (socket says 172.21.15.33:49974) 2023-11-06T21:03:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f818db9a700 1 -- 172.21.15.33:0/1571526643 learned_addr learned my addr 172.21.15.33:0/1571526643 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f818eb9c700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81881580e0 0x7f81880c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f818e39b700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81880a7910 0x7f8188132980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f818db9a700 1 -- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81880a7910 msgr2=0x7f8188132980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f818db9a700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81880a7910 0x7f8188132980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f818db9a700 1 -- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81881580e0 msgr2=0x7f81880c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f818db9a700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81881580e0 0x7f81880c2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f818db9a700 1 -- 172.21.15.33:0/1571526643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180009ae0 con 0x7f81880bdbe0 2023-11-06T21:03:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f818db9a700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880bdbe0 0x7f81880c1f50 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f817800c880 tx=0x7f8178008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f817e7fc700 1 -- 172.21.15.33:0/1571526643 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178010580 con 0x7f81880bdbe0 2023-11-06T21:03:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f817e7fc700 1 -- 172.21.15.33:0/1571526643 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81780106e0 con 0x7f81880bdbe0 2023-11-06T21:03:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f817e7fc700 1 -- 172.21.15.33:0/1571526643 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178010a60 con 0x7f81880bdbe0 2023-11-06T21:03:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.250+0000 7f81905ff700 1 -- 172.21.15.33:0/1571526643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81880adb80 con 0x7f81880bdbe0 2023-11-06T21:03:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.251+0000 7f81905ff700 1 -- 172.21.15.33:0/1571526643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81880ae140 con 0x7f81880bdbe0 2023-11-06T21:03:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.252+0000 7f81905ff700 1 -- 172.21.15.33:0/1571526643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81880ae660 con 0x7f81880bdbe0 2023-11-06T21:03:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.252+0000 7f817e7fc700 1 -- 172.21.15.33:0/1571526643 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f817801b460 con 0x7f81880bdbe0 2023-11-06T21:03:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.253+0000 7f817e7fc700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f816c05f240 0x7f816c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.253+0000 7f817e7fc700 1 -- 172.21.15.33:0/1571526643 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f8178079fe0 con 0x7f81880bdbe0 2023-11-06T21:03:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.254+0000 7f818e39b700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f816c05f240 0x7f816c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.254+0000 7f818e39b700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f816c05f240 0x7f816c061700 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f8184007fd0 tx=0x7f8184007d30).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.257+0000 7f817e7fc700 1 -- 172.21.15.33:0/1571526643 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f81880ae660 con 0x7f81880bdbe0 2023-11-06T21:03:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.439+0000 7f81905ff700 1 -- 172.21.15.33:0/1571526643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8188066bf0 con 0x7f81880bdbe0 2023-11-06T21:03:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.440+0000 7f817e7fc700 1 -- 172.21.15.33:0/1571526643 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+13771 (secure 0 0 0) 0x7f817804edd0 con 0x7f81880bdbe0 2023-11-06T21:03:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:18.441 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":143,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:03:17.510769+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:03:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.442+0000 7f81905ff700 1 -- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f816c05f240 msgr2=0x7f816c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.442+0000 7f81905ff700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f816c05f240 0x7f816c061700 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f8184007fd0 tx=0x7f8184007d30).stop 2023-11-06T21:03:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.442+0000 7f81905ff700 1 -- 172.21.15.33:0/1571526643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880bdbe0 msgr2=0x7f81880c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.442+0000 7f81905ff700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880bdbe0 0x7f81880c1f50 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f817800c880 tx=0x7f8178008040).stop 2023-11-06T21:03:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.442+0000 7f81905ff700 1 -- 172.21.15.33:0/1571526643 shutdown_connections 2023-11-06T21:03:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.442+0000 7f81905ff700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f816c05f240 0x7f816c061700 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.442+0000 7f81905ff700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81880a7910 0x7f8188132980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.442+0000 7f81905ff700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81881580e0 0x7f81880c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.443+0000 7f81905ff700 1 --2- 172.21.15.33:0/1571526643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81880bdbe0 0x7f81880c1f50 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.443+0000 7f81905ff700 1 -- 172.21.15.33:0/1571526643 >> 172.21.15.33:0/1571526643 conn(0x7f818805ec80 msgr2=0x7f81881395c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.443+0000 7f81905ff700 1 -- 172.21.15.33:0/1571526643 shutdown_connections 2023-11-06T21:03:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.443+0000 7f81905ff700 1 -- 172.21.15.33:0/1571526643 wait complete. 2023-11-06T21:03:18.461 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T21:03:18.462 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T21:03:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.660+0000 7fe83cbff700 1 Processor -- start 2023-11-06T21:03:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.660+0000 7fe83cbff700 1 -- start start 2023-11-06T21:03:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe83cbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 0x7fe8381520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe83cbff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8381525e0 con 0x7fe8380c3b60 2023-11-06T21:03:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe83cbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe838154f90 con 0x7fe8381342f0 2023-11-06T21:03:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe83cbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83814bc20 con 0x7fe838133f10 2023-11-06T21:03:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe836d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe8380c3b60 0x7fe838069530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36520/0 (socket says 172.21.15.33:36520) 2023-11-06T21:03:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe836d9d700 1 -- 172.21.15.33:0/501880260 learned_addr learned my addr 172.21.15.33:0/501880260 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe83659c700 1 --2- 172.21.15.33:0/501880260 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 0x7fe8381520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe83559a700 1 -- 172.21.15.33:0/501880260 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022059808 0 0) 0x7fe8381525e0 con 0x7fe8380c3b60 2023-11-06T21:03:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe83559a700 1 -- 172.21.15.33:0/501880260 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8200033f0 con 0x7fe8380c3b60 2023-11-06T21:03:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe83559a700 1 -- 172.21.15.33:0/501880260 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55301631 0 0) 0x7fe838154f90 con 0x7fe8381342f0 2023-11-06T21:03:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.662+0000 7fe83559a700 1 -- 172.21.15.33:0/501880260 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8381525e0 con 0x7fe8381342f0 2023-11-06T21:03:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.663+0000 7fe83659c700 1 -- 172.21.15.33:0/501880260 >> v1:172.21.15.69:6789/0 conn(0x7fe8381342f0 legacy=0x7fe8380c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.663+0000 7fe83659c700 1 -- 172.21.15.33:0/501880260 >> v1:172.21.15.33:6789/0 conn(0x7fe8380c3b60 legacy=0x7fe838069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.663+0000 7fe83659c700 1 -- 172.21.15.33:0/501880260 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838155270 con 0x7fe838133f10 2023-11-06T21:03:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.663+0000 7fe83659c700 1 --2- 172.21.15.33:0/501880260 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 0x7fe8381520f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fe828009790 tx=0x7fe828009060).ready entity=mon.2 client_cookie=4c736dabaa7dcb17 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.663+0000 7fe83559a700 1 -- 172.21.15.33:0/501880260 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800f510 con 0x7fe838133f10 2023-11-06T21:03:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.663+0000 7fe83559a700 1 -- 172.21.15.33:0/501880260 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82800f670 con 0x7fe838133f10 2023-11-06T21:03:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.663+0000 7fe83559a700 1 -- 172.21.15.33:0/501880260 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800f9f0 con 0x7fe838133f10 2023-11-06T21:03:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.663+0000 7fe83cbff700 1 -- 172.21.15.33:0/501880260 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 msgr2=0x7fe8381520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.663+0000 7fe83cbff700 1 --2- 172.21.15.33:0/501880260 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 0x7fe8381520f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fe828009790 tx=0x7fe828009060).stop 2023-11-06T21:03:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.664+0000 7fe83cbff700 1 -- 172.21.15.33:0/501880260 shutdown_connections 2023-11-06T21:03:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.664+0000 7fe83cbff700 1 --2- 172.21.15.33:0/501880260 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 0x7fe8381520f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.664+0000 7fe83cbff700 1 -- 172.21.15.33:0/501880260 >> 172.21.15.33:0/501880260 conn(0x7fe838062710 msgr2=0x7fe838062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.664+0000 7fe83cbff700 1 -- 172.21.15.33:0/501880260 shutdown_connections 2023-11-06T21:03:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.664+0000 7fe83cbff700 1 -- 172.21.15.33:0/501880260 wait complete. 2023-11-06T21:03:18.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.665+0000 7fe83cbff700 1 Processor -- start 2023-11-06T21:03:18.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.665+0000 7fe83cbff700 1 -- start start 2023-11-06T21:03:18.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.666+0000 7fe83cbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8380c3b60 0x7fe838132570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.666+0000 7fe83cbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 0x7fe838136e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.666+0000 7fe83cbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8381342f0 0x7fe838153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.666+0000 7fe83cbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380a9a10 con 0x7fe8380c3b60 2023-11-06T21:03:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.666+0000 7fe83cbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380aa2b0 con 0x7fe8381342f0 2023-11-06T21:03:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.666+0000 7fe83cbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe838132a60 con 0x7fe838133f10 2023-11-06T21:03:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.666+0000 7fe83659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8380c3b60 0x7fe838132570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.666+0000 7fe83659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8380c3b60 0x7fe838132570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49982/0 (socket says 172.21.15.33:49982) 2023-11-06T21:03:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.666+0000 7fe83659c700 1 -- 172.21.15.33:0/2311106158 learned_addr learned my addr 172.21.15.33:0/2311106158 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe836d9d700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8381342f0 0x7fe838153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe835d9b700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 0x7fe838136e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe83659c700 1 -- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 msgr2=0x7fe838136e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe83659c700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 0x7fe838136e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe83659c700 1 -- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8381342f0 msgr2=0x7fe838153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe83659c700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8381342f0 0x7fe838153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe83659c700 1 -- 172.21.15.33:0/2311106158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8280092a0 con 0x7fe8380c3b60 2023-11-06T21:03:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe835d9b700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 0x7fe838136e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe83659c700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8380c3b60 0x7fe838132570 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7fe828006d70 tx=0x7fe828017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe8267fc700 1 -- 172.21.15.33:0/2311106158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800f7e0 con 0x7fe8380c3b60 2023-11-06T21:03:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe8267fc700 1 -- 172.21.15.33:0/2311106158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82800f940 con 0x7fe8380c3b60 2023-11-06T21:03:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe83cbff700 1 -- 172.21.15.33:0/2311106158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8381377d0 con 0x7fe8380c3b60 2023-11-06T21:03:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.667+0000 7fe8267fc700 1 -- 172.21.15.33:0/2311106158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82801b660 con 0x7fe8380c3b60 2023-11-06T21:03:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.668+0000 7fe83cbff700 1 -- 172.21.15.33:0/2311106158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8380a9f40 con 0x7fe8380c3b60 2023-11-06T21:03:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.669+0000 7fe83cbff700 1 -- 172.21.15.33:0/2311106158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8380aa590 con 0x7fe8380c3b60 2023-11-06T21:03:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.669+0000 7fe8267fc700 1 -- 172.21.15.33:0/2311106158 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe82801ba50 con 0x7fe8380c3b60 2023-11-06T21:03:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.671+0000 7fe8267fc700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe81805f270 0x7fe818061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.671+0000 7fe8267fc700 1 -- 172.21.15.33:0/2311106158 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fe82807b8b0 con 0x7fe8380c3b60 2023-11-06T21:03:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.671+0000 7fe835d9b700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe81805f270 0x7fe818061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.672+0000 7fe835d9b700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe81805f270 0x7fe818061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fe82c007fd0 tx=0x7fe82c009040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.674+0000 7fe8267fc700 1 -- 172.21.15.33:0/2311106158 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe82804cd10 con 0x7fe8380c3b60 2023-11-06T21:03:18.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.855+0000 7fe83cbff700 1 -- 172.21.15.33:0/2311106158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe8380659f0 con 0x7fe8380c3b60 2023-11-06T21:03:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.855+0000 7fe8267fc700 1 -- 172.21.15.33:0/2311106158 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v143) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe828029020 con 0x7fe8380c3b60 2023-11-06T21:03:18.856 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T21:03:18.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.857+0000 7fe83cbff700 1 -- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe81805f270 msgr2=0x7fe818061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.857+0000 7fe83cbff700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe81805f270 0x7fe818061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fe82c007fd0 tx=0x7fe82c009040).stop 2023-11-06T21:03:18.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.857+0000 7fe83cbff700 1 -- 172.21.15.33:0/2311106158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8380c3b60 msgr2=0x7fe838132570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:18.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.857+0000 7fe83cbff700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8380c3b60 0x7fe838132570 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7fe828006d70 tx=0x7fe828017000).stop 2023-11-06T21:03:18.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.858+0000 7fe83cbff700 1 -- 172.21.15.33:0/2311106158 shutdown_connections 2023-11-06T21:03:18.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.858+0000 7fe83cbff700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe81805f270 0x7fe818061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.858+0000 7fe83cbff700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8381342f0 0x7fe838153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.858+0000 7fe83cbff700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8380c3b60 0x7fe838132570 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.858+0000 7fe83cbff700 1 --2- 172.21.15.33:0/2311106158 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe838133f10 0x7fe838136e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.858+0000 7fe83cbff700 1 -- 172.21.15.33:0/2311106158 >> 172.21.15.33:0/2311106158 conn(0x7fe838062710 msgr2=0x7fe8380abff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.858+0000 7fe83cbff700 1 -- 172.21.15.33:0/2311106158 shutdown_connections 2023-11-06T21:03:18.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:18.858+0000 7fe83cbff700 1 -- 172.21.15.33:0/2311106158 wait complete. 2023-11-06T21:03:18.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T21:03:19.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.074+0000 7f67a47ff700 1 Processor -- start 2023-11-06T21:03:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.074+0000 7f67a47ff700 1 -- start start 2023-11-06T21:03:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.075+0000 7f67a47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f679c154330 0x7f679c0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.075+0000 7f67a47ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f679c0c7ff0 0x7f679c0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.075+0000 7f67a47ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679c0c6b60 con 0x7f679c154710 2023-11-06T21:03:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.075+0000 7f67a47ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679c0c7700 con 0x7f679c0c7ff0 2023-11-06T21:03:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.075+0000 7f67a47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679c0c5cd0 con 0x7f679c154330 2023-11-06T21:03:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.075+0000 7f67a2d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f679c0c7ff0 0x7f679c0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.075+0000 7f67a2d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f679c0c7ff0 0x7f679c0646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49994/0 (socket says 172.21.15.33:49994) 2023-11-06T21:03:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.075+0000 7f67a2d9c700 1 -- 172.21.15.33:0/948464581 learned_addr learned my addr 172.21.15.33:0/948464581 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.076+0000 7f67a259b700 1 --2- 172.21.15.33:0/948464581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f679c154330 0x7f679c0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.076+0000 7f67a2d9c700 1 -- 172.21.15.33:0/948464581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f679c154330 msgr2=0x7f679c0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.076+0000 7f67a2d9c700 1 --2- 172.21.15.33:0/948464581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f679c154330 0x7f679c0c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.076+0000 7f67a2d9c700 1 -- 172.21.15.33:0/948464581 >> v1:172.21.15.69:6789/0 conn(0x7f679c154710 legacy=0x7f679c0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.076+0000 7f67a2d9c700 1 -- 172.21.15.33:0/948464581 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c0c5f50 con 0x7f679c0c7ff0 2023-11-06T21:03:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.076+0000 7f67a2d9c700 1 --2- 172.21.15.33:0/948464581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 0x7f679c0646c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f6794009f20 tx=0x7f6794009840).ready entity=mon.0 client_cookie=5ec2fd8a2c1c9156 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.076+0000 7f67a1599700 1 -- 172.21.15.33:0/948464581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679400e070 con 0x7f679c0c7ff0 2023-11-06T21:03:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.076+0000 7f67a1599700 1 -- 172.21.15.33:0/948464581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679400ba80 con 0x7f679c0c7ff0 2023-11-06T21:03:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.076+0000 7f67a1599700 1 -- 172.21.15.33:0/948464581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679400be00 con 0x7f679c0c7ff0 2023-11-06T21:03:19.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.077+0000 7f67a47ff700 1 -- 172.21.15.33:0/948464581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 msgr2=0x7f679c0646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.077+0000 7f67a47ff700 1 --2- 172.21.15.33:0/948464581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 0x7f679c0646c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f6794009f20 tx=0x7f6794009840).stop 2023-11-06T21:03:19.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.077+0000 7f67a47ff700 1 -- 172.21.15.33:0/948464581 shutdown_connections 2023-11-06T21:03:19.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.077+0000 7f67a47ff700 1 --2- 172.21.15.33:0/948464581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 0x7f679c0646c0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.077+0000 7f67a47ff700 1 --2- 172.21.15.33:0/948464581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f679c154330 0x7f679c0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.077+0000 7f67a47ff700 1 -- 172.21.15.33:0/948464581 >> 172.21.15.33:0/948464581 conn(0x7f679c0cb910 msgr2=0x7f679c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:19.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.077+0000 7f67a47ff700 1 -- 172.21.15.33:0/948464581 shutdown_connections 2023-11-06T21:03:19.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.077+0000 7f67a47ff700 1 -- 172.21.15.33:0/948464581 wait complete. 2023-11-06T21:03:19.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.079+0000 7f67a47ff700 1 Processor -- start 2023-11-06T21:03:19.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.079+0000 7f67a47ff700 1 -- start start 2023-11-06T21:03:19.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a47ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 0x7f679c13f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a47ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f679c154330 0x7f679c0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f679c154710 0x7f679c0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a47ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c0ae7b0 con 0x7f679c0c7ff0 2023-11-06T21:03:19.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a47ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c159910 con 0x7f679c154330 2023-11-06T21:03:19.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679c0c2ff0 con 0x7f679c154710 2023-11-06T21:03:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 0x7f679c13f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 0x7f679c13f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50000/0 (socket says 172.21.15.33:50000) 2023-11-06T21:03:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a259b700 1 -- 172.21.15.33:0/3256947339 learned_addr learned my addr 172.21.15.33:0/3256947339 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a1d9a700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f679c154330 0x7f679c0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a2d9c700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f679c154710 0x7f679c0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a259b700 1 -- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f679c154710 msgr2=0x7f679c0ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a259b700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f679c154710 0x7f679c0ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a259b700 1 -- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f679c154330 msgr2=0x7f679c0addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.080+0000 7f67a259b700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f679c154330 0x7f679c0addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.081+0000 7f67a259b700 1 -- 172.21.15.33:0/3256947339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6794009ae0 con 0x7f679c0c7ff0 2023-11-06T21:03:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.081+0000 7f67a1d9a700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f679c154330 0x7f679c0addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.081+0000 7f67a259b700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 0x7f679c13f1e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f679800d8c0 tx=0x7f6798008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.081+0000 7f67927fc700 1 -- 172.21.15.33:0/3256947339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67980095b0 con 0x7f679c0c7ff0 2023-11-06T21:03:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.081+0000 7f67927fc700 1 -- 172.21.15.33:0/3256947339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6798009710 con 0x7f679c0c7ff0 2023-11-06T21:03:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.081+0000 7f67927fc700 1 -- 172.21.15.33:0/3256947339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798009a90 con 0x7f679c0c7ff0 2023-11-06T21:03:19.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.081+0000 7f67a47ff700 1 -- 172.21.15.33:0/3256947339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c13f6d0 con 0x7f679c0c7ff0 2023-11-06T21:03:19.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.081+0000 7f67a47ff700 1 -- 172.21.15.33:0/3256947339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c0c29a0 con 0x7f679c0c7ff0 2023-11-06T21:03:19.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.082+0000 7f67a47ff700 1 -- 172.21.15.33:0/3256947339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679c0830b0 con 0x7f679c0c7ff0 2023-11-06T21:03:19.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.083+0000 7f67927fc700 1 -- 172.21.15.33:0/3256947339 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6798013480 con 0x7f679c0c7ff0 2023-11-06T21:03:19.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.084+0000 7f67927fc700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f678005f270 0x7f6780061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.084+0000 7f67927fc700 1 -- 172.21.15.33:0/3256947339 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f6798079f80 con 0x7f679c0c7ff0 2023-11-06T21:03:19.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.084+0000 7f67a1d9a700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f678005f270 0x7f6780061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.085+0000 7f67a1d9a700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f678005f270 0x7f6780061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f679c0aa710 tx=0x7f678c009040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:19.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.088+0000 7f67927fc700 1 -- 172.21.15.33:0/3256947339 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f679804b300 con 0x7f679c0c7ff0 2023-11-06T21:03:19.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.266+0000 7f67a47ff700 1 -- 172.21.15.33:0/3256947339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f679c0652d0 con 0x7f679c0c7ff0 2023-11-06T21:03:19.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.267+0000 7f67927fc700 1 -- 172.21.15.33:0/3256947339 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v143) v1 ==== 92+0+10 (secure 0 0 0) 0x7f679804e920 con 0x7f679c0c7ff0 2023-11-06T21:03:19.268 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T21:03:19.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 -- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f678005f270 msgr2=0x7f6780061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f678005f270 0x7f6780061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f679c0aa710 tx=0x7f678c009040).stop 2023-11-06T21:03:19.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 -- 172.21.15.33:0/3256947339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 msgr2=0x7f679c13f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 0x7f679c13f1e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f679800d8c0 tx=0x7f6798008040).stop 2023-11-06T21:03:19.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 -- 172.21.15.33:0/3256947339 shutdown_connections 2023-11-06T21:03:19.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f678005f270 0x7f6780061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f679c154710 0x7f679c0ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f679c0c7ff0 0x7f679c13f1e0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 --2- 172.21.15.33:0/3256947339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f679c154330 0x7f679c0addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 -- 172.21.15.33:0/3256947339 >> 172.21.15.33:0/3256947339 conn(0x7f679c0cb910 msgr2=0x7f679c0ca1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 -- 172.21.15.33:0/3256947339 shutdown_connections 2023-11-06T21:03:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.269+0000 7f67a47ff700 1 -- 172.21.15.33:0/3256947339 wait complete. 2023-11-06T21:03:19.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T21:03:19.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.482+0000 7fe54f1ff700 1 Processor -- start 2023-11-06T21:03:19.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.482+0000 7fe54f1ff700 1 -- start start 2023-11-06T21:03:19.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.484+0000 7fe54f1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe5480c1840 0x7fe5480a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.484+0000 7fe54f1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe5480c6460 0x7fe5480bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.484+0000 7fe54f1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5481554a0 con 0x7fe5480aa2d0 2023-11-06T21:03:19.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.484+0000 7fe54f1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe548154960 con 0x7fe5480c6460 2023-11-06T21:03:19.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.484+0000 7fe54f1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe548155930 con 0x7fe5480c1840 2023-11-06T21:03:19.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.484+0000 7fe54d79c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe5480c6460 0x7fe5480bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.484+0000 7fe54d79c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe5480c6460 0x7fe5480bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50008/0 (socket says 172.21.15.33:50008) 2023-11-06T21:03:19.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.484+0000 7fe54d79c700 1 -- 172.21.15.33:0/1412009330 learned_addr learned my addr 172.21.15.33:0/1412009330 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:19.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.484+0000 7fe54cf9b700 1 --2- 172.21.15.33:0/1412009330 >> v2:172.21.15.69:3300/0 conn(0x7fe5480c1840 0x7fe5480a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.485+0000 7fe54d79c700 1 -- 172.21.15.33:0/1412009330 >> v2:172.21.15.69:3300/0 conn(0x7fe5480c1840 msgr2=0x7fe5480a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:19.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.485+0000 7fe54d79c700 1 --2- 172.21.15.33:0/1412009330 >> v2:172.21.15.69:3300/0 conn(0x7fe5480c1840 0x7fe5480a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.485+0000 7fe54d79c700 1 -- 172.21.15.33:0/1412009330 >> v1:172.21.15.69:6789/0 conn(0x7fe5480aa2d0 legacy=0x7fe5480c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:19.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.485+0000 7fe54d79c700 1 -- 172.21.15.33:0/1412009330 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe548154b70 con 0x7fe5480c6460 2023-11-06T21:03:19.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.485+0000 7fe54d79c700 1 --2- 172.21.15.33:0/1412009330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c6460 0x7fe5480bfd70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fe5480840b0 tx=0x7fe540009840).ready entity=mon.0 client_cookie=92f68996a3b13f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:19.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.485+0000 7fe5477fe700 1 -- 172.21.15.33:0/1412009330 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54000e070 con 0x7fe5480c6460 2023-11-06T21:03:19.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.485+0000 7fe5477fe700 1 -- 172.21.15.33:0/1412009330 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54000ba10 con 0x7fe5480c6460 2023-11-06T21:03:19.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.485+0000 7fe5477fe700 1 -- 172.21.15.33:0/1412009330 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54000bd90 con 0x7fe5480c6460 2023-11-06T21:03:19.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.486+0000 7fe54f1ff700 1 -- 172.21.15.33:0/1412009330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c6460 msgr2=0x7fe5480bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.486+0000 7fe54f1ff700 1 --2- 172.21.15.33:0/1412009330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c6460 0x7fe5480bfd70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fe5480840b0 tx=0x7fe540009840).stop 2023-11-06T21:03:19.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.486+0000 7fe54f1ff700 1 -- 172.21.15.33:0/1412009330 shutdown_connections 2023-11-06T21:03:19.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.486+0000 7fe54f1ff700 1 --2- 172.21.15.33:0/1412009330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c6460 0x7fe5480bfd70 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.486+0000 7fe54f1ff700 1 --2- 172.21.15.33:0/1412009330 >> v2:172.21.15.69:3300/0 conn(0x7fe5480c1840 0x7fe5480a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.486+0000 7fe54f1ff700 1 -- 172.21.15.33:0/1412009330 >> 172.21.15.33:0/1412009330 conn(0x7fe54805ec80 msgr2=0x7fe5480cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.486+0000 7fe54f1ff700 1 -- 172.21.15.33:0/1412009330 shutdown_connections 2023-11-06T21:03:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.486+0000 7fe54f1ff700 1 -- 172.21.15.33:0/1412009330 wait complete. 2023-11-06T21:03:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.487+0000 7fe54f1ff700 1 Processor -- start 2023-11-06T21:03:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.487+0000 7fe54f1ff700 1 -- start start 2023-11-06T21:03:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.488+0000 7fe54f1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5480aa2d0 0x7fe548149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.488+0000 7fe54f1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c1840 0x7fe548153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.488+0000 7fe54f1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5480c6460 0x7fe5481541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.488+0000 7fe54f1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5481546a0 con 0x7fe5480c1840 2023-11-06T21:03:19.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.488+0000 7fe54f1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe548167a30 con 0x7fe5480c6460 2023-11-06T21:03:19.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.488+0000 7fe54f1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54814f9a0 con 0x7fe5480aa2d0 2023-11-06T21:03:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe547fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c1840 0x7fe548153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe547fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c1840 0x7fe548153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50016/0 (socket says 172.21.15.33:50016) 2023-11-06T21:03:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe547fff700 1 -- 172.21.15.33:0/340523236 learned_addr learned my addr 172.21.15.33:0/340523236 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe54cf9b700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5480aa2d0 0x7fe548149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe54d79c700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5480c6460 0x7fe5481541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe547fff700 1 -- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5480aa2d0 msgr2=0x7fe548149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe547fff700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5480aa2d0 0x7fe548149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe547fff700 1 -- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5480c6460 msgr2=0x7fe5481541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe547fff700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5480c6460 0x7fe5481541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe547fff700 1 -- 172.21.15.33:0/340523236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe540009ae0 con 0x7fe5480c1840 2023-11-06T21:03:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe547fff700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c1840 0x7fe548153cc0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fe53400b490 tx=0x7fe534006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.489+0000 7fe544ff9700 1 -- 172.21.15.33:0/340523236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534012450 con 0x7fe5480c1840 2023-11-06T21:03:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.490+0000 7fe544ff9700 1 -- 172.21.15.33:0/340523236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5340125d0 con 0x7fe5480c1840 2023-11-06T21:03:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.490+0000 7fe544ff9700 1 -- 172.21.15.33:0/340523236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534012990 con 0x7fe5480c1840 2023-11-06T21:03:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.490+0000 7fe54f1ff700 1 -- 172.21.15.33:0/340523236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe548167c40 con 0x7fe5480c1840 2023-11-06T21:03:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.490+0000 7fe54f1ff700 1 -- 172.21.15.33:0/340523236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe548168200 con 0x7fe5480c1840 2023-11-06T21:03:19.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.491+0000 7fe54f1ff700 1 -- 172.21.15.33:0/340523236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5481683f0 con 0x7fe5480c1840 2023-11-06T21:03:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.491+0000 7fe544ff9700 1 -- 172.21.15.33:0/340523236 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe53401c3f0 con 0x7fe5480c1840 2023-11-06T21:03:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.493+0000 7fe544ff9700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe52c05f270 0x7fe52c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.493+0000 7fe544ff9700 1 -- 172.21.15.33:0/340523236 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fe534079a80 con 0x7fe5480c1840 2023-11-06T21:03:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.493+0000 7fe54cf9b700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe52c05f270 0x7fe52c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.494+0000 7fe54cf9b700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe52c05f270 0x7fe52c061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fe54814e6a0 tx=0x7fe53c008040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.496+0000 7fe544ff9700 1 -- 172.21.15.33:0/340523236 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe53404ae00 con 0x7fe5480c1840 2023-11-06T21:03:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.681+0000 7fe54f1ff700 1 -- 172.21.15.33:0/340523236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe54806c6c0 con 0x7fe5480c1840 2023-11-06T21:03:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.682+0000 7fe544ff9700 1 -- 172.21.15.33:0/340523236 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v143) v1 ==== 84+0+143 (secure 0 0 0) 0x7fe53404e420 con 0x7fe5480c1840 2023-11-06T21:03:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:19.683 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T21:03:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T21:03:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T21:03:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:03:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T21:03:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T21:03:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:19.685 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T21:03:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 -- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe52c05f270 msgr2=0x7fe52c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe52c05f270 0x7fe52c061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fe54814e6a0 tx=0x7fe53c008040).stop 2023-11-06T21:03:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 -- 172.21.15.33:0/340523236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c1840 msgr2=0x7fe548153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c1840 0x7fe548153cc0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fe53400b490 tx=0x7fe534006ef0).stop 2023-11-06T21:03:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 -- 172.21.15.33:0/340523236 shutdown_connections 2023-11-06T21:03:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe52c05f270 0x7fe52c061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5480aa2d0 0x7fe548149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5480c6460 0x7fe5481541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 --2- 172.21.15.33:0/340523236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5480c1840 0x7fe548153cc0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 -- 172.21.15.33:0/340523236 >> 172.21.15.33:0/340523236 conn(0x7fe54805ec80 msgr2=0x7fe548151bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 -- 172.21.15.33:0/340523236 shutdown_connections 2023-11-06T21:03:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.684+0000 7fe54f1ff700 1 -- 172.21.15.33:0/340523236 wait complete. 2023-11-06T21:03:19.702 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T21:03:19.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T21:03:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.903+0000 7f1b4e9ff700 1 Processor -- start 2023-11-06T21:03:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.903+0000 7f1b4e9ff700 1 -- start start 2023-11-06T21:03:19.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.905+0000 7f1b4e9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1b480638b0 0x7f1b480ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.905+0000 7f1b4e9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1b481550d0 0x7f1b480c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.905+0000 7f1b4e9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b481586a0 con 0x7f1b480acfc0 2023-11-06T21:03:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.905+0000 7f1b4e9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480c6590 con 0x7f1b481550d0 2023-11-06T21:03:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.905+0000 7f1b4e9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b48155580 con 0x7f1b480638b0 2023-11-06T21:03:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.905+0000 7f1b4cf9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1b481550d0 0x7f1b480c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.905+0000 7f1b4cf9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1b481550d0 0x7f1b480c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50032/0 (socket says 172.21.15.33:50032) 2023-11-06T21:03:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.905+0000 7f1b4cf9c700 1 -- 172.21.15.33:0/1209379509 learned_addr learned my addr 172.21.15.33:0/1209379509 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.905+0000 7f1b477fe700 1 --2- 172.21.15.33:0/1209379509 >> v2:172.21.15.69:3300/0 conn(0x7f1b480638b0 0x7f1b480ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.906+0000 7f1b4cf9c700 1 -- 172.21.15.33:0/1209379509 >> v2:172.21.15.69:3300/0 conn(0x7f1b480638b0 msgr2=0x7f1b480ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.906+0000 7f1b4cf9c700 1 --2- 172.21.15.33:0/1209379509 >> v2:172.21.15.69:3300/0 conn(0x7f1b480638b0 0x7f1b480ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.906+0000 7f1b4cf9c700 1 -- 172.21.15.33:0/1209379509 >> v1:172.21.15.69:6789/0 conn(0x7f1b480acfc0 legacy=0x7f1b480631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.906+0000 7f1b4cf9c700 1 -- 172.21.15.33:0/1209379509 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b481544c0 con 0x7f1b481550d0 2023-11-06T21:03:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.906+0000 7f1b477fe700 1 --2- 172.21.15.33:0/1209379509 >> v2:172.21.15.69:3300/0 conn(0x7f1b480638b0 0x7f1b480ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.906+0000 7f1b46ffd700 1 -- 172.21.15.33:0/1209379509 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808527661 0 0) 0x7f1b481586a0 con 0x7f1b480acfc0 2023-11-06T21:03:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.906+0000 7f1b4cf9c700 1 --2- 172.21.15.33:0/1209379509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b481550d0 0x7f1b480c5e20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f1b40009f20 tx=0x7f1b40009840).ready entity=mon.0 client_cookie=5ef9ecf55881877c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.906+0000 7f1b46ffd700 1 -- 172.21.15.33:0/1209379509 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000e070 con 0x7f1b481550d0 2023-11-06T21:03:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.906+0000 7f1b46ffd700 1 -- 172.21.15.33:0/1209379509 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4000b990 con 0x7f1b481550d0 2023-11-06T21:03:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.906+0000 7f1b46ffd700 1 -- 172.21.15.33:0/1209379509 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000bd10 con 0x7f1b481550d0 2023-11-06T21:03:19.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.907+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/1209379509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b481550d0 msgr2=0x7f1b480c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.907+0000 7f1b4e9ff700 1 --2- 172.21.15.33:0/1209379509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b481550d0 0x7f1b480c5e20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f1b40009f20 tx=0x7f1b40009840).stop 2023-11-06T21:03:19.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.907+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/1209379509 shutdown_connections 2023-11-06T21:03:19.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.907+0000 7f1b4e9ff700 1 --2- 172.21.15.33:0/1209379509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b481550d0 0x7f1b480c5e20 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.907+0000 7f1b4e9ff700 1 --2- 172.21.15.33:0/1209379509 >> v2:172.21.15.69:3300/0 conn(0x7f1b480638b0 0x7f1b480ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.907+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/1209379509 >> 172.21.15.33:0/1209379509 conn(0x7f1b4805ec80 msgr2=0x7f1b480cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:19.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.907+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/1209379509 shutdown_connections 2023-11-06T21:03:19.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.907+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/1209379509 wait complete. 2023-11-06T21:03:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.908+0000 7f1b4e9ff700 1 Processor -- start 2023-11-06T21:03:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.909+0000 7f1b4e9ff700 1 -- start start 2023-11-06T21:03:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.909+0000 7f1b4e9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b480638b0 0x7f1b48164290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.909+0000 7f1b4e9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b480acfc0 0x7f1b481538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.909+0000 7f1b4e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b481550d0 0x7f1b481678a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.909+0000 7f1b4e9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48168820 con 0x7f1b480acfc0 2023-11-06T21:03:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.909+0000 7f1b4e9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480c2350 con 0x7f1b480638b0 2023-11-06T21:03:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.909+0000 7f1b4e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48168260 con 0x7f1b481550d0 2023-11-06T21:03:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b480acfc0 0x7f1b481538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b480acfc0 0x7f1b481538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50042/0 (socket says 172.21.15.33:50042) 2023-11-06T21:03:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b477fe700 1 -- 172.21.15.33:0/2764241215 learned_addr learned my addr 172.21.15.33:0/2764241215 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b4cf9c700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b481550d0 0x7f1b481678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b47fff700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b480638b0 0x7f1b48164290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b477fe700 1 -- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b481550d0 msgr2=0x7f1b481678a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b477fe700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b481550d0 0x7f1b481678a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b477fe700 1 -- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b480638b0 msgr2=0x7f1b48164290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b477fe700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b480638b0 0x7f1b48164290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b477fe700 1 -- 172.21.15.33:0/2764241215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c0092a0 con 0x7f1b480acfc0 2023-11-06T21:03:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.910+0000 7f1b477fe700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b480acfc0 0x7f1b481538a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f1b3c00e9d0 tx=0x7f1b3c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.911+0000 7f1b2ffff700 1 -- 172.21.15.33:0/2764241215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00c6a0 con 0x7f1b480acfc0 2023-11-06T21:03:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.911+0000 7f1b2ffff700 1 -- 172.21.15.33:0/2764241215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c00c800 con 0x7f1b480acfc0 2023-11-06T21:03:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.911+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/2764241215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b40009ae0 con 0x7f1b480acfc0 2023-11-06T21:03:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.911+0000 7f1b2ffff700 1 -- 172.21.15.33:0/2764241215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00cb80 con 0x7f1b480acfc0 2023-11-06T21:03:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.911+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/2764241215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b480c1fe0 con 0x7f1b480acfc0 2023-11-06T21:03:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.912+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/2764241215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b480c24e0 con 0x7f1b480acfc0 2023-11-06T21:03:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.913+0000 7f1b2ffff700 1 -- 172.21.15.33:0/2764241215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1b3c01b400 con 0x7f1b480acfc0 2023-11-06T21:03:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.914+0000 7f1b2ffff700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b3405f270 0x7f1b34061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.914+0000 7f1b2ffff700 1 -- 172.21.15.33:0/2764241215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f1b3c07a640 con 0x7f1b480acfc0 2023-11-06T21:03:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.914+0000 7f1b47fff700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b3405f270 0x7f1b34061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.915+0000 7f1b47fff700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b3405f270 0x7f1b34061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f1b38007fd0 tx=0x7f1b38007bb0).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:19.918+0000 7f1b2ffff700 1 -- 172.21.15.33:0/2764241215 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1b3c04ba70 con 0x7f1b480acfc0 2023-11-06T21:03:20.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.098+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/2764241215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f1b4806c6c0 con 0x7f1b480acfc0 2023-11-06T21:03:20.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.186+0000 7f1b2ffff700 1 -- 172.21.15.33:0/2764241215 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v144) v1 ==== 152+0+0 (secure 0 0 0) 0x7f1b3c016020 con 0x7f1b480acfc0 2023-11-06T21:03:20.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b3405f270 msgr2=0x7f1b34061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:20.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b3405f270 0x7f1b34061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f1b38007fd0 tx=0x7f1b38007bb0).stop 2023-11-06T21:03:20.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/2764241215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b480acfc0 msgr2=0x7f1b481538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:20.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b480acfc0 0x7f1b481538a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f1b3c00e9d0 tx=0x7f1b3c009060).stop 2023-11-06T21:03:20.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/2764241215 shutdown_connections 2023-11-06T21:03:20.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1b3405f270 0x7f1b34061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b481550d0 0x7f1b481678a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b480638b0 0x7f1b48164290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 --2- 172.21.15.33:0/2764241215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b480acfc0 0x7f1b481538a0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/2764241215 >> 172.21.15.33:0/2764241215 conn(0x7f1b4805ec80 msgr2=0x7f1b480c0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:20.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.189+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/2764241215 shutdown_connections 2023-11-06T21:03:20.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.190+0000 7f1b4e9ff700 1 -- 172.21.15.33:0/2764241215 wait complete. 2023-11-06T21:03:20.199 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T21:03:20.215 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T21:03:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.418+0000 7f6aaefff700 1 Processor -- start 2023-11-06T21:03:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.419+0000 7f6aaefff700 1 -- start start 2023-11-06T21:03:20.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.420+0000 7f6aaefff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6aa8154330 0x7f6aa80c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:20.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.420+0000 7f6aaefff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6aa8154710 0x7f6aa80c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:20.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.420+0000 7f6aaefff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aa80c7ff0 0x7f6aa80646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:20.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.420+0000 7f6aaefff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa80c6b60 con 0x7f6aa8154710 2023-11-06T21:03:20.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.420+0000 7f6aaefff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa80c5cd0 con 0x7f6aa8154330 2023-11-06T21:03:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.420+0000 7f6aaefff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80c5f50 con 0x7f6aa80c7ff0 2023-11-06T21:03:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.420+0000 7f6aa7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6aa8154710 0x7f6aa80c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:20.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.420+0000 7f6aa7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6aa8154710 0x7f6aa80c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50046/0 (socket says 172.21.15.33:50046) 2023-11-06T21:03:20.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.420+0000 7f6aa7fff700 1 -- 172.21.15.33:0/2516715401 learned_addr learned my addr 172.21.15.33:0/2516715401 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:20.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aacd9b700 1 --2- 172.21.15.33:0/2516715401 >> v2:172.21.15.69:3300/0 conn(0x7f6aa8154330 0x7f6aa80c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:20.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aad59c700 1 --2- 172.21.15.33:0/2516715401 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aa80c7ff0 0x7f6aa80646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:20.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aa7fff700 1 -- 172.21.15.33:0/2516715401 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aa80c7ff0 msgr2=0x7f6aa80646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aa7fff700 1 --2- 172.21.15.33:0/2516715401 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aa80c7ff0 0x7f6aa80646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aa7fff700 1 -- 172.21.15.33:0/2516715401 >> v2:172.21.15.69:3300/0 conn(0x7f6aa8154330 msgr2=0x7f6aa80c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aa7fff700 1 --2- 172.21.15.33:0/2516715401 >> v2:172.21.15.69:3300/0 conn(0x7f6aa8154330 0x7f6aa80c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aa7fff700 1 -- 172.21.15.33:0/2516715401 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa80c6230 con 0x7f6aa8154710 2023-11-06T21:03:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aa7fff700 1 --2- 172.21.15.33:0/2516715401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 0x7f6aa80c6ff0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f6aa80605d0 tx=0x7f6a940090f0).ready entity=mon.0 client_cookie=987f96b0df008a22 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aa77fe700 1 -- 172.21.15.33:0/2516715401 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400b600 con 0x7f6aa8154710 2023-11-06T21:03:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aa77fe700 1 -- 172.21.15.33:0/2516715401 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9400b760 con 0x7f6aa8154710 2023-11-06T21:03:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.421+0000 7f6aa77fe700 1 -- 172.21.15.33:0/2516715401 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400bae0 con 0x7f6aa8154710 2023-11-06T21:03:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.422+0000 7f6aaefff700 1 -- 172.21.15.33:0/2516715401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 msgr2=0x7f6aa80c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.422+0000 7f6aaefff700 1 --2- 172.21.15.33:0/2516715401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 0x7f6aa80c6ff0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f6aa80605d0 tx=0x7f6a940090f0).stop 2023-11-06T21:03:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.422+0000 7f6aaefff700 1 -- 172.21.15.33:0/2516715401 shutdown_connections 2023-11-06T21:03:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.422+0000 7f6aaefff700 1 --2- 172.21.15.33:0/2516715401 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aa80c7ff0 0x7f6aa80646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.422+0000 7f6aaefff700 1 --2- 172.21.15.33:0/2516715401 >> v2:172.21.15.69:3300/0 conn(0x7f6aa8154330 0x7f6aa80c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.422+0000 7f6aaefff700 1 --2- 172.21.15.33:0/2516715401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 0x7f6aa80c6ff0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.422+0000 7f6aaefff700 1 -- 172.21.15.33:0/2516715401 >> 172.21.15.33:0/2516715401 conn(0x7f6aa80cb910 msgr2=0x7f6aa80c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.422+0000 7f6aaefff700 1 -- 172.21.15.33:0/2516715401 shutdown_connections 2023-11-06T21:03:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.422+0000 7f6aaefff700 1 -- 172.21.15.33:0/2516715401 wait complete. 2023-11-06T21:03:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.423+0000 7f6aaefff700 1 Processor -- start 2023-11-06T21:03:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.424+0000 7f6aaefff700 1 -- start start 2023-11-06T21:03:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.424+0000 7f6aaefff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa80c7ff0 0x7f6aa8157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aaefff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aa8154330 0x7f6aa80c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aaefff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 0x7f6aa80630a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aaefff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80ae2a0 con 0x7f6aa8154710 2023-11-06T21:03:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aaefff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80c4580 con 0x7f6aa80c7ff0 2023-11-06T21:03:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aaefff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80adf50 con 0x7f6aa8154330 2023-11-06T21:03:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aad59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 0x7f6aa80630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aad59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 0x7f6aa80630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50054/0 (socket says 172.21.15.33:50054) 2023-11-06T21:03:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aad59c700 1 -- 172.21.15.33:0/4154514583 learned_addr learned my addr 172.21.15.33:0/4154514583 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aacd9b700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa80c7ff0 0x7f6aa8157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aa7fff700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aa8154330 0x7f6aa80c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aad59c700 1 -- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aa8154330 msgr2=0x7f6aa80c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aad59c700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aa8154330 0x7f6aa80c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aad59c700 1 -- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa80c7ff0 msgr2=0x7f6aa8157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aad59c700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa80c7ff0 0x7f6aa8157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.425+0000 7f6aad59c700 1 -- 172.21.15.33:0/4154514583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a94009370 con 0x7f6aa8154710 2023-11-06T21:03:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.426+0000 7f6aad59c700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 0x7f6aa80630a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f6aa0009fa0 tx=0x7f6aa0009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.426+0000 7f6aa4ff9700 1 -- 172.21.15.33:0/4154514583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000ec50 con 0x7f6aa8154710 2023-11-06T21:03:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.426+0000 7f6aa4ff9700 1 -- 172.21.15.33:0/4154514583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa000edb0 con 0x7f6aa8154710 2023-11-06T21:03:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.426+0000 7f6aaefff700 1 -- 172.21.15.33:0/4154514583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa80c4830 con 0x7f6aa8154710 2023-11-06T21:03:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.426+0000 7f6aa4ff9700 1 -- 172.21.15.33:0/4154514583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0014610 con 0x7f6aa8154710 2023-11-06T21:03:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.426+0000 7f6aaefff700 1 -- 172.21.15.33:0/4154514583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa8149f70 con 0x7f6aa8154710 2023-11-06T21:03:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.427+0000 7f6aaefff700 1 -- 172.21.15.33:0/4154514583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa80c4d50 con 0x7f6aa8154710 2023-11-06T21:03:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.428+0000 7f6aa4ff9700 1 -- 172.21.15.33:0/4154514583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6aa001d3b0 con 0x7f6aa8154710 2023-11-06T21:03:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.429+0000 7f6aa4ff9700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a8c05f270 0x7f6a8c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.429+0000 7f6aa4ff9700 1 -- 172.21.15.33:0/4154514583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f6aa007a7e0 con 0x7f6aa8154710 2023-11-06T21:03:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.429+0000 7f6aacd9b700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a8c05f270 0x7f6a8c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.430+0000 7f6aacd9b700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a8c05f270 0x7f6a8c061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f6a9c00bbc0 tx=0x7f6a9c008040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.433+0000 7f6aa4ff9700 1 -- 172.21.15.33:0/4154514583 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6aa004b9f0 con 0x7f6aa8154710 2023-11-06T21:03:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:20.614+0000 7f6aaefff700 1 -- 172.21.15.33:0/4154514583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6aa80ae3e0 con 0x7f6aa8154710 2023-11-06T21:03:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.197+0000 7f6aa4ff9700 1 -- 172.21.15.33:0/4154514583 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v145) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6aa004f010 con 0x7f6aa8154710 2023-11-06T21:03:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 -- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a8c05f270 msgr2=0x7f6a8c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a8c05f270 0x7f6a8c061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f6a9c00bbc0 tx=0x7f6a9c008040).stop 2023-11-06T21:03:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 -- 172.21.15.33:0/4154514583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 msgr2=0x7f6aa80630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 0x7f6aa80630a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f6aa0009fa0 tx=0x7f6aa0009900).stop 2023-11-06T21:03:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 -- 172.21.15.33:0/4154514583 shutdown_connections 2023-11-06T21:03:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6a8c05f270 0x7f6a8c061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa8154710 0x7f6aa80630a0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6aa80c7ff0 0x7f6aa8157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 --2- 172.21.15.33:0/4154514583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6aa8154330 0x7f6aa80c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 -- 172.21.15.33:0/4154514583 >> 172.21.15.33:0/4154514583 conn(0x7f6aa80cb910 msgr2=0x7f6aa80ab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 -- 172.21.15.33:0/4154514583 shutdown_connections 2023-11-06T21:03:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.202+0000 7f6aaefff700 1 -- 172.21.15.33:0/4154514583 wait complete. 2023-11-06T21:03:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T21:03:21.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.441+0000 7f8687bff700 1 Processor -- start 2023-11-06T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.441+0000 7f8687bff700 1 -- start start 2023-11-06T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.442+0000 7f8687bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8680154330 0x7f86800c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.442+0000 7f8687bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8680154710 0x7f86800c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.442+0000 7f8687bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86800c6b60 con 0x7f86800c7ff0 2023-11-06T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.442+0000 7f8687bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86800c5cd0 con 0x7f8680154330 2023-11-06T21:03:21.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.442+0000 7f8687bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86800c5f50 con 0x7f8680154710 2023-11-06T21:03:21.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.442+0000 7f868599b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8680154330 0x7f86800c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.442+0000 7f868599b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8680154330 0x7f86800c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50060/0 (socket says 172.21.15.33:50060) 2023-11-06T21:03:21.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.443+0000 7f868599b700 1 -- 172.21.15.33:0/3093560269 learned_addr learned my addr 172.21.15.33:0/3093560269 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:21.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.442+0000 7f868619c700 1 --1- 172.21.15.33:0/3093560269 >> v1:172.21.15.33:6789/0 conn(0x7f86800c7ff0 0x7f86800646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36530/0 (socket says 172.21.15.33:36530) 2023-11-06T21:03:21.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.443+0000 7f868519a700 1 --2- 172.21.15.33:0/3093560269 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8680154710 0x7f86800c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.443+0000 7f868599b700 1 -- 172.21.15.33:0/3093560269 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8680154710 msgr2=0x7f86800c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.443+0000 7f868599b700 1 --2- 172.21.15.33:0/3093560269 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8680154710 0x7f86800c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.443+0000 7f868599b700 1 -- 172.21.15.33:0/3093560269 >> v1:172.21.15.33:6789/0 conn(0x7f86800c7ff0 legacy=0x7f86800646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.443+0000 7f868599b700 1 -- 172.21.15.33:0/3093560269 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800c6230 con 0x7f8680154330 2023-11-06T21:03:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.443+0000 7f868599b700 1 --2- 172.21.15.33:0/3093560269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8680154330 0x7f86800c57e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f867c006fd0 tx=0x7f867c0090f0).ready entity=mon.0 client_cookie=6841e8ba5d2626f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.443+0000 7f8684999700 1 -- 172.21.15.33:0/3093560269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00b580 con 0x7f8680154330 2023-11-06T21:03:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.444+0000 7f8684999700 1 -- 172.21.15.33:0/3093560269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c00b6e0 con 0x7f8680154330 2023-11-06T21:03:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.444+0000 7f8684999700 1 -- 172.21.15.33:0/3093560269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00ba60 con 0x7f8680154330 2023-11-06T21:03:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.444+0000 7f8687bff700 1 -- 172.21.15.33:0/3093560269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8680154330 msgr2=0x7f86800c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.444+0000 7f8687bff700 1 --2- 172.21.15.33:0/3093560269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8680154330 0x7f86800c57e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f867c006fd0 tx=0x7f867c0090f0).stop 2023-11-06T21:03:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.444+0000 7f8687bff700 1 -- 172.21.15.33:0/3093560269 shutdown_connections 2023-11-06T21:03:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.444+0000 7f8687bff700 1 --2- 172.21.15.33:0/3093560269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8680154330 0x7f86800c57e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.444+0000 7f8687bff700 1 --2- 172.21.15.33:0/3093560269 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8680154710 0x7f86800c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.444+0000 7f8687bff700 1 -- 172.21.15.33:0/3093560269 >> 172.21.15.33:0/3093560269 conn(0x7f86800cb910 msgr2=0x7f86800c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.444+0000 7f8687bff700 1 -- 172.21.15.33:0/3093560269 shutdown_connections 2023-11-06T21:03:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.444+0000 7f8687bff700 1 -- 172.21.15.33:0/3093560269 wait complete. 2023-11-06T21:03:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.446+0000 7f8687bff700 1 Processor -- start 2023-11-06T21:03:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.447+0000 7f8687bff700 1 -- start start 2023-11-06T21:03:21.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f8687bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86800c7ff0 0x7f868014fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f8687bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680154330 0x7f868014ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f8687bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8680154710 0x7f86800aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f8687bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680150250 con 0x7f86800c7ff0 2023-11-06T21:03:21.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f8687bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800ade70 con 0x7f8680154330 2023-11-06T21:03:21.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f8687bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8680150440 con 0x7f8680154710 2023-11-06T21:03:21.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f868599b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86800c7ff0 0x7f868014fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f868599b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86800c7ff0 0x7f868014fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50076/0 (socket says 172.21.15.33:50076) 2023-11-06T21:03:21.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f868599b700 1 -- 172.21.15.33:0/3790474804 learned_addr learned my addr 172.21.15.33:0/3790474804 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:21.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f868519a700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680154330 0x7f868014ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f868619c700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8680154710 0x7f86800aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f868599b700 1 -- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8680154710 msgr2=0x7f86800aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f868599b700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8680154710 0x7f86800aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f868599b700 1 -- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680154330 msgr2=0x7f868014ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.448+0000 7f868599b700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680154330 0x7f868014ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.449+0000 7f868599b700 1 -- 172.21.15.33:0/3790474804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c009370 con 0x7f86800c7ff0 2023-11-06T21:03:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.449+0000 7f868599b700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86800c7ff0 0x7f868014fd60 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f867c009fd0 tx=0x7f867c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.449+0000 7f8675ffb700 1 -- 172.21.15.33:0/3790474804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00bea0 con 0x7f86800c7ff0 2023-11-06T21:03:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.449+0000 7f8675ffb700 1 -- 172.21.15.33:0/3790474804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c00b850 con 0x7f86800c7ff0 2023-11-06T21:03:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.449+0000 7f8687bff700 1 -- 172.21.15.33:0/3790474804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8680150720 con 0x7f86800c7ff0 2023-11-06T21:03:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.449+0000 7f8675ffb700 1 -- 172.21.15.33:0/3790474804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c01b660 con 0x7f86800c7ff0 2023-11-06T21:03:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.449+0000 7f8687bff700 1 -- 172.21.15.33:0/3790474804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86800ca210 con 0x7f86800c7ff0 2023-11-06T21:03:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.450+0000 7f8687bff700 1 -- 172.21.15.33:0/3790474804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86800ae170 con 0x7f86800c7ff0 2023-11-06T21:03:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.451+0000 7f8675ffb700 1 -- 172.21.15.33:0/3790474804 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f867c01b9d0 con 0x7f86800c7ff0 2023-11-06T21:03:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.452+0000 7f8675ffb700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f866405f270 0x7f8664061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.452+0000 7f8675ffb700 1 -- 172.21.15.33:0/3790474804 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f867c07bcf0 con 0x7f86800c7ff0 2023-11-06T21:03:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.453+0000 7f868519a700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f866405f270 0x7f8664061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.453+0000 7f868519a700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f866405f270 0x7f8664061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f86800c4c00 tx=0x7f8670008040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.457+0000 7f8675ffb700 1 -- 172.21.15.33:0/3790474804 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f867c04cea0 con 0x7f86800c7ff0 2023-11-06T21:03:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.681+0000 7f8687bff700 1 -- 172.21.15.33:0/3790474804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f86800830b0 con 0x7f86800c7ff0 2023-11-06T21:03:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.686+0000 7f8675ffb700 1 -- 172.21.15.33:0/3790474804 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v57) v1 ==== 174+0+0 (secure 0 0 0) 0x7f867c01f070 con 0x7f86800c7ff0 2023-11-06T21:03:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.689+0000 7f8687bff700 1 -- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f866405f270 msgr2=0x7f8664061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.689+0000 7f8687bff700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f866405f270 0x7f8664061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f86800c4c00 tx=0x7f8670008040).stop 2023-11-06T21:03:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.689+0000 7f8687bff700 1 -- 172.21.15.33:0/3790474804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86800c7ff0 msgr2=0x7f868014fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.689+0000 7f8687bff700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86800c7ff0 0x7f868014fd60 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f867c009fd0 tx=0x7f867c017000).stop 2023-11-06T21:03:21.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.690+0000 7f8687bff700 1 -- 172.21.15.33:0/3790474804 shutdown_connections 2023-11-06T21:03:21.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.690+0000 7f8687bff700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f866405f270 0x7f8664061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.690+0000 7f8687bff700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8680154710 0x7f86800aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.690+0000 7f8687bff700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86800c7ff0 0x7f868014fd60 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.690+0000 7f8687bff700 1 --2- 172.21.15.33:0/3790474804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680154330 0x7f868014ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.690+0000 7f8687bff700 1 -- 172.21.15.33:0/3790474804 >> 172.21.15.33:0/3790474804 conn(0x7f86800cb910 msgr2=0x7f86800630a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.691+0000 7f8687bff700 1 -- 172.21.15.33:0/3790474804 shutdown_connections 2023-11-06T21:03:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.691+0000 7f8687bff700 1 -- 172.21.15.33:0/3790474804 wait complete. 2023-11-06T21:03:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 17 and data pool 18 2023-11-06T21:03:21.727 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:03:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.922+0000 7f971ddff700 1 Processor -- start 2023-11-06T21:03:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.923+0000 7f971ddff700 1 -- start start 2023-11-06T21:03:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.924+0000 7f971ddff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718064ce0 0x7f97181544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.924+0000 7f971ddff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f97180aade0 0x7f97180c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.924+0000 7f971ddff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97180c0880 con 0x7f9718154110 2023-11-06T21:03:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.924+0000 7f971ddff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97180bfd40 con 0x7f97180aade0 2023-11-06T21:03:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.924+0000 7f971ddff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97180c8260 con 0x7f9718064ce0 2023-11-06T21:03:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.924+0000 7f97177fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9718154110 0x7f97180645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36532/0 (socket says 172.21.15.33:36532) 2023-11-06T21:03:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.924+0000 7f97177fe700 1 -- 172.21.15.33:0/3815253078 learned_addr learned my addr 172.21.15.33:0/3815253078 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f9716ffd700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718064ce0 0x7f97181544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562652079 0 0) 0x7f97180c0880 con 0x7f9718154110 2023-11-06T21:03:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f9717fff700 1 --2- 172.21.15.33:0/3815253078 >> v2:172.21.15.69:3300/0 conn(0x7f97180aade0 0x7f97180c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97000033f0 con 0x7f9718154110 2023-11-06T21:03:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4153129011 0 0) 0x7f97000033f0 con 0x7f9718154110 2023-11-06T21:03:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718064ce0 msgr2=0x7f97181544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718064ce0 0x7f97181544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 >> v2:172.21.15.69:3300/0 conn(0x7f97180aade0 msgr2=0x7f97180c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:21.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 --2- 172.21.15.33:0/3815253078 >> v2:172.21.15.69:3300/0 conn(0x7f97180aade0 0x7f97180c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97180bff50 con 0x7f9718154110 2023-11-06T21:03:21.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f9708001470 con 0x7f9718154110 2023-11-06T21:03:21.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 >> v1:172.21.15.33:6789/0 conn(0x7f9718154110 legacy=0x7f97180645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000038e0 0x7f9700003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718064ce0 0x7f9700004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9700004b80 0x7f9700005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97180bfd40 con 0x7f97000038e0 2023-11-06T21:03:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97180c8260 con 0x7f9700004b80 2023-11-06T21:03:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970000b8e0 con 0x7f9718064ce0 2023-11-06T21:03:21.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.925+0000 7f97177fe700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000038e0 0x7f9700003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f9716ffd700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718064ce0 0x7f9700004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f9717fff700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9700004b80 0x7f9700005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f97177fe700 1 -- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718064ce0 msgr2=0x7f9700004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:21.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f97177fe700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718064ce0 0x7f9700004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f97177fe700 1 -- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9700004b80 msgr2=0x7f9700005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:21.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f97177fe700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9700004b80 0x7f9700005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f97177fe700 1 -- 172.21.15.33:0/3815253078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97000033f0 con 0x7f97000038e0 2023-11-06T21:03:21.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f97177fe700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000038e0 0x7f9700003d00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f970800c210 tx=0x7f9708001e90).ready entity=mon.0 client_cookie=e1ddf7addfedab0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:21.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9708007a60 con 0x7f97000038e0 2023-11-06T21:03:21.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970800cae0 con 0x7f97000038e0 2023-11-06T21:03:21.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.926+0000 7f97167fc700 1 -- 172.21.15.33:0/3815253078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970800ce80 con 0x7f97000038e0 2023-11-06T21:03:21.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.927+0000 7f971ddff700 1 -- 172.21.15.33:0/3815253078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000038e0 msgr2=0x7f9700003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.927+0000 7f971ddff700 1 --2- 172.21.15.33:0/3815253078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000038e0 0x7f9700003d00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f970800c210 tx=0x7f9708001e90).stop 2023-11-06T21:03:21.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.927+0000 7f9717fff700 1 -- 172.21.15.33:0/3815253078 reap_dead start 2023-11-06T21:03:21.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.927+0000 7f971ddff700 1 -- 172.21.15.33:0/3815253078 shutdown_connections 2023-11-06T21:03:21.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.927+0000 7f971ddff700 1 -- 172.21.15.33:0/3815253078 >> 172.21.15.33:0/3815253078 conn(0x7f97180cb8b0 msgr2=0x7f97180c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:21.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.927+0000 7f971ddff700 1 -- 172.21.15.33:0/3815253078 shutdown_connections 2023-11-06T21:03:21.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.927+0000 7f971ddff700 1 -- 172.21.15.33:0/3815253078 wait complete. 2023-11-06T21:03:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.928+0000 7f971ddff700 1 Processor -- start 2023-11-06T21:03:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.929+0000 7f971ddff700 1 -- start start 2023-11-06T21:03:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f971ddff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718167830 0x7f97180ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f971ddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97180b0200 0x7f97181554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f971ddff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97181559c0 0x7f9718164930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f971ddff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97180a1bc0 con 0x7f97181559c0 2023-11-06T21:03:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f971ddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9718156e90 con 0x7f97180b0200 2023-11-06T21:03:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f971ddff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9718157a10 con 0x7f9718167830 2023-11-06T21:03:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f9717fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97181559c0 0x7f9718164930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f9717fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97181559c0 0x7f9718164930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50102/0 (socket says 172.21.15.33:50102) 2023-11-06T21:03:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f9717fff700 1 -- 172.21.15.33:0/2652917444 learned_addr learned my addr 172.21.15.33:0/2652917444 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:21.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f97177fe700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718167830 0x7f97180ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f9716ffd700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97180b0200 0x7f97181554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f9717fff700 1 -- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718167830 msgr2=0x7f97180ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f9717fff700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718167830 0x7f97180ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f9717fff700 1 -- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97180b0200 msgr2=0x7f97181554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f9717fff700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97180b0200 0x7f97181554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f9717fff700 1 -- 172.21.15.33:0/2652917444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9718155e40 con 0x7f97181559c0 2023-11-06T21:03:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f9716ffd700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97180b0200 0x7f97181554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.930+0000 7f97177fe700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718167830 0x7f97180ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.931+0000 7f9717fff700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97181559c0 0x7f9718164930 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f97180ab250 tx=0x7f971000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.931+0000 7f96ff7fe700 1 -- 172.21.15.33:0/2652917444 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971000e040 con 0x7f97181559c0 2023-11-06T21:03:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.931+0000 7f96ff7fe700 1 -- 172.21.15.33:0/2652917444 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971000f040 con 0x7f97181559c0 2023-11-06T21:03:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.931+0000 7f971ddff700 1 -- 172.21.15.33:0/2652917444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9718167c10 con 0x7f97181559c0 2023-11-06T21:03:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.931+0000 7f96ff7fe700 1 -- 172.21.15.33:0/2652917444 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971000fdf0 con 0x7f97181559c0 2023-11-06T21:03:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.931+0000 7f971ddff700 1 -- 172.21.15.33:0/2652917444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97181573c0 con 0x7f97181559c0 2023-11-06T21:03:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.932+0000 7f971ddff700 1 -- 172.21.15.33:0/2652917444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971806c6c0 con 0x7f97181559c0 2023-11-06T21:03:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.933+0000 7f96ff7fe700 1 -- 172.21.15.33:0/2652917444 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9710015660 con 0x7f97181559c0 2023-11-06T21:03:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.934+0000 7f96ff7fe700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f970405f270 0x7f9704061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.934+0000 7f96ff7fe700 1 -- 172.21.15.33:0/2652917444 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f971007aeb0 con 0x7f97181559c0 2023-11-06T21:03:21.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.934+0000 7f97177fe700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f970405f270 0x7f9704061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.935+0000 7f97177fe700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f970405f270 0x7f9704061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f97180844a0 tx=0x7f9708015000).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:21.938+0000 7f96ff7fe700 1 -- 172.21.15.33:0/2652917444 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f971004c000 con 0x7f97181559c0 2023-11-06T21:03:22.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.121+0000 7f971ddff700 1 -- 172.21.15.33:0/2652917444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9718157b50 con 0x7f97181559c0 2023-11-06T21:03:22.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.122+0000 7f96ff7fe700 1 -- 172.21.15.33:0/2652917444 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+17004 (secure 0 0 0) 0x7f971004f620 con 0x7f97181559c0 2023-11-06T21:03:22.123 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:22.123 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":146,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:03:21.682890+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:03:20.146564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T21:03:20.662302+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:03:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 -- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f970405f270 msgr2=0x7f9704061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f970405f270 0x7f9704061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f97180844a0 tx=0x7f9708015000).stop 2023-11-06T21:03:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 -- 172.21.15.33:0/2652917444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97181559c0 msgr2=0x7f9718164930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97181559c0 0x7f9718164930 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f97180ab250 tx=0x7f971000b040).stop 2023-11-06T21:03:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 -- 172.21.15.33:0/2652917444 shutdown_connections 2023-11-06T21:03:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f970405f270 0x7f9704061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9718167830 0x7f97180ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97181559c0 0x7f9718164930 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 --2- 172.21.15.33:0/2652917444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97180b0200 0x7f97181554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 -- 172.21.15.33:0/2652917444 >> 172.21.15.33:0/2652917444 conn(0x7f97180cb8b0 msgr2=0x7f97180acfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.124+0000 7f971ddff700 1 -- 172.21.15.33:0/2652917444 shutdown_connections 2023-11-06T21:03:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.125+0000 7f971ddff700 1 -- 172.21.15.33:0/2652917444 wait complete. 2023-11-06T21:03:22.143 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:03:22.177 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-11-06T21:03:22.177+0000 7f8716ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T21:03:22.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.338+0000 7f38659ff700 1 Processor -- start 2023-11-06T21:03:22.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.339+0000 7f38659ff700 1 -- start start 2023-11-06T21:03:22.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.340+0000 7f38659ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38600bdbe0 0x7f38601584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.340+0000 7f38659ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f38600a7910 0x7f38600a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.340+0000 7f38659ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3860064010 con 0x7f38601580e0 2023-11-06T21:03:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.340+0000 7f38659ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38600be430 con 0x7f38600a7910 2023-11-06T21:03:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.340+0000 7f38659ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38600be150 con 0x7f38600bdbe0 2023-11-06T21:03:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.340+0000 7f385effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f38601580e0 0x7f38600684c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36546/0 (socket says 172.21.15.33:36546) 2023-11-06T21:03:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.340+0000 7f385effd700 1 -- 172.21.15.33:0/2408820375 learned_addr learned my addr 172.21.15.33:0/2408820375 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.340+0000 7f385e7fc700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38600bdbe0 0x7f38601584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.340+0000 7f385f7fe700 1 --2- 172.21.15.33:0/2408820375 >> v2:172.21.15.69:3300/0 conn(0x7f38600a7910 0x7f38600a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338314430 0 0) 0x7f3860064010 con 0x7f38601580e0 2023-11-06T21:03:22.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38480033f0 con 0x7f38601580e0 2023-11-06T21:03:22.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1547416405 0 0) 0x7f38480033f0 con 0x7f38601580e0 2023-11-06T21:03:22.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38600bdbe0 msgr2=0x7f38601584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38600bdbe0 0x7f38601584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 >> v2:172.21.15.69:3300/0 conn(0x7f38600a7910 msgr2=0x7f38600a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 --2- 172.21.15.33:0/2408820375 >> v2:172.21.15.69:3300/0 conn(0x7f38600a7910 0x7f38600a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600639c0 con 0x7f38601580e0 2023-11-06T21:03:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f3850001470 con 0x7f38601580e0 2023-11-06T21:03:22.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 >> v1:172.21.15.33:6789/0 conn(0x7f38601580e0 legacy=0x7f38600684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38480038e0 0x7f3848003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38600bdbe0 0x7f3848004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3848004b80 0x7f3848005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38600be430 con 0x7f3848004b80 2023-11-06T21:03:22.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38600be150 con 0x7f38480038e0 2023-11-06T21:03:22.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384800b8e0 con 0x7f38600bdbe0 2023-11-06T21:03:22.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385f7fe700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3848004b80 0x7f3848005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.341+0000 7f385effd700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38480038e0 0x7f3848003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385e7fc700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38600bdbe0 0x7f3848004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385e7fc700 1 -- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38480038e0 msgr2=0x7f3848003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:22.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385e7fc700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38480038e0 0x7f3848003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385e7fc700 1 -- 172.21.15.33:0/2408820375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3848004b80 msgr2=0x7f3848005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:22.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385e7fc700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3848004b80 0x7f3848005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385e7fc700 1 -- 172.21.15.33:0/2408820375 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480033f0 con 0x7f38600bdbe0 2023-11-06T21:03:22.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385f7fe700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3848004b80 0x7f3848005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:22.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385e7fc700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38600bdbe0 0x7f3848004690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f3854009a80 tx=0x7f3854009060).ready entity=mon.2 client_cookie=84decbf66ac82af1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854009430 con 0x7f38600bdbe0 2023-11-06T21:03:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385400e640 con 0x7f38600bdbe0 2023-11-06T21:03:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.342+0000 7f385dffb700 1 -- 172.21.15.33:0/2408820375 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400e9c0 con 0x7f38600bdbe0 2023-11-06T21:03:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.343+0000 7f38659ff700 1 -- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38600bdbe0 msgr2=0x7f3848004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.343+0000 7f38659ff700 1 --2- 172.21.15.33:0/2408820375 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38600bdbe0 0x7f3848004690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f3854009a80 tx=0x7f3854009060).stop 2023-11-06T21:03:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.343+0000 7f385f7fe700 1 -- 172.21.15.33:0/2408820375 reap_dead start 2023-11-06T21:03:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.343+0000 7f38659ff700 1 -- 172.21.15.33:0/2408820375 shutdown_connections 2023-11-06T21:03:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.343+0000 7f38659ff700 1 -- 172.21.15.33:0/2408820375 >> 172.21.15.33:0/2408820375 conn(0x7f386005ec80 msgr2=0x7f38600664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.343+0000 7f38659ff700 1 -- 172.21.15.33:0/2408820375 shutdown_connections 2023-11-06T21:03:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.343+0000 7f38659ff700 1 -- 172.21.15.33:0/2408820375 wait complete. 2023-11-06T21:03:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.345+0000 7f38659ff700 1 Processor -- start 2023-11-06T21:03:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.345+0000 7f38659ff700 1 -- start start 2023-11-06T21:03:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f38659ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38600bdbe0 0x7f386013a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f38659ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386014e560 0x7f38600c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f38659ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f386014e9e0 0x7f38600c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f38659ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38600c28d0 con 0x7f386014e560 2023-11-06T21:03:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f38659ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860168340 con 0x7f38600bdbe0 2023-11-06T21:03:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f38659ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3860168620 con 0x7f386014e9e0 2023-11-06T21:03:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386014e560 0x7f38600c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386014e560 0x7f38600c1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50124/0 (socket says 172.21.15.33:50124) 2023-11-06T21:03:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385e7fc700 1 -- 172.21.15.33:0/2159901559 learned_addr learned my addr 172.21.15.33:0/2159901559 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385effd700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38600bdbe0 0x7f386013a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385f7fe700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f386014e9e0 0x7f38600c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385e7fc700 1 -- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f386014e9e0 msgr2=0x7f38600c23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385e7fc700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f386014e9e0 0x7f38600c23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385e7fc700 1 -- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38600bdbe0 msgr2=0x7f386013a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385e7fc700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38600bdbe0 0x7f386013a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385e7fc700 1 -- 172.21.15.33:0/2159901559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860168900 con 0x7f386014e560 2023-11-06T21:03:22.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.347+0000 7f385effd700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38600bdbe0 0x7f386013a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:22.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.348+0000 7f385e7fc700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386014e560 0x7f38600c1ef0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f385400e4b0 tx=0x7f385401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:22.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.348+0000 7f3846ffd700 1 -- 172.21.15.33:0/2159901559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854020070 con 0x7f386014e560 2023-11-06T21:03:22.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.348+0000 7f3846ffd700 1 -- 172.21.15.33:0/2159901559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3854009ec0 con 0x7f386014e560 2023-11-06T21:03:22.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.348+0000 7f3846ffd700 1 -- 172.21.15.33:0/2159901559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400e7b0 con 0x7f386014e560 2023-11-06T21:03:22.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.348+0000 7f38659ff700 1 -- 172.21.15.33:0/2159901559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3860168be0 con 0x7f386014e560 2023-11-06T21:03:22.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.348+0000 7f38659ff700 1 -- 172.21.15.33:0/2159901559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386013ab20 con 0x7f386014e560 2023-11-06T21:03:22.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.349+0000 7f38659ff700 1 -- 172.21.15.33:0/2159901559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38600648d0 con 0x7f386014e560 2023-11-06T21:03:22.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.350+0000 7f3846ffd700 1 -- 172.21.15.33:0/2159901559 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f38540123f0 con 0x7f386014e560 2023-11-06T21:03:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.351+0000 7f3846ffd700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384c05f270 0x7f384c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.351+0000 7f3846ffd700 1 -- 172.21.15.33:0/2159901559 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f385407b650 con 0x7f386014e560 2023-11-06T21:03:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.351+0000 7f385effd700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384c05f270 0x7f384c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.352+0000 7f385effd700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384c05f270 0x7f384c061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3850001620 tx=0x7f3850009040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:22.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.355+0000 7f3846ffd700 1 -- 172.21.15.33:0/2159901559 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f385404c730 con 0x7f386014e560 2023-11-06T21:03:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.533+0000 7f38659ff700 1 -- 172.21.15.33:0/2159901559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38600645f0 con 0x7f386014e560 2023-11-06T21:03:22.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.534+0000 7f3846ffd700 1 -- 172.21.15.33:0/2159901559 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+17004 (secure 0 0 0) 0x7f3854029090 con 0x7f386014e560 2023-11-06T21:03:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:22.536 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":146,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:03:21.682890+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:03:20.146564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T21:03:20.662302+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:03:22.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 -- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384c05f270 msgr2=0x7f384c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384c05f270 0x7f384c061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3850001620 tx=0x7f3850009040).stop 2023-11-06T21:03:22.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 -- 172.21.15.33:0/2159901559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386014e560 msgr2=0x7f38600c1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386014e560 0x7f38600c1ef0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f385400e4b0 tx=0x7f385401a000).stop 2023-11-06T21:03:22.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 -- 172.21.15.33:0/2159901559 shutdown_connections 2023-11-06T21:03:22.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f384c05f270 0x7f384c061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f386014e9e0 0x7f38600c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38600bdbe0 0x7f386013a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 --2- 172.21.15.33:0/2159901559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386014e560 0x7f38600c1ef0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 -- 172.21.15.33:0/2159901559 >> 172.21.15.33:0/2159901559 conn(0x7f386005ec80 msgr2=0x7f38600650d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:22.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 -- 172.21.15.33:0/2159901559 shutdown_connections 2023-11-06T21:03:22.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.537+0000 7f38659ff700 1 -- 172.21.15.33:0/2159901559 wait complete. 2023-11-06T21:03:22.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T21:03:22.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.794+0000 7f6fb25ff700 1 Processor -- start 2023-11-06T21:03:22.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.794+0000 7f6fb25ff700 1 -- start start 2023-11-06T21:03:22.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.796+0000 7f6fb25ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 0x7f6fac0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.796+0000 7f6fb25ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6fac0c7ff0 0x7f6fac0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.796+0000 7f6fb25ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac0c6b60 con 0x7f6fac154710 2023-11-06T21:03:22.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.796+0000 7f6fb25ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fac0c7700 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.796+0000 7f6fb25ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0c5cd0 con 0x7f6fac154330 2023-11-06T21:03:22.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.796+0000 7f6fb0b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6fac0c7ff0 0x7f6fac0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.796+0000 7f6fb0b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6fac0c7ff0 0x7f6fac0646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50134/0 (socket says 172.21.15.33:50134) 2023-11-06T21:03:22.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.796+0000 7f6fb0b9c700 1 -- 172.21.15.33:0/2215511546 learned_addr learned my addr 172.21.15.33:0/2215511546 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:22.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.797+0000 7f6fabfff700 1 --2- 172.21.15.33:0/2215511546 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 0x7f6fac0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.797+0000 7f6fb0b9c700 1 -- 172.21.15.33:0/2215511546 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 msgr2=0x7f6fac0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.797+0000 7f6fb0b9c700 1 --2- 172.21.15.33:0/2215511546 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 0x7f6fac0c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.797+0000 7f6fb0b9c700 1 -- 172.21.15.33:0/2215511546 >> v1:172.21.15.69:6789/0 conn(0x7f6fac154710 legacy=0x7f6fac0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.797+0000 7f6fb0b9c700 1 -- 172.21.15.33:0/2215511546 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0c5f50 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.797+0000 7f6fb0b9c700 1 --2- 172.21.15.33:0/2215511546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 0x7f6fac0646c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f6fa4009f20 tx=0x7f6fa4009840).ready entity=mon.0 client_cookie=26cf0cfbab7f1e4e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.797+0000 7f6faaffd700 1 -- 172.21.15.33:0/2215511546 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400e070 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.797+0000 7f6faaffd700 1 -- 172.21.15.33:0/2215511546 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa400ba80 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.797+0000 7f6faaffd700 1 -- 172.21.15.33:0/2215511546 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400be00 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.798+0000 7f6fb25ff700 1 -- 172.21.15.33:0/2215511546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 msgr2=0x7f6fac0646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.798+0000 7f6fb25ff700 1 --2- 172.21.15.33:0/2215511546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 0x7f6fac0646c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f6fa4009f20 tx=0x7f6fa4009840).stop 2023-11-06T21:03:22.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.798+0000 7f6fb25ff700 1 -- 172.21.15.33:0/2215511546 shutdown_connections 2023-11-06T21:03:22.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.798+0000 7f6fb25ff700 1 --2- 172.21.15.33:0/2215511546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 0x7f6fac0646c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.798+0000 7f6fb25ff700 1 --2- 172.21.15.33:0/2215511546 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 0x7f6fac0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.798+0000 7f6fb25ff700 1 -- 172.21.15.33:0/2215511546 >> 172.21.15.33:0/2215511546 conn(0x7f6fac0cb910 msgr2=0x7f6fac0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:22.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.798+0000 7f6fb25ff700 1 -- 172.21.15.33:0/2215511546 shutdown_connections 2023-11-06T21:03:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.798+0000 7f6fb25ff700 1 -- 172.21.15.33:0/2215511546 wait complete. 2023-11-06T21:03:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.799+0000 7f6fb25ff700 1 Processor -- start 2023-11-06T21:03:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.800+0000 7f6fb25ff700 1 -- start start 2023-11-06T21:03:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fb25ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 0x7f6fac165580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fb25ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 0x7f6fac0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fb25ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac154710 0x7f6fac0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fb25ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0aed00 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fb25ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac1650f0 con 0x7f6fac154710 2023-11-06T21:03:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fb25ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac165a70 con 0x7f6fac154330 2023-11-06T21:03:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 0x7f6fac165580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 0x7f6fac165580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50146/0 (socket says 172.21.15.33:50146) 2023-11-06T21:03:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fabfff700 1 -- 172.21.15.33:0/4216952799 learned_addr learned my addr 172.21.15.33:0/4216952799 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fb0b9c700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac154710 0x7f6fac0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fab7fe700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 0x7f6fac0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fabfff700 1 -- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 msgr2=0x7f6fac0b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fabfff700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 0x7f6fac0b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fabfff700 1 -- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac154710 msgr2=0x7f6fac0addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:22.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fabfff700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac154710 0x7f6fac0addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:22.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fabfff700 1 -- 172.21.15.33:0/4216952799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4009ae0 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fab7fe700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 0x7f6fac0b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:22.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.801+0000 7f6fabfff700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 0x7f6fac165580 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f6f9c00d8c0 tx=0x7f6f9c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:22.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.802+0000 7f6f93fff700 1 -- 172.21.15.33:0/4216952799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0095b0 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.802+0000 7f6f93fff700 1 -- 172.21.15.33:0/4216952799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c009710 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.802+0000 7f6f93fff700 1 -- 172.21.15.33:0/4216952799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c009a90 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.802+0000 7f6fb25ff700 1 -- 172.21.15.33:0/4216952799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac164ad0 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.802+0000 7f6fb25ff700 1 -- 172.21.15.33:0/4216952799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac0ae7c0 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.803+0000 7f6fb25ff700 1 -- 172.21.15.33:0/4216952799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac0aeae0 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.804+0000 7f6f93fff700 1 -- 172.21.15.33:0/4216952799 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6f9c013480 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.805+0000 7f6f93fff700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f9805f270 0x7f6f98061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.805+0000 7f6f93fff700 1 -- 172.21.15.33:0/4216952799 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6f9c07a570 con 0x7f6fac0c7ff0 2023-11-06T21:03:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.805+0000 7f6fab7fe700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f9805f270 0x7f6f98061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.806+0000 7f6fab7fe700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f9805f270 0x7f6f98061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6fac0ac270 tx=0x7f6fa0009040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:22.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:22.809+0000 7f6f93fff700 1 -- 172.21.15.33:0/4216952799 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6f9c04b660 con 0x7f6fac0c7ff0 2023-11-06T21:03:23.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.037+0000 7f6fb25ff700 1 -- 172.21.15.33:0/4216952799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6fac0830b0 con 0x7f6fac0c7ff0 2023-11-06T21:03:23.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.689+0000 7f6f93fff700 1 -- 172.21.15.33:0/4216952799 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6f9c04ec80 con 0x7f6fac0c7ff0 2023-11-06T21:03:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 -- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f9805f270 msgr2=0x7f6f98061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f9805f270 0x7f6f98061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6fac0ac270 tx=0x7f6fa0009040).stop 2023-11-06T21:03:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 -- 172.21.15.33:0/4216952799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 msgr2=0x7f6fac165580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 0x7f6fac165580 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f6f9c00d8c0 tx=0x7f6f9c008040).stop 2023-11-06T21:03:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 -- 172.21.15.33:0/4216952799 shutdown_connections 2023-11-06T21:03:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f9805f270 0x7f6f98061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fac154710 0x7f6fac0addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fac0c7ff0 0x7f6fac165580 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 --2- 172.21.15.33:0/4216952799 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6fac154330 0x7f6fac0b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 -- 172.21.15.33:0/4216952799 >> 172.21.15.33:0/4216952799 conn(0x7f6fac0cb910 msgr2=0x7f6fac0ab510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.691+0000 7f6fb25ff700 1 -- 172.21.15.33:0/4216952799 shutdown_connections 2023-11-06T21:03:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.692+0000 7f6fb25ff700 1 -- 172.21.15.33:0/4216952799 wait complete. 2023-11-06T21:03:23.711 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T21:03:23.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T21:03:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.911+0000 7f4b6cbff700 1 Processor -- start 2023-11-06T21:03:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.911+0000 7f4b6cbff700 1 -- start start 2023-11-06T21:03:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.912+0000 7f4b6cbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4b68154710 0x7f4b680c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.912+0000 7f4b6cbff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b680c6b60 con 0x7f4b680c7ff0 2023-11-06T21:03:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.912+0000 7f4b6cbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b680c7700 con 0x7f4b68154330 2023-11-06T21:03:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.912+0000 7f4b6cbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b680c5cd0 con 0x7f4b68154710 2023-11-06T21:03:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.912+0000 7f4b66d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4b680c7ff0 0x7f4b680646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36556/0 (socket says 172.21.15.33:36556) 2023-11-06T21:03:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.912+0000 7f4b66d9d700 1 -- 172.21.15.33:0/4151797920 learned_addr learned my addr 172.21.15.33:0/4151797920 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.913+0000 7f4b65d9b700 1 --2- 172.21.15.33:0/4151797920 >> v2:172.21.15.69:3300/0 conn(0x7f4b68154710 0x7f4b680c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.913+0000 7f4b6559a700 1 -- 172.21.15.33:0/4151797920 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115015749 0 0) 0x7f4b680c6b60 con 0x7f4b680c7ff0 2023-11-06T21:03:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.913+0000 7f4b6559a700 1 -- 172.21.15.33:0/4151797920 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b500033f0 con 0x7f4b680c7ff0 2023-11-06T21:03:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.913+0000 7f4b65d9b700 1 -- 172.21.15.33:0/4151797920 >> v1:172.21.15.69:6789/0 conn(0x7f4b68154330 legacy=0x7f4b680c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.913+0000 7f4b6559a700 1 -- 172.21.15.33:0/4151797920 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233109379 0 0) 0x7f4b680c7700 con 0x7f4b68154330 2023-11-06T21:03:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.913+0000 7f4b65d9b700 1 -- 172.21.15.33:0/4151797920 >> v1:172.21.15.33:6789/0 conn(0x7f4b680c7ff0 legacy=0x7f4b680646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.913+0000 7f4b65d9b700 1 -- 172.21.15.33:0/4151797920 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b680c5f50 con 0x7f4b68154710 2023-11-06T21:03:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.913+0000 7f4b65d9b700 1 --2- 172.21.15.33:0/4151797920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b68154710 0x7f4b680c57e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f4b5c009760 tx=0x7f4b5c009060).ready entity=mon.1 client_cookie=c6ff13228c898c15 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.914+0000 7f4b6559a700 1 -- 172.21.15.33:0/4151797920 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00f510 con 0x7f4b68154710 2023-11-06T21:03:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.914+0000 7f4b6559a700 1 -- 172.21.15.33:0/4151797920 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c00f670 con 0x7f4b68154710 2023-11-06T21:03:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.914+0000 7f4b6559a700 1 -- 172.21.15.33:0/4151797920 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00f9f0 con 0x7f4b68154710 2023-11-06T21:03:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.914+0000 7f4b6cbff700 1 -- 172.21.15.33:0/4151797920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b68154710 msgr2=0x7f4b680c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.914+0000 7f4b6cbff700 1 --2- 172.21.15.33:0/4151797920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b68154710 0x7f4b680c57e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f4b5c009760 tx=0x7f4b5c009060).stop 2023-11-06T21:03:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.914+0000 7f4b6cbff700 1 -- 172.21.15.33:0/4151797920 shutdown_connections 2023-11-06T21:03:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.914+0000 7f4b6cbff700 1 --2- 172.21.15.33:0/4151797920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b68154710 0x7f4b680c57e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.914+0000 7f4b6cbff700 1 -- 172.21.15.33:0/4151797920 >> 172.21.15.33:0/4151797920 conn(0x7f4b680cb910 msgr2=0x7f4b680c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.914+0000 7f4b6cbff700 1 -- 172.21.15.33:0/4151797920 shutdown_connections 2023-11-06T21:03:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.914+0000 7f4b6cbff700 1 -- 172.21.15.33:0/4151797920 wait complete. 2023-11-06T21:03:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.916+0000 7f4b6cbff700 1 Processor -- start 2023-11-06T21:03:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.916+0000 7f4b6cbff700 1 -- start start 2023-11-06T21:03:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b6cbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b680c7ff0 0x7f4b68147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b6cbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b68154330 0x7f4b680c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b6cbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b68154710 0x7f4b680c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b6cbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680c9520 con 0x7f4b68154710 2023-11-06T21:03:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b6cbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680ae480 con 0x7f4b68154330 2023-11-06T21:03:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b6cbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680ade70 con 0x7f4b680c7ff0 2023-11-06T21:03:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b66d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b68154710 0x7f4b680c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b66d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b68154710 0x7f4b680c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50150/0 (socket says 172.21.15.33:50150) 2023-11-06T21:03:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b66d9d700 1 -- 172.21.15.33:0/1543203756 learned_addr learned my addr 172.21.15.33:0/1543203756 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b65d9b700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b68154330 0x7f4b680c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b6659c700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b680c7ff0 0x7f4b68147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b66d9d700 1 -- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b680c7ff0 msgr2=0x7f4b68147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b66d9d700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b680c7ff0 0x7f4b68147ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b66d9d700 1 -- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b68154330 msgr2=0x7f4b680c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b66d9d700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b68154330 0x7f4b680c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.917+0000 7f4b66d9d700 1 -- 172.21.15.33:0/1543203756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c0092a0 con 0x7f4b68154710 2023-11-06T21:03:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.918+0000 7f4b66d9d700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b68154710 0x7f4b680c9030 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f4b60005ad0 tx=0x7f4b600015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.918+0000 7f4b567fc700 1 -- 172.21.15.33:0/1543203756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60005d30 con 0x7f4b68154710 2023-11-06T21:03:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.918+0000 7f4b567fc700 1 -- 172.21.15.33:0/1543203756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b60005e90 con 0x7f4b68154710 2023-11-06T21:03:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.918+0000 7f4b6cbff700 1 -- 172.21.15.33:0/1543203756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b68156950 con 0x7f4b68154710 2023-11-06T21:03:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.918+0000 7f4b567fc700 1 -- 172.21.15.33:0/1543203756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60007320 con 0x7f4b68154710 2023-11-06T21:03:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.918+0000 7f4b6cbff700 1 -- 172.21.15.33:0/1543203756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b68156c30 con 0x7f4b68154710 2023-11-06T21:03:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.919+0000 7f4b6cbff700 1 -- 172.21.15.33:0/1543203756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b680ae610 con 0x7f4b68154710 2023-11-06T21:03:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.920+0000 7f4b567fc700 1 -- 172.21.15.33:0/1543203756 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4b60007710 con 0x7f4b68154710 2023-11-06T21:03:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.921+0000 7f4b567fc700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4b4805f270 0x7f4b48061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.921+0000 7f4b567fc700 1 -- 172.21.15.33:0/1543203756 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4b6007a500 con 0x7f4b68154710 2023-11-06T21:03:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.921+0000 7f4b6659c700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4b4805f270 0x7f4b48061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.922+0000 7f4b6659c700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4b4805f270 0x7f4b48061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4b680aa8b0 tx=0x7f4b58009040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:23.925+0000 7f4b567fc700 1 -- 172.21.15.33:0/1543203756 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4b6004b590 con 0x7f4b68154710 2023-11-06T21:03:24.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.150+0000 7f4b6cbff700 1 -- 172.21.15.33:0/1543203756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4b680652d0 con 0x7f4b68154710 2023-11-06T21:03:24.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.696+0000 7f4b567fc700 1 -- 172.21.15.33:0/1543203756 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4b6004ebb0 con 0x7f4b68154710 2023-11-06T21:03:24.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.698+0000 7f4b6cbff700 1 -- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4b4805f270 msgr2=0x7f4b48061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:24.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.698+0000 7f4b6cbff700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4b4805f270 0x7f4b48061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4b680aa8b0 tx=0x7f4b58009040).stop 2023-11-06T21:03:24.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.698+0000 7f4b6cbff700 1 -- 172.21.15.33:0/1543203756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b68154710 msgr2=0x7f4b680c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:24.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.698+0000 7f4b6cbff700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b68154710 0x7f4b680c9030 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f4b60005ad0 tx=0x7f4b600015b0).stop 2023-11-06T21:03:24.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.698+0000 7f4b6cbff700 1 -- 172.21.15.33:0/1543203756 shutdown_connections 2023-11-06T21:03:24.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.698+0000 7f4b6cbff700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4b4805f270 0x7f4b48061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:24.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.698+0000 7f4b6cbff700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4b680c7ff0 0x7f4b68147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:24.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.698+0000 7f4b6cbff700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b68154710 0x7f4b680c9030 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:24.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.698+0000 7f4b6cbff700 1 --2- 172.21.15.33:0/1543203756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b68154330 0x7f4b680c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:24.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.699+0000 7f4b6cbff700 1 -- 172.21.15.33:0/1543203756 >> 172.21.15.33:0/1543203756 conn(0x7f4b680cb910 msgr2=0x7f4b680c01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:24.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.699+0000 7f4b6cbff700 1 -- 172.21.15.33:0/1543203756 shutdown_connections 2023-11-06T21:03:24.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.699+0000 7f4b6cbff700 1 -- 172.21.15.33:0/1543203756 wait complete. 2023-11-06T21:03:24.720 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T21:03:24.720 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:03:24.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.920+0000 7f7442fff700 1 Processor -- start 2023-11-06T21:03:24.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.920+0000 7f7442fff700 1 -- start start 2023-11-06T21:03:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.921+0000 7f7442fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f743c063280 0x7f743c063660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.921+0000 7f7442fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c0653f0 con 0x7f743c14e4a0 2023-11-06T21:03:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.921+0000 7f7442fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c158e20 con 0x7f743c0c15f0 2023-11-06T21:03:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.921+0000 7f7442fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c1582a0 con 0x7f743c063280 2023-11-06T21:03:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.921+0000 7f743bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f743c14e4a0 0x7f743c0645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36562/0 (socket says 172.21.15.33:36562) 2023-11-06T21:03:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.921+0000 7f743bfff700 1 -- 172.21.15.33:0/3830387372 learned_addr learned my addr 172.21.15.33:0/3830387372 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.921+0000 7f7440d9b700 1 --2- 172.21.15.33:0/3830387372 >> v2:172.21.15.69:3300/0 conn(0x7f743c063280 0x7f743c063660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.922+0000 7f743b7fe700 1 -- 172.21.15.33:0/3830387372 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258669658 0 0) 0x7f743c0653f0 con 0x7f743c14e4a0 2023-11-06T21:03:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.922+0000 7f743b7fe700 1 -- 172.21.15.33:0/3830387372 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f742c0033f0 con 0x7f743c14e4a0 2023-11-06T21:03:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.922+0000 7f7440d9b700 1 -- 172.21.15.33:0/3830387372 >> v1:172.21.15.69:6789/0 conn(0x7f743c0c15f0 legacy=0x7f743c064ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:24.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.922+0000 7f743b7fe700 1 -- 172.21.15.33:0/3830387372 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348829123 0 0) 0x7f743c158e20 con 0x7f743c0c15f0 2023-11-06T21:03:24.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.922+0000 7f7440d9b700 1 -- 172.21.15.33:0/3830387372 >> v1:172.21.15.33:6789/0 conn(0x7f743c14e4a0 legacy=0x7f743c0645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:24.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.922+0000 7f7440d9b700 1 -- 172.21.15.33:0/3830387372 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c158580 con 0x7f743c063280 2023-11-06T21:03:24.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.922+0000 7f7440d9b700 1 --2- 172.21.15.33:0/3830387372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f743c063280 0x7f743c063660 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f7430009760 tx=0x7f7430009060).ready entity=mon.1 client_cookie=1c3440bd643c0ddc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:24.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.922+0000 7f743b7fe700 1 -- 172.21.15.33:0/3830387372 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743000f510 con 0x7f743c063280 2023-11-06T21:03:24.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.923+0000 7f743b7fe700 1 -- 172.21.15.33:0/3830387372 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743000f670 con 0x7f743c063280 2023-11-06T21:03:24.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.923+0000 7f743b7fe700 1 -- 172.21.15.33:0/3830387372 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743000f9f0 con 0x7f743c063280 2023-11-06T21:03:24.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.923+0000 7f7442fff700 1 -- 172.21.15.33:0/3830387372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f743c063280 msgr2=0x7f743c063660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:24.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.923+0000 7f7442fff700 1 --2- 172.21.15.33:0/3830387372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f743c063280 0x7f743c063660 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f7430009760 tx=0x7f7430009060).stop 2023-11-06T21:03:24.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.923+0000 7f7442fff700 1 -- 172.21.15.33:0/3830387372 shutdown_connections 2023-11-06T21:03:24.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.923+0000 7f7442fff700 1 --2- 172.21.15.33:0/3830387372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f743c063280 0x7f743c063660 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:24.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.923+0000 7f7442fff700 1 -- 172.21.15.33:0/3830387372 >> 172.21.15.33:0/3830387372 conn(0x7f743c0cade0 msgr2=0x7f743c0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:24.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.923+0000 7f7442fff700 1 -- 172.21.15.33:0/3830387372 shutdown_connections 2023-11-06T21:03:24.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.923+0000 7f7442fff700 1 -- 172.21.15.33:0/3830387372 wait complete. 2023-11-06T21:03:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.925+0000 7f7442fff700 1 Processor -- start 2023-11-06T21:03:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.926+0000 7f7442fff700 1 -- start start 2023-11-06T21:03:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f7442fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f743c063280 0x7f743c0a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f7442fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f743c0c15f0 0x7f743c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f7442fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f743c14e4a0 0x7f743c0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f7442fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0c9860 con 0x7f743c14e4a0 2023-11-06T21:03:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f7442fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c13d290 con 0x7f743c0c15f0 2023-11-06T21:03:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f7442fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0c39f0 con 0x7f743c063280 2023-11-06T21:03:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f744159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f743c14e4a0 0x7f743c0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f744159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f743c14e4a0 0x7f743c0c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50164/0 (socket says 172.21.15.33:50164) 2023-11-06T21:03:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f744159c700 1 -- 172.21.15.33:0/182601897 learned_addr learned my addr 172.21.15.33:0/182601897 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f743bfff700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f743c0c15f0 0x7f743c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.927+0000 7f7440d9b700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f743c063280 0x7f743c0a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f744159c700 1 -- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f743c063280 msgr2=0x7f743c0a91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f744159c700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f743c063280 0x7f743c0a91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f744159c700 1 -- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f743c0c15f0 msgr2=0x7f743c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f744159c700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f743c0c15f0 0x7f743c0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f744159c700 1 -- 172.21.15.33:0/182601897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74300092a0 con 0x7f743c14e4a0 2023-11-06T21:03:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f7440d9b700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f743c063280 0x7f743c0a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f743bfff700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f743c0c15f0 0x7f743c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f744159c700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f743c14e4a0 0x7f743c0c9370 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f7434002230 tx=0x7f7434007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:24.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f7438ff9700 1 -- 172.21.15.33:0/182601897 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743400a580 con 0x7f743c14e4a0 2023-11-06T21:03:24.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f7438ff9700 1 -- 172.21.15.33:0/182601897 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743400a6e0 con 0x7f743c14e4a0 2023-11-06T21:03:24.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f7438ff9700 1 -- 172.21.15.33:0/182601897 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743400aa80 con 0x7f743c14e4a0 2023-11-06T21:03:24.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f7442fff700 1 -- 172.21.15.33:0/182601897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c0c2de0 con 0x7f743c14e4a0 2023-11-06T21:03:24.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.928+0000 7f7442fff700 1 -- 172.21.15.33:0/182601897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c0c33a0 con 0x7f743c14e4a0 2023-11-06T21:03:24.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.929+0000 7f7442fff700 1 -- 172.21.15.33:0/182601897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c0c3b60 con 0x7f743c14e4a0 2023-11-06T21:03:24.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.930+0000 7f7438ff9700 1 -- 172.21.15.33:0/182601897 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f743401b400 con 0x7f743c14e4a0 2023-11-06T21:03:24.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.931+0000 7f7438ff9700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f742005f270 0x7f7420061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:24.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.931+0000 7f7438ff9700 1 -- 172.21.15.33:0/182601897 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f743407a3a0 con 0x7f743c14e4a0 2023-11-06T21:03:24.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.931+0000 7f7440d9b700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f742005f270 0x7f7420061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:24.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.932+0000 7f7440d9b700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f742005f270 0x7f7420061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7430009760 tx=0x7f7430017000).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:24.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:24.935+0000 7f7438ff9700 1 -- 172.21.15.33:0/182601897 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f743404b3d0 con 0x7f743c14e4a0 2023-11-06T21:03:25.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.159+0000 7f7442fff700 1 -- 172.21.15.33:0/182601897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f743c06c6c0 con 0x7f743c14e4a0 2023-11-06T21:03:25.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.160+0000 7f7438ff9700 1 -- 172.21.15.33:0/182601897 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f743404e9f0 con 0x7f743c14e4a0 2023-11-06T21:03:25.161 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1809147689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1809147689},{"type":"v1","addr":"172.21.15.33:6835","nonce":1809147689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1390207299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1390207299},{"type":"v1","addr":"172.21.15.69:6837","nonce":1390207299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3179984136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3179984136},{"type":"v1","addr":"172.21.15.69:6836","nonce":3179984136}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:03:21.683218+0000","modified":"2023-11-06T21:03:24.689443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7720":{"gid":7720,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/4126832799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4126832799},{"type":"v1","addr":"172.21.15.33:6834","nonce":4126832799}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-06T21:03:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 -- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f742005f270 msgr2=0x7f7420061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f742005f270 0x7f7420061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7430009760 tx=0x7f7430017000).stop 2023-11-06T21:03:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 -- 172.21.15.33:0/182601897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f743c14e4a0 msgr2=0x7f743c0c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f743c14e4a0 0x7f743c0c9370 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f7434002230 tx=0x7f7434007f70).stop 2023-11-06T21:03:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 -- 172.21.15.33:0/182601897 shutdown_connections 2023-11-06T21:03:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f742005f270 0x7f7420061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f743c063280 0x7f743c0a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f743c14e4a0 0x7f743c0c9370 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 --2- 172.21.15.33:0/182601897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f743c0c15f0 0x7f743c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 -- 172.21.15.33:0/182601897 >> 172.21.15.33:0/182601897 conn(0x7f743c0cade0 msgr2=0x7f743c0aae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 -- 172.21.15.33:0/182601897 shutdown_connections 2023-11-06T21:03:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.162+0000 7f7442fff700 1 -- 172.21.15.33:0/182601897 wait complete. 2023-11-06T21:03:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 60 2023-11-06T21:03:25.181 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:03:25.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.376+0000 7f97d19ff700 1 Processor -- start 2023-11-06T21:03:25.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.376+0000 7f97d19ff700 1 -- start start 2023-11-06T21:03:25.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.377+0000 7f97d19ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97cc0c1840 0x7f97cc0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.377+0000 7f97d19ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97cc1554a0 con 0x7f97cc0c6460 2023-11-06T21:03:25.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.377+0000 7f97d19ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97cc155930 con 0x7f97cc0aa2d0 2023-11-06T21:03:25.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.377+0000 7f97d19ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97cc154960 con 0x7f97cc0c1840 2023-11-06T21:03:25.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.378+0000 7f97caffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97cc0c1840 0x7f97cc0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.378+0000 7f97caffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97cc0c1840 0x7f97cc0a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50166/0 (socket says 172.21.15.33:50166) 2023-11-06T21:03:25.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.378+0000 7f97caffd700 1 -- 172.21.15.33:0/1842097310 learned_addr learned my addr 172.21.15.33:0/1842097310 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:25.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.378+0000 7f97caffd700 1 -- 172.21.15.33:0/1842097310 >> v1:172.21.15.69:6789/0 conn(0x7f97cc0aa2d0 legacy=0x7f97cc0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.378+0000 7f97caffd700 1 -- 172.21.15.33:0/1842097310 >> v1:172.21.15.33:6789/0 conn(0x7f97cc0c6460 legacy=0x7f97cc0bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.378+0000 7f97caffd700 1 -- 172.21.15.33:0/1842097310 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc154b70 con 0x7f97cc0c1840 2023-11-06T21:03:25.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.378+0000 7f97caffd700 1 --2- 172.21.15.33:0/1842097310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c1840 0x7f97cc0a9de0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7f97bc006fd0 tx=0x7f97bc0090f0).ready entity=mon.0 client_cookie=4953030aac8e9258 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:25.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.378+0000 7f97c9ffb700 1 -- 172.21.15.33:0/1842097310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00b600 con 0x7f97cc0c1840 2023-11-06T21:03:25.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.379+0000 7f97c9ffb700 1 -- 172.21.15.33:0/1842097310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc00b760 con 0x7f97cc0c1840 2023-11-06T21:03:25.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.379+0000 7f97c9ffb700 1 -- 172.21.15.33:0/1842097310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00bae0 con 0x7f97cc0c1840 2023-11-06T21:03:25.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.379+0000 7f97d19ff700 1 -- 172.21.15.33:0/1842097310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c1840 msgr2=0x7f97cc0a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.379+0000 7f97d19ff700 1 --2- 172.21.15.33:0/1842097310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c1840 0x7f97cc0a9de0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7f97bc006fd0 tx=0x7f97bc0090f0).stop 2023-11-06T21:03:25.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.379+0000 7f97d19ff700 1 -- 172.21.15.33:0/1842097310 shutdown_connections 2023-11-06T21:03:25.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.379+0000 7f97d19ff700 1 --2- 172.21.15.33:0/1842097310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c1840 0x7f97cc0a9de0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.379+0000 7f97d19ff700 1 -- 172.21.15.33:0/1842097310 >> 172.21.15.33:0/1842097310 conn(0x7f97cc05ec80 msgr2=0x7f97cc0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:25.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.379+0000 7f97d19ff700 1 -- 172.21.15.33:0/1842097310 shutdown_connections 2023-11-06T21:03:25.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.379+0000 7f97d19ff700 1 -- 172.21.15.33:0/1842097310 wait complete. 2023-11-06T21:03:25.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.381+0000 7f97d19ff700 1 Processor -- start 2023-11-06T21:03:25.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.381+0000 7f97d19ff700 1 -- start start 2023-11-06T21:03:25.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97d19ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97cc0aa2d0 0x7f97cc14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97d19ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc0c1840 0x7f97cc0c0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97d19ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c6460 0x7f97cc0c0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97d19ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc0c0e10 con 0x7f97cc0c6460 2023-11-06T21:03:25.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97d19ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc0aeee0 con 0x7f97cc0c1840 2023-11-06T21:03:25.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97d19ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc0ae2d0 con 0x7f97cc0aa2d0 2023-11-06T21:03:25.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c6460 0x7f97cc0c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c6460 0x7f97cc0c0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50170/0 (socket says 172.21.15.33:50170) 2023-11-06T21:03:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97cb7fe700 1 -- 172.21.15.33:0/4135931871 learned_addr learned my addr 172.21.15.33:0/4135931871 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97caffd700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97cc0aa2d0 0x7f97cc14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.382+0000 7f97ca7fc700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc0c1840 0x7f97cc0c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97cb7fe700 1 -- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97cc0aa2d0 msgr2=0x7f97cc14f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97cb7fe700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97cc0aa2d0 0x7f97cc14f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97cb7fe700 1 -- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc0c1840 msgr2=0x7f97cc0c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97cb7fe700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc0c1840 0x7f97cc0c0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97cb7fe700 1 -- 172.21.15.33:0/4135931871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc009370 con 0x7f97cc0c6460 2023-11-06T21:03:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97ca7fc700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc0c1840 0x7f97cc0c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97cb7fe700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c6460 0x7f97cc0c0920 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f97c400c7c0 tx=0x7f97c4001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97b2ffd700 1 -- 172.21.15.33:0/4135931871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400e4a0 con 0x7f97cc0c6460 2023-11-06T21:03:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97b2ffd700 1 -- 172.21.15.33:0/4135931871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c400e600 con 0x7f97cc0c6460 2023-11-06T21:03:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97d19ff700 1 -- 172.21.15.33:0/4135931871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97cc0ae5b0 con 0x7f97cc0c6460 2023-11-06T21:03:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97b2ffd700 1 -- 172.21.15.33:0/4135931871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400e9a0 con 0x7f97cc0c6460 2023-11-06T21:03:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.383+0000 7f97d19ff700 1 -- 172.21.15.33:0/4135931871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc0aeb70 con 0x7f97cc0c6460 2023-11-06T21:03:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.384+0000 7f97d19ff700 1 -- 172.21.15.33:0/4135931871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc065590 con 0x7f97cc0c6460 2023-11-06T21:03:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.385+0000 7f97b2ffd700 1 -- 172.21.15.33:0/4135931871 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f97c401b3b0 con 0x7f97cc0c6460 2023-11-06T21:03:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.386+0000 7f97b2ffd700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f97b805f270 0x7f97b8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.386+0000 7f97b2ffd700 1 -- 172.21.15.33:0/4135931871 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f97c40799f0 con 0x7f97cc0c6460 2023-11-06T21:03:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.386+0000 7f97caffd700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f97b805f270 0x7f97b8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.387+0000 7f97caffd700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f97b805f270 0x7f97b8061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f97bc009130 tx=0x7f97bc017000).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.389+0000 7f97b2ffd700 1 -- 172.21.15.33:0/4135931871 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f97c4048af0 con 0x7f97cc0c6460 2023-11-06T21:03:25.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.568+0000 7f97d19ff700 1 -- 172.21.15.33:0/4135931871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97cc06c6c0 con 0x7f97cc0c6460 2023-11-06T21:03:25.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.569+0000 7f97b2ffd700 1 -- 172.21.15.33:0/4135931871 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16991 (secure 0 0 0) 0x7f97c4017030 con 0x7f97cc0c6460 2023-11-06T21:03:25.570 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:25.570 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":147,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:03:22.685279+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:03:20.146564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T21:03:20.662302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:03:25.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 -- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f97b805f270 msgr2=0x7f97b8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f97b805f270 0x7f97b8061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f97bc009130 tx=0x7f97bc017000).stop 2023-11-06T21:03:25.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 -- 172.21.15.33:0/4135931871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c6460 msgr2=0x7f97cc0c0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c6460 0x7f97cc0c0920 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f97c400c7c0 tx=0x7f97c4001510).stop 2023-11-06T21:03:25.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 -- 172.21.15.33:0/4135931871 shutdown_connections 2023-11-06T21:03:25.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f97b805f270 0x7f97b8061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97cc0aa2d0 0x7f97cc14f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97cc0c6460 0x7f97cc0c0920 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 --2- 172.21.15.33:0/4135931871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97cc0c1840 0x7f97cc0c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 -- 172.21.15.33:0/4135931871 >> 172.21.15.33:0/4135931871 conn(0x7f97cc05ec80 msgr2=0x7f97cc1517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:25.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 -- 172.21.15.33:0/4135931871 shutdown_connections 2023-11-06T21:03:25.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.571+0000 7f97d19ff700 1 -- 172.21.15.33:0/4135931871 wait complete. 2023-11-06T21:03:25.589 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T21:03:25.590 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T21:03:25.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.786+0000 7f4452fff700 1 Processor -- start 2023-11-06T21:03:25.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.786+0000 7f4452fff700 1 -- start start 2023-11-06T21:03:25.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.787+0000 7f4452fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f444c05e1b0 0x7f444c063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f4452fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f444c0c3680 0x7f444c154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f4452fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c0c31f0 con 0x7f444c063140 2023-11-06T21:03:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f4452fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c06a0f0 con 0x7f444c0c3680 2023-11-06T21:03:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f4452fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c0c2920 con 0x7f444c05e1b0 2023-11-06T21:03:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f445159c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f444c0c3680 0x7f444c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f445159c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f444c0c3680 0x7f444c154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50172/0 (socket says 172.21.15.33:50172) 2023-11-06T21:03:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f445159c700 1 -- 172.21.15.33:0/159493002 learned_addr learned my addr 172.21.15.33:0/159493002 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f4450d9b700 1 --1- 172.21.15.33:0/159493002 >> v1:172.21.15.33:6789/0 conn(0x7f444c063140 0x7f444c0c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36592/0 (socket says 172.21.15.33:36592) 2023-11-06T21:03:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f444bfff700 1 --2- 172.21.15.33:0/159493002 >> v2:172.21.15.69:3300/0 conn(0x7f444c05e1b0 0x7f444c063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f445159c700 1 -- 172.21.15.33:0/159493002 >> v2:172.21.15.69:3300/0 conn(0x7f444c05e1b0 msgr2=0x7f444c063520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f445159c700 1 --2- 172.21.15.33:0/159493002 >> v2:172.21.15.69:3300/0 conn(0x7f444c05e1b0 0x7f444c063520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f445159c700 1 -- 172.21.15.33:0/159493002 >> v1:172.21.15.33:6789/0 conn(0x7f444c063140 legacy=0x7f444c0c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f445159c700 1 -- 172.21.15.33:0/159493002 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c05e640 con 0x7f444c0c3680 2023-11-06T21:03:25.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.788+0000 7f445159c700 1 --2- 172.21.15.33:0/159493002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c0c3680 0x7f444c154050 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7f444c0c1890 tx=0x7f4444009840).ready entity=mon.0 client_cookie=e0308ed2a722b872 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:25.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f444b7fe700 1 -- 172.21.15.33:0/159493002 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400e070 con 0x7f444c0c3680 2023-11-06T21:03:25.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f444b7fe700 1 -- 172.21.15.33:0/159493002 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444400ba50 con 0x7f444c0c3680 2023-11-06T21:03:25.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f444b7fe700 1 -- 172.21.15.33:0/159493002 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400bdd0 con 0x7f444c0c3680 2023-11-06T21:03:25.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f4452fff700 1 -- 172.21.15.33:0/159493002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c0c3680 msgr2=0x7f444c154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f4452fff700 1 --2- 172.21.15.33:0/159493002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c0c3680 0x7f444c154050 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7f444c0c1890 tx=0x7f4444009840).stop 2023-11-06T21:03:25.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f4452fff700 1 -- 172.21.15.33:0/159493002 shutdown_connections 2023-11-06T21:03:25.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f4452fff700 1 --2- 172.21.15.33:0/159493002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c0c3680 0x7f444c154050 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f4452fff700 1 --2- 172.21.15.33:0/159493002 >> v2:172.21.15.69:3300/0 conn(0x7f444c05e1b0 0x7f444c063520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f4452fff700 1 -- 172.21.15.33:0/159493002 >> 172.21.15.33:0/159493002 conn(0x7f444c0c7d60 msgr2=0x7f444c0c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:25.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f4452fff700 1 -- 172.21.15.33:0/159493002 shutdown_connections 2023-11-06T21:03:25.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.789+0000 7f4452fff700 1 -- 172.21.15.33:0/159493002 wait complete. 2023-11-06T21:03:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.791+0000 7f4452fff700 1 Processor -- start 2023-11-06T21:03:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.791+0000 7f4452fff700 1 -- start start 2023-11-06T21:03:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.792+0000 7f4452fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c063140 0x7f444c154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.792+0000 7f4452fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c0c3680 0x7f444c142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.792+0000 7f4452fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f444c1432a0 0x7f444c156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.792+0000 7f4452fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c155f50 con 0x7f444c063140 2023-11-06T21:03:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.792+0000 7f4452fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c155340 con 0x7f444c0c3680 2023-11-06T21:03:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.792+0000 7f4452fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c155620 con 0x7f444c1432a0 2023-11-06T21:03:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.792+0000 7f4450d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c063140 0x7f444c154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.792+0000 7f4450d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c063140 0x7f444c154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50186/0 (socket says 172.21.15.33:50186) 2023-11-06T21:03:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.792+0000 7f4450d9b700 1 -- 172.21.15.33:0/573825765 learned_addr learned my addr 172.21.15.33:0/573825765 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f444bfff700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c0c3680 0x7f444c142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f445159c700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f444c1432a0 0x7f444c156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4450d9b700 1 -- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f444c1432a0 msgr2=0x7f444c156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4450d9b700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f444c1432a0 0x7f444c156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4450d9b700 1 -- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c0c3680 msgr2=0x7f444c142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4450d9b700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c0c3680 0x7f444c142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4450d9b700 1 -- 172.21.15.33:0/573825765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444009ae0 con 0x7f444c063140 2023-11-06T21:03:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4450d9b700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c063140 0x7f444c154a40 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f4440007fd0 tx=0x7f4440007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4448ff9700 1 -- 172.21.15.33:0/573825765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400072e0 con 0x7f444c063140 2023-11-06T21:03:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4448ff9700 1 -- 172.21.15.33:0/573825765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4440007440 con 0x7f444c063140 2023-11-06T21:03:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4448ff9700 1 -- 172.21.15.33:0/573825765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400058f0 con 0x7f444c063140 2023-11-06T21:03:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4452fff700 1 -- 172.21.15.33:0/573825765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c155900 con 0x7f444c063140 2023-11-06T21:03:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.793+0000 7f4452fff700 1 -- 172.21.15.33:0/573825765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c154f30 con 0x7f444c063140 2023-11-06T21:03:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.795+0000 7f4452fff700 1 -- 172.21.15.33:0/573825765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c064420 con 0x7f444c063140 2023-11-06T21:03:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.795+0000 7f4448ff9700 1 -- 172.21.15.33:0/573825765 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4440005ce0 con 0x7f444c063140 2023-11-06T21:03:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.796+0000 7f4448ff9700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f443005f270 0x7f4430061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.796+0000 7f4448ff9700 1 -- 172.21.15.33:0/573825765 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4440079c70 con 0x7f444c063140 2023-11-06T21:03:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.797+0000 7f444bfff700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f443005f270 0x7f4430061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:25.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.797+0000 7f444bfff700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f443005f270 0x7f4430061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f4438006fd0 tx=0x7f4438008040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:25.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.800+0000 7f4448ff9700 1 -- 172.21.15.33:0/573825765 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f444004ad00 con 0x7f444c063140 2023-11-06T21:03:25.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.979+0000 7f4452fff700 1 -- 172.21.15.33:0/573825765 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f444c065d70 con 0x7f443005f270 2023-11-06T21:03:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.993+0000 7f4448ff9700 1 -- 172.21.15.33:0/573825765 <== mgr.6692 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361880 (secure 0 0 0) 0x7f444c065d70 con 0x7f443005f270 2023-11-06T21:03:25.998 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.997+0000 7f4452fff700 1 -- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f443005f270 msgr2=0x7f4430061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f443005f270 0x7f4430061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f4438006fd0 tx=0x7f4438008040).stop 2023-11-06T21:03:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 -- 172.21.15.33:0/573825765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c063140 msgr2=0x7f444c154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c063140 0x7f444c154a40 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f4440007fd0 tx=0x7f4440007df0).stop 2023-11-06T21:03:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 -- 172.21.15.33:0/573825765 shutdown_connections 2023-11-06T21:03:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f443005f270 0x7f4430061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f444c1432a0 0x7f444c156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c063140 0x7f444c154a40 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 --2- 172.21.15.33:0/573825765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c0c3680 0x7f444c142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 -- 172.21.15.33:0/573825765 >> 172.21.15.33:0/573825765 conn(0x7f444c0c7d60 msgr2=0x7f444c0c5010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 -- 172.21.15.33:0/573825765 shutdown_connections 2023-11-06T21:03:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:25.998+0000 7f4452fff700 1 -- 172.21.15.33:0/573825765 wait complete. 2023-11-06T21:03:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T21:03:26.016 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T21:03:24.347070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2423828,"kb_used_data":14292,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752550892,"statfs":{"total":773094113280,"available":770612113408,"internally_reserved":0,"allocated":14635008,"data_stored":7401476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20937,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-73,"num_write_kb":-48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.003725"},"pg_stats":[{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719502+0000","last_change":"2023-11-06T21:03:21.716328+0000","last_active":"2023-11-06T21:03:21.719502+0000","last_peered":"2023-11-06T21:03:21.719502+0000","last_clean":"2023-11-06T21:03:21.719502+0000","last_became_active":"2023-11-06T21:03:21.714842+0000","last_became_peered":"2023-11-06T21:03:21.714842+0000","last_unstale":"2023-11-06T21:03:21.719502+0000","last_undegraded":"2023-11-06T21:03:21.719502+0000","last_fullsized":"2023-11-06T21:03:21.719502+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3f","version":"146'3","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.723126+0000","last_change":"2023-11-06T21:03:21.233431+0000","last_active":"2023-11-06T21:03:21.723126+0000","last_peered":"2023-11-06T21:03:21.723126+0000","last_clean":"2023-11-06T21:03:21.723126+0000","last_became_active":"2023-11-06T21:03:21.232386+0000","last_became_peered":"2023-11-06T21:03:21.232386+0000","last_unstale":"2023-11-06T21:03:21.723126+0000","last_undegraded":"2023-11-06T21:03:21.723126+0000","last_fullsized":"2023-11-06T21:03:21.723126+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.718248+0000","last_change":"2023-11-06T21:03:21.715517+0000","last_active":"2023-11-06T21:03:21.718248+0000","last_peered":"2023-11-06T21:03:21.718248+0000","last_clean":"2023-11-06T21:03:21.718248+0000","last_became_active":"2023-11-06T21:03:21.715227+0000","last_became_peered":"2023-11-06T21:03:21.715227+0000","last_unstale":"2023-11-06T21:03:21.718248+0000","last_undegraded":"2023-11-06T21:03:21.718248+0000","last_fullsized":"2023-11-06T21:03:21.718248+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.698683+0000","last_change":"2023-11-06T21:03:21.232828+0000","last_active":"2023-11-06T21:03:21.698683+0000","last_peered":"2023-11-06T21:03:21.698683+0000","last_clean":"2023-11-06T21:03:21.698683+0000","last_became_active":"2023-11-06T21:03:21.232202+0000","last_became_peered":"2023-11-06T21:03:21.232202+0000","last_unstale":"2023-11-06T21:03:21.698683+0000","last_undegraded":"2023-11-06T21:03:21.698683+0000","last_fullsized":"2023-11-06T21:03:21.698683+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.712831+0000","last_change":"2023-11-06T21:03:21.702662+0000","last_active":"2023-11-06T21:03:21.712831+0000","last_peered":"2023-11-06T21:03:21.712831+0000","last_clean":"2023-11-06T21:03:21.712831+0000","last_became_active":"2023-11-06T21:03:21.702378+0000","last_became_peered":"2023-11-06T21:03:21.702378+0000","last_unstale":"2023-11-06T21:03:21.712831+0000","last_undegraded":"2023-11-06T21:03:21.712831+0000","last_fullsized":"2023-11-06T21:03:21.712831+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693771+0000","last_change":"2023-11-06T21:03:21.225199+0000","last_active":"2023-11-06T21:03:21.693771+0000","last_peered":"2023-11-06T21:03:21.693771+0000","last_clean":"2023-11-06T21:03:21.693771+0000","last_became_active":"2023-11-06T21:03:21.224918+0000","last_became_peered":"2023-11-06T21:03:21.224918+0000","last_unstale":"2023-11-06T21:03:21.693771+0000","last_undegraded":"2023-11-06T21:03:21.693771+0000","last_fullsized":"2023-11-06T21:03:21.693771+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726200+0000","last_change":"2023-11-06T21:03:21.714432+0000","last_active":"2023-11-06T21:03:21.726200+0000","last_peered":"2023-11-06T21:03:21.726200+0000","last_clean":"2023-11-06T21:03:21.726200+0000","last_became_active":"2023-11-06T21:03:21.713732+0000","last_became_peered":"2023-11-06T21:03:21.713732+0000","last_unstale":"2023-11-06T21:03:21.726200+0000","last_undegraded":"2023-11-06T21:03:21.726200+0000","last_fullsized":"2023-11-06T21:03:21.726200+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3c","version":"146'2","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726934+0000","last_change":"2023-11-06T21:03:21.230736+0000","last_active":"2023-11-06T21:03:21.726934+0000","last_peered":"2023-11-06T21:03:21.726934+0000","last_clean":"2023-11-06T21:03:21.726934+0000","last_became_active":"2023-11-06T21:03:21.230381+0000","last_became_peered":"2023-11-06T21:03:21.230381+0000","last_unstale":"2023-11-06T21:03:21.726934+0000","last_undegraded":"2023-11-06T21:03:21.726934+0000","last_fullsized":"2023-11-06T21:03:21.726934+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.715218+0000","last_change":"2023-11-06T21:03:21.707538+0000","last_active":"2023-11-06T21:03:21.715218+0000","last_peered":"2023-11-06T21:03:21.715218+0000","last_clean":"2023-11-06T21:03:21.715218+0000","last_became_active":"2023-11-06T21:03:21.707165+0000","last_became_peered":"2023-11-06T21:03:21.707165+0000","last_unstale":"2023-11-06T21:03:21.715218+0000","last_undegraded":"2023-11-06T21:03:21.715218+0000","last_fullsized":"2023-11-06T21:03:21.715218+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.711564+0000","last_change":"2023-11-06T21:03:21.231433+0000","last_active":"2023-11-06T21:03:21.711564+0000","last_peered":"2023-11-06T21:03:21.711564+0000","last_clean":"2023-11-06T21:03:21.711564+0000","last_became_active":"2023-11-06T21:03:21.231142+0000","last_became_peered":"2023-11-06T21:03:21.231142+0000","last_unstale":"2023-11-06T21:03:21.711564+0000","last_undegraded":"2023-11-06T21:03:21.711564+0000","last_fullsized":"2023-11-06T21:03:21.711564+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716811+0000","last_change":"2023-11-06T21:03:21.711731+0000","last_active":"2023-11-06T21:03:21.716811+0000","last_peered":"2023-11-06T21:03:21.716811+0000","last_clean":"2023-11-06T21:03:21.716811+0000","last_became_active":"2023-11-06T21:03:21.710021+0000","last_became_peered":"2023-11-06T21:03:21.710021+0000","last_unstale":"2023-11-06T21:03:21.716811+0000","last_undegraded":"2023-11-06T21:03:21.716811+0000","last_fullsized":"2023-11-06T21:03:21.716811+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3a","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716302+0000","last_change":"2023-11-06T21:03:21.231979+0000","last_active":"2023-11-06T21:03:21.716302+0000","last_peered":"2023-11-06T21:03:21.716302+0000","last_clean":"2023-11-06T21:03:21.716302+0000","last_became_active":"2023-11-06T21:03:21.231191+0000","last_became_peered":"2023-11-06T21:03:21.231191+0000","last_unstale":"2023-11-06T21:03:21.716302+0000","last_undegraded":"2023-11-06T21:03:21.716302+0000","last_fullsized":"2023-11-06T21:03:21.716302+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.714466+0000","last_change":"2023-11-06T21:03:21.705387+0000","last_active":"2023-11-06T21:03:21.714466+0000","last_peered":"2023-11-06T21:03:21.714466+0000","last_clean":"2023-11-06T21:03:21.714466+0000","last_became_active":"2023-11-06T21:03:21.705124+0000","last_became_peered":"2023-11-06T21:03:21.705124+0000","last_unstale":"2023-11-06T21:03:21.714466+0000","last_undegraded":"2023-11-06T21:03:21.714466+0000","last_fullsized":"2023-11-06T21:03:21.714466+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.689695+0000","last_change":"2023-11-06T21:03:21.227622+0000","last_active":"2023-11-06T21:03:21.689695+0000","last_peered":"2023-11-06T21:03:21.689695+0000","last_clean":"2023-11-06T21:03:21.689695+0000","last_became_active":"2023-11-06T21:03:21.227277+0000","last_became_peered":"2023-11-06T21:03:21.227277+0000","last_unstale":"2023-11-06T21:03:21.689695+0000","last_undegraded":"2023-11-06T21:03:21.689695+0000","last_fullsized":"2023-11-06T21:03:21.689695+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726756+0000","last_change":"2023-11-06T21:03:21.723295+0000","last_active":"2023-11-06T21:03:21.726756+0000","last_peered":"2023-11-06T21:03:21.726756+0000","last_clean":"2023-11-06T21:03:21.726756+0000","last_became_active":"2023-11-06T21:03:21.723007+0000","last_became_peered":"2023-11-06T21:03:21.723007+0000","last_unstale":"2023-11-06T21:03:21.726756+0000","last_undegraded":"2023-11-06T21:03:21.726756+0000","last_fullsized":"2023-11-06T21:03:21.726756+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.689571+0000","last_change":"2023-11-06T21:03:21.210735+0000","last_active":"2023-11-06T21:03:21.689571+0000","last_peered":"2023-11-06T21:03:21.689571+0000","last_clean":"2023-11-06T21:03:21.689571+0000","last_became_active":"2023-11-06T21:03:21.210444+0000","last_became_peered":"2023-11-06T21:03:21.210444+0000","last_unstale":"2023-11-06T21:03:21.689571+0000","last_undegraded":"2023-11-06T21:03:21.689571+0000","last_fullsized":"2023-11-06T21:03:21.689571+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.728031+0000","last_change":"2023-11-06T21:03:21.726836+0000","last_active":"2023-11-06T21:03:21.728031+0000","last_peered":"2023-11-06T21:03:21.728031+0000","last_clean":"2023-11-06T21:03:21.728031+0000","last_became_active":"2023-11-06T21:03:21.726525+0000","last_became_peered":"2023-11-06T21:03:21.726525+0000","last_unstale":"2023-11-06T21:03:21.728031+0000","last_undegraded":"2023-11-06T21:03:21.728031+0000","last_fullsized":"2023-11-06T21:03:21.728031+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.704856+0000","last_change":"2023-11-06T21:03:21.227201+0000","last_active":"2023-11-06T21:03:21.704856+0000","last_peered":"2023-11-06T21:03:21.704856+0000","last_clean":"2023-11-06T21:03:21.704856+0000","last_became_active":"2023-11-06T21:03:21.226904+0000","last_became_peered":"2023-11-06T21:03:21.226904+0000","last_unstale":"2023-11-06T21:03:21.704856+0000","last_undegraded":"2023-11-06T21:03:21.704856+0000","last_fullsized":"2023-11-06T21:03:21.704856+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716415+0000","last_change":"2023-11-06T21:03:21.699822+0000","last_active":"2023-11-06T21:03:21.716415+0000","last_peered":"2023-11-06T21:03:21.716415+0000","last_clean":"2023-11-06T21:03:21.716415+0000","last_became_active":"2023-11-06T21:03:21.699554+0000","last_became_peered":"2023-11-06T21:03:21.699554+0000","last_unstale":"2023-11-06T21:03:21.716415+0000","last_undegraded":"2023-11-06T21:03:21.716415+0000","last_fullsized":"2023-11-06T21:03:21.716415+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.694262+0000","last_change":"2023-11-06T21:03:21.233807+0000","last_active":"2023-11-06T21:03:21.694262+0000","last_peered":"2023-11-06T21:03:21.694262+0000","last_clean":"2023-11-06T21:03:21.694262+0000","last_became_active":"2023-11-06T21:03:21.233535+0000","last_became_peered":"2023-11-06T21:03:21.233535+0000","last_unstale":"2023-11-06T21:03:21.694262+0000","last_undegraded":"2023-11-06T21:03:21.694262+0000","last_fullsized":"2023-11-06T21:03:21.694262+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.724676+0000","last_change":"2023-11-06T21:03:21.720216+0000","last_active":"2023-11-06T21:03:21.724676+0000","last_peered":"2023-11-06T21:03:21.724676+0000","last_clean":"2023-11-06T21:03:21.724676+0000","last_became_active":"2023-11-06T21:03:21.719882+0000","last_became_peered":"2023-11-06T21:03:21.719882+0000","last_unstale":"2023-11-06T21:03:21.724676+0000","last_undegraded":"2023-11-06T21:03:21.724676+0000","last_fullsized":"2023-11-06T21:03:21.724676+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691263+0000","last_change":"2023-11-06T21:03:21.225241+0000","last_active":"2023-11-06T21:03:21.691263+0000","last_peered":"2023-11-06T21:03:21.691263+0000","last_clean":"2023-11-06T21:03:21.691263+0000","last_became_active":"2023-11-06T21:03:21.224975+0000","last_became_peered":"2023-11-06T21:03:21.224975+0000","last_unstale":"2023-11-06T21:03:21.691263+0000","last_undegraded":"2023-11-06T21:03:21.691263+0000","last_fullsized":"2023-11-06T21:03:21.691263+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725809+0000","last_change":"2023-11-06T21:03:21.720263+0000","last_active":"2023-11-06T21:03:21.725809+0000","last_peered":"2023-11-06T21:03:21.725809+0000","last_clean":"2023-11-06T21:03:21.725809+0000","last_became_active":"2023-11-06T21:03:21.719974+0000","last_became_peered":"2023-11-06T21:03:21.719974+0000","last_unstale":"2023-11-06T21:03:21.725809+0000","last_undegraded":"2023-11-06T21:03:21.725809+0000","last_fullsized":"2023-11-06T21:03:21.725809+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.34","version":"146'4","reported_seq":14,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.718301+0000","last_change":"2023-11-06T21:03:21.233255+0000","last_active":"2023-11-06T21:03:21.718301+0000","last_peered":"2023-11-06T21:03:21.718301+0000","last_clean":"2023-11-06T21:03:21.718301+0000","last_became_active":"2023-11-06T21:03:21.232129+0000","last_became_peered":"2023-11-06T21:03:21.232129+0000","last_unstale":"2023-11-06T21:03:21.718301+0000","last_undegraded":"2023-11-06T21:03:21.718301+0000","last_fullsized":"2023-11-06T21:03:21.718301+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719750+0000","last_change":"2023-11-06T21:03:21.717930+0000","last_active":"2023-11-06T21:03:21.719750+0000","last_peered":"2023-11-06T21:03:21.719750+0000","last_clean":"2023-11-06T21:03:21.719750+0000","last_became_active":"2023-11-06T21:03:21.717637+0000","last_became_peered":"2023-11-06T21:03:21.717637+0000","last_unstale":"2023-11-06T21:03:21.719750+0000","last_undegraded":"2023-11-06T21:03:21.719750+0000","last_fullsized":"2023-11-06T21:03:21.719750+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.33","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.718767+0000","last_change":"2023-11-06T21:03:21.228252+0000","last_active":"2023-11-06T21:03:21.718767+0000","last_peered":"2023-11-06T21:03:21.718767+0000","last_clean":"2023-11-06T21:03:21.718767+0000","last_became_active":"2023-11-06T21:03:21.227965+0000","last_became_peered":"2023-11-06T21:03:21.227965+0000","last_unstale":"2023-11-06T21:03:21.718767+0000","last_undegraded":"2023-11-06T21:03:21.718767+0000","last_fullsized":"2023-11-06T21:03:21.718767+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.711898+0000","last_change":"2023-11-06T21:03:21.706612+0000","last_active":"2023-11-06T21:03:21.711898+0000","last_peered":"2023-11-06T21:03:21.711898+0000","last_clean":"2023-11-06T21:03:21.711898+0000","last_became_active":"2023-11-06T21:03:21.706299+0000","last_became_peered":"2023-11-06T21:03:21.706299+0000","last_unstale":"2023-11-06T21:03:21.711898+0000","last_undegraded":"2023-11-06T21:03:21.711898+0000","last_fullsized":"2023-11-06T21:03:21.711898+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.705894+0000","last_change":"2023-11-06T21:03:21.224674+0000","last_active":"2023-11-06T21:03:21.705894+0000","last_peered":"2023-11-06T21:03:21.705894+0000","last_clean":"2023-11-06T21:03:21.705894+0000","last_became_active":"2023-11-06T21:03:21.224087+0000","last_became_peered":"2023-11-06T21:03:21.224087+0000","last_unstale":"2023-11-06T21:03:21.705894+0000","last_undegraded":"2023-11-06T21:03:21.705894+0000","last_fullsized":"2023-11-06T21:03:21.705894+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719827+0000","last_change":"2023-11-06T21:03:21.717340+0000","last_active":"2023-11-06T21:03:21.719827+0000","last_peered":"2023-11-06T21:03:21.719827+0000","last_clean":"2023-11-06T21:03:21.719827+0000","last_became_active":"2023-11-06T21:03:21.717068+0000","last_became_peered":"2023-11-06T21:03:21.717068+0000","last_unstale":"2023-11-06T21:03:21.719827+0000","last_undegraded":"2023-11-06T21:03:21.719827+0000","last_fullsized":"2023-11-06T21:03:21.719827+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.706433+0000","last_change":"2023-11-06T21:03:21.217712+0000","last_active":"2023-11-06T21:03:21.706433+0000","last_peered":"2023-11-06T21:03:21.706433+0000","last_clean":"2023-11-06T21:03:21.706433+0000","last_became_active":"2023-11-06T21:03:21.217431+0000","last_became_peered":"2023-11-06T21:03:21.217431+0000","last_unstale":"2023-11-06T21:03:21.706433+0000","last_undegraded":"2023-11-06T21:03:21.706433+0000","last_fullsized":"2023-11-06T21:03:21.706433+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716668+0000","last_change":"2023-11-06T21:03:21.709166+0000","last_active":"2023-11-06T21:03:21.716668+0000","last_peered":"2023-11-06T21:03:21.716668+0000","last_clean":"2023-11-06T21:03:21.716668+0000","last_became_active":"2023-11-06T21:03:21.708145+0000","last_became_peered":"2023-11-06T21:03:21.708145+0000","last_unstale":"2023-11-06T21:03:21.716668+0000","last_undegraded":"2023-11-06T21:03:21.716668+0000","last_fullsized":"2023-11-06T21:03:21.716668+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.30","version":"146'4","reported_seq":14,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.721221+0000","last_change":"2023-11-06T21:03:21.231178+0000","last_active":"2023-11-06T21:03:21.721221+0000","last_peered":"2023-11-06T21:03:21.721221+0000","last_clean":"2023-11-06T21:03:21.721221+0000","last_became_active":"2023-11-06T21:03:21.230797+0000","last_became_peered":"2023-11-06T21:03:21.230797+0000","last_unstale":"2023-11-06T21:03:21.721221+0000","last_undegraded":"2023-11-06T21:03:21.721221+0000","last_fullsized":"2023-11-06T21:03:21.721221+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.724701+0000","last_change":"2023-11-06T21:03:21.720441+0000","last_active":"2023-11-06T21:03:21.724701+0000","last_peered":"2023-11-06T21:03:21.724701+0000","last_clean":"2023-11-06T21:03:21.724701+0000","last_became_active":"2023-11-06T21:03:21.720104+0000","last_became_peered":"2023-11-06T21:03:21.720104+0000","last_unstale":"2023-11-06T21:03:21.724701+0000","last_undegraded":"2023-11-06T21:03:21.724701+0000","last_fullsized":"2023-11-06T21:03:21.724701+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692898+0000","last_change":"2023-11-06T21:03:21.232111+0000","last_active":"2023-11-06T21:03:21.692898+0000","last_peered":"2023-11-06T21:03:21.692898+0000","last_clean":"2023-11-06T21:03:21.692898+0000","last_became_active":"2023-11-06T21:03:21.231796+0000","last_became_peered":"2023-11-06T21:03:21.231796+0000","last_unstale":"2023-11-06T21:03:21.692898+0000","last_undegraded":"2023-11-06T21:03:21.692898+0000","last_fullsized":"2023-11-06T21:03:21.692898+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725098+0000","last_change":"2023-11-06T21:03:21.719951+0000","last_active":"2023-11-06T21:03:21.725098+0000","last_peered":"2023-11-06T21:03:21.725098+0000","last_clean":"2023-11-06T21:03:21.725098+0000","last_became_active":"2023-11-06T21:03:21.717893+0000","last_became_peered":"2023-11-06T21:03:21.717893+0000","last_unstale":"2023-11-06T21:03:21.725098+0000","last_undegraded":"2023-11-06T21:03:21.725098+0000","last_fullsized":"2023-11-06T21:03:21.725098+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691214+0000","last_change":"2023-11-06T21:03:21.218758+0000","last_active":"2023-11-06T21:03:21.691214+0000","last_peered":"2023-11-06T21:03:21.691214+0000","last_clean":"2023-11-06T21:03:21.691214+0000","last_became_active":"2023-11-06T21:03:21.218094+0000","last_became_peered":"2023-11-06T21:03:21.218094+0000","last_unstale":"2023-11-06T21:03:21.691214+0000","last_undegraded":"2023-11-06T21:03:21.691214+0000","last_fullsized":"2023-11-06T21:03:21.691214+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.722571+0000","last_change":"2023-11-06T21:03:21.719952+0000","last_active":"2023-11-06T21:03:21.722571+0000","last_peered":"2023-11-06T21:03:21.722571+0000","last_clean":"2023-11-06T21:03:21.722571+0000","last_became_active":"2023-11-06T21:03:21.719160+0000","last_became_peered":"2023-11-06T21:03:21.719160+0000","last_unstale":"2023-11-06T21:03:21.722571+0000","last_undegraded":"2023-11-06T21:03:21.722571+0000","last_fullsized":"2023-11-06T21:03:21.722571+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.717408+0000","last_change":"2023-11-06T21:03:21.230392+0000","last_active":"2023-11-06T21:03:21.717408+0000","last_peered":"2023-11-06T21:03:21.717408+0000","last_clean":"2023-11-06T21:03:21.717408+0000","last_became_active":"2023-11-06T21:03:21.229515+0000","last_became_peered":"2023-11-06T21:03:21.229515+0000","last_unstale":"2023-11-06T21:03:21.717408+0000","last_undegraded":"2023-11-06T21:03:21.717408+0000","last_fullsized":"2023-11-06T21:03:21.717408+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.718040+0000","last_change":"2023-11-06T21:03:21.716474+0000","last_active":"2023-11-06T21:03:21.718040+0000","last_peered":"2023-11-06T21:03:21.718040+0000","last_clean":"2023-11-06T21:03:21.718040+0000","last_became_active":"2023-11-06T21:03:21.716211+0000","last_became_peered":"2023-11-06T21:03:21.716211+0000","last_unstale":"2023-11-06T21:03:21.718040+0000","last_undegraded":"2023-11-06T21:03:21.718040+0000","last_fullsized":"2023-11-06T21:03:21.718040+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693121+0000","last_change":"2023-11-06T21:03:21.233090+0000","last_active":"2023-11-06T21:03:21.693121+0000","last_peered":"2023-11-06T21:03:21.693121+0000","last_clean":"2023-11-06T21:03:21.693121+0000","last_became_active":"2023-11-06T21:03:21.231497+0000","last_became_peered":"2023-11-06T21:03:21.231497+0000","last_unstale":"2023-11-06T21:03:21.693121+0000","last_undegraded":"2023-11-06T21:03:21.693121+0000","last_fullsized":"2023-11-06T21:03:21.693121+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.723592+0000","last_change":"2023-11-06T21:03:21.721977+0000","last_active":"2023-11-06T21:03:21.723592+0000","last_peered":"2023-11-06T21:03:21.723592+0000","last_clean":"2023-11-06T21:03:21.723592+0000","last_became_active":"2023-11-06T21:03:21.720055+0000","last_became_peered":"2023-11-06T21:03:21.720055+0000","last_unstale":"2023-11-06T21:03:21.723592+0000","last_undegraded":"2023-11-06T21:03:21.723592+0000","last_fullsized":"2023-11-06T21:03:21.723592+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.727318+0000","last_change":"2023-11-06T21:03:21.228257+0000","last_active":"2023-11-06T21:03:21.727318+0000","last_peered":"2023-11-06T21:03:21.727318+0000","last_clean":"2023-11-06T21:03:21.727318+0000","last_became_active":"2023-11-06T21:03:21.227350+0000","last_became_peered":"2023-11-06T21:03:21.227350+0000","last_unstale":"2023-11-06T21:03:21.727318+0000","last_undegraded":"2023-11-06T21:03:21.727318+0000","last_fullsized":"2023-11-06T21:03:21.727318+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.711877+0000","last_change":"2023-11-06T21:03:21.706516+0000","last_active":"2023-11-06T21:03:21.711877+0000","last_peered":"2023-11-06T21:03:21.711877+0000","last_clean":"2023-11-06T21:03:21.711877+0000","last_became_active":"2023-11-06T21:03:21.706165+0000","last_became_peered":"2023-11-06T21:03:21.706165+0000","last_unstale":"2023-11-06T21:03:21.711877+0000","last_undegraded":"2023-11-06T21:03:21.711877+0000","last_fullsized":"2023-11-06T21:03:21.711877+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691523+0000","last_change":"2023-11-06T21:03:21.212700+0000","last_active":"2023-11-06T21:03:21.691523+0000","last_peered":"2023-11-06T21:03:21.691523+0000","last_clean":"2023-11-06T21:03:21.691523+0000","last_became_active":"2023-11-06T21:03:21.212367+0000","last_became_peered":"2023-11-06T21:03:21.212367+0000","last_unstale":"2023-11-06T21:03:21.691523+0000","last_undegraded":"2023-11-06T21:03:21.691523+0000","last_fullsized":"2023-11-06T21:03:21.691523+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.722781+0000","last_change":"2023-11-06T21:03:21.719540+0000","last_active":"2023-11-06T21:03:21.722781+0000","last_peered":"2023-11-06T21:03:21.722781+0000","last_clean":"2023-11-06T21:03:21.722781+0000","last_became_active":"2023-11-06T21:03:21.717861+0000","last_became_peered":"2023-11-06T21:03:21.717861+0000","last_unstale":"2023-11-06T21:03:21.722781+0000","last_undegraded":"2023-11-06T21:03:21.722781+0000","last_fullsized":"2023-11-06T21:03:21.722781+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.699075+0000","last_change":"2023-11-06T21:03:21.226920+0000","last_active":"2023-11-06T21:03:21.699075+0000","last_peered":"2023-11-06T21:03:21.699075+0000","last_clean":"2023-11-06T21:03:21.699075+0000","last_became_active":"2023-11-06T21:03:21.226630+0000","last_became_peered":"2023-11-06T21:03:21.226630+0000","last_unstale":"2023-11-06T21:03:21.699075+0000","last_undegraded":"2023-11-06T21:03:21.699075+0000","last_fullsized":"2023-11-06T21:03:21.699075+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725391+0000","last_change":"2023-11-06T21:03:21.720772+0000","last_active":"2023-11-06T21:03:21.725391+0000","last_peered":"2023-11-06T21:03:21.725391+0000","last_clean":"2023-11-06T21:03:21.725391+0000","last_became_active":"2023-11-06T21:03:21.720488+0000","last_became_peered":"2023-11-06T21:03:21.720488+0000","last_unstale":"2023-11-06T21:03:21.725391+0000","last_undegraded":"2023-11-06T21:03:21.725391+0000","last_fullsized":"2023-11-06T21:03:21.725391+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.699909+0000","last_change":"2023-11-06T21:03:21.223799+0000","last_active":"2023-11-06T21:03:21.699909+0000","last_peered":"2023-11-06T21:03:21.699909+0000","last_clean":"2023-11-06T21:03:21.699909+0000","last_became_active":"2023-11-06T21:03:21.223503+0000","last_became_peered":"2023-11-06T21:03:21.223503+0000","last_unstale":"2023-11-06T21:03:21.699909+0000","last_undegraded":"2023-11-06T21:03:21.699909+0000","last_fullsized":"2023-11-06T21:03:21.699909+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725102+0000","last_change":"2023-11-06T21:03:21.723673+0000","last_active":"2023-11-06T21:03:21.725102+0000","last_peered":"2023-11-06T21:03:21.725102+0000","last_clean":"2023-11-06T21:03:21.725102+0000","last_became_active":"2023-11-06T21:03:21.723387+0000","last_became_peered":"2023-11-06T21:03:21.723387+0000","last_unstale":"2023-11-06T21:03:21.725102+0000","last_undegraded":"2023-11-06T21:03:21.725102+0000","last_fullsized":"2023-11-06T21:03:21.725102+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691593+0000","last_change":"2023-11-06T21:03:21.220090+0000","last_active":"2023-11-06T21:03:21.691593+0000","last_peered":"2023-11-06T21:03:21.691593+0000","last_clean":"2023-11-06T21:03:21.691593+0000","last_became_active":"2023-11-06T21:03:21.219322+0000","last_became_peered":"2023-11-06T21:03:21.219322+0000","last_unstale":"2023-11-06T21:03:21.691593+0000","last_undegraded":"2023-11-06T21:03:21.691593+0000","last_fullsized":"2023-11-06T21:03:21.691593+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.727400+0000","last_change":"2023-11-06T21:03:21.724962+0000","last_active":"2023-11-06T21:03:21.727400+0000","last_peered":"2023-11-06T21:03:21.727400+0000","last_clean":"2023-11-06T21:03:21.727400+0000","last_became_active":"2023-11-06T21:03:21.720778+0000","last_became_peered":"2023-11-06T21:03:21.720778+0000","last_unstale":"2023-11-06T21:03:21.727400+0000","last_undegraded":"2023-11-06T21:03:21.727400+0000","last_fullsized":"2023-11-06T21:03:21.727400+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693606+0000","last_change":"2023-11-06T21:03:21.216485+0000","last_active":"2023-11-06T21:03:21.693606+0000","last_peered":"2023-11-06T21:03:21.693606+0000","last_clean":"2023-11-06T21:03:21.693606+0000","last_became_active":"2023-11-06T21:03:21.216091+0000","last_became_peered":"2023-11-06T21:03:21.216091+0000","last_unstale":"2023-11-06T21:03:21.693606+0000","last_undegraded":"2023-11-06T21:03:21.693606+0000","last_fullsized":"2023-11-06T21:03:21.693606+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.715402+0000","last_change":"2023-11-06T21:03:21.709618+0000","last_active":"2023-11-06T21:03:21.715402+0000","last_peered":"2023-11-06T21:03:21.715402+0000","last_clean":"2023-11-06T21:03:21.715402+0000","last_became_active":"2023-11-06T21:03:21.709301+0000","last_became_peered":"2023-11-06T21:03:21.709301+0000","last_unstale":"2023-11-06T21:03:21.715402+0000","last_undegraded":"2023-11-06T21:03:21.715402+0000","last_fullsized":"2023-11-06T21:03:21.715402+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.701602+0000","last_change":"2023-11-06T21:03:21.226333+0000","last_active":"2023-11-06T21:03:21.701602+0000","last_peered":"2023-11-06T21:03:21.701602+0000","last_clean":"2023-11-06T21:03:21.701602+0000","last_became_active":"2023-11-06T21:03:21.225339+0000","last_became_peered":"2023-11-06T21:03:21.225339+0000","last_unstale":"2023-11-06T21:03:21.701602+0000","last_undegraded":"2023-11-06T21:03:21.701602+0000","last_fullsized":"2023-11-06T21:03:21.701602+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726226+0000","last_change":"2023-11-06T21:03:21.714640+0000","last_active":"2023-11-06T21:03:21.726226+0000","last_peered":"2023-11-06T21:03:21.726226+0000","last_clean":"2023-11-06T21:03:21.726226+0000","last_became_active":"2023-11-06T21:03:21.714154+0000","last_became_peered":"2023-11-06T21:03:21.714154+0000","last_unstale":"2023-11-06T21:03:21.726226+0000","last_undegraded":"2023-11-06T21:03:21.726226+0000","last_fullsized":"2023-11-06T21:03:21.726226+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693524+0000","last_change":"2023-11-06T21:03:21.226812+0000","last_active":"2023-11-06T21:03:21.693524+0000","last_peered":"2023-11-06T21:03:21.693524+0000","last_clean":"2023-11-06T21:03:21.693524+0000","last_became_active":"2023-11-06T21:03:21.226530+0000","last_became_peered":"2023-11-06T21:03:21.226530+0000","last_unstale":"2023-11-06T21:03:21.693524+0000","last_undegraded":"2023-11-06T21:03:21.693524+0000","last_fullsized":"2023-11-06T21:03:21.693524+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.727516+0000","last_change":"2023-11-06T21:03:21.720525+0000","last_active":"2023-11-06T21:03:21.727516+0000","last_peered":"2023-11-06T21:03:21.727516+0000","last_clean":"2023-11-06T21:03:21.727516+0000","last_became_active":"2023-11-06T21:03:21.720254+0000","last_became_peered":"2023-11-06T21:03:21.720254+0000","last_unstale":"2023-11-06T21:03:21.727516+0000","last_undegraded":"2023-11-06T21:03:21.727516+0000","last_fullsized":"2023-11-06T21:03:21.727516+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.703896+0000","last_change":"2023-11-06T21:03:21.231668+0000","last_active":"2023-11-06T21:03:21.703896+0000","last_peered":"2023-11-06T21:03:21.703896+0000","last_clean":"2023-11-06T21:03:21.703896+0000","last_became_active":"2023-11-06T21:03:21.231323+0000","last_became_peered":"2023-11-06T21:03:21.231323+0000","last_unstale":"2023-11-06T21:03:21.703896+0000","last_undegraded":"2023-11-06T21:03:21.703896+0000","last_fullsized":"2023-11-06T21:03:21.703896+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726205+0000","last_change":"2023-11-06T21:03:21.718742+0000","last_active":"2023-11-06T21:03:21.726205+0000","last_peered":"2023-11-06T21:03:21.726205+0000","last_clean":"2023-11-06T21:03:21.726205+0000","last_became_active":"2023-11-06T21:03:21.718410+0000","last_became_peered":"2023-11-06T21:03:21.718410+0000","last_unstale":"2023-11-06T21:03:21.726205+0000","last_undegraded":"2023-11-06T21:03:21.726205+0000","last_fullsized":"2023-11-06T21:03:21.726205+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.690445+0000","last_change":"2023-11-06T21:03:21.210013+0000","last_active":"2023-11-06T21:03:21.690445+0000","last_peered":"2023-11-06T21:03:21.690445+0000","last_clean":"2023-11-06T21:03:21.690445+0000","last_became_active":"2023-11-06T21:03:21.207953+0000","last_became_peered":"2023-11-06T21:03:21.207953+0000","last_unstale":"2023-11-06T21:03:21.690445+0000","last_undegraded":"2023-11-06T21:03:21.690445+0000","last_fullsized":"2023-11-06T21:03:21.690445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.711357+0000","last_change":"2023-11-06T21:03:21.700858+0000","last_active":"2023-11-06T21:03:21.711357+0000","last_peered":"2023-11-06T21:03:21.711357+0000","last_clean":"2023-11-06T21:03:21.711357+0000","last_became_active":"2023-11-06T21:03:21.700573+0000","last_became_peered":"2023-11-06T21:03:21.700573+0000","last_unstale":"2023-11-06T21:03:21.711357+0000","last_undegraded":"2023-11-06T21:03:21.711357+0000","last_fullsized":"2023-11-06T21:03:21.711357+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.707738+0000","last_change":"2023-11-06T21:03:21.229372+0000","last_active":"2023-11-06T21:03:21.707738+0000","last_peered":"2023-11-06T21:03:21.707738+0000","last_clean":"2023-11-06T21:03:21.707738+0000","last_became_active":"2023-11-06T21:03:21.226410+0000","last_became_peered":"2023-11-06T21:03:21.226410+0000","last_unstale":"2023-11-06T21:03:21.707738+0000","last_undegraded":"2023-11-06T21:03:21.707738+0000","last_fullsized":"2023-11-06T21:03:21.707738+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716493+0000","last_change":"2023-11-06T21:03:21.711734+0000","last_active":"2023-11-06T21:03:21.716493+0000","last_peered":"2023-11-06T21:03:21.716493+0000","last_clean":"2023-11-06T21:03:21.716493+0000","last_became_active":"2023-11-06T21:03:21.711379+0000","last_became_peered":"2023-11-06T21:03:21.711379+0000","last_unstale":"2023-11-06T21:03:21.716493+0000","last_undegraded":"2023-11-06T21:03:21.716493+0000","last_fullsized":"2023-11-06T21:03:21.716493+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692623+0000","last_change":"2023-11-06T21:03:21.234030+0000","last_active":"2023-11-06T21:03:21.692623+0000","last_peered":"2023-11-06T21:03:21.692623+0000","last_clean":"2023-11-06T21:03:21.692623+0000","last_became_active":"2023-11-06T21:03:21.232752+0000","last_became_peered":"2023-11-06T21:03:21.232752+0000","last_unstale":"2023-11-06T21:03:21.692623+0000","last_undegraded":"2023-11-06T21:03:21.692623+0000","last_fullsized":"2023-11-06T21:03:21.692623+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.715852+0000","last_change":"2023-11-06T21:03:21.713467+0000","last_active":"2023-11-06T21:03:21.715852+0000","last_peered":"2023-11-06T21:03:21.715852+0000","last_clean":"2023-11-06T21:03:21.715852+0000","last_became_active":"2023-11-06T21:03:21.713064+0000","last_became_peered":"2023-11-06T21:03:21.713064+0000","last_unstale":"2023-11-06T21:03:21.715852+0000","last_undegraded":"2023-11-06T21:03:21.715852+0000","last_fullsized":"2023-11-06T21:03:21.715852+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.724200+0000","last_change":"2023-11-06T21:03:21.227032+0000","last_active":"2023-11-06T21:03:21.724200+0000","last_peered":"2023-11-06T21:03:21.724200+0000","last_clean":"2023-11-06T21:03:21.724200+0000","last_became_active":"2023-11-06T21:03:21.226754+0000","last_became_peered":"2023-11-06T21:03:21.226754+0000","last_unstale":"2023-11-06T21:03:21.724200+0000","last_undegraded":"2023-11-06T21:03:21.724200+0000","last_fullsized":"2023-11-06T21:03:21.724200+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.717926+0000","last_change":"2023-11-06T21:03:21.713041+0000","last_active":"2023-11-06T21:03:21.717926+0000","last_peered":"2023-11-06T21:03:21.717926+0000","last_clean":"2023-11-06T21:03:21.717926+0000","last_became_active":"2023-11-06T21:03:21.712543+0000","last_became_peered":"2023-11-06T21:03:21.712543+0000","last_unstale":"2023-11-06T21:03:21.717926+0000","last_undegraded":"2023-11-06T21:03:21.717926+0000","last_fullsized":"2023-11-06T21:03:21.717926+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716844+0000","last_change":"2023-11-06T21:03:21.227797+0000","last_active":"2023-11-06T21:03:21.716844+0000","last_peered":"2023-11-06T21:03:21.716844+0000","last_clean":"2023-11-06T21:03:21.716844+0000","last_became_active":"2023-11-06T21:03:21.227478+0000","last_became_peered":"2023-11-06T21:03:21.227478+0000","last_unstale":"2023-11-06T21:03:21.716844+0000","last_undegraded":"2023-11-06T21:03:21.716844+0000","last_fullsized":"2023-11-06T21:03:21.716844+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726089+0000","last_change":"2023-11-06T21:03:21.714676+0000","last_active":"2023-11-06T21:03:21.726089+0000","last_peered":"2023-11-06T21:03:21.726089+0000","last_clean":"2023-11-06T21:03:21.726089+0000","last_became_active":"2023-11-06T21:03:21.714331+0000","last_became_peered":"2023-11-06T21:03:21.714331+0000","last_unstale":"2023-11-06T21:03:21.726089+0000","last_undegraded":"2023-11-06T21:03:21.726089+0000","last_fullsized":"2023-11-06T21:03:21.726089+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"146'2","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719344+0000","last_change":"2023-11-06T21:03:21.228465+0000","last_active":"2023-11-06T21:03:21.719344+0000","last_peered":"2023-11-06T21:03:21.719344+0000","last_clean":"2023-11-06T21:03:21.719344+0000","last_became_active":"2023-11-06T21:03:21.228175+0000","last_became_peered":"2023-11-06T21:03:21.228175+0000","last_unstale":"2023-11-06T21:03:21.719344+0000","last_undegraded":"2023-11-06T21:03:21.719344+0000","last_fullsized":"2023-11-06T21:03:21.719344+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725788+0000","last_change":"2023-11-06T21:03:21.719655+0000","last_active":"2023-11-06T21:03:21.725788+0000","last_peered":"2023-11-06T21:03:21.725788+0000","last_clean":"2023-11-06T21:03:21.725788+0000","last_became_active":"2023-11-06T21:03:21.718337+0000","last_became_peered":"2023-11-06T21:03:21.718337+0000","last_unstale":"2023-11-06T21:03:21.725788+0000","last_undegraded":"2023-11-06T21:03:21.725788+0000","last_fullsized":"2023-11-06T21:03:21.725788+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693630+0000","last_change":"2023-11-06T21:03:21.224119+0000","last_active":"2023-11-06T21:03:21.693630+0000","last_peered":"2023-11-06T21:03:21.693630+0000","last_clean":"2023-11-06T21:03:21.693630+0000","last_became_active":"2023-11-06T21:03:21.223491+0000","last_became_peered":"2023-11-06T21:03:21.223491+0000","last_unstale":"2023-11-06T21:03:21.693630+0000","last_undegraded":"2023-11-06T21:03:21.693630+0000","last_fullsized":"2023-11-06T21:03:21.693630+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.723886+0000","last_change":"2023-11-06T21:03:21.720812+0000","last_active":"2023-11-06T21:03:21.723886+0000","last_peered":"2023-11-06T21:03:21.723886+0000","last_clean":"2023-11-06T21:03:21.723886+0000","last_became_active":"2023-11-06T21:03:21.720518+0000","last_became_peered":"2023-11-06T21:03:21.720518+0000","last_unstale":"2023-11-06T21:03:21.723886+0000","last_undegraded":"2023-11-06T21:03:21.723886+0000","last_fullsized":"2023-11-06T21:03:21.723886+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691476+0000","last_change":"2023-11-06T21:03:21.221763+0000","last_active":"2023-11-06T21:03:21.691476+0000","last_peered":"2023-11-06T21:03:21.691476+0000","last_clean":"2023-11-06T21:03:21.691476+0000","last_became_active":"2023-11-06T21:03:21.220623+0000","last_became_peered":"2023-11-06T21:03:21.220623+0000","last_unstale":"2023-11-06T21:03:21.691476+0000","last_undegraded":"2023-11-06T21:03:21.691476+0000","last_fullsized":"2023-11-06T21:03:21.691476+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716914+0000","last_change":"2023-11-06T21:03:21.705747+0000","last_active":"2023-11-06T21:03:21.716914+0000","last_peered":"2023-11-06T21:03:21.716914+0000","last_clean":"2023-11-06T21:03:21.716914+0000","last_became_active":"2023-11-06T21:03:21.705461+0000","last_became_peered":"2023-11-06T21:03:21.705461+0000","last_unstale":"2023-11-06T21:03:21.716914+0000","last_undegraded":"2023-11-06T21:03:21.716914+0000","last_fullsized":"2023-11-06T21:03:21.716914+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.700270+0000","last_change":"2023-11-06T21:03:21.232420+0000","last_active":"2023-11-06T21:03:21.700270+0000","last_peered":"2023-11-06T21:03:21.700270+0000","last_clean":"2023-11-06T21:03:21.700270+0000","last_became_active":"2023-11-06T21:03:21.232092+0000","last_became_peered":"2023-11-06T21:03:21.232092+0000","last_unstale":"2023-11-06T21:03:21.700270+0000","last_undegraded":"2023-11-06T21:03:21.700270+0000","last_fullsized":"2023-11-06T21:03:21.700270+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.714380+0000","last_change":"2023-11-06T21:03:21.706554+0000","last_active":"2023-11-06T21:03:21.714380+0000","last_peered":"2023-11-06T21:03:21.714380+0000","last_clean":"2023-11-06T21:03:21.714380+0000","last_became_active":"2023-11-06T21:03:21.706264+0000","last_became_peered":"2023-11-06T21:03:21.706264+0000","last_unstale":"2023-11-06T21:03:21.714380+0000","last_undegraded":"2023-11-06T21:03:21.714380+0000","last_fullsized":"2023-11-06T21:03:21.714380+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.689780+0000","last_change":"2023-11-06T21:03:21.213086+0000","last_active":"2023-11-06T21:03:21.689780+0000","last_peered":"2023-11-06T21:03:21.689780+0000","last_clean":"2023-11-06T21:03:21.689780+0000","last_became_active":"2023-11-06T21:03:21.212240+0000","last_became_peered":"2023-11-06T21:03:21.212240+0000","last_unstale":"2023-11-06T21:03:21.689780+0000","last_undegraded":"2023-11-06T21:03:21.689780+0000","last_fullsized":"2023-11-06T21:03:21.689780+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.712681+0000","last_change":"2023-11-06T21:03:21.706299+0000","last_active":"2023-11-06T21:03:21.712681+0000","last_peered":"2023-11-06T21:03:21.712681+0000","last_clean":"2023-11-06T21:03:21.712681+0000","last_became_active":"2023-11-06T21:03:21.705948+0000","last_became_peered":"2023-11-06T21:03:21.705948+0000","last_unstale":"2023-11-06T21:03:21.712681+0000","last_undegraded":"2023-11-06T21:03:21.712681+0000","last_fullsized":"2023-11-06T21:03:21.712681+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692727+0000","last_change":"2023-11-06T21:03:21.227820+0000","last_active":"2023-11-06T21:03:21.692727+0000","last_peered":"2023-11-06T21:03:21.692727+0000","last_clean":"2023-11-06T21:03:21.692727+0000","last_became_active":"2023-11-06T21:03:21.227529+0000","last_became_peered":"2023-11-06T21:03:21.227529+0000","last_unstale":"2023-11-06T21:03:21.692727+0000","last_undegraded":"2023-11-06T21:03:21.692727+0000","last_fullsized":"2023-11-06T21:03:21.692727+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719482+0000","last_change":"2023-11-06T21:03:21.716144+0000","last_active":"2023-11-06T21:03:21.719482+0000","last_peered":"2023-11-06T21:03:21.719482+0000","last_clean":"2023-11-06T21:03:21.719482+0000","last_became_active":"2023-11-06T21:03:21.714537+0000","last_became_peered":"2023-11-06T21:03:21.714537+0000","last_unstale":"2023-11-06T21:03:21.719482+0000","last_undegraded":"2023-11-06T21:03:21.719482+0000","last_fullsized":"2023-11-06T21:03:21.719482+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.717473+0000","last_change":"2023-11-06T21:03:21.230976+0000","last_active":"2023-11-06T21:03:21.717473+0000","last_peered":"2023-11-06T21:03:21.717473+0000","last_clean":"2023-11-06T21:03:21.717473+0000","last_became_active":"2023-11-06T21:03:21.230425+0000","last_became_peered":"2023-11-06T21:03:21.230425+0000","last_unstale":"2023-11-06T21:03:21.717473+0000","last_undegraded":"2023-11-06T21:03:21.717473+0000","last_fullsized":"2023-11-06T21:03:21.717473+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716434+0000","last_change":"2023-11-06T21:03:21.701884+0000","last_active":"2023-11-06T21:03:21.716434+0000","last_peered":"2023-11-06T21:03:21.716434+0000","last_clean":"2023-11-06T21:03:21.716434+0000","last_became_active":"2023-11-06T21:03:21.701605+0000","last_became_peered":"2023-11-06T21:03:21.701605+0000","last_unstale":"2023-11-06T21:03:21.716434+0000","last_undegraded":"2023-11-06T21:03:21.716434+0000","last_fullsized":"2023-11-06T21:03:21.716434+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"146'2","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719457+0000","last_change":"2023-11-06T21:03:21.222629+0000","last_active":"2023-11-06T21:03:21.719457+0000","last_peered":"2023-11-06T21:03:21.719457+0000","last_clean":"2023-11-06T21:03:21.719457+0000","last_became_active":"2023-11-06T21:03:21.221644+0000","last_became_peered":"2023-11-06T21:03:21.221644+0000","last_unstale":"2023-11-06T21:03:21.719457+0000","last_undegraded":"2023-11-06T21:03:21.719457+0000","last_fullsized":"2023-11-06T21:03:21.719457+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.714216+0000","last_change":"2023-11-06T21:03:21.704282+0000","last_active":"2023-11-06T21:03:21.714216+0000","last_peered":"2023-11-06T21:03:21.714216+0000","last_clean":"2023-11-06T21:03:21.714216+0000","last_became_active":"2023-11-06T21:03:21.703532+0000","last_became_peered":"2023-11-06T21:03:21.703532+0000","last_unstale":"2023-11-06T21:03:21.714216+0000","last_undegraded":"2023-11-06T21:03:21.714216+0000","last_fullsized":"2023-11-06T21:03:21.714216+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.704094+0000","last_change":"2023-11-06T21:03:21.228316+0000","last_active":"2023-11-06T21:03:21.704094+0000","last_peered":"2023-11-06T21:03:21.704094+0000","last_clean":"2023-11-06T21:03:21.704094+0000","last_became_active":"2023-11-06T21:03:21.227981+0000","last_became_peered":"2023-11-06T21:03:21.227981+0000","last_unstale":"2023-11-06T21:03:21.704094+0000","last_undegraded":"2023-11-06T21:03:21.704094+0000","last_fullsized":"2023-11-06T21:03:21.704094+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.723339+0000","last_change":"2023-11-06T21:03:21.717139+0000","last_active":"2023-11-06T21:03:21.723339+0000","last_peered":"2023-11-06T21:03:21.723339+0000","last_clean":"2023-11-06T21:03:21.723339+0000","last_became_active":"2023-11-06T21:03:21.716682+0000","last_became_peered":"2023-11-06T21:03:21.716682+0000","last_unstale":"2023-11-06T21:03:21.723339+0000","last_undegraded":"2023-11-06T21:03:21.723339+0000","last_fullsized":"2023-11-06T21:03:21.723339+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691236+0000","last_change":"2023-11-06T21:03:21.224941+0000","last_active":"2023-11-06T21:03:21.691236+0000","last_peered":"2023-11-06T21:03:21.691236+0000","last_clean":"2023-11-06T21:03:21.691236+0000","last_became_active":"2023-11-06T21:03:21.224578+0000","last_became_peered":"2023-11-06T21:03:21.224578+0000","last_unstale":"2023-11-06T21:03:21.691236+0000","last_undegraded":"2023-11-06T21:03:21.691236+0000","last_fullsized":"2023-11-06T21:03:21.691236+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.710028+0000","last_change":"2023-11-06T21:03:21.699710+0000","last_active":"2023-11-06T21:03:21.710028+0000","last_peered":"2023-11-06T21:03:21.710028+0000","last_clean":"2023-11-06T21:03:21.710028+0000","last_became_active":"2023-11-06T21:03:21.699412+0000","last_became_peered":"2023-11-06T21:03:21.699412+0000","last_unstale":"2023-11-06T21:03:21.710028+0000","last_undegraded":"2023-11-06T21:03:21.710028+0000","last_fullsized":"2023-11-06T21:03:21.710028+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.714961+0000","last_change":"2023-11-06T21:03:21.218741+0000","last_active":"2023-11-06T21:03:21.714961+0000","last_peered":"2023-11-06T21:03:21.714961+0000","last_clean":"2023-11-06T21:03:21.714961+0000","last_became_active":"2023-11-06T21:03:21.218402+0000","last_became_peered":"2023-11-06T21:03:21.218402+0000","last_unstale":"2023-11-06T21:03:21.714961+0000","last_undegraded":"2023-11-06T21:03:21.714961+0000","last_fullsized":"2023-11-06T21:03:21.714961+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719348+0000","last_change":"2023-11-06T21:03:21.717509+0000","last_active":"2023-11-06T21:03:21.719348+0000","last_peered":"2023-11-06T21:03:21.719348+0000","last_clean":"2023-11-06T21:03:21.719348+0000","last_became_active":"2023-11-06T21:03:21.716656+0000","last_became_peered":"2023-11-06T21:03:21.716656+0000","last_unstale":"2023-11-06T21:03:21.719348+0000","last_undegraded":"2023-11-06T21:03:21.719348+0000","last_fullsized":"2023-11-06T21:03:21.719348+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.694959+0000","last_change":"2023-11-06T21:03:21.217738+0000","last_active":"2023-11-06T21:03:21.694959+0000","last_peered":"2023-11-06T21:03:21.694959+0000","last_clean":"2023-11-06T21:03:21.694959+0000","last_became_active":"2023-11-06T21:03:21.217404+0000","last_became_peered":"2023-11-06T21:03:21.217404+0000","last_unstale":"2023-11-06T21:03:21.694959+0000","last_undegraded":"2023-11-06T21:03:21.694959+0000","last_fullsized":"2023-11-06T21:03:21.694959+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.723774+0000","last_change":"2023-11-06T21:03:21.721254+0000","last_active":"2023-11-06T21:03:21.723774+0000","last_peered":"2023-11-06T21:03:21.723774+0000","last_clean":"2023-11-06T21:03:21.723774+0000","last_became_active":"2023-11-06T21:03:21.720884+0000","last_became_peered":"2023-11-06T21:03:21.720884+0000","last_unstale":"2023-11-06T21:03:21.723774+0000","last_undegraded":"2023-11-06T21:03:21.723774+0000","last_fullsized":"2023-11-06T21:03:21.723774+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692724+0000","last_change":"2023-11-06T21:03:21.227649+0000","last_active":"2023-11-06T21:03:21.692724+0000","last_peered":"2023-11-06T21:03:21.692724+0000","last_clean":"2023-11-06T21:03:21.692724+0000","last_became_active":"2023-11-06T21:03:21.227323+0000","last_became_peered":"2023-11-06T21:03:21.227323+0000","last_unstale":"2023-11-06T21:03:21.692724+0000","last_undegraded":"2023-11-06T21:03:21.692724+0000","last_fullsized":"2023-11-06T21:03:21.692724+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719403+0000","last_change":"2023-11-06T21:03:21.702090+0000","last_active":"2023-11-06T21:03:21.719403+0000","last_peered":"2023-11-06T21:03:21.719403+0000","last_clean":"2023-11-06T21:03:21.719403+0000","last_became_active":"2023-11-06T21:03:21.701810+0000","last_became_peered":"2023-11-06T21:03:21.701810+0000","last_unstale":"2023-11-06T21:03:21.719403+0000","last_undegraded":"2023-11-06T21:03:21.719403+0000","last_fullsized":"2023-11-06T21:03:21.719403+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.700129+0000","last_change":"2023-11-06T21:03:21.223351+0000","last_active":"2023-11-06T21:03:21.700129+0000","last_peered":"2023-11-06T21:03:21.700129+0000","last_clean":"2023-11-06T21:03:21.700129+0000","last_became_active":"2023-11-06T21:03:21.222353+0000","last_became_peered":"2023-11-06T21:03:21.222353+0000","last_unstale":"2023-11-06T21:03:21.700129+0000","last_undegraded":"2023-11-06T21:03:21.700129+0000","last_fullsized":"2023-11-06T21:03:21.700129+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.722485+0000","last_change":"2023-11-06T21:03:21.719702+0000","last_active":"2023-11-06T21:03:21.722485+0000","last_peered":"2023-11-06T21:03:21.722485+0000","last_clean":"2023-11-06T21:03:21.722485+0000","last_became_active":"2023-11-06T21:03:21.719501+0000","last_became_peered":"2023-11-06T21:03:21.719501+0000","last_unstale":"2023-11-06T21:03:21.722485+0000","last_undegraded":"2023-11-06T21:03:21.722485+0000","last_fullsized":"2023-11-06T21:03:21.722485+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"146'2","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.718970+0000","last_change":"2023-11-06T21:03:21.215675+0000","last_active":"2023-11-06T21:03:21.718970+0000","last_peered":"2023-11-06T21:03:21.718970+0000","last_clean":"2023-11-06T21:03:21.718970+0000","last_became_active":"2023-11-06T21:03:21.215028+0000","last_became_peered":"2023-11-06T21:03:21.215028+0000","last_unstale":"2023-11-06T21:03:21.718970+0000","last_undegraded":"2023-11-06T21:03:21.718970+0000","last_fullsized":"2023-11-06T21:03:21.718970+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716366+0000","last_change":"2023-11-06T21:03:21.710295+0000","last_active":"2023-11-06T21:03:21.716366+0000","last_peered":"2023-11-06T21:03:21.716366+0000","last_clean":"2023-11-06T21:03:21.716366+0000","last_became_active":"2023-11-06T21:03:21.709915+0000","last_became_peered":"2023-11-06T21:03:21.709915+0000","last_unstale":"2023-11-06T21:03:21.716366+0000","last_undegraded":"2023-11-06T21:03:21.716366+0000","last_fullsized":"2023-11-06T21:03:21.716366+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.706968+0000","last_change":"2023-11-06T21:03:21.233585+0000","last_active":"2023-11-06T21:03:21.706968+0000","last_peered":"2023-11-06T21:03:21.706968+0000","last_clean":"2023-11-06T21:03:21.706968+0000","last_became_active":"2023-11-06T21:03:21.232662+0000","last_became_peered":"2023-11-06T21:03:21.232662+0000","last_unstale":"2023-11-06T21:03:21.706968+0000","last_undegraded":"2023-11-06T21:03:21.706968+0000","last_fullsized":"2023-11-06T21:03:21.706968+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716619+0000","last_change":"2023-11-06T21:03:21.712941+0000","last_active":"2023-11-06T21:03:21.716619+0000","last_peered":"2023-11-06T21:03:21.716619+0000","last_clean":"2023-11-06T21:03:21.716619+0000","last_became_active":"2023-11-06T21:03:21.712683+0000","last_became_peered":"2023-11-06T21:03:21.712683+0000","last_unstale":"2023-11-06T21:03:21.716619+0000","last_undegraded":"2023-11-06T21:03:21.716619+0000","last_fullsized":"2023-11-06T21:03:21.716619+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.704192+0000","last_change":"2023-11-06T21:03:21.230850+0000","last_active":"2023-11-06T21:03:21.704192+0000","last_peered":"2023-11-06T21:03:21.704192+0000","last_clean":"2023-11-06T21:03:21.704192+0000","last_became_active":"2023-11-06T21:03:21.230523+0000","last_became_peered":"2023-11-06T21:03:21.230523+0000","last_unstale":"2023-11-06T21:03:21.704192+0000","last_undegraded":"2023-11-06T21:03:21.704192+0000","last_fullsized":"2023-11-06T21:03:21.704192+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.722298+0000","last_change":"2023-11-06T21:03:21.720665+0000","last_active":"2023-11-06T21:03:21.722298+0000","last_peered":"2023-11-06T21:03:21.722298+0000","last_clean":"2023-11-06T21:03:21.722298+0000","last_became_active":"2023-11-06T21:03:21.720237+0000","last_became_peered":"2023-11-06T21:03:21.720237+0000","last_unstale":"2023-11-06T21:03:21.722298+0000","last_undegraded":"2023-11-06T21:03:21.722298+0000","last_fullsized":"2023-11-06T21:03:21.722298+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.710978+0000","last_change":"2023-11-06T21:03:21.226235+0000","last_active":"2023-11-06T21:03:21.710978+0000","last_peered":"2023-11-06T21:03:21.710978+0000","last_clean":"2023-11-06T21:03:21.710978+0000","last_became_active":"2023-11-06T21:03:21.225785+0000","last_became_peered":"2023-11-06T21:03:21.225785+0000","last_unstale":"2023-11-06T21:03:21.710978+0000","last_undegraded":"2023-11-06T21:03:21.710978+0000","last_fullsized":"2023-11-06T21:03:21.710978+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719727+0000","last_change":"2023-11-06T21:03:21.717883+0000","last_active":"2023-11-06T21:03:21.719727+0000","last_peered":"2023-11-06T21:03:21.719727+0000","last_clean":"2023-11-06T21:03:21.719727+0000","last_became_active":"2023-11-06T21:03:21.717612+0000","last_became_peered":"2023-11-06T21:03:21.717612+0000","last_unstale":"2023-11-06T21:03:21.719727+0000","last_undegraded":"2023-11-06T21:03:21.719727+0000","last_fullsized":"2023-11-06T21:03:21.719727+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.704001+0000","last_change":"2023-11-06T21:03:21.231531+0000","last_active":"2023-11-06T21:03:21.704001+0000","last_peered":"2023-11-06T21:03:21.704001+0000","last_clean":"2023-11-06T21:03:21.704001+0000","last_became_active":"2023-11-06T21:03:21.231250+0000","last_became_peered":"2023-11-06T21:03:21.231250+0000","last_unstale":"2023-11-06T21:03:21.704001+0000","last_undegraded":"2023-11-06T21:03:21.704001+0000","last_fullsized":"2023-11-06T21:03:21.704001+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716772+0000","last_change":"2023-11-06T21:03:21.710700+0000","last_active":"2023-11-06T21:03:21.716772+0000","last_peered":"2023-11-06T21:03:21.716772+0000","last_clean":"2023-11-06T21:03:21.716772+0000","last_became_active":"2023-11-06T21:03:21.709326+0000","last_became_peered":"2023-11-06T21:03:21.709326+0000","last_unstale":"2023-11-06T21:03:21.716772+0000","last_undegraded":"2023-11-06T21:03:21.716772+0000","last_fullsized":"2023-11-06T21:03:21.716772+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.694932+0000","last_change":"2023-11-06T21:03:21.227704+0000","last_active":"2023-11-06T21:03:21.694932+0000","last_peered":"2023-11-06T21:03:21.694932+0000","last_clean":"2023-11-06T21:03:21.694932+0000","last_became_active":"2023-11-06T21:03:21.227197+0000","last_became_peered":"2023-11-06T21:03:21.227197+0000","last_unstale":"2023-11-06T21:03:21.694932+0000","last_undegraded":"2023-11-06T21:03:21.694932+0000","last_fullsized":"2023-11-06T21:03:21.694932+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.717691+0000","last_change":"2023-11-06T21:03:21.715461+0000","last_active":"2023-11-06T21:03:21.717691+0000","last_peered":"2023-11-06T21:03:21.717691+0000","last_clean":"2023-11-06T21:03:21.717691+0000","last_became_active":"2023-11-06T21:03:21.715191+0000","last_became_peered":"2023-11-06T21:03:21.715191+0000","last_unstale":"2023-11-06T21:03:21.717691+0000","last_undegraded":"2023-11-06T21:03:21.717691+0000","last_fullsized":"2023-11-06T21:03:21.717691+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692143+0000","last_change":"2023-11-06T21:03:21.220621+0000","last_active":"2023-11-06T21:03:21.692143+0000","last_peered":"2023-11-06T21:03:21.692143+0000","last_clean":"2023-11-06T21:03:21.692143+0000","last_became_active":"2023-11-06T21:03:21.220330+0000","last_became_peered":"2023-11-06T21:03:21.220330+0000","last_unstale":"2023-11-06T21:03:21.692143+0000","last_undegraded":"2023-11-06T21:03:21.692143+0000","last_fullsized":"2023-11-06T21:03:21.692143+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.717796+0000","last_change":"2023-11-06T21:03:21.714604+0000","last_active":"2023-11-06T21:03:21.717796+0000","last_peered":"2023-11-06T21:03:21.717796+0000","last_clean":"2023-11-06T21:03:21.717796+0000","last_became_active":"2023-11-06T21:03:21.714321+0000","last_became_peered":"2023-11-06T21:03:21.714321+0000","last_unstale":"2023-11-06T21:03:21.717796+0000","last_undegraded":"2023-11-06T21:03:21.717796+0000","last_fullsized":"2023-11-06T21:03:21.717796+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.700214+0000","last_change":"2023-11-06T21:03:21.228240+0000","last_active":"2023-11-06T21:03:21.700214+0000","last_peered":"2023-11-06T21:03:21.700214+0000","last_clean":"2023-11-06T21:03:21.700214+0000","last_became_active":"2023-11-06T21:03:21.227747+0000","last_became_peered":"2023-11-06T21:03:21.227747+0000","last_unstale":"2023-11-06T21:03:21.700214+0000","last_undegraded":"2023-11-06T21:03:21.700214+0000","last_fullsized":"2023-11-06T21:03:21.700214+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.715127+0000","last_change":"2023-11-06T21:03:21.711289+0000","last_active":"2023-11-06T21:03:21.715127+0000","last_peered":"2023-11-06T21:03:21.715127+0000","last_clean":"2023-11-06T21:03:21.715127+0000","last_became_active":"2023-11-06T21:03:21.710992+0000","last_became_peered":"2023-11-06T21:03:21.710992+0000","last_unstale":"2023-11-06T21:03:21.715127+0000","last_undegraded":"2023-11-06T21:03:21.715127+0000","last_fullsized":"2023-11-06T21:03:21.715127+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.701093+0000","last_change":"2023-11-06T21:03:21.229674+0000","last_active":"2023-11-06T21:03:21.701093+0000","last_peered":"2023-11-06T21:03:21.701093+0000","last_clean":"2023-11-06T21:03:21.701093+0000","last_became_active":"2023-11-06T21:03:21.228519+0000","last_became_peered":"2023-11-06T21:03:21.228519+0000","last_unstale":"2023-11-06T21:03:21.701093+0000","last_undegraded":"2023-11-06T21:03:21.701093+0000","last_fullsized":"2023-11-06T21:03:21.701093+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693796+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T21:03:21.693796+0000","last_peered":"2023-11-06T21:03:21.693796+0000","last_clean":"2023-11-06T21:03:21.693796+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T21:03:21.693796+0000","last_undegraded":"2023-11-06T21:03:21.693796+0000","last_fullsized":"2023-11-06T21:03:21.693796+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719802+0000","last_change":"2023-11-06T21:03:21.717271+0000","last_active":"2023-11-06T21:03:21.719802+0000","last_peered":"2023-11-06T21:03:21.719802+0000","last_clean":"2023-11-06T21:03:21.719802+0000","last_became_active":"2023-11-06T21:03:21.716945+0000","last_became_peered":"2023-11-06T21:03:21.716945+0000","last_unstale":"2023-11-06T21:03:21.719802+0000","last_undegraded":"2023-11-06T21:03:21.719802+0000","last_fullsized":"2023-11-06T21:03:21.719802+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.700423+0000","last_change":"2023-11-06T21:03:21.225780+0000","last_active":"2023-11-06T21:03:21.700423+0000","last_peered":"2023-11-06T21:03:21.700423+0000","last_clean":"2023-11-06T21:03:21.700423+0000","last_became_active":"2023-11-06T21:03:21.225007+0000","last_became_peered":"2023-11-06T21:03:21.225007+0000","last_unstale":"2023-11-06T21:03:21.700423+0000","last_undegraded":"2023-11-06T21:03:21.700423+0000","last_fullsized":"2023-11-06T21:03:21.700423+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":131,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.690520+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T21:03:21.690520+0000","last_peered":"2023-11-06T21:03:21.690520+0000","last_clean":"2023-11-06T21:03:21.690520+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T21:03:21.690520+0000","last_undegraded":"2023-11-06T21:03:21.690520+0000","last_fullsized":"2023-11-06T21:03:21.690520+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716793+0000","last_change":"2023-11-06T21:03:21.710933+0000","last_active":"2023-11-06T21:03:21.716793+0000","last_peered":"2023-11-06T21:03:21.716793+0000","last_clean":"2023-11-06T21:03:21.716793+0000","last_became_active":"2023-11-06T21:03:21.709677+0000","last_became_peered":"2023-11-06T21:03:21.709677+0000","last_unstale":"2023-11-06T21:03:21.716793+0000","last_undegraded":"2023-11-06T21:03:21.716793+0000","last_fullsized":"2023-11-06T21:03:21.716793+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.694519+0000","last_change":"2023-11-06T21:03:21.235344+0000","last_active":"2023-11-06T21:03:21.694519+0000","last_peered":"2023-11-06T21:03:21.694519+0000","last_clean":"2023-11-06T21:03:21.694519+0000","last_became_active":"2023-11-06T21:03:21.234696+0000","last_became_peered":"2023-11-06T21:03:21.234696+0000","last_unstale":"2023-11-06T21:03:21.694519+0000","last_undegraded":"2023-11-06T21:03:21.694519+0000","last_fullsized":"2023-11-06T21:03:21.694519+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":135,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692869+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T21:03:21.692869+0000","last_peered":"2023-11-06T21:03:21.692869+0000","last_clean":"2023-11-06T21:03:21.692869+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T21:03:21.692869+0000","last_undegraded":"2023-11-06T21:03:21.692869+0000","last_fullsized":"2023-11-06T21:03:21.692869+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725410+0000","last_change":"2023-11-06T21:03:21.722170+0000","last_active":"2023-11-06T21:03:21.725410+0000","last_peered":"2023-11-06T21:03:21.725410+0000","last_clean":"2023-11-06T21:03:21.725410+0000","last_became_active":"2023-11-06T21:03:21.721420+0000","last_became_peered":"2023-11-06T21:03:21.721420+0000","last_unstale":"2023-11-06T21:03:21.725410+0000","last_undegraded":"2023-11-06T21:03:21.725410+0000","last_fullsized":"2023-11-06T21:03:21.725410+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"146'4","reported_seq":14,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.727620+0000","last_change":"2023-11-06T21:03:21.222049+0000","last_active":"2023-11-06T21:03:21.727620+0000","last_peered":"2023-11-06T21:03:21.727620+0000","last_clean":"2023-11-06T21:03:21.727620+0000","last_became_active":"2023-11-06T21:03:21.221015+0000","last_became_peered":"2023-11-06T21:03:21.221015+0000","last_unstale":"2023-11-06T21:03:21.727620+0000","last_undegraded":"2023-11-06T21:03:21.727620+0000","last_fullsized":"2023-11-06T21:03:21.727620+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":131,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691547+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T21:03:21.691547+0000","last_peered":"2023-11-06T21:03:21.691547+0000","last_clean":"2023-11-06T21:03:21.691547+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T21:03:21.691547+0000","last_undegraded":"2023-11-06T21:03:21.691547+0000","last_fullsized":"2023-11-06T21:03:21.691547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":123,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692503+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T21:03:21.692503+0000","last_peered":"2023-11-06T21:03:21.692503+0000","last_clean":"2023-11-06T21:03:21.692503+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T21:03:21.692503+0000","last_undegraded":"2023-11-06T21:03:21.692503+0000","last_fullsized":"2023-11-06T21:03:21.692503+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716684+0000","last_change":"2023-11-06T21:03:21.712901+0000","last_active":"2023-11-06T21:03:21.716684+0000","last_peered":"2023-11-06T21:03:21.716684+0000","last_clean":"2023-11-06T21:03:21.716684+0000","last_became_active":"2023-11-06T21:03:21.711909+0000","last_became_peered":"2023-11-06T21:03:21.711909+0000","last_unstale":"2023-11-06T21:03:21.716684+0000","last_undegraded":"2023-11-06T21:03:21.716684+0000","last_fullsized":"2023-11-06T21:03:21.716684+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.712475+0000","last_change":"2023-11-06T21:03:21.234062+0000","last_active":"2023-11-06T21:03:21.712475+0000","last_peered":"2023-11-06T21:03:21.712475+0000","last_clean":"2023-11-06T21:03:21.712475+0000","last_became_active":"2023-11-06T21:03:21.233776+0000","last_became_peered":"2023-11-06T21:03:21.233776+0000","last_unstale":"2023-11-06T21:03:21.712475+0000","last_undegraded":"2023-11-06T21:03:21.712475+0000","last_fullsized":"2023-11-06T21:03:21.712475+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.689651+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T21:03:21.689651+0000","last_peered":"2023-11-06T21:03:21.689651+0000","last_clean":"2023-11-06T21:03:21.689651+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T21:03:21.689651+0000","last_undegraded":"2023-11-06T21:03:21.689651+0000","last_fullsized":"2023-11-06T21:03:21.689651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.715950+0000","last_change":"2023-11-06T21:03:21.710019+0000","last_active":"2023-11-06T21:03:21.715950+0000","last_peered":"2023-11-06T21:03:21.715950+0000","last_clean":"2023-11-06T21:03:21.715950+0000","last_became_active":"2023-11-06T21:03:21.709753+0000","last_became_peered":"2023-11-06T21:03:21.709753+0000","last_unstale":"2023-11-06T21:03:21.715950+0000","last_undegraded":"2023-11-06T21:03:21.715950+0000","last_fullsized":"2023-11-06T21:03:21.715950+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.698773+0000","last_change":"2023-11-06T21:03:21.221514+0000","last_active":"2023-11-06T21:03:21.698773+0000","last_peered":"2023-11-06T21:03:21.698773+0000","last_clean":"2023-11-06T21:03:21.698773+0000","last_became_active":"2023-11-06T21:03:21.221198+0000","last_became_peered":"2023-11-06T21:03:21.221198+0000","last_unstale":"2023-11-06T21:03:21.698773+0000","last_undegraded":"2023-11-06T21:03:21.698773+0000","last_fullsized":"2023-11-06T21:03:21.698773+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691594+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T21:03:21.691594+0000","last_peered":"2023-11-06T21:03:21.691594+0000","last_clean":"2023-11-06T21:03:21.691594+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T21:03:21.691594+0000","last_undegraded":"2023-11-06T21:03:21.691594+0000","last_fullsized":"2023-11-06T21:03:21.691594+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.714548+0000","last_change":"2023-11-06T21:03:21.707848+0000","last_active":"2023-11-06T21:03:21.714548+0000","last_peered":"2023-11-06T21:03:21.714548+0000","last_clean":"2023-11-06T21:03:21.714548+0000","last_became_active":"2023-11-06T21:03:21.707529+0000","last_became_peered":"2023-11-06T21:03:21.707529+0000","last_unstale":"2023-11-06T21:03:21.714548+0000","last_undegraded":"2023-11-06T21:03:21.714548+0000","last_fullsized":"2023-11-06T21:03:21.714548+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"146'3","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726466+0000","last_change":"2023-11-06T21:03:21.220969+0000","last_active":"2023-11-06T21:03:21.726466+0000","last_peered":"2023-11-06T21:03:21.726466+0000","last_clean":"2023-11-06T21:03:21.726466+0000","last_became_active":"2023-11-06T21:03:21.220016+0000","last_became_peered":"2023-11-06T21:03:21.220016+0000","last_unstale":"2023-11-06T21:03:21.726466+0000","last_undegraded":"2023-11-06T21:03:21.726466+0000","last_fullsized":"2023-11-06T21:03:21.726466+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":120,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.700281+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T21:03:21.700281+0000","last_peered":"2023-11-06T21:03:21.700281+0000","last_clean":"2023-11-06T21:03:21.700281+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T21:03:21.700281+0000","last_undegraded":"2023-11-06T21:03:21.700281+0000","last_fullsized":"2023-11-06T21:03:21.700281+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.724971+0000","last_change":"2023-11-06T21:03:21.720099+0000","last_active":"2023-11-06T21:03:21.724971+0000","last_peered":"2023-11-06T21:03:21.724971+0000","last_clean":"2023-11-06T21:03:21.724971+0000","last_became_active":"2023-11-06T21:03:21.719863+0000","last_became_peered":"2023-11-06T21:03:21.719863+0000","last_unstale":"2023-11-06T21:03:21.724971+0000","last_undegraded":"2023-11-06T21:03:21.724971+0000","last_fullsized":"2023-11-06T21:03:21.724971+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"146'2","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.721956+0000","last_change":"2023-11-06T21:03:21.212520+0000","last_active":"2023-11-06T21:03:21.721956+0000","last_peered":"2023-11-06T21:03:21.721956+0000","last_clean":"2023-11-06T21:03:21.721956+0000","last_became_active":"2023-11-06T21:03:21.212147+0000","last_became_peered":"2023-11-06T21:03:21.212147+0000","last_unstale":"2023-11-06T21:03:21.721956+0000","last_undegraded":"2023-11-06T21:03:21.721956+0000","last_fullsized":"2023-11-06T21:03:21.721956+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691672+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T21:03:21.691672+0000","last_peered":"2023-11-06T21:03:21.691672+0000","last_clean":"2023-11-06T21:03:21.691672+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T21:03:21.691672+0000","last_undegraded":"2023-11-06T21:03:21.691672+0000","last_fullsized":"2023-11-06T21:03:21.691672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575076,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":926408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.377},"max":{"1min":0.98099999999999998,"5min":1.069,"15min":1.069},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66800000000000004,"15min":0.69499999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35099999999999998},"max":{"1min":0.88500000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.872}]},{"osd":1,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66300000000000003,"15min":0.69299999999999995},"min":{"1min":0.48299999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84199999999999997,"5min":0.90600000000000003,"15min":0.98599999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96299999999999997,"5min":1.0600000000000001,"15min":1.111},"last":0.67000000000000004}]},{"osd":2,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68500000000000005,"15min":0.72099999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.165},"last":0.751},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67800000000000005,"15min":0.71299999999999997},"min":{"1min":0.499,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91200000000000003,"5min":0.97599999999999998,"15min":1.024},"last":0.81999999999999995}]},{"osd":3,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66400000000000003,"15min":0.69499999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.875,"5min":0.89900000000000002,"15min":1.0129999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72299999999999998,"15min":0.748},"min":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0389999999999999,"5min":1.1180000000000001,"15min":1.274},"last":0.85699999999999998}]},{"osd":4,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79300000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.621},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84099999999999997}]},{"osd":5,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86199999999999999,"5min":1.349,"15min":1.349},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64300000000000002,"15min":0.65600000000000003},"min":{"1min":0.47499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":1.399,"15min":1.399},"last":0.71899999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69499999999999995,"15min":0.70799999999999996},"min":{"1min":0.45100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89700000000000002,"5min":1.0680000000000001,"15min":1.1379999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66000000000000003,"15min":0.68300000000000005},"min":{"1min":0.437,"5min":0.433,"15min":0.433},"max":{"1min":0.92900000000000005,"5min":0.96999999999999997,"15min":1.0549999999999999},"last":0.56599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575076,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":926022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66400000000000003,"15min":0.67900000000000005},"min":{"1min":0.41599999999999998,"5min":0.35899999999999999,"15min":0.33000000000000002},"max":{"1min":0.94999999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.68400000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93600000000000005,"5min":0.93700000000000006,"15min":1.216},"last":0.52800000000000002}]},{"osd":1,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66900000000000004,"15min":0.68899999999999995},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.97599999999999998},"last":0.623},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.98599999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.56799999999999995,"5min":0.49299999999999999,"15min":0.47099999999999997},"max":{"1min":0.96499999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.749,"15min":0.76200000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.41499999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.05},"last":0.68600000000000005}]},{"osd":3,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68500000000000005,"15min":0.70799999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.39900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.1459999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1,"5min":1,"15min":1.002},"last":0.81999999999999995}]},{"osd":4,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77800000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80100000000000005,"5min":0.93300000000000005,"15min":1.012},"last":0.74099999999999999}]},{"osd":5,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.35099999999999998},"max":{"1min":0.85699999999999998,"5min":0.93300000000000005,"15min":1.028},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69599999999999995,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.441,"15min":0.38600000000000001},"max":{"1min":0.81100000000000005,"5min":1.04,"15min":1.04},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.68999999999999995,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.378},"max":{"1min":0.878,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72399999999999998,"15min":0.73299999999999998},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89900000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575076,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":925606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46400000000000002,"15min":0.45700000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.25800000000000001},"max":{"1min":0.64200000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.308},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.66800000000000004,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.40500000000000003}]},{"osd":2,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63700000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63800000000000001,"15min":0.61499999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82399999999999995,"5min":4.431,"15min":4.431},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.61899999999999999,"15min":0.59199999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79300000000000004,"5min":4.5380000000000003,"15min":4.5380000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.35599999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.47799999999999998}]},{"osd":4,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.77800000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.42599999999999999},"max":{"1min":1.1519999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0720000000000001,"5min":1.883,"15min":1.883},"last":0.85499999999999998}]},{"osd":5,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0449999999999999,"5min":4.399,"15min":4.399},"last":0.754},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.78000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.171,"5min":2.649,"15min":2.649},"last":0.83599999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.81200000000000006,"15min":0.80800000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1240000000000001,"5min":3.024,"15min":3.024},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.52400000000000002},"max":{"1min":1.139,"5min":3.0609999999999999,"15min":3.0609999999999999},"last":0.79500000000000004}]},{"osd":7,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82099999999999995,"15min":0.83699999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51100000000000001},"max":{"1min":1.153,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.85599999999999998,"15min":0.83899999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0880000000000001,"5min":5.8230000000000004,"15min":5.8230000000000004},"last":0.88400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575076,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":926034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.46800000000000003},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.29699999999999999},"max":{"1min":0.58499999999999996,"5min":0.73899999999999999,"15min":0.84299999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34499999999999997,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.46000000000000002}]},{"osd":2,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45200000000000001,"15min":0.46999999999999997},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.623,"5min":0.67600000000000005,"15min":0.82099999999999995},"last":0.436},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52700000000000002,"15min":0.55000000000000004},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.91000000000000003},"last":0.51000000000000001}]},{"osd":3,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.97199999999999998},"last":0.502}]},{"osd":4,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77400000000000002,"15min":0.80600000000000005},"min":{"1min":0.61699999999999999,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.1299999999999999,"15min":1.1779999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.77400000000000002,"15min":0.80500000000000005},"min":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.056,"5min":1.153,"15min":1.298},"last":0.53400000000000003}]},{"osd":5,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78000000000000003,"15min":0.80400000000000005},"min":{"1min":0.59099999999999997,"5min":0.54700000000000004,"15min":0.48999999999999999},"max":{"1min":1.0549999999999999,"5min":1.069,"15min":1.2010000000000001},"last":0.625},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.51400000000000001},"max":{"1min":1.0940000000000001,"5min":1.2010000000000001,"15min":1.2829999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77800000000000002,"15min":0.79300000000000004},"min":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"max":{"1min":0.997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.79300000000000004,"15min":0.81599999999999995},"min":{"1min":0.59599999999999997,"5min":0.52200000000000002,"15min":0.51800000000000002},"max":{"1min":1.042,"5min":1.0629999999999999,"15min":1.304},"last":0.65800000000000003}]},{"osd":7,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80000000000000004,"15min":0.82199999999999995},"min":{"1min":0.56200000000000006,"5min":0.505,"15min":0.505},"max":{"1min":1.032,"5min":1.171,"15min":1.2490000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.82399999999999995,"15min":0.83899999999999997},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54600000000000004},"max":{"1min":1.083,"5min":1.1899999999999999,"15min":1.236},"last":0.56799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575076,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":926090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.32500000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":1.0920000000000001},"last":0.442},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29499999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.81699999999999995},"last":0.41899999999999998}]},{"osd":1,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.34799999999999998,"15min":0.34599999999999997},"max":{"1min":0.69199999999999995,"5min":0.86099999999999999,"15min":1.6359999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58499999999999996,"15min":0.60399999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.35999999999999999},"max":{"1min":0.83999999999999997,"5min":0.96399999999999997,"15min":1.4490000000000001},"last":0.45200000000000001}]},{"osd":3,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76300000000000001,"5min":0.85899999999999999,"15min":2.266},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79400000000000004,"5min":0.93999999999999995,"15min":1.732},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77700000000000002,"15min":0.78800000000000003},"min":{"1min":0.58999999999999997,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":0.95399999999999996,"5min":1.121,"15min":1.1459999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.77600000000000002},"min":{"1min":0.54300000000000004,"5min":0.46700000000000003,"15min":0.39400000000000002},"max":{"1min":0.96299999999999997,"5min":1.087,"15min":1.357},"last":0.79300000000000004}]},{"osd":5,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.752,"15min":0.77300000000000002},"min":{"1min":0.57899999999999996,"5min":0.51300000000000001,"15min":0.46300000000000002},"max":{"1min":1.052,"5min":1.1619999999999999,"15min":1.657},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0609999999999999,"5min":1.159,"15min":1.159},"last":0.95299999999999996}]},{"osd":6,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78500000000000003,"15min":0.80100000000000005},"min":{"1min":0.51300000000000001,"5min":0.495,"15min":0.46200000000000002},"max":{"1min":0.94399999999999995,"5min":1.1499999999999999,"15min":1.2829999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.81100000000000005},"min":{"1min":0.57099999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.997,"5min":1.099,"15min":1.7330000000000001},"last":1.0700000000000001}]},{"osd":7,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":0.82499999999999996},"min":{"1min":0.59899999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0189999999999999,"5min":1.1870000000000001,"15min":1.448},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80900000000000005,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.52100000000000002},"max":{"1min":1.044,"5min":1.129,"15min":2.202},"last":0.995}]}]},{"osd":3,"up_from":13,"seq":55834575077,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302976,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068864,"statfs":{"total":96636764160,"available":96326516736,"internally_reserved":0,"allocated":1826816,"data_stored":925536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.44500000000000001,"15min":0.45300000000000001},"min":{"1min":0.28499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69399999999999995,"5min":0.71399999999999997,"15min":0.82599999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34000000000000002,"15min":0.32600000000000001},"max":{"1min":0.70499999999999996,"5min":0.86699999999999999,"15min":0.93400000000000005},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.624,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53300000000000003,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69699999999999995,"5min":0.79900000000000004,"15min":0.80900000000000005},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.55400000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.505},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57999999999999996,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.33800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81000000000000005,"15min":0.81699999999999995},"min":{"1min":0.59199999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95499999999999996,"5min":1.234,"15min":1.2470000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80000000000000004,"15min":0.81299999999999994},"min":{"1min":0.504,"5min":0.504,"15min":0.47999999999999998},"max":{"1min":0.92800000000000005,"5min":1.276,"15min":1.286},"last":0.68000000000000005}]},{"osd":5,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.78400000000000003},"min":{"1min":0.59699999999999998,"5min":0.52200000000000002,"15min":0.496},"max":{"1min":0.96699999999999997,"5min":1.1519999999999999,"15min":1.1779999999999999},"last":0.752},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77700000000000002,"15min":0.79400000000000004},"min":{"1min":0.56100000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.99399999999999999,"5min":1.258,"15min":1.258},"last":0.58399999999999996}]},{"osd":6,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76800000000000002,"15min":0.78400000000000003},"min":{"1min":0.57799999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88900000000000001,"5min":1.323,"15min":1.323},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.79200000000000004,"15min":0.81399999999999995},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.53300000000000003},"max":{"1min":0.89100000000000001,"5min":1.054,"15min":1.2130000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79100000000000004,"15min":0.81399999999999995},"min":{"1min":0.621,"5min":0.497,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":1.0780000000000001,"15min":1.302},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.60299999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.93400000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.80100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575076,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302968,"kb_used_data":1776,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068872,"statfs":{"total":96636764160,"available":96326524928,"internally_reserved":0,"allocated":1818624,"data_stored":918798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64500000000000002,"15min":0.68000000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84099999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.40300000000000002},"max":{"1min":0.84699999999999998,"5min":1,"15min":1},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65300000000000002,"15min":0.69199999999999995},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82099999999999995,"5min":1.107,"15min":1.645},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.46300000000000002,"5min":0.45600000000000002,"15min":0.41199999999999998},"max":{"1min":0.877,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.76700000000000002}]},{"osd":2,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67000000000000004,"15min":0.71099999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.46300000000000002},"max":{"1min":0.873,"5min":0.92800000000000005,"15min":1.0569999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.81599999999999995}]},{"osd":3,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65300000000000002,"15min":0.68500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82399999999999995,"5min":0.90300000000000002,"15min":0.95799999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66400000000000003,"15min":0.68799999999999994},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86199999999999999,"5min":0.93200000000000005,"15min":1.1060000000000001},"last":0.745}]},{"osd":5,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80200000000000005,"5min":0.88200000000000001,"15min":1.0389999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.622,"15min":0.59499999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.87,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83199999999999996}]},{"osd":6,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63400000000000001,"15min":0.60699999999999998},"min":{"1min":0.42699999999999999,"5min":0.40899999999999997,"15min":0.376},"max":{"1min":0.84399999999999997,"5min":0.93100000000000005,"15min":1.22},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.66700000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.89700000000000002}]},{"osd":7,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67900000000000005,"15min":0.65800000000000003},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.44},"max":{"1min":0.90100000000000002,"5min":1.034,"15min":1.034},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70099999999999996,"15min":0.67600000000000005},"min":{"1min":0.501,"5min":0.47299999999999998,"15min":0.45400000000000001},"max":{"1min":0.88,"5min":0.95199999999999996,"15min":1.177},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575077,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1835008,"data_stored":926982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67300000000000004,"15min":0.67200000000000004},"min":{"1min":0.42099999999999999,"5min":0.40100000000000002,"15min":0.27300000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.34999999999999998},"max":{"1min":0.92300000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.874}]},{"osd":1,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70499999999999996,"15min":0.71199999999999997},"min":{"1min":0.52500000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90300000000000002,"5min":0.92300000000000004,"15min":0.99399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.995,"5min":1.026,"15min":1.026},"last":0.85799999999999998}]},{"osd":2,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71199999999999997,"15min":0.73499999999999999},"min":{"1min":0.54600000000000004,"5min":0.41199999999999998,"15min":0.36299999999999999},"max":{"1min":0.95799999999999996,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52900000000000003,"5min":0.46500000000000002,"15min":0.45600000000000002},"max":{"1min":0.96999999999999997,"5min":0.98099999999999998,"15min":1.0089999999999999},"last":0.79300000000000004}]},{"osd":3,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69399999999999995,"15min":0.70399999999999996},"min":{"1min":0.49399999999999999,"5min":0.45100000000000001,"15min":0.377},"max":{"1min":1,"5min":1,"15min":1},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71299999999999997,"15min":0.71399999999999997},"min":{"1min":0.47799999999999998,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.98399999999999999,"5min":1.175,"15min":1.175},"last":0.83899999999999997}]},{"osd":4,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0389999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.39400000000000002,"15min":0.35599999999999998},"max":{"1min":1.018,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.92300000000000004}]},{"osd":6,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64600000000000002,"15min":0.627},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.98099999999999998,"5min":1.214,"15min":1.214},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.52600000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0820000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.40600000000000003},"max":{"1min":0.94999999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0589999999999999,"5min":1.258,"15min":1.258},"last":0.89800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T21:03:26.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T21:03:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.251+0000 7f92fd7ff700 1 Processor -- start 2023-11-06T21:03:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.251+0000 7f92fd7ff700 1 -- start start 2023-11-06T21:03:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.252+0000 7f92fd7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f92f8154280 0x7f92f8154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.252+0000 7f92fd7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f92f80ad6e0 0x7f92f80c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.252+0000 7f92fd7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f80c5db0 con 0x7f92f80a9f20 2023-11-06T21:03:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.252+0000 7f92fd7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f80c6930 con 0x7f92f80ad6e0 2023-11-06T21:03:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.252+0000 7f92fd7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f80c6000 con 0x7f92f8154280 2023-11-06T21:03:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.253+0000 7f92f67fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f92f80ad6e0 0x7f92f80c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.253+0000 7f92f67fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f92f80ad6e0 0x7f92f80c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50194/0 (socket says 172.21.15.33:50194) 2023-11-06T21:03:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.253+0000 7f92f67fc700 1 -- 172.21.15.33:0/1376900040 learned_addr learned my addr 172.21.15.33:0/1376900040 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.253+0000 7f92f6ffd700 1 --2- 172.21.15.33:0/1376900040 >> v2:172.21.15.69:3300/0 conn(0x7f92f8154280 0x7f92f8154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.253+0000 7f92f67fc700 1 -- 172.21.15.33:0/1376900040 >> v2:172.21.15.69:3300/0 conn(0x7f92f8154280 msgr2=0x7f92f8154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.253+0000 7f92f67fc700 1 --2- 172.21.15.33:0/1376900040 >> v2:172.21.15.69:3300/0 conn(0x7f92f8154280 0x7f92f8154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.253+0000 7f92f67fc700 1 -- 172.21.15.33:0/1376900040 >> v1:172.21.15.69:6789/0 conn(0x7f92f80a9f20 legacy=0x7f92f80c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:26.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.253+0000 7f92f67fc700 1 -- 172.21.15.33:0/1376900040 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f80c62e0 con 0x7f92f80ad6e0 2023-11-06T21:03:26.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.253+0000 7f92f67fc700 1 --2- 172.21.15.33:0/1376900040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f80ad6e0 0x7f92f80c6dc0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f92f8065150 tx=0x7f92ec0090f0).ready entity=mon.0 client_cookie=6d6c458a08e37060 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:26.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92f5ffb700 1 -- 172.21.15.33:0/1376900040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec00b5a0 con 0x7f92f80ad6e0 2023-11-06T21:03:26.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92f5ffb700 1 -- 172.21.15.33:0/1376900040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec00b700 con 0x7f92f80ad6e0 2023-11-06T21:03:26.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92f5ffb700 1 -- 172.21.15.33:0/1376900040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec00ba80 con 0x7f92f80ad6e0 2023-11-06T21:03:26.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92fd7ff700 1 -- 172.21.15.33:0/1376900040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f80ad6e0 msgr2=0x7f92f80c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92fd7ff700 1 --2- 172.21.15.33:0/1376900040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f80ad6e0 0x7f92f80c6dc0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f92f8065150 tx=0x7f92ec0090f0).stop 2023-11-06T21:03:26.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92fd7ff700 1 -- 172.21.15.33:0/1376900040 shutdown_connections 2023-11-06T21:03:26.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92fd7ff700 1 --2- 172.21.15.33:0/1376900040 >> v2:172.21.15.69:3300/0 conn(0x7f92f8154280 0x7f92f8154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92fd7ff700 1 --2- 172.21.15.33:0/1376900040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f80ad6e0 0x7f92f80c6dc0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92fd7ff700 1 -- 172.21.15.33:0/1376900040 >> 172.21.15.33:0/1376900040 conn(0x7f92f805e1b0 msgr2=0x7f92f805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92fd7ff700 1 -- 172.21.15.33:0/1376900040 shutdown_connections 2023-11-06T21:03:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.254+0000 7f92fd7ff700 1 -- 172.21.15.33:0/1376900040 wait complete. 2023-11-06T21:03:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.256+0000 7f92fd7ff700 1 Processor -- start 2023-11-06T21:03:26.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.256+0000 7f92fd7ff700 1 -- start start 2023-11-06T21:03:26.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92fd7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f80a9f20 0x7f92f80cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92fd7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f92f80ad6e0 0x7f92f81428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92fd7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f8154280 0x7f92f80c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92fd7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8150230 con 0x7f92f8154280 2023-11-06T21:03:26.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92fd7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80ca8d0 con 0x7f92f80a9f20 2023-11-06T21:03:26.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92fd7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80cabb0 con 0x7f92f80ad6e0 2023-11-06T21:03:26.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f8154280 0x7f92f80c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92f6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f80a9f20 0x7f92f80cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f8154280 0x7f92f80c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50200/0 (socket says 172.21.15.33:50200) 2023-11-06T21:03:26.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92f67fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f92f80ad6e0 0x7f92f81428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.257+0000 7f92f77fe700 1 -- 172.21.15.33:0/721405447 learned_addr learned my addr 172.21.15.33:0/721405447 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:26.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92f77fe700 1 -- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f92f80ad6e0 msgr2=0x7f92f81428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92f77fe700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f92f80ad6e0 0x7f92f81428c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92f77fe700 1 -- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f80a9f20 msgr2=0x7f92f80cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92f77fe700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f80a9f20 0x7f92f80cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92f77fe700 1 -- 172.21.15.33:0/721405447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec009370 con 0x7f92f8154280 2023-11-06T21:03:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92f77fe700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f8154280 0x7f92f80c9d10 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f92f000c810 tx=0x7f92f0002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92deffd700 1 -- 172.21.15.33:0/721405447 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0010510 con 0x7f92f8154280 2023-11-06T21:03:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92deffd700 1 -- 172.21.15.33:0/721405447 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f00106b0 con 0x7f92f8154280 2023-11-06T21:03:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92fd7ff700 1 -- 172.21.15.33:0/721405447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f80cae90 con 0x7f92f8154280 2023-11-06T21:03:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92deffd700 1 -- 172.21.15.33:0/721405447 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0010a70 con 0x7f92f8154280 2023-11-06T21:03:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.258+0000 7f92fd7ff700 1 -- 172.21.15.33:0/721405447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f80cb450 con 0x7f92f8154280 2023-11-06T21:03:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.260+0000 7f92fd7ff700 1 -- 172.21.15.33:0/721405447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f80c65c0 con 0x7f92f8154280 2023-11-06T21:03:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.261+0000 7f92deffd700 1 -- 172.21.15.33:0/721405447 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f92f0012440 con 0x7f92f8154280 2023-11-06T21:03:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.261+0000 7f92deffd700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f92e405f270 0x7f92e4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.262+0000 7f92deffd700 1 -- 172.21.15.33:0/721405447 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f92f0079d00 con 0x7f92f8154280 2023-11-06T21:03:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.262+0000 7f92f6ffd700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f92e405f270 0x7f92e4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.263+0000 7f92f6ffd700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f92e405f270 0x7f92e4061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f92e800bb10 tx=0x7f92e8008040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:26.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.265+0000 7f92deffd700 1 -- 172.21.15.33:0/721405447 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f92f0048af0 con 0x7f92f8154280 2023-11-06T21:03:26.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.444+0000 7f92fd7ff700 1 -- 172.21.15.33:0/721405447 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92f80cad20 con 0x7f92e405f270 2023-11-06T21:03:26.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.455+0000 7f92deffd700 1 -- 172.21.15.33:0/721405447 <== mgr.6692 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361880 (secure 0 0 0) 0x7f92f80cad20 con 0x7f92e405f270 2023-11-06T21:03:26.460 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:26.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 -- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f92e405f270 msgr2=0x7f92e4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f92e405f270 0x7f92e4061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f92e800bb10 tx=0x7f92e8008040).stop 2023-11-06T21:03:26.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 -- 172.21.15.33:0/721405447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f8154280 msgr2=0x7f92f80c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f8154280 0x7f92f80c9d10 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f92f000c810 tx=0x7f92f0002560).stop 2023-11-06T21:03:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 -- 172.21.15.33:0/721405447 shutdown_connections 2023-11-06T21:03:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f92e405f270 0x7f92e4061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92f8154280 0x7f92f80c9d10 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f80a9f20 0x7f92f80cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 --2- 172.21.15.33:0/721405447 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f92f80ad6e0 0x7f92f81428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 -- 172.21.15.33:0/721405447 >> 172.21.15.33:0/721405447 conn(0x7f92f805e1b0 msgr2=0x7f92f80ac3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:26.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 -- 172.21.15.33:0/721405447 shutdown_connections 2023-11-06T21:03:26.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.459+0000 7f92fd7ff700 1 -- 172.21.15.33:0/721405447 wait complete. 2023-11-06T21:03:26.467 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T21:03:26.478 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T21:03:26.347636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2423828,"kb_used_data":14292,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752550892,"statfs":{"total":773094113280,"available":770612113408,"internally_reserved":0,"allocated":14635008,"data_stored":7401476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20937,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-73,"num_write_kb":-48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.004291"},"pg_stats":[{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719502+0000","last_change":"2023-11-06T21:03:21.716328+0000","last_active":"2023-11-06T21:03:21.719502+0000","last_peered":"2023-11-06T21:03:21.719502+0000","last_clean":"2023-11-06T21:03:21.719502+0000","last_became_active":"2023-11-06T21:03:21.714842+0000","last_became_peered":"2023-11-06T21:03:21.714842+0000","last_unstale":"2023-11-06T21:03:21.719502+0000","last_undegraded":"2023-11-06T21:03:21.719502+0000","last_fullsized":"2023-11-06T21:03:21.719502+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3f","version":"146'3","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.723126+0000","last_change":"2023-11-06T21:03:21.233431+0000","last_active":"2023-11-06T21:03:21.723126+0000","last_peered":"2023-11-06T21:03:21.723126+0000","last_clean":"2023-11-06T21:03:21.723126+0000","last_became_active":"2023-11-06T21:03:21.232386+0000","last_became_peered":"2023-11-06T21:03:21.232386+0000","last_unstale":"2023-11-06T21:03:21.723126+0000","last_undegraded":"2023-11-06T21:03:21.723126+0000","last_fullsized":"2023-11-06T21:03:21.723126+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.718248+0000","last_change":"2023-11-06T21:03:21.715517+0000","last_active":"2023-11-06T21:03:21.718248+0000","last_peered":"2023-11-06T21:03:21.718248+0000","last_clean":"2023-11-06T21:03:21.718248+0000","last_became_active":"2023-11-06T21:03:21.715227+0000","last_became_peered":"2023-11-06T21:03:21.715227+0000","last_unstale":"2023-11-06T21:03:21.718248+0000","last_undegraded":"2023-11-06T21:03:21.718248+0000","last_fullsized":"2023-11-06T21:03:21.718248+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.698683+0000","last_change":"2023-11-06T21:03:21.232828+0000","last_active":"2023-11-06T21:03:21.698683+0000","last_peered":"2023-11-06T21:03:21.698683+0000","last_clean":"2023-11-06T21:03:21.698683+0000","last_became_active":"2023-11-06T21:03:21.232202+0000","last_became_peered":"2023-11-06T21:03:21.232202+0000","last_unstale":"2023-11-06T21:03:21.698683+0000","last_undegraded":"2023-11-06T21:03:21.698683+0000","last_fullsized":"2023-11-06T21:03:21.698683+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.712831+0000","last_change":"2023-11-06T21:03:21.702662+0000","last_active":"2023-11-06T21:03:21.712831+0000","last_peered":"2023-11-06T21:03:21.712831+0000","last_clean":"2023-11-06T21:03:21.712831+0000","last_became_active":"2023-11-06T21:03:21.702378+0000","last_became_peered":"2023-11-06T21:03:21.702378+0000","last_unstale":"2023-11-06T21:03:21.712831+0000","last_undegraded":"2023-11-06T21:03:21.712831+0000","last_fullsized":"2023-11-06T21:03:21.712831+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693771+0000","last_change":"2023-11-06T21:03:21.225199+0000","last_active":"2023-11-06T21:03:21.693771+0000","last_peered":"2023-11-06T21:03:21.693771+0000","last_clean":"2023-11-06T21:03:21.693771+0000","last_became_active":"2023-11-06T21:03:21.224918+0000","last_became_peered":"2023-11-06T21:03:21.224918+0000","last_unstale":"2023-11-06T21:03:21.693771+0000","last_undegraded":"2023-11-06T21:03:21.693771+0000","last_fullsized":"2023-11-06T21:03:21.693771+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726200+0000","last_change":"2023-11-06T21:03:21.714432+0000","last_active":"2023-11-06T21:03:21.726200+0000","last_peered":"2023-11-06T21:03:21.726200+0000","last_clean":"2023-11-06T21:03:21.726200+0000","last_became_active":"2023-11-06T21:03:21.713732+0000","last_became_peered":"2023-11-06T21:03:21.713732+0000","last_unstale":"2023-11-06T21:03:21.726200+0000","last_undegraded":"2023-11-06T21:03:21.726200+0000","last_fullsized":"2023-11-06T21:03:21.726200+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3c","version":"146'2","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726934+0000","last_change":"2023-11-06T21:03:21.230736+0000","last_active":"2023-11-06T21:03:21.726934+0000","last_peered":"2023-11-06T21:03:21.726934+0000","last_clean":"2023-11-06T21:03:21.726934+0000","last_became_active":"2023-11-06T21:03:21.230381+0000","last_became_peered":"2023-11-06T21:03:21.230381+0000","last_unstale":"2023-11-06T21:03:21.726934+0000","last_undegraded":"2023-11-06T21:03:21.726934+0000","last_fullsized":"2023-11-06T21:03:21.726934+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.715218+0000","last_change":"2023-11-06T21:03:21.707538+0000","last_active":"2023-11-06T21:03:21.715218+0000","last_peered":"2023-11-06T21:03:21.715218+0000","last_clean":"2023-11-06T21:03:21.715218+0000","last_became_active":"2023-11-06T21:03:21.707165+0000","last_became_peered":"2023-11-06T21:03:21.707165+0000","last_unstale":"2023-11-06T21:03:21.715218+0000","last_undegraded":"2023-11-06T21:03:21.715218+0000","last_fullsized":"2023-11-06T21:03:21.715218+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.711564+0000","last_change":"2023-11-06T21:03:21.231433+0000","last_active":"2023-11-06T21:03:21.711564+0000","last_peered":"2023-11-06T21:03:21.711564+0000","last_clean":"2023-11-06T21:03:21.711564+0000","last_became_active":"2023-11-06T21:03:21.231142+0000","last_became_peered":"2023-11-06T21:03:21.231142+0000","last_unstale":"2023-11-06T21:03:21.711564+0000","last_undegraded":"2023-11-06T21:03:21.711564+0000","last_fullsized":"2023-11-06T21:03:21.711564+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716811+0000","last_change":"2023-11-06T21:03:21.711731+0000","last_active":"2023-11-06T21:03:21.716811+0000","last_peered":"2023-11-06T21:03:21.716811+0000","last_clean":"2023-11-06T21:03:21.716811+0000","last_became_active":"2023-11-06T21:03:21.710021+0000","last_became_peered":"2023-11-06T21:03:21.710021+0000","last_unstale":"2023-11-06T21:03:21.716811+0000","last_undegraded":"2023-11-06T21:03:21.716811+0000","last_fullsized":"2023-11-06T21:03:21.716811+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3a","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716302+0000","last_change":"2023-11-06T21:03:21.231979+0000","last_active":"2023-11-06T21:03:21.716302+0000","last_peered":"2023-11-06T21:03:21.716302+0000","last_clean":"2023-11-06T21:03:21.716302+0000","last_became_active":"2023-11-06T21:03:21.231191+0000","last_became_peered":"2023-11-06T21:03:21.231191+0000","last_unstale":"2023-11-06T21:03:21.716302+0000","last_undegraded":"2023-11-06T21:03:21.716302+0000","last_fullsized":"2023-11-06T21:03:21.716302+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.714466+0000","last_change":"2023-11-06T21:03:21.705387+0000","last_active":"2023-11-06T21:03:21.714466+0000","last_peered":"2023-11-06T21:03:21.714466+0000","last_clean":"2023-11-06T21:03:21.714466+0000","last_became_active":"2023-11-06T21:03:21.705124+0000","last_became_peered":"2023-11-06T21:03:21.705124+0000","last_unstale":"2023-11-06T21:03:21.714466+0000","last_undegraded":"2023-11-06T21:03:21.714466+0000","last_fullsized":"2023-11-06T21:03:21.714466+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.689695+0000","last_change":"2023-11-06T21:03:21.227622+0000","last_active":"2023-11-06T21:03:21.689695+0000","last_peered":"2023-11-06T21:03:21.689695+0000","last_clean":"2023-11-06T21:03:21.689695+0000","last_became_active":"2023-11-06T21:03:21.227277+0000","last_became_peered":"2023-11-06T21:03:21.227277+0000","last_unstale":"2023-11-06T21:03:21.689695+0000","last_undegraded":"2023-11-06T21:03:21.689695+0000","last_fullsized":"2023-11-06T21:03:21.689695+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726756+0000","last_change":"2023-11-06T21:03:21.723295+0000","last_active":"2023-11-06T21:03:21.726756+0000","last_peered":"2023-11-06T21:03:21.726756+0000","last_clean":"2023-11-06T21:03:21.726756+0000","last_became_active":"2023-11-06T21:03:21.723007+0000","last_became_peered":"2023-11-06T21:03:21.723007+0000","last_unstale":"2023-11-06T21:03:21.726756+0000","last_undegraded":"2023-11-06T21:03:21.726756+0000","last_fullsized":"2023-11-06T21:03:21.726756+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.689571+0000","last_change":"2023-11-06T21:03:21.210735+0000","last_active":"2023-11-06T21:03:21.689571+0000","last_peered":"2023-11-06T21:03:21.689571+0000","last_clean":"2023-11-06T21:03:21.689571+0000","last_became_active":"2023-11-06T21:03:21.210444+0000","last_became_peered":"2023-11-06T21:03:21.210444+0000","last_unstale":"2023-11-06T21:03:21.689571+0000","last_undegraded":"2023-11-06T21:03:21.689571+0000","last_fullsized":"2023-11-06T21:03:21.689571+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.728031+0000","last_change":"2023-11-06T21:03:21.726836+0000","last_active":"2023-11-06T21:03:21.728031+0000","last_peered":"2023-11-06T21:03:21.728031+0000","last_clean":"2023-11-06T21:03:21.728031+0000","last_became_active":"2023-11-06T21:03:21.726525+0000","last_became_peered":"2023-11-06T21:03:21.726525+0000","last_unstale":"2023-11-06T21:03:21.728031+0000","last_undegraded":"2023-11-06T21:03:21.728031+0000","last_fullsized":"2023-11-06T21:03:21.728031+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.704856+0000","last_change":"2023-11-06T21:03:21.227201+0000","last_active":"2023-11-06T21:03:21.704856+0000","last_peered":"2023-11-06T21:03:21.704856+0000","last_clean":"2023-11-06T21:03:21.704856+0000","last_became_active":"2023-11-06T21:03:21.226904+0000","last_became_peered":"2023-11-06T21:03:21.226904+0000","last_unstale":"2023-11-06T21:03:21.704856+0000","last_undegraded":"2023-11-06T21:03:21.704856+0000","last_fullsized":"2023-11-06T21:03:21.704856+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716415+0000","last_change":"2023-11-06T21:03:21.699822+0000","last_active":"2023-11-06T21:03:21.716415+0000","last_peered":"2023-11-06T21:03:21.716415+0000","last_clean":"2023-11-06T21:03:21.716415+0000","last_became_active":"2023-11-06T21:03:21.699554+0000","last_became_peered":"2023-11-06T21:03:21.699554+0000","last_unstale":"2023-11-06T21:03:21.716415+0000","last_undegraded":"2023-11-06T21:03:21.716415+0000","last_fullsized":"2023-11-06T21:03:21.716415+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.694262+0000","last_change":"2023-11-06T21:03:21.233807+0000","last_active":"2023-11-06T21:03:21.694262+0000","last_peered":"2023-11-06T21:03:21.694262+0000","last_clean":"2023-11-06T21:03:21.694262+0000","last_became_active":"2023-11-06T21:03:21.233535+0000","last_became_peered":"2023-11-06T21:03:21.233535+0000","last_unstale":"2023-11-06T21:03:21.694262+0000","last_undegraded":"2023-11-06T21:03:21.694262+0000","last_fullsized":"2023-11-06T21:03:21.694262+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.724676+0000","last_change":"2023-11-06T21:03:21.720216+0000","last_active":"2023-11-06T21:03:21.724676+0000","last_peered":"2023-11-06T21:03:21.724676+0000","last_clean":"2023-11-06T21:03:21.724676+0000","last_became_active":"2023-11-06T21:03:21.719882+0000","last_became_peered":"2023-11-06T21:03:21.719882+0000","last_unstale":"2023-11-06T21:03:21.724676+0000","last_undegraded":"2023-11-06T21:03:21.724676+0000","last_fullsized":"2023-11-06T21:03:21.724676+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691263+0000","last_change":"2023-11-06T21:03:21.225241+0000","last_active":"2023-11-06T21:03:21.691263+0000","last_peered":"2023-11-06T21:03:21.691263+0000","last_clean":"2023-11-06T21:03:21.691263+0000","last_became_active":"2023-11-06T21:03:21.224975+0000","last_became_peered":"2023-11-06T21:03:21.224975+0000","last_unstale":"2023-11-06T21:03:21.691263+0000","last_undegraded":"2023-11-06T21:03:21.691263+0000","last_fullsized":"2023-11-06T21:03:21.691263+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725809+0000","last_change":"2023-11-06T21:03:21.720263+0000","last_active":"2023-11-06T21:03:21.725809+0000","last_peered":"2023-11-06T21:03:21.725809+0000","last_clean":"2023-11-06T21:03:21.725809+0000","last_became_active":"2023-11-06T21:03:21.719974+0000","last_became_peered":"2023-11-06T21:03:21.719974+0000","last_unstale":"2023-11-06T21:03:21.725809+0000","last_undegraded":"2023-11-06T21:03:21.725809+0000","last_fullsized":"2023-11-06T21:03:21.725809+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.34","version":"146'4","reported_seq":14,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.718301+0000","last_change":"2023-11-06T21:03:21.233255+0000","last_active":"2023-11-06T21:03:21.718301+0000","last_peered":"2023-11-06T21:03:21.718301+0000","last_clean":"2023-11-06T21:03:21.718301+0000","last_became_active":"2023-11-06T21:03:21.232129+0000","last_became_peered":"2023-11-06T21:03:21.232129+0000","last_unstale":"2023-11-06T21:03:21.718301+0000","last_undegraded":"2023-11-06T21:03:21.718301+0000","last_fullsized":"2023-11-06T21:03:21.718301+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719750+0000","last_change":"2023-11-06T21:03:21.717930+0000","last_active":"2023-11-06T21:03:21.719750+0000","last_peered":"2023-11-06T21:03:21.719750+0000","last_clean":"2023-11-06T21:03:21.719750+0000","last_became_active":"2023-11-06T21:03:21.717637+0000","last_became_peered":"2023-11-06T21:03:21.717637+0000","last_unstale":"2023-11-06T21:03:21.719750+0000","last_undegraded":"2023-11-06T21:03:21.719750+0000","last_fullsized":"2023-11-06T21:03:21.719750+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.33","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.718767+0000","last_change":"2023-11-06T21:03:21.228252+0000","last_active":"2023-11-06T21:03:21.718767+0000","last_peered":"2023-11-06T21:03:21.718767+0000","last_clean":"2023-11-06T21:03:21.718767+0000","last_became_active":"2023-11-06T21:03:21.227965+0000","last_became_peered":"2023-11-06T21:03:21.227965+0000","last_unstale":"2023-11-06T21:03:21.718767+0000","last_undegraded":"2023-11-06T21:03:21.718767+0000","last_fullsized":"2023-11-06T21:03:21.718767+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.711898+0000","last_change":"2023-11-06T21:03:21.706612+0000","last_active":"2023-11-06T21:03:21.711898+0000","last_peered":"2023-11-06T21:03:21.711898+0000","last_clean":"2023-11-06T21:03:21.711898+0000","last_became_active":"2023-11-06T21:03:21.706299+0000","last_became_peered":"2023-11-06T21:03:21.706299+0000","last_unstale":"2023-11-06T21:03:21.711898+0000","last_undegraded":"2023-11-06T21:03:21.711898+0000","last_fullsized":"2023-11-06T21:03:21.711898+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.705894+0000","last_change":"2023-11-06T21:03:21.224674+0000","last_active":"2023-11-06T21:03:21.705894+0000","last_peered":"2023-11-06T21:03:21.705894+0000","last_clean":"2023-11-06T21:03:21.705894+0000","last_became_active":"2023-11-06T21:03:21.224087+0000","last_became_peered":"2023-11-06T21:03:21.224087+0000","last_unstale":"2023-11-06T21:03:21.705894+0000","last_undegraded":"2023-11-06T21:03:21.705894+0000","last_fullsized":"2023-11-06T21:03:21.705894+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719827+0000","last_change":"2023-11-06T21:03:21.717340+0000","last_active":"2023-11-06T21:03:21.719827+0000","last_peered":"2023-11-06T21:03:21.719827+0000","last_clean":"2023-11-06T21:03:21.719827+0000","last_became_active":"2023-11-06T21:03:21.717068+0000","last_became_peered":"2023-11-06T21:03:21.717068+0000","last_unstale":"2023-11-06T21:03:21.719827+0000","last_undegraded":"2023-11-06T21:03:21.719827+0000","last_fullsized":"2023-11-06T21:03:21.719827+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.706433+0000","last_change":"2023-11-06T21:03:21.217712+0000","last_active":"2023-11-06T21:03:21.706433+0000","last_peered":"2023-11-06T21:03:21.706433+0000","last_clean":"2023-11-06T21:03:21.706433+0000","last_became_active":"2023-11-06T21:03:21.217431+0000","last_became_peered":"2023-11-06T21:03:21.217431+0000","last_unstale":"2023-11-06T21:03:21.706433+0000","last_undegraded":"2023-11-06T21:03:21.706433+0000","last_fullsized":"2023-11-06T21:03:21.706433+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716668+0000","last_change":"2023-11-06T21:03:21.709166+0000","last_active":"2023-11-06T21:03:21.716668+0000","last_peered":"2023-11-06T21:03:21.716668+0000","last_clean":"2023-11-06T21:03:21.716668+0000","last_became_active":"2023-11-06T21:03:21.708145+0000","last_became_peered":"2023-11-06T21:03:21.708145+0000","last_unstale":"2023-11-06T21:03:21.716668+0000","last_undegraded":"2023-11-06T21:03:21.716668+0000","last_fullsized":"2023-11-06T21:03:21.716668+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.30","version":"146'4","reported_seq":14,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.721221+0000","last_change":"2023-11-06T21:03:21.231178+0000","last_active":"2023-11-06T21:03:21.721221+0000","last_peered":"2023-11-06T21:03:21.721221+0000","last_clean":"2023-11-06T21:03:21.721221+0000","last_became_active":"2023-11-06T21:03:21.230797+0000","last_became_peered":"2023-11-06T21:03:21.230797+0000","last_unstale":"2023-11-06T21:03:21.721221+0000","last_undegraded":"2023-11-06T21:03:21.721221+0000","last_fullsized":"2023-11-06T21:03:21.721221+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.724701+0000","last_change":"2023-11-06T21:03:21.720441+0000","last_active":"2023-11-06T21:03:21.724701+0000","last_peered":"2023-11-06T21:03:21.724701+0000","last_clean":"2023-11-06T21:03:21.724701+0000","last_became_active":"2023-11-06T21:03:21.720104+0000","last_became_peered":"2023-11-06T21:03:21.720104+0000","last_unstale":"2023-11-06T21:03:21.724701+0000","last_undegraded":"2023-11-06T21:03:21.724701+0000","last_fullsized":"2023-11-06T21:03:21.724701+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692898+0000","last_change":"2023-11-06T21:03:21.232111+0000","last_active":"2023-11-06T21:03:21.692898+0000","last_peered":"2023-11-06T21:03:21.692898+0000","last_clean":"2023-11-06T21:03:21.692898+0000","last_became_active":"2023-11-06T21:03:21.231796+0000","last_became_peered":"2023-11-06T21:03:21.231796+0000","last_unstale":"2023-11-06T21:03:21.692898+0000","last_undegraded":"2023-11-06T21:03:21.692898+0000","last_fullsized":"2023-11-06T21:03:21.692898+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725098+0000","last_change":"2023-11-06T21:03:21.719951+0000","last_active":"2023-11-06T21:03:21.725098+0000","last_peered":"2023-11-06T21:03:21.725098+0000","last_clean":"2023-11-06T21:03:21.725098+0000","last_became_active":"2023-11-06T21:03:21.717893+0000","last_became_peered":"2023-11-06T21:03:21.717893+0000","last_unstale":"2023-11-06T21:03:21.725098+0000","last_undegraded":"2023-11-06T21:03:21.725098+0000","last_fullsized":"2023-11-06T21:03:21.725098+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691214+0000","last_change":"2023-11-06T21:03:21.218758+0000","last_active":"2023-11-06T21:03:21.691214+0000","last_peered":"2023-11-06T21:03:21.691214+0000","last_clean":"2023-11-06T21:03:21.691214+0000","last_became_active":"2023-11-06T21:03:21.218094+0000","last_became_peered":"2023-11-06T21:03:21.218094+0000","last_unstale":"2023-11-06T21:03:21.691214+0000","last_undegraded":"2023-11-06T21:03:21.691214+0000","last_fullsized":"2023-11-06T21:03:21.691214+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.722571+0000","last_change":"2023-11-06T21:03:21.719952+0000","last_active":"2023-11-06T21:03:21.722571+0000","last_peered":"2023-11-06T21:03:21.722571+0000","last_clean":"2023-11-06T21:03:21.722571+0000","last_became_active":"2023-11-06T21:03:21.719160+0000","last_became_peered":"2023-11-06T21:03:21.719160+0000","last_unstale":"2023-11-06T21:03:21.722571+0000","last_undegraded":"2023-11-06T21:03:21.722571+0000","last_fullsized":"2023-11-06T21:03:21.722571+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.717408+0000","last_change":"2023-11-06T21:03:21.230392+0000","last_active":"2023-11-06T21:03:21.717408+0000","last_peered":"2023-11-06T21:03:21.717408+0000","last_clean":"2023-11-06T21:03:21.717408+0000","last_became_active":"2023-11-06T21:03:21.229515+0000","last_became_peered":"2023-11-06T21:03:21.229515+0000","last_unstale":"2023-11-06T21:03:21.717408+0000","last_undegraded":"2023-11-06T21:03:21.717408+0000","last_fullsized":"2023-11-06T21:03:21.717408+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.718040+0000","last_change":"2023-11-06T21:03:21.716474+0000","last_active":"2023-11-06T21:03:21.718040+0000","last_peered":"2023-11-06T21:03:21.718040+0000","last_clean":"2023-11-06T21:03:21.718040+0000","last_became_active":"2023-11-06T21:03:21.716211+0000","last_became_peered":"2023-11-06T21:03:21.716211+0000","last_unstale":"2023-11-06T21:03:21.718040+0000","last_undegraded":"2023-11-06T21:03:21.718040+0000","last_fullsized":"2023-11-06T21:03:21.718040+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693121+0000","last_change":"2023-11-06T21:03:21.233090+0000","last_active":"2023-11-06T21:03:21.693121+0000","last_peered":"2023-11-06T21:03:21.693121+0000","last_clean":"2023-11-06T21:03:21.693121+0000","last_became_active":"2023-11-06T21:03:21.231497+0000","last_became_peered":"2023-11-06T21:03:21.231497+0000","last_unstale":"2023-11-06T21:03:21.693121+0000","last_undegraded":"2023-11-06T21:03:21.693121+0000","last_fullsized":"2023-11-06T21:03:21.693121+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.723592+0000","last_change":"2023-11-06T21:03:21.721977+0000","last_active":"2023-11-06T21:03:21.723592+0000","last_peered":"2023-11-06T21:03:21.723592+0000","last_clean":"2023-11-06T21:03:21.723592+0000","last_became_active":"2023-11-06T21:03:21.720055+0000","last_became_peered":"2023-11-06T21:03:21.720055+0000","last_unstale":"2023-11-06T21:03:21.723592+0000","last_undegraded":"2023-11-06T21:03:21.723592+0000","last_fullsized":"2023-11-06T21:03:21.723592+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.727318+0000","last_change":"2023-11-06T21:03:21.228257+0000","last_active":"2023-11-06T21:03:21.727318+0000","last_peered":"2023-11-06T21:03:21.727318+0000","last_clean":"2023-11-06T21:03:21.727318+0000","last_became_active":"2023-11-06T21:03:21.227350+0000","last_became_peered":"2023-11-06T21:03:21.227350+0000","last_unstale":"2023-11-06T21:03:21.727318+0000","last_undegraded":"2023-11-06T21:03:21.727318+0000","last_fullsized":"2023-11-06T21:03:21.727318+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.711877+0000","last_change":"2023-11-06T21:03:21.706516+0000","last_active":"2023-11-06T21:03:21.711877+0000","last_peered":"2023-11-06T21:03:21.711877+0000","last_clean":"2023-11-06T21:03:21.711877+0000","last_became_active":"2023-11-06T21:03:21.706165+0000","last_became_peered":"2023-11-06T21:03:21.706165+0000","last_unstale":"2023-11-06T21:03:21.711877+0000","last_undegraded":"2023-11-06T21:03:21.711877+0000","last_fullsized":"2023-11-06T21:03:21.711877+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691523+0000","last_change":"2023-11-06T21:03:21.212700+0000","last_active":"2023-11-06T21:03:21.691523+0000","last_peered":"2023-11-06T21:03:21.691523+0000","last_clean":"2023-11-06T21:03:21.691523+0000","last_became_active":"2023-11-06T21:03:21.212367+0000","last_became_peered":"2023-11-06T21:03:21.212367+0000","last_unstale":"2023-11-06T21:03:21.691523+0000","last_undegraded":"2023-11-06T21:03:21.691523+0000","last_fullsized":"2023-11-06T21:03:21.691523+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.722781+0000","last_change":"2023-11-06T21:03:21.719540+0000","last_active":"2023-11-06T21:03:21.722781+0000","last_peered":"2023-11-06T21:03:21.722781+0000","last_clean":"2023-11-06T21:03:21.722781+0000","last_became_active":"2023-11-06T21:03:21.717861+0000","last_became_peered":"2023-11-06T21:03:21.717861+0000","last_unstale":"2023-11-06T21:03:21.722781+0000","last_undegraded":"2023-11-06T21:03:21.722781+0000","last_fullsized":"2023-11-06T21:03:21.722781+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.699075+0000","last_change":"2023-11-06T21:03:21.226920+0000","last_active":"2023-11-06T21:03:21.699075+0000","last_peered":"2023-11-06T21:03:21.699075+0000","last_clean":"2023-11-06T21:03:21.699075+0000","last_became_active":"2023-11-06T21:03:21.226630+0000","last_became_peered":"2023-11-06T21:03:21.226630+0000","last_unstale":"2023-11-06T21:03:21.699075+0000","last_undegraded":"2023-11-06T21:03:21.699075+0000","last_fullsized":"2023-11-06T21:03:21.699075+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725391+0000","last_change":"2023-11-06T21:03:21.720772+0000","last_active":"2023-11-06T21:03:21.725391+0000","last_peered":"2023-11-06T21:03:21.725391+0000","last_clean":"2023-11-06T21:03:21.725391+0000","last_became_active":"2023-11-06T21:03:21.720488+0000","last_became_peered":"2023-11-06T21:03:21.720488+0000","last_unstale":"2023-11-06T21:03:21.725391+0000","last_undegraded":"2023-11-06T21:03:21.725391+0000","last_fullsized":"2023-11-06T21:03:21.725391+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.699909+0000","last_change":"2023-11-06T21:03:21.223799+0000","last_active":"2023-11-06T21:03:21.699909+0000","last_peered":"2023-11-06T21:03:21.699909+0000","last_clean":"2023-11-06T21:03:21.699909+0000","last_became_active":"2023-11-06T21:03:21.223503+0000","last_became_peered":"2023-11-06T21:03:21.223503+0000","last_unstale":"2023-11-06T21:03:21.699909+0000","last_undegraded":"2023-11-06T21:03:21.699909+0000","last_fullsized":"2023-11-06T21:03:21.699909+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725102+0000","last_change":"2023-11-06T21:03:21.723673+0000","last_active":"2023-11-06T21:03:21.725102+0000","last_peered":"2023-11-06T21:03:21.725102+0000","last_clean":"2023-11-06T21:03:21.725102+0000","last_became_active":"2023-11-06T21:03:21.723387+0000","last_became_peered":"2023-11-06T21:03:21.723387+0000","last_unstale":"2023-11-06T21:03:21.725102+0000","last_undegraded":"2023-11-06T21:03:21.725102+0000","last_fullsized":"2023-11-06T21:03:21.725102+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691593+0000","last_change":"2023-11-06T21:03:21.220090+0000","last_active":"2023-11-06T21:03:21.691593+0000","last_peered":"2023-11-06T21:03:21.691593+0000","last_clean":"2023-11-06T21:03:21.691593+0000","last_became_active":"2023-11-06T21:03:21.219322+0000","last_became_peered":"2023-11-06T21:03:21.219322+0000","last_unstale":"2023-11-06T21:03:21.691593+0000","last_undegraded":"2023-11-06T21:03:21.691593+0000","last_fullsized":"2023-11-06T21:03:21.691593+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.727400+0000","last_change":"2023-11-06T21:03:21.724962+0000","last_active":"2023-11-06T21:03:21.727400+0000","last_peered":"2023-11-06T21:03:21.727400+0000","last_clean":"2023-11-06T21:03:21.727400+0000","last_became_active":"2023-11-06T21:03:21.720778+0000","last_became_peered":"2023-11-06T21:03:21.720778+0000","last_unstale":"2023-11-06T21:03:21.727400+0000","last_undegraded":"2023-11-06T21:03:21.727400+0000","last_fullsized":"2023-11-06T21:03:21.727400+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693606+0000","last_change":"2023-11-06T21:03:21.216485+0000","last_active":"2023-11-06T21:03:21.693606+0000","last_peered":"2023-11-06T21:03:21.693606+0000","last_clean":"2023-11-06T21:03:21.693606+0000","last_became_active":"2023-11-06T21:03:21.216091+0000","last_became_peered":"2023-11-06T21:03:21.216091+0000","last_unstale":"2023-11-06T21:03:21.693606+0000","last_undegraded":"2023-11-06T21:03:21.693606+0000","last_fullsized":"2023-11-06T21:03:21.693606+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.715402+0000","last_change":"2023-11-06T21:03:21.709618+0000","last_active":"2023-11-06T21:03:21.715402+0000","last_peered":"2023-11-06T21:03:21.715402+0000","last_clean":"2023-11-06T21:03:21.715402+0000","last_became_active":"2023-11-06T21:03:21.709301+0000","last_became_peered":"2023-11-06T21:03:21.709301+0000","last_unstale":"2023-11-06T21:03:21.715402+0000","last_undegraded":"2023-11-06T21:03:21.715402+0000","last_fullsized":"2023-11-06T21:03:21.715402+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.701602+0000","last_change":"2023-11-06T21:03:21.226333+0000","last_active":"2023-11-06T21:03:21.701602+0000","last_peered":"2023-11-06T21:03:21.701602+0000","last_clean":"2023-11-06T21:03:21.701602+0000","last_became_active":"2023-11-06T21:03:21.225339+0000","last_became_peered":"2023-11-06T21:03:21.225339+0000","last_unstale":"2023-11-06T21:03:21.701602+0000","last_undegraded":"2023-11-06T21:03:21.701602+0000","last_fullsized":"2023-11-06T21:03:21.701602+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726226+0000","last_change":"2023-11-06T21:03:21.714640+0000","last_active":"2023-11-06T21:03:21.726226+0000","last_peered":"2023-11-06T21:03:21.726226+0000","last_clean":"2023-11-06T21:03:21.726226+0000","last_became_active":"2023-11-06T21:03:21.714154+0000","last_became_peered":"2023-11-06T21:03:21.714154+0000","last_unstale":"2023-11-06T21:03:21.726226+0000","last_undegraded":"2023-11-06T21:03:21.726226+0000","last_fullsized":"2023-11-06T21:03:21.726226+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693524+0000","last_change":"2023-11-06T21:03:21.226812+0000","last_active":"2023-11-06T21:03:21.693524+0000","last_peered":"2023-11-06T21:03:21.693524+0000","last_clean":"2023-11-06T21:03:21.693524+0000","last_became_active":"2023-11-06T21:03:21.226530+0000","last_became_peered":"2023-11-06T21:03:21.226530+0000","last_unstale":"2023-11-06T21:03:21.693524+0000","last_undegraded":"2023-11-06T21:03:21.693524+0000","last_fullsized":"2023-11-06T21:03:21.693524+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.727516+0000","last_change":"2023-11-06T21:03:21.720525+0000","last_active":"2023-11-06T21:03:21.727516+0000","last_peered":"2023-11-06T21:03:21.727516+0000","last_clean":"2023-11-06T21:03:21.727516+0000","last_became_active":"2023-11-06T21:03:21.720254+0000","last_became_peered":"2023-11-06T21:03:21.720254+0000","last_unstale":"2023-11-06T21:03:21.727516+0000","last_undegraded":"2023-11-06T21:03:21.727516+0000","last_fullsized":"2023-11-06T21:03:21.727516+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.703896+0000","last_change":"2023-11-06T21:03:21.231668+0000","last_active":"2023-11-06T21:03:21.703896+0000","last_peered":"2023-11-06T21:03:21.703896+0000","last_clean":"2023-11-06T21:03:21.703896+0000","last_became_active":"2023-11-06T21:03:21.231323+0000","last_became_peered":"2023-11-06T21:03:21.231323+0000","last_unstale":"2023-11-06T21:03:21.703896+0000","last_undegraded":"2023-11-06T21:03:21.703896+0000","last_fullsized":"2023-11-06T21:03:21.703896+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726205+0000","last_change":"2023-11-06T21:03:21.718742+0000","last_active":"2023-11-06T21:03:21.726205+0000","last_peered":"2023-11-06T21:03:21.726205+0000","last_clean":"2023-11-06T21:03:21.726205+0000","last_became_active":"2023-11-06T21:03:21.718410+0000","last_became_peered":"2023-11-06T21:03:21.718410+0000","last_unstale":"2023-11-06T21:03:21.726205+0000","last_undegraded":"2023-11-06T21:03:21.726205+0000","last_fullsized":"2023-11-06T21:03:21.726205+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.690445+0000","last_change":"2023-11-06T21:03:21.210013+0000","last_active":"2023-11-06T21:03:21.690445+0000","last_peered":"2023-11-06T21:03:21.690445+0000","last_clean":"2023-11-06T21:03:21.690445+0000","last_became_active":"2023-11-06T21:03:21.207953+0000","last_became_peered":"2023-11-06T21:03:21.207953+0000","last_unstale":"2023-11-06T21:03:21.690445+0000","last_undegraded":"2023-11-06T21:03:21.690445+0000","last_fullsized":"2023-11-06T21:03:21.690445+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.711357+0000","last_change":"2023-11-06T21:03:21.700858+0000","last_active":"2023-11-06T21:03:21.711357+0000","last_peered":"2023-11-06T21:03:21.711357+0000","last_clean":"2023-11-06T21:03:21.711357+0000","last_became_active":"2023-11-06T21:03:21.700573+0000","last_became_peered":"2023-11-06T21:03:21.700573+0000","last_unstale":"2023-11-06T21:03:21.711357+0000","last_undegraded":"2023-11-06T21:03:21.711357+0000","last_fullsized":"2023-11-06T21:03:21.711357+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.707738+0000","last_change":"2023-11-06T21:03:21.229372+0000","last_active":"2023-11-06T21:03:21.707738+0000","last_peered":"2023-11-06T21:03:21.707738+0000","last_clean":"2023-11-06T21:03:21.707738+0000","last_became_active":"2023-11-06T21:03:21.226410+0000","last_became_peered":"2023-11-06T21:03:21.226410+0000","last_unstale":"2023-11-06T21:03:21.707738+0000","last_undegraded":"2023-11-06T21:03:21.707738+0000","last_fullsized":"2023-11-06T21:03:21.707738+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716493+0000","last_change":"2023-11-06T21:03:21.711734+0000","last_active":"2023-11-06T21:03:21.716493+0000","last_peered":"2023-11-06T21:03:21.716493+0000","last_clean":"2023-11-06T21:03:21.716493+0000","last_became_active":"2023-11-06T21:03:21.711379+0000","last_became_peered":"2023-11-06T21:03:21.711379+0000","last_unstale":"2023-11-06T21:03:21.716493+0000","last_undegraded":"2023-11-06T21:03:21.716493+0000","last_fullsized":"2023-11-06T21:03:21.716493+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692623+0000","last_change":"2023-11-06T21:03:21.234030+0000","last_active":"2023-11-06T21:03:21.692623+0000","last_peered":"2023-11-06T21:03:21.692623+0000","last_clean":"2023-11-06T21:03:21.692623+0000","last_became_active":"2023-11-06T21:03:21.232752+0000","last_became_peered":"2023-11-06T21:03:21.232752+0000","last_unstale":"2023-11-06T21:03:21.692623+0000","last_undegraded":"2023-11-06T21:03:21.692623+0000","last_fullsized":"2023-11-06T21:03:21.692623+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.715852+0000","last_change":"2023-11-06T21:03:21.713467+0000","last_active":"2023-11-06T21:03:21.715852+0000","last_peered":"2023-11-06T21:03:21.715852+0000","last_clean":"2023-11-06T21:03:21.715852+0000","last_became_active":"2023-11-06T21:03:21.713064+0000","last_became_peered":"2023-11-06T21:03:21.713064+0000","last_unstale":"2023-11-06T21:03:21.715852+0000","last_undegraded":"2023-11-06T21:03:21.715852+0000","last_fullsized":"2023-11-06T21:03:21.715852+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.724200+0000","last_change":"2023-11-06T21:03:21.227032+0000","last_active":"2023-11-06T21:03:21.724200+0000","last_peered":"2023-11-06T21:03:21.724200+0000","last_clean":"2023-11-06T21:03:21.724200+0000","last_became_active":"2023-11-06T21:03:21.226754+0000","last_became_peered":"2023-11-06T21:03:21.226754+0000","last_unstale":"2023-11-06T21:03:21.724200+0000","last_undegraded":"2023-11-06T21:03:21.724200+0000","last_fullsized":"2023-11-06T21:03:21.724200+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.717926+0000","last_change":"2023-11-06T21:03:21.713041+0000","last_active":"2023-11-06T21:03:21.717926+0000","last_peered":"2023-11-06T21:03:21.717926+0000","last_clean":"2023-11-06T21:03:21.717926+0000","last_became_active":"2023-11-06T21:03:21.712543+0000","last_became_peered":"2023-11-06T21:03:21.712543+0000","last_unstale":"2023-11-06T21:03:21.717926+0000","last_undegraded":"2023-11-06T21:03:21.717926+0000","last_fullsized":"2023-11-06T21:03:21.717926+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716844+0000","last_change":"2023-11-06T21:03:21.227797+0000","last_active":"2023-11-06T21:03:21.716844+0000","last_peered":"2023-11-06T21:03:21.716844+0000","last_clean":"2023-11-06T21:03:21.716844+0000","last_became_active":"2023-11-06T21:03:21.227478+0000","last_became_peered":"2023-11-06T21:03:21.227478+0000","last_unstale":"2023-11-06T21:03:21.716844+0000","last_undegraded":"2023-11-06T21:03:21.716844+0000","last_fullsized":"2023-11-06T21:03:21.716844+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726089+0000","last_change":"2023-11-06T21:03:21.714676+0000","last_active":"2023-11-06T21:03:21.726089+0000","last_peered":"2023-11-06T21:03:21.726089+0000","last_clean":"2023-11-06T21:03:21.726089+0000","last_became_active":"2023-11-06T21:03:21.714331+0000","last_became_peered":"2023-11-06T21:03:21.714331+0000","last_unstale":"2023-11-06T21:03:21.726089+0000","last_undegraded":"2023-11-06T21:03:21.726089+0000","last_fullsized":"2023-11-06T21:03:21.726089+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"146'2","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719344+0000","last_change":"2023-11-06T21:03:21.228465+0000","last_active":"2023-11-06T21:03:21.719344+0000","last_peered":"2023-11-06T21:03:21.719344+0000","last_clean":"2023-11-06T21:03:21.719344+0000","last_became_active":"2023-11-06T21:03:21.228175+0000","last_became_peered":"2023-11-06T21:03:21.228175+0000","last_unstale":"2023-11-06T21:03:21.719344+0000","last_undegraded":"2023-11-06T21:03:21.719344+0000","last_fullsized":"2023-11-06T21:03:21.719344+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725788+0000","last_change":"2023-11-06T21:03:21.719655+0000","last_active":"2023-11-06T21:03:21.725788+0000","last_peered":"2023-11-06T21:03:21.725788+0000","last_clean":"2023-11-06T21:03:21.725788+0000","last_became_active":"2023-11-06T21:03:21.718337+0000","last_became_peered":"2023-11-06T21:03:21.718337+0000","last_unstale":"2023-11-06T21:03:21.725788+0000","last_undegraded":"2023-11-06T21:03:21.725788+0000","last_fullsized":"2023-11-06T21:03:21.725788+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693630+0000","last_change":"2023-11-06T21:03:21.224119+0000","last_active":"2023-11-06T21:03:21.693630+0000","last_peered":"2023-11-06T21:03:21.693630+0000","last_clean":"2023-11-06T21:03:21.693630+0000","last_became_active":"2023-11-06T21:03:21.223491+0000","last_became_peered":"2023-11-06T21:03:21.223491+0000","last_unstale":"2023-11-06T21:03:21.693630+0000","last_undegraded":"2023-11-06T21:03:21.693630+0000","last_fullsized":"2023-11-06T21:03:21.693630+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.723886+0000","last_change":"2023-11-06T21:03:21.720812+0000","last_active":"2023-11-06T21:03:21.723886+0000","last_peered":"2023-11-06T21:03:21.723886+0000","last_clean":"2023-11-06T21:03:21.723886+0000","last_became_active":"2023-11-06T21:03:21.720518+0000","last_became_peered":"2023-11-06T21:03:21.720518+0000","last_unstale":"2023-11-06T21:03:21.723886+0000","last_undegraded":"2023-11-06T21:03:21.723886+0000","last_fullsized":"2023-11-06T21:03:21.723886+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691476+0000","last_change":"2023-11-06T21:03:21.221763+0000","last_active":"2023-11-06T21:03:21.691476+0000","last_peered":"2023-11-06T21:03:21.691476+0000","last_clean":"2023-11-06T21:03:21.691476+0000","last_became_active":"2023-11-06T21:03:21.220623+0000","last_became_peered":"2023-11-06T21:03:21.220623+0000","last_unstale":"2023-11-06T21:03:21.691476+0000","last_undegraded":"2023-11-06T21:03:21.691476+0000","last_fullsized":"2023-11-06T21:03:21.691476+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716914+0000","last_change":"2023-11-06T21:03:21.705747+0000","last_active":"2023-11-06T21:03:21.716914+0000","last_peered":"2023-11-06T21:03:21.716914+0000","last_clean":"2023-11-06T21:03:21.716914+0000","last_became_active":"2023-11-06T21:03:21.705461+0000","last_became_peered":"2023-11-06T21:03:21.705461+0000","last_unstale":"2023-11-06T21:03:21.716914+0000","last_undegraded":"2023-11-06T21:03:21.716914+0000","last_fullsized":"2023-11-06T21:03:21.716914+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.700270+0000","last_change":"2023-11-06T21:03:21.232420+0000","last_active":"2023-11-06T21:03:21.700270+0000","last_peered":"2023-11-06T21:03:21.700270+0000","last_clean":"2023-11-06T21:03:21.700270+0000","last_became_active":"2023-11-06T21:03:21.232092+0000","last_became_peered":"2023-11-06T21:03:21.232092+0000","last_unstale":"2023-11-06T21:03:21.700270+0000","last_undegraded":"2023-11-06T21:03:21.700270+0000","last_fullsized":"2023-11-06T21:03:21.700270+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.714380+0000","last_change":"2023-11-06T21:03:21.706554+0000","last_active":"2023-11-06T21:03:21.714380+0000","last_peered":"2023-11-06T21:03:21.714380+0000","last_clean":"2023-11-06T21:03:21.714380+0000","last_became_active":"2023-11-06T21:03:21.706264+0000","last_became_peered":"2023-11-06T21:03:21.706264+0000","last_unstale":"2023-11-06T21:03:21.714380+0000","last_undegraded":"2023-11-06T21:03:21.714380+0000","last_fullsized":"2023-11-06T21:03:21.714380+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.689780+0000","last_change":"2023-11-06T21:03:21.213086+0000","last_active":"2023-11-06T21:03:21.689780+0000","last_peered":"2023-11-06T21:03:21.689780+0000","last_clean":"2023-11-06T21:03:21.689780+0000","last_became_active":"2023-11-06T21:03:21.212240+0000","last_became_peered":"2023-11-06T21:03:21.212240+0000","last_unstale":"2023-11-06T21:03:21.689780+0000","last_undegraded":"2023-11-06T21:03:21.689780+0000","last_fullsized":"2023-11-06T21:03:21.689780+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.712681+0000","last_change":"2023-11-06T21:03:21.706299+0000","last_active":"2023-11-06T21:03:21.712681+0000","last_peered":"2023-11-06T21:03:21.712681+0000","last_clean":"2023-11-06T21:03:21.712681+0000","last_became_active":"2023-11-06T21:03:21.705948+0000","last_became_peered":"2023-11-06T21:03:21.705948+0000","last_unstale":"2023-11-06T21:03:21.712681+0000","last_undegraded":"2023-11-06T21:03:21.712681+0000","last_fullsized":"2023-11-06T21:03:21.712681+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692727+0000","last_change":"2023-11-06T21:03:21.227820+0000","last_active":"2023-11-06T21:03:21.692727+0000","last_peered":"2023-11-06T21:03:21.692727+0000","last_clean":"2023-11-06T21:03:21.692727+0000","last_became_active":"2023-11-06T21:03:21.227529+0000","last_became_peered":"2023-11-06T21:03:21.227529+0000","last_unstale":"2023-11-06T21:03:21.692727+0000","last_undegraded":"2023-11-06T21:03:21.692727+0000","last_fullsized":"2023-11-06T21:03:21.692727+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719482+0000","last_change":"2023-11-06T21:03:21.716144+0000","last_active":"2023-11-06T21:03:21.719482+0000","last_peered":"2023-11-06T21:03:21.719482+0000","last_clean":"2023-11-06T21:03:21.719482+0000","last_became_active":"2023-11-06T21:03:21.714537+0000","last_became_peered":"2023-11-06T21:03:21.714537+0000","last_unstale":"2023-11-06T21:03:21.719482+0000","last_undegraded":"2023-11-06T21:03:21.719482+0000","last_fullsized":"2023-11-06T21:03:21.719482+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.717473+0000","last_change":"2023-11-06T21:03:21.230976+0000","last_active":"2023-11-06T21:03:21.717473+0000","last_peered":"2023-11-06T21:03:21.717473+0000","last_clean":"2023-11-06T21:03:21.717473+0000","last_became_active":"2023-11-06T21:03:21.230425+0000","last_became_peered":"2023-11-06T21:03:21.230425+0000","last_unstale":"2023-11-06T21:03:21.717473+0000","last_undegraded":"2023-11-06T21:03:21.717473+0000","last_fullsized":"2023-11-06T21:03:21.717473+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716434+0000","last_change":"2023-11-06T21:03:21.701884+0000","last_active":"2023-11-06T21:03:21.716434+0000","last_peered":"2023-11-06T21:03:21.716434+0000","last_clean":"2023-11-06T21:03:21.716434+0000","last_became_active":"2023-11-06T21:03:21.701605+0000","last_became_peered":"2023-11-06T21:03:21.701605+0000","last_unstale":"2023-11-06T21:03:21.716434+0000","last_undegraded":"2023-11-06T21:03:21.716434+0000","last_fullsized":"2023-11-06T21:03:21.716434+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"146'2","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719457+0000","last_change":"2023-11-06T21:03:21.222629+0000","last_active":"2023-11-06T21:03:21.719457+0000","last_peered":"2023-11-06T21:03:21.719457+0000","last_clean":"2023-11-06T21:03:21.719457+0000","last_became_active":"2023-11-06T21:03:21.221644+0000","last_became_peered":"2023-11-06T21:03:21.221644+0000","last_unstale":"2023-11-06T21:03:21.719457+0000","last_undegraded":"2023-11-06T21:03:21.719457+0000","last_fullsized":"2023-11-06T21:03:21.719457+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.714216+0000","last_change":"2023-11-06T21:03:21.704282+0000","last_active":"2023-11-06T21:03:21.714216+0000","last_peered":"2023-11-06T21:03:21.714216+0000","last_clean":"2023-11-06T21:03:21.714216+0000","last_became_active":"2023-11-06T21:03:21.703532+0000","last_became_peered":"2023-11-06T21:03:21.703532+0000","last_unstale":"2023-11-06T21:03:21.714216+0000","last_undegraded":"2023-11-06T21:03:21.714216+0000","last_fullsized":"2023-11-06T21:03:21.714216+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.704094+0000","last_change":"2023-11-06T21:03:21.228316+0000","last_active":"2023-11-06T21:03:21.704094+0000","last_peered":"2023-11-06T21:03:21.704094+0000","last_clean":"2023-11-06T21:03:21.704094+0000","last_became_active":"2023-11-06T21:03:21.227981+0000","last_became_peered":"2023-11-06T21:03:21.227981+0000","last_unstale":"2023-11-06T21:03:21.704094+0000","last_undegraded":"2023-11-06T21:03:21.704094+0000","last_fullsized":"2023-11-06T21:03:21.704094+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.723339+0000","last_change":"2023-11-06T21:03:21.717139+0000","last_active":"2023-11-06T21:03:21.723339+0000","last_peered":"2023-11-06T21:03:21.723339+0000","last_clean":"2023-11-06T21:03:21.723339+0000","last_became_active":"2023-11-06T21:03:21.716682+0000","last_became_peered":"2023-11-06T21:03:21.716682+0000","last_unstale":"2023-11-06T21:03:21.723339+0000","last_undegraded":"2023-11-06T21:03:21.723339+0000","last_fullsized":"2023-11-06T21:03:21.723339+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691236+0000","last_change":"2023-11-06T21:03:21.224941+0000","last_active":"2023-11-06T21:03:21.691236+0000","last_peered":"2023-11-06T21:03:21.691236+0000","last_clean":"2023-11-06T21:03:21.691236+0000","last_became_active":"2023-11-06T21:03:21.224578+0000","last_became_peered":"2023-11-06T21:03:21.224578+0000","last_unstale":"2023-11-06T21:03:21.691236+0000","last_undegraded":"2023-11-06T21:03:21.691236+0000","last_fullsized":"2023-11-06T21:03:21.691236+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.710028+0000","last_change":"2023-11-06T21:03:21.699710+0000","last_active":"2023-11-06T21:03:21.710028+0000","last_peered":"2023-11-06T21:03:21.710028+0000","last_clean":"2023-11-06T21:03:21.710028+0000","last_became_active":"2023-11-06T21:03:21.699412+0000","last_became_peered":"2023-11-06T21:03:21.699412+0000","last_unstale":"2023-11-06T21:03:21.710028+0000","last_undegraded":"2023-11-06T21:03:21.710028+0000","last_fullsized":"2023-11-06T21:03:21.710028+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.714961+0000","last_change":"2023-11-06T21:03:21.218741+0000","last_active":"2023-11-06T21:03:21.714961+0000","last_peered":"2023-11-06T21:03:21.714961+0000","last_clean":"2023-11-06T21:03:21.714961+0000","last_became_active":"2023-11-06T21:03:21.218402+0000","last_became_peered":"2023-11-06T21:03:21.218402+0000","last_unstale":"2023-11-06T21:03:21.714961+0000","last_undegraded":"2023-11-06T21:03:21.714961+0000","last_fullsized":"2023-11-06T21:03:21.714961+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719348+0000","last_change":"2023-11-06T21:03:21.717509+0000","last_active":"2023-11-06T21:03:21.719348+0000","last_peered":"2023-11-06T21:03:21.719348+0000","last_clean":"2023-11-06T21:03:21.719348+0000","last_became_active":"2023-11-06T21:03:21.716656+0000","last_became_peered":"2023-11-06T21:03:21.716656+0000","last_unstale":"2023-11-06T21:03:21.719348+0000","last_undegraded":"2023-11-06T21:03:21.719348+0000","last_fullsized":"2023-11-06T21:03:21.719348+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.694959+0000","last_change":"2023-11-06T21:03:21.217738+0000","last_active":"2023-11-06T21:03:21.694959+0000","last_peered":"2023-11-06T21:03:21.694959+0000","last_clean":"2023-11-06T21:03:21.694959+0000","last_became_active":"2023-11-06T21:03:21.217404+0000","last_became_peered":"2023-11-06T21:03:21.217404+0000","last_unstale":"2023-11-06T21:03:21.694959+0000","last_undegraded":"2023-11-06T21:03:21.694959+0000","last_fullsized":"2023-11-06T21:03:21.694959+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.723774+0000","last_change":"2023-11-06T21:03:21.721254+0000","last_active":"2023-11-06T21:03:21.723774+0000","last_peered":"2023-11-06T21:03:21.723774+0000","last_clean":"2023-11-06T21:03:21.723774+0000","last_became_active":"2023-11-06T21:03:21.720884+0000","last_became_peered":"2023-11-06T21:03:21.720884+0000","last_unstale":"2023-11-06T21:03:21.723774+0000","last_undegraded":"2023-11-06T21:03:21.723774+0000","last_fullsized":"2023-11-06T21:03:21.723774+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692724+0000","last_change":"2023-11-06T21:03:21.227649+0000","last_active":"2023-11-06T21:03:21.692724+0000","last_peered":"2023-11-06T21:03:21.692724+0000","last_clean":"2023-11-06T21:03:21.692724+0000","last_became_active":"2023-11-06T21:03:21.227323+0000","last_became_peered":"2023-11-06T21:03:21.227323+0000","last_unstale":"2023-11-06T21:03:21.692724+0000","last_undegraded":"2023-11-06T21:03:21.692724+0000","last_fullsized":"2023-11-06T21:03:21.692724+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719403+0000","last_change":"2023-11-06T21:03:21.702090+0000","last_active":"2023-11-06T21:03:21.719403+0000","last_peered":"2023-11-06T21:03:21.719403+0000","last_clean":"2023-11-06T21:03:21.719403+0000","last_became_active":"2023-11-06T21:03:21.701810+0000","last_became_peered":"2023-11-06T21:03:21.701810+0000","last_unstale":"2023-11-06T21:03:21.719403+0000","last_undegraded":"2023-11-06T21:03:21.719403+0000","last_fullsized":"2023-11-06T21:03:21.719403+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.700129+0000","last_change":"2023-11-06T21:03:21.223351+0000","last_active":"2023-11-06T21:03:21.700129+0000","last_peered":"2023-11-06T21:03:21.700129+0000","last_clean":"2023-11-06T21:03:21.700129+0000","last_became_active":"2023-11-06T21:03:21.222353+0000","last_became_peered":"2023-11-06T21:03:21.222353+0000","last_unstale":"2023-11-06T21:03:21.700129+0000","last_undegraded":"2023-11-06T21:03:21.700129+0000","last_fullsized":"2023-11-06T21:03:21.700129+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.722485+0000","last_change":"2023-11-06T21:03:21.719702+0000","last_active":"2023-11-06T21:03:21.722485+0000","last_peered":"2023-11-06T21:03:21.722485+0000","last_clean":"2023-11-06T21:03:21.722485+0000","last_became_active":"2023-11-06T21:03:21.719501+0000","last_became_peered":"2023-11-06T21:03:21.719501+0000","last_unstale":"2023-11-06T21:03:21.722485+0000","last_undegraded":"2023-11-06T21:03:21.722485+0000","last_fullsized":"2023-11-06T21:03:21.722485+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"146'2","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.718970+0000","last_change":"2023-11-06T21:03:21.215675+0000","last_active":"2023-11-06T21:03:21.718970+0000","last_peered":"2023-11-06T21:03:21.718970+0000","last_clean":"2023-11-06T21:03:21.718970+0000","last_became_active":"2023-11-06T21:03:21.215028+0000","last_became_peered":"2023-11-06T21:03:21.215028+0000","last_unstale":"2023-11-06T21:03:21.718970+0000","last_undegraded":"2023-11-06T21:03:21.718970+0000","last_fullsized":"2023-11-06T21:03:21.718970+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716366+0000","last_change":"2023-11-06T21:03:21.710295+0000","last_active":"2023-11-06T21:03:21.716366+0000","last_peered":"2023-11-06T21:03:21.716366+0000","last_clean":"2023-11-06T21:03:21.716366+0000","last_became_active":"2023-11-06T21:03:21.709915+0000","last_became_peered":"2023-11-06T21:03:21.709915+0000","last_unstale":"2023-11-06T21:03:21.716366+0000","last_undegraded":"2023-11-06T21:03:21.716366+0000","last_fullsized":"2023-11-06T21:03:21.716366+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.706968+0000","last_change":"2023-11-06T21:03:21.233585+0000","last_active":"2023-11-06T21:03:21.706968+0000","last_peered":"2023-11-06T21:03:21.706968+0000","last_clean":"2023-11-06T21:03:21.706968+0000","last_became_active":"2023-11-06T21:03:21.232662+0000","last_became_peered":"2023-11-06T21:03:21.232662+0000","last_unstale":"2023-11-06T21:03:21.706968+0000","last_undegraded":"2023-11-06T21:03:21.706968+0000","last_fullsized":"2023-11-06T21:03:21.706968+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716619+0000","last_change":"2023-11-06T21:03:21.712941+0000","last_active":"2023-11-06T21:03:21.716619+0000","last_peered":"2023-11-06T21:03:21.716619+0000","last_clean":"2023-11-06T21:03:21.716619+0000","last_became_active":"2023-11-06T21:03:21.712683+0000","last_became_peered":"2023-11-06T21:03:21.712683+0000","last_unstale":"2023-11-06T21:03:21.716619+0000","last_undegraded":"2023-11-06T21:03:21.716619+0000","last_fullsized":"2023-11-06T21:03:21.716619+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.704192+0000","last_change":"2023-11-06T21:03:21.230850+0000","last_active":"2023-11-06T21:03:21.704192+0000","last_peered":"2023-11-06T21:03:21.704192+0000","last_clean":"2023-11-06T21:03:21.704192+0000","last_became_active":"2023-11-06T21:03:21.230523+0000","last_became_peered":"2023-11-06T21:03:21.230523+0000","last_unstale":"2023-11-06T21:03:21.704192+0000","last_undegraded":"2023-11-06T21:03:21.704192+0000","last_fullsized":"2023-11-06T21:03:21.704192+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.722298+0000","last_change":"2023-11-06T21:03:21.720665+0000","last_active":"2023-11-06T21:03:21.722298+0000","last_peered":"2023-11-06T21:03:21.722298+0000","last_clean":"2023-11-06T21:03:21.722298+0000","last_became_active":"2023-11-06T21:03:21.720237+0000","last_became_peered":"2023-11-06T21:03:21.720237+0000","last_unstale":"2023-11-06T21:03:21.722298+0000","last_undegraded":"2023-11-06T21:03:21.722298+0000","last_fullsized":"2023-11-06T21:03:21.722298+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.710978+0000","last_change":"2023-11-06T21:03:21.226235+0000","last_active":"2023-11-06T21:03:21.710978+0000","last_peered":"2023-11-06T21:03:21.710978+0000","last_clean":"2023-11-06T21:03:21.710978+0000","last_became_active":"2023-11-06T21:03:21.225785+0000","last_became_peered":"2023-11-06T21:03:21.225785+0000","last_unstale":"2023-11-06T21:03:21.710978+0000","last_undegraded":"2023-11-06T21:03:21.710978+0000","last_fullsized":"2023-11-06T21:03:21.710978+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719727+0000","last_change":"2023-11-06T21:03:21.717883+0000","last_active":"2023-11-06T21:03:21.719727+0000","last_peered":"2023-11-06T21:03:21.719727+0000","last_clean":"2023-11-06T21:03:21.719727+0000","last_became_active":"2023-11-06T21:03:21.717612+0000","last_became_peered":"2023-11-06T21:03:21.717612+0000","last_unstale":"2023-11-06T21:03:21.719727+0000","last_undegraded":"2023-11-06T21:03:21.719727+0000","last_fullsized":"2023-11-06T21:03:21.719727+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.704001+0000","last_change":"2023-11-06T21:03:21.231531+0000","last_active":"2023-11-06T21:03:21.704001+0000","last_peered":"2023-11-06T21:03:21.704001+0000","last_clean":"2023-11-06T21:03:21.704001+0000","last_became_active":"2023-11-06T21:03:21.231250+0000","last_became_peered":"2023-11-06T21:03:21.231250+0000","last_unstale":"2023-11-06T21:03:21.704001+0000","last_undegraded":"2023-11-06T21:03:21.704001+0000","last_fullsized":"2023-11-06T21:03:21.704001+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716772+0000","last_change":"2023-11-06T21:03:21.710700+0000","last_active":"2023-11-06T21:03:21.716772+0000","last_peered":"2023-11-06T21:03:21.716772+0000","last_clean":"2023-11-06T21:03:21.716772+0000","last_became_active":"2023-11-06T21:03:21.709326+0000","last_became_peered":"2023-11-06T21:03:21.709326+0000","last_unstale":"2023-11-06T21:03:21.716772+0000","last_undegraded":"2023-11-06T21:03:21.716772+0000","last_fullsized":"2023-11-06T21:03:21.716772+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.694932+0000","last_change":"2023-11-06T21:03:21.227704+0000","last_active":"2023-11-06T21:03:21.694932+0000","last_peered":"2023-11-06T21:03:21.694932+0000","last_clean":"2023-11-06T21:03:21.694932+0000","last_became_active":"2023-11-06T21:03:21.227197+0000","last_became_peered":"2023-11-06T21:03:21.227197+0000","last_unstale":"2023-11-06T21:03:21.694932+0000","last_undegraded":"2023-11-06T21:03:21.694932+0000","last_fullsized":"2023-11-06T21:03:21.694932+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.717691+0000","last_change":"2023-11-06T21:03:21.715461+0000","last_active":"2023-11-06T21:03:21.717691+0000","last_peered":"2023-11-06T21:03:21.717691+0000","last_clean":"2023-11-06T21:03:21.717691+0000","last_became_active":"2023-11-06T21:03:21.715191+0000","last_became_peered":"2023-11-06T21:03:21.715191+0000","last_unstale":"2023-11-06T21:03:21.717691+0000","last_undegraded":"2023-11-06T21:03:21.717691+0000","last_fullsized":"2023-11-06T21:03:21.717691+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692143+0000","last_change":"2023-11-06T21:03:21.220621+0000","last_active":"2023-11-06T21:03:21.692143+0000","last_peered":"2023-11-06T21:03:21.692143+0000","last_clean":"2023-11-06T21:03:21.692143+0000","last_became_active":"2023-11-06T21:03:21.220330+0000","last_became_peered":"2023-11-06T21:03:21.220330+0000","last_unstale":"2023-11-06T21:03:21.692143+0000","last_undegraded":"2023-11-06T21:03:21.692143+0000","last_fullsized":"2023-11-06T21:03:21.692143+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.717796+0000","last_change":"2023-11-06T21:03:21.714604+0000","last_active":"2023-11-06T21:03:21.717796+0000","last_peered":"2023-11-06T21:03:21.717796+0000","last_clean":"2023-11-06T21:03:21.717796+0000","last_became_active":"2023-11-06T21:03:21.714321+0000","last_became_peered":"2023-11-06T21:03:21.714321+0000","last_unstale":"2023-11-06T21:03:21.717796+0000","last_undegraded":"2023-11-06T21:03:21.717796+0000","last_fullsized":"2023-11-06T21:03:21.717796+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.700214+0000","last_change":"2023-11-06T21:03:21.228240+0000","last_active":"2023-11-06T21:03:21.700214+0000","last_peered":"2023-11-06T21:03:21.700214+0000","last_clean":"2023-11-06T21:03:21.700214+0000","last_became_active":"2023-11-06T21:03:21.227747+0000","last_became_peered":"2023-11-06T21:03:21.227747+0000","last_unstale":"2023-11-06T21:03:21.700214+0000","last_undegraded":"2023-11-06T21:03:21.700214+0000","last_fullsized":"2023-11-06T21:03:21.700214+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.715127+0000","last_change":"2023-11-06T21:03:21.711289+0000","last_active":"2023-11-06T21:03:21.715127+0000","last_peered":"2023-11-06T21:03:21.715127+0000","last_clean":"2023-11-06T21:03:21.715127+0000","last_became_active":"2023-11-06T21:03:21.710992+0000","last_became_peered":"2023-11-06T21:03:21.710992+0000","last_unstale":"2023-11-06T21:03:21.715127+0000","last_undegraded":"2023-11-06T21:03:21.715127+0000","last_fullsized":"2023-11-06T21:03:21.715127+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.701093+0000","last_change":"2023-11-06T21:03:21.229674+0000","last_active":"2023-11-06T21:03:21.701093+0000","last_peered":"2023-11-06T21:03:21.701093+0000","last_clean":"2023-11-06T21:03:21.701093+0000","last_became_active":"2023-11-06T21:03:21.228519+0000","last_became_peered":"2023-11-06T21:03:21.228519+0000","last_unstale":"2023-11-06T21:03:21.701093+0000","last_undegraded":"2023-11-06T21:03:21.701093+0000","last_fullsized":"2023-11-06T21:03:21.701093+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.693796+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T21:03:21.693796+0000","last_peered":"2023-11-06T21:03:21.693796+0000","last_clean":"2023-11-06T21:03:21.693796+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T21:03:21.693796+0000","last_undegraded":"2023-11-06T21:03:21.693796+0000","last_fullsized":"2023-11-06T21:03:21.693796+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.719802+0000","last_change":"2023-11-06T21:03:21.717271+0000","last_active":"2023-11-06T21:03:21.719802+0000","last_peered":"2023-11-06T21:03:21.719802+0000","last_clean":"2023-11-06T21:03:21.719802+0000","last_became_active":"2023-11-06T21:03:21.716945+0000","last_became_peered":"2023-11-06T21:03:21.716945+0000","last_unstale":"2023-11-06T21:03:21.719802+0000","last_undegraded":"2023-11-06T21:03:21.719802+0000","last_fullsized":"2023-11-06T21:03:21.719802+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.700423+0000","last_change":"2023-11-06T21:03:21.225780+0000","last_active":"2023-11-06T21:03:21.700423+0000","last_peered":"2023-11-06T21:03:21.700423+0000","last_clean":"2023-11-06T21:03:21.700423+0000","last_became_active":"2023-11-06T21:03:21.225007+0000","last_became_peered":"2023-11-06T21:03:21.225007+0000","last_unstale":"2023-11-06T21:03:21.700423+0000","last_undegraded":"2023-11-06T21:03:21.700423+0000","last_fullsized":"2023-11-06T21:03:21.700423+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":131,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.690520+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T21:03:21.690520+0000","last_peered":"2023-11-06T21:03:21.690520+0000","last_clean":"2023-11-06T21:03:21.690520+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T21:03:21.690520+0000","last_undegraded":"2023-11-06T21:03:21.690520+0000","last_fullsized":"2023-11-06T21:03:21.690520+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716793+0000","last_change":"2023-11-06T21:03:21.710933+0000","last_active":"2023-11-06T21:03:21.716793+0000","last_peered":"2023-11-06T21:03:21.716793+0000","last_clean":"2023-11-06T21:03:21.716793+0000","last_became_active":"2023-11-06T21:03:21.709677+0000","last_became_peered":"2023-11-06T21:03:21.709677+0000","last_unstale":"2023-11-06T21:03:21.716793+0000","last_undegraded":"2023-11-06T21:03:21.716793+0000","last_fullsized":"2023-11-06T21:03:21.716793+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.694519+0000","last_change":"2023-11-06T21:03:21.235344+0000","last_active":"2023-11-06T21:03:21.694519+0000","last_peered":"2023-11-06T21:03:21.694519+0000","last_clean":"2023-11-06T21:03:21.694519+0000","last_became_active":"2023-11-06T21:03:21.234696+0000","last_became_peered":"2023-11-06T21:03:21.234696+0000","last_unstale":"2023-11-06T21:03:21.694519+0000","last_undegraded":"2023-11-06T21:03:21.694519+0000","last_fullsized":"2023-11-06T21:03:21.694519+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":135,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692869+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T21:03:21.692869+0000","last_peered":"2023-11-06T21:03:21.692869+0000","last_clean":"2023-11-06T21:03:21.692869+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T21:03:21.692869+0000","last_undegraded":"2023-11-06T21:03:21.692869+0000","last_fullsized":"2023-11-06T21:03:21.692869+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.725410+0000","last_change":"2023-11-06T21:03:21.722170+0000","last_active":"2023-11-06T21:03:21.725410+0000","last_peered":"2023-11-06T21:03:21.725410+0000","last_clean":"2023-11-06T21:03:21.725410+0000","last_became_active":"2023-11-06T21:03:21.721420+0000","last_became_peered":"2023-11-06T21:03:21.721420+0000","last_unstale":"2023-11-06T21:03:21.725410+0000","last_undegraded":"2023-11-06T21:03:21.725410+0000","last_fullsized":"2023-11-06T21:03:21.725410+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"146'4","reported_seq":14,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.727620+0000","last_change":"2023-11-06T21:03:21.222049+0000","last_active":"2023-11-06T21:03:21.727620+0000","last_peered":"2023-11-06T21:03:21.727620+0000","last_clean":"2023-11-06T21:03:21.727620+0000","last_became_active":"2023-11-06T21:03:21.221015+0000","last_became_peered":"2023-11-06T21:03:21.221015+0000","last_unstale":"2023-11-06T21:03:21.727620+0000","last_undegraded":"2023-11-06T21:03:21.727620+0000","last_fullsized":"2023-11-06T21:03:21.727620+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":131,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691547+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T21:03:21.691547+0000","last_peered":"2023-11-06T21:03:21.691547+0000","last_clean":"2023-11-06T21:03:21.691547+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T21:03:21.691547+0000","last_undegraded":"2023-11-06T21:03:21.691547+0000","last_fullsized":"2023-11-06T21:03:21.691547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":123,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.692503+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T21:03:21.692503+0000","last_peered":"2023-11-06T21:03:21.692503+0000","last_clean":"2023-11-06T21:03:21.692503+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T21:03:21.692503+0000","last_undegraded":"2023-11-06T21:03:21.692503+0000","last_fullsized":"2023-11-06T21:03:21.692503+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.716684+0000","last_change":"2023-11-06T21:03:21.712901+0000","last_active":"2023-11-06T21:03:21.716684+0000","last_peered":"2023-11-06T21:03:21.716684+0000","last_clean":"2023-11-06T21:03:21.716684+0000","last_became_active":"2023-11-06T21:03:21.711909+0000","last_became_peered":"2023-11-06T21:03:21.711909+0000","last_unstale":"2023-11-06T21:03:21.716684+0000","last_undegraded":"2023-11-06T21:03:21.716684+0000","last_fullsized":"2023-11-06T21:03:21.716684+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"146'1","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.712475+0000","last_change":"2023-11-06T21:03:21.234062+0000","last_active":"2023-11-06T21:03:21.712475+0000","last_peered":"2023-11-06T21:03:21.712475+0000","last_clean":"2023-11-06T21:03:21.712475+0000","last_became_active":"2023-11-06T21:03:21.233776+0000","last_became_peered":"2023-11-06T21:03:21.233776+0000","last_unstale":"2023-11-06T21:03:21.712475+0000","last_undegraded":"2023-11-06T21:03:21.712475+0000","last_fullsized":"2023-11-06T21:03:21.712475+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.689651+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T21:03:21.689651+0000","last_peered":"2023-11-06T21:03:21.689651+0000","last_clean":"2023-11-06T21:03:21.689651+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T21:03:21.689651+0000","last_undegraded":"2023-11-06T21:03:21.689651+0000","last_fullsized":"2023-11-06T21:03:21.689651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.715950+0000","last_change":"2023-11-06T21:03:21.710019+0000","last_active":"2023-11-06T21:03:21.715950+0000","last_peered":"2023-11-06T21:03:21.715950+0000","last_clean":"2023-11-06T21:03:21.715950+0000","last_became_active":"2023-11-06T21:03:21.709753+0000","last_became_peered":"2023-11-06T21:03:21.709753+0000","last_unstale":"2023-11-06T21:03:21.715950+0000","last_undegraded":"2023-11-06T21:03:21.715950+0000","last_fullsized":"2023-11-06T21:03:21.715950+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.698773+0000","last_change":"2023-11-06T21:03:21.221514+0000","last_active":"2023-11-06T21:03:21.698773+0000","last_peered":"2023-11-06T21:03:21.698773+0000","last_clean":"2023-11-06T21:03:21.698773+0000","last_became_active":"2023-11-06T21:03:21.221198+0000","last_became_peered":"2023-11-06T21:03:21.221198+0000","last_unstale":"2023-11-06T21:03:21.698773+0000","last_undegraded":"2023-11-06T21:03:21.698773+0000","last_fullsized":"2023-11-06T21:03:21.698773+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691594+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T21:03:21.691594+0000","last_peered":"2023-11-06T21:03:21.691594+0000","last_clean":"2023-11-06T21:03:21.691594+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T21:03:21.691594+0000","last_undegraded":"2023-11-06T21:03:21.691594+0000","last_fullsized":"2023-11-06T21:03:21.691594+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.714548+0000","last_change":"2023-11-06T21:03:21.707848+0000","last_active":"2023-11-06T21:03:21.714548+0000","last_peered":"2023-11-06T21:03:21.714548+0000","last_clean":"2023-11-06T21:03:21.714548+0000","last_became_active":"2023-11-06T21:03:21.707529+0000","last_became_peered":"2023-11-06T21:03:21.707529+0000","last_unstale":"2023-11-06T21:03:21.714548+0000","last_undegraded":"2023-11-06T21:03:21.714548+0000","last_fullsized":"2023-11-06T21:03:21.714548+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"146'3","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.726466+0000","last_change":"2023-11-06T21:03:21.220969+0000","last_active":"2023-11-06T21:03:21.726466+0000","last_peered":"2023-11-06T21:03:21.726466+0000","last_clean":"2023-11-06T21:03:21.726466+0000","last_became_active":"2023-11-06T21:03:21.220016+0000","last_became_peered":"2023-11-06T21:03:21.220016+0000","last_unstale":"2023-11-06T21:03:21.726466+0000","last_undegraded":"2023-11-06T21:03:21.726466+0000","last_fullsized":"2023-11-06T21:03:21.726466+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":120,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.700281+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T21:03:21.700281+0000","last_peered":"2023-11-06T21:03:21.700281+0000","last_clean":"2023-11-06T21:03:21.700281+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T21:03:21.700281+0000","last_undegraded":"2023-11-06T21:03:21.700281+0000","last_fullsized":"2023-11-06T21:03:21.700281+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.724971+0000","last_change":"2023-11-06T21:03:21.720099+0000","last_active":"2023-11-06T21:03:21.724971+0000","last_peered":"2023-11-06T21:03:21.724971+0000","last_clean":"2023-11-06T21:03:21.724971+0000","last_became_active":"2023-11-06T21:03:21.719863+0000","last_became_peered":"2023-11-06T21:03:21.719863+0000","last_unstale":"2023-11-06T21:03:21.724971+0000","last_undegraded":"2023-11-06T21:03:21.724971+0000","last_fullsized":"2023-11-06T21:03:21.724971+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:21.187108+0000","last_clean_scrub_stamp":"2023-11-06T21:03:21.187108+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"146'2","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.721956+0000","last_change":"2023-11-06T21:03:21.212520+0000","last_active":"2023-11-06T21:03:21.721956+0000","last_peered":"2023-11-06T21:03:21.721956+0000","last_clean":"2023-11-06T21:03:21.721956+0000","last_became_active":"2023-11-06T21:03:21.212147+0000","last_became_peered":"2023-11-06T21:03:21.212147+0000","last_unstale":"2023-11-06T21:03:21.721956+0000","last_undegraded":"2023-11-06T21:03:21.721956+0000","last_fullsized":"2023-11-06T21:03:21.721956+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:03:20.180662+0000","last_clean_scrub_stamp":"2023-11-06T21:03:20.180662+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-11-06T21:03:21.691672+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T21:03:21.691672+0000","last_peered":"2023-11-06T21:03:21.691672+0000","last_clean":"2023-11-06T21:03:21.691672+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T21:03:21.691672+0000","last_undegraded":"2023-11-06T21:03:21.691672+0000","last_fullsized":"2023-11-06T21:03:21.691672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575076,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":926408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.377},"max":{"1min":0.98099999999999998,"5min":1.069,"15min":1.069},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66800000000000004,"15min":0.69499999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35099999999999998},"max":{"1min":0.88500000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.872}]},{"osd":1,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66300000000000003,"15min":0.69299999999999995},"min":{"1min":0.48299999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84199999999999997,"5min":0.90600000000000003,"15min":0.98599999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96299999999999997,"5min":1.0600000000000001,"15min":1.111},"last":0.67000000000000004}]},{"osd":2,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68500000000000005,"15min":0.72099999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.165},"last":0.751},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67800000000000005,"15min":0.71299999999999997},"min":{"1min":0.499,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91200000000000003,"5min":0.97599999999999998,"15min":1.024},"last":0.81999999999999995}]},{"osd":3,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66400000000000003,"15min":0.69499999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.875,"5min":0.89900000000000002,"15min":1.0129999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72299999999999998,"15min":0.748},"min":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0389999999999999,"5min":1.1180000000000001,"15min":1.274},"last":0.85699999999999998}]},{"osd":4,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79300000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.621},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84099999999999997}]},{"osd":5,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86199999999999999,"5min":1.349,"15min":1.349},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64300000000000002,"15min":0.65600000000000003},"min":{"1min":0.47499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":1.399,"15min":1.399},"last":0.71899999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69499999999999995,"15min":0.70799999999999996},"min":{"1min":0.45100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89700000000000002,"5min":1.0680000000000001,"15min":1.1379999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66000000000000003,"15min":0.68300000000000005},"min":{"1min":0.437,"5min":0.433,"15min":0.433},"max":{"1min":0.92900000000000005,"5min":0.96999999999999997,"15min":1.0549999999999999},"last":0.56599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575076,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":926022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66400000000000003,"15min":0.67900000000000005},"min":{"1min":0.41599999999999998,"5min":0.35899999999999999,"15min":0.33000000000000002},"max":{"1min":0.94999999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.68400000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93600000000000005,"5min":0.93700000000000006,"15min":1.216},"last":0.52800000000000002}]},{"osd":1,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66900000000000004,"15min":0.68899999999999995},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.97599999999999998},"last":0.623},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.98599999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.56799999999999995,"5min":0.49299999999999999,"15min":0.47099999999999997},"max":{"1min":0.96499999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.749,"15min":0.76200000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.41499999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.05},"last":0.68600000000000005}]},{"osd":3,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68500000000000005,"15min":0.70799999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.39900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.1459999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1,"5min":1,"15min":1.002},"last":0.81999999999999995}]},{"osd":4,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77800000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80100000000000005,"5min":0.93300000000000005,"15min":1.012},"last":0.74099999999999999}]},{"osd":5,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.35099999999999998},"max":{"1min":0.85699999999999998,"5min":0.93300000000000005,"15min":1.028},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69599999999999995,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.441,"15min":0.38600000000000001},"max":{"1min":0.81100000000000005,"5min":1.04,"15min":1.04},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Nov 6 21:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.68999999999999995,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.378},"max":{"1min":0.878,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72399999999999998,"15min":0.73299999999999998},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89900000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575076,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":925606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46400000000000002,"15min":0.45700000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.25800000000000001},"max":{"1min":0.64200000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.308},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.30199999999999999},"max":{"1min":0.66800000000000004,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.40500000000000003}]},{"osd":2,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63700000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63800000000000001,"15min":0.61499999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82399999999999995,"5min":4.431,"15min":4.431},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.61899999999999999,"15min":0.59199999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79300000000000004,"5min":4.5380000000000003,"15min":4.5380000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.35599999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.47799999999999998}]},{"osd":4,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.77800000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.42599999999999999},"max":{"1min":1.1519999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0720000000000001,"5min":1.883,"15min":1.883},"last":0.85499999999999998}]},{"osd":5,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0449999999999999,"5min":4.399,"15min":4.399},"last":0.754},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.78000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.171,"5min":2.649,"15min":2.649},"last":0.83599999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.81200000000000006,"15min":0.80800000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1240000000000001,"5min":3.024,"15min":3.024},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.52400000000000002},"max":{"1min":1.139,"5min":3.0609999999999999,"15min":3.0609999999999999},"last":0.79500000000000004}]},{"osd":7,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82099999999999995,"15min":0.83699999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51100000000000001},"max":{"1min":1.153,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.85599999999999998,"15min":0.83899999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0880000000000001,"5min":5.8230000000000004,"15min":5.8230000000000004},"last":0.88400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575076,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":926034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.46800000000000003},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.29699999999999999},"max":{"1min":0.58499999999999996,"5min":0.73899999999999999,"15min":0.84299999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34499999999999997,"5min":0.33700000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.46000000000000002}]},{"osd":2,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45200000000000001,"15min":0.46999999999999997},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.623,"5min":0.67600000000000005,"15min":0.82099999999999995},"last":0.436},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52700000000000002,"15min":0.55000000000000004},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.91000000000000003},"last":0.51000000000000001}]},{"osd":3,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.97199999999999998},"last":0.502}]},{"osd":4,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77400000000000002,"15min":0.80600000000000005},"min":{"1min":0.61699999999999999,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.1299999999999999,"15min":1.1779999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.77400000000000002,"15min":0.80500000000000005},"min":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.056,"5min":1.153,"15min":1.298},"last":0.53400000000000003}]},{"osd":5,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78000000000000003,"15min":0.80400000000000005},"min":{"1min":0.59099999999999997,"5min":0.54700000000000004,"15min":0.48999999999999999},"max":{"1min":1.0549999999999999,"5min":1.069,"15min":1.2010000000000001},"last":0.625},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.51400000000000001},"max":{"1min":1.0940000000000001,"5min":1.2010000000000001,"15min":1.2829999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77800000000000002,"15min":0.79300000000000004},"min":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"max":{"1min":0.997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.79300000000000004,"15min":0.81599999999999995},"min":{"1min":0.59599999999999997,"5min":0.52200000000000002,"15min":0.51800000000000002},"max":{"1min":1.042,"5min":1.0629999999999999,"15min":1.304},"last":0.65800000000000003}]},{"osd":7,"last update":"Mon Nov 6 21:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80000000000000004,"15min":0.82199999999999995},"min":{"1min":0.56200000000000006,"5min":0.505,"15min":0.505},"max":{"1min":1.032,"5min":1.171,"15min":1.2490000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.82399999999999995,"15min":0.83899999999999997},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54600000000000004},"max":{"1min":1.083,"5min":1.1899999999999999,"15min":1.236},"last":0.56799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575076,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":926090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.32500000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":1.0920000000000001},"last":0.442},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29499999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.81699999999999995},"last":0.41899999999999998}]},{"osd":1,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.56200000000000006},"min":{"1min":0.40799999999999997,"5min":0.34799999999999998,"15min":0.34599999999999997},"max":{"1min":0.69199999999999995,"5min":0.86099999999999999,"15min":1.6359999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58499999999999996,"15min":0.60399999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.35999999999999999},"max":{"1min":0.83999999999999997,"5min":0.96399999999999997,"15min":1.4490000000000001},"last":0.45200000000000001}]},{"osd":3,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76300000000000001,"5min":0.85899999999999999,"15min":2.266},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79400000000000004,"5min":0.93999999999999995,"15min":1.732},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77700000000000002,"15min":0.78800000000000003},"min":{"1min":0.58999999999999997,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":0.95399999999999996,"5min":1.121,"15min":1.1459999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.77600000000000002},"min":{"1min":0.54300000000000004,"5min":0.46700000000000003,"15min":0.39400000000000002},"max":{"1min":0.96299999999999997,"5min":1.087,"15min":1.357},"last":0.79300000000000004}]},{"osd":5,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.752,"15min":0.77300000000000002},"min":{"1min":0.57899999999999996,"5min":0.51300000000000001,"15min":0.46300000000000002},"max":{"1min":1.052,"5min":1.1619999999999999,"15min":1.657},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0609999999999999,"5min":1.159,"15min":1.159},"last":0.95299999999999996}]},{"osd":6,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78500000000000003,"15min":0.80100000000000005},"min":{"1min":0.51300000000000001,"5min":0.495,"15min":0.46200000000000002},"max":{"1min":0.94399999999999995,"5min":1.1499999999999999,"15min":1.2829999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.81100000000000005},"min":{"1min":0.57099999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.997,"5min":1.099,"15min":1.7330000000000001},"last":1.0700000000000001}]},{"osd":7,"last update":"Mon Nov 6 21:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":0.82499999999999996},"min":{"1min":0.59899999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0189999999999999,"5min":1.1870000000000001,"15min":1.448},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80900000000000005,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.52100000000000002},"max":{"1min":1.044,"5min":1.129,"15min":2.202},"last":0.995}]}]},{"osd":3,"up_from":13,"seq":55834575077,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302976,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068864,"statfs":{"total":96636764160,"available":96326516736,"internally_reserved":0,"allocated":1826816,"data_stored":925536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.44500000000000001,"15min":0.45300000000000001},"min":{"1min":0.28499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69399999999999995,"5min":0.71399999999999997,"15min":0.82599999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34000000000000002,"15min":0.32600000000000001},"max":{"1min":0.70499999999999996,"5min":0.86699999999999999,"15min":0.93400000000000005},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.624,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53300000000000003,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69699999999999995,"5min":0.79900000000000004,"15min":0.80900000000000005},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.55400000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.505},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57999999999999996,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.33800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81000000000000005,"15min":0.81699999999999995},"min":{"1min":0.59199999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95499999999999996,"5min":1.234,"15min":1.2470000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80000000000000004,"15min":0.81299999999999994},"min":{"1min":0.504,"5min":0.504,"15min":0.47999999999999998},"max":{"1min":0.92800000000000005,"5min":1.276,"15min":1.286},"last":0.68000000000000005}]},{"osd":5,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.78400000000000003},"min":{"1min":0.59699999999999998,"5min":0.52200000000000002,"15min":0.496},"max":{"1min":0.96699999999999997,"5min":1.1519999999999999,"15min":1.1779999999999999},"last":0.752},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77700000000000002,"15min":0.79400000000000004},"min":{"1min":0.56100000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.99399999999999999,"5min":1.258,"15min":1.258},"last":0.58399999999999996}]},{"osd":6,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76800000000000002,"15min":0.78400000000000003},"min":{"1min":0.57799999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88900000000000001,"5min":1.323,"15min":1.323},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.79200000000000004,"15min":0.81399999999999995},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.53300000000000003},"max":{"1min":0.89100000000000001,"5min":1.054,"15min":1.2130000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Mon Nov 6 21:02:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79100000000000004,"15min":0.81399999999999995},"min":{"1min":0.621,"5min":0.497,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":1.0780000000000001,"15min":1.302},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.60299999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.93400000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.80100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575076,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302968,"kb_used_data":1776,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068872,"statfs":{"total":96636764160,"available":96326524928,"internally_reserved":0,"allocated":1818624,"data_stored":918798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64500000000000002,"15min":0.68000000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84099999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.40300000000000002},"max":{"1min":0.84699999999999998,"5min":1,"15min":1},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65300000000000002,"15min":0.69199999999999995},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82099999999999995,"5min":1.107,"15min":1.645},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.46300000000000002,"5min":0.45600000000000002,"15min":0.41199999999999998},"max":{"1min":0.877,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.76700000000000002}]},{"osd":2,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67000000000000004,"15min":0.71099999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.46300000000000002},"max":{"1min":0.873,"5min":0.92800000000000005,"15min":1.0569999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.81599999999999995}]},{"osd":3,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65300000000000002,"15min":0.68500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82399999999999995,"5min":0.90300000000000002,"15min":0.95799999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66400000000000003,"15min":0.68799999999999994},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86199999999999999,"5min":0.93200000000000005,"15min":1.1060000000000001},"last":0.745}]},{"osd":5,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80200000000000005,"5min":0.88200000000000001,"15min":1.0389999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.622,"15min":0.59499999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.87,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83199999999999996}]},{"osd":6,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63400000000000001,"15min":0.60699999999999998},"min":{"1min":0.42699999999999999,"5min":0.40899999999999997,"15min":0.376},"max":{"1min":0.84399999999999997,"5min":0.93100000000000005,"15min":1.22},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.66700000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.89700000000000002}]},{"osd":7,"last update":"Mon Nov 6 21:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67900000000000005,"15min":0.65800000000000003},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.44},"max":{"1min":0.90100000000000002,"5min":1.034,"15min":1.034},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70099999999999996,"15min":0.67600000000000005},"min":{"1min":0.501,"5min":0.47299999999999998,"15min":0.45400000000000001},"max":{"1min":0.88,"5min":0.95199999999999996,"15min":1.177},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575077,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1835008,"data_stored":926982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67300000000000004,"15min":0.67200000000000004},"min":{"1min":0.42099999999999999,"5min":0.40100000000000002,"15min":0.27300000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.34999999999999998},"max":{"1min":0.92300000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.874}]},{"osd":1,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70499999999999996,"15min":0.71199999999999997},"min":{"1min":0.52500000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90300000000000002,"5min":0.92300000000000004,"15min":0.99399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.995,"5min":1.026,"15min":1.026},"last":0.85799999999999998}]},{"osd":2,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71199999999999997,"15min":0.73499999999999999},"min":{"1min":0.54600000000000004,"5min":0.41199999999999998,"15min":0.36299999999999999},"max":{"1min":0.95799999999999996,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52900000000000003,"5min":0.46500000000000002,"15min":0.45600000000000002},"max":{"1min":0.96999999999999997,"5min":0.98099999999999998,"15min":1.0089999999999999},"last":0.79300000000000004}]},{"osd":3,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69399999999999995,"15min":0.70399999999999996},"min":{"1min":0.49399999999999999,"5min":0.45100000000000001,"15min":0.377},"max":{"1min":1,"5min":1,"15min":1},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71299999999999997,"15min":0.71399999999999997},"min":{"1min":0.47799999999999998,"5min":0.46000000000000002,"15min":0.437},"max":{"1min":0.98399999999999999,"5min":1.175,"15min":1.175},"last":0.83899999999999997}]},{"osd":4,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0389999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.39400000000000002,"15min":0.35599999999999998},"max":{"1min":1.018,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.92300000000000004}]},{"osd":6,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64600000000000002,"15min":0.627},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.98099999999999998,"5min":1.214,"15min":1.214},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.52600000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0820000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Mon Nov 6 21:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.40600000000000003},"max":{"1min":0.94999999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0589999999999999,"5min":1.258,"15min":1.258},"last":0.89800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T21:03:26.486 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T21:03:26.487 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T21:03:26.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.729+0000 7fdf483ff700 1 Processor -- start 2023-11-06T21:03:26.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.729+0000 7fdf483ff700 1 -- start start 2023-11-06T21:03:26.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.730+0000 7fdf483ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf40154330 0x7fdf400c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.731+0000 7fdf483ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdf400c7ff0 0x7fdf400646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.731+0000 7fdf483ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf400c6b60 con 0x7fdf40154710 2023-11-06T21:03:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.731+0000 7fdf483ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf400c7700 con 0x7fdf400c7ff0 2023-11-06T21:03:26.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.731+0000 7fdf483ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400c5cd0 con 0x7fdf40154330 2023-11-06T21:03:26.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.731+0000 7fdf4599a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdf40154710 0x7fdf400c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36606/0 (socket says 172.21.15.33:36606) 2023-11-06T21:03:26.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.731+0000 7fdf4599a700 1 -- 172.21.15.33:0/3139093977 learned_addr learned my addr 172.21.15.33:0/3139093977 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:26.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.731+0000 7fdf4699c700 1 --2- 172.21.15.33:0/3139093977 >> v2:172.21.15.69:3300/0 conn(0x7fdf400c7ff0 0x7fdf400646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.731+0000 7fdf4619b700 1 --2- 172.21.15.33:0/3139093977 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf40154330 0x7fdf400c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.731+0000 7fdf45199700 1 -- 172.21.15.33:0/3139093977 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253426829 0 0) 0x7fdf400c6b60 con 0x7fdf40154710 2023-11-06T21:03:26.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.731+0000 7fdf45199700 1 -- 172.21.15.33:0/3139093977 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf2c0033f0 con 0x7fdf40154710 2023-11-06T21:03:26.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.732+0000 7fdf4699c700 1 -- 172.21.15.33:0/3139093977 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf40154330 msgr2=0x7fdf400c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:26.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.732+0000 7fdf4699c700 1 --2- 172.21.15.33:0/3139093977 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf40154330 0x7fdf400c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.732+0000 7fdf4699c700 1 -- 172.21.15.33:0/3139093977 >> v1:172.21.15.33:6789/0 conn(0x7fdf40154710 legacy=0x7fdf400c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.732+0000 7fdf4699c700 1 -- 172.21.15.33:0/3139093977 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf400c5f50 con 0x7fdf400c7ff0 2023-11-06T21:03:26.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.732+0000 7fdf4699c700 1 --2- 172.21.15.33:0/3139093977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf400c7ff0 0x7fdf400646c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fdf38009e60 tx=0x7fdf38009780).ready entity=mon.1 client_cookie=4b026ea7ed55481a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:26.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.732+0000 7fdf45199700 1 -- 172.21.15.33:0/3139093977 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800e070 con 0x7fdf400c7ff0 2023-11-06T21:03:26.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.732+0000 7fdf45199700 1 -- 172.21.15.33:0/3139093977 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3800bab0 con 0x7fdf400c7ff0 2023-11-06T21:03:26.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.732+0000 7fdf45199700 1 -- 172.21.15.33:0/3139093977 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800be30 con 0x7fdf400c7ff0 2023-11-06T21:03:26.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.732+0000 7fdf483ff700 1 -- 172.21.15.33:0/3139093977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf400c7ff0 msgr2=0x7fdf400646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.732+0000 7fdf483ff700 1 --2- 172.21.15.33:0/3139093977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf400c7ff0 0x7fdf400646c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fdf38009e60 tx=0x7fdf38009780).stop 2023-11-06T21:03:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.733+0000 7fdf483ff700 1 -- 172.21.15.33:0/3139093977 shutdown_connections 2023-11-06T21:03:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.733+0000 7fdf483ff700 1 --2- 172.21.15.33:0/3139093977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf400c7ff0 0x7fdf400646c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.733+0000 7fdf483ff700 1 --2- 172.21.15.33:0/3139093977 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf40154330 0x7fdf400c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.733+0000 7fdf483ff700 1 -- 172.21.15.33:0/3139093977 >> 172.21.15.33:0/3139093977 conn(0x7fdf400cb910 msgr2=0x7fdf400c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:26.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.733+0000 7fdf483ff700 1 -- 172.21.15.33:0/3139093977 shutdown_connections 2023-11-06T21:03:26.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.733+0000 7fdf483ff700 1 -- 172.21.15.33:0/3139093977 wait complete. 2023-11-06T21:03:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.734+0000 7fdf483ff700 1 Processor -- start 2023-11-06T21:03:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.734+0000 7fdf483ff700 1 -- start start 2023-11-06T21:03:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.735+0000 7fdf483ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf400c7ff0 0x7fdf40159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.735+0000 7fdf483ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf40154330 0x7fdf400b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.735+0000 7fdf483ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40154710 0x7fdf400a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.735+0000 7fdf483ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4015a0e0 con 0x7fdf40154710 2023-11-06T21:03:26.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.735+0000 7fdf483ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf401565d0 con 0x7fdf40154330 2023-11-06T21:03:26.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.735+0000 7fdf483ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400b0c80 con 0x7fdf400c7ff0 2023-11-06T21:03:26.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.735+0000 7fdf4699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40154710 0x7fdf400a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40154710 0x7fdf400a92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50206/0 (socket says 172.21.15.33:50206) 2023-11-06T21:03:26.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4699c700 1 -- 172.21.15.33:0/2387917863 learned_addr learned my addr 172.21.15.33:0/2387917863 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:26.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4599a700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf40154330 0x7fdf400b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4619b700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf400c7ff0 0x7fdf40159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4699c700 1 -- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf400c7ff0 msgr2=0x7fdf40159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4699c700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf400c7ff0 0x7fdf40159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4699c700 1 -- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf40154330 msgr2=0x7fdf400b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4699c700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf40154330 0x7fdf400b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4699c700 1 -- 172.21.15.33:0/2387917863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf38009a20 con 0x7fdf40154710 2023-11-06T21:03:26.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4619b700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf400c7ff0 0x7fdf40159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:26.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf4699c700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40154710 0x7fdf400a92a0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fdf38009460 tx=0x7fdf38009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:26.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf367fc700 1 -- 172.21.15.33:0/2387917863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3801d630 con 0x7fdf40154710 2023-11-06T21:03:26.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf367fc700 1 -- 172.21.15.33:0/2387917863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3801d790 con 0x7fdf40154710 2023-11-06T21:03:26.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf367fc700 1 -- 172.21.15.33:0/2387917863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3801db10 con 0x7fdf40154710 2023-11-06T21:03:26.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.736+0000 7fdf483ff700 1 -- 172.21.15.33:0/2387917863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf40155ce0 con 0x7fdf40154710 2023-11-06T21:03:26.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.737+0000 7fdf483ff700 1 -- 172.21.15.33:0/2387917863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf40156260 con 0x7fdf40154710 2023-11-06T21:03:26.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.738+0000 7fdf483ff700 1 -- 172.21.15.33:0/2387917863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf400b09f0 con 0x7fdf40154710 2023-11-06T21:03:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.738+0000 7fdf367fc700 1 -- 172.21.15.33:0/2387917863 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdf38003040 con 0x7fdf40154710 2023-11-06T21:03:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.739+0000 7fdf367fc700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf2405f240 0x7fdf24061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.739+0000 7fdf367fc700 1 -- 172.21.15.33:0/2387917863 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdf3807c280 con 0x7fdf40154710 2023-11-06T21:03:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.740+0000 7fdf4619b700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf2405f240 0x7fdf24061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.740+0000 7fdf4619b700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf2405f240 0x7fdf24061700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fdf3c009c00 tx=0x7fdf3c008040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.743+0000 7fdf367fc700 1 -- 172.21.15.33:0/2387917863 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdf3804d3d0 con 0x7fdf40154710 2023-11-06T21:03:26.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.990+0000 7fdf483ff700 1 -- 172.21.15.33:0/2387917863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdf400830b0 con 0x7fdf40154710 2023-11-06T21:03:26.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.994+0000 7fdf367fc700 1 -- 172.21.15.33:0/2387917863 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7fdf380509f0 con 0x7fdf40154710 2023-11-06T21:03:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 -- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf2405f240 msgr2=0x7fdf24061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf2405f240 0x7fdf24061700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fdf3c009c00 tx=0x7fdf3c008040).stop 2023-11-06T21:03:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 -- 172.21.15.33:0/2387917863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40154710 msgr2=0x7fdf400a92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:26.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40154710 0x7fdf400a92a0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fdf38009460 tx=0x7fdf38009780).stop 2023-11-06T21:03:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 -- 172.21.15.33:0/2387917863 shutdown_connections 2023-11-06T21:03:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf2405f240 0x7fdf24061700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf400c7ff0 0x7fdf40159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40154710 0x7fdf400a92a0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 --2- 172.21.15.33:0/2387917863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf40154330 0x7fdf400b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 -- 172.21.15.33:0/2387917863 >> 172.21.15.33:0/2387917863 conn(0x7fdf400cb910 msgr2=0x7fdf400646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 -- 172.21.15.33:0/2387917863 shutdown_connections 2023-11-06T21:03:26.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:26.996+0000 7fdf483ff700 1 -- 172.21.15.33:0/2387917863 wait complete. 2023-11-06T21:03:27.000 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T21:03:27.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:03:27.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.211+0000 7f18d77ff700 1 Processor -- start 2023-11-06T21:03:27.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.211+0000 7f18d77ff700 1 -- start start 2023-11-06T21:03:27.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.212+0000 7f18d77ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f18d00c1840 0x7f18d00a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:27.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.212+0000 7f18d77ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18d00c6460 0x7f18d00bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:27.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.212+0000 7f18d77ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d01554a0 con 0x7f18d00aa2d0 2023-11-06T21:03:27.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.212+0000 7f18d77ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d0154960 con 0x7f18d00c1840 2023-11-06T21:03:27.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.212+0000 7f18d77ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d0155930 con 0x7f18d00c6460 2023-11-06T21:03:27.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.213+0000 7f18d559b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f18d00c1840 0x7f18d00a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:27.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.213+0000 7f18d559b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f18d00c1840 0x7f18d00a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50222/0 (socket says 172.21.15.33:50222) 2023-11-06T21:03:27.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.213+0000 7f18d559b700 1 -- 172.21.15.33:0/44170746 learned_addr learned my addr 172.21.15.33:0/44170746 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:27.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.213+0000 7f18d4d9a700 1 --1- 172.21.15.33:0/44170746 >> v1:172.21.15.33:6789/0 conn(0x7f18d00aa2d0 0x7f18d00c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36620/0 (socket says 172.21.15.33:36620) 2023-11-06T21:03:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.213+0000 7f18d5d9c700 1 --2- 172.21.15.33:0/44170746 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18d00c6460 0x7f18d00bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.213+0000 7f18d559b700 1 -- 172.21.15.33:0/44170746 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18d00c6460 msgr2=0x7f18d00bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.213+0000 7f18d559b700 1 --2- 172.21.15.33:0/44170746 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18d00c6460 0x7f18d00bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.213+0000 7f18d559b700 1 -- 172.21.15.33:0/44170746 >> v1:172.21.15.33:6789/0 conn(0x7f18d00aa2d0 legacy=0x7f18d00c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.213+0000 7f18d559b700 1 -- 172.21.15.33:0/44170746 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d0154b70 con 0x7f18d00c1840 2023-11-06T21:03:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.213+0000 7f18d559b700 1 --2- 172.21.15.33:0/44170746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c1840 0x7f18d00a9de0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f18cc006fd0 tx=0x7f18cc0090f0).ready entity=mon.0 client_cookie=e415c99994798ef7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:27.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18c7fff700 1 -- 172.21.15.33:0/44170746 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00b580 con 0x7f18d00c1840 2023-11-06T21:03:27.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18c7fff700 1 -- 172.21.15.33:0/44170746 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc00b6e0 con 0x7f18d00c1840 2023-11-06T21:03:27.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18c7fff700 1 -- 172.21.15.33:0/44170746 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00ba60 con 0x7f18d00c1840 2023-11-06T21:03:27.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18d77ff700 1 -- 172.21.15.33:0/44170746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c1840 msgr2=0x7f18d00a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:27.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18d77ff700 1 --2- 172.21.15.33:0/44170746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c1840 0x7f18d00a9de0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f18cc006fd0 tx=0x7f18cc0090f0).stop 2023-11-06T21:03:27.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18d77ff700 1 -- 172.21.15.33:0/44170746 shutdown_connections 2023-11-06T21:03:27.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18d77ff700 1 --2- 172.21.15.33:0/44170746 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18d00c6460 0x7f18d00bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:27.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18d77ff700 1 --2- 172.21.15.33:0/44170746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c1840 0x7f18d00a9de0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:27.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18d77ff700 1 -- 172.21.15.33:0/44170746 >> 172.21.15.33:0/44170746 conn(0x7f18d005ec80 msgr2=0x7f18d00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:27.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18d77ff700 1 -- 172.21.15.33:0/44170746 shutdown_connections 2023-11-06T21:03:27.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.214+0000 7f18d77ff700 1 -- 172.21.15.33:0/44170746 wait complete. 2023-11-06T21:03:27.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.216+0000 7f18d77ff700 1 Processor -- start 2023-11-06T21:03:27.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.216+0000 7f18d77ff700 1 -- start start 2023-11-06T21:03:27.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d77ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18d00aa2d0 0x7f18d00acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:27.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d77ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18d00c1840 0x7f18d0149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:27.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d77ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c6460 0x7f18d0151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:27.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d77ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00abf10 con 0x7f18d00c6460 2023-11-06T21:03:27.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d77ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00ab300 con 0x7f18d00aa2d0 2023-11-06T21:03:27.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d77ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00ab5e0 con 0x7f18d00c1840 2023-11-06T21:03:27.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d5d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c6460 0x7f18d0151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:27.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d5d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c6460 0x7f18d0151690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50232/0 (socket says 172.21.15.33:50232) 2023-11-06T21:03:27.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d5d9c700 1 -- 172.21.15.33:0/2066903795 learned_addr learned my addr 172.21.15.33:0/2066903795 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:27.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d5d9c700 1 -- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18d00c1840 msgr2=0x7f18d0149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:27.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.217+0000 7f18d5d9c700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18d00c1840 0x7f18d0149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:27.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18d559b700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18d00aa2d0 0x7f18d00acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:27.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18d5d9c700 1 -- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18d00aa2d0 msgr2=0x7f18d00acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:27.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18d5d9c700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18d00aa2d0 0x7f18d00acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:27.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18d5d9c700 1 -- 172.21.15.33:0/2066903795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc009370 con 0x7f18d00c6460 2023-11-06T21:03:27.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18d559b700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18d00aa2d0 0x7f18d00acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:27.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18d5d9c700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c6460 0x7f18d0151690 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f18c8009d70 tx=0x7f18c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:27.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18c57fa700 1 -- 172.21.15.33:0/2066903795 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c800cb50 con 0x7f18d00c6460 2023-11-06T21:03:27.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18c57fa700 1 -- 172.21.15.33:0/2066903795 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c800f040 con 0x7f18d00c6460 2023-11-06T21:03:27.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18c57fa700 1 -- 172.21.15.33:0/2066903795 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8013400 con 0x7f18d00c6460 2023-11-06T21:03:27.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18d77ff700 1 -- 172.21.15.33:0/2066903795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d00ab8c0 con 0x7f18d00c6460 2023-11-06T21:03:27.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.218+0000 7f18d77ff700 1 -- 172.21.15.33:0/2066903795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d00c4bc0 con 0x7f18d00c6460 2023-11-06T21:03:27.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.219+0000 7f18d77ff700 1 -- 172.21.15.33:0/2066903795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d006c6c0 con 0x7f18d00c6460 2023-11-06T21:03:27.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.220+0000 7f18c57fa700 1 -- 172.21.15.33:0/2066903795 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f18c80137b0 con 0x7f18d00c6460 2023-11-06T21:03:27.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.221+0000 7f18c57fa700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18b405f270 0x7f18b4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:27.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.221+0000 7f18c57fa700 1 -- 172.21.15.33:0/2066903795 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f18c807ac50 con 0x7f18d00c6460 2023-11-06T21:03:27.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.222+0000 7f18d559b700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18b405f270 0x7f18b4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:27.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.223+0000 7f18d559b700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18b405f270 0x7f18b4061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f18d014dbb0 tx=0x7f18cc017000).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:27.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.225+0000 7f18c57fa700 1 -- 172.21.15.33:0/2066903795 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f18c804bce0 con 0x7f18d00c6460 2023-11-06T21:03:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.447+0000 7f18d77ff700 1 -- 172.21.15.33:0/2066903795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18d0063f20 con 0x7f18d00c6460 2023-11-06T21:03:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.448+0000 7f18c57fa700 1 -- 172.21.15.33:0/2066903795 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f18c804f300 con 0x7f18d00c6460 2023-11-06T21:03:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:27.449 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1809147689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1809147689},{"type":"v1","addr":"172.21.15.33:6835","nonce":1809147689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1390207299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1390207299},{"type":"v1","addr":"172.21.15.69:6837","nonce":1390207299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3179984136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3179984136},{"type":"v1","addr":"172.21.15.69:6836","nonce":3179984136}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:03:21.683218+0000","modified":"2023-11-06T21:03:24.689443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7720":{"gid":7720,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/4126832799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4126832799},{"type":"v1","addr":"172.21.15.33:6834","nonce":4126832799}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-06T21:03:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 -- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18b405f270 msgr2=0x7f18b4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18b405f270 0x7f18b4061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f18d014dbb0 tx=0x7f18cc017000).stop 2023-11-06T21:03:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 -- 172.21.15.33:0/2066903795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c6460 msgr2=0x7f18d0151690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c6460 0x7f18d0151690 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f18c8009d70 tx=0x7f18c800b040).stop 2023-11-06T21:03:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 -- 172.21.15.33:0/2066903795 shutdown_connections 2023-11-06T21:03:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f18b405f270 0x7f18b4061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18d00c6460 0x7f18d0151690 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18d00aa2d0 0x7f18d00acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 --2- 172.21.15.33:0/2066903795 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18d00c1840 0x7f18d0149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 -- 172.21.15.33:0/2066903795 >> 172.21.15.33:0/2066903795 conn(0x7f18d005ec80 msgr2=0x7f18d00ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 -- 172.21.15.33:0/2066903795 shutdown_connections 2023-11-06T21:03:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:27.450+0000 7f18d77ff700 1 -- 172.21.15.33:0/2066903795 wait complete. 2023-11-06T21:03:27.454 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 60 2023-11-06T21:03:27.871 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T21:03:21.683218+0000', 'modified': '2023-11-06T21:03:24.689443+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7720}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7720': {'gid': 7720, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6834/4126832799', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6832', 'nonce': 4126832799}, {'type': 'v1', 'addr': '172.21.15.33:6834', 'nonce': 4126832799}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T21:03:27.872 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T21:03:27.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T21:03:28.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.071+0000 7f87505ff700 1 Processor -- start 2023-11-06T21:03:28.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.071+0000 7f87505ff700 1 -- start start 2023-11-06T21:03:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.072+0000 7f87505ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8748064cb0 0x7f874815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.073+0000 7f87505ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8748155480 con 0x7f87480c7420 2023-11-06T21:03:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.073+0000 7f87505ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87480c6760 con 0x7f874805eae0 2023-11-06T21:03:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.073+0000 7f87505ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87480aa5f0 con 0x7f8748064cb0 2023-11-06T21:03:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.073+0000 7f874e39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f87480c7420 0x7f87480a9e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37328/0 (socket says 172.21.15.33:37328) 2023-11-06T21:03:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.073+0000 7f874e39b700 1 -- 172.21.15.33:0/2301947217 learned_addr learned my addr 172.21.15.33:0/2301947217 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.073+0000 7f874eb9c700 1 --2- 172.21.15.33:0/2301947217 >> v2:172.21.15.69:3300/0 conn(0x7f8748064cb0 0x7f874815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.073+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073723207 0 0) 0x7f8748155480 con 0x7f87480c7420 2023-11-06T21:03:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.073+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87340033f0 con 0x7f87480c7420 2023-11-06T21:03:28.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523504879 0 0) 0x7f87480c6760 con 0x7f874805eae0 2023-11-06T21:03:28.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8748155480 con 0x7f874805eae0 2023-11-06T21:03:28.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023588011 0 0) 0x7f87340033f0 con 0x7f87480c7420 2023-11-06T21:03:28.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 >> v2:172.21.15.69:3300/0 conn(0x7f8748064cb0 msgr2=0x7f874815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 --2- 172.21.15.33:0/2301947217 >> v2:172.21.15.69:3300/0 conn(0x7f8748064cb0 0x7f874815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 >> v1:172.21.15.69:6789/0 conn(0x7f874805eae0 legacy=0x7f87480645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8748154b50 con 0x7f87480c7420 2023-11-06T21:03:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f8744001470 con 0x7f87480c7420 2023-11-06T21:03:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874eb9c700 1 --2- 172.21.15.33:0/2301947217 >> v2:172.21.15.69:3300/0 conn(0x7f8748064cb0 0x7f874815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 >> v1:172.21.15.33:6789/0 conn(0x7f87480c7420 legacy=0x7f87480a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8734003a70 0x7f8734003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8734004690 0x7f8734004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8734005000 0x7f873400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87480aa5f0 con 0x7f8734003a70 2023-11-06T21:03:28.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873400be80 con 0x7f8734004690 2023-11-06T21:03:28.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873400c040 con 0x7f8734005000 2023-11-06T21:03:28.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874e39b700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8734003a70 0x7f8734003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874db9a700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8734004690 0x7f8734004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874eb9c700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8734005000 0x7f873400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.074+0000 7f874e39b700 1 -- 172.21.15.33:0/2301947217 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8734005000 msgr2=0x7f873400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f874e39b700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8734005000 0x7f873400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f874e39b700 1 -- 172.21.15.33:0/2301947217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8734004690 msgr2=0x7f8734004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f874e39b700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8734004690 0x7f8734004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f874e39b700 1 -- 172.21.15.33:0/2301947217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87340033f0 con 0x7f8734003a70 2023-11-06T21:03:28.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f874eb9c700 1 -- 172.21.15.33:0/2301947217 reap_dead start 2023-11-06T21:03:28.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f874e39b700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8734003a70 0x7f8734003e70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f874400c160 tx=0x7f8744001e10).ready entity=mon.0 client_cookie=beea41d28f68d104 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:28.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874400c9d0 con 0x7f8734003a70 2023-11-06T21:03:28.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874400cb30 con 0x7f8734003a70 2023-11-06T21:03:28.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f874d399700 1 -- 172.21.15.33:0/2301947217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87440070d0 con 0x7f8734003a70 2023-11-06T21:03:28.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f87505ff700 1 -- 172.21.15.33:0/2301947217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8734003a70 msgr2=0x7f8734003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f87505ff700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8734003a70 0x7f8734003e70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f874400c160 tx=0x7f8744001e10).stop 2023-11-06T21:03:28.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f87505ff700 1 -- 172.21.15.33:0/2301947217 shutdown_connections 2023-11-06T21:03:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f87505ff700 1 --2- 172.21.15.33:0/2301947217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8734003a70 0x7f8734003e70 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.075+0000 7f87505ff700 1 -- 172.21.15.33:0/2301947217 >> 172.21.15.33:0/2301947217 conn(0x7f87480c8320 msgr2=0x7f874805e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.076+0000 7f87505ff700 1 -- 172.21.15.33:0/2301947217 shutdown_connections 2023-11-06T21:03:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.076+0000 7f87505ff700 1 -- 172.21.15.33:0/2301947217 wait complete. 2023-11-06T21:03:28.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.077+0000 7f87505ff700 1 Processor -- start 2023-11-06T21:03:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.077+0000 7f87505ff700 1 -- start start 2023-11-06T21:03:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.078+0000 7f87505ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8734003a70 0x7f8748165b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.078+0000 7f87505ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87480c20f0 0x7f8748150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.078+0000 7f87505ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87480a99d0 0x7f8748157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.078+0000 7f87505ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87480c52a0 con 0x7f87480a99d0 2023-11-06T21:03:28.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.078+0000 7f87505ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87480c4a00 con 0x7f87480c20f0 2023-11-06T21:03:28.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.078+0000 7f87505ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87480c4ce0 con 0x7f8734003a70 2023-11-06T21:03:28.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.078+0000 7f874eb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87480a99d0 0x7f8748157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.078+0000 7f874eb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87480a99d0 0x7f8748157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59862/0 (socket says 172.21.15.33:59862) 2023-11-06T21:03:28.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.078+0000 7f874eb9c700 1 -- 172.21.15.33:0/3573627916 learned_addr learned my addr 172.21.15.33:0/3573627916 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:28.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f874db9a700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87480c20f0 0x7f8748150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f874e39b700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8734003a70 0x7f8748165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f874eb9c700 1 -- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8734003a70 msgr2=0x7f8748165b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f874eb9c700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8734003a70 0x7f8748165b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f874eb9c700 1 -- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87480c20f0 msgr2=0x7f8748150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f874eb9c700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87480c20f0 0x7f8748150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f874eb9c700 1 -- 172.21.15.33:0/3573627916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87480c4fc0 con 0x7f87480a99d0 2023-11-06T21:03:28.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f874eb9c700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87480a99d0 0x7f8748157fb0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f8748083d00 tx=0x7f874000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:28.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f873e7fc700 1 -- 172.21.15.33:0/3573627916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874000e070 con 0x7f87480a99d0 2023-11-06T21:03:28.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f873e7fc700 1 -- 172.21.15.33:0/3573627916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8740011c50 con 0x7f87480a99d0 2023-11-06T21:03:28.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f87505ff700 1 -- 172.21.15.33:0/3573627916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8748166070 con 0x7f87480a99d0 2023-11-06T21:03:28.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.079+0000 7f873e7fc700 1 -- 172.21.15.33:0/3573627916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740016450 con 0x7f87480a99d0 2023-11-06T21:03:28.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.080+0000 7f87505ff700 1 -- 172.21.15.33:0/3573627916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87480c4660 con 0x7f87480a99d0 2023-11-06T21:03:28.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.081+0000 7f873e7fc700 1 -- 172.21.15.33:0/3573627916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8740016800 con 0x7f87480a99d0 2023-11-06T21:03:28.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.082+0000 7f873e7fc700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f872c05f270 0x7f872c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.082+0000 7f873e7fc700 1 -- 172.21.15.33:0/3573627916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f874007a910 con 0x7f87480a99d0 2023-11-06T21:03:28.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.083+0000 7f874e39b700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f872c05f270 0x7f872c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.084+0000 7f874e39b700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f872c05f270 0x7f872c061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f8744001e70 tx=0x7f8744015000).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:28.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.084+0000 7f87679a6b80 1 Processor -- start 2023-11-06T21:03:28.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.085+0000 7f87679a6b80 1 -- start start 2023-11-06T21:03:28.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.085+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baee6080 0x55a9baee6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.085+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 0x55a9baee0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baf0e4c0 0x55a9baee13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f87679a6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bae222f0 con 0x55a9baee6080 2023-11-06T21:03:28.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f87679a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bae377e0 con 0x55a9baf2ebe0 2023-11-06T21:03:28.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f87679a6b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9bae37920 con 0x55a9baf0e4c0 2023-11-06T21:03:28.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baee6080 0x55a9baee6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baee6080 0x55a9baee6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59870/0 (socket says 172.21.15.33:59870) 2023-11-06T21:03:28.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874e39b700 1 -- 172.21.15.33:0/3085131797 learned_addr learned my addr 172.21.15.33:0/3085131797 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:28.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874db9a700 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 0x55a9baee0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874e39b700 1 -- 172.21.15.33:0/3085131797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baf0e4c0 msgr2=0x55a9baee13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874eb9c700 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baf0e4c0 0x55a9baee13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874e39b700 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baf0e4c0 0x55a9baee13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874e39b700 1 -- 172.21.15.33:0/3085131797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 msgr2=0x55a9baee0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874e39b700 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 0x55a9baee0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874e39b700 1 -- 172.21.15.33:0/3085131797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9bae30720 con 0x55a9baee6080 2023-11-06T21:03:28.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874db9a700 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 0x55a9baee0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:28.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874eb9c700 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baf0e4c0 0x55a9baee13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:28.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.086+0000 7f874e39b700 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baee6080 0x55a9baee6460 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f874400c750 tx=0x7f874401f000).ready entity=mon.0 client_cookie=342acfb7fcabebff server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:28.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f872b1fe700 1 -- 172.21.15.33:0/3085131797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744028070 con 0x55a9baee6080 2023-11-06T21:03:28.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f872b1fe700 1 -- 172.21.15.33:0/3085131797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8744001430 con 0x55a9baee6080 2023-11-06T21:03:28.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f872b1fe700 1 -- 172.21.15.33:0/3085131797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87440046b0 con 0x55a9baee6080 2023-11-06T21:03:28.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f87679a6b80 1 -- 172.21.15.33:0/3085131797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baee6080 msgr2=0x55a9baee6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f87679a6b80 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baee6080 0x55a9baee6460 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f874400c750 tx=0x7f874401f000).stop 2023-11-06T21:03:28.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f87679a6b80 1 -- 172.21.15.33:0/3085131797 shutdown_connections 2023-11-06T21:03:28.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f87679a6b80 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baf0e4c0 0x55a9baee13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f87679a6b80 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baee6080 0x55a9baee6460 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f87679a6b80 1 --2- 172.21.15.33:0/3085131797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 0x55a9baee0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f87679a6b80 1 -- 172.21.15.33:0/3085131797 >> 172.21.15.33:0/3085131797 conn(0x55a9bad4f940 msgr2=0x55a9baf2f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f87679a6b80 1 -- 172.21.15.33:0/3085131797 shutdown_connections 2023-11-06T21:03:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.087+0000 7f87679a6b80 1 -- 172.21.15.33:0/3085131797 wait complete. 2023-11-06T21:03:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.088+0000 7f87679a6b80 1 -- start start 2023-11-06T21:03:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.088+0000 7f87679a6b80 1 Processor -- start 2023-11-06T21:03:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.089+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 0x55a9baf2e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.089+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf0e4c0 0x55a9baf31f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.089+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 0x55a9baf303e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.089+0000 7f87679a6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bae37ba0 con 0x55a9baf0e4c0 2023-11-06T21:03:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.089+0000 7f87679a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bae02e60 con 0x55a9baf2ebe0 2023-11-06T21:03:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.089+0000 7f87679a6b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9baf3b250 con 0x55a9baee6080 2023-11-06T21:03:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.089+0000 7f87679a6b80 10 client.0 ll_register_callbacks cb 0x55a9badd7ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T21:03:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874db9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf0e4c0 0x55a9baf31f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874db9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf0e4c0 0x55a9baf31f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59882/0 (socket says 172.21.15.33:59882) 2023-11-06T21:03:28.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874db9a700 1 -- 172.21.15.33:0/2474431622 learned_addr learned my addr 172.21.15.33:0/2474431622 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:28.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874eb9c700 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 0x55a9baf303e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874e39b700 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 0x55a9baf2e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874db9a700 1 -- 172.21.15.33:0/2474431622 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 msgr2=0x55a9baf2e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874db9a700 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 0x55a9baf2e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874db9a700 1 -- 172.21.15.33:0/2474431622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 msgr2=0x55a9baf303e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874db9a700 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 0x55a9baf303e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874db9a700 1 -- 172.21.15.33:0/2474431622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87340033f0 con 0x55a9baf0e4c0 2023-11-06T21:03:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874eb9c700 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 0x55a9baf303e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f874db9a700 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf0e4c0 0x55a9baf31f20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f8738002620 tx=0x7f8738009040).ready entity=mon.0 client_cookie=657c730a04d90f3c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f87289f9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T21:03:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f87289f9700 1 -- 172.21.15.33:0/2474431622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738015690 con 0x55a9baf0e4c0 2023-11-06T21:03:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.090+0000 7f87289f9700 1 -- 172.21.15.33:0/2474431622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f87380157f0 con 0x55a9baf0e4c0 2023-11-06T21:03:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87679a6b80 1 -- 172.21.15.33:0/2474431622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9baed9e20 con 0x55a9baf0e4c0 2023-11-06T21:03:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87289f9700 1 -- 172.21.15.33:0/2474431622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8738015bc0 con 0x55a9baf0e4c0 2023-11-06T21:03:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87679a6b80 10 client.8115 fetch_fsmap learned FSMap version 60 2023-11-06T21:03:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87679a6b80 1 -- 172.21.15.33:0/2474431622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55a9bae2bac0 con 0x55a9baf0e4c0 2023-11-06T21:03:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87289f9700 1 -- 172.21.15.33:0/2474431622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a9baed9e20 con 0x55a9baf0e4c0 2023-11-06T21:03:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87289f9700 1 -- 172.21.15.33:0/2474431622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 60) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8738015d20 con 0x55a9baf0e4c0 2023-11-06T21:03:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87679a6b80 10 client.8115 fetch_fsmap finished waiting for FSMap version 60 2023-11-06T21:03:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87679a6b80 10 client.8115 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T21:03:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87679a6b80 20 client.8115 populate_metadata read hostname 'smithi033' 2023-11-06T21:03:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.33:6832/4126832799,v1:172.21.15.33:6834/4126832799] conn(0x55a9baee1770 0x55a9baee1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87679a6b80 4 client.8115 mds_command: new command op to 7720 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T21:03:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f87679a6b80 1 -- 172.21.15.33:0/2474431622 --> [v2:172.21.15.33:6832/4126832799,v1:172.21.15.33:6834/4126832799] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9bac9fef0 con 0x55a9baee1770 2023-11-06T21:03:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.091+0000 7f874e39b700 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.33:6832/4126832799,v1:172.21.15.33:6834/4126832799] conn(0x55a9baee1770 0x55a9baee1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.092+0000 7f874e39b700 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.33:6832/4126832799,v1:172.21.15.33:6834/4126832799] conn(0x55a9baee1770 0x55a9baee1b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=845742dee882e7 server_cookie=a2dcd602155b96d3 in_seq=0 out_seq=0 2023-11-06T21:03:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.092+0000 7f87289f9700 10 client.8115 ms_handle_connect on v2:172.21.15.33:6832/4126832799 2023-11-06T21:03:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87289f9700 1 -- 172.21.15.33:0/2474431622 <== mds.0 v2:172.21.15.33:6832/4126832799 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55a9bac9fef0 con 0x55a9baee1770 2023-11-06T21:03:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87289f9700 10 client.8115 handle_command_reply: tid=0 2023-11-06T21:03:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 client.8115 shutdown 2023-11-06T21:03:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 -- 172.21.15.33:0/2474431622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf0e4c0 msgr2=0x55a9baf31f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf0e4c0 0x55a9baf31f20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f8738002620 tx=0x7f8738009040).stop 2023-11-06T21:03:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 -- 172.21.15.33:0/2474431622 shutdown_connections 2023-11-06T21:03:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.33:6832/4126832799,v1:172.21.15.33:6834/4126832799] conn(0x55a9baee1770 0x55a9baee1b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 0x55a9baf2e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf2ebe0 0x55a9baf303e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87289f9700 0 client.8115 ms_handle_reset on v2:172.21.15.33:6832/4126832799 2023-11-06T21:03:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2474431622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf0e4c0 0x55a9baf31f20 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 -- 172.21.15.33:0/2474431622 >> 172.21.15.33:0/2474431622 conn(0x55a9bad4f940 msgr2=0x55a9baee3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 -- 172.21.15.33:0/2474431622 shutdown_connections 2023-11-06T21:03:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 1 -- 172.21.15.33:0/2474431622 wait complete. 2023-11-06T21:03:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.093+0000 7f87679a6b80 20 client.8115 trim_cache size 0 max 16384 2023-11-06T21:03:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.110+0000 7f87679a6b80 1 Processor -- start 2023-11-06T21:03:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.110+0000 7f87679a6b80 1 -- start start 2023-11-06T21:03:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 0x55a9baee6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf0e4c0 0x55a9baee3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baee40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f87679a6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9baee6f80 con 0x55a9baf2ebe0 2023-11-06T21:03:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f87679a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9baee5bd0 con 0x55a9baf0e4c0 2023-11-06T21:03:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f87679a6b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9baee8190 con 0x55a9baee6080 2023-11-06T21:03:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f874eb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baee40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f874eb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baee40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59898/0 (socket says 172.21.15.33:59898) 2023-11-06T21:03:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f874eb9c700 1 -- 172.21.15.33:0/1506784941 learned_addr learned my addr 172.21.15.33:0/1506784941 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f874e39b700 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 0x55a9baee6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.111+0000 7f874db9a700 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf0e4c0 0x55a9baee3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f874eb9c700 1 -- 172.21.15.33:0/1506784941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 msgr2=0x55a9baee6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f874eb9c700 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 0x55a9baee6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f874eb9c700 1 -- 172.21.15.33:0/1506784941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf0e4c0 msgr2=0x55a9baee3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f874eb9c700 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf0e4c0 0x55a9baee3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f874eb9c700 1 -- 172.21.15.33:0/1506784941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9bae2b810 con 0x55a9baf2ebe0 2023-11-06T21:03:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f874db9a700 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf0e4c0 0x55a9baee3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f874e39b700 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 0x55a9baee6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:28.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f874eb9c700 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baee40b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f8740012010 tx=0x7f8740009780).ready entity=mon.0 client_cookie=b6e714351e253be server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:28.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f872b9ff700 1 -- 172.21.15.33:0/1506784941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740054070 con 0x55a9baf2ebe0 2023-11-06T21:03:28.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f872b9ff700 1 -- 172.21.15.33:0/1506784941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874004b9f0 con 0x55a9baf2ebe0 2023-11-06T21:03:28.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.112+0000 7f872b9ff700 1 -- 172.21.15.33:0/1506784941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874004da50 con 0x55a9baf2ebe0 2023-11-06T21:03:28.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.113+0000 7f87679a6b80 1 -- 172.21.15.33:0/1506784941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 msgr2=0x55a9baee40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.113+0000 7f87679a6b80 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baee40b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f8740012010 tx=0x7f8740009780).stop 2023-11-06T21:03:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.113+0000 7f87679a6b80 1 -- 172.21.15.33:0/1506784941 shutdown_connections 2023-11-06T21:03:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.113+0000 7f87679a6b80 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baee6080 0x55a9baee6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.113+0000 7f87679a6b80 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baee40b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.113+0000 7f87679a6b80 1 --2- 172.21.15.33:0/1506784941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baf0e4c0 0x55a9baee3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.113+0000 7f87679a6b80 1 -- 172.21.15.33:0/1506784941 >> 172.21.15.33:0/1506784941 conn(0x55a9bad4f940 msgr2=0x55a9baf33810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:28.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.113+0000 7f87679a6b80 1 -- 172.21.15.33:0/1506784941 shutdown_connections 2023-11-06T21:03:28.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.113+0000 7f87679a6b80 1 -- 172.21.15.33:0/1506784941 wait complete. 2023-11-06T21:03:28.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.113+0000 7f87679a6b80 1 -- start start 2023-11-06T21:03:28.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.114+0000 7f87679a6b80 1 Processor -- start 2023-11-06T21:03:28.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baee6080 0x55a9baee2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baf0e4c0 0x55a9baf13de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f87679a6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baf142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f87679a6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9baf147c0 con 0x55a9baf2ebe0 2023-11-06T21:03:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f87679a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9baf389b0 con 0x55a9baee6080 2023-11-06T21:03:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f87679a6b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9baf43e70 con 0x55a9baf0e4c0 2023-11-06T21:03:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f87679a6b80 10 client.0 ll_register_callbacks cb 0x55a9badd7ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T21:03:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f874eb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baf142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f874eb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baf142d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59904/0 (socket says 172.21.15.33:59904) 2023-11-06T21:03:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f874eb9c700 1 -- 172.21.15.33:0/2243911082 learned_addr learned my addr 172.21.15.33:0/2243911082 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f874eb9c700 1 -- 172.21.15.33:0/2243911082 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baf0e4c0 msgr2=0x55a9baf13de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f874eb9c700 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baf0e4c0 0x55a9baf13de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f874eb9c700 1 -- 172.21.15.33:0/2243911082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baee6080 msgr2=0x55a9baee2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f874eb9c700 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baee6080 0x55a9baee2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f874eb9c700 1 -- 172.21.15.33:0/2243911082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87480c4660 con 0x55a9baf2ebe0 2023-11-06T21:03:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f874eb9c700 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baf142d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f8740007310 tx=0x7f8740057000).ready entity=mon.0 client_cookie=86d1bd35cc7567cc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.115+0000 7f87289f9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T21:03:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87289f9700 1 -- 172.21.15.33:0/2243911082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874004edb0 con 0x55a9baf2ebe0 2023-11-06T21:03:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87289f9700 1 -- 172.21.15.33:0/2243911082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f874005dc10 con 0x55a9baf2ebe0 2023-11-06T21:03:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87679a6b80 1 -- 172.21.15.33:0/2243911082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8744001430 con 0x55a9baf2ebe0 2023-11-06T21:03:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87289f9700 1 -- 172.21.15.33:0/2243911082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f874004b4a0 con 0x55a9baf2ebe0 2023-11-06T21:03:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87679a6b80 10 client.8121 fetch_fsmap learned FSMap version 60 2023-11-06T21:03:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87679a6b80 1 -- 172.21.15.33:0/2243911082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55a9baca51d0 con 0x55a9baf2ebe0 2023-11-06T21:03:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87289f9700 1 -- 172.21.15.33:0/2243911082 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8744001430 con 0x55a9baf2ebe0 2023-11-06T21:03:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87289f9700 1 -- 172.21.15.33:0/2243911082 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 60) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f874004b600 con 0x55a9baf2ebe0 2023-11-06T21:03:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87679a6b80 10 client.8121 fetch_fsmap finished waiting for FSMap version 60 2023-11-06T21:03:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87679a6b80 10 client.8121 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T21:03:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87679a6b80 20 client.8121 populate_metadata read hostname 'smithi033' 2023-11-06T21:03:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.33:6832/4126832799,v1:172.21.15.33:6834/4126832799] conn(0x55a9baf0c560 0x55a9baf0c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87679a6b80 4 client.8121 mds_command: new command op to 7720 tid=0[{"prefix": "status"}] 2023-11-06T21:03:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f87679a6b80 1 -- 172.21.15.33:0/2243911082 --> [v2:172.21.15.33:6832/4126832799,v1:172.21.15.33:6834/4126832799] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a9bacd5040 con 0x55a9baf0c560 2023-11-06T21:03:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.116+0000 7f874e39b700 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.33:6832/4126832799,v1:172.21.15.33:6834/4126832799] conn(0x55a9baf0c560 0x55a9baf0c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f874e39b700 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.33:6832/4126832799,v1:172.21.15.33:6834/4126832799] conn(0x55a9baf0c560 0x55a9baf0c940 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9c17fb1fe47262e8 server_cookie=370d8cc558f47d5c in_seq=0 out_seq=0 2023-11-06T21:03:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87289f9700 10 client.8121 ms_handle_connect on v2:172.21.15.33:6832/4126832799 2023-11-06T21:03:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87289f9700 1 -- 172.21.15.33:0/2243911082 <== mds.0 v2:172.21.15.33:6832/4126832799 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55a9bacd5040 con 0x55a9baf0c560 2023-11-06T21:03:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87289f9700 10 client.8121 handle_command_reply: tid=0 2023-11-06T21:03:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87679a6b80 1 client.8121 shutdown 2023-11-06T21:03:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87679a6b80 1 -- 172.21.15.33:0/2243911082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 msgr2=0x55a9baf142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baf142d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f8740007310 tx=0x7f8740057000).stop 2023-11-06T21:03:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87679a6b80 1 -- 172.21.15.33:0/2243911082 shutdown_connections 2023-11-06T21:03:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a9baf2ebe0 0x55a9baf142d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9baee6080 0x55a9baee2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.33:6832/4126832799,v1:172.21.15.33:6834/4126832799] conn(0x55a9baf0c560 0x55a9baf0c940 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87679a6b80 1 --2- 172.21.15.33:0/2243911082 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55a9baf0e4c0 0x55a9baf13de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87679a6b80 1 -- 172.21.15.33:0/2243911082 >> 172.21.15.33:0/2243911082 conn(0x55a9bad4f940 msgr2=0x55a9baee3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.117+0000 7f87289f9700 0 client.8121 ms_handle_reset on v2:172.21.15.33:6832/4126832799 2023-11-06T21:03:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87679a6b80 1 -- 172.21.15.33:0/2243911082 shutdown_connections 2023-11-06T21:03:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87679a6b80 1 -- 172.21.15.33:0/2243911082 wait complete. 2023-11-06T21:03:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87679a6b80 20 client.8121 trim_cache size 0 max 16384 2023-11-06T21:03:28.146 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T21:03:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T21:03:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T21:03:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 7720, 2023-11-06T21:03:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T21:03:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T21:03:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T21:03:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.4316713449999998, 2023-11-06T21:03:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 60, 2023-11-06T21:03:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 147, 2023-11-06T21:03:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 147, 2023-11-06T21:03:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 11.637508995999999 2023-11-06T21:03:28.149 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T21:03:28.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 -- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f872c05f270 msgr2=0x7f872c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f872c05f270 0x7f872c061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f8744001e70 tx=0x7f8744015000).stop 2023-11-06T21:03:28.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 -- 172.21.15.33:0/3573627916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87480a99d0 msgr2=0x7f8748157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87480a99d0 0x7f8748157fb0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f8748083d00 tx=0x7f874000b040).stop 2023-11-06T21:03:28.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 -- 172.21.15.33:0/3573627916 shutdown_connections 2023-11-06T21:03:28.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f872c05f270 0x7f872c061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8734003a70 0x7f8748165b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87480a99d0 0x7f8748157fb0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 --2- 172.21.15.33:0/3573627916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87480c20f0 0x7f8748150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 -- 172.21.15.33:0/3573627916 >> 172.21.15.33:0/3573627916 conn(0x7f87480c8320 msgr2=0x7f87480c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:28.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 -- 172.21.15.33:0/3573627916 shutdown_connections 2023-11-06T21:03:28.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.118+0000 7f87505ff700 1 -- 172.21.15.33:0/3573627916 wait complete. 2023-11-06T21:03:28.153 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T21:03:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.317+0000 7fe48bfff700 1 Processor -- start 2023-11-06T21:03:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.317+0000 7fe48bfff700 1 -- start start 2023-11-06T21:03:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.318+0000 7fe48bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4840a9ee0 0x7fe4840c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.318+0000 7fe48bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4840c5f80 con 0x7fe4840c07b0 2023-11-06T21:03:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.318+0000 7fe48bfff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4840aa740 con 0x7fe4841541a0 2023-11-06T21:03:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.318+0000 7fe48bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4840c6800 con 0x7fe4840a9ee0 2023-11-06T21:03:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.319+0000 7fe48959a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4840a9ee0 0x7fe4840c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.319+0000 7fe48959a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4840a9ee0 0x7fe4840c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59910/0 (socket says 172.21.15.33:59910) 2023-11-06T21:03:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.319+0000 7fe48959a700 1 -- 172.21.15.33:0/2291148678 learned_addr learned my addr 172.21.15.33:0/2291148678 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.319+0000 7fe48959a700 1 -- 172.21.15.33:0/2291148678 >> v1:172.21.15.69:6789/0 conn(0x7fe4841541a0 legacy=0x7fe4840c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.319+0000 7fe48959a700 1 -- 172.21.15.33:0/2291148678 >> v1:172.21.15.33:6789/0 conn(0x7fe4840c07b0 legacy=0x7fe4840a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.319+0000 7fe48959a700 1 -- 172.21.15.33:0/2291148678 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4840aa460 con 0x7fe4840a9ee0 2023-11-06T21:03:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.319+0000 7fe48959a700 1 --2- 172.21.15.33:0/2291148678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 0x7fe4840c6c90 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7fe474006fd0 tx=0x7fe4740090f0).ready entity=mon.0 client_cookie=cbcdd8dc3f7ca522 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.319+0000 7fe488d99700 1 -- 172.21.15.33:0/2291148678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47400b600 con 0x7fe4840a9ee0 2023-11-06T21:03:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.319+0000 7fe488d99700 1 -- 172.21.15.33:0/2291148678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47400b760 con 0x7fe4840a9ee0 2023-11-06T21:03:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.319+0000 7fe488d99700 1 -- 172.21.15.33:0/2291148678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47400bae0 con 0x7fe4840a9ee0 2023-11-06T21:03:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.320+0000 7fe48bfff700 1 -- 172.21.15.33:0/2291148678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 msgr2=0x7fe4840c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.320+0000 7fe48bfff700 1 --2- 172.21.15.33:0/2291148678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 0x7fe4840c6c90 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7fe474006fd0 tx=0x7fe4740090f0).stop 2023-11-06T21:03:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.320+0000 7fe48bfff700 1 -- 172.21.15.33:0/2291148678 shutdown_connections 2023-11-06T21:03:28.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.320+0000 7fe48bfff700 1 --2- 172.21.15.33:0/2291148678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 0x7fe4840c6c90 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.320+0000 7fe48bfff700 1 -- 172.21.15.33:0/2291148678 >> 172.21.15.33:0/2291148678 conn(0x7fe4840944e0 msgr2=0x7fe48406a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:28.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.320+0000 7fe48bfff700 1 -- 172.21.15.33:0/2291148678 shutdown_connections 2023-11-06T21:03:28.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.320+0000 7fe48bfff700 1 -- 172.21.15.33:0/2291148678 wait complete. 2023-11-06T21:03:28.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.322+0000 7fe48bfff700 1 Processor -- start 2023-11-06T21:03:28.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.322+0000 7fe48bfff700 1 -- start start 2023-11-06T21:03:28.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe48bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 0x7fe484151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe48bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4840c07b0 0x7fe484150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe48bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4841541a0 0x7fe484164190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe48bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484151400 con 0x7fe4840a9ee0 2023-11-06T21:03:28.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe48bfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484151ee0 con 0x7fe4841541a0 2023-11-06T21:03:28.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe48bfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841521c0 con 0x7fe4840c07b0 2023-11-06T21:03:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe489d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 0x7fe484151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe489d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 0x7fe484151920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59926/0 (socket says 172.21.15.33:59926) 2023-11-06T21:03:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe489d9b700 1 -- 172.21.15.33:0/3623009127 learned_addr learned my addr 172.21.15.33:0/3623009127 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe48a59c700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4841541a0 0x7fe484164190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe48959a700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4840c07b0 0x7fe484150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe489d9b700 1 -- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4840c07b0 msgr2=0x7fe484150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe489d9b700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4840c07b0 0x7fe484150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe489d9b700 1 -- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4841541a0 msgr2=0x7fe484164190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe489d9b700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4841541a0 0x7fe484164190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe489d9b700 1 -- 172.21.15.33:0/3623009127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe474009370 con 0x7fe4840a9ee0 2023-11-06T21:03:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.323+0000 7fe48959a700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4840c07b0 0x7fe484150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:03:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.324+0000 7fe489d9b700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 0x7fe484151920 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7fe48000c440 tx=0x7fe480007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.324+0000 7fe479ffb700 1 -- 172.21.15.33:0/3623009127 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48000ce40 con 0x7fe4840a9ee0 2023-11-06T21:03:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.324+0000 7fe479ffb700 1 -- 172.21.15.33:0/3623009127 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480007020 con 0x7fe4840a9ee0 2023-11-06T21:03:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.324+0000 7fe48bfff700 1 -- 172.21.15.33:0/3623009127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4841524a0 con 0x7fe4840a9ee0 2023-11-06T21:03:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.324+0000 7fe479ffb700 1 -- 172.21.15.33:0/3623009127 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4800073c0 con 0x7fe4840a9ee0 2023-11-06T21:03:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.324+0000 7fe48bfff700 1 -- 172.21.15.33:0/3623009127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe484152a60 con 0x7fe4840a9ee0 2023-11-06T21:03:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.325+0000 7fe48bfff700 1 -- 172.21.15.33:0/3623009127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4840475a0 con 0x7fe4840a9ee0 2023-11-06T21:03:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.326+0000 7fe479ffb700 1 -- 172.21.15.33:0/3623009127 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe480006900 con 0x7fe4840a9ee0 2023-11-06T21:03:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.327+0000 7fe479ffb700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46805f240 0x7fe468061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.327+0000 7fe479ffb700 1 -- 172.21.15.33:0/3623009127 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe4800796e0 con 0x7fe4840a9ee0 2023-11-06T21:03:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.328+0000 7fe48959a700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46805f240 0x7fe468061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.328+0000 7fe48959a700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46805f240 0x7fe468061700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe474009130 tx=0x7fe474017000).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.331+0000 7fe479ffb700 1 -- 172.21.15.33:0/3623009127 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe480047af0 con 0x7fe4840a9ee0 2023-11-06T21:03:28.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.554+0000 7fe48bfff700 1 -- 172.21.15.33:0/3623009127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe4840c61b0 con 0x7fe4840a9ee0 2023-11-06T21:03:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.555+0000 7fe479ffb700 1 -- 172.21.15.33:0/3623009127 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe48004a830 con 0x7fe4840a9ee0 2023-11-06T21:03:28.556 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T21:03:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 -- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46805f240 msgr2=0x7fe468061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46805f240 0x7fe468061700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe474009130 tx=0x7fe474017000).stop 2023-11-06T21:03:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 -- 172.21.15.33:0/3623009127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 msgr2=0x7fe484151920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 0x7fe484151920 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7fe48000c440 tx=0x7fe480007e50).stop 2023-11-06T21:03:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 -- 172.21.15.33:0/3623009127 shutdown_connections 2023-11-06T21:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46805f240 0x7fe468061700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4841541a0 0x7fe484164190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4840a9ee0 0x7fe484151920 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 --2- 172.21.15.33:0/3623009127 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4840c07b0 0x7fe484150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 -- 172.21.15.33:0/3623009127 >> 172.21.15.33:0/3623009127 conn(0x7fe4840944e0 msgr2=0x7fe484064a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 -- 172.21.15.33:0/3623009127 shutdown_connections 2023-11-06T21:03:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:28.557+0000 7fe48bfff700 1 -- 172.21.15.33:0/3623009127 wait complete. 2023-11-06T21:03:28.572 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T21:03:28.573 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T21:03:28.573 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T21:03:28.573 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T21:03:28.574 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T21:03:28.574 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T21:03:28.574 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T21:03:28.575 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T21:03:28.575 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T21:03:28.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:03:28.576 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T21:03:28.624 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T21:03:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T21:03:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T21:03:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:03:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T21:03:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:03:28.626 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:03:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:03:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T21:03:28.626 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:03:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:03:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T21:03:28.627 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T21:03:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:03:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T21:03:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T21:03:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41135sec preferred_lft 41135sec 2023-11-06T21:03:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T21:03:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:03:28.628 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:03:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:03:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T21:03:28.629 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T21:03:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T21:03:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T21:03:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:03:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T21:03:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:03:28.630 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T21:03:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T21:03:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T21:03:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:03:28.631 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T21:03:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T21:03:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T21:03:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:03:28.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:03:28.632 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T21:03:28.703 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T21:03:28.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:03:28.706 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T21:03:28.757 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T21:03:28.758 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T21:03:28.758 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T21:03:28.758 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T21:03:28.758 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T21:03:28.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:03:28.760 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T21:03:28.760 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T21:03:28.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T21:03:28.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T21:03:28.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T21:03:28.762 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T21:03:28.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:03:28.938 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T21:03:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T21:03:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T21:03:28.987 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T21:03:28.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:03:28.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T21:03:29.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:03:29.062 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T21:03:29.111 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T21:03:29.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:03:29.255 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:03:29.255 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T21:03:29.281 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:03:29.282 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T21:03:29.286 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T21:03:29.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:03:29.287 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T21:03:29.360 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T21:03:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.558+0000 7fc083fff700 1 Processor -- start 2023-11-06T21:03:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.558+0000 7fc083fff700 1 -- start start 2023-11-06T21:03:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.559+0000 7fc083fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc07c1373a0 0x7fc07c137780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.559+0000 7fc083fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc07c0c1020 0x7fc07c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.559+0000 7fc083fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc07c0c3770 0x7fc07c069440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.559+0000 7fc083fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c06a470 con 0x7fc07c1373a0 2023-11-06T21:03:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.559+0000 7fc083fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c06a190 con 0x7fc07c0c3770 2023-11-06T21:03:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.559+0000 7fc083fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0bbab0 con 0x7fc07c0c1020 2023-11-06T21:03:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc081d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc07c1373a0 0x7fc07c137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc081d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc07c1373a0 0x7fc07c137780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59934/0 (socket says 172.21.15.33:59934) 2023-11-06T21:03:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc081d9b700 1 -- 172.21.15.33:0/110799738 learned_addr learned my addr 172.21.15.33:0/110799738 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc08159a700 1 --2- 172.21.15.33:0/110799738 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc07c0c1020 0x7fc07c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc08259c700 1 --2- 172.21.15.33:0/110799738 >> v2:172.21.15.69:3300/0 conn(0x7fc07c0c3770 0x7fc07c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc081d9b700 1 -- 172.21.15.33:0/110799738 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc07c0c1020 msgr2=0x7fc07c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc081d9b700 1 --2- 172.21.15.33:0/110799738 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc07c0c1020 0x7fc07c0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc081d9b700 1 -- 172.21.15.33:0/110799738 >> v2:172.21.15.69:3300/0 conn(0x7fc07c0c3770 msgr2=0x7fc07c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc081d9b700 1 --2- 172.21.15.33:0/110799738 >> v2:172.21.15.69:3300/0 conn(0x7fc07c0c3770 0x7fc07c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc081d9b700 1 -- 172.21.15.33:0/110799738 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0bbd90 con 0x7fc07c1373a0 2023-11-06T21:03:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.560+0000 7fc081d9b700 1 --2- 172.21.15.33:0/110799738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 0x7fc07c137780 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7fc07c068340 tx=0x7fc0780090f0).ready entity=mon.0 client_cookie=601f9d8ef6c52188 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc080d99700 1 -- 172.21.15.33:0/110799738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800b600 con 0x7fc07c1373a0 2023-11-06T21:03:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc080d99700 1 -- 172.21.15.33:0/110799738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07800b760 con 0x7fc07c1373a0 2023-11-06T21:03:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc080d99700 1 -- 172.21.15.33:0/110799738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800bb00 con 0x7fc07c1373a0 2023-11-06T21:03:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc083fff700 1 -- 172.21.15.33:0/110799738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 msgr2=0x7fc07c137780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc083fff700 1 --2- 172.21.15.33:0/110799738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 0x7fc07c137780 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7fc07c068340 tx=0x7fc0780090f0).stop 2023-11-06T21:03:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc083fff700 1 -- 172.21.15.33:0/110799738 shutdown_connections 2023-11-06T21:03:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc083fff700 1 --2- 172.21.15.33:0/110799738 >> v2:172.21.15.69:3300/0 conn(0x7fc07c0c3770 0x7fc07c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc083fff700 1 --2- 172.21.15.33:0/110799738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 0x7fc07c137780 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc083fff700 1 --2- 172.21.15.33:0/110799738 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc07c0c1020 0x7fc07c0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc083fff700 1 -- 172.21.15.33:0/110799738 >> 172.21.15.33:0/110799738 conn(0x7fc07c062710 msgr2=0x7fc07c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:29.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc083fff700 1 -- 172.21.15.33:0/110799738 shutdown_connections 2023-11-06T21:03:29.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.561+0000 7fc083fff700 1 -- 172.21.15.33:0/110799738 wait complete. 2023-11-06T21:03:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.563+0000 7fc083fff700 1 Processor -- start 2023-11-06T21:03:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.563+0000 7fc083fff700 1 -- start start 2023-11-06T21:03:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc083fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc07c0c1020 0x7fc07c0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc083fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc07c0c3770 0x7fc07c0a9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc083fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 0x7fc07c0aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc083fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c167820 con 0x7fc07c1373a0 2023-11-06T21:03:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc083fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c167ac0 con 0x7fc07c0c1020 2023-11-06T21:03:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc083fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0a9600 con 0x7fc07c0c3770 2023-11-06T21:03:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc08259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 0x7fc07c0aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc08259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 0x7fc07c0aa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59946/0 (socket says 172.21.15.33:59946) 2023-11-06T21:03:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc08259c700 1 -- 172.21.15.33:0/471167840 learned_addr learned my addr 172.21.15.33:0/471167840 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc081d9b700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc07c0c1020 0x7fc07c0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.564+0000 7fc08159a700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc07c0c3770 0x7fc07c0a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc08259c700 1 -- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc07c0c3770 msgr2=0x7fc07c0a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc08259c700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc07c0c3770 0x7fc07c0a9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc08259c700 1 -- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc07c0c1020 msgr2=0x7fc07c0c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc08259c700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc07c0c1020 0x7fc07c0c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc08259c700 1 -- 172.21.15.33:0/471167840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078009370 con 0x7fc07c1373a0 2023-11-06T21:03:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc08259c700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 0x7fc07c0aa9e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fc074009fa0 tx=0x7fc074009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc071ffb700 1 -- 172.21.15.33:0/471167840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07400ebc0 con 0x7fc07c1373a0 2023-11-06T21:03:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc071ffb700 1 -- 172.21.15.33:0/471167840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07400ed20 con 0x7fc07c1373a0 2023-11-06T21:03:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc083fff700 1 -- 172.21.15.33:0/471167840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c0aaed0 con 0x7fc07c1373a0 2023-11-06T21:03:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc071ffb700 1 -- 172.21.15.33:0/471167840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074014610 con 0x7fc07c1373a0 2023-11-06T21:03:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.565+0000 7fc083fff700 1 -- 172.21.15.33:0/471167840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c0ab480 con 0x7fc07c1373a0 2023-11-06T21:03:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.566+0000 7fc083fff700 1 -- 172.21.15.33:0/471167840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c167c70 con 0x7fc07c1373a0 2023-11-06T21:03:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.567+0000 7fc071ffb700 1 -- 172.21.15.33:0/471167840 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc07401d3b0 con 0x7fc07c1373a0 2023-11-06T21:03:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.568+0000 7fc071ffb700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc06005f240 0x7fc060061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.568+0000 7fc071ffb700 1 -- 172.21.15.33:0/471167840 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc07407a860 con 0x7fc07c1373a0 2023-11-06T21:03:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.568+0000 7fc081d9b700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc06005f240 0x7fc060061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.569+0000 7fc081d9b700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc06005f240 0x7fc060061700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc07800b3c0 tx=0x7fc078017000).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.572+0000 7fc071ffb700 1 -- 172.21.15.33:0/471167840 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc07c167c70 con 0x7fc07c1373a0 2023-11-06T21:03:29.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.783+0000 7fc083fff700 1 -- 172.21.15.33:0/471167840 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc07c167c30 con 0x7fc06005f240 2023-11-06T21:03:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.786+0000 7fc071ffb700 1 -- 172.21.15.33:0/471167840 <== mgr.6692 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc07c167c30 con 0x7fc06005f240 2023-11-06T21:03:29.786 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T21:03:29.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:03:29.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T21:03:29.787 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:03:29.787 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T21:03:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 -- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc06005f240 msgr2=0x7fc060061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc06005f240 0x7fc060061700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc07800b3c0 tx=0x7fc078017000).stop 2023-11-06T21:03:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 -- 172.21.15.33:0/471167840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 msgr2=0x7fc07c0aa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 0x7fc07c0aa9e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fc074009fa0 tx=0x7fc074009900).stop 2023-11-06T21:03:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 -- 172.21.15.33:0/471167840 shutdown_connections 2023-11-06T21:03:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc06005f240 0x7fc060061700 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07c1373a0 0x7fc07c0aa9e0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc07c0c1020 0x7fc07c0c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 --2- 172.21.15.33:0/471167840 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc07c0c3770 0x7fc07c0a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 -- 172.21.15.33:0/471167840 >> 172.21.15.33:0/471167840 conn(0x7fc07c062710 msgr2=0x7fc07c1362c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 -- 172.21.15.33:0/471167840 shutdown_connections 2023-11-06T21:03:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:29.788+0000 7fc083fff700 1 -- 172.21.15.33:0/471167840 wait complete. 2023-11-06T21:03:29.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T21:03:30.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.002+0000 7fbcf4fff700 1 Processor -- start 2023-11-06T21:03:30.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.002+0000 7fbcf4fff700 1 -- start start 2023-11-06T21:03:30.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.003+0000 7fbcf4fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbcf0066620 0x7fbcf00c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.003+0000 7fbcf4fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcf0158780 0x7fbcf00681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.003+0000 7fbcf4fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf00a8330 con 0x7fbcf0066a00 2023-11-06T21:03:30.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.003+0000 7fbcf4fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf00be210 con 0x7fbcf0158780 2023-11-06T21:03:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.003+0000 7fbcf4fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf00bdc50 con 0x7fbcf0066620 2023-11-06T21:03:30.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.004+0000 7fbceed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcf0158780 0x7fbcf00681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.004+0000 7fbceed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcf0158780 0x7fbcf00681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59948/0 (socket says 172.21.15.33:59948) 2023-11-06T21:03:30.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.004+0000 7fbceed9d700 1 -- 172.21.15.33:0/2510990044 learned_addr learned my addr 172.21.15.33:0/2510990044 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:30.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.004+0000 7fbcee59c700 1 --2- 172.21.15.33:0/2510990044 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbcf0066620 0x7fbcf00c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.004+0000 7fbceed9d700 1 -- 172.21.15.33:0/2510990044 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbcf0066620 msgr2=0x7fbcf00c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:30.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.004+0000 7fbceed9d700 1 --2- 172.21.15.33:0/2510990044 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbcf0066620 0x7fbcf00c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.004+0000 7fbceed9d700 1 -- 172.21.15.33:0/2510990044 >> v1:172.21.15.33:6789/0 conn(0x7fbcf0066a00 legacy=0x7fbcf0067ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:30.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.004+0000 7fbceed9d700 1 -- 172.21.15.33:0/2510990044 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf00bdf30 con 0x7fbcf0158780 2023-11-06T21:03:30.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.004+0000 7fbceed9d700 1 --2- 172.21.15.33:0/2510990044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 0x7fbcf00681e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7fbce8009f20 tx=0x7fbce8009840).ready entity=mon.0 client_cookie=75ad0a0d102b4fb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:30.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.004+0000 7fbced59a700 1 -- 172.21.15.33:0/2510990044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce800e070 con 0x7fbcf0158780 2023-11-06T21:03:30.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.005+0000 7fbced59a700 1 -- 172.21.15.33:0/2510990044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce800b9d0 con 0x7fbcf0158780 2023-11-06T21:03:30.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.005+0000 7fbced59a700 1 -- 172.21.15.33:0/2510990044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce800bd50 con 0x7fbcf0158780 2023-11-06T21:03:30.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.005+0000 7fbcf4fff700 1 -- 172.21.15.33:0/2510990044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 msgr2=0x7fbcf00681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:30.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.005+0000 7fbcf4fff700 1 --2- 172.21.15.33:0/2510990044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 0x7fbcf00681e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7fbce8009f20 tx=0x7fbce8009840).stop 2023-11-06T21:03:30.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.005+0000 7fbcf4fff700 1 -- 172.21.15.33:0/2510990044 shutdown_connections 2023-11-06T21:03:30.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.005+0000 7fbcf4fff700 1 --2- 172.21.15.33:0/2510990044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 0x7fbcf00681e0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.005+0000 7fbcf4fff700 1 --2- 172.21.15.33:0/2510990044 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbcf0066620 0x7fbcf00c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.005+0000 7fbcf4fff700 1 -- 172.21.15.33:0/2510990044 >> 172.21.15.33:0/2510990044 conn(0x7fbcf00c8c50 msgr2=0x7fbcf00c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.005+0000 7fbcf4fff700 1 -- 172.21.15.33:0/2510990044 shutdown_connections 2023-11-06T21:03:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.005+0000 7fbcf4fff700 1 -- 172.21.15.33:0/2510990044 wait complete. 2023-11-06T21:03:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.007+0000 7fbcf4fff700 1 Processor -- start 2023-11-06T21:03:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.007+0000 7fbcf4fff700 1 -- start start 2023-11-06T21:03:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbcf4fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbcf0066620 0x7fbcf00ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbcf4fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcf0066a00 0x7fbcf00a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbcf4fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 0x7fbcf00ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbcf4fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf00a74f0 con 0x7fbcf0158780 2023-11-06T21:03:30.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbcf4fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf00a87d0 con 0x7fbcf0066a00 2023-11-06T21:03:30.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbcf4fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf00a8a40 con 0x7fbcf0066620 2023-11-06T21:03:30.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbceed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 0x7fbcf00ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbceed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 0x7fbcf00ac0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59950/0 (socket says 172.21.15.33:59950) 2023-11-06T21:03:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbceed9d700 1 -- 172.21.15.33:0/3536188631 learned_addr learned my addr 172.21.15.33:0/3536188631 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbcedd9b700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcf0066a00 0x7fbcf00a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbcee59c700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbcf0066620 0x7fbcf00ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbceed9d700 1 -- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbcf0066620 msgr2=0x7fbcf00ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbceed9d700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbcf0066620 0x7fbcf00ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbceed9d700 1 -- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcf0066a00 msgr2=0x7fbcf00a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbceed9d700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcf0066a00 0x7fbcf00a7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.008+0000 7fbceed9d700 1 -- 172.21.15.33:0/3536188631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce8009ae0 con 0x7fbcf0158780 2023-11-06T21:03:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.009+0000 7fbceed9d700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 0x7fbcf00ac0a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7fbce800b610 tx=0x7fbce8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.009+0000 7fbcde7fc700 1 -- 172.21.15.33:0/3536188631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce800bd50 con 0x7fbcf0158780 2023-11-06T21:03:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.009+0000 7fbcde7fc700 1 -- 172.21.15.33:0/3536188631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce80092e0 con 0x7fbcf0158780 2023-11-06T21:03:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.009+0000 7fbcde7fc700 1 -- 172.21.15.33:0/3536188631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce801d6b0 con 0x7fbcf0158780 2023-11-06T21:03:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.009+0000 7fbcf4fff700 1 -- 172.21.15.33:0/3536188631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf00a8d20 con 0x7fbcf0158780 2023-11-06T21:03:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.009+0000 7fbcf4fff700 1 -- 172.21.15.33:0/3536188631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf00a7730 con 0x7fbcf0158780 2023-11-06T21:03:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.010+0000 7fbcf4fff700 1 -- 172.21.15.33:0/3536188631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf00a9240 con 0x7fbcf0158780 2023-11-06T21:03:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.011+0000 7fbcde7fc700 1 -- 172.21.15.33:0/3536188631 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 55) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbce8003040 con 0x7fbcf0158780 2023-11-06T21:03:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.012+0000 7fbcde7fc700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbcd005f270 0x7fbcd0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.012+0000 7fbcde7fc700 1 -- 172.21.15.33:0/3536188631 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbce807b800 con 0x7fbcf0158780 2023-11-06T21:03:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.012+0000 7fbcee59c700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbcd005f270 0x7fbcd0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.013+0000 7fbcee59c700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbcd005f270 0x7fbcd0061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fbce0009bc0 tx=0x7fbce0008040).ready entity=mgr.6692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.016+0000 7fbcde7fc700 1 -- 172.21.15.33:0/3536188631 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbce804c950 con 0x7fbcf0158780 2023-11-06T21:03:30.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.246+0000 7fbcf4fff700 1 -- 172.21.15.33:0/3536188631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fbcf005f7a0 con 0x7fbcf0158780 2023-11-06T21:03:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.719+0000 7fbcde7fc700 1 -- 172.21.15.33:0/3536188631 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 56) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fbce8079450 con 0x7fbcf0158780 2023-11-06T21:03:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.720+0000 7fbcde7fc700 1 -- 172.21.15.33:0/3536188631 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v56) v1 ==== 92+0+0 (secure 0 0 0) 0x7fbce8042380 con 0x7fbcf0158780 2023-11-06T21:03:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 -- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbcd005f270 msgr2=0x7fbcd0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbcd005f270 0x7fbcd0061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fbce0009bc0 tx=0x7fbce0008040).stop 2023-11-06T21:03:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 -- 172.21.15.33:0/3536188631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 msgr2=0x7fbcf00ac0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 0x7fbcf00ac0a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7fbce800b610 tx=0x7fbce8009840).stop 2023-11-06T21:03:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 -- 172.21.15.33:0/3536188631 shutdown_connections 2023-11-06T21:03:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbcd005f270 0x7fbcd0061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbcf0066620 0x7fbcf00ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf0158780 0x7fbcf00ac0a0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 --2- 172.21.15.33:0/3536188631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcf0066a00 0x7fbcf00a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 -- 172.21.15.33:0/3536188631 >> 172.21.15.33:0/3536188631 conn(0x7fbcf00c8c50 msgr2=0x7fbcf00c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 -- 172.21.15.33:0/3536188631 shutdown_connections 2023-11-06T21:03:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.723+0000 7fbcf4fff700 1 -- 172.21.15.33:0/3536188631 wait complete. 2023-11-06T21:03:30.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T21:03:30.937 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:30.937+0000 7f6891b19680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:03:30.951 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:30.951+0000 7f694102a680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:03:30.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.953+0000 7feeb37ff700 1 Processor -- start 2023-11-06T21:03:30.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.953+0000 7feeb37ff700 1 -- start start 2023-11-06T21:03:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb37ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7feeac1544f0 0x7feeac0c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb37ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feeac0c0c40 0x7feeac0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb37ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feeac064bb0 0x7feeac157f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb37ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac1548d0 con 0x7feeac0c0c40 2023-11-06T21:03:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb37ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac1556d0 con 0x7feeac1544f0 2023-11-06T21:03:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb37ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac065010 con 0x7feeac064bb0 2023-11-06T21:03:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb0d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feeac0c0c40 0x7feeac0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb0d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feeac0c0c40 0x7feeac0646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59980/0 (socket says 172.21.15.33:59980) 2023-11-06T21:03:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb0d9a700 1 -- 172.21.15.33:0/1273838948 learned_addr learned my addr 172.21.15.33:0/1273838948 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb159b700 1 --2- 172.21.15.33:0/1273838948 >> v2:172.21.15.69:3300/0 conn(0x7feeac1544f0 0x7feeac0c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb1d9c700 1 --2- 172.21.15.33:0/1273838948 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feeac064bb0 0x7feeac157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb0d9a700 1 -- 172.21.15.33:0/1273838948 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feeac064bb0 msgr2=0x7feeac157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb0d9a700 1 --2- 172.21.15.33:0/1273838948 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feeac064bb0 0x7feeac157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb0d9a700 1 -- 172.21.15.33:0/1273838948 >> v2:172.21.15.69:3300/0 conn(0x7feeac1544f0 msgr2=0x7feeac0c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb0d9a700 1 --2- 172.21.15.33:0/1273838948 >> v2:172.21.15.69:3300/0 conn(0x7feeac1544f0 0x7feeac0c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.955+0000 7feeb0d9a700 1 -- 172.21.15.33:0/1273838948 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac154ac0 con 0x7feeac0c0c40 2023-11-06T21:03:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.956+0000 7feeb0d9a700 1 --2- 172.21.15.33:0/1273838948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeac0c0c40 0x7feeac0646c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7feeac084660 tx=0x7fee9c0090f0).ready entity=mon.0 client_cookie=fee903f7ec66e54c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.956+0000 7feea3fff700 1 -- 172.21.15.33:0/1273838948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00f510 con 0x7feeac0c0c40 2023-11-06T21:03:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.956+0000 7feea3fff700 1 -- 172.21.15.33:0/1273838948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c00f670 con 0x7feeac0c0c40 2023-11-06T21:03:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.956+0000 7feea3fff700 1 -- 172.21.15.33:0/1273838948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00f9f0 con 0x7feeac0c0c40 2023-11-06T21:03:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.956+0000 7feeb37ff700 1 -- 172.21.15.33:0/1273838948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeac0c0c40 msgr2=0x7feeac0646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.956+0000 7feeb37ff700 1 --2- 172.21.15.33:0/1273838948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeac0c0c40 0x7feeac0646c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7feeac084660 tx=0x7fee9c0090f0).stop 2023-11-06T21:03:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.957+0000 7feeb37ff700 1 -- 172.21.15.33:0/1273838948 shutdown_connections 2023-11-06T21:03:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.957+0000 7feeb37ff700 1 --2- 172.21.15.33:0/1273838948 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feeac064bb0 0x7feeac157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.957+0000 7feeb37ff700 1 --2- 172.21.15.33:0/1273838948 >> v2:172.21.15.69:3300/0 conn(0x7feeac1544f0 0x7feeac0c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.957+0000 7feeb37ff700 1 --2- 172.21.15.33:0/1273838948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeac0c0c40 0x7feeac0646c0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.957+0000 7feeb37ff700 1 -- 172.21.15.33:0/1273838948 >> 172.21.15.33:0/1273838948 conn(0x7feeac0cb900 msgr2=0x7feeac0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:30.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.957+0000 7feeb37ff700 1 -- 172.21.15.33:0/1273838948 shutdown_connections 2023-11-06T21:03:30.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.957+0000 7feeb37ff700 1 -- 172.21.15.33:0/1273838948 wait complete. 2023-11-06T21:03:30.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.958+0000 7feeb37ff700 1 Processor -- start 2023-11-06T21:03:30.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.958+0000 7feeb37ff700 1 -- start start 2023-11-06T21:03:30.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.959+0000 7feeb37ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feeac064bb0 0x7feeac0aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.959+0000 7feeb37ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feeac0c0c40 0x7feeac0afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.959+0000 7feeb37ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeac1544f0 0x7feeac0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.959+0000 7feeb37ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0c3ab0 con 0x7feeac1544f0 2023-11-06T21:03:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.959+0000 7feeb37ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0c2bc0 con 0x7feeac064bb0 2023-11-06T21:03:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.959+0000 7feeb37ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0c2ea0 con 0x7feeac0c0c40 2023-11-06T21:03:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb1d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeac1544f0 0x7feeac0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb159b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feeac064bb0 0x7feeac0aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb159b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feeac064bb0 0x7feeac0aa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:59780/0 (socket says 172.21.15.33:59780) 2023-11-06T21:03:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb159b700 1 -- 172.21.15.33:0/321365929 learned_addr learned my addr 172.21.15.33:0/321365929 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb0d9a700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feeac0c0c40 0x7feeac0afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb1d9c700 1 -- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feeac0c0c40 msgr2=0x7feeac0afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb1d9c700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feeac0c0c40 0x7feeac0afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb1d9c700 1 -- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feeac064bb0 msgr2=0x7feeac0aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb1d9c700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feeac064bb0 0x7feeac0aa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb1d9c700 1 -- 172.21.15.33:0/321365929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c009370 con 0x7feeac1544f0 2023-11-06T21:03:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb159b700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feeac064bb0 0x7feeac0aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:03:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feeb1d9c700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeac1544f0 0x7feeac0b03e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7feea4009e50 tx=0x7feea400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.960+0000 7feea17fa700 1 -- 172.21.15.33:0/321365929 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400caa0 con 0x7feeac1544f0 2023-11-06T21:03:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.961+0000 7feea17fa700 1 -- 172.21.15.33:0/321365929 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400f040 con 0x7feeac1544f0 2023-11-06T21:03:30.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.961+0000 7feea17fa700 1 -- 172.21.15.33:0/321365929 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400ce10 con 0x7feeac1544f0 2023-11-06T21:03:30.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.961+0000 7feeb37ff700 1 -- 172.21.15.33:0/321365929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac0c3180 con 0x7feeac1544f0 2023-11-06T21:03:30.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.961+0000 7feeb37ff700 1 -- 172.21.15.33:0/321365929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac0c3740 con 0x7feeac1544f0 2023-11-06T21:03:30.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.962+0000 7feeb37ff700 1 -- 172.21.15.33:0/321365929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac0c8390 con 0x7feeac1544f0 2023-11-06T21:03:30.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.963+0000 7feea17fa700 1 -- 172.21.15.33:0/321365929 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 56) v1 ==== 75565+0+0 (secure 0 0 0) 0x7feea4013610 con 0x7feeac1544f0 2023-11-06T21:03:30.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.964+0000 7feea17fa700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fee9005f290 0x7fee90061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.964+0000 7feea17fa700 1 -- 172.21.15.33:0/321365929 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feea407ac50 con 0x7feeac1544f0 2023-11-06T21:03:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.964+0000 7feeb159b700 1 -- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fee9005f290 msgr2=0x7fee90061750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.964+0000 7feeb159b700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fee9005f290 0x7fee90061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:30.972+0000 7feea17fa700 1 -- 172.21.15.33:0/321365929 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feea404bce0 con 0x7feeac1544f0 2023-11-06T21:03:31.042 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:31.041+0000 7f694102a680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:03:31.079 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:31.017+0000 7f6891b19680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:03:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.156+0000 7feeb37ff700 1 -- 172.21.15.33:0/321365929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7feeac0c8760 con 0x7feeac1544f0 2023-11-06T21:03:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.160+0000 7feea17fa700 1 -- 172.21.15.33:0/321365929 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) v1 ==== 138+0+0 (secure 0 0 0) 0x7feea404f300 con 0x7feeac1544f0 2023-11-06T21:03:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 -- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fee9005f290 msgr2=0x7fee90061750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fee9005f290 0x7fee90061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 -- 172.21.15.33:0/321365929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeac1544f0 msgr2=0x7feeac0b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeac1544f0 0x7feeac0b03e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7feea4009e50 tx=0x7feea400b040).stop 2023-11-06T21:03:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 -- 172.21.15.33:0/321365929 shutdown_connections 2023-11-06T21:03:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fee9005f290 0x7fee90061750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeac1544f0 0x7feeac0b03e0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feeac064bb0 0x7feeac0aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 --2- 172.21.15.33:0/321365929 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feeac0c0c40 0x7feeac0afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:31.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 -- 172.21.15.33:0/321365929 >> 172.21.15.33:0/321365929 conn(0x7feeac0cb900 msgr2=0x7feeac153b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:31.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 -- 172.21.15.33:0/321365929 shutdown_connections 2023-11-06T21:03:31.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.162+0000 7feeb37ff700 1 -- 172.21.15.33:0/321365929 wait complete. 2023-11-06T21:03:31.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:03:31.181 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-06T21:03:31.236 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1 1M 2023-11-06T21:03:31.349 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:31.349+0000 7f6891b19680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:03:31.394 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:31.394+0000 7f694102a680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:03:31.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.437+0000 7f5ffe9ff700 1 Processor -- start 2023-11-06T21:03:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.438+0000 7f5ffe9ff700 1 -- start start 2023-11-06T21:03:31.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.439+0000 7f5ffe9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ff813dea0 0x7f5ff8158060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:31.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.439+0000 7f5ffe9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ff80c2820 0x7f5ff8065c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:31.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.439+0000 7f5ffe9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff80a3da0 con 0x7f5ff8158550 2023-11-06T21:03:31.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.439+0000 7f5ffe9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff8158a00 con 0x7f5ff813dea0 2023-11-06T21:03:31.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.439+0000 7f5ffe9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8065090 con 0x7f5ff80c2820 2023-11-06T21:03:31.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.440+0000 7f5ff7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ff813dea0 0x7f5ff8158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.440+0000 7f5ff7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ff813dea0 0x7f5ff8158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59998/0 (socket says 172.21.15.33:59998) 2023-11-06T21:03:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.440+0000 7f5ff7fff700 1 -- 172.21.15.33:0/2122077339 learned_addr learned my addr 172.21.15.33:0/2122077339 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.440+0000 7f5ffcf9c700 1 --2- 172.21.15.33:0/2122077339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ff80c2820 0x7f5ff8065c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.440+0000 7f5ff7fff700 1 -- 172.21.15.33:0/2122077339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ff80c2820 msgr2=0x7f5ff8065c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:31.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.440+0000 7f5ff7fff700 1 --2- 172.21.15.33:0/2122077339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ff80c2820 0x7f5ff8065c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:31.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.440+0000 7f5ff7fff700 1 -- 172.21.15.33:0/2122077339 >> v1:172.21.15.69:6789/0 conn(0x7f5ff8158550 legacy=0x7f5ff8065520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:31.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.440+0000 7f5ff6ffd700 1 -- 172.21.15.33:0/2122077339 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859924598 0 0) 0x7f5ff80a3da0 con 0x7f5ff8158550 2023-11-06T21:03:31.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.440+0000 7f5ff7fff700 1 -- 172.21.15.33:0/2122077339 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80c3200 con 0x7f5ff813dea0 2023-11-06T21:03:31.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.440+0000 7f5ff7fff700 1 --2- 172.21.15.33:0/2122077339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff813dea0 0x7f5ff8158060 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f5fe8006fd0 tx=0x7f5fe80090f0).ready entity=mon.0 client_cookie=9db24cc6d2d673d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:31.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ff6ffd700 1 -- 172.21.15.33:0/2122077339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe800b580 con 0x7f5ff813dea0 2023-11-06T21:03:31.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ff6ffd700 1 -- 172.21.15.33:0/2122077339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe800b6e0 con 0x7f5ff813dea0 2023-11-06T21:03:31.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ff6ffd700 1 -- 172.21.15.33:0/2122077339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe800ba80 con 0x7f5ff813dea0 2023-11-06T21:03:31.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/2122077339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff813dea0 msgr2=0x7f5ff8158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:31.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ffe9ff700 1 --2- 172.21.15.33:0/2122077339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff813dea0 0x7f5ff8158060 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f5fe8006fd0 tx=0x7f5fe80090f0).stop 2023-11-06T21:03:31.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/2122077339 shutdown_connections 2023-11-06T21:03:31.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ffe9ff700 1 --2- 172.21.15.33:0/2122077339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff813dea0 0x7f5ff8158060 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:31.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ffe9ff700 1 --2- 172.21.15.33:0/2122077339 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ff80c2820 0x7f5ff8065c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:31.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/2122077339 >> 172.21.15.33:0/2122077339 conn(0x7f5ff80c6e70 msgr2=0x7f5ff80c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:31.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/2122077339 shutdown_connections 2023-11-06T21:03:31.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.441+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/2122077339 wait complete. 2023-11-06T21:03:31.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.443+0000 7f5ffe9ff700 1 Processor -- start 2023-11-06T21:03:31.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.443+0000 7f5ffe9ff700 1 -- start start 2023-11-06T21:03:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.444+0000 7f5ffe9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80c2820 0x7f5ff813bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.444+0000 7f5ffe9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff813dea0 0x7f5ff8167900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.444+0000 7f5ffe9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ff8158550 0x7f5ff8167df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.444+0000 7f5ffe9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81682e0 con 0x7f5ff80c2820 2023-11-06T21:03:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.444+0000 7f5ffe9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff813aa00 con 0x7f5ff813dea0 2023-11-06T21:03:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.444+0000 7f5ffe9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff813b860 con 0x7f5ff8158550 2023-11-06T21:03:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.444+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80c2820 0x7f5ff813bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.444+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80c2820 0x7f5ff813bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60010/0 (socket says 172.21.15.33:60010) 2023-11-06T21:03:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.445+0000 7f5ff7fff700 1 -- 172.21.15.33:0/4084128486 learned_addr learned my addr 172.21.15.33:0/4084128486 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.444+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff813dea0 0x7f5ff8167900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.445+0000 7f5ffcf9c700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ff8158550 0x7f5ff8167df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.445+0000 7f5ff7fff700 1 -- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ff8158550 msgr2=0x7f5ff8167df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.445+0000 7f5ff7fff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ff8158550 0x7f5ff8167df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.445+0000 7f5ff7fff700 1 -- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff813dea0 msgr2=0x7f5ff8167900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.445+0000 7f5ff7fff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff813dea0 0x7f5ff8167900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.445+0000 7f5ff7fff700 1 -- 172.21.15.33:0/4084128486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe8009370 con 0x7f5ff80c2820 2023-11-06T21:03:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.445+0000 7f5ff7fff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80c2820 0x7f5ff813bcf0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f5ff8063c60 tx=0x7f5fe8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.445+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8006d90 con 0x7f5ff80c2820 2023-11-06T21:03:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.446+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8006930 con 0x7f5ff80c2820 2023-11-06T21:03:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.446+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/4084128486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff813ac50 con 0x7f5ff80c2820 2023-11-06T21:03:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.446+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe800ba80 con 0x7f5ff80c2820 2023-11-06T21:03:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.446+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/4084128486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff813b210 con 0x7f5ff80c2820 2023-11-06T21:03:31.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.447+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/4084128486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff8064760 con 0x7f5ff80c2820 2023-11-06T21:03:31.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.448+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 56) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f5fe801c630 con 0x7f5ff80c2820 2023-11-06T21:03:31.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.449+0000 7f5fdffff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:31.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.449+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5fe807bd40 con 0x7f5ff80c2820 2023-11-06T21:03:31.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.449+0000 7f5ff77fe700 1 -- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 msgr2=0x7f5fe4061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.449+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:03:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.453+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5fe807f030 con 0x7f5ff80c2820 2023-11-06T21:03:31.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.650+0000 7f5ff77fe700 1 -- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 msgr2=0x7f5fe4061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:31.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.650+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:03:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:31.661+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/4084128486 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ff80ace50 con 0x7f5fe405f2c0 2023-11-06T21:03:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:32.051+0000 7f5ff77fe700 1 -- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 msgr2=0x7f5fe4061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:32.051+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T21:03:32.090 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:32.089+0000 7f6891b19680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:03:32.173 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:32.172+0000 7f6891b19680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:03:32.189 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:32.188+0000 7f694102a680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:03:32.278 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:32.276+0000 7f694102a680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:03:32.403 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:32.402+0000 7f6891b19680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:03:32.517 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:32.517+0000 7f694102a680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:03:32.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:32.852+0000 7f5ff77fe700 1 -- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 msgr2=0x7f5fe4061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:32.852+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T21:03:32.913 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:32.913+0000 7f6891b19680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:03:33.036 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:33.036+0000 7f6891b19680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:03:33.057 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:33.057+0000 7f694102a680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:03:33.189 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:33.189+0000 7f694102a680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:03:33.221 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:33.221+0000 7f6891b19680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:03:33.314 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:33.314+0000 7f6891b19680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:03:33.382 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:33.381+0000 7f694102a680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:03:33.399 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:33.399+0000 7f6891b19680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:03:33.481 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:33.481+0000 7f694102a680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:03:33.571 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:33.571+0000 7f694102a680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:03:33.706 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:33.706+0000 7f6891b19680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:03:33.901 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:33.900+0000 7f694102a680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:03:33.928 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:33.928+0000 7f6891b19680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:03:34.131 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:34.131+0000 7f694102a680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:03:34.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:34.454+0000 7f5ff77fe700 1 -- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 msgr2=0x7f5fe4061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:03:34.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:34.454+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T21:03:34.726 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:34.726+0000 7f6891b19680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:03:34.813 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:34.813+0000 7f6891b19680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:03:34.906 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:34.906+0000 7f6891b19680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:03:34.980 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:34.980+0000 7f694102a680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:03:35.073 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:35.072+0000 7f694102a680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:03:35.091 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.091+0000 7f6891b19680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:03:35.172 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:35.171+0000 7f694102a680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:03:35.180 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.179+0000 7f6891b19680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:03:35.321 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.321+0000 7f6891b19680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:03:35.367 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:35.366+0000 7f694102a680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:03:35.451 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.451+0000 7f6891b19680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:03:35.461 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:35.460+0000 7f694102a680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:03:35.610 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:35.610+0000 7f694102a680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:03:35.748 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:35.748+0000 7f694102a680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:03:35.796 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.796+0000 7f6891b19680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:03:35.885 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.885+0000 7f6891b19680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:03:35.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:35.892+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 57) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f5fe8079990 con 0x7f5ff80c2820 2023-11-06T21:03:35.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:35.892+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 msgr2=0x7f5fe4061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:03:35.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:35.892+0000 7f5fdffff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:03:35.974 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.974+0000 7f6807607700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:35.975 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.974+0000 7f6807607700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:35.975 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.974+0000 7f6807607700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:35.975 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.974+0000 7f6807607700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:35.975 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.974+0000 7f6807607700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:35.976 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.974+0000 7f6804601700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:35.976 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.974+0000 7f6804601700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:35.976 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.974+0000 7f6804601700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:35.976 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.974+0000 7f6804601700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:35.976 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:03:35.974+0000 7f6804601700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:03:36.112 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:36.111+0000 7f694102a680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:03:36.206 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:03:36.206+0000 7f694102a680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:03:36.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.897+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 58) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f5fe8079d10 con 0x7f5ff80c2820 2023-11-06T21:03:36.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.897+0000 7f5fdffff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:36.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.897+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ff80ace50 con 0x7f5fe405f2c0 2023-11-06T21:03:36.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.898+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:36.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.900+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f5ff81654b0 tx=0x7f5fec007d40).ready entity=mgr.7864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.913+0000 7f5fdffff700 1 -- 172.21.15.33:0/4084128486 <== mgr.7864 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f5ff80ace50 con 0x7f5fe405f2c0 2023-11-06T21:03:36.913 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1 2023-11-06T21:03:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 msgr2=0x7f5fe4061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f5ff81654b0 tx=0x7f5fec007d40).stop 2023-11-06T21:03:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/4084128486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80c2820 msgr2=0x7f5ff813bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80c2820 0x7f5ff813bcf0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f5ff8063c60 tx=0x7f5fe8017000).stop 2023-11-06T21:03:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/4084128486 shutdown_connections 2023-11-06T21:03:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5fe405f2c0 0x7f5fe4061700 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ff8158550 0x7f5ff8167df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80c2820 0x7f5ff813bcf0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 --2- 172.21.15.33:0/4084128486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff813dea0 0x7f5ff8167900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/4084128486 >> 172.21.15.33:0/4084128486 conn(0x7f5ff80c6e70 msgr2=0x7f5ff80a3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/4084128486 shutdown_connections 2023-11-06T21:03:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:36.915+0000 7f5ffe9ff700 1 -- 172.21.15.33:0/4084128486 wait complete. 2023-11-06T21:03:36.937 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-11-06T21:03:36.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs /snap_test_dir1 55n 2023-11-06T21:03:37.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.145+0000 7f2385bff700 1 Processor -- start 2023-11-06T21:03:37.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.146+0000 7f2385bff700 1 -- start start 2023-11-06T21:03:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f2385bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2380133f10 0x7f23801520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f2385bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23801525e0 con 0x7f23801342f0 2023-11-06T21:03:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f2385bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2380154f90 con 0x7f23800c3b60 2023-11-06T21:03:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f2385bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f238014bc20 con 0x7f2380133f10 2023-11-06T21:03:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2380133f10 0x7f23801520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f23801342f0 0x7f23800c3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37378/0 (socket says 172.21.15.33:37378) 2023-11-06T21:03:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237effd700 1 -- 172.21.15.33:0/2911366849 learned_addr learned my addr 172.21.15.33:0/2911366849 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:03:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237f7fe700 1 -- 172.21.15.33:0/2911366849 >> v1:172.21.15.69:6789/0 conn(0x7f23800c3b60 legacy=0x7f2380069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237f7fe700 1 -- 172.21.15.33:0/2911366849 >> v1:172.21.15.33:6789/0 conn(0x7f23801342f0 legacy=0x7f23800c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237f7fe700 1 -- 172.21.15.33:0/2911366849 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2380155270 con 0x7f2380133f10 2023-11-06T21:03:37.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237e7fc700 1 -- 172.21.15.33:0/2911366849 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152554832 0 0) 0x7f23801525e0 con 0x7f23801342f0 2023-11-06T21:03:37.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237f7fe700 1 --2- 172.21.15.33:0/2911366849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2380133f10 0x7f23801520f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f237000ba00 tx=0x7f237000b300).ready entity=mon.0 client_cookie=c2945e692b365d6e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:37.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237e7fc700 1 -- 172.21.15.33:0/2911366849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237000da30 con 0x7f2380133f10 2023-11-06T21:03:37.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237e7fc700 1 -- 172.21.15.33:0/2911366849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237000db90 con 0x7f2380133f10 2023-11-06T21:03:37.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.148+0000 7f237e7fc700 1 -- 172.21.15.33:0/2911366849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370010470 con 0x7f2380133f10 2023-11-06T21:03:37.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.149+0000 7f2385bff700 1 -- 172.21.15.33:0/2911366849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2380133f10 msgr2=0x7f23801520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.149+0000 7f2385bff700 1 --2- 172.21.15.33:0/2911366849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2380133f10 0x7f23801520f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f237000ba00 tx=0x7f237000b300).stop 2023-11-06T21:03:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.149+0000 7f2385bff700 1 -- 172.21.15.33:0/2911366849 shutdown_connections 2023-11-06T21:03:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.149+0000 7f2385bff700 1 --2- 172.21.15.33:0/2911366849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2380133f10 0x7f23801520f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.149+0000 7f2385bff700 1 -- 172.21.15.33:0/2911366849 >> 172.21.15.33:0/2911366849 conn(0x7f2380062710 msgr2=0x7f2380062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.149+0000 7f2385bff700 1 -- 172.21.15.33:0/2911366849 shutdown_connections 2023-11-06T21:03:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.149+0000 7f2385bff700 1 -- 172.21.15.33:0/2911366849 wait complete. 2023-11-06T21:03:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.150+0000 7f2385bff700 1 Processor -- start 2023-11-06T21:03:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.150+0000 7f2385bff700 1 -- start start 2023-11-06T21:03:37.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.151+0000 7f2385bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23800c3b60 0x7f238014ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.151+0000 7f2385bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2380133f10 0x7f23800ab900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.151+0000 7f2385bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23801342f0 0x7f23800abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.151+0000 7f2385bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23800ac2e0 con 0x7f23800c3b60 2023-11-06T21:03:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.151+0000 7f2385bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23800aad30 con 0x7f2380133f10 2023-11-06T21:03:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.151+0000 7f2385bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2380160500 con 0x7f23801342f0 2023-11-06T21:03:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23800c3b60 0x7f238014ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23800c3b60 0x7f238014ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60034/0 (socket says 172.21.15.33:60034) 2023-11-06T21:03:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237f7fe700 1 -- 172.21.15.33:0/929756734 learned_addr learned my addr 172.21.15.33:0/929756734 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237effd700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2380133f10 0x7f23800ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237ffff700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23801342f0 0x7f23800abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237f7fe700 1 -- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23801342f0 msgr2=0x7f23800abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237f7fe700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23801342f0 0x7f23800abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237f7fe700 1 -- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2380133f10 msgr2=0x7f23800ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237f7fe700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2380133f10 0x7f23800ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237f7fe700 1 -- 172.21.15.33:0/929756734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237000b540 con 0x7f23800c3b60 2023-11-06T21:03:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237ffff700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23801342f0 0x7f23800abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:03:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f237f7fe700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23800c3b60 0x7f238014ac00 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7f237000b340 tx=0x7f2380068420).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:37.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f23677fe700 1 -- 172.21.15.33:0/929756734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237000ded0 con 0x7f23800c3b60 2023-11-06T21:03:37.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.152+0000 7f23677fe700 1 -- 172.21.15.33:0/929756734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237001f040 con 0x7f23800c3b60 2023-11-06T21:03:37.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.153+0000 7f23677fe700 1 -- 172.21.15.33:0/929756734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23700198b0 con 0x7f23800c3b60 2023-11-06T21:03:37.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.153+0000 7f2385bff700 1 -- 172.21.15.33:0/929756734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23800c50f0 con 0x7f23800c3b60 2023-11-06T21:03:37.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.153+0000 7f2385bff700 1 -- 172.21.15.33:0/929756734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23800c56b0 con 0x7f23800c3b60 2023-11-06T21:03:37.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.154+0000 7f2385bff700 1 -- 172.21.15.33:0/929756734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23800c5fe0 con 0x7f23800c3b60 2023-11-06T21:03:37.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.154+0000 7f23677fe700 1 -- 172.21.15.33:0/929756734 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 58) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f2370019c20 con 0x7f23800c3b60 2023-11-06T21:03:37.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.155+0000 7f23677fe700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f236c05f080 0x7f236c061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.155+0000 7f23677fe700 1 -- 172.21.15.33:0/929756734 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f237007e120 con 0x7f23800c3b60 2023-11-06T21:03:37.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.155+0000 7f237effd700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f236c05f080 0x7f236c061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.156+0000 7f237effd700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f236c05f080 0x7f236c061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f23800c1630 tx=0x7f2374009040).ready entity=mgr.7864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:37.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.160+0000 7f23677fe700 1 -- 172.21.15.33:0/929756734 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f237004f4c0 con 0x7f23800c3b60 2023-11-06T21:03:37.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.370+0000 7f2385bff700 1 -- 172.21.15.33:0/929756734 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f23800659f0 con 0x7f236c05f080 2023-11-06T21:03:37.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.377+0000 7f23677fe700 1 -- 172.21.15.33:0/929756734 <== mgr.7864 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f23800659f0 con 0x7f236c05f080 2023-11-06T21:03:37.377 INFO:teuthology.orchestra.run.smithi033.stdout:Retention added to path /snap_test_dir1 2023-11-06T21:03:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 -- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f236c05f080 msgr2=0x7f236c061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f236c05f080 0x7f236c061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f23800c1630 tx=0x7f2374009040).stop 2023-11-06T21:03:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 -- 172.21.15.33:0/929756734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23800c3b60 msgr2=0x7f238014ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23800c3b60 0x7f238014ac00 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7f237000b340 tx=0x7f2380068420).stop 2023-11-06T21:03:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 -- 172.21.15.33:0/929756734 shutdown_connections 2023-11-06T21:03:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f236c05f080 0x7f236c061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23801342f0 0x7f23800abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23800c3b60 0x7f238014ac00 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 --2- 172.21.15.33:0/929756734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2380133f10 0x7f23800ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 -- 172.21.15.33:0/929756734 >> 172.21.15.33:0/929756734 conn(0x7f2380062710 msgr2=0x7f2380153880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 -- 172.21.15.33:0/929756734 shutdown_connections 2023-11-06T21:03:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.379+0000 7f2385bff700 1 -- 172.21.15.33:0/929756734 wait complete. 2023-11-06T21:03:37.398 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-11-06T21:03:37.398 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-11-06T21:03:37.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json /snap_test_dir1 2023-11-06T21:03:37.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.598+0000 7f6ca77ff700 1 Processor -- start 2023-11-06T21:03:37.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.598+0000 7f6ca77ff700 1 -- start start 2023-11-06T21:03:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca77ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ca00acfc0 0x7f6ca00ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca77ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6ca01550d0 0x7f6ca00c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca77ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca01586a0 con 0x7f6ca00631a0 2023-11-06T21:03:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca77ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca00c6590 con 0x7f6ca00acfc0 2023-11-06T21:03:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca77ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca0155580 con 0x7f6ca01550d0 2023-11-06T21:03:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca559b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ca00acfc0 0x7f6ca00ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca559b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ca00acfc0 0x7f6ca00ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60038/0 (socket says 172.21.15.33:60038) 2023-11-06T21:03:37.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca559b700 1 -- 172.21.15.33:0/2591086930 learned_addr learned my addr 172.21.15.33:0/2591086930 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca5d9c700 1 --1- 172.21.15.33:0/2591086930 >> v1:172.21.15.33:6789/0 conn(0x7f6ca00631a0 0x7f6ca0064680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37394/0 (socket says 172.21.15.33:37394) 2023-11-06T21:03:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca4d9a700 1 --2- 172.21.15.33:0/2591086930 >> v2:172.21.15.69:3300/0 conn(0x7f6ca01550d0 0x7f6ca00c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca559b700 1 -- 172.21.15.33:0/2591086930 >> v2:172.21.15.69:3300/0 conn(0x7f6ca01550d0 msgr2=0x7f6ca00c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:03:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca559b700 1 --2- 172.21.15.33:0/2591086930 >> v2:172.21.15.69:3300/0 conn(0x7f6ca01550d0 0x7f6ca00c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca559b700 1 -- 172.21.15.33:0/2591086930 >> v1:172.21.15.33:6789/0 conn(0x7f6ca00631a0 legacy=0x7f6ca0064680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca559b700 1 -- 172.21.15.33:0/2591086930 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca01544c0 con 0x7f6ca00acfc0 2023-11-06T21:03:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6c97fff700 1 -- 172.21.15.33:0/2591086930 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4286588625 0 0) 0x7f6ca01586a0 con 0x7f6ca00631a0 2023-11-06T21:03:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6ca559b700 1 --2- 172.21.15.33:0/2591086930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00acfc0 0x7f6ca00ad3a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7f6c9c006fd0 tx=0x7f6c9c0090f0).ready entity=mon.0 client_cookie=adca66e8cf2f43c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6c97fff700 1 -- 172.21.15.33:0/2591086930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c00b600 con 0x7f6ca00acfc0 2023-11-06T21:03:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6c97fff700 1 -- 172.21.15.33:0/2591086930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c00b760 con 0x7f6ca00acfc0 2023-11-06T21:03:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.600+0000 7f6c97fff700 1 -- 172.21.15.33:0/2591086930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c00bae0 con 0x7f6ca00acfc0 2023-11-06T21:03:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.601+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2591086930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00acfc0 msgr2=0x7f6ca00ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.601+0000 7f6ca77ff700 1 --2- 172.21.15.33:0/2591086930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00acfc0 0x7f6ca00ad3a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7f6c9c006fd0 tx=0x7f6c9c0090f0).stop 2023-11-06T21:03:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.601+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2591086930 shutdown_connections 2023-11-06T21:03:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.601+0000 7f6ca77ff700 1 --2- 172.21.15.33:0/2591086930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00acfc0 0x7f6ca00ad3a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.601+0000 7f6ca77ff700 1 --2- 172.21.15.33:0/2591086930 >> v2:172.21.15.69:3300/0 conn(0x7f6ca01550d0 0x7f6ca00c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.601+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2591086930 >> 172.21.15.33:0/2591086930 conn(0x7f6ca005ec80 msgr2=0x7f6ca00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.601+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2591086930 shutdown_connections 2023-11-06T21:03:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.601+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2591086930 wait complete. 2023-11-06T21:03:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.602+0000 7f6ca77ff700 1 Processor -- start 2023-11-06T21:03:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.603+0000 7f6ca77ff700 1 -- start start 2023-11-06T21:03:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.603+0000 7f6ca77ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6ca00631a0 0x7f6ca0152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca77ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ca00acfc0 0x7f6ca0151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca77ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca01550d0 0x7f6ca0150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca77ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0153070 con 0x7f6ca01550d0 2023-11-06T21:03:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca77ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0155e20 con 0x7f6ca00acfc0 2023-11-06T21:03:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca77ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0152d90 con 0x7f6ca00631a0 2023-11-06T21:03:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca5d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca01550d0 0x7f6ca0150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca5d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca01550d0 0x7f6ca0150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60040/0 (socket says 172.21.15.33:60040) 2023-11-06T21:03:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca5d9c700 1 -- 172.21.15.33:0/2290890176 learned_addr learned my addr 172.21.15.33:0/2290890176 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:03:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca4d9a700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ca00acfc0 0x7f6ca0151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca559b700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6ca00631a0 0x7f6ca0152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca5d9c700 1 -- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6ca00631a0 msgr2=0x7f6ca0152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca5d9c700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6ca00631a0 0x7f6ca0152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca5d9c700 1 -- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ca00acfc0 msgr2=0x7f6ca0151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca5d9c700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ca00acfc0 0x7f6ca0151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.604+0000 7f6ca5d9c700 1 -- 172.21.15.33:0/2290890176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c009370 con 0x7f6ca01550d0 2023-11-06T21:03:37.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.605+0000 7f6ca5d9c700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca01550d0 0x7f6ca0150b00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f6c9800c810 tx=0x7f6c98007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:37.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.605+0000 7f6c957fa700 1 -- 172.21.15.33:0/2290890176 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98005c70 con 0x7f6ca01550d0 2023-11-06T21:03:37.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.605+0000 7f6c957fa700 1 -- 172.21.15.33:0/2290890176 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c98005df0 con 0x7f6ca01550d0 2023-11-06T21:03:37.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.605+0000 7f6c957fa700 1 -- 172.21.15.33:0/2290890176 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98007360 con 0x7f6ca01550d0 2023-11-06T21:03:37.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.605+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2290890176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca01557d0 con 0x7f6ca01550d0 2023-11-06T21:03:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.605+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2290890176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca0151a80 con 0x7f6ca01550d0 2023-11-06T21:03:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.606+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2290890176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca0155f90 con 0x7f6ca01550d0 2023-11-06T21:03:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.607+0000 7f6c957fa700 1 -- 172.21.15.33:0/2290890176 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 58) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f6c98007750 con 0x7f6ca01550d0 2023-11-06T21:03:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.608+0000 7f6c957fa700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c8405f080 0x7f6c84061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:03:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.608+0000 7f6c957fa700 1 -- 172.21.15.33:0/2290890176 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f6c9807a0f0 con 0x7f6ca01550d0 2023-11-06T21:03:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.609+0000 7f6ca559b700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c8405f080 0x7f6c84061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:03:37.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.609+0000 7f6ca559b700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c8405f080 0x7f6c84061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6c9c009130 tx=0x7f6c9c017000).ready entity=mgr.7864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:03:37.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.613+0000 7f6c957fa700 1 -- 172.21.15.33:0/2290890176 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6c9804b380 con 0x7f6ca01550d0 2023-11-06T21:03:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.822+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2290890176 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ca00c86a0 con 0x7f6c8405f080 2023-11-06T21:03:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.826+0000 7f6c957fa700 1 -- 172.21.15.33:0/2290890176 <== mgr.7864 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f6ca00c86a0 con 0x7f6c8405f080 2023-11-06T21:03:37.826 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:03:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-11-06T21:03:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.827+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c8405f080 msgr2=0x7f6c84061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.827+0000 7f6ca77ff700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c8405f080 0x7f6c84061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6c9c009130 tx=0x7f6c9c017000).stop 2023-11-06T21:03:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.827+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2290890176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca01550d0 msgr2=0x7f6ca0150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:03:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.827+0000 7f6ca77ff700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca01550d0 0x7f6ca0150b00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f6c9800c810 tx=0x7f6c98007f10).stop 2023-11-06T21:03:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.828+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2290890176 shutdown_connections 2023-11-06T21:03:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.828+0000 7f6ca77ff700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6c8405f080 0x7f6c84061540 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.828+0000 7f6ca77ff700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6ca00631a0 0x7f6ca0152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.828+0000 7f6ca77ff700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca01550d0 0x7f6ca0150b00 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.828+0000 7f6ca77ff700 1 --2- 172.21.15.33:0/2290890176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ca00acfc0 0x7f6ca0151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:03:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.828+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2290890176 >> 172.21.15.33:0/2290890176 conn(0x7f6ca005ec80 msgr2=0x7f6ca01536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:03:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.828+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2290890176 shutdown_connections 2023-11-06T21:03:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:03:37.828+0000 7f6ca77ff700 1 -- 172.21.15.33:0/2290890176 wait complete. 2023-11-06T21:03:37.846 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-11-06T21:03:37.846 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2023-11-06T21:03:39.647 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:03:39.652 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:04:00.384 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:04:00.383+0000 7f67f65a5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:04:00.384 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:04:00.383+0000 7f67f65a5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:04:00.385 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:04:00.384+0000 7f67f65a5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:04:00.385 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:04:00.384+0000 7f67f65a5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:04:00.385 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:04:00.384+0000 7f67f65a5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:04:09.762 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:04:09.768 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:04:39.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:04:39.915 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:05:10.023 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:05:10.029 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:05:40.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:05:40.142 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:06:10.257 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:06:10.263 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:06:40.372 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:06:40.377 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:07:10.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:07:10.499 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:07:40.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:07:40.630 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:08:10.979 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:08:10.985 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:08:41.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:08:41.122 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:09:11.238 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:09:11.247 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:09:41.360 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:09:41.366 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:10:11.475 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:10:11.481 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:10:41.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:10:41.602 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:11:11.717 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:11:11.722 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:11:41.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:11:41.861 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:12:11.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:12:11.983 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:12:42.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:12:42.097 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:13:12.206 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:13:12.212 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:13:42.318 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:13:42.325 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:14:12.523 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:14:12.529 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:14:42.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:14:42.659 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:15:12.774 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:15:12.780 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:15:42.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:15:42.903 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:16:13.018 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:16:13.024 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:16:43.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:16:43.139 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:17:13.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:17:13.261 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:17:43.392 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:17:43.398 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:18:13.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:18:13.521 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:18:43.643 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:18:43.649 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:19:13.768 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:19:13.774 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:19:43.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:19:43.904 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:20:14.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:20:14.027 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:20:44.142 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:20:44.149 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:21:14.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:21:14.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:21:44.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:21:44.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:22:14.516 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:22:14.523 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:22:44.917 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:22:44.923 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:23:15.036 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:23:15.047 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:23:45.158 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:23:45.164 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:24:15.273 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:24:15.279 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:24:45.394 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:24:45.400 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:25:15.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:25:15.531 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:25:45.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:25:45.703 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:26:15.818 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:26:15.824 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:26:45.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:26:45.964 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:27:16.079 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:27:16.085 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:27:46.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:27:46.199 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:28:16.323 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:28:16.329 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:28:46.435 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:28:46.442 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:29:16.563 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:29:16.569 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:29:46.695 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:29:46.702 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:30:16.844 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:30:16.850 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:30:46.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:30:46.964 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:31:17.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:31:17.074 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:31:47.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:31:47.191 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:32:17.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:32:17.524 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:32:47.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:32:47.639 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:33:17.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:33:17.781 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:33:47.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:33:47.911 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:34:18.083 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:34:18.088 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:34:48.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:34:48.297 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:35:18.421 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:35:18.427 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:35:48.534 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:35:48.540 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:36:18.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:36:18.657 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:36:48.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:36:48.788 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:37:18.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:37:18.918 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:37:49.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:37:49.036 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:38:19.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:38:19.158 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:38:49.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:38:49.275 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:39:19.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:39:19.677 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:39:49.783 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:39:49.789 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:40:19.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:40:19.912 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:40:50.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:40:50.033 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:41:20.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:41:20.156 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:41:50.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:41:50.411 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:42:20.526 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:42:20.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:42:50.642 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:42:50.648 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:43:20.762 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:43:20.768 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:43:50.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:43:50.882 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:44:20.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:44:21.004 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:44:51.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:44:51.376 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:45:21.484 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:45:21.490 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:45:51.821 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:45:51.828 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:46:21.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:46:21.949 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:46:52.049 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:46:52.055 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:47:22.170 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:47:22.176 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:47:52.327 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:47:52.333 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:48:22.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:48:22.455 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:48:52.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:48:52.593 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:49:22.708 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:49:22.714 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:49:52.841 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:49:52.847 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:50:23.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:50:23.008 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:50:53.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:50:53.132 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:51:23.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:51:23.247 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:51:53.379 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:51:53.385 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:52:23.502 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:52:23.508 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:52:53.630 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:52:53.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:53:23.746 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:53:23.752 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:53:54.035 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:53:54.041 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:54:24.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:54:24.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:54:54.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:54:54.283 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:55:24.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:55:24.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:55:54.517 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:55:54.523 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:56:24.635 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:56:24.641 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:56:54.757 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:56:54.763 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:57:24.955 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:57:24.961 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:57:55.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:57:55.074 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:58:25.177 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:58:25.183 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:58:55.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:58:55.316 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:59:15.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:15.848 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:59:15.906 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_05_00_UTC 2023-11-06T21:59:15.906 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_06_00_UTC 2023-11-06T21:59:15.907 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_07_00_UTC 2023-11-06T21:59:15.907 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_08_00_UTC 2023-11-06T21:59:15.907 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_09_00_UTC 2023-11-06T21:59:15.907 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_10_00_UTC 2023-11-06T21:59:15.908 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_11_00_UTC 2023-11-06T21:59:15.908 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_12_00_UTC 2023-11-06T21:59:15.908 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_13_00_UTC 2023-11-06T21:59:15.908 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_14_00_UTC 2023-11-06T21:59:15.908 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_15_00_UTC 2023-11-06T21:59:15.909 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_16_00_UTC 2023-11-06T21:59:15.909 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_17_00_UTC 2023-11-06T21:59:15.909 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_18_00_UTC 2023-11-06T21:59:15.909 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_19_00_UTC 2023-11-06T21:59:15.909 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_20_00_UTC 2023-11-06T21:59:15.910 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_21_00_UTC 2023-11-06T21:59:15.910 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_22_00_UTC 2023-11-06T21:59:15.910 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_23_00_UTC 2023-11-06T21:59:15.910 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_24_00_UTC 2023-11-06T21:59:15.910 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_25_00_UTC 2023-11-06T21:59:15.911 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_26_00_UTC 2023-11-06T21:59:15.911 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_27_00_UTC 2023-11-06T21:59:15.911 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_28_00_UTC 2023-11-06T21:59:15.911 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_29_00_UTC 2023-11-06T21:59:15.911 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_30_00_UTC 2023-11-06T21:59:15.912 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_31_00_UTC 2023-11-06T21:59:15.912 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_32_00_UTC 2023-11-06T21:59:15.912 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_33_00_UTC 2023-11-06T21:59:15.912 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_34_00_UTC 2023-11-06T21:59:15.912 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_35_00_UTC 2023-11-06T21:59:15.913 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_36_00_UTC 2023-11-06T21:59:15.913 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_37_00_UTC 2023-11-06T21:59:15.913 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_38_00_UTC 2023-11-06T21:59:15.913 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_39_00_UTC 2023-11-06T21:59:15.913 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_40_00_UTC 2023-11-06T21:59:15.913 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_41_00_UTC 2023-11-06T21:59:15.914 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_42_00_UTC 2023-11-06T21:59:15.914 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_43_00_UTC 2023-11-06T21:59:15.914 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_44_00_UTC 2023-11-06T21:59:15.914 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_45_00_UTC 2023-11-06T21:59:15.914 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_46_00_UTC 2023-11-06T21:59:15.915 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_47_00_UTC 2023-11-06T21:59:15.915 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_48_00_UTC 2023-11-06T21:59:15.915 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_49_00_UTC 2023-11-06T21:59:15.915 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_50_00_UTC 2023-11-06T21:59:15.915 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_51_00_UTC 2023-11-06T21:59:15.915 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_52_00_UTC 2023-11-06T21:59:15.916 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_53_00_UTC 2023-11-06T21:59:15.916 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_54_00_UTC 2023-11-06T21:59:15.916 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_55_00_UTC 2023-11-06T21:59:15.916 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_56_00_UTC 2023-11-06T21:59:15.916 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_57_00_UTC 2023-11-06T21:59:15.917 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_58_00_UTC 2023-11-06T21:59:15.917 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_59_00_UTC 2023-11-06T21:59:15.918 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-21_05_00_UTC', 'scheduled-2023-11-06-21_06_00_UTC', 'scheduled-2023-11-06-21_07_00_UTC', 'scheduled-2023-11-06-21_08_00_UTC', 'scheduled-2023-11-06-21_09_00_UTC', 'scheduled-2023-11-06-21_10_00_UTC', 'scheduled-2023-11-06-21_11_00_UTC', 'scheduled-2023-11-06-21_12_00_UTC', 'scheduled-2023-11-06-21_13_00_UTC', 'scheduled-2023-11-06-21_14_00_UTC', 'scheduled-2023-11-06-21_15_00_UTC', 'scheduled-2023-11-06-21_16_00_UTC', 'scheduled-2023-11-06-21_17_00_UTC', 'scheduled-2023-11-06-21_18_00_UTC', 'scheduled-2023-11-06-21_19_00_UTC', 'scheduled-2023-11-06-21_20_00_UTC', 'scheduled-2023-11-06-21_21_00_UTC', 'scheduled-2023-11-06-21_22_00_UTC', 'scheduled-2023-11-06-21_23_00_UTC', 'scheduled-2023-11-06-21_24_00_UTC', 'scheduled-2023-11-06-21_25_00_UTC', 'scheduled-2023-11-06-21_26_00_UTC', 'scheduled-2023-11-06-21_27_00_UTC', 'scheduled-2023-11-06-21_28_00_UTC', 'scheduled-2023-11-06-21_29_00_UTC', 'scheduled-2023-11-06-21_30_00_UTC', 'scheduled-2023-11-06-21_31_00_UTC', 'scheduled-2023-11-06-21_32_00_UTC', 'scheduled-2023-11-06-21_33_00_UTC', 'scheduled-2023-11-06-21_34_00_UTC', 'scheduled-2023-11-06-21_35_00_UTC', 'scheduled-2023-11-06-21_36_00_UTC', 'scheduled-2023-11-06-21_37_00_UTC', 'scheduled-2023-11-06-21_38_00_UTC', 'scheduled-2023-11-06-21_39_00_UTC', 'scheduled-2023-11-06-21_40_00_UTC', 'scheduled-2023-11-06-21_41_00_UTC', 'scheduled-2023-11-06-21_42_00_UTC', 'scheduled-2023-11-06-21_43_00_UTC', 'scheduled-2023-11-06-21_44_00_UTC', 'scheduled-2023-11-06-21_45_00_UTC', 'scheduled-2023-11-06-21_46_00_UTC', 'scheduled-2023-11-06-21_47_00_UTC', 'scheduled-2023-11-06-21_48_00_UTC', 'scheduled-2023-11-06-21_49_00_UTC', 'scheduled-2023-11-06-21_50_00_UTC', 'scheduled-2023-11-06-21_51_00_UTC', 'scheduled-2023-11-06-21_52_00_UTC', 'scheduled-2023-11-06-21_53_00_UTC', 'scheduled-2023-11-06-21_54_00_UTC', 'scheduled-2023-11-06-21_55_00_UTC', 'scheduled-2023-11-06-21_56_00_UTC', 'scheduled-2023-11-06-21_57_00_UTC', 'scheduled-2023-11-06-21_58_00_UTC', 'scheduled-2023-11-06-21_59_00_UTC'] 2023-11-06T21:59:15.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1 2023-11-06T21:59:16.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.121+0000 7f834dbff700 1 Processor -- start 2023-11-06T21:59:16.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.121+0000 7f834dbff700 1 -- start start 2023-11-06T21:59:16.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.122+0000 7f834dbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83480c4a50 0x7f83480666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:16.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.123+0000 7f834dbff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8348155a20 con 0x7f8348067c20 2023-11-06T21:59:16.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.123+0000 7f834dbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8348068170 con 0x7f834814e630 2023-11-06T21:59:16.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.123+0000 7f834dbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8348068450 con 0x7f83480c4a50 2023-11-06T21:59:16.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.123+0000 7f8346ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8348067c20 0x7f8348155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38468/0 (socket says 172.21.15.33:38468) 2023-11-06T21:59:16.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.123+0000 7f8346ffd700 1 -- 172.21.15.33:0/1385545129 learned_addr learned my addr 172.21.15.33:0/1385545129 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:16.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.123+0000 7f8347fff700 1 --2- 172.21.15.33:0/1385545129 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83480c4a50 0x7f83480666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:16.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.123+0000 7f83467fc700 1 -- 172.21.15.33:0/1385545129 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080002847 0 0) 0x7f8348155a20 con 0x7f8348067c20 2023-11-06T21:59:16.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.123+0000 7f83467fc700 1 -- 172.21.15.33:0/1385545129 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83300033f0 con 0x7f8348067c20 2023-11-06T21:59:16.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.123+0000 7f83467fc700 1 -- 172.21.15.33:0/1385545129 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122523233 0 0) 0x7f8348068170 con 0x7f834814e630 2023-11-06T21:59:16.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.123+0000 7f83467fc700 1 -- 172.21.15.33:0/1385545129 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8348155a20 con 0x7f834814e630 2023-11-06T21:59:16.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.124+0000 7f83467fc700 1 -- 172.21.15.33:0/1385545129 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 215848325 0 0) 0x7f83300033f0 con 0x7f8348067c20 2023-11-06T21:59:16.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.124+0000 7f8347fff700 1 -- 172.21.15.33:0/1385545129 >> v1:172.21.15.69:6789/0 conn(0x7f834814e630 legacy=0x7f834814ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.124+0000 7f8347fff700 1 -- 172.21.15.33:0/1385545129 >> v1:172.21.15.33:6789/0 conn(0x7f8348067c20 legacy=0x7f8348155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.124+0000 7f8347fff700 1 -- 172.21.15.33:0/1385545129 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8348068730 con 0x7f83480c4a50 2023-11-06T21:59:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.124+0000 7f8347fff700 1 --2- 172.21.15.33:0/1385545129 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83480c4a50 0x7f83480666b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f8340009e90 tx=0x7f8340009780).ready entity=mon.2 client_cookie=4e7edde59c39ebf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.124+0000 7f83467fc700 1 -- 172.21.15.33:0/1385545129 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834000e070 con 0x7f83480c4a50 2023-11-06T21:59:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.124+0000 7f83467fc700 1 -- 172.21.15.33:0/1385545129 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834000b7a0 con 0x7f83480c4a50 2023-11-06T21:59:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.124+0000 7f83467fc700 1 -- 172.21.15.33:0/1385545129 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c001470 con 0x7f83480c4a50 2023-11-06T21:59:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.125+0000 7f834dbff700 1 -- 172.21.15.33:0/1385545129 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83480c4a50 msgr2=0x7f83480666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.125+0000 7f834dbff700 1 --2- 172.21.15.33:0/1385545129 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83480c4a50 0x7f83480666b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f8340009e90 tx=0x7f8340009780).stop 2023-11-06T21:59:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.125+0000 7f834dbff700 1 -- 172.21.15.33:0/1385545129 shutdown_connections 2023-11-06T21:59:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.125+0000 7f834dbff700 1 --2- 172.21.15.33:0/1385545129 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83480c4a50 0x7f83480666b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.125+0000 7f834dbff700 1 -- 172.21.15.33:0/1385545129 >> 172.21.15.33:0/1385545129 conn(0x7f83480c8e60 msgr2=0x7f83480c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.125+0000 7f834dbff700 1 -- 172.21.15.33:0/1385545129 shutdown_connections 2023-11-06T21:59:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.125+0000 7f834dbff700 1 -- 172.21.15.33:0/1385545129 wait complete. 2023-11-06T21:59:16.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.126+0000 7f834dbff700 1 Processor -- start 2023-11-06T21:59:16.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.126+0000 7f834dbff700 1 -- start start 2023-11-06T21:59:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.127+0000 7f834dbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8348067c20 0x7f83480ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.127+0000 7f834dbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83480c4a50 0x7f83481398d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.127+0000 7f834dbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f834814e630 0x7f83480c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.127+0000 7f834dbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83480aebe0 con 0x7f83480c4a50 2023-11-06T21:59:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.127+0000 7f834dbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83480bff00 con 0x7f834814e630 2023-11-06T21:59:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.127+0000 7f834dbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83480c04c0 con 0x7f8348067c20 2023-11-06T21:59:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8346ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83480c4a50 0x7f83481398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8346ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83480c4a50 0x7f83481398d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46608/0 (socket says 172.21.15.33:46608) 2023-11-06T21:59:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8346ffd700 1 -- 172.21.15.33:0/75869236 learned_addr learned my addr 172.21.15.33:0/75869236 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f83477fe700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8348067c20 0x7f83480ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8347fff700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f834814e630 0x7f83480c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8346ffd700 1 -- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8348067c20 msgr2=0x7f83480ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8346ffd700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8348067c20 0x7f83480ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8346ffd700 1 -- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f834814e630 msgr2=0x7f83480c27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8346ffd700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f834814e630 0x7f83480c27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8346ffd700 1 -- 172.21.15.33:0/75869236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340009a20 con 0x7f83480c4a50 2023-11-06T21:59:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f83477fe700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8348067c20 0x7f83480ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8347fff700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f834814e630 0x7f83480c27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f8346ffd700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83480c4a50 0x7f83481398d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f833c00c800 tx=0x7f833c001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f832f7fe700 1 -- 172.21.15.33:0/75869236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c007170 con 0x7f83480c4a50 2023-11-06T21:59:16.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.128+0000 7f832f7fe700 1 -- 172.21.15.33:0/75869236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c0072d0 con 0x7f83480c4a50 2023-11-06T21:59:16.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.129+0000 7f834dbff700 1 -- 172.21.15.33:0/75869236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8348139dc0 con 0x7f83480c4a50 2023-11-06T21:59:16.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.129+0000 7f832f7fe700 1 -- 172.21.15.33:0/75869236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c0048c0 con 0x7f83480c4a50 2023-11-06T21:59:16.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.129+0000 7f834dbff700 1 -- 172.21.15.33:0/75869236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83480ae900 con 0x7f83480c4a50 2023-11-06T21:59:16.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.130+0000 7f834dbff700 1 -- 172.21.15.33:0/75869236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83480c0630 con 0x7f83480c4a50 2023-11-06T21:59:16.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.130+0000 7f832f7fe700 1 -- 172.21.15.33:0/75869236 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 61) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f833c004c30 con 0x7f83480c4a50 2023-11-06T21:59:16.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.132+0000 7f832f7fe700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f290 0x7f8334061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:16.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.132+0000 7f832f7fe700 1 -- 172.21.15.33:0/75869236 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f833c07a420 con 0x7f83480c4a50 2023-11-06T21:59:16.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.132+0000 7f83477fe700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f290 0x7f8334061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.133+0000 7f83477fe700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f290 0x7f8334061750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f8348168050 tx=0x7f8338009040).ready entity=mgr.7864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.137+0000 7f832f7fe700 1 -- 172.21.15.33:0/75869236 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f833c04b3a0 con 0x7f83480c4a50 2023-11-06T21:59:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.344+0000 7f834dbff700 1 -- 172.21.15.33:0/75869236 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f834806ad10 con 0x7f833405f290 2023-11-06T21:59:16.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.346+0000 7f832f7fe700 1 -- 172.21.15.33:0/75869236 <== mgr.7864 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f834806ad10 con 0x7f833405f290 2023-11-06T21:59:16.348 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:16.348 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T21:03:36", "first": "2023-11-06T21:04:00", "last": "2023-11-06T21:59:00", "last_pruned": "2023-11-06T21:59:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-11-06T21:59:16.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 -- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f290 msgr2=0x7f8334061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f290 0x7f8334061750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f8348168050 tx=0x7f8338009040).stop 2023-11-06T21:59:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 -- 172.21.15.33:0/75869236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83480c4a50 msgr2=0x7f83481398d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83480c4a50 0x7f83481398d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f833c00c800 tx=0x7f833c001b50).stop 2023-11-06T21:59:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 -- 172.21.15.33:0/75869236 shutdown_connections 2023-11-06T21:59:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f290 0x7f8334061750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8348067c20 0x7f83480ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f834814e630 0x7f83480c27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 --2- 172.21.15.33:0/75869236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83480c4a50 0x7f83481398d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 -- 172.21.15.33:0/75869236 >> 172.21.15.33:0/75869236 conn(0x7f83480c8e60 msgr2=0x7f834813a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 -- 172.21.15.33:0/75869236 shutdown_connections 2023-11-06T21:59:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.348+0000 7f834dbff700 1 -- 172.21.15.33:0/75869236 wait complete. 2023-11-06T21:59:16.368 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T21:03:36", "first": "2023-11-06T21:04:00", "last": "2023-11-06T21:59:00", "last_pruned": "2023-11-06T21:59:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-11-06T21:59:16.368 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T21:03:36', 'first': '2023-11-06T21:04:00', 'last': '2023-11-06T21:59:00', 'last_pruned': '2023-11-06T21:59:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-11-06T21:59:16.368 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-11-06T21:59:16.369 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-11-06T21:59:16.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1 2023-11-06T21:59:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.568+0000 7f4da0bff700 1 Processor -- start 2023-11-06T21:59:16.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.568+0000 7f4da0bff700 1 -- start start 2023-11-06T21:59:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.569+0000 7f4da0bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4d9c1542b0 0x7f4d9c0c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:16.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.569+0000 7f4da0bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c0c9f80 con 0x7f4d9c0c8ee0 2023-11-06T21:59:16.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.569+0000 7f4da0bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c0cab20 con 0x7f4d9c153ed0 2023-11-06T21:59:16.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.569+0000 7f4da0bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9c0c6000 con 0x7f4d9c1542b0 2023-11-06T21:59:16.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.569+0000 7f4d9ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4d9c1542b0 0x7f4d9c0c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:16.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.570+0000 7f4d9ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4d9c1542b0 0x7f4d9c0c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46618/0 (socket says 172.21.15.33:46618) 2023-11-06T21:59:16.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.570+0000 7f4d9ad9d700 1 -- 172.21.15.33:0/804373367 learned_addr learned my addr 172.21.15.33:0/804373367 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:16.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.570+0000 7f4d99d9b700 1 --1- 172.21.15.33:0/804373367 >> v1:172.21.15.33:6789/0 conn(0x7f4d9c0c8ee0 0x7f4d9c0ca410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38484/0 (socket says 172.21.15.33:38484) 2023-11-06T21:59:16.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.570+0000 7f4d9ad9d700 1 -- 172.21.15.33:0/804373367 >> v1:172.21.15.69:6789/0 conn(0x7f4d9c153ed0 legacy=0x7f4d9c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.570+0000 7f4d9ad9d700 1 -- 172.21.15.33:0/804373367 >> v1:172.21.15.33:6789/0 conn(0x7f4d9c0c8ee0 legacy=0x7f4d9c0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.570+0000 7f4d9ad9d700 1 -- 172.21.15.33:0/804373367 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c0c9370 con 0x7f4d9c1542b0 2023-11-06T21:59:16.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.570+0000 7f4d9959a700 1 -- 172.21.15.33:0/804373367 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388530846 0 0) 0x7f4d9c0c9f80 con 0x7f4d9c0c8ee0 2023-11-06T21:59:16.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.570+0000 7f4d9ad9d700 1 --2- 172.21.15.33:0/804373367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1542b0 0x7f4d9c0c76f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f4d94009f20 tx=0x7f4d94009840).ready entity=mon.0 client_cookie=3640443c917f69ef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:16.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.570+0000 7f4d9959a700 1 -- 172.21.15.33:0/804373367 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9400e070 con 0x7f4d9c1542b0 2023-11-06T21:59:16.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.570+0000 7f4d9959a700 1 -- 172.21.15.33:0/804373367 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9400ba80 con 0x7f4d9c1542b0 2023-11-06T21:59:16.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.571+0000 7f4d9959a700 1 -- 172.21.15.33:0/804373367 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9400be00 con 0x7f4d9c1542b0 2023-11-06T21:59:16.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.571+0000 7f4da0bff700 1 -- 172.21.15.33:0/804373367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1542b0 msgr2=0x7f4d9c0c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.571+0000 7f4da0bff700 1 --2- 172.21.15.33:0/804373367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1542b0 0x7f4d9c0c76f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f4d94009f20 tx=0x7f4d94009840).stop 2023-11-06T21:59:16.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.571+0000 7f4da0bff700 1 -- 172.21.15.33:0/804373367 shutdown_connections 2023-11-06T21:59:16.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.571+0000 7f4da0bff700 1 --2- 172.21.15.33:0/804373367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1542b0 0x7f4d9c0c76f0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.571+0000 7f4da0bff700 1 -- 172.21.15.33:0/804373367 >> 172.21.15.33:0/804373367 conn(0x7f4d9c05e1b0 msgr2=0x7f4d9c05e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:16.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.571+0000 7f4da0bff700 1 -- 172.21.15.33:0/804373367 shutdown_connections 2023-11-06T21:59:16.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.571+0000 7f4da0bff700 1 -- 172.21.15.33:0/804373367 wait complete. 2023-11-06T21:59:16.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.573+0000 7f4da0bff700 1 Processor -- start 2023-11-06T21:59:16.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.573+0000 7f4da0bff700 1 -- start start 2023-11-06T21:59:16.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4da0bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d9c153ed0 0x7f4d9c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:16.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4da0bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d9c1542b0 0x7f4d9c14cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:16.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4da0bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1677e0 0x7f4d9c158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:16.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4da0bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c0af0c0 con 0x7f4d9c1677e0 2023-11-06T21:59:16.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4da0bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c14d1f0 con 0x7f4d9c1542b0 2023-11-06T21:59:16.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4da0bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c15ace0 con 0x7f4d9c153ed0 2023-11-06T21:59:16.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d9ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1677e0 0x7f4d9c158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:16.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d9ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1677e0 0x7f4d9c158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46626/0 (socket says 172.21.15.33:46626) 2023-11-06T21:59:16.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d9ad9d700 1 -- 172.21.15.33:0/1414205684 learned_addr learned my addr 172.21.15.33:0/1414205684 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:16.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d99d9b700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d9c1542b0 0x7f4d9c14cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:16.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d9a59c700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d9c153ed0 0x7f4d9c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:16.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d9ad9d700 1 -- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d9c153ed0 msgr2=0x7f4d9c0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d9ad9d700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d9c153ed0 0x7f4d9c0ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d9ad9d700 1 -- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d9c1542b0 msgr2=0x7f4d9c14cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d9ad9d700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d9c1542b0 0x7f4d9c14cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d9ad9d700 1 -- 172.21.15.33:0/1414205684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d94009ae0 con 0x7f4d9c1677e0 2023-11-06T21:59:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d9a59c700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d9c153ed0 0x7f4d9c0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.574+0000 7f4d99d9b700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d9c1542b0 0x7f4d9c14cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.575+0000 7f4d9ad9d700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1677e0 0x7f4d9c158da0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f4d94010000 tx=0x7f4d94009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:16.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.575+0000 7f4d8a7fc700 1 -- 172.21.15.33:0/1414205684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9401d410 con 0x7f4d9c1677e0 2023-11-06T21:59:16.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.575+0000 7f4d8a7fc700 1 -- 172.21.15.33:0/1414205684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9401d570 con 0x7f4d9c1677e0 2023-11-06T21:59:16.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.575+0000 7f4da0bff700 1 -- 172.21.15.33:0/1414205684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c0ae820 con 0x7f4d9c1677e0 2023-11-06T21:59:16.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.575+0000 7f4d8a7fc700 1 -- 172.21.15.33:0/1414205684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9401d910 con 0x7f4d9c1677e0 2023-11-06T21:59:16.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.575+0000 7f4da0bff700 1 -- 172.21.15.33:0/1414205684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c0aede0 con 0x7f4d9c1677e0 2023-11-06T21:59:16.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.576+0000 7f4da0bff700 1 -- 172.21.15.33:0/1414205684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9c06c6c0 con 0x7f4d9c1677e0 2023-11-06T21:59:16.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.577+0000 7f4d8a7fc700 1 -- 172.21.15.33:0/1414205684 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 61) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f4d9401dd00 con 0x7f4d9c1677e0 2023-11-06T21:59:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.578+0000 7f4d8a7fc700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d7c05f2c0 0x7f4d7c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.578+0000 7f4d8a7fc700 1 -- 172.21.15.33:0/1414205684 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4d9407c2c0 con 0x7f4d9c1677e0 2023-11-06T21:59:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.578+0000 7f4d9a59c700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d7c05f2c0 0x7f4d7c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.579+0000 7f4d9a59c700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d7c05f2c0 0x7f4d7c061780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f4d8c001710 tx=0x7f4d8c009040).ready entity=mgr.7864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.582+0000 7f4d8a7fc700 1 -- 172.21.15.33:0/1414205684 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4d9404d200 con 0x7f4d9c1677e0 2023-11-06T21:59:16.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.790+0000 7f4da0bff700 1 -- 172.21.15.33:0/1414205684 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d9c06abf0 con 0x7f4d7c05f2c0 2023-11-06T21:59:16.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.795+0000 7f4d8a7fc700 1 -- 172.21.15.33:0/1414205684 <== mgr.7864 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7f4d9c06abf0 con 0x7f4d7c05f2c0 2023-11-06T21:59:16.798 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1 2023-11-06T21:59:16.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 -- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d7c05f2c0 msgr2=0x7f4d7c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d7c05f2c0 0x7f4d7c061780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f4d8c001710 tx=0x7f4d8c009040).stop 2023-11-06T21:59:16.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 -- 172.21.15.33:0/1414205684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1677e0 msgr2=0x7f4d9c158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1677e0 0x7f4d9c158da0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f4d94010000 tx=0x7f4d94009840).stop 2023-11-06T21:59:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 -- 172.21.15.33:0/1414205684 shutdown_connections 2023-11-06T21:59:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4d7c05f2c0 0x7f4d7c061780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4d9c153ed0 0x7f4d9c0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d9c1677e0 0x7f4d9c158da0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 --2- 172.21.15.33:0/1414205684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d9c1542b0 0x7f4d9c14cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 -- 172.21.15.33:0/1414205684 >> 172.21.15.33:0/1414205684 conn(0x7f4d9c05e1b0 msgr2=0x7f4d9c0aba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 -- 172.21.15.33:0/1414205684 shutdown_connections 2023-11-06T21:59:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:16.798+0000 7f4da0bff700 1 -- 172.21.15.33:0/1414205684 wait complete. 2023-11-06T21:59:16.817 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-11-06T21:59:16.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:16.818 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T21:59:16.868 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_05_00_UTC 2023-11-06T21:59:16.869 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_06_00_UTC 2023-11-06T21:59:16.869 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_07_00_UTC 2023-11-06T21:59:16.869 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_08_00_UTC 2023-11-06T21:59:16.869 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_09_00_UTC 2023-11-06T21:59:16.870 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_10_00_UTC 2023-11-06T21:59:16.870 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_11_00_UTC 2023-11-06T21:59:16.870 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_12_00_UTC 2023-11-06T21:59:16.870 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_13_00_UTC 2023-11-06T21:59:16.870 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_14_00_UTC 2023-11-06T21:59:16.871 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_15_00_UTC 2023-11-06T21:59:16.871 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_16_00_UTC 2023-11-06T21:59:16.871 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_17_00_UTC 2023-11-06T21:59:16.871 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_18_00_UTC 2023-11-06T21:59:16.871 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_19_00_UTC 2023-11-06T21:59:16.871 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_20_00_UTC 2023-11-06T21:59:16.872 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_21_00_UTC 2023-11-06T21:59:16.872 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_22_00_UTC 2023-11-06T21:59:16.872 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_23_00_UTC 2023-11-06T21:59:16.872 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_24_00_UTC 2023-11-06T21:59:16.872 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_25_00_UTC 2023-11-06T21:59:16.873 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_26_00_UTC 2023-11-06T21:59:16.873 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_27_00_UTC 2023-11-06T21:59:16.873 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_28_00_UTC 2023-11-06T21:59:16.873 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_29_00_UTC 2023-11-06T21:59:16.873 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_30_00_UTC 2023-11-06T21:59:16.874 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_31_00_UTC 2023-11-06T21:59:16.874 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_32_00_UTC 2023-11-06T21:59:16.874 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_33_00_UTC 2023-11-06T21:59:16.874 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_34_00_UTC 2023-11-06T21:59:16.874 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_35_00_UTC 2023-11-06T21:59:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_36_00_UTC 2023-11-06T21:59:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_37_00_UTC 2023-11-06T21:59:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_38_00_UTC 2023-11-06T21:59:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_39_00_UTC 2023-11-06T21:59:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_40_00_UTC 2023-11-06T21:59:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_41_00_UTC 2023-11-06T21:59:16.876 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_42_00_UTC 2023-11-06T21:59:16.876 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_43_00_UTC 2023-11-06T21:59:16.876 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_44_00_UTC 2023-11-06T21:59:16.876 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_45_00_UTC 2023-11-06T21:59:16.876 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_46_00_UTC 2023-11-06T21:59:16.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_47_00_UTC 2023-11-06T21:59:16.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_48_00_UTC 2023-11-06T21:59:16.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_49_00_UTC 2023-11-06T21:59:16.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_50_00_UTC 2023-11-06T21:59:16.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_51_00_UTC 2023-11-06T21:59:16.878 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_52_00_UTC 2023-11-06T21:59:16.878 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_53_00_UTC 2023-11-06T21:59:16.878 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_54_00_UTC 2023-11-06T21:59:16.878 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_55_00_UTC 2023-11-06T21:59:16.878 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_56_00_UTC 2023-11-06T21:59:16.879 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_57_00_UTC 2023-11-06T21:59:16.879 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_58_00_UTC 2023-11-06T21:59:16.879 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-21_59_00_UTC 2023-11-06T21:59:16.880 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_05_00_UTC 2023-11-06T21:59:16.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:16.881 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_05_00_UTC) 2023-11-06T21:59:16.936 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_06_00_UTC 2023-11-06T21:59:16.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:16.937 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_06_00_UTC) 2023-11-06T21:59:16.992 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_07_00_UTC 2023-11-06T21:59:16.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:16.992 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_07_00_UTC) 2023-11-06T21:59:17.049 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_08_00_UTC 2023-11-06T21:59:17.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.050 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_08_00_UTC) 2023-11-06T21:59:17.105 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_09_00_UTC 2023-11-06T21:59:17.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.106 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_09_00_UTC) 2023-11-06T21:59:17.161 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_10_00_UTC 2023-11-06T21:59:17.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.161 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_10_00_UTC) 2023-11-06T21:59:17.216 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_11_00_UTC 2023-11-06T21:59:17.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.217 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_11_00_UTC) 2023-11-06T21:59:17.272 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_12_00_UTC 2023-11-06T21:59:17.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.273 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_12_00_UTC) 2023-11-06T21:59:17.328 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_13_00_UTC 2023-11-06T21:59:17.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.329 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_13_00_UTC) 2023-11-06T21:59:17.386 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_14_00_UTC 2023-11-06T21:59:17.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.387 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_14_00_UTC) 2023-11-06T21:59:17.443 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_15_00_UTC 2023-11-06T21:59:17.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.444 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_15_00_UTC) 2023-11-06T21:59:17.500 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_16_00_UTC 2023-11-06T21:59:17.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.501 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_16_00_UTC) 2023-11-06T21:59:17.555 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_17_00_UTC 2023-11-06T21:59:17.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.556 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_17_00_UTC) 2023-11-06T21:59:17.611 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_18_00_UTC 2023-11-06T21:59:17.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.612 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_18_00_UTC) 2023-11-06T21:59:17.667 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_19_00_UTC 2023-11-06T21:59:17.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.668 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_19_00_UTC) 2023-11-06T21:59:17.724 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_20_00_UTC 2023-11-06T21:59:17.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.725 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_20_00_UTC) 2023-11-06T21:59:17.782 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_21_00_UTC 2023-11-06T21:59:17.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.783 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_21_00_UTC) 2023-11-06T21:59:17.838 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_22_00_UTC 2023-11-06T21:59:17.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.839 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_22_00_UTC) 2023-11-06T21:59:17.895 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_23_00_UTC 2023-11-06T21:59:17.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.895 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_23_00_UTC) 2023-11-06T21:59:17.954 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_24_00_UTC 2023-11-06T21:59:17.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:17.955 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_24_00_UTC) 2023-11-06T21:59:18.028 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_25_00_UTC 2023-11-06T21:59:18.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.029 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_25_00_UTC) 2023-11-06T21:59:18.089 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_26_00_UTC 2023-11-06T21:59:18.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.090 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_26_00_UTC) 2023-11-06T21:59:18.147 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_27_00_UTC 2023-11-06T21:59:18.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.148 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_27_00_UTC) 2023-11-06T21:59:18.203 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_28_00_UTC 2023-11-06T21:59:18.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.204 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_28_00_UTC) 2023-11-06T21:59:18.259 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_29_00_UTC 2023-11-06T21:59:18.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.260 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_29_00_UTC) 2023-11-06T21:59:18.315 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_30_00_UTC 2023-11-06T21:59:18.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.316 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_30_00_UTC) 2023-11-06T21:59:18.370 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_31_00_UTC 2023-11-06T21:59:18.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.371 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_31_00_UTC) 2023-11-06T21:59:18.429 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_32_00_UTC 2023-11-06T21:59:18.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.431 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_32_00_UTC) 2023-11-06T21:59:18.486 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_33_00_UTC 2023-11-06T21:59:18.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.487 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_33_00_UTC) 2023-11-06T21:59:18.541 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_34_00_UTC 2023-11-06T21:59:18.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.542 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_34_00_UTC) 2023-11-06T21:59:18.597 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_35_00_UTC 2023-11-06T21:59:18.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.598 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_35_00_UTC) 2023-11-06T21:59:18.653 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_36_00_UTC 2023-11-06T21:59:18.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.654 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_36_00_UTC) 2023-11-06T21:59:18.709 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_37_00_UTC 2023-11-06T21:59:18.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.710 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_37_00_UTC) 2023-11-06T21:59:18.764 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_38_00_UTC 2023-11-06T21:59:18.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.765 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_38_00_UTC) 2023-11-06T21:59:18.820 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_39_00_UTC 2023-11-06T21:59:18.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.821 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_39_00_UTC) 2023-11-06T21:59:18.875 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_40_00_UTC 2023-11-06T21:59:18.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.876 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_40_00_UTC) 2023-11-06T21:59:18.930 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_41_00_UTC 2023-11-06T21:59:18.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:18.931 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_41_00_UTC) 2023-11-06T21:59:19.026 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_42_00_UTC 2023-11-06T21:59:19.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.027 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_42_00_UTC) 2023-11-06T21:59:19.148 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_43_00_UTC 2023-11-06T21:59:19.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.149 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_43_00_UTC) 2023-11-06T21:59:19.203 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_44_00_UTC 2023-11-06T21:59:19.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.204 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_44_00_UTC) 2023-11-06T21:59:19.260 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_45_00_UTC 2023-11-06T21:59:19.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.261 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_45_00_UTC) 2023-11-06T21:59:19.315 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_46_00_UTC 2023-11-06T21:59:19.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.316 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_46_00_UTC) 2023-11-06T21:59:19.370 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_47_00_UTC 2023-11-06T21:59:19.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.371 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_47_00_UTC) 2023-11-06T21:59:19.426 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_48_00_UTC 2023-11-06T21:59:19.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.432 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_48_00_UTC) 2023-11-06T21:59:19.485 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_49_00_UTC 2023-11-06T21:59:19.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.487 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_49_00_UTC) 2023-11-06T21:59:19.541 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_50_00_UTC 2023-11-06T21:59:19.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.542 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_50_00_UTC) 2023-11-06T21:59:19.596 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_51_00_UTC 2023-11-06T21:59:19.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.597 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_51_00_UTC) 2023-11-06T21:59:19.650 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_52_00_UTC 2023-11-06T21:59:19.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.651 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_52_00_UTC) 2023-11-06T21:59:19.705 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_53_00_UTC 2023-11-06T21:59:19.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.706 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_53_00_UTC) 2023-11-06T21:59:19.760 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_54_00_UTC 2023-11-06T21:59:19.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.761 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_54_00_UTC) 2023-11-06T21:59:19.815 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_55_00_UTC 2023-11-06T21:59:19.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.817 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_55_00_UTC) 2023-11-06T21:59:19.872 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_56_00_UTC 2023-11-06T21:59:19.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.872 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_56_00_UTC) 2023-11-06T21:59:19.926 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_57_00_UTC 2023-11-06T21:59:19.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:19.927 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_57_00_UTC) 2023-11-06T21:59:20.006 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_58_00_UTC 2023-11-06T21:59:20.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:20.007 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_58_00_UTC) 2023-11-06T21:59:20.108 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-21_59_00_UTC 2023-11-06T21:59:20.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:20.109 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-21_59_00_UTC) 2023-11-06T21:59:20.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:20.186 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-06T21:59:20.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T21:59:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.432+0000 7f030fbff700 1 Processor -- start 2023-11-06T21:59:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.432+0000 7f030fbff700 1 -- start start 2023-11-06T21:59:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.433+0000 7f030fbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0308154720 0x7f0308063150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.433+0000 7f030fbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0308063640 0x7f03080c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.433+0000 7f030fbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0308158ed0 con 0x7f0308154340 2023-11-06T21:59:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.433+0000 7f030fbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03081580b0 con 0x7f0308063640 2023-11-06T21:59:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.433+0000 7f030fbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0308063a90 con 0x7f0308154720 2023-11-06T21:59:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.434+0000 7f030e19c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0308063640 0x7f03080c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.434+0000 7f030e19c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0308063640 0x7f03080c5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35742/0 (socket says 172.21.15.33:35742) 2023-11-06T21:59:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.434+0000 7f030e19c700 1 -- 172.21.15.33:0/1684621131 learned_addr learned my addr 172.21.15.33:0/1684621131 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.434+0000 7f030d19a700 1 --2- 172.21.15.33:0/1684621131 >> v2:172.21.15.69:3300/0 conn(0x7f0308154720 0x7f0308063150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.434+0000 7f030e19c700 1 -- 172.21.15.33:0/1684621131 >> v2:172.21.15.69:3300/0 conn(0x7f0308154720 msgr2=0x7f0308063150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.434+0000 7f030e19c700 1 --2- 172.21.15.33:0/1684621131 >> v2:172.21.15.69:3300/0 conn(0x7f0308154720 0x7f0308063150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.434+0000 7f030e19c700 1 -- 172.21.15.33:0/1684621131 >> v1:172.21.15.69:6789/0 conn(0x7f0308154340 legacy=0x7f03080a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.434+0000 7f030e19c700 1 -- 172.21.15.33:0/1684621131 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03081582c0 con 0x7f0308063640 2023-11-06T21:59:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.434+0000 7f030e19c700 1 --2- 172.21.15.33:0/1684621131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308063640 0x7f03080c5d80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f03080c5420 tx=0x7f0300009840).ready entity=mon.0 client_cookie=b2338e0364959b7e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:20.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030c999700 1 -- 172.21.15.33:0/1684621131 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030000e070 con 0x7f0308063640 2023-11-06T21:59:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030c999700 1 -- 172.21.15.33:0/1684621131 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030000ba10 con 0x7f0308063640 2023-11-06T21:59:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030c999700 1 -- 172.21.15.33:0/1684621131 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030000bd90 con 0x7f0308063640 2023-11-06T21:59:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030fbff700 1 -- 172.21.15.33:0/1684621131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308063640 msgr2=0x7f03080c5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030fbff700 1 --2- 172.21.15.33:0/1684621131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308063640 0x7f03080c5d80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f03080c5420 tx=0x7f0300009840).stop 2023-11-06T21:59:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030fbff700 1 -- 172.21.15.33:0/1684621131 shutdown_connections 2023-11-06T21:59:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030fbff700 1 --2- 172.21.15.33:0/1684621131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308063640 0x7f03080c5d80 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030fbff700 1 --2- 172.21.15.33:0/1684621131 >> v2:172.21.15.69:3300/0 conn(0x7f0308154720 0x7f0308063150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030fbff700 1 -- 172.21.15.33:0/1684621131 >> 172.21.15.33:0/1684621131 conn(0x7f030805ec80 msgr2=0x7f03080cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030fbff700 1 -- 172.21.15.33:0/1684621131 shutdown_connections 2023-11-06T21:59:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.435+0000 7f030fbff700 1 -- 172.21.15.33:0/1684621131 wait complete. 2023-11-06T21:59:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.437+0000 7f030fbff700 1 Processor -- start 2023-11-06T21:59:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.437+0000 7f030fbff700 1 -- start start 2023-11-06T21:59:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030fbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0308063640 0x7f0308156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030fbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308154340 0x7f03081522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030fbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0308154720 0x7f03080c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:20.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030fbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03080af130 con 0x7f0308154340 2023-11-06T21:59:20.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030fbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030814d8a0 con 0x7f0308063640 2023-11-06T21:59:20.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030fbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03080ae290 con 0x7f0308154720 2023-11-06T21:59:20.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030d19a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308154340 0x7f03081522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:20.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030d19a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308154340 0x7f03081522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35748/0 (socket says 172.21.15.33:35748) 2023-11-06T21:59:20.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030d19a700 1 -- 172.21.15.33:0/2271018651 learned_addr learned my addr 172.21.15.33:0/2271018651 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:20.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030d99b700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0308063640 0x7f0308156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.438+0000 7f030e19c700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0308154720 0x7f03080c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f030d19a700 1 -- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0308154720 msgr2=0x7f03080c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f030d19a700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0308154720 0x7f03080c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f030d19a700 1 -- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0308063640 msgr2=0x7f0308156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f030d19a700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0308063640 0x7f0308156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f030d19a700 1 -- 172.21.15.33:0/2271018651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0300009ae0 con 0x7f0308154340 2023-11-06T21:59:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f030d99b700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0308063640 0x7f0308156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f030e19c700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0308154720 0x7f03080c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f030d19a700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308154340 0x7f03081522e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f02f800d860 tx=0x7f02f8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f02fdffb700 1 -- 172.21.15.33:0/2271018651 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f80125f0 con 0x7f0308154340 2023-11-06T21:59:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f02fdffb700 1 -- 172.21.15.33:0/2271018651 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f8012750 con 0x7f0308154340 2023-11-06T21:59:20.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f030fbff700 1 -- 172.21.15.33:0/2271018651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03080ae890 con 0x7f0308154340 2023-11-06T21:59:20.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f02fdffb700 1 -- 172.21.15.33:0/2271018651 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8012ad0 con 0x7f0308154340 2023-11-06T21:59:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.439+0000 7f030fbff700 1 -- 172.21.15.33:0/2271018651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03080aec20 con 0x7f0308154340 2023-11-06T21:59:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.440+0000 7f030fbff700 1 -- 172.21.15.33:0/2271018651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0308157460 con 0x7f0308154340 2023-11-06T21:59:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.441+0000 7f02fdffb700 1 -- 172.21.15.33:0/2271018651 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 61) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f02f8013480 con 0x7f0308154340 2023-11-06T21:59:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.442+0000 7f02fdffb700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f02ec05f2c0 0x7f02ec061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.442+0000 7f02fdffb700 1 -- 172.21.15.33:0/2271018651 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f02f807a400 con 0x7f0308154340 2023-11-06T21:59:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.442+0000 7f030d99b700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f02ec05f2c0 0x7f02ec061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.443+0000 7f030d99b700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f02ec05f2c0 0x7f02ec061780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f030400b8b0 tx=0x7f0304008040).ready entity=mgr.7864 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.446+0000 7f02fdffb700 1 -- 172.21.15.33:0/2271018651 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f02f804b390 con 0x7f0308154340 2023-11-06T21:59:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.674+0000 7f030fbff700 1 -- 172.21.15.33:0/2271018651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f03080cba20 con 0x7f0308154340 2023-11-06T21:59:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.966+0000 7f02fdffb700 1 -- 172.21.15.33:0/2271018651 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f02f807a0d0 con 0x7f0308154340 2023-11-06T21:59:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.967+0000 7f02fdffb700 1 -- 172.21.15.33:0/2271018651 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v62) v1 ==== 93+0+0 (secure 0 0 0) 0x7f02f8072c10 con 0x7f0308154340 2023-11-06T21:59:20.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 -- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f02ec05f2c0 msgr2=0x7f02ec061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:20.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f02ec05f2c0 0x7f02ec061780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f030400b8b0 tx=0x7f0304008040).stop 2023-11-06T21:59:20.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 -- 172.21.15.33:0/2271018651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308154340 msgr2=0x7f03081522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:20.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308154340 0x7f03081522e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f02f800d860 tx=0x7f02f8008040).stop 2023-11-06T21:59:20.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 -- 172.21.15.33:0/2271018651 shutdown_connections 2023-11-06T21:59:20.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f02ec05f2c0 0x7f02ec061780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:20.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0308154720 0x7f03080c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:20.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0308063640 0x7f0308156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:20.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 --2- 172.21.15.33:0/2271018651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0308154340 0x7f03081522e0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:20.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 -- 172.21.15.33:0/2271018651 >> 172.21.15.33:0/2271018651 conn(0x7f030805ec80 msgr2=0x7f0308152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:20.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 -- 172.21.15.33:0/2271018651 shutdown_connections 2023-11-06T21:59:20.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:20.971+0000 7f030fbff700 1 -- 172.21.15.33:0/2271018651 wait complete. 2023-11-06T21:59:21.002 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T21:59:21.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:59:21.153 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:59:21.243 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T21:59:21.244 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T21:59:21.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:21.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T21:59:21.273 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:21.270+0000 7f6401601680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:59:21.277 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:21.275+0000 7f216124c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:59:21.362 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T21:59:21.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:59:21.363 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T21:59:21.364 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:21.362+0000 7f216124c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:59:21.387 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:21.385+0000 7f6401601680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:59:21.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:59:21.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.641+0000 7fd5e2fff700 1 Processor -- start 2023-11-06T21:59:21.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.641+0000 7fd5e2fff700 1 -- start start 2023-11-06T21:59:21.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.643+0000 7fd5e2fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5dc151dc0 0x7fd5dc0c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:21.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.643+0000 7fd5e2fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5dc1521a0 0x7fd5dc063110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:21.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.643+0000 7fd5e2fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5dc154e90 con 0x7fd5dc155320 2023-11-06T21:59:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.643+0000 7fd5e2fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5dc0639d0 con 0x7fd5dc151dc0 2023-11-06T21:59:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.643+0000 7fd5e2fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc063cb0 con 0x7fd5dc1521a0 2023-11-06T21:59:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.643+0000 7fd5e0d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5dc151dc0 0x7fd5dc0c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.643+0000 7fd5e0d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5dc151dc0 0x7fd5dc0c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35770/0 (socket says 172.21.15.33:35770) 2023-11-06T21:59:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.643+0000 7fd5e0d9b700 1 -- 172.21.15.33:0/937788177 learned_addr learned my addr 172.21.15.33:0/937788177 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:21.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.643+0000 7fd5e159c700 1 --1- 172.21.15.33:0/937788177 >> v1:172.21.15.33:6789/0 conn(0x7fd5dc155320 0x7fd5dc0c7850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53206/0 (socket says 172.21.15.33:53206) 2023-11-06T21:59:21.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.643+0000 7fd5dbfff700 1 --2- 172.21.15.33:0/937788177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5dc1521a0 0x7fd5dc063110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.644+0000 7fd5e0d9b700 1 -- 172.21.15.33:0/937788177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5dc1521a0 msgr2=0x7fd5dc063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.644+0000 7fd5e0d9b700 1 --2- 172.21.15.33:0/937788177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5dc1521a0 0x7fd5dc063110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.644+0000 7fd5e0d9b700 1 -- 172.21.15.33:0/937788177 >> v1:172.21.15.33:6789/0 conn(0x7fd5dc155320 legacy=0x7fd5dc0c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.644+0000 7fd5db7fe700 1 -- 172.21.15.33:0/937788177 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658070902 0 0) 0x7fd5dc154e90 con 0x7fd5dc155320 2023-11-06T21:59:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.644+0000 7fd5e0d9b700 1 -- 172.21.15.33:0/937788177 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc063f90 con 0x7fd5dc151dc0 2023-11-06T21:59:21.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.644+0000 7fd5e0d9b700 1 --2- 172.21.15.33:0/937788177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc151dc0 0x7fd5dc0c0c50 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7fd5d0006fd0 tx=0x7fd5d00090f0).ready entity=mon.0 client_cookie=3ee55a768b97cf37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:21.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.644+0000 7fd5db7fe700 1 -- 172.21.15.33:0/937788177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000b580 con 0x7fd5dc151dc0 2023-11-06T21:59:21.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.644+0000 7fd5db7fe700 1 -- 172.21.15.33:0/937788177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d000b6e0 con 0x7fd5dc151dc0 2023-11-06T21:59:21.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.644+0000 7fd5db7fe700 1 -- 172.21.15.33:0/937788177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000ba60 con 0x7fd5dc151dc0 2023-11-06T21:59:21.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.645+0000 7fd5e2fff700 1 -- 172.21.15.33:0/937788177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc151dc0 msgr2=0x7fd5dc0c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:21.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.645+0000 7fd5e2fff700 1 --2- 172.21.15.33:0/937788177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc151dc0 0x7fd5dc0c0c50 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7fd5d0006fd0 tx=0x7fd5d00090f0).stop 2023-11-06T21:59:21.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.645+0000 7fd5e2fff700 1 -- 172.21.15.33:0/937788177 shutdown_connections 2023-11-06T21:59:21.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.645+0000 7fd5e2fff700 1 --2- 172.21.15.33:0/937788177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc151dc0 0x7fd5dc0c0c50 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:21.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.645+0000 7fd5e2fff700 1 --2- 172.21.15.33:0/937788177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5dc1521a0 0x7fd5dc063110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:21.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.645+0000 7fd5e2fff700 1 -- 172.21.15.33:0/937788177 >> 172.21.15.33:0/937788177 conn(0x7fd5dc05e1b0 msgr2=0x7fd5dc05e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.645+0000 7fd5e2fff700 1 -- 172.21.15.33:0/937788177 shutdown_connections 2023-11-06T21:59:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.645+0000 7fd5e2fff700 1 -- 172.21.15.33:0/937788177 wait complete. 2023-11-06T21:59:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.646+0000 7fd5e2fff700 1 Processor -- start 2023-11-06T21:59:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.647+0000 7fd5e2fff700 1 -- start start 2023-11-06T21:59:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.647+0000 7fd5e2fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5dc151dc0 0x7fd5dc144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.647+0000 7fd5e2fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc1521a0 0x7fd5dc0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5e2fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5dc155320 0x7fd5dc0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5e2fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0c7a30 con 0x7fd5dc1521a0 2023-11-06T21:59:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5e2fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0c5d20 con 0x7fd5dc155320 2023-11-06T21:59:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5e2fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0c6000 con 0x7fd5dc151dc0 2023-11-06T21:59:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5dbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc1521a0 0x7fd5dc0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5dbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc1521a0 0x7fd5dc0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35776/0 (socket says 172.21.15.33:35776) 2023-11-06T21:59:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5dbfff700 1 -- 172.21.15.33:0/2360397883 learned_addr learned my addr 172.21.15.33:0/2360397883 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5e0d9b700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5dc151dc0 0x7fd5dc144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5e159c700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5dc155320 0x7fd5dc0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5dbfff700 1 -- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5dc151dc0 msgr2=0x7fd5dc144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5dbfff700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5dc151dc0 0x7fd5dc144ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5dbfff700 1 -- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5dc155320 msgr2=0x7fd5dc0c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5dbfff700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5dc155320 0x7fd5dc0c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5dbfff700 1 -- 172.21.15.33:0/2360397883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d0009370 con 0x7fd5dc1521a0 2023-11-06T21:59:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.648+0000 7fd5dbfff700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc1521a0 0x7fd5dc0c7cb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7fd5c800e930 tx=0x7fd5c8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.649+0000 7fd5d8ff9700 1 -- 172.21.15.33:0/2360397883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8012600 con 0x7fd5dc1521a0 2023-11-06T21:59:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.649+0000 7fd5d8ff9700 1 -- 172.21.15.33:0/2360397883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c8012760 con 0x7fd5dc1521a0 2023-11-06T21:59:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.649+0000 7fd5e2fff700 1 -- 172.21.15.33:0/2360397883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5dc0c62e0 con 0x7fd5dc1521a0 2023-11-06T21:59:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.649+0000 7fd5d8ff9700 1 -- 172.21.15.33:0/2360397883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8012ae0 con 0x7fd5dc1521a0 2023-11-06T21:59:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.649+0000 7fd5e2fff700 1 -- 172.21.15.33:0/2360397883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc0c68a0 con 0x7fd5dc1521a0 2023-11-06T21:59:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.650+0000 7fd5e2fff700 1 -- 172.21.15.33:0/2360397883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5dc06bc10 con 0x7fd5dc1521a0 2023-11-06T21:59:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.651+0000 7fd5d8ff9700 1 -- 172.21.15.33:0/2360397883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fd5c801b400 con 0x7fd5dc1521a0 2023-11-06T21:59:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.652+0000 7fd5d8ff9700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5c005f270 0x7fd5c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.652+0000 7fd5d8ff9700 1 -- 172.21.15.33:0/2360397883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd5c807a9a0 con 0x7fd5dc1521a0 2023-11-06T21:59:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.652+0000 7fd5e0d9b700 1 -- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5c005f270 msgr2=0x7fd5c0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.652+0000 7fd5e0d9b700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5c005f270 0x7fd5c0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.660+0000 7fd5d8ff9700 1 -- 172.21.15.33:0/2360397883 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd5c804b920 con 0x7fd5dc1521a0 2023-11-06T21:59:21.698 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:21.697+0000 7f216124c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:59:21.755 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:21.753+0000 7f6401601680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:59:21.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.853+0000 7fd5e0d9b700 1 -- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5c005f270 msgr2=0x7fd5c0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:21.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.853+0000 7fd5e0d9b700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5c005f270 0x7fd5c0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:59:21.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.886+0000 7fd5e2fff700 1 -- 172.21.15.33:0/2360397883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5dc154b20 con 0x7fd5dc1521a0 2023-11-06T21:59:21.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.886+0000 7fd5d8ff9700 1 -- 172.21.15.33:0/2360397883 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4491 (secure 0 0 0) 0x7fd5c804ef40 con 0x7fd5dc1521a0 2023-11-06T21:59:21.889 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:21.889 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1809147689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1809147689},{"type":"v1","addr":"172.21.15.33:6835","nonce":1809147689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1390207299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1390207299},{"type":"v1","addr":"172.21.15.69:6837","nonce":1390207299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3179984136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3179984136},{"type":"v1","addr":"172.21.15.69:6836","nonce":3179984136}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:03:21.683218+0000","modified":"2023-11-06T21:03:24.689443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7720":{"gid":7720,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/4126832799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4126832799},{"type":"v1","addr":"172.21.15.33:6834","nonce":4126832799}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-06T21:59:21.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 -- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5c005f270 msgr2=0x7fd5c0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:21.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5c005f270 0x7fd5c0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 -- 172.21.15.33:0/2360397883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc1521a0 msgr2=0x7fd5dc0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc1521a0 0x7fd5dc0c7cb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7fd5c800e930 tx=0x7fd5c8009060).stop 2023-11-06T21:59:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 -- 172.21.15.33:0/2360397883 shutdown_connections 2023-11-06T21:59:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5c005f270 0x7fd5c0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5dc151dc0 0x7fd5dc144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5dc155320 0x7fd5dc0c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 --2- 172.21.15.33:0/2360397883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5dc1521a0 0x7fd5dc0c7cb0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 -- 172.21.15.33:0/2360397883 >> 172.21.15.33:0/2360397883 conn(0x7fd5dc05e1b0 msgr2=0x7fd5dc152640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 -- 172.21.15.33:0/2360397883 shutdown_connections 2023-11-06T21:59:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:21.889+0000 7fd5e2fff700 1 -- 172.21.15.33:0/2360397883 wait complete. 2023-11-06T21:59:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 60 2023-11-06T21:59:21.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:59:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.138+0000 7f7b1e19d700 1 Processor -- start 2023-11-06T21:59:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.138+0000 7f7b1e19d700 1 -- start start 2023-11-06T21:59:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.139+0000 7f7b1e19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b18152170 0x7f7b180645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.139+0000 7f7b1e19d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18155630 0x7f7b18064ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.139+0000 7f7b1e19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b180c6900 con 0x7f7b18151d90 2023-11-06T21:59:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.139+0000 7f7b1e19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b18154960 con 0x7f7b18152170 2023-11-06T21:59:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.139+0000 7f7b1e19d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b18064fc0 con 0x7f7b18155630 2023-11-06T21:59:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.139+0000 7f7b1c99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b18152170 0x7f7b180645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.140+0000 7f7b1c99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b18152170 0x7f7b180645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35786/0 (socket says 172.21.15.33:35786) 2023-11-06T21:59:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.140+0000 7f7b1c99a700 1 -- 172.21.15.33:0/51550717 learned_addr learned my addr 172.21.15.33:0/51550717 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.140+0000 7f7b1d99c700 1 --2- 172.21.15.33:0/51550717 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18155630 0x7f7b18064ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.140+0000 7f7b1c99a700 1 -- 172.21.15.33:0/51550717 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18155630 msgr2=0x7f7b18064ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.140+0000 7f7b1c99a700 1 --2- 172.21.15.33:0/51550717 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18155630 0x7f7b18064ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.140+0000 7f7b1c99a700 1 -- 172.21.15.33:0/51550717 >> v1:172.21.15.33:6789/0 conn(0x7f7b18151d90 legacy=0x7f7b180c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.140+0000 7f7b1c99a700 1 -- 172.21.15.33:0/51550717 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b181551a0 con 0x7f7b18152170 2023-11-06T21:59:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.140+0000 7f7b1d99c700 1 --2- 172.21.15.33:0/51550717 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18155630 0x7f7b18064ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.140+0000 7f7b1c99a700 1 --2- 172.21.15.33:0/51550717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 0x7f7b180645e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f7b08006fd0 tx=0x7f7b080090f0).ready entity=mon.0 client_cookie=20b7dead0c1989de server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:22.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.140+0000 7f7b0ffff700 1 -- 172.21.15.33:0/51550717 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800b600 con 0x7f7b18152170 2023-11-06T21:59:22.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.141+0000 7f7b0ffff700 1 -- 172.21.15.33:0/51550717 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0800b760 con 0x7f7b18152170 2023-11-06T21:59:22.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.141+0000 7f7b0ffff700 1 -- 172.21.15.33:0/51550717 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800bae0 con 0x7f7b18152170 2023-11-06T21:59:22.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.141+0000 7f7b1e19d700 1 -- 172.21.15.33:0/51550717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 msgr2=0x7f7b180645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.141+0000 7f7b1e19d700 1 --2- 172.21.15.33:0/51550717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 0x7f7b180645e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f7b08006fd0 tx=0x7f7b080090f0).stop 2023-11-06T21:59:22.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.141+0000 7f7b1e19d700 1 -- 172.21.15.33:0/51550717 shutdown_connections 2023-11-06T21:59:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.141+0000 7f7b1e19d700 1 --2- 172.21.15.33:0/51550717 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18155630 0x7f7b18064ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.141+0000 7f7b1e19d700 1 --2- 172.21.15.33:0/51550717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 0x7f7b180645e0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.141+0000 7f7b1e19d700 1 -- 172.21.15.33:0/51550717 >> 172.21.15.33:0/51550717 conn(0x7f7b180cb780 msgr2=0x7f7b180c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.141+0000 7f7b1e19d700 1 -- 172.21.15.33:0/51550717 shutdown_connections 2023-11-06T21:59:22.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.141+0000 7f7b1e19d700 1 -- 172.21.15.33:0/51550717 wait complete. 2023-11-06T21:59:22.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.143+0000 7f7b1e19d700 1 Processor -- start 2023-11-06T21:59:22.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.143+0000 7f7b1e19d700 1 -- start start 2023-11-06T21:59:22.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1e19d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18151d90 0x7f7b181647e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1e19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 0x7f7b180b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1e19d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b18155630 0x7f7b180c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1e19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180b0d10 con 0x7f7b18152170 2023-11-06T21:59:22.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1e19d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180c3290 con 0x7f7b18155630 2023-11-06T21:59:22.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1e19d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180c34b0 con 0x7f7b18151d90 2023-11-06T21:59:22.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1c99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 0x7f7b180b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1c99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 0x7f7b180b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35802/0 (socket says 172.21.15.33:35802) 2023-11-06T21:59:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1c99a700 1 -- 172.21.15.33:0/2851261463 learned_addr learned my addr 172.21.15.33:0/2851261463 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1d19b700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18151d90 0x7f7b181647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.144+0000 7f7b1d99c700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b18155630 0x7f7b180c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b1c99a700 1 -- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18151d90 msgr2=0x7f7b181647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b1c99a700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18151d90 0x7f7b181647e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b1c99a700 1 -- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b18155630 msgr2=0x7f7b180c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b1c99a700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b18155630 0x7f7b180c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b1c99a700 1 -- 172.21.15.33:0/2851261463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08009370 con 0x7f7b18152170 2023-11-06T21:59:22.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b1c99a700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 0x7f7b180b0820 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f7b08011000 tx=0x7f7b08017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:22.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b0d7fa700 1 -- 172.21.15.33:0/2851261463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800bea0 con 0x7f7b18152170 2023-11-06T21:59:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b0d7fa700 1 -- 172.21.15.33:0/2851261463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0800b8d0 con 0x7f7b18152170 2023-11-06T21:59:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b0d7fa700 1 -- 172.21.15.33:0/2851261463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0801b660 con 0x7f7b18152170 2023-11-06T21:59:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b1e19d700 1 -- 172.21.15.33:0/2851261463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b180ae560 con 0x7f7b18152170 2023-11-06T21:59:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.145+0000 7f7b1e19d700 1 -- 172.21.15.33:0/2851261463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b180aeb20 con 0x7f7b18152170 2023-11-06T21:59:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.146+0000 7f7b1e19d700 1 -- 172.21.15.33:0/2851261463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b180c5070 con 0x7f7b18152170 2023-11-06T21:59:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.147+0000 7f7b0d7fa700 1 -- 172.21.15.33:0/2851261463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f7b0801b9d0 con 0x7f7b18152170 2023-11-06T21:59:22.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.148+0000 7f7b0d7fa700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7afc05f2a0 0x7f7afc061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.148+0000 7f7b0d7fa700 1 -- 172.21.15.33:0/2851261463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7b0807bde0 con 0x7f7b18152170 2023-11-06T21:59:22.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.149+0000 7f7b1d19b700 1 -- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7afc05f2a0 msgr2=0x7f7afc061760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:22.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.149+0000 7f7b1d19b700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7afc05f2a0 0x7f7afc061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:22.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.156+0000 7f7b0d7fa700 1 -- 172.21.15.33:0/2851261463 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7b0804cd20 con 0x7f7b18152170 2023-11-06T21:59:22.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.340+0000 7f7b1e19d700 1 -- 172.21.15.33:0/2851261463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b18060830 con 0x7f7b18152170 2023-11-06T21:59:22.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.342+0000 7f7b0d7fa700 1 -- 172.21.15.33:0/2851261463 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+17573 (secure 0 0 0) 0x7f7b0801f070 con 0x7f7b18152170 2023-11-06T21:59:22.345 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:22.345 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":156,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:59:21.974687+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:03:20.146564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T21:03:20.662302+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":156,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2083752177":"2023-11-07T21:03:35.888563+0000","172.21.15.69:6833/114912":"2023-11-07T21:03:35.888563+0000","172.21.15.69:6832/114912":"2023-11-07T21:03:35.888563+0000","172.21.15.69:0/47836582":"2023-11-07T21:03:35.888563+0000","172.21.15.69:0/3836508278":"2023-11-07T21:03:35.888563+0000","172.21.15.69:0/2121315625":"2023-11-07T21:03:35.888563+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":54,"length":4},{"begin":110,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:59:22.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 -- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7afc05f2a0 msgr2=0x7f7afc061760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:22.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7afc05f2a0 0x7f7afc061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:22.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 -- 172.21.15.33:0/2851261463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 msgr2=0x7f7b180b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 0x7f7b180b0820 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f7b08011000 tx=0x7f7b08017000).stop 2023-11-06T21:59:22.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 -- 172.21.15.33:0/2851261463 shutdown_connections 2023-11-06T21:59:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7afc05f2a0 0x7f7afc061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b18151d90 0x7f7b181647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b18155630 0x7f7b180c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 --2- 172.21.15.33:0/2851261463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b18152170 0x7f7b180b0820 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 -- 172.21.15.33:0/2851261463 >> 172.21.15.33:0/2851261463 conn(0x7f7b180cb780 msgr2=0x7f7b18153f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.344+0000 7f7b1e19d700 1 -- 172.21.15.33:0/2851261463 shutdown_connections 2023-11-06T21:59:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.345+0000 7f7b1e19d700 1 -- 172.21.15.33:0/2851261463 wait complete. 2023-11-06T21:59:22.366 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T21:03:20.146564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T21:03:20.662302+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 113, 'snap_epoch': 156, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T21:59:22.367 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T21:59:22.470 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:22.467+0000 7f216124c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:59:22.553 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:22.551+0000 7f216124c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:59:22.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.597+0000 7efc1c9fe700 1 Processor -- start 2023-11-06T21:59:22.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.597+0000 7efc1c9fe700 1 -- start start 2023-11-06T21:59:22.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.599+0000 7efc1c9fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc18062e20 0x7efc180c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.599+0000 7efc1c9fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc18158090 con 0x7efc180be3f0 2023-11-06T21:59:22.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.599+0000 7efc1c9fe700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc180c3d50 con 0x7efc180c3970 2023-11-06T21:59:22.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.599+0000 7efc1c9fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc180bd020 con 0x7efc18062e20 2023-11-06T21:59:22.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.599+0000 7efc1659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc18062e20 0x7efc180c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.599+0000 7efc1759e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efc180be3f0 0x7efc18154510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53224/0 (socket says 172.21.15.33:53224) 2023-11-06T21:59:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.599+0000 7efc1759e700 1 -- 172.21.15.33:0/3929776810 learned_addr learned my addr 172.21.15.33:0/3929776810 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.599+0000 7efc15d9b700 1 -- 172.21.15.33:0/3929776810 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155830985 0 0) 0x7efc18158090 con 0x7efc180be3f0 2023-11-06T21:59:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.600+0000 7efc15d9b700 1 -- 172.21.15.33:0/3929776810 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc000033f0 con 0x7efc180be3f0 2023-11-06T21:59:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.600+0000 7efc1659c700 1 -- 172.21.15.33:0/3929776810 >> v1:172.21.15.69:6789/0 conn(0x7efc180c3970 legacy=0x7efc18062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.600+0000 7efc1659c700 1 -- 172.21.15.33:0/3929776810 >> v1:172.21.15.33:6789/0 conn(0x7efc180be3f0 legacy=0x7efc18154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.600+0000 7efc1659c700 1 -- 172.21.15.33:0/3929776810 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc180bca80 con 0x7efc18062e20 2023-11-06T21:59:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.600+0000 7efc1659c700 1 --2- 172.21.15.33:0/3929776810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 0x7efc180c80b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7efc0c0096e0 tx=0x7efc0c009060).ready entity=mon.0 client_cookie=3f763a72d81682a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.600+0000 7efc15d9b700 1 -- 172.21.15.33:0/3929776810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c009130 con 0x7efc18062e20 2023-11-06T21:59:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.600+0000 7efc15d9b700 1 -- 172.21.15.33:0/3929776810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c00d4a0 con 0x7efc18062e20 2023-11-06T21:59:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.600+0000 7efc15d9b700 1 -- 172.21.15.33:0/3929776810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00d820 con 0x7efc18062e20 2023-11-06T21:59:22.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.601+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3929776810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 msgr2=0x7efc180c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.601+0000 7efc1c9fe700 1 --2- 172.21.15.33:0/3929776810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 0x7efc180c80b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7efc0c0096e0 tx=0x7efc0c009060).stop 2023-11-06T21:59:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.601+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3929776810 shutdown_connections 2023-11-06T21:59:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.601+0000 7efc1c9fe700 1 --2- 172.21.15.33:0/3929776810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 0x7efc180c80b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.601+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3929776810 >> 172.21.15.33:0/3929776810 conn(0x7efc180642f0 msgr2=0x7efc180c79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:22.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.601+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3929776810 shutdown_connections 2023-11-06T21:59:22.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.601+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3929776810 wait complete. 2023-11-06T21:59:22.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.602+0000 7efc1c9fe700 1 Processor -- start 2023-11-06T21:59:22.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.603+0000 7efc1c9fe700 1 -- start start 2023-11-06T21:59:22.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.603+0000 7efc1c9fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 0x7efc18149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.603+0000 7efc1c9fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc180be3f0 0x7efc1814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.603+0000 7efc1c9fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc180c3970 0x7efc18152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.603+0000 7efc1c9fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc18149c40 con 0x7efc18062e20 2023-11-06T21:59:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc1c9fe700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc180a4980 con 0x7efc180c3970 2023-11-06T21:59:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc1c9fe700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc180a4c20 con 0x7efc180be3f0 2023-11-06T21:59:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc16d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 0x7efc18149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc16d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 0x7efc18149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35808/0 (socket says 172.21.15.33:35808) 2023-11-06T21:59:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc16d9d700 1 -- 172.21.15.33:0/3403621405 learned_addr learned my addr 172.21.15.33:0/3403621405 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc1659c700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc180be3f0 0x7efc1814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc1759e700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc180c3970 0x7efc18152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:22.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc16d9d700 1 -- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc180be3f0 msgr2=0x7efc1814d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc16d9d700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc180be3f0 0x7efc1814d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc16d9d700 1 -- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc180c3970 msgr2=0x7efc18152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc16d9d700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc180c3970 0x7efc18152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc16d9d700 1 -- 172.21.15.33:0/3403621405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c0092a0 con 0x7efc18062e20 2023-11-06T21:59:22.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.604+0000 7efc16d9d700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 0x7efc18149750 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7efc0800c530 tx=0x7efc08007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:22.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.605+0000 7efbfeffd700 1 -- 172.21.15.33:0/3403621405 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0800f470 con 0x7efc18062e20 2023-11-06T21:59:22.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.605+0000 7efbfeffd700 1 -- 172.21.15.33:0/3403621405 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0800f5d0 con 0x7efc18062e20 2023-11-06T21:59:22.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.605+0000 7efbfeffd700 1 -- 172.21.15.33:0/3403621405 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0800f970 con 0x7efc18062e20 2023-11-06T21:59:22.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.605+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3403621405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc180a4f00 con 0x7efc18062e20 2023-11-06T21:59:22.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.605+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3403621405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc18164f60 con 0x7efc18062e20 2023-11-06T21:59:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.606+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3403621405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc180475a0 con 0x7efc18062e20 2023-11-06T21:59:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.607+0000 7efbfeffd700 1 -- 172.21.15.33:0/3403621405 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7efc08012400 con 0x7efc18062e20 2023-11-06T21:59:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.607+0000 7efbfeffd700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0405f270 0x7efc04061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.608+0000 7efbfeffd700 1 -- 172.21.15.33:0/3403621405 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efc080799c0 con 0x7efc18062e20 2023-11-06T21:59:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.608+0000 7efc1659c700 1 -- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0405f270 msgr2=0x7efc04061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:22.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.608+0000 7efc1659c700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0405f270 0x7efc04061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:22.616 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:22.610+0000 7f6401601680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:59:22.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.612+0000 7efbfeffd700 1 -- 172.21.15.33:0/3403621405 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efc08048af0 con 0x7efc18062e20 2023-11-06T21:59:22.701 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:22.699+0000 7f6401601680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:59:22.783 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:22.781+0000 7f216124c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:59:22.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.791+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3403621405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efc18152d30 con 0x7efc18062e20 2023-11-06T21:59:22.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.792+0000 7efbfeffd700 1 -- 172.21.15.33:0/3403621405 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v156) v1 ==== 110+0+10 (secure 0 0 0) 0x7efc0804a9f0 con 0x7efc18062e20 2023-11-06T21:59:22.794 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T21:59:22.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0405f270 msgr2=0x7efc04061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:22.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0405f270 0x7efc04061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:22.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3403621405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 msgr2=0x7efc18149750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:22.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 0x7efc18149750 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7efc0800c530 tx=0x7efc08007db0).stop 2023-11-06T21:59:22.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3403621405 shutdown_connections 2023-11-06T21:59:22.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7efc0405f270 0x7efc04061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:22.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc180c3970 0x7efc18152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc18062e20 0x7efc18149750 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 --2- 172.21.15.33:0/3403621405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efc180be3f0 0x7efc1814d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:22.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3403621405 >> 172.21.15.33:0/3403621405 conn(0x7efc180642f0 msgr2=0x7efc18065160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:22.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3403621405 shutdown_connections 2023-11-06T21:59:22.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:22.794+0000 7efc1c9fe700 1 -- 172.21.15.33:0/3403621405 wait complete. 2023-11-06T21:59:22.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T21:59:22.952 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:22.950+0000 7f6401601680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:59:23.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.015+0000 7f058f9ff700 1 Processor -- start 2023-11-06T21:59:23.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.015+0000 7f058f9ff700 1 -- start start 2023-11-06T21:59:23.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058f9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f058813dff0 0x7f0588157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058f9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 0x7f05880c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058f9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0588064bf0 con 0x7f05880c4720 2023-11-06T21:59:23.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058f9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058805e490 con 0x7f058813dff0 2023-11-06T21:59:23.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058f9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0588062e10 con 0x7f05880c4b00 2023-11-06T21:59:23.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058cf9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f058813dff0 0x7f0588157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058cf9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f058813dff0 0x7f0588157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35816/0 (socket says 172.21.15.33:35816) 2023-11-06T21:59:23.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058cf9a700 1 -- 172.21.15.33:0/2264644178 learned_addr learned my addr 172.21.15.33:0/2264644178 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:23.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058df9c700 1 --2- 172.21.15.33:0/2264644178 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 0x7f05880c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058cf9a700 1 -- 172.21.15.33:0/2264644178 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 msgr2=0x7f05880c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:23.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058cf9a700 1 --2- 172.21.15.33:0/2264644178 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 0x7f05880c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058cf9a700 1 -- 172.21.15.33:0/2264644178 >> v1:172.21.15.69:6789/0 conn(0x7f05880c4720 legacy=0x7f058813d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.017+0000 7f058cf9a700 1 -- 172.21.15.33:0/2264644178 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05880630f0 con 0x7f058813dff0 2023-11-06T21:59:23.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.018+0000 7f058cf9a700 1 --2- 172.21.15.33:0/2264644178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 0x7f0588157e20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f0578006fd0 tx=0x7f05780090f0).ready entity=mon.0 client_cookie=adce07392b4300f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:23.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.018+0000 7f057ffff700 1 -- 172.21.15.33:0/2264644178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057800b580 con 0x7f058813dff0 2023-11-06T21:59:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.018+0000 7f057ffff700 1 -- 172.21.15.33:0/2264644178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057800b6e0 con 0x7f058813dff0 2023-11-06T21:59:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.018+0000 7f057ffff700 1 -- 172.21.15.33:0/2264644178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057800ba60 con 0x7f058813dff0 2023-11-06T21:59:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.019+0000 7f058f9ff700 1 -- 172.21.15.33:0/2264644178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 msgr2=0x7f0588157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.019+0000 7f058f9ff700 1 --2- 172.21.15.33:0/2264644178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 0x7f0588157e20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f0578006fd0 tx=0x7f05780090f0).stop 2023-11-06T21:59:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.019+0000 7f058f9ff700 1 -- 172.21.15.33:0/2264644178 shutdown_connections 2023-11-06T21:59:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.019+0000 7f058f9ff700 1 --2- 172.21.15.33:0/2264644178 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 0x7f05880c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.019+0000 7f058f9ff700 1 --2- 172.21.15.33:0/2264644178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 0x7f0588157e20 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.019+0000 7f058f9ff700 1 -- 172.21.15.33:0/2264644178 >> 172.21.15.33:0/2264644178 conn(0x7f05880c7dc0 msgr2=0x7f05880c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.019+0000 7f058f9ff700 1 -- 172.21.15.33:0/2264644178 shutdown_connections 2023-11-06T21:59:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.019+0000 7f058f9ff700 1 -- 172.21.15.33:0/2264644178 wait complete. 2023-11-06T21:59:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.021+0000 7f058f9ff700 1 Processor -- start 2023-11-06T21:59:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.021+0000 7f058f9ff700 1 -- start start 2023-11-06T21:59:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058f9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05880c4720 0x7f058813aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058f9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 0x7f05881389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058f9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 0x7f0588136680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058f9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05881526f0 con 0x7f058813dff0 2023-11-06T21:59:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058f9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05880aafd0 con 0x7f05880c4720 2023-11-06T21:59:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058f9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05881370a0 con 0x7f05880c4b00 2023-11-06T21:59:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058df9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 0x7f0588136680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058cf9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 0x7f05881389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058df9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 0x7f0588136680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35820/0 (socket says 172.21.15.33:35820) 2023-11-06T21:59:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058df9c700 1 -- 172.21.15.33:0/3618748368 learned_addr learned my addr 172.21.15.33:0/3618748368 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.023+0000 7f058d79b700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05880c4720 0x7f058813aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f058df9c700 1 -- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 msgr2=0x7f05881389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f058df9c700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 0x7f05881389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f058df9c700 1 -- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05880c4720 msgr2=0x7f058813aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f058df9c700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05880c4720 0x7f058813aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f058df9c700 1 -- 172.21.15.33:0/3618748368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0578009370 con 0x7f058813dff0 2023-11-06T21:59:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f058cf9a700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 0x7f05881389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f058df9c700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 0x7f0588136680 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f0580009df0 tx=0x7f058000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f057d7fa700 1 -- 172.21.15.33:0/3618748368 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058000caa0 con 0x7f058813dff0 2023-11-06T21:59:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f057d7fa700 1 -- 172.21.15.33:0/3618748368 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058000f040 con 0x7f058813dff0 2023-11-06T21:59:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f057d7fa700 1 -- 172.21.15.33:0/3618748368 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058000ce10 con 0x7f058813dff0 2023-11-06T21:59:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f058f9ff700 1 -- 172.21.15.33:0/3618748368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05880aa6a0 con 0x7f058813dff0 2023-11-06T21:59:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.024+0000 7f058f9ff700 1 -- 172.21.15.33:0/3618748368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05880aac60 con 0x7f058813dff0 2023-11-06T21:59:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.025+0000 7f058f9ff700 1 -- 172.21.15.33:0/3618748368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058806ab80 con 0x7f058813dff0 2023-11-06T21:59:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.026+0000 7f057d7fa700 1 -- 172.21.15.33:0/3618748368 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f0580013610 con 0x7f058813dff0 2023-11-06T21:59:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.027+0000 7f057d7fa700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f056c05f270 0x7f056c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.028+0000 7f057d7fa700 1 -- 172.21.15.33:0/3618748368 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f058007acf0 con 0x7f058813dff0 2023-11-06T21:59:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.028+0000 7f058d79b700 1 -- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f056c05f270 msgr2=0x7f056c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.028+0000 7f058d79b700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f056c05f270 0x7f056c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.032+0000 7f057d7fa700 1 -- 172.21.15.33:0/3618748368 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f058004bc30 con 0x7f058813dff0 2023-11-06T21:59:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.209+0000 7f058f9ff700 1 -- 172.21.15.33:0/3618748368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05880ab160 con 0x7f058813dff0 2023-11-06T21:59:23.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.210+0000 7f057d7fa700 1 -- 172.21.15.33:0/3618748368 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v156) v1 ==== 92+0+10 (secure 0 0 0) 0x7f058004f250 con 0x7f058813dff0 2023-11-06T21:59:23.212 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T21:59:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 -- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f056c05f270 msgr2=0x7f056c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f056c05f270 0x7f056c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 -- 172.21.15.33:0/3618748368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 msgr2=0x7f0588136680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 0x7f0588136680 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f0580009df0 tx=0x7f058000b040).stop 2023-11-06T21:59:23.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 -- 172.21.15.33:0/3618748368 shutdown_connections 2023-11-06T21:59:23.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f056c05f270 0x7f056c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:23.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f058813dff0 0x7f0588136680 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05880c4720 0x7f058813aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 --2- 172.21.15.33:0/3618748368 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05880c4b00 0x7f05881389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 -- 172.21.15.33:0/3618748368 >> 172.21.15.33:0/3618748368 conn(0x7f05880c7dc0 msgr2=0x7f05880c3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:23.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 -- 172.21.15.33:0/3618748368 shutdown_connections 2023-11-06T21:59:23.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.212+0000 7f058f9ff700 1 -- 172.21.15.33:0/3618748368 wait complete. 2023-11-06T21:59:23.233 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T21:59:23.298 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:23.296+0000 7f216124c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:59:23.423 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:23.422+0000 7f216124c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:59:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.435+0000 7f4dda1ff700 1 Processor -- start 2023-11-06T21:59:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.435+0000 7f4dda1ff700 1 -- start start 2023-11-06T21:59:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.436+0000 7f4dda1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4dd4154710 0x7f4dd40c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.436+0000 7f4dda1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4dd40c7ff0 0x7f4dd40646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.436+0000 7f4dda1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd40c6b60 con 0x7f4dd4154330 2023-11-06T21:59:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.436+0000 7f4dda1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd40c7700 con 0x7f4dd40c7ff0 2023-11-06T21:59:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.436+0000 7f4dda1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd40c5cd0 con 0x7f4dd4154710 2023-11-06T21:59:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4dd40c7ff0 0x7f4dd40646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4dd4154330 0x7f4dd40c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53230/0 (socket says 172.21.15.33:53230) 2023-11-06T21:59:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4dd40c7ff0 0x7f4dd40646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35824/0 (socket says 172.21.15.33:35824) 2023-11-06T21:59:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd37fe700 1 -- 172.21.15.33:0/3974648787 learned_addr learned my addr 172.21.15.33:0/3974648787 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd2ffd700 1 --2- 172.21.15.33:0/3974648787 >> v2:172.21.15.69:3300/0 conn(0x7f4dd4154710 0x7f4dd40c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd3fff700 1 -- 172.21.15.33:0/3974648787 >> v2:172.21.15.69:3300/0 conn(0x7f4dd4154710 msgr2=0x7f4dd40c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:23.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd3fff700 1 --2- 172.21.15.33:0/3974648787 >> v2:172.21.15.69:3300/0 conn(0x7f4dd4154710 0x7f4dd40c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd3fff700 1 -- 172.21.15.33:0/3974648787 >> v1:172.21.15.33:6789/0 conn(0x7f4dd4154330 legacy=0x7f4dd40c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd3fff700 1 -- 172.21.15.33:0/3974648787 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd40c5f50 con 0x7f4dd40c7ff0 2023-11-06T21:59:23.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd27fc700 1 -- 172.21.15.33:0/3974648787 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266353460 0 0) 0x7f4dd40c6b60 con 0x7f4dd4154330 2023-11-06T21:59:23.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.437+0000 7f4dd3fff700 1 --2- 172.21.15.33:0/3974648787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd40c7ff0 0x7f4dd40646c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f4dd40c8d10 tx=0x7f4dcc009780).ready entity=mon.0 client_cookie=3ac5a705219ad7ad server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:23.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dd27fc700 1 -- 172.21.15.33:0/3974648787 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc00e070 con 0x7f4dd40c7ff0 2023-11-06T21:59:23.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dd27fc700 1 -- 172.21.15.33:0/3974648787 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc00bb60 con 0x7f4dd40c7ff0 2023-11-06T21:59:23.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dd27fc700 1 -- 172.21.15.33:0/3974648787 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc012410 con 0x7f4dd40c7ff0 2023-11-06T21:59:23.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dda1ff700 1 -- 172.21.15.33:0/3974648787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd40c7ff0 msgr2=0x7f4dd40646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dda1ff700 1 --2- 172.21.15.33:0/3974648787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd40c7ff0 0x7f4dd40646c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f4dd40c8d10 tx=0x7f4dcc009780).stop 2023-11-06T21:59:23.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dda1ff700 1 -- 172.21.15.33:0/3974648787 shutdown_connections 2023-11-06T21:59:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dda1ff700 1 --2- 172.21.15.33:0/3974648787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd40c7ff0 0x7f4dd40646c0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dda1ff700 1 --2- 172.21.15.33:0/3974648787 >> v2:172.21.15.69:3300/0 conn(0x7f4dd4154710 0x7f4dd40c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dda1ff700 1 -- 172.21.15.33:0/3974648787 >> 172.21.15.33:0/3974648787 conn(0x7f4dd40cb910 msgr2=0x7f4dd40c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dda1ff700 1 -- 172.21.15.33:0/3974648787 shutdown_connections 2023-11-06T21:59:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.438+0000 7f4dda1ff700 1 -- 172.21.15.33:0/3974648787 wait complete. 2023-11-06T21:59:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.440+0000 7f4dda1ff700 1 Processor -- start 2023-11-06T21:59:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.440+0000 7f4dda1ff700 1 -- start start 2023-11-06T21:59:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dda1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dd40c7ff0 0x7f4dd4164b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dda1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4dd4154330 0x7f4dd4168b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dda1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd4154710 0x7f4dd40aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dda1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4164ff0 con 0x7f4dd4154710 2023-11-06T21:59:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dda1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4165a80 con 0x7f4dd40c7ff0 2023-11-06T21:59:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dda1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4169000 con 0x7f4dd4154330 2023-11-06T21:59:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dd3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd4154710 0x7f4dd40aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dd3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd4154710 0x7f4dd40aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35840/0 (socket says 172.21.15.33:35840) 2023-11-06T21:59:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dd3fff700 1 -- 172.21.15.33:0/1582729786 learned_addr learned my addr 172.21.15.33:0/1582729786 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dd37fe700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dd40c7ff0 0x7f4dd4164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.441+0000 7f4dd2ffd700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4dd4154330 0x7f4dd4168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dd3fff700 1 -- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4dd4154330 msgr2=0x7f4dd4168b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dd3fff700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4dd4154330 0x7f4dd4168b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dd3fff700 1 -- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dd40c7ff0 msgr2=0x7f4dd4164b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dd3fff700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dd40c7ff0 0x7f4dd4164b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dd3fff700 1 -- 172.21.15.33:0/1582729786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc009a20 con 0x7f4dd4154710 2023-11-06T21:59:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dd37fe700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dd40c7ff0 0x7f4dd4164b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dd2ffd700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4dd4154330 0x7f4dd4168b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dd3fff700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd4154710 0x7f4dd40aef70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f4dd406c170 tx=0x7f4dcc009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dbb7fe700 1 -- 172.21.15.33:0/1582729786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc01d630 con 0x7f4dd4154710 2023-11-06T21:59:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dbb7fe700 1 -- 172.21.15.33:0/1582729786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc01d790 con 0x7f4dd4154710 2023-11-06T21:59:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dda1ff700 1 -- 172.21.15.33:0/1582729786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd41651e0 con 0x7f4dd4154710 2023-11-06T21:59:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dbb7fe700 1 -- 172.21.15.33:0/1582729786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc01db30 con 0x7f4dd4154710 2023-11-06T21:59:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.442+0000 7f4dda1ff700 1 -- 172.21.15.33:0/1582729786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd41657a0 con 0x7f4dd4154710 2023-11-06T21:59:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.444+0000 7f4dda1ff700 1 -- 172.21.15.33:0/1582729786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd40830b0 con 0x7f4dd4154710 2023-11-06T21:59:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.444+0000 7f4dbb7fe700 1 -- 172.21.15.33:0/1582729786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f4dcc0083a0 con 0x7f4dd4154710 2023-11-06T21:59:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.445+0000 7f4dbb7fe700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4dc005f270 0x7f4dc0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.445+0000 7f4dbb7fe700 1 -- 172.21.15.33:0/1582729786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4dcc07c490 con 0x7f4dd4154710 2023-11-06T21:59:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.445+0000 7f4dd37fe700 1 -- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4dc005f270 msgr2=0x7f4dc0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.445+0000 7f4dd37fe700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4dc005f270 0x7f4dc0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.449+0000 7f4dbb7fe700 1 -- 172.21.15.33:0/1582729786 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4dcc04d410 con 0x7f4dd4154710 2023-11-06T21:59:23.509 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:23.507+0000 7f6401601680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:59:23.609 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:23.608+0000 7f216124c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:59:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.634+0000 7f4dda1ff700 1 -- 172.21.15.33:0/1582729786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4dd4062b00 con 0x7f4dd4154710 2023-11-06T21:59:23.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.635+0000 7f4dbb7fe700 1 -- 172.21.15.33:0/1582729786 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4dcc050a30 con 0x7f4dd4154710 2023-11-06T21:59:23.637 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T21:59:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 -- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4dc005f270 msgr2=0x7f4dc0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4dc005f270 0x7f4dc0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 -- 172.21.15.33:0/1582729786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd4154710 msgr2=0x7f4dd40aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd4154710 0x7f4dd40aef70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f4dd406c170 tx=0x7f4dcc009780).stop 2023-11-06T21:59:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 -- 172.21.15.33:0/1582729786 shutdown_connections 2023-11-06T21:59:23.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f4dc005f270 0x7f4dc0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:23.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dd4154710 0x7f4dd40aef70 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dd40c7ff0 0x7f4dd4164b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 --2- 172.21.15.33:0/1582729786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4dd4154330 0x7f4dd4168b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 -- 172.21.15.33:0/1582729786 >> 172.21.15.33:0/1582729786 conn(0x7f4dd40cb910 msgr2=0x7f4dd40ca310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:23.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 -- 172.21.15.33:0/1582729786 shutdown_connections 2023-11-06T21:59:23.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.637+0000 7f4dda1ff700 1 -- 172.21.15.33:0/1582729786 wait complete. 2023-11-06T21:59:23.643 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:23.641+0000 7f6401601680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:59:23.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T21:59:23.703 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:23.702+0000 7f216124c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:59:23.789 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:23.787+0000 7f216124c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:59:23.843 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:23.841+0000 7f6401601680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:59:23.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.859+0000 7f76e47ff700 1 Processor -- start 2023-11-06T21:59:23.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.859+0000 7f76e47ff700 1 -- start start 2023-11-06T21:59:23.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e47ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f76dc154330 0x7f76dc0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc154710 0x7f76dc0c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e47ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76dc0c7ff0 0x7f76dc0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e47ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc0c6b60 con 0x7f76dc0c7ff0 2023-11-06T21:59:23.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e47ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc0c5cd0 con 0x7f76dc154330 2023-11-06T21:59:23.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0c5f50 con 0x7f76dc154710 2023-11-06T21:59:23.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e2d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76dc0c7ff0 0x7f76dc0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e2d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76dc0c7ff0 0x7f76dc0646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35852/0 (socket says 172.21.15.33:35852) 2023-11-06T21:59:23.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e2d9c700 1 -- 172.21.15.33:0/1025342234 learned_addr learned my addr 172.21.15.33:0/1025342234 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:23.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e259b700 1 --2- 172.21.15.33:0/1025342234 >> v2:172.21.15.69:3300/0 conn(0x7f76dc154330 0x7f76dc0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.861+0000 7f76e1d9a700 1 --2- 172.21.15.33:0/1025342234 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc154710 0x7f76dc0c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.862+0000 7f76e2d9c700 1 -- 172.21.15.33:0/1025342234 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc154710 msgr2=0x7f76dc0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:23.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.862+0000 7f76e2d9c700 1 --2- 172.21.15.33:0/1025342234 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc154710 0x7f76dc0c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.862+0000 7f76e2d9c700 1 -- 172.21.15.33:0/1025342234 >> v2:172.21.15.69:3300/0 conn(0x7f76dc154330 msgr2=0x7f76dc0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:23.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.862+0000 7f76e2d9c700 1 --2- 172.21.15.33:0/1025342234 >> v2:172.21.15.69:3300/0 conn(0x7f76dc154330 0x7f76dc0c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.862+0000 7f76e2d9c700 1 -- 172.21.15.33:0/1025342234 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc0c6230 con 0x7f76dc0c7ff0 2023-11-06T21:59:23.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.862+0000 7f76e1d9a700 1 --2- 172.21.15.33:0/1025342234 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc154710 0x7f76dc0c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:23.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.862+0000 7f76e2d9c700 1 --2- 172.21.15.33:0/1025342234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc0c7ff0 0x7f76dc0646c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f76dc0605d0 tx=0x7f76d4009840).ready entity=mon.0 client_cookie=77a80c4214db140f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:23.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.862+0000 7f76e1599700 1 -- 172.21.15.33:0/1025342234 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d400e070 con 0x7f76dc0c7ff0 2023-11-06T21:59:23.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.862+0000 7f76e1599700 1 -- 172.21.15.33:0/1025342234 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d400ba20 con 0x7f76dc0c7ff0 2023-11-06T21:59:23.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.862+0000 7f76e1599700 1 -- 172.21.15.33:0/1025342234 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d400bda0 con 0x7f76dc0c7ff0 2023-11-06T21:59:23.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.863+0000 7f76e47ff700 1 -- 172.21.15.33:0/1025342234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc0c7ff0 msgr2=0x7f76dc0646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.863+0000 7f76e47ff700 1 --2- 172.21.15.33:0/1025342234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc0c7ff0 0x7f76dc0646c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f76dc0605d0 tx=0x7f76d4009840).stop 2023-11-06T21:59:23.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.863+0000 7f76e47ff700 1 -- 172.21.15.33:0/1025342234 shutdown_connections 2023-11-06T21:59:23.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.863+0000 7f76e47ff700 1 --2- 172.21.15.33:0/1025342234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc0c7ff0 0x7f76dc0646c0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.863+0000 7f76e47ff700 1 --2- 172.21.15.33:0/1025342234 >> v2:172.21.15.69:3300/0 conn(0x7f76dc154330 0x7f76dc0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.863+0000 7f76e47ff700 1 --2- 172.21.15.33:0/1025342234 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc154710 0x7f76dc0c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.863+0000 7f76e47ff700 1 -- 172.21.15.33:0/1025342234 >> 172.21.15.33:0/1025342234 conn(0x7f76dc0cb910 msgr2=0x7f76dc0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:23.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.863+0000 7f76e47ff700 1 -- 172.21.15.33:0/1025342234 shutdown_connections 2023-11-06T21:59:23.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.863+0000 7f76e47ff700 1 -- 172.21.15.33:0/1025342234 wait complete. 2023-11-06T21:59:23.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.864+0000 7f76e47ff700 1 Processor -- start 2023-11-06T21:59:23.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.864+0000 7f76e47ff700 1 -- start start 2023-11-06T21:59:23.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.865+0000 7f76e47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc0c7ff0 0x7f76dc13c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.865+0000 7f76e47ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76dc154330 0x7f76dc0b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.865+0000 7f76e47ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc154710 0x7f76dc0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.865+0000 7f76e47ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc13ca90 con 0x7f76dc154710 2023-11-06T21:59:23.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.865+0000 7f76e47ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0b0df0 con 0x7f76dc154330 2023-11-06T21:59:23.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.865+0000 7f76e47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0c2990 con 0x7f76dc0c7ff0 2023-11-06T21:59:23.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e2d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc154710 0x7f76dc0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e1d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76dc154330 0x7f76dc0b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e259b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc0c7ff0 0x7f76dc13c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e1d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76dc154330 0x7f76dc0b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:40882/0 (socket says 172.21.15.33:40882) 2023-11-06T21:59:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e2d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc154710 0x7f76dc0c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35866/0 (socket says 172.21.15.33:35866) 2023-11-06T21:59:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e259b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc0c7ff0 0x7f76dc13c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:49128/0 (socket says 172.21.15.33:49128) 2023-11-06T21:59:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e1d9a700 1 -- 172.21.15.33:0/3609557758 learned_addr learned my addr 172.21.15.33:0/3609557758 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:23.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e2d9c700 1 -- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc0c7ff0 msgr2=0x7f76dc13c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e2d9c700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc0c7ff0 0x7f76dc13c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e2d9c700 1 -- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76dc154330 msgr2=0x7f76dc0b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:23.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e2d9c700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76dc154330 0x7f76dc0b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:23.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e2d9c700 1 -- 172.21.15.33:0/3609557758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4009ae0 con 0x7f76dc154710 2023-11-06T21:59:23.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.866+0000 7f76e1d9a700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76dc154330 0x7f76dc0b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:23.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.867+0000 7f76e2d9c700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc154710 0x7f76dc0c21b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f76d4013000 tx=0x7f76d4009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:23.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.867+0000 7f76d27fc700 1 -- 172.21.15.33:0/3609557758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d40032f0 con 0x7f76dc154710 2023-11-06T21:59:23.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.867+0000 7f76d27fc700 1 -- 172.21.15.33:0/3609557758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d400bda0 con 0x7f76dc154710 2023-11-06T21:59:23.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.867+0000 7f76d27fc700 1 -- 172.21.15.33:0/3609557758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d401d690 con 0x7f76dc154710 2023-11-06T21:59:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.867+0000 7f76e47ff700 1 -- 172.21.15.33:0/3609557758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc0c2c70 con 0x7f76dc154710 2023-11-06T21:59:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.867+0000 7f76e47ff700 1 -- 172.21.15.33:0/3609557758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc0c3230 con 0x7f76dc154710 2023-11-06T21:59:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.868+0000 7f76e47ff700 1 -- 172.21.15.33:0/3609557758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc0830b0 con 0x7f76dc154710 2023-11-06T21:59:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.870+0000 7f76d27fc700 1 -- 172.21.15.33:0/3609557758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f76d4003040 con 0x7f76dc154710 2023-11-06T21:59:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.871+0000 7f76d27fc700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f76c005f270 0x7f76c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.871+0000 7f76d27fc700 1 -- 172.21.15.33:0/3609557758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f76d407b300 con 0x7f76dc154710 2023-11-06T21:59:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.871+0000 7f76e259b700 1 -- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f76c005f270 msgr2=0x7f76c0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.871+0000 7f76e259b700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f76c005f270 0x7f76c0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:23.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:23.875+0000 7f76d27fc700 1 -- 172.21.15.33:0/3609557758 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f76d404c370 con 0x7f76dc154710 2023-11-06T21:59:23.947 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:23.945+0000 7f6401601680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:59:24.039 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:24.037+0000 7f6401601680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:59:24.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.056+0000 7f76e47ff700 1 -- 172.21.15.33:0/3609557758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76dc062b00 con 0x7f76dc154710 2023-11-06T21:59:24.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.057+0000 7f76d27fc700 1 -- 172.21.15.33:0/3609557758 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) v1 ==== 100+0+11 (secure 0 0 0) 0x7f76d4022020 con 0x7f76dc154710 2023-11-06T21:59:24.059 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T21:59:24.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 -- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f76c005f270 msgr2=0x7f76c0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f76c005f270 0x7f76c0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 -- 172.21.15.33:0/3609557758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc154710 msgr2=0x7f76dc0c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc154710 0x7f76dc0c21b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f76d4013000 tx=0x7f76d4009840).stop 2023-11-06T21:59:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 -- 172.21.15.33:0/3609557758 shutdown_connections 2023-11-06T21:59:24.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f76c005f270 0x7f76c0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:24.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f76dc0c7ff0 0x7f76dc13c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76dc154710 0x7f76dc0c21b0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 --2- 172.21.15.33:0/3609557758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76dc154330 0x7f76dc0b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 -- 172.21.15.33:0/3609557758 >> 172.21.15.33:0/3609557758 conn(0x7f76dc0cb910 msgr2=0x7f76dc063510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:24.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 -- 172.21.15.33:0/3609557758 shutdown_connections 2023-11-06T21:59:24.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.059+0000 7f76e47ff700 1 -- 172.21.15.33:0/3609557758 wait complete. 2023-11-06T21:59:24.079 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:59:24.100 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:24.098+0000 7f216124c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:59:24.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.277+0000 7f9cd57ff700 1 Processor -- start 2023-11-06T21:59:24.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.277+0000 7f9cd57ff700 1 -- start start 2023-11-06T21:59:24.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.278+0000 7f9cd57ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cd00acda0 0x7f9cd00638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.278+0000 7f9cd57ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd00c4490 0x7f9cd00c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.278+0000 7f9cd57ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd00c3380 con 0x7f9cd0152250 2023-11-06T21:59:24.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.278+0000 7f9cd57ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd00c20d0 con 0x7f9cd00acda0 2023-11-06T21:59:24.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.278+0000 7f9cd57ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd005e520 con 0x7f9cd00c4490 2023-11-06T21:59:24.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.278+0000 7f9cce7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cd00acda0 0x7f9cd00638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.278+0000 7f9cce7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cd00acda0 0x7f9cd00638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35870/0 (socket says 172.21.15.33:35870) 2023-11-06T21:59:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.278+0000 7f9cce7fc700 1 -- 172.21.15.33:0/1872517038 learned_addr learned my addr 172.21.15.33:0/1872517038 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.278+0000 7f9ccf7fe700 1 --2- 172.21.15.33:0/1872517038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd00c4490 0x7f9cd00c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.279+0000 7f9cce7fc700 1 -- 172.21.15.33:0/1872517038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd00c4490 msgr2=0x7f9cd00c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.279+0000 7f9cce7fc700 1 --2- 172.21.15.33:0/1872517038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd00c4490 0x7f9cd00c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.279+0000 7f9cce7fc700 1 -- 172.21.15.33:0/1872517038 >> v1:172.21.15.69:6789/0 conn(0x7f9cd0152250 legacy=0x7f9cd00ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.279+0000 7f9cce7fc700 1 -- 172.21.15.33:0/1872517038 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd005e800 con 0x7f9cd00acda0 2023-11-06T21:59:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.279+0000 7f9cce7fc700 1 --2- 172.21.15.33:0/1872517038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 0x7f9cd00638b0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f9cc4006fd0 tx=0x7f9cc40090f0).ready entity=mon.0 client_cookie=d63b381f3ad9c3d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.279+0000 7f9ccdffb700 1 -- 172.21.15.33:0/1872517038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400b580 con 0x7f9cd00acda0 2023-11-06T21:59:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.279+0000 7f9ccdffb700 1 -- 172.21.15.33:0/1872517038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc400b6e0 con 0x7f9cd00acda0 2023-11-06T21:59:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.279+0000 7f9ccdffb700 1 -- 172.21.15.33:0/1872517038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400ba60 con 0x7f9cd00acda0 2023-11-06T21:59:24.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.280+0000 7f9cd57ff700 1 -- 172.21.15.33:0/1872517038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 msgr2=0x7f9cd00638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.280+0000 7f9cd57ff700 1 --2- 172.21.15.33:0/1872517038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 0x7f9cd00638b0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f9cc4006fd0 tx=0x7f9cc40090f0).stop 2023-11-06T21:59:24.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.280+0000 7f9cd57ff700 1 -- 172.21.15.33:0/1872517038 shutdown_connections 2023-11-06T21:59:24.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.280+0000 7f9cd57ff700 1 --2- 172.21.15.33:0/1872517038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd00c4490 0x7f9cd00c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.280+0000 7f9cd57ff700 1 --2- 172.21.15.33:0/1872517038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 0x7f9cd00638b0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.280+0000 7f9cd57ff700 1 -- 172.21.15.33:0/1872517038 >> 172.21.15.33:0/1872517038 conn(0x7f9cd00c7d60 msgr2=0x7f9cd00c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:24.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.280+0000 7f9cd57ff700 1 -- 172.21.15.33:0/1872517038 shutdown_connections 2023-11-06T21:59:24.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.280+0000 7f9cd57ff700 1 -- 172.21.15.33:0/1872517038 wait complete. 2023-11-06T21:59:24.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.282+0000 7f9cd57ff700 1 Processor -- start 2023-11-06T21:59:24.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.282+0000 7f9cd57ff700 1 -- start start 2023-11-06T21:59:24.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9cd57ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 0x7f9cd014cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9cd57ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cd00c4490 0x7f9cd0164400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9cd57ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd0152250 0x7f9cd00c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9cd57ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0154a90 con 0x7f9cd00acda0 2023-11-06T21:59:24.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9cd57ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0155330 con 0x7f9cd00c4490 2023-11-06T21:59:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9cd57ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd00ab2b0 con 0x7f9cd0152250 2023-11-06T21:59:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9cceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 0x7f9cd014cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9cceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 0x7f9cd014cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35874/0 (socket says 172.21.15.33:35874) 2023-11-06T21:59:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9cceffd700 1 -- 172.21.15.33:0/2797030092 learned_addr learned my addr 172.21.15.33:0/2797030092 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:24.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9ccf7fe700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd0152250 0x7f9cd00c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:24.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.283+0000 7f9cce7fc700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cd00c4490 0x7f9cd0164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:24.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cceffd700 1 -- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd0152250 msgr2=0x7f9cd00c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cceffd700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd0152250 0x7f9cd00c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cceffd700 1 -- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cd00c4490 msgr2=0x7f9cd0164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cceffd700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cd00c4490 0x7f9cd0164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cceffd700 1 -- 172.21.15.33:0/2797030092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4009370 con 0x7f9cd00acda0 2023-11-06T21:59:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9ccf7fe700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd0152250 0x7f9cd00c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cce7fc700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cd00c4490 0x7f9cd0164400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cceffd700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 0x7f9cd014cdb0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f9cc000c470 tx=0x7f9cc0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:24.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cb6ffd700 1 -- 172.21.15.33:0/2797030092 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0007020 con 0x7f9cd00acda0 2023-11-06T21:59:24.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cb6ffd700 1 -- 172.21.15.33:0/2797030092 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc0007180 con 0x7f9cd00acda0 2023-11-06T21:59:24.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cd57ff700 1 -- 172.21.15.33:0/2797030092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd0154ce0 con 0x7f9cd00acda0 2023-11-06T21:59:24.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cb6ffd700 1 -- 172.21.15.33:0/2797030092 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0007520 con 0x7f9cd00acda0 2023-11-06T21:59:24.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.284+0000 7f9cd57ff700 1 -- 172.21.15.33:0/2797030092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd014d2a0 con 0x7f9cd00acda0 2023-11-06T21:59:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.285+0000 7f9cd57ff700 1 -- 172.21.15.33:0/2797030092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd0064420 con 0x7f9cd00acda0 2023-11-06T21:59:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.286+0000 7f9cb6ffd700 1 -- 172.21.15.33:0/2797030092 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f9cc00068b0 con 0x7f9cd00acda0 2023-11-06T21:59:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.287+0000 7f9cb6ffd700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cbc05f270 0x7f9cbc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.287+0000 7f9cb6ffd700 1 -- 172.21.15.33:0/2797030092 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9cc00798c0 con 0x7f9cd00acda0 2023-11-06T21:59:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.287+0000 7f9cce7fc700 1 -- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cbc05f270 msgr2=0x7f9cbc061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.287+0000 7f9cce7fc700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cbc05f270 0x7f9cbc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.291+0000 7f9cb6ffd700 1 -- 172.21.15.33:0/2797030092 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9cc0047af0 con 0x7f9cd00acda0 2023-11-06T21:59:24.321 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:24.320+0000 7f216124c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:59:24.379 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:24.377+0000 7f6401601680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:59:24.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.488+0000 7f9cce7fc700 1 -- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cbc05f270 msgr2=0x7f9cbc061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:24.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.488+0000 7f9cce7fc700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cbc05f270 0x7f9cbc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:59:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.517+0000 7f9cd57ff700 1 -- 172.21.15.33:0/2797030092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cd00c2d30 con 0x7f9cd00acda0 2023-11-06T21:59:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.518+0000 7f9cb6ffd700 1 -- 172.21.15.33:0/2797030092 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f9cc004a8a0 con 0x7f9cd00acda0 2023-11-06T21:59:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:24.521 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1809147689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1809147689},{"type":"v1","addr":"172.21.15.33:6835","nonce":1809147689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1390207299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1390207299},{"type":"v1","addr":"172.21.15.69:6837","nonce":1390207299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3179984136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3179984136},{"type":"v1","addr":"172.21.15.69:6836","nonce":3179984136}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:03:21.683218+0000","modified":"2023-11-06T21:03:24.689443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7720":{"gid":7720,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/4126832799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4126832799},{"type":"v1","addr":"172.21.15.33:6834","nonce":4126832799}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-06T21:59:24.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.520+0000 7f9cd57ff700 1 -- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cbc05f270 msgr2=0x7f9cbc061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:24.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.520+0000 7f9cd57ff700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cbc05f270 0x7f9cbc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:24.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.520+0000 7f9cd57ff700 1 -- 172.21.15.33:0/2797030092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 msgr2=0x7f9cd014cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.520+0000 7f9cd57ff700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 0x7f9cd014cdb0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f9cc000c470 tx=0x7f9cc0007e00).stop 2023-11-06T21:59:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.520+0000 7f9cd57ff700 1 -- 172.21.15.33:0/2797030092 shutdown_connections 2023-11-06T21:59:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.520+0000 7f9cd57ff700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9cbc05f270 0x7f9cbc061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.520+0000 7f9cd57ff700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9cd0152250 0x7f9cd00c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.520+0000 7f9cd57ff700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd00acda0 0x7f9cd014cdb0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.520+0000 7f9cd57ff700 1 --2- 172.21.15.33:0/2797030092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cd00c4490 0x7f9cd0164400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.520+0000 7f9cd57ff700 1 -- 172.21.15.33:0/2797030092 >> 172.21.15.33:0/2797030092 conn(0x7f9cd00c7d60 msgr2=0x7f9cd0152630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.521+0000 7f9cd57ff700 1 -- 172.21.15.33:0/2797030092 shutdown_connections 2023-11-06T21:59:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.521+0000 7f9cd57ff700 1 -- 172.21.15.33:0/2797030092 wait complete. 2023-11-06T21:59:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 60 2023-11-06T21:59:24.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:59:24.614 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:24.612+0000 7f6401601680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:59:24.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.739+0000 7f90c9bff700 1 Processor -- start 2023-11-06T21:59:24.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.739+0000 7f90c9bff700 1 -- start start 2023-11-06T21:59:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.741+0000 7f90c9bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90c40c5d50 0x7f90c40c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.741+0000 7f90c9bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c41554a0 con 0x7f90c40c1840 2023-11-06T21:59:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.741+0000 7f90c9bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c40c66c0 con 0x7f90c40bfd70 2023-11-06T21:59:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.741+0000 7f90c9bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c4154960 con 0x7f90c40c5d50 2023-11-06T21:59:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.741+0000 7f90c2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90c40c5d50 0x7f90c40c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.741+0000 7f90c2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90c40c5d50 0x7f90c40c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35888/0 (socket says 172.21.15.33:35888) 2023-11-06T21:59:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.741+0000 7f90c2ffd700 1 -- 172.21.15.33:0/812351286 learned_addr learned my addr 172.21.15.33:0/812351286 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.741+0000 7f90c37fe700 1 --1- 172.21.15.33:0/812351286 >> v1:172.21.15.33:6789/0 conn(0x7f90c40c1840 0x7f90c40a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53246/0 (socket says 172.21.15.33:53246) 2023-11-06T21:59:24.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.742+0000 7f90c2ffd700 1 -- 172.21.15.33:0/812351286 >> v1:172.21.15.69:6789/0 conn(0x7f90c40bfd70 legacy=0x7f90c40c0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.742+0000 7f90c2ffd700 1 -- 172.21.15.33:0/812351286 >> v1:172.21.15.33:6789/0 conn(0x7f90c40c1840 legacy=0x7f90c40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.742+0000 7f90c27fc700 1 -- 172.21.15.33:0/812351286 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388917717 0 0) 0x7f90c41554a0 con 0x7f90c40c1840 2023-11-06T21:59:24.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.742+0000 7f90c2ffd700 1 -- 172.21.15.33:0/812351286 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c4155930 con 0x7f90c40c5d50 2023-11-06T21:59:24.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.742+0000 7f90c2ffd700 1 --2- 172.21.15.33:0/812351286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40c5d50 0x7f90c40c61d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7f90b8006fd0 tx=0x7f90b80090f0).ready entity=mon.0 client_cookie=b15ddf65ce76889c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:24.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.742+0000 7f90c27fc700 1 -- 172.21.15.33:0/812351286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b800f510 con 0x7f90c40c5d50 2023-11-06T21:59:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.742+0000 7f90c27fc700 1 -- 172.21.15.33:0/812351286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b800f670 con 0x7f90c40c5d50 2023-11-06T21:59:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.742+0000 7f90c27fc700 1 -- 172.21.15.33:0/812351286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b800fa10 con 0x7f90c40c5d50 2023-11-06T21:59:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.743+0000 7f90c9bff700 1 -- 172.21.15.33:0/812351286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40c5d50 msgr2=0x7f90c40c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.743+0000 7f90c9bff700 1 --2- 172.21.15.33:0/812351286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40c5d50 0x7f90c40c61d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7f90b8006fd0 tx=0x7f90b80090f0).stop 2023-11-06T21:59:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.743+0000 7f90c9bff700 1 -- 172.21.15.33:0/812351286 shutdown_connections 2023-11-06T21:59:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.743+0000 7f90c9bff700 1 --2- 172.21.15.33:0/812351286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40c5d50 0x7f90c40c61d0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.743+0000 7f90c9bff700 1 -- 172.21.15.33:0/812351286 >> 172.21.15.33:0/812351286 conn(0x7f90c405ec80 msgr2=0x7f90c40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.743+0000 7f90c9bff700 1 -- 172.21.15.33:0/812351286 shutdown_connections 2023-11-06T21:59:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.743+0000 7f90c9bff700 1 -- 172.21.15.33:0/812351286 wait complete. 2023-11-06T21:59:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.744+0000 7f90c9bff700 1 Processor -- start 2023-11-06T21:59:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.744+0000 7f90c9bff700 1 -- start start 2023-11-06T21:59:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.745+0000 7f90c9bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40bfd70 0x7f90c414c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.745+0000 7f90c9bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f90c40c1840 0x7f90c40ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.745+0000 7f90c9bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90c40c5d50 0x7f90c4167a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.745+0000 7f90c9bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c414c540 con 0x7f90c40bfd70 2023-11-06T21:59:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.745+0000 7f90c9bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c414e1a0 con 0x7f90c40c5d50 2023-11-06T21:59:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.745+0000 7f90c9bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c414c780 con 0x7f90c40c1840 2023-11-06T21:59:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40bfd70 0x7f90c414c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:24.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40bfd70 0x7f90c414c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35904/0 (socket says 172.21.15.33:35904) 2023-11-06T21:59:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c37fe700 1 -- 172.21.15.33:0/3564710444 learned_addr learned my addr 172.21.15.33:0/3564710444 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c2ffd700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f90c40c1840 0x7f90c40ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c3fff700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90c40c5d50 0x7f90c4167a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c37fe700 1 -- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f90c40c1840 msgr2=0x7f90c40ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c37fe700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f90c40c1840 0x7f90c40ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c37fe700 1 -- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90c40c5d50 msgr2=0x7f90c4167a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c37fe700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90c40c5d50 0x7f90c4167a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c37fe700 1 -- 172.21.15.33:0/3564710444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b8009370 con 0x7f90c40bfd70 2023-11-06T21:59:24.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c3fff700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90c40c5d50 0x7f90c4167a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:24.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90c37fe700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40bfd70 0x7f90c414c050 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f90b400c4c0 tx=0x7f90b4007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.746+0000 7f90ab7fe700 1 -- 172.21.15.33:0/3564710444 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40072e0 con 0x7f90c40bfd70 2023-11-06T21:59:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.747+0000 7f90ab7fe700 1 -- 172.21.15.33:0/3564710444 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b4007440 con 0x7f90c40bfd70 2023-11-06T21:59:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.747+0000 7f90ab7fe700 1 -- 172.21.15.33:0/3564710444 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4005930 con 0x7f90c40bfd70 2023-11-06T21:59:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.747+0000 7f90c9bff700 1 -- 172.21.15.33:0/3564710444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c414ca60 con 0x7f90c40bfd70 2023-11-06T21:59:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.747+0000 7f90c9bff700 1 -- 172.21.15.33:0/3564710444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c414d020 con 0x7f90c40bfd70 2023-11-06T21:59:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.748+0000 7f90c9bff700 1 -- 172.21.15.33:0/3564710444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c406c6c0 con 0x7f90c40bfd70 2023-11-06T21:59:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.750+0000 7f90ab7fe700 1 -- 172.21.15.33:0/3564710444 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f90b4005ce0 con 0x7f90c40bfd70 2023-11-06T21:59:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.750+0000 7f90ab7fe700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90b005f270 0x7f90b0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.750+0000 7f90ab7fe700 1 -- 172.21.15.33:0/3564710444 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f90b4079c70 con 0x7f90c40bfd70 2023-11-06T21:59:24.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.751+0000 7f90c2ffd700 1 -- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90b005f270 msgr2=0x7f90b0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:24.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.751+0000 7f90c2ffd700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90b005f270 0x7f90b0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:24.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.754+0000 7f90ab7fe700 1 -- 172.21.15.33:0/3564710444 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f90b404ac50 con 0x7f90c40bfd70 2023-11-06T21:59:24.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.936+0000 7f90c9bff700 1 -- 172.21.15.33:0/3564710444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90c4063f20 con 0x7f90c40bfd70 2023-11-06T21:59:24.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.937+0000 7f90ab7fe700 1 -- 172.21.15.33:0/3564710444 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+17573 (secure 0 0 0) 0x7f90b404e270 con 0x7f90c40bfd70 2023-11-06T21:59:24.940 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:24.940 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":156,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:59:21.974687+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:03:20.146564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T21:03:20.662302+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":156,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2083752177":"2023-11-07T21:03:35.888563+0000","172.21.15.69:6833/114912":"2023-11-07T21:03:35.888563+0000","172.21.15.69:6832/114912":"2023-11-07T21:03:35.888563+0000","172.21.15.69:0/47836582":"2023-11-07T21:03:35.888563+0000","172.21.15.69:0/3836508278":"2023-11-07T21:03:35.888563+0000","172.21.15.69:0/2121315625":"2023-11-07T21:03:35.888563+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":54,"length":4},{"begin":110,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:59:24.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.939+0000 7f90c9bff700 1 -- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90b005f270 msgr2=0x7f90b0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:24.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.939+0000 7f90c9bff700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90b005f270 0x7f90b0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:24.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.939+0000 7f90c9bff700 1 -- 172.21.15.33:0/3564710444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40bfd70 msgr2=0x7f90c414c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:24.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.939+0000 7f90c9bff700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40bfd70 0x7f90c414c050 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f90b400c4c0 tx=0x7f90b4007c10).stop 2023-11-06T21:59:24.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.939+0000 7f90c9bff700 1 -- 172.21.15.33:0/3564710444 shutdown_connections 2023-11-06T21:59:24.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.939+0000 7f90c9bff700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f90b005f270 0x7f90b0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:24.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.939+0000 7f90c9bff700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90c40c5d50 0x7f90c4167a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.939+0000 7f90c9bff700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c40bfd70 0x7f90c414c050 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.939+0000 7f90c9bff700 1 --2- 172.21.15.33:0/3564710444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f90c40c1840 0x7f90c40ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.939+0000 7f90c9bff700 1 -- 172.21.15.33:0/3564710444 >> 172.21.15.33:0/3564710444 conn(0x7f90c405ec80 msgr2=0x7f90c4152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.940+0000 7f90c9bff700 1 -- 172.21.15.33:0/3564710444 shutdown_connections 2023-11-06T21:59:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:24.940+0000 7f90c9bff700 1 -- 172.21.15.33:0/3564710444 wait complete. 2023-11-06T21:59:24.960 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T21:59:24.961 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:59:25.126 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:25.124+0000 7f216124c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:59:25.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.158+0000 7f39197ff700 1 Processor -- start 2023-11-06T21:59:25.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.158+0000 7f39197ff700 1 -- start start 2023-11-06T21:59:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.159+0000 7f39197ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f39140c7b80 0x7f3914154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.159+0000 7f39197ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39140c5db0 con 0x7f3914154280 2023-11-06T21:59:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.159+0000 7f39197ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39140c6930 con 0x7f39140ad6e0 2023-11-06T21:59:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.159+0000 7f39197ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39140aa630 con 0x7f39140c7b80 2023-11-06T21:59:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.160+0000 7f3912ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3914154280 0x7f39140a9f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53250/0 (socket says 172.21.15.33:53250) 2023-11-06T21:59:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.160+0000 7f3912ffd700 1 -- 172.21.15.33:0/3976139699 learned_addr learned my addr 172.21.15.33:0/3976139699 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.160+0000 7f39137fe700 1 --2- 172.21.15.33:0/3976139699 >> v2:172.21.15.33:3300/0 conn(0x7f39140c7b80 0x7f3914154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.160+0000 7f3911ffb700 1 -- 172.21.15.33:0/3976139699 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640115523 0 0) 0x7f39140c5db0 con 0x7f3914154280 2023-11-06T21:59:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.160+0000 7f3911ffb700 1 -- 172.21.15.33:0/3976139699 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38fc0033f0 con 0x7f3914154280 2023-11-06T21:59:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.160+0000 7f39137fe700 1 -- 172.21.15.33:0/3976139699 >> v1:172.21.15.69:6789/0 conn(0x7f39140ad6e0 legacy=0x7f39140c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.160+0000 7f39137fe700 1 -- 172.21.15.33:0/3976139699 >> v1:172.21.15.33:6789/0 conn(0x7f3914154280 legacy=0x7f39140a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.160+0000 7f39137fe700 1 -- 172.21.15.33:0/3976139699 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140c6000 con 0x7f39140c7b80 2023-11-06T21:59:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.161+0000 7f39137fe700 1 --2- 172.21.15.33:0/3976139699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140c7b80 0x7f3914154660 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f390c009e60 tx=0x7f390c009780).ready entity=mon.0 client_cookie=4f8c70d3dc56717a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.161+0000 7f3911ffb700 1 -- 172.21.15.33:0/3976139699 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c00e070 con 0x7f39140c7b80 2023-11-06T21:59:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.161+0000 7f3911ffb700 1 -- 172.21.15.33:0/3976139699 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c00bb60 con 0x7f39140c7b80 2023-11-06T21:59:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.161+0000 7f3911ffb700 1 -- 172.21.15.33:0/3976139699 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c012410 con 0x7f39140c7b80 2023-11-06T21:59:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.161+0000 7f39197ff700 1 -- 172.21.15.33:0/3976139699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140c7b80 msgr2=0x7f3914154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.161+0000 7f39197ff700 1 --2- 172.21.15.33:0/3976139699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140c7b80 0x7f3914154660 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f390c009e60 tx=0x7f390c009780).stop 2023-11-06T21:59:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.162+0000 7f39197ff700 1 -- 172.21.15.33:0/3976139699 shutdown_connections 2023-11-06T21:59:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.162+0000 7f39197ff700 1 --2- 172.21.15.33:0/3976139699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140c7b80 0x7f3914154660 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.162+0000 7f39197ff700 1 -- 172.21.15.33:0/3976139699 >> 172.21.15.33:0/3976139699 conn(0x7f391405e1b0 msgr2=0x7f391405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.162+0000 7f39197ff700 1 -- 172.21.15.33:0/3976139699 shutdown_connections 2023-11-06T21:59:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.162+0000 7f39197ff700 1 -- 172.21.15.33:0/3976139699 wait complete. 2023-11-06T21:59:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.163+0000 7f39197ff700 1 Processor -- start 2023-11-06T21:59:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.163+0000 7f39197ff700 1 -- start start 2023-11-06T21:59:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.164+0000 7f39197ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140ad6e0 0x7f3914167a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.164+0000 7f39197ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39140c7b80 0x7f39140a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.164+0000 7f39197ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914154280 0x7f39140c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.164+0000 7f39197ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140a37d0 con 0x7f39140ad6e0 2023-11-06T21:59:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.164+0000 7f39197ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140a2ea0 con 0x7f3914154280 2023-11-06T21:59:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.164+0000 7f39197ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39140c1ff0 con 0x7f39140c7b80 2023-11-06T21:59:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.164+0000 7f3912ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140ad6e0 0x7f3914167a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f39137fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914154280 0x7f39140c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f3912ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140ad6e0 0x7f3914167a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35924/0 (socket says 172.21.15.33:35924) 2023-11-06T21:59:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f3912ffd700 1 -- 172.21.15.33:0/141992011 learned_addr learned my addr 172.21.15.33:0/141992011 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f39127fc700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39140c7b80 0x7f39140a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f3912ffd700 1 -- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39140c7b80 msgr2=0x7f39140a32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f3912ffd700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39140c7b80 0x7f39140a32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f3912ffd700 1 -- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914154280 msgr2=0x7f39140c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f3912ffd700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914154280 0x7f39140c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f3912ffd700 1 -- 172.21.15.33:0/141992011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c009a20 con 0x7f39140ad6e0 2023-11-06T21:59:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f3912ffd700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140ad6e0 0x7f3914167a70 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f390400c420 tx=0x7f3904007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f38faffd700 1 -- 172.21.15.33:0/141992011 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39040070d0 con 0x7f39140ad6e0 2023-11-06T21:59:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f38faffd700 1 -- 172.21.15.33:0/141992011 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3904007230 con 0x7f39140ad6e0 2023-11-06T21:59:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f39197ff700 1 -- 172.21.15.33:0/141992011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3914168050 con 0x7f39140ad6e0 2023-11-06T21:59:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.165+0000 7f38faffd700 1 -- 172.21.15.33:0/141992011 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39040075b0 con 0x7f39140ad6e0 2023-11-06T21:59:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.166+0000 7f39197ff700 1 -- 172.21.15.33:0/141992011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3914168610 con 0x7f39140ad6e0 2023-11-06T21:59:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.167+0000 7f39197ff700 1 -- 172.21.15.33:0/141992011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39140c62e0 con 0x7f39140ad6e0 2023-11-06T21:59:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.167+0000 7f38faffd700 1 -- 172.21.15.33:0/141992011 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f39040059b0 con 0x7f39140ad6e0 2023-11-06T21:59:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.168+0000 7f38faffd700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f390005f270 0x7f3900061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.168+0000 7f38faffd700 1 -- 172.21.15.33:0/141992011 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3904079d30 con 0x7f39140ad6e0 2023-11-06T21:59:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.168+0000 7f39127fc700 1 -- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f390005f270 msgr2=0x7f3900061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.169+0000 7f39127fc700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f390005f270 0x7f3900061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.172+0000 7f38faffd700 1 -- 172.21.15.33:0/141992011 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f390404ac70 con 0x7f39140ad6e0 2023-11-06T21:59:25.215 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:25.213+0000 7f216124c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:59:25.309 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:25.307+0000 7f216124c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:59:25.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.370+0000 7f39127fc700 1 -- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f390005f270 msgr2=0x7f3900061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:25.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.370+0000 7f39127fc700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f390005f270 0x7f3900061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:59:25.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.397+0000 7f39197ff700 1 -- 172.21.15.33:0/141992011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3914062ac0 con 0x7f39140ad6e0 2023-11-06T21:59:25.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.398+0000 7f38faffd700 1 -- 172.21.15.33:0/141992011 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f390404e290 con 0x7f39140ad6e0 2023-11-06T21:59:25.400 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:25.400 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1809147689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1809147689},{"type":"v1","addr":"172.21.15.33:6835","nonce":1809147689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1390207299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1390207299},{"type":"v1","addr":"172.21.15.69:6837","nonce":1390207299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3179984136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3179984136},{"type":"v1","addr":"172.21.15.69:6836","nonce":3179984136}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:03:21.683218+0000","modified":"2023-11-06T21:03:24.689443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7720":{"gid":7720,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/4126832799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4126832799},{"type":"v1","addr":"172.21.15.33:6834","nonce":4126832799}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-06T21:59:25.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.400+0000 7f39197ff700 1 -- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f390005f270 msgr2=0x7f3900061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:25.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.400+0000 7f39197ff700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f390005f270 0x7f3900061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:25.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.400+0000 7f39197ff700 1 -- 172.21.15.33:0/141992011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140ad6e0 msgr2=0x7f3914167a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.400+0000 7f39197ff700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140ad6e0 0x7f3914167a70 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f390400c420 tx=0x7f3904007c80).stop 2023-11-06T21:59:25.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.400+0000 7f39197ff700 1 -- 172.21.15.33:0/141992011 shutdown_connections 2023-11-06T21:59:25.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.400+0000 7f39197ff700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f390005f270 0x7f3900061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:25.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.400+0000 7f39197ff700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914154280 0x7f39140c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.400+0000 7f39197ff700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39140ad6e0 0x7f3914167a70 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.400+0000 7f39197ff700 1 --2- 172.21.15.33:0/141992011 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39140c7b80 0x7f39140a32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.400+0000 7f39197ff700 1 -- 172.21.15.33:0/141992011 >> 172.21.15.33:0/141992011 conn(0x7f391405e1b0 msgr2=0x7f39140ac390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.401+0000 7f39197ff700 1 -- 172.21.15.33:0/141992011 shutdown_connections 2023-11-06T21:59:25.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.401+0000 7f39197ff700 1 -- 172.21.15.33:0/141992011 wait complete. 2023-11-06T21:59:25.406 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 60 2023-11-06T21:59:25.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:59:25.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:59:25.473 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:59:25.487 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:25.485+0000 7f6401601680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:59:25.497 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:25.496+0000 7f216124c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:59:25.585 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:25.583+0000 7f6401601680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:59:25.592 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:25.591+0000 7f216124c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:59:25.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.622+0000 7fe11e7ff700 1 Processor -- start 2023-11-06T21:59:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.622+0000 7fe11e7ff700 1 -- start start 2023-11-06T21:59:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.624+0000 7fe11e7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe1181521a0 0x7fe1180c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.624+0000 7fe11e7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe118154e90 con 0x7fe118151dc0 2023-11-06T21:59:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.624+0000 7fe11e7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1180639d0 con 0x7fe118155320 2023-11-06T21:59:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.624+0000 7fe11e7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe118063cb0 con 0x7fe1181521a0 2023-11-06T21:59:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.624+0000 7fe117fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe118151dc0 0x7fe118063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53254/0 (socket says 172.21.15.33:53254) 2023-11-06T21:59:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.624+0000 7fe117fff700 1 -- 172.21.15.33:0/2736958893 learned_addr learned my addr 172.21.15.33:0/2736958893 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.624+0000 7fe1177fe700 1 --2- 172.21.15.33:0/2736958893 >> v2:172.21.15.69:3300/0 conn(0x7fe1181521a0 0x7fe1180c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.625+0000 7fe116ffd700 1 -- 172.21.15.33:0/2736958893 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288233111 0 0) 0x7fe118154e90 con 0x7fe118151dc0 2023-11-06T21:59:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.625+0000 7fe116ffd700 1 -- 172.21.15.33:0/2736958893 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1000033f0 con 0x7fe118151dc0 2023-11-06T21:59:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.625+0000 7fe116ffd700 1 -- 172.21.15.33:0/2736958893 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139444905 0 0) 0x7fe1180639d0 con 0x7fe118155320 2023-11-06T21:59:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.625+0000 7fe116ffd700 1 -- 172.21.15.33:0/2736958893 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe118154e90 con 0x7fe118155320 2023-11-06T21:59:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.625+0000 7fe1177fe700 1 -- 172.21.15.33:0/2736958893 >> v1:172.21.15.69:6789/0 conn(0x7fe118155320 legacy=0x7fe1180c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.625+0000 7fe1177fe700 1 -- 172.21.15.33:0/2736958893 >> v1:172.21.15.33:6789/0 conn(0x7fe118151dc0 legacy=0x7fe118063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.625+0000 7fe1177fe700 1 -- 172.21.15.33:0/2736958893 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe118063f90 con 0x7fe1181521a0 2023-11-06T21:59:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.625+0000 7fe1177fe700 1 --2- 172.21.15.33:0/2736958893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1181521a0 0x7fe1180c0c50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fe10c009760 tx=0x7fe10c009060).ready entity=mon.1 client_cookie=5ef5af132b9cfe24 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.626+0000 7fe116ffd700 1 -- 172.21.15.33:0/2736958893 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00d4a0 con 0x7fe1181521a0 2023-11-06T21:59:25.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.626+0000 7fe116ffd700 1 -- 172.21.15.33:0/2736958893 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c00d600 con 0x7fe1181521a0 2023-11-06T21:59:25.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.626+0000 7fe116ffd700 1 -- 172.21.15.33:0/2736958893 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00d980 con 0x7fe1181521a0 2023-11-06T21:59:25.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.626+0000 7fe11e7ff700 1 -- 172.21.15.33:0/2736958893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1181521a0 msgr2=0x7fe1180c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.626+0000 7fe11e7ff700 1 --2- 172.21.15.33:0/2736958893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1181521a0 0x7fe1180c0c50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fe10c009760 tx=0x7fe10c009060).stop 2023-11-06T21:59:25.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.626+0000 7fe11e7ff700 1 -- 172.21.15.33:0/2736958893 shutdown_connections 2023-11-06T21:59:25.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.626+0000 7fe11e7ff700 1 --2- 172.21.15.33:0/2736958893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1181521a0 0x7fe1180c0c50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.626+0000 7fe11e7ff700 1 -- 172.21.15.33:0/2736958893 >> 172.21.15.33:0/2736958893 conn(0x7fe11805e1b0 msgr2=0x7fe11805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:25.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.626+0000 7fe11e7ff700 1 -- 172.21.15.33:0/2736958893 shutdown_connections 2023-11-06T21:59:25.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.626+0000 7fe11e7ff700 1 -- 172.21.15.33:0/2736958893 wait complete. 2023-11-06T21:59:25.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.628+0000 7fe11e7ff700 1 Processor -- start 2023-11-06T21:59:25.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.628+0000 7fe11e7ff700 1 -- start start 2023-11-06T21:59:25.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe11e7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe118151dc0 0x7fe1181506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:25.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe11e7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1181521a0 0x7fe1180af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:25.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe11e7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe118155320 0x7fe1180c2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:25.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe11e7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180c7910 con 0x7fe1181521a0 2023-11-06T21:59:25.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe11e7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180c4d80 con 0x7fe118151dc0 2023-11-06T21:59:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe11e7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180c23d0 con 0x7fe118155320 2023-11-06T21:59:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe1177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1181521a0 0x7fe1180af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe1177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1181521a0 0x7fe1180af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35928/0 (socket says 172.21.15.33:35928) 2023-11-06T21:59:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe1177fe700 1 -- 172.21.15.33:0/1435841561 learned_addr learned my addr 172.21.15.33:0/1435841561 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:25.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe117fff700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe118151dc0 0x7fe1181506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:25.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.629+0000 7fe11cd9c700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe118155320 0x7fe1180c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:25.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe1177fe700 1 -- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe118155320 msgr2=0x7fe1180c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe1177fe700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe118155320 0x7fe1180c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe1177fe700 1 -- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe118151dc0 msgr2=0x7fe1181506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe1177fe700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe118151dc0 0x7fe1181506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe1177fe700 1 -- 172.21.15.33:0/1435841561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c0092a0 con 0x7fe1181521a0 2023-11-06T21:59:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe11cd9c700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe118155320 0x7fe1180c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe1177fe700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1181521a0 0x7fe1180af390 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7fe10c011fb0 tx=0x7fe10c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe0fffff700 1 -- 172.21.15.33:0/1435841561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00dcf0 con 0x7fe1181521a0 2023-11-06T21:59:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe0fffff700 1 -- 172.21.15.33:0/1435841561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c00de50 con 0x7fe1181521a0 2023-11-06T21:59:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe11e7ff700 1 -- 172.21.15.33:0/1435841561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1180c47c0 con 0x7fe1181521a0 2023-11-06T21:59:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe0fffff700 1 -- 172.21.15.33:0/1435841561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c01d430 con 0x7fe1181521a0 2023-11-06T21:59:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.630+0000 7fe11e7ff700 1 -- 172.21.15.33:0/1435841561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1180c2060 con 0x7fe1181521a0 2023-11-06T21:59:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.632+0000 7fe11e7ff700 1 -- 172.21.15.33:0/1435841561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1180c0a30 con 0x7fe1181521a0 2023-11-06T21:59:25.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.632+0000 7fe0fffff700 1 -- 172.21.15.33:0/1435841561 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fe10c01d7a0 con 0x7fe1181521a0 2023-11-06T21:59:25.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.633+0000 7fe0fffff700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe10405f270 0x7fe104061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:25.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.633+0000 7fe0fffff700 1 -- 172.21.15.33:0/1435841561 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe10c07bdd0 con 0x7fe1181521a0 2023-11-06T21:59:25.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.633+0000 7fe117fff700 1 -- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe10405f270 msgr2=0x7fe104061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:25.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.633+0000 7fe117fff700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe10405f270 0x7fe104061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:25.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.637+0000 7fe0fffff700 1 -- 172.21.15.33:0/1435841561 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe10c04cd10 con 0x7fe1181521a0 2023-11-06T21:59:25.690 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:25.688+0000 7f6401601680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:59:25.735 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:25.733+0000 7f216124c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:59:25.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.834+0000 7fe117fff700 1 -- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe10405f270 msgr2=0x7fe104061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:25.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.834+0000 7fe117fff700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe10405f270 0x7fe104061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:59:25.865 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:25.863+0000 7f216124c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:59:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.863+0000 7fe11e7ff700 1 -- 172.21.15.33:0/1435841561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe11806ab40 con 0x7fe1181521a0 2023-11-06T21:59:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.864+0000 7fe0fffff700 1 -- 172.21.15.33:0/1435841561 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4491 (secure 0 0 0) 0x7fe10c050330 con 0x7fe1181521a0 2023-11-06T21:59:25.866 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:25.867 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1809147689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1809147689},{"type":"v1","addr":"172.21.15.33:6835","nonce":1809147689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1390207299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1390207299},{"type":"v1","addr":"172.21.15.69:6837","nonce":1390207299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3179984136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3179984136},{"type":"v1","addr":"172.21.15.69:6836","nonce":3179984136}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:03:21.683218+0000","modified":"2023-11-06T21:03:24.689443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7720":{"gid":7720,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/4126832799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4126832799},{"type":"v1","addr":"172.21.15.33:6834","nonce":4126832799}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-06T21:59:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 -- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe10405f270 msgr2=0x7fe104061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:25.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe10405f270 0x7fe104061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:25.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 -- 172.21.15.33:0/1435841561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1181521a0 msgr2=0x7fe1180af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:25.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1181521a0 0x7fe1180af390 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7fe10c011fb0 tx=0x7fe10c016000).stop 2023-11-06T21:59:25.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 -- 172.21.15.33:0/1435841561 shutdown_connections 2023-11-06T21:59:25.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe10405f270 0x7fe104061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:25.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe118155320 0x7fe1180c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe118151dc0 0x7fe1181506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 --2- 172.21.15.33:0/1435841561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1181521a0 0x7fe1180af390 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:25.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 -- 172.21.15.33:0/1435841561 >> 172.21.15.33:0/1435841561 conn(0x7fe11805e1b0 msgr2=0x7fe118156e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 -- 172.21.15.33:0/1435841561 shutdown_connections 2023-11-06T21:59:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:25.866+0000 7fe11e7ff700 1 -- 172.21.15.33:0/1435841561 wait complete. 2023-11-06T21:59:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 60 2023-11-06T21:59:25.886 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:59:25.889 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:25.888+0000 7f6401601680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:59:25.990 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:25.988+0000 7f6401601680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:59:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.086+0000 7f8cf47ff700 1 Processor -- start 2023-11-06T21:59:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.086+0000 7f8cf47ff700 1 -- start start 2023-11-06T21:59:26.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.087+0000 7f8cf47ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8cec0c5d50 0x7f8cec0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:26.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.087+0000 7f8cf47ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cec1554a0 con 0x7f8cec0c1840 2023-11-06T21:59:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.087+0000 7f8cf47ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cec0c66c0 con 0x7f8cec0bfd70 2023-11-06T21:59:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.087+0000 7f8cf47ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cec154960 con 0x7f8cec0c5d50 2023-11-06T21:59:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.087+0000 7f8cf259b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8cec0c1840 0x7f8cec0a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53270/0 (socket says 172.21.15.33:53270) 2023-11-06T21:59:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.087+0000 7f8cf259b700 1 -- 172.21.15.33:0/753444291 learned_addr learned my addr 172.21.15.33:0/753444291 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.088+0000 7f8cf1d9a700 1 --2- 172.21.15.33:0/753444291 >> v2:172.21.15.69:3300/0 conn(0x7f8cec0c5d50 0x7f8cec0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.088+0000 7f8cf1599700 1 -- 172.21.15.33:0/753444291 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326100038 0 0) 0x7f8cec1554a0 con 0x7f8cec0c1840 2023-11-06T21:59:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.088+0000 7f8cf1599700 1 -- 172.21.15.33:0/753444291 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cd80033f0 con 0x7f8cec0c1840 2023-11-06T21:59:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.088+0000 7f8cf1d9a700 1 -- 172.21.15.33:0/753444291 >> v1:172.21.15.69:6789/0 conn(0x7f8cec0bfd70 legacy=0x7f8cec0c0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.088+0000 7f8cf1d9a700 1 -- 172.21.15.33:0/753444291 >> v1:172.21.15.33:6789/0 conn(0x7f8cec0c1840 legacy=0x7f8cec0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.088+0000 7f8cf1d9a700 1 -- 172.21.15.33:0/753444291 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec155930 con 0x7f8cec0c5d50 2023-11-06T21:59:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.088+0000 7f8cf1d9a700 1 --2- 172.21.15.33:0/753444291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c5d50 0x7f8cec0c61d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f8cdc009760 tx=0x7f8cdc009060).ready entity=mon.1 client_cookie=f4ac62fdf0806488 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.089+0000 7f8cf1599700 1 -- 172.21.15.33:0/753444291 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00f510 con 0x7f8cec0c5d50 2023-11-06T21:59:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.089+0000 7f8cf1599700 1 -- 172.21.15.33:0/753444291 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cdc00f670 con 0x7f8cec0c5d50 2023-11-06T21:59:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.089+0000 7f8cf1599700 1 -- 172.21.15.33:0/753444291 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00f9f0 con 0x7f8cec0c5d50 2023-11-06T21:59:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.089+0000 7f8cf47ff700 1 -- 172.21.15.33:0/753444291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c5d50 msgr2=0x7f8cec0c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.089+0000 7f8cf47ff700 1 --2- 172.21.15.33:0/753444291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c5d50 0x7f8cec0c61d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f8cdc009760 tx=0x7f8cdc009060).stop 2023-11-06T21:59:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.089+0000 7f8cf47ff700 1 -- 172.21.15.33:0/753444291 shutdown_connections 2023-11-06T21:59:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.089+0000 7f8cf47ff700 1 --2- 172.21.15.33:0/753444291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c5d50 0x7f8cec0c61d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.089+0000 7f8cf47ff700 1 -- 172.21.15.33:0/753444291 >> 172.21.15.33:0/753444291 conn(0x7f8cec05ec80 msgr2=0x7f8cec0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.090+0000 7f8cf47ff700 1 -- 172.21.15.33:0/753444291 shutdown_connections 2023-11-06T21:59:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.090+0000 7f8cf47ff700 1 -- 172.21.15.33:0/753444291 wait complete. 2023-11-06T21:59:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.091+0000 7f8cf47ff700 1 Processor -- start 2023-11-06T21:59:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.091+0000 7f8cf47ff700 1 -- start start 2023-11-06T21:59:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.092+0000 7f8cf47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cec0bfd70 0x7f8cec149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.092+0000 7f8cf47ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c1840 0x7f8cec14cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.092+0000 7f8cf47ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cec0c5d50 0x7f8cec0c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.092+0000 7f8cf47ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec14d7c0 con 0x7f8cec0c5d50 2023-11-06T21:59:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.092+0000 7f8cf47ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec0c2ef0 con 0x7f8cec0c1840 2023-11-06T21:59:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.092+0000 7f8cf47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec0c31d0 con 0x7f8cec0bfd70 2023-11-06T21:59:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.092+0000 7f8cf2d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cec0c5d50 0x7f8cec0c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.092+0000 7f8cf1d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c1840 0x7f8cec14cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.092+0000 7f8cf1d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c1840 0x7f8cec14cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:40952/0 (socket says 172.21.15.33:40952) 2023-11-06T21:59:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8cf1d9a700 1 -- 172.21.15.33:0/1982092067 learned_addr learned my addr 172.21.15.33:0/1982092067 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8cf259b700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cec0bfd70 0x7f8cec149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8cf1d9a700 1 -- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cec0bfd70 msgr2=0x7f8cec149fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8cf1d9a700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cec0bfd70 0x7f8cec149fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8cf1d9a700 1 -- 172.21.15.33:0/1982092067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cec0c5d50 msgr2=0x7f8cec0c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8cf1d9a700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cec0c5d50 0x7f8cec0c05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8cf1d9a700 1 -- 172.21.15.33:0/1982092067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc0092a0 con 0x7f8cec0c1840 2023-11-06T21:59:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8cf2d9c700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cec0c5d50 0x7f8cec0c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8cf1d9a700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c1840 0x7f8cec14cde0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f8cdc009e00 tx=0x7f8cdc017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8ce27fc700 1 -- 172.21.15.33:0/1982092067 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00f7e0 con 0x7f8cec0c1840 2023-11-06T21:59:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8ce27fc700 1 -- 172.21.15.33:0/1982092067 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cdc00f940 con 0x7f8cec0c1840 2023-11-06T21:59:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8cf47ff700 1 -- 172.21.15.33:0/1982092067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cec0c34b0 con 0x7f8cec0c1840 2023-11-06T21:59:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.093+0000 7f8ce27fc700 1 -- 172.21.15.33:0/1982092067 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc01b690 con 0x7f8cec0c1840 2023-11-06T21:59:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.094+0000 7f8cf47ff700 1 -- 172.21.15.33:0/1982092067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cec0c2820 con 0x7f8cec0c1840 2023-11-06T21:59:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.095+0000 7f8cf47ff700 1 -- 172.21.15.33:0/1982092067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cec0c39d0 con 0x7f8cec0c1840 2023-11-06T21:59:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.096+0000 7f8ce27fc700 1 -- 172.21.15.33:0/1982092067 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 62) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f8cdc01ba40 con 0x7f8cec0c1840 2023-11-06T21:59:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.096+0000 7f8ce27fc700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cd005f270 0x7f8cd0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.096+0000 7f8ce27fc700 1 -- 172.21.15.33:0/1982092067 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8cdc07bbf0 con 0x7f8cec0c1840 2023-11-06T21:59:26.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.096+0000 7f8cf259b700 1 -- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cd005f270 msgr2=0x7f8cd0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:26.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.096+0000 7f8cf259b700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cd005f270 0x7f8cd0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.108+0000 7f8ce27fc700 1 -- 172.21.15.33:0/1982092067 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8cec0c39d0 con 0x7f8cec0c1840 2023-11-06T21:59:26.147 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:26.145+0000 7f6401601680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:59:26.214 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.212+0000 7f216124c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:59:26.286 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:26.284+0000 7f6401601680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:59:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.294+0000 7f8cf47ff700 1 -- 172.21.15.33:0/1982092067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cec0c39d0 con 0x7f8cec0c1840 2023-11-06T21:59:26.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.296+0000 7f8ce27fc700 1 -- 172.21.15.33:0/1982092067 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+17573 (secure 0 0 0) 0x7f8cec0c39d0 con 0x7f8cec0c1840 2023-11-06T21:59:26.298 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:26.299 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":156,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:59:21.974687+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:03:20.146564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T21:03:20.662302+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":156,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2083752177":"2023-11-07T21:03:35.888563+0000","172.21.15.69:6833/114912":"2023-11-07T21:03:35.888563+0000","172.21.15.69:6832/114912":"2023-11-07T21:03:35.888563+0000","172.21.15.69:0/47836582":"2023-11-07T21:03:35.888563+0000","172.21.15.69:0/3836508278":"2023-11-07T21:03:35.888563+0000","172.21.15.69:0/2121315625":"2023-11-07T21:03:35.888563+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":54,"length":4},{"begin":110,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:59:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.297+0000 7f8cf259b700 1 -- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cd005f270 msgr2=0x7f8cd0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.297+0000 7f8cf259b700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cd005f270 0x7f8cd0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:59:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 -- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cd005f270 msgr2=0x7f8cd0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cd005f270 0x7f8cd0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 -- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c1840 msgr2=0x7f8cec14cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c1840 0x7f8cec14cde0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f8cdc009e00 tx=0x7f8cdc017000).stop 2023-11-06T21:59:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 -- 172.21.15.33:0/1982092067 shutdown_connections 2023-11-06T21:59:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8cd005f270 0x7f8cd0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cec0bfd70 0x7f8cec149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cec0c5d50 0x7f8cec0c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 --2- 172.21.15.33:0/1982092067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cec0c1840 0x7f8cec14cde0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 -- 172.21.15.33:0/1982092067 >> 172.21.15.33:0/1982092067 conn(0x7f8cec05ec80 msgr2=0x7f8cec154210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 -- 172.21.15.33:0/1982092067 shutdown_connections 2023-11-06T21:59:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.298+0000 7f8cf47ff700 1 -- 172.21.15.33:0/1982092067 wait complete. 2023-11-06T21:59:26.304 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.302+0000 7f216124c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:59:26.322 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T21:59:26.377 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.375+0000 7f20dc552700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:26.378 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.375+0000 7f20dc552700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:26.378 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.375+0000 7f20dc552700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:26.378 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.375+0000 7f20dc552700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:26.379 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.375+0000 7f20dc552700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:26.379 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.376+0000 7f20d8d4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:26.379 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.376+0000 7f20d8d4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:26.379 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.376+0000 7f20d8d4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:26.379 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.376+0000 7f20d8d4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:26.380 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:26.376+0000 7f20d8d4b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:26.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.528+0000 7f89723ff700 1 Processor -- start 2023-11-06T21:59:26.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.528+0000 7f89723ff700 1 -- start start 2023-11-06T21:59:26.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.530+0000 7f89723ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c0c3b20 0x7f896c156770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:26.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.530+0000 7f89723ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896c155d20 con 0x7f896c14fb70 2023-11-06T21:59:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.530+0000 7f89723ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896c1562e0 con 0x7f896c14ff50 2023-11-06T21:59:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.530+0000 7f89723ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c156c60 con 0x7f896c0c3b20 2023-11-06T21:59:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.530+0000 7f896bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f896c14fb70 0x7f896c152750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53278/0 (socket says 172.21.15.33:53278) 2023-11-06T21:59:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.530+0000 7f896bfff700 1 -- 172.21.15.33:0/4218874972 learned_addr learned my addr 172.21.15.33:0/4218874972 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.530+0000 7f897099c700 1 --2- 172.21.15.33:0/4218874972 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c0c3b20 0x7f896c156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f896affd700 1 -- 172.21.15.33:0/4218874972 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447260225 0 0) 0x7f896c1562e0 con 0x7f896c14ff50 2023-11-06T21:59:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f896affd700 1 -- 172.21.15.33:0/4218874972 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89540033f0 con 0x7f896c14ff50 2023-11-06T21:59:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f896affd700 1 -- 172.21.15.33:0/4218874972 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708141873 0 0) 0x7f896c155d20 con 0x7f896c14fb70 2023-11-06T21:59:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f896affd700 1 -- 172.21.15.33:0/4218874972 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f896c1562e0 con 0x7f896c14fb70 2023-11-06T21:59:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f897099c700 1 -- 172.21.15.33:0/4218874972 >> v1:172.21.15.69:6789/0 conn(0x7f896c14ff50 legacy=0x7f896c0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f897099c700 1 -- 172.21.15.33:0/4218874972 >> v1:172.21.15.33:6789/0 conn(0x7f896c14fb70 legacy=0x7f896c152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f897099c700 1 -- 172.21.15.33:0/4218874972 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c153280 con 0x7f896c0c3b20 2023-11-06T21:59:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f897099c700 1 --2- 172.21.15.33:0/4218874972 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c0c3b20 0x7f896c156770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f8964009e90 tx=0x7f8964009780).ready entity=mon.2 client_cookie=58e6fe6fe6fd520 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f896affd700 1 -- 172.21.15.33:0/4218874972 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896400e070 con 0x7f896c0c3b20 2023-11-06T21:59:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f896affd700 1 -- 172.21.15.33:0/4218874972 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896400bab0 con 0x7f896c0c3b20 2023-11-06T21:59:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.531+0000 7f896affd700 1 -- 172.21.15.33:0/4218874972 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896400be30 con 0x7f896c0c3b20 2023-11-06T21:59:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.532+0000 7f89723ff700 1 -- 172.21.15.33:0/4218874972 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c0c3b20 msgr2=0x7f896c156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.532+0000 7f89723ff700 1 --2- 172.21.15.33:0/4218874972 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c0c3b20 0x7f896c156770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f8964009e90 tx=0x7f8964009780).stop 2023-11-06T21:59:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.532+0000 7f89723ff700 1 -- 172.21.15.33:0/4218874972 shutdown_connections 2023-11-06T21:59:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.532+0000 7f89723ff700 1 --2- 172.21.15.33:0/4218874972 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c0c3b20 0x7f896c156770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.532+0000 7f89723ff700 1 -- 172.21.15.33:0/4218874972 >> 172.21.15.33:0/4218874972 conn(0x7f896c062710 msgr2=0x7f896c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.532+0000 7f89723ff700 1 -- 172.21.15.33:0/4218874972 shutdown_connections 2023-11-06T21:59:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.532+0000 7f89723ff700 1 -- 172.21.15.33:0/4218874972 wait complete. 2023-11-06T21:59:26.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.534+0000 7f89723ff700 1 Processor -- start 2023-11-06T21:59:26.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.534+0000 7f89723ff700 1 -- start start 2023-11-06T21:59:26.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f89723ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f896c0c3b20 0x7f896c0a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:26.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f89723ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c14fb70 0x7f896c154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:26.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f89723ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c14ff50 0x7f896c154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:26.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f89723ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c155210 con 0x7f896c14fb70 2023-11-06T21:59:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f89723ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0c2d20 con 0x7f896c0c3b20 2023-11-06T21:59:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f89723ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0a95f0 con 0x7f896c14ff50 2023-11-06T21:59:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f896b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c14fb70 0x7f896c154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f897099c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c14ff50 0x7f896c154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:26.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f896bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f896c0c3b20 0x7f896c0a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:26.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f897099c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c14ff50 0x7f896c154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:49196/0 (socket says 172.21.15.33:49196) 2023-11-06T21:59:26.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.535+0000 7f897099c700 1 -- 172.21.15.33:0/540748891 learned_addr learned my addr 172.21.15.33:0/540748891 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:26.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.536+0000 7f897099c700 1 -- 172.21.15.33:0/540748891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f896c0c3b20 msgr2=0x7f896c0a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.536+0000 7f897099c700 1 --2- 172.21.15.33:0/540748891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f896c0c3b20 0x7f896c0a9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:26.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.536+0000 7f897099c700 1 -- 172.21.15.33:0/540748891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c14fb70 msgr2=0x7f896c154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:26.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.536+0000 7f897099c700 1 --2- 172.21.15.33:0/540748891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c14fb70 0x7f896c154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:26.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.536+0000 7f897099c700 1 -- 172.21.15.33:0/540748891 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8964009a20 con 0x7f896c14ff50 2023-11-06T21:59:26.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.536+0000 7f897099c700 1 --2- 172.21.15.33:0/540748891 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c14ff50 0x7f896c154d20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f896c05efa0 tx=0x7f896c156250).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:26.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.536+0000 7f8953fff700 1 -- 172.21.15.33:0/540748891 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964013e40 con 0x7f896c14ff50 2023-11-06T21:59:26.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.536+0000 7f8953fff700 1 -- 172.21.15.33:0/540748891 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896401d630 con 0x7f896c14ff50 2023-11-06T21:59:26.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.536+0000 7f8953fff700 1 -- 172.21.15.33:0/540748891 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896401d9f0 con 0x7f896c14ff50 2023-11-06T21:59:26.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.536+0000 7f89723ff700 1 -- 172.21.15.33:0/540748891 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896c0c2110 con 0x7f896c14ff50 2023-11-06T21:59:26.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.537+0000 7f89723ff700 1 -- 172.21.15.33:0/540748891 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c0c26d0 con 0x7f896c14ff50 2023-11-06T21:59:26.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.538+0000 7f89723ff700 1 -- 172.21.15.33:0/540748891 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896c0a9a10 con 0x7f896c14ff50 2023-11-06T21:59:26.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.539+0000 7f8953fff700 1 -- 172.21.15.33:0/540748891 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 63) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f8964003040 con 0x7f896c14ff50 2023-11-06T21:59:26.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.539+0000 7f8953fff700 1 -- 172.21.15.33:0/540748891 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6624+0+0 (secure 0 0 0) 0x7f896407bf10 con 0x7f896c14ff50 2023-11-06T21:59:26.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.544+0000 7f8953fff700 1 -- 172.21.15.33:0/540748891 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f896404d1c0 con 0x7f896c14ff50 2023-11-06T21:59:26.662 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:26.660+0000 7f6401601680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:59:26.758 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:26.756+0000 7f6401601680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:59:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:26.774+0000 7f89723ff700 1 -- 172.21.15.33:0/540748891 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f896c153560 con 0x7f896c14ff50 2023-11-06T21:59:27.309 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T21:59:27.308+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T21:59:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.315+0000 7f8953fff700 1 -- 172.21.15.33:0/540748891 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7f89640507e0 con 0x7f896c14ff50 2023-11-06T21:59:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.318+0000 7f89723ff700 1 -- 172.21.15.33:0/540748891 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c14ff50 msgr2=0x7f896c154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:27.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.318+0000 7f89723ff700 1 --2- 172.21.15.33:0/540748891 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c14ff50 0x7f896c154d20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f896c05efa0 tx=0x7f896c156250).stop 2023-11-06T21:59:27.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.318+0000 7f89723ff700 1 -- 172.21.15.33:0/540748891 shutdown_connections 2023-11-06T21:59:27.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.318+0000 7f89723ff700 1 --2- 172.21.15.33:0/540748891 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f896c14ff50 0x7f896c154d20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:27.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.318+0000 7f89723ff700 1 --2- 172.21.15.33:0/540748891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f896c0c3b20 0x7f896c0a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.318+0000 7f89723ff700 1 --2- 172.21.15.33:0/540748891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c14fb70 0x7f896c154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.318+0000 7f89723ff700 1 -- 172.21.15.33:0/540748891 >> 172.21.15.33:0/540748891 conn(0x7f896c062710 msgr2=0x7f896c064180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.318+0000 7f89723ff700 1 -- 172.21.15.33:0/540748891 shutdown_connections 2023-11-06T21:59:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.318+0000 7f89723ff700 1 -- 172.21.15.33:0/540748891 wait complete. 2023-11-06T21:59:27.323 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T21:59:27.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T21:59:27.381 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T21:59:27.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.542+0000 7fd2439ff700 1 Processor -- start 2023-11-06T21:59:27.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.542+0000 7fd2439ff700 1 -- start start 2023-11-06T21:59:27.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.543+0000 7fd2439ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c157700 0x7fd23c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:27.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.543+0000 7fd2439ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23c158c00 con 0x7fd23c0aaf50 2023-11-06T21:59:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.543+0000 7fd2439ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23c157ff0 con 0x7fd23c063840 2023-11-06T21:59:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.543+0000 7fd2439ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0c18f0 con 0x7fd23c157700 2023-11-06T21:59:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.543+0000 7fd240f9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd23c0aaf50 0x7fd23c063130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53282/0 (socket says 172.21.15.33:53282) 2023-11-06T21:59:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.543+0000 7fd240f9a700 1 -- 172.21.15.33:0/3122247059 learned_addr learned my addr 172.21.15.33:0/3122247059 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.544+0000 7fd24179b700 1 --2- 172.21.15.33:0/3122247059 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c157700 0x7fd23c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.544+0000 7fd233fff700 1 -- 172.21.15.33:0/3122247059 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530363388 0 0) 0x7fd23c158c00 con 0x7fd23c0aaf50 2023-11-06T21:59:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.544+0000 7fd233fff700 1 -- 172.21.15.33:0/3122247059 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2280033f0 con 0x7fd23c0aaf50 2023-11-06T21:59:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.544+0000 7fd233fff700 1 -- 172.21.15.33:0/3122247059 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383376097 0 0) 0x7fd23c157ff0 con 0x7fd23c063840 2023-11-06T21:59:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.544+0000 7fd233fff700 1 -- 172.21.15.33:0/3122247059 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd23c158c00 con 0x7fd23c063840 2023-11-06T21:59:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.544+0000 7fd233fff700 1 -- 172.21.15.33:0/3122247059 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2591101381 0 0) 0x7fd2280033f0 con 0x7fd23c0aaf50 2023-11-06T21:59:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.544+0000 7fd24179b700 1 -- 172.21.15.33:0/3122247059 >> v1:172.21.15.69:6789/0 conn(0x7fd23c063840 legacy=0x7fd23c0c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.544+0000 7fd24179b700 1 -- 172.21.15.33:0/3122247059 >> v1:172.21.15.33:6789/0 conn(0x7fd23c0aaf50 legacy=0x7fd23c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.544+0000 7fd24179b700 1 -- 172.21.15.33:0/3122247059 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c1582d0 con 0x7fd23c157700 2023-11-06T21:59:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.544+0000 7fd24179b700 1 --2- 172.21.15.33:0/3122247059 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c157700 0x7fd23c0c6f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fd238009790 tx=0x7fd238009060).ready entity=mon.2 client_cookie=2bff2c154b522d7f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.545+0000 7fd233fff700 1 -- 172.21.15.33:0/3122247059 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c001470 con 0x7fd23c157700 2023-11-06T21:59:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.545+0000 7fd233fff700 1 -- 172.21.15.33:0/3122247059 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd238011400 con 0x7fd23c157700 2023-11-06T21:59:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.545+0000 7fd233fff700 1 -- 172.21.15.33:0/3122247059 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238011780 con 0x7fd23c157700 2023-11-06T21:59:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.545+0000 7fd2439ff700 1 -- 172.21.15.33:0/3122247059 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c157700 msgr2=0x7fd23c0c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.545+0000 7fd2439ff700 1 --2- 172.21.15.33:0/3122247059 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c157700 0x7fd23c0c6f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fd238009790 tx=0x7fd238009060).stop 2023-11-06T21:59:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.545+0000 7fd2439ff700 1 -- 172.21.15.33:0/3122247059 shutdown_connections 2023-11-06T21:59:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.545+0000 7fd2439ff700 1 --2- 172.21.15.33:0/3122247059 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c157700 0x7fd23c0c6f00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.545+0000 7fd2439ff700 1 -- 172.21.15.33:0/3122247059 >> 172.21.15.33:0/3122247059 conn(0x7fd23c05ec80 msgr2=0x7fd23c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.545+0000 7fd2439ff700 1 -- 172.21.15.33:0/3122247059 shutdown_connections 2023-11-06T21:59:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.545+0000 7fd2439ff700 1 -- 172.21.15.33:0/3122247059 wait complete. 2023-11-06T21:59:27.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.547+0000 7fd2439ff700 1 Processor -- start 2023-11-06T21:59:27.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.547+0000 7fd2439ff700 1 -- start start 2023-11-06T21:59:27.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd2439ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd23c063840 0x7fd23c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:27.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd2439ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c0aaf50 0x7fd23c0c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:27.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd2439ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd23c157700 0x7fd23c0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:27.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd2439ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c14b640 con 0x7fd23c157700 2023-11-06T21:59:27.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd2439ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0ae350 con 0x7fd23c063840 2023-11-06T21:59:27.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd2439ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0ae630 con 0x7fd23c0aaf50 2023-11-06T21:59:27.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd241f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd23c157700 0x7fd23c0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:27.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd241f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd23c157700 0x7fd23c0c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35970/0 (socket says 172.21.15.33:35970) 2023-11-06T21:59:27.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd241f9c700 1 -- 172.21.15.33:0/3123639494 learned_addr learned my addr 172.21.15.33:0/3123639494 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:27.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd24179b700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd23c063840 0x7fd23c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.548+0000 7fd240f9a700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c0aaf50 0x7fd23c0c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd241f9c700 1 -- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c0aaf50 msgr2=0x7fd23c0c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd241f9c700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c0aaf50 0x7fd23c0c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd241f9c700 1 -- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd23c063840 msgr2=0x7fd23c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd241f9c700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd23c063840 0x7fd23c14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd241f9c700 1 -- 172.21.15.33:0/3123639494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380092a0 con 0x7fd23c157700 2023-11-06T21:59:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd241f9c700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd23c157700 0x7fd23c0c8c30 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7fd23400c7c0 tx=0x7fd234007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd2317fa700 1 -- 172.21.15.33:0/3123639494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400a520 con 0x7fd23c157700 2023-11-06T21:59:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd2317fa700 1 -- 172.21.15.33:0/3123639494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23400a680 con 0x7fd23c157700 2023-11-06T21:59:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd2439ff700 1 -- 172.21.15.33:0/3123639494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c0ae910 con 0x7fd23c157700 2023-11-06T21:59:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd2317fa700 1 -- 172.21.15.33:0/3123639494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400aa20 con 0x7fd23c157700 2023-11-06T21:59:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.549+0000 7fd2439ff700 1 -- 172.21.15.33:0/3123639494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23c0aeed0 con 0x7fd23c157700 2023-11-06T21:59:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.550+0000 7fd2439ff700 1 -- 172.21.15.33:0/3123639494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23c0af0f0 con 0x7fd23c157700 2023-11-06T21:59:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.551+0000 7fd2317fa700 1 -- 172.21.15.33:0/3123639494 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 64) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fd23401b400 con 0x7fd23c157700 2023-11-06T21:59:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.552+0000 7fd2317fa700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd22005f000 0x7fd2200614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.552+0000 7fd2317fa700 1 -- 172.21.15.33:0/3123639494 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6710+0+0 (secure 0 0 0) 0x7fd23407a170 con 0x7fd23c157700 2023-11-06T21:59:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.553+0000 7fd24179b700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd22005f000 0x7fd2200614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.553+0000 7fd24179b700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd22005f000 0x7fd2200614c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fd2380096e0 tx=0x7fd238016000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.557+0000 7fd2317fa700 1 -- 172.21.15.33:0/3123639494 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd23404b250 con 0x7fd23c157700 2023-11-06T21:59:27.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:27.779+0000 7fd2439ff700 1 -- 172.21.15.33:0/3123639494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd23c06c6c0 con 0x7fd23c157700 2023-11-06T21:59:28.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.325+0000 7fd2317fa700 1 -- 172.21.15.33:0/3123639494 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd23404e870 con 0x7fd23c157700 2023-11-06T21:59:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 -- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd22005f000 msgr2=0x7fd2200614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd22005f000 0x7fd2200614c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fd2380096e0 tx=0x7fd238016000).stop 2023-11-06T21:59:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 -- 172.21.15.33:0/3123639494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd23c157700 msgr2=0x7fd23c0c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd23c157700 0x7fd23c0c8c30 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7fd23400c7c0 tx=0x7fd234007f50).stop 2023-11-06T21:59:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 -- 172.21.15.33:0/3123639494 shutdown_connections 2023-11-06T21:59:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd22005f000 0x7fd2200614c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd23c157700 0x7fd23c0c8c30 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd23c063840 0x7fd23c14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 --2- 172.21.15.33:0/3123639494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd23c0aaf50 0x7fd23c0c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 -- 172.21.15.33:0/3123639494 >> 172.21.15.33:0/3123639494 conn(0x7fd23c05ec80 msgr2=0x7fd23c0c9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.327+0000 7fd2439ff700 1 -- 172.21.15.33:0/3123639494 shutdown_connections 2023-11-06T21:59:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.328+0000 7fd2439ff700 1 -- 172.21.15.33:0/3123639494 wait complete. 2023-11-06T21:59:28.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T21:59:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.590+0000 7f53423ff700 1 Processor -- start 2023-11-06T21:59:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.590+0000 7f53423ff700 1 -- start start 2023-11-06T21:59:28.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f53423ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f533c0acfc0 0x7f533c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:28.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f53423ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 0x7f533c0c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:28.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f53423ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533c1586a0 con 0x7f533c0631a0 2023-11-06T21:59:28.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f53423ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533c0c6590 con 0x7f533c0acfc0 2023-11-06T21:59:28.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f53423ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1544c0 con 0x7f533c1550d0 2023-11-06T21:59:28.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f533bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f533c0acfc0 0x7f533c0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f533bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f533c0acfc0 0x7f533c0ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41406/0 (socket says 172.21.15.33:41406) 2023-11-06T21:59:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f533bfff700 1 -- 172.21.15.33:0/761138873 learned_addr learned my addr 172.21.15.33:0/761138873 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f534099c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f533c0631a0 0x7f533c064680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40800/0 (socket says 172.21.15.33:40800) 2023-11-06T21:59:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f533b7fe700 1 --2- 172.21.15.33:0/761138873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 0x7f533c0c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f533bfff700 1 -- 172.21.15.33:0/761138873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 msgr2=0x7f533c0c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f533bfff700 1 --2- 172.21.15.33:0/761138873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 0x7f533c0c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f533bfff700 1 -- 172.21.15.33:0/761138873 >> v1:172.21.15.33:6789/0 conn(0x7f533c0631a0 legacy=0x7f533c064680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f533bfff700 1 -- 172.21.15.33:0/761138873 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c155550 con 0x7f533c0acfc0 2023-11-06T21:59:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.592+0000 7f533b7fe700 1 --2- 172.21.15.33:0/761138873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 0x7f533c0c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.593+0000 7f533bfff700 1 --2- 172.21.15.33:0/761138873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 0x7f533c0ad3a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f532c006fd0 tx=0x7f532c0090f0).ready entity=mon.0 client_cookie=730902e4dd17c370 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:28.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.593+0000 7f533affd700 1 -- 172.21.15.33:0/761138873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c00b600 con 0x7f533c0acfc0 2023-11-06T21:59:28.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.593+0000 7f533affd700 1 -- 172.21.15.33:0/761138873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c00b760 con 0x7f533c0acfc0 2023-11-06T21:59:28.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.593+0000 7f533affd700 1 -- 172.21.15.33:0/761138873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c00bae0 con 0x7f533c0acfc0 2023-11-06T21:59:28.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.593+0000 7f53423ff700 1 -- 172.21.15.33:0/761138873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 msgr2=0x7f533c0ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:28.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.593+0000 7f53423ff700 1 --2- 172.21.15.33:0/761138873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 0x7f533c0ad3a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f532c006fd0 tx=0x7f532c0090f0).stop 2023-11-06T21:59:28.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.593+0000 7f53423ff700 1 -- 172.21.15.33:0/761138873 shutdown_connections 2023-11-06T21:59:28.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.593+0000 7f53423ff700 1 --2- 172.21.15.33:0/761138873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 0x7f533c0ad3a0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:28.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.593+0000 7f53423ff700 1 --2- 172.21.15.33:0/761138873 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 0x7f533c0c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:28.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.593+0000 7f53423ff700 1 -- 172.21.15.33:0/761138873 >> 172.21.15.33:0/761138873 conn(0x7f533c05ec80 msgr2=0x7f533c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:28.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.594+0000 7f53423ff700 1 -- 172.21.15.33:0/761138873 shutdown_connections 2023-11-06T21:59:28.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.594+0000 7f53423ff700 1 -- 172.21.15.33:0/761138873 wait complete. 2023-11-06T21:59:28.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.595+0000 7f53423ff700 1 Processor -- start 2023-11-06T21:59:28.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.595+0000 7f53423ff700 1 -- start start 2023-11-06T21:59:28.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.596+0000 7f53423ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f533c0631a0 0x7f533c0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:28.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.596+0000 7f53423ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 0x7f533c14b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:28.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.596+0000 7f53423ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 0x7f533c14ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:28.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.596+0000 7f53423ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c0ae130 con 0x7f533c0acfc0 2023-11-06T21:59:28.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.596+0000 7f53423ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c0c55e0 con 0x7f533c0631a0 2023-11-06T21:59:28.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.596+0000 7f53423ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c151000 con 0x7f533c1550d0 2023-11-06T21:59:28.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.596+0000 7f533b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 0x7f533c14b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:28.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.596+0000 7f533b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 0x7f533c14b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41412/0 (socket says 172.21.15.33:41412) 2023-11-06T21:59:28.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.596+0000 7f533b7fe700 1 -- 172.21.15.33:0/2317332906 learned_addr learned my addr 172.21.15.33:0/2317332906 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f533bfff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f533c0631a0 0x7f533c0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f534099c700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 0x7f533c14ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f533b7fe700 1 -- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 msgr2=0x7f533c14ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f533b7fe700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 0x7f533c14ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f533b7fe700 1 -- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f533c0631a0 msgr2=0x7f533c0c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:28.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f533b7fe700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f533c0631a0 0x7f533c0c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:28.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f533b7fe700 1 -- 172.21.15.33:0/2317332906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c009370 con 0x7f533c0acfc0 2023-11-06T21:59:28.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f534099c700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 0x7f533c14ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:28.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f533bfff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f533c0631a0 0x7f533c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:28.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f533b7fe700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 0x7f533c14b450 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f533000c880 tx=0x7f5330008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:28.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f5323fff700 1 -- 172.21.15.33:0/2317332906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330010580 con 0x7f533c0acfc0 2023-11-06T21:59:28.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f5323fff700 1 -- 172.21.15.33:0/2317332906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53300106e0 con 0x7f533c0acfc0 2023-11-06T21:59:28.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f53423ff700 1 -- 172.21.15.33:0/2317332906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c1512e0 con 0x7f533c0acfc0 2023-11-06T21:59:28.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.597+0000 7f5323fff700 1 -- 172.21.15.33:0/2317332906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330010a80 con 0x7f533c0acfc0 2023-11-06T21:59:28.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.598+0000 7f53423ff700 1 -- 172.21.15.33:0/2317332906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c0addc0 con 0x7f533c0acfc0 2023-11-06T21:59:28.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.599+0000 7f53423ff700 1 -- 172.21.15.33:0/2317332906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c06c6c0 con 0x7f533c0acfc0 2023-11-06T21:59:28.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.599+0000 7f5323fff700 1 -- 172.21.15.33:0/2317332906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 64) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f533001b460 con 0x7f533c0acfc0 2023-11-06T21:59:28.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.600+0000 7f5323fff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f532805f000 0x7f53280614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:28.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.601+0000 7f5323fff700 1 -- 172.21.15.33:0/2317332906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6710+0+0 (secure 0 0 0) 0x7f533007a2e0 con 0x7f533c0acfc0 2023-11-06T21:59:28.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.601+0000 7f533bfff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f532805f000 0x7f53280614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:28.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.601+0000 7f533bfff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f532805f000 0x7f53280614c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f532c009130 tx=0x7f532c017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:28.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.605+0000 7f5323fff700 1 -- 172.21.15.33:0/2317332906 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f533004b3c0 con 0x7f533c0acfc0 2023-11-06T21:59:28.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:28.785+0000 7f53423ff700 1 -- 172.21.15.33:0/2317332906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f533c0adf50 con 0x7f533c0acfc0 2023-11-06T21:59:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.006+0000 7f5323fff700 1 -- 172.21.15.33:0/2317332906 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5330077f20 con 0x7f533c0acfc0 2023-11-06T21:59:29.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.330+0000 7f5323fff700 1 -- 172.21.15.33:0/2317332906 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v160) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5330073030 con 0x7f533c0acfc0 2023-11-06T21:59:29.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 -- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f532805f000 msgr2=0x7f53280614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:29.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f532805f000 0x7f53280614c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f532c009130 tx=0x7f532c017000).stop 2023-11-06T21:59:29.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 -- 172.21.15.33:0/2317332906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 msgr2=0x7f533c14b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:29.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 0x7f533c14b450 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f533000c880 tx=0x7f5330008040).stop 2023-11-06T21:59:29.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 -- 172.21.15.33:0/2317332906 shutdown_connections 2023-11-06T21:59:29.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f532805f000 0x7f53280614c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:29.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f533c1550d0 0x7f533c14ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:29.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f533c0631a0 0x7f533c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:29.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 --2- 172.21.15.33:0/2317332906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f533c0acfc0 0x7f533c14b450 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:29.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 -- 172.21.15.33:0/2317332906 >> 172.21.15.33:0/2317332906 conn(0x7f533c05ec80 msgr2=0x7f533c0c0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:29.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 -- 172.21.15.33:0/2317332906 shutdown_connections 2023-11-06T21:59:29.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.332+0000 7f53423ff700 1 -- 172.21.15.33:0/2317332906 wait complete. 2023-11-06T21:59:29.337 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T21:59:29.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T21:59:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.554+0000 7f98849ff700 1 Processor -- start 2023-11-06T21:59:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.554+0000 7f98849ff700 1 -- start start 2023-11-06T21:59:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.556+0000 7f98849ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f988013a960 0x7f9880155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.556+0000 7f98849ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f98800c9340 0x7f9880066490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.556+0000 7f98849ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9880066e30 con 0x7f9880066980 2023-11-06T21:59:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.556+0000 7f98849ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880064cc0 con 0x7f98800c9340 2023-11-06T21:59:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.556+0000 7f98849ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9880155930 con 0x7f988013a960 2023-11-06T21:59:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.556+0000 7f987e59c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f988013a960 0x7f9880155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.556+0000 7f987dd9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f98800c9340 0x7f9880066490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.556+0000 7f987dd9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f98800c9340 0x7f9880066490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:44778/0 (socket says 172.21.15.33:44778) 2023-11-06T21:59:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.556+0000 7f987dd9b700 1 -- 172.21.15.33:0/1170948205 learned_addr learned my addr 172.21.15.33:0/1170948205 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.557+0000 7f987e59c700 1 -- 172.21.15.33:0/1170948205 >> v2:172.21.15.69:3300/0 conn(0x7f98800c9340 msgr2=0x7f9880066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.557+0000 7f987e59c700 1 --2- 172.21.15.33:0/1170948205 >> v2:172.21.15.69:3300/0 conn(0x7f98800c9340 0x7f9880066490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.557+0000 7f987e59c700 1 -- 172.21.15.33:0/1170948205 >> v1:172.21.15.69:6789/0 conn(0x7f9880066980 legacy=0x7f98800679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.557+0000 7f987e59c700 1 -- 172.21.15.33:0/1170948205 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9880063950 con 0x7f988013a960 2023-11-06T21:59:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.557+0000 7f987dd9b700 1 --2- 172.21.15.33:0/1170948205 >> v2:172.21.15.69:3300/0 conn(0x7f98800c9340 0x7f9880066490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.557+0000 7f987e59c700 1 --2- 172.21.15.33:0/1170948205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f988013a960 0x7f9880155380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f9870009760 tx=0x7f9870009060).ready entity=mon.2 client_cookie=30f3089283a6764 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f987d59a700 1 -- 172.21.15.33:0/1170948205 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987000f510 con 0x7f988013a960 2023-11-06T21:59:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f987d59a700 1 -- 172.21.15.33:0/1170948205 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987000f670 con 0x7f988013a960 2023-11-06T21:59:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f987d59a700 1 -- 172.21.15.33:0/1170948205 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987000f9f0 con 0x7f988013a960 2023-11-06T21:59:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f98849ff700 1 -- 172.21.15.33:0/1170948205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f988013a960 msgr2=0x7f9880155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f98849ff700 1 --2- 172.21.15.33:0/1170948205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f988013a960 0x7f9880155380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f9870009760 tx=0x7f9870009060).stop 2023-11-06T21:59:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f98849ff700 1 -- 172.21.15.33:0/1170948205 shutdown_connections 2023-11-06T21:59:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f98849ff700 1 --2- 172.21.15.33:0/1170948205 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f988013a960 0x7f9880155380 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f98849ff700 1 --2- 172.21.15.33:0/1170948205 >> v2:172.21.15.69:3300/0 conn(0x7f98800c9340 0x7f9880066490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f98849ff700 1 -- 172.21.15.33:0/1170948205 >> 172.21.15.33:0/1170948205 conn(0x7f988005ec70 msgr2=0x7f98800c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f98849ff700 1 -- 172.21.15.33:0/1170948205 shutdown_connections 2023-11-06T21:59:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.558+0000 7f98849ff700 1 -- 172.21.15.33:0/1170948205 wait complete. 2023-11-06T21:59:29.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.560+0000 7f98849ff700 1 Processor -- start 2023-11-06T21:59:29.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.560+0000 7f98849ff700 1 -- start start 2023-11-06T21:59:29.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f98849ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9880066980 0x7f98800a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:29.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f98849ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98800c9340 0x7f98800a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:29.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f98849ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f988013a960 0x7f98801398b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:29.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f98849ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98800a50c0 con 0x7f98800c9340 2023-11-06T21:59:29.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f98849ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98800a53a0 con 0x7f988013a960 2023-11-06T21:59:29.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f98849ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98800a5680 con 0x7f9880066980 2023-11-06T21:59:29.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98800c9340 0x7f98800a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98800c9340 0x7f98800a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41424/0 (socket says 172.21.15.33:41424) 2023-11-06T21:59:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987dd9b700 1 -- 172.21.15.33:0/3105057033 learned_addr learned my addr 172.21.15.33:0/3105057033 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987e59c700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9880066980 0x7f98800a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987ed9d700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f988013a960 0x7f98801398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987dd9b700 1 -- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9880066980 msgr2=0x7f98800a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987dd9b700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9880066980 0x7f98800a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987dd9b700 1 -- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f988013a960 msgr2=0x7f98801398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987dd9b700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f988013a960 0x7f98801398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987dd9b700 1 -- 172.21.15.33:0/3105057033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9874009370 con 0x7f98800c9340 2023-11-06T21:59:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987e59c700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9880066980 0x7f98800a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.561+0000 7f987ed9d700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f988013a960 0x7f98801398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.562+0000 7f987dd9b700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98800c9340 0x7f98800a48e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f987400ea20 tx=0x7f98740090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:29.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.562+0000 7f986e7fc700 1 -- 172.21.15.33:0/3105057033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987400c7f0 con 0x7f98800c9340 2023-11-06T21:59:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.562+0000 7f986e7fc700 1 -- 172.21.15.33:0/3105057033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987400c950 con 0x7f98800c9340 2023-11-06T21:59:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.562+0000 7f98849ff700 1 -- 172.21.15.33:0/3105057033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98700092a0 con 0x7f98800c9340 2023-11-06T21:59:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.562+0000 7f986e7fc700 1 -- 172.21.15.33:0/3105057033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987400ccd0 con 0x7f98800c9340 2023-11-06T21:59:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.562+0000 7f98849ff700 1 -- 172.21.15.33:0/3105057033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98800a5c40 con 0x7f98800c9340 2023-11-06T21:59:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.563+0000 7f98849ff700 1 -- 172.21.15.33:0/3105057033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98800a5fb0 con 0x7f98800c9340 2023-11-06T21:59:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.564+0000 7f986e7fc700 1 -- 172.21.15.33:0/3105057033 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9874012600 con 0x7f98800c9340 2023-11-06T21:59:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.565+0000 7f986e7fc700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f986005f270 0x7f9860061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.565+0000 7f986e7fc700 1 -- 172.21.15.33:0/3105057033 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f987407ab70 con 0x7f98800c9340 2023-11-06T21:59:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.565+0000 7f987e59c700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f986005f270 0x7f9860061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.566+0000 7f987e59c700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f986005f270 0x7f9860061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f9870009fd0 tx=0x7f9870017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.569+0000 7f986e7fc700 1 -- 172.21.15.33:0/3105057033 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f987404bb80 con 0x7f98800c9340 2023-11-06T21:59:29.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:29.750+0000 7f98849ff700 1 -- 172.21.15.33:0/3105057033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9880065f30 con 0x7f98800c9340 2023-11-06T21:59:30.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.332+0000 7f986e7fc700 1 -- 172.21.15.33:0/3105057033 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v161) v1 ==== 169+0+0 (secure 0 0 0) 0x7f987404f1a0 con 0x7f98800c9340 2023-11-06T21:59:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 -- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f986005f270 msgr2=0x7f9860061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f986005f270 0x7f9860061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f9870009fd0 tx=0x7f9870017000).stop 2023-11-06T21:59:30.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 -- 172.21.15.33:0/3105057033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98800c9340 msgr2=0x7f98800a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98800c9340 0x7f98800a48e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f987400ea20 tx=0x7f98740090f0).stop 2023-11-06T21:59:30.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 -- 172.21.15.33:0/3105057033 shutdown_connections 2023-11-06T21:59:30.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f986005f270 0x7f9860061730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9880066980 0x7f98800a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f988013a960 0x7f98801398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 --2- 172.21.15.33:0/3105057033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98800c9340 0x7f98800a48e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 -- 172.21.15.33:0/3105057033 >> 172.21.15.33:0/3105057033 conn(0x7f988005ec70 msgr2=0x7f98800a9360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 -- 172.21.15.33:0/3105057033 shutdown_connections 2023-11-06T21:59:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.335+0000 7f98849ff700 1 -- 172.21.15.33:0/3105057033 wait complete. 2023-11-06T21:59:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T21:59:30.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T21:59:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.551+0000 7f2c5e9ff700 1 Processor -- start 2023-11-06T21:59:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.551+0000 7f2c5e9ff700 1 -- start start 2023-11-06T21:59:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.553+0000 7f2c5e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c58154720 0x7f2c58063150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.553+0000 7f2c5e9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c58158ed0 con 0x7f2c58154340 2023-11-06T21:59:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.553+0000 7f2c5e9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c58063ab0 con 0x7f2c58063640 2023-11-06T21:59:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.553+0000 7f2c5e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c581580b0 con 0x7f2c58154720 2023-11-06T21:59:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.553+0000 7f2c57fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2c58154340 0x7f2c580a9ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40812/0 (socket says 172.21.15.33:40812) 2023-11-06T21:59:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.553+0000 7f2c57fff700 1 -- 172.21.15.33:0/2869237300 learned_addr learned my addr 172.21.15.33:0/2869237300 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c577fe700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c58154720 0x7f2c58063150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202462713 0 0) 0x7f2c58158ed0 con 0x7f2c58154340 2023-11-06T21:59:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c400033f0 con 0x7f2c58154340 2023-11-06T21:59:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4271380405 0 0) 0x7f2c400033f0 con 0x7f2c58154340 2023-11-06T21:59:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c58154720 msgr2=0x7f2c58063150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c58154720 0x7f2c58063150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 >> v1:172.21.15.69:6789/0 conn(0x7f2c58063640 legacy=0x7f2c580c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c581582c0 con 0x7f2c58154340 2023-11-06T21:59:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f2c48001470 con 0x7f2c58154340 2023-11-06T21:59:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 >> v1:172.21.15.33:6789/0 conn(0x7f2c58154340 legacy=0x7f2c580a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c40003910 0x7f2c40003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c40004220 0x7f2c400046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c40004ea0 0x7f2c4000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c581580b0 con 0x7f2c40004ea0 2023-11-06T21:59:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4000bdd0 con 0x7f2c40004220 2023-11-06T21:59:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.554+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4000bf10 con 0x7f2c40003910 2023-11-06T21:59:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c577fe700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c58154720 0x7f2c58063150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c5cf9c700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c40004ea0 0x7f2c4000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c57fff700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c40003910 0x7f2c40003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c577fe700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c40004220 0x7f2c400046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c5cf9c700 1 -- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c40003910 msgr2=0x7f2c40003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c5cf9c700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c40003910 0x7f2c40003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c5cf9c700 1 -- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c40004220 msgr2=0x7f2c400046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c5cf9c700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c40004220 0x7f2c400046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c5cf9c700 1 -- 172.21.15.33:0/2869237300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c400033f0 con 0x7f2c40004ea0 2023-11-06T21:59:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c577fe700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c40004220 0x7f2c400046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.555+0000 7f2c5cf9c700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c40004ea0 0x7f2c4000b8e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f2c50001f20 tx=0x7f2c50007f10).ready entity=mon.0 client_cookie=46d5145c030445d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.556+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c500017e0 con 0x7f2c40004ea0 2023-11-06T21:59:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.556+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5000c6d0 con 0x7f2c40004ea0 2023-11-06T21:59:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.556+0000 7f2c56ffd700 1 -- 172.21.15.33:0/2869237300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5000ca50 con 0x7f2c40004ea0 2023-11-06T21:59:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.556+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/2869237300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c40004ea0 msgr2=0x7f2c4000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.556+0000 7f2c5e9ff700 1 --2- 172.21.15.33:0/2869237300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c40004ea0 0x7f2c4000b8e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f2c50001f20 tx=0x7f2c50007f10).stop 2023-11-06T21:59:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.556+0000 7f2c5cf9c700 1 -- 172.21.15.33:0/2869237300 reap_dead start 2023-11-06T21:59:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.556+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/2869237300 shutdown_connections 2023-11-06T21:59:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.556+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/2869237300 >> 172.21.15.33:0/2869237300 conn(0x7f2c5805ec80 msgr2=0x7f2c580cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.556+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/2869237300 shutdown_connections 2023-11-06T21:59:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.556+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/2869237300 wait complete. 2023-11-06T21:59:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.558+0000 7f2c5e9ff700 1 Processor -- start 2023-11-06T21:59:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.558+0000 7f2c5e9ff700 1 -- start start 2023-11-06T21:59:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c5e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c40004ea0 0x7f2c58156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c5e9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c581571d0 0x7f2c58167f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c5e9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c58164f80 0x7f2c58168460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c5e9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c58164af0 con 0x7f2c581571d0 2023-11-06T21:59:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c5e9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c581579b0 con 0x7f2c58164f80 2023-11-06T21:59:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c5e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c581576d0 con 0x7f2c40004ea0 2023-11-06T21:59:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c581571d0 0x7f2c58167f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c581571d0 0x7f2c58167f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41436/0 (socket says 172.21.15.33:41436) 2023-11-06T21:59:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c577fe700 1 -- 172.21.15.33:0/3304576846 learned_addr learned my addr 172.21.15.33:0/3304576846 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c57fff700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c40004ea0 0x7f2c58156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c5cf9c700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c58164f80 0x7f2c58168460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c577fe700 1 -- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c40004ea0 msgr2=0x7f2c58156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c577fe700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c40004ea0 0x7f2c58156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c577fe700 1 -- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c58164f80 msgr2=0x7f2c58168460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c577fe700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c58164f80 0x7f2c58168460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c577fe700 1 -- 172.21.15.33:0/3304576846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c581644a0 con 0x7f2c581571d0 2023-11-06T21:59:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.559+0000 7f2c5cf9c700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c58164f80 0x7f2c58168460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.560+0000 7f2c577fe700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c581571d0 0x7f2c58167f70 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f2c4c011970 tx=0x7f2c4c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.560+0000 7f2c3ffff700 1 -- 172.21.15.33:0/3304576846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c00f6d0 con 0x7f2c581571d0 2023-11-06T21:59:30.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.560+0000 7f2c3ffff700 1 -- 172.21.15.33:0/3304576846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c00f830 con 0x7f2c581571d0 2023-11-06T21:59:30.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.560+0000 7f2c3ffff700 1 -- 172.21.15.33:0/3304576846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c00fbb0 con 0x7f2c581571d0 2023-11-06T21:59:30.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.560+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/3304576846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c58164780 con 0x7f2c581571d0 2023-11-06T21:59:30.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.560+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/3304576846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c580c99b0 con 0x7f2c581571d0 2023-11-06T21:59:30.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.561+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/3304576846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c58164910 con 0x7f2c581571d0 2023-11-06T21:59:30.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.562+0000 7f2c3ffff700 1 -- 172.21.15.33:0/3304576846 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2c4c015480 con 0x7f2c581571d0 2023-11-06T21:59:30.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.563+0000 7f2c3ffff700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c4405f240 0x7f2c44061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.563+0000 7f2c3ffff700 1 -- 172.21.15.33:0/3304576846 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f2c4c07ac40 con 0x7f2c581571d0 2023-11-06T21:59:30.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.563+0000 7f2c57fff700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c4405f240 0x7f2c44061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.564+0000 7f2c57fff700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c4405f240 0x7f2c44061700 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2c581564d0 tx=0x7f2c48001e70).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:30.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.567+0000 7f2c3ffff700 1 -- 172.21.15.33:0/3304576846 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2c4c04bd90 con 0x7f2c581571d0 2023-11-06T21:59:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.747+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/3304576846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f2c58164c30 con 0x7f2c581571d0 2023-11-06T21:59:30.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.751+0000 7f2c3ffff700 1 -- 172.21.15.33:0/3304576846 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v28) v1 ==== 120+0+0 (secure 0 0 0) 0x7f2c4c04f3b0 con 0x7f2c581571d0 2023-11-06T21:59:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c4405f240 msgr2=0x7f2c44061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c4405f240 0x7f2c44061700 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2c581564d0 tx=0x7f2c48001e70).stop 2023-11-06T21:59:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/3304576846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c581571d0 msgr2=0x7f2c58167f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c581571d0 0x7f2c58167f70 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f2c4c011970 tx=0x7f2c4c008040).stop 2023-11-06T21:59:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/3304576846 shutdown_connections 2023-11-06T21:59:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2c4405f240 0x7f2c44061700 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2c40004ea0 0x7f2c58156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c58164f80 0x7f2c58168460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 --2- 172.21.15.33:0/3304576846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c581571d0 0x7f2c58167f70 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/3304576846 >> 172.21.15.33:0/3304576846 conn(0x7f2c5805ec80 msgr2=0x7f2c580ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:30.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/3304576846 shutdown_connections 2023-11-06T21:59:30.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.753+0000 7f2c5e9ff700 1 -- 172.21.15.33:0/3304576846 wait complete. 2023-11-06T21:59:30.783 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T21:59:30.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.980+0000 7fa59f1ff700 1 Processor -- start 2023-11-06T21:59:30.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.980+0000 7fa59f1ff700 1 -- start start 2023-11-06T21:59:30.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.981+0000 7fa59f1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa598155410 0x7fa5980644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.981+0000 7fa59f1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa5981547d0 0x7fa5980649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.981+0000 7fa59f1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa598063c00 con 0x7fa598155030 2023-11-06T21:59:30.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.981+0000 7fa59f1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa598064090 con 0x7fa598155410 2023-11-06T21:59:30.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.981+0000 7fa59f1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa598064ed0 con 0x7fa5981547d0 2023-11-06T21:59:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa597fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa598155410 0x7fa5980644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa597fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa598155410 0x7fa5980644f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41442/0 (socket says 172.21.15.33:41442) 2023-11-06T21:59:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa59cf9b700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fa598155030 0x7fa598062710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:51502/0 (socket says 172.21.15.33:51502) 2023-11-06T21:59:30.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa597fff700 1 -- 172.21.15.33:0/683087330 learned_addr learned my addr 172.21.15.33:0/683087330 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:30.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa59d79c700 1 --2- 172.21.15.33:0/683087330 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa5981547d0 0x7fa5980649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa597fff700 1 -- 172.21.15.33:0/683087330 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa5981547d0 msgr2=0x7fa5980649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:30.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa597fff700 1 --2- 172.21.15.33:0/683087330 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa5981547d0 0x7fa5980649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa597fff700 1 -- 172.21.15.33:0/683087330 >> v1:172.21.15.69:6789/0 conn(0x7fa598155030 legacy=0x7fa598062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa597fff700 1 -- 172.21.15.33:0/683087330 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa598062ff0 con 0x7fa598155410 2023-11-06T21:59:30.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa5977fe700 1 -- 172.21.15.33:0/683087330 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498390630 0 0) 0x7fa598063c00 con 0x7fa598155030 2023-11-06T21:59:30.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.982+0000 7fa597fff700 1 --2- 172.21.15.33:0/683087330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 0x7fa5980644f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fa584006fd0 tx=0x7fa5840090f0).ready entity=mon.0 client_cookie=215aa97508ec8055 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:30.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa5977fe700 1 -- 172.21.15.33:0/683087330 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400b580 con 0x7fa598155410 2023-11-06T21:59:30.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa5977fe700 1 -- 172.21.15.33:0/683087330 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58400b6e0 con 0x7fa598155410 2023-11-06T21:59:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa5977fe700 1 -- 172.21.15.33:0/683087330 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400ba60 con 0x7fa598155410 2023-11-06T21:59:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa59f1ff700 1 -- 172.21.15.33:0/683087330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 msgr2=0x7fa5980644f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa59f1ff700 1 --2- 172.21.15.33:0/683087330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 0x7fa5980644f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fa584006fd0 tx=0x7fa5840090f0).stop 2023-11-06T21:59:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa59f1ff700 1 -- 172.21.15.33:0/683087330 shutdown_connections 2023-11-06T21:59:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa59f1ff700 1 --2- 172.21.15.33:0/683087330 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa5981547d0 0x7fa5980649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa59f1ff700 1 --2- 172.21.15.33:0/683087330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 0x7fa5980644f0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa59f1ff700 1 -- 172.21.15.33:0/683087330 >> 172.21.15.33:0/683087330 conn(0x7fa5980c3d30 msgr2=0x7fa5980c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:30.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa59f1ff700 1 -- 172.21.15.33:0/683087330 shutdown_connections 2023-11-06T21:59:30.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.983+0000 7fa59f1ff700 1 -- 172.21.15.33:0/683087330 wait complete. 2023-11-06T21:59:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.985+0000 7fa59f1ff700 1 Processor -- start 2023-11-06T21:59:30.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.985+0000 7fa59f1ff700 1 -- start start 2023-11-06T21:59:30.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.986+0000 7fa59f1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa5981547d0 0x7fa59813d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.986+0000 7fa59f1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa598155030 0x7fa5980aa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.986+0000 7fa59f1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 0x7fa598167c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.986+0000 7fa59f1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59813bc10 con 0x7fa598155410 2023-11-06T21:59:30.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.986+0000 7fa59f1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980bfc20 con 0x7fa598155030 2023-11-06T21:59:30.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.986+0000 7fa59f1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980ac9a0 con 0x7fa5981547d0 2023-11-06T21:59:30.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.986+0000 7fa59d79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 0x7fa598167c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.986+0000 7fa59d79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 0x7fa598167c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41452/0 (socket says 172.21.15.33:41452) 2023-11-06T21:59:30.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.986+0000 7fa59d79c700 1 -- 172.21.15.33:0/3887765479 learned_addr learned my addr 172.21.15.33:0/3887765479 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa597fff700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa598155030 0x7fa5980aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa59cf9b700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa5981547d0 0x7fa59813d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa59d79c700 1 -- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa5981547d0 msgr2=0x7fa59813d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa59d79c700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa5981547d0 0x7fa59813d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa59d79c700 1 -- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa598155030 msgr2=0x7fa5980aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa59d79c700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa598155030 0x7fa5980aa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa59d79c700 1 -- 172.21.15.33:0/3887765479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009370 con 0x7fa598155410 2023-11-06T21:59:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa59d79c700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 0x7fa598167c10 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fa590009ef0 tx=0x7fa59000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa594ff9700 1 -- 172.21.15.33:0/3887765479 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590009c50 con 0x7fa598155410 2023-11-06T21:59:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa594ff9700 1 -- 172.21.15.33:0/3887765479 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59000f040 con 0x7fa598155410 2023-11-06T21:59:30.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa59f1ff700 1 -- 172.21.15.33:0/3887765479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5980acc40 con 0x7fa598155410 2023-11-06T21:59:30.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa594ff9700 1 -- 172.21.15.33:0/3887765479 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59000ce80 con 0x7fa598155410 2023-11-06T21:59:30.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.987+0000 7fa59f1ff700 1 -- 172.21.15.33:0/3887765479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5980ad200 con 0x7fa598155410 2023-11-06T21:59:30.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.989+0000 7fa59f1ff700 1 -- 172.21.15.33:0/3887765479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5980bfd90 con 0x7fa598155410 2023-11-06T21:59:30.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.989+0000 7fa594ff9700 1 -- 172.21.15.33:0/3887765479 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa590013610 con 0x7fa598155410 2023-11-06T21:59:30.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.990+0000 7fa594ff9700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa57c05f270 0x7fa57c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:30.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.990+0000 7fa594ff9700 1 -- 172.21.15.33:0/3887765479 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5829+0+0 (secure 0 0 0) 0x7fa59007ac60 con 0x7fa598155410 2023-11-06T21:59:30.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.991+0000 7fa59cf9b700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa57c05f270 0x7fa57c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:30.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.991+0000 7fa59cf9b700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa57c05f270 0x7fa57c061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fa58c0011e0 tx=0x7fa58c0070f0).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:31.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:30.994+0000 7fa594ff9700 1 -- 172.21.15.33:0/3887765479 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa59004be10 con 0x7fa598155410 2023-11-06T21:59:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.176+0000 7fa59f1ff700 1 -- 172.21.15.33:0/3887765479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa59806c080 con 0x7fa598155410 2023-11-06T21:59:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.180+0000 7fa594ff9700 1 -- 172.21.15.33:0/3887765479 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa59004f430 con 0x7fa598155410 2023-11-06T21:59:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 -- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa57c05f270 msgr2=0x7fa57c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa57c05f270 0x7fa57c061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fa58c0011e0 tx=0x7fa58c0070f0).stop 2023-11-06T21:59:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 -- 172.21.15.33:0/3887765479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 msgr2=0x7fa598167c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 0x7fa598167c10 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fa590009ef0 tx=0x7fa59000b040).stop 2023-11-06T21:59:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 -- 172.21.15.33:0/3887765479 shutdown_connections 2023-11-06T21:59:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa57c05f270 0x7fa57c061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa5981547d0 0x7fa59813d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598155410 0x7fa598167c10 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 --2- 172.21.15.33:0/3887765479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa598155030 0x7fa5980aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 -- 172.21.15.33:0/3887765479 >> 172.21.15.33:0/3887765479 conn(0x7fa5980c3d30 msgr2=0x7fa5980a5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 -- 172.21.15.33:0/3887765479 shutdown_connections 2023-11-06T21:59:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.182+0000 7fa59f1ff700 1 -- 172.21.15.33:0/3887765479 wait complete. 2023-11-06T21:59:31.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-11-06T21:59:31.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.399+0000 7f6dbabff700 1 Processor -- start 2023-11-06T21:59:31.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.399+0000 7f6dbabff700 1 -- start start 2023-11-06T21:59:31.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.400+0000 7f6dbabff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6db40c3600 0x7f6db40c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:31.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.400+0000 7f6dbabff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4069410 0x7f6db4069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:31.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.400+0000 7f6dbabff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40c17e0 con 0x7f6db4136f50 2023-11-06T21:59:31.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.400+0000 7f6dbabff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db406a740 con 0x7f6db40c3600 2023-11-06T21:59:31.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.400+0000 7f6dbabff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db406a180 con 0x7f6db4069410 2023-11-06T21:59:31.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db899b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6db40c3600 0x7f6db40c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:31.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db899b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6db40c3600 0x7f6db40c39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41466/0 (socket says 172.21.15.33:41466) 2023-11-06T21:59:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db899b700 1 -- 172.21.15.33:0/735574487 learned_addr learned my addr 172.21.15.33:0/735574487 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db3fff700 1 --1- 172.21.15.33:0/735574487 >> v1:172.21.15.33:6789/0 conn(0x7f6db4136f50 0x7f6db4137400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40818/0 (socket says 172.21.15.33:40818) 2023-11-06T21:59:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db919c700 1 --2- 172.21.15.33:0/735574487 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4069410 0x7f6db4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db899b700 1 -- 172.21.15.33:0/735574487 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4069410 msgr2=0x7f6db4069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db899b700 1 --2- 172.21.15.33:0/735574487 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4069410 0x7f6db4069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db899b700 1 -- 172.21.15.33:0/735574487 >> v1:172.21.15.33:6789/0 conn(0x7f6db4136f50 legacy=0x7f6db4137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db37fe700 1 -- 172.21.15.33:0/735574487 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109395528 0 0) 0x7f6db40c17e0 con 0x7f6db4136f50 2023-11-06T21:59:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db899b700 1 -- 172.21.15.33:0/735574487 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db406a460 con 0x7f6db40c3600 2023-11-06T21:59:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db919c700 1 --2- 172.21.15.33:0/735574487 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4069410 0x7f6db4069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.401+0000 7f6db899b700 1 --2- 172.21.15.33:0/735574487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db40c3600 0x7f6db40c39e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f6da8006fd0 tx=0x7f6da800b3b0).ready entity=mon.0 client_cookie=2fe0d0e4e1e2459b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:31.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6db37fe700 1 -- 172.21.15.33:0/735574487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800db40 con 0x7f6db40c3600 2023-11-06T21:59:31.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6db37fe700 1 -- 172.21.15.33:0/735574487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da800dca0 con 0x7f6db40c3600 2023-11-06T21:59:31.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6db37fe700 1 -- 172.21.15.33:0/735574487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8010680 con 0x7f6db40c3600 2023-11-06T21:59:31.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6dbabff700 1 -- 172.21.15.33:0/735574487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db40c3600 msgr2=0x7f6db40c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:31.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6dbabff700 1 --2- 172.21.15.33:0/735574487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db40c3600 0x7f6db40c39e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f6da8006fd0 tx=0x7f6da800b3b0).stop 2023-11-06T21:59:31.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6dbabff700 1 -- 172.21.15.33:0/735574487 shutdown_connections 2023-11-06T21:59:31.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6dbabff700 1 --2- 172.21.15.33:0/735574487 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4069410 0x7f6db4069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:31.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6dbabff700 1 --2- 172.21.15.33:0/735574487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db40c3600 0x7f6db40c39e0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:31.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6dbabff700 1 -- 172.21.15.33:0/735574487 >> 172.21.15.33:0/735574487 conn(0x7f6db4062710 msgr2=0x7f6db4062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:31.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6dbabff700 1 -- 172.21.15.33:0/735574487 shutdown_connections 2023-11-06T21:59:31.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.402+0000 7f6dbabff700 1 -- 172.21.15.33:0/735574487 wait complete. 2023-11-06T21:59:31.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.404+0000 7f6dbabff700 1 Processor -- start 2023-11-06T21:59:31.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.404+0000 7f6dbabff700 1 -- start start 2023-11-06T21:59:31.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6dbabff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db4069410 0x7f6db40abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:31.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6dbabff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6db40c3600 0x7f6db40aa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:31.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6dbabff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4136f50 0x7f6db414e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:31.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6dbabff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40aa990 con 0x7f6db4069410 2023-11-06T21:59:31.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6dbabff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40aac70 con 0x7f6db40c3600 2023-11-06T21:59:31.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6dbabff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40aaf50 con 0x7f6db4136f50 2023-11-06T21:59:31.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6db899b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db4069410 0x7f6db40abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:31.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6db899b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db4069410 0x7f6db40abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41478/0 (socket says 172.21.15.33:41478) 2023-11-06T21:59:31.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6db899b700 1 -- 172.21.15.33:0/4254734872 learned_addr learned my addr 172.21.15.33:0/4254734872 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:31.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6db919c700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4136f50 0x7f6db414e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:31.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6db3fff700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6db40c3600 0x7f6db40aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:31.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6db899b700 1 -- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4136f50 msgr2=0x7f6db414e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:31.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.405+0000 7f6db899b700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4136f50 0x7f6db414e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:31.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.406+0000 7f6db899b700 1 -- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6db40c3600 msgr2=0x7f6db40aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:31.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.406+0000 7f6db899b700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6db40c3600 0x7f6db40aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:31.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.406+0000 7f6db899b700 1 -- 172.21.15.33:0/4254734872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da800b610 con 0x7f6db4069410 2023-11-06T21:59:31.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.406+0000 7f6db899b700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db4069410 0x7f6db40abc90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f6da800bfd0 tx=0x7f6da800b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:31.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.406+0000 7f6db0ff9700 1 -- 172.21.15.33:0/4254734872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da801f070 con 0x7f6db4069410 2023-11-06T21:59:31.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.406+0000 7f6db0ff9700 1 -- 172.21.15.33:0/4254734872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da8019450 con 0x7f6db4069410 2023-11-06T21:59:31.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.406+0000 7f6dbabff700 1 -- 172.21.15.33:0/4254734872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db40ab230 con 0x7f6db4069410 2023-11-06T21:59:31.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.406+0000 7f6db0ff9700 1 -- 172.21.15.33:0/4254734872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80197f0 con 0x7f6db4069410 2023-11-06T21:59:31.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.406+0000 7f6dbabff700 1 -- 172.21.15.33:0/4254734872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db4155f50 con 0x7f6db4069410 2023-11-06T21:59:31.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.407+0000 7f6dbabff700 1 -- 172.21.15.33:0/4254734872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db40ab3f0 con 0x7f6db4069410 2023-11-06T21:59:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.408+0000 7f6db0ff9700 1 -- 172.21.15.33:0/4254734872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6da8019c20 con 0x7f6db4069410 2023-11-06T21:59:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.409+0000 7f6db0ff9700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d9805f270 0x7f6d98061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.409+0000 7f6db0ff9700 1 -- 172.21.15.33:0/4254734872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f6da807dbc0 con 0x7f6db4069410 2023-11-06T21:59:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.410+0000 7f6db3fff700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d9805f270 0x7f6d98061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.410+0000 7f6db3fff700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d9805f270 0x7f6d98061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f6db40c7840 tx=0x7f6da0007c90).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.413+0000 7f6db0ff9700 1 -- 172.21.15.33:0/4254734872 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6da804ed70 con 0x7f6db4069410 2023-11-06T21:59:31.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:31.676+0000 7f6dbabff700 1 -- 172.21.15.33:0/4254734872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f6db40be520 con 0x7f6db4069410 2023-11-06T21:59:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.342+0000 7f6db0ff9700 1 -- 172.21.15.33:0/4254734872 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2424) v1 ==== 178+0+0 (secure 0 0 0) 0x7f6da8028070 con 0x7f6db4069410 2023-11-06T21:59:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.345+0000 7f6dbabff700 1 -- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d9805f270 msgr2=0x7f6d98061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.345+0000 7f6dbabff700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d9805f270 0x7f6d98061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f6db40c7840 tx=0x7f6da0007c90).stop 2023-11-06T21:59:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.345+0000 7f6dbabff700 1 -- 172.21.15.33:0/4254734872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db4069410 msgr2=0x7f6db40abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.345+0000 7f6dbabff700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db4069410 0x7f6db40abc90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f6da800bfd0 tx=0x7f6da800b3b0).stop 2023-11-06T21:59:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.345+0000 7f6dbabff700 1 -- 172.21.15.33:0/4254734872 shutdown_connections 2023-11-06T21:59:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.345+0000 7f6dbabff700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d9805f270 0x7f6d98061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.345+0000 7f6dbabff700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6db4136f50 0x7f6db414e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.345+0000 7f6dbabff700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6db4069410 0x7f6db40abc90 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.345+0000 7f6dbabff700 1 --2- 172.21.15.33:0/4254734872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6db40c3600 0x7f6db40aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.345+0000 7f6dbabff700 1 -- 172.21.15.33:0/4254734872 >> 172.21.15.33:0/4254734872 conn(0x7f6db4062710 msgr2=0x7f6db40bba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.346+0000 7f6dbabff700 1 -- 172.21.15.33:0/4254734872 shutdown_connections 2023-11-06T21:59:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.346+0000 7f6dbabff700 1 -- 172.21.15.33:0/4254734872 wait complete. 2023-11-06T21:59:32.369 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-11-06T21:59:32.369 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-11-06T21:59:32.370 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-06T21:59:32.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-11-06T21:59:32.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.569+0000 7f5b14fff700 1 Processor -- start 2023-11-06T21:59:32.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.569+0000 7f5b14fff700 1 -- start start 2023-11-06T21:59:32.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b14fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 0x7f5b10063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:32.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b14fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b100c31f0 con 0x7f5b1005e1b0 2023-11-06T21:59:32.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b14fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b100c2920 con 0x7f5b10063140 2023-11-06T21:59:32.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b14fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1006a0f0 con 0x7f5b10154760 2023-11-06T21:59:32.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b0dd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5b1005e1b0 0x7f5b10154050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40824/0 (socket says 172.21.15.33:40824) 2023-11-06T21:59:32.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b0dd9b700 1 -- 172.21.15.33:0/1334967494 learned_addr learned my addr 172.21.15.33:0/1334967494 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:32.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b0ed9d700 1 --2- 172.21.15.33:0/1334967494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 0x7f5b10063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:32.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b0d59a700 1 -- 172.21.15.33:0/1334967494 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831702704 0 0) 0x7f5b100c31f0 con 0x7f5b1005e1b0 2023-11-06T21:59:32.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b0d59a700 1 -- 172.21.15.33:0/1334967494 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af80033f0 con 0x7f5b1005e1b0 2023-11-06T21:59:32.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b0d59a700 1 -- 172.21.15.33:0/1334967494 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524613144 0 0) 0x7f5b100c2920 con 0x7f5b10063140 2023-11-06T21:59:32.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.571+0000 7f5b0d59a700 1 -- 172.21.15.33:0/1334967494 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b100c31f0 con 0x7f5b10063140 2023-11-06T21:59:32.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.572+0000 7f5b0ed9d700 1 -- 172.21.15.33:0/1334967494 >> v1:172.21.15.69:6789/0 conn(0x7f5b10063140 legacy=0x7f5b100c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:32.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.572+0000 7f5b0ed9d700 1 -- 172.21.15.33:0/1334967494 >> v1:172.21.15.33:6789/0 conn(0x7f5b1005e1b0 legacy=0x7f5b10154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:32.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.572+0000 7f5b0ed9d700 1 -- 172.21.15.33:0/1334967494 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1005e640 con 0x7f5b10154760 2023-11-06T21:59:32.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.572+0000 7f5b0ed9d700 1 --2- 172.21.15.33:0/1334967494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 0x7f5b10063520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f5b08009e90 tx=0x7f5b08009780).ready entity=mon.2 client_cookie=18b752787431db56 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:32.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.572+0000 7f5b0d59a700 1 -- 172.21.15.33:0/1334967494 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800e070 con 0x7f5b10154760 2023-11-06T21:59:32.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.572+0000 7f5b0d59a700 1 -- 172.21.15.33:0/1334967494 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0800bab0 con 0x7f5b10154760 2023-11-06T21:59:32.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.572+0000 7f5b0d59a700 1 -- 172.21.15.33:0/1334967494 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800be30 con 0x7f5b10154760 2023-11-06T21:59:32.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.573+0000 7f5b14fff700 1 -- 172.21.15.33:0/1334967494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 msgr2=0x7f5b10063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:32.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.573+0000 7f5b14fff700 1 --2- 172.21.15.33:0/1334967494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 0x7f5b10063520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f5b08009e90 tx=0x7f5b08009780).stop 2023-11-06T21:59:32.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.573+0000 7f5b14fff700 1 -- 172.21.15.33:0/1334967494 shutdown_connections 2023-11-06T21:59:32.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.573+0000 7f5b14fff700 1 --2- 172.21.15.33:0/1334967494 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 0x7f5b10063520 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:32.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.573+0000 7f5b14fff700 1 -- 172.21.15.33:0/1334967494 >> 172.21.15.33:0/1334967494 conn(0x7f5b100c7d60 msgr2=0x7f5b100c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:32.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.573+0000 7f5b14fff700 1 -- 172.21.15.33:0/1334967494 shutdown_connections 2023-11-06T21:59:32.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.573+0000 7f5b14fff700 1 -- 172.21.15.33:0/1334967494 wait complete. 2023-11-06T21:59:32.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.574+0000 7f5b14fff700 1 Processor -- start 2023-11-06T21:59:32.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.574+0000 7f5b14fff700 1 -- start start 2023-11-06T21:59:32.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.575+0000 7f5b14fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10063140 0x7f5b10155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:32.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.575+0000 7f5b14fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 0x7f5b10150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:32.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.575+0000 7f5b14fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b100ac600 0x7f5b100aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:32.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.575+0000 7f5b14fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b10155010 con 0x7f5b10063140 2023-11-06T21:59:32.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.575+0000 7f5b14fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b100acf70 con 0x7f5b100ac600 2023-11-06T21:59:32.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.575+0000 7f5b14fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b101577f0 con 0x7f5b10154760 2023-11-06T21:59:32.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10063140 0x7f5b10155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:32.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10063140 0x7f5b10155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41484/0 (socket says 172.21.15.33:41484) 2023-11-06T21:59:32.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0e59c700 1 -- 172.21.15.33:0/1911548549 learned_addr learned my addr 172.21.15.33:0/1911548549 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:32.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0ed9d700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b100ac600 0x7f5b100aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:32.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0dd9b700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 0x7f5b10150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:32.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0e59c700 1 -- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 msgr2=0x7f5b10150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:32.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0e59c700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 0x7f5b10150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:32.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0e59c700 1 -- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b100ac600 msgr2=0x7f5b100aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:32.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0e59c700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b100ac600 0x7f5b100aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:32.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0e59c700 1 -- 172.21.15.33:0/1911548549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b08009a20 con 0x7f5b10063140 2023-11-06T21:59:32.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.576+0000 7f5b0e59c700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10063140 0x7f5b10155e30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f5b0000c4a0 tx=0x7f5b00007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:32.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.577+0000 7f5afe7fc700 1 -- 172.21.15.33:0/1911548549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000f470 con 0x7f5b10063140 2023-11-06T21:59:32.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.577+0000 7f5afe7fc700 1 -- 172.21.15.33:0/1911548549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0000f5d0 con 0x7f5b10063140 2023-11-06T21:59:32.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.577+0000 7f5b14fff700 1 -- 172.21.15.33:0/1911548549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b10157ad0 con 0x7f5b10063140 2023-11-06T21:59:32.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.577+0000 7f5afe7fc700 1 -- 172.21.15.33:0/1911548549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000f970 con 0x7f5b10063140 2023-11-06T21:59:32.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.577+0000 7f5b14fff700 1 -- 172.21.15.33:0/1911548549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b100a2020 con 0x7f5b10063140 2023-11-06T21:59:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.578+0000 7f5b14fff700 1 -- 172.21.15.33:0/1911548549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b101575a0 con 0x7f5b10063140 2023-11-06T21:59:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.579+0000 7f5afe7fc700 1 -- 172.21.15.33:0/1911548549 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5b0000fd20 con 0x7f5b10063140 2023-11-06T21:59:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.579+0000 7f5afe7fc700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5af005f240 0x7f5af0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.579+0000 7f5afe7fc700 1 -- 172.21.15.33:0/1911548549 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f5b00079a50 con 0x7f5b10063140 2023-11-06T21:59:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.580+0000 7f5b0dd9b700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5af005f240 0x7f5af0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:32.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.580+0000 7f5b0dd9b700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5af005f240 0x7f5af0061700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5b1015aaf0 tx=0x7f5b04009040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.583+0000 7f5afe7fc700 1 -- 172.21.15.33:0/1911548549 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5b0004ac00 con 0x7f5b10063140 2023-11-06T21:59:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:32.828+0000 7f5b14fff700 1 -- 172.21.15.33:0/1911548549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f5b1006bfd0 con 0x7f5b10063140 2023-11-06T21:59:33.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.347+0000 7f5afe7fc700 1 -- 172.21.15.33:0/1911548549 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2425) v1 ==== 177+0+0 (secure 0 0 0) 0x7f5b0004e220 con 0x7f5b10063140 2023-11-06T21:59:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.349+0000 7f5b14fff700 1 -- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5af005f240 msgr2=0x7f5af0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.349+0000 7f5b14fff700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5af005f240 0x7f5af0061700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5b1015aaf0 tx=0x7f5b04009040).stop 2023-11-06T21:59:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.349+0000 7f5b14fff700 1 -- 172.21.15.33:0/1911548549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10063140 msgr2=0x7f5b10155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:33.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.349+0000 7f5b14fff700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10063140 0x7f5b10155e30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f5b0000c4a0 tx=0x7f5b00007c50).stop 2023-11-06T21:59:33.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.350+0000 7f5b14fff700 1 -- 172.21.15.33:0/1911548549 shutdown_connections 2023-11-06T21:59:33.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.350+0000 7f5b14fff700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5af005f240 0x7f5af0061700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.350+0000 7f5b14fff700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b100ac600 0x7f5b100aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.350+0000 7f5b14fff700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b10063140 0x7f5b10155e30 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.350+0000 7f5b14fff700 1 --2- 172.21.15.33:0/1911548549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5b10154760 0x7f5b10150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.350+0000 7f5b14fff700 1 -- 172.21.15.33:0/1911548549 >> 172.21.15.33:0/1911548549 conn(0x7f5b100c7d60 msgr2=0x7f5b100c5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.350+0000 7f5b14fff700 1 -- 172.21.15.33:0/1911548549 shutdown_connections 2023-11-06T21:59:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.350+0000 7f5b14fff700 1 -- 172.21.15.33:0/1911548549 wait complete. 2023-11-06T21:59:33.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T21:59:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.573+0000 7fa9d73ff700 1 Processor -- start 2023-11-06T21:59:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.574+0000 7fa9d73ff700 1 -- start start 2023-11-06T21:59:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.575+0000 7fa9d73ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa9d0154710 0x7fa9d00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.575+0000 7fa9d73ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa9d00c7ff0 0x7fa9d00646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.575+0000 7fa9d73ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d00c6b60 con 0x7fa9d0154330 2023-11-06T21:59:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.575+0000 7fa9d73ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d00c7700 con 0x7fa9d0154710 2023-11-06T21:59:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.575+0000 7fa9d73ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00c5cd0 con 0x7fa9d00c7ff0 2023-11-06T21:59:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.575+0000 7fa9d499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa9d0154710 0x7fa9d00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.575+0000 7fa9d499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa9d0154710 0x7fa9d00c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41496/0 (socket says 172.21.15.33:41496) 2023-11-06T21:59:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.575+0000 7fa9d519b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa9d0154330 0x7fa9d00c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40832/0 (socket says 172.21.15.33:40832) 2023-11-06T21:59:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.575+0000 7fa9d499a700 1 -- 172.21.15.33:0/2272447388 learned_addr learned my addr 172.21.15.33:0/2272447388 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.575+0000 7fa9d599c700 1 --2- 172.21.15.33:0/2272447388 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa9d00c7ff0 0x7fa9d00646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.576+0000 7fa9c7fff700 1 -- 172.21.15.33:0/2272447388 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159559698 0 0) 0x7fa9d00c6b60 con 0x7fa9d0154330 2023-11-06T21:59:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.576+0000 7fa9c7fff700 1 -- 172.21.15.33:0/2272447388 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9bc0033f0 con 0x7fa9d0154330 2023-11-06T21:59:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.576+0000 7fa9d499a700 1 -- 172.21.15.33:0/2272447388 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa9d00c7ff0 msgr2=0x7fa9d00646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:33.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.576+0000 7fa9d499a700 1 --2- 172.21.15.33:0/2272447388 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa9d00c7ff0 0x7fa9d00646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.576+0000 7fa9d499a700 1 -- 172.21.15.33:0/2272447388 >> v1:172.21.15.33:6789/0 conn(0x7fa9d0154330 legacy=0x7fa9d00c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:33.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.576+0000 7fa9d499a700 1 -- 172.21.15.33:0/2272447388 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d00c5f50 con 0x7fa9d0154710 2023-11-06T21:59:33.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.576+0000 7fa9d499a700 1 --2- 172.21.15.33:0/2272447388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 0x7fa9d00c57e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7fa9c0006fd0 tx=0x7fa9c00090f0).ready entity=mon.0 client_cookie=16a43bd816192d50 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:33.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.576+0000 7fa9c7fff700 1 -- 172.21.15.33:0/2272447388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000f510 con 0x7fa9d0154710 2023-11-06T21:59:33.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.576+0000 7fa9c7fff700 1 -- 172.21.15.33:0/2272447388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c000f670 con 0x7fa9d0154710 2023-11-06T21:59:33.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.576+0000 7fa9c7fff700 1 -- 172.21.15.33:0/2272447388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000f9f0 con 0x7fa9d0154710 2023-11-06T21:59:33.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.577+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2272447388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 msgr2=0x7fa9d00c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:33.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.577+0000 7fa9d73ff700 1 --2- 172.21.15.33:0/2272447388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 0x7fa9d00c57e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7fa9c0006fd0 tx=0x7fa9c00090f0).stop 2023-11-06T21:59:33.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.577+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2272447388 shutdown_connections 2023-11-06T21:59:33.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.577+0000 7fa9d73ff700 1 --2- 172.21.15.33:0/2272447388 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa9d00c7ff0 0x7fa9d00646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.577+0000 7fa9d73ff700 1 --2- 172.21.15.33:0/2272447388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 0x7fa9d00c57e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.577+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2272447388 >> 172.21.15.33:0/2272447388 conn(0x7fa9d00cb910 msgr2=0x7fa9d00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:33.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.577+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2272447388 shutdown_connections 2023-11-06T21:59:33.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.577+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2272447388 wait complete. 2023-11-06T21:59:33.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.578+0000 7fa9d73ff700 1 Processor -- start 2023-11-06T21:59:33.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.578+0000 7fa9d73ff700 1 -- start start 2023-11-06T21:59:33.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.579+0000 7fa9d73ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d00c7ff0 0x7fa9d00c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:33.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.579+0000 7fa9d73ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa9d0154330 0x7fa9d00c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:33.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.579+0000 7fa9d73ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 0x7fa9d00c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:33.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.579+0000 7fa9d73ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00c4890 con 0x7fa9d0154710 2023-11-06T21:59:33.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.579+0000 7fa9d73ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00c0bb0 con 0x7fa9d00c7ff0 2023-11-06T21:59:33.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.579+0000 7fa9d73ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00c39b0 con 0x7fa9d0154330 2023-11-06T21:59:33.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 0x7fa9d00c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:33.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 0x7fa9d00c06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41502/0 (socket says 172.21.15.33:41502) 2023-11-06T21:59:33.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d599c700 1 -- 172.21.15.33:0/2582407442 learned_addr learned my addr 172.21.15.33:0/2582407442 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:33.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d499a700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa9d0154330 0x7fa9d00c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:33.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d519b700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d00c7ff0 0x7fa9d00c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d599c700 1 -- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa9d0154330 msgr2=0x7fa9d00c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d599c700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa9d0154330 0x7fa9d00c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d599c700 1 -- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d00c7ff0 msgr2=0x7fa9d00c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d599c700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d00c7ff0 0x7fa9d00c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d599c700 1 -- 172.21.15.33:0/2582407442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0009370 con 0x7fa9d0154710 2023-11-06T21:59:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.580+0000 7fa9d599c700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 0x7fa9d00c06c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7fa9c8009e90 tx=0x7fa9c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.581+0000 7fa9c57fa700 1 -- 172.21.15.33:0/2582407442 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c800caa0 con 0x7fa9d0154710 2023-11-06T21:59:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.581+0000 7fa9c57fa700 1 -- 172.21.15.33:0/2582407442 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c800f040 con 0x7fa9d0154710 2023-11-06T21:59:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.581+0000 7fa9c57fa700 1 -- 172.21.15.33:0/2582407442 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c800ce10 con 0x7fa9d0154710 2023-11-06T21:59:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.581+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2582407442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d00c3c80 con 0x7fa9d0154710 2023-11-06T21:59:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.581+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2582407442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d00c4240 con 0x7fa9d0154710 2023-11-06T21:59:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.582+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2582407442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d00830b0 con 0x7fa9d0154710 2023-11-06T21:59:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.583+0000 7fa9c57fa700 1 -- 172.21.15.33:0/2582407442 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa9c8013610 con 0x7fa9d0154710 2023-11-06T21:59:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.584+0000 7fa9c57fa700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa9b405f270 0x7fa9b4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.585+0000 7fa9c57fa700 1 -- 172.21.15.33:0/2582407442 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5829+0+0 (secure 0 0 0) 0x7fa9c807ab10 con 0x7fa9d0154710 2023-11-06T21:59:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.585+0000 7fa9d519b700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa9b405f270 0x7fa9b4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.585+0000 7fa9d519b700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa9b405f270 0x7fa9b4061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa9cc007640 tx=0x7fa9cc007bc0).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.589+0000 7fa9c57fa700 1 -- 172.21.15.33:0/2582407442 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa9c804bd80 con 0x7fa9d0154710 2023-11-06T21:59:33.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.770+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2582407442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa9d0062b00 con 0x7fa9d0154710 2023-11-06T21:59:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.777+0000 7fa9c57fa700 1 -- 172.21.15.33:0/2582407442 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa9c804f3a0 con 0x7fa9d0154710 2023-11-06T21:59:33.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa9b405f270 msgr2=0x7fa9b4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:33.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa9b405f270 0x7fa9b4061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa9cc007640 tx=0x7fa9cc007bc0).stop 2023-11-06T21:59:33.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2582407442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 msgr2=0x7fa9d00c06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 0x7fa9d00c06c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7fa9c8009e90 tx=0x7fa9c800b040).stop 2023-11-06T21:59:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2582407442 shutdown_connections 2023-11-06T21:59:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa9b405f270 0x7fa9b4061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d0154710 0x7fa9d00c06c0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9d00c7ff0 0x7fa9d00c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 --2- 172.21.15.33:0/2582407442 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa9d0154330 0x7fa9d00c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2582407442 >> 172.21.15.33:0/2582407442 conn(0x7fa9d00cb910 msgr2=0x7fa9d00c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2582407442 shutdown_connections 2023-11-06T21:59:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:33.780+0000 7fa9d73ff700 1 -- 172.21.15.33:0/2582407442 wait complete. 2023-11-06T21:59:33.800 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T21:59:33.800 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:59:33.908 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:59:34.003 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T21:59:34.003 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T21:59:34.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:59:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.197+0000 7f0e4f9ff700 1 Processor -- start 2023-11-06T21:59:34.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.197+0000 7f0e4f9ff700 1 -- start start 2023-11-06T21:59:34.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.198+0000 7f0e4f9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0e480c0c50 0x7f0e480c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:34.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.198+0000 7f0e4f9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e48154e90 con 0x7f0e481521a0 2023-11-06T21:59:34.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.198+0000 7f0e4f9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e480639d0 con 0x7f0e48151dc0 2023-11-06T21:59:34.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.198+0000 7f0e4f9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e48155a30 con 0x7f0e480c0c50 2023-11-06T21:59:34.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.199+0000 7f0e4cf9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0e481521a0 0x7f0e48155320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40840/0 (socket says 172.21.15.33:40840) 2023-11-06T21:59:34.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.199+0000 7f0e4cf9a700 1 -- 172.21.15.33:0/45914238 learned_addr learned my addr 172.21.15.33:0/45914238 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:34.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.199+0000 7f0e4df9c700 1 --2- 172.21.15.33:0/45914238 >> v2:172.21.15.69:3300/0 conn(0x7f0e480c0c50 0x7f0e480c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.199+0000 7f0e3ffff700 1 -- 172.21.15.33:0/45914238 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799555441 0 0) 0x7f0e48154e90 con 0x7f0e481521a0 2023-11-06T21:59:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.199+0000 7f0e3ffff700 1 -- 172.21.15.33:0/45914238 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e340033f0 con 0x7f0e481521a0 2023-11-06T21:59:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.199+0000 7f0e4df9c700 1 -- 172.21.15.33:0/45914238 >> v1:172.21.15.69:6789/0 conn(0x7f0e48151dc0 legacy=0x7f0e48063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.199+0000 7f0e4df9c700 1 -- 172.21.15.33:0/45914238 >> v1:172.21.15.33:6789/0 conn(0x7f0e481521a0 legacy=0x7f0e48155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.199+0000 7f0e4df9c700 1 -- 172.21.15.33:0/45914238 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e48063cb0 con 0x7f0e480c0c50 2023-11-06T21:59:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.200+0000 7f0e4df9c700 1 --2- 172.21.15.33:0/45914238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e480c0c50 0x7f0e480c7850 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f0e40009e60 tx=0x7f0e40009780).ready entity=mon.1 client_cookie=20ed5959f9554571 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.200+0000 7f0e3ffff700 1 -- 172.21.15.33:0/45914238 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4000e070 con 0x7f0e480c0c50 2023-11-06T21:59:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.200+0000 7f0e3ffff700 1 -- 172.21.15.33:0/45914238 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4000bb60 con 0x7f0e480c0c50 2023-11-06T21:59:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.200+0000 7f0e3ffff700 1 -- 172.21.15.33:0/45914238 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40012410 con 0x7f0e480c0c50 2023-11-06T21:59:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.200+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/45914238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e480c0c50 msgr2=0x7f0e480c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.200+0000 7f0e4f9ff700 1 --2- 172.21.15.33:0/45914238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e480c0c50 0x7f0e480c7850 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f0e40009e60 tx=0x7f0e40009780).stop 2023-11-06T21:59:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.200+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/45914238 shutdown_connections 2023-11-06T21:59:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.200+0000 7f0e4f9ff700 1 --2- 172.21.15.33:0/45914238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e480c0c50 0x7f0e480c7850 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.200+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/45914238 >> 172.21.15.33:0/45914238 conn(0x7f0e4805e1b0 msgr2=0x7f0e4805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.201+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/45914238 shutdown_connections 2023-11-06T21:59:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.201+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/45914238 wait complete. 2023-11-06T21:59:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.202+0000 7f0e4f9ff700 1 Processor -- start 2023-11-06T21:59:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.202+0000 7f0e4f9ff700 1 -- start start 2023-11-06T21:59:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.203+0000 7f0e4f9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0e480c0c50 0x7f0e48146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.203+0000 7f0e4f9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e48151dc0 0x7f0e480ab820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.203+0000 7f0e4f9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e481521a0 0x7f0e480abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.203+0000 7f0e4f9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480c7760 con 0x7f0e481521a0 2023-11-06T21:59:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.203+0000 7f0e4f9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480acf60 con 0x7f0e48151dc0 2023-11-06T21:59:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.203+0000 7f0e4f9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e481470e0 con 0x7f0e480c0c50 2023-11-06T21:59:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.203+0000 7f0e4df9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e481521a0 0x7f0e480abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:34.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.203+0000 7f0e4df9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e481521a0 0x7f0e480abd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41514/0 (socket says 172.21.15.33:41514) 2023-11-06T21:59:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.203+0000 7f0e4df9c700 1 -- 172.21.15.33:0/2187974298 learned_addr learned my addr 172.21.15.33:0/2187974298 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e4cf9a700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e48151dc0 0x7f0e480ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e4d79b700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0e480c0c50 0x7f0e48146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e4df9c700 1 -- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0e480c0c50 msgr2=0x7f0e48146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e4df9c700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0e480c0c50 0x7f0e48146bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:34.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e4df9c700 1 -- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e48151dc0 msgr2=0x7f0e480ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e4df9c700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e48151dc0 0x7f0e480ab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:34.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e4df9c700 1 -- 172.21.15.33:0/2187974298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e40009a20 con 0x7f0e481521a0 2023-11-06T21:59:34.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e4d79b700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0e480c0c50 0x7f0e48146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:34.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e4df9c700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e481521a0 0x7f0e480abd10 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f0e4000b6c0 tx=0x7f0e40009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e3d7fa700 1 -- 172.21.15.33:0/2187974298 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4001d630 con 0x7f0e481521a0 2023-11-06T21:59:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e3d7fa700 1 -- 172.21.15.33:0/2187974298 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4001d790 con 0x7f0e481521a0 2023-11-06T21:59:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/2187974298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e480ac2e0 con 0x7f0e481521a0 2023-11-06T21:59:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.204+0000 7f0e3d7fa700 1 -- 172.21.15.33:0/2187974298 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4001db30 con 0x7f0e481521a0 2023-11-06T21:59:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.205+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/2187974298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e480ac8a0 con 0x7f0e481521a0 2023-11-06T21:59:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.206+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/2187974298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e480ad0a0 con 0x7f0e481521a0 2023-11-06T21:59:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.206+0000 7f0e3d7fa700 1 -- 172.21.15.33:0/2187974298 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0e40026410 con 0x7f0e481521a0 2023-11-06T21:59:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.207+0000 7f0e3d7fa700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e2c05f240 0x7f0e2c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.208+0000 7f0e3d7fa700 1 -- 172.21.15.33:0/2187974298 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f0e4007c2d0 con 0x7f0e481521a0 2023-11-06T21:59:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.208+0000 7f0e4d79b700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e2c05f240 0x7f0e2c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.209+0000 7f0e4d79b700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e2c05f240 0x7f0e2c061700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f0e44001710 tx=0x7f0e44009040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:34.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.211+0000 7f0e3d7fa700 1 -- 172.21.15.33:0/2187974298 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0e40050e50 con 0x7f0e481521a0 2023-11-06T21:59:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.434+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/2187974298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e48062ac0 con 0x7f0e481521a0 2023-11-06T21:59:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.434+0000 7f0e3d7fa700 1 -- 172.21.15.33:0/2187974298 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f0e40050a40 con 0x7f0e481521a0 2023-11-06T21:59:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:34.437 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6848,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1809147689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1809147689},{"type":"v1","addr":"172.21.15.33:6835","nonce":1809147689}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7714,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1390207299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1390207299},{"type":"v1","addr":"172.21.15.69:6837","nonce":1390207299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3179984136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3179984136},{"type":"v1","addr":"172.21.15.69:6836","nonce":3179984136}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":8370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3005262821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3005262821},{"type":"v1","addr":"172.21.15.33:6834","nonce":3005262821}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[]} 2023-11-06T21:59:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.436+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e2c05f240 msgr2=0x7f0e2c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.436+0000 7f0e4f9ff700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e2c05f240 0x7f0e2c061700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f0e44001710 tx=0x7f0e44009040).stop 2023-11-06T21:59:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.436+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/2187974298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e481521a0 msgr2=0x7f0e480abd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.436+0000 7f0e4f9ff700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e481521a0 0x7f0e480abd10 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f0e4000b6c0 tx=0x7f0e40009780).stop 2023-11-06T21:59:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.436+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/2187974298 shutdown_connections 2023-11-06T21:59:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.436+0000 7f0e4f9ff700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f0e2c05f240 0x7f0e2c061700 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.436+0000 7f0e4f9ff700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0e480c0c50 0x7f0e48146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:34.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.436+0000 7f0e4f9ff700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e481521a0 0x7f0e480abd10 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:34.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.436+0000 7f0e4f9ff700 1 --2- 172.21.15.33:0/2187974298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e48151dc0 0x7f0e480ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:34.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.436+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/2187974298 >> 172.21.15.33:0/2187974298 conn(0x7f0e4805e1b0 msgr2=0x7f0e480a9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:34.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.437+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/2187974298 shutdown_connections 2023-11-06T21:59:34.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.437+0000 7f0e4f9ff700 1 -- 172.21.15.33:0/2187974298 wait complete. 2023-11-06T21:59:34.442 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-11-06T21:59:34.456 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T21:59:34.456 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T21:59:34.457 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T21:59:34.457 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:59:34.457 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T21:59:34.458 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T21:59:34.458 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T21:59:34.458 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:59:34.459 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T21:59:34.459 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T21:59:34.459 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T21:59:34.459 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:59:34.460 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T21:59:34.460 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T21:59:34.460 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T21:59:34.460 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T21:59:34.471 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T21:59:34.471 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T21:59:34.473 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T21:59:34.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T21:59:34.477 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T21:59:34.478 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T21:59:34.479 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T21:59:34.479 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T21:59:34.518 INFO:tasks.ceph.mds.a:Started 2023-11-06T21:59:34.519 DEBUG:teuthology.parallel:result is None 2023-11-06T21:59:34.522 INFO:tasks.ceph.mds.c:Started 2023-11-06T21:59:34.522 DEBUG:teuthology.parallel:result is None 2023-11-06T21:59:34.524 INFO:tasks.ceph.mds.b:Started 2023-11-06T21:59:34.524 DEBUG:teuthology.parallel:result is None 2023-11-06T21:59:34.527 INFO:tasks.ceph.mds.d:Started 2023-11-06T21:59:34.527 DEBUG:teuthology.parallel:result is None 2023-11-06T21:59:34.528 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T21:59:34.666 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T21:59:34.668 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T21:59:34.673 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T21:59:34.674 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T21:59:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.741+0000 7f55af9ff700 1 Processor -- start 2023-11-06T21:59:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.741+0000 7f55af9ff700 1 -- start start 2023-11-06T21:59:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.743+0000 7f55af9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f55a81373a0 0x7f55a8137780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.743+0000 7f55af9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a806a470 con 0x7f55a80c1020 2023-11-06T21:59:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.743+0000 7f55af9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a806a190 con 0x7f55a80c3990 2023-11-06T21:59:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.743+0000 7f55af9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a80bbab0 con 0x7f55a81373a0 2023-11-06T21:59:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.743+0000 7f55acf9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f55a80c1020 0x7f55a80c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40854/0 (socket says 172.21.15.33:40854) 2023-11-06T21:59:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.743+0000 7f55acf9a700 1 -- 172.21.15.33:0/3966258691 learned_addr learned my addr 172.21.15.33:0/3966258691 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.743+0000 7f55ad79b700 1 --2- 172.21.15.33:0/3966258691 >> v2:172.21.15.69:3300/0 conn(0x7f55a81373a0 0x7f55a8137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.743+0000 7f559ffff700 1 -- 172.21.15.33:0/3966258691 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613655565 0 0) 0x7f55a806a470 con 0x7f55a80c1020 2023-11-06T21:59:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.744+0000 7f559ffff700 1 -- 172.21.15.33:0/3966258691 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55940033f0 con 0x7f55a80c1020 2023-11-06T21:59:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.744+0000 7f55ad79b700 1 -- 172.21.15.33:0/3966258691 >> v1:172.21.15.69:6789/0 conn(0x7f55a80c3990 legacy=0x7f55a8069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.744+0000 7f559ffff700 1 -- 172.21.15.33:0/3966258691 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543289532 0 0) 0x7f55a806a190 con 0x7f55a80c3990 2023-11-06T21:59:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.744+0000 7f55ad79b700 1 -- 172.21.15.33:0/3966258691 >> v1:172.21.15.33:6789/0 conn(0x7f55a80c1020 legacy=0x7f55a80c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.744+0000 7f55ad79b700 1 -- 172.21.15.33:0/3966258691 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80bbd90 con 0x7f55a81373a0 2023-11-06T21:59:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.744+0000 7f55ad79b700 1 --2- 172.21.15.33:0/3966258691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a81373a0 0x7f55a8137780 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f55a4009760 tx=0x7f55a4009060).ready entity=mon.1 client_cookie=9c69d2175d1b2d47 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.744+0000 7f559ffff700 1 -- 172.21.15.33:0/3966258691 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400f510 con 0x7f55a81373a0 2023-11-06T21:59:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.744+0000 7f559ffff700 1 -- 172.21.15.33:0/3966258691 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a400f670 con 0x7f55a81373a0 2023-11-06T21:59:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.744+0000 7f559ffff700 1 -- 172.21.15.33:0/3966258691 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400f9f0 con 0x7f55a81373a0 2023-11-06T21:59:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.745+0000 7f55af9ff700 1 -- 172.21.15.33:0/3966258691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a81373a0 msgr2=0x7f55a8137780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.745+0000 7f55af9ff700 1 --2- 172.21.15.33:0/3966258691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a81373a0 0x7f55a8137780 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f55a4009760 tx=0x7f55a4009060).stop 2023-11-06T21:59:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.745+0000 7f55af9ff700 1 -- 172.21.15.33:0/3966258691 shutdown_connections 2023-11-06T21:59:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.745+0000 7f55af9ff700 1 --2- 172.21.15.33:0/3966258691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a81373a0 0x7f55a8137780 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.745+0000 7f55af9ff700 1 -- 172.21.15.33:0/3966258691 >> 172.21.15.33:0/3966258691 conn(0x7f55a8062710 msgr2=0x7f55a8062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.745+0000 7f55af9ff700 1 -- 172.21.15.33:0/3966258691 shutdown_connections 2023-11-06T21:59:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.745+0000 7f55af9ff700 1 -- 172.21.15.33:0/3966258691 wait complete. 2023-11-06T21:59:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.746+0000 7f55af9ff700 1 Processor -- start 2023-11-06T21:59:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.746+0000 7f55af9ff700 1 -- start start 2023-11-06T21:59:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.747+0000 7f55af9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a80c1020 0x7f55a814e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.747+0000 7f55af9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55a80c3990 0x7f55a8164c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.747+0000 7f55af9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a81373a0 0x7f55a80a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.747+0000 7f55af9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80a9f00 con 0x7f55a80c1020 2023-11-06T21:59:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.747+0000 7f55af9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80aa1d0 con 0x7f55a81373a0 2023-11-06T21:59:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.747+0000 7f55af9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80aa4b0 con 0x7f55a80c3990 2023-11-06T21:59:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55ad79b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a80c1020 0x7f55a814e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55ad79b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a80c1020 0x7f55a814e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41550/0 (socket says 172.21.15.33:41550) 2023-11-06T21:59:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55ad79b700 1 -- 172.21.15.33:0/2216676829 learned_addr learned my addr 172.21.15.33:0/2216676829 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55acf9a700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55a80c3990 0x7f55a8164c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55adf9c700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a81373a0 0x7f55a80a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55ad79b700 1 -- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55a80c3990 msgr2=0x7f55a8164c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55ad79b700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55a80c3990 0x7f55a8164c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55ad79b700 1 -- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a81373a0 msgr2=0x7f55a80a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55ad79b700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a81373a0 0x7f55a80a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55ad79b700 1 -- 172.21.15.33:0/2216676829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a40092a0 con 0x7f55a80c1020 2023-11-06T21:59:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55adf9c700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a81373a0 0x7f55a80a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.748+0000 7f55ad79b700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a80c1020 0x7f55a814e2e0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f55a4009e00 tx=0x7f55a4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.749+0000 7f559d7fa700 1 -- 172.21.15.33:0/2216676829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400f7e0 con 0x7f55a80c1020 2023-11-06T21:59:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.749+0000 7f559d7fa700 1 -- 172.21.15.33:0/2216676829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a400f940 con 0x7f55a80c1020 2023-11-06T21:59:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.749+0000 7f559d7fa700 1 -- 172.21.15.33:0/2216676829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a401b660 con 0x7f55a80c1020 2023-11-06T21:59:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.749+0000 7f55af9ff700 1 -- 172.21.15.33:0/2216676829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a80aa790 con 0x7f55a80c1020 2023-11-06T21:59:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.749+0000 7f55af9ff700 1 -- 172.21.15.33:0/2216676829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a8064270 con 0x7f55a80c1020 2023-11-06T21:59:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.750+0000 7f55af9ff700 1 -- 172.21.15.33:0/2216676829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a8165460 con 0x7f55a80c1020 2023-11-06T21:59:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.751+0000 7f559d7fa700 1 -- 172.21.15.33:0/2216676829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f55a401ba50 con 0x7f55a80c1020 2023-11-06T21:59:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.752+0000 7f559d7fa700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f558c05f270 0x7f558c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.752+0000 7f559d7fa700 1 -- 172.21.15.33:0/2216676829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f55a407bea0 con 0x7f55a80c1020 2023-11-06T21:59:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.752+0000 7f55acf9a700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f558c05f270 0x7f558c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.753+0000 7f55acf9a700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f558c05f270 0x7f558c061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f55a8064e40 tx=0x7f5598009040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.755+0000 7f559d7fa700 1 -- 172.21.15.33:0/2216676829 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f55a404d100 con 0x7f55a80c1020 2023-11-06T21:59:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:34.953+0000 7f55af9ff700 1 -- 172.21.15.33:0/2216676829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f55a8065a00 con 0x7f55a80c1020 2023-11-06T21:59:35.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.783+0000 7f559d7fa700 1 -- 172.21.15.33:0/2216676829 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v166) v1 ==== 97+0+0 (secure 0 0 0) 0x7f55a401f020 con 0x7f55a80c1020 2023-11-06T21:59:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.785+0000 7f55af9ff700 1 -- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f558c05f270 msgr2=0x7f558c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.785+0000 7f55af9ff700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f558c05f270 0x7f558c061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f55a8064e40 tx=0x7f5598009040).stop 2023-11-06T21:59:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.785+0000 7f55af9ff700 1 -- 172.21.15.33:0/2216676829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a80c1020 msgr2=0x7f55a814e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.785+0000 7f55af9ff700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a80c1020 0x7f55a814e2e0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f55a4009e00 tx=0x7f55a4017000).stop 2023-11-06T21:59:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.785+0000 7f55af9ff700 1 -- 172.21.15.33:0/2216676829 shutdown_connections 2023-11-06T21:59:35.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.785+0000 7f55af9ff700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f558c05f270 0x7f558c061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:35.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.785+0000 7f55af9ff700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a81373a0 0x7f55a80a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:35.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.785+0000 7f55af9ff700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a80c1020 0x7f55a814e2e0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:35.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.785+0000 7f55af9ff700 1 --2- 172.21.15.33:0/2216676829 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f55a80c3990 0x7f55a8164c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:35.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.785+0000 7f55af9ff700 1 -- 172.21.15.33:0/2216676829 >> 172.21.15.33:0/2216676829 conn(0x7f55a8062710 msgr2=0x7f55a8136460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:35.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.786+0000 7f55af9ff700 1 -- 172.21.15.33:0/2216676829 shutdown_connections 2023-11-06T21:59:35.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:35.786+0000 7f55af9ff700 1 -- 172.21.15.33:0/2216676829 wait complete. 2023-11-06T21:59:35.790 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T21:59:35.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T21:59:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.007+0000 7f39f95ff700 1 Processor -- start 2023-11-06T21:59:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.007+0000 7f39f95ff700 1 -- start start 2023-11-06T21:59:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.009+0000 7f39f95ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f39f40c1840 0x7f39f40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.009+0000 7f39f95ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f41554a0 con 0x7f39f40c6460 2023-11-06T21:59:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.009+0000 7f39f95ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f4155930 con 0x7f39f40aa2d0 2023-11-06T21:59:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.009+0000 7f39f95ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f4154960 con 0x7f39f40c1840 2023-11-06T21:59:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.009+0000 7f39f37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f39f40c6460 0x7f39f40bfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40862/0 (socket says 172.21.15.33:40862) 2023-11-06T21:59:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.009+0000 7f39f37fe700 1 -- 172.21.15.33:0/1614928270 learned_addr learned my addr 172.21.15.33:0/1614928270 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.009+0000 7f39f2ffd700 1 --2- 172.21.15.33:0/1614928270 >> v2:172.21.15.69:3300/0 conn(0x7f39f40c1840 0x7f39f40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.009+0000 7f39f1ffb700 1 -- 172.21.15.33:0/1614928270 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407659025 0 0) 0x7f39f41554a0 con 0x7f39f40c6460 2023-11-06T21:59:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.010+0000 7f39f1ffb700 1 -- 172.21.15.33:0/1614928270 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39dc0033f0 con 0x7f39f40c6460 2023-11-06T21:59:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.010+0000 7f39f2ffd700 1 -- 172.21.15.33:0/1614928270 >> v1:172.21.15.69:6789/0 conn(0x7f39f40aa2d0 legacy=0x7f39f40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.010+0000 7f39f2ffd700 1 -- 172.21.15.33:0/1614928270 >> v1:172.21.15.33:6789/0 conn(0x7f39f40c6460 legacy=0x7f39f40bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.010+0000 7f39f2ffd700 1 -- 172.21.15.33:0/1614928270 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f4154b70 con 0x7f39f40c1840 2023-11-06T21:59:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.010+0000 7f39f2ffd700 1 --2- 172.21.15.33:0/1614928270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40c1840 0x7f39f40a9de0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f39e4009760 tx=0x7f39e4009060).ready entity=mon.1 client_cookie=7c1a04d02ffddd50 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.010+0000 7f39f1ffb700 1 -- 172.21.15.33:0/1614928270 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e400f510 con 0x7f39f40c1840 2023-11-06T21:59:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.010+0000 7f39f1ffb700 1 -- 172.21.15.33:0/1614928270 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e400f670 con 0x7f39f40c1840 2023-11-06T21:59:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.010+0000 7f39f1ffb700 1 -- 172.21.15.33:0/1614928270 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e400f9f0 con 0x7f39f40c1840 2023-11-06T21:59:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.011+0000 7f39f95ff700 1 -- 172.21.15.33:0/1614928270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40c1840 msgr2=0x7f39f40a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.011+0000 7f39f95ff700 1 --2- 172.21.15.33:0/1614928270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40c1840 0x7f39f40a9de0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f39e4009760 tx=0x7f39e4009060).stop 2023-11-06T21:59:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.011+0000 7f39f95ff700 1 -- 172.21.15.33:0/1614928270 shutdown_connections 2023-11-06T21:59:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.011+0000 7f39f95ff700 1 --2- 172.21.15.33:0/1614928270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40c1840 0x7f39f40a9de0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.011+0000 7f39f95ff700 1 -- 172.21.15.33:0/1614928270 >> 172.21.15.33:0/1614928270 conn(0x7f39f405ec80 msgr2=0x7f39f40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.011+0000 7f39f95ff700 1 -- 172.21.15.33:0/1614928270 shutdown_connections 2023-11-06T21:59:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.011+0000 7f39f95ff700 1 -- 172.21.15.33:0/1614928270 wait complete. 2023-11-06T21:59:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.012+0000 7f39f95ff700 1 Processor -- start 2023-11-06T21:59:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.013+0000 7f39f95ff700 1 -- start start 2023-11-06T21:59:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.013+0000 7f39f95ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40aa2d0 0x7f39f4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.013+0000 7f39f95ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39f40c1840 0x7f39f413e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.013+0000 7f39f95ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f40c6460 0x7f39f413ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.013+0000 7f39f95ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f40aafe0 con 0x7f39f40c6460 2023-11-06T21:59:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.013+0000 7f39f95ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4157ef0 con 0x7f39f40aa2d0 2023-11-06T21:59:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.013+0000 7f39f95ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f41581d0 con 0x7f39f40c1840 2023-11-06T21:59:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f40c6460 0x7f39f413ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f40c6460 0x7f39f413ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41560/0 (socket says 172.21.15.33:41560) 2023-11-06T21:59:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f37fe700 1 -- 172.21.15.33:0/3142969173 learned_addr learned my addr 172.21.15.33:0/3142969173 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f2ffd700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40aa2d0 0x7f39f4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f27fc700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39f40c1840 0x7f39f413e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f37fe700 1 -- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39f40c1840 msgr2=0x7f39f413e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f37fe700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39f40c1840 0x7f39f413e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f37fe700 1 -- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40aa2d0 msgr2=0x7f39f4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f37fe700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40aa2d0 0x7f39f4155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f37fe700 1 -- 172.21.15.33:0/3142969173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e40092a0 con 0x7f39f40c6460 2023-11-06T21:59:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f27fc700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39f40c1840 0x7f39f413e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.014+0000 7f39f37fe700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f40c6460 0x7f39f413ea10 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f39ec005ad0 tx=0x7f39ec0015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.015+0000 7f39daffd700 1 -- 172.21.15.33:0/3142969173 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec005d30 con 0x7f39f40c6460 2023-11-06T21:59:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.015+0000 7f39daffd700 1 -- 172.21.15.33:0/3142969173 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec005e90 con 0x7f39f40c6460 2023-11-06T21:59:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.015+0000 7f39daffd700 1 -- 172.21.15.33:0/3142969173 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec007320 con 0x7f39f40c6460 2023-11-06T21:59:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.015+0000 7f39f95ff700 1 -- 172.21.15.33:0/3142969173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f41584b0 con 0x7f39f40c6460 2023-11-06T21:59:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.015+0000 7f39f95ff700 1 -- 172.21.15.33:0/3142969173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f4158a70 con 0x7f39f40c6460 2023-11-06T21:59:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.016+0000 7f39f95ff700 1 -- 172.21.15.33:0/3142969173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f406c6c0 con 0x7f39f40c6460 2023-11-06T21:59:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.017+0000 7f39daffd700 1 -- 172.21.15.33:0/3142969173 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f39ec007710 con 0x7f39f40c6460 2023-11-06T21:59:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.017+0000 7f39daffd700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39e005f270 0x7f39e0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.018+0000 7f39daffd700 1 -- 172.21.15.33:0/3142969173 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f39ec079fe0 con 0x7f39f40c6460 2023-11-06T21:59:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.018+0000 7f39f2ffd700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39e005f270 0x7f39e0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.018+0000 7f39f2ffd700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39e005f270 0x7f39e0061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f39e4009760 tx=0x7f39e4017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.021+0000 7f39daffd700 1 -- 172.21.15.33:0/3142969173 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f39ec04b410 con 0x7f39f40c6460 2023-11-06T21:59:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.270+0000 7f39f95ff700 1 -- 172.21.15.33:0/3142969173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f39f4063f20 con 0x7f39f40c6460 2023-11-06T21:59:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.271+0000 7f39daffd700 1 -- 172.21.15.33:0/3142969173 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f39ec04ea30 con 0x7f39f40c6460 2023-11-06T21:59:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T21:59:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T21:59:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T21:59:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T21:59:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:59:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:59:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:59:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T21:59:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T21:59:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:59:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:59:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:59:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T21:59:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T21:59:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:59:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:59:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:59:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T21:59:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T21:59:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:59:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:59:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:59:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T21:59:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T21:59:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:59:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:59:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T21:59:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T21:59:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:59:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:59:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T21:59:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T21:59:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:59:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:59:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T21:59:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T21:59:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:59:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:59:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T21:59:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T21:59:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:59:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:59:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T21:59:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T21:59:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:59:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:59:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T21:59:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T21:59:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:59:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:59:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T21:59:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T21:59:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T21:59:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T21:59:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T21:59:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T21:59:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:59:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-11-06T21:59:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T21:59:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T21:59:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T21:59:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:59:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:59:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:59:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:59:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T21:59:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T21:59:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:59:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:59:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:59:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:59:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T21:59:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T21:59:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T21:59:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T21:59:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T21:59:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T21:59:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T21:59:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T21:59:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:59:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T21:59:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T21:59:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T21:59:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T21:59:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:59:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T21:59:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T21:59:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T21:59:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T21:59:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T21:59:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T21:59:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T21:59:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T21:59:36.309 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T21:59:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 -- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39e005f270 msgr2=0x7f39e0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39e005f270 0x7f39e0061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f39e4009760 tx=0x7f39e4017000).stop 2023-11-06T21:59:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 -- 172.21.15.33:0/3142969173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f40c6460 msgr2=0x7f39f413ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f40c6460 0x7f39f413ea10 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f39ec005ad0 tx=0x7f39ec0015b0).stop 2023-11-06T21:59:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 -- 172.21.15.33:0/3142969173 shutdown_connections 2023-11-06T21:59:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f39e005f270 0x7f39e0061730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39f40c6460 0x7f39f413ea10 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40aa2d0 0x7f39f4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 --2- 172.21.15.33:0/3142969173 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f39f40c1840 0x7f39f413e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 -- 172.21.15.33:0/3142969173 >> 172.21.15.33:0/3142969173 conn(0x7f39f405ec80 msgr2=0x7f39f40ca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 -- 172.21.15.33:0/3142969173 shutdown_connections 2023-11-06T21:59:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.273+0000 7f39f95ff700 1 -- 172.21.15.33:0/3142969173 wait complete. 2023-11-06T21:59:36.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:59:36.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.507+0000 7f68153ff700 1 Processor -- start 2023-11-06T21:59:36.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.507+0000 7f68153ff700 1 -- start start 2023-11-06T21:59:36.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.508+0000 7f68153ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6810063140 0x7f6810063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.508+0000 7f68153ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f68100c3680 0x7f6810154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.508+0000 7f68153ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68100c31f0 con 0x7f681005e1b0 2023-11-06T21:59:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.508+0000 7f68153ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681006a0f0 con 0x7f6810063140 2023-11-06T21:59:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.508+0000 7f68153ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100c2920 con 0x7f68100c3680 2023-11-06T21:59:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.508+0000 7f680effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6810063140 0x7f6810063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.508+0000 7f680effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6810063140 0x7f6810063520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41564/0 (socket says 172.21.15.33:41564) 2023-11-06T21:59:36.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.508+0000 7f680effd700 1 -- 172.21.15.33:0/3016602122 learned_addr learned my addr 172.21.15.33:0/3016602122 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:36.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.508+0000 7f680e7fc700 1 --1- 172.21.15.33:0/3016602122 >> v1:172.21.15.33:6789/0 conn(0x7f681005e1b0 0x7f68100c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40870/0 (socket says 172.21.15.33:40870) 2023-11-06T21:59:36.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680f7fe700 1 --2- 172.21.15.33:0/3016602122 >> v2:172.21.15.69:3300/0 conn(0x7f68100c3680 0x7f6810154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680effd700 1 -- 172.21.15.33:0/3016602122 >> v2:172.21.15.69:3300/0 conn(0x7f68100c3680 msgr2=0x7f6810154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:36.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680effd700 1 --2- 172.21.15.33:0/3016602122 >> v2:172.21.15.69:3300/0 conn(0x7f68100c3680 0x7f6810154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680effd700 1 -- 172.21.15.33:0/3016602122 >> v1:172.21.15.33:6789/0 conn(0x7f681005e1b0 legacy=0x7f68100c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680effd700 1 -- 172.21.15.33:0/3016602122 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681005e640 con 0x7f6810063140 2023-11-06T21:59:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680dffb700 1 -- 172.21.15.33:0/3016602122 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766568881 0 0) 0x7f68100c31f0 con 0x7f681005e1b0 2023-11-06T21:59:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680f7fe700 1 --2- 172.21.15.33:0/3016602122 >> v2:172.21.15.69:3300/0 conn(0x7f68100c3680 0x7f6810154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680effd700 1 --2- 172.21.15.33:0/3016602122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6810063140 0x7f6810063520 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f68100c3160 tx=0x7f68000090f0).ready entity=mon.0 client_cookie=9ab5971724d11d88 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680dffb700 1 -- 172.21.15.33:0/3016602122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000b600 con 0x7f6810063140 2023-11-06T21:59:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680dffb700 1 -- 172.21.15.33:0/3016602122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680000b760 con 0x7f6810063140 2023-11-06T21:59:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.509+0000 7f680dffb700 1 -- 172.21.15.33:0/3016602122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000bae0 con 0x7f6810063140 2023-11-06T21:59:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.510+0000 7f68153ff700 1 -- 172.21.15.33:0/3016602122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6810063140 msgr2=0x7f6810063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.510+0000 7f68153ff700 1 --2- 172.21.15.33:0/3016602122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6810063140 0x7f6810063520 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f68100c3160 tx=0x7f68000090f0).stop 2023-11-06T21:59:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.510+0000 7f68153ff700 1 -- 172.21.15.33:0/3016602122 shutdown_connections 2023-11-06T21:59:36.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.510+0000 7f68153ff700 1 --2- 172.21.15.33:0/3016602122 >> v2:172.21.15.69:3300/0 conn(0x7f68100c3680 0x7f6810154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.510+0000 7f68153ff700 1 --2- 172.21.15.33:0/3016602122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6810063140 0x7f6810063520 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.510+0000 7f68153ff700 1 -- 172.21.15.33:0/3016602122 >> 172.21.15.33:0/3016602122 conn(0x7f68100c7d60 msgr2=0x7f68100c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:36.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.510+0000 7f68153ff700 1 -- 172.21.15.33:0/3016602122 shutdown_connections 2023-11-06T21:59:36.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.510+0000 7f68153ff700 1 -- 172.21.15.33:0/3016602122 wait complete. 2023-11-06T21:59:36.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.512+0000 7f68153ff700 1 Processor -- start 2023-11-06T21:59:36.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.512+0000 7f68153ff700 1 -- start start 2023-11-06T21:59:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f68153ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6810063140 0x7f68100aab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f68153ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100c3680 0x7f68100abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f68153ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f681013ebf0 0x7f6810164570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f68153ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68100ab000 con 0x7f68100c3680 2023-11-06T21:59:36.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f68153ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68100ab1f0 con 0x7f681013ebf0 2023-11-06T21:59:36.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f68153ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68100ac3e0 con 0x7f6810063140 2023-11-06T21:59:36.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f680e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100c3680 0x7f68100abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f680e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100c3680 0x7f68100abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41572/0 (socket says 172.21.15.33:41572) 2023-11-06T21:59:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f680e7fc700 1 -- 172.21.15.33:0/3901591859 learned_addr learned my addr 172.21.15.33:0/3901591859 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f680effd700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6810063140 0x7f68100aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f680f7fe700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f681013ebf0 0x7f6810164570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.513+0000 7f680e7fc700 1 -- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6810063140 msgr2=0x7f68100aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.514+0000 7f680e7fc700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6810063140 0x7f68100aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.514+0000 7f680e7fc700 1 -- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f681013ebf0 msgr2=0x7f6810164570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.514+0000 7f680e7fc700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f681013ebf0 0x7f6810164570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.514+0000 7f680e7fc700 1 -- 172.21.15.33:0/3901591859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800009370 con 0x7f68100c3680 2023-11-06T21:59:36.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.514+0000 7f680e7fc700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100c3680 0x7f68100abef0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f680400c4c0 tx=0x7f6804007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:36.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.514+0000 7f67f6ffd700 1 -- 172.21.15.33:0/3901591859 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040072e0 con 0x7f68100c3680 2023-11-06T21:59:36.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.514+0000 7f67f6ffd700 1 -- 172.21.15.33:0/3901591859 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6804007440 con 0x7f68100c3680 2023-11-06T21:59:36.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.514+0000 7f68153ff700 1 -- 172.21.15.33:0/3901591859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68100c5a60 con 0x7f68100c3680 2023-11-06T21:59:36.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.514+0000 7f67f6ffd700 1 -- 172.21.15.33:0/3901591859 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804005930 con 0x7f68100c3680 2023-11-06T21:59:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.514+0000 7f68153ff700 1 -- 172.21.15.33:0/3901591859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68100c5410 con 0x7f68100c3680 2023-11-06T21:59:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.515+0000 7f68153ff700 1 -- 172.21.15.33:0/3901591859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6810064420 con 0x7f68100c3680 2023-11-06T21:59:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.516+0000 7f67f6ffd700 1 -- 172.21.15.33:0/3901591859 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6804005ca0 con 0x7f68100c3680 2023-11-06T21:59:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.517+0000 7f67f6ffd700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67fc05f270 0x7f67fc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.517+0000 7f67f6ffd700 1 -- 172.21.15.33:0/3901591859 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f6804079840 con 0x7f68100c3680 2023-11-06T21:59:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.517+0000 7f680effd700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67fc05f270 0x7f67fc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.518+0000 7f680effd700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67fc05f270 0x7f67fc061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f6800009130 tx=0x7f6800017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.521+0000 7f67f6ffd700 1 -- 172.21.15.33:0/3901591859 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f680404abf0 con 0x7f68100c3680 2023-11-06T21:59:36.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.699+0000 7f68153ff700 1 -- 172.21.15.33:0/3901591859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68100c5bf0 con 0x7f68100c3680 2023-11-06T21:59:36.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.700+0000 7f67f6ffd700 1 -- 172.21.15.33:0/3901591859 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+13841 (secure 0 0 0) 0x7f680404e210 con 0x7f68100c3680 2023-11-06T21:59:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:36.703 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":166,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:59:35.779896+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:59:36.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.702+0000 7f68153ff700 1 -- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67fc05f270 msgr2=0x7f67fc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.702+0000 7f68153ff700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67fc05f270 0x7f67fc061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f6800009130 tx=0x7f6800017000).stop 2023-11-06T21:59:36.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.702+0000 7f68153ff700 1 -- 172.21.15.33:0/3901591859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100c3680 msgr2=0x7f68100abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.703+0000 7f68153ff700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100c3680 0x7f68100abef0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f680400c4c0 tx=0x7f6804007e00).stop 2023-11-06T21:59:36.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.703+0000 7f68153ff700 1 -- 172.21.15.33:0/3901591859 shutdown_connections 2023-11-06T21:59:36.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.703+0000 7f68153ff700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f67fc05f270 0x7f67fc061730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.703+0000 7f68153ff700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6810063140 0x7f68100aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.703+0000 7f68153ff700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f681013ebf0 0x7f6810164570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.703+0000 7f68153ff700 1 --2- 172.21.15.33:0/3901591859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100c3680 0x7f68100abef0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.703+0000 7f68153ff700 1 -- 172.21.15.33:0/3901591859 >> 172.21.15.33:0/3901591859 conn(0x7f68100c7d60 msgr2=0x7f68101535b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:36.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.703+0000 7f68153ff700 1 -- 172.21.15.33:0/3901591859 shutdown_connections 2023-11-06T21:59:36.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.703+0000 7f68153ff700 1 -- 172.21.15.33:0/3901591859 wait complete. 2023-11-06T21:59:36.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T21:59:36.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T21:59:36.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.921+0000 7f70649ff700 1 Processor -- start 2023-11-06T21:59:36.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.921+0000 7f70649ff700 1 -- start start 2023-11-06T21:59:36.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f70649ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f706013dff0 0x7f7060157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f70649ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f70600c4b00 0x7f70600c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f70649ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7060064bf0 con 0x7f70600c4720 2023-11-06T21:59:36.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f70649ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706005e490 con 0x7f706013dff0 2023-11-06T21:59:36.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f70649ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7060062e10 con 0x7f70600c4b00 2023-11-06T21:59:36.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f706013dff0 0x7f7060157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f706013dff0 0x7f7060157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41588/0 (socket says 172.21.15.33:41588) 2023-11-06T21:59:36.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705dd9b700 1 -- 172.21.15.33:0/2865100346 learned_addr learned my addr 172.21.15.33:0/2865100346 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:36.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705e59c700 1 --1- 172.21.15.33:0/2865100346 >> v1:172.21.15.33:6789/0 conn(0x7f70600c4720 0x7f706013d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40872/0 (socket says 172.21.15.33:40872) 2023-11-06T21:59:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705ed9d700 1 --2- 172.21.15.33:0/2865100346 >> v2:172.21.15.69:3300/0 conn(0x7f70600c4b00 0x7f70600c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705dd9b700 1 -- 172.21.15.33:0/2865100346 >> v2:172.21.15.69:3300/0 conn(0x7f70600c4b00 msgr2=0x7f70600c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705dd9b700 1 --2- 172.21.15.33:0/2865100346 >> v2:172.21.15.69:3300/0 conn(0x7f70600c4b00 0x7f70600c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705dd9b700 1 -- 172.21.15.33:0/2865100346 >> v1:172.21.15.33:6789/0 conn(0x7f70600c4720 legacy=0x7f706013d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705dd9b700 1 -- 172.21.15.33:0/2865100346 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70600630f0 con 0x7f706013dff0 2023-11-06T21:59:36.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705d59a700 1 -- 172.21.15.33:0/2865100346 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726092535 0 0) 0x7f7060064bf0 con 0x7f70600c4720 2023-11-06T21:59:36.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.923+0000 7f705ed9d700 1 --2- 172.21.15.33:0/2865100346 >> v2:172.21.15.69:3300/0 conn(0x7f70600c4b00 0x7f70600c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:36.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.924+0000 7f705dd9b700 1 --2- 172.21.15.33:0/2865100346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706013dff0 0x7f7060157e20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f70600c23a0 tx=0x7f70540090f0).ready entity=mon.0 client_cookie=248f4e40e67848f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:36.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.924+0000 7f705d59a700 1 -- 172.21.15.33:0/2865100346 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400b600 con 0x7f706013dff0 2023-11-06T21:59:36.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.924+0000 7f705d59a700 1 -- 172.21.15.33:0/2865100346 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705400b760 con 0x7f706013dff0 2023-11-06T21:59:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.924+0000 7f705d59a700 1 -- 172.21.15.33:0/2865100346 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400bae0 con 0x7f706013dff0 2023-11-06T21:59:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.924+0000 7f70649ff700 1 -- 172.21.15.33:0/2865100346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706013dff0 msgr2=0x7f7060157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.924+0000 7f70649ff700 1 --2- 172.21.15.33:0/2865100346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706013dff0 0x7f7060157e20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f70600c23a0 tx=0x7f70540090f0).stop 2023-11-06T21:59:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.924+0000 7f70649ff700 1 -- 172.21.15.33:0/2865100346 shutdown_connections 2023-11-06T21:59:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.924+0000 7f70649ff700 1 --2- 172.21.15.33:0/2865100346 >> v2:172.21.15.69:3300/0 conn(0x7f70600c4b00 0x7f70600c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.924+0000 7f70649ff700 1 --2- 172.21.15.33:0/2865100346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f706013dff0 0x7f7060157e20 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.924+0000 7f70649ff700 1 -- 172.21.15.33:0/2865100346 >> 172.21.15.33:0/2865100346 conn(0x7f70600c7dc0 msgr2=0x7f70600c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.925+0000 7f70649ff700 1 -- 172.21.15.33:0/2865100346 shutdown_connections 2023-11-06T21:59:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.925+0000 7f70649ff700 1 -- 172.21.15.33:0/2865100346 wait complete. 2023-11-06T21:59:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.926+0000 7f70649ff700 1 Processor -- start 2023-11-06T21:59:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.926+0000 7f70649ff700 1 -- start start 2023-11-06T21:59:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.927+0000 7f70649ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70600c4720 0x7f706013bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.927+0000 7f70649ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70600c4b00 0x7f70601529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.927+0000 7f70649ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f706013dff0 0x7f7060138bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.927+0000 7f70649ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600ac9a0 con 0x7f70600c4720 2023-11-06T21:59:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.927+0000 7f70649ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706013c9a0 con 0x7f70600c4b00 2023-11-06T21:59:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.927+0000 7f70649ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706013c020 con 0x7f706013dff0 2023-11-06T21:59:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.927+0000 7f705e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70600c4720 0x7f706013bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.927+0000 7f705e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70600c4720 0x7f706013bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41592/0 (socket says 172.21.15.33:41592) 2023-11-06T21:59:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.927+0000 7f705e59c700 1 -- 172.21.15.33:0/1704275704 learned_addr learned my addr 172.21.15.33:0/1704275704 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f705ed9d700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f706013dff0 0x7f7060138bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f705dd9b700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70600c4b00 0x7f70601529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f705e59c700 1 -- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f706013dff0 msgr2=0x7f7060138bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f705e59c700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f706013dff0 0x7f7060138bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f705e59c700 1 -- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70600c4b00 msgr2=0x7f70601529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f705e59c700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70600c4b00 0x7f70601529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f705e59c700 1 -- 172.21.15.33:0/1704275704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054009370 con 0x7f70600c4720 2023-11-06T21:59:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f705ed9d700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f706013dff0 0x7f7060138bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f705dd9b700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70600c4b00 0x7f70601529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f705e59c700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70600c4720 0x7f706013bb30 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f705000c440 tx=0x7f7050007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f704e7fc700 1 -- 172.21.15.33:0/1704275704 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70500072e0 con 0x7f70600c4720 2023-11-06T21:59:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f704e7fc700 1 -- 172.21.15.33:0/1704275704 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7050007440 con 0x7f70600c4720 2023-11-06T21:59:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f704e7fc700 1 -- 172.21.15.33:0/1704275704 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050005910 con 0x7f70600c4720 2023-11-06T21:59:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.928+0000 7f70649ff700 1 -- 172.21.15.33:0/1704275704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70600abd90 con 0x7f70600c4720 2023-11-06T21:59:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.929+0000 7f70649ff700 1 -- 172.21.15.33:0/1704275704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70600ac350 con 0x7f70600c4720 2023-11-06T21:59:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.930+0000 7f70649ff700 1 -- 172.21.15.33:0/1704275704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70600c2150 con 0x7f70600c4720 2023-11-06T21:59:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.931+0000 7f704e7fc700 1 -- 172.21.15.33:0/1704275704 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7050005d00 con 0x7f70600c4720 2023-11-06T21:59:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.931+0000 7f704e7fc700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f704005f270 0x7f7040061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.931+0000 7f704e7fc700 1 -- 172.21.15.33:0/1704275704 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f70500797e0 con 0x7f70600c4720 2023-11-06T21:59:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.932+0000 7f705dd9b700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f704005f270 0x7f7040061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.932+0000 7f705dd9b700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f704005f270 0x7f7040061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7054009130 tx=0x7f7054017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:36.935+0000 7f704e7fc700 1 -- 172.21.15.33:0/1704275704 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f705004ac50 con 0x7f70600c4720 2023-11-06T21:59:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.113+0000 7f70649ff700 1 -- 172.21.15.33:0/1704275704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f70600aa0d0 con 0x7f70600c4720 2023-11-06T21:59:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.113+0000 7f704e7fc700 1 -- 172.21.15.33:0/1704275704 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v166) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7050018030 con 0x7f70600c4720 2023-11-06T21:59:37.116 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T21:59:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.115+0000 7f70649ff700 1 -- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f704005f270 msgr2=0x7f7040061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.115+0000 7f70649ff700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f704005f270 0x7f7040061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7054009130 tx=0x7f7054017000).stop 2023-11-06T21:59:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.115+0000 7f70649ff700 1 -- 172.21.15.33:0/1704275704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70600c4720 msgr2=0x7f706013bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.115+0000 7f70649ff700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70600c4720 0x7f706013bb30 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f705000c440 tx=0x7f7050007e30).stop 2023-11-06T21:59:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.115+0000 7f70649ff700 1 -- 172.21.15.33:0/1704275704 shutdown_connections 2023-11-06T21:59:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.115+0000 7f70649ff700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f704005f270 0x7f7040061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.115+0000 7f70649ff700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f706013dff0 0x7f7060138bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.115+0000 7f70649ff700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70600c4720 0x7f706013bb30 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.115+0000 7f70649ff700 1 --2- 172.21.15.33:0/1704275704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70600c4b00 0x7f70601529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.115+0000 7f70649ff700 1 -- 172.21.15.33:0/1704275704 >> 172.21.15.33:0/1704275704 conn(0x7f70600c7dc0 msgr2=0x7f70600c6070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:37.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.116+0000 7f70649ff700 1 -- 172.21.15.33:0/1704275704 shutdown_connections 2023-11-06T21:59:37.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.116+0000 7f70649ff700 1 -- 172.21.15.33:0/1704275704 wait complete. 2023-11-06T21:59:37.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T21:59:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.332+0000 7faeb77ff700 1 Processor -- start 2023-11-06T21:59:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.332+0000 7faeb77ff700 1 -- start start 2023-11-06T21:59:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.333+0000 7faeb77ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faeb0064b40 0x7faeb00c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.333+0000 7faeb77ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb01556d0 con 0x7faeb01544f0 2023-11-06T21:59:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.333+0000 7faeb77ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb0064fd0 con 0x7faeb00646c0 2023-11-06T21:59:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.333+0000 7faeb77ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb01548d0 con 0x7faeb0064b40 2023-11-06T21:59:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.333+0000 7faeb5d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faeb0064b40 0x7faeb00c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.333+0000 7faeb559b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faeb01544f0 0x7faeb00c0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40882/0 (socket says 172.21.15.33:40882) 2023-11-06T21:59:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.333+0000 7faeb559b700 1 -- 172.21.15.33:0/2575492921 learned_addr learned my addr 172.21.15.33:0/2575492921 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.334+0000 7faeb5d9c700 1 -- 172.21.15.33:0/2575492921 >> v1:172.21.15.69:6789/0 conn(0x7faeb00646c0 legacy=0x7faeb00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.334+0000 7faeb5d9c700 1 -- 172.21.15.33:0/2575492921 >> v1:172.21.15.33:6789/0 conn(0x7faeb01544f0 legacy=0x7faeb00c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.334+0000 7faeb5d9c700 1 -- 172.21.15.33:0/2575492921 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0154ac0 con 0x7faeb0064b40 2023-11-06T21:59:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.334+0000 7faea7fff700 1 -- 172.21.15.33:0/2575492921 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139317101 0 0) 0x7faeb01556d0 con 0x7faeb01544f0 2023-11-06T21:59:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.334+0000 7faeb5d9c700 1 --2- 172.21.15.33:0/2575492921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb0064b40 0x7faeb00c5ef0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7faea8009e60 tx=0x7faea8009780).ready entity=mon.0 client_cookie=f2ba73fb93430450 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.334+0000 7faea7fff700 1 -- 172.21.15.33:0/2575492921 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea800e070 con 0x7faeb0064b40 2023-11-06T21:59:37.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.334+0000 7faea7fff700 1 -- 172.21.15.33:0/2575492921 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea800bc30 con 0x7faeb0064b40 2023-11-06T21:59:37.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.335+0000 7faea7fff700 1 -- 172.21.15.33:0/2575492921 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8012410 con 0x7faeb0064b40 2023-11-06T21:59:37.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.335+0000 7faeb77ff700 1 -- 172.21.15.33:0/2575492921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb0064b40 msgr2=0x7faeb00c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.335+0000 7faeb77ff700 1 --2- 172.21.15.33:0/2575492921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb0064b40 0x7faeb00c5ef0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7faea8009e60 tx=0x7faea8009780).stop 2023-11-06T21:59:37.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.335+0000 7faeb77ff700 1 -- 172.21.15.33:0/2575492921 shutdown_connections 2023-11-06T21:59:37.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.335+0000 7faeb77ff700 1 --2- 172.21.15.33:0/2575492921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb0064b40 0x7faeb00c5ef0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.335+0000 7faeb77ff700 1 -- 172.21.15.33:0/2575492921 >> 172.21.15.33:0/2575492921 conn(0x7faeb00cb900 msgr2=0x7faeb00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:37.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.335+0000 7faeb77ff700 1 -- 172.21.15.33:0/2575492921 shutdown_connections 2023-11-06T21:59:37.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.335+0000 7faeb77ff700 1 -- 172.21.15.33:0/2575492921 wait complete. 2023-11-06T21:59:37.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.337+0000 7faeb77ff700 1 Processor -- start 2023-11-06T21:59:37.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.337+0000 7faeb77ff700 1 -- start start 2023-11-06T21:59:37.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb77ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb00646c0 0x7faeb0147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb77ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb0064b40 0x7faeb0165110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb77ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faeb01544f0 0x7faeb00c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb77ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb01480b0 con 0x7faeb00646c0 2023-11-06T21:59:37.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb77ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb0165600 con 0x7faeb0064b40 2023-11-06T21:59:37.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb77ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00c1ee0 con 0x7faeb01544f0 2023-11-06T21:59:37.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb559b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb00646c0 0x7faeb0147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb559b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb00646c0 0x7faeb0147bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41602/0 (socket says 172.21.15.33:41602) 2023-11-06T21:59:37.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb559b700 1 -- 172.21.15.33:0/1728204109 learned_addr learned my addr 172.21.15.33:0/1728204109 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb4d9a700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb0064b40 0x7faeb0165110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb5d9c700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faeb01544f0 0x7faeb00c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb559b700 1 -- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faeb01544f0 msgr2=0x7faeb00c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.338+0000 7faeb559b700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faeb01544f0 0x7faeb00c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.339+0000 7faeb559b700 1 -- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb0064b40 msgr2=0x7faeb0165110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.339+0000 7faeb559b700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb0064b40 0x7faeb0165110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.339+0000 7faeb559b700 1 -- 172.21.15.33:0/1728204109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea8009a20 con 0x7faeb00646c0 2023-11-06T21:59:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.339+0000 7faeb4d9a700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb0064b40 0x7faeb0165110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.339+0000 7faeb559b700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb00646c0 0x7faeb0147bc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7faeac00c490 tx=0x7faeac007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.339+0000 7faea57fa700 1 -- 172.21.15.33:0/1728204109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0072e0 con 0x7faeb00646c0 2023-11-06T21:59:37.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.339+0000 7faea57fa700 1 -- 172.21.15.33:0/1728204109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac007440 con 0x7faeb00646c0 2023-11-06T21:59:37.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.339+0000 7faeb77ff700 1 -- 172.21.15.33:0/1728204109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb00c21c0 con 0x7faeb00646c0 2023-11-06T21:59:37.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.339+0000 7faea57fa700 1 -- 172.21.15.33:0/1728204109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0057a0 con 0x7faeb00646c0 2023-11-06T21:59:37.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.339+0000 7faeb77ff700 1 -- 172.21.15.33:0/1728204109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb00c2780 con 0x7faeb00646c0 2023-11-06T21:59:37.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.340+0000 7faeb77ff700 1 -- 172.21.15.33:0/1728204109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb00c30b0 con 0x7faeb00646c0 2023-11-06T21:59:37.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.341+0000 7faea57fa700 1 -- 172.21.15.33:0/1728204109 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7faeac005b90 con 0x7faeb00646c0 2023-11-06T21:59:37.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.342+0000 7faea57fa700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae9405f270 0x7fae94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.342+0000 7faea57fa700 1 -- 172.21.15.33:0/1728204109 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5313+0+0 (secure 0 0 0) 0x7faeac0799d0 con 0x7faeb00646c0 2023-11-06T21:59:37.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.343+0000 7faeb4d9a700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae9405f270 0x7fae94061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.343+0000 7faeb4d9a700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae9405f270 0x7fae94061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7faea0001730 tx=0x7faea0009040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:37.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.346+0000 7faea57fa700 1 -- 172.21.15.33:0/1728204109 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faeac04ad80 con 0x7faeb00646c0 2023-11-06T21:59:37.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.522+0000 7faeb77ff700 1 -- 172.21.15.33:0/1728204109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faeb00c8760 con 0x7faeb00646c0 2023-11-06T21:59:37.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.523+0000 7faea57fa700 1 -- 172.21.15.33:0/1728204109 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v166) v1 ==== 92+0+10 (secure 0 0 0) 0x7faeac018030 con 0x7faeb00646c0 2023-11-06T21:59:37.526 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T21:59:37.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 -- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae9405f270 msgr2=0x7fae94061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae9405f270 0x7fae94061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7faea0001730 tx=0x7faea0009040).stop 2023-11-06T21:59:37.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 -- 172.21.15.33:0/1728204109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb00646c0 msgr2=0x7faeb0147bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb00646c0 0x7faeb0147bc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7faeac00c490 tx=0x7faeac007e40).stop 2023-11-06T21:59:37.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 -- 172.21.15.33:0/1728204109 shutdown_connections 2023-11-06T21:59:37.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fae9405f270 0x7fae94061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faeb01544f0 0x7faeb00c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeb00646c0 0x7faeb0147bc0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 --2- 172.21.15.33:0/1728204109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb0064b40 0x7faeb0165110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 -- 172.21.15.33:0/1728204109 >> 172.21.15.33:0/1728204109 conn(0x7faeb00cb900 msgr2=0x7faeb0157250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:37.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 -- 172.21.15.33:0/1728204109 shutdown_connections 2023-11-06T21:59:37.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.525+0000 7faeb77ff700 1 -- 172.21.15.33:0/1728204109 wait complete. 2023-11-06T21:59:37.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T21:59:37.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.739+0000 7f17ec5ff700 1 Processor -- start 2023-11-06T21:59:37.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.739+0000 7f17ec5ff700 1 -- start start 2023-11-06T21:59:37.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.740+0000 7f17ec5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f17e40c1840 0x7f17e40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.740+0000 7f17ec5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f17e40aa2d0 0x7f17e40c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.740+0000 7f17ec5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e41554a0 con 0x7f17e40c6240 2023-11-06T21:59:37.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.740+0000 7f17ec5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e4154960 con 0x7f17e40aa2d0 2023-11-06T21:59:37.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.740+0000 7f17ec5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e4155930 con 0x7f17e40c1840 2023-11-06T21:59:37.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.741+0000 7f17e9b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f17e40aa2d0 0x7f17e40c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.741+0000 7f17e9b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f17e40aa2d0 0x7f17e40c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41610/0 (socket says 172.21.15.33:41610) 2023-11-06T21:59:37.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.741+0000 7f17e9b9a700 1 -- 172.21.15.33:0/2157942932 learned_addr learned my addr 172.21.15.33:0/2157942932 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:37.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.741+0000 7f17ea39b700 1 --2- 172.21.15.33:0/2157942932 >> v2:172.21.15.69:3300/0 conn(0x7f17e40c1840 0x7f17e40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.741+0000 7f17e9b9a700 1 -- 172.21.15.33:0/2157942932 >> v2:172.21.15.69:3300/0 conn(0x7f17e40c1840 msgr2=0x7f17e40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:37.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.741+0000 7f17e9b9a700 1 --2- 172.21.15.33:0/2157942932 >> v2:172.21.15.69:3300/0 conn(0x7f17e40c1840 0x7f17e40a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.741+0000 7f17e9b9a700 1 -- 172.21.15.33:0/2157942932 >> v1:172.21.15.33:6789/0 conn(0x7f17e40c6240 legacy=0x7f17e4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.741+0000 7f17e9b9a700 1 -- 172.21.15.33:0/2157942932 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4154b70 con 0x7f17e40aa2d0 2023-11-06T21:59:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.741+0000 7f17e9b9a700 1 --2- 172.21.15.33:0/2157942932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40aa2d0 0x7f17e40c5d50 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f17e40840b0 tx=0x7f17d40090f0).ready entity=mon.0 client_cookie=80fa7f7ef7a8e5e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17e9399700 1 -- 172.21.15.33:0/2157942932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400b600 con 0x7f17e40aa2d0 2023-11-06T21:59:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17e9399700 1 -- 172.21.15.33:0/2157942932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d400b760 con 0x7f17e40aa2d0 2023-11-06T21:59:37.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17e9399700 1 -- 172.21.15.33:0/2157942932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400bae0 con 0x7f17e40aa2d0 2023-11-06T21:59:37.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2157942932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40aa2d0 msgr2=0x7f17e40c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17ec5ff700 1 --2- 172.21.15.33:0/2157942932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40aa2d0 0x7f17e40c5d50 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f17e40840b0 tx=0x7f17d40090f0).stop 2023-11-06T21:59:37.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2157942932 shutdown_connections 2023-11-06T21:59:37.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17ec5ff700 1 --2- 172.21.15.33:0/2157942932 >> v2:172.21.15.69:3300/0 conn(0x7f17e40c1840 0x7f17e40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17ec5ff700 1 --2- 172.21.15.33:0/2157942932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40aa2d0 0x7f17e40c5d50 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2157942932 >> 172.21.15.33:0/2157942932 conn(0x7f17e405ec80 msgr2=0x7f17e40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:37.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2157942932 shutdown_connections 2023-11-06T21:59:37.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.742+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2157942932 wait complete. 2023-11-06T21:59:37.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.744+0000 7f17ec5ff700 1 Processor -- start 2023-11-06T21:59:37.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.744+0000 7f17ec5ff700 1 -- start start 2023-11-06T21:59:37.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17ec5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17e40aa2d0 0x7f17e4158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17ec5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40c1840 0x7f17e4144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17ec5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17e40c6240 0x7f17e40c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17ec5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e4157020 con 0x7f17e40c1840 2023-11-06T21:59:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17ec5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e4157300 con 0x7f17e40c6240 2023-11-06T21:59:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17ec5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e41575e0 con 0x7f17e40aa2d0 2023-11-06T21:59:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17e9b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40c1840 0x7f17e4144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17e9b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40c1840 0x7f17e4144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41620/0 (socket says 172.21.15.33:41620) 2023-11-06T21:59:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17e9b9a700 1 -- 172.21.15.33:0/2315577607 learned_addr learned my addr 172.21.15.33:0/2315577607 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17eab9c700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17e40c6240 0x7f17e40c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.745+0000 7f17ea39b700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17e40aa2d0 0x7f17e4158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17e9b9a700 1 -- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17e40aa2d0 msgr2=0x7f17e4158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17e9b9a700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17e40aa2d0 0x7f17e4158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17e9b9a700 1 -- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17e40c6240 msgr2=0x7f17e40c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17e9b9a700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17e40c6240 0x7f17e40c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17e9b9a700 1 -- 172.21.15.33:0/2315577607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d4009370 con 0x7f17e40c1840 2023-11-06T21:59:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17ea39b700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17e40aa2d0 0x7f17e4158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17e9b9a700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40c1840 0x7f17e4144ab0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f17d4011000 tx=0x7f17d4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17da7fc700 1 -- 172.21.15.33:0/2315577607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400b8d0 con 0x7f17e40c1840 2023-11-06T21:59:37.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17da7fc700 1 -- 172.21.15.33:0/2315577607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d400ba30 con 0x7f17e40c1840 2023-11-06T21:59:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2315577607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e41578c0 con 0x7f17e40c1840 2023-11-06T21:59:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17da7fc700 1 -- 172.21.15.33:0/2315577607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d401b620 con 0x7f17e40c1840 2023-11-06T21:59:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.746+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2315577607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e4144fa0 con 0x7f17e40c1840 2023-11-06T21:59:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.747+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2315577607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e406c6c0 con 0x7f17e40c1840 2023-11-06T21:59:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.748+0000 7f17da7fc700 1 -- 172.21.15.33:0/2315577607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f17d401ba10 con 0x7f17e40c1840 2023-11-06T21:59:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.749+0000 7f17da7fc700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f17c805f270 0x7f17c8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.749+0000 7f17da7fc700 1 -- 172.21.15.33:0/2315577607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f17d407baa0 con 0x7f17e40c1840 2023-11-06T21:59:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.750+0000 7f17ea39b700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f17c805f270 0x7f17c8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.750+0000 7f17ea39b700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f17c805f270 0x7f17c8061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f17e000bb60 tx=0x7f17e0008040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.754+0000 7f17da7fc700 1 -- 172.21.15.33:0/2315577607 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f17d404ce50 con 0x7f17e40c1840 2023-11-06T21:59:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.939+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2315577607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f17e4063f20 con 0x7f17e40c1840 2023-11-06T21:59:37.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.940+0000 7f17da7fc700 1 -- 172.21.15.33:0/2315577607 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v166) v1 ==== 84+0+143 (secure 0 0 0) 0x7f17d4029020 con 0x7f17e40c1840 2023-11-06T21:59:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:37.943 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T21:59:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T21:59:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T21:59:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T21:59:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T21:59:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T21:59:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:37.944 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T21:59:37.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f17c805f270 msgr2=0x7f17c8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f17c805f270 0x7f17c8061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f17e000bb60 tx=0x7f17e0008040).stop 2023-11-06T21:59:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2315577607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40c1840 msgr2=0x7f17e4144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40c1840 0x7f17e4144ab0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f17d4011000 tx=0x7f17d4017000).stop 2023-11-06T21:59:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2315577607 shutdown_connections 2023-11-06T21:59:37.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f17c805f270 0x7f17c8061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f17e40aa2d0 0x7f17e4158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17e40c6240 0x7f17e40c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 --2- 172.21.15.33:0/2315577607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17e40c1840 0x7f17e4144ab0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:37.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2315577607 >> 172.21.15.33:0/2315577607 conn(0x7f17e405ec80 msgr2=0x7f17e4150dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:37.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2315577607 shutdown_connections 2023-11-06T21:59:37.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:37.942+0000 7f17ec5ff700 1 -- 172.21.15.33:0/2315577607 wait complete. 2023-11-06T21:59:37.962 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T21:59:37.962 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T21:59:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.161+0000 7fb5933ff700 1 Processor -- start 2023-11-06T21:59:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.161+0000 7fb5933ff700 1 -- start start 2023-11-06T21:59:38.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb5933ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb58c14e630 0x7fb58c14ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:38.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb5933ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c067c20 0x7fb58c155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:38.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb5933ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58c068170 con 0x7fb58c155800 2023-11-06T21:59:38.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb5933ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb58c068450 con 0x7fb58c14e630 2023-11-06T21:59:38.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb5933ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c068730 con 0x7fb58c067c20 2023-11-06T21:59:38.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb59119b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb58c14e630 0x7fb58c14ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:38.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb59119b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb58c14e630 0x7fb58c14ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58718/0 (socket says 172.21.15.33:58718) 2023-11-06T21:59:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb59119b700 1 -- 172.21.15.33:0/3116761655 learned_addr learned my addr 172.21.15.33:0/3116761655 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb59099a700 1 --2- 172.21.15.33:0/3116761655 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c067c20 0x7fb58c155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb59119b700 1 -- 172.21.15.33:0/3116761655 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c067c20 msgr2=0x7fb58c155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb59119b700 1 --2- 172.21.15.33:0/3116761655 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c067c20 0x7fb58c155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb59119b700 1 -- 172.21.15.33:0/3116761655 >> v1:172.21.15.33:6789/0 conn(0x7fb58c155800 legacy=0x7fb58c0666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb59119b700 1 -- 172.21.15.33:0/3116761655 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c068a10 con 0x7fb58c14e630 2023-11-06T21:59:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.163+0000 7fb59099a700 1 --2- 172.21.15.33:0/3116761655 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c067c20 0x7fb58c155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.164+0000 7fb59119b700 1 --2- 172.21.15.33:0/3116761655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c14e630 0x7fb58c14ea10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7fb588006fd0 tx=0x7fb5880090f0).ready entity=mon.0 client_cookie=c84e55e77c9da95f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.164+0000 7fb583fff700 1 -- 172.21.15.33:0/3116761655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800b600 con 0x7fb58c14e630 2023-11-06T21:59:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.164+0000 7fb583fff700 1 -- 172.21.15.33:0/3116761655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58800b760 con 0x7fb58c14e630 2023-11-06T21:59:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.164+0000 7fb583fff700 1 -- 172.21.15.33:0/3116761655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800bae0 con 0x7fb58c14e630 2023-11-06T21:59:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.164+0000 7fb5933ff700 1 -- 172.21.15.33:0/3116761655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c14e630 msgr2=0x7fb58c14ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.164+0000 7fb5933ff700 1 --2- 172.21.15.33:0/3116761655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c14e630 0x7fb58c14ea10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7fb588006fd0 tx=0x7fb5880090f0).stop 2023-11-06T21:59:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.164+0000 7fb5933ff700 1 -- 172.21.15.33:0/3116761655 shutdown_connections 2023-11-06T21:59:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.164+0000 7fb5933ff700 1 --2- 172.21.15.33:0/3116761655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c14e630 0x7fb58c14ea10 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.164+0000 7fb5933ff700 1 --2- 172.21.15.33:0/3116761655 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c067c20 0x7fb58c155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.164+0000 7fb5933ff700 1 -- 172.21.15.33:0/3116761655 >> 172.21.15.33:0/3116761655 conn(0x7fb58c0c8e60 msgr2=0x7fb58c0c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.165+0000 7fb5933ff700 1 -- 172.21.15.33:0/3116761655 shutdown_connections 2023-11-06T21:59:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.165+0000 7fb5933ff700 1 -- 172.21.15.33:0/3116761655 wait complete. 2023-11-06T21:59:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.167+0000 7fb5933ff700 1 Processor -- start 2023-11-06T21:59:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.167+0000 7fb5933ff700 1 -- start start 2023-11-06T21:59:38.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.168+0000 7fb5933ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c067c20 0x7fb58c134b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:38.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.168+0000 7fb5933ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb58c14e630 0x7fb58c0abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:38.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.168+0000 7fb5933ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c155800 0x7fb58c0add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.168+0000 7fb5933ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0ad350 con 0x7fb58c067c20 2023-11-06T21:59:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.168+0000 7fb5933ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0ac740 con 0x7fb58c14e630 2023-11-06T21:59:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.168+0000 7fb5933ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0aca20 con 0x7fb58c155800 2023-11-06T21:59:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.168+0000 7fb59119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c067c20 0x7fb58c134b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.168+0000 7fb59119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c067c20 0x7fb58c134b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58726/0 (socket says 172.21.15.33:58726) 2023-11-06T21:59:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.168+0000 7fb59119b700 1 -- 172.21.15.33:0/2190839959 learned_addr learned my addr 172.21.15.33:0/2190839959 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:38.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb59099a700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb58c14e630 0x7fb58c0abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:38.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb59199c700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c155800 0x7fb58c0add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:38.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb59119b700 1 -- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c155800 msgr2=0x7fb58c0add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:38.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb59119b700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c155800 0x7fb58c0add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:38.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb59119b700 1 -- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb58c14e630 msgr2=0x7fb58c0abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:38.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb59119b700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb58c14e630 0x7fb58c0abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:38.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb59119b700 1 -- 172.21.15.33:0/2190839959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb588009370 con 0x7fb58c067c20 2023-11-06T21:59:38.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb59119b700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c067c20 0x7fb58c134b60 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7fb58c063370 tx=0x7fb588017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:38.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb5817fa700 1 -- 172.21.15.33:0/2190839959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800b8d0 con 0x7fb58c067c20 2023-11-06T21:59:38.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb5817fa700 1 -- 172.21.15.33:0/2190839959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58800ba30 con 0x7fb58c067c20 2023-11-06T21:59:38.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb5817fa700 1 -- 172.21.15.33:0/2190839959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58801c650 con 0x7fb58c067c20 2023-11-06T21:59:38.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb5933ff700 1 -- 172.21.15.33:0/2190839959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58c0acd00 con 0x7fb58c067c20 2023-11-06T21:59:38.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.169+0000 7fb5933ff700 1 -- 172.21.15.33:0/2190839959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58c0ad7e0 con 0x7fb58c067c20 2023-11-06T21:59:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.170+0000 7fb5933ff700 1 -- 172.21.15.33:0/2190839959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58c0ad630 con 0x7fb58c067c20 2023-11-06T21:59:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.171+0000 7fb5817fa700 1 -- 172.21.15.33:0/2190839959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb58801ca00 con 0x7fb58c067c20 2023-11-06T21:59:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.173+0000 7fb5817fa700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb57005f270 0x7fb570061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.173+0000 7fb5817fa700 1 -- 172.21.15.33:0/2190839959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5313+0+0 (secure 0 0 0) 0x7fb58807c010 con 0x7fb58c067c20 2023-11-06T21:59:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.173+0000 7fb59099a700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb57005f270 0x7fb570061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:38.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.174+0000 7fb59099a700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb57005f270 0x7fb570061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb58c14e220 tx=0x7fb57c008040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:38.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.177+0000 7fb5817fa700 1 -- 172.21.15.33:0/2190839959 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb58804d3c0 con 0x7fb58c067c20 2023-11-06T21:59:38.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:38.361+0000 7fb5933ff700 1 -- 172.21.15.33:0/2190839959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fb58c068cf0 con 0x7fb58c067c20 2023-11-06T21:59:39.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.370+0000 7fb5817fa700 1 -- 172.21.15.33:0/2190839959 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v167) v1 ==== 152+0+0 (secure 0 0 0) 0x7fb58802a030 con 0x7fb58c067c20 2023-11-06T21:59:39.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.375+0000 7fb5933ff700 1 -- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb57005f270 msgr2=0x7fb570061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:39.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.375+0000 7fb5933ff700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb57005f270 0x7fb570061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb58c14e220 tx=0x7fb57c008040).stop 2023-11-06T21:59:39.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.375+0000 7fb5933ff700 1 -- 172.21.15.33:0/2190839959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c067c20 msgr2=0x7fb58c134b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:39.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.375+0000 7fb5933ff700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c067c20 0x7fb58c134b60 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7fb58c063370 tx=0x7fb588017000).stop 2023-11-06T21:59:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.376+0000 7fb5933ff700 1 -- 172.21.15.33:0/2190839959 shutdown_connections 2023-11-06T21:59:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.376+0000 7fb5933ff700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb57005f270 0x7fb570061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.376+0000 7fb5933ff700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb58c155800 0x7fb58c0add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.376+0000 7fb5933ff700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb58c067c20 0x7fb58c134b60 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:39.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.376+0000 7fb5933ff700 1 --2- 172.21.15.33:0/2190839959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb58c14e630 0x7fb58c0abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:39.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.376+0000 7fb5933ff700 1 -- 172.21.15.33:0/2190839959 >> 172.21.15.33:0/2190839959 conn(0x7fb58c0c8e60 msgr2=0x7fb58c156900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:39.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.377+0000 7fb5933ff700 1 -- 172.21.15.33:0/2190839959 shutdown_connections 2023-11-06T21:59:39.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.378+0000 7fb5933ff700 1 -- 172.21.15.33:0/2190839959 wait complete. 2023-11-06T21:59:39.386 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T21:59:39.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T21:59:39.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.610+0000 7ff7417ff700 1 Processor -- start 2023-11-06T21:59:39.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.611+0000 7ff7417ff700 1 -- start start 2023-11-06T21:59:39.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.612+0000 7ff7417ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff73c154710 0x7ff73c0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:39.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.612+0000 7ff7417ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c0c7ff0 0x7ff73c0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.612+0000 7ff7417ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c0c6b60 con 0x7ff73c154330 2023-11-06T21:59:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.612+0000 7ff7417ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73c0c7700 con 0x7ff73c154710 2023-11-06T21:59:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.612+0000 7ff7417ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0c5cd0 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.612+0000 7ff73a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff73c154710 0x7ff73c0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.612+0000 7ff73a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff73c154710 0x7ff73c0c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58734/0 (socket says 172.21.15.33:58734) 2023-11-06T21:59:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.612+0000 7ff73a7fc700 1 -- 172.21.15.33:0/131120972 learned_addr learned my addr 172.21.15.33:0/131120972 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.613+0000 7ff73b7fe700 1 --2- 172.21.15.33:0/131120972 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c0c7ff0 0x7ff73c0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.613+0000 7ff73a7fc700 1 -- 172.21.15.33:0/131120972 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c0c7ff0 msgr2=0x7ff73c0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.613+0000 7ff73a7fc700 1 --2- 172.21.15.33:0/131120972 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c0c7ff0 0x7ff73c0646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.613+0000 7ff73a7fc700 1 -- 172.21.15.33:0/131120972 >> v1:172.21.15.69:6789/0 conn(0x7ff73c154330 legacy=0x7ff73c0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.613+0000 7ff73a7fc700 1 -- 172.21.15.33:0/131120972 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c0c5f50 con 0x7ff73c154710 2023-11-06T21:59:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.613+0000 7ff73b7fe700 1 --2- 172.21.15.33:0/131120972 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c0c7ff0 0x7ff73c0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:39.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.613+0000 7ff73a7fc700 1 --2- 172.21.15.33:0/131120972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c154710 0x7ff73c0c57e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7ff730006fd0 tx=0x7ff7300090f0).ready entity=mon.0 client_cookie=fbe745d8299cd54 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:39.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.613+0000 7ff739ffb700 1 -- 172.21.15.33:0/131120972 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73000b600 con 0x7ff73c154710 2023-11-06T21:59:39.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.613+0000 7ff739ffb700 1 -- 172.21.15.33:0/131120972 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73000b760 con 0x7ff73c154710 2023-11-06T21:59:39.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.613+0000 7ff739ffb700 1 -- 172.21.15.33:0/131120972 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73000bae0 con 0x7ff73c154710 2023-11-06T21:59:39.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.614+0000 7ff7417ff700 1 -- 172.21.15.33:0/131120972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c154710 msgr2=0x7ff73c0c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:39.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.614+0000 7ff7417ff700 1 --2- 172.21.15.33:0/131120972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c154710 0x7ff73c0c57e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7ff730006fd0 tx=0x7ff7300090f0).stop 2023-11-06T21:59:39.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.614+0000 7ff7417ff700 1 -- 172.21.15.33:0/131120972 shutdown_connections 2023-11-06T21:59:39.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.614+0000 7ff7417ff700 1 --2- 172.21.15.33:0/131120972 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c0c7ff0 0x7ff73c0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:39.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.614+0000 7ff7417ff700 1 --2- 172.21.15.33:0/131120972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c154710 0x7ff73c0c57e0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:39.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.614+0000 7ff7417ff700 1 -- 172.21.15.33:0/131120972 >> 172.21.15.33:0/131120972 conn(0x7ff73c0cb910 msgr2=0x7ff73c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:39.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.614+0000 7ff7417ff700 1 -- 172.21.15.33:0/131120972 shutdown_connections 2023-11-06T21:59:39.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.614+0000 7ff7417ff700 1 -- 172.21.15.33:0/131120972 wait complete. 2023-11-06T21:59:39.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.615+0000 7ff7417ff700 1 Processor -- start 2023-11-06T21:59:39.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.616+0000 7ff7417ff700 1 -- start start 2023-11-06T21:59:39.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.616+0000 7ff7417ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c0c7ff0 0x7ff73c0b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:39.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.616+0000 7ff7417ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff73c154330 0x7ff73c0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:39.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.616+0000 7ff7417ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c154710 0x7ff73c0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:39.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff7417ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0ae0b0 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff7417ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c168aa0 con 0x7ff73c154330 2023-11-06T21:59:39.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff7417ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c168d80 con 0x7ff73c154710 2023-11-06T21:59:39.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c0c7ff0 0x7ff73c0b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:39.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c0c7ff0 0x7ff73c0b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58746/0 (socket says 172.21.15.33:58746) 2023-11-06T21:59:39.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73affd700 1 -- 172.21.15.33:0/3685321214 learned_addr learned my addr 172.21.15.33:0/3685321214 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:39.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73a7fc700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff73c154330 0x7ff73c0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:39.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73b7fe700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c154710 0x7ff73c0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:39.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73affd700 1 -- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c154710 msgr2=0x7ff73c0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:39.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73affd700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c154710 0x7ff73c0af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:39.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73affd700 1 -- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff73c154330 msgr2=0x7ff73c0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:39.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73affd700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff73c154330 0x7ff73c0b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:39.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73affd700 1 -- 172.21.15.33:0/3685321214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff730009370 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.617+0000 7ff73affd700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c0c7ff0 0x7ff73c0b0560 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7ff72c00b490 tx=0x7ff72c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:39.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.618+0000 7ff722ffd700 1 -- 172.21.15.33:0/3685321214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00be40 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.618+0000 7ff722ffd700 1 -- 172.21.15.33:0/3685321214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c012410 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.618+0000 7ff722ffd700 1 -- 172.21.15.33:0/3685321214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0127b0 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.618+0000 7ff7417ff700 1 -- 172.21.15.33:0/3685321214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c155120 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.618+0000 7ff7417ff700 1 -- 172.21.15.33:0/3685321214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c1556e0 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.619+0000 7ff7417ff700 1 -- 172.21.15.33:0/3685321214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c0830b0 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.620+0000 7ff722ffd700 1 -- 172.21.15.33:0/3685321214 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff72c012b60 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.621+0000 7ff722ffd700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff72805f270 0x7ff728061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:39.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.621+0000 7ff722ffd700 1 -- 172.21.15.33:0/3685321214 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5683+0+0 (secure 0 0 0) 0x7ff72c079eb0 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.621+0000 7ff73a7fc700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff72805f270 0x7ff728061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:39.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.622+0000 7ff73a7fc700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff72805f270 0x7ff728061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff73c154b70 tx=0x7ff730017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:39.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.626+0000 7ff722ffd700 1 -- 172.21.15.33:0/3685321214 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff72c04b1b0 con 0x7ff73c0c7ff0 2023-11-06T21:59:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:39.806+0000 7ff7417ff700 1 -- 172.21.15.33:0/3685321214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff73c0652d0 con 0x7ff73c0c7ff0 2023-11-06T21:59:40.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.378+0000 7ff722ffd700 1 -- 172.21.15.33:0/3685321214 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v168) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff72c04e7d0 con 0x7ff73c0c7ff0 2023-11-06T21:59:40.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 -- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff72805f270 msgr2=0x7ff728061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:40.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff72805f270 0x7ff728061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff73c154b70 tx=0x7ff730017000).stop 2023-11-06T21:59:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 -- 172.21.15.33:0/3685321214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c0c7ff0 msgr2=0x7ff73c0b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c0c7ff0 0x7ff73c0b0560 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7ff72c00b490 tx=0x7ff72c009060).stop 2023-11-06T21:59:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 -- 172.21.15.33:0/3685321214 shutdown_connections 2023-11-06T21:59:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff72805f270 0x7ff728061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff73c154710 0x7ff73c0af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff73c0c7ff0 0x7ff73c0b0560 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 --2- 172.21.15.33:0/3685321214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff73c154330 0x7ff73c0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 -- 172.21.15.33:0/3685321214 >> 172.21.15.33:0/3685321214 conn(0x7ff73c0cb910 msgr2=0x7ff73c0c9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 -- 172.21.15.33:0/3685321214 shutdown_connections 2023-11-06T21:59:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.383+0000 7ff7417ff700 1 -- 172.21.15.33:0/3685321214 wait complete. 2023-11-06T21:59:40.388 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T21:59:40.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T21:59:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.620+0000 7fbca5fff700 1 Processor -- start 2023-11-06T21:59:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.620+0000 7fbca5fff700 1 -- start start 2023-11-06T21:59:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.622+0000 7fbca5fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbca0064b40 0x7fbca00c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.622+0000 7fbca5fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca01556d0 con 0x7fbca01544f0 2023-11-06T21:59:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.622+0000 7fbca5fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca0154ac0 con 0x7fbca00646c0 2023-11-06T21:59:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.622+0000 7fbca5fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca01548d0 con 0x7fbca0064b40 2023-11-06T21:59:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.622+0000 7fbc9f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbca01544f0 0x7fbca00c0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57670/0 (socket says 172.21.15.33:57670) 2023-11-06T21:59:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.622+0000 7fbc9f7fe700 1 -- 172.21.15.33:0/3138527405 learned_addr learned my addr 172.21.15.33:0/3138527405 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:40.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.622+0000 7fbc9ffff700 1 --2- 172.21.15.33:0/3138527405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbca0064b40 0x7fbca00c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.622+0000 7fbc9e7fc700 1 -- 172.21.15.33:0/3138527405 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161607984 0 0) 0x7fbca01556d0 con 0x7fbca01544f0 2023-11-06T21:59:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9e7fc700 1 -- 172.21.15.33:0/3138527405 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc880033f0 con 0x7fbca01544f0 2023-11-06T21:59:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9e7fc700 1 -- 172.21.15.33:0/3138527405 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261624261 0 0) 0x7fbca0154ac0 con 0x7fbca00646c0 2023-11-06T21:59:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9e7fc700 1 -- 172.21.15.33:0/3138527405 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca01556d0 con 0x7fbca00646c0 2023-11-06T21:59:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9e7fc700 1 -- 172.21.15.33:0/3138527405 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1413638498 0 0) 0x7fbc880033f0 con 0x7fbca01544f0 2023-11-06T21:59:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9ffff700 1 -- 172.21.15.33:0/3138527405 >> v1:172.21.15.69:6789/0 conn(0x7fbca00646c0 legacy=0x7fbca00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9ffff700 1 -- 172.21.15.33:0/3138527405 >> v1:172.21.15.33:6789/0 conn(0x7fbca01544f0 legacy=0x7fbca00c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9ffff700 1 -- 172.21.15.33:0/3138527405 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca0154da0 con 0x7fbca0064b40 2023-11-06T21:59:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9ffff700 1 --2- 172.21.15.33:0/3138527405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbca0064b40 0x7fbca00c5ef0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fbc98009e90 tx=0x7fbc98009780).ready entity=mon.2 client_cookie=ac8fad11671ab9c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9e7fc700 1 -- 172.21.15.33:0/3138527405 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9800e070 con 0x7fbca0064b40 2023-11-06T21:59:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9e7fc700 1 -- 172.21.15.33:0/3138527405 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9800b7a0 con 0x7fbca0064b40 2023-11-06T21:59:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.623+0000 7fbc9e7fc700 1 -- 172.21.15.33:0/3138527405 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90001470 con 0x7fbca0064b40 2023-11-06T21:59:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.624+0000 7fbca5fff700 1 -- 172.21.15.33:0/3138527405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbca0064b40 msgr2=0x7fbca00c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.624+0000 7fbca5fff700 1 --2- 172.21.15.33:0/3138527405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbca0064b40 0x7fbca00c5ef0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fbc98009e90 tx=0x7fbc98009780).stop 2023-11-06T21:59:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.624+0000 7fbca5fff700 1 -- 172.21.15.33:0/3138527405 shutdown_connections 2023-11-06T21:59:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.624+0000 7fbca5fff700 1 --2- 172.21.15.33:0/3138527405 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbca0064b40 0x7fbca00c5ef0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.624+0000 7fbca5fff700 1 -- 172.21.15.33:0/3138527405 >> 172.21.15.33:0/3138527405 conn(0x7fbca00cb900 msgr2=0x7fbca00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.624+0000 7fbca5fff700 1 -- 172.21.15.33:0/3138527405 shutdown_connections 2023-11-06T21:59:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.624+0000 7fbca5fff700 1 -- 172.21.15.33:0/3138527405 wait complete. 2023-11-06T21:59:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.626+0000 7fbca5fff700 1 Processor -- start 2023-11-06T21:59:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.626+0000 7fbca5fff700 1 -- start start 2023-11-06T21:59:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbca5fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbca00646c0 0x7fbca01528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbca5fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbca0064b40 0x7fbca00c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbca5fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbca01544f0 0x7fbca00c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbca5fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca00c3460 con 0x7fbca00646c0 2023-11-06T21:59:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbca5fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca00c2850 con 0x7fbca01544f0 2023-11-06T21:59:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbca5fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca00c2b30 con 0x7fbca0064b40 2023-11-06T21:59:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbc9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbca00646c0 0x7fbca01528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbc9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbca00646c0 0x7fbca01528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58754/0 (socket says 172.21.15.33:58754) 2023-11-06T21:59:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbc9f7fe700 1 -- 172.21.15.33:0/60395747 learned_addr learned my addr 172.21.15.33:0/60395747 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:40.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbc9f7fe700 1 -- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbca0064b40 msgr2=0x7fbca00c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:40.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbc9f7fe700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbca0064b40 0x7fbca00c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:40.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbc9f7fe700 1 -- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbca01544f0 msgr2=0x7fbca00c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:40.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbc9f7fe700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbca01544f0 0x7fbca00c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:40.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.627+0000 7fbc9f7fe700 1 -- 172.21.15.33:0/60395747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc98009a20 con 0x7fbca00646c0 2023-11-06T21:59:40.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.628+0000 7fbc9f7fe700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbca00646c0 0x7fbca01528b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7fbc9000c810 tx=0x7fbc90001ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:40.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.628+0000 7fbc877fe700 1 -- 172.21.15.33:0/60395747 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90007170 con 0x7fbca00646c0 2023-11-06T21:59:40.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.628+0000 7fbc877fe700 1 -- 172.21.15.33:0/60395747 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc900072f0 con 0x7fbca00646c0 2023-11-06T21:59:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.628+0000 7fbc877fe700 1 -- 172.21.15.33:0/60395747 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900048c0 con 0x7fbca00646c0 2023-11-06T21:59:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.628+0000 7fbca5fff700 1 -- 172.21.15.33:0/60395747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca00c2e10 con 0x7fbca00646c0 2023-11-06T21:59:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.628+0000 7fbca5fff700 1 -- 172.21.15.33:0/60395747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca0152da0 con 0x7fbca00646c0 2023-11-06T21:59:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.629+0000 7fbca5fff700 1 -- 172.21.15.33:0/60395747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca00c3740 con 0x7fbca00646c0 2023-11-06T21:59:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.630+0000 7fbc877fe700 1 -- 172.21.15.33:0/60395747 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbc90004c70 con 0x7fbca00646c0 2023-11-06T21:59:40.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.631+0000 7fbc877fe700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc8c05f270 0x7fbc8c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:40.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.631+0000 7fbc877fe700 1 -- 172.21.15.33:0/60395747 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6065+0+0 (secure 0 0 0) 0x7fbc90079e40 con 0x7fbca00646c0 2023-11-06T21:59:40.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.632+0000 7fbc9effd700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc8c05f270 0x7fbc8c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:40.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.632+0000 7fbc9effd700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc8c05f270 0x7fbc8c061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fbca01679a0 tx=0x7fbc94007810).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:40.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.636+0000 7fbc877fe700 1 -- 172.21.15.33:0/60395747 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbc9004afc0 con 0x7fbca00646c0 2023-11-06T21:59:40.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.863+0000 7fbca5fff700 1 -- 172.21.15.33:0/60395747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbca00c8760 con 0x7fbca00646c0 2023-11-06T21:59:40.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.872+0000 7fbc877fe700 1 -- 172.21.15.33:0/60395747 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v64) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbc90019070 con 0x7fbca00646c0 2023-11-06T21:59:40.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.878+0000 7fbca5fff700 1 -- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc8c05f270 msgr2=0x7fbc8c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:40.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.878+0000 7fbca5fff700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc8c05f270 0x7fbc8c061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fbca01679a0 tx=0x7fbc94007810).stop 2023-11-06T21:59:40.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.878+0000 7fbca5fff700 1 -- 172.21.15.33:0/60395747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbca00646c0 msgr2=0x7fbca01528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:40.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.878+0000 7fbca5fff700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbca00646c0 0x7fbca01528b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7fbc9000c810 tx=0x7fbc90001ab0).stop 2023-11-06T21:59:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.878+0000 7fbca5fff700 1 -- 172.21.15.33:0/60395747 shutdown_connections 2023-11-06T21:59:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.878+0000 7fbca5fff700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbc8c05f270 0x7fbc8c061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.878+0000 7fbca5fff700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbca01544f0 0x7fbca00c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.878+0000 7fbca5fff700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbca00646c0 0x7fbca01528b0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.878+0000 7fbca5fff700 1 --2- 172.21.15.33:0/60395747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbca0064b40 0x7fbca00c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:40.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.878+0000 7fbca5fff700 1 -- 172.21.15.33:0/60395747 >> 172.21.15.33:0/60395747 conn(0x7fbca00cb900 msgr2=0x7fbca0156de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:40.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.879+0000 7fbca5fff700 1 -- 172.21.15.33:0/60395747 shutdown_connections 2023-11-06T21:59:40.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:40.879+0000 7fbca5fff700 1 -- 172.21.15.33:0/60395747 wait complete. 2023-11-06T21:59:40.887 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 19 and data pool 20 2023-11-06T21:59:40.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:59:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.114+0000 7f16335ff700 1 Processor -- start 2023-11-06T21:59:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.114+0000 7f16335ff700 1 -- start start 2023-11-06T21:59:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.115+0000 7f16335ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f162c064ce0 0x7f162c1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.115+0000 7f16335ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f162c0aade0 0x7f162c0c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.115+0000 7f16335ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162c0c0880 con 0x7f162c154110 2023-11-06T21:59:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.115+0000 7f16335ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162c0bfd40 con 0x7f162c064ce0 2023-11-06T21:59:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.115+0000 7f16335ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0c8260 con 0x7f162c0aade0 2023-11-06T21:59:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.116+0000 7f1630b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f162c064ce0 0x7f162c1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.116+0000 7f1630b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f162c064ce0 0x7f162c1544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58762/0 (socket says 172.21.15.33:58762) 2023-11-06T21:59:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.116+0000 7f1630b9a700 1 -- 172.21.15.33:0/2408914434 learned_addr learned my addr 172.21.15.33:0/2408914434 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.116+0000 7f1631b9c700 1 --2- 172.21.15.33:0/2408914434 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f162c0aade0 0x7f162c0c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.116+0000 7f1630b9a700 1 -- 172.21.15.33:0/2408914434 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f162c0aade0 msgr2=0x7f162c0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.116+0000 7f1630b9a700 1 --2- 172.21.15.33:0/2408914434 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f162c0aade0 0x7f162c0c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.116+0000 7f1630b9a700 1 -- 172.21.15.33:0/2408914434 >> v1:172.21.15.69:6789/0 conn(0x7f162c154110 legacy=0x7f162c0645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.116+0000 7f1630b9a700 1 -- 172.21.15.33:0/2408914434 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c0bff50 con 0x7f162c064ce0 2023-11-06T21:59:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.116+0000 7f1630b9a700 1 --2- 172.21.15.33:0/2408914434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c064ce0 0x7f162c1544f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f161c006fd0 tx=0x7f161c0090f0).ready entity=mon.0 client_cookie=faf08da9220a8309 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.116+0000 7f1623fff700 1 -- 172.21.15.33:0/2408914434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00b600 con 0x7f162c064ce0 2023-11-06T21:59:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.117+0000 7f1623fff700 1 -- 172.21.15.33:0/2408914434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c00b760 con 0x7f162c064ce0 2023-11-06T21:59:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.117+0000 7f1623fff700 1 -- 172.21.15.33:0/2408914434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00bae0 con 0x7f162c064ce0 2023-11-06T21:59:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.117+0000 7f16335ff700 1 -- 172.21.15.33:0/2408914434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c064ce0 msgr2=0x7f162c1544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.117+0000 7f16335ff700 1 --2- 172.21.15.33:0/2408914434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c064ce0 0x7f162c1544f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f161c006fd0 tx=0x7f161c0090f0).stop 2023-11-06T21:59:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.117+0000 7f16335ff700 1 -- 172.21.15.33:0/2408914434 shutdown_connections 2023-11-06T21:59:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.117+0000 7f16335ff700 1 --2- 172.21.15.33:0/2408914434 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f162c0aade0 0x7f162c0c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.117+0000 7f16335ff700 1 --2- 172.21.15.33:0/2408914434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c064ce0 0x7f162c1544f0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.117+0000 7f16335ff700 1 -- 172.21.15.33:0/2408914434 >> 172.21.15.33:0/2408914434 conn(0x7f162c0cb8b0 msgr2=0x7f162c0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.117+0000 7f16335ff700 1 -- 172.21.15.33:0/2408914434 shutdown_connections 2023-11-06T21:59:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.117+0000 7f16335ff700 1 -- 172.21.15.33:0/2408914434 wait complete. 2023-11-06T21:59:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.119+0000 7f16335ff700 1 Processor -- start 2023-11-06T21:59:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.119+0000 7f16335ff700 1 -- start start 2023-11-06T21:59:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f16335ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f162c064ce0 0x7f162c14db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f16335ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c0aade0 0x7f162c152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f16335ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c154110 0x7f162c14f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f16335ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0b0010 con 0x7f162c154110 2023-11-06T21:59:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f16335ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c150fa0 con 0x7f162c0aade0 2023-11-06T21:59:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f16335ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c14e080 con 0x7f162c064ce0 2023-11-06T21:59:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f1631b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c154110 0x7f162c14f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f1631b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c154110 0x7f162c14f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58776/0 (socket says 172.21.15.33:58776) 2023-11-06T21:59:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f1631b9c700 1 -- 172.21.15.33:0/4242509077 learned_addr learned my addr 172.21.15.33:0/4242509077 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f1630b9a700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c0aade0 0x7f162c152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.120+0000 7f163139b700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f162c064ce0 0x7f162c14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f1631b9c700 1 -- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f162c064ce0 msgr2=0x7f162c14db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f1631b9c700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f162c064ce0 0x7f162c14db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f1631b9c700 1 -- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c0aade0 msgr2=0x7f162c152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f1631b9c700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c0aade0 0x7f162c152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f1631b9c700 1 -- 172.21.15.33:0/4242509077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c009370 con 0x7f162c154110 2023-11-06T21:59:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f1631b9c700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c154110 0x7f162c14f950 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7f1624007f30 tx=0x7f16240098e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f16217fa700 1 -- 172.21.15.33:0/4242509077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162400eb90 con 0x7f162c154110 2023-11-06T21:59:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f16217fa700 1 -- 172.21.15.33:0/4242509077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162400ecf0 con 0x7f162c154110 2023-11-06T21:59:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f16335ff700 1 -- 172.21.15.33:0/4242509077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c0b0200 con 0x7f162c154110 2023-11-06T21:59:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f16217fa700 1 -- 172.21.15.33:0/4242509077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624014610 con 0x7f162c154110 2023-11-06T21:59:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.121+0000 7f16335ff700 1 -- 172.21.15.33:0/4242509077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162c0b07c0 con 0x7f162c154110 2023-11-06T21:59:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.122+0000 7f16335ff700 1 -- 172.21.15.33:0/4242509077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162c151110 con 0x7f162c154110 2023-11-06T21:59:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.123+0000 7f16217fa700 1 -- 172.21.15.33:0/4242509077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f16240149c0 con 0x7f162c154110 2023-11-06T21:59:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.124+0000 7f16217fa700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f161005f270 0x7f1610061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.125+0000 7f16217fa700 1 -- 172.21.15.33:0/4242509077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f162407a850 con 0x7f162c154110 2023-11-06T21:59:41.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.125+0000 7f163139b700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f161005f270 0x7f1610061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.126+0000 7f163139b700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f161005f270 0x7f1610061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f1628003860 tx=0x7f1628007d40).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:41.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.128+0000 7f16217fa700 1 -- 172.21.15.33:0/4242509077 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f162c151110 con 0x7f162c154110 2023-11-06T21:59:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.308+0000 7f16335ff700 1 -- 172.21.15.33:0/4242509077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f162c151110 con 0x7f162c154110 2023-11-06T21:59:41.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.309+0000 7f16217fa700 1 -- 172.21.15.33:0/4242509077 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f162c151110 con 0x7f162c154110 2023-11-06T21:59:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:41.312 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":169,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:59:40.865543+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:59:38.416213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T21:59:39.859883+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:59:41.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 -- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f161005f270 msgr2=0x7f1610061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f161005f270 0x7f1610061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f1628003860 tx=0x7f1628007d40).stop 2023-11-06T21:59:41.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 -- 172.21.15.33:0/4242509077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c154110 msgr2=0x7f162c14f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c154110 0x7f162c14f950 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7f1624007f30 tx=0x7f16240098e0).stop 2023-11-06T21:59:41.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 -- 172.21.15.33:0/4242509077 shutdown_connections 2023-11-06T21:59:41.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f161005f270 0x7f1610061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f162c064ce0 0x7f162c14db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f162c154110 0x7f162c14f950 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 --2- 172.21.15.33:0/4242509077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c0aade0 0x7f162c152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 -- 172.21.15.33:0/4242509077 >> 172.21.15.33:0/4242509077 conn(0x7f162c0cb8b0 msgr2=0x7f162c0ac270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 -- 172.21.15.33:0/4242509077 shutdown_connections 2023-11-06T21:59:41.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.311+0000 7f16335ff700 1 -- 172.21.15.33:0/4242509077 wait complete. 2023-11-06T21:59:41.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:59:41.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.525+0000 7f8740bff700 1 Processor -- start 2023-11-06T21:59:41.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.525+0000 7f8740bff700 1 -- start start 2023-11-06T21:59:41.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f8740bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f873c154110 0x7f873c1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f8740bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f873c0aade0 0x7f873c0645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f8740bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873c064f70 con 0x7f873c064ac0 2023-11-06T21:59:41.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f8740bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873c0bfd40 con 0x7f873c0aade0 2023-11-06T21:59:41.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f8740bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0c0880 con 0x7f873c154110 2023-11-06T21:59:41.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f8739d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f873c0aade0 0x7f873c0645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f8739d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f873c0aade0 0x7f873c0645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58790/0 (socket says 172.21.15.33:58790) 2023-11-06T21:59:41.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f8739d9b700 1 -- 172.21.15.33:0/3104781140 learned_addr learned my addr 172.21.15.33:0/3104781140 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:41.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f873a59c700 1 --2- 172.21.15.33:0/3104781140 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f873c154110 0x7f873c1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f8739d9b700 1 -- 172.21.15.33:0/3104781140 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f873c154110 msgr2=0x7f873c1544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:41.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.527+0000 7f8739d9b700 1 --2- 172.21.15.33:0/3104781140 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f873c154110 0x7f873c1544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.528+0000 7f8739d9b700 1 -- 172.21.15.33:0/3104781140 >> v1:172.21.15.69:6789/0 conn(0x7f873c064ac0 legacy=0x7f873c0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.528+0000 7f8739d9b700 1 -- 172.21.15.33:0/3104781140 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c0c8260 con 0x7f873c0aade0 2023-11-06T21:59:41.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.528+0000 7f8739d9b700 1 --2- 172.21.15.33:0/3104781140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 0x7f873c0645d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f8730006fd0 tx=0x7f87300090f0).ready entity=mon.0 client_cookie=84665067a0ba87c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:41.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.528+0000 7f873959a700 1 -- 172.21.15.33:0/3104781140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873000b600 con 0x7f873c0aade0 2023-11-06T21:59:41.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.528+0000 7f873959a700 1 -- 172.21.15.33:0/3104781140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873000b760 con 0x7f873c0aade0 2023-11-06T21:59:41.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.528+0000 7f873959a700 1 -- 172.21.15.33:0/3104781140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873000bae0 con 0x7f873c0aade0 2023-11-06T21:59:41.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.528+0000 7f8740bff700 1 -- 172.21.15.33:0/3104781140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 msgr2=0x7f873c0645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.528+0000 7f8740bff700 1 --2- 172.21.15.33:0/3104781140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 0x7f873c0645d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f8730006fd0 tx=0x7f87300090f0).stop 2023-11-06T21:59:41.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.529+0000 7f8740bff700 1 -- 172.21.15.33:0/3104781140 shutdown_connections 2023-11-06T21:59:41.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.529+0000 7f8740bff700 1 --2- 172.21.15.33:0/3104781140 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f873c154110 0x7f873c1544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.529+0000 7f8740bff700 1 --2- 172.21.15.33:0/3104781140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 0x7f873c0645d0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.529+0000 7f8740bff700 1 -- 172.21.15.33:0/3104781140 >> 172.21.15.33:0/3104781140 conn(0x7f873c0cb8b0 msgr2=0x7f873c0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.529+0000 7f8740bff700 1 -- 172.21.15.33:0/3104781140 shutdown_connections 2023-11-06T21:59:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.529+0000 7f8740bff700 1 -- 172.21.15.33:0/3104781140 wait complete. 2023-11-06T21:59:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.530+0000 7f8740bff700 1 Processor -- start 2023-11-06T21:59:41.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.530+0000 7f8740bff700 1 -- start start 2023-11-06T21:59:41.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.531+0000 7f8740bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f873c064ac0 0x7f873c142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.531+0000 7f8740bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 0x7f873c0b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.531+0000 7f8740bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f873c154110 0x7f873c0b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.531+0000 7f8740bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1644e0 con 0x7f873c0aade0 2023-11-06T21:59:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.531+0000 7f8740bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1513c0 con 0x7f873c154110 2023-11-06T21:59:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.531+0000 7f8740bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1516a0 con 0x7f873c064ac0 2023-11-06T21:59:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.531+0000 7f8739d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 0x7f873c0b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f8739d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 0x7f873c0b0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58794/0 (socket says 172.21.15.33:58794) 2023-11-06T21:59:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f8739d9b700 1 -- 172.21.15.33:0/2793194905 learned_addr learned my addr 172.21.15.33:0/2793194905 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f873ad9d700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f873c154110 0x7f873c0b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f873a59c700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f873c064ac0 0x7f873c142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f8739d9b700 1 -- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f873c064ac0 msgr2=0x7f873c142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f8739d9b700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f873c064ac0 0x7f873c142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f8739d9b700 1 -- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f873c154110 msgr2=0x7f873c0b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f8739d9b700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f873c154110 0x7f873c0b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f8739d9b700 1 -- 172.21.15.33:0/2793194905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730009370 con 0x7f873c0aade0 2023-11-06T21:59:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f8739d9b700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 0x7f873c0b0390 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f8730011000 tx=0x7f8730017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f872a7fc700 1 -- 172.21.15.33:0/2793194905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873000bea0 con 0x7f873c0aade0 2023-11-06T21:59:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f872a7fc700 1 -- 172.21.15.33:0/2793194905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873000b8d0 con 0x7f873c0aade0 2023-11-06T21:59:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f872a7fc700 1 -- 172.21.15.33:0/2793194905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873001c670 con 0x7f873c0aade0 2023-11-06T21:59:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.532+0000 7f8740bff700 1 -- 172.21.15.33:0/2793194905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873c151980 con 0x7f873c0aade0 2023-11-06T21:59:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.533+0000 7f8740bff700 1 -- 172.21.15.33:0/2793194905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c151f40 con 0x7f873c0aade0 2023-11-06T21:59:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.534+0000 7f8740bff700 1 -- 172.21.15.33:0/2793194905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873c06c6c0 con 0x7f873c0aade0 2023-11-06T21:59:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.535+0000 7f872a7fc700 1 -- 172.21.15.33:0/2793194905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f873001c9e0 con 0x7f873c0aade0 2023-11-06T21:59:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.535+0000 7f872a7fc700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f871c05f270 0x7f871c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.536+0000 7f872a7fc700 1 -- 172.21.15.33:0/2793194905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f873007c180 con 0x7f873c0aade0 2023-11-06T21:59:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.536+0000 7f873a59c700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f871c05f270 0x7f871c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.536+0000 7f873a59c700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f871c05f270 0x7f871c061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f872c00bb10 tx=0x7f872c008040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.539+0000 7f872a7fc700 1 -- 172.21.15.33:0/2793194905 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8730050c10 con 0x7f873c0aade0 2023-11-06T21:59:41.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.719+0000 7f8740bff700 1 -- 172.21.15.33:0/2793194905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f873c152580 con 0x7f873c0aade0 2023-11-06T21:59:41.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.720+0000 7f872a7fc700 1 -- 172.21.15.33:0/2793194905 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f873c152580 con 0x7f873c0aade0 2023-11-06T21:59:41.722 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:41.723 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":169,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:59:40.865543+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:59:38.416213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T21:59:39.859883+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:59:41.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 -- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f871c05f270 msgr2=0x7f871c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f871c05f270 0x7f871c061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f872c00bb10 tx=0x7f872c008040).stop 2023-11-06T21:59:41.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 -- 172.21.15.33:0/2793194905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 msgr2=0x7f873c0b0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 0x7f873c0b0390 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f8730011000 tx=0x7f8730017000).stop 2023-11-06T21:59:41.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 -- 172.21.15.33:0/2793194905 shutdown_connections 2023-11-06T21:59:41.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f871c05f270 0x7f871c061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f873c064ac0 0x7f873c142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f873c154110 0x7f873c0b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 --2- 172.21.15.33:0/2793194905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f873c0aade0 0x7f873c0b0390 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 -- 172.21.15.33:0/2793194905 >> 172.21.15.33:0/2793194905 conn(0x7f873c0cb8b0 msgr2=0x7f873c0abad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:41.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 -- 172.21.15.33:0/2793194905 shutdown_connections 2023-11-06T21:59:41.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.722+0000 7f8740bff700 1 -- 172.21.15.33:0/2793194905 wait complete. 2023-11-06T21:59:41.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T21:59:41.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.941+0000 7fba039ff700 1 Processor -- start 2023-11-06T21:59:41.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.941+0000 7fba039ff700 1 -- start start 2023-11-06T21:59:41.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.942+0000 7fba039ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9fc154710 0x7fb9fc0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.942+0000 7fba039ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 0x7fb9fc0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.942+0000 7fba039ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc0c6b60 con 0x7fb9fc154330 2023-11-06T21:59:41.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.942+0000 7fba039ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9fc0c7700 con 0x7fb9fc154710 2023-11-06T21:59:41.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.942+0000 7fba039ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0c5cd0 con 0x7fb9fc0c7ff0 2023-11-06T21:59:41.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.942+0000 7fba00f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9fc154710 0x7fb9fc0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.942+0000 7fba00f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9fc154710 0x7fb9fc0c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58804/0 (socket says 172.21.15.33:58804) 2023-11-06T21:59:41.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.942+0000 7fba00f9a700 1 -- 172.21.15.33:0/3142118877 learned_addr learned my addr 172.21.15.33:0/3142118877 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:41.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.943+0000 7fba01f9c700 1 --2- 172.21.15.33:0/3142118877 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 0x7fb9fc0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.943+0000 7fba00f9a700 1 -- 172.21.15.33:0/3142118877 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 msgr2=0x7fb9fc0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:41.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.943+0000 7fba00f9a700 1 --2- 172.21.15.33:0/3142118877 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 0x7fb9fc0646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.943+0000 7fba00f9a700 1 -- 172.21.15.33:0/3142118877 >> v1:172.21.15.69:6789/0 conn(0x7fb9fc154330 legacy=0x7fb9fc0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.943+0000 7fba00f9a700 1 -- 172.21.15.33:0/3142118877 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0c5f50 con 0x7fb9fc154710 2023-11-06T21:59:41.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.943+0000 7fba00f9a700 1 --2- 172.21.15.33:0/3142118877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 0x7fb9fc0c57e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7fb9ec006fd0 tx=0x7fb9ec0090f0).ready entity=mon.0 client_cookie=8356a55bcdabf136 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.943+0000 7fb9f3fff700 1 -- 172.21.15.33:0/3142118877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00b600 con 0x7fb9fc154710 2023-11-06T21:59:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.943+0000 7fb9f3fff700 1 -- 172.21.15.33:0/3142118877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec00b760 con 0x7fb9fc154710 2023-11-06T21:59:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.943+0000 7fb9f3fff700 1 -- 172.21.15.33:0/3142118877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00bae0 con 0x7fb9fc154710 2023-11-06T21:59:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.944+0000 7fba039ff700 1 -- 172.21.15.33:0/3142118877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 msgr2=0x7fb9fc0c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.944+0000 7fba039ff700 1 --2- 172.21.15.33:0/3142118877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 0x7fb9fc0c57e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7fb9ec006fd0 tx=0x7fb9ec0090f0).stop 2023-11-06T21:59:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.944+0000 7fba039ff700 1 -- 172.21.15.33:0/3142118877 shutdown_connections 2023-11-06T21:59:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.944+0000 7fba039ff700 1 --2- 172.21.15.33:0/3142118877 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 0x7fb9fc0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.944+0000 7fba039ff700 1 --2- 172.21.15.33:0/3142118877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 0x7fb9fc0c57e0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.944+0000 7fba039ff700 1 -- 172.21.15.33:0/3142118877 >> 172.21.15.33:0/3142118877 conn(0x7fb9fc0cb910 msgr2=0x7fb9fc0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.944+0000 7fba039ff700 1 -- 172.21.15.33:0/3142118877 shutdown_connections 2023-11-06T21:59:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.944+0000 7fba039ff700 1 -- 172.21.15.33:0/3142118877 wait complete. 2023-11-06T21:59:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.945+0000 7fba039ff700 1 Processor -- start 2023-11-06T21:59:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.946+0000 7fba039ff700 1 -- start start 2023-11-06T21:59:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.946+0000 7fba039ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 0x7fb9fc138d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba039ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9fc154330 0x7fb9fc154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba039ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 0x7fb9fc155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba039ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc155580 con 0x7fb9fc154710 2023-11-06T21:59:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba039ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0c4ab0 con 0x7fb9fc154330 2023-11-06T21:59:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba039ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0c3ea0 con 0x7fb9fc0c7ff0 2023-11-06T21:59:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba01f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 0x7fb9fc155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba01f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 0x7fb9fc155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58814/0 (socket says 172.21.15.33:58814) 2023-11-06T21:59:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba01f9c700 1 -- 172.21.15.33:0/560871662 learned_addr learned my addr 172.21.15.33:0/560871662 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba00f9a700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9fc154330 0x7fb9fc154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba0179b700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 0x7fb9fc138d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba01f9c700 1 -- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 msgr2=0x7fb9fc138d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba01f9c700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 0x7fb9fc138d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba01f9c700 1 -- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9fc154330 msgr2=0x7fb9fc154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:41.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba01f9c700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9fc154330 0x7fb9fc154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:41.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba01f9c700 1 -- 172.21.15.33:0/560871662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec009370 con 0x7fb9fc154710 2023-11-06T21:59:41.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.947+0000 7fba0179b700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 0x7fb9fc138d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:41.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.948+0000 7fba01f9c700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 0x7fb9fc155090 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7fb9f4007f20 tx=0x7fb9f4009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.948+0000 7fb9f17fa700 1 -- 172.21.15.33:0/560871662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400ebf0 con 0x7fb9fc154710 2023-11-06T21:59:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.948+0000 7fb9f17fa700 1 -- 172.21.15.33:0/560871662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f400ed50 con 0x7fb9fc154710 2023-11-06T21:59:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.948+0000 7fb9f17fa700 1 -- 172.21.15.33:0/560871662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4014610 con 0x7fb9fc154710 2023-11-06T21:59:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.948+0000 7fba039ff700 1 -- 172.21.15.33:0/560871662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc0c4180 con 0x7fb9fc154710 2023-11-06T21:59:41.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.948+0000 7fba039ff700 1 -- 172.21.15.33:0/560871662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc0c4740 con 0x7fb9fc154710 2023-11-06T21:59:41.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.949+0000 7fba039ff700 1 -- 172.21.15.33:0/560871662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc0c4c40 con 0x7fb9fc154710 2023-11-06T21:59:41.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.950+0000 7fb9f17fa700 1 -- 172.21.15.33:0/560871662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb9f4014980 con 0x7fb9fc154710 2023-11-06T21:59:41.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.951+0000 7fb9f17fa700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb9e005f270 0x7fb9e0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:41.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.951+0000 7fb9f17fa700 1 -- 172.21.15.33:0/560871662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fb9f407a960 con 0x7fb9fc154710 2023-11-06T21:59:41.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.952+0000 7fba0179b700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb9e005f270 0x7fb9e0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:41.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.952+0000 7fba0179b700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb9e005f270 0x7fb9e0061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb9f8001730 tx=0x7fb9f8009040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:41.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:41.955+0000 7fb9f17fa700 1 -- 172.21.15.33:0/560871662 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb9f404b9c0 con 0x7fb9fc154710 2023-11-06T21:59:42.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.178+0000 7fba039ff700 1 -- 172.21.15.33:0/560871662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb9fc0652d0 con 0x7fb9fc154710 2023-11-06T21:59:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.884+0000 7fb9f17fa700 1 -- 172.21.15.33:0/560871662 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v67) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb9f404efe0 con 0x7fb9fc154710 2023-11-06T21:59:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 -- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb9e005f270 msgr2=0x7fb9e0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:42.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb9e005f270 0x7fb9e0061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb9f8001730 tx=0x7fb9f8009040).stop 2023-11-06T21:59:42.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 -- 172.21.15.33:0/560871662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 msgr2=0x7fb9fc155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:42.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 0x7fb9fc155090 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7fb9f4007f20 tx=0x7fb9f4009860).stop 2023-11-06T21:59:42.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 -- 172.21.15.33:0/560871662 shutdown_connections 2023-11-06T21:59:42.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb9e005f270 0x7fb9e0061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:42.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb9fc0c7ff0 0x7fb9fc138d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:42.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc154710 0x7fb9fc155090 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:42.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 --2- 172.21.15.33:0/560871662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9fc154330 0x7fb9fc154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:42.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 -- 172.21.15.33:0/560871662 >> 172.21.15.33:0/560871662 conn(0x7fb9fc0cb910 msgr2=0x7fb9fc0c90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:42.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 -- 172.21.15.33:0/560871662 shutdown_connections 2023-11-06T21:59:42.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:42.886+0000 7fba039ff700 1 -- 172.21.15.33:0/560871662 wait complete. 2023-11-06T21:59:42.908 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T21:59:42.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T21:59:43.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.107+0000 7f6f6ebff700 1 Processor -- start 2023-11-06T21:59:43.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.107+0000 7f6f6ebff700 1 -- start start 2023-11-06T21:59:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.109+0000 7f6f6ebff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6f68154710 0x7f6f680c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.109+0000 7f6f6ebff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f680c7ff0 0x7f6f680646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.109+0000 7f6f6ebff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f680c6b60 con 0x7f6f68154330 2023-11-06T21:59:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.109+0000 7f6f6ebff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f680c7700 con 0x7f6f680c7ff0 2023-11-06T21:59:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.109+0000 7f6f6ebff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f680c5cd0 con 0x7f6f68154710 2023-11-06T21:59:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.109+0000 7f6f6d19c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f680c7ff0 0x7f6f680646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.109+0000 7f6f6d19c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f680c7ff0 0x7f6f680646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58830/0 (socket says 172.21.15.33:58830) 2023-11-06T21:59:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.109+0000 7f6f6d19c700 1 -- 172.21.15.33:0/2460234890 learned_addr learned my addr 172.21.15.33:0/2460234890 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.109+0000 7f6f67fff700 1 --2- 172.21.15.33:0/2460234890 >> v2:172.21.15.69:3300/0 conn(0x7f6f68154710 0x7f6f680c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.110+0000 7f6f6d19c700 1 -- 172.21.15.33:0/2460234890 >> v2:172.21.15.69:3300/0 conn(0x7f6f68154710 msgr2=0x7f6f680c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.110+0000 7f6f6d19c700 1 --2- 172.21.15.33:0/2460234890 >> v2:172.21.15.69:3300/0 conn(0x7f6f68154710 0x7f6f680c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.110+0000 7f6f6d19c700 1 -- 172.21.15.33:0/2460234890 >> v1:172.21.15.69:6789/0 conn(0x7f6f68154330 legacy=0x7f6f680c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.110+0000 7f6f6d19c700 1 -- 172.21.15.33:0/2460234890 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680c5f50 con 0x7f6f680c7ff0 2023-11-06T21:59:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.110+0000 7f6f6d19c700 1 --2- 172.21.15.33:0/2460234890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f680c7ff0 0x7f6f680646c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f6f680c8d10 tx=0x7f6f60009840).ready entity=mon.0 client_cookie=28958f62c39b86e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.110+0000 7f6f677fe700 1 -- 172.21.15.33:0/2460234890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000e070 con 0x7f6f680c7ff0 2023-11-06T21:59:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.110+0000 7f6f677fe700 1 -- 172.21.15.33:0/2460234890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6000ba50 con 0x7f6f680c7ff0 2023-11-06T21:59:43.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.110+0000 7f6f677fe700 1 -- 172.21.15.33:0/2460234890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000bdd0 con 0x7f6f680c7ff0 2023-11-06T21:59:43.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.111+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2460234890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f680c7ff0 msgr2=0x7f6f680646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:43.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.111+0000 7f6f6ebff700 1 --2- 172.21.15.33:0/2460234890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f680c7ff0 0x7f6f680646c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f6f680c8d10 tx=0x7f6f60009840).stop 2023-11-06T21:59:43.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.111+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2460234890 shutdown_connections 2023-11-06T21:59:43.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.111+0000 7f6f6ebff700 1 --2- 172.21.15.33:0/2460234890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f680c7ff0 0x7f6f680646c0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:43.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.111+0000 7f6f6ebff700 1 --2- 172.21.15.33:0/2460234890 >> v2:172.21.15.69:3300/0 conn(0x7f6f68154710 0x7f6f680c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:43.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.111+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2460234890 >> 172.21.15.33:0/2460234890 conn(0x7f6f680cb910 msgr2=0x7f6f680c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:43.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.111+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2460234890 shutdown_connections 2023-11-06T21:59:43.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.111+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2460234890 wait complete. 2023-11-06T21:59:43.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.113+0000 7f6f6ebff700 1 Processor -- start 2023-11-06T21:59:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.113+0000 7f6f6ebff700 1 -- start start 2023-11-06T21:59:43.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.114+0000 7f6f6ebff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f680c7ff0 0x7f6f680ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:43.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.114+0000 7f6f6ebff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f68154330 0x7f6f680c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:43.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.114+0000 7f6f6ebff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f68154710 0x7f6f680c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:43.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6ebff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680c9c60 con 0x7f6f68154710 2023-11-06T21:59:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6ebff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6813f770 con 0x7f6f68154330 2023-11-06T21:59:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6ebff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681532d0 con 0x7f6f680c7ff0 2023-11-06T21:59:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6d19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f68154710 0x7f6f680c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6d19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f68154710 0x7f6f680c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58832/0 (socket says 172.21.15.33:58832) 2023-11-06T21:59:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6d19c700 1 -- 172.21.15.33:0/2481283399 learned_addr learned my addr 172.21.15.33:0/2481283399 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f67fff700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f68154330 0x7f6f680c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6c99b700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f680c7ff0 0x7f6f680ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:43.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6d19c700 1 -- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f680c7ff0 msgr2=0x7f6f680ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:43.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6d19c700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f680c7ff0 0x7f6f680ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:43.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6d19c700 1 -- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f68154330 msgr2=0x7f6f680c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6d19c700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f68154330 0x7f6f680c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6d19c700 1 -- 172.21.15.33:0/2481283399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f60009ae0 con 0x7f6f68154710 2023-11-06T21:59:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.115+0000 7f6f6d19c700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f68154710 0x7f6f680c9770 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f6f6806c170 tx=0x7f6f60009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.116+0000 7f6f64ff9700 1 -- 172.21.15.33:0/2481283399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f600092e0 con 0x7f6f68154710 2023-11-06T21:59:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.116+0000 7f6f64ff9700 1 -- 172.21.15.33:0/2481283399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6001d410 con 0x7f6f68154710 2023-11-06T21:59:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.116+0000 7f6f64ff9700 1 -- 172.21.15.33:0/2481283399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6001d790 con 0x7f6f68154710 2023-11-06T21:59:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.116+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2481283399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f680b0b10 con 0x7f6f68154710 2023-11-06T21:59:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.116+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2481283399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f681681a0 con 0x7f6f68154710 2023-11-06T21:59:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.117+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2481283399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f680830b0 con 0x7f6f68154710 2023-11-06T21:59:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.118+0000 7f6f64ff9700 1 -- 172.21.15.33:0/2481283399 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6f6001db00 con 0x7f6f68154710 2023-11-06T21:59:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.119+0000 7f6f64ff9700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f4c05f270 0x7f6f4c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.119+0000 7f6f64ff9700 1 -- 172.21.15.33:0/2481283399 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f6f6007c0a0 con 0x7f6f68154710 2023-11-06T21:59:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.119+0000 7f6f6c99b700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f4c05f270 0x7f6f4c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.120+0000 7f6f6c99b700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f4c05f270 0x7f6f4c061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f6f5c003900 tx=0x7f6f5c007d40).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.123+0000 7f6f64ff9700 1 -- 172.21.15.33:0/2481283399 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6f6004d1b0 con 0x7f6f68154710 2023-11-06T21:59:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.351+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2481283399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6f680652d0 con 0x7f6f68154710 2023-11-06T21:59:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.955+0000 7f6f64ff9700 1 -- 172.21.15.33:0/2481283399 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v68) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6f6001ddb0 con 0x7f6f68154710 2023-11-06T21:59:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.957+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f4c05f270 msgr2=0x7f6f4c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.957+0000 7f6f6ebff700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f4c05f270 0x7f6f4c061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f6f5c003900 tx=0x7f6f5c007d40).stop 2023-11-06T21:59:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.957+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2481283399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f68154710 msgr2=0x7f6f680c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.957+0000 7f6f6ebff700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f68154710 0x7f6f680c9770 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f6f6806c170 tx=0x7f6f60009840).stop 2023-11-06T21:59:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.958+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2481283399 shutdown_connections 2023-11-06T21:59:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.958+0000 7f6f6ebff700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6f4c05f270 0x7f6f4c061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.958+0000 7f6f6ebff700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f680c7ff0 0x7f6f680ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.958+0000 7f6f6ebff700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f68154710 0x7f6f680c9770 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.958+0000 7f6f6ebff700 1 --2- 172.21.15.33:0/2481283399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f68154330 0x7f6f680c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.958+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2481283399 >> 172.21.15.33:0/2481283399 conn(0x7f6f680cb910 msgr2=0x7f6f680ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.958+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2481283399 shutdown_connections 2023-11-06T21:59:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:43.958+0000 7f6f6ebff700 1 -- 172.21.15.33:0/2481283399 wait complete. 2023-11-06T21:59:43.981 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T21:59:43.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:59:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.180+0000 7f664b7ff700 1 Processor -- start 2023-11-06T21:59:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.180+0000 7f664b7ff700 1 -- start start 2023-11-06T21:59:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.181+0000 7f664b7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66440c1840 0x7f66440a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.181+0000 7f664b7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f66440aa2d0 0x7f66440c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.181+0000 7f664b7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66441554a0 con 0x7f66440c6240 2023-11-06T21:59:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.181+0000 7f664b7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6644154960 con 0x7f66440c1840 2023-11-06T21:59:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.181+0000 7f664b7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6644155930 con 0x7f66440aa2d0 2023-11-06T21:59:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.181+0000 7f664959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66440c1840 0x7f66440a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.181+0000 7f664959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66440c1840 0x7f66440a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58834/0 (socket says 172.21.15.33:58834) 2023-11-06T21:59:44.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.181+0000 7f664959b700 1 -- 172.21.15.33:0/285850091 learned_addr learned my addr 172.21.15.33:0/285850091 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:44.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.182+0000 7f6648d9a700 1 --2- 172.21.15.33:0/285850091 >> v2:172.21.15.69:3300/0 conn(0x7f66440aa2d0 0x7f66440c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.182+0000 7f664959b700 1 -- 172.21.15.33:0/285850091 >> v2:172.21.15.69:3300/0 conn(0x7f66440aa2d0 msgr2=0x7f66440c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:44.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.182+0000 7f664959b700 1 --2- 172.21.15.33:0/285850091 >> v2:172.21.15.69:3300/0 conn(0x7f66440aa2d0 0x7f66440c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.182+0000 7f664959b700 1 -- 172.21.15.33:0/285850091 >> v1:172.21.15.69:6789/0 conn(0x7f66440c6240 legacy=0x7f6644157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.182+0000 7f664959b700 1 -- 172.21.15.33:0/285850091 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644154b70 con 0x7f66440c1840 2023-11-06T21:59:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.182+0000 7f6648d9a700 1 --2- 172.21.15.33:0/285850091 >> v2:172.21.15.69:3300/0 conn(0x7f66440aa2d0 0x7f66440c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.182+0000 7f664959b700 1 --2- 172.21.15.33:0/285850091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 0x7f66440a9de0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7f66440adb40 tx=0x7f66400090f0).ready entity=mon.0 client_cookie=3e137594e6cbff0b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.182+0000 7f663bfff700 1 -- 172.21.15.33:0/285850091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664000b600 con 0x7f66440c1840 2023-11-06T21:59:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.182+0000 7f663bfff700 1 -- 172.21.15.33:0/285850091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664000b760 con 0x7f66440c1840 2023-11-06T21:59:44.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.182+0000 7f663bfff700 1 -- 172.21.15.33:0/285850091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664000bae0 con 0x7f66440c1840 2023-11-06T21:59:44.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.183+0000 7f664b7ff700 1 -- 172.21.15.33:0/285850091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 msgr2=0x7f66440a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.183+0000 7f664b7ff700 1 --2- 172.21.15.33:0/285850091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 0x7f66440a9de0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7f66440adb40 tx=0x7f66400090f0).stop 2023-11-06T21:59:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.183+0000 7f664b7ff700 1 -- 172.21.15.33:0/285850091 shutdown_connections 2023-11-06T21:59:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.183+0000 7f664b7ff700 1 --2- 172.21.15.33:0/285850091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 0x7f66440a9de0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.183+0000 7f664b7ff700 1 --2- 172.21.15.33:0/285850091 >> v2:172.21.15.69:3300/0 conn(0x7f66440aa2d0 0x7f66440c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.183+0000 7f664b7ff700 1 -- 172.21.15.33:0/285850091 >> 172.21.15.33:0/285850091 conn(0x7f664405ec80 msgr2=0x7f66440cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.183+0000 7f664b7ff700 1 -- 172.21.15.33:0/285850091 shutdown_connections 2023-11-06T21:59:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.183+0000 7f664b7ff700 1 -- 172.21.15.33:0/285850091 wait complete. 2023-11-06T21:59:44.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.185+0000 7f664b7ff700 1 Processor -- start 2023-11-06T21:59:44.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.185+0000 7f664b7ff700 1 -- start start 2023-11-06T21:59:44.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f664b7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f66440aa2d0 0x7f66440af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f664b7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 0x7f66440b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f664b7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66440c6240 0x7f66441567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f664b7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440b04e0 con 0x7f66440c1840 2023-11-06T21:59:44.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f664b7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664414f8b0 con 0x7f66440c6240 2023-11-06T21:59:44.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f664b7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66440af8d0 con 0x7f66440aa2d0 2023-11-06T21:59:44.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f6648d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 0x7f66440b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f6648d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 0x7f66440b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58850/0 (socket says 172.21.15.33:58850) 2023-11-06T21:59:44.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f6648d9a700 1 -- 172.21.15.33:0/1713425716 learned_addr learned my addr 172.21.15.33:0/1713425716 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f664959b700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f66440aa2d0 0x7f66440af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f6649d9c700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66440c6240 0x7f66441567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f6648d9a700 1 -- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f66440aa2d0 msgr2=0x7f66440af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f6648d9a700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f66440aa2d0 0x7f66440af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f6648d9a700 1 -- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66440c6240 msgr2=0x7f66441567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f6648d9a700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66440c6240 0x7f66441567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.186+0000 7f6648d9a700 1 -- 172.21.15.33:0/1713425716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6640009370 con 0x7f66440c1840 2023-11-06T21:59:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.187+0000 7f6648d9a700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 0x7f66440b0970 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f663400d860 tx=0x7f6634008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.187+0000 7f66397fa700 1 -- 172.21.15.33:0/1713425716 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66340125f0 con 0x7f66440c1840 2023-11-06T21:59:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.187+0000 7f66397fa700 1 -- 172.21.15.33:0/1713425716 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6634012750 con 0x7f66440c1840 2023-11-06T21:59:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.187+0000 7f664b7ff700 1 -- 172.21.15.33:0/1713425716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66440afbb0 con 0x7f66440c1840 2023-11-06T21:59:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.187+0000 7f66397fa700 1 -- 172.21.15.33:0/1713425716 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634012ad0 con 0x7f66440c1840 2023-11-06T21:59:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.187+0000 7f664b7ff700 1 -- 172.21.15.33:0/1713425716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66440b0170 con 0x7f66440c1840 2023-11-06T21:59:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.188+0000 7f664b7ff700 1 -- 172.21.15.33:0/1713425716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66440b0690 con 0x7f66440c1840 2023-11-06T21:59:44.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.189+0000 7f66397fa700 1 -- 172.21.15.33:0/1713425716 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6634013480 con 0x7f66440c1840 2023-11-06T21:59:44.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.190+0000 7f66397fa700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f662805f290 0x7f6628061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.190+0000 7f66397fa700 1 -- 172.21.15.33:0/1713425716 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f663407a440 con 0x7f66440c1840 2023-11-06T21:59:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.191+0000 7f664959b700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f662805f290 0x7f6628061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.191+0000 7f664959b700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f662805f290 0x7f6628061750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f6640009130 tx=0x7f6640017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.194+0000 7f66397fa700 1 -- 172.21.15.33:0/1713425716 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f66440b0690 con 0x7f66440c1840 2023-11-06T21:59:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.416+0000 7f664b7ff700 1 -- 172.21.15.33:0/1713425716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f664406c6c0 con 0x7f66440c1840 2023-11-06T21:59:44.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.416+0000 7f66397fa700 1 -- 172.21.15.33:0/1713425716 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f663404b5c0 con 0x7f66440c1840 2023-11-06T21:59:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:44.419 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3645039939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3645039939},{"type":"v1","addr":"172.21.15.33:6835","nonce":3645039939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1156785194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1156785194},{"type":"v1","addr":"172.21.15.69:6836","nonce":1156785194}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/4183165491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":4183165491},{"type":"v1","addr":"172.21.15.69:6837","nonce":4183165491}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:59:40.865889+0000","modified":"2023-11-06T21:59:43.953031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8445":{"gid":8445,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/1588551062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1588551062},{"type":"v1","addr":"172.21.15.33:6833","nonce":1588551062}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-06T21:59:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.418+0000 7f664b7ff700 1 -- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f662805f290 msgr2=0x7f6628061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.418+0000 7f664b7ff700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f662805f290 0x7f6628061750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f6640009130 tx=0x7f6640017000).stop 2023-11-06T21:59:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.418+0000 7f664b7ff700 1 -- 172.21.15.33:0/1713425716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 msgr2=0x7f66440b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.418+0000 7f664b7ff700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 0x7f66440b0970 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f663400d860 tx=0x7f6634008040).stop 2023-11-06T21:59:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.419+0000 7f664b7ff700 1 -- 172.21.15.33:0/1713425716 shutdown_connections 2023-11-06T21:59:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.419+0000 7f664b7ff700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f662805f290 0x7f6628061750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.419+0000 7f664b7ff700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f66440aa2d0 0x7f66440af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.419+0000 7f664b7ff700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66440c6240 0x7f66441567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.419+0000 7f664b7ff700 1 --2- 172.21.15.33:0/1713425716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66440c1840 0x7f66440b0970 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.419+0000 7f664b7ff700 1 -- 172.21.15.33:0/1713425716 >> 172.21.15.33:0/1713425716 conn(0x7f664405ec80 msgr2=0x7f66440ca970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:44.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.419+0000 7f664b7ff700 1 -- 172.21.15.33:0/1713425716 shutdown_connections 2023-11-06T21:59:44.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.419+0000 7f664b7ff700 1 -- 172.21.15.33:0/1713425716 wait complete. 2023-11-06T21:59:44.423 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 68 2023-11-06T21:59:44.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:59:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.633+0000 7fb11ddff700 1 Processor -- start 2023-11-06T21:59:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.633+0000 7fb11ddff700 1 -- start start 2023-11-06T21:59:44.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.634+0000 7fb11ddff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb1181521a0 0x7fb1180c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.634+0000 7fb11ddff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb118155320 0x7fb1180c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.634+0000 7fb11ddff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb118154e90 con 0x7fb118151dc0 2023-11-06T21:59:44.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.634+0000 7fb11ddff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1180639d0 con 0x7fb1181521a0 2023-11-06T21:59:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.634+0000 7fb11ddff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb118063cb0 con 0x7fb118155320 2023-11-06T21:59:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.634+0000 7fb116ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb1181521a0 0x7fb1180c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.634+0000 7fb116ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb1181521a0 0x7fb1180c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58862/0 (socket says 172.21.15.33:58862) 2023-11-06T21:59:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.634+0000 7fb116ffd700 1 -- 172.21.15.33:0/439827944 learned_addr learned my addr 172.21.15.33:0/439827944 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.634+0000 7fb1177fe700 1 --1- 172.21.15.33:0/439827944 >> v1:172.21.15.33:6789/0 conn(0x7fb118151dc0 0x7fb118063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57676/0 (socket says 172.21.15.33:57676) 2023-11-06T21:59:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb117fff700 1 --2- 172.21.15.33:0/439827944 >> v2:172.21.15.69:3300/0 conn(0x7fb118155320 0x7fb1180c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb116ffd700 1 -- 172.21.15.33:0/439827944 >> v2:172.21.15.69:3300/0 conn(0x7fb118155320 msgr2=0x7fb1180c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb116ffd700 1 --2- 172.21.15.33:0/439827944 >> v2:172.21.15.69:3300/0 conn(0x7fb118155320 0x7fb1180c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb116ffd700 1 -- 172.21.15.33:0/439827944 >> v1:172.21.15.33:6789/0 conn(0x7fb118151dc0 legacy=0x7fb118063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb116ffd700 1 -- 172.21.15.33:0/439827944 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb118063f90 con 0x7fb1181521a0 2023-11-06T21:59:44.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb1167fc700 1 -- 172.21.15.33:0/439827944 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130583381 0 0) 0x7fb118154e90 con 0x7fb118151dc0 2023-11-06T21:59:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb117fff700 1 --2- 172.21.15.33:0/439827944 >> v2:172.21.15.69:3300/0 conn(0x7fb118155320 0x7fb1180c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb116ffd700 1 --2- 172.21.15.33:0/439827944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1181521a0 0x7fb1180c0c50 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fb118154a10 tx=0x7fb10c0090f0).ready entity=mon.0 client_cookie=9268b3a966360d38 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:44.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb1167fc700 1 -- 172.21.15.33:0/439827944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c00b600 con 0x7fb1181521a0 2023-11-06T21:59:44.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb1167fc700 1 -- 172.21.15.33:0/439827944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c00b760 con 0x7fb1181521a0 2023-11-06T21:59:44.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.635+0000 7fb1167fc700 1 -- 172.21.15.33:0/439827944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c00bae0 con 0x7fb1181521a0 2023-11-06T21:59:44.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.636+0000 7fb11ddff700 1 -- 172.21.15.33:0/439827944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1181521a0 msgr2=0x7fb1180c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.636+0000 7fb11ddff700 1 --2- 172.21.15.33:0/439827944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1181521a0 0x7fb1180c0c50 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fb118154a10 tx=0x7fb10c0090f0).stop 2023-11-06T21:59:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.636+0000 7fb11ddff700 1 -- 172.21.15.33:0/439827944 shutdown_connections 2023-11-06T21:59:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.636+0000 7fb11ddff700 1 --2- 172.21.15.33:0/439827944 >> v2:172.21.15.69:3300/0 conn(0x7fb118155320 0x7fb1180c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.636+0000 7fb11ddff700 1 --2- 172.21.15.33:0/439827944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1181521a0 0x7fb1180c0c50 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.636+0000 7fb11ddff700 1 -- 172.21.15.33:0/439827944 >> 172.21.15.33:0/439827944 conn(0x7fb11805e1b0 msgr2=0x7fb11805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.636+0000 7fb11ddff700 1 -- 172.21.15.33:0/439827944 shutdown_connections 2023-11-06T21:59:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.636+0000 7fb11ddff700 1 -- 172.21.15.33:0/439827944 wait complete. 2023-11-06T21:59:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.638+0000 7fb11ddff700 1 Processor -- start 2023-11-06T21:59:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.638+0000 7fb11ddff700 1 -- start start 2023-11-06T21:59:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb11ddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb118151dc0 0x7fb1180addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb11ddff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1181521a0 0x7fb118150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb11ddff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb118155320 0x7fb1180ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb11ddff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180af2c0 con 0x7fb118155320 2023-11-06T21:59:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb11ddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180ae6b0 con 0x7fb118151dc0 2023-11-06T21:59:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb11ddff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180ae990 con 0x7fb1181521a0 2023-11-06T21:59:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb117fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb118155320 0x7fb1180ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb117fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb118155320 0x7fb1180ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58868/0 (socket says 172.21.15.33:58868) 2023-11-06T21:59:44.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb117fff700 1 -- 172.21.15.33:0/3775328916 learned_addr learned my addr 172.21.15.33:0/3775328916 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:44.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb116ffd700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1181521a0 0x7fb118150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb1177fe700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb118151dc0 0x7fb1180addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb117fff700 1 -- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1181521a0 msgr2=0x7fb118150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.639+0000 7fb117fff700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1181521a0 0x7fb118150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.640+0000 7fb117fff700 1 -- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb118151dc0 msgr2=0x7fb1180addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.640+0000 7fb117fff700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb118151dc0 0x7fb1180addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.640+0000 7fb117fff700 1 -- 172.21.15.33:0/3775328916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c009370 con 0x7fb118155320 2023-11-06T21:59:44.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.640+0000 7fb117fff700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb118155320 0x7fb1180ac470 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7fb110009fc0 tx=0x7fb1100099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:44.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.640+0000 7fb0ff7fe700 1 -- 172.21.15.33:0/3775328916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11000dbe0 con 0x7fb118155320 2023-11-06T21:59:44.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.640+0000 7fb0ff7fe700 1 -- 172.21.15.33:0/3775328916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11000f040 con 0x7fb118155320 2023-11-06T21:59:44.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.640+0000 7fb11ddff700 1 -- 172.21.15.33:0/3775328916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1180aec70 con 0x7fb118155320 2023-11-06T21:59:44.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.640+0000 7fb0ff7fe700 1 -- 172.21.15.33:0/3775328916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100133f0 con 0x7fb118155320 2023-11-06T21:59:44.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.640+0000 7fb11ddff700 1 -- 172.21.15.33:0/3775328916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1180af750 con 0x7fb118155320 2023-11-06T21:59:44.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.641+0000 7fb11ddff700 1 -- 172.21.15.33:0/3775328916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1180c0a30 con 0x7fb118155320 2023-11-06T21:59:44.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.642+0000 7fb0ff7fe700 1 -- 172.21.15.33:0/3775328916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb1100137e0 con 0x7fb118155320 2023-11-06T21:59:44.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.643+0000 7fb0ff7fe700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb10405f270 0x7fb104061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:44.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.643+0000 7fb0ff7fe700 1 -- 172.21.15.33:0/3775328916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fb11007aaf0 con 0x7fb118155320 2023-11-06T21:59:44.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.643+0000 7fb1177fe700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb10405f270 0x7fb104061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:44.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.644+0000 7fb1177fe700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb10405f270 0x7fb104061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb1181678b0 tx=0x7fb108007bd0).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:44.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.647+0000 7fb0ff7fe700 1 -- 172.21.15.33:0/3775328916 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb11004bb50 con 0x7fb118155320 2023-11-06T21:59:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.828+0000 7fb11ddff700 1 -- 172.21.15.33:0/3775328916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb11806ab40 con 0x7fb118155320 2023-11-06T21:59:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.829+0000 7fb0ff7fe700 1 -- 172.21.15.33:0/3775328916 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+17061 (secure 0 0 0) 0x7fb11004f170 con 0x7fb118155320 2023-11-06T21:59:44.832 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:44.832 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":170,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:59:41.868528+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:59:38.416213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T21:59:39.859883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:59:44.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.831+0000 7fb11ddff700 1 -- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb10405f270 msgr2=0x7fb104061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.831+0000 7fb11ddff700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb10405f270 0x7fb104061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb1181678b0 tx=0x7fb108007bd0).stop 2023-11-06T21:59:44.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.831+0000 7fb11ddff700 1 -- 172.21.15.33:0/3775328916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb118155320 msgr2=0x7fb1180ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:44.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.831+0000 7fb11ddff700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb118155320 0x7fb1180ac470 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7fb110009fc0 tx=0x7fb1100099b0).stop 2023-11-06T21:59:44.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.832+0000 7fb11ddff700 1 -- 172.21.15.33:0/3775328916 shutdown_connections 2023-11-06T21:59:44.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.832+0000 7fb11ddff700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb10405f270 0x7fb104061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.832+0000 7fb11ddff700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb118155320 0x7fb1180ac470 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.832+0000 7fb11ddff700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb118151dc0 0x7fb1180addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.832+0000 7fb11ddff700 1 --2- 172.21.15.33:0/3775328916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1181521a0 0x7fb118150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.832+0000 7fb11ddff700 1 -- 172.21.15.33:0/3775328916 >> 172.21.15.33:0/3775328916 conn(0x7fb11805e1b0 msgr2=0x7fb118152640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.832+0000 7fb11ddff700 1 -- 172.21.15.33:0/3775328916 shutdown_connections 2023-11-06T21:59:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:44.832+0000 7fb11ddff700 1 -- 172.21.15.33:0/3775328916 wait complete. 2023-11-06T21:59:44.852 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T21:59:44.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T21:59:45.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.047+0000 7fd5f53ff700 1 Processor -- start 2023-11-06T21:59:45.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.048+0000 7fd5f53ff700 1 -- start start 2023-11-06T21:59:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.049+0000 7fd5f53ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5f00c64c0 0x7fd5f01544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.049+0000 7fd5f53ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f00c0880 con 0x7fd5f0154110 2023-11-06T21:59:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.049+0000 7fd5f53ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f00c8260 con 0x7fd5f0064ce0 2023-11-06T21:59:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.049+0000 7fd5f53ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f00bfd40 con 0x7fd5f00c64c0 2023-11-06T21:59:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.049+0000 7fd5eeffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd5f0154110 0x7fd5f00645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57692/0 (socket says 172.21.15.33:57692) 2023-11-06T21:59:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.049+0000 7fd5eeffd700 1 -- 172.21.15.33:0/3604569448 learned_addr learned my addr 172.21.15.33:0/3604569448 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.049+0000 7fd5ef7fe700 1 --2- 172.21.15.33:0/3604569448 >> v2:172.21.15.33:3300/0 conn(0x7fd5f00c64c0 0x7fd5f01544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.050+0000 7fd5edffb700 1 -- 172.21.15.33:0/3604569448 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991797424 0 0) 0x7fd5f00c0880 con 0x7fd5f0154110 2023-11-06T21:59:45.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.050+0000 7fd5edffb700 1 -- 172.21.15.33:0/3604569448 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d80033f0 con 0x7fd5f0154110 2023-11-06T21:59:45.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.050+0000 7fd5ef7fe700 1 -- 172.21.15.33:0/3604569448 >> v1:172.21.15.69:6789/0 conn(0x7fd5f0064ce0 legacy=0x7fd5f00c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.050+0000 7fd5ef7fe700 1 -- 172.21.15.33:0/3604569448 >> v1:172.21.15.33:6789/0 conn(0x7fd5f0154110 legacy=0x7fd5f00645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.050+0000 7fd5ef7fe700 1 -- 172.21.15.33:0/3604569448 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f00bff50 con 0x7fd5f00c64c0 2023-11-06T21:59:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.050+0000 7fd5ef7fe700 1 --2- 172.21.15.33:0/3604569448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 0x7fd5f01544f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7fd5e8009e60 tx=0x7fd5e8009780).ready entity=mon.0 client_cookie=d1ced08cfa83d775 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.050+0000 7fd5edffb700 1 -- 172.21.15.33:0/3604569448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800e070 con 0x7fd5f00c64c0 2023-11-06T21:59:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.050+0000 7fd5edffb700 1 -- 172.21.15.33:0/3604569448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e800bb60 con 0x7fd5f00c64c0 2023-11-06T21:59:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.050+0000 7fd5edffb700 1 -- 172.21.15.33:0/3604569448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8012410 con 0x7fd5f00c64c0 2023-11-06T21:59:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.051+0000 7fd5f53ff700 1 -- 172.21.15.33:0/3604569448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 msgr2=0x7fd5f01544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.051+0000 7fd5f53ff700 1 --2- 172.21.15.33:0/3604569448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 0x7fd5f01544f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7fd5e8009e60 tx=0x7fd5e8009780).stop 2023-11-06T21:59:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.051+0000 7fd5f53ff700 1 -- 172.21.15.33:0/3604569448 shutdown_connections 2023-11-06T21:59:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.051+0000 7fd5f53ff700 1 --2- 172.21.15.33:0/3604569448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 0x7fd5f01544f0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.051+0000 7fd5f53ff700 1 -- 172.21.15.33:0/3604569448 >> 172.21.15.33:0/3604569448 conn(0x7fd5f00cb8b0 msgr2=0x7fd5f00c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:45.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.051+0000 7fd5f53ff700 1 -- 172.21.15.33:0/3604569448 shutdown_connections 2023-11-06T21:59:45.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.051+0000 7fd5f53ff700 1 -- 172.21.15.33:0/3604569448 wait complete. 2023-11-06T21:59:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.053+0000 7fd5f53ff700 1 Processor -- start 2023-11-06T21:59:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.053+0000 7fd5f53ff700 1 -- start start 2023-11-06T21:59:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.054+0000 7fd5f53ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5f0064ce0 0x7fd5f01642f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.054+0000 7fd5f53ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 0x7fd5f0165500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.054+0000 7fd5f53ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5f0154110 0x7fd5f0161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.054+0000 7fd5f53ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f01659f0 con 0x7fd5f00c64c0 2023-11-06T21:59:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.054+0000 7fd5f53ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f0166810 con 0x7fd5f0154110 2023-11-06T21:59:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5f53ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00a3260 con 0x7fd5f0064ce0 2023-11-06T21:59:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5ee7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 0x7fd5f0165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5ee7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 0x7fd5f0165500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58888/0 (socket says 172.21.15.33:58888) 2023-11-06T21:59:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5ee7fc700 1 -- 172.21.15.33:0/135611883 learned_addr learned my addr 172.21.15.33:0/135611883 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5eeffd700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5f0064ce0 0x7fd5f01642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5ef7fe700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5f0154110 0x7fd5f0161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5ee7fc700 1 -- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5f0064ce0 msgr2=0x7fd5f01642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5ee7fc700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5f0064ce0 0x7fd5f01642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5ee7fc700 1 -- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5f0154110 msgr2=0x7fd5f0161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5ee7fc700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5f0154110 0x7fd5f0161fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.055+0000 7fd5ee7fc700 1 -- 172.21.15.33:0/135611883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e8009a20 con 0x7fd5f00c64c0 2023-11-06T21:59:45.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.056+0000 7fd5ee7fc700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 0x7fd5f0165500 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fd5e400c4c0 tx=0x7fd5e4007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.056+0000 7fd5d6ffd700 1 -- 172.21.15.33:0/135611883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400ce10 con 0x7fd5f00c64c0 2023-11-06T21:59:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.056+0000 7fd5d6ffd700 1 -- 172.21.15.33:0/135611883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e4007020 con 0x7fd5f00c64c0 2023-11-06T21:59:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.056+0000 7fd5f53ff700 1 -- 172.21.15.33:0/135611883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f0165c00 con 0x7fd5f00c64c0 2023-11-06T21:59:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.056+0000 7fd5d6ffd700 1 -- 172.21.15.33:0/135611883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40073c0 con 0x7fd5f00c64c0 2023-11-06T21:59:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.056+0000 7fd5f53ff700 1 -- 172.21.15.33:0/135611883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f01661c0 con 0x7fd5f00c64c0 2023-11-06T21:59:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.057+0000 7fd5f53ff700 1 -- 172.21.15.33:0/135611883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f01669a0 con 0x7fd5f00c64c0 2023-11-06T21:59:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.058+0000 7fd5d6ffd700 1 -- 172.21.15.33:0/135611883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd5e40068b0 con 0x7fd5f00c64c0 2023-11-06T21:59:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.058+0000 7fd5d6ffd700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5dc05f240 0x7fd5dc061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.059+0000 7fd5d6ffd700 1 -- 172.21.15.33:0/135611883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fd5e40797c0 con 0x7fd5f00c64c0 2023-11-06T21:59:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.059+0000 7fd5eeffd700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5dc05f240 0x7fd5dc061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.059+0000 7fd5eeffd700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5dc05f240 0x7fd5dc061700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd5e0007fd0 tx=0x7fd5e0009040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.062+0000 7fd5d6ffd700 1 -- 172.21.15.33:0/135611883 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd5e4047af0 con 0x7fd5f00c64c0 2023-11-06T21:59:45.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.238+0000 7fd5f53ff700 1 -- 172.21.15.33:0/135611883 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5f006c6c0 con 0x7fd5dc05f240 2023-11-06T21:59:45.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.252+0000 7fd5d6ffd700 1 -- 172.21.15.33:0/135611883 <== mgr.7025 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361460 (secure 0 0 0) 0x7fd5f006c6c0 con 0x7fd5dc05f240 2023-11-06T21:59:45.259 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:45.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 -- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5dc05f240 msgr2=0x7fd5dc061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5dc05f240 0x7fd5dc061700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd5e0007fd0 tx=0x7fd5e0009040).stop 2023-11-06T21:59:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 -- 172.21.15.33:0/135611883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 msgr2=0x7fd5f0165500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 0x7fd5f0165500 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fd5e400c4c0 tx=0x7fd5e4007dd0).stop 2023-11-06T21:59:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 -- 172.21.15.33:0/135611883 shutdown_connections 2023-11-06T21:59:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd5dc05f240 0x7fd5dc061700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd5f0064ce0 0x7fd5f01642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5f0154110 0x7fd5f0161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 --2- 172.21.15.33:0/135611883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5f00c64c0 0x7fd5f0165500 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 -- 172.21.15.33:0/135611883 >> 172.21.15.33:0/135611883 conn(0x7fd5f00cb8b0 msgr2=0x7fd5f00afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:45.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 -- 172.21.15.33:0/135611883 shutdown_connections 2023-11-06T21:59:45.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.257+0000 7fd5f53ff700 1 -- 172.21.15.33:0/135611883 wait complete. 2023-11-06T21:59:45.264 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T21:59:45.277 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T21:59:44.331281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2442412,"kb_used_data":16492,"kb_used_omap":0,"kb_used_meta":2425856,"kb_avail":752532308,"statfs":{"total":773094113280,"available":770593083392,"internally_reserved":0,"allocated":16887808,"data_stored":8596208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2484076544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5037059,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-484,"num_write_kb":-5308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.995725"},"pg_stats":[{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907321+0000","last_change":"2023-11-06T21:59:40.898557+0000","last_active":"2023-11-06T21:59:40.907321+0000","last_peered":"2023-11-06T21:59:40.907321+0000","last_clean":"2023-11-06T21:59:40.907321+0000","last_became_active":"2023-11-06T21:59:40.898071+0000","last_became_peered":"2023-11-06T21:59:40.898071+0000","last_unstale":"2023-11-06T21:59:40.907321+0000","last_undegraded":"2023-11-06T21:59:40.907321+0000","last_fullsized":"2023-11-06T21:59:40.907321+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3f","version":"169'3","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.909120+0000","last_change":"2023-11-06T21:59:40.408459+0000","last_active":"2023-11-06T21:59:40.909120+0000","last_peered":"2023-11-06T21:59:40.909120+0000","last_clean":"2023-11-06T21:59:40.909120+0000","last_became_active":"2023-11-06T21:59:40.408127+0000","last_became_peered":"2023-11-06T21:59:40.408127+0000","last_unstale":"2023-11-06T21:59:40.909120+0000","last_undegraded":"2023-11-06T21:59:40.909120+0000","last_fullsized":"2023-11-06T21:59:40.909120+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.900961+0000","last_change":"2023-11-06T21:59:40.888973+0000","last_active":"2023-11-06T21:59:40.900961+0000","last_peered":"2023-11-06T21:59:40.900961+0000","last_clean":"2023-11-06T21:59:40.900961+0000","last_became_active":"2023-11-06T21:59:40.888001+0000","last_became_peered":"2023-11-06T21:59:40.888001+0000","last_unstale":"2023-11-06T21:59:40.900961+0000","last_undegraded":"2023-11-06T21:59:40.900961+0000","last_fullsized":"2023-11-06T21:59:40.900961+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882424+0000","last_change":"2023-11-06T21:59:40.413032+0000","last_active":"2023-11-06T21:59:40.882424+0000","last_peered":"2023-11-06T21:59:40.882424+0000","last_clean":"2023-11-06T21:59:40.882424+0000","last_became_active":"2023-11-06T21:59:40.412676+0000","last_became_peered":"2023-11-06T21:59:40.412676+0000","last_unstale":"2023-11-06T21:59:40.882424+0000","last_undegraded":"2023-11-06T21:59:40.882424+0000","last_fullsized":"2023-11-06T21:59:40.882424+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896380+0000","last_change":"2023-11-06T21:59:40.892101+0000","last_active":"2023-11-06T21:59:40.896380+0000","last_peered":"2023-11-06T21:59:40.896380+0000","last_clean":"2023-11-06T21:59:40.896380+0000","last_became_active":"2023-11-06T21:59:40.891699+0000","last_became_peered":"2023-11-06T21:59:40.891699+0000","last_unstale":"2023-11-06T21:59:40.896380+0000","last_undegraded":"2023-11-06T21:59:40.896380+0000","last_fullsized":"2023-11-06T21:59:40.896380+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.881559+0000","last_change":"2023-11-06T21:59:40.401569+0000","last_active":"2023-11-06T21:59:40.881559+0000","last_peered":"2023-11-06T21:59:40.881559+0000","last_clean":"2023-11-06T21:59:40.881559+0000","last_became_active":"2023-11-06T21:59:40.401019+0000","last_became_peered":"2023-11-06T21:59:40.401019+0000","last_unstale":"2023-11-06T21:59:40.881559+0000","last_undegraded":"2023-11-06T21:59:40.881559+0000","last_fullsized":"2023-11-06T21:59:40.881559+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896352+0000","last_change":"2023-11-06T21:59:40.891887+0000","last_active":"2023-11-06T21:59:40.896352+0000","last_peered":"2023-11-06T21:59:40.896352+0000","last_clean":"2023-11-06T21:59:40.896352+0000","last_became_active":"2023-11-06T21:59:40.888875+0000","last_became_peered":"2023-11-06T21:59:40.888875+0000","last_unstale":"2023-11-06T21:59:40.896352+0000","last_undegraded":"2023-11-06T21:59:40.896352+0000","last_fullsized":"2023-11-06T21:59:40.896352+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908353+0000","last_change":"2023-11-06T21:59:40.401201+0000","last_active":"2023-11-06T21:59:40.908353+0000","last_peered":"2023-11-06T21:59:40.908353+0000","last_clean":"2023-11-06T21:59:40.908353+0000","last_became_active":"2023-11-06T21:59:40.399541+0000","last_became_peered":"2023-11-06T21:59:40.399541+0000","last_unstale":"2023-11-06T21:59:40.908353+0000","last_undegraded":"2023-11-06T21:59:40.908353+0000","last_fullsized":"2023-11-06T21:59:40.908353+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908754+0000","last_change":"2023-11-06T21:59:40.903428+0000","last_active":"2023-11-06T21:59:40.908754+0000","last_peered":"2023-11-06T21:59:40.908754+0000","last_clean":"2023-11-06T21:59:40.908754+0000","last_became_active":"2023-11-06T21:59:40.903147+0000","last_became_peered":"2023-11-06T21:59:40.903147+0000","last_unstale":"2023-11-06T21:59:40.908754+0000","last_undegraded":"2023-11-06T21:59:40.908754+0000","last_fullsized":"2023-11-06T21:59:40.908754+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3b","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908333+0000","last_change":"2023-11-06T21:59:40.415502+0000","last_active":"2023-11-06T21:59:40.908333+0000","last_peered":"2023-11-06T21:59:40.908333+0000","last_clean":"2023-11-06T21:59:40.908333+0000","last_became_active":"2023-11-06T21:59:40.414166+0000","last_became_peered":"2023-11-06T21:59:40.414166+0000","last_unstale":"2023-11-06T21:59:40.908333+0000","last_undegraded":"2023-11-06T21:59:40.908333+0000","last_fullsized":"2023-11-06T21:59:40.908333+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908801+0000","last_change":"2023-11-06T21:59:40.903712+0000","last_active":"2023-11-06T21:59:40.908801+0000","last_peered":"2023-11-06T21:59:40.908801+0000","last_clean":"2023-11-06T21:59:40.908801+0000","last_became_active":"2023-11-06T21:59:40.902816+0000","last_became_peered":"2023-11-06T21:59:40.902816+0000","last_unstale":"2023-11-06T21:59:40.908801+0000","last_undegraded":"2023-11-06T21:59:40.908801+0000","last_fullsized":"2023-11-06T21:59:40.908801+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3a","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908756+0000","last_change":"2023-11-06T21:59:40.412482+0000","last_active":"2023-11-06T21:59:40.908756+0000","last_peered":"2023-11-06T21:59:40.908756+0000","last_clean":"2023-11-06T21:59:40.908756+0000","last_became_active":"2023-11-06T21:59:40.412144+0000","last_became_peered":"2023-11-06T21:59:40.412144+0000","last_unstale":"2023-11-06T21:59:40.908756+0000","last_undegraded":"2023-11-06T21:59:40.908756+0000","last_fullsized":"2023-11-06T21:59:40.908756+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.909140+0000","last_change":"2023-11-06T21:59:40.907038+0000","last_active":"2023-11-06T21:59:40.909140+0000","last_peered":"2023-11-06T21:59:40.909140+0000","last_clean":"2023-11-06T21:59:40.909140+0000","last_became_active":"2023-11-06T21:59:40.905589+0000","last_became_peered":"2023-11-06T21:59:40.905589+0000","last_unstale":"2023-11-06T21:59:40.909140+0000","last_undegraded":"2023-11-06T21:59:40.909140+0000","last_fullsized":"2023-11-06T21:59:40.909140+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879414+0000","last_change":"2023-11-06T21:59:40.404442+0000","last_active":"2023-11-06T21:59:40.879414+0000","last_peered":"2023-11-06T21:59:40.879414+0000","last_clean":"2023-11-06T21:59:40.879414+0000","last_became_active":"2023-11-06T21:59:40.404124+0000","last_became_peered":"2023-11-06T21:59:40.404124+0000","last_unstale":"2023-11-06T21:59:40.879414+0000","last_undegraded":"2023-11-06T21:59:40.879414+0000","last_fullsized":"2023-11-06T21:59:40.879414+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.895577+0000","last_change":"2023-11-06T21:59:40.889505+0000","last_active":"2023-11-06T21:59:40.895577+0000","last_peered":"2023-11-06T21:59:40.895577+0000","last_clean":"2023-11-06T21:59:40.895577+0000","last_became_active":"2023-11-06T21:59:40.889183+0000","last_became_peered":"2023-11-06T21:59:40.889183+0000","last_unstale":"2023-11-06T21:59:40.895577+0000","last_undegraded":"2023-11-06T21:59:40.895577+0000","last_fullsized":"2023-11-06T21:59:40.895577+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879975+0000","last_change":"2023-11-06T21:59:40.414173+0000","last_active":"2023-11-06T21:59:40.879975+0000","last_peered":"2023-11-06T21:59:40.879975+0000","last_clean":"2023-11-06T21:59:40.879975+0000","last_became_active":"2023-11-06T21:59:40.412912+0000","last_became_peered":"2023-11-06T21:59:40.412912+0000","last_unstale":"2023-11-06T21:59:40.879975+0000","last_undegraded":"2023-11-06T21:59:40.879975+0000","last_fullsized":"2023-11-06T21:59:40.879975+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.895532+0000","last_change":"2023-11-06T21:59:40.890003+0000","last_active":"2023-11-06T21:59:40.895532+0000","last_peered":"2023-11-06T21:59:40.895532+0000","last_clean":"2023-11-06T21:59:40.895532+0000","last_became_active":"2023-11-06T21:59:40.889447+0000","last_became_peered":"2023-11-06T21:59:40.889447+0000","last_unstale":"2023-11-06T21:59:40.895532+0000","last_undegraded":"2023-11-06T21:59:40.895532+0000","last_fullsized":"2023-11-06T21:59:40.895532+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.884076+0000","last_change":"2023-11-06T21:59:40.397578+0000","last_active":"2023-11-06T21:59:40.884076+0000","last_peered":"2023-11-06T21:59:40.884076+0000","last_clean":"2023-11-06T21:59:40.884076+0000","last_became_active":"2023-11-06T21:59:40.396898+0000","last_became_peered":"2023-11-06T21:59:40.396898+0000","last_unstale":"2023-11-06T21:59:40.884076+0000","last_undegraded":"2023-11-06T21:59:40.884076+0000","last_fullsized":"2023-11-06T21:59:40.884076+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896956+0000","last_change":"2023-11-06T21:59:40.892121+0000","last_active":"2023-11-06T21:59:40.896956+0000","last_peered":"2023-11-06T21:59:40.896956+0000","last_clean":"2023-11-06T21:59:40.896956+0000","last_became_active":"2023-11-06T21:59:40.891796+0000","last_became_peered":"2023-11-06T21:59:40.891796+0000","last_unstale":"2023-11-06T21:59:40.896956+0000","last_undegraded":"2023-11-06T21:59:40.896956+0000","last_fullsized":"2023-11-06T21:59:40.896956+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882577+0000","last_change":"2023-11-06T21:59:40.404012+0000","last_active":"2023-11-06T21:59:40.882577+0000","last_peered":"2023-11-06T21:59:40.882577+0000","last_clean":"2023-11-06T21:59:40.882577+0000","last_became_active":"2023-11-06T21:59:40.403699+0000","last_became_peered":"2023-11-06T21:59:40.403699+0000","last_unstale":"2023-11-06T21:59:40.882577+0000","last_undegraded":"2023-11-06T21:59:40.882577+0000","last_fullsized":"2023-11-06T21:59:40.882577+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896781+0000","last_change":"2023-11-06T21:59:40.891121+0000","last_active":"2023-11-06T21:59:40.896781+0000","last_peered":"2023-11-06T21:59:40.896781+0000","last_clean":"2023-11-06T21:59:40.896781+0000","last_became_active":"2023-11-06T21:59:40.890789+0000","last_became_peered":"2023-11-06T21:59:40.890789+0000","last_unstale":"2023-11-06T21:59:40.896781+0000","last_undegraded":"2023-11-06T21:59:40.896781+0000","last_fullsized":"2023-11-06T21:59:40.896781+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.883160+0000","last_change":"2023-11-06T21:59:40.404021+0000","last_active":"2023-11-06T21:59:40.883160+0000","last_peered":"2023-11-06T21:59:40.883160+0000","last_clean":"2023-11-06T21:59:40.883160+0000","last_became_active":"2023-11-06T21:59:40.403725+0000","last_became_peered":"2023-11-06T21:59:40.403725+0000","last_unstale":"2023-11-06T21:59:40.883160+0000","last_undegraded":"2023-11-06T21:59:40.883160+0000","last_fullsized":"2023-11-06T21:59:40.883160+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902111+0000","last_change":"2023-11-06T21:59:40.898421+0000","last_active":"2023-11-06T21:59:40.902111+0000","last_peered":"2023-11-06T21:59:40.902111+0000","last_clean":"2023-11-06T21:59:40.902111+0000","last_became_active":"2023-11-06T21:59:40.898098+0000","last_became_peered":"2023-11-06T21:59:40.898098+0000","last_unstale":"2023-11-06T21:59:40.902111+0000","last_undegraded":"2023-11-06T21:59:40.902111+0000","last_fullsized":"2023-11-06T21:59:40.902111+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.34","version":"169'4","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.912571+0000","last_change":"2023-11-06T21:59:40.411096+0000","last_active":"2023-11-06T21:59:40.912571+0000","last_peered":"2023-11-06T21:59:40.912571+0000","last_clean":"2023-11-06T21:59:40.912571+0000","last_became_active":"2023-11-06T21:59:40.410693+0000","last_became_peered":"2023-11-06T21:59:40.410693+0000","last_unstale":"2023-11-06T21:59:40.912571+0000","last_undegraded":"2023-11-06T21:59:40.912571+0000","last_fullsized":"2023-11-06T21:59:40.912571+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902492+0000","last_change":"2023-11-06T21:59:40.898318+0000","last_active":"2023-11-06T21:59:40.902492+0000","last_peered":"2023-11-06T21:59:40.902492+0000","last_clean":"2023-11-06T21:59:40.902492+0000","last_became_active":"2023-11-06T21:59:40.897995+0000","last_became_peered":"2023-11-06T21:59:40.897995+0000","last_unstale":"2023-11-06T21:59:40.902492+0000","last_undegraded":"2023-11-06T21:59:40.902492+0000","last_fullsized":"2023-11-06T21:59:40.902492+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.33","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.898602+0000","last_change":"2023-11-06T21:59:40.404856+0000","last_active":"2023-11-06T21:59:40.898602+0000","last_peered":"2023-11-06T21:59:40.898602+0000","last_clean":"2023-11-06T21:59:40.898602+0000","last_became_active":"2023-11-06T21:59:40.404521+0000","last_became_peered":"2023-11-06T21:59:40.404521+0000","last_unstale":"2023-11-06T21:59:40.898602+0000","last_undegraded":"2023-11-06T21:59:40.898602+0000","last_fullsized":"2023-11-06T21:59:40.898602+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897174+0000","last_change":"2023-11-06T21:59:40.888919+0000","last_active":"2023-11-06T21:59:40.897174+0000","last_peered":"2023-11-06T21:59:40.897174+0000","last_clean":"2023-11-06T21:59:40.897174+0000","last_became_active":"2023-11-06T21:59:40.888596+0000","last_became_peered":"2023-11-06T21:59:40.888596+0000","last_unstale":"2023-11-06T21:59:40.897174+0000","last_undegraded":"2023-11-06T21:59:40.897174+0000","last_fullsized":"2023-11-06T21:59:40.897174+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.873107+0000","last_change":"2023-11-06T21:59:40.401830+0000","last_active":"2023-11-06T21:59:40.873107+0000","last_peered":"2023-11-06T21:59:40.873107+0000","last_clean":"2023-11-06T21:59:40.873107+0000","last_became_active":"2023-11-06T21:59:40.400869+0000","last_became_peered":"2023-11-06T21:59:40.400869+0000","last_unstale":"2023-11-06T21:59:40.873107+0000","last_undegraded":"2023-11-06T21:59:40.873107+0000","last_fullsized":"2023-11-06T21:59:40.873107+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.891377+0000","last_change":"2023-11-06T21:59:40.886735+0000","last_active":"2023-11-06T21:59:40.891377+0000","last_peered":"2023-11-06T21:59:40.891377+0000","last_clean":"2023-11-06T21:59:40.891377+0000","last_became_active":"2023-11-06T21:59:40.884809+0000","last_became_peered":"2023-11-06T21:59:40.884809+0000","last_unstale":"2023-11-06T21:59:40.891377+0000","last_undegraded":"2023-11-06T21:59:40.891377+0000","last_fullsized":"2023-11-06T21:59:40.891377+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.874342+0000","last_change":"2023-11-06T21:59:40.407023+0000","last_active":"2023-11-06T21:59:40.874342+0000","last_peered":"2023-11-06T21:59:40.874342+0000","last_clean":"2023-11-06T21:59:40.874342+0000","last_became_active":"2023-11-06T21:59:40.406689+0000","last_became_peered":"2023-11-06T21:59:40.406689+0000","last_unstale":"2023-11-06T21:59:40.874342+0000","last_undegraded":"2023-11-06T21:59:40.874342+0000","last_fullsized":"2023-11-06T21:59:40.874342+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.889693+0000","last_change":"2023-11-06T21:59:40.886273+0000","last_active":"2023-11-06T21:59:40.889693+0000","last_peered":"2023-11-06T21:59:40.889693+0000","last_clean":"2023-11-06T21:59:40.889693+0000","last_became_active":"2023-11-06T21:59:40.885614+0000","last_became_peered":"2023-11-06T21:59:40.885614+0000","last_unstale":"2023-11-06T21:59:40.889693+0000","last_undegraded":"2023-11-06T21:59:40.889693+0000","last_fullsized":"2023-11-06T21:59:40.889693+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.30","version":"169'4","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.906411+0000","last_change":"2023-11-06T21:59:40.397662+0000","last_active":"2023-11-06T21:59:40.906411+0000","last_peered":"2023-11-06T21:59:40.906411+0000","last_clean":"2023-11-06T21:59:40.906411+0000","last_became_active":"2023-11-06T21:59:40.396926+0000","last_became_peered":"2023-11-06T21:59:40.396926+0000","last_unstale":"2023-11-06T21:59:40.906411+0000","last_undegraded":"2023-11-06T21:59:40.906411+0000","last_fullsized":"2023-11-06T21:59:40.906411+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.893388+0000","last_change":"2023-11-06T21:59:40.885581+0000","last_active":"2023-11-06T21:59:40.893388+0000","last_peered":"2023-11-06T21:59:40.893388+0000","last_clean":"2023-11-06T21:59:40.893388+0000","last_became_active":"2023-11-06T21:59:40.885279+0000","last_became_peered":"2023-11-06T21:59:40.885279+0000","last_unstale":"2023-11-06T21:59:40.893388+0000","last_undegraded":"2023-11-06T21:59:40.893388+0000","last_fullsized":"2023-11-06T21:59:40.893388+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.883892+0000","last_change":"2023-11-06T21:59:40.397928+0000","last_active":"2023-11-06T21:59:40.883892+0000","last_peered":"2023-11-06T21:59:40.883892+0000","last_clean":"2023-11-06T21:59:40.883892+0000","last_became_active":"2023-11-06T21:59:40.397632+0000","last_became_peered":"2023-11-06T21:59:40.397632+0000","last_unstale":"2023-11-06T21:59:40.883892+0000","last_undegraded":"2023-11-06T21:59:40.883892+0000","last_fullsized":"2023-11-06T21:59:40.883892+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899478+0000","last_change":"2023-11-06T21:59:40.893670+0000","last_active":"2023-11-06T21:59:40.899478+0000","last_peered":"2023-11-06T21:59:40.899478+0000","last_clean":"2023-11-06T21:59:40.899478+0000","last_became_active":"2023-11-06T21:59:40.893401+0000","last_became_peered":"2023-11-06T21:59:40.893401+0000","last_unstale":"2023-11-06T21:59:40.899478+0000","last_undegraded":"2023-11-06T21:59:40.899478+0000","last_fullsized":"2023-11-06T21:59:40.899478+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.876048+0000","last_change":"2023-11-06T21:59:40.396309+0000","last_active":"2023-11-06T21:59:40.876048+0000","last_peered":"2023-11-06T21:59:40.876048+0000","last_clean":"2023-11-06T21:59:40.876048+0000","last_became_active":"2023-11-06T21:59:40.395455+0000","last_became_peered":"2023-11-06T21:59:40.395455+0000","last_unstale":"2023-11-06T21:59:40.876048+0000","last_undegraded":"2023-11-06T21:59:40.876048+0000","last_fullsized":"2023-11-06T21:59:40.876048+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896852+0000","last_change":"2023-11-06T21:59:40.891043+0000","last_active":"2023-11-06T21:59:40.896852+0000","last_peered":"2023-11-06T21:59:40.896852+0000","last_clean":"2023-11-06T21:59:40.896852+0000","last_became_active":"2023-11-06T21:59:40.890751+0000","last_became_peered":"2023-11-06T21:59:40.890751+0000","last_unstale":"2023-11-06T21:59:40.896852+0000","last_undegraded":"2023-11-06T21:59:40.896852+0000","last_fullsized":"2023-11-06T21:59:40.896852+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897894+0000","last_change":"2023-11-06T21:59:40.404941+0000","last_active":"2023-11-06T21:59:40.897894+0000","last_peered":"2023-11-06T21:59:40.897894+0000","last_clean":"2023-11-06T21:59:40.897894+0000","last_became_active":"2023-11-06T21:59:40.404615+0000","last_became_peered":"2023-11-06T21:59:40.404615+0000","last_unstale":"2023-11-06T21:59:40.897894+0000","last_undegraded":"2023-11-06T21:59:40.897894+0000","last_fullsized":"2023-11-06T21:59:40.897894+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908775+0000","last_change":"2023-11-06T21:59:40.902954+0000","last_active":"2023-11-06T21:59:40.908775+0000","last_peered":"2023-11-06T21:59:40.908775+0000","last_clean":"2023-11-06T21:59:40.908775+0000","last_became_active":"2023-11-06T21:59:40.902683+0000","last_became_peered":"2023-11-06T21:59:40.902683+0000","last_unstale":"2023-11-06T21:59:40.908775+0000","last_undegraded":"2023-11-06T21:59:40.908775+0000","last_fullsized":"2023-11-06T21:59:40.908775+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.875744+0000","last_change":"2023-11-06T21:59:40.401395+0000","last_active":"2023-11-06T21:59:40.875744+0000","last_peered":"2023-11-06T21:59:40.875744+0000","last_clean":"2023-11-06T21:59:40.875744+0000","last_became_active":"2023-11-06T21:59:40.399769+0000","last_became_peered":"2023-11-06T21:59:40.399769+0000","last_unstale":"2023-11-06T21:59:40.875744+0000","last_undegraded":"2023-11-06T21:59:40.875744+0000","last_fullsized":"2023-11-06T21:59:40.875744+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.905087+0000","last_change":"2023-11-06T21:59:40.900645+0000","last_active":"2023-11-06T21:59:40.905087+0000","last_peered":"2023-11-06T21:59:40.905087+0000","last_clean":"2023-11-06T21:59:40.905087+0000","last_became_active":"2023-11-06T21:59:40.899407+0000","last_became_peered":"2023-11-06T21:59:40.899407+0000","last_unstale":"2023-11-06T21:59:40.905087+0000","last_undegraded":"2023-11-06T21:59:40.905087+0000","last_fullsized":"2023-11-06T21:59:40.905087+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2b","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.909846+0000","last_change":"2023-11-06T21:59:40.407004+0000","last_active":"2023-11-06T21:59:40.909846+0000","last_peered":"2023-11-06T21:59:40.909846+0000","last_clean":"2023-11-06T21:59:40.909846+0000","last_became_active":"2023-11-06T21:59:40.406732+0000","last_became_peered":"2023-11-06T21:59:40.406732+0000","last_unstale":"2023-11-06T21:59:40.909846+0000","last_undegraded":"2023-11-06T21:59:40.909846+0000","last_fullsized":"2023-11-06T21:59:40.909846+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899301+0000","last_change":"2023-11-06T21:59:40.895050+0000","last_active":"2023-11-06T21:59:40.899301+0000","last_peered":"2023-11-06T21:59:40.899301+0000","last_clean":"2023-11-06T21:59:40.899301+0000","last_became_active":"2023-11-06T21:59:40.894756+0000","last_became_peered":"2023-11-06T21:59:40.894756+0000","last_unstale":"2023-11-06T21:59:40.899301+0000","last_undegraded":"2023-11-06T21:59:40.899301+0000","last_fullsized":"2023-11-06T21:59:40.899301+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882543+0000","last_change":"2023-11-06T21:59:40.406414+0000","last_active":"2023-11-06T21:59:40.882543+0000","last_peered":"2023-11-06T21:59:40.882543+0000","last_clean":"2023-11-06T21:59:40.882543+0000","last_became_active":"2023-11-06T21:59:40.405739+0000","last_became_peered":"2023-11-06T21:59:40.405739+0000","last_unstale":"2023-11-06T21:59:40.882543+0000","last_undegraded":"2023-11-06T21:59:40.882543+0000","last_fullsized":"2023-11-06T21:59:40.882543+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902523+0000","last_change":"2023-11-06T21:59:40.900030+0000","last_active":"2023-11-06T21:59:40.902523+0000","last_peered":"2023-11-06T21:59:40.902523+0000","last_clean":"2023-11-06T21:59:40.902523+0000","last_became_active":"2023-11-06T21:59:40.899469+0000","last_became_peered":"2023-11-06T21:59:40.899469+0000","last_unstale":"2023-11-06T21:59:40.902523+0000","last_undegraded":"2023-11-06T21:59:40.902523+0000","last_fullsized":"2023-11-06T21:59:40.902523+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.887115+0000","last_change":"2023-11-06T21:59:40.404196+0000","last_active":"2023-11-06T21:59:40.887115+0000","last_peered":"2023-11-06T21:59:40.887115+0000","last_clean":"2023-11-06T21:59:40.887115+0000","last_became_active":"2023-11-06T21:59:40.403920+0000","last_became_peered":"2023-11-06T21:59:40.403920+0000","last_unstale":"2023-11-06T21:59:40.887115+0000","last_undegraded":"2023-11-06T21:59:40.887115+0000","last_fullsized":"2023-11-06T21:59:40.887115+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.895601+0000","last_change":"2023-11-06T21:59:40.889599+0000","last_active":"2023-11-06T21:59:40.895601+0000","last_peered":"2023-11-06T21:59:40.895601+0000","last_clean":"2023-11-06T21:59:40.895601+0000","last_became_active":"2023-11-06T21:59:40.889253+0000","last_became_peered":"2023-11-06T21:59:40.889253+0000","last_unstale":"2023-11-06T21:59:40.895601+0000","last_undegraded":"2023-11-06T21:59:40.895601+0000","last_fullsized":"2023-11-06T21:59:40.895601+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.874234+0000","last_change":"2023-11-06T21:59:40.406559+0000","last_active":"2023-11-06T21:59:40.874234+0000","last_peered":"2023-11-06T21:59:40.874234+0000","last_clean":"2023-11-06T21:59:40.874234+0000","last_became_active":"2023-11-06T21:59:40.405408+0000","last_became_peered":"2023-11-06T21:59:40.405408+0000","last_unstale":"2023-11-06T21:59:40.874234+0000","last_undegraded":"2023-11-06T21:59:40.874234+0000","last_fullsized":"2023-11-06T21:59:40.874234+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896395+0000","last_change":"2023-11-06T21:59:40.891547+0000","last_active":"2023-11-06T21:59:40.896395+0000","last_peered":"2023-11-06T21:59:40.896395+0000","last_clean":"2023-11-06T21:59:40.896395+0000","last_became_active":"2023-11-06T21:59:40.891179+0000","last_became_peered":"2023-11-06T21:59:40.891179+0000","last_unstale":"2023-11-06T21:59:40.896395+0000","last_undegraded":"2023-11-06T21:59:40.896395+0000","last_fullsized":"2023-11-06T21:59:40.896395+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.883412+0000","last_change":"2023-11-06T21:59:40.405498+0000","last_active":"2023-11-06T21:59:40.883412+0000","last_peered":"2023-11-06T21:59:40.883412+0000","last_clean":"2023-11-06T21:59:40.883412+0000","last_became_active":"2023-11-06T21:59:40.405201+0000","last_became_peered":"2023-11-06T21:59:40.405201+0000","last_unstale":"2023-11-06T21:59:40.883412+0000","last_undegraded":"2023-11-06T21:59:40.883412+0000","last_fullsized":"2023-11-06T21:59:40.883412+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899503+0000","last_change":"2023-11-06T21:59:40.894174+0000","last_active":"2023-11-06T21:59:40.899503+0000","last_peered":"2023-11-06T21:59:40.899503+0000","last_clean":"2023-11-06T21:59:40.899503+0000","last_became_active":"2023-11-06T21:59:40.893924+0000","last_became_peered":"2023-11-06T21:59:40.893924+0000","last_unstale":"2023-11-06T21:59:40.899503+0000","last_undegraded":"2023-11-06T21:59:40.899503+0000","last_fullsized":"2023-11-06T21:59:40.899503+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882699+0000","last_change":"2023-11-06T21:59:40.403965+0000","last_active":"2023-11-06T21:59:40.882699+0000","last_peered":"2023-11-06T21:59:40.882699+0000","last_clean":"2023-11-06T21:59:40.882699+0000","last_became_active":"2023-11-06T21:59:40.403290+0000","last_became_peered":"2023-11-06T21:59:40.403290+0000","last_unstale":"2023-11-06T21:59:40.882699+0000","last_undegraded":"2023-11-06T21:59:40.882699+0000","last_fullsized":"2023-11-06T21:59:40.882699+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899428+0000","last_change":"2023-11-06T21:59:40.895238+0000","last_active":"2023-11-06T21:59:40.899428+0000","last_peered":"2023-11-06T21:59:40.899428+0000","last_clean":"2023-11-06T21:59:40.899428+0000","last_became_active":"2023-11-06T21:59:40.894813+0000","last_became_peered":"2023-11-06T21:59:40.894813+0000","last_unstale":"2023-11-06T21:59:40.899428+0000","last_undegraded":"2023-11-06T21:59:40.899428+0000","last_fullsized":"2023-11-06T21:59:40.899428+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.872547+0000","last_change":"2023-11-06T21:59:40.401568+0000","last_active":"2023-11-06T21:59:40.872547+0000","last_peered":"2023-11-06T21:59:40.872547+0000","last_clean":"2023-11-06T21:59:40.872547+0000","last_became_active":"2023-11-06T21:59:40.401289+0000","last_became_peered":"2023-11-06T21:59:40.401289+0000","last_unstale":"2023-11-06T21:59:40.872547+0000","last_undegraded":"2023-11-06T21:59:40.872547+0000","last_fullsized":"2023-11-06T21:59:40.872547+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907952+0000","last_change":"2023-11-06T21:59:40.901564+0000","last_active":"2023-11-06T21:59:40.907952+0000","last_peered":"2023-11-06T21:59:40.907952+0000","last_clean":"2023-11-06T21:59:40.907952+0000","last_became_active":"2023-11-06T21:59:40.896678+0000","last_became_peered":"2023-11-06T21:59:40.896678+0000","last_unstale":"2023-11-06T21:59:40.907952+0000","last_undegraded":"2023-11-06T21:59:40.907952+0000","last_fullsized":"2023-11-06T21:59:40.907952+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.890213+0000","last_change":"2023-11-06T21:59:40.410065+0000","last_active":"2023-11-06T21:59:40.890213+0000","last_peered":"2023-11-06T21:59:40.890213+0000","last_clean":"2023-11-06T21:59:40.890213+0000","last_became_active":"2023-11-06T21:59:40.409787+0000","last_became_peered":"2023-11-06T21:59:40.409787+0000","last_unstale":"2023-11-06T21:59:40.890213+0000","last_undegraded":"2023-11-06T21:59:40.890213+0000","last_fullsized":"2023-11-06T21:59:40.890213+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.893389+0000","last_change":"2023-11-06T21:59:40.889735+0000","last_active":"2023-11-06T21:59:40.893389+0000","last_peered":"2023-11-06T21:59:40.893389+0000","last_clean":"2023-11-06T21:59:40.893389+0000","last_became_active":"2023-11-06T21:59:40.889423+0000","last_became_peered":"2023-11-06T21:59:40.889423+0000","last_unstale":"2023-11-06T21:59:40.893389+0000","last_undegraded":"2023-11-06T21:59:40.893389+0000","last_fullsized":"2023-11-06T21:59:40.893389+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882648+0000","last_change":"2023-11-06T21:59:40.404442+0000","last_active":"2023-11-06T21:59:40.882648+0000","last_peered":"2023-11-06T21:59:40.882648+0000","last_clean":"2023-11-06T21:59:40.882648+0000","last_became_active":"2023-11-06T21:59:40.404143+0000","last_became_peered":"2023-11-06T21:59:40.404143+0000","last_unstale":"2023-11-06T21:59:40.882648+0000","last_undegraded":"2023-11-06T21:59:40.882648+0000","last_fullsized":"2023-11-06T21:59:40.882648+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899289+0000","last_change":"2023-11-06T21:59:40.894443+0000","last_active":"2023-11-06T21:59:40.899289+0000","last_peered":"2023-11-06T21:59:40.899289+0000","last_clean":"2023-11-06T21:59:40.899289+0000","last_became_active":"2023-11-06T21:59:40.894140+0000","last_became_peered":"2023-11-06T21:59:40.894140+0000","last_unstale":"2023-11-06T21:59:40.899289+0000","last_undegraded":"2023-11-06T21:59:40.899289+0000","last_fullsized":"2023-11-06T21:59:40.899289+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882495+0000","last_change":"2023-11-06T21:59:40.409168+0000","last_active":"2023-11-06T21:59:40.882495+0000","last_peered":"2023-11-06T21:59:40.882495+0000","last_clean":"2023-11-06T21:59:40.882495+0000","last_became_active":"2023-11-06T21:59:40.406138+0000","last_became_peered":"2023-11-06T21:59:40.406138+0000","last_unstale":"2023-11-06T21:59:40.882495+0000","last_undegraded":"2023-11-06T21:59:40.882495+0000","last_fullsized":"2023-11-06T21:59:40.882495+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908701+0000","last_change":"2023-11-06T21:59:40.900931+0000","last_active":"2023-11-06T21:59:40.908701+0000","last_peered":"2023-11-06T21:59:40.908701+0000","last_clean":"2023-11-06T21:59:40.908701+0000","last_became_active":"2023-11-06T21:59:40.900699+0000","last_became_peered":"2023-11-06T21:59:40.900699+0000","last_unstale":"2023-11-06T21:59:40.908701+0000","last_undegraded":"2023-11-06T21:59:40.908701+0000","last_fullsized":"2023-11-06T21:59:40.908701+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.887205+0000","last_change":"2023-11-06T21:59:40.396742+0000","last_active":"2023-11-06T21:59:40.887205+0000","last_peered":"2023-11-06T21:59:40.887205+0000","last_clean":"2023-11-06T21:59:40.887205+0000","last_became_active":"2023-11-06T21:59:40.396434+0000","last_became_peered":"2023-11-06T21:59:40.396434+0000","last_unstale":"2023-11-06T21:59:40.887205+0000","last_undegraded":"2023-11-06T21:59:40.887205+0000","last_fullsized":"2023-11-06T21:59:40.887205+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907411+0000","last_change":"2023-11-06T21:59:40.900466+0000","last_active":"2023-11-06T21:59:40.907411+0000","last_peered":"2023-11-06T21:59:40.907411+0000","last_clean":"2023-11-06T21:59:40.907411+0000","last_became_active":"2023-11-06T21:59:40.900151+0000","last_became_peered":"2023-11-06T21:59:40.900151+0000","last_unstale":"2023-11-06T21:59:40.907411+0000","last_undegraded":"2023-11-06T21:59:40.907411+0000","last_fullsized":"2023-11-06T21:59:40.907411+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879328+0000","last_change":"2023-11-06T21:59:40.411171+0000","last_active":"2023-11-06T21:59:40.879328+0000","last_peered":"2023-11-06T21:59:40.879328+0000","last_clean":"2023-11-06T21:59:40.879328+0000","last_became_active":"2023-11-06T21:59:40.410909+0000","last_became_peered":"2023-11-06T21:59:40.410909+0000","last_unstale":"2023-11-06T21:59:40.879328+0000","last_undegraded":"2023-11-06T21:59:40.879328+0000","last_fullsized":"2023-11-06T21:59:40.879328+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.906614+0000","last_change":"2023-11-06T21:59:40.898377+0000","last_active":"2023-11-06T21:59:40.906614+0000","last_peered":"2023-11-06T21:59:40.906614+0000","last_clean":"2023-11-06T21:59:40.906614+0000","last_became_active":"2023-11-06T21:59:40.897661+0000","last_became_peered":"2023-11-06T21:59:40.897661+0000","last_unstale":"2023-11-06T21:59:40.906614+0000","last_undegraded":"2023-11-06T21:59:40.906614+0000","last_fullsized":"2023-11-06T21:59:40.906614+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1f","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902417+0000","last_change":"2023-11-06T21:59:40.397779+0000","last_active":"2023-11-06T21:59:40.902417+0000","last_peered":"2023-11-06T21:59:40.902417+0000","last_clean":"2023-11-06T21:59:40.902417+0000","last_became_active":"2023-11-06T21:59:40.397274+0000","last_became_peered":"2023-11-06T21:59:40.397274+0000","last_unstale":"2023-11-06T21:59:40.902417+0000","last_undegraded":"2023-11-06T21:59:40.902417+0000","last_fullsized":"2023-11-06T21:59:40.902417+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902241+0000","last_change":"2023-11-06T21:59:40.898423+0000","last_active":"2023-11-06T21:59:40.902241+0000","last_peered":"2023-11-06T21:59:40.902241+0000","last_clean":"2023-11-06T21:59:40.902241+0000","last_became_active":"2023-11-06T21:59:40.898136+0000","last_became_peered":"2023-11-06T21:59:40.898136+0000","last_unstale":"2023-11-06T21:59:40.902241+0000","last_undegraded":"2023-11-06T21:59:40.902241+0000","last_fullsized":"2023-11-06T21:59:40.902241+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.1e","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.904579+0000","last_change":"2023-11-06T21:59:40.401361+0000","last_active":"2023-11-06T21:59:40.904579+0000","last_peered":"2023-11-06T21:59:40.904579+0000","last_clean":"2023-11-06T21:59:40.904579+0000","last_became_active":"2023-11-06T21:59:40.401032+0000","last_became_peered":"2023-11-06T21:59:40.401032+0000","last_unstale":"2023-11-06T21:59:40.904579+0000","last_undegraded":"2023-11-06T21:59:40.904579+0000","last_fullsized":"2023-11-06T21:59:40.904579+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907738+0000","last_change":"2023-11-06T21:59:40.905442+0000","last_active":"2023-11-06T21:59:40.907738+0000","last_peered":"2023-11-06T21:59:40.907738+0000","last_clean":"2023-11-06T21:59:40.907738+0000","last_became_active":"2023-11-06T21:59:40.905087+0000","last_became_peered":"2023-11-06T21:59:40.905087+0000","last_unstale":"2023-11-06T21:59:40.907738+0000","last_undegraded":"2023-11-06T21:59:40.907738+0000","last_fullsized":"2023-11-06T21:59:40.907738+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1d","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908006+0000","last_change":"2023-11-06T21:59:40.413934+0000","last_active":"2023-11-06T21:59:40.908006+0000","last_peered":"2023-11-06T21:59:40.908006+0000","last_clean":"2023-11-06T21:59:40.908006+0000","last_became_active":"2023-11-06T21:59:40.413663+0000","last_became_peered":"2023-11-06T21:59:40.413663+0000","last_unstale":"2023-11-06T21:59:40.908006+0000","last_undegraded":"2023-11-06T21:59:40.908006+0000","last_fullsized":"2023-11-06T21:59:40.908006+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.898951+0000","last_change":"2023-11-06T21:59:40.895397+0000","last_active":"2023-11-06T21:59:40.898951+0000","last_peered":"2023-11-06T21:59:40.898951+0000","last_clean":"2023-11-06T21:59:40.898951+0000","last_became_active":"2023-11-06T21:59:40.895097+0000","last_became_peered":"2023-11-06T21:59:40.895097+0000","last_unstale":"2023-11-06T21:59:40.898951+0000","last_undegraded":"2023-11-06T21:59:40.898951+0000","last_fullsized":"2023-11-06T21:59:40.898951+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879723+0000","last_change":"2023-11-06T21:59:40.397238+0000","last_active":"2023-11-06T21:59:40.879723+0000","last_peered":"2023-11-06T21:59:40.879723+0000","last_clean":"2023-11-06T21:59:40.879723+0000","last_became_active":"2023-11-06T21:59:40.395006+0000","last_became_peered":"2023-11-06T21:59:40.395006+0000","last_unstale":"2023-11-06T21:59:40.879723+0000","last_undegraded":"2023-11-06T21:59:40.879723+0000","last_fullsized":"2023-11-06T21:59:40.879723+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897556+0000","last_change":"2023-11-06T21:59:40.888847+0000","last_active":"2023-11-06T21:59:40.897556+0000","last_peered":"2023-11-06T21:59:40.897556+0000","last_clean":"2023-11-06T21:59:40.897556+0000","last_became_active":"2023-11-06T21:59:40.888582+0000","last_became_peered":"2023-11-06T21:59:40.888582+0000","last_unstale":"2023-11-06T21:59:40.897556+0000","last_undegraded":"2023-11-06T21:59:40.897556+0000","last_fullsized":"2023-11-06T21:59:40.897556+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.881097+0000","last_change":"2023-11-06T21:59:40.411815+0000","last_active":"2023-11-06T21:59:40.881097+0000","last_peered":"2023-11-06T21:59:40.881097+0000","last_clean":"2023-11-06T21:59:40.881097+0000","last_became_active":"2023-11-06T21:59:40.411177+0000","last_became_peered":"2023-11-06T21:59:40.411177+0000","last_unstale":"2023-11-06T21:59:40.881097+0000","last_undegraded":"2023-11-06T21:59:40.881097+0000","last_fullsized":"2023-11-06T21:59:40.881097+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907277+0000","last_change":"2023-11-06T21:59:40.899751+0000","last_active":"2023-11-06T21:59:40.907277+0000","last_peered":"2023-11-06T21:59:40.907277+0000","last_clean":"2023-11-06T21:59:40.907277+0000","last_became_active":"2023-11-06T21:59:40.898435+0000","last_became_peered":"2023-11-06T21:59:40.898435+0000","last_unstale":"2023-11-06T21:59:40.907277+0000","last_undegraded":"2023-11-06T21:59:40.907277+0000","last_fullsized":"2023-11-06T21:59:40.907277+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.874450+0000","last_change":"2023-11-06T21:59:40.407711+0000","last_active":"2023-11-06T21:59:40.874450+0000","last_peered":"2023-11-06T21:59:40.874450+0000","last_clean":"2023-11-06T21:59:40.874450+0000","last_became_active":"2023-11-06T21:59:40.407411+0000","last_became_peered":"2023-11-06T21:59:40.407411+0000","last_unstale":"2023-11-06T21:59:40.874450+0000","last_undegraded":"2023-11-06T21:59:40.874450+0000","last_fullsized":"2023-11-06T21:59:40.874450+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897650+0000","last_change":"2023-11-06T21:59:40.894215+0000","last_active":"2023-11-06T21:59:40.897650+0000","last_peered":"2023-11-06T21:59:40.897650+0000","last_clean":"2023-11-06T21:59:40.897650+0000","last_became_active":"2023-11-06T21:59:40.893836+0000","last_became_peered":"2023-11-06T21:59:40.893836+0000","last_unstale":"2023-11-06T21:59:40.897650+0000","last_undegraded":"2023-11-06T21:59:40.897650+0000","last_fullsized":"2023-11-06T21:59:40.897650+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882303+0000","last_change":"2023-11-06T21:59:40.404318+0000","last_active":"2023-11-06T21:59:40.882303+0000","last_peered":"2023-11-06T21:59:40.882303+0000","last_clean":"2023-11-06T21:59:40.882303+0000","last_became_active":"2023-11-06T21:59:40.403389+0000","last_became_peered":"2023-11-06T21:59:40.403389+0000","last_unstale":"2023-11-06T21:59:40.882303+0000","last_undegraded":"2023-11-06T21:59:40.882303+0000","last_fullsized":"2023-11-06T21:59:40.882303+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907197+0000","last_change":"2023-11-06T21:59:40.899151+0000","last_active":"2023-11-06T21:59:40.907197+0000","last_peered":"2023-11-06T21:59:40.907197+0000","last_clean":"2023-11-06T21:59:40.907197+0000","last_became_active":"2023-11-06T21:59:40.898934+0000","last_became_peered":"2023-11-06T21:59:40.898934+0000","last_unstale":"2023-11-06T21:59:40.907197+0000","last_undegraded":"2023-11-06T21:59:40.907197+0000","last_fullsized":"2023-11-06T21:59:40.907197+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.875758+0000","last_change":"2023-11-06T21:59:40.407923+0000","last_active":"2023-11-06T21:59:40.875758+0000","last_peered":"2023-11-06T21:59:40.875758+0000","last_clean":"2023-11-06T21:59:40.875758+0000","last_became_active":"2023-11-06T21:59:40.406915+0000","last_became_peered":"2023-11-06T21:59:40.406915+0000","last_unstale":"2023-11-06T21:59:40.875758+0000","last_undegraded":"2023-11-06T21:59:40.875758+0000","last_fullsized":"2023-11-06T21:59:40.875758+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.905042+0000","last_change":"2023-11-06T21:59:40.899319+0000","last_active":"2023-11-06T21:59:40.905042+0000","last_peered":"2023-11-06T21:59:40.905042+0000","last_clean":"2023-11-06T21:59:40.905042+0000","last_became_active":"2023-11-06T21:59:40.897015+0000","last_became_peered":"2023-11-06T21:59:40.897015+0000","last_unstale":"2023-11-06T21:59:40.905042+0000","last_undegraded":"2023-11-06T21:59:40.905042+0000","last_fullsized":"2023-11-06T21:59:40.905042+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.f","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.903092+0000","last_change":"2023-11-06T21:59:40.409729+0000","last_active":"2023-11-06T21:59:40.903092+0000","last_peered":"2023-11-06T21:59:40.903092+0000","last_clean":"2023-11-06T21:59:40.903092+0000","last_became_active":"2023-11-06T21:59:40.409439+0000","last_became_peered":"2023-11-06T21:59:40.409439+0000","last_unstale":"2023-11-06T21:59:40.903092+0000","last_undegraded":"2023-11-06T21:59:40.903092+0000","last_fullsized":"2023-11-06T21:59:40.903092+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899642+0000","last_change":"2023-11-06T21:59:40.888966+0000","last_active":"2023-11-06T21:59:40.899642+0000","last_peered":"2023-11-06T21:59:40.899642+0000","last_clean":"2023-11-06T21:59:40.899642+0000","last_became_active":"2023-11-06T21:59:40.888692+0000","last_became_peered":"2023-11-06T21:59:40.888692+0000","last_unstale":"2023-11-06T21:59:40.899642+0000","last_undegraded":"2023-11-06T21:59:40.899642+0000","last_fullsized":"2023-11-06T21:59:40.899642+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.e","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.903172+0000","last_change":"2023-11-06T21:59:40.407412+0000","last_active":"2023-11-06T21:59:40.903172+0000","last_peered":"2023-11-06T21:59:40.903172+0000","last_clean":"2023-11-06T21:59:40.903172+0000","last_became_active":"2023-11-06T21:59:40.405257+0000","last_became_peered":"2023-11-06T21:59:40.405257+0000","last_unstale":"2023-11-06T21:59:40.903172+0000","last_undegraded":"2023-11-06T21:59:40.903172+0000","last_fullsized":"2023-11-06T21:59:40.903172+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.905479+0000","last_change":"2023-11-06T21:59:40.900815+0000","last_active":"2023-11-06T21:59:40.905479+0000","last_peered":"2023-11-06T21:59:40.905479+0000","last_clean":"2023-11-06T21:59:40.905479+0000","last_became_active":"2023-11-06T21:59:40.899554+0000","last_became_peered":"2023-11-06T21:59:40.899554+0000","last_unstale":"2023-11-06T21:59:40.905479+0000","last_undegraded":"2023-11-06T21:59:40.905479+0000","last_fullsized":"2023-11-06T21:59:40.905479+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882999+0000","last_change":"2023-11-06T21:59:40.400555+0000","last_active":"2023-11-06T21:59:40.882999+0000","last_peered":"2023-11-06T21:59:40.882999+0000","last_clean":"2023-11-06T21:59:40.882999+0000","last_became_active":"2023-11-06T21:59:40.400225+0000","last_became_peered":"2023-11-06T21:59:40.400225+0000","last_unstale":"2023-11-06T21:59:40.882999+0000","last_undegraded":"2023-11-06T21:59:40.882999+0000","last_fullsized":"2023-11-06T21:59:40.882999+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.893630+0000","last_change":"2023-11-06T21:59:40.887899+0000","last_active":"2023-11-06T21:59:40.893630+0000","last_peered":"2023-11-06T21:59:40.893630+0000","last_clean":"2023-11-06T21:59:40.893630+0000","last_became_active":"2023-11-06T21:59:40.887466+0000","last_became_peered":"2023-11-06T21:59:40.887466+0000","last_unstale":"2023-11-06T21:59:40.893630+0000","last_undegraded":"2023-11-06T21:59:40.893630+0000","last_fullsized":"2023-11-06T21:59:40.893630+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882870+0000","last_change":"2023-11-06T21:59:40.410553+0000","last_active":"2023-11-06T21:59:40.882870+0000","last_peered":"2023-11-06T21:59:40.882870+0000","last_clean":"2023-11-06T21:59:40.882870+0000","last_became_active":"2023-11-06T21:59:40.409980+0000","last_became_peered":"2023-11-06T21:59:40.409980+0000","last_unstale":"2023-11-06T21:59:40.882870+0000","last_undegraded":"2023-11-06T21:59:40.882870+0000","last_fullsized":"2023-11-06T21:59:40.882870+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899055+0000","last_change":"2023-11-06T21:59:40.894561+0000","last_active":"2023-11-06T21:59:40.899055+0000","last_peered":"2023-11-06T21:59:40.899055+0000","last_clean":"2023-11-06T21:59:40.899055+0000","last_became_active":"2023-11-06T21:59:40.894231+0000","last_became_peered":"2023-11-06T21:59:40.894231+0000","last_unstale":"2023-11-06T21:59:40.899055+0000","last_undegraded":"2023-11-06T21:59:40.899055+0000","last_fullsized":"2023-11-06T21:59:40.899055+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.b","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.905871+0000","last_change":"2023-11-06T21:59:40.406018+0000","last_active":"2023-11-06T21:59:40.905871+0000","last_peered":"2023-11-06T21:59:40.905871+0000","last_clean":"2023-11-06T21:59:40.905871+0000","last_became_active":"2023-11-06T21:59:40.405722+0000","last_became_peered":"2023-11-06T21:59:40.405722+0000","last_unstale":"2023-11-06T21:59:40.905871+0000","last_undegraded":"2023-11-06T21:59:40.905871+0000","last_fullsized":"2023-11-06T21:59:40.905871+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902648+0000","last_change":"2023-11-06T21:59:40.898369+0000","last_active":"2023-11-06T21:59:40.902648+0000","last_peered":"2023-11-06T21:59:40.902648+0000","last_clean":"2023-11-06T21:59:40.902648+0000","last_became_active":"2023-11-06T21:59:40.898072+0000","last_became_peered":"2023-11-06T21:59:40.898072+0000","last_unstale":"2023-11-06T21:59:40.902648+0000","last_undegraded":"2023-11-06T21:59:40.902648+0000","last_fullsized":"2023-11-06T21:59:40.902648+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882602+0000","last_change":"2023-11-06T21:59:40.410696+0000","last_active":"2023-11-06T21:59:40.882602+0000","last_peered":"2023-11-06T21:59:40.882602+0000","last_clean":"2023-11-06T21:59:40.882602+0000","last_became_active":"2023-11-06T21:59:40.409187+0000","last_became_peered":"2023-11-06T21:59:40.409187+0000","last_unstale":"2023-11-06T21:59:40.882602+0000","last_undegraded":"2023-11-06T21:59:40.882602+0000","last_fullsized":"2023-11-06T21:59:40.882602+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.898942+0000","last_change":"2023-11-06T21:59:40.895397+0000","last_active":"2023-11-06T21:59:40.898942+0000","last_peered":"2023-11-06T21:59:40.898942+0000","last_clean":"2023-11-06T21:59:40.898942+0000","last_became_active":"2023-11-06T21:59:40.894437+0000","last_became_peered":"2023-11-06T21:59:40.894437+0000","last_unstale":"2023-11-06T21:59:40.898942+0000","last_undegraded":"2023-11-06T21:59:40.898942+0000","last_fullsized":"2023-11-06T21:59:40.898942+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.877860+0000","last_change":"2023-11-06T21:59:40.399179+0000","last_active":"2023-11-06T21:59:40.877860+0000","last_peered":"2023-11-06T21:59:40.877860+0000","last_clean":"2023-11-06T21:59:40.877860+0000","last_became_active":"2023-11-06T21:59:40.398897+0000","last_became_peered":"2023-11-06T21:59:40.398897+0000","last_unstale":"2023-11-06T21:59:40.877860+0000","last_undegraded":"2023-11-06T21:59:40.877860+0000","last_fullsized":"2023-11-06T21:59:40.877860+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.900547+0000","last_change":"2023-11-06T21:59:40.891221+0000","last_active":"2023-11-06T21:59:40.900547+0000","last_peered":"2023-11-06T21:59:40.900547+0000","last_clean":"2023-11-06T21:59:40.900547+0000","last_became_active":"2023-11-06T21:59:40.890926+0000","last_became_peered":"2023-11-06T21:59:40.890926+0000","last_unstale":"2023-11-06T21:59:40.900547+0000","last_undegraded":"2023-11-06T21:59:40.900547+0000","last_fullsized":"2023-11-06T21:59:40.900547+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.873276+0000","last_change":"2023-11-06T21:59:40.397667+0000","last_active":"2023-11-06T21:59:40.873276+0000","last_peered":"2023-11-06T21:59:40.873276+0000","last_clean":"2023-11-06T21:59:40.873276+0000","last_became_active":"2023-11-06T21:59:40.397310+0000","last_became_peered":"2023-11-06T21:59:40.397310+0000","last_unstale":"2023-11-06T21:59:40.873276+0000","last_undegraded":"2023-11-06T21:59:40.873276+0000","last_fullsized":"2023-11-06T21:59:40.873276+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.890294+0000","last_change":"2023-11-06T21:59:40.883958+0000","last_active":"2023-11-06T21:59:40.890294+0000","last_peered":"2023-11-06T21:59:40.890294+0000","last_clean":"2023-11-06T21:59:40.890294+0000","last_became_active":"2023-11-06T21:59:40.883575+0000","last_became_peered":"2023-11-06T21:59:40.883575+0000","last_unstale":"2023-11-06T21:59:40.890294+0000","last_undegraded":"2023-11-06T21:59:40.890294+0000","last_fullsized":"2023-11-06T21:59:40.890294+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.7","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.901404+0000","last_change":"2023-11-06T21:59:40.413074+0000","last_active":"2023-11-06T21:59:40.901404+0000","last_peered":"2023-11-06T21:59:40.901404+0000","last_clean":"2023-11-06T21:59:40.901404+0000","last_became_active":"2023-11-06T21:59:40.412134+0000","last_became_peered":"2023-11-06T21:59:40.412134+0000","last_unstale":"2023-11-06T21:59:40.901404+0000","last_undegraded":"2023-11-06T21:59:40.901404+0000","last_fullsized":"2023-11-06T21:59:40.901404+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907395+0000","last_change":"2023-11-06T21:59:40.892081+0000","last_active":"2023-11-06T21:59:40.907395+0000","last_peered":"2023-11-06T21:59:40.907395+0000","last_clean":"2023-11-06T21:59:40.907395+0000","last_became_active":"2023-11-06T21:59:40.891804+0000","last_became_peered":"2023-11-06T21:59:40.891804+0000","last_unstale":"2023-11-06T21:59:40.907395+0000","last_undegraded":"2023-11-06T21:59:40.907395+0000","last_fullsized":"2023-11-06T21:59:40.907395+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.888947+0000","last_change":"2023-11-06T21:59:40.407023+0000","last_active":"2023-11-06T21:59:40.888947+0000","last_peered":"2023-11-06T21:59:40.888947+0000","last_clean":"2023-11-06T21:59:40.888947+0000","last_became_active":"2023-11-06T21:59:40.406741+0000","last_became_peered":"2023-11-06T21:59:40.406741+0000","last_unstale":"2023-11-06T21:59:40.888947+0000","last_undegraded":"2023-11-06T21:59:40.888947+0000","last_fullsized":"2023-11-06T21:59:40.888947+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908291+0000","last_change":"2023-11-06T21:59:40.900521+0000","last_active":"2023-11-06T21:59:40.908291+0000","last_peered":"2023-11-06T21:59:40.908291+0000","last_clean":"2023-11-06T21:59:40.908291+0000","last_became_active":"2023-11-06T21:59:40.900140+0000","last_became_peered":"2023-11-06T21:59:40.900140+0000","last_unstale":"2023-11-06T21:59:40.908291+0000","last_undegraded":"2023-11-06T21:59:40.908291+0000","last_fullsized":"2023-11-06T21:59:40.908291+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.881655+0000","last_change":"2023-11-06T21:59:40.400982+0000","last_active":"2023-11-06T21:59:40.881655+0000","last_peered":"2023-11-06T21:59:40.881655+0000","last_clean":"2023-11-06T21:59:40.881655+0000","last_became_active":"2023-11-06T21:59:40.400697+0000","last_became_peered":"2023-11-06T21:59:40.400697+0000","last_unstale":"2023-11-06T21:59:40.881655+0000","last_undegraded":"2023-11-06T21:59:40.881655+0000","last_fullsized":"2023-11-06T21:59:40.881655+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.898507+0000","last_change":"2023-11-06T21:59:40.892656+0000","last_active":"2023-11-06T21:59:40.898507+0000","last_peered":"2023-11-06T21:59:40.898507+0000","last_clean":"2023-11-06T21:59:40.898507+0000","last_became_active":"2023-11-06T21:59:40.892279+0000","last_became_peered":"2023-11-06T21:59:40.892279+0000","last_unstale":"2023-11-06T21:59:40.898507+0000","last_undegraded":"2023-11-06T21:59:40.898507+0000","last_fullsized":"2023-11-06T21:59:40.898507+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.873056+0000","last_change":"2023-11-06T21:59:40.402327+0000","last_active":"2023-11-06T21:59:40.873056+0000","last_peered":"2023-11-06T21:59:40.873056+0000","last_clean":"2023-11-06T21:59:40.873056+0000","last_became_active":"2023-11-06T21:59:40.401964+0000","last_became_peered":"2023-11-06T21:59:40.401964+0000","last_unstale":"2023-11-06T21:59:40.873056+0000","last_undegraded":"2023-11-06T21:59:40.873056+0000","last_fullsized":"2023-11-06T21:59:40.873056+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897046+0000","last_change":"2023-11-06T21:59:40.889256+0000","last_active":"2023-11-06T21:59:40.897046+0000","last_peered":"2023-11-06T21:59:40.897046+0000","last_clean":"2023-11-06T21:59:40.897046+0000","last_became_active":"2023-11-06T21:59:40.888945+0000","last_became_peered":"2023-11-06T21:59:40.888945+0000","last_unstale":"2023-11-06T21:59:40.897046+0000","last_undegraded":"2023-11-06T21:59:40.897046+0000","last_fullsized":"2023-11-06T21:59:40.897046+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.880423+0000","last_change":"2023-11-06T21:59:40.405739+0000","last_active":"2023-11-06T21:59:40.880423+0000","last_peered":"2023-11-06T21:59:40.880423+0000","last_clean":"2023-11-06T21:59:40.880423+0000","last_became_active":"2023-11-06T21:59:40.405442+0000","last_became_peered":"2023-11-06T21:59:40.405442+0000","last_unstale":"2023-11-06T21:59:40.880423+0000","last_undegraded":"2023-11-06T21:59:40.880423+0000","last_fullsized":"2023-11-06T21:59:40.880423+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896345+0000","last_change":"2023-11-06T21:59:40.893426+0000","last_active":"2023-11-06T21:59:40.896345+0000","last_peered":"2023-11-06T21:59:40.896345+0000","last_clean":"2023-11-06T21:59:40.896345+0000","last_became_active":"2023-11-06T21:59:40.890618+0000","last_became_peered":"2023-11-06T21:59:40.890618+0000","last_unstale":"2023-11-06T21:59:40.896345+0000","last_undegraded":"2023-11-06T21:59:40.896345+0000","last_fullsized":"2023-11-06T21:59:40.896345+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.881046+0000","last_change":"2023-11-06T21:59:40.398590+0000","last_active":"2023-11-06T21:59:40.881046+0000","last_peered":"2023-11-06T21:59:40.881046+0000","last_clean":"2023-11-06T21:59:40.881046+0000","last_became_active":"2023-11-06T21:59:40.398158+0000","last_became_peered":"2023-11-06T21:59:40.398158+0000","last_unstale":"2023-11-06T21:59:40.881046+0000","last_undegraded":"2023-11-06T21:59:40.881046+0000","last_fullsized":"2023-11-06T21:59:40.881046+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.885871+0000","last_change":"2023-11-06T21:59:40.883826+0000","last_active":"2023-11-06T21:59:40.885871+0000","last_peered":"2023-11-06T21:59:40.885871+0000","last_clean":"2023-11-06T21:59:40.885871+0000","last_became_active":"2023-11-06T21:59:40.883552+0000","last_became_peered":"2023-11-06T21:59:40.883552+0000","last_unstale":"2023-11-06T21:59:40.885871+0000","last_undegraded":"2023-11-06T21:59:40.885871+0000","last_fullsized":"2023-11-06T21:59:40.885871+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.872794+0000","last_change":"2023-11-06T21:59:40.400285+0000","last_active":"2023-11-06T21:59:40.872794+0000","last_peered":"2023-11-06T21:59:40.872794+0000","last_clean":"2023-11-06T21:59:40.872794+0000","last_became_active":"2023-11-06T21:59:40.400001+0000","last_became_peered":"2023-11-06T21:59:40.400001+0000","last_unstale":"2023-11-06T21:59:40.872794+0000","last_undegraded":"2023-11-06T21:59:40.872794+0000","last_fullsized":"2023-11-06T21:59:40.872794+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.904564+0000","last_change":"2023-11-06T21:59:40.898026+0000","last_active":"2023-11-06T21:59:40.904564+0000","last_peered":"2023-11-06T21:59:40.904564+0000","last_clean":"2023-11-06T21:59:40.904564+0000","last_became_active":"2023-11-06T21:59:40.895767+0000","last_became_peered":"2023-11-06T21:59:40.895767+0000","last_unstale":"2023-11-06T21:59:40.904564+0000","last_undegraded":"2023-11-06T21:59:40.904564+0000","last_fullsized":"2023-11-06T21:59:40.904564+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.874345+0000","last_change":"2023-11-06T21:59:40.406940+0000","last_active":"2023-11-06T21:59:40.874345+0000","last_peered":"2023-11-06T21:59:40.874345+0000","last_clean":"2023-11-06T21:59:40.874345+0000","last_became_active":"2023-11-06T21:59:40.406276+0000","last_became_peered":"2023-11-06T21:59:40.406276+0000","last_unstale":"2023-11-06T21:59:40.874345+0000","last_undegraded":"2023-11-06T21:59:40.874345+0000","last_fullsized":"2023-11-06T21:59:40.874345+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908720+0000","last_change":"2023-11-06T21:59:40.893113+0000","last_active":"2023-11-06T21:59:40.908720+0000","last_peered":"2023-11-06T21:59:40.908720+0000","last_clean":"2023-11-06T21:59:40.908720+0000","last_became_active":"2023-11-06T21:59:40.891126+0000","last_became_peered":"2023-11-06T21:59:40.891126+0000","last_unstale":"2023-11-06T21:59:40.908720+0000","last_undegraded":"2023-11-06T21:59:40.908720+0000","last_fullsized":"2023-11-06T21:59:40.908720+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"169'3","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.909427+0000","last_change":"2023-11-06T21:59:40.406818+0000","last_active":"2023-11-06T21:59:40.909427+0000","last_peered":"2023-11-06T21:59:40.909427+0000","last_clean":"2023-11-06T21:59:40.909427+0000","last_became_active":"2023-11-06T21:59:40.406429+0000","last_became_peered":"2023-11-06T21:59:40.406429+0000","last_unstale":"2023-11-06T21:59:40.909427+0000","last_undegraded":"2023-11-06T21:59:40.909427+0000","last_fullsized":"2023-11-06T21:59:40.909427+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882932+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T21:59:40.882932+0000","last_peered":"2023-11-06T21:59:40.882932+0000","last_clean":"2023-11-06T21:59:40.882932+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T21:59:40.882932+0000","last_undegraded":"2023-11-06T21:59:40.882932+0000","last_fullsized":"2023-11-06T21:59:40.882932+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897065+0000","last_change":"2023-11-06T21:59:40.889803+0000","last_active":"2023-11-06T21:59:40.897065+0000","last_peered":"2023-11-06T21:59:40.897065+0000","last_clean":"2023-11-06T21:59:40.897065+0000","last_became_active":"2023-11-06T21:59:40.889509+0000","last_became_peered":"2023-11-06T21:59:40.889509+0000","last_unstale":"2023-11-06T21:59:40.897065+0000","last_undegraded":"2023-11-06T21:59:40.897065+0000","last_fullsized":"2023-11-06T21:59:40.897065+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.13","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.900928+0000","last_change":"2023-11-06T21:59:40.402832+0000","last_active":"2023-11-06T21:59:40.900928+0000","last_peered":"2023-11-06T21:59:40.900928+0000","last_clean":"2023-11-06T21:59:40.900928+0000","last_became_active":"2023-11-06T21:59:40.402540+0000","last_became_peered":"2023-11-06T21:59:40.402540+0000","last_unstale":"2023-11-06T21:59:40.900928+0000","last_undegraded":"2023-11-06T21:59:40.900928+0000","last_fullsized":"2023-11-06T21:59:40.900928+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.876995+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T21:59:40.876995+0000","last_peered":"2023-11-06T21:59:40.876995+0000","last_clean":"2023-11-06T21:59:40.876995+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T21:59:40.876995+0000","last_undegraded":"2023-11-06T21:59:40.876995+0000","last_fullsized":"2023-11-06T21:59:40.876995+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897707+0000","last_change":"2023-11-06T21:59:40.894570+0000","last_active":"2023-11-06T21:59:40.897707+0000","last_peered":"2023-11-06T21:59:40.897707+0000","last_clean":"2023-11-06T21:59:40.897707+0000","last_became_active":"2023-11-06T21:59:40.894269+0000","last_became_peered":"2023-11-06T21:59:40.894269+0000","last_unstale":"2023-11-06T21:59:40.897707+0000","last_undegraded":"2023-11-06T21:59:40.897707+0000","last_fullsized":"2023-11-06T21:59:40.897707+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"169'4","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.906405+0000","last_change":"2023-11-06T21:59:40.407268+0000","last_active":"2023-11-06T21:59:40.906405+0000","last_peered":"2023-11-06T21:59:40.906405+0000","last_clean":"2023-11-06T21:59:40.906405+0000","last_became_active":"2023-11-06T21:59:40.406958+0000","last_became_peered":"2023-11-06T21:59:40.406958+0000","last_unstale":"2023-11-06T21:59:40.906405+0000","last_undegraded":"2023-11-06T21:59:40.906405+0000","last_fullsized":"2023-11-06T21:59:40.906405+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.874761+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T21:59:40.874761+0000","last_peered":"2023-11-06T21:59:40.874761+0000","last_clean":"2023-11-06T21:59:40.874761+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T21:59:40.874761+0000","last_undegraded":"2023-11-06T21:59:40.874761+0000","last_fullsized":"2023-11-06T21:59:40.874761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907610+0000","last_change":"2023-11-06T21:59:40.891989+0000","last_active":"2023-11-06T21:59:40.907610+0000","last_peered":"2023-11-06T21:59:40.907610+0000","last_clean":"2023-11-06T21:59:40.907610+0000","last_became_active":"2023-11-06T21:59:40.891726+0000","last_became_peered":"2023-11-06T21:59:40.891726+0000","last_unstale":"2023-11-06T21:59:40.907610+0000","last_undegraded":"2023-11-06T21:59:40.907610+0000","last_fullsized":"2023-11-06T21:59:40.907610+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.876899+0000","last_change":"2023-11-06T21:59:40.397282+0000","last_active":"2023-11-06T21:59:40.876899+0000","last_peered":"2023-11-06T21:59:40.876899+0000","last_clean":"2023-11-06T21:59:40.876899+0000","last_became_active":"2023-11-06T21:59:40.394993+0000","last_became_peered":"2023-11-06T21:59:40.394993+0000","last_unstale":"2023-11-06T21:59:40.876899+0000","last_undegraded":"2023-11-06T21:59:40.876899+0000","last_fullsized":"2023-11-06T21:59:40.876899+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.878993+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T21:59:40.878993+0000","last_peered":"2023-11-06T21:59:40.878993+0000","last_clean":"2023-11-06T21:59:40.878993+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T21:59:40.878993+0000","last_undegraded":"2023-11-06T21:59:40.878993+0000","last_fullsized":"2023-11-06T21:59:40.878993+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":143,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.886776+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T21:59:40.886776+0000","last_peered":"2023-11-06T21:59:40.886776+0000","last_clean":"2023-11-06T21:59:40.886776+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T21:59:40.886776+0000","last_undegraded":"2023-11-06T21:59:40.886776+0000","last_fullsized":"2023-11-06T21:59:40.886776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.904499+0000","last_change":"2023-11-06T21:59:40.896800+0000","last_active":"2023-11-06T21:59:40.904499+0000","last_peered":"2023-11-06T21:59:40.904499+0000","last_clean":"2023-11-06T21:59:40.904499+0000","last_became_active":"2023-11-06T21:59:40.896495+0000","last_became_peered":"2023-11-06T21:59:40.896495+0000","last_unstale":"2023-11-06T21:59:40.904499+0000","last_undegraded":"2023-11-06T21:59:40.904499+0000","last_fullsized":"2023-11-06T21:59:40.904499+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.877749+0000","last_change":"2023-11-06T21:59:40.408314+0000","last_active":"2023-11-06T21:59:40.877749+0000","last_peered":"2023-11-06T21:59:40.877749+0000","last_clean":"2023-11-06T21:59:40.877749+0000","last_became_active":"2023-11-06T21:59:40.407703+0000","last_became_peered":"2023-11-06T21:59:40.407703+0000","last_unstale":"2023-11-06T21:59:40.877749+0000","last_undegraded":"2023-11-06T21:59:40.877749+0000","last_fullsized":"2023-11-06T21:59:40.877749+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879618+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T21:59:40.879618+0000","last_peered":"2023-11-06T21:59:40.879618+0000","last_clean":"2023-11-06T21:59:40.879618+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T21:59:40.879618+0000","last_undegraded":"2023-11-06T21:59:40.879618+0000","last_fullsized":"2023-11-06T21:59:40.879618+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.906419+0000","last_change":"2023-11-06T21:59:40.898302+0000","last_active":"2023-11-06T21:59:40.906419+0000","last_peered":"2023-11-06T21:59:40.906419+0000","last_clean":"2023-11-06T21:59:40.906419+0000","last_became_active":"2023-11-06T21:59:40.898023+0000","last_became_peered":"2023-11-06T21:59:40.898023+0000","last_unstale":"2023-11-06T21:59:40.906419+0000","last_undegraded":"2023-11-06T21:59:40.906419+0000","last_fullsized":"2023-11-06T21:59:40.906419+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.910194+0000","last_change":"2023-11-06T21:59:40.410937+0000","last_active":"2023-11-06T21:59:40.910194+0000","last_peered":"2023-11-06T21:59:40.910194+0000","last_clean":"2023-11-06T21:59:40.910194+0000","last_became_active":"2023-11-06T21:59:40.410339+0000","last_became_peered":"2023-11-06T21:59:40.410339+0000","last_unstale":"2023-11-06T21:59:40.910194+0000","last_undegraded":"2023-11-06T21:59:40.910194+0000","last_fullsized":"2023-11-06T21:59:40.910194+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882063+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T21:59:40.882063+0000","last_peered":"2023-11-06T21:59:40.882063+0000","last_clean":"2023-11-06T21:59:40.882063+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T21:59:40.882063+0000","last_undegraded":"2023-11-06T21:59:40.882063+0000","last_fullsized":"2023-11-06T21:59:40.882063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896511+0000","last_change":"2023-11-06T21:59:40.891693+0000","last_active":"2023-11-06T21:59:40.896511+0000","last_peered":"2023-11-06T21:59:40.896511+0000","last_clean":"2023-11-06T21:59:40.896511+0000","last_became_active":"2023-11-06T21:59:40.891375+0000","last_became_peered":"2023-11-06T21:59:40.891375+0000","last_unstale":"2023-11-06T21:59:40.896511+0000","last_undegraded":"2023-11-06T21:59:40.896511+0000","last_fullsized":"2023-11-06T21:59:40.896511+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879924+0000","last_change":"2023-11-06T21:59:40.413385+0000","last_active":"2023-11-06T21:59:40.879924+0000","last_peered":"2023-11-06T21:59:40.879924+0000","last_clean":"2023-11-06T21:59:40.879924+0000","last_became_active":"2023-11-06T21:59:40.412579+0000","last_became_peered":"2023-11-06T21:59:40.412579+0000","last_unstale":"2023-11-06T21:59:40.879924+0000","last_undegraded":"2023-11-06T21:59:40.879924+0000","last_fullsized":"2023-11-06T21:59:40.879924+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882792+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T21:59:40.882792+0000","last_peered":"2023-11-06T21:59:40.882792+0000","last_clean":"2023-11-06T21:59:40.882792+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T21:59:40.882792+0000","last_undegraded":"2023-11-06T21:59:40.882792+0000","last_fullsized":"2023-11-06T21:59:40.882792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.884231+0000","last_change":"2023-11-06T21:59:40.879304+0000","last_active":"2023-11-06T21:59:40.884231+0000","last_peered":"2023-11-06T21:59:40.884231+0000","last_clean":"2023-11-06T21:59:40.884231+0000","last_became_active":"2023-11-06T21:59:40.879011+0000","last_became_peered":"2023-11-06T21:59:40.879011+0000","last_unstale":"2023-11-06T21:59:40.884231+0000","last_undegraded":"2023-11-06T21:59:40.884231+0000","last_fullsized":"2023-11-06T21:59:40.884231+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882696+0000","last_change":"2023-11-06T21:59:40.409786+0000","last_active":"2023-11-06T21:59:40.882696+0000","last_peered":"2023-11-06T21:59:40.882696+0000","last_clean":"2023-11-06T21:59:40.882696+0000","last_became_active":"2023-11-06T21:59:40.409281+0000","last_became_peered":"2023-11-06T21:59:40.409281+0000","last_unstale":"2023-11-06T21:59:40.882696+0000","last_undegraded":"2023-11-06T21:59:40.882696+0000","last_fullsized":"2023-11-06T21:59:40.882696+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.884151+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T21:59:40.884151+0000","last_peered":"2023-11-06T21:59:40.884151+0000","last_clean":"2023-11-06T21:59:40.884151+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T21:59:40.884151+0000","last_undegraded":"2023-11-06T21:59:40.884151+0000","last_fullsized":"2023-11-06T21:59:40.884151+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575750,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307352,"kb_used_data":2064,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064488,"statfs":{"total":96636764160,"available":96322035712,"internally_reserved":0,"allocated":2113536,"data_stored":1079854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.505,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90500000000000003,"5min":0.997,"15min":1.002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73199999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93100000000000005,"5min":1.022,"15min":1.022},"last":0.65200000000000002}]},{"osd":1,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.55300000000000005,"5min":0.48899999999999999,"15min":0.42399999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.42499999999999999},"max":{"1min":0.88800000000000001,"5min":1.083,"15min":1.083},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.745,"15min":0.752},"min":{"1min":0.54500000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.94599999999999995,"5min":1.004,"15min":1.004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.753,"15min":0.75},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.45200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.0029999999999999},"last":0.71099999999999997}]},{"osd":3,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.751,"15min":0.73899999999999999},"min":{"1min":0.56699999999999995,"5min":0.48499999999999999,"15min":0.47799999999999998},"max":{"1min":0.95599999999999996,"5min":1.038,"15min":1.038},"last":0.623},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54000000000000004},"max":{"1min":1,"5min":1.046,"15min":1.046},"last":0.68500000000000005}]},{"osd":4,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.61099999999999999},"min":{"1min":0.47699999999999998,"5min":0.45000000000000001,"15min":0.377},"max":{"1min":0.91200000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47099999999999997,"5min":0.433,"15min":0.41899999999999998},"max":{"1min":0.79900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.503}]},{"osd":5,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.503,"5min":0.45200000000000001,"15min":0.41499999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.68999999999999995},"min":{"1min":0.51600000000000001,"5min":0.46600000000000003,"15min":0.438},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.58199999999999996}]},{"osd":6,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.66200000000000003,"5min":0.53200000000000003,"15min":0.495},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70699999999999996,"15min":0.71099999999999997},"min":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.48199999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.56299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834575750,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307344,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064496,"statfs":{"total":96636764160,"available":96322043904,"internally_reserved":0,"allocated":2105344,"data_stored":1073140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.40000000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.04},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71099999999999997,"15min":0.71199999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.37},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.0209999999999999},"last":0.87}]},{"osd":1,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83099999999999996,"5min":0.93600000000000005,"15min":0.99399999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.32600000000000001},"max":{"1min":0.91100000000000003,"5min":1.004,"15min":1.004},"last":1.046}]},{"osd":2,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95299999999999996,"5min":1.034,"15min":1.034},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94499999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.98799999999999999}]},{"osd":3,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":1.02},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76200000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.503},"max":{"1min":0.89700000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":1.028}]},{"osd":4,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.39000000000000001},"max":{"1min":0.84899999999999998,"5min":0.85199999999999998,"15min":0.94399999999999995},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64400000000000002,"15min":0.63300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.434},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.75600000000000001}]},{"osd":5,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.627},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.70199999999999996},"min":{"1min":0.499,"5min":0.498,"15min":0.46200000000000002},"max":{"1min":0.95599999999999996,"5min":0.98099999999999998,"15min":0.98699999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.44400000000000001},"max":{"1min":0.84099999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.48899999999999999},"max":{"1min":0.93899999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575750,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":2117632,"data_stored":1074732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97299999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0209999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.875,"5min":0.875,"15min":1.1850000000000001},"last":0.49099999999999999}]},{"osd":3,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.60399999999999998},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.50700000000000001}]},{"osd":4,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0269999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83799999999999997,"15min":0.84699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59399999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.81299999999999994},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.55700000000000005},"max":{"1min":1.02,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.046,"5min":1.1559999999999999,"15min":1.198},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.82399999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.107,"5min":1.1160000000000001,"15min":1.151},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.83699999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.51600000000000001},"max":{"1min":1.083,"5min":1.206,"15min":1.206},"last":0.68200000000000005}]},{"osd":7,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85299999999999998,"15min":0.86199999999999999},"min":{"1min":0.70899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1499999999999999,"5min":1.2070000000000001,"15min":1.21},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0980000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.747}]}]},{"osd":0,"up_from":13,"seq":55834575750,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303264,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068576,"statfs":{"total":96636764160,"available":96326221824,"internally_reserved":0,"allocated":2121728,"data_stored":1075286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66200000000000003,"5min":0.749,"15min":0.82899999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54000000000000004,"15min":0.53300000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71399999999999997,"5min":0.83599999999999997,"15min":0.83799999999999997},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.91700000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.376},"max":{"1min":0.78900000000000003,"5min":0.80400000000000005,"15min":0.89800000000000002},"last":0.53500000000000003}]},{"osd":3,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":0.78600000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.5},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.44900000000000001,"5min":0.436,"15min":0.40300000000000002},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.88200000000000001},"last":0.63500000000000001}]},{"osd":4,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.62,"5min":0.61299999999999999,"15min":0.57099999999999995},"max":{"1min":1.0489999999999999,"5min":1.0920000000000001,"15min":1.2649999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67700000000000005,"5min":0.57399999999999995,"15min":0.51700000000000002},"max":{"1min":1.03,"5min":1.0800000000000001,"15min":1.137},"last":0.91400000000000003}]},{"osd":5,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.86099999999999999,"15min":0.85399999999999998},"min":{"1min":0.66800000000000004,"5min":0.61899999999999999,"15min":0.57999999999999996},"max":{"1min":0.99199999999999999,"5min":1.1970000000000001,"15min":1.306},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.879,"5min":0.871,"15min":0.86099999999999999},"min":{"1min":0.69099999999999995,"5min":0.60699999999999998,"15min":0.54900000000000004},"max":{"1min":1.0309999999999999,"5min":1.121,"15min":1.2729999999999999},"last":1.014}]},{"osd":6,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83399999999999996},"min":{"1min":0.66800000000000004,"5min":0.59599999999999997,"15min":0.55000000000000004},"max":{"1min":0.95299999999999996,"5min":1.1279999999999999,"15min":1.1910000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.84699999999999998,"15min":0.84799999999999998},"min":{"1min":0.74199999999999999,"5min":0.56599999999999995,"15min":0.50700000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.228},"last":0.876}]},{"osd":7,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86399999999999999,"15min":0.86099999999999999},"min":{"1min":0.73399999999999999,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.101,"5min":1.1439999999999999,"15min":1.2210000000000001},"last":0.998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.86399999999999999,"15min":0.86299999999999999},"min":{"1min":0.61099999999999999,"5min":0.59299999999999997,"15min":0.56899999999999995},"max":{"1min":1.0860000000000001,"5min":1.129,"15min":1.337},"last":0.92800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575750,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303252,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068588,"statfs":{"total":96636764160,"available":96326234112,"internally_reserved":0,"allocated":2109440,"data_stored":1073198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.46200000000000002},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.28999999999999998},"max":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.309},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.32800000000000001},"max":{"1min":0.66900000000000004,"5min":0.84799999999999998,"15min":0.86299999999999999},"last":0.503}]},{"osd":1,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.432},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61899999999999999}]},{"osd":3,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51400000000000001,"15min":0.51500000000000001},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70299999999999996,"5min":0.77100000000000002,"15min":0.80000000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.44800000000000001,"5min":0.432,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.89700000000000002,"15min":1.0940000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.622,"15min":0.55500000000000005},"max":{"1min":1.075,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83699999999999997,"15min":0.81699999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1020000000000001,"5min":1.256,"15min":1.256},"last":0.56899999999999995}]},{"osd":5,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.81000000000000005},"min":{"1min":0.64400000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.86099999999999999,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.58299999999999996},"max":{"1min":1.0840000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.53200000000000003}]},{"osd":6,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.84199999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.51800000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.69299999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.145,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.88300000000000001,"15min":0.871},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.57299999999999995},"max":{"1min":1.173,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.752,"5min":0.57699999999999996,"15min":0.55900000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.69399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307340,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064500,"statfs":{"total":96636764160,"available":96322048000,"internally_reserved":0,"allocated":2101248,"data_stored":1073106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.33400000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.39300000000000002}]},{"osd":1,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.32400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.308},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.376},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.375},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.40899999999999997},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.436}]},{"osd":4,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86199999999999999,"15min":0.84399999999999997},"min":{"1min":0.73799999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.85999999999999999,"15min":0.84599999999999997},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.47999999999999998},"max":{"1min":1.0309999999999999,"5min":1.202,"15min":1.202},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.85699999999999998,"15min":0.82699999999999996},"min":{"1min":0.66000000000000003,"5min":0.59499999999999997,"15min":0.57399999999999995},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.67600000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.64800000000000002}]},{"osd":6,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.86899999999999999,"15min":0.83599999999999997},"min":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89500000000000002,"15min":0.86699999999999999},"min":{"1min":0.69399999999999995,"5min":0.66500000000000004,"15min":0.59999999999999998},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.81599999999999995}]},{"osd":7,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.87,"15min":0.85899999999999999},"min":{"1min":0.69299999999999995,"5min":0.58899999999999997,"15min":0.55500000000000005},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.92000000000000004,"15min":0.88100000000000001},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.56799999999999995},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.77900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575750,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303252,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068588,"statfs":{"total":96636764160,"available":96326234112,"internally_reserved":0,"allocated":2109440,"data_stored":1073730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.54300000000000004,"5min":0.50900000000000001,"15min":0.45000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.1659999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89800000000000002,"5min":0.999,"15min":1},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91000000000000003,"5min":0.93999999999999995,"15min":1.046},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.74199999999999999},"min":{"1min":0.64600000000000002,"5min":0.55800000000000005,"15min":0.44700000000000001},"max":{"1min":0.84799999999999998,"5min":1.0700000000000001,"15min":1.131},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.746},"min":{"1min":0.64800000000000002,"5min":0.50800000000000001,"15min":0.41099999999999998},"max":{"1min":0.95799999999999996,"5min":0.99299999999999999,"15min":1.014},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.80700000000000005},"min":{"1min":0.71299999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.008,"5min":1.093,"15min":1.093},"last":0.60899999999999999}]},{"osd":3,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73299999999999998,"15min":0.72199999999999998},"min":{"1min":0.65900000000000003,"5min":0.48099999999999998,"15min":0.46000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.181},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.60699999999999998,"5min":0.52700000000000002,"15min":0.47099999999999997},"max":{"1min":1.0009999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.61499999999999999},"min":{"1min":0.502,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.92100000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.44500000000000001,"5min":0.42999999999999999,"15min":0.36899999999999999},"max":{"1min":0.88700000000000001,"5min":0.94499999999999995,"15min":1.228},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.48299999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.81000000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.56200000000000006,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.04,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.47999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.085},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73699999999999999,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.503,"15min":0.496},"max":{"1min":1.004,"5min":1.0189999999999999,"15min":1.091},"last":0.57299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575751,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307348,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064492,"statfs":{"total":96636764160,"available":96322039808,"internally_reserved":0,"allocated":2109440,"data_stored":1073162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67500000000000004,"15min":0.67800000000000005},"min":{"1min":0.48999999999999999,"5min":0.38600000000000001,"15min":0.33600000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.008},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.46899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.98199999999999998,"15min":0.99299999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75,"15min":0.753},"min":{"1min":0.57899999999999996,"5min":0.375,"15min":0.375},"max":{"1min":0.92100000000000004,"5min":1.0009999999999999,"15min":1.2370000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.496,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.0629999999999999,"15min":1.2470000000000001},"last":0.57499999999999996}]},{"osd":2,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.58499999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94399999999999995,"5min":1.085,"15min":1.085},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.753},"min":{"1min":0.66700000000000004,"5min":0.55100000000000005,"15min":0.47199999999999998},"max":{"1min":0.95599999999999996,"5min":1.036,"15min":1.036},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.60499999999999998,"5min":0.49099999999999999,"15min":0.47399999999999998},"max":{"1min":0.88300000000000001,"5min":1.02,"15min":1.02},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.63500000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90400000000000003,"5min":1.093,"15min":1.093},"last":0.72099999999999997}]},{"osd":4,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.69899999999999995},"min":{"1min":0.53400000000000003,"5min":0.437,"15min":0.39300000000000002},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":0.997},"last":0.77100000000000002}]},{"osd":6,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66900000000000004,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93400000000000005,"5min":0.99199999999999999,"15min":0.99299999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.751,"15min":0.72599999999999998},"min":{"1min":0.60299999999999998,"5min":0.46100000000000002,"15min":0.41499999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0449999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68500000000000005,"15min":0.66400000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.05},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72699999999999998,"15min":0.70099999999999996},"min":{"1min":0.48899999999999999,"5min":0.45700000000000002,"15min":0.436},"max":{"1min":0.95699999999999996,"5min":1.014,"15min":1.0780000000000001},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T21:59:45.326 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T21:59:45.360 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-11-06T21:59:45.358+0000 7fa24cdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T21:59:45.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.520+0000 7f973e1ff700 1 Processor -- start 2023-11-06T21:59:45.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.521+0000 7f973e1ff700 1 -- start start 2023-11-06T21:59:45.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.522+0000 7f973e1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380c1840 0x7f97380a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.522+0000 7f973e1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f97380aa2d0 0x7f97380c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.522+0000 7f973e1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97381554a0 con 0x7f97380c6240 2023-11-06T21:59:45.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.522+0000 7f973e1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9738154960 con 0x7f97380aa2d0 2023-11-06T21:59:45.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.522+0000 7f973e1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9738155930 con 0x7f97380c1840 2023-11-06T21:59:45.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.522+0000 7f97377fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380c1840 0x7f97380a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.522+0000 7f9736ffd700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f97380aa2d0 0x7f97380c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.522+0000 7f97377fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380c1840 0x7f97380a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:42274/0 (socket says 172.21.15.33:42274) 2023-11-06T21:59:45.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.522+0000 7f97377fe700 1 -- 172.21.15.33:0/2584829359 learned_addr learned my addr 172.21.15.33:0/2584829359 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:45.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.523+0000 7f97367fc700 1 -- 172.21.15.33:0/2584829359 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683666687 0 0) 0x7f97381554a0 con 0x7f97380c6240 2023-11-06T21:59:45.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.523+0000 7f97367fc700 1 -- 172.21.15.33:0/2584829359 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97200033f0 con 0x7f97380c6240 2023-11-06T21:59:45.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.523+0000 7f97377fe700 1 -- 172.21.15.33:0/2584829359 >> v2:172.21.15.69:3300/0 conn(0x7f97380aa2d0 msgr2=0x7f97380c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:45.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.523+0000 7f97377fe700 1 --2- 172.21.15.33:0/2584829359 >> v2:172.21.15.69:3300/0 conn(0x7f97380aa2d0 0x7f97380c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.523+0000 7f97377fe700 1 -- 172.21.15.33:0/2584829359 >> v1:172.21.15.69:6789/0 conn(0x7f97380c6240 legacy=0x7f9738157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.523+0000 7f97377fe700 1 -- 172.21.15.33:0/2584829359 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9738154b70 con 0x7f97380c1840 2023-11-06T21:59:45.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.523+0000 7f9736ffd700 1 --2- 172.21.15.33:0/2584829359 >> v2:172.21.15.69:3300/0 conn(0x7f97380aa2d0 0x7f97380c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:45.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.523+0000 7f97377fe700 1 --2- 172.21.15.33:0/2584829359 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380c1840 0x7f97380a9de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f9728006fd0 tx=0x7f97280090f0).ready entity=mon.2 client_cookie=a2e40eb7c976dbd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:45.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f97367fc700 1 -- 172.21.15.33:0/2584829359 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972800f510 con 0x7f97380c1840 2023-11-06T21:59:45.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f97367fc700 1 -- 172.21.15.33:0/2584829359 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972800f670 con 0x7f97380c1840 2023-11-06T21:59:45.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f97367fc700 1 -- 172.21.15.33:0/2584829359 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972800f9f0 con 0x7f97380c1840 2023-11-06T21:59:45.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f973e1ff700 1 -- 172.21.15.33:0/2584829359 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380c1840 msgr2=0x7f97380a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f973e1ff700 1 --2- 172.21.15.33:0/2584829359 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380c1840 0x7f97380a9de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f9728006fd0 tx=0x7f97280090f0).stop 2023-11-06T21:59:45.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f973e1ff700 1 -- 172.21.15.33:0/2584829359 shutdown_connections 2023-11-06T21:59:45.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f973e1ff700 1 --2- 172.21.15.33:0/2584829359 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380c1840 0x7f97380a9de0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f973e1ff700 1 --2- 172.21.15.33:0/2584829359 >> v2:172.21.15.69:3300/0 conn(0x7f97380aa2d0 0x7f97380c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f973e1ff700 1 -- 172.21.15.33:0/2584829359 >> 172.21.15.33:0/2584829359 conn(0x7f973805ec80 msgr2=0x7f97380cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:45.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f973e1ff700 1 -- 172.21.15.33:0/2584829359 shutdown_connections 2023-11-06T21:59:45.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.524+0000 7f973e1ff700 1 -- 172.21.15.33:0/2584829359 wait complete. 2023-11-06T21:59:45.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.526+0000 7f973e1ff700 1 Processor -- start 2023-11-06T21:59:45.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.526+0000 7f973e1ff700 1 -- start start 2023-11-06T21:59:45.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f973e1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380aa2d0 0x7f97380c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f973e1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97380c1840 0x7f97380c2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f973e1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97380c6240 0x7f97381569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f973e1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380c2d80 con 0x7f97380c1840 2023-11-06T21:59:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f973e1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380c2ff0 con 0x7f97380c6240 2023-11-06T21:59:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f973e1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97380c32d0 con 0x7f97380aa2d0 2023-11-06T21:59:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f9736ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97380c1840 0x7f97380c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f9736ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97380c1840 0x7f97380c2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58902/0 (socket says 172.21.15.33:58902) 2023-11-06T21:59:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f9736ffd700 1 -- 172.21.15.33:0/724000397 learned_addr learned my addr 172.21.15.33:0/724000397 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f97377fe700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380aa2d0 0x7f97380c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.527+0000 7f9737fff700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97380c6240 0x7f97381569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f9736ffd700 1 -- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380aa2d0 msgr2=0x7f97380c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f9736ffd700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380aa2d0 0x7f97380c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f9736ffd700 1 -- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97380c6240 msgr2=0x7f97381569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f9736ffd700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97380c6240 0x7f97381569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f9736ffd700 1 -- 172.21.15.33:0/724000397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c0092a0 con 0x7f97380c1840 2023-11-06T21:59:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f9736ffd700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97380c1840 0x7f97380c2890 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f972c00e980 tx=0x7f972c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:45.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f971f7fe700 1 -- 172.21.15.33:0/724000397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c012600 con 0x7f97380c1840 2023-11-06T21:59:45.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f971f7fe700 1 -- 172.21.15.33:0/724000397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c012760 con 0x7f97380c1840 2023-11-06T21:59:45.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f973e1ff700 1 -- 172.21.15.33:0/724000397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9728009370 con 0x7f97380c1840 2023-11-06T21:59:45.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f971f7fe700 1 -- 172.21.15.33:0/724000397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c012ae0 con 0x7f97380c1840 2023-11-06T21:59:45.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.528+0000 7f973e1ff700 1 -- 172.21.15.33:0/724000397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97380c3890 con 0x7f97380c1840 2023-11-06T21:59:45.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.529+0000 7f973e1ff700 1 -- 172.21.15.33:0/724000397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973806c6c0 con 0x7f97380c1840 2023-11-06T21:59:45.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.530+0000 7f971f7fe700 1 -- 172.21.15.33:0/724000397 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f972c01b400 con 0x7f97380c1840 2023-11-06T21:59:45.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.531+0000 7f971f7fe700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f972405f270 0x7f9724061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.531+0000 7f971f7fe700 1 -- 172.21.15.33:0/724000397 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f972c07a8f0 con 0x7f97380c1840 2023-11-06T21:59:45.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.531+0000 7f97377fe700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f972405f270 0x7f9724061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.532+0000 7f97377fe700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f972405f270 0x7f9724061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f9728009130 tx=0x7f9728017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:45.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.535+0000 7f971f7fe700 1 -- 172.21.15.33:0/724000397 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f972c04b8f0 con 0x7f97380c1840 2023-11-06T21:59:45.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.712+0000 7f973e1ff700 1 -- 172.21.15.33:0/724000397 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9738063f20 con 0x7f972405f270 2023-11-06T21:59:45.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.722+0000 7f971f7fe700 1 -- 172.21.15.33:0/724000397 <== mgr.7025 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361460 (secure 0 0 0) 0x7f9738063f20 con 0x7f972405f270 2023-11-06T21:59:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:45.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 -- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f972405f270 msgr2=0x7f9724061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f972405f270 0x7f9724061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f9728009130 tx=0x7f9728017000).stop 2023-11-06T21:59:45.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 -- 172.21.15.33:0/724000397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97380c1840 msgr2=0x7f97380c2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97380c1840 0x7f97380c2890 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f972c00e980 tx=0x7f972c009060).stop 2023-11-06T21:59:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 -- 172.21.15.33:0/724000397 shutdown_connections 2023-11-06T21:59:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f972405f270 0x7f9724061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f97380aa2d0 0x7f97380c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97380c6240 0x7f97381569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 --2- 172.21.15.33:0/724000397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97380c1840 0x7f97380c2890 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 -- 172.21.15.33:0/724000397 >> 172.21.15.33:0/724000397 conn(0x7f973805ec80 msgr2=0x7f97380ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 -- 172.21.15.33:0/724000397 shutdown_connections 2023-11-06T21:59:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.727+0000 7f973e1ff700 1 -- 172.21.15.33:0/724000397 wait complete. 2023-11-06T21:59:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T21:59:45.747 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T21:59:44.331281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2442412,"kb_used_data":16492,"kb_used_omap":0,"kb_used_meta":2425856,"kb_avail":752532308,"statfs":{"total":773094113280,"available":770593083392,"internally_reserved":0,"allocated":16887808,"data_stored":8596208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2484076544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5037059,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-484,"num_write_kb":-5308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.995725"},"pg_stats":[{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907321+0000","last_change":"2023-11-06T21:59:40.898557+0000","last_active":"2023-11-06T21:59:40.907321+0000","last_peered":"2023-11-06T21:59:40.907321+0000","last_clean":"2023-11-06T21:59:40.907321+0000","last_became_active":"2023-11-06T21:59:40.898071+0000","last_became_peered":"2023-11-06T21:59:40.898071+0000","last_unstale":"2023-11-06T21:59:40.907321+0000","last_undegraded":"2023-11-06T21:59:40.907321+0000","last_fullsized":"2023-11-06T21:59:40.907321+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3f","version":"169'3","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.909120+0000","last_change":"2023-11-06T21:59:40.408459+0000","last_active":"2023-11-06T21:59:40.909120+0000","last_peered":"2023-11-06T21:59:40.909120+0000","last_clean":"2023-11-06T21:59:40.909120+0000","last_became_active":"2023-11-06T21:59:40.408127+0000","last_became_peered":"2023-11-06T21:59:40.408127+0000","last_unstale":"2023-11-06T21:59:40.909120+0000","last_undegraded":"2023-11-06T21:59:40.909120+0000","last_fullsized":"2023-11-06T21:59:40.909120+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.900961+0000","last_change":"2023-11-06T21:59:40.888973+0000","last_active":"2023-11-06T21:59:40.900961+0000","last_peered":"2023-11-06T21:59:40.900961+0000","last_clean":"2023-11-06T21:59:40.900961+0000","last_became_active":"2023-11-06T21:59:40.888001+0000","last_became_peered":"2023-11-06T21:59:40.888001+0000","last_unstale":"2023-11-06T21:59:40.900961+0000","last_undegraded":"2023-11-06T21:59:40.900961+0000","last_fullsized":"2023-11-06T21:59:40.900961+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882424+0000","last_change":"2023-11-06T21:59:40.413032+0000","last_active":"2023-11-06T21:59:40.882424+0000","last_peered":"2023-11-06T21:59:40.882424+0000","last_clean":"2023-11-06T21:59:40.882424+0000","last_became_active":"2023-11-06T21:59:40.412676+0000","last_became_peered":"2023-11-06T21:59:40.412676+0000","last_unstale":"2023-11-06T21:59:40.882424+0000","last_undegraded":"2023-11-06T21:59:40.882424+0000","last_fullsized":"2023-11-06T21:59:40.882424+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896380+0000","last_change":"2023-11-06T21:59:40.892101+0000","last_active":"2023-11-06T21:59:40.896380+0000","last_peered":"2023-11-06T21:59:40.896380+0000","last_clean":"2023-11-06T21:59:40.896380+0000","last_became_active":"2023-11-06T21:59:40.891699+0000","last_became_peered":"2023-11-06T21:59:40.891699+0000","last_unstale":"2023-11-06T21:59:40.896380+0000","last_undegraded":"2023-11-06T21:59:40.896380+0000","last_fullsized":"2023-11-06T21:59:40.896380+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.881559+0000","last_change":"2023-11-06T21:59:40.401569+0000","last_active":"2023-11-06T21:59:40.881559+0000","last_peered":"2023-11-06T21:59:40.881559+0000","last_clean":"2023-11-06T21:59:40.881559+0000","last_became_active":"2023-11-06T21:59:40.401019+0000","last_became_peered":"2023-11-06T21:59:40.401019+0000","last_unstale":"2023-11-06T21:59:40.881559+0000","last_undegraded":"2023-11-06T21:59:40.881559+0000","last_fullsized":"2023-11-06T21:59:40.881559+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896352+0000","last_change":"2023-11-06T21:59:40.891887+0000","last_active":"2023-11-06T21:59:40.896352+0000","last_peered":"2023-11-06T21:59:40.896352+0000","last_clean":"2023-11-06T21:59:40.896352+0000","last_became_active":"2023-11-06T21:59:40.888875+0000","last_became_peered":"2023-11-06T21:59:40.888875+0000","last_unstale":"2023-11-06T21:59:40.896352+0000","last_undegraded":"2023-11-06T21:59:40.896352+0000","last_fullsized":"2023-11-06T21:59:40.896352+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908353+0000","last_change":"2023-11-06T21:59:40.401201+0000","last_active":"2023-11-06T21:59:40.908353+0000","last_peered":"2023-11-06T21:59:40.908353+0000","last_clean":"2023-11-06T21:59:40.908353+0000","last_became_active":"2023-11-06T21:59:40.399541+0000","last_became_peered":"2023-11-06T21:59:40.399541+0000","last_unstale":"2023-11-06T21:59:40.908353+0000","last_undegraded":"2023-11-06T21:59:40.908353+0000","last_fullsized":"2023-11-06T21:59:40.908353+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908754+0000","last_change":"2023-11-06T21:59:40.903428+0000","last_active":"2023-11-06T21:59:40.908754+0000","last_peered":"2023-11-06T21:59:40.908754+0000","last_clean":"2023-11-06T21:59:40.908754+0000","last_became_active":"2023-11-06T21:59:40.903147+0000","last_became_peered":"2023-11-06T21:59:40.903147+0000","last_unstale":"2023-11-06T21:59:40.908754+0000","last_undegraded":"2023-11-06T21:59:40.908754+0000","last_fullsized":"2023-11-06T21:59:40.908754+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3b","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908333+0000","last_change":"2023-11-06T21:59:40.415502+0000","last_active":"2023-11-06T21:59:40.908333+0000","last_peered":"2023-11-06T21:59:40.908333+0000","last_clean":"2023-11-06T21:59:40.908333+0000","last_became_active":"2023-11-06T21:59:40.414166+0000","last_became_peered":"2023-11-06T21:59:40.414166+0000","last_unstale":"2023-11-06T21:59:40.908333+0000","last_undegraded":"2023-11-06T21:59:40.908333+0000","last_fullsized":"2023-11-06T21:59:40.908333+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908801+0000","last_change":"2023-11-06T21:59:40.903712+0000","last_active":"2023-11-06T21:59:40.908801+0000","last_peered":"2023-11-06T21:59:40.908801+0000","last_clean":"2023-11-06T21:59:40.908801+0000","last_became_active":"2023-11-06T21:59:40.902816+0000","last_became_peered":"2023-11-06T21:59:40.902816+0000","last_unstale":"2023-11-06T21:59:40.908801+0000","last_undegraded":"2023-11-06T21:59:40.908801+0000","last_fullsized":"2023-11-06T21:59:40.908801+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3a","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908756+0000","last_change":"2023-11-06T21:59:40.412482+0000","last_active":"2023-11-06T21:59:40.908756+0000","last_peered":"2023-11-06T21:59:40.908756+0000","last_clean":"2023-11-06T21:59:40.908756+0000","last_became_active":"2023-11-06T21:59:40.412144+0000","last_became_peered":"2023-11-06T21:59:40.412144+0000","last_unstale":"2023-11-06T21:59:40.908756+0000","last_undegraded":"2023-11-06T21:59:40.908756+0000","last_fullsized":"2023-11-06T21:59:40.908756+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.909140+0000","last_change":"2023-11-06T21:59:40.907038+0000","last_active":"2023-11-06T21:59:40.909140+0000","last_peered":"2023-11-06T21:59:40.909140+0000","last_clean":"2023-11-06T21:59:40.909140+0000","last_became_active":"2023-11-06T21:59:40.905589+0000","last_became_peered":"2023-11-06T21:59:40.905589+0000","last_unstale":"2023-11-06T21:59:40.909140+0000","last_undegraded":"2023-11-06T21:59:40.909140+0000","last_fullsized":"2023-11-06T21:59:40.909140+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879414+0000","last_change":"2023-11-06T21:59:40.404442+0000","last_active":"2023-11-06T21:59:40.879414+0000","last_peered":"2023-11-06T21:59:40.879414+0000","last_clean":"2023-11-06T21:59:40.879414+0000","last_became_active":"2023-11-06T21:59:40.404124+0000","last_became_peered":"2023-11-06T21:59:40.404124+0000","last_unstale":"2023-11-06T21:59:40.879414+0000","last_undegraded":"2023-11-06T21:59:40.879414+0000","last_fullsized":"2023-11-06T21:59:40.879414+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.895577+0000","last_change":"2023-11-06T21:59:40.889505+0000","last_active":"2023-11-06T21:59:40.895577+0000","last_peered":"2023-11-06T21:59:40.895577+0000","last_clean":"2023-11-06T21:59:40.895577+0000","last_became_active":"2023-11-06T21:59:40.889183+0000","last_became_peered":"2023-11-06T21:59:40.889183+0000","last_unstale":"2023-11-06T21:59:40.895577+0000","last_undegraded":"2023-11-06T21:59:40.895577+0000","last_fullsized":"2023-11-06T21:59:40.895577+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879975+0000","last_change":"2023-11-06T21:59:40.414173+0000","last_active":"2023-11-06T21:59:40.879975+0000","last_peered":"2023-11-06T21:59:40.879975+0000","last_clean":"2023-11-06T21:59:40.879975+0000","last_became_active":"2023-11-06T21:59:40.412912+0000","last_became_peered":"2023-11-06T21:59:40.412912+0000","last_unstale":"2023-11-06T21:59:40.879975+0000","last_undegraded":"2023-11-06T21:59:40.879975+0000","last_fullsized":"2023-11-06T21:59:40.879975+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.895532+0000","last_change":"2023-11-06T21:59:40.890003+0000","last_active":"2023-11-06T21:59:40.895532+0000","last_peered":"2023-11-06T21:59:40.895532+0000","last_clean":"2023-11-06T21:59:40.895532+0000","last_became_active":"2023-11-06T21:59:40.889447+0000","last_became_peered":"2023-11-06T21:59:40.889447+0000","last_unstale":"2023-11-06T21:59:40.895532+0000","last_undegraded":"2023-11-06T21:59:40.895532+0000","last_fullsized":"2023-11-06T21:59:40.895532+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.884076+0000","last_change":"2023-11-06T21:59:40.397578+0000","last_active":"2023-11-06T21:59:40.884076+0000","last_peered":"2023-11-06T21:59:40.884076+0000","last_clean":"2023-11-06T21:59:40.884076+0000","last_became_active":"2023-11-06T21:59:40.396898+0000","last_became_peered":"2023-11-06T21:59:40.396898+0000","last_unstale":"2023-11-06T21:59:40.884076+0000","last_undegraded":"2023-11-06T21:59:40.884076+0000","last_fullsized":"2023-11-06T21:59:40.884076+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896956+0000","last_change":"2023-11-06T21:59:40.892121+0000","last_active":"2023-11-06T21:59:40.896956+0000","last_peered":"2023-11-06T21:59:40.896956+0000","last_clean":"2023-11-06T21:59:40.896956+0000","last_became_active":"2023-11-06T21:59:40.891796+0000","last_became_peered":"2023-11-06T21:59:40.891796+0000","last_unstale":"2023-11-06T21:59:40.896956+0000","last_undegraded":"2023-11-06T21:59:40.896956+0000","last_fullsized":"2023-11-06T21:59:40.896956+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882577+0000","last_change":"2023-11-06T21:59:40.404012+0000","last_active":"2023-11-06T21:59:40.882577+0000","last_peered":"2023-11-06T21:59:40.882577+0000","last_clean":"2023-11-06T21:59:40.882577+0000","last_became_active":"2023-11-06T21:59:40.403699+0000","last_became_peered":"2023-11-06T21:59:40.403699+0000","last_unstale":"2023-11-06T21:59:40.882577+0000","last_undegraded":"2023-11-06T21:59:40.882577+0000","last_fullsized":"2023-11-06T21:59:40.882577+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896781+0000","last_change":"2023-11-06T21:59:40.891121+0000","last_active":"2023-11-06T21:59:40.896781+0000","last_peered":"2023-11-06T21:59:40.896781+0000","last_clean":"2023-11-06T21:59:40.896781+0000","last_became_active":"2023-11-06T21:59:40.890789+0000","last_became_peered":"2023-11-06T21:59:40.890789+0000","last_unstale":"2023-11-06T21:59:40.896781+0000","last_undegraded":"2023-11-06T21:59:40.896781+0000","last_fullsized":"2023-11-06T21:59:40.896781+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.883160+0000","last_change":"2023-11-06T21:59:40.404021+0000","last_active":"2023-11-06T21:59:40.883160+0000","last_peered":"2023-11-06T21:59:40.883160+0000","last_clean":"2023-11-06T21:59:40.883160+0000","last_became_active":"2023-11-06T21:59:40.403725+0000","last_became_peered":"2023-11-06T21:59:40.403725+0000","last_unstale":"2023-11-06T21:59:40.883160+0000","last_undegraded":"2023-11-06T21:59:40.883160+0000","last_fullsized":"2023-11-06T21:59:40.883160+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902111+0000","last_change":"2023-11-06T21:59:40.898421+0000","last_active":"2023-11-06T21:59:40.902111+0000","last_peered":"2023-11-06T21:59:40.902111+0000","last_clean":"2023-11-06T21:59:40.902111+0000","last_became_active":"2023-11-06T21:59:40.898098+0000","last_became_peered":"2023-11-06T21:59:40.898098+0000","last_unstale":"2023-11-06T21:59:40.902111+0000","last_undegraded":"2023-11-06T21:59:40.902111+0000","last_fullsized":"2023-11-06T21:59:40.902111+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.34","version":"169'4","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.912571+0000","last_change":"2023-11-06T21:59:40.411096+0000","last_active":"2023-11-06T21:59:40.912571+0000","last_peered":"2023-11-06T21:59:40.912571+0000","last_clean":"2023-11-06T21:59:40.912571+0000","last_became_active":"2023-11-06T21:59:40.410693+0000","last_became_peered":"2023-11-06T21:59:40.410693+0000","last_unstale":"2023-11-06T21:59:40.912571+0000","last_undegraded":"2023-11-06T21:59:40.912571+0000","last_fullsized":"2023-11-06T21:59:40.912571+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902492+0000","last_change":"2023-11-06T21:59:40.898318+0000","last_active":"2023-11-06T21:59:40.902492+0000","last_peered":"2023-11-06T21:59:40.902492+0000","last_clean":"2023-11-06T21:59:40.902492+0000","last_became_active":"2023-11-06T21:59:40.897995+0000","last_became_peered":"2023-11-06T21:59:40.897995+0000","last_unstale":"2023-11-06T21:59:40.902492+0000","last_undegraded":"2023-11-06T21:59:40.902492+0000","last_fullsized":"2023-11-06T21:59:40.902492+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.33","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.898602+0000","last_change":"2023-11-06T21:59:40.404856+0000","last_active":"2023-11-06T21:59:40.898602+0000","last_peered":"2023-11-06T21:59:40.898602+0000","last_clean":"2023-11-06T21:59:40.898602+0000","last_became_active":"2023-11-06T21:59:40.404521+0000","last_became_peered":"2023-11-06T21:59:40.404521+0000","last_unstale":"2023-11-06T21:59:40.898602+0000","last_undegraded":"2023-11-06T21:59:40.898602+0000","last_fullsized":"2023-11-06T21:59:40.898602+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897174+0000","last_change":"2023-11-06T21:59:40.888919+0000","last_active":"2023-11-06T21:59:40.897174+0000","last_peered":"2023-11-06T21:59:40.897174+0000","last_clean":"2023-11-06T21:59:40.897174+0000","last_became_active":"2023-11-06T21:59:40.888596+0000","last_became_peered":"2023-11-06T21:59:40.888596+0000","last_unstale":"2023-11-06T21:59:40.897174+0000","last_undegraded":"2023-11-06T21:59:40.897174+0000","last_fullsized":"2023-11-06T21:59:40.897174+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.873107+0000","last_change":"2023-11-06T21:59:40.401830+0000","last_active":"2023-11-06T21:59:40.873107+0000","last_peered":"2023-11-06T21:59:40.873107+0000","last_clean":"2023-11-06T21:59:40.873107+0000","last_became_active":"2023-11-06T21:59:40.400869+0000","last_became_peered":"2023-11-06T21:59:40.400869+0000","last_unstale":"2023-11-06T21:59:40.873107+0000","last_undegraded":"2023-11-06T21:59:40.873107+0000","last_fullsized":"2023-11-06T21:59:40.873107+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.891377+0000","last_change":"2023-11-06T21:59:40.886735+0000","last_active":"2023-11-06T21:59:40.891377+0000","last_peered":"2023-11-06T21:59:40.891377+0000","last_clean":"2023-11-06T21:59:40.891377+0000","last_became_active":"2023-11-06T21:59:40.884809+0000","last_became_peered":"2023-11-06T21:59:40.884809+0000","last_unstale":"2023-11-06T21:59:40.891377+0000","last_undegraded":"2023-11-06T21:59:40.891377+0000","last_fullsized":"2023-11-06T21:59:40.891377+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.874342+0000","last_change":"2023-11-06T21:59:40.407023+0000","last_active":"2023-11-06T21:59:40.874342+0000","last_peered":"2023-11-06T21:59:40.874342+0000","last_clean":"2023-11-06T21:59:40.874342+0000","last_became_active":"2023-11-06T21:59:40.406689+0000","last_became_peered":"2023-11-06T21:59:40.406689+0000","last_unstale":"2023-11-06T21:59:40.874342+0000","last_undegraded":"2023-11-06T21:59:40.874342+0000","last_fullsized":"2023-11-06T21:59:40.874342+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.889693+0000","last_change":"2023-11-06T21:59:40.886273+0000","last_active":"2023-11-06T21:59:40.889693+0000","last_peered":"2023-11-06T21:59:40.889693+0000","last_clean":"2023-11-06T21:59:40.889693+0000","last_became_active":"2023-11-06T21:59:40.885614+0000","last_became_peered":"2023-11-06T21:59:40.885614+0000","last_unstale":"2023-11-06T21:59:40.889693+0000","last_undegraded":"2023-11-06T21:59:40.889693+0000","last_fullsized":"2023-11-06T21:59:40.889693+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.30","version":"169'4","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.906411+0000","last_change":"2023-11-06T21:59:40.397662+0000","last_active":"2023-11-06T21:59:40.906411+0000","last_peered":"2023-11-06T21:59:40.906411+0000","last_clean":"2023-11-06T21:59:40.906411+0000","last_became_active":"2023-11-06T21:59:40.396926+0000","last_became_peered":"2023-11-06T21:59:40.396926+0000","last_unstale":"2023-11-06T21:59:40.906411+0000","last_undegraded":"2023-11-06T21:59:40.906411+0000","last_fullsized":"2023-11-06T21:59:40.906411+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.893388+0000","last_change":"2023-11-06T21:59:40.885581+0000","last_active":"2023-11-06T21:59:40.893388+0000","last_peered":"2023-11-06T21:59:40.893388+0000","last_clean":"2023-11-06T21:59:40.893388+0000","last_became_active":"2023-11-06T21:59:40.885279+0000","last_became_peered":"2023-11-06T21:59:40.885279+0000","last_unstale":"2023-11-06T21:59:40.893388+0000","last_undegraded":"2023-11-06T21:59:40.893388+0000","last_fullsized":"2023-11-06T21:59:40.893388+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.883892+0000","last_change":"2023-11-06T21:59:40.397928+0000","last_active":"2023-11-06T21:59:40.883892+0000","last_peered":"2023-11-06T21:59:40.883892+0000","last_clean":"2023-11-06T21:59:40.883892+0000","last_became_active":"2023-11-06T21:59:40.397632+0000","last_became_peered":"2023-11-06T21:59:40.397632+0000","last_unstale":"2023-11-06T21:59:40.883892+0000","last_undegraded":"2023-11-06T21:59:40.883892+0000","last_fullsized":"2023-11-06T21:59:40.883892+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899478+0000","last_change":"2023-11-06T21:59:40.893670+0000","last_active":"2023-11-06T21:59:40.899478+0000","last_peered":"2023-11-06T21:59:40.899478+0000","last_clean":"2023-11-06T21:59:40.899478+0000","last_became_active":"2023-11-06T21:59:40.893401+0000","last_became_peered":"2023-11-06T21:59:40.893401+0000","last_unstale":"2023-11-06T21:59:40.899478+0000","last_undegraded":"2023-11-06T21:59:40.899478+0000","last_fullsized":"2023-11-06T21:59:40.899478+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.876048+0000","last_change":"2023-11-06T21:59:40.396309+0000","last_active":"2023-11-06T21:59:40.876048+0000","last_peered":"2023-11-06T21:59:40.876048+0000","last_clean":"2023-11-06T21:59:40.876048+0000","last_became_active":"2023-11-06T21:59:40.395455+0000","last_became_peered":"2023-11-06T21:59:40.395455+0000","last_unstale":"2023-11-06T21:59:40.876048+0000","last_undegraded":"2023-11-06T21:59:40.876048+0000","last_fullsized":"2023-11-06T21:59:40.876048+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896852+0000","last_change":"2023-11-06T21:59:40.891043+0000","last_active":"2023-11-06T21:59:40.896852+0000","last_peered":"2023-11-06T21:59:40.896852+0000","last_clean":"2023-11-06T21:59:40.896852+0000","last_became_active":"2023-11-06T21:59:40.890751+0000","last_became_peered":"2023-11-06T21:59:40.890751+0000","last_unstale":"2023-11-06T21:59:40.896852+0000","last_undegraded":"2023-11-06T21:59:40.896852+0000","last_fullsized":"2023-11-06T21:59:40.896852+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897894+0000","last_change":"2023-11-06T21:59:40.404941+0000","last_active":"2023-11-06T21:59:40.897894+0000","last_peered":"2023-11-06T21:59:40.897894+0000","last_clean":"2023-11-06T21:59:40.897894+0000","last_became_active":"2023-11-06T21:59:40.404615+0000","last_became_peered":"2023-11-06T21:59:40.404615+0000","last_unstale":"2023-11-06T21:59:40.897894+0000","last_undegraded":"2023-11-06T21:59:40.897894+0000","last_fullsized":"2023-11-06T21:59:40.897894+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908775+0000","last_change":"2023-11-06T21:59:40.902954+0000","last_active":"2023-11-06T21:59:40.908775+0000","last_peered":"2023-11-06T21:59:40.908775+0000","last_clean":"2023-11-06T21:59:40.908775+0000","last_became_active":"2023-11-06T21:59:40.902683+0000","last_became_peered":"2023-11-06T21:59:40.902683+0000","last_unstale":"2023-11-06T21:59:40.908775+0000","last_undegraded":"2023-11-06T21:59:40.908775+0000","last_fullsized":"2023-11-06T21:59:40.908775+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.875744+0000","last_change":"2023-11-06T21:59:40.401395+0000","last_active":"2023-11-06T21:59:40.875744+0000","last_peered":"2023-11-06T21:59:40.875744+0000","last_clean":"2023-11-06T21:59:40.875744+0000","last_became_active":"2023-11-06T21:59:40.399769+0000","last_became_peered":"2023-11-06T21:59:40.399769+0000","last_unstale":"2023-11-06T21:59:40.875744+0000","last_undegraded":"2023-11-06T21:59:40.875744+0000","last_fullsized":"2023-11-06T21:59:40.875744+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.905087+0000","last_change":"2023-11-06T21:59:40.900645+0000","last_active":"2023-11-06T21:59:40.905087+0000","last_peered":"2023-11-06T21:59:40.905087+0000","last_clean":"2023-11-06T21:59:40.905087+0000","last_became_active":"2023-11-06T21:59:40.899407+0000","last_became_peered":"2023-11-06T21:59:40.899407+0000","last_unstale":"2023-11-06T21:59:40.905087+0000","last_undegraded":"2023-11-06T21:59:40.905087+0000","last_fullsized":"2023-11-06T21:59:40.905087+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2b","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.909846+0000","last_change":"2023-11-06T21:59:40.407004+0000","last_active":"2023-11-06T21:59:40.909846+0000","last_peered":"2023-11-06T21:59:40.909846+0000","last_clean":"2023-11-06T21:59:40.909846+0000","last_became_active":"2023-11-06T21:59:40.406732+0000","last_became_peered":"2023-11-06T21:59:40.406732+0000","last_unstale":"2023-11-06T21:59:40.909846+0000","last_undegraded":"2023-11-06T21:59:40.909846+0000","last_fullsized":"2023-11-06T21:59:40.909846+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899301+0000","last_change":"2023-11-06T21:59:40.895050+0000","last_active":"2023-11-06T21:59:40.899301+0000","last_peered":"2023-11-06T21:59:40.899301+0000","last_clean":"2023-11-06T21:59:40.899301+0000","last_became_active":"2023-11-06T21:59:40.894756+0000","last_became_peered":"2023-11-06T21:59:40.894756+0000","last_unstale":"2023-11-06T21:59:40.899301+0000","last_undegraded":"2023-11-06T21:59:40.899301+0000","last_fullsized":"2023-11-06T21:59:40.899301+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882543+0000","last_change":"2023-11-06T21:59:40.406414+0000","last_active":"2023-11-06T21:59:40.882543+0000","last_peered":"2023-11-06T21:59:40.882543+0000","last_clean":"2023-11-06T21:59:40.882543+0000","last_became_active":"2023-11-06T21:59:40.405739+0000","last_became_peered":"2023-11-06T21:59:40.405739+0000","last_unstale":"2023-11-06T21:59:40.882543+0000","last_undegraded":"2023-11-06T21:59:40.882543+0000","last_fullsized":"2023-11-06T21:59:40.882543+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902523+0000","last_change":"2023-11-06T21:59:40.900030+0000","last_active":"2023-11-06T21:59:40.902523+0000","last_peered":"2023-11-06T21:59:40.902523+0000","last_clean":"2023-11-06T21:59:40.902523+0000","last_became_active":"2023-11-06T21:59:40.899469+0000","last_became_peered":"2023-11-06T21:59:40.899469+0000","last_unstale":"2023-11-06T21:59:40.902523+0000","last_undegraded":"2023-11-06T21:59:40.902523+0000","last_fullsized":"2023-11-06T21:59:40.902523+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.887115+0000","last_change":"2023-11-06T21:59:40.404196+0000","last_active":"2023-11-06T21:59:40.887115+0000","last_peered":"2023-11-06T21:59:40.887115+0000","last_clean":"2023-11-06T21:59:40.887115+0000","last_became_active":"2023-11-06T21:59:40.403920+0000","last_became_peered":"2023-11-06T21:59:40.403920+0000","last_unstale":"2023-11-06T21:59:40.887115+0000","last_undegraded":"2023-11-06T21:59:40.887115+0000","last_fullsized":"2023-11-06T21:59:40.887115+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.895601+0000","last_change":"2023-11-06T21:59:40.889599+0000","last_active":"2023-11-06T21:59:40.895601+0000","last_peered":"2023-11-06T21:59:40.895601+0000","last_clean":"2023-11-06T21:59:40.895601+0000","last_became_active":"2023-11-06T21:59:40.889253+0000","last_became_peered":"2023-11-06T21:59:40.889253+0000","last_unstale":"2023-11-06T21:59:40.895601+0000","last_undegraded":"2023-11-06T21:59:40.895601+0000","last_fullsized":"2023-11-06T21:59:40.895601+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.874234+0000","last_change":"2023-11-06T21:59:40.406559+0000","last_active":"2023-11-06T21:59:40.874234+0000","last_peered":"2023-11-06T21:59:40.874234+0000","last_clean":"2023-11-06T21:59:40.874234+0000","last_became_active":"2023-11-06T21:59:40.405408+0000","last_became_peered":"2023-11-06T21:59:40.405408+0000","last_unstale":"2023-11-06T21:59:40.874234+0000","last_undegraded":"2023-11-06T21:59:40.874234+0000","last_fullsized":"2023-11-06T21:59:40.874234+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896395+0000","last_change":"2023-11-06T21:59:40.891547+0000","last_active":"2023-11-06T21:59:40.896395+0000","last_peered":"2023-11-06T21:59:40.896395+0000","last_clean":"2023-11-06T21:59:40.896395+0000","last_became_active":"2023-11-06T21:59:40.891179+0000","last_became_peered":"2023-11-06T21:59:40.891179+0000","last_unstale":"2023-11-06T21:59:40.896395+0000","last_undegraded":"2023-11-06T21:59:40.896395+0000","last_fullsized":"2023-11-06T21:59:40.896395+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.883412+0000","last_change":"2023-11-06T21:59:40.405498+0000","last_active":"2023-11-06T21:59:40.883412+0000","last_peered":"2023-11-06T21:59:40.883412+0000","last_clean":"2023-11-06T21:59:40.883412+0000","last_became_active":"2023-11-06T21:59:40.405201+0000","last_became_peered":"2023-11-06T21:59:40.405201+0000","last_unstale":"2023-11-06T21:59:40.883412+0000","last_undegraded":"2023-11-06T21:59:40.883412+0000","last_fullsized":"2023-11-06T21:59:40.883412+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899503+0000","last_change":"2023-11-06T21:59:40.894174+0000","last_active":"2023-11-06T21:59:40.899503+0000","last_peered":"2023-11-06T21:59:40.899503+0000","last_clean":"2023-11-06T21:59:40.899503+0000","last_became_active":"2023-11-06T21:59:40.893924+0000","last_became_peered":"2023-11-06T21:59:40.893924+0000","last_unstale":"2023-11-06T21:59:40.899503+0000","last_undegraded":"2023-11-06T21:59:40.899503+0000","last_fullsized":"2023-11-06T21:59:40.899503+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882699+0000","last_change":"2023-11-06T21:59:40.403965+0000","last_active":"2023-11-06T21:59:40.882699+0000","last_peered":"2023-11-06T21:59:40.882699+0000","last_clean":"2023-11-06T21:59:40.882699+0000","last_became_active":"2023-11-06T21:59:40.403290+0000","last_became_peered":"2023-11-06T21:59:40.403290+0000","last_unstale":"2023-11-06T21:59:40.882699+0000","last_undegraded":"2023-11-06T21:59:40.882699+0000","last_fullsized":"2023-11-06T21:59:40.882699+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899428+0000","last_change":"2023-11-06T21:59:40.895238+0000","last_active":"2023-11-06T21:59:40.899428+0000","last_peered":"2023-11-06T21:59:40.899428+0000","last_clean":"2023-11-06T21:59:40.899428+0000","last_became_active":"2023-11-06T21:59:40.894813+0000","last_became_peered":"2023-11-06T21:59:40.894813+0000","last_unstale":"2023-11-06T21:59:40.899428+0000","last_undegraded":"2023-11-06T21:59:40.899428+0000","last_fullsized":"2023-11-06T21:59:40.899428+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.872547+0000","last_change":"2023-11-06T21:59:40.401568+0000","last_active":"2023-11-06T21:59:40.872547+0000","last_peered":"2023-11-06T21:59:40.872547+0000","last_clean":"2023-11-06T21:59:40.872547+0000","last_became_active":"2023-11-06T21:59:40.401289+0000","last_became_peered":"2023-11-06T21:59:40.401289+0000","last_unstale":"2023-11-06T21:59:40.872547+0000","last_undegraded":"2023-11-06T21:59:40.872547+0000","last_fullsized":"2023-11-06T21:59:40.872547+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907952+0000","last_change":"2023-11-06T21:59:40.901564+0000","last_active":"2023-11-06T21:59:40.907952+0000","last_peered":"2023-11-06T21:59:40.907952+0000","last_clean":"2023-11-06T21:59:40.907952+0000","last_became_active":"2023-11-06T21:59:40.896678+0000","last_became_peered":"2023-11-06T21:59:40.896678+0000","last_unstale":"2023-11-06T21:59:40.907952+0000","last_undegraded":"2023-11-06T21:59:40.907952+0000","last_fullsized":"2023-11-06T21:59:40.907952+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.890213+0000","last_change":"2023-11-06T21:59:40.410065+0000","last_active":"2023-11-06T21:59:40.890213+0000","last_peered":"2023-11-06T21:59:40.890213+0000","last_clean":"2023-11-06T21:59:40.890213+0000","last_became_active":"2023-11-06T21:59:40.409787+0000","last_became_peered":"2023-11-06T21:59:40.409787+0000","last_unstale":"2023-11-06T21:59:40.890213+0000","last_undegraded":"2023-11-06T21:59:40.890213+0000","last_fullsized":"2023-11-06T21:59:40.890213+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.893389+0000","last_change":"2023-11-06T21:59:40.889735+0000","last_active":"2023-11-06T21:59:40.893389+0000","last_peered":"2023-11-06T21:59:40.893389+0000","last_clean":"2023-11-06T21:59:40.893389+0000","last_became_active":"2023-11-06T21:59:40.889423+0000","last_became_peered":"2023-11-06T21:59:40.889423+0000","last_unstale":"2023-11-06T21:59:40.893389+0000","last_undegraded":"2023-11-06T21:59:40.893389+0000","last_fullsized":"2023-11-06T21:59:40.893389+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882648+0000","last_change":"2023-11-06T21:59:40.404442+0000","last_active":"2023-11-06T21:59:40.882648+0000","last_peered":"2023-11-06T21:59:40.882648+0000","last_clean":"2023-11-06T21:59:40.882648+0000","last_became_active":"2023-11-06T21:59:40.404143+0000","last_became_peered":"2023-11-06T21:59:40.404143+0000","last_unstale":"2023-11-06T21:59:40.882648+0000","last_undegraded":"2023-11-06T21:59:40.882648+0000","last_fullsized":"2023-11-06T21:59:40.882648+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899289+0000","last_change":"2023-11-06T21:59:40.894443+0000","last_active":"2023-11-06T21:59:40.899289+0000","last_peered":"2023-11-06T21:59:40.899289+0000","last_clean":"2023-11-06T21:59:40.899289+0000","last_became_active":"2023-11-06T21:59:40.894140+0000","last_became_peered":"2023-11-06T21:59:40.894140+0000","last_unstale":"2023-11-06T21:59:40.899289+0000","last_undegraded":"2023-11-06T21:59:40.899289+0000","last_fullsized":"2023-11-06T21:59:40.899289+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882495+0000","last_change":"2023-11-06T21:59:40.409168+0000","last_active":"2023-11-06T21:59:40.882495+0000","last_peered":"2023-11-06T21:59:40.882495+0000","last_clean":"2023-11-06T21:59:40.882495+0000","last_became_active":"2023-11-06T21:59:40.406138+0000","last_became_peered":"2023-11-06T21:59:40.406138+0000","last_unstale":"2023-11-06T21:59:40.882495+0000","last_undegraded":"2023-11-06T21:59:40.882495+0000","last_fullsized":"2023-11-06T21:59:40.882495+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908701+0000","last_change":"2023-11-06T21:59:40.900931+0000","last_active":"2023-11-06T21:59:40.908701+0000","last_peered":"2023-11-06T21:59:40.908701+0000","last_clean":"2023-11-06T21:59:40.908701+0000","last_became_active":"2023-11-06T21:59:40.900699+0000","last_became_peered":"2023-11-06T21:59:40.900699+0000","last_unstale":"2023-11-06T21:59:40.908701+0000","last_undegraded":"2023-11-06T21:59:40.908701+0000","last_fullsized":"2023-11-06T21:59:40.908701+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.887205+0000","last_change":"2023-11-06T21:59:40.396742+0000","last_active":"2023-11-06T21:59:40.887205+0000","last_peered":"2023-11-06T21:59:40.887205+0000","last_clean":"2023-11-06T21:59:40.887205+0000","last_became_active":"2023-11-06T21:59:40.396434+0000","last_became_peered":"2023-11-06T21:59:40.396434+0000","last_unstale":"2023-11-06T21:59:40.887205+0000","last_undegraded":"2023-11-06T21:59:40.887205+0000","last_fullsized":"2023-11-06T21:59:40.887205+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907411+0000","last_change":"2023-11-06T21:59:40.900466+0000","last_active":"2023-11-06T21:59:40.907411+0000","last_peered":"2023-11-06T21:59:40.907411+0000","last_clean":"2023-11-06T21:59:40.907411+0000","last_became_active":"2023-11-06T21:59:40.900151+0000","last_became_peered":"2023-11-06T21:59:40.900151+0000","last_unstale":"2023-11-06T21:59:40.907411+0000","last_undegraded":"2023-11-06T21:59:40.907411+0000","last_fullsized":"2023-11-06T21:59:40.907411+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879328+0000","last_change":"2023-11-06T21:59:40.411171+0000","last_active":"2023-11-06T21:59:40.879328+0000","last_peered":"2023-11-06T21:59:40.879328+0000","last_clean":"2023-11-06T21:59:40.879328+0000","last_became_active":"2023-11-06T21:59:40.410909+0000","last_became_peered":"2023-11-06T21:59:40.410909+0000","last_unstale":"2023-11-06T21:59:40.879328+0000","last_undegraded":"2023-11-06T21:59:40.879328+0000","last_fullsized":"2023-11-06T21:59:40.879328+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.906614+0000","last_change":"2023-11-06T21:59:40.898377+0000","last_active":"2023-11-06T21:59:40.906614+0000","last_peered":"2023-11-06T21:59:40.906614+0000","last_clean":"2023-11-06T21:59:40.906614+0000","last_became_active":"2023-11-06T21:59:40.897661+0000","last_became_peered":"2023-11-06T21:59:40.897661+0000","last_unstale":"2023-11-06T21:59:40.906614+0000","last_undegraded":"2023-11-06T21:59:40.906614+0000","last_fullsized":"2023-11-06T21:59:40.906614+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1f","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902417+0000","last_change":"2023-11-06T21:59:40.397779+0000","last_active":"2023-11-06T21:59:40.902417+0000","last_peered":"2023-11-06T21:59:40.902417+0000","last_clean":"2023-11-06T21:59:40.902417+0000","last_became_active":"2023-11-06T21:59:40.397274+0000","last_became_peered":"2023-11-06T21:59:40.397274+0000","last_unstale":"2023-11-06T21:59:40.902417+0000","last_undegraded":"2023-11-06T21:59:40.902417+0000","last_fullsized":"2023-11-06T21:59:40.902417+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902241+0000","last_change":"2023-11-06T21:59:40.898423+0000","last_active":"2023-11-06T21:59:40.902241+0000","last_peered":"2023-11-06T21:59:40.902241+0000","last_clean":"2023-11-06T21:59:40.902241+0000","last_became_active":"2023-11-06T21:59:40.898136+0000","last_became_peered":"2023-11-06T21:59:40.898136+0000","last_unstale":"2023-11-06T21:59:40.902241+0000","last_undegraded":"2023-11-06T21:59:40.902241+0000","last_fullsized":"2023-11-06T21:59:40.902241+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.1e","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.904579+0000","last_change":"2023-11-06T21:59:40.401361+0000","last_active":"2023-11-06T21:59:40.904579+0000","last_peered":"2023-11-06T21:59:40.904579+0000","last_clean":"2023-11-06T21:59:40.904579+0000","last_became_active":"2023-11-06T21:59:40.401032+0000","last_became_peered":"2023-11-06T21:59:40.401032+0000","last_unstale":"2023-11-06T21:59:40.904579+0000","last_undegraded":"2023-11-06T21:59:40.904579+0000","last_fullsized":"2023-11-06T21:59:40.904579+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907738+0000","last_change":"2023-11-06T21:59:40.905442+0000","last_active":"2023-11-06T21:59:40.907738+0000","last_peered":"2023-11-06T21:59:40.907738+0000","last_clean":"2023-11-06T21:59:40.907738+0000","last_became_active":"2023-11-06T21:59:40.905087+0000","last_became_peered":"2023-11-06T21:59:40.905087+0000","last_unstale":"2023-11-06T21:59:40.907738+0000","last_undegraded":"2023-11-06T21:59:40.907738+0000","last_fullsized":"2023-11-06T21:59:40.907738+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1d","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908006+0000","last_change":"2023-11-06T21:59:40.413934+0000","last_active":"2023-11-06T21:59:40.908006+0000","last_peered":"2023-11-06T21:59:40.908006+0000","last_clean":"2023-11-06T21:59:40.908006+0000","last_became_active":"2023-11-06T21:59:40.413663+0000","last_became_peered":"2023-11-06T21:59:40.413663+0000","last_unstale":"2023-11-06T21:59:40.908006+0000","last_undegraded":"2023-11-06T21:59:40.908006+0000","last_fullsized":"2023-11-06T21:59:40.908006+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.898951+0000","last_change":"2023-11-06T21:59:40.895397+0000","last_active":"2023-11-06T21:59:40.898951+0000","last_peered":"2023-11-06T21:59:40.898951+0000","last_clean":"2023-11-06T21:59:40.898951+0000","last_became_active":"2023-11-06T21:59:40.895097+0000","last_became_peered":"2023-11-06T21:59:40.895097+0000","last_unstale":"2023-11-06T21:59:40.898951+0000","last_undegraded":"2023-11-06T21:59:40.898951+0000","last_fullsized":"2023-11-06T21:59:40.898951+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879723+0000","last_change":"2023-11-06T21:59:40.397238+0000","last_active":"2023-11-06T21:59:40.879723+0000","last_peered":"2023-11-06T21:59:40.879723+0000","last_clean":"2023-11-06T21:59:40.879723+0000","last_became_active":"2023-11-06T21:59:40.395006+0000","last_became_peered":"2023-11-06T21:59:40.395006+0000","last_unstale":"2023-11-06T21:59:40.879723+0000","last_undegraded":"2023-11-06T21:59:40.879723+0000","last_fullsized":"2023-11-06T21:59:40.879723+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897556+0000","last_change":"2023-11-06T21:59:40.888847+0000","last_active":"2023-11-06T21:59:40.897556+0000","last_peered":"2023-11-06T21:59:40.897556+0000","last_clean":"2023-11-06T21:59:40.897556+0000","last_became_active":"2023-11-06T21:59:40.888582+0000","last_became_peered":"2023-11-06T21:59:40.888582+0000","last_unstale":"2023-11-06T21:59:40.897556+0000","last_undegraded":"2023-11-06T21:59:40.897556+0000","last_fullsized":"2023-11-06T21:59:40.897556+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.881097+0000","last_change":"2023-11-06T21:59:40.411815+0000","last_active":"2023-11-06T21:59:40.881097+0000","last_peered":"2023-11-06T21:59:40.881097+0000","last_clean":"2023-11-06T21:59:40.881097+0000","last_became_active":"2023-11-06T21:59:40.411177+0000","last_became_peered":"2023-11-06T21:59:40.411177+0000","last_unstale":"2023-11-06T21:59:40.881097+0000","last_undegraded":"2023-11-06T21:59:40.881097+0000","last_fullsized":"2023-11-06T21:59:40.881097+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907277+0000","last_change":"2023-11-06T21:59:40.899751+0000","last_active":"2023-11-06T21:59:40.907277+0000","last_peered":"2023-11-06T21:59:40.907277+0000","last_clean":"2023-11-06T21:59:40.907277+0000","last_became_active":"2023-11-06T21:59:40.898435+0000","last_became_peered":"2023-11-06T21:59:40.898435+0000","last_unstale":"2023-11-06T21:59:40.907277+0000","last_undegraded":"2023-11-06T21:59:40.907277+0000","last_fullsized":"2023-11-06T21:59:40.907277+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.874450+0000","last_change":"2023-11-06T21:59:40.407711+0000","last_active":"2023-11-06T21:59:40.874450+0000","last_peered":"2023-11-06T21:59:40.874450+0000","last_clean":"2023-11-06T21:59:40.874450+0000","last_became_active":"2023-11-06T21:59:40.407411+0000","last_became_peered":"2023-11-06T21:59:40.407411+0000","last_unstale":"2023-11-06T21:59:40.874450+0000","last_undegraded":"2023-11-06T21:59:40.874450+0000","last_fullsized":"2023-11-06T21:59:40.874450+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897650+0000","last_change":"2023-11-06T21:59:40.894215+0000","last_active":"2023-11-06T21:59:40.897650+0000","last_peered":"2023-11-06T21:59:40.897650+0000","last_clean":"2023-11-06T21:59:40.897650+0000","last_became_active":"2023-11-06T21:59:40.893836+0000","last_became_peered":"2023-11-06T21:59:40.893836+0000","last_unstale":"2023-11-06T21:59:40.897650+0000","last_undegraded":"2023-11-06T21:59:40.897650+0000","last_fullsized":"2023-11-06T21:59:40.897650+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882303+0000","last_change":"2023-11-06T21:59:40.404318+0000","last_active":"2023-11-06T21:59:40.882303+0000","last_peered":"2023-11-06T21:59:40.882303+0000","last_clean":"2023-11-06T21:59:40.882303+0000","last_became_active":"2023-11-06T21:59:40.403389+0000","last_became_peered":"2023-11-06T21:59:40.403389+0000","last_unstale":"2023-11-06T21:59:40.882303+0000","last_undegraded":"2023-11-06T21:59:40.882303+0000","last_fullsized":"2023-11-06T21:59:40.882303+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907197+0000","last_change":"2023-11-06T21:59:40.899151+0000","last_active":"2023-11-06T21:59:40.907197+0000","last_peered":"2023-11-06T21:59:40.907197+0000","last_clean":"2023-11-06T21:59:40.907197+0000","last_became_active":"2023-11-06T21:59:40.898934+0000","last_became_peered":"2023-11-06T21:59:40.898934+0000","last_unstale":"2023-11-06T21:59:40.907197+0000","last_undegraded":"2023-11-06T21:59:40.907197+0000","last_fullsized":"2023-11-06T21:59:40.907197+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.875758+0000","last_change":"2023-11-06T21:59:40.407923+0000","last_active":"2023-11-06T21:59:40.875758+0000","last_peered":"2023-11-06T21:59:40.875758+0000","last_clean":"2023-11-06T21:59:40.875758+0000","last_became_active":"2023-11-06T21:59:40.406915+0000","last_became_peered":"2023-11-06T21:59:40.406915+0000","last_unstale":"2023-11-06T21:59:40.875758+0000","last_undegraded":"2023-11-06T21:59:40.875758+0000","last_fullsized":"2023-11-06T21:59:40.875758+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.905042+0000","last_change":"2023-11-06T21:59:40.899319+0000","last_active":"2023-11-06T21:59:40.905042+0000","last_peered":"2023-11-06T21:59:40.905042+0000","last_clean":"2023-11-06T21:59:40.905042+0000","last_became_active":"2023-11-06T21:59:40.897015+0000","last_became_peered":"2023-11-06T21:59:40.897015+0000","last_unstale":"2023-11-06T21:59:40.905042+0000","last_undegraded":"2023-11-06T21:59:40.905042+0000","last_fullsized":"2023-11-06T21:59:40.905042+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.f","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.903092+0000","last_change":"2023-11-06T21:59:40.409729+0000","last_active":"2023-11-06T21:59:40.903092+0000","last_peered":"2023-11-06T21:59:40.903092+0000","last_clean":"2023-11-06T21:59:40.903092+0000","last_became_active":"2023-11-06T21:59:40.409439+0000","last_became_peered":"2023-11-06T21:59:40.409439+0000","last_unstale":"2023-11-06T21:59:40.903092+0000","last_undegraded":"2023-11-06T21:59:40.903092+0000","last_fullsized":"2023-11-06T21:59:40.903092+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899642+0000","last_change":"2023-11-06T21:59:40.888966+0000","last_active":"2023-11-06T21:59:40.899642+0000","last_peered":"2023-11-06T21:59:40.899642+0000","last_clean":"2023-11-06T21:59:40.899642+0000","last_became_active":"2023-11-06T21:59:40.888692+0000","last_became_peered":"2023-11-06T21:59:40.888692+0000","last_unstale":"2023-11-06T21:59:40.899642+0000","last_undegraded":"2023-11-06T21:59:40.899642+0000","last_fullsized":"2023-11-06T21:59:40.899642+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.e","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.903172+0000","last_change":"2023-11-06T21:59:40.407412+0000","last_active":"2023-11-06T21:59:40.903172+0000","last_peered":"2023-11-06T21:59:40.903172+0000","last_clean":"2023-11-06T21:59:40.903172+0000","last_became_active":"2023-11-06T21:59:40.405257+0000","last_became_peered":"2023-11-06T21:59:40.405257+0000","last_unstale":"2023-11-06T21:59:40.903172+0000","last_undegraded":"2023-11-06T21:59:40.903172+0000","last_fullsized":"2023-11-06T21:59:40.903172+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.905479+0000","last_change":"2023-11-06T21:59:40.900815+0000","last_active":"2023-11-06T21:59:40.905479+0000","last_peered":"2023-11-06T21:59:40.905479+0000","last_clean":"2023-11-06T21:59:40.905479+0000","last_became_active":"2023-11-06T21:59:40.899554+0000","last_became_peered":"2023-11-06T21:59:40.899554+0000","last_unstale":"2023-11-06T21:59:40.905479+0000","last_undegraded":"2023-11-06T21:59:40.905479+0000","last_fullsized":"2023-11-06T21:59:40.905479+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882999+0000","last_change":"2023-11-06T21:59:40.400555+0000","last_active":"2023-11-06T21:59:40.882999+0000","last_peered":"2023-11-06T21:59:40.882999+0000","last_clean":"2023-11-06T21:59:40.882999+0000","last_became_active":"2023-11-06T21:59:40.400225+0000","last_became_peered":"2023-11-06T21:59:40.400225+0000","last_unstale":"2023-11-06T21:59:40.882999+0000","last_undegraded":"2023-11-06T21:59:40.882999+0000","last_fullsized":"2023-11-06T21:59:40.882999+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.893630+0000","last_change":"2023-11-06T21:59:40.887899+0000","last_active":"2023-11-06T21:59:40.893630+0000","last_peered":"2023-11-06T21:59:40.893630+0000","last_clean":"2023-11-06T21:59:40.893630+0000","last_became_active":"2023-11-06T21:59:40.887466+0000","last_became_peered":"2023-11-06T21:59:40.887466+0000","last_unstale":"2023-11-06T21:59:40.893630+0000","last_undegraded":"2023-11-06T21:59:40.893630+0000","last_fullsized":"2023-11-06T21:59:40.893630+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882870+0000","last_change":"2023-11-06T21:59:40.410553+0000","last_active":"2023-11-06T21:59:40.882870+0000","last_peered":"2023-11-06T21:59:40.882870+0000","last_clean":"2023-11-06T21:59:40.882870+0000","last_became_active":"2023-11-06T21:59:40.409980+0000","last_became_peered":"2023-11-06T21:59:40.409980+0000","last_unstale":"2023-11-06T21:59:40.882870+0000","last_undegraded":"2023-11-06T21:59:40.882870+0000","last_fullsized":"2023-11-06T21:59:40.882870+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.899055+0000","last_change":"2023-11-06T21:59:40.894561+0000","last_active":"2023-11-06T21:59:40.899055+0000","last_peered":"2023-11-06T21:59:40.899055+0000","last_clean":"2023-11-06T21:59:40.899055+0000","last_became_active":"2023-11-06T21:59:40.894231+0000","last_became_peered":"2023-11-06T21:59:40.894231+0000","last_unstale":"2023-11-06T21:59:40.899055+0000","last_undegraded":"2023-11-06T21:59:40.899055+0000","last_fullsized":"2023-11-06T21:59:40.899055+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.b","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.905871+0000","last_change":"2023-11-06T21:59:40.406018+0000","last_active":"2023-11-06T21:59:40.905871+0000","last_peered":"2023-11-06T21:59:40.905871+0000","last_clean":"2023-11-06T21:59:40.905871+0000","last_became_active":"2023-11-06T21:59:40.405722+0000","last_became_peered":"2023-11-06T21:59:40.405722+0000","last_unstale":"2023-11-06T21:59:40.905871+0000","last_undegraded":"2023-11-06T21:59:40.905871+0000","last_fullsized":"2023-11-06T21:59:40.905871+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.902648+0000","last_change":"2023-11-06T21:59:40.898369+0000","last_active":"2023-11-06T21:59:40.902648+0000","last_peered":"2023-11-06T21:59:40.902648+0000","last_clean":"2023-11-06T21:59:40.902648+0000","last_became_active":"2023-11-06T21:59:40.898072+0000","last_became_peered":"2023-11-06T21:59:40.898072+0000","last_unstale":"2023-11-06T21:59:40.902648+0000","last_undegraded":"2023-11-06T21:59:40.902648+0000","last_fullsized":"2023-11-06T21:59:40.902648+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882602+0000","last_change":"2023-11-06T21:59:40.410696+0000","last_active":"2023-11-06T21:59:40.882602+0000","last_peered":"2023-11-06T21:59:40.882602+0000","last_clean":"2023-11-06T21:59:40.882602+0000","last_became_active":"2023-11-06T21:59:40.409187+0000","last_became_peered":"2023-11-06T21:59:40.409187+0000","last_unstale":"2023-11-06T21:59:40.882602+0000","last_undegraded":"2023-11-06T21:59:40.882602+0000","last_fullsized":"2023-11-06T21:59:40.882602+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.898942+0000","last_change":"2023-11-06T21:59:40.895397+0000","last_active":"2023-11-06T21:59:40.898942+0000","last_peered":"2023-11-06T21:59:40.898942+0000","last_clean":"2023-11-06T21:59:40.898942+0000","last_became_active":"2023-11-06T21:59:40.894437+0000","last_became_peered":"2023-11-06T21:59:40.894437+0000","last_unstale":"2023-11-06T21:59:40.898942+0000","last_undegraded":"2023-11-06T21:59:40.898942+0000","last_fullsized":"2023-11-06T21:59:40.898942+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.877860+0000","last_change":"2023-11-06T21:59:40.399179+0000","last_active":"2023-11-06T21:59:40.877860+0000","last_peered":"2023-11-06T21:59:40.877860+0000","last_clean":"2023-11-06T21:59:40.877860+0000","last_became_active":"2023-11-06T21:59:40.398897+0000","last_became_peered":"2023-11-06T21:59:40.398897+0000","last_unstale":"2023-11-06T21:59:40.877860+0000","last_undegraded":"2023-11-06T21:59:40.877860+0000","last_fullsized":"2023-11-06T21:59:40.877860+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.900547+0000","last_change":"2023-11-06T21:59:40.891221+0000","last_active":"2023-11-06T21:59:40.900547+0000","last_peered":"2023-11-06T21:59:40.900547+0000","last_clean":"2023-11-06T21:59:40.900547+0000","last_became_active":"2023-11-06T21:59:40.890926+0000","last_became_peered":"2023-11-06T21:59:40.890926+0000","last_unstale":"2023-11-06T21:59:40.900547+0000","last_undegraded":"2023-11-06T21:59:40.900547+0000","last_fullsized":"2023-11-06T21:59:40.900547+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.873276+0000","last_change":"2023-11-06T21:59:40.397667+0000","last_active":"2023-11-06T21:59:40.873276+0000","last_peered":"2023-11-06T21:59:40.873276+0000","last_clean":"2023-11-06T21:59:40.873276+0000","last_became_active":"2023-11-06T21:59:40.397310+0000","last_became_peered":"2023-11-06T21:59:40.397310+0000","last_unstale":"2023-11-06T21:59:40.873276+0000","last_undegraded":"2023-11-06T21:59:40.873276+0000","last_fullsized":"2023-11-06T21:59:40.873276+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.890294+0000","last_change":"2023-11-06T21:59:40.883958+0000","last_active":"2023-11-06T21:59:40.890294+0000","last_peered":"2023-11-06T21:59:40.890294+0000","last_clean":"2023-11-06T21:59:40.890294+0000","last_became_active":"2023-11-06T21:59:40.883575+0000","last_became_peered":"2023-11-06T21:59:40.883575+0000","last_unstale":"2023-11-06T21:59:40.890294+0000","last_undegraded":"2023-11-06T21:59:40.890294+0000","last_fullsized":"2023-11-06T21:59:40.890294+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.7","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.901404+0000","last_change":"2023-11-06T21:59:40.413074+0000","last_active":"2023-11-06T21:59:40.901404+0000","last_peered":"2023-11-06T21:59:40.901404+0000","last_clean":"2023-11-06T21:59:40.901404+0000","last_became_active":"2023-11-06T21:59:40.412134+0000","last_became_peered":"2023-11-06T21:59:40.412134+0000","last_unstale":"2023-11-06T21:59:40.901404+0000","last_undegraded":"2023-11-06T21:59:40.901404+0000","last_fullsized":"2023-11-06T21:59:40.901404+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907395+0000","last_change":"2023-11-06T21:59:40.892081+0000","last_active":"2023-11-06T21:59:40.907395+0000","last_peered":"2023-11-06T21:59:40.907395+0000","last_clean":"2023-11-06T21:59:40.907395+0000","last_became_active":"2023-11-06T21:59:40.891804+0000","last_became_peered":"2023-11-06T21:59:40.891804+0000","last_unstale":"2023-11-06T21:59:40.907395+0000","last_undegraded":"2023-11-06T21:59:40.907395+0000","last_fullsized":"2023-11-06T21:59:40.907395+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.888947+0000","last_change":"2023-11-06T21:59:40.407023+0000","last_active":"2023-11-06T21:59:40.888947+0000","last_peered":"2023-11-06T21:59:40.888947+0000","last_clean":"2023-11-06T21:59:40.888947+0000","last_became_active":"2023-11-06T21:59:40.406741+0000","last_became_peered":"2023-11-06T21:59:40.406741+0000","last_unstale":"2023-11-06T21:59:40.888947+0000","last_undegraded":"2023-11-06T21:59:40.888947+0000","last_fullsized":"2023-11-06T21:59:40.888947+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908291+0000","last_change":"2023-11-06T21:59:40.900521+0000","last_active":"2023-11-06T21:59:40.908291+0000","last_peered":"2023-11-06T21:59:40.908291+0000","last_clean":"2023-11-06T21:59:40.908291+0000","last_became_active":"2023-11-06T21:59:40.900140+0000","last_became_peered":"2023-11-06T21:59:40.900140+0000","last_unstale":"2023-11-06T21:59:40.908291+0000","last_undegraded":"2023-11-06T21:59:40.908291+0000","last_fullsized":"2023-11-06T21:59:40.908291+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.881655+0000","last_change":"2023-11-06T21:59:40.400982+0000","last_active":"2023-11-06T21:59:40.881655+0000","last_peered":"2023-11-06T21:59:40.881655+0000","last_clean":"2023-11-06T21:59:40.881655+0000","last_became_active":"2023-11-06T21:59:40.400697+0000","last_became_peered":"2023-11-06T21:59:40.400697+0000","last_unstale":"2023-11-06T21:59:40.881655+0000","last_undegraded":"2023-11-06T21:59:40.881655+0000","last_fullsized":"2023-11-06T21:59:40.881655+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.898507+0000","last_change":"2023-11-06T21:59:40.892656+0000","last_active":"2023-11-06T21:59:40.898507+0000","last_peered":"2023-11-06T21:59:40.898507+0000","last_clean":"2023-11-06T21:59:40.898507+0000","last_became_active":"2023-11-06T21:59:40.892279+0000","last_became_peered":"2023-11-06T21:59:40.892279+0000","last_unstale":"2023-11-06T21:59:40.898507+0000","last_undegraded":"2023-11-06T21:59:40.898507+0000","last_fullsized":"2023-11-06T21:59:40.898507+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.873056+0000","last_change":"2023-11-06T21:59:40.402327+0000","last_active":"2023-11-06T21:59:40.873056+0000","last_peered":"2023-11-06T21:59:40.873056+0000","last_clean":"2023-11-06T21:59:40.873056+0000","last_became_active":"2023-11-06T21:59:40.401964+0000","last_became_peered":"2023-11-06T21:59:40.401964+0000","last_unstale":"2023-11-06T21:59:40.873056+0000","last_undegraded":"2023-11-06T21:59:40.873056+0000","last_fullsized":"2023-11-06T21:59:40.873056+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897046+0000","last_change":"2023-11-06T21:59:40.889256+0000","last_active":"2023-11-06T21:59:40.897046+0000","last_peered":"2023-11-06T21:59:40.897046+0000","last_clean":"2023-11-06T21:59:40.897046+0000","last_became_active":"2023-11-06T21:59:40.888945+0000","last_became_peered":"2023-11-06T21:59:40.888945+0000","last_unstale":"2023-11-06T21:59:40.897046+0000","last_undegraded":"2023-11-06T21:59:40.897046+0000","last_fullsized":"2023-11-06T21:59:40.897046+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.880423+0000","last_change":"2023-11-06T21:59:40.405739+0000","last_active":"2023-11-06T21:59:40.880423+0000","last_peered":"2023-11-06T21:59:40.880423+0000","last_clean":"2023-11-06T21:59:40.880423+0000","last_became_active":"2023-11-06T21:59:40.405442+0000","last_became_peered":"2023-11-06T21:59:40.405442+0000","last_unstale":"2023-11-06T21:59:40.880423+0000","last_undegraded":"2023-11-06T21:59:40.880423+0000","last_fullsized":"2023-11-06T21:59:40.880423+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896345+0000","last_change":"2023-11-06T21:59:40.893426+0000","last_active":"2023-11-06T21:59:40.896345+0000","last_peered":"2023-11-06T21:59:40.896345+0000","last_clean":"2023-11-06T21:59:40.896345+0000","last_became_active":"2023-11-06T21:59:40.890618+0000","last_became_peered":"2023-11-06T21:59:40.890618+0000","last_unstale":"2023-11-06T21:59:40.896345+0000","last_undegraded":"2023-11-06T21:59:40.896345+0000","last_fullsized":"2023-11-06T21:59:40.896345+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.881046+0000","last_change":"2023-11-06T21:59:40.398590+0000","last_active":"2023-11-06T21:59:40.881046+0000","last_peered":"2023-11-06T21:59:40.881046+0000","last_clean":"2023-11-06T21:59:40.881046+0000","last_became_active":"2023-11-06T21:59:40.398158+0000","last_became_peered":"2023-11-06T21:59:40.398158+0000","last_unstale":"2023-11-06T21:59:40.881046+0000","last_undegraded":"2023-11-06T21:59:40.881046+0000","last_fullsized":"2023-11-06T21:59:40.881046+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.885871+0000","last_change":"2023-11-06T21:59:40.883826+0000","last_active":"2023-11-06T21:59:40.885871+0000","last_peered":"2023-11-06T21:59:40.885871+0000","last_clean":"2023-11-06T21:59:40.885871+0000","last_became_active":"2023-11-06T21:59:40.883552+0000","last_became_peered":"2023-11-06T21:59:40.883552+0000","last_unstale":"2023-11-06T21:59:40.885871+0000","last_undegraded":"2023-11-06T21:59:40.885871+0000","last_fullsized":"2023-11-06T21:59:40.885871+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.872794+0000","last_change":"2023-11-06T21:59:40.400285+0000","last_active":"2023-11-06T21:59:40.872794+0000","last_peered":"2023-11-06T21:59:40.872794+0000","last_clean":"2023-11-06T21:59:40.872794+0000","last_became_active":"2023-11-06T21:59:40.400001+0000","last_became_peered":"2023-11-06T21:59:40.400001+0000","last_unstale":"2023-11-06T21:59:40.872794+0000","last_undegraded":"2023-11-06T21:59:40.872794+0000","last_fullsized":"2023-11-06T21:59:40.872794+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.904564+0000","last_change":"2023-11-06T21:59:40.898026+0000","last_active":"2023-11-06T21:59:40.904564+0000","last_peered":"2023-11-06T21:59:40.904564+0000","last_clean":"2023-11-06T21:59:40.904564+0000","last_became_active":"2023-11-06T21:59:40.895767+0000","last_became_peered":"2023-11-06T21:59:40.895767+0000","last_unstale":"2023-11-06T21:59:40.904564+0000","last_undegraded":"2023-11-06T21:59:40.904564+0000","last_fullsized":"2023-11-06T21:59:40.904564+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.874345+0000","last_change":"2023-11-06T21:59:40.406940+0000","last_active":"2023-11-06T21:59:40.874345+0000","last_peered":"2023-11-06T21:59:40.874345+0000","last_clean":"2023-11-06T21:59:40.874345+0000","last_became_active":"2023-11-06T21:59:40.406276+0000","last_became_peered":"2023-11-06T21:59:40.406276+0000","last_unstale":"2023-11-06T21:59:40.874345+0000","last_undegraded":"2023-11-06T21:59:40.874345+0000","last_fullsized":"2023-11-06T21:59:40.874345+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.908720+0000","last_change":"2023-11-06T21:59:40.893113+0000","last_active":"2023-11-06T21:59:40.908720+0000","last_peered":"2023-11-06T21:59:40.908720+0000","last_clean":"2023-11-06T21:59:40.908720+0000","last_became_active":"2023-11-06T21:59:40.891126+0000","last_became_peered":"2023-11-06T21:59:40.891126+0000","last_unstale":"2023-11-06T21:59:40.908720+0000","last_undegraded":"2023-11-06T21:59:40.908720+0000","last_fullsized":"2023-11-06T21:59:40.908720+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"169'3","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.909427+0000","last_change":"2023-11-06T21:59:40.406818+0000","last_active":"2023-11-06T21:59:40.909427+0000","last_peered":"2023-11-06T21:59:40.909427+0000","last_clean":"2023-11-06T21:59:40.909427+0000","last_became_active":"2023-11-06T21:59:40.406429+0000","last_became_peered":"2023-11-06T21:59:40.406429+0000","last_unstale":"2023-11-06T21:59:40.909427+0000","last_undegraded":"2023-11-06T21:59:40.909427+0000","last_fullsized":"2023-11-06T21:59:40.909427+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882932+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T21:59:40.882932+0000","last_peered":"2023-11-06T21:59:40.882932+0000","last_clean":"2023-11-06T21:59:40.882932+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T21:59:40.882932+0000","last_undegraded":"2023-11-06T21:59:40.882932+0000","last_fullsized":"2023-11-06T21:59:40.882932+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897065+0000","last_change":"2023-11-06T21:59:40.889803+0000","last_active":"2023-11-06T21:59:40.897065+0000","last_peered":"2023-11-06T21:59:40.897065+0000","last_clean":"2023-11-06T21:59:40.897065+0000","last_became_active":"2023-11-06T21:59:40.889509+0000","last_became_peered":"2023-11-06T21:59:40.889509+0000","last_unstale":"2023-11-06T21:59:40.897065+0000","last_undegraded":"2023-11-06T21:59:40.897065+0000","last_fullsized":"2023-11-06T21:59:40.897065+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.13","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.900928+0000","last_change":"2023-11-06T21:59:40.402832+0000","last_active":"2023-11-06T21:59:40.900928+0000","last_peered":"2023-11-06T21:59:40.900928+0000","last_clean":"2023-11-06T21:59:40.900928+0000","last_became_active":"2023-11-06T21:59:40.402540+0000","last_became_peered":"2023-11-06T21:59:40.402540+0000","last_unstale":"2023-11-06T21:59:40.900928+0000","last_undegraded":"2023-11-06T21:59:40.900928+0000","last_fullsized":"2023-11-06T21:59:40.900928+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.876995+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T21:59:40.876995+0000","last_peered":"2023-11-06T21:59:40.876995+0000","last_clean":"2023-11-06T21:59:40.876995+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T21:59:40.876995+0000","last_undegraded":"2023-11-06T21:59:40.876995+0000","last_fullsized":"2023-11-06T21:59:40.876995+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.897707+0000","last_change":"2023-11-06T21:59:40.894570+0000","last_active":"2023-11-06T21:59:40.897707+0000","last_peered":"2023-11-06T21:59:40.897707+0000","last_clean":"2023-11-06T21:59:40.897707+0000","last_became_active":"2023-11-06T21:59:40.894269+0000","last_became_peered":"2023-11-06T21:59:40.894269+0000","last_unstale":"2023-11-06T21:59:40.897707+0000","last_undegraded":"2023-11-06T21:59:40.897707+0000","last_fullsized":"2023-11-06T21:59:40.897707+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"169'4","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.906405+0000","last_change":"2023-11-06T21:59:40.407268+0000","last_active":"2023-11-06T21:59:40.906405+0000","last_peered":"2023-11-06T21:59:40.906405+0000","last_clean":"2023-11-06T21:59:40.906405+0000","last_became_active":"2023-11-06T21:59:40.406958+0000","last_became_peered":"2023-11-06T21:59:40.406958+0000","last_unstale":"2023-11-06T21:59:40.906405+0000","last_undegraded":"2023-11-06T21:59:40.906405+0000","last_fullsized":"2023-11-06T21:59:40.906405+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.874761+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T21:59:40.874761+0000","last_peered":"2023-11-06T21:59:40.874761+0000","last_clean":"2023-11-06T21:59:40.874761+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T21:59:40.874761+0000","last_undegraded":"2023-11-06T21:59:40.874761+0000","last_fullsized":"2023-11-06T21:59:40.874761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.907610+0000","last_change":"2023-11-06T21:59:40.891989+0000","last_active":"2023-11-06T21:59:40.907610+0000","last_peered":"2023-11-06T21:59:40.907610+0000","last_clean":"2023-11-06T21:59:40.907610+0000","last_became_active":"2023-11-06T21:59:40.891726+0000","last_became_peered":"2023-11-06T21:59:40.891726+0000","last_unstale":"2023-11-06T21:59:40.907610+0000","last_undegraded":"2023-11-06T21:59:40.907610+0000","last_fullsized":"2023-11-06T21:59:40.907610+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.876899+0000","last_change":"2023-11-06T21:59:40.397282+0000","last_active":"2023-11-06T21:59:40.876899+0000","last_peered":"2023-11-06T21:59:40.876899+0000","last_clean":"2023-11-06T21:59:40.876899+0000","last_became_active":"2023-11-06T21:59:40.394993+0000","last_became_peered":"2023-11-06T21:59:40.394993+0000","last_unstale":"2023-11-06T21:59:40.876899+0000","last_undegraded":"2023-11-06T21:59:40.876899+0000","last_fullsized":"2023-11-06T21:59:40.876899+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.878993+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T21:59:40.878993+0000","last_peered":"2023-11-06T21:59:40.878993+0000","last_clean":"2023-11-06T21:59:40.878993+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T21:59:40.878993+0000","last_undegraded":"2023-11-06T21:59:40.878993+0000","last_fullsized":"2023-11-06T21:59:40.878993+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":143,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.886776+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T21:59:40.886776+0000","last_peered":"2023-11-06T21:59:40.886776+0000","last_clean":"2023-11-06T21:59:40.886776+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T21:59:40.886776+0000","last_undegraded":"2023-11-06T21:59:40.886776+0000","last_fullsized":"2023-11-06T21:59:40.886776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.904499+0000","last_change":"2023-11-06T21:59:40.896800+0000","last_active":"2023-11-06T21:59:40.904499+0000","last_peered":"2023-11-06T21:59:40.904499+0000","last_clean":"2023-11-06T21:59:40.904499+0000","last_became_active":"2023-11-06T21:59:40.896495+0000","last_became_peered":"2023-11-06T21:59:40.896495+0000","last_unstale":"2023-11-06T21:59:40.904499+0000","last_undegraded":"2023-11-06T21:59:40.904499+0000","last_fullsized":"2023-11-06T21:59:40.904499+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.877749+0000","last_change":"2023-11-06T21:59:40.408314+0000","last_active":"2023-11-06T21:59:40.877749+0000","last_peered":"2023-11-06T21:59:40.877749+0000","last_clean":"2023-11-06T21:59:40.877749+0000","last_became_active":"2023-11-06T21:59:40.407703+0000","last_became_peered":"2023-11-06T21:59:40.407703+0000","last_unstale":"2023-11-06T21:59:40.877749+0000","last_undegraded":"2023-11-06T21:59:40.877749+0000","last_fullsized":"2023-11-06T21:59:40.877749+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879618+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T21:59:40.879618+0000","last_peered":"2023-11-06T21:59:40.879618+0000","last_clean":"2023-11-06T21:59:40.879618+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T21:59:40.879618+0000","last_undegraded":"2023-11-06T21:59:40.879618+0000","last_fullsized":"2023-11-06T21:59:40.879618+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.906419+0000","last_change":"2023-11-06T21:59:40.898302+0000","last_active":"2023-11-06T21:59:40.906419+0000","last_peered":"2023-11-06T21:59:40.906419+0000","last_clean":"2023-11-06T21:59:40.906419+0000","last_became_active":"2023-11-06T21:59:40.898023+0000","last_became_peered":"2023-11-06T21:59:40.898023+0000","last_unstale":"2023-11-06T21:59:40.906419+0000","last_undegraded":"2023-11-06T21:59:40.906419+0000","last_fullsized":"2023-11-06T21:59:40.906419+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.910194+0000","last_change":"2023-11-06T21:59:40.410937+0000","last_active":"2023-11-06T21:59:40.910194+0000","last_peered":"2023-11-06T21:59:40.910194+0000","last_clean":"2023-11-06T21:59:40.910194+0000","last_became_active":"2023-11-06T21:59:40.410339+0000","last_became_peered":"2023-11-06T21:59:40.410339+0000","last_unstale":"2023-11-06T21:59:40.910194+0000","last_undegraded":"2023-11-06T21:59:40.910194+0000","last_fullsized":"2023-11-06T21:59:40.910194+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882063+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T21:59:40.882063+0000","last_peered":"2023-11-06T21:59:40.882063+0000","last_clean":"2023-11-06T21:59:40.882063+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T21:59:40.882063+0000","last_undegraded":"2023-11-06T21:59:40.882063+0000","last_fullsized":"2023-11-06T21:59:40.882063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.896511+0000","last_change":"2023-11-06T21:59:40.891693+0000","last_active":"2023-11-06T21:59:40.896511+0000","last_peered":"2023-11-06T21:59:40.896511+0000","last_clean":"2023-11-06T21:59:40.896511+0000","last_became_active":"2023-11-06T21:59:40.891375+0000","last_became_peered":"2023-11-06T21:59:40.891375+0000","last_unstale":"2023-11-06T21:59:40.896511+0000","last_undegraded":"2023-11-06T21:59:40.896511+0000","last_fullsized":"2023-11-06T21:59:40.896511+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.879924+0000","last_change":"2023-11-06T21:59:40.413385+0000","last_active":"2023-11-06T21:59:40.879924+0000","last_peered":"2023-11-06T21:59:40.879924+0000","last_clean":"2023-11-06T21:59:40.879924+0000","last_became_active":"2023-11-06T21:59:40.412579+0000","last_became_peered":"2023-11-06T21:59:40.412579+0000","last_unstale":"2023-11-06T21:59:40.879924+0000","last_undegraded":"2023-11-06T21:59:40.879924+0000","last_fullsized":"2023-11-06T21:59:40.879924+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882792+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T21:59:40.882792+0000","last_peered":"2023-11-06T21:59:40.882792+0000","last_clean":"2023-11-06T21:59:40.882792+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T21:59:40.882792+0000","last_undegraded":"2023-11-06T21:59:40.882792+0000","last_fullsized":"2023-11-06T21:59:40.882792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.884231+0000","last_change":"2023-11-06T21:59:40.879304+0000","last_active":"2023-11-06T21:59:40.884231+0000","last_peered":"2023-11-06T21:59:40.884231+0000","last_clean":"2023-11-06T21:59:40.884231+0000","last_became_active":"2023-11-06T21:59:40.879011+0000","last_became_peered":"2023-11-06T21:59:40.879011+0000","last_unstale":"2023-11-06T21:59:40.884231+0000","last_undegraded":"2023-11-06T21:59:40.884231+0000","last_fullsized":"2023-11-06T21:59:40.884231+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:40.370274+0000","last_clean_scrub_stamp":"2023-11-06T21:59:40.370274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.882696+0000","last_change":"2023-11-06T21:59:40.409786+0000","last_active":"2023-11-06T21:59:40.882696+0000","last_peered":"2023-11-06T21:59:40.882696+0000","last_clean":"2023-11-06T21:59:40.882696+0000","last_became_active":"2023-11-06T21:59:40.409281+0000","last_became_peered":"2023-11-06T21:59:40.409281+0000","last_unstale":"2023-11-06T21:59:40.882696+0000","last_undegraded":"2023-11-06T21:59:40.882696+0000","last_fullsized":"2023-11-06T21:59:40.882696+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T21:59:39.362796+0000","last_clean_scrub_stamp":"2023-11-06T21:59:39.362796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-11-06T21:59:40.884151+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T21:59:40.884151+0000","last_peered":"2023-11-06T21:59:40.884151+0000","last_clean":"2023-11-06T21:59:40.884151+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T21:59:40.884151+0000","last_undegraded":"2023-11-06T21:59:40.884151+0000","last_fullsized":"2023-11-06T21:59:40.884151+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575750,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307352,"kb_used_data":2064,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064488,"statfs":{"total":96636764160,"available":96322035712,"internally_reserved":0,"allocated":2113536,"data_stored":1079854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.505,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90500000000000003,"5min":0.997,"15min":1.002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73199999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93100000000000005,"5min":1.022,"15min":1.022},"last":0.65200000000000002}]},{"osd":1,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.55300000000000005,"5min":0.48899999999999999,"15min":0.42399999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.42499999999999999},"max":{"1min":0.88800000000000001,"5min":1.083,"15min":1.083},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.745,"15min":0.752},"min":{"1min":0.54500000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.94599999999999995,"5min":1.004,"15min":1.004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.753,"15min":0.75},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.45200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.0029999999999999},"last":0.71099999999999997}]},{"osd":3,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.751,"15min":0.73899999999999999},"min":{"1min":0.56699999999999995,"5min":0.48499999999999999,"15min":0.47799999999999998},"max":{"1min":0.95599999999999996,"5min":1.038,"15min":1.038},"last":0.623},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54000000000000004},"max":{"1min":1,"5min":1.046,"15min":1.046},"last":0.68500000000000005}]},{"osd":4,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.61099999999999999},"min":{"1min":0.47699999999999998,"5min":0.45000000000000001,"15min":0.377},"max":{"1min":0.91200000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47099999999999997,"5min":0.433,"15min":0.41899999999999998},"max":{"1min":0.79900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.503}]},{"osd":5,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.503,"5min":0.45200000000000001,"15min":0.41499999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.68999999999999995},"min":{"1min":0.51600000000000001,"5min":0.46600000000000003,"15min":0.438},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.58199999999999996}]},{"osd":6,"last update":"Mon Nov 6 21:59:00 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.66200000000000003,"5min":0.53200000000000003,"15min":0.495},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70699999999999996,"15min":0.71099999999999997},"min":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.48199999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.56299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834575750,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307344,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064496,"statfs":{"total":96636764160,"available":96322043904,"internally_reserved":0,"allocated":2105344,"data_stored":1073140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.40000000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.04},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71099999999999997,"15min":0.71199999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.37},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.0209999999999999},"last":0.87}]},{"osd":1,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83099999999999996,"5min":0.93600000000000005,"15min":0.99399999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.32600000000000001},"max":{"1min":0.91100000000000003,"5min":1.004,"15min":1.004},"last":1.046}]},{"osd":2,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95299999999999996,"5min":1.034,"15min":1.034},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94499999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.98799999999999999}]},{"osd":3,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":1.02},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76200000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.503},"max":{"1min":0.89700000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":1.028}]},{"osd":4,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.39000000000000001},"max":{"1min":0.84899999999999998,"5min":0.85199999999999998,"15min":0.94399999999999995},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64400000000000002,"15min":0.63300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.434},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.75600000000000001}]},{"osd":5,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.627},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.70199999999999996},"min":{"1min":0.499,"5min":0.498,"15min":0.46200000000000002},"max":{"1min":0.95599999999999996,"5min":0.98099999999999998,"15min":0.98699999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.44400000000000001},"max":{"1min":0.84099999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.48899999999999999},"max":{"1min":0.93899999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575750,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":2117632,"data_stored":1074732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97299999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0209999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.875,"5min":0.875,"15min":1.1850000000000001},"last":0.49099999999999999}]},{"osd":3,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.60399999999999998},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.50700000000000001}]},{"osd":4,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0269999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83799999999999997,"15min":0.84699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59399999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.81299999999999994},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.55700000000000005},"max":{"1min":1.02,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.046,"5min":1.1559999999999999,"15min":1.198},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.82399999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.107,"5min":1.1160000000000001,"15min":1.151},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.83699999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.51600000000000001},"max":{"1min":1.083,"5min":1.206,"15min":1.206},"last":0.68200000000000005}]},{"osd":7,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85299999999999998,"15min":0.86199999999999999},"min":{"1min":0.70899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1499999999999999,"5min":1.2070000000000001,"15min":1.21},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0980000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.747}]}]},{"osd":0,"up_from":13,"seq":55834575750,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303264,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068576,"statfs":{"total":96636764160,"available":96326221824,"internally_reserved":0,"allocated":2121728,"data_stored":1075286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66200000000000003,"5min":0.749,"15min":0.82899999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54000000000000004,"15min":0.53300000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71399999999999997,"5min":0.83599999999999997,"15min":0.83799999999999997},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.91700000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.376},"max":{"1min":0.78900000000000003,"5min":0.80400000000000005,"15min":0.89800000000000002},"last":0.53500000000000003}]},{"osd":3,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":0.78600000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.5},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.44900000000000001,"5min":0.436,"15min":0.40300000000000002},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.88200000000000001},"last":0.63500000000000001}]},{"osd":4,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.62,"5min":0.61299999999999999,"15min":0.57099999999999995},"max":{"1min":1.0489999999999999,"5min":1.0920000000000001,"15min":1.2649999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67700000000000005,"5min":0.57399999999999995,"15min":0.51700000000000002},"max":{"1min":1.03,"5min":1.0800000000000001,"15min":1.137},"last":0.91400000000000003}]},{"osd":5,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.86099999999999999,"15min":0.85399999999999998},"min":{"1min":0.66800000000000004,"5min":0.61899999999999999,"15min":0.57999999999999996},"max":{"1min":0.99199999999999999,"5min":1.1970000000000001,"15min":1.306},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.879,"5min":0.871,"15min":0.86099999999999999},"min":{"1min":0.69099999999999995,"5min":0.60699999999999998,"15min":0.54900000000000004},"max":{"1min":1.0309999999999999,"5min":1.121,"15min":1.2729999999999999},"last":1.014}]},{"osd":6,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83399999999999996},"min":{"1min":0.66800000000000004,"5min":0.59599999999999997,"15min":0.55000000000000004},"max":{"1min":0.95299999999999996,"5min":1.1279999999999999,"15min":1.1910000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.84699999999999998,"15min":0.84799999999999998},"min":{"1min":0.74199999999999999,"5min":0.56599999999999995,"15min":0.50700000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.228},"last":0.876}]},{"osd":7,"last update":"Mon Nov 6 21:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86399999999999999,"15min":0.86099999999999999},"min":{"1min":0.73399999999999999,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.101,"5min":1.1439999999999999,"15min":1.2210000000000001},"last":0.998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.86399999999999999,"15min":0.86299999999999999},"min":{"1min":0.61099999999999999,"5min":0.59299999999999997,"15min":0.56899999999999995},"max":{"1min":1.0860000000000001,"5min":1.129,"15min":1.337},"last":0.92800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575750,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303252,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068588,"statfs":{"total":96636764160,"available":96326234112,"internally_reserved":0,"allocated":2109440,"data_stored":1073198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.46200000000000002},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.28999999999999998},"max":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.309},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.32800000000000001},"max":{"1min":0.66900000000000004,"5min":0.84799999999999998,"15min":0.86299999999999999},"last":0.503}]},{"osd":1,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.432},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61899999999999999}]},{"osd":3,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51400000000000001,"15min":0.51500000000000001},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70299999999999996,"5min":0.77100000000000002,"15min":0.80000000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.44800000000000001,"5min":0.432,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.89700000000000002,"15min":1.0940000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.622,"15min":0.55500000000000005},"max":{"1min":1.075,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83699999999999997,"15min":0.81699999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1020000000000001,"5min":1.256,"15min":1.256},"last":0.56899999999999995}]},{"osd":5,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.81000000000000005},"min":{"1min":0.64400000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.86099999999999999,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.58299999999999996},"max":{"1min":1.0840000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.53200000000000003}]},{"osd":6,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.84199999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.51800000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.69299999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.145,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.88300000000000001,"15min":0.871},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.57299999999999995},"max":{"1min":1.173,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.752,"5min":0.57699999999999996,"15min":0.55900000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.69399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307340,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064500,"statfs":{"total":96636764160,"available":96322048000,"internally_reserved":0,"allocated":2101248,"data_stored":1073106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.33400000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.39300000000000002}]},{"osd":1,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.32400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.308},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.376},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.375},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.40899999999999997},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.436}]},{"osd":4,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86199999999999999,"15min":0.84399999999999997},"min":{"1min":0.73799999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.85999999999999999,"15min":0.84599999999999997},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.47999999999999998},"max":{"1min":1.0309999999999999,"5min":1.202,"15min":1.202},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.85699999999999998,"15min":0.82699999999999996},"min":{"1min":0.66000000000000003,"5min":0.59499999999999997,"15min":0.57399999999999995},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.67600000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.64800000000000002}]},{"osd":6,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.86899999999999999,"15min":0.83599999999999997},"min":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89500000000000002,"15min":0.86699999999999999},"min":{"1min":0.69399999999999995,"5min":0.66500000000000004,"15min":0.59999999999999998},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.81599999999999995}]},{"osd":7,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.87,"15min":0.85899999999999999},"min":{"1min":0.69299999999999995,"5min":0.58899999999999997,"15min":0.55500000000000005},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.92000000000000004,"15min":0.88100000000000001},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.56799999999999995},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.77900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575750,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303252,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068588,"statfs":{"total":96636764160,"available":96326234112,"internally_reserved":0,"allocated":2109440,"data_stored":1073730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.54300000000000004,"5min":0.50900000000000001,"15min":0.45000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.1659999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89800000000000002,"5min":0.999,"15min":1},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91000000000000003,"5min":0.93999999999999995,"15min":1.046},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.74199999999999999},"min":{"1min":0.64600000000000002,"5min":0.55800000000000005,"15min":0.44700000000000001},"max":{"1min":0.84799999999999998,"5min":1.0700000000000001,"15min":1.131},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.746},"min":{"1min":0.64800000000000002,"5min":0.50800000000000001,"15min":0.41099999999999998},"max":{"1min":0.95799999999999996,"5min":0.99299999999999999,"15min":1.014},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.80700000000000005},"min":{"1min":0.71299999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.008,"5min":1.093,"15min":1.093},"last":0.60899999999999999}]},{"osd":3,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73299999999999998,"15min":0.72199999999999998},"min":{"1min":0.65900000000000003,"5min":0.48099999999999998,"15min":0.46000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.181},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.60699999999999998,"5min":0.52700000000000002,"15min":0.47099999999999997},"max":{"1min":1.0009999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.61499999999999999},"min":{"1min":0.502,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.92100000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.44500000000000001,"5min":0.42999999999999999,"15min":0.36899999999999999},"max":{"1min":0.88700000000000001,"5min":0.94499999999999995,"15min":1.228},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.48299999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.81000000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.56200000000000006,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.04,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Mon Nov 6 21:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.47999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.085},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73699999999999999,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.503,"15min":0.496},"max":{"1min":1.004,"5min":1.0189999999999999,"15min":1.091},"last":0.57299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575751,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307348,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064492,"statfs":{"total":96636764160,"available":96322039808,"internally_reserved":0,"allocated":2109440,"data_stored":1073162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67500000000000004,"15min":0.67800000000000005},"min":{"1min":0.48999999999999999,"5min":0.38600000000000001,"15min":0.33600000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.008},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.46899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.98199999999999998,"15min":0.99299999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75,"15min":0.753},"min":{"1min":0.57899999999999996,"5min":0.375,"15min":0.375},"max":{"1min":0.92100000000000004,"5min":1.0009999999999999,"15min":1.2370000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.496,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.0629999999999999,"15min":1.2470000000000001},"last":0.57499999999999996}]},{"osd":2,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.58499999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94399999999999995,"5min":1.085,"15min":1.085},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.753},"min":{"1min":0.66700000000000004,"5min":0.55100000000000005,"15min":0.47199999999999998},"max":{"1min":0.95599999999999996,"5min":1.036,"15min":1.036},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.60499999999999998,"5min":0.49099999999999999,"15min":0.47399999999999998},"max":{"1min":0.88300000000000001,"5min":1.02,"15min":1.02},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.63500000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90400000000000003,"5min":1.093,"15min":1.093},"last":0.72099999999999997}]},{"osd":4,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.69899999999999995},"min":{"1min":0.53400000000000003,"5min":0.437,"15min":0.39300000000000002},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":0.997},"last":0.77100000000000002}]},{"osd":6,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66900000000000004,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93400000000000005,"5min":0.99199999999999999,"15min":0.99299999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.751,"15min":0.72599999999999998},"min":{"1min":0.60299999999999998,"5min":0.46100000000000002,"15min":0.41499999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0449999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68500000000000005,"15min":0.66400000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.05},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72699999999999998,"15min":0.70099999999999996},"min":{"1min":0.48899999999999999,"5min":0.45700000000000002,"15min":0.436},"max":{"1min":0.95699999999999996,"5min":1.014,"15min":1.0780000000000001},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T21:59:45.755 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T21:59:45.756 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T21:59:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.989+0000 7f8eacdff700 1 Processor -- start 2023-11-06T21:59:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.989+0000 7f8eacdff700 1 -- start start 2023-11-06T21:59:45.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.990+0000 7f8eacdff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8ea81546a0 0x7f8ea8064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.990+0000 7f8eacdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea80be2c0 0x7f8ea8066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:45.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.990+0000 7f8eacdff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea80641f0 con 0x7f8ea8066560 2023-11-06T21:59:45.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.990+0000 7f8eacdff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea8154a80 con 0x7f8ea81546a0 2023-11-06T21:59:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.990+0000 7f8eacdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea80638c0 con 0x7f8ea80be2c0 2023-11-06T21:59:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.991+0000 7f8ea659c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8ea81546a0 0x7f8ea8064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.991+0000 7f8ea6d9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea80be2c0 0x7f8ea8066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.991+0000 7f8ea6d9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea80be2c0 0x7f8ea8066a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:42292/0 (socket says 172.21.15.33:42292) 2023-11-06T21:59:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.991+0000 7f8ea6d9d700 1 -- 172.21.15.33:0/1351225141 learned_addr learned my addr 172.21.15.33:0/1351225141 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.991+0000 7f8ea6d9d700 1 -- 172.21.15.33:0/1351225141 >> v2:172.21.15.69:3300/0 conn(0x7f8ea81546a0 msgr2=0x7f8ea8064680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.991+0000 7f8ea6d9d700 1 --2- 172.21.15.33:0/1351225141 >> v2:172.21.15.69:3300/0 conn(0x7f8ea81546a0 0x7f8ea8064680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.991+0000 7f8ea6d9d700 1 -- 172.21.15.33:0/1351225141 >> v1:172.21.15.69:6789/0 conn(0x7f8ea8066560 legacy=0x7f8ea80bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.991+0000 7f8ea6d9d700 1 -- 172.21.15.33:0/1351225141 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea8063ba0 con 0x7f8ea80be2c0 2023-11-06T21:59:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.991+0000 7f8ea659c700 1 --2- 172.21.15.33:0/1351225141 >> v2:172.21.15.69:3300/0 conn(0x7f8ea81546a0 0x7f8ea8064680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.991+0000 7f8ea6d9d700 1 --2- 172.21.15.33:0/1351225141 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea80be2c0 0x7f8ea8066a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f8ea0009f20 tx=0x7f8ea0009840).ready entity=mon.2 client_cookie=2c7f4f1102433edb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8ea559a700 1 -- 172.21.15.33:0/1351225141 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000e070 con 0x7f8ea80be2c0 2023-11-06T21:59:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8ea559a700 1 -- 172.21.15.33:0/1351225141 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea000b9d0 con 0x7f8ea80be2c0 2023-11-06T21:59:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8ea559a700 1 -- 172.21.15.33:0/1351225141 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000bd50 con 0x7f8ea80be2c0 2023-11-06T21:59:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8eacdff700 1 -- 172.21.15.33:0/1351225141 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea80be2c0 msgr2=0x7f8ea8066a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8eacdff700 1 --2- 172.21.15.33:0/1351225141 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea80be2c0 0x7f8ea8066a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f8ea0009f20 tx=0x7f8ea0009840).stop 2023-11-06T21:59:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8eacdff700 1 -- 172.21.15.33:0/1351225141 shutdown_connections 2023-11-06T21:59:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8eacdff700 1 --2- 172.21.15.33:0/1351225141 >> v2:172.21.15.69:3300/0 conn(0x7f8ea81546a0 0x7f8ea8064680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8eacdff700 1 --2- 172.21.15.33:0/1351225141 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea80be2c0 0x7f8ea8066a10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8eacdff700 1 -- 172.21.15.33:0/1351225141 >> 172.21.15.33:0/1351225141 conn(0x7f8ea80c8c50 msgr2=0x7f8ea80c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8eacdff700 1 -- 172.21.15.33:0/1351225141 shutdown_connections 2023-11-06T21:59:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.992+0000 7f8eacdff700 1 -- 172.21.15.33:0/1351225141 wait complete. 2023-11-06T21:59:46.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.994+0000 7f8eacdff700 1 Processor -- start 2023-11-06T21:59:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.994+0000 7f8eacdff700 1 -- start start 2023-11-06T21:59:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.995+0000 7f8eacdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea8066560 0x7f8ea8168320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.995+0000 7f8eacdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ea80be2c0 0x7f8ea80ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.995+0000 7f8eacdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea81546a0 0x7f8ea813ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.995+0000 7f8eacdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81522c0 con 0x7f8ea80be2c0 2023-11-06T21:59:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.995+0000 7f8eacdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8155220 con 0x7f8ea8066560 2023-11-06T21:59:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.995+0000 7f8eacdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8168810 con 0x7f8ea81546a0 2023-11-06T21:59:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.995+0000 7f8ea5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ea80be2c0 0x7f8ea80ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.995+0000 7f8ea5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ea80be2c0 0x7f8ea80ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58918/0 (socket says 172.21.15.33:58918) 2023-11-06T21:59:46.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.995+0000 7f8ea5d9b700 1 -- 172.21.15.33:0/955328922 learned_addr learned my addr 172.21.15.33:0/955328922 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:46.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.995+0000 7f8ea659c700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea8066560 0x7f8ea8168320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:46.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8ea6d9d700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea81546a0 0x7f8ea813ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:46.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8ea5d9b700 1 -- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea81546a0 msgr2=0x7f8ea813ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8ea5d9b700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea81546a0 0x7f8ea813ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8ea5d9b700 1 -- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea8066560 msgr2=0x7f8ea8168320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8ea5d9b700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea8066560 0x7f8ea8168320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8ea5d9b700 1 -- 172.21.15.33:0/955328922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e980092a0 con 0x7f8ea80be2c0 2023-11-06T21:59:46.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8ea5d9b700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ea80be2c0 0x7f8ea80ab140 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f8e9c00c4c0 tx=0x7f8e9c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:46.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8e967fc700 1 -- 172.21.15.33:0/955328922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c00ce70 con 0x7f8ea80be2c0 2023-11-06T21:59:46.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8e967fc700 1 -- 172.21.15.33:0/955328922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c001e90 con 0x7f8ea80be2c0 2023-11-06T21:59:46.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8e967fc700 1 -- 172.21.15.33:0/955328922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c007250 con 0x7f8ea80be2c0 2023-11-06T21:59:46.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.996+0000 7f8eacdff700 1 -- 172.21.15.33:0/955328922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea0009ae0 con 0x7f8ea80be2c0 2023-11-06T21:59:46.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.997+0000 7f8eacdff700 1 -- 172.21.15.33:0/955328922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea80aa420 con 0x7f8ea80be2c0 2023-11-06T21:59:46.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.998+0000 7f8eacdff700 1 -- 172.21.15.33:0/955328922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea8066000 con 0x7f8ea80be2c0 2023-11-06T21:59:46.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.998+0000 7f8e967fc700 1 -- 172.21.15.33:0/955328922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8e9c0066a0 con 0x7f8ea80be2c0 2023-11-06T21:59:46.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.999+0000 7f8e967fc700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e8805f270 0x7f8e88061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:46.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:45.999+0000 7f8e967fc700 1 -- 172.21.15.33:0/955328922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f8e9c079b60 con 0x7f8ea80be2c0 2023-11-06T21:59:46.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.000+0000 7f8ea659c700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e8805f270 0x7f8e88061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:46.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.000+0000 7f8ea659c700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e8805f270 0x7f8e88061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8e98009d20 tx=0x7f8e98009060).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:46.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.003+0000 7f8e967fc700 1 -- 172.21.15.33:0/955328922 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8e9c04abc0 con 0x7f8ea80be2c0 2023-11-06T21:59:46.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.252+0000 7f8eacdff700 1 -- 172.21.15.33:0/955328922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8ea80a7be0 con 0x7f8ea80be2c0 2023-11-06T21:59:46.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.256+0000 7f8e967fc700 1 -- 172.21.15.33:0/955328922 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v38) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8e9c04e1e0 con 0x7f8ea80be2c0 2023-11-06T21:59:46.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.258+0000 7f8eacdff700 1 -- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e8805f270 msgr2=0x7f8e88061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.258+0000 7f8eacdff700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e8805f270 0x7f8e88061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8e98009d20 tx=0x7f8e98009060).stop 2023-11-06T21:59:46.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.258+0000 7f8eacdff700 1 -- 172.21.15.33:0/955328922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ea80be2c0 msgr2=0x7f8ea80ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.258+0000 7f8eacdff700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ea80be2c0 0x7f8ea80ab140 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f8e9c00c4c0 tx=0x7f8e9c007e00).stop 2023-11-06T21:59:46.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.259+0000 7f8eacdff700 1 -- 172.21.15.33:0/955328922 shutdown_connections 2023-11-06T21:59:46.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.259+0000 7f8eacdff700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e8805f270 0x7f8e88061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.259+0000 7f8eacdff700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8ea81546a0 0x7f8ea813ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.259+0000 7f8eacdff700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea8066560 0x7f8ea8168320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.259+0000 7f8eacdff700 1 --2- 172.21.15.33:0/955328922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ea80be2c0 0x7f8ea80ab140 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.259+0000 7f8eacdff700 1 -- 172.21.15.33:0/955328922 >> 172.21.15.33:0/955328922 conn(0x7f8ea80c8c50 msgr2=0x7f8ea8152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:46.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.259+0000 7f8eacdff700 1 -- 172.21.15.33:0/955328922 shutdown_connections 2023-11-06T21:59:46.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.259+0000 7f8eacdff700 1 -- 172.21.15.33:0/955328922 wait complete. 2023-11-06T21:59:46.266 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T21:59:46.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:59:46.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.472+0000 7f8e78bff700 1 Processor -- start 2023-11-06T21:59:46.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.473+0000 7f8e78bff700 1 -- start start 2023-11-06T21:59:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.474+0000 7f8e78bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c1840 0x7f8e740a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.474+0000 7f8e78bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8e740c6460 0x7f8e740bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.474+0000 7f8e78bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e741554a0 con 0x7f8e740aa2d0 2023-11-06T21:59:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.474+0000 7f8e78bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e74154960 con 0x7f8e740c6460 2023-11-06T21:59:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.474+0000 7f8e78bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e74155930 con 0x7f8e740c1840 2023-11-06T21:59:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.475+0000 7f8e7259c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c1840 0x7f8e740a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.475+0000 7f8e72d9d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8e740c6460 0x7f8e740bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.475+0000 7f8e72d9d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8e740c6460 0x7f8e740bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:50700/0 (socket says 172.21.15.33:50700) 2023-11-06T21:59:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.475+0000 7f8e7259c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c1840 0x7f8e740a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:42312/0 (socket says 172.21.15.33:42312) 2023-11-06T21:59:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.475+0000 7f8e72d9d700 1 -- 172.21.15.33:0/1759573684 learned_addr learned my addr 172.21.15.33:0/1759573684 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:46.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.475+0000 7f8e7259c700 1 -- 172.21.15.33:0/1759573684 >> v2:172.21.15.69:3300/0 conn(0x7f8e740c6460 msgr2=0x7f8e740bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:46.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.475+0000 7f8e7259c700 1 --2- 172.21.15.33:0/1759573684 >> v2:172.21.15.69:3300/0 conn(0x7f8e740c6460 0x7f8e740bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.475+0000 7f8e7259c700 1 -- 172.21.15.33:0/1759573684 >> v1:172.21.15.69:6789/0 conn(0x7f8e740aa2d0 legacy=0x7f8e740c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.475+0000 7f8e7259c700 1 -- 172.21.15.33:0/1759573684 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e74154b70 con 0x7f8e740c1840 2023-11-06T21:59:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.475+0000 7f8e7259c700 1 --2- 172.21.15.33:0/1759573684 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c1840 0x7f8e740a9de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f8e64009760 tx=0x7f8e64009060).ready entity=mon.2 client_cookie=fbacc43ba85f41a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e7159a700 1 -- 172.21.15.33:0/1759573684 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400f510 con 0x7f8e740c1840 2023-11-06T21:59:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e7159a700 1 -- 172.21.15.33:0/1759573684 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6400f670 con 0x7f8e740c1840 2023-11-06T21:59:46.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e7159a700 1 -- 172.21.15.33:0/1759573684 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400f9f0 con 0x7f8e740c1840 2023-11-06T21:59:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e78bff700 1 -- 172.21.15.33:0/1759573684 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c1840 msgr2=0x7f8e740a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e78bff700 1 --2- 172.21.15.33:0/1759573684 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c1840 0x7f8e740a9de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f8e64009760 tx=0x7f8e64009060).stop 2023-11-06T21:59:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e78bff700 1 -- 172.21.15.33:0/1759573684 shutdown_connections 2023-11-06T21:59:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e78bff700 1 --2- 172.21.15.33:0/1759573684 >> v2:172.21.15.69:3300/0 conn(0x7f8e740c6460 0x7f8e740bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e78bff700 1 --2- 172.21.15.33:0/1759573684 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c1840 0x7f8e740a9de0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e78bff700 1 -- 172.21.15.33:0/1759573684 >> 172.21.15.33:0/1759573684 conn(0x7f8e7405ec80 msgr2=0x7f8e740cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:46.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e78bff700 1 -- 172.21.15.33:0/1759573684 shutdown_connections 2023-11-06T21:59:46.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.476+0000 7f8e78bff700 1 -- 172.21.15.33:0/1759573684 wait complete. 2023-11-06T21:59:46.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.478+0000 7f8e78bff700 1 Processor -- start 2023-11-06T21:59:46.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.478+0000 7f8e78bff700 1 -- start start 2023-11-06T21:59:46.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e78bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e740aa2d0 0x7f8e740c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:46.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e78bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e740c1840 0x7f8e740c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:46.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e78bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c6460 0x7f8e741643b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:46.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e78bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7414bca0 con 0x7f8e740c1840 2023-11-06T21:59:46.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e78bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7414da20 con 0x7f8e740aa2d0 2023-11-06T21:59:46.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e78bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740aefc0 con 0x7f8e740c6460 2023-11-06T21:59:46.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e71d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e740c1840 0x7f8e740c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:46.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e71d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e740c1840 0x7f8e740c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58920/0 (socket says 172.21.15.33:58920) 2023-11-06T21:59:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e71d9b700 1 -- 172.21.15.33:0/4012934748 learned_addr learned my addr 172.21.15.33:0/4012934748 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e7259c700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e740aa2d0 0x7f8e740c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.479+0000 7f8e72d9d700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c6460 0x7f8e741643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e71d9b700 1 -- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c6460 msgr2=0x7f8e741643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e71d9b700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c6460 0x7f8e741643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e71d9b700 1 -- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e740aa2d0 msgr2=0x7f8e740c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e71d9b700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e740aa2d0 0x7f8e740c29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e71d9b700 1 -- 172.21.15.33:0/4012934748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e640092a0 con 0x7f8e740c1840 2023-11-06T21:59:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e71d9b700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e740c1840 0x7f8e740c4b40 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f8e6800c470 tx=0x7f8e68007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e627fc700 1 -- 172.21.15.33:0/4012934748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68007020 con 0x7f8e740c1840 2023-11-06T21:59:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e627fc700 1 -- 172.21.15.33:0/4012934748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e68007180 con 0x7f8e740c1840 2023-11-06T21:59:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e78bff700 1 -- 172.21.15.33:0/4012934748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e740c2fc0 con 0x7f8e740c1840 2023-11-06T21:59:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e627fc700 1 -- 172.21.15.33:0/4012934748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68007520 con 0x7f8e740c1840 2023-11-06T21:59:46.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.480+0000 7f8e78bff700 1 -- 172.21.15.33:0/4012934748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e740c3580 con 0x7f8e740c1840 2023-11-06T21:59:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.482+0000 7f8e78bff700 1 -- 172.21.15.33:0/4012934748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e740c3770 con 0x7f8e740c1840 2023-11-06T21:59:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.482+0000 7f8e627fc700 1 -- 172.21.15.33:0/4012934748 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8e680068b0 con 0x7f8e740c1840 2023-11-06T21:59:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.483+0000 7f8e627fc700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e5405f270 0x7f8e54061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.483+0000 7f8e627fc700 1 -- 172.21.15.33:0/4012934748 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f8e68079da0 con 0x7f8e740c1840 2023-11-06T21:59:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.484+0000 7f8e7259c700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e5405f270 0x7f8e54061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.484+0000 7f8e7259c700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e5405f270 0x7f8e54061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f8e64009fd0 tx=0x7f8e64017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.487+0000 7f8e627fc700 1 -- 172.21.15.33:0/4012934748 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8e6804ae00 con 0x7f8e740c1840 2023-11-06T21:59:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.710+0000 7f8e78bff700 1 -- 172.21.15.33:0/4012934748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e7406c6c0 con 0x7f8e740c1840 2023-11-06T21:59:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.710+0000 7f8e627fc700 1 -- 172.21.15.33:0/4012934748 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f8e6804e420 con 0x7f8e740c1840 2023-11-06T21:59:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:46.714 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3645039939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3645039939},{"type":"v1","addr":"172.21.15.33:6835","nonce":3645039939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1156785194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1156785194},{"type":"v1","addr":"172.21.15.69:6836","nonce":1156785194}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/4183165491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":4183165491},{"type":"v1","addr":"172.21.15.69:6837","nonce":4183165491}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:59:40.865889+0000","modified":"2023-11-06T21:59:43.953031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8445":{"gid":8445,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/1588551062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1588551062},{"type":"v1","addr":"172.21.15.33:6833","nonce":1588551062}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-06T21:59:46.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 -- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e5405f270 msgr2=0x7f8e54061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e5405f270 0x7f8e54061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f8e64009fd0 tx=0x7f8e64017000).stop 2023-11-06T21:59:46.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 -- 172.21.15.33:0/4012934748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e740c1840 msgr2=0x7f8e740c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e740c1840 0x7f8e740c4b40 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f8e6800c470 tx=0x7f8e68007d40).stop 2023-11-06T21:59:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 -- 172.21.15.33:0/4012934748 shutdown_connections 2023-11-06T21:59:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8e5405f270 0x7f8e54061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8e740c6460 0x7f8e741643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e740aa2d0 0x7f8e740c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 --2- 172.21.15.33:0/4012934748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e740c1840 0x7f8e740c4b40 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 -- 172.21.15.33:0/4012934748 >> 172.21.15.33:0/4012934748 conn(0x7f8e7405ec80 msgr2=0x7f8e74153a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 -- 172.21.15.33:0/4012934748 shutdown_connections 2023-11-06T21:59:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:46.713+0000 7f8e78bff700 1 -- 172.21.15.33:0/4012934748 wait complete. 2023-11-06T21:59:46.718 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 68 2023-11-06T21:59:47.135 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T21:59:40.865889+0000', 'modified': '2023-11-06T21:59:43.953031+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8445': {'gid': 8445, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6833/1588551062', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6832', 'nonce': 1588551062}, {'type': 'v1', 'addr': '172.21.15.33:6833', 'nonce': 1588551062}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T21:59:47.135 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T21:59:47.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T21:59:47.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.333+0000 7fca615ff700 1 Processor -- start 2023-11-06T21:59:47.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.333+0000 7fca615ff700 1 -- start start 2023-11-06T21:59:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.334+0000 7fca615ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 0x7fca5c137780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.334+0000 7fca615ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fca5c0c1020 0x7fca5c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.334+0000 7fca615ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c06a470 con 0x7fca5c0c3770 2023-11-06T21:59:47.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.334+0000 7fca615ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca5c06a190 con 0x7fca5c0c1020 2023-11-06T21:59:47.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.334+0000 7fca615ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0bbab0 con 0x7fca5c1373a0 2023-11-06T21:59:47.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.335+0000 7fca5b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fca5c0c3770 0x7fca5c069440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57698/0 (socket says 172.21.15.33:57698) 2023-11-06T21:59:47.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.335+0000 7fca5b7fe700 1 -- 172.21.15.33:0/2665393516 learned_addr learned my addr 172.21.15.33:0/2665393516 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:47.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.335+0000 7fca5affd700 1 --2- 172.21.15.33:0/2665393516 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 0x7fca5c137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.335+0000 7fca5a7fc700 1 --2- 172.21.15.33:0/2665393516 >> v2:172.21.15.69:3300/0 conn(0x7fca5c0c1020 0x7fca5c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.335+0000 7fca59ffb700 1 -- 172.21.15.33:0/2665393516 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170950503 0 0) 0x7fca5c06a470 con 0x7fca5c0c3770 2023-11-06T21:59:47.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.335+0000 7fca59ffb700 1 -- 172.21.15.33:0/2665393516 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca440033f0 con 0x7fca5c0c3770 2023-11-06T21:59:47.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.335+0000 7fca5affd700 1 -- 172.21.15.33:0/2665393516 >> v2:172.21.15.69:3300/0 conn(0x7fca5c0c1020 msgr2=0x7fca5c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:47.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.335+0000 7fca5affd700 1 --2- 172.21.15.33:0/2665393516 >> v2:172.21.15.69:3300/0 conn(0x7fca5c0c1020 0x7fca5c0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.335+0000 7fca5affd700 1 -- 172.21.15.33:0/2665393516 >> v1:172.21.15.33:6789/0 conn(0x7fca5c0c3770 legacy=0x7fca5c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.335+0000 7fca5affd700 1 -- 172.21.15.33:0/2665393516 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c0bbd90 con 0x7fca5c1373a0 2023-11-06T21:59:47.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca5affd700 1 --2- 172.21.15.33:0/2665393516 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 0x7fca5c137780 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fca4c009760 tx=0x7fca4c009060).ready entity=mon.2 client_cookie=6a7b0063b8c3c36 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:47.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca59ffb700 1 -- 172.21.15.33:0/2665393516 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c0090d0 con 0x7fca5c1373a0 2023-11-06T21:59:47.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca59ffb700 1 -- 172.21.15.33:0/2665393516 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c00d4a0 con 0x7fca5c1373a0 2023-11-06T21:59:47.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca59ffb700 1 -- 172.21.15.33:0/2665393516 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00d820 con 0x7fca5c1373a0 2023-11-06T21:59:47.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca615ff700 1 -- 172.21.15.33:0/2665393516 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 msgr2=0x7fca5c137780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca615ff700 1 --2- 172.21.15.33:0/2665393516 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 0x7fca5c137780 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fca4c009760 tx=0x7fca4c009060).stop 2023-11-06T21:59:47.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca615ff700 1 -- 172.21.15.33:0/2665393516 shutdown_connections 2023-11-06T21:59:47.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca615ff700 1 --2- 172.21.15.33:0/2665393516 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 0x7fca5c137780 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca615ff700 1 --2- 172.21.15.33:0/2665393516 >> v2:172.21.15.69:3300/0 conn(0x7fca5c0c1020 0x7fca5c0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca615ff700 1 -- 172.21.15.33:0/2665393516 >> 172.21.15.33:0/2665393516 conn(0x7fca5c062710 msgr2=0x7fca5c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:47.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca615ff700 1 -- 172.21.15.33:0/2665393516 shutdown_connections 2023-11-06T21:59:47.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.336+0000 7fca615ff700 1 -- 172.21.15.33:0/2665393516 wait complete. 2023-11-06T21:59:47.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.338+0000 7fca615ff700 1 Processor -- start 2023-11-06T21:59:47.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.338+0000 7fca615ff700 1 -- start start 2023-11-06T21:59:47.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca615ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca5c0c1020 0x7fca5c154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca615ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca5c0c3770 0x7fca5c135910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca615ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 0x7fca5c153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca615ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0c0600 con 0x7fca5c0c1020 2023-11-06T21:59:47.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca615ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c153ba0 con 0x7fca5c0c3770 2023-11-06T21:59:47.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca615ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c154a00 con 0x7fca5c1373a0 2023-11-06T21:59:47.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca5affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca5c0c1020 0x7fca5c154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca5affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca5c0c1020 0x7fca5c154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58930/0 (socket says 172.21.15.33:58930) 2023-11-06T21:59:47.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca5affd700 1 -- 172.21.15.33:0/883284006 learned_addr learned my addr 172.21.15.33:0/883284006 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:47.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca5b7fe700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 0x7fca5c153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.339+0000 7fca5a7fc700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca5c0c3770 0x7fca5c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca5affd700 1 -- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 msgr2=0x7fca5c153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca5affd700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 0x7fca5c153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca5affd700 1 -- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca5c0c3770 msgr2=0x7fca5c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca5affd700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca5c0c3770 0x7fca5c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca5affd700 1 -- 172.21.15.33:0/883284006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0092a0 con 0x7fca5c0c1020 2023-11-06T21:59:47.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca5affd700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca5c0c1020 0x7fca5c154e90 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7fca4c00d940 tx=0x7fca4c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:47.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca42ffd700 1 -- 172.21.15.33:0/883284006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c012830 con 0x7fca5c0c1020 2023-11-06T21:59:47.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca42ffd700 1 -- 172.21.15.33:0/883284006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c012990 con 0x7fca5c0c1020 2023-11-06T21:59:47.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca615ff700 1 -- 172.21.15.33:0/883284006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5c153df0 con 0x7fca5c0c1020 2023-11-06T21:59:47.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca42ffd700 1 -- 172.21.15.33:0/883284006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c012d30 con 0x7fca5c0c1020 2023-11-06T21:59:47.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.340+0000 7fca615ff700 1 -- 172.21.15.33:0/883284006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c1543b0 con 0x7fca5c0c1020 2023-11-06T21:59:47.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.342+0000 7fca42ffd700 1 -- 172.21.15.33:0/883284006 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fca4c01c630 con 0x7fca5c0c1020 2023-11-06T21:59:47.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.343+0000 7fca42ffd700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca4805f240 0x7fca48061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.343+0000 7fca42ffd700 1 -- 172.21.15.33:0/883284006 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fca4c07b970 con 0x7fca5c0c1020 2023-11-06T21:59:47.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.344+0000 7fca5a7fc700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca4805f240 0x7fca48061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.344+0000 7fca5a7fc700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca4805f240 0x7fca48061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fca5c155800 tx=0x7fca50008040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:47.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.345+0000 7fca78870b80 1 Processor -- start 2023-11-06T21:59:47.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.345+0000 7fca78870b80 1 -- start start 2023-11-06T21:59:47.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.346+0000 7fca78870b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.346+0000 7fca78870b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d641c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.346+0000 7fca78870b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641f44c0 0x563d641c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.346+0000 7fca78870b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d641082f0 con 0x563d641cc080 2023-11-06T21:59:47.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.346+0000 7fca78870b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6411d7e0 con 0x563d641f44c0 2023-11-06T21:59:47.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.346+0000 7fca78870b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d6411d920 con 0x563d64214be0 2023-11-06T21:59:47.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.346+0000 7fca5affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58932/0 (socket says 172.21.15.33:58932) 2023-11-06T21:59:47.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5affd700 1 -- 172.21.15.33:0/1306294413 learned_addr learned my addr 172.21.15.33:0/1306294413 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:47.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5a7fc700 1 --2- 172.21.15.33:0/1306294413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d641c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5b7fe700 1 --2- 172.21.15.33:0/1306294413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641f44c0 0x563d641c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5affd700 1 -- 172.21.15.33:0/1306294413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 msgr2=0x563d641c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:47.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5affd700 1 --2- 172.21.15.33:0/1306294413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d641c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5affd700 1 -- 172.21.15.33:0/1306294413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641f44c0 msgr2=0x563d641c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:47.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5affd700 1 --2- 172.21.15.33:0/1306294413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641f44c0 0x563d641c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5affd700 1 -- 172.21.15.33:0/1306294413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d64116720 con 0x563d641cc080 2023-11-06T21:59:47.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5b7fe700 1 --2- 172.21.15.33:0/1306294413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641f44c0 0x563d641c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:47.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca5affd700 1 --2- 172.21.15.33:0/1306294413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641cc460 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7fca4c000e50 tx=0x7fca4c00e000).ready entity=mon.0 client_cookie=d0e5123e9a90a854 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:47.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca3bfff700 1 -- 172.21.15.33:0/1306294413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c04e7d0 con 0x563d641cc080 2023-11-06T21:59:47.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca3bfff700 1 -- 172.21.15.33:0/1306294413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c04ea30 con 0x563d641cc080 2023-11-06T21:59:47.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.347+0000 7fca3bfff700 1 -- 172.21.15.33:0/1306294413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c04fb80 con 0x563d641cc080 2023-11-06T21:59:47.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.348+0000 7fca78870b80 1 -- 172.21.15.33:0/1306294413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 msgr2=0x563d641cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.348+0000 7fca78870b80 1 --2- 172.21.15.33:0/1306294413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641cc460 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7fca4c000e50 tx=0x7fca4c00e000).stop 2023-11-06T21:59:47.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.348+0000 7fca78870b80 1 -- 172.21.15.33:0/1306294413 shutdown_connections 2023-11-06T21:59:47.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.348+0000 7fca78870b80 1 --2- 172.21.15.33:0/1306294413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641f44c0 0x563d641c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.348+0000 7fca78870b80 1 --2- 172.21.15.33:0/1306294413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641cc460 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.348+0000 7fca78870b80 1 --2- 172.21.15.33:0/1306294413 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d641c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.348+0000 7fca78870b80 1 -- 172.21.15.33:0/1306294413 >> 172.21.15.33:0/1306294413 conn(0x563d64035940 msgr2=0x563d64215070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:47.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.348+0000 7fca78870b80 1 -- 172.21.15.33:0/1306294413 shutdown_connections 2023-11-06T21:59:47.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.348+0000 7fca78870b80 1 -- 172.21.15.33:0/1306294413 wait complete. 2023-11-06T21:59:47.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.349+0000 7fca78870b80 1 -- start start 2023-11-06T21:59:47.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.349+0000 7fca78870b80 1 Processor -- start 2023-11-06T21:59:47.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.350+0000 7fca78870b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 0x563d64210cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.350+0000 7fca78870b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641f44c0 0x563d64220840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.350+0000 7fca78870b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d64221a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.350+0000 7fca78870b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6411dba0 con 0x563d641f44c0 2023-11-06T21:59:47.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.350+0000 7fca78870b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d640e8e60 con 0x563d641cc080 2023-11-06T21:59:47.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.350+0000 7fca78870b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d64222850 con 0x563d64214be0 2023-11-06T21:59:47.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.350+0000 7fca78870b80 10 client.0 ll_register_callbacks cb 0x563d640bdce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T21:59:47.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.350+0000 7fca5a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641f44c0 0x563d64220840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.350+0000 7fca5a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641f44c0 0x563d64220840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58944/0 (socket says 172.21.15.33:58944) 2023-11-06T21:59:47.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.350+0000 7fca5a7fc700 1 -- 172.21.15.33:0/1180301757 learned_addr learned my addr 172.21.15.33:0/1180301757 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:47.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca5b7fe700 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d64221a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca5affd700 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 0x563d64210cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca5a7fc700 1 -- 172.21.15.33:0/1180301757 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 msgr2=0x563d64221a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:47.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca5a7fc700 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d64221a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca5a7fc700 1 -- 172.21.15.33:0/1180301757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 msgr2=0x563d64210cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:47.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca5a7fc700 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 0x563d64210cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca5a7fc700 1 -- 172.21.15.33:0/1180301757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c1543b0 con 0x563d641f44c0 2023-11-06T21:59:47.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca5b7fe700 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d64221a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:47.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca5affd700 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 0x563d64210cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:47.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca5a7fc700 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641f44c0 0x563d64220840 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fca5c0c7a30 tx=0x7fca50018040).ready entity=mon.0 client_cookie=e2a909c304df5600 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:47.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca397fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T21:59:47.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca397fa700 1 -- 172.21.15.33:0/1180301757 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca500145f0 con 0x563d641f44c0 2023-11-06T21:59:47.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca78870b80 1 -- 172.21.15.33:0/1180301757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d641bfe20 con 0x563d641f44c0 2023-11-06T21:59:47.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca397fa700 1 -- 172.21.15.33:0/1180301757 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fca50014750 con 0x563d641f44c0 2023-11-06T21:59:47.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca397fa700 1 -- 172.21.15.33:0/1180301757 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca50014b20 con 0x563d641f44c0 2023-11-06T21:59:47.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca78870b80 10 client.8592 fetch_fsmap learned FSMap version 68 2023-11-06T21:59:47.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.351+0000 7fca78870b80 1 -- 172.21.15.33:0/1180301757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x563d64111ac0 con 0x563d641f44c0 2023-11-06T21:59:47.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.352+0000 7fca397fa700 1 -- 172.21.15.33:0/1180301757 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563d641bfe20 con 0x563d641f44c0 2023-11-06T21:59:47.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.352+0000 7fca397fa700 1 -- 172.21.15.33:0/1180301757 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 68) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fca50014c80 con 0x563d641f44c0 2023-11-06T21:59:47.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.352+0000 7fca78870b80 10 client.8592 fetch_fsmap finished waiting for FSMap version 68 2023-11-06T21:59:47.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.352+0000 7fca78870b80 10 client.8592 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T21:59:47.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.352+0000 7fca78870b80 20 client.8592 populate_metadata read hostname 'smithi033' 2023-11-06T21:59:47.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.352+0000 7fca78870b80 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.33:6832/1588551062,v1:172.21.15.33:6833/1588551062] conn(0x563d641ee380 0x563d641ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.352+0000 7fca78870b80 4 client.8592 mds_command: new command op to 8445 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T21:59:47.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.352+0000 7fca78870b80 1 -- 172.21.15.33:0/1180301757 --> [v2:172.21.15.33:6832/1588551062,v1:172.21.15.33:6833/1588551062] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d63f85ef0 con 0x563d641ee380 2023-11-06T21:59:47.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.352+0000 7fca5affd700 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.33:6832/1588551062,v1:172.21.15.33:6833/1588551062] conn(0x563d641ee380 0x563d641ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.353+0000 7fca5affd700 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.33:6832/1588551062,v1:172.21.15.33:6833/1588551062] conn(0x563d641ee380 0x563d641ee760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b689e47bd3ca4846 server_cookie=cd87143a0868ea13 in_seq=0 out_seq=0 2023-11-06T21:59:47.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.353+0000 7fca397fa700 10 client.8592 ms_handle_connect on v2:172.21.15.33:6832/1588551062 2023-11-06T21:59:47.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.353+0000 7fca397fa700 1 -- 172.21.15.33:0/1180301757 <== mds.0 v2:172.21.15.33:6832/1588551062 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x563d63f85ef0 con 0x563d641ee380 2023-11-06T21:59:47.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.353+0000 7fca397fa700 10 client.8592 handle_command_reply: tid=0 2023-11-06T21:59:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.353+0000 7fca78870b80 1 client.8592 shutdown 2023-11-06T21:59:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 1 -- 172.21.15.33:0/1180301757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641f44c0 msgr2=0x563d64220840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641f44c0 0x563d64220840 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fca5c0c7a30 tx=0x7fca50018040).stop 2023-11-06T21:59:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 1 -- 172.21.15.33:0/1180301757 shutdown_connections 2023-11-06T21:59:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d64221a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.33:6832/1588551062,v1:172.21.15.33:6833/1588551062] conn(0x563d641ee380 0x563d641ee760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 0x563d64210cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 1 --2- 172.21.15.33:0/1180301757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641f44c0 0x563d64220840 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 1 -- 172.21.15.33:0/1180301757 >> 172.21.15.33:0/1180301757 conn(0x563d64035940 msgr2=0x563d641c8200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:47.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca397fa700 0 client.8592 ms_handle_reset on v2:172.21.15.33:6832/1588551062 2023-11-06T21:59:47.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 1 -- 172.21.15.33:0/1180301757 shutdown_connections 2023-11-06T21:59:47.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 1 -- 172.21.15.33:0/1180301757 wait complete. 2023-11-06T21:59:47.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.354+0000 7fca78870b80 20 client.8592 trim_cache size 0 max 16384 2023-11-06T21:59:47.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.371+0000 7fca78870b80 1 Processor -- start 2023-11-06T21:59:47.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.371+0000 7fca78870b80 1 -- start start 2023-11-06T21:59:47.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca78870b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641faac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca78870b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641f44c0 0x563d641ed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca78870b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d641ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca78870b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d641ef6c0 con 0x563d641cc080 2023-11-06T21:59:47.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca78870b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d641fafb0 con 0x563d641f44c0 2023-11-06T21:59:47.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca78870b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d641f0210 con 0x563d64214be0 2023-11-06T21:59:47.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca5affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca5affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641faac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58952/0 (socket says 172.21.15.33:58952) 2023-11-06T21:59:47.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca5affd700 1 -- 172.21.15.33:0/254547362 learned_addr learned my addr 172.21.15.33:0/254547362 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:47.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca5affd700 1 -- 172.21.15.33:0/254547362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 msgr2=0x563d641ee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:47.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca5affd700 1 --2- 172.21.15.33:0/254547362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d641ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:47.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca5affd700 1 -- 172.21.15.33:0/254547362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641f44c0 msgr2=0x563d641ed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:47.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca5affd700 1 --2- 172.21.15.33:0/254547362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641f44c0 0x563d641ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:47.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca5affd700 1 -- 172.21.15.33:0/254547362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d64111810 con 0x563d641cc080 2023-11-06T21:59:47.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.373+0000 7fca5affd700 1 --2- 172.21.15.33:0/254547362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641faac0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7fca4c079c00 tx=0x7fca4c033040).ready entity=mon.0 client_cookie=8b47746e533e8dda server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:47.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca409ff700 1 -- 172.21.15.33:0/254547362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c036070 con 0x563d641cc080 2023-11-06T21:59:47.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca409ff700 1 -- 172.21.15.33:0/254547362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c037040 con 0x563d641cc080 2023-11-06T21:59:47.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca409ff700 1 -- 172.21.15.33:0/254547362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c03a280 con 0x563d641cc080 2023-11-06T21:59:47.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca78870b80 1 -- 172.21.15.33:0/254547362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 msgr2=0x563d641faac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca78870b80 1 --2- 172.21.15.33:0/254547362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641faac0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7fca4c079c00 tx=0x7fca4c033040).stop 2023-11-06T21:59:47.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca78870b80 1 -- 172.21.15.33:0/254547362 shutdown_connections 2023-11-06T21:59:47.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca78870b80 1 --2- 172.21.15.33:0/254547362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d64214be0 0x563d641ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:47.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca78870b80 1 --2- 172.21.15.33:0/254547362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d641cc080 0x563d641faac0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca78870b80 1 --2- 172.21.15.33:0/254547362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641f44c0 0x563d641ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:47.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca78870b80 1 -- 172.21.15.33:0/254547362 >> 172.21.15.33:0/254547362 conn(0x563d64035940 msgr2=0x563d64215070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:47.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca78870b80 1 -- 172.21.15.33:0/254547362 shutdown_connections 2023-11-06T21:59:47.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.374+0000 7fca78870b80 1 -- 172.21.15.33:0/254547362 wait complete. 2023-11-06T21:59:47.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.375+0000 7fca78870b80 1 -- start start 2023-11-06T21:59:47.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.375+0000 7fca78870b80 1 Processor -- start 2023-11-06T21:59:47.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca78870b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 0x563d641ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca78870b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d641f44c0 0x563d641eec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca78870b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d64214be0 0x563d641c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca78870b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d641f31a0 con 0x563d64214be0 2023-11-06T21:59:47.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca78870b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d641f24e0 con 0x563d641cc080 2023-11-06T21:59:47.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca78870b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d641ef140 con 0x563d641f44c0 2023-11-06T21:59:47.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca78870b80 10 client.0 ll_register_callbacks cb 0x563d640bdce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T21:59:47.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d64214be0 0x563d641c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca5a7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d641f44c0 0x563d641eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d64214be0 0x563d641c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58964/0 (socket says 172.21.15.33:58964) 2023-11-06T21:59:47.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca5b7fe700 1 -- 172.21.15.33:0/1454829682 learned_addr learned my addr 172.21.15.33:0/1454829682 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:47.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca5a7fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d641f44c0 0x563d641eec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:42364/0 (socket says 172.21.15.33:42364) 2023-11-06T21:59:47.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.377+0000 7fca5affd700 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 0x563d641ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca5b7fe700 1 -- 172.21.15.33:0/1454829682 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d641f44c0 msgr2=0x563d641eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:47.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca5b7fe700 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d641f44c0 0x563d641eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca5b7fe700 1 -- 172.21.15.33:0/1454829682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 msgr2=0x563d641ee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:47.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca5b7fe700 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 0x563d641ee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca5b7fe700 1 -- 172.21.15.33:0/1454829682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d64116720 con 0x563d64214be0 2023-11-06T21:59:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca5a7fc700 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d641f44c0 0x563d641eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca5affd700 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 0x563d641ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca5b7fe700 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d64214be0 0x563d641c6110 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7fca54007fd0 tx=0x7fca54009040).ready entity=mon.0 client_cookie=425565c35fc37e01 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca397fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T21:59:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca397fa700 1 -- 172.21.15.33:0/1454829682 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54001d90 con 0x563d64214be0 2023-11-06T21:59:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca78870b80 1 -- 172.21.15.33:0/1454829682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca4c04ea30 con 0x563d64214be0 2023-11-06T21:59:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.378+0000 7fca397fa700 1 -- 172.21.15.33:0/1454829682 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fca54001ef0 con 0x563d64214be0 2023-11-06T21:59:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca397fa700 1 -- 172.21.15.33:0/1454829682 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca540020b0 con 0x563d64214be0 2023-11-06T21:59:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca78870b80 10 client.8598 fetch_fsmap learned FSMap version 68 2023-11-06T21:59:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca78870b80 1 -- 172.21.15.33:0/1454829682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x563d63f8b1d0 con 0x563d64214be0 2023-11-06T21:59:47.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca397fa700 1 -- 172.21.15.33:0/1454829682 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c04ea30 con 0x563d64214be0 2023-11-06T21:59:47.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca397fa700 1 -- 172.21.15.33:0/1454829682 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 68) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fca54016690 con 0x563d64214be0 2023-11-06T21:59:47.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca78870b80 10 client.8598 fetch_fsmap finished waiting for FSMap version 68 2023-11-06T21:59:47.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca78870b80 10 client.8598 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T21:59:47.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca78870b80 20 client.8598 populate_metadata read hostname 'smithi033' 2023-11-06T21:59:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca78870b80 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.33:6832/1588551062,v1:172.21.15.33:6833/1588551062] conn(0x563d641c6810 0x563d6420d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca78870b80 4 client.8598 mds_command: new command op to 8445 tid=0[{"prefix": "status"}] 2023-11-06T21:59:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca78870b80 1 -- 172.21.15.33:0/1454829682 --> [v2:172.21.15.33:6832/1588551062,v1:172.21.15.33:6833/1588551062] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563d63fbb040 con 0x563d641c6810 2023-11-06T21:59:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.379+0000 7fca5affd700 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.33:6832/1588551062,v1:172.21.15.33:6833/1588551062] conn(0x563d641c6810 0x563d6420d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca397fa700 10 client.8598 ms_handle_connect on v2:172.21.15.33:6832/1588551062 2023-11-06T21:59:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca5affd700 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.33:6832/1588551062,v1:172.21.15.33:6833/1588551062] conn(0x563d641c6810 0x563d6420d5f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=fa2723ea56b53404 server_cookie=6d6fb93cce177b22 in_seq=0 out_seq=0 2023-11-06T21:59:47.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca397fa700 1 -- 172.21.15.33:0/1454829682 <== mds.0 v2:172.21.15.33:6832/1588551062 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x563d63fbb040 con 0x563d641c6810 2023-11-06T21:59:47.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca397fa700 10 client.8598 handle_command_reply: tid=0 2023-11-06T21:59:47.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 client.8598 shutdown 2023-11-06T21:59:47.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 -- 172.21.15.33:0/1454829682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d64214be0 msgr2=0x563d641c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d64214be0 0x563d641c6110 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7fca54007fd0 tx=0x7fca54009040).stop 2023-11-06T21:59:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 -- 172.21.15.33:0/1454829682 shutdown_connections 2023-11-06T21:59:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.33:6832/1588551062,v1:172.21.15.33:6833/1588551062] conn(0x563d641c6810 0x563d6420d5f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d64214be0 0x563d641c6110 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563d641cc080 0x563d641ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca397fa700 0 client.8598 ms_handle_reset on v2:172.21.15.33:6832/1588551062 2023-11-06T21:59:47.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 --2- 172.21.15.33:0/1454829682 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x563d641f44c0 0x563d641eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 -- 172.21.15.33:0/1454829682 >> 172.21.15.33:0/1454829682 conn(0x563d64035940 msgr2=0x563d64221680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:47.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 -- 172.21.15.33:0/1454829682 shutdown_connections 2023-11-06T21:59:47.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.380+0000 7fca78870b80 1 -- 172.21.15.33:0/1454829682 wait complete. 2023-11-06T21:59:47.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca78870b80 20 client.8598 trim_cache size 0 max 16384 2023-11-06T21:59:47.400 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T21:59:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T21:59:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T21:59:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 8445, 2023-11-06T21:59:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T21:59:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T21:59:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T21:59:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.5129270039999998, 2023-11-06T21:59:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 68, 2023-11-06T21:59:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 170, 2023-11-06T21:59:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 170, 2023-11-06T21:59:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.711543477999999 2023-11-06T21:59:47.402 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T21:59:47.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 -- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca4805f240 msgr2=0x7fca48061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca4805f240 0x7fca48061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fca5c155800 tx=0x7fca50008040).stop 2023-11-06T21:59:47.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 -- 172.21.15.33:0/883284006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca5c0c1020 msgr2=0x7fca5c154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca5c0c1020 0x7fca5c154e90 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7fca4c00d940 tx=0x7fca4c016000).stop 2023-11-06T21:59:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 -- 172.21.15.33:0/883284006 shutdown_connections 2023-11-06T21:59:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fca4805f240 0x7fca48061700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fca5c1373a0 0x7fca5c153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca5c0c1020 0x7fca5c154e90 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 --2- 172.21.15.33:0/883284006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca5c0c3770 0x7fca5c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 -- 172.21.15.33:0/883284006 >> 172.21.15.33:0/883284006 conn(0x7fca5c062710 msgr2=0x7fca5c0bc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:47.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 -- 172.21.15.33:0/883284006 shutdown_connections 2023-11-06T21:59:47.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.381+0000 7fca615ff700 1 -- 172.21.15.33:0/883284006 wait complete. 2023-11-06T21:59:47.406 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T21:59:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.570+0000 7f78e3bff700 1 Processor -- start 2023-11-06T21:59:47.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.570+0000 7f78e3bff700 1 -- start start 2023-11-06T21:59:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.570+0000 7f78e3bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78dc0a9ee0 0x7f78dc0c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.570+0000 7f78e3bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78dc0c07b0 0x7f78dc0c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.570+0000 7f78e3bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78dc0c5f80 con 0x7f78dc1541a0 2023-11-06T21:59:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.570+0000 7f78e3bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78dc0aa740 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.570+0000 7f78e3bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0c6800 con 0x7f78dc0c07b0 2023-11-06T21:59:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.571+0000 7f78e119a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78dc0a9ee0 0x7f78dc0c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.571+0000 7f78e119a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78dc0a9ee0 0x7f78dc0c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58966/0 (socket says 172.21.15.33:58966) 2023-11-06T21:59:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.571+0000 7f78e119a700 1 -- 172.21.15.33:0/935357040 learned_addr learned my addr 172.21.15.33:0/935357040 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.571+0000 7f78e219c700 1 --2- 172.21.15.33:0/935357040 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78dc0c07b0 0x7f78dc0c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.571+0000 7f78e119a700 1 -- 172.21.15.33:0/935357040 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78dc0c07b0 msgr2=0x7f78dc0c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.571+0000 7f78e119a700 1 --2- 172.21.15.33:0/935357040 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78dc0c07b0 0x7f78dc0c0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.571+0000 7f78e119a700 1 -- 172.21.15.33:0/935357040 >> v1:172.21.15.69:6789/0 conn(0x7f78dc1541a0 legacy=0x7f78dc0c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.571+0000 7f78e119a700 1 -- 172.21.15.33:0/935357040 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc0c1130 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.571+0000 7f78e119a700 1 --2- 172.21.15.33:0/935357040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 0x7f78dc0c6c90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f78cc006fd0 tx=0x7f78cc0090f0).ready entity=mon.0 client_cookie=58b26781ee8ad328 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:47.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.571+0000 7f78e0999700 1 -- 172.21.15.33:0/935357040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00b580 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.572+0000 7f78e0999700 1 -- 172.21.15.33:0/935357040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc00b6e0 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.572+0000 7f78e0999700 1 -- 172.21.15.33:0/935357040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00ba60 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.572+0000 7f78e3bff700 1 -- 172.21.15.33:0/935357040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 msgr2=0x7f78dc0c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.572+0000 7f78e3bff700 1 --2- 172.21.15.33:0/935357040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 0x7f78dc0c6c90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f78cc006fd0 tx=0x7f78cc0090f0).stop 2023-11-06T21:59:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.572+0000 7f78e3bff700 1 -- 172.21.15.33:0/935357040 shutdown_connections 2023-11-06T21:59:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.572+0000 7f78e3bff700 1 --2- 172.21.15.33:0/935357040 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78dc0c07b0 0x7f78dc0c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.572+0000 7f78e3bff700 1 --2- 172.21.15.33:0/935357040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 0x7f78dc0c6c90 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.572+0000 7f78e3bff700 1 -- 172.21.15.33:0/935357040 >> 172.21.15.33:0/935357040 conn(0x7f78dc0944e0 msgr2=0x7f78dc06a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:47.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.572+0000 7f78e3bff700 1 -- 172.21.15.33:0/935357040 shutdown_connections 2023-11-06T21:59:47.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.572+0000 7f78e3bff700 1 -- 172.21.15.33:0/935357040 wait complete. 2023-11-06T21:59:47.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.574+0000 7f78e3bff700 1 Processor -- start 2023-11-06T21:59:47.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.574+0000 7f78e3bff700 1 -- start start 2023-11-06T21:59:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e3bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 0x7f78dc0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e3bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78dc0c07b0 0x7f78dc164cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e3bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc1541a0 0x7f78dc0ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e3bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0abc40 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e3bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc1651e0 con 0x7f78dc1541a0 2023-11-06T21:59:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e3bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0ab030 con 0x7f78dc0c07b0 2023-11-06T21:59:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 0x7f78dc0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 0x7f78dc0aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58974/0 (socket says 172.21.15.33:58974) 2023-11-06T21:59:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e199b700 1 -- 172.21.15.33:0/2812723874 learned_addr learned my addr 172.21.15.33:0/2812723874 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e119a700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78dc0c07b0 0x7f78dc164cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e219c700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc1541a0 0x7f78dc0ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e199b700 1 -- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78dc0c07b0 msgr2=0x7f78dc164cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e199b700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78dc0c07b0 0x7f78dc164cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e199b700 1 -- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc1541a0 msgr2=0x7f78dc0ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e199b700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc1541a0 0x7f78dc0ac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.575+0000 7f78e199b700 1 -- 172.21.15.33:0/2812723874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc009370 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.576+0000 7f78e199b700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 0x7f78dc0aff20 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f78d800c440 tx=0x7f78d8007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.576+0000 7f78d1ffb700 1 -- 172.21.15.33:0/2812723874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8007020 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.576+0000 7f78d1ffb700 1 -- 172.21.15.33:0/2812723874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d8007180 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.576+0000 7f78e3bff700 1 -- 172.21.15.33:0/2812723874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78dc0ab310 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.576+0000 7f78d1ffb700 1 -- 172.21.15.33:0/2812723874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8007520 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.576+0000 7f78e3bff700 1 -- 172.21.15.33:0/2812723874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc0ab8d0 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.577+0000 7f78e3bff700 1 -- 172.21.15.33:0/2812723874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78dc0475a0 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.578+0000 7f78d1ffb700 1 -- 172.21.15.33:0/2812723874 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f78d8006900 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.579+0000 7f78d1ffb700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78c005f270 0x7f78c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.579+0000 7f78d1ffb700 1 -- 172.21.15.33:0/2812723874 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f78d80798b0 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.579+0000 7f78e119a700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78c005f270 0x7f78c0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.580+0000 7f78e119a700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78c005f270 0x7f78c0061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f78cc009fd0 tx=0x7f78cc017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.583+0000 7f78d1ffb700 1 -- 172.21.15.33:0/2812723874 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f78d8047af0 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.805+0000 7f78e3bff700 1 -- 172.21.15.33:0/2812723874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f78dc0aa460 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.806+0000 7f78d1ffb700 1 -- 172.21.15.33:0/2812723874 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7f78d804a9d0 con 0x7f78dc0a9ee0 2023-11-06T21:59:47.809 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T21:59:47.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 -- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78c005f270 msgr2=0x7f78c0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78c005f270 0x7f78c0061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f78cc009fd0 tx=0x7f78cc017000).stop 2023-11-06T21:59:47.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 -- 172.21.15.33:0/2812723874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 msgr2=0x7f78dc0aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:47.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 0x7f78dc0aff20 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f78d800c440 tx=0x7f78d8007e50).stop 2023-11-06T21:59:47.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 -- 172.21.15.33:0/2812723874 shutdown_connections 2023-11-06T21:59:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78c005f270 0x7f78c0061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc1541a0 0x7f78dc0ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78dc0a9ee0 0x7f78dc0aff20 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 --2- 172.21.15.33:0/2812723874 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78dc0c07b0 0x7f78dc164cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 -- 172.21.15.33:0/2812723874 >> 172.21.15.33:0/2812723874 conn(0x7f78dc0944e0 msgr2=0x7f78dc0c9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:47.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 -- 172.21.15.33:0/2812723874 shutdown_connections 2023-11-06T21:59:47.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:47.808+0000 7f78e3bff700 1 -- 172.21.15.33:0/2812723874 wait complete. 2023-11-06T21:59:47.825 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T21:59:47.826 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T21:59:47.826 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T21:59:47.826 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T21:59:47.827 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T21:59:47.827 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T21:59:47.827 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T21:59:47.828 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T21:59:47.828 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T21:59:47.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:59:47.829 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T21:59:47.878 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T21:59:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T21:59:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T21:59:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:59:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T21:59:47.879 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:59:47.879 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:59:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:59:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T21:59:47.880 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:59:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:59:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T21:59:47.881 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T21:59:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:59:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T21:59:47.881 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T21:59:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 37756sec preferred_lft 37756sec 2023-11-06T21:59:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T21:59:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:59:47.882 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T21:59:47.882 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T21:59:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T21:59:47.883 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T21:59:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T21:59:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T21:59:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:59:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T21:59:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:59:47.884 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T21:59:47.884 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T21:59:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T21:59:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:59:47.885 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T21:59:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T21:59:47.885 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T21:59:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T21:59:47.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:59:47.886 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T21:59:47.962 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T21:59:47.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:59:47.966 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T21:59:48.017 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T21:59:48.017 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T21:59:48.017 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T21:59:48.017 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T21:59:48.018 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T21:59:48.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:59:48.019 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T21:59:48.020 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T21:59:48.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T21:59:48.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T21:59:48.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T21:59:48.022 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T21:59:48.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:59:48.198 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T21:59:48.247 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T21:59:48.248 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T21:59:48.248 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T21:59:48.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:59:48.250 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T21:59:48.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:59:48.324 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T21:59:48.372 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T21:59:48.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T21:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T21:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T21:59:48.509 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T21:59:48.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:59:48.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T21:59:48.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T21:59:48.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.779+0000 7f8f845ff700 1 Processor -- start 2023-11-06T21:59:48.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.779+0000 7f8f845ff700 1 -- start start 2023-11-06T21:59:48.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.781+0000 7f8f845ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8f7c062e20 0x7f8f7c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:48.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.781+0000 7f8f845ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7c069c80 con 0x7f8f7c0c8500 2023-11-06T21:59:48.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.781+0000 7f8f845ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7c069f60 con 0x7f8f7c1542d0 2023-11-06T21:59:48.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.781+0000 7f8f845ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7c0bcb10 con 0x7f8f7c062e20 2023-11-06T21:59:48.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.781+0000 7f8f81b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8f7c0c8500 0x7f8f7c062710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58566/0 (socket says 172.21.15.33:58566) 2023-11-06T21:59:48.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.781+0000 7f8f81b9a700 1 -- 172.21.15.33:0/956015557 learned_addr learned my addr 172.21.15.33:0/956015557 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:48.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.781+0000 7f8f82b9c700 1 --2- 172.21.15.33:0/956015557 >> v2:172.21.15.69:3300/0 conn(0x7f8f7c062e20 0x7f8f7c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:48.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.781+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977873095 0 0) 0x7f8f7c069c80 con 0x7f8f7c0c8500 2023-11-06T21:59:48.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.781+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f680033f0 con 0x7f8f7c0c8500 2023-11-06T21:59:48.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289155642 0 0) 0x7f8f7c069f60 con 0x7f8f7c1542d0 2023-11-06T21:59:48.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f7c069c80 con 0x7f8f7c1542d0 2023-11-06T21:59:48.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3820112439 0 0) 0x7f8f680033f0 con 0x7f8f7c0c8500 2023-11-06T21:59:48.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 >> v2:172.21.15.69:3300/0 conn(0x7f8f7c062e20 msgr2=0x7f8f7c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:48.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 --2- 172.21.15.33:0/956015557 >> v2:172.21.15.69:3300/0 conn(0x7f8f7c062e20 0x7f8f7c15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:48.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 >> v1:172.21.15.69:6789/0 conn(0x7f8f7c1542d0 legacy=0x7f8f7c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:48.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c0bcdf0 con 0x7f8f7c0c8500 2023-11-06T21:59:48.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f8f6c001470 con 0x7f8f7c0c8500 2023-11-06T21:59:48.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f82b9c700 1 --2- 172.21.15.33:0/956015557 >> v2:172.21.15.69:3300/0 conn(0x7f8f7c062e20 0x7f8f7c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:48.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 >> v1:172.21.15.33:6789/0 conn(0x7f8f7c0c8500 legacy=0x7f8f7c062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:48.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8f68003a70 0x7f8f68003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:48.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f68004690 0x7f8f68004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:48.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f6800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:48.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c0bcb10 con 0x7f8f68005000 2023-11-06T21:59:48.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6800be80 con 0x7f8f68004690 2023-11-06T21:59:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6800c040 con 0x7f8f68003a70 2023-11-06T21:59:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f82b9c700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f6800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f81b9a700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f68004690 0x7f8f68004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.782+0000 7f8f8239b700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8f68003a70 0x7f8f68003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.783+0000 7f8f82b9c700 1 -- 172.21.15.33:0/956015557 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8f68003a70 msgr2=0x7f8f68003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.783+0000 7f8f82b9c700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8f68003a70 0x7f8f68003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.783+0000 7f8f82b9c700 1 -- 172.21.15.33:0/956015557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f68004690 msgr2=0x7f8f68004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.783+0000 7f8f82b9c700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f68004690 0x7f8f68004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.783+0000 7f8f82b9c700 1 -- 172.21.15.33:0/956015557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f680033f0 con 0x7f8f68005000 2023-11-06T21:59:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.783+0000 7f8f82b9c700 1 -- 172.21.15.33:0/956015557 reap_dead start 2023-11-06T21:59:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.783+0000 7f8f82b9c700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f6800b8e0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f8f740094d0 tx=0x7f8f7400b040).ready entity=mon.0 client_cookie=fb2fd9f0985f2675 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.783+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74017070 con 0x7f8f68005000 2023-11-06T21:59:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.783+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7400f040 con 0x7f8f68005000 2023-11-06T21:59:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.783+0000 7f8f81399700 1 -- 172.21.15.33:0/956015557 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74009cf0 con 0x7f8f68005000 2023-11-06T21:59:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.784+0000 7f8f845ff700 1 -- 172.21.15.33:0/956015557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 msgr2=0x7f8f6800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.784+0000 7f8f845ff700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f6800b8e0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f8f740094d0 tx=0x7f8f7400b040).stop 2023-11-06T21:59:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.784+0000 7f8f845ff700 1 -- 172.21.15.33:0/956015557 shutdown_connections 2023-11-06T21:59:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.784+0000 7f8f845ff700 1 --2- 172.21.15.33:0/956015557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f6800b8e0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.784+0000 7f8f845ff700 1 -- 172.21.15.33:0/956015557 >> 172.21.15.33:0/956015557 conn(0x7f8f7c0642a0 msgr2=0x7f8f7c0c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.784+0000 7f8f845ff700 1 -- 172.21.15.33:0/956015557 shutdown_connections 2023-11-06T21:59:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.784+0000 7f8f845ff700 1 -- 172.21.15.33:0/956015557 wait complete. 2023-11-06T21:59:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.786+0000 7f8f845ff700 1 Processor -- start 2023-11-06T21:59:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.786+0000 7f8f845ff700 1 -- start start 2023-11-06T21:59:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.787+0000 7f8f845ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f7c0a9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.787+0000 7f8f845ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8f7c1642b0 0x7f8f7c0a80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.787+0000 7f8f845ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f7c166fb0 0x7f8f7c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.787+0000 7f8f845ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c164730 con 0x7f8f68005000 2023-11-06T21:59:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.787+0000 7f8f845ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c1546b0 con 0x7f8f7c166fb0 2023-11-06T21:59:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.787+0000 7f8f845ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c155230 con 0x7f8f7c1642b0 2023-11-06T21:59:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f8239b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f7c0a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f8239b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f7c0a9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59230/0 (socket says 172.21.15.33:59230) 2023-11-06T21:59:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f8239b700 1 -- 172.21.15.33:0/986207203 learned_addr learned my addr 172.21.15.33:0/986207203 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f81b9a700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8f7c1642b0 0x7f8f7c0a80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f82b9c700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f7c166fb0 0x7f8f7c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f8239b700 1 -- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8f7c1642b0 msgr2=0x7f8f7c0a80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:48.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f8239b700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8f7c1642b0 0x7f8f7c0a80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:48.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f8239b700 1 -- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f7c166fb0 msgr2=0x7f8f7c0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:48.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f8239b700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f7c166fb0 0x7f8f7c0acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:48.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f8239b700 1 -- 172.21.15.33:0/986207203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c0a8590 con 0x7f8f68005000 2023-11-06T21:59:48.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f82b9c700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f7c166fb0 0x7f8f7c0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:48.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f81b9a700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8f7c1642b0 0x7f8f7c0a80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T21:59:48.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f8239b700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f7c0a9930 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f8f78002a80 tx=0x7f8f78007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:48.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.788+0000 7f8f727fc700 1 -- 172.21.15.33:0/986207203 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800f940 con 0x7f8f68005000 2023-11-06T21:59:48.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.789+0000 7f8f727fc700 1 -- 172.21.15.33:0/986207203 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7800faa0 con 0x7f8f68005000 2023-11-06T21:59:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.789+0000 7f8f727fc700 1 -- 172.21.15.33:0/986207203 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800fe40 con 0x7f8f68005000 2023-11-06T21:59:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.789+0000 7f8f845ff700 1 -- 172.21.15.33:0/986207203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c154900 con 0x7f8f68005000 2023-11-06T21:59:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.789+0000 7f8f845ff700 1 -- 172.21.15.33:0/986207203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7c154ec0 con 0x7f8f68005000 2023-11-06T21:59:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.790+0000 7f8f845ff700 1 -- 172.21.15.33:0/986207203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7c06a240 con 0x7f8f68005000 2023-11-06T21:59:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.790+0000 7f8f727fc700 1 -- 172.21.15.33:0/986207203 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8f7801d3b0 con 0x7f8f68005000 2023-11-06T21:59:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.791+0000 7f8f727fc700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8f6005f240 0x7f8f60061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.791+0000 7f8f727fc700 1 -- 172.21.15.33:0/986207203 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f8f7807a4d0 con 0x7f8f68005000 2023-11-06T21:59:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.792+0000 7f8f81b9a700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8f6005f240 0x7f8f60061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.793+0000 7f8f81b9a700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8f6005f240 0x7f8f60061700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f8f6c001200 tx=0x7f8f6c009040).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:48.796+0000 7f8f727fc700 1 -- 172.21.15.33:0/986207203 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8f7804b590 con 0x7f8f68005000 2023-11-06T21:59:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.008+0000 7f8f845ff700 1 -- 172.21.15.33:0/986207203 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f7c06a510 con 0x7f8f6005f240 2023-11-06T21:59:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.010+0000 7f8f727fc700 1 -- 172.21.15.33:0/986207203 <== mgr.7025 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8f7c06a510 con 0x7f8f6005f240 2023-11-06T21:59:49.011 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T21:59:49.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T21:59:49.012 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T21:59:49.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T21:59:49.012 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T21:59:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.012+0000 7f8f845ff700 1 -- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8f6005f240 msgr2=0x7f8f60061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.012+0000 7f8f845ff700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8f6005f240 0x7f8f60061700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f8f6c001200 tx=0x7f8f6c009040).stop 2023-11-06T21:59:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.012+0000 7f8f845ff700 1 -- 172.21.15.33:0/986207203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 msgr2=0x7f8f7c0a9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.012+0000 7f8f845ff700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f7c0a9930 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f8f78002a80 tx=0x7f8f78007bb0).stop 2023-11-06T21:59:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.012+0000 7f8f845ff700 1 -- 172.21.15.33:0/986207203 shutdown_connections 2023-11-06T21:59:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.012+0000 7f8f845ff700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f8f6005f240 0x7f8f60061700 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.012+0000 7f8f845ff700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f7c166fb0 0x7f8f7c0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.012+0000 7f8f845ff700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f68005000 0x7f8f7c0a9930 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.012+0000 7f8f845ff700 1 --2- 172.21.15.33:0/986207203 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8f7c1642b0 0x7f8f7c0a80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.012+0000 7f8f845ff700 1 -- 172.21.15.33:0/986207203 >> 172.21.15.33:0/986207203 conn(0x7f8f7c0642a0 msgr2=0x7f8f7c0c65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.013+0000 7f8f845ff700 1 -- 172.21.15.33:0/986207203 shutdown_connections 2023-11-06T21:59:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.013+0000 7f8f845ff700 1 -- 172.21.15.33:0/986207203 wait complete. 2023-11-06T21:59:49.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T21:59:49.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.230+0000 7f1f60dff700 1 Processor -- start 2023-11-06T21:59:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.230+0000 7f1f60dff700 1 -- start start 2023-11-06T21:59:49.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.232+0000 7f1f60dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 0x7f1f5c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:49.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.232+0000 7f1f60dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1f5c0aaf50 0x7f1f5c063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:49.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.232+0000 7f1f60dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c158c00 con 0x7f1f5c0aa4f0 2023-11-06T21:59:49.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.232+0000 7f1f60dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c18f0 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.232+0000 7f1f60dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c6d80 con 0x7f1f5c157700 2023-11-06T21:59:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.232+0000 7f1f5a59c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 0x7f1f5c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.232+0000 7f1f5a59c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 0x7f1f5c0c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:41280/0 (socket says 172.21.15.33:41280) 2023-11-06T21:59:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.232+0000 7f1f5a59c700 1 -- 172.21.15.33:0/1799320263 learned_addr learned my addr 172.21.15.33:0/1799320263 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.232+0000 7f1f5ad9d700 1 --1- 172.21.15.33:0/1799320263 >> v1:172.21.15.69:6789/0 conn(0x7f1f5c0aa4f0 0x7f1f5c0646d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:48686/0 (socket says 172.21.15.33:48686) 2023-11-06T21:59:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.233+0000 7f1f59d9b700 1 --2- 172.21.15.33:0/1799320263 >> v2:172.21.15.69:3300/0 conn(0x7f1f5c0aaf50 0x7f1f5c063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.233+0000 7f1f5a59c700 1 -- 172.21.15.33:0/1799320263 >> v2:172.21.15.69:3300/0 conn(0x7f1f5c0aaf50 msgr2=0x7f1f5c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.233+0000 7f1f5a59c700 1 --2- 172.21.15.33:0/1799320263 >> v2:172.21.15.69:3300/0 conn(0x7f1f5c0aaf50 0x7f1f5c063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.233+0000 7f1f5a59c700 1 -- 172.21.15.33:0/1799320263 >> v1:172.21.15.69:6789/0 conn(0x7f1f5c0aa4f0 legacy=0x7f1f5c0646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.233+0000 7f1f5a59c700 1 -- 172.21.15.33:0/1799320263 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c157ff0 con 0x7f1f5c157700 2023-11-06T21:59:49.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.233+0000 7f1f5a59c700 1 --2- 172.21.15.33:0/1799320263 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 0x7f1f5c0c6f00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f1f4c006fd0 tx=0x7f1f4c0090f0).ready entity=mon.2 client_cookie=6c9e556e6b7df51a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:49.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.233+0000 7f1f5959a700 1 -- 172.21.15.33:0/1799320263 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00b580 con 0x7f1f5c157700 2023-11-06T21:59:49.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.234+0000 7f1f5959a700 1 -- 172.21.15.33:0/1799320263 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00b6e0 con 0x7f1f5c157700 2023-11-06T21:59:49.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.234+0000 7f1f5959a700 1 -- 172.21.15.33:0/1799320263 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00ba60 con 0x7f1f5c157700 2023-11-06T21:59:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.234+0000 7f1f60dff700 1 -- 172.21.15.33:0/1799320263 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 msgr2=0x7f1f5c0c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.234+0000 7f1f60dff700 1 --2- 172.21.15.33:0/1799320263 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 0x7f1f5c0c6f00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f1f4c006fd0 tx=0x7f1f4c0090f0).stop 2023-11-06T21:59:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.234+0000 7f1f60dff700 1 -- 172.21.15.33:0/1799320263 shutdown_connections 2023-11-06T21:59:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.234+0000 7f1f60dff700 1 --2- 172.21.15.33:0/1799320263 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 0x7f1f5c0c6f00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.234+0000 7f1f60dff700 1 --2- 172.21.15.33:0/1799320263 >> v2:172.21.15.69:3300/0 conn(0x7f1f5c0aaf50 0x7f1f5c063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.234+0000 7f1f60dff700 1 -- 172.21.15.33:0/1799320263 >> 172.21.15.33:0/1799320263 conn(0x7f1f5c05ec80 msgr2=0x7f1f5c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.234+0000 7f1f60dff700 1 -- 172.21.15.33:0/1799320263 shutdown_connections 2023-11-06T21:59:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.234+0000 7f1f60dff700 1 -- 172.21.15.33:0/1799320263 wait complete. 2023-11-06T21:59:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.236+0000 7f1f60dff700 1 Processor -- start 2023-11-06T21:59:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.236+0000 7f1f60dff700 1 -- start start 2023-11-06T21:59:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f60dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c0aa4f0 0x7f1f5c143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f60dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0aaf50 0x7f1f5c0c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f60dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 0x7f1f5c151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f60dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c3690 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f60dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c27f0 con 0x7f1f5c0aa4f0 2023-11-06T21:59:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f60dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c2ad0 con 0x7f1f5c157700 2023-11-06T21:59:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f59d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0aaf50 0x7f1f5c0c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f59d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0aaf50 0x7f1f5c0c1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59232/0 (socket says 172.21.15.33:59232) 2023-11-06T21:59:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f59d9b700 1 -- 172.21.15.33:0/597301774 learned_addr learned my addr 172.21.15.33:0/597301774 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f5ad9d700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 0x7f1f5c151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f5a59c700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c0aa4f0 0x7f1f5c143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f59d9b700 1 -- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 msgr2=0x7f1f5c151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f59d9b700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 0x7f1f5c151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f59d9b700 1 -- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c0aa4f0 msgr2=0x7f1f5c143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.237+0000 7f1f59d9b700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c0aa4f0 0x7f1f5c143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.238+0000 7f1f59d9b700 1 -- 172.21.15.33:0/597301774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c009370 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.238+0000 7f1f5a59c700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c0aa4f0 0x7f1f5c143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.238+0000 7f1f59d9b700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0aaf50 0x7f1f5c0c1ef0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f1f5000e900 tx=0x7f1f50009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.238+0000 7f1f4a7fc700 1 -- 172.21.15.33:0/597301774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000ee40 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.238+0000 7f1f4a7fc700 1 -- 172.21.15.33:0/597301774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5000c850 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.238+0000 7f1f4a7fc700 1 -- 172.21.15.33:0/597301774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000cbd0 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.238+0000 7f1f60dff700 1 -- 172.21.15.33:0/597301774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c0c2db0 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.238+0000 7f1f60dff700 1 -- 172.21.15.33:0/597301774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c143f70 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.239+0000 7f1f60dff700 1 -- 172.21.15.33:0/597301774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c0c3800 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.240+0000 7f1f4a7fc700 1 -- 172.21.15.33:0/597301774 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 65) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1f5001b400 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.241+0000 7f1f4a7fc700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f3c05f270 0x7f1f3c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:49.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.241+0000 7f1f4a7fc700 1 -- 172.21.15.33:0/597301774 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f1f5007a9c0 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.241+0000 7f1f5a59c700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f3c05f270 0x7f1f3c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:49.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.242+0000 7f1f5a59c700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f3c05f270 0x7f1f3c061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1f4c009fd0 tx=0x7f1f4c017000).ready entity=mgr.7025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:49.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.245+0000 7f1f4a7fc700 1 -- 172.21.15.33:0/597301774 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1f5004ba70 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.470+0000 7f1f60dff700 1 -- 172.21.15.33:0/597301774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f1f5c1585b0 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.977+0000 7f1f4a7fc700 1 -- 172.21.15.33:0/597301774 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 66) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f1f50078520 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.978+0000 7f1f4a7fc700 1 -- 172.21.15.33:0/597301774 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v66) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1f50072530 con 0x7f1f5c0aaf50 2023-11-06T21:59:49.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 -- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f3c05f270 msgr2=0x7f1f3c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:49.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f3c05f270 0x7f1f3c061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1f4c009fd0 tx=0x7f1f4c017000).stop 2023-11-06T21:59:49.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 -- 172.21.15.33:0/597301774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0aaf50 msgr2=0x7f1f5c0c1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:49.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0aaf50 0x7f1f5c0c1ef0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f1f5000e900 tx=0x7f1f50009060).stop 2023-11-06T21:59:49.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 -- 172.21.15.33:0/597301774 shutdown_connections 2023-11-06T21:59:49.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f1f3c05f270 0x7f1f3c061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1f5c157700 0x7f1f5c151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f5c0aa4f0 0x7f1f5c143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 --2- 172.21.15.33:0/597301774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0aaf50 0x7f1f5c0c1ef0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:49.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 -- 172.21.15.33:0/597301774 >> 172.21.15.33:0/597301774 conn(0x7f1f5c05ec80 msgr2=0x7f1f5c156960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:49.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 -- 172.21.15.33:0/597301774 shutdown_connections 2023-11-06T21:59:49.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:49.981+0000 7f1f60dff700 1 -- 172.21.15.33:0/597301774 wait complete. 2023-11-06T21:59:50.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T21:59:50.196 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:50.194+0000 7fa6804d7680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:59:50.200 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:50.199+0000 7fcbfc384680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:59:50.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.209+0000 7f009c5ff700 1 Processor -- start 2023-11-06T21:59:50.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.209+0000 7f009c5ff700 1 -- start start 2023-11-06T21:59:50.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.210+0000 7f009c5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0094154710 0x7f00940c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.210+0000 7f009c5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f00940c7ff0 0x7f00940646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.210+0000 7f009c5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00940c6b60 con 0x7f0094154330 2023-11-06T21:59:50.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.210+0000 7f009c5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00940c7700 con 0x7f0094154710 2023-11-06T21:59:50.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.210+0000 7f009c5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00940c5cd0 con 0x7f00940c7ff0 2023-11-06T21:59:50.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.211+0000 7f0099b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0094154710 0x7f00940c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:50.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.211+0000 7f0099b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0094154710 0x7f00940c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59252/0 (socket says 172.21.15.33:59252) 2023-11-06T21:59:50.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.211+0000 7f0099b9a700 1 -- 172.21.15.33:0/2726810684 learned_addr learned my addr 172.21.15.33:0/2726810684 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:50.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.211+0000 7f009ab9c700 1 --2- 172.21.15.33:0/2726810684 >> v2:172.21.15.69:3300/0 conn(0x7f00940c7ff0 0x7f00940646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:50.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.211+0000 7f0099b9a700 1 -- 172.21.15.33:0/2726810684 >> v2:172.21.15.69:3300/0 conn(0x7f00940c7ff0 msgr2=0x7f00940646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:50.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.211+0000 7f0099b9a700 1 --2- 172.21.15.33:0/2726810684 >> v2:172.21.15.69:3300/0 conn(0x7f00940c7ff0 0x7f00940646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.211+0000 7f0099b9a700 1 -- 172.21.15.33:0/2726810684 >> v1:172.21.15.69:6789/0 conn(0x7f0094154330 legacy=0x7f00940c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:50.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.211+0000 7f0099b9a700 1 -- 172.21.15.33:0/2726810684 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00940c5f50 con 0x7f0094154710 2023-11-06T21:59:50.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.211+0000 7f0099b9a700 1 --2- 172.21.15.33:0/2726810684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0094154710 0x7f00940c57e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f00940c1e60 tx=0x7f00840090f0).ready entity=mon.0 client_cookie=38f8cef749d537c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:50.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.211+0000 7f0099399700 1 -- 172.21.15.33:0/2726810684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008400b520 con 0x7f0094154710 2023-11-06T21:59:50.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.212+0000 7f0099399700 1 -- 172.21.15.33:0/2726810684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008400b680 con 0x7f0094154710 2023-11-06T21:59:50.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.212+0000 7f0099399700 1 -- 172.21.15.33:0/2726810684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008400ba00 con 0x7f0094154710 2023-11-06T21:59:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.212+0000 7f009c5ff700 1 -- 172.21.15.33:0/2726810684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0094154710 msgr2=0x7f00940c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.212+0000 7f009c5ff700 1 --2- 172.21.15.33:0/2726810684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0094154710 0x7f00940c57e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f00940c1e60 tx=0x7f00840090f0).stop 2023-11-06T21:59:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.212+0000 7f009c5ff700 1 -- 172.21.15.33:0/2726810684 shutdown_connections 2023-11-06T21:59:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.212+0000 7f009c5ff700 1 --2- 172.21.15.33:0/2726810684 >> v2:172.21.15.69:3300/0 conn(0x7f00940c7ff0 0x7f00940646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.212+0000 7f009c5ff700 1 --2- 172.21.15.33:0/2726810684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0094154710 0x7f00940c57e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.212+0000 7f009c5ff700 1 -- 172.21.15.33:0/2726810684 >> 172.21.15.33:0/2726810684 conn(0x7f00940cb910 msgr2=0x7f00940c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.212+0000 7f009c5ff700 1 -- 172.21.15.33:0/2726810684 shutdown_connections 2023-11-06T21:59:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.212+0000 7f009c5ff700 1 -- 172.21.15.33:0/2726810684 wait complete. 2023-11-06T21:59:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.214+0000 7f009c5ff700 1 Processor -- start 2023-11-06T21:59:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.214+0000 7f009c5ff700 1 -- start start 2023-11-06T21:59:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f009c5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00940c7ff0 0x7f0094157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f009c5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0094154330 0x7f0094147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f009c5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0094154710 0x7f00940af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f009c5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0094155570 con 0x7f0094154710 2023-11-06T21:59:50.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f009c5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00941565c0 con 0x7f00940c7ff0 2023-11-06T21:59:50.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f009c5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0094155800 con 0x7f0094154330 2023-11-06T21:59:50.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f0099b9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0094154330 0x7f0094147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:50.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f009a39b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00940c7ff0 0x7f0094157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:50.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f009ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0094154710 0x7f00940af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f0099b9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0094154330 0x7f0094147cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:41302/0 (socket says 172.21.15.33:41302) 2023-11-06T21:59:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.215+0000 7f0099b9a700 1 -- 172.21.15.33:0/2013197831 learned_addr learned my addr 172.21.15.33:0/2013197831 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f0099b9a700 1 -- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00940c7ff0 msgr2=0x7f0094157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f0099b9a700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00940c7ff0 0x7f0094157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f0099b9a700 1 -- 172.21.15.33:0/2013197831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0094154710 msgr2=0x7f00940af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:50.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f0099b9a700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0094154710 0x7f00940af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f0099b9a700 1 -- 172.21.15.33:0/2013197831 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0084009370 con 0x7f0094154330 2023-11-06T21:59:50.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f009a39b700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00940c7ff0 0x7f0094157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:50.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f0099b9a700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0094154330 0x7f0094147cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f0084011000 tx=0x7f0084017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:50.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f008a7fc700 1 -- 172.21.15.33:0/2013197831 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084000bc0 con 0x7f0094154330 2023-11-06T21:59:50.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f008a7fc700 1 -- 172.21.15.33:0/2013197831 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008400b7f0 con 0x7f0094154330 2023-11-06T21:59:50.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f009c5ff700 1 -- 172.21.15.33:0/2013197831 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0094155ae0 con 0x7f0094154330 2023-11-06T21:59:50.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.216+0000 7f008a7fc700 1 -- 172.21.15.33:0/2013197831 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084000d20 con 0x7f0094154330 2023-11-06T21:59:50.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.217+0000 7f009c5ff700 1 -- 172.21.15.33:0/2013197831 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00941560a0 con 0x7f0094154330 2023-11-06T21:59:50.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.218+0000 7f009c5ff700 1 -- 172.21.15.33:0/2013197831 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0094156290 con 0x7f0094154330 2023-11-06T21:59:50.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.218+0000 7f008a7fc700 1 -- 172.21.15.33:0/2013197831 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 66) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f008401c8c0 con 0x7f0094154330 2023-11-06T21:59:50.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.219+0000 7f008a7fc700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f007805f2c0 0x7f0078061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.219+0000 7f008a7fc700 1 -- 172.21.15.33:0/2013197831 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f008407bf60 con 0x7f0094154330 2023-11-06T21:59:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.219+0000 7f009a39b700 1 -- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f007805f2c0 msgr2=0x7f0078061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.220+0000 7f009a39b700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f007805f2c0 0x7f0078061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.224+0000 7f008a7fc700 1 -- 172.21.15.33:0/2013197831 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f008404cfc0 con 0x7f0094154330 2023-11-06T21:59:50.277 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:50.275+0000 7fa6804d7680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:59:50.285 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:50.284+0000 7fcbfc384680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:59:50.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.409+0000 7f009c5ff700 1 -- 172.21.15.33:0/2013197831 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f00940652d0 con 0x7f0094154330 2023-11-06T21:59:50.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.415+0000 7f008a7fc700 1 -- 172.21.15.33:0/2013197831 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f00840505e0 con 0x7f0094154330 2023-11-06T21:59:50.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.417+0000 7f009c5ff700 1 -- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f007805f2c0 msgr2=0x7f0078061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:50.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.417+0000 7f009c5ff700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f007805f2c0 0x7f0078061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:50.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.417+0000 7f009c5ff700 1 -- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0094154330 msgr2=0x7f0094147cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:50.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.417+0000 7f009c5ff700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0094154330 0x7f0094147cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f0084011000 tx=0x7f0084017000).stop 2023-11-06T21:59:50.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.418+0000 7f009c5ff700 1 -- 172.21.15.33:0/2013197831 shutdown_connections 2023-11-06T21:59:50.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.418+0000 7f009c5ff700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f007805f2c0 0x7f0078061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:50.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.418+0000 7f009c5ff700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0094154710 0x7f00940af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.418+0000 7f009c5ff700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00940c7ff0 0x7f0094157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.418+0000 7f009c5ff700 1 --2- 172.21.15.33:0/2013197831 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0094154330 0x7f0094147cb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.418+0000 7f009c5ff700 1 -- 172.21.15.33:0/2013197831 >> 172.21.15.33:0/2013197831 conn(0x7f00940cb910 msgr2=0x7f00940646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:50.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.418+0000 7f009c5ff700 1 -- 172.21.15.33:0/2013197831 shutdown_connections 2023-11-06T21:59:50.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.418+0000 7f009c5ff700 1 -- 172.21.15.33:0/2013197831 wait complete. 2023-11-06T21:59:50.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:50.438 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-06T21:59:50.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1h 2023-11-06T21:59:50.605 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:50.603+0000 7fa6804d7680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:59:50.648 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:50.647+0000 7fcbfc384680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:59:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.705+0000 7f83571ff700 1 Processor -- start 2023-11-06T21:59:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.705+0000 7f83571ff700 1 -- start start 2023-11-06T21:59:50.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.706+0000 7f83571ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f835013a960 0x7f8350155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.706+0000 7f83571ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8350066ba0 0x7f83500c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.706+0000 7f83571ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8350063950 con 0x7f83500c9340 2023-11-06T21:59:50.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.706+0000 7f83571ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8350064cc0 con 0x7f8350066ba0 2023-11-06T21:59:50.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.706+0000 7f83571ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8350155930 con 0x7f835013a960 2023-11-06T21:59:50.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f835579c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8350066ba0 0x7f83500c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:50.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f835579c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8350066ba0 0x7f83500c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59264/0 (socket says 172.21.15.33:59264) 2023-11-06T21:59:50.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f835579c700 1 -- 172.21.15.33:0/857507535 learned_addr learned my addr 172.21.15.33:0/857507535 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:50.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f834ffff700 1 --1- 172.21.15.33:0/857507535 >> v1:172.21.15.33:6789/0 conn(0x7f83500c9340 0x7f8350066490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58594/0 (socket says 172.21.15.33:58594) 2023-11-06T21:59:50.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f8354f9b700 1 --2- 172.21.15.33:0/857507535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f835013a960 0x7f8350155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:50.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f835579c700 1 -- 172.21.15.33:0/857507535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f835013a960 msgr2=0x7f8350155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:50.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f835579c700 1 --2- 172.21.15.33:0/857507535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f835013a960 0x7f8350155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f835579c700 1 -- 172.21.15.33:0/857507535 >> v1:172.21.15.33:6789/0 conn(0x7f83500c9340 legacy=0x7f8350066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:50.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f834f7fe700 1 -- 172.21.15.33:0/857507535 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486872978 0 0) 0x7f8350063950 con 0x7f83500c9340 2023-11-06T21:59:50.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f835579c700 1 -- 172.21.15.33:0/857507535 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8350063c30 con 0x7f8350066ba0 2023-11-06T21:59:50.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.707+0000 7f835579c700 1 --2- 172.21.15.33:0/857507535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8350066ba0 0x7f83500c37b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7f8348009f20 tx=0x7f8348009840).ready entity=mon.0 client_cookie=1f28c5965ee5829e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:50.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.708+0000 7f834f7fe700 1 -- 172.21.15.33:0/857507535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800e070 con 0x7f8350066ba0 2023-11-06T21:59:50.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.708+0000 7f834f7fe700 1 -- 172.21.15.33:0/857507535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834800b9d0 con 0x7f8350066ba0 2023-11-06T21:59:50.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.708+0000 7f834f7fe700 1 -- 172.21.15.33:0/857507535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800bd50 con 0x7f8350066ba0 2023-11-06T21:59:50.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.708+0000 7f83571ff700 1 -- 172.21.15.33:0/857507535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8350066ba0 msgr2=0x7f83500c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:50.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.708+0000 7f83571ff700 1 --2- 172.21.15.33:0/857507535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8350066ba0 0x7f83500c37b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7f8348009f20 tx=0x7f8348009840).stop 2023-11-06T21:59:50.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.708+0000 7f83571ff700 1 -- 172.21.15.33:0/857507535 shutdown_connections 2023-11-06T21:59:50.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.708+0000 7f83571ff700 1 --2- 172.21.15.33:0/857507535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8350066ba0 0x7f83500c37b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.708+0000 7f83571ff700 1 --2- 172.21.15.33:0/857507535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f835013a960 0x7f8350155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.708+0000 7f83571ff700 1 -- 172.21.15.33:0/857507535 >> 172.21.15.33:0/857507535 conn(0x7f835005ec70 msgr2=0x7f83500c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:50.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.709+0000 7f83571ff700 1 -- 172.21.15.33:0/857507535 shutdown_connections 2023-11-06T21:59:50.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.709+0000 7f83571ff700 1 -- 172.21.15.33:0/857507535 wait complete. 2023-11-06T21:59:50.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.710+0000 7f83571ff700 1 Processor -- start 2023-11-06T21:59:50.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.710+0000 7f83571ff700 1 -- start start 2023-11-06T21:59:50.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.711+0000 7f83571ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350066ba0 0x7f835014bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.711+0000 7f83571ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83500c9340 0x7f83500a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.711+0000 7f83571ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f835013a960 0x7f83500add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.711+0000 7f83571ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835014c0d0 con 0x7f835013a960 2023-11-06T21:59:50.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.711+0000 7f83571ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83500a7060 con 0x7f8350066ba0 2023-11-06T21:59:50.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.711+0000 7f83571ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83500a7340 con 0x7f83500c9340 2023-11-06T21:59:50.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.711+0000 7f835579c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f835013a960 0x7f83500add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:50.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f834ffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83500c9340 0x7f83500a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:50.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f8354f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350066ba0 0x7f835014bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:50.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f835579c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f835013a960 0x7f83500add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59272/0 (socket says 172.21.15.33:59272) 2023-11-06T21:59:50.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f835579c700 1 -- 172.21.15.33:0/1101097352 learned_addr learned my addr 172.21.15.33:0/1101097352 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:50.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f835579c700 1 -- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83500c9340 msgr2=0x7f83500a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:50.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f835579c700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83500c9340 0x7f83500a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f835579c700 1 -- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350066ba0 msgr2=0x7f835014bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:50.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f835579c700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350066ba0 0x7f835014bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:50.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f835579c700 1 -- 172.21.15.33:0/1101097352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8348009ae0 con 0x7f835013a960 2023-11-06T21:59:50.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f8354f9b700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350066ba0 0x7f835014bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:50.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.712+0000 7f835579c700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f835013a960 0x7f83500add20 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f835005f0e0 tx=0x7f8348009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:50.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.713+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800b640 con 0x7f835013a960 2023-11-06T21:59:50.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.713+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83480092e0 con 0x7f835013a960 2023-11-06T21:59:50.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.713+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834801e6a0 con 0x7f835013a960 2023-11-06T21:59:50.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.713+0000 7f83571ff700 1 -- 172.21.15.33:0/1101097352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83500a7620 con 0x7f835013a960 2023-11-06T21:59:50.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.713+0000 7f83571ff700 1 -- 172.21.15.33:0/1101097352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83500a7be0 con 0x7f835013a960 2023-11-06T21:59:50.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.714+0000 7f83571ff700 1 -- 172.21.15.33:0/1101097352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83500a7e00 con 0x7f835013a960 2023-11-06T21:59:50.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.715+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 66) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f8348003040 con 0x7f835013a960 2023-11-06T21:59:50.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.716+0000 7f834cff9700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:50.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.716+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f834807b960 con 0x7f835013a960 2023-11-06T21:59:50.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.716+0000 7f8354f9b700 1 -- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 msgr2=0x7f8334061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:50.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.716+0000 7f8354f9b700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:50.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.724+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f834804ca80 con 0x7f835013a960 2023-11-06T21:59:50.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.917+0000 7f8354f9b700 1 -- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 msgr2=0x7f8334061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:50.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.917+0000 7f8354f9b700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:59:50.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:50.931+0000 7f83571ff700 1 -- 172.21.15.33:0/1101097352 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f8350065f30 con 0x7f833405f2c0 2023-11-06T21:59:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:51.318+0000 7f8354f9b700 1 -- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 msgr2=0x7f8334061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:51.318+0000 7f8354f9b700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T21:59:51.353 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:51.351+0000 7fa6804d7680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:59:51.433 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:51.432+0000 7fcbfc384680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:59:51.438 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:51.436+0000 7fa6804d7680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:59:51.520 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:51.519+0000 7fcbfc384680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:59:51.669 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:51.667+0000 7fa6804d7680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:59:51.757 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:51.756+0000 7fcbfc384680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T21:59:52.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:52.119+0000 7f8354f9b700 1 -- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 msgr2=0x7f8334061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:52.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:52.119+0000 7f8354f9b700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T21:59:52.183 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:52.181+0000 7fa6804d7680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:59:52.287 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:52.286+0000 7fcbfc384680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T21:59:52.308 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:52.306+0000 7fa6804d7680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:59:52.419 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:52.418+0000 7fcbfc384680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:59:52.493 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:52.492+0000 7fa6804d7680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:59:52.588 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:52.586+0000 7fa6804d7680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:59:52.609 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:52.608+0000 7fcbfc384680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T21:59:52.674 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:52.672+0000 7fa6804d7680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:59:52.707 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:52.705+0000 7fcbfc384680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T21:59:52.794 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:52.793+0000 7fcbfc384680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T21:59:52.984 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:52.983+0000 7fa6804d7680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:59:53.119 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:53.118+0000 7fcbfc384680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T21:59:53.209 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:53.207+0000 7fa6804d7680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:59:53.348 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:53.347+0000 7fcbfc384680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T21:59:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:53.721+0000 7f8354f9b700 1 -- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 msgr2=0x7f8334061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:53.721+0000 7f8354f9b700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T21:59:54.012 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:54.010+0000 7fa6804d7680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:59:54.101 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:54.099+0000 7fa6804d7680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:59:54.184 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:54.183+0000 7fcbfc384680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T21:59:54.196 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:54.194+0000 7fa6804d7680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:59:54.275 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:54.274+0000 7fcbfc384680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T21:59:54.373 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:54.372+0000 7fcbfc384680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T21:59:54.382 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:54.380+0000 7fa6804d7680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:59:54.472 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:54.469+0000 7fa6804d7680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:59:54.565 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:54.564+0000 7fcbfc384680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T21:59:54.613 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:54.611+0000 7fa6804d7680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:59:54.658 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:54.657+0000 7fcbfc384680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T21:59:54.744 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:54.742+0000 7fa6804d7680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:59:54.806 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:54.804+0000 7fcbfc384680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T21:59:54.942 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:54.941+0000 7fcbfc384680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T21:59:55.094 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.092+0000 7fa6804d7680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:59:55.183 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.181+0000 7fa6804d7680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:59:55.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:55.190+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 67) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f8348079580 con 0x7f835013a960 2023-11-06T21:59:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:55.190+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 msgr2=0x7f8334061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:55.190+0000 7f834cff9700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:55.274 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.272+0000 7fa5f57c6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:55.274 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.272+0000 7fa5f57c6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:55.275 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.272+0000 7fa5f57c6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:55.275 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.272+0000 7fa5f57c6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:55.275 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.272+0000 7fa5f57c6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:55.276 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.273+0000 7fa5f2fc1700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:55.276 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.273+0000 7fa5f2fc1700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:55.276 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.273+0000 7fa5f2fc1700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:55.276 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.273+0000 7fa5f2fc1700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:55.277 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:55.273+0000 7fa5f2fc1700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T21:59:55.308 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:55.307+0000 7fcbfc384680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T21:59:55.402 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:55.400+0000 7fcbfc384680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T21:59:55.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:59:55.578 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T21:59:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.194+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 68) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f8348079900 con 0x7f835013a960 2023-11-06T21:59:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.194+0000 7f834cff9700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.194+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f8350065f30 con 0x7f833405f2c0 2023-11-06T21:59:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.195+0000 7f8354f9b700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.196+0000 7f8354f9b700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f8344009880 tx=0x7f8344009060).ready entity=mgr.7292 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.208+0000 7f834cff9700 1 -- 172.21.15.33:0/1101097352 <== mgr.7292 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8350065f30 con 0x7f833405f2c0 2023-11-06T21:59:56.209 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path snap_test_dir1 2023-11-06T21:59:56.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 -- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 msgr2=0x7f8334061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f8344009880 tx=0x7f8344009060).stop 2023-11-06T21:59:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 -- 172.21.15.33:0/1101097352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f835013a960 msgr2=0x7f83500add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f835013a960 0x7f83500add20 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f835005f0e0 tx=0x7f8348009840).stop 2023-11-06T21:59:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 -- 172.21.15.33:0/1101097352 shutdown_connections 2023-11-06T21:59:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f833405f2c0 0x7f8334061700 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f835013a960 0x7f83500add20 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350066ba0 0x7f835014bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 --2- 172.21.15.33:0/1101097352 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f83500c9340 0x7f83500a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 -- 172.21.15.33:0/1101097352 >> 172.21.15.33:0/1101097352 conn(0x7f835005ec70 msgr2=0x7f83501546e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 -- 172.21.15.33:0/1101097352 shutdown_connections 2023-11-06T21:59:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.210+0000 7f83571ff700 1 -- 172.21.15.33:0/1101097352 wait complete. 2023-11-06T21:59:56.230 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-06T21:59:56.230 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-11-06T21:59:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.435+0000 7f50a1dff700 1 Processor -- start 2023-11-06T21:59:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.435+0000 7f50a1dff700 1 -- start start 2023-11-06T21:59:56.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.436+0000 7f50a1dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f509c066ba0 0x7f509c155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.436+0000 7f50a1dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f509c0c9340 0x7f509c0c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.436+0000 7f50a1dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509c063950 con 0x7f509c13a960 2023-11-06T21:59:56.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.436+0000 7f50a1dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c064cc0 con 0x7f509c0c9340 2023-11-06T21:59:56.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.436+0000 7f50a1dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c155930 con 0x7f509c066ba0 2023-11-06T21:59:56.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f509c13a960 0x7f509c066490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58598/0 (socket says 172.21.15.33:58598) 2023-11-06T21:59:56.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509b7fe700 1 -- 172.21.15.33:0/1936079853 learned_addr learned my addr 172.21.15.33:0/1936079853 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:56.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509bfff700 1 --2- 172.21.15.33:0/1936079853 >> v2:172.21.15.33:3300/0 conn(0x7f509c0c9340 0x7f509c0c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509affd700 1 --2- 172.21.15.33:0/1936079853 >> v2:172.21.15.69:3300/0 conn(0x7f509c066ba0 0x7f509c155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509a7fc700 1 -- 172.21.15.33:0/1936079853 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093734793 0 0) 0x7f509c063950 con 0x7f509c13a960 2023-11-06T21:59:56.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509a7fc700 1 -- 172.21.15.33:0/1936079853 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50840033f0 con 0x7f509c13a960 2023-11-06T21:59:56.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509bfff700 1 -- 172.21.15.33:0/1936079853 >> v2:172.21.15.69:3300/0 conn(0x7f509c066ba0 msgr2=0x7f509c155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:56.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509bfff700 1 --2- 172.21.15.33:0/1936079853 >> v2:172.21.15.69:3300/0 conn(0x7f509c066ba0 0x7f509c155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509bfff700 1 -- 172.21.15.33:0/1936079853 >> v1:172.21.15.33:6789/0 conn(0x7f509c13a960 legacy=0x7f509c066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509bfff700 1 -- 172.21.15.33:0/1936079853 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c063c30 con 0x7f509c0c9340 2023-11-06T21:59:56.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.437+0000 7f509bfff700 1 --2- 172.21.15.33:0/1936079853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c0c9340 0x7f509c0c37b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f509c06bd80 tx=0x7f5094009780).ready entity=mon.0 client_cookie=3324998503242734 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:56.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f509a7fc700 1 -- 172.21.15.33:0/1936079853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509400e070 con 0x7f509c0c9340 2023-11-06T21:59:56.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f509a7fc700 1 -- 172.21.15.33:0/1936079853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509400bab0 con 0x7f509c0c9340 2023-11-06T21:59:56.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f509a7fc700 1 -- 172.21.15.33:0/1936079853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509400be30 con 0x7f509c0c9340 2023-11-06T21:59:56.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f50a1dff700 1 -- 172.21.15.33:0/1936079853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c0c9340 msgr2=0x7f509c0c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f50a1dff700 1 --2- 172.21.15.33:0/1936079853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c0c9340 0x7f509c0c37b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f509c06bd80 tx=0x7f5094009780).stop 2023-11-06T21:59:56.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f50a1dff700 1 -- 172.21.15.33:0/1936079853 shutdown_connections 2023-11-06T21:59:56.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f50a1dff700 1 --2- 172.21.15.33:0/1936079853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c0c9340 0x7f509c0c37b0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f50a1dff700 1 --2- 172.21.15.33:0/1936079853 >> v2:172.21.15.69:3300/0 conn(0x7f509c066ba0 0x7f509c155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f50a1dff700 1 -- 172.21.15.33:0/1936079853 >> 172.21.15.33:0/1936079853 conn(0x7f509c05ec70 msgr2=0x7f509c0c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:56.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f50a1dff700 1 -- 172.21.15.33:0/1936079853 shutdown_connections 2023-11-06T21:59:56.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.438+0000 7f50a1dff700 1 -- 172.21.15.33:0/1936079853 wait complete. 2023-11-06T21:59:56.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.440+0000 7f50a1dff700 1 Processor -- start 2023-11-06T21:59:56.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.440+0000 7f50a1dff700 1 -- start start 2023-11-06T21:59:56.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f50a1dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c066ba0 0x7f509c0a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f50a1dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f509c0c9340 0x7f509c0a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f50a1dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f509c13a960 0x7f509c154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f50a1dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a4dd0 con 0x7f509c066ba0 2023-11-06T21:59:56.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f50a1dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a5c20 con 0x7f509c0c9340 2023-11-06T21:59:56.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f50a1dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a5010 con 0x7f509c13a960 2023-11-06T21:59:56.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f509b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c066ba0 0x7f509c0a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f509b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c066ba0 0x7f509c0a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59286/0 (socket says 172.21.15.33:59286) 2023-11-06T21:59:56.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f509b7fe700 1 -- 172.21.15.33:0/1690709014 learned_addr learned my addr 172.21.15.33:0/1690709014 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:56.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f509bfff700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f509c13a960 0x7f509c154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.441+0000 7f509affd700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f509c0c9340 0x7f509c0a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f509b7fe700 1 -- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f509c13a960 msgr2=0x7f509c154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f509b7fe700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f509c13a960 0x7f509c154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f509b7fe700 1 -- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f509c0c9340 msgr2=0x7f509c0a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f509b7fe700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f509c0c9340 0x7f509c0a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f509b7fe700 1 -- 172.21.15.33:0/1690709014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094009a20 con 0x7f509c066ba0 2023-11-06T21:59:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f509b7fe700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c066ba0 0x7f509c0a6d20 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f508c00c590 tx=0x7f508c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f50837fe700 1 -- 172.21.15.33:0/1690709014 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c0072e0 con 0x7f509c066ba0 2023-11-06T21:59:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f50837fe700 1 -- 172.21.15.33:0/1690709014 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c007440 con 0x7f509c066ba0 2023-11-06T21:59:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f50837fe700 1 -- 172.21.15.33:0/1690709014 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c0057a0 con 0x7f509c066ba0 2023-11-06T21:59:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f50a1dff700 1 -- 172.21.15.33:0/1690709014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c0a52f0 con 0x7f509c066ba0 2023-11-06T21:59:56.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.442+0000 7f50a1dff700 1 -- 172.21.15.33:0/1690709014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c0a58b0 con 0x7f509c066ba0 2023-11-06T21:59:56.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.444+0000 7f50a1dff700 1 -- 172.21.15.33:0/1690709014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c065f30 con 0x7f509c066ba0 2023-11-06T21:59:56.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.444+0000 7f50837fe700 1 -- 172.21.15.33:0/1690709014 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 68) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f508c005bd0 con 0x7f509c066ba0 2023-11-06T21:59:56.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.445+0000 7f50837fe700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508805f080 0x7f5088061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.445+0000 7f50837fe700 1 -- 172.21.15.33:0/1690709014 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f508c079870 con 0x7f509c066ba0 2023-11-06T21:59:56.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.445+0000 7f509affd700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508805f080 0x7f5088061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.446+0000 7f509affd700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508805f080 0x7f5088061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f509c0c1750 tx=0x7f5090009060).ready entity=mgr.7292 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:56.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.449+0000 7f50837fe700 1 -- 172.21.15.33:0/1690709014 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f508c04abe0 con 0x7f509c066ba0 2023-11-06T21:59:56.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.661+0000 7f50a1dff700 1 -- 172.21.15.33:0/1690709014 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f509c0634c0 con 0x7f508805f080 2023-11-06T21:59:56.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.667+0000 7f50837fe700 1 -- 172.21.15.33:0/1690709014 <== mgr.7292 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f509c0634c0 con 0x7f508805f080 2023-11-06T21:59:56.668 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path snap_test_dir1 2023-11-06T21:59:56.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 -- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508805f080 msgr2=0x7f5088061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508805f080 0x7f5088061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f509c0c1750 tx=0x7f5090009060).stop 2023-11-06T21:59:56.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 -- 172.21.15.33:0/1690709014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c066ba0 msgr2=0x7f509c0a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c066ba0 0x7f509c0a6d20 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f508c00c590 tx=0x7f508c007c80).stop 2023-11-06T21:59:56.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 -- 172.21.15.33:0/1690709014 shutdown_connections 2023-11-06T21:59:56.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f508805f080 0x7f5088061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f509c13a960 0x7f509c154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f509c066ba0 0x7f509c0a6d20 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 --2- 172.21.15.33:0/1690709014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f509c0c9340 0x7f509c0a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 -- 172.21.15.33:0/1690709014 >> 172.21.15.33:0/1690709014 conn(0x7f509c05ec70 msgr2=0x7f509c0c25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:56.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 -- 172.21.15.33:0/1690709014 shutdown_connections 2023-11-06T21:59:56.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.669+0000 7f50a1dff700 1 -- 172.21.15.33:0/1690709014 wait complete. 2023-11-06T21:59:56.688 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-11-06T21:59:56.689 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-11-06T21:59:56.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.888+0000 7f34657ff700 1 Processor -- start 2023-11-06T21:59:56.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.888+0000 7f34657ff700 1 -- start start 2023-11-06T21:59:56.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.889+0000 7f34657ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34600c8ee0 0x7f34600ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.889+0000 7f34657ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f34601542b0 0x7f34600c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.889+0000 7f34657ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34600c9f80 con 0x7f3460153ed0 2023-11-06T21:59:56.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.889+0000 7f34657ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34600c6000 con 0x7f34601542b0 2023-11-06T21:59:56.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.889+0000 7f34657ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34600c9370 con 0x7f34600c8ee0 2023-11-06T21:59:56.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.890+0000 7f345effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3460153ed0 0x7f3460158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58606/0 (socket says 172.21.15.33:58606) 2023-11-06T21:59:56.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.890+0000 7f345effd700 1 -- 172.21.15.33:0/750185280 learned_addr learned my addr 172.21.15.33:0/750185280 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:56.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.890+0000 7f345f7fe700 1 --2- 172.21.15.33:0/750185280 >> v2:172.21.15.69:3300/0 conn(0x7f34601542b0 0x7f34600c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.890+0000 7f345e7fc700 1 --2- 172.21.15.33:0/750185280 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34600c8ee0 0x7f34600ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.890+0000 7f345dffb700 1 -- 172.21.15.33:0/750185280 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058670281 0 0) 0x7f34600c9f80 con 0x7f3460153ed0 2023-11-06T21:59:56.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.890+0000 7f345dffb700 1 -- 172.21.15.33:0/750185280 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34480033f0 con 0x7f3460153ed0 2023-11-06T21:59:56.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.890+0000 7f345e7fc700 1 -- 172.21.15.33:0/750185280 >> v2:172.21.15.69:3300/0 conn(0x7f34601542b0 msgr2=0x7f34600c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:56.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.890+0000 7f345e7fc700 1 --2- 172.21.15.33:0/750185280 >> v2:172.21.15.69:3300/0 conn(0x7f34601542b0 0x7f34600c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.890+0000 7f345e7fc700 1 -- 172.21.15.33:0/750185280 >> v1:172.21.15.33:6789/0 conn(0x7f3460153ed0 legacy=0x7f3460158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.890+0000 7f345e7fc700 1 -- 172.21.15.33:0/750185280 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34600c9650 con 0x7f34600c8ee0 2023-11-06T21:59:56.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.891+0000 7f345f7fe700 1 --2- 172.21.15.33:0/750185280 >> v2:172.21.15.69:3300/0 conn(0x7f34601542b0 0x7f34600c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:56.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.891+0000 7f345e7fc700 1 --2- 172.21.15.33:0/750185280 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34600c8ee0 0x7f34600ca410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f3454009760 tx=0x7f3454009060).ready entity=mon.2 client_cookie=c1565832921aace7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:56.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.891+0000 7f345dffb700 1 -- 172.21.15.33:0/750185280 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400d4a0 con 0x7f34600c8ee0 2023-11-06T21:59:56.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.891+0000 7f345dffb700 1 -- 172.21.15.33:0/750185280 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345400d600 con 0x7f34600c8ee0 2023-11-06T21:59:56.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.891+0000 7f345dffb700 1 -- 172.21.15.33:0/750185280 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400d980 con 0x7f34600c8ee0 2023-11-06T21:59:56.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.892+0000 7f34657ff700 1 -- 172.21.15.33:0/750185280 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34600c8ee0 msgr2=0x7f34600ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.892+0000 7f34657ff700 1 --2- 172.21.15.33:0/750185280 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34600c8ee0 0x7f34600ca410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f3454009760 tx=0x7f3454009060).stop 2023-11-06T21:59:56.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.892+0000 7f34657ff700 1 -- 172.21.15.33:0/750185280 shutdown_connections 2023-11-06T21:59:56.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.892+0000 7f34657ff700 1 --2- 172.21.15.33:0/750185280 >> v2:172.21.15.69:3300/0 conn(0x7f34601542b0 0x7f34600c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.892+0000 7f34657ff700 1 --2- 172.21.15.33:0/750185280 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34600c8ee0 0x7f34600ca410 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.892+0000 7f34657ff700 1 -- 172.21.15.33:0/750185280 >> 172.21.15.33:0/750185280 conn(0x7f346005e1b0 msgr2=0x7f346005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:56.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.892+0000 7f34657ff700 1 -- 172.21.15.33:0/750185280 shutdown_connections 2023-11-06T21:59:56.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.892+0000 7f34657ff700 1 -- 172.21.15.33:0/750185280 wait complete. 2023-11-06T21:59:56.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.894+0000 7f34657ff700 1 Processor -- start 2023-11-06T21:59:56.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.894+0000 7f34657ff700 1 -- start start 2023-11-06T21:59:56.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.895+0000 7f34657ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3460153ed0 0x7f34600af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.895+0000 7f34657ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34601542b0 0x7f3460153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.895+0000 7f34657ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34600b06a0 0x7f34600c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.895+0000 7f34657ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346016cbe0 con 0x7f3460153ed0 2023-11-06T21:59:56.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.895+0000 7f34657ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346016bfd0 con 0x7f34601542b0 2023-11-06T21:59:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.895+0000 7f34657ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346016c2b0 con 0x7f34600b06a0 2023-11-06T21:59:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.895+0000 7f345effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3460153ed0 0x7f34600af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f345e7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34601542b0 0x7f3460153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f345effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3460153ed0 0x7f34600af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59298/0 (socket says 172.21.15.33:59298) 2023-11-06T21:59:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f345effd700 1 -- 172.21.15.33:0/530214909 learned_addr learned my addr 172.21.15.33:0/530214909 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f345f7fe700 1 --2- 172.21.15.33:0/530214909 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34600b06a0 0x7f34600c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f345effd700 1 -- 172.21.15.33:0/530214909 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34600b06a0 msgr2=0x7f34600c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f345effd700 1 --2- 172.21.15.33:0/530214909 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34600b06a0 0x7f34600c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f345effd700 1 -- 172.21.15.33:0/530214909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34601542b0 msgr2=0x7f3460153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f345effd700 1 --2- 172.21.15.33:0/530214909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34601542b0 0x7f3460153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f345effd700 1 -- 172.21.15.33:0/530214909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3458009a20 con 0x7f3460153ed0 2023-11-06T21:59:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f345effd700 1 --2- 172.21.15.33:0/530214909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3460153ed0 0x7f34600af620 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f3450001ed0 tx=0x7f3450007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.896+0000 7f3446ffd700 1 -- 172.21.15.33:0/530214909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34500072e0 con 0x7f3460153ed0 2023-11-06T21:59:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.897+0000 7f3446ffd700 1 -- 172.21.15.33:0/530214909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3450007440 con 0x7f3460153ed0 2023-11-06T21:59:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.897+0000 7f34657ff700 1 -- 172.21.15.33:0/530214909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34540092a0 con 0x7f3460153ed0 2023-11-06T21:59:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.897+0000 7f3446ffd700 1 -- 172.21.15.33:0/530214909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34500057a0 con 0x7f3460153ed0 2023-11-06T21:59:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.897+0000 7f34657ff700 1 -- 172.21.15.33:0/530214909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346016c870 con 0x7f3460153ed0 2023-11-06T21:59:56.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.898+0000 7f34657ff700 1 -- 172.21.15.33:0/530214909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346006c6c0 con 0x7f3460153ed0 2023-11-06T21:59:56.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.899+0000 7f3446ffd700 1 -- 172.21.15.33:0/530214909 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 68) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f3450005b50 con 0x7f3460153ed0 2023-11-06T21:59:56.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.900+0000 7f3446ffd700 1 --2- 172.21.15.33:0/530214909 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f344c05f0b0 0x7f344c061570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:56.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.900+0000 7f3446ffd700 1 -- 172.21.15.33:0/530214909 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f3450079780 con 0x7f3460153ed0 2023-11-06T21:59:56.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.901+0000 7f345e7fc700 1 --2- 172.21.15.33:0/530214909 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f344c05f0b0 0x7f344c061570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:56.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.901+0000 7f345e7fc700 1 --2- 172.21.15.33:0/530214909 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f344c05f0b0 0x7f344c061570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3454006d30 tx=0x7f3454016000).ready entity=mgr.7292 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:56.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:56.904+0000 7f3446ffd700 1 -- 172.21.15.33:0/530214909 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f345004aaf0 con 0x7f3460153ed0 2023-11-06T21:59:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.113+0000 7f34657ff700 1 -- 172.21.15.33:0/530214909 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f34600c9930 con 0x7f344c05f0b0 2023-11-06T21:59:57.115 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:57.114+0000 7fa5fcfd5700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-11-06T21:59:57.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.115+0000 7f3446ffd700 1 -- 172.21.15.33:0/530214909 <== mgr.7292 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f34600c9930 con 0x7f344c05f0b0 2023-11-06T21:59:57.118 INFO:teuthology.orchestra.run.smithi033.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-11-06T21:59:57.125 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-06T21:59:57.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:57.126 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-06T21:59:57.177 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T21:59:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.378+0000 7f406b7ff700 1 Processor -- start 2023-11-06T21:59:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.378+0000 7f406b7ff700 1 -- start start 2023-11-06T21:59:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.379+0000 7f406b7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4064157700 0x7f40640c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.379+0000 7f406b7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f40640aaf50 0x7f4064063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.379+0000 7f406b7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 0x7f4064063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f406b7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40640c18f0 con 0x7f40640aaf50 2023-11-06T21:59:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f406b7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4064063b10 con 0x7f4064157700 2023-11-06T21:59:57.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f406b7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4064158c00 con 0x7f40640aa4f0 2023-11-06T21:59:57.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4068d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f40640aaf50 0x7f4064063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:57.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4068d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f40640aaf50 0x7f4064063130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59312/0 (socket says 172.21.15.33:59312) 2023-11-06T21:59:57.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4068d9a700 1 -- 172.21.15.33:0/3457777016 learned_addr learned my addr 172.21.15.33:0/3457777016 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:57.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4069d9c700 1 --2- 172.21.15.33:0/3457777016 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 0x7f4064063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:57.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f406959b700 1 --2- 172.21.15.33:0/3457777016 >> v2:172.21.15.69:3300/0 conn(0x7f4064157700 0x7f40640c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:57.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4068d9a700 1 -- 172.21.15.33:0/3457777016 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 msgr2=0x7f4064063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:57.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4068d9a700 1 --2- 172.21.15.33:0/3457777016 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 0x7f4064063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4068d9a700 1 -- 172.21.15.33:0/3457777016 >> v2:172.21.15.69:3300/0 conn(0x7f4064157700 msgr2=0x7f40640c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4068d9a700 1 --2- 172.21.15.33:0/3457777016 >> v2:172.21.15.69:3300/0 conn(0x7f4064157700 0x7f40640c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4068d9a700 1 -- 172.21.15.33:0/3457777016 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4064157ff0 con 0x7f40640aaf50 2023-11-06T21:59:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4069d9c700 1 --2- 172.21.15.33:0/3457777016 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 0x7f4064063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f406959b700 1 --2- 172.21.15.33:0/3457777016 >> v2:172.21.15.69:3300/0 conn(0x7f4064157700 0x7f40640c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T21:59:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.380+0000 7f4068d9a700 1 --2- 172.21.15.33:0/3457777016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 0x7f4064063130 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f406406c380 tx=0x7f40540090f0).ready entity=mon.0 client_cookie=8ca406289694ea4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f405bfff700 1 -- 172.21.15.33:0/3457777016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405400b600 con 0x7f40640aaf50 2023-11-06T21:59:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f405bfff700 1 -- 172.21.15.33:0/3457777016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405400b760 con 0x7f40640aaf50 2023-11-06T21:59:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f405bfff700 1 -- 172.21.15.33:0/3457777016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405400bae0 con 0x7f40640aaf50 2023-11-06T21:59:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f406b7ff700 1 -- 172.21.15.33:0/3457777016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 msgr2=0x7f4064063130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f406b7ff700 1 --2- 172.21.15.33:0/3457777016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 0x7f4064063130 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f406406c380 tx=0x7f40540090f0).stop 2023-11-06T21:59:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f406b7ff700 1 -- 172.21.15.33:0/3457777016 shutdown_connections 2023-11-06T21:59:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f406b7ff700 1 --2- 172.21.15.33:0/3457777016 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 0x7f4064063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f406b7ff700 1 --2- 172.21.15.33:0/3457777016 >> v2:172.21.15.69:3300/0 conn(0x7f4064157700 0x7f40640c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f406b7ff700 1 --2- 172.21.15.33:0/3457777016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 0x7f4064063130 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f406b7ff700 1 -- 172.21.15.33:0/3457777016 >> 172.21.15.33:0/3457777016 conn(0x7f406405ec80 msgr2=0x7f40640cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f406b7ff700 1 -- 172.21.15.33:0/3457777016 shutdown_connections 2023-11-06T21:59:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.381+0000 7f406b7ff700 1 -- 172.21.15.33:0/3457777016 wait complete. 2023-11-06T21:59:57.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.383+0000 7f406b7ff700 1 Processor -- start 2023-11-06T21:59:57.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.383+0000 7f406b7ff700 1 -- start start 2023-11-06T21:59:57.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.384+0000 7f406b7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 0x7f4064148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:57.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.384+0000 7f406b7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 0x7f40640c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:57.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.384+0000 7f406b7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4064157700 0x7f40640c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:57.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.384+0000 7f406b7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40640c33e0 con 0x7f40640aaf50 2023-11-06T21:59:57.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.384+0000 7f406b7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40640c3870 con 0x7f4064157700 2023-11-06T21:59:57.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.384+0000 7f406b7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40640c27d0 con 0x7f40640aa4f0 2023-11-06T21:59:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.384+0000 7f4068d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 0x7f40640c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.384+0000 7f4068d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 0x7f40640c1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59314/0 (socket says 172.21.15.33:59314) 2023-11-06T21:59:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.384+0000 7f4068d9a700 1 -- 172.21.15.33:0/1449092012 learned_addr learned my addr 172.21.15.33:0/1449092012 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f406959b700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 0x7f4064148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f4069d9c700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4064157700 0x7f40640c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f4068d9a700 1 -- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 msgr2=0x7f4064148280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f4068d9a700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 0x7f4064148280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f4068d9a700 1 -- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4064157700 msgr2=0x7f40640c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f4068d9a700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4064157700 0x7f40640c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f4068d9a700 1 -- 172.21.15.33:0/1449092012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4054009370 con 0x7f40640aaf50 2023-11-06T21:59:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f4068d9a700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 0x7f40640c1ef0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f4054009130 tx=0x7f4054017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f40597fa700 1 -- 172.21.15.33:0/1449092012 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405400bae0 con 0x7f40640aaf50 2023-11-06T21:59:57.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f40597fa700 1 -- 172.21.15.33:0/1449092012 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054000c90 con 0x7f40640aaf50 2023-11-06T21:59:57.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f40597fa700 1 -- 172.21.15.33:0/1449092012 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405401c670 con 0x7f40640aaf50 2023-11-06T21:59:57.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f406b7ff700 1 -- 172.21.15.33:0/1449092012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40640c2ab0 con 0x7f40640aaf50 2023-11-06T21:59:57.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.385+0000 7f406b7ff700 1 -- 172.21.15.33:0/1449092012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40640c3070 con 0x7f40640aaf50 2023-11-06T21:59:57.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.386+0000 7f406b7ff700 1 -- 172.21.15.33:0/1449092012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406406c6c0 con 0x7f40640aaf50 2023-11-06T21:59:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.387+0000 7f40597fa700 1 -- 172.21.15.33:0/1449092012 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 69) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f405401ca20 con 0x7f40640aaf50 2023-11-06T21:59:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.388+0000 7f40597fa700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f404805f350 0x7f4048061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.388+0000 7f40597fa700 1 -- 172.21.15.33:0/1449092012 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f405407c5d0 con 0x7f40640aaf50 2023-11-06T21:59:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.389+0000 7f406959b700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f404805f350 0x7f4048061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.389+0000 7f406959b700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f404805f350 0x7f4048061810 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f406000a9d0 tx=0x7f4060008040).ready entity=mgr.7292 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.392+0000 7f40597fa700 1 -- 172.21.15.33:0/1449092012 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f405404d470 con 0x7f40640aaf50 2023-11-06T21:59:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:57.618+0000 7f406b7ff700 1 -- 172.21.15.33:0/1449092012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f40641585b0 con 0x7f40640aaf50 2023-11-06T21:59:58.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.254+0000 7f40597fa700 1 -- 172.21.15.33:0/1449092012 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f4054079f70 con 0x7f40640aaf50 2023-11-06T21:59:58.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.255+0000 7f40597fa700 1 -- 172.21.15.33:0/1449092012 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v70) v1 ==== 93+0+0 (secure 0 0 0) 0x7f40540426e0 con 0x7f40640aaf50 2023-11-06T21:59:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 -- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f404805f350 msgr2=0x7f4048061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f404805f350 0x7f4048061810 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f406000a9d0 tx=0x7f4060008040).stop 2023-11-06T21:59:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 -- 172.21.15.33:0/1449092012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 msgr2=0x7f40640c1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 0x7f40640c1ef0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f4054009130 tx=0x7f4054017000).stop 2023-11-06T21:59:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 -- 172.21.15.33:0/1449092012 shutdown_connections 2023-11-06T21:59:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f404805f350 0x7f4048061810 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f40640aa4f0 0x7f4064148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4064157700 0x7f40640c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 --2- 172.21.15.33:0/1449092012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40640aaf50 0x7f40640c1ef0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 -- 172.21.15.33:0/1449092012 >> 172.21.15.33:0/1449092012 conn(0x7f406405ec80 msgr2=0x7f40641522f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 -- 172.21.15.33:0/1449092012 shutdown_connections 2023-11-06T21:59:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.258+0000 7f406b7ff700 1 -- 172.21.15.33:0/1449092012 wait complete. 2023-11-06T21:59:58.280 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T21:59:58.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:59:58.381 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T21:59:58.478 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:58.477+0000 7f62dc175680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:59:58.480 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T21:59:58.481 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T21:59:58.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T21:59:58.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T21:59:58.487 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:58.485+0000 7fa04a9ae680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T21:59:58.564 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:58.563+0000 7f62dc175680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:59:58.566 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:58.565+0000 7fa04a9ae680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T21:59:58.570 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T21:59:58.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T21:59:58.571 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T21:59:58.619 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T21:59:58.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.816+0000 7f96157ff700 1 Processor -- start 2023-11-06T21:59:58.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.816+0000 7f96157ff700 1 -- start start 2023-11-06T21:59:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f96157ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f96100c7b80 0x7f9610154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f96157ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96100c5db0 con 0x7f9610154280 2023-11-06T21:59:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f96157ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96100c6930 con 0x7f96100ad6e0 2023-11-06T21:59:58.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f96157ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96100aa630 con 0x7f96100c7b80 2023-11-06T21:59:58.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f960effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9610154280 0x7f96100a9f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45322/0 (socket says 172.21.15.33:45322) 2023-11-06T21:59:58.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f960effd700 1 -- 172.21.15.33:0/2806062608 learned_addr learned my addr 172.21.15.33:0/2806062608 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:58.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f960f7fe700 1 --2- 172.21.15.33:0/2806062608 >> v2:172.21.15.69:3300/0 conn(0x7f96100c7b80 0x7f9610154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:58.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f960dffb700 1 -- 172.21.15.33:0/2806062608 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293866331 0 0) 0x7f96100c5db0 con 0x7f9610154280 2023-11-06T21:59:58.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f960dffb700 1 -- 172.21.15.33:0/2806062608 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95f80033f0 con 0x7f9610154280 2023-11-06T21:59:58.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f960dffb700 1 -- 172.21.15.33:0/2806062608 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465702006 0 0) 0x7f96100c6930 con 0x7f96100ad6e0 2023-11-06T21:59:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.818+0000 7f960dffb700 1 -- 172.21.15.33:0/2806062608 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96100c5db0 con 0x7f96100ad6e0 2023-11-06T21:59:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.819+0000 7f960f7fe700 1 -- 172.21.15.33:0/2806062608 >> v1:172.21.15.69:6789/0 conn(0x7f96100ad6e0 legacy=0x7f96100c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.819+0000 7f960f7fe700 1 -- 172.21.15.33:0/2806062608 >> v1:172.21.15.33:6789/0 conn(0x7f9610154280 legacy=0x7f96100a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.819+0000 7f960f7fe700 1 -- 172.21.15.33:0/2806062608 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96100c6000 con 0x7f96100c7b80 2023-11-06T21:59:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.819+0000 7f960f7fe700 1 --2- 172.21.15.33:0/2806062608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96100c7b80 0x7f9610154660 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f9608009e60 tx=0x7f9608009780).ready entity=mon.1 client_cookie=6d59ab32fe884e27 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.819+0000 7f960dffb700 1 -- 172.21.15.33:0/2806062608 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800e070 con 0x7f96100c7b80 2023-11-06T21:59:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.819+0000 7f960dffb700 1 -- 172.21.15.33:0/2806062608 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960800bab0 con 0x7f96100c7b80 2023-11-06T21:59:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.819+0000 7f960dffb700 1 -- 172.21.15.33:0/2806062608 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800be30 con 0x7f96100c7b80 2023-11-06T21:59:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.820+0000 7f96157ff700 1 -- 172.21.15.33:0/2806062608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96100c7b80 msgr2=0x7f9610154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:58.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.820+0000 7f96157ff700 1 --2- 172.21.15.33:0/2806062608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96100c7b80 0x7f9610154660 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f9608009e60 tx=0x7f9608009780).stop 2023-11-06T21:59:58.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.820+0000 7f96157ff700 1 -- 172.21.15.33:0/2806062608 shutdown_connections 2023-11-06T21:59:58.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.820+0000 7f96157ff700 1 --2- 172.21.15.33:0/2806062608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96100c7b80 0x7f9610154660 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:58.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.820+0000 7f96157ff700 1 -- 172.21.15.33:0/2806062608 >> 172.21.15.33:0/2806062608 conn(0x7f961005e1b0 msgr2=0x7f961005e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:58.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.820+0000 7f96157ff700 1 -- 172.21.15.33:0/2806062608 shutdown_connections 2023-11-06T21:59:58.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.820+0000 7f96157ff700 1 -- 172.21.15.33:0/2806062608 wait complete. 2023-11-06T21:59:58.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.821+0000 7f96157ff700 1 Processor -- start 2023-11-06T21:59:58.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.821+0000 7f96157ff700 1 -- start start 2023-11-06T21:59:58.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.822+0000 7f96157ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96100ad6e0 0x7f9610150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:58.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.822+0000 7f96157ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96100c7b80 0x7f96100c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:58.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.822+0000 7f96157ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610154280 0x7f9610159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:58.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.822+0000 7f96157ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96100ca2b0 con 0x7f9610154280 2023-11-06T21:59:58.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.822+0000 7f96157ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96100ca740 con 0x7f96100ad6e0 2023-11-06T21:59:58.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.822+0000 7f96157ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96100c96a0 con 0x7f96100c7b80 2023-11-06T21:59:58.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610154280 0x7f9610159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:58.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610154280 0x7f9610159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46118/0 (socket says 172.21.15.33:46118) 2023-11-06T21:59:58.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960f7fe700 1 -- 172.21.15.33:0/3710933278 learned_addr learned my addr 172.21.15.33:0/3710933278 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:58.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96100ad6e0 0x7f9610150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:58.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960e7fc700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96100c7b80 0x7f96100c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:58.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960f7fe700 1 -- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96100c7b80 msgr2=0x7f96100c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:58.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960f7fe700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96100c7b80 0x7f96100c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:58.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960f7fe700 1 -- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96100ad6e0 msgr2=0x7f9610150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:58.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960f7fe700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96100ad6e0 0x7f9610150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:58.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960f7fe700 1 -- 172.21.15.33:0/3710933278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608009a20 con 0x7f9610154280 2023-11-06T21:59:58.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.823+0000 7f960f7fe700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610154280 0x7f9610159380 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f96100841f0 tx=0x7f9608009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:58.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.824+0000 7f95f6ffd700 1 -- 172.21.15.33:0/3710933278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608012e30 con 0x7f9610154280 2023-11-06T21:59:58.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.824+0000 7f95f6ffd700 1 -- 172.21.15.33:0/3710933278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960801d630 con 0x7f9610154280 2023-11-06T21:59:58.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.824+0000 7f95f6ffd700 1 -- 172.21.15.33:0/3710933278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960801d9b0 con 0x7f9610154280 2023-11-06T21:59:58.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.824+0000 7f96157ff700 1 -- 172.21.15.33:0/3710933278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96100c9980 con 0x7f9610154280 2023-11-06T21:59:58.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.824+0000 7f96157ff700 1 -- 172.21.15.33:0/3710933278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96100c9f40 con 0x7f9610154280 2023-11-06T21:59:58.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.825+0000 7f96157ff700 1 -- 172.21.15.33:0/3710933278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96100ca460 con 0x7f9610154280 2023-11-06T21:59:58.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.827+0000 7f95f6ffd700 1 -- 172.21.15.33:0/3710933278 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f96080083a0 con 0x7f9610154280 2023-11-06T21:59:58.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.828+0000 7f95f6ffd700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f95fc05f300 0x7f95fc0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:58.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.828+0000 7f95f6ffd700 1 -- 172.21.15.33:0/3710933278 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f960807c470 con 0x7f9610154280 2023-11-06T21:59:58.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.828+0000 7f960effd700 1 -- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f95fc05f300 msgr2=0x7f95fc0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:58.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.828+0000 7f960effd700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f95fc05f300 0x7f95fc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:58.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:58.833+0000 7f95f6ffd700 1 -- 172.21.15.33:0/3710933278 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f96100ca460 con 0x7f9610154280 2023-11-06T21:59:58.899 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:58.896+0000 7fa04a9ae680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:59:58.926 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:58.925+0000 7f62dc175680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T21:59:59.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.029+0000 7f960effd700 1 -- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f95fc05f300 msgr2=0x7f95fc0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:59.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.029+0000 7f960effd700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f95fc05f300 0x7f95fc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T21:59:59.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.057+0000 7f96157ff700 1 -- 172.21.15.33:0/3710933278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96100ca3f0 con 0x7f9610154280 2023-11-06T21:59:59.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.058+0000 7f95f6ffd700 1 -- 172.21.15.33:0/3710933278 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f960804d1a0 con 0x7f9610154280 2023-11-06T21:59:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:59.060 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3645039939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3645039939},{"type":"v1","addr":"172.21.15.33:6835","nonce":3645039939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1156785194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1156785194},{"type":"v1","addr":"172.21.15.69:6836","nonce":1156785194}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/4183165491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":4183165491},{"type":"v1","addr":"172.21.15.69:6837","nonce":4183165491}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:59:40.865889+0000","modified":"2023-11-06T21:59:43.953031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8445":{"gid":8445,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/1588551062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1588551062},{"type":"v1","addr":"172.21.15.33:6833","nonce":1588551062}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-06T21:59:59.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.060+0000 7f96157ff700 1 -- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f95fc05f300 msgr2=0x7f95fc0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:59.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.060+0000 7f96157ff700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f95fc05f300 0x7f95fc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:59.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.060+0000 7f96157ff700 1 -- 172.21.15.33:0/3710933278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610154280 msgr2=0x7f9610159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.060+0000 7f96157ff700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610154280 0x7f9610159380 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f96100841f0 tx=0x7f9608009780).stop 2023-11-06T21:59:59.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.060+0000 7f96157ff700 1 -- 172.21.15.33:0/3710933278 shutdown_connections 2023-11-06T21:59:59.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.060+0000 7f96157ff700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f95fc05f300 0x7f95fc0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:59.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.060+0000 7f96157ff700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610154280 0x7f9610159380 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.060+0000 7f96157ff700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96100ad6e0 0x7f9610150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.060+0000 7f96157ff700 1 --2- 172.21.15.33:0/3710933278 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96100c7b80 0x7f96100c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.060+0000 7f96157ff700 1 -- 172.21.15.33:0/3710933278 >> 172.21.15.33:0/3710933278 conn(0x7f961005e1b0 msgr2=0x7f96100aaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.061+0000 7f96157ff700 1 -- 172.21.15.33:0/3710933278 shutdown_connections 2023-11-06T21:59:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.061+0000 7f96157ff700 1 -- 172.21.15.33:0/3710933278 wait complete. 2023-11-06T21:59:59.065 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 68 2023-11-06T21:59:59.080 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T21:59:59.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.281+0000 7ff3083ff700 1 Processor -- start 2023-11-06T21:59:59.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.281+0000 7ff3083ff700 1 -- start start 2023-11-06T21:59:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.282+0000 7ff3083ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff3000c3600 0x7ff3000c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.282+0000 7ff3083ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3000c17e0 con 0x7ff300136f50 2023-11-06T21:59:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.282+0000 7ff3083ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30006a740 con 0x7ff300069410 2023-11-06T21:59:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.282+0000 7ff3083ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff300069f70 con 0x7ff3000c3600 2023-11-06T21:59:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.283+0000 7ff30599a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff300136f50 0x7ff300137400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45324/0 (socket says 172.21.15.33:45324) 2023-11-06T21:59:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.283+0000 7ff30599a700 1 -- 172.21.15.33:0/619283149 learned_addr learned my addr 172.21.15.33:0/619283149 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.283+0000 7ff30619b700 1 --2- 172.21.15.33:0/619283149 >> v2:172.21.15.69:3300/0 conn(0x7ff3000c3600 0x7ff3000c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.283+0000 7ff305199700 1 -- 172.21.15.33:0/619283149 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418671851 0 0) 0x7ff30006a740 con 0x7ff300069410 2023-11-06T21:59:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.283+0000 7ff305199700 1 -- 172.21.15.33:0/619283149 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2ec0033f0 con 0x7ff300069410 2023-11-06T21:59:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.283+0000 7ff305199700 1 -- 172.21.15.33:0/619283149 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872730435 0 0) 0x7ff3000c17e0 con 0x7ff300136f50 2023-11-06T21:59:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.283+0000 7ff30619b700 1 -- 172.21.15.33:0/619283149 >> v1:172.21.15.69:6789/0 conn(0x7ff300069410 legacy=0x7ff300069860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.283+0000 7ff30619b700 1 -- 172.21.15.33:0/619283149 >> v1:172.21.15.33:6789/0 conn(0x7ff300136f50 legacy=0x7ff300137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.283+0000 7ff30619b700 1 -- 172.21.15.33:0/619283149 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30006a180 con 0x7ff3000c3600 2023-11-06T21:59:59.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.284+0000 7ff30619b700 1 --2- 172.21.15.33:0/619283149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3000c3600 0x7ff3000c39e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7ff2fc009760 tx=0x7ff2fc009060).ready entity=mon.1 client_cookie=d2c309c0ba61fd78 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:59.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.284+0000 7ff305199700 1 -- 172.21.15.33:0/619283149 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00f510 con 0x7ff3000c3600 2023-11-06T21:59:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.284+0000 7ff305199700 1 -- 172.21.15.33:0/619283149 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc00f670 con 0x7ff3000c3600 2023-11-06T21:59:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.284+0000 7ff305199700 1 -- 172.21.15.33:0/619283149 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00f9f0 con 0x7ff3000c3600 2023-11-06T21:59:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.284+0000 7ff3083ff700 1 -- 172.21.15.33:0/619283149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3000c3600 msgr2=0x7ff3000c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.284+0000 7ff3083ff700 1 --2- 172.21.15.33:0/619283149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3000c3600 0x7ff3000c39e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7ff2fc009760 tx=0x7ff2fc009060).stop 2023-11-06T21:59:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.285+0000 7ff3083ff700 1 -- 172.21.15.33:0/619283149 shutdown_connections 2023-11-06T21:59:59.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.285+0000 7ff3083ff700 1 --2- 172.21.15.33:0/619283149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3000c3600 0x7ff3000c39e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.285+0000 7ff3083ff700 1 -- 172.21.15.33:0/619283149 >> 172.21.15.33:0/619283149 conn(0x7ff300062710 msgr2=0x7ff300062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:59.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.285+0000 7ff3083ff700 1 -- 172.21.15.33:0/619283149 shutdown_connections 2023-11-06T21:59:59.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.285+0000 7ff3083ff700 1 -- 172.21.15.33:0/619283149 wait complete. 2023-11-06T21:59:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.286+0000 7ff3083ff700 1 Processor -- start 2023-11-06T21:59:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.286+0000 7ff3083ff700 1 -- start start 2023-11-06T21:59:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.287+0000 7ff3083ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff300069410 0x7ff300136650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.287+0000 7ff3083ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3000c3600 0x7ff300152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.287+0000 7ff3083ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff300136f50 0x7ff300152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.287+0000 7ff3083ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff300153010 con 0x7ff300136f50 2023-11-06T21:59:59.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.287+0000 7ff3083ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff300165650 con 0x7ff300069410 2023-11-06T21:59:59.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.287+0000 7ff3083ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000c6650 con 0x7ff3000c3600 2023-11-06T21:59:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30619b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff300069410 0x7ff300136650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30619b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff300069410 0x7ff300136650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:35916/0 (socket says 172.21.15.33:35916) 2023-11-06T21:59:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30619b700 1 -- 172.21.15.33:0/2283384585 learned_addr learned my addr 172.21.15.33:0/2283384585 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30699c700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff300136f50 0x7ff300152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30599a700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3000c3600 0x7ff300152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30619b700 1 -- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3000c3600 msgr2=0x7ff300152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30619b700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3000c3600 0x7ff300152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30619b700 1 -- 172.21.15.33:0/2283384585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff300136f50 msgr2=0x7ff300152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30619b700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff300136f50 0x7ff300152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30619b700 1 -- 172.21.15.33:0/2283384585 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc0092a0 con 0x7ff300069410 2023-11-06T21:59:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff30619b700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff300069410 0x7ff300136650 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7ff2fc009e00 tx=0x7ff2fc017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff2f67fc700 1 -- 172.21.15.33:0/2283384585 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00f7e0 con 0x7ff300069410 2023-11-06T21:59:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff2f67fc700 1 -- 172.21.15.33:0/2283384585 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc00f940 con 0x7ff300069410 2023-11-06T21:59:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.288+0000 7ff2f67fc700 1 -- 172.21.15.33:0/2283384585 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc01b660 con 0x7ff300069410 2023-11-06T21:59:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.289+0000 7ff3083ff700 1 -- 172.21.15.33:0/2283384585 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3000c5db0 con 0x7ff300069410 2023-11-06T21:59:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.289+0000 7ff3083ff700 1 -- 172.21.15.33:0/2283384585 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3000c6370 con 0x7ff300069410 2023-11-06T21:59:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.290+0000 7ff3083ff700 1 -- 172.21.15.33:0/2283384585 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30006a460 con 0x7ff300069410 2023-11-06T21:59:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.290+0000 7ff2f67fc700 1 -- 172.21.15.33:0/2283384585 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7ff2fc01b9d0 con 0x7ff300069410 2023-11-06T21:59:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.291+0000 7ff2f67fc700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2e405f300 0x7ff2e40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.291+0000 7ff2f67fc700 1 -- 172.21.15.33:0/2283384585 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7ff2fc07bd50 con 0x7ff300069410 2023-11-06T21:59:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.291+0000 7ff30599a700 1 -- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2e405f300 msgr2=0x7ff2e40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.291+0000 7ff30599a700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2e405f300 0x7ff2e40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.296+0000 7ff2f67fc700 1 -- 172.21.15.33:0/2283384585 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff2fc04ccf0 con 0x7ff300069410 2023-11-06T21:59:59.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.479+0000 7ff3083ff700 1 -- 172.21.15.33:0/2283384585 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff300065750 con 0x7ff300069410 2023-11-06T21:59:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.480+0000 7ff2f67fc700 1 -- 172.21.15.33:0/2283384585 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+17427 (secure 0 0 0) 0x7ff2fc01f020 con 0x7ff300069410 2023-11-06T21:59:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T21:59:59.482 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":171,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:59:55.184836+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:59:38.416213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T21:59:39.859883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6833/114912":"2023-11-07T21:59:55.184802+0000","172.21.15.69:6832/114912":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/351947370":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/623316503":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/861101334":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/1093219585":"2023-11-07T21:59:55.184802+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T21:59:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 -- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2e405f300 msgr2=0x7ff2e40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2e405f300 0x7ff2e40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 -- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff300069410 msgr2=0x7ff300136650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff300069410 0x7ff300136650 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7ff2fc009e00 tx=0x7ff2fc017000).stop 2023-11-06T21:59:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 -- 172.21.15.33:0/2283384585 shutdown_connections 2023-11-06T21:59:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff2e405f300 0x7ff2e40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff300136f50 0x7ff300152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff300069410 0x7ff300136650 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 --2- 172.21.15.33:0/2283384585 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff3000c3600 0x7ff300152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 -- 172.21.15.33:0/2283384585 >> 172.21.15.33:0/2283384585 conn(0x7ff300062710 msgr2=0x7ff3001508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 -- 172.21.15.33:0/2283384585 shutdown_connections 2023-11-06T21:59:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.483+0000 7ff3083ff700 1 -- 172.21.15.33:0/2283384585 wait complete. 2023-11-06T21:59:59.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T21:59:38.416213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T21:59:39.859883+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T21:59:59.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T21:59:59.646 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:59.645+0000 7fa04a9ae680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:59:59.723 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:59.722+0000 7f62dc175680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T21:59:59.731 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:59.730+0000 7fa04a9ae680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:59:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.733+0000 7f78593ff700 1 Processor -- start 2023-11-06T21:59:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.733+0000 7f78593ff700 1 -- start start 2023-11-06T21:59:59.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.734+0000 7f78593ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78540c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.734+0000 7f78593ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f78540c7ff0 0x7f78540646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.734+0000 7f78593ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78540c6b60 con 0x7f7854154330 2023-11-06T21:59:59.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.734+0000 7f78593ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78540c7700 con 0x7f78540c7ff0 2023-11-06T21:59:59.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.734+0000 7f78593ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78540c5cd0 con 0x7f7854154710 2023-11-06T21:59:59.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.735+0000 7f78527fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78540c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:59.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.735+0000 7f7852ffd700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f7854154330 0x7f78540c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:55016/0 (socket says 172.21.15.33:55016) 2023-11-06T21:59:59.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.735+0000 7f7852ffd700 1 -- 172.21.15.33:0/3315839482 learned_addr learned my addr 172.21.15.33:0/3315839482 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T21:59:59.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.735+0000 7f78537fe700 1 --2- 172.21.15.33:0/3315839482 >> v2:172.21.15.69:3300/0 conn(0x7f78540c7ff0 0x7f78540646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:59.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.735+0000 7f78527fc700 1 -- 172.21.15.33:0/3315839482 >> v2:172.21.15.69:3300/0 conn(0x7f78540c7ff0 msgr2=0x7f78540646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T21:59:59.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.735+0000 7f78527fc700 1 --2- 172.21.15.33:0/3315839482 >> v2:172.21.15.69:3300/0 conn(0x7f78540c7ff0 0x7f78540646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.735+0000 7f7851ffb700 1 -- 172.21.15.33:0/3315839482 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531500015 0 0) 0x7f78540c6b60 con 0x7f7854154330 2023-11-06T21:59:59.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.735+0000 7f78527fc700 1 -- 172.21.15.33:0/3315839482 >> v1:172.21.15.69:6789/0 conn(0x7f7854154330 legacy=0x7f78540c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.735+0000 7f78527fc700 1 -- 172.21.15.33:0/3315839482 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78540c5f50 con 0x7f7854154710 2023-11-06T21:59:59.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.736+0000 7f78527fc700 1 --2- 172.21.15.33:0/3315839482 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78540c57e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f7848009760 tx=0x7f7848009060).ready entity=mon.2 client_cookie=c51ac084aa42ce78 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:59.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.736+0000 7f7851ffb700 1 -- 172.21.15.33:0/3315839482 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784800f510 con 0x7f7854154710 2023-11-06T21:59:59.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.736+0000 7f7851ffb700 1 -- 172.21.15.33:0/3315839482 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784800f670 con 0x7f7854154710 2023-11-06T21:59:59.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.736+0000 7f7851ffb700 1 -- 172.21.15.33:0/3315839482 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784800f9f0 con 0x7f7854154710 2023-11-06T21:59:59.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.736+0000 7f78593ff700 1 -- 172.21.15.33:0/3315839482 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 msgr2=0x7f78540c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.736+0000 7f78593ff700 1 --2- 172.21.15.33:0/3315839482 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78540c57e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f7848009760 tx=0x7f7848009060).stop 2023-11-06T21:59:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.736+0000 7f78593ff700 1 -- 172.21.15.33:0/3315839482 shutdown_connections 2023-11-06T21:59:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.736+0000 7f78593ff700 1 --2- 172.21.15.33:0/3315839482 >> v2:172.21.15.69:3300/0 conn(0x7f78540c7ff0 0x7f78540646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.736+0000 7f78593ff700 1 --2- 172.21.15.33:0/3315839482 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78540c57e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.736+0000 7f78593ff700 1 -- 172.21.15.33:0/3315839482 >> 172.21.15.33:0/3315839482 conn(0x7f78540cb910 msgr2=0x7f78540c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:59.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.737+0000 7f78593ff700 1 -- 172.21.15.33:0/3315839482 shutdown_connections 2023-11-06T21:59:59.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.737+0000 7f78593ff700 1 -- 172.21.15.33:0/3315839482 wait complete. 2023-11-06T21:59:59.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.738+0000 7f78593ff700 1 Processor -- start 2023-11-06T21:59:59.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.738+0000 7f78593ff700 1 -- start start 2023-11-06T21:59:59.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.739+0000 7f78593ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78540c7ff0 0x7f7854164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.739+0000 7f78593ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7854154330 0x7f78540aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.739+0000 7f78593ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78541539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.739+0000 7f78593ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540aee40 con 0x7f78540c7ff0 2023-11-06T21:59:59.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.739+0000 7f78593ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78541574d0 con 0x7f7854154330 2023-11-06T21:59:59.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.739+0000 7f78593ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78541577b0 con 0x7f7854154710 2023-11-06T21:59:59.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78527fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7854154330 0x7f78540aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:59.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f7852ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78540c7ff0 0x7f7854164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:59.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78537fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78541539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T21:59:59.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78537fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78541539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:60944/0 (socket says 172.21.15.33:60944) 2023-11-06T21:59:59.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78527fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7854154330 0x7f78540aba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:35932/0 (socket says 172.21.15.33:35932) 2023-11-06T21:59:59.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78537fe700 1 -- 172.21.15.33:0/1834777444 learned_addr learned my addr 172.21.15.33:0/1834777444 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T21:59:59.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78537fe700 1 -- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7854154330 msgr2=0x7f78540aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78537fe700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7854154330 0x7f78540aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78537fe700 1 -- 172.21.15.33:0/1834777444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78540c7ff0 msgr2=0x7f7854164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78537fe700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78540c7ff0 0x7f7854164b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78537fe700 1 -- 172.21.15.33:0/1834777444 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78480092a0 con 0x7f7854154710 2023-11-06T21:59:59.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78527fc700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7854154330 0x7f78540aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T21:59:59.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f78537fe700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78541539f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f784c009df0 tx=0x7f784c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T21:59:59.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.740+0000 7f783affd700 1 -- 172.21.15.33:0/1834777444 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c00caa0 con 0x7f7854154710 2023-11-06T21:59:59.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.741+0000 7f783affd700 1 -- 172.21.15.33:0/1834777444 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c00f040 con 0x7f7854154710 2023-11-06T21:59:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.741+0000 7f783affd700 1 -- 172.21.15.33:0/1834777444 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c00ce10 con 0x7f7854154710 2023-11-06T21:59:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.741+0000 7f78593ff700 1 -- 172.21.15.33:0/1834777444 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78540af0b0 con 0x7f7854154710 2023-11-06T21:59:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.741+0000 7f78593ff700 1 -- 172.21.15.33:0/1834777444 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78540abf40 con 0x7f7854154710 2023-11-06T21:59:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.742+0000 7f78593ff700 1 -- 172.21.15.33:0/1834777444 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78540830b0 con 0x7f7854154710 2023-11-06T21:59:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.743+0000 7f783affd700 1 -- 172.21.15.33:0/1834777444 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f784c013610 con 0x7f7854154710 2023-11-06T21:59:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.744+0000 7f783affd700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f784005f300 0x7f78400617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T21:59:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.744+0000 7f783affd700 1 -- 172.21.15.33:0/1834777444 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f784c07aed0 con 0x7f7854154710 2023-11-06T21:59:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.744+0000 7f7852ffd700 1 -- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f784005f300 msgr2=0x7f78400617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T21:59:59.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.744+0000 7f7852ffd700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f784005f300 0x7f78400617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T21:59:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.748+0000 7f783affd700 1 -- 172.21.15.33:0/1834777444 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f784c04bc60 con 0x7f7854154710 2023-11-06T21:59:59.812 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T21:59:59.811+0000 7f62dc175680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T21:59:59.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.929+0000 7f78593ff700 1 -- 172.21.15.33:0/1834777444 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7854062b00 con 0x7f7854154710 2023-11-06T21:59:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.930+0000 7f783affd700 1 -- 172.21.15.33:0/1834777444 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v171) v1 ==== 110+0+10 (secure 0 0 0) 0x7f784c04f280 con 0x7f7854154710 2023-11-06T21:59:59.931 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T21:59:59.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 -- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f784005f300 msgr2=0x7f78400617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T21:59:59.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f784005f300 0x7f78400617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:59.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 -- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 msgr2=0x7f78541539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T21:59:59.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78541539f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f784c009df0 tx=0x7f784c00b040).stop 2023-11-06T21:59:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 -- 172.21.15.33:0/1834777444 shutdown_connections 2023-11-06T21:59:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f784005f300 0x7f78400617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T21:59:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7854154710 0x7f78541539f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78540c7ff0 0x7f7854164b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 --2- 172.21.15.33:0/1834777444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7854154330 0x7f78540aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T21:59:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 -- 172.21.15.33:0/1834777444 >> 172.21.15.33:0/1834777444 conn(0x7f78540cb910 msgr2=0x7f78540646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T21:59:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 -- 172.21.15.33:0/1834777444 shutdown_connections 2023-11-06T21:59:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T21:59:59.932+0000 7f78593ff700 1 -- 172.21.15.33:0/1834777444 wait complete. 2023-11-06T21:59:59.952 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T21:59:59.967 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T21:59:59.965+0000 7fa04a9ae680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:00:00.059 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:00.058+0000 7f62dc175680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:00:00.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.166+0000 7f39901ff700 1 Processor -- start 2023-11-06T22:00:00.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.166+0000 7f39901ff700 1 -- start start 2023-11-06T22:00:00.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.167+0000 7f39901ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f39881544f0 0x7f39880c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.167+0000 7f39901ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39881556d0 con 0x7f3988064dd0 2023-11-06T22:00:00.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.167+0000 7f39901ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3988154ac0 con 0x7f39880c0c40 2023-11-06T22:00:00.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.167+0000 7f39901ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39881548d0 con 0x7f39881544f0 2023-11-06T22:00:00.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.167+0000 7f398e79c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3988064dd0 0x7f39880c57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45336/0 (socket says 172.21.15.33:45336) 2023-11-06T22:00:00.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.167+0000 7f398e79c700 1 -- 172.21.15.33:0/630010939 learned_addr learned my addr 172.21.15.33:0/630010939 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:00.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.167+0000 7f398df9b700 1 --2- 172.21.15.33:0/630010939 >> v2:172.21.15.69:3300/0 conn(0x7f39881544f0 0x7f39880c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:00.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.168+0000 7f398cf99700 1 -- 172.21.15.33:0/630010939 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169635372 0 0) 0x7f39881556d0 con 0x7f3988064dd0 2023-11-06T22:00:00.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.168+0000 7f398cf99700 1 -- 172.21.15.33:0/630010939 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39740033f0 con 0x7f3988064dd0 2023-11-06T22:00:00.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.168+0000 7f398cf99700 1 -- 172.21.15.33:0/630010939 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739633556 0 0) 0x7f3988154ac0 con 0x7f39880c0c40 2023-11-06T22:00:00.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.168+0000 7f398df9b700 1 -- 172.21.15.33:0/630010939 >> v1:172.21.15.69:6789/0 conn(0x7f39880c0c40 legacy=0x7f39880646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.168+0000 7f398df9b700 1 -- 172.21.15.33:0/630010939 >> v1:172.21.15.33:6789/0 conn(0x7f3988064dd0 legacy=0x7f39880c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.168+0000 7f398df9b700 1 -- 172.21.15.33:0/630010939 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3988154da0 con 0x7f39881544f0 2023-11-06T22:00:00.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.168+0000 7f398df9b700 1 --2- 172.21.15.33:0/630010939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39881544f0 0x7f39880c0750 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f3984009760 tx=0x7f3984009060).ready entity=mon.1 client_cookie=8388af407732fc9c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:00.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.168+0000 7f398cf99700 1 -- 172.21.15.33:0/630010939 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398400f510 con 0x7f39881544f0 2023-11-06T22:00:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.168+0000 7f398cf99700 1 -- 172.21.15.33:0/630010939 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398400f670 con 0x7f39881544f0 2023-11-06T22:00:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.169+0000 7f398cf99700 1 -- 172.21.15.33:0/630010939 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398400f9f0 con 0x7f39881544f0 2023-11-06T22:00:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.169+0000 7f39901ff700 1 -- 172.21.15.33:0/630010939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39881544f0 msgr2=0x7f39880c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.169+0000 7f39901ff700 1 --2- 172.21.15.33:0/630010939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39881544f0 0x7f39880c0750 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f3984009760 tx=0x7f3984009060).stop 2023-11-06T22:00:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.169+0000 7f39901ff700 1 -- 172.21.15.33:0/630010939 shutdown_connections 2023-11-06T22:00:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.169+0000 7f39901ff700 1 --2- 172.21.15.33:0/630010939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39881544f0 0x7f39880c0750 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.169+0000 7f39901ff700 1 -- 172.21.15.33:0/630010939 >> 172.21.15.33:0/630010939 conn(0x7f39880cb900 msgr2=0x7f39880c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.169+0000 7f39901ff700 1 -- 172.21.15.33:0/630010939 shutdown_connections 2023-11-06T22:00:00.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.169+0000 7f39901ff700 1 -- 172.21.15.33:0/630010939 wait complete. 2023-11-06T22:00:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.171+0000 7f39901ff700 1 Processor -- start 2023-11-06T22:00:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.171+0000 7f39901ff700 1 -- start start 2023-11-06T22:00:00.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f39901ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3988064dd0 0x7f39880afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f39901ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39880c0c40 0x7f39880c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f39901ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39881544f0 0x7f3988155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f39901ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39880afb20 con 0x7f39880c0c40 2023-11-06T22:00:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f39901ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39880ae270 con 0x7f39881544f0 2023-11-06T22:00:00.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f39901ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39880ae550 con 0x7f3988064dd0 2023-11-06T22:00:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f398d79a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39880c0c40 0x7f39880c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f398d79a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39880c0c40 0x7f39880c4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46148/0 (socket says 172.21.15.33:46148) 2023-11-06T22:00:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f398d79a700 1 -- 172.21.15.33:0/607038777 learned_addr learned my addr 172.21.15.33:0/607038777 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f398df9b700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3988064dd0 0x7f39880afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:00.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.172+0000 7f398e79c700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39881544f0 0x7f3988155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:00.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f398d79a700 1 -- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3988064dd0 msgr2=0x7f39880afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f398d79a700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3988064dd0 0x7f39880afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f398d79a700 1 -- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39881544f0 msgr2=0x7f3988155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f398d79a700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39881544f0 0x7f3988155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f398d79a700 1 -- 172.21.15.33:0/607038777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39840092a0 con 0x7f39880c0c40 2023-11-06T22:00:00.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f398d79a700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39880c0c40 0x7f39880c4600 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f3978001eb0 tx=0x7f3978007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:00.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f397dffb700 1 -- 172.21.15.33:0/607038777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397800ce60 con 0x7f39880c0c40 2023-11-06T22:00:00.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f397dffb700 1 -- 172.21.15.33:0/607038777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397800f470 con 0x7f39880c0c40 2023-11-06T22:00:00.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f397dffb700 1 -- 172.21.15.33:0/607038777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397800f810 con 0x7f39880c0c40 2023-11-06T22:00:00.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f39901ff700 1 -- 172.21.15.33:0/607038777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39880ae830 con 0x7f39880c0c40 2023-11-06T22:00:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.173+0000 7f39901ff700 1 -- 172.21.15.33:0/607038777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39880aedf0 con 0x7f39880c0c40 2023-11-06T22:00:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.174+0000 7f39901ff700 1 -- 172.21.15.33:0/607038777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39880b0430 con 0x7f39880c0c40 2023-11-06T22:00:00.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.175+0000 7f397dffb700 1 -- 172.21.15.33:0/607038777 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f397800fb80 con 0x7f39880c0c40 2023-11-06T22:00:00.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.176+0000 7f397dffb700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f396c05f300 0x7f396c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.176+0000 7f397dffb700 1 -- 172.21.15.33:0/607038777 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f3978079ad0 con 0x7f39880c0c40 2023-11-06T22:00:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.176+0000 7f398df9b700 1 -- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f396c05f300 msgr2=0x7f396c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.176+0000 7f398df9b700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f396c05f300 0x7f396c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:00:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.180+0000 7f397dffb700 1 -- 172.21.15.33:0/607038777 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f397804aa70 con 0x7f39880c0c40 2023-11-06T22:00:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.360+0000 7f39901ff700 1 -- 172.21.15.33:0/607038777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3988155080 con 0x7f39880c0c40 2023-11-06T22:00:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.360+0000 7f397dffb700 1 -- 172.21.15.33:0/607038777 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v171) v1 ==== 92+0+10 (secure 0 0 0) 0x7f397804e090 con 0x7f39880c0c40 2023-11-06T22:00:00.362 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T22:00:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 -- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f396c05f300 msgr2=0x7f396c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:00:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f396c05f300 0x7f396c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 -- 172.21.15.33:0/607038777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39880c0c40 msgr2=0x7f39880c4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39880c0c40 0x7f39880c4600 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f3978001eb0 tx=0x7f3978007c70).stop 2023-11-06T22:00:00.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 -- 172.21.15.33:0/607038777 shutdown_connections 2023-11-06T22:00:00.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f396c05f300 0x7f396c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:00.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3988064dd0 0x7f39880afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39881544f0 0x7f3988155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 --2- 172.21.15.33:0/607038777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39880c0c40 0x7f39880c4600 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 -- 172.21.15.33:0/607038777 >> 172.21.15.33:0/607038777 conn(0x7f39880cb900 msgr2=0x7f3988153b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 -- 172.21.15.33:0/607038777 shutdown_connections 2023-11-06T22:00:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.363+0000 7f39901ff700 1 -- 172.21.15.33:0/607038777 wait complete. 2023-11-06T22:00:00.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T22:00:00.483 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:00.481+0000 7fa04a9ae680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:00:00.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.583+0000 7f69623ff700 1 Processor -- start 2023-11-06T22:00:00.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.583+0000 7f69623ff700 1 -- start start 2023-11-06T22:00:00.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f69623ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f695c0c7590 0x7f695c156310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f69623ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f695c156800 0x7f695c138420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f69623ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f695c0bde10 con 0x7f695c155f30 2023-11-06T22:00:00.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f69623ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c156c30 con 0x7f695c156800 2023-11-06T22:00:00.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f69623ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c0bbde0 con 0x7f695c0c7590 2023-11-06T22:00:00.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f696099c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f695c156800 0x7f695c138420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:00.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f695b7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f695c0c7590 0x7f695c156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:00.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f696099c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f695c156800 0x7f695c138420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46158/0 (socket says 172.21.15.33:46158) 2023-11-06T22:00:00.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f695b7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f695c0c7590 0x7f695c156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:35950/0 (socket says 172.21.15.33:35950) 2023-11-06T22:00:00.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f696099c700 1 -- 172.21.15.33:0/1953543358 learned_addr learned my addr 172.21.15.33:0/1953543358 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:00.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.585+0000 7f695bfff700 1 --1- 172.21.15.33:0/1953543358 >> v1:172.21.15.69:6789/0 conn(0x7f695c155f30 0x7f695c0a6a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:55026/0 (socket says 172.21.15.33:55026) 2023-11-06T22:00:00.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.586+0000 7f696099c700 1 -- 172.21.15.33:0/1953543358 >> v2:172.21.15.69:3300/0 conn(0x7f695c0c7590 msgr2=0x7f695c156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:00.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.586+0000 7f696099c700 1 --2- 172.21.15.33:0/1953543358 >> v2:172.21.15.69:3300/0 conn(0x7f695c0c7590 0x7f695c156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.586+0000 7f696099c700 1 -- 172.21.15.33:0/1953543358 >> v1:172.21.15.69:6789/0 conn(0x7f695c155f30 legacy=0x7f695c0a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.586+0000 7f696099c700 1 -- 172.21.15.33:0/1953543358 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c0bc5f0 con 0x7f695c156800 2023-11-06T22:00:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.586+0000 7f695affd700 1 -- 172.21.15.33:0/1953543358 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728179090 0 0) 0x7f695c0bde10 con 0x7f695c155f30 2023-11-06T22:00:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.586+0000 7f695b7fe700 1 --2- 172.21.15.33:0/1953543358 >> v2:172.21.15.69:3300/0 conn(0x7f695c0c7590 0x7f695c156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:00:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.586+0000 7f696099c700 1 --2- 172.21.15.33:0/1953543358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c156800 0x7f695c138420 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f695c0a9ad0 tx=0x7f6954009840).ready entity=mon.0 client_cookie=a3659ddcbb2a2c08 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.586+0000 7f695affd700 1 -- 172.21.15.33:0/1953543358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695400e070 con 0x7f695c156800 2023-11-06T22:00:00.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.586+0000 7f695affd700 1 -- 172.21.15.33:0/1953543358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695400ba20 con 0x7f695c156800 2023-11-06T22:00:00.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.586+0000 7f695affd700 1 -- 172.21.15.33:0/1953543358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695400bda0 con 0x7f695c156800 2023-11-06T22:00:00.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.587+0000 7f69623ff700 1 -- 172.21.15.33:0/1953543358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c156800 msgr2=0x7f695c138420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.587+0000 7f69623ff700 1 --2- 172.21.15.33:0/1953543358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c156800 0x7f695c138420 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f695c0a9ad0 tx=0x7f6954009840).stop 2023-11-06T22:00:00.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.587+0000 7f69623ff700 1 -- 172.21.15.33:0/1953543358 shutdown_connections 2023-11-06T22:00:00.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.587+0000 7f69623ff700 1 --2- 172.21.15.33:0/1953543358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c156800 0x7f695c138420 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.587+0000 7f69623ff700 1 --2- 172.21.15.33:0/1953543358 >> v2:172.21.15.69:3300/0 conn(0x7f695c0c7590 0x7f695c156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.587+0000 7f69623ff700 1 -- 172.21.15.33:0/1953543358 >> 172.21.15.33:0/1953543358 conn(0x7f695c0631e0 msgr2=0x7f695c0c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:00.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.587+0000 7f69623ff700 1 -- 172.21.15.33:0/1953543358 shutdown_connections 2023-11-06T22:00:00.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.587+0000 7f69623ff700 1 -- 172.21.15.33:0/1953543358 wait complete. 2023-11-06T22:00:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.589+0000 7f69623ff700 1 Processor -- start 2023-11-06T22:00:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.590+0000 7f69623ff700 1 -- start start 2023-11-06T22:00:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.591+0000 7f69623ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c0c7590 0x7f695c155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.591+0000 7f69623ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c155f30 0x7f695c0c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.591+0000 7f69623ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f695c156800 0x7f695c151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.591+0000 7f69623ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c13bc90 con 0x7f695c155f30 2023-11-06T22:00:00.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.591+0000 7f69623ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c155670 con 0x7f695c0c7590 2023-11-06T22:00:00.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.591+0000 7f69623ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c13abf0 con 0x7f695c156800 2023-11-06T22:00:00.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f695b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c155f30 0x7f695c0c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:00.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f695b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c155f30 0x7f695c0c3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46174/0 (socket says 172.21.15.33:46174) 2023-11-06T22:00:00.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f695b7fe700 1 -- 172.21.15.33:0/1905900161 learned_addr learned my addr 172.21.15.33:0/1905900161 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:00.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f695bfff700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c0c7590 0x7f695c155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:00.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f696099c700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f695c156800 0x7f695c151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:00.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f695b7fe700 1 -- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f695c156800 msgr2=0x7f695c151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f695b7fe700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f695c156800 0x7f695c151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f695b7fe700 1 -- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c0c7590 msgr2=0x7f695c155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f695b7fe700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c0c7590 0x7f695c155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f695b7fe700 1 -- 172.21.15.33:0/1905900161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69500092a0 con 0x7f695c155f30 2023-11-06T22:00:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f695b7fe700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c155f30 0x7f695c0c3df0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f695000ea50 tx=0x7f6950009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f6943fff700 1 -- 172.21.15.33:0/1905900161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69500095a0 con 0x7f695c155f30 2023-11-06T22:00:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f6943fff700 1 -- 172.21.15.33:0/1905900161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695000c640 con 0x7f695c155f30 2023-11-06T22:00:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.592+0000 7f6943fff700 1 -- 172.21.15.33:0/1905900161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695000c9c0 con 0x7f695c155f30 2023-11-06T22:00:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.593+0000 7f69623ff700 1 -- 172.21.15.33:0/1905900161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6954009ae0 con 0x7f695c155f30 2023-11-06T22:00:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.593+0000 7f69623ff700 1 -- 172.21.15.33:0/1905900161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c13b1b0 con 0x7f695c155f30 2023-11-06T22:00:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.594+0000 7f69623ff700 1 -- 172.21.15.33:0/1905900161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695c0bc030 con 0x7f695c155f30 2023-11-06T22:00:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.594+0000 7f6943fff700 1 -- 172.21.15.33:0/1905900161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f695001b400 con 0x7f695c155f30 2023-11-06T22:00:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.595+0000 7f6943fff700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f694805f300 0x7f69480617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.595+0000 7f6943fff700 1 -- 172.21.15.33:0/1905900161 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f695007ab30 con 0x7f695c155f30 2023-11-06T22:00:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.595+0000 7f695bfff700 1 -- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f694805f300 msgr2=0x7f69480617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.595+0000 7f695bfff700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f694805f300 0x7f69480617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:00:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.599+0000 7f6943fff700 1 -- 172.21.15.33:0/1905900161 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f695004ba20 con 0x7f695c155f30 2023-11-06T22:00:00.608 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:00.607+0000 7f62dc175680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:00:00.609 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:00.607+0000 7fa04a9ae680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:00:00.741 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:00.740+0000 7f62dc175680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:00:00.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.779+0000 7f69623ff700 1 -- 172.21.15.33:0/1905900161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f695c064020 con 0x7f695c155f30 2023-11-06T22:00:00.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.780+0000 7f6943fff700 1 -- 172.21.15.33:0/1905900161 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v171) v1 ==== 104+0+11 (secure 0 0 0) 0x7f695000cdb0 con 0x7f695c155f30 2023-11-06T22:00:00.781 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T22:00:00.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 -- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f694805f300 msgr2=0x7f69480617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:00:00.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f694805f300 0x7f69480617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:00.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 -- 172.21.15.33:0/1905900161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c155f30 msgr2=0x7f695c0c3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:00.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c155f30 0x7f695c0c3df0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f695000ea50 tx=0x7f6950009060).stop 2023-11-06T22:00:00.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 -- 172.21.15.33:0/1905900161 shutdown_connections 2023-11-06T22:00:00.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f694805f300 0x7f69480617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:00.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f695c156800 0x7f695c151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f695c0c7590 0x7f695c155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 --2- 172.21.15.33:0/1905900161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f695c155f30 0x7f695c0c3df0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:00.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 -- 172.21.15.33:0/1905900161 >> 172.21.15.33:0/1905900161 conn(0x7f695c0631e0 msgr2=0x7f695c0c6400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:00.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 -- 172.21.15.33:0/1905900161 shutdown_connections 2023-11-06T22:00:00.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:00.782+0000 7f69623ff700 1 -- 172.21.15.33:0/1905900161 wait complete. 2023-11-06T22:00:00.794 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:00.793+0000 7fa04a9ae680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:00:00.802 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T22:00:00.890 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:00.888+0000 7fa04a9ae680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:00:00.937 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:00.936+0000 7f62dc175680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:00:00.975 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:00.973+0000 7fa04a9ae680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:00:01.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.006+0000 7f28a07ff700 1 Processor -- start 2023-11-06T22:00:01.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.006+0000 7f28a07ff700 1 -- start start 2023-11-06T22:00:01.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.007+0000 7f28a07ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898153f50 0x7f2898154330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.007+0000 7f28a07ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f28980a53c0 0x7f28980a5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.007+0000 7f28a07ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28980bc010 con 0x7f2898064270 2023-11-06T22:00:01.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.007+0000 7f28a07ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2898064e30 con 0x7f28980a53c0 2023-11-06T22:00:01.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.007+0000 7f28a07ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2898065110 con 0x7f2898153f50 2023-11-06T22:00:01.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.007+0000 7f289ed9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2898064270 0x7f28980c3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45350/0 (socket says 172.21.15.33:45350) 2023-11-06T22:00:01.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.007+0000 7f289ed9c700 1 -- 172.21.15.33:0/3914135666 learned_addr learned my addr 172.21.15.33:0/3914135666 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:01.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.007+0000 7f289dd9a700 1 --2- 172.21.15.33:0/3914135666 >> v2:172.21.15.69:3300/0 conn(0x7f28980a53c0 0x7f28980a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.007+0000 7f289e59b700 1 --2- 172.21.15.33:0/3914135666 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898153f50 0x7f2898154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.008+0000 7f289d599700 1 -- 172.21.15.33:0/3914135666 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583551317 0 0) 0x7f28980bc010 con 0x7f2898064270 2023-11-06T22:00:01.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.008+0000 7f289d599700 1 -- 172.21.15.33:0/3914135666 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28840033f0 con 0x7f2898064270 2023-11-06T22:00:01.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.008+0000 7f289dd9a700 1 -- 172.21.15.33:0/3914135666 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898153f50 msgr2=0x7f2898154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:01.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.008+0000 7f289dd9a700 1 --2- 172.21.15.33:0/3914135666 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898153f50 0x7f2898154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.008+0000 7f289dd9a700 1 -- 172.21.15.33:0/3914135666 >> v1:172.21.15.33:6789/0 conn(0x7f2898064270 legacy=0x7f28980c3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.008+0000 7f289dd9a700 1 -- 172.21.15.33:0/3914135666 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28980a48e0 con 0x7f28980a53c0 2023-11-06T22:00:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.008+0000 7f289e59b700 1 --2- 172.21.15.33:0/3914135666 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898153f50 0x7f2898154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:00:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.008+0000 7f289dd9a700 1 --2- 172.21.15.33:0/3914135666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28980a53c0 0x7f28980a5820 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f2888009760 tx=0x7f2888009060).ready entity=mon.1 client_cookie=167c09dbf8e6c64f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.008+0000 7f289d599700 1 -- 172.21.15.33:0/3914135666 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288800d4a0 con 0x7f28980a53c0 2023-11-06T22:00:01.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.009+0000 7f289d599700 1 -- 172.21.15.33:0/3914135666 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288800d600 con 0x7f28980a53c0 2023-11-06T22:00:01.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.009+0000 7f289d599700 1 -- 172.21.15.33:0/3914135666 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288800d980 con 0x7f28980a53c0 2023-11-06T22:00:01.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.009+0000 7f28a07ff700 1 -- 172.21.15.33:0/3914135666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28980a53c0 msgr2=0x7f28980a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.009+0000 7f28a07ff700 1 --2- 172.21.15.33:0/3914135666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28980a53c0 0x7f28980a5820 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f2888009760 tx=0x7f2888009060).stop 2023-11-06T22:00:01.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.009+0000 7f28a07ff700 1 -- 172.21.15.33:0/3914135666 shutdown_connections 2023-11-06T22:00:01.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.009+0000 7f28a07ff700 1 --2- 172.21.15.33:0/3914135666 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898153f50 0x7f2898154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.009+0000 7f28a07ff700 1 --2- 172.21.15.33:0/3914135666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28980a53c0 0x7f28980a5820 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.009+0000 7f28a07ff700 1 -- 172.21.15.33:0/3914135666 >> 172.21.15.33:0/3914135666 conn(0x7f2898062710 msgr2=0x7f2898062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:01.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.009+0000 7f28a07ff700 1 -- 172.21.15.33:0/3914135666 shutdown_connections 2023-11-06T22:00:01.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.010+0000 7f28a07ff700 1 -- 172.21.15.33:0/3914135666 wait complete. 2023-11-06T22:00:01.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.012+0000 7f28a07ff700 1 Processor -- start 2023-11-06T22:00:01.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.012+0000 7f28a07ff700 1 -- start start 2023-11-06T22:00:01.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.013+0000 7f28a07ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898064270 0x7f2898154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.013+0000 7f28a07ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28980a53c0 0x7f28980bf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.013+0000 7f28a07ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2898153f50 0x7f2898150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.013+0000 7f28a07ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2898154820 con 0x7f2898153f50 2023-11-06T22:00:01.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.013+0000 7f28a07ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980a6bf0 con 0x7f28980a53c0 2023-11-06T22:00:01.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.013+0000 7f28a07ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28980bfc50 con 0x7f2898064270 2023-11-06T22:00:01.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.013+0000 7f289ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2898153f50 0x7f2898150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.013+0000 7f289dd9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28980a53c0 0x7f28980bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.013+0000 7f289e59b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898064270 0x7f2898154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f289dd9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28980a53c0 0x7f28980bf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:35986/0 (socket says 172.21.15.33:35986) 2023-11-06T22:00:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f289ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2898153f50 0x7f2898150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46180/0 (socket says 172.21.15.33:46180) 2023-11-06T22:00:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f289e59b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898064270 0x7f2898154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:60974/0 (socket says 172.21.15.33:60974) 2023-11-06T22:00:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f289ed9c700 1 -- 172.21.15.33:0/2148728497 learned_addr learned my addr 172.21.15.33:0/2148728497 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f289ed9c700 1 -- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898064270 msgr2=0x7f2898154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f289ed9c700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898064270 0x7f2898154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f289ed9c700 1 -- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28980a53c0 msgr2=0x7f28980bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f289ed9c700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28980a53c0 0x7f28980bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f289ed9c700 1 -- 172.21.15.33:0/2148728497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940092a0 con 0x7f2898153f50 2023-11-06T22:00:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f289ed9c700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2898153f50 0x7f2898150740 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f2890005ad0 tx=0x7f2890001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f288e7fc700 1 -- 172.21.15.33:0/2148728497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890005d30 con 0x7f2898153f50 2023-11-06T22:00:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f288e7fc700 1 -- 172.21.15.33:0/2148728497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2890005e90 con 0x7f2898153f50 2023-11-06T22:00:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f288e7fc700 1 -- 172.21.15.33:0/2148728497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890007320 con 0x7f2898153f50 2023-11-06T22:00:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.014+0000 7f28a07ff700 1 -- 172.21.15.33:0/2148728497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28880092a0 con 0x7f2898153f50 2023-11-06T22:00:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.015+0000 7f28a07ff700 1 -- 172.21.15.33:0/2148728497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2898154cf0 con 0x7f2898153f50 2023-11-06T22:00:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.016+0000 7f28a07ff700 1 -- 172.21.15.33:0/2148728497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28980a69c0 con 0x7f2898153f50 2023-11-06T22:00:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.016+0000 7f288e7fc700 1 -- 172.21.15.33:0/2148728497 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f2890007710 con 0x7f2898153f50 2023-11-06T22:00:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.017+0000 7f288e7fc700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f287c05f300 0x7f287c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.017+0000 7f288e7fc700 1 -- 172.21.15.33:0/2148728497 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f289007a590 con 0x7f2898153f50 2023-11-06T22:00:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.017+0000 7f289e59b700 1 -- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f287c05f300 msgr2=0x7f287c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:01.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.017+0000 7f289e59b700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f287c05f300 0x7f287c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:00:01.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.021+0000 7f288e7fc700 1 -- 172.21.15.33:0/2148728497 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f28980a69c0 con 0x7f2898153f50 2023-11-06T22:00:01.055 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:01.054+0000 7f62dc175680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:00:01.179 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:01.178+0000 7f62dc175680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:00:01.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.218+0000 7f289e59b700 1 -- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f287c05f300 msgr2=0x7f287c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:01.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.218+0000 7f289e59b700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f287c05f300 0x7f287c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:00:01.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.266+0000 7f28a07ff700 1 -- 172.21.15.33:0/2148728497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28980a69c0 con 0x7f2898153f50 2023-11-06T22:00:01.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.266+0000 7f288e7fc700 1 -- 172.21.15.33:0/2148728497 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v171) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28980a69c0 con 0x7f2898153f50 2023-11-06T22:00:01.268 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T22:00:01.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 -- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f287c05f300 msgr2=0x7f287c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:00:01.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f287c05f300 0x7f287c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:01.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 -- 172.21.15.33:0/2148728497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2898153f50 msgr2=0x7f2898150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2898153f50 0x7f2898150740 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f2890005ad0 tx=0x7f2890001650).stop 2023-11-06T22:00:01.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 -- 172.21.15.33:0/2148728497 shutdown_connections 2023-11-06T22:00:01.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f287c05f300 0x7f287c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:01.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2898064270 0x7f2898154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2898153f50 0x7f2898150740 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 --2- 172.21.15.33:0/2148728497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28980a53c0 0x7f28980bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 -- 172.21.15.33:0/2148728497 >> 172.21.15.33:0/2148728497 conn(0x7f2898062710 msgr2=0x7f289806a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 -- 172.21.15.33:0/2148728497 shutdown_connections 2023-11-06T22:00:01.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.269+0000 7f28a07ff700 1 -- 172.21.15.33:0/2148728497 wait complete. 2023-11-06T22:00:01.289 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:01.287+0000 7fa04a9ae680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:00:01.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:00:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.492+0000 7f30fe7ff700 1 Processor -- start 2023-11-06T22:00:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.492+0000 7f30fe7ff700 1 -- start start 2023-11-06T22:00:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.493+0000 7f30fe7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f30f81521a0 0x7f30f80c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30fe7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8155320 0x7f30f80c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30fe7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f8154e90 con 0x7f30f8151dc0 2023-11-06T22:00:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30fe7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f80639d0 con 0x7f30f81521a0 2023-11-06T22:00:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30fe7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8063cb0 con 0x7f30f8155320 2023-11-06T22:00:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30f77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f30f81521a0 0x7f30f80c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30f77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f30f81521a0 0x7f30f80c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46192/0 (socket says 172.21.15.33:46192) 2023-11-06T22:00:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30f7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f30f8151dc0 0x7f30f8063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45352/0 (socket says 172.21.15.33:45352) 2023-11-06T22:00:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30f77fe700 1 -- 172.21.15.33:0/2751719559 learned_addr learned my addr 172.21.15.33:0/2751719559 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30fcd9c700 1 --2- 172.21.15.33:0/2751719559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8155320 0x7f30f80c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30f77fe700 1 -- 172.21.15.33:0/2751719559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8155320 msgr2=0x7f30f80c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30f77fe700 1 --2- 172.21.15.33:0/2751719559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8155320 0x7f30f80c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30f77fe700 1 -- 172.21.15.33:0/2751719559 >> v1:172.21.15.33:6789/0 conn(0x7f30f8151dc0 legacy=0x7f30f8063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.494+0000 7f30f77fe700 1 -- 172.21.15.33:0/2751719559 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8063f90 con 0x7f30f81521a0 2023-11-06T22:00:01.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.495+0000 7f30f77fe700 1 --2- 172.21.15.33:0/2751719559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f81521a0 0x7f30f80c0c50 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f30ec006fd0 tx=0x7f30ec0090f0).ready entity=mon.0 client_cookie=f49f4735b77f04ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:01.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.495+0000 7f30f6ffd700 1 -- 172.21.15.33:0/2751719559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00b580 con 0x7f30f81521a0 2023-11-06T22:00:01.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.495+0000 7f30f6ffd700 1 -- 172.21.15.33:0/2751719559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec00b6e0 con 0x7f30f81521a0 2023-11-06T22:00:01.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.495+0000 7f30f6ffd700 1 -- 172.21.15.33:0/2751719559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00ba60 con 0x7f30f81521a0 2023-11-06T22:00:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.495+0000 7f30fe7ff700 1 -- 172.21.15.33:0/2751719559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f81521a0 msgr2=0x7f30f80c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.495+0000 7f30fe7ff700 1 --2- 172.21.15.33:0/2751719559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f81521a0 0x7f30f80c0c50 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f30ec006fd0 tx=0x7f30ec0090f0).stop 2023-11-06T22:00:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.496+0000 7f30fe7ff700 1 -- 172.21.15.33:0/2751719559 shutdown_connections 2023-11-06T22:00:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.496+0000 7f30fe7ff700 1 --2- 172.21.15.33:0/2751719559 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8155320 0x7f30f80c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.496+0000 7f30fe7ff700 1 --2- 172.21.15.33:0/2751719559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f81521a0 0x7f30f80c0c50 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.496+0000 7f30fe7ff700 1 -- 172.21.15.33:0/2751719559 >> 172.21.15.33:0/2751719559 conn(0x7f30f805e1b0 msgr2=0x7f30f805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.496+0000 7f30fe7ff700 1 -- 172.21.15.33:0/2751719559 shutdown_connections 2023-11-06T22:00:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.496+0000 7f30fe7ff700 1 -- 172.21.15.33:0/2751719559 wait complete. 2023-11-06T22:00:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.497+0000 7f30fe7ff700 1 Processor -- start 2023-11-06T22:00:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.497+0000 7f30fe7ff700 1 -- start start 2023-11-06T22:00:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.498+0000 7f30fe7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8151dc0 0x7f30f80c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.498+0000 7f30fe7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30f81521a0 0x7f30f8156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.498+0000 7f30fe7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f8155320 0x7f30f8143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.498+0000 7f30fe7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f81689e0 con 0x7f30f8155320 2023-11-06T22:00:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.498+0000 7f30fe7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80c4c80 con 0x7f30f81521a0 2023-11-06T22:00:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.498+0000 7f30fe7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8167dd0 con 0x7f30f8151dc0 2023-11-06T22:00:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30fcd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f8155320 0x7f30f8143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30f7fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8151dc0 0x7f30f80c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30fcd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f8155320 0x7f30f8143660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46208/0 (socket says 172.21.15.33:46208) 2023-11-06T22:00:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30f77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30f81521a0 0x7f30f8156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30fcd9c700 1 -- 172.21.15.33:0/4131172639 learned_addr learned my addr 172.21.15.33:0/4131172639 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30f7fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8151dc0 0x7f30f80c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:32774/0 (socket says 172.21.15.33:32774) 2023-11-06T22:00:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30f7fff700 1 -- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30f81521a0 msgr2=0x7f30f8156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30f7fff700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30f81521a0 0x7f30f8156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30f7fff700 1 -- 172.21.15.33:0/4131172639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f8155320 msgr2=0x7f30f8143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30f7fff700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f8155320 0x7f30f8143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30f7fff700 1 -- 172.21.15.33:0/4131172639 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec009370 con 0x7f30f8151dc0 2023-11-06T22:00:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30fcd9c700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f8155320 0x7f30f8143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:00:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.499+0000 7f30f7fff700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8151dc0 0x7f30f80c4790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f30e800c440 tx=0x7f30e8007d50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.500+0000 7f30dffff700 1 -- 172.21.15.33:0/4131172639 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e80070d0 con 0x7f30f8151dc0 2023-11-06T22:00:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.500+0000 7f30dffff700 1 -- 172.21.15.33:0/4131172639 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e8007230 con 0x7f30f8151dc0 2023-11-06T22:00:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.500+0000 7f30fe7ff700 1 -- 172.21.15.33:0/4131172639 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f81680b0 con 0x7f30f8151dc0 2023-11-06T22:00:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.500+0000 7f30dffff700 1 -- 172.21.15.33:0/4131172639 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e80056e0 con 0x7f30f8151dc0 2023-11-06T22:00:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.500+0000 7f30fe7ff700 1 -- 172.21.15.33:0/4131172639 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f8168670 con 0x7f30f8151dc0 2023-11-06T22:00:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.501+0000 7f30fe7ff700 1 -- 172.21.15.33:0/4131172639 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f8168cc0 con 0x7f30f8151dc0 2023-11-06T22:00:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.502+0000 7f30dffff700 1 -- 172.21.15.33:0/4131172639 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f30e8005a50 con 0x7f30f8151dc0 2023-11-06T22:00:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.503+0000 7f30dffff700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f30e405f300 0x7f30e40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.503+0000 7f30dffff700 1 -- 172.21.15.33:0/4131172639 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f30e8079af0 con 0x7f30f8151dc0 2023-11-06T22:00:01.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.503+0000 7f30f77fe700 1 -- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f30e405f300 msgr2=0x7f30e40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:01.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.503+0000 7f30f77fe700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f30e405f300 0x7f30e40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:00:01.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.511+0000 7f30dffff700 1 -- 172.21.15.33:0/4131172639 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f30e804a990 con 0x7f30f8151dc0 2023-11-06T22:00:01.517 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:01.515+0000 7fa04a9ae680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:00:01.523 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:01.522+0000 7f62dc175680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:00:01.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.704+0000 7f30f77fe700 1 -- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f30e405f300 msgr2=0x7f30e40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:01.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.704+0000 7f30f77fe700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f30e405f300 0x7f30e40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:00:01.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.736+0000 7f30fe7ff700 1 -- 172.21.15.33:0/4131172639 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30f80c0a30 con 0x7f30f8151dc0 2023-11-06T22:00:01.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.737+0000 7f30dffff700 1 -- 172.21.15.33:0/4131172639 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f30e804dfb0 con 0x7f30f8151dc0 2023-11-06T22:00:01.738 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:01.739 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3645039939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3645039939},{"type":"v1","addr":"172.21.15.33:6835","nonce":3645039939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1156785194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1156785194},{"type":"v1","addr":"172.21.15.69:6836","nonce":1156785194}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/4183165491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":4183165491},{"type":"v1","addr":"172.21.15.69:6837","nonce":4183165491}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:59:40.865889+0000","modified":"2023-11-06T21:59:43.953031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8445":{"gid":8445,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/1588551062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1588551062},{"type":"v1","addr":"172.21.15.33:6833","nonce":1588551062}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-06T22:00:01.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 -- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f30e405f300 msgr2=0x7f30e40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:00:01.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f30e405f300 0x7f30e40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:01.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 -- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8151dc0 msgr2=0x7f30f80c4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8151dc0 0x7f30f80c4790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f30e800c440 tx=0x7f30e8007d50).stop 2023-11-06T22:00:01.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 -- 172.21.15.33:0/4131172639 shutdown_connections 2023-11-06T22:00:01.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f30e405f300 0x7f30e40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:01.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f30f8151dc0 0x7f30f80c4790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f8155320 0x7f30f8143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 --2- 172.21.15.33:0/4131172639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30f81521a0 0x7f30f8156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 -- 172.21.15.33:0/4131172639 >> 172.21.15.33:0/4131172639 conn(0x7f30f805e1b0 msgr2=0x7f30f80aa120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:01.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 -- 172.21.15.33:0/4131172639 shutdown_connections 2023-11-06T22:00:01.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.739+0000 7f30fe7ff700 1 -- 172.21.15.33:0/4131172639 wait complete. 2023-11-06T22:00:01.743 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 68 2023-11-06T22:00:01.757 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:01.756+0000 7f62dc175680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:00:01.758 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:00:01.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.955+0000 7fe4627ff700 1 Processor -- start 2023-11-06T22:00:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.956+0000 7fe4627ff700 1 -- start start 2023-11-06T22:00:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.957+0000 7fe4627ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0c5d50 0x7fe45c0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.957+0000 7fe4627ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe45c0bfd70 0x7fe45c0c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.957+0000 7fe4627ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45c1554a0 con 0x7fe45c0c1840 2023-11-06T22:00:01.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.957+0000 7fe4627ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45c154960 con 0x7fe45c0bfd70 2023-11-06T22:00:01.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.957+0000 7fe4627ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0c66c0 con 0x7fe45c0c5d50 2023-11-06T22:00:01.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.958+0000 7fe460d9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe45c0bfd70 0x7fe45c0c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.958+0000 7fe45b7fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0c5d50 0x7fe45c0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.958+0000 7fe45bfff700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fe45c0c1840 0x7fe45c0a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:55036/0 (socket says 172.21.15.33:55036) 2023-11-06T22:00:01.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.958+0000 7fe45bfff700 1 -- 172.21.15.33:0/1743388605 learned_addr learned my addr 172.21.15.33:0/1743388605 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:01.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.958+0000 7fe460d9c700 1 -- 172.21.15.33:0/1743388605 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0c5d50 msgr2=0x7fe45c0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:01.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.958+0000 7fe460d9c700 1 --2- 172.21.15.33:0/1743388605 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0c5d50 0x7fe45c0c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.958+0000 7fe460d9c700 1 -- 172.21.15.33:0/1743388605 >> v1:172.21.15.69:6789/0 conn(0x7fe45c0c1840 legacy=0x7fe45c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.958+0000 7fe460d9c700 1 -- 172.21.15.33:0/1743388605 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c155930 con 0x7fe45c0bfd70 2023-11-06T22:00:01.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.958+0000 7fe460d9c700 1 --2- 172.21.15.33:0/1743388605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe45c0bfd70 0x7fe45c0c01e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fe454009e60 tx=0x7fe454009780).ready entity=mon.1 client_cookie=1a26668935925cb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:01.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe45affd700 1 -- 172.21.15.33:0/1743388605 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45400e070 con 0x7fe45c0bfd70 2023-11-06T22:00:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe45affd700 1 -- 172.21.15.33:0/1743388605 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45400ba60 con 0x7fe45c0bfd70 2023-11-06T22:00:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe45affd700 1 -- 172.21.15.33:0/1743388605 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45400bde0 con 0x7fe45c0bfd70 2023-11-06T22:00:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe4627ff700 1 -- 172.21.15.33:0/1743388605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe45c0bfd70 msgr2=0x7fe45c0c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe4627ff700 1 --2- 172.21.15.33:0/1743388605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe45c0bfd70 0x7fe45c0c01e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fe454009e60 tx=0x7fe454009780).stop 2023-11-06T22:00:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe4627ff700 1 -- 172.21.15.33:0/1743388605 shutdown_connections 2023-11-06T22:00:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe4627ff700 1 --2- 172.21.15.33:0/1743388605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe45c0bfd70 0x7fe45c0c01e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe4627ff700 1 --2- 172.21.15.33:0/1743388605 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0c5d50 0x7fe45c0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe4627ff700 1 -- 172.21.15.33:0/1743388605 >> 172.21.15.33:0/1743388605 conn(0x7fe45c05ec80 msgr2=0x7fe45c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe4627ff700 1 -- 172.21.15.33:0/1743388605 shutdown_connections 2023-11-06T22:00:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.959+0000 7fe4627ff700 1 -- 172.21.15.33:0/1743388605 wait complete. 2023-11-06T22:00:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.961+0000 7fe4627ff700 1 Processor -- start 2023-11-06T22:00:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.961+0000 7fe4627ff700 1 -- start start 2023-11-06T22:00:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe4627ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0bfd70 0x7fe45c0ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe4627ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c0c1840 0x7fe45c153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe4627ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe45c0c5d50 0x7fe45c14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe4627ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0addc0 con 0x7fe45c0c1840 2023-11-06T22:00:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe4627ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0c4270 con 0x7fe45c0c5d50 2023-11-06T22:00:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe4627ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0c4550 con 0x7fe45c0bfd70 2023-11-06T22:00:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe45b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c0c1840 0x7fe45c153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe45bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0bfd70 0x7fe45c0ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe45b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c0c1840 0x7fe45c153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46214/0 (socket says 172.21.15.33:46214) 2023-11-06T22:00:01.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe45b7fe700 1 -- 172.21.15.33:0/698702179 learned_addr learned my addr 172.21.15.33:0/698702179 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:01.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe45bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0bfd70 0x7fe45c0ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:32784/0 (socket says 172.21.15.33:32784) 2023-11-06T22:00:01.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.962+0000 7fe460d9c700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe45c0c5d50 0x7fe45c14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:01.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe45b7fe700 1 -- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0bfd70 msgr2=0x7fe45c0ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe45b7fe700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0bfd70 0x7fe45c0ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe45b7fe700 1 -- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe45c0c5d50 msgr2=0x7fe45c14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:01.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe45b7fe700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe45c0c5d50 0x7fe45c14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:01.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe45b7fe700 1 -- 172.21.15.33:0/698702179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe454009a20 con 0x7fe45c0c1840 2023-11-06T22:00:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe45b7fe700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c0c1840 0x7fe45c153fd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7fe45000e900 tx=0x7fe450009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe443fff700 1 -- 172.21.15.33:0/698702179 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450012600 con 0x7fe45c0c1840 2023-11-06T22:00:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe443fff700 1 -- 172.21.15.33:0/698702179 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe450012760 con 0x7fe45c0c1840 2023-11-06T22:00:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe443fff700 1 -- 172.21.15.33:0/698702179 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450012ae0 con 0x7fe45c0c1840 2023-11-06T22:00:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe4627ff700 1 -- 172.21.15.33:0/698702179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45c0c4830 con 0x7fe45c0c1840 2023-11-06T22:00:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.963+0000 7fe4627ff700 1 -- 172.21.15.33:0/698702179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45c0ad500 con 0x7fe45c0c1840 2023-11-06T22:00:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.965+0000 7fe4627ff700 1 -- 172.21.15.33:0/698702179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45c0adf70 con 0x7fe45c0c1840 2023-11-06T22:00:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.966+0000 7fe443fff700 1 -- 172.21.15.33:0/698702179 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fe45001b400 con 0x7fe45c0c1840 2023-11-06T22:00:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.967+0000 7fe443fff700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe44805f300 0x7fe4480617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.968+0000 7fe443fff700 1 -- 172.21.15.33:0/698702179 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7fe45007a920 con 0x7fe45c0c1840 2023-11-06T22:00:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.968+0000 7fe45bfff700 1 -- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe44805f300 msgr2=0x7fe4480617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.968+0000 7fe45bfff700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe44805f300 0x7fe4480617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:00:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:01.972+0000 7fe443fff700 1 -- 172.21.15.33:0/698702179 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe45c0adf70 con 0x7fe45c0c1840 2023-11-06T22:00:02.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.158+0000 7fe4627ff700 1 -- 172.21.15.33:0/698702179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe45c06c6c0 con 0x7fe45c0c1840 2023-11-06T22:00:02.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.159+0000 7fe443fff700 1 -- 172.21.15.33:0/698702179 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+17427 (secure 0 0 0) 0x7fe45004b760 con 0x7fe45c0c1840 2023-11-06T22:00:02.160 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:02.161 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":171,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T21:59:55.184836+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:59:38.416213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T21:59:39.859883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6833/114912":"2023-11-07T21:59:55.184802+0000","172.21.15.69:6832/114912":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/351947370":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/623316503":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/861101334":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/1093219585":"2023-11-07T21:59:55.184802+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:00:02.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 -- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe44805f300 msgr2=0x7fe4480617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:00:02.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe44805f300 0x7fe4480617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:02.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 -- 172.21.15.33:0/698702179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c0c1840 msgr2=0x7fe45c153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:02.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c0c1840 0x7fe45c153fd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7fe45000e900 tx=0x7fe450009060).stop 2023-11-06T22:00:02.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 -- 172.21.15.33:0/698702179 shutdown_connections 2023-11-06T22:00:02.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe44805f300 0x7fe4480617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:02.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe45c0bfd70 0x7fe45c0ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe45c0c5d50 0x7fe45c14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 --2- 172.21.15.33:0/698702179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c0c1840 0x7fe45c153fd0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 -- 172.21.15.33:0/698702179 >> 172.21.15.33:0/698702179 conn(0x7fe45c05ec80 msgr2=0x7fe45c151620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:02.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 -- 172.21.15.33:0/698702179 shutdown_connections 2023-11-06T22:00:02.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.161+0000 7fe4627ff700 1 -- 172.21.15.33:0/698702179 wait complete. 2023-11-06T22:00:02.180 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T22:00:02.181 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:00:02.325 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:02.323+0000 7fa04a9ae680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:00:02.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.385+0000 7fc0a8dff700 1 Processor -- start 2023-11-06T22:00:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.386+0000 7fc0a8dff700 1 -- start start 2023-11-06T22:00:02.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.387+0000 7fc0a8dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0a405e1b0 0x7fc0a4063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.387+0000 7fc0a8dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a40c3680 0x7fc0a4154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.387+0000 7fc0a8dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a40c31f0 con 0x7fc0a4063140 2023-11-06T22:00:02.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.387+0000 7fc0a8dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a406a0f0 con 0x7fc0a405e1b0 2023-11-06T22:00:02.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.387+0000 7fc0a8dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40c2920 con 0x7fc0a40c3680 2023-11-06T22:00:02.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.387+0000 7fc0a1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0a405e1b0 0x7fc0a4063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:02.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.387+0000 7fc0a259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc0a4063140 0x7fc0a40c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45360/0 (socket says 172.21.15.33:45360) 2023-11-06T22:00:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.387+0000 7fc0a1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0a405e1b0 0x7fc0a4063520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46216/0 (socket says 172.21.15.33:46216) 2023-11-06T22:00:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.387+0000 7fc0a259c700 1 -- 172.21.15.33:0/2635970774 learned_addr learned my addr 172.21.15.33:0/2635970774 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.387+0000 7fc0a2d9d700 1 --2- 172.21.15.33:0/2635970774 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a40c3680 0x7fc0a4154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:02.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.388+0000 7fc0a1d9b700 1 -- 172.21.15.33:0/2635970774 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a40c3680 msgr2=0x7fc0a4154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:02.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.388+0000 7fc0a1d9b700 1 --2- 172.21.15.33:0/2635970774 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a40c3680 0x7fc0a4154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.388+0000 7fc0a1d9b700 1 -- 172.21.15.33:0/2635970774 >> v1:172.21.15.33:6789/0 conn(0x7fc0a4063140 legacy=0x7fc0a40c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:02.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.388+0000 7fc0a1d9b700 1 -- 172.21.15.33:0/2635970774 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a405e640 con 0x7fc0a405e1b0 2023-11-06T22:00:02.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.388+0000 7fc0a159a700 1 -- 172.21.15.33:0/2635970774 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880708855 0 0) 0x7fc0a40c31f0 con 0x7fc0a4063140 2023-11-06T22:00:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.388+0000 7fc0a1d9b700 1 --2- 172.21.15.33:0/2635970774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a405e1b0 0x7fc0a4063520 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fc098009760 tx=0x7fc098009060).ready entity=mon.0 client_cookie=b494fc00f00821b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.388+0000 7fc0a159a700 1 -- 172.21.15.33:0/2635970774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800f510 con 0x7fc0a405e1b0 2023-11-06T22:00:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.388+0000 7fc0a159a700 1 -- 172.21.15.33:0/2635970774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09800f670 con 0x7fc0a405e1b0 2023-11-06T22:00:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.388+0000 7fc0a159a700 1 -- 172.21.15.33:0/2635970774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800f9f0 con 0x7fc0a405e1b0 2023-11-06T22:00:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.389+0000 7fc0a8dff700 1 -- 172.21.15.33:0/2635970774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a405e1b0 msgr2=0x7fc0a4063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.389+0000 7fc0a8dff700 1 --2- 172.21.15.33:0/2635970774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a405e1b0 0x7fc0a4063520 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fc098009760 tx=0x7fc098009060).stop 2023-11-06T22:00:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.389+0000 7fc0a8dff700 1 -- 172.21.15.33:0/2635970774 shutdown_connections 2023-11-06T22:00:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.389+0000 7fc0a8dff700 1 --2- 172.21.15.33:0/2635970774 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a40c3680 0x7fc0a4154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.389+0000 7fc0a8dff700 1 --2- 172.21.15.33:0/2635970774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a405e1b0 0x7fc0a4063520 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.389+0000 7fc0a8dff700 1 -- 172.21.15.33:0/2635970774 >> 172.21.15.33:0/2635970774 conn(0x7fc0a40c7d60 msgr2=0x7fc0a40c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.389+0000 7fc0a8dff700 1 -- 172.21.15.33:0/2635970774 shutdown_connections 2023-11-06T22:00:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.389+0000 7fc0a8dff700 1 -- 172.21.15.33:0/2635970774 wait complete. 2023-11-06T22:00:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.391+0000 7fc0a8dff700 1 Processor -- start 2023-11-06T22:00:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.391+0000 7fc0a8dff700 1 -- start start 2023-11-06T22:00:02.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a8dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a4063140 0x7fc0a415aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a8dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a40c3680 0x7fc0a414d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a8dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a40ac620 0x7fc0a40aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a8dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a415af30 con 0x7fc0a40c3680 2023-11-06T22:00:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a8dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40ab580 con 0x7fc0a40ac620 2023-11-06T22:00:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a8dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40ab860 con 0x7fc0a4063140 2023-11-06T22:00:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a40c3680 0x7fc0a414d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a259c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a4063140 0x7fc0a415aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a40c3680 0x7fc0a414d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46228/0 (socket says 172.21.15.33:46228) 2023-11-06T22:00:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a1d9b700 1 -- 172.21.15.33:0/872923349 learned_addr learned my addr 172.21.15.33:0/872923349 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a2d9d700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a40ac620 0x7fc0a40aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a259c700 1 -- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a40ac620 msgr2=0x7fc0a40aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a259c700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a40ac620 0x7fc0a40aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a259c700 1 -- 172.21.15.33:0/872923349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a40c3680 msgr2=0x7fc0a414d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.392+0000 7fc0a259c700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a40c3680 0x7fc0a414d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.393+0000 7fc0a259c700 1 -- 172.21.15.33:0/872923349 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980092a0 con 0x7fc0a4063140 2023-11-06T22:00:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.393+0000 7fc0a1d9b700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a40c3680 0x7fc0a414d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:00:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.393+0000 7fc0a259c700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a4063140 0x7fc0a415aa40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fc094001eb0 tx=0x7fc094007bb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.393+0000 7fc0927fc700 1 -- 172.21.15.33:0/872923349 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0940070d0 con 0x7fc0a4063140 2023-11-06T22:00:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.393+0000 7fc0927fc700 1 -- 172.21.15.33:0/872923349 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc094007250 con 0x7fc0a4063140 2023-11-06T22:00:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.393+0000 7fc0927fc700 1 -- 172.21.15.33:0/872923349 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094007610 con 0x7fc0a4063140 2023-11-06T22:00:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.393+0000 7fc0a8dff700 1 -- 172.21.15.33:0/872923349 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a40abb40 con 0x7fc0a4063140 2023-11-06T22:00:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.393+0000 7fc0a8dff700 1 -- 172.21.15.33:0/872923349 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a40ac100 con 0x7fc0a4063140 2023-11-06T22:00:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.394+0000 7fc0a8dff700 1 -- 172.21.15.33:0/872923349 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a4064420 con 0x7fc0a4063140 2023-11-06T22:00:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.395+0000 7fc0927fc700 1 -- 172.21.15.33:0/872923349 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fc0940059b0 con 0x7fc0a4063140 2023-11-06T22:00:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.396+0000 7fc0927fc700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc08405f300 0x7fc0840617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.396+0000 7fc0927fc700 1 -- 172.21.15.33:0/872923349 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7fc094079c50 con 0x7fc0a4063140 2023-11-06T22:00:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.396+0000 7fc0a1d9b700 1 -- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc08405f300 msgr2=0x7fc0840617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.396+0000 7fc0a1d9b700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc08405f300 0x7fc0840617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:00:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.401+0000 7fc0927fc700 1 -- 172.21.15.33:0/872923349 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc09404aaf0 con 0x7fc0a4063140 2023-11-06T22:00:02.414 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:02.412+0000 7fa04a9ae680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:00:02.509 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:02.507+0000 7fa04a9ae680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:00:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.597+0000 7fc0a1d9b700 1 -- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc08405f300 msgr2=0x7fc0840617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.597+0000 7fc0a1d9b700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc08405f300 0x7fc0840617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:00:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.628+0000 7fc0a8dff700 1 -- 172.21.15.33:0/872923349 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0a40ab9a0 con 0x7fc0a4063140 2023-11-06T22:00:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.629+0000 7fc0927fc700 1 -- 172.21.15.33:0/872923349 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4491 (secure 0 0 0) 0x7fc0a40ab9a0 con 0x7fc0a4063140 2023-11-06T22:00:02.630 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:02.631 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3645039939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3645039939},{"type":"v1","addr":"172.21.15.33:6835","nonce":3645039939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1156785194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1156785194},{"type":"v1","addr":"172.21.15.69:6836","nonce":1156785194}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/4183165491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":4183165491},{"type":"v1","addr":"172.21.15.69:6837","nonce":4183165491}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:59:40.865889+0000","modified":"2023-11-06T21:59:43.953031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8445":{"gid":8445,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/1588551062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1588551062},{"type":"v1","addr":"172.21.15.33:6833","nonce":1588551062}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-06T22:00:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 -- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc08405f300 msgr2=0x7fc0840617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:00:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc08405f300 0x7fc0840617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 -- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a4063140 msgr2=0x7fc0a415aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a4063140 0x7fc0a415aa40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fc094001eb0 tx=0x7fc094007bb0).stop 2023-11-06T22:00:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 -- 172.21.15.33:0/872923349 shutdown_connections 2023-11-06T22:00:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc08405f300 0x7fc0840617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0a4063140 0x7fc0a415aa40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0a40ac620 0x7fc0a40aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 --2- 172.21.15.33:0/872923349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0a40c3680 0x7fc0a414d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 -- 172.21.15.33:0/872923349 >> 172.21.15.33:0/872923349 conn(0x7fc0a40c7d60 msgr2=0x7fc0a41528e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:02.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 -- 172.21.15.33:0/872923349 shutdown_connections 2023-11-06T22:00:02.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.631+0000 7fc0a8dff700 1 -- 172.21.15.33:0/872923349 wait complete. 2023-11-06T22:00:02.635 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 68 2023-11-06T22:00:02.640 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:02.639+0000 7f62dc175680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:00:02.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:00:02.696 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:02.694+0000 7fa04a9ae680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:00:02.740 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:02.739+0000 7f62dc175680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:00:02.785 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:02.783+0000 7fa04a9ae680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:00:02.840 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:02.839+0000 7f62dc175680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:00:02.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.848+0000 7f9ab9dff700 1 Processor -- start 2023-11-06T22:00:02.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.848+0000 7f9ab9dff700 1 -- start start 2023-11-06T22:00:02.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.849+0000 7f9ab9dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ab40bccc0 0x7f9ab40a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.849+0000 7f9ab9dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ab40a4e60 0x7f9ab40c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.849+0000 7f9ab9dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab40bbfb0 con 0x7f9ab4157720 2023-11-06T22:00:02.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.849+0000 7f9ab9dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab40a52f0 con 0x7f9ab40bccc0 2023-11-06T22:00:02.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.849+0000 7f9ab9dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40bdb90 con 0x7f9ab40a4e60 2023-11-06T22:00:02.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.849+0000 7f9ab2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ab40bccc0 0x7f9ab40a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:02.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.849+0000 7f9ab2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ab40bccc0 0x7f9ab40a4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46238/0 (socket says 172.21.15.33:46238) 2023-11-06T22:00:02.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.849+0000 7f9ab2ffd700 1 -- 172.21.15.33:0/3402454431 learned_addr learned my addr 172.21.15.33:0/3402454431 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:02.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.849+0000 7f9ab37fe700 1 --1- 172.21.15.33:0/3402454431 >> v1:172.21.15.33:6789/0 conn(0x7f9ab4157720 0x7f9ab4158840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45370/0 (socket says 172.21.15.33:45370) 2023-11-06T22:00:02.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.850+0000 7f9ab3fff700 1 --2- 172.21.15.33:0/3402454431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ab40a4e60 0x7f9ab40c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:02.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.850+0000 7f9ab2ffd700 1 -- 172.21.15.33:0/3402454431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ab40a4e60 msgr2=0x7f9ab40c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:02.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.850+0000 7f9ab2ffd700 1 --2- 172.21.15.33:0/3402454431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ab40a4e60 0x7f9ab40c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.850+0000 7f9ab2ffd700 1 -- 172.21.15.33:0/3402454431 >> v1:172.21.15.33:6789/0 conn(0x7f9ab4157720 legacy=0x7f9ab4158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:02.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.850+0000 7f9ab27fc700 1 -- 172.21.15.33:0/3402454431 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153103293 0 0) 0x7f9ab40bbfb0 con 0x7f9ab4157720 2023-11-06T22:00:02.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.850+0000 7f9ab2ffd700 1 -- 172.21.15.33:0/3402454431 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40bc1e0 con 0x7f9ab40bccc0 2023-11-06T22:00:02.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.850+0000 7f9ab2ffd700 1 --2- 172.21.15.33:0/3402454431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40bccc0 0x7f9ab40a4970 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f9aa8006fd0 tx=0x7f9aa80090f0).ready entity=mon.0 client_cookie=3f757f0b77fa6037 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:02.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.850+0000 7f9ab27fc700 1 -- 172.21.15.33:0/3402454431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa800b580 con 0x7f9ab40bccc0 2023-11-06T22:00:02.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.851+0000 7f9ab27fc700 1 -- 172.21.15.33:0/3402454431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa800b6e0 con 0x7f9ab40bccc0 2023-11-06T22:00:02.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.851+0000 7f9ab27fc700 1 -- 172.21.15.33:0/3402454431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa800ba60 con 0x7f9ab40bccc0 2023-11-06T22:00:02.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.851+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3402454431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40bccc0 msgr2=0x7f9ab40a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:02.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.851+0000 7f9ab9dff700 1 --2- 172.21.15.33:0/3402454431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40bccc0 0x7f9ab40a4970 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f9aa8006fd0 tx=0x7f9aa80090f0).stop 2023-11-06T22:00:02.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.851+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3402454431 shutdown_connections 2023-11-06T22:00:02.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.851+0000 7f9ab9dff700 1 --2- 172.21.15.33:0/3402454431 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ab40a4e60 0x7f9ab40c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.851+0000 7f9ab9dff700 1 --2- 172.21.15.33:0/3402454431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40bccc0 0x7f9ab40a4970 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.851+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3402454431 >> 172.21.15.33:0/3402454431 conn(0x7f9ab40631e0 msgr2=0x7f9ab40c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:02.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.851+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3402454431 shutdown_connections 2023-11-06T22:00:02.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.851+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3402454431 wait complete. 2023-11-06T22:00:02.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.853+0000 7f9ab9dff700 1 Processor -- start 2023-11-06T22:00:02.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.853+0000 7f9ab9dff700 1 -- start start 2023-11-06T22:00:02.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab9dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40a4e60 0x7f9ab40c0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab9dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40bccc0 0x7f9ab4155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab9dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ab4157720 0x7f9ab41562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab9dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab4156790 con 0x7f9ab40a4e60 2023-11-06T22:00:02.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab9dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab414f450 con 0x7f9ab40bccc0 2023-11-06T22:00:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab9dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab414fda0 con 0x7f9ab4157720 2023-11-06T22:00:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40a4e60 0x7f9ab40c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40a4e60 0x7f9ab40c0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46250/0 (socket says 172.21.15.33:46250) 2023-11-06T22:00:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab37fe700 1 -- 172.21.15.33:0/3317499051 learned_addr learned my addr 172.21.15.33:0/3317499051 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab3fff700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ab4157720 0x7f9ab41562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:02.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.854+0000 7f9ab2ffd700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40bccc0 0x7f9ab4155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:02.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9ab37fe700 1 -- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ab4157720 msgr2=0x7f9ab41562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:02.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9ab37fe700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ab4157720 0x7f9ab41562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9ab37fe700 1 -- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40bccc0 msgr2=0x7f9ab4155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:02.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9ab37fe700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40bccc0 0x7f9ab4155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:02.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9ab37fe700 1 -- 172.21.15.33:0/3317499051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa8009370 con 0x7f9ab40a4e60 2023-11-06T22:00:02.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9ab37fe700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40a4e60 0x7f9ab40c0770 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f9aa400c560 tx=0x7f9aa4007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:02.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9a9b7fe700 1 -- 172.21.15.33:0/3317499051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40072e0 con 0x7f9ab40a4e60 2023-11-06T22:00:02.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9a9b7fe700 1 -- 172.21.15.33:0/3317499051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa4007440 con 0x7f9ab40a4e60 2023-11-06T22:00:02.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9a9b7fe700 1 -- 172.21.15.33:0/3317499051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40058f0 con 0x7f9ab40a4e60 2023-11-06T22:00:02.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3317499051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab414f170 con 0x7f9ab40a4e60 2023-11-06T22:00:02.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.855+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3317499051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab414fa30 con 0x7f9ab40a4e60 2023-11-06T22:00:02.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.857+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3317499051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab40bc4c0 con 0x7f9ab40a4e60 2023-11-06T22:00:02.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.857+0000 7f9a9b7fe700 1 -- 172.21.15.33:0/3317499051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f9aa4005ce0 con 0x7f9ab40a4e60 2023-11-06T22:00:02.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.858+0000 7f9a9b7fe700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa005f300 0x7f9aa00617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:02.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.858+0000 7f9a9b7fe700 1 -- 172.21.15.33:0/3317499051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f9aa4079cc0 con 0x7f9ab40a4e60 2023-11-06T22:00:02.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.858+0000 7f9ab2ffd700 1 -- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa005f300 msgr2=0x7f9aa00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:02.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.858+0000 7f9ab2ffd700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa005f300 0x7f9aa00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:00:02.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:02.866+0000 7f9a9b7fe700 1 -- 172.21.15.33:0/3317499051 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9aa404ac60 con 0x7f9ab40a4e60 2023-11-06T22:00:02.928 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:02.926+0000 7fa04a9ae680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:00:03.046 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:03.045+0000 7f62dc175680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:00:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.059+0000 7f9ab2ffd700 1 -- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa005f300 msgr2=0x7f9aa00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.059+0000 7f9ab2ffd700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa005f300 0x7f9aa00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:00:03.061 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.058+0000 7fa04a9ae680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:00:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.092+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3317499051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ab40a4aa0 con 0x7f9ab40a4e60 2023-11-06T22:00:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.093+0000 7f9a9b7fe700 1 -- 172.21.15.33:0/3317499051 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4491 (secure 0 0 0) 0x7f9aa404e280 con 0x7f9ab40a4e60 2023-11-06T22:00:03.094 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:03.094 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3645039939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3645039939},{"type":"v1","addr":"172.21.15.33:6835","nonce":3645039939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1156785194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1156785194},{"type":"v1","addr":"172.21.15.69:6836","nonce":1156785194}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/4183165491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":4183165491},{"type":"v1","addr":"172.21.15.69:6837","nonce":4183165491}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T21:59:40.865889+0000","modified":"2023-11-06T21:59:43.953031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8445":{"gid":8445,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6833/1588551062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1588551062},{"type":"v1","addr":"172.21.15.33:6833","nonce":1588551062}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-06T22:00:03.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.095+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa005f300 msgr2=0x7f9aa00617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:00:03.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.095+0000 7f9ab9dff700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa005f300 0x7f9aa00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:03.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.095+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3317499051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40a4e60 msgr2=0x7f9ab40c0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.095+0000 7f9ab9dff700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40a4e60 0x7f9ab40c0770 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f9aa400c560 tx=0x7f9aa4007c90).stop 2023-11-06T22:00:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.095+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3317499051 shutdown_connections 2023-11-06T22:00:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.095+0000 7f9ab9dff700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f9aa005f300 0x7f9aa00617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:03.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.095+0000 7f9ab9dff700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ab4157720 0x7f9ab41562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.095+0000 7f9ab9dff700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ab40a4e60 0x7f9ab40c0770 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.095+0000 7f9ab9dff700 1 --2- 172.21.15.33:0/3317499051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40bccc0 0x7f9ab4155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.095+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3317499051 >> 172.21.15.33:0/3317499051 conn(0x7f9ab40631e0 msgr2=0x7f9ab414cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:03.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.096+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3317499051 shutdown_connections 2023-11-06T22:00:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.096+0000 7f9ab9dff700 1 -- 172.21.15.33:0/3317499051 wait complete. 2023-11-06T22:00:03.101 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 68 2023-11-06T22:00:03.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:00:03.146 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:03.145+0000 7f62dc175680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:00:03.302 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:03.301+0000 7f62dc175680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:00:03.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.313+0000 7fa80b3ff700 1 Processor -- start 2023-11-06T22:00:03.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.313+0000 7fa80b3ff700 1 -- start start 2023-11-06T22:00:03.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.314+0000 7fa80b3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa80405e1b0 0x7fa804063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:03.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.314+0000 7fa80b3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8040c31f0 con 0x7fa804063140 2023-11-06T22:00:03.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.314+0000 7fa80b3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8040c2920 con 0x7fa8040c3680 2023-11-06T22:00:03.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.314+0000 7fa80b3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80406a0f0 con 0x7fa80405e1b0 2023-11-06T22:00:03.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.314+0000 7fa80919b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa804063140 0x7fa8040c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45374/0 (socket says 172.21.15.33:45374) 2023-11-06T22:00:03.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.314+0000 7fa80919b700 1 -- 172.21.15.33:0/2140728514 learned_addr learned my addr 172.21.15.33:0/2140728514 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:03.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.315+0000 7fa80899a700 1 --2- 172.21.15.33:0/2140728514 >> v2:172.21.15.69:3300/0 conn(0x7fa80405e1b0 0x7fa804063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:03.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.315+0000 7fa7fbfff700 1 -- 172.21.15.33:0/2140728514 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214308065 0 0) 0x7fa8040c31f0 con 0x7fa804063140 2023-11-06T22:00:03.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.315+0000 7fa7fbfff700 1 -- 172.21.15.33:0/2140728514 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7f00033f0 con 0x7fa804063140 2023-11-06T22:00:03.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.315+0000 7fa80899a700 1 -- 172.21.15.33:0/2140728514 >> v1:172.21.15.69:6789/0 conn(0x7fa8040c3680 legacy=0x7fa804154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.315+0000 7fa7fbfff700 1 -- 172.21.15.33:0/2140728514 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943534667 0 0) 0x7fa8040c2920 con 0x7fa8040c3680 2023-11-06T22:00:03.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.315+0000 7fa80899a700 1 -- 172.21.15.33:0/2140728514 >> v1:172.21.15.33:6789/0 conn(0x7fa804063140 legacy=0x7fa8040c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.315+0000 7fa80899a700 1 -- 172.21.15.33:0/2140728514 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80405e640 con 0x7fa80405e1b0 2023-11-06T22:00:03.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.315+0000 7fa80899a700 1 --2- 172.21.15.33:0/2140728514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa80405e1b0 0x7fa804063520 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fa7f4009760 tx=0x7fa7f4009060).ready entity=mon.1 client_cookie=10bd16a81940d762 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:03.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.316+0000 7fa7fbfff700 1 -- 172.21.15.33:0/2140728514 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f400f510 con 0x7fa80405e1b0 2023-11-06T22:00:03.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.316+0000 7fa7fbfff700 1 -- 172.21.15.33:0/2140728514 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f400f670 con 0x7fa80405e1b0 2023-11-06T22:00:03.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.316+0000 7fa7fbfff700 1 -- 172.21.15.33:0/2140728514 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f400f9f0 con 0x7fa80405e1b0 2023-11-06T22:00:03.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.316+0000 7fa80b3ff700 1 -- 172.21.15.33:0/2140728514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa80405e1b0 msgr2=0x7fa804063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.316+0000 7fa80b3ff700 1 --2- 172.21.15.33:0/2140728514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa80405e1b0 0x7fa804063520 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fa7f4009760 tx=0x7fa7f4009060).stop 2023-11-06T22:00:03.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.316+0000 7fa80b3ff700 1 -- 172.21.15.33:0/2140728514 shutdown_connections 2023-11-06T22:00:03.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.316+0000 7fa80b3ff700 1 --2- 172.21.15.33:0/2140728514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa80405e1b0 0x7fa804063520 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.316+0000 7fa80b3ff700 1 -- 172.21.15.33:0/2140728514 >> 172.21.15.33:0/2140728514 conn(0x7fa8040c7d60 msgr2=0x7fa8040c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:03.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.316+0000 7fa80b3ff700 1 -- 172.21.15.33:0/2140728514 shutdown_connections 2023-11-06T22:00:03.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.316+0000 7fa80b3ff700 1 -- 172.21.15.33:0/2140728514 wait complete. 2023-11-06T22:00:03.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.318+0000 7fa80b3ff700 1 Processor -- start 2023-11-06T22:00:03.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.318+0000 7fa80b3ff700 1 -- start start 2023-11-06T22:00:03.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80b3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa804063140 0x7fa8041644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:03.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80b3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8040c3680 0x7fa804141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80b3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8041421f0 0x7fa80415a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80b3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80414f420 con 0x7fa804063140 2023-11-06T22:00:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80b3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80414f690 con 0x7fa8040c3680 2023-11-06T22:00:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80b3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80414f970 con 0x7fa8041421f0 2023-11-06T22:00:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa804063140 0x7fa8041644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa804063140 0x7fa8041644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46260/0 (socket says 172.21.15.33:46260) 2023-11-06T22:00:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80919b700 1 -- 172.21.15.33:0/566565133 learned_addr learned my addr 172.21.15.33:0/566565133 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80899a700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8040c3680 0x7fa804141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.319+0000 7fa80999c700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8041421f0 0x7fa80415a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa80919b700 1 -- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8041421f0 msgr2=0x7fa80415a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa80919b700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8041421f0 0x7fa80415a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa80919b700 1 -- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8040c3680 msgr2=0x7fa804141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa80919b700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8040c3680 0x7fa804141d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa80919b700 1 -- 172.21.15.33:0/566565133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40092a0 con 0x7fa804063140 2023-11-06T22:00:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa80919b700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa804063140 0x7fa8041644e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7fa800001ed0 tx=0x7fa800007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa7f97fa700 1 -- 172.21.15.33:0/566565133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8000072e0 con 0x7fa804063140 2023-11-06T22:00:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa7f97fa700 1 -- 172.21.15.33:0/566565133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa800007440 con 0x7fa804063140 2023-11-06T22:00:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa80b3ff700 1 -- 172.21.15.33:0/566565133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa804155040 con 0x7fa804063140 2023-11-06T22:00:03.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa7f97fa700 1 -- 172.21.15.33:0/566565133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8000057a0 con 0x7fa804063140 2023-11-06T22:00:03.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.320+0000 7fa80b3ff700 1 -- 172.21.15.33:0/566565133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8041578e0 con 0x7fa804063140 2023-11-06T22:00:03.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.322+0000 7fa80b3ff700 1 -- 172.21.15.33:0/566565133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa804155280 con 0x7fa804063140 2023-11-06T22:00:03.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.322+0000 7fa7f97fa700 1 -- 172.21.15.33:0/566565133 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fa800005b50 con 0x7fa804063140 2023-11-06T22:00:03.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.323+0000 7fa7f97fa700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7e805f2d0 0x7fa7e8061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:03.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.323+0000 7fa7f97fa700 1 -- 172.21.15.33:0/566565133 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6423+0+0 (secure 0 0 0) 0x7fa800079d40 con 0x7fa804063140 2023-11-06T22:00:03.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.324+0000 7fa80899a700 1 -- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7e805f2d0 msgr2=0x7fa7e8061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:03.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.324+0000 7fa80899a700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7e805f2d0 0x7fa7e8061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:00:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.328+0000 7fa7f97fa700 1 -- 172.21.15.33:0/566565133 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa804155280 con 0x7fa804063140 2023-11-06T22:00:03.409 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.407+0000 7fa04a9ae680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:00:03.446 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:03.443+0000 7f62dc175680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:00:03.499 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.497+0000 7fa04a9ae680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:00:03.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.511+0000 7fa7f97fa700 1 -- 172.21.15.33:0/566565133 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 71) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fa8000776e0 con 0x7fa804063140 2023-11-06T22:00:03.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.511+0000 7fa80b3ff700 1 -- 172.21.15.33:0/566565133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa804155280 con 0x7fa804063140 2023-11-06T22:00:03.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.512+0000 7fa7f97fa700 1 -- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7e805f2d0 msgr2=0x7fa7e8061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:00:03.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.512+0000 7fa7f97fa700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7e805f2d0 0x7fa7e8061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:03.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.514+0000 7fa7f97fa700 1 -- 172.21.15.33:0/566565133 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+17736 (secure 0 0 0) 0x7fa804155280 con 0x7fa804063140 2023-11-06T22:00:03.516 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:03.517 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":172,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:00:03.501533+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T21:59:38.416213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T21:59:39.859883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/808320956":"2023-11-07T22:00:03.501457+0000","172.21.15.69:0/2271633588":"2023-11-07T22:00:03.501457+0000","172.21.15.69:0/1344586871":"2023-11-07T22:00:03.501457+0000","172.21.15.69:0/3352936994":"2023-11-07T22:00:03.501457+0000","172.21.15.69:0/4097050624":"2023-11-07T22:00:03.501457+0000","172.21.15.69:6833/114912":"2023-11-07T21:59:55.184802+0000","172.21.15.69:6832/114912":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/351947370":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/623316503":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/861101334":"2023-11-07T21:59:55.184802+0000","172.21.15.69:0/1093219585":"2023-11-07T21:59:55.184802+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:00:03.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.518+0000 7fa80b3ff700 1 -- 172.21.15.33:0/566565133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa804063140 msgr2=0x7fa8041644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.518+0000 7fa80b3ff700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa804063140 0x7fa8041644e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7fa800001ed0 tx=0x7fa800007c70).stop 2023-11-06T22:00:03.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.518+0000 7fa80b3ff700 1 -- 172.21.15.33:0/566565133 shutdown_connections 2023-11-06T22:00:03.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.518+0000 7fa80b3ff700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fa7e805f2d0 0x7fa7e8061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:03.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.518+0000 7fa80b3ff700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8041421f0 0x7fa80415a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.518+0000 7fa80b3ff700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa804063140 0x7fa8041644e0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.518+0000 7fa80b3ff700 1 --2- 172.21.15.33:0/566565133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8040c3680 0x7fa804141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.518+0000 7fa80b3ff700 1 -- 172.21.15.33:0/566565133 >> 172.21.15.33:0/566565133 conn(0x7fa8040c7d60 msgr2=0x7fa8041516f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:03.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.518+0000 7fa80b3ff700 1 -- 172.21.15.33:0/566565133 shutdown_connections 2023-11-06T22:00:03.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.519+0000 7fa80b3ff700 1 -- 172.21.15.33:0/566565133 wait complete. 2023-11-06T22:00:03.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T22:00:03.578 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.575+0000 7f9fc64d3700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:03.578 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.575+0000 7f9fc64d3700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:03.578 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.575+0000 7f9fc64d3700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:03.579 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.575+0000 7f9fc64d3700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:03.579 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.575+0000 7f9fc64d3700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:03.579 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.576+0000 7f9fc34cd700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:03.580 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.576+0000 7f9fc34cd700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:03.580 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.576+0000 7f9fc34cd700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:03.580 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.576+0000 7f9fc34cd700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:03.580 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:03.576+0000 7f9fc34cd700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.784+0000 7f29de7ff700 1 Processor -- start 2023-11-06T22:00:03.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.784+0000 7f29de7ff700 1 -- start start 2023-11-06T22:00:03.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.786+0000 7f29de7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f29d81559d0 0x7f29d80bdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:03.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.786+0000 7f29de7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d80be0a0 0x7f29d80c4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.786+0000 7f29de7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d80be500 con 0x7f29d81555f0 2023-11-06T22:00:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.786+0000 7f29de7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d8067020 con 0x7f29d81559d0 2023-11-06T22:00:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.786+0000 7f29de7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d80c3790 con 0x7f29d80be0a0 2023-11-06T22:00:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.786+0000 7f29d7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f29d81555f0 0x7f29d8066560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45388/0 (socket says 172.21.15.33:45388) 2023-11-06T22:00:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.786+0000 7f29d7fff700 1 -- 172.21.15.33:0/2199123367 learned_addr learned my addr 172.21.15.33:0/2199123367 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.786+0000 7f29d77fe700 1 --2- 172.21.15.33:0/2199123367 >> v2:172.21.15.69:3300/0 conn(0x7f29d81559d0 0x7f29d80bdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.786+0000 7f29dcd9c700 1 --2- 172.21.15.33:0/2199123367 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d80be0a0 0x7f29d80c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:03.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.787+0000 7f29d6ffd700 1 -- 172.21.15.33:0/2199123367 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830651032 0 0) 0x7f29d80be500 con 0x7f29d81555f0 2023-11-06T22:00:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.787+0000 7f29d6ffd700 1 -- 172.21.15.33:0/2199123367 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c00033f0 con 0x7f29d81555f0 2023-11-06T22:00:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.787+0000 7f29d77fe700 1 -- 172.21.15.33:0/2199123367 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d80be0a0 msgr2=0x7f29d80c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.787+0000 7f29d77fe700 1 --2- 172.21.15.33:0/2199123367 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d80be0a0 0x7f29d80c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.787+0000 7f29d77fe700 1 -- 172.21.15.33:0/2199123367 >> v1:172.21.15.33:6789/0 conn(0x7f29d81555f0 legacy=0x7f29d8066560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.787+0000 7f29d77fe700 1 -- 172.21.15.33:0/2199123367 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d80c3a70 con 0x7f29d81559d0 2023-11-06T22:00:03.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.787+0000 7f29d77fe700 1 --2- 172.21.15.33:0/2199123367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29d81559d0 0x7f29d80bdbb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f29cc009760 tx=0x7f29cc009060).ready entity=mon.1 client_cookie=11b1fb1b30f1ba9b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:03.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.787+0000 7f29d6ffd700 1 -- 172.21.15.33:0/2199123367 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00d4a0 con 0x7f29d81559d0 2023-11-06T22:00:03.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.787+0000 7f29d6ffd700 1 -- 172.21.15.33:0/2199123367 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc00d600 con 0x7f29d81559d0 2023-11-06T22:00:03.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.787+0000 7f29d6ffd700 1 -- 172.21.15.33:0/2199123367 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00d980 con 0x7f29d81559d0 2023-11-06T22:00:03.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.788+0000 7f29de7ff700 1 -- 172.21.15.33:0/2199123367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29d81559d0 msgr2=0x7f29d80bdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.788+0000 7f29de7ff700 1 --2- 172.21.15.33:0/2199123367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29d81559d0 0x7f29d80bdbb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f29cc009760 tx=0x7f29cc009060).stop 2023-11-06T22:00:03.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.788+0000 7f29de7ff700 1 -- 172.21.15.33:0/2199123367 shutdown_connections 2023-11-06T22:00:03.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.788+0000 7f29de7ff700 1 --2- 172.21.15.33:0/2199123367 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d80be0a0 0x7f29d80c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.788+0000 7f29de7ff700 1 --2- 172.21.15.33:0/2199123367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29d81559d0 0x7f29d80bdbb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.788+0000 7f29de7ff700 1 -- 172.21.15.33:0/2199123367 >> 172.21.15.33:0/2199123367 conn(0x7f29d80c56f0 msgr2=0x7f29d80c5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.788+0000 7f29de7ff700 1 -- 172.21.15.33:0/2199123367 shutdown_connections 2023-11-06T22:00:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.788+0000 7f29de7ff700 1 -- 172.21.15.33:0/2199123367 wait complete. 2023-11-06T22:00:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.790+0000 7f29de7ff700 1 Processor -- start 2023-11-06T22:00:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.790+0000 7f29de7ff700 1 -- start start 2023-11-06T22:00:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29de7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d80be0a0 0x7f29d81501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29de7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d81555f0 0x7f29d8167df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29de7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29d81559d0 0x7f29d80ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29de7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d80abff0 con 0x7f29d80be0a0 2023-11-06T22:00:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29de7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d81686b0 con 0x7f29d81559d0 2023-11-06T22:00:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29de7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d8168990 con 0x7f29d81555f0 2023-11-06T22:00:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d80be0a0 0x7f29d81501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d80be0a0 0x7f29d81501d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46268/0 (socket says 172.21.15.33:46268) 2023-11-06T22:00:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29d7fff700 1 -- 172.21.15.33:0/2653790038 learned_addr learned my addr 172.21.15.33:0/2653790038 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29d77fe700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d81555f0 0x7f29d8167df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29dcd9c700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29d81559d0 0x7f29d80ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29d7fff700 1 -- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d81555f0 msgr2=0x7f29d8167df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29d7fff700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d81555f0 0x7f29d8167df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29d7fff700 1 -- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29d81559d0 msgr2=0x7f29d80ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29d7fff700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29d81559d0 0x7f29d80ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29d7fff700 1 -- 172.21.15.33:0/2653790038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc0092a0 con 0x7f29d80be0a0 2023-11-06T22:00:03.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29dcd9c700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29d81559d0 0x7f29d80ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:00:03.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.791+0000 7f29d77fe700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d81555f0 0x7f29d8167df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:00:03.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.792+0000 7f29d7fff700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d80be0a0 0x7f29d81501d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f29c800c590 tx=0x7f29c8007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:03.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.792+0000 7f29bffff700 1 -- 172.21.15.33:0/2653790038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8007e40 con 0x7f29d80be0a0 2023-11-06T22:00:03.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.792+0000 7f29bffff700 1 -- 172.21.15.33:0/2653790038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c80072e0 con 0x7f29d80be0a0 2023-11-06T22:00:03.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.792+0000 7f29de7ff700 1 -- 172.21.15.33:0/2653790038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d8168c70 con 0x7f29d80be0a0 2023-11-06T22:00:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.792+0000 7f29bffff700 1 -- 172.21.15.33:0/2653790038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c80057a0 con 0x7f29d80be0a0 2023-11-06T22:00:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.792+0000 7f29de7ff700 1 -- 172.21.15.33:0/2653790038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d80ae290 con 0x7f29d80be0a0 2023-11-06T22:00:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.793+0000 7f29de7ff700 1 -- 172.21.15.33:0/2653790038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d8066000 con 0x7f29d80be0a0 2023-11-06T22:00:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.795+0000 7f29bffff700 1 -- 172.21.15.33:0/2653790038 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 71) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f29c8005bd0 con 0x7f29d80be0a0 2023-11-06T22:00:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.796+0000 7f29bffff700 1 -- 172.21.15.33:0/2653790038 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f29c80797b0 con 0x7f29d80be0a0 2023-11-06T22:00:03.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:03.799+0000 7f29bffff700 1 -- 172.21.15.33:0/2653790038 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f29c804a940 con 0x7f29d80be0a0 2023-11-06T22:00:03.818 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:03.818+0000 7f62dc175680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:00:03.914 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:03.913+0000 7f62dc175680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:00:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.024+0000 7f29de7ff700 1 -- 172.21.15.33:0/2653790038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f29d80c3d50 con 0x7f29d80be0a0 2023-11-06T22:00:04.508 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T22:00:04.507+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T22:00:04.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.512+0000 7f29bffff700 1 -- 172.21.15.33:0/2653790038 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 72) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f29c8077440 con 0x7f29d80be0a0 2023-11-06T22:00:04.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.513+0000 7f29bffff700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29c40302c0 0x7f29c4067b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:04.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.513+0000 7f29d77fe700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29c40302c0 0x7f29c4067b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:04.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.518+0000 7f29d77fe700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29c40302c0 0x7f29c4067b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f29d80aecf0 tx=0x7f29cc016000).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.522+0000 7f29bffff700 1 -- 172.21.15.33:0/2653790038 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7f29c8074310 con 0x7f29d80be0a0 2023-11-06T22:00:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 -- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29c40302c0 msgr2=0x7f29c4067b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29c40302c0 0x7f29c4067b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f29d80aecf0 tx=0x7f29cc016000).stop 2023-11-06T22:00:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 -- 172.21.15.33:0/2653790038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d80be0a0 msgr2=0x7f29d81501d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:04.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d80be0a0 0x7f29d81501d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f29c800c590 tx=0x7f29c8007c80).stop 2023-11-06T22:00:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 -- 172.21.15.33:0/2653790038 shutdown_connections 2023-11-06T22:00:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f29c40302c0 0x7f29c4067b20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29d81559d0 0x7f29d80ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29d80be0a0 0x7f29d81501d0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 --2- 172.21.15.33:0/2653790038 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29d81555f0 0x7f29d8167df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 -- 172.21.15.33:0/2653790038 >> 172.21.15.33:0/2653790038 conn(0x7f29d80c56f0 msgr2=0x7f29d80c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.524+0000 7f29de7ff700 1 -- 172.21.15.33:0/2653790038 shutdown_connections 2023-11-06T22:00:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.525+0000 7f29de7ff700 1 -- 172.21.15.33:0/2653790038 wait complete. 2023-11-06T22:00:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T22:00:04.530 INFO:tasks.ceph.mds.a.smithi033.stderr: -2459> 2023-11-06T21:59:45.358+0000 7fa24cdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T22:00:04.546 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T22:00:04.572 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T22:00:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.745+0000 7fc069bff700 1 Processor -- start 2023-11-06T22:00:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.745+0000 7fc069bff700 1 -- start start 2023-11-06T22:00:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc069bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0640c72c0 0x7fc0641553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc069bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0640637f0 0x7fc0640c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc069bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc064152470 con 0x7fc064157fa0 2023-11-06T22:00:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc069bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc064154990 con 0x7fc0640637f0 2023-11-06T22:00:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc069bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0640c76a0 con 0x7fc0640c72c0 2023-11-06T22:00:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc063fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0640637f0 0x7fc0640c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc063fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc0640637f0 0x7fc0640c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46290/0 (socket says 172.21.15.33:46290) 2023-11-06T22:00:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc063fff700 1 -- 172.21.15.33:0/4168484331 learned_addr learned my addr 172.21.15.33:0/4168484331 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc0637fe700 1 --2- 172.21.15.33:0/4168484331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0640c72c0 0x7fc0641553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc063fff700 1 -- 172.21.15.33:0/4168484331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0640c72c0 msgr2=0x7fc0641553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc063fff700 1 --2- 172.21.15.33:0/4168484331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0640c72c0 0x7fc0641553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc063fff700 1 -- 172.21.15.33:0/4168484331 >> v1:172.21.15.69:6789/0 conn(0x7fc064157fa0 legacy=0x7fc0640630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.747+0000 7fc063fff700 1 -- 172.21.15.33:0/4168484331 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc064158910 con 0x7fc0640637f0 2023-11-06T22:00:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.748+0000 7fc063fff700 1 --2- 172.21.15.33:0/4168484331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0640637f0 0x7fc0640c5be0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7fc05c009f50 tx=0x7fc05c009840).ready entity=mon.0 client_cookie=c4d77702a832089f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.748+0000 7fc0627fc700 1 -- 172.21.15.33:0/4168484331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00e070 con 0x7fc0640637f0 2023-11-06T22:00:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.748+0000 7fc0627fc700 1 -- 172.21.15.33:0/4168484331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c00ba30 con 0x7fc0640637f0 2023-11-06T22:00:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.748+0000 7fc0627fc700 1 -- 172.21.15.33:0/4168484331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00bdb0 con 0x7fc0640637f0 2023-11-06T22:00:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.748+0000 7fc069bff700 1 -- 172.21.15.33:0/4168484331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0640637f0 msgr2=0x7fc0640c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.748+0000 7fc069bff700 1 --2- 172.21.15.33:0/4168484331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0640637f0 0x7fc0640c5be0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7fc05c009f50 tx=0x7fc05c009840).stop 2023-11-06T22:00:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.749+0000 7fc069bff700 1 -- 172.21.15.33:0/4168484331 shutdown_connections 2023-11-06T22:00:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.749+0000 7fc069bff700 1 --2- 172.21.15.33:0/4168484331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0640637f0 0x7fc0640c5be0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.749+0000 7fc069bff700 1 --2- 172.21.15.33:0/4168484331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0640c72c0 0x7fc0641553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.749+0000 7fc069bff700 1 -- 172.21.15.33:0/4168484331 >> 172.21.15.33:0/4168484331 conn(0x7fc06405ec20 msgr2=0x7fc0640cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.749+0000 7fc069bff700 1 -- 172.21.15.33:0/4168484331 shutdown_connections 2023-11-06T22:00:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.749+0000 7fc069bff700 1 -- 172.21.15.33:0/4168484331 wait complete. 2023-11-06T22:00:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.750+0000 7fc069bff700 1 Processor -- start 2023-11-06T22:00:04.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.750+0000 7fc069bff700 1 -- start start 2023-11-06T22:00:04.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.751+0000 7fc069bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0640637f0 0x7fc0640c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:04.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.751+0000 7fc069bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0640c72c0 0x7fc064157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:04.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.751+0000 7fc069bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc064157fa0 0x7fc0640c2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:04.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.751+0000 7fc069bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0640c42e0 con 0x7fc064157fa0 2023-11-06T22:00:04.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.751+0000 7fc069bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc064157a60 con 0x7fc0640c72c0 2023-11-06T22:00:04.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.751+0000 7fc069bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0640c4540 con 0x7fc0640637f0 2023-11-06T22:00:04.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.751+0000 7fc063fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc064157fa0 0x7fc0640c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc063fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc064157fa0 0x7fc0640c2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46306/0 (socket says 172.21.15.33:46306) 2023-11-06T22:00:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc063fff700 1 -- 172.21.15.33:0/1457162131 learned_addr learned my addr 172.21.15.33:0/1457162131 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc0637fe700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0640637f0 0x7fc0640c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc062ffd700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0640c72c0 0x7fc064157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc063fff700 1 -- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0640637f0 msgr2=0x7fc0640c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc063fff700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0640637f0 0x7fc0640c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc063fff700 1 -- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0640c72c0 msgr2=0x7fc064157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc063fff700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0640c72c0 0x7fc064157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc063fff700 1 -- 172.21.15.33:0/1457162131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c009ae0 con 0x7fc064157fa0 2023-11-06T22:00:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc063fff700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc064157fa0 0x7fc0640c2290 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7fc06406c2a0 tx=0x7fc05c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc04b7fe700 1 -- 172.21.15.33:0/1457162131 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c0092e0 con 0x7fc064157fa0 2023-11-06T22:00:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc04b7fe700 1 -- 172.21.15.33:0/1457162131 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c01e470 con 0x7fc064157fa0 2023-11-06T22:00:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc04b7fe700 1 -- 172.21.15.33:0/1457162131 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c01e810 con 0x7fc064157fa0 2023-11-06T22:00:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.752+0000 7fc069bff700 1 -- 172.21.15.33:0/1457162131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0640c4820 con 0x7fc064157fa0 2023-11-06T22:00:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.753+0000 7fc069bff700 1 -- 172.21.15.33:0/1457162131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0640c4de0 con 0x7fc064157fa0 2023-11-06T22:00:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.754+0000 7fc069bff700 1 -- 172.21.15.33:0/1457162131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc064158bf0 con 0x7fc064157fa0 2023-11-06T22:00:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.754+0000 7fc04b7fe700 1 -- 172.21.15.33:0/1457162131 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 72) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fc05c003040 con 0x7fc064157fa0 2023-11-06T22:00:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.755+0000 7fc04b7fe700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc05005f030 0x7fc0500614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.755+0000 7fc04b7fe700 1 -- 172.21.15.33:0/1457162131 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fc05c07b8b0 con 0x7fc064157fa0 2023-11-06T22:00:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.756+0000 7fc0637fe700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc05005f030 0x7fc0500614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.756+0000 7fc0637fe700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc05005f030 0x7fc0500614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fc05400bb20 tx=0x7fc054008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.760+0000 7fc04b7fe700 1 -- 172.21.15.33:0/1457162131 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc05c04cba0 con 0x7fc064157fa0 2023-11-06T22:00:04.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:04.994+0000 7fc069bff700 1 -- 172.21.15.33:0/1457162131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc06406abb0 con 0x7fc064157fa0 2023-11-06T22:00:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.527+0000 7fc04b7fe700 1 -- 172.21.15.33:0/1457162131 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc05c023020 con 0x7fc064157fa0 2023-11-06T22:00:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.530+0000 7fc069bff700 1 -- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc05005f030 msgr2=0x7fc0500614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.530+0000 7fc069bff700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc05005f030 0x7fc0500614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fc05400bb20 tx=0x7fc054008040).stop 2023-11-06T22:00:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.530+0000 7fc069bff700 1 -- 172.21.15.33:0/1457162131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc064157fa0 msgr2=0x7fc0640c2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.530+0000 7fc069bff700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc064157fa0 0x7fc0640c2290 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7fc06406c2a0 tx=0x7fc05c009840).stop 2023-11-06T22:00:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.530+0000 7fc069bff700 1 -- 172.21.15.33:0/1457162131 shutdown_connections 2023-11-06T22:00:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.530+0000 7fc069bff700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc05005f030 0x7fc0500614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.530+0000 7fc069bff700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc0640637f0 0x7fc0640c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.530+0000 7fc069bff700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc064157fa0 0x7fc0640c2290 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.530+0000 7fc069bff700 1 --2- 172.21.15.33:0/1457162131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0640c72c0 0x7fc064157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.530+0000 7fc069bff700 1 -- 172.21.15.33:0/1457162131 >> 172.21.15.33:0/1457162131 conn(0x7fc06405ec20 msgr2=0x7fc0641519b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.531+0000 7fc069bff700 1 -- 172.21.15.33:0/1457162131 shutdown_connections 2023-11-06T22:00:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.531+0000 7fc069bff700 1 -- 172.21.15.33:0/1457162131 wait complete. 2023-11-06T22:00:05.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T22:00:05.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.796+0000 7f6cc55ff700 1 Processor -- start 2023-11-06T22:00:05.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.796+0000 7f6cc55ff700 1 -- start start 2023-11-06T22:00:05.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.797+0000 7f6cc55ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc01542b0 0x7f6cc00c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.797+0000 7f6cc55ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc00c9f80 con 0x7f6cc00c8ee0 2023-11-06T22:00:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.797+0000 7f6cc55ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc00cab20 con 0x7f6cc0153ed0 2023-11-06T22:00:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.797+0000 7f6cc55ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc00c6000 con 0x7f6cc01542b0 2023-11-06T22:00:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.797+0000 7f6cbe7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6cc00c8ee0 0x7f6cc00ca410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45394/0 (socket says 172.21.15.33:45394) 2023-11-06T22:00:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.797+0000 7f6cbe7fc700 1 -- 172.21.15.33:0/2642572021 learned_addr learned my addr 172.21.15.33:0/2642572021 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:05.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.798+0000 7f6cbf7fe700 1 --2- 172.21.15.33:0/2642572021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc01542b0 0x7f6cc00c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:05.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.798+0000 7f6cbdffb700 1 -- 172.21.15.33:0/2642572021 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463448108 0 0) 0x7f6cc00c9f80 con 0x7f6cc00c8ee0 2023-11-06T22:00:05.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.798+0000 7f6cbdffb700 1 -- 172.21.15.33:0/2642572021 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca80033f0 con 0x7f6cc00c8ee0 2023-11-06T22:00:05.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.798+0000 7f6cbdffb700 1 -- 172.21.15.33:0/2642572021 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510260140 0 0) 0x7f6cc00cab20 con 0x7f6cc0153ed0 2023-11-06T22:00:05.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.798+0000 7f6cbdffb700 1 -- 172.21.15.33:0/2642572021 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc00c9f80 con 0x7f6cc0153ed0 2023-11-06T22:00:05.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.798+0000 7f6cbf7fe700 1 -- 172.21.15.33:0/2642572021 >> v1:172.21.15.69:6789/0 conn(0x7f6cc0153ed0 legacy=0x7f6cc0158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:05.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.798+0000 7f6cbf7fe700 1 -- 172.21.15.33:0/2642572021 >> v1:172.21.15.33:6789/0 conn(0x7f6cc00c8ee0 legacy=0x7f6cc00ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:05.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.798+0000 7f6cbf7fe700 1 -- 172.21.15.33:0/2642572021 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc00c9370 con 0x7f6cc01542b0 2023-11-06T22:00:05.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.798+0000 7f6cbf7fe700 1 --2- 172.21.15.33:0/2642572021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc01542b0 0x7f6cc00c76f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f6cb8009e90 tx=0x7f6cb8009780).ready entity=mon.2 client_cookie=34b4978b25c0a88a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:05.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.799+0000 7f6cbdffb700 1 -- 172.21.15.33:0/2642572021 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800e070 con 0x7f6cc01542b0 2023-11-06T22:00:05.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.799+0000 7f6cbdffb700 1 -- 172.21.15.33:0/2642572021 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb800bab0 con 0x7f6cc01542b0 2023-11-06T22:00:05.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.799+0000 7f6cbdffb700 1 -- 172.21.15.33:0/2642572021 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800be30 con 0x7f6cc01542b0 2023-11-06T22:00:05.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.799+0000 7f6cc55ff700 1 -- 172.21.15.33:0/2642572021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc01542b0 msgr2=0x7f6cc00c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:05.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.799+0000 7f6cc55ff700 1 --2- 172.21.15.33:0/2642572021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc01542b0 0x7f6cc00c76f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f6cb8009e90 tx=0x7f6cb8009780).stop 2023-11-06T22:00:05.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.799+0000 7f6cc55ff700 1 -- 172.21.15.33:0/2642572021 shutdown_connections 2023-11-06T22:00:05.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.799+0000 7f6cc55ff700 1 --2- 172.21.15.33:0/2642572021 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc01542b0 0x7f6cc00c76f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:05.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.799+0000 7f6cc55ff700 1 -- 172.21.15.33:0/2642572021 >> 172.21.15.33:0/2642572021 conn(0x7f6cc005e1b0 msgr2=0x7f6cc005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:05.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.799+0000 7f6cc55ff700 1 -- 172.21.15.33:0/2642572021 shutdown_connections 2023-11-06T22:00:05.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.800+0000 7f6cc55ff700 1 -- 172.21.15.33:0/2642572021 wait complete. 2023-11-06T22:00:05.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.801+0000 7f6cc55ff700 1 Processor -- start 2023-11-06T22:00:05.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.801+0000 7f6cc55ff700 1 -- start start 2023-11-06T22:00:05.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.802+0000 7f6cc55ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc0153ed0 0x7f6cc0168570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:05.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.802+0000 7f6cc55ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc01542b0 0x7f6cc014cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:05.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.802+0000 7f6cc55ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc0156460 0x7f6cc01546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:05.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.802+0000 7f6cc55ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc0155fd0 con 0x7f6cc0153ed0 2023-11-06T22:00:05.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.802+0000 7f6cc55ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc0168a60 con 0x7f6cc01542b0 2023-11-06T22:00:05.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.802+0000 7f6cc55ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc00afe60 con 0x7f6cc0156460 2023-11-06T22:00:05.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.802+0000 7f6cbeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc0153ed0 0x7f6cc0168570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:05.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.802+0000 7f6cbeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc0153ed0 0x7f6cc0168570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46316/0 (socket says 172.21.15.33:46316) 2023-11-06T22:00:05.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.802+0000 7f6cbeffd700 1 -- 172.21.15.33:0/1786967327 learned_addr learned my addr 172.21.15.33:0/1786967327 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:05.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6cbe7fc700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc01542b0 0x7f6cc014cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:05.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6cbf7fe700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc0156460 0x7f6cc01546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:05.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6cbeffd700 1 -- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc0156460 msgr2=0x7f6cc01546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:05.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6cbeffd700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc0156460 0x7f6cc01546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:05.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6cbeffd700 1 -- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc01542b0 msgr2=0x7f6cc014cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:05.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6cbeffd700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc01542b0 0x7f6cc014cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:05.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6cbeffd700 1 -- 172.21.15.33:0/1786967327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb8009a20 con 0x7f6cc0153ed0 2023-11-06T22:00:05.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6cbeffd700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc0153ed0 0x7f6cc0168570 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f6cb000c490 tx=0x7f6cb0007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:05.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/1786967327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000ce40 con 0x7f6cc0153ed0 2023-11-06T22:00:05.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/1786967327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb000f470 con 0x7f6cc0153ed0 2023-11-06T22:00:05.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/1786967327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000f810 con 0x7f6cc0153ed0 2023-11-06T22:00:05.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.803+0000 7f6cc55ff700 1 -- 172.21.15.33:0/1786967327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc00b0140 con 0x7f6cc0153ed0 2023-11-06T22:00:05.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.804+0000 7f6cc55ff700 1 -- 172.21.15.33:0/1786967327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc0155980 con 0x7f6cc0153ed0 2023-11-06T22:00:05.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.805+0000 7f6cc55ff700 1 -- 172.21.15.33:0/1786967327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc006c6c0 con 0x7f6cc0153ed0 2023-11-06T22:00:05.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.806+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/1786967327 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 72) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f6cb000fbc0 con 0x7f6cc0153ed0 2023-11-06T22:00:05.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.806+0000 7f6ca6ffd700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6cac05f030 0x7f6cac0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:05.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.806+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/1786967327 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f6cb00796c0 con 0x7f6cc0153ed0 2023-11-06T22:00:05.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.807+0000 7f6cbe7fc700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6cac05f030 0x7f6cac0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:05.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.808+0000 7f6cbe7fc700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6cac05f030 0x7f6cac0614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6cc00c4da0 tx=0x7f6cb4009040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:05.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.810+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/1786967327 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6cb004a900 con 0x7f6cc0153ed0 2023-11-06T22:00:05.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:05.989+0000 7f6cc55ff700 1 -- 172.21.15.33:0/1786967327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6cc00c9650 con 0x7f6cc0153ed0 2023-11-06T22:00:06.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.279+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/1786967327 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6cb004a080 con 0x7f6cc0153ed0 2023-11-06T22:00:06.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.534+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/1786967327 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v175) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6cb0072fe0 con 0x7f6cc0153ed0 2023-11-06T22:00:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.539+0000 7f6cc55ff700 1 -- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6cac05f030 msgr2=0x7f6cac0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.539+0000 7f6cc55ff700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6cac05f030 0x7f6cac0614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6cc00c4da0 tx=0x7f6cb4009040).stop 2023-11-06T22:00:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.539+0000 7f6cc55ff700 1 -- 172.21.15.33:0/1786967327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc0153ed0 msgr2=0x7f6cc0168570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.539+0000 7f6cc55ff700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc0153ed0 0x7f6cc0168570 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f6cb000c490 tx=0x7f6cb0007c90).stop 2023-11-06T22:00:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.539+0000 7f6cc55ff700 1 -- 172.21.15.33:0/1786967327 shutdown_connections 2023-11-06T22:00:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.539+0000 7f6cc55ff700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6cac05f030 0x7f6cac0614f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.539+0000 7f6cc55ff700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6cc0156460 0x7f6cc01546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.539+0000 7f6cc55ff700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc0153ed0 0x7f6cc0168570 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:06.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.539+0000 7f6cc55ff700 1 --2- 172.21.15.33:0/1786967327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cc01542b0 0x7f6cc014cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:06.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.539+0000 7f6cc55ff700 1 -- 172.21.15.33:0/1786967327 >> 172.21.15.33:0/1786967327 conn(0x7f6cc005e1b0 msgr2=0x7f6cc014d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:06.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.540+0000 7f6cc55ff700 1 -- 172.21.15.33:0/1786967327 shutdown_connections 2023-11-06T22:00:06.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.541+0000 7f6cc55ff700 1 -- 172.21.15.33:0/1786967327 wait complete. 2023-11-06T22:00:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T22:00:06.563 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T22:00:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.763+0000 7f120f3ff700 1 Processor -- start 2023-11-06T22:00:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.763+0000 7f120f3ff700 1 -- start start 2023-11-06T22:00:06.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.764+0000 7f120f3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1208153ed0 0x7f1208158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.764+0000 7f120f3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f12081542b0 0x7f12080c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.764+0000 7f120f3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12080c9f80 con 0x7f12080c8ee0 2023-11-06T22:00:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.764+0000 7f120f3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12080cab20 con 0x7f1208153ed0 2023-11-06T22:00:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.764+0000 7f120f3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12080c6000 con 0x7f12081542b0 2023-11-06T22:00:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.764+0000 7f120d19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1208153ed0 0x7f1208158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.764+0000 7f120d19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1208153ed0 0x7f1208158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46330/0 (socket says 172.21.15.33:46330) 2023-11-06T22:00:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.764+0000 7f120d19b700 1 -- 172.21.15.33:0/358607052 learned_addr learned my addr 172.21.15.33:0/358607052 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.765+0000 7f120d99c700 1 --2- 172.21.15.33:0/358607052 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f12081542b0 0x7f12080c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.765+0000 7f120d19b700 1 -- 172.21.15.33:0/358607052 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f12081542b0 msgr2=0x7f12080c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.765+0000 7f120d19b700 1 --2- 172.21.15.33:0/358607052 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f12081542b0 0x7f12080c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.765+0000 7f120d19b700 1 -- 172.21.15.33:0/358607052 >> v1:172.21.15.69:6789/0 conn(0x7f12080c8ee0 legacy=0x7f12080ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.765+0000 7f120d19b700 1 -- 172.21.15.33:0/358607052 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080c9370 con 0x7f1208153ed0 2023-11-06T22:00:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.765+0000 7f120d19b700 1 --2- 172.21.15.33:0/358607052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1208153ed0 0x7f1208158600 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f1204006fd0 tx=0x7f12040090f0).ready entity=mon.0 client_cookie=17d1b21c30c26bd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.765+0000 7f11fffff700 1 -- 172.21.15.33:0/358607052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120400b580 con 0x7f1208153ed0 2023-11-06T22:00:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.765+0000 7f11fffff700 1 -- 172.21.15.33:0/358607052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120400b6e0 con 0x7f1208153ed0 2023-11-06T22:00:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.765+0000 7f11fffff700 1 -- 172.21.15.33:0/358607052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120400ba60 con 0x7f1208153ed0 2023-11-06T22:00:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.766+0000 7f120f3ff700 1 -- 172.21.15.33:0/358607052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1208153ed0 msgr2=0x7f1208158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.766+0000 7f120f3ff700 1 --2- 172.21.15.33:0/358607052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1208153ed0 0x7f1208158600 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f1204006fd0 tx=0x7f12040090f0).stop 2023-11-06T22:00:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.766+0000 7f120f3ff700 1 -- 172.21.15.33:0/358607052 shutdown_connections 2023-11-06T22:00:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.766+0000 7f120f3ff700 1 --2- 172.21.15.33:0/358607052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1208153ed0 0x7f1208158600 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.766+0000 7f120f3ff700 1 --2- 172.21.15.33:0/358607052 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f12081542b0 0x7f12080c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.766+0000 7f120f3ff700 1 -- 172.21.15.33:0/358607052 >> 172.21.15.33:0/358607052 conn(0x7f120805e1b0 msgr2=0x7f120805e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.766+0000 7f120f3ff700 1 -- 172.21.15.33:0/358607052 shutdown_connections 2023-11-06T22:00:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.766+0000 7f120f3ff700 1 -- 172.21.15.33:0/358607052 wait complete. 2023-11-06T22:00:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.768+0000 7f120f3ff700 1 Processor -- start 2023-11-06T22:00:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.768+0000 7f120f3ff700 1 -- start start 2023-11-06T22:00:06.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120f3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1208153ed0 0x7f1208145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:06.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120f3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12081542b0 0x7f1208158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:06.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120f3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1208168160 0x7f1208156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:06.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120f3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12081595e0 con 0x7f12081542b0 2023-11-06T22:00:06.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120f3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12081598c0 con 0x7f1208168160 2023-11-06T22:00:06.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120f3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1208159ba0 con 0x7f1208153ed0 2023-11-06T22:00:06.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120c99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12081542b0 0x7f1208158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:06.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120c99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12081542b0 0x7f1208158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46338/0 (socket says 172.21.15.33:46338) 2023-11-06T22:00:06.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120c99a700 1 -- 172.21.15.33:0/2818069165 learned_addr learned my addr 172.21.15.33:0/2818069165 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:06.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120d19b700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1208153ed0 0x7f1208145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:06.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.769+0000 7f120d99c700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1208168160 0x7f1208156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:06.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f120c99a700 1 -- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1208153ed0 msgr2=0x7f1208145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:06.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f120c99a700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1208153ed0 0x7f1208145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:06.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f120c99a700 1 -- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1208168160 msgr2=0x7f1208156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f120c99a700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1208168160 0x7f1208156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f120c99a700 1 -- 172.21.15.33:0/2818069165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1204009370 con 0x7f12081542b0 2023-11-06T22:00:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f120c99a700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12081542b0 0x7f1208158b70 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f11f800c4c0 tx=0x7f11f8007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f11fd7fa700 1 -- 172.21.15.33:0/2818069165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f800ce70 con 0x7f12081542b0 2023-11-06T22:00:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f11fd7fa700 1 -- 172.21.15.33:0/2818069165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f8001e90 con 0x7f12081542b0 2023-11-06T22:00:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f11fd7fa700 1 -- 172.21.15.33:0/2818069165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8007250 con 0x7f12081542b0 2023-11-06T22:00:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f120f3ff700 1 -- 172.21.15.33:0/2818069165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12080c3010 con 0x7f12081542b0 2023-11-06T22:00:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.770+0000 7f120f3ff700 1 -- 172.21.15.33:0/2818069165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1208159060 con 0x7f12081542b0 2023-11-06T22:00:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.771+0000 7f120f3ff700 1 -- 172.21.15.33:0/2818069165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12080c3200 con 0x7f12081542b0 2023-11-06T22:00:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.772+0000 7f11fd7fa700 1 -- 172.21.15.33:0/2818069165 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f11f80066a0 con 0x7f12081542b0 2023-11-06T22:00:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.773+0000 7f11fd7fa700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11ec05f270 0x7f11ec061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.773+0000 7f11fd7fa700 1 -- 172.21.15.33:0/2818069165 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f11f8079ce0 con 0x7f12081542b0 2023-11-06T22:00:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.774+0000 7f120d19b700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11ec05f270 0x7f11ec061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.774+0000 7f120d19b700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11ec05f270 0x7f11ec061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f1204009fd0 tx=0x7f1204017000).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.777+0000 7f11fd7fa700 1 -- 172.21.15.33:0/2818069165 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f11f807e070 con 0x7f12081542b0 2023-11-06T22:00:06.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:06.956+0000 7f120f3ff700 1 -- 172.21.15.33:0/2818069165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f12080c90e0 con 0x7f12081542b0 2023-11-06T22:00:07.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.533+0000 7f11fd7fa700 1 -- 172.21.15.33:0/2818069165 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v176) v1 ==== 169+0+0 (secure 0 0 0) 0x7f11f804e7c0 con 0x7f12081542b0 2023-11-06T22:00:07.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.537+0000 7f120f3ff700 1 -- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11ec05f270 msgr2=0x7f11ec061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:07.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.537+0000 7f120f3ff700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11ec05f270 0x7f11ec061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f1204009fd0 tx=0x7f1204017000).stop 2023-11-06T22:00:07.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.537+0000 7f120f3ff700 1 -- 172.21.15.33:0/2818069165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12081542b0 msgr2=0x7f1208158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:07.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.537+0000 7f120f3ff700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12081542b0 0x7f1208158b70 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f11f800c4c0 tx=0x7f11f8007e00).stop 2023-11-06T22:00:07.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.537+0000 7f120f3ff700 1 -- 172.21.15.33:0/2818069165 shutdown_connections 2023-11-06T22:00:07.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.537+0000 7f120f3ff700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f11ec05f270 0x7f11ec061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.537+0000 7f120f3ff700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1208153ed0 0x7f1208145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.538+0000 7f120f3ff700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1208168160 0x7f1208156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.538+0000 7f120f3ff700 1 --2- 172.21.15.33:0/2818069165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12081542b0 0x7f1208158b70 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.538+0000 7f120f3ff700 1 -- 172.21.15.33:0/2818069165 >> 172.21.15.33:0/2818069165 conn(0x7f120805e1b0 msgr2=0x7f12080abcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:07.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.538+0000 7f120f3ff700 1 -- 172.21.15.33:0/2818069165 shutdown_connections 2023-11-06T22:00:07.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.538+0000 7f120f3ff700 1 -- 172.21.15.33:0/2818069165 wait complete. 2023-11-06T22:00:07.541 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T22:00:07.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T22:00:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.763+0000 7fb6fb19d700 1 Processor -- start 2023-11-06T22:00:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.763+0000 7fb6fb19d700 1 -- start start 2023-11-06T22:00:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.764+0000 7fb6fb19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6f4064bb0 0x7fb6f40c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.764+0000 7fb6fb19d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb6f40aade0 0x7fb6f40c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:07.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.764+0000 7fb6fb19d700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f40c0680 con 0x7fb6f41542c0 2023-11-06T22:00:07.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.764+0000 7fb6fb19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f40c1b20 con 0x7fb6f4064bb0 2023-11-06T22:00:07.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.764+0000 7fb6fb19d700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f40bfd90 con 0x7fb6f40aade0 2023-11-06T22:00:07.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.764+0000 7fb6f999a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6f4064bb0 0x7fb6f40c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:07.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.764+0000 7fb6f999a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6f4064bb0 0x7fb6f40c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46346/0 (socket says 172.21.15.33:46346) 2023-11-06T22:00:07.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.764+0000 7fb6f999a700 1 -- 172.21.15.33:0/556186644 learned_addr learned my addr 172.21.15.33:0/556186644 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:07.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.765+0000 7fb6fa99c700 1 --2- 172.21.15.33:0/556186644 >> v2:172.21.15.69:3300/0 conn(0x7fb6f40aade0 0x7fb6f40c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:07.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.765+0000 7fb6f999a700 1 -- 172.21.15.33:0/556186644 >> v2:172.21.15.69:3300/0 conn(0x7fb6f40aade0 msgr2=0x7fb6f40c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:07.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.765+0000 7fb6f999a700 1 --2- 172.21.15.33:0/556186644 >> v2:172.21.15.69:3300/0 conn(0x7fb6f40aade0 0x7fb6f40c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.765+0000 7fb6f999a700 1 -- 172.21.15.33:0/556186644 >> v1:172.21.15.69:6789/0 conn(0x7fb6f41542c0 legacy=0x7fb6f40644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:07.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.765+0000 7fb6f999a700 1 -- 172.21.15.33:0/556186644 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40c67f0 con 0x7fb6f4064bb0 2023-11-06T22:00:07.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.765+0000 7fb6fa99c700 1 --2- 172.21.15.33:0/556186644 >> v2:172.21.15.69:3300/0 conn(0x7fb6f40aade0 0x7fb6f40c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:07.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.765+0000 7fb6f999a700 1 --2- 172.21.15.33:0/556186644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 0x7fb6f40c5b50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7fb6f40aa720 tx=0x7fb6e40090f0).ready entity=mon.0 client_cookie=c0eea92a6590ba87 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:07.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.765+0000 7fb6f9199700 1 -- 172.21.15.33:0/556186644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400b600 con 0x7fb6f4064bb0 2023-11-06T22:00:07.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.765+0000 7fb6f9199700 1 -- 172.21.15.33:0/556186644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e400b760 con 0x7fb6f4064bb0 2023-11-06T22:00:07.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.765+0000 7fb6f9199700 1 -- 172.21.15.33:0/556186644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400bae0 con 0x7fb6f4064bb0 2023-11-06T22:00:07.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.766+0000 7fb6fb19d700 1 -- 172.21.15.33:0/556186644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 msgr2=0x7fb6f40c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:07.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.766+0000 7fb6fb19d700 1 --2- 172.21.15.33:0/556186644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 0x7fb6f40c5b50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7fb6f40aa720 tx=0x7fb6e40090f0).stop 2023-11-06T22:00:07.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.766+0000 7fb6fb19d700 1 -- 172.21.15.33:0/556186644 shutdown_connections 2023-11-06T22:00:07.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.766+0000 7fb6fb19d700 1 --2- 172.21.15.33:0/556186644 >> v2:172.21.15.69:3300/0 conn(0x7fb6f40aade0 0x7fb6f40c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.766+0000 7fb6fb19d700 1 --2- 172.21.15.33:0/556186644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 0x7fb6f40c5b50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.766+0000 7fb6fb19d700 1 -- 172.21.15.33:0/556186644 >> 172.21.15.33:0/556186644 conn(0x7fb6f40cb760 msgr2=0x7fb6f40c7730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:07.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.766+0000 7fb6fb19d700 1 -- 172.21.15.33:0/556186644 shutdown_connections 2023-11-06T22:00:07.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.766+0000 7fb6fb19d700 1 -- 172.21.15.33:0/556186644 wait complete. 2023-11-06T22:00:07.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.768+0000 7fb6fb19d700 1 Processor -- start 2023-11-06T22:00:07.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.768+0000 7fb6fb19d700 1 -- start start 2023-11-06T22:00:07.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fb19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 0x7fb6f4150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:07.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fb19d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6f40aade0 0x7fb6f40c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:07.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fb19d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f41542c0 0x7fb6f4164690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:07.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fb19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4150c00 con 0x7fb6f4064bb0 2023-11-06T22:00:07.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fb19d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40c4970 con 0x7fb6f41542c0 2023-11-06T22:00:07.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fb19d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4150df0 con 0x7fb6f40aade0 2023-11-06T22:00:07.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fa19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 0x7fb6f4150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:07.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fa19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 0x7fb6f4150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46354/0 (socket says 172.21.15.33:46354) 2023-11-06T22:00:07.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fa19b700 1 -- 172.21.15.33:0/3879563031 learned_addr learned my addr 172.21.15.33:0/3879563031 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:07.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fa99c700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f41542c0 0x7fb6f4164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:07.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6f999a700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6f40aade0 0x7fb6f40c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:07.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fa19b700 1 -- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6f40aade0 msgr2=0x7fb6f40c4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:07.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fa19b700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6f40aade0 0x7fb6f40c4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fa19b700 1 -- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f41542c0 msgr2=0x7fb6f4164690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:07.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fa19b700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f41542c0 0x7fb6f4164690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.769+0000 7fb6fa19b700 1 -- 172.21.15.33:0/3879563031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e4009370 con 0x7fb6f4064bb0 2023-11-06T22:00:07.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.770+0000 7fb6fa19b700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 0x7fb6f4150710 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7fb6f000c440 tx=0x7fb6f0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:07.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.770+0000 7fb6ea7fc700 1 -- 172.21.15.33:0/3879563031 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0007020 con 0x7fb6f4064bb0 2023-11-06T22:00:07.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.770+0000 7fb6ea7fc700 1 -- 172.21.15.33:0/3879563031 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f0007180 con 0x7fb6f4064bb0 2023-11-06T22:00:07.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.770+0000 7fb6fb19d700 1 -- 172.21.15.33:0/3879563031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f41510d0 con 0x7fb6f4064bb0 2023-11-06T22:00:07.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.770+0000 7fb6ea7fc700 1 -- 172.21.15.33:0/3879563031 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0007540 con 0x7fb6f4064bb0 2023-11-06T22:00:07.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.770+0000 7fb6fb19d700 1 -- 172.21.15.33:0/3879563031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f4151970 con 0x7fb6f4064bb0 2023-11-06T22:00:07.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.771+0000 7fb6fb19d700 1 -- 172.21.15.33:0/3879563031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f4151ce0 con 0x7fb6f4064bb0 2023-11-06T22:00:07.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.772+0000 7fb6ea7fc700 1 -- 172.21.15.33:0/3879563031 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb6f00068b0 con 0x7fb6f4064bb0 2023-11-06T22:00:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.773+0000 7fb6ea7fc700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6d805f270 0x7fb6d8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.773+0000 7fb6ea7fc700 1 -- 172.21.15.33:0/3879563031 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fb6f0079450 con 0x7fb6f4064bb0 2023-11-06T22:00:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.773+0000 7fb6f999a700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6d805f270 0x7fb6d8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.774+0000 7fb6f999a700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6d805f270 0x7fb6d8061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb6f4064fb0 tx=0x7fb6e4017000).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.777+0000 7fb6ea7fc700 1 -- 172.21.15.33:0/3879563031 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb6f0047af0 con 0x7fb6f4064bb0 2023-11-06T22:00:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.957+0000 7fb6fb19d700 1 -- 172.21.15.33:0/3879563031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fb6f406ad80 con 0x7fb6f4064bb0 2023-11-06T22:00:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.963+0000 7fb6ea7fc700 1 -- 172.21.15.33:0/3879563031 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v32) v1 ==== 120+0+0 (secure 0 0 0) 0x7fb6f004a600 con 0x7fb6f4064bb0 2023-11-06T22:00:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.965+0000 7fb6fb19d700 1 -- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6d805f270 msgr2=0x7fb6d8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.965+0000 7fb6fb19d700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6d805f270 0x7fb6d8061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb6f4064fb0 tx=0x7fb6e4017000).stop 2023-11-06T22:00:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.965+0000 7fb6fb19d700 1 -- 172.21.15.33:0/3879563031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 msgr2=0x7fb6f4150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.965+0000 7fb6fb19d700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 0x7fb6f4150710 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7fb6f000c440 tx=0x7fb6f0007e00).stop 2023-11-06T22:00:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.966+0000 7fb6fb19d700 1 -- 172.21.15.33:0/3879563031 shutdown_connections 2023-11-06T22:00:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.966+0000 7fb6fb19d700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb6d805f270 0x7fb6d8061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.966+0000 7fb6fb19d700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f41542c0 0x7fb6f4164690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.966+0000 7fb6fb19d700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6f4064bb0 0x7fb6f4150710 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.966+0000 7fb6fb19d700 1 --2- 172.21.15.33:0/3879563031 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6f40aade0 0x7fb6f40c4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.966+0000 7fb6fb19d700 1 -- 172.21.15.33:0/3879563031 >> 172.21.15.33:0/3879563031 conn(0x7fb6f40cb760 msgr2=0x7fb6f414f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.966+0000 7fb6fb19d700 1 -- 172.21.15.33:0/3879563031 shutdown_connections 2023-11-06T22:00:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:07.966+0000 7fb6fb19d700 1 -- 172.21.15.33:0/3879563031 wait complete. 2023-11-06T22:00:07.985 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T22:00:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.184+0000 7fc029fff700 1 Processor -- start 2023-11-06T22:00:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.184+0000 7fc029fff700 1 -- start start 2023-11-06T22:00:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.185+0000 7fc029fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024157700 0x7fc0240c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.185+0000 7fc029fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc024063840 0x7fc0240c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.185+0000 7fc029fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc024158c00 con 0x7fc0240aaf50 2023-11-06T22:00:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.185+0000 7fc029fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0240c18f0 con 0x7fc024063840 2023-11-06T22:00:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.185+0000 7fc029fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc024157ff0 con 0x7fc024157700 2023-11-06T22:00:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.186+0000 7fc023fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc024063840 0x7fc0240c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.186+0000 7fc023fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc024063840 0x7fc0240c5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58316/0 (socket says 172.21.15.33:58316) 2023-11-06T22:00:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.186+0000 7fc023fff700 1 -- 172.21.15.33:0/3978386848 learned_addr learned my addr 172.21.15.33:0/3978386848 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.186+0000 7fc0237fe700 1 --2- 172.21.15.33:0/3978386848 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024157700 0x7fc0240c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.186+0000 7fc023fff700 1 -- 172.21.15.33:0/3978386848 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024157700 msgr2=0x7fc0240c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.186+0000 7fc023fff700 1 --2- 172.21.15.33:0/3978386848 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024157700 0x7fc0240c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.186+0000 7fc023fff700 1 -- 172.21.15.33:0/3978386848 >> v1:172.21.15.69:6789/0 conn(0x7fc0240aaf50 legacy=0x7fc024063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.186+0000 7fc023fff700 1 -- 172.21.15.33:0/3978386848 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0241582d0 con 0x7fc024063840 2023-11-06T22:00:08.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.186+0000 7fc023fff700 1 --2- 172.21.15.33:0/3978386848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc024063840 0x7fc0240c5d70 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7fc01c009f20 tx=0x7fc01c009840).ready entity=mon.0 client_cookie=b0f5909c4ffea572 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc0227fc700 1 -- 172.21.15.33:0/3978386848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00e070 con 0x7fc024063840 2023-11-06T22:00:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc0227fc700 1 -- 172.21.15.33:0/3978386848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c00ba80 con 0x7fc024063840 2023-11-06T22:00:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc0227fc700 1 -- 172.21.15.33:0/3978386848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00be00 con 0x7fc024063840 2023-11-06T22:00:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc029fff700 1 -- 172.21.15.33:0/3978386848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc024063840 msgr2=0x7fc0240c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc029fff700 1 --2- 172.21.15.33:0/3978386848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc024063840 0x7fc0240c5d70 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7fc01c009f20 tx=0x7fc01c009840).stop 2023-11-06T22:00:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc029fff700 1 -- 172.21.15.33:0/3978386848 shutdown_connections 2023-11-06T22:00:08.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc029fff700 1 --2- 172.21.15.33:0/3978386848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc024063840 0x7fc0240c5d70 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc029fff700 1 --2- 172.21.15.33:0/3978386848 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024157700 0x7fc0240c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc029fff700 1 -- 172.21.15.33:0/3978386848 >> 172.21.15.33:0/3978386848 conn(0x7fc02405ec80 msgr2=0x7fc0240cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:08.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc029fff700 1 -- 172.21.15.33:0/3978386848 shutdown_connections 2023-11-06T22:00:08.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.187+0000 7fc029fff700 1 -- 172.21.15.33:0/3978386848 wait complete. 2023-11-06T22:00:08.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.189+0000 7fc029fff700 1 Processor -- start 2023-11-06T22:00:08.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.189+0000 7fc029fff700 1 -- start start 2023-11-06T22:00:08.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc029fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024063840 0x7fc024147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc029fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0240aaf50 0x7fc0240c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc029fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024157700 0x7fc0240adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc029fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240afef0 con 0x7fc0240aaf50 2023-11-06T22:00:08.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc029fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240b01d0 con 0x7fc024157700 2023-11-06T22:00:08.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc029fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240c2090 con 0x7fc024063840 2023-11-06T22:00:08.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc0237fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024063840 0x7fc024147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc0237fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024063840 0x7fc024147ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:54562/0 (socket says 172.21.15.33:54562) 2023-11-06T22:00:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc0237fe700 1 -- 172.21.15.33:0/1740074714 learned_addr learned my addr 172.21.15.33:0/1740074714 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc023fff700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024157700 0x7fc0240adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.190+0000 7fc022ffd700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0240aaf50 0x7fc0240c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc0237fe700 1 -- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024157700 msgr2=0x7fc0240adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:08.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc0237fe700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024157700 0x7fc0240adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc0237fe700 1 -- 172.21.15.33:0/1740074714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0240aaf50 msgr2=0x7fc0240c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:08.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc0237fe700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0240aaf50 0x7fc0240c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc0237fe700 1 -- 172.21.15.33:0/1740074714 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c009ae0 con 0x7fc024063840 2023-11-06T22:00:08.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc022ffd700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0240aaf50 0x7fc0240c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:00:08.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc0237fe700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024063840 0x7fc024147ac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fc01400d890 tx=0x7fc014008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:08.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc00b7fe700 1 -- 172.21.15.33:0/1740074714 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014011590 con 0x7fc024063840 2023-11-06T22:00:08.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc00b7fe700 1 -- 172.21.15.33:0/1740074714 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0140116f0 con 0x7fc024063840 2023-11-06T22:00:08.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc00b7fe700 1 -- 172.21.15.33:0/1740074714 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014011a70 con 0x7fc024063840 2023-11-06T22:00:08.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.191+0000 7fc029fff700 1 -- 172.21.15.33:0/1740074714 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0240c2370 con 0x7fc024063840 2023-11-06T22:00:08.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.192+0000 7fc029fff700 1 -- 172.21.15.33:0/1740074714 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc024147fb0 con 0x7fc024063840 2023-11-06T22:00:08.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.193+0000 7fc029fff700 1 -- 172.21.15.33:0/1740074714 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0240c26e0 con 0x7fc024063840 2023-11-06T22:00:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.194+0000 7fc00b7fe700 1 -- 172.21.15.33:0/1740074714 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc01401c460 con 0x7fc024063840 2023-11-06T22:00:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.194+0000 7fc00b7fe700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc01005f270 0x7fc010061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.194+0000 7fc00b7fe700 1 -- 172.21.15.33:0/1740074714 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fc01407a080 con 0x7fc024063840 2023-11-06T22:00:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.195+0000 7fc022ffd700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc01005f270 0x7fc010061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.195+0000 7fc022ffd700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc01005f270 0x7fc010061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fc024157ec0 tx=0x7fc018009040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:08.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.200+0000 7fc00b7fe700 1 -- 172.21.15.33:0/1740074714 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc0140a8af0 con 0x7fc024063840 2023-11-06T22:00:08.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.383+0000 7fc029fff700 1 -- 172.21.15.33:0/1740074714 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc0240c1ed0 con 0x7fc024063840 2023-11-06T22:00:08.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.388+0000 7fc00b7fe700 1 -- 172.21.15.33:0/1740074714 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc01404b1d0 con 0x7fc024063840 2023-11-06T22:00:08.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.390+0000 7fc029fff700 1 -- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc01005f270 msgr2=0x7fc010061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:08.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.390+0000 7fc029fff700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc01005f270 0x7fc010061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fc024157ec0 tx=0x7fc018009040).stop 2023-11-06T22:00:08.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.390+0000 7fc029fff700 1 -- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024063840 msgr2=0x7fc024147ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:08.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.390+0000 7fc029fff700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024063840 0x7fc024147ac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fc01400d890 tx=0x7fc014008040).stop 2023-11-06T22:00:08.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.391+0000 7fc029fff700 1 -- 172.21.15.33:0/1740074714 shutdown_connections 2023-11-06T22:00:08.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.391+0000 7fc029fff700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc01005f270 0x7fc010061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.391+0000 7fc029fff700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc024063840 0x7fc024147ac0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.391+0000 7fc029fff700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc024157700 0x7fc0240adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.391+0000 7fc029fff700 1 --2- 172.21.15.33:0/1740074714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0240aaf50 0x7fc0240c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.391+0000 7fc029fff700 1 -- 172.21.15.33:0/1740074714 >> 172.21.15.33:0/1740074714 conn(0x7fc02405ec80 msgr2=0x7fc0240c6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:08.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.391+0000 7fc029fff700 1 -- 172.21.15.33:0/1740074714 shutdown_connections 2023-11-06T22:00:08.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.391+0000 7fc029fff700 1 -- 172.21.15.33:0/1740074714 wait complete. 2023-11-06T22:00:08.409 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-11-06T22:00:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.606+0000 7f39481ff700 1 Processor -- start 2023-11-06T22:00:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.606+0000 7f39481ff700 1 -- start start 2023-11-06T22:00:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.608+0000 7f39481ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3940063140 0x7f3940063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.608+0000 7f39481ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f39400c3680 0x7f3940154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.608+0000 7f39481ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39400c31f0 con 0x7f394005e1b0 2023-11-06T22:00:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.608+0000 7f39481ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394006a0f0 con 0x7f39400c3680 2023-11-06T22:00:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.608+0000 7f39481ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39400c2920 con 0x7f3940063140 2023-11-06T22:00:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.608+0000 7f394679c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f39400c3680 0x7f3940154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.608+0000 7f394679c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f39400c3680 0x7f3940154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58344/0 (socket says 172.21.15.33:58344) 2023-11-06T22:00:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.608+0000 7f394679c700 1 -- 172.21.15.33:0/1584955484 learned_addr learned my addr 172.21.15.33:0/1584955484 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.608+0000 7f3945f9b700 1 --2- 172.21.15.33:0/1584955484 >> v2:172.21.15.69:3300/0 conn(0x7f3940063140 0x7f3940063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.608+0000 7f394679c700 1 -- 172.21.15.33:0/1584955484 >> v2:172.21.15.69:3300/0 conn(0x7f3940063140 msgr2=0x7f3940063520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.609+0000 7f394679c700 1 --2- 172.21.15.33:0/1584955484 >> v2:172.21.15.69:3300/0 conn(0x7f3940063140 0x7f3940063520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.609+0000 7f394679c700 1 -- 172.21.15.33:0/1584955484 >> v1:172.21.15.69:6789/0 conn(0x7f394005e1b0 legacy=0x7f39400c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.609+0000 7f394679c700 1 -- 172.21.15.33:0/1584955484 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394005e640 con 0x7f39400c3680 2023-11-06T22:00:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.609+0000 7f394679c700 1 --2- 172.21.15.33:0/1584955484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 0x7f3940154050 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f3940068530 tx=0x7f3938009840).ready entity=mon.0 client_cookie=b5e210b1c1e6b136 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.609+0000 7f3944f99700 1 -- 172.21.15.33:0/1584955484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800e070 con 0x7f39400c3680 2023-11-06T22:00:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.609+0000 7f3944f99700 1 -- 172.21.15.33:0/1584955484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393800ba10 con 0x7f39400c3680 2023-11-06T22:00:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.609+0000 7f3944f99700 1 -- 172.21.15.33:0/1584955484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800bd90 con 0x7f39400c3680 2023-11-06T22:00:08.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.609+0000 7f39481ff700 1 -- 172.21.15.33:0/1584955484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 msgr2=0x7f3940154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:08.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.609+0000 7f39481ff700 1 --2- 172.21.15.33:0/1584955484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 0x7f3940154050 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f3940068530 tx=0x7f3938009840).stop 2023-11-06T22:00:08.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.610+0000 7f39481ff700 1 -- 172.21.15.33:0/1584955484 shutdown_connections 2023-11-06T22:00:08.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.610+0000 7f39481ff700 1 --2- 172.21.15.33:0/1584955484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 0x7f3940154050 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.610+0000 7f39481ff700 1 --2- 172.21.15.33:0/1584955484 >> v2:172.21.15.69:3300/0 conn(0x7f3940063140 0x7f3940063520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.610+0000 7f39481ff700 1 -- 172.21.15.33:0/1584955484 >> 172.21.15.33:0/1584955484 conn(0x7f39400c7d60 msgr2=0x7f39400c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.610+0000 7f39481ff700 1 -- 172.21.15.33:0/1584955484 shutdown_connections 2023-11-06T22:00:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.610+0000 7f39481ff700 1 -- 172.21.15.33:0/1584955484 wait complete. 2023-11-06T22:00:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.611+0000 7f39481ff700 1 Processor -- start 2023-11-06T22:00:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.611+0000 7f39481ff700 1 -- start start 2023-11-06T22:00:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.612+0000 7f39481ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3940063140 0x7f3940138f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.612+0000 7f39481ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 0x7f394014bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.612+0000 7f39481ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f394014c150 0x7f3940167870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.612+0000 7f39481ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394014d430 con 0x7f39400c3680 2023-11-06T22:00:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.612+0000 7f39481ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394014c5d0 con 0x7f394014c150 2023-11-06T22:00:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.612+0000 7f39481ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394014c820 con 0x7f3940063140 2023-11-06T22:00:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f394579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 0x7f394014bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f394579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 0x7f394014bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58358/0 (socket says 172.21.15.33:58358) 2023-11-06T22:00:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f394579a700 1 -- 172.21.15.33:0/1232935485 learned_addr learned my addr 172.21.15.33:0/1232935485 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f3945f9b700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3940063140 0x7f3940138f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f394679c700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f394014c150 0x7f3940167870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f394579a700 1 -- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3940063140 msgr2=0x7f3940138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f394579a700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3940063140 0x7f3940138f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f394579a700 1 -- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f394014c150 msgr2=0x7f3940167870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f394579a700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f394014c150 0x7f3940167870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f394579a700 1 -- 172.21.15.33:0/1232935485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3938009ae0 con 0x7f39400c3680 2023-11-06T22:00:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f394579a700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 0x7f394014bc60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f393000b4c0 tx=0x7f3930006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.613+0000 7f3935ffb700 1 -- 172.21.15.33:0/1232935485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930012410 con 0x7f39400c3680 2023-11-06T22:00:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.614+0000 7f3935ffb700 1 -- 172.21.15.33:0/1232935485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3930012590 con 0x7f39400c3680 2023-11-06T22:00:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.614+0000 7f3935ffb700 1 -- 172.21.15.33:0/1232935485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930012950 con 0x7f39400c3680 2023-11-06T22:00:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.614+0000 7f39481ff700 1 -- 172.21.15.33:0/1232935485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394014cb00 con 0x7f39400c3680 2023-11-06T22:00:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.614+0000 7f39481ff700 1 -- 172.21.15.33:0/1232935485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394014d0c0 con 0x7f39400c3680 2023-11-06T22:00:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.615+0000 7f39481ff700 1 -- 172.21.15.33:0/1232935485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394014d5e0 con 0x7f39400c3680 2023-11-06T22:00:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.616+0000 7f3935ffb700 1 -- 172.21.15.33:0/1232935485 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3930012d40 con 0x7f39400c3680 2023-11-06T22:00:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.616+0000 7f3935ffb700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f392405f270 0x7f3924061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.616+0000 7f3935ffb700 1 -- 172.21.15.33:0/1232935485 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f393007a180 con 0x7f39400c3680 2023-11-06T22:00:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.617+0000 7f3945f9b700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f392405f270 0x7f3924061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.618+0000 7f3945f9b700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f392405f270 0x7f3924061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f393c00bb20 tx=0x7f393c008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.621+0000 7f3935ffb700 1 -- 172.21.15.33:0/1232935485 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f394014d5e0 con 0x7f39400c3680 2023-11-06T22:00:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:08.870+0000 7f39481ff700 1 -- 172.21.15.33:0/1232935485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f3940064420 con 0x7f39400c3680 2023-11-06T22:00:09.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.540+0000 7f3935ffb700 1 -- 172.21.15.33:0/1232935485 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2456) v1 ==== 174+0+0 (secure 0 0 0) 0x7f393004b330 con 0x7f39400c3680 2023-11-06T22:00:09.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.542+0000 7f39481ff700 1 -- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f392405f270 msgr2=0x7f3924061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:09.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.542+0000 7f39481ff700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f392405f270 0x7f3924061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f393c00bb20 tx=0x7f393c008040).stop 2023-11-06T22:00:09.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.543+0000 7f39481ff700 1 -- 172.21.15.33:0/1232935485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 msgr2=0x7f394014bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:09.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.543+0000 7f39481ff700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 0x7f394014bc60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f393000b4c0 tx=0x7f3930006ef0).stop 2023-11-06T22:00:09.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.543+0000 7f39481ff700 1 -- 172.21.15.33:0/1232935485 shutdown_connections 2023-11-06T22:00:09.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.543+0000 7f39481ff700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f392405f270 0x7f3924061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:09.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.543+0000 7f39481ff700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3940063140 0x7f3940138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.543+0000 7f39481ff700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f394014c150 0x7f3940167870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.543+0000 7f39481ff700 1 --2- 172.21.15.33:0/1232935485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400c3680 0x7f394014bc60 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.543+0000 7f39481ff700 1 -- 172.21.15.33:0/1232935485 >> 172.21.15.33:0/1232935485 conn(0x7f39400c7d60 msgr2=0x7f39401518a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.543+0000 7f39481ff700 1 -- 172.21.15.33:0/1232935485 shutdown_connections 2023-11-06T22:00:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.543+0000 7f39481ff700 1 -- 172.21.15.33:0/1232935485 wait complete. 2023-11-06T22:00:09.570 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-11-06T22:00:09.571 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-11-06T22:00:09.571 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-06T22:00:09.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-11-06T22:00:09.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.771+0000 7f7b673ff700 1 Processor -- start 2023-11-06T22:00:09.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.771+0000 7f7b673ff700 1 -- start start 2023-11-06T22:00:09.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b673ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b60064ce0 0x7f7b601544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:09.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b673ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7b600aade0 0x7f7b600c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:09.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b673ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b600c0880 con 0x7f7b60154110 2023-11-06T22:00:09.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b673ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b600bfd40 con 0x7f7b60064ce0 2023-11-06T22:00:09.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b673ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b600c8260 con 0x7f7b600aade0 2023-11-06T22:00:09.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b6499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b60064ce0 0x7f7b601544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:09.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b6499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b60064ce0 0x7f7b601544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58364/0 (socket says 172.21.15.33:58364) 2023-11-06T22:00:09.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b6499a700 1 -- 172.21.15.33:0/3954681277 learned_addr learned my addr 172.21.15.33:0/3954681277 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:09.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b6599c700 1 --2- 172.21.15.33:0/3954681277 >> v2:172.21.15.69:3300/0 conn(0x7f7b600aade0 0x7f7b600c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:09.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b6499a700 1 -- 172.21.15.33:0/3954681277 >> v2:172.21.15.69:3300/0 conn(0x7f7b600aade0 msgr2=0x7f7b600c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:09.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b6499a700 1 --2- 172.21.15.33:0/3954681277 >> v2:172.21.15.69:3300/0 conn(0x7f7b600aade0 0x7f7b600c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:09.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b6499a700 1 -- 172.21.15.33:0/3954681277 >> v1:172.21.15.69:6789/0 conn(0x7f7b60154110 legacy=0x7f7b600645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:09.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b6499a700 1 -- 172.21.15.33:0/3954681277 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b600bff50 con 0x7f7b60064ce0 2023-11-06T22:00:09.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.773+0000 7f7b6599c700 1 --2- 172.21.15.33:0/3954681277 >> v2:172.21.15.69:3300/0 conn(0x7f7b600aade0 0x7f7b600c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:00:09.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.774+0000 7f7b6499a700 1 --2- 172.21.15.33:0/3954681277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b60064ce0 0x7f7b601544f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f7b600654d0 tx=0x7f7b500090f0).ready entity=mon.0 client_cookie=1508916b76e462cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:09.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.774+0000 7f7b57fff700 1 -- 172.21.15.33:0/3954681277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000b5a0 con 0x7f7b60064ce0 2023-11-06T22:00:09.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.774+0000 7f7b57fff700 1 -- 172.21.15.33:0/3954681277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5000b700 con 0x7f7b60064ce0 2023-11-06T22:00:09.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.774+0000 7f7b57fff700 1 -- 172.21.15.33:0/3954681277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000ba80 con 0x7f7b60064ce0 2023-11-06T22:00:09.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.774+0000 7f7b673ff700 1 -- 172.21.15.33:0/3954681277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b60064ce0 msgr2=0x7f7b601544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:09.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.774+0000 7f7b673ff700 1 --2- 172.21.15.33:0/3954681277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b60064ce0 0x7f7b601544f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f7b600654d0 tx=0x7f7b500090f0).stop 2023-11-06T22:00:09.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.774+0000 7f7b673ff700 1 -- 172.21.15.33:0/3954681277 shutdown_connections 2023-11-06T22:00:09.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.774+0000 7f7b673ff700 1 --2- 172.21.15.33:0/3954681277 >> v2:172.21.15.69:3300/0 conn(0x7f7b600aade0 0x7f7b600c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:09.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.774+0000 7f7b673ff700 1 --2- 172.21.15.33:0/3954681277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b60064ce0 0x7f7b601544f0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:09.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.774+0000 7f7b673ff700 1 -- 172.21.15.33:0/3954681277 >> 172.21.15.33:0/3954681277 conn(0x7f7b600cb8b0 msgr2=0x7f7b600c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:09.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.775+0000 7f7b673ff700 1 -- 172.21.15.33:0/3954681277 shutdown_connections 2023-11-06T22:00:09.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.775+0000 7f7b673ff700 1 -- 172.21.15.33:0/3954681277 wait complete. 2023-11-06T22:00:09.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.776+0000 7f7b673ff700 1 Processor -- start 2023-11-06T22:00:09.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.776+0000 7f7b673ff700 1 -- start start 2023-11-06T22:00:09.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.777+0000 7f7b673ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b60064ce0 0x7f7b60164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:09.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.777+0000 7f7b673ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600aade0 0x7f7b601503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:09.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.777+0000 7f7b673ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60154110 0x7f7b6014eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:09.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.777+0000 7f7b673ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b60151770 con 0x7f7b600aade0 2023-11-06T22:00:09.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.777+0000 7f7b673ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b601508b0 con 0x7f7b60154110 2023-11-06T22:00:09.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.777+0000 7f7b673ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b60150b60 con 0x7f7b60064ce0 2023-11-06T22:00:09.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600aade0 0x7f7b601503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:09.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600aade0 0x7f7b601503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58368/0 (socket says 172.21.15.33:58368) 2023-11-06T22:00:09.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6499a700 1 -- 172.21.15.33:0/2093882043 learned_addr learned my addr 172.21.15.33:0/2093882043 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:09.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6599c700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60154110 0x7f7b6014eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:09.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6519b700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b60064ce0 0x7f7b60164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:09.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6499a700 1 -- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b60064ce0 msgr2=0x7f7b60164fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:09.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6499a700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b60064ce0 0x7f7b60164fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:09.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6499a700 1 -- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60154110 msgr2=0x7f7b6014eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:09.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6499a700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60154110 0x7f7b6014eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:09.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6499a700 1 -- 172.21.15.33:0/2093882043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b50009370 con 0x7f7b600aade0 2023-11-06T22:00:09.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6599c700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60154110 0x7f7b6014eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:09.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6519b700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b60064ce0 0x7f7b60164fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:09.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b6499a700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600aade0 0x7f7b601503c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f7b50009130 tx=0x7f7b50015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:09.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b557fa700 1 -- 172.21.15.33:0/2093882043 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000ba80 con 0x7f7b600aade0 2023-11-06T22:00:09.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b557fa700 1 -- 172.21.15.33:0/2093882043 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b50000d10 con 0x7f7b600aade0 2023-11-06T22:00:09.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b673ff700 1 -- 172.21.15.33:0/2093882043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b60150e40 con 0x7f7b600aade0 2023-11-06T22:00:09.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.778+0000 7f7b557fa700 1 -- 172.21.15.33:0/2093882043 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5001c670 con 0x7f7b600aade0 2023-11-06T22:00:09.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.779+0000 7f7b673ff700 1 -- 172.21.15.33:0/2093882043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b60151400 con 0x7f7b600aade0 2023-11-06T22:00:09.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.780+0000 7f7b673ff700 1 -- 172.21.15.33:0/2093882043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b601518e0 con 0x7f7b600aade0 2023-11-06T22:00:09.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.780+0000 7f7b557fa700 1 -- 172.21.15.33:0/2093882043 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7b5001c9e0 con 0x7f7b600aade0 2023-11-06T22:00:09.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.781+0000 7f7b557fa700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b4405f270 0x7f7b44061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:09.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.782+0000 7f7b557fa700 1 -- 172.21.15.33:0/2093882043 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f7b5007c210 con 0x7f7b600aade0 2023-11-06T22:00:09.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.782+0000 7f7b6519b700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b4405f270 0x7f7b44061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:09.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.783+0000 7f7b6519b700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b4405f270 0x7f7b44061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7b5c00b8b0 tx=0x7f7b5c008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:09.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:09.785+0000 7f7b557fa700 1 -- 172.21.15.33:0/2093882043 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7b5004d3c0 con 0x7f7b600aade0 2023-11-06T22:00:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.035+0000 7f7b673ff700 1 -- 172.21.15.33:0/2093882043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f7b6006c6c0 con 0x7f7b600aade0 2023-11-06T22:00:10.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.542+0000 7f7b557fa700 1 -- 172.21.15.33:0/2093882043 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2457) v1 ==== 168+0+0 (secure 0 0 0) 0x7f7b500509e0 con 0x7f7b600aade0 2023-11-06T22:00:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.544+0000 7f7b673ff700 1 -- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b4405f270 msgr2=0x7f7b44061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.544+0000 7f7b673ff700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b4405f270 0x7f7b44061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7b5c00b8b0 tx=0x7f7b5c008040).stop 2023-11-06T22:00:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.544+0000 7f7b673ff700 1 -- 172.21.15.33:0/2093882043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600aade0 msgr2=0x7f7b601503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.544+0000 7f7b673ff700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600aade0 0x7f7b601503c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f7b50009130 tx=0x7f7b50015000).stop 2023-11-06T22:00:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.544+0000 7f7b673ff700 1 -- 172.21.15.33:0/2093882043 shutdown_connections 2023-11-06T22:00:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.544+0000 7f7b673ff700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7b4405f270 0x7f7b44061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.544+0000 7f7b673ff700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7b60064ce0 0x7f7b60164fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.544+0000 7f7b673ff700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60154110 0x7f7b6014eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.544+0000 7f7b673ff700 1 --2- 172.21.15.33:0/2093882043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b600aade0 0x7f7b601503c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.544+0000 7f7b673ff700 1 -- 172.21.15.33:0/2093882043 >> 172.21.15.33:0/2093882043 conn(0x7f7b600cb8b0 msgr2=0x7f7b60062ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:10.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.545+0000 7f7b673ff700 1 -- 172.21.15.33:0/2093882043 shutdown_connections 2023-11-06T22:00:10.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.545+0000 7f7b673ff700 1 -- 172.21.15.33:0/2093882043 wait complete. 2023-11-06T22:00:10.565 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T22:00:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.764+0000 7fc7da5ff700 1 Processor -- start 2023-11-06T22:00:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.764+0000 7fc7da5ff700 1 -- start start 2023-11-06T22:00:10.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.765+0000 7fc7da5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7d41544f0 0x7fc7d40c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:10.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.766+0000 7fc7da5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc7d40c0c40 0x7fc7d40646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:10.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.766+0000 7fc7da5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d41556d0 con 0x7fc7d4064bb0 2023-11-06T22:00:10.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.766+0000 7fc7da5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d41548d0 con 0x7fc7d40c0c40 2023-11-06T22:00:10.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.766+0000 7fc7da5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4065030 con 0x7fc7d41544f0 2023-11-06T22:00:10.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.766+0000 7fc7d8b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc7d4064bb0 0x7fc7d4157f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48954/0 (socket says 172.21.15.33:48954) 2023-11-06T22:00:10.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.766+0000 7fc7d8b9c700 1 -- 172.21.15.33:0/2463116800 learned_addr learned my addr 172.21.15.33:0/2463116800 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:10.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.766+0000 7fc7d37fe700 1 --2- 172.21.15.33:0/2463116800 >> v2:172.21.15.69:3300/0 conn(0x7fc7d40c0c40 0x7fc7d40646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.766+0000 7fc7d3fff700 1 --2- 172.21.15.33:0/2463116800 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7d41544f0 0x7fc7d40c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.766+0000 7fc7d2ffd700 1 -- 172.21.15.33:0/2463116800 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191210931 0 0) 0x7fc7d41556d0 con 0x7fc7d4064bb0 2023-11-06T22:00:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.766+0000 7fc7d2ffd700 1 -- 172.21.15.33:0/2463116800 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7bc0033f0 con 0x7fc7d4064bb0 2023-11-06T22:00:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.767+0000 7fc7d37fe700 1 -- 172.21.15.33:0/2463116800 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7d41544f0 msgr2=0x7fc7d40c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.767+0000 7fc7d37fe700 1 --2- 172.21.15.33:0/2463116800 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7d41544f0 0x7fc7d40c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.767+0000 7fc7d37fe700 1 -- 172.21.15.33:0/2463116800 >> v1:172.21.15.33:6789/0 conn(0x7fc7d4064bb0 legacy=0x7fc7d4157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:10.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.767+0000 7fc7d37fe700 1 -- 172.21.15.33:0/2463116800 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d4154ac0 con 0x7fc7d40c0c40 2023-11-06T22:00:10.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.767+0000 7fc7d37fe700 1 --2- 172.21.15.33:0/2463116800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7d40c0c40 0x7fc7d40646c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fc7c8009760 tx=0x7fc7c8009060).ready entity=mon.1 client_cookie=5493dbf41c4ccde2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:10.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.767+0000 7fc7d2ffd700 1 -- 172.21.15.33:0/2463116800 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800d4a0 con 0x7fc7d40c0c40 2023-11-06T22:00:10.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.767+0000 7fc7d2ffd700 1 -- 172.21.15.33:0/2463116800 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c800d600 con 0x7fc7d40c0c40 2023-11-06T22:00:10.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.767+0000 7fc7d2ffd700 1 -- 172.21.15.33:0/2463116800 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800d980 con 0x7fc7d40c0c40 2023-11-06T22:00:10.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.768+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2463116800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7d40c0c40 msgr2=0x7fc7d40646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.768+0000 7fc7da5ff700 1 --2- 172.21.15.33:0/2463116800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7d40c0c40 0x7fc7d40646c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fc7c8009760 tx=0x7fc7c8009060).stop 2023-11-06T22:00:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.768+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2463116800 shutdown_connections 2023-11-06T22:00:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.768+0000 7fc7da5ff700 1 --2- 172.21.15.33:0/2463116800 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7d41544f0 0x7fc7d40c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.768+0000 7fc7da5ff700 1 --2- 172.21.15.33:0/2463116800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7d40c0c40 0x7fc7d40646c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.768+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2463116800 >> 172.21.15.33:0/2463116800 conn(0x7fc7d40cb900 msgr2=0x7fc7d40c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:10.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.768+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2463116800 shutdown_connections 2023-11-06T22:00:10.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.768+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2463116800 wait complete. 2023-11-06T22:00:10.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.770+0000 7fc7da5ff700 1 Processor -- start 2023-11-06T22:00:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.770+0000 7fc7da5ff700 1 -- start start 2023-11-06T22:00:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.771+0000 7fc7da5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7d4064bb0 0x7fc7d4149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.771+0000 7fc7da5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7d40c0c40 0x7fc7d40c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.771+0000 7fc7da5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d41544f0 0x7fc7d4156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7da5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40c2c20 con 0x7fc7d41544f0 2023-11-06T22:00:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7da5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40c30b0 con 0x7fc7d4064bb0 2023-11-06T22:00:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7da5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40c2010 con 0x7fc7d40c0c40 2023-11-06T22:00:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d8b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d41544f0 0x7fc7d4156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d8b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d41544f0 0x7fc7d4156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58378/0 (socket says 172.21.15.33:58378) 2023-11-06T22:00:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d8b9c700 1 -- 172.21.15.33:0/2053725969 learned_addr learned my addr 172.21.15.33:0/2053725969 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d3fff700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7d4064bb0 0x7fc7d4149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d37fe700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7d40c0c40 0x7fc7d40c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d8b9c700 1 -- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7d40c0c40 msgr2=0x7fc7d40c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d8b9c700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7d40c0c40 0x7fc7d40c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d8b9c700 1 -- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7d4064bb0 msgr2=0x7fc7d4149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d8b9c700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7d4064bb0 0x7fc7d4149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d8b9c700 1 -- 172.21.15.33:0/2053725969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80092a0 con 0x7fc7d41544f0 2023-11-06T22:00:10.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.772+0000 7fc7d8b9c700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d41544f0 0x7fc7d4156dd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7fc7cc005ad0 tx=0x7fc7cc001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:10.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.773+0000 7fc7bbfff700 1 -- 172.21.15.33:0/2053725969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc005d30 con 0x7fc7d41544f0 2023-11-06T22:00:10.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.773+0000 7fc7bbfff700 1 -- 172.21.15.33:0/2053725969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc005e90 con 0x7fc7d41544f0 2023-11-06T22:00:10.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.773+0000 7fc7bbfff700 1 -- 172.21.15.33:0/2053725969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc007320 con 0x7fc7d41544f0 2023-11-06T22:00:10.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.773+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2053725969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d40c22f0 con 0x7fc7d41544f0 2023-11-06T22:00:10.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.773+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2053725969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d40c28b0 con 0x7fc7d41544f0 2023-11-06T22:00:10.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.774+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2053725969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d4154da0 con 0x7fc7d41544f0 2023-11-06T22:00:10.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.775+0000 7fc7bbfff700 1 -- 172.21.15.33:0/2053725969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc7cc007710 con 0x7fc7d41544f0 2023-11-06T22:00:10.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.776+0000 7fc7bbfff700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7c005f270 0x7fc7c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:10.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.776+0000 7fc7bbfff700 1 -- 172.21.15.33:0/2053725969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fc7cc07a1e0 con 0x7fc7d41544f0 2023-11-06T22:00:10.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.777+0000 7fc7d3fff700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7c005f270 0x7fc7c0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:10.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.777+0000 7fc7d3fff700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7c005f270 0x7fc7c0061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc7d406bc30 tx=0x7fc7c4009060).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:10.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.780+0000 7fc7bbfff700 1 -- 172.21.15.33:0/2053725969 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc7cc04b3d0 con 0x7fc7d41544f0 2023-11-06T22:00:10.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.961+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2053725969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc7d4065340 con 0x7fc7d41544f0 2023-11-06T22:00:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.967+0000 7fc7bbfff700 1 -- 172.21.15.33:0/2053725969 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc7cc04e9f0 con 0x7fc7d41544f0 2023-11-06T22:00:10.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7c005f270 msgr2=0x7fc7c0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7c005f270 0x7fc7c0061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc7d406bc30 tx=0x7fc7c4009060).stop 2023-11-06T22:00:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2053725969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d41544f0 msgr2=0x7fc7d4156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d41544f0 0x7fc7d4156dd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7fc7cc005ad0 tx=0x7fc7cc001650).stop 2023-11-06T22:00:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2053725969 shutdown_connections 2023-11-06T22:00:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc7c005f270 0x7fc7c0061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d41544f0 0x7fc7d4156dd0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7d4064bb0 0x7fc7d4149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 --2- 172.21.15.33:0/2053725969 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc7d40c0c40 0x7fc7d40c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2053725969 >> 172.21.15.33:0/2053725969 conn(0x7fc7d40cb900 msgr2=0x7fc7d40ca3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2053725969 shutdown_connections 2023-11-06T22:00:10.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:10.969+0000 7fc7da5ff700 1 -- 172.21.15.33:0/2053725969 wait complete. 2023-11-06T22:00:10.989 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T22:00:10.989 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:00:11.079 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:00:11.177 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T22:00:11.177 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T22:00:11.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.370+0000 7f71d5dff700 1 Processor -- start 2023-11-06T22:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.370+0000 7f71d5dff700 1 -- start start 2023-11-06T22:00:11.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71d5dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71d01580e0 0x7f71d01584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71d5dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f71d00bdbe0 0x7f71d00684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71d5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71d00a7910 0x7f71d00a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71d5dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d00be430 con 0x7f71d01580e0 2023-11-06T22:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71d5dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d0064010 con 0x7f71d00bdbe0 2023-11-06T22:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71d5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d00689b0 con 0x7f71d00a7910 2023-11-06T22:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71cf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71d01580e0 0x7f71d01584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71cf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71d01580e0 0x7f71d01584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58380/0 (socket says 172.21.15.33:58380) 2023-11-06T22:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71cf7fe700 1 -- 172.21.15.33:0/2320988674 learned_addr learned my addr 172.21.15.33:0/2320988674 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71ceffd700 1 --2- 172.21.15.33:0/2320988674 >> v2:172.21.15.69:3300/0 conn(0x7f71d00bdbe0 0x7f71d00684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.372+0000 7f71cffff700 1 --2- 172.21.15.33:0/2320988674 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71d00a7910 0x7f71d00a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.373+0000 7f71cf7fe700 1 -- 172.21.15.33:0/2320988674 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71d00a7910 msgr2=0x7f71d00a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:11.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.373+0000 7f71cf7fe700 1 --2- 172.21.15.33:0/2320988674 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71d00a7910 0x7f71d00a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.373+0000 7f71cf7fe700 1 -- 172.21.15.33:0/2320988674 >> v2:172.21.15.69:3300/0 conn(0x7f71d00bdbe0 msgr2=0x7f71d00684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:11.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.373+0000 7f71cf7fe700 1 --2- 172.21.15.33:0/2320988674 >> v2:172.21.15.69:3300/0 conn(0x7f71d00bdbe0 0x7f71d00684c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.373+0000 7f71cf7fe700 1 -- 172.21.15.33:0/2320988674 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d00a8290 con 0x7f71d01580e0 2023-11-06T22:00:11.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.373+0000 7f71cf7fe700 1 --2- 172.21.15.33:0/2320988674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 0x7f71d01584c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f71d00bfd70 tx=0x7f71c00090f0).ready entity=mon.0 client_cookie=cfbcb0f11b19de3d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.373+0000 7f71ce7fc700 1 -- 172.21.15.33:0/2320988674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000f510 con 0x7f71d01580e0 2023-11-06T22:00:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.373+0000 7f71ce7fc700 1 -- 172.21.15.33:0/2320988674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c000f670 con 0x7f71d01580e0 2023-11-06T22:00:11.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.373+0000 7f71ce7fc700 1 -- 172.21.15.33:0/2320988674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000fa10 con 0x7f71d01580e0 2023-11-06T22:00:11.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.374+0000 7f71d5dff700 1 -- 172.21.15.33:0/2320988674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 msgr2=0x7f71d01584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:11.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.374+0000 7f71d5dff700 1 --2- 172.21.15.33:0/2320988674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 0x7f71d01584c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f71d00bfd70 tx=0x7f71c00090f0).stop 2023-11-06T22:00:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.374+0000 7f71d5dff700 1 -- 172.21.15.33:0/2320988674 shutdown_connections 2023-11-06T22:00:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.374+0000 7f71d5dff700 1 --2- 172.21.15.33:0/2320988674 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71d00a7910 0x7f71d00a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.374+0000 7f71d5dff700 1 --2- 172.21.15.33:0/2320988674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 0x7f71d01584c0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.374+0000 7f71d5dff700 1 --2- 172.21.15.33:0/2320988674 >> v2:172.21.15.69:3300/0 conn(0x7f71d00bdbe0 0x7f71d00684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.374+0000 7f71d5dff700 1 -- 172.21.15.33:0/2320988674 >> 172.21.15.33:0/2320988674 conn(0x7f71d005ec80 msgr2=0x7f71d00664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:11.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.374+0000 7f71d5dff700 1 -- 172.21.15.33:0/2320988674 shutdown_connections 2023-11-06T22:00:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.374+0000 7f71d5dff700 1 -- 172.21.15.33:0/2320988674 wait complete. 2023-11-06T22:00:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.376+0000 7f71d5dff700 1 Processor -- start 2023-11-06T22:00:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.376+0000 7f71d5dff700 1 -- start start 2023-11-06T22:00:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.377+0000 7f71d5dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71d00a7910 0x7f71d0137600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.377+0000 7f71d5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71d00bdbe0 0x7f71d00c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.377+0000 7f71d5dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 0x7f71d00c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.377+0000 7f71d5dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d00c84f0 con 0x7f71d01580e0 2023-11-06T22:00:11.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.377+0000 7f71d5dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d00c2bf0 con 0x7f71d00a7910 2023-11-06T22:00:11.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.377+0000 7f71d5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d00c2910 con 0x7f71d00bdbe0 2023-11-06T22:00:11.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 0x7f71d00c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 0x7f71d00c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58394/0 (socket says 172.21.15.33:58394) 2023-11-06T22:00:11.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71cffff700 1 -- 172.21.15.33:0/2781337097 learned_addr learned my addr 172.21.15.33:0/2781337097 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:11.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71cf7fe700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71d00a7910 0x7f71d0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71ceffd700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71d00bdbe0 0x7f71d00c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71cffff700 1 -- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71d00bdbe0 msgr2=0x7f71d00c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71cffff700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71d00bdbe0 0x7f71d00c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71cffff700 1 -- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71d00a7910 msgr2=0x7f71d0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71cffff700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71d00a7910 0x7f71d0137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71cffff700 1 -- 172.21.15.33:0/2781337097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c0009370 con 0x7f71d01580e0 2023-11-06T22:00:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.378+0000 7f71cffff700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 0x7f71d00c8000 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f71c8009df0 tx=0x7f71c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.379+0000 7f71b77fe700 1 -- 172.21.15.33:0/2781337097 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c800caa0 con 0x7f71d01580e0 2023-11-06T22:00:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.379+0000 7f71b77fe700 1 -- 172.21.15.33:0/2781337097 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c800f040 con 0x7f71d01580e0 2023-11-06T22:00:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.379+0000 7f71d5dff700 1 -- 172.21.15.33:0/2781337097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d00bfed0 con 0x7f71d01580e0 2023-11-06T22:00:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.379+0000 7f71b77fe700 1 -- 172.21.15.33:0/2781337097 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c800ce10 con 0x7f71d01580e0 2023-11-06T22:00:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.379+0000 7f71d5dff700 1 -- 172.21.15.33:0/2781337097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d00c0490 con 0x7f71d01580e0 2023-11-06T22:00:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.380+0000 7f71d5dff700 1 -- 172.21.15.33:0/2781337097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d00c06b0 con 0x7f71d01580e0 2023-11-06T22:00:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.381+0000 7f71b77fe700 1 -- 172.21.15.33:0/2781337097 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f71c8013610 con 0x7f71d01580e0 2023-11-06T22:00:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.381+0000 7f71b77fe700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71bc05f270 0x7f71bc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.382+0000 7f71b77fe700 1 -- 172.21.15.33:0/2781337097 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f71c807a980 con 0x7f71d01580e0 2023-11-06T22:00:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.382+0000 7f71cf7fe700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71bc05f270 0x7f71bc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.382+0000 7f71cf7fe700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71bc05f270 0x7f71bc061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f71d00bfd70 tx=0x7f71c0017000).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:11.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.386+0000 7f71b77fe700 1 -- 172.21.15.33:0/2781337097 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f71c804bbd0 con 0x7f71d01580e0 2023-11-06T22:00:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.610+0000 7f71d5dff700 1 -- 172.21.15.33:0/2781337097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71d006be80 con 0x7f71d01580e0 2023-11-06T22:00:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.610+0000 7f71b77fe700 1 -- 172.21.15.33:0/2781337097 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f71c804f1f0 con 0x7f71d01580e0 2023-11-06T22:00:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:11.613 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":70,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3645039939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3645039939},{"type":"v1","addr":"172.21.15.33:6835","nonce":3645039939}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1156785194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1156785194},{"type":"v1","addr":"172.21.15.69:6836","nonce":1156785194}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/4183165491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":4183165491},{"type":"v1","addr":"172.21.15.69:6837","nonce":4183165491}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2586335400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2586335400},{"type":"v1","addr":"172.21.15.33:6833","nonce":2586335400}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[]} 2023-11-06T22:00:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.612+0000 7f71d5dff700 1 -- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71bc05f270 msgr2=0x7f71bc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.612+0000 7f71d5dff700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71bc05f270 0x7f71bc061730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f71d00bfd70 tx=0x7f71c0017000).stop 2023-11-06T22:00:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.612+0000 7f71d5dff700 1 -- 172.21.15.33:0/2781337097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 msgr2=0x7f71d00c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.612+0000 7f71d5dff700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 0x7f71d00c8000 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f71c8009df0 tx=0x7f71c800b040).stop 2023-11-06T22:00:11.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.613+0000 7f71d5dff700 1 -- 172.21.15.33:0/2781337097 shutdown_connections 2023-11-06T22:00:11.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.613+0000 7f71d5dff700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f71bc05f270 0x7f71bc061730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.613+0000 7f71d5dff700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71d01580e0 0x7f71d00c8000 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.613+0000 7f71d5dff700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71d00a7910 0x7f71d0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.613+0000 7f71d5dff700 1 --2- 172.21.15.33:0/2781337097 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f71d00bdbe0 0x7f71d00c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.613+0000 7f71d5dff700 1 -- 172.21.15.33:0/2781337097 >> 172.21.15.33:0/2781337097 conn(0x7f71d005ec80 msgr2=0x7f71d0068290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:11.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.613+0000 7f71d5dff700 1 -- 172.21.15.33:0/2781337097 shutdown_connections 2023-11-06T22:00:11.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.613+0000 7f71d5dff700 1 -- 172.21.15.33:0/2781337097 wait complete. 2023-11-06T22:00:11.617 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 70 2023-11-06T22:00:11.633 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T22:00:11.634 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T22:00:11.634 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T22:00:11.635 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:00:11.635 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T22:00:11.636 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T22:00:11.636 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T22:00:11.637 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:00:11.637 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T22:00:11.638 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T22:00:11.638 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T22:00:11.639 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:00:11.639 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T22:00:11.640 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T22:00:11.640 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T22:00:11.641 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:00:11.648 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T22:00:11.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T22:00:11.651 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T22:00:11.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T22:00:11.656 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T22:00:11.657 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T22:00:11.658 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T22:00:11.658 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T22:00:11.695 INFO:tasks.ceph.mds.a:Started 2023-11-06T22:00:11.696 DEBUG:teuthology.parallel:result is None 2023-11-06T22:00:11.699 INFO:tasks.ceph.mds.c:Started 2023-11-06T22:00:11.700 DEBUG:teuthology.parallel:result is None 2023-11-06T22:00:11.701 INFO:tasks.ceph.mds.d:Started 2023-11-06T22:00:11.702 DEBUG:teuthology.parallel:result is None 2023-11-06T22:00:11.704 INFO:tasks.ceph.mds.b:Started 2023-11-06T22:00:11.705 DEBUG:teuthology.parallel:result is None 2023-11-06T22:00:11.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T22:00:11.845 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T22:00:11.846 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T22:00:11.852 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T22:00:11.853 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T22:00:11.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.916+0000 7f65a05ff700 1 Processor -- start 2023-11-06T22:00:11.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.917+0000 7f65a05ff700 1 -- start start 2023-11-06T22:00:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.918+0000 7f65a05ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f65980c0470 0x7f65980c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.918+0000 7f65a05ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65980c7700 con 0x7f65980c6d90 2023-11-06T22:00:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.918+0000 7f65a05ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659805efc0 con 0x7f6598154210 2023-11-06T22:00:11.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.918+0000 7f65a05ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65980c8ea0 con 0x7f65980c0470 2023-11-06T22:00:11.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.918+0000 7f659db9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f65980c6d90 0x7f65980bfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48982/0 (socket says 172.21.15.33:48982) 2023-11-06T22:00:11.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.918+0000 7f659db9a700 1 -- 172.21.15.33:0/4082336547 learned_addr learned my addr 172.21.15.33:0/4082336547 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:11.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.918+0000 7f659eb9c700 1 --2- 172.21.15.33:0/4082336547 >> v2:172.21.15.69:3300/0 conn(0x7f65980c0470 0x7f65980c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.918+0000 7f659d399700 1 -- 172.21.15.33:0/4082336547 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322204992 0 0) 0x7f6588001110 con 0x7f65980c6d90 2023-11-06T22:00:11.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.919+0000 7f659d399700 1 -- 172.21.15.33:0/4082336547 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65840033f0 con 0x7f65980c6d90 2023-11-06T22:00:11.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.919+0000 7f659d399700 1 -- 172.21.15.33:0/4082336547 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455269692 0 0) 0x7f659805efc0 con 0x7f6598154210 2023-11-06T22:00:11.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.919+0000 7f659d399700 1 -- 172.21.15.33:0/4082336547 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6588001110 con 0x7f6598154210 2023-11-06T22:00:11.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.919+0000 7f659eb9c700 1 -- 172.21.15.33:0/4082336547 >> v1:172.21.15.69:6789/0 conn(0x7f6598154210 legacy=0x7f65980a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:11.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.919+0000 7f659eb9c700 1 -- 172.21.15.33:0/4082336547 >> v1:172.21.15.33:6789/0 conn(0x7f65980c6d90 legacy=0x7f65980bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:11.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.919+0000 7f659eb9c700 1 -- 172.21.15.33:0/4082336547 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65980c9140 con 0x7f65980c0470 2023-11-06T22:00:11.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.919+0000 7f659eb9c700 1 --2- 172.21.15.33:0/4082336547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65980c0470 0x7f65980c7210 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f6590009e60 tx=0x7f6590009780).ready entity=mon.1 client_cookie=8a719baa278156b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:11.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.919+0000 7f659d399700 1 -- 172.21.15.33:0/4082336547 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659000e070 con 0x7f65980c0470 2023-11-06T22:00:11.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.919+0000 7f659d399700 1 -- 172.21.15.33:0/4082336547 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659000bab0 con 0x7f65980c0470 2023-11-06T22:00:11.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.919+0000 7f659d399700 1 -- 172.21.15.33:0/4082336547 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659000be30 con 0x7f65980c0470 2023-11-06T22:00:11.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.920+0000 7f65a05ff700 1 -- 172.21.15.33:0/4082336547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65980c0470 msgr2=0x7f65980c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:11.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.920+0000 7f65a05ff700 1 --2- 172.21.15.33:0/4082336547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65980c0470 0x7f65980c7210 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f6590009e60 tx=0x7f6590009780).stop 2023-11-06T22:00:11.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.920+0000 7f65a05ff700 1 -- 172.21.15.33:0/4082336547 shutdown_connections 2023-11-06T22:00:11.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.920+0000 7f65a05ff700 1 --2- 172.21.15.33:0/4082336547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65980c0470 0x7f65980c7210 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.920+0000 7f65a05ff700 1 -- 172.21.15.33:0/4082336547 >> 172.21.15.33:0/4082336547 conn(0x7f659805e990 msgr2=0x7f65980cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:11.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.920+0000 7f65a05ff700 1 -- 172.21.15.33:0/4082336547 shutdown_connections 2023-11-06T22:00:11.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.920+0000 7f65a05ff700 1 -- 172.21.15.33:0/4082336547 wait complete. 2023-11-06T22:00:11.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.921+0000 7f65a05ff700 1 Processor -- start 2023-11-06T22:00:11.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.921+0000 7f65a05ff700 1 -- start start 2023-11-06T22:00:11.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.922+0000 7f65a05ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65980c0470 0x7f65980c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.922+0000 7f65a05ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f65980c6d90 0x7f65980c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.922+0000 7f65a05ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6598154210 0x7f6598150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.922+0000 7f65a05ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65980c3940 con 0x7f65980c0470 2023-11-06T22:00:11.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.922+0000 7f65a05ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65981547b0 con 0x7f6598154210 2023-11-06T22:00:11.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.922+0000 7f65a05ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65980c2e20 con 0x7f65980c6d90 2023-11-06T22:00:11.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.922+0000 7f659e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65980c0470 0x7f65980c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65980c0470 0x7f65980c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58428/0 (socket says 172.21.15.33:58428) 2023-11-06T22:00:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659e39b700 1 -- 172.21.15.33:0/4022629421 learned_addr learned my addr 172.21.15.33:0/4022629421 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659eb9c700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6598154210 0x7f6598150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659db9a700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f65980c6d90 0x7f65980c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659e39b700 1 -- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f65980c6d90 msgr2=0x7f65980c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659e39b700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f65980c6d90 0x7f65980c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659e39b700 1 -- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6598154210 msgr2=0x7f6598150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659e39b700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6598154210 0x7f6598150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659e39b700 1 -- 172.21.15.33:0/4022629421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590009a20 con 0x7f65980c0470 2023-11-06T22:00:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659eb9c700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6598154210 0x7f6598150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f659e39b700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65980c0470 0x7f65980c4320 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f659400c470 tx=0x7f6594007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f658e7fc700 1 -- 172.21.15.33:0/4022629421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400ce90 con 0x7f65980c0470 2023-11-06T22:00:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f658e7fc700 1 -- 172.21.15.33:0/4022629421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659400f470 con 0x7f65980c0470 2023-11-06T22:00:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f65a05ff700 1 -- 172.21.15.33:0/4022629421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6598150970 con 0x7f65980c0470 2023-11-06T22:00:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.923+0000 7f658e7fc700 1 -- 172.21.15.33:0/4022629421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400f830 con 0x7f65980c0470 2023-11-06T22:00:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.924+0000 7f65a05ff700 1 -- 172.21.15.33:0/4022629421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65980c3380 con 0x7f65980c0470 2023-11-06T22:00:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.925+0000 7f65a05ff700 1 -- 172.21.15.33:0/4022629421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659806c6c0 con 0x7f65980c0470 2023-11-06T22:00:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.925+0000 7f658e7fc700 1 -- 172.21.15.33:0/4022629421 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f659400fbe0 con 0x7f65980c0470 2023-11-06T22:00:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.926+0000 7f658e7fc700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f657c05f270 0x7f657c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.926+0000 7f658e7fc700 1 -- 172.21.15.33:0/4022629421 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f6594079880 con 0x7f65980c0470 2023-11-06T22:00:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.927+0000 7f659db9a700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f657c05f270 0x7f657c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.927+0000 7f659db9a700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f657c05f270 0x7f657c061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f6598098ac0 tx=0x7f6588009040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:11.931+0000 7f658e7fc700 1 -- 172.21.15.33:0/4022629421 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f659404aa30 con 0x7f65980c0470 2023-11-06T22:00:12.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.126+0000 7f65a05ff700 1 -- 172.21.15.33:0/4022629421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f65980c9420 con 0x7f65980c0470 2023-11-06T22:00:12.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.970+0000 7f658e7fc700 1 -- 172.21.15.33:0/4022629421 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v181) v1 ==== 97+0+0 (secure 0 0 0) 0x7f659404e050 con 0x7f65980c0470 2023-11-06T22:00:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 -- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f657c05f270 msgr2=0x7f657c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f657c05f270 0x7f657c061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f6598098ac0 tx=0x7f6588009040).stop 2023-11-06T22:00:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 -- 172.21.15.33:0/4022629421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65980c0470 msgr2=0x7f65980c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65980c0470 0x7f65980c4320 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f659400c470 tx=0x7f6594007c90).stop 2023-11-06T22:00:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 -- 172.21.15.33:0/4022629421 shutdown_connections 2023-11-06T22:00:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f657c05f270 0x7f657c061730 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6598154210 0x7f6598150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:12.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65980c0470 0x7f65980c4320 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:12.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 --2- 172.21.15.33:0/4022629421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f65980c6d90 0x7f65980c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:12.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 -- 172.21.15.33:0/4022629421 >> 172.21.15.33:0/4022629421 conn(0x7f659805e990 msgr2=0x7f65980a7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:12.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 -- 172.21.15.33:0/4022629421 shutdown_connections 2023-11-06T22:00:12.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:12.973+0000 7f65a05ff700 1 -- 172.21.15.33:0/4022629421 wait complete. 2023-11-06T22:00:12.978 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T22:00:12.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T22:00:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.194+0000 7f3a647ff700 1 Processor -- start 2023-11-06T22:00:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.194+0000 7f3a647ff700 1 -- start start 2023-11-06T22:00:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.195+0000 7f3a647ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3a5c0c2de0 0x7f3a5c064b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.195+0000 7f3a647ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5c0c1940 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.195+0000 7f3a647ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5c065160 con 0x7f3a5c14ae80 2023-11-06T22:00:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.195+0000 7f3a647ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5c14b260 con 0x7f3a5c0c2de0 2023-11-06T22:00:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.196+0000 7f3a61d9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3a5c0bc7e0 0x7f3a5c064400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48984/0 (socket says 172.21.15.33:48984) 2023-11-06T22:00:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.196+0000 7f3a61d9a700 1 -- 172.21.15.33:0/2252613798 learned_addr learned my addr 172.21.15.33:0/2252613798 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.196+0000 7f3a62d9c700 1 --2- 172.21.15.33:0/2252613798 >> v2:172.21.15.69:3300/0 conn(0x7f3a5c0c2de0 0x7f3a5c064b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.196+0000 7f3a61599700 1 -- 172.21.15.33:0/2252613798 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682480958 0 0) 0x7f3a5c0c1940 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.196+0000 7f3a61599700 1 -- 172.21.15.33:0/2252613798 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a48004500 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.196+0000 7f3a62d9c700 1 -- 172.21.15.33:0/2252613798 >> v1:172.21.15.69:6789/0 conn(0x7f3a5c14ae80 legacy=0x7f3a5c0bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.196+0000 7f3a61599700 1 -- 172.21.15.33:0/2252613798 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4066471314 0 0) 0x7f3a48004500 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.196+0000 7f3a62d9c700 1 -- 172.21.15.33:0/2252613798 >> v1:172.21.15.33:6789/0 conn(0x7f3a5c0bc7e0 legacy=0x7f3a5c064400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.196+0000 7f3a62d9c700 1 -- 172.21.15.33:0/2252613798 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c065440 con 0x7f3a5c0c2de0 2023-11-06T22:00:13.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.197+0000 7f3a62d9c700 1 --2- 172.21.15.33:0/2252613798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a5c0c2de0 0x7f3a5c064b10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f3a54009de0 tx=0x7f3a54009780).ready entity=mon.1 client_cookie=5493289c40533017 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:13.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.197+0000 7f3a61599700 1 -- 172.21.15.33:0/2252613798 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400e070 con 0x7f3a5c0c2de0 2023-11-06T22:00:13.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.197+0000 7f3a61599700 1 -- 172.21.15.33:0/2252613798 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5400b850 con 0x7f3a5c0c2de0 2023-11-06T22:00:13.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.197+0000 7f3a61599700 1 -- 172.21.15.33:0/2252613798 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c001470 con 0x7f3a5c0c2de0 2023-11-06T22:00:13.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.197+0000 7f3a647ff700 1 -- 172.21.15.33:0/2252613798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a5c0c2de0 msgr2=0x7f3a5c064b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.197+0000 7f3a647ff700 1 --2- 172.21.15.33:0/2252613798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a5c0c2de0 0x7f3a5c064b10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f3a54009de0 tx=0x7f3a54009780).stop 2023-11-06T22:00:13.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.198+0000 7f3a647ff700 1 -- 172.21.15.33:0/2252613798 shutdown_connections 2023-11-06T22:00:13.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.198+0000 7f3a647ff700 1 --2- 172.21.15.33:0/2252613798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a5c0c2de0 0x7f3a5c064b10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.198+0000 7f3a647ff700 1 -- 172.21.15.33:0/2252613798 >> 172.21.15.33:0/2252613798 conn(0x7f3a5c062710 msgr2=0x7f3a5c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:13.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.198+0000 7f3a647ff700 1 -- 172.21.15.33:0/2252613798 shutdown_connections 2023-11-06T22:00:13.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.198+0000 7f3a647ff700 1 -- 172.21.15.33:0/2252613798 wait complete. 2023-11-06T22:00:13.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.199+0000 7f3a647ff700 1 Processor -- start 2023-11-06T22:00:13.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.199+0000 7f3a647ff700 1 -- start start 2023-11-06T22:00:13.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.200+0000 7f3a647ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a5c0bc7e0 0x7f3a5c164ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.200+0000 7f3a647ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a5c0c2de0 0x7f3a5c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.200+0000 7f3a647ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a5c14ae80 0x7f3a5c0ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.200+0000 7f3a647ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c164fd0 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.200+0000 7f3a647ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c153830 con 0x7f3a5c14ae80 2023-11-06T22:00:13.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.200+0000 7f3a647ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c154cf0 con 0x7f3a5c0c2de0 2023-11-06T22:00:13.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a6259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a5c0bc7e0 0x7f3a5c164ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a6259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a5c0bc7e0 0x7f3a5c164ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58430/0 (socket says 172.21.15.33:58430) 2023-11-06T22:00:13.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a6259b700 1 -- 172.21.15.33:0/134677876 learned_addr learned my addr 172.21.15.33:0/134677876 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:13.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a62d9c700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a5c14ae80 0x7f3a5c0ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a61d9a700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a5c0c2de0 0x7f3a5c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a6259b700 1 -- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a5c0c2de0 msgr2=0x7f3a5c154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a6259b700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a5c0c2de0 0x7f3a5c154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a6259b700 1 -- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a5c14ae80 msgr2=0x7f3a5c0ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a6259b700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a5c14ae80 0x7f3a5c0ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a6259b700 1 -- 172.21.15.33:0/134677876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54009a20 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.201+0000 7f3a6259b700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a5c0bc7e0 0x7f3a5c164ae0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f3a5800c490 tx=0x7f3a58007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:13.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.202+0000 7f3a527fc700 1 -- 172.21.15.33:0/134677876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58007020 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.202+0000 7f3a527fc700 1 -- 172.21.15.33:0/134677876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a580071a0 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.202+0000 7f3a647ff700 1 -- 172.21.15.33:0/134677876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c154370 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.202+0000 7f3a527fc700 1 -- 172.21.15.33:0/134677876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58007560 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.202+0000 7f3a647ff700 1 -- 172.21.15.33:0/134677876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c153a40 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.203+0000 7f3a647ff700 1 -- 172.21.15.33:0/134677876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c154650 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.204+0000 7f3a527fc700 1 -- 172.21.15.33:0/134677876 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3a580068b0 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.204+0000 7f3a527fc700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4005f270 0x7f3a40061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.204+0000 7f3a527fc700 1 -- 172.21.15.33:0/134677876 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f3a58079340 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.205+0000 7f3a61d9a700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4005f270 0x7f3a40061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.205+0000 7f3a61d9a700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4005f270 0x7f3a40061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3a5c149510 tx=0x7f3a4c000bc0).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:13.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.208+0000 7f3a527fc700 1 -- 172.21.15.33:0/134677876 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3a58047af0 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.456+0000 7f3a647ff700 1 -- 172.21.15.33:0/134677876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3a5c05f0b0 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.456+0000 7f3a527fc700 1 -- 172.21.15.33:0/134677876 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v39) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3a5804a7b0 con 0x7f3a5c0bc7e0 2023-11-06T22:00:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:13.459 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:00:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T22:00:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T22:00:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T22:00:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:00:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:00:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:00:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T22:00:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T22:00:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:00:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:00:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:00:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T22:00:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T22:00:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:00:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:00:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:00:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T22:00:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T22:00:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:00:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:00:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:00:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T22:00:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T22:00:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:00:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:00:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T22:00:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T22:00:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:00:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:00:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T22:00:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T22:00:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:00:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:00:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T22:00:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T22:00:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:00:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:00:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T22:00:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T22:00:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:00:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:00:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T22:00:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T22:00:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:00:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:00:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T22:00:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T22:00:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:00:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:00:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T22:00:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T22:00:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:00:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:00:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T22:00:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T22:00:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:00:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-11-06T22:00:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T22:00:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T22:00:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T22:00:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:00:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T22:00:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T22:00:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T22:00:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T22:00:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T22:00:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:00:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T22:00:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T22:00:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T22:00:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T22:00:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T22:00:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:00:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T22:00:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T22:00:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T22:00:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T22:00:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T22:00:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T22:00:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:00:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:00:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T22:00:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T22:00:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T22:00:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T22:00:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T22:00:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T22:00:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:00:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T22:00:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T22:00:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:00:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:13.495 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T22:00:13.495 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:00:13.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.458+0000 7f3a647ff700 1 -- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4005f270 msgr2=0x7f3a40061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.458+0000 7f3a647ff700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4005f270 0x7f3a40061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3a5c149510 tx=0x7f3a4c000bc0).stop 2023-11-06T22:00:13.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.458+0000 7f3a647ff700 1 -- 172.21.15.33:0/134677876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a5c0bc7e0 msgr2=0x7f3a5c164ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.458+0000 7f3a647ff700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a5c0bc7e0 0x7f3a5c164ae0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f3a5800c490 tx=0x7f3a58007da0).stop 2023-11-06T22:00:13.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.459+0000 7f3a647ff700 1 -- 172.21.15.33:0/134677876 shutdown_connections 2023-11-06T22:00:13.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.459+0000 7f3a647ff700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3a4005f270 0x7f3a40061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.459+0000 7f3a647ff700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a5c14ae80 0x7f3a5c0ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.459+0000 7f3a647ff700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a5c0bc7e0 0x7f3a5c164ae0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.459+0000 7f3a647ff700 1 --2- 172.21.15.33:0/134677876 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a5c0c2de0 0x7f3a5c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.459+0000 7f3a647ff700 1 -- 172.21.15.33:0/134677876 >> 172.21.15.33:0/134677876 conn(0x7f3a5c062710 msgr2=0x7f3a5c069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:13.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.459+0000 7f3a647ff700 1 -- 172.21.15.33:0/134677876 shutdown_connections 2023-11-06T22:00:13.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.459+0000 7f3a647ff700 1 -- 172.21.15.33:0/134677876 wait complete. 2023-11-06T22:00:13.499 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:00:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.694+0000 7f78f4fff700 1 Processor -- start 2023-11-06T22:00:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.695+0000 7f78f4fff700 1 -- start start 2023-11-06T22:00:13.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.696+0000 7f78f4fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f78f01580e0 0x7f78f01584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.696+0000 7f78f4fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78f00a7910 0x7f78f00a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.696+0000 7f78f4fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f0064010 con 0x7f78f00bdbe0 2023-11-06T22:00:13.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.696+0000 7f78f4fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f00be430 con 0x7f78f00a7910 2023-11-06T22:00:13.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.696+0000 7f78f4fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f00be150 con 0x7f78f01580e0 2023-11-06T22:00:13.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.696+0000 7f78eed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78f00a7910 0x7f78f00a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.696+0000 7f78eed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78f00a7910 0x7f78f00a7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58434/0 (socket says 172.21.15.33:58434) 2023-11-06T22:00:13.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.696+0000 7f78eed9d700 1 -- 172.21.15.33:0/3807903972 learned_addr learned my addr 172.21.15.33:0/3807903972 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:13.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.696+0000 7f78edd9b700 1 --1- 172.21.15.33:0/3807903972 >> v1:172.21.15.33:6789/0 conn(0x7f78f00bdbe0 0x7f78f00684c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48986/0 (socket says 172.21.15.33:48986) 2023-11-06T22:00:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.697+0000 7f78ee59c700 1 --2- 172.21.15.33:0/3807903972 >> v2:172.21.15.69:3300/0 conn(0x7f78f01580e0 0x7f78f01584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.697+0000 7f78eed9d700 1 -- 172.21.15.33:0/3807903972 >> v2:172.21.15.69:3300/0 conn(0x7f78f01580e0 msgr2=0x7f78f01584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.697+0000 7f78eed9d700 1 --2- 172.21.15.33:0/3807903972 >> v2:172.21.15.69:3300/0 conn(0x7f78f01580e0 0x7f78f01584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.697+0000 7f78eed9d700 1 -- 172.21.15.33:0/3807903972 >> v1:172.21.15.33:6789/0 conn(0x7f78f00bdbe0 legacy=0x7f78f00684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.697+0000 7f78eed9d700 1 -- 172.21.15.33:0/3807903972 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f00639c0 con 0x7f78f00a7910 2023-11-06T22:00:13.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.697+0000 7f78ed59a700 1 -- 172.21.15.33:0/3807903972 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778267259 0 0) 0x7f78f0064010 con 0x7f78f00bdbe0 2023-11-06T22:00:13.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.697+0000 7f78eed9d700 1 --2- 172.21.15.33:0/3807903972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f00a7910 0x7f78f00a7d60 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f78f005f090 tx=0x7f78e8009840).ready entity=mon.0 client_cookie=139ce4817448a7b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.697+0000 7f78ed59a700 1 -- 172.21.15.33:0/3807903972 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e800e070 con 0x7f78f00a7910 2023-11-06T22:00:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.697+0000 7f78ed59a700 1 -- 172.21.15.33:0/3807903972 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e800ba50 con 0x7f78f00a7910 2023-11-06T22:00:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.697+0000 7f78ed59a700 1 -- 172.21.15.33:0/3807903972 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e800bdd0 con 0x7f78f00a7910 2023-11-06T22:00:13.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.698+0000 7f78f4fff700 1 -- 172.21.15.33:0/3807903972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f00a7910 msgr2=0x7f78f00a7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.698+0000 7f78f4fff700 1 --2- 172.21.15.33:0/3807903972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f00a7910 0x7f78f00a7d60 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f78f005f090 tx=0x7f78e8009840).stop 2023-11-06T22:00:13.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.698+0000 7f78f4fff700 1 -- 172.21.15.33:0/3807903972 shutdown_connections 2023-11-06T22:00:13.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.698+0000 7f78f4fff700 1 --2- 172.21.15.33:0/3807903972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f00a7910 0x7f78f00a7d60 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.698+0000 7f78f4fff700 1 --2- 172.21.15.33:0/3807903972 >> v2:172.21.15.69:3300/0 conn(0x7f78f01580e0 0x7f78f01584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.698+0000 7f78f4fff700 1 -- 172.21.15.33:0/3807903972 >> 172.21.15.33:0/3807903972 conn(0x7f78f005ec80 msgr2=0x7f78f00664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:13.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.698+0000 7f78f4fff700 1 -- 172.21.15.33:0/3807903972 shutdown_connections 2023-11-06T22:00:13.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.698+0000 7f78f4fff700 1 -- 172.21.15.33:0/3807903972 wait complete. 2023-11-06T22:00:13.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.699+0000 7f78f4fff700 1 Processor -- start 2023-11-06T22:00:13.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.700+0000 7f78f4fff700 1 -- start start 2023-11-06T22:00:13.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78f4fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f00a7910 0x7f78f00c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78f4fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78f00bdbe0 0x7f78f0139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78f4fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f01580e0 0x7f78f0167af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78f4fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00adde0 con 0x7f78f01580e0 2023-11-06T22:00:13.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78f4fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00ae0c0 con 0x7f78f00a7910 2023-11-06T22:00:13.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78f4fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00ae3a0 con 0x7f78f00bdbe0 2023-11-06T22:00:13.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78eed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f01580e0 0x7f78f0167af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78eed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f01580e0 0x7f78f0167af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58448/0 (socket says 172.21.15.33:58448) 2023-11-06T22:00:13.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78eed9d700 1 -- 172.21.15.33:0/2923948313 learned_addr learned my addr 172.21.15.33:0/2923948313 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:13.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78ee59c700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f00a7910 0x7f78f00c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78edd9b700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78f00bdbe0 0x7f78f0139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78eed9d700 1 -- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78f00bdbe0 msgr2=0x7f78f0139920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78eed9d700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78f00bdbe0 0x7f78f0139920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78eed9d700 1 -- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f00a7910 msgr2=0x7f78f00c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78eed9d700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f00a7910 0x7f78f00c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78eed9d700 1 -- 172.21.15.33:0/2923948313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8009ae0 con 0x7f78f01580e0 2023-11-06T22:00:13.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.701+0000 7f78edd9b700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78f00bdbe0 0x7f78f0139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:13.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.702+0000 7f78eed9d700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f01580e0 0x7f78f0167af0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f78f0069bd0 tx=0x7f78e8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:13.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.702+0000 7f78de7fc700 1 -- 172.21.15.33:0/2923948313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e800bdd0 con 0x7f78f01580e0 2023-11-06T22:00:13.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.702+0000 7f78de7fc700 1 -- 172.21.15.33:0/2923948313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e8009500 con 0x7f78f01580e0 2023-11-06T22:00:13.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.702+0000 7f78f4fff700 1 -- 172.21.15.33:0/2923948313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f00ae680 con 0x7f78f01580e0 2023-11-06T22:00:13.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.702+0000 7f78de7fc700 1 -- 172.21.15.33:0/2923948313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e801c630 con 0x7f78f01580e0 2023-11-06T22:00:13.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.702+0000 7f78f4fff700 1 -- 172.21.15.33:0/2923948313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f00adac0 con 0x7f78f01580e0 2023-11-06T22:00:13.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.703+0000 7f78f4fff700 1 -- 172.21.15.33:0/2923948313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f0066bf0 con 0x7f78f01580e0 2023-11-06T22:00:13.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.704+0000 7f78de7fc700 1 -- 172.21.15.33:0/2923948313 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f78e801ca60 con 0x7f78f01580e0 2023-11-06T22:00:13.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.705+0000 7f78de7fc700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78d005f270 0x7f78d0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:13.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.705+0000 7f78de7fc700 1 -- 172.21.15.33:0/2923948313 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f78e807b9b0 con 0x7f78f01580e0 2023-11-06T22:00:13.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.706+0000 7f78ee59c700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78d005f270 0x7f78d0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:13.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.706+0000 7f78ee59c700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78d005f270 0x7f78d0061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f78e000bbc0 tx=0x7f78e0008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:13.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.710+0000 7f78de7fc700 1 -- 172.21.15.33:0/2923948313 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f78e804ce20 con 0x7f78f01580e0 2023-11-06T22:00:13.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.891+0000 7f78f4fff700 1 -- 172.21.15.33:0/2923948313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78f013ad90 con 0x7f78f01580e0 2023-11-06T22:00:13.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.892+0000 7f78de7fc700 1 -- 172.21.15.33:0/2923948313 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+13841 (secure 0 0 0) 0x7f78e8050440 con 0x7f78f01580e0 2023-11-06T22:00:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:13.895 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":181,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:00:12.967927+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:00:13.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 -- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78d005f270 msgr2=0x7f78d0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78d005f270 0x7f78d0061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f78e000bbc0 tx=0x7f78e0008040).stop 2023-11-06T22:00:13.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 -- 172.21.15.33:0/2923948313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f01580e0 msgr2=0x7f78f0167af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:13.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f01580e0 0x7f78f0167af0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f78f0069bd0 tx=0x7f78e8009840).stop 2023-11-06T22:00:13.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 -- 172.21.15.33:0/2923948313 shutdown_connections 2023-11-06T22:00:13.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f78d005f270 0x7f78d0061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f01580e0 0x7f78f0167af0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f00a7910 0x7f78f00c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 --2- 172.21.15.33:0/2923948313 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f78f00bdbe0 0x7f78f0139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:13.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 -- 172.21.15.33:0/2923948313 >> 172.21.15.33:0/2923948313 conn(0x7f78f005ec80 msgr2=0x7f78f0064db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:13.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 -- 172.21.15.33:0/2923948313 shutdown_connections 2023-11-06T22:00:13.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:13.895+0000 7f78f4fff700 1 -- 172.21.15.33:0/2923948313 wait complete. 2023-11-06T22:00:13.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T22:00:13.915 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T22:00:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.113+0000 7f6d247ff700 1 Processor -- start 2023-11-06T22:00:14.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.113+0000 7f6d247ff700 1 -- start start 2023-11-06T22:00:14.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.114+0000 7f6d247ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d1c1544f0 0x7f6d1c0c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.114+0000 7f6d247ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6d1c0c0c40 0x7f6d1c0646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.114+0000 7f6d247ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d1c064bb0 0x7f6d1c157f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.114+0000 7f6d247ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c1548d0 con 0x7f6d1c064bb0 2023-11-06T22:00:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.114+0000 7f6d247ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c1556d0 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.114+0000 7f6d247ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c065030 con 0x7f6d1c1544f0 2023-11-06T22:00:14.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d22d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d1c064bb0 0x7f6d1c157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d22d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d1c064bb0 0x7f6d1c157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58450/0 (socket says 172.21.15.33:58450) 2023-11-06T22:00:14.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d22d9c700 1 -- 172.21.15.33:0/1746029549 learned_addr learned my addr 172.21.15.33:0/1746029549 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:14.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d2259b700 1 --2- 172.21.15.33:0/1746029549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d1c1544f0 0x7f6d1c0c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d21d9a700 1 --2- 172.21.15.33:0/1746029549 >> v2:172.21.15.69:3300/0 conn(0x7f6d1c0c0c40 0x7f6d1c0646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d22d9c700 1 -- 172.21.15.33:0/1746029549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d1c1544f0 msgr2=0x7f6d1c0c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d22d9c700 1 --2- 172.21.15.33:0/1746029549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d1c1544f0 0x7f6d1c0c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d22d9c700 1 -- 172.21.15.33:0/1746029549 >> v2:172.21.15.69:3300/0 conn(0x7f6d1c0c0c40 msgr2=0x7f6d1c0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d22d9c700 1 --2- 172.21.15.33:0/1746029549 >> v2:172.21.15.69:3300/0 conn(0x7f6d1c0c0c40 0x7f6d1c0646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d22d9c700 1 -- 172.21.15.33:0/1746029549 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c154ac0 con 0x7f6d1c064bb0 2023-11-06T22:00:14.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d21d9a700 1 --2- 172.21.15.33:0/1746029549 >> v2:172.21.15.69:3300/0 conn(0x7f6d1c0c0c40 0x7f6d1c0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:14.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.115+0000 7f6d22d9c700 1 --2- 172.21.15.33:0/1746029549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c064bb0 0x7f6d1c157f20 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f6d1c06c1a0 tx=0x7f6d14009840).ready entity=mon.0 client_cookie=46358f59b3d9d9e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:14.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d21599700 1 -- 172.21.15.33:0/1746029549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400e070 con 0x7f6d1c064bb0 2023-11-06T22:00:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d21599700 1 -- 172.21.15.33:0/1746029549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1400ba50 con 0x7f6d1c064bb0 2023-11-06T22:00:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d21599700 1 -- 172.21.15.33:0/1746029549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400bdd0 con 0x7f6d1c064bb0 2023-11-06T22:00:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d247ff700 1 -- 172.21.15.33:0/1746029549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c064bb0 msgr2=0x7f6d1c157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d247ff700 1 --2- 172.21.15.33:0/1746029549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c064bb0 0x7f6d1c157f20 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f6d1c06c1a0 tx=0x7f6d14009840).stop 2023-11-06T22:00:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d247ff700 1 -- 172.21.15.33:0/1746029549 shutdown_connections 2023-11-06T22:00:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d247ff700 1 --2- 172.21.15.33:0/1746029549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c064bb0 0x7f6d1c157f20 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d247ff700 1 --2- 172.21.15.33:0/1746029549 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d1c1544f0 0x7f6d1c0c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d247ff700 1 --2- 172.21.15.33:0/1746029549 >> v2:172.21.15.69:3300/0 conn(0x7f6d1c0c0c40 0x7f6d1c0646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d247ff700 1 -- 172.21.15.33:0/1746029549 >> 172.21.15.33:0/1746029549 conn(0x7f6d1c0cb900 msgr2=0x7f6d1c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d247ff700 1 -- 172.21.15.33:0/1746029549 shutdown_connections 2023-11-06T22:00:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.116+0000 7f6d247ff700 1 -- 172.21.15.33:0/1746029549 wait complete. 2023-11-06T22:00:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.118+0000 7f6d247ff700 1 Processor -- start 2023-11-06T22:00:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.118+0000 7f6d247ff700 1 -- start start 2023-11-06T22:00:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.119+0000 7f6d247ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d1c064bb0 0x7f6d1c1413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.119+0000 7f6d247ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c0c0c40 0x7f6d1c1418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.119+0000 7f6d247ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c1544f0 0x7f6d1c0c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.119+0000 7f6d247ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c5250 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.119+0000 7f6d247ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c4690 con 0x7f6d1c1544f0 2023-11-06T22:00:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.119+0000 7f6d247ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c35f0 con 0x7f6d1c064bb0 2023-11-06T22:00:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.119+0000 7f6d21d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c0c0c40 0x7f6d1c1418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.119+0000 7f6d21d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c0c0c40 0x7f6d1c1418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58458/0 (socket says 172.21.15.33:58458) 2023-11-06T22:00:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.119+0000 7f6d21d9a700 1 -- 172.21.15.33:0/3532516763 learned_addr learned my addr 172.21.15.33:0/3532516763 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.119+0000 7f6d22d9c700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c1544f0 0x7f6d1c0c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d2259b700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d1c064bb0 0x7f6d1c1413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d21d9a700 1 -- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d1c064bb0 msgr2=0x7f6d1c1413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d21d9a700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d1c064bb0 0x7f6d1c1413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d21d9a700 1 -- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c1544f0 msgr2=0x7f6d1c0c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d21d9a700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c1544f0 0x7f6d1c0c28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d21d9a700 1 -- 172.21.15.33:0/3532516763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d14009ae0 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d22d9c700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c1544f0 0x7f6d1c0c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d21d9a700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c0c0c40 0x7f6d1c1418b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f6d0c00d890 tx=0x7f6d0c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d127fc700 1 -- 172.21.15.33:0/3532516763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c011590 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d127fc700 1 -- 172.21.15.33:0/3532516763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c0116f0 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d247ff700 1 -- 172.21.15.33:0/3532516763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c0c38d0 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d127fc700 1 -- 172.21.15.33:0/3532516763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c011a70 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.120+0000 7f6d247ff700 1 -- 172.21.15.33:0/3532516763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c0c3e90 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.121+0000 7f6d247ff700 1 -- 172.21.15.33:0/3532516763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c154da0 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.122+0000 7f6d127fc700 1 -- 172.21.15.33:0/3532516763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6d0c012420 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.123+0000 7f6d127fc700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0005f270 0x7f6d00061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.123+0000 7f6d127fc700 1 -- 172.21.15.33:0/3532516763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f6d0c079b90 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.124+0000 7f6d2259b700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0005f270 0x7f6d00061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.124+0000 7f6d2259b700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0005f270 0x7f6d00061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f6d1800bb10 tx=0x7f6d18008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.127+0000 7f6d127fc700 1 -- 172.21.15.33:0/3532516763 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6d0c04afa0 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.309+0000 7f6d247ff700 1 -- 172.21.15.33:0/3532516763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6d1c0c8760 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.310+0000 7f6d127fc700 1 -- 172.21.15.33:0/3532516763 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v181) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6d0c04e5c0 con 0x7f6d1c0c0c40 2023-11-06T22:00:14.312 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T22:00:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 -- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0005f270 msgr2=0x7f6d00061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0005f270 0x7f6d00061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f6d1800bb10 tx=0x7f6d18008040).stop 2023-11-06T22:00:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 -- 172.21.15.33:0/3532516763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c0c0c40 msgr2=0x7f6d1c1418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c0c0c40 0x7f6d1c1418b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f6d0c00d890 tx=0x7f6d0c008040).stop 2023-11-06T22:00:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 -- 172.21.15.33:0/3532516763 shutdown_connections 2023-11-06T22:00:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f6d0005f270 0x7f6d00061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6d1c064bb0 0x7f6d1c1413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c1544f0 0x7f6d1c0c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 --2- 172.21.15.33:0/3532516763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d1c0c0c40 0x7f6d1c1418b0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 -- 172.21.15.33:0/3532516763 >> 172.21.15.33:0/3532516763 conn(0x7f6d1c0cb900 msgr2=0x7f6d1c152ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:14.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 -- 172.21.15.33:0/3532516763 shutdown_connections 2023-11-06T22:00:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.312+0000 7f6d247ff700 1 -- 172.21.15.33:0/3532516763 wait complete. 2023-11-06T22:00:14.332 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T22:00:14.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.529+0000 7f2e199ff700 1 Processor -- start 2023-11-06T22:00:14.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.530+0000 7f2e199ff700 1 -- start start 2023-11-06T22:00:14.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.531+0000 7f2e199ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e140c8500 0x7f2e14062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.531+0000 7f2e199ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e14062c00 0x7f2e14065380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e199ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e14063050 con 0x7f2e141542d0 2023-11-06T22:00:14.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e199ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e14069c80 con 0x7f2e140c8500 2023-11-06T22:00:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e199ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14069f60 con 0x7f2e14062c00 2023-11-06T22:00:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e127fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e140c8500 0x7f2e14062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e127fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e140c8500 0x7f2e14062710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58464/0 (socket says 172.21.15.33:58464) 2023-11-06T22:00:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e127fc700 1 -- 172.21.15.33:0/2297304049 learned_addr learned my addr 172.21.15.33:0/2297304049 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e137fe700 1 --2- 172.21.15.33:0/2297304049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e14062c00 0x7f2e14065380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e127fc700 1 -- 172.21.15.33:0/2297304049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e14062c00 msgr2=0x7f2e14065380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e127fc700 1 --2- 172.21.15.33:0/2297304049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e14062c00 0x7f2e14065380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e127fc700 1 -- 172.21.15.33:0/2297304049 >> v1:172.21.15.69:6789/0 conn(0x7f2e141542d0 legacy=0x7f2e140a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e127fc700 1 -- 172.21.15.33:0/2297304049 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140bcb10 con 0x7f2e140c8500 2023-11-06T22:00:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e137fe700 1 --2- 172.21.15.33:0/2297304049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e14062c00 0x7f2e14065380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:00:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.532+0000 7f2e127fc700 1 --2- 172.21.15.33:0/2297304049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 0x7f2e14062710 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f2e08006fd0 tx=0x7f2e080090f0).ready entity=mon.0 client_cookie=ea193a339416efa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e11ffb700 1 -- 172.21.15.33:0/2297304049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800b600 con 0x7f2e140c8500 2023-11-06T22:00:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e11ffb700 1 -- 172.21.15.33:0/2297304049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0800b760 con 0x7f2e140c8500 2023-11-06T22:00:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e11ffb700 1 -- 172.21.15.33:0/2297304049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800bae0 con 0x7f2e140c8500 2023-11-06T22:00:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e199ff700 1 -- 172.21.15.33:0/2297304049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 msgr2=0x7f2e14062710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e199ff700 1 --2- 172.21.15.33:0/2297304049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 0x7f2e14062710 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f2e08006fd0 tx=0x7f2e080090f0).stop 2023-11-06T22:00:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e199ff700 1 -- 172.21.15.33:0/2297304049 shutdown_connections 2023-11-06T22:00:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e199ff700 1 --2- 172.21.15.33:0/2297304049 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e14062c00 0x7f2e14065380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e199ff700 1 --2- 172.21.15.33:0/2297304049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 0x7f2e14062710 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e199ff700 1 -- 172.21.15.33:0/2297304049 >> 172.21.15.33:0/2297304049 conn(0x7f2e140642a0 msgr2=0x7f2e140c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:14.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e199ff700 1 -- 172.21.15.33:0/2297304049 shutdown_connections 2023-11-06T22:00:14.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.533+0000 7f2e199ff700 1 -- 172.21.15.33:0/2297304049 wait complete. 2023-11-06T22:00:14.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.535+0000 7f2e199ff700 1 Processor -- start 2023-11-06T22:00:14.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.535+0000 7f2e199ff700 1 -- start start 2023-11-06T22:00:14.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.536+0000 7f2e199ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e14062c00 0x7f2e1414ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.536+0000 7f2e199ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 0x7f2e14164b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.536+0000 7f2e199ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e141542d0 0x7f2e140a8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.536+0000 7f2e199ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1414cef0 con 0x7f2e140c8500 2023-11-06T22:00:14.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.536+0000 7f2e199ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14165b00 con 0x7f2e14062c00 2023-11-06T22:00:14.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.536+0000 7f2e199ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e140ac2a0 con 0x7f2e141542d0 2023-11-06T22:00:14.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.536+0000 7f2e127fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 0x7f2e14164b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.536+0000 7f2e127fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 0x7f2e14164b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58478/0 (socket says 172.21.15.33:58478) 2023-11-06T22:00:14.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.536+0000 7f2e127fc700 1 -- 172.21.15.33:0/1745777201 learned_addr learned my addr 172.21.15.33:0/1745777201 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:14.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2e137fe700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e141542d0 0x7f2e140a8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2e12ffd700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e14062c00 0x7f2e1414ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2e127fc700 1 -- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e141542d0 msgr2=0x7f2e140a8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2e127fc700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e141542d0 0x7f2e140a8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2e127fc700 1 -- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e14062c00 msgr2=0x7f2e1414ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2e127fc700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e14062c00 0x7f2e1414ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2e127fc700 1 -- 172.21.15.33:0/1745777201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e08009370 con 0x7f2e140c8500 2023-11-06T22:00:14.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2e127fc700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 0x7f2e14164b20 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f2e1406af50 tx=0x7f2e08017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:14.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2dfaffd700 1 -- 172.21.15.33:0/1745777201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800b8d0 con 0x7f2e140c8500 2023-11-06T22:00:14.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2dfaffd700 1 -- 172.21.15.33:0/1745777201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0800ba30 con 0x7f2e140c8500 2023-11-06T22:00:14.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2e199ff700 1 -- 172.21.15.33:0/1745777201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e14165540 con 0x7f2e140c8500 2023-11-06T22:00:14.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2dfaffd700 1 -- 172.21.15.33:0/1745777201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0801c650 con 0x7f2e140c8500 2023-11-06T22:00:14.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.537+0000 7f2e199ff700 1 -- 172.21.15.33:0/1745777201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e140abf30 con 0x7f2e140c8500 2023-11-06T22:00:14.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.539+0000 7f2e199ff700 1 -- 172.21.15.33:0/1745777201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e140ac410 con 0x7f2e140c8500 2023-11-06T22:00:14.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.539+0000 7f2dfaffd700 1 -- 172.21.15.33:0/1745777201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2e0801c9c0 con 0x7f2e140c8500 2023-11-06T22:00:14.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.540+0000 7f2dfaffd700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2e0005f240 0x7f2e00061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.540+0000 7f2dfaffd700 1 -- 172.21.15.33:0/1745777201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f2e0807be80 con 0x7f2e140c8500 2023-11-06T22:00:14.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.541+0000 7f2e12ffd700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2e0005f240 0x7f2e00061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.541+0000 7f2e12ffd700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2e0005f240 0x7f2e00061700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2e140a5d80 tx=0x7f2e04008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:14.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.544+0000 7f2dfaffd700 1 -- 172.21.15.33:0/1745777201 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2e0804d2f0 con 0x7f2e140c8500 2023-11-06T22:00:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.735+0000 7f2e199ff700 1 -- 172.21.15.33:0/1745777201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2e14063250 con 0x7f2e140c8500 2023-11-06T22:00:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.736+0000 7f2dfaffd700 1 -- 172.21.15.33:0/1745777201 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v181) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2e0802a070 con 0x7f2e140c8500 2023-11-06T22:00:14.739 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T22:00:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 -- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2e0005f240 msgr2=0x7f2e00061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2e0005f240 0x7f2e00061700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2e140a5d80 tx=0x7f2e04008040).stop 2023-11-06T22:00:14.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 -- 172.21.15.33:0/1745777201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 msgr2=0x7f2e14164b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 0x7f2e14164b20 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f2e1406af50 tx=0x7f2e08017000).stop 2023-11-06T22:00:14.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 -- 172.21.15.33:0/1745777201 shutdown_connections 2023-11-06T22:00:14.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2e0005f240 0x7f2e00061700 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2e141542d0 0x7f2e140a8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e14062c00 0x7f2e1414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 --2- 172.21.15.33:0/1745777201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e140c8500 0x7f2e14164b20 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 -- 172.21.15.33:0/1745777201 >> 172.21.15.33:0/1745777201 conn(0x7f2e140642a0 msgr2=0x7f2e140652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:14.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 -- 172.21.15.33:0/1745777201 shutdown_connections 2023-11-06T22:00:14.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.738+0000 7f2e199ff700 1 -- 172.21.15.33:0/1745777201 wait complete. 2023-11-06T22:00:14.758 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T22:00:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.953+0000 7f204cbff700 1 Processor -- start 2023-11-06T22:00:14.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.953+0000 7f204cbff700 1 -- start start 2023-11-06T22:00:14.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f204cbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f204814c640 0x7f20480bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f204cbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2048152ff0 0x7f2048065160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f204cbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20480c33f0 0x7f20480c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f204cbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20480bd8f0 con 0x7f2048152ff0 2023-11-06T22:00:14.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f204cbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20480c3d90 con 0x7f204814c640 2023-11-06T22:00:14.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f204cbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2048153470 con 0x7f20480c33f0 2023-11-06T22:00:14.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f2045d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2048152ff0 0x7f2048065160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f2045d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2048152ff0 0x7f2048065160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58486/0 (socket says 172.21.15.33:58486) 2023-11-06T22:00:14.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f2045d9b700 1 -- 172.21.15.33:0/1142749840 learned_addr learned my addr 172.21.15.33:0/1142749840 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:14.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f2046d9d700 1 --2- 172.21.15.33:0/1142749840 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20480c33f0 0x7f20480c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f204659c700 1 --2- 172.21.15.33:0/1142749840 >> v2:172.21.15.69:3300/0 conn(0x7f204814c640 0x7f20480bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f2045d9b700 1 -- 172.21.15.33:0/1142749840 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20480c33f0 msgr2=0x7f20480c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:14.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f2045d9b700 1 --2- 172.21.15.33:0/1142749840 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20480c33f0 0x7f20480c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f2045d9b700 1 -- 172.21.15.33:0/1142749840 >> v2:172.21.15.69:3300/0 conn(0x7f204814c640 msgr2=0x7f20480bb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:14.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f2045d9b700 1 --2- 172.21.15.33:0/1142749840 >> v2:172.21.15.69:3300/0 conn(0x7f204814c640 0x7f20480bb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.955+0000 7f2045d9b700 1 -- 172.21.15.33:0/1142749840 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20480643a0 con 0x7f2048152ff0 2023-11-06T22:00:14.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.956+0000 7f2045d9b700 1 --2- 172.21.15.33:0/1142749840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 0x7f2048065160 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f2048063bd0 tx=0x7f203c0090f0).ready entity=mon.0 client_cookie=da77fc130cb00713 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:14.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.956+0000 7f204559a700 1 -- 172.21.15.33:0/1142749840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c00b600 con 0x7f2048152ff0 2023-11-06T22:00:14.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.956+0000 7f204559a700 1 -- 172.21.15.33:0/1142749840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c00b760 con 0x7f2048152ff0 2023-11-06T22:00:14.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.956+0000 7f204559a700 1 -- 172.21.15.33:0/1142749840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c00bae0 con 0x7f2048152ff0 2023-11-06T22:00:14.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.956+0000 7f204cbff700 1 -- 172.21.15.33:0/1142749840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 msgr2=0x7f2048065160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.956+0000 7f204cbff700 1 --2- 172.21.15.33:0/1142749840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 0x7f2048065160 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f2048063bd0 tx=0x7f203c0090f0).stop 2023-11-06T22:00:14.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.957+0000 7f204cbff700 1 -- 172.21.15.33:0/1142749840 shutdown_connections 2023-11-06T22:00:14.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.957+0000 7f204cbff700 1 --2- 172.21.15.33:0/1142749840 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20480c33f0 0x7f20480c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.957+0000 7f204cbff700 1 --2- 172.21.15.33:0/1142749840 >> v2:172.21.15.69:3300/0 conn(0x7f204814c640 0x7f20480bb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.957+0000 7f204cbff700 1 --2- 172.21.15.33:0/1142749840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 0x7f2048065160 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.957+0000 7f204cbff700 1 -- 172.21.15.33:0/1142749840 >> 172.21.15.33:0/1142749840 conn(0x7f2048062710 msgr2=0x7f2048062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.957+0000 7f204cbff700 1 -- 172.21.15.33:0/1142749840 shutdown_connections 2023-11-06T22:00:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.957+0000 7f204cbff700 1 -- 172.21.15.33:0/1142749840 wait complete. 2023-11-06T22:00:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.958+0000 7f204cbff700 1 Processor -- start 2023-11-06T22:00:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.958+0000 7f204cbff700 1 -- start start 2023-11-06T22:00:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.959+0000 7f204cbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20480c33f0 0x7f2048069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.959+0000 7f204cbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f204814c640 0x7f204814aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.959+0000 7f204cbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 0x7f20480bfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.959+0000 7f204cbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20480a9cf0 con 0x7f2048152ff0 2023-11-06T22:00:14.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.959+0000 7f204cbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204806a260 con 0x7f204814c640 2023-11-06T22:00:14.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.959+0000 7f204cbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204814b3e0 con 0x7f20480c33f0 2023-11-06T22:00:14.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f2046d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 0x7f20480bfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f2046d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 0x7f20480bfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58496/0 (socket says 172.21.15.33:58496) 2023-11-06T22:00:14.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f2046d9d700 1 -- 172.21.15.33:0/3763004282 learned_addr learned my addr 172.21.15.33:0/3763004282 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:14.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f2045d9b700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f204814c640 0x7f204814aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f204659c700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20480c33f0 0x7f2048069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f2046d9d700 1 -- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20480c33f0 msgr2=0x7f2048069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f2046d9d700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20480c33f0 0x7f2048069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f2046d9d700 1 -- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f204814c640 msgr2=0x7f204814aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f2046d9d700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f204814c640 0x7f204814aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f2046d9d700 1 -- 172.21.15.33:0/3763004282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c009370 con 0x7f2048152ff0 2023-11-06T22:00:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f2046d9d700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 0x7f20480bfc60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f2040009fa0 tx=0x7f2040009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.960+0000 7f20367fc700 1 -- 172.21.15.33:0/3763004282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204000ebc0 con 0x7f2048152ff0 2023-11-06T22:00:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.961+0000 7f20367fc700 1 -- 172.21.15.33:0/3763004282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204000ed20 con 0x7f2048152ff0 2023-11-06T22:00:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.961+0000 7f20367fc700 1 -- 172.21.15.33:0/3763004282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040014610 con 0x7f2048152ff0 2023-11-06T22:00:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.961+0000 7f204cbff700 1 -- 172.21.15.33:0/3763004282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204814def0 con 0x7f2048152ff0 2023-11-06T22:00:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.961+0000 7f204cbff700 1 -- 172.21.15.33:0/3763004282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204814e4b0 con 0x7f2048152ff0 2023-11-06T22:00:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.962+0000 7f204cbff700 1 -- 172.21.15.33:0/3763004282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20480ab540 con 0x7f2048152ff0 2023-11-06T22:00:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.962+0000 7f20367fc700 1 -- 172.21.15.33:0/3763004282 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f204001d3b0 con 0x7f2048152ff0 2023-11-06T22:00:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.963+0000 7f20367fc700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202805f270 0x7f2028061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:14.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.963+0000 7f20367fc700 1 -- 172.21.15.33:0/3763004282 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f204007a4f0 con 0x7f2048152ff0 2023-11-06T22:00:14.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.964+0000 7f204659c700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202805f270 0x7f2028061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:14.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.964+0000 7f204659c700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202805f270 0x7f2028061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f203800bbc0 tx=0x7f2038008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:14.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:14.968+0000 7f20367fc700 1 -- 172.21.15.33:0/3763004282 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f20480ab540 con 0x7f2048152ff0 2023-11-06T22:00:15.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.150+0000 7f204cbff700 1 -- 172.21.15.33:0/3763004282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2048064680 con 0x7f2048152ff0 2023-11-06T22:00:15.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.150+0000 7f20367fc700 1 -- 172.21.15.33:0/3763004282 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v181) v1 ==== 84+0+143 (secure 0 0 0) 0x7f204004b900 con 0x7f2048152ff0 2023-11-06T22:00:15.153 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T22:00:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T22:00:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T22:00:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:00:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T22:00:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T22:00:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:15.155 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T22:00:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 -- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202805f270 msgr2=0x7f2028061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202805f270 0x7f2028061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f203800bbc0 tx=0x7f2038008040).stop 2023-11-06T22:00:15.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 -- 172.21.15.33:0/3763004282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 msgr2=0x7f20480bfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:15.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 0x7f20480bfc60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f2040009fa0 tx=0x7f2040009900).stop 2023-11-06T22:00:15.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 -- 172.21.15.33:0/3763004282 shutdown_connections 2023-11-06T22:00:15.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f202805f270 0x7f2028061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:15.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20480c33f0 0x7f2048069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2048152ff0 0x7f20480bfc60 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 --2- 172.21.15.33:0/3763004282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f204814c640 0x7f204814aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 -- 172.21.15.33:0/3763004282 >> 172.21.15.33:0/3763004282 conn(0x7f2048062710 msgr2=0x7f2048151b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 -- 172.21.15.33:0/3763004282 shutdown_connections 2023-11-06T22:00:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.153+0000 7f204cbff700 1 -- 172.21.15.33:0/3763004282 wait complete. 2023-11-06T22:00:15.172 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T22:00:15.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T22:00:15.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.371+0000 7fb2cdfff700 1 Processor -- start 2023-11-06T22:00:15.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.371+0000 7fb2cdfff700 1 -- start start 2023-11-06T22:00:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.372+0000 7fb2cdfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2c8154720 0x7fb2c80aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.372+0000 7fb2cdfff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb2c8158720 0x7fb2c8063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.372+0000 7fb2cdfff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c80c0e00 con 0x7fb2c8154340 2023-11-06T22:00:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.372+0000 7fb2cdfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c8158bf0 con 0x7fb2c8154720 2023-11-06T22:00:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.372+0000 7fb2cdfff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c8158ed0 con 0x7fb2c8158720 2023-11-06T22:00:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.373+0000 7fb2c6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2c8154720 0x7fb2c80aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.373+0000 7fb2c6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2c8154720 0x7fb2c80aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58504/0 (socket says 172.21.15.33:58504) 2023-11-06T22:00:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.373+0000 7fb2c6ffd700 1 -- 172.21.15.33:0/3260165539 learned_addr learned my addr 172.21.15.33:0/3260165539 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:15.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.373+0000 7fb2c7fff700 1 --2- 172.21.15.33:0/3260165539 >> v2:172.21.15.69:3300/0 conn(0x7fb2c8158720 0x7fb2c8063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:15.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.373+0000 7fb2c6ffd700 1 -- 172.21.15.33:0/3260165539 >> v2:172.21.15.69:3300/0 conn(0x7fb2c8158720 msgr2=0x7fb2c8063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:15.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.373+0000 7fb2c6ffd700 1 --2- 172.21.15.33:0/3260165539 >> v2:172.21.15.69:3300/0 conn(0x7fb2c8158720 0x7fb2c8063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:15.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.373+0000 7fb2c6ffd700 1 -- 172.21.15.33:0/3260165539 >> v1:172.21.15.69:6789/0 conn(0x7fb2c8154340 legacy=0x7fb2c8158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:15.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.373+0000 7fb2c6ffd700 1 -- 172.21.15.33:0/3260165539 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80c5d80 con 0x7fb2c8154720 2023-11-06T22:00:15.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.373+0000 7fb2c6ffd700 1 --2- 172.21.15.33:0/3260165539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8154720 0x7fb2c80aae40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7fb2c80adba0 tx=0x7fb2bc0090f0).ready entity=mon.0 client_cookie=2df708ceb2eae9a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:15.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2c67fc700 1 -- 172.21.15.33:0/3260165539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc00b5a0 con 0x7fb2c8154720 2023-11-06T22:00:15.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2c67fc700 1 -- 172.21.15.33:0/3260165539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc00b700 con 0x7fb2c8154720 2023-11-06T22:00:15.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2c67fc700 1 -- 172.21.15.33:0/3260165539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc00ba80 con 0x7fb2c8154720 2023-11-06T22:00:15.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2cdfff700 1 -- 172.21.15.33:0/3260165539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8154720 msgr2=0x7fb2c80aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2cdfff700 1 --2- 172.21.15.33:0/3260165539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8154720 0x7fb2c80aae40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7fb2c80adba0 tx=0x7fb2bc0090f0).stop 2023-11-06T22:00:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2cdfff700 1 -- 172.21.15.33:0/3260165539 shutdown_connections 2023-11-06T22:00:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2cdfff700 1 --2- 172.21.15.33:0/3260165539 >> v2:172.21.15.69:3300/0 conn(0x7fb2c8158720 0x7fb2c8063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2cdfff700 1 --2- 172.21.15.33:0/3260165539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8154720 0x7fb2c80aae40 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2cdfff700 1 -- 172.21.15.33:0/3260165539 >> 172.21.15.33:0/3260165539 conn(0x7fb2c805ec80 msgr2=0x7fb2c80cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:15.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2cdfff700 1 -- 172.21.15.33:0/3260165539 shutdown_connections 2023-11-06T22:00:15.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.374+0000 7fb2cdfff700 1 -- 172.21.15.33:0/3260165539 wait complete. 2023-11-06T22:00:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.376+0000 7fb2cdfff700 1 Processor -- start 2023-11-06T22:00:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.376+0000 7fb2cdfff700 1 -- start start 2023-11-06T22:00:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.377+0000 7fb2cdfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2c8154340 0x7fb2c81680f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.377+0000 7fb2cdfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c8154720 0x7fb2c8157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.377+0000 7fb2cdfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8158720 0x7fb2c8165300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.377+0000 7fb2cdfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8164b20 con 0x7fb2c8158720 2023-11-06T22:00:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.377+0000 7fb2cdfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8168920 con 0x7fb2c8154720 2023-11-06T22:00:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.377+0000 7fb2cdfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8168c00 con 0x7fb2c8154340 2023-11-06T22:00:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.377+0000 7fb2c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8158720 0x7fb2c8165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:15.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.377+0000 7fb2c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8158720 0x7fb2c8165300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58518/0 (socket says 172.21.15.33:58518) 2023-11-06T22:00:15.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.377+0000 7fb2c7fff700 1 -- 172.21.15.33:0/463860581 learned_addr learned my addr 172.21.15.33:0/463860581 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:15.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2c6ffd700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c8154720 0x7fb2c8157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:15.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2c77fe700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2c8154340 0x7fb2c81680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:15.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2c7fff700 1 -- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2c8154340 msgr2=0x7fb2c81680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2c7fff700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2c8154340 0x7fb2c81680f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2c7fff700 1 -- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c8154720 msgr2=0x7fb2c8157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2c7fff700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c8154720 0x7fb2c8157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2c7fff700 1 -- 172.21.15.33:0/463860581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc009370 con 0x7fb2c8158720 2023-11-06T22:00:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2c6ffd700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c8154720 0x7fb2c8157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2c7fff700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8158720 0x7fb2c8165300 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fb2c0007f20 tx=0x7fb2c0009800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2af7fe700 1 -- 172.21.15.33:0/463860581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c000ec40 con 0x7fb2c8158720 2023-11-06T22:00:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2af7fe700 1 -- 172.21.15.33:0/463860581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c000eda0 con 0x7fb2c8158720 2023-11-06T22:00:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2cdfff700 1 -- 172.21.15.33:0/463860581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c8168ee0 con 0x7fb2c8158720 2023-11-06T22:00:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2af7fe700 1 -- 172.21.15.33:0/463860581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0014610 con 0x7fb2c8158720 2023-11-06T22:00:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.378+0000 7fb2cdfff700 1 -- 172.21.15.33:0/463860581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c81685e0 con 0x7fb2c8158720 2023-11-06T22:00:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.380+0000 7fb2cdfff700 1 -- 172.21.15.33:0/463860581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c8165020 con 0x7fb2c8158720 2023-11-06T22:00:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.380+0000 7fb2af7fe700 1 -- 172.21.15.33:0/463860581 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb2c001d3b0 con 0x7fb2c8158720 2023-11-06T22:00:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.381+0000 7fb2af7fe700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2b405f2a0 0x7fb2b4061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.381+0000 7fb2af7fe700 1 -- 172.21.15.33:0/463860581 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5313+0+0 (secure 0 0 0) 0x7fb2c007a630 con 0x7fb2c8158720 2023-11-06T22:00:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.381+0000 7fb2c77fe700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2b405f2a0 0x7fb2b4061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.382+0000 7fb2c77fe700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2b405f2a0 0x7fb2b4061760 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb2b80036e0 tx=0x7fb2b8009040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.385+0000 7fb2af7fe700 1 -- 172.21.15.33:0/463860581 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb2c8165020 con 0x7fb2c8158720 2023-11-06T22:00:15.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:15.570+0000 7fb2cdfff700 1 -- 172.21.15.33:0/463860581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fb2c8165020 con 0x7fb2c8158720 2023-11-06T22:00:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.575+0000 7fb2af7fe700 1 -- 172.21.15.33:0/463860581 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v182) v1 ==== 152+0+0 (secure 0 0 0) 0x7fb2c8165020 con 0x7fb2c8158720 2023-11-06T22:00:16.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.582+0000 7fb2cdfff700 1 -- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2b405f2a0 msgr2=0x7fb2b4061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:16.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.582+0000 7fb2cdfff700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2b405f2a0 0x7fb2b4061760 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb2b80036e0 tx=0x7fb2b8009040).stop 2023-11-06T22:00:16.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.582+0000 7fb2cdfff700 1 -- 172.21.15.33:0/463860581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8158720 msgr2=0x7fb2c8165300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:16.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.582+0000 7fb2cdfff700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8158720 0x7fb2c8165300 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fb2c0007f20 tx=0x7fb2c0009800).stop 2023-11-06T22:00:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.583+0000 7fb2cdfff700 1 -- 172.21.15.33:0/463860581 shutdown_connections 2023-11-06T22:00:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.583+0000 7fb2cdfff700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb2b405f2a0 0x7fb2b4061760 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.583+0000 7fb2cdfff700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb2c8154340 0x7fb2c81680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.583+0000 7fb2cdfff700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8158720 0x7fb2c8165300 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.583+0000 7fb2cdfff700 1 --2- 172.21.15.33:0/463860581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c8154720 0x7fb2c8157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.583+0000 7fb2cdfff700 1 -- 172.21.15.33:0/463860581 >> 172.21.15.33:0/463860581 conn(0x7fb2c805ec80 msgr2=0x7fb2c814bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.583+0000 7fb2cdfff700 1 -- 172.21.15.33:0/463860581 shutdown_connections 2023-11-06T22:00:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.583+0000 7fb2cdfff700 1 -- 172.21.15.33:0/463860581 wait complete. 2023-11-06T22:00:16.590 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T22:00:16.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T22:00:16.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.814+0000 7f726cfff700 1 Processor -- start 2023-11-06T22:00:16.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.814+0000 7f726cfff700 1 -- start start 2023-11-06T22:00:16.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.816+0000 7f726cfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7268154330 0x7f72680c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:16.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.816+0000 7f726cfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72680c6b60 con 0x7f7268154710 2023-11-06T22:00:16.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.816+0000 7f726cfff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72680c7700 con 0x7f72680c7ff0 2023-11-06T22:00:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.816+0000 7f726cfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72680c5cd0 con 0x7f7268154330 2023-11-06T22:00:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.816+0000 7f726659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7268154330 0x7f72680c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.816+0000 7f726659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7268154330 0x7f72680c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58522/0 (socket says 172.21.15.33:58522) 2023-11-06T22:00:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.816+0000 7f726659c700 1 -- 172.21.15.33:0/3956897690 learned_addr learned my addr 172.21.15.33:0/3956897690 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:16.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.817+0000 7f726659c700 1 -- 172.21.15.33:0/3956897690 >> v1:172.21.15.69:6789/0 conn(0x7f72680c7ff0 legacy=0x7f72680646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:16.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.817+0000 7f726659c700 1 -- 172.21.15.33:0/3956897690 >> v1:172.21.15.33:6789/0 conn(0x7f7268154710 legacy=0x7f72680c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:16.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.817+0000 7f726659c700 1 -- 172.21.15.33:0/3956897690 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72680c5f50 con 0x7f7268154330 2023-11-06T22:00:16.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.817+0000 7f726559a700 1 -- 172.21.15.33:0/3956897690 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048787733 0 0) 0x7f72680c6b60 con 0x7f7268154710 2023-11-06T22:00:16.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.817+0000 7f726659c700 1 --2- 172.21.15.33:0/3956897690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268154330 0x7f72680c57e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f7258006fd0 tx=0x7f72580090f0).ready entity=mon.0 client_cookie=a6af20b7cc6013b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:16.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.817+0000 7f726559a700 1 -- 172.21.15.33:0/3956897690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800b600 con 0x7f7268154330 2023-11-06T22:00:16.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.817+0000 7f726559a700 1 -- 172.21.15.33:0/3956897690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725800b760 con 0x7f7268154330 2023-11-06T22:00:16.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.817+0000 7f726559a700 1 -- 172.21.15.33:0/3956897690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800bae0 con 0x7f7268154330 2023-11-06T22:00:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.818+0000 7f726cfff700 1 -- 172.21.15.33:0/3956897690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268154330 msgr2=0x7f72680c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.818+0000 7f726cfff700 1 --2- 172.21.15.33:0/3956897690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268154330 0x7f72680c57e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f7258006fd0 tx=0x7f72580090f0).stop 2023-11-06T22:00:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.818+0000 7f726cfff700 1 -- 172.21.15.33:0/3956897690 shutdown_connections 2023-11-06T22:00:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.818+0000 7f726cfff700 1 --2- 172.21.15.33:0/3956897690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268154330 0x7f72680c57e0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.818+0000 7f726cfff700 1 -- 172.21.15.33:0/3956897690 >> 172.21.15.33:0/3956897690 conn(0x7f72680cb910 msgr2=0x7f72680c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.818+0000 7f726cfff700 1 -- 172.21.15.33:0/3956897690 shutdown_connections 2023-11-06T22:00:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.818+0000 7f726cfff700 1 -- 172.21.15.33:0/3956897690 wait complete. 2023-11-06T22:00:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.819+0000 7f726cfff700 1 Processor -- start 2023-11-06T22:00:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.819+0000 7f726cfff700 1 -- start start 2023-11-06T22:00:16.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.820+0000 7f726cfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72680c7ff0 0x7f72680ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:16.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.820+0000 7f726cfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7268154330 0x7f72680ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:16.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.820+0000 7f726cfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7268154710 0x7f7268164de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:16.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.820+0000 7f726cfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680caae0 con 0x7f72680c7ff0 2023-11-06T22:00:16.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.820+0000 7f726cfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7268168400 con 0x7f7268154710 2023-11-06T22:00:16.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.820+0000 7f726cfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72681686e0 con 0x7f7268154330 2023-11-06T22:00:16.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.820+0000 7f726659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72680c7ff0 0x7f72680ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:16.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f726659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72680c7ff0 0x7f72680ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58526/0 (socket says 172.21.15.33:58526) 2023-11-06T22:00:16.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f726659c700 1 -- 172.21.15.33:0/3604372508 learned_addr learned my addr 172.21.15.33:0/3604372508 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:16.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f7266d9d700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7268154710 0x7f7268164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:16.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f7265d9b700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7268154330 0x7f72680ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:16.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f726659c700 1 -- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7268154330 msgr2=0x7f72680ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f726659c700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7268154330 0x7f72680ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f726659c700 1 -- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7268154710 msgr2=0x7f7268164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f726659c700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7268154710 0x7f7268164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f726659c700 1 -- 172.21.15.33:0/3604372508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7258009370 con 0x7f72680c7ff0 2023-11-06T22:00:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f7266d9d700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7268154710 0x7f7268164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f726659c700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72680c7ff0 0x7f72680ca100 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f7258009130 tx=0x7f7258017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f72567fc700 1 -- 172.21.15.33:0/3604372508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800bae0 con 0x7f72680c7ff0 2023-11-06T22:00:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f72567fc700 1 -- 172.21.15.33:0/3604372508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7258000c90 con 0x7f72680c7ff0 2023-11-06T22:00:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f726cfff700 1 -- 172.21.15.33:0/3604372508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72681689c0 con 0x7f72680c7ff0 2023-11-06T22:00:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.821+0000 7f72567fc700 1 -- 172.21.15.33:0/3604372508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725801c670 con 0x7f72680c7ff0 2023-11-06T22:00:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.822+0000 7f726cfff700 1 -- 172.21.15.33:0/3604372508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72681578f0 con 0x7f72680c7ff0 2023-11-06T22:00:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.823+0000 7f726cfff700 1 -- 172.21.15.33:0/3604372508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7268165d70 con 0x7f72680c7ff0 2023-11-06T22:00:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.824+0000 7f72567fc700 1 -- 172.21.15.33:0/3604372508 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f725801c9e0 con 0x7f72680c7ff0 2023-11-06T22:00:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.825+0000 7f72567fc700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724805f270 0x7f7248061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.825+0000 7f72567fc700 1 -- 172.21.15.33:0/3604372508 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5683+0+0 (secure 0 0 0) 0x7f725807c0c0 con 0x7f72680c7ff0 2023-11-06T22:00:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.825+0000 7f7265d9b700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724805f270 0x7f7248061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.826+0000 7f7265d9b700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724805f270 0x7f7248061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f72680a3080 tx=0x7f725c009040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:16.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:16.829+0000 7f72567fc700 1 -- 172.21.15.33:0/3604372508 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f725804d3c0 con 0x7f72680c7ff0 2023-11-06T22:00:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.008+0000 7f726cfff700 1 -- 172.21.15.33:0/3604372508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f72680830b0 con 0x7f72680c7ff0 2023-11-06T22:00:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.581+0000 7f72567fc700 1 -- 172.21.15.33:0/3604372508 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v183) v1 ==== 170+0+0 (secure 0 0 0) 0x7f725802a020 con 0x7f72680c7ff0 2023-11-06T22:00:17.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.584+0000 7f726cfff700 1 -- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724805f270 msgr2=0x7f7248061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:17.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.584+0000 7f726cfff700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724805f270 0x7f7248061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f72680a3080 tx=0x7f725c009040).stop 2023-11-06T22:00:17.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.584+0000 7f726cfff700 1 -- 172.21.15.33:0/3604372508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72680c7ff0 msgr2=0x7f72680ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.584+0000 7f726cfff700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72680c7ff0 0x7f72680ca100 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f7258009130 tx=0x7f7258017000).stop 2023-11-06T22:00:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.585+0000 7f726cfff700 1 -- 172.21.15.33:0/3604372508 shutdown_connections 2023-11-06T22:00:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.585+0000 7f726cfff700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f724805f270 0x7f7248061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.585+0000 7f726cfff700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7268154710 0x7f7268164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.585+0000 7f726cfff700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72680c7ff0 0x7f72680ca100 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.585+0000 7f726cfff700 1 --2- 172.21.15.33:0/3604372508 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7268154330 0x7f72680ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.585+0000 7f726cfff700 1 -- 172.21.15.33:0/3604372508 >> 172.21.15.33:0/3604372508 conn(0x7f72680cb910 msgr2=0x7f72680aaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.585+0000 7f726cfff700 1 -- 172.21.15.33:0/3604372508 shutdown_connections 2023-11-06T22:00:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.585+0000 7f726cfff700 1 -- 172.21.15.33:0/3604372508 wait complete. 2023-11-06T22:00:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T22:00:17.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T22:00:17.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.829+0000 7fe4839ff700 1 Processor -- start 2023-11-06T22:00:17.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.829+0000 7fe4839ff700 1 -- start start 2023-11-06T22:00:17.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.830+0000 7fe4839ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe47c154330 0x7fe47c0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:17.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.830+0000 7fe4839ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe47c154710 0x7fe47c0c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:17.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.830+0000 7fe4839ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47c0c6b60 con 0x7fe47c0c7ff0 2023-11-06T22:00:17.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.830+0000 7fe4839ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47c0c5cd0 con 0x7fe47c154710 2023-11-06T22:00:17.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.830+0000 7fe4839ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47c0c5f50 con 0x7fe47c154330 2023-11-06T22:00:17.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.830+0000 7fe480f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe47c154710 0x7fe47c0c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.831+0000 7fe480f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe47c154710 0x7fe47c0c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58540/0 (socket says 172.21.15.33:58540) 2023-11-06T22:00:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.831+0000 7fe480f9a700 1 -- 172.21.15.33:0/1851984034 learned_addr learned my addr 172.21.15.33:0/1851984034 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.831+0000 7fe48179b700 1 --2- 172.21.15.33:0/1851984034 >> v2:172.21.15.69:3300/0 conn(0x7fe47c154330 0x7fe47c0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.831+0000 7fe480f9a700 1 -- 172.21.15.33:0/1851984034 >> v2:172.21.15.69:3300/0 conn(0x7fe47c154330 msgr2=0x7fe47c0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.831+0000 7fe480f9a700 1 --2- 172.21.15.33:0/1851984034 >> v2:172.21.15.69:3300/0 conn(0x7fe47c154330 0x7fe47c0c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.831+0000 7fe480f9a700 1 -- 172.21.15.33:0/1851984034 >> v1:172.21.15.69:6789/0 conn(0x7fe47c0c7ff0 legacy=0x7fe47c0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.831+0000 7fe480f9a700 1 -- 172.21.15.33:0/1851984034 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c0c6230 con 0x7fe47c154710 2023-11-06T22:00:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.831+0000 7fe480f9a700 1 --2- 172.21.15.33:0/1851984034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154710 0x7fe47c0c6ff0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7fe47c0605d0 tx=0x7fe46c0090f0).ready entity=mon.0 client_cookie=4a03ca5655a3d160 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.831+0000 7fe473fff700 1 -- 172.21.15.33:0/1851984034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00b520 con 0x7fe47c154710 2023-11-06T22:00:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.831+0000 7fe473fff700 1 -- 172.21.15.33:0/1851984034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c00b680 con 0x7fe47c154710 2023-11-06T22:00:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.832+0000 7fe473fff700 1 -- 172.21.15.33:0/1851984034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00ba00 con 0x7fe47c154710 2023-11-06T22:00:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.832+0000 7fe4839ff700 1 -- 172.21.15.33:0/1851984034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154710 msgr2=0x7fe47c0c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.832+0000 7fe4839ff700 1 --2- 172.21.15.33:0/1851984034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154710 0x7fe47c0c6ff0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7fe47c0605d0 tx=0x7fe46c0090f0).stop 2023-11-06T22:00:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.832+0000 7fe4839ff700 1 -- 172.21.15.33:0/1851984034 shutdown_connections 2023-11-06T22:00:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.832+0000 7fe4839ff700 1 --2- 172.21.15.33:0/1851984034 >> v2:172.21.15.69:3300/0 conn(0x7fe47c154330 0x7fe47c0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.832+0000 7fe4839ff700 1 --2- 172.21.15.33:0/1851984034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154710 0x7fe47c0c6ff0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.832+0000 7fe4839ff700 1 -- 172.21.15.33:0/1851984034 >> 172.21.15.33:0/1851984034 conn(0x7fe47c0cb910 msgr2=0x7fe47c0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.832+0000 7fe4839ff700 1 -- 172.21.15.33:0/1851984034 shutdown_connections 2023-11-06T22:00:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.832+0000 7fe4839ff700 1 -- 172.21.15.33:0/1851984034 wait complete. 2023-11-06T22:00:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.834+0000 7fe4839ff700 1 Processor -- start 2023-11-06T22:00:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.834+0000 7fe4839ff700 1 -- start start 2023-11-06T22:00:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe4839ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe47c0c7ff0 0x7fe47c157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe4839ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154330 0x7fe47c0a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe4839ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe47c154710 0x7fe47c151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe4839ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0addd0 con 0x7fe47c154330 2023-11-06T22:00:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe4839ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0aa060 con 0x7fe47c154710 2023-11-06T22:00:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe4839ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c151af0 con 0x7fe47c0c7ff0 2023-11-06T22:00:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe480f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154330 0x7fe47c0a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe480f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154330 0x7fe47c0a9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58556/0 (socket says 172.21.15.33:58556) 2023-11-06T22:00:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe480f9a700 1 -- 172.21.15.33:0/953077735 learned_addr learned my addr 172.21.15.33:0/953077735 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe481f9c700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe47c154710 0x7fe47c151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe48179b700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe47c0c7ff0 0x7fe47c157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe480f9a700 1 -- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe47c0c7ff0 msgr2=0x7fe47c157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.836+0000 7fe480f9a700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe47c0c7ff0 0x7fe47c157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe480f9a700 1 -- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe47c154710 msgr2=0x7fe47c151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe480f9a700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe47c154710 0x7fe47c151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe480f9a700 1 -- 172.21.15.33:0/953077735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c009370 con 0x7fe47c154330 2023-11-06T22:00:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe48179b700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe47c0c7ff0 0x7fe47c157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:00:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe481f9c700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe47c154710 0x7fe47c151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:00:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe480f9a700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154330 0x7fe47c0a9b70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7fe47c06c170 tx=0x7fe46c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe4717fa700 1 -- 172.21.15.33:0/953077735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00b7f0 con 0x7fe47c154330 2023-11-06T22:00:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe4717fa700 1 -- 172.21.15.33:0/953077735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c00b950 con 0x7fe47c154330 2023-11-06T22:00:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe4839ff700 1 -- 172.21.15.33:0/953077735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c0ae910 con 0x7fe47c154330 2023-11-06T22:00:17.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe4717fa700 1 -- 172.21.15.33:0/953077735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c01d660 con 0x7fe47c154330 2023-11-06T22:00:17.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.837+0000 7fe4839ff700 1 -- 172.21.15.33:0/953077735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47c0ae2c0 con 0x7fe47c154330 2023-11-06T22:00:17.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.838+0000 7fe4839ff700 1 -- 172.21.15.33:0/953077735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47c0aeaa0 con 0x7fe47c154330 2023-11-06T22:00:17.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.839+0000 7fe4717fa700 1 -- 172.21.15.33:0/953077735 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe46c01da50 con 0x7fe47c154330 2023-11-06T22:00:17.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.841+0000 7fe4717fa700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46005f270 0x7fe460061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:17.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.841+0000 7fe4717fa700 1 -- 172.21.15.33:0/953077735 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6065+0+0 (secure 0 0 0) 0x7fe46c07bfb0 con 0x7fe47c154330 2023-11-06T22:00:17.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.841+0000 7fe48179b700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46005f270 0x7fe460061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:17.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.842+0000 7fe48179b700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46005f270 0x7fe460061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fe478006d80 tx=0x7fe478008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:17.846+0000 7fe4717fa700 1 -- 172.21.15.33:0/953077735 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe46c04d130 con 0x7fe47c154330 2023-11-06T22:00:18.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.067+0000 7fe4839ff700 1 -- 172.21.15.33:0/953077735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe47c0830b0 con 0x7fe47c154330 2023-11-06T22:00:18.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.072+0000 7fe4717fa700 1 -- 172.21.15.33:0/953077735 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v72) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe46c021070 con 0x7fe47c154330 2023-11-06T22:00:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 -- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46005f270 msgr2=0x7fe460061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46005f270 0x7fe460061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fe478006d80 tx=0x7fe478008040).stop 2023-11-06T22:00:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 -- 172.21.15.33:0/953077735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154330 msgr2=0x7fe47c0a9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154330 0x7fe47c0a9b70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7fe47c06c170 tx=0x7fe46c017000).stop 2023-11-06T22:00:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 -- 172.21.15.33:0/953077735 shutdown_connections 2023-11-06T22:00:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fe46005f270 0x7fe460061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe47c0c7ff0 0x7fe47c157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe47c154710 0x7fe47c151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 --2- 172.21.15.33:0/953077735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe47c154330 0x7fe47c0a9b70 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 -- 172.21.15.33:0/953077735 >> 172.21.15.33:0/953077735 conn(0x7fe47c0cb910 msgr2=0x7fe47c0646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:18.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 -- 172.21.15.33:0/953077735 shutdown_connections 2023-11-06T22:00:18.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.076+0000 7fe4839ff700 1 -- 172.21.15.33:0/953077735 wait complete. 2023-11-06T22:00:18.081 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 21 and data pool 22 2023-11-06T22:00:18.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:00:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.310+0000 7fbad09ff700 1 Processor -- start 2023-11-06T22:00:18.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.310+0000 7fbad09ff700 1 -- start start 2023-11-06T22:00:18.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbad09ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbacc0c1840 0x7fbacc0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbad09ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbacc0aa2d0 0x7fbacc0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbad09ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbacc1554a0 con 0x7fbacc0c6240 2023-11-06T22:00:18.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbad09ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc154960 con 0x7fbacc0c1840 2023-11-06T22:00:18.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbad09ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc155930 con 0x7fbacc0aa2d0 2023-11-06T22:00:18.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbaca59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbacc0c1840 0x7fbacc0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbacad9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbacc0c6240 0x7fbacc157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51992/0 (socket says 172.21.15.33:51992) 2023-11-06T22:00:18.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbaca59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbacc0c1840 0x7fbacc0a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44990/0 (socket says 172.21.15.33:44990) 2023-11-06T22:00:18.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbacad9d700 1 -- 172.21.15.33:0/420788348 learned_addr learned my addr 172.21.15.33:0/420788348 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:18.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbac9d9b700 1 --2- 172.21.15.33:0/420788348 >> v2:172.21.15.69:3300/0 conn(0x7fbacc0aa2d0 0x7fbacc0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbaca59c700 1 -- 172.21.15.33:0/420788348 >> v2:172.21.15.69:3300/0 conn(0x7fbacc0aa2d0 msgr2=0x7fbacc0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:18.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbaca59c700 1 --2- 172.21.15.33:0/420788348 >> v2:172.21.15.69:3300/0 conn(0x7fbacc0aa2d0 0x7fbacc0c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbaca59c700 1 -- 172.21.15.33:0/420788348 >> v1:172.21.15.33:6789/0 conn(0x7fbacc0c6240 legacy=0x7fbacc157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbaca59c700 1 -- 172.21.15.33:0/420788348 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc154b70 con 0x7fbacc0c1840 2023-11-06T22:00:18.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.313+0000 7fbac9d9b700 1 --2- 172.21.15.33:0/420788348 >> v2:172.21.15.69:3300/0 conn(0x7fbacc0aa2d0 0x7fbacc0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:18.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.312+0000 7fbac959a700 1 -- 172.21.15.33:0/420788348 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820425838 0 0) 0x7fbacc1554a0 con 0x7fbacc0c6240 2023-11-06T22:00:18.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.313+0000 7fbaca59c700 1 --2- 172.21.15.33:0/420788348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c1840 0x7fbacc0a9de0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7fbacc0adb40 tx=0x7fbabc009060).ready entity=mon.0 client_cookie=a2a8a35d505a83eb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:18.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.313+0000 7fbac959a700 1 -- 172.21.15.33:0/420788348 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00b580 con 0x7fbacc0c1840 2023-11-06T22:00:18.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.313+0000 7fbac959a700 1 -- 172.21.15.33:0/420788348 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc00b6e0 con 0x7fbacc0c1840 2023-11-06T22:00:18.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.313+0000 7fbac959a700 1 -- 172.21.15.33:0/420788348 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00ba60 con 0x7fbacc0c1840 2023-11-06T22:00:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.313+0000 7fbad09ff700 1 -- 172.21.15.33:0/420788348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c1840 msgr2=0x7fbacc0a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.313+0000 7fbad09ff700 1 --2- 172.21.15.33:0/420788348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c1840 0x7fbacc0a9de0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7fbacc0adb40 tx=0x7fbabc009060).stop 2023-11-06T22:00:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.314+0000 7fbad09ff700 1 -- 172.21.15.33:0/420788348 shutdown_connections 2023-11-06T22:00:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.314+0000 7fbad09ff700 1 --2- 172.21.15.33:0/420788348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c1840 0x7fbacc0a9de0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.314+0000 7fbad09ff700 1 --2- 172.21.15.33:0/420788348 >> v2:172.21.15.69:3300/0 conn(0x7fbacc0aa2d0 0x7fbacc0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.314+0000 7fbad09ff700 1 -- 172.21.15.33:0/420788348 >> 172.21.15.33:0/420788348 conn(0x7fbacc05ec80 msgr2=0x7fbacc0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.314+0000 7fbad09ff700 1 -- 172.21.15.33:0/420788348 shutdown_connections 2023-11-06T22:00:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.314+0000 7fbad09ff700 1 -- 172.21.15.33:0/420788348 wait complete. 2023-11-06T22:00:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.315+0000 7fbad09ff700 1 Processor -- start 2023-11-06T22:00:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.315+0000 7fbad09ff700 1 -- start start 2023-11-06T22:00:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.316+0000 7fbad09ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbacc0aa2d0 0x7fbacc1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.316+0000 7fbad09ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbacc0c1840 0x7fbacc0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.316+0000 7fbad09ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c6240 0x7fbacc0b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.316+0000 7fbad09ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0aee30 con 0x7fbacc0c6240 2023-11-06T22:00:18.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.316+0000 7fbad09ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0af110 con 0x7fbacc0aa2d0 2023-11-06T22:00:18.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.316+0000 7fbad09ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0af3f0 con 0x7fbacc0c1840 2023-11-06T22:00:18.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.316+0000 7fbacad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c6240 0x7fbacc0b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbaca59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbacc0aa2d0 0x7fbacc1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbacad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c6240 0x7fbacc0b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44992/0 (socket says 172.21.15.33:44992) 2023-11-06T22:00:18.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbacad9d700 1 -- 172.21.15.33:0/1660469574 learned_addr learned my addr 172.21.15.33:0/1660469574 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:18.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbac9d9b700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbacc0c1840 0x7fbacc0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbacad9d700 1 -- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbacc0c1840 msgr2=0x7fbacc0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbacad9d700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbacc0c1840 0x7fbacc0afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbacad9d700 1 -- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbacc0aa2d0 msgr2=0x7fbacc1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbacad9d700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbacc0aa2d0 0x7fbacc1504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbacad9d700 1 -- 172.21.15.33:0/1660469574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc0092a0 con 0x7fbacc0c6240 2023-11-06T22:00:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbaca59c700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbacc0aa2d0 0x7fbacc1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:00:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbacad9d700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c6240 0x7fbacc0b03c0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7fbac400c7e0 tx=0x7fbac4007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbaba7fc700 1 -- 172.21.15.33:0/1660469574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4005d30 con 0x7fbacc0c6240 2023-11-06T22:00:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.317+0000 7fbaba7fc700 1 -- 172.21.15.33:0/1660469574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac4005e90 con 0x7fbacc0c6240 2023-11-06T22:00:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.318+0000 7fbaba7fc700 1 -- 172.21.15.33:0/1660469574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4007320 con 0x7fbacc0c6240 2023-11-06T22:00:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.318+0000 7fbad09ff700 1 -- 172.21.15.33:0/1660469574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc0af6d0 con 0x7fbacc0c6240 2023-11-06T22:00:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.318+0000 7fbad09ff700 1 -- 172.21.15.33:0/1660469574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc0c2130 con 0x7fbacc0c6240 2023-11-06T22:00:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.319+0000 7fbad09ff700 1 -- 172.21.15.33:0/1660469574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbacc06c6c0 con 0x7fbacc0c6240 2023-11-06T22:00:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.320+0000 7fbaba7fc700 1 -- 172.21.15.33:0/1660469574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbac4007710 con 0x7fbacc0c6240 2023-11-06T22:00:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.320+0000 7fbaba7fc700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbaac05f270 0x7fbaac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.321+0000 7fbaba7fc700 1 -- 172.21.15.33:0/1660469574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fbac407a310 con 0x7fbacc0c6240 2023-11-06T22:00:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.321+0000 7fbaca59c700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbaac05f270 0x7fbaac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.322+0000 7fbaca59c700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbaac05f270 0x7fbaac061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fbabc009fd0 tx=0x7fbabc017000).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.325+0000 7fbaba7fc700 1 -- 172.21.15.33:0/1660469574 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbac404b3d0 con 0x7fbacc0c6240 2023-11-06T22:00:18.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.507+0000 7fbad09ff700 1 -- 172.21.15.33:0/1660469574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbacc063f20 con 0x7fbacc0c6240 2023-11-06T22:00:18.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.508+0000 7fbaba7fc700 1 -- 172.21.15.33:0/1660469574 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+17074 (secure 0 0 0) 0x7fbac404e9f0 con 0x7fbacc0c6240 2023-11-06T22:00:18.511 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:18.511 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":184,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:00:18.069425+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:00:15.624386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T22:00:17.062698+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:00:18.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 -- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbaac05f270 msgr2=0x7fbaac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbaac05f270 0x7fbaac061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fbabc009fd0 tx=0x7fbabc017000).stop 2023-11-06T22:00:18.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 -- 172.21.15.33:0/1660469574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c6240 msgr2=0x7fbacc0b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c6240 0x7fbacc0b03c0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7fbac400c7e0 tx=0x7fbac4007f90).stop 2023-11-06T22:00:18.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 -- 172.21.15.33:0/1660469574 shutdown_connections 2023-11-06T22:00:18.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fbaac05f270 0x7fbaac061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0c6240 0x7fbacc0b03c0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbacc0aa2d0 0x7fbacc1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 --2- 172.21.15.33:0/1660469574 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbacc0c1840 0x7fbacc0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 -- 172.21.15.33:0/1660469574 >> 172.21.15.33:0/1660469574 conn(0x7fbacc05ec80 msgr2=0x7fbacc151b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:18.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 -- 172.21.15.33:0/1660469574 shutdown_connections 2023-11-06T22:00:18.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.511+0000 7fbad09ff700 1 -- 172.21.15.33:0/1660469574 wait complete. 2023-11-06T22:00:18.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:00:18.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.727+0000 7f525bdff700 1 Processor -- start 2023-11-06T22:00:18.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.727+0000 7f525bdff700 1 -- start start 2023-11-06T22:00:18.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.728+0000 7f525bdff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f52540c1840 0x7f52540a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.728+0000 7f525bdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c6460 0x7f52540bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.728+0000 7f525bdff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52541554a0 con 0x7f52540aa2d0 2023-11-06T22:00:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.728+0000 7f525bdff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5254154960 con 0x7f52540c1840 2023-11-06T22:00:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.728+0000 7f525bdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5254155930 con 0x7f52540c6460 2023-11-06T22:00:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.728+0000 7f5259b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f52540c1840 0x7f52540a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f5259b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f52540c1840 0x7f52540a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44994/0 (socket says 172.21.15.33:44994) 2023-11-06T22:00:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f5259b9b700 1 -- 172.21.15.33:0/979148586 learned_addr learned my addr 172.21.15.33:0/979148586 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f525a39c700 1 --2- 172.21.15.33:0/979148586 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c6460 0x7f52540bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f5259b9b700 1 -- 172.21.15.33:0/979148586 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c6460 msgr2=0x7f52540bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f5259b9b700 1 --2- 172.21.15.33:0/979148586 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c6460 0x7f52540bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f5259b9b700 1 -- 172.21.15.33:0/979148586 >> v1:172.21.15.69:6789/0 conn(0x7f52540aa2d0 legacy=0x7f52540c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f5259b9b700 1 -- 172.21.15.33:0/979148586 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5254154b70 con 0x7f52540c1840 2023-11-06T22:00:18.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f525a39c700 1 --2- 172.21.15.33:0/979148586 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c6460 0x7f52540bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:18.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f5259b9b700 1 --2- 172.21.15.33:0/979148586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540c1840 0x7f52540a9de0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f5250006fd0 tx=0x7f52500090f0).ready entity=mon.0 client_cookie=94ef22ac6be0afad server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:18.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f5258b99700 1 -- 172.21.15.33:0/979148586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525000b600 con 0x7f52540c1840 2023-11-06T22:00:18.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f5258b99700 1 -- 172.21.15.33:0/979148586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525000b760 con 0x7f52540c1840 2023-11-06T22:00:18.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.729+0000 7f5258b99700 1 -- 172.21.15.33:0/979148586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525000bae0 con 0x7f52540c1840 2023-11-06T22:00:18.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.730+0000 7f525bdff700 1 -- 172.21.15.33:0/979148586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540c1840 msgr2=0x7f52540a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.730+0000 7f525bdff700 1 --2- 172.21.15.33:0/979148586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540c1840 0x7f52540a9de0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f5250006fd0 tx=0x7f52500090f0).stop 2023-11-06T22:00:18.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.730+0000 7f525bdff700 1 -- 172.21.15.33:0/979148586 shutdown_connections 2023-11-06T22:00:18.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.730+0000 7f525bdff700 1 --2- 172.21.15.33:0/979148586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540c1840 0x7f52540a9de0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.730+0000 7f525bdff700 1 --2- 172.21.15.33:0/979148586 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c6460 0x7f52540bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.730+0000 7f525bdff700 1 -- 172.21.15.33:0/979148586 >> 172.21.15.33:0/979148586 conn(0x7f525405ec80 msgr2=0x7f52540cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:18.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.730+0000 7f525bdff700 1 -- 172.21.15.33:0/979148586 shutdown_connections 2023-11-06T22:00:18.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.730+0000 7f525bdff700 1 -- 172.21.15.33:0/979148586 wait complete. 2023-11-06T22:00:18.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.732+0000 7f525bdff700 1 Processor -- start 2023-11-06T22:00:18.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.732+0000 7f525bdff700 1 -- start start 2023-11-06T22:00:18.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f525bdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540aa2d0 0x7f52541677f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f525bdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c1840 0x7f52540ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f525bdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52540c6460 0x7f52540cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f525bdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540cb130 con 0x7f52540aa2d0 2023-11-06T22:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f525bdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254165870 con 0x7f52540c6460 2023-11-06T22:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f525bdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52540adf40 con 0x7f52540c1840 2023-11-06T22:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f5259b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540aa2d0 0x7f52541677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f5259b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540aa2d0 0x7f52541677f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44998/0 (socket says 172.21.15.33:44998) 2023-11-06T22:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f5259b9b700 1 -- 172.21.15.33:0/478602586 learned_addr learned my addr 172.21.15.33:0/478602586 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f525a39c700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52540c6460 0x7f52540cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f525939a700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c1840 0x7f52540ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f5259b9b700 1 -- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c1840 msgr2=0x7f52540ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f5259b9b700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c1840 0x7f52540ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f5259b9b700 1 -- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52540c6460 msgr2=0x7f52540cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f5259b9b700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52540c6460 0x7f52540cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f5259b9b700 1 -- 172.21.15.33:0/478602586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5250009370 con 0x7f52540aa2d0 2023-11-06T22:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.733+0000 7f525a39c700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52540c6460 0x7f52540cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.734+0000 7f5259b9b700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540aa2d0 0x7f52541677f0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f525406bc90 tx=0x7f5250017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.734+0000 7f5249ffb700 1 -- 172.21.15.33:0/478602586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525000b8d0 con 0x7f52540aa2d0 2023-11-06T22:00:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.734+0000 7f5249ffb700 1 -- 172.21.15.33:0/478602586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525000ba30 con 0x7f52540aa2d0 2023-11-06T22:00:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.734+0000 7f525bdff700 1 -- 172.21.15.33:0/478602586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52540ae220 con 0x7f52540aa2d0 2023-11-06T22:00:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.734+0000 7f5249ffb700 1 -- 172.21.15.33:0/478602586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525001b660 con 0x7f52540aa2d0 2023-11-06T22:00:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.734+0000 7f525bdff700 1 -- 172.21.15.33:0/478602586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5254167ce0 con 0x7f52540aa2d0 2023-11-06T22:00:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.735+0000 7f525bdff700 1 -- 172.21.15.33:0/478602586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525406c6c0 con 0x7f52540aa2d0 2023-11-06T22:00:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.736+0000 7f5249ffb700 1 -- 172.21.15.33:0/478602586 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f525001ba50 con 0x7f52540aa2d0 2023-11-06T22:00:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.737+0000 7f5249ffb700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f523805f240 0x7f5238061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.737+0000 7f5249ffb700 1 -- 172.21.15.33:0/478602586 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f525007bf00 con 0x7f52540aa2d0 2023-11-06T22:00:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.738+0000 7f525939a700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f523805f240 0x7f5238061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.738+0000 7f525939a700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f523805f240 0x7f5238061700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f52540c37b0 tx=0x7f5244009040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:18.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.741+0000 7f5249ffb700 1 -- 172.21.15.33:0/478602586 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f525004cf60 con 0x7f52540aa2d0 2023-11-06T22:00:18.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.921+0000 7f525bdff700 1 -- 172.21.15.33:0/478602586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52540c1390 con 0x7f52540aa2d0 2023-11-06T22:00:18.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.922+0000 7f5249ffb700 1 -- 172.21.15.33:0/478602586 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f525001f070 con 0x7f52540aa2d0 2023-11-06T22:00:18.925 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:18.925 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":184,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:00:18.069425+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:00:15.624386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T22:00:17.062698+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:00:18.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 -- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f523805f240 msgr2=0x7f5238061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f523805f240 0x7f5238061700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f52540c37b0 tx=0x7f5244009040).stop 2023-11-06T22:00:18.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 -- 172.21.15.33:0/478602586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540aa2d0 msgr2=0x7f52541677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:18.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540aa2d0 0x7f52541677f0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f525406bc90 tx=0x7f5250017000).stop 2023-11-06T22:00:18.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 -- 172.21.15.33:0/478602586 shutdown_connections 2023-11-06T22:00:18.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f523805f240 0x7f5238061700 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52540c6460 0x7f52540cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52540aa2d0 0x7f52541677f0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 --2- 172.21.15.33:0/478602586 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f52540c1840 0x7f52540ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:18.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 -- 172.21.15.33:0/478602586 >> 172.21.15.33:0/478602586 conn(0x7f525405ec80 msgr2=0x7f52540c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:18.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 -- 172.21.15.33:0/478602586 shutdown_connections 2023-11-06T22:00:18.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:18.924+0000 7f525bdff700 1 -- 172.21.15.33:0/478602586 wait complete. 2023-11-06T22:00:18.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T22:00:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.145+0000 7fb52b1ff700 1 Processor -- start 2023-11-06T22:00:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.145+0000 7fb52b1ff700 1 -- start start 2023-11-06T22:00:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.147+0000 7fb52b1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb5240646c0 0x7fb524064b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.147+0000 7fb52b1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5241556d0 con 0x7fb5240c57e0 2023-11-06T22:00:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.147+0000 7fb52b1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb524065030 con 0x7fb5241544f0 2023-11-06T22:00:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.147+0000 7fb52b1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5241548d0 con 0x7fb5240646c0 2023-11-06T22:00:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.147+0000 7fb52979c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb5240c57e0 0x7fb5240c5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52006/0 (socket says 172.21.15.33:52006) 2023-11-06T22:00:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.147+0000 7fb52979c700 1 -- 172.21.15.33:0/595611824 learned_addr learned my addr 172.21.15.33:0/595611824 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.147+0000 7fb523fff700 1 --2- 172.21.15.33:0/595611824 >> v2:172.21.15.69:3300/0 conn(0x7fb5240646c0 0x7fb524064b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.147+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950594764 0 0) 0x7fb5241556d0 con 0x7fb5240c57e0 2023-11-06T22:00:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.147+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5140033f0 con 0x7fb5240c57e0 2023-11-06T22:00:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644242806 0 0) 0x7fb524065030 con 0x7fb5241544f0 2023-11-06T22:00:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5241556d0 con 0x7fb5241544f0 2023-11-06T22:00:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014513819 0 0) 0x7fb5140033f0 con 0x7fb5240c57e0 2023-11-06T22:00:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 >> v2:172.21.15.69:3300/0 conn(0x7fb5240646c0 msgr2=0x7fb524064b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 --2- 172.21.15.33:0/595611824 >> v2:172.21.15.69:3300/0 conn(0x7fb5240646c0 0x7fb524064b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:19.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 >> v1:172.21.15.69:6789/0 conn(0x7fb5241544f0 legacy=0x7fb5240c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524154ac0 con 0x7fb5240c57e0 2023-11-06T22:00:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fb51c0017f0 con 0x7fb5240c57e0 2023-11-06T22:00:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 >> v1:172.21.15.33:6789/0 conn(0x7fb5240c57e0 legacy=0x7fb5240c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb514003a70 0x7fb514003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb514004690 0x7fb514004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb514005000 0x7fb51400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51400be80 con 0x7fb514003a70 2023-11-06T22:00:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51400bfc0 con 0x7fb514004690 2023-11-06T22:00:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51400c180 con 0x7fb514005000 2023-11-06T22:00:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb528f9b700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb514003a70 0x7fb514003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb523fff700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb514004690 0x7fb514004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb52979c700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb514005000 0x7fb51400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb528f9b700 1 -- 172.21.15.33:0/595611824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb514005000 msgr2=0x7fb51400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.148+0000 7fb528f9b700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb514005000 0x7fb51400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb528f9b700 1 -- 172.21.15.33:0/595611824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb514004690 msgr2=0x7fb514004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb528f9b700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb514004690 0x7fb514004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb528f9b700 1 -- 172.21.15.33:0/595611824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5140033f0 con 0x7fb514003a70 2023-11-06T22:00:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb52979c700 1 -- 172.21.15.33:0/595611824 reap_dead start 2023-11-06T22:00:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb528f9b700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb514003a70 0x7fb514003e70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fb518001860 tx=0x7fb518007bb0).ready entity=mon.0 client_cookie=e7fba2d55313d0a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518007020 con 0x7fb514003a70 2023-11-06T22:00:19.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb518007180 con 0x7fb514003a70 2023-11-06T22:00:19.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb5237fe700 1 -- 172.21.15.33:0/595611824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518007500 con 0x7fb514003a70 2023-11-06T22:00:19.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb52b1ff700 1 -- 172.21.15.33:0/595611824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb514003a70 msgr2=0x7fb514003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:19.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb52b1ff700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb514003a70 0x7fb514003e70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fb518001860 tx=0x7fb518007bb0).stop 2023-11-06T22:00:19.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb52b1ff700 1 -- 172.21.15.33:0/595611824 shutdown_connections 2023-11-06T22:00:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb52b1ff700 1 --2- 172.21.15.33:0/595611824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb514003a70 0x7fb514003e70 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.149+0000 7fb52b1ff700 1 -- 172.21.15.33:0/595611824 >> 172.21.15.33:0/595611824 conn(0x7fb5240cb900 msgr2=0x7fb5240c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.150+0000 7fb52b1ff700 1 -- 172.21.15.33:0/595611824 shutdown_connections 2023-11-06T22:00:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.150+0000 7fb52b1ff700 1 -- 172.21.15.33:0/595611824 wait complete. 2023-11-06T22:00:19.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.152+0000 7fb52b1ff700 1 Processor -- start 2023-11-06T22:00:19.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.152+0000 7fb52b1ff700 1 -- start start 2023-11-06T22:00:19.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.153+0000 7fb52b1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb514003a70 0x7fb52413f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.153+0000 7fb52b1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5240c4a90 0x7fb5240c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.153+0000 7fb52b1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5240c4f10 0x7fb5240ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.153+0000 7fb52b1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240ca730 con 0x7fb5240c4f10 2023-11-06T22:00:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.153+0000 7fb52b1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240ac3d0 con 0x7fb5240c4a90 2023-11-06T22:00:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.153+0000 7fb52b1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240ac6b0 con 0x7fb514003a70 2023-11-06T22:00:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.153+0000 7fb52979c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5240c4f10 0x7fb5240ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb528f9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb514003a70 0x7fb52413f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb523fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5240c4a90 0x7fb5240c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb52979c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5240c4f10 0x7fb5240ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45018/0 (socket says 172.21.15.33:45018) 2023-11-06T22:00:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb52979c700 1 -- 172.21.15.33:0/250244270 learned_addr learned my addr 172.21.15.33:0/250244270 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb52979c700 1 -- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb514003a70 msgr2=0x7fb52413f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb52979c700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb514003a70 0x7fb52413f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb52979c700 1 -- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5240c4a90 msgr2=0x7fb5240c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb52979c700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5240c4a90 0x7fb5240c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb52979c700 1 -- 172.21.15.33:0/250244270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5240ac990 con 0x7fb5240c4f10 2023-11-06T22:00:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb52979c700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5240c4f10 0x7fb5240ca240 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7fb51c000f80 tx=0x7fb51c007f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.154+0000 7fb520ff9700 1 -- 172.21.15.33:0/250244270 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c010520 con 0x7fb5240c4f10 2023-11-06T22:00:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.155+0000 7fb520ff9700 1 -- 172.21.15.33:0/250244270 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c010680 con 0x7fb5240c4f10 2023-11-06T22:00:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.155+0000 7fb52b1ff700 1 -- 172.21.15.33:0/250244270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5240acc70 con 0x7fb5240c4f10 2023-11-06T22:00:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.155+0000 7fb520ff9700 1 -- 172.21.15.33:0/250244270 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c010a20 con 0x7fb5240c4f10 2023-11-06T22:00:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.155+0000 7fb52b1ff700 1 -- 172.21.15.33:0/250244270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52413fcf0 con 0x7fb5240c4f10 2023-11-06T22:00:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.156+0000 7fb52b1ff700 1 -- 172.21.15.33:0/250244270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb524154da0 con 0x7fb5240c4f10 2023-11-06T22:00:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.157+0000 7fb520ff9700 1 -- 172.21.15.33:0/250244270 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb51c014430 con 0x7fb5240c4f10 2023-11-06T22:00:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.158+0000 7fb520ff9700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb50805f270 0x7fb508061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.158+0000 7fb520ff9700 1 -- 172.21.15.33:0/250244270 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fb51c07a1a0 con 0x7fb5240c4f10 2023-11-06T22:00:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.158+0000 7fb528f9b700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb50805f270 0x7fb508061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.159+0000 7fb528f9b700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb50805f270 0x7fb508061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb518001670 tx=0x7fb518016000).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.162+0000 7fb520ff9700 1 -- 172.21.15.33:0/250244270 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb51c049af0 con 0x7fb5240c4f10 2023-11-06T22:00:19.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:19.386+0000 7fb52b1ff700 1 -- 172.21.15.33:0/250244270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb5240ad890 con 0x7fb5240c4f10 2023-11-06T22:00:20.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.084+0000 7fb520ff9700 1 -- 172.21.15.33:0/250244270 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb51c04ebd0 con 0x7fb5240c4f10 2023-11-06T22:00:20.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 -- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb50805f270 msgr2=0x7fb508061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:20.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb50805f270 0x7fb508061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb518001670 tx=0x7fb518016000).stop 2023-11-06T22:00:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 -- 172.21.15.33:0/250244270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5240c4f10 msgr2=0x7fb5240ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5240c4f10 0x7fb5240ca240 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7fb51c000f80 tx=0x7fb51c007f20).stop 2023-11-06T22:00:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 -- 172.21.15.33:0/250244270 shutdown_connections 2023-11-06T22:00:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fb50805f270 0x7fb508061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb514003a70 0x7fb52413f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5240c4f10 0x7fb5240ca240 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 --2- 172.21.15.33:0/250244270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5240c4a90 0x7fb5240c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 -- 172.21.15.33:0/250244270 >> 172.21.15.33:0/250244270 conn(0x7fb5240cb900 msgr2=0x7fb524157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:20.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 -- 172.21.15.33:0/250244270 shutdown_connections 2023-11-06T22:00:20.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.087+0000 7fb52b1ff700 1 -- 172.21.15.33:0/250244270 wait complete. 2023-11-06T22:00:20.110 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T22:00:20.110 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T22:00:20.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.312+0000 7f007ffff700 1 Processor -- start 2023-11-06T22:00:20.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.312+0000 7f007ffff700 1 -- start start 2023-11-06T22:00:20.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.313+0000 7f007ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0078154330 0x7f00780c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:20.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.313+0000 7f007ffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0078154710 0x7f00780c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:20.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.313+0000 7f007ffff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f00780c7ff0 0x7f00780646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:20.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.313+0000 7f007ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00780c6b60 con 0x7f0078154330 2023-11-06T22:00:20.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.313+0000 7f007ffff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00780c5cd0 con 0x7f00780c7ff0 2023-11-06T22:00:20.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.313+0000 7f007ffff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c5f50 con 0x7f0078154710 2023-11-06T22:00:20.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0078154330 0x7f00780c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:20.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0078154330 0x7f00780c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45028/0 (socket says 172.21.15.33:45028) 2023-11-06T22:00:20.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007dd9b700 1 -- 172.21.15.33:0/3021108883 learned_addr learned my addr 172.21.15.33:0/3021108883 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:20.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007e59c700 1 --2- 172.21.15.33:0/3021108883 >> v2:172.21.15.69:3300/0 conn(0x7f00780c7ff0 0x7f00780646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:20.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007d59a700 1 --2- 172.21.15.33:0/3021108883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0078154710 0x7f00780c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007dd9b700 1 -- 172.21.15.33:0/3021108883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0078154710 msgr2=0x7f00780c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007dd9b700 1 --2- 172.21.15.33:0/3021108883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0078154710 0x7f00780c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007dd9b700 1 -- 172.21.15.33:0/3021108883 >> v2:172.21.15.69:3300/0 conn(0x7f00780c7ff0 msgr2=0x7f00780646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007dd9b700 1 --2- 172.21.15.33:0/3021108883 >> v2:172.21.15.69:3300/0 conn(0x7f00780c7ff0 0x7f00780646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007dd9b700 1 -- 172.21.15.33:0/3021108883 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00780c6230 con 0x7f0078154330 2023-11-06T22:00:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.314+0000 7f007dd9b700 1 --2- 172.21.15.33:0/3021108883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 0x7f00780c57e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f00780c1e60 tx=0x7f00740090f0).ready entity=mon.0 client_cookie=ac517846801b95d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:20.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007cd99700 1 -- 172.21.15.33:0/3021108883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400b600 con 0x7f0078154330 2023-11-06T22:00:20.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007cd99700 1 -- 172.21.15.33:0/3021108883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007400b760 con 0x7f0078154330 2023-11-06T22:00:20.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007cd99700 1 -- 172.21.15.33:0/3021108883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400bae0 con 0x7f0078154330 2023-11-06T22:00:20.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007ffff700 1 -- 172.21.15.33:0/3021108883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 msgr2=0x7f00780c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:20.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007ffff700 1 --2- 172.21.15.33:0/3021108883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 0x7f00780c57e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f00780c1e60 tx=0x7f00740090f0).stop 2023-11-06T22:00:20.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007ffff700 1 -- 172.21.15.33:0/3021108883 shutdown_connections 2023-11-06T22:00:20.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007ffff700 1 --2- 172.21.15.33:0/3021108883 >> v2:172.21.15.69:3300/0 conn(0x7f00780c7ff0 0x7f00780646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007ffff700 1 --2- 172.21.15.33:0/3021108883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 0x7f00780c57e0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007ffff700 1 --2- 172.21.15.33:0/3021108883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0078154710 0x7f00780c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007ffff700 1 -- 172.21.15.33:0/3021108883 >> 172.21.15.33:0/3021108883 conn(0x7f00780cb910 msgr2=0x7f00780c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.315+0000 7f007ffff700 1 -- 172.21.15.33:0/3021108883 shutdown_connections 2023-11-06T22:00:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.316+0000 7f007ffff700 1 -- 172.21.15.33:0/3021108883 wait complete. 2023-11-06T22:00:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.317+0000 7f007ffff700 1 Processor -- start 2023-11-06T22:00:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.317+0000 7f007ffff700 1 -- start start 2023-11-06T22:00:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00780c7ff0 0x7f00780adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 0x7f00780ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007ffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0078154710 0x7f00780a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780aec70 con 0x7f0078154330 2023-11-06T22:00:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007ffff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780aeef0 con 0x7f00780c7ff0 2023-11-06T22:00:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007ffff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00781647a0 con 0x7f0078154710 2023-11-06T22:00:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007d59a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 0x7f00780ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007d59a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 0x7f00780ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45044/0 (socket says 172.21.15.33:45044) 2023-11-06T22:00:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007d59a700 1 -- 172.21.15.33:0/1794881419 learned_addr learned my addr 172.21.15.33:0/1794881419 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007e59c700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0078154710 0x7f00780a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.318+0000 7f007dd9b700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00780c7ff0 0x7f00780adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f007d59a700 1 -- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0078154710 msgr2=0x7f00780a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f007d59a700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0078154710 0x7f00780a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f007d59a700 1 -- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00780c7ff0 msgr2=0x7f00780adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f007d59a700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00780c7ff0 0x7f00780adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f007d59a700 1 -- 172.21.15.33:0/1794881419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0074009370 con 0x7f0078154330 2023-11-06T22:00:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f007d59a700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 0x7f00780ae780 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f0068006980 tx=0x7f0068008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f006dffb700 1 -- 172.21.15.33:0/1794881419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068011590 con 0x7f0078154330 2023-11-06T22:00:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f006dffb700 1 -- 172.21.15.33:0/1794881419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00680116f0 con 0x7f0078154330 2023-11-06T22:00:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f007ffff700 1 -- 172.21.15.33:0/1794881419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0078164a80 con 0x7f0078154330 2023-11-06T22:00:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f006dffb700 1 -- 172.21.15.33:0/1794881419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068011a70 con 0x7f0078154330 2023-11-06T22:00:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.319+0000 7f007ffff700 1 -- 172.21.15.33:0/1794881419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0078164450 con 0x7f0078154330 2023-11-06T22:00:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.321+0000 7f007ffff700 1 -- 172.21.15.33:0/1794881419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0078164f20 con 0x7f0078154330 2023-11-06T22:00:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.321+0000 7f006dffb700 1 -- 172.21.15.33:0/1794881419 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f006801c460 con 0x7f0078154330 2023-11-06T22:00:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.322+0000 7f006dffb700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f270 0x7f005c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.322+0000 7f006dffb700 1 -- 172.21.15.33:0/1794881419 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f006807a100 con 0x7f0078154330 2023-11-06T22:00:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.323+0000 7f007dd9b700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f270 0x7f005c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:20.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.323+0000 7f007dd9b700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f270 0x7f005c061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0074009130 tx=0x7f0074017000).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:20.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.326+0000 7f006dffb700 1 -- 172.21.15.33:0/1794881419 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f006804b100 con 0x7f0078154330 2023-11-06T22:00:20.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:20.551+0000 7f007ffff700 1 -- 172.21.15.33:0/1794881419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f00780652d0 con 0x7f0078154330 2023-11-06T22:00:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.086+0000 7f006dffb700 1 -- 172.21.15.33:0/1794881419 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f006804e720 con 0x7f0078154330 2023-11-06T22:00:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 -- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f270 msgr2=0x7f005c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f270 0x7f005c061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0074009130 tx=0x7f0074017000).stop 2023-11-06T22:00:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 -- 172.21.15.33:0/1794881419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 msgr2=0x7f00780ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 0x7f00780ae780 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f0068006980 tx=0x7f0068008040).stop 2023-11-06T22:00:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 -- 172.21.15.33:0/1794881419 shutdown_connections 2023-11-06T22:00:21.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f005c05f270 0x7f005c061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0078154710 0x7f00780a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00780c7ff0 0x7f00780adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 --2- 172.21.15.33:0/1794881419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0078154330 0x7f00780ae780 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 -- 172.21.15.33:0/1794881419 >> 172.21.15.33:0/1794881419 conn(0x7f00780cb910 msgr2=0x7f00780ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:21.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.089+0000 7f007ffff700 1 -- 172.21.15.33:0/1794881419 shutdown_connections 2023-11-06T22:00:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.090+0000 7f007ffff700 1 -- 172.21.15.33:0/1794881419 wait complete. 2023-11-06T22:00:21.112 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T22:00:21.113 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:00:21.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.311+0000 7fd64b5ff700 1 Processor -- start 2023-11-06T22:00:21.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.311+0000 7fd64b5ff700 1 -- start start 2023-11-06T22:00:21.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.312+0000 7fd64b5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd6440c1840 0x7fd6440a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.312+0000 7fd64b5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6440c6460 0x7fd6440bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd64b5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6441554a0 con 0x7fd6440aa2d0 2023-11-06T22:00:21.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd64b5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd644154960 con 0x7fd6440c6460 2023-11-06T22:00:21.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd64b5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd644155930 con 0x7fd6440c1840 2023-11-06T22:00:21.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd649b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6440c6460 0x7fd6440bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd649b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6440c6460 0x7fd6440bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45060/0 (socket says 172.21.15.33:45060) 2023-11-06T22:00:21.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd649b9c700 1 -- 172.21.15.33:0/1734951498 learned_addr learned my addr 172.21.15.33:0/1734951498 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:21.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd64939b700 1 --2- 172.21.15.33:0/1734951498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd6440c1840 0x7fd6440a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd649b9c700 1 -- 172.21.15.33:0/1734951498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd6440c1840 msgr2=0x7fd6440a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:21.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd649b9c700 1 --2- 172.21.15.33:0/1734951498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd6440c1840 0x7fd6440a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd649b9c700 1 -- 172.21.15.33:0/1734951498 >> v1:172.21.15.69:6789/0 conn(0x7fd6440aa2d0 legacy=0x7fd6440c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.313+0000 7fd649b9c700 1 -- 172.21.15.33:0/1734951498 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd644154b70 con 0x7fd6440c6460 2023-11-06T22:00:21.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.314+0000 7fd649b9c700 1 --2- 172.21.15.33:0/1734951498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c6460 0x7fd6440bfd70 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7fd63c009f20 tx=0x7fd63c009840).ready entity=mon.0 client_cookie=70f1b0c3d35320c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:21.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.314+0000 7fd63bfff700 1 -- 172.21.15.33:0/1734951498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00e070 con 0x7fd6440c6460 2023-11-06T22:00:21.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.314+0000 7fd63bfff700 1 -- 172.21.15.33:0/1734951498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c00ba80 con 0x7fd6440c6460 2023-11-06T22:00:21.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.314+0000 7fd63bfff700 1 -- 172.21.15.33:0/1734951498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00be00 con 0x7fd6440c6460 2023-11-06T22:00:21.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.314+0000 7fd64b5ff700 1 -- 172.21.15.33:0/1734951498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c6460 msgr2=0x7fd6440bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.314+0000 7fd64b5ff700 1 --2- 172.21.15.33:0/1734951498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c6460 0x7fd6440bfd70 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7fd63c009f20 tx=0x7fd63c009840).stop 2023-11-06T22:00:21.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.314+0000 7fd64b5ff700 1 -- 172.21.15.33:0/1734951498 shutdown_connections 2023-11-06T22:00:21.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.314+0000 7fd64b5ff700 1 --2- 172.21.15.33:0/1734951498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c6460 0x7fd6440bfd70 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.314+0000 7fd64b5ff700 1 --2- 172.21.15.33:0/1734951498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd6440c1840 0x7fd6440a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.314+0000 7fd64b5ff700 1 -- 172.21.15.33:0/1734951498 >> 172.21.15.33:0/1734951498 conn(0x7fd64405ec80 msgr2=0x7fd6440cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:21.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.315+0000 7fd64b5ff700 1 -- 172.21.15.33:0/1734951498 shutdown_connections 2023-11-06T22:00:21.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.315+0000 7fd64b5ff700 1 -- 172.21.15.33:0/1734951498 wait complete. 2023-11-06T22:00:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.316+0000 7fd64b5ff700 1 Processor -- start 2023-11-06T22:00:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.316+0000 7fd64b5ff700 1 -- start start 2023-11-06T22:00:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd64b5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd6440aa2d0 0x7fd6440c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd64b5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c1840 0x7fd6440c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd64b5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6440c6460 0x7fd6440ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd64b5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440ca980 con 0x7fd6440c1840 2023-11-06T22:00:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd64b5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64414d530 con 0x7fd6440c6460 2023-11-06T22:00:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd64b5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440c5020 con 0x7fd6440aa2d0 2023-11-06T22:00:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd648b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c1840 0x7fd6440c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd648b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c1840 0x7fd6440c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45066/0 (socket says 172.21.15.33:45066) 2023-11-06T22:00:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd648b9a700 1 -- 172.21.15.33:0/703058241 learned_addr learned my addr 172.21.15.33:0/703058241 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd649b9c700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6440c6460 0x7fd6440ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.317+0000 7fd64939b700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd6440aa2d0 0x7fd6440c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.318+0000 7fd648b9a700 1 -- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd6440aa2d0 msgr2=0x7fd6440c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.318+0000 7fd648b9a700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd6440aa2d0 0x7fd6440c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.318+0000 7fd648b9a700 1 -- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6440c6460 msgr2=0x7fd6440ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.318+0000 7fd648b9a700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6440c6460 0x7fd6440ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.318+0000 7fd648b9a700 1 -- 172.21.15.33:0/703058241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c009ae0 con 0x7fd6440c1840 2023-11-06T22:00:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.318+0000 7fd648b9a700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c1840 0x7fd6440c9fa0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7fd63400c440 tx=0x7fd634007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.318+0000 7fd6397fa700 1 -- 172.21.15.33:0/703058241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634007020 con 0x7fd6440c1840 2023-11-06T22:00:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.318+0000 7fd6397fa700 1 -- 172.21.15.33:0/703058241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634007180 con 0x7fd6440c1840 2023-11-06T22:00:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.318+0000 7fd64b5ff700 1 -- 172.21.15.33:0/703058241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64414c920 con 0x7fd6440c1840 2023-11-06T22:00:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.318+0000 7fd6397fa700 1 -- 172.21.15.33:0/703058241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634007520 con 0x7fd6440c1840 2023-11-06T22:00:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.319+0000 7fd64b5ff700 1 -- 172.21.15.33:0/703058241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64414cee0 con 0x7fd6440c1840 2023-11-06T22:00:21.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.319+0000 7fd64b5ff700 1 -- 172.21.15.33:0/703058241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64414d6c0 con 0x7fd6440c1840 2023-11-06T22:00:21.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.320+0000 7fd6397fa700 1 -- 172.21.15.33:0/703058241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd6340068b0 con 0x7fd6440c1840 2023-11-06T22:00:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.322+0000 7fd6397fa700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd62805f270 0x7fd628061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.322+0000 7fd6397fa700 1 -- 172.21.15.33:0/703058241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fd634079db0 con 0x7fd6440c1840 2023-11-06T22:00:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.322+0000 7fd64939b700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd62805f270 0x7fd628061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.322+0000 7fd64939b700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd62805f270 0x7fd628061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd64000bbc0 tx=0x7fd640008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:21.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.325+0000 7fd6397fa700 1 -- 172.21.15.33:0/703058241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd63407e070 con 0x7fd6440c1840 2023-11-06T22:00:21.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.544+0000 7fd64b5ff700 1 -- 172.21.15.33:0/703058241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd64406c6c0 con 0x7fd6440c1840 2023-11-06T22:00:21.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.544+0000 7fd6397fa700 1 -- 172.21.15.33:0/703058241 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4490 (secure 0 0 0) 0x7fd63404ae10 con 0x7fd6440c1840 2023-11-06T22:00:21.547 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:21.547 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":76,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3392183580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3392183580},{"type":"v1","addr":"172.21.15.33:6835","nonce":3392183580}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2124998325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2124998325},{"type":"v1","addr":"172.21.15.69:6837","nonce":2124998325}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3826680893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3826680893},{"type":"v1","addr":"172.21.15.33:6834","nonce":3826680893}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:00:18.069753+0000","modified":"2023-11-06T22:00:21.084401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8551":{"gid":8551,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/249801946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":249801946},{"type":"v1","addr":"172.21.15.69:6836","nonce":249801946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-06T22:00:21.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.546+0000 7fd64b5ff700 1 -- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd62805f270 msgr2=0x7fd628061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.546+0000 7fd64b5ff700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd62805f270 0x7fd628061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd64000bbc0 tx=0x7fd640008040).stop 2023-11-06T22:00:21.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.546+0000 7fd64b5ff700 1 -- 172.21.15.33:0/703058241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c1840 msgr2=0x7fd6440c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.546+0000 7fd64b5ff700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c1840 0x7fd6440c9fa0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7fd63400c440 tx=0x7fd634007d40).stop 2023-11-06T22:00:21.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.547+0000 7fd64b5ff700 1 -- 172.21.15.33:0/703058241 shutdown_connections 2023-11-06T22:00:21.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.547+0000 7fd64b5ff700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd62805f270 0x7fd628061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.547+0000 7fd64b5ff700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd6440aa2d0 0x7fd6440c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.547+0000 7fd64b5ff700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6440c6460 0x7fd6440ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.547+0000 7fd64b5ff700 1 --2- 172.21.15.33:0/703058241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6440c1840 0x7fd6440c9fa0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.547+0000 7fd64b5ff700 1 -- 172.21.15.33:0/703058241 >> 172.21.15.33:0/703058241 conn(0x7fd64405ec80 msgr2=0x7fd644063140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:21.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.547+0000 7fd64b5ff700 1 -- 172.21.15.33:0/703058241 shutdown_connections 2023-11-06T22:00:21.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.547+0000 7fd64b5ff700 1 -- 172.21.15.33:0/703058241 wait complete. 2023-11-06T22:00:21.552 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 76 2023-11-06T22:00:21.567 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:00:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.760+0000 7f57d9dff700 1 Processor -- start 2023-11-06T22:00:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.761+0000 7f57d9dff700 1 -- start start 2023-11-06T22:00:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.762+0000 7f57d9dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57d40c1840 0x7f57d40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.762+0000 7f57d9dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f57d40c6460 0x7f57d40bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.762+0000 7f57d9dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d41554a0 con 0x7f57d40aa2d0 2023-11-06T22:00:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.762+0000 7f57d9dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d4154960 con 0x7f57d40c6460 2023-11-06T22:00:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.762+0000 7f57d9dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d4155930 con 0x7f57d40c1840 2023-11-06T22:00:21.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.762+0000 7f57d2ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f57d40aa2d0 0x7f57d40c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52012/0 (socket says 172.21.15.33:52012) 2023-11-06T22:00:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.762+0000 7f57d2ffd700 1 -- 172.21.15.33:0/3446099328 learned_addr learned my addr 172.21.15.33:0/3446099328 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.762+0000 7f57d37fe700 1 --2- 172.21.15.33:0/3446099328 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57d40c1840 0x7f57d40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.762+0000 7f57d3fff700 1 --2- 172.21.15.33:0/3446099328 >> v2:172.21.15.69:3300/0 conn(0x7f57d40c6460 0x7f57d40bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.763+0000 7f57d27fc700 1 -- 172.21.15.33:0/3446099328 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354803318 0 0) 0x7f57d41554a0 con 0x7f57d40aa2d0 2023-11-06T22:00:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.763+0000 7f57d27fc700 1 -- 172.21.15.33:0/3446099328 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57bc0033f0 con 0x7f57d40aa2d0 2023-11-06T22:00:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.763+0000 7f57d3fff700 1 -- 172.21.15.33:0/3446099328 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57d40c1840 msgr2=0x7f57d40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.763+0000 7f57d3fff700 1 --2- 172.21.15.33:0/3446099328 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57d40c1840 0x7f57d40a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.763+0000 7f57d3fff700 1 -- 172.21.15.33:0/3446099328 >> v1:172.21.15.33:6789/0 conn(0x7f57d40aa2d0 legacy=0x7f57d40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.763+0000 7f57d3fff700 1 -- 172.21.15.33:0/3446099328 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4154b70 con 0x7f57d40c6460 2023-11-06T22:00:21.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.763+0000 7f57d3fff700 1 --2- 172.21.15.33:0/3446099328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57d40c6460 0x7f57d40bfd70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f57cc009e60 tx=0x7f57cc009780).ready entity=mon.1 client_cookie=62df0c72b7d9de42 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:21.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.763+0000 7f57d27fc700 1 -- 172.21.15.33:0/3446099328 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc00e070 con 0x7f57d40c6460 2023-11-06T22:00:21.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.763+0000 7f57d27fc700 1 -- 172.21.15.33:0/3446099328 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc00bab0 con 0x7f57d40c6460 2023-11-06T22:00:21.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.763+0000 7f57d27fc700 1 -- 172.21.15.33:0/3446099328 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc00be30 con 0x7f57d40c6460 2023-11-06T22:00:21.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.764+0000 7f57d9dff700 1 -- 172.21.15.33:0/3446099328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57d40c6460 msgr2=0x7f57d40bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.764+0000 7f57d9dff700 1 --2- 172.21.15.33:0/3446099328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57d40c6460 0x7f57d40bfd70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f57cc009e60 tx=0x7f57cc009780).stop 2023-11-06T22:00:21.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.764+0000 7f57d9dff700 1 -- 172.21.15.33:0/3446099328 shutdown_connections 2023-11-06T22:00:21.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.764+0000 7f57d9dff700 1 --2- 172.21.15.33:0/3446099328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57d40c6460 0x7f57d40bfd70 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.764+0000 7f57d9dff700 1 --2- 172.21.15.33:0/3446099328 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57d40c1840 0x7f57d40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.764+0000 7f57d9dff700 1 -- 172.21.15.33:0/3446099328 >> 172.21.15.33:0/3446099328 conn(0x7f57d405ec80 msgr2=0x7f57d40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:21.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.764+0000 7f57d9dff700 1 -- 172.21.15.33:0/3446099328 shutdown_connections 2023-11-06T22:00:21.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.764+0000 7f57d9dff700 1 -- 172.21.15.33:0/3446099328 wait complete. 2023-11-06T22:00:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.766+0000 7f57d9dff700 1 Processor -- start 2023-11-06T22:00:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.766+0000 7f57d9dff700 1 -- start start 2023-11-06T22:00:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d9dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57d40aa2d0 0x7f57d4143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d9dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57d40c1840 0x7f57d40af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d9dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57d40c6460 0x7f57d40c05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d9dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40b0e10 con 0x7f57d40c6460 2023-11-06T22:00:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d9dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40b0980 con 0x7f57d40c1840 2023-11-06T22:00:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d9dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40afd70 con 0x7f57d40aa2d0 2023-11-06T22:00:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57d40c6460 0x7f57d40c05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57d40c6460 0x7f57d40c05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45070/0 (socket says 172.21.15.33:45070) 2023-11-06T22:00:21.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d3fff700 1 -- 172.21.15.33:0/1680860201 learned_addr learned my addr 172.21.15.33:0/1680860201 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:21.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d2ffd700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57d40c1840 0x7f57d40af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d37fe700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57d40aa2d0 0x7f57d4143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d3fff700 1 -- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57d40aa2d0 msgr2=0x7f57d4143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d3fff700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57d40aa2d0 0x7f57d4143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d3fff700 1 -- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57d40c1840 msgr2=0x7f57d40af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d3fff700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57d40c1840 0x7f57d40af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d3fff700 1 -- 172.21.15.33:0/1680860201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc009a20 con 0x7f57d40c6460 2023-11-06T22:00:21.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.768+0000 7f57d2ffd700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57d40c1840 0x7f57d40af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.769+0000 7f57d3fff700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57d40c6460 0x7f57d40c05a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f57d406bc90 tx=0x7f57d4155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.769+0000 7f57bb7fe700 1 -- 172.21.15.33:0/1680860201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc009440 con 0x7f57d40c6460 2023-11-06T22:00:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.769+0000 7f57bb7fe700 1 -- 172.21.15.33:0/1680860201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc01d630 con 0x7f57d40c6460 2023-11-06T22:00:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.769+0000 7f57bb7fe700 1 -- 172.21.15.33:0/1680860201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc01d9f0 con 0x7f57d40c6460 2023-11-06T22:00:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.769+0000 7f57d9dff700 1 -- 172.21.15.33:0/1680860201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d40b0050 con 0x7f57d40c6460 2023-11-06T22:00:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.769+0000 7f57d9dff700 1 -- 172.21.15.33:0/1680860201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d40b0610 con 0x7f57d40c6460 2023-11-06T22:00:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.770+0000 7f57d9dff700 1 -- 172.21.15.33:0/1680860201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d40c14f0 con 0x7f57d40c6460 2023-11-06T22:00:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.771+0000 7f57bb7fe700 1 -- 172.21.15.33:0/1680860201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f57cc027460 con 0x7f57d40c6460 2023-11-06T22:00:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.772+0000 7f57bb7fe700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f57c005f240 0x7f57c0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.772+0000 7f57bb7fe700 1 -- 172.21.15.33:0/1680860201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f57cc07c1e0 con 0x7f57d40c6460 2023-11-06T22:00:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.772+0000 7f57d37fe700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f57c005f240 0x7f57c0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.773+0000 7f57d37fe700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f57c005f240 0x7f57c0061700 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f57c4009c30 tx=0x7f57c4009060).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.776+0000 7f57bb7fe700 1 -- 172.21.15.33:0/1680860201 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f57cc04d1e0 con 0x7f57d40c6460 2023-11-06T22:00:21.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.960+0000 7f57d9dff700 1 -- 172.21.15.33:0/1680860201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57d40b0af0 con 0x7f57d40c6460 2023-11-06T22:00:21.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.961+0000 7f57bb7fe700 1 -- 172.21.15.33:0/1680860201 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+17061 (secure 0 0 0) 0x7f57cc050800 con 0x7f57d40c6460 2023-11-06T22:00:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:21.965 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":185,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:00:19.072004+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:00:15.624386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T22:00:17.062698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:00:21.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.963+0000 7f57d9dff700 1 -- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f57c005f240 msgr2=0x7f57c0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.963+0000 7f57d9dff700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f57c005f240 0x7f57c0061700 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f57c4009c30 tx=0x7f57c4009060).stop 2023-11-06T22:00:21.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.963+0000 7f57d9dff700 1 -- 172.21.15.33:0/1680860201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57d40c6460 msgr2=0x7f57d40c05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.963+0000 7f57d9dff700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57d40c6460 0x7f57d40c05a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f57d406bc90 tx=0x7f57d4155410).stop 2023-11-06T22:00:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.964+0000 7f57d9dff700 1 -- 172.21.15.33:0/1680860201 shutdown_connections 2023-11-06T22:00:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.964+0000 7f57d9dff700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f57c005f240 0x7f57c0061700 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.964+0000 7f57d9dff700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57d40aa2d0 0x7f57d4143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.964+0000 7f57d9dff700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57d40c6460 0x7f57d40c05a0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.964+0000 7f57d9dff700 1 --2- 172.21.15.33:0/1680860201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57d40c1840 0x7f57d40af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.964+0000 7f57d9dff700 1 -- 172.21.15.33:0/1680860201 >> 172.21.15.33:0/1680860201 conn(0x7f57d405ec80 msgr2=0x7f57d40ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.964+0000 7f57d9dff700 1 -- 172.21.15.33:0/1680860201 shutdown_connections 2023-11-06T22:00:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:21.964+0000 7f57d9dff700 1 -- 172.21.15.33:0/1680860201 wait complete. 2023-11-06T22:00:21.984 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T22:00:21.985 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:00:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.178+0000 7ff7287ff700 1 Processor -- start 2023-11-06T22:00:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.178+0000 7ff7287ff700 1 -- start start 2023-11-06T22:00:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.180+0000 7ff7287ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff7200c61d0 0x7ff7200c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.180+0000 7ff7287ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7201554a0 con 0x7ff7200c5d50 2023-11-06T22:00:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.180+0000 7ff7287ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff720155930 con 0x7ff7200c1840 2023-11-06T22:00:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.180+0000 7ff7287ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff720154960 con 0x7ff7200c61d0 2023-11-06T22:00:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.180+0000 7ff725d9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff7200c5d50 0x7ff7200bfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52028/0 (socket says 172.21.15.33:52028) 2023-11-06T22:00:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.180+0000 7ff725d9a700 1 -- 172.21.15.33:0/629143529 learned_addr learned my addr 172.21.15.33:0/629143529 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.180+0000 7ff726d9c700 1 --2- 172.21.15.33:0/629143529 >> v2:172.21.15.69:3300/0 conn(0x7ff7200c61d0 0x7ff7200c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765091557 0 0) 0x7ff7201554a0 con 0x7ff7200c5d50 2023-11-06T22:00:22.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff70c0033f0 con 0x7ff7200c5d50 2023-11-06T22:00:22.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599437036 0 0) 0x7ff720155930 con 0x7ff7200c1840 2023-11-06T22:00:22.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7201554a0 con 0x7ff7200c1840 2023-11-06T22:00:22.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3874303080 0 0) 0x7ff70c0033f0 con 0x7ff7200c5d50 2023-11-06T22:00:22.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 >> v2:172.21.15.69:3300/0 conn(0x7ff7200c61d0 msgr2=0x7ff7200c0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:22.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 --2- 172.21.15.33:0/629143529 >> v2:172.21.15.69:3300/0 conn(0x7ff7200c61d0 0x7ff7200c0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 >> v1:172.21.15.69:6789/0 conn(0x7ff7200c1840 legacy=0x7ff7200a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200aa4f0 con 0x7ff7200c5d50 2023-11-06T22:00:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7ff710001470 con 0x7ff7200c5d50 2023-11-06T22:00:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff726d9c700 1 --2- 172.21.15.33:0/629143529 >> v2:172.21.15.69:3300/0 conn(0x7ff7200c61d0 0x7ff7200c0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:00:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 >> v1:172.21.15.33:6789/0 conn(0x7ff7200c5d50 legacy=0x7ff7200bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff70c003a70 0x7ff70c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff70c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff70c005000 0x7ff70c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff720154960 con 0x7ff70c004690 2023-11-06T22:00:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c00bf60 con 0x7ff70c003a70 2023-11-06T22:00:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c00c0d0 con 0x7ff70c005000 2023-11-06T22:00:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff725d9a700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff70c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.181+0000 7ff72659b700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff70c003a70 0x7ff70c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff726d9c700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff70c005000 0x7ff70c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff725d9a700 1 -- 172.21.15.33:0/629143529 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff70c005000 msgr2=0x7ff70c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff725d9a700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff70c005000 0x7ff70c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff725d9a700 1 -- 172.21.15.33:0/629143529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff70c003a70 msgr2=0x7ff70c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff725d9a700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff70c003a70 0x7ff70c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff725d9a700 1 -- 172.21.15.33:0/629143529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c0033f0 con 0x7ff70c004690 2023-11-06T22:00:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff72659b700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff70c003a70 0x7ff70c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:00:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff726d9c700 1 -- 172.21.15.33:0/629143529 reap_dead start 2023-11-06T22:00:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff725d9a700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff70c004b10 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7ff71000c110 tx=0x7ff720155410).ready entity=mon.0 client_cookie=ffbdec539cd924bf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7100079b0 con 0x7ff70c004690 2023-11-06T22:00:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71000ca40 con 0x7ff70c004690 2023-11-06T22:00:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.182+0000 7ff725599700 1 -- 172.21.15.33:0/629143529 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71000cde0 con 0x7ff70c004690 2023-11-06T22:00:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.183+0000 7ff7287ff700 1 -- 172.21.15.33:0/629143529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 msgr2=0x7ff70c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.183+0000 7ff7287ff700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff70c004b10 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7ff71000c110 tx=0x7ff720155410).stop 2023-11-06T22:00:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.183+0000 7ff7287ff700 1 -- 172.21.15.33:0/629143529 shutdown_connections 2023-11-06T22:00:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.183+0000 7ff7287ff700 1 --2- 172.21.15.33:0/629143529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff70c004b10 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.183+0000 7ff7287ff700 1 -- 172.21.15.33:0/629143529 >> 172.21.15.33:0/629143529 conn(0x7ff72005ec80 msgr2=0x7ff7200cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.183+0000 7ff7287ff700 1 -- 172.21.15.33:0/629143529 shutdown_connections 2023-11-06T22:00:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.183+0000 7ff7287ff700 1 -- 172.21.15.33:0/629143529 wait complete. 2023-11-06T22:00:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.185+0000 7ff7287ff700 1 Processor -- start 2023-11-06T22:00:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.185+0000 7ff7287ff700 1 -- start start 2023-11-06T22:00:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff7287ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff720144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff7287ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff720153b60 0x7ff720153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff7287ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff720155dd0 0x7ff720156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff7287ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200c3850 con 0x7ff70c004690 2023-11-06T22:00:22.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff7287ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7201544d0 con 0x7ff720153b60 2023-11-06T22:00:22.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff7287ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff720145130 con 0x7ff720155dd0 2023-11-06T22:00:22.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff72659b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff720144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff72659b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff720144c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45100/0 (socket says 172.21.15.33:45100) 2023-11-06T22:00:22.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff72659b700 1 -- 172.21.15.33:0/2488801750 learned_addr learned my addr 172.21.15.33:0/2488801750 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:22.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff725d9a700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff720153b60 0x7ff720153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff726d9c700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff720155dd0 0x7ff720156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff72659b700 1 -- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff720155dd0 msgr2=0x7ff720156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff72659b700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff720155dd0 0x7ff720156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff72659b700 1 -- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff720153b60 msgr2=0x7ff720153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff72659b700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff720153b60 0x7ff720153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff72659b700 1 -- 172.21.15.33:0/2488801750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200c2f20 con 0x7ff70c004690 2023-11-06T22:00:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff726d9c700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff720155dd0 0x7ff720156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff725d9a700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff720153b60 0x7ff720153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.186+0000 7ff72659b700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff720144c40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7ff71c007720 tx=0x7ff71c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.187+0000 7ff7167fc700 1 -- 172.21.15.33:0/2488801750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c002240 con 0x7ff70c004690 2023-11-06T22:00:22.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.187+0000 7ff7167fc700 1 -- 172.21.15.33:0/2488801750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c0023a0 con 0x7ff70c004690 2023-11-06T22:00:22.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.187+0000 7ff7287ff700 1 -- 172.21.15.33:0/2488801750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7200c3200 con 0x7ff70c004690 2023-11-06T22:00:22.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.187+0000 7ff7167fc700 1 -- 172.21.15.33:0/2488801750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0017a0 con 0x7ff70c004690 2023-11-06T22:00:22.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.187+0000 7ff7287ff700 1 -- 172.21.15.33:0/2488801750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff720156b50 con 0x7ff70c004690 2023-11-06T22:00:22.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.188+0000 7ff7287ff700 1 -- 172.21.15.33:0/2488801750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72006ab10 con 0x7ff70c004690 2023-11-06T22:00:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.189+0000 7ff7167fc700 1 -- 172.21.15.33:0/2488801750 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff71c001b10 con 0x7ff70c004690 2023-11-06T22:00:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.189+0000 7ff7167fc700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff70405f270 0x7ff704061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.190+0000 7ff7167fc700 1 -- 172.21.15.33:0/2488801750 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7ff71c079e70 con 0x7ff70c004690 2023-11-06T22:00:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.190+0000 7ff725d9a700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff70405f270 0x7ff704061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.190+0000 7ff725d9a700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff70405f270 0x7ff704061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff710001ed0 tx=0x7ff710015000).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.193+0000 7ff7167fc700 1 -- 172.21.15.33:0/2488801750 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff71c049af0 con 0x7ff70c004690 2023-11-06T22:00:22.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.372+0000 7ff7287ff700 1 -- 172.21.15.33:0/2488801750 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7200c3990 con 0x7ff70405f270 2023-11-06T22:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.385+0000 7ff7167fc700 1 -- 172.21.15.33:0/2488801750 <== mgr.7352 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361406 (secure 0 0 0) 0x7ff710000cd0 con 0x7ff70405f270 2023-11-06T22:00:22.392 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:22.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 -- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff70405f270 msgr2=0x7ff704061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff70405f270 0x7ff704061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff710001ed0 tx=0x7ff710015000).stop 2023-11-06T22:00:22.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 -- 172.21.15.33:0/2488801750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 msgr2=0x7ff720144c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff720144c40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7ff71c007720 tx=0x7ff71c007bb0).stop 2023-11-06T22:00:22.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 -- 172.21.15.33:0/2488801750 shutdown_connections 2023-11-06T22:00:22.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7ff70405f270 0x7ff704061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff720155dd0 0x7ff720156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70c004690 0x7ff720144c40 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 --2- 172.21.15.33:0/2488801750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff720153b60 0x7ff720153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 -- 172.21.15.33:0/2488801750 >> 172.21.15.33:0/2488801750 conn(0x7ff72005ec80 msgr2=0x7ff7200a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 -- 172.21.15.33:0/2488801750 shutdown_connections 2023-11-06T22:00:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.390+0000 7ff7287ff700 1 -- 172.21.15.33:0/2488801750 wait complete. 2023-11-06T22:00:22.398 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:00:22.409 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T22:00:21.525332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2443916,"kb_used_data":17996,"kb_used_omap":0,"kb_used_meta":2425856,"kb_avail":752530804,"statfs":{"total":773094113280,"available":770591543296,"internally_reserved":0,"allocated":18427904,"data_stored":9387328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2484076544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11936,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-65,"num_write_kb":-34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.009263"},"pg_stats":[{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.105501+0000","last_change":"2023-11-06T22:00:18.101568+0000","last_active":"2023-11-06T22:00:18.105501+0000","last_peered":"2023-11-06T22:00:18.105501+0000","last_clean":"2023-11-06T22:00:18.105501+0000","last_became_active":"2023-11-06T22:00:18.100916+0000","last_became_peered":"2023-11-06T22:00:18.100916+0000","last_unstale":"2023-11-06T22:00:18.105501+0000","last_undegraded":"2023-11-06T22:00:18.105501+0000","last_fullsized":"2023-11-06T22:00:18.105501+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3f","version":"184'3","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.114829+0000","last_change":"2023-11-06T22:00:17.612436+0000","last_active":"2023-11-06T22:00:18.114829+0000","last_peered":"2023-11-06T22:00:18.114829+0000","last_clean":"2023-11-06T22:00:18.114829+0000","last_became_active":"2023-11-06T22:00:17.612113+0000","last_became_peered":"2023-11-06T22:00:17.612113+0000","last_unstale":"2023-11-06T22:00:18.114829+0000","last_undegraded":"2023-11-06T22:00:18.114829+0000","last_fullsized":"2023-11-06T22:00:18.114829+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.103436+0000","last_change":"2023-11-06T22:00:18.103436+0000","last_active":"2023-11-06T22:00:18.103436+0000","last_peered":"2023-11-06T22:00:18.103436+0000","last_clean":"2023-11-06T22:00:18.103436+0000","last_became_active":"2023-11-06T22:00:18.103072+0000","last_became_peered":"2023-11-06T22:00:18.103072+0000","last_unstale":"2023-11-06T22:00:18.103436+0000","last_undegraded":"2023-11-06T22:00:18.103436+0000","last_fullsized":"2023-11-06T22:00:18.103436+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094216+0000","last_change":"2023-11-06T22:00:17.604922+0000","last_active":"2023-11-06T22:00:18.094216+0000","last_peered":"2023-11-06T22:00:18.094216+0000","last_clean":"2023-11-06T22:00:18.094216+0000","last_became_active":"2023-11-06T22:00:17.604269+0000","last_became_peered":"2023-11-06T22:00:17.604269+0000","last_unstale":"2023-11-06T22:00:18.094216+0000","last_undegraded":"2023-11-06T22:00:18.094216+0000","last_fullsized":"2023-11-06T22:00:18.094216+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.109225+0000","last_change":"2023-11-06T22:00:18.104901+0000","last_active":"2023-11-06T22:00:18.109225+0000","last_peered":"2023-11-06T22:00:18.109225+0000","last_clean":"2023-11-06T22:00:18.109225+0000","last_became_active":"2023-11-06T22:00:18.104606+0000","last_became_peered":"2023-11-06T22:00:18.104606+0000","last_unstale":"2023-11-06T22:00:18.109225+0000","last_undegraded":"2023-11-06T22:00:18.109225+0000","last_fullsized":"2023-11-06T22:00:18.109225+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077200+0000","last_change":"2023-11-06T22:00:17.602782+0000","last_active":"2023-11-06T22:00:18.077200+0000","last_peered":"2023-11-06T22:00:18.077200+0000","last_clean":"2023-11-06T22:00:18.077200+0000","last_became_active":"2023-11-06T22:00:17.602492+0000","last_became_peered":"2023-11-06T22:00:17.602492+0000","last_unstale":"2023-11-06T22:00:18.077200+0000","last_undegraded":"2023-11-06T22:00:18.077200+0000","last_fullsized":"2023-11-06T22:00:18.077200+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.107260+0000","last_change":"2023-11-06T22:00:18.098304+0000","last_active":"2023-11-06T22:00:18.107260+0000","last_peered":"2023-11-06T22:00:18.107260+0000","last_clean":"2023-11-06T22:00:18.107260+0000","last_became_active":"2023-11-06T22:00:18.097993+0000","last_became_peered":"2023-11-06T22:00:18.097993+0000","last_unstale":"2023-11-06T22:00:18.107260+0000","last_undegraded":"2023-11-06T22:00:18.107260+0000","last_fullsized":"2023-11-06T22:00:18.107260+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"184'2","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.114158+0000","last_change":"2023-11-06T22:00:17.608170+0000","last_active":"2023-11-06T22:00:18.114158+0000","last_peered":"2023-11-06T22:00:18.114158+0000","last_clean":"2023-11-06T22:00:18.114158+0000","last_became_active":"2023-11-06T22:00:17.606510+0000","last_became_peered":"2023-11-06T22:00:17.606510+0000","last_unstale":"2023-11-06T22:00:18.114158+0000","last_undegraded":"2023-11-06T22:00:18.114158+0000","last_fullsized":"2023-11-06T22:00:18.114158+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.105343+0000","last_change":"2023-11-06T22:00:18.099107+0000","last_active":"2023-11-06T22:00:18.105343+0000","last_peered":"2023-11-06T22:00:18.105343+0000","last_clean":"2023-11-06T22:00:18.105343+0000","last_became_active":"2023-11-06T22:00:18.098655+0000","last_became_peered":"2023-11-06T22:00:18.098655+0000","last_unstale":"2023-11-06T22:00:18.105343+0000","last_undegraded":"2023-11-06T22:00:18.105343+0000","last_fullsized":"2023-11-06T22:00:18.105343+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3b","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.117863+0000","last_change":"2023-11-06T22:00:17.616386+0000","last_active":"2023-11-06T22:00:18.117863+0000","last_peered":"2023-11-06T22:00:18.117863+0000","last_clean":"2023-11-06T22:00:18.117863+0000","last_became_active":"2023-11-06T22:00:17.616040+0000","last_became_peered":"2023-11-06T22:00:17.616040+0000","last_unstale":"2023-11-06T22:00:18.117863+0000","last_undegraded":"2023-11-06T22:00:18.117863+0000","last_fullsized":"2023-11-06T22:00:18.117863+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.104984+0000","last_change":"2023-11-06T22:00:18.091646+0000","last_active":"2023-11-06T22:00:18.104984+0000","last_peered":"2023-11-06T22:00:18.104984+0000","last_clean":"2023-11-06T22:00:18.104984+0000","last_became_active":"2023-11-06T22:00:18.091385+0000","last_became_peered":"2023-11-06T22:00:18.091385+0000","last_unstale":"2023-11-06T22:00:18.104984+0000","last_undegraded":"2023-11-06T22:00:18.104984+0000","last_fullsized":"2023-11-06T22:00:18.104984+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3a","version":"184'1","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.085765+0000","last_change":"2023-11-06T22:00:17.609892+0000","last_active":"2023-11-06T22:00:18.085765+0000","last_peered":"2023-11-06T22:00:18.085765+0000","last_clean":"2023-11-06T22:00:18.085765+0000","last_became_active":"2023-11-06T22:00:17.609602+0000","last_became_peered":"2023-11-06T22:00:17.609602+0000","last_unstale":"2023-11-06T22:00:18.085765+0000","last_undegraded":"2023-11-06T22:00:18.085765+0000","last_fullsized":"2023-11-06T22:00:18.085765+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094163+0000","last_change":"2023-11-06T22:00:18.088979+0000","last_active":"2023-11-06T22:00:18.094163+0000","last_peered":"2023-11-06T22:00:18.094163+0000","last_clean":"2023-11-06T22:00:18.094163+0000","last_became_active":"2023-11-06T22:00:18.088700+0000","last_became_peered":"2023-11-06T22:00:18.088700+0000","last_unstale":"2023-11-06T22:00:18.094163+0000","last_undegraded":"2023-11-06T22:00:18.094163+0000","last_fullsized":"2023-11-06T22:00:18.094163+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.620463+0000","last_change":"2023-11-06T22:00:17.610304+0000","last_active":"2023-11-06T22:00:17.620463+0000","last_peered":"2023-11-06T22:00:17.620463+0000","last_clean":"2023-11-06T22:00:17.620463+0000","last_became_active":"2023-11-06T22:00:17.610036+0000","last_became_peered":"2023-11-06T22:00:17.610036+0000","last_unstale":"2023-11-06T22:00:17.620463+0000","last_undegraded":"2023-11-06T22:00:17.620463+0000","last_fullsized":"2023-11-06T22:00:17.620463+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111179+0000","last_change":"2023-11-06T22:00:18.107809+0000","last_active":"2023-11-06T22:00:18.111179+0000","last_peered":"2023-11-06T22:00:18.111179+0000","last_clean":"2023-11-06T22:00:18.111179+0000","last_became_active":"2023-11-06T22:00:18.105872+0000","last_became_peered":"2023-11-06T22:00:18.105872+0000","last_unstale":"2023-11-06T22:00:18.111179+0000","last_undegraded":"2023-11-06T22:00:18.111179+0000","last_fullsized":"2023-11-06T22:00:18.111179+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.085425+0000","last_change":"2023-11-06T22:00:17.608963+0000","last_active":"2023-11-06T22:00:18.085425+0000","last_peered":"2023-11-06T22:00:18.085425+0000","last_clean":"2023-11-06T22:00:18.085425+0000","last_became_active":"2023-11-06T22:00:17.608612+0000","last_became_peered":"2023-11-06T22:00:17.608612+0000","last_unstale":"2023-11-06T22:00:18.085425+0000","last_undegraded":"2023-11-06T22:00:18.085425+0000","last_fullsized":"2023-11-06T22:00:18.085425+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112318+0000","last_change":"2023-11-06T22:00:18.101936+0000","last_active":"2023-11-06T22:00:18.112318+0000","last_peered":"2023-11-06T22:00:18.112318+0000","last_clean":"2023-11-06T22:00:18.112318+0000","last_became_active":"2023-11-06T22:00:18.101642+0000","last_became_peered":"2023-11-06T22:00:18.101642+0000","last_unstale":"2023-11-06T22:00:18.112318+0000","last_undegraded":"2023-11-06T22:00:18.112318+0000","last_fullsized":"2023-11-06T22:00:18.112318+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.614629+0000","last_change":"2023-11-06T22:00:17.608609+0000","last_active":"2023-11-06T22:00:17.614629+0000","last_peered":"2023-11-06T22:00:17.614629+0000","last_clean":"2023-11-06T22:00:17.614629+0000","last_became_active":"2023-11-06T22:00:17.606730+0000","last_became_peered":"2023-11-06T22:00:17.606730+0000","last_unstale":"2023-11-06T22:00:17.614629+0000","last_undegraded":"2023-11-06T22:00:17.614629+0000","last_fullsized":"2023-11-06T22:00:17.614629+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.101863+0000","last_change":"2023-11-06T22:00:18.101863+0000","last_active":"2023-11-06T22:00:18.101863+0000","last_peered":"2023-11-06T22:00:18.101863+0000","last_clean":"2023-11-06T22:00:18.101863+0000","last_became_active":"2023-11-06T22:00:18.101548+0000","last_became_peered":"2023-11-06T22:00:18.101548+0000","last_unstale":"2023-11-06T22:00:18.101863+0000","last_undegraded":"2023-11-06T22:00:18.101863+0000","last_fullsized":"2023-11-06T22:00:18.101863+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077239+0000","last_change":"2023-11-06T22:00:17.614429+0000","last_active":"2023-11-06T22:00:18.077239+0000","last_peered":"2023-11-06T22:00:18.077239+0000","last_clean":"2023-11-06T22:00:18.077239+0000","last_became_active":"2023-11-06T22:00:17.614046+0000","last_became_peered":"2023-11-06T22:00:17.614046+0000","last_unstale":"2023-11-06T22:00:18.077239+0000","last_undegraded":"2023-11-06T22:00:18.077239+0000","last_fullsized":"2023-11-06T22:00:18.077239+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.098871+0000","last_change":"2023-11-06T22:00:18.098871+0000","last_active":"2023-11-06T22:00:18.098871+0000","last_peered":"2023-11-06T22:00:18.098871+0000","last_clean":"2023-11-06T22:00:18.098871+0000","last_became_active":"2023-11-06T22:00:18.098597+0000","last_became_peered":"2023-11-06T22:00:18.098597+0000","last_unstale":"2023-11-06T22:00:18.098871+0000","last_undegraded":"2023-11-06T22:00:18.098871+0000","last_fullsized":"2023-11-06T22:00:18.098871+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.614700+0000","last_change":"2023-11-06T22:00:17.610682+0000","last_active":"2023-11-06T22:00:17.614700+0000","last_peered":"2023-11-06T22:00:17.614700+0000","last_clean":"2023-11-06T22:00:17.614700+0000","last_became_active":"2023-11-06T22:00:17.610367+0000","last_became_peered":"2023-11-06T22:00:17.610367+0000","last_unstale":"2023-11-06T22:00:17.614700+0000","last_undegraded":"2023-11-06T22:00:17.614700+0000","last_fullsized":"2023-11-06T22:00:17.614700+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.109961+0000","last_change":"2023-11-06T22:00:18.104332+0000","last_active":"2023-11-06T22:00:18.109961+0000","last_peered":"2023-11-06T22:00:18.109961+0000","last_clean":"2023-11-06T22:00:18.109961+0000","last_became_active":"2023-11-06T22:00:18.104075+0000","last_became_peered":"2023-11-06T22:00:18.104075+0000","last_unstale":"2023-11-06T22:00:18.109961+0000","last_undegraded":"2023-11-06T22:00:18.109961+0000","last_fullsized":"2023-11-06T22:00:18.109961+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.34","version":"184'4","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.119521+0000","last_change":"2023-11-06T22:00:17.612964+0000","last_active":"2023-11-06T22:00:18.119521+0000","last_peered":"2023-11-06T22:00:18.119521+0000","last_clean":"2023-11-06T22:00:18.119521+0000","last_became_active":"2023-11-06T22:00:17.612561+0000","last_became_peered":"2023-11-06T22:00:17.612561+0000","last_unstale":"2023-11-06T22:00:18.119521+0000","last_undegraded":"2023-11-06T22:00:18.119521+0000","last_fullsized":"2023-11-06T22:00:18.119521+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.104894+0000","last_change":"2023-11-06T22:00:18.104894+0000","last_active":"2023-11-06T22:00:18.104894+0000","last_peered":"2023-11-06T22:00:18.104894+0000","last_clean":"2023-11-06T22:00:18.104894+0000","last_became_active":"2023-11-06T22:00:18.104013+0000","last_became_peered":"2023-11-06T22:00:18.104013+0000","last_unstale":"2023-11-06T22:00:18.104894+0000","last_undegraded":"2023-11-06T22:00:18.104894+0000","last_fullsized":"2023-11-06T22:00:18.104894+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.33","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.117849+0000","last_change":"2023-11-06T22:00:17.614760+0000","last_active":"2023-11-06T22:00:18.117849+0000","last_peered":"2023-11-06T22:00:18.117849+0000","last_clean":"2023-11-06T22:00:18.117849+0000","last_became_active":"2023-11-06T22:00:17.614272+0000","last_became_peered":"2023-11-06T22:00:17.614272+0000","last_unstale":"2023-11-06T22:00:18.117849+0000","last_undegraded":"2023-11-06T22:00:18.117849+0000","last_fullsized":"2023-11-06T22:00:18.117849+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.097913+0000","last_change":"2023-11-06T22:00:18.097913+0000","last_active":"2023-11-06T22:00:18.097913+0000","last_peered":"2023-11-06T22:00:18.097913+0000","last_clean":"2023-11-06T22:00:18.097913+0000","last_became_active":"2023-11-06T22:00:18.097578+0000","last_became_peered":"2023-11-06T22:00:18.097578+0000","last_unstale":"2023-11-06T22:00:18.097913+0000","last_undegraded":"2023-11-06T22:00:18.097913+0000","last_fullsized":"2023-11-06T22:00:18.097913+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094467+0000","last_change":"2023-11-06T22:00:17.617428+0000","last_active":"2023-11-06T22:00:18.094467+0000","last_peered":"2023-11-06T22:00:18.094467+0000","last_clean":"2023-11-06T22:00:18.094467+0000","last_became_active":"2023-11-06T22:00:17.616754+0000","last_became_peered":"2023-11-06T22:00:17.616754+0000","last_unstale":"2023-11-06T22:00:18.094467+0000","last_undegraded":"2023-11-06T22:00:18.094467+0000","last_fullsized":"2023-11-06T22:00:18.094467+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.100649+0000","last_change":"2023-11-06T22:00:18.100649+0000","last_active":"2023-11-06T22:00:18.100649+0000","last_peered":"2023-11-06T22:00:18.100649+0000","last_clean":"2023-11-06T22:00:18.100649+0000","last_became_active":"2023-11-06T22:00:18.100153+0000","last_became_peered":"2023-11-06T22:00:18.100153+0000","last_unstale":"2023-11-06T22:00:18.100649+0000","last_undegraded":"2023-11-06T22:00:18.100649+0000","last_fullsized":"2023-11-06T22:00:18.100649+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.083352+0000","last_change":"2023-11-06T22:00:17.613326+0000","last_active":"2023-11-06T22:00:18.083352+0000","last_peered":"2023-11-06T22:00:18.083352+0000","last_clean":"2023-11-06T22:00:18.083352+0000","last_became_active":"2023-11-06T22:00:17.613075+0000","last_became_peered":"2023-11-06T22:00:17.613075+0000","last_unstale":"2023-11-06T22:00:18.083352+0000","last_undegraded":"2023-11-06T22:00:18.083352+0000","last_fullsized":"2023-11-06T22:00:18.083352+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111197+0000","last_change":"2023-11-06T22:00:18.107834+0000","last_active":"2023-11-06T22:00:18.111197+0000","last_peered":"2023-11-06T22:00:18.111197+0000","last_clean":"2023-11-06T22:00:18.111197+0000","last_became_active":"2023-11-06T22:00:18.105105+0000","last_became_peered":"2023-11-06T22:00:18.105105+0000","last_unstale":"2023-11-06T22:00:18.111197+0000","last_undegraded":"2023-11-06T22:00:18.111197+0000","last_fullsized":"2023-11-06T22:00:18.111197+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"184'4","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.113122+0000","last_change":"2023-11-06T22:00:17.607230+0000","last_active":"2023-11-06T22:00:18.113122+0000","last_peered":"2023-11-06T22:00:18.113122+0000","last_clean":"2023-11-06T22:00:18.113122+0000","last_became_active":"2023-11-06T22:00:17.606957+0000","last_became_peered":"2023-11-06T22:00:17.606957+0000","last_unstale":"2023-11-06T22:00:18.113122+0000","last_undegraded":"2023-11-06T22:00:18.113122+0000","last_fullsized":"2023-11-06T22:00:18.113122+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.096800+0000","last_change":"2023-11-06T22:00:18.096800+0000","last_active":"2023-11-06T22:00:18.096800+0000","last_peered":"2023-11-06T22:00:18.096800+0000","last_clean":"2023-11-06T22:00:18.096800+0000","last_became_active":"2023-11-06T22:00:18.096444+0000","last_became_peered":"2023-11-06T22:00:18.096444+0000","last_unstale":"2023-11-06T22:00:18.096800+0000","last_undegraded":"2023-11-06T22:00:18.096800+0000","last_fullsized":"2023-11-06T22:00:18.096800+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077597+0000","last_change":"2023-11-06T22:00:17.612270+0000","last_active":"2023-11-06T22:00:18.077597+0000","last_peered":"2023-11-06T22:00:18.077597+0000","last_clean":"2023-11-06T22:00:18.077597+0000","last_became_active":"2023-11-06T22:00:17.611074+0000","last_became_peered":"2023-11-06T22:00:17.611074+0000","last_unstale":"2023-11-06T22:00:18.077597+0000","last_undegraded":"2023-11-06T22:00:18.077597+0000","last_fullsized":"2023-11-06T22:00:18.077597+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.090903+0000","last_change":"2023-11-06T22:00:18.084493+0000","last_active":"2023-11-06T22:00:18.090903+0000","last_peered":"2023-11-06T22:00:18.090903+0000","last_clean":"2023-11-06T22:00:18.090903+0000","last_became_active":"2023-11-06T22:00:18.084177+0000","last_became_peered":"2023-11-06T22:00:18.084177+0000","last_unstale":"2023-11-06T22:00:18.090903+0000","last_undegraded":"2023-11-06T22:00:18.090903+0000","last_fullsized":"2023-11-06T22:00:18.090903+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.083273+0000","last_change":"2023-11-06T22:00:17.615629+0000","last_active":"2023-11-06T22:00:18.083273+0000","last_peered":"2023-11-06T22:00:18.083273+0000","last_clean":"2023-11-06T22:00:18.083273+0000","last_became_active":"2023-11-06T22:00:17.614981+0000","last_became_peered":"2023-11-06T22:00:17.614981+0000","last_unstale":"2023-11-06T22:00:18.083273+0000","last_undegraded":"2023-11-06T22:00:18.083273+0000","last_fullsized":"2023-11-06T22:00:18.083273+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.114660+0000","last_change":"2023-11-06T22:00:18.114032+0000","last_active":"2023-11-06T22:00:18.114660+0000","last_peered":"2023-11-06T22:00:18.114660+0000","last_clean":"2023-11-06T22:00:18.114660+0000","last_became_active":"2023-11-06T22:00:18.113064+0000","last_became_peered":"2023-11-06T22:00:18.113064+0000","last_unstale":"2023-11-06T22:00:18.114660+0000","last_undegraded":"2023-11-06T22:00:18.114660+0000","last_fullsized":"2023-11-06T22:00:18.114660+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2d","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111195+0000","last_change":"2023-11-06T22:00:17.616312+0000","last_active":"2023-11-06T22:00:18.111195+0000","last_peered":"2023-11-06T22:00:18.111195+0000","last_clean":"2023-11-06T22:00:18.111195+0000","last_became_active":"2023-11-06T22:00:17.614951+0000","last_became_peered":"2023-11-06T22:00:17.614951+0000","last_unstale":"2023-11-06T22:00:18.111195+0000","last_undegraded":"2023-11-06T22:00:18.111195+0000","last_fullsized":"2023-11-06T22:00:18.111195+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.110063+0000","last_change":"2023-11-06T22:00:18.103381+0000","last_active":"2023-11-06T22:00:18.110063+0000","last_peered":"2023-11-06T22:00:18.110063+0000","last_clean":"2023-11-06T22:00:18.110063+0000","last_became_active":"2023-11-06T22:00:18.103105+0000","last_became_peered":"2023-11-06T22:00:18.103105+0000","last_unstale":"2023-11-06T22:00:18.110063+0000","last_undegraded":"2023-11-06T22:00:18.110063+0000","last_fullsized":"2023-11-06T22:00:18.110063+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.087791+0000","last_change":"2023-11-06T22:00:17.606658+0000","last_active":"2023-11-06T22:00:18.087791+0000","last_peered":"2023-11-06T22:00:18.087791+0000","last_clean":"2023-11-06T22:00:18.087791+0000","last_became_active":"2023-11-06T22:00:17.606247+0000","last_became_peered":"2023-11-06T22:00:17.606247+0000","last_unstale":"2023-11-06T22:00:18.087791+0000","last_undegraded":"2023-11-06T22:00:18.087791+0000","last_fullsized":"2023-11-06T22:00:18.087791+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.093250+0000","last_change":"2023-11-06T22:00:18.093250+0000","last_active":"2023-11-06T22:00:18.093250+0000","last_peered":"2023-11-06T22:00:18.093250+0000","last_clean":"2023-11-06T22:00:18.093250+0000","last_became_active":"2023-11-06T22:00:18.092901+0000","last_became_peered":"2023-11-06T22:00:18.092901+0000","last_unstale":"2023-11-06T22:00:18.093250+0000","last_undegraded":"2023-11-06T22:00:18.093250+0000","last_fullsized":"2023-11-06T22:00:18.093250+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2b","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112615+0000","last_change":"2023-11-06T22:00:17.616718+0000","last_active":"2023-11-06T22:00:18.112615+0000","last_peered":"2023-11-06T22:00:18.112615+0000","last_clean":"2023-11-06T22:00:18.112615+0000","last_became_active":"2023-11-06T22:00:17.616259+0000","last_became_peered":"2023-11-06T22:00:17.616259+0000","last_unstale":"2023-11-06T22:00:18.112615+0000","last_undegraded":"2023-11-06T22:00:18.112615+0000","last_fullsized":"2023-11-06T22:00:18.112615+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.104222+0000","last_change":"2023-11-06T22:00:18.098319+0000","last_active":"2023-11-06T22:00:18.104222+0000","last_peered":"2023-11-06T22:00:18.104222+0000","last_clean":"2023-11-06T22:00:18.104222+0000","last_became_active":"2023-11-06T22:00:18.093084+0000","last_became_peered":"2023-11-06T22:00:18.093084+0000","last_unstale":"2023-11-06T22:00:18.104222+0000","last_undegraded":"2023-11-06T22:00:18.104222+0000","last_fullsized":"2023-11-06T22:00:18.104222+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094566+0000","last_change":"2023-11-06T22:00:17.610350+0000","last_active":"2023-11-06T22:00:18.094566+0000","last_peered":"2023-11-06T22:00:18.094566+0000","last_clean":"2023-11-06T22:00:18.094566+0000","last_became_active":"2023-11-06T22:00:17.607925+0000","last_became_peered":"2023-11-06T22:00:17.607925+0000","last_unstale":"2023-11-06T22:00:18.094566+0000","last_undegraded":"2023-11-06T22:00:18.094566+0000","last_fullsized":"2023-11-06T22:00:18.094566+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.110612+0000","last_change":"2023-11-06T22:00:18.094031+0000","last_active":"2023-11-06T22:00:18.110612+0000","last_peered":"2023-11-06T22:00:18.110612+0000","last_clean":"2023-11-06T22:00:18.110612+0000","last_became_active":"2023-11-06T22:00:18.093775+0000","last_became_peered":"2023-11-06T22:00:18.093775+0000","last_unstale":"2023-11-06T22:00:18.110612+0000","last_undegraded":"2023-11-06T22:00:18.110612+0000","last_fullsized":"2023-11-06T22:00:18.110612+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.620592+0000","last_change":"2023-11-06T22:00:17.616295+0000","last_active":"2023-11-06T22:00:17.620592+0000","last_peered":"2023-11-06T22:00:17.620592+0000","last_clean":"2023-11-06T22:00:17.620592+0000","last_became_active":"2023-11-06T22:00:17.615366+0000","last_became_peered":"2023-11-06T22:00:17.615366+0000","last_unstale":"2023-11-06T22:00:17.620592+0000","last_undegraded":"2023-11-06T22:00:17.620592+0000","last_fullsized":"2023-11-06T22:00:17.620592+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.095957+0000","last_change":"2023-11-06T22:00:18.095957+0000","last_active":"2023-11-06T22:00:18.095957+0000","last_peered":"2023-11-06T22:00:18.095957+0000","last_clean":"2023-11-06T22:00:18.095957+0000","last_became_active":"2023-11-06T22:00:18.095652+0000","last_became_peered":"2023-11-06T22:00:18.095652+0000","last_unstale":"2023-11-06T22:00:18.095957+0000","last_undegraded":"2023-11-06T22:00:18.095957+0000","last_fullsized":"2023-11-06T22:00:18.095957+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.082747+0000","last_change":"2023-11-06T22:00:17.621460+0000","last_active":"2023-11-06T22:00:18.082747+0000","last_peered":"2023-11-06T22:00:18.082747+0000","last_clean":"2023-11-06T22:00:18.082747+0000","last_became_active":"2023-11-06T22:00:17.621121+0000","last_became_peered":"2023-11-06T22:00:17.621121+0000","last_unstale":"2023-11-06T22:00:18.082747+0000","last_undegraded":"2023-11-06T22:00:18.082747+0000","last_fullsized":"2023-11-06T22:00:18.082747+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111324+0000","last_change":"2023-11-06T22:00:18.108060+0000","last_active":"2023-11-06T22:00:18.111324+0000","last_peered":"2023-11-06T22:00:18.111324+0000","last_clean":"2023-11-06T22:00:18.111324+0000","last_became_active":"2023-11-06T22:00:18.107850+0000","last_became_peered":"2023-11-06T22:00:18.107850+0000","last_unstale":"2023-11-06T22:00:18.111324+0000","last_undegraded":"2023-11-06T22:00:18.111324+0000","last_fullsized":"2023-11-06T22:00:18.111324+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.090737+0000","last_change":"2023-11-06T22:00:17.614473+0000","last_active":"2023-11-06T22:00:18.090737+0000","last_peered":"2023-11-06T22:00:18.090737+0000","last_clean":"2023-11-06T22:00:18.090737+0000","last_became_active":"2023-11-06T22:00:17.611658+0000","last_became_peered":"2023-11-06T22:00:17.611658+0000","last_unstale":"2023-11-06T22:00:18.090737+0000","last_undegraded":"2023-11-06T22:00:18.090737+0000","last_fullsized":"2023-11-06T22:00:18.090737+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.097808+0000","last_change":"2023-11-06T22:00:18.097808+0000","last_active":"2023-11-06T22:00:18.097808+0000","last_peered":"2023-11-06T22:00:18.097808+0000","last_clean":"2023-11-06T22:00:18.097808+0000","last_became_active":"2023-11-06T22:00:18.097474+0000","last_became_peered":"2023-11-06T22:00:18.097474+0000","last_unstale":"2023-11-06T22:00:18.097808+0000","last_undegraded":"2023-11-06T22:00:18.097808+0000","last_fullsized":"2023-11-06T22:00:18.097808+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.089652+0000","last_change":"2023-11-06T22:00:17.606853+0000","last_active":"2023-11-06T22:00:18.089652+0000","last_peered":"2023-11-06T22:00:18.089652+0000","last_clean":"2023-11-06T22:00:18.089652+0000","last_became_active":"2023-11-06T22:00:17.606369+0000","last_became_peered":"2023-11-06T22:00:17.606369+0000","last_unstale":"2023-11-06T22:00:18.089652+0000","last_undegraded":"2023-11-06T22:00:18.089652+0000","last_fullsized":"2023-11-06T22:00:18.089652+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.107124+0000","last_change":"2023-11-06T22:00:18.107124+0000","last_active":"2023-11-06T22:00:18.107124+0000","last_peered":"2023-11-06T22:00:18.107124+0000","last_clean":"2023-11-06T22:00:18.107124+0000","last_became_active":"2023-11-06T22:00:18.106062+0000","last_became_peered":"2023-11-06T22:00:18.106062+0000","last_unstale":"2023-11-06T22:00:18.107124+0000","last_undegraded":"2023-11-06T22:00:18.107124+0000","last_fullsized":"2023-11-06T22:00:18.107124+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.091318+0000","last_change":"2023-11-06T22:00:17.608211+0000","last_active":"2023-11-06T22:00:18.091318+0000","last_peered":"2023-11-06T22:00:18.091318+0000","last_clean":"2023-11-06T22:00:18.091318+0000","last_became_active":"2023-11-06T22:00:17.607775+0000","last_became_peered":"2023-11-06T22:00:17.607775+0000","last_unstale":"2023-11-06T22:00:18.091318+0000","last_undegraded":"2023-11-06T22:00:18.091318+0000","last_fullsized":"2023-11-06T22:00:18.091318+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.091052+0000","last_change":"2023-11-06T22:00:18.089006+0000","last_active":"2023-11-06T22:00:18.091052+0000","last_peered":"2023-11-06T22:00:18.091052+0000","last_clean":"2023-11-06T22:00:18.091052+0000","last_became_active":"2023-11-06T22:00:18.088686+0000","last_became_peered":"2023-11-06T22:00:18.088686+0000","last_unstale":"2023-11-06T22:00:18.091052+0000","last_undegraded":"2023-11-06T22:00:18.091052+0000","last_fullsized":"2023-11-06T22:00:18.091052+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094462+0000","last_change":"2023-11-06T22:00:17.606162+0000","last_active":"2023-11-06T22:00:18.094462+0000","last_peered":"2023-11-06T22:00:18.094462+0000","last_clean":"2023-11-06T22:00:18.094462+0000","last_became_active":"2023-11-06T22:00:17.605889+0000","last_became_peered":"2023-11-06T22:00:17.605889+0000","last_unstale":"2023-11-06T22:00:18.094462+0000","last_undegraded":"2023-11-06T22:00:18.094462+0000","last_fullsized":"2023-11-06T22:00:18.094462+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.093133+0000","last_change":"2023-11-06T22:00:18.093133+0000","last_active":"2023-11-06T22:00:18.093133+0000","last_peered":"2023-11-06T22:00:18.093133+0000","last_clean":"2023-11-06T22:00:18.093133+0000","last_became_active":"2023-11-06T22:00:18.092867+0000","last_became_peered":"2023-11-06T22:00:18.092867+0000","last_unstale":"2023-11-06T22:00:18.093133+0000","last_undegraded":"2023-11-06T22:00:18.093133+0000","last_fullsized":"2023-11-06T22:00:18.093133+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.089200+0000","last_change":"2023-11-06T22:00:17.616087+0000","last_active":"2023-11-06T22:00:18.089200+0000","last_peered":"2023-11-06T22:00:18.089200+0000","last_clean":"2023-11-06T22:00:18.089200+0000","last_became_active":"2023-11-06T22:00:17.615015+0000","last_became_peered":"2023-11-06T22:00:17.615015+0000","last_unstale":"2023-11-06T22:00:18.089200+0000","last_undegraded":"2023-11-06T22:00:18.089200+0000","last_fullsized":"2023-11-06T22:00:18.089200+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.091163+0000","last_change":"2023-11-06T22:00:18.091163+0000","last_active":"2023-11-06T22:00:18.091163+0000","last_peered":"2023-11-06T22:00:18.091163+0000","last_clean":"2023-11-06T22:00:18.091163+0000","last_became_active":"2023-11-06T22:00:18.090878+0000","last_became_peered":"2023-11-06T22:00:18.090878+0000","last_unstale":"2023-11-06T22:00:18.091163+0000","last_undegraded":"2023-11-06T22:00:18.091163+0000","last_fullsized":"2023-11-06T22:00:18.091163+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094653+0000","last_change":"2023-11-06T22:00:17.611021+0000","last_active":"2023-11-06T22:00:18.094653+0000","last_peered":"2023-11-06T22:00:18.094653+0000","last_clean":"2023-11-06T22:00:18.094653+0000","last_became_active":"2023-11-06T22:00:17.607530+0000","last_became_peered":"2023-11-06T22:00:17.607530+0000","last_unstale":"2023-11-06T22:00:18.094653+0000","last_undegraded":"2023-11-06T22:00:18.094653+0000","last_fullsized":"2023-11-06T22:00:18.094653+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.097980+0000","last_change":"2023-11-06T22:00:18.097980+0000","last_active":"2023-11-06T22:00:18.097980+0000","last_peered":"2023-11-06T22:00:18.097980+0000","last_clean":"2023-11-06T22:00:18.097980+0000","last_became_active":"2023-11-06T22:00:18.097540+0000","last_became_peered":"2023-11-06T22:00:18.097540+0000","last_unstale":"2023-11-06T22:00:18.097980+0000","last_undegraded":"2023-11-06T22:00:18.097980+0000","last_fullsized":"2023-11-06T22:00:18.097980+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.091932+0000","last_change":"2023-11-06T22:00:17.614519+0000","last_active":"2023-11-06T22:00:18.091932+0000","last_peered":"2023-11-06T22:00:18.091932+0000","last_clean":"2023-11-06T22:00:18.091932+0000","last_became_active":"2023-11-06T22:00:17.613523+0000","last_became_peered":"2023-11-06T22:00:17.613523+0000","last_unstale":"2023-11-06T22:00:18.091932+0000","last_undegraded":"2023-11-06T22:00:18.091932+0000","last_fullsized":"2023-11-06T22:00:18.091932+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.103162+0000","last_change":"2023-11-06T22:00:18.094492+0000","last_active":"2023-11-06T22:00:18.103162+0000","last_peered":"2023-11-06T22:00:18.103162+0000","last_clean":"2023-11-06T22:00:18.103162+0000","last_became_active":"2023-11-06T22:00:18.093830+0000","last_became_peered":"2023-11-06T22:00:18.093830+0000","last_unstale":"2023-11-06T22:00:18.103162+0000","last_undegraded":"2023-11-06T22:00:18.103162+0000","last_fullsized":"2023-11-06T22:00:18.103162+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.090205+0000","last_change":"2023-11-06T22:00:17.617058+0000","last_active":"2023-11-06T22:00:18.090205+0000","last_peered":"2023-11-06T22:00:18.090205+0000","last_clean":"2023-11-06T22:00:18.090205+0000","last_became_active":"2023-11-06T22:00:17.616702+0000","last_became_peered":"2023-11-06T22:00:17.616702+0000","last_unstale":"2023-11-06T22:00:18.090205+0000","last_undegraded":"2023-11-06T22:00:18.090205+0000","last_fullsized":"2023-11-06T22:00:18.090205+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.102299+0000","last_change":"2023-11-06T22:00:18.102299+0000","last_active":"2023-11-06T22:00:18.102299+0000","last_peered":"2023-11-06T22:00:18.102299+0000","last_clean":"2023-11-06T22:00:18.102299+0000","last_became_active":"2023-11-06T22:00:18.102010+0000","last_became_peered":"2023-11-06T22:00:18.102010+0000","last_unstale":"2023-11-06T22:00:18.102299+0000","last_undegraded":"2023-11-06T22:00:18.102299+0000","last_fullsized":"2023-11-06T22:00:18.102299+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1f","version":"184'1","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.075109+0000","last_change":"2023-11-06T22:00:17.609196+0000","last_active":"2023-11-06T22:00:18.075109+0000","last_peered":"2023-11-06T22:00:18.075109+0000","last_clean":"2023-11-06T22:00:18.075109+0000","last_became_active":"2023-11-06T22:00:17.606206+0000","last_became_peered":"2023-11-06T22:00:17.606206+0000","last_unstale":"2023-11-06T22:00:18.075109+0000","last_undegraded":"2023-11-06T22:00:18.075109+0000","last_fullsized":"2023-11-06T22:00:18.075109+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.109130+0000","last_change":"2023-11-06T22:00:18.104491+0000","last_active":"2023-11-06T22:00:18.109130+0000","last_peered":"2023-11-06T22:00:18.109130+0000","last_clean":"2023-11-06T22:00:18.109130+0000","last_became_active":"2023-11-06T22:00:18.103932+0000","last_became_peered":"2023-11-06T22:00:18.103932+0000","last_unstale":"2023-11-06T22:00:18.109130+0000","last_undegraded":"2023-11-06T22:00:18.109130+0000","last_fullsized":"2023-11-06T22:00:18.109130+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.117722+0000","last_change":"2023-11-06T22:00:17.615812+0000","last_active":"2023-11-06T22:00:18.117722+0000","last_peered":"2023-11-06T22:00:18.117722+0000","last_clean":"2023-11-06T22:00:18.117722+0000","last_became_active":"2023-11-06T22:00:17.615330+0000","last_became_peered":"2023-11-06T22:00:17.615330+0000","last_unstale":"2023-11-06T22:00:18.117722+0000","last_undegraded":"2023-11-06T22:00:18.117722+0000","last_fullsized":"2023-11-06T22:00:18.117722+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.110694+0000","last_change":"2023-11-06T22:00:18.106332+0000","last_active":"2023-11-06T22:00:18.110694+0000","last_peered":"2023-11-06T22:00:18.110694+0000","last_clean":"2023-11-06T22:00:18.110694+0000","last_became_active":"2023-11-06T22:00:18.105651+0000","last_became_peered":"2023-11-06T22:00:18.105651+0000","last_unstale":"2023-11-06T22:00:18.110694+0000","last_undegraded":"2023-11-06T22:00:18.110694+0000","last_fullsized":"2023-11-06T22:00:18.110694+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1d","version":"184'2","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.114242+0000","last_change":"2023-11-06T22:00:17.607641+0000","last_active":"2023-11-06T22:00:18.114242+0000","last_peered":"2023-11-06T22:00:18.114242+0000","last_clean":"2023-11-06T22:00:18.114242+0000","last_became_active":"2023-11-06T22:00:17.607366+0000","last_became_peered":"2023-11-06T22:00:17.607366+0000","last_unstale":"2023-11-06T22:00:18.114242+0000","last_undegraded":"2023-11-06T22:00:18.114242+0000","last_fullsized":"2023-11-06T22:00:18.114242+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.096494+0000","last_change":"2023-11-06T22:00:18.096494+0000","last_active":"2023-11-06T22:00:18.096494+0000","last_peered":"2023-11-06T22:00:18.096494+0000","last_clean":"2023-11-06T22:00:18.096494+0000","last_became_active":"2023-11-06T22:00:18.096179+0000","last_became_peered":"2023-11-06T22:00:18.096179+0000","last_unstale":"2023-11-06T22:00:18.096494+0000","last_undegraded":"2023-11-06T22:00:18.096494+0000","last_fullsized":"2023-11-06T22:00:18.096494+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.625803+0000","last_change":"2023-11-06T22:00:17.621422+0000","last_active":"2023-11-06T22:00:17.625803+0000","last_peered":"2023-11-06T22:00:17.625803+0000","last_clean":"2023-11-06T22:00:17.625803+0000","last_became_active":"2023-11-06T22:00:17.621149+0000","last_became_peered":"2023-11-06T22:00:17.621149+0000","last_unstale":"2023-11-06T22:00:17.625803+0000","last_undegraded":"2023-11-06T22:00:17.625803+0000","last_fullsized":"2023-11-06T22:00:17.625803+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.105141+0000","last_change":"2023-11-06T22:00:18.105141+0000","last_active":"2023-11-06T22:00:18.105141+0000","last_peered":"2023-11-06T22:00:18.105141+0000","last_clean":"2023-11-06T22:00:18.105141+0000","last_became_active":"2023-11-06T22:00:18.104545+0000","last_became_peered":"2023-11-06T22:00:18.104545+0000","last_unstale":"2023-11-06T22:00:18.105141+0000","last_undegraded":"2023-11-06T22:00:18.105141+0000","last_fullsized":"2023-11-06T22:00:18.105141+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.620240+0000","last_change":"2023-11-06T22:00:17.613641+0000","last_active":"2023-11-06T22:00:17.620240+0000","last_peered":"2023-11-06T22:00:17.620240+0000","last_clean":"2023-11-06T22:00:17.620240+0000","last_became_active":"2023-11-06T22:00:17.612511+0000","last_became_peered":"2023-11-06T22:00:17.612511+0000","last_unstale":"2023-11-06T22:00:17.620240+0000","last_undegraded":"2023-11-06T22:00:17.620240+0000","last_fullsized":"2023-11-06T22:00:17.620240+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094256+0000","last_change":"2023-11-06T22:00:18.094256+0000","last_active":"2023-11-06T22:00:18.094256+0000","last_peered":"2023-11-06T22:00:18.094256+0000","last_clean":"2023-11-06T22:00:18.094256+0000","last_became_active":"2023-11-06T22:00:18.093983+0000","last_became_peered":"2023-11-06T22:00:18.093983+0000","last_unstale":"2023-11-06T22:00:18.094256+0000","last_undegraded":"2023-11-06T22:00:18.094256+0000","last_fullsized":"2023-11-06T22:00:18.094256+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.092770+0000","last_change":"2023-11-06T22:00:17.612695+0000","last_active":"2023-11-06T22:00:18.092770+0000","last_peered":"2023-11-06T22:00:18.092770+0000","last_clean":"2023-11-06T22:00:18.092770+0000","last_became_active":"2023-11-06T22:00:17.612080+0000","last_became_peered":"2023-11-06T22:00:17.612080+0000","last_unstale":"2023-11-06T22:00:18.092770+0000","last_undegraded":"2023-11-06T22:00:18.092770+0000","last_fullsized":"2023-11-06T22:00:18.092770+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.097937+0000","last_change":"2023-11-06T22:00:18.093851+0000","last_active":"2023-11-06T22:00:18.097937+0000","last_peered":"2023-11-06T22:00:18.097937+0000","last_clean":"2023-11-06T22:00:18.097937+0000","last_became_active":"2023-11-06T22:00:18.093064+0000","last_became_peered":"2023-11-06T22:00:18.093064+0000","last_unstale":"2023-11-06T22:00:18.097937+0000","last_undegraded":"2023-11-06T22:00:18.097937+0000","last_fullsized":"2023-11-06T22:00:18.097937+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.616729+0000","last_change":"2023-11-06T22:00:17.609329+0000","last_active":"2023-11-06T22:00:17.616729+0000","last_peered":"2023-11-06T22:00:17.616729+0000","last_clean":"2023-11-06T22:00:17.616729+0000","last_became_active":"2023-11-06T22:00:17.608768+0000","last_became_peered":"2023-11-06T22:00:17.608768+0000","last_unstale":"2023-11-06T22:00:17.616729+0000","last_undegraded":"2023-11-06T22:00:17.616729+0000","last_fullsized":"2023-11-06T22:00:17.616729+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.096257+0000","last_change":"2023-11-06T22:00:18.096257+0000","last_active":"2023-11-06T22:00:18.096257+0000","last_peered":"2023-11-06T22:00:18.096257+0000","last_clean":"2023-11-06T22:00:18.096257+0000","last_became_active":"2023-11-06T22:00:18.095899+0000","last_became_peered":"2023-11-06T22:00:18.095899+0000","last_unstale":"2023-11-06T22:00:18.096257+0000","last_undegraded":"2023-11-06T22:00:18.096257+0000","last_fullsized":"2023-11-06T22:00:18.096257+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.091705+0000","last_change":"2023-11-06T22:00:17.621160+0000","last_active":"2023-11-06T22:00:18.091705+0000","last_peered":"2023-11-06T22:00:18.091705+0000","last_clean":"2023-11-06T22:00:18.091705+0000","last_became_active":"2023-11-06T22:00:17.620808+0000","last_became_peered":"2023-11-06T22:00:17.620808+0000","last_unstale":"2023-11-06T22:00:18.091705+0000","last_undegraded":"2023-11-06T22:00:18.091705+0000","last_fullsized":"2023-11-06T22:00:18.091705+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.095948+0000","last_change":"2023-11-06T22:00:18.089020+0000","last_active":"2023-11-06T22:00:18.095948+0000","last_peered":"2023-11-06T22:00:18.095948+0000","last_clean":"2023-11-06T22:00:18.095948+0000","last_became_active":"2023-11-06T22:00:18.088747+0000","last_became_peered":"2023-11-06T22:00:18.088747+0000","last_unstale":"2023-11-06T22:00:18.095948+0000","last_undegraded":"2023-11-06T22:00:18.095948+0000","last_fullsized":"2023-11-06T22:00:18.095948+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.f","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112809+0000","last_change":"2023-11-06T22:00:17.615798+0000","last_active":"2023-11-06T22:00:18.112809+0000","last_peered":"2023-11-06T22:00:18.112809+0000","last_clean":"2023-11-06T22:00:18.112809+0000","last_became_active":"2023-11-06T22:00:17.613829+0000","last_became_peered":"2023-11-06T22:00:17.613829+0000","last_unstale":"2023-11-06T22:00:18.112809+0000","last_undegraded":"2023-11-06T22:00:18.112809+0000","last_fullsized":"2023-11-06T22:00:18.112809+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.103273+0000","last_change":"2023-11-06T22:00:18.097845+0000","last_active":"2023-11-06T22:00:18.103273+0000","last_peered":"2023-11-06T22:00:18.103273+0000","last_clean":"2023-11-06T22:00:18.103273+0000","last_became_active":"2023-11-06T22:00:18.097610+0000","last_became_peered":"2023-11-06T22:00:18.097610+0000","last_unstale":"2023-11-06T22:00:18.103273+0000","last_undegraded":"2023-11-06T22:00:18.103273+0000","last_fullsized":"2023-11-06T22:00:18.103273+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.e","version":"184'2","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111281+0000","last_change":"2023-11-06T22:00:17.622266+0000","last_active":"2023-11-06T22:00:18.111281+0000","last_peered":"2023-11-06T22:00:18.111281+0000","last_clean":"2023-11-06T22:00:18.111281+0000","last_became_active":"2023-11-06T22:00:17.621950+0000","last_became_peered":"2023-11-06T22:00:17.621950+0000","last_unstale":"2023-11-06T22:00:18.111281+0000","last_undegraded":"2023-11-06T22:00:18.111281+0000","last_fullsized":"2023-11-06T22:00:18.111281+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.107611+0000","last_change":"2023-11-06T22:00:18.100393+0000","last_active":"2023-11-06T22:00:18.107611+0000","last_peered":"2023-11-06T22:00:18.107611+0000","last_clean":"2023-11-06T22:00:18.107611+0000","last_became_active":"2023-11-06T22:00:18.100096+0000","last_became_peered":"2023-11-06T22:00:18.100096+0000","last_unstale":"2023-11-06T22:00:18.107611+0000","last_undegraded":"2023-11-06T22:00:18.107611+0000","last_fullsized":"2023-11-06T22:00:18.107611+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.625820+0000","last_change":"2023-11-06T22:00:17.622030+0000","last_active":"2023-11-06T22:00:17.625820+0000","last_peered":"2023-11-06T22:00:17.625820+0000","last_clean":"2023-11-06T22:00:17.625820+0000","last_became_active":"2023-11-06T22:00:17.621721+0000","last_became_peered":"2023-11-06T22:00:17.621721+0000","last_unstale":"2023-11-06T22:00:17.625820+0000","last_undegraded":"2023-11-06T22:00:17.625820+0000","last_fullsized":"2023-11-06T22:00:17.625820+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.104253+0000","last_change":"2023-11-06T22:00:18.099012+0000","last_active":"2023-11-06T22:00:18.104253+0000","last_peered":"2023-11-06T22:00:18.104253+0000","last_clean":"2023-11-06T22:00:18.104253+0000","last_became_active":"2023-11-06T22:00:18.098329+0000","last_became_peered":"2023-11-06T22:00:18.098329+0000","last_unstale":"2023-11-06T22:00:18.104253+0000","last_undegraded":"2023-11-06T22:00:18.104253+0000","last_fullsized":"2023-11-06T22:00:18.104253+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.083283+0000","last_change":"2023-11-06T22:00:17.622289+0000","last_active":"2023-11-06T22:00:18.083283+0000","last_peered":"2023-11-06T22:00:18.083283+0000","last_clean":"2023-11-06T22:00:18.083283+0000","last_became_active":"2023-11-06T22:00:17.621995+0000","last_became_peered":"2023-11-06T22:00:17.621995+0000","last_unstale":"2023-11-06T22:00:18.083283+0000","last_undegraded":"2023-11-06T22:00:18.083283+0000","last_fullsized":"2023-11-06T22:00:18.083283+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.106650+0000","last_change":"2023-11-06T22:00:18.101583+0000","last_active":"2023-11-06T22:00:18.106650+0000","last_peered":"2023-11-06T22:00:18.106650+0000","last_clean":"2023-11-06T22:00:18.106650+0000","last_became_active":"2023-11-06T22:00:18.101269+0000","last_became_peered":"2023-11-06T22:00:18.101269+0000","last_unstale":"2023-11-06T22:00:18.106650+0000","last_undegraded":"2023-11-06T22:00:18.106650+0000","last_fullsized":"2023-11-06T22:00:18.106650+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.b","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112454+0000","last_change":"2023-11-06T22:00:17.614210+0000","last_active":"2023-11-06T22:00:18.112454+0000","last_peered":"2023-11-06T22:00:18.112454+0000","last_clean":"2023-11-06T22:00:18.112454+0000","last_became_active":"2023-11-06T22:00:17.612970+0000","last_became_peered":"2023-11-06T22:00:17.612970+0000","last_unstale":"2023-11-06T22:00:18.112454+0000","last_undegraded":"2023-11-06T22:00:18.112454+0000","last_fullsized":"2023-11-06T22:00:18.112454+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.099289+0000","last_change":"2023-11-06T22:00:18.096595+0000","last_active":"2023-11-06T22:00:18.099289+0000","last_peered":"2023-11-06T22:00:18.099289+0000","last_clean":"2023-11-06T22:00:18.099289+0000","last_became_active":"2023-11-06T22:00:18.096322+0000","last_became_peered":"2023-11-06T22:00:18.096322+0000","last_unstale":"2023-11-06T22:00:18.099289+0000","last_undegraded":"2023-11-06T22:00:18.099289+0000","last_fullsized":"2023-11-06T22:00:18.099289+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094772+0000","last_change":"2023-11-06T22:00:17.605537+0000","last_active":"2023-11-06T22:00:18.094772+0000","last_peered":"2023-11-06T22:00:18.094772+0000","last_clean":"2023-11-06T22:00:18.094772+0000","last_became_active":"2023-11-06T22:00:17.605222+0000","last_became_peered":"2023-11-06T22:00:17.605222+0000","last_unstale":"2023-11-06T22:00:18.094772+0000","last_undegraded":"2023-11-06T22:00:18.094772+0000","last_fullsized":"2023-11-06T22:00:18.094772+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.110632+0000","last_change":"2023-11-06T22:00:18.099860+0000","last_active":"2023-11-06T22:00:18.110632+0000","last_peered":"2023-11-06T22:00:18.110632+0000","last_clean":"2023-11-06T22:00:18.110632+0000","last_became_active":"2023-11-06T22:00:18.099597+0000","last_became_peered":"2023-11-06T22:00:18.099597+0000","last_unstale":"2023-11-06T22:00:18.110632+0000","last_undegraded":"2023-11-06T22:00:18.110632+0000","last_fullsized":"2023-11-06T22:00:18.110632+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.092160+0000","last_change":"2023-11-06T22:00:17.614059+0000","last_active":"2023-11-06T22:00:18.092160+0000","last_peered":"2023-11-06T22:00:18.092160+0000","last_clean":"2023-11-06T22:00:18.092160+0000","last_became_active":"2023-11-06T22:00:17.613718+0000","last_became_peered":"2023-11-06T22:00:17.613718+0000","last_unstale":"2023-11-06T22:00:18.092160+0000","last_undegraded":"2023-11-06T22:00:18.092160+0000","last_fullsized":"2023-11-06T22:00:18.092160+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.081889+0000","last_change":"2023-11-06T22:00:18.081889+0000","last_active":"2023-11-06T22:00:18.081889+0000","last_peered":"2023-11-06T22:00:18.081889+0000","last_clean":"2023-11-06T22:00:18.081889+0000","last_became_active":"2023-11-06T22:00:18.081313+0000","last_became_peered":"2023-11-06T22:00:18.081313+0000","last_unstale":"2023-11-06T22:00:18.081889+0000","last_undegraded":"2023-11-06T22:00:18.081889+0000","last_fullsized":"2023-11-06T22:00:18.081889+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.093276+0000","last_change":"2023-11-06T22:00:17.615319+0000","last_active":"2023-11-06T22:00:18.093276+0000","last_peered":"2023-11-06T22:00:18.093276+0000","last_clean":"2023-11-06T22:00:18.093276+0000","last_became_active":"2023-11-06T22:00:17.615055+0000","last_became_peered":"2023-11-06T22:00:17.615055+0000","last_unstale":"2023-11-06T22:00:18.093276+0000","last_undegraded":"2023-11-06T22:00:18.093276+0000","last_fullsized":"2023-11-06T22:00:18.093276+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.081880+0000","last_change":"2023-11-06T22:00:18.081880+0000","last_active":"2023-11-06T22:00:18.081880+0000","last_peered":"2023-11-06T22:00:18.081880+0000","last_clean":"2023-11-06T22:00:18.081880+0000","last_became_active":"2023-11-06T22:00:18.081323+0000","last_became_peered":"2023-11-06T22:00:18.081323+0000","last_unstale":"2023-11-06T22:00:18.081880+0000","last_undegraded":"2023-11-06T22:00:18.081880+0000","last_fullsized":"2023-11-06T22:00:18.081880+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.7","version":"184'2","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.117829+0000","last_change":"2023-11-06T22:00:17.610204+0000","last_active":"2023-11-06T22:00:18.117829+0000","last_peered":"2023-11-06T22:00:18.117829+0000","last_clean":"2023-11-06T22:00:18.117829+0000","last_became_active":"2023-11-06T22:00:17.609777+0000","last_became_peered":"2023-11-06T22:00:17.609777+0000","last_unstale":"2023-11-06T22:00:18.117829+0000","last_undegraded":"2023-11-06T22:00:18.117829+0000","last_fullsized":"2023-11-06T22:00:18.117829+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.106792+0000","last_change":"2023-11-06T22:00:18.106792+0000","last_active":"2023-11-06T22:00:18.106792+0000","last_peered":"2023-11-06T22:00:18.106792+0000","last_clean":"2023-11-06T22:00:18.106792+0000","last_became_active":"2023-11-06T22:00:18.106457+0000","last_became_peered":"2023-11-06T22:00:18.106457+0000","last_unstale":"2023-11-06T22:00:18.106792+0000","last_undegraded":"2023-11-06T22:00:18.106792+0000","last_fullsized":"2023-11-06T22:00:18.106792+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.614124+0000","last_change":"2023-11-06T22:00:17.608612+0000","last_active":"2023-11-06T22:00:17.614124+0000","last_peered":"2023-11-06T22:00:17.614124+0000","last_clean":"2023-11-06T22:00:17.614124+0000","last_became_active":"2023-11-06T22:00:17.608190+0000","last_became_peered":"2023-11-06T22:00:17.608190+0000","last_unstale":"2023-11-06T22:00:17.614124+0000","last_undegraded":"2023-11-06T22:00:17.614124+0000","last_fullsized":"2023-11-06T22:00:17.614124+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.108226+0000","last_change":"2023-11-06T22:00:18.095118+0000","last_active":"2023-11-06T22:00:18.108226+0000","last_peered":"2023-11-06T22:00:18.108226+0000","last_clean":"2023-11-06T22:00:18.108226+0000","last_became_active":"2023-11-06T22:00:18.094856+0000","last_became_peered":"2023-11-06T22:00:18.094856+0000","last_unstale":"2023-11-06T22:00:18.108226+0000","last_undegraded":"2023-11-06T22:00:18.108226+0000","last_fullsized":"2023-11-06T22:00:18.108226+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.087144+0000","last_change":"2023-11-06T22:00:17.609707+0000","last_active":"2023-11-06T22:00:18.087144+0000","last_peered":"2023-11-06T22:00:18.087144+0000","last_clean":"2023-11-06T22:00:18.087144+0000","last_became_active":"2023-11-06T22:00:17.609400+0000","last_became_peered":"2023-11-06T22:00:17.609400+0000","last_unstale":"2023-11-06T22:00:18.087144+0000","last_undegraded":"2023-11-06T22:00:18.087144+0000","last_fullsized":"2023-11-06T22:00:18.087144+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.109937+0000","last_change":"2023-11-06T22:00:18.103751+0000","last_active":"2023-11-06T22:00:18.109937+0000","last_peered":"2023-11-06T22:00:18.109937+0000","last_clean":"2023-11-06T22:00:18.109937+0000","last_became_active":"2023-11-06T22:00:18.098912+0000","last_became_peered":"2023-11-06T22:00:18.098912+0000","last_unstale":"2023-11-06T22:00:18.109937+0000","last_undegraded":"2023-11-06T22:00:18.109937+0000","last_fullsized":"2023-11-06T22:00:18.109937+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.087889+0000","last_change":"2023-11-06T22:00:17.606496+0000","last_active":"2023-11-06T22:00:18.087889+0000","last_peered":"2023-11-06T22:00:18.087889+0000","last_clean":"2023-11-06T22:00:18.087889+0000","last_became_active":"2023-11-06T22:00:17.605847+0000","last_became_peered":"2023-11-06T22:00:17.605847+0000","last_unstale":"2023-11-06T22:00:18.087889+0000","last_undegraded":"2023-11-06T22:00:18.087889+0000","last_fullsized":"2023-11-06T22:00:18.087889+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.092946+0000","last_change":"2023-11-06T22:00:18.092946+0000","last_active":"2023-11-06T22:00:18.092946+0000","last_peered":"2023-11-06T22:00:18.092946+0000","last_clean":"2023-11-06T22:00:18.092946+0000","last_became_active":"2023-11-06T22:00:18.092704+0000","last_became_peered":"2023-11-06T22:00:18.092704+0000","last_unstale":"2023-11-06T22:00:18.092946+0000","last_undegraded":"2023-11-06T22:00:18.092946+0000","last_fullsized":"2023-11-06T22:00:18.092946+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.620214+0000","last_change":"2023-11-06T22:00:17.612831+0000","last_active":"2023-11-06T22:00:17.620214+0000","last_peered":"2023-11-06T22:00:17.620214+0000","last_clean":"2023-11-06T22:00:17.620214+0000","last_became_active":"2023-11-06T22:00:17.612448+0000","last_became_peered":"2023-11-06T22:00:17.612448+0000","last_unstale":"2023-11-06T22:00:17.620214+0000","last_undegraded":"2023-11-06T22:00:17.620214+0000","last_fullsized":"2023-11-06T22:00:17.620214+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.107228+0000","last_change":"2023-11-06T22:00:18.107228+0000","last_active":"2023-11-06T22:00:18.107228+0000","last_peered":"2023-11-06T22:00:18.107228+0000","last_clean":"2023-11-06T22:00:18.107228+0000","last_became_active":"2023-11-06T22:00:18.106967+0000","last_became_peered":"2023-11-06T22:00:18.106967+0000","last_unstale":"2023-11-06T22:00:18.107228+0000","last_undegraded":"2023-11-06T22:00:18.107228+0000","last_fullsized":"2023-11-06T22:00:18.107228+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.608853+0000","last_change":"2023-11-06T22:00:17.603418+0000","last_active":"2023-11-06T22:00:17.608853+0000","last_peered":"2023-11-06T22:00:17.608853+0000","last_clean":"2023-11-06T22:00:17.608853+0000","last_became_active":"2023-11-06T22:00:17.603124+0000","last_became_peered":"2023-11-06T22:00:17.603124+0000","last_unstale":"2023-11-06T22:00:17.608853+0000","last_undegraded":"2023-11-06T22:00:17.608853+0000","last_fullsized":"2023-11-06T22:00:17.608853+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.102966+0000","last_change":"2023-11-06T22:00:18.102966+0000","last_active":"2023-11-06T22:00:18.102966+0000","last_peered":"2023-11-06T22:00:18.102966+0000","last_clean":"2023-11-06T22:00:18.102966+0000","last_became_active":"2023-11-06T22:00:18.102611+0000","last_became_peered":"2023-11-06T22:00:18.102611+0000","last_unstale":"2023-11-06T22:00:18.102966+0000","last_undegraded":"2023-11-06T22:00:18.102966+0000","last_fullsized":"2023-11-06T22:00:18.102966+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.076583+0000","last_change":"2023-11-06T22:00:17.604459+0000","last_active":"2023-11-06T22:00:18.076583+0000","last_peered":"2023-11-06T22:00:18.076583+0000","last_clean":"2023-11-06T22:00:18.076583+0000","last_became_active":"2023-11-06T22:00:17.604112+0000","last_became_peered":"2023-11-06T22:00:17.604112+0000","last_unstale":"2023-11-06T22:00:18.076583+0000","last_undegraded":"2023-11-06T22:00:18.076583+0000","last_fullsized":"2023-11-06T22:00:18.076583+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.103190+0000","last_change":"2023-11-06T22:00:18.094531+0000","last_active":"2023-11-06T22:00:18.103190+0000","last_peered":"2023-11-06T22:00:18.103190+0000","last_clean":"2023-11-06T22:00:18.103190+0000","last_became_active":"2023-11-06T22:00:18.094240+0000","last_became_peered":"2023-11-06T22:00:18.094240+0000","last_unstale":"2023-11-06T22:00:18.103190+0000","last_undegraded":"2023-11-06T22:00:18.103190+0000","last_fullsized":"2023-11-06T22:00:18.103190+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.610512+0000","last_change":"2023-11-06T22:00:17.604567+0000","last_active":"2023-11-06T22:00:17.610512+0000","last_peered":"2023-11-06T22:00:17.610512+0000","last_clean":"2023-11-06T22:00:17.610512+0000","last_became_active":"2023-11-06T22:00:17.604261+0000","last_became_peered":"2023-11-06T22:00:17.604261+0000","last_unstale":"2023-11-06T22:00:17.610512+0000","last_undegraded":"2023-11-06T22:00:17.610512+0000","last_fullsized":"2023-11-06T22:00:17.610512+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.093135+0000","last_change":"2023-11-06T22:00:18.089020+0000","last_active":"2023-11-06T22:00:18.093135+0000","last_peered":"2023-11-06T22:00:18.093135+0000","last_clean":"2023-11-06T22:00:18.093135+0000","last_became_active":"2023-11-06T22:00:18.088718+0000","last_became_peered":"2023-11-06T22:00:18.088718+0000","last_unstale":"2023-11-06T22:00:18.093135+0000","last_undegraded":"2023-11-06T22:00:18.093135+0000","last_fullsized":"2023-11-06T22:00:18.093135+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111451+0000","last_change":"2023-11-06T22:00:17.603597+0000","last_active":"2023-11-06T22:00:18.111451+0000","last_peered":"2023-11-06T22:00:18.111451+0000","last_clean":"2023-11-06T22:00:18.111451+0000","last_became_active":"2023-11-06T22:00:17.603245+0000","last_became_peered":"2023-11-06T22:00:17.603245+0000","last_unstale":"2023-11-06T22:00:18.111451+0000","last_undegraded":"2023-11-06T22:00:18.111451+0000","last_fullsized":"2023-11-06T22:00:18.111451+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.591800+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T22:00:17.591800+0000","last_peered":"2023-11-06T22:00:17.591800+0000","last_clean":"2023-11-06T22:00:17.591800+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:00:17.591800+0000","last_undegraded":"2023-11-06T22:00:17.591800+0000","last_fullsized":"2023-11-06T22:00:17.591800+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.110595+0000","last_change":"2023-11-06T22:00:18.107017+0000","last_active":"2023-11-06T22:00:18.110595+0000","last_peered":"2023-11-06T22:00:18.110595+0000","last_clean":"2023-11-06T22:00:18.110595+0000","last_became_active":"2023-11-06T22:00:18.106699+0000","last_became_peered":"2023-11-06T22:00:18.106699+0000","last_unstale":"2023-11-06T22:00:18.110595+0000","last_undegraded":"2023-11-06T22:00:18.110595+0000","last_fullsized":"2023-11-06T22:00:18.110595+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"184'3","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112508+0000","last_change":"2023-11-06T22:00:17.606776+0000","last_active":"2023-11-06T22:00:18.112508+0000","last_peered":"2023-11-06T22:00:18.112508+0000","last_clean":"2023-11-06T22:00:18.112508+0000","last_became_active":"2023-11-06T22:00:17.605542+0000","last_became_peered":"2023-11-06T22:00:17.605542+0000","last_unstale":"2023-11-06T22:00:18.112508+0000","last_undegraded":"2023-11-06T22:00:18.112508+0000","last_fullsized":"2023-11-06T22:00:18.112508+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077152+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T22:00:18.077152+0000","last_peered":"2023-11-06T22:00:18.077152+0000","last_clean":"2023-11-06T22:00:18.077152+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:00:18.077152+0000","last_undegraded":"2023-11-06T22:00:18.077152+0000","last_fullsized":"2023-11-06T22:00:18.077152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.096360+0000","last_change":"2023-11-06T22:00:18.096360+0000","last_active":"2023-11-06T22:00:18.096360+0000","last_peered":"2023-11-06T22:00:18.096360+0000","last_clean":"2023-11-06T22:00:18.096360+0000","last_became_active":"2023-11-06T22:00:18.096025+0000","last_became_peered":"2023-11-06T22:00:18.096025+0000","last_unstale":"2023-11-06T22:00:18.096360+0000","last_undegraded":"2023-11-06T22:00:18.096360+0000","last_fullsized":"2023-11-06T22:00:18.096360+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.617231+0000","last_change":"2023-11-06T22:00:17.610418+0000","last_active":"2023-11-06T22:00:17.617231+0000","last_peered":"2023-11-06T22:00:17.617231+0000","last_clean":"2023-11-06T22:00:17.617231+0000","last_became_active":"2023-11-06T22:00:17.608860+0000","last_became_peered":"2023-11-06T22:00:17.608860+0000","last_unstale":"2023-11-06T22:00:17.617231+0000","last_undegraded":"2023-11-06T22:00:17.617231+0000","last_fullsized":"2023-11-06T22:00:17.617231+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.092254+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T22:00:18.092254+0000","last_peered":"2023-11-06T22:00:18.092254+0000","last_clean":"2023-11-06T22:00:18.092254+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:00:18.092254+0000","last_undegraded":"2023-11-06T22:00:18.092254+0000","last_fullsized":"2023-11-06T22:00:18.092254+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.104272+0000","last_change":"2023-11-06T22:00:18.099178+0000","last_active":"2023-11-06T22:00:18.104272+0000","last_peered":"2023-11-06T22:00:18.104272+0000","last_clean":"2023-11-06T22:00:18.104272+0000","last_became_active":"2023-11-06T22:00:18.098739+0000","last_became_peered":"2023-11-06T22:00:18.098739+0000","last_unstale":"2023-11-06T22:00:18.104272+0000","last_undegraded":"2023-11-06T22:00:18.104272+0000","last_fullsized":"2023-11-06T22:00:18.104272+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.14","version":"184'2","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112191+0000","last_change":"2023-11-06T22:00:17.597193+0000","last_active":"2023-11-06T22:00:18.112191+0000","last_peered":"2023-11-06T22:00:18.112191+0000","last_clean":"2023-11-06T22:00:18.112191+0000","last_became_active":"2023-11-06T22:00:17.596847+0000","last_became_peered":"2023-11-06T22:00:17.596847+0000","last_unstale":"2023-11-06T22:00:18.112191+0000","last_undegraded":"2023-11-06T22:00:18.112191+0000","last_fullsized":"2023-11-06T22:00:18.112191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.076920+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T22:00:18.076920+0000","last_peered":"2023-11-06T22:00:18.076920+0000","last_clean":"2023-11-06T22:00:18.076920+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:00:18.076920+0000","last_undegraded":"2023-11-06T22:00:18.076920+0000","last_fullsized":"2023-11-06T22:00:18.076920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":157,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.082848+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T22:00:18.082848+0000","last_peered":"2023-11-06T22:00:18.082848+0000","last_clean":"2023-11-06T22:00:18.082848+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:00:18.082848+0000","last_undegraded":"2023-11-06T22:00:18.082848+0000","last_fullsized":"2023-11-06T22:00:18.082848+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.098384+0000","last_change":"2023-11-06T22:00:18.090235+0000","last_active":"2023-11-06T22:00:18.098384+0000","last_peered":"2023-11-06T22:00:18.098384+0000","last_clean":"2023-11-06T22:00:18.098384+0000","last_became_active":"2023-11-06T22:00:18.089449+0000","last_became_peered":"2023-11-06T22:00:18.089449+0000","last_unstale":"2023-11-06T22:00:18.098384+0000","last_undegraded":"2023-11-06T22:00:18.098384+0000","last_fullsized":"2023-11-06T22:00:18.098384+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.093271+0000","last_change":"2023-11-06T22:00:17.609973+0000","last_active":"2023-11-06T22:00:18.093271+0000","last_peered":"2023-11-06T22:00:18.093271+0000","last_clean":"2023-11-06T22:00:18.093271+0000","last_became_active":"2023-11-06T22:00:17.609628+0000","last_became_peered":"2023-11-06T22:00:17.609628+0000","last_unstale":"2023-11-06T22:00:18.093271+0000","last_undegraded":"2023-11-06T22:00:18.093271+0000","last_fullsized":"2023-11-06T22:00:18.093271+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.076646+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T22:00:18.076646+0000","last_peered":"2023-11-06T22:00:18.076646+0000","last_clean":"2023-11-06T22:00:18.076646+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:00:18.076646+0000","last_undegraded":"2023-11-06T22:00:18.076646+0000","last_fullsized":"2023-11-06T22:00:18.076646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.098485+0000","last_change":"2023-11-06T22:00:18.098485+0000","last_active":"2023-11-06T22:00:18.098485+0000","last_peered":"2023-11-06T22:00:18.098485+0000","last_clean":"2023-11-06T22:00:18.098485+0000","last_became_active":"2023-11-06T22:00:18.098208+0000","last_became_peered":"2023-11-06T22:00:18.098208+0000","last_unstale":"2023-11-06T22:00:18.098485+0000","last_undegraded":"2023-11-06T22:00:18.098485+0000","last_fullsized":"2023-11-06T22:00:18.098485+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.621745+0000","last_change":"2023-11-06T22:00:17.618031+0000","last_active":"2023-11-06T22:00:17.621745+0000","last_peered":"2023-11-06T22:00:17.621745+0000","last_clean":"2023-11-06T22:00:17.621745+0000","last_became_active":"2023-11-06T22:00:17.617769+0000","last_became_peered":"2023-11-06T22:00:17.617769+0000","last_unstale":"2023-11-06T22:00:17.621745+0000","last_undegraded":"2023-11-06T22:00:17.621745+0000","last_fullsized":"2023-11-06T22:00:17.621745+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077170+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:00:18.077170+0000","last_peered":"2023-11-06T22:00:18.077170+0000","last_clean":"2023-11-06T22:00:18.077170+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:00:18.077170+0000","last_undegraded":"2023-11-06T22:00:18.077170+0000","last_fullsized":"2023-11-06T22:00:18.077170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111515+0000","last_change":"2023-11-06T22:00:18.109508+0000","last_active":"2023-11-06T22:00:18.111515+0000","last_peered":"2023-11-06T22:00:18.111515+0000","last_clean":"2023-11-06T22:00:18.111515+0000","last_became_active":"2023-11-06T22:00:18.109086+0000","last_became_peered":"2023-11-06T22:00:18.109086+0000","last_unstale":"2023-11-06T22:00:18.111515+0000","last_undegraded":"2023-11-06T22:00:18.111515+0000","last_fullsized":"2023-11-06T22:00:18.111515+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.617878+0000","last_change":"2023-11-06T22:00:17.611622+0000","last_active":"2023-11-06T22:00:17.617878+0000","last_peered":"2023-11-06T22:00:17.617878+0000","last_clean":"2023-11-06T22:00:17.617878+0000","last_became_active":"2023-11-06T22:00:17.609294+0000","last_became_peered":"2023-11-06T22:00:17.609294+0000","last_unstale":"2023-11-06T22:00:17.617878+0000","last_undegraded":"2023-11-06T22:00:17.617878+0000","last_fullsized":"2023-11-06T22:00:17.617878+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.089731+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T22:00:18.089731+0000","last_peered":"2023-11-06T22:00:18.089731+0000","last_clean":"2023-11-06T22:00:18.089731+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:00:18.089731+0000","last_undegraded":"2023-11-06T22:00:18.089731+0000","last_fullsized":"2023-11-06T22:00:18.089731+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094208+0000","last_change":"2023-11-06T22:00:18.092304+0000","last_active":"2023-11-06T22:00:18.094208+0000","last_peered":"2023-11-06T22:00:18.094208+0000","last_clean":"2023-11-06T22:00:18.094208+0000","last_became_active":"2023-11-06T22:00:18.091811+0000","last_became_peered":"2023-11-06T22:00:18.091811+0000","last_unstale":"2023-11-06T22:00:18.094208+0000","last_undegraded":"2023-11-06T22:00:18.094208+0000","last_fullsized":"2023-11-06T22:00:18.094208+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.10","version":"184'4","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.118038+0000","last_change":"2023-11-06T22:00:17.607854+0000","last_active":"2023-11-06T22:00:18.118038+0000","last_peered":"2023-11-06T22:00:18.118038+0000","last_clean":"2023-11-06T22:00:18.118038+0000","last_became_active":"2023-11-06T22:00:17.607602+0000","last_became_peered":"2023-11-06T22:00:17.607602+0000","last_unstale":"2023-11-06T22:00:18.118038+0000","last_undegraded":"2023-11-06T22:00:18.118038+0000","last_fullsized":"2023-11-06T22:00:18.118038+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077512+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:00:18.077512+0000","last_peered":"2023-11-06T22:00:18.077512+0000","last_clean":"2023-11-06T22:00:18.077512+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:00:18.077512+0000","last_undegraded":"2023-11-06T22:00:18.077512+0000","last_fullsized":"2023-11-06T22:00:18.077512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575756,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307548,"kb_used_data":2260,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064292,"statfs":{"total":96636764160,"available":96321835008,"internally_reserved":0,"allocated":2314240,"data_stored":1178856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68799999999999994,"15min":0.71099999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.93999999999999995,"5min":0.997,"15min":1.002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.012,"5min":1.022,"15min":1.022},"last":0.66000000000000003}]},{"osd":1,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88400000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":1.083,"15min":1.083},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71999999999999997,"15min":0.746},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.90200000000000002,"5min":1.004,"15min":1.004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":0.97299999999999998,"15min":1.0029999999999999},"last":0.48399999999999999}]},{"osd":3,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88300000000000001,"5min":1.038,"15min":1.038},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.747,"15min":0.75800000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.86099999999999999,"5min":1.046,"15min":1.046},"last":0.73399999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.377},"max":{"1min":0.83099999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75900000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96499999999999997,"5min":1.145,"15min":1.145},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.68799999999999994},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92000000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.747}]},{"osd":6,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.495},"max":{"1min":1.0629999999999999,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69999999999999996,"15min":0.70699999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.84499999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.59699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575756,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307532,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064308,"statfs":{"total":96636764160,"available":96321851392,"internally_reserved":0,"allocated":2297856,"data_stored":1172066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.40000000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.04},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71099999999999997,"15min":0.71199999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.37},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.0209999999999999},"last":0.81000000000000005}]},{"osd":1,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83099999999999996,"5min":0.93600000000000005,"15min":0.99399999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.32600000000000001},"max":{"1min":0.91100000000000003,"5min":1.004,"15min":1.004},"last":0.79500000000000004}]},{"osd":2,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95299999999999996,"5min":1.034,"15min":1.034},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94499999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":1.02},"last":0.753},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76200000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.503},"max":{"1min":0.89700000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.69599999999999995}]},{"osd":4,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.39000000000000001},"max":{"1min":0.84899999999999998,"5min":0.85199999999999998,"15min":0.94399999999999995},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64400000000000002,"15min":0.63300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.434},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.90300000000000002}]},{"osd":5,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.70199999999999996},"min":{"1min":0.499,"5min":0.498,"15min":0.46200000000000002},"max":{"1min":0.95599999999999996,"5min":0.98099999999999998,"15min":0.98699999999999999},"last":0.878}]},{"osd":7,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.44400000000000001},"max":{"1min":0.84099999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.48899999999999999},"max":{"1min":0.93899999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.85999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2297856,"data_stored":1172018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97299999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"last":0.44900000000000001}]},{"osd":2,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0209999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.875,"5min":0.875,"15min":1.1850000000000001},"last":0.46700000000000003}]},{"osd":3,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.60399999999999998},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57099999999999995}]},{"osd":4,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0269999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83799999999999997,"15min":0.84699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59399999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.96499999999999997}]},{"osd":5,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.81299999999999994},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.55700000000000005},"max":{"1min":1.02,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.046,"5min":1.1559999999999999,"15min":1.198},"last":0.88600000000000001}]},{"osd":6,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.82399999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.107,"5min":1.1160000000000001,"15min":1.151},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.83699999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.51600000000000001},"max":{"1min":1.083,"5min":1.206,"15min":1.206},"last":0.95699999999999996}]},{"osd":7,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85299999999999998,"15min":0.86199999999999999},"min":{"1min":0.70899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1499999999999999,"5min":1.2070000000000001,"15min":1.21},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0980000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.97799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575756,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2301952,"data_stored":1172922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46300000000000002,"15min":0.47199999999999998},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.749,"15min":0.82899999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52400000000000002,"15min":0.52900000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.38800000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.34000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65400000000000003,"5min":0.72699999999999998,"15min":0.91700000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.376},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.48299999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.56299999999999994},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80500000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.59999999999999998},"min":{"1min":0.45500000000000002,"5min":0.436,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.88200000000000001},"last":0.44600000000000001}]},{"osd":4,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.57099999999999995},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.84699999999999998,"15min":0.84799999999999998},"min":{"1min":0.626,"5min":0.57399999999999995,"15min":0.51700000000000002},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.83799999999999997}]},{"osd":5,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85299999999999998,"15min":0.85399999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.306},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85699999999999998,"15min":0.86099999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.155,"5min":1.155,"15min":1.2729999999999999},"last":0.88400000000000001}]},{"osd":6,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82399999999999995,"15min":0.83399999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.55000000000000004},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.872},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.85399999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.56000000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.81899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575756,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2301952,"data_stored":1173442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.46200000000000002},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.28999999999999998},"max":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.32800000000000001},"max":{"1min":0.66900000000000004,"5min":0.84799999999999998,"15min":0.86299999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.432},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.499}]},{"osd":3,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51400000000000001,"15min":0.51500000000000001},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70299999999999996,"5min":0.77100000000000002,"15min":0.80000000000000004},"last":0.434},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.44800000000000001,"5min":0.432,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.89700000000000002,"15min":1.0940000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.622,"15min":0.55500000000000005},"max":{"1min":1.075,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83699999999999997,"15min":0.81699999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1020000000000001,"5min":1.256,"15min":1.256},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.81000000000000005},"min":{"1min":0.64400000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.86099999999999999,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.58299999999999996},"max":{"1min":1.0840000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.84199999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.51800000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.69299999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.145,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.88300000000000001,"15min":0.871},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.57299999999999995},"max":{"1min":1.173,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.752,"5min":0.57699999999999996,"15min":0.55900000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.79400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575757,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307536,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064304,"statfs":{"total":96636764160,"available":96321847296,"internally_reserved":0,"allocated":2301952,"data_stored":1172640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.33400000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.48599999999999999}]},{"osd":1,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.32400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.376},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.64400000000000002}]},{"osd":2,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.375},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.40899999999999997},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86199999999999999,"15min":0.84399999999999997},"min":{"1min":0.73799999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.85999999999999999,"15min":0.84599999999999997},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.47999999999999998},"max":{"1min":1.0309999999999999,"5min":1.202,"15min":1.202},"last":0.70999999999999996}]},{"osd":5,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.85699999999999998,"15min":0.82699999999999996},"min":{"1min":0.66000000000000003,"5min":0.59499999999999997,"15min":0.57399999999999995},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.67600000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.80600000000000005}]},{"osd":6,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.86899999999999999,"15min":0.83599999999999997},"min":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.63},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89500000000000002,"15min":0.86699999999999999},"min":{"1min":0.69399999999999995,"5min":0.66500000000000004,"15min":0.59999999999999998},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.69599999999999995}]},{"osd":7,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.87,"15min":0.85899999999999999},"min":{"1min":0.69299999999999995,"5min":0.58899999999999997,"15min":0.55500000000000005},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.92000000000000004,"15min":0.88100000000000001},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.56799999999999995},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575756,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303444,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068396,"statfs":{"total":96636764160,"available":96326037504,"internally_reserved":0,"allocated":2306048,"data_stored":1172654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":1.0589999999999999,"15min":1.1659999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82699999999999996,"5min":0.999,"15min":0.999},"last":0.73699999999999999}]},{"osd":1,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.747,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.44700000000000001},"max":{"1min":1.05,"5min":1.0700000000000001,"15min":1.131},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.99299999999999999,"15min":1.014},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.80500000000000005,"15min":0.79900000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91700000000000004,"5min":0.95299999999999996,"15min":1.181},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.73799999999999999,"15min":0.72499999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.47099999999999997},"max":{"1min":0.93100000000000005,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.81899999999999995}]},{"osd":5,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.61799999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.36899999999999999},"max":{"1min":0.90500000000000003,"5min":0.94499999999999995,"15min":1.228},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.443},"max":{"1min":0.80200000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.42799999999999999},"max":{"1min":0.88400000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.41999999999999998},"max":{"1min":1,"5min":1.0209999999999999,"15min":1.085},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.71799999999999997},"min":{"1min":0.51700000000000002,"5min":0.504,"15min":0.496},"max":{"1min":1.077,"5min":1.077,"15min":1.091},"last":0.88500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575757,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307540,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064300,"statfs":{"total":96636764160,"available":96321843200,"internally_reserved":0,"allocated":2306048,"data_stored":1172730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67500000000000004,"15min":0.67800000000000005},"min":{"1min":0.48999999999999999,"5min":0.38600000000000001,"15min":0.33600000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.008},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.46899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.98199999999999998,"15min":0.99299999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75,"15min":0.753},"min":{"1min":0.57899999999999996,"5min":0.375,"15min":0.375},"max":{"1min":0.92100000000000004,"5min":1.0009999999999999,"15min":1.2370000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.496,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.0629999999999999,"15min":1.2470000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.58499999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94399999999999995,"5min":1.085,"15min":1.085},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.753},"min":{"1min":0.66700000000000004,"5min":0.55100000000000005,"15min":0.47199999999999998},"max":{"1min":0.95599999999999996,"5min":1.036,"15min":1.036},"last":0.52200000000000002}]},{"osd":3,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.60499999999999998,"5min":0.49099999999999999,"15min":0.47399999999999998},"max":{"1min":0.88300000000000001,"5min":1.02,"15min":1.02},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.63500000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90400000000000003,"5min":1.093,"15min":1.093},"last":0.5}]},{"osd":4,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.628},{"interface":"front","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.69899999999999995},"min":{"1min":0.53400000000000003,"5min":0.437,"15min":0.39300000000000002},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":0.997},"last":0.51300000000000001}]},{"osd":6,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66900000000000004,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93400000000000005,"5min":0.99199999999999999,"15min":0.99299999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.751,"15min":0.72599999999999998},"min":{"1min":0.60299999999999998,"5min":0.46100000000000002,"15min":0.41499999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0449999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68500000000000005,"15min":0.66400000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.05},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72699999999999998,"15min":0.70099999999999996},"min":{"1min":0.48899999999999999,"5min":0.45700000000000002,"15min":0.436},"max":{"1min":0.95699999999999996,"5min":1.014,"15min":1.0780000000000001},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:00:22.418 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:00:22.560 INFO:tasks.ceph.mds.d.smithi069.stderr:2023-11-06T22:00:22.558+0000 7f50405f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T22:00:22.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.643+0000 7fdf777ff700 1 Processor -- start 2023-11-06T22:00:22.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.643+0000 7fdf777ff700 1 -- start start 2023-11-06T22:00:22.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.644+0000 7fdf777ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdf70154110 0x7fdf701544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.644+0000 7fdf777ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf700aade0 0x7fdf700645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.644+0000 7fdf777ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf70064ac0 0x7fdf700c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.644+0000 7fdf777ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf700bfd40 con 0x7fdf70064ac0 2023-11-06T22:00:22.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.644+0000 7fdf777ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf70064f70 con 0x7fdf70154110 2023-11-06T22:00:22.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.644+0000 7fdf777ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700c0880 con 0x7fdf700aade0 2023-11-06T22:00:22.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf75d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf70064ac0 0x7fdf700c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf75d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf70064ac0 0x7fdf700c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45102/0 (socket says 172.21.15.33:45102) 2023-11-06T22:00:22.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf75d9c700 1 -- 172.21.15.33:0/2044248105 learned_addr learned my addr 172.21.15.33:0/2044248105 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:22.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf7559b700 1 --2- 172.21.15.33:0/2044248105 >> v2:172.21.15.69:3300/0 conn(0x7fdf70154110 0x7fdf701544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf74d9a700 1 --2- 172.21.15.33:0/2044248105 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf700aade0 0x7fdf700645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf75d9c700 1 -- 172.21.15.33:0/2044248105 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf700aade0 msgr2=0x7fdf700645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:22.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf75d9c700 1 --2- 172.21.15.33:0/2044248105 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf700aade0 0x7fdf700645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf75d9c700 1 -- 172.21.15.33:0/2044248105 >> v2:172.21.15.69:3300/0 conn(0x7fdf70154110 msgr2=0x7fdf701544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:22.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf75d9c700 1 --2- 172.21.15.33:0/2044248105 >> v2:172.21.15.69:3300/0 conn(0x7fdf70154110 0x7fdf701544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf75d9c700 1 -- 172.21.15.33:0/2044248105 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700c8260 con 0x7fdf70064ac0 2023-11-06T22:00:22.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf75d9c700 1 --2- 172.21.15.33:0/2044248105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70064ac0 0x7fdf700c5db0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7fdf7006c130 tx=0x7fdf68009840).ready entity=mon.0 client_cookie=af75700ed04cd281 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf67fff700 1 -- 172.21.15.33:0/2044248105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6800e070 con 0x7fdf70064ac0 2023-11-06T22:00:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.645+0000 7fdf67fff700 1 -- 172.21.15.33:0/2044248105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6800ba50 con 0x7fdf70064ac0 2023-11-06T22:00:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.646+0000 7fdf67fff700 1 -- 172.21.15.33:0/2044248105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6800bdd0 con 0x7fdf70064ac0 2023-11-06T22:00:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.646+0000 7fdf777ff700 1 -- 172.21.15.33:0/2044248105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70064ac0 msgr2=0x7fdf700c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.646+0000 7fdf777ff700 1 --2- 172.21.15.33:0/2044248105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70064ac0 0x7fdf700c5db0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7fdf7006c130 tx=0x7fdf68009840).stop 2023-11-06T22:00:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.646+0000 7fdf777ff700 1 -- 172.21.15.33:0/2044248105 shutdown_connections 2023-11-06T22:00:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.646+0000 7fdf777ff700 1 --2- 172.21.15.33:0/2044248105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70064ac0 0x7fdf700c5db0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.646+0000 7fdf777ff700 1 --2- 172.21.15.33:0/2044248105 >> v2:172.21.15.69:3300/0 conn(0x7fdf70154110 0x7fdf701544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.646+0000 7fdf777ff700 1 --2- 172.21.15.33:0/2044248105 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf700aade0 0x7fdf700645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.646+0000 7fdf777ff700 1 -- 172.21.15.33:0/2044248105 >> 172.21.15.33:0/2044248105 conn(0x7fdf700cb8b0 msgr2=0x7fdf700c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.646+0000 7fdf777ff700 1 -- 172.21.15.33:0/2044248105 shutdown_connections 2023-11-06T22:00:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.646+0000 7fdf777ff700 1 -- 172.21.15.33:0/2044248105 wait complete. 2023-11-06T22:00:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.648+0000 7fdf777ff700 1 Processor -- start 2023-11-06T22:00:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.648+0000 7fdf777ff700 1 -- start start 2023-11-06T22:00:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf777ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf70064ac0 0x7fdf700ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf777ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf700aade0 0x7fdf70167690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf777ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70154110 0x7fdf7014f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf777ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf70154ad0 con 0x7fdf70154110 2023-11-06T22:00:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf777ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700ae500 con 0x7fdf70064ac0 2023-11-06T22:00:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf777ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf70154d40 con 0x7fdf700aade0 2023-11-06T22:00:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf75d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70154110 0x7fdf7014f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf75d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70154110 0x7fdf7014f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45114/0 (socket says 172.21.15.33:45114) 2023-11-06T22:00:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf75d9c700 1 -- 172.21.15.33:0/1628426437 learned_addr learned my addr 172.21.15.33:0/1628426437 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf75d9c700 1 -- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf700aade0 msgr2=0x7fdf70167690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf75d9c700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf700aade0 0x7fdf70167690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf75d9c700 1 -- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf70064ac0 msgr2=0x7fdf700ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf7559b700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf70064ac0 0x7fdf700ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf75d9c700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf70064ac0 0x7fdf700ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf75d9c700 1 -- 172.21.15.33:0/1628426437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf68009ae0 con 0x7fdf70154110 2023-11-06T22:00:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.649+0000 7fdf7559b700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf70064ac0 0x7fdf700ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:00:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.650+0000 7fdf75d9c700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70154110 0x7fdf7014f970 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7fdf68009fd0 tx=0x7fdf68009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.650+0000 7fdf657fa700 1 -- 172.21.15.33:0/1628426437 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6800bdd0 con 0x7fdf70154110 2023-11-06T22:00:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.650+0000 7fdf657fa700 1 -- 172.21.15.33:0/1628426437 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6801d460 con 0x7fdf70154110 2023-11-06T22:00:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.650+0000 7fdf777ff700 1 -- 172.21.15.33:0/1628426437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf70155020 con 0x7fdf70154110 2023-11-06T22:00:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.650+0000 7fdf657fa700 1 -- 172.21.15.33:0/1628426437 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6801d820 con 0x7fdf70154110 2023-11-06T22:00:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.650+0000 7fdf777ff700 1 -- 172.21.15.33:0/1628426437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf701536e0 con 0x7fdf70154110 2023-11-06T22:00:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.651+0000 7fdf777ff700 1 -- 172.21.15.33:0/1628426437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7006c6c0 con 0x7fdf70154110 2023-11-06T22:00:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.652+0000 7fdf657fa700 1 -- 172.21.15.33:0/1628426437 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdf68003040 con 0x7fdf70154110 2023-11-06T22:00:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.653+0000 7fdf657fa700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf5405f270 0x7fdf54061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.653+0000 7fdf657fa700 1 -- 172.21.15.33:0/1628426437 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fdf6807b9b0 con 0x7fdf70154110 2023-11-06T22:00:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.653+0000 7fdf7559b700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf5405f270 0x7fdf54061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.654+0000 7fdf7559b700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf5405f270 0x7fdf54061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fdf6c0098c0 tx=0x7fdf6c008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.657+0000 7fdf657fa700 1 -- 172.21.15.33:0/1628426437 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdf6804ca10 con 0x7fdf70154110 2023-11-06T22:00:22.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.834+0000 7fdf777ff700 1 -- 172.21.15.33:0/1628426437 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf700bff50 con 0x7fdf5405f270 2023-11-06T22:00:22.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.845+0000 7fdf657fa700 1 -- 172.21.15.33:0/1628426437 <== mgr.7352 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361406 (secure 0 0 0) 0x7fdf700bff50 con 0x7fdf5405f270 2023-11-06T22:00:22.852 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.849+0000 7fdf777ff700 1 -- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf5405f270 msgr2=0x7fdf54061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.849+0000 7fdf777ff700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf5405f270 0x7fdf54061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fdf6c0098c0 tx=0x7fdf6c008040).stop 2023-11-06T22:00:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.849+0000 7fdf777ff700 1 -- 172.21.15.33:0/1628426437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70154110 msgr2=0x7fdf7014f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.849+0000 7fdf777ff700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70154110 0x7fdf7014f970 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7fdf68009fd0 tx=0x7fdf68009840).stop 2023-11-06T22:00:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.849+0000 7fdf777ff700 1 -- 172.21.15.33:0/1628426437 shutdown_connections 2023-11-06T22:00:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.850+0000 7fdf777ff700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fdf5405f270 0x7fdf54061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.850+0000 7fdf777ff700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf70154110 0x7fdf7014f970 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.850+0000 7fdf777ff700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf70064ac0 0x7fdf700ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.850+0000 7fdf777ff700 1 --2- 172.21.15.33:0/1628426437 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdf700aade0 0x7fdf70167690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.850+0000 7fdf777ff700 1 -- 172.21.15.33:0/1628426437 >> 172.21.15.33:0/1628426437 conn(0x7fdf700cb8b0 msgr2=0x7fdf700ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.850+0000 7fdf777ff700 1 -- 172.21.15.33:0/1628426437 shutdown_connections 2023-11-06T22:00:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:22.850+0000 7fdf777ff700 1 -- 172.21.15.33:0/1628426437 wait complete. 2023-11-06T22:00:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:00:22.869 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T22:00:21.525332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2443916,"kb_used_data":17996,"kb_used_omap":0,"kb_used_meta":2425856,"kb_avail":752530804,"statfs":{"total":773094113280,"available":770591543296,"internally_reserved":0,"allocated":18427904,"data_stored":9387328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2484076544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11936,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-65,"num_write_kb":-34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.009263"},"pg_stats":[{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.105501+0000","last_change":"2023-11-06T22:00:18.101568+0000","last_active":"2023-11-06T22:00:18.105501+0000","last_peered":"2023-11-06T22:00:18.105501+0000","last_clean":"2023-11-06T22:00:18.105501+0000","last_became_active":"2023-11-06T22:00:18.100916+0000","last_became_peered":"2023-11-06T22:00:18.100916+0000","last_unstale":"2023-11-06T22:00:18.105501+0000","last_undegraded":"2023-11-06T22:00:18.105501+0000","last_fullsized":"2023-11-06T22:00:18.105501+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3f","version":"184'3","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.114829+0000","last_change":"2023-11-06T22:00:17.612436+0000","last_active":"2023-11-06T22:00:18.114829+0000","last_peered":"2023-11-06T22:00:18.114829+0000","last_clean":"2023-11-06T22:00:18.114829+0000","last_became_active":"2023-11-06T22:00:17.612113+0000","last_became_peered":"2023-11-06T22:00:17.612113+0000","last_unstale":"2023-11-06T22:00:18.114829+0000","last_undegraded":"2023-11-06T22:00:18.114829+0000","last_fullsized":"2023-11-06T22:00:18.114829+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.103436+0000","last_change":"2023-11-06T22:00:18.103436+0000","last_active":"2023-11-06T22:00:18.103436+0000","last_peered":"2023-11-06T22:00:18.103436+0000","last_clean":"2023-11-06T22:00:18.103436+0000","last_became_active":"2023-11-06T22:00:18.103072+0000","last_became_peered":"2023-11-06T22:00:18.103072+0000","last_unstale":"2023-11-06T22:00:18.103436+0000","last_undegraded":"2023-11-06T22:00:18.103436+0000","last_fullsized":"2023-11-06T22:00:18.103436+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094216+0000","last_change":"2023-11-06T22:00:17.604922+0000","last_active":"2023-11-06T22:00:18.094216+0000","last_peered":"2023-11-06T22:00:18.094216+0000","last_clean":"2023-11-06T22:00:18.094216+0000","last_became_active":"2023-11-06T22:00:17.604269+0000","last_became_peered":"2023-11-06T22:00:17.604269+0000","last_unstale":"2023-11-06T22:00:18.094216+0000","last_undegraded":"2023-11-06T22:00:18.094216+0000","last_fullsized":"2023-11-06T22:00:18.094216+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.109225+0000","last_change":"2023-11-06T22:00:18.104901+0000","last_active":"2023-11-06T22:00:18.109225+0000","last_peered":"2023-11-06T22:00:18.109225+0000","last_clean":"2023-11-06T22:00:18.109225+0000","last_became_active":"2023-11-06T22:00:18.104606+0000","last_became_peered":"2023-11-06T22:00:18.104606+0000","last_unstale":"2023-11-06T22:00:18.109225+0000","last_undegraded":"2023-11-06T22:00:18.109225+0000","last_fullsized":"2023-11-06T22:00:18.109225+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077200+0000","last_change":"2023-11-06T22:00:17.602782+0000","last_active":"2023-11-06T22:00:18.077200+0000","last_peered":"2023-11-06T22:00:18.077200+0000","last_clean":"2023-11-06T22:00:18.077200+0000","last_became_active":"2023-11-06T22:00:17.602492+0000","last_became_peered":"2023-11-06T22:00:17.602492+0000","last_unstale":"2023-11-06T22:00:18.077200+0000","last_undegraded":"2023-11-06T22:00:18.077200+0000","last_fullsized":"2023-11-06T22:00:18.077200+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.107260+0000","last_change":"2023-11-06T22:00:18.098304+0000","last_active":"2023-11-06T22:00:18.107260+0000","last_peered":"2023-11-06T22:00:18.107260+0000","last_clean":"2023-11-06T22:00:18.107260+0000","last_became_active":"2023-11-06T22:00:18.097993+0000","last_became_peered":"2023-11-06T22:00:18.097993+0000","last_unstale":"2023-11-06T22:00:18.107260+0000","last_undegraded":"2023-11-06T22:00:18.107260+0000","last_fullsized":"2023-11-06T22:00:18.107260+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"184'2","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.114158+0000","last_change":"2023-11-06T22:00:17.608170+0000","last_active":"2023-11-06T22:00:18.114158+0000","last_peered":"2023-11-06T22:00:18.114158+0000","last_clean":"2023-11-06T22:00:18.114158+0000","last_became_active":"2023-11-06T22:00:17.606510+0000","last_became_peered":"2023-11-06T22:00:17.606510+0000","last_unstale":"2023-11-06T22:00:18.114158+0000","last_undegraded":"2023-11-06T22:00:18.114158+0000","last_fullsized":"2023-11-06T22:00:18.114158+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.105343+0000","last_change":"2023-11-06T22:00:18.099107+0000","last_active":"2023-11-06T22:00:18.105343+0000","last_peered":"2023-11-06T22:00:18.105343+0000","last_clean":"2023-11-06T22:00:18.105343+0000","last_became_active":"2023-11-06T22:00:18.098655+0000","last_became_peered":"2023-11-06T22:00:18.098655+0000","last_unstale":"2023-11-06T22:00:18.105343+0000","last_undegraded":"2023-11-06T22:00:18.105343+0000","last_fullsized":"2023-11-06T22:00:18.105343+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3b","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.117863+0000","last_change":"2023-11-06T22:00:17.616386+0000","last_active":"2023-11-06T22:00:18.117863+0000","last_peered":"2023-11-06T22:00:18.117863+0000","last_clean":"2023-11-06T22:00:18.117863+0000","last_became_active":"2023-11-06T22:00:17.616040+0000","last_became_peered":"2023-11-06T22:00:17.616040+0000","last_unstale":"2023-11-06T22:00:18.117863+0000","last_undegraded":"2023-11-06T22:00:18.117863+0000","last_fullsized":"2023-11-06T22:00:18.117863+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.104984+0000","last_change":"2023-11-06T22:00:18.091646+0000","last_active":"2023-11-06T22:00:18.104984+0000","last_peered":"2023-11-06T22:00:18.104984+0000","last_clean":"2023-11-06T22:00:18.104984+0000","last_became_active":"2023-11-06T22:00:18.091385+0000","last_became_peered":"2023-11-06T22:00:18.091385+0000","last_unstale":"2023-11-06T22:00:18.104984+0000","last_undegraded":"2023-11-06T22:00:18.104984+0000","last_fullsized":"2023-11-06T22:00:18.104984+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3a","version":"184'1","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.085765+0000","last_change":"2023-11-06T22:00:17.609892+0000","last_active":"2023-11-06T22:00:18.085765+0000","last_peered":"2023-11-06T22:00:18.085765+0000","last_clean":"2023-11-06T22:00:18.085765+0000","last_became_active":"2023-11-06T22:00:17.609602+0000","last_became_peered":"2023-11-06T22:00:17.609602+0000","last_unstale":"2023-11-06T22:00:18.085765+0000","last_undegraded":"2023-11-06T22:00:18.085765+0000","last_fullsized":"2023-11-06T22:00:18.085765+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094163+0000","last_change":"2023-11-06T22:00:18.088979+0000","last_active":"2023-11-06T22:00:18.094163+0000","last_peered":"2023-11-06T22:00:18.094163+0000","last_clean":"2023-11-06T22:00:18.094163+0000","last_became_active":"2023-11-06T22:00:18.088700+0000","last_became_peered":"2023-11-06T22:00:18.088700+0000","last_unstale":"2023-11-06T22:00:18.094163+0000","last_undegraded":"2023-11-06T22:00:18.094163+0000","last_fullsized":"2023-11-06T22:00:18.094163+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.620463+0000","last_change":"2023-11-06T22:00:17.610304+0000","last_active":"2023-11-06T22:00:17.620463+0000","last_peered":"2023-11-06T22:00:17.620463+0000","last_clean":"2023-11-06T22:00:17.620463+0000","last_became_active":"2023-11-06T22:00:17.610036+0000","last_became_peered":"2023-11-06T22:00:17.610036+0000","last_unstale":"2023-11-06T22:00:17.620463+0000","last_undegraded":"2023-11-06T22:00:17.620463+0000","last_fullsized":"2023-11-06T22:00:17.620463+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111179+0000","last_change":"2023-11-06T22:00:18.107809+0000","last_active":"2023-11-06T22:00:18.111179+0000","last_peered":"2023-11-06T22:00:18.111179+0000","last_clean":"2023-11-06T22:00:18.111179+0000","last_became_active":"2023-11-06T22:00:18.105872+0000","last_became_peered":"2023-11-06T22:00:18.105872+0000","last_unstale":"2023-11-06T22:00:18.111179+0000","last_undegraded":"2023-11-06T22:00:18.111179+0000","last_fullsized":"2023-11-06T22:00:18.111179+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.085425+0000","last_change":"2023-11-06T22:00:17.608963+0000","last_active":"2023-11-06T22:00:18.085425+0000","last_peered":"2023-11-06T22:00:18.085425+0000","last_clean":"2023-11-06T22:00:18.085425+0000","last_became_active":"2023-11-06T22:00:17.608612+0000","last_became_peered":"2023-11-06T22:00:17.608612+0000","last_unstale":"2023-11-06T22:00:18.085425+0000","last_undegraded":"2023-11-06T22:00:18.085425+0000","last_fullsized":"2023-11-06T22:00:18.085425+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112318+0000","last_change":"2023-11-06T22:00:18.101936+0000","last_active":"2023-11-06T22:00:18.112318+0000","last_peered":"2023-11-06T22:00:18.112318+0000","last_clean":"2023-11-06T22:00:18.112318+0000","last_became_active":"2023-11-06T22:00:18.101642+0000","last_became_peered":"2023-11-06T22:00:18.101642+0000","last_unstale":"2023-11-06T22:00:18.112318+0000","last_undegraded":"2023-11-06T22:00:18.112318+0000","last_fullsized":"2023-11-06T22:00:18.112318+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.614629+0000","last_change":"2023-11-06T22:00:17.608609+0000","last_active":"2023-11-06T22:00:17.614629+0000","last_peered":"2023-11-06T22:00:17.614629+0000","last_clean":"2023-11-06T22:00:17.614629+0000","last_became_active":"2023-11-06T22:00:17.606730+0000","last_became_peered":"2023-11-06T22:00:17.606730+0000","last_unstale":"2023-11-06T22:00:17.614629+0000","last_undegraded":"2023-11-06T22:00:17.614629+0000","last_fullsized":"2023-11-06T22:00:17.614629+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.101863+0000","last_change":"2023-11-06T22:00:18.101863+0000","last_active":"2023-11-06T22:00:18.101863+0000","last_peered":"2023-11-06T22:00:18.101863+0000","last_clean":"2023-11-06T22:00:18.101863+0000","last_became_active":"2023-11-06T22:00:18.101548+0000","last_became_peered":"2023-11-06T22:00:18.101548+0000","last_unstale":"2023-11-06T22:00:18.101863+0000","last_undegraded":"2023-11-06T22:00:18.101863+0000","last_fullsized":"2023-11-06T22:00:18.101863+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077239+0000","last_change":"2023-11-06T22:00:17.614429+0000","last_active":"2023-11-06T22:00:18.077239+0000","last_peered":"2023-11-06T22:00:18.077239+0000","last_clean":"2023-11-06T22:00:18.077239+0000","last_became_active":"2023-11-06T22:00:17.614046+0000","last_became_peered":"2023-11-06T22:00:17.614046+0000","last_unstale":"2023-11-06T22:00:18.077239+0000","last_undegraded":"2023-11-06T22:00:18.077239+0000","last_fullsized":"2023-11-06T22:00:18.077239+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.098871+0000","last_change":"2023-11-06T22:00:18.098871+0000","last_active":"2023-11-06T22:00:18.098871+0000","last_peered":"2023-11-06T22:00:18.098871+0000","last_clean":"2023-11-06T22:00:18.098871+0000","last_became_active":"2023-11-06T22:00:18.098597+0000","last_became_peered":"2023-11-06T22:00:18.098597+0000","last_unstale":"2023-11-06T22:00:18.098871+0000","last_undegraded":"2023-11-06T22:00:18.098871+0000","last_fullsized":"2023-11-06T22:00:18.098871+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.614700+0000","last_change":"2023-11-06T22:00:17.610682+0000","last_active":"2023-11-06T22:00:17.614700+0000","last_peered":"2023-11-06T22:00:17.614700+0000","last_clean":"2023-11-06T22:00:17.614700+0000","last_became_active":"2023-11-06T22:00:17.610367+0000","last_became_peered":"2023-11-06T22:00:17.610367+0000","last_unstale":"2023-11-06T22:00:17.614700+0000","last_undegraded":"2023-11-06T22:00:17.614700+0000","last_fullsized":"2023-11-06T22:00:17.614700+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.109961+0000","last_change":"2023-11-06T22:00:18.104332+0000","last_active":"2023-11-06T22:00:18.109961+0000","last_peered":"2023-11-06T22:00:18.109961+0000","last_clean":"2023-11-06T22:00:18.109961+0000","last_became_active":"2023-11-06T22:00:18.104075+0000","last_became_peered":"2023-11-06T22:00:18.104075+0000","last_unstale":"2023-11-06T22:00:18.109961+0000","last_undegraded":"2023-11-06T22:00:18.109961+0000","last_fullsized":"2023-11-06T22:00:18.109961+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.34","version":"184'4","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.119521+0000","last_change":"2023-11-06T22:00:17.612964+0000","last_active":"2023-11-06T22:00:18.119521+0000","last_peered":"2023-11-06T22:00:18.119521+0000","last_clean":"2023-11-06T22:00:18.119521+0000","last_became_active":"2023-11-06T22:00:17.612561+0000","last_became_peered":"2023-11-06T22:00:17.612561+0000","last_unstale":"2023-11-06T22:00:18.119521+0000","last_undegraded":"2023-11-06T22:00:18.119521+0000","last_fullsized":"2023-11-06T22:00:18.119521+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.104894+0000","last_change":"2023-11-06T22:00:18.104894+0000","last_active":"2023-11-06T22:00:18.104894+0000","last_peered":"2023-11-06T22:00:18.104894+0000","last_clean":"2023-11-06T22:00:18.104894+0000","last_became_active":"2023-11-06T22:00:18.104013+0000","last_became_peered":"2023-11-06T22:00:18.104013+0000","last_unstale":"2023-11-06T22:00:18.104894+0000","last_undegraded":"2023-11-06T22:00:18.104894+0000","last_fullsized":"2023-11-06T22:00:18.104894+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.33","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.117849+0000","last_change":"2023-11-06T22:00:17.614760+0000","last_active":"2023-11-06T22:00:18.117849+0000","last_peered":"2023-11-06T22:00:18.117849+0000","last_clean":"2023-11-06T22:00:18.117849+0000","last_became_active":"2023-11-06T22:00:17.614272+0000","last_became_peered":"2023-11-06T22:00:17.614272+0000","last_unstale":"2023-11-06T22:00:18.117849+0000","last_undegraded":"2023-11-06T22:00:18.117849+0000","last_fullsized":"2023-11-06T22:00:18.117849+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.097913+0000","last_change":"2023-11-06T22:00:18.097913+0000","last_active":"2023-11-06T22:00:18.097913+0000","last_peered":"2023-11-06T22:00:18.097913+0000","last_clean":"2023-11-06T22:00:18.097913+0000","last_became_active":"2023-11-06T22:00:18.097578+0000","last_became_peered":"2023-11-06T22:00:18.097578+0000","last_unstale":"2023-11-06T22:00:18.097913+0000","last_undegraded":"2023-11-06T22:00:18.097913+0000","last_fullsized":"2023-11-06T22:00:18.097913+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094467+0000","last_change":"2023-11-06T22:00:17.617428+0000","last_active":"2023-11-06T22:00:18.094467+0000","last_peered":"2023-11-06T22:00:18.094467+0000","last_clean":"2023-11-06T22:00:18.094467+0000","last_became_active":"2023-11-06T22:00:17.616754+0000","last_became_peered":"2023-11-06T22:00:17.616754+0000","last_unstale":"2023-11-06T22:00:18.094467+0000","last_undegraded":"2023-11-06T22:00:18.094467+0000","last_fullsized":"2023-11-06T22:00:18.094467+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.100649+0000","last_change":"2023-11-06T22:00:18.100649+0000","last_active":"2023-11-06T22:00:18.100649+0000","last_peered":"2023-11-06T22:00:18.100649+0000","last_clean":"2023-11-06T22:00:18.100649+0000","last_became_active":"2023-11-06T22:00:18.100153+0000","last_became_peered":"2023-11-06T22:00:18.100153+0000","last_unstale":"2023-11-06T22:00:18.100649+0000","last_undegraded":"2023-11-06T22:00:18.100649+0000","last_fullsized":"2023-11-06T22:00:18.100649+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.083352+0000","last_change":"2023-11-06T22:00:17.613326+0000","last_active":"2023-11-06T22:00:18.083352+0000","last_peered":"2023-11-06T22:00:18.083352+0000","last_clean":"2023-11-06T22:00:18.083352+0000","last_became_active":"2023-11-06T22:00:17.613075+0000","last_became_peered":"2023-11-06T22:00:17.613075+0000","last_unstale":"2023-11-06T22:00:18.083352+0000","last_undegraded":"2023-11-06T22:00:18.083352+0000","last_fullsized":"2023-11-06T22:00:18.083352+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111197+0000","last_change":"2023-11-06T22:00:18.107834+0000","last_active":"2023-11-06T22:00:18.111197+0000","last_peered":"2023-11-06T22:00:18.111197+0000","last_clean":"2023-11-06T22:00:18.111197+0000","last_became_active":"2023-11-06T22:00:18.105105+0000","last_became_peered":"2023-11-06T22:00:18.105105+0000","last_unstale":"2023-11-06T22:00:18.111197+0000","last_undegraded":"2023-11-06T22:00:18.111197+0000","last_fullsized":"2023-11-06T22:00:18.111197+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"184'4","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.113122+0000","last_change":"2023-11-06T22:00:17.607230+0000","last_active":"2023-11-06T22:00:18.113122+0000","last_peered":"2023-11-06T22:00:18.113122+0000","last_clean":"2023-11-06T22:00:18.113122+0000","last_became_active":"2023-11-06T22:00:17.606957+0000","last_became_peered":"2023-11-06T22:00:17.606957+0000","last_unstale":"2023-11-06T22:00:18.113122+0000","last_undegraded":"2023-11-06T22:00:18.113122+0000","last_fullsized":"2023-11-06T22:00:18.113122+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.096800+0000","last_change":"2023-11-06T22:00:18.096800+0000","last_active":"2023-11-06T22:00:18.096800+0000","last_peered":"2023-11-06T22:00:18.096800+0000","last_clean":"2023-11-06T22:00:18.096800+0000","last_became_active":"2023-11-06T22:00:18.096444+0000","last_became_peered":"2023-11-06T22:00:18.096444+0000","last_unstale":"2023-11-06T22:00:18.096800+0000","last_undegraded":"2023-11-06T22:00:18.096800+0000","last_fullsized":"2023-11-06T22:00:18.096800+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077597+0000","last_change":"2023-11-06T22:00:17.612270+0000","last_active":"2023-11-06T22:00:18.077597+0000","last_peered":"2023-11-06T22:00:18.077597+0000","last_clean":"2023-11-06T22:00:18.077597+0000","last_became_active":"2023-11-06T22:00:17.611074+0000","last_became_peered":"2023-11-06T22:00:17.611074+0000","last_unstale":"2023-11-06T22:00:18.077597+0000","last_undegraded":"2023-11-06T22:00:18.077597+0000","last_fullsized":"2023-11-06T22:00:18.077597+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.090903+0000","last_change":"2023-11-06T22:00:18.084493+0000","last_active":"2023-11-06T22:00:18.090903+0000","last_peered":"2023-11-06T22:00:18.090903+0000","last_clean":"2023-11-06T22:00:18.090903+0000","last_became_active":"2023-11-06T22:00:18.084177+0000","last_became_peered":"2023-11-06T22:00:18.084177+0000","last_unstale":"2023-11-06T22:00:18.090903+0000","last_undegraded":"2023-11-06T22:00:18.090903+0000","last_fullsized":"2023-11-06T22:00:18.090903+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.083273+0000","last_change":"2023-11-06T22:00:17.615629+0000","last_active":"2023-11-06T22:00:18.083273+0000","last_peered":"2023-11-06T22:00:18.083273+0000","last_clean":"2023-11-06T22:00:18.083273+0000","last_became_active":"2023-11-06T22:00:17.614981+0000","last_became_peered":"2023-11-06T22:00:17.614981+0000","last_unstale":"2023-11-06T22:00:18.083273+0000","last_undegraded":"2023-11-06T22:00:18.083273+0000","last_fullsized":"2023-11-06T22:00:18.083273+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.114660+0000","last_change":"2023-11-06T22:00:18.114032+0000","last_active":"2023-11-06T22:00:18.114660+0000","last_peered":"2023-11-06T22:00:18.114660+0000","last_clean":"2023-11-06T22:00:18.114660+0000","last_became_active":"2023-11-06T22:00:18.113064+0000","last_became_peered":"2023-11-06T22:00:18.113064+0000","last_unstale":"2023-11-06T22:00:18.114660+0000","last_undegraded":"2023-11-06T22:00:18.114660+0000","last_fullsized":"2023-11-06T22:00:18.114660+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2d","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111195+0000","last_change":"2023-11-06T22:00:17.616312+0000","last_active":"2023-11-06T22:00:18.111195+0000","last_peered":"2023-11-06T22:00:18.111195+0000","last_clean":"2023-11-06T22:00:18.111195+0000","last_became_active":"2023-11-06T22:00:17.614951+0000","last_became_peered":"2023-11-06T22:00:17.614951+0000","last_unstale":"2023-11-06T22:00:18.111195+0000","last_undegraded":"2023-11-06T22:00:18.111195+0000","last_fullsized":"2023-11-06T22:00:18.111195+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.110063+0000","last_change":"2023-11-06T22:00:18.103381+0000","last_active":"2023-11-06T22:00:18.110063+0000","last_peered":"2023-11-06T22:00:18.110063+0000","last_clean":"2023-11-06T22:00:18.110063+0000","last_became_active":"2023-11-06T22:00:18.103105+0000","last_became_peered":"2023-11-06T22:00:18.103105+0000","last_unstale":"2023-11-06T22:00:18.110063+0000","last_undegraded":"2023-11-06T22:00:18.110063+0000","last_fullsized":"2023-11-06T22:00:18.110063+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.087791+0000","last_change":"2023-11-06T22:00:17.606658+0000","last_active":"2023-11-06T22:00:18.087791+0000","last_peered":"2023-11-06T22:00:18.087791+0000","last_clean":"2023-11-06T22:00:18.087791+0000","last_became_active":"2023-11-06T22:00:17.606247+0000","last_became_peered":"2023-11-06T22:00:17.606247+0000","last_unstale":"2023-11-06T22:00:18.087791+0000","last_undegraded":"2023-11-06T22:00:18.087791+0000","last_fullsized":"2023-11-06T22:00:18.087791+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.093250+0000","last_change":"2023-11-06T22:00:18.093250+0000","last_active":"2023-11-06T22:00:18.093250+0000","last_peered":"2023-11-06T22:00:18.093250+0000","last_clean":"2023-11-06T22:00:18.093250+0000","last_became_active":"2023-11-06T22:00:18.092901+0000","last_became_peered":"2023-11-06T22:00:18.092901+0000","last_unstale":"2023-11-06T22:00:18.093250+0000","last_undegraded":"2023-11-06T22:00:18.093250+0000","last_fullsized":"2023-11-06T22:00:18.093250+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2b","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112615+0000","last_change":"2023-11-06T22:00:17.616718+0000","last_active":"2023-11-06T22:00:18.112615+0000","last_peered":"2023-11-06T22:00:18.112615+0000","last_clean":"2023-11-06T22:00:18.112615+0000","last_became_active":"2023-11-06T22:00:17.616259+0000","last_became_peered":"2023-11-06T22:00:17.616259+0000","last_unstale":"2023-11-06T22:00:18.112615+0000","last_undegraded":"2023-11-06T22:00:18.112615+0000","last_fullsized":"2023-11-06T22:00:18.112615+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.104222+0000","last_change":"2023-11-06T22:00:18.098319+0000","last_active":"2023-11-06T22:00:18.104222+0000","last_peered":"2023-11-06T22:00:18.104222+0000","last_clean":"2023-11-06T22:00:18.104222+0000","last_became_active":"2023-11-06T22:00:18.093084+0000","last_became_peered":"2023-11-06T22:00:18.093084+0000","last_unstale":"2023-11-06T22:00:18.104222+0000","last_undegraded":"2023-11-06T22:00:18.104222+0000","last_fullsized":"2023-11-06T22:00:18.104222+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094566+0000","last_change":"2023-11-06T22:00:17.610350+0000","last_active":"2023-11-06T22:00:18.094566+0000","last_peered":"2023-11-06T22:00:18.094566+0000","last_clean":"2023-11-06T22:00:18.094566+0000","last_became_active":"2023-11-06T22:00:17.607925+0000","last_became_peered":"2023-11-06T22:00:17.607925+0000","last_unstale":"2023-11-06T22:00:18.094566+0000","last_undegraded":"2023-11-06T22:00:18.094566+0000","last_fullsized":"2023-11-06T22:00:18.094566+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.110612+0000","last_change":"2023-11-06T22:00:18.094031+0000","last_active":"2023-11-06T22:00:18.110612+0000","last_peered":"2023-11-06T22:00:18.110612+0000","last_clean":"2023-11-06T22:00:18.110612+0000","last_became_active":"2023-11-06T22:00:18.093775+0000","last_became_peered":"2023-11-06T22:00:18.093775+0000","last_unstale":"2023-11-06T22:00:18.110612+0000","last_undegraded":"2023-11-06T22:00:18.110612+0000","last_fullsized":"2023-11-06T22:00:18.110612+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.620592+0000","last_change":"2023-11-06T22:00:17.616295+0000","last_active":"2023-11-06T22:00:17.620592+0000","last_peered":"2023-11-06T22:00:17.620592+0000","last_clean":"2023-11-06T22:00:17.620592+0000","last_became_active":"2023-11-06T22:00:17.615366+0000","last_became_peered":"2023-11-06T22:00:17.615366+0000","last_unstale":"2023-11-06T22:00:17.620592+0000","last_undegraded":"2023-11-06T22:00:17.620592+0000","last_fullsized":"2023-11-06T22:00:17.620592+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.095957+0000","last_change":"2023-11-06T22:00:18.095957+0000","last_active":"2023-11-06T22:00:18.095957+0000","last_peered":"2023-11-06T22:00:18.095957+0000","last_clean":"2023-11-06T22:00:18.095957+0000","last_became_active":"2023-11-06T22:00:18.095652+0000","last_became_peered":"2023-11-06T22:00:18.095652+0000","last_unstale":"2023-11-06T22:00:18.095957+0000","last_undegraded":"2023-11-06T22:00:18.095957+0000","last_fullsized":"2023-11-06T22:00:18.095957+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.082747+0000","last_change":"2023-11-06T22:00:17.621460+0000","last_active":"2023-11-06T22:00:18.082747+0000","last_peered":"2023-11-06T22:00:18.082747+0000","last_clean":"2023-11-06T22:00:18.082747+0000","last_became_active":"2023-11-06T22:00:17.621121+0000","last_became_peered":"2023-11-06T22:00:17.621121+0000","last_unstale":"2023-11-06T22:00:18.082747+0000","last_undegraded":"2023-11-06T22:00:18.082747+0000","last_fullsized":"2023-11-06T22:00:18.082747+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111324+0000","last_change":"2023-11-06T22:00:18.108060+0000","last_active":"2023-11-06T22:00:18.111324+0000","last_peered":"2023-11-06T22:00:18.111324+0000","last_clean":"2023-11-06T22:00:18.111324+0000","last_became_active":"2023-11-06T22:00:18.107850+0000","last_became_peered":"2023-11-06T22:00:18.107850+0000","last_unstale":"2023-11-06T22:00:18.111324+0000","last_undegraded":"2023-11-06T22:00:18.111324+0000","last_fullsized":"2023-11-06T22:00:18.111324+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.090737+0000","last_change":"2023-11-06T22:00:17.614473+0000","last_active":"2023-11-06T22:00:18.090737+0000","last_peered":"2023-11-06T22:00:18.090737+0000","last_clean":"2023-11-06T22:00:18.090737+0000","last_became_active":"2023-11-06T22:00:17.611658+0000","last_became_peered":"2023-11-06T22:00:17.611658+0000","last_unstale":"2023-11-06T22:00:18.090737+0000","last_undegraded":"2023-11-06T22:00:18.090737+0000","last_fullsized":"2023-11-06T22:00:18.090737+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.097808+0000","last_change":"2023-11-06T22:00:18.097808+0000","last_active":"2023-11-06T22:00:18.097808+0000","last_peered":"2023-11-06T22:00:18.097808+0000","last_clean":"2023-11-06T22:00:18.097808+0000","last_became_active":"2023-11-06T22:00:18.097474+0000","last_became_peered":"2023-11-06T22:00:18.097474+0000","last_unstale":"2023-11-06T22:00:18.097808+0000","last_undegraded":"2023-11-06T22:00:18.097808+0000","last_fullsized":"2023-11-06T22:00:18.097808+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.089652+0000","last_change":"2023-11-06T22:00:17.606853+0000","last_active":"2023-11-06T22:00:18.089652+0000","last_peered":"2023-11-06T22:00:18.089652+0000","last_clean":"2023-11-06T22:00:18.089652+0000","last_became_active":"2023-11-06T22:00:17.606369+0000","last_became_peered":"2023-11-06T22:00:17.606369+0000","last_unstale":"2023-11-06T22:00:18.089652+0000","last_undegraded":"2023-11-06T22:00:18.089652+0000","last_fullsized":"2023-11-06T22:00:18.089652+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.107124+0000","last_change":"2023-11-06T22:00:18.107124+0000","last_active":"2023-11-06T22:00:18.107124+0000","last_peered":"2023-11-06T22:00:18.107124+0000","last_clean":"2023-11-06T22:00:18.107124+0000","last_became_active":"2023-11-06T22:00:18.106062+0000","last_became_peered":"2023-11-06T22:00:18.106062+0000","last_unstale":"2023-11-06T22:00:18.107124+0000","last_undegraded":"2023-11-06T22:00:18.107124+0000","last_fullsized":"2023-11-06T22:00:18.107124+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.091318+0000","last_change":"2023-11-06T22:00:17.608211+0000","last_active":"2023-11-06T22:00:18.091318+0000","last_peered":"2023-11-06T22:00:18.091318+0000","last_clean":"2023-11-06T22:00:18.091318+0000","last_became_active":"2023-11-06T22:00:17.607775+0000","last_became_peered":"2023-11-06T22:00:17.607775+0000","last_unstale":"2023-11-06T22:00:18.091318+0000","last_undegraded":"2023-11-06T22:00:18.091318+0000","last_fullsized":"2023-11-06T22:00:18.091318+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.091052+0000","last_change":"2023-11-06T22:00:18.089006+0000","last_active":"2023-11-06T22:00:18.091052+0000","last_peered":"2023-11-06T22:00:18.091052+0000","last_clean":"2023-11-06T22:00:18.091052+0000","last_became_active":"2023-11-06T22:00:18.088686+0000","last_became_peered":"2023-11-06T22:00:18.088686+0000","last_unstale":"2023-11-06T22:00:18.091052+0000","last_undegraded":"2023-11-06T22:00:18.091052+0000","last_fullsized":"2023-11-06T22:00:18.091052+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094462+0000","last_change":"2023-11-06T22:00:17.606162+0000","last_active":"2023-11-06T22:00:18.094462+0000","last_peered":"2023-11-06T22:00:18.094462+0000","last_clean":"2023-11-06T22:00:18.094462+0000","last_became_active":"2023-11-06T22:00:17.605889+0000","last_became_peered":"2023-11-06T22:00:17.605889+0000","last_unstale":"2023-11-06T22:00:18.094462+0000","last_undegraded":"2023-11-06T22:00:18.094462+0000","last_fullsized":"2023-11-06T22:00:18.094462+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.093133+0000","last_change":"2023-11-06T22:00:18.093133+0000","last_active":"2023-11-06T22:00:18.093133+0000","last_peered":"2023-11-06T22:00:18.093133+0000","last_clean":"2023-11-06T22:00:18.093133+0000","last_became_active":"2023-11-06T22:00:18.092867+0000","last_became_peered":"2023-11-06T22:00:18.092867+0000","last_unstale":"2023-11-06T22:00:18.093133+0000","last_undegraded":"2023-11-06T22:00:18.093133+0000","last_fullsized":"2023-11-06T22:00:18.093133+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.089200+0000","last_change":"2023-11-06T22:00:17.616087+0000","last_active":"2023-11-06T22:00:18.089200+0000","last_peered":"2023-11-06T22:00:18.089200+0000","last_clean":"2023-11-06T22:00:18.089200+0000","last_became_active":"2023-11-06T22:00:17.615015+0000","last_became_peered":"2023-11-06T22:00:17.615015+0000","last_unstale":"2023-11-06T22:00:18.089200+0000","last_undegraded":"2023-11-06T22:00:18.089200+0000","last_fullsized":"2023-11-06T22:00:18.089200+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.091163+0000","last_change":"2023-11-06T22:00:18.091163+0000","last_active":"2023-11-06T22:00:18.091163+0000","last_peered":"2023-11-06T22:00:18.091163+0000","last_clean":"2023-11-06T22:00:18.091163+0000","last_became_active":"2023-11-06T22:00:18.090878+0000","last_became_peered":"2023-11-06T22:00:18.090878+0000","last_unstale":"2023-11-06T22:00:18.091163+0000","last_undegraded":"2023-11-06T22:00:18.091163+0000","last_fullsized":"2023-11-06T22:00:18.091163+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094653+0000","last_change":"2023-11-06T22:00:17.611021+0000","last_active":"2023-11-06T22:00:18.094653+0000","last_peered":"2023-11-06T22:00:18.094653+0000","last_clean":"2023-11-06T22:00:18.094653+0000","last_became_active":"2023-11-06T22:00:17.607530+0000","last_became_peered":"2023-11-06T22:00:17.607530+0000","last_unstale":"2023-11-06T22:00:18.094653+0000","last_undegraded":"2023-11-06T22:00:18.094653+0000","last_fullsized":"2023-11-06T22:00:18.094653+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.097980+0000","last_change":"2023-11-06T22:00:18.097980+0000","last_active":"2023-11-06T22:00:18.097980+0000","last_peered":"2023-11-06T22:00:18.097980+0000","last_clean":"2023-11-06T22:00:18.097980+0000","last_became_active":"2023-11-06T22:00:18.097540+0000","last_became_peered":"2023-11-06T22:00:18.097540+0000","last_unstale":"2023-11-06T22:00:18.097980+0000","last_undegraded":"2023-11-06T22:00:18.097980+0000","last_fullsized":"2023-11-06T22:00:18.097980+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.091932+0000","last_change":"2023-11-06T22:00:17.614519+0000","last_active":"2023-11-06T22:00:18.091932+0000","last_peered":"2023-11-06T22:00:18.091932+0000","last_clean":"2023-11-06T22:00:18.091932+0000","last_became_active":"2023-11-06T22:00:17.613523+0000","last_became_peered":"2023-11-06T22:00:17.613523+0000","last_unstale":"2023-11-06T22:00:18.091932+0000","last_undegraded":"2023-11-06T22:00:18.091932+0000","last_fullsized":"2023-11-06T22:00:18.091932+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.103162+0000","last_change":"2023-11-06T22:00:18.094492+0000","last_active":"2023-11-06T22:00:18.103162+0000","last_peered":"2023-11-06T22:00:18.103162+0000","last_clean":"2023-11-06T22:00:18.103162+0000","last_became_active":"2023-11-06T22:00:18.093830+0000","last_became_peered":"2023-11-06T22:00:18.093830+0000","last_unstale":"2023-11-06T22:00:18.103162+0000","last_undegraded":"2023-11-06T22:00:18.103162+0000","last_fullsized":"2023-11-06T22:00:18.103162+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.090205+0000","last_change":"2023-11-06T22:00:17.617058+0000","last_active":"2023-11-06T22:00:18.090205+0000","last_peered":"2023-11-06T22:00:18.090205+0000","last_clean":"2023-11-06T22:00:18.090205+0000","last_became_active":"2023-11-06T22:00:17.616702+0000","last_became_peered":"2023-11-06T22:00:17.616702+0000","last_unstale":"2023-11-06T22:00:18.090205+0000","last_undegraded":"2023-11-06T22:00:18.090205+0000","last_fullsized":"2023-11-06T22:00:18.090205+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.102299+0000","last_change":"2023-11-06T22:00:18.102299+0000","last_active":"2023-11-06T22:00:18.102299+0000","last_peered":"2023-11-06T22:00:18.102299+0000","last_clean":"2023-11-06T22:00:18.102299+0000","last_became_active":"2023-11-06T22:00:18.102010+0000","last_became_peered":"2023-11-06T22:00:18.102010+0000","last_unstale":"2023-11-06T22:00:18.102299+0000","last_undegraded":"2023-11-06T22:00:18.102299+0000","last_fullsized":"2023-11-06T22:00:18.102299+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1f","version":"184'1","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.075109+0000","last_change":"2023-11-06T22:00:17.609196+0000","last_active":"2023-11-06T22:00:18.075109+0000","last_peered":"2023-11-06T22:00:18.075109+0000","last_clean":"2023-11-06T22:00:18.075109+0000","last_became_active":"2023-11-06T22:00:17.606206+0000","last_became_peered":"2023-11-06T22:00:17.606206+0000","last_unstale":"2023-11-06T22:00:18.075109+0000","last_undegraded":"2023-11-06T22:00:18.075109+0000","last_fullsized":"2023-11-06T22:00:18.075109+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.109130+0000","last_change":"2023-11-06T22:00:18.104491+0000","last_active":"2023-11-06T22:00:18.109130+0000","last_peered":"2023-11-06T22:00:18.109130+0000","last_clean":"2023-11-06T22:00:18.109130+0000","last_became_active":"2023-11-06T22:00:18.103932+0000","last_became_peered":"2023-11-06T22:00:18.103932+0000","last_unstale":"2023-11-06T22:00:18.109130+0000","last_undegraded":"2023-11-06T22:00:18.109130+0000","last_fullsized":"2023-11-06T22:00:18.109130+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.117722+0000","last_change":"2023-11-06T22:00:17.615812+0000","last_active":"2023-11-06T22:00:18.117722+0000","last_peered":"2023-11-06T22:00:18.117722+0000","last_clean":"2023-11-06T22:00:18.117722+0000","last_became_active":"2023-11-06T22:00:17.615330+0000","last_became_peered":"2023-11-06T22:00:17.615330+0000","last_unstale":"2023-11-06T22:00:18.117722+0000","last_undegraded":"2023-11-06T22:00:18.117722+0000","last_fullsized":"2023-11-06T22:00:18.117722+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.110694+0000","last_change":"2023-11-06T22:00:18.106332+0000","last_active":"2023-11-06T22:00:18.110694+0000","last_peered":"2023-11-06T22:00:18.110694+0000","last_clean":"2023-11-06T22:00:18.110694+0000","last_became_active":"2023-11-06T22:00:18.105651+0000","last_became_peered":"2023-11-06T22:00:18.105651+0000","last_unstale":"2023-11-06T22:00:18.110694+0000","last_undegraded":"2023-11-06T22:00:18.110694+0000","last_fullsized":"2023-11-06T22:00:18.110694+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1d","version":"184'2","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.114242+0000","last_change":"2023-11-06T22:00:17.607641+0000","last_active":"2023-11-06T22:00:18.114242+0000","last_peered":"2023-11-06T22:00:18.114242+0000","last_clean":"2023-11-06T22:00:18.114242+0000","last_became_active":"2023-11-06T22:00:17.607366+0000","last_became_peered":"2023-11-06T22:00:17.607366+0000","last_unstale":"2023-11-06T22:00:18.114242+0000","last_undegraded":"2023-11-06T22:00:18.114242+0000","last_fullsized":"2023-11-06T22:00:18.114242+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.096494+0000","last_change":"2023-11-06T22:00:18.096494+0000","last_active":"2023-11-06T22:00:18.096494+0000","last_peered":"2023-11-06T22:00:18.096494+0000","last_clean":"2023-11-06T22:00:18.096494+0000","last_became_active":"2023-11-06T22:00:18.096179+0000","last_became_peered":"2023-11-06T22:00:18.096179+0000","last_unstale":"2023-11-06T22:00:18.096494+0000","last_undegraded":"2023-11-06T22:00:18.096494+0000","last_fullsized":"2023-11-06T22:00:18.096494+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.625803+0000","last_change":"2023-11-06T22:00:17.621422+0000","last_active":"2023-11-06T22:00:17.625803+0000","last_peered":"2023-11-06T22:00:17.625803+0000","last_clean":"2023-11-06T22:00:17.625803+0000","last_became_active":"2023-11-06T22:00:17.621149+0000","last_became_peered":"2023-11-06T22:00:17.621149+0000","last_unstale":"2023-11-06T22:00:17.625803+0000","last_undegraded":"2023-11-06T22:00:17.625803+0000","last_fullsized":"2023-11-06T22:00:17.625803+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.105141+0000","last_change":"2023-11-06T22:00:18.105141+0000","last_active":"2023-11-06T22:00:18.105141+0000","last_peered":"2023-11-06T22:00:18.105141+0000","last_clean":"2023-11-06T22:00:18.105141+0000","last_became_active":"2023-11-06T22:00:18.104545+0000","last_became_peered":"2023-11-06T22:00:18.104545+0000","last_unstale":"2023-11-06T22:00:18.105141+0000","last_undegraded":"2023-11-06T22:00:18.105141+0000","last_fullsized":"2023-11-06T22:00:18.105141+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.620240+0000","last_change":"2023-11-06T22:00:17.613641+0000","last_active":"2023-11-06T22:00:17.620240+0000","last_peered":"2023-11-06T22:00:17.620240+0000","last_clean":"2023-11-06T22:00:17.620240+0000","last_became_active":"2023-11-06T22:00:17.612511+0000","last_became_peered":"2023-11-06T22:00:17.612511+0000","last_unstale":"2023-11-06T22:00:17.620240+0000","last_undegraded":"2023-11-06T22:00:17.620240+0000","last_fullsized":"2023-11-06T22:00:17.620240+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094256+0000","last_change":"2023-11-06T22:00:18.094256+0000","last_active":"2023-11-06T22:00:18.094256+0000","last_peered":"2023-11-06T22:00:18.094256+0000","last_clean":"2023-11-06T22:00:18.094256+0000","last_became_active":"2023-11-06T22:00:18.093983+0000","last_became_peered":"2023-11-06T22:00:18.093983+0000","last_unstale":"2023-11-06T22:00:18.094256+0000","last_undegraded":"2023-11-06T22:00:18.094256+0000","last_fullsized":"2023-11-06T22:00:18.094256+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.092770+0000","last_change":"2023-11-06T22:00:17.612695+0000","last_active":"2023-11-06T22:00:18.092770+0000","last_peered":"2023-11-06T22:00:18.092770+0000","last_clean":"2023-11-06T22:00:18.092770+0000","last_became_active":"2023-11-06T22:00:17.612080+0000","last_became_peered":"2023-11-06T22:00:17.612080+0000","last_unstale":"2023-11-06T22:00:18.092770+0000","last_undegraded":"2023-11-06T22:00:18.092770+0000","last_fullsized":"2023-11-06T22:00:18.092770+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.097937+0000","last_change":"2023-11-06T22:00:18.093851+0000","last_active":"2023-11-06T22:00:18.097937+0000","last_peered":"2023-11-06T22:00:18.097937+0000","last_clean":"2023-11-06T22:00:18.097937+0000","last_became_active":"2023-11-06T22:00:18.093064+0000","last_became_peered":"2023-11-06T22:00:18.093064+0000","last_unstale":"2023-11-06T22:00:18.097937+0000","last_undegraded":"2023-11-06T22:00:18.097937+0000","last_fullsized":"2023-11-06T22:00:18.097937+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.616729+0000","last_change":"2023-11-06T22:00:17.609329+0000","last_active":"2023-11-06T22:00:17.616729+0000","last_peered":"2023-11-06T22:00:17.616729+0000","last_clean":"2023-11-06T22:00:17.616729+0000","last_became_active":"2023-11-06T22:00:17.608768+0000","last_became_peered":"2023-11-06T22:00:17.608768+0000","last_unstale":"2023-11-06T22:00:17.616729+0000","last_undegraded":"2023-11-06T22:00:17.616729+0000","last_fullsized":"2023-11-06T22:00:17.616729+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.096257+0000","last_change":"2023-11-06T22:00:18.096257+0000","last_active":"2023-11-06T22:00:18.096257+0000","last_peered":"2023-11-06T22:00:18.096257+0000","last_clean":"2023-11-06T22:00:18.096257+0000","last_became_active":"2023-11-06T22:00:18.095899+0000","last_became_peered":"2023-11-06T22:00:18.095899+0000","last_unstale":"2023-11-06T22:00:18.096257+0000","last_undegraded":"2023-11-06T22:00:18.096257+0000","last_fullsized":"2023-11-06T22:00:18.096257+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.091705+0000","last_change":"2023-11-06T22:00:17.621160+0000","last_active":"2023-11-06T22:00:18.091705+0000","last_peered":"2023-11-06T22:00:18.091705+0000","last_clean":"2023-11-06T22:00:18.091705+0000","last_became_active":"2023-11-06T22:00:17.620808+0000","last_became_peered":"2023-11-06T22:00:17.620808+0000","last_unstale":"2023-11-06T22:00:18.091705+0000","last_undegraded":"2023-11-06T22:00:18.091705+0000","last_fullsized":"2023-11-06T22:00:18.091705+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.095948+0000","last_change":"2023-11-06T22:00:18.089020+0000","last_active":"2023-11-06T22:00:18.095948+0000","last_peered":"2023-11-06T22:00:18.095948+0000","last_clean":"2023-11-06T22:00:18.095948+0000","last_became_active":"2023-11-06T22:00:18.088747+0000","last_became_peered":"2023-11-06T22:00:18.088747+0000","last_unstale":"2023-11-06T22:00:18.095948+0000","last_undegraded":"2023-11-06T22:00:18.095948+0000","last_fullsized":"2023-11-06T22:00:18.095948+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.f","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112809+0000","last_change":"2023-11-06T22:00:17.615798+0000","last_active":"2023-11-06T22:00:18.112809+0000","last_peered":"2023-11-06T22:00:18.112809+0000","last_clean":"2023-11-06T22:00:18.112809+0000","last_became_active":"2023-11-06T22:00:17.613829+0000","last_became_peered":"2023-11-06T22:00:17.613829+0000","last_unstale":"2023-11-06T22:00:18.112809+0000","last_undegraded":"2023-11-06T22:00:18.112809+0000","last_fullsized":"2023-11-06T22:00:18.112809+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.103273+0000","last_change":"2023-11-06T22:00:18.097845+0000","last_active":"2023-11-06T22:00:18.103273+0000","last_peered":"2023-11-06T22:00:18.103273+0000","last_clean":"2023-11-06T22:00:18.103273+0000","last_became_active":"2023-11-06T22:00:18.097610+0000","last_became_peered":"2023-11-06T22:00:18.097610+0000","last_unstale":"2023-11-06T22:00:18.103273+0000","last_undegraded":"2023-11-06T22:00:18.103273+0000","last_fullsized":"2023-11-06T22:00:18.103273+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.e","version":"184'2","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111281+0000","last_change":"2023-11-06T22:00:17.622266+0000","last_active":"2023-11-06T22:00:18.111281+0000","last_peered":"2023-11-06T22:00:18.111281+0000","last_clean":"2023-11-06T22:00:18.111281+0000","last_became_active":"2023-11-06T22:00:17.621950+0000","last_became_peered":"2023-11-06T22:00:17.621950+0000","last_unstale":"2023-11-06T22:00:18.111281+0000","last_undegraded":"2023-11-06T22:00:18.111281+0000","last_fullsized":"2023-11-06T22:00:18.111281+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.107611+0000","last_change":"2023-11-06T22:00:18.100393+0000","last_active":"2023-11-06T22:00:18.107611+0000","last_peered":"2023-11-06T22:00:18.107611+0000","last_clean":"2023-11-06T22:00:18.107611+0000","last_became_active":"2023-11-06T22:00:18.100096+0000","last_became_peered":"2023-11-06T22:00:18.100096+0000","last_unstale":"2023-11-06T22:00:18.107611+0000","last_undegraded":"2023-11-06T22:00:18.107611+0000","last_fullsized":"2023-11-06T22:00:18.107611+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.625820+0000","last_change":"2023-11-06T22:00:17.622030+0000","last_active":"2023-11-06T22:00:17.625820+0000","last_peered":"2023-11-06T22:00:17.625820+0000","last_clean":"2023-11-06T22:00:17.625820+0000","last_became_active":"2023-11-06T22:00:17.621721+0000","last_became_peered":"2023-11-06T22:00:17.621721+0000","last_unstale":"2023-11-06T22:00:17.625820+0000","last_undegraded":"2023-11-06T22:00:17.625820+0000","last_fullsized":"2023-11-06T22:00:17.625820+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.104253+0000","last_change":"2023-11-06T22:00:18.099012+0000","last_active":"2023-11-06T22:00:18.104253+0000","last_peered":"2023-11-06T22:00:18.104253+0000","last_clean":"2023-11-06T22:00:18.104253+0000","last_became_active":"2023-11-06T22:00:18.098329+0000","last_became_peered":"2023-11-06T22:00:18.098329+0000","last_unstale":"2023-11-06T22:00:18.104253+0000","last_undegraded":"2023-11-06T22:00:18.104253+0000","last_fullsized":"2023-11-06T22:00:18.104253+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.083283+0000","last_change":"2023-11-06T22:00:17.622289+0000","last_active":"2023-11-06T22:00:18.083283+0000","last_peered":"2023-11-06T22:00:18.083283+0000","last_clean":"2023-11-06T22:00:18.083283+0000","last_became_active":"2023-11-06T22:00:17.621995+0000","last_became_peered":"2023-11-06T22:00:17.621995+0000","last_unstale":"2023-11-06T22:00:18.083283+0000","last_undegraded":"2023-11-06T22:00:18.083283+0000","last_fullsized":"2023-11-06T22:00:18.083283+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.106650+0000","last_change":"2023-11-06T22:00:18.101583+0000","last_active":"2023-11-06T22:00:18.106650+0000","last_peered":"2023-11-06T22:00:18.106650+0000","last_clean":"2023-11-06T22:00:18.106650+0000","last_became_active":"2023-11-06T22:00:18.101269+0000","last_became_peered":"2023-11-06T22:00:18.101269+0000","last_unstale":"2023-11-06T22:00:18.106650+0000","last_undegraded":"2023-11-06T22:00:18.106650+0000","last_fullsized":"2023-11-06T22:00:18.106650+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.b","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112454+0000","last_change":"2023-11-06T22:00:17.614210+0000","last_active":"2023-11-06T22:00:18.112454+0000","last_peered":"2023-11-06T22:00:18.112454+0000","last_clean":"2023-11-06T22:00:18.112454+0000","last_became_active":"2023-11-06T22:00:17.612970+0000","last_became_peered":"2023-11-06T22:00:17.612970+0000","last_unstale":"2023-11-06T22:00:18.112454+0000","last_undegraded":"2023-11-06T22:00:18.112454+0000","last_fullsized":"2023-11-06T22:00:18.112454+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.099289+0000","last_change":"2023-11-06T22:00:18.096595+0000","last_active":"2023-11-06T22:00:18.099289+0000","last_peered":"2023-11-06T22:00:18.099289+0000","last_clean":"2023-11-06T22:00:18.099289+0000","last_became_active":"2023-11-06T22:00:18.096322+0000","last_became_peered":"2023-11-06T22:00:18.096322+0000","last_unstale":"2023-11-06T22:00:18.099289+0000","last_undegraded":"2023-11-06T22:00:18.099289+0000","last_fullsized":"2023-11-06T22:00:18.099289+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094772+0000","last_change":"2023-11-06T22:00:17.605537+0000","last_active":"2023-11-06T22:00:18.094772+0000","last_peered":"2023-11-06T22:00:18.094772+0000","last_clean":"2023-11-06T22:00:18.094772+0000","last_became_active":"2023-11-06T22:00:17.605222+0000","last_became_peered":"2023-11-06T22:00:17.605222+0000","last_unstale":"2023-11-06T22:00:18.094772+0000","last_undegraded":"2023-11-06T22:00:18.094772+0000","last_fullsized":"2023-11-06T22:00:18.094772+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.110632+0000","last_change":"2023-11-06T22:00:18.099860+0000","last_active":"2023-11-06T22:00:18.110632+0000","last_peered":"2023-11-06T22:00:18.110632+0000","last_clean":"2023-11-06T22:00:18.110632+0000","last_became_active":"2023-11-06T22:00:18.099597+0000","last_became_peered":"2023-11-06T22:00:18.099597+0000","last_unstale":"2023-11-06T22:00:18.110632+0000","last_undegraded":"2023-11-06T22:00:18.110632+0000","last_fullsized":"2023-11-06T22:00:18.110632+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.092160+0000","last_change":"2023-11-06T22:00:17.614059+0000","last_active":"2023-11-06T22:00:18.092160+0000","last_peered":"2023-11-06T22:00:18.092160+0000","last_clean":"2023-11-06T22:00:18.092160+0000","last_became_active":"2023-11-06T22:00:17.613718+0000","last_became_peered":"2023-11-06T22:00:17.613718+0000","last_unstale":"2023-11-06T22:00:18.092160+0000","last_undegraded":"2023-11-06T22:00:18.092160+0000","last_fullsized":"2023-11-06T22:00:18.092160+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.081889+0000","last_change":"2023-11-06T22:00:18.081889+0000","last_active":"2023-11-06T22:00:18.081889+0000","last_peered":"2023-11-06T22:00:18.081889+0000","last_clean":"2023-11-06T22:00:18.081889+0000","last_became_active":"2023-11-06T22:00:18.081313+0000","last_became_peered":"2023-11-06T22:00:18.081313+0000","last_unstale":"2023-11-06T22:00:18.081889+0000","last_undegraded":"2023-11-06T22:00:18.081889+0000","last_fullsized":"2023-11-06T22:00:18.081889+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.093276+0000","last_change":"2023-11-06T22:00:17.615319+0000","last_active":"2023-11-06T22:00:18.093276+0000","last_peered":"2023-11-06T22:00:18.093276+0000","last_clean":"2023-11-06T22:00:18.093276+0000","last_became_active":"2023-11-06T22:00:17.615055+0000","last_became_peered":"2023-11-06T22:00:17.615055+0000","last_unstale":"2023-11-06T22:00:18.093276+0000","last_undegraded":"2023-11-06T22:00:18.093276+0000","last_fullsized":"2023-11-06T22:00:18.093276+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.081880+0000","last_change":"2023-11-06T22:00:18.081880+0000","last_active":"2023-11-06T22:00:18.081880+0000","last_peered":"2023-11-06T22:00:18.081880+0000","last_clean":"2023-11-06T22:00:18.081880+0000","last_became_active":"2023-11-06T22:00:18.081323+0000","last_became_peered":"2023-11-06T22:00:18.081323+0000","last_unstale":"2023-11-06T22:00:18.081880+0000","last_undegraded":"2023-11-06T22:00:18.081880+0000","last_fullsized":"2023-11-06T22:00:18.081880+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.7","version":"184'2","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.117829+0000","last_change":"2023-11-06T22:00:17.610204+0000","last_active":"2023-11-06T22:00:18.117829+0000","last_peered":"2023-11-06T22:00:18.117829+0000","last_clean":"2023-11-06T22:00:18.117829+0000","last_became_active":"2023-11-06T22:00:17.609777+0000","last_became_peered":"2023-11-06T22:00:17.609777+0000","last_unstale":"2023-11-06T22:00:18.117829+0000","last_undegraded":"2023-11-06T22:00:18.117829+0000","last_fullsized":"2023-11-06T22:00:18.117829+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.106792+0000","last_change":"2023-11-06T22:00:18.106792+0000","last_active":"2023-11-06T22:00:18.106792+0000","last_peered":"2023-11-06T22:00:18.106792+0000","last_clean":"2023-11-06T22:00:18.106792+0000","last_became_active":"2023-11-06T22:00:18.106457+0000","last_became_peered":"2023-11-06T22:00:18.106457+0000","last_unstale":"2023-11-06T22:00:18.106792+0000","last_undegraded":"2023-11-06T22:00:18.106792+0000","last_fullsized":"2023-11-06T22:00:18.106792+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.614124+0000","last_change":"2023-11-06T22:00:17.608612+0000","last_active":"2023-11-06T22:00:17.614124+0000","last_peered":"2023-11-06T22:00:17.614124+0000","last_clean":"2023-11-06T22:00:17.614124+0000","last_became_active":"2023-11-06T22:00:17.608190+0000","last_became_peered":"2023-11-06T22:00:17.608190+0000","last_unstale":"2023-11-06T22:00:17.614124+0000","last_undegraded":"2023-11-06T22:00:17.614124+0000","last_fullsized":"2023-11-06T22:00:17.614124+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.108226+0000","last_change":"2023-11-06T22:00:18.095118+0000","last_active":"2023-11-06T22:00:18.108226+0000","last_peered":"2023-11-06T22:00:18.108226+0000","last_clean":"2023-11-06T22:00:18.108226+0000","last_became_active":"2023-11-06T22:00:18.094856+0000","last_became_peered":"2023-11-06T22:00:18.094856+0000","last_unstale":"2023-11-06T22:00:18.108226+0000","last_undegraded":"2023-11-06T22:00:18.108226+0000","last_fullsized":"2023-11-06T22:00:18.108226+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.087144+0000","last_change":"2023-11-06T22:00:17.609707+0000","last_active":"2023-11-06T22:00:18.087144+0000","last_peered":"2023-11-06T22:00:18.087144+0000","last_clean":"2023-11-06T22:00:18.087144+0000","last_became_active":"2023-11-06T22:00:17.609400+0000","last_became_peered":"2023-11-06T22:00:17.609400+0000","last_unstale":"2023-11-06T22:00:18.087144+0000","last_undegraded":"2023-11-06T22:00:18.087144+0000","last_fullsized":"2023-11-06T22:00:18.087144+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.109937+0000","last_change":"2023-11-06T22:00:18.103751+0000","last_active":"2023-11-06T22:00:18.109937+0000","last_peered":"2023-11-06T22:00:18.109937+0000","last_clean":"2023-11-06T22:00:18.109937+0000","last_became_active":"2023-11-06T22:00:18.098912+0000","last_became_peered":"2023-11-06T22:00:18.098912+0000","last_unstale":"2023-11-06T22:00:18.109937+0000","last_undegraded":"2023-11-06T22:00:18.109937+0000","last_fullsized":"2023-11-06T22:00:18.109937+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.087889+0000","last_change":"2023-11-06T22:00:17.606496+0000","last_active":"2023-11-06T22:00:18.087889+0000","last_peered":"2023-11-06T22:00:18.087889+0000","last_clean":"2023-11-06T22:00:18.087889+0000","last_became_active":"2023-11-06T22:00:17.605847+0000","last_became_peered":"2023-11-06T22:00:17.605847+0000","last_unstale":"2023-11-06T22:00:18.087889+0000","last_undegraded":"2023-11-06T22:00:18.087889+0000","last_fullsized":"2023-11-06T22:00:18.087889+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.092946+0000","last_change":"2023-11-06T22:00:18.092946+0000","last_active":"2023-11-06T22:00:18.092946+0000","last_peered":"2023-11-06T22:00:18.092946+0000","last_clean":"2023-11-06T22:00:18.092946+0000","last_became_active":"2023-11-06T22:00:18.092704+0000","last_became_peered":"2023-11-06T22:00:18.092704+0000","last_unstale":"2023-11-06T22:00:18.092946+0000","last_undegraded":"2023-11-06T22:00:18.092946+0000","last_fullsized":"2023-11-06T22:00:18.092946+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.620214+0000","last_change":"2023-11-06T22:00:17.612831+0000","last_active":"2023-11-06T22:00:17.620214+0000","last_peered":"2023-11-06T22:00:17.620214+0000","last_clean":"2023-11-06T22:00:17.620214+0000","last_became_active":"2023-11-06T22:00:17.612448+0000","last_became_peered":"2023-11-06T22:00:17.612448+0000","last_unstale":"2023-11-06T22:00:17.620214+0000","last_undegraded":"2023-11-06T22:00:17.620214+0000","last_fullsized":"2023-11-06T22:00:17.620214+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.107228+0000","last_change":"2023-11-06T22:00:18.107228+0000","last_active":"2023-11-06T22:00:18.107228+0000","last_peered":"2023-11-06T22:00:18.107228+0000","last_clean":"2023-11-06T22:00:18.107228+0000","last_became_active":"2023-11-06T22:00:18.106967+0000","last_became_peered":"2023-11-06T22:00:18.106967+0000","last_unstale":"2023-11-06T22:00:18.107228+0000","last_undegraded":"2023-11-06T22:00:18.107228+0000","last_fullsized":"2023-11-06T22:00:18.107228+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.608853+0000","last_change":"2023-11-06T22:00:17.603418+0000","last_active":"2023-11-06T22:00:17.608853+0000","last_peered":"2023-11-06T22:00:17.608853+0000","last_clean":"2023-11-06T22:00:17.608853+0000","last_became_active":"2023-11-06T22:00:17.603124+0000","last_became_peered":"2023-11-06T22:00:17.603124+0000","last_unstale":"2023-11-06T22:00:17.608853+0000","last_undegraded":"2023-11-06T22:00:17.608853+0000","last_fullsized":"2023-11-06T22:00:17.608853+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.102966+0000","last_change":"2023-11-06T22:00:18.102966+0000","last_active":"2023-11-06T22:00:18.102966+0000","last_peered":"2023-11-06T22:00:18.102966+0000","last_clean":"2023-11-06T22:00:18.102966+0000","last_became_active":"2023-11-06T22:00:18.102611+0000","last_became_peered":"2023-11-06T22:00:18.102611+0000","last_unstale":"2023-11-06T22:00:18.102966+0000","last_undegraded":"2023-11-06T22:00:18.102966+0000","last_fullsized":"2023-11-06T22:00:18.102966+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.076583+0000","last_change":"2023-11-06T22:00:17.604459+0000","last_active":"2023-11-06T22:00:18.076583+0000","last_peered":"2023-11-06T22:00:18.076583+0000","last_clean":"2023-11-06T22:00:18.076583+0000","last_became_active":"2023-11-06T22:00:17.604112+0000","last_became_peered":"2023-11-06T22:00:17.604112+0000","last_unstale":"2023-11-06T22:00:18.076583+0000","last_undegraded":"2023-11-06T22:00:18.076583+0000","last_fullsized":"2023-11-06T22:00:18.076583+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.103190+0000","last_change":"2023-11-06T22:00:18.094531+0000","last_active":"2023-11-06T22:00:18.103190+0000","last_peered":"2023-11-06T22:00:18.103190+0000","last_clean":"2023-11-06T22:00:18.103190+0000","last_became_active":"2023-11-06T22:00:18.094240+0000","last_became_peered":"2023-11-06T22:00:18.094240+0000","last_unstale":"2023-11-06T22:00:18.103190+0000","last_undegraded":"2023-11-06T22:00:18.103190+0000","last_fullsized":"2023-11-06T22:00:18.103190+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.610512+0000","last_change":"2023-11-06T22:00:17.604567+0000","last_active":"2023-11-06T22:00:17.610512+0000","last_peered":"2023-11-06T22:00:17.610512+0000","last_clean":"2023-11-06T22:00:17.610512+0000","last_became_active":"2023-11-06T22:00:17.604261+0000","last_became_peered":"2023-11-06T22:00:17.604261+0000","last_unstale":"2023-11-06T22:00:17.610512+0000","last_undegraded":"2023-11-06T22:00:17.610512+0000","last_fullsized":"2023-11-06T22:00:17.610512+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.093135+0000","last_change":"2023-11-06T22:00:18.089020+0000","last_active":"2023-11-06T22:00:18.093135+0000","last_peered":"2023-11-06T22:00:18.093135+0000","last_clean":"2023-11-06T22:00:18.093135+0000","last_became_active":"2023-11-06T22:00:18.088718+0000","last_became_peered":"2023-11-06T22:00:18.088718+0000","last_unstale":"2023-11-06T22:00:18.093135+0000","last_undegraded":"2023-11-06T22:00:18.093135+0000","last_fullsized":"2023-11-06T22:00:18.093135+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111451+0000","last_change":"2023-11-06T22:00:17.603597+0000","last_active":"2023-11-06T22:00:18.111451+0000","last_peered":"2023-11-06T22:00:18.111451+0000","last_clean":"2023-11-06T22:00:18.111451+0000","last_became_active":"2023-11-06T22:00:17.603245+0000","last_became_peered":"2023-11-06T22:00:17.603245+0000","last_unstale":"2023-11-06T22:00:18.111451+0000","last_undegraded":"2023-11-06T22:00:18.111451+0000","last_fullsized":"2023-11-06T22:00:18.111451+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.591800+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T22:00:17.591800+0000","last_peered":"2023-11-06T22:00:17.591800+0000","last_clean":"2023-11-06T22:00:17.591800+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:00:17.591800+0000","last_undegraded":"2023-11-06T22:00:17.591800+0000","last_fullsized":"2023-11-06T22:00:17.591800+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.110595+0000","last_change":"2023-11-06T22:00:18.107017+0000","last_active":"2023-11-06T22:00:18.110595+0000","last_peered":"2023-11-06T22:00:18.110595+0000","last_clean":"2023-11-06T22:00:18.110595+0000","last_became_active":"2023-11-06T22:00:18.106699+0000","last_became_peered":"2023-11-06T22:00:18.106699+0000","last_unstale":"2023-11-06T22:00:18.110595+0000","last_undegraded":"2023-11-06T22:00:18.110595+0000","last_fullsized":"2023-11-06T22:00:18.110595+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"184'3","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112508+0000","last_change":"2023-11-06T22:00:17.606776+0000","last_active":"2023-11-06T22:00:18.112508+0000","last_peered":"2023-11-06T22:00:18.112508+0000","last_clean":"2023-11-06T22:00:18.112508+0000","last_became_active":"2023-11-06T22:00:17.605542+0000","last_became_peered":"2023-11-06T22:00:17.605542+0000","last_unstale":"2023-11-06T22:00:18.112508+0000","last_undegraded":"2023-11-06T22:00:18.112508+0000","last_fullsized":"2023-11-06T22:00:18.112508+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077152+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T22:00:18.077152+0000","last_peered":"2023-11-06T22:00:18.077152+0000","last_clean":"2023-11-06T22:00:18.077152+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:00:18.077152+0000","last_undegraded":"2023-11-06T22:00:18.077152+0000","last_fullsized":"2023-11-06T22:00:18.077152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.096360+0000","last_change":"2023-11-06T22:00:18.096360+0000","last_active":"2023-11-06T22:00:18.096360+0000","last_peered":"2023-11-06T22:00:18.096360+0000","last_clean":"2023-11-06T22:00:18.096360+0000","last_became_active":"2023-11-06T22:00:18.096025+0000","last_became_peered":"2023-11-06T22:00:18.096025+0000","last_unstale":"2023-11-06T22:00:18.096360+0000","last_undegraded":"2023-11-06T22:00:18.096360+0000","last_fullsized":"2023-11-06T22:00:18.096360+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.617231+0000","last_change":"2023-11-06T22:00:17.610418+0000","last_active":"2023-11-06T22:00:17.617231+0000","last_peered":"2023-11-06T22:00:17.617231+0000","last_clean":"2023-11-06T22:00:17.617231+0000","last_became_active":"2023-11-06T22:00:17.608860+0000","last_became_peered":"2023-11-06T22:00:17.608860+0000","last_unstale":"2023-11-06T22:00:17.617231+0000","last_undegraded":"2023-11-06T22:00:17.617231+0000","last_fullsized":"2023-11-06T22:00:17.617231+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.092254+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T22:00:18.092254+0000","last_peered":"2023-11-06T22:00:18.092254+0000","last_clean":"2023-11-06T22:00:18.092254+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:00:18.092254+0000","last_undegraded":"2023-11-06T22:00:18.092254+0000","last_fullsized":"2023-11-06T22:00:18.092254+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.104272+0000","last_change":"2023-11-06T22:00:18.099178+0000","last_active":"2023-11-06T22:00:18.104272+0000","last_peered":"2023-11-06T22:00:18.104272+0000","last_clean":"2023-11-06T22:00:18.104272+0000","last_became_active":"2023-11-06T22:00:18.098739+0000","last_became_peered":"2023-11-06T22:00:18.098739+0000","last_unstale":"2023-11-06T22:00:18.104272+0000","last_undegraded":"2023-11-06T22:00:18.104272+0000","last_fullsized":"2023-11-06T22:00:18.104272+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.14","version":"184'2","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.112191+0000","last_change":"2023-11-06T22:00:17.597193+0000","last_active":"2023-11-06T22:00:18.112191+0000","last_peered":"2023-11-06T22:00:18.112191+0000","last_clean":"2023-11-06T22:00:18.112191+0000","last_became_active":"2023-11-06T22:00:17.596847+0000","last_became_peered":"2023-11-06T22:00:17.596847+0000","last_unstale":"2023-11-06T22:00:18.112191+0000","last_undegraded":"2023-11-06T22:00:18.112191+0000","last_fullsized":"2023-11-06T22:00:18.112191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.076920+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T22:00:18.076920+0000","last_peered":"2023-11-06T22:00:18.076920+0000","last_clean":"2023-11-06T22:00:18.076920+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:00:18.076920+0000","last_undegraded":"2023-11-06T22:00:18.076920+0000","last_fullsized":"2023-11-06T22:00:18.076920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":157,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.082848+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T22:00:18.082848+0000","last_peered":"2023-11-06T22:00:18.082848+0000","last_clean":"2023-11-06T22:00:18.082848+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:00:18.082848+0000","last_undegraded":"2023-11-06T22:00:18.082848+0000","last_fullsized":"2023-11-06T22:00:18.082848+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.098384+0000","last_change":"2023-11-06T22:00:18.090235+0000","last_active":"2023-11-06T22:00:18.098384+0000","last_peered":"2023-11-06T22:00:18.098384+0000","last_clean":"2023-11-06T22:00:18.098384+0000","last_became_active":"2023-11-06T22:00:18.089449+0000","last_became_peered":"2023-11-06T22:00:18.089449+0000","last_unstale":"2023-11-06T22:00:18.098384+0000","last_undegraded":"2023-11-06T22:00:18.098384+0000","last_fullsized":"2023-11-06T22:00:18.098384+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.093271+0000","last_change":"2023-11-06T22:00:17.609973+0000","last_active":"2023-11-06T22:00:18.093271+0000","last_peered":"2023-11-06T22:00:18.093271+0000","last_clean":"2023-11-06T22:00:18.093271+0000","last_became_active":"2023-11-06T22:00:17.609628+0000","last_became_peered":"2023-11-06T22:00:17.609628+0000","last_unstale":"2023-11-06T22:00:18.093271+0000","last_undegraded":"2023-11-06T22:00:18.093271+0000","last_fullsized":"2023-11-06T22:00:18.093271+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.076646+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T22:00:18.076646+0000","last_peered":"2023-11-06T22:00:18.076646+0000","last_clean":"2023-11-06T22:00:18.076646+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:00:18.076646+0000","last_undegraded":"2023-11-06T22:00:18.076646+0000","last_fullsized":"2023-11-06T22:00:18.076646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.098485+0000","last_change":"2023-11-06T22:00:18.098485+0000","last_active":"2023-11-06T22:00:18.098485+0000","last_peered":"2023-11-06T22:00:18.098485+0000","last_clean":"2023-11-06T22:00:18.098485+0000","last_became_active":"2023-11-06T22:00:18.098208+0000","last_became_peered":"2023-11-06T22:00:18.098208+0000","last_unstale":"2023-11-06T22:00:18.098485+0000","last_undegraded":"2023-11-06T22:00:18.098485+0000","last_fullsized":"2023-11-06T22:00:18.098485+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.621745+0000","last_change":"2023-11-06T22:00:17.618031+0000","last_active":"2023-11-06T22:00:17.621745+0000","last_peered":"2023-11-06T22:00:17.621745+0000","last_clean":"2023-11-06T22:00:17.621745+0000","last_became_active":"2023-11-06T22:00:17.617769+0000","last_became_peered":"2023-11-06T22:00:17.617769+0000","last_unstale":"2023-11-06T22:00:17.621745+0000","last_undegraded":"2023-11-06T22:00:17.621745+0000","last_fullsized":"2023-11-06T22:00:17.621745+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077170+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:00:18.077170+0000","last_peered":"2023-11-06T22:00:18.077170+0000","last_clean":"2023-11-06T22:00:18.077170+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:00:18.077170+0000","last_undegraded":"2023-11-06T22:00:18.077170+0000","last_fullsized":"2023-11-06T22:00:18.077170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.111515+0000","last_change":"2023-11-06T22:00:18.109508+0000","last_active":"2023-11-06T22:00:18.111515+0000","last_peered":"2023-11-06T22:00:18.111515+0000","last_clean":"2023-11-06T22:00:18.111515+0000","last_became_active":"2023-11-06T22:00:18.109086+0000","last_became_peered":"2023-11-06T22:00:18.109086+0000","last_unstale":"2023-11-06T22:00:18.111515+0000","last_undegraded":"2023-11-06T22:00:18.111515+0000","last_fullsized":"2023-11-06T22:00:18.111515+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-11-06T22:00:17.617878+0000","last_change":"2023-11-06T22:00:17.611622+0000","last_active":"2023-11-06T22:00:17.617878+0000","last_peered":"2023-11-06T22:00:17.617878+0000","last_clean":"2023-11-06T22:00:17.617878+0000","last_became_active":"2023-11-06T22:00:17.609294+0000","last_became_peered":"2023-11-06T22:00:17.609294+0000","last_unstale":"2023-11-06T22:00:17.617878+0000","last_undegraded":"2023-11-06T22:00:17.617878+0000","last_fullsized":"2023-11-06T22:00:17.617878+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.089731+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T22:00:18.089731+0000","last_peered":"2023-11-06T22:00:18.089731+0000","last_clean":"2023-11-06T22:00:18.089731+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:00:18.089731+0000","last_undegraded":"2023-11-06T22:00:18.089731+0000","last_fullsized":"2023-11-06T22:00:18.089731+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.094208+0000","last_change":"2023-11-06T22:00:18.092304+0000","last_active":"2023-11-06T22:00:18.094208+0000","last_peered":"2023-11-06T22:00:18.094208+0000","last_clean":"2023-11-06T22:00:18.094208+0000","last_became_active":"2023-11-06T22:00:18.091811+0000","last_became_peered":"2023-11-06T22:00:18.091811+0000","last_unstale":"2023-11-06T22:00:18.094208+0000","last_undegraded":"2023-11-06T22:00:18.094208+0000","last_fullsized":"2023-11-06T22:00:18.094208+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:17.573960+0000","last_clean_scrub_stamp":"2023-11-06T22:00:17.573960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.10","version":"184'4","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.118038+0000","last_change":"2023-11-06T22:00:17.607854+0000","last_active":"2023-11-06T22:00:18.118038+0000","last_peered":"2023-11-06T22:00:18.118038+0000","last_clean":"2023-11-06T22:00:18.118038+0000","last_became_active":"2023-11-06T22:00:17.607602+0000","last_became_peered":"2023-11-06T22:00:17.607602+0000","last_unstale":"2023-11-06T22:00:18.118038+0000","last_undegraded":"2023-11-06T22:00:18.118038+0000","last_fullsized":"2023-11-06T22:00:18.118038+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:00:16.567023+0000","last_clean_scrub_stamp":"2023-11-06T22:00:16.567023+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":167,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-11-06T22:00:18.077512+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:00:18.077512+0000","last_peered":"2023-11-06T22:00:18.077512+0000","last_clean":"2023-11-06T22:00:18.077512+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:00:18.077512+0000","last_undegraded":"2023-11-06T22:00:18.077512+0000","last_fullsized":"2023-11-06T22:00:18.077512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575756,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307548,"kb_used_data":2260,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064292,"statfs":{"total":96636764160,"available":96321835008,"internally_reserved":0,"allocated":2314240,"data_stored":1178856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68799999999999994,"15min":0.71099999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.93999999999999995,"5min":0.997,"15min":1.002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.012,"5min":1.022,"15min":1.022},"last":0.66000000000000003}]},{"osd":1,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88400000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":1.083,"15min":1.083},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71999999999999997,"15min":0.746},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.90200000000000002,"5min":1.004,"15min":1.004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":0.97299999999999998,"15min":1.0029999999999999},"last":0.48399999999999999}]},{"osd":3,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88300000000000001,"5min":1.038,"15min":1.038},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.747,"15min":0.75800000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.86099999999999999,"5min":1.046,"15min":1.046},"last":0.73399999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.377},"max":{"1min":0.83099999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.75900000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96499999999999997,"5min":1.145,"15min":1.145},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.68799999999999994},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92000000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.747}]},{"osd":6,"last update":"Mon Nov 6 22:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.495},"max":{"1min":1.0629999999999999,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69999999999999996,"15min":0.70699999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.84499999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.59699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575756,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307532,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064308,"statfs":{"total":96636764160,"available":96321851392,"internally_reserved":0,"allocated":2297856,"data_stored":1172066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46700000000000003,"5min":0.45500000000000002,"15min":0.40000000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.04},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71099999999999997,"15min":0.71199999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.37},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.0209999999999999},"last":0.81000000000000005}]},{"osd":1,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83099999999999996,"5min":0.93600000000000005,"15min":0.99399999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.32600000000000001},"max":{"1min":0.91100000000000003,"5min":1.004,"15min":1.004},"last":0.79500000000000004}]},{"osd":2,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95299999999999996,"5min":1.034,"15min":1.034},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94499999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":1.02},"last":0.753},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76200000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.503},"max":{"1min":0.89700000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.69599999999999995}]},{"osd":4,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.39000000000000001},"max":{"1min":0.84899999999999998,"5min":0.85199999999999998,"15min":0.94399999999999995},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64400000000000002,"15min":0.63300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.434},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.90300000000000002}]},{"osd":5,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.70199999999999996},"min":{"1min":0.499,"5min":0.498,"15min":0.46200000000000002},"max":{"1min":0.95599999999999996,"5min":0.98099999999999998,"15min":0.98699999999999999},"last":0.878}]},{"osd":7,"last update":"Mon Nov 6 21:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.44400000000000001},"max":{"1min":0.84099999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.48899999999999999},"max":{"1min":0.93899999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.85999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2297856,"data_stored":1172018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97299999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"last":0.44900000000000001}]},{"osd":2,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.33900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0209999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.875,"5min":0.875,"15min":1.1850000000000001},"last":0.46700000000000003}]},{"osd":3,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.60399999999999998},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57099999999999995}]},{"osd":4,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0269999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83799999999999997,"15min":0.84699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59399999999999997},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.96499999999999997}]},{"osd":5,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.81299999999999994},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.55700000000000005},"max":{"1min":1.02,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.046,"5min":1.1559999999999999,"15min":1.198},"last":0.88600000000000001}]},{"osd":6,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.82399999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.107,"5min":1.1160000000000001,"15min":1.151},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.83699999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.51600000000000001},"max":{"1min":1.083,"5min":1.206,"15min":1.206},"last":0.95699999999999996}]},{"osd":7,"last update":"Mon Nov 6 21:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85299999999999998,"15min":0.86199999999999999},"min":{"1min":0.70899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1499999999999999,"5min":1.2070000000000001,"15min":1.21},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0980000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.97799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575756,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2301952,"data_stored":1172922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46300000000000002,"15min":0.47199999999999998},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.749,"15min":0.82899999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52400000000000002,"15min":0.52900000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.38800000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.34000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65400000000000003,"5min":0.72699999999999998,"15min":0.91700000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.376},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.48299999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.56299999999999994},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80500000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.59999999999999998},"min":{"1min":0.45500000000000002,"5min":0.436,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.88200000000000001},"last":0.44600000000000001}]},{"osd":4,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.57099999999999995},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.84699999999999998,"15min":0.84799999999999998},"min":{"1min":0.626,"5min":0.57399999999999995,"15min":0.51700000000000002},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.83799999999999997}]},{"osd":5,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85299999999999998,"15min":0.85399999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.306},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85699999999999998,"15min":0.86099999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.155,"5min":1.155,"15min":1.2729999999999999},"last":0.88400000000000001}]},{"osd":6,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82399999999999995,"15min":0.83399999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.55000000000000004},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.872},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.85399999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:00:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.56000000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.81899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575756,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2301952,"data_stored":1173442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.46200000000000002},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.28999999999999998},"max":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.32800000000000001},"max":{"1min":0.66900000000000004,"5min":0.84799999999999998,"15min":0.86299999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.432},"max":{"1min":0.81899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.499}]},{"osd":3,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51400000000000001,"15min":0.51500000000000001},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70299999999999996,"5min":0.77100000000000002,"15min":0.80000000000000004},"last":0.434},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.44800000000000001,"5min":0.432,"15min":0.41499999999999998},"max":{"1min":0.77700000000000002,"5min":0.89700000000000002,"15min":1.0940000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.622,"15min":0.55500000000000005},"max":{"1min":1.075,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83699999999999997,"15min":0.81699999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1020000000000001,"5min":1.256,"15min":1.256},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.81000000000000005},"min":{"1min":0.64400000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.86099999999999999,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.58299999999999996},"max":{"1min":1.0840000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.84199999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.51800000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.69299999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.145,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Nov 6 21:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.88300000000000001,"15min":0.871},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.57299999999999995},"max":{"1min":1.173,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.752,"5min":0.57699999999999996,"15min":0.55900000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.79400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575757,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307536,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064304,"statfs":{"total":96636764160,"available":96321847296,"internally_reserved":0,"allocated":2301952,"data_stored":1172640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.33400000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.48599999999999999}]},{"osd":1,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.32400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.376},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.64400000000000002}]},{"osd":2,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.375},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.40899999999999997},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86199999999999999,"15min":0.84399999999999997},"min":{"1min":0.73799999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.85999999999999999,"15min":0.84599999999999997},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.47999999999999998},"max":{"1min":1.0309999999999999,"5min":1.202,"15min":1.202},"last":0.70999999999999996}]},{"osd":5,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.85699999999999998,"15min":0.82699999999999996},"min":{"1min":0.66000000000000003,"5min":0.59499999999999997,"15min":0.57399999999999995},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.67600000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.80600000000000005}]},{"osd":6,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.86899999999999999,"15min":0.83599999999999997},"min":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.63},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89500000000000002,"15min":0.86699999999999999},"min":{"1min":0.69399999999999995,"5min":0.66500000000000004,"15min":0.59999999999999998},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.69599999999999995}]},{"osd":7,"last update":"Mon Nov 6 21:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.87,"15min":0.85899999999999999},"min":{"1min":0.69299999999999995,"5min":0.58899999999999997,"15min":0.55500000000000005},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.92000000000000004,"15min":0.88100000000000001},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.56799999999999995},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575756,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303444,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068396,"statfs":{"total":96636764160,"available":96326037504,"internally_reserved":0,"allocated":2306048,"data_stored":1172654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":1.0589999999999999,"15min":1.1659999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82699999999999996,"5min":0.999,"15min":0.999},"last":0.73699999999999999}]},{"osd":1,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.747,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.44700000000000001},"max":{"1min":1.05,"5min":1.0700000000000001,"15min":1.131},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.99299999999999999,"15min":1.014},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.80500000000000005,"15min":0.79900000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91700000000000004,"5min":0.95299999999999996,"15min":1.181},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.73799999999999999,"15min":0.72499999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.47099999999999997},"max":{"1min":0.93100000000000005,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.81899999999999995}]},{"osd":5,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.61799999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.36899999999999999},"max":{"1min":0.90500000000000003,"5min":0.94499999999999995,"15min":1.228},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.443},"max":{"1min":0.80200000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.42799999999999999},"max":{"1min":0.88400000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Mon Nov 6 22:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.41999999999999998},"max":{"1min":1,"5min":1.0209999999999999,"15min":1.085},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.71799999999999997},"min":{"1min":0.51700000000000002,"5min":0.504,"15min":0.496},"max":{"1min":1.077,"5min":1.077,"15min":1.091},"last":0.88500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575757,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307540,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064300,"statfs":{"total":96636764160,"available":96321843200,"internally_reserved":0,"allocated":2306048,"data_stored":1172730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67500000000000004,"15min":0.67800000000000005},"min":{"1min":0.48999999999999999,"5min":0.38600000000000001,"15min":0.33600000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.008},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.46899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.98199999999999998,"15min":0.99299999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75,"15min":0.753},"min":{"1min":0.57899999999999996,"5min":0.375,"15min":0.375},"max":{"1min":0.92100000000000004,"5min":1.0009999999999999,"15min":1.2370000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.496,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.0629999999999999,"15min":1.2470000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.58499999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94399999999999995,"5min":1.085,"15min":1.085},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.753},"min":{"1min":0.66700000000000004,"5min":0.55100000000000005,"15min":0.47199999999999998},"max":{"1min":0.95599999999999996,"5min":1.036,"15min":1.036},"last":0.52200000000000002}]},{"osd":3,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.60499999999999998,"5min":0.49099999999999999,"15min":0.47399999999999998},"max":{"1min":0.88300000000000001,"5min":1.02,"15min":1.02},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.63500000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90400000000000003,"5min":1.093,"15min":1.093},"last":0.5}]},{"osd":4,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.628},{"interface":"front","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.69899999999999995},"min":{"1min":0.53400000000000003,"5min":0.437,"15min":0.39300000000000002},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":0.997},"last":0.51300000000000001}]},{"osd":6,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66900000000000004,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93400000000000005,"5min":0.99199999999999999,"15min":0.99299999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.751,"15min":0.72599999999999998},"min":{"1min":0.60299999999999998,"5min":0.46100000000000002,"15min":0.41499999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0449999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Mon Nov 6 21:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68500000000000005,"15min":0.66400000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.05},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72699999999999998,"15min":0.70099999999999996},"min":{"1min":0.48899999999999999,"5min":0.45700000000000002,"15min":0.436},"max":{"1min":0.95699999999999996,"5min":1.014,"15min":1.0780000000000001},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:00:22.876 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T22:00:22.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T22:00:23.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.113+0000 7f2a271ff700 1 Processor -- start 2023-11-06T22:00:23.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.113+0000 7f2a271ff700 1 -- start start 2023-11-06T22:00:23.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.114+0000 7f2a271ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2a201544f0 0x7f2a200c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.114+0000 7f2a271ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a200c0c40 0x7f2a200646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.114+0000 7f2a271ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a20064bb0 0x7f2a20157f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.114+0000 7f2a271ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a201548d0 con 0x7f2a200c0c40 2023-11-06T22:00:23.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.114+0000 7f2a271ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a201556d0 con 0x7f2a201544f0 2023-11-06T22:00:23.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.114+0000 7f2a271ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a20065010 con 0x7f2a20064bb0 2023-11-06T22:00:23.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a1ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a200c0c40 0x7f2a200646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a1ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a200c0c40 0x7f2a200646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45120/0 (socket says 172.21.15.33:45120) 2023-11-06T22:00:23.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a1ffff700 1 -- 172.21.15.33:0/1256176812 learned_addr learned my addr 172.21.15.33:0/1256176812 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a24f9b700 1 --2- 172.21.15.33:0/1256176812 >> v2:172.21.15.69:3300/0 conn(0x7f2a201544f0 0x7f2a200c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a2579c700 1 --2- 172.21.15.33:0/1256176812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a20064bb0 0x7f2a20157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a1ffff700 1 -- 172.21.15.33:0/1256176812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a20064bb0 msgr2=0x7f2a20157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a1ffff700 1 --2- 172.21.15.33:0/1256176812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a20064bb0 0x7f2a20157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a1ffff700 1 -- 172.21.15.33:0/1256176812 >> v2:172.21.15.69:3300/0 conn(0x7f2a201544f0 msgr2=0x7f2a200c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a1ffff700 1 --2- 172.21.15.33:0/1256176812 >> v2:172.21.15.69:3300/0 conn(0x7f2a201544f0 0x7f2a200c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a1ffff700 1 -- 172.21.15.33:0/1256176812 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a20154ac0 con 0x7f2a200c0c40 2023-11-06T22:00:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a24f9b700 1 --2- 172.21.15.33:0/1256176812 >> v2:172.21.15.69:3300/0 conn(0x7f2a201544f0 0x7f2a200c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.115+0000 7f2a1ffff700 1 --2- 172.21.15.33:0/1256176812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a200c0c40 0x7f2a200646c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f2a20084660 tx=0x7f2a0c0090f0).ready entity=mon.0 client_cookie=47255d292f7a386d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:23.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a1f7fe700 1 -- 172.21.15.33:0/1256176812 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c00b600 con 0x7f2a200c0c40 2023-11-06T22:00:23.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a1f7fe700 1 -- 172.21.15.33:0/1256176812 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c00b760 con 0x7f2a200c0c40 2023-11-06T22:00:23.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a1f7fe700 1 -- 172.21.15.33:0/1256176812 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c00bae0 con 0x7f2a200c0c40 2023-11-06T22:00:23.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a271ff700 1 -- 172.21.15.33:0/1256176812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a200c0c40 msgr2=0x7f2a200646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a271ff700 1 --2- 172.21.15.33:0/1256176812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a200c0c40 0x7f2a200646c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f2a20084660 tx=0x7f2a0c0090f0).stop 2023-11-06T22:00:23.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a271ff700 1 -- 172.21.15.33:0/1256176812 shutdown_connections 2023-11-06T22:00:23.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a271ff700 1 --2- 172.21.15.33:0/1256176812 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a20064bb0 0x7f2a20157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a271ff700 1 --2- 172.21.15.33:0/1256176812 >> v2:172.21.15.69:3300/0 conn(0x7f2a201544f0 0x7f2a200c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a271ff700 1 --2- 172.21.15.33:0/1256176812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a200c0c40 0x7f2a200646c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a271ff700 1 -- 172.21.15.33:0/1256176812 >> 172.21.15.33:0/1256176812 conn(0x7f2a200cb900 msgr2=0x7f2a200c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a271ff700 1 -- 172.21.15.33:0/1256176812 shutdown_connections 2023-11-06T22:00:23.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.116+0000 7f2a271ff700 1 -- 172.21.15.33:0/1256176812 wait complete. 2023-11-06T22:00:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.118+0000 7f2a271ff700 1 Processor -- start 2023-11-06T22:00:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.118+0000 7f2a271ff700 1 -- start start 2023-11-06T22:00:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a271ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a20064bb0 0x7f2a20150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a271ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a200c0c40 0x7f2a200c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a271ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a201544f0 0x7f2a200c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a271ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a200c4400 con 0x7f2a201544f0 2023-11-06T22:00:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a271ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a20165800 con 0x7f2a200c0c40 2023-11-06T22:00:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a271ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a200b0db0 con 0x7f2a20064bb0 2023-11-06T22:00:23.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a2579c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a201544f0 0x7f2a200c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a2579c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a201544f0 0x7f2a200c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45128/0 (socket says 172.21.15.33:45128) 2023-11-06T22:00:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a2579c700 1 -- 172.21.15.33:0/1022100062 learned_addr learned my addr 172.21.15.33:0/1022100062 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a24f9b700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a20064bb0 0x7f2a20150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.119+0000 7f2a1ffff700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a200c0c40 0x7f2a200c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a2579c700 1 -- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a20064bb0 msgr2=0x7f2a20150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a2579c700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a20064bb0 0x7f2a20150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a2579c700 1 -- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a200c0c40 msgr2=0x7f2a200c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a2579c700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a200c0c40 0x7f2a200c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a2579c700 1 -- 172.21.15.33:0/1022100062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c009370 con 0x7f2a201544f0 2023-11-06T22:00:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a2579c700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a201544f0 0x7f2a200c3d80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f2a18009fa0 tx=0x7f2a18009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a1cff9700 1 -- 172.21.15.33:0/1022100062 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800ebc0 con 0x7f2a201544f0 2023-11-06T22:00:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a1cff9700 1 -- 172.21.15.33:0/1022100062 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1800ed20 con 0x7f2a201544f0 2023-11-06T22:00:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a271ff700 1 -- 172.21.15.33:0/1022100062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a20164a40 con 0x7f2a201544f0 2023-11-06T22:00:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a1cff9700 1 -- 172.21.15.33:0/1022100062 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18014610 con 0x7f2a201544f0 2023-11-06T22:00:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.120+0000 7f2a271ff700 1 -- 172.21.15.33:0/1022100062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a20165000 con 0x7f2a201544f0 2023-11-06T22:00:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.121+0000 7f2a271ff700 1 -- 172.21.15.33:0/1022100062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a20154da0 con 0x7f2a201544f0 2023-11-06T22:00:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.122+0000 7f2a1cff9700 1 -- 172.21.15.33:0/1022100062 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2a1801d3b0 con 0x7f2a201544f0 2023-11-06T22:00:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.123+0000 7f2a1cff9700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a0405f2c0 0x7f2a04061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.123+0000 7f2a1cff9700 1 -- 172.21.15.33:0/1022100062 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f2a1807a880 con 0x7f2a201544f0 2023-11-06T22:00:23.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.124+0000 7f2a24f9b700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a0405f2c0 0x7f2a04061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.124+0000 7f2a24f9b700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a0405f2c0 0x7f2a04061780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2a14006fd0 tx=0x7f2a14008040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.127+0000 7f2a1cff9700 1 -- 172.21.15.33:0/1022100062 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2a1804b900 con 0x7f2a201544f0 2023-11-06T22:00:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.375+0000 7f2a271ff700 1 -- 172.21.15.33:0/1022100062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2a200c8760 con 0x7f2a201544f0 2023-11-06T22:00:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.379+0000 7f2a1cff9700 1 -- 172.21.15.33:0/1022100062 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v40) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2a1804ef20 con 0x7f2a201544f0 2023-11-06T22:00:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 -- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a0405f2c0 msgr2=0x7f2a04061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a0405f2c0 0x7f2a04061780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2a14006fd0 tx=0x7f2a14008040).stop 2023-11-06T22:00:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 -- 172.21.15.33:0/1022100062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a201544f0 msgr2=0x7f2a200c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a201544f0 0x7f2a200c3d80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f2a18009fa0 tx=0x7f2a18009900).stop 2023-11-06T22:00:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 -- 172.21.15.33:0/1022100062 shutdown_connections 2023-11-06T22:00:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f2a0405f2c0 0x7f2a04061780 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a20064bb0 0x7f2a20150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a201544f0 0x7f2a200c3d80 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 --2- 172.21.15.33:0/1022100062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a200c0c40 0x7f2a200c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 -- 172.21.15.33:0/1022100062 >> 172.21.15.33:0/1022100062 conn(0x7f2a200cb900 msgr2=0x7f2a200c5650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:23.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 -- 172.21.15.33:0/1022100062 shutdown_connections 2023-11-06T22:00:23.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.381+0000 7f2a271ff700 1 -- 172.21.15.33:0/1022100062 wait complete. 2023-11-06T22:00:23.386 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T22:00:23.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:00:23.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.594+0000 7fcba7dff700 1 Processor -- start 2023-11-06T22:00:23.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.594+0000 7fcba7dff700 1 -- start start 2023-11-06T22:00:23.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.596+0000 7fcba7dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcba00c5d50 0x7fcba00c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.596+0000 7fcba7dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcba00bfd70 0x7fcba00c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.596+0000 7fcba7dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba01554a0 con 0x7fcba00c1840 2023-11-06T22:00:23.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.596+0000 7fcba7dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba0154960 con 0x7fcba00bfd70 2023-11-06T22:00:23.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.596+0000 7fcba7dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00c66c0 con 0x7fcba00c5d50 2023-11-06T22:00:23.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.596+0000 7fcba639c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcba00bfd70 0x7fcba00c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.596+0000 7fcba639c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcba00bfd70 0x7fcba00c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45142/0 (socket says 172.21.15.33:45142) 2023-11-06T22:00:23.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.596+0000 7fcba639c700 1 -- 172.21.15.33:0/4162331675 learned_addr learned my addr 172.21.15.33:0/4162331675 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:23.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.596+0000 7fcba539a700 1 --2- 172.21.15.33:0/4162331675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcba00c5d50 0x7fcba00c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.597+0000 7fcba639c700 1 -- 172.21.15.33:0/4162331675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcba00c5d50 msgr2=0x7fcba00c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.597+0000 7fcba639c700 1 --2- 172.21.15.33:0/4162331675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcba00c5d50 0x7fcba00c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.597+0000 7fcba639c700 1 -- 172.21.15.33:0/4162331675 >> v1:172.21.15.69:6789/0 conn(0x7fcba00c1840 legacy=0x7fcba00a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.597+0000 7fcba639c700 1 -- 172.21.15.33:0/4162331675 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba0155930 con 0x7fcba00bfd70 2023-11-06T22:00:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.597+0000 7fcba639c700 1 --2- 172.21.15.33:0/4162331675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00bfd70 0x7fcba00c01e0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7fcb98009f20 tx=0x7fcb98009840).ready entity=mon.0 client_cookie=428044b55c3dc621 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.597+0000 7fcba4b99700 1 -- 172.21.15.33:0/4162331675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800e070 con 0x7fcba00bfd70 2023-11-06T22:00:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.597+0000 7fcba4b99700 1 -- 172.21.15.33:0/4162331675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9800b9d0 con 0x7fcba00bfd70 2023-11-06T22:00:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.597+0000 7fcba4b99700 1 -- 172.21.15.33:0/4162331675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800bd50 con 0x7fcba00bfd70 2023-11-06T22:00:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.598+0000 7fcba7dff700 1 -- 172.21.15.33:0/4162331675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00bfd70 msgr2=0x7fcba00c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.598+0000 7fcba7dff700 1 --2- 172.21.15.33:0/4162331675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00bfd70 0x7fcba00c01e0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7fcb98009f20 tx=0x7fcb98009840).stop 2023-11-06T22:00:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.598+0000 7fcba7dff700 1 -- 172.21.15.33:0/4162331675 shutdown_connections 2023-11-06T22:00:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.598+0000 7fcba7dff700 1 --2- 172.21.15.33:0/4162331675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00bfd70 0x7fcba00c01e0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.598+0000 7fcba7dff700 1 --2- 172.21.15.33:0/4162331675 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcba00c5d50 0x7fcba00c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.598+0000 7fcba7dff700 1 -- 172.21.15.33:0/4162331675 >> 172.21.15.33:0/4162331675 conn(0x7fcba005ec80 msgr2=0x7fcba00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.598+0000 7fcba7dff700 1 -- 172.21.15.33:0/4162331675 shutdown_connections 2023-11-06T22:00:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.598+0000 7fcba7dff700 1 -- 172.21.15.33:0/4162331675 wait complete. 2023-11-06T22:00:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.599+0000 7fcba7dff700 1 Processor -- start 2023-11-06T22:00:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.599+0000 7fcba7dff700 1 -- start start 2023-11-06T22:00:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.600+0000 7fcba7dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcba00bfd70 0x7fcba00c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.600+0000 7fcba7dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00c1840 0x7fcba0149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.600+0000 7fcba7dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba00c5d50 0x7fcba00c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.600+0000 7fcba7dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba014ecc0 con 0x7fcba00c1840 2023-11-06T22:00:23.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.600+0000 7fcba7dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00c2a20 con 0x7fcba00c5d50 2023-11-06T22:00:23.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.600+0000 7fcba7dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00c2d00 con 0x7fcba00bfd70 2023-11-06T22:00:23.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00c1840 0x7fcba0149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00c1840 0x7fcba0149aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45158/0 (socket says 172.21.15.33:45158) 2023-11-06T22:00:23.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba539a700 1 -- 172.21.15.33:0/1185084621 learned_addr learned my addr 172.21.15.33:0/1185084621 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:23.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba639c700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba00c5d50 0x7fcba00c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba5b9b700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcba00bfd70 0x7fcba00c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba539a700 1 -- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcba00bfd70 msgr2=0x7fcba00c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba539a700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcba00bfd70 0x7fcba00c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba539a700 1 -- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba00c5d50 msgr2=0x7fcba00c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba539a700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba00c5d50 0x7fcba00c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba539a700 1 -- 172.21.15.33:0/1185084621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb98009ae0 con 0x7fcba00c1840 2023-11-06T22:00:23.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.601+0000 7fcba539a700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00c1840 0x7fcba0149aa0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7fcb9000e900 tx=0x7fcb90009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:23.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.602+0000 7fcb95ffb700 1 -- 172.21.15.33:0/1185084621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90012600 con 0x7fcba00c1840 2023-11-06T22:00:23.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.602+0000 7fcb95ffb700 1 -- 172.21.15.33:0/1185084621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90012760 con 0x7fcba00c1840 2023-11-06T22:00:23.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.602+0000 7fcb95ffb700 1 -- 172.21.15.33:0/1185084621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90012ae0 con 0x7fcba00c1840 2023-11-06T22:00:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.602+0000 7fcba7dff700 1 -- 172.21.15.33:0/1185084621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba00c2fe0 con 0x7fcba00c1840 2023-11-06T22:00:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.602+0000 7fcba7dff700 1 -- 172.21.15.33:0/1185084621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba00c35a0 con 0x7fcba00c1840 2023-11-06T22:00:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.603+0000 7fcba7dff700 1 -- 172.21.15.33:0/1185084621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba00c3910 con 0x7fcba00c1840 2023-11-06T22:00:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.604+0000 7fcb95ffb700 1 -- 172.21.15.33:0/1185084621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcb9001b400 con 0x7fcba00c1840 2023-11-06T22:00:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.604+0000 7fcb95ffb700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb8405f270 0x7fcb84061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.605+0000 7fcb95ffb700 1 -- 172.21.15.33:0/1185084621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fcb9007a720 con 0x7fcba00c1840 2023-11-06T22:00:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.605+0000 7fcba5b9b700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb8405f270 0x7fcb84061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.605+0000 7fcba5b9b700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb8405f270 0x7fcb84061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fcba014f2d0 tx=0x7fcb9c009040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:23.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.609+0000 7fcb95ffb700 1 -- 172.21.15.33:0/1185084621 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcb9004b720 con 0x7fcba00c1840 2023-11-06T22:00:23.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.832+0000 7fcba7dff700 1 -- 172.21.15.33:0/1185084621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcba00c27c0 con 0x7fcba00c1840 2023-11-06T22:00:23.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.833+0000 7fcb95ffb700 1 -- 172.21.15.33:0/1185084621 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4490 (secure 0 0 0) 0x7fcb9004ed40 con 0x7fcba00c1840 2023-11-06T22:00:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:00:23.835 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":76,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3392183580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3392183580},{"type":"v1","addr":"172.21.15.33:6835","nonce":3392183580}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2124998325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2124998325},{"type":"v1","addr":"172.21.15.69:6837","nonce":2124998325}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3826680893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3826680893},{"type":"v1","addr":"172.21.15.33:6834","nonce":3826680893}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:00:18.069753+0000","modified":"2023-11-06T22:00:21.084401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8551":{"gid":8551,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/249801946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":249801946},{"type":"v1","addr":"172.21.15.69:6836","nonce":249801946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-06T22:00:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.834+0000 7fcba7dff700 1 -- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb8405f270 msgr2=0x7fcb84061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.834+0000 7fcba7dff700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb8405f270 0x7fcb84061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fcba014f2d0 tx=0x7fcb9c009040).stop 2023-11-06T22:00:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.835+0000 7fcba7dff700 1 -- 172.21.15.33:0/1185084621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00c1840 msgr2=0x7fcba0149aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.835+0000 7fcba7dff700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00c1840 0x7fcba0149aa0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7fcb9000e900 tx=0x7fcb90009060).stop 2023-11-06T22:00:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.835+0000 7fcba7dff700 1 -- 172.21.15.33:0/1185084621 shutdown_connections 2023-11-06T22:00:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.835+0000 7fcba7dff700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fcb8405f270 0x7fcb84061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.835+0000 7fcba7dff700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcba00bfd70 0x7fcba00c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.835+0000 7fcba7dff700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba00c5d50 0x7fcba00c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.835+0000 7fcba7dff700 1 --2- 172.21.15.33:0/1185084621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba00c1840 0x7fcba0149aa0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.835+0000 7fcba7dff700 1 -- 172.21.15.33:0/1185084621 >> 172.21.15.33:0/1185084621 conn(0x7fcba005ec80 msgr2=0x7fcba00ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.835+0000 7fcba7dff700 1 -- 172.21.15.33:0/1185084621 shutdown_connections 2023-11-06T22:00:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:23.835+0000 7fcba7dff700 1 -- 172.21.15.33:0/1185084621 wait complete. 2023-11-06T22:00:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 76 2023-11-06T22:00:24.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T22:00:18.069753+0000', 'modified': '2023-11-06T22:00:21.084401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8551}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8551': {'gid': 8551, 'name': 'd', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.69:6836/249801946', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 249801946}, {'type': 'v1', 'addr': '172.21.15.69:6836', 'nonce': 249801946}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T22:00:24.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T22:00:24.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-06T22:00:24.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.457+0000 7f5eec5ff700 1 Processor -- start 2023-11-06T22:00:24.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.457+0000 7f5eec5ff700 1 -- start start 2023-11-06T22:00:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.458+0000 7f5eec5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5ee40c6d90 0x7f5ee40c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.458+0000 7f5eec5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ee40bfd60 0x7f5ee40c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.458+0000 7f5eec5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee40c7700 con 0x7f5ee4154210 2023-11-06T22:00:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.458+0000 7f5eec5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee40c06c0 con 0x7f5ee40bfd60 2023-11-06T22:00:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.458+0000 7f5eec5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee405efc0 con 0x7f5ee40c6d90 2023-11-06T22:00:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.459+0000 7f5eeab9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ee40bfd60 0x7f5ee40c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.459+0000 7f5eeab9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ee40bfd60 0x7f5ee40c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45162/0 (socket says 172.21.15.33:45162) 2023-11-06T22:00:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.459+0000 7f5eeab9c700 1 -- 172.21.15.33:0/973701009 learned_addr learned my addr 172.21.15.33:0/973701009 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.459+0000 7f5ee9b9a700 1 --2- 172.21.15.33:0/973701009 >> v2:172.21.15.69:3300/0 conn(0x7f5ee40c6d90 0x7f5ee40c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.459+0000 7f5eeab9c700 1 -- 172.21.15.33:0/973701009 >> v2:172.21.15.69:3300/0 conn(0x7f5ee40c6d90 msgr2=0x7f5ee40c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.459+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/973701009 >> v2:172.21.15.69:3300/0 conn(0x7f5ee40c6d90 0x7f5ee40c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.459+0000 7f5eeab9c700 1 -- 172.21.15.33:0/973701009 >> v1:172.21.15.69:6789/0 conn(0x7f5ee4154210 legacy=0x7f5ee40a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.459+0000 7f5eeab9c700 1 -- 172.21.15.33:0/973701009 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee40c8ea0 con 0x7f5ee40bfd60 2023-11-06T22:00:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.459+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/973701009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee40bfd60 0x7f5ee40c01d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f5ee40cbf10 tx=0x7f5edc009840).ready entity=mon.0 client_cookie=205259d6fbc75ff4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5ee9399700 1 -- 172.21.15.33:0/973701009 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc00e070 con 0x7f5ee40bfd60 2023-11-06T22:00:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5ee9399700 1 -- 172.21.15.33:0/973701009 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc00ba20 con 0x7f5ee40bfd60 2023-11-06T22:00:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5ee9399700 1 -- 172.21.15.33:0/973701009 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc00bda0 con 0x7f5ee40bfd60 2023-11-06T22:00:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5eec5ff700 1 -- 172.21.15.33:0/973701009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee40bfd60 msgr2=0x7f5ee40c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5eec5ff700 1 --2- 172.21.15.33:0/973701009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee40bfd60 0x7f5ee40c01d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f5ee40cbf10 tx=0x7f5edc009840).stop 2023-11-06T22:00:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5eec5ff700 1 -- 172.21.15.33:0/973701009 shutdown_connections 2023-11-06T22:00:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5eec5ff700 1 --2- 172.21.15.33:0/973701009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee40bfd60 0x7f5ee40c01d0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5eec5ff700 1 --2- 172.21.15.33:0/973701009 >> v2:172.21.15.69:3300/0 conn(0x7f5ee40c6d90 0x7f5ee40c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5eec5ff700 1 -- 172.21.15.33:0/973701009 >> 172.21.15.33:0/973701009 conn(0x7f5ee405e990 msgr2=0x7f5ee40cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5eec5ff700 1 -- 172.21.15.33:0/973701009 shutdown_connections 2023-11-06T22:00:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.460+0000 7f5eec5ff700 1 -- 172.21.15.33:0/973701009 wait complete. 2023-11-06T22:00:24.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.462+0000 7f5eec5ff700 1 Processor -- start 2023-11-06T22:00:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.462+0000 7f5eec5ff700 1 -- start start 2023-11-06T22:00:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.463+0000 7f5eec5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee40bfd60 0x7f5ee413dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.463+0000 7f5eec5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee40c6d90 0x7f5ee41545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.463+0000 7f5eec5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee4154210 0x7f5ee4154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.463+0000 7f5eec5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4154fd0 con 0x7f5ee4154210 2023-11-06T22:00:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.463+0000 7f5eec5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee40c2ef0 con 0x7f5ee40c6d90 2023-11-06T22:00:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.463+0000 7f5eec5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee40c34b0 con 0x7f5ee40bfd60 2023-11-06T22:00:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.463+0000 7f5eeab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee4154210 0x7f5ee4154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.463+0000 7f5eeab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee4154210 0x7f5ee4154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45174/0 (socket says 172.21.15.33:45174) 2023-11-06T22:00:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.463+0000 7f5eeab9c700 1 -- 172.21.15.33:0/2639296680 learned_addr learned my addr 172.21.15.33:0/2639296680 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.463+0000 7f5ee9b9a700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee40c6d90 0x7f5ee41545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eea39b700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee40bfd60 0x7f5ee413dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eeab9c700 1 -- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee40bfd60 msgr2=0x7f5ee413dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee40bfd60 0x7f5ee413dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eeab9c700 1 -- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee40c6d90 msgr2=0x7f5ee41545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee40c6d90 0x7f5ee41545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eeab9c700 1 -- 172.21.15.33:0/2639296680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc009ae0 con 0x7f5ee4154210 2023-11-06T22:00:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eea39b700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee40bfd60 0x7f5ee413dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee4154210 0x7f5ee4154ae0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f5edc00bec0 tx=0x7f5edc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eda7fc700 1 -- 172.21.15.33:0/2639296680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc00b6f0 con 0x7f5ee4154210 2023-11-06T22:00:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eda7fc700 1 -- 172.21.15.33:0/2639296680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc0092e0 con 0x7f5ee4154210 2023-11-06T22:00:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eec5ff700 1 -- 172.21.15.33:0/2639296680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee4157630 con 0x7f5ee4154210 2023-11-06T22:00:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eda7fc700 1 -- 172.21.15.33:0/2639296680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc01e6a0 con 0x7f5ee4154210 2023-11-06T22:00:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.464+0000 7f5eec5ff700 1 -- 172.21.15.33:0/2639296680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee40c3140 con 0x7f5ee4154210 2023-11-06T22:00:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.466+0000 7f5eda7fc700 1 -- 172.21.15.33:0/2639296680 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5edc003040 con 0x7f5ee4154210 2023-11-06T22:00:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.467+0000 7f5eda7fc700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ec805f270 0x7f5ec8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.467+0000 7f5eda7fc700 1 -- 172.21.15.33:0/2639296680 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f5edc07b9b0 con 0x7f5ee4154210 2023-11-06T22:00:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.467+0000 7f5eea39b700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ec805f270 0x7f5ec8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.468+0000 7f5eea39b700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ec805f270 0x7f5ec8061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5ee0000c00 tx=0x7f5ee0009040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.469+0000 7f5f039a8b80 1 Processor -- start 2023-11-06T22:00:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.469+0000 7f5f039a8b80 1 -- start start 2023-11-06T22:00:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.470+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d8215d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.470+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.470+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d8215cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.470+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d82150d2f0 con 0x55d8215d1080 2023-11-06T22:00:24.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.470+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8215227e0 con 0x55d821619be0 2023-11-06T22:00:24.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.470+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d821522920 con 0x55d8215f94c0 2023-11-06T22:00:24.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5eea39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d8215d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5ee9b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5eea39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d8215d1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45176/0 (socket says 172.21.15.33:45176) 2023-11-06T22:00:24.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5eea39b700 1 -- 172.21.15.33:0/22896232 learned_addr learned my addr 172.21.15.33:0/22896232 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:24.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5ee9b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:60784/0 (socket says 172.21.15.33:60784) 2023-11-06T22:00:24.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/22896232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d8215cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5eea39b700 1 -- 172.21.15.33:0/22896232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 msgr2=0x55d8215cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5eea39b700 1 --2- 172.21.15.33:0/22896232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d8215cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5eea39b700 1 -- 172.21.15.33:0/22896232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 msgr2=0x55d8215cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5eea39b700 1 --2- 172.21.15.33:0/22896232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5eea39b700 1 -- 172.21.15.33:0/22896232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d82151b720 con 0x55d8215d1080 2023-11-06T22:00:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5eea39b700 1 --2- 172.21.15.33:0/22896232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d8215d1460 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f5ee0018680 tx=0x7f5ee0003000).ready entity=mon.0 client_cookie=46fe677b347f29f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:24.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5ec71fe700 1 -- 172.21.15.33:0/22896232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0018240 con 0x55d8215d1080 2023-11-06T22:00:24.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5ec71fe700 1 -- 172.21.15.33:0/22896232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee00183a0 con 0x55d8215d1080 2023-11-06T22:00:24.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.471+0000 7f5ec71fe700 1 -- 172.21.15.33:0/22896232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0015620 con 0x55d8215d1080 2023-11-06T22:00:24.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.472+0000 7f5f039a8b80 1 -- 172.21.15.33:0/22896232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 msgr2=0x55d8215d1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.472+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/22896232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d8215d1460 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f5ee0018680 tx=0x7f5ee0003000).stop 2023-11-06T22:00:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.472+0000 7f5f039a8b80 1 -- 172.21.15.33:0/22896232 shutdown_connections 2023-11-06T22:00:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.472+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/22896232 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d8215cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.472+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/22896232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d8215d1460 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.472+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/22896232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.472+0000 7f5f039a8b80 1 -- 172.21.15.33:0/22896232 >> 172.21.15.33:0/22896232 conn(0x55d82143a940 msgr2=0x55d82161a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.472+0000 7f5f039a8b80 1 -- 172.21.15.33:0/22896232 shutdown_connections 2023-11-06T22:00:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.472+0000 7f5f039a8b80 1 -- 172.21.15.33:0/22896232 wait complete. 2023-11-06T22:00:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.473+0000 7f5f039a8b80 1 -- start start 2023-11-06T22:00:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.473+0000 7f5f039a8b80 1 Processor -- start 2023-11-06T22:00:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.474+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215d1080 0x55d821614020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.474+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8215f94c0 0x55d8215ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.474+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d821619be0 0x55d8215cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.474+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d821522ba0 con 0x55d821619be0 2023-11-06T22:00:24.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.474+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8214ede60 con 0x55d8215f94c0 2023-11-06T22:00:24.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.474+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8215fcbb0 con 0x55d8215d1080 2023-11-06T22:00:24.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.474+0000 7f5f039a8b80 10 client.0 ll_register_callbacks cb 0x55d8214c2ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T22:00:24.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eeab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d821619be0 0x55d8215cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eeab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d821619be0 0x55d8215cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45192/0 (socket says 172.21.15.33:45192) 2023-11-06T22:00:24.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eeab9c700 1 -- 172.21.15.33:0/3880314 learned_addr learned my addr 172.21.15.33:0/3880314 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:24.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5ee9b9a700 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8215f94c0 0x55d8215ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eea39b700 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215d1080 0x55d821614020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eeab9c700 1 -- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215d1080 msgr2=0x55d821614020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:24.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215d1080 0x55d821614020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eeab9c700 1 -- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8215f94c0 msgr2=0x55d8215ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:24.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8215f94c0 0x55d8215ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eeab9c700 1 -- 172.21.15.33:0/3880314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee000f040 con 0x55d821619be0 2023-11-06T22:00:24.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eea39b700 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215d1080 0x55d821614020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:00:24.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d821619be0 0x55d8215cd3b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f5edc0506b0 tx=0x7f5edc02a040).ready entity=mon.0 client_cookie=ac922bdeaeb64805 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:24.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5ec49f9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T22:00:24.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5ec49f9700 1 -- 172.21.15.33:0/3880314 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc04c570 con 0x55d821619be0 2023-11-06T22:00:24.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5ec49f9700 1 -- 172.21.15.33:0/3880314 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f5edc04c6d0 con 0x55d821619be0 2023-11-06T22:00:24.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.475+0000 7f5f039a8b80 1 -- 172.21.15.33:0/3880314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8215c4e20 con 0x55d821619be0 2023-11-06T22:00:24.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5ec49f9700 1 -- 172.21.15.33:0/3880314 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5edc04c890 con 0x55d821619be0 2023-11-06T22:00:24.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5f039a8b80 10 client.9012 fetch_fsmap learned FSMap version 76 2023-11-06T22:00:24.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5f039a8b80 1 -- 172.21.15.33:0/3880314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55d821516ac0 con 0x55d821619be0 2023-11-06T22:00:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5ec49f9700 1 -- 172.21.15.33:0/3880314 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8215c4e20 con 0x55d821619be0 2023-11-06T22:00:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5ec49f9700 1 -- 172.21.15.33:0/3880314 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 76) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5edc04f8f0 con 0x55d821619be0 2023-11-06T22:00:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5f039a8b80 10 client.9012 fetch_fsmap finished waiting for FSMap version 76 2023-11-06T22:00:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5f039a8b80 10 client.9012 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T22:00:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5f039a8b80 20 client.9012 populate_metadata read hostname 'smithi033' 2023-11-06T22:00:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:6834/249801946,v1:172.21.15.69:6836/249801946] conn(0x55d8215fa270 0x55d821625f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5f039a8b80 4 client.9012 mds_command: new command op to 8551 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T22:00:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5f039a8b80 1 -- 172.21.15.33:0/3880314 --> [v2:172.21.15.69:6834/249801946,v1:172.21.15.69:6836/249801946] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d82138aef0 con 0x55d8215fa270 2023-11-06T22:00:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.476+0000 7f5eea39b700 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:6834/249801946,v1:172.21.15.69:6836/249801946] conn(0x55d8215fa270 0x55d821625f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.477+0000 7f5eea39b700 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:6834/249801946,v1:172.21.15.69:6836/249801946] conn(0x55d8215fa270 0x55d821625f80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6b2d62263459872a server_cookie=b844e7a60677b230 in_seq=0 out_seq=0 2023-11-06T22:00:24.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.477+0000 7f5ec49f9700 10 client.9012 ms_handle_connect on v2:172.21.15.69:6834/249801946 2023-11-06T22:00:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5ec49f9700 1 -- 172.21.15.33:0/3880314 <== mds.0 v2:172.21.15.69:6834/249801946 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55d82138aef0 con 0x55d8215fa270 2023-11-06T22:00:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5ec49f9700 10 client.9012 handle_command_reply: tid=0 2023-11-06T22:00:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5f039a8b80 1 client.9012 shutdown 2023-11-06T22:00:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5f039a8b80 1 -- 172.21.15.33:0/3880314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d821619be0 msgr2=0x55d8215cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d821619be0 0x55d8215cd3b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f5edc0506b0 tx=0x7f5edc02a040).stop 2023-11-06T22:00:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5f039a8b80 1 -- 172.21.15.33:0/3880314 shutdown_connections 2023-11-06T22:00:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:6834/249801946,v1:172.21.15.69:6836/249801946] conn(0x55d8215fa270 0x55d821625f80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215d1080 0x55d821614020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d821619be0 0x55d8215cd3b0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5ec49f9700 0 client.9012 ms_handle_reset on v2:172.21.15.69:6834/249801946 2023-11-06T22:00:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/3880314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8215f94c0 0x55d8215ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.478+0000 7f5f039a8b80 1 -- 172.21.15.33:0/3880314 >> 172.21.15.33:0/3880314 conn(0x55d82143a940 msgr2=0x55d8215cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.479+0000 7f5f039a8b80 1 -- 172.21.15.33:0/3880314 shutdown_connections 2023-11-06T22:00:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.479+0000 7f5f039a8b80 1 -- 172.21.15.33:0/3880314 wait complete. 2023-11-06T22:00:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.479+0000 7f5f039a8b80 20 client.9012 trim_cache size 0 max 16384 2023-11-06T22:00:24.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.496+0000 7f5f039a8b80 1 Processor -- start 2023-11-06T22:00:24.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.496+0000 7f5f039a8b80 1 -- start start 2023-11-06T22:00:24.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821613b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d821614060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8215f51d0 con 0x55d8215d1080 2023-11-06T22:00:24.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d821619a10 con 0x55d821619be0 2023-11-06T22:00:24.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d82161aa30 con 0x55d8215f94c0 2023-11-06T22:00:24.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5eea39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821613b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5eea39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821613b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45202/0 (socket says 172.21.15.33:45202) 2023-11-06T22:00:24.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5eea39b700 1 -- 172.21.15.33:0/2383148502 learned_addr learned my addr 172.21.15.33:0/2383148502 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:24.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5ee9b9a700 1 --2- 172.21.15.33:0/2383148502 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d821614060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/2383148502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5eea39b700 1 -- 172.21.15.33:0/2383148502 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 msgr2=0x55d821614060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:24.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5eea39b700 1 --2- 172.21.15.33:0/2383148502 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d821614060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5eea39b700 1 -- 172.21.15.33:0/2383148502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 msgr2=0x55d8215f87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:24.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5eea39b700 1 --2- 172.21.15.33:0/2383148502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215f87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5eea39b700 1 -- 172.21.15.33:0/2383148502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8215cdf10 con 0x55d8215d1080 2023-11-06T22:00:24.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.497+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/2383148502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:00:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5eea39b700 1 --2- 172.21.15.33:0/2383148502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821613b70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f5ee0018c70 tx=0x7f5ee0016000).ready entity=mon.0 client_cookie=b375d7692d4dc49f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5ec79ff700 1 -- 172.21.15.33:0/2383148502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0029040 con 0x55d8215d1080 2023-11-06T22:00:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5ec79ff700 1 -- 172.21.15.33:0/2383148502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee00154b0 con 0x55d8215d1080 2023-11-06T22:00:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5ec79ff700 1 -- 172.21.15.33:0/2383148502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee002d280 con 0x55d8215d1080 2023-11-06T22:00:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5f039a8b80 1 -- 172.21.15.33:0/2383148502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 msgr2=0x55d821613b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/2383148502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821613b70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f5ee0018c70 tx=0x7f5ee0016000).stop 2023-11-06T22:00:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5f039a8b80 1 -- 172.21.15.33:0/2383148502 shutdown_connections 2023-11-06T22:00:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/2383148502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/2383148502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821613b70 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/2383148502 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d821614060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5f039a8b80 1 -- 172.21.15.33:0/2383148502 >> 172.21.15.33:0/2383148502 conn(0x55d82143a940 msgr2=0x55d8215d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.498+0000 7f5f039a8b80 1 -- 172.21.15.33:0/2383148502 shutdown_connections 2023-11-06T22:00:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.499+0000 7f5f039a8b80 1 -- 172.21.15.33:0/2383148502 wait complete. 2023-11-06T22:00:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.499+0000 7f5f039a8b80 1 -- start start 2023-11-06T22:00:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.499+0000 7f5f039a8b80 1 Processor -- start 2023-11-06T22:00:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.500+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821678930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.500+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d8215cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.500+0000 7f5f039a8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.500+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8215ccbe0 con 0x55d8215d1080 2023-11-06T22:00:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.500+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d82167ba60 con 0x55d821619be0 2023-11-06T22:00:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.500+0000 7f5f039a8b80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8215f8410 con 0x55d8215f94c0 2023-11-06T22:00:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.500+0000 7f5f039a8b80 10 client.0 ll_register_callbacks cb 0x55d8214c2ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T22:00:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5eea39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821678930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5eea39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821678930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45216/0 (socket says 172.21.15.33:45216) 2023-11-06T22:00:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5eea39b700 1 -- 172.21.15.33:0/913764658 learned_addr learned my addr 172.21.15.33:0/913764658 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5eeab9c700 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5ee9b9a700 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d8215cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5eea39b700 1 -- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 msgr2=0x55d8215cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5eea39b700 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d8215cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5eea39b700 1 -- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 msgr2=0x55d8215cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5eea39b700 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5eea39b700 1 -- 172.21.15.33:0/913764658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d82151b720 con 0x55d8215d1080 2023-11-06T22:00:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5eea39b700 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821678930 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f5ee000e1c0 tx=0x7f5ee002a000).ready entity=mon.0 client_cookie=ed5c8283f38b0bc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5ec49f9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T22:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.501+0000 7f5ec49f9700 1 -- 172.21.15.33:0/913764658 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0029070 con 0x55d8215d1080 2023-11-06T22:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5ec49f9700 1 -- 172.21.15.33:0/913764658 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f5ee002e760 con 0x55d8215d1080 2023-11-06T22:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5f039a8b80 1 -- 172.21.15.33:0/913764658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ee00183a0 con 0x55d8215d1080 2023-11-06T22:00:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5ec49f9700 1 -- 172.21.15.33:0/913764658 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee002eb30 con 0x55d8215d1080 2023-11-06T22:00:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5f039a8b80 10 client.9018 fetch_fsmap learned FSMap version 76 2023-11-06T22:00:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5f039a8b80 1 -- 172.21.15.33:0/913764658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x7f5ee4064bc0 con 0x55d8215d1080 2023-11-06T22:00:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5ec49f9700 1 -- 172.21.15.33:0/913764658 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee00183a0 con 0x55d8215d1080 2023-11-06T22:00:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5ec49f9700 1 -- 172.21.15.33:0/913764658 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 76) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5ee0020020 con 0x55d8215d1080 2023-11-06T22:00:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5f039a8b80 10 client.9018 fetch_fsmap finished waiting for FSMap version 76 2023-11-06T22:00:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5f039a8b80 10 client.9018 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T22:00:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5f039a8b80 20 client.9018 populate_metadata read hostname 'smithi033' 2023-11-06T22:00:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:6834/249801946,v1:172.21.15.69:6836/249801946] conn(0x55d8215f7740 0x55d8215f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5f039a8b80 4 client.9018 mds_command: new command op to 8551 tid=0[{"prefix": "status"}] 2023-11-06T22:00:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.502+0000 7f5f039a8b80 1 -- 172.21.15.33:0/913764658 --> [v2:172.21.15.69:6834/249801946,v1:172.21.15.69:6836/249801946] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d8213c0040 con 0x55d8215f7740 2023-11-06T22:00:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.503+0000 7f5ee9b9a700 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:6834/249801946,v1:172.21.15.69:6836/249801946] conn(0x55d8215f7740 0x55d8215f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.503+0000 7f5ee9b9a700 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:6834/249801946,v1:172.21.15.69:6836/249801946] conn(0x55d8215f7740 0x55d8215f7b20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=3a66c118f6845b6e server_cookie=b64e57efc9a80391 in_seq=0 out_seq=0 2023-11-06T22:00:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.503+0000 7f5ec49f9700 10 client.9018 ms_handle_connect on v2:172.21.15.69:6834/249801946 2023-11-06T22:00:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.503+0000 7f5ec49f9700 1 -- 172.21.15.33:0/913764658 <== mds.0 v2:172.21.15.69:6834/249801946 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55d8213c0040 con 0x55d8215f7740 2023-11-06T22:00:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.503+0000 7f5ec49f9700 10 client.9018 handle_command_reply: tid=0 2023-11-06T22:00:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.503+0000 7f5f039a8b80 1 client.9018 shutdown 2023-11-06T22:00:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 1 -- 172.21.15.33:0/913764658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 msgr2=0x55d821678930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821678930 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f5ee000e1c0 tx=0x7f5ee002a000).stop 2023-11-06T22:00:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 1 -- 172.21.15.33:0/913764658 shutdown_connections 2023-11-06T22:00:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:6834/249801946,v1:172.21.15.69:6836/249801946] conn(0x55d8215f7740 0x55d8215f7b20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d821619be0 0x55d8215cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8215d1080 0x55d821678930 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 1 --2- 172.21.15.33:0/913764658 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x55d8215f94c0 0x55d8215cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5ec49f9700 0 client.9018 ms_handle_reset on v2:172.21.15.69:6834/249801946 2023-11-06T22:00:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 1 -- 172.21.15.33:0/913764658 >> 172.21.15.33:0/913764658 conn(0x55d82143a940 msgr2=0x55d8215fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:24.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 1 -- 172.21.15.33:0/913764658 shutdown_connections 2023-11-06T22:00:24.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 1 -- 172.21.15.33:0/913764658 wait complete. 2023-11-06T22:00:24.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5f039a8b80 20 client.9018 trim_cache size 0 max 16384 2023-11-06T22:00:24.523 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:00:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T22:00:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T22:00:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 8551, 2023-11-06T22:00:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T22:00:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T22:00:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T22:00:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.4301155169999999, 2023-11-06T22:00:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 76, 2023-11-06T22:00:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 185, 2023-11-06T22:00:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 185, 2023-11-06T22:00:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.649138218999999 2023-11-06T22:00:24.525 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:00:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5eec5ff700 1 -- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ec805f270 msgr2=0x7f5ec8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5eec5ff700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ec805f270 0x7f5ec8061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5ee0000c00 tx=0x7f5ee0009040).stop 2023-11-06T22:00:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5eec5ff700 1 -- 172.21.15.33:0/2639296680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee4154210 msgr2=0x7f5ee4154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.504+0000 7f5eec5ff700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee4154210 0x7f5ee4154ae0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f5edc00bec0 tx=0x7f5edc009840).stop 2023-11-06T22:00:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.505+0000 7f5eec5ff700 1 -- 172.21.15.33:0/2639296680 shutdown_connections 2023-11-06T22:00:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.505+0000 7f5eec5ff700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f5ec805f270 0x7f5ec8061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.505+0000 7f5eec5ff700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee40bfd60 0x7f5ee413dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.505+0000 7f5eec5ff700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee4154210 0x7f5ee4154ae0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.505+0000 7f5eec5ff700 1 --2- 172.21.15.33:0/2639296680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee40c6d90 0x7f5ee41545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.505+0000 7f5eec5ff700 1 -- 172.21.15.33:0/2639296680 >> 172.21.15.33:0/2639296680 conn(0x7f5ee405e990 msgr2=0x7f5ee40c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.505+0000 7f5eec5ff700 1 -- 172.21.15.33:0/2639296680 shutdown_connections 2023-11-06T22:00:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.505+0000 7f5eec5ff700 1 -- 172.21.15.33:0/2639296680 wait complete. 2023-11-06T22:00:24.529 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T22:00:24.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.691+0000 7f7d03dff700 1 Processor -- start 2023-11-06T22:00:24.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.691+0000 7f7d03dff700 1 -- start start 2023-11-06T22:00:24.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.693+0000 7f7d03dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc0c63c0 0x7f7cfc0be6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.693+0000 7f7d03dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0c5f30 con 0x7f7cfc155760 2023-11-06T22:00:24.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.693+0000 7f7d03dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0a8440 con 0x7f7cfc1543d0 2023-11-06T22:00:24.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.693+0000 7f7d03dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0c5600 con 0x7f7cfc0c63c0 2023-11-06T22:00:24.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.693+0000 7f7d0239c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7cfc155760 0x7f7cfc064280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52038/0 (socket says 172.21.15.33:52038) 2023-11-06T22:00:24.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.693+0000 7f7d0239c700 1 -- 172.21.15.33:0/3957512982 learned_addr learned my addr 172.21.15.33:0/3957512982 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:00:24.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.693+0000 7f7d0139a700 1 --2- 172.21.15.33:0/3957512982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc0c63c0 0x7f7cfc0be6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d00b99700 1 -- 172.21.15.33:0/3957512982 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221147771 0 0) 0x7f7cfc0c5f30 con 0x7f7cfc155760 2023-11-06T22:00:24.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d00b99700 1 -- 172.21.15.33:0/3957512982 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce80033f0 con 0x7f7cfc155760 2023-11-06T22:00:24.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d00b99700 1 -- 172.21.15.33:0/3957512982 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473537619 0 0) 0x7f7cfc0a8440 con 0x7f7cfc1543d0 2023-11-06T22:00:24.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d00b99700 1 -- 172.21.15.33:0/3957512982 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cfc0c5f30 con 0x7f7cfc1543d0 2023-11-06T22:00:24.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d0139a700 1 -- 172.21.15.33:0/3957512982 >> v1:172.21.15.69:6789/0 conn(0x7f7cfc1543d0 legacy=0x7f7cfc0a7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d0139a700 1 -- 172.21.15.33:0/3957512982 >> v1:172.21.15.33:6789/0 conn(0x7f7cfc155760 legacy=0x7f7cfc064280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d0139a700 1 -- 172.21.15.33:0/3957512982 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0c58e0 con 0x7f7cfc0c63c0 2023-11-06T22:00:24.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d0139a700 1 --2- 172.21.15.33:0/3957512982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc0c63c0 0x7f7cfc0be6d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f7cec009790 tx=0x7f7cec009060).ready entity=mon.2 client_cookie=cb6999f1d6513a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:24.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d00b99700 1 -- 172.21.15.33:0/3957512982 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec00f510 con 0x7f7cfc0c63c0 2023-11-06T22:00:24.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d00b99700 1 -- 172.21.15.33:0/3957512982 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec00f670 con 0x7f7cfc0c63c0 2023-11-06T22:00:24.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.694+0000 7f7d00b99700 1 -- 172.21.15.33:0/3957512982 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec00f9f0 con 0x7f7cfc0c63c0 2023-11-06T22:00:24.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.695+0000 7f7d03dff700 1 -- 172.21.15.33:0/3957512982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc0c63c0 msgr2=0x7f7cfc0be6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.695+0000 7f7d03dff700 1 --2- 172.21.15.33:0/3957512982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc0c63c0 0x7f7cfc0be6d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f7cec009790 tx=0x7f7cec009060).stop 2023-11-06T22:00:24.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.695+0000 7f7d03dff700 1 -- 172.21.15.33:0/3957512982 shutdown_connections 2023-11-06T22:00:24.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.695+0000 7f7d03dff700 1 --2- 172.21.15.33:0/3957512982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc0c63c0 0x7f7cfc0be6d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.695+0000 7f7d03dff700 1 -- 172.21.15.33:0/3957512982 >> 172.21.15.33:0/3957512982 conn(0x7f7cfc092300 msgr2=0x7f7cfc062ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:24.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.695+0000 7f7d03dff700 1 -- 172.21.15.33:0/3957512982 shutdown_connections 2023-11-06T22:00:24.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.695+0000 7f7d03dff700 1 -- 172.21.15.33:0/3957512982 wait complete. 2023-11-06T22:00:24.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.696+0000 7f7d03dff700 1 Processor -- start 2023-11-06T22:00:24.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.697+0000 7f7d03dff700 1 -- start start 2023-11-06T22:00:24.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.697+0000 7f7d03dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cfc0c63c0 0x7f7cfc0ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.697+0000 7f7d03dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc1543d0 0x7f7cfc168020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.697+0000 7f7d03dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc155760 0x7f7cfc0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.697+0000 7f7d03dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0aa9a0 con 0x7f7cfc155760 2023-11-06T22:00:24.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.697+0000 7f7d03dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0aa0b0 con 0x7f7cfc0c63c0 2023-11-06T22:00:24.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.697+0000 7f7d03dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0aa350 con 0x7f7cfc1543d0 2023-11-06T22:00:24.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0239c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc155760 0x7f7cfc0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0239c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc155760 0x7f7cfc0ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45230/0 (socket says 172.21.15.33:45230) 2023-11-06T22:00:24.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0239c700 1 -- 172.21.15.33:0/3398384644 learned_addr learned my addr 172.21.15.33:0/3398384644 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:24.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d01b9b700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cfc0c63c0 0x7f7cfc0ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0139a700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc1543d0 0x7f7cfc168020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0239c700 1 -- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc1543d0 msgr2=0x7f7cfc168020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0239c700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc1543d0 0x7f7cfc168020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0239c700 1 -- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cfc0c63c0 msgr2=0x7f7cfc0ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0239c700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cfc0c63c0 0x7f7cfc0ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0239c700 1 -- 172.21.15.33:0/3398384644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec0092a0 con 0x7f7cfc155760 2023-11-06T22:00:24.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0139a700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc1543d0 0x7f7cfc168020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:00:24.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.698+0000 7f7d0239c700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc155760 0x7f7cfc0ae2c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f7cf4001560 tx=0x7f7cf40015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:24.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.699+0000 7f7cf1ffb700 1 -- 172.21.15.33:0/3398384644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4005d30 con 0x7f7cfc155760 2023-11-06T22:00:24.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.699+0000 7f7cf1ffb700 1 -- 172.21.15.33:0/3398384644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf4005e90 con 0x7f7cfc155760 2023-11-06T22:00:24.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.699+0000 7f7cf1ffb700 1 -- 172.21.15.33:0/3398384644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4007340 con 0x7f7cfc155760 2023-11-06T22:00:24.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.699+0000 7f7d03dff700 1 -- 172.21.15.33:0/3398384644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc0aa630 con 0x7f7cfc155760 2023-11-06T22:00:24.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.699+0000 7f7d03dff700 1 -- 172.21.15.33:0/3398384644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc151150 con 0x7f7cfc155760 2023-11-06T22:00:24.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.700+0000 7f7d03dff700 1 -- 172.21.15.33:0/3398384644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc0aab10 con 0x7f7cfc155760 2023-11-06T22:00:24.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.701+0000 7f7cf1ffb700 1 -- 172.21.15.33:0/3398384644 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7cf40076b0 con 0x7f7cfc155760 2023-11-06T22:00:24.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.701+0000 7f7cf1ffb700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ce005f270 0x7f7ce0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:24.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.702+0000 7f7cf1ffb700 1 -- 172.21.15.33:0/3398384644 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f7cf407a3d0 con 0x7f7cfc155760 2023-11-06T22:00:24.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.702+0000 7f7d01b9b700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ce005f270 0x7f7ce0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:24.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.702+0000 7f7d01b9b700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ce005f270 0x7f7ce0061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f7cf8003920 tx=0x7f7cf8007b90).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:24.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.705+0000 7f7cf1ffb700 1 -- 172.21.15.33:0/3398384644 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7cf404b3d0 con 0x7f7cfc155760 2023-11-06T22:00:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.929+0000 7f7d03dff700 1 -- 172.21.15.33:0/3398384644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7cfc0475a0 con 0x7f7cfc155760 2023-11-06T22:00:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.930+0000 7f7cf1ffb700 1 -- 172.21.15.33:0/3398384644 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7cf404e9f0 con 0x7f7cfc155760 2023-11-06T22:00:24.932 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T22:00:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 -- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ce005f270 msgr2=0x7f7ce0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ce005f270 0x7f7ce0061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f7cf8003920 tx=0x7f7cf8007b90).stop 2023-11-06T22:00:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 -- 172.21.15.33:0/3398384644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc155760 msgr2=0x7f7cfc0ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc155760 0x7f7cfc0ae2c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f7cf4001560 tx=0x7f7cf40015b0).stop 2023-11-06T22:00:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 -- 172.21.15.33:0/3398384644 shutdown_connections 2023-11-06T22:00:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f7ce005f270 0x7f7ce0061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc155760 0x7f7cfc0ae2c0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cfc0c63c0 0x7f7cfc0ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 --2- 172.21.15.33:0/3398384644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7cfc1543d0 0x7f7cfc168020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:24.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 -- 172.21.15.33:0/3398384644 >> 172.21.15.33:0/3398384644 conn(0x7f7cfc092300 msgr2=0x7f7cfc1547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:24.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 -- 172.21.15.33:0/3398384644 shutdown_connections 2023-11-06T22:00:24.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:24.932+0000 7f7d03dff700 1 -- 172.21.15.33:0/3398384644 wait complete. 2023-11-06T22:00:24.949 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T22:00:24.949 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T22:00:24.950 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T22:00:24.950 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T22:00:24.950 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T22:00:24.951 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T22:00:24.951 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T22:00:24.951 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T22:00:24.952 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T22:00:24.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:00:24.952 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T22:00:25.001 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T22:00:25.001 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T22:00:25.001 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T22:00:25.002 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:00:25.002 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T22:00:25.002 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:00:25.002 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T22:00:25.002 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:00:25.003 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T22:00:25.003 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T22:00:25.003 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:00:25.003 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T22:00:25.003 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T22:00:25.004 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:00:25.004 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T22:00:25.004 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T22:00:25.004 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 37719sec preferred_lft 37719sec 2023-11-06T22:00:25.004 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T22:00:25.005 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:00:25.005 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T22:00:25.005 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:00:25.005 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T22:00:25.005 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T22:00:25.006 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T22:00:25.006 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T22:00:25.006 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:00:25.006 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T22:00:25.006 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:00:25.007 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T22:00:25.007 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T22:00:25.007 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T22:00:25.007 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:00:25.007 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T22:00:25.008 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T22:00:25.008 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T22:00:25.008 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:00:25.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:00:25.009 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T22:00:25.079 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T22:00:25.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:00:25.083 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T22:00:25.134 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T22:00:25.134 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T22:00:25.134 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T22:00:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T22:00:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T22:00:25.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:00:25.136 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T22:00:25.137 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T22:00:25.137 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T22:00:25.137 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T22:00:25.138 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T22:00:25.138 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T22:00:25.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:00:25.313 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T22:00:25.361 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T22:00:25.361 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T22:00:25.362 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T22:00:25.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:00:25.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T22:00:25.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:00:25.437 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T22:00:25.485 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T22:00:25.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T22:00:25.587 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T22:00:25.588 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T22:00:25.615 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T22:00:25.615 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T22:00:25.620 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T22:00:25.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:00:25.621 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T22:00:25.687 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:00:25.692 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:00:25.694 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T22:00:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.930+0000 7fc31a3ff700 1 Processor -- start 2023-11-06T22:00:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.930+0000 7fc31a3ff700 1 -- start start 2023-11-06T22:00:25.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.932+0000 7fc31a3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc31413ce50 0x7fc31413d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:25.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.932+0000 7fc31a3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc3141586f0 0x7fc3140c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.932+0000 7fc31a3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3140c4f00 con 0x7fc3140c4560 2023-11-06T22:00:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.932+0000 7fc31a3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc314158d50 con 0x7fc31413ce50 2023-11-06T22:00:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.932+0000 7fc31a3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc314158af0 con 0x7fc3141586f0 2023-11-06T22:00:25.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.932+0000 7fc313fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc31413ce50 0x7fc31413d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.932+0000 7fc313fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc31413ce50 0x7fc31413d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45240/0 (socket says 172.21.15.33:45240) 2023-11-06T22:00:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.932+0000 7fc313fff700 1 -- 172.21.15.33:0/2035681398 learned_addr learned my addr 172.21.15.33:0/2035681398 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.932+0000 7fc3137fe700 1 --1- 172.21.15.33:0/2035681398 >> v1:172.21.15.33:6789/0 conn(0x7fc3140c4560 0x7fc314157fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52046/0 (socket says 172.21.15.33:52046) 2023-11-06T22:00:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.932+0000 7fc31899c700 1 --2- 172.21.15.33:0/2035681398 >> v2:172.21.15.69:3300/0 conn(0x7fc3141586f0 0x7fc3140c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc313fff700 1 -- 172.21.15.33:0/2035681398 >> v2:172.21.15.69:3300/0 conn(0x7fc3141586f0 msgr2=0x7fc3140c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc313fff700 1 --2- 172.21.15.33:0/2035681398 >> v2:172.21.15.69:3300/0 conn(0x7fc3141586f0 0x7fc3140c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc313fff700 1 -- 172.21.15.33:0/2035681398 >> v1:172.21.15.33:6789/0 conn(0x7fc3140c4560 legacy=0x7fc314157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc313fff700 1 -- 172.21.15.33:0/2035681398 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc314062840 con 0x7fc31413ce50 2023-11-06T22:00:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc312ffd700 1 -- 172.21.15.33:0/2035681398 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387331620 0 0) 0x7fc308001110 con 0x7fc3140c4560 2023-11-06T22:00:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc313fff700 1 --2- 172.21.15.33:0/2035681398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc31413ce50 0x7fc31413d230 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7fc31406ad70 tx=0x7fc3040090f0).ready entity=mon.0 client_cookie=506b6fa2392f83ea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc312ffd700 1 -- 172.21.15.33:0/2035681398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30400b600 con 0x7fc31413ce50 2023-11-06T22:00:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc312ffd700 1 -- 172.21.15.33:0/2035681398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30400b760 con 0x7fc31413ce50 2023-11-06T22:00:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc312ffd700 1 -- 172.21.15.33:0/2035681398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30400bae0 con 0x7fc31413ce50 2023-11-06T22:00:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2035681398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc31413ce50 msgr2=0x7fc31413d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.933+0000 7fc31a3ff700 1 --2- 172.21.15.33:0/2035681398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc31413ce50 0x7fc31413d230 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7fc31406ad70 tx=0x7fc3040090f0).stop 2023-11-06T22:00:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.934+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2035681398 shutdown_connections 2023-11-06T22:00:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.934+0000 7fc31a3ff700 1 --2- 172.21.15.33:0/2035681398 >> v2:172.21.15.69:3300/0 conn(0x7fc3141586f0 0x7fc3140c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.934+0000 7fc31a3ff700 1 --2- 172.21.15.33:0/2035681398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc31413ce50 0x7fc31413d230 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.934+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2035681398 >> 172.21.15.33:0/2035681398 conn(0x7fc3140c7dc0 msgr2=0x7fc3140c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:25.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.934+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2035681398 shutdown_connections 2023-11-06T22:00:25.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.934+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2035681398 wait complete. 2023-11-06T22:00:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.935+0000 7fc31a3ff700 1 Processor -- start 2023-11-06T22:00:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.935+0000 7fc31a3ff700 1 -- start start 2023-11-06T22:00:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.936+0000 7fc31a3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140c4560 0x7fc314153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.936+0000 7fc31a3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc31413ce50 0x7fc3141578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.936+0000 7fc31a3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3141586f0 0x7fc314157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:25.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.936+0000 7fc31a3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc314158280 con 0x7fc3141586f0 2023-11-06T22:00:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.936+0000 7fc31a3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3140a1aa0 con 0x7fc31413ce50 2023-11-06T22:00:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.936+0000 7fc31a3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3140a14e0 con 0x7fc3140c4560 2023-11-06T22:00:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc31899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3141586f0 0x7fc314157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc31899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3141586f0 0x7fc314157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45246/0 (socket says 172.21.15.33:45246) 2023-11-06T22:00:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc31899c700 1 -- 172.21.15.33:0/2385886611 learned_addr learned my addr 172.21.15.33:0/2385886611 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc3137fe700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc31413ce50 0x7fc3141578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc313fff700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140c4560 0x7fc314153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc31899c700 1 -- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140c4560 msgr2=0x7fc314153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc31899c700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140c4560 0x7fc314153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:25.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc31899c700 1 -- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc31413ce50 msgr2=0x7fc3141578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc31899c700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc31413ce50 0x7fc3141578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc31899c700 1 -- 172.21.15.33:0/2385886611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304009370 con 0x7fc3141586f0 2023-11-06T22:00:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc31899c700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3141586f0 0x7fc314157d90 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7fc30c007f20 tx=0x7fc30c009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc2fbfff700 1 -- 172.21.15.33:0/2385886611 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00eb90 con 0x7fc3141586f0 2023-11-06T22:00:25.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc2fbfff700 1 -- 172.21.15.33:0/2385886611 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c00ecf0 con 0x7fc3141586f0 2023-11-06T22:00:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.937+0000 7fc2fbfff700 1 -- 172.21.15.33:0/2385886611 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c014610 con 0x7fc3141586f0 2023-11-06T22:00:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.938+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2385886611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3140a17c0 con 0x7fc3141586f0 2023-11-06T22:00:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.938+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2385886611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc314154e90 con 0x7fc3141586f0 2023-11-06T22:00:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.939+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2385886611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc314155080 con 0x7fc3141586f0 2023-11-06T22:00:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.939+0000 7fc2fbfff700 1 -- 172.21.15.33:0/2385886611 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc30c014980 con 0x7fc3141586f0 2023-11-06T22:00:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.940+0000 7fc2fbfff700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc30005f270 0x7fc300061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.940+0000 7fc2fbfff700 1 -- 172.21.15.33:0/2385886611 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fc30c07a840 con 0x7fc3141586f0 2023-11-06T22:00:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.941+0000 7fc313fff700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc30005f270 0x7fc300061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.941+0000 7fc313fff700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc30005f270 0x7fc300061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fc304009130 tx=0x7fc304017000).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:25.944+0000 7fc2fbfff700 1 -- 172.21.15.33:0/2385886611 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc30c04b8a0 con 0x7fc3141586f0 2023-11-06T22:00:26.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.155+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2385886611 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc314062b20 con 0x7fc30005f270 2023-11-06T22:00:26.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.158+0000 7fc2fbfff700 1 -- 172.21.15.33:0/2385886611 <== mgr.7352 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc314062b20 con 0x7fc30005f270 2023-11-06T22:00:26.159 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T22:00:26.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:00:26.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T22:00:26.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:00:26.160 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T22:00:26.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc30005f270 msgr2=0x7fc300061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc30005f270 0x7fc300061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fc304009130 tx=0x7fc304017000).stop 2023-11-06T22:00:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2385886611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3141586f0 msgr2=0x7fc314157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3141586f0 0x7fc314157d90 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7fc30c007f20 tx=0x7fc30c009860).stop 2023-11-06T22:00:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2385886611 shutdown_connections 2023-11-06T22:00:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fc30005f270 0x7fc300061730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc3140c4560 0x7fc314153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3141586f0 0x7fc314157d90 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 --2- 172.21.15.33:0/2385886611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc31413ce50 0x7fc3141578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2385886611 >> 172.21.15.33:0/2385886611 conn(0x7fc3140c7dc0 msgr2=0x7fc3140c52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2385886611 shutdown_connections 2023-11-06T22:00:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.159+0000 7fc31a3ff700 1 -- 172.21.15.33:0/2385886611 wait complete. 2023-11-06T22:00:26.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T22:00:26.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.375+0000 7fd4b73ff700 1 Processor -- start 2023-11-06T22:00:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.375+0000 7fd4b73ff700 1 -- start start 2023-11-06T22:00:26.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.376+0000 7fd4b73ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4b0155410 0x7fd4b00644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:26.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.376+0000 7fd4b73ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd4b01547d0 0x7fd4b00649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:26.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b73ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b0063c00 con 0x7fd4b0155030 2023-11-06T22:00:26.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b73ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b0064090 con 0x7fd4b0155410 2023-11-06T22:00:26.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b73ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b0064ed0 con 0x7fd4b01547d0 2023-11-06T22:00:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4b0155410 0x7fd4b00644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4b0155410 0x7fd4b00644f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45260/0 (socket says 172.21.15.33:45260) 2023-11-06T22:00:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b499a700 1 -- 172.21.15.33:0/1722947102 learned_addr learned my addr 172.21.15.33:0/1722947102 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b599c700 1 --2- 172.21.15.33:0/1722947102 >> v2:172.21.15.69:3300/0 conn(0x7fd4b01547d0 0x7fd4b00649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b499a700 1 -- 172.21.15.33:0/1722947102 >> v2:172.21.15.69:3300/0 conn(0x7fd4b01547d0 msgr2=0x7fd4b00649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b499a700 1 --2- 172.21.15.33:0/1722947102 >> v2:172.21.15.69:3300/0 conn(0x7fd4b01547d0 0x7fd4b00649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b499a700 1 -- 172.21.15.33:0/1722947102 >> v1:172.21.15.33:6789/0 conn(0x7fd4b0155030 legacy=0x7fd4b0062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4b499a700 1 -- 172.21.15.33:0/1722947102 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b0062ff0 con 0x7fd4b0155410 2023-11-06T22:00:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.377+0000 7fd4a7fff700 1 -- 172.21.15.33:0/1722947102 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543140811 0 0) 0x7fd4b0063c00 con 0x7fd4b0155030 2023-11-06T22:00:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4b499a700 1 --2- 172.21.15.33:0/1722947102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 0x7fd4b00644f0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7fd4b0152250 tx=0x7fd4a00090f0).ready entity=mon.0 client_cookie=b2afba7713398fe0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4a7fff700 1 -- 172.21.15.33:0/1722947102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000b600 con 0x7fd4b0155410 2023-11-06T22:00:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4a7fff700 1 -- 172.21.15.33:0/1722947102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a000b760 con 0x7fd4b0155410 2023-11-06T22:00:26.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4a7fff700 1 -- 172.21.15.33:0/1722947102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000bae0 con 0x7fd4b0155410 2023-11-06T22:00:26.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4b73ff700 1 -- 172.21.15.33:0/1722947102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 msgr2=0x7fd4b00644f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:26.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4b73ff700 1 --2- 172.21.15.33:0/1722947102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 0x7fd4b00644f0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7fd4b0152250 tx=0x7fd4a00090f0).stop 2023-11-06T22:00:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4b73ff700 1 -- 172.21.15.33:0/1722947102 shutdown_connections 2023-11-06T22:00:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4b73ff700 1 --2- 172.21.15.33:0/1722947102 >> v2:172.21.15.69:3300/0 conn(0x7fd4b01547d0 0x7fd4b00649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4b73ff700 1 --2- 172.21.15.33:0/1722947102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 0x7fd4b00644f0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4b73ff700 1 -- 172.21.15.33:0/1722947102 >> 172.21.15.33:0/1722947102 conn(0x7fd4b00c3d30 msgr2=0x7fd4b00c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.378+0000 7fd4b73ff700 1 -- 172.21.15.33:0/1722947102 shutdown_connections 2023-11-06T22:00:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.379+0000 7fd4b73ff700 1 -- 172.21.15.33:0/1722947102 wait complete. 2023-11-06T22:00:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.380+0000 7fd4b73ff700 1 Processor -- start 2023-11-06T22:00:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.380+0000 7fd4b73ff700 1 -- start start 2023-11-06T22:00:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.381+0000 7fd4b73ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4b01547d0 0x7fd4b00acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.381+0000 7fd4b73ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4b0155030 0x7fd4b013bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.381+0000 7fd4b73ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 0x7fd4b013bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.381+0000 7fd4b73ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b013c4e0 con 0x7fd4b0155410 2023-11-06T22:00:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.381+0000 7fd4b73ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0165b50 con 0x7fd4b0155030 2023-11-06T22:00:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.381+0000 7fd4b73ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0164b80 con 0x7fd4b01547d0 2023-11-06T22:00:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.381+0000 7fd4b599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 0x7fd4b013bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:26.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.381+0000 7fd4b599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 0x7fd4b013bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45264/0 (socket says 172.21.15.33:45264) 2023-11-06T22:00:26.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.382+0000 7fd4b599c700 1 -- 172.21.15.33:0/3860378777 learned_addr learned my addr 172.21.15.33:0/3860378777 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:26.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.382+0000 7fd4b519b700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4b01547d0 0x7fd4b00acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:26.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.382+0000 7fd4b499a700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4b0155030 0x7fd4b013bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.382+0000 7fd4b599c700 1 -- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4b01547d0 msgr2=0x7fd4b00acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.382+0000 7fd4b599c700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4b01547d0 0x7fd4b00acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.382+0000 7fd4b599c700 1 -- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4b0155030 msgr2=0x7fd4b013bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.382+0000 7fd4b599c700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4b0155030 0x7fd4b013bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.382+0000 7fd4b599c700 1 -- 172.21.15.33:0/3860378777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a0009370 con 0x7fd4b0155410 2023-11-06T22:00:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.382+0000 7fd4b599c700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 0x7fd4b013bff0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7fd4a8009fd0 tx=0x7fd4a8009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.382+0000 7fd4a57fa700 1 -- 172.21.15.33:0/3860378777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800eb90 con 0x7fd4b0155410 2023-11-06T22:00:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.383+0000 7fd4a57fa700 1 -- 172.21.15.33:0/3860378777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a800ecf0 con 0x7fd4b0155410 2023-11-06T22:00:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.383+0000 7fd4a57fa700 1 -- 172.21.15.33:0/3860378777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8014610 con 0x7fd4b0155410 2023-11-06T22:00:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.383+0000 7fd4b73ff700 1 -- 172.21.15.33:0/3860378777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b01656c0 con 0x7fd4b0155410 2023-11-06T22:00:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.383+0000 7fd4b73ff700 1 -- 172.21.15.33:0/3860378777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b013c6e0 con 0x7fd4b0155410 2023-11-06T22:00:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.384+0000 7fd4b73ff700 1 -- 172.21.15.33:0/3860378777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b0165850 con 0x7fd4b0155410 2023-11-06T22:00:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.385+0000 7fd4a57fa700 1 -- 172.21.15.33:0/3860378777 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 73) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd4a801d3b0 con 0x7fd4b0155410 2023-11-06T22:00:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.386+0000 7fd4a57fa700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd49405f270 0x7fd494061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.386+0000 7fd4a57fa700 1 -- 172.21.15.33:0/3860378777 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fd4a807a890 con 0x7fd4b0155410 2023-11-06T22:00:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.386+0000 7fd4b519b700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd49405f270 0x7fd494061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.387+0000 7fd4b519b700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd49405f270 0x7fd494061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fd4b00c13a0 tx=0x7fd4ac009040).ready entity=mgr.7352 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.390+0000 7fd4a57fa700 1 -- 172.21.15.33:0/3860378777 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd4a804b890 con 0x7fd4b0155410 2023-11-06T22:00:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:26.619+0000 7fd4b73ff700 1 -- 172.21.15.33:0/3860378777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fd4b006c080 con 0x7fd4b0155410 2023-11-06T22:00:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.107+0000 7fd4a57fa700 1 -- 172.21.15.33:0/3860378777 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 74) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fd4a80783f0 con 0x7fd4b0155410 2023-11-06T22:00:27.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.109+0000 7fd4a57fa700 1 -- 172.21.15.33:0/3860378777 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v74) v1 ==== 92+0+0 (secure 0 0 0) 0x7fd4a8073a80 con 0x7fd4b0155410 2023-11-06T22:00:27.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 -- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd49405f270 msgr2=0x7fd494061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd49405f270 0x7fd494061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fd4b00c13a0 tx=0x7fd4ac009040).stop 2023-11-06T22:00:27.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 -- 172.21.15.33:0/3860378777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 msgr2=0x7fd4b013bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 0x7fd4b013bff0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7fd4a8009fd0 tx=0x7fd4a8009900).stop 2023-11-06T22:00:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 -- 172.21.15.33:0/3860378777 shutdown_connections 2023-11-06T22:00:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd49405f270 0x7fd494061730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd4b01547d0 0x7fd4b00acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4b0155410 0x7fd4b013bff0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 --2- 172.21.15.33:0/3860378777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4b0155030 0x7fd4b013bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 -- 172.21.15.33:0/3860378777 >> 172.21.15.33:0/3860378777 conn(0x7fd4b00c3d30 msgr2=0x7fd4b00a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:27.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 -- 172.21.15.33:0/3860378777 shutdown_connections 2023-11-06T22:00:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.113+0000 7fd4b73ff700 1 -- 172.21.15.33:0/3860378777 wait complete. 2023-11-06T22:00:27.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T22:00:27.327 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:27.325+0000 7f9d5041e680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:00:27.333 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:27.332+0000 7fbf889e7680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:00:27.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.340+0000 7f44bb7ff700 1 Processor -- start 2023-11-06T22:00:27.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.341+0000 7f44bb7ff700 1 -- start start 2023-11-06T22:00:27.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44bb7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44b4154330 0x7f44b40c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44bb7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f44b4154710 0x7f44b40c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44bb7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f44b40c7ff0 0x7f44b40646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44bb7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b40c6b60 con 0x7f44b4154710 2023-11-06T22:00:27.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44bb7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b40c5cd0 con 0x7f44b40c7ff0 2023-11-06T22:00:27.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44bb7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40c5f50 con 0x7f44b4154330 2023-11-06T22:00:27.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44b8d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f44b4154710 0x7f44b40c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:27.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44b8d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f44b4154710 0x7f44b40c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45290/0 (socket says 172.21.15.33:45290) 2023-11-06T22:00:27.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44b8d9a700 1 -- 172.21.15.33:0/1035695925 learned_addr learned my addr 172.21.15.33:0/1035695925 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:27.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44b9d9c700 1 --2- 172.21.15.33:0/1035695925 >> v2:172.21.15.69:3300/0 conn(0x7f44b40c7ff0 0x7f44b40646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:27.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.342+0000 7f44b959b700 1 --2- 172.21.15.33:0/1035695925 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44b4154330 0x7f44b40c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:27.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.343+0000 7f44b8d9a700 1 -- 172.21.15.33:0/1035695925 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44b4154330 msgr2=0x7f44b40c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:27.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.343+0000 7f44b8d9a700 1 --2- 172.21.15.33:0/1035695925 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44b4154330 0x7f44b40c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.343+0000 7f44b8d9a700 1 -- 172.21.15.33:0/1035695925 >> v2:172.21.15.69:3300/0 conn(0x7f44b40c7ff0 msgr2=0x7f44b40646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:27.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.343+0000 7f44b8d9a700 1 --2- 172.21.15.33:0/1035695925 >> v2:172.21.15.69:3300/0 conn(0x7f44b40c7ff0 0x7f44b40646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.343+0000 7f44b8d9a700 1 -- 172.21.15.33:0/1035695925 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b40c6230 con 0x7f44b4154710 2023-11-06T22:00:27.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.343+0000 7f44b8d9a700 1 --2- 172.21.15.33:0/1035695925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 0x7f44b40c6ff0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f44b40c1e90 tx=0x7f44a40090f0).ready entity=mon.0 client_cookie=c964fcede4a245de server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:27.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.343+0000 7f44abfff700 1 -- 172.21.15.33:0/1035695925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400f510 con 0x7f44b4154710 2023-11-06T22:00:27.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.343+0000 7f44abfff700 1 -- 172.21.15.33:0/1035695925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a400f670 con 0x7f44b4154710 2023-11-06T22:00:27.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.343+0000 7f44abfff700 1 -- 172.21.15.33:0/1035695925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400f9f0 con 0x7f44b4154710 2023-11-06T22:00:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.344+0000 7f44bb7ff700 1 -- 172.21.15.33:0/1035695925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 msgr2=0x7f44b40c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.344+0000 7f44bb7ff700 1 --2- 172.21.15.33:0/1035695925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 0x7f44b40c6ff0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f44b40c1e90 tx=0x7f44a40090f0).stop 2023-11-06T22:00:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.344+0000 7f44bb7ff700 1 -- 172.21.15.33:0/1035695925 shutdown_connections 2023-11-06T22:00:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.344+0000 7f44bb7ff700 1 --2- 172.21.15.33:0/1035695925 >> v2:172.21.15.69:3300/0 conn(0x7f44b40c7ff0 0x7f44b40646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.344+0000 7f44bb7ff700 1 --2- 172.21.15.33:0/1035695925 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44b4154330 0x7f44b40c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.344+0000 7f44bb7ff700 1 --2- 172.21.15.33:0/1035695925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 0x7f44b40c6ff0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.344+0000 7f44bb7ff700 1 -- 172.21.15.33:0/1035695925 >> 172.21.15.33:0/1035695925 conn(0x7f44b40cb910 msgr2=0x7f44b40c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:27.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.344+0000 7f44bb7ff700 1 -- 172.21.15.33:0/1035695925 shutdown_connections 2023-11-06T22:00:27.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.344+0000 7f44bb7ff700 1 -- 172.21.15.33:0/1035695925 wait complete. 2023-11-06T22:00:27.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.346+0000 7f44bb7ff700 1 Processor -- start 2023-11-06T22:00:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.346+0000 7f44bb7ff700 1 -- start start 2023-11-06T22:00:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44bb7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b40c7ff0 0x7f44b41432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44bb7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44b4154330 0x7f44b41531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44bb7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 0x7f44b40ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44bb7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40aee70 con 0x7f44b4154710 2023-11-06T22:00:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44bb7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40aebf0 con 0x7f44b40c7ff0 2023-11-06T22:00:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44bb7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40c31a0 con 0x7f44b4154330 2023-11-06T22:00:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44b9d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 0x7f44b40ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44b9d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 0x7f44b40ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45304/0 (socket says 172.21.15.33:45304) 2023-11-06T22:00:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44b9d9c700 1 -- 172.21.15.33:0/3424156944 learned_addr learned my addr 172.21.15.33:0/3424156944 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44b8d9a700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44b4154330 0x7f44b41531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44b959b700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b40c7ff0 0x7f44b41432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44b9d9c700 1 -- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44b4154330 msgr2=0x7f44b41531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44b9d9c700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44b4154330 0x7f44b41531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44b9d9c700 1 -- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b40c7ff0 msgr2=0x7f44b41432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44b9d9c700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b40c7ff0 0x7f44b41432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.347+0000 7f44b9d9c700 1 -- 172.21.15.33:0/3424156944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a4009370 con 0x7f44b4154710 2023-11-06T22:00:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.348+0000 7f44b9d9c700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 0x7f44b40ae700 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f44ac009f20 tx=0x7f44ac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.348+0000 7f44a97fa700 1 -- 172.21.15.33:0/3424156944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac009c50 con 0x7f44b4154710 2023-11-06T22:00:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.348+0000 7f44a97fa700 1 -- 172.21.15.33:0/3424156944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac00f040 con 0x7f44b4154710 2023-11-06T22:00:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.348+0000 7f44bb7ff700 1 -- 172.21.15.33:0/3424156944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b40c3480 con 0x7f44b4154710 2023-11-06T22:00:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.348+0000 7f44a97fa700 1 -- 172.21.15.33:0/3424156944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00cda0 con 0x7f44b4154710 2023-11-06T22:00:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.348+0000 7f44bb7ff700 1 -- 172.21.15.33:0/3424156944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b40c3a40 con 0x7f44b4154710 2023-11-06T22:00:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.349+0000 7f44bb7ff700 1 -- 172.21.15.33:0/3424156944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b40c3c60 con 0x7f44b4154710 2023-11-06T22:00:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.352+0000 7f44a97fa700 1 -- 172.21.15.33:0/3424156944 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 74) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f44ac013610 con 0x7f44b4154710 2023-11-06T22:00:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.353+0000 7f44a97fa700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f449805f2c0 0x7f4498061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.353+0000 7f44a97fa700 1 -- 172.21.15.33:0/3424156944 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f44ac07ac00 con 0x7f44b4154710 2023-11-06T22:00:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.354+0000 7f44b959b700 1 -- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f449805f2c0 msgr2=0x7f4498061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.354+0000 7f44b959b700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f449805f2c0 0x7f4498061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:00:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.358+0000 7f44a97fa700 1 -- 172.21.15.33:0/3424156944 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f44ac04bd20 con 0x7f44b4154710 2023-11-06T22:00:27.408 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:27.405+0000 7f9d5041e680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:00:27.420 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:27.418+0000 7fbf889e7680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:00:27.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.541+0000 7f44bb7ff700 1 -- 172.21.15.33:0/3424156944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f44b40830b0 con 0x7f44b4154710 2023-11-06T22:00:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.546+0000 7f44a97fa700 1 -- 172.21.15.33:0/3424156944 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f44ac04f340 con 0x7f44b4154710 2023-11-06T22:00:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.548+0000 7f44bb7ff700 1 -- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f449805f2c0 msgr2=0x7f4498061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:00:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.548+0000 7f44bb7ff700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f449805f2c0 0x7f4498061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.549+0000 7f44bb7ff700 1 -- 172.21.15.33:0/3424156944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 msgr2=0x7f44b40ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.549+0000 7f44bb7ff700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 0x7f44b40ae700 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f44ac009f20 tx=0x7f44ac00b040).stop 2023-11-06T22:00:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.549+0000 7f44bb7ff700 1 -- 172.21.15.33:0/3424156944 shutdown_connections 2023-11-06T22:00:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.549+0000 7f44bb7ff700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f449805f2c0 0x7f4498061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.549+0000 7f44bb7ff700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b4154710 0x7f44b40ae700 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.549+0000 7f44bb7ff700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b40c7ff0 0x7f44b41432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.549+0000 7f44bb7ff700 1 --2- 172.21.15.33:0/3424156944 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f44b4154330 0x7f44b41531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.549+0000 7f44bb7ff700 1 -- 172.21.15.33:0/3424156944 >> 172.21.15.33:0/3424156944 conn(0x7f44b40cb910 msgr2=0x7f44b40c9350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.549+0000 7f44bb7ff700 1 -- 172.21.15.33:0/3424156944 shutdown_connections 2023-11-06T22:00:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.549+0000 7f44bb7ff700 1 -- 172.21.15.33:0/3424156944 wait complete. 2023-11-06T22:00:27.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:00:27.570 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-06T22:00:27.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:00:27.629 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-11-06T22:00:27.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/test_restart 1M 2023-11-06T22:00:27.737 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:27.735+0000 7f9d5041e680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:00:27.788 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:27.786+0000 7fbf889e7680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:00:27.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.883+0000 7fd8299ff700 1 Processor -- start 2023-11-06T22:00:27.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.884+0000 7fd8299ff700 1 -- start start 2023-11-06T22:00:27.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.885+0000 7fd8299ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd8240c8ee0 0x7fd8240ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.885+0000 7fd8299ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8240c9f80 con 0x7fd8241542b0 2023-11-06T22:00:27.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.885+0000 7fd8299ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8240c6000 con 0x7fd824153ed0 2023-11-06T22:00:27.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.885+0000 7fd8299ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8240c9370 con 0x7fd8240c8ee0 2023-11-06T22:00:27.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.885+0000 7fd8227fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd8240c8ee0 0x7fd8240ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:27.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.885+0000 7fd8227fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd8240c8ee0 0x7fd8240ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45316/0 (socket says 172.21.15.33:45316) 2023-11-06T22:00:27.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.885+0000 7fd8227fc700 1 -- 172.21.15.33:0/4249357656 learned_addr learned my addr 172.21.15.33:0/4249357656 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:27.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.885+0000 7fd8237fe700 1 --1- 172.21.15.33:0/4249357656 >> v1:172.21.15.33:6789/0 conn(0x7fd8241542b0 0x7fd8240c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52070/0 (socket says 172.21.15.33:52070) 2023-11-06T22:00:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.886+0000 7fd821ffb700 1 -- 172.21.15.33:0/4249357656 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528668002 0 0) 0x7fd8240c6000 con 0x7fd824153ed0 2023-11-06T22:00:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.886+0000 7fd8227fc700 1 -- 172.21.15.33:0/4249357656 >> v1:172.21.15.69:6789/0 conn(0x7fd824153ed0 legacy=0x7fd824158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.886+0000 7fd8227fc700 1 -- 172.21.15.33:0/4249357656 >> v1:172.21.15.33:6789/0 conn(0x7fd8241542b0 legacy=0x7fd8240c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.886+0000 7fd8227fc700 1 -- 172.21.15.33:0/4249357656 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8240c9650 con 0x7fd8240c8ee0 2023-11-06T22:00:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.886+0000 7fd8227fc700 1 --2- 172.21.15.33:0/4249357656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8240c8ee0 0x7fd8240ca410 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fd818006fd0 tx=0x7fd8180090f0).ready entity=mon.0 client_cookie=26e22137745dabba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.886+0000 7fd821ffb700 1 -- 172.21.15.33:0/4249357656 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81800f510 con 0x7fd8240c8ee0 2023-11-06T22:00:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.886+0000 7fd821ffb700 1 -- 172.21.15.33:0/4249357656 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81800f670 con 0x7fd8240c8ee0 2023-11-06T22:00:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.886+0000 7fd821ffb700 1 -- 172.21.15.33:0/4249357656 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81800fa10 con 0x7fd8240c8ee0 2023-11-06T22:00:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.887+0000 7fd8299ff700 1 -- 172.21.15.33:0/4249357656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8240c8ee0 msgr2=0x7fd8240ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.887+0000 7fd8299ff700 1 --2- 172.21.15.33:0/4249357656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8240c8ee0 0x7fd8240ca410 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fd818006fd0 tx=0x7fd8180090f0).stop 2023-11-06T22:00:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.887+0000 7fd8299ff700 1 -- 172.21.15.33:0/4249357656 shutdown_connections 2023-11-06T22:00:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.887+0000 7fd8299ff700 1 --2- 172.21.15.33:0/4249357656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8240c8ee0 0x7fd8240ca410 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.887+0000 7fd8299ff700 1 -- 172.21.15.33:0/4249357656 >> 172.21.15.33:0/4249357656 conn(0x7fd82405e1b0 msgr2=0x7fd82405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.887+0000 7fd8299ff700 1 -- 172.21.15.33:0/4249357656 shutdown_connections 2023-11-06T22:00:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.887+0000 7fd8299ff700 1 -- 172.21.15.33:0/4249357656 wait complete. 2023-11-06T22:00:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.888+0000 7fd8299ff700 1 Processor -- start 2023-11-06T22:00:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.889+0000 7fd8299ff700 1 -- start start 2023-11-06T22:00:27.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.889+0000 7fd8299ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824153ed0 0x7fd8241645b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd8299ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8241542b0 0x7fd82414cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd8299ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd82414cfa0 0x7fd824156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd8299ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8240afaf0 con 0x7fd824153ed0 2023-11-06T22:00:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd8299ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd824164aa0 con 0x7fd8241542b0 2023-11-06T22:00:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd8299ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8240aeee0 con 0x7fd82414cfa0 2023-11-06T22:00:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd822ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824153ed0 0x7fd8241645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:27.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd822ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824153ed0 0x7fd8241645b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45328/0 (socket says 172.21.15.33:45328) 2023-11-06T22:00:27.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd822ffd700 1 -- 172.21.15.33:0/1062089406 learned_addr learned my addr 172.21.15.33:0/1062089406 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:27.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd8227fc700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8241542b0 0x7fd82414cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:27.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd8237fe700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd82414cfa0 0x7fd824156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:27.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd822ffd700 1 -- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd82414cfa0 msgr2=0x7fd824156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd822ffd700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd82414cfa0 0x7fd824156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd822ffd700 1 -- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8241542b0 msgr2=0x7fd82414cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:27.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd822ffd700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8241542b0 0x7fd82414cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:27.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.890+0000 7fd822ffd700 1 -- 172.21.15.33:0/1062089406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd818009370 con 0x7fd824153ed0 2023-11-06T22:00:27.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.891+0000 7fd822ffd700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824153ed0 0x7fd8241645b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7fd814001eb0 tx=0x7fd814007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.891+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400ce60 con 0x7fd824153ed0 2023-11-06T22:00:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.891+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81400f470 con 0x7fd824153ed0 2023-11-06T22:00:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.891+0000 7fd8299ff700 1 -- 172.21.15.33:0/1062089406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8240af1c0 con 0x7fd824153ed0 2023-11-06T22:00:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.891+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400f810 con 0x7fd824153ed0 2023-11-06T22:00:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.891+0000 7fd8299ff700 1 -- 172.21.15.33:0/1062089406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8240af550 con 0x7fd824153ed0 2023-11-06T22:00:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.892+0000 7fd8299ff700 1 -- 172.21.15.33:0/1062089406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8240aec30 con 0x7fd824153ed0 2023-11-06T22:00:27.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.893+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 74) v1 ==== 75565+0+0 (secure 0 0 0) 0x7fd81400fb80 con 0x7fd824153ed0 2023-11-06T22:00:27.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.894+0000 7fd80affd700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:27.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.894+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fd8140798c0 con 0x7fd824153ed0 2023-11-06T22:00:27.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.894+0000 7fd8227fc700 1 -- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 msgr2=0x7fd810061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.894+0000 7fd8227fc700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:00:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:27.898+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd81404a9e0 con 0x7fd824153ed0 2023-11-06T22:00:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:28.095+0000 7fd8227fc700 1 -- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 msgr2=0x7fd810061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:28.095+0000 7fd8227fc700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:00:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:28.108+0000 7fd8299ff700 1 -- 172.21.15.33:0/1062089406 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8240afc30 con 0x7fd81005f2c0 2023-11-06T22:00:28.487 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:28.486+0000 7f9d5041e680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:00:28.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:28.496+0000 7fd8227fc700 1 -- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 msgr2=0x7fd810061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:28.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:28.496+0000 7fd8227fc700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T22:00:28.570 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:28.569+0000 7f9d5041e680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:00:28.573 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:28.571+0000 7fbf889e7680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:00:28.660 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:28.658+0000 7fbf889e7680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:00:28.801 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:28.800+0000 7f9d5041e680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:00:28.898 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:28.896+0000 7fbf889e7680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:00:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:29.297+0000 7fd8227fc700 1 -- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 msgr2=0x7fd810061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:29.297+0000 7fd8227fc700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T22:00:29.313 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:29.312+0000 7f9d5041e680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:00:29.432 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:29.430+0000 7fbf889e7680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:00:29.438 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:29.436+0000 7f9d5041e680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:00:29.562 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:29.560+0000 7fbf889e7680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:00:29.622 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:29.620+0000 7f9d5041e680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:00:29.717 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:29.715+0000 7f9d5041e680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:00:29.752 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:29.750+0000 7fbf889e7680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:00:29.802 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:29.801+0000 7f9d5041e680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:00:29.850 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:29.848+0000 7fbf889e7680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:00:29.939 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:29.937+0000 7fbf889e7680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:00:30.113 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:30.111+0000 7f9d5041e680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:00:30.264 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:30.262+0000 7fbf889e7680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:00:30.336 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:30.335+0000 7f9d5041e680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:00:30.493 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:30.491+0000 7fbf889e7680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:00:30.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:30.899+0000 7fd8227fc700 1 -- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 msgr2=0x7fd810061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/114912 2023-11-06T22:00:30.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:30.899+0000 7fd8227fc700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T22:00:31.140 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:31.138+0000 7f9d5041e680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:00:31.228 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:31.226+0000 7f9d5041e680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:00:31.322 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:31.320+0000 7f9d5041e680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:00:31.332 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:31.330+0000 7fbf889e7680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:00:31.423 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:31.421+0000 7fbf889e7680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:00:31.510 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:31.508+0000 7f9d5041e680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:00:31.521 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:31.519+0000 7fbf889e7680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:00:31.600 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:31.599+0000 7f9d5041e680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:00:31.715 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:31.713+0000 7fbf889e7680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:00:31.742 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:31.741+0000 7f9d5041e680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:00:31.808 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:31.806+0000 7fbf889e7680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:00:31.874 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:31.872+0000 7f9d5041e680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:00:31.957 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:31.955+0000 7fbf889e7680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:00:32.094 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:32.092+0000 7fbf889e7680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:00:32.221 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.219+0000 7f9d5041e680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:00:32.312 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.310+0000 7f9d5041e680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:00:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:32.319+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 75) v1 ==== 75287+0+0 (secure 0 0 0) 0x7fd8140774e0 con 0x7fd824153ed0 2023-11-06T22:00:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:32.319+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 msgr2=0x7fd810061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:00:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:32.319+0000 7fd80affd700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:00:32.399 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.397+0000 7f9cc5f47700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:32.400 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.397+0000 7f9cc5f47700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:32.400 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.397+0000 7f9cc5f47700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:32.400 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.397+0000 7f9cc5f47700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:32.400 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.397+0000 7f9cc5f47700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:32.401 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.398+0000 7f9cc2f41700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:32.401 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.398+0000 7f9cc2f41700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:32.401 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.398+0000 7f9cc2f41700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:32.402 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.398+0000 7f9cc2f41700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:32.402 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:00:32.398+0000 7f9cc2f41700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:00:32.459 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:32.457+0000 7fbf889e7680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:00:32.553 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:00:32.551+0000 7fbf889e7680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:00:33.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.322+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 76) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fd814077860 con 0x7fd824153ed0 2023-11-06T22:00:33.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.322+0000 7fd80affd700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:33.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.322+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd810030b80 con 0x7fd81005f2c0 2023-11-06T22:00:33.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.323+0000 7fd8227fc700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:33.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.324+0000 7fd8227fc700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061700 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fd82414f280 tx=0x7fd818017000).ready entity=mgr.8722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:33.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.335+0000 7fd80affd700 1 -- 172.21.15.33:0/1062089406 <== mgr.8722 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd81800f860 con 0x7fd81005f2c0 2023-11-06T22:00:33.338 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-11-06T22:00:33.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 -- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 msgr2=0x7fd810061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:33.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061700 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fd82414f280 tx=0x7fd818017000).stop 2023-11-06T22:00:33.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 -- 172.21.15.33:0/1062089406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824153ed0 msgr2=0x7fd8241645b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824153ed0 0x7fd8241645b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7fd814001eb0 tx=0x7fd814007c70).stop 2023-11-06T22:00:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 -- 172.21.15.33:0/1062089406 shutdown_connections 2023-11-06T22:00:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7fd81005f2c0 0x7fd810061700 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd82414cfa0 0x7fd824156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824153ed0 0x7fd8241645b0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 --2- 172.21.15.33:0/1062089406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8241542b0 0x7fd82414cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 -- 172.21.15.33:0/1062089406 >> 172.21.15.33:0/1062089406 conn(0x7fd82405e1b0 msgr2=0x7fd82414da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:33.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 -- 172.21.15.33:0/1062089406 shutdown_connections 2023-11-06T22:00:33.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.338+0000 7fd8299ff700 1 -- 172.21.15.33:0/1062089406 wait complete. 2023-11-06T22:00:33.361 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-11-06T22:00:33.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/test_restart 1M 2023-11-06T22:00:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.565+0000 7f3df73ff700 1 Processor -- start 2023-11-06T22:00:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.566+0000 7f3df73ff700 1 -- start start 2023-11-06T22:00:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.567+0000 7f3df73ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3df00638b0 0x7f3df00ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.567+0000 7f3df73ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3df01550d0 0x7f3df00c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.567+0000 7f3df73ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df01586a0 con 0x7f3df00acfc0 2023-11-06T22:00:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.567+0000 7f3df73ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df00c6590 con 0x7f3df00638b0 2023-11-06T22:00:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.567+0000 7f3df73ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df0155580 con 0x7f3df01550d0 2023-11-06T22:00:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.567+0000 7f3df499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3df00638b0 0x7f3df00ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.567+0000 7f3df499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3df00638b0 0x7f3df00ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46554/0 (socket says 172.21.15.33:46554) 2023-11-06T22:00:33.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.567+0000 7f3df499a700 1 -- 172.21.15.33:0/2038006504 learned_addr learned my addr 172.21.15.33:0/2038006504 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.567+0000 7f3df519b700 1 --1- 172.21.15.33:0/2038006504 >> v1:172.21.15.33:6789/0 conn(0x7f3df00acfc0 0x7f3df00631a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50944/0 (socket says 172.21.15.33:50944) 2023-11-06T22:00:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.567+0000 7f3df599c700 1 --2- 172.21.15.33:0/2038006504 >> v2:172.21.15.69:3300/0 conn(0x7f3df01550d0 0x7f3df00c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.568+0000 7f3df499a700 1 -- 172.21.15.33:0/2038006504 >> v2:172.21.15.69:3300/0 conn(0x7f3df01550d0 msgr2=0x7f3df00c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:00:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.568+0000 7f3df499a700 1 --2- 172.21.15.33:0/2038006504 >> v2:172.21.15.69:3300/0 conn(0x7f3df01550d0 0x7f3df00c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.568+0000 7f3df499a700 1 -- 172.21.15.33:0/2038006504 >> v1:172.21.15.33:6789/0 conn(0x7f3df00acfc0 legacy=0x7f3df00631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.568+0000 7f3df499a700 1 -- 172.21.15.33:0/2038006504 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df01544c0 con 0x7f3df00638b0 2023-11-06T22:00:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.568+0000 7f3de7fff700 1 -- 172.21.15.33:0/2038006504 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973269921 0 0) 0x7f3df01586a0 con 0x7f3df00acfc0 2023-11-06T22:00:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.568+0000 7f3df499a700 1 --2- 172.21.15.33:0/2038006504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 0x7f3df00ad3a0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f3de0006fd0 tx=0x7f3de00090f0).ready entity=mon.0 client_cookie=1f2df76708e3dc0e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.568+0000 7f3de7fff700 1 -- 172.21.15.33:0/2038006504 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de000b600 con 0x7f3df00638b0 2023-11-06T22:00:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.568+0000 7f3de7fff700 1 -- 172.21.15.33:0/2038006504 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de000b760 con 0x7f3df00638b0 2023-11-06T22:00:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.568+0000 7f3de7fff700 1 -- 172.21.15.33:0/2038006504 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de000bae0 con 0x7f3df00638b0 2023-11-06T22:00:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.569+0000 7f3df73ff700 1 -- 172.21.15.33:0/2038006504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 msgr2=0x7f3df00ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.569+0000 7f3df73ff700 1 --2- 172.21.15.33:0/2038006504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 0x7f3df00ad3a0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f3de0006fd0 tx=0x7f3de00090f0).stop 2023-11-06T22:00:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.569+0000 7f3df73ff700 1 -- 172.21.15.33:0/2038006504 shutdown_connections 2023-11-06T22:00:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.569+0000 7f3df73ff700 1 --2- 172.21.15.33:0/2038006504 >> v2:172.21.15.69:3300/0 conn(0x7f3df01550d0 0x7f3df00c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.569+0000 7f3df73ff700 1 --2- 172.21.15.33:0/2038006504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 0x7f3df00ad3a0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.569+0000 7f3df73ff700 1 -- 172.21.15.33:0/2038006504 >> 172.21.15.33:0/2038006504 conn(0x7f3df005ec80 msgr2=0x7f3df00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.569+0000 7f3df73ff700 1 -- 172.21.15.33:0/2038006504 shutdown_connections 2023-11-06T22:00:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.569+0000 7f3df73ff700 1 -- 172.21.15.33:0/2038006504 wait complete. 2023-11-06T22:00:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.570+0000 7f3df73ff700 1 Processor -- start 2023-11-06T22:00:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.570+0000 7f3df73ff700 1 -- start start 2023-11-06T22:00:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.571+0000 7f3df73ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 0x7f3df0167a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.571+0000 7f3df73ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3df00acfc0 0x7f3df0152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.571+0000 7f3df73ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3df01550d0 0x7f3df01450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.571+0000 7f3df73ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00c4140 con 0x7f3df00638b0 2023-11-06T22:00:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.571+0000 7f3df73ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df0167fe0 con 0x7f3df01550d0 2023-11-06T22:00:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.571+0000 7f3df73ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df01682c0 con 0x7f3df00acfc0 2023-11-06T22:00:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.572+0000 7f3df519b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 0x7f3df0167a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.572+0000 7f3df519b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 0x7f3df0167a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46558/0 (socket says 172.21.15.33:46558) 2023-11-06T22:00:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.572+0000 7f3df519b700 1 -- 172.21.15.33:0/1099551923 learned_addr learned my addr 172.21.15.33:0/1099551923 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:00:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.572+0000 7f3df599c700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3df01550d0 0x7f3df01450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.572+0000 7f3df499a700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3df00acfc0 0x7f3df0152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.572+0000 7f3df519b700 1 -- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3df00acfc0 msgr2=0x7f3df0152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.572+0000 7f3df519b700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3df00acfc0 0x7f3df0152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.572+0000 7f3df519b700 1 -- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3df01550d0 msgr2=0x7f3df01450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.572+0000 7f3df519b700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3df01550d0 0x7f3df01450b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.572+0000 7f3df519b700 1 -- 172.21.15.33:0/1099551923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de0009370 con 0x7f3df00638b0 2023-11-06T22:00:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.573+0000 7f3df519b700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 0x7f3df0167a00 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f3dec00c560 tx=0x7f3dec007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:33.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.573+0000 7f3de57fa700 1 -- 172.21.15.33:0/1099551923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0072e0 con 0x7f3df00638b0 2023-11-06T22:00:33.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.573+0000 7f3de57fa700 1 -- 172.21.15.33:0/1099551923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec007440 con 0x7f3df00638b0 2023-11-06T22:00:33.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.573+0000 7f3de57fa700 1 -- 172.21.15.33:0/1099551923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0058f0 con 0x7f3df00638b0 2023-11-06T22:00:33.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.573+0000 7f3df73ff700 1 -- 172.21.15.33:0/1099551923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df01685a0 con 0x7f3df00638b0 2023-11-06T22:00:33.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.573+0000 7f3df73ff700 1 -- 172.21.15.33:0/1099551923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df0152f90 con 0x7f3df00638b0 2023-11-06T22:00:33.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.574+0000 7f3df73ff700 1 -- 172.21.15.33:0/1099551923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df00c3f10 con 0x7f3df00638b0 2023-11-06T22:00:33.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.575+0000 7f3de57fa700 1 -- 172.21.15.33:0/1099551923 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 76) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f3dec005ce0 con 0x7f3df00638b0 2023-11-06T22:00:33.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.576+0000 7f3de57fa700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3dd405f080 0x7f3dd4061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:00:33.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.576+0000 7f3de57fa700 1 -- 172.21.15.33:0/1099551923 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f3dec079950 con 0x7f3df00638b0 2023-11-06T22:00:33.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.576+0000 7f3df499a700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3dd405f080 0x7f3dd4061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:00:33.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.577+0000 7f3df499a700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3dd405f080 0x7f3dd4061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3de0006fd0 tx=0x7f3de0017000).ready entity=mgr.8722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:00:33.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.580+0000 7f3de57fa700 1 -- 172.21.15.33:0/1099551923 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3df00c3f10 con 0x7f3df00638b0 2023-11-06T22:00:33.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.788+0000 7f3df73ff700 1 -- 172.21.15.33:0/1099551923 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3df00c4280 con 0x7f3dd405f080 2023-11-06T22:00:33.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.796+0000 7f3de57fa700 1 -- 172.21.15.33:0/1099551923 <== mgr.8722 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f3de000b8d0 con 0x7f3dd405f080 2023-11-06T22:00:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-11-06T22:00:33.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 -- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3dd405f080 msgr2=0x7f3dd4061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:33.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3dd405f080 0x7f3dd4061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3de0006fd0 tx=0x7f3de0017000).stop 2023-11-06T22:00:33.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 -- 172.21.15.33:0/1099551923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 msgr2=0x7f3df0167a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:00:33.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 0x7f3df0167a00 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f3dec00c560 tx=0x7f3dec007c90).stop 2023-11-06T22:00:33.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 -- 172.21.15.33:0/1099551923 shutdown_connections 2023-11-06T22:00:33.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f3dd405f080 0x7f3dd4061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3df01550d0 0x7f3df01450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3df00638b0 0x7f3df0167a00 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 --2- 172.21.15.33:0/1099551923 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3df00acfc0 0x7f3df0152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:00:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 -- 172.21.15.33:0/1099551923 >> 172.21.15.33:0/1099551923 conn(0x7f3df005ec80 msgr2=0x7f3df00c0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:00:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 -- 172.21.15.33:0/1099551923 shutdown_connections 2023-11-06T22:00:33.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:00:33.798+0000 7f3df73ff700 1 -- 172.21.15.33:0/1099551923 wait complete. 2023-11-06T22:00:33.817 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-11-06T22:00:55.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:00:55.825 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:01:00.804 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:01:00.802+0000 7f9cb4ee5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:01:00.805 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:01:00.802+0000 7f9cb4ee5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:01:00.805 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:01:00.802+0000 7f9cb4ee5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:01:00.805 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:01:00.802+0000 7f9cb4ee5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:01:00.805 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:01:00.802+0000 7f9cb4ee5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:01:25.956 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:01:25.962 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:01:56.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:01:56.159 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:02:26.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:02:26.280 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:02:56.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:02:56.562 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:03:26.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:03:26.684 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:03:56.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:03:56.807 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:04:26.917 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:04:26.922 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:04:57.035 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:04:57.042 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:05:27.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:05:27.421 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:05:57.523 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:05:57.529 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:06:27.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:06:27.643 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:06:57.763 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:06:57.769 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:07:27.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:07:27.891 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:07:58.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:07:58.027 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:08:28.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:08:28.155 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:08:58.268 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:08:58.274 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:09:28.382 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:09:28.387 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:09:58.523 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:09:58.529 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:10:28.641 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:10:28.647 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:10:58.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:10:58.904 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:11:15.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:11:15.820 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-11-06T22:11:15.876 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_01_00_UTC 2023-11-06T22:11:15.876 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_02_00_UTC 2023-11-06T22:11:15.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_03_00_UTC 2023-11-06T22:11:15.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_04_00_UTC 2023-11-06T22:11:15.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_05_00_UTC 2023-11-06T22:11:15.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_06_00_UTC 2023-11-06T22:11:15.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_07_00_UTC 2023-11-06T22:11:15.877 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_08_00_UTC 2023-11-06T22:11:15.878 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_09_00_UTC 2023-11-06T22:11:15.878 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_10_00_UTC 2023-11-06T22:11:15.878 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_11_00_UTC 2023-11-06T22:11:15.879 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-22_01_00_UTC', 'scheduled-2023-11-06-22_02_00_UTC', 'scheduled-2023-11-06-22_03_00_UTC', 'scheduled-2023-11-06-22_04_00_UTC', 'scheduled-2023-11-06-22_05_00_UTC', 'scheduled-2023-11-06-22_06_00_UTC', 'scheduled-2023-11-06-22_07_00_UTC', 'scheduled-2023-11-06-22_08_00_UTC', 'scheduled-2023-11-06-22_09_00_UTC', 'scheduled-2023-11-06-22_10_00_UTC', 'scheduled-2023-11-06-22_11_00_UTC'] 2023-11-06T22:11:15.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-11-06T22:11:16.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.082+0000 7f04811ff700 1 Processor -- start 2023-11-06T22:11:16.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.082+0000 7f04811ff700 1 -- start start 2023-11-06T22:11:16.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.083+0000 7f04811ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f047c150870 0x7f047c150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.083+0000 7f04811ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f047c0bc690 0x7f047c064230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.083+0000 7f04811ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c0bb980 con 0x7f047c064720 2023-11-06T22:11:16.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.083+0000 7f04811ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f047c064b50 con 0x7f047c150870 2023-11-06T22:11:16.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.083+0000 7f04811ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c1553b0 con 0x7f047c0bc690 2023-11-06T22:11:16.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.083+0000 7f047b59e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f047c064720 0x7f047c154680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43476/0 (socket says 172.21.15.33:43476) 2023-11-06T22:11:16.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.083+0000 7f047b59e700 1 -- 172.21.15.33:0/4031938988 learned_addr learned my addr 172.21.15.33:0/4031938988 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:11:16.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.084+0000 7f047ad9d700 1 --2- 172.21.15.33:0/4031938988 >> v2:172.21.15.69:3300/0 conn(0x7f047c150870 0x7f047c150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.084+0000 7f047a59c700 1 --2- 172.21.15.33:0/4031938988 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f047c0bc690 0x7f047c064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.084+0000 7f0479d9b700 1 -- 172.21.15.33:0/4031938988 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248246542 0 0) 0x7f047c0bb980 con 0x7f047c064720 2023-11-06T22:11:16.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.084+0000 7f0479d9b700 1 -- 172.21.15.33:0/4031938988 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04640033f0 con 0x7f047c064720 2023-11-06T22:11:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.084+0000 7f047ad9d700 1 -- 172.21.15.33:0/4031938988 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f047c0bc690 msgr2=0x7f047c064230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:11:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.084+0000 7f047ad9d700 1 --2- 172.21.15.33:0/4031938988 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f047c0bc690 0x7f047c064230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.084+0000 7f047ad9d700 1 -- 172.21.15.33:0/4031938988 >> v1:172.21.15.33:6789/0 conn(0x7f047c064720 legacy=0x7f047c154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.084+0000 7f047ad9d700 1 -- 172.21.15.33:0/4031938988 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c0bbbd0 con 0x7f047c150870 2023-11-06T22:11:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.084+0000 7f047ad9d700 1 --2- 172.21.15.33:0/4031938988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047c150870 0x7f047c150c50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f046c009760 tx=0x7f046c009060).ready entity=mon.1 client_cookie=e73740c61c21ca8a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:11:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f0479d9b700 1 -- 172.21.15.33:0/4031938988 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c00d4a0 con 0x7f047c150870 2023-11-06T22:11:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f0479d9b700 1 -- 172.21.15.33:0/4031938988 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c00d600 con 0x7f047c150870 2023-11-06T22:11:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f0479d9b700 1 -- 172.21.15.33:0/4031938988 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c00d980 con 0x7f047c150870 2023-11-06T22:11:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f04811ff700 1 -- 172.21.15.33:0/4031938988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047c150870 msgr2=0x7f047c150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f04811ff700 1 --2- 172.21.15.33:0/4031938988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047c150870 0x7f047c150c50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f046c009760 tx=0x7f046c009060).stop 2023-11-06T22:11:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f04811ff700 1 -- 172.21.15.33:0/4031938988 shutdown_connections 2023-11-06T22:11:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f04811ff700 1 --2- 172.21.15.33:0/4031938988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047c150870 0x7f047c150c50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f04811ff700 1 --2- 172.21.15.33:0/4031938988 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f047c0bc690 0x7f047c064230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f04811ff700 1 -- 172.21.15.33:0/4031938988 >> 172.21.15.33:0/4031938988 conn(0x7f047c062710 msgr2=0x7f047c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:11:16.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f04811ff700 1 -- 172.21.15.33:0/4031938988 shutdown_connections 2023-11-06T22:11:16.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.085+0000 7f04811ff700 1 -- 172.21.15.33:0/4031938988 wait complete. 2023-11-06T22:11:16.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.087+0000 7f04811ff700 1 Processor -- start 2023-11-06T22:11:16.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.087+0000 7f04811ff700 1 -- start start 2023-11-06T22:11:16.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.088+0000 7f04811ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f047c064720 0x7f047c0a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.088+0000 7f04811ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047c0bc690 0x7f047c169000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.088+0000 7f04811ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f047c150870 0x7f047c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.088+0000 7f04811ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0aab10 con 0x7f047c150870 2023-11-06T22:11:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.088+0000 7f04811ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c1694f0 con 0x7f047c0bc690 2023-11-06T22:11:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.088+0000 7f04811ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0a9f90 con 0x7f047c064720 2023-11-06T22:11:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.088+0000 7f047b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f047c150870 0x7f047c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.088+0000 7f047b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f047c150870 0x7f047c0a5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40290/0 (socket says 172.21.15.33:40290) 2023-11-06T22:11:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.088+0000 7f047b59e700 1 -- 172.21.15.33:0/1164801555 learned_addr learned my addr 172.21.15.33:0/1164801555 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:11:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f047ad9d700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f047c064720 0x7f047c0a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f047a59c700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047c0bc690 0x7f047c169000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f047b59e700 1 -- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f047c064720 msgr2=0x7f047c0a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f047b59e700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f047c064720 0x7f047c0a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f047b59e700 1 -- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047c0bc690 msgr2=0x7f047c169000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f047b59e700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047c0bc690 0x7f047c169000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f047b59e700 1 -- 172.21.15.33:0/1164801555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c0092a0 con 0x7f047c150870 2023-11-06T22:11:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f047b59e700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f047c150870 0x7f047c0a5ad0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f0474005ad0 tx=0x7f0474001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:11:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f0462ffd700 1 -- 172.21.15.33:0/1164801555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0474005d30 con 0x7f047c150870 2023-11-06T22:11:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f0462ffd700 1 -- 172.21.15.33:0/1164801555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0474005e90 con 0x7f047c150870 2023-11-06T22:11:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f0462ffd700 1 -- 172.21.15.33:0/1164801555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0474007320 con 0x7f047c150870 2023-11-06T22:11:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f04811ff700 1 -- 172.21.15.33:0/1164801555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c0aa270 con 0x7f047c150870 2023-11-06T22:11:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.089+0000 7f04811ff700 1 -- 172.21.15.33:0/1164801555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c0aa830 con 0x7f047c150870 2023-11-06T22:11:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.091+0000 7f04811ff700 1 -- 172.21.15.33:0/1164801555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047c0bbeb0 con 0x7f047c150870 2023-11-06T22:11:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.091+0000 7f0462ffd700 1 -- 172.21.15.33:0/1164801555 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 79) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f0474007710 con 0x7f047c150870 2023-11-06T22:11:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.092+0000 7f0462ffd700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046805f290 0x7f0468061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.092+0000 7f0462ffd700 1 -- 172.21.15.33:0/1164801555 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f047407a740 con 0x7f047c150870 2023-11-06T22:11:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.093+0000 7f047ad9d700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046805f290 0x7f0468061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.093+0000 7f047ad9d700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046805f290 0x7f0468061750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f047c152f80 tx=0x7f046c016000).ready entity=mgr.8722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:11:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.097+0000 7f0462ffd700 1 -- 172.21.15.33:0/1164801555 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f047404b660 con 0x7f047c150870 2023-11-06T22:11:16.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.303+0000 7f04811ff700 1 -- 172.21.15.33:0/1164801555 --> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f047c063310 con 0x7f046805f290 2023-11-06T22:11:16.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.306+0000 7f0462ffd700 1 -- 172.21.15.33:0/1164801555 <== mgr.8722 v2:172.21.15.69:6832/114912 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f047c063310 con 0x7f046805f290 2023-11-06T22:11:16.308 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:11:16.308 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T22:00:33", "first": "2023-11-06T22:01:00", "last": "2023-11-06T22:11:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-11-06T22:11:16.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 -- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046805f290 msgr2=0x7f0468061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046805f290 0x7f0468061750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f047c152f80 tx=0x7f046c016000).stop 2023-11-06T22:11:16.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 -- 172.21.15.33:0/1164801555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f047c150870 msgr2=0x7f047c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f047c150870 0x7f047c0a5ad0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f0474005ad0 tx=0x7f0474001650).stop 2023-11-06T22:11:16.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 -- 172.21.15.33:0/1164801555 shutdown_connections 2023-11-06T22:11:16.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f046805f290 0x7f0468061750 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f047c064720 0x7f047c0a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f047c150870 0x7f047c0a5ad0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 --2- 172.21.15.33:0/1164801555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047c0bc690 0x7f047c169000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 -- 172.21.15.33:0/1164801555 >> 172.21.15.33:0/1164801555 conn(0x7f047c062710 msgr2=0x7f047c154610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:11:16.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 -- 172.21.15.33:0/1164801555 shutdown_connections 2023-11-06T22:11:16.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.308+0000 7f04811ff700 1 -- 172.21.15.33:0/1164801555 wait complete. 2023-11-06T22:11:16.327 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T22:00:33", "first": "2023-11-06T22:01:00", "last": "2023-11-06T22:11:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-11-06T22:11:16.328 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T22:00:33', 'first': '2023-11-06T22:01:00', 'last': '2023-11-06T22:11:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-11-06T22:11:16.328 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-06T22:11:16.329 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-06T22:11:16.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-06T22:11:16.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.522+0000 7f1827fff700 1 Processor -- start 2023-11-06T22:11:16.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.522+0000 7f1827fff700 1 -- start start 2023-11-06T22:11:16.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f1827fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f182014ebb0 0x7f1820063660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f1827fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18200c05e0 0x7f18200c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f1827fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1820158e20 con 0x7f1820063280 2023-11-06T22:11:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f1827fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18201582a0 con 0x7f182014ebb0 2023-11-06T22:11:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f1827fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1820158580 con 0x7f18200c05e0 2023-11-06T22:11:16.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f182559a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f182014ebb0 0x7f1820063660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f182559a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f182014ebb0 0x7f1820063660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40298/0 (socket says 172.21.15.33:40298) 2023-11-06T22:11:16.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f182559a700 1 -- 172.21.15.33:0/657011399 learned_addr learned my addr 172.21.15.33:0/657011399 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:11:16.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f182659c700 1 --2- 172.21.15.33:0/657011399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18200c05e0 0x7f18200c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f182559a700 1 -- 172.21.15.33:0/657011399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18200c05e0 msgr2=0x7f18200c0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:11:16.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.524+0000 7f182559a700 1 --2- 172.21.15.33:0/657011399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18200c05e0 0x7f18200c0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.525+0000 7f182559a700 1 -- 172.21.15.33:0/657011399 >> v1:172.21.15.69:6789/0 conn(0x7f1820063280 legacy=0x7f182014e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.525+0000 7f182559a700 1 -- 172.21.15.33:0/657011399 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820158860 con 0x7f182014ebb0 2023-11-06T22:11:16.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.525+0000 7f182559a700 1 --2- 172.21.15.33:0/657011399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 0x7f1820063660 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f1810006fd0 tx=0x7f18100090f0).ready entity=mon.0 client_cookie=4b3fdd8e6ed74a7a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:11:16.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.525+0000 7f1824d99700 1 -- 172.21.15.33:0/657011399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181000b580 con 0x7f182014ebb0 2023-11-06T22:11:16.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.525+0000 7f1824d99700 1 -- 172.21.15.33:0/657011399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181000b6e0 con 0x7f182014ebb0 2023-11-06T22:11:16.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.525+0000 7f1824d99700 1 -- 172.21.15.33:0/657011399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181000ba60 con 0x7f182014ebb0 2023-11-06T22:11:16.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.525+0000 7f1827fff700 1 -- 172.21.15.33:0/657011399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 msgr2=0x7f1820063660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.525+0000 7f1827fff700 1 --2- 172.21.15.33:0/657011399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 0x7f1820063660 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f1810006fd0 tx=0x7f18100090f0).stop 2023-11-06T22:11:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.526+0000 7f1827fff700 1 -- 172.21.15.33:0/657011399 shutdown_connections 2023-11-06T22:11:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.526+0000 7f1827fff700 1 --2- 172.21.15.33:0/657011399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18200c05e0 0x7f18200c0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.526+0000 7f1827fff700 1 --2- 172.21.15.33:0/657011399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 0x7f1820063660 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.526+0000 7f1827fff700 1 -- 172.21.15.33:0/657011399 >> 172.21.15.33:0/657011399 conn(0x7f18200cade0 msgr2=0x7f18200cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:11:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.526+0000 7f1827fff700 1 -- 172.21.15.33:0/657011399 shutdown_connections 2023-11-06T22:11:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.526+0000 7f1827fff700 1 -- 172.21.15.33:0/657011399 wait complete. 2023-11-06T22:11:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.527+0000 7f1827fff700 1 Processor -- start 2023-11-06T22:11:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.527+0000 7f1827fff700 1 -- start start 2023-11-06T22:11:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.528+0000 7f1827fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1820063280 0x7f18200c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.528+0000 7f1827fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18200c05e0 0x7f182014c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.528+0000 7f1827fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 0x7f18200a31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.528+0000 7f1827fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18200c4790 con 0x7f182014ebb0 2023-11-06T22:11:16.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.528+0000 7f1827fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18201656f0 con 0x7f1820063280 2023-11-06T22:11:16.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.528+0000 7f1827fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182014cd60 con 0x7f18200c05e0 2023-11-06T22:11:16.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f182659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 0x7f18200a31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f182659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 0x7f18200a31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40300/0 (socket says 172.21.15.33:40300) 2023-11-06T22:11:16.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f182659c700 1 -- 172.21.15.33:0/989490033 learned_addr learned my addr 172.21.15.33:0/989490033 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:11:16.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f1825d9b700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1820063280 0x7f18200c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f182559a700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18200c05e0 0x7f182014c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f182659c700 1 -- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18200c05e0 msgr2=0x7f182014c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f182659c700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18200c05e0 0x7f182014c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f182659c700 1 -- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1820063280 msgr2=0x7f18200c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f182659c700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1820063280 0x7f18200c1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f182659c700 1 -- 172.21.15.33:0/989490033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1810009370 con 0x7f182014ebb0 2023-11-06T22:11:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f182659c700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 0x7f18200a31d0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f1818009ec0 tx=0x7f181800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:11:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.529+0000 7f1815ffb700 1 -- 172.21.15.33:0/989490033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818009bf0 con 0x7f182014ebb0 2023-11-06T22:11:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.530+0000 7f1815ffb700 1 -- 172.21.15.33:0/989490033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181800f040 con 0x7f182014ebb0 2023-11-06T22:11:16.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.530+0000 7f1827fff700 1 -- 172.21.15.33:0/989490033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18200c41d0 con 0x7f182014ebb0 2023-11-06T22:11:16.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.530+0000 7f1815ffb700 1 -- 172.21.15.33:0/989490033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181800cd80 con 0x7f182014ebb0 2023-11-06T22:11:16.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.530+0000 7f1827fff700 1 -- 172.21.15.33:0/989490033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18200a1c80 con 0x7f182014ebb0 2023-11-06T22:11:16.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.531+0000 7f1827fff700 1 -- 172.21.15.33:0/989490033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182006c6c0 con 0x7f182014ebb0 2023-11-06T22:11:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.532+0000 7f1815ffb700 1 -- 172.21.15.33:0/989490033 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 79) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f1818013610 con 0x7f182014ebb0 2023-11-06T22:11:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.532+0000 7f1815ffb700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f180405f2c0 0x7f1804061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:11:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.532+0000 7f1815ffb700 1 -- 172.21.15.33:0/989490033 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f181807acb0 con 0x7f182014ebb0 2023-11-06T22:11:16.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.533+0000 7f1825d9b700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f180405f2c0 0x7f1804061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:11:16.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.533+0000 7f1825d9b700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f180405f2c0 0x7f1804061780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f18200c51d0 tx=0x7f181c009040).ready entity=mgr.8722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:11:16.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.537+0000 7f1815ffb700 1 -- 172.21.15.33:0/989490033 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f181804bc90 con 0x7f182014ebb0 2023-11-06T22:11:16.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.763+0000 7f1827fff700 1 -- 172.21.15.33:0/989490033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f18200c1c40 con 0x7f182014ebb0 2023-11-06T22:11:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.767+0000 7f1815ffb700 1 -- 172.21.15.33:0/989490033 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v79) v1 ==== 74+0+149159 (secure 0 0 0) 0x7f181804f2b0 con 0x7f182014ebb0 2023-11-06T22:11:16.771 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:11:16.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 -- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f180405f2c0 msgr2=0x7f1804061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f180405f2c0 0x7f1804061780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f18200c51d0 tx=0x7f181c009040).stop 2023-11-06T22:11:16.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 -- 172.21.15.33:0/989490033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 msgr2=0x7f18200a31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:11:16.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 0x7f18200a31d0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f1818009ec0 tx=0x7f181800b040).stop 2023-11-06T22:11:16.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 -- 172.21.15.33:0/989490033 shutdown_connections 2023-11-06T22:11:16.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:6832/114912,v1:172.21.15.69:6833/114912] conn(0x7f180405f2c0 0x7f1804061780 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f182014ebb0 0x7f18200a31d0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1820063280 0x7f18200c1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 --2- 172.21.15.33:0/989490033 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f18200c05e0 0x7f182014c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:11:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 -- 172.21.15.33:0/989490033 >> 172.21.15.33:0/989490033 conn(0x7f18200cade0 msgr2=0x7f18200c9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:11:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 -- 172.21.15.33:0/989490033 shutdown_connections 2023-11-06T22:11:16.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:11:16.770+0000 7f1827fff700 1 -- 172.21.15.33:0/989490033 wait complete. 2023-11-06T22:11:16.790 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":79,"active_gid":8722,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":114912},{"type":"v1","addr":"172.21.15.69:6833","nonce":114912}]},"active_addr":"172.21.15.69:6833/114912","active_change":"2023-11-06T22:00:32.314204+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":8719,"name":"y","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":186,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":540532913}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3284886886}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2903646842}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3999769341}]}]} 2023-11-06T22:11:16.794 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2023-11-06T22:11:16.795 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-06T22:11:16.795 INFO:tasks.ceph.mgr.x:Stopping old one... 2023-11-06T22:11:16.795 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-06T22:11:16.796 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:11:16.861 INFO:tasks.ceph.mgr.x:Stopped 2023-11-06T22:11:16.861 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-06T22:11:16.866 INFO:tasks.ceph.mgr.x:Started 2023-11-06T22:11:17.138 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:17.136+0000 7fc40ad6f680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:11:17.215 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:17.213+0000 7fc40ad6f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:11:17.547 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:17.545+0000 7fc40ad6f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:11:18.290 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:18.288+0000 7fc40ad6f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:11:18.374 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:18.372+0000 7fc40ad6f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:11:18.606 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:18.604+0000 7fc40ad6f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:11:19.118 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:19.116+0000 7fc40ad6f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:11:19.243 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:19.241+0000 7fc40ad6f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:11:19.428 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:19.426+0000 7fc40ad6f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:11:19.524 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:19.522+0000 7fc40ad6f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:11:19.609 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:19.607+0000 7fc40ad6f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:11:19.922 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:19.920+0000 7fc40ad6f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:11:20.144 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:20.142+0000 7fc40ad6f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:11:20.947 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:20.945+0000 7fc40ad6f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:11:21.035 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:21.033+0000 7fc40ad6f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:11:21.129 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:21.127+0000 7fc40ad6f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:11:21.316 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:21.314+0000 7fc40ad6f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:11:21.405 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:21.403+0000 7fc40ad6f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:11:21.547 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:21.545+0000 7fc40ad6f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:11:21.680 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:21.678+0000 7fc40ad6f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:11:22.026 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.024+0000 7fc40ad6f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:11:22.118 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.116+0000 7fc40ad6f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:11:22.210 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.207+0000 7fc380046700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:11:22.210 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.207+0000 7fc380046700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:11:22.210 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.207+0000 7fc380046700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:11:22.210 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.207+0000 7fc380046700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:11:22.211 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.207+0000 7fc380046700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:11:22.211 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.208+0000 7fc37d040700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:11:22.212 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.208+0000 7fc37d040700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:11:22.212 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.208+0000 7fc37d040700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:11:22.212 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.208+0000 7fc37d040700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:11:22.212 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:11:22.208+0000 7fc37d040700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:11:29.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:11:29.019 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:11:59.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:11:59.141 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:12:00.197 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:12:00.195+0000 7fc388056700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:12:00.198 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:12:00.195+0000 7fc388056700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:12:00.198 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:12:00.195+0000 7fc388056700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:12:00.198 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:12:00.195+0000 7fc388056700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:12:00.198 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:12:00.195+0000 7fc388056700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:12:29.247 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:12:29.253 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:12:59.371 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:12:59.377 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:13:29.506 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:13:29.512 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:13:59.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:13:59.637 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:14:29.868 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:14:29.874 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:14:59.989 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:14:59.995 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:15:30.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:15:30.142 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:16:00.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:16:00.273 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:16:16.868 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/test_restart 1M 2023-11-06T22:16:17.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.078+0000 7f446bbff700 1 Processor -- start 2023-11-06T22:16:17.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.078+0000 7f446bbff700 1 -- start start 2023-11-06T22:16:17.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446bbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4464154340 0x7f44640aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446bbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4464154720 0x7f4464158010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446bbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4464158500 0x7f4464063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446bbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44641589b0 con 0x7f4464154720 2023-11-06T22:16:17.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446bbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44640c0e00 con 0x7f4464158500 2023-11-06T22:16:17.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446bbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4464158bf0 con 0x7f4464154340 2023-11-06T22:16:17.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4464154720 0x7f4464158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4464154720 0x7f4464158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41528/0 (socket says 172.21.15.33:41528) 2023-11-06T22:16:17.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446919a700 1 -- 172.21.15.33:0/2536240940 learned_addr learned my addr 172.21.15.33:0/2536240940 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:17.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446919a700 1 -- 172.21.15.33:0/2536240940 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4464154340 msgr2=0x7f44640aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:17.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446919a700 1 --2- 172.21.15.33:0/2536240940 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4464154340 0x7f44640aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:17.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446919a700 1 -- 172.21.15.33:0/2536240940 >> v2:172.21.15.69:3300/0 conn(0x7f4464158500 msgr2=0x7f4464063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:17.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446919a700 1 --2- 172.21.15.33:0/2536240940 >> v2:172.21.15.69:3300/0 conn(0x7f4464158500 0x7f4464063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:17.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.080+0000 7f446919a700 1 -- 172.21.15.33:0/2536240940 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464158ed0 con 0x7f4464154720 2023-11-06T22:16:17.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f446919a700 1 --2- 172.21.15.33:0/2536240940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154720 0x7f4464158010 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f44640aac10 tx=0x7f44540090f0).ready entity=mon.0 client_cookie=41f9778bba4a3b18 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:17.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f4468999700 1 -- 172.21.15.33:0/2536240940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445400b600 con 0x7f4464154720 2023-11-06T22:16:17.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f4468999700 1 -- 172.21.15.33:0/2536240940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445400b760 con 0x7f4464154720 2023-11-06T22:16:17.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f4468999700 1 -- 172.21.15.33:0/2536240940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445400bae0 con 0x7f4464154720 2023-11-06T22:16:17.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f446bbff700 1 -- 172.21.15.33:0/2536240940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154720 msgr2=0x7f4464158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f446bbff700 1 --2- 172.21.15.33:0/2536240940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154720 0x7f4464158010 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f44640aac10 tx=0x7f44540090f0).stop 2023-11-06T22:16:17.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f446bbff700 1 -- 172.21.15.33:0/2536240940 shutdown_connections 2023-11-06T22:16:17.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f446bbff700 1 --2- 172.21.15.33:0/2536240940 >> v2:172.21.15.69:3300/0 conn(0x7f4464158500 0x7f4464063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:17.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f446bbff700 1 --2- 172.21.15.33:0/2536240940 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4464154340 0x7f44640aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:17.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f446bbff700 1 --2- 172.21.15.33:0/2536240940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154720 0x7f4464158010 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.081+0000 7f446bbff700 1 -- 172.21.15.33:0/2536240940 >> 172.21.15.33:0/2536240940 conn(0x7f446405ec80 msgr2=0x7f44640cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:17.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.082+0000 7f446bbff700 1 -- 172.21.15.33:0/2536240940 shutdown_connections 2023-11-06T22:16:17.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.082+0000 7f446bbff700 1 -- 172.21.15.33:0/2536240940 wait complete. 2023-11-06T22:16:17.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.083+0000 7f446bbff700 1 Processor -- start 2023-11-06T22:16:17.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.083+0000 7f446bbff700 1 -- start start 2023-11-06T22:16:17.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.084+0000 7f446bbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154340 0x7f446414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.084+0000 7f446bbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4464154720 0x7f4464152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.084+0000 7f446bbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4464158500 0x7f446414ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.084+0000 7f446bbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446414e4a0 con 0x7f4464154340 2023-11-06T22:16:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.084+0000 7f446bbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446414e780 con 0x7f4464154720 2023-11-06T22:16:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.084+0000 7f446bbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446414ac40 con 0x7f4464158500 2023-11-06T22:16:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.084+0000 7f446999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154340 0x7f446414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.084+0000 7f446999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154340 0x7f446414a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41536/0 (socket says 172.21.15.33:41536) 2023-11-06T22:16:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.084+0000 7f446999b700 1 -- 172.21.15.33:0/2312583724 learned_addr learned my addr 172.21.15.33:0/2312583724 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446919a700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4464154720 0x7f4464152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446a19c700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4464158500 0x7f446414ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446999b700 1 -- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4464158500 msgr2=0x7f446414ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446999b700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4464158500 0x7f446414ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446999b700 1 -- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4464154720 msgr2=0x7f4464152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446999b700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4464154720 0x7f4464152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446999b700 1 -- 172.21.15.33:0/2312583724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4454009370 con 0x7f4464154340 2023-11-06T22:16:17.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446a19c700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4464158500 0x7f446414ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:17.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446999b700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154340 0x7f446414a3f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f446000b530 tx=0x7f44600090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:17.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f4459ffb700 1 -- 172.21.15.33:0/2312583724 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446000d4a0 con 0x7f4464154340 2023-11-06T22:16:17.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f4459ffb700 1 -- 172.21.15.33:0/2312583724 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446000d620 con 0x7f4464154340 2023-11-06T22:16:17.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f4459ffb700 1 -- 172.21.15.33:0/2312583724 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446000d9e0 con 0x7f4464154340 2023-11-06T22:16:17.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446bbff700 1 -- 172.21.15.33:0/2312583724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446414af20 con 0x7f4464154340 2023-11-06T22:16:17.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.085+0000 7f446bbff700 1 -- 172.21.15.33:0/2312583724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446414a8e0 con 0x7f4464154340 2023-11-06T22:16:17.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.086+0000 7f446bbff700 1 -- 172.21.15.33:0/2312583724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44640c09e0 con 0x7f4464154340 2023-11-06T22:16:17.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.087+0000 7f4459ffb700 1 -- 172.21.15.33:0/2312583724 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 84) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f4460012410 con 0x7f4464154340 2023-11-06T22:16:17.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.088+0000 7f4459ffb700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f444805f290 0x7f4448061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.088+0000 7f4459ffb700 1 -- 172.21.15.33:0/2312583724 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f446007a570 con 0x7f4464154340 2023-11-06T22:16:17.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.089+0000 7f446919a700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f444805f290 0x7f4448061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.089+0000 7f446919a700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f444805f290 0x7f4448061750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f44640aac10 tx=0x7f4454017000).ready entity=mgr.8797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:17.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.093+0000 7f4459ffb700 1 -- 172.21.15.33:0/2312583724 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f446004b450 con 0x7f4464154340 2023-11-06T22:16:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.301+0000 7f446bbff700 1 -- 172.21.15.33:0/2312583724 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f44640c5d80 con 0x7f444805f290 2023-11-06T22:16:17.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.307+0000 7f4459ffb700 1 -- 172.21.15.33:0/2312583724 <== mgr.8797 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f44640c5d80 con 0x7f444805f290 2023-11-06T22:16:17.310 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-11-06T22:16:17.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 -- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f444805f290 msgr2=0x7f4448061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f444805f290 0x7f4448061750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f44640aac10 tx=0x7f4454017000).stop 2023-11-06T22:16:17.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 -- 172.21.15.33:0/2312583724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154340 msgr2=0x7f446414a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154340 0x7f446414a3f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f446000b530 tx=0x7f44600090f0).stop 2023-11-06T22:16:17.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 -- 172.21.15.33:0/2312583724 shutdown_connections 2023-11-06T22:16:17.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f444805f290 0x7f4448061750 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4464158500 0x7f446414ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464154340 0x7f446414a3f0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 --2- 172.21.15.33:0/2312583724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4464154720 0x7f4464152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 -- 172.21.15.33:0/2312583724 >> 172.21.15.33:0/2312583724 conn(0x7f446405ec80 msgr2=0x7f446414ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:17.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 -- 172.21.15.33:0/2312583724 shutdown_connections 2023-11-06T22:16:17.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.309+0000 7f446bbff700 1 -- 172.21.15.33:0/2312583724 wait complete. 2023-11-06T22:16:17.329 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-11-06T22:16:17.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:17.330 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-11-06T22:16:17.379 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_01_00_UTC 2023-11-06T22:16:17.380 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_02_00_UTC 2023-11-06T22:16:17.380 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_03_00_UTC 2023-11-06T22:16:17.380 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_04_00_UTC 2023-11-06T22:16:17.380 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_05_00_UTC 2023-11-06T22:16:17.380 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_06_00_UTC 2023-11-06T22:16:17.381 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_07_00_UTC 2023-11-06T22:16:17.381 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_08_00_UTC 2023-11-06T22:16:17.381 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_09_00_UTC 2023-11-06T22:16:17.381 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_10_00_UTC 2023-11-06T22:16:17.381 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_11_00_UTC 2023-11-06T22:16:17.382 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_12_00_UTC 2023-11-06T22:16:17.382 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_13_00_UTC 2023-11-06T22:16:17.382 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_14_00_UTC 2023-11-06T22:16:17.382 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_15_00_UTC 2023-11-06T22:16:17.382 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_16_00_UTC 2023-11-06T22:16:17.384 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-06-22_01_00_UTC', 'scheduled-2023-11-06-22_02_00_UTC', 'scheduled-2023-11-06-22_03_00_UTC', 'scheduled-2023-11-06-22_04_00_UTC', 'scheduled-2023-11-06-22_05_00_UTC', 'scheduled-2023-11-06-22_06_00_UTC', 'scheduled-2023-11-06-22_07_00_UTC', 'scheduled-2023-11-06-22_08_00_UTC', 'scheduled-2023-11-06-22_09_00_UTC', 'scheduled-2023-11-06-22_10_00_UTC', 'scheduled-2023-11-06-22_11_00_UTC', 'scheduled-2023-11-06-22_12_00_UTC', 'scheduled-2023-11-06-22_13_00_UTC', 'scheduled-2023-11-06-22_14_00_UTC', 'scheduled-2023-11-06-22_15_00_UTC', 'scheduled-2023-11-06-22_16_00_UTC'] 2023-11-06T22:16:17.384 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-11-06T22:16:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.581+0000 7faf277ff700 1 Processor -- start 2023-11-06T22:16:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.581+0000 7faf277ff700 1 -- start start 2023-11-06T22:16:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.582+0000 7faf277ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200acfc0 0x7faf200ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.582+0000 7faf277ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf201586a0 con 0x7faf201550d0 2023-11-06T22:16:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.582+0000 7faf277ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf201544c0 con 0x7faf200638b0 2023-11-06T22:16:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.582+0000 7faf277ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf200c6590 con 0x7faf200acfc0 2023-11-06T22:16:17.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.582+0000 7faf24d9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faf201550d0 0x7faf200631a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42648/0 (socket says 172.21.15.33:42648) 2023-11-06T22:16:17.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.582+0000 7faf24d9a700 1 -- 172.21.15.33:0/1293400453 learned_addr learned my addr 172.21.15.33:0/1293400453 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf2559b700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200acfc0 0x7faf200ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887497047 0 0) 0x7faf201586a0 con 0x7faf201550d0 2023-11-06T22:16:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf0c0033f0 con 0x7faf201550d0 2023-11-06T22:16:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2407351120 0 0) 0x7faf0c0033f0 con 0x7faf201550d0 2023-11-06T22:16:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200acfc0 msgr2=0x7faf200ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200acfc0 0x7faf200ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 >> v1:172.21.15.69:6789/0 conn(0x7faf200638b0 legacy=0x7faf200c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf20155550 con 0x7faf201550d0 2023-11-06T22:16:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7faf10001470 con 0x7faf201550d0 2023-11-06T22:16:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 >> v1:172.21.15.33:6789/0 conn(0x7faf201550d0 legacy=0x7faf200631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf0c003910 0x7faf0c003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200acfc0 0x7faf0c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c004b80 0x7faf0c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf200c6590 con 0x7faf0c004b80 2023-11-06T22:16:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c00b8e0 con 0x7faf0c003910 2023-11-06T22:16:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.583+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c00ba50 con 0x7faf200acfc0 2023-11-06T22:16:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.584+0000 7faf25d9c700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c004b80 0x7faf0c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.584+0000 7faf24d9a700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200acfc0 0x7faf0c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.584+0000 7faf2559b700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf0c003910 0x7faf0c003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.584+0000 7faf25d9c700 1 -- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200acfc0 msgr2=0x7faf0c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.584+0000 7faf25d9c700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf200acfc0 0x7faf0c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.584+0000 7faf25d9c700 1 -- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf0c003910 msgr2=0x7faf0c003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.584+0000 7faf25d9c700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf0c003910 0x7faf0c003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.584+0000 7faf25d9c700 1 -- 172.21.15.33:0/1293400453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0033f0 con 0x7faf0c004b80 2023-11-06T22:16:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.584+0000 7faf25d9c700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c004b80 0x7faf0c005000 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7faf20063cb0 tx=0x7faf18007f10).ready entity=mon.0 client_cookie=2fb10952e0f2df36 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.585+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180017e0 con 0x7faf0c004b80 2023-11-06T22:16:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.585+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1800c640 con 0x7faf0c004b80 2023-11-06T22:16:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.585+0000 7faf17fff700 1 -- 172.21.15.33:0/1293400453 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800c9c0 con 0x7faf0c004b80 2023-11-06T22:16:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.585+0000 7faf277ff700 1 -- 172.21.15.33:0/1293400453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c004b80 msgr2=0x7faf0c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.585+0000 7faf277ff700 1 --2- 172.21.15.33:0/1293400453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c004b80 0x7faf0c005000 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7faf20063cb0 tx=0x7faf18007f10).stop 2023-11-06T22:16:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.585+0000 7faf25d9c700 1 -- 172.21.15.33:0/1293400453 reap_dead start 2023-11-06T22:16:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.585+0000 7faf277ff700 1 -- 172.21.15.33:0/1293400453 shutdown_connections 2023-11-06T22:16:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.585+0000 7faf277ff700 1 -- 172.21.15.33:0/1293400453 >> 172.21.15.33:0/1293400453 conn(0x7faf2005ec80 msgr2=0x7faf200cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.585+0000 7faf277ff700 1 -- 172.21.15.33:0/1293400453 shutdown_connections 2023-11-06T22:16:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.585+0000 7faf277ff700 1 -- 172.21.15.33:0/1293400453 wait complete. 2023-11-06T22:16:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.587+0000 7faf277ff700 1 Processor -- start 2023-11-06T22:16:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.587+0000 7faf277ff700 1 -- start start 2023-11-06T22:16:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf277ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c004b80 0x7faf20155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf277ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20164990 0x7faf200c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf277ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c3cf0 0x7faf20159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf277ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf20156b50 con 0x7faf20164990 2023-11-06T22:16:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf277ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf20155f40 con 0x7faf200c3cf0 2023-11-06T22:16:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf277ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf20156220 con 0x7faf0c004b80 2023-11-06T22:16:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf24d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20164990 0x7faf200c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf24d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20164990 0x7faf200c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41548/0 (socket says 172.21.15.33:41548) 2023-11-06T22:16:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf24d9a700 1 -- 172.21.15.33:0/3087635711 learned_addr learned my addr 172.21.15.33:0/3087635711 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf2559b700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c004b80 0x7faf20155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.588+0000 7faf25d9c700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c3cf0 0x7faf20159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf24d9a700 1 -- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c004b80 msgr2=0x7faf20155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf24d9a700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c004b80 0x7faf20155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf24d9a700 1 -- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c3cf0 msgr2=0x7faf20159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf24d9a700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c3cf0 0x7faf20159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf24d9a700 1 -- 172.21.15.33:0/3087635711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf20156500 con 0x7faf20164990 2023-11-06T22:16:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf25d9c700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c3cf0 0x7faf20159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf24d9a700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20164990 0x7faf200c3800 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7faf10007fb0 tx=0x7faf200c6fd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf157fa700 1 -- 172.21.15.33:0/3087635711 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10011590 con 0x7faf20164990 2023-11-06T22:16:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf157fa700 1 -- 172.21.15.33:0/3087635711 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf100116f0 con 0x7faf20164990 2023-11-06T22:16:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf157fa700 1 -- 172.21.15.33:0/3087635711 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10011a70 con 0x7faf20164990 2023-11-06T22:16:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf277ff700 1 -- 172.21.15.33:0/3087635711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf201567e0 con 0x7faf20164990 2023-11-06T22:16:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.589+0000 7faf277ff700 1 -- 172.21.15.33:0/3087635711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2015a1f0 con 0x7faf20164990 2023-11-06T22:16:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.590+0000 7faf277ff700 1 -- 172.21.15.33:0/3087635711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf20156ee0 con 0x7faf20164990 2023-11-06T22:16:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.591+0000 7faf157fa700 1 -- 172.21.15.33:0/3087635711 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 84) v1 ==== 75577+0+0 (secure 0 0 0) 0x7faf1001d3b0 con 0x7faf20164990 2023-11-06T22:16:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.592+0000 7faf157fa700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf0405f2c0 0x7faf04061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.592+0000 7faf157fa700 1 -- 172.21.15.33:0/3087635711 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6653+0+0 (secure 0 0 0) 0x7faf10079f50 con 0x7faf20164990 2023-11-06T22:16:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.593+0000 7faf2559b700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf0405f2c0 0x7faf04061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.593+0000 7faf2559b700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf0405f2c0 0x7faf04061780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7faf1c006d50 tx=0x7faf1c008040).ready entity=mgr.8797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.596+0000 7faf157fa700 1 -- 172.21.15.33:0/3087635711 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf1004ae20 con 0x7faf20164990 2023-11-06T22:16:17.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.803+0000 7faf277ff700 1 -- 172.21.15.33:0/3087635711 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf2006c6c0 con 0x7faf0405f2c0 2023-11-06T22:16:17.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.806+0000 7faf157fa700 1 -- 172.21.15.33:0/3087635711 <== mgr.8797 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7faf2006c6c0 con 0x7faf0405f2c0 2023-11-06T22:16:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:17.808 INFO:teuthology.orchestra.run.smithi033.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T22:00:33", "first": "2023-11-06T22:01:00", "last": "2023-11-06T22:16:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-11-06T22:16:17.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.807+0000 7faf277ff700 1 -- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf0405f2c0 msgr2=0x7faf04061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.807+0000 7faf277ff700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf0405f2c0 0x7faf04061780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7faf1c006d50 tx=0x7faf1c008040).stop 2023-11-06T22:16:17.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.807+0000 7faf277ff700 1 -- 172.21.15.33:0/3087635711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20164990 msgr2=0x7faf200c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:17.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.807+0000 7faf277ff700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20164990 0x7faf200c3800 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7faf10007fb0 tx=0x7faf200c6fd0).stop 2023-11-06T22:16:17.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.807+0000 7faf277ff700 1 -- 172.21.15.33:0/3087635711 shutdown_connections 2023-11-06T22:16:17.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.807+0000 7faf277ff700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf0405f2c0 0x7faf04061780 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.807+0000 7faf277ff700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c004b80 0x7faf20155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.807+0000 7faf277ff700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf200c3cf0 0x7faf20159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.807+0000 7faf277ff700 1 --2- 172.21.15.33:0/3087635711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf20164990 0x7faf200c3800 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.807+0000 7faf277ff700 1 -- 172.21.15.33:0/3087635711 >> 172.21.15.33:0/3087635711 conn(0x7faf2005ec80 msgr2=0x7faf20154670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.808+0000 7faf277ff700 1 -- 172.21.15.33:0/3087635711 shutdown_connections 2023-11-06T22:16:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:17.808+0000 7faf277ff700 1 -- 172.21.15.33:0/3087635711 wait complete. 2023-11-06T22:16:17.828 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-11-06T00:00:00", "created": "2023-11-06T22:00:33", "first": "2023-11-06T22:01:00", "last": "2023-11-06T22:16:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-11-06T22:16:17.828 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-11-06T00:00:00', 'created': '2023-11-06T22:00:33', 'first': '2023-11-06T22:01:00', 'last': '2023-11-06T22:16:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-11-06T22:16:17.829 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-11-06T22:16:17.829 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-11-06T22:16:17.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/test_restart 1M 2023-11-06T22:16:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.027+0000 7fc1b71ff700 1 Processor -- start 2023-11-06T22:16:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.027+0000 7fc1b71ff700 1 -- start start 2023-11-06T22:16:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1b71ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1b013a960 0x7fc1b0155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1b71ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b0063950 con 0x7fc1b00c9340 2023-11-06T22:16:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1b71ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b0063c30 con 0x7fc1b0066ba0 2023-11-06T22:16:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1b71ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b0064cc0 con 0x7fc1b013a960 2023-11-06T22:16:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1b4f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1b013a960 0x7fc1b0155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1affff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc1b00c9340 0x7fc1b0066490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35222/0 (socket says 172.21.15.33:35222) 2023-11-06T22:16:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1affff700 1 -- 172.21.15.33:0/105435323 learned_addr learned my addr 172.21.15.33:0/105435323 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1b4f9b700 1 -- 172.21.15.33:0/105435323 >> v1:172.21.15.69:6789/0 conn(0x7fc1b0066ba0 legacy=0x7fc1b00c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1b4f9b700 1 -- 172.21.15.33:0/105435323 >> v1:172.21.15.33:6789/0 conn(0x7fc1b00c9340 legacy=0x7fc1b0066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1b4f9b700 1 -- 172.21.15.33:0/105435323 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b013ad40 con 0x7fc1b013a960 2023-11-06T22:16:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.029+0000 7fc1af7fe700 1 -- 172.21.15.33:0/105435323 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767658549 0 0) 0x7fc1b0063950 con 0x7fc1b00c9340 2023-11-06T22:16:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.030+0000 7fc1b4f9b700 1 --2- 172.21.15.33:0/105435323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b013a960 0x7fc1b0155380 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7fc1a4009760 tx=0x7fc1a4009060).ready entity=mon.0 client_cookie=c02ce61186b8cb2c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.030+0000 7fc1af7fe700 1 -- 172.21.15.33:0/105435323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400b580 con 0x7fc1b013a960 2023-11-06T22:16:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.030+0000 7fc1af7fe700 1 -- 172.21.15.33:0/105435323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a400b6e0 con 0x7fc1b013a960 2023-11-06T22:16:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.030+0000 7fc1af7fe700 1 -- 172.21.15.33:0/105435323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400ba60 con 0x7fc1b013a960 2023-11-06T22:16:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.030+0000 7fc1b71ff700 1 -- 172.21.15.33:0/105435323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b013a960 msgr2=0x7fc1b0155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.030+0000 7fc1b71ff700 1 --2- 172.21.15.33:0/105435323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b013a960 0x7fc1b0155380 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7fc1a4009760 tx=0x7fc1a4009060).stop 2023-11-06T22:16:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.030+0000 7fc1b71ff700 1 -- 172.21.15.33:0/105435323 shutdown_connections 2023-11-06T22:16:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.030+0000 7fc1b71ff700 1 --2- 172.21.15.33:0/105435323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b013a960 0x7fc1b0155380 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.030+0000 7fc1b71ff700 1 -- 172.21.15.33:0/105435323 >> 172.21.15.33:0/105435323 conn(0x7fc1b005ec70 msgr2=0x7fc1b00c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.031+0000 7fc1b71ff700 1 -- 172.21.15.33:0/105435323 shutdown_connections 2023-11-06T22:16:18.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.031+0000 7fc1b71ff700 1 -- 172.21.15.33:0/105435323 wait complete. 2023-11-06T22:16:18.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.032+0000 7fc1b71ff700 1 Processor -- start 2023-11-06T22:16:18.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.032+0000 7fc1b71ff700 1 -- start start 2023-11-06T22:16:18.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.033+0000 7fc1b71ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1b0066ba0 0x7fc1b01678d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.033+0000 7fc1b71ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b00c9340 0x7fc1b00a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.033+0000 7fc1b71ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b013a960 0x7fc1b00a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.033+0000 7fc1b71ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0155930 con 0x7fc1b00c9340 2023-11-06T22:16:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.033+0000 7fc1b71ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00a6ff0 con 0x7fc1b013a960 2023-11-06T22:16:18.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.033+0000 7fc1b71ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00a6370 con 0x7fc1b0066ba0 2023-11-06T22:16:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.033+0000 7fc1affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b00c9340 0x7fc1b00a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.033+0000 7fc1affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b00c9340 0x7fc1b00a58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56610/0 (socket says 172.21.15.33:56610) 2023-11-06T22:16:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.033+0000 7fc1affff700 1 -- 172.21.15.33:0/1396077359 learned_addr learned my addr 172.21.15.33:0/1396077359 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1b4f9b700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1b0066ba0 0x7fc1b01678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1b579c700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b013a960 0x7fc1b00a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1affff700 1 -- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1b0066ba0 msgr2=0x7fc1b01678d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1affff700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1b0066ba0 0x7fc1b01678d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1affff700 1 -- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b013a960 msgr2=0x7fc1b00a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1affff700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b013a960 0x7fc1b00a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1affff700 1 -- 172.21.15.33:0/1396077359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40092a0 con 0x7fc1b00c9340 2023-11-06T22:16:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1b579c700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b013a960 0x7fc1b00a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1affff700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b00c9340 0x7fc1b00a58b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7fc19c00c4c0 tx=0x7fc19c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1acff9700 1 -- 172.21.15.33:0/1396077359 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0072e0 con 0x7fc1b00c9340 2023-11-06T22:16:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1acff9700 1 -- 172.21.15.33:0/1396077359 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c007440 con 0x7fc1b00c9340 2023-11-06T22:16:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1acff9700 1 -- 172.21.15.33:0/1396077359 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0057a0 con 0x7fc1b00c9340 2023-11-06T22:16:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1b71ff700 1 -- 172.21.15.33:0/1396077359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b00a66b0 con 0x7fc1b00c9340 2023-11-06T22:16:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.034+0000 7fc1b71ff700 1 -- 172.21.15.33:0/1396077359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b00a8f50 con 0x7fc1b00c9340 2023-11-06T22:16:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.036+0000 7fc1b71ff700 1 -- 172.21.15.33:0/1396077359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b00bfa40 con 0x7fc1b00c9340 2023-11-06T22:16:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.036+0000 7fc1acff9700 1 -- 172.21.15.33:0/1396077359 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 84) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fc19c005b90 con 0x7fc1b00c9340 2023-11-06T22:16:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.037+0000 7fc1acff9700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc19405f2c0 0x7fc194061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.038+0000 7fc1acff9700 1 -- 172.21.15.33:0/1396077359 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6653+0+0 (secure 0 0 0) 0x7fc19c079d60 con 0x7fc1b00c9340 2023-11-06T22:16:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.038+0000 7fc1b4f9b700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc19405f2c0 0x7fc194061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.038+0000 7fc1b4f9b700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc19405f2c0 0x7fc194061780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc1a400b340 tx=0x7fc1a4017000).ready entity=mgr.8797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.041+0000 7fc1acff9700 1 -- 172.21.15.33:0/1396077359 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc19c04aca0 con 0x7fc1b00c9340 2023-11-06T22:16:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.254+0000 7fc1b71ff700 1 -- 172.21.15.33:0/1396077359 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1b0065f30 con 0x7fc19405f2c0 2023-11-06T22:16:18.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.260+0000 7fc1acff9700 1 -- 172.21.15.33:0/1396077359 <== mgr.8797 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7fc1b0065f30 con 0x7fc19405f2c0 2023-11-06T22:16:18.262 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-11-06T22:16:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 -- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc19405f2c0 msgr2=0x7fc194061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc19405f2c0 0x7fc194061780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc1a400b340 tx=0x7fc1a4017000).stop 2023-11-06T22:16:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 -- 172.21.15.33:0/1396077359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b00c9340 msgr2=0x7fc1b00a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:18.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b00c9340 0x7fc1b00a58b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7fc19c00c4c0 tx=0x7fc19c007c10).stop 2023-11-06T22:16:18.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 -- 172.21.15.33:0/1396077359 shutdown_connections 2023-11-06T22:16:18.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc19405f2c0 0x7fc194061780 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:18.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc1b0066ba0 0x7fc1b01678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b013a960 0x7fc1b00a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 --2- 172.21.15.33:0/1396077359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b00c9340 0x7fc1b00a58b0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 -- 172.21.15.33:0/1396077359 >> 172.21.15.33:0/1396077359 conn(0x7fc1b005ec70 msgr2=0x7fc1b00c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 -- 172.21.15.33:0/1396077359 shutdown_connections 2023-11-06T22:16:18.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:18.262+0000 7fc1b71ff700 1 -- 172.21.15.33:0/1396077359 wait complete. 2023-11-06T22:16:18.282 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-11-06T22:16:18.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.283 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-11-06T22:16:18.332 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_01_00_UTC 2023-11-06T22:16:18.332 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_02_00_UTC 2023-11-06T22:16:18.333 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_03_00_UTC 2023-11-06T22:16:18.333 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_04_00_UTC 2023-11-06T22:16:18.333 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_05_00_UTC 2023-11-06T22:16:18.333 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_06_00_UTC 2023-11-06T22:16:18.333 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_07_00_UTC 2023-11-06T22:16:18.334 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_08_00_UTC 2023-11-06T22:16:18.334 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_09_00_UTC 2023-11-06T22:16:18.334 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_10_00_UTC 2023-11-06T22:16:18.334 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_11_00_UTC 2023-11-06T22:16:18.335 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_12_00_UTC 2023-11-06T22:16:18.335 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_13_00_UTC 2023-11-06T22:16:18.335 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_14_00_UTC 2023-11-06T22:16:18.335 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_15_00_UTC 2023-11-06T22:16:18.336 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_16_00_UTC 2023-11-06T22:16:18.337 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_01_00_UTC 2023-11-06T22:16:18.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.338 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_01_00_UTC) 2023-11-06T22:16:18.394 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_02_00_UTC 2023-11-06T22:16:18.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.394 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_02_00_UTC) 2023-11-06T22:16:18.452 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_03_00_UTC 2023-11-06T22:16:18.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.452 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_03_00_UTC) 2023-11-06T22:16:18.512 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_04_00_UTC 2023-11-06T22:16:18.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.513 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_04_00_UTC) 2023-11-06T22:16:18.572 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_05_00_UTC 2023-11-06T22:16:18.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.573 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_05_00_UTC) 2023-11-06T22:16:18.628 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_06_00_UTC 2023-11-06T22:16:18.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.629 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_06_00_UTC) 2023-11-06T22:16:18.685 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_07_00_UTC 2023-11-06T22:16:18.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.686 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_07_00_UTC) 2023-11-06T22:16:18.742 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_08_00_UTC 2023-11-06T22:16:18.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.743 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_08_00_UTC) 2023-11-06T22:16:18.798 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_09_00_UTC 2023-11-06T22:16:18.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.799 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_09_00_UTC) 2023-11-06T22:16:18.855 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_10_00_UTC 2023-11-06T22:16:18.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.856 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_10_00_UTC) 2023-11-06T22:16:18.914 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_11_00_UTC 2023-11-06T22:16:18.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.915 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_11_00_UTC) 2023-11-06T22:16:18.975 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_12_00_UTC 2023-11-06T22:16:18.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:18.976 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_12_00_UTC) 2023-11-06T22:16:19.030 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_13_00_UTC 2023-11-06T22:16:19.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:19.031 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_13_00_UTC) 2023-11-06T22:16:19.088 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_14_00_UTC 2023-11-06T22:16:19.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:19.089 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_14_00_UTC) 2023-11-06T22:16:19.147 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_15_00_UTC 2023-11-06T22:16:19.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:19.148 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_15_00_UTC) 2023-11-06T22:16:19.203 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_16_00_UTC 2023-11-06T22:16:19.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:19.204 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-06-22_16_00_UTC) 2023-11-06T22:16:19.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:19.259 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-11-06T22:16:19.311 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T22:16:19.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.507+0000 7fbd1c5ff700 1 Processor -- start 2023-11-06T22:16:19.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.507+0000 7fbd1c5ff700 1 -- start start 2023-11-06T22:16:19.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.508+0000 7fbd1c5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbd140aaf50 0x7fbd14063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:19.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.508+0000 7fbd1c5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd14158c00 con 0x7fbd140aa4f0 2023-11-06T22:16:19.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.508+0000 7fbd1c5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd140c7610 con 0x7fbd14157700 2023-11-06T22:16:19.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.508+0000 7fbd1c5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd140c18f0 con 0x7fbd140aaf50 2023-11-06T22:16:19.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.508+0000 7fbd1ab9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbd140aa4f0 0x7fbd140c5d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35226/0 (socket says 172.21.15.33:35226) 2023-11-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd1ab9c700 1 -- 172.21.15.33:0/3947148373 learned_addr learned my addr 172.21.15.33:0/3947148373 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19b9a700 1 --2- 172.21.15.33:0/3947148373 >> v2:172.21.15.69:3300/0 conn(0x7fbd140aaf50 0x7fbd14063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249867629 0 0) 0x7fbd14158c00 con 0x7fbd140aa4f0 2023-11-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd000033f0 con 0x7fbd140aa4f0 2023-11-06T22:16:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677320208 0 0) 0x7fbd140c7610 con 0x7fbd14157700 2023-11-06T22:16:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd14158c00 con 0x7fbd14157700 2023-11-06T22:16:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2123411347 0 0) 0x7fbd000033f0 con 0x7fbd140aa4f0 2023-11-06T22:16:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 >> v2:172.21.15.69:3300/0 conn(0x7fbd140aaf50 msgr2=0x7fbd14063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 --2- 172.21.15.33:0/3947148373 >> v2:172.21.15.69:3300/0 conn(0x7fbd140aaf50 0x7fbd14063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 >> v1:172.21.15.69:6789/0 conn(0x7fbd14157700 legacy=0x7fbd140c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd14157ff0 con 0x7fbd140aa4f0 2023-11-06T22:16:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fbd0c0017f0 con 0x7fbd140aa4f0 2023-11-06T22:16:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 >> v1:172.21.15.33:6789/0 conn(0x7fbd140aa4f0 legacy=0x7fbd140c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:19.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd00003a70 0x7fbd00003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:19.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbd00004690 0x7fbd00004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:19.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd00005000 0x7fbd0000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:19.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140c18f0 con 0x7fbd00003a70 2023-11-06T22:16:19.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.509+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0000be80 con 0x7fbd00005000 2023-11-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0000c040 con 0x7fbd00004690 2023-11-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd1a39b700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd00003a70 0x7fbd00003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd19b9a700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbd00004690 0x7fbd00004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd1ab9c700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd00005000 0x7fbd0000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd1a39b700 1 -- 172.21.15.33:0/3947148373 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbd00004690 msgr2=0x7fbd00004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd1a39b700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbd00004690 0x7fbd00004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd1a39b700 1 -- 172.21.15.33:0/3947148373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd00005000 msgr2=0x7fbd0000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd1a39b700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd00005000 0x7fbd0000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd1a39b700 1 -- 172.21.15.33:0/3947148373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd000033f0 con 0x7fbd00003a70 2023-11-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd1ab9c700 1 -- 172.21.15.33:0/3947148373 reap_dead start 2023-11-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd1a39b700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd00003a70 0x7fbd00003e70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7fbd10001860 tx=0x7fbd10007bb0).ready entity=mon.0 client_cookie=31f94d1884848924 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.510+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10007020 con 0x7fbd00003a70 2023-11-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.511+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd10007180 con 0x7fbd00003a70 2023-11-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.511+0000 7fbd19399700 1 -- 172.21.15.33:0/3947148373 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10007500 con 0x7fbd00003a70 2023-11-06T22:16:19.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.511+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/3947148373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd00003a70 msgr2=0x7fbd00003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:19.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.511+0000 7fbd1c5ff700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd00003a70 0x7fbd00003e70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7fbd10001860 tx=0x7fbd10007bb0).stop 2023-11-06T22:16:19.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.511+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/3947148373 shutdown_connections 2023-11-06T22:16:19.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.511+0000 7fbd1c5ff700 1 --2- 172.21.15.33:0/3947148373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd00003a70 0x7fbd00003e70 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:19.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.511+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/3947148373 >> 172.21.15.33:0/3947148373 conn(0x7fbd1405ec80 msgr2=0x7fbd140cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:19.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.511+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/3947148373 shutdown_connections 2023-11-06T22:16:19.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.511+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/3947148373 wait complete. 2023-11-06T22:16:19.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.513+0000 7fbd1c5ff700 1 Processor -- start 2023-11-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.513+0000 7fbd1c5ff700 1 -- start start 2023-11-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd1c5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbd00003a70 0x7fbd14168050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd1c5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd140acbf0 0x7fbd140ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:19.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd1c5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd14147270 0x7fbd141476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:19.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd1c5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140c2c10 con 0x7fbd140acbf0 2023-11-06T22:16:19.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd1c5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140ad560 con 0x7fbd14147270 2023-11-06T22:16:19.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd1c5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140c1f90 con 0x7fbd00003a70 2023-11-06T22:16:19.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd19b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd140acbf0 0x7fbd140ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:19.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd19b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd140acbf0 0x7fbd140ad070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56622/0 (socket says 172.21.15.33:56622) 2023-11-06T22:16:19.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd19b9a700 1 -- 172.21.15.33:0/2383373786 learned_addr learned my addr 172.21.15.33:0/2383373786 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:19.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd1a39b700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbd00003a70 0x7fbd14168050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:19.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd1ab9c700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd14147270 0x7fbd141476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:19.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd19b9a700 1 -- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbd00003a70 msgr2=0x7fbd14168050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:19.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd19b9a700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbd00003a70 0x7fbd14168050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:19.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd19b9a700 1 -- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd14147270 msgr2=0x7fbd141476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:19.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd19b9a700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd14147270 0x7fbd141476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:19.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.514+0000 7fbd19b9a700 1 -- 172.21.15.33:0/2383373786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd140c2270 con 0x7fbd140acbf0 2023-11-06T22:16:19.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.515+0000 7fbd19b9a700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd140acbf0 0x7fbd140ad070 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7fbd040119d0 tx=0x7fbd04009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:19.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.515+0000 7fbd0a7fc700 1 -- 172.21.15.33:0/2383373786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400f940 con 0x7fbd140acbf0 2023-11-06T22:16:19.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.515+0000 7fbd0a7fc700 1 -- 172.21.15.33:0/2383373786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0400faa0 con 0x7fbd140acbf0 2023-11-06T22:16:19.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.515+0000 7fbd0a7fc700 1 -- 172.21.15.33:0/2383373786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400fe20 con 0x7fbd140acbf0 2023-11-06T22:16:19.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.515+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/2383373786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd140c2550 con 0x7fbd140acbf0 2023-11-06T22:16:19.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.515+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/2383373786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd14147be0 con 0x7fbd140acbf0 2023-11-06T22:16:19.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.516+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/2383373786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1406c6c0 con 0x7fbd140acbf0 2023-11-06T22:16:19.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.517+0000 7fbd0a7fc700 1 -- 172.21.15.33:0/2383373786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 84) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fbd04014650 con 0x7fbd140acbf0 2023-11-06T22:16:19.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.518+0000 7fbd0a7fc700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbcf805f290 0x7fbcf8061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:19.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.518+0000 7fbd0a7fc700 1 -- 172.21.15.33:0/2383373786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6653+0+0 (secure 0 0 0) 0x7fbd0407b190 con 0x7fbd140acbf0 2023-11-06T22:16:19.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.518+0000 7fbd1a39b700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbcf805f290 0x7fbcf8061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:19.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.519+0000 7fbd1a39b700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbcf805f290 0x7fbcf8061750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fbd10000c00 tx=0x7fbd10015000).ready entity=mgr.8797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:19.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.523+0000 7fbd0a7fc700 1 -- 172.21.15.33:0/2383373786 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbd0404c010 con 0x7fbd140acbf0 2023-11-06T22:16:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:19.749+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/2383373786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fbd140bfe50 con 0x7fbd140acbf0 2023-11-06T22:16:20.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.259+0000 7fbd0a7fc700 1 -- 172.21.15.33:0/2383373786 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fbd04078b10 con 0x7fbd140acbf0 2023-11-06T22:16:20.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.261+0000 7fbd0a7fc700 1 -- 172.21.15.33:0/2383373786 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v85) v1 ==== 93+0+0 (secure 0 0 0) 0x7fbd0406fe40 con 0x7fbd140acbf0 2023-11-06T22:16:20.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbcf805f290 msgr2=0x7fbcf8061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:20.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbcf805f290 0x7fbcf8061750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fbd10000c00 tx=0x7fbd10015000).stop 2023-11-06T22:16:20.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/2383373786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd140acbf0 msgr2=0x7fbd140ad070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:20.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd140acbf0 0x7fbd140ad070 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7fbd040119d0 tx=0x7fbd04009060).stop 2023-11-06T22:16:20.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/2383373786 shutdown_connections 2023-11-06T22:16:20.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbcf805f290 0x7fbcf8061750 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:20.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbd00003a70 0x7fbd14168050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:20.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd14147270 0x7fbd141476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 --2- 172.21.15.33:0/2383373786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd140acbf0 0x7fbd140ad070 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/2383373786 >> 172.21.15.33:0/2383373786 conn(0x7fbd1405ec80 msgr2=0x7fbd140c6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/2383373786 shutdown_connections 2023-11-06T22:16:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.264+0000 7fbd1c5ff700 1 -- 172.21.15.33:0/2383373786 wait complete. 2023-11-06T22:16:20.286 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T22:16:20.287 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:16:20.391 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:16:20.483 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:20.481+0000 7f5957778680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:16:20.488 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T22:16:20.488 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T22:16:20.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:20.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T22:16:20.490 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:20.488+0000 7ff8aec72680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:16:20.567 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:20.566+0000 7f5957778680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:16:20.570 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:20.569+0000 7ff8aec72680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:16:20.573 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T22:16:20.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:16:20.574 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T22:16:20.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:16:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.824+0000 7f0806dff700 1 Processor -- start 2023-11-06T22:16:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.824+0000 7f0806dff700 1 -- start start 2023-11-06T22:16:20.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.826+0000 7f0806dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f080014c640 0x7f08000bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.826+0000 7f0806dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f08000c33f0 0x7f08000c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.826+0000 7f0806dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0800153470 con 0x7f0800152ff0 2023-11-06T22:16:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.826+0000 7f0806dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08000bd8f0 con 0x7f080014c640 2023-11-06T22:16:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.826+0000 7f0806dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08000643a0 con 0x7f08000c33f0 2023-11-06T22:16:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.826+0000 7f0804b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f080014c640 0x7f08000bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.826+0000 7f0804b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f080014c640 0x7f08000bb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56656/0 (socket says 172.21.15.33:56656) 2023-11-06T22:16:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.826+0000 7f0804b9b700 1 -- 172.21.15.33:0/1006399828 learned_addr learned my addr 172.21.15.33:0/1006399828 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.826+0000 7f07fffff700 1 --1- 172.21.15.33:0/1006399828 >> v1:172.21.15.33:6789/0 conn(0x7f0800152ff0 0x7f0800065160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35228/0 (socket says 172.21.15.33:35228) 2023-11-06T22:16:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.826+0000 7f080539c700 1 --2- 172.21.15.33:0/1006399828 >> v2:172.21.15.69:3300/0 conn(0x7f08000c33f0 0x7f08000c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f07ff7fe700 1 -- 172.21.15.33:0/1006399828 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281346584 0 0) 0x7f0800153470 con 0x7f0800152ff0 2023-11-06T22:16:20.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f07ff7fe700 1 -- 172.21.15.33:0/1006399828 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f00033f0 con 0x7f0800152ff0 2023-11-06T22:16:20.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f0804b9b700 1 -- 172.21.15.33:0/1006399828 >> v2:172.21.15.69:3300/0 conn(0x7f08000c33f0 msgr2=0x7f08000c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:20.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f0804b9b700 1 --2- 172.21.15.33:0/1006399828 >> v2:172.21.15.69:3300/0 conn(0x7f08000c33f0 0x7f08000c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:20.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f0804b9b700 1 -- 172.21.15.33:0/1006399828 >> v1:172.21.15.33:6789/0 conn(0x7f0800152ff0 legacy=0x7f0800065160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:20.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f0804b9b700 1 -- 172.21.15.33:0/1006399828 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800064680 con 0x7f080014c640 2023-11-06T22:16:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f080539c700 1 --2- 172.21.15.33:0/1006399828 >> v2:172.21.15.69:3300/0 conn(0x7f08000c33f0 0x7f08000c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f0804b9b700 1 --2- 172.21.15.33:0/1006399828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 0x7f08000bb990 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f08000c0fc0 tx=0x7f07f40090f0).ready entity=mon.0 client_cookie=bf019ea48aa91bd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f07ff7fe700 1 -- 172.21.15.33:0/1006399828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f400f510 con 0x7f080014c640 2023-11-06T22:16:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f07ff7fe700 1 -- 172.21.15.33:0/1006399828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f400f670 con 0x7f080014c640 2023-11-06T22:16:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.827+0000 7f07ff7fe700 1 -- 172.21.15.33:0/1006399828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f400f9f0 con 0x7f080014c640 2023-11-06T22:16:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.828+0000 7f0806dff700 1 -- 172.21.15.33:0/1006399828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 msgr2=0x7f08000bb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.828+0000 7f0806dff700 1 --2- 172.21.15.33:0/1006399828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 0x7f08000bb990 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f08000c0fc0 tx=0x7f07f40090f0).stop 2023-11-06T22:16:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.828+0000 7f0806dff700 1 -- 172.21.15.33:0/1006399828 shutdown_connections 2023-11-06T22:16:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.828+0000 7f0806dff700 1 --2- 172.21.15.33:0/1006399828 >> v2:172.21.15.69:3300/0 conn(0x7f08000c33f0 0x7f08000c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.828+0000 7f0806dff700 1 --2- 172.21.15.33:0/1006399828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 0x7f08000bb990 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.828+0000 7f0806dff700 1 -- 172.21.15.33:0/1006399828 >> 172.21.15.33:0/1006399828 conn(0x7f0800062710 msgr2=0x7f0800062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.828+0000 7f0806dff700 1 -- 172.21.15.33:0/1006399828 shutdown_connections 2023-11-06T22:16:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.828+0000 7f0806dff700 1 -- 172.21.15.33:0/1006399828 wait complete. 2023-11-06T22:16:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.829+0000 7f0806dff700 1 Processor -- start 2023-11-06T22:16:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.829+0000 7f0806dff700 1 -- start start 2023-11-06T22:16:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.830+0000 7f0806dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08000c33f0 0x7f08000ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.830+0000 7f0806dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 0x7f0800162240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.830+0000 7f0806dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0800152ff0 0x7f0800162730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.830+0000 7f0806dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08000aa700 con 0x7f080014c640 2023-11-06T22:16:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.830+0000 7f0806dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08000a9af0 con 0x7f0800152ff0 2023-11-06T22:16:20.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.830+0000 7f0806dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000a9dd0 con 0x7f08000c33f0 2023-11-06T22:16:20.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f07fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 0x7f0800162240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:20.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f0804b9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08000c33f0 0x7f08000ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:20.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f07fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 0x7f0800162240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56666/0 (socket says 172.21.15.33:56666) 2023-11-06T22:16:20.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f07fffff700 1 -- 172.21.15.33:0/1630657550 learned_addr learned my addr 172.21.15.33:0/1630657550 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f080539c700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0800152ff0 0x7f0800162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f07fffff700 1 -- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08000c33f0 msgr2=0x7f08000ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f07fffff700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08000c33f0 0x7f08000ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f07fffff700 1 -- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0800152ff0 msgr2=0x7f0800162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f07fffff700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0800152ff0 0x7f0800162730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:20.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f07fffff700 1 -- 172.21.15.33:0/1630657550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8009a20 con 0x7f080014c640 2023-11-06T22:16:20.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f0804b9b700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08000c33f0 0x7f08000ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:20.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.831+0000 7f07fffff700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 0x7f0800162240 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f07ec00c420 tx=0x7f07ec007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:20.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.832+0000 7f07fcff9700 1 -- 172.21.15.33:0/1630657550 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec00ce20 con 0x7f080014c640 2023-11-06T22:16:20.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.832+0000 7f07fcff9700 1 -- 172.21.15.33:0/1630657550 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec0072e0 con 0x7f080014c640 2023-11-06T22:16:20.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.832+0000 7f0806dff700 1 -- 172.21.15.33:0/1630657550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f4009370 con 0x7f080014c640 2023-11-06T22:16:20.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.832+0000 7f07fcff9700 1 -- 172.21.15.33:0/1630657550 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec0056e0 con 0x7f080014c640 2023-11-06T22:16:20.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.832+0000 7f0806dff700 1 -- 172.21.15.33:0/1630657550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08000aa390 con 0x7f080014c640 2023-11-06T22:16:20.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.833+0000 7f0806dff700 1 -- 172.21.15.33:0/1630657550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0800064960 con 0x7f080014c640 2023-11-06T22:16:20.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.834+0000 7f07fcff9700 1 -- 172.21.15.33:0/1630657550 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f07ec005a50 con 0x7f080014c640 2023-11-06T22:16:20.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.835+0000 7f07fcff9700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f07e405f270 0x7f07e4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:20.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.835+0000 7f07fcff9700 1 -- 172.21.15.33:0/1630657550 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f07ec079bd0 con 0x7f080014c640 2023-11-06T22:16:20.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.835+0000 7f0804b9b700 1 -- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f07e405f270 msgr2=0x7f07e4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:20.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.835+0000 7f0804b9b700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f07e405f270 0x7f07e4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:20.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:20.839+0000 7f07fcff9700 1 -- 172.21.15.33:0/1630657550 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f07ec04ab10 con 0x7f080014c640 2023-11-06T22:16:20.899 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:20.897+0000 7ff8aec72680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:16:20.927 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:20.925+0000 7f5957778680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:16:21.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.036+0000 7f0804b9b700 1 -- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f07e405f270 msgr2=0x7f07e4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:21.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.036+0000 7f0804b9b700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f07e405f270 0x7f07e4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:16:21.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.067+0000 7f0806dff700 1 -- 172.21.15.33:0/1630657550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08000bc4a0 con 0x7f080014c640 2023-11-06T22:16:21.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.068+0000 7f07fcff9700 1 -- 172.21.15.33:0/1630657550 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4490 (secure 0 0 0) 0x7f07ec018030 con 0x7f080014c640 2023-11-06T22:16:21.070 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:21.070 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":76,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3392183580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3392183580},{"type":"v1","addr":"172.21.15.33:6835","nonce":3392183580}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2124998325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2124998325},{"type":"v1","addr":"172.21.15.69:6837","nonce":2124998325}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3826680893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3826680893},{"type":"v1","addr":"172.21.15.33:6834","nonce":3826680893}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:00:18.069753+0000","modified":"2023-11-06T22:00:21.084401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8551":{"gid":8551,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/249801946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":249801946},{"type":"v1","addr":"172.21.15.69:6836","nonce":249801946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-06T22:16:21.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 -- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f07e405f270 msgr2=0x7f07e4061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:21.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f07e405f270 0x7f07e4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:21.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 -- 172.21.15.33:0/1630657550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 msgr2=0x7f0800162240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 0x7f0800162240 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f07ec00c420 tx=0x7f07ec007c10).stop 2023-11-06T22:16:21.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 -- 172.21.15.33:0/1630657550 shutdown_connections 2023-11-06T22:16:21.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f07e405f270 0x7f07e4061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:21.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f08000c33f0 0x7f08000ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0800152ff0 0x7f0800162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 --2- 172.21.15.33:0/1630657550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080014c640 0x7f0800162240 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 -- 172.21.15.33:0/1630657550 >> 172.21.15.33:0/1630657550 conn(0x7f0800062710 msgr2=0x7f08001501a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 -- 172.21.15.33:0/1630657550 shutdown_connections 2023-11-06T22:16:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.070+0000 7f0806dff700 1 -- 172.21.15.33:0/1630657550 wait complete. 2023-11-06T22:16:21.075 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 76 2023-11-06T22:16:21.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:16:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.288+0000 7f96e47ff700 1 Processor -- start 2023-11-06T22:16:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.288+0000 7f96e47ff700 1 -- start start 2023-11-06T22:16:21.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e47ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f96dc151dc0 0x7f96dc0c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e47ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f96dc1521a0 0x7f96dc063110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e47ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96dc154e90 con 0x7f96dc155320 2023-11-06T22:16:21.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e47ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc0639d0 con 0x7f96dc1521a0 2023-11-06T22:16:21.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e47ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc063cb0 con 0x7f96dc151dc0 2023-11-06T22:16:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e1d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f96dc1521a0 0x7f96dc063110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e259b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f96dc151dc0 0x7f96dc0c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e259b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f96dc151dc0 0x7f96dc0c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:46192/0 (socket says 172.21.15.33:46192) 2023-11-06T22:16:21.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e259b700 1 -- 172.21.15.33:0/4071862730 learned_addr learned my addr 172.21.15.33:0/4071862730 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e259b700 1 -- 172.21.15.33:0/4071862730 >> v2:172.21.15.33:3300/0 conn(0x7f96dc1521a0 msgr2=0x7f96dc063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e259b700 1 --2- 172.21.15.33:0/4071862730 >> v2:172.21.15.33:3300/0 conn(0x7f96dc1521a0 0x7f96dc063110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e259b700 1 -- 172.21.15.33:0/4071862730 >> v1:172.21.15.69:6789/0 conn(0x7f96dc155320 legacy=0x7f96dc0c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.290+0000 7f96e259b700 1 -- 172.21.15.33:0/4071862730 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc063f90 con 0x7f96dc151dc0 2023-11-06T22:16:21.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.291+0000 7f96e259b700 1 --2- 172.21.15.33:0/4071862730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc151dc0 0x7f96dc0c0c50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f96dc0651b0 tx=0x7f96d80090f0).ready entity=mon.1 client_cookie=f4168c689861dc37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:21.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.291+0000 7f96e1599700 1 -- 172.21.15.33:0/4071862730 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d800b520 con 0x7f96dc151dc0 2023-11-06T22:16:21.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.291+0000 7f96e1599700 1 -- 172.21.15.33:0/4071862730 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d800b680 con 0x7f96dc151dc0 2023-11-06T22:16:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.291+0000 7f96e1599700 1 -- 172.21.15.33:0/4071862730 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d800ba00 con 0x7f96dc151dc0 2023-11-06T22:16:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.291+0000 7f96e47ff700 1 -- 172.21.15.33:0/4071862730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc151dc0 msgr2=0x7f96dc0c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.291+0000 7f96e47ff700 1 --2- 172.21.15.33:0/4071862730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc151dc0 0x7f96dc0c0c50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f96dc0651b0 tx=0x7f96d80090f0).stop 2023-11-06T22:16:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.292+0000 7f96e47ff700 1 -- 172.21.15.33:0/4071862730 shutdown_connections 2023-11-06T22:16:21.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.292+0000 7f96e47ff700 1 --2- 172.21.15.33:0/4071862730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc151dc0 0x7f96dc0c0c50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.292+0000 7f96e47ff700 1 --2- 172.21.15.33:0/4071862730 >> v2:172.21.15.33:3300/0 conn(0x7f96dc1521a0 0x7f96dc063110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.292+0000 7f96e47ff700 1 -- 172.21.15.33:0/4071862730 >> 172.21.15.33:0/4071862730 conn(0x7f96dc05e1b0 msgr2=0x7f96dc05e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:21.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.292+0000 7f96e47ff700 1 -- 172.21.15.33:0/4071862730 shutdown_connections 2023-11-06T22:16:21.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.292+0000 7f96e47ff700 1 -- 172.21.15.33:0/4071862730 wait complete. 2023-11-06T22:16:21.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.293+0000 7f96e47ff700 1 Processor -- start 2023-11-06T22:16:21.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.293+0000 7f96e47ff700 1 -- start start 2023-11-06T22:16:21.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.294+0000 7f96e47ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96dc151dc0 0x7f96dc164f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.294+0000 7f96e47ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96dc1521a0 0x7f96dc0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.294+0000 7f96e47ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc155320 0x7f96dc156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.294+0000 7f96e47ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0aeda0 con 0x7f96dc151dc0 2023-11-06T22:16:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.294+0000 7f96e47ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0afc30 con 0x7f96dc155320 2023-11-06T22:16:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.294+0000 7f96e47ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc165480 con 0x7f96dc1521a0 2023-11-06T22:16:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.294+0000 7f96e259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96dc151dc0 0x7f96dc164f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:21.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.294+0000 7f96e259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96dc151dc0 0x7f96dc164f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56698/0 (socket says 172.21.15.33:56698) 2023-11-06T22:16:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96e259b700 1 -- 172.21.15.33:0/1554647271 learned_addr learned my addr 172.21.15.33:0/1554647271 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96e1d9a700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96dc1521a0 0x7f96dc0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96e2d9c700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc155320 0x7f96dc156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96e259b700 1 -- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96dc1521a0 msgr2=0x7f96dc0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96e259b700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96dc1521a0 0x7f96dc0c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96e259b700 1 -- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc155320 msgr2=0x7f96dc156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96e259b700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc155320 0x7f96dc156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96e259b700 1 -- 172.21.15.33:0/1554647271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d8009370 con 0x7f96dc151dc0 2023-11-06T22:16:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96e259b700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96dc151dc0 0x7f96dc164f90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f96d8009130 tx=0x7f96d8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:21.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96d27fc700 1 -- 172.21.15.33:0/1554647271 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8000bc0 con 0x7f96dc151dc0 2023-11-06T22:16:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96d27fc700 1 -- 172.21.15.33:0/1554647271 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d800b7f0 con 0x7f96dc151dc0 2023-11-06T22:16:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96e47ff700 1 -- 172.21.15.33:0/1554647271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc0ae190 con 0x7f96dc151dc0 2023-11-06T22:16:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.295+0000 7f96d27fc700 1 -- 172.21.15.33:0/1554647271 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8000d20 con 0x7f96dc151dc0 2023-11-06T22:16:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.296+0000 7f96e47ff700 1 -- 172.21.15.33:0/1554647271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc0ae750 con 0x7f96dc151dc0 2023-11-06T22:16:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.297+0000 7f96e47ff700 1 -- 172.21.15.33:0/1554647271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc0c1ea0 con 0x7f96dc151dc0 2023-11-06T22:16:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.297+0000 7f96d27fc700 1 -- 172.21.15.33:0/1554647271 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f96d801c8c0 con 0x7f96dc151dc0 2023-11-06T22:16:21.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.298+0000 7f96d27fc700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f96c005f270 0x7f96c0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.298+0000 7f96d27fc700 1 -- 172.21.15.33:0/1554647271 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f96d807c050 con 0x7f96dc151dc0 2023-11-06T22:16:21.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.298+0000 7f96e1d9a700 1 -- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f96c005f270 msgr2=0x7f96c0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:21.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.298+0000 7f96e1d9a700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f96c005f270 0x7f96c0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:21.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.306+0000 7f96d27fc700 1 -- 172.21.15.33:0/1554647271 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f96d804cf90 con 0x7f96dc151dc0 2023-11-06T22:16:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.488+0000 7f96e47ff700 1 -- 172.21.15.33:0/1554647271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96dc06bc10 con 0x7f96dc151dc0 2023-11-06T22:16:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.490+0000 7f96d27fc700 1 -- 172.21.15.33:0/1554647271 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+17773 (secure 0 0 0) 0x7f96d802a070 con 0x7f96dc151dc0 2023-11-06T22:16:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:21.492 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":189,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:11:22.119451+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:00:15.624386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T22:00:17.062698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1d","mappings":[{"from":2,"to":5}]},{"pgid":"21.1e","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/540532913":"2023-11-07T22:11:22.119414+0000","172.21.15.69:0/2903646842":"2023-11-07T22:11:22.119414+0000","172.21.15.69:0/3284886886":"2023-11-07T22:11:22.119414+0000","172.21.15.69:0/3999769341":"2023-11-07T22:11:22.119414+0000","172.21.15.69:6833/114912":"2023-11-07T22:00:32.314047+0000","172.21.15.69:6832/114912":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/2728366018":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/515574665":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/1433505155":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/805490711":"2023-11-07T22:00:32.314047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:16:21.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 -- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f96c005f270 msgr2=0x7f96c0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:21.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f96c005f270 0x7f96c0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:21.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 -- 172.21.15.33:0/1554647271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96dc151dc0 msgr2=0x7f96dc164f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96dc151dc0 0x7f96dc164f90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f96d8009130 tx=0x7f96d8017000).stop 2023-11-06T22:16:21.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 -- 172.21.15.33:0/1554647271 shutdown_connections 2023-11-06T22:16:21.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f96c005f270 0x7f96c0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:21.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc155320 0x7f96dc156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96dc151dc0 0x7f96dc164f90 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 --2- 172.21.15.33:0/1554647271 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96dc1521a0 0x7f96dc0c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 -- 172.21.15.33:0/1554647271 >> 172.21.15.33:0/1554647271 conn(0x7f96dc05e1b0 msgr2=0x7f96dc0c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:21.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 -- 172.21.15.33:0/1554647271 shutdown_connections 2023-11-06T22:16:21.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.492+0000 7f96e47ff700 1 -- 172.21.15.33:0/1554647271 wait complete. 2023-11-06T22:16:21.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T22:00:15.624386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T22:00:17.062698+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T22:16:21.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T22:16:21.641 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:21.639+0000 7ff8aec72680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:16:21.724 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:21.722+0000 7ff8aec72680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:16:21.728 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:21.727+0000 7f5957778680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:16:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.742+0000 7ffb4b3ff700 1 Processor -- start 2023-11-06T22:16:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.742+0000 7ffb4b3ff700 1 -- start start 2023-11-06T22:16:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.744+0000 7ffb4b3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb44064680 0x7ffb440bdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.744+0000 7ffb4b3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ffb440be0a0 0x7ffb440c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.744+0000 7ffb4b3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb44064b00 con 0x7ffb441546a0 2023-11-06T22:16:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.744+0000 7ffb4b3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb44154a80 con 0x7ffb440be0a0 2023-11-06T22:16:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.744+0000 7ffb4b3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb440641f0 con 0x7ffb44064680 2023-11-06T22:16:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.744+0000 7ffb4919b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffb441546a0 0x7ffb44066560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35240/0 (socket says 172.21.15.33:35240) 2023-11-06T22:16:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.744+0000 7ffb4919b700 1 -- 172.21.15.33:0/2163124729 learned_addr learned my addr 172.21.15.33:0/2163124729 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.744+0000 7ffb4999c700 1 --2- 172.21.15.33:0/2163124729 >> v2:172.21.15.69:3300/0 conn(0x7ffb440be0a0 0x7ffb440c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:21.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.744+0000 7ffb4899a700 1 --2- 172.21.15.33:0/2163124729 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb44064680 0x7ffb440bdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.745+0000 7ffb3bfff700 1 -- 172.21.15.33:0/2163124729 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601813926 0 0) 0x7ffb40001110 con 0x7ffb441546a0 2023-11-06T22:16:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.745+0000 7ffb3bfff700 1 -- 172.21.15.33:0/2163124729 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb300033f0 con 0x7ffb441546a0 2023-11-06T22:16:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.745+0000 7ffb4999c700 1 -- 172.21.15.33:0/2163124729 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb44064680 msgr2=0x7ffb440bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.745+0000 7ffb4999c700 1 --2- 172.21.15.33:0/2163124729 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb44064680 0x7ffb440bdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.745+0000 7ffb4999c700 1 -- 172.21.15.33:0/2163124729 >> v1:172.21.15.33:6789/0 conn(0x7ffb441546a0 legacy=0x7ffb44066560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.745+0000 7ffb4999c700 1 -- 172.21.15.33:0/2163124729 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb440be530 con 0x7ffb440be0a0 2023-11-06T22:16:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.745+0000 7ffb4999c700 1 --2- 172.21.15.33:0/2163124729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb440be0a0 0x7ffb440c35f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7ffb3c009e60 tx=0x7ffb3c009780).ready entity=mon.1 client_cookie=77dd8da385875ef5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.745+0000 7ffb3bfff700 1 -- 172.21.15.33:0/2163124729 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00e070 con 0x7ffb440be0a0 2023-11-06T22:16:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.745+0000 7ffb3bfff700 1 -- 172.21.15.33:0/2163124729 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c00bab0 con 0x7ffb440be0a0 2023-11-06T22:16:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.745+0000 7ffb3bfff700 1 -- 172.21.15.33:0/2163124729 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00be30 con 0x7ffb440be0a0 2023-11-06T22:16:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.746+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/2163124729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb440be0a0 msgr2=0x7ffb440c35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.746+0000 7ffb4b3ff700 1 --2- 172.21.15.33:0/2163124729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb440be0a0 0x7ffb440c35f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7ffb3c009e60 tx=0x7ffb3c009780).stop 2023-11-06T22:16:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.746+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/2163124729 shutdown_connections 2023-11-06T22:16:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.746+0000 7ffb4b3ff700 1 --2- 172.21.15.33:0/2163124729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb440be0a0 0x7ffb440c35f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.746+0000 7ffb4b3ff700 1 --2- 172.21.15.33:0/2163124729 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb44064680 0x7ffb440bdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.746+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/2163124729 >> 172.21.15.33:0/2163124729 conn(0x7ffb440c8c50 msgr2=0x7ffb440c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.746+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/2163124729 shutdown_connections 2023-11-06T22:16:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.746+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/2163124729 wait complete. 2023-11-06T22:16:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.748+0000 7ffb4b3ff700 1 Processor -- start 2023-11-06T22:16:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.748+0000 7ffb4b3ff700 1 -- start start 2023-11-06T22:16:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4b3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb44064680 0x7ffb44067c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4b3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb440be0a0 0x7ffb441565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4b3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb441546a0 0x7ffb44156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4b3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb44157060 con 0x7ffb440be0a0 2023-11-06T22:16:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4b3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb44157340 con 0x7ffb44064680 2023-11-06T22:16:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4b3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb44157620 con 0x7ffb441546a0 2023-11-06T22:16:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4919b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb44064680 0x7ffb44067c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4999c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb441546a0 0x7ffb44156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb440be0a0 0x7ffb441565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb440be0a0 0x7ffb441565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56714/0 (socket says 172.21.15.33:56714) 2023-11-06T22:16:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4999c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb441546a0 0x7ffb44156a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:53194/0 (socket says 172.21.15.33:53194) 2023-11-06T22:16:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4899a700 1 -- 172.21.15.33:0/484503300 learned_addr learned my addr 172.21.15.33:0/484503300 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.749+0000 7ffb4919b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb44064680 0x7ffb44067c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:46220/0 (socket says 172.21.15.33:46220) 2023-11-06T22:16:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb4919b700 1 -- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb441546a0 msgr2=0x7ffb44156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb4919b700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb441546a0 0x7ffb44156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb4919b700 1 -- 172.21.15.33:0/484503300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb440be0a0 msgr2=0x7ffb441565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb4919b700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb440be0a0 0x7ffb441565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb4919b700 1 -- 172.21.15.33:0/484503300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c009a20 con 0x7ffb44064680 2023-11-06T22:16:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb4899a700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb440be0a0 0x7ffb441565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb4919b700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb44064680 0x7ffb44067c70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7ffb40005750 tx=0x7ffb40007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb397fa700 1 -- 172.21.15.33:0/484503300 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb400070d0 con 0x7ffb44064680 2023-11-06T22:16:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb397fa700 1 -- 172.21.15.33:0/484503300 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb40007230 con 0x7ffb44064680 2023-11-06T22:16:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb397fa700 1 -- 172.21.15.33:0/484503300 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb400075d0 con 0x7ffb44064680 2023-11-06T22:16:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/484503300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb44157900 con 0x7ffb44064680 2023-11-06T22:16:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.750+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/484503300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb44068160 con 0x7ffb44064680 2023-11-06T22:16:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.751+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/484503300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb44068440 con 0x7ffb44064680 2023-11-06T22:16:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.752+0000 7ffb397fa700 1 -- 172.21.15.33:0/484503300 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7ffb400059b0 con 0x7ffb44064680 2023-11-06T22:16:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.753+0000 7ffb397fa700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb2805f270 0x7ffb28061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.753+0000 7ffb397fa700 1 -- 172.21.15.33:0/484503300 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6653+0+0 (secure 0 0 0) 0x7ffb40079b40 con 0x7ffb44064680 2023-11-06T22:16:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.754+0000 7ffb4899a700 1 -- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb2805f270 msgr2=0x7ffb28061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.754+0000 7ffb4899a700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb2805f270 0x7ffb28061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.758+0000 7ffb397fa700 1 -- 172.21.15.33:0/484503300 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffb4004a9c0 con 0x7ffb44064680 2023-11-06T22:16:21.818 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:21.817+0000 7f5957778680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:16:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.939+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/484503300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ffb44064b00 con 0x7ffb44064680 2023-11-06T22:16:21.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.940+0000 7ffb397fa700 1 -- 172.21.15.33:0/484503300 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v189) v1 ==== 110+0+10 (secure 0 0 0) 0x7ffb44064b00 con 0x7ffb44064680 2023-11-06T22:16:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T22:16:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.942+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb2805f270 msgr2=0x7ffb28061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:21.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.942+0000 7ffb4b3ff700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb2805f270 0x7ffb28061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.942+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb44064680 msgr2=0x7ffb44067c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.942+0000 7ffb4b3ff700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb44064680 0x7ffb44067c70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7ffb40005750 tx=0x7ffb40007a70).stop 2023-11-06T22:16:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.942+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/484503300 shutdown_connections 2023-11-06T22:16:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.942+0000 7ffb4b3ff700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb2805f270 0x7ffb28061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:21.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.942+0000 7ffb4b3ff700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb441546a0 0x7ffb44156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.942+0000 7ffb4b3ff700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb44064680 0x7ffb44067c70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.942+0000 7ffb4b3ff700 1 --2- 172.21.15.33:0/484503300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb440be0a0 0x7ffb441565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.942+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/484503300 >> 172.21.15.33:0/484503300 conn(0x7ffb440c8c50 msgr2=0x7ffb441533e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.943+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/484503300 shutdown_connections 2023-11-06T22:16:21.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:21.943+0000 7ffb4b3ff700 1 -- 172.21.15.33:0/484503300 wait complete. 2023-11-06T22:16:21.958 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:21.956+0000 7ff8aec72680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:16:21.962 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T22:16:22.062 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:22.061+0000 7f5957778680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:16:22.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.161+0000 7f895efff700 1 Processor -- start 2023-11-06T22:16:22.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.162+0000 7f895efff700 1 -- start start 2023-11-06T22:16:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.163+0000 7f895efff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f895813dff0 0x7f8958157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.163+0000 7f895efff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f89580c4b00 0x7f89580c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.163+0000 7f895efff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8958064bf0 con 0x7f89580c4720 2023-11-06T22:16:22.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.163+0000 7f895efff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895805e490 con 0x7f89580c4b00 2023-11-06T22:16:22.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.163+0000 7f895efff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8958062e10 con 0x7f895813dff0 2023-11-06T22:16:22.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.164+0000 7f895cd9b700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f89580c4720 0x7f895813d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:47800/0 (socket says 172.21.15.33:47800) 2023-11-06T22:16:22.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.164+0000 7f895cd9b700 1 -- 172.21.15.33:0/4028787919 learned_addr learned my addr 172.21.15.33:0/4028787919 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:22.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.164+0000 7f8957fff700 1 --2- 172.21.15.33:0/4028787919 >> v2:172.21.15.69:3300/0 conn(0x7f895813dff0 0x7f8958157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:22.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.164+0000 7f895d59c700 1 --2- 172.21.15.33:0/4028787919 >> v2:172.21.15.33:3300/0 conn(0x7f89580c4b00 0x7f89580c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:22.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.164+0000 7f89577fe700 1 -- 172.21.15.33:0/4028787919 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666907414 0 0) 0x7f8958064bf0 con 0x7f89580c4720 2023-11-06T22:16:22.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.164+0000 7f89577fe700 1 -- 172.21.15.33:0/4028787919 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89480033f0 con 0x7f89580c4720 2023-11-06T22:16:22.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.164+0000 7f8957fff700 1 -- 172.21.15.33:0/4028787919 >> v2:172.21.15.33:3300/0 conn(0x7f89580c4b00 msgr2=0x7f89580c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:22.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.164+0000 7f8957fff700 1 --2- 172.21.15.33:0/4028787919 >> v2:172.21.15.33:3300/0 conn(0x7f89580c4b00 0x7f89580c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.164+0000 7f8957fff700 1 -- 172.21.15.33:0/4028787919 >> v1:172.21.15.69:6789/0 conn(0x7f89580c4720 legacy=0x7f895813d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:22.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.164+0000 7f8957fff700 1 -- 172.21.15.33:0/4028787919 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89580630f0 con 0x7f895813dff0 2023-11-06T22:16:22.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f8957fff700 1 --2- 172.21.15.33:0/4028787919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f895813dff0 0x7f8958157e20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f8958067c60 tx=0x7f8944009060).ready entity=mon.1 client_cookie=509b50afc730bc28 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:22.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f89577fe700 1 -- 172.21.15.33:0/4028787919 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894400d4a0 con 0x7f895813dff0 2023-11-06T22:16:22.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f89577fe700 1 -- 172.21.15.33:0/4028787919 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894400d600 con 0x7f895813dff0 2023-11-06T22:16:22.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f89577fe700 1 -- 172.21.15.33:0/4028787919 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894400d980 con 0x7f895813dff0 2023-11-06T22:16:22.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f895efff700 1 -- 172.21.15.33:0/4028787919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f895813dff0 msgr2=0x7f8958157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:22.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f895efff700 1 --2- 172.21.15.33:0/4028787919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f895813dff0 0x7f8958157e20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f8958067c60 tx=0x7f8944009060).stop 2023-11-06T22:16:22.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f895efff700 1 -- 172.21.15.33:0/4028787919 shutdown_connections 2023-11-06T22:16:22.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f895efff700 1 --2- 172.21.15.33:0/4028787919 >> v2:172.21.15.33:3300/0 conn(0x7f89580c4b00 0x7f89580c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f895efff700 1 --2- 172.21.15.33:0/4028787919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f895813dff0 0x7f8958157e20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f895efff700 1 -- 172.21.15.33:0/4028787919 >> 172.21.15.33:0/4028787919 conn(0x7f89580c7dc0 msgr2=0x7f89580c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.165+0000 7f895efff700 1 -- 172.21.15.33:0/4028787919 shutdown_connections 2023-11-06T22:16:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.166+0000 7f895efff700 1 -- 172.21.15.33:0/4028787919 wait complete. 2023-11-06T22:16:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.167+0000 7f895efff700 1 Processor -- start 2023-11-06T22:16:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.167+0000 7f895efff700 1 -- start start 2023-11-06T22:16:22.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.168+0000 7f895efff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89580c4720 0x7f895813c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.168+0000 7f895efff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f89580c4b00 0x7f8958137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.168+0000 7f895efff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f895813dff0 0x7f8958135430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.168+0000 7f895efff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895813c8f0 con 0x7f895813dff0 2023-11-06T22:16:22.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.168+0000 7f895efff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8958135fe0 con 0x7f89580c4720 2023-11-06T22:16:22.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.168+0000 7f895efff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89581362c0 con 0x7f89580c4b00 2023-11-06T22:16:22.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f895d59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f895813dff0 0x7f8958135430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:22.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f895cd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89580c4720 0x7f895813c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:22.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f895d59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f895813dff0 0x7f8958135430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56736/0 (socket says 172.21.15.33:56736) 2023-11-06T22:16:22.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f8957fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f89580c4b00 0x7f8958137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:22.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f895d59c700 1 -- 172.21.15.33:0/3273170017 learned_addr learned my addr 172.21.15.33:0/3273170017 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f895d59c700 1 -- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f89580c4b00 msgr2=0x7f8958137080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f895d59c700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f89580c4b00 0x7f8958137080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f895d59c700 1 -- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89580c4720 msgr2=0x7f895813c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f895d59c700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89580c4720 0x7f895813c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f895d59c700 1 -- 172.21.15.33:0/3273170017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89440092a0 con 0x7f895813dff0 2023-11-06T22:16:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f895d59c700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f895813dff0 0x7f8958135430 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f8950009e90 tx=0x7f895000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.169+0000 7f8954ff9700 1 -- 172.21.15.33:0/3273170017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000caa0 con 0x7f895813dff0 2023-11-06T22:16:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.170+0000 7f8954ff9700 1 -- 172.21.15.33:0/3273170017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895000f040 con 0x7f895813dff0 2023-11-06T22:16:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.170+0000 7f8954ff9700 1 -- 172.21.15.33:0/3273170017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000ce10 con 0x7f895813dff0 2023-11-06T22:16:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.170+0000 7f895efff700 1 -- 172.21.15.33:0/3273170017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89581365a0 con 0x7f895813dff0 2023-11-06T22:16:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.170+0000 7f895efff700 1 -- 172.21.15.33:0/3273170017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8958136b60 con 0x7f895813dff0 2023-11-06T22:16:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.171+0000 7f895efff700 1 -- 172.21.15.33:0/3273170017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8958065a50 con 0x7f895813dff0 2023-11-06T22:16:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.172+0000 7f8954ff9700 1 -- 172.21.15.33:0/3273170017 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f8950013610 con 0x7f895813dff0 2023-11-06T22:16:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.173+0000 7f8954ff9700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f893c05f270 0x7f893c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.173+0000 7f8954ff9700 1 -- 172.21.15.33:0/3273170017 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f895007af10 con 0x7f895813dff0 2023-11-06T22:16:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.174+0000 7f895cd9b700 1 -- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f893c05f270 msgr2=0x7f893c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.174+0000 7f895cd9b700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f893c05f270 0x7f893c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.177+0000 7f8954ff9700 1 -- 172.21.15.33:0/3273170017 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f895004bd90 con 0x7f895813dff0 2023-11-06T22:16:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.375+0000 7f895cd9b700 1 -- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f893c05f270 msgr2=0x7f893c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.375+0000 7f895cd9b700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f893c05f270 0x7f893c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:16:22.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.392+0000 7f895efff700 1 -- 172.21.15.33:0/3273170017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8958136d50 con 0x7f895813dff0 2023-11-06T22:16:22.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.392+0000 7f8954ff9700 1 -- 172.21.15.33:0/3273170017 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v190) v1 ==== 92+0+10 (secure 0 0 0) 0x7f895004f3b0 con 0x7f895813dff0 2023-11-06T22:16:22.394 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T22:16:22.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 -- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f893c05f270 msgr2=0x7f893c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f893c05f270 0x7f893c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 -- 172.21.15.33:0/3273170017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f895813dff0 msgr2=0x7f8958135430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f895813dff0 0x7f8958135430 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f8950009e90 tx=0x7f895000b040).stop 2023-11-06T22:16:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 -- 172.21.15.33:0/3273170017 shutdown_connections 2023-11-06T22:16:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f893c05f270 0x7f893c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:22.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f895813dff0 0x7f8958135430 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89580c4720 0x7f895813c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 --2- 172.21.15.33:0/3273170017 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f89580c4b00 0x7f8958137080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 -- 172.21.15.33:0/3273170017 >> 172.21.15.33:0/3273170017 conn(0x7f89580c7dc0 msgr2=0x7f89580c3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:22.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 -- 172.21.15.33:0/3273170017 shutdown_connections 2023-11-06T22:16:22.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.395+0000 7f895efff700 1 -- 172.21.15.33:0/3273170017 wait complete. 2023-11-06T22:16:22.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T22:16:22.500 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:22.498+0000 7ff8aec72680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:16:22.637 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:22.635+0000 7ff8aec72680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:16:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.655+0000 7f15db1ff700 1 Processor -- start 2023-11-06T22:16:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.656+0000 7f15db1ff700 1 -- start start 2023-11-06T22:16:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.659+0000 7f15db1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d4154710 0x7f15d40c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.659+0000 7f15db1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f15d40c7ff0 0x7f15d40646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.659+0000 7f15db1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d40c6b60 con 0x7f15d4154330 2023-11-06T22:16:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.659+0000 7f15db1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d40c7700 con 0x7f15d40c7ff0 2023-11-06T22:16:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.659+0000 7f15db1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40c5cd0 con 0x7f15d4154710 2023-11-06T22:16:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.659+0000 7f15d8f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f15d4154330 0x7f15d40c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35242/0 (socket says 172.21.15.33:35242) 2023-11-06T22:16:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.659+0000 7f15d8f9b700 1 -- 172.21.15.33:0/2083349498 learned_addr learned my addr 172.21.15.33:0/2083349498 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.659+0000 7f15d3fff700 1 --2- 172.21.15.33:0/2083349498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d4154710 0x7f15d40c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.659+0000 7f15d979c700 1 --2- 172.21.15.33:0/2083349498 >> v2:172.21.15.69:3300/0 conn(0x7f15d40c7ff0 0x7f15d40646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.660+0000 7f15d3fff700 1 -- 172.21.15.33:0/2083349498 >> v2:172.21.15.69:3300/0 conn(0x7f15d40c7ff0 msgr2=0x7f15d40646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.660+0000 7f15d3fff700 1 --2- 172.21.15.33:0/2083349498 >> v2:172.21.15.69:3300/0 conn(0x7f15d40c7ff0 0x7f15d40646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.660+0000 7f15d3fff700 1 -- 172.21.15.33:0/2083349498 >> v1:172.21.15.33:6789/0 conn(0x7f15d4154330 legacy=0x7f15d40c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.660+0000 7f15d3fff700 1 -- 172.21.15.33:0/2083349498 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40c5f50 con 0x7f15d4154710 2023-11-06T22:16:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.660+0000 7f15d37fe700 1 -- 172.21.15.33:0/2083349498 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973984208 0 0) 0x7f15d40c6b60 con 0x7f15d4154330 2023-11-06T22:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.660+0000 7f15d3fff700 1 --2- 172.21.15.33:0/2083349498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d4154710 0x7f15d40c57e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f15c0009760 tx=0x7f15c0009060).ready entity=mon.2 client_cookie=7eaa0ee6a2ca4df2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.660+0000 7f15d37fe700 1 -- 172.21.15.33:0/2083349498 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c000f510 con 0x7f15d4154710 2023-11-06T22:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.660+0000 7f15d37fe700 1 -- 172.21.15.33:0/2083349498 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c000f670 con 0x7f15d4154710 2023-11-06T22:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.660+0000 7f15d37fe700 1 -- 172.21.15.33:0/2083349498 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c000f9f0 con 0x7f15d4154710 2023-11-06T22:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.661+0000 7f15db1ff700 1 -- 172.21.15.33:0/2083349498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d4154710 msgr2=0x7f15d40c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.661+0000 7f15db1ff700 1 --2- 172.21.15.33:0/2083349498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d4154710 0x7f15d40c57e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f15c0009760 tx=0x7f15c0009060).stop 2023-11-06T22:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.661+0000 7f15db1ff700 1 -- 172.21.15.33:0/2083349498 shutdown_connections 2023-11-06T22:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.661+0000 7f15db1ff700 1 --2- 172.21.15.33:0/2083349498 >> v2:172.21.15.69:3300/0 conn(0x7f15d40c7ff0 0x7f15d40646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.661+0000 7f15db1ff700 1 --2- 172.21.15.33:0/2083349498 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d4154710 0x7f15d40c57e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.661+0000 7f15db1ff700 1 -- 172.21.15.33:0/2083349498 >> 172.21.15.33:0/2083349498 conn(0x7f15d40cb910 msgr2=0x7f15d40c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.661+0000 7f15db1ff700 1 -- 172.21.15.33:0/2083349498 shutdown_connections 2023-11-06T22:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.661+0000 7f15db1ff700 1 -- 172.21.15.33:0/2083349498 wait complete. 2023-11-06T22:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.664+0000 7f15db1ff700 1 Processor -- start 2023-11-06T22:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.664+0000 7f15db1ff700 1 -- start start 2023-11-06T22:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15db1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d40c7ff0 0x7f15d4145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15db1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d4154330 0x7f15d4065a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15db1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15d4154710 0x7f15d4065f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15db1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40b07a0 con 0x7f15d4154710 2023-11-06T22:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15db1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4066410 con 0x7f15d4154330 2023-11-06T22:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15db1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4168b10 con 0x7f15d40c7ff0 2023-11-06T22:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15d8f9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d40c7ff0 0x7f15d4145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15d3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d4154330 0x7f15d4065a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15d8f9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d40c7ff0 0x7f15d4145a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:53216/0 (socket says 172.21.15.33:53216) 2023-11-06T22:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15d8f9b700 1 -- 172.21.15.33:0/2512427220 learned_addr learned my addr 172.21.15.33:0/2512427220 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.666+0000 7f15d979c700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15d4154710 0x7f15d4065f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15d8f9b700 1 -- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d4154330 msgr2=0x7f15d4065a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15d8f9b700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d4154330 0x7f15d4065a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15d8f9b700 1 -- 172.21.15.33:0/2512427220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15d4154710 msgr2=0x7f15d4065f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15d8f9b700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15d4154710 0x7f15d4065f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15d8f9b700 1 -- 172.21.15.33:0/2512427220 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c00092a0 con 0x7f15d40c7ff0 2023-11-06T22:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15d8f9b700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d40c7ff0 0x7f15d4145a00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f15c800c4e0 tx=0x7f15c8007c70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15d0ff9700 1 -- 172.21.15.33:0/2512427220 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c800ce90 con 0x7f15d40c7ff0 2023-11-06T22:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15d0ff9700 1 -- 172.21.15.33:0/2512427220 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c80072e0 con 0x7f15d40c7ff0 2023-11-06T22:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15d0ff9700 1 -- 172.21.15.33:0/2512427220 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8007660 con 0x7f15d40c7ff0 2023-11-06T22:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15db1ff700 1 -- 172.21.15.33:0/2512427220 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d4168d60 con 0x7f15d40c7ff0 2023-11-06T22:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.667+0000 7f15db1ff700 1 -- 172.21.15.33:0/2512427220 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d4169320 con 0x7f15d40c7ff0 2023-11-06T22:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.669+0000 7f15db1ff700 1 -- 172.21.15.33:0/2512427220 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d40830b0 con 0x7f15d40c7ff0 2023-11-06T22:16:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.669+0000 7f15d0ff9700 1 -- 172.21.15.33:0/2512427220 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f15c80059b0 con 0x7f15d40c7ff0 2023-11-06T22:16:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.670+0000 7f15d0ff9700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f15b805f270 0x7f15b8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.671+0000 7f15d0ff9700 1 -- 172.21.15.33:0/2512427220 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f15c8079db0 con 0x7f15d40c7ff0 2023-11-06T22:16:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.671+0000 7f15d3fff700 1 -- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f15b805f270 msgr2=0x7f15b8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.671+0000 7f15d3fff700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f15b805f270 0x7f15b8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.675+0000 7f15d0ff9700 1 -- 172.21.15.33:0/2512427220 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f15c807e070 con 0x7f15d40c7ff0 2023-11-06T22:16:22.682 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:22.681+0000 7f5957778680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:16:22.853 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:22.852+0000 7f5957778680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:16:22.859 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:22.858+0000 7ff8aec72680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:16:22.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.872+0000 7f15d3fff700 1 -- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f15b805f270 msgr2=0x7f15b8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:22.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.872+0000 7f15d3fff700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f15b805f270 0x7f15b8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:16:22.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.896+0000 7f15db1ff700 1 -- 172.21.15.33:0/2512427220 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15d4062b00 con 0x7f15d40c7ff0 2023-11-06T22:16:22.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.897+0000 7f15d0ff9700 1 -- 172.21.15.33:0/2512427220 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v190) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15c804abf0 con 0x7f15d40c7ff0 2023-11-06T22:16:22.899 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T22:16:22.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 -- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f15b805f270 msgr2=0x7f15b8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:22.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f15b805f270 0x7f15b8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:22.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 -- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d40c7ff0 msgr2=0x7f15d4145a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:22.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d40c7ff0 0x7f15d4145a00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f15c800c4e0 tx=0x7f15c8007c70).stop 2023-11-06T22:16:22.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 -- 172.21.15.33:0/2512427220 shutdown_connections 2023-11-06T22:16:22.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f15b805f270 0x7f15b8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:22.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f15d40c7ff0 0x7f15d4145a00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15d4154710 0x7f15d4065f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 --2- 172.21.15.33:0/2512427220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15d4154330 0x7f15d4065a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:22.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 -- 172.21.15.33:0/2512427220 >> 172.21.15.33:0/2512427220 conn(0x7f15d40cb910 msgr2=0x7f15d40ac770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:22.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.899+0000 7f15db1ff700 1 -- 172.21.15.33:0/2512427220 shutdown_connections 2023-11-06T22:16:22.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:22.900+0000 7f15db1ff700 1 -- 172.21.15.33:0/2512427220 wait complete. 2023-11-06T22:16:22.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T22:16:22.955 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:22.953+0000 7ff8aec72680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:16:23.041 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:23.039+0000 7ff8aec72680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:16:23.054 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:23.053+0000 7f5957778680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:16:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.126+0000 7f1b86dff700 1 Processor -- start 2023-11-06T22:16:23.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.126+0000 7f1b86dff700 1 -- start start 2023-11-06T22:16:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b86dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1b80154330 0x7f1b800c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b86dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1b80154710 0x7f1b800c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b86dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b800c7ff0 0x7f1b800646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b86dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b800c6b60 con 0x7f1b80154710 2023-11-06T22:16:23.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b86dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b800c5cd0 con 0x7f1b80154330 2023-11-06T22:16:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b86dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800c5f50 con 0x7f1b800c7ff0 2023-11-06T22:16:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b7ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1b80154710 0x7f1b800c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b7ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1b80154710 0x7f1b800c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56752/0 (socket says 172.21.15.33:56752) 2023-11-06T22:16:23.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b7ffff700 1 -- 172.21.15.33:0/247582680 learned_addr learned my addr 172.21.15.33:0/247582680 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b84b9b700 1 --2- 172.21.15.33:0/247582680 >> v2:172.21.15.69:3300/0 conn(0x7f1b80154330 0x7f1b800c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.128+0000 7f1b8539c700 1 --2- 172.21.15.33:0/247582680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b800c7ff0 0x7f1b800646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.129+0000 7f1b7ffff700 1 -- 172.21.15.33:0/247582680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b800c7ff0 msgr2=0x7f1b800646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:23.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.129+0000 7f1b7ffff700 1 --2- 172.21.15.33:0/247582680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b800c7ff0 0x7f1b800646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.129+0000 7f1b7ffff700 1 -- 172.21.15.33:0/247582680 >> v2:172.21.15.69:3300/0 conn(0x7f1b80154330 msgr2=0x7f1b800c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:23.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.129+0000 7f1b7ffff700 1 --2- 172.21.15.33:0/247582680 >> v2:172.21.15.69:3300/0 conn(0x7f1b80154330 0x7f1b800c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.129+0000 7f1b7ffff700 1 -- 172.21.15.33:0/247582680 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800c6230 con 0x7f1b80154710 2023-11-06T22:16:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.129+0000 7f1b7ffff700 1 --2- 172.21.15.33:0/247582680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b80154710 0x7f1b800c6ff0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f1b800605d0 tx=0x7f1b6c0090f0).ready entity=mon.0 client_cookie=538a29745927f577 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.129+0000 7f1b7f7fe700 1 -- 172.21.15.33:0/247582680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00f510 con 0x7f1b80154710 2023-11-06T22:16:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.129+0000 7f1b7f7fe700 1 -- 172.21.15.33:0/247582680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6c00f670 con 0x7f1b80154710 2023-11-06T22:16:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.129+0000 7f1b7f7fe700 1 -- 172.21.15.33:0/247582680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00f9f0 con 0x7f1b80154710 2023-11-06T22:16:23.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.130+0000 7f1b86dff700 1 -- 172.21.15.33:0/247582680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b80154710 msgr2=0x7f1b800c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:23.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.130+0000 7f1b86dff700 1 --2- 172.21.15.33:0/247582680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b80154710 0x7f1b800c6ff0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f1b800605d0 tx=0x7f1b6c0090f0).stop 2023-11-06T22:16:23.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.130+0000 7f1b86dff700 1 -- 172.21.15.33:0/247582680 shutdown_connections 2023-11-06T22:16:23.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.130+0000 7f1b86dff700 1 --2- 172.21.15.33:0/247582680 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b800c7ff0 0x7f1b800646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.130+0000 7f1b86dff700 1 --2- 172.21.15.33:0/247582680 >> v2:172.21.15.69:3300/0 conn(0x7f1b80154330 0x7f1b800c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.130+0000 7f1b86dff700 1 --2- 172.21.15.33:0/247582680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b80154710 0x7f1b800c6ff0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.130+0000 7f1b86dff700 1 -- 172.21.15.33:0/247582680 >> 172.21.15.33:0/247582680 conn(0x7f1b800cb910 msgr2=0x7f1b800c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.130+0000 7f1b86dff700 1 -- 172.21.15.33:0/247582680 shutdown_connections 2023-11-06T22:16:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.130+0000 7f1b86dff700 1 -- 172.21.15.33:0/247582680 wait complete. 2023-11-06T22:16:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.131+0000 7f1b86dff700 1 Processor -- start 2023-11-06T22:16:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.132+0000 7f1b86dff700 1 -- start start 2023-11-06T22:16:23.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.132+0000 7f1b86dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b800c7ff0 0x7f1b8014c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.132+0000 7f1b86dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b80154330 0x7f1b80154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b86dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b80154710 0x7f1b80155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b86dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800af110 con 0x7f1b80154330 2023-11-06T22:16:23.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b86dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80156090 con 0x7f1b80154710 2023-11-06T22:16:23.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b86dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800af620 con 0x7f1b800c7ff0 2023-11-06T22:16:23.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b80154330 0x7f1b80154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b80154330 0x7f1b80154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56764/0 (socket says 172.21.15.33:56764) 2023-11-06T22:16:23.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b7ffff700 1 -- 172.21.15.33:0/4108744391 learned_addr learned my addr 172.21.15.33:0/4108744391 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:23.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b84b9b700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b800c7ff0 0x7f1b8014c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b8539c700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b80154710 0x7f1b80155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b8539c700 1 -- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b800c7ff0 msgr2=0x7f1b8014c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:23.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b8539c700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b800c7ff0 0x7f1b8014c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b8539c700 1 -- 172.21.15.33:0/4108744391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b80154330 msgr2=0x7f1b80154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:23.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b8539c700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b80154330 0x7f1b80154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.133+0000 7f1b8539c700 1 -- 172.21.15.33:0/4108744391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c009370 con 0x7f1b80154710 2023-11-06T22:16:23.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.134+0000 7f1b8539c700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b80154710 0x7f1b80155090 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f1b78009e50 tx=0x7f1b7800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:23.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.134+0000 7f1b7cff9700 1 -- 172.21.15.33:0/4108744391 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800caa0 con 0x7f1b80154710 2023-11-06T22:16:23.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.134+0000 7f1b7cff9700 1 -- 172.21.15.33:0/4108744391 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7800f040 con 0x7f1b80154710 2023-11-06T22:16:23.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.134+0000 7f1b7cff9700 1 -- 172.21.15.33:0/4108744391 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800ce10 con 0x7f1b80154710 2023-11-06T22:16:23.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.134+0000 7f1b86dff700 1 -- 172.21.15.33:0/4108744391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b800af8c0 con 0x7f1b80154710 2023-11-06T22:16:23.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.134+0000 7f1b86dff700 1 -- 172.21.15.33:0/4108744391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b80155ad0 con 0x7f1b80154710 2023-11-06T22:16:23.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.135+0000 7f1b86dff700 1 -- 172.21.15.33:0/4108744391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b800afa50 con 0x7f1b80154710 2023-11-06T22:16:23.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.136+0000 7f1b7cff9700 1 -- 172.21.15.33:0/4108744391 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f1b78013610 con 0x7f1b80154710 2023-11-06T22:16:23.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.137+0000 7f1b7cff9700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1b6405f270 0x7f1b64061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.137+0000 7f1b7cff9700 1 -- 172.21.15.33:0/4108744391 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1b7807ae50 con 0x7f1b80154710 2023-11-06T22:16:23.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.137+0000 7f1b84b9b700 1 -- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1b6405f270 msgr2=0x7f1b64061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:23.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.137+0000 7f1b84b9b700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1b6405f270 0x7f1b64061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:23.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.141+0000 7f1b7cff9700 1 -- 172.21.15.33:0/4108744391 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1b7804bc90 con 0x7f1b80154710 2023-11-06T22:16:23.155 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:23.154+0000 7f5957778680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:16:23.247 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:23.245+0000 7f5957778680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:16:23.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.322+0000 7f1b86dff700 1 -- 172.21.15.33:0/4108744391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b800830b0 con 0x7f1b80154710 2023-11-06T22:16:23.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.323+0000 7f1b7cff9700 1 -- 172.21.15.33:0/4108744391 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v190) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b7804f2b0 con 0x7f1b80154710 2023-11-06T22:16:23.324 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T22:16:23.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 -- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1b6405f270 msgr2=0x7f1b64061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:23.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1b6405f270 0x7f1b64061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:23.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 -- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b80154710 msgr2=0x7f1b80155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:23.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b80154710 0x7f1b80155090 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f1b78009e50 tx=0x7f1b7800b040).stop 2023-11-06T22:16:23.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 -- 172.21.15.33:0/4108744391 shutdown_connections 2023-11-06T22:16:23.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1b6405f270 0x7f1b64061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:23.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1b800c7ff0 0x7f1b8014c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b80154710 0x7f1b80155090 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 --2- 172.21.15.33:0/4108744391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b80154330 0x7f1b80154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 -- 172.21.15.33:0/4108744391 >> 172.21.15.33:0/4108744391 conn(0x7f1b800cb910 msgr2=0x7f1b800aa070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 -- 172.21.15.33:0/4108744391 shutdown_connections 2023-11-06T22:16:23.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.326+0000 7f1b86dff700 1 -- 172.21.15.33:0/4108744391 wait complete. 2023-11-06T22:16:23.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:16:23.357 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:23.355+0000 7ff8aec72680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:16:23.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.551+0000 7fe9647ff700 1 Processor -- start 2023-11-06T22:16:23.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.551+0000 7fe9647ff700 1 -- start start 2023-11-06T22:16:23.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.552+0000 7fe9647ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe95c154280 0x7fe95c154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.552+0000 7fe9647ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe95c0ad6e0 0x7fe95c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.552+0000 7fe9647ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe95c0a9f20 0x7fe95c0c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.552+0000 7fe9647ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c0c5db0 con 0x7fe95c0ad6e0 2023-11-06T22:16:23.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.552+0000 7fe9647ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c0c6930 con 0x7fe95c154280 2023-11-06T22:16:23.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.552+0000 7fe9647ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c6000 con 0x7fe95c0a9f20 2023-11-06T22:16:23.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe961d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe95c0ad6e0 0x7fe95c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe961d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe95c0ad6e0 0x7fe95c0c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56778/0 (socket says 172.21.15.33:56778) 2023-11-06T22:16:23.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe961d9a700 1 -- 172.21.15.33:0/3803682925 learned_addr learned my addr 172.21.15.33:0/3803682925 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:23.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe962d9c700 1 --2- 172.21.15.33:0/3803682925 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe95c0a9f20 0x7fe95c0c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe96259b700 1 --2- 172.21.15.33:0/3803682925 >> v2:172.21.15.69:3300/0 conn(0x7fe95c154280 0x7fe95c154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe961d9a700 1 -- 172.21.15.33:0/3803682925 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe95c0a9f20 msgr2=0x7fe95c0c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe961d9a700 1 --2- 172.21.15.33:0/3803682925 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe95c0a9f20 0x7fe95c0c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe961d9a700 1 -- 172.21.15.33:0/3803682925 >> v2:172.21.15.69:3300/0 conn(0x7fe95c154280 msgr2=0x7fe95c154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe961d9a700 1 --2- 172.21.15.33:0/3803682925 >> v2:172.21.15.69:3300/0 conn(0x7fe95c154280 0x7fe95c154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe961d9a700 1 -- 172.21.15.33:0/3803682925 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c0c62e0 con 0x7fe95c0ad6e0 2023-11-06T22:16:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.553+0000 7fe96259b700 1 --2- 172.21.15.33:0/3803682925 >> v2:172.21.15.69:3300/0 conn(0x7fe95c154280 0x7fe95c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.554+0000 7fe961d9a700 1 --2- 172.21.15.33:0/3803682925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c0ad6e0 0x7fe95c0c6dc0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7fe95c065150 tx=0x7fe94c0090f0).ready entity=mon.0 client_cookie=77d8eb592524978d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.554+0000 7fe961599700 1 -- 172.21.15.33:0/3803682925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c00f510 con 0x7fe95c0ad6e0 2023-11-06T22:16:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.554+0000 7fe961599700 1 -- 172.21.15.33:0/3803682925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c00f670 con 0x7fe95c0ad6e0 2023-11-06T22:16:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.554+0000 7fe961599700 1 -- 172.21.15.33:0/3803682925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c00f9f0 con 0x7fe95c0ad6e0 2023-11-06T22:16:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.554+0000 7fe9647ff700 1 -- 172.21.15.33:0/3803682925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c0ad6e0 msgr2=0x7fe95c0c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.554+0000 7fe9647ff700 1 --2- 172.21.15.33:0/3803682925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c0ad6e0 0x7fe95c0c6dc0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7fe95c065150 tx=0x7fe94c0090f0).stop 2023-11-06T22:16:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.555+0000 7fe9647ff700 1 -- 172.21.15.33:0/3803682925 shutdown_connections 2023-11-06T22:16:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.555+0000 7fe9647ff700 1 --2- 172.21.15.33:0/3803682925 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe95c0a9f20 0x7fe95c0c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.555+0000 7fe9647ff700 1 --2- 172.21.15.33:0/3803682925 >> v2:172.21.15.69:3300/0 conn(0x7fe95c154280 0x7fe95c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.555+0000 7fe9647ff700 1 --2- 172.21.15.33:0/3803682925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c0ad6e0 0x7fe95c0c6dc0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.555+0000 7fe9647ff700 1 -- 172.21.15.33:0/3803682925 >> 172.21.15.33:0/3803682925 conn(0x7fe95c05e1b0 msgr2=0x7fe95c05e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.555+0000 7fe9647ff700 1 -- 172.21.15.33:0/3803682925 shutdown_connections 2023-11-06T22:16:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.555+0000 7fe9647ff700 1 -- 172.21.15.33:0/3803682925 wait complete. 2023-11-06T22:16:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.556+0000 7fe9647ff700 1 Processor -- start 2023-11-06T22:16:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.556+0000 7fe9647ff700 1 -- start start 2023-11-06T22:16:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.557+0000 7fe9647ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe95c0a9f20 0x7fe95c0c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.557+0000 7fe9647ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe95c0ad6e0 0x7fe95c14aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.557+0000 7fe9647ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c154280 0x7fe95c152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.557+0000 7fe9647ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c2890 con 0x7fe95c154280 2023-11-06T22:16:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.557+0000 7fe9647ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c165270 con 0x7fe95c0ad6e0 2023-11-06T22:16:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.557+0000 7fe9647ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c1f60 con 0x7fe95c0a9f20 2023-11-06T22:16:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe962d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c154280 0x7fe95c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe96259b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe95c0a9f20 0x7fe95c0c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe962d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c154280 0x7fe95c152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56794/0 (socket says 172.21.15.33:56794) 2023-11-06T22:16:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe962d9c700 1 -- 172.21.15.33:0/3555170256 learned_addr learned my addr 172.21.15.33:0/3555170256 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe961d9a700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe95c0ad6e0 0x7fe95c14aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe962d9c700 1 -- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe95c0a9f20 msgr2=0x7fe95c0c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe962d9c700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe95c0a9f20 0x7fe95c0c2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe962d9c700 1 -- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe95c0ad6e0 msgr2=0x7fe95c14aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe962d9c700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe95c0ad6e0 0x7fe95c14aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe962d9c700 1 -- 172.21.15.33:0/3555170256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580092a0 con 0x7fe95c154280 2023-11-06T22:16:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe962d9c700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c154280 0x7fe95c152390 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7fe954009e90 tx=0x7fe95400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe9527fc700 1 -- 172.21.15.33:0/3555170256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95400caa0 con 0x7fe95c154280 2023-11-06T22:16:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe9527fc700 1 -- 172.21.15.33:0/3555170256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95400f040 con 0x7fe95c154280 2023-11-06T22:16:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.558+0000 7fe9527fc700 1 -- 172.21.15.33:0/3555170256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95400ce10 con 0x7fe95c154280 2023-11-06T22:16:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.559+0000 7fe9647ff700 1 -- 172.21.15.33:0/3555170256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c009370 con 0x7fe95c154280 2023-11-06T22:16:23.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.559+0000 7fe9647ff700 1 -- 172.21.15.33:0/3555170256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c0c2520 con 0x7fe95c154280 2023-11-06T22:16:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.560+0000 7fe9647ff700 1 -- 172.21.15.33:0/3555170256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c0c65c0 con 0x7fe95c154280 2023-11-06T22:16:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.560+0000 7fe9527fc700 1 -- 172.21.15.33:0/3555170256 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fe954013610 con 0x7fe95c154280 2023-11-06T22:16:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.561+0000 7fe9527fc700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe94005f270 0x7fe940061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.561+0000 7fe9527fc700 1 -- 172.21.15.33:0/3555170256 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe95407aee0 con 0x7fe95c154280 2023-11-06T22:16:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.561+0000 7fe96259b700 1 -- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe94005f270 msgr2=0x7fe940061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.561+0000 7fe96259b700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe94005f270 0x7fe940061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.566+0000 7fe9527fc700 1 -- 172.21.15.33:0/3555170256 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe95404bd20 con 0x7fe95c154280 2023-11-06T22:16:23.578 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:23.577+0000 7ff8aec72680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:16:23.594 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:23.593+0000 7f5957778680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:16:23.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.762+0000 7fe96259b700 1 -- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe94005f270 msgr2=0x7fe940061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:23.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.762+0000 7fe96259b700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe94005f270 0x7fe940061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:16:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.790+0000 7fe9647ff700 1 -- 172.21.15.33:0/3555170256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe95c062ac0 con 0x7fe95c154280 2023-11-06T22:16:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.791+0000 7fe9527fc700 1 -- 172.21.15.33:0/3555170256 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4490 (secure 0 0 0) 0x7fe95404f340 con 0x7fe95c154280 2023-11-06T22:16:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:23.793 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":76,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3392183580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3392183580},{"type":"v1","addr":"172.21.15.33:6835","nonce":3392183580}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2124998325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2124998325},{"type":"v1","addr":"172.21.15.69:6837","nonce":2124998325}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3826680893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3826680893},{"type":"v1","addr":"172.21.15.33:6834","nonce":3826680893}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:00:18.069753+0000","modified":"2023-11-06T22:00:21.084401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8551":{"gid":8551,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/249801946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":249801946},{"type":"v1","addr":"172.21.15.69:6836","nonce":249801946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-06T22:16:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.793+0000 7fe9647ff700 1 -- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe94005f270 msgr2=0x7fe940061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.793+0000 7fe9647ff700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe94005f270 0x7fe940061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.793+0000 7fe9647ff700 1 -- 172.21.15.33:0/3555170256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c154280 msgr2=0x7fe95c152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.793+0000 7fe9647ff700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c154280 0x7fe95c152390 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7fe954009e90 tx=0x7fe95400b040).stop 2023-11-06T22:16:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.793+0000 7fe9647ff700 1 -- 172.21.15.33:0/3555170256 shutdown_connections 2023-11-06T22:16:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.793+0000 7fe9647ff700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe94005f270 0x7fe940061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.793+0000 7fe9647ff700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe95c0a9f20 0x7fe95c0c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.793+0000 7fe9647ff700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe95c154280 0x7fe95c152390 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.793+0000 7fe9647ff700 1 --2- 172.21.15.33:0/3555170256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe95c0ad6e0 0x7fe95c14aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.793+0000 7fe9647ff700 1 -- 172.21.15.33:0/3555170256 >> 172.21.15.33:0/3555170256 conn(0x7fe95c05e1b0 msgr2=0x7fe95c153520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.794+0000 7fe9647ff700 1 -- 172.21.15.33:0/3555170256 shutdown_connections 2023-11-06T22:16:23.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:23.794+0000 7fe9647ff700 1 -- 172.21.15.33:0/3555170256 wait complete. 2023-11-06T22:16:23.798 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 76 2023-11-06T22:16:23.815 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:16:23.828 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:23.826+0000 7f5957778680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:16:24.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.013+0000 7f17625ff700 1 Processor -- start 2023-11-06T22:16:24.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.013+0000 7f17625ff700 1 -- start start 2023-11-06T22:16:24.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.014+0000 7f17625ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f175c0be280 0x7f175c15f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f17625ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c1553b0 con 0x7f175c0c4e60 2023-11-06T22:16:24.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f17625ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c0c45c0 con 0x7f175c0c5240 2023-11-06T22:16:24.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f17625ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175c066c50 con 0x7f175c0be280 2023-11-06T22:16:24.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f175bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f175c0c4e60 0x7f175c066540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35254/0 (socket says 172.21.15.33:35254) 2023-11-06T22:16:24.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f175bfff700 1 -- 172.21.15.33:0/3050509469 learned_addr learned my addr 172.21.15.33:0/3050509469 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:24.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f1760b9c700 1 --2- 172.21.15.33:0/3050509469 >> v2:172.21.15.69:3300/0 conn(0x7f175c0be280 0x7f175c15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f175affd700 1 -- 172.21.15.33:0/3050509469 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377362233 0 0) 0x7f175c1553b0 con 0x7f175c0c4e60 2023-11-06T22:16:24.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f175affd700 1 -- 172.21.15.33:0/3050509469 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17440033f0 con 0x7f175c0c4e60 2023-11-06T22:16:24.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f175affd700 1 -- 172.21.15.33:0/3050509469 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855933557 0 0) 0x7f175c0c45c0 con 0x7f175c0c5240 2023-11-06T22:16:24.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f175affd700 1 -- 172.21.15.33:0/3050509469 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f175c1553b0 con 0x7f175c0c5240 2023-11-06T22:16:24.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f1760b9c700 1 -- 172.21.15.33:0/3050509469 >> v1:172.21.15.69:6789/0 conn(0x7f175c0c5240 legacy=0x7f175c0bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.015+0000 7f1760b9c700 1 -- 172.21.15.33:0/3050509469 >> v1:172.21.15.33:6789/0 conn(0x7f175c0c4e60 legacy=0x7f175c066540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.016+0000 7f1760b9c700 1 -- 172.21.15.33:0/3050509469 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c154ac0 con 0x7f175c0be280 2023-11-06T22:16:24.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.016+0000 7f1760b9c700 1 --2- 172.21.15.33:0/3050509469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f175c0be280 0x7f175c15f110 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f1754009e90 tx=0x7f1754009780).ready entity=mon.1 client_cookie=4dfa229a0a042519 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:24.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.016+0000 7f175affd700 1 -- 172.21.15.33:0/3050509469 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175400e070 con 0x7f175c0be280 2023-11-06T22:16:24.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.016+0000 7f175affd700 1 -- 172.21.15.33:0/3050509469 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175400bac0 con 0x7f175c0be280 2023-11-06T22:16:24.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.016+0000 7f175affd700 1 -- 172.21.15.33:0/3050509469 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175400be40 con 0x7f175c0be280 2023-11-06T22:16:24.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.017+0000 7f17625ff700 1 -- 172.21.15.33:0/3050509469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f175c0be280 msgr2=0x7f175c15f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.017+0000 7f17625ff700 1 --2- 172.21.15.33:0/3050509469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f175c0be280 0x7f175c15f110 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f1754009e90 tx=0x7f1754009780).stop 2023-11-06T22:16:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.017+0000 7f17625ff700 1 -- 172.21.15.33:0/3050509469 shutdown_connections 2023-11-06T22:16:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.017+0000 7f17625ff700 1 --2- 172.21.15.33:0/3050509469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f175c0be280 0x7f175c15f110 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.017+0000 7f17625ff700 1 -- 172.21.15.33:0/3050509469 >> 172.21.15.33:0/3050509469 conn(0x7f175c0c8c10 msgr2=0x7f175c0c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.017+0000 7f17625ff700 1 -- 172.21.15.33:0/3050509469 shutdown_connections 2023-11-06T22:16:24.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.017+0000 7f17625ff700 1 -- 172.21.15.33:0/3050509469 wait complete. 2023-11-06T22:16:24.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.018+0000 7f17625ff700 1 Processor -- start 2023-11-06T22:16:24.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.019+0000 7f17625ff700 1 -- start start 2023-11-06T22:16:24.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f17625ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f175c0be280 0x7f175c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f17625ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f175c0c4e60 0x7f175c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f17625ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f175c0c5240 0x7f175c0c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f17625ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1688d0 con 0x7f175c0be280 2023-11-06T22:16:24.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f17625ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c0a52a0 con 0x7f175c0c5240 2023-11-06T22:16:24.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f17625ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175c0a4c90 con 0x7f175c0c4e60 2023-11-06T22:16:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f175bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f175c0be280 0x7f175c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f175bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f175c0be280 0x7f175c153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56808/0 (socket says 172.21.15.33:56808) 2023-11-06T22:16:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f175bfff700 1 -- 172.21.15.33:0/1297660408 learned_addr learned my addr 172.21.15.33:0/1297660408 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f175b7fe700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f175c0c4e60 0x7f175c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f1760b9c700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f175c0c5240 0x7f175c0c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f175bfff700 1 -- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f175c0c4e60 msgr2=0x7f175c0a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f175bfff700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f175c0c4e60 0x7f175c0a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f175bfff700 1 -- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f175c0c5240 msgr2=0x7f175c0c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f175bfff700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f175c0c5240 0x7f175c0c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.020+0000 7f175bfff700 1 -- 172.21.15.33:0/1297660408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1754009a20 con 0x7f175c0be280 2023-11-06T22:16:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.021+0000 7f175bfff700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f175c0be280 0x7f175c153b20 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f174c00c560 tx=0x7f174c007d00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.021+0000 7f1743fff700 1 -- 172.21.15.33:0/1297660408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c00ce70 con 0x7f175c0be280 2023-11-06T22:16:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.021+0000 7f1743fff700 1 -- 172.21.15.33:0/1297660408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c0072e0 con 0x7f175c0be280 2023-11-06T22:16:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.021+0000 7f1743fff700 1 -- 172.21.15.33:0/1297660408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c007660 con 0x7f175c0be280 2023-11-06T22:16:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.021+0000 7f17625ff700 1 -- 172.21.15.33:0/1297660408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c1685f0 con 0x7f175c0be280 2023-11-06T22:16:24.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.021+0000 7f17625ff700 1 -- 172.21.15.33:0/1297660408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c154010 con 0x7f175c0be280 2023-11-06T22:16:24.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.022+0000 7f17625ff700 1 -- 172.21.15.33:0/1297660408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c0a4af0 con 0x7f175c0be280 2023-11-06T22:16:24.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.023+0000 7f1743fff700 1 -- 172.21.15.33:0/1297660408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f174c0059b0 con 0x7f175c0be280 2023-11-06T22:16:24.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.024+0000 7f1743fff700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f174805f270 0x7f1748061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.024+0000 7f1743fff700 1 -- 172.21.15.33:0/1297660408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f174c079ca0 con 0x7f175c0be280 2023-11-06T22:16:24.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.024+0000 7f175b7fe700 1 -- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f174805f270 msgr2=0x7f1748061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.024+0000 7f175b7fe700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f174805f270 0x7f1748061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.028+0000 7f1743fff700 1 -- 172.21.15.33:0/1297660408 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f175c0a4af0 con 0x7f175c0be280 2023-11-06T22:16:24.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.211+0000 7f17625ff700 1 -- 172.21.15.33:0/1297660408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f175c0a4af0 con 0x7f175c0be280 2023-11-06T22:16:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.212+0000 7f1743fff700 1 -- 172.21.15.33:0/1297660408 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+17888 (secure 0 0 0) 0x7f175c0a4af0 con 0x7f175c0be280 2023-11-06T22:16:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:24.215 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":190,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:16:22.260209+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:00:15.624386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T22:00:17.062698+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":190,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1d","mappings":[{"from":2,"to":5}]},{"pgid":"21.1e","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/540532913":"2023-11-07T22:11:22.119414+0000","172.21.15.69:0/2903646842":"2023-11-07T22:11:22.119414+0000","172.21.15.69:0/3284886886":"2023-11-07T22:11:22.119414+0000","172.21.15.69:0/3999769341":"2023-11-07T22:11:22.119414+0000","172.21.15.69:6833/114912":"2023-11-07T22:00:32.314047+0000","172.21.15.69:6832/114912":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/2728366018":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/515574665":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/1433505155":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/805490711":"2023-11-07T22:00:32.314047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:16:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.214+0000 7f17625ff700 1 -- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f174805f270 msgr2=0x7f1748061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.214+0000 7f17625ff700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f174805f270 0x7f1748061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.214+0000 7f17625ff700 1 -- 172.21.15.33:0/1297660408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f175c0be280 msgr2=0x7f175c153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.214+0000 7f17625ff700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f175c0be280 0x7f175c153b20 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f174c00c560 tx=0x7f174c007d00).stop 2023-11-06T22:16:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.215+0000 7f17625ff700 1 -- 172.21.15.33:0/1297660408 shutdown_connections 2023-11-06T22:16:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.215+0000 7f17625ff700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f174805f270 0x7f1748061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.215+0000 7f17625ff700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f175c0c5240 0x7f175c0c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.215+0000 7f17625ff700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f175c0be280 0x7f175c153b20 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.215+0000 7f17625ff700 1 --2- 172.21.15.33:0/1297660408 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f175c0c4e60 0x7f175c0a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.215+0000 7f17625ff700 1 -- 172.21.15.33:0/1297660408 >> 172.21.15.33:0/1297660408 conn(0x7f175c0c8c10 msgr2=0x7f175c065330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.215+0000 7f17625ff700 1 -- 172.21.15.33:0/1297660408 shutdown_connections 2023-11-06T22:16:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.215+0000 7f17625ff700 1 -- 172.21.15.33:0/1297660408 wait complete. 2023-11-06T22:16:24.234 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T22:16:24.235 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:16:24.378 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:24.376+0000 7ff8aec72680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:16:24.465 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:24.463+0000 7ff8aec72680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:16:24.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.471+0000 7ff59a9ff700 1 Processor -- start 2023-11-06T22:16:24.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.471+0000 7ff59a9ff700 1 -- start start 2023-11-06T22:16:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.473+0000 7ff59a9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff5940c0c50 0x7ff5940c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.473+0000 7ff59a9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff594154e90 con 0x7ff5941521a0 2023-11-06T22:16:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.473+0000 7ff59a9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5940639d0 con 0x7ff594151dc0 2023-11-06T22:16:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.473+0000 7ff59a9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff594155a30 con 0x7ff5940c0c50 2023-11-06T22:16:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.473+0000 7ff5937fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff5941521a0 0x7ff594155320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35268/0 (socket says 172.21.15.33:35268) 2023-11-06T22:16:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.473+0000 7ff5937fe700 1 -- 172.21.15.33:0/436660203 learned_addr learned my addr 172.21.15.33:0/436660203 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.473+0000 7ff598f9c700 1 --2- 172.21.15.33:0/436660203 >> v2:172.21.15.69:3300/0 conn(0x7ff5940c0c50 0x7ff5940c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.474+0000 7ff592ffd700 1 -- 172.21.15.33:0/436660203 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489938598 0 0) 0x7ff594154e90 con 0x7ff5941521a0 2023-11-06T22:16:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.474+0000 7ff592ffd700 1 -- 172.21.15.33:0/436660203 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5800033f0 con 0x7ff5941521a0 2023-11-06T22:16:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.474+0000 7ff598f9c700 1 -- 172.21.15.33:0/436660203 >> v1:172.21.15.69:6789/0 conn(0x7ff594151dc0 legacy=0x7ff594063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.474+0000 7ff592ffd700 1 -- 172.21.15.33:0/436660203 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885509191 0 0) 0x7ff5940639d0 con 0x7ff594151dc0 2023-11-06T22:16:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.474+0000 7ff598f9c700 1 -- 172.21.15.33:0/436660203 >> v1:172.21.15.33:6789/0 conn(0x7ff5941521a0 legacy=0x7ff594155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.474+0000 7ff598f9c700 1 -- 172.21.15.33:0/436660203 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594063cb0 con 0x7ff5940c0c50 2023-11-06T22:16:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.474+0000 7ff598f9c700 1 --2- 172.21.15.33:0/436660203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5940c0c50 0x7ff5940c7850 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7ff584009760 tx=0x7ff584009060).ready entity=mon.1 client_cookie=4994f48afe332341 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.474+0000 7ff592ffd700 1 -- 172.21.15.33:0/436660203 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400f510 con 0x7ff5940c0c50 2023-11-06T22:16:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.474+0000 7ff592ffd700 1 -- 172.21.15.33:0/436660203 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58400f670 con 0x7ff5940c0c50 2023-11-06T22:16:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.474+0000 7ff592ffd700 1 -- 172.21.15.33:0/436660203 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400f9f0 con 0x7ff5940c0c50 2023-11-06T22:16:24.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.475+0000 7ff59a9ff700 1 -- 172.21.15.33:0/436660203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5940c0c50 msgr2=0x7ff5940c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.475+0000 7ff59a9ff700 1 --2- 172.21.15.33:0/436660203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5940c0c50 0x7ff5940c7850 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7ff584009760 tx=0x7ff584009060).stop 2023-11-06T22:16:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.475+0000 7ff59a9ff700 1 -- 172.21.15.33:0/436660203 shutdown_connections 2023-11-06T22:16:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.475+0000 7ff59a9ff700 1 --2- 172.21.15.33:0/436660203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5940c0c50 0x7ff5940c7850 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.475+0000 7ff59a9ff700 1 -- 172.21.15.33:0/436660203 >> 172.21.15.33:0/436660203 conn(0x7ff59405e1b0 msgr2=0x7ff59405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.475+0000 7ff59a9ff700 1 -- 172.21.15.33:0/436660203 shutdown_connections 2023-11-06T22:16:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.475+0000 7ff59a9ff700 1 -- 172.21.15.33:0/436660203 wait complete. 2023-11-06T22:16:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.477+0000 7ff59a9ff700 1 Processor -- start 2023-11-06T22:16:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.477+0000 7ff59a9ff700 1 -- start start 2023-11-06T22:16:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff59a9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5940c0c50 0x7ff594151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff59a9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff594151dc0 0x7ff5940c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff59a9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5941521a0 0x7ff5940abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff59a9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff594151610 con 0x7ff594151dc0 2023-11-06T22:16:24.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff59a9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940c2860 con 0x7ff5941521a0 2023-11-06T22:16:24.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff59a9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940c2b40 con 0x7ff5940c0c50 2023-11-06T22:16:24.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff5937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff594151dc0 0x7ff5940c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff5937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff594151dc0 0x7ff5940c1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56822/0 (socket says 172.21.15.33:56822) 2023-11-06T22:16:24.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff5937fe700 1 -- 172.21.15.33:0/2385312421 learned_addr learned my addr 172.21.15.33:0/2385312421 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:24.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff593fff700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5940c0c50 0x7ff594151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.478+0000 7ff598f9c700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5941521a0 0x7ff5940abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff5937fe700 1 -- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5940c0c50 msgr2=0x7ff594151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff5937fe700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5940c0c50 0x7ff594151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff5937fe700 1 -- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5941521a0 msgr2=0x7ff5940abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff5937fe700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5941521a0 0x7ff5940abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff5937fe700 1 -- 172.21.15.33:0/2385312421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5840092a0 con 0x7ff594151dc0 2023-11-06T22:16:24.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff5937fe700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff594151dc0 0x7ff5940c1ec0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7ff57c002250 tx=0x7ff57c0015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:24.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff58bfff700 1 -- 172.21.15.33:0/2385312421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c005d30 con 0x7ff594151dc0 2023-11-06T22:16:24.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff58bfff700 1 -- 172.21.15.33:0/2385312421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c005e90 con 0x7ff594151dc0 2023-11-06T22:16:24.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff59a9ff700 1 -- 172.21.15.33:0/2385312421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5940c2e20 con 0x7ff594151dc0 2023-11-06T22:16:24.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff58bfff700 1 -- 172.21.15.33:0/2385312421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c007340 con 0x7ff594151dc0 2023-11-06T22:16:24.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.479+0000 7ff59a9ff700 1 -- 172.21.15.33:0/2385312421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5940c33e0 con 0x7ff594151dc0 2023-11-06T22:16:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.481+0000 7ff59a9ff700 1 -- 172.21.15.33:0/2385312421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5940aa6d0 con 0x7ff594151dc0 2023-11-06T22:16:24.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.482+0000 7ff58bfff700 1 -- 172.21.15.33:0/2385312421 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7ff57c0076f0 con 0x7ff594151dc0 2023-11-06T22:16:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.482+0000 7ff58bfff700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff57405f240 0x7ff574061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.482+0000 7ff58bfff700 1 -- 172.21.15.33:0/2385312421 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff57c07a610 con 0x7ff594151dc0 2023-11-06T22:16:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.483+0000 7ff593fff700 1 -- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff57405f240 msgr2=0x7ff574061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.483+0000 7ff593fff700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff57405f240 0x7ff574061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:24.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.486+0000 7ff58bfff700 1 -- 172.21.15.33:0/2385312421 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff57c04b450 con 0x7ff594151dc0 2023-11-06T22:16:24.560 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:24.558+0000 7ff8aec72680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:16:24.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.684+0000 7ff593fff700 1 -- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff57405f240 msgr2=0x7ff574061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:24.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.684+0000 7ff593fff700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff57405f240 0x7ff574061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:16:24.688 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:24.686+0000 7f5957778680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:16:24.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.711+0000 7ff59a9ff700 1 -- 172.21.15.33:0/2385312421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5940c0a30 con 0x7ff594151dc0 2023-11-06T22:16:24.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.712+0000 7ff58bfff700 1 -- 172.21.15.33:0/2385312421 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4490 (secure 0 0 0) 0x7ff57c0079a0 con 0x7ff594151dc0 2023-11-06T22:16:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:24.714 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":76,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3392183580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3392183580},{"type":"v1","addr":"172.21.15.33:6835","nonce":3392183580}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2124998325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2124998325},{"type":"v1","addr":"172.21.15.69:6837","nonce":2124998325}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3826680893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3826680893},{"type":"v1","addr":"172.21.15.33:6834","nonce":3826680893}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:00:18.069753+0000","modified":"2023-11-06T22:00:21.084401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8551":{"gid":8551,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/249801946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":249801946},{"type":"v1","addr":"172.21.15.69:6836","nonce":249801946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-06T22:16:24.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 -- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff57405f240 msgr2=0x7ff574061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:24.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff57405f240 0x7ff574061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:24.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 -- 172.21.15.33:0/2385312421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff594151dc0 msgr2=0x7ff5940c1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff594151dc0 0x7ff5940c1ec0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7ff57c002250 tx=0x7ff57c0015b0).stop 2023-11-06T22:16:24.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 -- 172.21.15.33:0/2385312421 shutdown_connections 2023-11-06T22:16:24.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff57405f240 0x7ff574061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:24.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5940c0c50 0x7ff594151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5941521a0 0x7ff5940abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 --2- 172.21.15.33:0/2385312421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff594151dc0 0x7ff5940c1ec0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 -- 172.21.15.33:0/2385312421 >> 172.21.15.33:0/2385312421 conn(0x7ff59405e1b0 msgr2=0x7ff5940c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:24.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 -- 172.21.15.33:0/2385312421 shutdown_connections 2023-11-06T22:16:24.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.714+0000 7ff59a9ff700 1 -- 172.21.15.33:0/2385312421 wait complete. 2023-11-06T22:16:24.719 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 76 2023-11-06T22:16:24.741 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:16:24.745 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:24.743+0000 7ff8aec72680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:16:24.797 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:24.795+0000 7f5957778680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:16:24.833 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:24.831+0000 7ff8aec72680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:16:24.899 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:24.897+0000 7f5957778680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:16:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.943+0000 7fbadffff700 1 Processor -- start 2023-11-06T22:16:24.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.943+0000 7fbadffff700 1 -- start start 2023-11-06T22:16:24.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.944+0000 7fbadffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad8154280 0x7fbad8154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.944+0000 7fbadffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbad80ad6e0 0x7fbad80c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.944+0000 7fbadffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad80c5db0 con 0x7fbad80a9f20 2023-11-06T22:16:24.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.944+0000 7fbadffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad80c6930 con 0x7fbad80ad6e0 2023-11-06T22:16:24.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.944+0000 7fbadffff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80c6000 con 0x7fbad8154280 2023-11-06T22:16:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.944+0000 7fbadd59a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbad80ad6e0 0x7fbad80c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.945+0000 7fbade59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbad80a9f20 0x7fbad80c7470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35274/0 (socket says 172.21.15.33:35274) 2023-11-06T22:16:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.945+0000 7fbade59c700 1 -- 172.21.15.33:0/2906047325 learned_addr learned my addr 172.21.15.33:0/2906047325 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.945+0000 7fbaddd9b700 1 --2- 172.21.15.33:0/2906047325 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad8154280 0x7fbad8154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.945+0000 7fbadd59a700 1 -- 172.21.15.33:0/2906047325 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad8154280 msgr2=0x7fbad8154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.945+0000 7fbadd59a700 1 --2- 172.21.15.33:0/2906047325 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad8154280 0x7fbad8154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.945+0000 7fbadd59a700 1 -- 172.21.15.33:0/2906047325 >> v1:172.21.15.33:6789/0 conn(0x7fbad80a9f20 legacy=0x7fbad80c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.945+0000 7fbadd59a700 1 -- 172.21.15.33:0/2906047325 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80c62e0 con 0x7fbad80ad6e0 2023-11-06T22:16:24.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.945+0000 7fbadd59a700 1 --2- 172.21.15.33:0/2906047325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80ad6e0 0x7fbad80c6dc0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fbac8009790 tx=0x7fbac8009060).ready entity=mon.0 client_cookie=90280a1928c015a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:24.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.945+0000 7fbadcd99700 1 -- 172.21.15.33:0/2906047325 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800f510 con 0x7fbad80ad6e0 2023-11-06T22:16:24.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.946+0000 7fbadcd99700 1 -- 172.21.15.33:0/2906047325 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac800f670 con 0x7fbad80ad6e0 2023-11-06T22:16:24.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.946+0000 7fbadcd99700 1 -- 172.21.15.33:0/2906047325 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800f9f0 con 0x7fbad80ad6e0 2023-11-06T22:16:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.946+0000 7fbadffff700 1 -- 172.21.15.33:0/2906047325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80ad6e0 msgr2=0x7fbad80c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.946+0000 7fbadffff700 1 --2- 172.21.15.33:0/2906047325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80ad6e0 0x7fbad80c6dc0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fbac8009790 tx=0x7fbac8009060).stop 2023-11-06T22:16:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.946+0000 7fbadffff700 1 -- 172.21.15.33:0/2906047325 shutdown_connections 2023-11-06T22:16:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.946+0000 7fbadffff700 1 --2- 172.21.15.33:0/2906047325 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad8154280 0x7fbad8154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.946+0000 7fbadffff700 1 --2- 172.21.15.33:0/2906047325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80ad6e0 0x7fbad80c6dc0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.946+0000 7fbadffff700 1 -- 172.21.15.33:0/2906047325 >> 172.21.15.33:0/2906047325 conn(0x7fbad805e1b0 msgr2=0x7fbad805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:24.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.946+0000 7fbadffff700 1 -- 172.21.15.33:0/2906047325 shutdown_connections 2023-11-06T22:16:24.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.946+0000 7fbadffff700 1 -- 172.21.15.33:0/2906047325 wait complete. 2023-11-06T22:16:24.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.948+0000 7fbadffff700 1 Processor -- start 2023-11-06T22:16:24.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.948+0000 7fbadffff700 1 -- start start 2023-11-06T22:16:24.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbadffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80a9f20 0x7fbad80a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbadffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80ad6e0 0x7fbad8164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbadffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad8154280 0x7fbad80a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbadffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80b02d0 con 0x7fbad80a9f20 2023-11-06T22:16:24.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbadffff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80a9c20 con 0x7fbad8154280 2023-11-06T22:16:24.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbadffff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad81654e0 con 0x7fbad80ad6e0 2023-11-06T22:16:24.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbaddd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80a9f20 0x7fbad80a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbaddd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80a9f20 0x7fbad80a9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56838/0 (socket says 172.21.15.33:56838) 2023-11-06T22:16:24.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbaddd9b700 1 -- 172.21.15.33:0/3741270706 learned_addr learned my addr 172.21.15.33:0/3741270706 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:24.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbadd59a700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80ad6e0 0x7fbad8164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbade59c700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad8154280 0x7fbad80a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:24.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbaddd9b700 1 -- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80ad6e0 msgr2=0x7fbad8164ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.949+0000 7fbaddd9b700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80ad6e0 0x7fbad8164ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.950+0000 7fbaddd9b700 1 -- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad8154280 msgr2=0x7fbad80a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:24.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.950+0000 7fbaddd9b700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad8154280 0x7fbad80a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:24.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.950+0000 7fbaddd9b700 1 -- 172.21.15.33:0/3741270706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac80092a0 con 0x7fbad80a9f20 2023-11-06T22:16:24.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.950+0000 7fbaddd9b700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80a9f20 0x7fbad80a9730 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fbad400e900 tx=0x7fbad4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:24.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.950+0000 7fbacdffb700 1 -- 172.21.15.33:0/3741270706 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad400c640 con 0x7fbad80a9f20 2023-11-06T22:16:24.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.950+0000 7fbacdffb700 1 -- 172.21.15.33:0/3741270706 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad400c7a0 con 0x7fbad80a9f20 2023-11-06T22:16:24.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.950+0000 7fbacdffb700 1 -- 172.21.15.33:0/3741270706 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad400cb20 con 0x7fbad80a9f20 2023-11-06T22:16:24.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.950+0000 7fbadffff700 1 -- 172.21.15.33:0/3741270706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad80af650 con 0x7fbad80a9f20 2023-11-06T22:16:24.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.950+0000 7fbadffff700 1 -- 172.21.15.33:0/3741270706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad80afc10 con 0x7fbad80a9f20 2023-11-06T22:16:24.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.951+0000 7fbadffff700 1 -- 172.21.15.33:0/3741270706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad80c65c0 con 0x7fbad80a9f20 2023-11-06T22:16:24.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.952+0000 7fbacdffb700 1 -- 172.21.15.33:0/3741270706 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fbad401b400 con 0x7fbad80a9f20 2023-11-06T22:16:24.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.953+0000 7fbacdffb700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbabc05f240 0x7fbabc061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:24.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.953+0000 7fbacdffb700 1 -- 172.21.15.33:0/3741270706 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbad407ab00 con 0x7fbad80a9f20 2023-11-06T22:16:24.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.953+0000 7fbadd59a700 1 -- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbabc05f240 msgr2=0x7fbabc061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:24.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.953+0000 7fbadd59a700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbabc05f240 0x7fbabc061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:24.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:24.961+0000 7fbacdffb700 1 -- 172.21.15.33:0/3741270706 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbad407ce60 con 0x7fbad80a9f20 2023-11-06T22:16:24.975 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:24.973+0000 7ff8aec72680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:16:25.101 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:25.099+0000 7f5957778680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:16:25.106 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.104+0000 7ff8aec72680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:16:25.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.154+0000 7fbadd59a700 1 -- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbabc05f240 msgr2=0x7fbabc061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:25.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.154+0000 7fbadd59a700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbabc05f240 0x7fbabc061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:16:25.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.190+0000 7fbadffff700 1 -- 172.21.15.33:0/3741270706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbad806ab40 con 0x7fbad80a9f20 2023-11-06T22:16:25.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.191+0000 7fbacdffb700 1 -- 172.21.15.33:0/3741270706 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4490 (secure 0 0 0) 0x7fbad4016070 con 0x7fbad80a9f20 2023-11-06T22:16:25.192 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:25.193 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":76,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3392183580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3392183580},{"type":"v1","addr":"172.21.15.33:6835","nonce":3392183580}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2124998325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2124998325},{"type":"v1","addr":"172.21.15.69:6837","nonce":2124998325}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3826680893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3826680893},{"type":"v1","addr":"172.21.15.33:6834","nonce":3826680893}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:00:18.069753+0000","modified":"2023-11-06T22:00:21.084401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8551":{"gid":8551,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/249801946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":249801946},{"type":"v1","addr":"172.21.15.69:6836","nonce":249801946}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-06T22:16:25.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 -- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbabc05f240 msgr2=0x7fbabc061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:25.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbabc05f240 0x7fbabc061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:25.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 -- 172.21.15.33:0/3741270706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80a9f20 msgr2=0x7fbad80a9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80a9f20 0x7fbad80a9730 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fbad400e900 tx=0x7fbad4009060).stop 2023-11-06T22:16:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 -- 172.21.15.33:0/3741270706 shutdown_connections 2023-11-06T22:16:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbabc05f240 0x7fbabc061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbad8154280 0x7fbad80a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbad80a9f20 0x7fbad80a9730 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 --2- 172.21.15.33:0/3741270706 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbad80ad6e0 0x7fbad8164ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 -- 172.21.15.33:0/3741270706 >> 172.21.15.33:0/3741270706 conn(0x7fbad805e1b0 msgr2=0x7fbad8063040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:25.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 -- 172.21.15.33:0/3741270706 shutdown_connections 2023-11-06T22:16:25.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.193+0000 7fbadffff700 1 -- 172.21.15.33:0/3741270706 wait complete. 2023-11-06T22:16:25.197 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:25.194+0000 7f5957778680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:16:25.198 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 76 2023-11-06T22:16:25.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:16:25.353 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:25.352+0000 7f5957778680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:16:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.415+0000 7f0ab91ff700 1 Processor -- start 2023-11-06T22:16:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.415+0000 7f0ab91ff700 1 -- start start 2023-11-06T22:16:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.416+0000 7f0ab91ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0ab40ad6e0 0x7f0ab4154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.416+0000 7f0ab91ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab40c5db0 con 0x7f0ab4154280 2023-11-06T22:16:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.416+0000 7f0ab91ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab40c6930 con 0x7f0ab40c6dc0 2023-11-06T22:16:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.416+0000 7f0ab91ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab40aa630 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.417+0000 7f0ab2d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0ab4154280 0x7f0ab40a9f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35278/0 (socket says 172.21.15.33:35278) 2023-11-06T22:16:25.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.417+0000 7f0ab2d9d700 1 -- 172.21.15.33:0/1252124859 learned_addr learned my addr 172.21.15.33:0/1252124859 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:25.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.417+0000 7f0ab259c700 1 --2- 172.21.15.33:0/1252124859 >> v2:172.21.15.69:3300/0 conn(0x7f0ab40ad6e0 0x7f0ab4154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:25.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.417+0000 7f0ab1d9b700 1 -- 172.21.15.33:0/1252124859 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608170409 0 0) 0x7f0ab40c5db0 con 0x7f0ab4154280 2023-11-06T22:16:25.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.417+0000 7f0ab1d9b700 1 -- 172.21.15.33:0/1252124859 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a9c0033f0 con 0x7f0ab4154280 2023-11-06T22:16:25.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.417+0000 7f0ab259c700 1 -- 172.21.15.33:0/1252124859 >> v1:172.21.15.69:6789/0 conn(0x7f0ab40c6dc0 legacy=0x7f0ab40c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.417+0000 7f0ab1d9b700 1 -- 172.21.15.33:0/1252124859 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393827686 0 0) 0x7f0ab40c6930 con 0x7f0ab40c6dc0 2023-11-06T22:16:25.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.417+0000 7f0ab259c700 1 -- 172.21.15.33:0/1252124859 >> v1:172.21.15.33:6789/0 conn(0x7f0ab4154280 legacy=0x7f0ab40a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.418+0000 7f0ab259c700 1 -- 172.21.15.33:0/1252124859 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40c6000 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.418+0000 7f0ab259c700 1 --2- 172.21.15.33:0/1252124859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ab40ad6e0 0x7f0ab4154660 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f0aa8009760 tx=0x7f0aa8009060).ready entity=mon.1 client_cookie=359e84471d834bd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:25.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.418+0000 7f0ab1d9b700 1 -- 172.21.15.33:0/1252124859 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4001470 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.418+0000 7f0ab1d9b700 1 -- 172.21.15.33:0/1252124859 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8012410 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.418+0000 7f0ab1d9b700 1 -- 172.21.15.33:0/1252124859 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8012790 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.419+0000 7f0ab91ff700 1 -- 172.21.15.33:0/1252124859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ab40ad6e0 msgr2=0x7f0ab4154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.419+0000 7f0ab91ff700 1 --2- 172.21.15.33:0/1252124859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ab40ad6e0 0x7f0ab4154660 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f0aa8009760 tx=0x7f0aa8009060).stop 2023-11-06T22:16:25.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.419+0000 7f0ab91ff700 1 -- 172.21.15.33:0/1252124859 shutdown_connections 2023-11-06T22:16:25.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.419+0000 7f0ab91ff700 1 --2- 172.21.15.33:0/1252124859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ab40ad6e0 0x7f0ab4154660 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.419+0000 7f0ab91ff700 1 -- 172.21.15.33:0/1252124859 >> 172.21.15.33:0/1252124859 conn(0x7f0ab405e1b0 msgr2=0x7f0ab405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:25.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.419+0000 7f0ab91ff700 1 -- 172.21.15.33:0/1252124859 shutdown_connections 2023-11-06T22:16:25.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.419+0000 7f0ab91ff700 1 -- 172.21.15.33:0/1252124859 wait complete. 2023-11-06T22:16:25.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.420+0000 7f0ab91ff700 1 Processor -- start 2023-11-06T22:16:25.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.420+0000 7f0ab91ff700 1 -- start start 2023-11-06T22:16:25.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.421+0000 7f0ab91ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ab40ad6e0 0x7f0ab40c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:25.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.421+0000 7f0ab91ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ab40c6dc0 0x7f0ab41587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:25.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.421+0000 7f0ab91ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab4154280 0x7f0ab40cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:25.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.421+0000 7f0ab91ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40ca6d0 con 0x7f0ab4154280 2023-11-06T22:16:25.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.421+0000 7f0ab91ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40c9100 con 0x7f0ab40c6dc0 2023-11-06T22:16:25.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.421+0000 7f0ab91ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40c93e0 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab2d9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ab40ad6e0 0x7f0ab40c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:25.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab4154280 0x7f0ab40cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ab40c6dc0 0x7f0ab41587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab2d9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ab40ad6e0 0x7f0ab40c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:53292/0 (socket says 172.21.15.33:53292) 2023-11-06T22:16:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab2d9d700 1 -- 172.21.15.33:0/3925446534 learned_addr learned my addr 172.21.15.33:0/3925446534 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:25.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab2d9d700 1 -- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ab40c6dc0 msgr2=0x7f0ab41587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab2d9d700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ab40c6dc0 0x7f0ab41587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab2d9d700 1 -- 172.21.15.33:0/3925446534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab4154280 msgr2=0x7f0ab40cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab2d9d700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab4154280 0x7f0ab40cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab2d9d700 1 -- 172.21.15.33:0/3925446534 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80092a0 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab259c700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ab40c6dc0 0x7f0ab41587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:25.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.422+0000 7f0ab2d9d700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ab40ad6e0 0x7f0ab40c8840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f0aa400c760 tx=0x7f0aa4001ad0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:25.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.423+0000 7f0a9affd700 1 -- 172.21.15.33:0/3925446534 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4007170 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.423+0000 7f0a9affd700 1 -- 172.21.15.33:0/3925446534 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa40072d0 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.423+0000 7f0ab91ff700 1 -- 172.21.15.33:0/3925446534 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab40c96c0 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.423+0000 7f0a9affd700 1 -- 172.21.15.33:0/3925446534 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40048c0 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.423+0000 7f0ab91ff700 1 -- 172.21.15.33:0/3925446534 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab40c8d30 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.424+0000 7f0ab91ff700 1 -- 172.21.15.33:0/3925446534 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab4062ac0 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.425+0000 7f0a9affd700 1 -- 172.21.15.33:0/3925446534 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f0aa4004c70 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.425+0000 7f0a9affd700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0aa005f270 0x7f0aa0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:25.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.426+0000 7f0a9affd700 1 -- 172.21.15.33:0/3925446534 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0aa407a280 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.426+0000 7f0ab259c700 1 -- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0aa005f270 msgr2=0x7f0aa0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:25.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.426+0000 7f0ab259c700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0aa005f270 0x7f0aa0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:25.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.430+0000 7f0a9affd700 1 -- 172.21.15.33:0/3925446534 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0aa404b170 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.455 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.453+0000 7ff8aec72680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:16:25.494 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:25.492+0000 7f5957778680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:16:25.545 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.543+0000 7ff8aec72680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:16:25.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.559+0000 7f0a9affd700 1 -- 172.21.15.33:0/3925446534 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mgrmap(e 86) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f0aa4077bc0 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.559+0000 7f0a9affd700 1 -- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0aa005f270 msgr2=0x7f0aa0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:25.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.559+0000 7f0a9affd700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0aa005f270 0x7f0aa0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:25.620 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.617+0000 7ff82874f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:25.620 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.617+0000 7ff82874f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:25.620 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.617+0000 7ff82874f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:25.620 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.617+0000 7ff82874f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:25.620 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.617+0000 7ff82874f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:25.621 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.617+0000 7ff826f4c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:25.621 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.617+0000 7ff826f4c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:25.621 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.617+0000 7ff826f4c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:25.621 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.617+0000 7ff826f4c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:25.621 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:25.617+0000 7ff826f4c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:25.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.623+0000 7f0ab91ff700 1 -- 172.21.15.33:0/3925446534 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ab406ab40 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.624+0000 7f0a9affd700 1 -- 172.21.15.33:0/3925446534 <== mon.2 v2:172.21.15.69:3301/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+18210 (secure 0 0 0) 0x7f0aa406e730 con 0x7f0ab40ad6e0 2023-11-06T22:16:25.626 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:25.626 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":191,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:16:25.547212+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:00:15.624386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T22:00:17.062698+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":190,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1d","mappings":[{"from":2,"to":5}]},{"pgid":"21.1e","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/555515371":"2023-11-07T22:16:25.547171+0000","172.21.15.69:6833/129946":"2023-11-07T22:16:25.547171+0000","172.21.15.69:0/220399095":"2023-11-07T22:16:25.547171+0000","172.21.15.69:6832/129946":"2023-11-07T22:16:25.547171+0000","172.21.15.69:0/2733795845":"2023-11-07T22:16:25.547171+0000","172.21.15.69:0/540532913":"2023-11-07T22:11:22.119414+0000","172.21.15.69:0/2903646842":"2023-11-07T22:11:22.119414+0000","172.21.15.69:0/3284886886":"2023-11-07T22:11:22.119414+0000","172.21.15.69:0/3999769341":"2023-11-07T22:11:22.119414+0000","172.21.15.69:6833/114912":"2023-11-07T22:00:32.314047+0000","172.21.15.69:6832/114912":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/2728366018":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/515574665":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/1433505155":"2023-11-07T22:00:32.314047+0000","172.21.15.69:0/151889627":"2023-11-07T22:16:25.547171+0000","172.21.15.69:0/805490711":"2023-11-07T22:00:32.314047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:16:25.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.626+0000 7f0ab91ff700 1 -- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ab40ad6e0 msgr2=0x7f0ab40c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.626+0000 7f0ab91ff700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ab40ad6e0 0x7f0ab40c8840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f0aa400c760 tx=0x7f0aa4001ad0).stop 2023-11-06T22:16:25.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.626+0000 7f0ab91ff700 1 -- 172.21.15.33:0/3925446534 shutdown_connections 2023-11-06T22:16:25.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.626+0000 7f0ab91ff700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0aa005f270 0x7f0aa0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.626+0000 7f0ab91ff700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0ab40ad6e0 0x7f0ab40c8840 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.626+0000 7f0ab91ff700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab4154280 0x7f0ab40cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.626+0000 7f0ab91ff700 1 --2- 172.21.15.33:0/3925446534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ab40c6dc0 0x7f0ab41587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.626+0000 7f0ab91ff700 1 -- 172.21.15.33:0/3925446534 >> 172.21.15.33:0/3925446534 conn(0x7f0ab405e1b0 msgr2=0x7f0ab40b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.626+0000 7f0ab91ff700 1 -- 172.21.15.33:0/3925446534 shutdown_connections 2023-11-06T22:16:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.627+0000 7f0ab91ff700 1 -- 172.21.15.33:0/3925446534 wait complete. 2023-11-06T22:16:25.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T22:16:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.852+0000 7fac94dff700 1 Processor -- start 2023-11-06T22:16:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.853+0000 7fac94dff700 1 -- start start 2023-11-06T22:16:25.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.854+0000 7fac94dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fac900c2900 0x7fac900c3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:25.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.854+0000 7fac94dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:25.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.854+0000 7fac94dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac900c2470 con 0x7fac900c3530 2023-11-06T22:16:25.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.854+0000 7fac94dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac900c15e0 con 0x7fac900c2900 2023-11-06T22:16:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.854+0000 7fac94dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac900c1860 con 0x7fac90153d00 2023-11-06T22:16:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.854+0000 7fac8ed9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.854+0000 7fac8ed9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:53306/0 (socket says 172.21.15.33:53306) 2023-11-06T22:16:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.854+0000 7fac8dd9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fac900c2900 0x7fac900c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.854+0000 7fac8e59c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fac900c3530 0x7fac900bc470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:47846/0 (socket says 172.21.15.33:47846) 2023-11-06T22:16:25.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.854+0000 7fac8ed9d700 1 -- 172.21.15.33:0/2551519432 learned_addr learned my addr 172.21.15.33:0/2551519432 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:25.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.855+0000 7fac8d59a700 1 -- 172.21.15.33:0/2551519432 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275293641 0 0) 0x7fac900c2470 con 0x7fac900c3530 2023-11-06T22:16:25.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.855+0000 7fac8d59a700 1 -- 172.21.15.33:0/2551519432 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac780033f0 con 0x7fac900c3530 2023-11-06T22:16:25.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.855+0000 7fac8dd9b700 1 -- 172.21.15.33:0/2551519432 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 msgr2=0x7fac901541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:25.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.855+0000 7fac8dd9b700 1 --2- 172.21.15.33:0/2551519432 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.855+0000 7fac8dd9b700 1 -- 172.21.15.33:0/2551519432 >> v1:172.21.15.69:6789/0 conn(0x7fac900c3530 legacy=0x7fac900bc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.855+0000 7fac8dd9b700 1 -- 172.21.15.33:0/2551519432 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac900c1b40 con 0x7fac900c2900 2023-11-06T22:16:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.855+0000 7fac8ed9d700 1 --2- 172.21.15.33:0/2551519432 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.855+0000 7fac8dd9b700 1 --2- 172.21.15.33:0/2551519432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c2900 0x7fac900c3910 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fac84009760 tx=0x7fac84009060).ready entity=mon.1 client_cookie=8ef22e08bf920620 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.855+0000 7fac8d59a700 1 -- 172.21.15.33:0/2551519432 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400d4a0 con 0x7fac900c2900 2023-11-06T22:16:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.856+0000 7fac8d59a700 1 -- 172.21.15.33:0/2551519432 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8400d600 con 0x7fac900c2900 2023-11-06T22:16:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.856+0000 7fac8d59a700 1 -- 172.21.15.33:0/2551519432 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400d980 con 0x7fac900c2900 2023-11-06T22:16:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.856+0000 7fac94dff700 1 -- 172.21.15.33:0/2551519432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c2900 msgr2=0x7fac900c3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.856+0000 7fac94dff700 1 --2- 172.21.15.33:0/2551519432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c2900 0x7fac900c3910 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fac84009760 tx=0x7fac84009060).stop 2023-11-06T22:16:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.856+0000 7fac94dff700 1 -- 172.21.15.33:0/2551519432 shutdown_connections 2023-11-06T22:16:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.856+0000 7fac94dff700 1 --2- 172.21.15.33:0/2551519432 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.856+0000 7fac94dff700 1 --2- 172.21.15.33:0/2551519432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c2900 0x7fac900c3910 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.856+0000 7fac94dff700 1 -- 172.21.15.33:0/2551519432 >> 172.21.15.33:0/2551519432 conn(0x7fac90062710 msgr2=0x7fac90062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.856+0000 7fac94dff700 1 -- 172.21.15.33:0/2551519432 shutdown_connections 2023-11-06T22:16:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.856+0000 7fac94dff700 1 -- 172.21.15.33:0/2551519432 wait complete. 2023-11-06T22:16:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.858+0000 7fac94dff700 1 Processor -- start 2023-11-06T22:16:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.858+0000 7fac94dff700 1 -- start start 2023-11-06T22:16:25.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac94dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c2900 0x7fac9014c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac94dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900c3530 0x7fac900a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac94dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac94dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac901560e0 con 0x7fac900c3530 2023-11-06T22:16:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac94dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90155db0 con 0x7fac900c2900 2023-11-06T22:16:25.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac94dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac90154b00 con 0x7fac90153d00 2023-11-06T22:16:25.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac8dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900c3530 0x7fac900a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:25.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac8dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900c3530 0x7fac900a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56858/0 (socket says 172.21.15.33:56858) 2023-11-06T22:16:25.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac8dd9b700 1 -- 172.21.15.33:0/4229718678 learned_addr learned my addr 172.21.15.33:0/4229718678 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:25.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac8e59c700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c2900 0x7fac9014c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:25.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.859+0000 7fac8ed9d700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac8dd9b700 1 -- 172.21.15.33:0/4229718678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 msgr2=0x7fac901541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac8dd9b700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac8dd9b700 1 -- 172.21.15.33:0/4229718678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c2900 msgr2=0x7fac9014c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:25.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac8dd9b700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c2900 0x7fac9014c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:25.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac8dd9b700 1 -- 172.21.15.33:0/4229718678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88009ae0 con 0x7fac900c3530 2023-11-06T22:16:25.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac8e59c700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c2900 0x7fac9014c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:25.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac8ed9d700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:25.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac8dd9b700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900c3530 0x7fac900a9a10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fac9005efa0 tx=0x7fac84016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:25.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4229718678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac84021070 con 0x7fac900c3530 2023-11-06T22:16:25.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4229718678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac84000bc0 con 0x7fac900c3530 2023-11-06T22:16:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4229718678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400d980 con 0x7fac900c3530 2023-11-06T22:16:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac94dff700 1 -- 172.21.15.33:0/4229718678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac840092a0 con 0x7fac900c3530 2023-11-06T22:16:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.860+0000 7fac94dff700 1 -- 172.21.15.33:0/4229718678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac901550c0 con 0x7fac900c3530 2023-11-06T22:16:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.861+0000 7fac94dff700 1 -- 172.21.15.33:0/4229718678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac90065900 con 0x7fac900c3530 2023-11-06T22:16:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.862+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4229718678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 86) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fac8401d640 con 0x7fac900c3530 2023-11-06T22:16:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.863+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4229718678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6939+0+0 (secure 0 0 0) 0x7fac8407b7c0 con 0x7fac900c3530 2023-11-06T22:16:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:25.867+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4229718678 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fac8404c930 con 0x7fac900c3530 2023-11-06T22:16:25.872 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:25.869+0000 7f5957778680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:16:25.967 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:25.965+0000 7f5957778680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:16:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.091+0000 7fac94dff700 1 -- 172.21.15.33:0/4229718678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fac9005f080 con 0x7fac900c3530 2023-11-06T22:16:26.552 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T22:16:26.551+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T22:16:26.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.556+0000 7fac7e7fc700 1 -- 172.21.15.33:0/4229718678 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) v1 ==== 161+0+0 (secure 0 0 0) 0x7fac8404ff50 con 0x7fac900c3530 2023-11-06T22:16:26.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.558+0000 7fac94dff700 1 -- 172.21.15.33:0/4229718678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900c3530 msgr2=0x7fac900a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:26.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.558+0000 7fac94dff700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900c3530 0x7fac900a9a10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fac9005efa0 tx=0x7fac84016000).stop 2023-11-06T22:16:26.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.559+0000 7fac94dff700 1 -- 172.21.15.33:0/4229718678 shutdown_connections 2023-11-06T22:16:26.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.559+0000 7fac94dff700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fac90153d00 0x7fac901541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:26.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.559+0000 7fac94dff700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac900c2900 0x7fac9014c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:26.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.559+0000 7fac94dff700 1 --2- 172.21.15.33:0/4229718678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900c3530 0x7fac900a9a10 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:26.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.559+0000 7fac94dff700 1 -- 172.21.15.33:0/4229718678 >> 172.21.15.33:0/4229718678 conn(0x7fac90062710 msgr2=0x7fac90065020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:26.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.559+0000 7fac94dff700 1 -- 172.21.15.33:0/4229718678 shutdown_connections 2023-11-06T22:16:26.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.559+0000 7fac94dff700 1 -- 172.21.15.33:0/4229718678 wait complete. 2023-11-06T22:16:26.562 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T22:16:26.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T22:16:26.624 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T22:16:26.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.779+0000 7f05cb5ff700 1 Processor -- start 2023-11-06T22:16:26.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.779+0000 7f05cb5ff700 1 -- start start 2023-11-06T22:16:26.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.781+0000 7f05cb5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f05c4157700 0x7f05c40c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:26.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.781+0000 7f05cb5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f05c4063840 0x7f05c40c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:26.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.781+0000 7f05cb5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c4158c00 con 0x7f05c40aaf50 2023-11-06T22:16:26.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.781+0000 7f05cb5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c40c18f0 con 0x7f05c4157700 2023-11-06T22:16:26.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.781+0000 7f05cb5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c4157ff0 con 0x7f05c4063840 2023-11-06T22:16:26.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.781+0000 7f05c939b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f05c4157700 0x7f05c40c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:26.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.781+0000 7f05c8b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f05c40aaf50 0x7f05c4063130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35290/0 (socket says 172.21.15.33:35290) 2023-11-06T22:16:26.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.781+0000 7f05c8b9a700 1 -- 172.21.15.33:0/1265588186 learned_addr learned my addr 172.21.15.33:0/1265588186 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:26.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.781+0000 7f05c9b9c700 1 --2- 172.21.15.33:0/1265588186 >> v2:172.21.15.69:3300/0 conn(0x7f05c4063840 0x7f05c40c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:26.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05c939b700 1 -- 172.21.15.33:0/1265588186 >> v2:172.21.15.69:3300/0 conn(0x7f05c4063840 msgr2=0x7f05c40c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:26.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05c939b700 1 --2- 172.21.15.33:0/1265588186 >> v2:172.21.15.69:3300/0 conn(0x7f05c4063840 0x7f05c40c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:26.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05c939b700 1 -- 172.21.15.33:0/1265588186 >> v1:172.21.15.33:6789/0 conn(0x7f05c40aaf50 legacy=0x7f05c4063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:26.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05c939b700 1 -- 172.21.15.33:0/1265588186 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c41582d0 con 0x7f05c4157700 2023-11-06T22:16:26.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05c9b9c700 1 --2- 172.21.15.33:0/1265588186 >> v2:172.21.15.69:3300/0 conn(0x7f05c4063840 0x7f05c40c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:26.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05c939b700 1 --2- 172.21.15.33:0/1265588186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4157700 0x7f05c40c6f00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f05c40c5400 tx=0x7f05c0009060).ready entity=mon.0 client_cookie=f0b391cdf7885895 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:26.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05bbfff700 1 -- 172.21.15.33:0/1265588186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c000b580 con 0x7f05c4157700 2023-11-06T22:16:26.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05bbfff700 1 -- 172.21.15.33:0/1265588186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c000b6e0 con 0x7f05c4157700 2023-11-06T22:16:26.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05bbfff700 1 -- 172.21.15.33:0/1265588186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c000ba60 con 0x7f05c4157700 2023-11-06T22:16:26.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05cb5ff700 1 -- 172.21.15.33:0/1265588186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4157700 msgr2=0x7f05c40c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:26.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.782+0000 7f05cb5ff700 1 --2- 172.21.15.33:0/1265588186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4157700 0x7f05c40c6f00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f05c40c5400 tx=0x7f05c0009060).stop 2023-11-06T22:16:26.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.783+0000 7f05cb5ff700 1 -- 172.21.15.33:0/1265588186 shutdown_connections 2023-11-06T22:16:26.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.783+0000 7f05cb5ff700 1 --2- 172.21.15.33:0/1265588186 >> v2:172.21.15.69:3300/0 conn(0x7f05c4063840 0x7f05c40c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:26.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.783+0000 7f05cb5ff700 1 --2- 172.21.15.33:0/1265588186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c4157700 0x7f05c40c6f00 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:26.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.783+0000 7f05cb5ff700 1 -- 172.21.15.33:0/1265588186 >> 172.21.15.33:0/1265588186 conn(0x7f05c405ec80 msgr2=0x7f05c40cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:26.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.783+0000 7f05cb5ff700 1 -- 172.21.15.33:0/1265588186 shutdown_connections 2023-11-06T22:16:26.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.783+0000 7f05cb5ff700 1 -- 172.21.15.33:0/1265588186 wait complete. 2023-11-06T22:16:26.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.784+0000 7f05cb5ff700 1 Processor -- start 2023-11-06T22:16:26.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.784+0000 7f05cb5ff700 1 -- start start 2023-11-06T22:16:26.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.785+0000 7f05cb5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c4063840 0x7f05c414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:26.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.785+0000 7f05cb5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c40aaf50 0x7f05c40afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:26.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.785+0000 7f05cb5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05c4157700 0x7f05c4152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.785+0000 7f05cb5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40b0f90 con 0x7f05c40aaf50 2023-11-06T22:16:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.785+0000 7f05cb5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40ae600 con 0x7f05c4063840 2023-11-06T22:16:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.785+0000 7f05cb5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40b06f0 con 0x7f05c4157700 2023-11-06T22:16:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c40aaf50 0x7f05c40afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:26.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c939b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c4063840 0x7f05c414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c40aaf50 0x7f05c40afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56872/0 (socket says 172.21.15.33:56872) 2023-11-06T22:16:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c8b9a700 1 -- 172.21.15.33:0/3826690607 learned_addr learned my addr 172.21.15.33:0/3826690607 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c9b9c700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05c4157700 0x7f05c4152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c8b9a700 1 -- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05c4157700 msgr2=0x7f05c4152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c8b9a700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05c4157700 0x7f05c4152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:26.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c8b9a700 1 -- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c4063840 msgr2=0x7f05c414cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:26.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c8b9a700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c4063840 0x7f05c414cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:26.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c8b9a700 1 -- 172.21.15.33:0/3826690607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c00092a0 con 0x7f05c40aaf50 2023-11-06T22:16:26.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c939b700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c4063840 0x7f05c414cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:26.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.786+0000 7f05c8b9a700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c40aaf50 0x7f05c40afdf0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f05b4007fd0 tx=0x7f05b4007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:26.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.787+0000 7f05b97fa700 1 -- 172.21.15.33:0/3826690607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b400ce10 con 0x7f05c40aaf50 2023-11-06T22:16:26.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.787+0000 7f05b97fa700 1 -- 172.21.15.33:0/3826690607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b40072e0 con 0x7f05c40aaf50 2023-11-06T22:16:26.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.787+0000 7f05b97fa700 1 -- 172.21.15.33:0/3826690607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4007660 con 0x7f05c40aaf50 2023-11-06T22:16:26.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.787+0000 7f05cb5ff700 1 -- 172.21.15.33:0/3826690607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c40b09d0 con 0x7f05c40aaf50 2023-11-06T22:16:26.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.787+0000 7f05cb5ff700 1 -- 172.21.15.33:0/3826690607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c414d1f0 con 0x7f05c40aaf50 2023-11-06T22:16:26.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.788+0000 7f05cb5ff700 1 -- 172.21.15.33:0/3826690607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c40ae3d0 con 0x7f05c40aaf50 2023-11-06T22:16:26.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.789+0000 7f05b97fa700 1 -- 172.21.15.33:0/3826690607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 87) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f05b40059b0 con 0x7f05c40aaf50 2023-11-06T22:16:26.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.790+0000 7f05b97fa700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f05a805f030 0x7f05a80614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:26.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.790+0000 7f05b97fa700 1 -- 172.21.15.33:0/3826690607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 7025+0+0 (secure 0 0 0) 0x7f05b4079a70 con 0x7f05c40aaf50 2023-11-06T22:16:26.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.790+0000 7f05c939b700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f05a805f030 0x7f05a80614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:26.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.791+0000 7f05c939b700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f05a805f030 0x7f05a80614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f05c0009fd0 tx=0x7f05c0017000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:26.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:26.793+0000 7f05b97fa700 1 -- 172.21.15.33:0/3826690607 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f05b404aa70 con 0x7f05c40aaf50 2023-11-06T22:16:27.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.022+0000 7f05cb5ff700 1 -- 172.21.15.33:0/3826690607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f05c406c6c0 con 0x7f05c40aaf50 2023-11-06T22:16:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.566+0000 7f05b97fa700 1 -- 172.21.15.33:0/3826690607 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) v1 ==== 104+0+0 (secure 0 0 0) 0x7f05b404e090 con 0x7f05c40aaf50 2023-11-06T22:16:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.568+0000 7f05cb5ff700 1 -- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f05a805f030 msgr2=0x7f05a80614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.568+0000 7f05cb5ff700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f05a805f030 0x7f05a80614f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f05c0009fd0 tx=0x7f05c0017000).stop 2023-11-06T22:16:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.569+0000 7f05cb5ff700 1 -- 172.21.15.33:0/3826690607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c40aaf50 msgr2=0x7f05c40afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.569+0000 7f05cb5ff700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c40aaf50 0x7f05c40afdf0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f05b4007fd0 tx=0x7f05b4007df0).stop 2023-11-06T22:16:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.569+0000 7f05cb5ff700 1 -- 172.21.15.33:0/3826690607 shutdown_connections 2023-11-06T22:16:27.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.569+0000 7f05cb5ff700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f05a805f030 0x7f05a80614f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:27.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.569+0000 7f05cb5ff700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f05c4157700 0x7f05c4152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:27.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.569+0000 7f05cb5ff700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c4063840 0x7f05c414cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.569+0000 7f05cb5ff700 1 --2- 172.21.15.33:0/3826690607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05c40aaf50 0x7f05c40afdf0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.569+0000 7f05cb5ff700 1 -- 172.21.15.33:0/3826690607 >> 172.21.15.33:0/3826690607 conn(0x7f05c405ec80 msgr2=0x7f05c4156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.569+0000 7f05cb5ff700 1 -- 172.21.15.33:0/3826690607 shutdown_connections 2023-11-06T22:16:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.569+0000 7f05cb5ff700 1 -- 172.21.15.33:0/3826690607 wait complete. 2023-11-06T22:16:27.590 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T22:16:27.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.790+0000 7fd9dbbff700 1 Processor -- start 2023-11-06T22:16:27.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.790+0000 7fd9dbbff700 1 -- start start 2023-11-06T22:16:27.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.792+0000 7fd9dbbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c8ee0 0x7fd9d40ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:27.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.792+0000 7fd9dbbff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9d41542b0 0x7fd9d40c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:27.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.792+0000 7fd9dbbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d40c9f80 con 0x7fd9d4153ed0 2023-11-06T22:16:27.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.792+0000 7fd9dbbff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40c6000 con 0x7fd9d41542b0 2023-11-06T22:16:27.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.792+0000 7fd9dbbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40c9370 con 0x7fd9d40c8ee0 2023-11-06T22:16:27.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.792+0000 7fd9da19c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9d41542b0 0x7fd9d40c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:27.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.792+0000 7fd9da19c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9d41542b0 0x7fd9d40c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56886/0 (socket says 172.21.15.33:56886) 2023-11-06T22:16:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.792+0000 7fd9da19c700 1 -- 172.21.15.33:0/4264670061 learned_addr learned my addr 172.21.15.33:0/4264670061 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.792+0000 7fd9d919a700 1 --2- 172.21.15.33:0/4264670061 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c8ee0 0x7fd9d40ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.793+0000 7fd9da19c700 1 -- 172.21.15.33:0/4264670061 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c8ee0 msgr2=0x7fd9d40ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.793+0000 7fd9da19c700 1 --2- 172.21.15.33:0/4264670061 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c8ee0 0x7fd9d40ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.793+0000 7fd9da19c700 1 -- 172.21.15.33:0/4264670061 >> v1:172.21.15.69:6789/0 conn(0x7fd9d4153ed0 legacy=0x7fd9d4158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.793+0000 7fd9da19c700 1 -- 172.21.15.33:0/4264670061 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40c9650 con 0x7fd9d41542b0 2023-11-06T22:16:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.793+0000 7fd9da19c700 1 --2- 172.21.15.33:0/4264670061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 0x7fd9d40c76f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7fd9cc009f20 tx=0x7fd9cc009840).ready entity=mon.0 client_cookie=f3bea9c98602fce7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:27.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.793+0000 7fd9d8999700 1 -- 172.21.15.33:0/4264670061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00e070 con 0x7fd9d41542b0 2023-11-06T22:16:27.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.793+0000 7fd9d8999700 1 -- 172.21.15.33:0/4264670061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc00b9d0 con 0x7fd9d41542b0 2023-11-06T22:16:27.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.793+0000 7fd9d8999700 1 -- 172.21.15.33:0/4264670061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00bd50 con 0x7fd9d41542b0 2023-11-06T22:16:27.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.794+0000 7fd9dbbff700 1 -- 172.21.15.33:0/4264670061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 msgr2=0x7fd9d40c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:27.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.794+0000 7fd9dbbff700 1 --2- 172.21.15.33:0/4264670061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 0x7fd9d40c76f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7fd9cc009f20 tx=0x7fd9cc009840).stop 2023-11-06T22:16:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.794+0000 7fd9dbbff700 1 -- 172.21.15.33:0/4264670061 shutdown_connections 2023-11-06T22:16:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.794+0000 7fd9dbbff700 1 --2- 172.21.15.33:0/4264670061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 0x7fd9d40c76f0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.794+0000 7fd9dbbff700 1 --2- 172.21.15.33:0/4264670061 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c8ee0 0x7fd9d40ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.794+0000 7fd9dbbff700 1 -- 172.21.15.33:0/4264670061 >> 172.21.15.33:0/4264670061 conn(0x7fd9d405e1b0 msgr2=0x7fd9d405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.794+0000 7fd9dbbff700 1 -- 172.21.15.33:0/4264670061 shutdown_connections 2023-11-06T22:16:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.794+0000 7fd9dbbff700 1 -- 172.21.15.33:0/4264670061 wait complete. 2023-11-06T22:16:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.795+0000 7fd9dbbff700 1 Processor -- start 2023-11-06T22:16:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.795+0000 7fd9dbbff700 1 -- start start 2023-11-06T22:16:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.796+0000 7fd9dbbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d4153ed0 0x7fd9d4141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.796+0000 7fd9dbbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 0x7fd9d40c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.796+0000 7fd9dbbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c3380 0x7fd9d40c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.796+0000 7fd9dbbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d4150490 con 0x7fd9d41542b0 2023-11-06T22:16:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.796+0000 7fd9dbbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40af410 con 0x7fd9d4153ed0 2023-11-06T22:16:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.796+0000 7fd9dbbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40c3cb0 con 0x7fd9d40c3380 2023-11-06T22:16:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9d919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 0x7fd9d40c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9d919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 0x7fd9d40c2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56896/0 (socket says 172.21.15.33:56896) 2023-11-06T22:16:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9d919a700 1 -- 172.21.15.33:0/495211968 learned_addr learned my addr 172.21.15.33:0/495211968 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9da19c700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c3380 0x7fd9d40c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9d999b700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d4153ed0 0x7fd9d4141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9d919a700 1 -- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c3380 msgr2=0x7fd9d40c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9d919a700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c3380 0x7fd9d40c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:27.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9d919a700 1 -- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d4153ed0 msgr2=0x7fd9d4141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:27.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9d919a700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d4153ed0 0x7fd9d4141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:27.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9d919a700 1 -- 172.21.15.33:0/495211968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc009ae0 con 0x7fd9d41542b0 2023-11-06T22:16:27.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9da19c700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c3380 0x7fd9d40c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:27.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9d919a700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 0x7fd9d40c2e90 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7fd9c400e930 tx=0x7fd9c4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:27.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.797+0000 7fd9c9ffb700 1 -- 172.21.15.33:0/495211968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c400eed0 con 0x7fd9d41542b0 2023-11-06T22:16:27.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.798+0000 7fd9c9ffb700 1 -- 172.21.15.33:0/495211968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c400c850 con 0x7fd9d41542b0 2023-11-06T22:16:27.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.798+0000 7fd9dbbff700 1 -- 172.21.15.33:0/495211968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d4153560 con 0x7fd9d41542b0 2023-11-06T22:16:27.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.798+0000 7fd9c9ffb700 1 -- 172.21.15.33:0/495211968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c400cbd0 con 0x7fd9d41542b0 2023-11-06T22:16:27.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.798+0000 7fd9dbbff700 1 -- 172.21.15.33:0/495211968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d40aee40 con 0x7fd9d41542b0 2023-11-06T22:16:27.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.799+0000 7fd9dbbff700 1 -- 172.21.15.33:0/495211968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d40af580 con 0x7fd9d41542b0 2023-11-06T22:16:27.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.799+0000 7fd9c9ffb700 1 -- 172.21.15.33:0/495211968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 87) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fd9c401b400 con 0x7fd9d41542b0 2023-11-06T22:16:27.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.800+0000 7fd9c9ffb700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd9b805f030 0x7fd9b80614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:27.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.800+0000 7fd9c9ffb700 1 -- 172.21.15.33:0/495211968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7025+0+0 (secure 0 0 0) 0x7fd9c407a960 con 0x7fd9d41542b0 2023-11-06T22:16:27.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.801+0000 7fd9d999b700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd9b805f030 0x7fd9b80614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:27.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.801+0000 7fd9d999b700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd9b805f030 0x7fd9b80614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fd9d00036d0 tx=0x7fd9d0009040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:27.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.804+0000 7fd9c9ffb700 1 -- 172.21.15.33:0/495211968 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd9c404ba40 con 0x7fd9d41542b0 2023-11-06T22:16:27.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:27.985+0000 7fd9dbbff700 1 -- 172.21.15.33:0/495211968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9d40c90e0 con 0x7fd9d41542b0 2023-11-06T22:16:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.226+0000 7fd9c9ffb700 1 -- 172.21.15.33:0/495211968 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v194) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd9c404f060 con 0x7fd9d41542b0 2023-11-06T22:16:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.230+0000 7fd9dbbff700 1 -- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd9b805f030 msgr2=0x7fd9b80614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.230+0000 7fd9dbbff700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd9b805f030 0x7fd9b80614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fd9d00036d0 tx=0x7fd9d0009040).stop 2023-11-06T22:16:28.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.230+0000 7fd9dbbff700 1 -- 172.21.15.33:0/495211968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 msgr2=0x7fd9d40c2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:28.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.230+0000 7fd9dbbff700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 0x7fd9d40c2e90 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7fd9c400e930 tx=0x7fd9c4009060).stop 2023-11-06T22:16:28.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.230+0000 7fd9dbbff700 1 -- 172.21.15.33:0/495211968 shutdown_connections 2023-11-06T22:16:28.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.230+0000 7fd9dbbff700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd9b805f030 0x7fd9b80614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:28.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.230+0000 7fd9dbbff700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd9d40c3380 0x7fd9d40c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:28.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.230+0000 7fd9dbbff700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d4153ed0 0x7fd9d4141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:28.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.230+0000 7fd9dbbff700 1 --2- 172.21.15.33:0/495211968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9d41542b0 0x7fd9d40c2e90 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:28.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.230+0000 7fd9dbbff700 1 -- 172.21.15.33:0/495211968 >> 172.21.15.33:0/495211968 conn(0x7fd9d405e1b0 msgr2=0x7fd9d414d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:28.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.231+0000 7fd9dbbff700 1 -- 172.21.15.33:0/495211968 shutdown_connections 2023-11-06T22:16:28.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.231+0000 7fd9dbbff700 1 -- 172.21.15.33:0/495211968 wait complete. 2023-11-06T22:16:28.235 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T22:16:28.253 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T22:16:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.454+0000 7f422e9ff700 1 Processor -- start 2023-11-06T22:16:28.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.454+0000 7f422e9ff700 1 -- start start 2023-11-06T22:16:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.456+0000 7f422e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f422815d390 0x7f422815f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.456+0000 7f422e9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42280a3da0 con 0x7f422813dea0 2023-11-06T22:16:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.456+0000 7f422e9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4228065090 con 0x7f4228158770 2023-11-06T22:16:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.456+0000 7f422e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42280c2f30 con 0x7f422815d390 2023-11-06T22:16:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.456+0000 7f4227fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f422813dea0 0x7f4228158060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44704/0 (socket says 172.21.15.33:44704) 2023-11-06T22:16:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.456+0000 7f4227fff700 1 -- 172.21.15.33:0/3072278648 learned_addr learned my addr 172.21.15.33:0/3072278648 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.456+0000 7f422cf9c700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f422815d390 0x7f422815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164985903 0 0) 0x7f42280a3da0 con 0x7f422813dea0 2023-11-06T22:16:28.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42100033f0 con 0x7f422813dea0 2023-11-06T22:16:28.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011394546 0 0) 0x7f4228065090 con 0x7f4228158770 2023-11-06T22:16:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42280a3da0 con 0x7f4228158770 2023-11-06T22:16:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2622109941 0 0) 0x7f42100033f0 con 0x7f422813dea0 2023-11-06T22:16:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f422815d390 msgr2=0x7f422815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f422815d390 0x7f422815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 >> v1:172.21.15.69:6789/0 conn(0x7f4228158770 legacy=0x7f42280c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42280c3200 con 0x7f422813dea0 2023-11-06T22:16:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f4218001470 con 0x7f422813dea0 2023-11-06T22:16:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 >> v1:172.21.15.33:6789/0 conn(0x7f422813dea0 legacy=0x7f4228158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4210003aa0 0x7f4210003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f422815d390 0x7f4210004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4210004b80 0x7f4210005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42280c2f30 con 0x7f4210004b80 2023-11-06T22:16:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421000b9c0 con 0x7f4210003aa0 2023-11-06T22:16:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421000bb80 con 0x7f422815d390 2023-11-06T22:16:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f422cf9c700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4210004b80 0x7f4210005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.457+0000 7f4227fff700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4210003aa0 0x7f4210003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.458+0000 7f42277fe700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f422815d390 0x7f4210004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.458+0000 7f422cf9c700 1 -- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f422815d390 msgr2=0x7f4210004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.458+0000 7f422cf9c700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f422815d390 0x7f4210004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.458+0000 7f422cf9c700 1 -- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4210003aa0 msgr2=0x7f4210003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.458+0000 7f422cf9c700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4210003aa0 0x7f4210003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.458+0000 7f422cf9c700 1 -- 172.21.15.33:0/3072278648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100033f0 con 0x7f4210004b80 2023-11-06T22:16:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.458+0000 7f422cf9c700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4210004b80 0x7f4210005000 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f422805f2e0 tx=0x7f422000b040).ready entity=mon.0 client_cookie=70d480705b286e00 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.458+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220009ac0 con 0x7f4210004b80 2023-11-06T22:16:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.458+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422000f040 con 0x7f4210004b80 2023-11-06T22:16:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.458+0000 7f4226ffd700 1 -- 172.21.15.33:0/3072278648 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422000eca0 con 0x7f4210004b80 2023-11-06T22:16:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.459+0000 7f422e9ff700 1 -- 172.21.15.33:0/3072278648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4210004b80 msgr2=0x7f4210005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.459+0000 7f422e9ff700 1 --2- 172.21.15.33:0/3072278648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4210004b80 0x7f4210005000 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f422805f2e0 tx=0x7f422000b040).stop 2023-11-06T22:16:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.459+0000 7f422cf9c700 1 -- 172.21.15.33:0/3072278648 reap_dead start 2023-11-06T22:16:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.459+0000 7f422e9ff700 1 -- 172.21.15.33:0/3072278648 shutdown_connections 2023-11-06T22:16:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.459+0000 7f422e9ff700 1 -- 172.21.15.33:0/3072278648 >> 172.21.15.33:0/3072278648 conn(0x7f42280c6e70 msgr2=0x7f42280c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.459+0000 7f422e9ff700 1 -- 172.21.15.33:0/3072278648 shutdown_connections 2023-11-06T22:16:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.459+0000 7f422e9ff700 1 -- 172.21.15.33:0/3072278648 wait complete. 2023-11-06T22:16:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.460+0000 7f422e9ff700 1 Processor -- start 2023-11-06T22:16:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.461+0000 7f422e9ff700 1 -- start start 2023-11-06T22:16:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.461+0000 7f422e9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42280c1fe0 0x7f4228168900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.461+0000 7f422e9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42280c23c0 0x7f4228093320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.461+0000 7f422e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4228091a70 0x7f422813bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.461+0000 7f422e9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42280c2820 con 0x7f42280c1fe0 2023-11-06T22:16:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.461+0000 7f422e9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4228093810 con 0x7f42280c23c0 2023-11-06T22:16:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.461+0000 7f422e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4228092e90 con 0x7f4228091a70 2023-11-06T22:16:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f4227fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42280c1fe0 0x7f4228168900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f4227fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42280c1fe0 0x7f4228168900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58168/0 (socket says 172.21.15.33:58168) 2023-11-06T22:16:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f4227fff700 1 -- 172.21.15.33:0/3447303790 learned_addr learned my addr 172.21.15.33:0/3447303790 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f42277fe700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42280c23c0 0x7f4228093320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f422cf9c700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4228091a70 0x7f422813bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f4227fff700 1 -- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4228091a70 msgr2=0x7f422813bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f4227fff700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4228091a70 0x7f422813bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f4227fff700 1 -- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42280c23c0 msgr2=0x7f4228093320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f4227fff700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42280c23c0 0x7f4228093320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f4227fff700 1 -- 172.21.15.33:0/3447303790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4228092280 con 0x7f42280c1fe0 2023-11-06T22:16:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f4227fff700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42280c1fe0 0x7f4228168900 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f4218007840 tx=0x7f4218001e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f420ffff700 1 -- 172.21.15.33:0/3447303790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421800f5f0 con 0x7f42280c1fe0 2023-11-06T22:16:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.462+0000 7f420ffff700 1 -- 172.21.15.33:0/3447303790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421800f750 con 0x7f42280c1fe0 2023-11-06T22:16:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.463+0000 7f420ffff700 1 -- 172.21.15.33:0/3447303790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421800faf0 con 0x7f42280c1fe0 2023-11-06T22:16:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.463+0000 7f422e9ff700 1 -- 172.21.15.33:0/3447303790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4228092560 con 0x7f42280c1fe0 2023-11-06T22:16:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.463+0000 7f422e9ff700 1 -- 172.21.15.33:0/3447303790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4228092ba0 con 0x7f42280c1fe0 2023-11-06T22:16:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.464+0000 7f422e9ff700 1 -- 172.21.15.33:0/3447303790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4228093170 con 0x7f42280c1fe0 2023-11-06T22:16:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.465+0000 7f420ffff700 1 -- 172.21.15.33:0/3447303790 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4218015410 con 0x7f42280c1fe0 2023-11-06T22:16:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.465+0000 7f420ffff700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f421405f270 0x7f4214061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.465+0000 7f420ffff700 1 -- 172.21.15.33:0/3447303790 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f421807a840 con 0x7f42280c1fe0 2023-11-06T22:16:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.466+0000 7f42277fe700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f421405f270 0x7f4214061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.466+0000 7f42277fe700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f421405f270 0x7f4214061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f421c007cc0 tx=0x7f421c009040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.470+0000 7f420ffff700 1 -- 172.21.15.33:0/3447303790 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f421804b6d0 con 0x7f42280c1fe0 2023-11-06T22:16:28.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:28.668+0000 7f422e9ff700 1 -- 172.21.15.33:0/3447303790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4228064760 con 0x7f42280c1fe0 2023-11-06T22:16:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.570+0000 7f420ffff700 1 -- 172.21.15.33:0/3447303790 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v195) v1 ==== 169+0+0 (secure 0 0 0) 0x7f421804ecf0 con 0x7f42280c1fe0 2023-11-06T22:16:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 -- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f421405f270 msgr2=0x7f4214061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f421405f270 0x7f4214061730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f421c007cc0 tx=0x7f421c009040).stop 2023-11-06T22:16:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 -- 172.21.15.33:0/3447303790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42280c1fe0 msgr2=0x7f4228168900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42280c1fe0 0x7f4228168900 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f4218007840 tx=0x7f4218001e10).stop 2023-11-06T22:16:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 -- 172.21.15.33:0/3447303790 shutdown_connections 2023-11-06T22:16:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f421405f270 0x7f4214061730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4228091a70 0x7f422813bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42280c1fe0 0x7f4228168900 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 --2- 172.21.15.33:0/3447303790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42280c23c0 0x7f4228093320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 -- 172.21.15.33:0/3447303790 >> 172.21.15.33:0/3447303790 conn(0x7f42280c6e70 msgr2=0x7f422813ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.573+0000 7f422e9ff700 1 -- 172.21.15.33:0/3447303790 shutdown_connections 2023-11-06T22:16:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.574+0000 7f422e9ff700 1 -- 172.21.15.33:0/3447303790 wait complete. 2023-11-06T22:16:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T22:16:29.595 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T22:16:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.798+0000 7f374d7ff700 1 Processor -- start 2023-11-06T22:16:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.798+0000 7f374d7ff700 1 -- start start 2023-11-06T22:16:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.800+0000 7f374d7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748063840 0x7f374815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:29.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.800+0000 7f374d7ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3748158c00 con 0x7f37480aaf50 2023-11-06T22:16:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.800+0000 7f374d7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37480c7610 con 0x7f3748157700 2023-11-06T22:16:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.800+0000 7f374d7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37480c18f0 con 0x7f3748063840 2023-11-06T22:16:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.800+0000 7f37467fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f37480aaf50 0x7f3748063130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44714/0 (socket says 172.21.15.33:44714) 2023-11-06T22:16:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.800+0000 7f37467fc700 1 -- 172.21.15.33:0/2739985084 learned_addr learned my addr 172.21.15.33:0/2739985084 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.800+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682826188 0 0) 0x7f3748158c00 con 0x7f37480aaf50 2023-11-06T22:16:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.800+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37300033f0 con 0x7f37480aaf50 2023-11-06T22:16:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.800+0000 7f37477fe700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748063840 0x7f374815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:29.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2416119878 0 0) 0x7f37300033f0 con 0x7f37480aaf50 2023-11-06T22:16:29.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748063840 msgr2=0x7f374815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:29.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748063840 0x7f374815f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:29.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 >> v1:172.21.15.69:6789/0 conn(0x7f3748157700 legacy=0x7f37480c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:29.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3748157ff0 con 0x7f37480aaf50 2023-11-06T22:16:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f373c001470 con 0x7f37480aaf50 2023-11-06T22:16:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 >> v1:172.21.15.33:6789/0 conn(0x7f37480aaf50 legacy=0x7f3748063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748063840 0x7f3730003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 0x7f3730004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3730004b80 0x7f3730005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:29.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37480c18f0 con 0x7f3730003e40 2023-11-06T22:16:29.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373000b990 con 0x7f3730004b80 2023-11-06T22:16:29.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373000bb50 con 0x7f3748063840 2023-11-06T22:16:29.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f37467fc700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 0x7f3730004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:29.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f37477fe700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3730004b80 0x7f3730005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:29.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.801+0000 7f3746ffd700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748063840 0x7f3730003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f37467fc700 1 -- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748063840 msgr2=0x7f3730003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f37467fc700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748063840 0x7f3730003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f37467fc700 1 -- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3730004b80 msgr2=0x7f3730005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f37467fc700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3730004b80 0x7f3730005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:29.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f37467fc700 1 -- 172.21.15.33:0/2739985084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37300033f0 con 0x7f3730003e40 2023-11-06T22:16:29.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f37467fc700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 0x7f3730004690 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f373c00c190 tx=0x7f373c001e50).ready entity=mon.0 client_cookie=f731bc1067a09df7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:29.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c007a20 con 0x7f3730003e40 2023-11-06T22:16:29.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c00cac0 con 0x7f3730003e40 2023-11-06T22:16:29.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f3745ffb700 1 -- 172.21.15.33:0/2739985084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c00ce60 con 0x7f3730003e40 2023-11-06T22:16:29.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f374d7ff700 1 -- 172.21.15.33:0/2739985084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 msgr2=0x7f3730004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f374d7ff700 1 --2- 172.21.15.33:0/2739985084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 0x7f3730004690 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f373c00c190 tx=0x7f373c001e50).stop 2023-11-06T22:16:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.802+0000 7f37477fe700 1 -- 172.21.15.33:0/2739985084 reap_dead start 2023-11-06T22:16:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.803+0000 7f374d7ff700 1 -- 172.21.15.33:0/2739985084 shutdown_connections 2023-11-06T22:16:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.803+0000 7f374d7ff700 1 -- 172.21.15.33:0/2739985084 >> 172.21.15.33:0/2739985084 conn(0x7f374805ec80 msgr2=0x7f37480cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:29.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.803+0000 7f374d7ff700 1 -- 172.21.15.33:0/2739985084 shutdown_connections 2023-11-06T22:16:29.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.803+0000 7f374d7ff700 1 -- 172.21.15.33:0/2739985084 wait complete. 2023-11-06T22:16:29.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.805+0000 7f374d7ff700 1 Processor -- start 2023-11-06T22:16:29.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.805+0000 7f374d7ff700 1 -- start start 2023-11-06T22:16:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.806+0000 7f374d7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 0x7f37481532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:29.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.806+0000 7f374d7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748155c60 0x7f37481560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:29.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.806+0000 7f374d7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37481426e0 0x7f3748142b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:29.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.806+0000 7f374d7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3748143030 con 0x7f3730003e40 2023-11-06T22:16:29.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.806+0000 7f374d7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3748153b70 con 0x7f37481426e0 2023-11-06T22:16:29.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.806+0000 7f374d7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3748153e50 con 0x7f3748155c60 2023-11-06T22:16:29.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.806+0000 7f3746ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 0x7f37481532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:29.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f3746ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 0x7f37481532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58180/0 (socket says 172.21.15.33:58180) 2023-11-06T22:16:29.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f3746ffd700 1 -- 172.21.15.33:0/4114939298 learned_addr learned my addr 172.21.15.33:0/4114939298 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:29.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f37467fc700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748155c60 0x7f37481560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f37477fe700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37481426e0 0x7f3748142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f3746ffd700 1 -- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748155c60 msgr2=0x7f37481560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f3746ffd700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748155c60 0x7f37481560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f3746ffd700 1 -- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37481426e0 msgr2=0x7f3748142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f3746ffd700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37481426e0 0x7f3748142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:29.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f3746ffd700 1 -- 172.21.15.33:0/4114939298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3748154130 con 0x7f3730003e40 2023-11-06T22:16:29.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f37477fe700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37481426e0 0x7f3748142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:29.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f3746ffd700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 0x7f37481532b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f37380028c0 tx=0x7f3738007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:29.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f372effd700 1 -- 172.21.15.33:0/4114939298 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373800f610 con 0x7f3730003e40 2023-11-06T22:16:29.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f372effd700 1 -- 172.21.15.33:0/4114939298 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373800f770 con 0x7f3730003e40 2023-11-06T22:16:29.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f372effd700 1 -- 172.21.15.33:0/4114939298 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373800faf0 con 0x7f3730003e40 2023-11-06T22:16:29.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.807+0000 7f374d7ff700 1 -- 172.21.15.33:0/4114939298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3748154410 con 0x7f3730003e40 2023-11-06T22:16:29.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.808+0000 7f374d7ff700 1 -- 172.21.15.33:0/4114939298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37480af190 con 0x7f3730003e40 2023-11-06T22:16:29.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.809+0000 7f374d7ff700 1 -- 172.21.15.33:0/4114939298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37480af500 con 0x7f3730003e40 2023-11-06T22:16:29.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.809+0000 7f372effd700 1 -- 172.21.15.33:0/4114939298 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f373801d3b0 con 0x7f3730003e40 2023-11-06T22:16:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.810+0000 7f372effd700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f373405f270 0x7f3734061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.810+0000 7f372effd700 1 -- 172.21.15.33:0/4114939298 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f373807a0b0 con 0x7f3730003e40 2023-11-06T22:16:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.810+0000 7f37467fc700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f373405f270 0x7f3734061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.811+0000 7f37467fc700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f373405f270 0x7f3734061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3748152f20 tx=0x7f373c015000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.814+0000 7f372effd700 1 -- 172.21.15.33:0/4114939298 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f373804b1b0 con 0x7f3730003e40 2023-11-06T22:16:29.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.993+0000 7f374d7ff700 1 -- 172.21.15.33:0/4114939298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f374806c6c0 con 0x7f3730003e40 2023-11-06T22:16:29.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.996+0000 7f372effd700 1 -- 172.21.15.33:0/4114939298 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v36) v1 ==== 120+0+0 (secure 0 0 0) 0x7f37380190b0 con 0x7f3730003e40 2023-11-06T22:16:30.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.998+0000 7f374d7ff700 1 -- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f373405f270 msgr2=0x7f3734061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.998+0000 7f374d7ff700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f373405f270 0x7f3734061730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3748152f20 tx=0x7f373c015000).stop 2023-11-06T22:16:30.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.998+0000 7f374d7ff700 1 -- 172.21.15.33:0/4114939298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 msgr2=0x7f37481532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.998+0000 7f374d7ff700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 0x7f37481532b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f37380028c0 tx=0x7f3738007bb0).stop 2023-11-06T22:16:30.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.998+0000 7f374d7ff700 1 -- 172.21.15.33:0/4114939298 shutdown_connections 2023-11-06T22:16:30.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.998+0000 7f374d7ff700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f373405f270 0x7f3734061730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.998+0000 7f374d7ff700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37481426e0 0x7f3748142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.998+0000 7f374d7ff700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3730003e40 0x7f37481532b0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.999+0000 7f374d7ff700 1 --2- 172.21.15.33:0/4114939298 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3748155c60 0x7f37481560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.999+0000 7f374d7ff700 1 -- 172.21.15.33:0/4114939298 >> 172.21.15.33:0/4114939298 conn(0x7f374805ec80 msgr2=0x7f3748151f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:30.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.999+0000 7f374d7ff700 1 -- 172.21.15.33:0/4114939298 shutdown_connections 2023-11-06T22:16:30.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:29.999+0000 7f374d7ff700 1 -- 172.21.15.33:0/4114939298 wait complete. 2023-11-06T22:16:30.017 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T22:16:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.214+0000 7f7efa3ff700 1 Processor -- start 2023-11-06T22:16:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.214+0000 7f7efa3ff700 1 -- start start 2023-11-06T22:16:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.215+0000 7f7efa3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7ef4157700 0x7f7ef40c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.215+0000 7f7efa3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40aaf50 0x7f7ef4063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.215+0000 7f7efa3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef4158c00 con 0x7f7ef40aa4f0 2023-11-06T22:16:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.215+0000 7f7efa3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef40c18f0 con 0x7f7ef4157700 2023-11-06T22:16:30.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.215+0000 7f7efa3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef40c6d80 con 0x7f7ef40aaf50 2023-11-06T22:16:30.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef899c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7ef40aa4f0 0x7f7ef40646d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44718/0 (socket says 172.21.15.33:44718) 2023-11-06T22:16:30.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef899c700 1 -- 172.21.15.33:0/4242639927 learned_addr learned my addr 172.21.15.33:0/4242639927 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:30.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef37fe700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40aaf50 0x7f7ef4063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef3fff700 1 --2- 172.21.15.33:0/4242639927 >> v2:172.21.15.69:3300/0 conn(0x7f7ef4157700 0x7f7ef40c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322041148 0 0) 0x7f7ef4158c00 con 0x7f7ef40aa4f0 2023-11-06T22:16:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7edc0033f0 con 0x7f7ef40aa4f0 2023-11-06T22:16:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1310266329 0 0) 0x7f7edc0033f0 con 0x7f7ef40aa4f0 2023-11-06T22:16:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40aaf50 msgr2=0x7f7ef4063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:30.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef2ffd700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40aaf50 0x7f7ef4063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 >> v2:172.21.15.69:3300/0 conn(0x7f7ef4157700 msgr2=0x7f7ef40c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:30.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.216+0000 7f7ef2ffd700 1 --2- 172.21.15.33:0/4242639927 >> v2:172.21.15.69:3300/0 conn(0x7f7ef4157700 0x7f7ef40c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef4157ff0 con 0x7f7ef40aa4f0 2023-11-06T22:16:30.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f7eec0017f0 con 0x7f7ef40aa4f0 2023-11-06T22:16:30.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 >> v1:172.21.15.33:6789/0 conn(0x7f7ef40aa4f0 legacy=0x7f7ef40646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef2ffd700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7edc003910 0x7f7edc003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef2ffd700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40aaf50 0x7f7edc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef2ffd700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc004b80 0x7f7edc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef40c18f0 con 0x7f7edc003910 2023-11-06T22:16:30.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef40c6d80 con 0x7f7edc004b80 2023-11-06T22:16:30.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc00b8e0 con 0x7f7ef40aaf50 2023-11-06T22:16:30.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef3fff700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7edc003910 0x7f7edc003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef899c700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc004b80 0x7f7edc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef37fe700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40aaf50 0x7f7edc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef3fff700 1 -- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40aaf50 msgr2=0x7f7edc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:30.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef3fff700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40aaf50 0x7f7edc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef3fff700 1 -- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc004b80 msgr2=0x7f7edc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:30.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef3fff700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc004b80 0x7f7edc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.217+0000 7f7ef3fff700 1 -- 172.21.15.33:0/4242639927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc0033f0 con 0x7f7edc003910 2023-11-06T22:16:30.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.218+0000 7f7ef3fff700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7edc003910 0x7f7edc003d30 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f7ee4009a80 tx=0x7f7ee4009060).ready entity=mon.0 client_cookie=684ce6820030dacf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:30.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.218+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4009430 con 0x7f7edc003910 2023-11-06T22:16:30.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.218+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee400e640 con 0x7f7edc003910 2023-11-06T22:16:30.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.218+0000 7f7ef2ffd700 1 -- 172.21.15.33:0/4242639927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400e9c0 con 0x7f7edc003910 2023-11-06T22:16:30.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.218+0000 7f7efa3ff700 1 -- 172.21.15.33:0/4242639927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7edc003910 msgr2=0x7f7edc003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.218+0000 7f7efa3ff700 1 --2- 172.21.15.33:0/4242639927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7edc003910 0x7f7edc003d30 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f7ee4009a80 tx=0x7f7ee4009060).stop 2023-11-06T22:16:30.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.218+0000 7f7ef899c700 1 -- 172.21.15.33:0/4242639927 reap_dead start 2023-11-06T22:16:30.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.218+0000 7f7efa3ff700 1 -- 172.21.15.33:0/4242639927 shutdown_connections 2023-11-06T22:16:30.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.218+0000 7f7efa3ff700 1 -- 172.21.15.33:0/4242639927 >> 172.21.15.33:0/4242639927 conn(0x7f7ef405ec80 msgr2=0x7f7ef40cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:30.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.219+0000 7f7efa3ff700 1 -- 172.21.15.33:0/4242639927 shutdown_connections 2023-11-06T22:16:30.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.219+0000 7f7efa3ff700 1 -- 172.21.15.33:0/4242639927 wait complete. 2023-11-06T22:16:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.220+0000 7f7efa3ff700 1 Processor -- start 2023-11-06T22:16:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.220+0000 7f7efa3ff700 1 -- start start 2023-11-06T22:16:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.221+0000 7f7efa3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc003910 0x7f7ef4144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.221+0000 7f7efa3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40c3a40 0x7f7ef40c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.221+0000 7f7efa3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef4153080 0x7f7ef41534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.221+0000 7f7efa3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4145360 con 0x7f7ef4153080 2023-11-06T22:16:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.221+0000 7f7efa3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef40c43b0 con 0x7f7edc003910 2023-11-06T22:16:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.221+0000 7f7efa3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef41558e0 con 0x7f7ef40c3a40 2023-11-06T22:16:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.221+0000 7f7ef899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef4153080 0x7f7ef41534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.221+0000 7f7ef899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef4153080 0x7f7ef41534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58192/0 (socket says 172.21.15.33:58192) 2023-11-06T22:16:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.221+0000 7f7ef899c700 1 -- 172.21.15.33:0/3670285776 learned_addr learned my addr 172.21.15.33:0/3670285776 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7ef3fff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc003910 0x7f7ef4144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7ef37fe700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40c3a40 0x7f7ef40c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7ef899c700 1 -- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40c3a40 msgr2=0x7f7ef40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7ef899c700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40c3a40 0x7f7ef40c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7ef899c700 1 -- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc003910 msgr2=0x7f7ef4144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7ef899c700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc003910 0x7f7ef4144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7ef899c700 1 -- 172.21.15.33:0/3670285776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef41539d0 con 0x7f7ef4153080 2023-11-06T22:16:30.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7ef3fff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc003910 0x7f7ef4144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:30.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7ef37fe700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40c3a40 0x7f7ef40c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:30.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7ef899c700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef4153080 0x7f7ef41534e0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f7eec000f80 tx=0x7f7eec001340).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:30.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7edbfff700 1 -- 172.21.15.33:0/3670285776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00f970 con 0x7f7ef4153080 2023-11-06T22:16:30.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7edbfff700 1 -- 172.21.15.33:0/3670285776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec00fad0 con 0x7f7ef4153080 2023-11-06T22:16:30.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7edbfff700 1 -- 172.21.15.33:0/3670285776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00fe70 con 0x7f7ef4153080 2023-11-06T22:16:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7efa3ff700 1 -- 172.21.15.33:0/3670285776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef4145800 con 0x7f7ef4153080 2023-11-06T22:16:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.222+0000 7f7efa3ff700 1 -- 172.21.15.33:0/3670285776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef40c48d0 con 0x7f7ef4153080 2023-11-06T22:16:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.223+0000 7f7efa3ff700 1 -- 172.21.15.33:0/3670285776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef40c4a60 con 0x7f7ef4153080 2023-11-06T22:16:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.224+0000 7f7edbfff700 1 -- 172.21.15.33:0/3670285776 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7eec012560 con 0x7f7ef4153080 2023-11-06T22:16:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.225+0000 7f7edbfff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7ee005f270 0x7f7ee0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.225+0000 7f7edbfff700 1 -- 172.21.15.33:0/3670285776 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7eec079ff0 con 0x7f7ef4153080 2023-11-06T22:16:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.225+0000 7f7ef3fff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7ee005f270 0x7f7ee0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.226+0000 7f7ef3fff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7ee005f270 0x7f7ee0061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f7ee4006be0 tx=0x7f7ee401a000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.229+0000 7f7edbfff700 1 -- 172.21.15.33:0/3670285776 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7eec04b130 con 0x7f7ef4153080 2023-11-06T22:16:30.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.407+0000 7f7efa3ff700 1 -- 172.21.15.33:0/3670285776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7ef406c6c0 con 0x7f7ef4153080 2023-11-06T22:16:30.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.411+0000 7f7edbfff700 1 -- 172.21.15.33:0/3670285776 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7eec019030 con 0x7f7ef4153080 2023-11-06T22:16:30.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 -- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7ee005f270 msgr2=0x7f7ee0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7ee005f270 0x7f7ee0061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f7ee4006be0 tx=0x7f7ee401a000).stop 2023-11-06T22:16:30.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 -- 172.21.15.33:0/3670285776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef4153080 msgr2=0x7f7ef41534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef4153080 0x7f7ef41534e0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f7eec000f80 tx=0x7f7eec001340).stop 2023-11-06T22:16:30.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 -- 172.21.15.33:0/3670285776 shutdown_connections 2023-11-06T22:16:30.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7ee005f270 0x7f7ee0061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef4153080 0x7f7ef41534e0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc003910 0x7f7ef4144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 --2- 172.21.15.33:0/3670285776 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7ef40c3a40 0x7f7ef40c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 -- 172.21.15.33:0/3670285776 >> 172.21.15.33:0/3670285776 conn(0x7f7ef405ec80 msgr2=0x7f7ef4151140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:30.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 -- 172.21.15.33:0/3670285776 shutdown_connections 2023-11-06T22:16:30.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.413+0000 7f7efa3ff700 1 -- 172.21.15.33:0/3670285776 wait complete. 2023-11-06T22:16:30.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-11-06T22:16:30.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:16:30.593 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:16:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.640+0000 7f3898dff700 1 Processor -- start 2023-11-06T22:16:30.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.640+0000 7f3898dff700 1 -- start start 2023-11-06T22:16:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.641+0000 7f3898dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940c5d50 0x7f38940c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.641+0000 7f3898dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38941554a0 con 0x7f38940bfd70 2023-11-06T22:16:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.641+0000 7f3898dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38940c66c0 con 0x7f38940c1840 2023-11-06T22:16:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.641+0000 7f3898dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3894154960 con 0x7f38940c5d50 2023-11-06T22:16:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.642+0000 7f3892d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f38940bfd70 0x7f38940c01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44722/0 (socket says 172.21.15.33:44722) 2023-11-06T22:16:30.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.642+0000 7f3892d9d700 1 -- 172.21.15.33:0/3299028422 learned_addr learned my addr 172.21.15.33:0/3299028422 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:30.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.642+0000 7f3891d9b700 1 --2- 172.21.15.33:0/3299028422 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940c5d50 0x7f38940c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.642+0000 7f389159a700 1 -- 172.21.15.33:0/3299028422 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757496207 0 0) 0x7f38941554a0 con 0x7f38940bfd70 2023-11-06T22:16:30.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.642+0000 7f389159a700 1 -- 172.21.15.33:0/3299028422 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f387c0033f0 con 0x7f38940bfd70 2023-11-06T22:16:30.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.642+0000 7f389159a700 1 -- 172.21.15.33:0/3299028422 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592460785 0 0) 0x7f38940c66c0 con 0x7f38940c1840 2023-11-06T22:16:30.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.642+0000 7f389159a700 1 -- 172.21.15.33:0/3299028422 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38941554a0 con 0x7f38940c1840 2023-11-06T22:16:30.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.642+0000 7f3891d9b700 1 -- 172.21.15.33:0/3299028422 >> v1:172.21.15.69:6789/0 conn(0x7f38940c1840 legacy=0x7f38940a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.642+0000 7f3891d9b700 1 -- 172.21.15.33:0/3299028422 >> v1:172.21.15.33:6789/0 conn(0x7f38940bfd70 legacy=0x7f38940c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.642+0000 7f3891d9b700 1 -- 172.21.15.33:0/3299028422 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3894155930 con 0x7f38940c5d50 2023-11-06T22:16:30.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.643+0000 7f3891d9b700 1 --2- 172.21.15.33:0/3299028422 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940c5d50 0x7f38940c61d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f3888009790 tx=0x7f3888009060).ready entity=mon.2 client_cookie=c89eeb9fd08618e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:30.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.643+0000 7f389159a700 1 -- 172.21.15.33:0/3299028422 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388800f510 con 0x7f38940c5d50 2023-11-06T22:16:30.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.643+0000 7f389159a700 1 -- 172.21.15.33:0/3299028422 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388800f670 con 0x7f38940c5d50 2023-11-06T22:16:30.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.643+0000 7f389159a700 1 -- 172.21.15.33:0/3299028422 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388800f9f0 con 0x7f38940c5d50 2023-11-06T22:16:30.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.644+0000 7f3898dff700 1 -- 172.21.15.33:0/3299028422 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940c5d50 msgr2=0x7f38940c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.644+0000 7f3898dff700 1 --2- 172.21.15.33:0/3299028422 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940c5d50 0x7f38940c61d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f3888009790 tx=0x7f3888009060).stop 2023-11-06T22:16:30.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.644+0000 7f3898dff700 1 -- 172.21.15.33:0/3299028422 shutdown_connections 2023-11-06T22:16:30.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.644+0000 7f3898dff700 1 --2- 172.21.15.33:0/3299028422 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940c5d50 0x7f38940c61d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.644+0000 7f3898dff700 1 -- 172.21.15.33:0/3299028422 >> 172.21.15.33:0/3299028422 conn(0x7f389405ec80 msgr2=0x7f38940cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:30.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.644+0000 7f3898dff700 1 -- 172.21.15.33:0/3299028422 shutdown_connections 2023-11-06T22:16:30.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.644+0000 7f3898dff700 1 -- 172.21.15.33:0/3299028422 wait complete. 2023-11-06T22:16:30.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.646+0000 7f3898dff700 1 Processor -- start 2023-11-06T22:16:30.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.646+0000 7f3898dff700 1 -- start start 2023-11-06T22:16:30.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.647+0000 7f3898dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940bfd70 0x7f3894149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.647+0000 7f3898dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38940c1840 0x7f38940c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.647+0000 7f3898dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38940c5d50 0x7f3894150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.647+0000 7f3898dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389414f470 con 0x7f38940c1840 2023-11-06T22:16:30.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.647+0000 7f3898dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389414a2f0 con 0x7f38940c5d50 2023-11-06T22:16:30.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.647+0000 7f3898dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389414e860 con 0x7f38940bfd70 2023-11-06T22:16:30.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3891d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38940c1840 0x7f38940c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3891d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38940c1840 0x7f38940c28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58194/0 (socket says 172.21.15.33:58194) 2023-11-06T22:16:30.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3891d9b700 1 -- 172.21.15.33:0/3592156848 learned_addr learned my addr 172.21.15.33:0/3592156848 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:30.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f389259c700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940bfd70 0x7f3894149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3892d9d700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38940c5d50 0x7f3894150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3891d9b700 1 -- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940bfd70 msgr2=0x7f3894149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3891d9b700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940bfd70 0x7f3894149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3891d9b700 1 -- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38940c5d50 msgr2=0x7f3894150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:30.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3891d9b700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38940c5d50 0x7f3894150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:30.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3891d9b700 1 -- 172.21.15.33:0/3592156848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38880092a0 con 0x7f38940c1840 2023-11-06T22:16:30.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3892d9d700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38940c5d50 0x7f3894150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:30.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.648+0000 7f3891d9b700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38940c1840 0x7f38940c28c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f389406bc90 tx=0x7f3888017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:30.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.649+0000 7f38827fc700 1 -- 172.21.15.33:0/3592156848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388800f7e0 con 0x7f38940c1840 2023-11-06T22:16:30.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.649+0000 7f38827fc700 1 -- 172.21.15.33:0/3592156848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388800f940 con 0x7f38940c1840 2023-11-06T22:16:30.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.649+0000 7f3898dff700 1 -- 172.21.15.33:0/3592156848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389414eb40 con 0x7f38940c1840 2023-11-06T22:16:30.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.649+0000 7f38827fc700 1 -- 172.21.15.33:0/3592156848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388801c650 con 0x7f38940c1840 2023-11-06T22:16:30.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.649+0000 7f3898dff700 1 -- 172.21.15.33:0/3592156848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389414f100 con 0x7f38940c1840 2023-11-06T22:16:30.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.650+0000 7f3898dff700 1 -- 172.21.15.33:0/3592156848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389406c6c0 con 0x7f38940c1840 2023-11-06T22:16:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.651+0000 7f38827fc700 1 -- 172.21.15.33:0/3592156848 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f388801ca40 con 0x7f38940c1840 2023-11-06T22:16:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.652+0000 7f38827fc700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f387405f270 0x7f3874061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.652+0000 7f38827fc700 1 -- 172.21.15.33:0/3592156848 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f388807b940 con 0x7f38940c1840 2023-11-06T22:16:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.652+0000 7f389259c700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f387405f270 0x7f3874061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.653+0000 7f389259c700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f387405f270 0x7f3874061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f3884003930 tx=0x7f3884007bd0).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:30.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.656+0000 7f38827fc700 1 -- 172.21.15.33:0/3592156848 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f388804cb60 con 0x7f38940c1840 2023-11-06T22:16:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:30.905+0000 7f3898dff700 1 -- 172.21.15.33:0/3592156848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f389414f5b0 con 0x7f38940c1840 2023-11-06T22:16:31.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.585+0000 7f38827fc700 1 -- 172.21.15.33:0/3592156848 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2965) v1 ==== 165+0+0 (secure 0 0 0) 0x7f3888020020 con 0x7f38940c1840 2023-11-06T22:16:31.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.587+0000 7f3898dff700 1 -- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f387405f270 msgr2=0x7f3874061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:31.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.587+0000 7f3898dff700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f387405f270 0x7f3874061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f3884003930 tx=0x7f3884007bd0).stop 2023-11-06T22:16:31.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.587+0000 7f3898dff700 1 -- 172.21.15.33:0/3592156848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38940c1840 msgr2=0x7f38940c28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:31.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.587+0000 7f3898dff700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38940c1840 0x7f38940c28c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f389406bc90 tx=0x7f3888017000).stop 2023-11-06T22:16:31.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.587+0000 7f3898dff700 1 -- 172.21.15.33:0/3592156848 shutdown_connections 2023-11-06T22:16:31.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.587+0000 7f3898dff700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f387405f270 0x7f3874061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:31.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.587+0000 7f3898dff700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f38940bfd70 0x7f3894149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:31.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.587+0000 7f3898dff700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38940c5d50 0x7f3894150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:31.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.587+0000 7f3898dff700 1 --2- 172.21.15.33:0/3592156848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38940c1840 0x7f38940c28c0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:31.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.587+0000 7f3898dff700 1 -- 172.21.15.33:0/3592156848 >> 172.21.15.33:0/3592156848 conn(0x7f389405ec80 msgr2=0x7f38940ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.588+0000 7f3898dff700 1 -- 172.21.15.33:0/3592156848 shutdown_connections 2023-11-06T22:16:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.588+0000 7f3898dff700 1 -- 172.21.15.33:0/3592156848 wait complete. 2023-11-06T22:16:31.610 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-11-06T22:16:31.610 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-11-06T22:16:31.611 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-06T22:16:31.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-11-06T22:16:31.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.810+0000 7faf133ff700 1 Processor -- start 2023-11-06T22:16:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.810+0000 7faf133ff700 1 -- start start 2023-11-06T22:16:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.811+0000 7faf133ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c05e1b0 0x7faf0c063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.811+0000 7faf133ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0c0c31f0 con 0x7faf0c063140 2023-11-06T22:16:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.811+0000 7faf133ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0c0c2920 con 0x7faf0c0c3680 2023-11-06T22:16:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.811+0000 7faf133ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c06a0f0 con 0x7faf0c05e1b0 2023-11-06T22:16:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf1119b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faf0c063140 0x7faf0c0c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44724/0 (socket says 172.21.15.33:44724) 2023-11-06T22:16:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf1119b700 1 -- 172.21.15.33:0/1145024193 learned_addr learned my addr 172.21.15.33:0/1145024193 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf1099a700 1 --2- 172.21.15.33:0/1145024193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c05e1b0 0x7faf0c063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf03fff700 1 -- 172.21.15.33:0/1145024193 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914248959 0 0) 0x7faf0c0c31f0 con 0x7faf0c063140 2023-11-06T22:16:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf03fff700 1 -- 172.21.15.33:0/1145024193 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faef80033f0 con 0x7faf0c063140 2023-11-06T22:16:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf03fff700 1 -- 172.21.15.33:0/1145024193 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046971007 0 0) 0x7faf0c0c2920 con 0x7faf0c0c3680 2023-11-06T22:16:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf03fff700 1 -- 172.21.15.33:0/1145024193 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf0c0c31f0 con 0x7faf0c0c3680 2023-11-06T22:16:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf03fff700 1 -- 172.21.15.33:0/1145024193 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2499065483 0 0) 0x7faef80033f0 con 0x7faf0c063140 2023-11-06T22:16:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf1099a700 1 -- 172.21.15.33:0/1145024193 >> v1:172.21.15.69:6789/0 conn(0x7faf0c0c3680 legacy=0x7faf0c154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf1099a700 1 -- 172.21.15.33:0/1145024193 >> v1:172.21.15.33:6789/0 conn(0x7faf0c063140 legacy=0x7faf0c0c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.812+0000 7faf1099a700 1 -- 172.21.15.33:0/1145024193 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c05e640 con 0x7faf0c05e1b0 2023-11-06T22:16:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.813+0000 7faf1099a700 1 --2- 172.21.15.33:0/1145024193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c05e1b0 0x7faf0c063520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7faefc009790 tx=0x7faefc009060).ready entity=mon.2 client_cookie=8a948f6d14589633 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.813+0000 7faf03fff700 1 -- 172.21.15.33:0/1145024193 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08001470 con 0x7faf0c05e1b0 2023-11-06T22:16:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.813+0000 7faf03fff700 1 -- 172.21.15.33:0/1145024193 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc011400 con 0x7faf0c05e1b0 2023-11-06T22:16:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.813+0000 7faf03fff700 1 -- 172.21.15.33:0/1145024193 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc011780 con 0x7faf0c05e1b0 2023-11-06T22:16:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.813+0000 7faf133ff700 1 -- 172.21.15.33:0/1145024193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c05e1b0 msgr2=0x7faf0c063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.813+0000 7faf133ff700 1 --2- 172.21.15.33:0/1145024193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c05e1b0 0x7faf0c063520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7faefc009790 tx=0x7faefc009060).stop 2023-11-06T22:16:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.814+0000 7faf133ff700 1 -- 172.21.15.33:0/1145024193 shutdown_connections 2023-11-06T22:16:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.814+0000 7faf133ff700 1 --2- 172.21.15.33:0/1145024193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c05e1b0 0x7faf0c063520 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.814+0000 7faf133ff700 1 -- 172.21.15.33:0/1145024193 >> 172.21.15.33:0/1145024193 conn(0x7faf0c0c7d60 msgr2=0x7faf0c0c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.814+0000 7faf133ff700 1 -- 172.21.15.33:0/1145024193 shutdown_connections 2023-11-06T22:16:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.814+0000 7faf133ff700 1 -- 172.21.15.33:0/1145024193 wait complete. 2023-11-06T22:16:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.816+0000 7faf133ff700 1 Processor -- start 2023-11-06T22:16:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.816+0000 7faf133ff700 1 -- start start 2023-11-06T22:16:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.817+0000 7faf133ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c063140 0x7faf0c13bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.817+0000 7faf133ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf0c0c3680 0x7faf0c0abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.817+0000 7faf133ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c147d40 0x7faf0c0aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.817+0000 7faf133ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0ab970 con 0x7faf0c063140 2023-11-06T22:16:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.817+0000 7faf133ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c13c050 con 0x7faf0c0c3680 2023-11-06T22:16:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.817+0000 7faf133ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0aad60 con 0x7faf0c147d40 2023-11-06T22:16:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.817+0000 7faf1119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c063140 0x7faf0c13bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.817+0000 7faf1119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c063140 0x7faf0c13bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58210/0 (socket says 172.21.15.33:58210) 2023-11-06T22:16:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.817+0000 7faf1119b700 1 -- 172.21.15.33:0/1539358560 learned_addr learned my addr 172.21.15.33:0/1539358560 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf1199c700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c147d40 0x7faf0c0aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf1119b700 1 -- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c147d40 msgr2=0x7faf0c0aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf1099a700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf0c0c3680 0x7faf0c0abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf1119b700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c147d40 0x7faf0c0aa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf1119b700 1 -- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf0c0c3680 msgr2=0x7faf0c0abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf1119b700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf0c0c3680 0x7faf0c0abe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf1119b700 1 -- 172.21.15.33:0/1539358560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc0092a0 con 0x7faf0c063140 2023-11-06T22:16:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf1199c700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c147d40 0x7faf0c0aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf1099a700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf0c0c3680 0x7faf0c0abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf1119b700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c063140 0x7faf0c13bb60 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7faf0800c7c0 tx=0x7faf08001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf017fa700 1 -- 172.21.15.33:0/1539358560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08007170 con 0x7faf0c063140 2023-11-06T22:16:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf017fa700 1 -- 172.21.15.33:0/1539358560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf080072f0 con 0x7faf0c063140 2023-11-06T22:16:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf017fa700 1 -- 172.21.15.33:0/1539358560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf080048c0 con 0x7faf0c063140 2023-11-06T22:16:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf133ff700 1 -- 172.21.15.33:0/1539358560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c0ab040 con 0x7faf0c063140 2023-11-06T22:16:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.818+0000 7faf133ff700 1 -- 172.21.15.33:0/1539358560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c0ab600 con 0x7faf0c063140 2023-11-06T22:16:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.820+0000 7faf133ff700 1 -- 172.21.15.33:0/1539358560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0c064420 con 0x7faf0c063140 2023-11-06T22:16:31.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.820+0000 7faf017fa700 1 -- 172.21.15.33:0/1539358560 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7faf08004c70 con 0x7faf0c063140 2023-11-06T22:16:31.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.822+0000 7faf017fa700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faef005f2a0 0x7faef0061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:31.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.822+0000 7faf017fa700 1 -- 172.21.15.33:0/1539358560 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6087+0+0 (secure 0 0 0) 0x7faf08079eb0 con 0x7faf0c063140 2023-11-06T22:16:31.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.822+0000 7faf1099a700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faef005f2a0 0x7faef0061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:31.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.823+0000 7faf1099a700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faef005f2a0 0x7faef0061760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7faefc009790 tx=0x7faefc016000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:31.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:31.826+0000 7faf017fa700 1 -- 172.21.15.33:0/1539358560 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf0804b0d0 con 0x7faf0c063140 2023-11-06T22:16:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.073+0000 7faf133ff700 1 -- 172.21.15.33:0/1539358560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7faf0c06bfd0 con 0x7faf0c063140 2023-11-06T22:16:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.587+0000 7faf017fa700 1 -- 172.21.15.33:0/1539358560 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2966) v1 ==== 166+0+0 (secure 0 0 0) 0x7faf08001c70 con 0x7faf0c063140 2023-11-06T22:16:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 -- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faef005f2a0 msgr2=0x7faef0061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faef005f2a0 0x7faef0061760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7faefc009790 tx=0x7faefc016000).stop 2023-11-06T22:16:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 -- 172.21.15.33:0/1539358560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c063140 msgr2=0x7faf0c13bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c063140 0x7faf0c13bb60 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7faf0800c7c0 tx=0x7faf08001b50).stop 2023-11-06T22:16:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 -- 172.21.15.33:0/1539358560 shutdown_connections 2023-11-06T22:16:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faef005f2a0 0x7faef0061760 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf0c147d40 0x7faf0c0aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf0c063140 0x7faf0c13bb60 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 --2- 172.21.15.33:0/1539358560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf0c0c3680 0x7faf0c0abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 -- 172.21.15.33:0/1539358560 >> 172.21.15.33:0/1539358560 conn(0x7faf0c0c7d60 msgr2=0x7faf0c064ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 -- 172.21.15.33:0/1539358560 shutdown_connections 2023-11-06T22:16:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.590+0000 7faf133ff700 1 -- 172.21.15.33:0/1539358560 wait complete. 2023-11-06T22:16:32.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T22:16:32.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.811+0000 7f36b75ff700 1 Processor -- start 2023-11-06T22:16:32.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.811+0000 7f36b75ff700 1 -- start start 2023-11-06T22:16:32.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.812+0000 7f36b75ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f36b01546a0 0x7f36b0064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:32.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.812+0000 7f36b75ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f36b0066560 0x7f36b0066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b75ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b00641f0 con 0x7f36b00bdbb0 2023-11-06T22:16:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b75ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b0154a80 con 0x7f36b01546a0 2023-11-06T22:16:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b75ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b00638c0 con 0x7f36b0066560 2023-11-06T22:16:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b539b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f36b01546a0 0x7f36b0064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b539b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f36b01546a0 0x7f36b0064680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58216/0 (socket says 172.21.15.33:58216) 2023-11-06T22:16:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b539b700 1 -- 172.21.15.33:0/1865582165 learned_addr learned my addr 172.21.15.33:0/1865582165 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b4b9a700 1 --2- 172.21.15.33:0/1865582165 >> v2:172.21.15.69:3300/0 conn(0x7f36b0066560 0x7f36b0066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b539b700 1 -- 172.21.15.33:0/1865582165 >> v2:172.21.15.69:3300/0 conn(0x7f36b0066560 msgr2=0x7f36b0066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b539b700 1 --2- 172.21.15.33:0/1865582165 >> v2:172.21.15.69:3300/0 conn(0x7f36b0066560 0x7f36b0066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b539b700 1 -- 172.21.15.33:0/1865582165 >> v1:172.21.15.33:6789/0 conn(0x7f36b00bdbb0 legacy=0x7f36b00c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b539b700 1 -- 172.21.15.33:0/1865582165 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b0063ba0 con 0x7f36b01546a0 2023-11-06T22:16:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36a7fff700 1 -- 172.21.15.33:0/1865582165 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774469168 0 0) 0x7f36b00641f0 con 0x7f36b00bdbb0 2023-11-06T22:16:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.813+0000 7f36b4b9a700 1 --2- 172.21.15.33:0/1865582165 >> v2:172.21.15.69:3300/0 conn(0x7f36b0066560 0x7f36b0066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36b539b700 1 --2- 172.21.15.33:0/1865582165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b01546a0 0x7f36b0064680 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f36b00c3180 tx=0x7f36ac0090f0).ready entity=mon.0 client_cookie=bf9f206dd19d8f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36a7fff700 1 -- 172.21.15.33:0/1865582165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00b600 con 0x7f36b01546a0 2023-11-06T22:16:32.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36a7fff700 1 -- 172.21.15.33:0/1865582165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac00b760 con 0x7f36b01546a0 2023-11-06T22:16:32.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36a7fff700 1 -- 172.21.15.33:0/1865582165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00bae0 con 0x7f36b01546a0 2023-11-06T22:16:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36b75ff700 1 -- 172.21.15.33:0/1865582165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b01546a0 msgr2=0x7f36b0064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36b75ff700 1 --2- 172.21.15.33:0/1865582165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b01546a0 0x7f36b0064680 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f36b00c3180 tx=0x7f36ac0090f0).stop 2023-11-06T22:16:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36b75ff700 1 -- 172.21.15.33:0/1865582165 shutdown_connections 2023-11-06T22:16:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36b75ff700 1 --2- 172.21.15.33:0/1865582165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b01546a0 0x7f36b0064680 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36b75ff700 1 --2- 172.21.15.33:0/1865582165 >> v2:172.21.15.69:3300/0 conn(0x7f36b0066560 0x7f36b0066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36b75ff700 1 -- 172.21.15.33:0/1865582165 >> 172.21.15.33:0/1865582165 conn(0x7f36b00c8c50 msgr2=0x7f36b00c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.814+0000 7f36b75ff700 1 -- 172.21.15.33:0/1865582165 shutdown_connections 2023-11-06T22:16:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.815+0000 7f36b75ff700 1 -- 172.21.15.33:0/1865582165 wait complete. 2023-11-06T22:16:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.816+0000 7f36b75ff700 1 Processor -- start 2023-11-06T22:16:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.816+0000 7f36b75ff700 1 -- start start 2023-11-06T22:16:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.817+0000 7f36b75ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f36b0066560 0x7f36b0151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.817+0000 7f36b75ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b00bdbb0 0x7f36b01673d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.817+0000 7f36b75ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b01546a0 0x7f36b0166090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.817+0000 7f36b75ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0156d70 con 0x7f36b00bdbb0 2023-11-06T22:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.817+0000 7f36b75ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b01678c0 con 0x7f36b01546a0 2023-11-06T22:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.817+0000 7f36b75ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0165640 con 0x7f36b0066560 2023-11-06T22:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.817+0000 7f36b4b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b00bdbb0 0x7f36b01673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.817+0000 7f36b4b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b00bdbb0 0x7f36b01673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58230/0 (socket says 172.21.15.33:58230) 2023-11-06T22:16:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.817+0000 7f36b4b9a700 1 -- 172.21.15.33:0/1287022763 learned_addr learned my addr 172.21.15.33:0/1287022763 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.817+0000 7f36b539b700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f36b0066560 0x7f36b0151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36b5b9c700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b01546a0 0x7f36b0166090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36b4b9a700 1 -- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f36b0066560 msgr2=0x7f36b0151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36b4b9a700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f36b0066560 0x7f36b0151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36b4b9a700 1 -- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b01546a0 msgr2=0x7f36b0166090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36b4b9a700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b01546a0 0x7f36b0166090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36b4b9a700 1 -- 172.21.15.33:0/1287022763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac009370 con 0x7f36b00bdbb0 2023-11-06T22:16:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36b5b9c700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b01546a0 0x7f36b0166090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36b4b9a700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b00bdbb0 0x7f36b01673d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f36a000d890 tx=0x7f36a0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36a57fa700 1 -- 172.21.15.33:0/1287022763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0011590 con 0x7f36b00bdbb0 2023-11-06T22:16:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36a57fa700 1 -- 172.21.15.33:0/1287022763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a00116f0 con 0x7f36b00bdbb0 2023-11-06T22:16:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36a57fa700 1 -- 172.21.15.33:0/1287022763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0011a70 con 0x7f36b00bdbb0 2023-11-06T22:16:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36b75ff700 1 -- 172.21.15.33:0/1287022763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b01518a0 con 0x7f36b00bdbb0 2023-11-06T22:16:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.818+0000 7f36b75ff700 1 -- 172.21.15.33:0/1287022763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b0151e60 con 0x7f36b00bdbb0 2023-11-06T22:16:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.819+0000 7f36b75ff700 1 -- 172.21.15.33:0/1287022763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b0165ee0 con 0x7f36b00bdbb0 2023-11-06T22:16:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.820+0000 7f36a57fa700 1 -- 172.21.15.33:0/1287022763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f36a0012420 con 0x7f36b00bdbb0 2023-11-06T22:16:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.821+0000 7f36a57fa700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f369405f270 0x7f3694061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.821+0000 7f36a57fa700 1 -- 172.21.15.33:0/1287022763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f36a0079e70 con 0x7f36b00bdbb0 2023-11-06T22:16:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.822+0000 7f36b539b700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f369405f270 0x7f3694061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.822+0000 7f36b539b700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f369405f270 0x7f3694061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f36ac009fd0 tx=0x7f36ac017000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:32.825+0000 7f36a57fa700 1 -- 172.21.15.33:0/1287022763 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f36a004aec0 con 0x7f36b00bdbb0 2023-11-06T22:16:33.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.006+0000 7f36b75ff700 1 -- 172.21.15.33:0/1287022763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f36b0066000 con 0x7f36b00bdbb0 2023-11-06T22:16:33.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.010+0000 7f36a57fa700 1 -- 172.21.15.33:0/1287022763 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7f36a004e4e0 con 0x7f36b00bdbb0 2023-11-06T22:16:33.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 -- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f369405f270 msgr2=0x7f3694061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f369405f270 0x7f3694061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f36ac009fd0 tx=0x7f36ac017000).stop 2023-11-06T22:16:33.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 -- 172.21.15.33:0/1287022763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b00bdbb0 msgr2=0x7f36b01673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b00bdbb0 0x7f36b01673d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f36a000d890 tx=0x7f36a0008040).stop 2023-11-06T22:16:33.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 -- 172.21.15.33:0/1287022763 shutdown_connections 2023-11-06T22:16:33.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f369405f270 0x7f3694061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f36b0066560 0x7f36b0151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b01546a0 0x7f36b0166090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 --2- 172.21.15.33:0/1287022763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36b00bdbb0 0x7f36b01673d0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 -- 172.21.15.33:0/1287022763 >> 172.21.15.33:0/1287022763 conn(0x7f36b00c8c50 msgr2=0x7f36b00c4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:33.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 -- 172.21.15.33:0/1287022763 shutdown_connections 2023-11-06T22:16:33.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.012+0000 7f36b75ff700 1 -- 172.21.15.33:0/1287022763 wait complete. 2023-11-06T22:16:33.031 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T22:16:33.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:16:33.122 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:16:33.220 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T22:16:33.220 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T22:16:33.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:16:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.417+0000 7f1f023ff700 1 Processor -- start 2023-11-06T22:16:33.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.417+0000 7f1f023ff700 1 -- start start 2023-11-06T22:16:33.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.418+0000 7f1f023ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1efc0be280 0x7f1efc15f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.418+0000 7f1f023ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc1553b0 con 0x7f1efc0c5240 2023-11-06T22:16:33.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.418+0000 7f1f023ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc0c45c0 con 0x7f1efc0c4e60 2023-11-06T22:16:33.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.418+0000 7f1f023ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc066c50 con 0x7f1efc0be280 2023-11-06T22:16:33.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.419+0000 7f1efb7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1efc0c5240 0x7f1efc0bdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44750/0 (socket says 172.21.15.33:44750) 2023-11-06T22:16:33.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.419+0000 7f1efb7fe700 1 -- 172.21.15.33:0/326581175 learned_addr learned my addr 172.21.15.33:0/326581175 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.419+0000 7f1f0099c700 1 --2- 172.21.15.33:0/326581175 >> v2:172.21.15.69:3300/0 conn(0x7f1efc0be280 0x7f1efc15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.419+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102082103 0 0) 0x7f1efc1553b0 con 0x7f1efc0c5240 2023-11-06T22:16:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.419+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ee40033f0 con 0x7f1efc0c5240 2023-11-06T22:16:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3556359590 0 0) 0x7f1ee40033f0 con 0x7f1efc0c5240 2023-11-06T22:16:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 >> v2:172.21.15.69:3300/0 conn(0x7f1efc0be280 msgr2=0x7f1efc15f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 --2- 172.21.15.33:0/326581175 >> v2:172.21.15.69:3300/0 conn(0x7f1efc0be280 0x7f1efc15f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 >> v1:172.21.15.69:6789/0 conn(0x7f1efc0c4e60 legacy=0x7f1efc066540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc154ac0 con 0x7f1efc0c5240 2023-11-06T22:16:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f1ef0001470 con 0x7f1efc0c5240 2023-11-06T22:16:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 >> v1:172.21.15.33:6789/0 conn(0x7f1efc0c5240 legacy=0x7f1efc0bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4003910 0x7f1ee4003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4004690 0x7f1ee4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ee4005000 0x7f1ee400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc066c50 con 0x7f1ee4003910 2023-11-06T22:16:33.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee400beb0 con 0x7f1ee4004690 2023-11-06T22:16:33.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee400c070 con 0x7f1ee4005000 2023-11-06T22:16:33.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efbfff700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4003910 0x7f1ee4003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efb7fe700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4004690 0x7f1ee4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1f0099c700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ee4005000 0x7f1ee400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efb7fe700 1 -- 172.21.15.33:0/326581175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ee4005000 msgr2=0x7f1ee400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:33.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efb7fe700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1ee4005000 0x7f1ee400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efb7fe700 1 -- 172.21.15.33:0/326581175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4003910 msgr2=0x7f1ee4003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:33.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.420+0000 7f1efb7fe700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4003910 0x7f1ee4003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.421+0000 7f1efb7fe700 1 -- 172.21.15.33:0/326581175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40033f0 con 0x7f1ee4004690 2023-11-06T22:16:33.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.421+0000 7f1f0099c700 1 -- 172.21.15.33:0/326581175 reap_dead start 2023-11-06T22:16:33.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.421+0000 7f1efb7fe700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4004690 0x7f1ee4004b10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f1efc0c5680 tx=0x7f1efc154c50).ready entity=mon.1 client_cookie=7c3a58bbc447a204 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:33.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.421+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000ca60 con 0x7f1ee4004690 2023-11-06T22:16:33.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.421+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef000cbc0 con 0x7f1ee4004690 2023-11-06T22:16:33.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.421+0000 7f1efaffd700 1 -- 172.21.15.33:0/326581175 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00070d0 con 0x7f1ee4004690 2023-11-06T22:16:33.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.421+0000 7f1f023ff700 1 -- 172.21.15.33:0/326581175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4004690 msgr2=0x7f1ee4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.421+0000 7f1f023ff700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4004690 0x7f1ee4004b10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f1efc0c5680 tx=0x7f1efc154c50).stop 2023-11-06T22:16:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.422+0000 7f1f023ff700 1 -- 172.21.15.33:0/326581175 shutdown_connections 2023-11-06T22:16:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.422+0000 7f1f023ff700 1 --2- 172.21.15.33:0/326581175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4004690 0x7f1ee4004b10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.422+0000 7f1f023ff700 1 -- 172.21.15.33:0/326581175 >> 172.21.15.33:0/326581175 conn(0x7f1efc0c8c10 msgr2=0x7f1efc0c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:33.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.422+0000 7f1f023ff700 1 -- 172.21.15.33:0/326581175 shutdown_connections 2023-11-06T22:16:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.422+0000 7f1f023ff700 1 -- 172.21.15.33:0/326581175 wait complete. 2023-11-06T22:16:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.423+0000 7f1f023ff700 1 Processor -- start 2023-11-06T22:16:33.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.423+0000 7f1f023ff700 1 -- start start 2023-11-06T22:16:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.424+0000 7f1f023ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4004690 0x7f1efc151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.424+0000 7f1f023ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc0a52b0 0x7f1efc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.424+0000 7f1f023ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1efc157150 0x7f1efc1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.424+0000 7f1f023ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0a5c20 con 0x7f1ee4004690 2023-11-06T22:16:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.424+0000 7f1f023ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0ac820 con 0x7f1efc0a52b0 2023-11-06T22:16:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.424+0000 7f1f023ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc157ac0 con 0x7f1efc157150 2023-11-06T22:16:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4004690 0x7f1efc151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4004690 0x7f1efc151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58256/0 (socket says 172.21.15.33:58256) 2023-11-06T22:16:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efbfff700 1 -- 172.21.15.33:0/960992251 learned_addr learned my addr 172.21.15.33:0/960992251 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1f0099c700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1efc157150 0x7f1efc1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efb7fe700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc0a52b0 0x7f1efc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efbfff700 1 -- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1efc157150 msgr2=0x7f1efc1575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efbfff700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1efc157150 0x7f1efc1575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efbfff700 1 -- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc0a52b0 msgr2=0x7f1efc0a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efbfff700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc0a52b0 0x7f1efc0a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efbfff700 1 -- 172.21.15.33:0/960992251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc0a6830 con 0x7f1ee4004690 2023-11-06T22:16:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1f0099c700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1efc157150 0x7f1efc1575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:33.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efb7fe700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc0a52b0 0x7f1efc0a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:33.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1efbfff700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4004690 0x7f1efc151c70 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f1eec0026a0 tx=0x7f1eec007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:33.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1ee3fff700 1 -- 172.21.15.33:0/960992251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec007240 con 0x7f1ee4004690 2023-11-06T22:16:33.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1ee3fff700 1 -- 172.21.15.33:0/960992251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec0073a0 con 0x7f1ee4004690 2023-11-06T22:16:33.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1ee3fff700 1 -- 172.21.15.33:0/960992251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0066a0 con 0x7f1ee4004690 2023-11-06T22:16:33.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.425+0000 7f1f023ff700 1 -- 172.21.15.33:0/960992251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc0a5e60 con 0x7f1ee4004690 2023-11-06T22:16:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.426+0000 7f1f023ff700 1 -- 172.21.15.33:0/960992251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc0ac4b0 con 0x7f1ee4004690 2023-11-06T22:16:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.427+0000 7f1f023ff700 1 -- 172.21.15.33:0/960992251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc0acb00 con 0x7f1ee4004690 2023-11-06T22:16:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.428+0000 7f1ee3fff700 1 -- 172.21.15.33:0/960992251 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1eec006a50 con 0x7f1ee4004690 2023-11-06T22:16:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.428+0000 7f1ee3fff700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1ee805f270 0x7f1ee8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.428+0000 7f1ee3fff700 1 -- 172.21.15.33:0/960992251 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f1eec07a630 con 0x7f1ee4004690 2023-11-06T22:16:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.429+0000 7f1efb7fe700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1ee805f270 0x7f1ee8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.429+0000 7f1efb7fe700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1ee805f270 0x7f1ee8061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1efc167bd0 tx=0x7f1ef0015000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.433+0000 7f1ee3fff700 1 -- 172.21.15.33:0/960992251 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1eec04b6e0 con 0x7f1ee4004690 2023-11-06T22:16:33.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.662+0000 7f1f023ff700 1 -- 172.21.15.33:0/960992251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1efc154d60 con 0x7f1ee4004690 2023-11-06T22:16:33.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.663+0000 7f1ee3fff700 1 -- 172.21.15.33:0/960992251 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f1eec04ed00 con 0x7f1ee4004690 2023-11-06T22:16:33.665 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:33.665 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":78,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3392183580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3392183580},{"type":"v1","addr":"172.21.15.33:6835","nonce":3392183580}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2124998325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":2124998325},{"type":"v1","addr":"172.21.15.69:6837","nonce":2124998325}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3826680893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3826680893},{"type":"v1","addr":"172.21.15.33:6834","nonce":3826680893}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8980,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/2782074947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2782074947},{"type":"v1","addr":"172.21.15.69:6836","nonce":2782074947}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[]} 2023-11-06T22:16:33.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 -- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1ee805f270 msgr2=0x7f1ee8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1ee805f270 0x7f1ee8061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1efc167bd0 tx=0x7f1ef0015000).stop 2023-11-06T22:16:33.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 -- 172.21.15.33:0/960992251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4004690 msgr2=0x7f1efc151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4004690 0x7f1efc151c70 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f1eec0026a0 tx=0x7f1eec007bb0).stop 2023-11-06T22:16:33.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 -- 172.21.15.33:0/960992251 shutdown_connections 2023-11-06T22:16:33.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1ee805f270 0x7f1ee8061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1efc157150 0x7f1efc1575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4004690 0x7f1efc151c70 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 --2- 172.21.15.33:0/960992251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc0a52b0 0x7f1efc0a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 -- 172.21.15.33:0/960992251 >> 172.21.15.33:0/960992251 conn(0x7f1efc0c8c10 msgr2=0x7f1efc0c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:33.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 -- 172.21.15.33:0/960992251 shutdown_connections 2023-11-06T22:16:33.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.665+0000 7f1f023ff700 1 -- 172.21.15.33:0/960992251 wait complete. 2023-11-06T22:16:33.670 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 78 2023-11-06T22:16:33.684 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T22:16:33.685 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T22:16:33.685 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T22:16:33.685 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:16:33.686 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T22:16:33.686 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T22:16:33.687 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T22:16:33.687 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:16:33.687 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T22:16:33.687 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T22:16:33.688 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T22:16:33.688 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:16:33.688 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T22:16:33.689 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T22:16:33.689 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T22:16:33.689 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:16:33.699 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T22:16:33.700 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T22:16:33.701 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T22:16:33.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T22:16:33.705 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T22:16:33.705 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T22:16:33.706 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T22:16:33.706 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T22:16:33.745 INFO:tasks.ceph.mds.a:Started 2023-11-06T22:16:33.746 DEBUG:teuthology.parallel:result is None 2023-11-06T22:16:33.749 INFO:tasks.ceph.mds.c:Started 2023-11-06T22:16:33.749 DEBUG:teuthology.parallel:result is None 2023-11-06T22:16:33.752 INFO:tasks.ceph.mds.d:Started 2023-11-06T22:16:33.753 DEBUG:teuthology.parallel:result is None 2023-11-06T22:16:33.755 INFO:tasks.ceph.mds.b:Started 2023-11-06T22:16:33.755 DEBUG:teuthology.parallel:result is None 2023-11-06T22:16:33.756 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T22:16:33.896 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T22:16:33.897 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T22:16:33.898 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T22:16:33.901 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T22:16:33.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.972+0000 7f7f473ff700 1 Processor -- start 2023-11-06T22:16:33.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.973+0000 7f7f473ff700 1 -- start start 2023-11-06T22:16:33.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f473ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7f401373a0 0x7f7f40137780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f473ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c1020 0x7f7f400c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f473ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f400c3770 0x7f7f40069440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f473ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4006a470 con 0x7f7f400c3770 2023-11-06T22:16:33.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f473ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4006a190 con 0x7f7f401373a0 2023-11-06T22:16:33.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f473ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f400bbab0 con 0x7f7f400c1020 2023-11-06T22:16:33.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f4599c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f400c3770 0x7f7f40069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f4599c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f400c3770 0x7f7f40069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58308/0 (socket says 172.21.15.33:58308) 2023-11-06T22:16:33.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f4599c700 1 -- 172.21.15.33:0/255107291 learned_addr learned my addr 172.21.15.33:0/255107291 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:33.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f4519b700 1 --2- 172.21.15.33:0/255107291 >> v2:172.21.15.69:3300/0 conn(0x7f7f401373a0 0x7f7f40137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.974+0000 7f7f4499a700 1 --2- 172.21.15.33:0/255107291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c1020 0x7f7f400c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.975+0000 7f7f4599c700 1 -- 172.21.15.33:0/255107291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c1020 msgr2=0x7f7f400c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.975+0000 7f7f4599c700 1 --2- 172.21.15.33:0/255107291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c1020 0x7f7f400c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.975+0000 7f7f4599c700 1 -- 172.21.15.33:0/255107291 >> v2:172.21.15.69:3300/0 conn(0x7f7f401373a0 msgr2=0x7f7f40137780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.975+0000 7f7f4599c700 1 --2- 172.21.15.33:0/255107291 >> v2:172.21.15.69:3300/0 conn(0x7f7f401373a0 0x7f7f40137780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.975+0000 7f7f4599c700 1 -- 172.21.15.33:0/255107291 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f400bbd90 con 0x7f7f400c3770 2023-11-06T22:16:33.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.975+0000 7f7f4499a700 1 --2- 172.21.15.33:0/255107291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c1020 0x7f7f400c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:33.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.975+0000 7f7f4599c700 1 --2- 172.21.15.33:0/255107291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c3770 0x7f7f40069440 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f7f400c6930 tx=0x7f7f38009840).ready entity=mon.0 client_cookie=312e9a28905bfef4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:33.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.975+0000 7f7f37fff700 1 -- 172.21.15.33:0/255107291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800e070 con 0x7f7f400c3770 2023-11-06T22:16:33.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.975+0000 7f7f37fff700 1 -- 172.21.15.33:0/255107291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3800ba20 con 0x7f7f400c3770 2023-11-06T22:16:33.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.975+0000 7f7f37fff700 1 -- 172.21.15.33:0/255107291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800bda0 con 0x7f7f400c3770 2023-11-06T22:16:33.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.976+0000 7f7f473ff700 1 -- 172.21.15.33:0/255107291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c3770 msgr2=0x7f7f40069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.976+0000 7f7f473ff700 1 --2- 172.21.15.33:0/255107291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c3770 0x7f7f40069440 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f7f400c6930 tx=0x7f7f38009840).stop 2023-11-06T22:16:33.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.976+0000 7f7f473ff700 1 -- 172.21.15.33:0/255107291 shutdown_connections 2023-11-06T22:16:33.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.976+0000 7f7f473ff700 1 --2- 172.21.15.33:0/255107291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c3770 0x7f7f40069440 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.976+0000 7f7f473ff700 1 --2- 172.21.15.33:0/255107291 >> v2:172.21.15.69:3300/0 conn(0x7f7f401373a0 0x7f7f40137780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.976+0000 7f7f473ff700 1 --2- 172.21.15.33:0/255107291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c1020 0x7f7f400c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.976+0000 7f7f473ff700 1 -- 172.21.15.33:0/255107291 >> 172.21.15.33:0/255107291 conn(0x7f7f40062710 msgr2=0x7f7f40062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:33.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.976+0000 7f7f473ff700 1 -- 172.21.15.33:0/255107291 shutdown_connections 2023-11-06T22:16:33.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.976+0000 7f7f473ff700 1 -- 172.21.15.33:0/255107291 wait complete. 2023-11-06T22:16:33.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.977+0000 7f7f473ff700 1 Processor -- start 2023-11-06T22:16:33.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.978+0000 7f7f473ff700 1 -- start start 2023-11-06T22:16:33.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.978+0000 7f7f473ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c1020 0x7f7f4015b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f473ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c3770 0x7f7f400bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f473ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f401373a0 0x7f7f4014dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f473ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f400be350 con 0x7f7f400c1020 2023-11-06T22:16:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f473ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f400be630 con 0x7f7f401373a0 2023-11-06T22:16:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f473ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f400be910 con 0x7f7f400c3770 2023-11-06T22:16:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4519b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c1020 0x7f7f4015b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4519b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c1020 0x7f7f4015b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58314/0 (socket says 172.21.15.33:58314) 2023-11-06T22:16:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4519b700 1 -- 172.21.15.33:0/4026246664 learned_addr learned my addr 172.21.15.33:0/4026246664 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4499a700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c3770 0x7f7f400bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4599c700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f401373a0 0x7f7f4014dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4519b700 1 -- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c3770 msgr2=0x7f7f400bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4519b700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c3770 0x7f7f400bdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4519b700 1 -- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f401373a0 msgr2=0x7f7f4014dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:33.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4519b700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f401373a0 0x7f7f4014dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:33.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4519b700 1 -- 172.21.15.33:0/4026246664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38009ae0 con 0x7f7f400c1020 2023-11-06T22:16:33.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.979+0000 7f7f4519b700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c1020 0x7f7f4015b390 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f7f3c00e930 tx=0x7f7f3c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:33.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.980+0000 7f7f357fa700 1 -- 172.21.15.33:0/4026246664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c012600 con 0x7f7f400c1020 2023-11-06T22:16:33.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.980+0000 7f7f357fa700 1 -- 172.21.15.33:0/4026246664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c012760 con 0x7f7f400c1020 2023-11-06T22:16:33.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.980+0000 7f7f473ff700 1 -- 172.21.15.33:0/4026246664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f400bebf0 con 0x7f7f400c1020 2023-11-06T22:16:33.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.980+0000 7f7f357fa700 1 -- 172.21.15.33:0/4026246664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c012ae0 con 0x7f7f400c1020 2023-11-06T22:16:33.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.980+0000 7f7f473ff700 1 -- 172.21.15.33:0/4026246664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f400bdfb0 con 0x7f7f400c1020 2023-11-06T22:16:33.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.981+0000 7f7f473ff700 1 -- 172.21.15.33:0/4026246664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f400a9bd0 con 0x7f7f400c1020 2023-11-06T22:16:33.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.982+0000 7f7f357fa700 1 -- 172.21.15.33:0/4026246664 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7f3c01b400 con 0x7f7f400c1020 2023-11-06T22:16:33.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.983+0000 7f7f357fa700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7f2405f270 0x7f7f24061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:33.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.983+0000 7f7f357fa700 1 -- 172.21.15.33:0/4026246664 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7f3c07a850 con 0x7f7f400c1020 2023-11-06T22:16:33.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.983+0000 7f7f4499a700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7f2405f270 0x7f7f24061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:33.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.984+0000 7f7f4499a700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7f2405f270 0x7f7f24061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f7f4015b880 tx=0x7f7f30008040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:33.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:33.987+0000 7f7f357fa700 1 -- 172.21.15.33:0/4026246664 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7f3c04b8a0 con 0x7f7f400c1020 2023-11-06T22:16:34.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.185+0000 7f7f473ff700 1 -- 172.21.15.33:0/4026246664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7f400a9d80 con 0x7f7f400c1020 2023-11-06T22:16:34.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.935+0000 7f7f357fa700 1 -- 172.21.15.33:0/4026246664 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v200) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7f3c04eec0 con 0x7f7f400c1020 2023-11-06T22:16:34.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 -- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7f2405f270 msgr2=0x7f7f24061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:34.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7f2405f270 0x7f7f24061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f7f4015b880 tx=0x7f7f30008040).stop 2023-11-06T22:16:34.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 -- 172.21.15.33:0/4026246664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c1020 msgr2=0x7f7f4015b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:34.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c1020 0x7f7f4015b390 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f7f3c00e930 tx=0x7f7f3c009060).stop 2023-11-06T22:16:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 -- 172.21.15.33:0/4026246664 shutdown_connections 2023-11-06T22:16:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7f2405f270 0x7f7f24061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f401373a0 0x7f7f4014dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f400c1020 0x7f7f4015b390 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 --2- 172.21.15.33:0/4026246664 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7f400c3770 0x7f7f400bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 -- 172.21.15.33:0/4026246664 >> 172.21.15.33:0/4026246664 conn(0x7f7f40062710 msgr2=0x7f7f40064720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 -- 172.21.15.33:0/4026246664 shutdown_connections 2023-11-06T22:16:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:34.938+0000 7f7f473ff700 1 -- 172.21.15.33:0/4026246664 wait complete. 2023-11-06T22:16:34.990 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T22:16:34.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T22:16:35.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.191+0000 7f86a05ff700 1 Processor -- start 2023-11-06T22:16:35.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.191+0000 7f86a05ff700 1 -- start start 2023-11-06T22:16:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.192+0000 7f86a05ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c5d50 0x7f86980c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.193+0000 7f86a05ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f86980bfd70 0x7f86980c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.193+0000 7f86a05ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86981554a0 con 0x7f86980c1840 2023-11-06T22:16:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.193+0000 7f86a05ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8698154960 con 0x7f86980bfd70 2023-11-06T22:16:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.193+0000 7f86a05ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86980c66c0 con 0x7f86980c5d50 2023-11-06T22:16:35.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.193+0000 7f869db9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c5d50 0x7f86980c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.193+0000 7f869eb9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f86980bfd70 0x7f86980c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.193+0000 7f869e39b700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f86980c1840 0x7f86980a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:37966/0 (socket says 172.21.15.33:37966) 2023-11-06T22:16:35.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.193+0000 7f869db9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c5d50 0x7f86980c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:43850/0 (socket says 172.21.15.33:43850) 2023-11-06T22:16:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.193+0000 7f869eb9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f86980bfd70 0x7f86980c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:40802/0 (socket says 172.21.15.33:40802) 2023-11-06T22:16:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.193+0000 7f869e39b700 1 -- 172.21.15.33:0/1571654615 learned_addr learned my addr 172.21.15.33:0/1571654615 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.194+0000 7f869db9a700 1 -- 172.21.15.33:0/1571654615 >> v2:172.21.15.69:3300/0 conn(0x7f86980bfd70 msgr2=0x7f86980c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.194+0000 7f869db9a700 1 --2- 172.21.15.33:0/1571654615 >> v2:172.21.15.69:3300/0 conn(0x7f86980bfd70 0x7f86980c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.194+0000 7f869d399700 1 -- 172.21.15.33:0/1571654615 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89018866 0 0) 0x7f86981554a0 con 0x7f86980c1840 2023-11-06T22:16:35.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.194+0000 7f869db9a700 1 -- 172.21.15.33:0/1571654615 >> v1:172.21.15.69:6789/0 conn(0x7f86980c1840 legacy=0x7f86980a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.194+0000 7f869db9a700 1 -- 172.21.15.33:0/1571654615 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8698155930 con 0x7f86980c5d50 2023-11-06T22:16:35.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.194+0000 7f869eb9c700 1 --2- 172.21.15.33:0/1571654615 >> v2:172.21.15.69:3300/0 conn(0x7f86980bfd70 0x7f86980c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:35.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.194+0000 7f869db9a700 1 --2- 172.21.15.33:0/1571654615 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c5d50 0x7f86980c61d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f8688009760 tx=0x7f8688009060).ready entity=mon.2 client_cookie=cadc8021795253c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:35.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.194+0000 7f869d399700 1 -- 172.21.15.33:0/1571654615 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800f510 con 0x7f86980c5d50 2023-11-06T22:16:35.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.194+0000 7f869d399700 1 -- 172.21.15.33:0/1571654615 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868800f670 con 0x7f86980c5d50 2023-11-06T22:16:35.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.194+0000 7f869d399700 1 -- 172.21.15.33:0/1571654615 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800f9f0 con 0x7f86980c5d50 2023-11-06T22:16:35.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.195+0000 7f86a05ff700 1 -- 172.21.15.33:0/1571654615 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c5d50 msgr2=0x7f86980c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.195+0000 7f86a05ff700 1 --2- 172.21.15.33:0/1571654615 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c5d50 0x7f86980c61d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f8688009760 tx=0x7f8688009060).stop 2023-11-06T22:16:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.195+0000 7f86a05ff700 1 -- 172.21.15.33:0/1571654615 shutdown_connections 2023-11-06T22:16:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.195+0000 7f86a05ff700 1 --2- 172.21.15.33:0/1571654615 >> v2:172.21.15.69:3300/0 conn(0x7f86980bfd70 0x7f86980c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.195+0000 7f86a05ff700 1 --2- 172.21.15.33:0/1571654615 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c5d50 0x7f86980c61d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.195+0000 7f86a05ff700 1 -- 172.21.15.33:0/1571654615 >> 172.21.15.33:0/1571654615 conn(0x7f869805ec80 msgr2=0x7f86980cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:35.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.195+0000 7f86a05ff700 1 -- 172.21.15.33:0/1571654615 shutdown_connections 2023-11-06T22:16:35.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.195+0000 7f86a05ff700 1 -- 172.21.15.33:0/1571654615 wait complete. 2023-11-06T22:16:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.196+0000 7f86a05ff700 1 Processor -- start 2023-11-06T22:16:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.197+0000 7f86a05ff700 1 -- start start 2023-11-06T22:16:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.197+0000 7f86a05ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86980bfd70 0x7f8698149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.197+0000 7f86a05ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c1840 0x7f8698164df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f86a05ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86980c5d50 0x7f869814d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f86a05ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869814ede0 con 0x7f86980bfd70 2023-11-06T22:16:35.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f86a05ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86981652e0 con 0x7f86980c5d50 2023-11-06T22:16:35.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f86a05ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869814e1d0 con 0x7f86980c1840 2023-11-06T22:16:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86980bfd70 0x7f8698149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86980bfd70 0x7f8698149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58322/0 (socket says 172.21.15.33:58322) 2023-11-06T22:16:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869e39b700 1 -- 172.21.15.33:0/3177352246 learned_addr learned my addr 172.21.15.33:0/3177352246 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869e39b700 1 -- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c1840 msgr2=0x7f8698164df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869e39b700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c1840 0x7f8698164df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869eb9c700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86980c5d50 0x7f869814d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869e39b700 1 -- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86980c5d50 msgr2=0x7f869814d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869e39b700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86980c5d50 0x7f869814d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869e39b700 1 -- 172.21.15.33:0/3177352246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8690009a20 con 0x7f86980bfd70 2023-11-06T22:16:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869eb9c700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86980c5d50 0x7f869814d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.198+0000 7f869e39b700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86980bfd70 0x7f8698149640 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f869400c4e0 tx=0x7f8694007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.199+0000 7f868e7fc700 1 -- 172.21.15.33:0/3177352246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940072e0 con 0x7f86980bfd70 2023-11-06T22:16:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.199+0000 7f868e7fc700 1 -- 172.21.15.33:0/3177352246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8694007440 con 0x7f86980bfd70 2023-11-06T22:16:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.199+0000 7f868e7fc700 1 -- 172.21.15.33:0/3177352246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940057a0 con 0x7f86980bfd70 2023-11-06T22:16:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.199+0000 7f86a05ff700 1 -- 172.21.15.33:0/3177352246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86880092a0 con 0x7f86980bfd70 2023-11-06T22:16:35.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.199+0000 7f86a05ff700 1 -- 172.21.15.33:0/3177352246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869814e560 con 0x7f86980bfd70 2023-11-06T22:16:35.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.200+0000 7f86a05ff700 1 -- 172.21.15.33:0/3177352246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869806c6c0 con 0x7f86980bfd70 2023-11-06T22:16:35.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.200+0000 7f868e7fc700 1 -- 172.21.15.33:0/3177352246 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8694005bd0 con 0x7f86980bfd70 2023-11-06T22:16:35.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.201+0000 7f868e7fc700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f867c05f270 0x7f867c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.201+0000 7f868e7fc700 1 -- 172.21.15.33:0/3177352246 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f8694079a50 con 0x7f86980bfd70 2023-11-06T22:16:35.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.202+0000 7f869db9a700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f867c05f270 0x7f867c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.202+0000 7f869db9a700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f867c05f270 0x7f867c061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f86980c1390 tx=0x7f8688017000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:35.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.205+0000 7f868e7fc700 1 -- 172.21.15.33:0/3177352246 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f869404aec0 con 0x7f86980bfd70 2023-11-06T22:16:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.450+0000 7f86a05ff700 1 -- 172.21.15.33:0/3177352246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f86980c1460 con 0x7f86980bfd70 2023-11-06T22:16:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.451+0000 7f868e7fc700 1 -- 172.21.15.33:0/3177352246 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v47) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f869404e4e0 con 0x7f86980bfd70 2023-11-06T22:16:35.452 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:35.453 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:16:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T22:16:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T22:16:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T22:16:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:16:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:16:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:16:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T22:16:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T22:16:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:16:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:16:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:16:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T22:16:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T22:16:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:16:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:16:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:16:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T22:16:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T22:16:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:16:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:16:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:16:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T22:16:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T22:16:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:16:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:16:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T22:16:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T22:16:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:16:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:16:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.465 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T22:16:35.465 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T22:16:35.466 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.466 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:16:35.466 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:16:35.466 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.467 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T22:16:35.467 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T22:16:35.467 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:16:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:16:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T22:16:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T22:16:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:16:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:16:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T22:16:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T22:16:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:16:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:16:35.472 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.472 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T22:16:35.473 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T22:16:35.473 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.473 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:16:35.473 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:16:35.474 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.474 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.475 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T22:16:35.475 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T22:16:35.475 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.475 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:16:35.475 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:16:35.476 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.476 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T22:16:35.477 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T22:16:35.477 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.477 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:16:35.477 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-11-06T22:16:35.477 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T22:16:35.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.478 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T22:16:35.478 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T22:16:35.479 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.479 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:16:35.479 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T22:16:35.479 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T22:16:35.479 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T22:16:35.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.480 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T22:16:35.480 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T22:16:35.481 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.481 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:16:35.481 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T22:16:35.481 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T22:16:35.481 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T22:16:35.482 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.482 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T22:16:35.482 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T22:16:35.482 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.483 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:16:35.483 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T22:16:35.483 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T22:16:35.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T22:16:35.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.484 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T22:16:35.484 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T22:16:35.484 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.485 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T22:16:35.485 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:16:35.485 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:16:35.485 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.485 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.486 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T22:16:35.486 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T22:16:35.486 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.486 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T22:16:35.487 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T22:16:35.487 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.487 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:35.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:35.488 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T22:16:35.488 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T22:16:35.488 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:16:35.488 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T22:16:35.488 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T22:16:35.489 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:16:35.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:35.489 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T22:16:35.489 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:16:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 -- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f867c05f270 msgr2=0x7f867c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f867c05f270 0x7f867c061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f86980c1390 tx=0x7f8688017000).stop 2023-11-06T22:16:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 -- 172.21.15.33:0/3177352246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86980bfd70 msgr2=0x7f8698149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86980bfd70 0x7f8698149640 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f869400c4e0 tx=0x7f8694007c70).stop 2023-11-06T22:16:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 -- 172.21.15.33:0/3177352246 shutdown_connections 2023-11-06T22:16:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f867c05f270 0x7f867c061730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86980c5d50 0x7f869814d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86980bfd70 0x7f8698149640 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 --2- 172.21.15.33:0/3177352246 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f86980c1840 0x7f8698164df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 -- 172.21.15.33:0/3177352246 >> 172.21.15.33:0/3177352246 conn(0x7f869805ec80 msgr2=0x7f86980cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 -- 172.21.15.33:0/3177352246 shutdown_connections 2023-11-06T22:16:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.453+0000 7f86a05ff700 1 -- 172.21.15.33:0/3177352246 wait complete. 2023-11-06T22:16:35.494 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:16:35.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.689+0000 7fb1a67ff700 1 Processor -- start 2023-11-06T22:16:35.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.689+0000 7fb1a67ff700 1 -- start start 2023-11-06T22:16:35.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.690+0000 7fb1a67ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00c1840 0x7fb1a00a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.690+0000 7fb1a67ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb1a00aa2d0 0x7fb1a00c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.691+0000 7fb1a67ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a01554a0 con 0x7fb1a00c6240 2023-11-06T22:16:35.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.691+0000 7fb1a67ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a0154960 con 0x7fb1a00aa2d0 2023-11-06T22:16:35.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.691+0000 7fb1a67ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0155930 con 0x7fb1a00c1840 2023-11-06T22:16:35.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.691+0000 7fb19ffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00c1840 0x7fb1a00a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.691+0000 7fb1a4d9c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fb1a00c6240 0x7fb1a0157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:37978/0 (socket says 172.21.15.33:37978) 2023-11-06T22:16:35.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.691+0000 7fb1a4d9c700 1 -- 172.21.15.33:0/3635058056 learned_addr learned my addr 172.21.15.33:0/3635058056 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:35.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.691+0000 7fb19f7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb1a00aa2d0 0x7fb1a00c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.692+0000 7fb19ffff700 1 -- 172.21.15.33:0/3635058056 >> v2:172.21.15.69:3300/0 conn(0x7fb1a00aa2d0 msgr2=0x7fb1a00c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.692+0000 7fb19ffff700 1 --2- 172.21.15.33:0/3635058056 >> v2:172.21.15.69:3300/0 conn(0x7fb1a00aa2d0 0x7fb1a00c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.692+0000 7fb19ffff700 1 -- 172.21.15.33:0/3635058056 >> v1:172.21.15.69:6789/0 conn(0x7fb1a00c6240 legacy=0x7fb1a0157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.692+0000 7fb19ffff700 1 -- 172.21.15.33:0/3635058056 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a0154b70 con 0x7fb1a00c1840 2023-11-06T22:16:35.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.692+0000 7fb19f7fe700 1 --2- 172.21.15.33:0/3635058056 >> v2:172.21.15.69:3300/0 conn(0x7fb1a00aa2d0 0x7fb1a00c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:35.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.692+0000 7fb19ffff700 1 --2- 172.21.15.33:0/3635058056 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00c1840 0x7fb1a00a9de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fb190009760 tx=0x7fb190009060).ready entity=mon.2 client_cookie=748a30bee6e593ff server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:35.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.692+0000 7fb19effd700 1 -- 172.21.15.33:0/3635058056 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19000f510 con 0x7fb1a00c1840 2023-11-06T22:16:35.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.692+0000 7fb19effd700 1 -- 172.21.15.33:0/3635058056 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19000f670 con 0x7fb1a00c1840 2023-11-06T22:16:35.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.692+0000 7fb19effd700 1 -- 172.21.15.33:0/3635058056 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19000f9f0 con 0x7fb1a00c1840 2023-11-06T22:16:35.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.693+0000 7fb1a67ff700 1 -- 172.21.15.33:0/3635058056 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00c1840 msgr2=0x7fb1a00a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.693+0000 7fb1a67ff700 1 --2- 172.21.15.33:0/3635058056 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00c1840 0x7fb1a00a9de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fb190009760 tx=0x7fb190009060).stop 2023-11-06T22:16:35.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.693+0000 7fb1a67ff700 1 -- 172.21.15.33:0/3635058056 shutdown_connections 2023-11-06T22:16:35.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.693+0000 7fb1a67ff700 1 --2- 172.21.15.33:0/3635058056 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00c1840 0x7fb1a00a9de0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.693+0000 7fb1a67ff700 1 --2- 172.21.15.33:0/3635058056 >> v2:172.21.15.69:3300/0 conn(0x7fb1a00aa2d0 0x7fb1a00c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.693+0000 7fb1a67ff700 1 -- 172.21.15.33:0/3635058056 >> 172.21.15.33:0/3635058056 conn(0x7fb1a005ec80 msgr2=0x7fb1a00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.693+0000 7fb1a67ff700 1 -- 172.21.15.33:0/3635058056 shutdown_connections 2023-11-06T22:16:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.693+0000 7fb1a67ff700 1 -- 172.21.15.33:0/3635058056 wait complete. 2023-11-06T22:16:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.695+0000 7fb1a67ff700 1 Processor -- start 2023-11-06T22:16:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.695+0000 7fb1a67ff700 1 -- start start 2023-11-06T22:16:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.696+0000 7fb1a67ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00aa2d0 0x7fb1a014ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.696+0000 7fb1a67ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a00c1840 0x7fb1a00af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a67ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a00c6240 0x7fb1a00afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a67ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0158670 con 0x7fb1a00c6240 2023-11-06T22:16:35.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a67ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0157ad0 con 0x7fb1a00c1840 2023-11-06T22:16:35.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a67ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0157d40 con 0x7fb1a00aa2d0 2023-11-06T22:16:35.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a4d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a00c6240 0x7fb1a00afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a4d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a00c6240 0x7fb1a00afe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58324/0 (socket says 172.21.15.33:58324) 2023-11-06T22:16:35.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a4d9c700 1 -- 172.21.15.33:0/1566951404 learned_addr learned my addr 172.21.15.33:0/1566951404 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:35.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb19ffff700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00aa2d0 0x7fb1a014ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb19f7fe700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a00c1840 0x7fb1a00af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a4d9c700 1 -- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00aa2d0 msgr2=0x7fb1a014ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a4d9c700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00aa2d0 0x7fb1a014ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a4d9c700 1 -- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a00c1840 msgr2=0x7fb1a00af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a4d9c700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a00c1840 0x7fb1a00af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.697+0000 7fb1a4d9c700 1 -- 172.21.15.33:0/1566951404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940092a0 con 0x7fb1a00c6240 2023-11-06T22:16:35.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.698+0000 7fb1a4d9c700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a00c6240 0x7fb1a00afe60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7fb19800c7f0 tx=0x7fb198001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.698+0000 7fb187fff700 1 -- 172.21.15.33:0/1566951404 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198005d30 con 0x7fb1a00c6240 2023-11-06T22:16:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.698+0000 7fb187fff700 1 -- 172.21.15.33:0/1566951404 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb198005e90 con 0x7fb1a00c6240 2023-11-06T22:16:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.698+0000 7fb1a67ff700 1 -- 172.21.15.33:0/1566951404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1900092a0 con 0x7fb1a00c6240 2023-11-06T22:16:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.698+0000 7fb187fff700 1 -- 172.21.15.33:0/1566951404 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980072e0 con 0x7fb1a00c6240 2023-11-06T22:16:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.698+0000 7fb1a67ff700 1 -- 172.21.15.33:0/1566951404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a0158300 con 0x7fb1a00c6240 2023-11-06T22:16:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.699+0000 7fb1a67ff700 1 -- 172.21.15.33:0/1566951404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a006c6c0 con 0x7fb1a00c6240 2023-11-06T22:16:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.700+0000 7fb187fff700 1 -- 172.21.15.33:0/1566951404 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb198007710 con 0x7fb1a00c6240 2023-11-06T22:16:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.701+0000 7fb187fff700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb18c05f270 0x7fb18c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.701+0000 7fb187fff700 1 -- 172.21.15.33:0/1566951404 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5313+0+0 (secure 0 0 0) 0x7fb198079ae0 con 0x7fb1a00c6240 2023-11-06T22:16:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.701+0000 7fb19ffff700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb18c05f270 0x7fb18c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:35.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.702+0000 7fb19ffff700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb18c05f270 0x7fb18c061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb190009fd0 tx=0x7fb190017000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:35.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.705+0000 7fb187fff700 1 -- 172.21.15.33:0/1566951404 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb19804ae90 con 0x7fb1a00c6240 2023-11-06T22:16:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.887+0000 7fb1a67ff700 1 -- 172.21.15.33:0/1566951404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1a01587b0 con 0x7fb1a00c6240 2023-11-06T22:16:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.888+0000 7fb187fff700 1 -- 172.21.15.33:0/1566951404 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+13841 (secure 0 0 0) 0x7fb19804e4b0 con 0x7fb1a00c6240 2023-11-06T22:16:35.889 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:35.890 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":200,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:16:34.931055+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:16:35.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 -- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb18c05f270 msgr2=0x7fb18c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb18c05f270 0x7fb18c061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb190009fd0 tx=0x7fb190017000).stop 2023-11-06T22:16:35.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 -- 172.21.15.33:0/1566951404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a00c6240 msgr2=0x7fb1a00afe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:35.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a00c6240 0x7fb1a00afe60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7fb19800c7f0 tx=0x7fb198001510).stop 2023-11-06T22:16:35.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 -- 172.21.15.33:0/1566951404 shutdown_connections 2023-11-06T22:16:35.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb18c05f270 0x7fb18c061730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb1a00aa2d0 0x7fb1a014ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a00c6240 0x7fb1a00afe60 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 --2- 172.21.15.33:0/1566951404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a00c1840 0x7fb1a00af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:35.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 -- 172.21.15.33:0/1566951404 >> 172.21.15.33:0/1566951404 conn(0x7fb1a005ec80 msgr2=0x7fb1a0154250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:35.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 -- 172.21.15.33:0/1566951404 shutdown_connections 2023-11-06T22:16:35.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:35.890+0000 7fb1a67ff700 1 -- 172.21.15.33:0/1566951404 wait complete. 2023-11-06T22:16:35.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T22:16:35.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T22:16:36.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.109+0000 7f9dc4bff700 1 Processor -- start 2023-11-06T22:16:36.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.109+0000 7f9dc4bff700 1 -- start start 2023-11-06T22:16:36.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.111+0000 7f9dc4bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9dc01544f0 0x7f9dc00c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.111+0000 7f9dc4bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc01556d0 con 0x7f9dc00c0c40 2023-11-06T22:16:36.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.111+0000 7f9dc4bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc0154ac0 con 0x7f9dc0064dd0 2023-11-06T22:16:36.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.111+0000 7f9dc4bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc01548d0 con 0x7f9dc01544f0 2023-11-06T22:16:36.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.111+0000 7f9dbdd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9dc00c0c40 0x7f9dc00646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44760/0 (socket says 172.21.15.33:44760) 2023-11-06T22:16:36.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.111+0000 7f9dbdd9b700 1 -- 172.21.15.33:0/3996541219 learned_addr learned my addr 172.21.15.33:0/3996541219 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:36.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.111+0000 7f9dbe59c700 1 --2- 172.21.15.33:0/3996541219 >> v2:172.21.15.69:3300/0 conn(0x7f9dc01544f0 0x7f9dc00c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.111+0000 7f9dbd59a700 1 -- 172.21.15.33:0/3996541219 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221440448 0 0) 0x7f9dc01556d0 con 0x7f9dc00c0c40 2023-11-06T22:16:36.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.111+0000 7f9dbd59a700 1 -- 172.21.15.33:0/3996541219 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da80033f0 con 0x7f9dc00c0c40 2023-11-06T22:16:36.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.112+0000 7f9dbe59c700 1 -- 172.21.15.33:0/3996541219 >> v1:172.21.15.69:6789/0 conn(0x7f9dc0064dd0 legacy=0x7f9dc00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.112+0000 7f9dbe59c700 1 -- 172.21.15.33:0/3996541219 >> v1:172.21.15.33:6789/0 conn(0x7f9dc00c0c40 legacy=0x7f9dc00646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.112+0000 7f9dbe59c700 1 -- 172.21.15.33:0/3996541219 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc0154da0 con 0x7f9dc01544f0 2023-11-06T22:16:36.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.112+0000 7f9dbe59c700 1 --2- 172.21.15.33:0/3996541219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc01544f0 0x7f9dc00c0750 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f9db0009760 tx=0x7f9db0009060).ready entity=mon.1 client_cookie=a2e1afa1921f1d73 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:36.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.112+0000 7f9dbd59a700 1 -- 172.21.15.33:0/3996541219 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db000f510 con 0x7f9dc01544f0 2023-11-06T22:16:36.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.112+0000 7f9dbd59a700 1 -- 172.21.15.33:0/3996541219 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db000f670 con 0x7f9dc01544f0 2023-11-06T22:16:36.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.112+0000 7f9dbd59a700 1 -- 172.21.15.33:0/3996541219 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db000f9f0 con 0x7f9dc01544f0 2023-11-06T22:16:36.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.113+0000 7f9dc4bff700 1 -- 172.21.15.33:0/3996541219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc01544f0 msgr2=0x7f9dc00c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.113+0000 7f9dc4bff700 1 --2- 172.21.15.33:0/3996541219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc01544f0 0x7f9dc00c0750 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f9db0009760 tx=0x7f9db0009060).stop 2023-11-06T22:16:36.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.113+0000 7f9dc4bff700 1 -- 172.21.15.33:0/3996541219 shutdown_connections 2023-11-06T22:16:36.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.113+0000 7f9dc4bff700 1 --2- 172.21.15.33:0/3996541219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc01544f0 0x7f9dc00c0750 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.113+0000 7f9dc4bff700 1 -- 172.21.15.33:0/3996541219 >> 172.21.15.33:0/3996541219 conn(0x7f9dc00cb900 msgr2=0x7f9dc00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:36.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.113+0000 7f9dc4bff700 1 -- 172.21.15.33:0/3996541219 shutdown_connections 2023-11-06T22:16:36.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.113+0000 7f9dc4bff700 1 -- 172.21.15.33:0/3996541219 wait complete. 2023-11-06T22:16:36.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.115+0000 7f9dc4bff700 1 Processor -- start 2023-11-06T22:16:36.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.115+0000 7f9dc4bff700 1 -- start start 2023-11-06T22:16:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dc4bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc0064dd0 0x7f9dc0147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dc4bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc00c0c40 0x7f9dc0167d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dc4bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9dc01544f0 0x7f9dc0168240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dc4bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0145540 con 0x7f9dc0064dd0 2023-11-06T22:16:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dc4bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00c2790 con 0x7f9dc00c0c40 2023-11-06T22:16:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dc4bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00b0ae0 con 0x7f9dc01544f0 2023-11-06T22:16:36.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc0064dd0 0x7f9dc0147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc0064dd0 0x7f9dc0147da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58334/0 (socket says 172.21.15.33:58334) 2023-11-06T22:16:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbe59c700 1 -- 172.21.15.33:0/2965593292 learned_addr learned my addr 172.21.15.33:0/2965593292 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbdd9b700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc00c0c40 0x7f9dc0167d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbed9d700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9dc01544f0 0x7f9dc0168240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbe59c700 1 -- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9dc01544f0 msgr2=0x7f9dc0168240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbe59c700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9dc01544f0 0x7f9dc0168240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbe59c700 1 -- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc00c0c40 msgr2=0x7f9dc0167d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbe59c700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc00c0c40 0x7f9dc0167d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbe59c700 1 -- 172.21.15.33:0/2965593292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db00092a0 con 0x7f9dc0064dd0 2023-11-06T22:16:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.116+0000 7f9dbe59c700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc0064dd0 0x7f9dc0147da0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f9db0009e00 tx=0x7f9db0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.117+0000 7f9dae7fc700 1 -- 172.21.15.33:0/2965593292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db000f9f0 con 0x7f9dc0064dd0 2023-11-06T22:16:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.117+0000 7f9dae7fc700 1 -- 172.21.15.33:0/2965593292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db0000cb0 con 0x7f9dc0064dd0 2023-11-06T22:16:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.117+0000 7f9dae7fc700 1 -- 172.21.15.33:0/2965593292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db001b670 con 0x7f9dc0064dd0 2023-11-06T22:16:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.117+0000 7f9dc4bff700 1 -- 172.21.15.33:0/2965593292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc01688c0 con 0x7f9dc0064dd0 2023-11-06T22:16:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.117+0000 7f9dc4bff700 1 -- 172.21.15.33:0/2965593292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc00c2140 con 0x7f9dc0064dd0 2023-11-06T22:16:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.118+0000 7f9dc4bff700 1 -- 172.21.15.33:0/2965593292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc0155080 con 0x7f9dc0064dd0 2023-11-06T22:16:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.119+0000 7f9dae7fc700 1 -- 172.21.15.33:0/2965593292 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9db001ba20 con 0x7f9dc0064dd0 2023-11-06T22:16:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.120+0000 7f9dae7fc700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9da005f270 0x7f9da0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.120+0000 7f9dae7fc700 1 -- 172.21.15.33:0/2965593292 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f9db007b840 con 0x7f9dc0064dd0 2023-11-06T22:16:36.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.120+0000 7f9dbdd9b700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9da005f270 0x7f9da0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.121+0000 7f9dbdd9b700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9da005f270 0x7f9da0061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9dc0063630 tx=0x7f9db4009040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:36.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.124+0000 7f9dae7fc700 1 -- 172.21.15.33:0/2965593292 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9db004cbf0 con 0x7f9dc0064dd0 2023-11-06T22:16:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.304+0000 7f9dc4bff700 1 -- 172.21.15.33:0/2965593292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9dc0065340 con 0x7f9dc0064dd0 2023-11-06T22:16:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.305+0000 7f9dae7fc700 1 -- 172.21.15.33:0/2965593292 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v200) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9db00280b0 con 0x7f9dc0064dd0 2023-11-06T22:16:36.307 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T22:16:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 -- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9da005f270 msgr2=0x7f9da0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9da005f270 0x7f9da0061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9dc0063630 tx=0x7f9db4009040).stop 2023-11-06T22:16:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 -- 172.21.15.33:0/2965593292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc0064dd0 msgr2=0x7f9dc0147da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc0064dd0 0x7f9dc0147da0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f9db0009e00 tx=0x7f9db0017000).stop 2023-11-06T22:16:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 -- 172.21.15.33:0/2965593292 shutdown_connections 2023-11-06T22:16:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9da005f270 0x7f9da0061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9dc01544f0 0x7f9dc0168240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc0064dd0 0x7f9dc0147da0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 --2- 172.21.15.33:0/2965593292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc00c0c40 0x7f9dc0167d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 -- 172.21.15.33:0/2965593292 >> 172.21.15.33:0/2965593292 conn(0x7f9dc00cb900 msgr2=0x7f9dc00ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 -- 172.21.15.33:0/2965593292 shutdown_connections 2023-11-06T22:16:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.307+0000 7f9dc4bff700 1 -- 172.21.15.33:0/2965593292 wait complete. 2023-11-06T22:16:36.326 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T22:16:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.523+0000 7f4beafff700 1 Processor -- start 2023-11-06T22:16:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.523+0000 7f4beafff700 1 -- start start 2023-11-06T22:16:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.524+0000 7f4beafff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4be414e0b0 0x7f4be414e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.525+0000 7f4beafff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be40bcc90 con 0x7f4be40bc210 2023-11-06T22:16:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.525+0000 7f4beafff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be40bcf70 con 0x7f4be40c1f00 2023-11-06T22:16:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.525+0000 7f4beafff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be40bd250 con 0x7f4be414e0b0 2023-11-06T22:16:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.525+0000 7f4be3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4be40bc210 0x7f4be40c17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44768/0 (socket says 172.21.15.33:44768) 2023-11-06T22:16:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.525+0000 7f4be3fff700 1 -- 172.21.15.33:0/4243991993 learned_addr learned my addr 172.21.15.33:0/4243991993 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.525+0000 7f4be8d9b700 1 --2- 172.21.15.33:0/4243991993 >> v2:172.21.15.69:3300/0 conn(0x7f4be414e0b0 0x7f4be414e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.525+0000 7f4be37fe700 1 -- 172.21.15.33:0/4243991993 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032716211 0 0) 0x7f4be40bcc90 con 0x7f4be40bc210 2023-11-06T22:16:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.525+0000 7f4be37fe700 1 -- 172.21.15.33:0/4243991993 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd40033f0 con 0x7f4be40bc210 2023-11-06T22:16:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.525+0000 7f4be37fe700 1 -- 172.21.15.33:0/4243991993 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515872969 0 0) 0x7f4be40bcf70 con 0x7f4be40c1f00 2023-11-06T22:16:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.525+0000 7f4be37fe700 1 -- 172.21.15.33:0/4243991993 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be40bcc90 con 0x7f4be40c1f00 2023-11-06T22:16:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.526+0000 7f4be8d9b700 1 -- 172.21.15.33:0/4243991993 >> v1:172.21.15.69:6789/0 conn(0x7f4be40c1f00 legacy=0x7f4be4064550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.526+0000 7f4be8d9b700 1 -- 172.21.15.33:0/4243991993 >> v1:172.21.15.33:6789/0 conn(0x7f4be40bc210 legacy=0x7f4be40c17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.526+0000 7f4be8d9b700 1 -- 172.21.15.33:0/4243991993 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40bd530 con 0x7f4be414e0b0 2023-11-06T22:16:36.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.526+0000 7f4be8d9b700 1 --2- 172.21.15.33:0/4243991993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4be414e0b0 0x7f4be414e490 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f4bd800a900 tx=0x7f4bd800a200).ready entity=mon.1 client_cookie=be4b1b4770b0987 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:36.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.526+0000 7f4be37fe700 1 -- 172.21.15.33:0/4243991993 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800c6b0 con 0x7f4be414e0b0 2023-11-06T22:16:36.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.526+0000 7f4be37fe700 1 -- 172.21.15.33:0/4243991993 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd800c810 con 0x7f4be414e0b0 2023-11-06T22:16:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.526+0000 7f4be37fe700 1 -- 172.21.15.33:0/4243991993 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800cb90 con 0x7f4be414e0b0 2023-11-06T22:16:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.526+0000 7f4beafff700 1 -- 172.21.15.33:0/4243991993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4be414e0b0 msgr2=0x7f4be414e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.526+0000 7f4beafff700 1 --2- 172.21.15.33:0/4243991993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4be414e0b0 0x7f4be414e490 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f4bd800a900 tx=0x7f4bd800a200).stop 2023-11-06T22:16:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.527+0000 7f4beafff700 1 -- 172.21.15.33:0/4243991993 shutdown_connections 2023-11-06T22:16:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.527+0000 7f4beafff700 1 --2- 172.21.15.33:0/4243991993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4be414e0b0 0x7f4be414e490 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.527+0000 7f4beafff700 1 -- 172.21.15.33:0/4243991993 >> 172.21.15.33:0/4243991993 conn(0x7f4be4062710 msgr2=0x7f4be4062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.527+0000 7f4beafff700 1 -- 172.21.15.33:0/4243991993 shutdown_connections 2023-11-06T22:16:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.527+0000 7f4beafff700 1 -- 172.21.15.33:0/4243991993 wait complete. 2023-11-06T22:16:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.528+0000 7f4beafff700 1 Processor -- start 2023-11-06T22:16:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.528+0000 7f4beafff700 1 -- start start 2023-11-06T22:16:36.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.529+0000 7f4beafff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4be40bc210 0x7f4be414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.529+0000 7f4beafff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be40c1f00 0x7f4be414fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.529+0000 7f4beafff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4be414e0b0 0x7f4be41466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.529+0000 7f4beafff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be4150420 con 0x7f4be40c1f00 2023-11-06T22:16:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.529+0000 7f4beafff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be4168540 con 0x7f4be414e0b0 2023-11-06T22:16:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.529+0000 7f4beafff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be41501a0 con 0x7f4be40bc210 2023-11-06T22:16:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.529+0000 7f4be3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be40c1f00 0x7f4be414fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be40c1f00 0x7f4be414fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58342/0 (socket says 172.21.15.33:58342) 2023-11-06T22:16:36.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be3fff700 1 -- 172.21.15.33:0/4192419928 learned_addr learned my addr 172.21.15.33:0/4192419928 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:36.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be8d9b700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4be40bc210 0x7f4be414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be959c700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4be414e0b0 0x7f4be41466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be3fff700 1 -- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4be40bc210 msgr2=0x7f4be414cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be3fff700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4be40bc210 0x7f4be414cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be3fff700 1 -- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4be414e0b0 msgr2=0x7f4be41466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be3fff700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4be414e0b0 0x7f4be41466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be3fff700 1 -- 172.21.15.33:0/4192419928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd800a440 con 0x7f4be40c1f00 2023-11-06T22:16:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be3fff700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be40c1f00 0x7f4be414fcb0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f4bd000c590 tx=0x7f4bd0007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be0ff9700 1 -- 172.21.15.33:0/4192419928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000ce60 con 0x7f4be40c1f00 2023-11-06T22:16:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be0ff9700 1 -- 172.21.15.33:0/4192419928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd00072e0 con 0x7f4be40c1f00 2023-11-06T22:16:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4beafff700 1 -- 172.21.15.33:0/4192419928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be41689d0 con 0x7f4be40c1f00 2023-11-06T22:16:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.530+0000 7f4be0ff9700 1 -- 172.21.15.33:0/4192419928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0007660 con 0x7f4be40c1f00 2023-11-06T22:16:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.531+0000 7f4beafff700 1 -- 172.21.15.33:0/4192419928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be4167ef0 con 0x7f4be40c1f00 2023-11-06T22:16:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.532+0000 7f4beafff700 1 -- 172.21.15.33:0/4192419928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be41686f0 con 0x7f4be40c1f00 2023-11-06T22:16:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.532+0000 7f4be0ff9700 1 -- 172.21.15.33:0/4192419928 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4bd00059b0 con 0x7f4be40c1f00 2023-11-06T22:16:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.534+0000 7f4be0ff9700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4bc805f270 0x7f4bc8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.534+0000 7f4be0ff9700 1 -- 172.21.15.33:0/4192419928 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f4bd0079730 con 0x7f4be40c1f00 2023-11-06T22:16:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.534+0000 7f4be8d9b700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4bc805f270 0x7f4bc8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.535+0000 7f4be8d9b700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4bc805f270 0x7f4bc8061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4bd8006d80 tx=0x7f4bd800a200).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.537+0000 7f4be0ff9700 1 -- 172.21.15.33:0/4192419928 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4be41686f0 con 0x7f4be40c1f00 2023-11-06T22:16:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.717+0000 7f4beafff700 1 -- 172.21.15.33:0/4192419928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4be41686f0 con 0x7f4be40c1f00 2023-11-06T22:16:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.717+0000 7f4be0ff9700 1 -- 172.21.15.33:0/4192419928 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v200) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4bd0018020 con 0x7f4be40c1f00 2023-11-06T22:16:36.719 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T22:16:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 -- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4bc805f270 msgr2=0x7f4bc8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4bc805f270 0x7f4bc8061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4bd8006d80 tx=0x7f4bd800a200).stop 2023-11-06T22:16:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 -- 172.21.15.33:0/4192419928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be40c1f00 msgr2=0x7f4be414fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be40c1f00 0x7f4be414fcb0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f4bd000c590 tx=0x7f4bd0007d20).stop 2023-11-06T22:16:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 -- 172.21.15.33:0/4192419928 shutdown_connections 2023-11-06T22:16:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4bc805f270 0x7f4bc8061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4be40bc210 0x7f4be414cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4be414e0b0 0x7f4be41466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 --2- 172.21.15.33:0/4192419928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be40c1f00 0x7f4be414fcb0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 -- 172.21.15.33:0/4192419928 >> 172.21.15.33:0/4192419928 conn(0x7f4be4062710 msgr2=0x7f4be40c13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.719+0000 7f4beafff700 1 -- 172.21.15.33:0/4192419928 shutdown_connections 2023-11-06T22:16:36.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.720+0000 7f4beafff700 1 -- 172.21.15.33:0/4192419928 wait complete. 2023-11-06T22:16:36.739 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T22:16:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.933+0000 7f87a99ff700 1 Processor -- start 2023-11-06T22:16:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.933+0000 7f87a99ff700 1 -- start start 2023-11-06T22:16:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.934+0000 7f87a99ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f87a4151dc0 0x7f87a40c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.935+0000 7f87a99ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a4063110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.935+0000 7f87a99ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a4154e90 con 0x7f87a4155320 2023-11-06T22:16:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.935+0000 7f87a99ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a40639d0 con 0x7f87a4151dc0 2023-11-06T22:16:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.935+0000 7f87a99ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a4063cb0 con 0x7f87a41521a0 2023-11-06T22:16:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.935+0000 7f87a27fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a4063110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.935+0000 7f87a27fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a4063110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:43908/0 (socket says 172.21.15.33:43908) 2023-11-06T22:16:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.935+0000 7f87a27fc700 1 -- 172.21.15.33:0/2976718454 learned_addr learned my addr 172.21.15.33:0/2976718454 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.935+0000 7f87a2ffd700 1 --2- 172.21.15.33:0/2976718454 >> v2:172.21.15.69:3300/0 conn(0x7f87a4151dc0 0x7f87a40c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.936+0000 7f87a27fc700 1 -- 172.21.15.33:0/2976718454 >> v2:172.21.15.69:3300/0 conn(0x7f87a4151dc0 msgr2=0x7f87a40c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.936+0000 7f87a27fc700 1 --2- 172.21.15.33:0/2976718454 >> v2:172.21.15.69:3300/0 conn(0x7f87a4151dc0 0x7f87a40c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.936+0000 7f87a27fc700 1 -- 172.21.15.33:0/2976718454 >> v1:172.21.15.69:6789/0 conn(0x7f87a4155320 legacy=0x7f87a40c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.936+0000 7f87a27fc700 1 -- 172.21.15.33:0/2976718454 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a4063f90 con 0x7f87a41521a0 2023-11-06T22:16:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.936+0000 7f87a1ffb700 1 -- 172.21.15.33:0/2976718454 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279875510 0 0) 0x7f87a4154e90 con 0x7f87a4155320 2023-11-06T22:16:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.936+0000 7f87a27fc700 1 --2- 172.21.15.33:0/2976718454 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a4063110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f8798006fd0 tx=0x7f87980090f0).ready entity=mon.2 client_cookie=4f9e675992ae04a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.936+0000 7f87a1ffb700 1 -- 172.21.15.33:0/2976718454 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800b580 con 0x7f87a41521a0 2023-11-06T22:16:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.936+0000 7f87a1ffb700 1 -- 172.21.15.33:0/2976718454 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879800b6e0 con 0x7f87a41521a0 2023-11-06T22:16:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.936+0000 7f87a1ffb700 1 -- 172.21.15.33:0/2976718454 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800ba60 con 0x7f87a41521a0 2023-11-06T22:16:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.937+0000 7f87a99ff700 1 -- 172.21.15.33:0/2976718454 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 msgr2=0x7f87a4063110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.937+0000 7f87a99ff700 1 --2- 172.21.15.33:0/2976718454 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a4063110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f8798006fd0 tx=0x7f87980090f0).stop 2023-11-06T22:16:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.937+0000 7f87a99ff700 1 -- 172.21.15.33:0/2976718454 shutdown_connections 2023-11-06T22:16:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.937+0000 7f87a99ff700 1 --2- 172.21.15.33:0/2976718454 >> v2:172.21.15.69:3300/0 conn(0x7f87a4151dc0 0x7f87a40c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.937+0000 7f87a99ff700 1 --2- 172.21.15.33:0/2976718454 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a4063110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.937+0000 7f87a99ff700 1 -- 172.21.15.33:0/2976718454 >> 172.21.15.33:0/2976718454 conn(0x7f87a405e1b0 msgr2=0x7f87a405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.937+0000 7f87a99ff700 1 -- 172.21.15.33:0/2976718454 shutdown_connections 2023-11-06T22:16:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.937+0000 7f87a99ff700 1 -- 172.21.15.33:0/2976718454 wait complete. 2023-11-06T22:16:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.938+0000 7f87a99ff700 1 Processor -- start 2023-11-06T22:16:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.938+0000 7f87a99ff700 1 -- start start 2023-11-06T22:16:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.939+0000 7f87a99ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87a4151dc0 0x7f87a40c3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.939+0000 7f87a99ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a414da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.939+0000 7f87a99ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a4155320 0x7f87a414d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.939+0000 7f87a99ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a414df30 con 0x7f87a4155320 2023-11-06T22:16:36.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.939+0000 7f87a99ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a4168670 con 0x7f87a4151dc0 2023-11-06T22:16:36.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.939+0000 7f87a99ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a414d560 con 0x7f87a41521a0 2023-11-06T22:16:36.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a4155320 0x7f87a414d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a4155320 0x7f87a414d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58356/0 (socket says 172.21.15.33:58356) 2023-11-06T22:16:36.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a37fe700 1 -- 172.21.15.33:0/2688123884 learned_addr learned my addr 172.21.15.33:0/2688123884 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:36.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a27fc700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a414da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a2ffd700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87a4151dc0 0x7f87a40c3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a37fe700 1 -- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 msgr2=0x7f87a414da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a37fe700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a414da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a37fe700 1 -- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87a4151dc0 msgr2=0x7f87a40c3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a37fe700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87a4151dc0 0x7f87a40c3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a37fe700 1 -- 172.21.15.33:0/2688123884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8798009370 con 0x7f87a4155320 2023-11-06T22:16:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a27fc700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a414da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.940+0000 7f87a37fe700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a4155320 0x7f87a414d070 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7f879c00c810 tx=0x7f879c001500).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.941+0000 7f878affd700 1 -- 172.21.15.33:0/2688123884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c00e4a0 con 0x7f87a4155320 2023-11-06T22:16:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.941+0000 7f878affd700 1 -- 172.21.15.33:0/2688123884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c00e600 con 0x7f87a4155320 2023-11-06T22:16:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.941+0000 7f878affd700 1 -- 172.21.15.33:0/2688123884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c00e9a0 con 0x7f87a4155320 2023-11-06T22:16:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.941+0000 7f87a99ff700 1 -- 172.21.15.33:0/2688123884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a4167a60 con 0x7f87a4155320 2023-11-06T22:16:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.941+0000 7f87a99ff700 1 -- 172.21.15.33:0/2688123884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a4168020 con 0x7f87a4155320 2023-11-06T22:16:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.942+0000 7f87a99ff700 1 -- 172.21.15.33:0/2688123884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a41687b0 con 0x7f87a4155320 2023-11-06T22:16:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.943+0000 7f878affd700 1 -- 172.21.15.33:0/2688123884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f879c012400 con 0x7f87a4155320 2023-11-06T22:16:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.944+0000 7f878affd700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f879005f240 0x7f8790061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.944+0000 7f878affd700 1 -- 172.21.15.33:0/2688123884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f879c0797f0 con 0x7f87a4155320 2023-11-06T22:16:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.945+0000 7f87a2ffd700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f879005f240 0x7f8790061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.945+0000 7f87a2ffd700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f879005f240 0x7f8790061700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f87a4167820 tx=0x7f8794009060).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:36.948+0000 7f878affd700 1 -- 172.21.15.33:0/2688123884 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f879c048af0 con 0x7f87a4155320 2023-11-06T22:16:37.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.134+0000 7f87a99ff700 1 -- 172.21.15.33:0/2688123884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f87a4154b20 con 0x7f87a4155320 2023-11-06T22:16:37.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.135+0000 7f878affd700 1 -- 172.21.15.33:0/2688123884 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v200) v1 ==== 84+0+143 (secure 0 0 0) 0x7f879c017070 con 0x7f87a4155320 2023-11-06T22:16:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:37.137 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T22:16:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T22:16:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T22:16:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:16:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T22:16:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T22:16:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:37.139 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T22:16:37.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 -- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f879005f240 msgr2=0x7f8790061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f879005f240 0x7f8790061700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f87a4167820 tx=0x7f8794009060).stop 2023-11-06T22:16:37.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 -- 172.21.15.33:0/2688123884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a4155320 msgr2=0x7f87a414d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a4155320 0x7f87a414d070 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7f879c00c810 tx=0x7f879c001500).stop 2023-11-06T22:16:37.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 -- 172.21.15.33:0/2688123884 shutdown_connections 2023-11-06T22:16:37.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f879005f240 0x7f8790061700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a4155320 0x7f87a414d070 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87a4151dc0 0x7f87a40c3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 --2- 172.21.15.33:0/2688123884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87a41521a0 0x7f87a414da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 -- 172.21.15.33:0/2688123884 >> 172.21.15.33:0/2688123884 conn(0x7f87a405e1b0 msgr2=0x7f87a4150c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:37.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 -- 172.21.15.33:0/2688123884 shutdown_connections 2023-11-06T22:16:37.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.137+0000 7f87a99ff700 1 -- 172.21.15.33:0/2688123884 wait complete. 2023-11-06T22:16:37.156 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T22:16:37.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T22:16:37.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.356+0000 7f805c5ff700 1 Processor -- start 2023-11-06T22:16:37.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.356+0000 7f805c5ff700 1 -- start start 2023-11-06T22:16:37.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.357+0000 7f805c5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8054067c20 0x7f8054155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.357+0000 7f805c5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8054155800 0x7f80540666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.357+0000 7f805c5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8054068170 con 0x7f805414e630 2023-11-06T22:16:37.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.357+0000 7f805c5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8054068450 con 0x7f8054067c20 2023-11-06T22:16:37.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.357+0000 7f805c5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8054068730 con 0x7f8054155800 2023-11-06T22:16:37.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.357+0000 7f805a39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f805414e630 0x7f805414ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44778/0 (socket says 172.21.15.33:44778) 2023-11-06T22:16:37.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.357+0000 7f805a39b700 1 -- 172.21.15.33:0/1039879485 learned_addr learned my addr 172.21.15.33:0/1039879485 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:37.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.357+0000 7f8059b9a700 1 --2- 172.21.15.33:0/1039879485 >> v2:172.21.15.33:3300/0 conn(0x7f8054067c20 0x7f8054155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.358+0000 7f805ab9c700 1 --2- 172.21.15.33:0/1039879485 >> v2:172.21.15.69:3300/0 conn(0x7f8054155800 0x7f80540666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.358+0000 7f8059399700 1 -- 172.21.15.33:0/1039879485 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205204426 0 0) 0x7f8054068170 con 0x7f805414e630 2023-11-06T22:16:37.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.358+0000 7f8059b9a700 1 -- 172.21.15.33:0/1039879485 >> v2:172.21.15.69:3300/0 conn(0x7f8054155800 msgr2=0x7f80540666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:37.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.358+0000 7f8059b9a700 1 --2- 172.21.15.33:0/1039879485 >> v2:172.21.15.69:3300/0 conn(0x7f8054155800 0x7f80540666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.358+0000 7f8059b9a700 1 -- 172.21.15.33:0/1039879485 >> v1:172.21.15.33:6789/0 conn(0x7f805414e630 legacy=0x7f805414ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.358+0000 7f8059b9a700 1 -- 172.21.15.33:0/1039879485 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8054068a10 con 0x7f8054067c20 2023-11-06T22:16:37.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.358+0000 7f8059b9a700 1 --2- 172.21.15.33:0/1039879485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8054067c20 0x7f8054155310 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7f80540692b0 tx=0x7f8044009060).ready entity=mon.0 client_cookie=b4fa96eb0baca7c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:37.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.358+0000 7f8059399700 1 -- 172.21.15.33:0/1039879485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804400f510 con 0x7f8054067c20 2023-11-06T22:16:37.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.358+0000 7f8059399700 1 -- 172.21.15.33:0/1039879485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804400f670 con 0x7f8054067c20 2023-11-06T22:16:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.358+0000 7f8059399700 1 -- 172.21.15.33:0/1039879485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804400f9f0 con 0x7f8054067c20 2023-11-06T22:16:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.359+0000 7f805c5ff700 1 -- 172.21.15.33:0/1039879485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8054067c20 msgr2=0x7f8054155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.359+0000 7f805c5ff700 1 --2- 172.21.15.33:0/1039879485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8054067c20 0x7f8054155310 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7f80540692b0 tx=0x7f8044009060).stop 2023-11-06T22:16:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.359+0000 7f805c5ff700 1 -- 172.21.15.33:0/1039879485 shutdown_connections 2023-11-06T22:16:37.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.359+0000 7f805c5ff700 1 --2- 172.21.15.33:0/1039879485 >> v2:172.21.15.69:3300/0 conn(0x7f8054155800 0x7f80540666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.359+0000 7f805c5ff700 1 --2- 172.21.15.33:0/1039879485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8054067c20 0x7f8054155310 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.359+0000 7f805c5ff700 1 -- 172.21.15.33:0/1039879485 >> 172.21.15.33:0/1039879485 conn(0x7f80540c8e60 msgr2=0x7f80540c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:37.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.359+0000 7f805c5ff700 1 -- 172.21.15.33:0/1039879485 shutdown_connections 2023-11-06T22:16:37.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.359+0000 7f805c5ff700 1 -- 172.21.15.33:0/1039879485 wait complete. 2023-11-06T22:16:37.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.361+0000 7f805c5ff700 1 Processor -- start 2023-11-06T22:16:37.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.361+0000 7f805c5ff700 1 -- start start 2023-11-06T22:16:37.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f805c5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8054067c20 0x7f805414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f805c5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f805414e630 0x7f80540ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f805c5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8054155800 0x7f80540c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f805c5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8054137be0 con 0x7f805414e630 2023-11-06T22:16:37.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f805c5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8054138a40 con 0x7f8054155800 2023-11-06T22:16:37.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f805c5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8054137e30 con 0x7f8054067c20 2023-11-06T22:16:37.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f8059b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f805414e630 0x7f80540ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f8059b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f805414e630 0x7f80540ac7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58374/0 (socket says 172.21.15.33:58374) 2023-11-06T22:16:37.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f8059b9a700 1 -- 172.21.15.33:0/1613569834 learned_addr learned my addr 172.21.15.33:0/1613569834 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:37.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f805a39b700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8054067c20 0x7f805414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f805ab9c700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8054155800 0x7f80540c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f8059b9a700 1 -- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8054067c20 msgr2=0x7f805414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f8059b9a700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8054067c20 0x7f805414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f8059b9a700 1 -- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8054155800 msgr2=0x7f80540c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f8059b9a700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8054155800 0x7f80540c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f8059b9a700 1 -- 172.21.15.33:0/1613569834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440092a0 con 0x7f805414e630 2023-11-06T22:16:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f805a39b700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8054067c20 0x7f805414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.362+0000 7f805ab9c700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8054155800 0x7f80540c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.363+0000 7f8059b9a700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f805414e630 0x7f80540ac7b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7f8044009dd0 tx=0x7f8044017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.363+0000 7f804a7fc700 1 -- 172.21.15.33:0/1613569834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804400f9f0 con 0x7f805414e630 2023-11-06T22:16:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.363+0000 7f804a7fc700 1 -- 172.21.15.33:0/1613569834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8044000c90 con 0x7f805414e630 2023-11-06T22:16:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.363+0000 7f805c5ff700 1 -- 172.21.15.33:0/1613569834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8054138110 con 0x7f805414e630 2023-11-06T22:16:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.363+0000 7f804a7fc700 1 -- 172.21.15.33:0/1613569834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804401b660 con 0x7f805414e630 2023-11-06T22:16:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.363+0000 7f805c5ff700 1 -- 172.21.15.33:0/1613569834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80541386d0 con 0x7f805414e630 2023-11-06T22:16:37.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.364+0000 7f805c5ff700 1 -- 172.21.15.33:0/1613569834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8054138dd0 con 0x7f805414e630 2023-11-06T22:16:37.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.365+0000 7f804a7fc700 1 -- 172.21.15.33:0/1613569834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f804401ba10 con 0x7f805414e630 2023-11-06T22:16:37.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.366+0000 7f804a7fc700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f803805f270 0x7f8038061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.366+0000 7f804a7fc700 1 -- 172.21.15.33:0/1613569834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f804407bc40 con 0x7f805414e630 2023-11-06T22:16:37.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.366+0000 7f805a39b700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f803805f270 0x7f8038061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.367+0000 7f805a39b700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f803805f270 0x7f8038061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f8050001750 tx=0x7f8050009040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:37.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.370+0000 7f804a7fc700 1 -- 172.21.15.33:0/1613569834 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8054138dd0 con 0x7f805414e630 2023-11-06T22:16:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.549+0000 7f805c5ff700 1 -- 172.21.15.33:0/1613569834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f8054138dd0 con 0x7f805414e630 2023-11-06T22:16:37.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.657+0000 7f804a7fc700 1 -- 172.21.15.33:0/1613569834 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v201) v1 ==== 152+0+0 (secure 0 0 0) 0x7f8054138dd0 con 0x7f805414e630 2023-11-06T22:16:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.662+0000 7f805c5ff700 1 -- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f803805f270 msgr2=0x7f8038061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.662+0000 7f805c5ff700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f803805f270 0x7f8038061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f8050001750 tx=0x7f8050009040).stop 2023-11-06T22:16:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.662+0000 7f805c5ff700 1 -- 172.21.15.33:0/1613569834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f805414e630 msgr2=0x7f80540ac7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.662+0000 7f805c5ff700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f805414e630 0x7f80540ac7b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7f8044009dd0 tx=0x7f8044017000).stop 2023-11-06T22:16:37.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.663+0000 7f805c5ff700 1 -- 172.21.15.33:0/1613569834 shutdown_connections 2023-11-06T22:16:37.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.663+0000 7f805c5ff700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f803805f270 0x7f8038061730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.663+0000 7f805c5ff700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8054067c20 0x7f805414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.663+0000 7f805c5ff700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8054155800 0x7f80540c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.663+0000 7f805c5ff700 1 --2- 172.21.15.33:0/1613569834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f805414e630 0x7f80540ac7b0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.663+0000 7f805c5ff700 1 -- 172.21.15.33:0/1613569834 >> 172.21.15.33:0/1613569834 conn(0x7f80540c8e60 msgr2=0x7f8054135ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:37.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.663+0000 7f805c5ff700 1 -- 172.21.15.33:0/1613569834 shutdown_connections 2023-11-06T22:16:37.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.663+0000 7f805c5ff700 1 -- 172.21.15.33:0/1613569834 wait complete. 2023-11-06T22:16:37.668 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T22:16:37.687 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T22:16:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.887+0000 7f673a7ff700 1 Processor -- start 2023-11-06T22:16:37.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.887+0000 7f673a7ff700 1 -- start start 2023-11-06T22:16:37.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.889+0000 7f673a7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f673414faa0 0x7f6734064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.889+0000 7f673a7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414fe80 0x7f6734152750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.889+0000 7f673a7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6734152c40 0x7f67340c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.889+0000 7f673a7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6734064270 con 0x7f6734152c40 2023-11-06T22:16:37.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.889+0000 7f673a7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67340651b0 con 0x7f673414faa0 2023-11-06T22:16:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.889+0000 7f673a7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67341531f0 con 0x7f673414fe80 2023-11-06T22:16:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.889+0000 7f6738d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6734152c40 0x7f67340c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.889+0000 7f6738d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6734152c40 0x7f67340c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58380/0 (socket says 172.21.15.33:58380) 2023-11-06T22:16:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.889+0000 7f6738d9c700 1 -- 172.21.15.33:0/1086871043 learned_addr learned my addr 172.21.15.33:0/1086871043 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f67337fe700 1 --2- 172.21.15.33:0/1086871043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414fe80 0x7f6734152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f6733fff700 1 --2- 172.21.15.33:0/1086871043 >> v2:172.21.15.69:3300/0 conn(0x7f673414faa0 0x7f6734064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f6738d9c700 1 -- 172.21.15.33:0/1086871043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414fe80 msgr2=0x7f6734152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f6738d9c700 1 --2- 172.21.15.33:0/1086871043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414fe80 0x7f6734152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f6738d9c700 1 -- 172.21.15.33:0/1086871043 >> v2:172.21.15.69:3300/0 conn(0x7f673414faa0 msgr2=0x7f6734064cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f6738d9c700 1 --2- 172.21.15.33:0/1086871043 >> v2:172.21.15.69:3300/0 conn(0x7f673414faa0 0x7f6734064cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f6738d9c700 1 -- 172.21.15.33:0/1086871043 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67341534d0 con 0x7f6734152c40 2023-11-06T22:16:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f67337fe700 1 --2- 172.21.15.33:0/1086871043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414fe80 0x7f6734152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f6738d9c700 1 --2- 172.21.15.33:0/1086871043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6734152c40 0x7f67340c3450 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f67340676b0 tx=0x7f672c009840).ready entity=mon.0 client_cookie=51f14a2c28b54f5e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f6732ffd700 1 -- 172.21.15.33:0/1086871043 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00e070 con 0x7f6734152c40 2023-11-06T22:16:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f6732ffd700 1 -- 172.21.15.33:0/1086871043 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c00ba50 con 0x7f6734152c40 2023-11-06T22:16:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.890+0000 7f6732ffd700 1 -- 172.21.15.33:0/1086871043 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00bdd0 con 0x7f6734152c40 2023-11-06T22:16:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.891+0000 7f673a7ff700 1 -- 172.21.15.33:0/1086871043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6734152c40 msgr2=0x7f67340c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.891+0000 7f673a7ff700 1 --2- 172.21.15.33:0/1086871043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6734152c40 0x7f67340c3450 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f67340676b0 tx=0x7f672c009840).stop 2023-11-06T22:16:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.891+0000 7f673a7ff700 1 -- 172.21.15.33:0/1086871043 shutdown_connections 2023-11-06T22:16:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.891+0000 7f673a7ff700 1 --2- 172.21.15.33:0/1086871043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6734152c40 0x7f67340c3450 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.891+0000 7f673a7ff700 1 --2- 172.21.15.33:0/1086871043 >> v2:172.21.15.69:3300/0 conn(0x7f673414faa0 0x7f6734064cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.891+0000 7f673a7ff700 1 --2- 172.21.15.33:0/1086871043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414fe80 0x7f6734152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.891+0000 7f673a7ff700 1 -- 172.21.15.33:0/1086871043 >> 172.21.15.33:0/1086871043 conn(0x7f6734062710 msgr2=0x7f6734062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:37.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.891+0000 7f673a7ff700 1 -- 172.21.15.33:0/1086871043 shutdown_connections 2023-11-06T22:16:37.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.891+0000 7f673a7ff700 1 -- 172.21.15.33:0/1086871043 wait complete. 2023-11-06T22:16:37.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.892+0000 7f673a7ff700 1 Processor -- start 2023-11-06T22:16:37.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.893+0000 7f673a7ff700 1 -- start start 2023-11-06T22:16:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f673a7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414faa0 0x7f673413b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f673a7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f673414fe80 0x7f67340ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f673a7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6734152c40 0x7f67340ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f673a7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67340ac8e0 con 0x7f673414fe80 2023-11-06T22:16:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f673a7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67340c4a40 con 0x7f6734152c40 2023-11-06T22:16:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f673a7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673414b570 con 0x7f673414faa0 2023-11-06T22:16:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f67337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f673414fe80 0x7f67340ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f67337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f673414fe80 0x7f67340ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58386/0 (socket says 172.21.15.33:58386) 2023-11-06T22:16:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f67337fe700 1 -- 172.21.15.33:0/2259136281 learned_addr learned my addr 172.21.15.33:0/2259136281 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f6733fff700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414faa0 0x7f673413b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f6738d9c700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6734152c40 0x7f67340ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f67337fe700 1 -- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414faa0 msgr2=0x7f673413b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f67337fe700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414faa0 0x7f673413b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f67337fe700 1 -- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6734152c40 msgr2=0x7f67340ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f67337fe700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6734152c40 0x7f67340ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:37.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.894+0000 7f67337fe700 1 -- 172.21.15.33:0/2259136281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c009ae0 con 0x7f673414fe80 2023-11-06T22:16:37.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.895+0000 7f67337fe700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f673414fe80 0x7f67340ab4f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f6728006fd0 tx=0x7f6728008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:37.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.895+0000 7f671bfff700 1 -- 172.21.15.33:0/2259136281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728016070 con 0x7f673414fe80 2023-11-06T22:16:37.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.895+0000 7f671bfff700 1 -- 172.21.15.33:0/2259136281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728011590 con 0x7f673414fe80 2023-11-06T22:16:37.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.895+0000 7f671bfff700 1 -- 172.21.15.33:0/2259136281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728011910 con 0x7f673414fe80 2023-11-06T22:16:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.895+0000 7f673a7ff700 1 -- 172.21.15.33:0/2259136281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67340ac040 con 0x7f673414fe80 2023-11-06T22:16:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.895+0000 7f673a7ff700 1 -- 172.21.15.33:0/2259136281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67340ac600 con 0x7f673414fe80 2023-11-06T22:16:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.896+0000 7f673a7ff700 1 -- 172.21.15.33:0/2259136281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67340c4830 con 0x7f673414fe80 2023-11-06T22:16:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.897+0000 7f671bfff700 1 -- 172.21.15.33:0/2259136281 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6728011d00 con 0x7f673414fe80 2023-11-06T22:16:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.898+0000 7f671bfff700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f672005f270 0x7f6720061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.898+0000 7f671bfff700 1 -- 172.21.15.33:0/2259136281 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5683+0+0 (secure 0 0 0) 0x7f6728079cc0 con 0x7f673414fe80 2023-11-06T22:16:37.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.899+0000 7f6733fff700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f672005f270 0x7f6720061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:37.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.900+0000 7f6733fff700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f672005f270 0x7f6720061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f672400bbc0 tx=0x7f6724008040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:37.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:37.903+0000 7f671bfff700 1 -- 172.21.15.33:0/2259136281 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6728080080 con 0x7f673414fe80 2023-11-06T22:16:38.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.085+0000 7f673a7ff700 1 -- 172.21.15.33:0/2259136281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f67340644c0 con 0x7f673414fe80 2023-11-06T22:16:38.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.661+0000 7f671bfff700 1 -- 172.21.15.33:0/2259136281 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v202) v1 ==== 170+0+0 (secure 0 0 0) 0x7f672804e8d0 con 0x7f673414fe80 2023-11-06T22:16:38.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.666+0000 7f673a7ff700 1 -- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f672005f270 msgr2=0x7f6720061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:38.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.666+0000 7f673a7ff700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f672005f270 0x7f6720061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f672400bbc0 tx=0x7f6724008040).stop 2023-11-06T22:16:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.666+0000 7f673a7ff700 1 -- 172.21.15.33:0/2259136281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f673414fe80 msgr2=0x7f67340ab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.666+0000 7f673a7ff700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f673414fe80 0x7f67340ab4f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f6728006fd0 tx=0x7f6728008040).stop 2023-11-06T22:16:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.667+0000 7f673a7ff700 1 -- 172.21.15.33:0/2259136281 shutdown_connections 2023-11-06T22:16:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.667+0000 7f673a7ff700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f672005f270 0x7f6720061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.667+0000 7f673a7ff700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f673414faa0 0x7f673413b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.667+0000 7f673a7ff700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6734152c40 0x7f67340ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.667+0000 7f673a7ff700 1 --2- 172.21.15.33:0/2259136281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f673414fe80 0x7f67340ab4f0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.667+0000 7f673a7ff700 1 -- 172.21.15.33:0/2259136281 >> 172.21.15.33:0/2259136281 conn(0x7f6734062710 msgr2=0x7f6734150e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.667+0000 7f673a7ff700 1 -- 172.21.15.33:0/2259136281 shutdown_connections 2023-11-06T22:16:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.667+0000 7f673a7ff700 1 -- 172.21.15.33:0/2259136281 wait complete. 2023-11-06T22:16:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T22:16:38.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T22:16:38.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.905+0000 7f207c5ff700 1 Processor -- start 2023-11-06T22:16:38.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.905+0000 7f207c5ff700 1 -- start start 2023-11-06T22:16:38.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.906+0000 7f207c5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2074133f10 0x7f20741520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:38.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.906+0000 7f207c5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20741525e0 con 0x7f20740c3b60 2023-11-06T22:16:38.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.906+0000 7f207c5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2074154f90 con 0x7f20741342f0 2023-11-06T22:16:38.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.906+0000 7f207c5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207414bc20 con 0x7f2074133f10 2023-11-06T22:16:38.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.907+0000 7f207a39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2074133f10 0x7f20741520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:38.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.907+0000 7f207ab9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f20740c3b60 0x7f2074069530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58010/0 (socket says 172.21.15.33:58010) 2023-11-06T22:16:38.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.907+0000 7f207ab9c700 1 -- 172.21.15.33:0/250906576 learned_addr learned my addr 172.21.15.33:0/250906576 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:38.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.907+0000 7f207a39b700 1 -- 172.21.15.33:0/250906576 >> v1:172.21.15.69:6789/0 conn(0x7f20741342f0 legacy=0x7f20740c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:38.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.907+0000 7f207a39b700 1 -- 172.21.15.33:0/250906576 >> v1:172.21.15.33:6789/0 conn(0x7f20740c3b60 legacy=0x7f2074069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:38.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.907+0000 7f207a39b700 1 -- 172.21.15.33:0/250906576 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2074155270 con 0x7f2074133f10 2023-11-06T22:16:38.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.907+0000 7f2079399700 1 -- 172.21.15.33:0/250906576 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134944984 0 0) 0x7f20741525e0 con 0x7f20740c3b60 2023-11-06T22:16:38.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.907+0000 7f207a39b700 1 --2- 172.21.15.33:0/250906576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074133f10 0x7f20741520f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f20700096e0 tx=0x7f2070009060).ready entity=mon.0 client_cookie=2871b1da5ee0e572 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:38.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.908+0000 7f2079399700 1 -- 172.21.15.33:0/250906576 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207000f510 con 0x7f2074133f10 2023-11-06T22:16:38.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.908+0000 7f2079399700 1 -- 172.21.15.33:0/250906576 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207000f670 con 0x7f2074133f10 2023-11-06T22:16:38.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.908+0000 7f2079399700 1 -- 172.21.15.33:0/250906576 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207000f9f0 con 0x7f2074133f10 2023-11-06T22:16:38.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.908+0000 7f207c5ff700 1 -- 172.21.15.33:0/250906576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074133f10 msgr2=0x7f20741520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.908+0000 7f207c5ff700 1 --2- 172.21.15.33:0/250906576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074133f10 0x7f20741520f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f20700096e0 tx=0x7f2070009060).stop 2023-11-06T22:16:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.908+0000 7f207c5ff700 1 -- 172.21.15.33:0/250906576 shutdown_connections 2023-11-06T22:16:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.908+0000 7f207c5ff700 1 --2- 172.21.15.33:0/250906576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2074133f10 0x7f20741520f0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:38.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.908+0000 7f207c5ff700 1 -- 172.21.15.33:0/250906576 >> 172.21.15.33:0/250906576 conn(0x7f2074062710 msgr2=0x7f2074062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:38.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.908+0000 7f207c5ff700 1 -- 172.21.15.33:0/250906576 shutdown_connections 2023-11-06T22:16:38.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.908+0000 7f207c5ff700 1 -- 172.21.15.33:0/250906576 wait complete. 2023-11-06T22:16:38.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.910+0000 7f207c5ff700 1 Processor -- start 2023-11-06T22:16:38.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.910+0000 7f207c5ff700 1 -- start start 2023-11-06T22:16:38.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207c5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20740c3b60 0x7f2074134d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207c5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2074133f10 0x7f2074135d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207c5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20741342f0 0x7f207414a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207c5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2074136230 con 0x7f20740c3b60 2023-11-06T22:16:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207c5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207414acf0 con 0x7f20741342f0 2023-11-06T22:16:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207c5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20741364f0 con 0x7f2074133f10 2023-11-06T22:16:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207a39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20740c3b60 0x7f2074134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207a39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20740c3b60 0x7f2074134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41766/0 (socket says 172.21.15.33:41766) 2023-11-06T22:16:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207a39b700 1 -- 172.21.15.33:0/3052231361 learned_addr learned my addr 172.21.15.33:0/3052231361 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207a39b700 1 -- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2074133f10 msgr2=0x7f2074135d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:38.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207a39b700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2074133f10 0x7f2074135d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:38.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207a39b700 1 -- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20741342f0 msgr2=0x7f207414a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:38.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207a39b700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20741342f0 0x7f207414a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:38.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.911+0000 7f207a39b700 1 -- 172.21.15.33:0/3052231361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20700092a0 con 0x7f20740c3b60 2023-11-06T22:16:38.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.912+0000 7f207a39b700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20740c3b60 0x7f2074134d20 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f2070009e30 tx=0x7f2070017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.912+0000 7f206a7fc700 1 -- 172.21.15.33:0/3052231361 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207000f9f0 con 0x7f20740c3b60 2023-11-06T22:16:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.912+0000 7f206a7fc700 1 -- 172.21.15.33:0/3052231361 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2070000cc0 con 0x7f20740c3b60 2023-11-06T22:16:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.912+0000 7f207c5ff700 1 -- 172.21.15.33:0/3052231361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20741367d0 con 0x7f20740c3b60 2023-11-06T22:16:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.912+0000 7f206a7fc700 1 -- 172.21.15.33:0/3052231361 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700246a0 con 0x7f20740c3b60 2023-11-06T22:16:38.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.912+0000 7f207c5ff700 1 -- 172.21.15.33:0/3052231361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2074136d90 con 0x7f20740c3b60 2023-11-06T22:16:38.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.913+0000 7f207c5ff700 1 -- 172.21.15.33:0/3052231361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20741573d0 con 0x7f20740c3b60 2023-11-06T22:16:38.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.914+0000 7f206a7fc700 1 -- 172.21.15.33:0/3052231361 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2070021070 con 0x7f20740c3b60 2023-11-06T22:16:38.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.915+0000 7f206a7fc700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f205805f270 0x7f2058061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:38.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.915+0000 7f206a7fc700 1 -- 172.21.15.33:0/3052231361 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6065+0+0 (secure 0 0 0) 0x7f207007c0c0 con 0x7f20740c3b60 2023-11-06T22:16:38.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.915+0000 7f2079b9a700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f205805f270 0x7f2058061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:38.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.916+0000 7f2079b9a700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f205805f270 0x7f2058061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f20740c7700 tx=0x7f2064007940).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:38.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:38.920+0000 7f206a7fc700 1 -- 172.21.15.33:0/3052231361 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f207004d180 con 0x7f20740c3b60 2023-11-06T22:16:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.144+0000 7f207c5ff700 1 -- 172.21.15.33:0/3052231361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2074155550 con 0x7f20740c3b60 2023-11-06T22:16:39.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.152+0000 7f206a7fc700 1 -- 172.21.15.33:0/3052231361 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v80) v1 ==== 174+0+0 (secure 0 0 0) 0x7f20700507a0 con 0x7f20740c3b60 2023-11-06T22:16:39.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.162+0000 7f207c5ff700 1 -- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f205805f270 msgr2=0x7f2058061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.162+0000 7f207c5ff700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f205805f270 0x7f2058061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f20740c7700 tx=0x7f2064007940).stop 2023-11-06T22:16:39.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.162+0000 7f207c5ff700 1 -- 172.21.15.33:0/3052231361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20740c3b60 msgr2=0x7f2074134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.162+0000 7f207c5ff700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20740c3b60 0x7f2074134d20 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f2070009e30 tx=0x7f2070017000).stop 2023-11-06T22:16:39.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.163+0000 7f207c5ff700 1 -- 172.21.15.33:0/3052231361 shutdown_connections 2023-11-06T22:16:39.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.163+0000 7f207c5ff700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f205805f270 0x7f2058061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.163+0000 7f207c5ff700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20741342f0 0x7f207414a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:39.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.163+0000 7f207c5ff700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20740c3b60 0x7f2074134d20 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.163+0000 7f207c5ff700 1 --2- 172.21.15.33:0/3052231361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2074133f10 0x7f2074135d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:39.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.163+0000 7f207c5ff700 1 -- 172.21.15.33:0/3052231361 >> 172.21.15.33:0/3052231361 conn(0x7f2074062710 msgr2=0x7f2074151910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:39.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.164+0000 7f207c5ff700 1 -- 172.21.15.33:0/3052231361 shutdown_connections 2023-11-06T22:16:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.164+0000 7f207c5ff700 1 -- 172.21.15.33:0/3052231361 wait complete. 2023-11-06T22:16:39.171 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 23 and data pool 24 2023-11-06T22:16:39.198 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:16:39.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.394+0000 7fe0a05ff700 1 Processor -- start 2023-11-06T22:16:39.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.394+0000 7fe0a05ff700 1 -- start start 2023-11-06T22:16:39.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.395+0000 7fe0a05ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe0980c1840 0x7fe0980a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.395+0000 7fe0a05ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980c6460 0x7fe0980bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.395+0000 7fe0a05ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0981554a0 con 0x7fe0980aa2d0 2023-11-06T22:16:39.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.395+0000 7fe0a05ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe098154960 con 0x7fe0980c1840 2023-11-06T22:16:39.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.395+0000 7fe0a05ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe098155930 con 0x7fe0980c6460 2023-11-06T22:16:39.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.396+0000 7fe09eb9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980c6460 0x7fe0980bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.396+0000 7fe09db9a700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fe0980aa2d0 0x7fe0980c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:33260/0 (socket says 172.21.15.33:33260) 2023-11-06T22:16:39.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.396+0000 7fe09eb9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980c6460 0x7fe0980bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:46340/0 (socket says 172.21.15.33:46340) 2023-11-06T22:16:39.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.396+0000 7fe09db9a700 1 -- 172.21.15.33:0/1144721706 learned_addr learned my addr 172.21.15.33:0/1144721706 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:39.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.396+0000 7fe09e39b700 1 --2- 172.21.15.33:0/1144721706 >> v2:172.21.15.69:3300/0 conn(0x7fe0980c1840 0x7fe0980a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.396+0000 7fe09eb9c700 1 -- 172.21.15.33:0/1144721706 >> v2:172.21.15.69:3300/0 conn(0x7fe0980c1840 msgr2=0x7fe0980a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:39.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.396+0000 7fe09d399700 1 -- 172.21.15.33:0/1144721706 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350332671 0 0) 0x7fe0981554a0 con 0x7fe0980aa2d0 2023-11-06T22:16:39.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.396+0000 7fe09eb9c700 1 --2- 172.21.15.33:0/1144721706 >> v2:172.21.15.69:3300/0 conn(0x7fe0980c1840 0x7fe0980a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.396+0000 7fe09eb9c700 1 -- 172.21.15.33:0/1144721706 >> v1:172.21.15.69:6789/0 conn(0x7fe0980aa2d0 legacy=0x7fe0980c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.397+0000 7fe09eb9c700 1 -- 172.21.15.33:0/1144721706 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe098154b70 con 0x7fe0980c6460 2023-11-06T22:16:39.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.397+0000 7fe09e39b700 1 --2- 172.21.15.33:0/1144721706 >> v2:172.21.15.69:3300/0 conn(0x7fe0980c1840 0x7fe0980a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:39.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.397+0000 7fe09eb9c700 1 --2- 172.21.15.33:0/1144721706 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980c6460 0x7fe0980bfd70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fe090009e60 tx=0x7fe090009780).ready entity=mon.2 client_cookie=ebef381e1ce156d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:39.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.397+0000 7fe09d399700 1 -- 172.21.15.33:0/1144721706 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09000e070 con 0x7fe0980c6460 2023-11-06T22:16:39.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.397+0000 7fe09d399700 1 -- 172.21.15.33:0/1144721706 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09000bb60 con 0x7fe0980c6460 2023-11-06T22:16:39.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.397+0000 7fe09d399700 1 -- 172.21.15.33:0/1144721706 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090012410 con 0x7fe0980c6460 2023-11-06T22:16:39.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.397+0000 7fe0a05ff700 1 -- 172.21.15.33:0/1144721706 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980c6460 msgr2=0x7fe0980bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.397+0000 7fe0a05ff700 1 --2- 172.21.15.33:0/1144721706 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980c6460 0x7fe0980bfd70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fe090009e60 tx=0x7fe090009780).stop 2023-11-06T22:16:39.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.398+0000 7fe0a05ff700 1 -- 172.21.15.33:0/1144721706 shutdown_connections 2023-11-06T22:16:39.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.398+0000 7fe0a05ff700 1 --2- 172.21.15.33:0/1144721706 >> v2:172.21.15.69:3300/0 conn(0x7fe0980c1840 0x7fe0980a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.398+0000 7fe0a05ff700 1 --2- 172.21.15.33:0/1144721706 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980c6460 0x7fe0980bfd70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.398+0000 7fe0a05ff700 1 -- 172.21.15.33:0/1144721706 >> 172.21.15.33:0/1144721706 conn(0x7fe09805ec80 msgr2=0x7fe0980cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:39.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.398+0000 7fe0a05ff700 1 -- 172.21.15.33:0/1144721706 shutdown_connections 2023-11-06T22:16:39.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.398+0000 7fe0a05ff700 1 -- 172.21.15.33:0/1144721706 wait complete. 2023-11-06T22:16:39.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.399+0000 7fe0a05ff700 1 Processor -- start 2023-11-06T22:16:39.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.399+0000 7fe0a05ff700 1 -- start start 2023-11-06T22:16:39.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.400+0000 7fe0a05ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980aa2d0 0x7fe098149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.400+0000 7fe0a05ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0980c1840 0x7fe0981560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.400+0000 7fe0a05ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0980c6460 0x7fe09814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.400+0000 7fe0a05ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0981650c0 con 0x7fe0980c6460 2023-11-06T22:16:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.400+0000 7fe0a05ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980abc20 con 0x7fe0980c1840 2023-11-06T22:16:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.400+0000 7fe0a05ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe098156a50 con 0x7fe0980aa2d0 2023-11-06T22:16:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.400+0000 7fe09eb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0980c6460 0x7fe09814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09eb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0980c6460 0x7fe09814fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41768/0 (socket says 172.21.15.33:41768) 2023-11-06T22:16:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09eb9c700 1 -- 172.21.15.33:0/4192201982 learned_addr learned my addr 172.21.15.33:0/4192201982 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09e39b700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980aa2d0 0x7fe098149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09db9a700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0980c1840 0x7fe0981560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09eb9c700 1 -- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980aa2d0 msgr2=0x7fe098149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09eb9c700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980aa2d0 0x7fe098149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09eb9c700 1 -- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0980c1840 msgr2=0x7fe0981560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09eb9c700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0980c1840 0x7fe0981560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09eb9c700 1 -- 172.21.15.33:0/4192201982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0940092a0 con 0x7fe0980c6460 2023-11-06T22:16:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09db9a700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0980c1840 0x7fe0981560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe09eb9c700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0980c6460 0x7fe09814fe10 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7fe090011000 tx=0x7fe098155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe08e7fc700 1 -- 172.21.15.33:0/4192201982 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09001d630 con 0x7fe0980c6460 2023-11-06T22:16:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe08e7fc700 1 -- 172.21.15.33:0/4192201982 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09001d790 con 0x7fe0980c6460 2023-11-06T22:16:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.401+0000 7fe0a05ff700 1 -- 172.21.15.33:0/4192201982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe090009a20 con 0x7fe0980c6460 2023-11-06T22:16:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.402+0000 7fe08e7fc700 1 -- 172.21.15.33:0/4192201982 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09001db50 con 0x7fe0980c6460 2023-11-06T22:16:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.402+0000 7fe0a05ff700 1 -- 172.21.15.33:0/4192201982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0980ab8b0 con 0x7fe0980c6460 2023-11-06T22:16:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.403+0000 7fe0a05ff700 1 -- 172.21.15.33:0/4192201982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0980abdd0 con 0x7fe0980c6460 2023-11-06T22:16:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.404+0000 7fe08e7fc700 1 -- 172.21.15.33:0/4192201982 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe090003040 con 0x7fe0980c6460 2023-11-06T22:16:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.404+0000 7fe08e7fc700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe07c05f270 0x7fe07c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.404+0000 7fe08e7fc700 1 -- 172.21.15.33:0/4192201982 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fe09007c320 con 0x7fe0980c6460 2023-11-06T22:16:39.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.405+0000 7fe09e39b700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe07c05f270 0x7fe07c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.405+0000 7fe09e39b700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe07c05f270 0x7fe07c061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe094009dc0 tx=0x7fe094009060).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:39.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.409+0000 7fe08e7fc700 1 -- 172.21.15.33:0/4192201982 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe0980abdd0 con 0x7fe0980c6460 2023-11-06T22:16:39.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.591+0000 7fe0a05ff700 1 -- 172.21.15.33:0/4192201982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0980abd90 con 0x7fe0980c6460 2023-11-06T22:16:39.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.592+0000 7fe08e7fc700 1 -- 172.21.15.33:0/4192201982 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+17074 (secure 0 0 0) 0x7fe090050db0 con 0x7fe0980c6460 2023-11-06T22:16:39.593 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:39.594 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":203,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:16:39.144871+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:16:37.603904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T22:16:38.139152+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:16:39.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 -- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe07c05f270 msgr2=0x7fe07c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe07c05f270 0x7fe07c061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe094009dc0 tx=0x7fe094009060).stop 2023-11-06T22:16:39.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 -- 172.21.15.33:0/4192201982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0980c6460 msgr2=0x7fe09814fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0980c6460 0x7fe09814fe10 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7fe090011000 tx=0x7fe098155410).stop 2023-11-06T22:16:39.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 -- 172.21.15.33:0/4192201982 shutdown_connections 2023-11-06T22:16:39.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe07c05f270 0x7fe07c061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe0980aa2d0 0x7fe098149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0980c6460 0x7fe09814fe10 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 --2- 172.21.15.33:0/4192201982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0980c1840 0x7fe0981560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 -- 172.21.15.33:0/4192201982 >> 172.21.15.33:0/4192201982 conn(0x7fe09805ec80 msgr2=0x7fe098152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 -- 172.21.15.33:0/4192201982 shutdown_connections 2023-11-06T22:16:39.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.594+0000 7fe0a05ff700 1 -- 172.21.15.33:0/4192201982 wait complete. 2023-11-06T22:16:39.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:16:39.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.808+0000 7f9322dff700 1 Processor -- start 2023-11-06T22:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.809+0000 7f9322dff700 1 -- start start 2023-11-06T22:16:39.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.810+0000 7f9322dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f931c0c1840 0x7f931c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.810+0000 7f9322dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f931c0c6460 0x7f931c0bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.810+0000 7f9322dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c1554a0 con 0x7f931c0aa2d0 2023-11-06T22:16:39.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.810+0000 7f9322dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c154960 con 0x7f931c0c1840 2023-11-06T22:16:39.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.810+0000 7f9322dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c155930 con 0x7f931c0c6460 2023-11-06T22:16:39.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.810+0000 7f9320b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f931c0c1840 0x7f931c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.810+0000 7f931bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f931c0aa2d0 0x7f931c0c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58026/0 (socket says 172.21.15.33:58026) 2023-11-06T22:16:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.810+0000 7f931bfff700 1 -- 172.21.15.33:0/901634616 learned_addr learned my addr 172.21.15.33:0/901634616 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.811+0000 7f932139c700 1 --2- 172.21.15.33:0/901634616 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f931c0c6460 0x7f931c0bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.811+0000 7f9320b9b700 1 -- 172.21.15.33:0/901634616 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f931c0c6460 msgr2=0x7f931c0bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:39.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.811+0000 7f9320b9b700 1 --2- 172.21.15.33:0/901634616 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f931c0c6460 0x7f931c0bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.811+0000 7f931b7fe700 1 -- 172.21.15.33:0/901634616 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404986031 0 0) 0x7f931c1554a0 con 0x7f931c0aa2d0 2023-11-06T22:16:39.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.811+0000 7f9320b9b700 1 -- 172.21.15.33:0/901634616 >> v1:172.21.15.33:6789/0 conn(0x7f931c0aa2d0 legacy=0x7f931c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.811+0000 7f9320b9b700 1 -- 172.21.15.33:0/901634616 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c154b70 con 0x7f931c0c1840 2023-11-06T22:16:39.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.811+0000 7f9320b9b700 1 --2- 172.21.15.33:0/901634616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 0x7f931c0a9de0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f9310009760 tx=0x7f9310009060).ready entity=mon.0 client_cookie=99075fcb0228fa31 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:39.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.811+0000 7f931b7fe700 1 -- 172.21.15.33:0/901634616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931000b580 con 0x7f931c0c1840 2023-11-06T22:16:39.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.811+0000 7f931b7fe700 1 -- 172.21.15.33:0/901634616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931000b6e0 con 0x7f931c0c1840 2023-11-06T22:16:39.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.811+0000 7f931b7fe700 1 -- 172.21.15.33:0/901634616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931000ba60 con 0x7f931c0c1840 2023-11-06T22:16:39.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.812+0000 7f9322dff700 1 -- 172.21.15.33:0/901634616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 msgr2=0x7f931c0a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.812+0000 7f9322dff700 1 --2- 172.21.15.33:0/901634616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 0x7f931c0a9de0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f9310009760 tx=0x7f9310009060).stop 2023-11-06T22:16:39.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.812+0000 7f9322dff700 1 -- 172.21.15.33:0/901634616 shutdown_connections 2023-11-06T22:16:39.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.812+0000 7f9322dff700 1 --2- 172.21.15.33:0/901634616 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f931c0c6460 0x7f931c0bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.812+0000 7f9322dff700 1 --2- 172.21.15.33:0/901634616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 0x7f931c0a9de0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.812+0000 7f9322dff700 1 -- 172.21.15.33:0/901634616 >> 172.21.15.33:0/901634616 conn(0x7f931c05ec80 msgr2=0x7f931c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:39.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.812+0000 7f9322dff700 1 -- 172.21.15.33:0/901634616 shutdown_connections 2023-11-06T22:16:39.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.812+0000 7f9322dff700 1 -- 172.21.15.33:0/901634616 wait complete. 2023-11-06T22:16:39.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.813+0000 7f9322dff700 1 Processor -- start 2023-11-06T22:16:39.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.814+0000 7f9322dff700 1 -- start start 2023-11-06T22:16:39.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.814+0000 7f9322dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f931c0aa2d0 0x7f931c0ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.814+0000 7f9322dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 0x7f931c149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f9322dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f931c0c6460 0x7f931c0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f9322dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c156770 con 0x7f931c0c1840 2023-11-06T22:16:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f9322dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c158ad0 con 0x7f931c0aa2d0 2023-11-06T22:16:39.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f9322dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0ab590 con 0x7f931c0c6460 2023-11-06T22:16:39.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f931bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 0x7f931c149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f931bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 0x7f931c149b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41780/0 (socket says 172.21.15.33:41780) 2023-11-06T22:16:39.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f931bfff700 1 -- 172.21.15.33:0/2701661338 learned_addr learned my addr 172.21.15.33:0/2701661338 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:39.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f9320b9b700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f931c0aa2d0 0x7f931c0ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f932139c700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f931c0c6460 0x7f931c0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f931bfff700 1 -- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f931c0c6460 msgr2=0x7f931c0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f931bfff700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f931c0c6460 0x7f931c0c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f931bfff700 1 -- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f931c0aa2d0 msgr2=0x7f931c0ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:39.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f931bfff700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f931c0aa2d0 0x7f931c0ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f931bfff700 1 -- 172.21.15.33:0/2701661338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93100092a0 con 0x7f931c0c1840 2023-11-06T22:16:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.815+0000 7f931bfff700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 0x7f931c149b70 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f930800c4a0 tx=0x7f9308007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.816+0000 7f9318ff9700 1 -- 172.21.15.33:0/2701661338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930800ce50 con 0x7f931c0c1840 2023-11-06T22:16:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.816+0000 7f9318ff9700 1 -- 172.21.15.33:0/2701661338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93080072e0 con 0x7f931c0c1840 2023-11-06T22:16:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.816+0000 7f9322dff700 1 -- 172.21.15.33:0/2701661338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c0ab870 con 0x7f931c0c1840 2023-11-06T22:16:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.816+0000 7f9318ff9700 1 -- 172.21.15.33:0/2701661338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308007660 con 0x7f931c0c1840 2023-11-06T22:16:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.816+0000 7f9322dff700 1 -- 172.21.15.33:0/2701661338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c0abe30 con 0x7f931c0c1840 2023-11-06T22:16:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.817+0000 7f9322dff700 1 -- 172.21.15.33:0/2701661338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c06c6c0 con 0x7f931c0c1840 2023-11-06T22:16:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.818+0000 7f9318ff9700 1 -- 172.21.15.33:0/2701661338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f93080059b0 con 0x7f931c0c1840 2023-11-06T22:16:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.819+0000 7f9318ff9700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f930005f240 0x7f9300061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.819+0000 7f9318ff9700 1 -- 172.21.15.33:0/2701661338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f9308079aa0 con 0x7f931c0c1840 2023-11-06T22:16:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.820+0000 7f9320b9b700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f930005f240 0x7f9300061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:39.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.820+0000 7f9320b9b700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f930005f240 0x7f9300061700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9310006d30 tx=0x7f9310017000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:39.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:39.822+0000 7f9318ff9700 1 -- 172.21.15.33:0/2701661338 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f930804ab00 con 0x7f931c0c1840 2023-11-06T22:16:40.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.005+0000 7f9322dff700 1 -- 172.21.15.33:0/2701661338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f931c063f20 con 0x7f931c0c1840 2023-11-06T22:16:40.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.006+0000 7f9318ff9700 1 -- 172.21.15.33:0/2701661338 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f930804e120 con 0x7f931c0c1840 2023-11-06T22:16:40.008 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:40.009 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":203,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:16:39.144871+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:16:37.603904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T22:16:38.139152+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:16:40.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.008+0000 7f9322dff700 1 -- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f930005f240 msgr2=0x7f9300061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f930005f240 0x7f9300061700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9310006d30 tx=0x7f9310017000).stop 2023-11-06T22:16:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 -- 172.21.15.33:0/2701661338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 msgr2=0x7f931c149b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 0x7f931c149b70 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f930800c4a0 tx=0x7f9308007c30).stop 2023-11-06T22:16:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 -- 172.21.15.33:0/2701661338 shutdown_connections 2023-11-06T22:16:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f930005f240 0x7f9300061700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f931c0c6460 0x7f931c0c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f931c0aa2d0 0x7f931c0ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 --2- 172.21.15.33:0/2701661338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0c1840 0x7f931c149b70 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 -- 172.21.15.33:0/2701661338 >> 172.21.15.33:0/2701661338 conn(0x7f931c05ec80 msgr2=0x7f931c063140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 -- 172.21.15.33:0/2701661338 shutdown_connections 2023-11-06T22:16:40.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.009+0000 7f9322dff700 1 -- 172.21.15.33:0/2701661338 wait complete. 2023-11-06T22:16:40.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T22:16:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.232+0000 7f8b2c3ff700 1 Processor -- start 2023-11-06T22:16:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.232+0000 7f8b2c3ff700 1 -- start start 2023-11-06T22:16:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.233+0000 7f8b2c3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b24154330 0x7f8b240c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.233+0000 7f8b2c3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f8b240c7ff0 0x7f8b240646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.233+0000 7f8b2c3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b240c6b60 con 0x7f8b24154710 2023-11-06T22:16:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.233+0000 7f8b2c3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b240c7700 con 0x7f8b240c7ff0 2023-11-06T22:16:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.233+0000 7f8b2c3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b240c5cd0 con 0x7f8b24154330 2023-11-06T22:16:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.233+0000 7f8b2999a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8b24154710 0x7f8b240c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58040/0 (socket says 172.21.15.33:58040) 2023-11-06T22:16:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.233+0000 7f8b2999a700 1 -- 172.21.15.33:0/3775417621 learned_addr learned my addr 172.21.15.33:0/3775417621 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b2a99c700 1 --2- 172.21.15.33:0/3775417621 >> v2:172.21.15.69:3300/0 conn(0x7f8b240c7ff0 0x7f8b240646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b2a19b700 1 --2- 172.21.15.33:0/3775417621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b24154330 0x7f8b240c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b29199700 1 -- 172.21.15.33:0/3775417621 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814219313 0 0) 0x7f8b240c6b60 con 0x7f8b24154710 2023-11-06T22:16:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b29199700 1 -- 172.21.15.33:0/3775417621 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b100033f0 con 0x7f8b24154710 2023-11-06T22:16:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b2a19b700 1 -- 172.21.15.33:0/3775417621 >> v2:172.21.15.69:3300/0 conn(0x7f8b240c7ff0 msgr2=0x7f8b240646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b29199700 1 -- 172.21.15.33:0/3775417621 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2689136203 0 0) 0x7f8b100033f0 con 0x7f8b24154710 2023-11-06T22:16:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b2a19b700 1 --2- 172.21.15.33:0/3775417621 >> v2:172.21.15.69:3300/0 conn(0x7f8b240c7ff0 0x7f8b240646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:40.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b2a19b700 1 -- 172.21.15.33:0/3775417621 >> v1:172.21.15.33:6789/0 conn(0x7f8b24154710 legacy=0x7f8b240c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:40.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b2a19b700 1 -- 172.21.15.33:0/3775417621 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b240c5f50 con 0x7f8b24154330 2023-11-06T22:16:40.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b2a99c700 1 --2- 172.21.15.33:0/3775417621 >> v2:172.21.15.69:3300/0 conn(0x7f8b240c7ff0 0x7f8b240646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:40.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.234+0000 7f8b2a19b700 1 --2- 172.21.15.33:0/3775417621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b24154330 0x7f8b240c57e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8b20009760 tx=0x7f8b20009060).ready entity=mon.2 client_cookie=b8cf314ef2227898 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:40.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b29199700 1 -- 172.21.15.33:0/3775417621 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14001470 con 0x7f8b24154330 2023-11-06T22:16:40.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b29199700 1 -- 172.21.15.33:0/3775417621 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b20011400 con 0x7f8b24154330 2023-11-06T22:16:40.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b29199700 1 -- 172.21.15.33:0/3775417621 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20011780 con 0x7f8b24154330 2023-11-06T22:16:40.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/3775417621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b24154330 msgr2=0x7f8b240c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:40.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b2c3ff700 1 --2- 172.21.15.33:0/3775417621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b24154330 0x7f8b240c57e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8b20009760 tx=0x7f8b20009060).stop 2023-11-06T22:16:40.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/3775417621 shutdown_connections 2023-11-06T22:16:40.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b2c3ff700 1 --2- 172.21.15.33:0/3775417621 >> v2:172.21.15.69:3300/0 conn(0x7f8b240c7ff0 0x7f8b240646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:40.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b2c3ff700 1 --2- 172.21.15.33:0/3775417621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b24154330 0x7f8b240c57e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/3775417621 >> 172.21.15.33:0/3775417621 conn(0x7f8b240cb910 msgr2=0x7f8b240c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/3775417621 shutdown_connections 2023-11-06T22:16:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.235+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/3775417621 wait complete. 2023-11-06T22:16:40.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.237+0000 7f8b2c3ff700 1 Processor -- start 2023-11-06T22:16:40.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.237+0000 7f8b2c3ff700 1 -- start start 2023-11-06T22:16:40.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2c3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b240c7ff0 0x7f8b241530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2c3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b24154330 0x7f8b24157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2c3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b24154710 0x7f8b2415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2c3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24157570 con 0x7f8b24154330 2023-11-06T22:16:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2c3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b241579e0 con 0x7f8b24154710 2023-11-06T22:16:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2c3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2415b340 con 0x7f8b240c7ff0 2023-11-06T22:16:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b24154330 0x7f8b24157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b24154330 0x7f8b24157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41786/0 (socket says 172.21.15.33:41786) 2023-11-06T22:16:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2999a700 1 -- 172.21.15.33:0/2916360827 learned_addr learned my addr 172.21.15.33:0/2916360827 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2a19b700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b240c7ff0 0x7f8b241530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.238+0000 7f8b2a99c700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b24154710 0x7f8b2415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b2999a700 1 -- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b240c7ff0 msgr2=0x7f8b241530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b2999a700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b240c7ff0 0x7f8b241530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b2999a700 1 -- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b24154710 msgr2=0x7f8b2415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b2999a700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b24154710 0x7f8b2415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b2999a700 1 -- 172.21.15.33:0/2916360827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c009a20 con 0x7f8b24154330 2023-11-06T22:16:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b2a99c700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b24154710 0x7f8b2415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b2999a700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b24154330 0x7f8b24157e70 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f8b1400c810 tx=0x7f8b14001a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b1a7fc700 1 -- 172.21.15.33:0/2916360827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b140072e0 con 0x7f8b24154330 2023-11-06T22:16:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b1a7fc700 1 -- 172.21.15.33:0/2916360827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14007460 con 0x7f8b24154330 2023-11-06T22:16:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/2916360827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b200092a0 con 0x7f8b24154330 2023-11-06T22:16:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b1a7fc700 1 -- 172.21.15.33:0/2916360827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14005990 con 0x7f8b24154330 2023-11-06T22:16:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.239+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/2916360827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2415abf0 con 0x7f8b24154330 2023-11-06T22:16:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.240+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/2916360827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b241577b0 con 0x7f8b24154330 2023-11-06T22:16:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.241+0000 7f8b1a7fc700 1 -- 172.21.15.33:0/2916360827 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f8b14005d40 con 0x7f8b24154330 2023-11-06T22:16:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.242+0000 7f8b1a7fc700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8b0805f270 0x7f8b08061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.242+0000 7f8b1a7fc700 1 -- 172.21.15.33:0/2916360827 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f8b14079fe0 con 0x7f8b24154330 2023-11-06T22:16:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.243+0000 7f8b2a19b700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8b0805f270 0x7f8b08061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.243+0000 7f8b2a19b700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8b0805f270 0x7f8b08061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8b20009760 tx=0x7f8b20016000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.248+0000 7f8b1a7fc700 1 -- 172.21.15.33:0/2916360827 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8b241577b0 con 0x7f8b24154330 2023-11-06T22:16:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:40.472+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/2916360827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8b241577b0 con 0x7f8b24154330 2023-11-06T22:16:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.180+0000 7f8b1a7fc700 1 -- 172.21.15.33:0/2916360827 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8b241577b0 con 0x7f8b24154330 2023-11-06T22:16:41.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8b0805f270 msgr2=0x7f8b08061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:41.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8b0805f270 0x7f8b08061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8b20009760 tx=0x7f8b20016000).stop 2023-11-06T22:16:41.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/2916360827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b24154330 msgr2=0x7f8b24157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:41.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b24154330 0x7f8b24157e70 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f8b1400c810 tx=0x7f8b14001a00).stop 2023-11-06T22:16:41.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/2916360827 shutdown_connections 2023-11-06T22:16:41.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8b0805f270 0x7f8b08061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:41.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8b240c7ff0 0x7f8b241530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:41.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b24154710 0x7f8b2415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:41.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 --2- 172.21.15.33:0/2916360827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b24154330 0x7f8b24157e70 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:41.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/2916360827 >> 172.21.15.33:0/2916360827 conn(0x7f8b240cb910 msgr2=0x7f8b240abb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:41.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/2916360827 shutdown_connections 2023-11-06T22:16:41.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.183+0000 7f8b2c3ff700 1 -- 172.21.15.33:0/2916360827 wait complete. 2023-11-06T22:16:41.208 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T22:16:41.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T22:16:41.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.413+0000 7f0a591ff700 1 Processor -- start 2023-11-06T22:16:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.413+0000 7f0a591ff700 1 -- start start 2023-11-06T22:16:41.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.415+0000 7f0a591ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0a54154710 0x7f0a540c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.415+0000 7f0a591ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a540c7ff0 0x7f0a540646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.415+0000 7f0a591ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a540c6b60 con 0x7f0a54154330 2023-11-06T22:16:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.415+0000 7f0a591ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540c7700 con 0x7f0a540c7ff0 2023-11-06T22:16:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.415+0000 7f0a591ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540c5cd0 con 0x7f0a54154710 2023-11-06T22:16:41.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.415+0000 7f0a52d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0a54154330 0x7f0a540c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58054/0 (socket says 172.21.15.33:58054) 2023-11-06T22:16:41.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.415+0000 7f0a52d9d700 1 -- 172.21.15.33:0/2037303859 learned_addr learned my addr 172.21.15.33:0/2037303859 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:41.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.415+0000 7f0a5359e700 1 --2- 172.21.15.33:0/2037303859 >> v2:172.21.15.33:3300/0 conn(0x7f0a540c7ff0 0x7f0a540646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:41.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.415+0000 7f0a5259c700 1 --2- 172.21.15.33:0/2037303859 >> v2:172.21.15.69:3300/0 conn(0x7f0a54154710 0x7f0a540c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:41.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.416+0000 7f0a51d9b700 1 -- 172.21.15.33:0/2037303859 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140228945 0 0) 0x7f0a540c6b60 con 0x7f0a54154330 2023-11-06T22:16:41.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.416+0000 7f0a51d9b700 1 -- 172.21.15.33:0/2037303859 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a3c0033f0 con 0x7f0a54154330 2023-11-06T22:16:41.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.416+0000 7f0a5359e700 1 -- 172.21.15.33:0/2037303859 >> v2:172.21.15.69:3300/0 conn(0x7f0a54154710 msgr2=0x7f0a540c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:41.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.416+0000 7f0a5359e700 1 --2- 172.21.15.33:0/2037303859 >> v2:172.21.15.69:3300/0 conn(0x7f0a54154710 0x7f0a540c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:41.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.416+0000 7f0a5359e700 1 -- 172.21.15.33:0/2037303859 >> v1:172.21.15.33:6789/0 conn(0x7f0a54154330 legacy=0x7f0a540c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:41.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.416+0000 7f0a5359e700 1 -- 172.21.15.33:0/2037303859 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a540c5f50 con 0x7f0a540c7ff0 2023-11-06T22:16:41.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.416+0000 7f0a5359e700 1 --2- 172.21.15.33:0/2037303859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a540c7ff0 0x7f0a540646c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f0a540c8d10 tx=0x7f0a4c009780).ready entity=mon.0 client_cookie=96cddd81efddf3cf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:41.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.416+0000 7f0a51d9b700 1 -- 172.21.15.33:0/2037303859 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00e070 con 0x7f0a540c7ff0 2023-11-06T22:16:41.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.416+0000 7f0a51d9b700 1 -- 172.21.15.33:0/2037303859 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c00bab0 con 0x7f0a540c7ff0 2023-11-06T22:16:41.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.416+0000 7f0a51d9b700 1 -- 172.21.15.33:0/2037303859 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00be30 con 0x7f0a540c7ff0 2023-11-06T22:16:41.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.417+0000 7f0a591ff700 1 -- 172.21.15.33:0/2037303859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a540c7ff0 msgr2=0x7f0a540646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:41.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.417+0000 7f0a591ff700 1 --2- 172.21.15.33:0/2037303859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a540c7ff0 0x7f0a540646c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f0a540c8d10 tx=0x7f0a4c009780).stop 2023-11-06T22:16:41.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.417+0000 7f0a591ff700 1 -- 172.21.15.33:0/2037303859 shutdown_connections 2023-11-06T22:16:41.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.417+0000 7f0a591ff700 1 --2- 172.21.15.33:0/2037303859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a540c7ff0 0x7f0a540646c0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:41.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.417+0000 7f0a591ff700 1 --2- 172.21.15.33:0/2037303859 >> v2:172.21.15.69:3300/0 conn(0x7f0a54154710 0x7f0a540c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:41.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.417+0000 7f0a591ff700 1 -- 172.21.15.33:0/2037303859 >> 172.21.15.33:0/2037303859 conn(0x7f0a540cb910 msgr2=0x7f0a540c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:41.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.417+0000 7f0a591ff700 1 -- 172.21.15.33:0/2037303859 shutdown_connections 2023-11-06T22:16:41.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.417+0000 7f0a591ff700 1 -- 172.21.15.33:0/2037303859 wait complete. 2023-11-06T22:16:41.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.419+0000 7f0a591ff700 1 Processor -- start 2023-11-06T22:16:41.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.419+0000 7f0a591ff700 1 -- start start 2023-11-06T22:16:41.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.420+0000 7f0a591ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a540c7ff0 0x7f0a54154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:41.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.420+0000 7f0a591ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a54154330 0x7f0a540c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:41.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.420+0000 7f0a591ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a54154710 0x7f0a540c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:41.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.420+0000 7f0a591ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540c3890 con 0x7f0a54154330 2023-11-06T22:16:41.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.420+0000 7f0a591ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54157d50 con 0x7f0a540c7ff0 2023-11-06T22:16:41.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.420+0000 7f0a591ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540aeb60 con 0x7f0a54154710 2023-11-06T22:16:41.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a5259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a54154330 0x7f0a540c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:41.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a5259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a54154330 0x7f0a540c2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41808/0 (socket says 172.21.15.33:41808) 2023-11-06T22:16:41.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a5259c700 1 -- 172.21.15.33:0/2695623724 learned_addr learned my addr 172.21.15.33:0/2695623724 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:41.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a52d9d700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a540c7ff0 0x7f0a54154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:41.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a5359e700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a54154710 0x7f0a540c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:41.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a5259c700 1 -- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a54154710 msgr2=0x7f0a540c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:41.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a5259c700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a54154710 0x7f0a540c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:41.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a5259c700 1 -- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a540c7ff0 msgr2=0x7f0a54154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:41.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a5259c700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a540c7ff0 0x7f0a54154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:41.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a5259c700 1 -- 172.21.15.33:0/2695623724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c009a20 con 0x7f0a54154330 2023-11-06T22:16:41.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.421+0000 7f0a5259c700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a54154330 0x7f0a540c2eb0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f0a4800e8d0 tx=0x7f0a48009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:41.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.422+0000 7f0a3affd700 1 -- 172.21.15.33:0/2695623724 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48012600 con 0x7f0a54154330 2023-11-06T22:16:41.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.422+0000 7f0a3affd700 1 -- 172.21.15.33:0/2695623724 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a48012760 con 0x7f0a54154330 2023-11-06T22:16:41.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.422+0000 7f0a3affd700 1 -- 172.21.15.33:0/2695623724 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48012ae0 con 0x7f0a54154330 2023-11-06T22:16:41.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.422+0000 7f0a591ff700 1 -- 172.21.15.33:0/2695623724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5413f800 con 0x7f0a54154330 2023-11-06T22:16:41.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.422+0000 7f0a591ff700 1 -- 172.21.15.33:0/2695623724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a540ae510 con 0x7f0a54154330 2023-11-06T22:16:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.423+0000 7f0a591ff700 1 -- 172.21.15.33:0/2695623724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a540aecd0 con 0x7f0a54154330 2023-11-06T22:16:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.424+0000 7f0a3affd700 1 -- 172.21.15.33:0/2695623724 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0a4801b400 con 0x7f0a54154330 2023-11-06T22:16:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.425+0000 7f0a3affd700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0a4005f270 0x7f0a40061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.425+0000 7f0a3affd700 1 -- 172.21.15.33:0/2695623724 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f0a4807a7a0 con 0x7f0a54154330 2023-11-06T22:16:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.425+0000 7f0a52d9d700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0a4005f270 0x7f0a40061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.426+0000 7f0a52d9d700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0a4005f270 0x7f0a40061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f0a44005750 tx=0x7f0a44009040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.429+0000 7f0a3affd700 1 -- 172.21.15.33:0/2695623724 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0a4804b800 con 0x7f0a54154330 2023-11-06T22:16:41.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:41.656+0000 7f0a591ff700 1 -- 172.21.15.33:0/2695623724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0a540652d0 con 0x7f0a54154330 2023-11-06T22:16:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.187+0000 7f0a3affd700 1 -- 172.21.15.33:0/2695623724 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0a480160c0 con 0x7f0a54154330 2023-11-06T22:16:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.189+0000 7f0a591ff700 1 -- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0a4005f270 msgr2=0x7f0a40061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0a4005f270 0x7f0a40061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f0a44005750 tx=0x7f0a44009040).stop 2023-11-06T22:16:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 -- 172.21.15.33:0/2695623724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a54154330 msgr2=0x7f0a540c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a54154330 0x7f0a540c2eb0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f0a4800e8d0 tx=0x7f0a48009060).stop 2023-11-06T22:16:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 -- 172.21.15.33:0/2695623724 shutdown_connections 2023-11-06T22:16:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0a4005f270 0x7f0a40061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0a54154710 0x7f0a540c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a540c7ff0 0x7f0a54154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 --2- 172.21.15.33:0/2695623724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a54154330 0x7f0a540c2eb0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 -- 172.21.15.33:0/2695623724 >> 172.21.15.33:0/2695623724 conn(0x7f0a540cb910 msgr2=0x7f0a540aaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 -- 172.21.15.33:0/2695623724 shutdown_connections 2023-11-06T22:16:42.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.190+0000 7f0a591ff700 1 -- 172.21.15.33:0/2695623724 wait complete. 2023-11-06T22:16:42.212 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T22:16:42.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:16:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.410+0000 7f582e1ff700 1 Processor -- start 2023-11-06T22:16:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.410+0000 7f582e1ff700 1 -- start start 2023-11-06T22:16:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.411+0000 7f582e1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58280c1840 0x7f58280a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.411+0000 7f582e1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58280aa2d0 0x7f58280c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.411+0000 7f582e1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58281554a0 con 0x7f58280c6240 2023-11-06T22:16:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.411+0000 7f582e1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5828154960 con 0x7f58280aa2d0 2023-11-06T22:16:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.411+0000 7f582e1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5828155930 con 0x7f58280c1840 2023-11-06T22:16:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.411+0000 7f5826ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58280aa2d0 0x7f58280c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.411+0000 7f5826ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58280aa2d0 0x7f58280c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41814/0 (socket says 172.21.15.33:41814) 2023-11-06T22:16:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.411+0000 7f5826ffd700 1 -- 172.21.15.33:0/2402517129 learned_addr learned my addr 172.21.15.33:0/2402517129 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.412+0000 7f58277fe700 1 --2- 172.21.15.33:0/2402517129 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58280c1840 0x7f58280a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.412+0000 7f5826ffd700 1 -- 172.21.15.33:0/2402517129 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58280c1840 msgr2=0x7f58280a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.412+0000 7f5826ffd700 1 --2- 172.21.15.33:0/2402517129 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58280c1840 0x7f58280a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.412+0000 7f5826ffd700 1 -- 172.21.15.33:0/2402517129 >> v1:172.21.15.69:6789/0 conn(0x7f58280c6240 legacy=0x7f5828157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.412+0000 7f5826ffd700 1 -- 172.21.15.33:0/2402517129 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5828154b70 con 0x7f58280aa2d0 2023-11-06T22:16:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.412+0000 7f5826ffd700 1 --2- 172.21.15.33:0/2402517129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280aa2d0 0x7f58280c5d50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f581c006fd0 tx=0x7f581c0090f0).ready entity=mon.0 client_cookie=42b15b9c0927a0c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.412+0000 7f58267fc700 1 -- 172.21.15.33:0/2402517129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c00b600 con 0x7f58280aa2d0 2023-11-06T22:16:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.412+0000 7f58267fc700 1 -- 172.21.15.33:0/2402517129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c00b760 con 0x7f58280aa2d0 2023-11-06T22:16:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.412+0000 7f58267fc700 1 -- 172.21.15.33:0/2402517129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c00bae0 con 0x7f58280aa2d0 2023-11-06T22:16:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.413+0000 7f582e1ff700 1 -- 172.21.15.33:0/2402517129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280aa2d0 msgr2=0x7f58280c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.413+0000 7f582e1ff700 1 --2- 172.21.15.33:0/2402517129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280aa2d0 0x7f58280c5d50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f581c006fd0 tx=0x7f581c0090f0).stop 2023-11-06T22:16:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.413+0000 7f582e1ff700 1 -- 172.21.15.33:0/2402517129 shutdown_connections 2023-11-06T22:16:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.413+0000 7f582e1ff700 1 --2- 172.21.15.33:0/2402517129 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58280c1840 0x7f58280a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.413+0000 7f582e1ff700 1 --2- 172.21.15.33:0/2402517129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280aa2d0 0x7f58280c5d50 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.413+0000 7f582e1ff700 1 -- 172.21.15.33:0/2402517129 >> 172.21.15.33:0/2402517129 conn(0x7f582805ec80 msgr2=0x7f58280cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.413+0000 7f582e1ff700 1 -- 172.21.15.33:0/2402517129 shutdown_connections 2023-11-06T22:16:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.413+0000 7f582e1ff700 1 -- 172.21.15.33:0/2402517129 wait complete. 2023-11-06T22:16:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.414+0000 7f582e1ff700 1 Processor -- start 2023-11-06T22:16:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.415+0000 7f582e1ff700 1 -- start start 2023-11-06T22:16:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f582e1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58280aa2d0 0x7f582814ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f582e1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58280c1840 0x7f5828155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f582e1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280c6240 0x7f58280ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f582e1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58280c3890 con 0x7f58280c6240 2023-11-06T22:16:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f582e1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58280c2f60 con 0x7f58280aa2d0 2023-11-06T22:16:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f582e1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58280c3240 con 0x7f58280c1840 2023-11-06T22:16:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f5827fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280c6240 0x7f58280ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f5827fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280c6240 0x7f58280ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41818/0 (socket says 172.21.15.33:41818) 2023-11-06T22:16:42.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f5827fff700 1 -- 172.21.15.33:0/6552777 learned_addr learned my addr 172.21.15.33:0/6552777 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:42.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f5826ffd700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58280c1840 0x7f5828155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f58277fe700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58280aa2d0 0x7f582814ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f5827fff700 1 -- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58280c1840 msgr2=0x7f5828155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.416+0000 7f5827fff700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58280c1840 0x7f5828155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.417+0000 7f5827fff700 1 -- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58280aa2d0 msgr2=0x7f582814ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.417+0000 7f5827fff700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58280aa2d0 0x7f582814ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.417+0000 7f5827fff700 1 -- 172.21.15.33:0/6552777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c009370 con 0x7f58280c6240 2023-11-06T22:16:42.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.417+0000 7f5827fff700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280c6240 0x7f58280ae700 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f582000c7c0 tx=0x7f5820001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:42.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.417+0000 7f580f7fe700 1 -- 172.21.15.33:0/6552777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000e4a0 con 0x7f58280c6240 2023-11-06T22:16:42.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.417+0000 7f580f7fe700 1 -- 172.21.15.33:0/6552777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582000e600 con 0x7f58280c6240 2023-11-06T22:16:42.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.417+0000 7f582e1ff700 1 -- 172.21.15.33:0/6552777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58280c3520 con 0x7f58280c6240 2023-11-06T22:16:42.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.417+0000 7f580f7fe700 1 -- 172.21.15.33:0/6552777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000e980 con 0x7f58280c6240 2023-11-06T22:16:42.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.417+0000 7f582e1ff700 1 -- 172.21.15.33:0/6552777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58280aef30 con 0x7f58280c6240 2023-11-06T22:16:42.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.418+0000 7f582e1ff700 1 -- 172.21.15.33:0/6552777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58280c3a00 con 0x7f58280c6240 2023-11-06T22:16:42.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.419+0000 7f580f7fe700 1 -- 172.21.15.33:0/6552777 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f58200123f0 con 0x7f58280c6240 2023-11-06T22:16:42.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.420+0000 7f580f7fe700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f581405f270 0x7f5814061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.420+0000 7f580f7fe700 1 -- 172.21.15.33:0/6552777 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f5820079cb0 con 0x7f58280c6240 2023-11-06T22:16:42.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.420+0000 7f58277fe700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f581405f270 0x7f5814061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.421+0000 7f58277fe700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f581405f270 0x7f5814061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f581800bbc0 tx=0x7f5818008040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:42.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.424+0000 7f580f7fe700 1 -- 172.21.15.33:0/6552777 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5820047af0 con 0x7f58280c6240 2023-11-06T22:16:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.650+0000 7f582e1ff700 1 -- 172.21.15.33:0/6552777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f582806c6c0 con 0x7f58280c6240 2023-11-06T22:16:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.651+0000 7f580f7fe700 1 -- 172.21.15.33:0/6552777 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f582004acb0 con 0x7f58280c6240 2023-11-06T22:16:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:42.654 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3815920714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3815920714},{"type":"v1","addr":"172.21.15.69:6836","nonce":3815920714}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9052,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3614716748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3614716748},{"type":"v1","addr":"172.21.15.69:6837","nonce":3614716748}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1513187241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1513187241},{"type":"v1","addr":"172.21.15.33:6834","nonce":1513187241}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:16:39.145216+0000","modified":"2023-11-06T22:16:42.180077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3197264665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3197264665},{"type":"v1","addr":"172.21.15.33:6835","nonce":3197264665}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-06T22:16:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 -- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f581405f270 msgr2=0x7f5814061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f581405f270 0x7f5814061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f581800bbc0 tx=0x7f5818008040).stop 2023-11-06T22:16:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 -- 172.21.15.33:0/6552777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280c6240 msgr2=0x7f58280ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280c6240 0x7f58280ae700 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f582000c7c0 tx=0x7f5820001510).stop 2023-11-06T22:16:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 -- 172.21.15.33:0/6552777 shutdown_connections 2023-11-06T22:16:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f581405f270 0x7f5814061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58280c6240 0x7f58280ae700 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58280aa2d0 0x7f582814ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 --2- 172.21.15.33:0/6552777 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f58280c1840 0x7f5828155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 -- 172.21.15.33:0/6552777 >> 172.21.15.33:0/6552777 conn(0x7f582805ec80 msgr2=0x7f5828152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.653+0000 7f582e1ff700 1 -- 172.21.15.33:0/6552777 shutdown_connections 2023-11-06T22:16:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.654+0000 7f582e1ff700 1 -- 172.21.15.33:0/6552777 wait complete. 2023-11-06T22:16:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 84 2023-11-06T22:16:42.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:16:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.866+0000 7f3f7dbff700 1 Processor -- start 2023-11-06T22:16:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.867+0000 7f3f7dbff700 1 -- start start 2023-11-06T22:16:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.868+0000 7f3f7dbff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3f780c5240 0x7f3f780bdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.868+0000 7f3f7dbff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f780be510 con 0x7f3f780be060 2023-11-06T22:16:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.868+0000 7f3f7dbff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f781553b0 con 0x7f3f780c4e60 2023-11-06T22:16:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.868+0000 7f3f7dbff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f78066c50 con 0x7f3f780c5240 2023-11-06T22:16:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.868+0000 7f3f77fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3f780be060 0x7f3f780c35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58062/0 (socket says 172.21.15.33:58062) 2023-11-06T22:16:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.868+0000 7f3f77fff700 1 -- 172.21.15.33:0/1302533841 learned_addr learned my addr 172.21.15.33:0/1302533841 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.868+0000 7f3f76ffd700 1 --2- 172.21.15.33:0/1302533841 >> v2:172.21.15.69:3300/0 conn(0x7f3f780c5240 0x7f3f780bdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955135013 0 0) 0x7f3f70001490 con 0x7f3f780be060 2023-11-06T22:16:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f600033f0 con 0x7f3f780be060 2023-11-06T22:16:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2794748261 0 0) 0x7f3f600033f0 con 0x7f3f780be060 2023-11-06T22:16:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 >> v2:172.21.15.69:3300/0 conn(0x7f3f780c5240 msgr2=0x7f3f780bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 --2- 172.21.15.33:0/1302533841 >> v2:172.21.15.69:3300/0 conn(0x7f3f780c5240 0x7f3f780bdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 >> v1:172.21.15.69:6789/0 conn(0x7f3f780c4e60 legacy=0x7f3f78066540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f78154ac0 con 0x7f3f780be060 2023-11-06T22:16:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f3f70001980 con 0x7f3f780be060 2023-11-06T22:16:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 >> v1:172.21.15.33:6789/0 conn(0x7f3f780be060 legacy=0x7f3f780c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f600038e0 0x7f3f60003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60004690 0x7f3f60004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f60005000 0x7f3f6000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78066c50 con 0x7f3f60004690 2023-11-06T22:16:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6000be00 con 0x7f3f600038e0 2023-11-06T22:16:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6000bfc0 con 0x7f3f60005000 2023-11-06T22:16:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.869+0000 7f3f76ffd700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60004690 0x7f3f60004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f77fff700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f60005000 0x7f3f6000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f777fe700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f600038e0 0x7f3f60003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f76ffd700 1 -- 172.21.15.33:0/1302533841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f60005000 msgr2=0x7f3f6000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f76ffd700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f60005000 0x7f3f6000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f76ffd700 1 -- 172.21.15.33:0/1302533841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f600038e0 msgr2=0x7f3f60003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f76ffd700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f600038e0 0x7f3f60003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f76ffd700 1 -- 172.21.15.33:0/1302533841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f600033f0 con 0x7f3f60004690 2023-11-06T22:16:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f77fff700 1 -- 172.21.15.33:0/1302533841 reap_dead start 2023-11-06T22:16:42.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f76ffd700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60004690 0x7f3f60004b10 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f3f780c5680 tx=0x7f3f6c009060).ready entity=mon.0 client_cookie=973ea0d4c469e341 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:42.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c009e90 con 0x7f3f60004690 2023-11-06T22:16:42.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c009430 con 0x7f3f60004690 2023-11-06T22:16:42.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.870+0000 7f3f767fc700 1 -- 172.21.15.33:0/1302533841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00e850 con 0x7f3f60004690 2023-11-06T22:16:42.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.871+0000 7f3f7dbff700 1 -- 172.21.15.33:0/1302533841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60004690 msgr2=0x7f3f60004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.871+0000 7f3f7dbff700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60004690 0x7f3f60004b10 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f3f780c5680 tx=0x7f3f6c009060).stop 2023-11-06T22:16:42.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.871+0000 7f3f7dbff700 1 -- 172.21.15.33:0/1302533841 shutdown_connections 2023-11-06T22:16:42.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.871+0000 7f3f7dbff700 1 --2- 172.21.15.33:0/1302533841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60004690 0x7f3f60004b10 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.871+0000 7f3f7dbff700 1 -- 172.21.15.33:0/1302533841 >> 172.21.15.33:0/1302533841 conn(0x7f3f780c8c10 msgr2=0x7f3f780c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.871+0000 7f3f7dbff700 1 -- 172.21.15.33:0/1302533841 shutdown_connections 2023-11-06T22:16:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.871+0000 7f3f7dbff700 1 -- 172.21.15.33:0/1302533841 wait complete. 2023-11-06T22:16:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.872+0000 7f3f7dbff700 1 Processor -- start 2023-11-06T22:16:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.872+0000 7f3f7dbff700 1 -- start start 2023-11-06T22:16:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.873+0000 7f3f7dbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f60004690 0x7f3f781395b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.873+0000 7f3f7dbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f78139aa0 0x7f3f78157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.873+0000 7f3f7dbff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f78155c30 0x7f3f781583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.873+0000 7f3f7dbff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f781588a0 con 0x7f3f78139aa0 2023-11-06T22:16:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.873+0000 7f3f7dbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f780ad580 con 0x7f3f60004690 2023-11-06T22:16:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.873+0000 7f3f7dbff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f780bfad0 con 0x7f3f78155c30 2023-11-06T22:16:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f76ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f78139aa0 0x7f3f78157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f76ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f78139aa0 0x7f3f78157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41836/0 (socket says 172.21.15.33:41836) 2023-11-06T22:16:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f76ffd700 1 -- 172.21.15.33:0/2239151047 learned_addr learned my addr 172.21.15.33:0/2239151047 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:42.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f77fff700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f78155c30 0x7f3f781583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f777fe700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f60004690 0x7f3f781395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f76ffd700 1 -- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f78155c30 msgr2=0x7f3f781583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f76ffd700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f78155c30 0x7f3f781583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f76ffd700 1 -- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f60004690 msgr2=0x7f3f781395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:42.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f76ffd700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f60004690 0x7f3f781395b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:42.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f76ffd700 1 -- 172.21.15.33:0/2239151047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f780a5be0 con 0x7f3f78139aa0 2023-11-06T22:16:42.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.874+0000 7f3f76ffd700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f78139aa0 0x7f3f78157ec0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f3f6c006d50 tx=0x7f3f6c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:42.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.875+0000 7f3f5f7fe700 1 -- 172.21.15.33:0/2239151047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c01c810 con 0x7f3f78139aa0 2023-11-06T22:16:42.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.875+0000 7f3f5f7fe700 1 -- 172.21.15.33:0/2239151047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c01c970 con 0x7f3f78139aa0 2023-11-06T22:16:42.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.875+0000 7f3f7dbff700 1 -- 172.21.15.33:0/2239151047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f780a5ec0 con 0x7f3f78139aa0 2023-11-06T22:16:42.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.875+0000 7f3f5f7fe700 1 -- 172.21.15.33:0/2239151047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c01cd30 con 0x7f3f78139aa0 2023-11-06T22:16:42.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.875+0000 7f3f7dbff700 1 -- 172.21.15.33:0/2239151047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f780ad210 con 0x7f3f78139aa0 2023-11-06T22:16:42.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.876+0000 7f3f7dbff700 1 -- 172.21.15.33:0/2239151047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f780ad070 con 0x7f3f78139aa0 2023-11-06T22:16:42.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.877+0000 7f3f5f7fe700 1 -- 172.21.15.33:0/2239151047 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3f6c02e440 con 0x7f3f78139aa0 2023-11-06T22:16:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.877+0000 7f3f5f7fe700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3f6405f270 0x7f3f64061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.878+0000 7f3f5f7fe700 1 -- 172.21.15.33:0/2239151047 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f3f6c07bd60 con 0x7f3f78139aa0 2023-11-06T22:16:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.878+0000 7f3f777fe700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3f6405f270 0x7f3f64061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.879+0000 7f3f777fe700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3f6405f270 0x7f3f64061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f3f68000c00 tx=0x7f3f68009040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:42.882+0000 7f3f5f7fe700 1 -- 172.21.15.33:0/2239151047 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3f780ad070 con 0x7f3f78139aa0 2023-11-06T22:16:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.064+0000 7f3f7dbff700 1 -- 172.21.15.33:0/2239151047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f780ad070 con 0x7f3f78139aa0 2023-11-06T22:16:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.065+0000 7f3f5f7fe700 1 -- 172.21.15.33:0/2239151047 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v204) v1 ==== 74+0+17061 (secure 0 0 0) 0x7f3f780ad070 con 0x7f3f78139aa0 2023-11-06T22:16:43.068 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:43.068 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":204,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:16:40.147893+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:16:37.603904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T22:16:38.139152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:16:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 -- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3f6405f270 msgr2=0x7f3f64061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3f6405f270 0x7f3f64061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f3f68000c00 tx=0x7f3f68009040).stop 2023-11-06T22:16:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 -- 172.21.15.33:0/2239151047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f78139aa0 msgr2=0x7f3f78157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f78139aa0 0x7f3f78157ec0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f3f6c006d50 tx=0x7f3f6c01a000).stop 2023-11-06T22:16:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 -- 172.21.15.33:0/2239151047 shutdown_connections 2023-11-06T22:16:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3f6405f270 0x7f3f64061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3f78155c30 0x7f3f781583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f60004690 0x7f3f781395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 --2- 172.21.15.33:0/2239151047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f78139aa0 0x7f3f78157ec0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 -- 172.21.15.33:0/2239151047 >> 172.21.15.33:0/2239151047 conn(0x7f3f780c8c10 msgr2=0x7f3f78063850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 -- 172.21.15.33:0/2239151047 shutdown_connections 2023-11-06T22:16:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.067+0000 7f3f7dbff700 1 -- 172.21.15.33:0/2239151047 wait complete. 2023-11-06T22:16:43.088 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T22:16:43.088 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:16:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.285+0000 7ffb5b5ff700 1 Processor -- start 2023-11-06T22:16:43.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.286+0000 7ffb5b5ff700 1 -- start start 2023-11-06T22:16:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.287+0000 7ffb5b5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ffb54154110 0x7ffb541544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.287+0000 7ffb5b5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb54064ce0 0x7ffb540c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.287+0000 7ffb5b5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb540c0880 con 0x7ffb540aade0 2023-11-06T22:16:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.287+0000 7ffb5b5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb540bfd40 con 0x7ffb54064ce0 2023-11-06T22:16:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.287+0000 7ffb5b5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb540c8260 con 0x7ffb54154110 2023-11-06T22:16:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.287+0000 7ffb59b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb54064ce0 0x7ffb540c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.287+0000 7ffb59b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb54064ce0 0x7ffb540c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41844/0 (socket says 172.21.15.33:41844) 2023-11-06T22:16:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.287+0000 7ffb59b9c700 1 -- 172.21.15.33:0/4093461203 learned_addr learned my addr 172.21.15.33:0/4093461203 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.287+0000 7ffb5939b700 1 --2- 172.21.15.33:0/4093461203 >> v2:172.21.15.69:3300/0 conn(0x7ffb54154110 0x7ffb541544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.288+0000 7ffb59b9c700 1 -- 172.21.15.33:0/4093461203 >> v2:172.21.15.69:3300/0 conn(0x7ffb54154110 msgr2=0x7ffb541544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.288+0000 7ffb59b9c700 1 --2- 172.21.15.33:0/4093461203 >> v2:172.21.15.69:3300/0 conn(0x7ffb54154110 0x7ffb541544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.288+0000 7ffb59b9c700 1 -- 172.21.15.33:0/4093461203 >> v1:172.21.15.69:6789/0 conn(0x7ffb540aade0 legacy=0x7ffb540645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.288+0000 7ffb59b9c700 1 -- 172.21.15.33:0/4093461203 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb540bff50 con 0x7ffb54064ce0 2023-11-06T22:16:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.288+0000 7ffb5939b700 1 --2- 172.21.15.33:0/4093461203 >> v2:172.21.15.69:3300/0 conn(0x7ffb54154110 0x7ffb541544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.288+0000 7ffb59b9c700 1 --2- 172.21.15.33:0/4093461203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 0x7ffb540c5db0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7ffb5406c130 tx=0x7ffb4c009840).ready entity=mon.0 client_cookie=b59b3500eaff9cc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.288+0000 7ffb4bfff700 1 -- 172.21.15.33:0/4093461203 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c00e070 con 0x7ffb54064ce0 2023-11-06T22:16:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.288+0000 7ffb4bfff700 1 -- 172.21.15.33:0/4093461203 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c00ba50 con 0x7ffb54064ce0 2023-11-06T22:16:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.288+0000 7ffb4bfff700 1 -- 172.21.15.33:0/4093461203 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c00bdd0 con 0x7ffb54064ce0 2023-11-06T22:16:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.289+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4093461203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 msgr2=0x7ffb540c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.289+0000 7ffb5b5ff700 1 --2- 172.21.15.33:0/4093461203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 0x7ffb540c5db0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7ffb5406c130 tx=0x7ffb4c009840).stop 2023-11-06T22:16:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.289+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4093461203 shutdown_connections 2023-11-06T22:16:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.289+0000 7ffb5b5ff700 1 --2- 172.21.15.33:0/4093461203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 0x7ffb540c5db0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.289+0000 7ffb5b5ff700 1 --2- 172.21.15.33:0/4093461203 >> v2:172.21.15.69:3300/0 conn(0x7ffb54154110 0x7ffb541544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.289+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4093461203 >> 172.21.15.33:0/4093461203 conn(0x7ffb540cb8b0 msgr2=0x7ffb540c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.289+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4093461203 shutdown_connections 2023-11-06T22:16:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.289+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4093461203 wait complete. 2023-11-06T22:16:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.290+0000 7ffb5b5ff700 1 Processor -- start 2023-11-06T22:16:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.291+0000 7ffb5b5ff700 1 -- start start 2023-11-06T22:16:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.291+0000 7ffb5b5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 0x7ffb540c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5b5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb540aade0 0x7ffb54142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5b5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb54154110 0x7ffb54167960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5b5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540afd50 con 0x7ffb54064ce0 2023-11-06T22:16:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5b5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540af140 con 0x7ffb540aade0 2023-11-06T22:16:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5b5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540af420 con 0x7ffb54154110 2023-11-06T22:16:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 0x7ffb540c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 0x7ffb540c3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41856/0 (socket says 172.21.15.33:41856) 2023-11-06T22:16:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5939b700 1 -- 172.21.15.33:0/4072586916 learned_addr learned my addr 172.21.15.33:0/4072586916 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5939b700 1 -- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb54154110 msgr2=0x7ffb54167960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb59b9c700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb54154110 0x7ffb54167960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb58b9a700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb540aade0 0x7ffb54142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5939b700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb54154110 0x7ffb54167960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5939b700 1 -- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb540aade0 msgr2=0x7ffb54142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5939b700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb540aade0 0x7ffb54142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5939b700 1 -- 172.21.15.33:0/4072586916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c009ae0 con 0x7ffb54064ce0 2023-11-06T22:16:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb59b9c700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb54154110 0x7ffb54167960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb58b9a700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb540aade0 0x7ffb54142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.292+0000 7ffb5939b700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 0x7ffb540c3df0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7ffb5000d890 tx=0x7ffb50008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.293+0000 7ffb497fa700 1 -- 172.21.15.33:0/4072586916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500125f0 con 0x7ffb54064ce0 2023-11-06T22:16:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.293+0000 7ffb497fa700 1 -- 172.21.15.33:0/4072586916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb50012750 con 0x7ffb54064ce0 2023-11-06T22:16:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.293+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4072586916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb540af700 con 0x7ffb54064ce0 2023-11-06T22:16:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.293+0000 7ffb497fa700 1 -- 172.21.15.33:0/4072586916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50012ad0 con 0x7ffb54064ce0 2023-11-06T22:16:43.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.293+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4072586916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb54142c80 con 0x7ffb54064ce0 2023-11-06T22:16:43.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.294+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4072586916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb540c6700 con 0x7ffb54064ce0 2023-11-06T22:16:43.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.295+0000 7ffb497fa700 1 -- 172.21.15.33:0/4072586916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ffb50013480 con 0x7ffb54064ce0 2023-11-06T22:16:43.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.296+0000 7ffb497fa700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb3805f270 0x7ffb38061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.296+0000 7ffb497fa700 1 -- 172.21.15.33:0/4072586916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7ffb5007a240 con 0x7ffb54064ce0 2023-11-06T22:16:43.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.297+0000 7ffb58b9a700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb3805f270 0x7ffb38061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.297+0000 7ffb58b9a700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb3805f270 0x7ffb38061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7ffb540c4b30 tx=0x7ffb44009040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:43.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.300+0000 7ffb497fa700 1 -- 172.21.15.33:0/4072586916 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffb5004b240 con 0x7ffb54064ce0 2023-11-06T22:16:43.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.478+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4072586916 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb540afe90 con 0x7ffb3805f270 2023-11-06T22:16:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.491+0000 7ffb497fa700 1 -- 172.21.15.33:0/4072586916 <== mgr.7733 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+352229 (secure 0 0 0) 0x7ffb44001360 con 0x7ffb3805f270 2023-11-06T22:16:43.498 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.495+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb3805f270 msgr2=0x7ffb38061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.495+0000 7ffb5b5ff700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb3805f270 0x7ffb38061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7ffb540c4b30 tx=0x7ffb44009040).stop 2023-11-06T22:16:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.495+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4072586916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 msgr2=0x7ffb540c3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.495+0000 7ffb5b5ff700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 0x7ffb540c3df0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7ffb5000d890 tx=0x7ffb50008040).stop 2023-11-06T22:16:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.496+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4072586916 shutdown_connections 2023-11-06T22:16:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.496+0000 7ffb5b5ff700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffb3805f270 0x7ffb38061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.496+0000 7ffb5b5ff700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffb54154110 0x7ffb54167960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.496+0000 7ffb5b5ff700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb54064ce0 0x7ffb540c3df0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.496+0000 7ffb5b5ff700 1 --2- 172.21.15.33:0/4072586916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb540aade0 0x7ffb54142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.496+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4072586916 >> 172.21.15.33:0/4072586916 conn(0x7ffb540cb8b0 msgr2=0x7ffb54062ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.496+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4072586916 shutdown_connections 2023-11-06T22:16:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.496+0000 7ffb5b5ff700 1 -- 172.21.15.33:0/4072586916 wait complete. 2023-11-06T22:16:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:16:43.516 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T22:16:41.570530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":926,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":62,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2457836,"kb_used_data":19628,"kb_used_omap":0,"kb_used_meta":2438144,"kb_avail":752516884,"statfs":{"total":773094113280,"available":770577289216,"internally_reserved":0,"allocated":20099072,"data_stored":10305406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2496659456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-594056,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":-209,"num_write_kb":-719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.650632"},"pg_stats":[{"pgid":"24.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182097+0000","last_change":"2023-11-06T22:16:39.177666+0000","last_active":"2023-11-06T22:16:39.182097+0000","last_peered":"2023-11-06T22:16:39.182097+0000","last_clean":"2023-11-06T22:16:39.182097+0000","last_became_active":"2023-11-06T22:16:39.177358+0000","last_became_peered":"2023-11-06T22:16:39.177358+0000","last_unstale":"2023-11-06T22:16:39.182097+0000","last_undegraded":"2023-11-06T22:16:39.182097+0000","last_fullsized":"2023-11-06T22:16:39.182097+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"203'4","reported_seq":14,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.192449+0000","last_change":"2023-11-06T22:16:38.691776+0000","last_active":"2023-11-06T22:16:39.192449+0000","last_peered":"2023-11-06T22:16:39.192449+0000","last_clean":"2023-11-06T22:16:39.192449+0000","last_became_active":"2023-11-06T22:16:38.691516+0000","last_became_peered":"2023-11-06T22:16:38.691516+0000","last_unstale":"2023-11-06T22:16:39.192449+0000","last_undegraded":"2023-11-06T22:16:39.192449+0000","last_fullsized":"2023-11-06T22:16:39.192449+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.173440+0000","last_change":"2023-11-06T22:16:38.678970+0000","last_active":"2023-11-06T22:16:39.173440+0000","last_peered":"2023-11-06T22:16:39.173440+0000","last_clean":"2023-11-06T22:16:39.173440+0000","last_became_active":"2023-11-06T22:16:38.678575+0000","last_became_peered":"2023-11-06T22:16:38.678575+0000","last_unstale":"2023-11-06T22:16:39.173440+0000","last_undegraded":"2023-11-06T22:16:39.173440+0000","last_fullsized":"2023-11-06T22:16:39.173440+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182828+0000","last_change":"2023-11-06T22:16:39.163644+0000","last_active":"2023-11-06T22:16:39.182828+0000","last_peered":"2023-11-06T22:16:39.182828+0000","last_clean":"2023-11-06T22:16:39.182828+0000","last_became_active":"2023-11-06T22:16:39.163320+0000","last_became_peered":"2023-11-06T22:16:39.163320+0000","last_unstale":"2023-11-06T22:16:39.182828+0000","last_undegraded":"2023-11-06T22:16:39.182828+0000","last_fullsized":"2023-11-06T22:16:39.182828+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.153745+0000","last_change":"2023-11-06T22:16:38.677947+0000","last_active":"2023-11-06T22:16:39.153745+0000","last_peered":"2023-11-06T22:16:39.153745+0000","last_clean":"2023-11-06T22:16:39.153745+0000","last_became_active":"2023-11-06T22:16:38.677660+0000","last_became_peered":"2023-11-06T22:16:38.677660+0000","last_unstale":"2023-11-06T22:16:39.153745+0000","last_undegraded":"2023-11-06T22:16:39.153745+0000","last_fullsized":"2023-11-06T22:16:39.153745+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.184047+0000","last_change":"2023-11-06T22:16:39.182550+0000","last_active":"2023-11-06T22:16:39.184047+0000","last_peered":"2023-11-06T22:16:39.184047+0000","last_clean":"2023-11-06T22:16:39.184047+0000","last_became_active":"2023-11-06T22:16:39.182257+0000","last_became_peered":"2023-11-06T22:16:39.182257+0000","last_unstale":"2023-11-06T22:16:39.184047+0000","last_undegraded":"2023-11-06T22:16:39.184047+0000","last_fullsized":"2023-11-06T22:16:39.184047+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.164660+0000","last_change":"2023-11-06T22:16:38.678374+0000","last_active":"2023-11-06T22:16:39.164660+0000","last_peered":"2023-11-06T22:16:39.164660+0000","last_clean":"2023-11-06T22:16:39.164660+0000","last_became_active":"2023-11-06T22:16:38.678085+0000","last_became_peered":"2023-11-06T22:16:38.678085+0000","last_unstale":"2023-11-06T22:16:39.164660+0000","last_undegraded":"2023-11-06T22:16:39.164660+0000","last_fullsized":"2023-11-06T22:16:39.164660+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.167510+0000","last_change":"2023-11-06T22:16:38.686680+0000","last_active":"2023-11-06T22:16:39.167510+0000","last_peered":"2023-11-06T22:16:39.167510+0000","last_clean":"2023-11-06T22:16:39.167510+0000","last_became_active":"2023-11-06T22:16:38.682448+0000","last_became_peered":"2023-11-06T22:16:38.682448+0000","last_unstale":"2023-11-06T22:16:39.167510+0000","last_undegraded":"2023-11-06T22:16:39.167510+0000","last_fullsized":"2023-11-06T22:16:39.167510+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.187540+0000","last_change":"2023-11-06T22:16:39.180008+0000","last_active":"2023-11-06T22:16:39.187540+0000","last_peered":"2023-11-06T22:16:39.187540+0000","last_clean":"2023-11-06T22:16:39.187540+0000","last_became_active":"2023-11-06T22:16:39.178723+0000","last_became_peered":"2023-11-06T22:16:39.178723+0000","last_unstale":"2023-11-06T22:16:39.187540+0000","last_undegraded":"2023-11-06T22:16:39.187540+0000","last_fullsized":"2023-11-06T22:16:39.187540+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.186849+0000","last_change":"2023-11-06T22:16:39.176565+0000","last_active":"2023-11-06T22:16:39.186849+0000","last_peered":"2023-11-06T22:16:39.186849+0000","last_clean":"2023-11-06T22:16:39.186849+0000","last_became_active":"2023-11-06T22:16:39.175687+0000","last_became_peered":"2023-11-06T22:16:39.175687+0000","last_unstale":"2023-11-06T22:16:39.186849+0000","last_undegraded":"2023-11-06T22:16:39.186849+0000","last_fullsized":"2023-11-06T22:16:39.186849+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.163594+0000","last_change":"2023-11-06T22:16:38.675389+0000","last_active":"2023-11-06T22:16:39.163594+0000","last_peered":"2023-11-06T22:16:39.163594+0000","last_clean":"2023-11-06T22:16:39.163594+0000","last_became_active":"2023-11-06T22:16:38.675034+0000","last_became_peered":"2023-11-06T22:16:38.675034+0000","last_unstale":"2023-11-06T22:16:39.163594+0000","last_undegraded":"2023-11-06T22:16:39.163594+0000","last_fullsized":"2023-11-06T22:16:39.163594+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.170796+0000","last_change":"2023-11-06T22:16:38.687430+0000","last_active":"2023-11-06T22:16:39.170796+0000","last_peered":"2023-11-06T22:16:39.170796+0000","last_clean":"2023-11-06T22:16:39.170796+0000","last_became_active":"2023-11-06T22:16:38.687129+0000","last_became_peered":"2023-11-06T22:16:38.687129+0000","last_unstale":"2023-11-06T22:16:39.170796+0000","last_undegraded":"2023-11-06T22:16:39.170796+0000","last_fullsized":"2023-11-06T22:16:39.170796+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.b","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.183202+0000","last_change":"2023-11-06T22:16:38.691051+0000","last_active":"2023-11-06T22:16:39.183202+0000","last_peered":"2023-11-06T22:16:39.183202+0000","last_clean":"2023-11-06T22:16:39.183202+0000","last_became_active":"2023-11-06T22:16:38.690525+0000","last_became_peered":"2023-11-06T22:16:38.690525+0000","last_unstale":"2023-11-06T22:16:39.183202+0000","last_undegraded":"2023-11-06T22:16:39.183202+0000","last_fullsized":"2023-11-06T22:16:39.183202+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.185236+0000","last_change":"2023-11-06T22:16:39.183192+0000","last_active":"2023-11-06T22:16:39.185236+0000","last_peered":"2023-11-06T22:16:39.185236+0000","last_clean":"2023-11-06T22:16:39.185236+0000","last_became_active":"2023-11-06T22:16:39.182818+0000","last_became_peered":"2023-11-06T22:16:39.182818+0000","last_unstale":"2023-11-06T22:16:39.185236+0000","last_undegraded":"2023-11-06T22:16:39.185236+0000","last_fullsized":"2023-11-06T22:16:39.185236+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.184027+0000","last_change":"2023-11-06T22:16:39.182180+0000","last_active":"2023-11-06T22:16:39.184027+0000","last_peered":"2023-11-06T22:16:39.184027+0000","last_clean":"2023-11-06T22:16:39.184027+0000","last_became_active":"2023-11-06T22:16:39.181886+0000","last_became_peered":"2023-11-06T22:16:39.181886+0000","last_unstale":"2023-11-06T22:16:39.184027+0000","last_undegraded":"2023-11-06T22:16:39.184027+0000","last_fullsized":"2023-11-06T22:16:39.184027+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.183539+0000","last_change":"2023-11-06T22:16:39.166103+0000","last_active":"2023-11-06T22:16:39.183539+0000","last_peered":"2023-11-06T22:16:39.183539+0000","last_clean":"2023-11-06T22:16:39.183539+0000","last_became_active":"2023-11-06T22:16:39.165765+0000","last_became_peered":"2023-11-06T22:16:39.165765+0000","last_unstale":"2023-11-06T22:16:39.183539+0000","last_undegraded":"2023-11-06T22:16:39.183539+0000","last_fullsized":"2023-11-06T22:16:39.183539+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.186807+0000","last_change":"2023-11-06T22:16:39.176283+0000","last_active":"2023-11-06T22:16:39.186807+0000","last_peered":"2023-11-06T22:16:39.186807+0000","last_clean":"2023-11-06T22:16:39.186807+0000","last_became_active":"2023-11-06T22:16:39.175905+0000","last_became_peered":"2023-11-06T22:16:39.175905+0000","last_unstale":"2023-11-06T22:16:39.186807+0000","last_undegraded":"2023-11-06T22:16:39.186807+0000","last_fullsized":"2023-11-06T22:16:39.186807+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.167607+0000","last_change":"2023-11-06T22:16:38.690230+0000","last_active":"2023-11-06T22:16:39.167607+0000","last_peered":"2023-11-06T22:16:39.167607+0000","last_clean":"2023-11-06T22:16:39.167607+0000","last_became_active":"2023-11-06T22:16:38.681801+0000","last_became_peered":"2023-11-06T22:16:38.681801+0000","last_unstale":"2023-11-06T22:16:39.167607+0000","last_undegraded":"2023-11-06T22:16:39.167607+0000","last_fullsized":"2023-11-06T22:16:39.167607+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182564+0000","last_change":"2023-11-06T22:16:39.164031+0000","last_active":"2023-11-06T22:16:39.182564+0000","last_peered":"2023-11-06T22:16:39.182564+0000","last_clean":"2023-11-06T22:16:39.182564+0000","last_became_active":"2023-11-06T22:16:39.163724+0000","last_became_peered":"2023-11-06T22:16:39.163724+0000","last_unstale":"2023-11-06T22:16:39.182564+0000","last_undegraded":"2023-11-06T22:16:39.182564+0000","last_fullsized":"2023-11-06T22:16:39.182564+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.187580+0000","last_change":"2023-11-06T22:16:39.183935+0000","last_active":"2023-11-06T22:16:39.187580+0000","last_peered":"2023-11-06T22:16:39.187580+0000","last_clean":"2023-11-06T22:16:39.187580+0000","last_became_active":"2023-11-06T22:16:39.183660+0000","last_became_peered":"2023-11-06T22:16:39.183660+0000","last_unstale":"2023-11-06T22:16:39.187580+0000","last_undegraded":"2023-11-06T22:16:39.187580+0000","last_fullsized":"2023-11-06T22:16:39.187580+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":173,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.164790+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T22:16:39.164790+0000","last_peered":"2023-11-06T22:16:39.164790+0000","last_clean":"2023-11-06T22:16:39.164790+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:16:39.164790+0000","last_undegraded":"2023-11-06T22:16:39.164790+0000","last_fullsized":"2023-11-06T22:16:39.164790+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":184,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.152940+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T22:16:39.152940+0000","last_peered":"2023-11-06T22:16:39.152940+0000","last_clean":"2023-11-06T22:16:39.152940+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:16:39.152940+0000","last_undegraded":"2023-11-06T22:16:39.152940+0000","last_fullsized":"2023-11-06T22:16:39.152940+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":188,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-11-06T22:16:33.925047+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T22:16:33.925047+0000","last_peered":"2023-11-06T22:16:33.925047+0000","last_clean":"2023-11-06T22:16:33.925047+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:16:33.925047+0000","last_undegraded":"2023-11-06T22:16:33.925047+0000","last_fullsized":"2023-11-06T22:16:33.925047+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":184,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.153226+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T22:16:39.153226+0000","last_peered":"2023-11-06T22:16:39.153226+0000","last_clean":"2023-11-06T22:16:39.153226+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:16:39.153226+0000","last_undegraded":"2023-11-06T22:16:39.153226+0000","last_fullsized":"2023-11-06T22:16:39.153226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":168,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-11-06T22:16:33.928236+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T22:16:33.928236+0000","last_peered":"2023-11-06T22:16:33.928236+0000","last_clean":"2023-11-06T22:16:33.928236+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:16:33.928236+0000","last_undegraded":"2023-11-06T22:16:33.928236+0000","last_fullsized":"2023-11-06T22:16:33.928236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":184,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.152821+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T22:16:39.152821+0000","last_peered":"2023-11-06T22:16:39.152821+0000","last_clean":"2023-11-06T22:16:39.152821+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:16:39.152821+0000","last_undegraded":"2023-11-06T22:16:39.152821+0000","last_fullsized":"2023-11-06T22:16:39.152821+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182007+0000","last_change":"2023-11-06T22:16:39.177469+0000","last_active":"2023-11-06T22:16:39.182007+0000","last_peered":"2023-11-06T22:16:39.182007+0000","last_clean":"2023-11-06T22:16:39.182007+0000","last_became_active":"2023-11-06T22:16:39.177167+0000","last_became_peered":"2023-11-06T22:16:39.177167+0000","last_unstale":"2023-11-06T22:16:39.182007+0000","last_undegraded":"2023-11-06T22:16:39.182007+0000","last_fullsized":"2023-11-06T22:16:39.182007+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":184,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.153156+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:16:39.153156+0000","last_peered":"2023-11-06T22:16:39.153156+0000","last_clean":"2023-11-06T22:16:39.153156+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:16:39.153156+0000","last_undegraded":"2023-11-06T22:16:39.153156+0000","last_fullsized":"2023-11-06T22:16:39.153156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.13","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.185455+0000","last_change":"2023-11-06T22:16:38.673087+0000","last_active":"2023-11-06T22:16:39.185455+0000","last_peered":"2023-11-06T22:16:39.185455+0000","last_clean":"2023-11-06T22:16:39.185455+0000","last_became_active":"2023-11-06T22:16:38.672786+0000","last_became_peered":"2023-11-06T22:16:38.672786+0000","last_unstale":"2023-11-06T22:16:39.185455+0000","last_undegraded":"2023-11-06T22:16:39.185455+0000","last_fullsized":"2023-11-06T22:16:39.185455+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-11-06T22:16:34.367616+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T22:16:34.367616+0000","last_peered":"2023-11-06T22:16:34.367616+0000","last_clean":"2023-11-06T22:16:34.367616+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:16:34.367616+0000","last_undegraded":"2023-11-06T22:16:34.367616+0000","last_fullsized":"2023-11-06T22:16:34.367616+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:38.656491+0000","last_change":"2023-11-06T22:16:38.656491+0000","last_active":"2023-11-06T22:16:38.656491+0000","last_peered":"2023-11-06T22:16:38.656491+0000","last_clean":"2023-11-06T22:16:38.656491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:38.656491+0000","last_undegraded":"2023-11-06T22:16:38.656491+0000","last_fullsized":"2023-11-06T22:16:38.656491+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T22:16:37.654116+0000","last_change":"2023-11-06T22:16:37.654116+0000","last_active":"2023-11-06T22:16:37.654116+0000","last_peered":"2023-11-06T22:16:37.654116+0000","last_clean":"2023-11-06T22:16:37.654116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T22:16:37.654116+0000","last_undegraded":"2023-11-06T22:16:37.654116+0000","last_fullsized":"2023-11-06T22:16:37.654116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":184,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.155730+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:16:39.155730+0000","last_peered":"2023-11-06T22:16:39.155730+0000","last_clean":"2023-11-06T22:16:39.155730+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:16:39.155730+0000","last_undegraded":"2023-11-06T22:16:39.155730+0000","last_fullsized":"2023-11-06T22:16:39.155730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":926,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":20,"acting":20,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307728,"kb_used_data":2440,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064112,"statfs":{"total":96636764160,"available":96321650688,"internally_reserved":0,"allocated":2498560,"data_stored":1281316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.71999999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90700000000000003,"5min":0.97199999999999998,"15min":1.0980000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71499999999999997,"15min":0.72999999999999998},"min":{"1min":0.47599999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.49199999999999999}]},{"osd":1,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70599999999999996,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.48699999999999999,"15min":0.46400000000000002},"max":{"1min":0.90300000000000002,"5min":0.98799999999999999,"15min":1.038},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93200000000000005,"5min":0.96599999999999997,"15min":1.0680000000000001},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.61699999999999999,"5min":0.53100000000000003,"15min":0.502},"max":{"1min":0.98799999999999999,"5min":1.0069999999999999,"15min":1.016},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.52700000000000002,"5min":0.50900000000000001,"15min":0.505},"max":{"1min":0.996,"5min":0.996,"15min":1.0209999999999999},"last":0.42999999999999999}]},{"osd":3,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73899999999999999,"15min":0.74099999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.499},"max":{"1min":0.89600000000000002,"5min":0.95799999999999996,"15min":0.96999999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.76400000000000001,"15min":0.76900000000000002},"min":{"1min":0.58999999999999997,"5min":0.504,"15min":0.504},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.0229999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.40899999999999997},"max":{"1min":0.88300000000000001,"5min":0.94399999999999995,"15min":0.97199999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.47699999999999998,"5min":0.45200000000000001,"15min":0.42099999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61599999999999999}]},{"osd":5,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94099999999999995,"5min":1.008,"15min":1.0389999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.70799999999999996,"15min":0.68700000000000006},"min":{"1min":0.501,"5min":0.432,"15min":0.432},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.48399999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.55000000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.069},"last":0.55700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307728,"kb_used_data":2440,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064112,"statfs":{"total":96636764160,"available":96321650688,"internally_reserved":0,"allocated":2498560,"data_stored":1281316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.56599999999999995,"5min":0.501,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.96199999999999997,"15min":1.0569999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.54200000000000004,"5min":0.51800000000000002,"15min":0.434},"max":{"1min":0.87,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.749,"15min":0.73899999999999999},"min":{"1min":0.54600000000000004,"5min":0.44,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":1.0660000000000001,"15min":1.0700000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.45000000000000001},"max":{"1min":0.879,"5min":1.016,"15min":1.0960000000000001},"last":0.67600000000000005}]},{"osd":2,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.59999999999999998,"5min":0.498,"15min":0.498},"max":{"1min":0.94699999999999995,"5min":0.97999999999999998,"15min":1.167},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79400000000000004,"15min":0.79500000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.47099999999999997},"max":{"1min":0.93899999999999995,"5min":1.046,"15min":1.1839999999999999},"last":0.78300000000000003}]},{"osd":3,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.41599999999999998},"max":{"1min":0.91500000000000004,"5min":0.97799999999999998,"15min":1.002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.51700000000000002},"max":{"1min":0.89700000000000002,"5min":0.93500000000000005,"15min":1.0309999999999999},"last":0.80500000000000005}]},{"osd":4,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.50600000000000001,"5min":0.42299999999999999,"15min":0.41599999999999998},"max":{"1min":0.85399999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.51600000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98999999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.442},"max":{"1min":0.996,"5min":0.996,"15min":1.056},"last":0.82099999999999995}]},{"osd":7,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96199999999999997,"5min":1.012,"15min":1.0429999999999999},"last":0.749},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98099999999999998,"5min":1.0569999999999999,"15min":1.1279999999999999},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307728,"kb_used_data":2440,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064112,"statfs":{"total":96636764160,"available":96321650688,"internally_reserved":0,"allocated":2498560,"data_stored":1281316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46000000000000002,"15min":0.46500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":0.52900000000000003,"5min":0.748,"15min":0.97999999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.35199999999999998,"15min":0.33800000000000002},"max":{"1min":0.65900000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51800000000000002,"15min":0.51200000000000001},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.33800000000000002},"max":{"1min":0.77100000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.373},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.627,"15min":0.62},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41199999999999998},"max":{"1min":0.79900000000000004,"5min":0.95699999999999996,"15min":1.0189999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":1.0509999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.39200000000000002},"max":{"1min":0.79900000000000004,"5min":0.87,"15min":0.94999999999999996},"last":0.495}]},{"osd":4,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.45200000000000001},"max":{"1min":1.0649999999999999,"5min":1.155,"15min":1.218},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.84899999999999998,"15min":0.85599999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.59499999999999997},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82799999999999996,"15min":0.81999999999999995},"min":{"1min":0.63200000000000001,"5min":0.60699999999999998,"15min":0.57399999999999995},"max":{"1min":1.2210000000000001,"5min":1.2669999999999999,"15min":1.2709999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.61699999999999999,"5min":0.58199999999999996,"15min":0.55400000000000005},"max":{"1min":1.1160000000000001,"5min":1.2210000000000001,"15min":1.3109999999999999},"last":0.64200000000000002}]},{"osd":6,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.83399999999999996,"15min":0.84099999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1299999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84799999999999998,"15min":0.85099999999999998},"min":{"1min":0.64400000000000002,"5min":0.61699999999999999,"15min":0.59399999999999997},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.60399999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.87,"15min":0.876},"min":{"1min":0.71299999999999997,"5min":0.61899999999999999,"15min":0.60899999999999999},"max":{"1min":1.089,"5min":1.21,"15min":1.2549999999999999},"last":0.621},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.66400000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.2509999999999999},"last":0.67900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307728,"kb_used_data":2440,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064112,"statfs":{"total":96636764160,"available":96321650688,"internally_reserved":0,"allocated":2498560,"data_stored":1281316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.33900000000000002,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":0.61799999999999999,"5min":0.65800000000000003,"15min":0.74199999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.501}]},{"osd":2,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.876},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38600000000000001},"max":{"1min":0.78600000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.34699999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.40699999999999997,"15min":0.375},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.95899999999999996},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.42499999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.85199999999999998,"15min":0.85799999999999998},"last":0.42299999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.66100000000000003,"5min":0.622,"15min":0.56100000000000005},"max":{"1min":1.054,"5min":1.147,"15min":1.357},"last":0.749},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84399999999999997,"15min":0.83899999999999997},"min":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.54400000000000004},"max":{"1min":1.0369999999999999,"5min":1.1879999999999999,"15min":1.2150000000000001},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.71299999999999997,"5min":0.56699999999999995,"15min":0.52400000000000002},"max":{"1min":1.036,"5min":1.2170000000000001,"15min":1.2669999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.874,"15min":0.86899999999999999},"min":{"1min":0.67800000000000005,"5min":0.64500000000000002,"15min":0.58599999999999997},"max":{"1min":1.113,"5min":1.2350000000000001,"15min":1.2949999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.63700000000000001,"5min":0.58899999999999997,"15min":0.52600000000000002},"max":{"1min":1.0009999999999999,"5min":1.0980000000000001,"15min":1.3120000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.873,"15min":0.84999999999999998},"min":{"1min":0.66600000000000004,"5min":0.64500000000000002,"15min":0.54300000000000004},"max":{"1min":1.026,"5min":1.127,"15min":1.26},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86299999999999999,"15min":0.85999999999999999},"min":{"1min":0.628,"5min":0.621,"15min":0.57499999999999996},"max":{"1min":1.1040000000000001,"5min":1.1639999999999999,"15min":1.268},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.74399999999999999,"5min":0.59999999999999998,"15min":0.58999999999999997},"max":{"1min":1.0960000000000001,"5min":1.1140000000000001,"15min":1.4330000000000001},"last":0.69599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303632,"kb_used_data":2440,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068208,"statfs":{"total":96636764160,"available":96325844992,"internally_reserved":0,"allocated":2498560,"data_stored":1281316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.34200000000000003,"5min":0.29599999999999999,"15min":0.29499999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77900000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.36799999999999999},"max":{"1min":0.82699999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.628},"min":{"1min":0.50600000000000001,"5min":0.46300000000000002,"15min":0.34200000000000003},"max":{"1min":0.80000000000000004,"5min":0.89900000000000002,"15min":0.97299999999999998},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.78300000000000003},"last":0.371},{"interface":"front","average":{"1min":0.62,"5min":0.621,"15min":0.61699999999999999},"min":{"1min":0.45900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93400000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53600000000000003}]},{"osd":4,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.82499999999999996,"15min":0.82799999999999996},"min":{"1min":0.61499999999999999,"5min":0.58599999999999997,"15min":0.53900000000000003},"max":{"1min":1.0740000000000001,"5min":1.089,"15min":1.349},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.82299999999999995,"15min":0.82199999999999995},"min":{"1min":0.58499999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0429999999999999,"5min":1.1439999999999999,"15min":1.339},"last":0.82399999999999995}]},{"osd":5,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.56999999999999995,"5min":0.54300000000000004,"15min":0.48999999999999999},"max":{"1min":1.0649999999999999,"5min":1.1579999999999999,"15min":1.337},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86599999999999999,"15min":0.85899999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.56000000000000005},"max":{"1min":1.212,"5min":1.212,"15min":1.468},"last":0.83799999999999997}]},{"osd":6,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.83899999999999997,"15min":0.84999999999999998},"min":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.54600000000000004},"max":{"1min":1.0860000000000001,"5min":1.1930000000000001,"15min":1.387},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85299999999999998,"15min":0.85099999999999998},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.56100000000000005},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.4299999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.878,"15min":0.875},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.55700000000000005},"max":{"1min":1.1899999999999999,"5min":1.266,"15min":1.496},"last":0.872},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.87,"15min":0.874},"min":{"1min":0.66300000000000003,"5min":0.59199999999999997,"15min":0.54100000000000004},"max":{"1min":1.069,"5min":1.234,"15min":1.508},"last":0.90000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575951,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307784,"kb_used_data":2496,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064056,"statfs":{"total":96636764160,"available":96321593344,"internally_reserved":0,"allocated":2555904,"data_stored":1309201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46600000000000003,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.78000000000000003},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.35199999999999998,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53100000000000003}]},{"osd":1,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.498},"min":{"1min":0.371,"5min":0.34799999999999998,"15min":0.34100000000000003},"max":{"1min":0.63,"5min":0.71199999999999997,"15min":0.76900000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.58399999999999996},"min":{"1min":0.42199999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68500000000000005,"5min":0.82699999999999996,"15min":0.84799999999999998},"last":0.41499999999999998}]},{"osd":2,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.39400000000000002},"max":{"1min":0.71699999999999997,"5min":0.85299999999999998,"15min":0.87},"last":0.373},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.748,"5min":0.84199999999999997,"15min":0.89300000000000002},"last":0.47999999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.84499999999999997},"min":{"1min":0.627,"5min":0.60899999999999999,"15min":0.57999999999999996},"max":{"1min":1.0029999999999999,"5min":1.2370000000000001,"15min":1.365},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83299999999999996,"15min":0.84599999999999997},"min":{"1min":0.60999999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0669999999999999,"5min":1.1519999999999999,"15min":1.2989999999999999},"last":0.88700000000000001}]},{"osd":5,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81000000000000005,"15min":0.82099999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.2270000000000001,"15min":1.3440000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82299999999999995,"15min":0.83399999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.046,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82899999999999996,"15min":0.83899999999999997},"min":{"1min":0.58299999999999996,"5min":0.54500000000000004,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":1.212,"15min":1.331},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.86599999999999999,"15min":0.871},"min":{"1min":0.66900000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0660000000000001,"5min":1.1639999999999999,"15min":1.2749999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.86199999999999999,"15min":0.873},"min":{"1min":0.69699999999999995,"5min":0.61899999999999999,"15min":0.55900000000000005},"max":{"1min":1.034,"5min":1.2829999999999999,"15min":1.3080000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.86599999999999999,"15min":0.877},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.085,"5min":1.248,"15min":1.248},"last":0.86299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307728,"kb_used_data":2440,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064112,"statfs":{"total":96636764160,"available":96321650688,"internally_reserved":0,"allocated":2498560,"data_stored":1281316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.56299999999999994,"5min":0.43099999999999999,"15min":0.371},"max":{"1min":0.96599999999999997,"5min":0.96699999999999997,"15min":0.98999999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.72499999999999998,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.41699999999999998},"max":{"1min":0.84799999999999998,"5min":1.0009999999999999,"15min":1.006},"last":0.74199999999999999}]},{"osd":1,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":0.91200000000000003,"5min":1.0129999999999999,"15min":1.0289999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.503,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.92100000000000004,"5min":1.0600000000000001,"15min":1.085},"last":0.753}]},{"osd":2,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.629,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.087},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81899999999999995,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.48299999999999998},"max":{"1min":1.002,"5min":1.052,"15min":1.071},"last":0.78100000000000003}]},{"osd":3,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54200000000000004,"5min":0.52000000000000002,"15min":0.48699999999999999},"max":{"1min":0.90400000000000003,"5min":0.92400000000000004,"15min":1.0329999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.72899999999999998},"min":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.45600000000000002},"max":{"1min":0.95099999999999996,"5min":0.96999999999999997,"15min":1.0369999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.498,"5min":0.42399999999999999,"15min":0.39500000000000002},"max":{"1min":0.92300000000000004,"5min":0.93100000000000005,"15min":1.0469999999999999},"last":0.496},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.96799999999999997},"last":0.46200000000000002}]},{"osd":6,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78100000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.59299999999999997,"5min":0.48299999999999998,"15min":0.45800000000000002},"max":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":1.0269999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70699999999999996,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.93400000000000005,"5min":0.98799999999999999,"15min":1.018},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72699999999999998,"15min":0.72199999999999998},"min":{"1min":0.59199999999999997,"5min":0.54900000000000004,"15min":0.49099999999999999},"max":{"1min":0.86899999999999999,"5min":0.98699999999999999,"15min":1.054},"last":0.63300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575951,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307780,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064060,"statfs":{"total":96636764160,"available":96321597440,"internally_reserved":0,"allocated":2551808,"data_stored":1308309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69599999999999995,"15min":0.68700000000000006},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.309},"max":{"1min":0.96199999999999997,"5min":1.0269999999999999,"15min":1.032},"last":1.226},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69599999999999995,"15min":0.69299999999999995},"min":{"1min":0.44700000000000001,"5min":0.40300000000000002,"15min":0.35999999999999999},"max":{"1min":0.88200000000000001,"5min":0.95499999999999996,"15min":0.98899999999999999},"last":3.3559999999999999}]},{"osd":1,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.34799999999999998},"max":{"1min":0.877,"5min":0.997,"15min":1.21},"last":5.0279999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.77800000000000002},"min":{"1min":0.58799999999999997,"5min":0.47999999999999998,"15min":0.47299999999999998},"max":{"1min":0.98499999999999999,"5min":1.087,"15min":1.2290000000000001},"last":1.105}]},{"osd":2,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.54400000000000004,"5min":0.46400000000000002,"15min":0.45100000000000001},"max":{"1min":0.86399999999999999,"5min":1.113,"15min":1.113},"last":3.4089999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76500000000000001,"15min":0.754},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.48299999999999998},"max":{"1min":0.93100000000000005,"5min":1.103,"15min":1.103},"last":4.9779999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.57599999999999996,"5min":0.46200000000000002,"15min":0.36499999999999999},"max":{"1min":0.94599999999999995,"5min":0.996,"15min":1.044},"last":1.2729999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.74099999999999999},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.45800000000000002},"max":{"1min":0.94799999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":3.4489999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.90400000000000003,"15min":0.91700000000000004},"last":5.0659999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68600000000000005,"15min":0.68700000000000006},"min":{"1min":0.54600000000000004,"5min":0.38800000000000001,"15min":0.34999999999999998},"max":{"1min":0.94899999999999995,"5min":1.002,"15min":1.004},"last":1.585}]},{"osd":6,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":0.88700000000000001,"5min":0.97999999999999998,"15min":1.119},"last":3.4889999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.36299999999999999},"max":{"1min":0.93700000000000006,"5min":0.95599999999999996,"15min":1.0780000000000001},"last":5.1029999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.39200000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0780000000000001},"last":1.365},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70599999999999996,"15min":0.69499999999999995},"min":{"1min":0.51100000000000001,"5min":0.47499999999999998,"15min":0.45200000000000001},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":6.891}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:16:43.524 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:16:43.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.749+0000 7f5ee4fff700 1 Processor -- start 2023-11-06T22:16:43.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.750+0000 7f5ee4fff700 1 -- start start 2023-11-06T22:16:43.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.751+0000 7f5ee4fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5ee0064580 0x7f5ee0064960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.751+0000 7f5ee4fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ee00c3050 0x7f5ee00a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.751+0000 7f5ee4fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee00c2bc0 con 0x7f5ee00c3490 2023-11-06T22:16:43.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.751+0000 7f5ee4fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee00bbe00 con 0x7f5ee00c3050 2023-11-06T22:16:43.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.751+0000 7f5ee4fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee0152620 con 0x7f5ee0064580 2023-11-06T22:16:43.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.751+0000 7f5eddd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ee00c3050 0x7f5ee00a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.751+0000 7f5eded9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5ee00c3490 0x7f5ee00a6f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58070/0 (socket says 172.21.15.33:58070) 2023-11-06T22:16:43.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.751+0000 7f5eded9d700 1 -- 172.21.15.33:0/3225595086 learned_addr learned my addr 172.21.15.33:0/3225595086 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:43.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.752+0000 7f5ede59c700 1 --2- 172.21.15.33:0/3225595086 >> v2:172.21.15.69:3300/0 conn(0x7f5ee0064580 0x7f5ee0064960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.752+0000 7f5eddd9b700 1 -- 172.21.15.33:0/3225595086 >> v2:172.21.15.69:3300/0 conn(0x7f5ee0064580 msgr2=0x7f5ee0064960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:43.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.752+0000 7f5eddd9b700 1 --2- 172.21.15.33:0/3225595086 >> v2:172.21.15.69:3300/0 conn(0x7f5ee0064580 0x7f5ee0064960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.752+0000 7f5eddd9b700 1 -- 172.21.15.33:0/3225595086 >> v1:172.21.15.33:6789/0 conn(0x7f5ee00c3490 legacy=0x7f5ee00a6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.752+0000 7f5eddd9b700 1 -- 172.21.15.33:0/3225595086 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee00bbfe0 con 0x7f5ee00c3050 2023-11-06T22:16:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.752+0000 7f5edd59a700 1 -- 172.21.15.33:0/3225595086 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487685381 0 0) 0x7f5ee00c2bc0 con 0x7f5ee00c3490 2023-11-06T22:16:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.752+0000 7f5eddd9b700 1 --2- 172.21.15.33:0/3225595086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 0x7f5ee00a6a50 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f5ee00684b0 tx=0x7f5ed4009060).ready entity=mon.0 client_cookie=34f07dd95deb66f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.752+0000 7f5edd59a700 1 -- 172.21.15.33:0/3225595086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400b580 con 0x7f5ee00c3050 2023-11-06T22:16:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.752+0000 7f5edd59a700 1 -- 172.21.15.33:0/3225595086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed400b6e0 con 0x7f5ee00c3050 2023-11-06T22:16:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.752+0000 7f5edd59a700 1 -- 172.21.15.33:0/3225595086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400ba60 con 0x7f5ee00c3050 2023-11-06T22:16:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.753+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3225595086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 msgr2=0x7f5ee00a6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.753+0000 7f5ee4fff700 1 --2- 172.21.15.33:0/3225595086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 0x7f5ee00a6a50 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f5ee00684b0 tx=0x7f5ed4009060).stop 2023-11-06T22:16:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.753+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3225595086 shutdown_connections 2023-11-06T22:16:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.753+0000 7f5ee4fff700 1 --2- 172.21.15.33:0/3225595086 >> v2:172.21.15.69:3300/0 conn(0x7f5ee0064580 0x7f5ee0064960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.753+0000 7f5ee4fff700 1 --2- 172.21.15.33:0/3225595086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 0x7f5ee00a6a50 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.753+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3225595086 >> 172.21.15.33:0/3225595086 conn(0x7f5ee00631e0 msgr2=0x7f5ee00c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.753+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3225595086 shutdown_connections 2023-11-06T22:16:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.753+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3225595086 wait complete. 2023-11-06T22:16:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.754+0000 7f5ee4fff700 1 Processor -- start 2023-11-06T22:16:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.755+0000 7f5ee4fff700 1 -- start start 2023-11-06T22:16:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.755+0000 7f5ee4fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee0064580 0x7f5ee00c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.755+0000 7f5ee4fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 0x7f5ee00c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.755+0000 7f5ee4fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee00c3490 0x7f5ee00c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5ee4fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00c6b30 con 0x7f5ee00c3050 2023-11-06T22:16:43.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5ee4fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee0155400 con 0x7f5ee00c3490 2023-11-06T22:16:43.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5ee4fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee0154ad0 con 0x7f5ee0064580 2023-11-06T22:16:43.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eddd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 0x7f5ee00c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eddd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 0x7f5ee00c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41884/0 (socket says 172.21.15.33:41884) 2023-11-06T22:16:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eddd9b700 1 -- 172.21.15.33:0/3541783369 learned_addr learned my addr 172.21.15.33:0/3541783369 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5ede59c700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee0064580 0x7f5ee00c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eded9d700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee00c3490 0x7f5ee00c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eddd9b700 1 -- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee0064580 msgr2=0x7f5ee00c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eddd9b700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee0064580 0x7f5ee00c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eddd9b700 1 -- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee00c3490 msgr2=0x7f5ee00c0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eddd9b700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee00c3490 0x7f5ee00c0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eddd9b700 1 -- 172.21.15.33:0/3541783369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed40092a0 con 0x7f5ee00c3050 2023-11-06T22:16:43.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eded9d700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee00c3490 0x7f5ee00c0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:43.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.756+0000 7f5eddd9b700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 0x7f5ee00c6640 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7f5ed4009fd0 tx=0x7f5ed4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:43.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.757+0000 7f5ece7fc700 1 -- 172.21.15.33:0/3541783369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400b850 con 0x7f5ee00c3050 2023-11-06T22:16:43.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.757+0000 7f5ece7fc700 1 -- 172.21.15.33:0/3541783369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed400b9b0 con 0x7f5ee00c3050 2023-11-06T22:16:43.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.757+0000 7f5ece7fc700 1 -- 172.21.15.33:0/3541783369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed401b660 con 0x7f5ee00c3050 2023-11-06T22:16:43.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.757+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3541783369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee0154db0 con 0x7f5ee00c3050 2023-11-06T22:16:43.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.757+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3541783369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee014a6a0 con 0x7f5ee00c3050 2023-11-06T22:16:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.758+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3541783369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee01555b0 con 0x7f5ee00c3050 2023-11-06T22:16:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.759+0000 7f5ece7fc700 1 -- 172.21.15.33:0/3541783369 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5ed401ba50 con 0x7f5ee00c3050 2023-11-06T22:16:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.760+0000 7f5ece7fc700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5ec005f270 0x7f5ec0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.760+0000 7f5ece7fc700 1 -- 172.21.15.33:0/3541783369 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f5ed407bfa0 con 0x7f5ee00c3050 2023-11-06T22:16:43.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.760+0000 7f5ede59c700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5ec005f270 0x7f5ec0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:43.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.761+0000 7f5ede59c700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5ec005f270 0x7f5ec0061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f5ed000bbc0 tx=0x7f5ed0008040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:43.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.764+0000 7f5ece7fc700 1 -- 172.21.15.33:0/3541783369 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5ed404d000 con 0x7f5ee00c3050 2023-11-06T22:16:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.939+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3541783369 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ee014a8c0 con 0x7f5ec005f270 2023-11-06T22:16:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.950+0000 7f5ece7fc700 1 -- 172.21.15.33:0/3541783369 <== mgr.7733 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361736 (secure 0 0 0) 0x7f5ee014a8c0 con 0x7f5ec005f270 2023-11-06T22:16:43.957 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5ec005f270 msgr2=0x7f5ec0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5ec005f270 0x7f5ec0061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f5ed000bbc0 tx=0x7f5ed0008040).stop 2023-11-06T22:16:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3541783369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 msgr2=0x7f5ee00c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 0x7f5ee00c6640 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7f5ed4009fd0 tx=0x7f5ed4017000).stop 2023-11-06T22:16:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3541783369 shutdown_connections 2023-11-06T22:16:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5ec005f270 0x7f5ec0061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5ee0064580 0x7f5ee00c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ee00c3490 0x7f5ee00c0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 --2- 172.21.15.33:0/3541783369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ee00c3050 0x7f5ee00c6640 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3541783369 >> 172.21.15.33:0/3541783369 conn(0x7f5ee00631e0 msgr2=0x7f5ee00aa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3541783369 shutdown_connections 2023-11-06T22:16:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:43.954+0000 7f5ee4fff700 1 -- 172.21.15.33:0/3541783369 wait complete. 2023-11-06T22:16:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:16:43.974 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T22:16:43.571286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":229,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2458176,"kb_used_data":19968,"kb_used_omap":0,"kb_used_meta":2438144,"kb_avail":752516544,"statfs":{"total":773094113280,"available":770576941056,"internally_reserved":0,"allocated":20447232,"data_stored":10470812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2496659456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-592676,"num_objects":-7,"num_object_clones":0,"num_object_copies":-14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":-173,"num_write_kb":-713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.651388"},"pg_stats":[{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.181425+0000","last_change":"2023-11-06T22:16:39.172566+0000","last_active":"2023-11-06T22:16:39.181425+0000","last_peered":"2023-11-06T22:16:39.181425+0000","last_clean":"2023-11-06T22:16:39.181425+0000","last_became_active":"2023-11-06T22:16:39.172306+0000","last_became_peered":"2023-11-06T22:16:39.172306+0000","last_unstale":"2023-11-06T22:16:39.181425+0000","last_undegraded":"2023-11-06T22:16:39.181425+0000","last_fullsized":"2023-11-06T22:16:39.181425+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3f","version":"203'3","reported_seq":13,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.192777+0000","last_change":"2023-11-06T22:16:38.694207+0000","last_active":"2023-11-06T22:16:39.192777+0000","last_peered":"2023-11-06T22:16:39.192777+0000","last_clean":"2023-11-06T22:16:39.192777+0000","last_became_active":"2023-11-06T22:16:38.691472+0000","last_became_peered":"2023-11-06T22:16:38.691472+0000","last_unstale":"2023-11-06T22:16:39.192777+0000","last_undegraded":"2023-11-06T22:16:39.192777+0000","last_fullsized":"2023-11-06T22:16:39.192777+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.189950+0000","last_change":"2023-11-06T22:16:39.184748+0000","last_active":"2023-11-06T22:16:39.189950+0000","last_peered":"2023-11-06T22:16:39.189950+0000","last_clean":"2023-11-06T22:16:39.189950+0000","last_became_active":"2023-11-06T22:16:39.184488+0000","last_became_peered":"2023-11-06T22:16:39.184488+0000","last_unstale":"2023-11-06T22:16:39.189950+0000","last_undegraded":"2023-11-06T22:16:39.189950+0000","last_fullsized":"2023-11-06T22:16:39.189950+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.174307+0000","last_change":"2023-11-06T22:16:38.693068+0000","last_active":"2023-11-06T22:16:39.174307+0000","last_peered":"2023-11-06T22:16:39.174307+0000","last_clean":"2023-11-06T22:16:39.174307+0000","last_became_active":"2023-11-06T22:16:38.691450+0000","last_became_peered":"2023-11-06T22:16:38.691450+0000","last_unstale":"2023-11-06T22:16:39.174307+0000","last_undegraded":"2023-11-06T22:16:39.174307+0000","last_fullsized":"2023-11-06T22:16:39.174307+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182097+0000","last_change":"2023-11-06T22:16:39.177666+0000","last_active":"2023-11-06T22:16:39.182097+0000","last_peered":"2023-11-06T22:16:39.182097+0000","last_clean":"2023-11-06T22:16:39.182097+0000","last_became_active":"2023-11-06T22:16:39.177358+0000","last_became_peered":"2023-11-06T22:16:39.177358+0000","last_unstale":"2023-11-06T22:16:39.182097+0000","last_undegraded":"2023-11-06T22:16:39.182097+0000","last_fullsized":"2023-11-06T22:16:39.182097+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.175397+0000","last_change":"2023-11-06T22:16:38.688001+0000","last_active":"2023-11-06T22:16:39.175397+0000","last_peered":"2023-11-06T22:16:39.175397+0000","last_clean":"2023-11-06T22:16:39.175397+0000","last_became_active":"2023-11-06T22:16:38.687570+0000","last_became_peered":"2023-11-06T22:16:38.687570+0000","last_unstale":"2023-11-06T22:16:39.175397+0000","last_undegraded":"2023-11-06T22:16:39.175397+0000","last_fullsized":"2023-11-06T22:16:39.175397+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.175704+0000","last_change":"2023-11-06T22:16:39.170554+0000","last_active":"2023-11-06T22:16:39.175704+0000","last_peered":"2023-11-06T22:16:39.175704+0000","last_clean":"2023-11-06T22:16:39.175704+0000","last_became_active":"2023-11-06T22:16:39.170293+0000","last_became_peered":"2023-11-06T22:16:39.170293+0000","last_unstale":"2023-11-06T22:16:39.175704+0000","last_undegraded":"2023-11-06T22:16:39.175704+0000","last_fullsized":"2023-11-06T22:16:39.175704+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3c","version":"203'2","reported_seq":12,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.179349+0000","last_change":"2023-11-06T22:16:38.683358+0000","last_active":"2023-11-06T22:16:39.179349+0000","last_peered":"2023-11-06T22:16:39.179349+0000","last_clean":"2023-11-06T22:16:39.179349+0000","last_became_active":"2023-11-06T22:16:38.682317+0000","last_became_peered":"2023-11-06T22:16:38.682317+0000","last_unstale":"2023-11-06T22:16:39.179349+0000","last_undegraded":"2023-11-06T22:16:39.179349+0000","last_fullsized":"2023-11-06T22:16:39.179349+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.192055+0000","last_change":"2023-11-06T22:16:39.184927+0000","last_active":"2023-11-06T22:16:39.192055+0000","last_peered":"2023-11-06T22:16:39.192055+0000","last_clean":"2023-11-06T22:16:39.192055+0000","last_became_active":"2023-11-06T22:16:39.184656+0000","last_became_peered":"2023-11-06T22:16:39.184656+0000","last_unstale":"2023-11-06T22:16:39.192055+0000","last_undegraded":"2023-11-06T22:16:39.192055+0000","last_fullsized":"2023-11-06T22:16:39.192055+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.3b","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.177525+0000","last_change":"2023-11-06T22:16:38.689017+0000","last_active":"2023-11-06T22:16:39.177525+0000","last_peered":"2023-11-06T22:16:39.177525+0000","last_clean":"2023-11-06T22:16:39.177525+0000","last_became_active":"2023-11-06T22:16:38.688721+0000","last_became_peered":"2023-11-06T22:16:38.688721+0000","last_unstale":"2023-11-06T22:16:39.177525+0000","last_undegraded":"2023-11-06T22:16:39.177525+0000","last_fullsized":"2023-11-06T22:16:39.177525+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182594+0000","last_change":"2023-11-06T22:16:39.179223+0000","last_active":"2023-11-06T22:16:39.182594+0000","last_peered":"2023-11-06T22:16:39.182594+0000","last_clean":"2023-11-06T22:16:39.182594+0000","last_became_active":"2023-11-06T22:16:39.178944+0000","last_became_peered":"2023-11-06T22:16:39.178944+0000","last_unstale":"2023-11-06T22:16:39.182594+0000","last_undegraded":"2023-11-06T22:16:39.182594+0000","last_fullsized":"2023-11-06T22:16:39.182594+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.174505+0000","last_change":"2023-11-06T22:16:38.690454+0000","last_active":"2023-11-06T22:16:39.174505+0000","last_peered":"2023-11-06T22:16:39.174505+0000","last_clean":"2023-11-06T22:16:39.174505+0000","last_became_active":"2023-11-06T22:16:38.686051+0000","last_became_peered":"2023-11-06T22:16:38.686051+0000","last_unstale":"2023-11-06T22:16:39.174505+0000","last_undegraded":"2023-11-06T22:16:39.174505+0000","last_fullsized":"2023-11-06T22:16:39.174505+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.192122+0000","last_change":"2023-11-06T22:16:39.186250+0000","last_active":"2023-11-06T22:16:39.192122+0000","last_peered":"2023-11-06T22:16:39.192122+0000","last_clean":"2023-11-06T22:16:39.192122+0000","last_became_active":"2023-11-06T22:16:39.185949+0000","last_became_peered":"2023-11-06T22:16:39.185949+0000","last_unstale":"2023-11-06T22:16:39.192122+0000","last_undegraded":"2023-11-06T22:16:39.192122+0000","last_fullsized":"2023-11-06T22:16:39.192122+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.160214+0000","last_change":"2023-11-06T22:16:38.694030+0000","last_active":"2023-11-06T22:16:39.160214+0000","last_peered":"2023-11-06T22:16:39.160214+0000","last_clean":"2023-11-06T22:16:39.160214+0000","last_became_active":"2023-11-06T22:16:38.693750+0000","last_became_peered":"2023-11-06T22:16:38.693750+0000","last_unstale":"2023-11-06T22:16:39.160214+0000","last_undegraded":"2023-11-06T22:16:39.160214+0000","last_fullsized":"2023-11-06T22:16:39.160214+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.180112+0000","last_change":"2023-11-06T22:16:39.169921+0000","last_active":"2023-11-06T22:16:39.180112+0000","last_peered":"2023-11-06T22:16:39.180112+0000","last_clean":"2023-11-06T22:16:39.180112+0000","last_became_active":"2023-11-06T22:16:39.169676+0000","last_became_peered":"2023-11-06T22:16:39.169676+0000","last_unstale":"2023-11-06T22:16:39.180112+0000","last_undegraded":"2023-11-06T22:16:39.180112+0000","last_fullsized":"2023-11-06T22:16:39.180112+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.160015+0000","last_change":"2023-11-06T22:16:38.686921+0000","last_active":"2023-11-06T22:16:39.160015+0000","last_peered":"2023-11-06T22:16:39.160015+0000","last_clean":"2023-11-06T22:16:39.160015+0000","last_became_active":"2023-11-06T22:16:38.686647+0000","last_became_peered":"2023-11-06T22:16:38.686647+0000","last_unstale":"2023-11-06T22:16:39.160015+0000","last_undegraded":"2023-11-06T22:16:39.160015+0000","last_fullsized":"2023-11-06T22:16:39.160015+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.179270+0000","last_change":"2023-11-06T22:16:39.174349+0000","last_active":"2023-11-06T22:16:39.179270+0000","last_peered":"2023-11-06T22:16:39.179270+0000","last_clean":"2023-11-06T22:16:39.179270+0000","last_became_active":"2023-11-06T22:16:39.173775+0000","last_became_peered":"2023-11-06T22:16:39.173775+0000","last_unstale":"2023-11-06T22:16:39.179270+0000","last_undegraded":"2023-11-06T22:16:39.179270+0000","last_fullsized":"2023-11-06T22:16:39.179270+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.163133+0000","last_change":"2023-11-06T22:16:38.683310+0000","last_active":"2023-11-06T22:16:39.163133+0000","last_peered":"2023-11-06T22:16:39.163133+0000","last_clean":"2023-11-06T22:16:39.163133+0000","last_became_active":"2023-11-06T22:16:38.682988+0000","last_became_peered":"2023-11-06T22:16:38.682988+0000","last_unstale":"2023-11-06T22:16:39.163133+0000","last_undegraded":"2023-11-06T22:16:39.163133+0000","last_fullsized":"2023-11-06T22:16:39.163133+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.176507+0000","last_change":"2023-11-06T22:16:39.173303+0000","last_active":"2023-11-06T22:16:39.176507+0000","last_peered":"2023-11-06T22:16:39.176507+0000","last_clean":"2023-11-06T22:16:39.176507+0000","last_became_active":"2023-11-06T22:16:39.172088+0000","last_became_peered":"2023-11-06T22:16:39.172088+0000","last_unstale":"2023-11-06T22:16:39.176507+0000","last_undegraded":"2023-11-06T22:16:39.176507+0000","last_fullsized":"2023-11-06T22:16:39.176507+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.161009+0000","last_change":"2023-11-06T22:16:38.686710+0000","last_active":"2023-11-06T22:16:39.161009+0000","last_peered":"2023-11-06T22:16:39.161009+0000","last_clean":"2023-11-06T22:16:39.161009+0000","last_became_active":"2023-11-06T22:16:38.686366+0000","last_became_peered":"2023-11-06T22:16:38.686366+0000","last_unstale":"2023-11-06T22:16:39.161009+0000","last_undegraded":"2023-11-06T22:16:39.161009+0000","last_fullsized":"2023-11-06T22:16:39.161009+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.173684+0000","last_change":"2023-11-06T22:16:39.169147+0000","last_active":"2023-11-06T22:16:39.173684+0000","last_peered":"2023-11-06T22:16:39.173684+0000","last_clean":"2023-11-06T22:16:39.173684+0000","last_became_active":"2023-11-06T22:16:39.168831+0000","last_became_peered":"2023-11-06T22:16:39.168831+0000","last_unstale":"2023-11-06T22:16:39.173684+0000","last_undegraded":"2023-11-06T22:16:39.173684+0000","last_fullsized":"2023-11-06T22:16:39.173684+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.165536+0000","last_change":"2023-11-06T22:16:38.674603+0000","last_active":"2023-11-06T22:16:39.165536+0000","last_peered":"2023-11-06T22:16:39.165536+0000","last_clean":"2023-11-06T22:16:39.165536+0000","last_became_active":"2023-11-06T22:16:38.674300+0000","last_became_peered":"2023-11-06T22:16:38.674300+0000","last_unstale":"2023-11-06T22:16:39.165536+0000","last_undegraded":"2023-11-06T22:16:39.165536+0000","last_fullsized":"2023-11-06T22:16:39.165536+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.180351+0000","last_change":"2023-11-06T22:16:39.174787+0000","last_active":"2023-11-06T22:16:39.180351+0000","last_peered":"2023-11-06T22:16:39.180351+0000","last_clean":"2023-11-06T22:16:39.180351+0000","last_became_active":"2023-11-06T22:16:39.174510+0000","last_became_peered":"2023-11-06T22:16:39.174510+0000","last_unstale":"2023-11-06T22:16:39.180351+0000","last_undegraded":"2023-11-06T22:16:39.180351+0000","last_fullsized":"2023-11-06T22:16:39.180351+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"203'4","reported_seq":14,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.192449+0000","last_change":"2023-11-06T22:16:38.691776+0000","last_active":"2023-11-06T22:16:39.192449+0000","last_peered":"2023-11-06T22:16:39.192449+0000","last_clean":"2023-11-06T22:16:39.192449+0000","last_became_active":"2023-11-06T22:16:38.691516+0000","last_became_peered":"2023-11-06T22:16:38.691516+0000","last_unstale":"2023-11-06T22:16:39.192449+0000","last_undegraded":"2023-11-06T22:16:39.192449+0000","last_fullsized":"2023-11-06T22:16:39.192449+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.192028+0000","last_change":"2023-11-06T22:16:39.184006+0000","last_active":"2023-11-06T22:16:39.192028+0000","last_peered":"2023-11-06T22:16:39.192028+0000","last_clean":"2023-11-06T22:16:39.192028+0000","last_became_active":"2023-11-06T22:16:39.183722+0000","last_became_peered":"2023-11-06T22:16:39.183722+0000","last_unstale":"2023-11-06T22:16:39.192028+0000","last_undegraded":"2023-11-06T22:16:39.192028+0000","last_fullsized":"2023-11-06T22:16:39.192028+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.33","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.173440+0000","last_change":"2023-11-06T22:16:38.678970+0000","last_active":"2023-11-06T22:16:39.173440+0000","last_peered":"2023-11-06T22:16:39.173440+0000","last_clean":"2023-11-06T22:16:39.173440+0000","last_became_active":"2023-11-06T22:16:38.678575+0000","last_became_peered":"2023-11-06T22:16:38.678575+0000","last_unstale":"2023-11-06T22:16:39.173440+0000","last_undegraded":"2023-11-06T22:16:39.173440+0000","last_fullsized":"2023-11-06T22:16:39.173440+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.189314+0000","last_change":"2023-11-06T22:16:39.182698+0000","last_active":"2023-11-06T22:16:39.189314+0000","last_peered":"2023-11-06T22:16:39.189314+0000","last_clean":"2023-11-06T22:16:39.189314+0000","last_became_active":"2023-11-06T22:16:39.182370+0000","last_became_peered":"2023-11-06T22:16:39.182370+0000","last_unstale":"2023-11-06T22:16:39.189314+0000","last_undegraded":"2023-11-06T22:16:39.189314+0000","last_fullsized":"2023-11-06T22:16:39.189314+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.160202+0000","last_change":"2023-11-06T22:16:38.686557+0000","last_active":"2023-11-06T22:16:39.160202+0000","last_peered":"2023-11-06T22:16:39.160202+0000","last_clean":"2023-11-06T22:16:39.160202+0000","last_became_active":"2023-11-06T22:16:38.686042+0000","last_became_peered":"2023-11-06T22:16:38.686042+0000","last_unstale":"2023-11-06T22:16:39.160202+0000","last_undegraded":"2023-11-06T22:16:39.160202+0000","last_fullsized":"2023-11-06T22:16:39.160202+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.192283+0000","last_change":"2023-11-06T22:16:39.185170+0000","last_active":"2023-11-06T22:16:39.192283+0000","last_peered":"2023-11-06T22:16:39.192283+0000","last_clean":"2023-11-06T22:16:39.192283+0000","last_became_active":"2023-11-06T22:16:39.184115+0000","last_became_peered":"2023-11-06T22:16:39.184115+0000","last_unstale":"2023-11-06T22:16:39.192283+0000","last_undegraded":"2023-11-06T22:16:39.192283+0000","last_fullsized":"2023-11-06T22:16:39.192283+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.159968+0000","last_change":"2023-11-06T22:16:38.697034+0000","last_active":"2023-11-06T22:16:39.159968+0000","last_peered":"2023-11-06T22:16:39.159968+0000","last_clean":"2023-11-06T22:16:39.159968+0000","last_became_active":"2023-11-06T22:16:38.696716+0000","last_became_peered":"2023-11-06T22:16:38.696716+0000","last_unstale":"2023-11-06T22:16:39.159968+0000","last_undegraded":"2023-11-06T22:16:39.159968+0000","last_fullsized":"2023-11-06T22:16:39.159968+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.183522+0000","last_change":"2023-11-06T22:16:39.180498+0000","last_active":"2023-11-06T22:16:39.183522+0000","last_peered":"2023-11-06T22:16:39.183522+0000","last_clean":"2023-11-06T22:16:39.183522+0000","last_became_active":"2023-11-06T22:16:39.179266+0000","last_became_peered":"2023-11-06T22:16:39.179266+0000","last_unstale":"2023-11-06T22:16:39.183522+0000","last_undegraded":"2023-11-06T22:16:39.183522+0000","last_fullsized":"2023-11-06T22:16:39.183522+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.30","version":"203'4","reported_seq":14,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.187191+0000","last_change":"2023-11-06T22:16:38.689207+0000","last_active":"2023-11-06T22:16:39.187191+0000","last_peered":"2023-11-06T22:16:39.187191+0000","last_clean":"2023-11-06T22:16:39.187191+0000","last_became_active":"2023-11-06T22:16:38.688935+0000","last_became_peered":"2023-11-06T22:16:38.688935+0000","last_unstale":"2023-11-06T22:16:39.187191+0000","last_undegraded":"2023-11-06T22:16:39.187191+0000","last_fullsized":"2023-11-06T22:16:39.187191+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.192537+0000","last_change":"2023-11-06T22:16:39.187851+0000","last_active":"2023-11-06T22:16:39.192537+0000","last_peered":"2023-11-06T22:16:39.192537+0000","last_clean":"2023-11-06T22:16:39.192537+0000","last_became_active":"2023-11-06T22:16:39.187555+0000","last_became_peered":"2023-11-06T22:16:39.187555+0000","last_unstale":"2023-11-06T22:16:39.192537+0000","last_undegraded":"2023-11-06T22:16:39.192537+0000","last_fullsized":"2023-11-06T22:16:39.192537+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.154714+0000","last_change":"2023-11-06T22:16:38.685682+0000","last_active":"2023-11-06T22:16:39.154714+0000","last_peered":"2023-11-06T22:16:39.154714+0000","last_clean":"2023-11-06T22:16:39.154714+0000","last_became_active":"2023-11-06T22:16:38.684719+0000","last_became_peered":"2023-11-06T22:16:38.684719+0000","last_unstale":"2023-11-06T22:16:39.154714+0000","last_undegraded":"2023-11-06T22:16:39.154714+0000","last_fullsized":"2023-11-06T22:16:39.154714+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.186059+0000","last_change":"2023-11-06T22:16:39.176570+0000","last_active":"2023-11-06T22:16:39.186059+0000","last_peered":"2023-11-06T22:16:39.186059+0000","last_clean":"2023-11-06T22:16:39.186059+0000","last_became_active":"2023-11-06T22:16:39.174652+0000","last_became_peered":"2023-11-06T22:16:39.174652+0000","last_unstale":"2023-11-06T22:16:39.186059+0000","last_undegraded":"2023-11-06T22:16:39.186059+0000","last_fullsized":"2023-11-06T22:16:39.186059+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.157581+0000","last_change":"2023-11-06T22:16:38.692388+0000","last_active":"2023-11-06T22:16:39.157581+0000","last_peered":"2023-11-06T22:16:39.157581+0000","last_clean":"2023-11-06T22:16:39.157581+0000","last_became_active":"2023-11-06T22:16:38.691407+0000","last_became_peered":"2023-11-06T22:16:38.691407+0000","last_unstale":"2023-11-06T22:16:39.157581+0000","last_undegraded":"2023-11-06T22:16:39.157581+0000","last_fullsized":"2023-11-06T22:16:39.157581+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.172839+0000","last_change":"2023-11-06T22:16:39.168750+0000","last_active":"2023-11-06T22:16:39.172839+0000","last_peered":"2023-11-06T22:16:39.172839+0000","last_clean":"2023-11-06T22:16:39.172839+0000","last_became_active":"2023-11-06T22:16:39.168486+0000","last_became_peered":"2023-11-06T22:16:39.168486+0000","last_unstale":"2023-11-06T22:16:39.172839+0000","last_undegraded":"2023-11-06T22:16:39.172839+0000","last_fullsized":"2023-11-06T22:16:39.172839+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.2d","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.186589+0000","last_change":"2023-11-06T22:16:38.691783+0000","last_active":"2023-11-06T22:16:39.186589+0000","last_peered":"2023-11-06T22:16:39.186589+0000","last_clean":"2023-11-06T22:16:39.186589+0000","last_became_active":"2023-11-06T22:16:38.690922+0000","last_became_peered":"2023-11-06T22:16:38.690922+0000","last_unstale":"2023-11-06T22:16:39.186589+0000","last_undegraded":"2023-11-06T22:16:39.186589+0000","last_fullsized":"2023-11-06T22:16:39.186589+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182828+0000","last_change":"2023-11-06T22:16:39.163644+0000","last_active":"2023-11-06T22:16:39.182828+0000","last_peered":"2023-11-06T22:16:39.182828+0000","last_clean":"2023-11-06T22:16:39.182828+0000","last_became_active":"2023-11-06T22:16:39.163320+0000","last_became_peered":"2023-11-06T22:16:39.163320+0000","last_unstale":"2023-11-06T22:16:39.182828+0000","last_undegraded":"2023-11-06T22:16:39.182828+0000","last_fullsized":"2023-11-06T22:16:39.182828+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.153745+0000","last_change":"2023-11-06T22:16:38.677947+0000","last_active":"2023-11-06T22:16:39.153745+0000","last_peered":"2023-11-06T22:16:39.153745+0000","last_clean":"2023-11-06T22:16:39.153745+0000","last_became_active":"2023-11-06T22:16:38.677660+0000","last_became_peered":"2023-11-06T22:16:38.677660+0000","last_unstale":"2023-11-06T22:16:39.153745+0000","last_undegraded":"2023-11-06T22:16:39.153745+0000","last_fullsized":"2023-11-06T22:16:39.153745+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.163960+0000","last_change":"2023-11-06T22:16:39.161891+0000","last_active":"2023-11-06T22:16:39.163960+0000","last_peered":"2023-11-06T22:16:39.163960+0000","last_clean":"2023-11-06T22:16:39.163960+0000","last_became_active":"2023-11-06T22:16:39.161030+0000","last_became_peered":"2023-11-06T22:16:39.161030+0000","last_unstale":"2023-11-06T22:16:39.163960+0000","last_undegraded":"2023-11-06T22:16:39.163960+0000","last_fullsized":"2023-11-06T22:16:39.163960+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2b","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.179566+0000","last_change":"2023-11-06T22:16:38.686256+0000","last_active":"2023-11-06T22:16:39.179566+0000","last_peered":"2023-11-06T22:16:39.179566+0000","last_clean":"2023-11-06T22:16:39.179566+0000","last_became_active":"2023-11-06T22:16:38.685698+0000","last_became_peered":"2023-11-06T22:16:38.685698+0000","last_unstale":"2023-11-06T22:16:39.179566+0000","last_undegraded":"2023-11-06T22:16:39.179566+0000","last_fullsized":"2023-11-06T22:16:39.179566+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.192541+0000","last_change":"2023-11-06T22:16:39.188498+0000","last_active":"2023-11-06T22:16:39.192541+0000","last_peered":"2023-11-06T22:16:39.192541+0000","last_clean":"2023-11-06T22:16:39.192541+0000","last_became_active":"2023-11-06T22:16:39.188186+0000","last_became_peered":"2023-11-06T22:16:39.188186+0000","last_unstale":"2023-11-06T22:16:39.192541+0000","last_undegraded":"2023-11-06T22:16:39.192541+0000","last_fullsized":"2023-11-06T22:16:39.192541+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.165866+0000","last_change":"2023-11-06T22:16:38.681806+0000","last_active":"2023-11-06T22:16:39.165866+0000","last_peered":"2023-11-06T22:16:39.165866+0000","last_clean":"2023-11-06T22:16:39.165866+0000","last_became_active":"2023-11-06T22:16:38.681534+0000","last_became_peered":"2023-11-06T22:16:38.681534+0000","last_unstale":"2023-11-06T22:16:39.165866+0000","last_undegraded":"2023-11-06T22:16:39.165866+0000","last_fullsized":"2023-11-06T22:16:39.165866+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.184047+0000","last_change":"2023-11-06T22:16:39.182550+0000","last_active":"2023-11-06T22:16:39.184047+0000","last_peered":"2023-11-06T22:16:39.184047+0000","last_clean":"2023-11-06T22:16:39.184047+0000","last_became_active":"2023-11-06T22:16:39.182257+0000","last_became_peered":"2023-11-06T22:16:39.182257+0000","last_unstale":"2023-11-06T22:16:39.184047+0000","last_undegraded":"2023-11-06T22:16:39.184047+0000","last_fullsized":"2023-11-06T22:16:39.184047+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.164660+0000","last_change":"2023-11-06T22:16:38.678374+0000","last_active":"2023-11-06T22:16:39.164660+0000","last_peered":"2023-11-06T22:16:39.164660+0000","last_clean":"2023-11-06T22:16:39.164660+0000","last_became_active":"2023-11-06T22:16:38.678085+0000","last_became_peered":"2023-11-06T22:16:38.678085+0000","last_unstale":"2023-11-06T22:16:39.164660+0000","last_undegraded":"2023-11-06T22:16:39.164660+0000","last_fullsized":"2023-11-06T22:16:39.164660+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182196+0000","last_change":"2023-11-06T22:16:39.177821+0000","last_active":"2023-11-06T22:16:39.182196+0000","last_peered":"2023-11-06T22:16:39.182196+0000","last_clean":"2023-11-06T22:16:39.182196+0000","last_became_active":"2023-11-06T22:16:39.177521+0000","last_became_peered":"2023-11-06T22:16:39.177521+0000","last_unstale":"2023-11-06T22:16:39.182196+0000","last_undegraded":"2023-11-06T22:16:39.182196+0000","last_fullsized":"2023-11-06T22:16:39.182196+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.163966+0000","last_change":"2023-11-06T22:16:38.685270+0000","last_active":"2023-11-06T22:16:39.163966+0000","last_peered":"2023-11-06T22:16:39.163966+0000","last_clean":"2023-11-06T22:16:39.163966+0000","last_became_active":"2023-11-06T22:16:38.685000+0000","last_became_peered":"2023-11-06T22:16:38.685000+0000","last_unstale":"2023-11-06T22:16:39.163966+0000","last_undegraded":"2023-11-06T22:16:39.163966+0000","last_fullsized":"2023-11-06T22:16:39.163966+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.183433+0000","last_change":"2023-11-06T22:16:39.174263+0000","last_active":"2023-11-06T22:16:39.183433+0000","last_peered":"2023-11-06T22:16:39.183433+0000","last_clean":"2023-11-06T22:16:39.183433+0000","last_became_active":"2023-11-06T22:16:39.173969+0000","last_became_peered":"2023-11-06T22:16:39.173969+0000","last_unstale":"2023-11-06T22:16:39.183433+0000","last_undegraded":"2023-11-06T22:16:39.183433+0000","last_fullsized":"2023-11-06T22:16:39.183433+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.165514+0000","last_change":"2023-11-06T22:16:38.677925+0000","last_active":"2023-11-06T22:16:39.165514+0000","last_peered":"2023-11-06T22:16:39.165514+0000","last_clean":"2023-11-06T22:16:39.165514+0000","last_became_active":"2023-11-06T22:16:38.677642+0000","last_became_peered":"2023-11-06T22:16:38.677642+0000","last_unstale":"2023-11-06T22:16:39.165514+0000","last_undegraded":"2023-11-06T22:16:39.165514+0000","last_fullsized":"2023-11-06T22:16:39.165514+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.180155+0000","last_change":"2023-11-06T22:16:39.174320+0000","last_active":"2023-11-06T22:16:39.180155+0000","last_peered":"2023-11-06T22:16:39.180155+0000","last_clean":"2023-11-06T22:16:39.180155+0000","last_became_active":"2023-11-06T22:16:39.168510+0000","last_became_peered":"2023-11-06T22:16:39.168510+0000","last_unstale":"2023-11-06T22:16:39.180155+0000","last_undegraded":"2023-11-06T22:16:39.180155+0000","last_fullsized":"2023-11-06T22:16:39.180155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.167510+0000","last_change":"2023-11-06T22:16:38.686680+0000","last_active":"2023-11-06T22:16:39.167510+0000","last_peered":"2023-11-06T22:16:39.167510+0000","last_clean":"2023-11-06T22:16:39.167510+0000","last_became_active":"2023-11-06T22:16:38.682448+0000","last_became_peered":"2023-11-06T22:16:38.682448+0000","last_unstale":"2023-11-06T22:16:39.167510+0000","last_undegraded":"2023-11-06T22:16:39.167510+0000","last_fullsized":"2023-11-06T22:16:39.167510+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.181984+0000","last_change":"2023-11-06T22:16:39.173057+0000","last_active":"2023-11-06T22:16:39.181984+0000","last_peered":"2023-11-06T22:16:39.181984+0000","last_clean":"2023-11-06T22:16:39.181984+0000","last_became_active":"2023-11-06T22:16:39.172662+0000","last_became_peered":"2023-11-06T22:16:39.172662+0000","last_unstale":"2023-11-06T22:16:39.181984+0000","last_undegraded":"2023-11-06T22:16:39.181984+0000","last_fullsized":"2023-11-06T22:16:39.181984+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.165630+0000","last_change":"2023-11-06T22:16:38.687685+0000","last_active":"2023-11-06T22:16:39.165630+0000","last_peered":"2023-11-06T22:16:39.165630+0000","last_clean":"2023-11-06T22:16:39.165630+0000","last_became_active":"2023-11-06T22:16:38.687424+0000","last_became_peered":"2023-11-06T22:16:38.687424+0000","last_unstale":"2023-11-06T22:16:39.165630+0000","last_undegraded":"2023-11-06T22:16:39.165630+0000","last_fullsized":"2023-11-06T22:16:39.165630+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.171429+0000","last_change":"2023-11-06T22:16:39.168559+0000","last_active":"2023-11-06T22:16:39.171429+0000","last_peered":"2023-11-06T22:16:39.171429+0000","last_clean":"2023-11-06T22:16:39.171429+0000","last_became_active":"2023-11-06T22:16:39.165530+0000","last_became_peered":"2023-11-06T22:16:39.165530+0000","last_unstale":"2023-11-06T22:16:39.171429+0000","last_undegraded":"2023-11-06T22:16:39.171429+0000","last_fullsized":"2023-11-06T22:16:39.171429+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.160103+0000","last_change":"2023-11-06T22:16:38.692331+0000","last_active":"2023-11-06T22:16:39.160103+0000","last_peered":"2023-11-06T22:16:39.160103+0000","last_clean":"2023-11-06T22:16:39.160103+0000","last_became_active":"2023-11-06T22:16:38.688132+0000","last_became_peered":"2023-11-06T22:16:38.688132+0000","last_unstale":"2023-11-06T22:16:39.160103+0000","last_undegraded":"2023-11-06T22:16:39.160103+0000","last_fullsized":"2023-11-06T22:16:39.160103+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.180440+0000","last_change":"2023-11-06T22:16:39.178019+0000","last_active":"2023-11-06T22:16:39.180440+0000","last_peered":"2023-11-06T22:16:39.180440+0000","last_clean":"2023-11-06T22:16:39.180440+0000","last_became_active":"2023-11-06T22:16:39.177652+0000","last_became_peered":"2023-11-06T22:16:39.177652+0000","last_unstale":"2023-11-06T22:16:39.180440+0000","last_undegraded":"2023-11-06T22:16:39.180440+0000","last_fullsized":"2023-11-06T22:16:39.180440+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.171613+0000","last_change":"2023-11-06T22:16:38.684617+0000","last_active":"2023-11-06T22:16:39.171613+0000","last_peered":"2023-11-06T22:16:39.171613+0000","last_clean":"2023-11-06T22:16:39.171613+0000","last_became_active":"2023-11-06T22:16:38.684366+0000","last_became_peered":"2023-11-06T22:16:38.684366+0000","last_unstale":"2023-11-06T22:16:39.171613+0000","last_undegraded":"2023-11-06T22:16:39.171613+0000","last_fullsized":"2023-11-06T22:16:39.171613+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.188218+0000","last_change":"2023-11-06T22:16:39.180546+0000","last_active":"2023-11-06T22:16:39.188218+0000","last_peered":"2023-11-06T22:16:39.188218+0000","last_clean":"2023-11-06T22:16:39.188218+0000","last_became_active":"2023-11-06T22:16:39.179894+0000","last_became_peered":"2023-11-06T22:16:39.179894+0000","last_unstale":"2023-11-06T22:16:39.188218+0000","last_undegraded":"2023-11-06T22:16:39.188218+0000","last_fullsized":"2023-11-06T22:16:39.188218+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.160295+0000","last_change":"2023-11-06T22:16:38.686342+0000","last_active":"2023-11-06T22:16:39.160295+0000","last_peered":"2023-11-06T22:16:39.160295+0000","last_clean":"2023-11-06T22:16:39.160295+0000","last_became_active":"2023-11-06T22:16:38.685632+0000","last_became_peered":"2023-11-06T22:16:38.685632+0000","last_unstale":"2023-11-06T22:16:39.160295+0000","last_undegraded":"2023-11-06T22:16:39.160295+0000","last_fullsized":"2023-11-06T22:16:39.160295+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.187540+0000","last_change":"2023-11-06T22:16:39.180008+0000","last_active":"2023-11-06T22:16:39.187540+0000","last_peered":"2023-11-06T22:16:39.187540+0000","last_clean":"2023-11-06T22:16:39.187540+0000","last_became_active":"2023-11-06T22:16:39.178723+0000","last_became_peered":"2023-11-06T22:16:39.178723+0000","last_unstale":"2023-11-06T22:16:39.187540+0000","last_undegraded":"2023-11-06T22:16:39.187540+0000","last_fullsized":"2023-11-06T22:16:39.187540+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.172783+0000","last_change":"2023-11-06T22:16:38.694328+0000","last_active":"2023-11-06T22:16:39.172783+0000","last_peered":"2023-11-06T22:16:39.172783+0000","last_clean":"2023-11-06T22:16:39.172783+0000","last_became_active":"2023-11-06T22:16:38.692213+0000","last_became_peered":"2023-11-06T22:16:38.692213+0000","last_unstale":"2023-11-06T22:16:39.172783+0000","last_undegraded":"2023-11-06T22:16:39.172783+0000","last_fullsized":"2023-11-06T22:16:39.172783+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.180086+0000","last_change":"2023-11-06T22:16:39.169480+0000","last_active":"2023-11-06T22:16:39.180086+0000","last_peered":"2023-11-06T22:16:39.180086+0000","last_clean":"2023-11-06T22:16:39.180086+0000","last_became_active":"2023-11-06T22:16:39.169140+0000","last_became_peered":"2023-11-06T22:16:39.169140+0000","last_unstale":"2023-11-06T22:16:39.180086+0000","last_undegraded":"2023-11-06T22:16:39.180086+0000","last_fullsized":"2023-11-06T22:16:39.180086+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.152021+0000","last_change":"2023-11-06T22:16:38.688944+0000","last_active":"2023-11-06T22:16:39.152021+0000","last_peered":"2023-11-06T22:16:39.152021+0000","last_clean":"2023-11-06T22:16:39.152021+0000","last_became_active":"2023-11-06T22:16:38.684576+0000","last_became_peered":"2023-11-06T22:16:38.684576+0000","last_unstale":"2023-11-06T22:16:39.152021+0000","last_undegraded":"2023-11-06T22:16:39.152021+0000","last_fullsized":"2023-11-06T22:16:39.152021+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.183471+0000","last_change":"2023-11-06T22:16:39.178655+0000","last_active":"2023-11-06T22:16:39.183471+0000","last_peered":"2023-11-06T22:16:39.183471+0000","last_clean":"2023-11-06T22:16:39.183471+0000","last_became_active":"2023-11-06T22:16:39.178337+0000","last_became_peered":"2023-11-06T22:16:39.178337+0000","last_unstale":"2023-11-06T22:16:39.183471+0000","last_undegraded":"2023-11-06T22:16:39.183471+0000","last_fullsized":"2023-11-06T22:16:39.183471+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.188070+0000","last_change":"2023-11-06T22:16:38.693668+0000","last_active":"2023-11-06T22:16:39.188070+0000","last_peered":"2023-11-06T22:16:39.188070+0000","last_clean":"2023-11-06T22:16:39.188070+0000","last_became_active":"2023-11-06T22:16:38.693395+0000","last_became_peered":"2023-11-06T22:16:38.693395+0000","last_unstale":"2023-11-06T22:16:39.188070+0000","last_undegraded":"2023-11-06T22:16:39.188070+0000","last_fullsized":"2023-11-06T22:16:39.188070+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.171183+0000","last_change":"2023-11-06T22:16:39.165189+0000","last_active":"2023-11-06T22:16:39.171183+0000","last_peered":"2023-11-06T22:16:39.171183+0000","last_clean":"2023-11-06T22:16:39.171183+0000","last_became_active":"2023-11-06T22:16:39.164872+0000","last_became_peered":"2023-11-06T22:16:39.164872+0000","last_unstale":"2023-11-06T22:16:39.171183+0000","last_undegraded":"2023-11-06T22:16:39.171183+0000","last_fullsized":"2023-11-06T22:16:39.171183+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.177663+0000","last_change":"2023-11-06T22:16:38.683846+0000","last_active":"2023-11-06T22:16:39.177663+0000","last_peered":"2023-11-06T22:16:39.177663+0000","last_clean":"2023-11-06T22:16:39.177663+0000","last_became_active":"2023-11-06T22:16:38.683524+0000","last_became_peered":"2023-11-06T22:16:38.683524+0000","last_unstale":"2023-11-06T22:16:39.177663+0000","last_undegraded":"2023-11-06T22:16:39.177663+0000","last_fullsized":"2023-11-06T22:16:39.177663+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.179325+0000","last_change":"2023-11-06T22:16:39.168526+0000","last_active":"2023-11-06T22:16:39.179325+0000","last_peered":"2023-11-06T22:16:39.179325+0000","last_clean":"2023-11-06T22:16:39.179325+0000","last_became_active":"2023-11-06T22:16:39.168148+0000","last_became_peered":"2023-11-06T22:16:39.168148+0000","last_unstale":"2023-11-06T22:16:39.179325+0000","last_undegraded":"2023-11-06T22:16:39.179325+0000","last_fullsized":"2023-11-06T22:16:39.179325+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1d","version":"203'2","reported_seq":12,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.177017+0000","last_change":"2023-11-06T22:16:38.696846+0000","last_active":"2023-11-06T22:16:39.177017+0000","last_peered":"2023-11-06T22:16:39.177017+0000","last_clean":"2023-11-06T22:16:39.177017+0000","last_became_active":"2023-11-06T22:16:38.695942+0000","last_became_peered":"2023-11-06T22:16:38.695942+0000","last_unstale":"2023-11-06T22:16:39.177017+0000","last_undegraded":"2023-11-06T22:16:39.177017+0000","last_fullsized":"2023-11-06T22:16:39.177017+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.175433+0000","last_change":"2023-11-06T22:16:39.167819+0000","last_active":"2023-11-06T22:16:39.175433+0000","last_peered":"2023-11-06T22:16:39.175433+0000","last_clean":"2023-11-06T22:16:39.175433+0000","last_became_active":"2023-11-06T22:16:39.167017+0000","last_became_peered":"2023-11-06T22:16:39.167017+0000","last_unstale":"2023-11-06T22:16:39.175433+0000","last_undegraded":"2023-11-06T22:16:39.175433+0000","last_fullsized":"2023-11-06T22:16:39.175433+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.154408+0000","last_change":"2023-11-06T22:16:38.680418+0000","last_active":"2023-11-06T22:16:39.154408+0000","last_peered":"2023-11-06T22:16:39.154408+0000","last_clean":"2023-11-06T22:16:39.154408+0000","last_became_active":"2023-11-06T22:16:38.680144+0000","last_became_peered":"2023-11-06T22:16:38.680144+0000","last_unstale":"2023-11-06T22:16:39.154408+0000","last_undegraded":"2023-11-06T22:16:39.154408+0000","last_fullsized":"2023-11-06T22:16:39.154408+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.175320+0000","last_change":"2023-11-06T22:16:39.169369+0000","last_active":"2023-11-06T22:16:39.175320+0000","last_peered":"2023-11-06T22:16:39.175320+0000","last_clean":"2023-11-06T22:16:39.175320+0000","last_became_active":"2023-11-06T22:16:39.168849+0000","last_became_peered":"2023-11-06T22:16:39.168849+0000","last_unstale":"2023-11-06T22:16:39.175320+0000","last_undegraded":"2023-11-06T22:16:39.175320+0000","last_fullsized":"2023-11-06T22:16:39.175320+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.154458+0000","last_change":"2023-11-06T22:16:38.683508+0000","last_active":"2023-11-06T22:16:39.154458+0000","last_peered":"2023-11-06T22:16:39.154458+0000","last_clean":"2023-11-06T22:16:39.154458+0000","last_became_active":"2023-11-06T22:16:38.683235+0000","last_became_peered":"2023-11-06T22:16:38.683235+0000","last_unstale":"2023-11-06T22:16:39.154458+0000","last_undegraded":"2023-11-06T22:16:39.154458+0000","last_fullsized":"2023-11-06T22:16:39.154458+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.170281+0000","last_change":"2023-11-06T22:16:39.166173+0000","last_active":"2023-11-06T22:16:39.170281+0000","last_peered":"2023-11-06T22:16:39.170281+0000","last_clean":"2023-11-06T22:16:39.170281+0000","last_became_active":"2023-11-06T22:16:39.165896+0000","last_became_peered":"2023-11-06T22:16:39.165896+0000","last_unstale":"2023-11-06T22:16:39.170281+0000","last_undegraded":"2023-11-06T22:16:39.170281+0000","last_fullsized":"2023-11-06T22:16:39.170281+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.158480+0000","last_change":"2023-11-06T22:16:38.683907+0000","last_active":"2023-11-06T22:16:39.158480+0000","last_peered":"2023-11-06T22:16:39.158480+0000","last_clean":"2023-11-06T22:16:39.158480+0000","last_became_active":"2023-11-06T22:16:38.683619+0000","last_became_peered":"2023-11-06T22:16:38.683619+0000","last_unstale":"2023-11-06T22:16:39.158480+0000","last_undegraded":"2023-11-06T22:16:39.158480+0000","last_fullsized":"2023-11-06T22:16:39.158480+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.180518+0000","last_change":"2023-11-06T22:16:39.175179+0000","last_active":"2023-11-06T22:16:39.180518+0000","last_peered":"2023-11-06T22:16:39.180518+0000","last_clean":"2023-11-06T22:16:39.180518+0000","last_became_active":"2023-11-06T22:16:39.169790+0000","last_became_peered":"2023-11-06T22:16:39.169790+0000","last_unstale":"2023-11-06T22:16:39.180518+0000","last_undegraded":"2023-11-06T22:16:39.180518+0000","last_fullsized":"2023-11-06T22:16:39.180518+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.156915+0000","last_change":"2023-11-06T22:16:38.678978+0000","last_active":"2023-11-06T22:16:39.156915+0000","last_peered":"2023-11-06T22:16:39.156915+0000","last_clean":"2023-11-06T22:16:39.156915+0000","last_became_active":"2023-11-06T22:16:38.678651+0000","last_became_peered":"2023-11-06T22:16:38.678651+0000","last_unstale":"2023-11-06T22:16:39.156915+0000","last_undegraded":"2023-11-06T22:16:39.156915+0000","last_fullsized":"2023-11-06T22:16:39.156915+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.186849+0000","last_change":"2023-11-06T22:16:39.176565+0000","last_active":"2023-11-06T22:16:39.186849+0000","last_peered":"2023-11-06T22:16:39.186849+0000","last_clean":"2023-11-06T22:16:39.186849+0000","last_became_active":"2023-11-06T22:16:39.175687+0000","last_became_peered":"2023-11-06T22:16:39.175687+0000","last_unstale":"2023-11-06T22:16:39.186849+0000","last_undegraded":"2023-11-06T22:16:39.186849+0000","last_fullsized":"2023-11-06T22:16:39.186849+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.163594+0000","last_change":"2023-11-06T22:16:38.675389+0000","last_active":"2023-11-06T22:16:39.163594+0000","last_peered":"2023-11-06T22:16:39.163594+0000","last_clean":"2023-11-06T22:16:39.163594+0000","last_became_active":"2023-11-06T22:16:38.675034+0000","last_became_peered":"2023-11-06T22:16:38.675034+0000","last_unstale":"2023-11-06T22:16:39.163594+0000","last_undegraded":"2023-11-06T22:16:39.163594+0000","last_fullsized":"2023-11-06T22:16:39.163594+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182480+0000","last_change":"2023-11-06T22:16:39.179177+0000","last_active":"2023-11-06T22:16:39.182480+0000","last_peered":"2023-11-06T22:16:39.182480+0000","last_clean":"2023-11-06T22:16:39.182480+0000","last_became_active":"2023-11-06T22:16:39.176354+0000","last_became_peered":"2023-11-06T22:16:39.176354+0000","last_unstale":"2023-11-06T22:16:39.182480+0000","last_undegraded":"2023-11-06T22:16:39.182480+0000","last_fullsized":"2023-11-06T22:16:39.182480+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.f","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.185648+0000","last_change":"2023-11-06T22:16:38.681685+0000","last_active":"2023-11-06T22:16:39.185648+0000","last_peered":"2023-11-06T22:16:39.185648+0000","last_clean":"2023-11-06T22:16:39.185648+0000","last_became_active":"2023-11-06T22:16:38.681179+0000","last_became_peered":"2023-11-06T22:16:38.681179+0000","last_unstale":"2023-11-06T22:16:39.185648+0000","last_undegraded":"2023-11-06T22:16:39.185648+0000","last_fullsized":"2023-11-06T22:16:39.185648+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.173820+0000","last_change":"2023-11-06T22:16:39.171069+0000","last_active":"2023-11-06T22:16:39.173820+0000","last_peered":"2023-11-06T22:16:39.173820+0000","last_clean":"2023-11-06T22:16:39.173820+0000","last_became_active":"2023-11-06T22:16:39.170807+0000","last_became_peered":"2023-11-06T22:16:39.170807+0000","last_unstale":"2023-11-06T22:16:39.173820+0000","last_undegraded":"2023-11-06T22:16:39.173820+0000","last_fullsized":"2023-11-06T22:16:39.173820+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"203'2","reported_seq":12,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.183327+0000","last_change":"2023-11-06T22:16:38.698135+0000","last_active":"2023-11-06T22:16:39.183327+0000","last_peered":"2023-11-06T22:16:39.183327+0000","last_clean":"2023-11-06T22:16:39.183327+0000","last_became_active":"2023-11-06T22:16:38.697792+0000","last_became_peered":"2023-11-06T22:16:38.697792+0000","last_unstale":"2023-11-06T22:16:39.183327+0000","last_undegraded":"2023-11-06T22:16:39.183327+0000","last_fullsized":"2023-11-06T22:16:39.183327+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.186928+0000","last_change":"2023-11-06T22:16:39.183317+0000","last_active":"2023-11-06T22:16:39.186928+0000","last_peered":"2023-11-06T22:16:39.186928+0000","last_clean":"2023-11-06T22:16:39.186928+0000","last_became_active":"2023-11-06T22:16:39.183030+0000","last_became_peered":"2023-11-06T22:16:39.183030+0000","last_unstale":"2023-11-06T22:16:39.186928+0000","last_undegraded":"2023-11-06T22:16:39.186928+0000","last_fullsized":"2023-11-06T22:16:39.186928+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.170796+0000","last_change":"2023-11-06T22:16:38.687430+0000","last_active":"2023-11-06T22:16:39.170796+0000","last_peered":"2023-11-06T22:16:39.170796+0000","last_clean":"2023-11-06T22:16:39.170796+0000","last_became_active":"2023-11-06T22:16:38.687129+0000","last_became_peered":"2023-11-06T22:16:38.687129+0000","last_unstale":"2023-11-06T22:16:39.170796+0000","last_undegraded":"2023-11-06T22:16:39.170796+0000","last_fullsized":"2023-11-06T22:16:39.170796+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182156+0000","last_change":"2023-11-06T22:16:39.177725+0000","last_active":"2023-11-06T22:16:39.182156+0000","last_peered":"2023-11-06T22:16:39.182156+0000","last_clean":"2023-11-06T22:16:39.182156+0000","last_became_active":"2023-11-06T22:16:39.177049+0000","last_became_peered":"2023-11-06T22:16:39.177049+0000","last_unstale":"2023-11-06T22:16:39.182156+0000","last_undegraded":"2023-11-06T22:16:39.182156+0000","last_fullsized":"2023-11-06T22:16:39.182156+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.156006+0000","last_change":"2023-11-06T22:16:38.692114+0000","last_active":"2023-11-06T22:16:39.156006+0000","last_peered":"2023-11-06T22:16:39.156006+0000","last_clean":"2023-11-06T22:16:39.156006+0000","last_became_active":"2023-11-06T22:16:38.690208+0000","last_became_peered":"2023-11-06T22:16:38.690208+0000","last_unstale":"2023-11-06T22:16:39.156006+0000","last_undegraded":"2023-11-06T22:16:39.156006+0000","last_fullsized":"2023-11-06T22:16:39.156006+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.175341+0000","last_change":"2023-11-06T22:16:39.170283+0000","last_active":"2023-11-06T22:16:39.175341+0000","last_peered":"2023-11-06T22:16:39.175341+0000","last_clean":"2023-11-06T22:16:39.175341+0000","last_became_active":"2023-11-06T22:16:39.169875+0000","last_became_peered":"2023-11-06T22:16:39.169875+0000","last_unstale":"2023-11-06T22:16:39.175341+0000","last_undegraded":"2023-11-06T22:16:39.175341+0000","last_fullsized":"2023-11-06T22:16:39.175341+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.b","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.183202+0000","last_change":"2023-11-06T22:16:38.691051+0000","last_active":"2023-11-06T22:16:39.183202+0000","last_peered":"2023-11-06T22:16:39.183202+0000","last_clean":"2023-11-06T22:16:39.183202+0000","last_became_active":"2023-11-06T22:16:38.690525+0000","last_became_peered":"2023-11-06T22:16:38.690525+0000","last_unstale":"2023-11-06T22:16:39.183202+0000","last_undegraded":"2023-11-06T22:16:39.183202+0000","last_fullsized":"2023-11-06T22:16:39.183202+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.185236+0000","last_change":"2023-11-06T22:16:39.183192+0000","last_active":"2023-11-06T22:16:39.185236+0000","last_peered":"2023-11-06T22:16:39.185236+0000","last_clean":"2023-11-06T22:16:39.185236+0000","last_became_active":"2023-11-06T22:16:39.182818+0000","last_became_peered":"2023-11-06T22:16:39.182818+0000","last_unstale":"2023-11-06T22:16:39.185236+0000","last_undegraded":"2023-11-06T22:16:39.185236+0000","last_fullsized":"2023-11-06T22:16:39.185236+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.171274+0000","last_change":"2023-11-06T22:16:38.682687+0000","last_active":"2023-11-06T22:16:39.171274+0000","last_peered":"2023-11-06T22:16:39.171274+0000","last_clean":"2023-11-06T22:16:39.171274+0000","last_became_active":"2023-11-06T22:16:38.682394+0000","last_became_peered":"2023-11-06T22:16:38.682394+0000","last_unstale":"2023-11-06T22:16:39.171274+0000","last_undegraded":"2023-11-06T22:16:39.171274+0000","last_fullsized":"2023-11-06T22:16:39.171274+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.184027+0000","last_change":"2023-11-06T22:16:39.182180+0000","last_active":"2023-11-06T22:16:39.184027+0000","last_peered":"2023-11-06T22:16:39.184027+0000","last_clean":"2023-11-06T22:16:39.184027+0000","last_became_active":"2023-11-06T22:16:39.181886+0000","last_became_peered":"2023-11-06T22:16:39.181886+0000","last_unstale":"2023-11-06T22:16:39.184027+0000","last_undegraded":"2023-11-06T22:16:39.184027+0000","last_fullsized":"2023-11-06T22:16:39.184027+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.160315+0000","last_change":"2023-11-06T22:16:38.687753+0000","last_active":"2023-11-06T22:16:39.160315+0000","last_peered":"2023-11-06T22:16:39.160315+0000","last_clean":"2023-11-06T22:16:39.160315+0000","last_became_active":"2023-11-06T22:16:38.687443+0000","last_became_peered":"2023-11-06T22:16:38.687443+0000","last_unstale":"2023-11-06T22:16:39.160315+0000","last_undegraded":"2023-11-06T22:16:39.160315+0000","last_fullsized":"2023-11-06T22:16:39.160315+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.183539+0000","last_change":"2023-11-06T22:16:39.166103+0000","last_active":"2023-11-06T22:16:39.183539+0000","last_peered":"2023-11-06T22:16:39.183539+0000","last_clean":"2023-11-06T22:16:39.183539+0000","last_became_active":"2023-11-06T22:16:39.165765+0000","last_became_peered":"2023-11-06T22:16:39.165765+0000","last_unstale":"2023-11-06T22:16:39.183539+0000","last_undegraded":"2023-11-06T22:16:39.183539+0000","last_fullsized":"2023-11-06T22:16:39.183539+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.172087+0000","last_change":"2023-11-06T22:16:38.683770+0000","last_active":"2023-11-06T22:16:39.172087+0000","last_peered":"2023-11-06T22:16:39.172087+0000","last_clean":"2023-11-06T22:16:39.172087+0000","last_became_active":"2023-11-06T22:16:38.682682+0000","last_became_peered":"2023-11-06T22:16:38.682682+0000","last_unstale":"2023-11-06T22:16:39.172087+0000","last_undegraded":"2023-11-06T22:16:39.172087+0000","last_fullsized":"2023-11-06T22:16:39.172087+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.186807+0000","last_change":"2023-11-06T22:16:39.176283+0000","last_active":"2023-11-06T22:16:39.186807+0000","last_peered":"2023-11-06T22:16:39.186807+0000","last_clean":"2023-11-06T22:16:39.186807+0000","last_became_active":"2023-11-06T22:16:39.175905+0000","last_became_peered":"2023-11-06T22:16:39.175905+0000","last_unstale":"2023-11-06T22:16:39.186807+0000","last_undegraded":"2023-11-06T22:16:39.186807+0000","last_fullsized":"2023-11-06T22:16:39.186807+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"203'2","reported_seq":12,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.185244+0000","last_change":"2023-11-06T22:16:38.684716+0000","last_active":"2023-11-06T22:16:39.185244+0000","last_peered":"2023-11-06T22:16:39.185244+0000","last_clean":"2023-11-06T22:16:39.185244+0000","last_became_active":"2023-11-06T22:16:38.684449+0000","last_became_peered":"2023-11-06T22:16:38.684449+0000","last_unstale":"2023-11-06T22:16:39.185244+0000","last_undegraded":"2023-11-06T22:16:39.185244+0000","last_fullsized":"2023-11-06T22:16:39.185244+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182424+0000","last_change":"2023-11-06T22:16:39.179400+0000","last_active":"2023-11-06T22:16:39.182424+0000","last_peered":"2023-11-06T22:16:39.182424+0000","last_clean":"2023-11-06T22:16:39.182424+0000","last_became_active":"2023-11-06T22:16:39.177060+0000","last_became_peered":"2023-11-06T22:16:39.177060+0000","last_unstale":"2023-11-06T22:16:39.182424+0000","last_undegraded":"2023-11-06T22:16:39.182424+0000","last_fullsized":"2023-11-06T22:16:39.182424+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.167607+0000","last_change":"2023-11-06T22:16:38.690230+0000","last_active":"2023-11-06T22:16:39.167607+0000","last_peered":"2023-11-06T22:16:39.167607+0000","last_clean":"2023-11-06T22:16:39.167607+0000","last_became_active":"2023-11-06T22:16:38.681801+0000","last_became_peered":"2023-11-06T22:16:38.681801+0000","last_unstale":"2023-11-06T22:16:39.167607+0000","last_undegraded":"2023-11-06T22:16:39.167607+0000","last_fullsized":"2023-11-06T22:16:39.167607+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.184552+0000","last_change":"2023-11-06T22:16:39.183226+0000","last_active":"2023-11-06T22:16:39.184552+0000","last_peered":"2023-11-06T22:16:39.184552+0000","last_clean":"2023-11-06T22:16:39.184552+0000","last_became_active":"2023-11-06T22:16:39.182930+0000","last_became_peered":"2023-11-06T22:16:39.182930+0000","last_unstale":"2023-11-06T22:16:39.184552+0000","last_undegraded":"2023-11-06T22:16:39.184552+0000","last_fullsized":"2023-11-06T22:16:39.184552+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.154491+0000","last_change":"2023-11-06T22:16:38.690481+0000","last_active":"2023-11-06T22:16:39.154491+0000","last_peered":"2023-11-06T22:16:39.154491+0000","last_clean":"2023-11-06T22:16:39.154491+0000","last_became_active":"2023-11-06T22:16:38.690145+0000","last_became_peered":"2023-11-06T22:16:38.690145+0000","last_unstale":"2023-11-06T22:16:39.154491+0000","last_undegraded":"2023-11-06T22:16:39.154491+0000","last_fullsized":"2023-11-06T22:16:39.154491+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.163908+0000","last_change":"2023-11-06T22:16:39.161097+0000","last_active":"2023-11-06T22:16:39.163908+0000","last_peered":"2023-11-06T22:16:39.163908+0000","last_clean":"2023-11-06T22:16:39.163908+0000","last_became_active":"2023-11-06T22:16:39.160807+0000","last_became_peered":"2023-11-06T22:16:39.160807+0000","last_unstale":"2023-11-06T22:16:39.163908+0000","last_undegraded":"2023-11-06T22:16:39.163908+0000","last_fullsized":"2023-11-06T22:16:39.163908+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.156108+0000","last_change":"2023-11-06T22:16:38.684191+0000","last_active":"2023-11-06T22:16:39.156108+0000","last_peered":"2023-11-06T22:16:39.156108+0000","last_clean":"2023-11-06T22:16:39.156108+0000","last_became_active":"2023-11-06T22:16:38.683317+0000","last_became_peered":"2023-11-06T22:16:38.683317+0000","last_unstale":"2023-11-06T22:16:39.156108+0000","last_undegraded":"2023-11-06T22:16:39.156108+0000","last_fullsized":"2023-11-06T22:16:39.156108+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182564+0000","last_change":"2023-11-06T22:16:39.164031+0000","last_active":"2023-11-06T22:16:39.182564+0000","last_peered":"2023-11-06T22:16:39.182564+0000","last_clean":"2023-11-06T22:16:39.182564+0000","last_became_active":"2023-11-06T22:16:39.163724+0000","last_became_peered":"2023-11-06T22:16:39.163724+0000","last_unstale":"2023-11-06T22:16:39.182564+0000","last_undegraded":"2023-11-06T22:16:39.182564+0000","last_fullsized":"2023-11-06T22:16:39.182564+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.171491+0000","last_change":"2023-11-06T22:16:38.687442+0000","last_active":"2023-11-06T22:16:39.171491+0000","last_peered":"2023-11-06T22:16:39.171491+0000","last_clean":"2023-11-06T22:16:39.171491+0000","last_became_active":"2023-11-06T22:16:38.687104+0000","last_became_peered":"2023-11-06T22:16:38.687104+0000","last_unstale":"2023-11-06T22:16:39.171491+0000","last_undegraded":"2023-11-06T22:16:39.171491+0000","last_fullsized":"2023-11-06T22:16:39.171491+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.187580+0000","last_change":"2023-11-06T22:16:39.183935+0000","last_active":"2023-11-06T22:16:39.187580+0000","last_peered":"2023-11-06T22:16:39.187580+0000","last_clean":"2023-11-06T22:16:39.187580+0000","last_became_active":"2023-11-06T22:16:39.183660+0000","last_became_peered":"2023-11-06T22:16:39.183660+0000","last_unstale":"2023-11-06T22:16:39.187580+0000","last_undegraded":"2023-11-06T22:16:39.187580+0000","last_fullsized":"2023-11-06T22:16:39.187580+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.152526+0000","last_change":"2023-11-06T22:16:38.692176+0000","last_active":"2023-11-06T22:16:39.152526+0000","last_peered":"2023-11-06T22:16:39.152526+0000","last_clean":"2023-11-06T22:16:39.152526+0000","last_became_active":"2023-11-06T22:16:38.690162+0000","last_became_peered":"2023-11-06T22:16:38.690162+0000","last_unstale":"2023-11-06T22:16:39.152526+0000","last_undegraded":"2023-11-06T22:16:39.152526+0000","last_fullsized":"2023-11-06T22:16:39.152526+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.170665+0000","last_change":"2023-11-06T22:16:39.165948+0000","last_active":"2023-11-06T22:16:39.170665+0000","last_peered":"2023-11-06T22:16:39.170665+0000","last_clean":"2023-11-06T22:16:39.170665+0000","last_became_active":"2023-11-06T22:16:39.165681+0000","last_became_peered":"2023-11-06T22:16:39.165681+0000","last_unstale":"2023-11-06T22:16:39.170665+0000","last_undegraded":"2023-11-06T22:16:39.170665+0000","last_fullsized":"2023-11-06T22:16:39.170665+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.172909+0000","last_change":"2023-11-06T22:16:38.684206+0000","last_active":"2023-11-06T22:16:39.172909+0000","last_peered":"2023-11-06T22:16:39.172909+0000","last_clean":"2023-11-06T22:16:39.172909+0000","last_became_active":"2023-11-06T22:16:38.683872+0000","last_became_peered":"2023-11-06T22:16:38.683872+0000","last_unstale":"2023-11-06T22:16:39.172909+0000","last_undegraded":"2023-11-06T22:16:39.172909+0000","last_fullsized":"2023-11-06T22:16:39.172909+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.176947+0000","last_change":"2023-11-06T22:16:39.174011+0000","last_active":"2023-11-06T22:16:39.176947+0000","last_peered":"2023-11-06T22:16:39.176947+0000","last_clean":"2023-11-06T22:16:39.176947+0000","last_became_active":"2023-11-06T22:16:39.171928+0000","last_became_peered":"2023-11-06T22:16:39.171928+0000","last_unstale":"2023-11-06T22:16:39.176947+0000","last_undegraded":"2023-11-06T22:16:39.176947+0000","last_fullsized":"2023-11-06T22:16:39.176947+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.159959+0000","last_change":"2023-11-06T22:16:38.694589+0000","last_active":"2023-11-06T22:16:39.159959+0000","last_peered":"2023-11-06T22:16:39.159959+0000","last_clean":"2023-11-06T22:16:39.159959+0000","last_became_active":"2023-11-06T22:16:38.694320+0000","last_became_peered":"2023-11-06T22:16:38.694320+0000","last_unstale":"2023-11-06T22:16:39.159959+0000","last_undegraded":"2023-11-06T22:16:39.159959+0000","last_fullsized":"2023-11-06T22:16:39.159959+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.176726+0000","last_change":"2023-11-06T22:16:39.170833+0000","last_active":"2023-11-06T22:16:39.176726+0000","last_peered":"2023-11-06T22:16:39.176726+0000","last_clean":"2023-11-06T22:16:39.176726+0000","last_became_active":"2023-11-06T22:16:39.170552+0000","last_became_peered":"2023-11-06T22:16:39.170552+0000","last_unstale":"2023-11-06T22:16:39.176726+0000","last_undegraded":"2023-11-06T22:16:39.176726+0000","last_fullsized":"2023-11-06T22:16:39.176726+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.160062+0000","last_change":"2023-11-06T22:16:38.695640+0000","last_active":"2023-11-06T22:16:39.160062+0000","last_peered":"2023-11-06T22:16:39.160062+0000","last_clean":"2023-11-06T22:16:39.160062+0000","last_became_active":"2023-11-06T22:16:38.695286+0000","last_became_peered":"2023-11-06T22:16:38.695286+0000","last_unstale":"2023-11-06T22:16:39.160062+0000","last_undegraded":"2023-11-06T22:16:39.160062+0000","last_fullsized":"2023-11-06T22:16:39.160062+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":173,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.164790+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T22:16:39.164790+0000","last_peered":"2023-11-06T22:16:39.164790+0000","last_clean":"2023-11-06T22:16:39.164790+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:16:39.164790+0000","last_undegraded":"2023-11-06T22:16:39.164790+0000","last_fullsized":"2023-11-06T22:16:39.164790+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.183454+0000","last_change":"2023-11-06T22:16:39.178087+0000","last_active":"2023-11-06T22:16:39.183454+0000","last_peered":"2023-11-06T22:16:39.183454+0000","last_clean":"2023-11-06T22:16:39.183454+0000","last_became_active":"2023-11-06T22:16:39.177595+0000","last_became_peered":"2023-11-06T22:16:39.177595+0000","last_unstale":"2023-11-06T22:16:39.183454+0000","last_undegraded":"2023-11-06T22:16:39.183454+0000","last_fullsized":"2023-11-06T22:16:39.183454+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.160752+0000","last_change":"2023-11-06T22:16:38.692384+0000","last_active":"2023-11-06T22:16:39.160752+0000","last_peered":"2023-11-06T22:16:39.160752+0000","last_clean":"2023-11-06T22:16:39.160752+0000","last_became_active":"2023-11-06T22:16:38.692115+0000","last_became_peered":"2023-11-06T22:16:38.692115+0000","last_unstale":"2023-11-06T22:16:39.160752+0000","last_undegraded":"2023-11-06T22:16:39.160752+0000","last_fullsized":"2023-11-06T22:16:39.160752+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":184,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.152940+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T22:16:39.152940+0000","last_peered":"2023-11-06T22:16:39.152940+0000","last_clean":"2023-11-06T22:16:39.152940+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:16:39.152940+0000","last_undegraded":"2023-11-06T22:16:39.152940+0000","last_fullsized":"2023-11-06T22:16:39.152940+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.176793+0000","last_change":"2023-11-06T22:16:39.173368+0000","last_active":"2023-11-06T22:16:39.176793+0000","last_peered":"2023-11-06T22:16:39.176793+0000","last_clean":"2023-11-06T22:16:39.176793+0000","last_became_active":"2023-11-06T22:16:39.172645+0000","last_became_peered":"2023-11-06T22:16:39.172645+0000","last_unstale":"2023-11-06T22:16:39.176793+0000","last_undegraded":"2023-11-06T22:16:39.176793+0000","last_fullsized":"2023-11-06T22:16:39.176793+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"203'2","reported_seq":12,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.181769+0000","last_change":"2023-11-06T22:16:38.690659+0000","last_active":"2023-11-06T22:16:39.181769+0000","last_peered":"2023-11-06T22:16:39.181769+0000","last_clean":"2023-11-06T22:16:39.181769+0000","last_became_active":"2023-11-06T22:16:38.686550+0000","last_became_peered":"2023-11-06T22:16:38.686550+0000","last_unstale":"2023-11-06T22:16:39.181769+0000","last_undegraded":"2023-11-06T22:16:39.181769+0000","last_fullsized":"2023-11-06T22:16:39.181769+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":192,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.153444+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T22:16:39.153444+0000","last_peered":"2023-11-06T22:16:39.153444+0000","last_clean":"2023-11-06T22:16:39.153444+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:16:39.153444+0000","last_undegraded":"2023-11-06T22:16:39.153444+0000","last_fullsized":"2023-11-06T22:16:39.153444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.188402+0000","last_change":"2023-11-06T22:16:39.178242+0000","last_active":"2023-11-06T22:16:39.188402+0000","last_peered":"2023-11-06T22:16:39.188402+0000","last_clean":"2023-11-06T22:16:39.188402+0000","last_became_active":"2023-11-06T22:16:39.177946+0000","last_became_peered":"2023-11-06T22:16:39.177946+0000","last_unstale":"2023-11-06T22:16:39.188402+0000","last_undegraded":"2023-11-06T22:16:39.188402+0000","last_fullsized":"2023-11-06T22:16:39.188402+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.161103+0000","last_change":"2023-11-06T22:16:38.695670+0000","last_active":"2023-11-06T22:16:39.161103+0000","last_peered":"2023-11-06T22:16:39.161103+0000","last_clean":"2023-11-06T22:16:39.161103+0000","last_became_active":"2023-11-06T22:16:38.695359+0000","last_became_peered":"2023-11-06T22:16:38.695359+0000","last_unstale":"2023-11-06T22:16:39.161103+0000","last_undegraded":"2023-11-06T22:16:39.161103+0000","last_fullsized":"2023-11-06T22:16:39.161103+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":184,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.153226+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T22:16:39.153226+0000","last_peered":"2023-11-06T22:16:39.153226+0000","last_clean":"2023-11-06T22:16:39.153226+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:16:39.153226+0000","last_undegraded":"2023-11-06T22:16:39.153226+0000","last_fullsized":"2023-11-06T22:16:39.153226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":172,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.164041+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T22:16:39.164041+0000","last_peered":"2023-11-06T22:16:39.164041+0000","last_clean":"2023-11-06T22:16:39.164041+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:16:39.164041+0000","last_undegraded":"2023-11-06T22:16:39.164041+0000","last_fullsized":"2023-11-06T22:16:39.164041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.176621+0000","last_change":"2023-11-06T22:16:39.171632+0000","last_active":"2023-11-06T22:16:39.176621+0000","last_peered":"2023-11-06T22:16:39.176621+0000","last_clean":"2023-11-06T22:16:39.176621+0000","last_became_active":"2023-11-06T22:16:39.171304+0000","last_became_peered":"2023-11-06T22:16:39.171304+0000","last_unstale":"2023-11-06T22:16:39.176621+0000","last_undegraded":"2023-11-06T22:16:39.176621+0000","last_fullsized":"2023-11-06T22:16:39.176621+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"203'3","reported_seq":13,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.186402+0000","last_change":"2023-11-06T22:16:38.687911+0000","last_active":"2023-11-06T22:16:39.186402+0000","last_peered":"2023-11-06T22:16:39.186402+0000","last_clean":"2023-11-06T22:16:39.186402+0000","last_became_active":"2023-11-06T22:16:38.687169+0000","last_became_peered":"2023-11-06T22:16:38.687169+0000","last_unstale":"2023-11-06T22:16:39.186402+0000","last_undegraded":"2023-11-06T22:16:39.186402+0000","last_fullsized":"2023-11-06T22:16:39.186402+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":184,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.152821+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T22:16:39.152821+0000","last_peered":"2023-11-06T22:16:39.152821+0000","last_clean":"2023-11-06T22:16:39.152821+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:16:39.152821+0000","last_undegraded":"2023-11-06T22:16:39.152821+0000","last_fullsized":"2023-11-06T22:16:39.152821+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182007+0000","last_change":"2023-11-06T22:16:39.177469+0000","last_active":"2023-11-06T22:16:39.182007+0000","last_peered":"2023-11-06T22:16:39.182007+0000","last_clean":"2023-11-06T22:16:39.182007+0000","last_became_active":"2023-11-06T22:16:39.177167+0000","last_became_peered":"2023-11-06T22:16:39.177167+0000","last_unstale":"2023-11-06T22:16:39.182007+0000","last_undegraded":"2023-11-06T22:16:39.182007+0000","last_fullsized":"2023-11-06T22:16:39.182007+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.10","version":"203'4","reported_seq":14,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.177182+0000","last_change":"2023-11-06T22:16:38.689267+0000","last_active":"2023-11-06T22:16:39.177182+0000","last_peered":"2023-11-06T22:16:39.177182+0000","last_clean":"2023-11-06T22:16:39.177182+0000","last_became_active":"2023-11-06T22:16:38.688725+0000","last_became_peered":"2023-11-06T22:16:38.688725+0000","last_unstale":"2023-11-06T22:16:39.177182+0000","last_undegraded":"2023-11-06T22:16:39.177182+0000","last_fullsized":"2023-11-06T22:16:39.177182+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":184,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.153156+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:16:39.153156+0000","last_peered":"2023-11-06T22:16:39.153156+0000","last_clean":"2023-11-06T22:16:39.153156+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:16:39.153156+0000","last_undegraded":"2023-11-06T22:16:39.153156+0000","last_fullsized":"2023-11-06T22:16:39.153156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.175498+0000","last_change":"2023-11-06T22:16:39.169007+0000","last_active":"2023-11-06T22:16:39.175498+0000","last_peered":"2023-11-06T22:16:39.175498+0000","last_clean":"2023-11-06T22:16:39.175498+0000","last_became_active":"2023-11-06T22:16:39.168738+0000","last_became_peered":"2023-11-06T22:16:39.168738+0000","last_unstale":"2023-11-06T22:16:39.175498+0000","last_undegraded":"2023-11-06T22:16:39.175498+0000","last_fullsized":"2023-11-06T22:16:39.175498+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.13","version":"203'1","reported_seq":11,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.185455+0000","last_change":"2023-11-06T22:16:38.673087+0000","last_active":"2023-11-06T22:16:39.185455+0000","last_peered":"2023-11-06T22:16:39.185455+0000","last_clean":"2023-11-06T22:16:39.185455+0000","last_became_active":"2023-11-06T22:16:38.672786+0000","last_became_peered":"2023-11-06T22:16:38.672786+0000","last_unstale":"2023-11-06T22:16:39.185455+0000","last_undegraded":"2023-11-06T22:16:39.185455+0000","last_fullsized":"2023-11-06T22:16:39.185455+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":173,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.157756+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T22:16:39.157756+0000","last_peered":"2023-11-06T22:16:39.157756+0000","last_clean":"2023-11-06T22:16:39.157756+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:16:39.157756+0000","last_undegraded":"2023-11-06T22:16:39.157756+0000","last_fullsized":"2023-11-06T22:16:39.157756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.182281+0000","last_change":"2023-11-06T22:16:39.179690+0000","last_active":"2023-11-06T22:16:39.182281+0000","last_peered":"2023-11-06T22:16:39.182281+0000","last_clean":"2023-11-06T22:16:39.182281+0000","last_became_active":"2023-11-06T22:16:39.179186+0000","last_became_peered":"2023-11-06T22:16:39.179186+0000","last_unstale":"2023-11-06T22:16:39.182281+0000","last_undegraded":"2023-11-06T22:16:39.182281+0000","last_fullsized":"2023-11-06T22:16:39.182281+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:38.656491+0000","last_clean_scrub_stamp":"2023-11-06T22:16:38.656491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.160342+0000","last_change":"2023-11-06T22:16:38.695813+0000","last_active":"2023-11-06T22:16:39.160342+0000","last_peered":"2023-11-06T22:16:39.160342+0000","last_clean":"2023-11-06T22:16:39.160342+0000","last_became_active":"2023-11-06T22:16:38.693535+0000","last_became_peered":"2023-11-06T22:16:38.693535+0000","last_unstale":"2023-11-06T22:16:39.160342+0000","last_undegraded":"2023-11-06T22:16:39.160342+0000","last_fullsized":"2023-11-06T22:16:39.160342+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:16:37.654116+0000","last_clean_scrub_stamp":"2023-11-06T22:16:37.654116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":184,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-11-06T22:16:39.155730+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:16:39.155730+0000","last_peered":"2023-11-06T22:16:39.155730+0000","last_clean":"2023-11-06T22:16:39.155730+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:16:39.155730+0000","last_undegraded":"2023-11-06T22:16:39.155730+0000","last_fullsized":"2023-11-06T22:16:39.155730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575950,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307784,"kb_used_data":2496,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064056,"statfs":{"total":96636764160,"available":96321593344,"internally_reserved":0,"allocated":2555904,"data_stored":1308919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.71999999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90700000000000003,"5min":0.97199999999999998,"15min":1.0980000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71499999999999997,"15min":0.72999999999999998},"min":{"1min":0.47599999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.93600000000000005}]},{"osd":1,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70599999999999996,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.48699999999999999,"15min":0.46400000000000002},"max":{"1min":0.90300000000000002,"5min":0.98799999999999999,"15min":1.038},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93200000000000005,"5min":0.96599999999999997,"15min":1.0680000000000001},"last":0.77100000000000002}]},{"osd":2,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.61699999999999999,"5min":0.53100000000000003,"15min":0.502},"max":{"1min":0.98799999999999999,"5min":1.0069999999999999,"15min":1.016},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.52700000000000002,"5min":0.50900000000000001,"15min":0.505},"max":{"1min":0.996,"5min":0.996,"15min":1.0209999999999999},"last":0.85599999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73899999999999999,"15min":0.74099999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.499},"max":{"1min":0.89600000000000002,"5min":0.95799999999999996,"15min":0.96999999999999997},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.76400000000000001,"15min":0.76900000000000002},"min":{"1min":0.58999999999999997,"5min":0.504,"15min":0.504},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.0229999999999999},"last":0.95199999999999996}]},{"osd":4,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.40899999999999997},"max":{"1min":0.88300000000000001,"5min":0.94399999999999995,"15min":0.97199999999999998},"last":0.747},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.47699999999999998,"5min":0.45200000000000001,"15min":0.42099999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63100000000000001}]},{"osd":5,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94099999999999995,"5min":1.008,"15min":1.0389999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.749,"5min":0.70799999999999996,"15min":0.68700000000000006},"min":{"1min":0.501,"5min":0.432,"15min":0.432},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.88}]},{"osd":6,"last update":"Mon Nov 6 22:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.48399999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.55000000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.069},"last":0.71699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307792,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064048,"statfs":{"total":96636764160,"available":96321585152,"internally_reserved":0,"allocated":2564096,"data_stored":1309881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.56599999999999995,"5min":0.501,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.96199999999999997,"15min":1.0569999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.54200000000000004,"5min":0.51800000000000002,"15min":0.434},"max":{"1min":0.87,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.749,"15min":0.73899999999999999},"min":{"1min":0.54600000000000004,"5min":0.44,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":1.0660000000000001,"15min":1.0700000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.45000000000000001},"max":{"1min":0.879,"5min":1.016,"15min":1.0960000000000001},"last":0.75700000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.59999999999999998,"5min":0.498,"15min":0.498},"max":{"1min":0.94699999999999995,"5min":0.97999999999999998,"15min":1.167},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79400000000000004,"15min":0.79500000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.47099999999999997},"max":{"1min":0.93899999999999995,"5min":1.046,"15min":1.1839999999999999},"last":0.68899999999999995}]},{"osd":3,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.41599999999999998},"max":{"1min":0.91500000000000004,"5min":0.97799999999999998,"15min":1.002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.54000000000000004,"15min":0.51700000000000002},"max":{"1min":0.89700000000000002,"5min":0.93500000000000005,"15min":1.0309999999999999},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.50600000000000001,"5min":0.42299999999999999,"15min":0.41599999999999998},"max":{"1min":0.85399999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.51600000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73099999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98999999999999999},"last":0.745},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.442},"max":{"1min":0.996,"5min":0.996,"15min":1.056},"last":0.72399999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96199999999999997,"5min":1.012,"15min":1.0429999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98099999999999998,"5min":1.0569999999999999,"15min":1.1279999999999999},"last":0.69799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575950,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307780,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064060,"statfs":{"total":96636764160,"available":96321597440,"internally_reserved":0,"allocated":2551808,"data_stored":1308365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46000000000000002,"15min":0.46500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":0.52900000000000003,"5min":0.748,"15min":0.97999999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.35199999999999998,"15min":0.33800000000000002},"max":{"1min":0.65900000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.5}]},{"osd":2,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51800000000000002,"15min":0.51200000000000001},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.33800000000000002},"max":{"1min":0.77100000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.627,"15min":0.62},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41199999999999998},"max":{"1min":0.79900000000000004,"5min":0.95699999999999996,"15min":1.0189999999999999},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":1.0509999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.39200000000000002},"max":{"1min":0.79900000000000004,"5min":0.87,"15min":0.94999999999999996},"last":0.54400000000000004}]},{"osd":4,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.45200000000000001},"max":{"1min":1.0649999999999999,"5min":1.155,"15min":1.218},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.84899999999999998,"15min":0.85599999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.59499999999999997},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.81699999999999995}]},{"osd":5,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82799999999999996,"15min":0.81999999999999995},"min":{"1min":0.63200000000000001,"5min":0.60699999999999998,"15min":0.57399999999999995},"max":{"1min":1.2210000000000001,"5min":1.2669999999999999,"15min":1.2709999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.61699999999999999,"5min":0.58199999999999996,"15min":0.55400000000000005},"max":{"1min":1.1160000000000001,"5min":1.2210000000000001,"15min":1.3109999999999999},"last":0.76100000000000001}]},{"osd":6,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.83399999999999996,"15min":0.84099999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1299999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84799999999999998,"15min":0.85099999999999998},"min":{"1min":0.64400000000000002,"5min":0.61699999999999999,"15min":0.59399999999999997},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.77000000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.87,"15min":0.876},"min":{"1min":0.71299999999999997,"5min":0.61899999999999999,"15min":0.60899999999999999},"max":{"1min":1.089,"5min":1.21,"15min":1.2549999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.66400000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.2509999999999999},"last":0.89700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575950,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307780,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064060,"statfs":{"total":96636764160,"available":96321597440,"internally_reserved":0,"allocated":2551808,"data_stored":1308829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.33900000000000002,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":0.61799999999999999,"5min":0.65800000000000003,"15min":0.74199999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51000000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.876},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38600000000000001},"max":{"1min":0.78600000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.40699999999999997,"15min":0.375},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.95899999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.42499999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.85199999999999998,"15min":0.85799999999999998},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.66100000000000003,"5min":0.622,"15min":0.56100000000000005},"max":{"1min":1.054,"5min":1.147,"15min":1.357},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84399999999999997,"15min":0.83899999999999997},"min":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.54400000000000004},"max":{"1min":1.0369999999999999,"5min":1.1879999999999999,"15min":1.2150000000000001},"last":0.92600000000000005}]},{"osd":5,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.71299999999999997,"5min":0.56699999999999995,"15min":0.52400000000000002},"max":{"1min":1.036,"5min":1.2170000000000001,"15min":1.2669999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.874,"15min":0.86899999999999999},"min":{"1min":0.67800000000000005,"5min":0.64500000000000002,"15min":0.58599999999999997},"max":{"1min":1.113,"5min":1.2350000000000001,"15min":1.2949999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.63700000000000001,"5min":0.58899999999999997,"15min":0.52600000000000002},"max":{"1min":1.0009999999999999,"5min":1.0980000000000001,"15min":1.3120000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.873,"15min":0.84999999999999998},"min":{"1min":0.66600000000000004,"5min":0.64500000000000002,"15min":0.54300000000000004},"max":{"1min":1.026,"5min":1.127,"15min":1.26},"last":0.70599999999999996}]},{"osd":7,"last update":"Mon Nov 6 22:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86299999999999999,"15min":0.85999999999999999},"min":{"1min":0.628,"5min":0.621,"15min":0.57499999999999996},"max":{"1min":1.1040000000000001,"5min":1.1639999999999999,"15min":1.268},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.74399999999999999,"5min":0.59999999999999998,"15min":0.58999999999999997},"max":{"1min":1.0960000000000001,"5min":1.1140000000000001,"15min":1.4330000000000001},"last":0.74199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303696,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068144,"statfs":{"total":96636764160,"available":96325779456,"internally_reserved":0,"allocated":2564096,"data_stored":1309011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.34200000000000003,"5min":0.29599999999999999,"15min":0.29499999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.443},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77900000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.55300000000000005}]},{"osd":1,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.36799999999999999},"max":{"1min":0.82699999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.628},"min":{"1min":0.50600000000000001,"5min":0.46300000000000002,"15min":0.34200000000000003},"max":{"1min":0.80000000000000004,"5min":0.89900000000000002,"15min":0.97299999999999998},"last":0.46000000000000002}]},{"osd":3,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.78300000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.621,"15min":0.61699999999999999},"min":{"1min":0.45900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93400000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.82499999999999996,"15min":0.82799999999999996},"min":{"1min":0.61499999999999999,"5min":0.58599999999999997,"15min":0.53900000000000003},"max":{"1min":1.0740000000000001,"5min":1.089,"15min":1.349},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.82299999999999995,"15min":0.82199999999999995},"min":{"1min":0.58499999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0429999999999999,"5min":1.1439999999999999,"15min":1.339},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.56999999999999995,"5min":0.54300000000000004,"15min":0.48999999999999999},"max":{"1min":1.0649999999999999,"5min":1.1579999999999999,"15min":1.337},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86599999999999999,"15min":0.85899999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.56000000000000005},"max":{"1min":1.212,"5min":1.212,"15min":1.468},"last":0.77200000000000002}]},{"osd":6,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.83899999999999997,"15min":0.84999999999999998},"min":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.54600000000000004},"max":{"1min":1.0860000000000001,"5min":1.1930000000000001,"15min":1.387},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85299999999999998,"15min":0.85099999999999998},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.56100000000000005},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.4299999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.878,"15min":0.875},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.55700000000000005},"max":{"1min":1.1899999999999999,"5min":1.266,"15min":1.496},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.87,"15min":0.874},"min":{"1min":0.66300000000000003,"5min":0.59199999999999997,"15min":0.54100000000000004},"max":{"1min":1.069,"5min":1.234,"15min":1.508},"last":0.68300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575951,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307784,"kb_used_data":2496,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064056,"statfs":{"total":96636764160,"available":96321593344,"internally_reserved":0,"allocated":2555904,"data_stored":1309201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46600000000000003,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.78000000000000003},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.35199999999999998,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53100000000000003}]},{"osd":1,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.498},"min":{"1min":0.371,"5min":0.34799999999999998,"15min":0.34100000000000003},"max":{"1min":0.63,"5min":0.71199999999999997,"15min":0.76900000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.58399999999999996},"min":{"1min":0.42199999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68500000000000005,"5min":0.82699999999999996,"15min":0.84799999999999998},"last":0.41499999999999998}]},{"osd":2,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.39400000000000002},"max":{"1min":0.71699999999999997,"5min":0.85299999999999998,"15min":0.87},"last":0.373},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.48699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.748,"5min":0.84199999999999997,"15min":0.89300000000000002},"last":0.47999999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84899999999999998,"15min":0.84499999999999997},"min":{"1min":0.627,"5min":0.60899999999999999,"15min":0.57999999999999996},"max":{"1min":1.0029999999999999,"5min":1.2370000000000001,"15min":1.365},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83299999999999996,"15min":0.84599999999999997},"min":{"1min":0.60999999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0669999999999999,"5min":1.1519999999999999,"15min":1.2989999999999999},"last":0.88700000000000001}]},{"osd":5,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81000000000000005,"15min":0.82099999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.2270000000000001,"15min":1.3440000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82299999999999995,"15min":0.83399999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.046,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82899999999999996,"15min":0.83899999999999997},"min":{"1min":0.58299999999999996,"5min":0.54500000000000004,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":1.212,"15min":1.331},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.86599999999999999,"15min":0.871},"min":{"1min":0.66900000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0660000000000001,"5min":1.1639999999999999,"15min":1.2749999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.86199999999999999,"15min":0.873},"min":{"1min":0.69699999999999995,"5min":0.61899999999999999,"15min":0.55900000000000005},"max":{"1min":1.034,"5min":1.2829999999999999,"15min":1.3080000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.86599999999999999,"15min":0.877},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.085,"5min":1.248,"15min":1.248},"last":0.86299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575950,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307780,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064060,"statfs":{"total":96636764160,"available":96321597440,"internally_reserved":0,"allocated":2551808,"data_stored":1308297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.56299999999999994,"5min":0.43099999999999999,"15min":0.371},"max":{"1min":0.96599999999999997,"5min":0.96699999999999997,"15min":0.98999999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.72499999999999998,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.41699999999999998},"max":{"1min":0.84799999999999998,"5min":1.0009999999999999,"15min":1.006},"last":0.49299999999999999}]},{"osd":1,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":0.91200000000000003,"5min":1.0129999999999999,"15min":1.0289999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.503,"5min":0.48399999999999999,"15min":0.46999999999999997},"max":{"1min":0.92100000000000004,"5min":1.0600000000000001,"15min":1.085},"last":0.71599999999999997}]},{"osd":2,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.629,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.087},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81899999999999995,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.48299999999999998},"max":{"1min":1.002,"5min":1.052,"15min":1.071},"last":0.84599999999999997}]},{"osd":3,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54200000000000004,"5min":0.52000000000000002,"15min":0.48699999999999999},"max":{"1min":0.90400000000000003,"5min":0.92400000000000004,"15min":1.0329999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.72899999999999998},"min":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.45600000000000002},"max":{"1min":0.95099999999999996,"5min":0.96999999999999997,"15min":1.0369999999999999},"last":0.628}]},{"osd":5,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.498,"5min":0.42399999999999999,"15min":0.39500000000000002},"max":{"1min":0.92300000000000004,"5min":0.93100000000000005,"15min":1.0469999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.96799999999999997},"last":0.61099999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78100000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.59299999999999997,"5min":0.48299999999999998,"15min":0.45800000000000002},"max":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":1.0269999999999999},"last":0.81799999999999995}]},{"osd":7,"last update":"Mon Nov 6 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70699999999999996,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.93400000000000005,"5min":0.98799999999999999,"15min":1.018},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72699999999999998,"15min":0.72199999999999998},"min":{"1min":0.59199999999999997,"5min":0.54900000000000004,"15min":0.49099999999999999},"max":{"1min":0.86899999999999999,"5min":0.98699999999999999,"15min":1.054},"last":0.72499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575951,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307780,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064060,"statfs":{"total":96636764160,"available":96321597440,"internally_reserved":0,"allocated":2551808,"data_stored":1308309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69599999999999995,"15min":0.68700000000000006},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.309},"max":{"1min":0.96199999999999997,"5min":1.0269999999999999,"15min":1.032},"last":1.226},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69599999999999995,"15min":0.69299999999999995},"min":{"1min":0.44700000000000001,"5min":0.40300000000000002,"15min":0.35999999999999999},"max":{"1min":0.88200000000000001,"5min":0.95499999999999996,"15min":0.98899999999999999},"last":3.3559999999999999}]},{"osd":1,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.34799999999999998},"max":{"1min":0.877,"5min":0.997,"15min":1.21},"last":5.0279999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.77800000000000002},"min":{"1min":0.58799999999999997,"5min":0.47999999999999998,"15min":0.47299999999999998},"max":{"1min":0.98499999999999999,"5min":1.087,"15min":1.2290000000000001},"last":1.105}]},{"osd":2,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.54400000000000004,"5min":0.46400000000000002,"15min":0.45100000000000001},"max":{"1min":0.86399999999999999,"5min":1.113,"15min":1.113},"last":3.4089999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76500000000000001,"15min":0.754},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.48299999999999998},"max":{"1min":0.93100000000000005,"5min":1.103,"15min":1.103},"last":4.9779999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.57599999999999996,"5min":0.46200000000000002,"15min":0.36499999999999999},"max":{"1min":0.94599999999999995,"5min":0.996,"15min":1.044},"last":1.2729999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.74099999999999999},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.45800000000000002},"max":{"1min":0.94799999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":3.4489999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.90400000000000003,"15min":0.91700000000000004},"last":5.0659999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68600000000000005,"15min":0.68700000000000006},"min":{"1min":0.54600000000000004,"5min":0.38800000000000001,"15min":0.34999999999999998},"max":{"1min":0.94899999999999995,"5min":1.002,"15min":1.004},"last":1.585}]},{"osd":6,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":0.88700000000000001,"5min":0.97999999999999998,"15min":1.119},"last":3.4889999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.36299999999999999},"max":{"1min":0.93700000000000006,"5min":0.95599999999999996,"15min":1.0780000000000001},"last":5.1029999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.39200000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0780000000000001},"last":1.365},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70599999999999996,"15min":0.69499999999999995},"min":{"1min":0.51100000000000001,"5min":0.47499999999999998,"15min":0.45200000000000001},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":6.891}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:16:43.982 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T22:16:43.983 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T22:16:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.213+0000 7f877b3ff700 1 Processor -- start 2023-11-06T22:16:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.214+0000 7f877b3ff700 1 -- start start 2023-11-06T22:16:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.215+0000 7f877b3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740c4720 0x7f8774157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.215+0000 7f877b3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8774064bf0 con 0x7f87740c4b00 2023-11-06T22:16:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.215+0000 7f877b3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8774062e10 con 0x7f877413dff0 2023-11-06T22:16:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.215+0000 7f877b3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740c4f60 con 0x7f87740c4720 2023-11-06T22:16:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.215+0000 7f877899a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f87740c4b00 0x7f877413d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58076/0 (socket says 172.21.15.33:58076) 2023-11-06T22:16:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.215+0000 7f877899a700 1 -- 172.21.15.33:0/2105011435 learned_addr learned my addr 172.21.15.33:0/2105011435 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f877919b700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740c4720 0x7f8774157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421651884 0 0) 0x7f8774064bf0 con 0x7f87740c4b00 2023-11-06T22:16:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87600033f0 con 0x7f87740c4b00 2023-11-06T22:16:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764950406 0 0) 0x7f8774062e10 con 0x7f877413dff0 2023-11-06T22:16:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8774064bf0 con 0x7f877413dff0 2023-11-06T22:16:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1092337488 0 0) 0x7f87600033f0 con 0x7f87740c4b00 2023-11-06T22:16:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740c4720 msgr2=0x7f8774157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740c4720 0x7f8774157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 >> v1:172.21.15.69:6789/0 conn(0x7f877413dff0 legacy=0x7f8774065080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87740630f0 con 0x7f87740c4b00 2023-11-06T22:16:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f8764001470 con 0x7f87740c4b00 2023-11-06T22:16:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 >> v1:172.21.15.33:6789/0 conn(0x7f87740c4b00 legacy=0x7f877413d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 0x7f8760003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8760004690 0x7f8760004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740c4720 0x7f8760005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740c4f60 con 0x7f8760003aa0 2023-11-06T22:16:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876000b910 con 0x7f8760004690 2023-11-06T22:16:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.216+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876000bad0 con 0x7f87740c4720 2023-11-06T22:16:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f877919b700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 0x7f8760003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f877899a700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8760004690 0x7f8760004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f877999c700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740c4720 0x7f8760005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f877919b700 1 -- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740c4720 msgr2=0x7f8760005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f877919b700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740c4720 0x7f8760005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f877919b700 1 -- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8760004690 msgr2=0x7f8760004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f877919b700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8760004690 0x7f8760004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f877919b700 1 -- 172.21.15.33:0/2105011435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87600033f0 con 0x7f8760003aa0 2023-11-06T22:16:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f877919b700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 0x7f8760003ea0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f8770009a80 tx=0x7f8770009060).ready entity=mon.0 client_cookie=674b1ea04c646c72 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770009430 con 0x7f8760003aa0 2023-11-06T22:16:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877000e640 con 0x7f8760003aa0 2023-11-06T22:16:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.217+0000 7f876bfff700 1 -- 172.21.15.33:0/2105011435 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877000e9c0 con 0x7f8760003aa0 2023-11-06T22:16:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.218+0000 7f877b3ff700 1 -- 172.21.15.33:0/2105011435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 msgr2=0x7f8760003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.218+0000 7f877b3ff700 1 --2- 172.21.15.33:0/2105011435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 0x7f8760003ea0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f8770009a80 tx=0x7f8770009060).stop 2023-11-06T22:16:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.218+0000 7f877999c700 1 -- 172.21.15.33:0/2105011435 reap_dead start 2023-11-06T22:16:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.218+0000 7f877b3ff700 1 -- 172.21.15.33:0/2105011435 shutdown_connections 2023-11-06T22:16:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.218+0000 7f877b3ff700 1 -- 172.21.15.33:0/2105011435 >> 172.21.15.33:0/2105011435 conn(0x7f87740c7dc0 msgr2=0x7f87740c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.218+0000 7f877b3ff700 1 -- 172.21.15.33:0/2105011435 shutdown_connections 2023-11-06T22:16:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.218+0000 7f877b3ff700 1 -- 172.21.15.33:0/2105011435 wait complete. 2023-11-06T22:16:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.220+0000 7f877b3ff700 1 Processor -- start 2023-11-06T22:16:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.220+0000 7f877b3ff700 1 -- start start 2023-11-06T22:16:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877b3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 0x7f87740acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877b3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740aaf10 0x7f8774153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877b3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774134640 0x7f8774134ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877b3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740ab940 con 0x7f8760003aa0 2023-11-06T22:16:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877b3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8774133950 con 0x7f8774134640 2023-11-06T22:16:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877b3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740ab390 con 0x7f87740aaf10 2023-11-06T22:16:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 0x7f87740acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 0x7f87740acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41908/0 (socket says 172.21.15.33:41908) 2023-11-06T22:16:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877919b700 1 -- 172.21.15.33:0/24376142 learned_addr learned my addr 172.21.15.33:0/24376142 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877899a700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740aaf10 0x7f8774153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877999c700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774134640 0x7f8774134ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877919b700 1 -- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740aaf10 msgr2=0x7f8774153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877919b700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740aaf10 0x7f8774153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877919b700 1 -- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774134640 msgr2=0x7f8774134ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877919b700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774134640 0x7f8774134ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877919b700 1 -- 172.21.15.33:0/24376142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87740ab660 con 0x7f8760003aa0 2023-11-06T22:16:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877999c700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774134640 0x7f8774134ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.221+0000 7f877919b700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 0x7f87740acef0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7f877000e4b0 tx=0x7f877001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.222+0000 7f87697fa700 1 -- 172.21.15.33:0/24376142 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877000ed30 con 0x7f8760003aa0 2023-11-06T22:16:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.222+0000 7f87697fa700 1 -- 172.21.15.33:0/24376142 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877000ee90 con 0x7f8760003aa0 2023-11-06T22:16:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.222+0000 7f87697fa700 1 -- 172.21.15.33:0/24376142 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770026440 con 0x7f8760003aa0 2023-11-06T22:16:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.222+0000 7f877b3ff700 1 -- 172.21.15.33:0/24376142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8774133b60 con 0x7f8760003aa0 2023-11-06T22:16:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.222+0000 7f877b3ff700 1 -- 172.21.15.33:0/24376142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8774134120 con 0x7f8760003aa0 2023-11-06T22:16:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.223+0000 7f877b3ff700 1 -- 172.21.15.33:0/24376142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87740aa0d0 con 0x7f8760003aa0 2023-11-06T22:16:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.224+0000 7f87697fa700 1 -- 172.21.15.33:0/24376142 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f877002f440 con 0x7f8760003aa0 2023-11-06T22:16:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.225+0000 7f87697fa700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f875805f270 0x7f8758061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.225+0000 7f87697fa700 1 -- 172.21.15.33:0/24376142 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f877007c2a0 con 0x7f8760003aa0 2023-11-06T22:16:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.225+0000 7f877899a700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f875805f270 0x7f8758061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.226+0000 7f877899a700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f875805f270 0x7f8758061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8774133900 tx=0x7f8764009040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.229+0000 7f87697fa700 1 -- 172.21.15.33:0/24376142 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f877004d360 con 0x7f8760003aa0 2023-11-06T22:16:44.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.478+0000 7f877b3ff700 1 -- 172.21.15.33:0/24376142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8774069c20 con 0x7f8760003aa0 2023-11-06T22:16:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.483+0000 7f87697fa700 1 -- 172.21.15.33:0/24376142 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v48) v1 ==== 190+0+0 (secure 0 0 0) 0x7f87700160e0 con 0x7f8760003aa0 2023-11-06T22:16:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 -- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f875805f270 msgr2=0x7f8758061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f875805f270 0x7f8758061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f8774133900 tx=0x7f8764009040).stop 2023-11-06T22:16:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 -- 172.21.15.33:0/24376142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 msgr2=0x7f87740acef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 0x7f87740acef0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7f877000e4b0 tx=0x7f877001a000).stop 2023-11-06T22:16:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 -- 172.21.15.33:0/24376142 shutdown_connections 2023-11-06T22:16:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f875805f270 0x7f8758061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774134640 0x7f8774134ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760003aa0 0x7f87740acef0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 --2- 172.21.15.33:0/24376142 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f87740aaf10 0x7f8774153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 -- 172.21.15.33:0/24376142 >> 172.21.15.33:0/24376142 conn(0x7f87740c7dc0 msgr2=0x7f87741397b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 -- 172.21.15.33:0/24376142 shutdown_connections 2023-11-06T22:16:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.485+0000 7f877b3ff700 1 -- 172.21.15.33:0/24376142 wait complete. 2023-11-06T22:16:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T22:16:44.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:16:44.597 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-11-06T22:16:44.595+0000 7f5ce39f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T22:16:44.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.699+0000 7fc19c7ff700 1 Processor -- start 2023-11-06T22:16:44.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.699+0000 7fc19c7ff700 1 -- start start 2023-11-06T22:16:44.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.700+0000 7fc19c7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc194153e60 0x7fc1940a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc19c7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc194154240 0x7fc194069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc19c7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1940c2ec0 con 0x7fc194062710 2023-11-06T22:16:44.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc19c7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1941546c0 con 0x7fc194154240 2023-11-06T22:16:44.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc19c7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940c4360 con 0x7fc194153e60 2023-11-06T22:16:44.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc199d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc194154240 0x7fc194069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc199d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc194154240 0x7fc194069d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41916/0 (socket says 172.21.15.33:41916) 2023-11-06T22:16:44.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc199d9a700 1 -- 172.21.15.33:0/2224080323 learned_addr learned my addr 172.21.15.33:0/2224080323 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:44.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc19a59b700 1 --2- 172.21.15.33:0/2224080323 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc194153e60 0x7fc1940a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc199d9a700 1 -- 172.21.15.33:0/2224080323 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc194153e60 msgr2=0x7fc1940a6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:44.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc199d9a700 1 --2- 172.21.15.33:0/2224080323 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc194153e60 0x7fc1940a6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc199d9a700 1 -- 172.21.15.33:0/2224080323 >> v1:172.21.15.69:6789/0 conn(0x7fc194062710 legacy=0x7fc194157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.701+0000 7fc199d9a700 1 -- 172.21.15.33:0/2224080323 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1940a73d0 con 0x7fc194154240 2023-11-06T22:16:44.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.702+0000 7fc199d9a700 1 --2- 172.21.15.33:0/2224080323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 0x7fc194069d70 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7fc184006fd0 tx=0x7fc1840090f0).ready entity=mon.0 client_cookie=6f2b6c38daf02dc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.702+0000 7fc199599700 1 -- 172.21.15.33:0/2224080323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18400b580 con 0x7fc194154240 2023-11-06T22:16:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.702+0000 7fc199599700 1 -- 172.21.15.33:0/2224080323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18400b6e0 con 0x7fc194154240 2023-11-06T22:16:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.702+0000 7fc199599700 1 -- 172.21.15.33:0/2224080323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18400ba60 con 0x7fc194154240 2023-11-06T22:16:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.702+0000 7fc19c7ff700 1 -- 172.21.15.33:0/2224080323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 msgr2=0x7fc194069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.702+0000 7fc19c7ff700 1 --2- 172.21.15.33:0/2224080323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 0x7fc194069d70 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7fc184006fd0 tx=0x7fc1840090f0).stop 2023-11-06T22:16:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.703+0000 7fc19c7ff700 1 -- 172.21.15.33:0/2224080323 shutdown_connections 2023-11-06T22:16:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.703+0000 7fc19c7ff700 1 --2- 172.21.15.33:0/2224080323 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc194153e60 0x7fc1940a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.703+0000 7fc19c7ff700 1 --2- 172.21.15.33:0/2224080323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 0x7fc194069d70 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.703+0000 7fc19c7ff700 1 -- 172.21.15.33:0/2224080323 >> 172.21.15.33:0/2224080323 conn(0x7fc1940c7b50 msgr2=0x7fc1940c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.703+0000 7fc19c7ff700 1 -- 172.21.15.33:0/2224080323 shutdown_connections 2023-11-06T22:16:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.703+0000 7fc19c7ff700 1 -- 172.21.15.33:0/2224080323 wait complete. 2023-11-06T22:16:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.704+0000 7fc19c7ff700 1 Processor -- start 2023-11-06T22:16:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.704+0000 7fc19c7ff700 1 -- start start 2023-11-06T22:16:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.705+0000 7fc19c7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc194062710 0x7fc1941484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.705+0000 7fc19c7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc194153e60 0x7fc1940bf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.705+0000 7fc19c7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 0x7fc1940666a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.705+0000 7fc19c7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940ab480 con 0x7fc194154240 2023-11-06T22:16:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.705+0000 7fc19c7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940ac970 con 0x7fc194062710 2023-11-06T22:16:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.705+0000 7fc19c7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940c0370 con 0x7fc194153e60 2023-11-06T22:16:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.705+0000 7fc19ad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 0x7fc1940666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc19ad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 0x7fc1940666a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41932/0 (socket says 172.21.15.33:41932) 2023-11-06T22:16:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc19ad9c700 1 -- 172.21.15.33:0/3252184974 learned_addr learned my addr 172.21.15.33:0/3252184974 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc199d9a700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc194153e60 0x7fc1940bf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc19a59b700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc194062710 0x7fc1941484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc19ad9c700 1 -- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc194153e60 msgr2=0x7fc1940bf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc19ad9c700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc194153e60 0x7fc1940bf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc19ad9c700 1 -- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc194062710 msgr2=0x7fc1941484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc19ad9c700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc194062710 0x7fc1941484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc19ad9c700 1 -- 172.21.15.33:0/3252184974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc184009370 con 0x7fc194154240 2023-11-06T22:16:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc19ad9c700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 0x7fc1940666a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7fc18c002230 tx=0x7fc18c0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:44.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc18a7fc700 1 -- 172.21.15.33:0/3252184974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00e4a0 con 0x7fc194154240 2023-11-06T22:16:44.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc18a7fc700 1 -- 172.21.15.33:0/3252184974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c00e600 con 0x7fc194154240 2023-11-06T22:16:44.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc19c7ff700 1 -- 172.21.15.33:0/3252184974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1940aaae0 con 0x7fc194154240 2023-11-06T22:16:44.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.706+0000 7fc18a7fc700 1 -- 172.21.15.33:0/3252184974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00e980 con 0x7fc194154240 2023-11-06T22:16:44.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.707+0000 7fc19c7ff700 1 -- 172.21.15.33:0/3252184974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1940ab0a0 con 0x7fc194154240 2023-11-06T22:16:44.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.708+0000 7fc19c7ff700 1 -- 172.21.15.33:0/3252184974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1940c2590 con 0x7fc194154240 2023-11-06T22:16:44.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.709+0000 7fc18a7fc700 1 -- 172.21.15.33:0/3252184974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc18c012400 con 0x7fc194154240 2023-11-06T22:16:44.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.709+0000 7fc18a7fc700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc17805f270 0x7fc178061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:44.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.709+0000 7fc18a7fc700 1 -- 172.21.15.33:0/3252184974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fc18c0799e0 con 0x7fc194154240 2023-11-06T22:16:44.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.710+0000 7fc19a59b700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc17805f270 0x7fc178061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:44.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.711+0000 7fc19a59b700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc17805f270 0x7fc178061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fc190009c30 tx=0x7fc190009060).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:44.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.713+0000 7fc18a7fc700 1 -- 172.21.15.33:0/3252184974 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc18c048af0 con 0x7fc194154240 2023-11-06T22:16:44.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.936+0000 7fc19c7ff700 1 -- 172.21.15.33:0/3252184974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc19406a710 con 0x7fc194154240 2023-11-06T22:16:44.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.937+0000 7fc18a7fc700 1 -- 172.21.15.33:0/3252184974 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4493 (secure 0 0 0) 0x7fc18c04ab50 con 0x7fc194154240 2023-11-06T22:16:44.939 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:44.940 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3815920714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3815920714},{"type":"v1","addr":"172.21.15.69:6836","nonce":3815920714}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9052,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3614716748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3614716748},{"type":"v1","addr":"172.21.15.69:6837","nonce":3614716748}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1513187241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1513187241},{"type":"v1","addr":"172.21.15.33:6834","nonce":1513187241}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:16:39.145216+0000","modified":"2023-11-06T22:16:42.180077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3197264665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3197264665},{"type":"v1","addr":"172.21.15.33:6835","nonce":3197264665}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-06T22:16:44.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.938+0000 7fc19c7ff700 1 -- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc17805f270 msgr2=0x7fc178061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc17805f270 0x7fc178061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fc190009c30 tx=0x7fc190009060).stop 2023-11-06T22:16:44.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 -- 172.21.15.33:0/3252184974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 msgr2=0x7fc1940666a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:44.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 0x7fc1940666a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7fc18c002230 tx=0x7fc18c0015e0).stop 2023-11-06T22:16:44.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 -- 172.21.15.33:0/3252184974 shutdown_connections 2023-11-06T22:16:44.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc17805f270 0x7fc178061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc194154240 0x7fc1940666a0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc194062710 0x7fc1941484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 --2- 172.21.15.33:0/3252184974 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc194153e60 0x7fc1940bf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:44.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 -- 172.21.15.33:0/3252184974 >> 172.21.15.33:0/3252184974 conn(0x7fc1940c7b50 msgr2=0x7fc1940c4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:44.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 -- 172.21.15.33:0/3252184974 shutdown_connections 2023-11-06T22:16:44.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:44.939+0000 7fc19c7ff700 1 -- 172.21.15.33:0/3252184974 wait complete. 2023-11-06T22:16:44.944 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 84 2023-11-06T22:16:45.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T22:16:39.145216+0000', 'modified': '2023-11-06T22:16:42.180077+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9387': {'gid': 9387, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/3197264665', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6833', 'nonce': 3197264665}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 3197264665}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T22:16:45.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T22:16:45.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T22:16:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.561+0000 7f01ca3ff700 1 Processor -- start 2023-11-06T22:16:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.561+0000 7f01ca3ff700 1 -- start start 2023-11-06T22:16:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.563+0000 7f01ca3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f01c40c5ea0 0x7f01c4157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.563+0000 7f01ca3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c40640c0 con 0x7f01c4158190 2023-11-06T22:16:45.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.563+0000 7f01ca3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c40c5a10 con 0x7f01c414e6a0 2023-11-06T22:16:45.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.563+0000 7f01ca3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c4064d10 con 0x7f01c40c5ea0 2023-11-06T22:16:45.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.563+0000 7f01c899c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f01c4158190 0x7f01c40c3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58080/0 (socket says 172.21.15.33:58080) 2023-11-06T22:16:45.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.563+0000 7f01c899c700 1 -- 172.21.15.33:0/1273882786 learned_addr learned my addr 172.21.15.33:0/1273882786 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:45.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.564+0000 7f01c37fe700 1 --2- 172.21.15.33:0/1273882786 >> v2:172.21.15.69:3300/0 conn(0x7f01c40c5ea0 0x7f01c4157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.564+0000 7f01c2ffd700 1 -- 172.21.15.33:0/1273882786 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996467182 0 0) 0x7f01bc001490 con 0x7f01c4158190 2023-11-06T22:16:45.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.564+0000 7f01c2ffd700 1 -- 172.21.15.33:0/1273882786 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ac0033f0 con 0x7f01c4158190 2023-11-06T22:16:45.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.564+0000 7f01c2ffd700 1 -- 172.21.15.33:0/1273882786 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689742685 0 0) 0x7f01c40c5a10 con 0x7f01c414e6a0 2023-11-06T22:16:45.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.564+0000 7f01c2ffd700 1 -- 172.21.15.33:0/1273882786 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01bc001490 con 0x7f01c414e6a0 2023-11-06T22:16:45.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.564+0000 7f01c37fe700 1 -- 172.21.15.33:0/1273882786 >> v1:172.21.15.69:6789/0 conn(0x7f01c414e6a0 legacy=0x7f01c4066e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.564+0000 7f01c37fe700 1 -- 172.21.15.33:0/1273882786 >> v1:172.21.15.33:6789/0 conn(0x7f01c4158190 legacy=0x7f01c40c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.564+0000 7f01c37fe700 1 -- 172.21.15.33:0/1273882786 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c4158600 con 0x7f01c40c5ea0 2023-11-06T22:16:45.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.564+0000 7f01c37fe700 1 --2- 172.21.15.33:0/1273882786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c40c5ea0 0x7f01c4157ca0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f01b8009760 tx=0x7f01b8009060).ready entity=mon.1 client_cookie=d5397cf68d93adf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:45.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.565+0000 7f01c2ffd700 1 -- 172.21.15.33:0/1273882786 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b800d4a0 con 0x7f01c40c5ea0 2023-11-06T22:16:45.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.565+0000 7f01c2ffd700 1 -- 172.21.15.33:0/1273882786 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b800d600 con 0x7f01c40c5ea0 2023-11-06T22:16:45.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.565+0000 7f01c2ffd700 1 -- 172.21.15.33:0/1273882786 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b800d980 con 0x7f01c40c5ea0 2023-11-06T22:16:45.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.565+0000 7f01ca3ff700 1 -- 172.21.15.33:0/1273882786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c40c5ea0 msgr2=0x7f01c4157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.565+0000 7f01ca3ff700 1 --2- 172.21.15.33:0/1273882786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c40c5ea0 0x7f01c4157ca0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f01b8009760 tx=0x7f01b8009060).stop 2023-11-06T22:16:45.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.565+0000 7f01ca3ff700 1 -- 172.21.15.33:0/1273882786 shutdown_connections 2023-11-06T22:16:45.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.565+0000 7f01ca3ff700 1 --2- 172.21.15.33:0/1273882786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c40c5ea0 0x7f01c4157ca0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.565+0000 7f01ca3ff700 1 -- 172.21.15.33:0/1273882786 >> 172.21.15.33:0/1273882786 conn(0x7f01c405e1b0 msgr2=0x7f01c405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:45.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.565+0000 7f01ca3ff700 1 -- 172.21.15.33:0/1273882786 shutdown_connections 2023-11-06T22:16:45.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.565+0000 7f01ca3ff700 1 -- 172.21.15.33:0/1273882786 wait complete. 2023-11-06T22:16:45.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.567+0000 7f01ca3ff700 1 Processor -- start 2023-11-06T22:16:45.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.567+0000 7f01ca3ff700 1 -- start start 2023-11-06T22:16:45.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01ca3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c40c5ea0 0x7f01c414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01ca3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c414e6a0 0x7f01c40ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01ca3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c4158190 0x7f01c40c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01ca3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40aeb80 con 0x7f01c414e6a0 2023-11-06T22:16:45.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01ca3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40a9b30 con 0x7f01c4158190 2023-11-06T22:16:45.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01ca3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c414d0b0 con 0x7f01c40c5ea0 2023-11-06T22:16:45.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c414e6a0 0x7f01c40ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c414e6a0 0x7f01c40ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41944/0 (socket says 172.21.15.33:41944) 2023-11-06T22:16:45.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01c37fe700 1 -- 172.21.15.33:0/2331814735 learned_addr learned my addr 172.21.15.33:0/2331814735 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:45.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01c899c700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c4158190 0x7f01c40c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.568+0000 7f01c3fff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c40c5ea0 0x7f01c414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01c37fe700 1 -- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c40c5ea0 msgr2=0x7f01c414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01c37fe700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c40c5ea0 0x7f01c414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01c37fe700 1 -- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c4158190 msgr2=0x7f01c40c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01c37fe700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c4158190 0x7f01c40c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01c37fe700 1 -- 172.21.15.33:0/2331814735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b80092a0 con 0x7f01c414e6a0 2023-11-06T22:16:45.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01c3fff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c40c5ea0 0x7f01c414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:45.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01c899c700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c4158190 0x7f01c40c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:45.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01c37fe700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c414e6a0 0x7f01c40ad7b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f01b8006d60 tx=0x7f01b8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:45.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01abfff700 1 -- 172.21.15.33:0/2331814735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b800d770 con 0x7f01c414e6a0 2023-11-06T22:16:45.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01abfff700 1 -- 172.21.15.33:0/2331814735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b800d8d0 con 0x7f01c414e6a0 2023-11-06T22:16:45.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01abfff700 1 -- 172.21.15.33:0/2331814735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b801d660 con 0x7f01c414e6a0 2023-11-06T22:16:45.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01ca3ff700 1 -- 172.21.15.33:0/2331814735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c40ae000 con 0x7f01c414e6a0 2023-11-06T22:16:45.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.569+0000 7f01ca3ff700 1 -- 172.21.15.33:0/2331814735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c40ae5c0 con 0x7f01c414e6a0 2023-11-06T22:16:45.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.571+0000 7f01abfff700 1 -- 172.21.15.33:0/2331814735 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f01b801da10 con 0x7f01c414e6a0 2023-11-06T22:16:45.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.573+0000 7f01abfff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01b005f240 0x7f01b0061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.573+0000 7f01abfff700 1 -- 172.21.15.33:0/2331814735 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f01b807bcb0 con 0x7f01c414e6a0 2023-11-06T22:16:45.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.573+0000 7f01c3fff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01b005f240 0x7f01b0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.574+0000 7f01c3fff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01b005f240 0x7f01b0061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f01c40c2cd0 tx=0x7f01b4009040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:45.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.575+0000 7f01e16ecb80 1 Processor -- start 2023-11-06T22:16:45.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.575+0000 7f01e16ecb80 1 -- start start 2023-11-06T22:16:45.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 0x556593c9d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593c97ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593cc54c0 0x556593c983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556593bd92f0 con 0x556593ce5be0 2023-11-06T22:16:45.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556593bee7e0 con 0x556593cc54c0 2023-11-06T22:16:45.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x556593bee920 con 0x556593c9d080 2023-11-06T22:16:45.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593c97ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593c97ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41946/0 (socket says 172.21.15.33:41946) 2023-11-06T22:16:45.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01c3fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 0x556593c9d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01c37fe700 1 -- 172.21.15.33:0/2511589814 learned_addr learned my addr 172.21.15.33:0/2511589814 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01c899c700 1 --2- 172.21.15.33:0/2511589814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593cc54c0 0x556593c983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01c37fe700 1 -- 172.21.15.33:0/2511589814 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 msgr2=0x556593c9d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01c37fe700 1 --2- 172.21.15.33:0/2511589814 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 0x556593c9d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01c37fe700 1 -- 172.21.15.33:0/2511589814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593cc54c0 msgr2=0x556593c983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01c37fe700 1 --2- 172.21.15.33:0/2511589814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593cc54c0 0x556593c983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.576+0000 7f01c37fe700 1 -- 172.21.15.33:0/2511589814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556593be7720 con 0x556593ce5be0 2023-11-06T22:16:45.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01c37fe700 1 --2- 172.21.15.33:0/2511589814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593c97ef0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f01b8003000 tx=0x7f01b800e000).ready entity=mon.0 client_cookie=cc901c23e011f0ba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:45.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01a99fe700 1 -- 172.21.15.33:0/2511589814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8050510 con 0x556593ce5be0 2023-11-06T22:16:45.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01a99fe700 1 -- 172.21.15.33:0/2511589814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b8050330 con 0x556593ce5be0 2023-11-06T22:16:45.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01a99fe700 1 -- 172.21.15.33:0/2511589814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8050970 con 0x556593ce5be0 2023-11-06T22:16:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2511589814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 msgr2=0x556593c97ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2511589814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593c97ef0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f01b8003000 tx=0x7f01b800e000).stop 2023-11-06T22:16:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2511589814 shutdown_connections 2023-11-06T22:16:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2511589814 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 0x556593c9d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2511589814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593cc54c0 0x556593c983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2511589814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593c97ef0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.577+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2511589814 >> 172.21.15.33:0/2511589814 conn(0x556593b06940 msgr2=0x556593ce6070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:45.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.578+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2511589814 shutdown_connections 2023-11-06T22:16:45.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.578+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2511589814 wait complete. 2023-11-06T22:16:45.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.578+0000 7f01e16ecb80 1 -- start start 2023-11-06T22:16:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.579+0000 7f01e16ecb80 1 Processor -- start 2023-11-06T22:16:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.579+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 0x556593ce46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 0x556593cf5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593cf5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556593beeba0 con 0x556593ce5be0 2023-11-06T22:16:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556593bb9e60 con 0x556593c9d080 2023-11-06T22:16:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x556593cf5b70 con 0x556593cc54c0 2023-11-06T22:16:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01e16ecb80 10 client.0 ll_register_callbacks cb 0x556593b8ece0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T22:16:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593cf5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593cf5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41962/0 (socket says 172.21.15.33:41962) 2023-11-06T22:16:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c899c700 1 -- 172.21.15.33:0/3402453772 learned_addr learned my addr 172.21.15.33:0/3402453772 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c3fff700 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 0x556593ce46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c37fe700 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 0x556593cf5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c899c700 1 -- 172.21.15.33:0/3402453772 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 msgr2=0x556593cf5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c899c700 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 0x556593cf5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c899c700 1 -- 172.21.15.33:0/3402453772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 msgr2=0x556593ce46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:45.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c899c700 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 0x556593ce46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c899c700 1 -- 172.21.15.33:0/3402453772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c40ae5c0 con 0x556593ce5be0 2023-11-06T22:16:45.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c3fff700 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 0x556593ce46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:45.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c37fe700 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 0x556593cf5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01c899c700 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593cf5680 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f01bc011750 tx=0x7f01bc009040).ready entity=mon.0 client_cookie=7b2f43454b17c931 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.580+0000 7f01a27fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01a27fc700 1 -- 172.21.15.33:0/3402453772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc005d10 con 0x556593ce5be0 2023-11-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01a27fc700 1 -- 172.21.15.33:0/3402453772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f01bc0070d0 con 0x556593ce5be0 2023-11-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01e16ecb80 1 -- 172.21.15.33:0/3402453772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556593c90e20 con 0x556593ce5be0 2023-11-06T22:16:45.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01a27fc700 1 -- 172.21.15.33:0/3402453772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01bc005e70 con 0x556593ce5be0 2023-11-06T22:16:45.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01e16ecb80 10 client.9516 fetch_fsmap learned FSMap version 84 2023-11-06T22:16:45.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01e16ecb80 1 -- 172.21.15.33:0/3402453772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x556593be2ac0 con 0x556593ce5be0 2023-11-06T22:16:45.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01a27fc700 1 -- 172.21.15.33:0/3402453772 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556593c90e20 con 0x556593ce5be0 2023-11-06T22:16:45.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01a27fc700 1 -- 172.21.15.33:0/3402453772 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f01bc0074a0 con 0x556593ce5be0 2023-11-06T22:16:45.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01e16ecb80 10 client.9516 fetch_fsmap finished waiting for FSMap version 84 2023-11-06T22:16:45.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01e16ecb80 10 client.9516 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T22:16:45.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01e16ecb80 20 client.9516 populate_metadata read hostname 'smithi033' 2023-11-06T22:16:45.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.33:6833/3197264665,v1:172.21.15.33:6835/3197264665] conn(0x556593cbe540 0x556593cff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01e16ecb80 4 client.9516 mds_command: new command op to 9387 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T22:16:45.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.581+0000 7f01e16ecb80 1 -- 172.21.15.33:0/3402453772 --> [v2:172.21.15.33:6833/3197264665,v1:172.21.15.33:6835/3197264665] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556593a56ef0 con 0x556593cbe540 2023-11-06T22:16:45.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.582+0000 7f01c3fff700 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.33:6833/3197264665,v1:172.21.15.33:6835/3197264665] conn(0x556593cbe540 0x556593cff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.582+0000 7f01c3fff700 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.33:6833/3197264665,v1:172.21.15.33:6835/3197264665] conn(0x556593cbe540 0x556593cff4b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ee824b8b9e55ff9c server_cookie=b1f3dd353ab315b1 in_seq=0 out_seq=0 2023-11-06T22:16:45.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.582+0000 7f01a27fc700 10 client.9516 ms_handle_connect on v2:172.21.15.33:6833/3197264665 2023-11-06T22:16:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.583+0000 7f01a27fc700 1 -- 172.21.15.33:0/3402453772 <== mds.0 v2:172.21.15.33:6833/3197264665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x556593a56ef0 con 0x556593cbe540 2023-11-06T22:16:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.583+0000 7f01a27fc700 10 client.9516 handle_command_reply: tid=0 2023-11-06T22:16:45.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.583+0000 7f01e16ecb80 1 client.9516 shutdown 2023-11-06T22:16:45.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 1 -- 172.21.15.33:0/3402453772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 msgr2=0x556593cf5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593cf5680 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f01bc011750 tx=0x7f01bc009040).stop 2023-11-06T22:16:45.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 1 -- 172.21.15.33:0/3402453772 shutdown_connections 2023-11-06T22:16:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.33:6833/3197264665,v1:172.21.15.33:6835/3197264665] conn(0x556593cbe540 0x556593cff4b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593cf5680 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 0x556593ce46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01a27fc700 0 client.9516 ms_handle_reset on v2:172.21.15.33:6833/3197264665 2023-11-06T22:16:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/3402453772 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 0x556593cf5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 1 -- 172.21.15.33:0/3402453772 >> 172.21.15.33:0/3402453772 conn(0x556593b06940 msgr2=0x556593c9abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 1 -- 172.21.15.33:0/3402453772 shutdown_connections 2023-11-06T22:16:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 1 -- 172.21.15.33:0/3402453772 wait complete. 2023-11-06T22:16:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.584+0000 7f01e16ecb80 20 client.9516 trim_cache size 0 max 16384 2023-11-06T22:16:45.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.601+0000 7f01e16ecb80 1 Processor -- start 2023-11-06T22:16:45.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.601+0000 7f01e16ecb80 1 -- start start 2023-11-06T22:16:45.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.602+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 0x556593cf4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.602+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593cc54c0 0x556593c99140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.602+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593ce5be0 0x556593cc2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.602+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556593cc9690 con 0x556593cc54c0 2023-11-06T22:16:45.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.602+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556593c99e60 con 0x556593ce5be0 2023-11-06T22:16:45.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.602+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x556593ce0670 con 0x556593c9d080 2023-11-06T22:16:45.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.602+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593cc54c0 0x556593c99140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.602+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593cc54c0 0x556593c99140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41964/0 (socket says 172.21.15.33:41964) 2023-11-06T22:16:45.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.602+0000 7f01c37fe700 1 -- 172.21.15.33:0/2816824863 learned_addr learned my addr 172.21.15.33:0/2816824863 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:45.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.602+0000 7f01c3fff700 1 --2- 172.21.15.33:0/2816824863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 0x556593cf4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01c899c700 1 --2- 172.21.15.33:0/2816824863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593ce5be0 0x556593cc2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01c37fe700 1 -- 172.21.15.33:0/2816824863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 msgr2=0x556593cf4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:45.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01c37fe700 1 --2- 172.21.15.33:0/2816824863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 0x556593cf4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01c37fe700 1 -- 172.21.15.33:0/2816824863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593ce5be0 msgr2=0x556593cc2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:45.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01c37fe700 1 --2- 172.21.15.33:0/2816824863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593ce5be0 0x556593cc2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01c37fe700 1 -- 172.21.15.33:0/2816824863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556593be2810 con 0x556593cc54c0 2023-11-06T22:16:45.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01c3fff700 1 --2- 172.21.15.33:0/2816824863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 0x556593cf4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:45.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01c37fe700 1 --2- 172.21.15.33:0/2816824863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593cc54c0 0x556593c99140 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f01b80504a0 tx=0x7f01b8032000).ready entity=mon.0 client_cookie=9a516492c200b554 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:45.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01aa1ff700 1 -- 172.21.15.33:0/2816824863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8050510 con 0x556593cc54c0 2023-11-06T22:16:45.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01aa1ff700 1 -- 172.21.15.33:0/2816824863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b8050970 con 0x556593cc54c0 2023-11-06T22:16:45.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.603+0000 7f01aa1ff700 1 -- 172.21.15.33:0/2816824863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b800dcf0 con 0x556593cc54c0 2023-11-06T22:16:45.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.604+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2816824863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593cc54c0 msgr2=0x556593c99140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.604+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2816824863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593cc54c0 0x556593c99140 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f01b80504a0 tx=0x7f01b8032000).stop 2023-11-06T22:16:45.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.604+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2816824863 shutdown_connections 2023-11-06T22:16:45.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.604+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2816824863 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593c9d080 0x556593cf4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.604+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2816824863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593ce5be0 0x556593cc2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.604+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2816824863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593cc54c0 0x556593c99140 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.604+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2816824863 >> 172.21.15.33:0/2816824863 conn(0x556593b06940 msgr2=0x556593c9d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:45.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.604+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2816824863 shutdown_connections 2023-11-06T22:16:45.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.604+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2816824863 wait complete. 2023-11-06T22:16:45.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.605+0000 7f01e16ecb80 1 -- start start 2023-11-06T22:16:45.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.605+0000 7f01e16ecb80 1 Processor -- start 2023-11-06T22:16:45.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 0x556593d41e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 0x556593d47dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01e16ecb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593ce10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556593d48880 con 0x556593ce5be0 2023-11-06T22:16:45.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556593cc5f60 con 0x556593c9d080 2023-11-06T22:16:45.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01e16ecb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x556593cc61a0 con 0x556593cc54c0 2023-11-06T22:16:45.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01e16ecb80 10 client.0 ll_register_callbacks cb 0x556593b8ece0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T22:16:45.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01c899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593ce10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01c899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593ce10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41980/0 (socket says 172.21.15.33:41980) 2023-11-06T22:16:45.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01c899c700 1 -- 172.21.15.33:0/2473201180 learned_addr learned my addr 172.21.15.33:0/2473201180 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:45.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01c3fff700 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 0x556593d41e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.606+0000 7f01c37fe700 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 0x556593d47dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01c899c700 1 -- 172.21.15.33:0/2473201180 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 msgr2=0x556593d47dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:45.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01c899c700 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 0x556593d47dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01c899c700 1 -- 172.21.15.33:0/2473201180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 msgr2=0x556593d41e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:45.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01c899c700 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 0x556593d41e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01c899c700 1 -- 172.21.15.33:0/2473201180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b400f040 con 0x556593ce5be0 2023-11-06T22:16:45.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01c899c700 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593ce10f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f01bc011750 tx=0x7f01bc021000).ready entity=mon.0 client_cookie=7370897874e224fd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:45.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01a27fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T22:16:45.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01a27fc700 1 -- 172.21.15.33:0/2473201180 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc001200 con 0x556593ce5be0 2023-11-06T22:16:45.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01a27fc700 1 -- 172.21.15.33:0/2473201180 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f01bc0113c0 con 0x556593ce5be0 2023-11-06T22:16:45.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2473201180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01b8050330 con 0x556593ce5be0 2023-11-06T22:16:45.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01a27fc700 1 -- 172.21.15.33:0/2473201180 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01bc0074a0 con 0x556593ce5be0 2023-11-06T22:16:45.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01e16ecb80 10 client.9522 fetch_fsmap learned FSMap version 84 2023-11-06T22:16:45.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.607+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2473201180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x7f01b8035040 con 0x556593ce5be0 2023-11-06T22:16:45.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01a27fc700 1 -- 172.21.15.33:0/2473201180 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b8050330 con 0x556593ce5be0 2023-11-06T22:16:45.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01a27fc700 1 -- 172.21.15.33:0/2473201180 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f01bc005d10 con 0x556593ce5be0 2023-11-06T22:16:45.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01e16ecb80 10 client.9522 fetch_fsmap finished waiting for FSMap version 84 2023-11-06T22:16:45.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01e16ecb80 10 client.9522 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T22:16:45.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01e16ecb80 20 client.9522 populate_metadata read hostname 'smithi033' 2023-11-06T22:16:45.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.33:6833/3197264665,v1:172.21.15.33:6835/3197264665] conn(0x556593ce2130 0x556593ce2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01e16ecb80 4 client.9522 mds_command: new command op to 9387 tid=0[{"prefix": "status"}] 2023-11-06T22:16:45.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2473201180 --> [v2:172.21.15.33:6833/3197264665,v1:172.21.15.33:6835/3197264665] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556593a8c040 con 0x556593ce2130 2023-11-06T22:16:45.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01c3fff700 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.33:6833/3197264665,v1:172.21.15.33:6835/3197264665] conn(0x556593ce2130 0x556593ce2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01c3fff700 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.33:6833/3197264665,v1:172.21.15.33:6835/3197264665] conn(0x556593ce2130 0x556593ce2510 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=dc6fd914f46df1ac server_cookie=aae9e89819a5c071 in_seq=0 out_seq=0 2023-11-06T22:16:45.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.608+0000 7f01a27fc700 10 client.9522 ms_handle_connect on v2:172.21.15.33:6833/3197264665 2023-11-06T22:16:45.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01a27fc700 1 -- 172.21.15.33:0/2473201180 <== mds.0 v2:172.21.15.33:6833/3197264665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x556593a8c040 con 0x556593ce2130 2023-11-06T22:16:45.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01a27fc700 10 client.9522 handle_command_reply: tid=0 2023-11-06T22:16:45.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 client.9522 shutdown 2023-11-06T22:16:45.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2473201180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 msgr2=0x556593ce10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593ce10f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f01bc011750 tx=0x7f01bc021000).stop 2023-11-06T22:16:45.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2473201180 shutdown_connections 2023-11-06T22:16:45.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.33:6833/3197264665,v1:172.21.15.33:6835/3197264665] conn(0x556593ce2130 0x556593ce2510 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556593ce5be0 0x556593ce10f0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556593c9d080 0x556593d41e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 --2- 172.21.15.33:0/2473201180 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x556593cc54c0 0x556593d47dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01a27fc700 0 client.9522 ms_handle_reset on v2:172.21.15.33:6833/3197264665 2023-11-06T22:16:45.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2473201180 >> 172.21.15.33:0/2473201180 conn(0x556593b06940 msgr2=0x556593c9d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:45.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2473201180 shutdown_connections 2023-11-06T22:16:45.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 1 -- 172.21.15.33:0/2473201180 wait complete. 2023-11-06T22:16:45.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.609+0000 7f01e16ecb80 20 client.9522 trim_cache size 0 max 16384 2023-11-06T22:16:45.627 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:16:45.627 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T22:16:45.627 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T22:16:45.627 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 9387, 2023-11-06T22:16:45.627 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T22:16:45.628 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T22:16:45.628 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T22:16:45.628 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.4623486889999997, 2023-11-06T22:16:45.628 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 84, 2023-11-06T22:16:45.628 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 204, 2023-11-06T22:16:45.628 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 204, 2023-11-06T22:16:45.629 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 11.708291983000001 2023-11-06T22:16:45.629 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:16:45.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 -- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01b005f240 msgr2=0x7f01b0061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01b005f240 0x7f01b0061700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f01c40c2cd0 tx=0x7f01b4009040).stop 2023-11-06T22:16:45.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 -- 172.21.15.33:0/2331814735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c414e6a0 msgr2=0x7f01c40ad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c414e6a0 0x7f01c40ad7b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f01b8006d60 tx=0x7f01b8016000).stop 2023-11-06T22:16:45.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 -- 172.21.15.33:0/2331814735 shutdown_connections 2023-11-06T22:16:45.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01b005f240 0x7f01b0061700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c40c5ea0 0x7f01c414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c4158190 0x7f01c40c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 --2- 172.21.15.33:0/2331814735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c414e6a0 0x7f01c40ad7b0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 -- 172.21.15.33:0/2331814735 >> 172.21.15.33:0/2331814735 conn(0x7f01c405e1b0 msgr2=0x7f01c414d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 -- 172.21.15.33:0/2331814735 shutdown_connections 2023-11-06T22:16:45.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.610+0000 7f01ca3ff700 1 -- 172.21.15.33:0/2331814735 wait complete. 2023-11-06T22:16:45.633 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T22:16:45.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.794+0000 7ff53b5ff700 1 Processor -- start 2023-11-06T22:16:45.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.795+0000 7ff53b5ff700 1 -- start start 2023-11-06T22:16:45.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.796+0000 7ff53b5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5341541a0 0x7ff5340c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.796+0000 7ff53b5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff5340a9ee0 0x7ff5340c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.796+0000 7ff53b5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5340c5f80 con 0x7ff5340c07b0 2023-11-06T22:16:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.796+0000 7ff53b5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5340aa740 con 0x7ff5340a9ee0 2023-11-06T22:16:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.796+0000 7ff53b5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c6800 con 0x7ff5341541a0 2023-11-06T22:16:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.796+0000 7ff539b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff5340c07b0 0x7ff5340a7c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58090/0 (socket says 172.21.15.33:58090) 2023-11-06T22:16:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.796+0000 7ff539b9c700 1 -- 172.21.15.33:0/1257076442 learned_addr learned my addr 172.21.15.33:0/1257076442 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.797+0000 7ff53939b700 1 --2- 172.21.15.33:0/1257076442 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5341541a0 0x7ff5340c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.797+0000 7ff538b9a700 1 --2- 172.21.15.33:0/1257076442 >> v2:172.21.15.69:3300/0 conn(0x7ff5340a9ee0 0x7ff5340c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.797+0000 7ff52bfff700 1 -- 172.21.15.33:0/1257076442 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397934702 0 0) 0x7ff5340c5f80 con 0x7ff5340c07b0 2023-11-06T22:16:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.797+0000 7ff52bfff700 1 -- 172.21.15.33:0/1257076442 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5200033f0 con 0x7ff5340c07b0 2023-11-06T22:16:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.797+0000 7ff53939b700 1 -- 172.21.15.33:0/1257076442 >> v2:172.21.15.69:3300/0 conn(0x7ff5340a9ee0 msgr2=0x7ff5340c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.797+0000 7ff53939b700 1 --2- 172.21.15.33:0/1257076442 >> v2:172.21.15.69:3300/0 conn(0x7ff5340a9ee0 0x7ff5340c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.797+0000 7ff53939b700 1 -- 172.21.15.33:0/1257076442 >> v1:172.21.15.33:6789/0 conn(0x7ff5340c07b0 legacy=0x7ff5340a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.797+0000 7ff52bfff700 1 -- 172.21.15.33:0/1257076442 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1874540009 0 0) 0x7ff5200033f0 con 0x7ff5340c07b0 2023-11-06T22:16:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.797+0000 7ff53939b700 1 -- 172.21.15.33:0/1257076442 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340aa460 con 0x7ff5341541a0 2023-11-06T22:16:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.797+0000 7ff53939b700 1 --2- 172.21.15.33:0/1257076442 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5341541a0 0x7ff5340c8220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7ff530006fd0 tx=0x7ff530006f40).ready entity=mon.2 client_cookie=89e666aa43250422 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff52bfff700 1 -- 172.21.15.33:0/1257076442 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0017f0 con 0x7ff5341541a0 2023-11-06T22:16:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff52bfff700 1 -- 172.21.15.33:0/1257076442 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff530005880 con 0x7ff5341541a0 2023-11-06T22:16:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff52bfff700 1 -- 172.21.15.33:0/1257076442 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530005c00 con 0x7ff5341541a0 2023-11-06T22:16:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1257076442 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5341541a0 msgr2=0x7ff5340c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff53b5ff700 1 --2- 172.21.15.33:0/1257076442 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5341541a0 0x7ff5340c8220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7ff530006fd0 tx=0x7ff530006f40).stop 2023-11-06T22:16:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1257076442 shutdown_connections 2023-11-06T22:16:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff53b5ff700 1 --2- 172.21.15.33:0/1257076442 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5341541a0 0x7ff5340c8220 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff53b5ff700 1 --2- 172.21.15.33:0/1257076442 >> v2:172.21.15.69:3300/0 conn(0x7ff5340a9ee0 0x7ff5340c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1257076442 >> 172.21.15.33:0/1257076442 conn(0x7ff5340944e0 msgr2=0x7ff53406a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1257076442 shutdown_connections 2023-11-06T22:16:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.798+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1257076442 wait complete. 2023-11-06T22:16:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.800+0000 7ff53b5ff700 1 Processor -- start 2023-11-06T22:16:45.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.800+0000 7ff53b5ff700 1 -- start start 2023-11-06T22:16:45.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff53b5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5340a9ee0 0x7ff53414f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff53b5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5340c07b0 0x7ff5340c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff53b5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5341541a0 0x7ff5340c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff53b5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c4b90 con 0x7ff5341541a0 2023-11-06T22:16:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff53b5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c4260 con 0x7ff5340c07b0 2023-11-06T22:16:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff53b5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c4540 con 0x7ff5340a9ee0 2023-11-06T22:16:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff539b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5341541a0 0x7ff5340c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff53939b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5340a9ee0 0x7ff53414f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff538b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5340c07b0 0x7ff5340c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff538b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5340c07b0 0x7ff5340c33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:41050/0 (socket says 172.21.15.33:41050) 2023-11-06T22:16:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff53939b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5340a9ee0 0x7ff53414f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:46558/0 (socket says 172.21.15.33:46558) 2023-11-06T22:16:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff538b9a700 1 -- 172.21.15.33:0/1039154521 learned_addr learned my addr 172.21.15.33:0/1039154521 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.801+0000 7ff539b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5341541a0 0x7ff5340c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41992/0 (socket says 172.21.15.33:41992) 2023-11-06T22:16:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff538b9a700 1 -- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5340a9ee0 msgr2=0x7ff53414f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff538b9a700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5340a9ee0 0x7ff53414f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff538b9a700 1 -- 172.21.15.33:0/1039154521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5341541a0 msgr2=0x7ff5340c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff538b9a700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5341541a0 0x7ff5340c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff538b9a700 1 -- 172.21.15.33:0/1039154521 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff530009250 con 0x7ff5340c07b0 2023-11-06T22:16:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff539b9c700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5341541a0 0x7ff5340c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:16:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff538b9a700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5340c07b0 0x7ff5340c33d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7ff52400e900 tx=0x7ff524009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff5297fa700 1 -- 172.21.15.33:0/1039154521 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524012600 con 0x7ff5340c07b0 2023-11-06T22:16:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff5297fa700 1 -- 172.21.15.33:0/1039154521 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff524012760 con 0x7ff5340c07b0 2023-11-06T22:16:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff5297fa700 1 -- 172.21.15.33:0/1039154521 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524012ae0 con 0x7ff5340c07b0 2023-11-06T22:16:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.802+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1039154521 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5340c4820 con 0x7ff5340c07b0 2023-11-06T22:16:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.803+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1039154521 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53414f640 con 0x7ff5340c07b0 2023-11-06T22:16:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.804+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1039154521 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5340475a0 con 0x7ff5340c07b0 2023-11-06T22:16:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.804+0000 7ff5297fa700 1 -- 172.21.15.33:0/1039154521 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff52401b400 con 0x7ff5340c07b0 2023-11-06T22:16:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.806+0000 7ff5297fa700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff51805f270 0x7ff518061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.806+0000 7ff5297fa700 1 -- 172.21.15.33:0/1039154521 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7ff52407a8a0 con 0x7ff5340c07b0 2023-11-06T22:16:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.806+0000 7ff53939b700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff51805f270 0x7ff518061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.806+0000 7ff53939b700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff51805f270 0x7ff518061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff530006f80 tx=0x7ff530015000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:45.810+0000 7ff5297fa700 1 -- 172.21.15.33:0/1039154521 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff52404b8a0 con 0x7ff5340c07b0 2023-11-06T22:16:46.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.036+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1039154521 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff5340c61b0 con 0x7ff5340c07b0 2023-11-06T22:16:46.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.037+0000 7ff5297fa700 1 -- 172.21.15.33:0/1039154521 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff52404eec0 con 0x7ff5340c07b0 2023-11-06T22:16:46.040 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T22:16:46.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff51805f270 msgr2=0x7ff518061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:46.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff51805f270 0x7ff518061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff530006f80 tx=0x7ff530015000).stop 2023-11-06T22:16:46.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5340c07b0 msgr2=0x7ff5340c33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:46.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5340c07b0 0x7ff5340c33d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7ff52400e900 tx=0x7ff524009060).stop 2023-11-06T22:16:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1039154521 shutdown_connections 2023-11-06T22:16:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff51805f270 0x7ff518061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff5340a9ee0 0x7ff53414f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5341541a0 0x7ff5340c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 --2- 172.21.15.33:0/1039154521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff5340c07b0 0x7ff5340c33d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1039154521 >> 172.21.15.33:0/1039154521 conn(0x7ff5340944e0 msgr2=0x7ff5340c6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1039154521 shutdown_connections 2023-11-06T22:16:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.039+0000 7ff53b5ff700 1 -- 172.21.15.33:0/1039154521 wait complete. 2023-11-06T22:16:46.056 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T22:16:46.057 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T22:16:46.057 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T22:16:46.058 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T22:16:46.058 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T22:16:46.058 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T22:16:46.059 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T22:16:46.059 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T22:16:46.059 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T22:16:46.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:16:46.060 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T22:16:46.109 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T22:16:46.109 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T22:16:46.109 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T22:16:46.109 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:16:46.110 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T22:16:46.110 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:16:46.110 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T22:16:46.110 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:16:46.110 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T22:16:46.111 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T22:16:46.111 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:16:46.111 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T22:16:46.111 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T22:16:46.111 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:16:46.112 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T22:16:46.112 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T22:16:46.112 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 36738sec preferred_lft 36738sec 2023-11-06T22:16:46.112 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T22:16:46.112 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:16:46.113 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T22:16:46.113 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:16:46.113 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T22:16:46.113 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T22:16:46.113 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T22:16:46.114 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T22:16:46.114 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:16:46.114 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T22:16:46.114 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:16:46.115 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T22:16:46.115 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T22:16:46.115 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T22:16:46.115 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:16:46.115 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T22:16:46.115 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T22:16:46.116 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T22:16:46.116 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:16:46.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:16:46.118 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T22:16:46.187 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T22:16:46.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:16:46.191 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T22:16:46.243 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T22:16:46.243 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T22:16:46.243 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T22:16:46.243 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T22:16:46.244 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T22:16:46.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:16:46.245 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T22:16:46.245 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T22:16:46.246 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T22:16:46.246 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T22:16:46.247 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T22:16:46.247 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T22:16:46.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:16:46.423 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T22:16:46.470 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T22:16:46.470 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T22:16:46.471 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T22:16:46.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:16:46.472 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T22:16:46.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:16:46.546 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T22:16:46.593 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T22:16:46.594 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T22:16:46.697 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T22:16:46.698 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T22:16:46.725 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T22:16:46.725 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T22:16:46.730 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T22:16:46.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:16:46.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T22:16:46.805 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T22:16:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.999+0000 7f1418bff700 1 Processor -- start 2023-11-06T22:16:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:46.999+0000 7f1418bff700 1 -- start start 2023-11-06T22:16:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.001+0000 7f1418bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f141413ce50 0x7f141413d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.001+0000 7f1418bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1414062840 con 0x7f14141586f0 2023-11-06T22:16:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.001+0000 7f1418bff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1414062b20 con 0x7f14140c4560 2023-11-06T22:16:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.001+0000 7f1418bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1414158d50 con 0x7f141413ce50 2023-11-06T22:16:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.001+0000 7f1412d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f14141586f0 0x7f1414064bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58092/0 (socket says 172.21.15.33:58092) 2023-11-06T22:16:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.001+0000 7f1412d9d700 1 -- 172.21.15.33:0/2768704941 learned_addr learned my addr 172.21.15.33:0/2768704941 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.001+0000 7f141259c700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f141413ce50 0x7f141413d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.001+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936876066 0 0) 0x7f1414062840 con 0x7f14141586f0 2023-11-06T22:16:47.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13fc0033f0 con 0x7f14141586f0 2023-11-06T22:16:47.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386257399 0 0) 0x7f1414062b20 con 0x7f14140c4560 2023-11-06T22:16:47.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1414062840 con 0x7f14140c4560 2023-11-06T22:16:47.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1455777598 0 0) 0x7f13fc0033f0 con 0x7f14141586f0 2023-11-06T22:16:47.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f141413ce50 msgr2=0x7f141413d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:47.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f141413ce50 0x7f141413d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 >> v1:172.21.15.69:6789/0 conn(0x7f14140c4560 legacy=0x7f1414157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1414062e00 con 0x7f14141586f0 2023-11-06T22:16:47.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f140c0017f0 con 0x7f14141586f0 2023-11-06T22:16:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 >> v1:172.21.15.33:6789/0 conn(0x7f14141586f0 legacy=0x7f1414064bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13fc003aa0 0x7f13fc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f141413ce50 0x7f13fc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13fc004b80 0x7f13fc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1414158d50 con 0x7f13fc004b80 2023-11-06T22:16:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc00b910 con 0x7f13fc003aa0 2023-11-06T22:16:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc00bad0 con 0x7f141413ce50 2023-11-06T22:16:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f141259c700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13fc003aa0 0x7f13fc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f1412d9d700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13fc004b80 0x7f13fc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.002+0000 7f1411d9b700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f141413ce50 0x7f13fc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.003+0000 7f141259c700 1 -- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f141413ce50 msgr2=0x7f13fc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:47.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.003+0000 7f141259c700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f141413ce50 0x7f13fc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.003+0000 7f141259c700 1 -- 172.21.15.33:0/2768704941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13fc004b80 msgr2=0x7f13fc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:47.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.003+0000 7f141259c700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13fc004b80 0x7f13fc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.003+0000 7f141259c700 1 -- 172.21.15.33:0/2768704941 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc0033f0 con 0x7f13fc003aa0 2023-11-06T22:16:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.003+0000 7f141259c700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13fc003aa0 0x7f13fc003ea0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f1404009a80 tx=0x7f1404009060).ready entity=mon.1 client_cookie=e7e89acb10765fb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.003+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404009430 con 0x7f13fc003aa0 2023-11-06T22:16:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.003+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140400e640 con 0x7f13fc003aa0 2023-11-06T22:16:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.003+0000 7f141159a700 1 -- 172.21.15.33:0/2768704941 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400e9c0 con 0x7f13fc003aa0 2023-11-06T22:16:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.004+0000 7f1418bff700 1 -- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13fc003aa0 msgr2=0x7f13fc003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.004+0000 7f1418bff700 1 --2- 172.21.15.33:0/2768704941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13fc003aa0 0x7f13fc003ea0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f1404009a80 tx=0x7f1404009060).stop 2023-11-06T22:16:47.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.004+0000 7f1412d9d700 1 -- 172.21.15.33:0/2768704941 reap_dead start 2023-11-06T22:16:47.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.004+0000 7f1418bff700 1 -- 172.21.15.33:0/2768704941 shutdown_connections 2023-11-06T22:16:47.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.004+0000 7f1418bff700 1 -- 172.21.15.33:0/2768704941 >> 172.21.15.33:0/2768704941 conn(0x7f14140c7dc0 msgr2=0x7f14140c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:47.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.004+0000 7f1418bff700 1 -- 172.21.15.33:0/2768704941 shutdown_connections 2023-11-06T22:16:47.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.004+0000 7f1418bff700 1 -- 172.21.15.33:0/2768704941 wait complete. 2023-11-06T22:16:47.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.006+0000 7f1418bff700 1 Processor -- start 2023-11-06T22:16:47.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.006+0000 7f1418bff700 1 -- start start 2023-11-06T22:16:47.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f1418bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f13fc003aa0 0x7f14140aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f1418bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14140acc70 0x7f1414139c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f1418bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141413a130 0x7f14141566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f1418bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1414164800 con 0x7f14140acc70 2023-11-06T22:16:47.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f1418bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14140ab7a0 con 0x7f141413a130 2023-11-06T22:16:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f1418bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14140aba80 con 0x7f13fc003aa0 2023-11-06T22:16:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f1411d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14140acc70 0x7f1414139c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f1411d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14140acc70 0x7f1414139c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42022/0 (socket says 172.21.15.33:42022) 2023-11-06T22:16:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f1411d9b700 1 -- 172.21.15.33:0/1203575473 learned_addr learned my addr 172.21.15.33:0/1203575473 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f141259c700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f13fc003aa0 0x7f14140aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.007+0000 7f1412d9d700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141413a130 0x7f14141566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f1411d9b700 1 -- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f13fc003aa0 msgr2=0x7f14140aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f1411d9b700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f13fc003aa0 0x7f14140aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f1411d9b700 1 -- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141413a130 msgr2=0x7f14141566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f1411d9b700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141413a130 0x7f14141566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f1411d9b700 1 -- 172.21.15.33:0/1203575473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14140abd60 con 0x7f14140acc70 2023-11-06T22:16:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f141259c700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f13fc003aa0 0x7f14140aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f1412d9d700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141413a130 0x7f14141566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:16:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f1411d9b700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14140acc70 0x7f1414139c40 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7f14080010e0 tx=0x7f1408007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f14027fc700 1 -- 172.21.15.33:0/1203575473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140800f8d0 con 0x7f14140acc70 2023-11-06T22:16:47.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f14027fc700 1 -- 172.21.15.33:0/1203575473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140800fa30 con 0x7f14140acc70 2023-11-06T22:16:47.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f1418bff700 1 -- 172.21.15.33:0/1203575473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14140ac040 con 0x7f14140acc70 2023-11-06T22:16:47.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f14027fc700 1 -- 172.21.15.33:0/1203575473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140800fdb0 con 0x7f14140acc70 2023-11-06T22:16:47.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.008+0000 7f1418bff700 1 -- 172.21.15.33:0/1203575473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1414156ba0 con 0x7f14140acc70 2023-11-06T22:16:47.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.009+0000 7f1418bff700 1 -- 172.21.15.33:0/1203575473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1414069be0 con 0x7f14140acc70 2023-11-06T22:16:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.010+0000 7f14027fc700 1 -- 172.21.15.33:0/1203575473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f1408014660 con 0x7f14140acc70 2023-11-06T22:16:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.011+0000 7f14027fc700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f13f405f270 0x7f13f4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.011+0000 7f14027fc700 1 -- 172.21.15.33:0/1203575473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f140807a2f0 con 0x7f14140acc70 2023-11-06T22:16:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.011+0000 7f141259c700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f13f405f270 0x7f13f4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.012+0000 7f141259c700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f13f405f270 0x7f13f4061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1404006d50 tx=0x7f140401a000).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:47.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.015+0000 7f14027fc700 1 -- 172.21.15.33:0/1203575473 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f140804aaf0 con 0x7f14140acc70 2023-11-06T22:16:47.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.224+0000 7f1418bff700 1 -- 172.21.15.33:0/1203575473 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1414063280 con 0x7f13f405f270 2023-11-06T22:16:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.226+0000 7f14027fc700 1 -- 172.21.15.33:0/1203575473 <== mgr.7733 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f1414063280 con 0x7f13f405f270 2023-11-06T22:16:47.229 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T22:16:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:16:47.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T22:16:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:16:47.230 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T22:16:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.228+0000 7f1418bff700 1 -- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f13f405f270 msgr2=0x7f13f4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.228+0000 7f1418bff700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f13f405f270 0x7f13f4061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1404006d50 tx=0x7f140401a000).stop 2023-11-06T22:16:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.228+0000 7f1418bff700 1 -- 172.21.15.33:0/1203575473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14140acc70 msgr2=0x7f1414139c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.228+0000 7f1418bff700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14140acc70 0x7f1414139c40 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7f14080010e0 tx=0x7f1408007bb0).stop 2023-11-06T22:16:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.228+0000 7f1418bff700 1 -- 172.21.15.33:0/1203575473 shutdown_connections 2023-11-06T22:16:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.228+0000 7f1418bff700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f13f405f270 0x7f13f4061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.228+0000 7f1418bff700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f13fc003aa0 0x7f14140aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.228+0000 7f1418bff700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f141413a130 0x7f14141566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.228+0000 7f1418bff700 1 --2- 172.21.15.33:0/1203575473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14140acc70 0x7f1414139c40 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.228+0000 7f1418bff700 1 -- 172.21.15.33:0/1203575473 >> 172.21.15.33:0/1203575473 conn(0x7f14140c7dc0 msgr2=0x7f14140c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.229+0000 7f1418bff700 1 -- 172.21.15.33:0/1203575473 shutdown_connections 2023-11-06T22:16:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.229+0000 7f1418bff700 1 -- 172.21.15.33:0/1203575473 wait complete. 2023-11-06T22:16:47.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T22:16:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.445+0000 7ff6f21ff700 1 Processor -- start 2023-11-06T22:16:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.445+0000 7ff6f21ff700 1 -- start start 2023-11-06T22:16:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.446+0000 7ff6f21ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff6ec155030 0x7ff6ec062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.446+0000 7ff6f21ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff6ec155410 0x7ff6ec0644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.446+0000 7ff6f21ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec063c00 con 0x7ff6ec1547d0 2023-11-06T22:16:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.446+0000 7ff6f21ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec064090 con 0x7ff6ec155410 2023-11-06T22:16:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.446+0000 7ff6f21ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec062ff0 con 0x7ff6ec155030 2023-11-06T22:16:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.447+0000 7ff6eaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff6ec155410 0x7ff6ec0644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.447+0000 7ff6eaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff6ec155410 0x7ff6ec0644f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42034/0 (socket says 172.21.15.33:42034) 2023-11-06T22:16:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.447+0000 7ff6eaffd700 1 -- 172.21.15.33:0/3003242207 learned_addr learned my addr 172.21.15.33:0/3003242207 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.447+0000 7ff6eb7fe700 1 --2- 172.21.15.33:0/3003242207 >> v2:172.21.15.69:3300/0 conn(0x7ff6ec155030 0x7ff6ec062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.447+0000 7ff6eaffd700 1 -- 172.21.15.33:0/3003242207 >> v2:172.21.15.69:3300/0 conn(0x7ff6ec155030 msgr2=0x7ff6ec062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.447+0000 7ff6eaffd700 1 --2- 172.21.15.33:0/3003242207 >> v2:172.21.15.69:3300/0 conn(0x7ff6ec155030 0x7ff6ec062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.447+0000 7ff6eaffd700 1 -- 172.21.15.33:0/3003242207 >> v1:172.21.15.69:6789/0 conn(0x7ff6ec1547d0 legacy=0x7ff6ec157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.447+0000 7ff6eaffd700 1 -- 172.21.15.33:0/3003242207 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0632d0 con 0x7ff6ec155410 2023-11-06T22:16:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.447+0000 7ff6eaffd700 1 --2- 172.21.15.33:0/3003242207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 0x7ff6ec0644f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7ff6ec06bc40 tx=0x7ff6e00090f0).ready entity=mon.0 client_cookie=12d9011e7e4d434d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6ea7fc700 1 -- 172.21.15.33:0/3003242207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000b520 con 0x7ff6ec155410 2023-11-06T22:16:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6ea7fc700 1 -- 172.21.15.33:0/3003242207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e000b680 con 0x7ff6ec155410 2023-11-06T22:16:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6ea7fc700 1 -- 172.21.15.33:0/3003242207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000ba00 con 0x7ff6ec155410 2023-11-06T22:16:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6f21ff700 1 -- 172.21.15.33:0/3003242207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 msgr2=0x7ff6ec0644f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6f21ff700 1 --2- 172.21.15.33:0/3003242207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 0x7ff6ec0644f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7ff6ec06bc40 tx=0x7ff6e00090f0).stop 2023-11-06T22:16:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6f21ff700 1 -- 172.21.15.33:0/3003242207 shutdown_connections 2023-11-06T22:16:47.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6f21ff700 1 --2- 172.21.15.33:0/3003242207 >> v2:172.21.15.69:3300/0 conn(0x7ff6ec155030 0x7ff6ec062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6f21ff700 1 --2- 172.21.15.33:0/3003242207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 0x7ff6ec0644f0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6f21ff700 1 -- 172.21.15.33:0/3003242207 >> 172.21.15.33:0/3003242207 conn(0x7ff6ec0c3d30 msgr2=0x7ff6ec0c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:47.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6f21ff700 1 -- 172.21.15.33:0/3003242207 shutdown_connections 2023-11-06T22:16:47.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.448+0000 7ff6f21ff700 1 -- 172.21.15.33:0/3003242207 wait complete. 2023-11-06T22:16:47.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.450+0000 7ff6f21ff700 1 Processor -- start 2023-11-06T22:16:47.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.450+0000 7ff6f21ff700 1 -- start start 2023-11-06T22:16:47.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.451+0000 7ff6f21ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6ec1547d0 0x7ff6ec167a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.451+0000 7ff6f21ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6ec155030 0x7ff6ec13d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.451+0000 7ff6f21ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 0x7ff6ec164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.451+0000 7ff6f21ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec136570 con 0x7ff6ec155410 2023-11-06T22:16:47.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.451+0000 7ff6f21ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec167f60 con 0x7ff6ec155030 2023-11-06T22:16:47.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.451+0000 7ff6f21ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec164440 con 0x7ff6ec1547d0 2023-11-06T22:16:47.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.451+0000 7ff6ebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 0x7ff6ec164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.451+0000 7ff6ebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 0x7ff6ec164bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42048/0 (socket says 172.21.15.33:42048) 2023-11-06T22:16:47.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.451+0000 7ff6ebfff700 1 -- 172.21.15.33:0/4067128842 learned_addr learned my addr 172.21.15.33:0/4067128842 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:47.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6eaffd700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6ec155030 0x7ff6ec13d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6eb7fe700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6ec1547d0 0x7ff6ec167a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6ebfff700 1 -- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6ec1547d0 msgr2=0x7ff6ec167a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6ebfff700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6ec1547d0 0x7ff6ec167a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6ebfff700 1 -- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6ec155030 msgr2=0x7ff6ec13d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:47.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6ebfff700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6ec155030 0x7ff6ec13d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:47.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6ebfff700 1 -- 172.21.15.33:0/4067128842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e0009370 con 0x7ff6ec155410 2023-11-06T22:16:47.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6ebfff700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 0x7ff6ec164bb0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7ff6e400c7c0 tx=0x7ff6e4001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:47.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6d37fe700 1 -- 172.21.15.33:0/4067128842 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4010510 con 0x7ff6ec155410 2023-11-06T22:16:47.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6d37fe700 1 -- 172.21.15.33:0/4067128842 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e4010670 con 0x7ff6ec155410 2023-11-06T22:16:47.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6d37fe700 1 -- 172.21.15.33:0/4067128842 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40109f0 con 0x7ff6ec155410 2023-11-06T22:16:47.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6f21ff700 1 -- 172.21.15.33:0/4067128842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec164720 con 0x7ff6ec155410 2023-11-06T22:16:47.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.452+0000 7ff6f21ff700 1 -- 172.21.15.33:0/4067128842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec168450 con 0x7ff6ec155410 2023-11-06T22:16:47.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.453+0000 7ff6f21ff700 1 -- 172.21.15.33:0/4067128842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec1648b0 con 0x7ff6ec155410 2023-11-06T22:16:47.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.454+0000 7ff6d37fe700 1 -- 172.21.15.33:0/4067128842 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 88) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff6e401b400 con 0x7ff6ec155410 2023-11-06T22:16:47.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.455+0000 7ff6d37fe700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff6d805f270 0x7ff6d8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:47.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.455+0000 7ff6d37fe700 1 -- 172.21.15.33:0/4067128842 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7ff6e407a190 con 0x7ff6ec155410 2023-11-06T22:16:47.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.455+0000 7ff6eb7fe700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff6d805f270 0x7ff6d8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:47.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.456+0000 7ff6eb7fe700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff6d805f270 0x7ff6d8061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff6dc009bc0 tx=0x7ff6dc008040).ready entity=mgr.7733 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:47.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.459+0000 7ff6d37fe700 1 -- 172.21.15.33:0/4067128842 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff6e404b250 con 0x7ff6ec155410 2023-11-06T22:16:47.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:47.688+0000 7ff6f21ff700 1 -- 172.21.15.33:0/4067128842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7ff6ec05e280 con 0x7ff6ec155410 2023-11-06T22:16:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.251+0000 7ff6d37fe700 1 -- 172.21.15.33:0/4067128842 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 89) v1 ==== 75565+0+0 (secure 0 0 0) 0x7ff6e4018d60 con 0x7ff6ec155410 2023-11-06T22:16:48.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.253+0000 7ff6d37fe700 1 -- 172.21.15.33:0/4067128842 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v89) v1 ==== 92+0+0 (secure 0 0 0) 0x7ff6e4072bd0 con 0x7ff6ec155410 2023-11-06T22:16:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.256+0000 7ff6f21ff700 1 -- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff6d805f270 msgr2=0x7ff6d8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.256+0000 7ff6f21ff700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff6d805f270 0x7ff6d8061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff6dc009bc0 tx=0x7ff6dc008040).stop 2023-11-06T22:16:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.257+0000 7ff6f21ff700 1 -- 172.21.15.33:0/4067128842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 msgr2=0x7ff6ec164bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.257+0000 7ff6f21ff700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 0x7ff6ec164bb0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7ff6e400c7c0 tx=0x7ff6e4001510).stop 2023-11-06T22:16:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.257+0000 7ff6f21ff700 1 -- 172.21.15.33:0/4067128842 shutdown_connections 2023-11-06T22:16:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.257+0000 7ff6f21ff700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff6d805f270 0x7ff6d8061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.257+0000 7ff6f21ff700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff6ec1547d0 0x7ff6ec167a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.257+0000 7ff6f21ff700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec155410 0x7ff6ec164bb0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.257+0000 7ff6f21ff700 1 --2- 172.21.15.33:0/4067128842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6ec155030 0x7ff6ec13d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.257+0000 7ff6f21ff700 1 -- 172.21.15.33:0/4067128842 >> 172.21.15.33:0/4067128842 conn(0x7ff6ec0c3d30 msgr2=0x7ff6ec0bc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.257+0000 7ff6f21ff700 1 -- 172.21.15.33:0/4067128842 shutdown_connections 2023-11-06T22:16:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.257+0000 7ff6f21ff700 1 -- 172.21.15.33:0/4067128842 wait complete. 2023-11-06T22:16:48.284 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T22:16:48.471 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:48.469+0000 7f88a4206680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:16:48.487 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:48.484+0000 7f0580f53680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:16:48.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.488+0000 7f23a9dff700 1 Processor -- start 2023-11-06T22:16:48.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.488+0000 7f23a9dff700 1 -- start start 2023-11-06T22:16:48.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.490+0000 7f23a9dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 0x7f23a40c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.490+0000 7f23a9dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a41556d0 con 0x7f23a41544f0 2023-11-06T22:16:48.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.490+0000 7f23a9dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a4154ac0 con 0x7f23a40646c0 2023-11-06T22:16:48.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.490+0000 7f23a9dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a41548d0 con 0x7f23a4064b40 2023-11-06T22:16:48.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.490+0000 7f23a37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f23a41544f0 0x7f23a40c0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51338/0 (socket says 172.21.15.33:51338) 2023-11-06T22:16:48.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.490+0000 7f23a37fe700 1 -- 172.21.15.33:0/3784171768 learned_addr learned my addr 172.21.15.33:0/3784171768 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:48.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.490+0000 7f23a3fff700 1 --2- 172.21.15.33:0/3784171768 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 0x7f23a40c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:48.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a27fc700 1 -- 172.21.15.33:0/3784171768 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463336962 0 0) 0x7f23a41556d0 con 0x7f23a41544f0 2023-11-06T22:16:48.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a27fc700 1 -- 172.21.15.33:0/3784171768 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f238c0033f0 con 0x7f23a41544f0 2023-11-06T22:16:48.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a27fc700 1 -- 172.21.15.33:0/3784171768 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907900498 0 0) 0x7f23a4154ac0 con 0x7f23a40646c0 2023-11-06T22:16:48.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a27fc700 1 -- 172.21.15.33:0/3784171768 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a41556d0 con 0x7f23a40646c0 2023-11-06T22:16:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a3fff700 1 -- 172.21.15.33:0/3784171768 >> v1:172.21.15.69:6789/0 conn(0x7f23a40646c0 legacy=0x7f23a40c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a3fff700 1 -- 172.21.15.33:0/3784171768 >> v1:172.21.15.33:6789/0 conn(0x7f23a41544f0 legacy=0x7f23a40c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a3fff700 1 -- 172.21.15.33:0/3784171768 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a4154da0 con 0x7f23a4064b40 2023-11-06T22:16:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a3fff700 1 --2- 172.21.15.33:0/3784171768 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 0x7f23a40c5ef0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f239c009e90 tx=0x7f239c009780).ready entity=mon.2 client_cookie=30577ba2bd90e84 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:48.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a27fc700 1 -- 172.21.15.33:0/3784171768 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c00e070 con 0x7f23a4064b40 2023-11-06T22:16:48.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a27fc700 1 -- 172.21.15.33:0/3784171768 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239c00bab0 con 0x7f23a4064b40 2023-11-06T22:16:48.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.491+0000 7f23a27fc700 1 -- 172.21.15.33:0/3784171768 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c00be30 con 0x7f23a4064b40 2023-11-06T22:16:48.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.492+0000 7f23a9dff700 1 -- 172.21.15.33:0/3784171768 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 msgr2=0x7f23a40c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.492+0000 7f23a9dff700 1 --2- 172.21.15.33:0/3784171768 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 0x7f23a40c5ef0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f239c009e90 tx=0x7f239c009780).stop 2023-11-06T22:16:48.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.492+0000 7f23a9dff700 1 -- 172.21.15.33:0/3784171768 shutdown_connections 2023-11-06T22:16:48.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.492+0000 7f23a9dff700 1 --2- 172.21.15.33:0/3784171768 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 0x7f23a40c5ef0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.492+0000 7f23a9dff700 1 -- 172.21.15.33:0/3784171768 >> 172.21.15.33:0/3784171768 conn(0x7f23a40cb900 msgr2=0x7f23a40c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.492+0000 7f23a9dff700 1 -- 172.21.15.33:0/3784171768 shutdown_connections 2023-11-06T22:16:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.492+0000 7f23a9dff700 1 -- 172.21.15.33:0/3784171768 wait complete. 2023-11-06T22:16:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.494+0000 7f23a9dff700 1 Processor -- start 2023-11-06T22:16:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.494+0000 7f23a9dff700 1 -- start start 2023-11-06T22:16:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a9dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a40646c0 0x7f23a4147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a9dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 0x7f23a40c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a9dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23a41544f0 0x7f23a40c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a9dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a40c3990 con 0x7f23a40646c0 2023-11-06T22:16:48.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a9dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a40c2d80 con 0x7f23a41544f0 2023-11-06T22:16:48.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a9dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a40c3060 con 0x7f23a4064b40 2023-11-06T22:16:48.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a40646c0 0x7f23a4147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a40646c0 0x7f23a4147d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39770/0 (socket says 172.21.15.33:39770) 2023-11-06T22:16:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a37fe700 1 -- 172.21.15.33:0/3199273645 learned_addr learned my addr 172.21.15.33:0/3199273645 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a2ffd700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 0x7f23a40c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a3fff700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23a41544f0 0x7f23a40c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a37fe700 1 -- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 msgr2=0x7f23a40c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a37fe700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 0x7f23a40c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a37fe700 1 -- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23a41544f0 msgr2=0x7f23a40c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a37fe700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23a41544f0 0x7f23a40c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.495+0000 7f23a37fe700 1 -- 172.21.15.33:0/3199273645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c009a20 con 0x7f23a40646c0 2023-11-06T22:16:48.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.496+0000 7f23a37fe700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a40646c0 0x7f23a4147d20 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7f239400c490 tx=0x7f2394007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:48.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.496+0000 7f238b7fe700 1 -- 172.21.15.33:0/3199273645 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23940070d0 con 0x7f23a40646c0 2023-11-06T22:16:48.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.496+0000 7f238b7fe700 1 -- 172.21.15.33:0/3199273645 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394007230 con 0x7f23a40646c0 2023-11-06T22:16:48.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.496+0000 7f238b7fe700 1 -- 172.21.15.33:0/3199273645 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23940075b0 con 0x7f23a40646c0 2023-11-06T22:16:48.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.496+0000 7f23a9dff700 1 -- 172.21.15.33:0/3199273645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a40c3340 con 0x7f23a40646c0 2023-11-06T22:16:48.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.496+0000 7f23a9dff700 1 -- 172.21.15.33:0/3199273645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a4148210 con 0x7f23a40646c0 2023-11-06T22:16:48.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.497+0000 7f23a9dff700 1 -- 172.21.15.33:0/3199273645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a40c3d00 con 0x7f23a40646c0 2023-11-06T22:16:48.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.498+0000 7f238b7fe700 1 -- 172.21.15.33:0/3199273645 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 89) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f23940059b0 con 0x7f23a40646c0 2023-11-06T22:16:48.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.499+0000 7f238b7fe700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f239005f2c0 0x7f2390061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.499+0000 7f238b7fe700 1 -- 172.21.15.33:0/3199273645 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f2394079b40 con 0x7f23a40646c0 2023-11-06T22:16:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.499+0000 7f23a2ffd700 1 -- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f239005f2c0 msgr2=0x7f2390061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.499+0000 7f23a2ffd700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f239005f2c0 0x7f2390061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.503+0000 7f238b7fe700 1 -- 172.21.15.33:0/3199273645 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f239404aba0 con 0x7f23a40646c0 2023-11-06T22:16:48.551 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:48.549+0000 7f88a4206680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:16:48.573 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:48.570+0000 7f0580f53680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:16:48.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.687+0000 7f23a9dff700 1 -- 172.21.15.33:0/3199273645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f23a4155080 con 0x7f23a40646c0 2023-11-06T22:16:48.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.690+0000 7f238b7fe700 1 -- 172.21.15.33:0/3199273645 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v39)=0 v39) v1 ==== 138+0+0 (secure 0 0 0) 0x7f239404e1c0 con 0x7f23a40646c0 2023-11-06T22:16:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 -- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f239005f2c0 msgr2=0x7f2390061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:48.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f239005f2c0 0x7f2390061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:48.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 -- 172.21.15.33:0/3199273645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a40646c0 msgr2=0x7f23a4147d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a40646c0 0x7f23a4147d20 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7f239400c490 tx=0x7f2394007d20).stop 2023-11-06T22:16:48.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 -- 172.21.15.33:0/3199273645 shutdown_connections 2023-11-06T22:16:48.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f239005f2c0 0x7f2390061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:48.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23a41544f0 0x7f23a40c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a40646c0 0x7f23a4147d20 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 --2- 172.21.15.33:0/3199273645 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f23a4064b40 0x7f23a40c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 -- 172.21.15.33:0/3199273645 >> 172.21.15.33:0/3199273645 conn(0x7f23a40cb900 msgr2=0x7f23a4153c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:48.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 -- 172.21.15.33:0/3199273645 shutdown_connections 2023-11-06T22:16:48.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.693+0000 7f23a9dff700 1 -- 172.21.15.33:0/3199273645 wait complete. 2023-11-06T22:16:48.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:48.714 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-06T22:16:48.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-11-06T22:16:48.877 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:48.875+0000 7f88a4206680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:16:48.930 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:48.928+0000 7f0580f53680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:16:48.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.970+0000 7f682adff700 1 Processor -- start 2023-11-06T22:16:48.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.970+0000 7f682adff700 1 -- start start 2023-11-06T22:16:48.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.971+0000 7f682adff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6824153ed0 0x7f68240aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.971+0000 7f682adff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68241542b0 0x7f6824157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.971+0000 7f682adff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68240631c0 0x7f68240635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.971+0000 7f682adff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68240c02c0 con 0x7f68240631c0 2023-11-06T22:16:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.971+0000 7f682adff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68241581e0 con 0x7f6824153ed0 2023-11-06T22:16:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.971+0000 7f682adff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6824063ad0 con 0x7f68241542b0 2023-11-06T22:16:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f682939c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68240631c0 0x7f68240635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:48.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f682939c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68240631c0 0x7f68240635e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39782/0 (socket says 172.21.15.33:39782) 2023-11-06T22:16:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f6828b9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6824153ed0 0x7f68240aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f682939c700 1 -- 172.21.15.33:0/1124952262 learned_addr learned my addr 172.21.15.33:0/1124952262 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f6823fff700 1 --2- 172.21.15.33:0/1124952262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68241542b0 0x7f6824157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:48.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f682939c700 1 -- 172.21.15.33:0/1124952262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68241542b0 msgr2=0x7f6824157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:48.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f682939c700 1 --2- 172.21.15.33:0/1124952262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68241542b0 0x7f6824157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f682939c700 1 -- 172.21.15.33:0/1124952262 >> v2:172.21.15.69:3300/0 conn(0x7f6824153ed0 msgr2=0x7f68240aa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:48.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f682939c700 1 --2- 172.21.15.33:0/1124952262 >> v2:172.21.15.69:3300/0 conn(0x7f6824153ed0 0x7f68240aa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f682939c700 1 -- 172.21.15.33:0/1124952262 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6824158460 con 0x7f68240631c0 2023-11-06T22:16:48.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.972+0000 7f682939c700 1 --2- 172.21.15.33:0/1124952262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68240631c0 0x7f68240635e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f68240642b0 tx=0x7f681c009840).ready entity=mon.0 client_cookie=6d144fc3ecb212ca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:48.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f68237fe700 1 -- 172.21.15.33:0/1124952262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c00e070 con 0x7f68240631c0 2023-11-06T22:16:48.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f68237fe700 1 -- 172.21.15.33:0/1124952262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c00b970 con 0x7f68240631c0 2023-11-06T22:16:48.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f68237fe700 1 -- 172.21.15.33:0/1124952262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c00bcf0 con 0x7f68240631c0 2023-11-06T22:16:48.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f682adff700 1 -- 172.21.15.33:0/1124952262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68240631c0 msgr2=0x7f68240635e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f682adff700 1 --2- 172.21.15.33:0/1124952262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68240631c0 0x7f68240635e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f68240642b0 tx=0x7f681c009840).stop 2023-11-06T22:16:48.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f682adff700 1 -- 172.21.15.33:0/1124952262 shutdown_connections 2023-11-06T22:16:48.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f682adff700 1 --2- 172.21.15.33:0/1124952262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68240631c0 0x7f68240635e0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f682adff700 1 --2- 172.21.15.33:0/1124952262 >> v2:172.21.15.69:3300/0 conn(0x7f6824153ed0 0x7f68240aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f682adff700 1 --2- 172.21.15.33:0/1124952262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68241542b0 0x7f6824157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f682adff700 1 -- 172.21.15.33:0/1124952262 >> 172.21.15.33:0/1124952262 conn(0x7f682405ec80 msgr2=0x7f68240cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.973+0000 7f682adff700 1 -- 172.21.15.33:0/1124952262 shutdown_connections 2023-11-06T22:16:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.974+0000 7f682adff700 1 -- 172.21.15.33:0/1124952262 wait complete. 2023-11-06T22:16:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.975+0000 7f682adff700 1 Processor -- start 2023-11-06T22:16:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.975+0000 7f682adff700 1 -- start start 2023-11-06T22:16:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.976+0000 7f682adff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68240631c0 0x7f68240c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.976+0000 7f682adff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6824153ed0 0x7f6824152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.976+0000 7f682adff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68241542b0 0x7f68240c0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.976+0000 7f682adff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6824146db0 con 0x7f68241542b0 2023-11-06T22:16:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.976+0000 7f682adff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6824146130 con 0x7f68240631c0 2023-11-06T22:16:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.976+0000 7f682adff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6824146410 con 0x7f6824153ed0 2023-11-06T22:16:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.976+0000 7f682939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68241542b0 0x7f68240c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.976+0000 7f682939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68241542b0 0x7f68240c0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39792/0 (socket says 172.21.15.33:39792) 2023-11-06T22:16:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.976+0000 7f682939c700 1 -- 172.21.15.33:0/4018652277 learned_addr learned my addr 172.21.15.33:0/4018652277 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.976+0000 7f6823fff700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6824153ed0 0x7f6824152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f6828b9b700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68240631c0 0x7f68240c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f682939c700 1 -- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6824153ed0 msgr2=0x7f6824152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f682939c700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6824153ed0 0x7f6824152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f682939c700 1 -- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68240631c0 msgr2=0x7f68240c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f682939c700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68240631c0 0x7f68240c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f682939c700 1 -- 172.21.15.33:0/4018652277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c009ae0 con 0x7f68241542b0 2023-11-06T22:16:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f682939c700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68241542b0 0x7f68240c0750 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f68240c8950 tx=0x7f68240c0230).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c00bee0 con 0x7f68241542b0 2023-11-06T22:16:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c00bcf0 con 0x7f68241542b0 2023-11-06T22:16:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c01d6b0 con 0x7f68241542b0 2023-11-06T22:16:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.977+0000 7f682adff700 1 -- 172.21.15.33:0/4018652277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68241466f0 con 0x7f68241542b0 2023-11-06T22:16:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.978+0000 7f682adff700 1 -- 172.21.15.33:0/4018652277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6824145e20 con 0x7f68241542b0 2023-11-06T22:16:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.979+0000 7f682adff700 1 -- 172.21.15.33:0/4018652277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6824146ef0 con 0x7f68241542b0 2023-11-06T22:16:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.979+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 89) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f681c003040 con 0x7f68241542b0 2023-11-06T22:16:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.980+0000 7f6820ff9700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.981+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f681c07b660 con 0x7f68241542b0 2023-11-06T22:16:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.981+0000 7f6828b9b700 1 -- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 msgr2=0x7f6808061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.981+0000 7f6828b9b700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:16:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:48.985+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6824146ef0 con 0x7f68241542b0 2023-11-06T22:16:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:49.182+0000 7f6828b9b700 1 -- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 msgr2=0x7f6808061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:49.182+0000 7f6828b9b700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:16:49.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:49.196+0000 7f682adff700 1 -- 172.21.15.33:0/4018652277 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6824146ef0 con 0x7f680805f2c0 2023-11-06T22:16:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:49.583+0000 7f6828b9b700 1 -- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 msgr2=0x7f6808061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:49.583+0000 7f6828b9b700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T22:16:49.618 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:49.616+0000 7f88a4206680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:16:49.701 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:49.699+0000 7f88a4206680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:16:49.725 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:49.723+0000 7f0580f53680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:16:49.814 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:49.812+0000 7f0580f53680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:16:49.931 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:49.929+0000 7f88a4206680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:16:50.056 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:50.054+0000 7f0580f53680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:16:50.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:50.384+0000 7f6828b9b700 1 -- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 msgr2=0x7f6808061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:50.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:50.384+0000 7f6828b9b700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T22:16:50.441 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:50.439+0000 7f88a4206680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:16:50.565 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:50.563+0000 7f88a4206680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:16:50.596 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:50.594+0000 7f0580f53680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:16:50.730 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:50.727+0000 7f0580f53680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:16:50.752 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:50.750+0000 7f88a4206680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:16:50.847 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:50.845+0000 7f88a4206680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:16:50.923 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:50.921+0000 7f0580f53680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:16:50.933 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:50.931+0000 7f88a4206680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:16:51.024 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:51.021+0000 7f0580f53680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:16:51.115 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:51.112+0000 7f0580f53680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:16:51.243 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:51.240+0000 7f88a4206680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:16:51.445 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:51.443+0000 7f0580f53680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:16:51.466 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:51.464+0000 7f88a4206680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:16:51.677 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:51.674+0000 7f0580f53680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:16:51.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:51.986+0000 7f6828b9b700 1 -- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 msgr2=0x7f6808061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:16:51.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:51.986+0000 7f6828b9b700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T22:16:52.270 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:52.268+0000 7f88a4206680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:16:52.357 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:52.355+0000 7f88a4206680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:16:52.453 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:52.451+0000 7f88a4206680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:16:52.532 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:52.530+0000 7f0580f53680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:16:52.625 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:52.622+0000 7f0580f53680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:16:52.640 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:52.638+0000 7f88a4206680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:16:52.725 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:52.723+0000 7f0580f53680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:16:52.730 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:52.728+0000 7f88a4206680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:16:52.874 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:52.871+0000 7f88a4206680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:16:52.921 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:52.918+0000 7f0580f53680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:16:53.004 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.002+0000 7f88a4206680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:16:53.015 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:53.013+0000 7f0580f53680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:16:53.166 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:53.163+0000 7f0580f53680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:16:53.306 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:53.304+0000 7f0580f53680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:16:53.352 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.349+0000 7f88a4206680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:16:53.443 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.441+0000 7f88a4206680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:16:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:53.449+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 90) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f681c079240 con 0x7f68241542b0 2023-11-06T22:16:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:53.450+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 msgr2=0x7f6808061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:16:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:53.450+0000 7f6820ff9700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:16:53.525 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.523+0000 7f8819d07700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:53.525 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.523+0000 7f8819d07700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:53.525 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.523+0000 7f8819d07700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:53.526 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.523+0000 7f8819d07700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:53.526 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.523+0000 7f8819d07700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:53.526 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.523+0000 7f88154fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:53.527 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.523+0000 7f88154fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:53.527 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.523+0000 7f88154fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:53.527 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.523+0000 7f88154fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:53.527 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:16:53.523+0000 7f88154fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:16:53.675 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:53.672+0000 7f0580f53680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:16:53.770 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:16:53.768+0000 7f0580f53680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:16:54.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.456+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 91) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f681c0795c0 con 0x7f68241542b0 2023-11-06T22:16:54.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.456+0000 7f6820ff9700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:54.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.456+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6824146ef0 con 0x7f680805f2c0 2023-11-06T22:16:54.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.457+0000 7f6828b9b700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:54.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.459+0000 7f6828b9b700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f6818009640 tx=0x7f6818009060).ready entity=mgr.7985 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.471+0000 7f6820ff9700 1 -- 172.21.15.33:0/4018652277 <== mgr.7985 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6824146ef0 con 0x7f680805f2c0 2023-11-06T22:16:54.473 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path snap_test_dir1 2023-11-06T22:16:54.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.473+0000 7f682adff700 1 -- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 msgr2=0x7f6808061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:54.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.473+0000 7f682adff700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f6818009640 tx=0x7f6818009060).stop 2023-11-06T22:16:54.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.473+0000 7f682adff700 1 -- 172.21.15.33:0/4018652277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68241542b0 msgr2=0x7f68240c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:54.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.473+0000 7f682adff700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68241542b0 0x7f68240c0750 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7f68240c8950 tx=0x7f68240c0230).stop 2023-11-06T22:16:54.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.473+0000 7f682adff700 1 -- 172.21.15.33:0/4018652277 shutdown_connections 2023-11-06T22:16:54.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.473+0000 7f682adff700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680805f2c0 0x7f6808061700 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.473+0000 7f682adff700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68241542b0 0x7f68240c0750 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.473+0000 7f682adff700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68240631c0 0x7f68240c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.473+0000 7f682adff700 1 --2- 172.21.15.33:0/4018652277 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6824153ed0 0x7f6824152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.473+0000 7f682adff700 1 -- 172.21.15.33:0/4018652277 >> 172.21.15.33:0/4018652277 conn(0x7f682405ec80 msgr2=0x7f682414e140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.474+0000 7f682adff700 1 -- 172.21.15.33:0/4018652277 shutdown_connections 2023-11-06T22:16:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.474+0000 7f682adff700 1 -- 172.21.15.33:0/4018652277 wait complete. 2023-11-06T22:16:54.497 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-06T22:16:54.498 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-11-06T22:16:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.704+0000 7f1225dff700 1 Processor -- start 2023-11-06T22:16:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.704+0000 7f1225dff700 1 -- start start 2023-11-06T22:16:54.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.705+0000 7f1225dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1220064b40 0x7f12200c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:54.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.705+0000 7f1225dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12201556d0 con 0x7f12201544f0 2023-11-06T22:16:54.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.705+0000 7f1225dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1220064fd0 con 0x7f12200646c0 2023-11-06T22:16:54.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.705+0000 7f1225dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12201548d0 con 0x7f1220064b40 2023-11-06T22:16:54.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.705+0000 7f121f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f12201544f0 0x7f12200c0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51340/0 (socket says 172.21.15.33:51340) 2023-11-06T22:16:54.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.706+0000 7f121f7fe700 1 -- 172.21.15.33:0/1028762152 learned_addr learned my addr 172.21.15.33:0/1028762152 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:16:54.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.706+0000 7f121ffff700 1 --2- 172.21.15.33:0/1028762152 >> v2:172.21.15.33:3300/0 conn(0x7f1220064b40 0x7f12200c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:54.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.706+0000 7f121e7fc700 1 -- 172.21.15.33:0/1028762152 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191289023 0 0) 0x7f12201556d0 con 0x7f12201544f0 2023-11-06T22:16:54.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.706+0000 7f121e7fc700 1 -- 172.21.15.33:0/1028762152 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12080033f0 con 0x7f12201544f0 2023-11-06T22:16:54.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.706+0000 7f121ffff700 1 -- 172.21.15.33:0/1028762152 >> v1:172.21.15.69:6789/0 conn(0x7f12200646c0 legacy=0x7f12200c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:54.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.706+0000 7f121ffff700 1 -- 172.21.15.33:0/1028762152 >> v1:172.21.15.33:6789/0 conn(0x7f12201544f0 legacy=0x7f12200c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:54.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.706+0000 7f121ffff700 1 -- 172.21.15.33:0/1028762152 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1220154ac0 con 0x7f1220064b40 2023-11-06T22:16:54.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.706+0000 7f121ffff700 1 --2- 172.21.15.33:0/1028762152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1220064b40 0x7f12200c5ef0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f1218009e60 tx=0x7f1218009780).ready entity=mon.0 client_cookie=88266bda044e1237 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:54.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.706+0000 7f121e7fc700 1 -- 172.21.15.33:0/1028762152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121800e070 con 0x7f1220064b40 2023-11-06T22:16:54.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.707+0000 7f121e7fc700 1 -- 172.21.15.33:0/1028762152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121800bb60 con 0x7f1220064b40 2023-11-06T22:16:54.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.707+0000 7f121e7fc700 1 -- 172.21.15.33:0/1028762152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218012410 con 0x7f1220064b40 2023-11-06T22:16:54.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.707+0000 7f1225dff700 1 -- 172.21.15.33:0/1028762152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1220064b40 msgr2=0x7f12200c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:54.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.707+0000 7f1225dff700 1 --2- 172.21.15.33:0/1028762152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1220064b40 0x7f12200c5ef0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f1218009e60 tx=0x7f1218009780).stop 2023-11-06T22:16:54.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.707+0000 7f1225dff700 1 -- 172.21.15.33:0/1028762152 shutdown_connections 2023-11-06T22:16:54.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.707+0000 7f1225dff700 1 --2- 172.21.15.33:0/1028762152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1220064b40 0x7f12200c5ef0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.707+0000 7f1225dff700 1 -- 172.21.15.33:0/1028762152 >> 172.21.15.33:0/1028762152 conn(0x7f12200cb900 msgr2=0x7f12200c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:54.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.707+0000 7f1225dff700 1 -- 172.21.15.33:0/1028762152 shutdown_connections 2023-11-06T22:16:54.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.707+0000 7f1225dff700 1 -- 172.21.15.33:0/1028762152 wait complete. 2023-11-06T22:16:54.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.709+0000 7f1225dff700 1 Processor -- start 2023-11-06T22:16:54.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.709+0000 7f1225dff700 1 -- start start 2023-11-06T22:16:54.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f1225dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12200646c0 0x7f1220151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:54.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f1225dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1220064b40 0x7f12200af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:54.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f1225dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12201544f0 0x7f12200c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:54.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f1225dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12200adf00 con 0x7f12201544f0 2023-11-06T22:16:54.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f1225dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12200ae1e0 con 0x7f12200646c0 2023-11-06T22:16:54.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f1225dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12200ae4c0 con 0x7f1220064b40 2023-11-06T22:16:54.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f121ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12201544f0 0x7f12200c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:54.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f121f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12200646c0 0x7f1220151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:54.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f121ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12201544f0 0x7f12200c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39814/0 (socket says 172.21.15.33:39814) 2023-11-06T22:16:54.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f121f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12200646c0 0x7f1220151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:35190/0 (socket says 172.21.15.33:35190) 2023-11-06T22:16:54.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f121ffff700 1 -- 172.21.15.33:0/1601037621 learned_addr learned my addr 172.21.15.33:0/1601037621 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:54.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.710+0000 7f121effd700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1220064b40 0x7f12200af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:54.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f121ffff700 1 -- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1220064b40 msgr2=0x7f12200af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:54.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f121ffff700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1220064b40 0x7f12200af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f121ffff700 1 -- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12200646c0 msgr2=0x7f1220151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:54.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f121ffff700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12200646c0 0x7f1220151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f121ffff700 1 -- 172.21.15.33:0/1601037621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218009a20 con 0x7f12201544f0 2023-11-06T22:16:54.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f121ffff700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12201544f0 0x7f12200c58b0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f12200843a0 tx=0x7f1218009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:54.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f12077fe700 1 -- 172.21.15.33:0/1601037621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218012410 con 0x7f12201544f0 2023-11-06T22:16:54.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f12077fe700 1 -- 172.21.15.33:0/1601037621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121801d630 con 0x7f12201544f0 2023-11-06T22:16:54.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f12077fe700 1 -- 172.21.15.33:0/1601037621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121801d9f0 con 0x7f12201544f0 2023-11-06T22:16:54.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f1225dff700 1 -- 172.21.15.33:0/1601037621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12200ae7a0 con 0x7f12201544f0 2023-11-06T22:16:54.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.711+0000 7f1225dff700 1 -- 172.21.15.33:0/1601037621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12200b0890 con 0x7f12201544f0 2023-11-06T22:16:54.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.712+0000 7f1225dff700 1 -- 172.21.15.33:0/1601037621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12200aec40 con 0x7f12201544f0 2023-11-06T22:16:54.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.713+0000 7f12077fe700 1 -- 172.21.15.33:0/1601037621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 91) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f1218026410 con 0x7f12201544f0 2023-11-06T22:16:54.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.714+0000 7f12077fe700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f120c05f080 0x7f120c061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:54.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.714+0000 7f12077fe700 1 -- 172.21.15.33:0/1601037621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f121807bf80 con 0x7f12201544f0 2023-11-06T22:16:54.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.714+0000 7f121f7fe700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f120c05f080 0x7f120c061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:54.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.715+0000 7f121f7fe700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f120c05f080 0x7f120c061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f1210007fd0 tx=0x7f1210009040).ready entity=mgr.7985 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:54.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.718+0000 7f12077fe700 1 -- 172.21.15.33:0/1601037621 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f121804d200 con 0x7f12201544f0 2023-11-06T22:16:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.926+0000 7f1225dff700 1 -- 172.21.15.33:0/1601037621 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f12200aedf0 con 0x7f120c05f080 2023-11-06T22:16:54.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.933+0000 7f12077fe700 1 -- 172.21.15.33:0/1601037621 <== mgr.7985 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f12200aedf0 con 0x7f120c05f080 2023-11-06T22:16:54.936 INFO:teuthology.orchestra.run.smithi033.stdout:Retention added to path snap_test_dir1 2023-11-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 -- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f120c05f080 msgr2=0x7f120c061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f120c05f080 0x7f120c061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f1210007fd0 tx=0x7f1210009040).stop 2023-11-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 -- 172.21.15.33:0/1601037621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12201544f0 msgr2=0x7f12200c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12201544f0 0x7f12200c58b0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f12200843a0 tx=0x7f1218009780).stop 2023-11-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 -- 172.21.15.33:0/1601037621 shutdown_connections 2023-11-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f120c05f080 0x7f120c061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12201544f0 0x7f12200c58b0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12200646c0 0x7f1220151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 --2- 172.21.15.33:0/1601037621 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1220064b40 0x7f12200af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:54.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 -- 172.21.15.33:0/1601037621 >> 172.21.15.33:0/1601037621 conn(0x7f12200cb900 msgr2=0x7f12200aa240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:54.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 -- 172.21.15.33:0/1601037621 shutdown_connections 2023-11-06T22:16:54.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:54.935+0000 7f1225dff700 1 -- 172.21.15.33:0/1601037621 wait complete. 2023-11-06T22:16:54.955 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-11-06T22:16:54.956 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-11-06-22_17_00 in ~6s... 2023-11-06T22:16:54.956 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-11-06T22:16:54.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-11-06T22:16:55.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.153+0000 7f7c14fff700 1 Processor -- start 2023-11-06T22:16:55.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.153+0000 7f7c14fff700 1 -- start start 2023-11-06T22:16:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c14fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c10153ed0 0x7f7c10158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c14fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c100c8ee0 0x7f7c100ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c14fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7c101542b0 0x7f7c100c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c14fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c100c9f80 con 0x7f7c100c8ee0 2023-11-06T22:16:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c14fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c100c6000 con 0x7f7c101542b0 2023-11-06T22:16:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c14fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c100c9370 con 0x7f7c10153ed0 2023-11-06T22:16:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c0dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c100c8ee0 0x7f7c100ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:55.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c0dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c100c8ee0 0x7f7c100ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39830/0 (socket says 172.21.15.33:39830) 2023-11-06T22:16:55.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c0dd9b700 1 -- 172.21.15.33:0/531461175 learned_addr learned my addr 172.21.15.33:0/531461175 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:55.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c0e59c700 1 --2- 172.21.15.33:0/531461175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c10153ed0 0x7f7c10158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:55.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c0ed9d700 1 --2- 172.21.15.33:0/531461175 >> v2:172.21.15.69:3300/0 conn(0x7f7c101542b0 0x7f7c100c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:55.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c0dd9b700 1 -- 172.21.15.33:0/531461175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c10153ed0 msgr2=0x7f7c10158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:55.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c0dd9b700 1 --2- 172.21.15.33:0/531461175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c10153ed0 0x7f7c10158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c0dd9b700 1 -- 172.21.15.33:0/531461175 >> v2:172.21.15.69:3300/0 conn(0x7f7c101542b0 msgr2=0x7f7c100c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:16:55.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c0dd9b700 1 --2- 172.21.15.33:0/531461175 >> v2:172.21.15.69:3300/0 conn(0x7f7c101542b0 0x7f7c100c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.155+0000 7f7c0dd9b700 1 -- 172.21.15.33:0/531461175 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c100c9650 con 0x7f7c100c8ee0 2023-11-06T22:16:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.156+0000 7f7c0dd9b700 1 --2- 172.21.15.33:0/531461175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c100c8ee0 0x7f7c100ca410 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f7c100c8c60 tx=0x7f7c040090f0).ready entity=mon.0 client_cookie=50aaafc0a947a87c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.156+0000 7f7c0d59a700 1 -- 172.21.15.33:0/531461175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400b600 con 0x7f7c100c8ee0 2023-11-06T22:16:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.156+0000 7f7c0d59a700 1 -- 172.21.15.33:0/531461175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0400b760 con 0x7f7c100c8ee0 2023-11-06T22:16:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.156+0000 7f7c0d59a700 1 -- 172.21.15.33:0/531461175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400bae0 con 0x7f7c100c8ee0 2023-11-06T22:16:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.156+0000 7f7c14fff700 1 -- 172.21.15.33:0/531461175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c100c8ee0 msgr2=0x7f7c100ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.156+0000 7f7c14fff700 1 --2- 172.21.15.33:0/531461175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c100c8ee0 0x7f7c100ca410 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f7c100c8c60 tx=0x7f7c040090f0).stop 2023-11-06T22:16:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.157+0000 7f7c14fff700 1 -- 172.21.15.33:0/531461175 shutdown_connections 2023-11-06T22:16:55.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.157+0000 7f7c14fff700 1 --2- 172.21.15.33:0/531461175 >> v2:172.21.15.69:3300/0 conn(0x7f7c101542b0 0x7f7c100c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.157+0000 7f7c14fff700 1 --2- 172.21.15.33:0/531461175 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c10153ed0 0x7f7c10158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.157+0000 7f7c14fff700 1 --2- 172.21.15.33:0/531461175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c100c8ee0 0x7f7c100ca410 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.157+0000 7f7c14fff700 1 -- 172.21.15.33:0/531461175 >> 172.21.15.33:0/531461175 conn(0x7f7c1005e1b0 msgr2=0x7f7c1005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:55.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.157+0000 7f7c14fff700 1 -- 172.21.15.33:0/531461175 shutdown_connections 2023-11-06T22:16:55.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.157+0000 7f7c14fff700 1 -- 172.21.15.33:0/531461175 wait complete. 2023-11-06T22:16:55.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.158+0000 7f7c14fff700 1 Processor -- start 2023-11-06T22:16:55.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.158+0000 7f7c14fff700 1 -- start start 2023-11-06T22:16:55.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.159+0000 7f7c14fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c10153ed0 0x7f7c1014fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:55.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.159+0000 7f7c14fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c101542b0 0x7f7c10164ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:55.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.159+0000 7f7c14fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c100affe0 0x7f7c10156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:55.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.159+0000 7f7c14fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1014fff0 con 0x7f7c101542b0 2023-11-06T22:16:55.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.159+0000 7f7c14fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c100aef40 con 0x7f7c100affe0 2023-11-06T22:16:55.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.159+0000 7f7c14fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c100af220 con 0x7f7c10153ed0 2023-11-06T22:16:55.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c101542b0 0x7f7c10164ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c101542b0 0x7f7c10164ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39840/0 (socket says 172.21.15.33:39840) 2023-11-06T22:16:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0dd9b700 1 -- 172.21.15.33:0/1844291801 learned_addr learned my addr 172.21.15.33:0/1844291801 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:16:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0e59c700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c10153ed0 0x7f7c1014fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0ed9d700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c100affe0 0x7f7c10156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0dd9b700 1 -- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c10153ed0 msgr2=0x7f7c1014fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0dd9b700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c10153ed0 0x7f7c1014fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0dd9b700 1 -- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c100affe0 msgr2=0x7f7c10156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0dd9b700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c100affe0 0x7f7c10156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0dd9b700 1 -- 172.21.15.33:0/1844291801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c04009370 con 0x7f7c101542b0 2023-11-06T22:16:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.160+0000 7f7c0dd9b700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c101542b0 0x7f7c10164ca0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f7c04009130 tx=0x7f7c04017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.161+0000 7f7bfe7fc700 1 -- 172.21.15.33:0/1844291801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400b8d0 con 0x7f7c101542b0 2023-11-06T22:16:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.161+0000 7f7bfe7fc700 1 -- 172.21.15.33:0/1844291801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0400ba30 con 0x7f7c101542b0 2023-11-06T22:16:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.161+0000 7f7bfe7fc700 1 -- 172.21.15.33:0/1844291801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0401c670 con 0x7f7c101542b0 2023-11-06T22:16:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.161+0000 7f7c14fff700 1 -- 172.21.15.33:0/1844291801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c100af500 con 0x7f7c101542b0 2023-11-06T22:16:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.161+0000 7f7c14fff700 1 -- 172.21.15.33:0/1844291801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c100afac0 con 0x7f7c101542b0 2023-11-06T22:16:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.162+0000 7f7c14fff700 1 -- 172.21.15.33:0/1844291801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c1006c6c0 con 0x7f7c101542b0 2023-11-06T22:16:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.163+0000 7f7bfe7fc700 1 -- 172.21.15.33:0/1844291801 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 91) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f7c0401ca20 con 0x7f7c101542b0 2023-11-06T22:16:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.163+0000 7f7bfe7fc700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f080 0x7f7bf0061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:16:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.164+0000 7f7bfe7fc700 1 -- 172.21.15.33:0/1844291801 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f7c0407bfc0 con 0x7f7c101542b0 2023-11-06T22:16:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.164+0000 7f7c0e59c700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f080 0x7f7bf0061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:16:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.165+0000 7f7c0e59c700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f080 0x7f7bf0061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7c10157c10 tx=0x7f7c00008040).ready entity=mgr.7985 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:16:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.168+0000 7f7bfe7fc700 1 -- 172.21.15.33:0/1844291801 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7c0404d2b0 con 0x7f7c101542b0 2023-11-06T22:16:55.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.374+0000 7f7c14fff700 1 -- 172.21.15.33:0/1844291801 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c100c9930 con 0x7f7bf005f080 2023-11-06T22:16:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.378+0000 7f7bfe7fc700 1 -- 172.21.15.33:0/1844291801 <== mgr.7985 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f7c100c9930 con 0x7f7bf005f080 2023-11-06T22:16:55.380 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:16:55.380 INFO:teuthology.orchestra.run.smithi033.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-11-06T22:16:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.379+0000 7f7c14fff700 1 -- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f080 msgr2=0x7f7bf0061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.379+0000 7f7c14fff700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f080 0x7f7bf0061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7c10157c10 tx=0x7f7c00008040).stop 2023-11-06T22:16:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.379+0000 7f7c14fff700 1 -- 172.21.15.33:0/1844291801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c101542b0 msgr2=0x7f7c10164ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:16:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.379+0000 7f7c14fff700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c101542b0 0x7f7c10164ca0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f7c04009130 tx=0x7f7c04017000).stop 2023-11-06T22:16:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.379+0000 7f7c14fff700 1 -- 172.21.15.33:0/1844291801 shutdown_connections 2023-11-06T22:16:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.379+0000 7f7c14fff700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f080 0x7f7bf0061540 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.379+0000 7f7c14fff700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c10153ed0 0x7f7c1014fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.379+0000 7f7c14fff700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c100affe0 0x7f7c10156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.379+0000 7f7c14fff700 1 --2- 172.21.15.33:0/1844291801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c101542b0 0x7f7c10164ca0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:16:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.379+0000 7f7c14fff700 1 -- 172.21.15.33:0/1844291801 >> 172.21.15.33:0/1844291801 conn(0x7f7c1005e1b0 msgr2=0x7f7c100cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:16:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.380+0000 7f7c14fff700 1 -- 172.21.15.33:0/1844291801 shutdown_connections 2023-11-06T22:16:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:16:55.380+0000 7f7c14fff700 1 -- 172.21.15.33:0/1844291801 wait complete. 2023-11-06T22:16:55.400 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-11-06T22:16:55.400 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-11-06T22:16:55.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:55.401 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:16:56.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:56.451 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:16:57.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:57.509 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:16:58.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:58.565 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:16:59.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:16:59.621 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:00.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:00.678 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:00.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:17:00.729 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:17:00.941 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:17:00.938+0000 7f8808ca5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:17:00.941 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:17:00.938+0000 7f8808ca5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:17:00.941 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:17:00.938+0000 7f8808ca5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:17:00.941 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:17:00.938+0000 7f8808ca5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:17:00.942 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:17:00.938+0000 7f8808ca5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:17:01.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:01.735 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:01.789 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:01.793 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-11-06-22_17_00_UTC'] 2023-11-06T22:17:01.794 DEBUG:tasks.cephfs.test_snap_schedules:exec=1699309014.9562597, now = 1699309021.7941744, timo = 6 2023-11-06T22:17:02.795 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-11-06-22_17_00 in ~66s... 2023-11-06T22:17:02.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:02.796 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:02.849 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:03.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:03.852 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:03.905 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:04.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:04.908 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:04.962 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:05.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:05.965 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:06.018 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:07.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:07.020 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:08.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:08.077 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:08.131 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:09.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:09.134 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:09.188 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:10.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:10.191 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:10.245 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:11.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:11.247 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:11.301 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:12.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:12.304 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:12.357 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:13.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:13.360 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:13.413 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:14.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:14.416 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:14.470 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:15.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:15.472 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:15.525 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:16.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:16.529 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:16.582 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:17.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:17.585 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:17.639 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:18.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:18.641 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:18.698 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:19.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:19.700 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:19.754 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:20.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:20.757 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:20.810 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:21.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:21.813 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:21.867 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:22.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:22.869 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:22.923 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:23.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:23.925 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:23.979 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:24.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:24.981 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:25.034 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:26.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:26.037 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:26.091 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:27.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:27.094 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:27.148 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:28.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:28.150 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:28.203 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:29.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:29.206 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:29.261 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:30.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:30.263 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:30.318 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:30.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:17:30.883 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:17:31.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:31.321 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:31.374 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:32.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:32.376 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:32.430 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:33.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:33.433 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:33.487 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:34.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:34.489 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:34.543 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:35.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:35.546 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:35.599 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:36.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:36.602 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:36.655 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:37.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:37.658 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:37.712 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:38.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:38.714 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:38.768 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:39.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:39.771 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:39.824 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:40.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:40.827 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:40.880 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:41.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:41.883 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:41.937 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:42.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:42.940 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:42.993 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:43.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:43.996 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:44.051 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:45.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:45.053 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:45.107 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:46.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:46.110 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:46.163 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:47.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:47.166 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:47.220 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:48.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:48.223 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:48.276 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:49.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:49.279 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:49.333 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:50.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:50.336 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:50.390 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:51.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:51.393 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:51.446 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:52.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:52.449 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:52.503 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:53.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:53.505 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:53.559 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:54.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:54.562 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:54.615 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:55.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:55.618 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:55.672 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:56.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:56.674 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:56.728 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:57.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:57.730 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:57.783 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:58.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:58.786 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:58.839 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:17:59.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:17:59.842 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:17:59.896 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:18:00.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:18:00.899 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:18:00.952 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_17_00_UTC 2023-11-06T22:18:00.994 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:18:01.000 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:18:01.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:18:01.955 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:18:02.008 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_18_00_UTC 2023-11-06T22:18:02.009 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-11-06-22_17_00_UTC'] 2023-11-06T22:18:02.010 DEBUG:tasks.cephfs.test_snap_schedules:exec=1699309014.9562597, now = 1699309082.0102904, timo = 66 2023-11-06T22:18:03.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-11-06T22:18:03.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.220+0000 7f99481ff700 1 Processor -- start 2023-11-06T22:18:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.220+0000 7f99481ff700 1 -- start start 2023-11-06T22:18:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.221+0000 7f99481ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f99401514c0 0x7f99400bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.221+0000 7f99481ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99400bbe70 0x7f994015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.221+0000 7f99481ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9940154c60 con 0x7f99401510e0 2023-11-06T22:18:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.221+0000 7f99481ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99400bc2e0 con 0x7f99401514c0 2023-11-06T22:18:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.221+0000 7f99481ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9940068f50 con 0x7f99400bbe70 2023-11-06T22:18:03.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.221+0000 7f9945f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f99401510e0 0x7f99400a5d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40164/0 (socket says 172.21.15.33:40164) 2023-11-06T22:18:03.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.221+0000 7f9945f9b700 1 -- 172.21.15.33:0/2226145300 learned_addr learned my addr 172.21.15.33:0/2226145300 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:03.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.222+0000 7f994679c700 1 --2- 172.21.15.33:0/2226145300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99400bbe70 0x7f994015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.222+0000 7f994579a700 1 --2- 172.21.15.33:0/2226145300 >> v2:172.21.15.69:3300/0 conn(0x7f99401514c0 0x7f99400bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.222+0000 7f9944f99700 1 -- 172.21.15.33:0/2226145300 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934618366 0 0) 0x7f9940154c60 con 0x7f99401510e0 2023-11-06T22:18:03.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.222+0000 7f9944f99700 1 -- 172.21.15.33:0/2226145300 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f992c0033f0 con 0x7f99401510e0 2023-11-06T22:18:03.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.222+0000 7f994579a700 1 -- 172.21.15.33:0/2226145300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99400bbe70 msgr2=0x7f994015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:03.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.222+0000 7f994579a700 1 --2- 172.21.15.33:0/2226145300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99400bbe70 0x7f994015ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.222+0000 7f994579a700 1 -- 172.21.15.33:0/2226145300 >> v1:172.21.15.33:6789/0 conn(0x7f99401510e0 legacy=0x7f99400a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.222+0000 7f994579a700 1 -- 172.21.15.33:0/2226145300 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9940151dc0 con 0x7f99401514c0 2023-11-06T22:18:03.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.222+0000 7f994579a700 1 --2- 172.21.15.33:0/2226145300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99401514c0 0x7f99400bb980 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f9930009760 tx=0x7f9930009060).ready entity=mon.1 client_cookie=7fe46516d7c51e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:03.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f9944f99700 1 -- 172.21.15.33:0/2226145300 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993000d4a0 con 0x7f99401514c0 2023-11-06T22:18:03.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f9944f99700 1 -- 172.21.15.33:0/2226145300 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993000d600 con 0x7f99401514c0 2023-11-06T22:18:03.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f9944f99700 1 -- 172.21.15.33:0/2226145300 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993000d980 con 0x7f99401514c0 2023-11-06T22:18:03.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f99481ff700 1 -- 172.21.15.33:0/2226145300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99401514c0 msgr2=0x7f99400bb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f99481ff700 1 --2- 172.21.15.33:0/2226145300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99401514c0 0x7f99400bb980 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f9930009760 tx=0x7f9930009060).stop 2023-11-06T22:18:03.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f99481ff700 1 -- 172.21.15.33:0/2226145300 shutdown_connections 2023-11-06T22:18:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f99481ff700 1 --2- 172.21.15.33:0/2226145300 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99400bbe70 0x7f994015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f99481ff700 1 --2- 172.21.15.33:0/2226145300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99401514c0 0x7f99400bb980 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f99481ff700 1 -- 172.21.15.33:0/2226145300 >> 172.21.15.33:0/2226145300 conn(0x7f9940062710 msgr2=0x7f9940062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f99481ff700 1 -- 172.21.15.33:0/2226145300 shutdown_connections 2023-11-06T22:18:03.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.223+0000 7f99481ff700 1 -- 172.21.15.33:0/2226145300 wait complete. 2023-11-06T22:18:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.225+0000 7f99481ff700 1 Processor -- start 2023-11-06T22:18:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.225+0000 7f99481ff700 1 -- start start 2023-11-06T22:18:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f99481ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99400bbe70 0x7f99400a8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f99481ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99401510e0 0x7f9940153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f99481ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99401514c0 0x7f9940165000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f99481ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994014f130 con 0x7f99401510e0 2023-11-06T22:18:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f99481ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400be9e0 con 0x7f99401514c0 2023-11-06T22:18:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f99481ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99400bde10 con 0x7f99400bbe70 2023-11-06T22:18:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f994579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99401510e0 0x7f9940153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f994579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99401510e0 0x7f9940153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45634/0 (socket says 172.21.15.33:45634) 2023-11-06T22:18:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f994579a700 1 -- 172.21.15.33:0/2538525069 learned_addr learned my addr 172.21.15.33:0/2538525069 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f994679c700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99401514c0 0x7f9940165000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.226+0000 7f9945f9b700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99400bbe70 0x7f99400a8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f994579a700 1 -- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99400bbe70 msgr2=0x7f99400a8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f994579a700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99400bbe70 0x7f99400a8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f994579a700 1 -- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99401514c0 msgr2=0x7f9940165000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f994579a700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99401514c0 0x7f9940165000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f994579a700 1 -- 172.21.15.33:0/2538525069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300092a0 con 0x7f99401510e0 2023-11-06T22:18:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f994679c700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99401514c0 0x7f9940165000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f994579a700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99401510e0 0x7f9940153ae0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f9930009d50 tx=0x7f9930016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f9935ffb700 1 -- 172.21.15.33:0/2538525069 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993000d770 con 0x7f99401510e0 2023-11-06T22:18:03.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f9935ffb700 1 -- 172.21.15.33:0/2538525069 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993000d8d0 con 0x7f99401510e0 2023-11-06T22:18:03.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f99481ff700 1 -- 172.21.15.33:0/2538525069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99400be0b0 con 0x7f99401510e0 2023-11-06T22:18:03.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f9935ffb700 1 -- 172.21.15.33:0/2538525069 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993001d660 con 0x7f99401510e0 2023-11-06T22:18:03.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.227+0000 7f99481ff700 1 -- 172.21.15.33:0/2538525069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99400be670 con 0x7f99401510e0 2023-11-06T22:18:03.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.228+0000 7f99481ff700 1 -- 172.21.15.33:0/2538525069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99400beb50 con 0x7f99401510e0 2023-11-06T22:18:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.229+0000 7f9935ffb700 1 -- 172.21.15.33:0/2538525069 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 93) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f993001da50 con 0x7f99401510e0 2023-11-06T22:18:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.230+0000 7f9935ffb700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f992405f410 0x7f99240618d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.230+0000 7f9935ffb700 1 -- 172.21.15.33:0/2538525069 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6510+0+0 (secure 0 0 0) 0x7f993007bf50 con 0x7f99401510e0 2023-11-06T22:18:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.230+0000 7f9945f9b700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f992405f410 0x7f99240618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.231+0000 7f9945f9b700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f992405f410 0x7f99240618d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f993c005750 tx=0x7f993c009040).ready entity=mgr.7985 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.235+0000 7f9935ffb700 1 -- 172.21.15.33:0/2538525069 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f993004cda0 con 0x7f99401510e0 2023-11-06T22:18:03.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.444+0000 7f99481ff700 1 -- 172.21.15.33:0/2538525069 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f99401520a0 con 0x7f992405f410 2023-11-06T22:18:03.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.449+0000 7f9935ffb700 1 -- 172.21.15.33:0/2538525069 <== mgr.7985 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f99401520a0 con 0x7f992405f410 2023-11-06T22:18:03.452 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path snap_test_dir1 2023-11-06T22:18:03.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.451+0000 7f99481ff700 1 -- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f992405f410 msgr2=0x7f99240618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.451+0000 7f99481ff700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f992405f410 0x7f99240618d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f993c005750 tx=0x7f993c009040).stop 2023-11-06T22:18:03.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.451+0000 7f99481ff700 1 -- 172.21.15.33:0/2538525069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99401510e0 msgr2=0x7f9940153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.451+0000 7f99481ff700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99401510e0 0x7f9940153ae0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f9930009d50 tx=0x7f9930016000).stop 2023-11-06T22:18:03.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.451+0000 7f99481ff700 1 -- 172.21.15.33:0/2538525069 shutdown_connections 2023-11-06T22:18:03.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.451+0000 7f99481ff700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f992405f410 0x7f99240618d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.451+0000 7f99481ff700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f99400bbe70 0x7f99400a8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.451+0000 7f99481ff700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99401514c0 0x7f9940165000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.451+0000 7f99481ff700 1 --2- 172.21.15.33:0/2538525069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99401510e0 0x7f9940153ae0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.451+0000 7f99481ff700 1 -- 172.21.15.33:0/2538525069 >> 172.21.15.33:0/2538525069 conn(0x7f9940062710 msgr2=0x7f9940152990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:03.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.452+0000 7f99481ff700 1 -- 172.21.15.33:0/2538525069 shutdown_connections 2023-11-06T22:18:03.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.452+0000 7f99481ff700 1 -- 172.21.15.33:0/2538525069 wait complete. 2023-11-06T22:18:03.473 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-11-06T22:18:03.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:18:03.474 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:18:03.522 INFO:teuthology.orchestra.run.smithi033.stdout:scheduled-2023-11-06-22_18_00_UTC 2023-11-06T22:18:03.525 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-06-22_18_00_UTC 2023-11-06T22:18:03.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:18:03.526 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-06-22_18_00_UTC) 2023-11-06T22:18:03.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:18:03.580 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-06T22:18:03.632 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T22:18:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.827+0000 7f2a083ff700 1 Processor -- start 2023-11-06T22:18:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.827+0000 7f2a083ff700 1 -- start start 2023-11-06T22:18:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.829+0000 7f2a083ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a00157fa0 0x7f2a000630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.829+0000 7f2a083ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a00152470 con 0x7f2a000c72c0 2023-11-06T22:18:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.829+0000 7f2a083ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a00154990 con 0x7f2a00158420 2023-11-06T22:18:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.829+0000 7f2a083ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000c76a0 con 0x7f2a00157fa0 2023-11-06T22:18:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.829+0000 7f2a0619b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2a000c72c0 0x7f2a001553e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40170/0 (socket says 172.21.15.33:40170) 2023-11-06T22:18:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.829+0000 7f2a0619b700 1 -- 172.21.15.33:0/3014863351 learned_addr learned my addr 172.21.15.33:0/3014863351 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.829+0000 7f2a0599a700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a00157fa0 0x7f2a000630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.829+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252923510 0 0) 0x7f2a00152470 con 0x7f2a000c72c0 2023-11-06T22:18:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29ec0033f0 con 0x7f2a000c72c0 2023-11-06T22:18:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133947996 0 0) 0x7f2a00154990 con 0x7f2a00158420 2023-11-06T22:18:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a00152470 con 0x7f2a00158420 2023-11-06T22:18:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1845752847 0 0) 0x7f29ec0033f0 con 0x7f2a000c72c0 2023-11-06T22:18:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a00157fa0 msgr2=0x7f2a000630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a00157fa0 0x7f2a000630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 >> v1:172.21.15.69:6789/0 conn(0x7f2a00158420 legacy=0x7f2a000c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a00158910 con 0x7f2a000c72c0 2023-11-06T22:18:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f29fc001470 con 0x7f2a000c72c0 2023-11-06T22:18:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 >> v1:172.21.15.33:6789/0 conn(0x7f2a000c72c0 legacy=0x7f2a001553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 0x7f29ec003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a00157fa0 0x7f29ec004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29ec004b80 0x7f29ec005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000c76a0 con 0x7f29ec003aa0 2023-11-06T22:18:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec00b990 con 0x7f29ec004b80 2023-11-06T22:18:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec00bb00 con 0x7f2a00157fa0 2023-11-06T22:18:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a0619b700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 0x7f29ec003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a0599a700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a00157fa0 0x7f29ec004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.830+0000 7f2a0699c700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29ec004b80 0x7f29ec005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.831+0000 7f2a0619b700 1 -- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a00157fa0 msgr2=0x7f29ec004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.831+0000 7f2a0619b700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a00157fa0 0x7f29ec004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.831+0000 7f2a0619b700 1 -- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29ec004b80 msgr2=0x7f29ec005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.831+0000 7f2a0619b700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29ec004b80 0x7f29ec005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.831+0000 7f2a0619b700 1 -- 172.21.15.33:0/3014863351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec0033f0 con 0x7f29ec003aa0 2023-11-06T22:18:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.831+0000 7f2a0699c700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29ec004b80 0x7f29ec005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.831+0000 7f2a0619b700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 0x7f29ec003ea0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7f29fc007f80 tx=0x7f29fc001d40).ready entity=mon.0 client_cookie=677c5443091056c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.831+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc00c9e0 con 0x7f29ec003aa0 2023-11-06T22:18:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.831+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc00cb40 con 0x7f29ec003aa0 2023-11-06T22:18:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.831+0000 7f2a05199700 1 -- 172.21.15.33:0/3014863351 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0070d0 con 0x7f29ec003aa0 2023-11-06T22:18:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.832+0000 7f2a083ff700 1 -- 172.21.15.33:0/3014863351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 msgr2=0x7f29ec003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.832+0000 7f2a083ff700 1 --2- 172.21.15.33:0/3014863351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 0x7f29ec003ea0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7f29fc007f80 tx=0x7f29fc001d40).stop 2023-11-06T22:18:03.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.832+0000 7f2a0699c700 1 -- 172.21.15.33:0/3014863351 reap_dead start 2023-11-06T22:18:03.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.832+0000 7f2a083ff700 1 -- 172.21.15.33:0/3014863351 shutdown_connections 2023-11-06T22:18:03.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.832+0000 7f2a083ff700 1 -- 172.21.15.33:0/3014863351 >> 172.21.15.33:0/3014863351 conn(0x7f2a0005ec20 msgr2=0x7f2a000cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:03.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.832+0000 7f2a083ff700 1 -- 172.21.15.33:0/3014863351 shutdown_connections 2023-11-06T22:18:03.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.832+0000 7f2a083ff700 1 -- 172.21.15.33:0/3014863351 wait complete. 2023-11-06T22:18:03.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.833+0000 7f2a083ff700 1 Processor -- start 2023-11-06T22:18:03.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.833+0000 7f2a083ff700 1 -- start start 2023-11-06T22:18:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.834+0000 7f2a083ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 0x7f2a000c2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.834+0000 7f2a083ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a000c4cb0 0x7f2a000c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.834+0000 7f2a083ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a000aee30 0x7f2a000af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.834+0000 7f2a083ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000ae9a0 con 0x7f29ec003aa0 2023-11-06T22:18:03.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.834+0000 7f2a083ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0014f580 con 0x7f2a000aee30 2023-11-06T22:18:03.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.834+0000 7f2a083ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000add90 con 0x7f2a000c4cb0 2023-11-06T22:18:03.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f2a0619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 0x7f2a000c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f2a0619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 0x7f2a000c2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45650/0 (socket says 172.21.15.33:45650) 2023-11-06T22:18:03.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f2a0619b700 1 -- 172.21.15.33:0/3043231935 learned_addr learned my addr 172.21.15.33:0/3043231935 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:03.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f2a0619b700 1 -- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a000c4cb0 msgr2=0x7f2a000c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f2a0619b700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a000c4cb0 0x7f2a000c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:03.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f2a0619b700 1 -- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a000aee30 msgr2=0x7f2a000af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:03.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f2a0619b700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a000aee30 0x7f2a000af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:03.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f2a0619b700 1 -- 172.21.15.33:0/3043231935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a000ae070 con 0x7f29ec003aa0 2023-11-06T22:18:03.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f2a0619b700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 0x7f2a000c2080 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f29ec005620 tx=0x7f29fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:03.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f29f67fc700 1 -- 172.21.15.33:0/3043231935 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0070d0 con 0x7f29ec003aa0 2023-11-06T22:18:03.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f29f67fc700 1 -- 172.21.15.33:0/3043231935 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc00cec0 con 0x7f29ec003aa0 2023-11-06T22:18:03.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f29f67fc700 1 -- 172.21.15.33:0/3043231935 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc001370 con 0x7f29ec003aa0 2023-11-06T22:18:03.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.835+0000 7f2a083ff700 1 -- 172.21.15.33:0/3043231935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a000ae9a0 con 0x7f29ec003aa0 2023-11-06T22:18:03.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.836+0000 7f2a083ff700 1 -- 172.21.15.33:0/3043231935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a000ae630 con 0x7f29ec003aa0 2023-11-06T22:18:03.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.837+0000 7f2a083ff700 1 -- 172.21.15.33:0/3043231935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a00158bf0 con 0x7f29ec003aa0 2023-11-06T22:18:03.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.837+0000 7f29f67fc700 1 -- 172.21.15.33:0/3043231935 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 93) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f29fc020480 con 0x7f29ec003aa0 2023-11-06T22:18:03.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.838+0000 7f29f67fc700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f29e405f3e0 0x7f29e40618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:03.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.838+0000 7f29f67fc700 1 -- 172.21.15.33:0/3043231935 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6510+0+0 (secure 0 0 0) 0x7f29fc07b6e0 con 0x7f29ec003aa0 2023-11-06T22:18:03.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.838+0000 7f2a0599a700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f29e405f3e0 0x7f29e40618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:03.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.839+0000 7f2a0599a700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f29e405f3e0 0x7f29e40618a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f2a00157040 tx=0x7f29f0009060).ready entity=mgr.7985 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:03.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:03.843+0000 7f29f67fc700 1 -- 172.21.15.33:0/3043231935 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f29fc04c410 con 0x7f29ec003aa0 2023-11-06T22:18:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.069+0000 7f2a083ff700 1 -- 172.21.15.33:0/3043231935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f2a00062b80 con 0x7f29ec003aa0 2023-11-06T22:18:04.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.276+0000 7f29f67fc700 1 -- 172.21.15.33:0/3043231935 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f29fc078f70 con 0x7f29ec003aa0 2023-11-06T22:18:04.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.279+0000 7f29f67fc700 1 -- 172.21.15.33:0/3043231935 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v94) v1 ==== 93+0+0 (secure 0 0 0) 0x7f29fc041450 con 0x7f29ec003aa0 2023-11-06T22:18:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.281+0000 7f2a083ff700 1 -- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f29e405f3e0 msgr2=0x7f29e40618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.281+0000 7f2a083ff700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f29e405f3e0 0x7f29e40618a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f2a00157040 tx=0x7f29f0009060).stop 2023-11-06T22:18:04.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.281+0000 7f2a083ff700 1 -- 172.21.15.33:0/3043231935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 msgr2=0x7f2a000c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:04.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.281+0000 7f2a083ff700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 0x7f2a000c2080 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f29ec005620 tx=0x7f29fc015000).stop 2023-11-06T22:18:04.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.281+0000 7f2a083ff700 1 -- 172.21.15.33:0/3043231935 shutdown_connections 2023-11-06T22:18:04.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.281+0000 7f2a083ff700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f29e405f3e0 0x7f29e40618a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:04.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.281+0000 7f2a083ff700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a000aee30 0x7f2a000af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:04.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.281+0000 7f2a083ff700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29ec003aa0 0x7f2a000c2080 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:04.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.281+0000 7f2a083ff700 1 --2- 172.21.15.33:0/3043231935 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a000c4cb0 0x7f2a000c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:04.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.281+0000 7f2a083ff700 1 -- 172.21.15.33:0/3043231935 >> 172.21.15.33:0/3043231935 conn(0x7f2a0005ec20 msgr2=0x7f2a0014f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:04.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.282+0000 7f2a083ff700 1 -- 172.21.15.33:0/3043231935 shutdown_connections 2023-11-06T22:18:04.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.282+0000 7f2a083ff700 1 -- 172.21.15.33:0/3043231935 wait complete. 2023-11-06T22:18:04.304 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T22:18:04.304 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:18:04.404 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:18:04.505 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:04.502+0000 7f065526c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:18:04.506 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T22:18:04.507 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T22:18:04.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:18:04.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T22:18:04.513 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:04.511+0000 7f48a4a58680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:18:04.591 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T22:18:04.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:04.592 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T22:18:04.593 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:04.590+0000 7f065526c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:18:04.596 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:04.594+0000 7f48a4a58680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:18:04.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.841+0000 7f75d11ff700 1 Processor -- start 2023-11-06T22:18:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.841+0000 7f75d11ff700 1 -- start start 2023-11-06T22:18:04.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.842+0000 7f75d11ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc0bb990 0x7f75cc152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:04.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.842+0000 7f75d11ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f75cc0c33f0 0x7f75cc0c3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.842+0000 7f75d11ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc1534e0 con 0x7f75cc14c640 2023-11-06T22:18:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.843+0000 7f75d11ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75cc0bd8f0 con 0x7f75cc0c33f0 2023-11-06T22:18:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.843+0000 7f75d11ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0c3d70 con 0x7f75cc0bb990 2023-11-06T22:18:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.843+0000 7f75cad9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f75cc14c640 0x7f75cc065160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40196/0 (socket says 172.21.15.33:40196) 2023-11-06T22:18:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.843+0000 7f75cad9d700 1 -- 172.21.15.33:0/733469440 learned_addr learned my addr 172.21.15.33:0/733469440 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:04.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.843+0000 7f75cb59e700 1 --2- 172.21.15.33:0/733469440 >> v2:172.21.15.69:3300/0 conn(0x7f75cc0c33f0 0x7f75cc0c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:04.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.843+0000 7f75ca59c700 1 --2- 172.21.15.33:0/733469440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc0bb990 0x7f75cc152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.843+0000 7f75c9d9b700 1 -- 172.21.15.33:0/733469440 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594664928 0 0) 0x7f75cc1534e0 con 0x7f75cc14c640 2023-11-06T22:18:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.843+0000 7f75c9d9b700 1 -- 172.21.15.33:0/733469440 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b40033f0 con 0x7f75cc14c640 2023-11-06T22:18:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.843+0000 7f75cb59e700 1 -- 172.21.15.33:0/733469440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc0bb990 msgr2=0x7f75cc152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.844+0000 7f75cb59e700 1 --2- 172.21.15.33:0/733469440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc0bb990 0x7f75cc152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.844+0000 7f75cb59e700 1 -- 172.21.15.33:0/733469440 >> v1:172.21.15.33:6789/0 conn(0x7f75cc14c640 legacy=0x7f75cc065160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.844+0000 7f75c9d9b700 1 -- 172.21.15.33:0/733469440 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1684639367 0 0) 0x7f75b40033f0 con 0x7f75cc14c640 2023-11-06T22:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.844+0000 7f75cb59e700 1 -- 172.21.15.33:0/733469440 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc0643a0 con 0x7f75cc0c33f0 2023-11-06T22:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.844+0000 7f75ca59c700 1 --2- 172.21.15.33:0/733469440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc0bb990 0x7f75cc152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.844+0000 7f75cb59e700 1 --2- 172.21.15.33:0/733469440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75cc0c33f0 0x7f75cc0c3880 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f75c4009e60 tx=0x7f75c4009780).ready entity=mon.1 client_cookie=1ae13c7e071053a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.844+0000 7f75c9d9b700 1 -- 172.21.15.33:0/733469440 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400e070 con 0x7f75cc0c33f0 2023-11-06T22:18:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.844+0000 7f75c9d9b700 1 -- 172.21.15.33:0/733469440 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c400bab0 con 0x7f75cc0c33f0 2023-11-06T22:18:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.844+0000 7f75c9d9b700 1 -- 172.21.15.33:0/733469440 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400be30 con 0x7f75cc0c33f0 2023-11-06T22:18:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.845+0000 7f75d11ff700 1 -- 172.21.15.33:0/733469440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75cc0c33f0 msgr2=0x7f75cc0c3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.845+0000 7f75d11ff700 1 --2- 172.21.15.33:0/733469440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75cc0c33f0 0x7f75cc0c3880 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f75c4009e60 tx=0x7f75c4009780).stop 2023-11-06T22:18:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.845+0000 7f75d11ff700 1 -- 172.21.15.33:0/733469440 shutdown_connections 2023-11-06T22:18:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.845+0000 7f75d11ff700 1 --2- 172.21.15.33:0/733469440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75cc0c33f0 0x7f75cc0c3880 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.845+0000 7f75d11ff700 1 --2- 172.21.15.33:0/733469440 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc0bb990 0x7f75cc152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.845+0000 7f75d11ff700 1 -- 172.21.15.33:0/733469440 >> 172.21.15.33:0/733469440 conn(0x7f75cc062710 msgr2=0x7f75cc062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.845+0000 7f75d11ff700 1 -- 172.21.15.33:0/733469440 shutdown_connections 2023-11-06T22:18:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.845+0000 7f75d11ff700 1 -- 172.21.15.33:0/733469440 wait complete. 2023-11-06T22:18:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.846+0000 7f75d11ff700 1 Processor -- start 2023-11-06T22:18:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.846+0000 7f75d11ff700 1 -- start start 2023-11-06T22:18:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.847+0000 7f75d11ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75cc0bb990 0x7f75cc148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.847+0000 7f75d11ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75cc0c33f0 0x7f75cc0abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.847+0000 7f75d11ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc14c640 0x7f75cc0ac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.847+0000 7f75d11ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc148b40 con 0x7f75cc0c33f0 2023-11-06T22:18:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.847+0000 7f75d11ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0ac4f0 con 0x7f75cc0bb990 2023-11-06T22:18:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.847+0000 7f75d11ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0c0c40 con 0x7f75cc14c640 2023-11-06T22:18:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75ca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75cc0c33f0 0x7f75cc0abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75ca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75cc0c33f0 0x7f75cc0abb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45674/0 (socket says 172.21.15.33:45674) 2023-11-06T22:18:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75ca59c700 1 -- 172.21.15.33:0/1617879781 learned_addr learned my addr 172.21.15.33:0/1617879781 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75cad9d700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75cc0bb990 0x7f75cc148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75cb59e700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc14c640 0x7f75cc0ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75ca59c700 1 -- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc14c640 msgr2=0x7f75cc0ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75ca59c700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc14c640 0x7f75cc0ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75ca59c700 1 -- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75cc0bb990 msgr2=0x7f75cc148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75ca59c700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75cc0bb990 0x7f75cc148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75ca59c700 1 -- 172.21.15.33:0/1617879781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c00092a0 con 0x7f75cc0c33f0 2023-11-06T22:18:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.848+0000 7f75ca59c700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75cc0c33f0 0x7f75cc0abb10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f75c000ea20 tx=0x7f75c0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.849+0000 7f75b2ffd700 1 -- 172.21.15.33:0/1617879781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c000ee60 con 0x7f75cc0c33f0 2023-11-06T22:18:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.849+0000 7f75b2ffd700 1 -- 172.21.15.33:0/1617879781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c000c7f0 con 0x7f75cc0c33f0 2023-11-06T22:18:04.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.849+0000 7f75b2ffd700 1 -- 172.21.15.33:0/1617879781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c000cb70 con 0x7f75cc0c33f0 2023-11-06T22:18:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.849+0000 7f75d11ff700 1 -- 172.21.15.33:0/1617879781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c4009a20 con 0x7f75cc0c33f0 2023-11-06T22:18:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.849+0000 7f75d11ff700 1 -- 172.21.15.33:0/1617879781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc0c63b0 con 0x7f75cc0c33f0 2023-11-06T22:18:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.850+0000 7f75d11ff700 1 -- 172.21.15.33:0/1617879781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc0c6540 con 0x7f75cc0c33f0 2023-11-06T22:18:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.852+0000 7f75b2ffd700 1 -- 172.21.15.33:0/1617879781 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f75c001b400 con 0x7f75cc0c33f0 2023-11-06T22:18:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.853+0000 7f75b2ffd700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f75b805f390 0x7f75b8061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.853+0000 7f75b2ffd700 1 -- 172.21.15.33:0/1617879781 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6598+0+0 (secure 0 0 0) 0x7f75c007acd0 con 0x7f75cc0c33f0 2023-11-06T22:18:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.853+0000 7f75cad9d700 1 -- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f75b805f390 msgr2=0x7f75b8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:04.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.853+0000 7f75cad9d700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f75b805f390 0x7f75b8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:04.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:04.858+0000 7f75b2ffd700 1 -- 172.21.15.33:0/1617879781 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f75c004ba50 con 0x7f75cc0c33f0 2023-11-06T22:18:04.926 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:04.924+0000 7f48a4a58680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:18:04.951 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:04.949+0000 7f065526c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:18:05.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.054+0000 7f75cad9d700 1 -- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f75b805f390 msgr2=0x7f75b8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.054+0000 7f75cad9d700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f75b805f390 0x7f75b8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:18:05.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.083+0000 7f75d11ff700 1 -- 172.21.15.33:0/1617879781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75cc064680 con 0x7f75cc0c33f0 2023-11-06T22:18:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.083+0000 7f75b2ffd700 1 -- 172.21.15.33:0/1617879781 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f75c004f070 con 0x7f75cc0c33f0 2023-11-06T22:18:05.086 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:05.087 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3815920714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3815920714},{"type":"v1","addr":"172.21.15.69:6836","nonce":3815920714}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9052,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3614716748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3614716748},{"type":"v1","addr":"172.21.15.69:6837","nonce":3614716748}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1513187241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1513187241},{"type":"v1","addr":"172.21.15.33:6834","nonce":1513187241}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:16:39.145216+0000","modified":"2023-11-06T22:16:42.180077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3197264665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3197264665},{"type":"v1","addr":"172.21.15.33:6835","nonce":3197264665}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-06T22:18:05.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.085+0000 7f75d11ff700 1 -- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f75b805f390 msgr2=0x7f75b8061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:05.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.085+0000 7f75d11ff700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f75b805f390 0x7f75b8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:05.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.086+0000 7f75d11ff700 1 -- 172.21.15.33:0/1617879781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75cc0c33f0 msgr2=0x7f75cc0abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.086+0000 7f75d11ff700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75cc0c33f0 0x7f75cc0abb10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f75c000ea20 tx=0x7f75c0009060).stop 2023-11-06T22:18:05.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.086+0000 7f75d11ff700 1 -- 172.21.15.33:0/1617879781 shutdown_connections 2023-11-06T22:18:05.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.086+0000 7f75d11ff700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f75b805f390 0x7f75b8061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:05.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.086+0000 7f75d11ff700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75cc14c640 0x7f75cc0ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.086+0000 7f75d11ff700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75cc0bb990 0x7f75cc148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.086+0000 7f75d11ff700 1 --2- 172.21.15.33:0/1617879781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75cc0c33f0 0x7f75cc0abb10 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.086+0000 7f75d11ff700 1 -- 172.21.15.33:0/1617879781 >> 172.21.15.33:0/1617879781 conn(0x7f75cc062710 msgr2=0x7f75cc1545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:05.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.086+0000 7f75d11ff700 1 -- 172.21.15.33:0/1617879781 shutdown_connections 2023-11-06T22:18:05.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.086+0000 7f75d11ff700 1 -- 172.21.15.33:0/1617879781 wait complete. 2023-11-06T22:18:05.091 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 84 2023-11-06T22:18:05.107 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:05.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.308+0000 7fcf647ff700 1 Processor -- start 2023-11-06T22:18:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.308+0000 7fcf647ff700 1 -- start start 2023-11-06T22:18:05.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.309+0000 7fcf647ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c064ce0 0x7fcf5c1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.309+0000 7fcf647ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcf5c0aade0 0x7fcf5c0c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.309+0000 7fcf647ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5c0c0880 con 0x7fcf5c154110 2023-11-06T22:18:05.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.309+0000 7fcf647ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5c0bfd40 con 0x7fcf5c0aade0 2023-11-06T22:18:05.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.309+0000 7fcf647ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0c8260 con 0x7fcf5c064ce0 2023-11-06T22:18:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf62d9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcf5c0aade0 0x7fcf5c0c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf61d9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c064ce0 0x7fcf5c1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf61d9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c064ce0 0x7fcf5c1544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:52270/0 (socket says 172.21.15.33:52270) 2023-11-06T22:18:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf62d9c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcf5c0aade0 0x7fcf5c0c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:51280/0 (socket says 172.21.15.33:51280) 2023-11-06T22:18:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf61d9a700 1 -- 172.21.15.33:0/3035285982 learned_addr learned my addr 172.21.15.33:0/3035285982 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:05.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf6259b700 1 --1- 172.21.15.33:0/3035285982 >> v1:172.21.15.69:6789/0 conn(0x7fcf5c154110 0x7fcf5c0645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:39100/0 (socket says 172.21.15.33:39100) 2023-11-06T22:18:05.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf61d9a700 1 -- 172.21.15.33:0/3035285982 >> v2:172.21.15.69:3300/0 conn(0x7fcf5c0aade0 msgr2=0x7fcf5c0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:05.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf61d9a700 1 --2- 172.21.15.33:0/3035285982 >> v2:172.21.15.69:3300/0 conn(0x7fcf5c0aade0 0x7fcf5c0c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf61d9a700 1 -- 172.21.15.33:0/3035285982 >> v1:172.21.15.69:6789/0 conn(0x7fcf5c154110 legacy=0x7fcf5c0645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf61d9a700 1 -- 172.21.15.33:0/3035285982 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c0bff50 con 0x7fcf5c064ce0 2023-11-06T22:18:05.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.310+0000 7fcf61d9a700 1 --2- 172.21.15.33:0/3035285982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c064ce0 0x7fcf5c1544f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fcf4c006fd0 tx=0x7fcf4c0090f0).ready entity=mon.2 client_cookie=76003e4fbbf6b469 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:05.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf61599700 1 -- 172.21.15.33:0/3035285982 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00b580 con 0x7fcf5c064ce0 2023-11-06T22:18:05.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf61599700 1 -- 172.21.15.33:0/3035285982 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c00b6e0 con 0x7fcf5c064ce0 2023-11-06T22:18:05.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf61599700 1 -- 172.21.15.33:0/3035285982 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00ba60 con 0x7fcf5c064ce0 2023-11-06T22:18:05.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf647ff700 1 -- 172.21.15.33:0/3035285982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c064ce0 msgr2=0x7fcf5c1544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf647ff700 1 --2- 172.21.15.33:0/3035285982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c064ce0 0x7fcf5c1544f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fcf4c006fd0 tx=0x7fcf4c0090f0).stop 2023-11-06T22:18:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf647ff700 1 -- 172.21.15.33:0/3035285982 shutdown_connections 2023-11-06T22:18:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf647ff700 1 --2- 172.21.15.33:0/3035285982 >> v2:172.21.15.69:3300/0 conn(0x7fcf5c0aade0 0x7fcf5c0c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf647ff700 1 --2- 172.21.15.33:0/3035285982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c064ce0 0x7fcf5c1544f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf647ff700 1 -- 172.21.15.33:0/3035285982 >> 172.21.15.33:0/3035285982 conn(0x7fcf5c0cb8b0 msgr2=0x7fcf5c0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf647ff700 1 -- 172.21.15.33:0/3035285982 shutdown_connections 2023-11-06T22:18:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.311+0000 7fcf647ff700 1 -- 172.21.15.33:0/3035285982 wait complete. 2023-11-06T22:18:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.313+0000 7fcf647ff700 1 Processor -- start 2023-11-06T22:18:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.313+0000 7fcf647ff700 1 -- start start 2023-11-06T22:18:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf647ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf5c064ce0 0x7fcf5c0ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf647ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf5c0aade0 0x7fcf5c0c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf647ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c154110 0x7fcf5c0c3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf647ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0aeb60 con 0x7fcf5c064ce0 2023-11-06T22:18:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf647ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0adf50 con 0x7fcf5c0aade0 2023-11-06T22:18:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf647ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0ae230 con 0x7fcf5c154110 2023-11-06T22:18:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf6259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf5c064ce0 0x7fcf5c0ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf6259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf5c064ce0 0x7fcf5c0ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45686/0 (socket says 172.21.15.33:45686) 2023-11-06T22:18:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf6259b700 1 -- 172.21.15.33:0/1728087222 learned_addr learned my addr 172.21.15.33:0/1728087222 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf61d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf5c0aade0 0x7fcf5c0c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.314+0000 7fcf62d9c700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c154110 0x7fcf5c0c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf62d9c700 1 -- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf5c0aade0 msgr2=0x7fcf5c0c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf62d9c700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf5c0aade0 0x7fcf5c0c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf62d9c700 1 -- 172.21.15.33:0/1728087222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf5c064ce0 msgr2=0x7fcf5c0ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf62d9c700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf5c064ce0 0x7fcf5c0ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf62d9c700 1 -- 172.21.15.33:0/1728087222 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c009370 con 0x7fcf5c154110 2023-11-06T22:18:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf6259b700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf5c064ce0 0x7fcf5c0ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf62d9c700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c154110 0x7fcf5c0c3340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fcf54009e20 tx=0x7fcf5400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf527fc700 1 -- 172.21.15.33:0/1728087222 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400caa0 con 0x7fcf5c154110 2023-11-06T22:18:05.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf527fc700 1 -- 172.21.15.33:0/1728087222 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5400f040 con 0x7fcf5c154110 2023-11-06T22:18:05.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf527fc700 1 -- 172.21.15.33:0/1728087222 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400ce10 con 0x7fcf5c154110 2023-11-06T22:18:05.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.315+0000 7fcf647ff700 1 -- 172.21.15.33:0/1728087222 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c0ae510 con 0x7fcf5c154110 2023-11-06T22:18:05.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.316+0000 7fcf647ff700 1 -- 172.21.15.33:0/1728087222 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c0ac6a0 con 0x7fcf5c154110 2023-11-06T22:18:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.317+0000 7fcf647ff700 1 -- 172.21.15.33:0/1728087222 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5c06c6c0 con 0x7fcf5c154110 2023-11-06T22:18:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.317+0000 7fcf527fc700 1 -- 172.21.15.33:0/1728087222 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fcf54013610 con 0x7fcf5c154110 2023-11-06T22:18:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.318+0000 7fcf527fc700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcf4005f390 0x7fcf40061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.318+0000 7fcf527fc700 1 -- 172.21.15.33:0/1728087222 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fcf5407afe0 con 0x7fcf5c154110 2023-11-06T22:18:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.318+0000 7fcf6259b700 1 -- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcf4005f390 msgr2=0x7fcf40061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.318+0000 7fcf6259b700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcf4005f390 0x7fcf40061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.323+0000 7fcf527fc700 1 -- 172.21.15.33:0/1728087222 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcf5404bcd0 con 0x7fcf5c154110 2023-11-06T22:18:05.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.513+0000 7fcf647ff700 1 -- 172.21.15.33:0/1728087222 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf5c0c3c50 con 0x7fcf5c154110 2023-11-06T22:18:05.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.514+0000 7fcf527fc700 1 -- 172.21.15.33:0/1728087222 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) v1 ==== 74+0+17711 (secure 0 0 0) 0x7fcf5404f2f0 con 0x7fcf5c154110 2023-11-06T22:18:05.518 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:05.519 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":209,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:05.275003+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:16:37.603904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T22:16:38.139152+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":209,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.1f","mappings":[{"from":5,"to":3}]},{"pgid":"23.33","mappings":[{"from":5,"to":7}]},{"pgid":"23.3c","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6833/129946":"2023-11-07T22:16:53.444669+0000","172.21.15.69:6832/129946":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/2960476778":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/1246238046":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/4116472923":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/4003747652":"2023-11-07T22:16:53.444669+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:05.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 -- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcf4005f390 msgr2=0x7fcf40061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:05.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcf4005f390 0x7fcf40061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:05.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 -- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c154110 msgr2=0x7fcf5c0c3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c154110 0x7fcf5c0c3340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fcf54009e20 tx=0x7fcf5400b040).stop 2023-11-06T22:18:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 -- 172.21.15.33:0/1728087222 shutdown_connections 2023-11-06T22:18:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcf4005f390 0x7fcf40061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcf5c154110 0x7fcf5c0c3340 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf5c064ce0 0x7fcf5c0ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 --2- 172.21.15.33:0/1728087222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf5c0aade0 0x7fcf5c0c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 -- 172.21.15.33:0/1728087222 >> 172.21.15.33:0/1728087222 conn(0x7fcf5c0cb8b0 msgr2=0x7fcf5c14f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 -- 172.21.15.33:0/1728087222 shutdown_connections 2023-11-06T22:18:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.517+0000 7fcf647ff700 1 -- 172.21.15.33:0/1728087222 wait complete. 2023-11-06T22:18:05.538 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '207', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T22:16:37.603904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '203', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T22:16:38.139152+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '209', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 209, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T22:18:05.539 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T22:18:05.675 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:05.673+0000 7f48a4a58680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:18:05.758 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:05.756+0000 7f48a4a58680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:18:05.772 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:05.769+0000 7f065526c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:18:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.777+0000 7f1e0af9d700 1 Processor -- start 2023-11-06T22:18:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.777+0000 7f1e0af9d700 1 -- start start 2023-11-06T22:18:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.778+0000 7f1e0af9d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1dfc038b70 0x7f1dfc037f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.778+0000 7f1e0af9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1dfc0e1fa0 0x7f1dfc053a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.778+0000 7f1e0af9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc04f8e0 con 0x7f1dfc0e1bc0 2023-11-06T22:18:05.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e0af9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc007790 con 0x7f1dfc0e1fa0 2023-11-06T22:18:05.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e0af9d700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc0062f0 con 0x7f1dfc038b70 2023-11-06T22:18:05.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e09f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1dfc0e1bc0 0x7f1dfc04eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40208/0 (socket says 172.21.15.33:40208) 2023-11-06T22:18:05.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e09f9b700 1 -- 172.21.15.33:0/3546406503 learned_addr learned my addr 172.21.15.33:0/3546406503 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:05.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e0a79c700 1 --2- 172.21.15.33:0/3546406503 >> v2:172.21.15.33:3300/0 conn(0x7f1dfc0e1fa0 0x7f1dfc053a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:05.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e0979a700 1 --2- 172.21.15.33:0/3546406503 >> v2:172.21.15.69:3300/0 conn(0x7f1dfc038b70 0x7f1dfc037f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e08f99700 1 -- 172.21.15.33:0/3546406503 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976533698 0 0) 0x7f1e00001110 con 0x7f1dfc0e1bc0 2023-11-06T22:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e08f99700 1 -- 172.21.15.33:0/3546406503 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df00033f0 con 0x7f1dfc0e1bc0 2023-11-06T22:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e0a79c700 1 -- 172.21.15.33:0/3546406503 >> v2:172.21.15.69:3300/0 conn(0x7f1dfc038b70 msgr2=0x7f1dfc037f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e0a79c700 1 --2- 172.21.15.33:0/3546406503 >> v2:172.21.15.69:3300/0 conn(0x7f1dfc038b70 0x7f1dfc037f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.779+0000 7f1e0a79c700 1 -- 172.21.15.33:0/3546406503 >> v1:172.21.15.33:6789/0 conn(0x7f1dfc0e1bc0 legacy=0x7f1dfc04eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.780+0000 7f1e0a79c700 1 -- 172.21.15.33:0/3546406503 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc04f320 con 0x7f1dfc0e1fa0 2023-11-06T22:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.780+0000 7f1e0a79c700 1 --2- 172.21.15.33:0/3546406503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0e1fa0 0x7f1dfc053a10 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f1e040614b0 tx=0x7f1e04062130).ready entity=mon.0 client_cookie=55583e32d35771e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.780+0000 7f1e08f99700 1 -- 172.21.15.33:0/3546406503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04047340 con 0x7f1dfc0e1fa0 2023-11-06T22:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.780+0000 7f1e08f99700 1 -- 172.21.15.33:0/3546406503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e04086070 con 0x7f1dfc0e1fa0 2023-11-06T22:18:05.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.780+0000 7f1e08f99700 1 -- 172.21.15.33:0/3546406503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04046d40 con 0x7f1dfc0e1fa0 2023-11-06T22:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.780+0000 7f1e0af9d700 1 -- 172.21.15.33:0/3546406503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0e1fa0 msgr2=0x7f1dfc053a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.780+0000 7f1e0af9d700 1 --2- 172.21.15.33:0/3546406503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0e1fa0 0x7f1dfc053a10 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f1e040614b0 tx=0x7f1e04062130).stop 2023-11-06T22:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.781+0000 7f1e0af9d700 1 -- 172.21.15.33:0/3546406503 shutdown_connections 2023-11-06T22:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.781+0000 7f1e0af9d700 1 --2- 172.21.15.33:0/3546406503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0e1fa0 0x7f1dfc053a10 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.781+0000 7f1e0af9d700 1 --2- 172.21.15.33:0/3546406503 >> v2:172.21.15.69:3300/0 conn(0x7f1dfc038b70 0x7f1dfc037f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.781+0000 7f1e0af9d700 1 -- 172.21.15.33:0/3546406503 >> 172.21.15.33:0/3546406503 conn(0x7f1dfc0554a0 msgr2=0x7f1dfc0558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.781+0000 7f1e0af9d700 1 -- 172.21.15.33:0/3546406503 shutdown_connections 2023-11-06T22:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.781+0000 7f1e0af9d700 1 -- 172.21.15.33:0/3546406503 wait complete. 2023-11-06T22:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.782+0000 7f1e0af9d700 1 Processor -- start 2023-11-06T22:18:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.782+0000 7f1e0af9d700 1 -- start start 2023-11-06T22:18:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.783+0000 7f1e0af9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc038b70 0x7f1dfc03bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.783+0000 7f1e0af9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0e1bc0 0x7f1dfc03c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.783+0000 7f1e0af9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1dfc0e1fa0 0x7f1dfc03c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.783+0000 7f1e0af9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0e0d50 con 0x7f1dfc0e1bc0 2023-11-06T22:18:05.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.783+0000 7f1e0af9d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc051f50 con 0x7f1dfc038b70 2023-11-06T22:18:05.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.783+0000 7f1e0af9d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0523e0 con 0x7f1dfc0e1fa0 2023-11-06T22:18:05.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e0979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0e1bc0 0x7f1dfc03c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:05.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e09f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc038b70 0x7f1dfc03bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:05.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e0979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0e1bc0 0x7f1dfc03c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45694/0 (socket says 172.21.15.33:45694) 2023-11-06T22:18:05.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e0979a700 1 -- 172.21.15.33:0/95861132 learned_addr learned my addr 172.21.15.33:0/95861132 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:05.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e0a79c700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1dfc0e1fa0 0x7f1dfc03c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:05.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e09f9b700 1 -- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1dfc0e1fa0 msgr2=0x7f1dfc03c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e09f9b700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1dfc0e1fa0 0x7f1dfc03c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e09f9b700 1 -- 172.21.15.33:0/95861132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0e1bc0 msgr2=0x7f1dfc03c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e09f9b700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0e1bc0 0x7f1dfc03c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e09f9b700 1 -- 172.21.15.33:0/95861132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc051620 con 0x7f1dfc038b70 2023-11-06T22:18:05.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.784+0000 7f1e09f9b700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc038b70 0x7f1dfc03bd70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f1e00005750 tx=0x7f1e00007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:05.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.785+0000 7f1df9ffb700 1 -- 172.21.15.33:0/95861132 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e000070d0 con 0x7f1dfc038b70 2023-11-06T22:18:05.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.785+0000 7f1df9ffb700 1 -- 172.21.15.33:0/95861132 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e00007230 con 0x7f1dfc038b70 2023-11-06T22:18:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.785+0000 7f1e0af9d700 1 -- 172.21.15.33:0/95861132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc051900 con 0x7f1dfc038b70 2023-11-06T22:18:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.785+0000 7f1df9ffb700 1 -- 172.21.15.33:0/95861132 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e000075d0 con 0x7f1dfc038b70 2023-11-06T22:18:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.785+0000 7f1e0af9d700 1 -- 172.21.15.33:0/95861132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc051300 con 0x7f1dfc038b70 2023-11-06T22:18:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.786+0000 7f1e0af9d700 1 -- 172.21.15.33:0/95861132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc052cd0 con 0x7f1dfc038b70 2023-11-06T22:18:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.787+0000 7f1df9ffb700 1 -- 172.21.15.33:0/95861132 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f1e000059b0 con 0x7f1dfc038b70 2023-11-06T22:18:05.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.788+0000 7f1df9ffb700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1de805f3e0 0x7f1de80618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:05.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.788+0000 7f1df9ffb700 1 -- 172.21.15.33:0/95861132 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f1e00079a30 con 0x7f1dfc038b70 2023-11-06T22:18:05.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.788+0000 7f1e0979a700 1 -- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1de805f3e0 msgr2=0x7f1de80618a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:05.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.788+0000 7f1e0979a700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1de805f3e0 0x7f1de80618a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:05.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.792+0000 7f1df9ffb700 1 -- 172.21.15.33:0/95861132 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1e0004a7f0 con 0x7f1dfc038b70 2023-11-06T22:18:05.863 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:05.860+0000 7f065526c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:18:05.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.971+0000 7f1e0af9d700 1 -- 172.21.15.33:0/95861132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1dfc04f8e0 con 0x7f1dfc038b70 2023-11-06T22:18:05.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.972+0000 7f1df9ffb700 1 -- 172.21.15.33:0/95861132 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v209) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1dfc04f8e0 con 0x7f1dfc038b70 2023-11-06T22:18:05.975 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T22:18:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 -- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1de805f3e0 msgr2=0x7f1de80618a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1de805f3e0 0x7f1de80618a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 -- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc038b70 msgr2=0x7f1dfc03bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc038b70 0x7f1dfc03bd70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f1e00005750 tx=0x7f1e00007a70).stop 2023-11-06T22:18:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 -- 172.21.15.33:0/95861132 shutdown_connections 2023-11-06T22:18:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1de805f3e0 0x7f1de80618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1dfc0e1fa0 0x7f1dfc03c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc038b70 0x7f1dfc03bd70 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 --2- 172.21.15.33:0/95861132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0e1bc0 0x7f1dfc03c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 -- 172.21.15.33:0/95861132 >> 172.21.15.33:0/95861132 conn(0x7f1dfc0554a0 msgr2=0x7f1dfc03a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 -- 172.21.15.33:0/95861132 shutdown_connections 2023-11-06T22:18:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:05.974+0000 7f1e0af9d700 1 -- 172.21.15.33:0/95861132 wait complete. 2023-11-06T22:18:05.991 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:05.989+0000 7f48a4a58680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:18:05.995 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T22:18:06.112 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:06.109+0000 7f065526c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:18:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.194+0000 7f1e601ff700 1 Processor -- start 2023-11-06T22:18:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.194+0000 7f1e601ff700 1 -- start start 2023-11-06T22:18:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.195+0000 7f1e601ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1e58154710 0x7f1e580c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.195+0000 7f1e601ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 0x7f1e580646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.195+0000 7f1e601ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e580c6b60 con 0x7f1e58154330 2023-11-06T22:18:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.195+0000 7f1e601ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e580c7700 con 0x7f1e58154710 2023-11-06T22:18:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.195+0000 7f1e601ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e580c5cd0 con 0x7f1e580c7ff0 2023-11-06T22:18:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5df9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1e58154330 0x7f1e580c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40222/0 (socket says 172.21.15.33:40222) 2023-11-06T22:18:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5df9b700 1 -- 172.21.15.33:0/2066934345 learned_addr learned my addr 172.21.15.33:0/2066934345 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5d79a700 1 --2- 172.21.15.33:0/2066934345 >> v2:172.21.15.69:3300/0 conn(0x7f1e58154710 0x7f1e580c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5e79c700 1 --2- 172.21.15.33:0/2066934345 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 0x7f1e580646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:06.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5cf99700 1 -- 172.21.15.33:0/2066934345 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414440086 0 0) 0x7f1e580c6b60 con 0x7f1e58154330 2023-11-06T22:18:06.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5cf99700 1 -- 172.21.15.33:0/2066934345 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e440033f0 con 0x7f1e58154330 2023-11-06T22:18:06.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5d79a700 1 -- 172.21.15.33:0/2066934345 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 msgr2=0x7f1e580646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:06.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5d79a700 1 --2- 172.21.15.33:0/2066934345 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 0x7f1e580646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5d79a700 1 -- 172.21.15.33:0/2066934345 >> v1:172.21.15.33:6789/0 conn(0x7f1e58154330 legacy=0x7f1e580c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5d79a700 1 -- 172.21.15.33:0/2066934345 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e580c5f50 con 0x7f1e58154710 2023-11-06T22:18:06.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.196+0000 7f1e5e79c700 1 --2- 172.21.15.33:0/2066934345 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 0x7f1e580646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:06.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.197+0000 7f1e5d79a700 1 --2- 172.21.15.33:0/2066934345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e58154710 0x7f1e580c57e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f1e48009760 tx=0x7f1e48009060).ready entity=mon.1 client_cookie=5a21232d0295950e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.197+0000 7f1e5cf99700 1 -- 172.21.15.33:0/2066934345 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800d4a0 con 0x7f1e58154710 2023-11-06T22:18:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.197+0000 7f1e5cf99700 1 -- 172.21.15.33:0/2066934345 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4800d600 con 0x7f1e58154710 2023-11-06T22:18:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.197+0000 7f1e5cf99700 1 -- 172.21.15.33:0/2066934345 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800d980 con 0x7f1e58154710 2023-11-06T22:18:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.197+0000 7f1e601ff700 1 -- 172.21.15.33:0/2066934345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e58154710 msgr2=0x7f1e580c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.197+0000 7f1e601ff700 1 --2- 172.21.15.33:0/2066934345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e58154710 0x7f1e580c57e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f1e48009760 tx=0x7f1e48009060).stop 2023-11-06T22:18:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.198+0000 7f1e601ff700 1 -- 172.21.15.33:0/2066934345 shutdown_connections 2023-11-06T22:18:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.198+0000 7f1e601ff700 1 --2- 172.21.15.33:0/2066934345 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 0x7f1e580646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.198+0000 7f1e601ff700 1 --2- 172.21.15.33:0/2066934345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e58154710 0x7f1e580c57e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.198+0000 7f1e601ff700 1 -- 172.21.15.33:0/2066934345 >> 172.21.15.33:0/2066934345 conn(0x7f1e580cb910 msgr2=0x7f1e580c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.198+0000 7f1e601ff700 1 -- 172.21.15.33:0/2066934345 shutdown_connections 2023-11-06T22:18:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.198+0000 7f1e601ff700 1 -- 172.21.15.33:0/2066934345 wait complete. 2023-11-06T22:18:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.199+0000 7f1e601ff700 1 Processor -- start 2023-11-06T22:18:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.199+0000 7f1e601ff700 1 -- start start 2023-11-06T22:18:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.200+0000 7f1e601ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 0x7f1e580af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.200+0000 7f1e601ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e58154330 0x7f1e581499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.200+0000 7f1e601ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e58154710 0x7f1e580c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.200+0000 7f1e601ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e580ae120 con 0x7f1e58154330 2023-11-06T22:18:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.200+0000 7f1e601ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e580ae400 con 0x7f1e58154710 2023-11-06T22:18:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.200+0000 7f1e601ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e580ae6e0 con 0x7f1e580c7ff0 2023-11-06T22:18:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5d79a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e58154330 0x7f1e581499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5d79a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e58154330 0x7f1e581499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45708/0 (socket says 172.21.15.33:45708) 2023-11-06T22:18:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5df9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 0x7f1e580af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5d79a700 1 -- 172.21.15.33:0/3050431193 learned_addr learned my addr 172.21.15.33:0/3050431193 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5e79c700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e58154710 0x7f1e580c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5df9b700 1 -- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e58154710 msgr2=0x7f1e580c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5df9b700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e58154710 0x7f1e580c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5df9b700 1 -- 172.21.15.33:0/3050431193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e58154330 msgr2=0x7f1e581499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5df9b700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e58154330 0x7f1e581499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5df9b700 1 -- 172.21.15.33:0/3050431193 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e50009a20 con 0x7f1e580c7ff0 2023-11-06T22:18:06.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5d79a700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e58154330 0x7f1e581499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:06.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.201+0000 7f1e5df9b700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 0x7f1e580af1c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f1e54001ed0 tx=0x7f1e54007bd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:06.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.202+0000 7f1e4dffb700 1 -- 172.21.15.33:0/3050431193 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5400ce40 con 0x7f1e580c7ff0 2023-11-06T22:18:06.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.202+0000 7f1e4dffb700 1 -- 172.21.15.33:0/3050431193 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e540072e0 con 0x7f1e580c7ff0 2023-11-06T22:18:06.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.202+0000 7f1e4dffb700 1 -- 172.21.15.33:0/3050431193 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540057a0 con 0x7f1e580c7ff0 2023-11-06T22:18:06.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.202+0000 7f1e601ff700 1 -- 172.21.15.33:0/3050431193 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e480092a0 con 0x7f1e580c7ff0 2023-11-06T22:18:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.202+0000 7f1e601ff700 1 -- 172.21.15.33:0/3050431193 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e580aeca0 con 0x7f1e580c7ff0 2023-11-06T22:18:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.203+0000 7f1e601ff700 1 -- 172.21.15.33:0/3050431193 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e580830b0 con 0x7f1e580c7ff0 2023-11-06T22:18:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.204+0000 7f1e4dffb700 1 -- 172.21.15.33:0/3050431193 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f1e54005b90 con 0x7f1e580c7ff0 2023-11-06T22:18:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.204+0000 7f1e4dffb700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1e3c05f390 0x7f1e3c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.205+0000 7f1e4dffb700 1 -- 172.21.15.33:0/3050431193 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f1e54079eb0 con 0x7f1e580c7ff0 2023-11-06T22:18:06.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.205+0000 7f1e5d79a700 1 -- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1e3c05f390 msgr2=0x7f1e3c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:06.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.205+0000 7f1e5d79a700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1e3c05f390 0x7f1e3c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:06.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.209+0000 7f1e4dffb700 1 -- 172.21.15.33:0/3050431193 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1e5404abf0 con 0x7f1e580c7ff0 2023-11-06T22:18:06.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.389+0000 7f1e601ff700 1 -- 172.21.15.33:0/3050431193 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e580ae820 con 0x7f1e580c7ff0 2023-11-06T22:18:06.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.390+0000 7f1e4dffb700 1 -- 172.21.15.33:0/3050431193 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v209) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e580ae820 con 0x7f1e580c7ff0 2023-11-06T22:18:06.393 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T22:18:06.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 -- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1e3c05f390 msgr2=0x7f1e3c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:06.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1e3c05f390 0x7f1e3c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:06.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 -- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 msgr2=0x7f1e580af1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 0x7f1e580af1c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f1e54001ed0 tx=0x7f1e54007bd0).stop 2023-11-06T22:18:06.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 -- 172.21.15.33:0/3050431193 shutdown_connections 2023-11-06T22:18:06.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f1e3c05f390 0x7f1e3c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:06.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f1e580c7ff0 0x7f1e580af1c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e58154710 0x7f1e580c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 --2- 172.21.15.33:0/3050431193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e58154330 0x7f1e581499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 -- 172.21.15.33:0/3050431193 >> 172.21.15.33:0/3050431193 conn(0x7f1e580cb910 msgr2=0x7f1e580c88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:06.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 -- 172.21.15.33:0/3050431193 shutdown_connections 2023-11-06T22:18:06.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.392+0000 7f1e601ff700 1 -- 172.21.15.33:0/3050431193 wait complete. 2023-11-06T22:18:06.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T22:18:06.505 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:06.503+0000 7f48a4a58680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:18:06.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.626+0000 7f3a0a5ff700 1 Processor -- start 2023-11-06T22:18:06.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.626+0000 7f3a0a5ff700 1 -- start start 2023-11-06T22:18:06.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.627+0000 7f3a0a5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3a0414fe80 0x7f3a04064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.628+0000 7f3a0a5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a04064270 con 0x7f3a0414faa0 2023-11-06T22:18:06.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.628+0000 7f3a0a5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a040651b0 con 0x7f3a040c3450 2023-11-06T22:18:06.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.628+0000 7f3a0a5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a041531f0 con 0x7f3a0414fe80 2023-11-06T22:18:06.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.628+0000 7f3a037fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3a0414fe80 0x7f3a04064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:06.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.628+0000 7f3a03fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3a0414faa0 0x7f3a04152750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40230/0 (socket says 172.21.15.33:40230) 2023-11-06T22:18:06.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.628+0000 7f3a03fff700 1 -- 172.21.15.33:0/1909123459 learned_addr learned my addr 172.21.15.33:0/1909123459 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:06.632 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:06.628+0000 7f48a4a58680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:06.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.628+0000 7f3a037fe700 1 -- 172.21.15.33:0/1909123459 >> v1:172.21.15.69:6789/0 conn(0x7f3a040c3450 legacy=0x7f3a040bbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.628+0000 7f3a037fe700 1 -- 172.21.15.33:0/1909123459 >> v1:172.21.15.33:6789/0 conn(0x7f3a0414faa0 legacy=0x7f3a04152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.628+0000 7f3a037fe700 1 -- 172.21.15.33:0/1909123459 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a041534d0 con 0x7f3a0414fe80 2023-11-06T22:18:06.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.628+0000 7f3a02ffd700 1 -- 172.21.15.33:0/1909123459 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997314954 0 0) 0x7f3a04064270 con 0x7f3a0414faa0 2023-11-06T22:18:06.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.629+0000 7f3a037fe700 1 --2- 172.21.15.33:0/1909123459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a0414fe80 0x7f3a04064cc0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f39f8009760 tx=0x7f39f8009060).ready entity=mon.0 client_cookie=40c64170acccc50f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:06.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.629+0000 7f3a02ffd700 1 -- 172.21.15.33:0/1909123459 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f800b580 con 0x7f3a0414fe80 2023-11-06T22:18:06.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.629+0000 7f3a02ffd700 1 -- 172.21.15.33:0/1909123459 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f800b6e0 con 0x7f3a0414fe80 2023-11-06T22:18:06.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.629+0000 7f3a02ffd700 1 -- 172.21.15.33:0/1909123459 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f800ba60 con 0x7f3a0414fe80 2023-11-06T22:18:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.629+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/1909123459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a0414fe80 msgr2=0x7f3a04064cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.629+0000 7f3a0a5ff700 1 --2- 172.21.15.33:0/1909123459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a0414fe80 0x7f3a04064cc0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f39f8009760 tx=0x7f39f8009060).stop 2023-11-06T22:18:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.630+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/1909123459 shutdown_connections 2023-11-06T22:18:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.630+0000 7f3a0a5ff700 1 --2- 172.21.15.33:0/1909123459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a0414fe80 0x7f3a04064cc0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.630+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/1909123459 >> 172.21.15.33:0/1909123459 conn(0x7f3a04062710 msgr2=0x7f3a04062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.630+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/1909123459 shutdown_connections 2023-11-06T22:18:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.630+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/1909123459 wait complete. 2023-11-06T22:18:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.631+0000 7f3a0a5ff700 1 Processor -- start 2023-11-06T22:18:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.631+0000 7f3a0a5ff700 1 -- start start 2023-11-06T22:18:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.632+0000 7f3a0a5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a040c3450 0x7f3a0414a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.632+0000 7f3a0a5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a0414faa0 0x7f3a040abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.632+0000 7f3a0a5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a0414fe80 0x7f3a0414e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.632+0000 7f3a0a5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0414ae80 con 0x7f3a0414fe80 2023-11-06T22:18:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.632+0000 7f3a0a5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040c5620 con 0x7f3a0414faa0 2023-11-06T22:18:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.632+0000 7f3a0a5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0414bdb0 con 0x7f3a040c3450 2023-11-06T22:18:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a037fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a0414faa0 0x7f3a040abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a03fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a040c3450 0x7f3a0414a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a037fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a0414faa0 0x7f3a040abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:51340/0 (socket says 172.21.15.33:51340) 2023-11-06T22:18:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a08b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a0414fe80 0x7f3a0414e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a037fe700 1 -- 172.21.15.33:0/2273227531 learned_addr learned my addr 172.21.15.33:0/2273227531 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:06.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a037fe700 1 -- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a040c3450 msgr2=0x7f3a0414a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a037fe700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a040c3450 0x7f3a0414a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a037fe700 1 -- 172.21.15.33:0/2273227531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a0414fe80 msgr2=0x7f3a0414e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a037fe700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a0414fe80 0x7f3a0414e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a037fe700 1 -- 172.21.15.33:0/2273227531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f80092a0 con 0x7f3a0414faa0 2023-11-06T22:18:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a08b9c700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a0414fe80 0x7f3a0414e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.633+0000 7f3a037fe700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a0414faa0 0x7f3a040abda0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f3a040c7a90 tx=0x7f39f8017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.634+0000 7f39ebfff700 1 -- 172.21.15.33:0/2273227531 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f800b850 con 0x7f3a0414faa0 2023-11-06T22:18:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.634+0000 7f39ebfff700 1 -- 172.21.15.33:0/2273227531 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f800b9b0 con 0x7f3a0414faa0 2023-11-06T22:18:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.634+0000 7f39ebfff700 1 -- 172.21.15.33:0/2273227531 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f801b670 con 0x7f3a0414faa0 2023-11-06T22:18:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.634+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/2273227531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0414b1a0 con 0x7f3a0414faa0 2023-11-06T22:18:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.634+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/2273227531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0414b760 con 0x7f3a0414faa0 2023-11-06T22:18:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.635+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/2273227531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0414bf60 con 0x7f3a0414faa0 2023-11-06T22:18:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.636+0000 7f39ebfff700 1 -- 172.21.15.33:0/2273227531 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f39f801ba20 con 0x7f3a0414faa0 2023-11-06T22:18:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.636+0000 7f39ebfff700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f39f005f390 0x7f39f0061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:06.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.637+0000 7f39ebfff700 1 -- 172.21.15.33:0/2273227531 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f39f807bee0 con 0x7f3a0414faa0 2023-11-06T22:18:06.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.637+0000 7f3a03fff700 1 -- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f39f005f390 msgr2=0x7f39f0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:06.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.637+0000 7f3a03fff700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f39f005f390 0x7f39f0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:06.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.641+0000 7f39ebfff700 1 -- 172.21.15.33:0/2273227531 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3a0414bf60 con 0x7f3a0414faa0 2023-11-06T22:18:06.673 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:06.671+0000 7f065526c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:18:06.810 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:06.808+0000 7f065526c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:06.819 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:06.817+0000 7f48a4a58680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:06.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.827+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/2273227531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a0414bf60 con 0x7f3a0414faa0 2023-11-06T22:18:06.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.828+0000 7f39ebfff700 1 -- 172.21.15.33:0/2273227531 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v209) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a0414bf60 con 0x7f3a0414faa0 2023-11-06T22:18:06.831 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T22:18:06.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.830+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f39f005f390 msgr2=0x7f39f0061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:06.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.830+0000 7f3a0a5ff700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f39f005f390 0x7f39f0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:06.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.830+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a0414faa0 msgr2=0x7f3a040abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:06.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.830+0000 7f3a0a5ff700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a0414faa0 0x7f3a040abda0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f3a040c7a90 tx=0x7f39f8017000).stop 2023-11-06T22:18:06.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.830+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/2273227531 shutdown_connections 2023-11-06T22:18:06.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.830+0000 7f3a0a5ff700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f39f005f390 0x7f39f0061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:06.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.830+0000 7f3a0a5ff700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3a040c3450 0x7f3a0414a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.830+0000 7f3a0a5ff700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a0414fe80 0x7f3a0414e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.830+0000 7f3a0a5ff700 1 --2- 172.21.15.33:0/2273227531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a0414faa0 0x7f3a040abda0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:06.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.830+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/2273227531 >> 172.21.15.33:0/2273227531 conn(0x7f3a04062710 msgr2=0x7f3a04068160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:06.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.831+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/2273227531 shutdown_connections 2023-11-06T22:18:06.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:06.831+0000 7f3a0a5ff700 1 -- 172.21.15.33:0/2273227531 wait complete. 2023-11-06T22:18:06.851 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T22:18:06.915 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:06.913+0000 7f48a4a58680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:18:07.001 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:06.998+0000 7f48a4a58680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:18:07.007 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:07.004+0000 7f065526c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.053+0000 7faf6f5ff700 1 Processor -- start 2023-11-06T22:18:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.053+0000 7faf6f5ff700 1 -- start start 2023-11-06T22:18:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf6f5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf68064b40 0x7faf680c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf6f5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf681556d0 con 0x7faf681544f0 2023-11-06T22:18:07.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf6f5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf68154ac0 con 0x7faf680646c0 2023-11-06T22:18:07.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf6f5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf681548d0 con 0x7faf68064b40 2023-11-06T22:18:07.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf6d39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faf681544f0 0x7faf680c0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40232/0 (socket says 172.21.15.33:40232) 2023-11-06T22:18:07.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf6d39b700 1 -- 172.21.15.33:0/4251506704 learned_addr learned my addr 172.21.15.33:0/4251506704 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf6db9c700 1 --2- 172.21.15.33:0/4251506704 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf68064b40 0x7faf680c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf5ffff700 1 -- 172.21.15.33:0/4251506704 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286176712 0 0) 0x7faf681556d0 con 0x7faf681544f0 2023-11-06T22:18:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf5ffff700 1 -- 172.21.15.33:0/4251506704 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf540033f0 con 0x7faf681544f0 2023-11-06T22:18:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf5ffff700 1 -- 172.21.15.33:0/4251506704 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729734176 0 0) 0x7faf68154ac0 con 0x7faf680646c0 2023-11-06T22:18:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.056+0000 7faf5ffff700 1 -- 172.21.15.33:0/4251506704 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf681556d0 con 0x7faf680646c0 2023-11-06T22:18:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.057+0000 7faf6db9c700 1 -- 172.21.15.33:0/4251506704 >> v1:172.21.15.69:6789/0 conn(0x7faf680646c0 legacy=0x7faf680c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.057+0000 7faf6db9c700 1 -- 172.21.15.33:0/4251506704 >> v1:172.21.15.33:6789/0 conn(0x7faf681544f0 legacy=0x7faf680c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.057+0000 7faf6db9c700 1 -- 172.21.15.33:0/4251506704 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf68154da0 con 0x7faf68064b40 2023-11-06T22:18:07.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.057+0000 7faf6db9c700 1 --2- 172.21.15.33:0/4251506704 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf68064b40 0x7faf680c5ef0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7faf60009e90 tx=0x7faf60009780).ready entity=mon.2 client_cookie=618e0ee1e4af5dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:07.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.057+0000 7faf5ffff700 1 -- 172.21.15.33:0/4251506704 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000e070 con 0x7faf68064b40 2023-11-06T22:18:07.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.057+0000 7faf5ffff700 1 -- 172.21.15.33:0/4251506704 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6000bab0 con 0x7faf68064b40 2023-11-06T22:18:07.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.057+0000 7faf5ffff700 1 -- 172.21.15.33:0/4251506704 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000be30 con 0x7faf68064b40 2023-11-06T22:18:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.058+0000 7faf6f5ff700 1 -- 172.21.15.33:0/4251506704 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf68064b40 msgr2=0x7faf680c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.058+0000 7faf6f5ff700 1 --2- 172.21.15.33:0/4251506704 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf68064b40 0x7faf680c5ef0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7faf60009e90 tx=0x7faf60009780).stop 2023-11-06T22:18:07.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.058+0000 7faf6f5ff700 1 -- 172.21.15.33:0/4251506704 shutdown_connections 2023-11-06T22:18:07.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.058+0000 7faf6f5ff700 1 --2- 172.21.15.33:0/4251506704 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf68064b40 0x7faf680c5ef0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.058+0000 7faf6f5ff700 1 -- 172.21.15.33:0/4251506704 >> 172.21.15.33:0/4251506704 conn(0x7faf680cb900 msgr2=0x7faf680c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:07.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.058+0000 7faf6f5ff700 1 -- 172.21.15.33:0/4251506704 shutdown_connections 2023-11-06T22:18:07.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.058+0000 7faf6f5ff700 1 -- 172.21.15.33:0/4251506704 wait complete. 2023-11-06T22:18:07.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.059+0000 7faf6f5ff700 1 Processor -- start 2023-11-06T22:18:07.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.059+0000 7faf6f5ff700 1 -- start start 2023-11-06T22:18:07.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.060+0000 7faf6f5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680646c0 0x7faf68147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.060+0000 7faf6f5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf68064b40 0x7faf680c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.060+0000 7faf6f5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681544f0 0x7faf680c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.060+0000 7faf6f5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf680c3990 con 0x7faf681544f0 2023-11-06T22:18:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.060+0000 7faf6f5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf680c2d80 con 0x7faf68064b40 2023-11-06T22:18:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.060+0000 7faf6f5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf680c3060 con 0x7faf680646c0 2023-11-06T22:18:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6db9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681544f0 0x7faf680c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6d39b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680646c0 0x7faf68147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6db9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681544f0 0x7faf680c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45746/0 (socket says 172.21.15.33:45746) 2023-11-06T22:18:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6db9c700 1 -- 172.21.15.33:0/1242545254 learned_addr learned my addr 172.21.15.33:0/1242545254 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6cb9a700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf68064b40 0x7faf680c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6db9c700 1 -- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680646c0 msgr2=0x7faf68147d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6db9c700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680646c0 0x7faf68147d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6db9c700 1 -- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf68064b40 msgr2=0x7faf680c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6db9c700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf68064b40 0x7faf680c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6db9c700 1 -- 172.21.15.33:0/1242545254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60009a20 con 0x7faf681544f0 2023-11-06T22:18:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.061+0000 7faf6db9c700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681544f0 0x7faf680c7870 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7faf6000b610 tx=0x7faf60009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.062+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1242545254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60013e90 con 0x7faf681544f0 2023-11-06T22:18:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.062+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1242545254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6001d630 con 0x7faf681544f0 2023-11-06T22:18:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.062+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1242545254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6001d9f0 con 0x7faf681544f0 2023-11-06T22:18:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.062+0000 7faf6f5ff700 1 -- 172.21.15.33:0/1242545254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf680c3340 con 0x7faf681544f0 2023-11-06T22:18:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.062+0000 7faf6f5ff700 1 -- 172.21.15.33:0/1242545254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf68148210 con 0x7faf681544f0 2023-11-06T22:18:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.063+0000 7faf6f5ff700 1 -- 172.21.15.33:0/1242545254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf680c3d20 con 0x7faf681544f0 2023-11-06T22:18:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.064+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1242545254 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7faf60003040 con 0x7faf681544f0 2023-11-06T22:18:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.065+0000 7faf5d7fa700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf4c05f390 0x7faf4c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.065+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1242545254 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6582+0+0 (secure 0 0 0) 0x7faf6007c790 con 0x7faf681544f0 2023-11-06T22:18:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.065+0000 7faf6d39b700 1 -- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf4c05f390 msgr2=0x7faf4c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.065+0000 7faf6d39b700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf4c05f390 0x7faf4c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.068+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1242545254 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf60050df0 con 0x7faf681544f0 2023-11-06T22:18:07.111 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:07.108+0000 7f065526c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:18:07.204 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:07.202+0000 7f065526c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:18:07.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.253+0000 7faf6f5ff700 1 -- 172.21.15.33:0/1242545254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf680c3ad0 con 0x7faf681544f0 2023-11-06T22:18:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.254+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1242545254 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v209) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf600509e0 con 0x7faf681544f0 2023-11-06T22:18:07.257 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T22:18:07.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 -- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf4c05f390 msgr2=0x7faf4c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:07.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf4c05f390 0x7faf4c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:07.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 -- 172.21.15.33:0/1242545254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681544f0 msgr2=0x7faf680c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681544f0 0x7faf680c7870 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7faf6000b610 tx=0x7faf60009780).stop 2023-11-06T22:18:07.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 -- 172.21.15.33:0/1242545254 shutdown_connections 2023-11-06T22:18:07.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7faf4c05f390 0x7faf4c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:07.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faf680646c0 0x7faf68147d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681544f0 0x7faf680c7870 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 --2- 172.21.15.33:0/1242545254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf68064b40 0x7faf680c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 -- 172.21.15.33:0/1242545254 >> 172.21.15.33:0/1242545254 conn(0x7faf680cb900 msgr2=0x7faf68153c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.256+0000 7faf6f5ff700 1 -- 172.21.15.33:0/1242545254 shutdown_connections 2023-11-06T22:18:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.257+0000 7faf6f5ff700 1 -- 172.21.15.33:0/1242545254 wait complete. 2023-11-06T22:18:07.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:07.317 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:07.315+0000 7f48a4a58680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:18:07.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.482+0000 7fabaabff700 1 Processor -- start 2023-11-06T22:18:07.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.482+0000 7fabaabff700 1 -- start start 2023-11-06T22:18:07.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.483+0000 7fabaabff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faba41521a0 0x7faba40c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.483+0000 7fabaabff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faba4155320 0x7faba40c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7fabaabff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba4154e90 con 0x7faba4151dc0 2023-11-06T22:18:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7fabaabff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba40639d0 con 0x7faba41521a0 2023-11-06T22:18:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7fabaabff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba4063cb0 con 0x7faba4155320 2023-11-06T22:18:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7faba3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faba41521a0 0x7faba40c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7faba899b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faba4151dc0 0x7faba4063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40244/0 (socket says 172.21.15.33:40244) 2023-11-06T22:18:07.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7faba899b700 1 -- 172.21.15.33:0/1127575983 learned_addr learned my addr 172.21.15.33:0/1127575983 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:07.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7faba919c700 1 --2- 172.21.15.33:0/1127575983 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faba4155320 0x7faba40c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7faba3fff700 1 -- 172.21.15.33:0/1127575983 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faba4155320 msgr2=0x7faba40c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:07.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7faba3fff700 1 --2- 172.21.15.33:0/1127575983 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faba4155320 0x7faba40c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7faba3fff700 1 -- 172.21.15.33:0/1127575983 >> v1:172.21.15.33:6789/0 conn(0x7faba4151dc0 legacy=0x7faba4063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7faba3fff700 1 -- 172.21.15.33:0/1127575983 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba4063f90 con 0x7faba41521a0 2023-11-06T22:18:07.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.484+0000 7faba37fe700 1 -- 172.21.15.33:0/1127575983 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059168339 0 0) 0x7faba4154e90 con 0x7faba4151dc0 2023-11-06T22:18:07.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.485+0000 7faba3fff700 1 --2- 172.21.15.33:0/1127575983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 0x7faba40c0c50 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fab90009760 tx=0x7fab90009060).ready entity=mon.0 client_cookie=823a45acf681ba7c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:07.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.485+0000 7faba37fe700 1 -- 172.21.15.33:0/1127575983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9000f510 con 0x7faba41521a0 2023-11-06T22:18:07.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.485+0000 7faba37fe700 1 -- 172.21.15.33:0/1127575983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9000f670 con 0x7faba41521a0 2023-11-06T22:18:07.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.485+0000 7faba37fe700 1 -- 172.21.15.33:0/1127575983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9000f9f0 con 0x7faba41521a0 2023-11-06T22:18:07.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.485+0000 7fabaabff700 1 -- 172.21.15.33:0/1127575983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 msgr2=0x7faba40c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.485+0000 7fabaabff700 1 --2- 172.21.15.33:0/1127575983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 0x7faba40c0c50 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fab90009760 tx=0x7fab90009060).stop 2023-11-06T22:18:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.486+0000 7fabaabff700 1 -- 172.21.15.33:0/1127575983 shutdown_connections 2023-11-06T22:18:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.486+0000 7fabaabff700 1 --2- 172.21.15.33:0/1127575983 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faba4155320 0x7faba40c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.486+0000 7fabaabff700 1 --2- 172.21.15.33:0/1127575983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 0x7faba40c0c50 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.486+0000 7fabaabff700 1 -- 172.21.15.33:0/1127575983 >> 172.21.15.33:0/1127575983 conn(0x7faba405e1b0 msgr2=0x7faba405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.486+0000 7fabaabff700 1 -- 172.21.15.33:0/1127575983 shutdown_connections 2023-11-06T22:18:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.486+0000 7fabaabff700 1 -- 172.21.15.33:0/1127575983 wait complete. 2023-11-06T22:18:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.487+0000 7fabaabff700 1 Processor -- start 2023-11-06T22:18:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.487+0000 7fabaabff700 1 -- start start 2023-11-06T22:18:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.488+0000 7fabaabff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4151dc0 0x7faba41654d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.488+0000 7fabaabff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 0x7faba40b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7fabaabff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faba4155320 0x7faba4144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7fabaabff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba40b0580 con 0x7faba41521a0 2023-11-06T22:18:07.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7fabaabff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba40af970 con 0x7faba4151dc0 2023-11-06T22:18:07.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7fabaabff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba40afc50 con 0x7faba4155320 2023-11-06T22:18:07.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7faba3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 0x7faba40b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7faba3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 0x7faba40b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45774/0 (socket says 172.21.15.33:45774) 2023-11-06T22:18:07.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7faba3fff700 1 -- 172.21.15.33:0/1288525584 learned_addr learned my addr 172.21.15.33:0/1288525584 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:07.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7faba899b700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4151dc0 0x7faba41654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7faba919c700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faba4155320 0x7faba4144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7faba3fff700 1 -- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faba4155320 msgr2=0x7faba4144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7faba3fff700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faba4155320 0x7faba4144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7faba3fff700 1 -- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4151dc0 msgr2=0x7faba41654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7faba3fff700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4151dc0 0x7faba41654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.489+0000 7faba3fff700 1 -- 172.21.15.33:0/1288525584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900092a0 con 0x7faba41521a0 2023-11-06T22:18:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.490+0000 7faba3fff700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 0x7faba40b0a10 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7faba405f190 tx=0x7fab90017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.490+0000 7faba0ff9700 1 -- 172.21.15.33:0/1288525584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9000f7e0 con 0x7faba41521a0 2023-11-06T22:18:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.490+0000 7faba0ff9700 1 -- 172.21.15.33:0/1288525584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9000f940 con 0x7faba41521a0 2023-11-06T22:18:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.490+0000 7fabaabff700 1 -- 172.21.15.33:0/1288525584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba40aff30 con 0x7faba41521a0 2023-11-06T22:18:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.490+0000 7faba0ff9700 1 -- 172.21.15.33:0/1288525584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9001b660 con 0x7faba41521a0 2023-11-06T22:18:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.490+0000 7fabaabff700 1 -- 172.21.15.33:0/1288525584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba40af060 con 0x7faba41521a0 2023-11-06T22:18:07.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.491+0000 7fabaabff700 1 -- 172.21.15.33:0/1288525584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba40b06f0 con 0x7faba41521a0 2023-11-06T22:18:07.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.493+0000 7faba0ff9700 1 -- 172.21.15.33:0/1288525584 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fab9001ba50 con 0x7faba41521a0 2023-11-06T22:18:07.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.494+0000 7faba0ff9700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fab8805f390 0x7fab88061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.494+0000 7faba0ff9700 1 -- 172.21.15.33:0/1288525584 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fab9007c240 con 0x7faba41521a0 2023-11-06T22:18:07.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.494+0000 7faba899b700 1 -- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fab8805f390 msgr2=0x7fab88061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:07.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.494+0000 7faba899b700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fab8805f390 0x7fab88061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:07.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.498+0000 7faba0ff9700 1 -- 172.21.15.33:0/1288525584 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fab9004cf80 con 0x7faba41521a0 2023-11-06T22:18:07.544 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:07.542+0000 7f48a4a58680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:07.551 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:07.548+0000 7f065526c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:18:07.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.695+0000 7faba899b700 1 -- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fab8805f390 msgr2=0x7fab88061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:07.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.695+0000 7faba899b700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fab8805f390 0x7fab88061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:18:07.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.727+0000 7fabaabff700 1 -- 172.21.15.33:0/1288525584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faba40af730 con 0x7faba41521a0 2023-11-06T22:18:07.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.727+0000 7faba0ff9700 1 -- 172.21.15.33:0/1288525584 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4493 (secure 0 0 0) 0x7faba40af730 con 0x7faba41521a0 2023-11-06T22:18:07.730 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:07.731 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3815920714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3815920714},{"type":"v1","addr":"172.21.15.69:6836","nonce":3815920714}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9052,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3614716748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3614716748},{"type":"v1","addr":"172.21.15.69:6837","nonce":3614716748}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1513187241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1513187241},{"type":"v1","addr":"172.21.15.33:6834","nonce":1513187241}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:16:39.145216+0000","modified":"2023-11-06T22:16:42.180077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3197264665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3197264665},{"type":"v1","addr":"172.21.15.33:6835","nonce":3197264665}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-06T22:18:07.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 -- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fab8805f390 msgr2=0x7fab88061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:07.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fab8805f390 0x7fab88061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:07.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 -- 172.21.15.33:0/1288525584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 msgr2=0x7faba40b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 0x7faba40b0a10 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7faba405f190 tx=0x7fab90017000).stop 2023-11-06T22:18:07.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 -- 172.21.15.33:0/1288525584 shutdown_connections 2023-11-06T22:18:07.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fab8805f390 0x7fab88061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:07.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7faba4155320 0x7faba4144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4151dc0 0x7faba41654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 --2- 172.21.15.33:0/1288525584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba41521a0 0x7faba40b0a10 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 -- 172.21.15.33:0/1288525584 >> 172.21.15.33:0/1288525584 conn(0x7faba405e1b0 msgr2=0x7faba40c9640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:07.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 -- 172.21.15.33:0/1288525584 shutdown_connections 2023-11-06T22:18:07.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.730+0000 7fabaabff700 1 -- 172.21.15.33:0/1288525584 wait complete. 2023-11-06T22:18:07.735 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 84 2023-11-06T22:18:07.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:07.785 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:07.783+0000 7f065526c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.955+0000 7fb695fff700 1 Processor -- start 2023-11-06T22:18:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.955+0000 7fb695fff700 1 -- start start 2023-11-06T22:18:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.956+0000 7fb695fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb69014c640 0x7fb6900bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.956+0000 7fb695fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb690153470 con 0x7fb690152ff0 2023-11-06T22:18:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.956+0000 7fb695fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6900643a0 con 0x7fb6900c33f0 2023-11-06T22:18:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.956+0000 7fb695fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900bd8f0 con 0x7fb69014c640 2023-11-06T22:18:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.956+0000 7fb68effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb690152ff0 0x7fb690065160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44412/0 (socket says 172.21.15.33:44412) 2023-11-06T22:18:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.956+0000 7fb68effd700 1 -- 172.21.15.33:0/1154462496 learned_addr learned my addr 172.21.15.33:0/1154462496 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.957+0000 7fb68f7fe700 1 --2- 172.21.15.33:0/1154462496 >> v2:172.21.15.69:3300/0 conn(0x7fb69014c640 0x7fb6900bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.957+0000 7fb68e7fc700 1 -- 172.21.15.33:0/1154462496 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215096130 0 0) 0x7fb690153470 con 0x7fb690152ff0 2023-11-06T22:18:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.957+0000 7fb68e7fc700 1 -- 172.21.15.33:0/1154462496 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6780033f0 con 0x7fb690152ff0 2023-11-06T22:18:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.957+0000 7fb68f7fe700 1 -- 172.21.15.33:0/1154462496 >> v1:172.21.15.69:6789/0 conn(0x7fb6900c33f0 legacy=0x7fb6900c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.957+0000 7fb68e7fc700 1 -- 172.21.15.33:0/1154462496 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743753567 0 0) 0x7fb6900643a0 con 0x7fb6900c33f0 2023-11-06T22:18:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.957+0000 7fb68f7fe700 1 -- 172.21.15.33:0/1154462496 >> v1:172.21.15.33:6789/0 conn(0x7fb690152ff0 legacy=0x7fb690065160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.957+0000 7fb68f7fe700 1 -- 172.21.15.33:0/1154462496 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb690064680 con 0x7fb69014c640 2023-11-06T22:18:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.957+0000 7fb68f7fe700 1 --2- 172.21.15.33:0/1154462496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69014c640 0x7fb6900bb990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fb680009760 tx=0x7fb680009060).ready entity=mon.1 client_cookie=1cfae63cc4596c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.958+0000 7fb68e7fc700 1 -- 172.21.15.33:0/1154462496 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68000f510 con 0x7fb69014c640 2023-11-06T22:18:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.958+0000 7fb68e7fc700 1 -- 172.21.15.33:0/1154462496 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68000f670 con 0x7fb69014c640 2023-11-06T22:18:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.958+0000 7fb68e7fc700 1 -- 172.21.15.33:0/1154462496 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68000f9f0 con 0x7fb69014c640 2023-11-06T22:18:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.958+0000 7fb695fff700 1 -- 172.21.15.33:0/1154462496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69014c640 msgr2=0x7fb6900bb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.958+0000 7fb695fff700 1 --2- 172.21.15.33:0/1154462496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69014c640 0x7fb6900bb990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fb680009760 tx=0x7fb680009060).stop 2023-11-06T22:18:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.958+0000 7fb695fff700 1 -- 172.21.15.33:0/1154462496 shutdown_connections 2023-11-06T22:18:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.958+0000 7fb695fff700 1 --2- 172.21.15.33:0/1154462496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69014c640 0x7fb6900bb990 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.959+0000 7fb695fff700 1 -- 172.21.15.33:0/1154462496 >> 172.21.15.33:0/1154462496 conn(0x7fb690062710 msgr2=0x7fb690062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.959+0000 7fb695fff700 1 -- 172.21.15.33:0/1154462496 shutdown_connections 2023-11-06T22:18:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.959+0000 7fb695fff700 1 -- 172.21.15.33:0/1154462496 wait complete. 2023-11-06T22:18:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.961+0000 7fb695fff700 1 Processor -- start 2023-11-06T22:18:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.961+0000 7fb695fff700 1 -- start start 2023-11-06T22:18:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb695fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6900c33f0 0x7fb6901483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb695fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69014c640 0x7fb69014a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb695fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb690152ff0 0x7fb69014a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb695fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69014adf0 con 0x7fb690152ff0 2023-11-06T22:18:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb695fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900a9b40 con 0x7fb69014c640 2023-11-06T22:18:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb695fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900a9e00 con 0x7fb6900c33f0 2023-11-06T22:18:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb690152ff0 0x7fb69014a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb68ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb690152ff0 0x7fb69014a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37120/0 (socket says 172.21.15.33:37120) 2023-11-06T22:18:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb68ffff700 1 -- 172.21.15.33:0/777829510 learned_addr learned my addr 172.21.15.33:0/777829510 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb68effd700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69014c640 0x7fb69014a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.963+0000 7fb68f7fe700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6900c33f0 0x7fb6901483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:07.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb68ffff700 1 -- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6900c33f0 msgr2=0x7fb6901483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb68ffff700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6900c33f0 0x7fb6901483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb68ffff700 1 -- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69014c640 msgr2=0x7fb69014a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:07.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb68ffff700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69014c640 0x7fb69014a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:07.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb68ffff700 1 -- 172.21.15.33:0/777829510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6800092a0 con 0x7fb690152ff0 2023-11-06T22:18:07.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb68ffff700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb690152ff0 0x7fb69014a900 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7fb688001560 tx=0x7fb688007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:07.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb6777fe700 1 -- 172.21.15.33:0/777829510 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800e4a0 con 0x7fb690152ff0 2023-11-06T22:18:07.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb6777fe700 1 -- 172.21.15.33:0/777829510 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68800e600 con 0x7fb690152ff0 2023-11-06T22:18:07.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb6777fe700 1 -- 172.21.15.33:0/777829510 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800e9a0 con 0x7fb690152ff0 2023-11-06T22:18:07.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb695fff700 1 -- 172.21.15.33:0/777829510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6900aa0e0 con 0x7fb690152ff0 2023-11-06T22:18:07.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.964+0000 7fb695fff700 1 -- 172.21.15.33:0/777829510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6900aa6a0 con 0x7fb690152ff0 2023-11-06T22:18:07.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.966+0000 7fb695fff700 1 -- 172.21.15.33:0/777829510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb690064960 con 0x7fb690152ff0 2023-11-06T22:18:07.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.966+0000 7fb6777fe700 1 -- 172.21.15.33:0/777829510 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fb688012400 con 0x7fb690152ff0 2023-11-06T22:18:07.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.968+0000 7fb6777fe700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb67c05f390 0x7fb67c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:07.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.968+0000 7fb6777fe700 1 -- 172.21.15.33:0/777829510 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb688079eb0 con 0x7fb690152ff0 2023-11-06T22:18:07.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.968+0000 7fb68f7fe700 1 -- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb67c05f390 msgr2=0x7fb67c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:07.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.968+0000 7fb68f7fe700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb67c05f390 0x7fb67c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:07.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:07.971+0000 7fb6777fe700 1 -- 172.21.15.33:0/777829510 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb688048af0 con 0x7fb690152ff0 2023-11-06T22:18:08.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.163+0000 7fb695fff700 1 -- 172.21.15.33:0/777829510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb69005f0b0 con 0x7fb690152ff0 2023-11-06T22:18:08.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.164+0000 7fb6777fe700 1 -- 172.21.15.33:0/777829510 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) v1 ==== 74+0+17711 (secure 0 0 0) 0x7fb68804ac40 con 0x7fb690152ff0 2023-11-06T22:18:08.167 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:08.167 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":209,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:05.275003+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:16:37.603904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T22:16:38.139152+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":209,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.1f","mappings":[{"from":5,"to":3}]},{"pgid":"23.33","mappings":[{"from":5,"to":7}]},{"pgid":"23.3c","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6833/129946":"2023-11-07T22:16:53.444669+0000","172.21.15.69:6832/129946":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/2960476778":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/1246238046":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/4116472923":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/4003747652":"2023-11-07T22:16:53.444669+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 -- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb67c05f390 msgr2=0x7fb67c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb67c05f390 0x7fb67c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 -- 172.21.15.33:0/777829510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb690152ff0 msgr2=0x7fb69014a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb690152ff0 0x7fb69014a900 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7fb688001560 tx=0x7fb688007f40).stop 2023-11-06T22:18:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 -- 172.21.15.33:0/777829510 shutdown_connections 2023-11-06T22:18:08.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb67c05f390 0x7fb67c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:08.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb6900c33f0 0x7fb6901483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb690152ff0 0x7fb69014a900 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 --2- 172.21.15.33:0/777829510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb69014c640 0x7fb69014a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 -- 172.21.15.33:0/777829510 >> 172.21.15.33:0/777829510 conn(0x7fb690062710 msgr2=0x7fb6901501a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 -- 172.21.15.33:0/777829510 shutdown_connections 2023-11-06T22:18:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.166+0000 7fb695fff700 1 -- 172.21.15.33:0/777829510 wait complete. 2023-11-06T22:18:08.189 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T22:18:08.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:08.349 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:08.347+0000 7f48a4a58680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:18:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.425+0000 7f8dc61ff700 1 Processor -- start 2023-11-06T22:18:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.426+0000 7f8dc61ff700 1 -- start start 2023-11-06T22:18:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.427+0000 7f8dc61ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00ad6e0 0x7f8dc0154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.427+0000 7f8dc61ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8dc00c6dc0 0x7f8dc00c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.427+0000 7f8dc61ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc00c5db0 con 0x7f8dc0154280 2023-11-06T22:18:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.427+0000 7f8dc61ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc00c6930 con 0x7f8dc00c6dc0 2023-11-06T22:18:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.427+0000 7f8dc61ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00aa630 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.427+0000 7f8dbf7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8dc0154280 0x7f8dc00a9f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44426/0 (socket says 172.21.15.33:44426) 2023-11-06T22:18:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.427+0000 7f8dbf7fe700 1 -- 172.21.15.33:0/625390444 learned_addr learned my addr 172.21.15.33:0/625390444 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.427+0000 7f8dbeffd700 1 --2- 172.21.15.33:0/625390444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00ad6e0 0x7f8dc0154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.427+0000 7f8dbffff700 1 --2- 172.21.15.33:0/625390444 >> v2:172.21.15.33:3300/0 conn(0x7f8dc00c6dc0 0x7f8dc00c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.428+0000 7f8dbe7fc700 1 -- 172.21.15.33:0/625390444 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669812561 0 0) 0x7f8dc00c5db0 con 0x7f8dc0154280 2023-11-06T22:18:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.428+0000 7f8dbe7fc700 1 -- 172.21.15.33:0/625390444 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8da80033f0 con 0x7f8dc0154280 2023-11-06T22:18:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.428+0000 7f8dbeffd700 1 -- 172.21.15.33:0/625390444 >> v2:172.21.15.33:3300/0 conn(0x7f8dc00c6dc0 msgr2=0x7f8dc00c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.428+0000 7f8dbeffd700 1 --2- 172.21.15.33:0/625390444 >> v2:172.21.15.33:3300/0 conn(0x7f8dc00c6dc0 0x7f8dc00c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.428+0000 7f8dbeffd700 1 -- 172.21.15.33:0/625390444 >> v1:172.21.15.33:6789/0 conn(0x7f8dc0154280 legacy=0x7f8dc00a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.428+0000 7f8dbe7fc700 1 -- 172.21.15.33:0/625390444 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 806902800 0 0) 0x7f8da80033f0 con 0x7f8dc0154280 2023-11-06T22:18:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.428+0000 7f8dbeffd700 1 -- 172.21.15.33:0/625390444 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00c6000 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.428+0000 7f8dbffff700 1 --2- 172.21.15.33:0/625390444 >> v2:172.21.15.33:3300/0 conn(0x7f8dc00c6dc0 0x7f8dc00c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.428+0000 7f8dbeffd700 1 --2- 172.21.15.33:0/625390444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00ad6e0 0x7f8dc0154660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f8db4009760 tx=0x7f8db4009060).ready entity=mon.2 client_cookie=fbd2ec6239f2906b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.429+0000 7f8dbe7fc700 1 -- 172.21.15.33:0/625390444 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0001470 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.429+0000 7f8dbe7fc700 1 -- 172.21.15.33:0/625390444 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db4011400 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.429+0000 7f8dbe7fc700 1 -- 172.21.15.33:0/625390444 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db4011780 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.429+0000 7f8dc61ff700 1 -- 172.21.15.33:0/625390444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00ad6e0 msgr2=0x7f8dc0154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.429+0000 7f8dc61ff700 1 --2- 172.21.15.33:0/625390444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00ad6e0 0x7f8dc0154660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f8db4009760 tx=0x7f8db4009060).stop 2023-11-06T22:18:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.429+0000 7f8dc61ff700 1 -- 172.21.15.33:0/625390444 shutdown_connections 2023-11-06T22:18:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.429+0000 7f8dc61ff700 1 --2- 172.21.15.33:0/625390444 >> v2:172.21.15.33:3300/0 conn(0x7f8dc00c6dc0 0x7f8dc00c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.429+0000 7f8dc61ff700 1 --2- 172.21.15.33:0/625390444 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00ad6e0 0x7f8dc0154660 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.429+0000 7f8dc61ff700 1 -- 172.21.15.33:0/625390444 >> 172.21.15.33:0/625390444 conn(0x7f8dc005e1b0 msgr2=0x7f8dc005e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.429+0000 7f8dc61ff700 1 -- 172.21.15.33:0/625390444 shutdown_connections 2023-11-06T22:18:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.430+0000 7f8dc61ff700 1 -- 172.21.15.33:0/625390444 wait complete. 2023-11-06T22:18:08.438 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:08.436+0000 7f48a4a58680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:18:08.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.431+0000 7f8dc61ff700 1 Processor -- start 2023-11-06T22:18:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.431+0000 7f8dc61ff700 1 -- start start 2023-11-06T22:18:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dc61ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dc00ad6e0 0x7f8dc0143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dc61ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00c6dc0 0x7f8dc01527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dc61ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dc0154280 0x7f8dc00c2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dc61ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc0164750 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dc61ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00af040 con 0x7f8dc0154280 2023-11-06T22:18:08.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dc61ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00aea30 con 0x7f8dc00c6dc0 2023-11-06T22:18:08.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dbf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dc00ad6e0 0x7f8dc0143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:08.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dbf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dc00ad6e0 0x7f8dc0143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37136/0 (socket says 172.21.15.33:37136) 2023-11-06T22:18:08.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dbf7fe700 1 -- 172.21.15.33:0/2439255778 learned_addr learned my addr 172.21.15.33:0/2439255778 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:08.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dbeffd700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00c6dc0 0x7f8dc01527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:08.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.432+0000 7f8dbffff700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dc0154280 0x7f8dc00c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:08.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8dbf7fe700 1 -- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00c6dc0 msgr2=0x7f8dc01527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8dbf7fe700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00c6dc0 0x7f8dc01527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8dbf7fe700 1 -- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dc0154280 msgr2=0x7f8dc00c2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8dbf7fe700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dc0154280 0x7f8dc00c2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8dbf7fe700 1 -- 172.21.15.33:0/2439255778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db8009a20 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8dbffff700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dc0154280 0x7f8dc00c2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:08.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8dbf7fe700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dc00ad6e0 0x7f8dc0143ab0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7f8db000c810 tx=0x7f8db0001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:08.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8da77fe700 1 -- 172.21.15.33:0/2439255778 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0007170 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8da77fe700 1 -- 172.21.15.33:0/2439255778 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db00072d0 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8dc61ff700 1 -- 172.21.15.33:0/2439255778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db40092a0 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.433+0000 7f8da77fe700 1 -- 172.21.15.33:0/2439255778 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00048c0 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.434+0000 7f8dc61ff700 1 -- 172.21.15.33:0/2439255778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc00aecd0 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.435+0000 7f8dc61ff700 1 -- 172.21.15.33:0/2439255778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc00b0e00 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.436+0000 7f8da77fe700 1 -- 172.21.15.33:0/2439255778 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f8db0004c70 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.437+0000 7f8da77fe700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8dac05f390 0x7f8dac061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.437+0000 7f8da77fe700 1 -- 172.21.15.33:0/2439255778 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f8db007a870 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.437+0000 7f8dbeffd700 1 -- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8dac05f390 msgr2=0x7f8dac061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.437+0000 7f8dbeffd700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8dac05f390 0x7f8dac061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.441+0000 7f8da77fe700 1 -- 172.21.15.33:0/2439255778 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8db004b670 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.533 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:08.531+0000 7f48a4a58680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:18:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.638+0000 7f8dbeffd700 1 -- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8dac05f390 msgr2=0x7f8dac061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.638+0000 7f8dbeffd700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8dac05f390 0x7f8dac061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:18:08.662 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:08.660+0000 7f065526c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:18:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.670+0000 7f8dc61ff700 1 -- 172.21.15.33:0/2439255778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dc0062ac0 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.670+0000 7f8da77fe700 1 -- 172.21.15.33:0/2439255778 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f8db004ec90 con 0x7f8dc00ad6e0 2023-11-06T22:18:08.673 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:08.673 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3815920714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3815920714},{"type":"v1","addr":"172.21.15.69:6836","nonce":3815920714}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9052,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3614716748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3614716748},{"type":"v1","addr":"172.21.15.69:6837","nonce":3614716748}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1513187241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1513187241},{"type":"v1","addr":"172.21.15.33:6834","nonce":1513187241}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:16:39.145216+0000","modified":"2023-11-06T22:16:42.180077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3197264665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3197264665},{"type":"v1","addr":"172.21.15.33:6835","nonce":3197264665}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-06T22:18:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.672+0000 7f8dc61ff700 1 -- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8dac05f390 msgr2=0x7f8dac061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.672+0000 7f8dc61ff700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8dac05f390 0x7f8dac061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.672+0000 7f8dc61ff700 1 -- 172.21.15.33:0/2439255778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dc00ad6e0 msgr2=0x7f8dc0143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.672+0000 7f8dc61ff700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dc00ad6e0 0x7f8dc0143ab0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7f8db000c810 tx=0x7f8db0001b50).stop 2023-11-06T22:18:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.673+0000 7f8dc61ff700 1 -- 172.21.15.33:0/2439255778 shutdown_connections 2023-11-06T22:18:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.673+0000 7f8dc61ff700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f8dac05f390 0x7f8dac061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.673+0000 7f8dc61ff700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dc0154280 0x7f8dc00c2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.673+0000 7f8dc61ff700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dc00ad6e0 0x7f8dc0143ab0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.673+0000 7f8dc61ff700 1 --2- 172.21.15.33:0/2439255778 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8dc00c6dc0 0x7f8dc01527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.673+0000 7f8dc61ff700 1 -- 172.21.15.33:0/2439255778 >> 172.21.15.33:0/2439255778 conn(0x7f8dc005e1b0 msgr2=0x7f8dc00aaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.673+0000 7f8dc61ff700 1 -- 172.21.15.33:0/2439255778 shutdown_connections 2023-11-06T22:18:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.673+0000 7f8dc61ff700 1 -- 172.21.15.33:0/2439255778 wait complete. 2023-11-06T22:18:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 84 2023-11-06T22:18:08.693 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:08.718 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:08.717+0000 7f48a4a58680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:18:08.761 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:08.758+0000 7f065526c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:18:08.808 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:08.806+0000 7f48a4a58680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:18:08.863 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:08.860+0000 7f065526c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:18:08.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.888+0000 7fea675ff700 1 Processor -- start 2023-11-06T22:18:08.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.888+0000 7fea675ff700 1 -- start start 2023-11-06T22:18:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea675ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fea600a5080 0x7fea600c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea675ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea600bbfb0 con 0x7fea600bccc0 2023-11-06T22:18:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea675ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea600bdb90 con 0x7fea60157720 2023-11-06T22:18:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea675ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea600bc1e0 con 0x7fea600a5080 2023-11-06T22:18:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea65b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fea600a5080 0x7fea600c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea64b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fea600bccc0 0x7fea600a4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44434/0 (socket says 172.21.15.33:44434) 2023-11-06T22:18:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea64b9a700 1 -- 172.21.15.33:0/2128396558 learned_addr learned my addr 172.21.15.33:0/2128396558 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea65b9c700 1 -- 172.21.15.33:0/2128396558 >> v1:172.21.15.69:6789/0 conn(0x7fea60157720 legacy=0x7fea60158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea57fff700 1 -- 172.21.15.33:0/2128396558 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527881301 0 0) 0x7fea600bdb90 con 0x7fea60157720 2023-11-06T22:18:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea65b9c700 1 -- 172.21.15.33:0/2128396558 >> v1:172.21.15.33:6789/0 conn(0x7fea600bccc0 legacy=0x7fea600a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.890+0000 7fea65b9c700 1 -- 172.21.15.33:0/2128396558 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea600bc4c0 con 0x7fea600a5080 2023-11-06T22:18:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.891+0000 7fea65b9c700 1 --2- 172.21.15.33:0/2128396558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600a5080 0x7fea600c1840 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7fea58009e60 tx=0x7fea58009780).ready entity=mon.0 client_cookie=677bd6d7c8be132f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.891+0000 7fea57fff700 1 -- 172.21.15.33:0/2128396558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5800e070 con 0x7fea600a5080 2023-11-06T22:18:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.891+0000 7fea57fff700 1 -- 172.21.15.33:0/2128396558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5800bc30 con 0x7fea600a5080 2023-11-06T22:18:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.891+0000 7fea57fff700 1 -- 172.21.15.33:0/2128396558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58012410 con 0x7fea600a5080 2023-11-06T22:18:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.891+0000 7fea675ff700 1 -- 172.21.15.33:0/2128396558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600a5080 msgr2=0x7fea600c1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.891+0000 7fea675ff700 1 --2- 172.21.15.33:0/2128396558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600a5080 0x7fea600c1840 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7fea58009e60 tx=0x7fea58009780).stop 2023-11-06T22:18:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.892+0000 7fea675ff700 1 -- 172.21.15.33:0/2128396558 shutdown_connections 2023-11-06T22:18:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.892+0000 7fea675ff700 1 --2- 172.21.15.33:0/2128396558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600a5080 0x7fea600c1840 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.892+0000 7fea675ff700 1 -- 172.21.15.33:0/2128396558 >> 172.21.15.33:0/2128396558 conn(0x7fea600631e0 msgr2=0x7fea600c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.892+0000 7fea675ff700 1 -- 172.21.15.33:0/2128396558 shutdown_connections 2023-11-06T22:18:08.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.892+0000 7fea675ff700 1 -- 172.21.15.33:0/2128396558 wait complete. 2023-11-06T22:18:08.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.893+0000 7fea675ff700 1 Processor -- start 2023-11-06T22:18:08.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.893+0000 7fea675ff700 1 -- start start 2023-11-06T22:18:08.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.894+0000 7fea675ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea600a5080 0x7fea60069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.894+0000 7fea675ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600bccc0 0x7fea6014b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.894+0000 7fea675ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea60157720 0x7fea6014ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.894+0000 7fea675ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6014bef0 con 0x7fea600bccc0 2023-11-06T22:18:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.894+0000 7fea675ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea600ab620 con 0x7fea600a5080 2023-11-06T22:18:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.894+0000 7fea675ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea600ab900 con 0x7fea60157720 2023-11-06T22:18:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea64b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600bccc0 0x7fea6014b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea64b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600bccc0 0x7fea6014b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37148/0 (socket says 172.21.15.33:37148) 2023-11-06T22:18:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea64b9a700 1 -- 172.21.15.33:0/2859849578 learned_addr learned my addr 172.21.15.33:0/2859849578 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea6539b700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea600a5080 0x7fea60069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea65b9c700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea60157720 0x7fea6014ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea64b9a700 1 -- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea60157720 msgr2=0x7fea6014ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea64b9a700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea60157720 0x7fea6014ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea64b9a700 1 -- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea600a5080 msgr2=0x7fea60069840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea64b9a700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea600a5080 0x7fea60069840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea64b9a700 1 -- 172.21.15.33:0/2859849578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58009a20 con 0x7fea600bccc0 2023-11-06T22:18:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.895+0000 7fea64b9a700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600bccc0 0x7fea6014b510 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7fea50007fd0 tx=0x7fea50007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.896+0000 7fea557fa700 1 -- 172.21.15.33:0/2859849578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5000ce10 con 0x7fea600bccc0 2023-11-06T22:18:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.896+0000 7fea557fa700 1 -- 172.21.15.33:0/2859849578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea500072e0 con 0x7fea600bccc0 2023-11-06T22:18:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.896+0000 7fea557fa700 1 -- 172.21.15.33:0/2859849578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50007660 con 0x7fea600bccc0 2023-11-06T22:18:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.896+0000 7fea675ff700 1 -- 172.21.15.33:0/2859849578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea600abbe0 con 0x7fea600bccc0 2023-11-06T22:18:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.896+0000 7fea675ff700 1 -- 172.21.15.33:0/2859849578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea600ac1a0 con 0x7fea600bccc0 2023-11-06T22:18:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.897+0000 7fea675ff700 1 -- 172.21.15.33:0/2859849578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea600ac390 con 0x7fea600bccc0 2023-11-06T22:18:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.898+0000 7fea557fa700 1 -- 172.21.15.33:0/2859849578 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fea500059b0 con 0x7fea600bccc0 2023-11-06T22:18:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.899+0000 7fea557fa700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fea4405f390 0x7fea44061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.899+0000 7fea557fa700 1 -- 172.21.15.33:0/2859849578 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fea50079ea0 con 0x7fea600bccc0 2023-11-06T22:18:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.899+0000 7fea6539b700 1 -- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fea4405f390 msgr2=0x7fea44061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.899+0000 7fea6539b700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fea4405f390 0x7fea44061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:08.902+0000 7fea557fa700 1 -- 172.21.15.33:0/2859849578 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fea5004abe0 con 0x7fea600bccc0 2023-11-06T22:18:08.952 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:08.950+0000 7f48a4a58680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:18:09.065 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:09.062+0000 7f065526c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:18:09.083 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.081+0000 7f48a4a58680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:09.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.100+0000 7fea6539b700 1 -- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fea4405f390 msgr2=0x7fea44061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:09.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.100+0000 7fea6539b700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fea4405f390 0x7fea44061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:18:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.131+0000 7fea675ff700 1 -- 172.21.15.33:0/2859849578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea600bc7a0 con 0x7fea600bccc0 2023-11-06T22:18:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.131+0000 7fea557fa700 1 -- 172.21.15.33:0/2859849578 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4493 (secure 0 0 0) 0x7fea5004e200 con 0x7fea600bccc0 2023-11-06T22:18:09.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:09.134 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3815920714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3815920714},{"type":"v1","addr":"172.21.15.69:6836","nonce":3815920714}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9052,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3614716748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3614716748},{"type":"v1","addr":"172.21.15.69:6837","nonce":3614716748}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1513187241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1513187241},{"type":"v1","addr":"172.21.15.33:6834","nonce":1513187241}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:16:39.145216+0000","modified":"2023-11-06T22:16:42.180077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3197264665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3197264665},{"type":"v1","addr":"172.21.15.33:6835","nonce":3197264665}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-06T22:18:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 -- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fea4405f390 msgr2=0x7fea44061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fea4405f390 0x7fea44061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 -- 172.21.15.33:0/2859849578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600bccc0 msgr2=0x7fea6014b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600bccc0 0x7fea6014b510 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7fea50007fd0 tx=0x7fea50007df0).stop 2023-11-06T22:18:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 -- 172.21.15.33:0/2859849578 shutdown_connections 2023-11-06T22:18:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fea4405f390 0x7fea44061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fea60157720 0x7fea6014ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea600a5080 0x7fea60069840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 --2- 172.21.15.33:0/2859849578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea600bccc0 0x7fea6014b510 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 -- 172.21.15.33:0/2859849578 >> 172.21.15.33:0/2859849578 conn(0x7fea600631e0 msgr2=0x7fea600c3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 -- 172.21.15.33:0/2859849578 shutdown_connections 2023-11-06T22:18:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.134+0000 7fea675ff700 1 -- 172.21.15.33:0/2859849578 wait complete. 2023-11-06T22:18:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 84 2023-11-06T22:18:09.155 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:09.162 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:09.159+0000 7f065526c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:18:09.318 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:09.315+0000 7f065526c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:18:09.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.352+0000 7f63195ff700 1 Processor -- start 2023-11-06T22:18:09.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.353+0000 7f63195ff700 1 -- start start 2023-11-06T22:18:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.354+0000 7f63195ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c1840 0x7f63140a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.354+0000 7f63195ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f63140c6460 0x7f63140bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.354+0000 7f63195ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63141554a0 con 0x7f63140aa2d0 2023-11-06T22:18:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.354+0000 7f63195ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6314154960 con 0x7f63140c6460 2023-11-06T22:18:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.354+0000 7f63195ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6314155930 con 0x7f63140c1840 2023-11-06T22:18:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.354+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c1840 0x7f63140a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.354+0000 7f63137fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f63140c6460 0x7f63140bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.354+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c1840 0x7f63140a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:35166/0 (socket says 172.21.15.33:35166) 2023-11-06T22:18:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.354+0000 7f63127fc700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f63140aa2d0 0x7f63140c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:47474/0 (socket says 172.21.15.33:47474) 2023-11-06T22:18:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.354+0000 7f6312ffd700 1 -- 172.21.15.33:0/125926395 learned_addr learned my addr 172.21.15.33:0/125926395 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f6311ffb700 1 -- 172.21.15.33:0/125926395 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295369028 0 0) 0x7f63141554a0 con 0x7f63140aa2d0 2023-11-06T22:18:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f6311ffb700 1 -- 172.21.15.33:0/125926395 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62fc0033f0 con 0x7f63140aa2d0 2023-11-06T22:18:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f6312ffd700 1 -- 172.21.15.33:0/125926395 >> v2:172.21.15.69:3300/0 conn(0x7f63140c6460 msgr2=0x7f63140bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f6312ffd700 1 --2- 172.21.15.33:0/125926395 >> v2:172.21.15.69:3300/0 conn(0x7f63140c6460 0x7f63140bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f6312ffd700 1 -- 172.21.15.33:0/125926395 >> v1:172.21.15.69:6789/0 conn(0x7f63140aa2d0 legacy=0x7f63140c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f6312ffd700 1 -- 172.21.15.33:0/125926395 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6314154b70 con 0x7f63140c1840 2023-11-06T22:18:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f63137fe700 1 --2- 172.21.15.33:0/125926395 >> v2:172.21.15.69:3300/0 conn(0x7f63140c6460 0x7f63140bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f6312ffd700 1 --2- 172.21.15.33:0/125926395 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c1840 0x7f63140a9de0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f6304006fd0 tx=0x7f63040090f0).ready entity=mon.2 client_cookie=9a4e7a1ec32f93c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f6311ffb700 1 -- 172.21.15.33:0/125926395 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400f510 con 0x7f63140c1840 2023-11-06T22:18:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f6311ffb700 1 -- 172.21.15.33:0/125926395 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630400f670 con 0x7f63140c1840 2023-11-06T22:18:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.355+0000 7f6311ffb700 1 -- 172.21.15.33:0/125926395 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400f9f0 con 0x7f63140c1840 2023-11-06T22:18:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.356+0000 7f63195ff700 1 -- 172.21.15.33:0/125926395 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c1840 msgr2=0x7f63140a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.356+0000 7f63195ff700 1 --2- 172.21.15.33:0/125926395 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c1840 0x7f63140a9de0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f6304006fd0 tx=0x7f63040090f0).stop 2023-11-06T22:18:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.356+0000 7f63195ff700 1 -- 172.21.15.33:0/125926395 shutdown_connections 2023-11-06T22:18:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.356+0000 7f63195ff700 1 --2- 172.21.15.33:0/125926395 >> v2:172.21.15.69:3300/0 conn(0x7f63140c6460 0x7f63140bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.356+0000 7f63195ff700 1 --2- 172.21.15.33:0/125926395 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c1840 0x7f63140a9de0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.356+0000 7f63195ff700 1 -- 172.21.15.33:0/125926395 >> 172.21.15.33:0/125926395 conn(0x7f631405ec80 msgr2=0x7f63140cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.356+0000 7f63195ff700 1 -- 172.21.15.33:0/125926395 shutdown_connections 2023-11-06T22:18:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.356+0000 7f63195ff700 1 -- 172.21.15.33:0/125926395 wait complete. 2023-11-06T22:18:09.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.358+0000 7f63195ff700 1 Processor -- start 2023-11-06T22:18:09.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.358+0000 7f63195ff700 1 -- start start 2023-11-06T22:18:09.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f63195ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63140aa2d0 0x7f631413e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f63195ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63140c1840 0x7f63140aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f63195ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c6460 0x7f63140af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f63195ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63140c2940 con 0x7f63140aa2d0 2023-11-06T22:18:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f63195ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63141509b0 con 0x7f63140c1840 2023-11-06T22:18:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f63195ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6314150c90 con 0x7f63140c6460 2023-11-06T22:18:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63140aa2d0 0x7f631413e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63140aa2d0 0x7f631413e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37156/0 (socket says 172.21.15.33:37156) 2023-11-06T22:18:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f6312ffd700 1 -- 172.21.15.33:0/3722023153 learned_addr learned my addr 172.21.15.33:0/3722023153 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f63127fc700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63140c1840 0x7f63140aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.359+0000 7f63137fe700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c6460 0x7f63140af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f6312ffd700 1 -- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c6460 msgr2=0x7f63140af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f6312ffd700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c6460 0x7f63140af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f6312ffd700 1 -- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63140c1840 msgr2=0x7f63140aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f6312ffd700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63140c1840 0x7f63140aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f6312ffd700 1 -- 172.21.15.33:0/3722023153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c009a20 con 0x7f63140aa2d0 2023-11-06T22:18:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f6312ffd700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63140aa2d0 0x7f631413e9f0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f6314154aa0 tx=0x7f6304017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f62faffd700 1 -- 172.21.15.33:0/3722023153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400fe80 con 0x7f63140aa2d0 2023-11-06T22:18:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f62faffd700 1 -- 172.21.15.33:0/3722023153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630400f7e0 con 0x7f63140aa2d0 2023-11-06T22:18:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f63195ff700 1 -- 172.21.15.33:0/3722023153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6304009370 con 0x7f63140aa2d0 2023-11-06T22:18:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f62faffd700 1 -- 172.21.15.33:0/3722023153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304024680 con 0x7f63140aa2d0 2023-11-06T22:18:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.360+0000 7f63195ff700 1 -- 172.21.15.33:0/3722023153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63140c25d0 con 0x7f63140aa2d0 2023-11-06T22:18:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.361+0000 7f63195ff700 1 -- 172.21.15.33:0/3722023153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631406c6c0 con 0x7f63140aa2d0 2023-11-06T22:18:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.362+0000 7f62faffd700 1 -- 172.21.15.33:0/3722023153 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f6304021070 con 0x7f63140aa2d0 2023-11-06T22:18:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.363+0000 7f62faffd700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f630005f390 0x7f6300061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.363+0000 7f62faffd700 1 -- 172.21.15.33:0/3722023153 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f630407c340 con 0x7f63140aa2d0 2023-11-06T22:18:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.363+0000 7f63127fc700 1 -- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f630005f390 msgr2=0x7f6300061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.363+0000 7f63127fc700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f630005f390 0x7f6300061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:09.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.367+0000 7f62faffd700 1 -- 172.21.15.33:0/3722023153 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f630404d130 con 0x7f63140aa2d0 2023-11-06T22:18:09.432 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.429+0000 7f48a4a58680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:18:09.462 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:09.460+0000 7f065526c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:09.522 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.520+0000 7f48a4a58680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:18:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.530+0000 7f62faffd700 1 -- 172.21.15.33:0/3722023153 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 95) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f6304079b80 con 0x7f63140aa2d0 2023-11-06T22:18:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.530+0000 7f62faffd700 1 -- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f630005f390 msgr2=0x7f6300061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.530+0000 7f62faffd700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f630005f390 0x7f6300061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.556+0000 7f63195ff700 1 -- 172.21.15.33:0/3722023153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63140afb70 con 0x7f63140aa2d0 2023-11-06T22:18:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.557+0000 7f62faffd700 1 -- 172.21.15.33:0/3722023153 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) v1 ==== 74+0+18015 (secure 0 0 0) 0x7f63040557a0 con 0x7f63140aa2d0 2023-11-06T22:18:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:09.561 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":210,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:09.523876+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:16:37.603904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T22:16:38.139152+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":209,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.1f","mappings":[{"from":5,"to":3}]},{"pgid":"23.33","mappings":[{"from":5,"to":7}]},{"pgid":"23.3c","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/4186711226":"2023-11-07T22:18:09.523839+0000","172.21.15.69:0/173906519":"2023-11-07T22:18:09.523839+0000","172.21.15.69:0/2087153184":"2023-11-07T22:18:09.523839+0000","172.21.15.69:0/3301859808":"2023-11-07T22:18:09.523839+0000","172.21.15.69:6833/129946":"2023-11-07T22:16:53.444669+0000","172.21.15.69:6832/129946":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/3940127481":"2023-11-07T22:18:09.523839+0000","172.21.15.69:0/3893096594":"2023-11-07T22:18:09.523839+0000","172.21.15.69:0/2960476778":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/1246238046":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/4116472923":"2023-11-07T22:16:53.444669+0000","172.21.15.69:0/4003747652":"2023-11-07T22:16:53.444669+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.560+0000 7f63195ff700 1 -- 172.21.15.33:0/3722023153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63140aa2d0 msgr2=0x7f631413e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.560+0000 7f63195ff700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63140aa2d0 0x7f631413e9f0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f6314154aa0 tx=0x7f6304017000).stop 2023-11-06T22:18:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.560+0000 7f63195ff700 1 -- 172.21.15.33:0/3722023153 shutdown_connections 2023-11-06T22:18:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.560+0000 7f63195ff700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f630005f390 0x7f6300061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.560+0000 7f63195ff700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f63140c6460 0x7f63140af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.560+0000 7f63195ff700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63140aa2d0 0x7f631413e9f0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.560+0000 7f63195ff700 1 --2- 172.21.15.33:0/3722023153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63140c1840 0x7f63140aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.560+0000 7f63195ff700 1 -- 172.21.15.33:0/3722023153 >> 172.21.15.33:0/3722023153 conn(0x7f631405ec80 msgr2=0x7f6314152bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.560+0000 7f63195ff700 1 -- 172.21.15.33:0/3722023153 shutdown_connections 2023-11-06T22:18:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.560+0000 7f63195ff700 1 -- 172.21.15.33:0/3722023153 wait complete. 2023-11-06T22:18:09.581 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T22:18:09.604 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.602+0000 7f4820553700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:09.605 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.602+0000 7f4820553700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:09.605 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.602+0000 7f4820553700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:09.605 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.602+0000 7f4820553700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:09.605 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.602+0000 7f4820553700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:09.606 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.602+0000 7f481cd4c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:09.606 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.602+0000 7f481cd4c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:09.606 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.602+0000 7f481cd4c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:09.606 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.602+0000 7f481cd4c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:09.607 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:09.602+0000 7f481cd4c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:09.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.821+0000 7ffad91ff700 1 Processor -- start 2023-11-06T22:18:09.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.821+0000 7ffad91ff700 1 -- start start 2023-11-06T22:18:09.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.823+0000 7ffad91ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 0x7ffad40a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.823+0000 7ffad91ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ffad40ca610 0x7ffad40c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.823+0000 7ffad91ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad40c06d0 con 0x7ffad40bfd60 2023-11-06T22:18:09.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.823+0000 7ffad91ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad405efc0 con 0x7ffad40ca610 2023-11-06T22:18:09.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.823+0000 7ffad91ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40c8ea0 con 0x7ffad4154210 2023-11-06T22:18:09.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.823+0000 7ffad259c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7ffad40bfd60 0x7ffad40c9f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:47478/0 (socket says 172.21.15.33:47478) 2023-11-06T22:18:09.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.823+0000 7ffad259c700 1 -- 172.21.15.33:0/7156143 learned_addr learned my addr 172.21.15.33:0/7156143 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:09.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.823+0000 7ffad2d9d700 1 --2- 172.21.15.33:0/7156143 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 0x7ffad40a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:09.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.823+0000 7ffad359e700 1 --2- 172.21.15.33:0/7156143 >> v2:172.21.15.69:3300/0 conn(0x7ffad40ca610 0x7ffad40c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:09.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.823+0000 7ffad1d9b700 1 -- 172.21.15.33:0/7156143 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710203363 0 0) 0x7ffad40c06d0 con 0x7ffad40bfd60 2023-11-06T22:18:09.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.824+0000 7ffad1d9b700 1 -- 172.21.15.33:0/7156143 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffabc0033f0 con 0x7ffad40bfd60 2023-11-06T22:18:09.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.824+0000 7ffad2d9d700 1 -- 172.21.15.33:0/7156143 >> v2:172.21.15.69:3300/0 conn(0x7ffad40ca610 msgr2=0x7ffad40c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:09.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.824+0000 7ffad2d9d700 1 --2- 172.21.15.33:0/7156143 >> v2:172.21.15.69:3300/0 conn(0x7ffad40ca610 0x7ffad40c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.824+0000 7ffad2d9d700 1 -- 172.21.15.33:0/7156143 >> v1:172.21.15.69:6789/0 conn(0x7ffad40bfd60 legacy=0x7ffad40c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:09.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.824+0000 7ffad2d9d700 1 -- 172.21.15.33:0/7156143 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad40c9140 con 0x7ffad4154210 2023-11-06T22:18:09.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.824+0000 7ffad359e700 1 --2- 172.21.15.33:0/7156143 >> v2:172.21.15.69:3300/0 conn(0x7ffad40ca610 0x7ffad40c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:09.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.824+0000 7ffad2d9d700 1 --2- 172.21.15.33:0/7156143 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 0x7ffad40a7bd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7ffac4009760 tx=0x7ffac4009060).ready entity=mon.2 client_cookie=6b92de3d402f2b38 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:09.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.824+0000 7ffad1d9b700 1 -- 172.21.15.33:0/7156143 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400d4a0 con 0x7ffad4154210 2023-11-06T22:18:09.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.824+0000 7ffad1d9b700 1 -- 172.21.15.33:0/7156143 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac400d600 con 0x7ffad4154210 2023-11-06T22:18:09.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.824+0000 7ffad1d9b700 1 -- 172.21.15.33:0/7156143 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400d980 con 0x7ffad4154210 2023-11-06T22:18:09.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.825+0000 7ffad91ff700 1 -- 172.21.15.33:0/7156143 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 msgr2=0x7ffad40a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:09.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.825+0000 7ffad91ff700 1 --2- 172.21.15.33:0/7156143 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 0x7ffad40a7bd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7ffac4009760 tx=0x7ffac4009060).stop 2023-11-06T22:18:09.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.825+0000 7ffad91ff700 1 -- 172.21.15.33:0/7156143 shutdown_connections 2023-11-06T22:18:09.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.825+0000 7ffad91ff700 1 --2- 172.21.15.33:0/7156143 >> v2:172.21.15.69:3300/0 conn(0x7ffad40ca610 0x7ffad40c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.825+0000 7ffad91ff700 1 --2- 172.21.15.33:0/7156143 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 0x7ffad40a7bd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.825+0000 7ffad91ff700 1 -- 172.21.15.33:0/7156143 >> 172.21.15.33:0/7156143 conn(0x7ffad405e990 msgr2=0x7ffad40cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:09.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.825+0000 7ffad91ff700 1 -- 172.21.15.33:0/7156143 shutdown_connections 2023-11-06T22:18:09.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.825+0000 7ffad91ff700 1 -- 172.21.15.33:0/7156143 wait complete. 2023-11-06T22:18:09.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.826+0000 7ffad91ff700 1 Processor -- start 2023-11-06T22:18:09.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.827+0000 7ffad91ff700 1 -- start start 2023-11-06T22:18:09.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad91ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad40bfd60 0x7ffad40addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad91ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad40ca610 0x7ffad41655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad91ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 0x7ffad4165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:09.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad91ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40c81f0 con 0x7ffad40ca610 2023-11-06T22:18:09.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad91ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40ae390 con 0x7ffad40bfd60 2023-11-06T22:18:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad91ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40ae670 con 0x7ffad4154210 2023-11-06T22:18:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad40ca610 0x7ffad41655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad40ca610 0x7ffad41655e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37172/0 (socket says 172.21.15.33:37172) 2023-11-06T22:18:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad259c700 1 -- 172.21.15.33:0/2596230562 learned_addr learned my addr 172.21.15.33:0/2596230562 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad2d9d700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad40bfd60 0x7ffad40addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:09.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.828+0000 7ffad359e700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 0x7ffad4165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:09.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffad259c700 1 -- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 msgr2=0x7ffad4165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:09.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffad259c700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 0x7ffad4165ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffad259c700 1 -- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad40bfd60 msgr2=0x7ffad40addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:09.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffad259c700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad40bfd60 0x7ffad40addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:09.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffad259c700 1 -- 172.21.15.33:0/2596230562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc009a20 con 0x7ffad40ca610 2023-11-06T22:18:09.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffad359e700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 0x7ffad4165ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:09.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffad2d9d700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad40bfd60 0x7ffad40addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:09.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffad259c700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad40ca610 0x7ffad41655e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7ffac800c4c0 tx=0x7ffac8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:09.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffabaffd700 1 -- 172.21.15.33:0/2596230562 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80072e0 con 0x7ffad40ca610 2023-11-06T22:18:09.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffabaffd700 1 -- 172.21.15.33:0/2596230562 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac8007440 con 0x7ffad40ca610 2023-11-06T22:18:09.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffad91ff700 1 -- 172.21.15.33:0/2596230562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac40092a0 con 0x7ffad40ca610 2023-11-06T22:18:09.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffabaffd700 1 -- 172.21.15.33:0/2596230562 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80057a0 con 0x7ffad40ca610 2023-11-06T22:18:09.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.829+0000 7ffad91ff700 1 -- 172.21.15.33:0/2596230562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad40aec30 con 0x7ffad40ca610 2023-11-06T22:18:09.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.831+0000 7ffad91ff700 1 -- 172.21.15.33:0/2596230562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad40b0e90 con 0x7ffad40ca610 2023-11-06T22:18:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.831+0000 7ffabaffd700 1 -- 172.21.15.33:0/2596230562 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 95) v1 ==== 75270+0+0 (secure 0 0 0) 0x7ffac8005bd0 con 0x7ffad40ca610 2023-11-06T22:18:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.832+0000 7ffabaffd700 1 -- 172.21.15.33:0/2596230562 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6796+0+0 (secure 0 0 0) 0x7ffac8079a90 con 0x7ffad40ca610 2023-11-06T22:18:09.842 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:09.834+0000 7f065526c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:18:09.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:09.836+0000 7ffabaffd700 1 -- 172.21.15.33:0/2596230562 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffac804ab80 con 0x7ffad40ca610 2023-11-06T22:18:09.935 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:09.933+0000 7f065526c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:18:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.064+0000 7ffad91ff700 1 -- 172.21.15.33:0/2596230562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ffad406c6c0 con 0x7ffad40ca610 2023-11-06T22:18:10.529 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T22:18:10.527+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T22:18:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.533+0000 7ffabaffd700 1 -- 172.21.15.33:0/2596230562 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 96) v1 ==== 75340+0+0 (secure 0 0 0) 0x7ffac8077680 con 0x7ffad40ca610 2023-11-06T22:18:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.533+0000 7ffabaffd700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffac00302c0 0x7ffac0067d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.534+0000 7ffad2d9d700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffac00302c0 0x7ffac0067d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:10.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.536+0000 7ffad2d9d700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffac00302c0 0x7ffac0067d80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7ffac4009760 tx=0x7ffac4016000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.544+0000 7ffabaffd700 1 -- 172.21.15.33:0/2596230562 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v85) v1 ==== 161+0+0 (secure 0 0 0) 0x7ffac806f7a0 con 0x7ffad40ca610 2023-11-06T22:18:10.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 -- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffac00302c0 msgr2=0x7ffac0067d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffac00302c0 0x7ffac0067d80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7ffac4009760 tx=0x7ffac4016000).stop 2023-11-06T22:18:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 -- 172.21.15.33:0/2596230562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad40ca610 msgr2=0x7ffad41655e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad40ca610 0x7ffad41655e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7ffac800c4c0 tx=0x7ffac8007c50).stop 2023-11-06T22:18:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 -- 172.21.15.33:0/2596230562 shutdown_connections 2023-11-06T22:18:10.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffac00302c0 0x7ffac0067d80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:10.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffad4154210 0x7ffad4165ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:10.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffad40bfd60 0x7ffad40addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:10.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 --2- 172.21.15.33:0/2596230562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad40ca610 0x7ffad41655e0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:10.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 -- 172.21.15.33:0/2596230562 >> 172.21.15.33:0/2596230562 conn(0x7ffad405e990 msgr2=0x7ffad40c6240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:10.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 -- 172.21.15.33:0/2596230562 shutdown_connections 2023-11-06T22:18:10.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.547+0000 7ffad91ff700 1 -- 172.21.15.33:0/2596230562 wait complete. 2023-11-06T22:18:10.553 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T22:18:10.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T22:18:10.612 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T22:18:10.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.771+0000 7f574abff700 1 Processor -- start 2023-11-06T22:18:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.771+0000 7f574abff700 1 -- start start 2023-11-06T22:18:10.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.772+0000 7f574abff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5744157700 0x7f57440c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:10.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.772+0000 7f574abff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5744158c00 con 0x7f5744063840 2023-11-06T22:18:10.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.772+0000 7f574abff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5744157ff0 con 0x7f57440aaf50 2023-11-06T22:18:10.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.772+0000 7f574abff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57440c18f0 con 0x7f5744157700 2023-11-06T22:18:10.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.773+0000 7f574899b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5744157700 0x7f57440c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:10.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.773+0000 7f574919c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5744063840 0x7f57440c5d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44440/0 (socket says 172.21.15.33:44440) 2023-11-06T22:18:10.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.773+0000 7f574919c700 1 -- 172.21.15.33:0/4155371646 learned_addr learned my addr 172.21.15.33:0/4155371646 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.773+0000 7f574899b700 1 -- 172.21.15.33:0/4155371646 >> v1:172.21.15.69:6789/0 conn(0x7f57440aaf50 legacy=0x7f5744063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.773+0000 7f574899b700 1 -- 172.21.15.33:0/4155371646 >> v1:172.21.15.33:6789/0 conn(0x7f5744063840 legacy=0x7f57440c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.773+0000 7f574899b700 1 -- 172.21.15.33:0/4155371646 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57441582d0 con 0x7f5744157700 2023-11-06T22:18:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.773+0000 7f57437fe700 1 -- 172.21.15.33:0/4155371646 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663436850 0 0) 0x7f5744158c00 con 0x7f5744063840 2023-11-06T22:18:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.773+0000 7f574899b700 1 --2- 172.21.15.33:0/4155371646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 0x7f57440c6f00 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7f5738009760 tx=0x7f5738009060).ready entity=mon.0 client_cookie=6010cf8d5f12d08a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.773+0000 7f57437fe700 1 -- 172.21.15.33:0/4155371646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800b580 con 0x7f5744157700 2023-11-06T22:18:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.774+0000 7f57437fe700 1 -- 172.21.15.33:0/4155371646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573800b6e0 con 0x7f5744157700 2023-11-06T22:18:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.774+0000 7f57437fe700 1 -- 172.21.15.33:0/4155371646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800ba60 con 0x7f5744157700 2023-11-06T22:18:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.774+0000 7f574abff700 1 -- 172.21.15.33:0/4155371646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 msgr2=0x7f57440c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.774+0000 7f574abff700 1 --2- 172.21.15.33:0/4155371646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 0x7f57440c6f00 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7f5738009760 tx=0x7f5738009060).stop 2023-11-06T22:18:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.774+0000 7f574abff700 1 -- 172.21.15.33:0/4155371646 shutdown_connections 2023-11-06T22:18:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.774+0000 7f574abff700 1 --2- 172.21.15.33:0/4155371646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 0x7f57440c6f00 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.774+0000 7f574abff700 1 -- 172.21.15.33:0/4155371646 >> 172.21.15.33:0/4155371646 conn(0x7f574405ec80 msgr2=0x7f57440cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.774+0000 7f574abff700 1 -- 172.21.15.33:0/4155371646 shutdown_connections 2023-11-06T22:18:10.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.774+0000 7f574abff700 1 -- 172.21.15.33:0/4155371646 wait complete. 2023-11-06T22:18:10.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.776+0000 7f574abff700 1 Processor -- start 2023-11-06T22:18:10.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.776+0000 7f574abff700 1 -- start start 2023-11-06T22:18:10.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f574abff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744063840 0x7f574414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:10.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f574abff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57440aaf50 0x7f57440af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:10.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f574abff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 0x7f57440ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:10.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f574abff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57440c2400 con 0x7f5744157700 2023-11-06T22:18:10.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f574abff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57440b0840 con 0x7f5744063840 2023-11-06T22:18:10.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f574abff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57440b0280 con 0x7f57440aaf50 2023-11-06T22:18:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f574919c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 0x7f57440ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f574919c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 0x7f57440ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37208/0 (socket says 172.21.15.33:37208) 2023-11-06T22:18:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f574919c700 1 -- 172.21.15.33:0/3569067758 learned_addr learned my addr 172.21.15.33:0/3569067758 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f574899b700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744063840 0x7f574414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:10.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.777+0000 7f5743fff700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57440aaf50 0x7f57440af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:10.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f574919c700 1 -- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57440aaf50 msgr2=0x7f57440af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:10.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f574919c700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57440aaf50 0x7f57440af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:10.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f574919c700 1 -- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744063840 msgr2=0x7f574414e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:10.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f574919c700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744063840 0x7f574414e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:10.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f574919c700 1 -- 172.21.15.33:0/3569067758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57380092a0 con 0x7f5744157700 2023-11-06T22:18:10.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f574899b700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744063840 0x7f574414e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:10.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f574919c700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 0x7f57440ade00 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f573c00c810 tx=0x7f573c0014d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:10.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f5740ff9700 1 -- 172.21.15.33:0/3569067758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c005d30 con 0x7f5744157700 2023-11-06T22:18:10.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f5740ff9700 1 -- 172.21.15.33:0/3569067758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573c005e90 con 0x7f5744157700 2023-11-06T22:18:10.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f5740ff9700 1 -- 172.21.15.33:0/3569067758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c007340 con 0x7f5744157700 2023-11-06T22:18:10.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f574abff700 1 -- 172.21.15.33:0/3569067758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57440b0560 con 0x7f5744157700 2023-11-06T22:18:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.778+0000 7f574abff700 1 -- 172.21.15.33:0/3569067758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574414e990 con 0x7f5744157700 2023-11-06T22:18:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.779+0000 7f574abff700 1 -- 172.21.15.33:0/3569067758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57440c1ef0 con 0x7f5744157700 2023-11-06T22:18:10.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.780+0000 7f5740ff9700 1 -- 172.21.15.33:0/3569067758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 96) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f573c007730 con 0x7f5744157700 2023-11-06T22:18:10.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.781+0000 7f5740ff9700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f572805f030 0x7f57280614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:10.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.781+0000 7f5740ff9700 1 -- 172.21.15.33:0/3569067758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f573c07a3f0 con 0x7f5744157700 2023-11-06T22:18:10.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.782+0000 7f574899b700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f572805f030 0x7f57280614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:10.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.782+0000 7f574899b700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f572805f030 0x7f57280614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f5738006e00 tx=0x7f5738017000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:10.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:10.785+0000 7f5740ff9700 1 -- 172.21.15.33:0/3569067758 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f57440c1ef0 con 0x7f5744157700 2023-11-06T22:18:11.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.009+0000 7f574abff700 1 -- 172.21.15.33:0/3569067758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f57440c1ef0 con 0x7f5744157700 2023-11-06T22:18:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.546+0000 7f5740ff9700 1 -- 172.21.15.33:0/3569067758 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v86) v1 ==== 104+0+0 (secure 0 0 0) 0x7f57440c1ef0 con 0x7f5744157700 2023-11-06T22:18:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 -- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f572805f030 msgr2=0x7f57280614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f572805f030 0x7f57280614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f5738006e00 tx=0x7f5738017000).stop 2023-11-06T22:18:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 -- 172.21.15.33:0/3569067758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 msgr2=0x7f57440ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 0x7f57440ade00 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f573c00c810 tx=0x7f573c0014d0).stop 2023-11-06T22:18:11.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 -- 172.21.15.33:0/3569067758 shutdown_connections 2023-11-06T22:18:11.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f572805f030 0x7f57280614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:11.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5744157700 0x7f57440ade00 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:11.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744063840 0x7f574414e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:11.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 --2- 172.21.15.33:0/3569067758 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f57440aaf50 0x7f57440af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 -- 172.21.15.33:0/3569067758 >> 172.21.15.33:0/3569067758 conn(0x7f574405ec80 msgr2=0x7f57440c5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 -- 172.21.15.33:0/3569067758 shutdown_connections 2023-11-06T22:18:11.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.549+0000 7f574abff700 1 -- 172.21.15.33:0/3569067758 wait complete. 2023-11-06T22:18:11.573 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T22:18:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.812+0000 7f01cedff700 1 Processor -- start 2023-11-06T22:18:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.812+0000 7f01cedff700 1 -- start start 2023-11-06T22:18:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.813+0000 7f01cedff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01c80c9340 0x7f01c8155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.813+0000 7f01cedff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c8063950 con 0x7f01c813a960 2023-11-06T22:18:11.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.813+0000 7f01cedff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c8063c30 con 0x7f01c8066ba0 2023-11-06T22:18:11.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.813+0000 7f01cedff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c8064cc0 con 0x7f01c80c9340 2023-11-06T22:18:11.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.814+0000 7f01cd39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01c80c9340 0x7f01c8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:11.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.814+0000 7f01ccb9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f01c813a960 0x7f01c8066490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44450/0 (socket says 172.21.15.33:44450) 2023-11-06T22:18:11.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.814+0000 7f01ccb9b700 1 -- 172.21.15.33:0/2632976022 learned_addr learned my addr 172.21.15.33:0/2632976022 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:11.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.814+0000 7f01c77fe700 1 -- 172.21.15.33:0/2632976022 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294319581 0 0) 0x7f01c8063950 con 0x7f01c813a960 2023-11-06T22:18:11.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.814+0000 7f01cd39c700 1 -- 172.21.15.33:0/2632976022 >> v1:172.21.15.69:6789/0 conn(0x7f01c8066ba0 legacy=0x7f01c80c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:11.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.814+0000 7f01cd39c700 1 -- 172.21.15.33:0/2632976022 >> v1:172.21.15.33:6789/0 conn(0x7f01c813a960 legacy=0x7f01c8066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:11.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.814+0000 7f01cd39c700 1 -- 172.21.15.33:0/2632976022 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c813ad40 con 0x7f01c80c9340 2023-11-06T22:18:11.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.815+0000 7f01cd39c700 1 --2- 172.21.15.33:0/2632976022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c80c9340 0x7f01c8155380 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f01c0009e60 tx=0x7f01c0009780).ready entity=mon.0 client_cookie=796c18f26a051264 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:11.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.815+0000 7f01c77fe700 1 -- 172.21.15.33:0/2632976022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c000e070 con 0x7f01c80c9340 2023-11-06T22:18:11.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.815+0000 7f01c77fe700 1 -- 172.21.15.33:0/2632976022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c000bc30 con 0x7f01c80c9340 2023-11-06T22:18:11.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.815+0000 7f01c77fe700 1 -- 172.21.15.33:0/2632976022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0012410 con 0x7f01c80c9340 2023-11-06T22:18:11.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.815+0000 7f01cedff700 1 -- 172.21.15.33:0/2632976022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c80c9340 msgr2=0x7f01c8155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:11.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.815+0000 7f01cedff700 1 --2- 172.21.15.33:0/2632976022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c80c9340 0x7f01c8155380 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f01c0009e60 tx=0x7f01c0009780).stop 2023-11-06T22:18:11.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.815+0000 7f01cedff700 1 -- 172.21.15.33:0/2632976022 shutdown_connections 2023-11-06T22:18:11.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.815+0000 7f01cedff700 1 --2- 172.21.15.33:0/2632976022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c80c9340 0x7f01c8155380 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:11.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.815+0000 7f01cedff700 1 -- 172.21.15.33:0/2632976022 >> 172.21.15.33:0/2632976022 conn(0x7f01c805ec70 msgr2=0x7f01c80c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:11.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.815+0000 7f01cedff700 1 -- 172.21.15.33:0/2632976022 shutdown_connections 2023-11-06T22:18:11.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.816+0000 7f01cedff700 1 -- 172.21.15.33:0/2632976022 wait complete. 2023-11-06T22:18:11.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.817+0000 7f01cedff700 1 Processor -- start 2023-11-06T22:18:11.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.817+0000 7f01cedff700 1 -- start start 2023-11-06T22:18:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.818+0000 7f01cedff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8066ba0 0x7f01c80ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.818+0000 7f01cedff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c80c9340 0x7f01c80c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.818+0000 7f01cedff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c813a960 0x7f01c81397e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.818+0000 7f01cedff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c8155930 con 0x7f01c8066ba0 2023-11-06T22:18:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.818+0000 7f01cedff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80aebe0 con 0x7f01c813a960 2023-11-06T22:18:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.818+0000 7f01cedff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80abbc0 con 0x7f01c80c9340 2023-11-06T22:18:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.818+0000 7f01ccb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8066ba0 0x7f01c80ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.818+0000 7f01ccb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8066ba0 0x7f01c80ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37216/0 (socket says 172.21.15.33:37216) 2023-11-06T22:18:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.818+0000 7f01ccb9b700 1 -- 172.21.15.33:0/2050163055 learned_addr learned my addr 172.21.15.33:0/2050163055 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:11.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01cd39c700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c813a960 0x7f01c81397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:11.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01c7fff700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c80c9340 0x7f01c80c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:11.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01ccb9b700 1 -- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c80c9340 msgr2=0x7f01c80c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:11.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01ccb9b700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c80c9340 0x7f01c80c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:11.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01ccb9b700 1 -- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c813a960 msgr2=0x7f01c81397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01ccb9b700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c813a960 0x7f01c81397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01ccb9b700 1 -- 172.21.15.33:0/2050163055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c0009a20 con 0x7f01c8066ba0 2023-11-06T22:18:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01cd39c700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c813a960 0x7f01c81397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01ccb9b700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8066ba0 0x7f01c80ae6f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7f01bc00c4c0 tx=0x7f01bc007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01c4ff9700 1 -- 172.21.15.33:0/2050163055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0072e0 con 0x7f01c8066ba0 2023-11-06T22:18:11.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01c4ff9700 1 -- 172.21.15.33:0/2050163055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc007460 con 0x7f01c8066ba0 2023-11-06T22:18:11.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01c4ff9700 1 -- 172.21.15.33:0/2050163055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0059f0 con 0x7f01c8066ba0 2023-11-06T22:18:11.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.819+0000 7f01cedff700 1 -- 172.21.15.33:0/2050163055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c80ac180 con 0x7f01c8066ba0 2023-11-06T22:18:11.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.820+0000 7f01cedff700 1 -- 172.21.15.33:0/2050163055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c80c7d00 con 0x7f01c8066ba0 2023-11-06T22:18:11.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.821+0000 7f01cedff700 1 -- 172.21.15.33:0/2050163055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c8065f30 con 0x7f01c8066ba0 2023-11-06T22:18:11.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.822+0000 7f01c4ff9700 1 -- 172.21.15.33:0/2050163055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 96) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f01bc013450 con 0x7f01c8066ba0 2023-11-06T22:18:11.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.823+0000 7f01c4ff9700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01ac05f030 0x7f01ac0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.823+0000 7f01c4ff9700 1 -- 172.21.15.33:0/2050163055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f01bc079a60 con 0x7f01c8066ba0 2023-11-06T22:18:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.824+0000 7f01c7fff700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01ac05f030 0x7f01ac0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.824+0000 7f01c7fff700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01ac05f030 0x7f01ac0614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f01c81679c0 tx=0x7f01b4009040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:11.827+0000 7f01c4ff9700 1 -- 172.21.15.33:0/2050163055 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f01bc04aa90 con 0x7f01c8066ba0 2023-11-06T22:18:12.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.005+0000 7f01cedff700 1 -- 172.21.15.33:0/2050163055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01c80be920 con 0x7f01c8066ba0 2023-11-06T22:18:12.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.303+0000 7f01c4ff9700 1 -- 172.21.15.33:0/2050163055 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f01bc0775f0 con 0x7f01c8066ba0 2023-11-06T22:18:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.552+0000 7f01c4ff9700 1 -- 172.21.15.33:0/2050163055 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v213) v1 ==== 181+0+0 (secure 0 0 0) 0x7f01bc073330 con 0x7f01c8066ba0 2023-11-06T22:18:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 -- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01ac05f030 msgr2=0x7f01ac0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01ac05f030 0x7f01ac0614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f01c81679c0 tx=0x7f01b4009040).stop 2023-11-06T22:18:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 -- 172.21.15.33:0/2050163055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8066ba0 msgr2=0x7f01c80ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8066ba0 0x7f01c80ae6f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7f01bc00c4c0 tx=0x7f01bc007e40).stop 2023-11-06T22:18:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 -- 172.21.15.33:0/2050163055 shutdown_connections 2023-11-06T22:18:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f01ac05f030 0x7f01ac0614f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c813a960 0x7f01c81397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8066ba0 0x7f01c80ae6f0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 --2- 172.21.15.33:0/2050163055 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f01c80c9340 0x7f01c80c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 -- 172.21.15.33:0/2050163055 >> 172.21.15.33:0/2050163055 conn(0x7f01c805ec70 msgr2=0x7f01c80c0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 -- 172.21.15.33:0/2050163055 shutdown_connections 2023-11-06T22:18:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.554+0000 7f01cedff700 1 -- 172.21.15.33:0/2050163055 wait complete. 2023-11-06T22:18:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T22:18:12.577 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T22:18:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.774+0000 7f54f81ff700 1 Processor -- start 2023-11-06T22:18:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.775+0000 7f54f81ff700 1 -- start start 2023-11-06T22:18:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.776+0000 7f54f81ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f54f0154340 0x7f54f00aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.776+0000 7f54f81ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f54f0154720 0x7f54f0158010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.776+0000 7f54f81ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54f0158500 0x7f54f0063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.776+0000 7f54f81ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f0158960 con 0x7f54f0154340 2023-11-06T22:18:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.776+0000 7f54f81ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f00c0e00 con 0x7f54f0154720 2023-11-06T22:18:12.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.776+0000 7f54f81ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f0158bf0 con 0x7f54f0158500 2023-11-06T22:18:12.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.776+0000 7f54f5f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f54f0154340 0x7f54f00aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:12.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.776+0000 7f54f5f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f54f0154340 0x7f54f00aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37228/0 (socket says 172.21.15.33:37228) 2023-11-06T22:18:12.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.776+0000 7f54f5f9b700 1 -- 172.21.15.33:0/2917749665 learned_addr learned my addr 172.21.15.33:0/2917749665 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:12.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f679c700 1 --2- 172.21.15.33:0/2917749665 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54f0158500 0x7f54f0063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:12.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f579a700 1 --2- 172.21.15.33:0/2917749665 >> v2:172.21.15.69:3300/0 conn(0x7f54f0154720 0x7f54f0158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:12.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f5f9b700 1 -- 172.21.15.33:0/2917749665 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54f0158500 msgr2=0x7f54f0063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:12.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f5f9b700 1 --2- 172.21.15.33:0/2917749665 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54f0158500 0x7f54f0063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f5f9b700 1 -- 172.21.15.33:0/2917749665 >> v2:172.21.15.69:3300/0 conn(0x7f54f0154720 msgr2=0x7f54f0158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:12.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f5f9b700 1 --2- 172.21.15.33:0/2917749665 >> v2:172.21.15.69:3300/0 conn(0x7f54f0154720 0x7f54f0158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f5f9b700 1 -- 172.21.15.33:0/2917749665 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f0158ed0 con 0x7f54f0154340 2023-11-06T22:18:12.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f579a700 1 --2- 172.21.15.33:0/2917749665 >> v2:172.21.15.69:3300/0 conn(0x7f54f0154720 0x7f54f0158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:12.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f5f9b700 1 --2- 172.21.15.33:0/2917749665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 0x7f54f00aae40 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7f54f005f0c0 tx=0x7f54ec0090f0).ready entity=mon.0 client_cookie=739fb0f9a3ba9377 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:12.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f4f99700 1 -- 172.21.15.33:0/2917749665 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00b600 con 0x7f54f0154340 2023-11-06T22:18:12.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f4f99700 1 -- 172.21.15.33:0/2917749665 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec00b760 con 0x7f54f0154340 2023-11-06T22:18:12.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.777+0000 7f54f4f99700 1 -- 172.21.15.33:0/2917749665 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00bae0 con 0x7f54f0154340 2023-11-06T22:18:12.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.778+0000 7f54f81ff700 1 -- 172.21.15.33:0/2917749665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 msgr2=0x7f54f00aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:12.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.778+0000 7f54f81ff700 1 --2- 172.21.15.33:0/2917749665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 0x7f54f00aae40 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7f54f005f0c0 tx=0x7f54ec0090f0).stop 2023-11-06T22:18:12.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.778+0000 7f54f81ff700 1 -- 172.21.15.33:0/2917749665 shutdown_connections 2023-11-06T22:18:12.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.778+0000 7f54f81ff700 1 --2- 172.21.15.33:0/2917749665 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54f0158500 0x7f54f0063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.778+0000 7f54f81ff700 1 --2- 172.21.15.33:0/2917749665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 0x7f54f00aae40 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.778+0000 7f54f81ff700 1 --2- 172.21.15.33:0/2917749665 >> v2:172.21.15.69:3300/0 conn(0x7f54f0154720 0x7f54f0158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.778+0000 7f54f81ff700 1 -- 172.21.15.33:0/2917749665 >> 172.21.15.33:0/2917749665 conn(0x7f54f005ec80 msgr2=0x7f54f00cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:12.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.778+0000 7f54f81ff700 1 -- 172.21.15.33:0/2917749665 shutdown_connections 2023-11-06T22:18:12.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.778+0000 7f54f81ff700 1 -- 172.21.15.33:0/2917749665 wait complete. 2023-11-06T22:18:12.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.779+0000 7f54f81ff700 1 Processor -- start 2023-11-06T22:18:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.780+0000 7f54f81ff700 1 -- start start 2023-11-06T22:18:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.780+0000 7f54f81ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 0x7f54f00c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.780+0000 7f54f81ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54f0154720 0x7f54f0153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.780+0000 7f54f81ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54f0158500 0x7f54f00ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.780+0000 7f54f81ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f0155680 con 0x7f54f0154340 2023-11-06T22:18:12.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.780+0000 7f54f81ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00ca4b0 con 0x7f54f0154720 2023-11-06T22:18:12.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.780+0000 7f54f81ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00ca790 con 0x7f54f0158500 2023-11-06T22:18:12.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f5f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 0x7f54f00c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:12.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f5f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 0x7f54f00c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37236/0 (socket says 172.21.15.33:37236) 2023-11-06T22:18:12.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f5f9b700 1 -- 172.21.15.33:0/1616972622 learned_addr learned my addr 172.21.15.33:0/1616972622 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f679c700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54f0158500 0x7f54f00ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f579a700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54f0154720 0x7f54f0153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f5f9b700 1 -- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54f0158500 msgr2=0x7f54f00ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f5f9b700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54f0158500 0x7f54f00ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f5f9b700 1 -- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54f0154720 msgr2=0x7f54f0153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:12.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f5f9b700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54f0154720 0x7f54f0153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:12.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f5f9b700 1 -- 172.21.15.33:0/1616972622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec009370 con 0x7f54f0154340 2023-11-06T22:18:12.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f579a700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54f0154720 0x7f54f0153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:12.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.781+0000 7f54f5f9b700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 0x7f54f00c9e90 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f54ec009130 tx=0x7f54ec017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:12.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.782+0000 7f54e5ffb700 1 -- 172.21.15.33:0/1616972622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00b8d0 con 0x7f54f0154340 2023-11-06T22:18:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.782+0000 7f54e5ffb700 1 -- 172.21.15.33:0/1616972622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec00ba30 con 0x7f54f0154340 2023-11-06T22:18:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.782+0000 7f54e5ffb700 1 -- 172.21.15.33:0/1616972622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec01c670 con 0x7f54f0154340 2023-11-06T22:18:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.782+0000 7f54f81ff700 1 -- 172.21.15.33:0/1616972622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f00caa70 con 0x7f54f0154340 2023-11-06T22:18:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.782+0000 7f54f81ff700 1 -- 172.21.15.33:0/1616972622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f0153610 con 0x7f54f0154340 2023-11-06T22:18:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.783+0000 7f54f81ff700 1 -- 172.21.15.33:0/1616972622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f00c5d80 con 0x7f54f0154340 2023-11-06T22:18:12.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.784+0000 7f54e5ffb700 1 -- 172.21.15.33:0/1616972622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f54ec01ca20 con 0x7f54f0154340 2023-11-06T22:18:12.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.785+0000 7f54e5ffb700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f54d405f270 0x7f54d4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:12.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.785+0000 7f54e5ffb700 1 -- 172.21.15.33:0/1616972622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6357+0+0 (secure 0 0 0) 0x7f54ec07bd80 con 0x7f54f0154340 2023-11-06T22:18:12.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.785+0000 7f54f579a700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f54d405f270 0x7f54d4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.786+0000 7f54f579a700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f54d405f270 0x7f54d4061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f54f00ae550 tx=0x7f54e0008040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.789+0000 7f54e5ffb700 1 -- 172.21.15.33:0/1616972622 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f54ec04cd20 con 0x7f54f0154340 2023-11-06T22:18:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:12.972+0000 7f54f81ff700 1 -- 172.21.15.33:0/1616972622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f54f00cbcf0 con 0x7f54f0154340 2023-11-06T22:18:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.272+0000 7f54e5ffb700 1 -- 172.21.15.33:0/1616972622 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v214) v1 ==== 169+0+0 (secure 0 0 0) 0x7f54ec020020 con 0x7f54f0154340 2023-11-06T22:18:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.275+0000 7f54f81ff700 1 -- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f54d405f270 msgr2=0x7f54d4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.275+0000 7f54f81ff700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f54d405f270 0x7f54d4061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f54f00ae550 tx=0x7f54e0008040).stop 2023-11-06T22:18:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.275+0000 7f54f81ff700 1 -- 172.21.15.33:0/1616972622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 msgr2=0x7f54f00c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.275+0000 7f54f81ff700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 0x7f54f00c9e90 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f54ec009130 tx=0x7f54ec017000).stop 2023-11-06T22:18:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.276+0000 7f54f81ff700 1 -- 172.21.15.33:0/1616972622 shutdown_connections 2023-11-06T22:18:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.276+0000 7f54f81ff700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f54d405f270 0x7f54d4061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.276+0000 7f54f81ff700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f54f0158500 0x7f54f00ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.276+0000 7f54f81ff700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f0154340 0x7f54f00c9e90 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.276+0000 7f54f81ff700 1 --2- 172.21.15.33:0/1616972622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54f0154720 0x7f54f0153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.276+0000 7f54f81ff700 1 -- 172.21.15.33:0/1616972622 >> 172.21.15.33:0/1616972622 conn(0x7f54f005ec80 msgr2=0x7f54f014e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.276+0000 7f54f81ff700 1 -- 172.21.15.33:0/1616972622 shutdown_connections 2023-11-06T22:18:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.276+0000 7f54f81ff700 1 -- 172.21.15.33:0/1616972622 wait complete. 2023-11-06T22:18:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T22:18:13.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T22:18:13.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.493+0000 7f43bedff700 1 Processor -- start 2023-11-06T22:18:13.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.493+0000 7f43bedff700 1 -- start start 2023-11-06T22:18:13.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43bedff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b8157fa0 0x7f43b80630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43bedff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43b8158420 0x7f43b80635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43bedff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b8152470 con 0x7f43b80c72c0 2023-11-06T22:18:13.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43bedff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b8063ac0 con 0x7f43b8158420 2023-11-06T22:18:13.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43bedff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8154990 con 0x7f43b8157fa0 2023-11-06T22:18:13.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43bcb9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f43b80c72c0 0x7f43b81553e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44458/0 (socket says 172.21.15.33:44458) 2023-11-06T22:18:13.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43bcb9b700 1 -- 172.21.15.33:0/187188747 learned_addr learned my addr 172.21.15.33:0/187188747 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:13.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43bd39c700 1 --2- 172.21.15.33:0/187188747 >> v2:172.21.15.33:3300/0 conn(0x7f43b8158420 0x7f43b80635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43b7fff700 1 --2- 172.21.15.33:0/187188747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b8157fa0 0x7f43b80630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43bd39c700 1 -- 172.21.15.33:0/187188747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b8157fa0 msgr2=0x7f43b80630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:13.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.495+0000 7f43bd39c700 1 --2- 172.21.15.33:0/187188747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b8157fa0 0x7f43b80630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.496+0000 7f43bd39c700 1 -- 172.21.15.33:0/187188747 >> v1:172.21.15.33:6789/0 conn(0x7f43b80c72c0 legacy=0x7f43b81553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.496+0000 7f43bd39c700 1 -- 172.21.15.33:0/187188747 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b80c76a0 con 0x7f43b8158420 2023-11-06T22:18:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.496+0000 7f43b77fe700 1 -- 172.21.15.33:0/187188747 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406998864 0 0) 0x7f43b8152470 con 0x7f43b80c72c0 2023-11-06T22:18:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.496+0000 7f43bd39c700 1 --2- 172.21.15.33:0/187188747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b8158420 0x7f43b80635d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f43b0009e60 tx=0x7f43b0009780).ready entity=mon.0 client_cookie=817fda1e5b94b70c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.496+0000 7f43b77fe700 1 -- 172.21.15.33:0/187188747 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000e070 con 0x7f43b8158420 2023-11-06T22:18:13.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.496+0000 7f43b77fe700 1 -- 172.21.15.33:0/187188747 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b000bc30 con 0x7f43b8158420 2023-11-06T22:18:13.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.496+0000 7f43b77fe700 1 -- 172.21.15.33:0/187188747 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0012410 con 0x7f43b8158420 2023-11-06T22:18:13.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.496+0000 7f43bedff700 1 -- 172.21.15.33:0/187188747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b8158420 msgr2=0x7f43b80635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.496+0000 7f43bedff700 1 --2- 172.21.15.33:0/187188747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b8158420 0x7f43b80635d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f43b0009e60 tx=0x7f43b0009780).stop 2023-11-06T22:18:13.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.497+0000 7f43bedff700 1 -- 172.21.15.33:0/187188747 shutdown_connections 2023-11-06T22:18:13.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.497+0000 7f43bedff700 1 --2- 172.21.15.33:0/187188747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b8158420 0x7f43b80635d0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.497+0000 7f43bedff700 1 --2- 172.21.15.33:0/187188747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b8157fa0 0x7f43b80630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.497+0000 7f43bedff700 1 -- 172.21.15.33:0/187188747 >> 172.21.15.33:0/187188747 conn(0x7f43b805ec20 msgr2=0x7f43b80cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:13.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.497+0000 7f43bedff700 1 -- 172.21.15.33:0/187188747 shutdown_connections 2023-11-06T22:18:13.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.497+0000 7f43bedff700 1 -- 172.21.15.33:0/187188747 wait complete. 2023-11-06T22:18:13.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.498+0000 7f43bedff700 1 Processor -- start 2023-11-06T22:18:13.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.498+0000 7f43bedff700 1 -- start start 2023-11-06T22:18:13.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.499+0000 7f43bedff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b80c72c0 0x7f43b8150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.499+0000 7f43bedff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8157fa0 0x7f43b814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.499+0000 7f43bedff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b8158420 0x7f43b80abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.499+0000 7f43bedff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8150c30 con 0x7f43b80c72c0 2023-11-06T22:18:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.499+0000 7f43bedff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80af8c0 con 0x7f43b8157fa0 2023-11-06T22:18:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.499+0000 7f43bedff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80afba0 con 0x7f43b8158420 2023-11-06T22:18:13.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bcb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b80c72c0 0x7f43b8150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bcb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b80c72c0 0x7f43b8150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37260/0 (socket says 172.21.15.33:37260) 2023-11-06T22:18:13.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bcb9b700 1 -- 172.21.15.33:0/3624462043 learned_addr learned my addr 172.21.15.33:0/3624462043 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:13.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43b7fff700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8157fa0 0x7f43b814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bd39c700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b8158420 0x7f43b80abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bcb9b700 1 -- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b8158420 msgr2=0x7f43b80abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bcb9b700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b8158420 0x7f43b80abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bcb9b700 1 -- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8157fa0 msgr2=0x7f43b814f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bcb9b700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8157fa0 0x7f43b814f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bcb9b700 1 -- 172.21.15.33:0/3624462043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0009a20 con 0x7f43b80c72c0 2023-11-06T22:18:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bcb9b700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b80c72c0 0x7f43b8150740 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7f43ac00c4c0 tx=0x7f43ac007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43b4ff9700 1 -- 172.21.15.33:0/3624462043 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac0072e0 con 0x7f43b80c72c0 2023-11-06T22:18:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43b4ff9700 1 -- 172.21.15.33:0/3624462043 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ac007440 con 0x7f43b80c72c0 2023-11-06T22:18:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43bedff700 1 -- 172.21.15.33:0/3624462043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b80afe80 con 0x7f43b80c72c0 2023-11-06T22:18:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.500+0000 7f43b4ff9700 1 -- 172.21.15.33:0/3624462043 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac0057a0 con 0x7f43b80c72c0 2023-11-06T22:18:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.501+0000 7f43bedff700 1 -- 172.21.15.33:0/3624462043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b8151840 con 0x7f43b80c72c0 2023-11-06T22:18:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.501+0000 7f43bedff700 1 -- 172.21.15.33:0/3624462043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b80c2920 con 0x7f43b80c72c0 2023-11-06T22:18:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.502+0000 7f43b4ff9700 1 -- 172.21.15.33:0/3624462043 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f43ac005b90 con 0x7f43b80c72c0 2023-11-06T22:18:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.503+0000 7f43b4ff9700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f439c05f240 0x7f439c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.503+0000 7f43b4ff9700 1 -- 172.21.15.33:0/3624462043 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5943+0+0 (secure 0 0 0) 0x7f43ac079c90 con 0x7f43b80c72c0 2023-11-06T22:18:13.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.504+0000 7f43b7fff700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f439c05f240 0x7f439c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.505+0000 7f43b7fff700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f439c05f240 0x7f439c061700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f43b80aa510 tx=0x7f43a4008040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:13.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.507+0000 7f43b4ff9700 1 -- 172.21.15.33:0/3624462043 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f43ac04add0 con 0x7f43b80c72c0 2023-11-06T22:18:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.702+0000 7f43bedff700 1 -- 172.21.15.33:0/3624462043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f43b8062b80 con 0x7f43b80c72c0 2023-11-06T22:18:13.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.707+0000 7f43b4ff9700 1 -- 172.21.15.33:0/3624462043 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v40) v1 ==== 120+0+0 (secure 0 0 0) 0x7f43ac018070 con 0x7f43b80c72c0 2023-11-06T22:18:13.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 -- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f439c05f240 msgr2=0x7f439c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f439c05f240 0x7f439c061700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f43b80aa510 tx=0x7f43a4008040).stop 2023-11-06T22:18:13.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 -- 172.21.15.33:0/3624462043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b80c72c0 msgr2=0x7f43b8150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b80c72c0 0x7f43b8150740 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7f43ac00c4c0 tx=0x7f43ac007c10).stop 2023-11-06T22:18:13.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 -- 172.21.15.33:0/3624462043 shutdown_connections 2023-11-06T22:18:13.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f439c05f240 0x7f439c061700 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43b8158420 0x7f43b80abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b80c72c0 0x7f43b8150740 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 --2- 172.21.15.33:0/3624462043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43b8157fa0 0x7f43b814f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 -- 172.21.15.33:0/3624462043 >> 172.21.15.33:0/3624462043 conn(0x7f43b805ec20 msgr2=0x7f43b8155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:13.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 -- 172.21.15.33:0/3624462043 shutdown_connections 2023-11-06T22:18:13.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.709+0000 7f43bedff700 1 -- 172.21.15.33:0/3624462043 wait complete. 2023-11-06T22:18:13.730 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T22:18:13.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.926+0000 7f43a7bff700 1 Processor -- start 2023-11-06T22:18:13.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.927+0000 7f43a7bff700 1 -- start start 2023-11-06T22:18:13.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.928+0000 7f43a7bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43a01541e0 0x7f43a01545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.928+0000 7f43a7bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f43a0062e20 0x7f43a00c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.928+0000 7f43a7bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a00bcbe0 con 0x7f43a00c3f30 2023-11-06T22:18:13.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.928+0000 7f43a7bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a00a6f00 con 0x7f43a01541e0 2023-11-06T22:18:13.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.928+0000 7f43a7bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a00bd070 con 0x7f43a0062e20 2023-11-06T22:18:13.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.928+0000 7f43a519a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f43a00c3f30 0x7f43a0062710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44468/0 (socket says 172.21.15.33:44468) 2023-11-06T22:18:13.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.928+0000 7f43a599b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43a01541e0 0x7f43a01545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.928+0000 7f43a519a700 1 -- 172.21.15.33:0/2114256527 learned_addr learned my addr 172.21.15.33:0/2114256527 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.929+0000 7f43a619c700 1 --2- 172.21.15.33:0/2114256527 >> v2:172.21.15.69:3300/0 conn(0x7f43a0062e20 0x7f43a00c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.929+0000 7f43a599b700 1 -- 172.21.15.33:0/2114256527 >> v2:172.21.15.69:3300/0 conn(0x7f43a0062e20 msgr2=0x7f43a00c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.929+0000 7f43a599b700 1 --2- 172.21.15.33:0/2114256527 >> v2:172.21.15.69:3300/0 conn(0x7f43a0062e20 0x7f43a00c24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.929+0000 7f43a599b700 1 -- 172.21.15.33:0/2114256527 >> v1:172.21.15.33:6789/0 conn(0x7f43a00c3f30 legacy=0x7f43a0062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.929+0000 7f43a4999700 1 -- 172.21.15.33:0/2114256527 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807588591 0 0) 0x7f43a00bcbe0 con 0x7f43a00c3f30 2023-11-06T22:18:13.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.929+0000 7f43a599b700 1 -- 172.21.15.33:0/2114256527 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a0158db0 con 0x7f43a01541e0 2023-11-06T22:18:13.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.929+0000 7f43a599b700 1 --2- 172.21.15.33:0/2114256527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a01541e0 0x7f43a01545c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f43a0067e80 tx=0x7f439c009060).ready entity=mon.0 client_cookie=2d4a89473bcbb547 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:13.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.929+0000 7f43a4999700 1 -- 172.21.15.33:0/2114256527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00b580 con 0x7f43a01541e0 2023-11-06T22:18:13.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.929+0000 7f43a4999700 1 -- 172.21.15.33:0/2114256527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c00b6e0 con 0x7f43a01541e0 2023-11-06T22:18:13.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.929+0000 7f43a4999700 1 -- 172.21.15.33:0/2114256527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00ba60 con 0x7f43a01541e0 2023-11-06T22:18:13.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.930+0000 7f43a7bff700 1 -- 172.21.15.33:0/2114256527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a01541e0 msgr2=0x7f43a01545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.930+0000 7f43a7bff700 1 --2- 172.21.15.33:0/2114256527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a01541e0 0x7f43a01545c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f43a0067e80 tx=0x7f439c009060).stop 2023-11-06T22:18:13.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.930+0000 7f43a7bff700 1 -- 172.21.15.33:0/2114256527 shutdown_connections 2023-11-06T22:18:13.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.930+0000 7f43a7bff700 1 --2- 172.21.15.33:0/2114256527 >> v2:172.21.15.69:3300/0 conn(0x7f43a0062e20 0x7f43a00c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.930+0000 7f43a7bff700 1 --2- 172.21.15.33:0/2114256527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a01541e0 0x7f43a01545c0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.930+0000 7f43a7bff700 1 -- 172.21.15.33:0/2114256527 >> 172.21.15.33:0/2114256527 conn(0x7f43a00c7b50 msgr2=0x7f43a00c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:13.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.930+0000 7f43a7bff700 1 -- 172.21.15.33:0/2114256527 shutdown_connections 2023-11-06T22:18:13.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.930+0000 7f43a7bff700 1 -- 172.21.15.33:0/2114256527 wait complete. 2023-11-06T22:18:13.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.931+0000 7f43a7bff700 1 Processor -- start 2023-11-06T22:18:13.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.932+0000 7f43a7bff700 1 -- start start 2023-11-06T22:18:13.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.932+0000 7f43a7bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43a0062e20 0x7f43a00abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.932+0000 7f43a7bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a00c3f30 0x7f43a0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a7bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a01541e0 0x7f43a0151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a7bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a00a8450 con 0x7f43a00c3f30 2023-11-06T22:18:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a7bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a00ac300 con 0x7f43a01541e0 2023-11-06T22:18:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a7bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a00ac5e0 con 0x7f43a0062e20 2023-11-06T22:18:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a00c3f30 0x7f43a0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a00c3f30 0x7f43a0066570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37268/0 (socket says 172.21.15.33:37268) 2023-11-06T22:18:13.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a519a700 1 -- 172.21.15.33:0/2142221709 learned_addr learned my addr 172.21.15.33:0/2142221709 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:13.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a599b700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43a0062e20 0x7f43a00abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a519a700 1 -- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43a0062e20 msgr2=0x7f43a00abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a619c700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a01541e0 0x7f43a0151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a519a700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43a0062e20 0x7f43a00abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a519a700 1 -- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a01541e0 msgr2=0x7f43a0151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:13.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a519a700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a01541e0 0x7f43a0151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:13.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a519a700 1 -- 172.21.15.33:0/2142221709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c0092a0 con 0x7f43a00c3f30 2023-11-06T22:18:13.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a619c700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a01541e0 0x7f43a0151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:13.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.933+0000 7f43a519a700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a00c3f30 0x7f43a0066570 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7f439000c4e0 tx=0x7f4390007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:13.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.934+0000 7f4395ffb700 1 -- 172.21.15.33:0/2142221709 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43900072e0 con 0x7f43a00c3f30 2023-11-06T22:18:13.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.934+0000 7f4395ffb700 1 -- 172.21.15.33:0/2142221709 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4390007440 con 0x7f43a00c3f30 2023-11-06T22:18:13.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.934+0000 7f43a7bff700 1 -- 172.21.15.33:0/2142221709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a00ac8c0 con 0x7f43a00c3f30 2023-11-06T22:18:13.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.934+0000 7f4395ffb700 1 -- 172.21.15.33:0/2142221709 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43900057a0 con 0x7f43a00c3f30 2023-11-06T22:18:13.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.934+0000 7f43a7bff700 1 -- 172.21.15.33:0/2142221709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a00abfa0 con 0x7f43a00c3f30 2023-11-06T22:18:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.935+0000 7f43a7bff700 1 -- 172.21.15.33:0/2142221709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a00a8590 con 0x7f43a00c3f30 2023-11-06T22:18:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.936+0000 7f4395ffb700 1 -- 172.21.15.33:0/2142221709 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4390005bd0 con 0x7f43a00c3f30 2023-11-06T22:18:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.937+0000 7f4395ffb700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f438405f270 0x7f4384061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.937+0000 7f4395ffb700 1 -- 172.21.15.33:0/2142221709 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5943+0+0 (secure 0 0 0) 0x7f4390079d20 con 0x7f43a00c3f30 2023-11-06T22:18:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.937+0000 7f43a599b700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f438405f270 0x7f4384061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:13.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.938+0000 7f43a599b700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f438405f270 0x7f4384061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f439c00b340 tx=0x7f439c017000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:13.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:13.941+0000 7f4395ffb700 1 -- 172.21.15.33:0/2142221709 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f43a00a8590 con 0x7f43a00c3f30 2023-11-06T22:18:14.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.120+0000 7f43a7bff700 1 -- 172.21.15.33:0/2142221709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f43a00a8590 con 0x7f43a00c3f30 2023-11-06T22:18:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.124+0000 7f4395ffb700 1 -- 172.21.15.33:0/2142221709 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f43a00a8590 con 0x7f43a00c3f30 2023-11-06T22:18:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 -- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f438405f270 msgr2=0x7f4384061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f438405f270 0x7f4384061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f439c00b340 tx=0x7f439c017000).stop 2023-11-06T22:18:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 -- 172.21.15.33:0/2142221709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a00c3f30 msgr2=0x7f43a0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a00c3f30 0x7f43a0066570 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7f439000c4e0 tx=0x7f4390007c70).stop 2023-11-06T22:18:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 -- 172.21.15.33:0/2142221709 shutdown_connections 2023-11-06T22:18:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f438405f270 0x7f4384061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43a0062e20 0x7f43a00abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a01541e0 0x7f43a0151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 --2- 172.21.15.33:0/2142221709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a00c3f30 0x7f43a0066570 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 -- 172.21.15.33:0/2142221709 >> 172.21.15.33:0/2142221709 conn(0x7f43a00c7b50 msgr2=0x7f43a0062710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 -- 172.21.15.33:0/2142221709 shutdown_connections 2023-11-06T22:18:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.127+0000 7f43a7bff700 1 -- 172.21.15.33:0/2142221709 wait complete. 2023-11-06T22:18:14.147 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-11-06T22:18:14.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.343+0000 7f0c215ff700 1 Processor -- start 2023-11-06T22:18:14.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.344+0000 7f0c215ff700 1 -- start start 2023-11-06T22:18:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.345+0000 7f0c215ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 0x7f0c1c063660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.345+0000 7f0c215ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c1c14e4a0 0x7f0c1c14e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.345+0000 7f0c215ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0c1c0645d0 0x7f0c1c064a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.345+0000 7f0c215ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1c14ee10 con 0x7f0c1c14e4a0 2023-11-06T22:18:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.345+0000 7f0c215ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1c158e20 con 0x7f0c1c0645d0 2023-11-06T22:18:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.345+0000 7f0c215ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c1582a0 con 0x7f0c1c063280 2023-11-06T22:18:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.345+0000 7f0c1a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c1c14e4a0 0x7f0c1c14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.345+0000 7f0c1a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c1c14e4a0 0x7f0c1c14e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37282/0 (socket says 172.21.15.33:37282) 2023-11-06T22:18:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.345+0000 7f0c1a7fc700 1 -- 172.21.15.33:0/547294691 learned_addr learned my addr 172.21.15.33:0/547294691 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c1a7fc700 1 -- 172.21.15.33:0/547294691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 msgr2=0x7f0c1c063660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c1affd700 1 --2- 172.21.15.33:0/547294691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 0x7f0c1c063660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:14.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c1b7fe700 1 --2- 172.21.15.33:0/547294691 >> v2:172.21.15.69:3300/0 conn(0x7f0c1c0645d0 0x7f0c1c064a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:14.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c1a7fc700 1 --2- 172.21.15.33:0/547294691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 0x7f0c1c063660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c1a7fc700 1 -- 172.21.15.33:0/547294691 >> v2:172.21.15.69:3300/0 conn(0x7f0c1c0645d0 msgr2=0x7f0c1c064a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:14.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c1a7fc700 1 --2- 172.21.15.33:0/547294691 >> v2:172.21.15.69:3300/0 conn(0x7f0c1c0645d0 0x7f0c1c064a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c1a7fc700 1 -- 172.21.15.33:0/547294691 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c158580 con 0x7f0c1c14e4a0 2023-11-06T22:18:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c1affd700 1 --2- 172.21.15.33:0/547294691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 0x7f0c1c063660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c1b7fe700 1 --2- 172.21.15.33:0/547294691 >> v2:172.21.15.69:3300/0 conn(0x7f0c1c0645d0 0x7f0c1c064a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c1a7fc700 1 --2- 172.21.15.33:0/547294691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c14e4a0 0x7f0c1c14e920 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7f0c1c0aab40 tx=0x7f0c100090f0).ready entity=mon.0 client_cookie=9276d9fba7e916fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c19ffb700 1 -- 172.21.15.33:0/547294691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000b600 con 0x7f0c1c14e4a0 2023-11-06T22:18:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c19ffb700 1 -- 172.21.15.33:0/547294691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1000b760 con 0x7f0c1c14e4a0 2023-11-06T22:18:14.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.346+0000 7f0c19ffb700 1 -- 172.21.15.33:0/547294691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000bae0 con 0x7f0c1c14e4a0 2023-11-06T22:18:14.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.347+0000 7f0c215ff700 1 -- 172.21.15.33:0/547294691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c14e4a0 msgr2=0x7f0c1c14e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:14.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.347+0000 7f0c215ff700 1 --2- 172.21.15.33:0/547294691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c14e4a0 0x7f0c1c14e920 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7f0c1c0aab40 tx=0x7f0c100090f0).stop 2023-11-06T22:18:14.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.347+0000 7f0c215ff700 1 -- 172.21.15.33:0/547294691 shutdown_connections 2023-11-06T22:18:14.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.347+0000 7f0c215ff700 1 --2- 172.21.15.33:0/547294691 >> v2:172.21.15.69:3300/0 conn(0x7f0c1c0645d0 0x7f0c1c064a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.347+0000 7f0c215ff700 1 --2- 172.21.15.33:0/547294691 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 0x7f0c1c063660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.347+0000 7f0c215ff700 1 --2- 172.21.15.33:0/547294691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c14e4a0 0x7f0c1c14e920 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.347+0000 7f0c215ff700 1 -- 172.21.15.33:0/547294691 >> 172.21.15.33:0/547294691 conn(0x7f0c1c0cade0 msgr2=0x7f0c1c0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:14.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.347+0000 7f0c215ff700 1 -- 172.21.15.33:0/547294691 shutdown_connections 2023-11-06T22:18:14.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.347+0000 7f0c215ff700 1 -- 172.21.15.33:0/547294691 wait complete. 2023-11-06T22:18:14.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.348+0000 7f0c215ff700 1 Processor -- start 2023-11-06T22:18:14.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.349+0000 7f0c215ff700 1 -- start start 2023-11-06T22:18:14.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.349+0000 7f0c215ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 0x7f0c1c0c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:14.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.349+0000 7f0c215ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c0645d0 0x7f0c1c13f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:14.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.349+0000 7f0c215ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c1c14e4a0 0x7f0c1c0c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:14.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.349+0000 7f0c215ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c14ada0 con 0x7f0c1c0645d0 2023-11-06T22:18:14.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.349+0000 7f0c215ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0c2020 con 0x7f0c1c14e4a0 2023-11-06T22:18:14.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.349+0000 7f0c215ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0c46e0 con 0x7f0c1c063280 2023-11-06T22:18:14.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c0645d0 0x7f0c1c13f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:14.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c0645d0 0x7f0c1c13f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37290/0 (socket says 172.21.15.33:37290) 2023-11-06T22:18:14.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1a7fc700 1 -- 172.21.15.33:0/1134383014 learned_addr learned my addr 172.21.15.33:0/1134383014 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:14.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1affd700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 0x7f0c1c0c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:14.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1a7fc700 1 -- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 msgr2=0x7f0c1c0c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:14.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1a7fc700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 0x7f0c1c0c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1a7fc700 1 -- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c1c14e4a0 msgr2=0x7f0c1c0c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:14.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1b7fe700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c1c14e4a0 0x7f0c1c0c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:14.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1a7fc700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c1c14e4a0 0x7f0c1c0c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:14.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1a7fc700 1 -- 172.21.15.33:0/1134383014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10009370 con 0x7f0c1c0645d0 2023-11-06T22:18:14.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1affd700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 0x7f0c1c0c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:14.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1b7fe700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c1c14e4a0 0x7f0c1c0c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:14.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.350+0000 7f0c1a7fc700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c0645d0 0x7f0c1c13f800 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7f0c10009130 tx=0x7f0c10017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:14.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.351+0000 7f0c02ffd700 1 -- 172.21.15.33:0/1134383014 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000bae0 con 0x7f0c1c0645d0 2023-11-06T22:18:14.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.351+0000 7f0c02ffd700 1 -- 172.21.15.33:0/1134383014 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10000c90 con 0x7f0c1c0645d0 2023-11-06T22:18:14.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.351+0000 7f0c02ffd700 1 -- 172.21.15.33:0/1134383014 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1001d660 con 0x7f0c1c0645d0 2023-11-06T22:18:14.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.351+0000 7f0c215ff700 1 -- 172.21.15.33:0/1134383014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1c0c22c0 con 0x7f0c1c0645d0 2023-11-06T22:18:14.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.351+0000 7f0c215ff700 1 -- 172.21.15.33:0/1134383014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1c0c2880 con 0x7f0c1c0645d0 2023-11-06T22:18:14.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.352+0000 7f0c215ff700 1 -- 172.21.15.33:0/1134383014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1c06c6c0 con 0x7f0c1c0645d0 2023-11-06T22:18:14.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.353+0000 7f0c02ffd700 1 -- 172.21.15.33:0/1134383014 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0c1001da10 con 0x7f0c1c0645d0 2023-11-06T22:18:14.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.353+0000 7f0c02ffd700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0c0805f270 0x7f0c08061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:14.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.353+0000 7f0c02ffd700 1 -- 172.21.15.33:0/1134383014 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5943+0+0 (secure 0 0 0) 0x7f0c1007c0c0 con 0x7f0c1c0645d0 2023-11-06T22:18:14.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.354+0000 7f0c1affd700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0c0805f270 0x7f0c08061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:14.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.354+0000 7f0c1affd700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0c0805f270 0x7f0c08061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0c0c006fd0 tx=0x7f0c0c008040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:14.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.357+0000 7f0c02ffd700 1 -- 172.21.15.33:0/1134383014 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0c1004d2c0 con 0x7f0c1c0645d0 2023-11-06T22:18:14.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:14.603+0000 7f0c215ff700 1 -- 172.21.15.33:0/1134383014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f0c1c0c4430 con 0x7f0c1c0645d0 2023-11-06T22:18:15.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.313+0000 7f0c02ffd700 1 -- 172.21.15.33:0/1134383014 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3031) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0c100210e0 con 0x7f0c1c0645d0 2023-11-06T22:18:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.315+0000 7f0c215ff700 1 -- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0c0805f270 msgr2=0x7f0c08061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.315+0000 7f0c215ff700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0c0805f270 0x7f0c08061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0c0c006fd0 tx=0x7f0c0c008040).stop 2023-11-06T22:18:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.315+0000 7f0c215ff700 1 -- 172.21.15.33:0/1134383014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c0645d0 msgr2=0x7f0c1c13f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.315+0000 7f0c215ff700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c0645d0 0x7f0c1c13f800 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7f0c10009130 tx=0x7f0c10017000).stop 2023-11-06T22:18:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.316+0000 7f0c215ff700 1 -- 172.21.15.33:0/1134383014 shutdown_connections 2023-11-06T22:18:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.316+0000 7f0c215ff700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0c0805f270 0x7f0c08061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.316+0000 7f0c215ff700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0c1c063280 0x7f0c1c0c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.316+0000 7f0c215ff700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c1c14e4a0 0x7f0c1c0c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.316+0000 7f0c215ff700 1 --2- 172.21.15.33:0/1134383014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c1c0645d0 0x7f0c1c13f800 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.316+0000 7f0c215ff700 1 -- 172.21.15.33:0/1134383014 >> 172.21.15.33:0/1134383014 conn(0x7f0c1c0cade0 msgr2=0x7f0c1c14bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:15.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.316+0000 7f0c215ff700 1 -- 172.21.15.33:0/1134383014 shutdown_connections 2023-11-06T22:18:15.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.316+0000 7f0c215ff700 1 -- 172.21.15.33:0/1134383014 wait complete. 2023-11-06T22:18:15.338 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-11-06T22:18:15.339 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-11-06T22:18:15.339 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-11-06T22:18:15.339 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-11-06T22:18:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.537+0000 7f2fc63ff700 1 Processor -- start 2023-11-06T22:18:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.537+0000 7f2fc63ff700 1 -- start start 2023-11-06T22:18:15.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.539+0000 7f2fc63ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2fc014ebb0 0x7f2fc0063660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:15.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.539+0000 7f2fc63ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2fc00c05e0 0x7f2fc00c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:15.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.539+0000 7f2fc63ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc0158e20 con 0x7f2fc0063280 2023-11-06T22:18:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.539+0000 7f2fc63ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc01582a0 con 0x7f2fc00c05e0 2023-11-06T22:18:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.539+0000 7f2fc63ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc0158580 con 0x7f2fc014ebb0 2023-11-06T22:18:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.539+0000 7f2fc499c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2fc00c05e0 0x7f2fc00c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.539+0000 7f2fbffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2fc0063280 0x7f2fc014e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44470/0 (socket says 172.21.15.33:44470) 2023-11-06T22:18:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.539+0000 7f2fbffff700 1 -- 172.21.15.33:0/84026280 learned_addr learned my addr 172.21.15.33:0/84026280 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.540+0000 7f2fbf7fe700 1 --2- 172.21.15.33:0/84026280 >> v2:172.21.15.69:3300/0 conn(0x7f2fc014ebb0 0x7f2fc0063660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:15.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.540+0000 7f2fc499c700 1 -- 172.21.15.33:0/84026280 >> v2:172.21.15.69:3300/0 conn(0x7f2fc014ebb0 msgr2=0x7f2fc0063660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:15.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.540+0000 7f2fc499c700 1 --2- 172.21.15.33:0/84026280 >> v2:172.21.15.69:3300/0 conn(0x7f2fc014ebb0 0x7f2fc0063660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:15.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.540+0000 7f2fbeffd700 1 -- 172.21.15.33:0/84026280 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751750130 0 0) 0x7f2fc0158e20 con 0x7f2fc0063280 2023-11-06T22:18:15.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.540+0000 7f2fc499c700 1 -- 172.21.15.33:0/84026280 >> v1:172.21.15.33:6789/0 conn(0x7f2fc0063280 legacy=0x7f2fc014e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:15.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.540+0000 7f2fc499c700 1 -- 172.21.15.33:0/84026280 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc0158860 con 0x7f2fc00c05e0 2023-11-06T22:18:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.540+0000 7f2fc499c700 1 --2- 172.21.15.33:0/84026280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00c05e0 0x7f2fc00c0a50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f2fc00b0f90 tx=0x7f2fb8009780).ready entity=mon.0 client_cookie=d40c8c15b7532a5c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.540+0000 7f2fbeffd700 1 -- 172.21.15.33:0/84026280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb800e070 con 0x7f2fc00c05e0 2023-11-06T22:18:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.540+0000 7f2fbeffd700 1 -- 172.21.15.33:0/84026280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb800bb60 con 0x7f2fc00c05e0 2023-11-06T22:18:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.540+0000 7f2fbeffd700 1 -- 172.21.15.33:0/84026280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8012410 con 0x7f2fc00c05e0 2023-11-06T22:18:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.541+0000 7f2fc63ff700 1 -- 172.21.15.33:0/84026280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00c05e0 msgr2=0x7f2fc00c0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:15.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.541+0000 7f2fc63ff700 1 --2- 172.21.15.33:0/84026280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00c05e0 0x7f2fc00c0a50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f2fc00b0f90 tx=0x7f2fb8009780).stop 2023-11-06T22:18:15.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.541+0000 7f2fc63ff700 1 -- 172.21.15.33:0/84026280 shutdown_connections 2023-11-06T22:18:15.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.541+0000 7f2fc63ff700 1 --2- 172.21.15.33:0/84026280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00c05e0 0x7f2fc00c0a50 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:15.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.541+0000 7f2fc63ff700 1 --2- 172.21.15.33:0/84026280 >> v2:172.21.15.69:3300/0 conn(0x7f2fc014ebb0 0x7f2fc0063660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:15.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.541+0000 7f2fc63ff700 1 -- 172.21.15.33:0/84026280 >> 172.21.15.33:0/84026280 conn(0x7f2fc00cade0 msgr2=0x7f2fc00cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:15.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.541+0000 7f2fc63ff700 1 -- 172.21.15.33:0/84026280 shutdown_connections 2023-11-06T22:18:15.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.541+0000 7f2fc63ff700 1 -- 172.21.15.33:0/84026280 wait complete. 2023-11-06T22:18:15.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.543+0000 7f2fc63ff700 1 Processor -- start 2023-11-06T22:18:15.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.543+0000 7f2fc63ff700 1 -- start start 2023-11-06T22:18:15.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.544+0000 7f2fc63ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fc0063280 0x7f2fc00a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:15.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.544+0000 7f2fc63ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fc00c05e0 0x7f2fc00a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:15.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.544+0000 7f2fc63ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc014ebb0 0x7f2fc00a2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.544+0000 7f2fc63ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0164ab0 con 0x7f2fc014ebb0 2023-11-06T22:18:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.544+0000 7f2fc63ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0167d00 con 0x7f2fc00c05e0 2023-11-06T22:18:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.544+0000 7f2fc63ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00a3370 con 0x7f2fc0063280 2023-11-06T22:18:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.544+0000 7f2fc499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc014ebb0 0x7f2fc00a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.544+0000 7f2fbffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fc0063280 0x7f2fc00a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.544+0000 7f2fbf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fc00c05e0 0x7f2fc00a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fbf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fc00c05e0 0x7f2fc00a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52982/0 (socket says 172.21.15.33:52982) 2023-11-06T22:18:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fc499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc014ebb0 0x7f2fc00a2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37310/0 (socket says 172.21.15.33:37310) 2023-11-06T22:18:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fbf7fe700 1 -- 172.21.15.33:0/1324234535 learned_addr learned my addr 172.21.15.33:0/1324234535 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fc499c700 1 -- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fc0063280 msgr2=0x7f2fc00a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fc499c700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fc0063280 0x7f2fc00a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fc499c700 1 -- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fc00c05e0 msgr2=0x7f2fc00a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fc499c700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fc00c05e0 0x7f2fc00a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fc499c700 1 -- 172.21.15.33:0/1324234535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8009a20 con 0x7f2fc014ebb0 2023-11-06T22:18:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fc499c700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc014ebb0 0x7f2fc00a2920 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f2fb8014000 tx=0x7f2fb8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fa7fff700 1 -- 172.21.15.33:0/1324234535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8012410 con 0x7f2fc014ebb0 2023-11-06T22:18:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fa7fff700 1 -- 172.21.15.33:0/1324234535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb801d630 con 0x7f2fc014ebb0 2023-11-06T22:18:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fc63ff700 1 -- 172.21.15.33:0/1324234535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc00a3650 con 0x7f2fc014ebb0 2023-11-06T22:18:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.545+0000 7f2fa7fff700 1 -- 172.21.15.33:0/1324234535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb801d9b0 con 0x7f2fc014ebb0 2023-11-06T22:18:15.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.546+0000 7f2fc63ff700 1 -- 172.21.15.33:0/1324234535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc00a3c10 con 0x7f2fc014ebb0 2023-11-06T22:18:15.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.547+0000 7f2fc63ff700 1 -- 172.21.15.33:0/1324234535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc0164c20 con 0x7f2fc014ebb0 2023-11-06T22:18:15.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.547+0000 7f2fa7fff700 1 -- 172.21.15.33:0/1324234535 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2fb8026410 con 0x7f2fc014ebb0 2023-11-06T22:18:15.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.548+0000 7f2fa7fff700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2fac05f270 0x7f2fac061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:15.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.548+0000 7f2fa7fff700 1 -- 172.21.15.33:0/1324234535 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5943+0+0 (secure 0 0 0) 0x7f2fb807bff0 con 0x7f2fc014ebb0 2023-11-06T22:18:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.549+0000 7f2fbffff700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2fac05f270 0x7f2fac061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.549+0000 7f2fbffff700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2fac05f270 0x7f2fac061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f2fc00c5020 tx=0x7f2fb0007bb0).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.552+0000 7f2fa7fff700 1 -- 172.21.15.33:0/1324234535 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2fc0164c20 con 0x7f2fc014ebb0 2023-11-06T22:18:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:15.803+0000 7f2fc63ff700 1 -- 172.21.15.33:0/1324234535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f2fc0164bf0 con 0x7f2fc014ebb0 2023-11-06T22:18:16.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.317+0000 7f2fa7fff700 1 -- 172.21.15.33:0/1324234535 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3032) v1 ==== 158+0+0 (secure 0 0 0) 0x7f2fb8050bc0 con 0x7f2fc014ebb0 2023-11-06T22:18:16.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 -- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2fac05f270 msgr2=0x7f2fac061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:16.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2fac05f270 0x7f2fac061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f2fc00c5020 tx=0x7f2fb0007bb0).stop 2023-11-06T22:18:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 -- 172.21.15.33:0/1324234535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc014ebb0 msgr2=0x7f2fc00a2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc014ebb0 0x7f2fc00a2920 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f2fb8014000 tx=0x7f2fb8009780).stop 2023-11-06T22:18:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 -- 172.21.15.33:0/1324234535 shutdown_connections 2023-11-06T22:18:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2fac05f270 0x7f2fac061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2fc0063280 0x7f2fc00a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc014ebb0 0x7f2fc00a2920 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 --2- 172.21.15.33:0/1324234535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fc00c05e0 0x7f2fc00a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 -- 172.21.15.33:0/1324234535 >> 172.21.15.33:0/1324234535 conn(0x7f2fc00cade0 msgr2=0x7f2fc014e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:16.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 -- 172.21.15.33:0/1324234535 shutdown_connections 2023-11-06T22:18:16.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.320+0000 7f2fc63ff700 1 -- 172.21.15.33:0/1324234535 wait complete. 2023-11-06T22:18:16.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T22:18:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.540+0000 7fe4365ff700 1 Processor -- start 2023-11-06T22:18:16.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.540+0000 7fe4365ff700 1 -- start start 2023-11-06T22:18:16.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.540+0000 7fe4365ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe430154330 0x7fe4300c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:16.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.541+0000 7fe4365ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4300c7ff0 0x7fe4300646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:16.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.541+0000 7fe4365ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4300c6b60 con 0x7fe430154710 2023-11-06T22:18:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.541+0000 7fe4365ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4300c7700 con 0x7fe430154330 2023-11-06T22:18:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.541+0000 7fe4365ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300c5cd0 con 0x7fe4300c7ff0 2023-11-06T22:18:16.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.541+0000 7fe434b9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4300c7ff0 0x7fe4300646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:16.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.541+0000 7fe42ffff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe430154330 0x7fe4300c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:16.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.541+0000 7fe434b9c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4300c7ff0 0x7fe4300646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:35282/0 (socket says 172.21.15.33:35282) 2023-11-06T22:18:16.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.541+0000 7fe42f7fe700 1 --1- 172.21.15.33:0/1923958272 >> v1:172.21.15.69:6789/0 conn(0x7fe430154710 0x7fe4300c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:47502/0 (socket says 172.21.15.33:47502) 2023-11-06T22:18:16.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.541+0000 7fe434b9c700 1 -- 172.21.15.33:0/1923958272 learned_addr learned my addr 172.21.15.33:0/1923958272 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:16.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe42ffff700 1 -- 172.21.15.33:0/1923958272 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4300c7ff0 msgr2=0x7fe4300646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:16.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe42ffff700 1 --2- 172.21.15.33:0/1923958272 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4300c7ff0 0x7fe4300646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe42ffff700 1 -- 172.21.15.33:0/1923958272 >> v1:172.21.15.69:6789/0 conn(0x7fe430154710 legacy=0x7fe4300c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:16.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe42ffff700 1 -- 172.21.15.33:0/1923958272 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4300c5f50 con 0x7fe430154330 2023-11-06T22:18:16.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe434b9c700 1 --2- 172.21.15.33:0/1923958272 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4300c7ff0 0x7fe4300646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:16.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe42effd700 1 -- 172.21.15.33:0/1923958272 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957538931 0 0) 0x7fe4300c6b60 con 0x7fe430154710 2023-11-06T22:18:16.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe42ffff700 1 --2- 172.21.15.33:0/1923958272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe430154330 0x7fe4300c57e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fe420009760 tx=0x7fe420009060).ready entity=mon.1 client_cookie=e9b2c4898842ee19 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:16.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe42effd700 1 -- 172.21.15.33:0/1923958272 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42000f510 con 0x7fe430154330 2023-11-06T22:18:16.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe42effd700 1 -- 172.21.15.33:0/1923958272 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42000f670 con 0x7fe430154330 2023-11-06T22:18:16.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe42effd700 1 -- 172.21.15.33:0/1923958272 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42000f9f0 con 0x7fe430154330 2023-11-06T22:18:16.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe4365ff700 1 -- 172.21.15.33:0/1923958272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe430154330 msgr2=0x7fe4300c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:16.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.542+0000 7fe4365ff700 1 --2- 172.21.15.33:0/1923958272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe430154330 0x7fe4300c57e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fe420009760 tx=0x7fe420009060).stop 2023-11-06T22:18:16.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.543+0000 7fe4365ff700 1 -- 172.21.15.33:0/1923958272 shutdown_connections 2023-11-06T22:18:16.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.543+0000 7fe4365ff700 1 --2- 172.21.15.33:0/1923958272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe430154330 0x7fe4300c57e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.543+0000 7fe4365ff700 1 --2- 172.21.15.33:0/1923958272 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe4300c7ff0 0x7fe4300646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.543+0000 7fe4365ff700 1 -- 172.21.15.33:0/1923958272 >> 172.21.15.33:0/1923958272 conn(0x7fe4300cb910 msgr2=0x7fe4300c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:16.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.543+0000 7fe4365ff700 1 -- 172.21.15.33:0/1923958272 shutdown_connections 2023-11-06T22:18:16.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.543+0000 7fe4365ff700 1 -- 172.21.15.33:0/1923958272 wait complete. 2023-11-06T22:18:16.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.545+0000 7fe4365ff700 1 Processor -- start 2023-11-06T22:18:16.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.545+0000 7fe4365ff700 1 -- start start 2023-11-06T22:18:16.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.545+0000 7fe4365ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300c7ff0 0x7fe430154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:16.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.545+0000 7fe4365ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe430154330 0x7fe430153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:16.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.545+0000 7fe4365ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430154710 0x7fe4300c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:16.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.545+0000 7fe4365ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe430153f10 con 0x7fe430154710 2023-11-06T22:18:16.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.545+0000 7fe4365ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe430155e40 con 0x7fe4300c7ff0 2023-11-06T22:18:16.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.545+0000 7fe4365ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe430155230 con 0x7fe430154330 2023-11-06T22:18:16.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe42ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300c7ff0 0x7fe430154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:16.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe434b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430154710 0x7fe4300c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:16.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe42ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300c7ff0 0x7fe430154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52998/0 (socket says 172.21.15.33:52998) 2023-11-06T22:18:16.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe42ffff700 1 -- 172.21.15.33:0/3455926623 learned_addr learned my addr 172.21.15.33:0/3455926623 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:16.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe42f7fe700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe430154330 0x7fe430153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:16.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe434b9c700 1 -- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe430154330 msgr2=0x7fe430153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:16.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe434b9c700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe430154330 0x7fe430153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe434b9c700 1 -- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300c7ff0 msgr2=0x7fe430154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe434b9c700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300c7ff0 0x7fe430154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe434b9c700 1 -- 172.21.15.33:0/3455926623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe428009ae0 con 0x7fe430154710 2023-11-06T22:18:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe42ffff700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300c7ff0 0x7fe430154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe434b9c700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430154710 0x7fe4300c1ee0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7fe428009f40 tx=0x7fe42800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.546+0000 7fe417fff700 1 -- 172.21.15.33:0/3455926623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4280092e0 con 0x7fe430154710 2023-11-06T22:18:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.547+0000 7fe417fff700 1 -- 172.21.15.33:0/3455926623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42800f040 con 0x7fe430154710 2023-11-06T22:18:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.547+0000 7fe4365ff700 1 -- 172.21.15.33:0/3455926623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4200092a0 con 0x7fe430154710 2023-11-06T22:18:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.547+0000 7fe417fff700 1 -- 172.21.15.33:0/3455926623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42800cb70 con 0x7fe430154710 2023-11-06T22:18:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.547+0000 7fe4365ff700 1 -- 172.21.15.33:0/3455926623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4301557f0 con 0x7fe430154710 2023-11-06T22:18:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.548+0000 7fe4365ff700 1 -- 172.21.15.33:0/3455926623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4300ca440 con 0x7fe430154710 2023-11-06T22:18:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.550+0000 7fe417fff700 1 -- 172.21.15.33:0/3455926623 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe428014460 con 0x7fe430154710 2023-11-06T22:18:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.550+0000 7fe417fff700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe41c05f270 0x7fe41c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.550+0000 7fe417fff700 1 -- 172.21.15.33:0/3455926623 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5943+0+0 (secure 0 0 0) 0x7fe42807a220 con 0x7fe430154710 2023-11-06T22:18:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.550+0000 7fe42ffff700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe41c05f270 0x7fe41c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:16.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.551+0000 7fe42ffff700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe41c05f270 0x7fe41c061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe420009fd0 tx=0x7fe420017000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:16.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.553+0000 7fe417fff700 1 -- 172.21.15.33:0/3455926623 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe42804aaf0 con 0x7fe430154710 2023-11-06T22:18:16.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.737+0000 7fe4365ff700 1 -- 172.21.15.33:0/3455926623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe4300652d0 con 0x7fe430154710 2023-11-06T22:18:16.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.742+0000 7fe417fff700 1 -- 172.21.15.33:0/3455926623 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe42804b3b0 con 0x7fe430154710 2023-11-06T22:18:16.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 -- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe41c05f270 msgr2=0x7fe41c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:16.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe41c05f270 0x7fe41c061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe420009fd0 tx=0x7fe420017000).stop 2023-11-06T22:18:16.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 -- 172.21.15.33:0/3455926623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430154710 msgr2=0x7fe4300c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:16.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430154710 0x7fe4300c1ee0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7fe428009f40 tx=0x7fe42800b040).stop 2023-11-06T22:18:16.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 -- 172.21.15.33:0/3455926623 shutdown_connections 2023-11-06T22:18:16.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe41c05f270 0x7fe41c061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430154710 0x7fe4300c1ee0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300c7ff0 0x7fe430154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 --2- 172.21.15.33:0/3455926623 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe430154330 0x7fe430153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:16.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 -- 172.21.15.33:0/3455926623 >> 172.21.15.33:0/3455926623 conn(0x7fe4300cb910 msgr2=0x7fe43015a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 -- 172.21.15.33:0/3455926623 shutdown_connections 2023-11-06T22:18:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:16.745+0000 7fe4365ff700 1 -- 172.21.15.33:0/3455926623 wait complete. 2023-11-06T22:18:16.765 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T22:18:16.766 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:18:16.852 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:18:16.949 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T22:18:16.949 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T22:18:16.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:17.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.144+0000 7ff760dff700 1 Processor -- start 2023-11-06T22:18:17.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.144+0000 7ff760dff700 1 -- start start 2023-11-06T22:18:17.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.145+0000 7ff760dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff75c151dc0 0x7ff75c0c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.145+0000 7ff760dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c154e90 con 0x7ff75c1521a0 2023-11-06T22:18:17.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.145+0000 7ff760dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0639d0 con 0x7ff75c155320 2023-11-06T22:18:17.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.145+0000 7ff760dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c063cb0 con 0x7ff75c151dc0 2023-11-06T22:18:17.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff759d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff75c1521a0 0x7ff75c063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44476/0 (socket says 172.21.15.33:44476) 2023-11-06T22:18:17.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff759d9b700 1 -- 172.21.15.33:0/3956258235 learned_addr learned my addr 172.21.15.33:0/3956258235 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:17.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff75a59c700 1 --2- 172.21.15.33:0/3956258235 >> v2:172.21.15.69:3300/0 conn(0x7ff75c151dc0 0x7ff75c0c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183222850 0 0) 0x7ff75c154e90 con 0x7ff75c1521a0 2023-11-06T22:18:17.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7440033f0 con 0x7ff75c1521a0 2023-11-06T22:18:17.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2377691898 0 0) 0x7ff7440033f0 con 0x7ff75c1521a0 2023-11-06T22:18:17.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 >> v2:172.21.15.69:3300/0 conn(0x7ff75c151dc0 msgr2=0x7ff75c0c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:17.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff75959a700 1 --2- 172.21.15.33:0/3956258235 >> v2:172.21.15.69:3300/0 conn(0x7ff75c151dc0 0x7ff75c0c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 >> v1:172.21.15.69:6789/0 conn(0x7ff75c155320 legacy=0x7ff75c0c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:17.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c063f90 con 0x7ff75c1521a0 2023-11-06T22:18:17.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.146+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7ff750001470 con 0x7ff75c1521a0 2023-11-06T22:18:17.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 >> v1:172.21.15.33:6789/0 conn(0x7ff75c1521a0 legacy=0x7ff75c063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff75959a700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7440038e0 0x7ff744003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff75959a700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff744004690 0x7ff744004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff75959a700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff744005000 0x7ff74400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c063cb0 con 0x7ff744004690 2023-11-06T22:18:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74400beb0 con 0x7ff7440038e0 2023-11-06T22:18:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74400c070 con 0x7ff744005000 2023-11-06T22:18:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff759d9b700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff744004690 0x7ff744004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff75a59c700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7440038e0 0x7ff744003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff75ad9d700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff744005000 0x7ff74400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff759d9b700 1 -- 172.21.15.33:0/3956258235 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff744005000 msgr2=0x7ff74400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:17.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff759d9b700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff744005000 0x7ff74400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff759d9b700 1 -- 172.21.15.33:0/3956258235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7440038e0 msgr2=0x7ff744003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:17.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff759d9b700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7440038e0 0x7ff744003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff759d9b700 1 -- 172.21.15.33:0/3956258235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440033f0 con 0x7ff744004690 2023-11-06T22:18:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff75ad9d700 1 -- 172.21.15.33:0/3956258235 reap_dead start 2023-11-06T22:18:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.147+0000 7ff759d9b700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff744004690 0x7ff744004b10 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7ff75000c190 tx=0x7ff750001e50).ready entity=mon.0 client_cookie=8f275462818f2a5b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.148+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750007a20 con 0x7ff744004690 2023-11-06T22:18:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.148+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75000cac0 con 0x7ff744004690 2023-11-06T22:18:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.148+0000 7ff75959a700 1 -- 172.21.15.33:0/3956258235 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75000ce40 con 0x7ff744004690 2023-11-06T22:18:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.148+0000 7ff760dff700 1 -- 172.21.15.33:0/3956258235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff744004690 msgr2=0x7ff744004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.148+0000 7ff760dff700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff744004690 0x7ff744004b10 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7ff75000c190 tx=0x7ff750001e50).stop 2023-11-06T22:18:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.148+0000 7ff760dff700 1 -- 172.21.15.33:0/3956258235 shutdown_connections 2023-11-06T22:18:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.148+0000 7ff760dff700 1 --2- 172.21.15.33:0/3956258235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff744004690 0x7ff744004b10 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.148+0000 7ff760dff700 1 -- 172.21.15.33:0/3956258235 >> 172.21.15.33:0/3956258235 conn(0x7ff75c05e1b0 msgr2=0x7ff75c05e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.148+0000 7ff760dff700 1 -- 172.21.15.33:0/3956258235 shutdown_connections 2023-11-06T22:18:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.148+0000 7ff760dff700 1 -- 172.21.15.33:0/3956258235 wait complete. 2023-11-06T22:18:17.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.150+0000 7ff760dff700 1 Processor -- start 2023-11-06T22:18:17.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.150+0000 7ff760dff700 1 -- start start 2023-11-06T22:18:17.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff760dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff744004690 0x7ff75c0c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff760dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c0c2890 0x7ff75c1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff760dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff75c0af6a0 0x7ff75c0afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff760dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1589b0 con 0x7ff75c0af6a0 2023-11-06T22:18:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff760dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0c4290 con 0x7ff75c0c2890 2023-11-06T22:18:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff760dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c158b40 con 0x7ff744004690 2023-11-06T22:18:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff75ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff75c0af6a0 0x7ff75c0afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff75ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff75c0af6a0 0x7ff75c0afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37322/0 (socket says 172.21.15.33:37322) 2023-11-06T22:18:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff75ad9d700 1 -- 172.21.15.33:0/2693467040 learned_addr learned my addr 172.21.15.33:0/2693467040 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff75a59c700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff744004690 0x7ff75c0c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.151+0000 7ff759d9b700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c0c2890 0x7ff75c1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff75ad9d700 1 -- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff744004690 msgr2=0x7ff75c0c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff75ad9d700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff744004690 0x7ff75c0c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff75ad9d700 1 -- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c0c2890 msgr2=0x7ff75c1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff75ad9d700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c0c2890 0x7ff75c1514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff75ad9d700 1 -- 172.21.15.33:0/2693467040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0b0010 con 0x7ff75c0af6a0 2023-11-06T22:18:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff75ad9d700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff75c0af6a0 0x7ff75c0afb20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7ff75c084280 tx=0x7ff754009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff74a7fc700 1 -- 172.21.15.33:0/2693467040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400ee80 con 0x7ff75c0af6a0 2023-11-06T22:18:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff74a7fc700 1 -- 172.21.15.33:0/2693467040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff754003030 con 0x7ff75c0af6a0 2023-11-06T22:18:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff74a7fc700 1 -- 172.21.15.33:0/2693467040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7540033b0 con 0x7ff75c0af6a0 2023-11-06T22:18:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff760dff700 1 -- 172.21.15.33:0/2693467040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c0c2d10 con 0x7ff75c0af6a0 2023-11-06T22:18:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.152+0000 7ff760dff700 1 -- 172.21.15.33:0/2693467040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c0c32b0 con 0x7ff75c0af6a0 2023-11-06T22:18:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.153+0000 7ff760dff700 1 -- 172.21.15.33:0/2693467040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c06bc10 con 0x7ff75c0af6a0 2023-11-06T22:18:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.154+0000 7ff74a7fc700 1 -- 172.21.15.33:0/2693467040 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff7540019d0 con 0x7ff75c0af6a0 2023-11-06T22:18:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.155+0000 7ff74a7fc700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff73c05f2a0 0x7ff73c061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.155+0000 7ff74a7fc700 1 -- 172.21.15.33:0/2693467040 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5943+0+0 (secure 0 0 0) 0x7ff7540796c0 con 0x7ff75c0af6a0 2023-11-06T22:18:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.155+0000 7ff75a59c700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff73c05f2a0 0x7ff73c061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.156+0000 7ff75a59c700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff73c05f2a0 0x7ff73c061760 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff74c006d80 tx=0x7ff74c009060).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.158+0000 7ff74a7fc700 1 -- 172.21.15.33:0/2693467040 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff75404e230 con 0x7ff75c0af6a0 2023-11-06T22:18:17.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.387+0000 7ff760dff700 1 -- 172.21.15.33:0/2693467040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff75c158cd0 con 0x7ff75c0af6a0 2023-11-06T22:18:17.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.387+0000 7ff74a7fc700 1 -- 172.21.15.33:0/2693467040 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+3529 (secure 0 0 0) 0x7ff75404de20 con 0x7ff75c0af6a0 2023-11-06T22:18:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:17.390 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":86,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3815920714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3815920714},{"type":"v1","addr":"172.21.15.69:6836","nonce":3815920714}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9052,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3614716748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3614716748},{"type":"v1","addr":"172.21.15.69:6837","nonce":3614716748}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1513187241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1513187241},{"type":"v1","addr":"172.21.15.33:6834","nonce":1513187241}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9753,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3226030536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3226030536},{"type":"v1","addr":"172.21.15.33:6835","nonce":3226030536}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[]} 2023-11-06T22:18:17.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.389+0000 7ff760dff700 1 -- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff73c05f2a0 msgr2=0x7ff73c061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:17.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.389+0000 7ff760dff700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff73c05f2a0 0x7ff73c061760 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff74c006d80 tx=0x7ff74c009060).stop 2023-11-06T22:18:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.389+0000 7ff760dff700 1 -- 172.21.15.33:0/2693467040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff75c0af6a0 msgr2=0x7ff75c0afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.389+0000 7ff760dff700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff75c0af6a0 0x7ff75c0afb20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7ff75c084280 tx=0x7ff754009040).stop 2023-11-06T22:18:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.389+0000 7ff760dff700 1 -- 172.21.15.33:0/2693467040 shutdown_connections 2023-11-06T22:18:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.389+0000 7ff760dff700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff73c05f2a0 0x7ff73c061760 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.389+0000 7ff760dff700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff744004690 0x7ff75c0c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.389+0000 7ff760dff700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff75c0af6a0 0x7ff75c0afb20 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.389+0000 7ff760dff700 1 --2- 172.21.15.33:0/2693467040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff75c0c2890 0x7ff75c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.389+0000 7ff760dff700 1 -- 172.21.15.33:0/2693467040 >> 172.21.15.33:0/2693467040 conn(0x7ff75c05e1b0 msgr2=0x7ff75c153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:17.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.390+0000 7ff760dff700 1 -- 172.21.15.33:0/2693467040 shutdown_connections 2023-11-06T22:18:17.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.390+0000 7ff760dff700 1 -- 172.21.15.33:0/2693467040 wait complete. 2023-11-06T22:18:17.394 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 86 2023-11-06T22:18:17.409 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T22:18:17.409 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T22:18:17.410 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T22:18:17.410 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:18:17.410 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T22:18:17.411 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T22:18:17.411 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T22:18:17.411 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:18:17.411 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T22:18:17.412 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T22:18:17.412 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T22:18:17.412 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:18:17.413 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T22:18:17.413 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T22:18:17.413 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T22:18:17.414 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:18:17.423 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T22:18:17.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T22:18:17.425 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T22:18:17.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T22:18:17.429 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T22:18:17.429 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T22:18:17.431 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T22:18:17.431 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T22:18:17.470 INFO:tasks.ceph.mds.a:Started 2023-11-06T22:18:17.470 DEBUG:teuthology.parallel:result is None 2023-11-06T22:18:17.473 INFO:tasks.ceph.mds.c:Started 2023-11-06T22:18:17.474 DEBUG:teuthology.parallel:result is None 2023-11-06T22:18:17.475 INFO:tasks.ceph.mds.b:Started 2023-11-06T22:18:17.475 DEBUG:teuthology.parallel:result is None 2023-11-06T22:18:17.479 INFO:tasks.ceph.mds.d:Started 2023-11-06T22:18:17.479 DEBUG:teuthology.parallel:result is None 2023-11-06T22:18:17.480 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T22:18:17.619 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-11-06T22:18:17.620 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-11-06T22:18:17.629 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T22:18:17.630 INFO:tasks.ceph.mds.b.smithi069.stdout:starting mds.b at 2023-11-06T22:18:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.690+0000 7f9ac8bff700 1 Processor -- start 2023-11-06T22:18:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.690+0000 7f9ac8bff700 1 -- start start 2023-11-06T22:18:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.691+0000 7f9ac8bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9ac4069f80 0x7f9ac40c1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.691+0000 7f9ac8bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ac406a360 0x7f9ac4151810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.691+0000 7f9ac8bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac40c19f0 0x7f9ac40c1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.691+0000 7f9ac8bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac406a810 con 0x7f9ac406a360 2023-11-06T22:18:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.691+0000 7f9ac8bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac4155010 con 0x7f9ac4069f80 2023-11-06T22:18:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.691+0000 7f9ac8bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40c2340 con 0x7f9ac40c19f0 2023-11-06T22:18:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ac406a360 0x7f9ac4151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ac406a360 0x7f9ac4151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37378/0 (socket says 172.21.15.33:37378) 2023-11-06T22:18:17.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac1d9b700 1 -- 172.21.15.33:0/1464740604 learned_addr learned my addr 172.21.15.33:0/1464740604 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:17.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac259c700 1 --2- 172.21.15.33:0/1464740604 >> v2:172.21.15.69:3300/0 conn(0x7f9ac4069f80 0x7f9ac40c1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac2d9d700 1 --2- 172.21.15.33:0/1464740604 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac40c19f0 0x7f9ac40c1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac1d9b700 1 -- 172.21.15.33:0/1464740604 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac40c19f0 msgr2=0x7f9ac40c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:17.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac1d9b700 1 --2- 172.21.15.33:0/1464740604 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac40c19f0 0x7f9ac40c1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac1d9b700 1 -- 172.21.15.33:0/1464740604 >> v2:172.21.15.69:3300/0 conn(0x7f9ac4069f80 msgr2=0x7f9ac40c1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac1d9b700 1 --2- 172.21.15.33:0/1464740604 >> v2:172.21.15.69:3300/0 conn(0x7f9ac4069f80 0x7f9ac40c1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac1d9b700 1 -- 172.21.15.33:0/1464740604 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4154440 con 0x7f9ac406a360 2023-11-06T22:18:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac259c700 1 --2- 172.21.15.33:0/1464740604 >> v2:172.21.15.69:3300/0 conn(0x7f9ac4069f80 0x7f9ac40c1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.692+0000 7f9ac1d9b700 1 --2- 172.21.15.33:0/1464740604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac406a360 0x7f9ac4151810 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 rx=0x7f9ac4066410 tx=0x7f9ab80090f0).ready entity=mon.0 client_cookie=bb536b38d5fdf6d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac159a700 1 -- 172.21.15.33:0/1464740604 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800f510 con 0x7f9ac406a360 2023-11-06T22:18:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac159a700 1 -- 172.21.15.33:0/1464740604 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab800f670 con 0x7f9ac406a360 2023-11-06T22:18:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac159a700 1 -- 172.21.15.33:0/1464740604 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800f9f0 con 0x7f9ac406a360 2023-11-06T22:18:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac8bff700 1 -- 172.21.15.33:0/1464740604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac406a360 msgr2=0x7f9ac4151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac8bff700 1 --2- 172.21.15.33:0/1464740604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac406a360 0x7f9ac4151810 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 rx=0x7f9ac4066410 tx=0x7f9ab80090f0).stop 2023-11-06T22:18:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac8bff700 1 -- 172.21.15.33:0/1464740604 shutdown_connections 2023-11-06T22:18:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac8bff700 1 --2- 172.21.15.33:0/1464740604 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac40c19f0 0x7f9ac40c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac8bff700 1 --2- 172.21.15.33:0/1464740604 >> v2:172.21.15.69:3300/0 conn(0x7f9ac4069f80 0x7f9ac40c1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac8bff700 1 --2- 172.21.15.33:0/1464740604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac406a360 0x7f9ac4151810 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac8bff700 1 -- 172.21.15.33:0/1464740604 >> 172.21.15.33:0/1464740604 conn(0x7f9ac4062710 msgr2=0x7f9ac4062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac8bff700 1 -- 172.21.15.33:0/1464740604 shutdown_connections 2023-11-06T22:18:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.693+0000 7f9ac8bff700 1 -- 172.21.15.33:0/1464740604 wait complete. 2023-11-06T22:18:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.695+0000 7f9ac8bff700 1 Processor -- start 2023-11-06T22:18:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.696+0000 7f9ac8bff700 1 -- start start 2023-11-06T22:18:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac8bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac4069f80 0x7f9ac4146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac8bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ac406a360 0x7f9ac40bf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac8bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac40c19f0 0x7f9ac40bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac8bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4148600 con 0x7f9ac40c19f0 2023-11-06T22:18:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac8bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4146e40 con 0x7f9ac406a360 2023-11-06T22:18:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac8bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40bffc0 con 0x7f9ac4069f80 2023-11-06T22:18:17.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac40c19f0 0x7f9ac40bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac259c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac4069f80 0x7f9ac4146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ac406a360 0x7f9ac40bf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac259c700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac4069f80 0x7f9ac4146950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:35370/0 (socket says 172.21.15.33:35370) 2023-11-06T22:18:17.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.697+0000 7f9ac259c700 1 -- 172.21.15.33:0/2565595259 learned_addr learned my addr 172.21.15.33:0/2565595259 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:17.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ac259c700 1 -- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ac406a360 msgr2=0x7f9ac40bf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:17.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ac259c700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ac406a360 0x7f9ac40bf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ac259c700 1 -- 172.21.15.33:0/2565595259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac40c19f0 msgr2=0x7f9ac40bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:17.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ac259c700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac40c19f0 0x7f9ac40bdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:17.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ac259c700 1 -- 172.21.15.33:0/2565595259 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40092a0 con 0x7f9ac4069f80 2023-11-06T22:18:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ac2d9d700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac40c19f0 0x7f9ac40bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ac259c700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac4069f80 0x7f9ac4146950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f9ab400e980 tx=0x7f9ab4009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ab27fc700 1 -- 172.21.15.33:0/2565595259 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4012600 con 0x7f9ac4069f80 2023-11-06T22:18:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ab27fc700 1 -- 172.21.15.33:0/2565595259 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab4012760 con 0x7f9ac4069f80 2023-11-06T22:18:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ac8bff700 1 -- 172.21.15.33:0/2565595259 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab8009370 con 0x7f9ac4069f80 2023-11-06T22:18:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ab27fc700 1 -- 172.21.15.33:0/2565595259 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4012ae0 con 0x7f9ac4069f80 2023-11-06T22:18:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.698+0000 7f9ac8bff700 1 -- 172.21.15.33:0/2565595259 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac40c0580 con 0x7f9ac4069f80 2023-11-06T22:18:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.699+0000 7f9ac8bff700 1 -- 172.21.15.33:0/2565595259 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac40c07a0 con 0x7f9ac4069f80 2023-11-06T22:18:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.700+0000 7f9ab27fc700 1 -- 172.21.15.33:0/2565595259 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9ab401b400 con 0x7f9ac4069f80 2023-11-06T22:18:17.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.701+0000 7f9ab27fc700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9aa405f270 0x7f9aa4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:17.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.701+0000 7f9ab27fc700 1 -- 172.21.15.33:0/2565595259 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5915+0+0 (secure 0 0 0) 0x7f9ab407a7b0 con 0x7f9ac4069f80 2023-11-06T22:18:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.701+0000 7f9ac1d9b700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9aa405f270 0x7f9aa4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.702+0000 7f9ac1d9b700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9aa405f270 0x7f9aa4061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9ab8008000 tx=0x7f9ab8017000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.706+0000 7f9ab27fc700 1 -- 172.21.15.33:0/2565595259 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9ab404b970 con 0x7f9ac4069f80 2023-11-06T22:18:17.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:17.900+0000 7f9ac8bff700 1 -- 172.21.15.33:0/2565595259 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9ac4063c00 con 0x7f9ac4069f80 2023-11-06T22:18:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.661+0000 7f9ab27fc700 1 -- 172.21.15.33:0/2565595259 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v219) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9ab404ef90 con 0x7f9ac4069f80 2023-11-06T22:18:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 -- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9aa405f270 msgr2=0x7f9aa4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9aa405f270 0x7f9aa4061730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9ab8008000 tx=0x7f9ab8017000).stop 2023-11-06T22:18:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 -- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac4069f80 msgr2=0x7f9ac4146950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac4069f80 0x7f9ac4146950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f9ab400e980 tx=0x7f9ab4009060).stop 2023-11-06T22:18:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 -- 172.21.15.33:0/2565595259 shutdown_connections 2023-11-06T22:18:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9aa405f270 0x7f9aa4061730 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9ac4069f80 0x7f9ac4146950 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ac40c19f0 0x7f9ac40bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 --2- 172.21.15.33:0/2565595259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ac406a360 0x7f9ac40bf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 -- 172.21.15.33:0/2565595259 >> 172.21.15.33:0/2565595259 conn(0x7f9ac4062710 msgr2=0x7f9ac40646e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 -- 172.21.15.33:0/2565595259 shutdown_connections 2023-11-06T22:18:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.664+0000 7f9ac8bff700 1 -- 172.21.15.33:0/2565595259 wait complete. 2023-11-06T22:18:18.668 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-11-06T22:18:18.685 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T22:18:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.884+0000 7f29263ff700 1 Processor -- start 2023-11-06T22:18:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.884+0000 7f29263ff700 1 -- start start 2023-11-06T22:18:18.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.886+0000 7f29263ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2920063140 0x7f2920063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:18.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.886+0000 7f29263ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f292005e1b0 0x7f29200c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:18.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.886+0000 7f29263ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29200c31f0 con 0x7f29200c4490 2023-11-06T22:18:18.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.886+0000 7f29263ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f292006a0f0 con 0x7f2920063140 2023-11-06T22:18:18.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.886+0000 7f29263ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200c2920 con 0x7f292005e1b0 2023-11-06T22:18:18.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.886+0000 7f291ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2920063140 0x7f2920063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.886+0000 7f292499c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f29200c4490 0x7f2920154050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55536/0 (socket says 172.21.15.33:55536) 2023-11-06T22:18:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.886+0000 7f291ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2920063140 0x7f2920063520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48974/0 (socket says 172.21.15.33:48974) 2023-11-06T22:18:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.886+0000 7f292499c700 1 -- 172.21.15.33:0/433802644 learned_addr learned my addr 172.21.15.33:0/433802644 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.886+0000 7f291f7fe700 1 --2- 172.21.15.33:0/433802644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f292005e1b0 0x7f29200c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.887+0000 7f291ffff700 1 -- 172.21.15.33:0/433802644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f292005e1b0 msgr2=0x7f29200c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.887+0000 7f291ffff700 1 --2- 172.21.15.33:0/433802644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f292005e1b0 0x7f29200c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.887+0000 7f291ffff700 1 -- 172.21.15.33:0/433802644 >> v1:172.21.15.33:6789/0 conn(0x7f29200c4490 legacy=0x7f2920154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.887+0000 7f291ffff700 1 -- 172.21.15.33:0/433802644 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29200c3b70 con 0x7f2920063140 2023-11-06T22:18:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.887+0000 7f291effd700 1 -- 172.21.15.33:0/433802644 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023776195 0 0) 0x7f29200c31f0 con 0x7f29200c4490 2023-11-06T22:18:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.887+0000 7f291ffff700 1 --2- 172.21.15.33:0/433802644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 0x7f2920063520 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7f2910009760 tx=0x7f2910009060).ready entity=mon.0 client_cookie=6a0d5cd6fe60902d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.887+0000 7f291effd700 1 -- 172.21.15.33:0/433802644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291000f510 con 0x7f2920063140 2023-11-06T22:18:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.887+0000 7f291effd700 1 -- 172.21.15.33:0/433802644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291000f670 con 0x7f2920063140 2023-11-06T22:18:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.887+0000 7f291effd700 1 -- 172.21.15.33:0/433802644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291000f9f0 con 0x7f2920063140 2023-11-06T22:18:18.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.888+0000 7f29263ff700 1 -- 172.21.15.33:0/433802644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 msgr2=0x7f2920063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:18.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.888+0000 7f29263ff700 1 --2- 172.21.15.33:0/433802644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 0x7f2920063520 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 rx=0x7f2910009760 tx=0x7f2910009060).stop 2023-11-06T22:18:18.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.888+0000 7f29263ff700 1 -- 172.21.15.33:0/433802644 shutdown_connections 2023-11-06T22:18:18.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.888+0000 7f29263ff700 1 --2- 172.21.15.33:0/433802644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 0x7f2920063520 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:18.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.888+0000 7f29263ff700 1 --2- 172.21.15.33:0/433802644 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f292005e1b0 0x7f29200c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:18.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.888+0000 7f29263ff700 1 -- 172.21.15.33:0/433802644 >> 172.21.15.33:0/433802644 conn(0x7f29200c7d60 msgr2=0x7f29200c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:18.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.888+0000 7f29263ff700 1 -- 172.21.15.33:0/433802644 shutdown_connections 2023-11-06T22:18:18.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.888+0000 7f29263ff700 1 -- 172.21.15.33:0/433802644 wait complete. 2023-11-06T22:18:18.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.889+0000 7f29263ff700 1 Processor -- start 2023-11-06T22:18:18.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.889+0000 7f29263ff700 1 -- start start 2023-11-06T22:18:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.890+0000 7f29263ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 0x7f292014abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.890+0000 7f29263ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29200c4490 0x7f2920149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.890+0000 7f29263ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29201528e0 0x7f2920152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.890+0000 7f29263ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920158d80 con 0x7f2920063140 2023-11-06T22:18:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.890+0000 7f29263ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920153230 con 0x7f29200c4490 2023-11-06T22:18:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.890+0000 7f29263ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2920158170 con 0x7f29201528e0 2023-11-06T22:18:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f291ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 0x7f292014abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f291ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 0x7f292014abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48988/0 (socket says 172.21.15.33:48988) 2023-11-06T22:18:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f291ffff700 1 -- 172.21.15.33:0/3936492107 learned_addr learned my addr 172.21.15.33:0/3936492107 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f292499c700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29201528e0 0x7f2920152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f291f7fe700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29200c4490 0x7f2920149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f291ffff700 1 -- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29201528e0 msgr2=0x7f2920152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f291ffff700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29201528e0 0x7f2920152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f291ffff700 1 -- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29200c4490 msgr2=0x7f2920149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f291ffff700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29200c4490 0x7f2920149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f291ffff700 1 -- 172.21.15.33:0/3936492107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100092a0 con 0x7f2920063140 2023-11-06T22:18:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f291ffff700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 0x7f292014abd0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7f292005ebe0 tx=0x7f2910017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:18.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f2907fff700 1 -- 172.21.15.33:0/3936492107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291000f7e0 con 0x7f2920063140 2023-11-06T22:18:18.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.891+0000 7f2907fff700 1 -- 172.21.15.33:0/3936492107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291000f940 con 0x7f2920063140 2023-11-06T22:18:18.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.892+0000 7f29263ff700 1 -- 172.21.15.33:0/3936492107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2920158450 con 0x7f2920063140 2023-11-06T22:18:18.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.892+0000 7f2907fff700 1 -- 172.21.15.33:0/3936492107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291001b660 con 0x7f2920063140 2023-11-06T22:18:18.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.892+0000 7f29263ff700 1 -- 172.21.15.33:0/3936492107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2920158a10 con 0x7f2920063140 2023-11-06T22:18:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.893+0000 7f29263ff700 1 -- 172.21.15.33:0/3936492107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2920064420 con 0x7f2920063140 2023-11-06T22:18:18.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.894+0000 7f2907fff700 1 -- 172.21.15.33:0/3936492107 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f291001ba50 con 0x7f2920063140 2023-11-06T22:18:18.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.894+0000 7f2907fff700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f290c05f270 0x7f290c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:18.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.895+0000 7f2907fff700 1 -- 172.21.15.33:0/3936492107 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f291007bc00 con 0x7f2920063140 2023-11-06T22:18:18.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.895+0000 7f291f7fe700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f290c05f270 0x7f290c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:18.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.896+0000 7f291f7fe700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f290c05f270 0x7f290c061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f29200c53b0 tx=0x7f2914008040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:18.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:18.899+0000 7f2907fff700 1 -- 172.21.15.33:0/3936492107 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f291004d070 con 0x7f2920063140 2023-11-06T22:18:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.149+0000 7f29263ff700 1 -- 172.21.15.33:0/3936492107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f29200c4270 con 0x7f2920063140 2023-11-06T22:18:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.149+0000 7f2907fff700 1 -- 172.21.15.33:0/3936492107 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v48) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2910050690 con 0x7f2920063140 2023-11-06T22:18:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:19.152 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:18:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-11-06T22:18:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-11-06T22:18:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llLtciLxAAK4hQio8IdxWu4s/oFJA6rg==", 2023-11-06T22:18:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:18:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:18:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:18:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-11-06T22:18:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llO3ePChAAU20S6caQncNfL7E8DPvdvQ==", 2023-11-06T22:18:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:18:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:18:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:18:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-11-06T22:18:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llRYi5OhAAsYUYqZYXNOSDu84rl06s+w==", 2023-11-06T22:18:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:18:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:18:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:18:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-11-06T22:18:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0lllVXbFRAAxOa7Z3FtnA/BqBvbUmRtlg==", 2023-11-06T22:18:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:18:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:18:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:18:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-11-06T22:18:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDgT0llpO88FxAAYAToEi5FIXZnWR2RvmROCw==", 2023-11-06T22:18:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:18:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:18:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-11-06T22:18:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDjT0llSqQCKhAA74gEhDp5bO4ug0EUcihvXg==", 2023-11-06T22:18:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:18:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:18:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-11-06T22:18:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnT0llvznRABAAjNI7H9aorMjUmk87FtsjKg==", 2023-11-06T22:18:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:18:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:18:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-11-06T22:18:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqT0lljxSgExAA++2c35HoqsMyDMrK5OQenQ==", 2023-11-06T22:18:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:18:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:18:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-11-06T22:18:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwT0lle8aDOBAAgOzY0/KjX6vK35c/kjK7fw==", 2023-11-06T22:18:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:18:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:18:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-11-06T22:18:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD0T0lleGx5DxAAE9E+PbX8xXkLRpSqpbZVxw==", 2023-11-06T22:18:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:18:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:18:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-11-06T22:18:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD3T0llJF6yIRAAYOpRLtVm4ldMnge5W0+Ssg==", 2023-11-06T22:18:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:18:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:18:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-11-06T22:18:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD6T0ll7vgeMxAA4ODVFp8fKE78yhmtWlWIvA==", 2023-11-06T22:18:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-11-06T22:18:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-11-06T22:18:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-11-06T22:18:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llxPL9HxAAgOOCocaO0fyYcCpbzrwJcA==", 2023-11-06T22:18:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:18:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-11-06T22:18:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T22:18:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-11-06T22:18:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0ll8xSZLhAAoAF5Lo4Tk4trsEYidhsy7A==", 2023-11-06T22:18:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:18:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T22:18:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T22:18:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T22:18:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-11-06T22:18:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llXZJaJxAA5tWN2j2sOWk2jwopk+P7Lg==", 2023-11-06T22:18:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:18:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T22:18:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T22:18:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T22:18:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-11-06T22:18:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDcT0llC7/DNRAAr6Nj3n27UEjoeVvQVk6ShQ==", 2023-11-06T22:18:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-11-06T22:18:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-11-06T22:18:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-11-06T22:18:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-11-06T22:18:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-11-06T22:18:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDaT0llBH5bLxAA1iPOg4pkiNOzwiLKVYcEmA==", 2023-11-06T22:18:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T22:18:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-11-06T22:18:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-11-06T22:18:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-11-06T22:18:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llkcihJxAAOmCXNzAnMMFNYv5fogTSmQ==", 2023-11-06T22:18:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T22:18:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T22:18:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-11-06T22:18:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDbT0llTdiZGxAAw6CAJ6R9A5z/rqLu5AYCUg==", 2023-11-06T22:18:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-11-06T22:18:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-11-06T22:18:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-11-06T22:18:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-11-06T22:18:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-11-06T22:18:19.205 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:18:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.151+0000 7f29263ff700 1 -- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f290c05f270 msgr2=0x7f290c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.151+0000 7f29263ff700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f290c05f270 0x7f290c061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f29200c53b0 tx=0x7f2914008040).stop 2023-11-06T22:18:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.152+0000 7f29263ff700 1 -- 172.21.15.33:0/3936492107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 msgr2=0x7f292014abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.152+0000 7f29263ff700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 0x7f292014abd0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7f292005ebe0 tx=0x7f2910017000).stop 2023-11-06T22:18:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.152+0000 7f29263ff700 1 -- 172.21.15.33:0/3936492107 shutdown_connections 2023-11-06T22:18:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.152+0000 7f29263ff700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f290c05f270 0x7f290c061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.152+0000 7f29263ff700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f29201528e0 0x7f2920152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.152+0000 7f29263ff700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2920063140 0x7f292014abd0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.152+0000 7f29263ff700 1 --2- 172.21.15.33:0/3936492107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29200c4490 0x7f2920149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.152+0000 7f29263ff700 1 -- 172.21.15.33:0/3936492107 >> 172.21.15.33:0/3936492107 conn(0x7f29200c7d60 msgr2=0x7f29201500f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.152+0000 7f29263ff700 1 -- 172.21.15.33:0/3936492107 shutdown_connections 2023-11-06T22:18:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.152+0000 7f29263ff700 1 -- 172.21.15.33:0/3936492107 wait complete. 2023-11-06T22:18:19.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.404+0000 7f48f85ff700 1 Processor -- start 2023-11-06T22:18:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.405+0000 7f48f85ff700 1 -- start start 2023-11-06T22:18:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.406+0000 7f48f85ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48f00c5d50 0x7f48f00c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.406+0000 7f48f85ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48f00bfd70 0x7f48f00c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.406+0000 7f48f85ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f01554a0 con 0x7f48f00c1840 2023-11-06T22:18:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.406+0000 7f48f85ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f0154960 con 0x7f48f00c5d50 2023-11-06T22:18:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.406+0000 7f48f85ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f00c66c0 con 0x7f48f00bfd70 2023-11-06T22:18:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.406+0000 7f48f5b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48f00c5d50 0x7f48f00c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.406+0000 7f48f5b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48f00c5d50 0x7f48f00c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49002/0 (socket says 172.21.15.33:49002) 2023-11-06T22:18:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.406+0000 7f48f5b9a700 1 -- 172.21.15.33:0/1330795548 learned_addr learned my addr 172.21.15.33:0/1330795548 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.406+0000 7f48f6b9c700 1 --2- 172.21.15.33:0/1330795548 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48f00bfd70 0x7f48f00c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.407+0000 7f48f5b9a700 1 -- 172.21.15.33:0/1330795548 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48f00bfd70 msgr2=0x7f48f00c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:19.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.407+0000 7f48f5b9a700 1 --2- 172.21.15.33:0/1330795548 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48f00bfd70 0x7f48f00c01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.407+0000 7f48f5b9a700 1 -- 172.21.15.33:0/1330795548 >> v1:172.21.15.69:6789/0 conn(0x7f48f00c1840 legacy=0x7f48f00a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.407+0000 7f48f5b9a700 1 -- 172.21.15.33:0/1330795548 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f0155930 con 0x7f48f00c5d50 2023-11-06T22:18:19.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.407+0000 7f48f5b9a700 1 --2- 172.21.15.33:0/1330795548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00c5d50 0x7f48f00c61d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7f48e0006fd0 tx=0x7f48e00090f0).ready entity=mon.0 client_cookie=a3c2c9a58380af82 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.407+0000 7f48f5399700 1 -- 172.21.15.33:0/1330795548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e000b580 con 0x7f48f00c5d50 2023-11-06T22:18:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.407+0000 7f48f5399700 1 -- 172.21.15.33:0/1330795548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e000b6e0 con 0x7f48f00c5d50 2023-11-06T22:18:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.407+0000 7f48f5399700 1 -- 172.21.15.33:0/1330795548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e000ba60 con 0x7f48f00c5d50 2023-11-06T22:18:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.408+0000 7f48f85ff700 1 -- 172.21.15.33:0/1330795548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00c5d50 msgr2=0x7f48f00c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.408+0000 7f48f85ff700 1 --2- 172.21.15.33:0/1330795548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00c5d50 0x7f48f00c61d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7f48e0006fd0 tx=0x7f48e00090f0).stop 2023-11-06T22:18:19.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.408+0000 7f48f85ff700 1 -- 172.21.15.33:0/1330795548 shutdown_connections 2023-11-06T22:18:19.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.408+0000 7f48f85ff700 1 --2- 172.21.15.33:0/1330795548 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48f00bfd70 0x7f48f00c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.408+0000 7f48f85ff700 1 --2- 172.21.15.33:0/1330795548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00c5d50 0x7f48f00c61d0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.408+0000 7f48f85ff700 1 -- 172.21.15.33:0/1330795548 >> 172.21.15.33:0/1330795548 conn(0x7f48f005ec80 msgr2=0x7f48f00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:19.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.408+0000 7f48f85ff700 1 -- 172.21.15.33:0/1330795548 shutdown_connections 2023-11-06T22:18:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.408+0000 7f48f85ff700 1 -- 172.21.15.33:0/1330795548 wait complete. 2023-11-06T22:18:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.409+0000 7f48f85ff700 1 Processor -- start 2023-11-06T22:18:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.410+0000 7f48f85ff700 1 -- start start 2023-11-06T22:18:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.410+0000 7f48f85ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00bfd70 0x7f48f014e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.410+0000 7f48f85ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48f00c1840 0x7f48f00c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.410+0000 7f48f85ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48f00c5d50 0x7f48f0165050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.410+0000 7f48f85ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f0164bc0 con 0x7f48f00bfd70 2023-11-06T22:18:19.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.410+0000 7f48f85ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f01645e0 con 0x7f48f00c5d50 2023-11-06T22:18:19.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.410+0000 7f48f85ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f0165540 con 0x7f48f00c1840 2023-11-06T22:18:19.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f639b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00bfd70 0x7f48f014e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f639b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00bfd70 0x7f48f014e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49014/0 (socket says 172.21.15.33:49014) 2023-11-06T22:18:19.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f639b700 1 -- 172.21.15.33:0/731650827 learned_addr learned my addr 172.21.15.33:0/731650827 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:19.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f5b9a700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48f00c1840 0x7f48f00c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f6b9c700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48f00c5d50 0x7f48f0165050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f639b700 1 -- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48f00c1840 msgr2=0x7f48f00c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f639b700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48f00c1840 0x7f48f00c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f639b700 1 -- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48f00c5d50 msgr2=0x7f48f0165050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f639b700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48f00c5d50 0x7f48f0165050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f639b700 1 -- 172.21.15.33:0/731650827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e0009370 con 0x7f48f00bfd70 2023-11-06T22:18:19.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.411+0000 7f48f639b700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00bfd70 0x7f48f014e690 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7f48ec00c470 tx=0x7f48ec007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:19.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.412+0000 7f48e67fc700 1 -- 172.21.15.33:0/731650827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00f470 con 0x7f48f00bfd70 2023-11-06T22:18:19.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.412+0000 7f48e67fc700 1 -- 172.21.15.33:0/731650827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec00f5d0 con 0x7f48f00bfd70 2023-11-06T22:18:19.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.412+0000 7f48e67fc700 1 -- 172.21.15.33:0/731650827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00f970 con 0x7f48f00bfd70 2023-11-06T22:18:19.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.412+0000 7f48f85ff700 1 -- 172.21.15.33:0/731650827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f00c3790 con 0x7f48f00bfd70 2023-11-06T22:18:19.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.412+0000 7f48f85ff700 1 -- 172.21.15.33:0/731650827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f0164850 con 0x7f48f00bfd70 2023-11-06T22:18:19.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.413+0000 7f48f85ff700 1 -- 172.21.15.33:0/731650827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f00c3cb0 con 0x7f48f00bfd70 2023-11-06T22:18:19.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.414+0000 7f48e67fc700 1 -- 172.21.15.33:0/731650827 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f48ec01b3b0 con 0x7f48f00bfd70 2023-11-06T22:18:19.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.415+0000 7f48e67fc700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f48d405f270 0x7f48d4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.415+0000 7f48e67fc700 1 -- 172.21.15.33:0/731650827 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f48ec079580 con 0x7f48f00bfd70 2023-11-06T22:18:19.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.415+0000 7f48f5b9a700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f48d405f270 0x7f48d4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.416+0000 7f48f5b9a700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f48d405f270 0x7f48d4061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f48e0009130 tx=0x7f48e0017000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:19.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.419+0000 7f48e67fc700 1 -- 172.21.15.33:0/731650827 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f48ec048af0 con 0x7f48f00bfd70 2023-11-06T22:18:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.598+0000 7f48f85ff700 1 -- 172.21.15.33:0/731650827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48f0164ea0 con 0x7f48f00bfd70 2023-11-06T22:18:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.599+0000 7f48e67fc700 1 -- 172.21.15.33:0/731650827 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v219) v1 ==== 74+0+13841 (secure 0 0 0) 0x7f48f00c3cb0 con 0x7f48f00bfd70 2023-11-06T22:18:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:19.601 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":219,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:18.653604+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 -- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f48d405f270 msgr2=0x7f48d4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f48d405f270 0x7f48d4061730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f48e0009130 tx=0x7f48e0017000).stop 2023-11-06T22:18:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 -- 172.21.15.33:0/731650827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00bfd70 msgr2=0x7f48f014e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00bfd70 0x7f48f014e690 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7f48ec00c470 tx=0x7f48ec007dc0).stop 2023-11-06T22:18:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 -- 172.21.15.33:0/731650827 shutdown_connections 2023-11-06T22:18:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f48d405f270 0x7f48d4061730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48f00c5d50 0x7f48f0165050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48f00bfd70 0x7f48f014e690 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 --2- 172.21.15.33:0/731650827 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f48f00c1840 0x7f48f00c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 -- 172.21.15.33:0/731650827 >> 172.21.15.33:0/731650827 conn(0x7f48f005ec80 msgr2=0x7f48f0152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 -- 172.21.15.33:0/731650827 shutdown_connections 2023-11-06T22:18:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.602+0000 7f48f85ff700 1 -- 172.21.15.33:0/731650827 wait complete. 2023-11-06T22:18:19.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-06T22:18:19.622 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T22:18:19.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.819+0000 7fc9d67ff700 1 Processor -- start 2023-11-06T22:18:19.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.819+0000 7fc9d67ff700 1 -- start start 2023-11-06T22:18:19.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.821+0000 7fc9d67ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d0154010 0x7fc9d01543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.821+0000 7fc9d67ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc9d00aafc0 0x7fc9d00c0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.821+0000 7fc9d67ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d00c6280 con 0x7fc9d00c0b70 2023-11-06T22:18:19.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.821+0000 7fc9d67ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d00c7720 con 0x7fc9d00aafc0 2023-11-06T22:18:19.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.821+0000 7fc9d67ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d00c5990 con 0x7fc9d0154010 2023-11-06T22:18:19.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9cffff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d0154010 0x7fc9d01543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9cf7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc9d00aafc0 0x7fc9d00c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9d4d9c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fc9d00c0b70 0x7fc9d0064670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:49936/0 (socket says 172.21.15.33:49936) 2023-11-06T22:18:19.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9d4d9c700 1 -- 172.21.15.33:0/1667605326 learned_addr learned my addr 172.21.15.33:0/1667605326 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:19.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9cffff700 1 -- 172.21.15.33:0/1667605326 >> v2:172.21.15.69:3300/0 conn(0x7fc9d00aafc0 msgr2=0x7fc9d00c0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:19.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9cffff700 1 --2- 172.21.15.33:0/1667605326 >> v2:172.21.15.69:3300/0 conn(0x7fc9d00aafc0 0x7fc9d00c0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9cffff700 1 -- 172.21.15.33:0/1667605326 >> v1:172.21.15.69:6789/0 conn(0x7fc9d00c0b70 legacy=0x7fc9d0064670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9cffff700 1 -- 172.21.15.33:0/1667605326 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d00c1000 con 0x7fc9d0154010 2023-11-06T22:18:19.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9ceffd700 1 -- 172.21.15.33:0/1667605326 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502804072 0 0) 0x7fc9d00c6280 con 0x7fc9d00c0b70 2023-11-06T22:18:19.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9cf7fe700 1 --2- 172.21.15.33:0/1667605326 >> v2:172.21.15.69:3300/0 conn(0x7fc9d00aafc0 0x7fc9d00c0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:19.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.822+0000 7fc9cffff700 1 --2- 172.21.15.33:0/1667605326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d0154010 0x7fc9d01543f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fc9c0009760 tx=0x7fc9c0009060).ready entity=mon.2 client_cookie=bc86d91900155acc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:19.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9ceffd700 1 -- 172.21.15.33:0/1667605326 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000f510 con 0x7fc9d0154010 2023-11-06T22:18:19.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9ceffd700 1 -- 172.21.15.33:0/1667605326 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c000f670 con 0x7fc9d0154010 2023-11-06T22:18:19.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9ceffd700 1 -- 172.21.15.33:0/1667605326 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000f9f0 con 0x7fc9d0154010 2023-11-06T22:18:19.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9d67ff700 1 -- 172.21.15.33:0/1667605326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d0154010 msgr2=0x7fc9d01543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9d67ff700 1 --2- 172.21.15.33:0/1667605326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d0154010 0x7fc9d01543f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fc9c0009760 tx=0x7fc9c0009060).stop 2023-11-06T22:18:19.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9d67ff700 1 -- 172.21.15.33:0/1667605326 shutdown_connections 2023-11-06T22:18:19.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9d67ff700 1 --2- 172.21.15.33:0/1667605326 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d0154010 0x7fc9d01543f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9d67ff700 1 --2- 172.21.15.33:0/1667605326 >> v2:172.21.15.69:3300/0 conn(0x7fc9d00aafc0 0x7fc9d00c0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9d67ff700 1 -- 172.21.15.33:0/1667605326 >> 172.21.15.33:0/1667605326 conn(0x7fc9d00cb8e0 msgr2=0x7fc9d00c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:19.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9d67ff700 1 -- 172.21.15.33:0/1667605326 shutdown_connections 2023-11-06T22:18:19.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.823+0000 7fc9d67ff700 1 -- 172.21.15.33:0/1667605326 wait complete. 2023-11-06T22:18:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.825+0000 7fc9d67ff700 1 Processor -- start 2023-11-06T22:18:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.825+0000 7fc9d67ff700 1 -- start start 2023-11-06T22:18:19.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.826+0000 7fc9d67ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d0154010 0x7fc9d01526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.826+0000 7fc9d67ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d00c4db0 0x7fc9d0150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.826+0000 7fc9d67ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d013f900 0x7fc9d00ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.826+0000 7fc9d67ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d00c5230 con 0x7fc9d013f900 2023-11-06T22:18:19.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.826+0000 7fc9d67ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d01515d0 con 0x7fc9d0154010 2023-11-06T22:18:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.826+0000 7fc9d67ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d0152250 con 0x7fc9d00c4db0 2023-11-06T22:18:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.826+0000 7fc9d4d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d013f900 0x7fc9d00ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.826+0000 7fc9d4d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d013f900 0x7fc9d00ca130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49028/0 (socket says 172.21.15.33:49028) 2023-11-06T22:18:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.826+0000 7fc9d4d9c700 1 -- 172.21.15.33:0/2866299861 learned_addr learned my addr 172.21.15.33:0/2866299861 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9cf7fe700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d00c4db0 0x7fc9d0150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9cffff700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d0154010 0x7fc9d01526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9d4d9c700 1 -- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d00c4db0 msgr2=0x7fc9d0150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9d4d9c700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d00c4db0 0x7fc9d0150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9d4d9c700 1 -- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d0154010 msgr2=0x7fc9d01526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:19.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9d4d9c700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d0154010 0x7fc9d01526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:19.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9d4d9c700 1 -- 172.21.15.33:0/2866299861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c40092a0 con 0x7fc9d013f900 2023-11-06T22:18:19.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9d4d9c700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d013f900 0x7fc9d00ca130 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7fc9c800c840 tx=0x7fc9c8007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:19.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9b7fff700 1 -- 172.21.15.33:0/2866299861 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8005d30 con 0x7fc9d013f900 2023-11-06T22:18:19.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9b7fff700 1 -- 172.21.15.33:0/2866299861 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c8005e90 con 0x7fc9d013f900 2023-11-06T22:18:19.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9b7fff700 1 -- 172.21.15.33:0/2866299861 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8007300 con 0x7fc9d013f900 2023-11-06T22:18:19.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.827+0000 7fc9d67ff700 1 -- 172.21.15.33:0/2866299861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c00092a0 con 0x7fc9d013f900 2023-11-06T22:18:19.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.828+0000 7fc9d67ff700 1 -- 172.21.15.33:0/2866299861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d0151920 con 0x7fc9d013f900 2023-11-06T22:18:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.829+0000 7fc9d67ff700 1 -- 172.21.15.33:0/2866299861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d0152390 con 0x7fc9d013f900 2023-11-06T22:18:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.829+0000 7fc9b7fff700 1 -- 172.21.15.33:0/2866299861 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fc9c80076b0 con 0x7fc9d013f900 2023-11-06T22:18:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.830+0000 7fc9b7fff700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc9bc05f270 0x7fc9bc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.830+0000 7fc9b7fff700 1 -- 172.21.15.33:0/2866299861 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5313+0+0 (secure 0 0 0) 0x7fc9c80799f0 con 0x7fc9d013f900 2023-11-06T22:18:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.831+0000 7fc9cffff700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc9bc05f270 0x7fc9bc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.831+0000 7fc9cffff700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc9bc05f270 0x7fc9bc061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fc9d00c4910 tx=0x7fc9c0017000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:19.834+0000 7fc9b7fff700 1 -- 172.21.15.33:0/2866299861 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc9c804ada0 con 0x7fc9d013f900 2023-11-06T22:18:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.014+0000 7fc9d67ff700 1 -- 172.21.15.33:0/2866299861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc9d00c5c30 con 0x7fc9d013f900 2023-11-06T22:18:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.015+0000 7fc9b7fff700 1 -- 172.21.15.33:0/2866299861 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v219) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc9c804e3c0 con 0x7fc9d013f900 2023-11-06T22:18:20.017 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T22:18:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 -- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc9bc05f270 msgr2=0x7fc9bc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc9bc05f270 0x7fc9bc061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fc9d00c4910 tx=0x7fc9c0017000).stop 2023-11-06T22:18:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 -- 172.21.15.33:0/2866299861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d013f900 msgr2=0x7fc9d00ca130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d013f900 0x7fc9d00ca130 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7fc9c800c840 tx=0x7fc9c8007ed0).stop 2023-11-06T22:18:20.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 -- 172.21.15.33:0/2866299861 shutdown_connections 2023-11-06T22:18:20.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fc9bc05f270 0x7fc9bc061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d013f900 0x7fc9d00ca130 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d0154010 0x7fc9d01526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 --2- 172.21.15.33:0/2866299861 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fc9d00c4db0 0x7fc9d0150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 -- 172.21.15.33:0/2866299861 >> 172.21.15.33:0/2866299861 conn(0x7fc9d00cb8e0 msgr2=0x7fc9d00abd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:20.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 -- 172.21.15.33:0/2866299861 shutdown_connections 2023-11-06T22:18:20.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.017+0000 7fc9d67ff700 1 -- 172.21.15.33:0/2866299861 wait complete. 2023-11-06T22:18:20.037 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T22:18:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.234+0000 7fa8e09ff700 1 Processor -- start 2023-11-06T22:18:20.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.234+0000 7fa8e09ff700 1 -- start start 2023-11-06T22:18:20.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.235+0000 7fa8e09ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa8dc1544f0 0x7fa8dc0c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.235+0000 7fa8e09ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8dc064dd0 0x7fa8dc0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.235+0000 7fa8e09ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc1556d0 con 0x7fa8dc0c0c40 2023-11-06T22:18:20.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.235+0000 7fa8e09ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc1548d0 con 0x7fa8dc064dd0 2023-11-06T22:18:20.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.235+0000 7fa8e09ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc154ac0 con 0x7fa8dc1544f0 2023-11-06T22:18:20.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.235+0000 7fa8dad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8dc064dd0 0x7fa8dc0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.236+0000 7fa8dad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8dc064dd0 0x7fa8dc0c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49040/0 (socket says 172.21.15.33:49040) 2023-11-06T22:18:20.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.236+0000 7fa8dad9d700 1 -- 172.21.15.33:0/1084781654 learned_addr learned my addr 172.21.15.33:0/1084781654 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:20.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.236+0000 7fa8da59c700 1 --2- 172.21.15.33:0/1084781654 >> v2:172.21.15.69:3300/0 conn(0x7fa8dc1544f0 0x7fa8dc0c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.236+0000 7fa8dad9d700 1 -- 172.21.15.33:0/1084781654 >> v2:172.21.15.69:3300/0 conn(0x7fa8dc1544f0 msgr2=0x7fa8dc0c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:20.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.236+0000 7fa8dad9d700 1 --2- 172.21.15.33:0/1084781654 >> v2:172.21.15.69:3300/0 conn(0x7fa8dc1544f0 0x7fa8dc0c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.236+0000 7fa8dad9d700 1 -- 172.21.15.33:0/1084781654 >> v1:172.21.15.69:6789/0 conn(0x7fa8dc0c0c40 legacy=0x7fa8dc0646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:20.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.236+0000 7fa8dad9d700 1 -- 172.21.15.33:0/1084781654 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc154da0 con 0x7fa8dc064dd0 2023-11-06T22:18:20.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.236+0000 7fa8dad9d700 1 --2- 172.21.15.33:0/1084781654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 0x7fa8dc0c57e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7fa8dc084660 tx=0x7fa8d4009840).ready entity=mon.0 client_cookie=a82784a7d450c897 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:20.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.236+0000 7fa8d959a700 1 -- 172.21.15.33:0/1084781654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400e070 con 0x7fa8dc064dd0 2023-11-06T22:18:20.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.236+0000 7fa8d959a700 1 -- 172.21.15.33:0/1084781654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d400ba10 con 0x7fa8dc064dd0 2023-11-06T22:18:20.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.237+0000 7fa8d959a700 1 -- 172.21.15.33:0/1084781654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400bd90 con 0x7fa8dc064dd0 2023-11-06T22:18:20.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.237+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1084781654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 msgr2=0x7fa8dc0c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.237+0000 7fa8e09ff700 1 --2- 172.21.15.33:0/1084781654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 0x7fa8dc0c57e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7fa8dc084660 tx=0x7fa8d4009840).stop 2023-11-06T22:18:20.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.237+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1084781654 shutdown_connections 2023-11-06T22:18:20.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.237+0000 7fa8e09ff700 1 --2- 172.21.15.33:0/1084781654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 0x7fa8dc0c57e0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.237+0000 7fa8e09ff700 1 --2- 172.21.15.33:0/1084781654 >> v2:172.21.15.69:3300/0 conn(0x7fa8dc1544f0 0x7fa8dc0c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.237+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1084781654 >> 172.21.15.33:0/1084781654 conn(0x7fa8dc0cb900 msgr2=0x7fa8dc0c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:20.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.237+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1084781654 shutdown_connections 2023-11-06T22:18:20.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.237+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1084781654 wait complete. 2023-11-06T22:18:20.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.239+0000 7fa8e09ff700 1 Processor -- start 2023-11-06T22:18:20.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.239+0000 7fa8e09ff700 1 -- start start 2023-11-06T22:18:20.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8e09ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 0x7fa8dc13f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8e09ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8dc0c0c40 0x7fa8dc0c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8e09ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8dc1544f0 0x7fa8dc164a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8e09ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0c45e0 con 0x7fa8dc064dd0 2023-11-06T22:18:20.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8e09ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1654f0 con 0x7fa8dc1544f0 2023-11-06T22:18:20.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8e09ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1657d0 con 0x7fa8dc0c0c40 2023-11-06T22:18:20.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8da59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 0x7fa8dc13f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8da59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 0x7fa8dc13f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49054/0 (socket says 172.21.15.33:49054) 2023-11-06T22:18:20.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8da59c700 1 -- 172.21.15.33:0/1295913066 learned_addr learned my addr 172.21.15.33:0/1295913066 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:20.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8dad9d700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8dc1544f0 0x7fa8dc164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8d9d9b700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8dc0c0c40 0x7fa8dc0c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8da59c700 1 -- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8dc0c0c40 msgr2=0x7fa8dc0c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8da59c700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8dc0c0c40 0x7fa8dc0c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8da59c700 1 -- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8dc1544f0 msgr2=0x7fa8dc164a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8da59c700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8dc1544f0 0x7fa8dc164a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8da59c700 1 -- 172.21.15.33:0/1295913066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d4009ae0 con 0x7fa8dc064dd0 2023-11-06T22:18:20.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.240+0000 7fa8d9d9b700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8dc0c0c40 0x7fa8dc0c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:20.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.241+0000 7fa8da59c700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 0x7fa8dc13f670 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7fa8cc00d890 tx=0x7fa8cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:20.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.241+0000 7fa8ca7fc700 1 -- 172.21.15.33:0/1295913066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc011590 con 0x7fa8dc064dd0 2023-11-06T22:18:20.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.241+0000 7fa8ca7fc700 1 -- 172.21.15.33:0/1295913066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc0116f0 con 0x7fa8dc064dd0 2023-11-06T22:18:20.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.241+0000 7fa8ca7fc700 1 -- 172.21.15.33:0/1295913066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc011a70 con 0x7fa8dc064dd0 2023-11-06T22:18:20.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.241+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1295913066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc0c4270 con 0x7fa8dc064dd0 2023-11-06T22:18:20.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.241+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1295913066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc0ad200 con 0x7fa8dc064dd0 2023-11-06T22:18:20.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.242+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1295913066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc0c48c0 con 0x7fa8dc064dd0 2023-11-06T22:18:20.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.243+0000 7fa8ca7fc700 1 -- 172.21.15.33:0/1295913066 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa8cc012420 con 0x7fa8dc064dd0 2023-11-06T22:18:20.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.244+0000 7fa8ca7fc700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa8bc05f2a0 0x7fa8bc061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.244+0000 7fa8ca7fc700 1 -- 172.21.15.33:0/1295913066 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5313+0+0 (secure 0 0 0) 0x7fa8cc079ab0 con 0x7fa8dc064dd0 2023-11-06T22:18:20.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.245+0000 7fa8d9d9b700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa8bc05f2a0 0x7fa8bc061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.245+0000 7fa8d9d9b700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa8bc05f2a0 0x7fa8bc061760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fa8dc156590 tx=0x7fa8d0009040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:20.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.248+0000 7fa8ca7fc700 1 -- 172.21.15.33:0/1295913066 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa8cc04ae00 con 0x7fa8dc064dd0 2023-11-06T22:18:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.430+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1295913066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8dc155080 con 0x7fa8dc064dd0 2023-11-06T22:18:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.430+0000 7fa8ca7fc700 1 -- 172.21.15.33:0/1295913066 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v219) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa8cc011e60 con 0x7fa8dc064dd0 2023-11-06T22:18:20.432 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T22:18:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa8bc05f2a0 msgr2=0x7fa8bc061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa8bc05f2a0 0x7fa8bc061760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fa8dc156590 tx=0x7fa8d0009040).stop 2023-11-06T22:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1295913066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 msgr2=0x7fa8dc13f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 0x7fa8dc13f670 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7fa8cc00d890 tx=0x7fa8cc008040).stop 2023-11-06T22:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1295913066 shutdown_connections 2023-11-06T22:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa8bc05f2a0 0x7fa8bc061760 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8dc1544f0 0x7fa8dc164a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8dc064dd0 0x7fa8dc13f670 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 --2- 172.21.15.33:0/1295913066 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa8dc0c0c40 0x7fa8dc0c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1295913066 >> 172.21.15.33:0/1295913066 conn(0x7fa8dc0cb900 msgr2=0x7fa8dc155b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.432+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1295913066 shutdown_connections 2023-11-06T22:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.433+0000 7fa8e09ff700 1 -- 172.21.15.33:0/1295913066 wait complete. 2023-11-06T22:18:20.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T22:18:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.646+0000 7f88d83ff700 1 Processor -- start 2023-11-06T22:18:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.646+0000 7f88d83ff700 1 -- start start 2023-11-06T22:18:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.647+0000 7f88d83ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00c1840 0x7f88d00a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.647+0000 7f88d83ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d01554a0 con 0x7f88d00aa2d0 2023-11-06T22:18:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.647+0000 7f88d83ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d0155930 con 0x7f88d00c6460 2023-11-06T22:18:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.647+0000 7f88d83ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0154960 con 0x7f88d00c1840 2023-11-06T22:18:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d599a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f88d00aa2d0 0x7f88d00c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55550/0 (socket says 172.21.15.33:55550) 2023-11-06T22:18:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d599a700 1 -- 172.21.15.33:0/302379883 learned_addr learned my addr 172.21.15.33:0/302379883 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749212120 0 0) 0x7f88d01554a0 con 0x7f88d00aa2d0 2023-11-06T22:18:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88bc0033f0 con 0x7f88d00aa2d0 2023-11-06T22:18:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d619b700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00c1840 0x7f88d00a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4223768096 0 0) 0x7f88bc0033f0 con 0x7f88d00aa2d0 2023-11-06T22:18:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00c1840 msgr2=0x7f88d00a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:20.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00c1840 0x7f88d00a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 >> v1:172.21.15.69:6789/0 conn(0x7f88d00c6460 legacy=0x7f88d00bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0154b70 con 0x7f88d00aa2d0 2023-11-06T22:18:20.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f88c0001470 con 0x7f88d00aa2d0 2023-11-06T22:18:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 >> v1:172.21.15.33:6789/0 conn(0x7f88d00aa2d0 legacy=0x7f88d00c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 0x7f88bc003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.648+0000 7f88d5199700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88bc004690 0x7f88bc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d5199700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00c1840 0x7f88bc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0154960 con 0x7f88bc003910 2023-11-06T22:18:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc00b910 con 0x7f88bc004690 2023-11-06T22:18:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc00bad0 con 0x7f88d00c1840 2023-11-06T22:18:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d619b700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 0x7f88bc003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d599a700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88bc004690 0x7f88bc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d699c700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00c1840 0x7f88bc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d619b700 1 -- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00c1840 msgr2=0x7f88bc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:20.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d619b700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00c1840 0x7f88bc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d619b700 1 -- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88bc004690 msgr2=0x7f88bc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:20.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d619b700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88bc004690 0x7f88bc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d619b700 1 -- 172.21.15.33:0/302379883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0033f0 con 0x7f88bc003910 2023-11-06T22:18:20.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.649+0000 7f88d619b700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 0x7f88bc003d30 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f88cc009a80 tx=0x7f88cc009060).ready entity=mon.0 client_cookie=b6b84f86a1fb98fe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:20.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.650+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc009430 con 0x7f88bc003910 2023-11-06T22:18:20.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.650+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc00e640 con 0x7f88bc003910 2023-11-06T22:18:20.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.650+0000 7f88d5199700 1 -- 172.21.15.33:0/302379883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00e9c0 con 0x7f88bc003910 2023-11-06T22:18:20.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.650+0000 7f88d83ff700 1 -- 172.21.15.33:0/302379883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 msgr2=0x7f88bc003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.650+0000 7f88d83ff700 1 --2- 172.21.15.33:0/302379883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 0x7f88bc003d30 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f88cc009a80 tx=0x7f88cc009060).stop 2023-11-06T22:18:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.650+0000 7f88d699c700 1 -- 172.21.15.33:0/302379883 reap_dead start 2023-11-06T22:18:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.650+0000 7f88d83ff700 1 -- 172.21.15.33:0/302379883 shutdown_connections 2023-11-06T22:18:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.650+0000 7f88d83ff700 1 -- 172.21.15.33:0/302379883 >> 172.21.15.33:0/302379883 conn(0x7f88d005ec80 msgr2=0x7f88d00cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.650+0000 7f88d83ff700 1 -- 172.21.15.33:0/302379883 shutdown_connections 2023-11-06T22:18:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.650+0000 7f88d83ff700 1 -- 172.21.15.33:0/302379883 wait complete. 2023-11-06T22:18:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.652+0000 7f88d83ff700 1 Processor -- start 2023-11-06T22:18:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.652+0000 7f88d83ff700 1 -- start start 2023-11-06T22:18:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.653+0000 7f88d83ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 0x7f88d014e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.653+0000 7f88d83ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00ac3b0 0x7f88d00ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.653+0000 7f88d83ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d00adda0 0x7f88d00ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.653+0000 7f88d83ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d014e560 con 0x7f88bc003910 2023-11-06T22:18:20.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.653+0000 7f88d83ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00c3370 con 0x7f88d00adda0 2023-11-06T22:18:20.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.653+0000 7f88d83ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00acd20 con 0x7f88d00ac3b0 2023-11-06T22:18:20.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.653+0000 7f88d619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 0x7f88d014e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.653+0000 7f88d619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 0x7f88d014e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49076/0 (socket says 172.21.15.33:49076) 2023-11-06T22:18:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.653+0000 7f88d619b700 1 -- 172.21.15.33:0/2405224247 learned_addr learned my addr 172.21.15.33:0/2405224247 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88d619b700 1 -- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00ac3b0 msgr2=0x7f88d00ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88d619b700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00ac3b0 0x7f88d00ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88d619b700 1 -- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d00adda0 msgr2=0x7f88d00ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88d619b700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d00adda0 0x7f88d00ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88d619b700 1 -- 172.21.15.33:0/2405224247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00c2a40 con 0x7f88bc003910 2023-11-06T22:18:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88d619b700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 0x7f88d014e070 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f88cc0030b0 tx=0x7f88cc01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88c67fc700 1 -- 172.21.15.33:0/2405224247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00ed30 con 0x7f88bc003910 2023-11-06T22:18:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88c67fc700 1 -- 172.21.15.33:0/2405224247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc00ee90 con 0x7f88bc003910 2023-11-06T22:18:20.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88c67fc700 1 -- 172.21.15.33:0/2405224247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc026440 con 0x7f88bc003910 2023-11-06T22:18:20.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88d83ff700 1 -- 172.21.15.33:0/2405224247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d00c2d20 con 0x7f88bc003910 2023-11-06T22:18:20.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.654+0000 7f88d83ff700 1 -- 172.21.15.33:0/2405224247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d00ae710 con 0x7f88bc003910 2023-11-06T22:18:20.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.655+0000 7f88d83ff700 1 -- 172.21.15.33:0/2405224247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d006c6c0 con 0x7f88bc003910 2023-11-06T22:18:20.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.656+0000 7f88c67fc700 1 -- 172.21.15.33:0/2405224247 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f88cc02f440 con 0x7f88bc003910 2023-11-06T22:18:20.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.657+0000 7f88c67fc700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f88b405f270 0x7f88b4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:20.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.657+0000 7f88c67fc700 1 -- 172.21.15.33:0/2405224247 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f88cc07be70 con 0x7f88bc003910 2023-11-06T22:18:20.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.657+0000 7f88d599a700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f88b405f270 0x7f88b4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:20.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.658+0000 7f88d599a700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f88b405f270 0x7f88b4061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f88d00aa880 tx=0x7f88d0155410).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:20.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.661+0000 7f88c67fc700 1 -- 172.21.15.33:0/2405224247 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f88cc04d1c0 con 0x7f88bc003910 2023-11-06T22:18:20.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.844+0000 7f88d83ff700 1 -- 172.21.15.33:0/2405224247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f88d0063f20 con 0x7f88bc003910 2023-11-06T22:18:20.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.844+0000 7f88c67fc700 1 -- 172.21.15.33:0/2405224247 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v219) v1 ==== 84+0+143 (secure 0 0 0) 0x7f88cc016020 con 0x7f88bc003910 2023-11-06T22:18:20.847 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:20.847 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T22:18:20.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:20.848 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-11-06T22:18:20.848 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "device_health_metrics" 2023-11-06T22:18:20.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-11-06T22:18:20.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:20.849 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-11-06T22:18:20.849 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-11-06T22:18:20.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:20.850 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T22:18:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.846+0000 7f88d83ff700 1 -- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f88b405f270 msgr2=0x7f88b4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.846+0000 7f88d83ff700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f88b405f270 0x7f88b4061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f88d00aa880 tx=0x7f88d0155410).stop 2023-11-06T22:18:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.846+0000 7f88d83ff700 1 -- 172.21.15.33:0/2405224247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 msgr2=0x7f88d014e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.846+0000 7f88d83ff700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 0x7f88d014e070 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f88cc0030b0 tx=0x7f88cc01a000).stop 2023-11-06T22:18:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.847+0000 7f88d83ff700 1 -- 172.21.15.33:0/2405224247 shutdown_connections 2023-11-06T22:18:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.847+0000 7f88d83ff700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f88b405f270 0x7f88b4061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.847+0000 7f88d83ff700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d00adda0 0x7f88d00ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.847+0000 7f88d83ff700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88bc003910 0x7f88d014e070 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.847+0000 7f88d83ff700 1 --2- 172.21.15.33:0/2405224247 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f88d00ac3b0 0x7f88d00ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.847+0000 7f88d83ff700 1 -- 172.21.15.33:0/2405224247 >> 172.21.15.33:0/2405224247 conn(0x7f88d005ec80 msgr2=0x7f88d00c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.847+0000 7f88d83ff700 1 -- 172.21.15.33:0/2405224247 shutdown_connections 2023-11-06T22:18:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:20.847+0000 7f88d83ff700 1 -- 172.21.15.33:0/2405224247 wait complete. 2023-11-06T22:18:20.866 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T22:18:20.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-06T22:18:21.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.066+0000 7f6b017ff700 1 Processor -- start 2023-11-06T22:18:21.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.066+0000 7f6b017ff700 1 -- start start 2023-11-06T22:18:21.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.067+0000 7f6b017ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6afc153ed0 0x7f6afc158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.067+0000 7f6b017ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc1542b0 0x7f6afc0c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.067+0000 7f6b017ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6afc0c9f80 con 0x7f6afc0c8ee0 2023-11-06T22:18:21.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.067+0000 7f6b017ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6afc0cab20 con 0x7f6afc153ed0 2023-11-06T22:18:21.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.067+0000 7f6b017ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc0c6000 con 0x7f6afc1542b0 2023-11-06T22:18:21.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.067+0000 7f6afa7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6afc0c8ee0 0x7f6afc0ca410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55556/0 (socket says 172.21.15.33:55556) 2023-11-06T22:18:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.067+0000 7f6afa7fc700 1 -- 172.21.15.33:0/3479717535 learned_addr learned my addr 172.21.15.33:0/3479717535 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6afaffd700 1 --2- 172.21.15.33:0/3479717535 >> v2:172.21.15.69:3300/0 conn(0x7f6afc153ed0 0x7f6afc158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6afb7fe700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc1542b0 0x7f6afc0c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739350126 0 0) 0x7f6afc0c9f80 con 0x7f6afc0c8ee0 2023-11-06T22:18:21.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae40033f0 con 0x7f6afc0c8ee0 2023-11-06T22:18:21.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3669079423 0 0) 0x7f6ae40033f0 con 0x7f6afc0c8ee0 2023-11-06T22:18:21.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc1542b0 msgr2=0x7f6afc0c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:21.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc1542b0 0x7f6afc0c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 >> v2:172.21.15.69:3300/0 conn(0x7f6afc153ed0 msgr2=0x7f6afc158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:21.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 --2- 172.21.15.33:0/3479717535 >> v2:172.21.15.69:3300/0 conn(0x7f6afc153ed0 0x7f6afc158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc0c9370 con 0x7f6afc0c8ee0 2023-11-06T22:18:21.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f6af0001470 con 0x7f6afc0c8ee0 2023-11-06T22:18:21.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 >> v1:172.21.15.33:6789/0 conn(0x7f6afc0c8ee0 legacy=0x7f6afc0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:21.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc1542b0 0x7f6ae40038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 0x7f6ae4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ae4004b80 0x7f6ae4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc0cab20 con 0x7f6ae4003e10 2023-11-06T22:18:21.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc0c6000 con 0x7f6ae4004b80 2023-11-06T22:18:21.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.068+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae400b8e0 con 0x7f6afc1542b0 2023-11-06T22:18:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.069+0000 7f6afa7fc700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 0x7f6ae4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.069+0000 7f6afaffd700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc1542b0 0x7f6ae40038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.069+0000 7f6afb7fe700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ae4004b80 0x7f6ae4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.069+0000 7f6afa7fc700 1 -- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc1542b0 msgr2=0x7f6ae40038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.069+0000 7f6afa7fc700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc1542b0 0x7f6ae40038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.069+0000 7f6afa7fc700 1 -- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ae4004b80 msgr2=0x7f6ae4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.069+0000 7f6afa7fc700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ae4004b80 0x7f6ae4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.069+0000 7f6afa7fc700 1 -- 172.21.15.33:0/3479717535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae40033f0 con 0x7f6ae4003e10 2023-11-06T22:18:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.069+0000 7f6afa7fc700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 0x7f6ae4004690 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f6af000c240 tx=0x7f6af0001e90).ready entity=mon.0 client_cookie=5dd5fd9ca0a40d3c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.069+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0007a60 con 0x7f6ae4003e10 2023-11-06T22:18:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.070+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af000cad0 con 0x7f6ae4003e10 2023-11-06T22:18:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.070+0000 7f6af9ffb700 1 -- 172.21.15.33:0/3479717535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af000ce70 con 0x7f6ae4003e10 2023-11-06T22:18:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.070+0000 7f6b017ff700 1 -- 172.21.15.33:0/3479717535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 msgr2=0x7f6ae4004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.070+0000 7f6b017ff700 1 --2- 172.21.15.33:0/3479717535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 0x7f6ae4004690 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f6af000c240 tx=0x7f6af0001e90).stop 2023-11-06T22:18:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.070+0000 7f6afb7fe700 1 -- 172.21.15.33:0/3479717535 reap_dead start 2023-11-06T22:18:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.070+0000 7f6b017ff700 1 -- 172.21.15.33:0/3479717535 shutdown_connections 2023-11-06T22:18:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.070+0000 7f6b017ff700 1 -- 172.21.15.33:0/3479717535 >> 172.21.15.33:0/3479717535 conn(0x7f6afc05e1b0 msgr2=0x7f6afc05e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.070+0000 7f6b017ff700 1 -- 172.21.15.33:0/3479717535 shutdown_connections 2023-11-06T22:18:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.070+0000 7f6b017ff700 1 -- 172.21.15.33:0/3479717535 wait complete. 2023-11-06T22:18:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.072+0000 7f6b017ff700 1 Processor -- start 2023-11-06T22:18:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.072+0000 7f6b017ff700 1 -- start start 2023-11-06T22:18:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6b017ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 0x7f6afc0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6b017ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6afc155c40 0x7f6afc0b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6b017ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc0c3230 0x7f6afc0c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6b017ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc1560c0 con 0x7f6ae4003e10 2023-11-06T22:18:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6b017ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc0b0c20 con 0x7f6afc155c40 2023-11-06T22:18:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6b017ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc0c2da0 con 0x7f6afc0c3230 2023-11-06T22:18:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 0x7f6afc0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 0x7f6afc0addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49102/0 (socket says 172.21.15.33:49102) 2023-11-06T22:18:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6afaffd700 1 -- 172.21.15.33:0/1878626133 learned_addr learned my addr 172.21.15.33:0/1878626133 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6afa7fc700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6afc155c40 0x7f6afc0b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6afb7fe700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc0c3230 0x7f6afc0c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6afaffd700 1 -- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc0c3230 msgr2=0x7f6afc0c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6afaffd700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc0c3230 0x7f6afc0c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6afaffd700 1 -- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6afc155c40 msgr2=0x7f6afc0b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6afaffd700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6afc155c40 0x7f6afc0b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.073+0000 7f6afaffd700 1 -- 172.21.15.33:0/1878626133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc0c2470 con 0x7f6ae4003e10 2023-11-06T22:18:21.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.074+0000 7f6afaffd700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 0x7f6afc0addd0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7f6aec0119f0 tx=0x7f6aec009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.074+0000 7f6ae2ffd700 1 -- 172.21.15.33:0/1878626133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00f8c0 con 0x7f6ae4003e10 2023-11-06T22:18:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.074+0000 7f6ae2ffd700 1 -- 172.21.15.33:0/1878626133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec00fa20 con 0x7f6ae4003e10 2023-11-06T22:18:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.074+0000 7f6b017ff700 1 -- 172.21.15.33:0/1878626133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6afc0c2750 con 0x7f6ae4003e10 2023-11-06T22:18:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.074+0000 7f6ae2ffd700 1 -- 172.21.15.33:0/1878626133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00fda0 con 0x7f6ae4003e10 2023-11-06T22:18:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.074+0000 7f6b017ff700 1 -- 172.21.15.33:0/1878626133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc0b0930 con 0x7f6ae4003e10 2023-11-06T22:18:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.075+0000 7f6b017ff700 1 -- 172.21.15.33:0/1878626133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6afc06c6c0 con 0x7f6ae4003e10 2023-11-06T22:18:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.076+0000 7f6ae2ffd700 1 -- 172.21.15.33:0/1878626133 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f6aec014650 con 0x7f6ae4003e10 2023-11-06T22:18:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.077+0000 7f6ae2ffd700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6ae805f270 0x7f6ae8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.077+0000 7f6ae2ffd700 1 -- 172.21.15.33:0/1878626133 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5313+0+0 (secure 0 0 0) 0x7f6aec07ad40 con 0x7f6ae4003e10 2023-11-06T22:18:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.077+0000 7f6afa7fc700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6ae805f270 0x7f6ae8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.078+0000 7f6afa7fc700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6ae805f270 0x7f6ae8061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6af0001ed0 tx=0x7f6af0015000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.081+0000 7f6ae2ffd700 1 -- 172.21.15.33:0/1878626133 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6aec04c0f0 con 0x7f6ae4003e10 2023-11-06T22:18:21.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.262+0000 7f6b017ff700 1 -- 172.21.15.33:0/1878626133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f6afc0c9650 con 0x7f6ae4003e10 2023-11-06T22:18:21.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.339+0000 7f6ae2ffd700 1 -- 172.21.15.33:0/1878626133 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v220) v1 ==== 152+0+0 (secure 0 0 0) 0x7f6aec04f710 con 0x7f6ae4003e10 2023-11-06T22:18:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.342+0000 7f6b017ff700 1 -- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6ae805f270 msgr2=0x7f6ae8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.342+0000 7f6b017ff700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6ae805f270 0x7f6ae8061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6af0001ed0 tx=0x7f6af0015000).stop 2023-11-06T22:18:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.343+0000 7f6b017ff700 1 -- 172.21.15.33:0/1878626133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 msgr2=0x7f6afc0addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.343+0000 7f6b017ff700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 0x7f6afc0addd0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7f6aec0119f0 tx=0x7f6aec009060).stop 2023-11-06T22:18:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.343+0000 7f6b017ff700 1 -- 172.21.15.33:0/1878626133 shutdown_connections 2023-11-06T22:18:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.343+0000 7f6b017ff700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6ae805f270 0x7f6ae8061730 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.343+0000 7f6b017ff700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6afc0c3230 0x7f6afc0c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.343+0000 7f6b017ff700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ae4003e10 0x7f6afc0addd0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.343+0000 7f6b017ff700 1 --2- 172.21.15.33:0/1878626133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6afc155c40 0x7f6afc0b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.343+0000 7f6b017ff700 1 -- 172.21.15.33:0/1878626133 >> 172.21.15.33:0/1878626133 conn(0x7f6afc05e1b0 msgr2=0x7f6afc0ab460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.343+0000 7f6b017ff700 1 -- 172.21.15.33:0/1878626133 shutdown_connections 2023-11-06T22:18:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.343+0000 7f6b017ff700 1 -- 172.21.15.33:0/1878626133 wait complete. 2023-11-06T22:18:21.349 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-11-06T22:18:21.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T22:18:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.571+0000 7f7e37fff700 1 Processor -- start 2023-11-06T22:18:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.571+0000 7f7e37fff700 1 -- start start 2023-11-06T22:18:21.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.572+0000 7f7e37fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e301546a0 0x7f7e30064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.572+0000 7f7e37fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7e30066560 0x7f7e300669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.572+0000 7f7e37fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7e300bdbb0 0x7f7e300be060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.572+0000 7f7e37fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e30154a80 con 0x7f7e300bdbb0 2023-11-06T22:18:21.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.572+0000 7f7e37fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e300be550 con 0x7f7e30066560 2023-11-06T22:18:21.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.572+0000 7f7e37fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300641f0 con 0x7f7e301546a0 2023-11-06T22:18:21.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.572+0000 7f7e3659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7e300bdbb0 0x7f7e300be060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e3659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7e300bdbb0 0x7f7e300be060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49114/0 (socket says 172.21.15.33:49114) 2023-11-06T22:18:21.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e3659c700 1 -- 172.21.15.33:0/2610652456 learned_addr learned my addr 172.21.15.33:0/2610652456 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:21.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e3559a700 1 --2- 172.21.15.33:0/2610652456 >> v2:172.21.15.69:3300/0 conn(0x7f7e30066560 0x7f7e300669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e35d9b700 1 --2- 172.21.15.33:0/2610652456 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e301546a0 0x7f7e30064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e3659c700 1 -- 172.21.15.33:0/2610652456 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e301546a0 msgr2=0x7f7e30064680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e3659c700 1 --2- 172.21.15.33:0/2610652456 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e301546a0 0x7f7e30064680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e3659c700 1 -- 172.21.15.33:0/2610652456 >> v2:172.21.15.69:3300/0 conn(0x7f7e30066560 msgr2=0x7f7e300669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e3659c700 1 --2- 172.21.15.33:0/2610652456 >> v2:172.21.15.69:3300/0 conn(0x7f7e30066560 0x7f7e300669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e3659c700 1 -- 172.21.15.33:0/2610652456 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e300638c0 con 0x7f7e300bdbb0 2023-11-06T22:18:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e35d9b700 1 --2- 172.21.15.33:0/2610652456 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e301546a0 0x7f7e30064680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e3659c700 1 --2- 172.21.15.33:0/2610652456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e300bdbb0 0x7f7e300be060 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 rx=0x7f7e300c6700 tx=0x7f7e28009840).ready entity=mon.0 client_cookie=2889c3cab703e4ea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:21.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e34d99700 1 -- 172.21.15.33:0/2610652456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800e070 con 0x7f7e300bdbb0 2023-11-06T22:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e34d99700 1 -- 172.21.15.33:0/2610652456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2800ba50 con 0x7f7e300bdbb0 2023-11-06T22:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.573+0000 7f7e34d99700 1 -- 172.21.15.33:0/2610652456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800bdd0 con 0x7f7e300bdbb0 2023-11-06T22:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.574+0000 7f7e37fff700 1 -- 172.21.15.33:0/2610652456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e300bdbb0 msgr2=0x7f7e300be060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.574+0000 7f7e37fff700 1 --2- 172.21.15.33:0/2610652456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e300bdbb0 0x7f7e300be060 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 rx=0x7f7e300c6700 tx=0x7f7e28009840).stop 2023-11-06T22:18:21.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.574+0000 7f7e37fff700 1 -- 172.21.15.33:0/2610652456 shutdown_connections 2023-11-06T22:18:21.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.574+0000 7f7e37fff700 1 --2- 172.21.15.33:0/2610652456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e300bdbb0 0x7f7e300be060 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.574+0000 7f7e37fff700 1 --2- 172.21.15.33:0/2610652456 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e301546a0 0x7f7e30064680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.574+0000 7f7e37fff700 1 --2- 172.21.15.33:0/2610652456 >> v2:172.21.15.69:3300/0 conn(0x7f7e30066560 0x7f7e300669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.574+0000 7f7e37fff700 1 -- 172.21.15.33:0/2610652456 >> 172.21.15.33:0/2610652456 conn(0x7f7e300c8c50 msgr2=0x7f7e300c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.574+0000 7f7e37fff700 1 -- 172.21.15.33:0/2610652456 shutdown_connections 2023-11-06T22:18:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.574+0000 7f7e37fff700 1 -- 172.21.15.33:0/2610652456 wait complete. 2023-11-06T22:18:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.576+0000 7f7e37fff700 1 Processor -- start 2023-11-06T22:18:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.576+0000 7f7e37fff700 1 -- start start 2023-11-06T22:18:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e37fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e30066560 0x7f7e30155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e37fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e300bdbb0 0x7f7e300c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e37fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e301546a0 0x7f7e30150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e37fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e301646c0 con 0x7f7e301546a0 2023-11-06T22:18:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e37fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30151af0 con 0x7f7e300bdbb0 2023-11-06T22:18:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e37fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30155020 con 0x7f7e30066560 2023-11-06T22:18:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e3659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e301546a0 0x7f7e30150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e3659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e301546a0 0x7f7e30150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49126/0 (socket says 172.21.15.33:49126) 2023-11-06T22:18:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e3659c700 1 -- 172.21.15.33:0/4055133838 learned_addr learned my addr 172.21.15.33:0/4055133838 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:21.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e35d9b700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e30066560 0x7f7e30155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e3559a700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e300bdbb0 0x7f7e300c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e3659c700 1 -- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e30066560 msgr2=0x7f7e30155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e3659c700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e30066560 0x7f7e30155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e3659c700 1 -- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e300bdbb0 msgr2=0x7f7e300c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e3659c700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e300bdbb0 0x7f7e300c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.577+0000 7f7e3659c700 1 -- 172.21.15.33:0/4055133838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e28009ae0 con 0x7f7e301546a0 2023-11-06T22:18:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.578+0000 7f7e3659c700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e301546a0 0x7f7e30150c40 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f7e2800bef0 tx=0x7f7e28009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:21.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.578+0000 7f7e25ffb700 1 -- 172.21.15.33:0/4055133838 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28013e60 con 0x7f7e301546a0 2023-11-06T22:18:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.578+0000 7f7e25ffb700 1 -- 172.21.15.33:0/4055133838 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e28009500 con 0x7f7e301546a0 2023-11-06T22:18:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.578+0000 7f7e37fff700 1 -- 172.21.15.33:0/4055133838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e301648d0 con 0x7f7e301546a0 2023-11-06T22:18:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.578+0000 7f7e25ffb700 1 -- 172.21.15.33:0/4055133838 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2801e6c0 con 0x7f7e301546a0 2023-11-06T22:18:21.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.578+0000 7f7e37fff700 1 -- 172.21.15.33:0/4055133838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e30164e90 con 0x7f7e301546a0 2023-11-06T22:18:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.579+0000 7f7e37fff700 1 -- 172.21.15.33:0/4055133838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e300bf160 con 0x7f7e301546a0 2023-11-06T22:18:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.580+0000 7f7e25ffb700 1 -- 172.21.15.33:0/4055133838 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7e28003040 con 0x7f7e301546a0 2023-11-06T22:18:21.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.581+0000 7f7e25ffb700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e1405f270 0x7f7e14061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.581+0000 7f7e25ffb700 1 -- 172.21.15.33:0/4055133838 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 5683+0+0 (secure 0 0 0) 0x7f7e2807b830 con 0x7f7e301546a0 2023-11-06T22:18:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.581+0000 7f7e35d9b700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e1405f270 0x7f7e14061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:21.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.582+0000 7f7e35d9b700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e1405f270 0x7f7e14061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7e2c00ab40 tx=0x7f7e2c008040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:21.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.586+0000 7f7e25ffb700 1 -- 172.21.15.33:0/4055133838 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7e2804ca70 con 0x7f7e301546a0 2023-11-06T22:18:21.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:21.768+0000 7f7e37fff700 1 -- 172.21.15.33:0/4055133838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7e30066000 con 0x7f7e301546a0 2023-11-06T22:18:22.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.347+0000 7f7e25ffb700 1 -- 172.21.15.33:0/4055133838 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v221) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7e28023070 con 0x7f7e301546a0 2023-11-06T22:18:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.354+0000 7f7e37fff700 1 -- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e1405f270 msgr2=0x7f7e14061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.354+0000 7f7e37fff700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e1405f270 0x7f7e14061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7e2c00ab40 tx=0x7f7e2c008040).stop 2023-11-06T22:18:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.354+0000 7f7e37fff700 1 -- 172.21.15.33:0/4055133838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e301546a0 msgr2=0x7f7e30150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.354+0000 7f7e37fff700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e301546a0 0x7f7e30150c40 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f7e2800bef0 tx=0x7f7e28009840).stop 2023-11-06T22:18:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.355+0000 7f7e37fff700 1 -- 172.21.15.33:0/4055133838 shutdown_connections 2023-11-06T22:18:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.355+0000 7f7e37fff700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e1405f270 0x7f7e14061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.355+0000 7f7e37fff700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e30066560 0x7f7e30155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.355+0000 7f7e37fff700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e301546a0 0x7f7e30150c40 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.355+0000 7f7e37fff700 1 --2- 172.21.15.33:0/4055133838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e300bdbb0 0x7f7e300c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.355+0000 7f7e37fff700 1 -- 172.21.15.33:0/4055133838 >> 172.21.15.33:0/4055133838 conn(0x7f7e300c8c50 msgr2=0x7f7e30067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.356+0000 7f7e37fff700 1 -- 172.21.15.33:0/4055133838 shutdown_connections 2023-11-06T22:18:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.356+0000 7f7e37fff700 1 -- 172.21.15.33:0/4055133838 wait complete. 2023-11-06T22:18:22.365 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-11-06T22:18:22.393 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T22:18:22.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.592+0000 7fdac97ff700 1 Processor -- start 2023-11-06T22:18:22.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.592+0000 7fdac97ff700 1 -- start start 2023-11-06T22:18:22.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac97ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdac4154710 0x7fdac40c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:22.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac97ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdac40c7ff0 0x7fdac40646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:22.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac97ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac40c6b60 con 0x7fdac4154330 2023-11-06T22:18:22.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac97ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac40c7700 con 0x7fdac40c7ff0 2023-11-06T22:18:22.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac97ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac40c5cd0 con 0x7fdac4154710 2023-11-06T22:18:22.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdac40c7ff0 0x7fdac40646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:22.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdac40c7ff0 0x7fdac40646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49134/0 (socket says 172.21.15.33:49134) 2023-11-06T22:18:22.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac37fe700 1 -- 172.21.15.33:0/2632892797 learned_addr learned my addr 172.21.15.33:0/2632892797 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:22.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac27fc700 1 --2- 172.21.15.33:0/2632892797 >> v2:172.21.15.69:3300/0 conn(0x7fdac4154710 0x7fdac40c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:22.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac37fe700 1 -- 172.21.15.33:0/2632892797 >> v2:172.21.15.69:3300/0 conn(0x7fdac4154710 msgr2=0x7fdac40c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:22.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.594+0000 7fdac37fe700 1 --2- 172.21.15.33:0/2632892797 >> v2:172.21.15.69:3300/0 conn(0x7fdac4154710 0x7fdac40c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.595+0000 7fdac37fe700 1 -- 172.21.15.33:0/2632892797 >> v1:172.21.15.69:6789/0 conn(0x7fdac4154330 legacy=0x7fdac40c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:22.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.595+0000 7fdac37fe700 1 -- 172.21.15.33:0/2632892797 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac40c5f50 con 0x7fdac40c7ff0 2023-11-06T22:18:22.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.595+0000 7fdac37fe700 1 --2- 172.21.15.33:0/2632892797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac40c7ff0 0x7fdac40646c0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7fdac40c8d10 tx=0x7fdabc009840).ready entity=mon.0 client_cookie=4f071597781a1bf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:22.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.595+0000 7fdac1ffb700 1 -- 172.21.15.33:0/2632892797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00e070 con 0x7fdac40c7ff0 2023-11-06T22:18:22.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.595+0000 7fdac1ffb700 1 -- 172.21.15.33:0/2632892797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc00ba20 con 0x7fdac40c7ff0 2023-11-06T22:18:22.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.595+0000 7fdac1ffb700 1 -- 172.21.15.33:0/2632892797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00bda0 con 0x7fdac40c7ff0 2023-11-06T22:18:22.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.595+0000 7fdac97ff700 1 -- 172.21.15.33:0/2632892797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac40c7ff0 msgr2=0x7fdac40646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.595+0000 7fdac97ff700 1 --2- 172.21.15.33:0/2632892797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac40c7ff0 0x7fdac40646c0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7fdac40c8d10 tx=0x7fdabc009840).stop 2023-11-06T22:18:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.596+0000 7fdac97ff700 1 -- 172.21.15.33:0/2632892797 shutdown_connections 2023-11-06T22:18:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.596+0000 7fdac97ff700 1 --2- 172.21.15.33:0/2632892797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac40c7ff0 0x7fdac40646c0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.596+0000 7fdac97ff700 1 --2- 172.21.15.33:0/2632892797 >> v2:172.21.15.69:3300/0 conn(0x7fdac4154710 0x7fdac40c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.596+0000 7fdac97ff700 1 -- 172.21.15.33:0/2632892797 >> 172.21.15.33:0/2632892797 conn(0x7fdac40cb910 msgr2=0x7fdac40c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.596+0000 7fdac97ff700 1 -- 172.21.15.33:0/2632892797 shutdown_connections 2023-11-06T22:18:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.596+0000 7fdac97ff700 1 -- 172.21.15.33:0/2632892797 wait complete. 2023-11-06T22:18:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.597+0000 7fdac97ff700 1 Processor -- start 2023-11-06T22:18:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.597+0000 7fdac97ff700 1 -- start start 2023-11-06T22:18:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.598+0000 7fdac97ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdac40c7ff0 0x7fdac40ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.598+0000 7fdac97ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdac4154330 0x7fdac40c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.598+0000 7fdac97ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac4154710 0x7fdac40c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.598+0000 7fdac97ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac40c9c60 con 0x7fdac4154710 2023-11-06T22:18:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.598+0000 7fdac97ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac413f770 con 0x7fdac4154330 2023-11-06T22:18:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.598+0000 7fdac97ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41532d0 con 0x7fdac40c7ff0 2023-11-06T22:18:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac4154710 0x7fdac40c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac4154710 0x7fdac40c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49150/0 (socket says 172.21.15.33:49150) 2023-11-06T22:18:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac37fe700 1 -- 172.21.15.33:0/1869403134 learned_addr learned my addr 172.21.15.33:0/1869403134 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac2ffd700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdac40c7ff0 0x7fdac40ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac27fc700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdac4154330 0x7fdac40c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:22.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac37fe700 1 -- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdac40c7ff0 msgr2=0x7fdac40ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:22.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac37fe700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdac40c7ff0 0x7fdac40ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac37fe700 1 -- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdac4154330 msgr2=0x7fdac40c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:22.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac37fe700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdac4154330 0x7fdac40c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac37fe700 1 -- 172.21.15.33:0/1869403134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc009ae0 con 0x7fdac4154710 2023-11-06T22:18:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdac37fe700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac4154710 0x7fdac40c9770 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7fdac406c170 tx=0x7fdabc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.599+0000 7fdaaaffd700 1 -- 172.21.15.33:0/1869403134 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0092e0 con 0x7fdac4154710 2023-11-06T22:18:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.600+0000 7fdaaaffd700 1 -- 172.21.15.33:0/1869403134 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc01e470 con 0x7fdac4154710 2023-11-06T22:18:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.600+0000 7fdaaaffd700 1 -- 172.21.15.33:0/1869403134 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc01e810 con 0x7fdac4154710 2023-11-06T22:18:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.600+0000 7fdac97ff700 1 -- 172.21.15.33:0/1869403134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac40b0b10 con 0x7fdac4154710 2023-11-06T22:18:22.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.600+0000 7fdac97ff700 1 -- 172.21.15.33:0/1869403134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac41681a0 con 0x7fdac4154710 2023-11-06T22:18:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.601+0000 7fdac97ff700 1 -- 172.21.15.33:0/1869403134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac40830b0 con 0x7fdac4154710 2023-11-06T22:18:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.602+0000 7fdaaaffd700 1 -- 172.21.15.33:0/1869403134 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdabc003040 con 0x7fdac4154710 2023-11-06T22:18:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.603+0000 7fdaaaffd700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdab005f270 0x7fdab0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.603+0000 7fdaaaffd700 1 -- 172.21.15.33:0/1869403134 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6065+0+0 (secure 0 0 0) 0x7fdabc07b880 con 0x7fdac4154710 2023-11-06T22:18:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.603+0000 7fdac2ffd700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdab005f270 0x7fdab0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.604+0000 7fdac2ffd700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdab005f270 0x7fdab0061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fdab4003900 tx=0x7fdab4007d40).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.608+0000 7fdaaaffd700 1 -- 172.21.15.33:0/1869403134 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdabc04c9f0 con 0x7fdac4154710 2023-11-06T22:18:22.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.833+0000 7fdac97ff700 1 -- 172.21.15.33:0/1869403134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdac41687e0 con 0x7fdac4154710 2023-11-06T22:18:22.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.838+0000 7fdaaaffd700 1 -- 172.21.15.33:0/1869403134 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v88) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdac41687e0 con 0x7fdac4154710 2023-11-06T22:18:22.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.842+0000 7fdac97ff700 1 -- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdab005f270 msgr2=0x7fdab0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.842+0000 7fdac97ff700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdab005f270 0x7fdab0061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fdab4003900 tx=0x7fdab4007d40).stop 2023-11-06T22:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.842+0000 7fdac97ff700 1 -- 172.21.15.33:0/1869403134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac4154710 msgr2=0x7fdac40c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.842+0000 7fdac97ff700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac4154710 0x7fdac40c9770 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7fdac406c170 tx=0x7fdabc009840).stop 2023-11-06T22:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.843+0000 7fdac97ff700 1 -- 172.21.15.33:0/1869403134 shutdown_connections 2023-11-06T22:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.843+0000 7fdac97ff700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdab005f270 0x7fdab0061730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.843+0000 7fdac97ff700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdac40c7ff0 0x7fdac40ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.843+0000 7fdac97ff700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac4154710 0x7fdac40c9770 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.843+0000 7fdac97ff700 1 --2- 172.21.15.33:0/1869403134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdac4154330 0x7fdac40c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.843+0000 7fdac97ff700 1 -- 172.21.15.33:0/1869403134 >> 172.21.15.33:0/1869403134 conn(0x7fdac40cb910 msgr2=0x7fdac40ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:22.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.843+0000 7fdac97ff700 1 -- 172.21.15.33:0/1869403134 shutdown_connections 2023-11-06T22:18:22.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:22.844+0000 7fdac97ff700 1 -- 172.21.15.33:0/1869403134 wait complete. 2023-11-06T22:18:22.849 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 25 and data pool 26 2023-11-06T22:18:22.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.074+0000 7f26fb3ff700 1 Processor -- start 2023-11-06T22:18:23.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.074+0000 7f26fb3ff700 1 -- start start 2023-11-06T22:18:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.075+0000 7f26fb3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26f40c1840 0x7f26f40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.075+0000 7f26fb3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26f40aa2d0 0x7f26f40c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.075+0000 7f26fb3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f26f40c6240 0x7f26f4157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.075+0000 7f26fb3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f4154960 con 0x7f26f40c1840 2023-11-06T22:18:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.075+0000 7f26fb3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f41554a0 con 0x7f26f40c6240 2023-11-06T22:18:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.075+0000 7f26fb3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4155930 con 0x7f26f40aa2d0 2023-11-06T22:18:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.075+0000 7f26f919b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26f40c1840 0x7f26f40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.075+0000 7f26f919b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26f40c1840 0x7f26f40a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49164/0 (socket says 172.21.15.33:49164) 2023-11-06T22:18:23.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.075+0000 7f26f919b700 1 -- 172.21.15.33:0/2420713598 learned_addr learned my addr 172.21.15.33:0/2420713598 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26f999c700 1 --2- 172.21.15.33:0/2420713598 >> v2:172.21.15.69:3300/0 conn(0x7f26f40c6240 0x7f26f4157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26f899a700 1 --2- 172.21.15.33:0/2420713598 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26f40aa2d0 0x7f26f40c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26f919b700 1 -- 172.21.15.33:0/2420713598 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26f40aa2d0 msgr2=0x7f26f40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26f919b700 1 --2- 172.21.15.33:0/2420713598 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26f40aa2d0 0x7f26f40c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26f919b700 1 -- 172.21.15.33:0/2420713598 >> v2:172.21.15.69:3300/0 conn(0x7f26f40c6240 msgr2=0x7f26f4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26f919b700 1 --2- 172.21.15.33:0/2420713598 >> v2:172.21.15.69:3300/0 conn(0x7f26f40c6240 0x7f26f4157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26f919b700 1 -- 172.21.15.33:0/2420713598 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40c66b0 con 0x7f26f40c1840 2023-11-06T22:18:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26f919b700 1 --2- 172.21.15.33:0/2420713598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 0x7f26f40a9de0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 rx=0x7f26f4084630 tx=0x7f26f00090f0).ready entity=mon.0 client_cookie=1e98f0369b21f5f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26ebfff700 1 -- 172.21.15.33:0/2420713598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000b600 con 0x7f26f40c1840 2023-11-06T22:18:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26ebfff700 1 -- 172.21.15.33:0/2420713598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f000b760 con 0x7f26f40c1840 2023-11-06T22:18:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.076+0000 7f26ebfff700 1 -- 172.21.15.33:0/2420713598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000bae0 con 0x7f26f40c1840 2023-11-06T22:18:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.077+0000 7f26fb3ff700 1 -- 172.21.15.33:0/2420713598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 msgr2=0x7f26f40a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.077+0000 7f26fb3ff700 1 --2- 172.21.15.33:0/2420713598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 0x7f26f40a9de0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 rx=0x7f26f4084630 tx=0x7f26f00090f0).stop 2023-11-06T22:18:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.077+0000 7f26fb3ff700 1 -- 172.21.15.33:0/2420713598 shutdown_connections 2023-11-06T22:18:23.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.077+0000 7f26fb3ff700 1 --2- 172.21.15.33:0/2420713598 >> v2:172.21.15.69:3300/0 conn(0x7f26f40c6240 0x7f26f4157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.077+0000 7f26fb3ff700 1 --2- 172.21.15.33:0/2420713598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 0x7f26f40a9de0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.077+0000 7f26fb3ff700 1 --2- 172.21.15.33:0/2420713598 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26f40aa2d0 0x7f26f40c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.077+0000 7f26fb3ff700 1 -- 172.21.15.33:0/2420713598 >> 172.21.15.33:0/2420713598 conn(0x7f26f405ec80 msgr2=0x7f26f40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:23.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.077+0000 7f26fb3ff700 1 -- 172.21.15.33:0/2420713598 shutdown_connections 2023-11-06T22:18:23.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.077+0000 7f26fb3ff700 1 -- 172.21.15.33:0/2420713598 wait complete. 2023-11-06T22:18:23.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.078+0000 7f26fb3ff700 1 Processor -- start 2023-11-06T22:18:23.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.079+0000 7f26fb3ff700 1 -- start start 2023-11-06T22:18:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.079+0000 7f26fb3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26f40aa2d0 0x7f26f4149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.079+0000 7f26fb3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 0x7f26f414f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26fb3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26f40c6240 0x7f26f4152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26fb3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40b0480 con 0x7f26f40c1840 2023-11-06T22:18:23.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26fb3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f414fd40 con 0x7f26f40c6240 2023-11-06T22:18:23.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26fb3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4156430 con 0x7f26f40aa2d0 2023-11-06T22:18:23.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 0x7f26f414f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 0x7f26f414f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49170/0 (socket says 172.21.15.33:49170) 2023-11-06T22:18:23.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f899a700 1 -- 172.21.15.33:0/3966465870 learned_addr learned my addr 172.21.15.33:0/3966465870 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:23.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f919b700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26f40aa2d0 0x7f26f4149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f999c700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26f40c6240 0x7f26f4152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f899a700 1 -- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26f40aa2d0 msgr2=0x7f26f4149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f899a700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26f40aa2d0 0x7f26f4149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f899a700 1 -- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26f40c6240 msgr2=0x7f26f4152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f899a700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26f40c6240 0x7f26f4152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f899a700 1 -- 172.21.15.33:0/3966465870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0009370 con 0x7f26f40c1840 2023-11-06T22:18:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.080+0000 7f26f899a700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 0x7f26f414f850 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7f26e4006980 tx=0x7f26e4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.081+0000 7f26e97fa700 1 -- 172.21.15.33:0/3966465870 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4011590 con 0x7f26f40c1840 2023-11-06T22:18:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.081+0000 7f26e97fa700 1 -- 172.21.15.33:0/3966465870 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e40116f0 con 0x7f26f40c1840 2023-11-06T22:18:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.081+0000 7f26e97fa700 1 -- 172.21.15.33:0/3966465870 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4011a70 con 0x7f26f40c1840 2023-11-06T22:18:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.081+0000 7f26fb3ff700 1 -- 172.21.15.33:0/3966465870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f40afec0 con 0x7f26f40c1840 2023-11-06T22:18:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.081+0000 7f26fb3ff700 1 -- 172.21.15.33:0/3966465870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f41560c0 con 0x7f26f40c1840 2023-11-06T22:18:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.082+0000 7f26fb3ff700 1 -- 172.21.15.33:0/3966465870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f4155f20 con 0x7f26f40c1840 2023-11-06T22:18:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.083+0000 7f26e97fa700 1 -- 172.21.15.33:0/3966465870 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f26e4012420 con 0x7f26f40c1840 2023-11-06T22:18:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.084+0000 7f26e97fa700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f26d805f240 0x7f26d8061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.084+0000 7f26e97fa700 1 -- 172.21.15.33:0/3966465870 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f26e4079e40 con 0x7f26f40c1840 2023-11-06T22:18:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.084+0000 7f26f919b700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f26d805f240 0x7f26d8061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.085+0000 7f26f919b700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f26d805f240 0x7f26d8061700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f26f0009130 tx=0x7f26f0017000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.088+0000 7f26e97fa700 1 -- 172.21.15.33:0/3966465870 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f26f4155f20 con 0x7f26f40c1840 2023-11-06T22:18:23.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.267+0000 7f26fb3ff700 1 -- 172.21.15.33:0/3966465870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26f4155f20 con 0x7f26f40c1840 2023-11-06T22:18:23.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.268+0000 7f26e97fa700 1 -- 172.21.15.33:0/3966465870 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f26f4155f20 con 0x7f26f40c1840 2023-11-06T22:18:23.270 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:23.270 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":222,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:22.833901+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:18:21.317255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T22:18:21.822232+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 -- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f26d805f240 msgr2=0x7f26d8061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f26d805f240 0x7f26d8061700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f26f0009130 tx=0x7f26f0017000).stop 2023-11-06T22:18:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 -- 172.21.15.33:0/3966465870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 msgr2=0x7f26f414f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 0x7f26f414f850 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7f26e4006980 tx=0x7f26e4008040).stop 2023-11-06T22:18:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 -- 172.21.15.33:0/3966465870 shutdown_connections 2023-11-06T22:18:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f26d805f240 0x7f26d8061700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f26f40aa2d0 0x7f26f4149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26f40c6240 0x7f26f4152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 --2- 172.21.15.33:0/3966465870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26f40c1840 0x7f26f414f850 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 -- 172.21.15.33:0/3966465870 >> 172.21.15.33:0/3966465870 conn(0x7f26f405ec80 msgr2=0x7f26f40ad1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.270+0000 7f26fb3ff700 1 -- 172.21.15.33:0/3966465870 shutdown_connections 2023-11-06T22:18:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.271+0000 7f26fb3ff700 1 -- 172.21.15.33:0/3966465870 wait complete. 2023-11-06T22:18:23.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:23.333 INFO:tasks.ceph.mds.d.smithi069.stderr:2023-11-06T22:18:23.331+0000 7f5b4e7f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T22:18:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.486+0000 7f9a281ff700 1 Processor -- start 2023-11-06T22:18:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.486+0000 7f9a281ff700 1 -- start start 2023-11-06T22:18:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.487+0000 7f9a281ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9a200c1840 0x7f9a200a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.487+0000 7f9a281ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a201554a0 con 0x7f9a200aa2d0 2023-11-06T22:18:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.488+0000 7f9a281ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a20155930 con 0x7f9a200c6460 2023-11-06T22:18:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.488+0000 7f9a281ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a20154960 con 0x7f9a200c1840 2023-11-06T22:18:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.488+0000 7f9a25f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9a200c1840 0x7f9a200a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.488+0000 7f9a25f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9a200c1840 0x7f9a200a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49174/0 (socket says 172.21.15.33:49174) 2023-11-06T22:18:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.488+0000 7f9a25f9b700 1 -- 172.21.15.33:0/153788721 learned_addr learned my addr 172.21.15.33:0/153788721 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.488+0000 7f9a2579a700 1 --1- 172.21.15.33:0/153788721 >> v1:172.21.15.33:6789/0 conn(0x7f9a200aa2d0 0x7f9a200c5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55572/0 (socket says 172.21.15.33:55572) 2023-11-06T22:18:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.488+0000 7f9a25f9b700 1 -- 172.21.15.33:0/153788721 >> v1:172.21.15.69:6789/0 conn(0x7f9a200c6460 legacy=0x7f9a200bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.488+0000 7f9a25f9b700 1 -- 172.21.15.33:0/153788721 >> v1:172.21.15.33:6789/0 conn(0x7f9a200aa2d0 legacy=0x7f9a200c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.488+0000 7f9a25f9b700 1 -- 172.21.15.33:0/153788721 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a20154b70 con 0x7f9a200c1840 2023-11-06T22:18:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.489+0000 7f9a25f9b700 1 --2- 172.21.15.33:0/153788721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c1840 0x7f9a200a9de0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7f9a1c006fd0 tx=0x7f9a1c0090f0).ready entity=mon.0 client_cookie=fa8f4b7fc150fb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.489+0000 7f9a24f99700 1 -- 172.21.15.33:0/153788721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00b600 con 0x7f9a200c1840 2023-11-06T22:18:23.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.489+0000 7f9a24f99700 1 -- 172.21.15.33:0/153788721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c00b760 con 0x7f9a200c1840 2023-11-06T22:18:23.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.489+0000 7f9a24f99700 1 -- 172.21.15.33:0/153788721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00bae0 con 0x7f9a200c1840 2023-11-06T22:18:23.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.489+0000 7f9a281ff700 1 -- 172.21.15.33:0/153788721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c1840 msgr2=0x7f9a200a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.489+0000 7f9a281ff700 1 --2- 172.21.15.33:0/153788721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c1840 0x7f9a200a9de0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7f9a1c006fd0 tx=0x7f9a1c0090f0).stop 2023-11-06T22:18:23.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.489+0000 7f9a281ff700 1 -- 172.21.15.33:0/153788721 shutdown_connections 2023-11-06T22:18:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.489+0000 7f9a281ff700 1 --2- 172.21.15.33:0/153788721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c1840 0x7f9a200a9de0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.489+0000 7f9a281ff700 1 -- 172.21.15.33:0/153788721 >> 172.21.15.33:0/153788721 conn(0x7f9a2005ec80 msgr2=0x7f9a200cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.489+0000 7f9a281ff700 1 -- 172.21.15.33:0/153788721 shutdown_connections 2023-11-06T22:18:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.490+0000 7f9a281ff700 1 -- 172.21.15.33:0/153788721 wait complete. 2023-11-06T22:18:23.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.491+0000 7f9a281ff700 1 Processor -- start 2023-11-06T22:18:23.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.491+0000 7f9a281ff700 1 -- start start 2023-11-06T22:18:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a281ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a200aa2d0 0x7f9a201587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a281ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a200c1840 0x7f9a2014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a281ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c6460 0x7f9a200ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a281ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200c3ad0 con 0x7f9a200c6460 2023-11-06T22:18:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a281ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20168390 con 0x7f9a200aa2d0 2023-11-06T22:18:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a281ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20168670 con 0x7f9a200c1840 2023-11-06T22:18:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a2679c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c6460 0x7f9a200ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a2679c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c6460 0x7f9a200ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49178/0 (socket says 172.21.15.33:49178) 2023-11-06T22:18:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a2679c700 1 -- 172.21.15.33:0/2713286433 learned_addr learned my addr 172.21.15.33:0/2713286433 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a25f9b700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a200aa2d0 0x7f9a201587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.492+0000 7f9a2579a700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a200c1840 0x7f9a2014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a2679c700 1 -- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a200c1840 msgr2=0x7f9a2014ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a2679c700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a200c1840 0x7f9a2014ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a2679c700 1 -- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a200aa2d0 msgr2=0x7f9a201587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a2679c700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a200aa2d0 0x7f9a201587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a2679c700 1 -- 172.21.15.33:0/2713286433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c009370 con 0x7f9a200c6460 2023-11-06T22:18:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a2679c700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c6460 0x7f9a200ca0f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7f9a1800b810 tx=0x7f9a18009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a15ffb700 1 -- 172.21.15.33:0/2713286433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800d4a0 con 0x7f9a200c6460 2023-11-06T22:18:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a15ffb700 1 -- 172.21.15.33:0/2713286433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1800d600 con 0x7f9a200c6460 2023-11-06T22:18:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a15ffb700 1 -- 172.21.15.33:0/2713286433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800d9c0 con 0x7f9a200c6460 2023-11-06T22:18:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a281ff700 1 -- 172.21.15.33:0/2713286433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a20168950 con 0x7f9a200c6460 2023-11-06T22:18:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.493+0000 7f9a281ff700 1 -- 172.21.15.33:0/2713286433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a20167fd0 con 0x7f9a200c6460 2023-11-06T22:18:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.495+0000 7f9a281ff700 1 -- 172.21.15.33:0/2713286433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a200addc0 con 0x7f9a200c6460 2023-11-06T22:18:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.496+0000 7f9a15ffb700 1 -- 172.21.15.33:0/2713286433 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9a18011400 con 0x7f9a200c6460 2023-11-06T22:18:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.497+0000 7f9a15ffb700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9a0405f270 0x7f9a04061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.497+0000 7f9a15ffb700 1 -- 172.21.15.33:0/2713286433 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f9a18079c30 con 0x7f9a200c6460 2023-11-06T22:18:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.497+0000 7f9a25f9b700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9a0405f270 0x7f9a04061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.498+0000 7f9a25f9b700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9a0405f270 0x7f9a04061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f9a1c009130 tx=0x7f9a1c017000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:23.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.501+0000 7f9a15ffb700 1 -- 172.21.15.33:0/2713286433 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9a18047af0 con 0x7f9a200c6460 2023-11-06T22:18:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.684+0000 7f9a281ff700 1 -- 172.21.15.33:0/2713286433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a2006c6c0 con 0x7f9a200c6460 2023-11-06T22:18:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.685+0000 7f9a15ffb700 1 -- 172.21.15.33:0/2713286433 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f9a1804ace0 con 0x7f9a200c6460 2023-11-06T22:18:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:23.687 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":222,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:22.833901+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:18:21.317255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T22:18:21.822232+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 -- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9a0405f270 msgr2=0x7f9a04061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9a0405f270 0x7f9a04061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f9a1c009130 tx=0x7f9a1c017000).stop 2023-11-06T22:18:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 -- 172.21.15.33:0/2713286433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c6460 msgr2=0x7f9a200ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c6460 0x7f9a200ca0f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7f9a1800b810 tx=0x7f9a18009290).stop 2023-11-06T22:18:23.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 -- 172.21.15.33:0/2713286433 shutdown_connections 2023-11-06T22:18:23.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9a0405f270 0x7f9a04061730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a200c6460 0x7f9a200ca0f0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a200aa2d0 0x7f9a201587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 --2- 172.21.15.33:0/2713286433 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9a200c1840 0x7f9a2014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 -- 172.21.15.33:0/2713286433 >> 172.21.15.33:0/2713286433 conn(0x7f9a2005ec80 msgr2=0x7f9a201518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 -- 172.21.15.33:0/2713286433 shutdown_connections 2023-11-06T22:18:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.687+0000 7f9a281ff700 1 -- 172.21.15.33:0/2713286433 wait complete. 2023-11-06T22:18:23.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T22:18:23.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.905+0000 7f419ebff700 1 Processor -- start 2023-11-06T22:18:23.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.905+0000 7f419ebff700 1 -- start start 2023-11-06T22:18:23.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.906+0000 7f419ebff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419813dff0 0x7f4198157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.906+0000 7f419ebff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f41980c4b00 0x7f41980c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.906+0000 7f419ebff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4198064bf0 con 0x7f41980c4720 2023-11-06T22:18:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.906+0000 7f419ebff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419805e490 con 0x7f41980c4b00 2023-11-06T22:18:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.906+0000 7f419ebff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4198062e10 con 0x7f419813dff0 2023-11-06T22:18:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.907+0000 7f419d19c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f41980c4b00 0x7f41980c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.907+0000 7f419d19c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f41980c4b00 0x7f41980c2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:52480/0 (socket says 172.21.15.33:52480) 2023-11-06T22:18:23.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.907+0000 7f4197fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419813dff0 0x7f4198157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.907+0000 7f419d19c700 1 -- 172.21.15.33:0/2570371331 learned_addr learned my addr 172.21.15.33:0/2570371331 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:23.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.907+0000 7f419c99b700 1 --1- 172.21.15.33:0/2570371331 >> v1:172.21.15.69:6789/0 conn(0x7f41980c4720 0x7f419813d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:49980/0 (socket says 172.21.15.33:49980) 2023-11-06T22:18:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.907+0000 7f419d19c700 1 -- 172.21.15.33:0/2570371331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419813dff0 msgr2=0x7f4198157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.907+0000 7f419d19c700 1 --2- 172.21.15.33:0/2570371331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419813dff0 0x7f4198157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.907+0000 7f419d19c700 1 -- 172.21.15.33:0/2570371331 >> v1:172.21.15.69:6789/0 conn(0x7f41980c4720 legacy=0x7f419813d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.907+0000 7f419d19c700 1 -- 172.21.15.33:0/2570371331 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41980630f0 con 0x7f41980c4b00 2023-11-06T22:18:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.907+0000 7f41977fe700 1 -- 172.21.15.33:0/2570371331 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247621572 0 0) 0x7f4198064bf0 con 0x7f41980c4720 2023-11-06T22:18:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.908+0000 7f419d19c700 1 --2- 172.21.15.33:0/2570371331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41980c4b00 0x7f41980c2860 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f4190009f20 tx=0x7f4190009840).ready entity=mon.1 client_cookie=ce6dbb4c5293add9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.908+0000 7f41977fe700 1 -- 172.21.15.33:0/2570371331 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419000e070 con 0x7f41980c4b00 2023-11-06T22:18:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.908+0000 7f41977fe700 1 -- 172.21.15.33:0/2570371331 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419000b9d0 con 0x7f41980c4b00 2023-11-06T22:18:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.908+0000 7f41977fe700 1 -- 172.21.15.33:0/2570371331 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419000bd50 con 0x7f41980c4b00 2023-11-06T22:18:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.908+0000 7f419ebff700 1 -- 172.21.15.33:0/2570371331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41980c4b00 msgr2=0x7f41980c2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.908+0000 7f419ebff700 1 --2- 172.21.15.33:0/2570371331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41980c4b00 0x7f41980c2860 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f4190009f20 tx=0x7f4190009840).stop 2023-11-06T22:18:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.909+0000 7f419ebff700 1 -- 172.21.15.33:0/2570371331 shutdown_connections 2023-11-06T22:18:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.909+0000 7f419ebff700 1 --2- 172.21.15.33:0/2570371331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41980c4b00 0x7f41980c2860 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.909+0000 7f419ebff700 1 --2- 172.21.15.33:0/2570371331 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419813dff0 0x7f4198157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.909+0000 7f419ebff700 1 -- 172.21.15.33:0/2570371331 >> 172.21.15.33:0/2570371331 conn(0x7f41980c7dc0 msgr2=0x7f41980c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.909+0000 7f419ebff700 1 -- 172.21.15.33:0/2570371331 shutdown_connections 2023-11-06T22:18:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.909+0000 7f419ebff700 1 -- 172.21.15.33:0/2570371331 wait complete. 2023-11-06T22:18:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.910+0000 7f419ebff700 1 Processor -- start 2023-11-06T22:18:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.910+0000 7f419ebff700 1 -- start start 2023-11-06T22:18:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.911+0000 7f419ebff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41980c4720 0x7f4198164340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.911+0000 7f419ebff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41980c4b00 0x7f4198068b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.911+0000 7f419ebff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419813dff0 0x7f4198069030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.911+0000 7f419ebff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4198069520 con 0x7f41980c4b00 2023-11-06T22:18:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.911+0000 7f419ebff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4198135350 con 0x7f41980c4720 2023-11-06T22:18:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.911+0000 7f419ebff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4198134740 con 0x7f419813dff0 2023-11-06T22:18:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.911+0000 7f4197fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41980c4b00 0x7f4198068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.911+0000 7f4197fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41980c4b00 0x7f4198068b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49180/0 (socket says 172.21.15.33:49180) 2023-11-06T22:18:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.911+0000 7f4197fff700 1 -- 172.21.15.33:0/831186916 learned_addr learned my addr 172.21.15.33:0/831186916 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.911+0000 7f419c99b700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41980c4720 0x7f4198164340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f419d19c700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419813dff0 0x7f4198069030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f4197fff700 1 -- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419813dff0 msgr2=0x7f4198069030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f4197fff700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419813dff0 0x7f4198069030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f4197fff700 1 -- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41980c4720 msgr2=0x7f4198164340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f4197fff700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41980c4720 0x7f4198164340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f4197fff700 1 -- 172.21.15.33:0/831186916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190009ae0 con 0x7f41980c4b00 2023-11-06T22:18:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f4197fff700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41980c4b00 0x7f4198068b40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7f418400e930 tx=0x7f4184009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f4194ff9700 1 -- 172.21.15.33:0/831186916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4184009510 con 0x7f41980c4b00 2023-11-06T22:18:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f4194ff9700 1 -- 172.21.15.33:0/831186916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4184012600 con 0x7f41980c4b00 2023-11-06T22:18:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f419ebff700 1 -- 172.21.15.33:0/831186916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4198134a20 con 0x7f41980c4b00 2023-11-06T22:18:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f4194ff9700 1 -- 172.21.15.33:0/831186916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4184012980 con 0x7f41980c4b00 2023-11-06T22:18:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.912+0000 7f419ebff700 1 -- 172.21.15.33:0/831186916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4198134fe0 con 0x7f41980c4b00 2023-11-06T22:18:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.914+0000 7f419ebff700 1 -- 172.21.15.33:0/831186916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4198135500 con 0x7f41980c4b00 2023-11-06T22:18:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.914+0000 7f4194ff9700 1 -- 172.21.15.33:0/831186916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4184012d30 con 0x7f41980c4b00 2023-11-06T22:18:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.916+0000 7f4194ff9700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f417c05f270 0x7f417c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.916+0000 7f4194ff9700 1 -- 172.21.15.33:0/831186916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f418407a7f0 con 0x7f41980c4b00 2023-11-06T22:18:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.916+0000 7f419c99b700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f417c05f270 0x7f417c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.917+0000 7f419c99b700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f417c05f270 0x7f417c061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f418c007fd0 tx=0x7f418c007d70).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:23.920+0000 7f4194ff9700 1 -- 172.21.15.33:0/831186916 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4198135500 con 0x7f41980c4b00 2023-11-06T22:18:24.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.148+0000 7f419ebff700 1 -- 172.21.15.33:0/831186916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f41981354c0 con 0x7f41980c4b00 2023-11-06T22:18:24.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.842+0000 7f4194ff9700 1 -- 172.21.15.33:0/831186916 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) v1 ==== 118+0+0 (secure 0 0 0) 0x7f418404f280 con 0x7f41980c4b00 2023-11-06T22:18:24.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 -- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f417c05f270 msgr2=0x7f417c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:24.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f417c05f270 0x7f417c061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f418c007fd0 tx=0x7f418c007d70).stop 2023-11-06T22:18:24.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 -- 172.21.15.33:0/831186916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41980c4b00 msgr2=0x7f4198068b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:24.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41980c4b00 0x7f4198068b40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7f418400e930 tx=0x7f4184009060).stop 2023-11-06T22:18:24.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 -- 172.21.15.33:0/831186916 shutdown_connections 2023-11-06T22:18:24.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f417c05f270 0x7f417c061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:24.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419813dff0 0x7f4198069030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:24.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41980c4720 0x7f4198164340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:24.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 --2- 172.21.15.33:0/831186916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41980c4b00 0x7f4198068b40 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:24.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 -- 172.21.15.33:0/831186916 >> 172.21.15.33:0/831186916 conn(0x7f41980c7dc0 msgr2=0x7f4198153460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:24.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 -- 172.21.15.33:0/831186916 shutdown_connections 2023-11-06T22:18:24.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:24.845+0000 7f419ebff700 1 -- 172.21.15.33:0/831186916 wait complete. 2023-11-06T22:18:24.867 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T22:18:24.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T22:18:25.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.070+0000 7f7e893ff700 1 Processor -- start 2023-11-06T22:18:25.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.070+0000 7f7e893ff700 1 -- start start 2023-11-06T22:18:25.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.071+0000 7f7e893ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7e841546a0 0x7f7e84064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:25.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.071+0000 7f7e893ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e84066560 0x7f7e840669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:25.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.071+0000 7f7e893ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e840641f0 con 0x7f7e840bdbb0 2023-11-06T22:18:25.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.071+0000 7f7e893ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e84154a80 con 0x7f7e841546a0 2023-11-06T22:18:25.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.071+0000 7f7e893ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e840638c0 con 0x7f7e84066560 2023-11-06T22:18:25.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.072+0000 7f7e827fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e84066560 0x7f7e840669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:25.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.072+0000 7f7e82ffd700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7e841546a0 0x7f7e84064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:25.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.072+0000 7f7e827fc700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e84066560 0x7f7e840669e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:50210/0 (socket says 172.21.15.33:50210) 2023-11-06T22:18:25.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.072+0000 7f7e827fc700 1 -- 172.21.15.33:0/3274764337 learned_addr learned my addr 172.21.15.33:0/3274764337 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:25.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.073+0000 7f7e82ffd700 1 -- 172.21.15.33:0/3274764337 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e84066560 msgr2=0x7f7e840669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:25.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.073+0000 7f7e82ffd700 1 --2- 172.21.15.33:0/3274764337 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e84066560 0x7f7e840669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:25.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.073+0000 7f7e82ffd700 1 -- 172.21.15.33:0/3274764337 >> v1:172.21.15.69:6789/0 conn(0x7f7e840bdbb0 legacy=0x7f7e840c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:25.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.073+0000 7f7e82ffd700 1 -- 172.21.15.33:0/3274764337 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e84063ba0 con 0x7f7e841546a0 2023-11-06T22:18:25.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.073+0000 7f7e81ffb700 1 -- 172.21.15.33:0/3274764337 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457690088 0 0) 0x7f7e840641f0 con 0x7f7e840bdbb0 2023-11-06T22:18:25.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.073+0000 7f7e827fc700 1 --2- 172.21.15.33:0/3274764337 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e84066560 0x7f7e840669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:25.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.073+0000 7f7e82ffd700 1 --2- 172.21.15.33:0/3274764337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e841546a0 0x7f7e84064680 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f7e74009760 tx=0x7f7e74009060).ready entity=mon.1 client_cookie=e33854d781b811f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:25.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.073+0000 7f7e81ffb700 1 -- 172.21.15.33:0/3274764337 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400f510 con 0x7f7e841546a0 2023-11-06T22:18:25.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.073+0000 7f7e81ffb700 1 -- 172.21.15.33:0/3274764337 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7400f670 con 0x7f7e841546a0 2023-11-06T22:18:25.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.073+0000 7f7e81ffb700 1 -- 172.21.15.33:0/3274764337 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400f9f0 con 0x7f7e841546a0 2023-11-06T22:18:25.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.074+0000 7f7e893ff700 1 -- 172.21.15.33:0/3274764337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e841546a0 msgr2=0x7f7e84064680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:25.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.074+0000 7f7e893ff700 1 --2- 172.21.15.33:0/3274764337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e841546a0 0x7f7e84064680 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f7e74009760 tx=0x7f7e74009060).stop 2023-11-06T22:18:25.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.074+0000 7f7e893ff700 1 -- 172.21.15.33:0/3274764337 shutdown_connections 2023-11-06T22:18:25.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.074+0000 7f7e893ff700 1 --2- 172.21.15.33:0/3274764337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e841546a0 0x7f7e84064680 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:25.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.074+0000 7f7e893ff700 1 --2- 172.21.15.33:0/3274764337 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e84066560 0x7f7e840669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:25.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.074+0000 7f7e893ff700 1 -- 172.21.15.33:0/3274764337 >> 172.21.15.33:0/3274764337 conn(0x7f7e840c8c50 msgr2=0x7f7e840c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:25.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.074+0000 7f7e893ff700 1 -- 172.21.15.33:0/3274764337 shutdown_connections 2023-11-06T22:18:25.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.074+0000 7f7e893ff700 1 -- 172.21.15.33:0/3274764337 wait complete. 2023-11-06T22:18:25.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.075+0000 7f7e893ff700 1 Processor -- start 2023-11-06T22:18:25.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.076+0000 7f7e893ff700 1 -- start start 2023-11-06T22:18:25.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e893ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84066560 0x7f7e8413efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:25.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e893ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e840bdbb0 0x7f7e84164e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:25.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e893ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e841546a0 0x7f7e841523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:25.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e893ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8413f4b0 con 0x7f7e84066560 2023-11-06T22:18:25.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e893ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e841682f0 con 0x7f7e840bdbb0 2023-11-06T22:18:25.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e893ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e84154f40 con 0x7f7e841546a0 2023-11-06T22:18:25.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e82ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84066560 0x7f7e8413efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:25.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e82ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84066560 0x7f7e8413efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49186/0 (socket says 172.21.15.33:49186) 2023-11-06T22:18:25.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e82ffd700 1 -- 172.21.15.33:0/1417405416 learned_addr learned my addr 172.21.15.33:0/1417405416 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:25.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e827fc700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e840bdbb0 0x7f7e84164e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:25.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e837fe700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e841546a0 0x7f7e841523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:25.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e82ffd700 1 -- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e841546a0 msgr2=0x7f7e841523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:25.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e82ffd700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e841546a0 0x7f7e841523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:25.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e82ffd700 1 -- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e840bdbb0 msgr2=0x7f7e84164e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:25.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e82ffd700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e840bdbb0 0x7f7e84164e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:25.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.077+0000 7f7e82ffd700 1 -- 172.21.15.33:0/1417405416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e78009370 con 0x7f7e84066560 2023-11-06T22:18:25.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.078+0000 7f7e82ffd700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84066560 0x7f7e8413efc0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7f7e74009e00 tx=0x7f7e74017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:25.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.078+0000 7f7e6affd700 1 -- 172.21.15.33:0/1417405416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400fe80 con 0x7f7e84066560 2023-11-06T22:18:25.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.078+0000 7f7e6affd700 1 -- 172.21.15.33:0/1417405416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7400f7e0 con 0x7f7e84066560 2023-11-06T22:18:25.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.078+0000 7f7e6affd700 1 -- 172.21.15.33:0/1417405416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74022610 con 0x7f7e84066560 2023-11-06T22:18:25.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.078+0000 7f7e893ff700 1 -- 172.21.15.33:0/1417405416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e740092a0 con 0x7f7e84066560 2023-11-06T22:18:25.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.078+0000 7f7e893ff700 1 -- 172.21.15.33:0/1417405416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e84167f80 con 0x7f7e84066560 2023-11-06T22:18:25.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.079+0000 7f7e893ff700 1 -- 172.21.15.33:0/1417405416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e84167b00 con 0x7f7e84066560 2023-11-06T22:18:25.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.080+0000 7f7e6affd700 1 -- 172.21.15.33:0/1417405416 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7e7401a400 con 0x7f7e84066560 2023-11-06T22:18:25.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.081+0000 7f7e6affd700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e7005f270 0x7f7e70061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:25.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.081+0000 7f7e6affd700 1 -- 172.21.15.33:0/1417405416 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f7e7407b8b0 con 0x7f7e84066560 2023-11-06T22:18:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.081+0000 7f7e827fc700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e7005f270 0x7f7e70061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.082+0000 7f7e827fc700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e7005f270 0x7f7e70061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7e84152240 tx=0x7f7e780090f0).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.085+0000 7f7e6affd700 1 -- 172.21.15.33:0/1417405416 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7e84167b00 con 0x7f7e84066560 2023-11-06T22:18:25.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.316+0000 7f7e893ff700 1 -- 172.21.15.33:0/1417405416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7e84167b00 con 0x7f7e84066560 2023-11-06T22:18:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.845+0000 7f7e6affd700 1 -- 172.21.15.33:0/1417405416 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7e84167b00 con 0x7f7e84066560 2023-11-06T22:18:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 -- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e7005f270 msgr2=0x7f7e70061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e7005f270 0x7f7e70061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7e84152240 tx=0x7f7e780090f0).stop 2023-11-06T22:18:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 -- 172.21.15.33:0/1417405416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84066560 msgr2=0x7f7e8413efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84066560 0x7f7e8413efc0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7f7e74009e00 tx=0x7f7e74017000).stop 2023-11-06T22:18:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 -- 172.21.15.33:0/1417405416 shutdown_connections 2023-11-06T22:18:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7e7005f270 0x7f7e70061730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7e841546a0 0x7f7e841523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84066560 0x7f7e8413efc0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 --2- 172.21.15.33:0/1417405416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e840bdbb0 0x7f7e84164e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 -- 172.21.15.33:0/1417405416 >> 172.21.15.33:0/1417405416 conn(0x7f7e840c8c50 msgr2=0x7f7e840c3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:25.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 -- 172.21.15.33:0/1417405416 shutdown_connections 2023-11-06T22:18:25.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:25.848+0000 7f7e893ff700 1 -- 172.21.15.33:0/1417405416 wait complete. 2023-11-06T22:18:25.869 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T22:18:25.869 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:26.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.067+0000 7fe3ea3ff700 1 Processor -- start 2023-11-06T22:18:26.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.067+0000 7fe3ea3ff700 1 -- start start 2023-11-06T22:18:26.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.068+0000 7fe3ea3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4062e20 0x7fe3e4154240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.068+0000 7fe3ea3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e40c2ec0 con 0x7fe3e4153e60 2023-11-06T22:18:26.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.068+0000 7fe3ea3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e40c4360 con 0x7fe3e40a6d40 2023-11-06T22:18:26.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.068+0000 7fe3ea3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4154730 con 0x7fe3e4062e20 2023-11-06T22:18:26.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.069+0000 7fe3e3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe3e4153e60 0x7fe3e4069d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55574/0 (socket says 172.21.15.33:55574) 2023-11-06T22:18:26.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.069+0000 7fe3e3fff700 1 -- 172.21.15.33:0/827846965 learned_addr learned my addr 172.21.15.33:0/827846965 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.069+0000 7fe3e899c700 1 --2- 172.21.15.33:0/827846965 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4062e20 0x7fe3e4154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.069+0000 7fe3e2ffd700 1 -- 172.21.15.33:0/827846965 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239289868 0 0) 0x7fe3e40c2ec0 con 0x7fe3e4153e60 2023-11-06T22:18:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.069+0000 7fe3e2ffd700 1 -- 172.21.15.33:0/827846965 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3cc0033f0 con 0x7fe3e4153e60 2023-11-06T22:18:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.069+0000 7fe3e2ffd700 1 -- 172.21.15.33:0/827846965 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419889257 0 0) 0x7fe3e40c4360 con 0x7fe3e40a6d40 2023-11-06T22:18:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.069+0000 7fe3e2ffd700 1 -- 172.21.15.33:0/827846965 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e40c2ec0 con 0x7fe3e40a6d40 2023-11-06T22:18:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.069+0000 7fe3e899c700 1 -- 172.21.15.33:0/827846965 >> v1:172.21.15.69:6789/0 conn(0x7fe3e40a6d40 legacy=0x7fe3e4062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.069+0000 7fe3e899c700 1 -- 172.21.15.33:0/827846965 >> v1:172.21.15.33:6789/0 conn(0x7fe3e4153e60 legacy=0x7fe3e4069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.069+0000 7fe3e899c700 1 -- 172.21.15.33:0/827846965 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40a73d0 con 0x7fe3e4062e20 2023-11-06T22:18:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.070+0000 7fe3e899c700 1 --2- 172.21.15.33:0/827846965 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4062e20 0x7fe3e4154240 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fe3dc009e90 tx=0x7fe3dc009780).ready entity=mon.2 client_cookie=7d3826fcee3084d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.070+0000 7fe3e2ffd700 1 -- 172.21.15.33:0/827846965 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc00e070 con 0x7fe3e4062e20 2023-11-06T22:18:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.070+0000 7fe3e2ffd700 1 -- 172.21.15.33:0/827846965 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc00bab0 con 0x7fe3e4062e20 2023-11-06T22:18:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.070+0000 7fe3e2ffd700 1 -- 172.21.15.33:0/827846965 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc00be30 con 0x7fe3e4062e20 2023-11-06T22:18:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.070+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/827846965 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4062e20 msgr2=0x7fe3e4154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.070+0000 7fe3ea3ff700 1 --2- 172.21.15.33:0/827846965 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4062e20 0x7fe3e4154240 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fe3dc009e90 tx=0x7fe3dc009780).stop 2023-11-06T22:18:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.070+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/827846965 shutdown_connections 2023-11-06T22:18:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.070+0000 7fe3ea3ff700 1 --2- 172.21.15.33:0/827846965 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4062e20 0x7fe3e4154240 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.070+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/827846965 >> 172.21.15.33:0/827846965 conn(0x7fe3e40c7b50 msgr2=0x7fe3e40c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.071+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/827846965 shutdown_connections 2023-11-06T22:18:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.071+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/827846965 wait complete. 2023-11-06T22:18:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.072+0000 7fe3ea3ff700 1 Processor -- start 2023-11-06T22:18:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.072+0000 7fe3ea3ff700 1 -- start start 2023-11-06T22:18:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.073+0000 7fe3ea3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e4062e20 0x7fe3e41516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.073+0000 7fe3ea3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3e40a6d40 0x7fe3e4152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.073+0000 7fe3ea3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4153e60 0x7fe3e4152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.073+0000 7fe3ea3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4152e10 con 0x7fe3e40a6d40 2023-11-06T22:18:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.073+0000 7fe3ea3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4164d40 con 0x7fe3e4062e20 2023-11-06T22:18:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.073+0000 7fe3ea3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4164410 con 0x7fe3e4153e60 2023-11-06T22:18:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.073+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3e40a6d40 0x7fe3e4152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.073+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3e40a6d40 0x7fe3e4152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49200/0 (socket says 172.21.15.33:49200) 2023-11-06T22:18:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.073+0000 7fe3e37fe700 1 -- 172.21.15.33:0/2891576688 learned_addr learned my addr 172.21.15.33:0/2891576688 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3e899c700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4153e60 0x7fe3e4152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3e3fff700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e4062e20 0x7fe3e41516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3e37fe700 1 -- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4153e60 msgr2=0x7fe3e4152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3e37fe700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4153e60 0x7fe3e4152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3e37fe700 1 -- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e4062e20 msgr2=0x7fe3e41516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3e37fe700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e4062e20 0x7fe3e41516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3e37fe700 1 -- 172.21.15.33:0/2891576688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc009a20 con 0x7fe3e40a6d40 2023-11-06T22:18:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3e37fe700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3e40a6d40 0x7fe3e4152430 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7fe3d800c4a0 tx=0x7fe3d8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3cbfff700 1 -- 172.21.15.33:0/2891576688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800f470 con 0x7fe3e40a6d40 2023-11-06T22:18:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3cbfff700 1 -- 172.21.15.33:0/2891576688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d800f5d0 con 0x7fe3e40a6d40 2023-11-06T22:18:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3cbfff700 1 -- 172.21.15.33:0/2891576688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800f970 con 0x7fe3e40a6d40 2023-11-06T22:18:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/2891576688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e41646f0 con 0x7fe3e40a6d40 2023-11-06T22:18:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.074+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/2891576688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e41640c0 con 0x7fe3e40a6d40 2023-11-06T22:18:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.076+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/2891576688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e4164ef0 con 0x7fe3e40a6d40 2023-11-06T22:18:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.076+0000 7fe3cbfff700 1 -- 172.21.15.33:0/2891576688 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe3d800fd20 con 0x7fe3e40a6d40 2023-11-06T22:18:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.077+0000 7fe3cbfff700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe3d005f2a0 0x7fe3d0061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.077+0000 7fe3cbfff700 1 -- 172.21.15.33:0/2891576688 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fe3d8079ae0 con 0x7fe3e40a6d40 2023-11-06T22:18:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.078+0000 7fe3e3fff700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe3d005f2a0 0x7fe3d0061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.078+0000 7fe3e3fff700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe3d005f2a0 0x7fe3d0061760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fe3d4005750 tx=0x7fe3d4009040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.081+0000 7fe3cbfff700 1 -- 172.21.15.33:0/2891576688 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe3e4164ef0 con 0x7fe3e40a6d40 2023-11-06T22:18:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.305+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/2891576688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3e41650b0 con 0x7fe3e40a6d40 2023-11-06T22:18:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.306+0000 7fe3cbfff700 1 -- 172.21.15.33:0/2891576688 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4493 (secure 0 0 0) 0x7fe3d804aae0 con 0x7fe3e40a6d40 2023-11-06T22:18:26.307 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:26.308 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":91,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3097639551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3097639551},{"type":"v1","addr":"172.21.15.69:6837","nonce":3097639551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9828,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1256245016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1256245016},{"type":"v1","addr":"172.21.15.33:6834","nonce":1256245016}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1680210655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1680210655},{"type":"v1","addr":"172.21.15.33:6835","nonce":1680210655}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:18:22.834249+0000","modified":"2023-11-06T22:18:25.840897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9837":{"gid":9837,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/3921249578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3921249578},{"type":"v1","addr":"172.21.15.69:6836","nonce":3921249578}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-06T22:18:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe3d005f2a0 msgr2=0x7fe3d0061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe3d005f2a0 0x7fe3d0061760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fe3d4005750 tx=0x7fe3d4009040).stop 2023-11-06T22:18:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/2891576688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3e40a6d40 msgr2=0x7fe3e4152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3e40a6d40 0x7fe3e4152430 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7fe3d800c4a0 tx=0x7fe3d8007c50).stop 2023-11-06T22:18:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/2891576688 shutdown_connections 2023-11-06T22:18:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe3d005f2a0 0x7fe3d0061760 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe3e4153e60 0x7fe3e4152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e4062e20 0x7fe3e41516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 --2- 172.21.15.33:0/2891576688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3e40a6d40 0x7fe3e4152430 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/2891576688 >> 172.21.15.33:0/2891576688 conn(0x7fe3e40c7b50 msgr2=0x7fe3e40bd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/2891576688 shutdown_connections 2023-11-06T22:18:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.308+0000 7fe3ea3ff700 1 -- 172.21.15.33:0/2891576688 wait complete. 2023-11-06T22:18:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 91 2023-11-06T22:18:26.327 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:26.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.522+0000 7f0675fff700 1 Processor -- start 2023-11-06T22:18:26.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.522+0000 7f0675fff700 1 -- start start 2023-11-06T22:18:26.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.523+0000 7f0675fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0670063140 0x7f0670063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.523+0000 7f0675fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06700c3680 0x7f0670154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.523+0000 7f0675fff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06700c31f0 con 0x7f067005e1b0 2023-11-06T22:18:26.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.523+0000 7f0675fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067006a0f0 con 0x7f0670063140 2023-11-06T22:18:26.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.523+0000 7f0675fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06700c2920 con 0x7f06700c3680 2023-11-06T22:18:26.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.523+0000 7f066f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0670063140 0x7f0670063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.523+0000 7f066f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0670063140 0x7f0670063520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49206/0 (socket says 172.21.15.33:49206) 2023-11-06T22:18:26.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.523+0000 7f066f7fe700 1 -- 172.21.15.33:0/1598689517 learned_addr learned my addr 172.21.15.33:0/1598689517 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:26.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.524+0000 7f066ffff700 1 --2- 172.21.15.33:0/1598689517 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06700c3680 0x7f0670154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.524+0000 7f066f7fe700 1 -- 172.21.15.33:0/1598689517 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06700c3680 msgr2=0x7f0670154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:26.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.524+0000 7f066f7fe700 1 --2- 172.21.15.33:0/1598689517 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06700c3680 0x7f0670154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.524+0000 7f066f7fe700 1 -- 172.21.15.33:0/1598689517 >> v1:172.21.15.69:6789/0 conn(0x7f067005e1b0 legacy=0x7f06700c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.524+0000 7f066f7fe700 1 -- 172.21.15.33:0/1598689517 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067005e640 con 0x7f0670063140 2023-11-06T22:18:26.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.524+0000 7f066f7fe700 1 --2- 172.21.15.33:0/1598689517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 0x7f0670063520 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 rx=0x7f0660006fd0 tx=0x7f06600090f0).ready entity=mon.0 client_cookie=778baf1608faa13a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:26.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.524+0000 7f066e7fc700 1 -- 172.21.15.33:0/1598689517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000b600 con 0x7f0670063140 2023-11-06T22:18:26.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.524+0000 7f066e7fc700 1 -- 172.21.15.33:0/1598689517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066000b760 con 0x7f0670063140 2023-11-06T22:18:26.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.524+0000 7f066e7fc700 1 -- 172.21.15.33:0/1598689517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000bae0 con 0x7f0670063140 2023-11-06T22:18:26.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.525+0000 7f0675fff700 1 -- 172.21.15.33:0/1598689517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 msgr2=0x7f0670063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.525+0000 7f0675fff700 1 --2- 172.21.15.33:0/1598689517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 0x7f0670063520 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 rx=0x7f0660006fd0 tx=0x7f06600090f0).stop 2023-11-06T22:18:26.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.525+0000 7f0675fff700 1 -- 172.21.15.33:0/1598689517 shutdown_connections 2023-11-06T22:18:26.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.525+0000 7f0675fff700 1 --2- 172.21.15.33:0/1598689517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 0x7f0670063520 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.525+0000 7f0675fff700 1 --2- 172.21.15.33:0/1598689517 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06700c3680 0x7f0670154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.525+0000 7f0675fff700 1 -- 172.21.15.33:0/1598689517 >> 172.21.15.33:0/1598689517 conn(0x7f06700c7d60 msgr2=0x7f06700c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.525+0000 7f0675fff700 1 -- 172.21.15.33:0/1598689517 shutdown_connections 2023-11-06T22:18:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.525+0000 7f0675fff700 1 -- 172.21.15.33:0/1598689517 wait complete. 2023-11-06T22:18:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.526+0000 7f0675fff700 1 Processor -- start 2023-11-06T22:18:26.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.527+0000 7f0675fff700 1 -- start start 2023-11-06T22:18:26.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f0675fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 0x7f067013f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f0675fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06700c3680 0x7f067013c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f0675fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0670068a20 0x7f0670068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f0675fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0670069370 con 0x7f0670063140 2023-11-06T22:18:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f0675fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067013e790 con 0x7f0670068a20 2023-11-06T22:18:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f0675fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067013ea70 con 0x7f06700c3680 2023-11-06T22:18:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 0x7f067013f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 0x7f067013f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49214/0 (socket says 172.21.15.33:49214) 2023-11-06T22:18:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066f7fe700 1 -- 172.21.15.33:0/533708135 learned_addr learned my addr 172.21.15.33:0/533708135 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066effd700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06700c3680 0x7f067013c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066ffff700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0670068a20 0x7f0670068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066f7fe700 1 -- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06700c3680 msgr2=0x7f067013c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066f7fe700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06700c3680 0x7f067013c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066f7fe700 1 -- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0670068a20 msgr2=0x7f0670068e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066f7fe700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0670068a20 0x7f0670068e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066f7fe700 1 -- 172.21.15.33:0/533708135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0660009370 con 0x7f0670063140 2023-11-06T22:18:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.528+0000 7f066f7fe700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 0x7f067013f830 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f067005ebe0 tx=0x7f0660017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.529+0000 7f06577fe700 1 -- 172.21.15.33:0/533708135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000b8d0 con 0x7f0670063140 2023-11-06T22:18:26.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.529+0000 7f06577fe700 1 -- 172.21.15.33:0/533708135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066000ba30 con 0x7f0670063140 2023-11-06T22:18:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.529+0000 7f0675fff700 1 -- 172.21.15.33:0/533708135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067013ed50 con 0x7f0670063140 2023-11-06T22:18:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.529+0000 7f06577fe700 1 -- 172.21.15.33:0/533708135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066001c670 con 0x7f0670063140 2023-11-06T22:18:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.529+0000 7f0675fff700 1 -- 172.21.15.33:0/533708135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067013f310 con 0x7f0670063140 2023-11-06T22:18:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.530+0000 7f0675fff700 1 -- 172.21.15.33:0/533708135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0670064420 con 0x7f0670063140 2023-11-06T22:18:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.531+0000 7f06577fe700 1 -- 172.21.15.33:0/533708135 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f066001ca20 con 0x7f0670063140 2023-11-06T22:18:26.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.531+0000 7f06577fe700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f065c05f270 0x7f065c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.532+0000 7f06577fe700 1 -- 172.21.15.33:0/533708135 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f066007c2c0 con 0x7f0670063140 2023-11-06T22:18:26.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.532+0000 7f066effd700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f065c05f270 0x7f065c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.532+0000 7f066effd700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f065c05f270 0x7f065c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f06700c5690 tx=0x7f0664009040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:26.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.536+0000 7f06577fe700 1 -- 172.21.15.33:0/533708135 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0660050d50 con 0x7f0670063140 2023-11-06T22:18:26.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.717+0000 7f0675fff700 1 -- 172.21.15.33:0/533708135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06700ad050 con 0x7f0670063140 2023-11-06T22:18:26.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.718+0000 7f06577fe700 1 -- 172.21.15.33:0/533708135 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v223) v1 ==== 74+0+17061 (secure 0 0 0) 0x7f066002a020 con 0x7f0670063140 2023-11-06T22:18:26.720 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:26.720 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":223,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:23.836657+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:18:21.317255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T22:18:21.822232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:26.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 -- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f065c05f270 msgr2=0x7f065c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f065c05f270 0x7f065c061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f06700c5690 tx=0x7f0664009040).stop 2023-11-06T22:18:26.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 -- 172.21.15.33:0/533708135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 msgr2=0x7f067013f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 0x7f067013f830 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f067005ebe0 tx=0x7f0660017000).stop 2023-11-06T22:18:26.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 -- 172.21.15.33:0/533708135 shutdown_connections 2023-11-06T22:18:26.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f065c05f270 0x7f065c061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0670068a20 0x7f0670068e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0670063140 0x7f067013f830 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 --2- 172.21.15.33:0/533708135 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f06700c3680 0x7f067013c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 -- 172.21.15.33:0/533708135 >> 172.21.15.33:0/533708135 conn(0x7f06700c7d60 msgr2=0x7f0670151cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:26.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 -- 172.21.15.33:0/533708135 shutdown_connections 2023-11-06T22:18:26.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.720+0000 7f0675fff700 1 -- 172.21.15.33:0/533708135 wait complete. 2023-11-06T22:18:26.740 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T22:18:26.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:18:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.936+0000 7f06317ff700 1 Processor -- start 2023-11-06T22:18:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.936+0000 7f06317ff700 1 -- start start 2023-11-06T22:18:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.937+0000 7f06317ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f062c0c1840 0x7f062c0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.937+0000 7f06317ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f062c0aa2d0 0x7f062c0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.937+0000 7f06317ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062c1554a0 con 0x7f062c0c6240 2023-11-06T22:18:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.937+0000 7f06317ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062c154960 con 0x7f062c0aa2d0 2023-11-06T22:18:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.937+0000 7f06317ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062c155930 con 0x7f062c0c1840 2023-11-06T22:18:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.938+0000 7f062a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f062c0aa2d0 0x7f062c0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.938+0000 7f062a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f062c0aa2d0 0x7f062c0c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49228/0 (socket says 172.21.15.33:49228) 2023-11-06T22:18:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.938+0000 7f062a7fc700 1 -- 172.21.15.33:0/3217949864 learned_addr learned my addr 172.21.15.33:0/3217949864 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.938+0000 7f062affd700 1 --2- 172.21.15.33:0/3217949864 >> v2:172.21.15.69:3300/0 conn(0x7f062c0c1840 0x7f062c0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.938+0000 7f062a7fc700 1 -- 172.21.15.33:0/3217949864 >> v2:172.21.15.69:3300/0 conn(0x7f062c0c1840 msgr2=0x7f062c0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.938+0000 7f062a7fc700 1 --2- 172.21.15.33:0/3217949864 >> v2:172.21.15.69:3300/0 conn(0x7f062c0c1840 0x7f062c0a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.938+0000 7f062a7fc700 1 -- 172.21.15.33:0/3217949864 >> v1:172.21.15.33:6789/0 conn(0x7f062c0c6240 legacy=0x7f062c157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.938+0000 7f0629ffb700 1 -- 172.21.15.33:0/3217949864 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967451262 0 0) 0x7f062c1554a0 con 0x7f062c0c6240 2023-11-06T22:18:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.938+0000 7f062a7fc700 1 -- 172.21.15.33:0/3217949864 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c154b70 con 0x7f062c0aa2d0 2023-11-06T22:18:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.938+0000 7f062a7fc700 1 --2- 172.21.15.33:0/3217949864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 0x7f062c0c5d50 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7f062c0840b0 tx=0x7f06200090f0).ready entity=mon.0 client_cookie=88aa124188832329 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f0629ffb700 1 -- 172.21.15.33:0/3217949864 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062000b520 con 0x7f062c0aa2d0 2023-11-06T22:18:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f0629ffb700 1 -- 172.21.15.33:0/3217949864 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062000b680 con 0x7f062c0aa2d0 2023-11-06T22:18:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f0629ffb700 1 -- 172.21.15.33:0/3217949864 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062000ba00 con 0x7f062c0aa2d0 2023-11-06T22:18:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f06317ff700 1 -- 172.21.15.33:0/3217949864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 msgr2=0x7f062c0c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f06317ff700 1 --2- 172.21.15.33:0/3217949864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 0x7f062c0c5d50 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7f062c0840b0 tx=0x7f06200090f0).stop 2023-11-06T22:18:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f06317ff700 1 -- 172.21.15.33:0/3217949864 shutdown_connections 2023-11-06T22:18:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f06317ff700 1 --2- 172.21.15.33:0/3217949864 >> v2:172.21.15.69:3300/0 conn(0x7f062c0c1840 0x7f062c0a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f06317ff700 1 --2- 172.21.15.33:0/3217949864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 0x7f062c0c5d50 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f06317ff700 1 -- 172.21.15.33:0/3217949864 >> 172.21.15.33:0/3217949864 conn(0x7f062c05ec80 msgr2=0x7f062c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f06317ff700 1 -- 172.21.15.33:0/3217949864 shutdown_connections 2023-11-06T22:18:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.939+0000 7f06317ff700 1 -- 172.21.15.33:0/3217949864 wait complete. 2023-11-06T22:18:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.941+0000 7f06317ff700 1 Processor -- start 2023-11-06T22:18:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.941+0000 7f06317ff700 1 -- start start 2023-11-06T22:18:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f06317ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 0x7f062c0a9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f06317ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f062c0c1840 0x7f062c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f06317ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f062c0c6240 0x7f062c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f06317ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c154490 con 0x7f062c0aa2d0 2023-11-06T22:18:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f06317ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c167a50 con 0x7f062c0c1840 2023-11-06T22:18:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f06317ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062c165220 con 0x7f062c0c6240 2023-11-06T22:18:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f062affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 0x7f062c0a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f062affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 0x7f062c0a9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49240/0 (socket says 172.21.15.33:49240) 2023-11-06T22:18:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f062affd700 1 -- 172.21.15.33:0/673678601 learned_addr learned my addr 172.21.15.33:0/673678601 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f062b7fe700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f062c0c6240 0x7f062c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.942+0000 7f062a7fc700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f062c0c1840 0x7f062c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f062affd700 1 -- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f062c0c6240 msgr2=0x7f062c153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f062affd700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f062c0c6240 0x7f062c153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f062affd700 1 -- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f062c0c1840 msgr2=0x7f062c153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f062affd700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f062c0c1840 0x7f062c153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f062affd700 1 -- 172.21.15.33:0/673678601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0620009370 con 0x7f062c0aa2d0 2023-11-06T22:18:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f062a7fc700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f062c0c1840 0x7f062c153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f062affd700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 0x7f062c0a9530 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f061c00e930 tx=0x7f061c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f0612ffd700 1 -- 172.21.15.33:0/673678601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c00eed0 con 0x7f062c0aa2d0 2023-11-06T22:18:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f0612ffd700 1 -- 172.21.15.33:0/673678601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c00c850 con 0x7f062c0aa2d0 2023-11-06T22:18:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f0612ffd700 1 -- 172.21.15.33:0/673678601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c00cbd0 con 0x7f062c0aa2d0 2023-11-06T22:18:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f06317ff700 1 -- 172.21.15.33:0/673678601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c167ca0 con 0x7f062c0aa2d0 2023-11-06T22:18:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.943+0000 7f06317ff700 1 -- 172.21.15.33:0/673678601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c168260 con 0x7f062c0aa2d0 2023-11-06T22:18:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.944+0000 7f06317ff700 1 -- 172.21.15.33:0/673678601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062c06c6c0 con 0x7f062c0aa2d0 2023-11-06T22:18:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.945+0000 7f0612ffd700 1 -- 172.21.15.33:0/673678601 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f061c01b400 con 0x7f062c0aa2d0 2023-11-06T22:18:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.946+0000 7f0612ffd700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f061805f270 0x7f0618061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.946+0000 7f0612ffd700 1 -- 172.21.15.33:0/673678601 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f061c07aa70 con 0x7f062c0aa2d0 2023-11-06T22:18:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.946+0000 7f062a7fc700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f061805f270 0x7f0618061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.947+0000 7f062a7fc700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f061805f270 0x7f0618061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f062c0aa6d0 tx=0x7f0620017000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:26.950+0000 7f0612ffd700 1 -- 172.21.15.33:0/673678601 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f061c04ba70 con 0x7f062c0aa2d0 2023-11-06T22:18:27.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.125+0000 7f06317ff700 1 -- 172.21.15.33:0/673678601 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f062c063f20 con 0x7f061805f270 2023-11-06T22:18:27.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.139+0000 7f0612ffd700 1 -- 172.21.15.33:0/673678601 <== mgr.9310 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361758 (secure 0 0 0) 0x7f062c063f20 con 0x7f061805f270 2023-11-06T22:18:27.145 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:27.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 -- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f061805f270 msgr2=0x7f0618061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f061805f270 0x7f0618061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f062c0aa6d0 tx=0x7f0620017000).stop 2023-11-06T22:18:27.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 -- 172.21.15.33:0/673678601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 msgr2=0x7f062c0a9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 0x7f062c0a9530 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f061c00e930 tx=0x7f061c009060).stop 2023-11-06T22:18:27.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 -- 172.21.15.33:0/673678601 shutdown_connections 2023-11-06T22:18:27.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f061805f270 0x7f0618061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f062c0c6240 0x7f062c153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f062c0aa2d0 0x7f062c0a9530 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 --2- 172.21.15.33:0/673678601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f062c0c1840 0x7f062c153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 -- 172.21.15.33:0/673678601 >> 172.21.15.33:0/673678601 conn(0x7f062c05ec80 msgr2=0x7f062c152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:27.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 -- 172.21.15.33:0/673678601 shutdown_connections 2023-11-06T22:18:27.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.143+0000 7f06317ff700 1 -- 172.21.15.33:0/673678601 wait complete. 2023-11-06T22:18:27.151 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:18:27.163 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T22:18:25.551583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2464112,"kb_used_data":21808,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752510608,"statfs":{"total":773094113280,"available":770570862592,"internally_reserved":0,"allocated":22331392,"data_stored":11469156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-31423,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-84,"num_write_kb":-67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":-10,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.005926"},"pg_stats":[{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871810+0000","last_change":"2023-11-06T22:18:22.861504+0000","last_active":"2023-11-06T22:18:22.871810+0000","last_peered":"2023-11-06T22:18:22.871810+0000","last_clean":"2023-11-06T22:18:22.871810+0000","last_became_active":"2023-11-06T22:18:22.861190+0000","last_became_peered":"2023-11-06T22:18:22.861190+0000","last_unstale":"2023-11-06T22:18:22.871810+0000","last_undegraded":"2023-11-06T22:18:22.871810+0000","last_fullsized":"2023-11-06T22:18:22.871810+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3f","version":"222'3","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.874350+0000","last_change":"2023-11-06T22:18:22.381412+0000","last_active":"2023-11-06T22:18:22.874350+0000","last_peered":"2023-11-06T22:18:22.874350+0000","last_clean":"2023-11-06T22:18:22.874350+0000","last_became_active":"2023-11-06T22:18:22.379227+0000","last_became_peered":"2023-11-06T22:18:22.379227+0000","last_unstale":"2023-11-06T22:18:22.874350+0000","last_undegraded":"2023-11-06T22:18:22.874350+0000","last_fullsized":"2023-11-06T22:18:22.874350+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.873793+0000","last_change":"2023-11-06T22:18:22.867712+0000","last_active":"2023-11-06T22:18:22.873793+0000","last_peered":"2023-11-06T22:18:22.873793+0000","last_clean":"2023-11-06T22:18:22.873793+0000","last_became_active":"2023-11-06T22:18:22.867287+0000","last_became_peered":"2023-11-06T22:18:22.867287+0000","last_unstale":"2023-11-06T22:18:22.873793+0000","last_undegraded":"2023-11-06T22:18:22.873793+0000","last_fullsized":"2023-11-06T22:18:22.873793+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.382979+0000","last_change":"2023-11-06T22:18:22.378731+0000","last_active":"2023-11-06T22:18:22.382979+0000","last_peered":"2023-11-06T22:18:22.382979+0000","last_clean":"2023-11-06T22:18:22.382979+0000","last_became_active":"2023-11-06T22:18:22.378439+0000","last_became_peered":"2023-11-06T22:18:22.378439+0000","last_unstale":"2023-11-06T22:18:22.382979+0000","last_undegraded":"2023-11-06T22:18:22.382979+0000","last_fullsized":"2023-11-06T22:18:22.382979+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.881008+0000","last_change":"2023-11-06T22:18:22.879970+0000","last_active":"2023-11-06T22:18:22.881008+0000","last_peered":"2023-11-06T22:18:22.881008+0000","last_clean":"2023-11-06T22:18:22.881008+0000","last_became_active":"2023-11-06T22:18:22.879584+0000","last_became_peered":"2023-11-06T22:18:22.879584+0000","last_unstale":"2023-11-06T22:18:22.881008+0000","last_undegraded":"2023-11-06T22:18:22.881008+0000","last_fullsized":"2023-11-06T22:18:22.881008+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.369038+0000","last_change":"2023-11-06T22:18:22.356342+0000","last_active":"2023-11-06T22:18:22.369038+0000","last_peered":"2023-11-06T22:18:22.369038+0000","last_clean":"2023-11-06T22:18:22.369038+0000","last_became_active":"2023-11-06T22:18:22.355225+0000","last_became_peered":"2023-11-06T22:18:22.355225+0000","last_unstale":"2023-11-06T22:18:22.369038+0000","last_undegraded":"2023-11-06T22:18:22.369038+0000","last_fullsized":"2023-11-06T22:18:22.369038+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3f","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.850680+0000","last_change":"2023-11-06T22:18:22.850680+0000","last_active":"2023-11-06T22:18:22.850680+0000","last_peered":"2023-11-06T22:18:22.850680+0000","last_clean":"2023-11-06T22:18:22.850680+0000","last_became_active":"2023-11-06T22:18:22.849640+0000","last_became_peered":"2023-11-06T22:18:22.849640+0000","last_unstale":"2023-11-06T22:18:22.850680+0000","last_undegraded":"2023-11-06T22:18:22.850680+0000","last_fullsized":"2023-11-06T22:18:22.850680+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"222'2","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.856191+0000","last_change":"2023-11-06T22:18:22.376417+0000","last_active":"2023-11-06T22:18:22.856191+0000","last_peered":"2023-11-06T22:18:22.856191+0000","last_clean":"2023-11-06T22:18:22.856191+0000","last_became_active":"2023-11-06T22:18:22.376125+0000","last_became_peered":"2023-11-06T22:18:22.376125+0000","last_unstale":"2023-11-06T22:18:22.856191+0000","last_undegraded":"2023-11-06T22:18:22.856191+0000","last_fullsized":"2023-11-06T22:18:22.856191+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866585+0000","last_change":"2023-11-06T22:18:22.862316+0000","last_active":"2023-11-06T22:18:22.866585+0000","last_peered":"2023-11-06T22:18:22.866585+0000","last_clean":"2023-11-06T22:18:22.866585+0000","last_became_active":"2023-11-06T22:18:22.862083+0000","last_became_peered":"2023-11-06T22:18:22.862083+0000","last_unstale":"2023-11-06T22:18:22.866585+0000","last_undegraded":"2023-11-06T22:18:22.866585+0000","last_fullsized":"2023-11-06T22:18:22.866585+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3b","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.876467+0000","last_change":"2023-11-06T22:18:22.380952+0000","last_active":"2023-11-06T22:18:22.876467+0000","last_peered":"2023-11-06T22:18:22.876467+0000","last_clean":"2023-11-06T22:18:22.876467+0000","last_became_active":"2023-11-06T22:18:22.380507+0000","last_became_peered":"2023-11-06T22:18:22.380507+0000","last_unstale":"2023-11-06T22:18:22.876467+0000","last_undegraded":"2023-11-06T22:18:22.876467+0000","last_fullsized":"2023-11-06T22:18:22.876467+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.850758+0000","last_change":"2023-11-06T22:18:22.850758+0000","last_active":"2023-11-06T22:18:22.850758+0000","last_peered":"2023-11-06T22:18:22.850758+0000","last_clean":"2023-11-06T22:18:22.850758+0000","last_became_active":"2023-11-06T22:18:22.850443+0000","last_became_peered":"2023-11-06T22:18:22.850443+0000","last_unstale":"2023-11-06T22:18:22.850758+0000","last_undegraded":"2023-11-06T22:18:22.850758+0000","last_fullsized":"2023-11-06T22:18:22.850758+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3a","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.874181+0000","last_change":"2023-11-06T22:18:22.377088+0000","last_active":"2023-11-06T22:18:22.874181+0000","last_peered":"2023-11-06T22:18:22.874181+0000","last_clean":"2023-11-06T22:18:22.874181+0000","last_became_active":"2023-11-06T22:18:22.374923+0000","last_became_peered":"2023-11-06T22:18:22.374923+0000","last_unstale":"2023-11-06T22:18:22.874181+0000","last_undegraded":"2023-11-06T22:18:22.874181+0000","last_fullsized":"2023-11-06T22:18:22.874181+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.875306+0000","last_change":"2023-11-06T22:18:22.867250+0000","last_active":"2023-11-06T22:18:22.875306+0000","last_peered":"2023-11-06T22:18:22.875306+0000","last_clean":"2023-11-06T22:18:22.875306+0000","last_became_active":"2023-11-06T22:18:22.866842+0000","last_became_peered":"2023-11-06T22:18:22.866842+0000","last_unstale":"2023-11-06T22:18:22.875306+0000","last_undegraded":"2023-11-06T22:18:22.875306+0000","last_fullsized":"2023-11-06T22:18:22.875306+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.858010+0000","last_change":"2023-11-06T22:18:22.371098+0000","last_active":"2023-11-06T22:18:22.858010+0000","last_peered":"2023-11-06T22:18:22.858010+0000","last_clean":"2023-11-06T22:18:22.858010+0000","last_became_active":"2023-11-06T22:18:22.369819+0000","last_became_peered":"2023-11-06T22:18:22.369819+0000","last_unstale":"2023-11-06T22:18:22.858010+0000","last_undegraded":"2023-11-06T22:18:22.858010+0000","last_fullsized":"2023-11-06T22:18:22.858010+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.864341+0000","last_change":"2023-11-06T22:18:22.864341+0000","last_active":"2023-11-06T22:18:22.864341+0000","last_peered":"2023-11-06T22:18:22.864341+0000","last_clean":"2023-11-06T22:18:22.864341+0000","last_became_active":"2023-11-06T22:18:22.864034+0000","last_became_peered":"2023-11-06T22:18:22.864034+0000","last_unstale":"2023-11-06T22:18:22.864341+0000","last_undegraded":"2023-11-06T22:18:22.864341+0000","last_fullsized":"2023-11-06T22:18:22.864341+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.859812+0000","last_change":"2023-11-06T22:18:22.382620+0000","last_active":"2023-11-06T22:18:22.859812+0000","last_peered":"2023-11-06T22:18:22.859812+0000","last_clean":"2023-11-06T22:18:22.859812+0000","last_became_active":"2023-11-06T22:18:22.381716+0000","last_became_peered":"2023-11-06T22:18:22.381716+0000","last_unstale":"2023-11-06T22:18:22.859812+0000","last_undegraded":"2023-11-06T22:18:22.859812+0000","last_fullsized":"2023-11-06T22:18:22.859812+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878443+0000","last_change":"2023-11-06T22:18:22.874837+0000","last_active":"2023-11-06T22:18:22.878443+0000","last_peered":"2023-11-06T22:18:22.878443+0000","last_clean":"2023-11-06T22:18:22.878443+0000","last_became_active":"2023-11-06T22:18:22.872927+0000","last_became_peered":"2023-11-06T22:18:22.872927+0000","last_unstale":"2023-11-06T22:18:22.878443+0000","last_undegraded":"2023-11-06T22:18:22.878443+0000","last_fullsized":"2023-11-06T22:18:22.878443+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.381800+0000","last_change":"2023-11-06T22:18:22.377682+0000","last_active":"2023-11-06T22:18:22.381800+0000","last_peered":"2023-11-06T22:18:22.381800+0000","last_clean":"2023-11-06T22:18:22.381800+0000","last_became_active":"2023-11-06T22:18:22.377304+0000","last_became_peered":"2023-11-06T22:18:22.377304+0000","last_unstale":"2023-11-06T22:18:22.381800+0000","last_undegraded":"2023-11-06T22:18:22.381800+0000","last_fullsized":"2023-11-06T22:18:22.381800+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.849872+0000","last_change":"2023-11-06T22:18:22.849872+0000","last_active":"2023-11-06T22:18:22.849872+0000","last_peered":"2023-11-06T22:18:22.849872+0000","last_clean":"2023-11-06T22:18:22.849872+0000","last_became_active":"2023-11-06T22:18:22.849613+0000","last_became_peered":"2023-11-06T22:18:22.849613+0000","last_unstale":"2023-11-06T22:18:22.849872+0000","last_undegraded":"2023-11-06T22:18:22.849872+0000","last_fullsized":"2023-11-06T22:18:22.849872+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.849063+0000","last_change":"2023-11-06T22:18:22.375248+0000","last_active":"2023-11-06T22:18:22.849063+0000","last_peered":"2023-11-06T22:18:22.849063+0000","last_clean":"2023-11-06T22:18:22.849063+0000","last_became_active":"2023-11-06T22:18:22.374934+0000","last_became_peered":"2023-11-06T22:18:22.374934+0000","last_unstale":"2023-11-06T22:18:22.849063+0000","last_undegraded":"2023-11-06T22:18:22.849063+0000","last_fullsized":"2023-11-06T22:18:22.849063+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868558+0000","last_change":"2023-11-06T22:18:22.864984+0000","last_active":"2023-11-06T22:18:22.868558+0000","last_peered":"2023-11-06T22:18:22.868558+0000","last_clean":"2023-11-06T22:18:22.868558+0000","last_became_active":"2023-11-06T22:18:22.864595+0000","last_became_peered":"2023-11-06T22:18:22.864595+0000","last_unstale":"2023-11-06T22:18:22.868558+0000","last_undegraded":"2023-11-06T22:18:22.868558+0000","last_fullsized":"2023-11-06T22:18:22.868558+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840784+0000","last_change":"2023-11-06T22:18:22.378145+0000","last_active":"2023-11-06T22:18:22.840784+0000","last_peered":"2023-11-06T22:18:22.840784+0000","last_clean":"2023-11-06T22:18:22.840784+0000","last_became_active":"2023-11-06T22:18:22.377860+0000","last_became_peered":"2023-11-06T22:18:22.377860+0000","last_unstale":"2023-11-06T22:18:22.840784+0000","last_undegraded":"2023-11-06T22:18:22.840784+0000","last_fullsized":"2023-11-06T22:18:22.840784+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.874646+0000","last_change":"2023-11-06T22:18:22.865609+0000","last_active":"2023-11-06T22:18:22.874646+0000","last_peered":"2023-11-06T22:18:22.874646+0000","last_clean":"2023-11-06T22:18:22.874646+0000","last_became_active":"2023-11-06T22:18:22.865210+0000","last_became_peered":"2023-11-06T22:18:22.865210+0000","last_unstale":"2023-11-06T22:18:22.874646+0000","last_undegraded":"2023-11-06T22:18:22.874646+0000","last_fullsized":"2023-11-06T22:18:22.874646+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.34","version":"222'4","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.885284+0000","last_change":"2023-11-06T22:18:22.367605+0000","last_active":"2023-11-06T22:18:22.885284+0000","last_peered":"2023-11-06T22:18:22.885284+0000","last_clean":"2023-11-06T22:18:22.885284+0000","last_became_active":"2023-11-06T22:18:22.367308+0000","last_became_peered":"2023-11-06T22:18:22.367308+0000","last_unstale":"2023-11-06T22:18:22.885284+0000","last_undegraded":"2023-11-06T22:18:22.885284+0000","last_fullsized":"2023-11-06T22:18:22.885284+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878521+0000","last_change":"2023-11-06T22:18:22.876197+0000","last_active":"2023-11-06T22:18:22.878521+0000","last_peered":"2023-11-06T22:18:22.878521+0000","last_clean":"2023-11-06T22:18:22.878521+0000","last_became_active":"2023-11-06T22:18:22.874183+0000","last_became_peered":"2023-11-06T22:18:22.874183+0000","last_unstale":"2023-11-06T22:18:22.878521+0000","last_undegraded":"2023-11-06T22:18:22.878521+0000","last_fullsized":"2023-11-06T22:18:22.878521+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.33","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868592+0000","last_change":"2023-11-06T22:18:22.374349+0000","last_active":"2023-11-06T22:18:22.868592+0000","last_peered":"2023-11-06T22:18:22.868592+0000","last_clean":"2023-11-06T22:18:22.868592+0000","last_became_active":"2023-11-06T22:18:22.374063+0000","last_became_peered":"2023-11-06T22:18:22.374063+0000","last_unstale":"2023-11-06T22:18:22.868592+0000","last_undegraded":"2023-11-06T22:18:22.868592+0000","last_fullsized":"2023-11-06T22:18:22.868592+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.873960+0000","last_change":"2023-11-06T22:18:22.863483+0000","last_active":"2023-11-06T22:18:22.873960+0000","last_peered":"2023-11-06T22:18:22.873960+0000","last_clean":"2023-11-06T22:18:22.873960+0000","last_became_active":"2023-11-06T22:18:22.863213+0000","last_became_peered":"2023-11-06T22:18:22.863213+0000","last_unstale":"2023-11-06T22:18:22.873960+0000","last_undegraded":"2023-11-06T22:18:22.873960+0000","last_fullsized":"2023-11-06T22:18:22.873960+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.382476+0000","last_change":"2023-11-06T22:18:22.377248+0000","last_active":"2023-11-06T22:18:22.382476+0000","last_peered":"2023-11-06T22:18:22.382476+0000","last_clean":"2023-11-06T22:18:22.382476+0000","last_became_active":"2023-11-06T22:18:22.376941+0000","last_became_peered":"2023-11-06T22:18:22.376941+0000","last_unstale":"2023-11-06T22:18:22.382476+0000","last_undegraded":"2023-11-06T22:18:22.382476+0000","last_fullsized":"2023-11-06T22:18:22.382476+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868451+0000","last_change":"2023-11-06T22:18:22.858079+0000","last_active":"2023-11-06T22:18:22.868451+0000","last_peered":"2023-11-06T22:18:22.868451+0000","last_clean":"2023-11-06T22:18:22.868451+0000","last_became_active":"2023-11-06T22:18:22.857735+0000","last_became_peered":"2023-11-06T22:18:22.857735+0000","last_unstale":"2023-11-06T22:18:22.868451+0000","last_undegraded":"2023-11-06T22:18:22.868451+0000","last_fullsized":"2023-11-06T22:18:22.868451+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840963+0000","last_change":"2023-11-06T22:18:22.369394+0000","last_active":"2023-11-06T22:18:22.840963+0000","last_peered":"2023-11-06T22:18:22.840963+0000","last_clean":"2023-11-06T22:18:22.840963+0000","last_became_active":"2023-11-06T22:18:22.369068+0000","last_became_peered":"2023-11-06T22:18:22.369068+0000","last_unstale":"2023-11-06T22:18:22.840963+0000","last_undegraded":"2023-11-06T22:18:22.840963+0000","last_fullsized":"2023-11-06T22:18:22.840963+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.859229+0000","last_change":"2023-11-06T22:18:22.849923+0000","last_active":"2023-11-06T22:18:22.859229+0000","last_peered":"2023-11-06T22:18:22.859229+0000","last_clean":"2023-11-06T22:18:22.859229+0000","last_became_active":"2023-11-06T22:18:22.849630+0000","last_became_peered":"2023-11-06T22:18:22.849630+0000","last_unstale":"2023-11-06T22:18:22.859229+0000","last_undegraded":"2023-11-06T22:18:22.859229+0000","last_fullsized":"2023-11-06T22:18:22.859229+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"222'4","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866065+0000","last_change":"2023-11-06T22:18:22.356545+0000","last_active":"2023-11-06T22:18:22.866065+0000","last_peered":"2023-11-06T22:18:22.866065+0000","last_clean":"2023-11-06T22:18:22.866065+0000","last_became_active":"2023-11-06T22:18:22.355132+0000","last_became_peered":"2023-11-06T22:18:22.355132+0000","last_unstale":"2023-11-06T22:18:22.866065+0000","last_undegraded":"2023-11-06T22:18:22.866065+0000","last_fullsized":"2023-11-06T22:18:22.866065+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.875350+0000","last_change":"2023-11-06T22:18:22.866087+0000","last_active":"2023-11-06T22:18:22.875350+0000","last_peered":"2023-11-06T22:18:22.875350+0000","last_clean":"2023-11-06T22:18:22.875350+0000","last_became_active":"2023-11-06T22:18:22.865667+0000","last_became_peered":"2023-11-06T22:18:22.865667+0000","last_unstale":"2023-11-06T22:18:22.875350+0000","last_undegraded":"2023-11-06T22:18:22.875350+0000","last_fullsized":"2023-11-06T22:18:22.875350+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.856844+0000","last_change":"2023-11-06T22:18:22.374053+0000","last_active":"2023-11-06T22:18:22.856844+0000","last_peered":"2023-11-06T22:18:22.856844+0000","last_clean":"2023-11-06T22:18:22.856844+0000","last_became_active":"2023-11-06T22:18:22.372649+0000","last_became_peered":"2023-11-06T22:18:22.372649+0000","last_unstale":"2023-11-06T22:18:22.856844+0000","last_undegraded":"2023-11-06T22:18:22.856844+0000","last_fullsized":"2023-11-06T22:18:22.856844+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878724+0000","last_change":"2023-11-06T22:18:22.872896+0000","last_active":"2023-11-06T22:18:22.878724+0000","last_peered":"2023-11-06T22:18:22.878724+0000","last_clean":"2023-11-06T22:18:22.878724+0000","last_became_active":"2023-11-06T22:18:22.872616+0000","last_became_peered":"2023-11-06T22:18:22.872616+0000","last_unstale":"2023-11-06T22:18:22.878724+0000","last_undegraded":"2023-11-06T22:18:22.878724+0000","last_fullsized":"2023-11-06T22:18:22.878724+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860889+0000","last_change":"2023-11-06T22:18:22.380112+0000","last_active":"2023-11-06T22:18:22.860889+0000","last_peered":"2023-11-06T22:18:22.860889+0000","last_clean":"2023-11-06T22:18:22.860889+0000","last_became_active":"2023-11-06T22:18:22.379807+0000","last_became_peered":"2023-11-06T22:18:22.379807+0000","last_unstale":"2023-11-06T22:18:22.860889+0000","last_undegraded":"2023-11-06T22:18:22.860889+0000","last_fullsized":"2023-11-06T22:18:22.860889+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866476+0000","last_change":"2023-11-06T22:18:22.866476+0000","last_active":"2023-11-06T22:18:22.866476+0000","last_peered":"2023-11-06T22:18:22.866476+0000","last_clean":"2023-11-06T22:18:22.866476+0000","last_became_active":"2023-11-06T22:18:22.866109+0000","last_became_peered":"2023-11-06T22:18:22.866109+0000","last_unstale":"2023-11-06T22:18:22.866476+0000","last_undegraded":"2023-11-06T22:18:22.866476+0000","last_fullsized":"2023-11-06T22:18:22.866476+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2d","version":"222'1","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.846309+0000","last_change":"2023-11-06T22:18:22.375723+0000","last_active":"2023-11-06T22:18:22.846309+0000","last_peered":"2023-11-06T22:18:22.846309+0000","last_clean":"2023-11-06T22:18:22.846309+0000","last_became_active":"2023-11-06T22:18:22.374350+0000","last_became_peered":"2023-11-06T22:18:22.374350+0000","last_unstale":"2023-11-06T22:18:22.846309+0000","last_undegraded":"2023-11-06T22:18:22.846309+0000","last_fullsized":"2023-11-06T22:18:22.846309+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.880263+0000","last_change":"2023-11-06T22:18:22.878279+0000","last_active":"2023-11-06T22:18:22.880263+0000","last_peered":"2023-11-06T22:18:22.880263+0000","last_clean":"2023-11-06T22:18:22.880263+0000","last_became_active":"2023-11-06T22:18:22.877828+0000","last_became_peered":"2023-11-06T22:18:22.877828+0000","last_unstale":"2023-11-06T22:18:22.880263+0000","last_undegraded":"2023-11-06T22:18:22.880263+0000","last_fullsized":"2023-11-06T22:18:22.880263+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862906+0000","last_change":"2023-11-06T22:18:22.364440+0000","last_active":"2023-11-06T22:18:22.862906+0000","last_peered":"2023-11-06T22:18:22.862906+0000","last_clean":"2023-11-06T22:18:22.862906+0000","last_became_active":"2023-11-06T22:18:22.364153+0000","last_became_peered":"2023-11-06T22:18:22.364153+0000","last_unstale":"2023-11-06T22:18:22.862906+0000","last_undegraded":"2023-11-06T22:18:22.862906+0000","last_fullsized":"2023-11-06T22:18:22.862906+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.879010+0000","last_change":"2023-11-06T22:18:22.872375+0000","last_active":"2023-11-06T22:18:22.879010+0000","last_peered":"2023-11-06T22:18:22.879010+0000","last_clean":"2023-11-06T22:18:22.879010+0000","last_became_active":"2023-11-06T22:18:22.871857+0000","last_became_peered":"2023-11-06T22:18:22.871857+0000","last_unstale":"2023-11-06T22:18:22.879010+0000","last_undegraded":"2023-11-06T22:18:22.879010+0000","last_fullsized":"2023-11-06T22:18:22.879010+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"222'1","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.844892+0000","last_change":"2023-11-06T22:18:22.379138+0000","last_active":"2023-11-06T22:18:22.844892+0000","last_peered":"2023-11-06T22:18:22.844892+0000","last_clean":"2023-11-06T22:18:22.844892+0000","last_became_active":"2023-11-06T22:18:22.376082+0000","last_became_peered":"2023-11-06T22:18:22.376082+0000","last_unstale":"2023-11-06T22:18:22.844892+0000","last_undegraded":"2023-11-06T22:18:22.844892+0000","last_fullsized":"2023-11-06T22:18:22.844892+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.873183+0000","last_change":"2023-11-06T22:18:22.868802+0000","last_active":"2023-11-06T22:18:22.873183+0000","last_peered":"2023-11-06T22:18:22.873183+0000","last_clean":"2023-11-06T22:18:22.873183+0000","last_became_active":"2023-11-06T22:18:22.868499+0000","last_became_peered":"2023-11-06T22:18:22.868499+0000","last_unstale":"2023-11-06T22:18:22.873183+0000","last_undegraded":"2023-11-06T22:18:22.873183+0000","last_fullsized":"2023-11-06T22:18:22.873183+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.869561+0000","last_change":"2023-11-06T22:18:22.367410+0000","last_active":"2023-11-06T22:18:22.869561+0000","last_peered":"2023-11-06T22:18:22.869561+0000","last_clean":"2023-11-06T22:18:22.869561+0000","last_became_active":"2023-11-06T22:18:22.366809+0000","last_became_peered":"2023-11-06T22:18:22.366809+0000","last_unstale":"2023-11-06T22:18:22.869561+0000","last_undegraded":"2023-11-06T22:18:22.869561+0000","last_fullsized":"2023-11-06T22:18:22.869561+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.880506+0000","last_change":"2023-11-06T22:18:22.878761+0000","last_active":"2023-11-06T22:18:22.880506+0000","last_peered":"2023-11-06T22:18:22.880506+0000","last_clean":"2023-11-06T22:18:22.880506+0000","last_became_active":"2023-11-06T22:18:22.877180+0000","last_became_peered":"2023-11-06T22:18:22.877180+0000","last_unstale":"2023-11-06T22:18:22.880506+0000","last_undegraded":"2023-11-06T22:18:22.880506+0000","last_fullsized":"2023-11-06T22:18:22.880506+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.850901+0000","last_change":"2023-11-06T22:18:22.372123+0000","last_active":"2023-11-06T22:18:22.850901+0000","last_peered":"2023-11-06T22:18:22.850901+0000","last_clean":"2023-11-06T22:18:22.850901+0000","last_became_active":"2023-11-06T22:18:22.371777+0000","last_became_peered":"2023-11-06T22:18:22.371777+0000","last_unstale":"2023-11-06T22:18:22.850901+0000","last_undegraded":"2023-11-06T22:18:22.850901+0000","last_fullsized":"2023-11-06T22:18:22.850901+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866100+0000","last_change":"2023-11-06T22:18:22.866100+0000","last_active":"2023-11-06T22:18:22.866100+0000","last_peered":"2023-11-06T22:18:22.866100+0000","last_clean":"2023-11-06T22:18:22.866100+0000","last_became_active":"2023-11-06T22:18:22.865808+0000","last_became_peered":"2023-11-06T22:18:22.865808+0000","last_unstale":"2023-11-06T22:18:22.866100+0000","last_undegraded":"2023-11-06T22:18:22.866100+0000","last_fullsized":"2023-11-06T22:18:22.866100+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.859921+0000","last_change":"2023-11-06T22:18:22.378361+0000","last_active":"2023-11-06T22:18:22.859921+0000","last_peered":"2023-11-06T22:18:22.859921+0000","last_clean":"2023-11-06T22:18:22.859921+0000","last_became_active":"2023-11-06T22:18:22.377935+0000","last_became_peered":"2023-11-06T22:18:22.377935+0000","last_unstale":"2023-11-06T22:18:22.859921+0000","last_undegraded":"2023-11-06T22:18:22.859921+0000","last_fullsized":"2023-11-06T22:18:22.859921+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878627+0000","last_change":"2023-11-06T22:18:22.876877+0000","last_active":"2023-11-06T22:18:22.878627+0000","last_peered":"2023-11-06T22:18:22.878627+0000","last_clean":"2023-11-06T22:18:22.878627+0000","last_became_active":"2023-11-06T22:18:22.875822+0000","last_became_peered":"2023-11-06T22:18:22.875822+0000","last_unstale":"2023-11-06T22:18:22.878627+0000","last_undegraded":"2023-11-06T22:18:22.878627+0000","last_fullsized":"2023-11-06T22:18:22.878627+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.383282+0000","last_change":"2023-11-06T22:18:22.378945+0000","last_active":"2023-11-06T22:18:22.383282+0000","last_peered":"2023-11-06T22:18:22.383282+0000","last_clean":"2023-11-06T22:18:22.383282+0000","last_became_active":"2023-11-06T22:18:22.378630+0000","last_became_peered":"2023-11-06T22:18:22.378630+0000","last_unstale":"2023-11-06T22:18:22.383282+0000","last_undegraded":"2023-11-06T22:18:22.383282+0000","last_fullsized":"2023-11-06T22:18:22.383282+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871291+0000","last_change":"2023-11-06T22:18:22.866193+0000","last_active":"2023-11-06T22:18:22.871291+0000","last_peered":"2023-11-06T22:18:22.871291+0000","last_clean":"2023-11-06T22:18:22.871291+0000","last_became_active":"2023-11-06T22:18:22.865914+0000","last_became_peered":"2023-11-06T22:18:22.865914+0000","last_unstale":"2023-11-06T22:18:22.871291+0000","last_undegraded":"2023-11-06T22:18:22.871291+0000","last_fullsized":"2023-11-06T22:18:22.871291+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.847695+0000","last_change":"2023-11-06T22:18:22.371924+0000","last_active":"2023-11-06T22:18:22.847695+0000","last_peered":"2023-11-06T22:18:22.847695+0000","last_clean":"2023-11-06T22:18:22.847695+0000","last_became_active":"2023-11-06T22:18:22.370790+0000","last_became_peered":"2023-11-06T22:18:22.370790+0000","last_unstale":"2023-11-06T22:18:22.847695+0000","last_undegraded":"2023-11-06T22:18:22.847695+0000","last_fullsized":"2023-11-06T22:18:22.847695+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.864039+0000","last_change":"2023-11-06T22:18:22.864039+0000","last_active":"2023-11-06T22:18:22.864039+0000","last_peered":"2023-11-06T22:18:22.864039+0000","last_clean":"2023-11-06T22:18:22.864039+0000","last_became_active":"2023-11-06T22:18:22.863551+0000","last_became_peered":"2023-11-06T22:18:22.863551+0000","last_unstale":"2023-11-06T22:18:22.864039+0000","last_undegraded":"2023-11-06T22:18:22.864039+0000","last_fullsized":"2023-11-06T22:18:22.864039+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.378713+0000","last_change":"2023-11-06T22:18:22.374807+0000","last_active":"2023-11-06T22:18:22.378713+0000","last_peered":"2023-11-06T22:18:22.378713+0000","last_clean":"2023-11-06T22:18:22.378713+0000","last_became_active":"2023-11-06T22:18:22.373936+0000","last_became_peered":"2023-11-06T22:18:22.373936+0000","last_unstale":"2023-11-06T22:18:22.378713+0000","last_undegraded":"2023-11-06T22:18:22.378713+0000","last_fullsized":"2023-11-06T22:18:22.378713+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868783+0000","last_change":"2023-11-06T22:18:22.867080+0000","last_active":"2023-11-06T22:18:22.868783+0000","last_peered":"2023-11-06T22:18:22.868783+0000","last_clean":"2023-11-06T22:18:22.868783+0000","last_became_active":"2023-11-06T22:18:22.866775+0000","last_became_peered":"2023-11-06T22:18:22.866775+0000","last_unstale":"2023-11-06T22:18:22.868783+0000","last_undegraded":"2023-11-06T22:18:22.868783+0000","last_fullsized":"2023-11-06T22:18:22.868783+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.851706+0000","last_change":"2023-11-06T22:18:22.377422+0000","last_active":"2023-11-06T22:18:22.851706+0000","last_peered":"2023-11-06T22:18:22.851706+0000","last_clean":"2023-11-06T22:18:22.851706+0000","last_became_active":"2023-11-06T22:18:22.376164+0000","last_became_peered":"2023-11-06T22:18:22.376164+0000","last_unstale":"2023-11-06T22:18:22.851706+0000","last_undegraded":"2023-11-06T22:18:22.851706+0000","last_fullsized":"2023-11-06T22:18:22.851706+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.872115+0000","last_change":"2023-11-06T22:18:22.861982+0000","last_active":"2023-11-06T22:18:22.872115+0000","last_peered":"2023-11-06T22:18:22.872115+0000","last_clean":"2023-11-06T22:18:22.872115+0000","last_became_active":"2023-11-06T22:18:22.861368+0000","last_became_peered":"2023-11-06T22:18:22.861368+0000","last_unstale":"2023-11-06T22:18:22.872115+0000","last_undegraded":"2023-11-06T22:18:22.872115+0000","last_fullsized":"2023-11-06T22:18:22.872115+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.845637+0000","last_change":"2023-11-06T22:18:22.373932+0000","last_active":"2023-11-06T22:18:22.845637+0000","last_peered":"2023-11-06T22:18:22.845637+0000","last_clean":"2023-11-06T22:18:22.845637+0000","last_became_active":"2023-11-06T22:18:22.373620+0000","last_became_peered":"2023-11-06T22:18:22.373620+0000","last_unstale":"2023-11-06T22:18:22.845637+0000","last_undegraded":"2023-11-06T22:18:22.845637+0000","last_fullsized":"2023-11-06T22:18:22.845637+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.875315+0000","last_change":"2023-11-06T22:18:22.873133+0000","last_active":"2023-11-06T22:18:22.875315+0000","last_peered":"2023-11-06T22:18:22.875315+0000","last_clean":"2023-11-06T22:18:22.875315+0000","last_became_active":"2023-11-06T22:18:22.872788+0000","last_became_peered":"2023-11-06T22:18:22.872788+0000","last_unstale":"2023-11-06T22:18:22.875315+0000","last_undegraded":"2023-11-06T22:18:22.875315+0000","last_fullsized":"2023-11-06T22:18:22.875315+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.847946+0000","last_change":"2023-11-06T22:18:22.371486+0000","last_active":"2023-11-06T22:18:22.847946+0000","last_peered":"2023-11-06T22:18:22.847946+0000","last_clean":"2023-11-06T22:18:22.847946+0000","last_became_active":"2023-11-06T22:18:22.371115+0000","last_became_peered":"2023-11-06T22:18:22.371115+0000","last_unstale":"2023-11-06T22:18:22.847946+0000","last_undegraded":"2023-11-06T22:18:22.847946+0000","last_fullsized":"2023-11-06T22:18:22.847946+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.874458+0000","last_change":"2023-11-06T22:18:22.871864+0000","last_active":"2023-11-06T22:18:22.874458+0000","last_peered":"2023-11-06T22:18:22.874458+0000","last_clean":"2023-11-06T22:18:22.874458+0000","last_became_active":"2023-11-06T22:18:22.871514+0000","last_became_peered":"2023-11-06T22:18:22.871514+0000","last_unstale":"2023-11-06T22:18:22.874458+0000","last_undegraded":"2023-11-06T22:18:22.874458+0000","last_fullsized":"2023-11-06T22:18:22.874458+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860629+0000","last_change":"2023-11-06T22:18:22.370501+0000","last_active":"2023-11-06T22:18:22.860629+0000","last_peered":"2023-11-06T22:18:22.860629+0000","last_clean":"2023-11-06T22:18:22.860629+0000","last_became_active":"2023-11-06T22:18:22.370072+0000","last_became_peered":"2023-11-06T22:18:22.370072+0000","last_unstale":"2023-11-06T22:18:22.860629+0000","last_undegraded":"2023-11-06T22:18:22.860629+0000","last_fullsized":"2023-11-06T22:18:22.860629+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871349+0000","last_change":"2023-11-06T22:18:22.866083+0000","last_active":"2023-11-06T22:18:22.871349+0000","last_peered":"2023-11-06T22:18:22.871349+0000","last_clean":"2023-11-06T22:18:22.871349+0000","last_became_active":"2023-11-06T22:18:22.865704+0000","last_became_peered":"2023-11-06T22:18:22.865704+0000","last_unstale":"2023-11-06T22:18:22.871349+0000","last_undegraded":"2023-11-06T22:18:22.871349+0000","last_fullsized":"2023-11-06T22:18:22.871349+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860430+0000","last_change":"2023-11-06T22:18:22.382428+0000","last_active":"2023-11-06T22:18:22.860430+0000","last_peered":"2023-11-06T22:18:22.860430+0000","last_clean":"2023-11-06T22:18:22.860430+0000","last_became_active":"2023-11-06T22:18:22.382140+0000","last_became_peered":"2023-11-06T22:18:22.382140+0000","last_unstale":"2023-11-06T22:18:22.860430+0000","last_undegraded":"2023-11-06T22:18:22.860430+0000","last_fullsized":"2023-11-06T22:18:22.860430+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871378+0000","last_change":"2023-11-06T22:18:22.865207+0000","last_active":"2023-11-06T22:18:22.871378+0000","last_peered":"2023-11-06T22:18:22.871378+0000","last_clean":"2023-11-06T22:18:22.871378+0000","last_became_active":"2023-11-06T22:18:22.863071+0000","last_became_peered":"2023-11-06T22:18:22.863071+0000","last_unstale":"2023-11-06T22:18:22.871378+0000","last_undegraded":"2023-11-06T22:18:22.871378+0000","last_fullsized":"2023-11-06T22:18:22.871378+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.851391+0000","last_change":"2023-11-06T22:18:22.376697+0000","last_active":"2023-11-06T22:18:22.851391+0000","last_peered":"2023-11-06T22:18:22.851391+0000","last_clean":"2023-11-06T22:18:22.851391+0000","last_became_active":"2023-11-06T22:18:22.376410+0000","last_became_peered":"2023-11-06T22:18:22.376410+0000","last_unstale":"2023-11-06T22:18:22.851391+0000","last_undegraded":"2023-11-06T22:18:22.851391+0000","last_fullsized":"2023-11-06T22:18:22.851391+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868814+0000","last_change":"2023-11-06T22:18:22.867323+0000","last_active":"2023-11-06T22:18:22.868814+0000","last_peered":"2023-11-06T22:18:22.868814+0000","last_clean":"2023-11-06T22:18:22.868814+0000","last_became_active":"2023-11-06T22:18:22.866882+0000","last_became_peered":"2023-11-06T22:18:22.866882+0000","last_unstale":"2023-11-06T22:18:22.868814+0000","last_undegraded":"2023-11-06T22:18:22.868814+0000","last_fullsized":"2023-11-06T22:18:22.868814+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.386488+0000","last_change":"2023-11-06T22:18:22.381100+0000","last_active":"2023-11-06T22:18:22.386488+0000","last_peered":"2023-11-06T22:18:22.386488+0000","last_clean":"2023-11-06T22:18:22.386488+0000","last_became_active":"2023-11-06T22:18:22.379002+0000","last_became_peered":"2023-11-06T22:18:22.379002+0000","last_unstale":"2023-11-06T22:18:22.386488+0000","last_undegraded":"2023-11-06T22:18:22.386488+0000","last_fullsized":"2023-11-06T22:18:22.386488+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.859108+0000","last_change":"2023-11-06T22:18:22.859108+0000","last_active":"2023-11-06T22:18:22.859108+0000","last_peered":"2023-11-06T22:18:22.859108+0000","last_clean":"2023-11-06T22:18:22.859108+0000","last_became_active":"2023-11-06T22:18:22.858510+0000","last_became_peered":"2023-11-06T22:18:22.858510+0000","last_unstale":"2023-11-06T22:18:22.859108+0000","last_undegraded":"2023-11-06T22:18:22.859108+0000","last_fullsized":"2023-11-06T22:18:22.859108+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"222'3","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.873629+0000","last_change":"2023-11-06T22:18:22.374165+0000","last_active":"2023-11-06T22:18:22.873629+0000","last_peered":"2023-11-06T22:18:22.873629+0000","last_clean":"2023-11-06T22:18:22.873629+0000","last_became_active":"2023-11-06T22:18:22.373881+0000","last_became_peered":"2023-11-06T22:18:22.373881+0000","last_unstale":"2023-11-06T22:18:22.873629+0000","last_undegraded":"2023-11-06T22:18:22.873629+0000","last_fullsized":"2023-11-06T22:18:22.873629+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.880528+0000","last_change":"2023-11-06T22:18:22.879042+0000","last_active":"2023-11-06T22:18:22.880528+0000","last_peered":"2023-11-06T22:18:22.880528+0000","last_clean":"2023-11-06T22:18:22.880528+0000","last_became_active":"2023-11-06T22:18:22.878492+0000","last_became_peered":"2023-11-06T22:18:22.878492+0000","last_unstale":"2023-11-06T22:18:22.880528+0000","last_undegraded":"2023-11-06T22:18:22.880528+0000","last_fullsized":"2023-11-06T22:18:22.880528+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.14","version":"222'2","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.872617+0000","last_change":"2023-11-06T22:18:22.370708+0000","last_active":"2023-11-06T22:18:22.872617+0000","last_peered":"2023-11-06T22:18:22.872617+0000","last_clean":"2023-11-06T22:18:22.872617+0000","last_became_active":"2023-11-06T22:18:22.370429+0000","last_became_peered":"2023-11-06T22:18:22.370429+0000","last_unstale":"2023-11-06T22:18:22.872617+0000","last_undegraded":"2023-11-06T22:18:22.872617+0000","last_fullsized":"2023-11-06T22:18:22.872617+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.867777+0000","last_change":"2023-11-06T22:18:22.863683+0000","last_active":"2023-11-06T22:18:22.867777+0000","last_peered":"2023-11-06T22:18:22.867777+0000","last_clean":"2023-11-06T22:18:22.867777+0000","last_became_active":"2023-11-06T22:18:22.863321+0000","last_became_peered":"2023-11-06T22:18:22.863321+0000","last_unstale":"2023-11-06T22:18:22.867777+0000","last_undegraded":"2023-11-06T22:18:22.867777+0000","last_fullsized":"2023-11-06T22:18:22.867777+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.13","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.872041+0000","last_change":"2023-11-06T22:18:22.372773+0000","last_active":"2023-11-06T22:18:22.872041+0000","last_peered":"2023-11-06T22:18:22.872041+0000","last_clean":"2023-11-06T22:18:22.872041+0000","last_became_active":"2023-11-06T22:18:22.372494+0000","last_became_peered":"2023-11-06T22:18:22.372494+0000","last_unstale":"2023-11-06T22:18:22.872041+0000","last_undegraded":"2023-11-06T22:18:22.872041+0000","last_fullsized":"2023-11-06T22:18:22.872041+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868713+0000","last_change":"2023-11-06T22:18:22.863563+0000","last_active":"2023-11-06T22:18:22.868713+0000","last_peered":"2023-11-06T22:18:22.868713+0000","last_clean":"2023-11-06T22:18:22.868713+0000","last_became_active":"2023-11-06T22:18:22.863256+0000","last_became_peered":"2023-11-06T22:18:22.863256+0000","last_unstale":"2023-11-06T22:18:22.868713+0000","last_undegraded":"2023-11-06T22:18:22.868713+0000","last_fullsized":"2023-11-06T22:18:22.868713+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.850088+0000","last_change":"2023-11-06T22:18:22.372728+0000","last_active":"2023-11-06T22:18:22.850088+0000","last_peered":"2023-11-06T22:18:22.850088+0000","last_clean":"2023-11-06T22:18:22.850088+0000","last_became_active":"2023-11-06T22:18:22.372410+0000","last_became_peered":"2023-11-06T22:18:22.372410+0000","last_unstale":"2023-11-06T22:18:22.850088+0000","last_undegraded":"2023-11-06T22:18:22.850088+0000","last_fullsized":"2023-11-06T22:18:22.850088+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878712+0000","last_change":"2023-11-06T22:18:22.873073+0000","last_active":"2023-11-06T22:18:22.878712+0000","last_peered":"2023-11-06T22:18:22.878712+0000","last_clean":"2023-11-06T22:18:22.878712+0000","last_became_active":"2023-11-06T22:18:22.872754+0000","last_became_peered":"2023-11-06T22:18:22.872754+0000","last_unstale":"2023-11-06T22:18:22.878712+0000","last_undegraded":"2023-11-06T22:18:22.878712+0000","last_fullsized":"2023-11-06T22:18:22.878712+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860756+0000","last_change":"2023-11-06T22:18:22.373892+0000","last_active":"2023-11-06T22:18:22.860756+0000","last_peered":"2023-11-06T22:18:22.860756+0000","last_clean":"2023-11-06T22:18:22.860756+0000","last_became_active":"2023-11-06T22:18:22.373449+0000","last_became_peered":"2023-11-06T22:18:22.373449+0000","last_unstale":"2023-11-06T22:18:22.860756+0000","last_undegraded":"2023-11-06T22:18:22.860756+0000","last_fullsized":"2023-11-06T22:18:22.860756+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871381+0000","last_change":"2023-11-06T22:18:22.866172+0000","last_active":"2023-11-06T22:18:22.871381+0000","last_peered":"2023-11-06T22:18:22.871381+0000","last_clean":"2023-11-06T22:18:22.871381+0000","last_became_active":"2023-11-06T22:18:22.865814+0000","last_became_peered":"2023-11-06T22:18:22.865814+0000","last_unstale":"2023-11-06T22:18:22.871381+0000","last_undegraded":"2023-11-06T22:18:22.871381+0000","last_fullsized":"2023-11-06T22:18:22.871381+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"222'4","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866319+0000","last_change":"2023-11-06T22:18:22.383033+0000","last_active":"2023-11-06T22:18:22.866319+0000","last_peered":"2023-11-06T22:18:22.866319+0000","last_clean":"2023-11-06T22:18:22.866319+0000","last_became_active":"2023-11-06T22:18:22.382689+0000","last_became_peered":"2023-11-06T22:18:22.382689+0000","last_unstale":"2023-11-06T22:18:22.866319+0000","last_undegraded":"2023-11-06T22:18:22.866319+0000","last_fullsized":"2023-11-06T22:18:22.866319+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862873+0000","last_change":"2023-11-06T22:18:22.862873+0000","last_active":"2023-11-06T22:18:22.862873+0000","last_peered":"2023-11-06T22:18:22.862873+0000","last_clean":"2023-11-06T22:18:22.862873+0000","last_became_active":"2023-11-06T22:18:22.862537+0000","last_became_peered":"2023-11-06T22:18:22.862537+0000","last_unstale":"2023-11-06T22:18:22.862873+0000","last_undegraded":"2023-11-06T22:18:22.862873+0000","last_fullsized":"2023-11-06T22:18:22.862873+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.f","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871524+0000","last_change":"2023-11-06T22:18:22.373469+0000","last_active":"2023-11-06T22:18:22.871524+0000","last_peered":"2023-11-06T22:18:22.871524+0000","last_clean":"2023-11-06T22:18:22.871524+0000","last_became_active":"2023-11-06T22:18:22.373033+0000","last_became_peered":"2023-11-06T22:18:22.373033+0000","last_unstale":"2023-11-06T22:18:22.871524+0000","last_undegraded":"2023-11-06T22:18:22.871524+0000","last_fullsized":"2023-11-06T22:18:22.871524+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.875336+0000","last_change":"2023-11-06T22:18:22.867949+0000","last_active":"2023-11-06T22:18:22.875336+0000","last_peered":"2023-11-06T22:18:22.875336+0000","last_clean":"2023-11-06T22:18:22.875336+0000","last_became_active":"2023-11-06T22:18:22.867650+0000","last_became_peered":"2023-11-06T22:18:22.867650+0000","last_unstale":"2023-11-06T22:18:22.875336+0000","last_undegraded":"2023-11-06T22:18:22.875336+0000","last_fullsized":"2023-11-06T22:18:22.875336+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.e","version":"222'2","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860994+0000","last_change":"2023-11-06T22:18:22.374697+0000","last_active":"2023-11-06T22:18:22.860994+0000","last_peered":"2023-11-06T22:18:22.860994+0000","last_clean":"2023-11-06T22:18:22.860994+0000","last_became_active":"2023-11-06T22:18:22.374438+0000","last_became_peered":"2023-11-06T22:18:22.374438+0000","last_unstale":"2023-11-06T22:18:22.860994+0000","last_undegraded":"2023-11-06T22:18:22.860994+0000","last_fullsized":"2023-11-06T22:18:22.860994+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.872221+0000","last_change":"2023-11-06T22:18:22.867285+0000","last_active":"2023-11-06T22:18:22.872221+0000","last_peered":"2023-11-06T22:18:22.872221+0000","last_clean":"2023-11-06T22:18:22.872221+0000","last_became_active":"2023-11-06T22:18:22.867031+0000","last_became_peered":"2023-11-06T22:18:22.867031+0000","last_unstale":"2023-11-06T22:18:22.872221+0000","last_undegraded":"2023-11-06T22:18:22.872221+0000","last_fullsized":"2023-11-06T22:18:22.872221+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.845737+0000","last_change":"2023-11-06T22:18:22.371970+0000","last_active":"2023-11-06T22:18:22.845737+0000","last_peered":"2023-11-06T22:18:22.845737+0000","last_clean":"2023-11-06T22:18:22.845737+0000","last_became_active":"2023-11-06T22:18:22.371625+0000","last_became_peered":"2023-11-06T22:18:22.371625+0000","last_unstale":"2023-11-06T22:18:22.845737+0000","last_undegraded":"2023-11-06T22:18:22.845737+0000","last_fullsized":"2023-11-06T22:18:22.845737+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.874007+0000","last_change":"2023-11-06T22:18:22.874007+0000","last_active":"2023-11-06T22:18:22.874007+0000","last_peered":"2023-11-06T22:18:22.874007+0000","last_clean":"2023-11-06T22:18:22.874007+0000","last_became_active":"2023-11-06T22:18:22.873709+0000","last_became_peered":"2023-11-06T22:18:22.873709+0000","last_unstale":"2023-11-06T22:18:22.874007+0000","last_undegraded":"2023-11-06T22:18:22.874007+0000","last_fullsized":"2023-11-06T22:18:22.874007+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.381620+0000","last_change":"2023-11-06T22:18:22.376402+0000","last_active":"2023-11-06T22:18:22.381620+0000","last_peered":"2023-11-06T22:18:22.381620+0000","last_clean":"2023-11-06T22:18:22.381620+0000","last_became_active":"2023-11-06T22:18:22.375761+0000","last_became_peered":"2023-11-06T22:18:22.375761+0000","last_unstale":"2023-11-06T22:18:22.381620+0000","last_undegraded":"2023-11-06T22:18:22.381620+0000","last_fullsized":"2023-11-06T22:18:22.381620+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.870465+0000","last_change":"2023-11-06T22:18:22.867925+0000","last_active":"2023-11-06T22:18:22.870465+0000","last_peered":"2023-11-06T22:18:22.870465+0000","last_clean":"2023-11-06T22:18:22.870465+0000","last_became_active":"2023-11-06T22:18:22.866848+0000","last_became_peered":"2023-11-06T22:18:22.866848+0000","last_unstale":"2023-11-06T22:18:22.870465+0000","last_undegraded":"2023-11-06T22:18:22.870465+0000","last_fullsized":"2023-11-06T22:18:22.870465+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.b","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.870137+0000","last_change":"2023-11-06T22:18:22.377476+0000","last_active":"2023-11-06T22:18:22.870137+0000","last_peered":"2023-11-06T22:18:22.870137+0000","last_clean":"2023-11-06T22:18:22.870137+0000","last_became_active":"2023-11-06T22:18:22.376175+0000","last_became_peered":"2023-11-06T22:18:22.376175+0000","last_unstale":"2023-11-06T22:18:22.870137+0000","last_undegraded":"2023-11-06T22:18:22.870137+0000","last_fullsized":"2023-11-06T22:18:22.870137+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.879191+0000","last_change":"2023-11-06T22:18:22.875569+0000","last_active":"2023-11-06T22:18:22.879191+0000","last_peered":"2023-11-06T22:18:22.879191+0000","last_clean":"2023-11-06T22:18:22.879191+0000","last_became_active":"2023-11-06T22:18:22.875145+0000","last_became_peered":"2023-11-06T22:18:22.875145+0000","last_unstale":"2023-11-06T22:18:22.879191+0000","last_undegraded":"2023-11-06T22:18:22.879191+0000","last_fullsized":"2023-11-06T22:18:22.879191+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.855701+0000","last_change":"2023-11-06T22:18:22.372366+0000","last_active":"2023-11-06T22:18:22.855701+0000","last_peered":"2023-11-06T22:18:22.855701+0000","last_clean":"2023-11-06T22:18:22.855701+0000","last_became_active":"2023-11-06T22:18:22.372076+0000","last_became_peered":"2023-11-06T22:18:22.372076+0000","last_unstale":"2023-11-06T22:18:22.855701+0000","last_undegraded":"2023-11-06T22:18:22.855701+0000","last_fullsized":"2023-11-06T22:18:22.855701+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878513+0000","last_change":"2023-11-06T22:18:22.876274+0000","last_active":"2023-11-06T22:18:22.878513+0000","last_peered":"2023-11-06T22:18:22.878513+0000","last_clean":"2023-11-06T22:18:22.878513+0000","last_became_active":"2023-11-06T22:18:22.875579+0000","last_became_peered":"2023-11-06T22:18:22.875579+0000","last_unstale":"2023-11-06T22:18:22.878513+0000","last_undegraded":"2023-11-06T22:18:22.878513+0000","last_fullsized":"2023-11-06T22:18:22.878513+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.852036+0000","last_change":"2023-11-06T22:18:22.372478+0000","last_active":"2023-11-06T22:18:22.852036+0000","last_peered":"2023-11-06T22:18:22.852036+0000","last_clean":"2023-11-06T22:18:22.852036+0000","last_became_active":"2023-11-06T22:18:22.372198+0000","last_became_peered":"2023-11-06T22:18:22.372198+0000","last_unstale":"2023-11-06T22:18:22.852036+0000","last_undegraded":"2023-11-06T22:18:22.852036+0000","last_fullsized":"2023-11-06T22:18:22.852036+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.873335+0000","last_change":"2023-11-06T22:18:22.864459+0000","last_active":"2023-11-06T22:18:22.873335+0000","last_peered":"2023-11-06T22:18:22.873335+0000","last_clean":"2023-11-06T22:18:22.873335+0000","last_became_active":"2023-11-06T22:18:22.863876+0000","last_became_peered":"2023-11-06T22:18:22.863876+0000","last_unstale":"2023-11-06T22:18:22.873335+0000","last_undegraded":"2023-11-06T22:18:22.873335+0000","last_fullsized":"2023-11-06T22:18:22.873335+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840609+0000","last_change":"2023-11-06T22:18:22.369598+0000","last_active":"2023-11-06T22:18:22.840609+0000","last_peered":"2023-11-06T22:18:22.840609+0000","last_clean":"2023-11-06T22:18:22.840609+0000","last_became_active":"2023-11-06T22:18:22.367779+0000","last_became_peered":"2023-11-06T22:18:22.367779+0000","last_unstale":"2023-11-06T22:18:22.840609+0000","last_undegraded":"2023-11-06T22:18:22.840609+0000","last_fullsized":"2023-11-06T22:18:22.840609+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.849570+0000","last_change":"2023-11-06T22:18:22.849570+0000","last_active":"2023-11-06T22:18:22.849570+0000","last_peered":"2023-11-06T22:18:22.849570+0000","last_clean":"2023-11-06T22:18:22.849570+0000","last_became_active":"2023-11-06T22:18:22.849277+0000","last_became_peered":"2023-11-06T22:18:22.849277+0000","last_unstale":"2023-11-06T22:18:22.849570+0000","last_undegraded":"2023-11-06T22:18:22.849570+0000","last_fullsized":"2023-11-06T22:18:22.849570+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.7","version":"222'2","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868972+0000","last_change":"2023-11-06T22:18:22.370733+0000","last_active":"2023-11-06T22:18:22.868972+0000","last_peered":"2023-11-06T22:18:22.868972+0000","last_clean":"2023-11-06T22:18:22.868972+0000","last_became_active":"2023-11-06T22:18:22.367912+0000","last_became_peered":"2023-11-06T22:18:22.367912+0000","last_unstale":"2023-11-06T22:18:22.868972+0000","last_undegraded":"2023-11-06T22:18:22.868972+0000","last_fullsized":"2023-11-06T22:18:22.868972+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871660+0000","last_change":"2023-11-06T22:18:22.871660+0000","last_active":"2023-11-06T22:18:22.871660+0000","last_peered":"2023-11-06T22:18:22.871660+0000","last_clean":"2023-11-06T22:18:22.871660+0000","last_became_active":"2023-11-06T22:18:22.871386+0000","last_became_peered":"2023-11-06T22:18:22.871386+0000","last_unstale":"2023-11-06T22:18:22.871660+0000","last_undegraded":"2023-11-06T22:18:22.871660+0000","last_fullsized":"2023-11-06T22:18:22.871660+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.386507+0000","last_change":"2023-11-06T22:18:22.381263+0000","last_active":"2023-11-06T22:18:22.386507+0000","last_peered":"2023-11-06T22:18:22.386507+0000","last_clean":"2023-11-06T22:18:22.386507+0000","last_became_active":"2023-11-06T22:18:22.379493+0000","last_became_peered":"2023-11-06T22:18:22.379493+0000","last_unstale":"2023-11-06T22:18:22.386507+0000","last_undegraded":"2023-11-06T22:18:22.386507+0000","last_fullsized":"2023-11-06T22:18:22.386507+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871301+0000","last_change":"2023-11-06T22:18:22.867675+0000","last_active":"2023-11-06T22:18:22.871301+0000","last_peered":"2023-11-06T22:18:22.871301+0000","last_clean":"2023-11-06T22:18:22.871301+0000","last_became_active":"2023-11-06T22:18:22.867370+0000","last_became_peered":"2023-11-06T22:18:22.867370+0000","last_unstale":"2023-11-06T22:18:22.871301+0000","last_undegraded":"2023-11-06T22:18:22.871301+0000","last_fullsized":"2023-11-06T22:18:22.871301+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862448+0000","last_change":"2023-11-06T22:18:22.380369+0000","last_active":"2023-11-06T22:18:22.862448+0000","last_peered":"2023-11-06T22:18:22.862448+0000","last_clean":"2023-11-06T22:18:22.862448+0000","last_became_active":"2023-11-06T22:18:22.380041+0000","last_became_peered":"2023-11-06T22:18:22.380041+0000","last_unstale":"2023-11-06T22:18:22.862448+0000","last_undegraded":"2023-11-06T22:18:22.862448+0000","last_fullsized":"2023-11-06T22:18:22.862448+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878418+0000","last_change":"2023-11-06T22:18:22.873725+0000","last_active":"2023-11-06T22:18:22.878418+0000","last_peered":"2023-11-06T22:18:22.878418+0000","last_clean":"2023-11-06T22:18:22.878418+0000","last_became_active":"2023-11-06T22:18:22.871619+0000","last_became_peered":"2023-11-06T22:18:22.871619+0000","last_unstale":"2023-11-06T22:18:22.878418+0000","last_undegraded":"2023-11-06T22:18:22.878418+0000","last_fullsized":"2023-11-06T22:18:22.878418+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.369329+0000","last_change":"2023-11-06T22:18:22.360996+0000","last_active":"2023-11-06T22:18:22.369329+0000","last_peered":"2023-11-06T22:18:22.369329+0000","last_clean":"2023-11-06T22:18:22.369329+0000","last_became_active":"2023-11-06T22:18:22.359436+0000","last_became_peered":"2023-11-06T22:18:22.359436+0000","last_unstale":"2023-11-06T22:18:22.369329+0000","last_undegraded":"2023-11-06T22:18:22.369329+0000","last_fullsized":"2023-11-06T22:18:22.369329+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.864911+0000","last_change":"2023-11-06T22:18:22.864911+0000","last_active":"2023-11-06T22:18:22.864911+0000","last_peered":"2023-11-06T22:18:22.864911+0000","last_clean":"2023-11-06T22:18:22.864911+0000","last_became_active":"2023-11-06T22:18:22.864056+0000","last_became_peered":"2023-11-06T22:18:22.864056+0000","last_unstale":"2023-11-06T22:18:22.864911+0000","last_undegraded":"2023-11-06T22:18:22.864911+0000","last_fullsized":"2023-11-06T22:18:22.864911+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.842005+0000","last_change":"2023-11-06T22:18:22.361864+0000","last_active":"2023-11-06T22:18:22.842005+0000","last_peered":"2023-11-06T22:18:22.842005+0000","last_clean":"2023-11-06T22:18:22.842005+0000","last_became_active":"2023-11-06T22:18:22.361581+0000","last_became_peered":"2023-11-06T22:18:22.361581+0000","last_unstale":"2023-11-06T22:18:22.842005+0000","last_undegraded":"2023-11-06T22:18:22.842005+0000","last_fullsized":"2023-11-06T22:18:22.842005+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862008+0000","last_change":"2023-11-06T22:18:22.856002+0000","last_active":"2023-11-06T22:18:22.862008+0000","last_peered":"2023-11-06T22:18:22.862008+0000","last_clean":"2023-11-06T22:18:22.862008+0000","last_became_active":"2023-11-06T22:18:22.855710+0000","last_became_peered":"2023-11-06T22:18:22.855710+0000","last_unstale":"2023-11-06T22:18:22.862008+0000","last_undegraded":"2023-11-06T22:18:22.862008+0000","last_fullsized":"2023-11-06T22:18:22.862008+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.382450+0000","last_change":"2023-11-06T22:18:22.375257+0000","last_active":"2023-11-06T22:18:22.382450+0000","last_peered":"2023-11-06T22:18:22.382450+0000","last_clean":"2023-11-06T22:18:22.382450+0000","last_became_active":"2023-11-06T22:18:22.374635+0000","last_became_peered":"2023-11-06T22:18:22.374635+0000","last_unstale":"2023-11-06T22:18:22.382450+0000","last_undegraded":"2023-11-06T22:18:22.382450+0000","last_fullsized":"2023-11-06T22:18:22.382450+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.880301+0000","last_change":"2023-11-06T22:18:22.878630+0000","last_active":"2023-11-06T22:18:22.880301+0000","last_peered":"2023-11-06T22:18:22.880301+0000","last_clean":"2023-11-06T22:18:22.880301+0000","last_became_active":"2023-11-06T22:18:22.878087+0000","last_became_peered":"2023-11-06T22:18:22.878087+0000","last_unstale":"2023-11-06T22:18:22.880301+0000","last_undegraded":"2023-11-06T22:18:22.880301+0000","last_fullsized":"2023-11-06T22:18:22.880301+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860786+0000","last_change":"2023-11-06T22:18:22.367305+0000","last_active":"2023-11-06T22:18:22.860786+0000","last_peered":"2023-11-06T22:18:22.860786+0000","last_clean":"2023-11-06T22:18:22.860786+0000","last_became_active":"2023-11-06T22:18:22.367027+0000","last_became_peered":"2023-11-06T22:18:22.367027+0000","last_unstale":"2023-11-06T22:18:22.860786+0000","last_undegraded":"2023-11-06T22:18:22.860786+0000","last_fullsized":"2023-11-06T22:18:22.860786+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.864753+0000","last_change":"2023-11-06T22:18:22.864753+0000","last_active":"2023-11-06T22:18:22.864753+0000","last_peered":"2023-11-06T22:18:22.864753+0000","last_clean":"2023-11-06T22:18:22.864753+0000","last_became_active":"2023-11-06T22:18:22.863702+0000","last_became_peered":"2023-11-06T22:18:22.863702+0000","last_unstale":"2023-11-06T22:18:22.864753+0000","last_undegraded":"2023-11-06T22:18:22.864753+0000","last_fullsized":"2023-11-06T22:18:22.864753+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.383386+0000","last_change":"2023-11-06T22:18:22.378850+0000","last_active":"2023-11-06T22:18:22.383386+0000","last_peered":"2023-11-06T22:18:22.383386+0000","last_clean":"2023-11-06T22:18:22.383386+0000","last_became_active":"2023-11-06T22:18:22.378453+0000","last_became_peered":"2023-11-06T22:18:22.378453+0000","last_unstale":"2023-11-06T22:18:22.383386+0000","last_undegraded":"2023-11-06T22:18:22.383386+0000","last_fullsized":"2023-11-06T22:18:22.383386+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862302+0000","last_change":"2023-11-06T22:18:22.862302+0000","last_active":"2023-11-06T22:18:22.862302+0000","last_peered":"2023-11-06T22:18:22.862302+0000","last_clean":"2023-11-06T22:18:22.862302+0000","last_became_active":"2023-11-06T22:18:22.861817+0000","last_became_peered":"2023-11-06T22:18:22.861817+0000","last_unstale":"2023-11-06T22:18:22.862302+0000","last_undegraded":"2023-11-06T22:18:22.862302+0000","last_fullsized":"2023-11-06T22:18:22.862302+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1f","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.875842+0000","last_change":"2023-11-06T22:18:22.373358+0000","last_active":"2023-11-06T22:18:22.875842+0000","last_peered":"2023-11-06T22:18:22.875842+0000","last_clean":"2023-11-06T22:18:22.875842+0000","last_became_active":"2023-11-06T22:18:22.372926+0000","last_became_peered":"2023-11-06T22:18:22.372926+0000","last_unstale":"2023-11-06T22:18:22.875842+0000","last_undegraded":"2023-11-06T22:18:22.875842+0000","last_fullsized":"2023-11-06T22:18:22.875842+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":188,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.349951+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T22:18:22.349951+0000","last_peered":"2023-11-06T22:18:22.349951+0000","last_clean":"2023-11-06T22:18:22.349951+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:18:22.349951+0000","last_undegraded":"2023-11-06T22:18:22.349951+0000","last_fullsized":"2023-11-06T22:18:22.349951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.858259+0000","last_change":"2023-11-06T22:18:22.858259+0000","last_active":"2023-11-06T22:18:22.858259+0000","last_peered":"2023-11-06T22:18:22.858259+0000","last_clean":"2023-11-06T22:18:22.858259+0000","last_became_active":"2023-11-06T22:18:22.857808+0000","last_became_peered":"2023-11-06T22:18:22.857808+0000","last_unstale":"2023-11-06T22:18:22.858259+0000","last_undegraded":"2023-11-06T22:18:22.858259+0000","last_fullsized":"2023-11-06T22:18:22.858259+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.841054+0000","last_change":"2023-11-06T22:18:22.367265+0000","last_active":"2023-11-06T22:18:22.841054+0000","last_peered":"2023-11-06T22:18:22.841054+0000","last_clean":"2023-11-06T22:18:22.841054+0000","last_became_active":"2023-11-06T22:18:22.366914+0000","last_became_peered":"2023-11-06T22:18:22.366914+0000","last_unstale":"2023-11-06T22:18:22.841054+0000","last_undegraded":"2023-11-06T22:18:22.841054+0000","last_fullsized":"2023-11-06T22:18:22.841054+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":202,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.841522+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T22:18:22.841522+0000","last_peered":"2023-11-06T22:18:22.841522+0000","last_clean":"2023-11-06T22:18:22.841522+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:18:22.841522+0000","last_undegraded":"2023-11-06T22:18:22.841522+0000","last_fullsized":"2023-11-06T22:18:22.841522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.876152+0000","last_change":"2023-11-06T22:18:22.870963+0000","last_active":"2023-11-06T22:18:22.876152+0000","last_peered":"2023-11-06T22:18:22.876152+0000","last_clean":"2023-11-06T22:18:22.876152+0000","last_became_active":"2023-11-06T22:18:22.870685+0000","last_became_peered":"2023-11-06T22:18:22.870685+0000","last_unstale":"2023-11-06T22:18:22.876152+0000","last_undegraded":"2023-11-06T22:18:22.876152+0000","last_fullsized":"2023-11-06T22:18:22.876152+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.855462+0000","last_change":"2023-11-06T22:18:22.367095+0000","last_active":"2023-11-06T22:18:22.855462+0000","last_peered":"2023-11-06T22:18:22.855462+0000","last_clean":"2023-11-06T22:18:22.855462+0000","last_became_active":"2023-11-06T22:18:22.366804+0000","last_became_peered":"2023-11-06T22:18:22.366804+0000","last_unstale":"2023-11-06T22:18:22.855462+0000","last_undegraded":"2023-11-06T22:18:22.855462+0000","last_fullsized":"2023-11-06T22:18:22.855462+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":211,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860847+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T22:18:22.860847+0000","last_peered":"2023-11-06T22:18:22.860847+0000","last_clean":"2023-11-06T22:18:22.860847+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:18:22.860847+0000","last_undegraded":"2023-11-06T22:18:22.860847+0000","last_fullsized":"2023-11-06T22:18:22.860847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866442+0000","last_change":"2023-11-06T22:18:22.863959+0000","last_active":"2023-11-06T22:18:22.866442+0000","last_peered":"2023-11-06T22:18:22.866442+0000","last_clean":"2023-11-06T22:18:22.866442+0000","last_became_active":"2023-11-06T22:18:22.863660+0000","last_became_peered":"2023-11-06T22:18:22.863660+0000","last_unstale":"2023-11-06T22:18:22.866442+0000","last_undegraded":"2023-11-06T22:18:22.866442+0000","last_fullsized":"2023-11-06T22:18:22.866442+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.857909+0000","last_change":"2023-11-06T22:18:22.370944+0000","last_active":"2023-11-06T22:18:22.857909+0000","last_peered":"2023-11-06T22:18:22.857909+0000","last_clean":"2023-11-06T22:18:22.857909+0000","last_became_active":"2023-11-06T22:18:22.370561+0000","last_became_peered":"2023-11-06T22:18:22.370561+0000","last_unstale":"2023-11-06T22:18:22.857909+0000","last_undegraded":"2023-11-06T22:18:22.857909+0000","last_fullsized":"2023-11-06T22:18:22.857909+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":202,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.842365+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T22:18:22.842365+0000","last_peered":"2023-11-06T22:18:22.842365+0000","last_clean":"2023-11-06T22:18:22.842365+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:18:22.842365+0000","last_undegraded":"2023-11-06T22:18:22.842365+0000","last_fullsized":"2023-11-06T22:18:22.842365+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":187,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.360248+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T22:18:22.360248+0000","last_peered":"2023-11-06T22:18:22.360248+0000","last_clean":"2023-11-06T22:18:22.360248+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:18:22.360248+0000","last_undegraded":"2023-11-06T22:18:22.360248+0000","last_fullsized":"2023-11-06T22:18:22.360248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.864704+0000","last_change":"2023-11-06T22:18:22.864704+0000","last_active":"2023-11-06T22:18:22.864704+0000","last_peered":"2023-11-06T22:18:22.864704+0000","last_clean":"2023-11-06T22:18:22.864704+0000","last_became_active":"2023-11-06T22:18:22.864436+0000","last_became_peered":"2023-11-06T22:18:22.864436+0000","last_unstale":"2023-11-06T22:18:22.864704+0000","last_undegraded":"2023-11-06T22:18:22.864704+0000","last_fullsized":"2023-11-06T22:18:22.864704+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862433+0000","last_change":"2023-11-06T22:18:22.380772+0000","last_active":"2023-11-06T22:18:22.862433+0000","last_peered":"2023-11-06T22:18:22.862433+0000","last_clean":"2023-11-06T22:18:22.862433+0000","last_became_active":"2023-11-06T22:18:22.380172+0000","last_became_peered":"2023-11-06T22:18:22.380172+0000","last_unstale":"2023-11-06T22:18:22.862433+0000","last_undegraded":"2023-11-06T22:18:22.862433+0000","last_fullsized":"2023-11-06T22:18:22.862433+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":202,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840691+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T22:18:22.840691+0000","last_peered":"2023-11-06T22:18:22.840691+0000","last_clean":"2023-11-06T22:18:22.840691+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:18:22.840691+0000","last_undegraded":"2023-11-06T22:18:22.840691+0000","last_fullsized":"2023-11-06T22:18:22.840691+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878532+0000","last_change":"2023-11-06T22:18:22.876417+0000","last_active":"2023-11-06T22:18:22.878532+0000","last_peered":"2023-11-06T22:18:22.878532+0000","last_clean":"2023-11-06T22:18:22.878532+0000","last_became_active":"2023-11-06T22:18:22.875633+0000","last_became_peered":"2023-11-06T22:18:22.875633+0000","last_unstale":"2023-11-06T22:18:22.878532+0000","last_undegraded":"2023-11-06T22:18:22.878532+0000","last_fullsized":"2023-11-06T22:18:22.878532+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.877140+0000","last_change":"2023-11-06T22:18:22.365549+0000","last_active":"2023-11-06T22:18:22.877140+0000","last_peered":"2023-11-06T22:18:22.877140+0000","last_clean":"2023-11-06T22:18:22.877140+0000","last_became_active":"2023-11-06T22:18:22.365162+0000","last_became_peered":"2023-11-06T22:18:22.365162+0000","last_unstale":"2023-11-06T22:18:22.877140+0000","last_undegraded":"2023-11-06T22:18:22.877140+0000","last_fullsized":"2023-11-06T22:18:22.877140+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":202,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840863+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:18:22.840863+0000","last_peered":"2023-11-06T22:18:22.840863+0000","last_clean":"2023-11-06T22:18:22.840863+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:18:22.840863+0000","last_undegraded":"2023-11-06T22:18:22.840863+0000","last_fullsized":"2023-11-06T22:18:22.840863+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866770+0000","last_change":"2023-11-06T22:18:22.861012+0000","last_active":"2023-11-06T22:18:22.866770+0000","last_peered":"2023-11-06T22:18:22.866770+0000","last_clean":"2023-11-06T22:18:22.866770+0000","last_became_active":"2023-11-06T22:18:22.860031+0000","last_became_peered":"2023-11-06T22:18:22.860031+0000","last_unstale":"2023-11-06T22:18:22.866770+0000","last_undegraded":"2023-11-06T22:18:22.866770+0000","last_fullsized":"2023-11-06T22:18:22.866770+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1d","version":"222'2","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.869990+0000","last_change":"2023-11-06T22:18:22.375900+0000","last_active":"2023-11-06T22:18:22.869990+0000","last_peered":"2023-11-06T22:18:22.869990+0000","last_clean":"2023-11-06T22:18:22.869990+0000","last_became_active":"2023-11-06T22:18:22.375442+0000","last_became_peered":"2023-11-06T22:18:22.375442+0000","last_unstale":"2023-11-06T22:18:22.869990+0000","last_undegraded":"2023-11-06T22:18:22.869990+0000","last_fullsized":"2023-11-06T22:18:22.869990+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":188,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.857648+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T22:18:22.857648+0000","last_peered":"2023-11-06T22:18:22.857648+0000","last_clean":"2023-11-06T22:18:22.857648+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:18:22.857648+0000","last_undegraded":"2023-11-06T22:18:22.857648+0000","last_fullsized":"2023-11-06T22:18:22.857648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862244+0000","last_change":"2023-11-06T22:18:22.857254+0000","last_active":"2023-11-06T22:18:22.862244+0000","last_peered":"2023-11-06T22:18:22.862244+0000","last_clean":"2023-11-06T22:18:22.862244+0000","last_became_active":"2023-11-06T22:18:22.856484+0000","last_became_peered":"2023-11-06T22:18:22.856484+0000","last_unstale":"2023-11-06T22:18:22.862244+0000","last_undegraded":"2023-11-06T22:18:22.862244+0000","last_fullsized":"2023-11-06T22:18:22.862244+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.383197+0000","last_change":"2023-11-06T22:18:22.379336+0000","last_active":"2023-11-06T22:18:22.383197+0000","last_peered":"2023-11-06T22:18:22.383197+0000","last_clean":"2023-11-06T22:18:22.383197+0000","last_became_active":"2023-11-06T22:18:22.379027+0000","last_became_peered":"2023-11-06T22:18:22.379027+0000","last_unstale":"2023-11-06T22:18:22.383197+0000","last_undegraded":"2023-11-06T22:18:22.383197+0000","last_fullsized":"2023-11-06T22:18:22.383197+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":202,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840398+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:18:22.840398+0000","last_peered":"2023-11-06T22:18:22.840398+0000","last_clean":"2023-11-06T22:18:22.840398+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:18:22.840398+0000","last_undegraded":"2023-11-06T22:18:22.840398+0000","last_fullsized":"2023-11-06T22:18:22.840398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575969,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308016,"kb_used_data":2728,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063824,"statfs":{"total":96636764160,"available":96321355776,"internally_reserved":0,"allocated":2793472,"data_stored":1436464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.41599999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90700000000000003,"5min":0.97199999999999998,"15min":1.0980000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.72799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93600000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.94799999999999995}]},{"osd":1,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.71899999999999997},"min":{"1min":0.58699999999999997,"5min":0.48699999999999999,"15min":0.46400000000000002},"max":{"1min":0.94899999999999995,"5min":0.98799999999999999,"15min":1.038},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72099999999999997,"15min":0.73399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.93400000000000005,"5min":0.96599999999999997,"15min":1.0680000000000001},"last":0.75700000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.502},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90200000000000002,"5min":0.996,"15min":1.0209999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.499},"max":{"1min":0.90700000000000003,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.504},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.0229999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.94399999999999995,"15min":0.97199999999999998},"last":1.052},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.47899999999999998,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.94299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69499999999999995}]},{"osd":5,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.40400000000000003},"max":{"1min":0.80600000000000005,"5min":1.008,"15min":1.0389999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70499999999999996,"15min":0.68700000000000006},"min":{"1min":0.53300000000000003,"5min":0.47099999999999997,"15min":0.432},"max":{"1min":0.88,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84399999999999997}]},{"osd":6,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.48399999999999999},"max":{"1min":0.84799999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.51500000000000001,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.069},"last":0.99299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575969,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308008,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063832,"statfs":{"total":96636764160,"available":96321363968,"internally_reserved":0,"allocated":2785280,"data_stored":1430338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68100000000000005,"15min":0.70999999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":1.0469999999999999,"15min":1.0660000000000001},"last":0.372},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":1.016},"last":0.55100000000000005}]},{"osd":2,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0780000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.046},"last":0.63600000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.873,"5min":0.94799999999999995,"15min":1.002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.53600000000000003,"5min":0.42299999999999999,"15min":0.41699999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.98299999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76900000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.67100000000000004},"min":{"1min":0.44,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83499999999999996,"5min":0.98499999999999999,"15min":0.98999999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.442},"max":{"1min":0.80000000000000004,"5min":0.996,"15min":1.056},"last":0.754}]},{"osd":7,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79800000000000004,"5min":1.012,"15min":1.0429999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0569999999999999,"15min":1.1279999999999999},"last":0.73699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575969,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308000,"kb_used_data":2712,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063840,"statfs":{"total":96636764160,"available":96321372160,"internally_reserved":0,"allocated":2777088,"data_stored":1429694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.97999999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.33800000000000002},"max":{"1min":0.66700000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.38700000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.307,"15min":0.307},"max":{"1min":0.67400000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.371},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.95699999999999996,"15min":1.0189999999999999},"last":0.53700000000000003}]},{"osd":3,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74099999999999999,"5min":0.76200000000000001,"15min":1.0509999999999999},"last":0.498},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":0.92700000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80900000000000005,"15min":0.80800000000000005},"min":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.45200000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84899999999999998,"15min":0.85199999999999998},"min":{"1min":0.68300000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.78300000000000003}]},{"osd":5,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.81599999999999995},"min":{"1min":0.58899999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.052,"5min":1.2210000000000001,"15min":1.2709999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81499999999999995,"15min":0.80200000000000005},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.55400000000000005},"max":{"1min":1.224,"5min":1.224,"15min":1.3109999999999999},"last":0.75900000000000001}]},{"osd":6,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.3149999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.84399999999999997,"15min":0.84899999999999998},"min":{"1min":0.64900000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.65200000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1459999999999999,"5min":1.21,"15min":1.21},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1970000000000001,"15min":1.2509999999999999},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308032,"kb_used_data":2744,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063808,"statfs":{"total":96636764160,"available":96321339392,"internally_reserved":0,"allocated":2809856,"data_stored":1438070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64200000000000002,"5min":0.65800000000000003,"15min":0.74199999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.72499999999999998,"5min":0.79900000000000004,"15min":0.83299999999999996},"last":0.36299999999999999}]},{"osd":2,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.32700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.61799999999999999,"5min":0.84299999999999997,"15min":0.876},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.43099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.80900000000000005,"15min":0.83899999999999997},"last":0.441}]},{"osd":3,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.48099999999999998,"5min":0.39600000000000002,"15min":0.375},"max":{"1min":0.76400000000000001,"5min":0.81799999999999995,"15min":0.95899999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.755,"5min":0.79700000000000004,"15min":0.85799999999999998},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.622},"max":{"1min":1.008,"5min":1.083,"15min":1.357},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.83799999999999997,"15min":0.84299999999999997},"min":{"1min":0.61699999999999999,"5min":0.59299999999999997,"15min":0.54800000000000004},"max":{"1min":1.0820000000000001,"5min":1.091,"15min":1.2150000000000001},"last":0.90800000000000003}]},{"osd":5,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.85599999999999998},"min":{"1min":0.63300000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.2669999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.85199999999999998,"15min":0.86799999999999999},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.133,"5min":1.1579999999999999,"15min":1.2949999999999999},"last":0.95499999999999996}]},{"osd":6,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81699999999999995,"15min":0.81999999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.52600000000000002},"max":{"1min":1.04,"5min":1.123,"15min":1.3120000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.84799999999999998,"15min":0.84899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1160000000000001,"5min":1.127,"15min":1.26},"last":0.875}]},{"osd":7,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83599999999999997,"15min":0.85699999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1000000000000001,"5min":1.1040000000000001,"15min":1.268},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.86699999999999999,"15min":0.88200000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.4330000000000001},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575969,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308012,"kb_used_data":2724,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063828,"statfs":{"total":96636764160,"available":96321359872,"internally_reserved":0,"allocated":2789376,"data_stored":1430372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.629,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70499999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.77400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.48799999999999999,"5min":0.46000000000000002,"15min":0.45100000000000001},"max":{"1min":0.79300000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47499999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.40999999999999998,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.74299999999999999,"5min":0.74399999999999999,"15min":0.78300000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.42499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.82199999999999995,"15min":0.82699999999999996},"min":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.53900000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.349},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2609999999999999,"5min":1.2669999999999999,"15min":1.339},"last":0.92600000000000005}]},{"osd":5,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81200000000000006,"15min":0.81699999999999995},"min":{"1min":0.54600000000000004,"5min":0.54300000000000004,"15min":0.48999999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.337},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84199999999999997,"15min":0.85499999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1899999999999999,"5min":1.212,"15min":1.468},"last":0.752}]},{"osd":6,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.83099999999999996,"15min":0.84899999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.54600000000000004},"max":{"1min":1.147,"5min":1.155,"15min":1.387},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.84499999999999997,"15min":0.85099999999999998},"min":{"1min":0.627,"5min":0.56499999999999995,"15min":0.56100000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.4299999999999999},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.86599999999999999,"15min":0.876},"min":{"1min":0.61199999999999999,"5min":0.58599999999999997,"15min":0.55700000000000005},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.496},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85999999999999999,"15min":0.876},"min":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1459999999999999,"5min":1.2470000000000001,"15min":1.508},"last":0.86799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308020,"kb_used_data":2732,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063820,"statfs":{"total":96636764160,"available":96321351680,"internally_reserved":0,"allocated":2797568,"data_stored":1437888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.47299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.65500000000000003,"5min":0.67800000000000005,"15min":0.78000000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.31},"max":{"1min":0.66400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.33900000000000002}]},{"osd":1,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48399999999999999,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.46400000000000002}]},{"osd":2,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.85299999999999998,"15min":0.87},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60799999999999998,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":4,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83799999999999997,"15min":0.83599999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.107,"5min":1.161,"15min":1.2370000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.83799999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1220000000000001,"5min":1.147,"15min":1.226},"last":0.72299999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.81100000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97399999999999998,"5min":1.071,"15min":1.2270000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.82199999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.3140000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.82899999999999996,"15min":0.83199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.2290000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.84099999999999997,"15min":0.85799999999999998},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1879999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83699999999999997,"15min":0.85899999999999999},"min":{"1min":0.55900000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.117,"5min":1.117,"15min":1.2829999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.84799999999999998,"15min":0.86599999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.51300000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.248},"last":0.63600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575969,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308016,"kb_used_data":2728,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063824,"statfs":{"total":96636764160,"available":96321355776,"internally_reserved":0,"allocated":2793472,"data_stored":1436476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.71699999999999997},"min":{"1min":0.58399999999999996,"5min":0.378,"15min":0.378},"max":{"1min":0.94899999999999995,"5min":0.96599999999999997,"15min":0.98999999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69099999999999995,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.006},"last":0.58099999999999996}]},{"osd":1,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.0289999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.018,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73999999999999999,"15min":0.75700000000000001},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.42599999999999999},"max":{"1min":0.90600000000000003,"5min":0.98599999999999999,"15min":1.087},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.63800000000000001,"5min":0.53500000000000003,"15min":0.48299999999999998},"max":{"1min":0.98499999999999999,"5min":1.04,"15min":1.071},"last":0.78600000000000003}]},{"osd":3,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.57599999999999996,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.0329999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90900000000000003,"5min":1.236,"15min":1.236},"last":0.53500000000000003}]},{"osd":5,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.42399999999999999,"15min":0.40100000000000002},"max":{"1min":0.81499999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63700000000000001,"15min":0.63800000000000001},"min":{"1min":0.46400000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.85599999999999998,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.871,"5min":1.018,"15min":1.018},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.46300000000000002},"max":{"1min":0.91700000000000004,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.86699999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.441,"15min":0.441},"max":{"1min":0.96499999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.50800000000000001},"max":{"1min":0.94199999999999995,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575970,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308008,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063832,"statfs":{"total":96636764160,"available":96321363968,"internally_reserved":0,"allocated":2785280,"data_stored":1429854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.68700000000000006},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.309},"max":{"1min":0.83999999999999997,"5min":1.226,"15min":1.226},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.72399999999999998,"15min":0.70899999999999996},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83599999999999997,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78500000000000003,"15min":0.75600000000000001},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85299999999999998,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.78000000000000003},"min":{"1min":0.54900000000000004,"5min":0.499,"15min":0.47999999999999998},"max":{"1min":0.91400000000000003,"5min":1.105,"15min":1.2290000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.45100000000000001},"max":{"1min":0.90000000000000002,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.77300000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.48299999999999998},"max":{"1min":0.878,"5min":4.9779999999999998,"15min":4.9779999999999998},"last":0.70299999999999996}]},{"osd":3,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.36499999999999999},"max":{"1min":0.79900000000000004,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.76800000000000002,"15min":0.749},"min":{"1min":0.57099999999999995,"5min":0.52600000000000002,"15min":0.45800000000000002},"max":{"1min":0.86099999999999999,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":0.66200000000000003}]},{"osd":4,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.63800000000000001,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.69499999999999995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.38800000000000001},"max":{"1min":0.84399999999999997,"5min":1.585,"15min":1.585},"last":0.752}]},{"osd":6,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.66900000000000004,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.77500000000000002,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.441,"15min":0.42899999999999999},"max":{"1min":0.86099999999999999,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":0.76800000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.41399999999999998},"max":{"1min":0.879,"5min":1.365,"15min":1.365},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.77900000000000003,"15min":0.71899999999999997},"min":{"1min":0.50800000000000001,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":0.91200000000000003,"5min":6.891,"15min":6.891},"last":0.78600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:18:27.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:18:27.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.398+0000 7fd2095ff700 1 Processor -- start 2023-11-06T22:18:27.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.398+0000 7fd2095ff700 1 -- start start 2023-11-06T22:18:27.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.399+0000 7fd2095ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c61d0 0x7fd2040c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.399+0000 7fd2095ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2041554a0 con 0x7fd2040c5d50 2023-11-06T22:18:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.399+0000 7fd2095ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd204155930 con 0x7fd2040c1840 2023-11-06T22:18:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.399+0000 7fd2095ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd204154960 con 0x7fd2040c61d0 2023-11-06T22:18:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd2027fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd2040c5d50 0x7fd2040bfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55596/0 (socket says 172.21.15.33:55596) 2023-11-06T22:18:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd2027fc700 1 -- 172.21.15.33:0/2844381423 learned_addr learned my addr 172.21.15.33:0/2844381423 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd2037fe700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c61d0 0x7fd2040c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842188770 0 0) 0x7fd2041554a0 con 0x7fd2040c5d50 2023-11-06T22:18:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1ec0033f0 con 0x7fd2040c5d50 2023-11-06T22:18:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 930191202 0 0) 0x7fd1ec0033f0 con 0x7fd2040c5d50 2023-11-06T22:18:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c61d0 msgr2=0x7fd2040c0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd201ffb700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c61d0 0x7fd2040c0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 >> v1:172.21.15.69:6789/0 conn(0x7fd2040c1840 legacy=0x7fd2040a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040aa4f0 con 0x7fd2040c5d50 2023-11-06T22:18:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fd1f8001470 con 0x7fd2040c5d50 2023-11-06T22:18:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.400+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 >> v1:172.21.15.33:6789/0 conn(0x7fd2040c5d50 legacy=0x7fd2040bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd201ffb700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1ec003910 0x7fd1ec003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd201ffb700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c61d0 0x7fd1ec004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd201ffb700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ec004b80 0x7fd1ec005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204154960 con 0x7fd1ec004b80 2023-11-06T22:18:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec00b8e0 con 0x7fd1ec003910 2023-11-06T22:18:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec00ba50 con 0x7fd2040c61d0 2023-11-06T22:18:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd2037fe700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ec004b80 0x7fd1ec005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd202ffd700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1ec003910 0x7fd1ec003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd2027fc700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c61d0 0x7fd1ec004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd2037fe700 1 -- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c61d0 msgr2=0x7fd1ec004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd2037fe700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c61d0 0x7fd1ec004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd2037fe700 1 -- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1ec003910 msgr2=0x7fd1ec003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd2037fe700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1ec003910 0x7fd1ec003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd2037fe700 1 -- 172.21.15.33:0/2844381423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0033f0 con 0x7fd1ec004b80 2023-11-06T22:18:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.401+0000 7fd2037fe700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ec004b80 0x7fd1ec005000 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7fd1fc00e210 tx=0x7fd1fc00b040).ready entity=mon.0 client_cookie=3039df528e027a1d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.402+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc009ac0 con 0x7fd1ec004b80 2023-11-06T22:18:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.402+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc00f040 con 0x7fd1ec004b80 2023-11-06T22:18:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.402+0000 7fd201ffb700 1 -- 172.21.15.33:0/2844381423 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc00ec70 con 0x7fd1ec004b80 2023-11-06T22:18:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.402+0000 7fd2095ff700 1 -- 172.21.15.33:0/2844381423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ec004b80 msgr2=0x7fd1ec005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.402+0000 7fd2095ff700 1 --2- 172.21.15.33:0/2844381423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ec004b80 0x7fd1ec005000 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7fd1fc00e210 tx=0x7fd1fc00b040).stop 2023-11-06T22:18:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.402+0000 7fd2037fe700 1 -- 172.21.15.33:0/2844381423 reap_dead start 2023-11-06T22:18:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.402+0000 7fd2095ff700 1 -- 172.21.15.33:0/2844381423 shutdown_connections 2023-11-06T22:18:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.402+0000 7fd2095ff700 1 -- 172.21.15.33:0/2844381423 >> 172.21.15.33:0/2844381423 conn(0x7fd20405ec80 msgr2=0x7fd2040cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.402+0000 7fd2095ff700 1 -- 172.21.15.33:0/2844381423 shutdown_connections 2023-11-06T22:18:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.402+0000 7fd2095ff700 1 -- 172.21.15.33:0/2844381423 wait complete. 2023-11-06T22:18:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.404+0000 7fd2095ff700 1 Processor -- start 2023-11-06T22:18:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.404+0000 7fd2095ff700 1 -- start start 2023-11-06T22:18:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.405+0000 7fd2095ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1ec004b80 0x7fd20413f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.405+0000 7fd2095ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd204158ef0 0x7fd2040c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.405+0000 7fd2095ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c0a90 0x7fd2040c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.405+0000 7fd2095ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204151ce0 con 0x7fd204158ef0 2023-11-06T22:18:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.405+0000 7fd2095ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040c21b0 con 0x7fd1ec004b80 2023-11-06T22:18:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.405+0000 7fd2095ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040c1400 con 0x7fd2040c0a90 2023-11-06T22:18:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.405+0000 7fd2027fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd204158ef0 0x7fd2040c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.405+0000 7fd2027fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd204158ef0 0x7fd2040c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49256/0 (socket says 172.21.15.33:49256) 2023-11-06T22:18:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.405+0000 7fd2027fc700 1 -- 172.21.15.33:0/2276915603 learned_addr learned my addr 172.21.15.33:0/2276915603 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd202ffd700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1ec004b80 0x7fd20413f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd2037fe700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c0a90 0x7fd2040c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd2027fc700 1 -- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c0a90 msgr2=0x7fd2040c0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd2027fc700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c0a90 0x7fd2040c0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd2027fc700 1 -- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1ec004b80 msgr2=0x7fd20413f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd2027fc700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1ec004b80 0x7fd20413f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd2027fc700 1 -- 172.21.15.33:0/2276915603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204159440 con 0x7fd204158ef0 2023-11-06T22:18:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd2027fc700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd204158ef0 0x7fd2040c4a30 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7fd1f80049e0 tx=0x7fd204155410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd1eaffd700 1 -- 172.21.15.33:0/2276915603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8004e50 con 0x7fd204158ef0 2023-11-06T22:18:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd1eaffd700 1 -- 172.21.15.33:0/2276915603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f800f8d0 con 0x7fd204158ef0 2023-11-06T22:18:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd1eaffd700 1 -- 172.21.15.33:0/2276915603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f800fc50 con 0x7fd204158ef0 2023-11-06T22:18:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd2095ff700 1 -- 172.21.15.33:0/2276915603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd204159720 con 0x7fd204158ef0 2023-11-06T22:18:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.406+0000 7fd2095ff700 1 -- 172.21.15.33:0/2276915603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2040c4f20 con 0x7fd204158ef0 2023-11-06T22:18:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.408+0000 7fd2095ff700 1 -- 172.21.15.33:0/2276915603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20406c6c0 con 0x7fd204158ef0 2023-11-06T22:18:27.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.408+0000 7fd1eaffd700 1 -- 172.21.15.33:0/2276915603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fd1f8012560 con 0x7fd204158ef0 2023-11-06T22:18:27.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.409+0000 7fd1eaffd700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1f005f270 0x7fd1f0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.409+0000 7fd1eaffd700 1 -- 172.21.15.33:0/2276915603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fd1f8079dd0 con 0x7fd204158ef0 2023-11-06T22:18:27.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.410+0000 7fd202ffd700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1f005f270 0x7fd1f0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.410+0000 7fd202ffd700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1f005f270 0x7fd1f0061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd1f40010e0 tx=0x7fd1f4009040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:27.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.413+0000 7fd1eaffd700 1 -- 172.21.15.33:0/2276915603 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd1f804aee0 con 0x7fd204158ef0 2023-11-06T22:18:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.588+0000 7fd2095ff700 1 -- 172.21.15.33:0/2276915603 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2040c51a0 con 0x7fd1f005f270 2023-11-06T22:18:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.600+0000 7fd1eaffd700 1 -- 172.21.15.33:0/2276915603 <== mgr.9310 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361758 (secure 0 0 0) 0x7fd1f4003620 con 0x7fd1f005f270 2023-11-06T22:18:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 -- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1f005f270 msgr2=0x7fd1f0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1f005f270 0x7fd1f0061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd1f40010e0 tx=0x7fd1f4009040).stop 2023-11-06T22:18:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 -- 172.21.15.33:0/2276915603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd204158ef0 msgr2=0x7fd2040c4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd204158ef0 0x7fd2040c4a30 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7fd1f80049e0 tx=0x7fd204155410).stop 2023-11-06T22:18:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 -- 172.21.15.33:0/2276915603 shutdown_connections 2023-11-06T22:18:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1f005f270 0x7fd1f0061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd2040c0a90 0x7fd2040c0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1ec004b80 0x7fd20413f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 --2- 172.21.15.33:0/2276915603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd204158ef0 0x7fd2040c4a30 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 -- 172.21.15.33:0/2276915603 >> 172.21.15.33:0/2276915603 conn(0x7fd20405ec80 msgr2=0x7fd2040ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 -- 172.21.15.33:0/2276915603 shutdown_connections 2023-11-06T22:18:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.604+0000 7fd2095ff700 1 -- 172.21.15.33:0/2276915603 wait complete. 2023-11-06T22:18:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:18:27.670 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T22:18:27.552272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2464112,"kb_used_data":21808,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752510608,"statfs":{"total":773094113280,"available":770570862592,"internally_reserved":0,"allocated":22331392,"data_stored":11469156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-31423,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-84,"num_write_kb":-67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":-10,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.006615"},"pg_stats":[{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871810+0000","last_change":"2023-11-06T22:18:22.861504+0000","last_active":"2023-11-06T22:18:22.871810+0000","last_peered":"2023-11-06T22:18:22.871810+0000","last_clean":"2023-11-06T22:18:22.871810+0000","last_became_active":"2023-11-06T22:18:22.861190+0000","last_became_peered":"2023-11-06T22:18:22.861190+0000","last_unstale":"2023-11-06T22:18:22.871810+0000","last_undegraded":"2023-11-06T22:18:22.871810+0000","last_fullsized":"2023-11-06T22:18:22.871810+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3f","version":"222'3","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.874350+0000","last_change":"2023-11-06T22:18:22.381412+0000","last_active":"2023-11-06T22:18:22.874350+0000","last_peered":"2023-11-06T22:18:22.874350+0000","last_clean":"2023-11-06T22:18:22.874350+0000","last_became_active":"2023-11-06T22:18:22.379227+0000","last_became_peered":"2023-11-06T22:18:22.379227+0000","last_unstale":"2023-11-06T22:18:22.874350+0000","last_undegraded":"2023-11-06T22:18:22.874350+0000","last_fullsized":"2023-11-06T22:18:22.874350+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.873793+0000","last_change":"2023-11-06T22:18:22.867712+0000","last_active":"2023-11-06T22:18:22.873793+0000","last_peered":"2023-11-06T22:18:22.873793+0000","last_clean":"2023-11-06T22:18:22.873793+0000","last_became_active":"2023-11-06T22:18:22.867287+0000","last_became_peered":"2023-11-06T22:18:22.867287+0000","last_unstale":"2023-11-06T22:18:22.873793+0000","last_undegraded":"2023-11-06T22:18:22.873793+0000","last_fullsized":"2023-11-06T22:18:22.873793+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.382979+0000","last_change":"2023-11-06T22:18:22.378731+0000","last_active":"2023-11-06T22:18:22.382979+0000","last_peered":"2023-11-06T22:18:22.382979+0000","last_clean":"2023-11-06T22:18:22.382979+0000","last_became_active":"2023-11-06T22:18:22.378439+0000","last_became_peered":"2023-11-06T22:18:22.378439+0000","last_unstale":"2023-11-06T22:18:22.382979+0000","last_undegraded":"2023-11-06T22:18:22.382979+0000","last_fullsized":"2023-11-06T22:18:22.382979+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.881008+0000","last_change":"2023-11-06T22:18:22.879970+0000","last_active":"2023-11-06T22:18:22.881008+0000","last_peered":"2023-11-06T22:18:22.881008+0000","last_clean":"2023-11-06T22:18:22.881008+0000","last_became_active":"2023-11-06T22:18:22.879584+0000","last_became_peered":"2023-11-06T22:18:22.879584+0000","last_unstale":"2023-11-06T22:18:22.881008+0000","last_undegraded":"2023-11-06T22:18:22.881008+0000","last_fullsized":"2023-11-06T22:18:22.881008+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.369038+0000","last_change":"2023-11-06T22:18:22.356342+0000","last_active":"2023-11-06T22:18:22.369038+0000","last_peered":"2023-11-06T22:18:22.369038+0000","last_clean":"2023-11-06T22:18:22.369038+0000","last_became_active":"2023-11-06T22:18:22.355225+0000","last_became_peered":"2023-11-06T22:18:22.355225+0000","last_unstale":"2023-11-06T22:18:22.369038+0000","last_undegraded":"2023-11-06T22:18:22.369038+0000","last_fullsized":"2023-11-06T22:18:22.369038+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3f","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.850680+0000","last_change":"2023-11-06T22:18:22.850680+0000","last_active":"2023-11-06T22:18:22.850680+0000","last_peered":"2023-11-06T22:18:22.850680+0000","last_clean":"2023-11-06T22:18:22.850680+0000","last_became_active":"2023-11-06T22:18:22.849640+0000","last_became_peered":"2023-11-06T22:18:22.849640+0000","last_unstale":"2023-11-06T22:18:22.850680+0000","last_undegraded":"2023-11-06T22:18:22.850680+0000","last_fullsized":"2023-11-06T22:18:22.850680+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"222'2","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.856191+0000","last_change":"2023-11-06T22:18:22.376417+0000","last_active":"2023-11-06T22:18:22.856191+0000","last_peered":"2023-11-06T22:18:22.856191+0000","last_clean":"2023-11-06T22:18:22.856191+0000","last_became_active":"2023-11-06T22:18:22.376125+0000","last_became_peered":"2023-11-06T22:18:22.376125+0000","last_unstale":"2023-11-06T22:18:22.856191+0000","last_undegraded":"2023-11-06T22:18:22.856191+0000","last_fullsized":"2023-11-06T22:18:22.856191+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866585+0000","last_change":"2023-11-06T22:18:22.862316+0000","last_active":"2023-11-06T22:18:22.866585+0000","last_peered":"2023-11-06T22:18:22.866585+0000","last_clean":"2023-11-06T22:18:22.866585+0000","last_became_active":"2023-11-06T22:18:22.862083+0000","last_became_peered":"2023-11-06T22:18:22.862083+0000","last_unstale":"2023-11-06T22:18:22.866585+0000","last_undegraded":"2023-11-06T22:18:22.866585+0000","last_fullsized":"2023-11-06T22:18:22.866585+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3b","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.876467+0000","last_change":"2023-11-06T22:18:22.380952+0000","last_active":"2023-11-06T22:18:22.876467+0000","last_peered":"2023-11-06T22:18:22.876467+0000","last_clean":"2023-11-06T22:18:22.876467+0000","last_became_active":"2023-11-06T22:18:22.380507+0000","last_became_peered":"2023-11-06T22:18:22.380507+0000","last_unstale":"2023-11-06T22:18:22.876467+0000","last_undegraded":"2023-11-06T22:18:22.876467+0000","last_fullsized":"2023-11-06T22:18:22.876467+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.850758+0000","last_change":"2023-11-06T22:18:22.850758+0000","last_active":"2023-11-06T22:18:22.850758+0000","last_peered":"2023-11-06T22:18:22.850758+0000","last_clean":"2023-11-06T22:18:22.850758+0000","last_became_active":"2023-11-06T22:18:22.850443+0000","last_became_peered":"2023-11-06T22:18:22.850443+0000","last_unstale":"2023-11-06T22:18:22.850758+0000","last_undegraded":"2023-11-06T22:18:22.850758+0000","last_fullsized":"2023-11-06T22:18:22.850758+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3a","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.874181+0000","last_change":"2023-11-06T22:18:22.377088+0000","last_active":"2023-11-06T22:18:22.874181+0000","last_peered":"2023-11-06T22:18:22.874181+0000","last_clean":"2023-11-06T22:18:22.874181+0000","last_became_active":"2023-11-06T22:18:22.374923+0000","last_became_peered":"2023-11-06T22:18:22.374923+0000","last_unstale":"2023-11-06T22:18:22.874181+0000","last_undegraded":"2023-11-06T22:18:22.874181+0000","last_fullsized":"2023-11-06T22:18:22.874181+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.875306+0000","last_change":"2023-11-06T22:18:22.867250+0000","last_active":"2023-11-06T22:18:22.875306+0000","last_peered":"2023-11-06T22:18:22.875306+0000","last_clean":"2023-11-06T22:18:22.875306+0000","last_became_active":"2023-11-06T22:18:22.866842+0000","last_became_peered":"2023-11-06T22:18:22.866842+0000","last_unstale":"2023-11-06T22:18:22.875306+0000","last_undegraded":"2023-11-06T22:18:22.875306+0000","last_fullsized":"2023-11-06T22:18:22.875306+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.858010+0000","last_change":"2023-11-06T22:18:22.371098+0000","last_active":"2023-11-06T22:18:22.858010+0000","last_peered":"2023-11-06T22:18:22.858010+0000","last_clean":"2023-11-06T22:18:22.858010+0000","last_became_active":"2023-11-06T22:18:22.369819+0000","last_became_peered":"2023-11-06T22:18:22.369819+0000","last_unstale":"2023-11-06T22:18:22.858010+0000","last_undegraded":"2023-11-06T22:18:22.858010+0000","last_fullsized":"2023-11-06T22:18:22.858010+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.864341+0000","last_change":"2023-11-06T22:18:22.864341+0000","last_active":"2023-11-06T22:18:22.864341+0000","last_peered":"2023-11-06T22:18:22.864341+0000","last_clean":"2023-11-06T22:18:22.864341+0000","last_became_active":"2023-11-06T22:18:22.864034+0000","last_became_peered":"2023-11-06T22:18:22.864034+0000","last_unstale":"2023-11-06T22:18:22.864341+0000","last_undegraded":"2023-11-06T22:18:22.864341+0000","last_fullsized":"2023-11-06T22:18:22.864341+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.859812+0000","last_change":"2023-11-06T22:18:22.382620+0000","last_active":"2023-11-06T22:18:22.859812+0000","last_peered":"2023-11-06T22:18:22.859812+0000","last_clean":"2023-11-06T22:18:22.859812+0000","last_became_active":"2023-11-06T22:18:22.381716+0000","last_became_peered":"2023-11-06T22:18:22.381716+0000","last_unstale":"2023-11-06T22:18:22.859812+0000","last_undegraded":"2023-11-06T22:18:22.859812+0000","last_fullsized":"2023-11-06T22:18:22.859812+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878443+0000","last_change":"2023-11-06T22:18:22.874837+0000","last_active":"2023-11-06T22:18:22.878443+0000","last_peered":"2023-11-06T22:18:22.878443+0000","last_clean":"2023-11-06T22:18:22.878443+0000","last_became_active":"2023-11-06T22:18:22.872927+0000","last_became_peered":"2023-11-06T22:18:22.872927+0000","last_unstale":"2023-11-06T22:18:22.878443+0000","last_undegraded":"2023-11-06T22:18:22.878443+0000","last_fullsized":"2023-11-06T22:18:22.878443+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.381800+0000","last_change":"2023-11-06T22:18:22.377682+0000","last_active":"2023-11-06T22:18:22.381800+0000","last_peered":"2023-11-06T22:18:22.381800+0000","last_clean":"2023-11-06T22:18:22.381800+0000","last_became_active":"2023-11-06T22:18:22.377304+0000","last_became_peered":"2023-11-06T22:18:22.377304+0000","last_unstale":"2023-11-06T22:18:22.381800+0000","last_undegraded":"2023-11-06T22:18:22.381800+0000","last_fullsized":"2023-11-06T22:18:22.381800+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.849872+0000","last_change":"2023-11-06T22:18:22.849872+0000","last_active":"2023-11-06T22:18:22.849872+0000","last_peered":"2023-11-06T22:18:22.849872+0000","last_clean":"2023-11-06T22:18:22.849872+0000","last_became_active":"2023-11-06T22:18:22.849613+0000","last_became_peered":"2023-11-06T22:18:22.849613+0000","last_unstale":"2023-11-06T22:18:22.849872+0000","last_undegraded":"2023-11-06T22:18:22.849872+0000","last_fullsized":"2023-11-06T22:18:22.849872+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.849063+0000","last_change":"2023-11-06T22:18:22.375248+0000","last_active":"2023-11-06T22:18:22.849063+0000","last_peered":"2023-11-06T22:18:22.849063+0000","last_clean":"2023-11-06T22:18:22.849063+0000","last_became_active":"2023-11-06T22:18:22.374934+0000","last_became_peered":"2023-11-06T22:18:22.374934+0000","last_unstale":"2023-11-06T22:18:22.849063+0000","last_undegraded":"2023-11-06T22:18:22.849063+0000","last_fullsized":"2023-11-06T22:18:22.849063+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868558+0000","last_change":"2023-11-06T22:18:22.864984+0000","last_active":"2023-11-06T22:18:22.868558+0000","last_peered":"2023-11-06T22:18:22.868558+0000","last_clean":"2023-11-06T22:18:22.868558+0000","last_became_active":"2023-11-06T22:18:22.864595+0000","last_became_peered":"2023-11-06T22:18:22.864595+0000","last_unstale":"2023-11-06T22:18:22.868558+0000","last_undegraded":"2023-11-06T22:18:22.868558+0000","last_fullsized":"2023-11-06T22:18:22.868558+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840784+0000","last_change":"2023-11-06T22:18:22.378145+0000","last_active":"2023-11-06T22:18:22.840784+0000","last_peered":"2023-11-06T22:18:22.840784+0000","last_clean":"2023-11-06T22:18:22.840784+0000","last_became_active":"2023-11-06T22:18:22.377860+0000","last_became_peered":"2023-11-06T22:18:22.377860+0000","last_unstale":"2023-11-06T22:18:22.840784+0000","last_undegraded":"2023-11-06T22:18:22.840784+0000","last_fullsized":"2023-11-06T22:18:22.840784+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.874646+0000","last_change":"2023-11-06T22:18:22.865609+0000","last_active":"2023-11-06T22:18:22.874646+0000","last_peered":"2023-11-06T22:18:22.874646+0000","last_clean":"2023-11-06T22:18:22.874646+0000","last_became_active":"2023-11-06T22:18:22.865210+0000","last_became_peered":"2023-11-06T22:18:22.865210+0000","last_unstale":"2023-11-06T22:18:22.874646+0000","last_undegraded":"2023-11-06T22:18:22.874646+0000","last_fullsized":"2023-11-06T22:18:22.874646+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.34","version":"222'4","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.885284+0000","last_change":"2023-11-06T22:18:22.367605+0000","last_active":"2023-11-06T22:18:22.885284+0000","last_peered":"2023-11-06T22:18:22.885284+0000","last_clean":"2023-11-06T22:18:22.885284+0000","last_became_active":"2023-11-06T22:18:22.367308+0000","last_became_peered":"2023-11-06T22:18:22.367308+0000","last_unstale":"2023-11-06T22:18:22.885284+0000","last_undegraded":"2023-11-06T22:18:22.885284+0000","last_fullsized":"2023-11-06T22:18:22.885284+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878521+0000","last_change":"2023-11-06T22:18:22.876197+0000","last_active":"2023-11-06T22:18:22.878521+0000","last_peered":"2023-11-06T22:18:22.878521+0000","last_clean":"2023-11-06T22:18:22.878521+0000","last_became_active":"2023-11-06T22:18:22.874183+0000","last_became_peered":"2023-11-06T22:18:22.874183+0000","last_unstale":"2023-11-06T22:18:22.878521+0000","last_undegraded":"2023-11-06T22:18:22.878521+0000","last_fullsized":"2023-11-06T22:18:22.878521+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.33","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868592+0000","last_change":"2023-11-06T22:18:22.374349+0000","last_active":"2023-11-06T22:18:22.868592+0000","last_peered":"2023-11-06T22:18:22.868592+0000","last_clean":"2023-11-06T22:18:22.868592+0000","last_became_active":"2023-11-06T22:18:22.374063+0000","last_became_peered":"2023-11-06T22:18:22.374063+0000","last_unstale":"2023-11-06T22:18:22.868592+0000","last_undegraded":"2023-11-06T22:18:22.868592+0000","last_fullsized":"2023-11-06T22:18:22.868592+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.873960+0000","last_change":"2023-11-06T22:18:22.863483+0000","last_active":"2023-11-06T22:18:22.873960+0000","last_peered":"2023-11-06T22:18:22.873960+0000","last_clean":"2023-11-06T22:18:22.873960+0000","last_became_active":"2023-11-06T22:18:22.863213+0000","last_became_peered":"2023-11-06T22:18:22.863213+0000","last_unstale":"2023-11-06T22:18:22.873960+0000","last_undegraded":"2023-11-06T22:18:22.873960+0000","last_fullsized":"2023-11-06T22:18:22.873960+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.382476+0000","last_change":"2023-11-06T22:18:22.377248+0000","last_active":"2023-11-06T22:18:22.382476+0000","last_peered":"2023-11-06T22:18:22.382476+0000","last_clean":"2023-11-06T22:18:22.382476+0000","last_became_active":"2023-11-06T22:18:22.376941+0000","last_became_peered":"2023-11-06T22:18:22.376941+0000","last_unstale":"2023-11-06T22:18:22.382476+0000","last_undegraded":"2023-11-06T22:18:22.382476+0000","last_fullsized":"2023-11-06T22:18:22.382476+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868451+0000","last_change":"2023-11-06T22:18:22.858079+0000","last_active":"2023-11-06T22:18:22.868451+0000","last_peered":"2023-11-06T22:18:22.868451+0000","last_clean":"2023-11-06T22:18:22.868451+0000","last_became_active":"2023-11-06T22:18:22.857735+0000","last_became_peered":"2023-11-06T22:18:22.857735+0000","last_unstale":"2023-11-06T22:18:22.868451+0000","last_undegraded":"2023-11-06T22:18:22.868451+0000","last_fullsized":"2023-11-06T22:18:22.868451+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840963+0000","last_change":"2023-11-06T22:18:22.369394+0000","last_active":"2023-11-06T22:18:22.840963+0000","last_peered":"2023-11-06T22:18:22.840963+0000","last_clean":"2023-11-06T22:18:22.840963+0000","last_became_active":"2023-11-06T22:18:22.369068+0000","last_became_peered":"2023-11-06T22:18:22.369068+0000","last_unstale":"2023-11-06T22:18:22.840963+0000","last_undegraded":"2023-11-06T22:18:22.840963+0000","last_fullsized":"2023-11-06T22:18:22.840963+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.859229+0000","last_change":"2023-11-06T22:18:22.849923+0000","last_active":"2023-11-06T22:18:22.859229+0000","last_peered":"2023-11-06T22:18:22.859229+0000","last_clean":"2023-11-06T22:18:22.859229+0000","last_became_active":"2023-11-06T22:18:22.849630+0000","last_became_peered":"2023-11-06T22:18:22.849630+0000","last_unstale":"2023-11-06T22:18:22.859229+0000","last_undegraded":"2023-11-06T22:18:22.859229+0000","last_fullsized":"2023-11-06T22:18:22.859229+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"222'4","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866065+0000","last_change":"2023-11-06T22:18:22.356545+0000","last_active":"2023-11-06T22:18:22.866065+0000","last_peered":"2023-11-06T22:18:22.866065+0000","last_clean":"2023-11-06T22:18:22.866065+0000","last_became_active":"2023-11-06T22:18:22.355132+0000","last_became_peered":"2023-11-06T22:18:22.355132+0000","last_unstale":"2023-11-06T22:18:22.866065+0000","last_undegraded":"2023-11-06T22:18:22.866065+0000","last_fullsized":"2023-11-06T22:18:22.866065+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.875350+0000","last_change":"2023-11-06T22:18:22.866087+0000","last_active":"2023-11-06T22:18:22.875350+0000","last_peered":"2023-11-06T22:18:22.875350+0000","last_clean":"2023-11-06T22:18:22.875350+0000","last_became_active":"2023-11-06T22:18:22.865667+0000","last_became_peered":"2023-11-06T22:18:22.865667+0000","last_unstale":"2023-11-06T22:18:22.875350+0000","last_undegraded":"2023-11-06T22:18:22.875350+0000","last_fullsized":"2023-11-06T22:18:22.875350+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.856844+0000","last_change":"2023-11-06T22:18:22.374053+0000","last_active":"2023-11-06T22:18:22.856844+0000","last_peered":"2023-11-06T22:18:22.856844+0000","last_clean":"2023-11-06T22:18:22.856844+0000","last_became_active":"2023-11-06T22:18:22.372649+0000","last_became_peered":"2023-11-06T22:18:22.372649+0000","last_unstale":"2023-11-06T22:18:22.856844+0000","last_undegraded":"2023-11-06T22:18:22.856844+0000","last_fullsized":"2023-11-06T22:18:22.856844+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878724+0000","last_change":"2023-11-06T22:18:22.872896+0000","last_active":"2023-11-06T22:18:22.878724+0000","last_peered":"2023-11-06T22:18:22.878724+0000","last_clean":"2023-11-06T22:18:22.878724+0000","last_became_active":"2023-11-06T22:18:22.872616+0000","last_became_peered":"2023-11-06T22:18:22.872616+0000","last_unstale":"2023-11-06T22:18:22.878724+0000","last_undegraded":"2023-11-06T22:18:22.878724+0000","last_fullsized":"2023-11-06T22:18:22.878724+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860889+0000","last_change":"2023-11-06T22:18:22.380112+0000","last_active":"2023-11-06T22:18:22.860889+0000","last_peered":"2023-11-06T22:18:22.860889+0000","last_clean":"2023-11-06T22:18:22.860889+0000","last_became_active":"2023-11-06T22:18:22.379807+0000","last_became_peered":"2023-11-06T22:18:22.379807+0000","last_unstale":"2023-11-06T22:18:22.860889+0000","last_undegraded":"2023-11-06T22:18:22.860889+0000","last_fullsized":"2023-11-06T22:18:22.860889+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866476+0000","last_change":"2023-11-06T22:18:22.866476+0000","last_active":"2023-11-06T22:18:22.866476+0000","last_peered":"2023-11-06T22:18:22.866476+0000","last_clean":"2023-11-06T22:18:22.866476+0000","last_became_active":"2023-11-06T22:18:22.866109+0000","last_became_peered":"2023-11-06T22:18:22.866109+0000","last_unstale":"2023-11-06T22:18:22.866476+0000","last_undegraded":"2023-11-06T22:18:22.866476+0000","last_fullsized":"2023-11-06T22:18:22.866476+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2d","version":"222'1","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.846309+0000","last_change":"2023-11-06T22:18:22.375723+0000","last_active":"2023-11-06T22:18:22.846309+0000","last_peered":"2023-11-06T22:18:22.846309+0000","last_clean":"2023-11-06T22:18:22.846309+0000","last_became_active":"2023-11-06T22:18:22.374350+0000","last_became_peered":"2023-11-06T22:18:22.374350+0000","last_unstale":"2023-11-06T22:18:22.846309+0000","last_undegraded":"2023-11-06T22:18:22.846309+0000","last_fullsized":"2023-11-06T22:18:22.846309+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.880263+0000","last_change":"2023-11-06T22:18:22.878279+0000","last_active":"2023-11-06T22:18:22.880263+0000","last_peered":"2023-11-06T22:18:22.880263+0000","last_clean":"2023-11-06T22:18:22.880263+0000","last_became_active":"2023-11-06T22:18:22.877828+0000","last_became_peered":"2023-11-06T22:18:22.877828+0000","last_unstale":"2023-11-06T22:18:22.880263+0000","last_undegraded":"2023-11-06T22:18:22.880263+0000","last_fullsized":"2023-11-06T22:18:22.880263+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862906+0000","last_change":"2023-11-06T22:18:22.364440+0000","last_active":"2023-11-06T22:18:22.862906+0000","last_peered":"2023-11-06T22:18:22.862906+0000","last_clean":"2023-11-06T22:18:22.862906+0000","last_became_active":"2023-11-06T22:18:22.364153+0000","last_became_peered":"2023-11-06T22:18:22.364153+0000","last_unstale":"2023-11-06T22:18:22.862906+0000","last_undegraded":"2023-11-06T22:18:22.862906+0000","last_fullsized":"2023-11-06T22:18:22.862906+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.879010+0000","last_change":"2023-11-06T22:18:22.872375+0000","last_active":"2023-11-06T22:18:22.879010+0000","last_peered":"2023-11-06T22:18:22.879010+0000","last_clean":"2023-11-06T22:18:22.879010+0000","last_became_active":"2023-11-06T22:18:22.871857+0000","last_became_peered":"2023-11-06T22:18:22.871857+0000","last_unstale":"2023-11-06T22:18:22.879010+0000","last_undegraded":"2023-11-06T22:18:22.879010+0000","last_fullsized":"2023-11-06T22:18:22.879010+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"222'1","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.844892+0000","last_change":"2023-11-06T22:18:22.379138+0000","last_active":"2023-11-06T22:18:22.844892+0000","last_peered":"2023-11-06T22:18:22.844892+0000","last_clean":"2023-11-06T22:18:22.844892+0000","last_became_active":"2023-11-06T22:18:22.376082+0000","last_became_peered":"2023-11-06T22:18:22.376082+0000","last_unstale":"2023-11-06T22:18:22.844892+0000","last_undegraded":"2023-11-06T22:18:22.844892+0000","last_fullsized":"2023-11-06T22:18:22.844892+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.873183+0000","last_change":"2023-11-06T22:18:22.868802+0000","last_active":"2023-11-06T22:18:22.873183+0000","last_peered":"2023-11-06T22:18:22.873183+0000","last_clean":"2023-11-06T22:18:22.873183+0000","last_became_active":"2023-11-06T22:18:22.868499+0000","last_became_peered":"2023-11-06T22:18:22.868499+0000","last_unstale":"2023-11-06T22:18:22.873183+0000","last_undegraded":"2023-11-06T22:18:22.873183+0000","last_fullsized":"2023-11-06T22:18:22.873183+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.869561+0000","last_change":"2023-11-06T22:18:22.367410+0000","last_active":"2023-11-06T22:18:22.869561+0000","last_peered":"2023-11-06T22:18:22.869561+0000","last_clean":"2023-11-06T22:18:22.869561+0000","last_became_active":"2023-11-06T22:18:22.366809+0000","last_became_peered":"2023-11-06T22:18:22.366809+0000","last_unstale":"2023-11-06T22:18:22.869561+0000","last_undegraded":"2023-11-06T22:18:22.869561+0000","last_fullsized":"2023-11-06T22:18:22.869561+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.880506+0000","last_change":"2023-11-06T22:18:22.878761+0000","last_active":"2023-11-06T22:18:22.880506+0000","last_peered":"2023-11-06T22:18:22.880506+0000","last_clean":"2023-11-06T22:18:22.880506+0000","last_became_active":"2023-11-06T22:18:22.877180+0000","last_became_peered":"2023-11-06T22:18:22.877180+0000","last_unstale":"2023-11-06T22:18:22.880506+0000","last_undegraded":"2023-11-06T22:18:22.880506+0000","last_fullsized":"2023-11-06T22:18:22.880506+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.850901+0000","last_change":"2023-11-06T22:18:22.372123+0000","last_active":"2023-11-06T22:18:22.850901+0000","last_peered":"2023-11-06T22:18:22.850901+0000","last_clean":"2023-11-06T22:18:22.850901+0000","last_became_active":"2023-11-06T22:18:22.371777+0000","last_became_peered":"2023-11-06T22:18:22.371777+0000","last_unstale":"2023-11-06T22:18:22.850901+0000","last_undegraded":"2023-11-06T22:18:22.850901+0000","last_fullsized":"2023-11-06T22:18:22.850901+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866100+0000","last_change":"2023-11-06T22:18:22.866100+0000","last_active":"2023-11-06T22:18:22.866100+0000","last_peered":"2023-11-06T22:18:22.866100+0000","last_clean":"2023-11-06T22:18:22.866100+0000","last_became_active":"2023-11-06T22:18:22.865808+0000","last_became_peered":"2023-11-06T22:18:22.865808+0000","last_unstale":"2023-11-06T22:18:22.866100+0000","last_undegraded":"2023-11-06T22:18:22.866100+0000","last_fullsized":"2023-11-06T22:18:22.866100+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.859921+0000","last_change":"2023-11-06T22:18:22.378361+0000","last_active":"2023-11-06T22:18:22.859921+0000","last_peered":"2023-11-06T22:18:22.859921+0000","last_clean":"2023-11-06T22:18:22.859921+0000","last_became_active":"2023-11-06T22:18:22.377935+0000","last_became_peered":"2023-11-06T22:18:22.377935+0000","last_unstale":"2023-11-06T22:18:22.859921+0000","last_undegraded":"2023-11-06T22:18:22.859921+0000","last_fullsized":"2023-11-06T22:18:22.859921+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878627+0000","last_change":"2023-11-06T22:18:22.876877+0000","last_active":"2023-11-06T22:18:22.878627+0000","last_peered":"2023-11-06T22:18:22.878627+0000","last_clean":"2023-11-06T22:18:22.878627+0000","last_became_active":"2023-11-06T22:18:22.875822+0000","last_became_peered":"2023-11-06T22:18:22.875822+0000","last_unstale":"2023-11-06T22:18:22.878627+0000","last_undegraded":"2023-11-06T22:18:22.878627+0000","last_fullsized":"2023-11-06T22:18:22.878627+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.383282+0000","last_change":"2023-11-06T22:18:22.378945+0000","last_active":"2023-11-06T22:18:22.383282+0000","last_peered":"2023-11-06T22:18:22.383282+0000","last_clean":"2023-11-06T22:18:22.383282+0000","last_became_active":"2023-11-06T22:18:22.378630+0000","last_became_peered":"2023-11-06T22:18:22.378630+0000","last_unstale":"2023-11-06T22:18:22.383282+0000","last_undegraded":"2023-11-06T22:18:22.383282+0000","last_fullsized":"2023-11-06T22:18:22.383282+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871291+0000","last_change":"2023-11-06T22:18:22.866193+0000","last_active":"2023-11-06T22:18:22.871291+0000","last_peered":"2023-11-06T22:18:22.871291+0000","last_clean":"2023-11-06T22:18:22.871291+0000","last_became_active":"2023-11-06T22:18:22.865914+0000","last_became_peered":"2023-11-06T22:18:22.865914+0000","last_unstale":"2023-11-06T22:18:22.871291+0000","last_undegraded":"2023-11-06T22:18:22.871291+0000","last_fullsized":"2023-11-06T22:18:22.871291+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.847695+0000","last_change":"2023-11-06T22:18:22.371924+0000","last_active":"2023-11-06T22:18:22.847695+0000","last_peered":"2023-11-06T22:18:22.847695+0000","last_clean":"2023-11-06T22:18:22.847695+0000","last_became_active":"2023-11-06T22:18:22.370790+0000","last_became_peered":"2023-11-06T22:18:22.370790+0000","last_unstale":"2023-11-06T22:18:22.847695+0000","last_undegraded":"2023-11-06T22:18:22.847695+0000","last_fullsized":"2023-11-06T22:18:22.847695+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.864039+0000","last_change":"2023-11-06T22:18:22.864039+0000","last_active":"2023-11-06T22:18:22.864039+0000","last_peered":"2023-11-06T22:18:22.864039+0000","last_clean":"2023-11-06T22:18:22.864039+0000","last_became_active":"2023-11-06T22:18:22.863551+0000","last_became_peered":"2023-11-06T22:18:22.863551+0000","last_unstale":"2023-11-06T22:18:22.864039+0000","last_undegraded":"2023-11-06T22:18:22.864039+0000","last_fullsized":"2023-11-06T22:18:22.864039+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.378713+0000","last_change":"2023-11-06T22:18:22.374807+0000","last_active":"2023-11-06T22:18:22.378713+0000","last_peered":"2023-11-06T22:18:22.378713+0000","last_clean":"2023-11-06T22:18:22.378713+0000","last_became_active":"2023-11-06T22:18:22.373936+0000","last_became_peered":"2023-11-06T22:18:22.373936+0000","last_unstale":"2023-11-06T22:18:22.378713+0000","last_undegraded":"2023-11-06T22:18:22.378713+0000","last_fullsized":"2023-11-06T22:18:22.378713+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868783+0000","last_change":"2023-11-06T22:18:22.867080+0000","last_active":"2023-11-06T22:18:22.868783+0000","last_peered":"2023-11-06T22:18:22.868783+0000","last_clean":"2023-11-06T22:18:22.868783+0000","last_became_active":"2023-11-06T22:18:22.866775+0000","last_became_peered":"2023-11-06T22:18:22.866775+0000","last_unstale":"2023-11-06T22:18:22.868783+0000","last_undegraded":"2023-11-06T22:18:22.868783+0000","last_fullsized":"2023-11-06T22:18:22.868783+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.851706+0000","last_change":"2023-11-06T22:18:22.377422+0000","last_active":"2023-11-06T22:18:22.851706+0000","last_peered":"2023-11-06T22:18:22.851706+0000","last_clean":"2023-11-06T22:18:22.851706+0000","last_became_active":"2023-11-06T22:18:22.376164+0000","last_became_peered":"2023-11-06T22:18:22.376164+0000","last_unstale":"2023-11-06T22:18:22.851706+0000","last_undegraded":"2023-11-06T22:18:22.851706+0000","last_fullsized":"2023-11-06T22:18:22.851706+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.872115+0000","last_change":"2023-11-06T22:18:22.861982+0000","last_active":"2023-11-06T22:18:22.872115+0000","last_peered":"2023-11-06T22:18:22.872115+0000","last_clean":"2023-11-06T22:18:22.872115+0000","last_became_active":"2023-11-06T22:18:22.861368+0000","last_became_peered":"2023-11-06T22:18:22.861368+0000","last_unstale":"2023-11-06T22:18:22.872115+0000","last_undegraded":"2023-11-06T22:18:22.872115+0000","last_fullsized":"2023-11-06T22:18:22.872115+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.845637+0000","last_change":"2023-11-06T22:18:22.373932+0000","last_active":"2023-11-06T22:18:22.845637+0000","last_peered":"2023-11-06T22:18:22.845637+0000","last_clean":"2023-11-06T22:18:22.845637+0000","last_became_active":"2023-11-06T22:18:22.373620+0000","last_became_peered":"2023-11-06T22:18:22.373620+0000","last_unstale":"2023-11-06T22:18:22.845637+0000","last_undegraded":"2023-11-06T22:18:22.845637+0000","last_fullsized":"2023-11-06T22:18:22.845637+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.875315+0000","last_change":"2023-11-06T22:18:22.873133+0000","last_active":"2023-11-06T22:18:22.875315+0000","last_peered":"2023-11-06T22:18:22.875315+0000","last_clean":"2023-11-06T22:18:22.875315+0000","last_became_active":"2023-11-06T22:18:22.872788+0000","last_became_peered":"2023-11-06T22:18:22.872788+0000","last_unstale":"2023-11-06T22:18:22.875315+0000","last_undegraded":"2023-11-06T22:18:22.875315+0000","last_fullsized":"2023-11-06T22:18:22.875315+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.847946+0000","last_change":"2023-11-06T22:18:22.371486+0000","last_active":"2023-11-06T22:18:22.847946+0000","last_peered":"2023-11-06T22:18:22.847946+0000","last_clean":"2023-11-06T22:18:22.847946+0000","last_became_active":"2023-11-06T22:18:22.371115+0000","last_became_peered":"2023-11-06T22:18:22.371115+0000","last_unstale":"2023-11-06T22:18:22.847946+0000","last_undegraded":"2023-11-06T22:18:22.847946+0000","last_fullsized":"2023-11-06T22:18:22.847946+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.874458+0000","last_change":"2023-11-06T22:18:22.871864+0000","last_active":"2023-11-06T22:18:22.874458+0000","last_peered":"2023-11-06T22:18:22.874458+0000","last_clean":"2023-11-06T22:18:22.874458+0000","last_became_active":"2023-11-06T22:18:22.871514+0000","last_became_peered":"2023-11-06T22:18:22.871514+0000","last_unstale":"2023-11-06T22:18:22.874458+0000","last_undegraded":"2023-11-06T22:18:22.874458+0000","last_fullsized":"2023-11-06T22:18:22.874458+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860629+0000","last_change":"2023-11-06T22:18:22.370501+0000","last_active":"2023-11-06T22:18:22.860629+0000","last_peered":"2023-11-06T22:18:22.860629+0000","last_clean":"2023-11-06T22:18:22.860629+0000","last_became_active":"2023-11-06T22:18:22.370072+0000","last_became_peered":"2023-11-06T22:18:22.370072+0000","last_unstale":"2023-11-06T22:18:22.860629+0000","last_undegraded":"2023-11-06T22:18:22.860629+0000","last_fullsized":"2023-11-06T22:18:22.860629+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871349+0000","last_change":"2023-11-06T22:18:22.866083+0000","last_active":"2023-11-06T22:18:22.871349+0000","last_peered":"2023-11-06T22:18:22.871349+0000","last_clean":"2023-11-06T22:18:22.871349+0000","last_became_active":"2023-11-06T22:18:22.865704+0000","last_became_peered":"2023-11-06T22:18:22.865704+0000","last_unstale":"2023-11-06T22:18:22.871349+0000","last_undegraded":"2023-11-06T22:18:22.871349+0000","last_fullsized":"2023-11-06T22:18:22.871349+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860430+0000","last_change":"2023-11-06T22:18:22.382428+0000","last_active":"2023-11-06T22:18:22.860430+0000","last_peered":"2023-11-06T22:18:22.860430+0000","last_clean":"2023-11-06T22:18:22.860430+0000","last_became_active":"2023-11-06T22:18:22.382140+0000","last_became_peered":"2023-11-06T22:18:22.382140+0000","last_unstale":"2023-11-06T22:18:22.860430+0000","last_undegraded":"2023-11-06T22:18:22.860430+0000","last_fullsized":"2023-11-06T22:18:22.860430+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871378+0000","last_change":"2023-11-06T22:18:22.865207+0000","last_active":"2023-11-06T22:18:22.871378+0000","last_peered":"2023-11-06T22:18:22.871378+0000","last_clean":"2023-11-06T22:18:22.871378+0000","last_became_active":"2023-11-06T22:18:22.863071+0000","last_became_peered":"2023-11-06T22:18:22.863071+0000","last_unstale":"2023-11-06T22:18:22.871378+0000","last_undegraded":"2023-11-06T22:18:22.871378+0000","last_fullsized":"2023-11-06T22:18:22.871378+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.851391+0000","last_change":"2023-11-06T22:18:22.376697+0000","last_active":"2023-11-06T22:18:22.851391+0000","last_peered":"2023-11-06T22:18:22.851391+0000","last_clean":"2023-11-06T22:18:22.851391+0000","last_became_active":"2023-11-06T22:18:22.376410+0000","last_became_peered":"2023-11-06T22:18:22.376410+0000","last_unstale":"2023-11-06T22:18:22.851391+0000","last_undegraded":"2023-11-06T22:18:22.851391+0000","last_fullsized":"2023-11-06T22:18:22.851391+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868814+0000","last_change":"2023-11-06T22:18:22.867323+0000","last_active":"2023-11-06T22:18:22.868814+0000","last_peered":"2023-11-06T22:18:22.868814+0000","last_clean":"2023-11-06T22:18:22.868814+0000","last_became_active":"2023-11-06T22:18:22.866882+0000","last_became_peered":"2023-11-06T22:18:22.866882+0000","last_unstale":"2023-11-06T22:18:22.868814+0000","last_undegraded":"2023-11-06T22:18:22.868814+0000","last_fullsized":"2023-11-06T22:18:22.868814+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.386488+0000","last_change":"2023-11-06T22:18:22.381100+0000","last_active":"2023-11-06T22:18:22.386488+0000","last_peered":"2023-11-06T22:18:22.386488+0000","last_clean":"2023-11-06T22:18:22.386488+0000","last_became_active":"2023-11-06T22:18:22.379002+0000","last_became_peered":"2023-11-06T22:18:22.379002+0000","last_unstale":"2023-11-06T22:18:22.386488+0000","last_undegraded":"2023-11-06T22:18:22.386488+0000","last_fullsized":"2023-11-06T22:18:22.386488+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.859108+0000","last_change":"2023-11-06T22:18:22.859108+0000","last_active":"2023-11-06T22:18:22.859108+0000","last_peered":"2023-11-06T22:18:22.859108+0000","last_clean":"2023-11-06T22:18:22.859108+0000","last_became_active":"2023-11-06T22:18:22.858510+0000","last_became_peered":"2023-11-06T22:18:22.858510+0000","last_unstale":"2023-11-06T22:18:22.859108+0000","last_undegraded":"2023-11-06T22:18:22.859108+0000","last_fullsized":"2023-11-06T22:18:22.859108+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"222'3","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.873629+0000","last_change":"2023-11-06T22:18:22.374165+0000","last_active":"2023-11-06T22:18:22.873629+0000","last_peered":"2023-11-06T22:18:22.873629+0000","last_clean":"2023-11-06T22:18:22.873629+0000","last_became_active":"2023-11-06T22:18:22.373881+0000","last_became_peered":"2023-11-06T22:18:22.373881+0000","last_unstale":"2023-11-06T22:18:22.873629+0000","last_undegraded":"2023-11-06T22:18:22.873629+0000","last_fullsized":"2023-11-06T22:18:22.873629+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.880528+0000","last_change":"2023-11-06T22:18:22.879042+0000","last_active":"2023-11-06T22:18:22.880528+0000","last_peered":"2023-11-06T22:18:22.880528+0000","last_clean":"2023-11-06T22:18:22.880528+0000","last_became_active":"2023-11-06T22:18:22.878492+0000","last_became_peered":"2023-11-06T22:18:22.878492+0000","last_unstale":"2023-11-06T22:18:22.880528+0000","last_undegraded":"2023-11-06T22:18:22.880528+0000","last_fullsized":"2023-11-06T22:18:22.880528+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.14","version":"222'2","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.872617+0000","last_change":"2023-11-06T22:18:22.370708+0000","last_active":"2023-11-06T22:18:22.872617+0000","last_peered":"2023-11-06T22:18:22.872617+0000","last_clean":"2023-11-06T22:18:22.872617+0000","last_became_active":"2023-11-06T22:18:22.370429+0000","last_became_peered":"2023-11-06T22:18:22.370429+0000","last_unstale":"2023-11-06T22:18:22.872617+0000","last_undegraded":"2023-11-06T22:18:22.872617+0000","last_fullsized":"2023-11-06T22:18:22.872617+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.867777+0000","last_change":"2023-11-06T22:18:22.863683+0000","last_active":"2023-11-06T22:18:22.867777+0000","last_peered":"2023-11-06T22:18:22.867777+0000","last_clean":"2023-11-06T22:18:22.867777+0000","last_became_active":"2023-11-06T22:18:22.863321+0000","last_became_peered":"2023-11-06T22:18:22.863321+0000","last_unstale":"2023-11-06T22:18:22.867777+0000","last_undegraded":"2023-11-06T22:18:22.867777+0000","last_fullsized":"2023-11-06T22:18:22.867777+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.13","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.872041+0000","last_change":"2023-11-06T22:18:22.372773+0000","last_active":"2023-11-06T22:18:22.872041+0000","last_peered":"2023-11-06T22:18:22.872041+0000","last_clean":"2023-11-06T22:18:22.872041+0000","last_became_active":"2023-11-06T22:18:22.372494+0000","last_became_peered":"2023-11-06T22:18:22.372494+0000","last_unstale":"2023-11-06T22:18:22.872041+0000","last_undegraded":"2023-11-06T22:18:22.872041+0000","last_fullsized":"2023-11-06T22:18:22.872041+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868713+0000","last_change":"2023-11-06T22:18:22.863563+0000","last_active":"2023-11-06T22:18:22.868713+0000","last_peered":"2023-11-06T22:18:22.868713+0000","last_clean":"2023-11-06T22:18:22.868713+0000","last_became_active":"2023-11-06T22:18:22.863256+0000","last_became_peered":"2023-11-06T22:18:22.863256+0000","last_unstale":"2023-11-06T22:18:22.868713+0000","last_undegraded":"2023-11-06T22:18:22.868713+0000","last_fullsized":"2023-11-06T22:18:22.868713+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.850088+0000","last_change":"2023-11-06T22:18:22.372728+0000","last_active":"2023-11-06T22:18:22.850088+0000","last_peered":"2023-11-06T22:18:22.850088+0000","last_clean":"2023-11-06T22:18:22.850088+0000","last_became_active":"2023-11-06T22:18:22.372410+0000","last_became_peered":"2023-11-06T22:18:22.372410+0000","last_unstale":"2023-11-06T22:18:22.850088+0000","last_undegraded":"2023-11-06T22:18:22.850088+0000","last_fullsized":"2023-11-06T22:18:22.850088+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878712+0000","last_change":"2023-11-06T22:18:22.873073+0000","last_active":"2023-11-06T22:18:22.878712+0000","last_peered":"2023-11-06T22:18:22.878712+0000","last_clean":"2023-11-06T22:18:22.878712+0000","last_became_active":"2023-11-06T22:18:22.872754+0000","last_became_peered":"2023-11-06T22:18:22.872754+0000","last_unstale":"2023-11-06T22:18:22.878712+0000","last_undegraded":"2023-11-06T22:18:22.878712+0000","last_fullsized":"2023-11-06T22:18:22.878712+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860756+0000","last_change":"2023-11-06T22:18:22.373892+0000","last_active":"2023-11-06T22:18:22.860756+0000","last_peered":"2023-11-06T22:18:22.860756+0000","last_clean":"2023-11-06T22:18:22.860756+0000","last_became_active":"2023-11-06T22:18:22.373449+0000","last_became_peered":"2023-11-06T22:18:22.373449+0000","last_unstale":"2023-11-06T22:18:22.860756+0000","last_undegraded":"2023-11-06T22:18:22.860756+0000","last_fullsized":"2023-11-06T22:18:22.860756+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871381+0000","last_change":"2023-11-06T22:18:22.866172+0000","last_active":"2023-11-06T22:18:22.871381+0000","last_peered":"2023-11-06T22:18:22.871381+0000","last_clean":"2023-11-06T22:18:22.871381+0000","last_became_active":"2023-11-06T22:18:22.865814+0000","last_became_peered":"2023-11-06T22:18:22.865814+0000","last_unstale":"2023-11-06T22:18:22.871381+0000","last_undegraded":"2023-11-06T22:18:22.871381+0000","last_fullsized":"2023-11-06T22:18:22.871381+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"222'4","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866319+0000","last_change":"2023-11-06T22:18:22.383033+0000","last_active":"2023-11-06T22:18:22.866319+0000","last_peered":"2023-11-06T22:18:22.866319+0000","last_clean":"2023-11-06T22:18:22.866319+0000","last_became_active":"2023-11-06T22:18:22.382689+0000","last_became_peered":"2023-11-06T22:18:22.382689+0000","last_unstale":"2023-11-06T22:18:22.866319+0000","last_undegraded":"2023-11-06T22:18:22.866319+0000","last_fullsized":"2023-11-06T22:18:22.866319+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862873+0000","last_change":"2023-11-06T22:18:22.862873+0000","last_active":"2023-11-06T22:18:22.862873+0000","last_peered":"2023-11-06T22:18:22.862873+0000","last_clean":"2023-11-06T22:18:22.862873+0000","last_became_active":"2023-11-06T22:18:22.862537+0000","last_became_peered":"2023-11-06T22:18:22.862537+0000","last_unstale":"2023-11-06T22:18:22.862873+0000","last_undegraded":"2023-11-06T22:18:22.862873+0000","last_fullsized":"2023-11-06T22:18:22.862873+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.f","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871524+0000","last_change":"2023-11-06T22:18:22.373469+0000","last_active":"2023-11-06T22:18:22.871524+0000","last_peered":"2023-11-06T22:18:22.871524+0000","last_clean":"2023-11-06T22:18:22.871524+0000","last_became_active":"2023-11-06T22:18:22.373033+0000","last_became_peered":"2023-11-06T22:18:22.373033+0000","last_unstale":"2023-11-06T22:18:22.871524+0000","last_undegraded":"2023-11-06T22:18:22.871524+0000","last_fullsized":"2023-11-06T22:18:22.871524+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.875336+0000","last_change":"2023-11-06T22:18:22.867949+0000","last_active":"2023-11-06T22:18:22.875336+0000","last_peered":"2023-11-06T22:18:22.875336+0000","last_clean":"2023-11-06T22:18:22.875336+0000","last_became_active":"2023-11-06T22:18:22.867650+0000","last_became_peered":"2023-11-06T22:18:22.867650+0000","last_unstale":"2023-11-06T22:18:22.875336+0000","last_undegraded":"2023-11-06T22:18:22.875336+0000","last_fullsized":"2023-11-06T22:18:22.875336+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.e","version":"222'2","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860994+0000","last_change":"2023-11-06T22:18:22.374697+0000","last_active":"2023-11-06T22:18:22.860994+0000","last_peered":"2023-11-06T22:18:22.860994+0000","last_clean":"2023-11-06T22:18:22.860994+0000","last_became_active":"2023-11-06T22:18:22.374438+0000","last_became_peered":"2023-11-06T22:18:22.374438+0000","last_unstale":"2023-11-06T22:18:22.860994+0000","last_undegraded":"2023-11-06T22:18:22.860994+0000","last_fullsized":"2023-11-06T22:18:22.860994+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.872221+0000","last_change":"2023-11-06T22:18:22.867285+0000","last_active":"2023-11-06T22:18:22.872221+0000","last_peered":"2023-11-06T22:18:22.872221+0000","last_clean":"2023-11-06T22:18:22.872221+0000","last_became_active":"2023-11-06T22:18:22.867031+0000","last_became_peered":"2023-11-06T22:18:22.867031+0000","last_unstale":"2023-11-06T22:18:22.872221+0000","last_undegraded":"2023-11-06T22:18:22.872221+0000","last_fullsized":"2023-11-06T22:18:22.872221+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.845737+0000","last_change":"2023-11-06T22:18:22.371970+0000","last_active":"2023-11-06T22:18:22.845737+0000","last_peered":"2023-11-06T22:18:22.845737+0000","last_clean":"2023-11-06T22:18:22.845737+0000","last_became_active":"2023-11-06T22:18:22.371625+0000","last_became_peered":"2023-11-06T22:18:22.371625+0000","last_unstale":"2023-11-06T22:18:22.845737+0000","last_undegraded":"2023-11-06T22:18:22.845737+0000","last_fullsized":"2023-11-06T22:18:22.845737+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.874007+0000","last_change":"2023-11-06T22:18:22.874007+0000","last_active":"2023-11-06T22:18:22.874007+0000","last_peered":"2023-11-06T22:18:22.874007+0000","last_clean":"2023-11-06T22:18:22.874007+0000","last_became_active":"2023-11-06T22:18:22.873709+0000","last_became_peered":"2023-11-06T22:18:22.873709+0000","last_unstale":"2023-11-06T22:18:22.874007+0000","last_undegraded":"2023-11-06T22:18:22.874007+0000","last_fullsized":"2023-11-06T22:18:22.874007+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.381620+0000","last_change":"2023-11-06T22:18:22.376402+0000","last_active":"2023-11-06T22:18:22.381620+0000","last_peered":"2023-11-06T22:18:22.381620+0000","last_clean":"2023-11-06T22:18:22.381620+0000","last_became_active":"2023-11-06T22:18:22.375761+0000","last_became_peered":"2023-11-06T22:18:22.375761+0000","last_unstale":"2023-11-06T22:18:22.381620+0000","last_undegraded":"2023-11-06T22:18:22.381620+0000","last_fullsized":"2023-11-06T22:18:22.381620+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.870465+0000","last_change":"2023-11-06T22:18:22.867925+0000","last_active":"2023-11-06T22:18:22.870465+0000","last_peered":"2023-11-06T22:18:22.870465+0000","last_clean":"2023-11-06T22:18:22.870465+0000","last_became_active":"2023-11-06T22:18:22.866848+0000","last_became_peered":"2023-11-06T22:18:22.866848+0000","last_unstale":"2023-11-06T22:18:22.870465+0000","last_undegraded":"2023-11-06T22:18:22.870465+0000","last_fullsized":"2023-11-06T22:18:22.870465+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.b","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.870137+0000","last_change":"2023-11-06T22:18:22.377476+0000","last_active":"2023-11-06T22:18:22.870137+0000","last_peered":"2023-11-06T22:18:22.870137+0000","last_clean":"2023-11-06T22:18:22.870137+0000","last_became_active":"2023-11-06T22:18:22.376175+0000","last_became_peered":"2023-11-06T22:18:22.376175+0000","last_unstale":"2023-11-06T22:18:22.870137+0000","last_undegraded":"2023-11-06T22:18:22.870137+0000","last_fullsized":"2023-11-06T22:18:22.870137+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.879191+0000","last_change":"2023-11-06T22:18:22.875569+0000","last_active":"2023-11-06T22:18:22.879191+0000","last_peered":"2023-11-06T22:18:22.879191+0000","last_clean":"2023-11-06T22:18:22.879191+0000","last_became_active":"2023-11-06T22:18:22.875145+0000","last_became_peered":"2023-11-06T22:18:22.875145+0000","last_unstale":"2023-11-06T22:18:22.879191+0000","last_undegraded":"2023-11-06T22:18:22.879191+0000","last_fullsized":"2023-11-06T22:18:22.879191+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.855701+0000","last_change":"2023-11-06T22:18:22.372366+0000","last_active":"2023-11-06T22:18:22.855701+0000","last_peered":"2023-11-06T22:18:22.855701+0000","last_clean":"2023-11-06T22:18:22.855701+0000","last_became_active":"2023-11-06T22:18:22.372076+0000","last_became_peered":"2023-11-06T22:18:22.372076+0000","last_unstale":"2023-11-06T22:18:22.855701+0000","last_undegraded":"2023-11-06T22:18:22.855701+0000","last_fullsized":"2023-11-06T22:18:22.855701+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878513+0000","last_change":"2023-11-06T22:18:22.876274+0000","last_active":"2023-11-06T22:18:22.878513+0000","last_peered":"2023-11-06T22:18:22.878513+0000","last_clean":"2023-11-06T22:18:22.878513+0000","last_became_active":"2023-11-06T22:18:22.875579+0000","last_became_peered":"2023-11-06T22:18:22.875579+0000","last_unstale":"2023-11-06T22:18:22.878513+0000","last_undegraded":"2023-11-06T22:18:22.878513+0000","last_fullsized":"2023-11-06T22:18:22.878513+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.852036+0000","last_change":"2023-11-06T22:18:22.372478+0000","last_active":"2023-11-06T22:18:22.852036+0000","last_peered":"2023-11-06T22:18:22.852036+0000","last_clean":"2023-11-06T22:18:22.852036+0000","last_became_active":"2023-11-06T22:18:22.372198+0000","last_became_peered":"2023-11-06T22:18:22.372198+0000","last_unstale":"2023-11-06T22:18:22.852036+0000","last_undegraded":"2023-11-06T22:18:22.852036+0000","last_fullsized":"2023-11-06T22:18:22.852036+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.873335+0000","last_change":"2023-11-06T22:18:22.864459+0000","last_active":"2023-11-06T22:18:22.873335+0000","last_peered":"2023-11-06T22:18:22.873335+0000","last_clean":"2023-11-06T22:18:22.873335+0000","last_became_active":"2023-11-06T22:18:22.863876+0000","last_became_peered":"2023-11-06T22:18:22.863876+0000","last_unstale":"2023-11-06T22:18:22.873335+0000","last_undegraded":"2023-11-06T22:18:22.873335+0000","last_fullsized":"2023-11-06T22:18:22.873335+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840609+0000","last_change":"2023-11-06T22:18:22.369598+0000","last_active":"2023-11-06T22:18:22.840609+0000","last_peered":"2023-11-06T22:18:22.840609+0000","last_clean":"2023-11-06T22:18:22.840609+0000","last_became_active":"2023-11-06T22:18:22.367779+0000","last_became_peered":"2023-11-06T22:18:22.367779+0000","last_unstale":"2023-11-06T22:18:22.840609+0000","last_undegraded":"2023-11-06T22:18:22.840609+0000","last_fullsized":"2023-11-06T22:18:22.840609+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.849570+0000","last_change":"2023-11-06T22:18:22.849570+0000","last_active":"2023-11-06T22:18:22.849570+0000","last_peered":"2023-11-06T22:18:22.849570+0000","last_clean":"2023-11-06T22:18:22.849570+0000","last_became_active":"2023-11-06T22:18:22.849277+0000","last_became_peered":"2023-11-06T22:18:22.849277+0000","last_unstale":"2023-11-06T22:18:22.849570+0000","last_undegraded":"2023-11-06T22:18:22.849570+0000","last_fullsized":"2023-11-06T22:18:22.849570+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.7","version":"222'2","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.868972+0000","last_change":"2023-11-06T22:18:22.370733+0000","last_active":"2023-11-06T22:18:22.868972+0000","last_peered":"2023-11-06T22:18:22.868972+0000","last_clean":"2023-11-06T22:18:22.868972+0000","last_became_active":"2023-11-06T22:18:22.367912+0000","last_became_peered":"2023-11-06T22:18:22.367912+0000","last_unstale":"2023-11-06T22:18:22.868972+0000","last_undegraded":"2023-11-06T22:18:22.868972+0000","last_fullsized":"2023-11-06T22:18:22.868972+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871660+0000","last_change":"2023-11-06T22:18:22.871660+0000","last_active":"2023-11-06T22:18:22.871660+0000","last_peered":"2023-11-06T22:18:22.871660+0000","last_clean":"2023-11-06T22:18:22.871660+0000","last_became_active":"2023-11-06T22:18:22.871386+0000","last_became_peered":"2023-11-06T22:18:22.871386+0000","last_unstale":"2023-11-06T22:18:22.871660+0000","last_undegraded":"2023-11-06T22:18:22.871660+0000","last_fullsized":"2023-11-06T22:18:22.871660+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.386507+0000","last_change":"2023-11-06T22:18:22.381263+0000","last_active":"2023-11-06T22:18:22.386507+0000","last_peered":"2023-11-06T22:18:22.386507+0000","last_clean":"2023-11-06T22:18:22.386507+0000","last_became_active":"2023-11-06T22:18:22.379493+0000","last_became_peered":"2023-11-06T22:18:22.379493+0000","last_unstale":"2023-11-06T22:18:22.386507+0000","last_undegraded":"2023-11-06T22:18:22.386507+0000","last_fullsized":"2023-11-06T22:18:22.386507+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.871301+0000","last_change":"2023-11-06T22:18:22.867675+0000","last_active":"2023-11-06T22:18:22.871301+0000","last_peered":"2023-11-06T22:18:22.871301+0000","last_clean":"2023-11-06T22:18:22.871301+0000","last_became_active":"2023-11-06T22:18:22.867370+0000","last_became_peered":"2023-11-06T22:18:22.867370+0000","last_unstale":"2023-11-06T22:18:22.871301+0000","last_undegraded":"2023-11-06T22:18:22.871301+0000","last_fullsized":"2023-11-06T22:18:22.871301+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862448+0000","last_change":"2023-11-06T22:18:22.380369+0000","last_active":"2023-11-06T22:18:22.862448+0000","last_peered":"2023-11-06T22:18:22.862448+0000","last_clean":"2023-11-06T22:18:22.862448+0000","last_became_active":"2023-11-06T22:18:22.380041+0000","last_became_peered":"2023-11-06T22:18:22.380041+0000","last_unstale":"2023-11-06T22:18:22.862448+0000","last_undegraded":"2023-11-06T22:18:22.862448+0000","last_fullsized":"2023-11-06T22:18:22.862448+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878418+0000","last_change":"2023-11-06T22:18:22.873725+0000","last_active":"2023-11-06T22:18:22.878418+0000","last_peered":"2023-11-06T22:18:22.878418+0000","last_clean":"2023-11-06T22:18:22.878418+0000","last_became_active":"2023-11-06T22:18:22.871619+0000","last_became_peered":"2023-11-06T22:18:22.871619+0000","last_unstale":"2023-11-06T22:18:22.878418+0000","last_undegraded":"2023-11-06T22:18:22.878418+0000","last_fullsized":"2023-11-06T22:18:22.878418+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.369329+0000","last_change":"2023-11-06T22:18:22.360996+0000","last_active":"2023-11-06T22:18:22.369329+0000","last_peered":"2023-11-06T22:18:22.369329+0000","last_clean":"2023-11-06T22:18:22.369329+0000","last_became_active":"2023-11-06T22:18:22.359436+0000","last_became_peered":"2023-11-06T22:18:22.359436+0000","last_unstale":"2023-11-06T22:18:22.369329+0000","last_undegraded":"2023-11-06T22:18:22.369329+0000","last_fullsized":"2023-11-06T22:18:22.369329+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.864911+0000","last_change":"2023-11-06T22:18:22.864911+0000","last_active":"2023-11-06T22:18:22.864911+0000","last_peered":"2023-11-06T22:18:22.864911+0000","last_clean":"2023-11-06T22:18:22.864911+0000","last_became_active":"2023-11-06T22:18:22.864056+0000","last_became_peered":"2023-11-06T22:18:22.864056+0000","last_unstale":"2023-11-06T22:18:22.864911+0000","last_undegraded":"2023-11-06T22:18:22.864911+0000","last_fullsized":"2023-11-06T22:18:22.864911+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.842005+0000","last_change":"2023-11-06T22:18:22.361864+0000","last_active":"2023-11-06T22:18:22.842005+0000","last_peered":"2023-11-06T22:18:22.842005+0000","last_clean":"2023-11-06T22:18:22.842005+0000","last_became_active":"2023-11-06T22:18:22.361581+0000","last_became_peered":"2023-11-06T22:18:22.361581+0000","last_unstale":"2023-11-06T22:18:22.842005+0000","last_undegraded":"2023-11-06T22:18:22.842005+0000","last_fullsized":"2023-11-06T22:18:22.842005+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862008+0000","last_change":"2023-11-06T22:18:22.856002+0000","last_active":"2023-11-06T22:18:22.862008+0000","last_peered":"2023-11-06T22:18:22.862008+0000","last_clean":"2023-11-06T22:18:22.862008+0000","last_became_active":"2023-11-06T22:18:22.855710+0000","last_became_peered":"2023-11-06T22:18:22.855710+0000","last_unstale":"2023-11-06T22:18:22.862008+0000","last_undegraded":"2023-11-06T22:18:22.862008+0000","last_fullsized":"2023-11-06T22:18:22.862008+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.382450+0000","last_change":"2023-11-06T22:18:22.375257+0000","last_active":"2023-11-06T22:18:22.382450+0000","last_peered":"2023-11-06T22:18:22.382450+0000","last_clean":"2023-11-06T22:18:22.382450+0000","last_became_active":"2023-11-06T22:18:22.374635+0000","last_became_peered":"2023-11-06T22:18:22.374635+0000","last_unstale":"2023-11-06T22:18:22.382450+0000","last_undegraded":"2023-11-06T22:18:22.382450+0000","last_fullsized":"2023-11-06T22:18:22.382450+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.880301+0000","last_change":"2023-11-06T22:18:22.878630+0000","last_active":"2023-11-06T22:18:22.880301+0000","last_peered":"2023-11-06T22:18:22.880301+0000","last_clean":"2023-11-06T22:18:22.880301+0000","last_became_active":"2023-11-06T22:18:22.878087+0000","last_became_peered":"2023-11-06T22:18:22.878087+0000","last_unstale":"2023-11-06T22:18:22.880301+0000","last_undegraded":"2023-11-06T22:18:22.880301+0000","last_fullsized":"2023-11-06T22:18:22.880301+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860786+0000","last_change":"2023-11-06T22:18:22.367305+0000","last_active":"2023-11-06T22:18:22.860786+0000","last_peered":"2023-11-06T22:18:22.860786+0000","last_clean":"2023-11-06T22:18:22.860786+0000","last_became_active":"2023-11-06T22:18:22.367027+0000","last_became_peered":"2023-11-06T22:18:22.367027+0000","last_unstale":"2023-11-06T22:18:22.860786+0000","last_undegraded":"2023-11-06T22:18:22.860786+0000","last_fullsized":"2023-11-06T22:18:22.860786+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.864753+0000","last_change":"2023-11-06T22:18:22.864753+0000","last_active":"2023-11-06T22:18:22.864753+0000","last_peered":"2023-11-06T22:18:22.864753+0000","last_clean":"2023-11-06T22:18:22.864753+0000","last_became_active":"2023-11-06T22:18:22.863702+0000","last_became_peered":"2023-11-06T22:18:22.863702+0000","last_unstale":"2023-11-06T22:18:22.864753+0000","last_undegraded":"2023-11-06T22:18:22.864753+0000","last_fullsized":"2023-11-06T22:18:22.864753+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.383386+0000","last_change":"2023-11-06T22:18:22.378850+0000","last_active":"2023-11-06T22:18:22.383386+0000","last_peered":"2023-11-06T22:18:22.383386+0000","last_clean":"2023-11-06T22:18:22.383386+0000","last_became_active":"2023-11-06T22:18:22.378453+0000","last_became_peered":"2023-11-06T22:18:22.378453+0000","last_unstale":"2023-11-06T22:18:22.383386+0000","last_undegraded":"2023-11-06T22:18:22.383386+0000","last_fullsized":"2023-11-06T22:18:22.383386+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862302+0000","last_change":"2023-11-06T22:18:22.862302+0000","last_active":"2023-11-06T22:18:22.862302+0000","last_peered":"2023-11-06T22:18:22.862302+0000","last_clean":"2023-11-06T22:18:22.862302+0000","last_became_active":"2023-11-06T22:18:22.861817+0000","last_became_peered":"2023-11-06T22:18:22.861817+0000","last_unstale":"2023-11-06T22:18:22.862302+0000","last_undegraded":"2023-11-06T22:18:22.862302+0000","last_fullsized":"2023-11-06T22:18:22.862302+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1f","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.875842+0000","last_change":"2023-11-06T22:18:22.373358+0000","last_active":"2023-11-06T22:18:22.875842+0000","last_peered":"2023-11-06T22:18:22.875842+0000","last_clean":"2023-11-06T22:18:22.875842+0000","last_became_active":"2023-11-06T22:18:22.372926+0000","last_became_peered":"2023-11-06T22:18:22.372926+0000","last_unstale":"2023-11-06T22:18:22.875842+0000","last_undegraded":"2023-11-06T22:18:22.875842+0000","last_fullsized":"2023-11-06T22:18:22.875842+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":188,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.349951+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T22:18:22.349951+0000","last_peered":"2023-11-06T22:18:22.349951+0000","last_clean":"2023-11-06T22:18:22.349951+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:18:22.349951+0000","last_undegraded":"2023-11-06T22:18:22.349951+0000","last_fullsized":"2023-11-06T22:18:22.349951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.858259+0000","last_change":"2023-11-06T22:18:22.858259+0000","last_active":"2023-11-06T22:18:22.858259+0000","last_peered":"2023-11-06T22:18:22.858259+0000","last_clean":"2023-11-06T22:18:22.858259+0000","last_became_active":"2023-11-06T22:18:22.857808+0000","last_became_peered":"2023-11-06T22:18:22.857808+0000","last_unstale":"2023-11-06T22:18:22.858259+0000","last_undegraded":"2023-11-06T22:18:22.858259+0000","last_fullsized":"2023-11-06T22:18:22.858259+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.841054+0000","last_change":"2023-11-06T22:18:22.367265+0000","last_active":"2023-11-06T22:18:22.841054+0000","last_peered":"2023-11-06T22:18:22.841054+0000","last_clean":"2023-11-06T22:18:22.841054+0000","last_became_active":"2023-11-06T22:18:22.366914+0000","last_became_peered":"2023-11-06T22:18:22.366914+0000","last_unstale":"2023-11-06T22:18:22.841054+0000","last_undegraded":"2023-11-06T22:18:22.841054+0000","last_fullsized":"2023-11-06T22:18:22.841054+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":202,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.841522+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T22:18:22.841522+0000","last_peered":"2023-11-06T22:18:22.841522+0000","last_clean":"2023-11-06T22:18:22.841522+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:18:22.841522+0000","last_undegraded":"2023-11-06T22:18:22.841522+0000","last_fullsized":"2023-11-06T22:18:22.841522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.876152+0000","last_change":"2023-11-06T22:18:22.870963+0000","last_active":"2023-11-06T22:18:22.876152+0000","last_peered":"2023-11-06T22:18:22.876152+0000","last_clean":"2023-11-06T22:18:22.876152+0000","last_became_active":"2023-11-06T22:18:22.870685+0000","last_became_peered":"2023-11-06T22:18:22.870685+0000","last_unstale":"2023-11-06T22:18:22.876152+0000","last_undegraded":"2023-11-06T22:18:22.876152+0000","last_fullsized":"2023-11-06T22:18:22.876152+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.855462+0000","last_change":"2023-11-06T22:18:22.367095+0000","last_active":"2023-11-06T22:18:22.855462+0000","last_peered":"2023-11-06T22:18:22.855462+0000","last_clean":"2023-11-06T22:18:22.855462+0000","last_became_active":"2023-11-06T22:18:22.366804+0000","last_became_peered":"2023-11-06T22:18:22.366804+0000","last_unstale":"2023-11-06T22:18:22.855462+0000","last_undegraded":"2023-11-06T22:18:22.855462+0000","last_fullsized":"2023-11-06T22:18:22.855462+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":211,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.860847+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T22:18:22.860847+0000","last_peered":"2023-11-06T22:18:22.860847+0000","last_clean":"2023-11-06T22:18:22.860847+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:18:22.860847+0000","last_undegraded":"2023-11-06T22:18:22.860847+0000","last_fullsized":"2023-11-06T22:18:22.860847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866442+0000","last_change":"2023-11-06T22:18:22.863959+0000","last_active":"2023-11-06T22:18:22.866442+0000","last_peered":"2023-11-06T22:18:22.866442+0000","last_clean":"2023-11-06T22:18:22.866442+0000","last_became_active":"2023-11-06T22:18:22.863660+0000","last_became_peered":"2023-11-06T22:18:22.863660+0000","last_unstale":"2023-11-06T22:18:22.866442+0000","last_undegraded":"2023-11-06T22:18:22.866442+0000","last_fullsized":"2023-11-06T22:18:22.866442+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.857909+0000","last_change":"2023-11-06T22:18:22.370944+0000","last_active":"2023-11-06T22:18:22.857909+0000","last_peered":"2023-11-06T22:18:22.857909+0000","last_clean":"2023-11-06T22:18:22.857909+0000","last_became_active":"2023-11-06T22:18:22.370561+0000","last_became_peered":"2023-11-06T22:18:22.370561+0000","last_unstale":"2023-11-06T22:18:22.857909+0000","last_undegraded":"2023-11-06T22:18:22.857909+0000","last_fullsized":"2023-11-06T22:18:22.857909+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":202,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.842365+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T22:18:22.842365+0000","last_peered":"2023-11-06T22:18:22.842365+0000","last_clean":"2023-11-06T22:18:22.842365+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:18:22.842365+0000","last_undegraded":"2023-11-06T22:18:22.842365+0000","last_fullsized":"2023-11-06T22:18:22.842365+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":187,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.360248+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T22:18:22.360248+0000","last_peered":"2023-11-06T22:18:22.360248+0000","last_clean":"2023-11-06T22:18:22.360248+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:18:22.360248+0000","last_undegraded":"2023-11-06T22:18:22.360248+0000","last_fullsized":"2023-11-06T22:18:22.360248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":8,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.864704+0000","last_change":"2023-11-06T22:18:22.864704+0000","last_active":"2023-11-06T22:18:22.864704+0000","last_peered":"2023-11-06T22:18:22.864704+0000","last_clean":"2023-11-06T22:18:22.864704+0000","last_became_active":"2023-11-06T22:18:22.864436+0000","last_became_peered":"2023-11-06T22:18:22.864436+0000","last_unstale":"2023-11-06T22:18:22.864704+0000","last_undegraded":"2023-11-06T22:18:22.864704+0000","last_fullsized":"2023-11-06T22:18:22.864704+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":10,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862433+0000","last_change":"2023-11-06T22:18:22.380772+0000","last_active":"2023-11-06T22:18:22.862433+0000","last_peered":"2023-11-06T22:18:22.862433+0000","last_clean":"2023-11-06T22:18:22.862433+0000","last_became_active":"2023-11-06T22:18:22.380172+0000","last_became_peered":"2023-11-06T22:18:22.380172+0000","last_unstale":"2023-11-06T22:18:22.862433+0000","last_undegraded":"2023-11-06T22:18:22.862433+0000","last_fullsized":"2023-11-06T22:18:22.862433+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":202,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840691+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T22:18:22.840691+0000","last_peered":"2023-11-06T22:18:22.840691+0000","last_clean":"2023-11-06T22:18:22.840691+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:18:22.840691+0000","last_undegraded":"2023-11-06T22:18:22.840691+0000","last_fullsized":"2023-11-06T22:18:22.840691+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.878532+0000","last_change":"2023-11-06T22:18:22.876417+0000","last_active":"2023-11-06T22:18:22.878532+0000","last_peered":"2023-11-06T22:18:22.878532+0000","last_clean":"2023-11-06T22:18:22.878532+0000","last_became_active":"2023-11-06T22:18:22.875633+0000","last_became_peered":"2023-11-06T22:18:22.875633+0000","last_unstale":"2023-11-06T22:18:22.878532+0000","last_undegraded":"2023-11-06T22:18:22.878532+0000","last_fullsized":"2023-11-06T22:18:22.878532+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"222'1","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.877140+0000","last_change":"2023-11-06T22:18:22.365549+0000","last_active":"2023-11-06T22:18:22.877140+0000","last_peered":"2023-11-06T22:18:22.877140+0000","last_clean":"2023-11-06T22:18:22.877140+0000","last_became_active":"2023-11-06T22:18:22.365162+0000","last_became_peered":"2023-11-06T22:18:22.365162+0000","last_unstale":"2023-11-06T22:18:22.877140+0000","last_undegraded":"2023-11-06T22:18:22.877140+0000","last_fullsized":"2023-11-06T22:18:22.877140+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":202,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840863+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:18:22.840863+0000","last_peered":"2023-11-06T22:18:22.840863+0000","last_clean":"2023-11-06T22:18:22.840863+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:18:22.840863+0000","last_undegraded":"2023-11-06T22:18:22.840863+0000","last_fullsized":"2023-11-06T22:18:22.840863+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.866770+0000","last_change":"2023-11-06T22:18:22.861012+0000","last_active":"2023-11-06T22:18:22.866770+0000","last_peered":"2023-11-06T22:18:22.866770+0000","last_clean":"2023-11-06T22:18:22.866770+0000","last_became_active":"2023-11-06T22:18:22.860031+0000","last_became_peered":"2023-11-06T22:18:22.860031+0000","last_unstale":"2023-11-06T22:18:22.866770+0000","last_undegraded":"2023-11-06T22:18:22.866770+0000","last_fullsized":"2023-11-06T22:18:22.866770+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1d","version":"222'2","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.869990+0000","last_change":"2023-11-06T22:18:22.375900+0000","last_active":"2023-11-06T22:18:22.869990+0000","last_peered":"2023-11-06T22:18:22.869990+0000","last_clean":"2023-11-06T22:18:22.869990+0000","last_became_active":"2023-11-06T22:18:22.375442+0000","last_became_peered":"2023-11-06T22:18:22.375442+0000","last_unstale":"2023-11-06T22:18:22.869990+0000","last_undegraded":"2023-11-06T22:18:22.869990+0000","last_fullsized":"2023-11-06T22:18:22.869990+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":188,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.857648+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T22:18:22.857648+0000","last_peered":"2023-11-06T22:18:22.857648+0000","last_clean":"2023-11-06T22:18:22.857648+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:18:22.857648+0000","last_undegraded":"2023-11-06T22:18:22.857648+0000","last_fullsized":"2023-11-06T22:18:22.857648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.862244+0000","last_change":"2023-11-06T22:18:22.857254+0000","last_active":"2023-11-06T22:18:22.862244+0000","last_peered":"2023-11-06T22:18:22.862244+0000","last_clean":"2023-11-06T22:18:22.862244+0000","last_became_active":"2023-11-06T22:18:22.856484+0000","last_became_peered":"2023-11-06T22:18:22.856484+0000","last_unstale":"2023-11-06T22:18:22.862244+0000","last_undegraded":"2023-11-06T22:18:22.862244+0000","last_fullsized":"2023-11-06T22:18:22.862244+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:22.339185+0000","last_clean_scrub_stamp":"2023-11-06T22:18:22.339185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.383197+0000","last_change":"2023-11-06T22:18:22.379336+0000","last_active":"2023-11-06T22:18:22.383197+0000","last_peered":"2023-11-06T22:18:22.383197+0000","last_clean":"2023-11-06T22:18:22.383197+0000","last_became_active":"2023-11-06T22:18:22.379027+0000","last_became_peered":"2023-11-06T22:18:22.379027+0000","last_unstale":"2023-11-06T22:18:22.383197+0000","last_undegraded":"2023-11-06T22:18:22.383197+0000","last_fullsized":"2023-11-06T22:18:22.383197+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:21.331987+0000","last_clean_scrub_stamp":"2023-11-06T22:18:21.331987+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":202,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-11-06T22:18:22.840398+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:18:22.840398+0000","last_peered":"2023-11-06T22:18:22.840398+0000","last_clean":"2023-11-06T22:18:22.840398+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:18:22.840398+0000","last_undegraded":"2023-11-06T22:18:22.840398+0000","last_fullsized":"2023-11-06T22:18:22.840398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575969,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308016,"kb_used_data":2728,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063824,"statfs":{"total":96636764160,"available":96321355776,"internally_reserved":0,"allocated":2793472,"data_stored":1436464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.41599999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90700000000000003,"5min":0.97199999999999998,"15min":1.0980000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.72799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93600000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.94799999999999995}]},{"osd":1,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.71899999999999997},"min":{"1min":0.58699999999999997,"5min":0.48699999999999999,"15min":0.46400000000000002},"max":{"1min":0.94899999999999995,"5min":0.98799999999999999,"15min":1.038},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72099999999999997,"15min":0.73399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.93400000000000005,"5min":0.96599999999999997,"15min":1.0680000000000001},"last":0.75700000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.502},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90200000000000002,"5min":0.996,"15min":1.0209999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.499},"max":{"1min":0.90700000000000003,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.504},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.0229999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.94399999999999995,"15min":0.97199999999999998},"last":1.052},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.47899999999999998,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.94299999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69499999999999995}]},{"osd":5,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.40400000000000003},"max":{"1min":0.80600000000000005,"5min":1.008,"15min":1.0389999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70499999999999996,"15min":0.68700000000000006},"min":{"1min":0.53300000000000003,"5min":0.47099999999999997,"15min":0.432},"max":{"1min":0.88,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84399999999999997}]},{"osd":6,"last update":"Mon Nov 6 22:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.48399999999999999},"max":{"1min":0.84799999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.51500000000000001,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":0.86799999999999999,"5min":1.01,"15min":1.069},"last":0.99299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575969,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308008,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063832,"statfs":{"total":96636764160,"available":96321363968,"internally_reserved":0,"allocated":2785280,"data_stored":1430338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68100000000000005,"15min":0.70999999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":1.0469999999999999,"15min":1.0660000000000001},"last":0.372},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":1.016},"last":0.55100000000000005}]},{"osd":2,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0780000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.046},"last":0.63600000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.873,"5min":0.94799999999999995,"15min":1.002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.53600000000000003,"5min":0.42299999999999999,"15min":0.41699999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.98299999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76900000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.67100000000000004},"min":{"1min":0.44,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83499999999999996,"5min":0.98499999999999999,"15min":0.98999999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.442},"max":{"1min":0.80000000000000004,"5min":0.996,"15min":1.056},"last":0.754}]},{"osd":7,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79800000000000004,"5min":1.012,"15min":1.0429999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0569999999999999,"15min":1.1279999999999999},"last":0.73699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575969,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308000,"kb_used_data":2712,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063840,"statfs":{"total":96636764160,"available":96321372160,"internally_reserved":0,"allocated":2777088,"data_stored":1429694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.97999999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.33800000000000002},"max":{"1min":0.66700000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.38700000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.307,"15min":0.307},"max":{"1min":0.67400000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.371},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.95699999999999996,"15min":1.0189999999999999},"last":0.53700000000000003}]},{"osd":3,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74099999999999999,"5min":0.76200000000000001,"15min":1.0509999999999999},"last":0.498},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":0.92700000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80900000000000005,"15min":0.80800000000000005},"min":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.45200000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84899999999999998,"15min":0.85199999999999998},"min":{"1min":0.68300000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.78300000000000003}]},{"osd":5,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.81599999999999995},"min":{"1min":0.58899999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.052,"5min":1.2210000000000001,"15min":1.2709999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81499999999999995,"15min":0.80200000000000005},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.55400000000000005},"max":{"1min":1.224,"5min":1.224,"15min":1.3109999999999999},"last":0.75900000000000001}]},{"osd":6,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.3149999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.84399999999999997,"15min":0.84899999999999998},"min":{"1min":0.64900000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.65200000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1459999999999999,"5min":1.21,"15min":1.21},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1970000000000001,"15min":1.2509999999999999},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308032,"kb_used_data":2744,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063808,"statfs":{"total":96636764160,"available":96321339392,"internally_reserved":0,"allocated":2809856,"data_stored":1438070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64200000000000002,"5min":0.65800000000000003,"15min":0.74199999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.72499999999999998,"5min":0.79900000000000004,"15min":0.83299999999999996},"last":0.36299999999999999}]},{"osd":2,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.32700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.61799999999999999,"5min":0.84299999999999997,"15min":0.876},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.43099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.80900000000000005,"15min":0.83899999999999997},"last":0.441}]},{"osd":3,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.48099999999999998,"5min":0.39600000000000002,"15min":0.375},"max":{"1min":0.76400000000000001,"5min":0.81799999999999995,"15min":0.95899999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.755,"5min":0.79700000000000004,"15min":0.85799999999999998},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.622},"max":{"1min":1.008,"5min":1.083,"15min":1.357},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.83799999999999997,"15min":0.84299999999999997},"min":{"1min":0.61699999999999999,"5min":0.59299999999999997,"15min":0.54800000000000004},"max":{"1min":1.0820000000000001,"5min":1.091,"15min":1.2150000000000001},"last":0.90800000000000003}]},{"osd":5,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.85599999999999998},"min":{"1min":0.63300000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.2669999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.85199999999999998,"15min":0.86799999999999999},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.133,"5min":1.1579999999999999,"15min":1.2949999999999999},"last":0.95499999999999996}]},{"osd":6,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81699999999999995,"15min":0.81999999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.52600000000000002},"max":{"1min":1.04,"5min":1.123,"15min":1.3120000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.84799999999999998,"15min":0.84899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1160000000000001,"5min":1.127,"15min":1.26},"last":0.875}]},{"osd":7,"last update":"Mon Nov 6 22:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83599999999999997,"15min":0.85699999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1000000000000001,"5min":1.1040000000000001,"15min":1.268},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.86699999999999999,"15min":0.88200000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.4330000000000001},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575969,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308012,"kb_used_data":2724,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063828,"statfs":{"total":96636764160,"available":96321359872,"internally_reserved":0,"allocated":2789376,"data_stored":1430372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.629,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70499999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.77400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.48799999999999999,"5min":0.46000000000000002,"15min":0.45100000000000001},"max":{"1min":0.79300000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47499999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.40999999999999998,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.74299999999999999,"5min":0.74399999999999999,"15min":0.78300000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.42499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.82199999999999995,"15min":0.82699999999999996},"min":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.53900000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.349},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2609999999999999,"5min":1.2669999999999999,"15min":1.339},"last":0.92600000000000005}]},{"osd":5,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81200000000000006,"15min":0.81699999999999995},"min":{"1min":0.54600000000000004,"5min":0.54300000000000004,"15min":0.48999999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.337},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84199999999999997,"15min":0.85499999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1899999999999999,"5min":1.212,"15min":1.468},"last":0.752}]},{"osd":6,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.83099999999999996,"15min":0.84899999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.54600000000000004},"max":{"1min":1.147,"5min":1.155,"15min":1.387},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.84499999999999997,"15min":0.85099999999999998},"min":{"1min":0.627,"5min":0.56499999999999995,"15min":0.56100000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.4299999999999999},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.86599999999999999,"15min":0.876},"min":{"1min":0.61199999999999999,"5min":0.58599999999999997,"15min":0.55700000000000005},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.496},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85999999999999999,"15min":0.876},"min":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1459999999999999,"5min":1.2470000000000001,"15min":1.508},"last":0.86799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308020,"kb_used_data":2732,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063820,"statfs":{"total":96636764160,"available":96321351680,"internally_reserved":0,"allocated":2797568,"data_stored":1437888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.47299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.65500000000000003,"5min":0.67800000000000005,"15min":0.78000000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.31},"max":{"1min":0.66400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.33900000000000002}]},{"osd":1,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48399999999999999,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55300000000000005,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.46400000000000002}]},{"osd":2,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.67800000000000005,"5min":0.85299999999999998,"15min":0.87},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60799999999999998,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":4,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83799999999999997,"15min":0.83599999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.107,"5min":1.161,"15min":1.2370000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.83799999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1220000000000001,"5min":1.147,"15min":1.226},"last":0.72299999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.81100000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97399999999999998,"5min":1.071,"15min":1.2270000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.82199999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.3140000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.82899999999999996,"15min":0.83199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.2290000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.84099999999999997,"15min":0.85799999999999998},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1879999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83699999999999997,"15min":0.85899999999999999},"min":{"1min":0.55900000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.117,"5min":1.117,"15min":1.2829999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.84799999999999998,"15min":0.86599999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.51300000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.248},"last":0.63600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575969,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308016,"kb_used_data":2728,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063824,"statfs":{"total":96636764160,"available":96321355776,"internally_reserved":0,"allocated":2793472,"data_stored":1436476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.71699999999999997},"min":{"1min":0.58399999999999996,"5min":0.378,"15min":0.378},"max":{"1min":0.94899999999999995,"5min":0.96599999999999997,"15min":0.98999999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69099999999999995,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.006},"last":0.58099999999999996}]},{"osd":1,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.0289999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.018,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73999999999999999,"15min":0.75700000000000001},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.42599999999999999},"max":{"1min":0.90600000000000003,"5min":0.98599999999999999,"15min":1.087},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.63800000000000001,"5min":0.53500000000000003,"15min":0.48299999999999998},"max":{"1min":0.98499999999999999,"5min":1.04,"15min":1.071},"last":0.78600000000000003}]},{"osd":3,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.57599999999999996,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.0329999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90900000000000003,"5min":1.236,"15min":1.236},"last":0.53500000000000003}]},{"osd":5,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.42399999999999999,"15min":0.40100000000000002},"max":{"1min":0.81499999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63700000000000001,"15min":0.63800000000000001},"min":{"1min":0.46400000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.85599999999999998,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.871,"5min":1.018,"15min":1.018},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.46300000000000002},"max":{"1min":0.91700000000000004,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.86699999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.441,"15min":0.441},"max":{"1min":0.96499999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.50800000000000001},"max":{"1min":0.94199999999999995,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575970,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308008,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063832,"statfs":{"total":96636764160,"available":96321363968,"internally_reserved":0,"allocated":2785280,"data_stored":1429854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.68700000000000006},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.309},"max":{"1min":0.83999999999999997,"5min":1.226,"15min":1.226},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.72399999999999998,"15min":0.70899999999999996},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83599999999999997,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78500000000000003,"15min":0.75600000000000001},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85299999999999998,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.78000000000000003},"min":{"1min":0.54900000000000004,"5min":0.499,"15min":0.47999999999999998},"max":{"1min":0.91400000000000003,"5min":1.105,"15min":1.2290000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.45100000000000001},"max":{"1min":0.90000000000000002,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.77300000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.48299999999999998},"max":{"1min":0.878,"5min":4.9779999999999998,"15min":4.9779999999999998},"last":0.70299999999999996}]},{"osd":3,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.36499999999999999},"max":{"1min":0.79900000000000004,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.76800000000000002,"15min":0.749},"min":{"1min":0.57099999999999995,"5min":0.52600000000000002,"15min":0.45800000000000002},"max":{"1min":0.86099999999999999,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":0.66200000000000003}]},{"osd":4,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.63800000000000001,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.69499999999999995,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.38800000000000001},"max":{"1min":0.84399999999999997,"5min":1.585,"15min":1.585},"last":0.752}]},{"osd":6,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.66900000000000004,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.77500000000000002,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.441,"15min":0.42899999999999999},"max":{"1min":0.86099999999999999,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":0.76800000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.41399999999999998},"max":{"1min":0.879,"5min":1.365,"15min":1.365},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.77900000000000003,"15min":0.71899999999999997},"min":{"1min":0.50800000000000001,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":0.91200000000000003,"5min":6.891,"15min":6.891},"last":0.78600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:18:27.682 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T22:18:27.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T22:18:27.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.886+0000 7fe04e1ff700 1 Processor -- start 2023-11-06T22:18:27.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.887+0000 7fe04e1ff700 1 -- start start 2023-11-06T22:18:27.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.888+0000 7fe04e1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe04814faa0 0x7fe048064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.888+0000 7fe04e1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe04814fe80 0x7fe048152750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.888+0000 7fe04e1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe048064270 con 0x7fe048152c40 2023-11-06T22:18:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.888+0000 7fe04e1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0480651b0 con 0x7fe04814faa0 2023-11-06T22:18:27.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.888+0000 7fe04e1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481531f0 con 0x7fe04814fe80 2023-11-06T22:18:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.888+0000 7fe0477fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe04814faa0 0x7fe048064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.888+0000 7fe0477fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe04814faa0 0x7fe048064cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49266/0 (socket says 172.21.15.33:49266) 2023-11-06T22:18:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.888+0000 7fe0477fe700 1 -- 172.21.15.33:0/1214069857 learned_addr learned my addr 172.21.15.33:0/1214069857 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.888+0000 7fe047fff700 1 --1- 172.21.15.33:0/1214069857 >> v1:172.21.15.33:6789/0 conn(0x7fe048152c40 0x7fe0480c3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55604/0 (socket says 172.21.15.33:55604) 2023-11-06T22:18:27.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.888+0000 7fe046ffd700 1 --2- 172.21.15.33:0/1214069857 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe04814fe80 0x7fe048152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.889+0000 7fe0477fe700 1 -- 172.21.15.33:0/1214069857 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe04814fe80 msgr2=0x7fe048152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.889+0000 7fe0477fe700 1 --2- 172.21.15.33:0/1214069857 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe04814fe80 0x7fe048152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.889+0000 7fe0477fe700 1 -- 172.21.15.33:0/1214069857 >> v1:172.21.15.33:6789/0 conn(0x7fe048152c40 legacy=0x7fe0480c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.889+0000 7fe0477fe700 1 -- 172.21.15.33:0/1214069857 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0481534d0 con 0x7fe04814faa0 2023-11-06T22:18:27.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.889+0000 7fe0467fc700 1 -- 172.21.15.33:0/1214069857 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598479911 0 0) 0x7fe048064270 con 0x7fe048152c40 2023-11-06T22:18:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.889+0000 7fe046ffd700 1 --2- 172.21.15.33:0/1214069857 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe04814fe80 0x7fe048152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.889+0000 7fe0477fe700 1 --2- 172.21.15.33:0/1214069857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 0x7fe048064cc0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7fe038006fd0 tx=0x7fe0380090f0).ready entity=mon.0 client_cookie=e8a745c0d90f6bec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.889+0000 7fe0467fc700 1 -- 172.21.15.33:0/1214069857 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03800b600 con 0x7fe04814faa0 2023-11-06T22:18:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.889+0000 7fe0467fc700 1 -- 172.21.15.33:0/1214069857 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03800b760 con 0x7fe04814faa0 2023-11-06T22:18:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.889+0000 7fe0467fc700 1 -- 172.21.15.33:0/1214069857 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03800bae0 con 0x7fe04814faa0 2023-11-06T22:18:27.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.890+0000 7fe04e1ff700 1 -- 172.21.15.33:0/1214069857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 msgr2=0x7fe048064cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.890+0000 7fe04e1ff700 1 --2- 172.21.15.33:0/1214069857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 0x7fe048064cc0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7fe038006fd0 tx=0x7fe0380090f0).stop 2023-11-06T22:18:27.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.890+0000 7fe04e1ff700 1 -- 172.21.15.33:0/1214069857 shutdown_connections 2023-11-06T22:18:27.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.890+0000 7fe04e1ff700 1 --2- 172.21.15.33:0/1214069857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 0x7fe048064cc0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.890+0000 7fe04e1ff700 1 --2- 172.21.15.33:0/1214069857 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe04814fe80 0x7fe048152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.890+0000 7fe04e1ff700 1 -- 172.21.15.33:0/1214069857 >> 172.21.15.33:0/1214069857 conn(0x7fe048062710 msgr2=0x7fe048062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.890+0000 7fe04e1ff700 1 -- 172.21.15.33:0/1214069857 shutdown_connections 2023-11-06T22:18:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.890+0000 7fe04e1ff700 1 -- 172.21.15.33:0/1214069857 wait complete. 2023-11-06T22:18:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.891+0000 7fe04e1ff700 1 Processor -- start 2023-11-06T22:18:27.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.891+0000 7fe04e1ff700 1 -- start start 2023-11-06T22:18:27.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.892+0000 7fe04e1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 0x7fe048142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.892+0000 7fe04e1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe04814fe80 0x7fe0480abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.892+0000 7fe04e1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe048152c40 0x7fe0480bfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.892+0000 7fe04e1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480c48d0 con 0x7fe04814faa0 2023-11-06T22:18:27.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.892+0000 7fe04e1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480c4b70 con 0x7fe04814fe80 2023-11-06T22:18:27.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.892+0000 7fe04e1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480c4e50 con 0x7fe048152c40 2023-11-06T22:18:27.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe0477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 0x7fe048142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe0477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 0x7fe048142a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49280/0 (socket says 172.21.15.33:49280) 2023-11-06T22:18:27.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe0477fe700 1 -- 172.21.15.33:0/2526569628 learned_addr learned my addr 172.21.15.33:0/2526569628 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe046ffd700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe04814fe80 0x7fe0480abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe047fff700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe048152c40 0x7fe0480bfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe0477fe700 1 -- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe048152c40 msgr2=0x7fe0480bfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe0477fe700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe048152c40 0x7fe0480bfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe0477fe700 1 -- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe04814fe80 msgr2=0x7fe0480abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:27.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe0477fe700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe04814fe80 0x7fe0480abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:27.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe0477fe700 1 -- 172.21.15.33:0/2526569628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038009370 con 0x7fe04814faa0 2023-11-06T22:18:27.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe0477fe700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 0x7fe048142a10 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7fe038009130 tx=0x7fe038017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:27.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe02f7fe700 1 -- 172.21.15.33:0/2526569628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03800b8d0 con 0x7fe04814faa0 2023-11-06T22:18:27.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe02f7fe700 1 -- 172.21.15.33:0/2526569628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03800ba30 con 0x7fe04814faa0 2023-11-06T22:18:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe04e1ff700 1 -- 172.21.15.33:0/2526569628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0480c5130 con 0x7fe04814faa0 2023-11-06T22:18:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.893+0000 7fe02f7fe700 1 -- 172.21.15.33:0/2526569628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03801b660 con 0x7fe04814faa0 2023-11-06T22:18:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.894+0000 7fe04e1ff700 1 -- 172.21.15.33:0/2526569628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0480c5930 con 0x7fe04814faa0 2023-11-06T22:18:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.895+0000 7fe04e1ff700 1 -- 172.21.15.33:0/2526569628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0480c0d00 con 0x7fe04814faa0 2023-11-06T22:18:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.895+0000 7fe02f7fe700 1 -- 172.21.15.33:0/2526569628 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe03801b9d0 con 0x7fe04814faa0 2023-11-06T22:18:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.896+0000 7fe02f7fe700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe03405f270 0x7fe034061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.896+0000 7fe02f7fe700 1 -- 172.21.15.33:0/2526569628 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7fe03807bfa0 con 0x7fe04814faa0 2023-11-06T22:18:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.897+0000 7fe046ffd700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe03405f270 0x7fe034061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.897+0000 7fe046ffd700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe03405f270 0x7fe034061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fe03c006fd0 tx=0x7fe03c008040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:27.900+0000 7fe02f7fe700 1 -- 172.21.15.33:0/2526569628 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe03804d000 con 0x7fe04814faa0 2023-11-06T22:18:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.155+0000 7fe04e1ff700 1 -- 172.21.15.33:0/2526569628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe0480644c0 con 0x7fe04814faa0 2023-11-06T22:18:28.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.160+0000 7fe02f7fe700 1 -- 172.21.15.33:0/2526569628 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v49) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe038028030 con 0x7fe04814faa0 2023-11-06T22:18:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 -- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe03405f270 msgr2=0x7fe034061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe03405f270 0x7fe034061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fe03c006fd0 tx=0x7fe03c008040).stop 2023-11-06T22:18:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 -- 172.21.15.33:0/2526569628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 msgr2=0x7fe048142a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 0x7fe048142a10 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7fe038009130 tx=0x7fe038017000).stop 2023-11-06T22:18:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 -- 172.21.15.33:0/2526569628 shutdown_connections 2023-11-06T22:18:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe03405f270 0x7fe034061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe048152c40 0x7fe0480bfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe04814faa0 0x7fe048142a10 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 --2- 172.21.15.33:0/2526569628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe04814fe80 0x7fe0480abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 -- 172.21.15.33:0/2526569628 >> 172.21.15.33:0/2526569628 conn(0x7fe048062710 msgr2=0x7fe04814da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:28.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 -- 172.21.15.33:0/2526569628 shutdown_connections 2023-11-06T22:18:28.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.162+0000 7fe04e1ff700 1 -- 172.21.15.33:0/2526569628 wait complete. 2023-11-06T22:18:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-11-06T22:18:28.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.377+0000 7f85353ff700 1 Processor -- start 2023-11-06T22:18:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.377+0000 7f85353ff700 1 -- start start 2023-11-06T22:18:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.378+0000 7f85353ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 0x7f85300a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.378+0000 7f85353ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85301554a0 con 0x7f85300c6460 2023-11-06T22:18:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.378+0000 7f85353ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8530155930 con 0x7f85300aa2d0 2023-11-06T22:18:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.378+0000 7f85353ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8530154960 con 0x7f85300c1840 2023-11-06T22:18:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.378+0000 7f852f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f85300c6460 0x7f85300bfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33540/0 (socket says 172.21.15.33:33540) 2023-11-06T22:18:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.378+0000 7f852f7fe700 1 -- 172.21.15.33:0/3996003588 learned_addr learned my addr 172.21.15.33:0/3996003588 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:28.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.379+0000 7f852effd700 1 --2- 172.21.15.33:0/3996003588 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 0x7f85300a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.379+0000 7f852dffb700 1 -- 172.21.15.33:0/3996003588 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755262067 0 0) 0x7f85301554a0 con 0x7f85300c6460 2023-11-06T22:18:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.379+0000 7f852dffb700 1 -- 172.21.15.33:0/3996003588 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85180033f0 con 0x7f85300c6460 2023-11-06T22:18:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.379+0000 7f852dffb700 1 -- 172.21.15.33:0/3996003588 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172925699 0 0) 0x7f8530155930 con 0x7f85300aa2d0 2023-11-06T22:18:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.379+0000 7f852dffb700 1 -- 172.21.15.33:0/3996003588 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85301554a0 con 0x7f85300aa2d0 2023-11-06T22:18:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.379+0000 7f852effd700 1 -- 172.21.15.33:0/3996003588 >> v1:172.21.15.69:6789/0 conn(0x7f85300aa2d0 legacy=0x7f85300c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.379+0000 7f852effd700 1 -- 172.21.15.33:0/3996003588 >> v1:172.21.15.33:6789/0 conn(0x7f85300c6460 legacy=0x7f85300bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.379+0000 7f852effd700 1 -- 172.21.15.33:0/3996003588 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530154b70 con 0x7f85300c1840 2023-11-06T22:18:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.379+0000 7f852effd700 1 --2- 172.21.15.33:0/3996003588 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 0x7f85300a9de0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f8520009790 tx=0x7f8520009060).ready entity=mon.2 client_cookie=d9eee8db403b11fc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.380+0000 7f852dffb700 1 -- 172.21.15.33:0/3996003588 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000f510 con 0x7f85300c1840 2023-11-06T22:18:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.380+0000 7f852dffb700 1 -- 172.21.15.33:0/3996003588 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852000f670 con 0x7f85300c1840 2023-11-06T22:18:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.380+0000 7f852dffb700 1 -- 172.21.15.33:0/3996003588 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000f9f0 con 0x7f85300c1840 2023-11-06T22:18:28.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.380+0000 7f85353ff700 1 -- 172.21.15.33:0/3996003588 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 msgr2=0x7f85300a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.380+0000 7f85353ff700 1 --2- 172.21.15.33:0/3996003588 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 0x7f85300a9de0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f8520009790 tx=0x7f8520009060).stop 2023-11-06T22:18:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.380+0000 7f85353ff700 1 -- 172.21.15.33:0/3996003588 shutdown_connections 2023-11-06T22:18:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.380+0000 7f85353ff700 1 --2- 172.21.15.33:0/3996003588 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 0x7f85300a9de0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.380+0000 7f85353ff700 1 -- 172.21.15.33:0/3996003588 >> 172.21.15.33:0/3996003588 conn(0x7f853005ec80 msgr2=0x7f85300cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:28.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.380+0000 7f85353ff700 1 -- 172.21.15.33:0/3996003588 shutdown_connections 2023-11-06T22:18:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.380+0000 7f85353ff700 1 -- 172.21.15.33:0/3996003588 wait complete. 2023-11-06T22:18:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.382+0000 7f85353ff700 1 Processor -- start 2023-11-06T22:18:28.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.382+0000 7f85353ff700 1 -- start start 2023-11-06T22:18:28.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.383+0000 7f85353ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85300aa2d0 0x7f8530156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:28.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.383+0000 7f85353ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 0x7f8530155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:28.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.383+0000 7f85353ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85300c6460 0x7f85300c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:28.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.383+0000 7f85353ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85300af820 con 0x7f85300aa2d0 2023-11-06T22:18:28.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.383+0000 7f85353ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85300af9b0 con 0x7f85300c6460 2023-11-06T22:18:28.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.383+0000 7f85353ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8530156b40 con 0x7f85300c1840 2023-11-06T22:18:28.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.383+0000 7f852effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85300aa2d0 0x7f8530156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:28.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.383+0000 7f852effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85300aa2d0 0x7f8530156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42556/0 (socket says 172.21.15.33:42556) 2023-11-06T22:18:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.383+0000 7f852effd700 1 -- 172.21.15.33:0/578035982 learned_addr learned my addr 172.21.15.33:0/578035982 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f852f7fe700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85300c6460 0x7f85300c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f852e7fc700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 0x7f8530155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f852effd700 1 -- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 msgr2=0x7f8530155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f852effd700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 0x7f8530155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f852effd700 1 -- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85300c6460 msgr2=0x7f85300c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f852effd700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85300c6460 0x7f85300c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f852effd700 1 -- 172.21.15.33:0/578035982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85200092a0 con 0x7f85300aa2d0 2023-11-06T22:18:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f852e7fc700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 0x7f8530155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f852effd700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85300aa2d0 0x7f8530156580 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f853006bc90 tx=0x7f8520017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f8516ffd700 1 -- 172.21.15.33:0/578035982 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000f7e0 con 0x7f85300aa2d0 2023-11-06T22:18:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f8516ffd700 1 -- 172.21.15.33:0/578035982 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852000f940 con 0x7f85300aa2d0 2023-11-06T22:18:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f8516ffd700 1 -- 172.21.15.33:0/578035982 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852001b660 con 0x7f85300aa2d0 2023-11-06T22:18:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f85353ff700 1 -- 172.21.15.33:0/578035982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8530156e20 con 0x7f85300aa2d0 2023-11-06T22:18:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.384+0000 7f85353ff700 1 -- 172.21.15.33:0/578035982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85301573e0 con 0x7f85300aa2d0 2023-11-06T22:18:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.386+0000 7f85353ff700 1 -- 172.21.15.33:0/578035982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853006c6c0 con 0x7f85300aa2d0 2023-11-06T22:18:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.386+0000 7f8516ffd700 1 -- 172.21.15.33:0/578035982 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f852001ba10 con 0x7f85300aa2d0 2023-11-06T22:18:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.387+0000 7f8516ffd700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f851c05f240 0x7f851c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.387+0000 7f8516ffd700 1 -- 172.21.15.33:0/578035982 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f852007be70 con 0x7f85300aa2d0 2023-11-06T22:18:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.388+0000 7f852e7fc700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f851c05f240 0x7f851c061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:28.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.388+0000 7f852e7fc700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f851c05f240 0x7f851c061700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f8530156a70 tx=0x7f8524009040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:28.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.392+0000 7f8516ffd700 1 -- 172.21.15.33:0/578035982 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f85200509c0 con 0x7f85300aa2d0 2023-11-06T22:18:28.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.621+0000 7f85353ff700 1 -- 172.21.15.33:0/578035982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85300afc90 con 0x7f85300aa2d0 2023-11-06T22:18:28.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.621+0000 7f8516ffd700 1 -- 172.21.15.33:0/578035982 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f8520028070 con 0x7f85300aa2d0 2023-11-06T22:18:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:28.623 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":91,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3097639551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3097639551},{"type":"v1","addr":"172.21.15.69:6837","nonce":3097639551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9828,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1256245016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1256245016},{"type":"v1","addr":"172.21.15.33:6834","nonce":1256245016}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1680210655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1680210655},{"type":"v1","addr":"172.21.15.33:6835","nonce":1680210655}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:18:22.834249+0000","modified":"2023-11-06T22:18:25.840897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9837":{"gid":9837,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/3921249578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3921249578},{"type":"v1","addr":"172.21.15.69:6836","nonce":3921249578}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-06T22:18:28.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.623+0000 7f85353ff700 1 -- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f851c05f240 msgr2=0x7f851c061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:28.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.623+0000 7f85353ff700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f851c05f240 0x7f851c061700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f8530156a70 tx=0x7f8524009040).stop 2023-11-06T22:18:28.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.623+0000 7f85353ff700 1 -- 172.21.15.33:0/578035982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85300aa2d0 msgr2=0x7f8530156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:28.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.623+0000 7f85353ff700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85300aa2d0 0x7f8530156580 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f853006bc90 tx=0x7f8520017000).stop 2023-11-06T22:18:28.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.624+0000 7f85353ff700 1 -- 172.21.15.33:0/578035982 shutdown_connections 2023-11-06T22:18:28.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.624+0000 7f85353ff700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f851c05f240 0x7f851c061700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.624+0000 7f85353ff700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85300c6460 0x7f85300c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.624+0000 7f85353ff700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85300aa2d0 0x7f8530156580 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.624+0000 7f85353ff700 1 --2- 172.21.15.33:0/578035982 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85300c1840 0x7f8530155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:28.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.624+0000 7f85353ff700 1 -- 172.21.15.33:0/578035982 >> 172.21.15.33:0/578035982 conn(0x7f853005ec80 msgr2=0x7f8530150710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:28.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.624+0000 7f85353ff700 1 -- 172.21.15.33:0/578035982 shutdown_connections 2023-11-06T22:18:28.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:28.624+0000 7f85353ff700 1 -- 172.21.15.33:0/578035982 wait complete. 2023-11-06T22:18:28.628 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 91 2023-11-06T22:18:29.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T22:18:22.834249+0000', 'modified': '2023-11-06T22:18:25.840897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9837}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9837': {'gid': 9837, 'name': 'd', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.69:6836/3921249578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 3921249578}, {'type': 'v1', 'addr': '172.21.15.69:6836', 'nonce': 3921249578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-06T22:18:29.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T22:18:29.047 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-06T22:18:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.245+0000 7f0f9d5ff700 1 Processor -- start 2023-11-06T22:18:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.245+0000 7f0f9d5ff700 1 -- start start 2023-11-06T22:18:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.246+0000 7f0f9d5ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0f9805eae0 0x7f0f980645a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.246+0000 7f0f9d5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f98064a90 0x7f0f980c5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.246+0000 7f0f9d5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f98155480 con 0x7f0f980c7420 2023-11-06T22:18:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.246+0000 7f0f9d5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f98064f20 con 0x7f0f98064a90 2023-11-06T22:18:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.246+0000 7f0f9d5ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f980aa5f0 con 0x7f0f9805eae0 2023-11-06T22:18:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.246+0000 7f0f96ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0f980c7420 0x7f0f980a9e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33554/0 (socket says 172.21.15.33:33554) 2023-11-06T22:18:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.246+0000 7f0f96ffd700 1 -- 172.21.15.33:0/597522513 learned_addr learned my addr 172.21.15.33:0/597522513 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.246+0000 7f0f977fe700 1 --2- 172.21.15.33:0/597522513 >> v2:172.21.15.33:3300/0 conn(0x7f0f98064a90 0x7f0f980c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.247+0000 7f0f967fc700 1 --2- 172.21.15.33:0/597522513 >> v2:172.21.15.69:3300/0 conn(0x7f0f9805eae0 0x7f0f980645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.247+0000 7f0f95ffb700 1 -- 172.21.15.33:0/597522513 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355864780 0 0) 0x7f0f98155480 con 0x7f0f980c7420 2023-11-06T22:18:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.247+0000 7f0f95ffb700 1 -- 172.21.15.33:0/597522513 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f800033f0 con 0x7f0f980c7420 2023-11-06T22:18:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.247+0000 7f0f977fe700 1 -- 172.21.15.33:0/597522513 >> v2:172.21.15.69:3300/0 conn(0x7f0f9805eae0 msgr2=0x7f0f980645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.247+0000 7f0f977fe700 1 --2- 172.21.15.33:0/597522513 >> v2:172.21.15.69:3300/0 conn(0x7f0f9805eae0 0x7f0f980645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.247+0000 7f0f977fe700 1 -- 172.21.15.33:0/597522513 >> v1:172.21.15.33:6789/0 conn(0x7f0f980c7420 legacy=0x7f0f980a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.247+0000 7f0f977fe700 1 -- 172.21.15.33:0/597522513 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f98154b50 con 0x7f0f98064a90 2023-11-06T22:18:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.247+0000 7f0f977fe700 1 --2- 172.21.15.33:0/597522513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 0x7f0f980c5750 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7f0f980aa520 tx=0x7f0f90009780).ready entity=mon.0 client_cookie=15e1e4feeb7b06cf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.247+0000 7f0f95ffb700 1 -- 172.21.15.33:0/597522513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000e070 con 0x7f0f98064a90 2023-11-06T22:18:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.247+0000 7f0f95ffb700 1 -- 172.21.15.33:0/597522513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9000bb40 con 0x7f0f98064a90 2023-11-06T22:18:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.248+0000 7f0f95ffb700 1 -- 172.21.15.33:0/597522513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f90012410 con 0x7f0f98064a90 2023-11-06T22:18:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.248+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/597522513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 msgr2=0x7f0f980c5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.248+0000 7f0f9d5ff700 1 --2- 172.21.15.33:0/597522513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 0x7f0f980c5750 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7f0f980aa520 tx=0x7f0f90009780).stop 2023-11-06T22:18:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.248+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/597522513 shutdown_connections 2023-11-06T22:18:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.248+0000 7f0f9d5ff700 1 --2- 172.21.15.33:0/597522513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 0x7f0f980c5750 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.248+0000 7f0f9d5ff700 1 --2- 172.21.15.33:0/597522513 >> v2:172.21.15.69:3300/0 conn(0x7f0f9805eae0 0x7f0f980645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.248+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/597522513 >> 172.21.15.33:0/597522513 conn(0x7f0f980c8320 msgr2=0x7f0f9805e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.248+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/597522513 shutdown_connections 2023-11-06T22:18:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.248+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/597522513 wait complete. 2023-11-06T22:18:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.250+0000 7f0f9d5ff700 1 Processor -- start 2023-11-06T22:18:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.251+0000 7f0f9d5ff700 1 -- start start 2023-11-06T22:18:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f9d5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f9805eae0 0x7f0f980c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f9d5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 0x7f0f980c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f9d5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f980c7420 0x7f0f98153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f9d5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f980c4090 con 0x7f0f98064a90 2023-11-06T22:18:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f9d5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f980c4370 con 0x7f0f980c7420 2023-11-06T22:18:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f9d5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f980c4650 con 0x7f0f9805eae0 2023-11-06T22:18:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 0x7f0f980c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 0x7f0f980c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42584/0 (socket says 172.21.15.33:42584) 2023-11-06T22:18:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f967fc700 1 -- 172.21.15.33:0/2954242628 learned_addr learned my addr 172.21.15.33:0/2954242628 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f977fe700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f980c7420 0x7f0f98153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.252+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f9805eae0 0x7f0f980c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f967fc700 1 -- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f9805eae0 msgr2=0x7f0f980c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f967fc700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f9805eae0 0x7f0f980c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f967fc700 1 -- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f980c7420 msgr2=0x7f0f98153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f967fc700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f980c7420 0x7f0f98153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f967fc700 1 -- 172.21.15.33:0/2954242628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f90009a20 con 0x7f0f98064a90 2023-11-06T22:18:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f977fe700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f980c7420 0x7f0f98153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f967fc700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 0x7f0f980c3800 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7f0f8c00e930 tx=0x7f0f8c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f7effd700 1 -- 172.21.15.33:0/2954242628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00eed0 con 0x7f0f98064a90 2023-11-06T22:18:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f7effd700 1 -- 172.21.15.33:0/2954242628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c00c850 con 0x7f0f98064a90 2023-11-06T22:18:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f7effd700 1 -- 172.21.15.33:0/2954242628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00cbd0 con 0x7f0f98064a90 2023-11-06T22:18:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/2954242628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9814a4d0 con 0x7f0f98064a90 2023-11-06T22:18:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.253+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/2954242628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f98153580 con 0x7f0f98064a90 2023-11-06T22:18:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.255+0000 7f0f7effd700 1 -- 172.21.15.33:0/2954242628 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0f8c01b400 con 0x7f0f98064a90 2023-11-06T22:18:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.256+0000 7f0f7effd700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0f8405f270 0x7f0f84061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.256+0000 7f0f7effd700 1 -- 172.21.15.33:0/2954242628 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f0f8c07a9b0 con 0x7f0f98064a90 2023-11-06T22:18:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.256+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0f8405f270 0x7f0f84061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.257+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0f8405f270 0x7f0f84061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0f88005750 tx=0x7f0f88009040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:29.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.258+0000 7f0fb491cb80 1 Processor -- start 2023-11-06T22:18:29.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.259+0000 7f0fb491cb80 1 -- start start 2023-11-06T22:18:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.259+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6828080 0x5585e6828460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.259+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6870be0 0x5585e6822ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.259+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e68504c0 0x5585e68233e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.259+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e67642f0 con 0x5585e6828080 2023-11-06T22:18:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.259+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e67797e0 con 0x5585e68504c0 2023-11-06T22:18:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.259+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585e6779920 con 0x5585e6870be0 2023-11-06T22:18:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f96ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6828080 0x5585e6828460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f96ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6828080 0x5585e6828460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42598/0 (socket says 172.21.15.33:42598) 2023-11-06T22:18:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f96ffd700 1 -- 172.21.15.33:0/2735165791 learned_addr learned my addr 172.21.15.33:0/2735165791 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f967fc700 1 --2- 172.21.15.33:0/2735165791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6870be0 0x5585e6822ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f977fe700 1 --2- 172.21.15.33:0/2735165791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e68504c0 0x5585e68233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f96ffd700 1 -- 172.21.15.33:0/2735165791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6870be0 msgr2=0x5585e6822ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/2735165791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6870be0 0x5585e6822ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f96ffd700 1 -- 172.21.15.33:0/2735165791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e68504c0 msgr2=0x5585e68233e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/2735165791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e68504c0 0x5585e68233e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f96ffd700 1 -- 172.21.15.33:0/2735165791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585e6772720 con 0x5585e6828080 2023-11-06T22:18:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.260+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/2735165791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6828080 0x5585e6828460 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7f0f88007340 tx=0x7f0f88007000).ready entity=mon.0 client_cookie=f7bfd4c1c7569658 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0f77fff700 1 -- 172.21.15.33:0/2735165791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88015410 con 0x5585e6828080 2023-11-06T22:18:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0f77fff700 1 -- 172.21.15.33:0/2735165791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88015570 con 0x5585e6828080 2023-11-06T22:18:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0f77fff700 1 -- 172.21.15.33:0/2735165791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880158f0 con 0x5585e6828080 2023-11-06T22:18:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0fb491cb80 1 -- 172.21.15.33:0/2735165791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6828080 msgr2=0x5585e6828460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/2735165791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6828080 0x5585e6828460 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7f0f88007340 tx=0x7f0f88007000).stop 2023-11-06T22:18:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0fb491cb80 1 -- 172.21.15.33:0/2735165791 shutdown_connections 2023-11-06T22:18:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/2735165791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e68504c0 0x5585e68233e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/2735165791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6828080 0x5585e6828460 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/2735165791 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6870be0 0x5585e6822ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0fb491cb80 1 -- 172.21.15.33:0/2735165791 >> 172.21.15.33:0/2735165791 conn(0x5585e6691940 msgr2=0x5585e6871070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0fb491cb80 1 -- 172.21.15.33:0/2735165791 shutdown_connections 2023-11-06T22:18:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.261+0000 7f0fb491cb80 1 -- 172.21.15.33:0/2735165791 wait complete. 2023-11-06T22:18:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.262+0000 7f0fb491cb80 1 -- start start 2023-11-06T22:18:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.262+0000 7f0fb491cb80 1 Processor -- start 2023-11-06T22:18:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.263+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6828080 0x5585e684b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.263+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e68504c0 0x5585e684c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.263+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e686ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.263+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6779ba0 con 0x5585e6870be0 2023-11-06T22:18:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.263+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6744e60 con 0x5585e68504c0 2023-11-06T22:18:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.263+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585e686abf0 con 0x5585e6828080 2023-11-06T22:18:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0fb491cb80 10 client.0 ll_register_callbacks cb 0x5585e6719ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T22:18:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e686ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e686ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42606/0 (socket says 172.21.15.33:42606) 2023-11-06T22:18:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f977fe700 1 -- 172.21.15.33:0/1070675108 learned_addr learned my addr 172.21.15.33:0/1070675108 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6828080 0x5585e684b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f967fc700 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e68504c0 0x5585e684c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f977fe700 1 -- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6828080 msgr2=0x5585e684b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f977fe700 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6828080 0x5585e684b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f977fe700 1 -- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e68504c0 msgr2=0x5585e684c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f977fe700 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e68504c0 0x5585e684c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f977fe700 1 -- 172.21.15.33:0/1070675108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8800f040 con 0x5585e6870be0 2023-11-06T22:18:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f977fe700 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e686ae60 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7f0f9000b610 tx=0x7f0f90009780).ready entity=mon.0 client_cookie=4a4410af4c37b25c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.264+0000 7f0f757fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T22:18:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0f757fa700 1 -- 172.21.15.33:0/1070675108 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000b640 con 0x5585e6870be0 2023-11-06T22:18:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0f757fa700 1 -- 172.21.15.33:0/1070675108 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f0f9001d440 con 0x5585e6870be0 2023-11-06T22:18:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0fb491cb80 1 -- 172.21.15.33:0/1070675108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585e681be20 con 0x5585e6870be0 2023-11-06T22:18:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0f757fa700 1 -- 172.21.15.33:0/1070675108 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f9001d810 con 0x5585e6870be0 2023-11-06T22:18:29.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0fb491cb80 10 client.9972 fetch_fsmap learned FSMap version 91 2023-11-06T22:18:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0fb491cb80 1 -- 172.21.15.33:0/1070675108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5585e676dac0 con 0x5585e6870be0 2023-11-06T22:18:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0f757fa700 1 -- 172.21.15.33:0/1070675108 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585e681be20 con 0x5585e6870be0 2023-11-06T22:18:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0f757fa700 1 -- 172.21.15.33:0/1070675108 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 91) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0f9001d970 con 0x5585e6870be0 2023-11-06T22:18:29.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0fb491cb80 10 client.9972 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T22:18:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0fb491cb80 10 client.9972 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T22:18:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0fb491cb80 20 client.9972 populate_metadata read hostname 'smithi033' 2023-11-06T22:18:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:6834/3921249578,v1:172.21.15.69:6836/3921249578] conn(0x5585e6856710 0x5585e6856af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0fb491cb80 4 client.9972 mds_command: new command op to 9837 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T22:18:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.265+0000 7f0fb491cb80 1 -- 172.21.15.33:0/1070675108 --> [v2:172.21.15.69:6834/3921249578,v1:172.21.15.69:6836/3921249578] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585e65e1ef0 con 0x5585e6856710 2023-11-06T22:18:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.266+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:6834/3921249578,v1:172.21.15.69:6836/3921249578] conn(0x5585e6856710 0x5585e6856af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.267+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:6834/3921249578,v1:172.21.15.69:6836/3921249578] conn(0x5585e6856710 0x5585e6856af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=edfc59d46ced7b2d server_cookie=31ed581ee2b40c79 in_seq=0 out_seq=0 2023-11-06T22:18:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.267+0000 7f0f757fa700 10 client.9972 ms_handle_connect on v2:172.21.15.69:6834/3921249578 2023-11-06T22:18:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.267+0000 7f0f757fa700 1 -- 172.21.15.33:0/1070675108 <== mds.0 v2:172.21.15.69:6834/3921249578 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x5585e65e1ef0 con 0x5585e6856710 2023-11-06T22:18:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.267+0000 7f0f757fa700 10 client.9972 handle_command_reply: tid=0 2023-11-06T22:18:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.267+0000 7f0fb491cb80 1 client.9972 shutdown 2023-11-06T22:18:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 1 -- 172.21.15.33:0/1070675108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 msgr2=0x5585e686ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e686ae60 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7f0f9000b610 tx=0x7f0f90009780).stop 2023-11-06T22:18:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 1 -- 172.21.15.33:0/1070675108 shutdown_connections 2023-11-06T22:18:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:6834/3921249578,v1:172.21.15.69:6836/3921249578] conn(0x5585e6856710 0x5585e6856af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6828080 0x5585e684b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e686ae60 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/1070675108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e68504c0 0x5585e684c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0f757fa700 0 client.9972 ms_handle_reset on v2:172.21.15.69:6834/3921249578 2023-11-06T22:18:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 1 -- 172.21.15.33:0/1070675108 >> 172.21.15.33:0/1070675108 conn(0x5585e6691940 msgr2=0x5585e6829380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 1 -- 172.21.15.33:0/1070675108 shutdown_connections 2023-11-06T22:18:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 1 -- 172.21.15.33:0/1070675108 wait complete. 2023-11-06T22:18:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.268+0000 7f0fb491cb80 20 client.9972 trim_cache size 0 max 16384 2023-11-06T22:18:29.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.285+0000 7f0fb491cb80 1 Processor -- start 2023-11-06T22:18:29.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.285+0000 7f0fb491cb80 1 -- start start 2023-11-06T22:18:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.286+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6828080 0x5585e6821f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.286+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e68504c0 0x5585e686ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.286+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e684ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.286+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6870a60 con 0x5585e6870be0 2023-11-06T22:18:29.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.286+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6825560 con 0x5585e6828080 2023-11-06T22:18:29.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.286+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585e68298b0 con 0x5585e68504c0 2023-11-06T22:18:29.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e684ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e684ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42620/0 (socket says 172.21.15.33:42620) 2023-11-06T22:18:29.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f977fe700 1 -- 172.21.15.33:0/3034201262 learned_addr learned my addr 172.21.15.33:0/3034201262 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:29.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f967fc700 1 --2- 172.21.15.33:0/3034201262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e68504c0 0x5585e686ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/3034201262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6828080 0x5585e6821f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f977fe700 1 -- 172.21.15.33:0/3034201262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e68504c0 msgr2=0x5585e686ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:29.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f977fe700 1 --2- 172.21.15.33:0/3034201262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e68504c0 0x5585e686ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f977fe700 1 -- 172.21.15.33:0/3034201262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6828080 msgr2=0x5585e6821f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:29.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f977fe700 1 --2- 172.21.15.33:0/3034201262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6828080 0x5585e6821f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f977fe700 1 -- 172.21.15.33:0/3034201262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585e676d810 con 0x5585e6870be0 2023-11-06T22:18:29.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/3034201262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6828080 0x5585e6821f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:29.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f977fe700 1 --2- 172.21.15.33:0/3034201262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e684ca30 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7f0f900094d0 tx=0x7f0f9001f040).ready entity=mon.0 client_cookie=eade67603333c4cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:29.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f7c9ff700 1 -- 172.21.15.33:0/3034201262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f90037070 con 0x5585e6870be0 2023-11-06T22:18:29.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f7c9ff700 1 -- 172.21.15.33:0/3034201262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9000b640 con 0x5585e6870be0 2023-11-06T22:18:29.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.287+0000 7f0f7c9ff700 1 -- 172.21.15.33:0/3034201262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f90012b00 con 0x5585e6870be0 2023-11-06T22:18:29.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.288+0000 7f0fb491cb80 1 -- 172.21.15.33:0/3034201262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 msgr2=0x5585e684ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.288+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/3034201262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e684ca30 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7f0f900094d0 tx=0x7f0f9001f040).stop 2023-11-06T22:18:29.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.288+0000 7f0fb491cb80 1 -- 172.21.15.33:0/3034201262 shutdown_connections 2023-11-06T22:18:29.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.288+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/3034201262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e6870be0 0x5585e684ca30 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.288+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/3034201262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6828080 0x5585e6821f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.288+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/3034201262 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e68504c0 0x5585e686ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.288+0000 7f0fb491cb80 1 -- 172.21.15.33:0/3034201262 >> 172.21.15.33:0/3034201262 conn(0x5585e6691940 msgr2=0x5585e6824cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:29.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.288+0000 7f0fb491cb80 1 -- 172.21.15.33:0/3034201262 shutdown_connections 2023-11-06T22:18:29.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.288+0000 7f0fb491cb80 1 -- 172.21.15.33:0/3034201262 wait complete. 2023-11-06T22:18:29.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.289+0000 7f0fb491cb80 1 -- start start 2023-11-06T22:18:29.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.289+0000 7f0fb491cb80 1 Processor -- start 2023-11-06T22:18:29.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6828080 0x5585e6854870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e68504c0 0x5585e68d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0fb491cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6870be0 0x5585e6825bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e68555f0 con 0x5585e68504c0 2023-11-06T22:18:29.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e6856580 con 0x5585e6870be0 2023-11-06T22:18:29.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0fb491cb80 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585e68d6c70 con 0x5585e6828080 2023-11-06T22:18:29.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0fb491cb80 10 client.0 ll_register_callbacks cb 0x5585e6719ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T22:18:29.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0f967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e68504c0 0x5585e68d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0f967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e68504c0 0x5585e68d2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42632/0 (socket says 172.21.15.33:42632) 2023-11-06T22:18:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0f967fc700 1 -- 172.21.15.33:0/334288343 learned_addr learned my addr 172.21.15.33:0/334288343 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6828080 0x5585e6854870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.291+0000 7f0f977fe700 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6870be0 0x5585e6825bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f967fc700 1 -- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6828080 msgr2=0x5585e6854870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:29.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f967fc700 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6828080 0x5585e6854870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f967fc700 1 -- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6870be0 msgr2=0x5585e6825bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:29.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f967fc700 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6870be0 0x5585e6825bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f967fc700 1 -- 172.21.15.33:0/334288343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585e676dac0 con 0x5585e68504c0 2023-11-06T22:18:29.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f977fe700 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6870be0 0x5585e6825bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:29.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f967fc700 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e68504c0 0x5585e68d2130 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7f0f8c04bc30 tx=0x7f0f8c052040).ready entity=mon.0 client_cookie=d46d23f340f56946 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:29.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f757fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-11-06T22:18:29.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f757fa700 1 -- 172.21.15.33:0/334288343 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c057040 con 0x5585e68504c0 2023-11-06T22:18:29.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0fb491cb80 1 -- 172.21.15.33:0/334288343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585e6617040 con 0x5585e68504c0 2023-11-06T22:18:29.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f757fa700 1 -- 172.21.15.33:0/334288343 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f0f8c05dcc0 con 0x5585e68504c0 2023-11-06T22:18:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f757fa700 1 -- 172.21.15.33:0/334288343 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f8c05de80 con 0x5585e68504c0 2023-11-06T22:18:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0fb491cb80 10 client.9978 fetch_fsmap learned FSMap version 91 2023-11-06T22:18:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0fb491cb80 1 -- 172.21.15.33:0/334288343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5585e65e71d0 con 0x5585e68504c0 2023-11-06T22:18:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.292+0000 7f0f757fa700 1 -- 172.21.15.33:0/334288343 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585e6617040 con 0x5585e68504c0 2023-11-06T22:18:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.293+0000 7f0f757fa700 1 -- 172.21.15.33:0/334288343 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 91) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0f8c04b520 con 0x5585e68504c0 2023-11-06T22:18:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.293+0000 7f0fb491cb80 10 client.9978 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T22:18:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.293+0000 7f0fb491cb80 10 client.9978 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T22:18:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.293+0000 7f0fb491cb80 20 client.9978 populate_metadata read hostname 'smithi033' 2023-11-06T22:18:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.293+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:6834/3921249578,v1:172.21.15.69:6836/3921249578] conn(0x5585e68d5dc0 0x5585e68d61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.293+0000 7f0fb491cb80 4 client.9978 mds_command: new command op to 9837 tid=0[{"prefix": "status"}] 2023-11-06T22:18:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.293+0000 7f0fb491cb80 1 -- 172.21.15.33:0/334288343 --> [v2:172.21.15.69:6834/3921249578,v1:172.21.15.69:6836/3921249578] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5585e66080c0 con 0x5585e68d5dc0 2023-11-06T22:18:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.293+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:6834/3921249578,v1:172.21.15.69:6836/3921249578] conn(0x5585e68d5dc0 0x5585e68d61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.293+0000 7f0f757fa700 10 client.9978 ms_handle_connect on v2:172.21.15.69:6834/3921249578 2023-11-06T22:18:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.293+0000 7f0f96ffd700 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:6834/3921249578,v1:172.21.15.69:6836/3921249578] conn(0x5585e68d5dc0 0x5585e68d61a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=78e1c88e7ae6a8ee server_cookie=6c96429747843c46 in_seq=0 out_seq=0 2023-11-06T22:18:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0f757fa700 1 -- 172.21.15.33:0/334288343 <== mds.0 v2:172.21.15.69:6834/3921249578 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5585e66080c0 con 0x5585e68d5dc0 2023-11-06T22:18:29.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0f757fa700 10 client.9978 handle_command_reply: tid=0 2023-11-06T22:18:29.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 client.9978 shutdown 2023-11-06T22:18:29.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 -- 172.21.15.33:0/334288343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e68504c0 msgr2=0x5585e68d2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e68504c0 0x5585e68d2130 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7f0f8c04bc30 tx=0x7f0f8c052040).stop 2023-11-06T22:18:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 -- 172.21.15.33:0/334288343 shutdown_connections 2023-11-06T22:18:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:6834/3921249578,v1:172.21.15.69:6836/3921249578] conn(0x5585e68d5dc0 0x5585e68d61a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x5585e6828080 0x5585e6854870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585e6870be0 0x5585e6825bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 --2- 172.21.15.33:0/334288343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585e68504c0 0x5585e68d2130 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0f757fa700 0 client.9978 ms_handle_reset on v2:172.21.15.69:6834/3921249578 2023-11-06T22:18:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 -- 172.21.15.33:0/334288343 >> 172.21.15.33:0/334288343 conn(0x5585e6691940 msgr2=0x5585e68271a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 -- 172.21.15.33:0/334288343 shutdown_connections 2023-11-06T22:18:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 1 -- 172.21.15.33:0/334288343 wait complete. 2023-11-06T22:18:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.294+0000 7f0fb491cb80 20 client.9978 trim_cache size 0 max 16384 2023-11-06T22:18:29.324 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:18:29.325 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "097ce162-8fe3-4461-8445-f7551959ee06", 2023-11-06T22:18:29.325 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-11-06T22:18:29.325 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 9837, 2023-11-06T22:18:29.325 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-11-06T22:18:29.325 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-11-06T22:18:29.326 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-11-06T22:18:29.326 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.4568329689999997, 2023-11-06T22:18:29.326 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 91, 2023-11-06T22:18:29.326 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 223, 2023-11-06T22:18:29.326 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 223, 2023-11-06T22:18:29.327 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 11.660493943000001 2023-11-06T22:18:29.327 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:18:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0f8405f270 msgr2=0x7f0f84061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0f8405f270 0x7f0f84061730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0f88005750 tx=0x7f0f88009040).stop 2023-11-06T22:18:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/2954242628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 msgr2=0x7f0f980c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 0x7f0f980c3800 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7f0f8c00e930 tx=0x7f0f8c009060).stop 2023-11-06T22:18:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/2954242628 shutdown_connections 2023-11-06T22:18:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0f8405f270 0x7f0f84061730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0f9805eae0 0x7f0f980c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f980c7420 0x7f0f98153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 --2- 172.21.15.33:0/2954242628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98064a90 0x7f0f980c3800 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/2954242628 >> 172.21.15.33:0/2954242628 conn(0x7f0f980c8320 msgr2=0x7f0f98155810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:29.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/2954242628 shutdown_connections 2023-11-06T22:18:29.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.295+0000 7f0f9d5ff700 1 -- 172.21.15.33:0/2954242628 wait complete. 2023-11-06T22:18:29.331 DEBUG:teuthology.orchestra.run.smithi033:> ceph fs ls 2023-11-06T22:18:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.494+0000 7f43ca5ff700 1 Processor -- start 2023-11-06T22:18:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.494+0000 7f43ca5ff700 1 -- start start 2023-11-06T22:18:29.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.496+0000 7f43ca5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 0x7f43c40c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.496+0000 7f43ca5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43c40c07b0 0x7f43c40c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.496+0000 7f43ca5ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c40c5f80 con 0x7f43c41541a0 2023-11-06T22:18:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.496+0000 7f43ca5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c40aa740 con 0x7f43c40c07b0 2023-11-06T22:18:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.496+0000 7f43ca5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40c6800 con 0x7f43c40a9ee0 2023-11-06T22:18:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.496+0000 7f43c8b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43c40c07b0 0x7f43c40c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.496+0000 7f43c8b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43c40c07b0 0x7f43c40c0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42648/0 (socket says 172.21.15.33:42648) 2023-11-06T22:18:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.496+0000 7f43c8b9c700 1 -- 172.21.15.33:0/4130995989 learned_addr learned my addr 172.21.15.33:0/4130995989 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.497+0000 7f43c37fe700 1 --2- 172.21.15.33:0/4130995989 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 0x7f43c40c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.497+0000 7f43c8b9c700 1 -- 172.21.15.33:0/4130995989 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 msgr2=0x7f43c40c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.497+0000 7f43c8b9c700 1 --2- 172.21.15.33:0/4130995989 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 0x7f43c40c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.497+0000 7f43c8b9c700 1 -- 172.21.15.33:0/4130995989 >> v1:172.21.15.69:6789/0 conn(0x7f43c41541a0 legacy=0x7f43c40c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.497+0000 7f43c8b9c700 1 -- 172.21.15.33:0/4130995989 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40c1130 con 0x7f43c40c07b0 2023-11-06T22:18:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.497+0000 7f43c37fe700 1 --2- 172.21.15.33:0/4130995989 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 0x7f43c40c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.497+0000 7f43c8b9c700 1 --2- 172.21.15.33:0/4130995989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c40c07b0 0x7f43c40c0c40 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7f43bc009f20 tx=0x7f43bc009840).ready entity=mon.0 client_cookie=2819195aef3b91bc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:29.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.497+0000 7f43c2ffd700 1 -- 172.21.15.33:0/4130995989 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00e070 con 0x7f43c40c07b0 2023-11-06T22:18:29.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.497+0000 7f43c2ffd700 1 -- 172.21.15.33:0/4130995989 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc00ba80 con 0x7f43c40c07b0 2023-11-06T22:18:29.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.497+0000 7f43c2ffd700 1 -- 172.21.15.33:0/4130995989 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00be00 con 0x7f43c40c07b0 2023-11-06T22:18:29.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.498+0000 7f43ca5ff700 1 -- 172.21.15.33:0/4130995989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c40c07b0 msgr2=0x7f43c40c0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.498+0000 7f43ca5ff700 1 --2- 172.21.15.33:0/4130995989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c40c07b0 0x7f43c40c0c40 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7f43bc009f20 tx=0x7f43bc009840).stop 2023-11-06T22:18:29.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.498+0000 7f43ca5ff700 1 -- 172.21.15.33:0/4130995989 shutdown_connections 2023-11-06T22:18:29.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.498+0000 7f43ca5ff700 1 --2- 172.21.15.33:0/4130995989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c40c07b0 0x7f43c40c0c40 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.498+0000 7f43ca5ff700 1 --2- 172.21.15.33:0/4130995989 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 0x7f43c40c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.498+0000 7f43ca5ff700 1 -- 172.21.15.33:0/4130995989 >> 172.21.15.33:0/4130995989 conn(0x7f43c40944e0 msgr2=0x7f43c406a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.498+0000 7f43ca5ff700 1 -- 172.21.15.33:0/4130995989 shutdown_connections 2023-11-06T22:18:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.498+0000 7f43ca5ff700 1 -- 172.21.15.33:0/4130995989 wait complete. 2023-11-06T22:18:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.499+0000 7f43ca5ff700 1 Processor -- start 2023-11-06T22:18:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.500+0000 7f43ca5ff700 1 -- start start 2023-11-06T22:18:29.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.500+0000 7f43ca5ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 0x7f43c413c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43ca5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43c40c07b0 0x7f43c4152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43ca5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c41541a0 0x7f43c40645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43ca5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40c4060 con 0x7f43c41541a0 2023-11-06T22:18:29.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43ca5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40c2fc0 con 0x7f43c40c07b0 2023-11-06T22:18:29.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43ca5ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40c32a0 con 0x7f43c40a9ee0 2023-11-06T22:18:29.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c8b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c41541a0 0x7f43c40645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c8b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c41541a0 0x7f43c40645a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42660/0 (socket says 172.21.15.33:42660) 2023-11-06T22:18:29.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c8b9c700 1 -- 172.21.15.33:0/3242119678 learned_addr learned my addr 172.21.15.33:0/3242119678 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:29.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c37fe700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43c40c07b0 0x7f43c4152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c3fff700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 0x7f43c413c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c8b9c700 1 -- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 msgr2=0x7f43c413c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c8b9c700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 0x7f43c413c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c8b9c700 1 -- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43c40c07b0 msgr2=0x7f43c4152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c8b9c700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43c40c07b0 0x7f43c4152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c8b9c700 1 -- 172.21.15.33:0/3242119678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc009ae0 con 0x7f43c41541a0 2023-11-06T22:18:29.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.501+0000 7f43c8b9c700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c41541a0 0x7f43c40645a0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7f43bc00b6c0 tx=0x7f43bc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:29.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.502+0000 7f43abfff700 1 -- 172.21.15.33:0/3242119678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc0032f0 con 0x7f43c41541a0 2023-11-06T22:18:29.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.502+0000 7f43abfff700 1 -- 172.21.15.33:0/3242119678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc01e470 con 0x7f43c41541a0 2023-11-06T22:18:29.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.502+0000 7f43ca5ff700 1 -- 172.21.15.33:0/3242119678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c40c3580 con 0x7f43c41541a0 2023-11-06T22:18:29.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.502+0000 7f43abfff700 1 -- 172.21.15.33:0/3242119678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc01e810 con 0x7f43c41541a0 2023-11-06T22:18:29.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.502+0000 7f43ca5ff700 1 -- 172.21.15.33:0/3242119678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c40c3b40 con 0x7f43c41541a0 2023-11-06T22:18:29.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.503+0000 7f43ca5ff700 1 -- 172.21.15.33:0/3242119678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c40475a0 con 0x7f43c41541a0 2023-11-06T22:18:29.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.504+0000 7f43abfff700 1 -- 172.21.15.33:0/3242119678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f43bc003040 con 0x7f43c41541a0 2023-11-06T22:18:29.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.504+0000 7f43abfff700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f43b005f270 0x7f43b0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:29.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.505+0000 7f43abfff700 1 -- 172.21.15.33:0/3242119678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f43bc07ba00 con 0x7f43c41541a0 2023-11-06T22:18:29.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.505+0000 7f43c3fff700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f43b005f270 0x7f43b0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:29.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.505+0000 7f43c3fff700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f43b005f270 0x7f43b0061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f43b4003610 tx=0x7f43b4009040).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:29.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.508+0000 7f43abfff700 1 -- 172.21.15.33:0/3242119678 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f43bc04cb10 con 0x7f43c41541a0 2023-11-06T22:18:29.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.736+0000 7f43ca5ff700 1 -- 172.21.15.33:0/3242119678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f43c40c3d30 con 0x7f43c41541a0 2023-11-06T22:18:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.736+0000 7f43abfff700 1 -- 172.21.15.33:0/3242119678 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7f43bc023030 con 0x7f43c41541a0 2023-11-06T22:18:29.738 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T22:18:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.738+0000 7f43ca5ff700 1 -- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f43b005f270 msgr2=0x7f43b0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.738+0000 7f43ca5ff700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f43b005f270 0x7f43b0061730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f43b4003610 tx=0x7f43b4009040).stop 2023-11-06T22:18:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.738+0000 7f43ca5ff700 1 -- 172.21.15.33:0/3242119678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c41541a0 msgr2=0x7f43c40645a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.738+0000 7f43ca5ff700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c41541a0 0x7f43c40645a0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7f43bc00b6c0 tx=0x7f43bc009840).stop 2023-11-06T22:18:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.738+0000 7f43ca5ff700 1 -- 172.21.15.33:0/3242119678 shutdown_connections 2023-11-06T22:18:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.738+0000 7f43ca5ff700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f43b005f270 0x7f43b0061730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.738+0000 7f43ca5ff700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43c40a9ee0 0x7f43c413c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.738+0000 7f43ca5ff700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c41541a0 0x7f43c40645a0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.738+0000 7f43ca5ff700 1 --2- 172.21.15.33:0/3242119678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43c40c07b0 0x7f43c4152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.738+0000 7f43ca5ff700 1 -- 172.21.15.33:0/3242119678 >> 172.21.15.33:0/3242119678 conn(0x7f43c40944e0 msgr2=0x7f43c40ca2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.739+0000 7f43ca5ff700 1 -- 172.21.15.33:0/3242119678 shutdown_connections 2023-11-06T22:18:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:29.739+0000 7f43ca5ff700 1 -- 172.21.15.33:0/3242119678 wait complete. 2023-11-06T22:18:29.755 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-06T22:18:29.756 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T22:18:29.756 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-11-06T22:18:29.757 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T22:18:29.758 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T22:18:29.758 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-06T22:18:29.759 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-06T22:18:29.759 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T22:18:29.759 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T22:18:29.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:29.760 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-11-06T22:18:29.808 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T22:18:29.808 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T22:18:29.809 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T22:18:29.809 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:18:29.809 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-11-06T22:18:29.809 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:18:29.810 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T22:18:29.810 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:18:29.810 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-11-06T22:18:29.810 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T22:18:29.810 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:18:29.811 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-11-06T22:18:29.811 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T22:18:29.811 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:18:29.811 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-11-06T22:18:29.811 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-06T22:18:29.812 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 36634sec preferred_lft 36634sec 2023-11-06T22:18:29.812 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-11-06T22:18:29.812 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:18:29.812 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-06T22:18:29.812 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-11-06T22:18:29.813 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-11-06T22:18:29.814 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T22:18:29.814 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff 2023-11-06T22:18:29.814 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T22:18:29.814 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:18:29.815 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::e3:1aff:fe0a:b214/64 scope link 2023-11-06T22:18:29.815 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:18:29.815 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T22:18:29.815 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 6a:d0:c4:d3:1a:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T22:18:29.815 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::68d0:c4ff:fed3:1a25/64 scope link 2023-11-06T22:18:29.815 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:18:29.816 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T22:18:29.816 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 1e:bd:10:cc:37:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T22:18:29.816 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::1cbd:10ff:fecc:375d/64 scope link 2023-11-06T22:18:29.816 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-11-06T22:18:29.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:29.818 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T22:18:29.889 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-11-06T22:18:29.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:29.893 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-11-06T22:18:29.944 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-11-06T22:18:29.944 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T22:18:29.944 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-06T22:18:29.944 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-06T22:18:29.945 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T22:18:29.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:29.945 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-11-06T22:18:29.946 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-11-06T22:18:29.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T22:18:29.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T22:18:29.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T22:18:29.947 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-11-06T22:18:30.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:30.122 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T22:18:30.171 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T22:18:30.171 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T22:18:30.172 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T22:18:30.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:30.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-11-06T22:18:30.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:30.246 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-06T22:18:30.294 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-06T22:18:30.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T22:18:30.437 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T22:18:30.437 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T22:18:30.463 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-06T22:18:30.463 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-06T22:18:30.468 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-06T22:18:30.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:30.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T22:18:30.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-06T22:18:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.738+0000 7f9c483ff700 1 Processor -- start 2023-11-06T22:18:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.738+0000 7f9c483ff700 1 -- start start 2023-11-06T22:18:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.739+0000 7f9c483ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9c40154210 0x7f9c400a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.739+0000 7f9c483ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9c400bfd60 0x7f9c400c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.739+0000 7f9c483ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c400c06d0 con 0x7f9c400c9f00 2023-11-06T22:18:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.739+0000 7f9c483ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c400caa80 con 0x7f9c400bfd60 2023-11-06T22:18:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.739+0000 7f9c483ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4005efc0 con 0x7f9c40154210 2023-11-06T22:18:30.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.740+0000 7f9c4599a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9c400bfd60 0x7f9c400c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:30.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.740+0000 7f9c4599a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9c400bfd60 0x7f9c400c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42668/0 (socket says 172.21.15.33:42668) 2023-11-06T22:18:30.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.740+0000 7f9c4599a700 1 -- 172.21.15.33:0/2222784237 learned_addr learned my addr 172.21.15.33:0/2222784237 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:30.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.740+0000 7f9c4619b700 1 --2- 172.21.15.33:0/2222784237 >> v2:172.21.15.69:3300/0 conn(0x7f9c40154210 0x7f9c400a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:30.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.740+0000 7f9c4599a700 1 -- 172.21.15.33:0/2222784237 >> v2:172.21.15.69:3300/0 conn(0x7f9c40154210 msgr2=0x7f9c400a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:30.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.740+0000 7f9c4599a700 1 --2- 172.21.15.33:0/2222784237 >> v2:172.21.15.69:3300/0 conn(0x7f9c40154210 0x7f9c400a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:30.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.740+0000 7f9c4599a700 1 -- 172.21.15.33:0/2222784237 >> v1:172.21.15.69:6789/0 conn(0x7f9c400c9f00 legacy=0x7f9c400ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:30.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.740+0000 7f9c4599a700 1 -- 172.21.15.33:0/2222784237 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c400c8ea0 con 0x7f9c400bfd60 2023-11-06T22:18:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.740+0000 7f9c4599a700 1 --2- 172.21.15.33:0/2222784237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 0x7f9c400c01e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7f9c400adab0 tx=0x7f9c300090f0).ready entity=mon.0 client_cookie=6e3b6464418bea38 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c45199700 1 -- 172.21.15.33:0/2222784237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000b5a0 con 0x7f9c400bfd60 2023-11-06T22:18:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c45199700 1 -- 172.21.15.33:0/2222784237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3000b700 con 0x7f9c400bfd60 2023-11-06T22:18:30.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c45199700 1 -- 172.21.15.33:0/2222784237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000ba80 con 0x7f9c400bfd60 2023-11-06T22:18:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c483ff700 1 -- 172.21.15.33:0/2222784237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 msgr2=0x7f9c400c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c483ff700 1 --2- 172.21.15.33:0/2222784237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 0x7f9c400c01e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7f9c400adab0 tx=0x7f9c300090f0).stop 2023-11-06T22:18:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c483ff700 1 -- 172.21.15.33:0/2222784237 shutdown_connections 2023-11-06T22:18:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c483ff700 1 --2- 172.21.15.33:0/2222784237 >> v2:172.21.15.69:3300/0 conn(0x7f9c40154210 0x7f9c400a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c483ff700 1 --2- 172.21.15.33:0/2222784237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 0x7f9c400c01e0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c483ff700 1 -- 172.21.15.33:0/2222784237 >> 172.21.15.33:0/2222784237 conn(0x7f9c4005e990 msgr2=0x7f9c400cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c483ff700 1 -- 172.21.15.33:0/2222784237 shutdown_connections 2023-11-06T22:18:30.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.741+0000 7f9c483ff700 1 -- 172.21.15.33:0/2222784237 wait complete. 2023-11-06T22:18:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.743+0000 7f9c483ff700 1 Processor -- start 2023-11-06T22:18:30.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.743+0000 7f9c483ff700 1 -- start start 2023-11-06T22:18:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c483ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 0x7f9c40064850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c483ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c400c9f00 0x7f9c400a8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c483ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c40154210 0x7f9c4014e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c483ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400ad6a0 con 0x7f9c400bfd60 2023-11-06T22:18:30.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c483ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401573a0 con 0x7f9c40154210 2023-11-06T22:18:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c483ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c40156ae0 con 0x7f9c400c9f00 2023-11-06T22:18:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c4619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 0x7f9c40064850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c4619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 0x7f9c40064850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42672/0 (socket says 172.21.15.33:42672) 2023-11-06T22:18:30.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c4619b700 1 -- 172.21.15.33:0/551404123 learned_addr learned my addr 172.21.15.33:0/551404123 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:30.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c4599a700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c400c9f00 0x7f9c400a8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:30.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.744+0000 7f9c4699c700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c40154210 0x7f9c4014e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:30.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c4619b700 1 -- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c400c9f00 msgr2=0x7f9c400a8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:30.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c4619b700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c400c9f00 0x7f9c400a8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:30.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c4619b700 1 -- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c40154210 msgr2=0x7f9c4014e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:30.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c4619b700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c40154210 0x7f9c4014e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:30.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c4619b700 1 -- 172.21.15.33:0/551404123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c30009370 con 0x7f9c400bfd60 2023-11-06T22:18:30.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c4699c700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c40154210 0x7f9c4014e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:30.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c4619b700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 0x7f9c40064850 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7f9c3c00d8c0 tx=0x7f9c3c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:30.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c367fc700 1 -- 172.21.15.33:0/551404123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c011590 con 0x7f9c400bfd60 2023-11-06T22:18:30.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c367fc700 1 -- 172.21.15.33:0/551404123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c0116f0 con 0x7f9c400bfd60 2023-11-06T22:18:30.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c483ff700 1 -- 172.21.15.33:0/551404123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c400c5200 con 0x7f9c400bfd60 2023-11-06T22:18:30.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c367fc700 1 -- 172.21.15.33:0/551404123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c011a70 con 0x7f9c400bfd60 2023-11-06T22:18:30.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.745+0000 7f9c483ff700 1 -- 172.21.15.33:0/551404123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c401570b0 con 0x7f9c400bfd60 2023-11-06T22:18:30.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.746+0000 7f9c483ff700 1 -- 172.21.15.33:0/551404123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c40065000 con 0x7f9c400bfd60 2023-11-06T22:18:30.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.747+0000 7f9c367fc700 1 -- 172.21.15.33:0/551404123 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f9c3c01c460 con 0x7f9c400bfd60 2023-11-06T22:18:30.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.748+0000 7f9c367fc700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9c2405f270 0x7f9c24061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:30.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.748+0000 7f9c367fc700 1 -- 172.21.15.33:0/551404123 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f9c3c07a1d0 con 0x7f9c400bfd60 2023-11-06T22:18:30.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.748+0000 7f9c4599a700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9c2405f270 0x7f9c24061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:30.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.749+0000 7f9c4599a700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9c2405f270 0x7f9c24061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9c30009130 tx=0x7f9c30015000).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:30.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.752+0000 7f9c367fc700 1 -- 172.21.15.33:0/551404123 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9c3c04b1d0 con 0x7f9c400bfd60 2023-11-06T22:18:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.964+0000 7f9c483ff700 1 -- 172.21.15.33:0/551404123 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c4006c6c0 con 0x7f9c2405f270 2023-11-06T22:18:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.966+0000 7f9c367fc700 1 -- 172.21.15.33:0/551404123 <== mgr.9310 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9c4006c6c0 con 0x7f9c2405f270 2023-11-06T22:18:30.968 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-11-06T22:18:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-11-06T22:18:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephfs" 2023-11-06T22:18:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-11-06T22:18:30.969 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-11-06T22:18:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 -- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9c2405f270 msgr2=0x7f9c24061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9c2405f270 0x7f9c24061730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9c30009130 tx=0x7f9c30015000).stop 2023-11-06T22:18:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 -- 172.21.15.33:0/551404123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 msgr2=0x7f9c40064850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 0x7f9c40064850 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7f9c3c00d8c0 tx=0x7f9c3c008040).stop 2023-11-06T22:18:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 -- 172.21.15.33:0/551404123 shutdown_connections 2023-11-06T22:18:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f9c2405f270 0x7f9c24061730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c40154210 0x7f9c4014e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400bfd60 0x7f9c40064850 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 --2- 172.21.15.33:0/551404123 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f9c400c9f00 0x7f9c400a8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 -- 172.21.15.33:0/551404123 >> 172.21.15.33:0/551404123 conn(0x7f9c4005e990 msgr2=0x7f9c4014fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 -- 172.21.15.33:0/551404123 shutdown_connections 2023-11-06T22:18:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:30.968+0000 7f9c483ff700 1 -- 172.21.15.33:0/551404123 wait complete. 2023-11-06T22:18:30.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-06T22:18:31.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:18:31.122 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:18:31.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.188+0000 7f30823ff700 1 Processor -- start 2023-11-06T22:18:31.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.188+0000 7f30823ff700 1 -- start start 2023-11-06T22:18:31.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f30823ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f307c0bbe40 0x7f307c0c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f30823ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f307c0c18d0 0x7f307c065580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f30823ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c0650f0 con 0x7f307c1542e0 2023-11-06T22:18:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f30823ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0647c0 con 0x7f307c0bbe40 2023-11-06T22:18:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f30823ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c064aa0 con 0x7f307c0c18d0 2023-11-06T22:18:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f307b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f307c0bbe40 0x7f307c0c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f307b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f307c0bbe40 0x7f307c0c13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42674/0 (socket says 172.21.15.33:42674) 2023-11-06T22:18:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f307b7fe700 1 -- 172.21.15.33:0/1238677426 learned_addr learned my addr 172.21.15.33:0/1238677426 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f308099c700 1 --2- 172.21.15.33:0/1238677426 >> v2:172.21.15.69:3300/0 conn(0x7f307c0c18d0 0x7f307c065580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f307b7fe700 1 -- 172.21.15.33:0/1238677426 >> v2:172.21.15.69:3300/0 conn(0x7f307c0c18d0 msgr2=0x7f307c065580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f307b7fe700 1 --2- 172.21.15.33:0/1238677426 >> v2:172.21.15.69:3300/0 conn(0x7f307c0c18d0 0x7f307c065580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:31.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f307b7fe700 1 -- 172.21.15.33:0/1238677426 >> v1:172.21.15.69:6789/0 conn(0x7f307c1542e0 legacy=0x7f307c0bc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.190+0000 7f307b7fe700 1 -- 172.21.15.33:0/1238677426 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c064d80 con 0x7f307c0bbe40 2023-11-06T22:18:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.191+0000 7f307b7fe700 1 --2- 172.21.15.33:0/1238677426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c0bbe40 0x7f307c0c13e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f307c0c69e0 tx=0x7f30700090f0).ready entity=mon.0 client_cookie=bbb5bf1843559451 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.191+0000 7f307affd700 1 -- 172.21.15.33:0/1238677426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307000b520 con 0x7f307c0bbe40 2023-11-06T22:18:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.191+0000 7f307affd700 1 -- 172.21.15.33:0/1238677426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307000b680 con 0x7f307c0bbe40 2023-11-06T22:18:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.191+0000 7f307affd700 1 -- 172.21.15.33:0/1238677426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307000ba00 con 0x7f307c0bbe40 2023-11-06T22:18:31.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.191+0000 7f30823ff700 1 -- 172.21.15.33:0/1238677426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c0bbe40 msgr2=0x7f307c0c13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:31.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.191+0000 7f30823ff700 1 --2- 172.21.15.33:0/1238677426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c0bbe40 0x7f307c0c13e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f307c0c69e0 tx=0x7f30700090f0).stop 2023-11-06T22:18:31.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.191+0000 7f30823ff700 1 -- 172.21.15.33:0/1238677426 shutdown_connections 2023-11-06T22:18:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.191+0000 7f30823ff700 1 --2- 172.21.15.33:0/1238677426 >> v2:172.21.15.69:3300/0 conn(0x7f307c0c18d0 0x7f307c065580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.191+0000 7f30823ff700 1 --2- 172.21.15.33:0/1238677426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c0bbe40 0x7f307c0c13e0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.191+0000 7f30823ff700 1 -- 172.21.15.33:0/1238677426 >> 172.21.15.33:0/1238677426 conn(0x7f307c0c6a40 msgr2=0x7f307c0c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.192+0000 7f30823ff700 1 -- 172.21.15.33:0/1238677426 shutdown_connections 2023-11-06T22:18:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.192+0000 7f30823ff700 1 -- 172.21.15.33:0/1238677426 wait complete. 2023-11-06T22:18:31.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.194+0000 7f30823ff700 1 Processor -- start 2023-11-06T22:18:31.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.194+0000 7f30823ff700 1 -- start start 2023-11-06T22:18:31.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.195+0000 7f30823ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f307c0bbe40 0x7f307c14bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:31.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.195+0000 7f30823ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f307c0c18d0 0x7f307c0bf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:31.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.195+0000 7f30823ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c1542e0 0x7f307c149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:31.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.195+0000 7f30823ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c154ff0 con 0x7f307c1542e0 2023-11-06T22:18:31.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.195+0000 7f30823ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0c0180 con 0x7f307c0bbe40 2023-11-06T22:18:31.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.195+0000 7f30823ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c1546c0 con 0x7f307c0c18d0 2023-11-06T22:18:31.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.195+0000 7f308099c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c1542e0 0x7f307c149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:31.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.195+0000 7f308099c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c1542e0 0x7f307c149a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42686/0 (socket says 172.21.15.33:42686) 2023-11-06T22:18:31.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.195+0000 7f308099c700 1 -- 172.21.15.33:0/551443491 learned_addr learned my addr 172.21.15.33:0/551443491 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:31.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.195+0000 7f307bfff700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f307c0bbe40 0x7f307c14bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:31.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f307b7fe700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f307c0c18d0 0x7f307c0bf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:31.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f308099c700 1 -- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f307c0c18d0 msgr2=0x7f307c0bf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:31.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f308099c700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f307c0c18d0 0x7f307c0bf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:31.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f308099c700 1 -- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f307c0bbe40 msgr2=0x7f307c14bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f308099c700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f307c0bbe40 0x7f307c14bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f308099c700 1 -- 172.21.15.33:0/551443491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3070009370 con 0x7f307c1542e0 2023-11-06T22:18:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f307b7fe700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f307c0c18d0 0x7f307c0bf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f308099c700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c1542e0 0x7f307c149a70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 rx=0x7f3074009e90 tx=0x7f307400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f3063fff700 1 -- 172.21.15.33:0/551443491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400cb50 con 0x7f307c1542e0 2023-11-06T22:18:31.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f3063fff700 1 -- 172.21.15.33:0/551443491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307400f040 con 0x7f307c1542e0 2023-11-06T22:18:31.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f3063fff700 1 -- 172.21.15.33:0/551443491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400cec0 con 0x7f307c1542e0 2023-11-06T22:18:31.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f30823ff700 1 -- 172.21.15.33:0/551443491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c1549a0 con 0x7f307c1542e0 2023-11-06T22:18:31.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.196+0000 7f30823ff700 1 -- 172.21.15.33:0/551443491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c0bfe90 con 0x7f307c1542e0 2023-11-06T22:18:31.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.198+0000 7f30823ff700 1 -- 172.21.15.33:0/551443491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c155160 con 0x7f307c1542e0 2023-11-06T22:18:31.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.198+0000 7f3063fff700 1 -- 172.21.15.33:0/551443491 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 97) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3074013610 con 0x7f307c1542e0 2023-11-06T22:18:31.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.199+0000 7f3063fff700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f306805f270 0x7f3068061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:31.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.199+0000 7f3063fff700 1 -- 172.21.15.33:0/551443491 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7f307407ad90 con 0x7f307c1542e0 2023-11-06T22:18:31.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.199+0000 7f307bfff700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f306805f270 0x7f3068061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:31.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.200+0000 7f307bfff700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f306805f270 0x7f3068061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f306c003900 tx=0x7f306c007d40).ready entity=mgr.9310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:31.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.203+0000 7f3063fff700 1 -- 172.21.15.33:0/551443491 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f307404f820 con 0x7f307c1542e0 2023-11-06T22:18:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.435+0000 7f30823ff700 1 -- 172.21.15.33:0/551443491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f307c05f160 con 0x7f307c1542e0 2023-11-06T22:18:31.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.872+0000 7f3063fff700 1 -- 172.21.15.33:0/551443491 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 98) v1 ==== 75565+0+0 (secure 0 0 0) 0x7f30740788f0 con 0x7f307c1542e0 2023-11-06T22:18:31.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.874+0000 7f3063fff700 1 -- 172.21.15.33:0/551443491 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v98) v1 ==== 92+0+0 (secure 0 0 0) 0x7f3074071da0 con 0x7f307c1542e0 2023-11-06T22:18:31.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 -- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f306805f270 msgr2=0x7f3068061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:31.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f306805f270 0x7f3068061730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f306c003900 tx=0x7f306c007d40).stop 2023-11-06T22:18:31.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 -- 172.21.15.33:0/551443491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c1542e0 msgr2=0x7f307c149a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:31.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c1542e0 0x7f307c149a70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 rx=0x7f3074009e90 tx=0x7f307400b040).stop 2023-11-06T22:18:31.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 -- 172.21.15.33:0/551443491 shutdown_connections 2023-11-06T22:18:31.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f306805f270 0x7f3068061730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:31.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307c1542e0 0x7f307c149a70 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:31.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f307c0bbe40 0x7f307c14bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:31.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 --2- 172.21.15.33:0/551443491 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f307c0c18d0 0x7f307c0bf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:31.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 -- 172.21.15.33:0/551443491 >> 172.21.15.33:0/551443491 conn(0x7f307c0c6a40 msgr2=0x7f307c14c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:31.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.877+0000 7f30823ff700 1 -- 172.21.15.33:0/551443491 shutdown_connections 2023-11-06T22:18:31.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:31.878+0000 7f30823ff700 1 -- 172.21.15.33:0/551443491 wait complete. 2023-11-06T22:18:31.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-06T22:18:32.091 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:32.089+0000 7fb11acb0680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:18:32.098 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:32.096+0000 7fde086ac680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:18:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.104+0000 7ffb035ff700 1 Processor -- start 2023-11-06T22:18:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.104+0000 7ffb035ff700 1 -- start start 2023-11-06T22:18:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.105+0000 7ffb035ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0ab1d0 0x7ffafc0bdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.105+0000 7ffb035ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc152940 con 0x7ffafc0be0a0 2023-11-06T22:18:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.105+0000 7ffb035ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc153790 con 0x7ffafc0aadf0 2023-11-06T22:18:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.105+0000 7ffb035ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0be530 con 0x7ffafc0ab1d0 2023-11-06T22:18:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffb00b9a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0ab1d0 0x7ffafc0bdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffb01b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffafc0be0a0 0x7ffafc0c36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33558/0 (socket says 172.21.15.33:33558) 2023-11-06T22:18:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffb0139b700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7ffafc0aadf0 0x7ffafc068350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:37204/0 (socket says 172.21.15.33:37204) 2023-11-06T22:18:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffb01b9c700 1 -- 172.21.15.33:0/4140325443 learned_addr learned my addr 172.21.15.33:0/4140325443 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:32.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffaf3fff700 1 -- 172.21.15.33:0/4140325443 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325146235 0 0) 0x7ffafc153790 con 0x7ffafc0aadf0 2023-11-06T22:18:32.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffaf3fff700 1 -- 172.21.15.33:0/4140325443 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae80033f0 con 0x7ffafc0aadf0 2023-11-06T22:18:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffaf3fff700 1 -- 172.21.15.33:0/4140325443 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345749904 0 0) 0x7ffaf4001490 con 0x7ffafc0be0a0 2023-11-06T22:18:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffb00b9a700 1 -- 172.21.15.33:0/4140325443 >> v1:172.21.15.69:6789/0 conn(0x7ffafc0aadf0 legacy=0x7ffafc068350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffb00b9a700 1 -- 172.21.15.33:0/4140325443 >> v1:172.21.15.33:6789/0 conn(0x7ffafc0be0a0 legacy=0x7ffafc0c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffb00b9a700 1 -- 172.21.15.33:0/4140325443 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc152b80 con 0x7ffafc0ab1d0 2023-11-06T22:18:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.106+0000 7ffb00b9a700 1 --2- 172.21.15.33:0/4140325443 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0ab1d0 0x7ffafc0bdbb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7ffaec009790 tx=0x7ffaec009060).ready entity=mon.2 client_cookie=a7e19c18fa1aa39f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.107+0000 7ffaf3fff700 1 -- 172.21.15.33:0/4140325443 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00f510 con 0x7ffafc0ab1d0 2023-11-06T22:18:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.107+0000 7ffaf3fff700 1 -- 172.21.15.33:0/4140325443 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaec00f670 con 0x7ffafc0ab1d0 2023-11-06T22:18:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.107+0000 7ffaf3fff700 1 -- 172.21.15.33:0/4140325443 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00f9f0 con 0x7ffafc0ab1d0 2023-11-06T22:18:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.107+0000 7ffb035ff700 1 -- 172.21.15.33:0/4140325443 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0ab1d0 msgr2=0x7ffafc0bdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.107+0000 7ffb035ff700 1 --2- 172.21.15.33:0/4140325443 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0ab1d0 0x7ffafc0bdbb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7ffaec009790 tx=0x7ffaec009060).stop 2023-11-06T22:18:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.107+0000 7ffb035ff700 1 -- 172.21.15.33:0/4140325443 shutdown_connections 2023-11-06T22:18:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.107+0000 7ffb035ff700 1 --2- 172.21.15.33:0/4140325443 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0ab1d0 0x7ffafc0bdbb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.107+0000 7ffb035ff700 1 -- 172.21.15.33:0/4140325443 >> 172.21.15.33:0/4140325443 conn(0x7ffafc0c8c50 msgr2=0x7ffafc0c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.107+0000 7ffb035ff700 1 -- 172.21.15.33:0/4140325443 shutdown_connections 2023-11-06T22:18:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.107+0000 7ffb035ff700 1 -- 172.21.15.33:0/4140325443 wait complete. 2023-11-06T22:18:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.109+0000 7ffb035ff700 1 Processor -- start 2023-11-06T22:18:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.109+0000 7ffb035ff700 1 -- start start 2023-11-06T22:18:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb035ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0aadf0 0x7ffafc168470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb035ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffafc0ab1d0 0x7ffafc151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb035ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffafc0be0a0 0x7ffafc164630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb035ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc137f30 con 0x7ffafc0ab1d0 2023-11-06T22:18:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb035ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc064080 con 0x7ffafc0be0a0 2023-11-06T22:18:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb035ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc168a50 con 0x7ffafc0aadf0 2023-11-06T22:18:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb0139b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0aadf0 0x7ffafc168470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb00b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffafc0ab1d0 0x7ffafc151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb01b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffafc0be0a0 0x7ffafc164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb00b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffafc0ab1d0 0x7ffafc151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42706/0 (socket says 172.21.15.33:42706) 2023-11-06T22:18:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb0139b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0aadf0 0x7ffafc168470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:42096/0 (socket says 172.21.15.33:42096) 2023-11-06T22:18:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.110+0000 7ffb00b9a700 1 -- 172.21.15.33:0/179299239 learned_addr learned my addr 172.21.15.33:0/179299239 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffb0139b700 1 -- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffafc0be0a0 msgr2=0x7ffafc164630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffb0139b700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffafc0be0a0 0x7ffafc164630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffb0139b700 1 -- 172.21.15.33:0/179299239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffafc0ab1d0 msgr2=0x7ffafc151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffb0139b700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffafc0ab1d0 0x7ffafc151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffb0139b700 1 -- 172.21.15.33:0/179299239 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec0092a0 con 0x7ffafc0aadf0 2023-11-06T22:18:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffb00b9a700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffafc0ab1d0 0x7ffafc151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffb0139b700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0aadf0 0x7ffafc168470 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7ffaf800c440 tx=0x7ffaf8007c40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:32.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffaf17fa700 1 -- 172.21.15.33:0/179299239 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80070d0 con 0x7ffafc0aadf0 2023-11-06T22:18:32.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffaf17fa700 1 -- 172.21.15.33:0/179299239 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf8007230 con 0x7ffafc0aadf0 2023-11-06T22:18:32.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffb035ff700 1 -- 172.21.15.33:0/179299239 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc168d30 con 0x7ffafc0aadf0 2023-11-06T22:18:32.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffaf17fa700 1 -- 172.21.15.33:0/179299239 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80056e0 con 0x7ffafc0aadf0 2023-11-06T22:18:32.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.111+0000 7ffb035ff700 1 -- 172.21.15.33:0/179299239 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc1692f0 con 0x7ffafc0aadf0 2023-11-06T22:18:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.112+0000 7ffb035ff700 1 -- 172.21.15.33:0/179299239 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffafc066000 con 0x7ffafc0aadf0 2023-11-06T22:18:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.113+0000 7ffaf17fa700 1 -- 172.21.15.33:0/179299239 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 98) v1 ==== 75565+0+0 (secure 0 0 0) 0x7ffaf8005b10 con 0x7ffafc0aadf0 2023-11-06T22:18:32.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.115+0000 7ffaf17fa700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffae005f2c0 0x7ffae0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.115+0000 7ffaf17fa700 1 -- 172.21.15.33:0/179299239 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7ffaf8079ac0 con 0x7ffafc0aadf0 2023-11-06T22:18:32.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.115+0000 7ffb00b9a700 1 -- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffae005f2c0 msgr2=0x7ffae0061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:32.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.115+0000 7ffb00b9a700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffae005f2c0 0x7ffae0061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:32.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.119+0000 7ffaf17fa700 1 -- 172.21.15.33:0/179299239 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffaf804ab20 con 0x7ffafc0aadf0 2023-11-06T22:18:32.171 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:32.168+0000 7fb11acb0680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:18:32.183 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:32.182+0000 7fde086ac680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:18:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.303+0000 7ffb035ff700 1 -- 172.21.15.33:0/179299239 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ffafc151fd0 con 0x7ffafc0aadf0 2023-11-06T22:18:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.308+0000 7ffaf17fa700 1 -- 172.21.15.33:0/179299239 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v43)=0 v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7ffaf8079020 con 0x7ffafc0aadf0 2023-11-06T22:18:32.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 -- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffae005f2c0 msgr2=0x7ffae0061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:32.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffae005f2c0 0x7ffae0061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:32.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 -- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0aadf0 msgr2=0x7ffafc168470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0aadf0 0x7ffafc168470 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7ffaf800c440 tx=0x7ffaf8007c40).stop 2023-11-06T22:18:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 -- 172.21.15.33:0/179299239 shutdown_connections 2023-11-06T22:18:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ffae005f2c0 0x7ffae0061780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ffafc0aadf0 0x7ffafc168470 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffafc0be0a0 0x7ffafc164630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 --2- 172.21.15.33:0/179299239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffafc0ab1d0 0x7ffafc151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 -- 172.21.15.33:0/179299239 >> 172.21.15.33:0/179299239 conn(0x7ffafc0c8c50 msgr2=0x7ffafc154000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.310+0000 7ffb035ff700 1 -- 172.21.15.33:0/179299239 shutdown_connections 2023-11-06T22:18:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.311+0000 7ffb035ff700 1 -- 172.21.15.33:0/179299239 wait complete. 2023-11-06T22:18:32.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:18:32.331 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-06T22:18:32.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-11-06T22:18:32.496 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:32.494+0000 7fb11acb0680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:18:32.539 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:32.538+0000 7fde086ac680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:18:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.590+0000 7efd4279d700 1 Processor -- start 2023-11-06T22:18:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.590+0000 7efd4279d700 1 -- start start 2023-11-06T22:18:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.591+0000 7efd4279d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efd3c0ca410 0x7efd3c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.591+0000 7efd4279d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c0c60c0 0x7efd3c064510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.591+0000 7efd4279d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd3c064a00 0x7efd3c14def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd4279d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c065240 con 0x7efd3c064a00 2023-11-06T22:18:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd4279d700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c154630 con 0x7efd3c0ca410 2023-11-06T22:18:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd4279d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c155170 con 0x7efd3c0c60c0 2023-11-06T22:18:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd41f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd3c064a00 0x7efd3c14def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd41f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd3c064a00 0x7efd3c14def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42716/0 (socket says 172.21.15.33:42716) 2023-11-06T22:18:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd41f9c700 1 -- 172.21.15.33:0/2568396968 learned_addr learned my addr 172.21.15.33:0/2568396968 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:32.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd4179b700 1 --2- 172.21.15.33:0/2568396968 >> v2:172.21.15.69:3300/0 conn(0x7efd3c0ca410 0x7efd3c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:32.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd40f9a700 1 --2- 172.21.15.33:0/2568396968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c0c60c0 0x7efd3c064510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:32.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd41f9c700 1 -- 172.21.15.33:0/2568396968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c0c60c0 msgr2=0x7efd3c064510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:32.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd41f9c700 1 --2- 172.21.15.33:0/2568396968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c0c60c0 0x7efd3c064510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd41f9c700 1 -- 172.21.15.33:0/2568396968 >> v2:172.21.15.69:3300/0 conn(0x7efd3c0ca410 msgr2=0x7efd3c0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:32.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd41f9c700 1 --2- 172.21.15.33:0/2568396968 >> v2:172.21.15.69:3300/0 conn(0x7efd3c0ca410 0x7efd3c0c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.592+0000 7efd41f9c700 1 -- 172.21.15.33:0/2568396968 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0ca7f0 con 0x7efd3c064a00 2023-11-06T22:18:32.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd41f9c700 1 --2- 172.21.15.33:0/2568396968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c064a00 0x7efd3c14def0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7efd3c083690 tx=0x7efd34009840).ready entity=mon.0 client_cookie=76cf794e830dfe4e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:32.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd33fff700 1 -- 172.21.15.33:0/2568396968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400e070 con 0x7efd3c064a00 2023-11-06T22:18:32.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd33fff700 1 -- 172.21.15.33:0/2568396968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3400b970 con 0x7efd3c064a00 2023-11-06T22:18:32.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd33fff700 1 -- 172.21.15.33:0/2568396968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400bcf0 con 0x7efd3c064a00 2023-11-06T22:18:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd4279d700 1 -- 172.21.15.33:0/2568396968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c064a00 msgr2=0x7efd3c14def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd4279d700 1 --2- 172.21.15.33:0/2568396968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c064a00 0x7efd3c14def0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7efd3c083690 tx=0x7efd34009840).stop 2023-11-06T22:18:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd4279d700 1 -- 172.21.15.33:0/2568396968 shutdown_connections 2023-11-06T22:18:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd4279d700 1 --2- 172.21.15.33:0/2568396968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c064a00 0x7efd3c14def0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd4279d700 1 --2- 172.21.15.33:0/2568396968 >> v2:172.21.15.69:3300/0 conn(0x7efd3c0ca410 0x7efd3c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd4279d700 1 --2- 172.21.15.33:0/2568396968 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c0c60c0 0x7efd3c064510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.593+0000 7efd4279d700 1 -- 172.21.15.33:0/2568396968 >> 172.21.15.33:0/2568396968 conn(0x7efd3c0cb6f0 msgr2=0x7efd3c0c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.594+0000 7efd4279d700 1 -- 172.21.15.33:0/2568396968 shutdown_connections 2023-11-06T22:18:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.594+0000 7efd4279d700 1 -- 172.21.15.33:0/2568396968 wait complete. 2023-11-06T22:18:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.595+0000 7efd4279d700 1 Processor -- start 2023-11-06T22:18:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.595+0000 7efd4279d700 1 -- start start 2023-11-06T22:18:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.596+0000 7efd4279d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c064a00 0x7efd3c150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.596+0000 7efd4279d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd3c0c60c0 0x7efd3c153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.596+0000 7efd4279d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0ca410 0x7efd3c1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.596+0000 7efd4279d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c153de0 con 0x7efd3c0ca410 2023-11-06T22:18:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.596+0000 7efd4279d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0c2de0 con 0x7efd3c0c60c0 2023-11-06T22:18:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.596+0000 7efd4279d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0c2950 con 0x7efd3c064a00 2023-11-06T22:18:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd41f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0ca410 0x7efd3c1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd40f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd3c0c60c0 0x7efd3c153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd4179b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c064a00 0x7efd3c150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd40f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd3c0c60c0 0x7efd3c153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:37964/0 (socket says 172.21.15.33:37964) 2023-11-06T22:18:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd4179b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c064a00 0x7efd3c150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:42110/0 (socket says 172.21.15.33:42110) 2023-11-06T22:18:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd40f9a700 1 -- 172.21.15.33:0/3142031823 learned_addr learned my addr 172.21.15.33:0/3142031823 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd41f9c700 1 -- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c064a00 msgr2=0x7efd3c150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd41f9c700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c064a00 0x7efd3c150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd41f9c700 1 -- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd3c0c60c0 msgr2=0x7efd3c153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:32.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd41f9c700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd3c0c60c0 0x7efd3c153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:32.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd41f9c700 1 -- 172.21.15.33:0/3142031823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34009ae0 con 0x7efd3c0ca410 2023-11-06T22:18:32.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.597+0000 7efd41f9c700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0ca410 0x7efd3c1538f0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 rx=0x7efd34013000 tx=0x7efd34009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:32.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.598+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400bee0 con 0x7efd3c0ca410 2023-11-06T22:18:32.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.598+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd340092e0 con 0x7efd3c0ca410 2023-11-06T22:18:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.598+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3401d660 con 0x7efd3c0ca410 2023-11-06T22:18:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.598+0000 7efd4279d700 1 -- 172.21.15.33:0/3142031823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c0c1d40 con 0x7efd3c0ca410 2023-11-06T22:18:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.598+0000 7efd4279d700 1 -- 172.21.15.33:0/3142031823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c0c2300 con 0x7efd3c0ca410 2023-11-06T22:18:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.599+0000 7efd4279d700 1 -- 172.21.15.33:0/3142031823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c154490 con 0x7efd3c0ca410 2023-11-06T22:18:32.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.601+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 98) v1 ==== 75565+0+0 (secure 0 0 0) 0x7efd3401da10 con 0x7efd3c0ca410 2023-11-06T22:18:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.602+0000 7efd317fa700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:32.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.602+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6165+0+0 (secure 0 0 0) 0x7efd3407be90 con 0x7efd3c0ca410 2023-11-06T22:18:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.603+0000 7efd4179b700 1 -- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 msgr2=0x7efd20061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.603+0000 7efd4179b700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.607+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efd3c154490 con 0x7efd3c0ca410 2023-11-06T22:18:32.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.804+0000 7efd4179b700 1 -- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 msgr2=0x7efd20061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.804+0000 7efd4179b700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:18:32.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:32.813+0000 7efd4279d700 1 -- 172.21.15.33:0/3142031823 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efd3c154490 con 0x7efd2005f2c0 2023-11-06T22:18:33.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:33.205+0000 7efd4179b700 1 -- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 msgr2=0x7efd20061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:33.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:33.205+0000 7efd4179b700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-06T22:18:33.235 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:33.233+0000 7fb11acb0680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:18:33.319 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:33.317+0000 7fb11acb0680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:18:33.335 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:33.333+0000 7fde086ac680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:18:33.422 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:33.421+0000 7fde086ac680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:18:33.548 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:33.546+0000 7fb11acb0680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:18:33.663 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:33.662+0000 7fde086ac680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:18:34.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:34.006+0000 7efd4179b700 1 -- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 msgr2=0x7efd20061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:34.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:34.006+0000 7efd4179b700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-06T22:18:34.057 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:34.055+0000 7fb11acb0680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:18:34.181 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:34.179+0000 7fb11acb0680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:34.201 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:34.199+0000 7fde086ac680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:18:34.332 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:34.331+0000 7fde086ac680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:34.364 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:34.362+0000 7fb11acb0680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:34.458 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:34.456+0000 7fb11acb0680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:18:34.525 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:34.524+0000 7fde086ac680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:34.543 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:34.541+0000 7fb11acb0680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:18:34.624 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:34.622+0000 7fde086ac680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:18:34.714 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:34.712+0000 7fde086ac680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:18:34.849 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:34.847+0000 7fb11acb0680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:18:35.045 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:35.043+0000 7fde086ac680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:18:35.073 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:35.071+0000 7fb11acb0680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:35.276 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:35.275+0000 7fde086ac680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:35.608+0000 7efd4179b700 1 -- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 msgr2=0x7efd20061780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:35.608+0000 7efd4179b700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-06T22:18:35.873 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:35.871+0000 7fb11acb0680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:18:35.960 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:35.958+0000 7fb11acb0680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:18:36.053 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:36.051+0000 7fb11acb0680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:18:36.127 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:36.125+0000 7fde086ac680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:18:36.219 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:36.218+0000 7fde086ac680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:18:36.239 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:36.237+0000 7fb11acb0680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:18:36.319 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:36.317+0000 7fde086ac680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:18:36.329 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:36.327+0000 7fb11acb0680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:18:36.469 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:36.467+0000 7fb11acb0680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:18:36.514 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:36.513+0000 7fde086ac680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:18:36.600 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:36.598+0000 7fb11acb0680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:36.609 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:36.607+0000 7fde086ac680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:18:36.759 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:36.757+0000 7fde086ac680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:18:36.897 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:36.895+0000 7fde086ac680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:36.946 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:36.943+0000 7fb11acb0680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:18:37.036 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.034+0000 7fb11acb0680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:18:37.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:37.044+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 99) v1 ==== 75287+0+0 (secure 0 0 0) 0x7efd340799f0 con 0x7efd3c0ca410 2023-11-06T22:18:37.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:37.044+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 msgr2=0x7efd20061780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:37.044+0000 7efd317fa700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:37.124 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.121+0000 7fb090787700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:37.125 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.121+0000 7fb090787700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:37.125 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.121+0000 7fb090787700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:37.125 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.121+0000 7fb090787700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:37.125 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.121+0000 7fb090787700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:37.126 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.122+0000 7fb08cf80700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:37.126 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.122+0000 7fb08cf80700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:37.126 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.122+0000 7fb08cf80700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:37.126 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.122+0000 7fb08cf80700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:37.126 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:37.122+0000 7fb08cf80700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:37.261 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:37.259+0000 7fde086ac680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:18:37.357 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:37.355+0000 7fde086ac680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:18:38.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.048+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 100) v1 ==== 75357+0+0 (secure 0 0 0) 0x7efd34079d70 con 0x7efd3c0ca410 2023-11-06T22:18:38.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.049+0000 7efd317fa700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.049+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efd3c154490 con 0x7efd2005f2c0 2023-11-06T22:18:38.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.051+0000 7efd4179b700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.053+0000 7efd4179b700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061700 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7efd380094c0 tx=0x7efd38009060).ready entity=mgr.8330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:38.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.062+0000 7efd317fa700 1 -- 172.21.15.33:0/3142031823 <== mgr.8330 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efd3c154490 con 0x7efd2005f2c0 2023-11-06T22:18:38.064 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path snap_test_dir1 2023-11-06T22:18:38.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 -- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 msgr2=0x7efd20061700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061700 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7efd380094c0 tx=0x7efd38009060).stop 2023-11-06T22:18:38.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 -- 172.21.15.33:0/3142031823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0ca410 msgr2=0x7efd3c1538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0ca410 0x7efd3c1538f0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 rx=0x7efd34013000 tx=0x7efd34009840).stop 2023-11-06T22:18:38.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 -- 172.21.15.33:0/3142031823 shutdown_connections 2023-11-06T22:18:38.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efd2005f2c0 0x7efd20061700 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efd3c064a00 0x7efd3c150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0ca410 0x7efd3c1538f0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 --2- 172.21.15.33:0/3142031823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd3c0c60c0 0x7efd3c153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 -- 172.21.15.33:0/3142031823 >> 172.21.15.33:0/3142031823 conn(0x7efd3c0cb6f0 msgr2=0x7efd3c0c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:38.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 -- 172.21.15.33:0/3142031823 shutdown_connections 2023-11-06T22:18:38.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.065+0000 7efd4279d700 1 -- 172.21.15.33:0/3142031823 wait complete. 2023-11-06T22:18:38.087 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-06T22:18:38.088 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-11-06T22:18:38.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.291+0000 7fa3a0dff700 1 Processor -- start 2023-11-06T22:18:38.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.291+0000 7fa3a0dff700 1 -- start start 2023-11-06T22:18:38.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.292+0000 7fa3a0dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c1546a0 0x7fa39c064680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.292+0000 7fa3a0dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa39c0be2c0 0x7fa39c0669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.292+0000 7fa3a0dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39c0641f0 con 0x7fa39c066560 2023-11-06T22:18:38.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.292+0000 7fa3a0dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39c154a80 con 0x7fa39c0be2c0 2023-11-06T22:18:38.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.292+0000 7fa3a0dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0638c0 con 0x7fa39c1546a0 2023-11-06T22:18:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.293+0000 7fa39ad9d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa39c0be2c0 0x7fa39c0669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.293+0000 7fa39ad9d700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa39c0be2c0 0x7fa39c0669e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:37504/0 (socket says 172.21.15.33:37504) 2023-11-06T22:18:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.293+0000 7fa39ad9d700 1 -- 172.21.15.33:0/441075820 learned_addr learned my addr 172.21.15.33:0/441075820 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.293+0000 7fa39a59c700 1 --2- 172.21.15.33:0/441075820 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c1546a0 0x7fa39c064680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.293+0000 7fa39ad9d700 1 -- 172.21.15.33:0/441075820 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c1546a0 msgr2=0x7fa39c064680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.293+0000 7fa39ad9d700 1 --2- 172.21.15.33:0/441075820 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c1546a0 0x7fa39c064680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.293+0000 7fa39ad9d700 1 -- 172.21.15.33:0/441075820 >> v1:172.21.15.69:6789/0 conn(0x7fa39c066560 legacy=0x7fa39c0bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.293+0000 7fa39ad9d700 1 -- 172.21.15.33:0/441075820 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c063ba0 con 0x7fa39c0be2c0 2023-11-06T22:18:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.293+0000 7fa39ad9d700 1 --2- 172.21.15.33:0/441075820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa39c0be2c0 0x7fa39c0669e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fa394009f20 tx=0x7fa394009840).ready entity=mon.1 client_cookie=f552e9bed7981bfd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa39959a700 1 -- 172.21.15.33:0/441075820 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39400e070 con 0x7fa39c0be2c0 2023-11-06T22:18:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa39959a700 1 -- 172.21.15.33:0/441075820 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39400b9d0 con 0x7fa39c0be2c0 2023-11-06T22:18:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa39959a700 1 -- 172.21.15.33:0/441075820 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39400bd50 con 0x7fa39c0be2c0 2023-11-06T22:18:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa3a0dff700 1 -- 172.21.15.33:0/441075820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa39c0be2c0 msgr2=0x7fa39c0669e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa3a0dff700 1 --2- 172.21.15.33:0/441075820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa39c0be2c0 0x7fa39c0669e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fa394009f20 tx=0x7fa394009840).stop 2023-11-06T22:18:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa3a0dff700 1 -- 172.21.15.33:0/441075820 shutdown_connections 2023-11-06T22:18:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa3a0dff700 1 --2- 172.21.15.33:0/441075820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa39c0be2c0 0x7fa39c0669e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa3a0dff700 1 --2- 172.21.15.33:0/441075820 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c1546a0 0x7fa39c064680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa3a0dff700 1 -- 172.21.15.33:0/441075820 >> 172.21.15.33:0/441075820 conn(0x7fa39c0c8c50 msgr2=0x7fa39c0c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa3a0dff700 1 -- 172.21.15.33:0/441075820 shutdown_connections 2023-11-06T22:18:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.294+0000 7fa3a0dff700 1 -- 172.21.15.33:0/441075820 wait complete. 2023-11-06T22:18:38.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.296+0000 7fa3a0dff700 1 Processor -- start 2023-11-06T22:18:38.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.296+0000 7fa3a0dff700 1 -- start start 2023-11-06T22:18:38.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.297+0000 7fa3a0dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa39c066560 0x7fa39c0c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.297+0000 7fa3a0dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c0be2c0 0x7fa39c0bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.297+0000 7fa3a0dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa39c1546a0 0x7fa39c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.297+0000 7fa3a0dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0acde0 con 0x7fa39c066560 2023-11-06T22:18:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.297+0000 7fa3a0dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c157b80 con 0x7fa39c1546a0 2023-11-06T22:18:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.297+0000 7fa3a0dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0c01e0 con 0x7fa39c0be2c0 2023-11-06T22:18:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.297+0000 7fa39a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa39c066560 0x7fa39c0c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.297+0000 7fa39a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa39c066560 0x7fa39c0c2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33768/0 (socket says 172.21.15.33:33768) 2023-11-06T22:18:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.297+0000 7fa39a59c700 1 -- 172.21.15.33:0/3376548319 learned_addr learned my addr 172.21.15.33:0/3376548319 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:38.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.297+0000 7fa399d9b700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c0be2c0 0x7fa39c0bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa39ad9d700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa39c1546a0 0x7fa39c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa39a59c700 1 -- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c0be2c0 msgr2=0x7fa39c0bfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa39a59c700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c0be2c0 0x7fa39c0bfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa39a59c700 1 -- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa39c1546a0 msgr2=0x7fa39c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa39a59c700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa39c1546a0 0x7fa39c0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa39a59c700 1 -- 172.21.15.33:0/3376548319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa394009ae0 con 0x7fa39c066560 2023-11-06T22:18:38.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa399d9b700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c0be2c0 0x7fa39c0bfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:38.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa39a59c700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa39c066560 0x7fa39c0c2900 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7fa38c00e930 tx=0x7fa38c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:38.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa38a7fc700 1 -- 172.21.15.33:0/3376548319 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c00c640 con 0x7fa39c066560 2023-11-06T22:18:38.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa38a7fc700 1 -- 172.21.15.33:0/3376548319 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c00c7a0 con 0x7fa39c066560 2023-11-06T22:18:38.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa38a7fc700 1 -- 172.21.15.33:0/3376548319 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c00cb20 con 0x7fa39c066560 2023-11-06T22:18:38.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa3a0dff700 1 -- 172.21.15.33:0/3376548319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c13efa0 con 0x7fa39c066560 2023-11-06T22:18:38.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.298+0000 7fa3a0dff700 1 -- 172.21.15.33:0/3376548319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c1575c0 con 0x7fa39c066560 2023-11-06T22:18:38.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.299+0000 7fa3a0dff700 1 -- 172.21.15.33:0/3376548319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39c0ad080 con 0x7fa39c066560 2023-11-06T22:18:38.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.300+0000 7fa38a7fc700 1 -- 172.21.15.33:0/3376548319 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 100) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fa38c01b400 con 0x7fa39c066560 2023-11-06T22:18:38.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.301+0000 7fa38a7fc700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa37c05f050 0x7fa37c061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.301+0000 7fa38a7fc700 1 -- 172.21.15.33:0/3376548319 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7fa38c07a400 con 0x7fa39c066560 2023-11-06T22:18:38.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.302+0000 7fa399d9b700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa37c05f050 0x7fa37c061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.302+0000 7fa399d9b700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa37c05f050 0x7fa37c061510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa39c0c5dd0 tx=0x7fa390009040).ready entity=mgr.8330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:38.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.306+0000 7fa38a7fc700 1 -- 172.21.15.33:0/3376548319 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa38c04b690 con 0x7fa39c066560 2023-11-06T22:18:38.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.517+0000 7fa3a0dff700 1 -- 172.21.15.33:0/3376548319 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa39c0adb40 con 0x7fa37c05f050 2023-11-06T22:18:38.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.523+0000 7fa38a7fc700 1 -- 172.21.15.33:0/3376548319 <== mgr.8330 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7fa39c0adb40 con 0x7fa37c05f050 2023-11-06T22:18:38.526 INFO:teuthology.orchestra.run.smithi033.stdout:Retention added to path snap_test_dir1 2023-11-06T22:18:38.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 -- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa37c05f050 msgr2=0x7fa37c061510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa37c05f050 0x7fa37c061510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa39c0c5dd0 tx=0x7fa390009040).stop 2023-11-06T22:18:38.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 -- 172.21.15.33:0/3376548319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa39c066560 msgr2=0x7fa39c0c2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa39c066560 0x7fa39c0c2900 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7fa38c00e930 tx=0x7fa38c009060).stop 2023-11-06T22:18:38.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 -- 172.21.15.33:0/3376548319 shutdown_connections 2023-11-06T22:18:38.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa37c05f050 0x7fa37c061510 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa39c1546a0 0x7fa39c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa39c066560 0x7fa39c0c2900 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 --2- 172.21.15.33:0/3376548319 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa39c0be2c0 0x7fa39c0bfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 -- 172.21.15.33:0/3376548319 >> 172.21.15.33:0/3376548319 conn(0x7fa39c0c8c50 msgr2=0x7fa39c153b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:38.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 -- 172.21.15.33:0/3376548319 shutdown_connections 2023-11-06T22:18:38.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.526+0000 7fa3a0dff700 1 -- 172.21.15.33:0/3376548319 wait complete. 2023-11-06T22:18:38.546 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-11-06T22:18:38.550 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-11-06-22_19_00 in ~22s... 2023-11-06T22:18:38.551 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-11-06T22:18:38.551 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-11-06T22:18:38.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.750+0000 7efeeadff700 1 Processor -- start 2023-11-06T22:18:38.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.750+0000 7efeeadff700 1 -- start start 2023-11-06T22:18:38.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.751+0000 7efeeadff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efee40c4a50 0x7efee40666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.751+0000 7efeeadff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee4155a20 con 0x7efee414e630 2023-11-06T22:18:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.751+0000 7efeeadff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee4068170 con 0x7efee4067c20 2023-11-06T22:18:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.751+0000 7efeeadff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee4068450 con 0x7efee40c4a50 2023-11-06T22:18:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.752+0000 7efee939c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efee40c4a50 0x7efee40666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.752+0000 7efee939c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efee40c4a50 0x7efee40666b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33774/0 (socket says 172.21.15.33:33774) 2023-11-06T22:18:38.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.752+0000 7efee939c700 1 -- 172.21.15.33:0/1998154189 learned_addr learned my addr 172.21.15.33:0/1998154189 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:38.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.752+0000 7efee939c700 1 -- 172.21.15.33:0/1998154189 >> v1:172.21.15.69:6789/0 conn(0x7efee4067c20 legacy=0x7efee4155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:38.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.752+0000 7efee939c700 1 -- 172.21.15.33:0/1998154189 >> v1:172.21.15.33:6789/0 conn(0x7efee414e630 legacy=0x7efee414ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.752+0000 7efee37fe700 1 -- 172.21.15.33:0/1998154189 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936354703 0 0) 0x7efee4155a20 con 0x7efee414e630 2023-11-06T22:18:38.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.752+0000 7efee939c700 1 -- 172.21.15.33:0/1998154189 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4068730 con 0x7efee40c4a50 2023-11-06T22:18:38.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.752+0000 7efee939c700 1 --2- 172.21.15.33:0/1998154189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee40c4a50 0x7efee40666b0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 rx=0x7efedc009f20 tx=0x7efedc009840).ready entity=mon.0 client_cookie=e82c1581abc3bf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:38.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.753+0000 7efee37fe700 1 -- 172.21.15.33:0/1998154189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc00e070 con 0x7efee40c4a50 2023-11-06T22:18:38.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.753+0000 7efee37fe700 1 -- 172.21.15.33:0/1998154189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc00ba80 con 0x7efee40c4a50 2023-11-06T22:18:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.753+0000 7efee37fe700 1 -- 172.21.15.33:0/1998154189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc00be00 con 0x7efee40c4a50 2023-11-06T22:18:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.753+0000 7efeeadff700 1 -- 172.21.15.33:0/1998154189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee40c4a50 msgr2=0x7efee40666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.753+0000 7efeeadff700 1 --2- 172.21.15.33:0/1998154189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee40c4a50 0x7efee40666b0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 rx=0x7efedc009f20 tx=0x7efedc009840).stop 2023-11-06T22:18:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.753+0000 7efeeadff700 1 -- 172.21.15.33:0/1998154189 shutdown_connections 2023-11-06T22:18:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.753+0000 7efeeadff700 1 --2- 172.21.15.33:0/1998154189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee40c4a50 0x7efee40666b0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.753+0000 7efeeadff700 1 -- 172.21.15.33:0/1998154189 >> 172.21.15.33:0/1998154189 conn(0x7efee40c8e60 msgr2=0x7efee40c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.753+0000 7efeeadff700 1 -- 172.21.15.33:0/1998154189 shutdown_connections 2023-11-06T22:18:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.753+0000 7efeeadff700 1 -- 172.21.15.33:0/1998154189 wait complete. 2023-11-06T22:18:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.755+0000 7efeeadff700 1 Processor -- start 2023-11-06T22:18:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.755+0000 7efeeadff700 1 -- start start 2023-11-06T22:18:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.756+0000 7efeeadff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee4067c20 0x7efee414ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.756+0000 7efeeadff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efee40c4a50 0x7efee40abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.756+0000 7efeeadff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efee414e630 0x7efee4155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.756+0000 7efeeadff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee40ac7b0 con 0x7efee4067c20 2023-11-06T22:18:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.756+0000 7efeeadff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee40aca90 con 0x7efee40c4a50 2023-11-06T22:18:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.756+0000 7efeeadff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee40acd70 con 0x7efee414e630 2023-11-06T22:18:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.756+0000 7efee8b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee4067c20 0x7efee414ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.756+0000 7efee8b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee4067c20 0x7efee414ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33788/0 (socket says 172.21.15.33:33788) 2023-11-06T22:18:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.756+0000 7efee8b9b700 1 -- 172.21.15.33:0/2568875362 learned_addr learned my addr 172.21.15.33:0/2568875362 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee3fff700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efee40c4a50 0x7efee40abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee939c700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efee414e630 0x7efee4155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee8b9b700 1 -- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efee414e630 msgr2=0x7efee4155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee8b9b700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efee414e630 0x7efee4155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee8b9b700 1 -- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efee40c4a50 msgr2=0x7efee40abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee8b9b700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efee40c4a50 0x7efee40abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee8b9b700 1 -- 172.21.15.33:0/2568875362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc009ae0 con 0x7efee4067c20 2023-11-06T22:18:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee8b9b700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee4067c20 0x7efee414ceb0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7efed800c490 tx=0x7efed8007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee0ff9700 1 -- 172.21.15.33:0/2568875362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800f470 con 0x7efee4067c20 2023-11-06T22:18:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee0ff9700 1 -- 172.21.15.33:0/2568875362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed800f5f0 con 0x7efee4067c20 2023-11-06T22:18:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efee0ff9700 1 -- 172.21.15.33:0/2568875362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800f9b0 con 0x7efee4067c20 2023-11-06T22:18:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efeeadff700 1 -- 172.21.15.33:0/2568875362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee40ad050 con 0x7efee4067c20 2023-11-06T22:18:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.757+0000 7efeeadff700 1 -- 172.21.15.33:0/2568875362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee414d3a0 con 0x7efee4067c20 2023-11-06T22:18:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.759+0000 7efeeadff700 1 -- 172.21.15.33:0/2568875362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee40c11e0 con 0x7efee4067c20 2023-11-06T22:18:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.759+0000 7efee0ff9700 1 -- 172.21.15.33:0/2568875362 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 100) v1 ==== 75357+0+0 (secure 0 0 0) 0x7efed80123f0 con 0x7efee4067c20 2023-11-06T22:18:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.760+0000 7efee0ff9700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efec805f080 0x7efec8061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.760+0000 7efee0ff9700 1 -- 172.21.15.33:0/2568875362 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7efed80797e0 con 0x7efee4067c20 2023-11-06T22:18:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.761+0000 7efee3fff700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efec805f080 0x7efec8061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.761+0000 7efee3fff700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efec805f080 0x7efec8061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7efee414dbe0 tx=0x7efed0009040).ready entity=mgr.8330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.764+0000 7efee0ff9700 1 -- 172.21.15.33:0/2568875362 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efee40c11e0 con 0x7efee4067c20 2023-11-06T22:18:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.973+0000 7efeeadff700 1 -- 172.21.15.33:0/2568875362 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efee40c11e0 con 0x7efec805f080 2023-11-06T22:18:38.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.976+0000 7efee0ff9700 1 -- 172.21.15.33:0/2568875362 <== mgr.8330 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7efee40c11e0 con 0x7efec805f080 2023-11-06T22:18:38.979 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:38.979 INFO:teuthology.orchestra.run.smithi033.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-11-06T22:18:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 -- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efec805f080 msgr2=0x7efec8061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efec805f080 0x7efec8061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7efee414dbe0 tx=0x7efed0009040).stop 2023-11-06T22:18:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 -- 172.21.15.33:0/2568875362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee4067c20 msgr2=0x7efee414ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee4067c20 0x7efee414ceb0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7efed800c490 tx=0x7efed8007de0).stop 2023-11-06T22:18:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 -- 172.21.15.33:0/2568875362 shutdown_connections 2023-11-06T22:18:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7efec805f080 0x7efec8061540 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7efee414e630 0x7efee4155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efee4067c20 0x7efee414ceb0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 --2- 172.21.15.33:0/2568875362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efee40c4a50 0x7efee40abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 -- 172.21.15.33:0/2568875362 >> 172.21.15.33:0/2568875362 conn(0x7efee40c8e60 msgr2=0x7efee4156bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 -- 172.21.15.33:0/2568875362 shutdown_connections 2023-11-06T22:18:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:38.978+0000 7efeeadff700 1 -- 172.21.15.33:0/2568875362 wait complete. 2023-11-06T22:18:38.998 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-11-06T22:18:38.999 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-11-06T22:18:38.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-11-06T22:18:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.197+0000 7f960afff700 1 Processor -- start 2023-11-06T22:18:39.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.197+0000 7f960afff700 1 -- start start 2023-11-06T22:18:39.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.198+0000 7f960afff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f96040acfc0 0x7f96040ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.198+0000 7f960afff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f96041550d0 0x7f96040c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.198+0000 7f960afff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96041586a0 con 0x7f96040631a0 2023-11-06T22:18:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.198+0000 7f960afff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96040c6590 con 0x7f96041550d0 2023-11-06T22:18:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.198+0000 7f960afff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9604155580 con 0x7f96040acfc0 2023-11-06T22:18:39.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.198+0000 7f9603fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f96041550d0 0x7f96040c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f960959c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f96040631a0 0x7f9604064680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47340/0 (socket says 172.21.15.33:47340) 2023-11-06T22:18:39.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f9603fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f96041550d0 0x7f96040c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33792/0 (socket says 172.21.15.33:33792) 2023-11-06T22:18:39.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f960959c700 1 -- 172.21.15.33:0/3664877400 learned_addr learned my addr 172.21.15.33:0/3664877400 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:39.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f9608d9b700 1 --2- 172.21.15.33:0/3664877400 >> v2:172.21.15.69:3300/0 conn(0x7f96040acfc0 0x7f96040ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f9603fff700 1 -- 172.21.15.33:0/3664877400 >> v2:172.21.15.69:3300/0 conn(0x7f96040acfc0 msgr2=0x7f96040ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:39.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f9603fff700 1 --2- 172.21.15.33:0/3664877400 >> v2:172.21.15.69:3300/0 conn(0x7f96040acfc0 0x7f96040ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f9603fff700 1 -- 172.21.15.33:0/3664877400 >> v1:172.21.15.33:6789/0 conn(0x7f96040631a0 legacy=0x7f9604064680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:39.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f9603fff700 1 -- 172.21.15.33:0/3664877400 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96041544c0 con 0x7f96041550d0 2023-11-06T22:18:39.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f96037fe700 1 -- 172.21.15.33:0/3664877400 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213559721 0 0) 0x7f96041586a0 con 0x7f96040631a0 2023-11-06T22:18:39.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f9608d9b700 1 --2- 172.21.15.33:0/3664877400 >> v2:172.21.15.69:3300/0 conn(0x7f96040acfc0 0x7f96040ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:39.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f9603fff700 1 --2- 172.21.15.33:0/3664877400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96041550d0 0x7f96040c5e20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7f95f0009760 tx=0x7f95f0009060).ready entity=mon.0 client_cookie=6e54339ffac385aa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:39.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f96037fe700 1 -- 172.21.15.33:0/3664877400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f000b520 con 0x7f96041550d0 2023-11-06T22:18:39.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f96037fe700 1 -- 172.21.15.33:0/3664877400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f000b680 con 0x7f96041550d0 2023-11-06T22:18:39.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.199+0000 7f96037fe700 1 -- 172.21.15.33:0/3664877400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f000ba00 con 0x7f96041550d0 2023-11-06T22:18:39.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.200+0000 7f960afff700 1 -- 172.21.15.33:0/3664877400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96041550d0 msgr2=0x7f96040c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:39.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.200+0000 7f960afff700 1 --2- 172.21.15.33:0/3664877400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96041550d0 0x7f96040c5e20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7f95f0009760 tx=0x7f95f0009060).stop 2023-11-06T22:18:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.200+0000 7f960afff700 1 -- 172.21.15.33:0/3664877400 shutdown_connections 2023-11-06T22:18:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.200+0000 7f960afff700 1 --2- 172.21.15.33:0/3664877400 >> v2:172.21.15.69:3300/0 conn(0x7f96040acfc0 0x7f96040ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.200+0000 7f960afff700 1 --2- 172.21.15.33:0/3664877400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96041550d0 0x7f96040c5e20 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.200+0000 7f960afff700 1 -- 172.21.15.33:0/3664877400 >> 172.21.15.33:0/3664877400 conn(0x7f960405ec80 msgr2=0x7f96040cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:39.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.200+0000 7f960afff700 1 -- 172.21.15.33:0/3664877400 shutdown_connections 2023-11-06T22:18:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.200+0000 7f960afff700 1 -- 172.21.15.33:0/3664877400 wait complete. 2023-11-06T22:18:39.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.202+0000 7f960afff700 1 Processor -- start 2023-11-06T22:18:39.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.202+0000 7f960afff700 1 -- start start 2023-11-06T22:18:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f960afff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96040631a0 0x7f96041509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f960afff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96040acfc0 0x7f9604155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f960afff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96041550d0 0x7f9604155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f960afff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96041565f0 con 0x7f96040631a0 2023-11-06T22:18:39.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f960afff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9604156110 con 0x7f96040acfc0 2023-11-06T22:18:39.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f960afff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9604167b10 con 0x7f96041550d0 2023-11-06T22:18:39.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f9608d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96040631a0 0x7f96041509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f9608d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96040631a0 0x7f96041509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33806/0 (socket says 172.21.15.33:33806) 2023-11-06T22:18:39.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f9608d9b700 1 -- 172.21.15.33:0/211878763 learned_addr learned my addr 172.21.15.33:0/211878763 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:39.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f9603fff700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96040acfc0 0x7f9604155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f960959c700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96041550d0 0x7f9604155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f9608d9b700 1 -- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96041550d0 msgr2=0x7f9604155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:39.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f9608d9b700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96041550d0 0x7f9604155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f9608d9b700 1 -- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96040acfc0 msgr2=0x7f9604155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.203+0000 7f9608d9b700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96040acfc0 0x7f9604155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.204+0000 7f9608d9b700 1 -- 172.21.15.33:0/211878763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f00092a0 con 0x7f96040631a0 2023-11-06T22:18:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.204+0000 7f9603fff700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96040acfc0 0x7f9604155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:39.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.204+0000 7f9608d9b700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96040631a0 0x7f96041509f0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7f95f800d890 tx=0x7f95f8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.204+0000 7f9600ff9700 1 -- 172.21.15.33:0/211878763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8011590 con 0x7f96040631a0 2023-11-06T22:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.204+0000 7f9600ff9700 1 -- 172.21.15.33:0/211878763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f80116f0 con 0x7f96040631a0 2023-11-06T22:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.204+0000 7f960afff700 1 -- 172.21.15.33:0/211878763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9604151a00 con 0x7f96040631a0 2023-11-06T22:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.204+0000 7f9600ff9700 1 -- 172.21.15.33:0/211878763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8011a70 con 0x7f96040631a0 2023-11-06T22:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.204+0000 7f960afff700 1 -- 172.21.15.33:0/211878763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96041674c0 con 0x7f96040631a0 2023-11-06T22:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.205+0000 7f960afff700 1 -- 172.21.15.33:0/211878763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9604167df0 con 0x7f96040631a0 2023-11-06T22:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.206+0000 7f9600ff9700 1 -- 172.21.15.33:0/211878763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 100) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f95f801c460 con 0x7f96040631a0 2023-11-06T22:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.208+0000 7f9600ff9700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f95e805f050 0x7f95e8061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.208+0000 7f9600ff9700 1 -- 172.21.15.33:0/211878763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f95f807a100 con 0x7f96040631a0 2023-11-06T22:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.208+0000 7f9603fff700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f95e805f050 0x7f95e8061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.209+0000 7f9603fff700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f95e805f050 0x7f95e8061510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f9604156df0 tx=0x7f95f0017000).ready entity=mgr.8330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.212+0000 7f9600ff9700 1 -- 172.21.15.33:0/211878763 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f95f804b300 con 0x7f96040631a0 2023-11-06T22:18:39.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.287+0000 7f9600ff9700 1 -- 172.21.15.33:0/211878763 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 101) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f95f8077e60 con 0x7f96040631a0 2023-11-06T22:18:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.424+0000 7f960afff700 1 -- 172.21.15.33:0/211878763 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f960406c6c0 con 0x7f95e805f050 2023-11-06T22:18:39.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.428+0000 7f9600ff9700 1 -- 172.21.15.33:0/211878763 <== mgr.8330 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f960406c6c0 con 0x7f95e805f050 2023-11-06T22:18:39.431 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule removed for path snap_test_dir1 2023-11-06T22:18:39.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 -- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f95e805f050 msgr2=0x7f95e8061510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:39.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f95e805f050 0x7f95e8061510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f9604156df0 tx=0x7f95f0017000).stop 2023-11-06T22:18:39.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 -- 172.21.15.33:0/211878763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96040631a0 msgr2=0x7f96041509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:39.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96040631a0 0x7f96041509f0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7f95f800d890 tx=0x7f95f8008040).stop 2023-11-06T22:18:39.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 -- 172.21.15.33:0/211878763 shutdown_connections 2023-11-06T22:18:39.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f95e805f050 0x7f95e8061510 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f96041550d0 0x7f9604155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96040631a0 0x7f96041509f0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 --2- 172.21.15.33:0/211878763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96040acfc0 0x7f9604155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 -- 172.21.15.33:0/211878763 >> 172.21.15.33:0/211878763 conn(0x7f960405ec80 msgr2=0x7f96040c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:39.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 -- 172.21.15.33:0/211878763 shutdown_connections 2023-11-06T22:18:39.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.431+0000 7f960afff700 1 -- 172.21.15.33:0/211878763 wait complete. 2023-11-06T22:18:39.452 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-11-06T22:18:39.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:18:39.453 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-06T22:18:39.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:18:39.503 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-06T22:18:39.553 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-06T22:18:39.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.749+0000 7f79891ff700 1 Processor -- start 2023-11-06T22:18:39.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.749+0000 7f79891ff700 1 -- start start 2023-11-06T22:18:39.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.751+0000 7f79891ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79840c72c0 0x7f79841553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.751+0000 7f79891ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7984157fa0 0x7f79840630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.751+0000 7f79891ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7984152470 con 0x7f7984158420 2023-11-06T22:18:39.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.751+0000 7f79891ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7984154990 con 0x7f79840c72c0 2023-11-06T22:18:39.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.751+0000 7f79891ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79840c76a0 con 0x7f7984157fa0 2023-11-06T22:18:39.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.751+0000 7f7982d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79840c72c0 0x7f79841553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.751+0000 7f798359e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7984158420 0x7f79840c5be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47356/0 (socket says 172.21.15.33:47356) 2023-11-06T22:18:39.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.751+0000 7f798359e700 1 -- 172.21.15.33:0/652908652 learned_addr learned my addr 172.21.15.33:0/652908652 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.751+0000 7f798259c700 1 --2- 172.21.15.33:0/652908652 >> v2:172.21.15.69:3300/0 conn(0x7f7984157fa0 0x7f79840630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f7982d9d700 1 -- 172.21.15.33:0/652908652 >> v2:172.21.15.69:3300/0 conn(0x7f7984157fa0 msgr2=0x7f79840630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f7982d9d700 1 --2- 172.21.15.33:0/652908652 >> v2:172.21.15.69:3300/0 conn(0x7f7984157fa0 0x7f79840630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f7982d9d700 1 -- 172.21.15.33:0/652908652 >> v1:172.21.15.33:6789/0 conn(0x7f7984158420 legacy=0x7f79840c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f7982d9d700 1 -- 172.21.15.33:0/652908652 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7984158910 con 0x7f79840c72c0 2023-11-06T22:18:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f798259c700 1 --2- 172.21.15.33:0/652908652 >> v2:172.21.15.69:3300/0 conn(0x7f7984157fa0 0x7f79840630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f7981d9b700 1 -- 172.21.15.33:0/652908652 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211508137 0 0) 0x7f7984152470 con 0x7f7984158420 2023-11-06T22:18:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f7982d9d700 1 --2- 172.21.15.33:0/652908652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79840c72c0 0x7f79841553e0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f798406c340 tx=0x7f7974009060).ready entity=mon.0 client_cookie=ee59be695264cd9f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f7981d9b700 1 -- 172.21.15.33:0/652908652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400b580 con 0x7f79840c72c0 2023-11-06T22:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f7981d9b700 1 -- 172.21.15.33:0/652908652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797400b6e0 con 0x7f79840c72c0 2023-11-06T22:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f7981d9b700 1 -- 172.21.15.33:0/652908652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400ba60 con 0x7f79840c72c0 2023-11-06T22:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f79891ff700 1 -- 172.21.15.33:0/652908652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79840c72c0 msgr2=0x7f79841553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:39.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.752+0000 7f79891ff700 1 --2- 172.21.15.33:0/652908652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79840c72c0 0x7f79841553e0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f798406c340 tx=0x7f7974009060).stop 2023-11-06T22:18:39.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.753+0000 7f79891ff700 1 -- 172.21.15.33:0/652908652 shutdown_connections 2023-11-06T22:18:39.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.753+0000 7f79891ff700 1 --2- 172.21.15.33:0/652908652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79840c72c0 0x7f79841553e0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.753+0000 7f79891ff700 1 --2- 172.21.15.33:0/652908652 >> v2:172.21.15.69:3300/0 conn(0x7f7984157fa0 0x7f79840630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.753+0000 7f79891ff700 1 -- 172.21.15.33:0/652908652 >> 172.21.15.33:0/652908652 conn(0x7f798405ec20 msgr2=0x7f79840cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.753+0000 7f79891ff700 1 -- 172.21.15.33:0/652908652 shutdown_connections 2023-11-06T22:18:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.753+0000 7f79891ff700 1 -- 172.21.15.33:0/652908652 wait complete. 2023-11-06T22:18:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.754+0000 7f79891ff700 1 Processor -- start 2023-11-06T22:18:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.754+0000 7f79891ff700 1 -- start start 2023-11-06T22:18:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.755+0000 7f79891ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79840c72c0 0x7f798414fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.755+0000 7f79891ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7984157fa0 0x7f79840aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.755+0000 7f79891ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7984158420 0x7f79840af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.755+0000 7f79891ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840add90 con 0x7f7984158420 2023-11-06T22:18:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.755+0000 7f79891ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840ae8a0 con 0x7f7984157fa0 2023-11-06T22:18:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.755+0000 7f79891ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79840adf70 con 0x7f79840c72c0 2023-11-06T22:18:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.755+0000 7f798359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7984158420 0x7f79840af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f798359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7984158420 0x7f79840af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33822/0 (socket says 172.21.15.33:33822) 2023-11-06T22:18:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f798359e700 1 -- 172.21.15.33:0/1198160434 learned_addr learned my addr 172.21.15.33:0/1198160434 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f7982d9d700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79840c72c0 0x7f798414fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f798259c700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7984157fa0 0x7f79840aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f798359e700 1 -- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79840c72c0 msgr2=0x7f798414fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:39.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f798359e700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79840c72c0 0x7f798414fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f798359e700 1 -- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7984157fa0 msgr2=0x7f79840aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:39.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f798359e700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7984157fa0 0x7f79840aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:39.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f798359e700 1 -- 172.21.15.33:0/1198160434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79740092a0 con 0x7f7984158420 2023-11-06T22:18:39.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f798359e700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7984158420 0x7f79840af220 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f797c00c730 tx=0x7f797c001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f796affd700 1 -- 172.21.15.33:0/1198160434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c005d30 con 0x7f7984158420 2023-11-06T22:18:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f796affd700 1 -- 172.21.15.33:0/1198160434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c005e90 con 0x7f7984158420 2023-11-06T22:18:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f79891ff700 1 -- 172.21.15.33:0/1198160434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79840ae250 con 0x7f7984158420 2023-11-06T22:18:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.756+0000 7f796affd700 1 -- 172.21.15.33:0/1198160434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c007300 con 0x7f7984158420 2023-11-06T22:18:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.757+0000 7f79891ff700 1 -- 172.21.15.33:0/1198160434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7984150260 con 0x7f7984158420 2023-11-06T22:18:39.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.758+0000 7f79891ff700 1 -- 172.21.15.33:0/1198160434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7984150450 con 0x7f7984158420 2023-11-06T22:18:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.758+0000 7f796affd700 1 -- 172.21.15.33:0/1198160434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 101) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f797c0076b0 con 0x7f7984158420 2023-11-06T22:18:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.759+0000 7f796affd700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f797005f350 0x7f7970061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.759+0000 7f796affd700 1 -- 172.21.15.33:0/1198160434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f797c07a180 con 0x7f7984158420 2023-11-06T22:18:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.760+0000 7f7982d9d700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f797005f350 0x7f7970061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.760+0000 7f7982d9d700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f797005f350 0x7f7970061810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f7974009fd0 tx=0x7f7974017000).ready entity=mgr.8330 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.763+0000 7f796affd700 1 -- 172.21.15.33:0/1198160434 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f797c04b020 con 0x7f7984158420 2023-11-06T22:18:39.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:39.990+0000 7f79891ff700 1 -- 172.21.15.33:0/1198160434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f79840aea30 con 0x7f7984158420 2023-11-06T22:18:40.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.291+0000 7f796affd700 1 -- 172.21.15.33:0/1198160434 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f797c077b20 con 0x7f7984158420 2023-11-06T22:18:40.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.292+0000 7f796affd700 1 -- 172.21.15.33:0/1198160434 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v102) v1 ==== 93+0+0 (secure 0 0 0) 0x7f797c072260 con 0x7f7984158420 2023-11-06T22:18:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 -- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f797005f350 msgr2=0x7f7970061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f797005f350 0x7f7970061810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f7974009fd0 tx=0x7f7974017000).stop 2023-11-06T22:18:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 -- 172.21.15.33:0/1198160434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7984158420 msgr2=0x7f79840af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7984158420 0x7f79840af220 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f797c00c730 tx=0x7f797c001520).stop 2023-11-06T22:18:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 -- 172.21.15.33:0/1198160434 shutdown_connections 2023-11-06T22:18:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f797005f350 0x7f7970061810 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f79840c72c0 0x7f798414fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7984158420 0x7f79840af220 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 --2- 172.21.15.33:0/1198160434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7984157fa0 0x7f79840aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 -- 172.21.15.33:0/1198160434 >> 172.21.15.33:0/1198160434 conn(0x7f798405ec20 msgr2=0x7f79840c8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 -- 172.21.15.33:0/1198160434 shutdown_connections 2023-11-06T22:18:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.295+0000 7f79891ff700 1 -- 172.21.15.33:0/1198160434 wait complete. 2023-11-06T22:18:40.318 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T22:18:40.319 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:18:40.412 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T22:18:40.508 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T22:18:40.508 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-06T22:18:40.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-06T22:18:40.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-06T22:18:40.514 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:40.512+0000 7f42cd07f680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:18:40.521 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:40.519+0000 7f2e7097e680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T22:18:40.595 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T22:18:40.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:40.596 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T22:18:40.601 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:40.597+0000 7f42cd07f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:18:40.602 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:40.600+0000 7f2e7097e680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T22:18:40.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:40.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.842+0000 7fd9217ff700 1 Processor -- start 2023-11-06T22:18:40.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.843+0000 7fd9217ff700 1 -- start start 2023-11-06T22:18:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.844+0000 7fd9217ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c153e60 0x7fd91c0a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.844+0000 7fd9217ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd91c062710 0x7fd91c062bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.844+0000 7fd9217ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91c0630b0 con 0x7fd91c154240 2023-11-06T22:18:40.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.844+0000 7fd9217ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91c0c2ec0 con 0x7fd91c062710 2023-11-06T22:18:40.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.844+0000 7fd9217ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1546c0 con 0x7fd91c153e60 2023-11-06T22:18:40.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.844+0000 7fd91affd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c153e60 0x7fd91c0a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:40.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.844+0000 7fd91b7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd91c062710 0x7fd91c062bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:40.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.845+0000 7fd91b7fe700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd91c062710 0x7fd91c062bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:37560/0 (socket says 172.21.15.33:37560) 2023-11-06T22:18:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.845+0000 7fd91b7fe700 1 -- 172.21.15.33:0/1789362141 learned_addr learned my addr 172.21.15.33:0/1789362141 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.845+0000 7fd91a7fc700 1 --1- 172.21.15.33:0/1789362141 >> v1:172.21.15.69:6789/0 conn(0x7fd91c154240 0x7fd91c069d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:60338/0 (socket says 172.21.15.33:60338) 2023-11-06T22:18:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.845+0000 7fd919ffb700 1 -- 172.21.15.33:0/1789362141 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625426750 0 0) 0x7fd910001110 con 0x7fd91c154240 2023-11-06T22:18:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.845+0000 7fd91affd700 1 -- 172.21.15.33:0/1789362141 >> v2:172.21.15.69:3300/0 conn(0x7fd91c062710 msgr2=0x7fd91c062bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.845+0000 7fd91affd700 1 --2- 172.21.15.33:0/1789362141 >> v2:172.21.15.69:3300/0 conn(0x7fd91c062710 0x7fd91c062bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.845+0000 7fd91affd700 1 -- 172.21.15.33:0/1789362141 >> v1:172.21.15.69:6789/0 conn(0x7fd91c154240 legacy=0x7fd91c069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.845+0000 7fd91affd700 1 -- 172.21.15.33:0/1789362141 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c0c4360 con 0x7fd91c153e60 2023-11-06T22:18:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.845+0000 7fd91affd700 1 --2- 172.21.15.33:0/1789362141 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c153e60 0x7fd91c0a6d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fd90c009790 tx=0x7fd90c009060).ready entity=mon.2 client_cookie=48b273c854d620fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.845+0000 7fd919ffb700 1 -- 172.21.15.33:0/1789362141 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00f510 con 0x7fd91c153e60 2023-11-06T22:18:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.846+0000 7fd919ffb700 1 -- 172.21.15.33:0/1789362141 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00f670 con 0x7fd91c153e60 2023-11-06T22:18:40.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.846+0000 7fd919ffb700 1 -- 172.21.15.33:0/1789362141 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00f9f0 con 0x7fd91c153e60 2023-11-06T22:18:40.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.846+0000 7fd9217ff700 1 -- 172.21.15.33:0/1789362141 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c153e60 msgr2=0x7fd91c0a6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:40.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.846+0000 7fd9217ff700 1 --2- 172.21.15.33:0/1789362141 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c153e60 0x7fd91c0a6d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fd90c009790 tx=0x7fd90c009060).stop 2023-11-06T22:18:40.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.846+0000 7fd9217ff700 1 -- 172.21.15.33:0/1789362141 shutdown_connections 2023-11-06T22:18:40.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.846+0000 7fd9217ff700 1 --2- 172.21.15.33:0/1789362141 >> v2:172.21.15.69:3300/0 conn(0x7fd91c062710 0x7fd91c062bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.846+0000 7fd9217ff700 1 --2- 172.21.15.33:0/1789362141 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c153e60 0x7fd91c0a6d40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.846+0000 7fd9217ff700 1 -- 172.21.15.33:0/1789362141 >> 172.21.15.33:0/1789362141 conn(0x7fd91c0c7b50 msgr2=0x7fd91c0c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.846+0000 7fd9217ff700 1 -- 172.21.15.33:0/1789362141 shutdown_connections 2023-11-06T22:18:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.846+0000 7fd9217ff700 1 -- 172.21.15.33:0/1789362141 wait complete. 2023-11-06T22:18:40.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.848+0000 7fd9217ff700 1 Processor -- start 2023-11-06T22:18:40.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.848+0000 7fd9217ff700 1 -- start start 2023-11-06T22:18:40.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd9217ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd91c062710 0x7fd91c0660d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:40.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd9217ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd91c153e60 0x7fd91c150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:40.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd9217ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c154240 0x7fd91c0abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:40.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd9217ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0adb30 con 0x7fd91c062710 2023-11-06T22:18:40.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd9217ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0abb20 con 0x7fd91c153e60 2023-11-06T22:18:40.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd9217ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0ad850 con 0x7fd91c154240 2023-11-06T22:18:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd91affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd91c062710 0x7fd91c0660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd91affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd91c062710 0x7fd91c0660d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33842/0 (socket says 172.21.15.33:33842) 2023-11-06T22:18:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd91affd700 1 -- 172.21.15.33:0/1112066362 learned_addr learned my addr 172.21.15.33:0/1112066362 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd91a7fc700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd91c153e60 0x7fd91c150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.849+0000 7fd91b7fe700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c154240 0x7fd91c0abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:40.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd91affd700 1 -- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c154240 msgr2=0x7fd91c0abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:40.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd91affd700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c154240 0x7fd91c0abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:40.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd91affd700 1 -- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd91c153e60 msgr2=0x7fd91c150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:40.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd91affd700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd91c153e60 0x7fd91c150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:40.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd91affd700 1 -- 172.21.15.33:0/1112066362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c0092a0 con 0x7fd91c062710 2023-11-06T22:18:40.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd91affd700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd91c062710 0x7fd91c0660d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7fd90c009e30 tx=0x7fd90c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:40.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd902ffd700 1 -- 172.21.15.33:0/1112066362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00f7e0 con 0x7fd91c062710 2023-11-06T22:18:40.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd902ffd700 1 -- 172.21.15.33:0/1112066362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00f940 con 0x7fd91c062710 2023-11-06T22:18:40.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd9217ff700 1 -- 172.21.15.33:0/1112066362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91c0ab4d0 con 0x7fd91c062710 2023-11-06T22:18:40.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd902ffd700 1 -- 172.21.15.33:0/1112066362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c01b640 con 0x7fd91c062710 2023-11-06T22:18:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.850+0000 7fd9217ff700 1 -- 172.21.15.33:0/1112066362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c0665c0 con 0x7fd91c062710 2023-11-06T22:18:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.852+0000 7fd9217ff700 1 -- 172.21.15.33:0/1112066362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91c0a73d0 con 0x7fd91c062710 2023-11-06T22:18:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.853+0000 7fd902ffd700 1 -- 172.21.15.33:0/1112066362 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fd90c01ba30 con 0x7fd91c062710 2023-11-06T22:18:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.854+0000 7fd902ffd700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd90805f300 0x7fd9080617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:40.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.855+0000 7fd902ffd700 1 -- 172.21.15.33:0/1112066362 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7fd90c07bd40 con 0x7fd91c062710 2023-11-06T22:18:40.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.855+0000 7fd91a7fc700 1 -- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd90805f300 msgr2=0x7fd9080617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:40.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.855+0000 7fd91a7fc700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd90805f300 0x7fd9080617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:40.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:40.860+0000 7fd902ffd700 1 -- 172.21.15.33:0/1112066362 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd90c04cce0 con 0x7fd91c062710 2023-11-06T22:18:40.934 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:40.932+0000 7f2e7097e680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:18:40.958 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:40.955+0000 7f42cd07f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T22:18:41.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.056+0000 7fd91a7fc700 1 -- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd90805f300 msgr2=0x7fd9080617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.056+0000 7fd91a7fc700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd90805f300 0x7fd9080617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:18:41.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.088+0000 7fd9217ff700 1 -- 172.21.15.33:0/1112066362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd91c06a710 con 0x7fd91c062710 2023-11-06T22:18:41.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.088+0000 7fd902ffd700 1 -- 172.21.15.33:0/1112066362 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4493 (secure 0 0 0) 0x7fd90c029030 con 0x7fd91c062710 2023-11-06T22:18:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:41.092 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":91,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3097639551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3097639551},{"type":"v1","addr":"172.21.15.69:6837","nonce":3097639551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9828,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1256245016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1256245016},{"type":"v1","addr":"172.21.15.33:6834","nonce":1256245016}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1680210655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1680210655},{"type":"v1","addr":"172.21.15.33:6835","nonce":1680210655}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:18:22.834249+0000","modified":"2023-11-06T22:18:25.840897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9837":{"gid":9837,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/3921249578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3921249578},{"type":"v1","addr":"172.21.15.69:6836","nonce":3921249578}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-06T22:18:41.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.090+0000 7fd9217ff700 1 -- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd90805f300 msgr2=0x7fd9080617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:41.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.090+0000 7fd9217ff700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd90805f300 0x7fd9080617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:41.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.090+0000 7fd9217ff700 1 -- 172.21.15.33:0/1112066362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd91c062710 msgr2=0x7fd91c0660d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.090+0000 7fd9217ff700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd91c062710 0x7fd91c0660d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7fd90c009e30 tx=0x7fd90c017000).stop 2023-11-06T22:18:41.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.090+0000 7fd9217ff700 1 -- 172.21.15.33:0/1112066362 shutdown_connections 2023-11-06T22:18:41.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.090+0000 7fd9217ff700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd90805f300 0x7fd9080617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:41.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.090+0000 7fd9217ff700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd91c154240 0x7fd91c0abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.090+0000 7fd9217ff700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd91c062710 0x7fd91c0660d0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.090+0000 7fd9217ff700 1 --2- 172.21.15.33:0/1112066362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd91c153e60 0x7fd91c150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.090+0000 7fd9217ff700 1 -- 172.21.15.33:0/1112066362 >> 172.21.15.33:0/1112066362 conn(0x7fd91c0c7b50 msgr2=0x7fd91c0a48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.091+0000 7fd9217ff700 1 -- 172.21.15.33:0/1112066362 shutdown_connections 2023-11-06T22:18:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.091+0000 7fd9217ff700 1 -- 172.21.15.33:0/1112066362 wait complete. 2023-11-06T22:18:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 91 2023-11-06T22:18:41.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.308+0000 7f650bfff700 1 Processor -- start 2023-11-06T22:18:41.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.308+0000 7f650bfff700 1 -- start start 2023-11-06T22:18:41.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.309+0000 7f650bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f65041521a0 0x7f65040c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.309+0000 7f650bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6504155320 0x7f65040c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.309+0000 7f650bfff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6504154e90 con 0x7f6504151dc0 2023-11-06T22:18:41.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.309+0000 7f650bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040639d0 con 0x7f6504155320 2023-11-06T22:18:41.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.309+0000 7f650bfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6504063cb0 con 0x7f65041521a0 2023-11-06T22:18:41.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.310+0000 7f650a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6504155320 0x7f65040c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:41.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.310+0000 7f650a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6504155320 0x7f65040c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33846/0 (socket says 172.21.15.33:33846) 2023-11-06T22:18:41.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.310+0000 7f650a59c700 1 -- 172.21.15.33:0/896567332 learned_addr learned my addr 172.21.15.33:0/896567332 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:41.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.310+0000 7f650959a700 1 --2- 172.21.15.33:0/896567332 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f65041521a0 0x7f65040c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:41.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.310+0000 7f650a59c700 1 -- 172.21.15.33:0/896567332 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f65041521a0 msgr2=0x7f65040c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.310+0000 7f650a59c700 1 --2- 172.21.15.33:0/896567332 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f65041521a0 0x7f65040c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.310+0000 7f6508d99700 1 -- 172.21.15.33:0/896567332 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884457741 0 0) 0x7f6504154e90 con 0x7f6504151dc0 2023-11-06T22:18:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.310+0000 7f650a59c700 1 -- 172.21.15.33:0/896567332 >> v1:172.21.15.69:6789/0 conn(0x7f6504151dc0 legacy=0x7f6504063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.310+0000 7f650a59c700 1 -- 172.21.15.33:0/896567332 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6504063f90 con 0x7f6504155320 2023-11-06T22:18:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.310+0000 7f650a59c700 1 --2- 172.21.15.33:0/896567332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504155320 0x7f65040c7850 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 rx=0x7f64fc009f20 tx=0x7f64fc009840).ready entity=mon.0 client_cookie=331c1aa8a7c4a5ee server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:41.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.311+0000 7f6508d99700 1 -- 172.21.15.33:0/896567332 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00e070 con 0x7f6504155320 2023-11-06T22:18:41.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.311+0000 7f6508d99700 1 -- 172.21.15.33:0/896567332 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc00b9d0 con 0x7f6504155320 2023-11-06T22:18:41.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.311+0000 7f6508d99700 1 -- 172.21.15.33:0/896567332 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00bd50 con 0x7f6504155320 2023-11-06T22:18:41.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.311+0000 7f650bfff700 1 -- 172.21.15.33:0/896567332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504155320 msgr2=0x7f65040c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.311+0000 7f650bfff700 1 --2- 172.21.15.33:0/896567332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504155320 0x7f65040c7850 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 rx=0x7f64fc009f20 tx=0x7f64fc009840).stop 2023-11-06T22:18:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.311+0000 7f650bfff700 1 -- 172.21.15.33:0/896567332 shutdown_connections 2023-11-06T22:18:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.311+0000 7f650bfff700 1 --2- 172.21.15.33:0/896567332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504155320 0x7f65040c7850 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.311+0000 7f650bfff700 1 --2- 172.21.15.33:0/896567332 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f65041521a0 0x7f65040c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.311+0000 7f650bfff700 1 -- 172.21.15.33:0/896567332 >> 172.21.15.33:0/896567332 conn(0x7f650405e1b0 msgr2=0x7f650405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.311+0000 7f650bfff700 1 -- 172.21.15.33:0/896567332 shutdown_connections 2023-11-06T22:18:41.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.312+0000 7f650bfff700 1 -- 172.21.15.33:0/896567332 wait complete. 2023-11-06T22:18:41.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.314+0000 7f650bfff700 1 Processor -- start 2023-11-06T22:18:41.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.314+0000 7f650bfff700 1 -- start start 2023-11-06T22:18:41.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.315+0000 7f650bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504151dc0 0x7f65040afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.315+0000 7f650bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65041521a0 0x7f65040c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.315+0000 7f650bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6504155320 0x7f65041501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.315+0000 7f650bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040aff30 con 0x7f6504151dc0 2023-11-06T22:18:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.315+0000 7f650bfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040c82e0 con 0x7f65041521a0 2023-11-06T22:18:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.315+0000 7f650bfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65040b0120 con 0x7f6504155320 2023-11-06T22:18:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.315+0000 7f6509d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504151dc0 0x7f65040afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f650959a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65041521a0 0x7f65040c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f6509d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504151dc0 0x7f65040afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33848/0 (socket says 172.21.15.33:33848) 2023-11-06T22:18:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f6509d9b700 1 -- 172.21.15.33:0/2645204631 learned_addr learned my addr 172.21.15.33:0/2645204631 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f650a59c700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6504155320 0x7f65041501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f6509d9b700 1 -- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6504155320 msgr2=0x7f65041501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f6509d9b700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6504155320 0x7f65041501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f6509d9b700 1 -- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65041521a0 msgr2=0x7f65040c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f6509d9b700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65041521a0 0x7f65040c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f6509d9b700 1 -- 172.21.15.33:0/2645204631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc009ae0 con 0x7f6504151dc0 2023-11-06T22:18:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f650959a700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65041521a0 0x7f65040c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f6509d9b700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504151dc0 0x7f65040afa40 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7f650000c410 tx=0x7f6500007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f64f9ffb700 1 -- 172.21.15.33:0/2645204631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500007020 con 0x7f6504151dc0 2023-11-06T22:18:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f64f9ffb700 1 -- 172.21.15.33:0/2645204631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65000071a0 con 0x7f6504151dc0 2023-11-06T22:18:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f650bfff700 1 -- 172.21.15.33:0/2645204631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65040b0400 con 0x7f6504151dc0 2023-11-06T22:18:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.316+0000 7f64f9ffb700 1 -- 172.21.15.33:0/2645204631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500007560 con 0x7f6504151dc0 2023-11-06T22:18:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.317+0000 7f650bfff700 1 -- 172.21.15.33:0/2645204631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65040b09c0 con 0x7f6504151dc0 2023-11-06T22:18:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.318+0000 7f650bfff700 1 -- 172.21.15.33:0/2645204631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65040c0790 con 0x7f6504151dc0 2023-11-06T22:18:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.318+0000 7f64f9ffb700 1 -- 172.21.15.33:0/2645204631 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f65000068b0 con 0x7f6504151dc0 2023-11-06T22:18:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.319+0000 7f64f9ffb700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f64e805f300 0x7f64e80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.319+0000 7f64f9ffb700 1 -- 172.21.15.33:0/2645204631 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f6500079980 con 0x7f6504151dc0 2023-11-06T22:18:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.319+0000 7f650959a700 1 -- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f64e805f300 msgr2=0x7f64e80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.319+0000 7f650959a700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f64e805f300 0x7f64e80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.323+0000 7f64f9ffb700 1 -- 172.21.15.33:0/2645204631 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6500047af0 con 0x7f6504151dc0 2023-11-06T22:18:41.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.508+0000 7f650bfff700 1 -- 172.21.15.33:0/2645204631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65040c0a30 con 0x7f6504151dc0 2023-11-06T22:18:41.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.509+0000 7f64f9ffb700 1 -- 172.21.15.33:0/2645204631 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v224) v1 ==== 74+0+17429 (secure 0 0 0) 0x7f650004a820 con 0x7f6504151dc0 2023-11-06T22:18:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:41.513 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":224,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:37.037489+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:18:21.317255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T22:18:21.822232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/672707063":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/2065981382":"2023-11-07T22:18:37.037458+0000","172.21.15.69:6833/129946":"2023-11-07T22:18:37.037458+0000","172.21.15.69:6832/129946":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/3481533320":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/2510047576":"2023-11-07T22:18:37.037458+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:41.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.511+0000 7f650bfff700 1 -- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f64e805f300 msgr2=0x7f64e80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:41.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.511+0000 7f650bfff700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f64e805f300 0x7f64e80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:41.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.511+0000 7f650bfff700 1 -- 172.21.15.33:0/2645204631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504151dc0 msgr2=0x7f65040afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.511+0000 7f650bfff700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504151dc0 0x7f65040afa40 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7f650000c410 tx=0x7f6500007e00).stop 2023-11-06T22:18:41.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.511+0000 7f650bfff700 1 -- 172.21.15.33:0/2645204631 shutdown_connections 2023-11-06T22:18:41.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.511+0000 7f650bfff700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f64e805f300 0x7f64e80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:41.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.511+0000 7f650bfff700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6504155320 0x7f65041501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.511+0000 7f650bfff700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6504151dc0 0x7f65040afa40 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.511+0000 7f650bfff700 1 --2- 172.21.15.33:0/2645204631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65041521a0 0x7f65040c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.511+0000 7f650bfff700 1 -- 172.21.15.33:0/2645204631 >> 172.21.15.33:0/2645204631 conn(0x7f650405e1b0 msgr2=0x7f6504157060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:41.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.512+0000 7f650bfff700 1 -- 172.21.15.33:0/2645204631 shutdown_connections 2023-11-06T22:18:41.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.512+0000 7f650bfff700 1 -- 172.21.15.33:0/2645204631 wait complete. 2023-11-06T22:18:41.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-06T20:43:57.893748+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T20:43:59.610088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T22:18:21.317255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '222', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T22:18:21.822232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '223', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-06T22:18:41.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-06T22:18:41.678 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:41.674+0000 7f2e7097e680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:18:41.757 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:41.755+0000 7f42cd07f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T22:18:41.761 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:41.759+0000 7f2e7097e680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:18:41.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.762+0000 7f682679d700 1 Processor -- start 2023-11-06T22:18:41.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.762+0000 7f682679d700 1 -- start start 2023-11-06T22:18:41.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.763+0000 7f682679d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6818054120 0x7f6818037f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.763+0000 7f682679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6818007790 con 0x7f6818038b70 2023-11-06T22:18:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.763+0000 7f682679d700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681804f8e0 con 0x7f68180e1bc0 2023-11-06T22:18:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.763+0000 7f682679d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68180062f0 con 0x7f6818054120 2023-11-06T22:18:41.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.763+0000 7f6824f9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6818038b70 0x7f6818053a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47368/0 (socket says 172.21.15.33:47368) 2023-11-06T22:18:41.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.763+0000 7f6824f9a700 1 -- 172.21.15.33:0/4073109783 learned_addr learned my addr 172.21.15.33:0/4073109783 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:41.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.764+0000 7f6825f9c700 1 --2- 172.21.15.33:0/4073109783 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6818054120 0x7f6818037f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:41.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.764+0000 7f6817fff700 1 -- 172.21.15.33:0/4073109783 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823786218 0 0) 0x7f681c001040 con 0x7f68180e1bc0 2023-11-06T22:18:41.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.764+0000 7f6817fff700 1 -- 172.21.15.33:0/4073109783 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f680c0033f0 con 0x7f68180e1bc0 2023-11-06T22:18:41.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.764+0000 7f6817fff700 1 -- 172.21.15.33:0/4073109783 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024391014 0 0) 0x7f6818007790 con 0x7f6818038b70 2023-11-06T22:18:41.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.764+0000 7f6817fff700 1 -- 172.21.15.33:0/4073109783 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f681c001040 con 0x7f6818038b70 2023-11-06T22:18:41.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.764+0000 7f6825f9c700 1 -- 172.21.15.33:0/4073109783 >> v1:172.21.15.69:6789/0 conn(0x7f68180e1bc0 legacy=0x7f681804eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.764+0000 7f6825f9c700 1 -- 172.21.15.33:0/4073109783 >> v1:172.21.15.33:6789/0 conn(0x7f6818038b70 legacy=0x7f6818053a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.764+0000 7f6825f9c700 1 -- 172.21.15.33:0/4073109783 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681804f320 con 0x7f6818054120 2023-11-06T22:18:41.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.764+0000 7f6825f9c700 1 --2- 172.21.15.33:0/4073109783 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6818054120 0x7f6818037f40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f68180062a0 tx=0x7f6820062130).ready entity=mon.2 client_cookie=56f3c0bfde19574a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:41.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.765+0000 7f6817fff700 1 -- 172.21.15.33:0/4073109783 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6820086030 con 0x7f6818054120 2023-11-06T22:18:41.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.765+0000 7f6817fff700 1 -- 172.21.15.33:0/4073109783 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68200472c0 con 0x7f6818054120 2023-11-06T22:18:41.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.765+0000 7f6817fff700 1 -- 172.21.15.33:0/4073109783 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c001530 con 0x7f6818054120 2023-11-06T22:18:41.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.765+0000 7f682679d700 1 -- 172.21.15.33:0/4073109783 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6818054120 msgr2=0x7f6818037f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.765+0000 7f682679d700 1 --2- 172.21.15.33:0/4073109783 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6818054120 0x7f6818037f40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f68180062a0 tx=0x7f6820062130).stop 2023-11-06T22:18:41.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.765+0000 7f682679d700 1 -- 172.21.15.33:0/4073109783 shutdown_connections 2023-11-06T22:18:41.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.765+0000 7f682679d700 1 --2- 172.21.15.33:0/4073109783 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6818054120 0x7f6818037f40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.765+0000 7f682679d700 1 -- 172.21.15.33:0/4073109783 >> 172.21.15.33:0/4073109783 conn(0x7f68180554a0 msgr2=0x7f68180558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:41.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.765+0000 7f682679d700 1 -- 172.21.15.33:0/4073109783 shutdown_connections 2023-11-06T22:18:41.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.765+0000 7f682679d700 1 -- 172.21.15.33:0/4073109783 wait complete. 2023-11-06T22:18:41.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.767+0000 7f682679d700 1 Processor -- start 2023-11-06T22:18:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.768+0000 7f682679d700 1 -- start start 2023-11-06T22:18:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818038b70 0x7f68180ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682679d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818054120 0x7f6818050450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682679d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68180e1bc0 0x7f6818050940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6818050e30 con 0x7f6818038b70 2023-11-06T22:18:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682679d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681803c6f0 con 0x7f6818054120 2023-11-06T22:18:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682679d700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68180582a0 con 0x7f68180e1bc0 2023-11-06T22:18:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818038b70 0x7f68180ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f6824f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818054120 0x7f6818050450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818038b70 0x7f68180ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33860/0 (socket says 172.21.15.33:33860) 2023-11-06T22:18:41.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682579b700 1 -- 172.21.15.33:0/1878219900 learned_addr learned my addr 172.21.15.33:0/1878219900 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:41.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f6825f9c700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68180e1bc0 0x7f6818050940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:41.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682579b700 1 -- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68180e1bc0 msgr2=0x7f6818050940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682579b700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68180e1bc0 0x7f6818050940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682579b700 1 -- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818054120 msgr2=0x7f6818050450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682579b700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818054120 0x7f6818050450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f682579b700 1 -- 172.21.15.33:0/1878219900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6818057690 con 0x7f6818038b70 2023-11-06T22:18:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.769+0000 7f6824f9a700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818054120 0x7f6818050450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.770+0000 7f682579b700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818038b70 0x7f68180ce290 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7f681c00cd20 tx=0x7f681c007630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:41.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.770+0000 7f68157fa700 1 -- 172.21.15.33:0/1878219900 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c00ce20 con 0x7f6818038b70 2023-11-06T22:18:41.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.770+0000 7f68157fa700 1 -- 172.21.15.33:0/1878219900 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c00ac80 con 0x7f6818038b70 2023-11-06T22:18:41.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.770+0000 7f682679d700 1 -- 172.21.15.33:0/1878219900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6818057970 con 0x7f6818038b70 2023-11-06T22:18:41.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.770+0000 7f68157fa700 1 -- 172.21.15.33:0/1878219900 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c001a00 con 0x7f6818038b70 2023-11-06T22:18:41.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.770+0000 7f682679d700 1 -- 172.21.15.33:0/1878219900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6818057f30 con 0x7f6818038b70 2023-11-06T22:18:41.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.771+0000 7f682679d700 1 -- 172.21.15.33:0/1878219900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6818005200 con 0x7f6818038b70 2023-11-06T22:18:41.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.772+0000 7f68157fa700 1 -- 172.21.15.33:0/1878219900 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f681c003110 con 0x7f6818038b70 2023-11-06T22:18:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.774+0000 7f68157fa700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680405f2d0 0x7f6804061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.774+0000 7f68157fa700 1 -- 172.21.15.33:0/1878219900 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f681c008040 con 0x7f6818038b70 2023-11-06T22:18:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.774+0000 7f6824f9a700 1 -- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680405f2d0 msgr2=0x7f6804061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.774+0000 7f6824f9a700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680405f2d0 0x7f6804061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.778+0000 7f68157fa700 1 -- 172.21.15.33:0/1878219900 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f681c04b980 con 0x7f6818038b70 2023-11-06T22:18:41.849 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:41.846+0000 7f42cd07f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T22:18:41.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.958+0000 7f682679d700 1 -- 172.21.15.33:0/1878219900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f681804f8e0 con 0x7f6818038b70 2023-11-06T22:18:41.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.959+0000 7f68157fa700 1 -- 172.21.15.33:0/1878219900 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v224) v1 ==== 110+0+10 (secure 0 0 0) 0x7f681c04efa0 con 0x7f6818038b70 2023-11-06T22:18:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-11-06T22:18:41.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 -- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680405f2d0 msgr2=0x7f6804061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:41.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680405f2d0 0x7f6804061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 -- 172.21.15.33:0/1878219900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818038b70 msgr2=0x7f68180ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818038b70 0x7f68180ce290 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7f681c00cd20 tx=0x7f681c007630).stop 2023-11-06T22:18:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 -- 172.21.15.33:0/1878219900 shutdown_connections 2023-11-06T22:18:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f680405f2d0 0x7f6804061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f68180e1bc0 0x7f6818050940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818038b70 0x7f68180ce290 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 --2- 172.21.15.33:0/1878219900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818054120 0x7f6818050450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 -- 172.21.15.33:0/1878219900 >> 172.21.15.33:0/1878219900 conn(0x7f68180554a0 msgr2=0x7f681803e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 -- 172.21.15.33:0/1878219900 shutdown_connections 2023-11-06T22:18:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:41.961+0000 7f682679d700 1 -- 172.21.15.33:0/1878219900 wait complete. 2023-11-06T22:18:41.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T22:18:41.995 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:41.993+0000 7f2e7097e680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:18:42.093 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:42.091+0000 7f42cd07f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T22:18:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.181+0000 7f75157ff700 1 Processor -- start 2023-11-06T22:18:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.181+0000 7f75157ff700 1 -- start start 2023-11-06T22:18:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.183+0000 7f75157ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7510154710 0x7f75100c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.183+0000 7f75157ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75100c6b60 con 0x7f7510154330 2023-11-06T22:18:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.183+0000 7f75157ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75100c7700 con 0x7f75100c7ff0 2023-11-06T22:18:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.183+0000 7f75157ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75100c5cd0 con 0x7f7510154710 2023-11-06T22:18:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.183+0000 7f750effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7510154330 0x7f75100c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47374/0 (socket says 172.21.15.33:47374) 2023-11-06T22:18:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.183+0000 7f750effd700 1 -- 172.21.15.33:0/1325715614 learned_addr learned my addr 172.21.15.33:0/1325715614 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:42.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.183+0000 7f750e7fc700 1 --2- 172.21.15.33:0/1325715614 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7510154710 0x7f75100c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.183+0000 7f750dffb700 1 -- 172.21.15.33:0/1325715614 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659836733 0 0) 0x7f75100c6b60 con 0x7f7510154330 2023-11-06T22:18:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.184+0000 7f750dffb700 1 -- 172.21.15.33:0/1325715614 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74f80033f0 con 0x7f7510154330 2023-11-06T22:18:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.184+0000 7f750dffb700 1 -- 172.21.15.33:0/1325715614 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219476980 0 0) 0x7f75100c7700 con 0x7f75100c7ff0 2023-11-06T22:18:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.184+0000 7f750dffb700 1 -- 172.21.15.33:0/1325715614 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75100c6b60 con 0x7f75100c7ff0 2023-11-06T22:18:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.184+0000 7f750e7fc700 1 -- 172.21.15.33:0/1325715614 >> v1:172.21.15.69:6789/0 conn(0x7f75100c7ff0 legacy=0x7f75100646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.184+0000 7f750e7fc700 1 -- 172.21.15.33:0/1325715614 >> v1:172.21.15.33:6789/0 conn(0x7f7510154330 legacy=0x7f75100c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.184+0000 7f750e7fc700 1 -- 172.21.15.33:0/1325715614 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75100c5f50 con 0x7f7510154710 2023-11-06T22:18:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.184+0000 7f750e7fc700 1 --2- 172.21.15.33:0/1325715614 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7510154710 0x7f75100c57e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f7504009790 tx=0x7f7504009060).ready entity=mon.2 client_cookie=2b4908cce29e21f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.184+0000 7f750dffb700 1 -- 172.21.15.33:0/1325715614 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750400f510 con 0x7f7510154710 2023-11-06T22:18:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.184+0000 7f750dffb700 1 -- 172.21.15.33:0/1325715614 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750400f670 con 0x7f7510154710 2023-11-06T22:18:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.184+0000 7f750dffb700 1 -- 172.21.15.33:0/1325715614 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750400f9f0 con 0x7f7510154710 2023-11-06T22:18:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.185+0000 7f75157ff700 1 -- 172.21.15.33:0/1325715614 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7510154710 msgr2=0x7f75100c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.185+0000 7f75157ff700 1 --2- 172.21.15.33:0/1325715614 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7510154710 0x7f75100c57e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f7504009790 tx=0x7f7504009060).stop 2023-11-06T22:18:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.185+0000 7f75157ff700 1 -- 172.21.15.33:0/1325715614 shutdown_connections 2023-11-06T22:18:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.185+0000 7f75157ff700 1 --2- 172.21.15.33:0/1325715614 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7510154710 0x7f75100c57e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.185+0000 7f75157ff700 1 -- 172.21.15.33:0/1325715614 >> 172.21.15.33:0/1325715614 conn(0x7f75100cb910 msgr2=0x7f75100c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.185+0000 7f75157ff700 1 -- 172.21.15.33:0/1325715614 shutdown_connections 2023-11-06T22:18:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.185+0000 7f75157ff700 1 -- 172.21.15.33:0/1325715614 wait complete. 2023-11-06T22:18:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.186+0000 7f75157ff700 1 Processor -- start 2023-11-06T22:18:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.187+0000 7f75157ff700 1 -- start start 2023-11-06T22:18:42.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.187+0000 7f75157ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75100c7ff0 0x7f75100b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.187+0000 7f75157ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7510154330 0x7f75101588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.187+0000 7f75157ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7510154710 0x7f7510153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.187+0000 7f75157ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75100b0e30 con 0x7f7510154710 2023-11-06T22:18:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.187+0000 7f75157ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7510158d90 con 0x7f7510154330 2023-11-06T22:18:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.187+0000 7f75157ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7510159010 con 0x7f75100c7ff0 2023-11-06T22:18:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750effd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75100c7ff0 0x7f75100b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7510154710 0x7f7510153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750e7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7510154330 0x7f75101588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7510154710 0x7f7510153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33862/0 (socket says 172.21.15.33:33862) 2023-11-06T22:18:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750f7fe700 1 -- 172.21.15.33:0/2885783721 learned_addr learned my addr 172.21.15.33:0/2885783721 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750effd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75100c7ff0 0x7f75100b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:44318/0 (socket says 172.21.15.33:44318) 2023-11-06T22:18:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750f7fe700 1 -- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75100c7ff0 msgr2=0x7f75100b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750f7fe700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75100c7ff0 0x7f75100b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750f7fe700 1 -- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7510154330 msgr2=0x7f75101588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750f7fe700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7510154330 0x7f75101588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750f7fe700 1 -- 172.21.15.33:0/2885783721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040092a0 con 0x7f7510154710 2023-11-06T22:18:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.188+0000 7f750f7fe700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7510154710 0x7f7510153460 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7f750800c710 tx=0x7f750800ca20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.189+0000 7f74f6ffd700 1 -- 172.21.15.33:0/2885783721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750800e4a0 con 0x7f7510154710 2023-11-06T22:18:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.189+0000 7f74f6ffd700 1 -- 172.21.15.33:0/2885783721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750800e600 con 0x7f7510154710 2023-11-06T22:18:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.189+0000 7f74f6ffd700 1 -- 172.21.15.33:0/2885783721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750800e980 con 0x7f7510154710 2023-11-06T22:18:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.189+0000 7f75157ff700 1 -- 172.21.15.33:0/2885783721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75101592f0 con 0x7f7510154710 2023-11-06T22:18:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.189+0000 7f75157ff700 1 -- 172.21.15.33:0/2885783721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75101598b0 con 0x7f7510154710 2023-11-06T22:18:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.190+0000 7f75157ff700 1 -- 172.21.15.33:0/2885783721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75100830b0 con 0x7f7510154710 2023-11-06T22:18:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.191+0000 7f74f6ffd700 1 -- 172.21.15.33:0/2885783721 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f7508012400 con 0x7f7510154710 2023-11-06T22:18:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.192+0000 7f74f6ffd700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f74fc05f330 0x7f74fc0617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.192+0000 7f74f6ffd700 1 -- 172.21.15.33:0/2885783721 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f7508079ca0 con 0x7f7510154710 2023-11-06T22:18:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.192+0000 7f750effd700 1 -- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f74fc05f330 msgr2=0x7f74fc0617f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.192+0000 7f750effd700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f74fc05f330 0x7f74fc0617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.200+0000 7f74f6ffd700 1 -- 172.21.15.33:0/2885783721 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7508048af0 con 0x7f7510154710 2023-11-06T22:18:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.383+0000 7f75157ff700 1 -- 172.21.15.33:0/2885783721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7510062b00 con 0x7f7510154710 2023-11-06T22:18:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.384+0000 7f74f6ffd700 1 -- 172.21.15.33:0/2885783721 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v224) v1 ==== 92+0+10 (secure 0 0 0) 0x7f750804ab90 con 0x7f7510154710 2023-11-06T22:18:42.387 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-11-06T22:18:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 -- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f74fc05f330 msgr2=0x7f74fc0617f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f74fc05f330 0x7f74fc0617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 -- 172.21.15.33:0/2885783721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7510154710 msgr2=0x7f7510153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7510154710 0x7f7510153460 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7f750800c710 tx=0x7f750800ca20).stop 2023-11-06T22:18:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 -- 172.21.15.33:0/2885783721 shutdown_connections 2023-11-06T22:18:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f74fc05f330 0x7f74fc0617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f75100c7ff0 0x7f75100b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7510154710 0x7f7510153460 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 --2- 172.21.15.33:0/2885783721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7510154330 0x7f75101588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 -- 172.21.15.33:0/2885783721 >> 172.21.15.33:0/2885783721 conn(0x7f75100cb910 msgr2=0x7f75100aa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 -- 172.21.15.33:0/2885783721 shutdown_connections 2023-11-06T22:18:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.386+0000 7f75157ff700 1 -- 172.21.15.33:0/2885783721 wait complete. 2023-11-06T22:18:42.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T22:18:42.510 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:42.508+0000 7f2e7097e680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:18:42.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.613+0000 7f2a603ff700 1 Processor -- start 2023-11-06T22:18:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.613+0000 7f2a603ff700 1 -- start start 2023-11-06T22:18:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.614+0000 7f2a603ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 0x7f2a58062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.614+0000 7f2a603ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2a58062c00 0x7f2a58065380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.614+0000 7f2a603ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a58063070 con 0x7f2a581542d0 2023-11-06T22:18:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.614+0000 7f2a603ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a58069c80 con 0x7f2a58062c00 2023-11-06T22:18:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.614+0000 7f2a603ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58069f60 con 0x7f2a580c8500 2023-11-06T22:18:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5d99a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 0x7f2a58062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5e99c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2a58062c00 0x7f2a58065380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5e19b700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f2a581542d0 0x7f2a580a6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:60368/0 (socket says 172.21.15.33:60368) 2023-11-06T22:18:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5e19b700 1 -- 172.21.15.33:0/286749547 learned_addr learned my addr 172.21.15.33:0/286749547 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5d199700 1 -- 172.21.15.33:0/286749547 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887007210 0 0) 0x7f2a58063070 con 0x7f2a581542d0 2023-11-06T22:18:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5d199700 1 -- 172.21.15.33:0/286749547 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a440033f0 con 0x7f2a581542d0 2023-11-06T22:18:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5e99c700 1 -- 172.21.15.33:0/286749547 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 msgr2=0x7f2a58062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5e99c700 1 --2- 172.21.15.33:0/286749547 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 0x7f2a58062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5e99c700 1 -- 172.21.15.33:0/286749547 >> v1:172.21.15.69:6789/0 conn(0x7f2a581542d0 legacy=0x7f2a580a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5e99c700 1 -- 172.21.15.33:0/286749547 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a580bcb10 con 0x7f2a58062c00 2023-11-06T22:18:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.615+0000 7f2a5d99a700 1 --2- 172.21.15.33:0/286749547 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 0x7f2a58062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.616+0000 7f2a5e99c700 1 --2- 172.21.15.33:0/286749547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a58062c00 0x7f2a58065380 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f2a50009e60 tx=0x7f2a50009780).ready entity=mon.1 client_cookie=57d42352f1670fae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.616+0000 7f2a5d199700 1 -- 172.21.15.33:0/286749547 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5000e070 con 0x7f2a58062c00 2023-11-06T22:18:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.616+0000 7f2a5d199700 1 -- 172.21.15.33:0/286749547 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5000bab0 con 0x7f2a58062c00 2023-11-06T22:18:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.616+0000 7f2a5d199700 1 -- 172.21.15.33:0/286749547 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5000be30 con 0x7f2a58062c00 2023-11-06T22:18:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.616+0000 7f2a603ff700 1 -- 172.21.15.33:0/286749547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a58062c00 msgr2=0x7f2a58065380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.616+0000 7f2a603ff700 1 --2- 172.21.15.33:0/286749547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a58062c00 0x7f2a58065380 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f2a50009e60 tx=0x7f2a50009780).stop 2023-11-06T22:18:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.616+0000 7f2a603ff700 1 -- 172.21.15.33:0/286749547 shutdown_connections 2023-11-06T22:18:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.616+0000 7f2a603ff700 1 --2- 172.21.15.33:0/286749547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a58062c00 0x7f2a58065380 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.616+0000 7f2a603ff700 1 --2- 172.21.15.33:0/286749547 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 0x7f2a58062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.616+0000 7f2a603ff700 1 -- 172.21.15.33:0/286749547 >> 172.21.15.33:0/286749547 conn(0x7f2a580642a0 msgr2=0x7f2a580c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.617+0000 7f2a603ff700 1 -- 172.21.15.33:0/286749547 shutdown_connections 2023-11-06T22:18:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.617+0000 7f2a603ff700 1 -- 172.21.15.33:0/286749547 wait complete. 2023-11-06T22:18:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.618+0000 7f2a603ff700 1 Processor -- start 2023-11-06T22:18:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.618+0000 7f2a603ff700 1 -- start start 2023-11-06T22:18:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a603ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a58062c00 0x7f2a58158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a603ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 0x7f2a5814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a603ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a581542d0 0x7f2a58158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a603ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580abad0 con 0x7f2a58062c00 2023-11-06T22:18:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a603ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580ac090 con 0x7f2a581542d0 2023-11-06T22:18:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a603ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58159ce0 con 0x7f2a580c8500 2023-11-06T22:18:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a5e19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a58062c00 0x7f2a58158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a5d99a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 0x7f2a5814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a5e99c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a581542d0 0x7f2a58158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a5e19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a58062c00 0x7f2a58158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33874/0 (socket says 172.21.15.33:33874) 2023-11-06T22:18:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a5e19b700 1 -- 172.21.15.33:0/2206207961 learned_addr learned my addr 172.21.15.33:0/2206207961 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.619+0000 7f2a5d99a700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 0x7f2a5814f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:44338/0 (socket says 172.21.15.33:44338) 2023-11-06T22:18:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.620+0000 7f2a5e99c700 1 -- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 msgr2=0x7f2a5814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.620+0000 7f2a5e99c700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 0x7f2a5814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.620+0000 7f2a5e99c700 1 -- 172.21.15.33:0/2206207961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a58062c00 msgr2=0x7f2a58158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.620+0000 7f2a5e99c700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a58062c00 0x7f2a58158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.620+0000 7f2a5e99c700 1 -- 172.21.15.33:0/2206207961 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480092a0 con 0x7f2a581542d0 2023-11-06T22:18:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.620+0000 7f2a5e99c700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a581542d0 0x7f2a58158ea0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f2a50014000 tx=0x7f2a50009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.620+0000 7f2a4e7fc700 1 -- 172.21.15.33:0/2206207961 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50013e40 con 0x7f2a581542d0 2023-11-06T22:18:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.620+0000 7f2a4e7fc700 1 -- 172.21.15.33:0/2206207961 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5001d630 con 0x7f2a581542d0 2023-11-06T22:18:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.620+0000 7f2a4e7fc700 1 -- 172.21.15.33:0/2206207961 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5001d9f0 con 0x7f2a581542d0 2023-11-06T22:18:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.620+0000 7f2a603ff700 1 -- 172.21.15.33:0/2206207961 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a50009a20 con 0x7f2a581542d0 2023-11-06T22:18:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.621+0000 7f2a603ff700 1 -- 172.21.15.33:0/2206207961 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5815afe0 con 0x7f2a581542d0 2023-11-06T22:18:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.622+0000 7f2a603ff700 1 -- 172.21.15.33:0/2206207961 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a580bcdf0 con 0x7f2a581542d0 2023-11-06T22:18:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.622+0000 7f2a4e7fc700 1 -- 172.21.15.33:0/2206207961 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f2a50003040 con 0x7f2a581542d0 2023-11-06T22:18:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.623+0000 7f2a4e7fc700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2a3c05f330 0x7f2a3c0617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.623+0000 7f2a4e7fc700 1 -- 172.21.15.33:0/2206207961 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f2a5007c370 con 0x7f2a581542d0 2023-11-06T22:18:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.623+0000 7f2a5e19b700 1 -- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2a3c05f330 msgr2=0x7f2a3c0617f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.623+0000 7f2a5e19b700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2a3c05f330 0x7f2a3c0617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.628+0000 7f2a4e7fc700 1 -- 172.21.15.33:0/2206207961 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2a5004d100 con 0x7f2a581542d0 2023-11-06T22:18:42.636 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:42.634+0000 7f2e7097e680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:42.653 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:42.650+0000 7f42cd07f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T22:18:42.785 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:42.783+0000 7f42cd07f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.812+0000 7f2a603ff700 1 -- 172.21.15.33:0/2206207961 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a58064d10 con 0x7f2a581542d0 2023-11-06T22:18:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.812+0000 7f2a4e7fc700 1 -- 172.21.15.33:0/2206207961 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v224) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a50050720 con 0x7f2a581542d0 2023-11-06T22:18:42.815 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T22:18:42.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.814+0000 7f2a603ff700 1 -- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2a3c05f330 msgr2=0x7f2a3c0617f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:42.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.814+0000 7f2a603ff700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2a3c05f330 0x7f2a3c0617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:42.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.815+0000 7f2a603ff700 1 -- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a581542d0 msgr2=0x7f2a58158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.815+0000 7f2a603ff700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a581542d0 0x7f2a58158ea0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f2a50014000 tx=0x7f2a50009780).stop 2023-11-06T22:18:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.815+0000 7f2a603ff700 1 -- 172.21.15.33:0/2206207961 shutdown_connections 2023-11-06T22:18:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.815+0000 7f2a603ff700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2a3c05f330 0x7f2a3c0617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.815+0000 7f2a603ff700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a581542d0 0x7f2a58158ea0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.815+0000 7f2a603ff700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a58062c00 0x7f2a58158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.815+0000 7f2a603ff700 1 --2- 172.21.15.33:0/2206207961 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2a580c8500 0x7f2a5814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.815+0000 7f2a603ff700 1 -- 172.21.15.33:0/2206207961 >> 172.21.15.33:0/2206207961 conn(0x7f2a580642a0 msgr2=0x7f2a580652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.815+0000 7f2a603ff700 1 -- 172.21.15.33:0/2206207961 shutdown_connections 2023-11-06T22:18:42.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:42.815+0000 7f2a603ff700 1 -- 172.21.15.33:0/2206207961 wait complete. 2023-11-06T22:18:42.822 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:42.819+0000 7f2e7097e680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:42.836 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T22:18:42.917 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:42.915+0000 7f2e7097e680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:18:42.983 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:42.981+0000 7f42cd07f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:43.002 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:43.000+0000 7f2e7097e680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:18:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.038+0000 7ff974bff700 1 Processor -- start 2023-11-06T22:18:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.038+0000 7ff974bff700 1 -- start start 2023-11-06T22:18:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff974bff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9700c4720 0x7ff970157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff974bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9700c4b00 0x7ff9700c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff974bff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff970062710 0x7ff97013d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff974bff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9700c4f60 con 0x7ff9700c4720 2023-11-06T22:18:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff974bff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff97005e490 con 0x7ff970062710 2023-11-06T22:18:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff974bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff970064bf0 con 0x7ff9700c4b00 2023-11-06T22:18:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff96e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9700c4720 0x7ff970157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff96e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9700c4720 0x7ff970157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33886/0 (socket says 172.21.15.33:33886) 2023-11-06T22:18:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff96e59c700 1 -- 172.21.15.33:0/3584979464 learned_addr learned my addr 172.21.15.33:0/3584979464 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:43.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff96dd9b700 1 --2- 172.21.15.33:0/3584979464 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9700c4b00 0x7ff9700c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff96ed9d700 1 --2- 172.21.15.33:0/3584979464 >> v2:172.21.15.69:3300/0 conn(0x7ff970062710 0x7ff97013d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff96e59c700 1 -- 172.21.15.33:0/3584979464 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9700c4b00 msgr2=0x7ff9700c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:43.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.040+0000 7ff96e59c700 1 --2- 172.21.15.33:0/3584979464 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9700c4b00 0x7ff9700c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.041+0000 7ff96e59c700 1 -- 172.21.15.33:0/3584979464 >> v2:172.21.15.69:3300/0 conn(0x7ff970062710 msgr2=0x7ff97013d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.041+0000 7ff96e59c700 1 --2- 172.21.15.33:0/3584979464 >> v2:172.21.15.69:3300/0 conn(0x7ff970062710 0x7ff97013d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.041+0000 7ff96e59c700 1 -- 172.21.15.33:0/3584979464 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff970062ba0 con 0x7ff9700c4720 2023-11-06T22:18:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.041+0000 7ff96e59c700 1 --2- 172.21.15.33:0/3584979464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4720 0x7ff970157e20 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7ff9700c2370 tx=0x7ff9600090f0).ready entity=mon.0 client_cookie=ecf1ee4dcae45015 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:43.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.041+0000 7ff96d59a700 1 -- 172.21.15.33:0/3584979464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000b600 con 0x7ff9700c4720 2023-11-06T22:18:43.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.041+0000 7ff96d59a700 1 -- 172.21.15.33:0/3584979464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96000b760 con 0x7ff9700c4720 2023-11-06T22:18:43.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.041+0000 7ff96d59a700 1 -- 172.21.15.33:0/3584979464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000bae0 con 0x7ff9700c4720 2023-11-06T22:18:43.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.042+0000 7ff974bff700 1 -- 172.21.15.33:0/3584979464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4720 msgr2=0x7ff970157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.042+0000 7ff974bff700 1 --2- 172.21.15.33:0/3584979464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4720 0x7ff970157e20 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7ff9700c2370 tx=0x7ff9600090f0).stop 2023-11-06T22:18:43.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.042+0000 7ff974bff700 1 -- 172.21.15.33:0/3584979464 shutdown_connections 2023-11-06T22:18:43.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.042+0000 7ff974bff700 1 --2- 172.21.15.33:0/3584979464 >> v2:172.21.15.69:3300/0 conn(0x7ff970062710 0x7ff97013d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.042+0000 7ff974bff700 1 --2- 172.21.15.33:0/3584979464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4720 0x7ff970157e20 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.042+0000 7ff974bff700 1 --2- 172.21.15.33:0/3584979464 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9700c4b00 0x7ff9700c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.042+0000 7ff974bff700 1 -- 172.21.15.33:0/3584979464 >> 172.21.15.33:0/3584979464 conn(0x7ff9700c7dc0 msgr2=0x7ff9700c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.042+0000 7ff974bff700 1 -- 172.21.15.33:0/3584979464 shutdown_connections 2023-11-06T22:18:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.042+0000 7ff974bff700 1 -- 172.21.15.33:0/3584979464 wait complete. 2023-11-06T22:18:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.043+0000 7ff974bff700 1 Processor -- start 2023-11-06T22:18:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.043+0000 7ff974bff700 1 -- start start 2023-11-06T22:18:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.044+0000 7ff974bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970062710 0x7ff970165430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.044+0000 7ff974bff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9700c4720 0x7ff9701382e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.044+0000 7ff974bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4b00 0x7ff9701362b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.044+0000 7ff974bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff970134730 con 0x7ff9700c4b00 2023-11-06T22:18:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.044+0000 7ff974bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff970138c10 con 0x7ff970062710 2023-11-06T22:18:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.044+0000 7ff974bff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff970138ef0 con 0x7ff9700c4720 2023-11-06T22:18:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4b00 0x7ff9701362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4b00 0x7ff9701362b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33892/0 (socket says 172.21.15.33:33892) 2023-11-06T22:18:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96ed9d700 1 -- 172.21.15.33:0/1388646632 learned_addr learned my addr 172.21.15.33:0/1388646632 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96e59c700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970062710 0x7ff970165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96dd9b700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9700c4720 0x7ff9701382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96ed9d700 1 -- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9700c4720 msgr2=0x7ff9701382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96ed9d700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9700c4720 0x7ff9701382e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96ed9d700 1 -- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970062710 msgr2=0x7ff970165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96ed9d700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970062710 0x7ff970165430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96ed9d700 1 -- 172.21.15.33:0/1388646632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960009370 con 0x7ff9700c4b00 2023-11-06T22:18:43.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff96ed9d700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4b00 0x7ff9701362b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7ff968009fa0 tx=0x7ff968009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:43.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.045+0000 7ff95e7fc700 1 -- 172.21.15.33:0/1388646632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800ebc0 con 0x7ff9700c4b00 2023-11-06T22:18:43.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.046+0000 7ff95e7fc700 1 -- 172.21.15.33:0/1388646632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96800ed20 con 0x7ff9700c4b00 2023-11-06T22:18:43.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.046+0000 7ff974bff700 1 -- 172.21.15.33:0/1388646632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9701391d0 con 0x7ff9700c4b00 2023-11-06T22:18:43.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.046+0000 7ff95e7fc700 1 -- 172.21.15.33:0/1388646632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968014610 con 0x7ff9700c4b00 2023-11-06T22:18:43.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.046+0000 7ff974bff700 1 -- 172.21.15.33:0/1388646632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9701387d0 con 0x7ff9700c4b00 2023-11-06T22:18:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.047+0000 7ff974bff700 1 -- 172.21.15.33:0/1388646632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9701348e0 con 0x7ff9700c4b00 2023-11-06T22:18:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.047+0000 7ff95e7fc700 1 -- 172.21.15.33:0/1388646632 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7ff96801d3b0 con 0x7ff9700c4b00 2023-11-06T22:18:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.048+0000 7ff95e7fc700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff95005f300 0x7ff9500617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.048+0000 7ff95e7fc700 1 -- 172.21.15.33:0/1388646632 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7ff96807a990 con 0x7ff9700c4b00 2023-11-06T22:18:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.049+0000 7ff96e59c700 1 -- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff95005f300 msgr2=0x7ff9500617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.049+0000 7ff96e59c700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff95005f300 0x7ff9500617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.053+0000 7ff95e7fc700 1 -- 172.21.15.33:0/1388646632 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff96804b8d0 con 0x7ff9700c4b00 2023-11-06T22:18:43.088 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:43.086+0000 7f42cd07f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T22:18:43.178 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:43.176+0000 7f42cd07f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T22:18:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.236+0000 7ff974bff700 1 -- 172.21.15.33:0/1388646632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff9700aa0d0 con 0x7ff9700c4b00 2023-11-06T22:18:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.236+0000 7ff95e7fc700 1 -- 172.21.15.33:0/1388646632 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v224) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff96804eef0 con 0x7ff9700c4b00 2023-11-06T22:18:43.239 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-11-06T22:18:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.238+0000 7ff974bff700 1 -- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff95005f300 msgr2=0x7ff9500617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.238+0000 7ff974bff700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff95005f300 0x7ff9500617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.239+0000 7ff974bff700 1 -- 172.21.15.33:0/1388646632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4b00 msgr2=0x7ff9701362b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.239+0000 7ff974bff700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4b00 0x7ff9701362b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7ff968009fa0 tx=0x7ff968009900).stop 2023-11-06T22:18:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.239+0000 7ff974bff700 1 -- 172.21.15.33:0/1388646632 shutdown_connections 2023-11-06T22:18:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.239+0000 7ff974bff700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff95005f300 0x7ff9500617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.239+0000 7ff974bff700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9700c4b00 0x7ff9701362b0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.239+0000 7ff974bff700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff970062710 0x7ff970165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.239+0000 7ff974bff700 1 --2- 172.21.15.33:0/1388646632 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff9700c4720 0x7ff9701382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.239+0000 7ff974bff700 1 -- 172.21.15.33:0/1388646632 >> 172.21.15.33:0/1388646632 conn(0x7ff9700c7dc0 msgr2=0x7ff970064f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.239+0000 7ff974bff700 1 -- 172.21.15.33:0/1388646632 shutdown_connections 2023-11-06T22:18:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.239+0000 7ff974bff700 1 -- 172.21.15.33:0/1388646632 wait complete. 2023-11-06T22:18:43.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:43.315 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:43.313+0000 7f2e7097e680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:18:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.457+0000 7f27521ff700 1 Processor -- start 2023-11-06T22:18:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.457+0000 7f27521ff700 1 -- start start 2023-11-06T22:18:43.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.458+0000 7f27521ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f274c151dc0 0x7f274c0c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.458+0000 7f27521ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c155320 0x7f274c0c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.458+0000 7f27521ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274c154e90 con 0x7f274c1521a0 2023-11-06T22:18:43.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.458+0000 7f27521ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274c0639d0 con 0x7f274c151dc0 2023-11-06T22:18:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.458+0000 7f27521ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c063cb0 con 0x7f274c155320 2023-11-06T22:18:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.458+0000 7f274b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f274c151dc0 0x7f274c0c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.458+0000 7f274b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f274c151dc0 0x7f274c0c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33904/0 (socket says 172.21.15.33:33904) 2023-11-06T22:18:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.459+0000 7f274b7fe700 1 -- 172.21.15.33:0/526649580 learned_addr learned my addr 172.21.15.33:0/526649580 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.459+0000 7f274bfff700 1 --2- 172.21.15.33:0/526649580 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c155320 0x7f274c0c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.459+0000 7f274b7fe700 1 -- 172.21.15.33:0/526649580 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c155320 msgr2=0x7f274c0c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.459+0000 7f274b7fe700 1 --2- 172.21.15.33:0/526649580 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c155320 0x7f274c0c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.459+0000 7f274b7fe700 1 -- 172.21.15.33:0/526649580 >> v1:172.21.15.69:6789/0 conn(0x7f274c1521a0 legacy=0x7f274c063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.459+0000 7f274b7fe700 1 -- 172.21.15.33:0/526649580 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c063f90 con 0x7f274c151dc0 2023-11-06T22:18:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.459+0000 7f274b7fe700 1 --2- 172.21.15.33:0/526649580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c151dc0 0x7f274c0c0c50 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f273c006fd0 tx=0x7f273c0090f0).ready entity=mon.0 client_cookie=bebc622608b0e378 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.459+0000 7f274a7fc700 1 -- 172.21.15.33:0/526649580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c00b580 con 0x7f274c151dc0 2023-11-06T22:18:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.460+0000 7f274a7fc700 1 -- 172.21.15.33:0/526649580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c00b6e0 con 0x7f274c151dc0 2023-11-06T22:18:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.460+0000 7f274a7fc700 1 -- 172.21.15.33:0/526649580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c00ba60 con 0x7f274c151dc0 2023-11-06T22:18:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.460+0000 7f27521ff700 1 -- 172.21.15.33:0/526649580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c151dc0 msgr2=0x7f274c0c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.460+0000 7f27521ff700 1 --2- 172.21.15.33:0/526649580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c151dc0 0x7f274c0c0c50 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f273c006fd0 tx=0x7f273c0090f0).stop 2023-11-06T22:18:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.460+0000 7f27521ff700 1 -- 172.21.15.33:0/526649580 shutdown_connections 2023-11-06T22:18:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.460+0000 7f27521ff700 1 --2- 172.21.15.33:0/526649580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c151dc0 0x7f274c0c0c50 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.460+0000 7f27521ff700 1 --2- 172.21.15.33:0/526649580 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c155320 0x7f274c0c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.460+0000 7f27521ff700 1 -- 172.21.15.33:0/526649580 >> 172.21.15.33:0/526649580 conn(0x7f274c05e1b0 msgr2=0x7f274c05e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.460+0000 7f27521ff700 1 -- 172.21.15.33:0/526649580 shutdown_connections 2023-11-06T22:18:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.460+0000 7f27521ff700 1 -- 172.21.15.33:0/526649580 wait complete. 2023-11-06T22:18:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.462+0000 7f27521ff700 1 Processor -- start 2023-11-06T22:18:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.462+0000 7f27521ff700 1 -- start start 2023-11-06T22:18:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f27521ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c151dc0 0x7f274c0b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f27521ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c1521a0 0x7f274c0c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f27521ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f274c155320 0x7f274c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f27521ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0c4cc0 con 0x7f274c1521a0 2023-11-06T22:18:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f27521ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c165910 con 0x7f274c155320 2023-11-06T22:18:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f27521ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c164a70 con 0x7f274c151dc0 2023-11-06T22:18:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f274affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c1521a0 0x7f274c0c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f274bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f274c155320 0x7f274c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f274b7fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c151dc0 0x7f274c0b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f274affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c1521a0 0x7f274c0c34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33906/0 (socket says 172.21.15.33:33906) 2023-11-06T22:18:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.463+0000 7f274affd700 1 -- 172.21.15.33:0/3772934944 learned_addr learned my addr 172.21.15.33:0/3772934944 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f274b7fe700 1 -- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f274c155320 msgr2=0x7f274c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f274b7fe700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f274c155320 0x7f274c0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f274b7fe700 1 -- 172.21.15.33:0/3772934944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c1521a0 msgr2=0x7f274c0c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f274b7fe700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c1521a0 0x7f274c0c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f274b7fe700 1 -- 172.21.15.33:0/3772934944 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c009370 con 0x7f274c151dc0 2023-11-06T22:18:43.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f274affd700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c1521a0 0x7f274c0c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:43.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f274b7fe700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c151dc0 0x7f274c0b0710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f273c014000 tx=0x7f273c017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:43.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f27337fe700 1 -- 172.21.15.33:0/3772934944 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c006d90 con 0x7f274c151dc0 2023-11-06T22:18:43.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f27337fe700 1 -- 172.21.15.33:0/3772934944 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c000bc0 con 0x7f274c151dc0 2023-11-06T22:18:43.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f27337fe700 1 -- 172.21.15.33:0/3772934944 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c01b410 con 0x7f274c151dc0 2023-11-06T22:18:43.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.464+0000 7f27521ff700 1 -- 172.21.15.33:0/3772934944 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c164d00 con 0x7f274c151dc0 2023-11-06T22:18:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.465+0000 7f27521ff700 1 -- 172.21.15.33:0/3772934944 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c1652c0 con 0x7f274c151dc0 2023-11-06T22:18:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.466+0000 7f27521ff700 1 -- 172.21.15.33:0/3772934944 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c06bc10 con 0x7f274c151dc0 2023-11-06T22:18:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.466+0000 7f27337fe700 1 -- 172.21.15.33:0/3772934944 <== mon.2 v2:172.21.15.69:3301/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f273c01b9d0 con 0x7f274c151dc0 2023-11-06T22:18:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.467+0000 7f27337fe700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f273805f300 0x7f27380617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.467+0000 7f27337fe700 1 -- 172.21.15.33:0/3772934944 <== mon.2 v2:172.21.15.69:3301/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f273c07c1b0 con 0x7f274c151dc0 2023-11-06T22:18:43.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.468+0000 7f274affd700 1 -- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f273805f300 msgr2=0x7f27380617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:43.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.468+0000 7f274affd700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f273805f300 0x7f27380617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:43.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.472+0000 7f27337fe700 1 -- 172.21.15.33:0/3772934944 <== mon.2 v2:172.21.15.69:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f273c04d050 con 0x7f274c151dc0 2023-11-06T22:18:43.517 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:43.515+0000 7f42cd07f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T22:18:43.552 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:43.550+0000 7f2e7097e680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:43.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.669+0000 7f274affd700 1 -- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f273805f300 msgr2=0x7f27380617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:43.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.669+0000 7f274affd700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f273805f300 0x7f27380617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:18:43.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.701+0000 7f27521ff700 1 -- 172.21.15.33:0/3772934944 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f274c062ac0 con 0x7f274c151dc0 2023-11-06T22:18:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.702+0000 7f27337fe700 1 -- 172.21.15.33:0/3772934944 <== mon.2 v2:172.21.15.69:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f273c029090 con 0x7f274c151dc0 2023-11-06T22:18:43.704 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:43.705 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":91,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3097639551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3097639551},{"type":"v1","addr":"172.21.15.69:6837","nonce":3097639551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9828,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1256245016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1256245016},{"type":"v1","addr":"172.21.15.33:6834","nonce":1256245016}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1680210655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1680210655},{"type":"v1","addr":"172.21.15.33:6835","nonce":1680210655}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:18:22.834249+0000","modified":"2023-11-06T22:18:25.840897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9837":{"gid":9837,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/3921249578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3921249578},{"type":"v1","addr":"172.21.15.69:6836","nonce":3921249578}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-06T22:18:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 -- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f273805f300 msgr2=0x7f27380617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f273805f300 0x7f27380617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 -- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c151dc0 msgr2=0x7f274c0b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c151dc0 0x7f274c0b0710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f273c014000 tx=0x7f273c017000).stop 2023-11-06T22:18:43.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 -- 172.21.15.33:0/3772934944 shutdown_connections 2023-11-06T22:18:43.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f273805f300 0x7f27380617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:43.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f274c151dc0 0x7f274c0b0710 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f274c155320 0x7f274c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 --2- 172.21.15.33:0/3772934944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f274c1521a0 0x7f274c0c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 -- 172.21.15.33:0/3772934944 >> 172.21.15.33:0/3772934944 conn(0x7f274c05e1b0 msgr2=0x7f274c0a9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 -- 172.21.15.33:0/3772934944 shutdown_connections 2023-11-06T22:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.704+0000 7f27521ff700 1 -- 172.21.15.33:0/3772934944 wait complete. 2023-11-06T22:18:43.710 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 91 2023-11-06T22:18:43.725 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:43.750 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:43.748+0000 7f42cd07f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T22:18:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.922+0000 7f6f261ff700 1 Processor -- start 2023-11-06T22:18:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.922+0000 7f6f261ff700 1 -- start start 2023-11-06T22:18:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.923+0000 7f6f261ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6f20157720 0x7f6f20158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f261ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f200a5080 0x7f6f200c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f261ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f200bbfb0 con 0x7f6f200bccc0 2023-11-06T22:18:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f261ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f20158d30 con 0x7f6f200a5080 2023-11-06T22:18:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f261ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f200bdb90 con 0x7f6f20157720 2023-11-06T22:18:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f1ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f200a5080 0x7f6f200c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f1ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f200a5080 0x7f6f200c1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33912/0 (socket says 172.21.15.33:33912) 2023-11-06T22:18:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f1ffff700 1 -- 172.21.15.33:0/422541893 learned_addr learned my addr 172.21.15.33:0/422541893 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f1f7fe700 1 --2- 172.21.15.33:0/422541893 >> v2:172.21.15.69:3300/0 conn(0x7f6f20157720 0x7f6f20158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f1ffff700 1 -- 172.21.15.33:0/422541893 >> v2:172.21.15.69:3300/0 conn(0x7f6f20157720 msgr2=0x7f6f20158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f1ffff700 1 --2- 172.21.15.33:0/422541893 >> v2:172.21.15.69:3300/0 conn(0x7f6f20157720 0x7f6f20158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.924+0000 7f6f1ffff700 1 -- 172.21.15.33:0/422541893 >> v1:172.21.15.69:6789/0 conn(0x7f6f200bccc0 legacy=0x7f6f200a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.925+0000 7f6f1ffff700 1 -- 172.21.15.33:0/422541893 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f200bc1e0 con 0x7f6f200a5080 2023-11-06T22:18:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.925+0000 7f6f1f7fe700 1 --2- 172.21.15.33:0/422541893 >> v2:172.21.15.69:3300/0 conn(0x7f6f20157720 0x7f6f20158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.925+0000 7f6f1ffff700 1 --2- 172.21.15.33:0/422541893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f200a5080 0x7f6f200c1840 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f6f20067510 tx=0x7f6f18009840).ready entity=mon.0 client_cookie=cd93d55f12eeadd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.925+0000 7f6f1e7fc700 1 -- 172.21.15.33:0/422541893 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1800e070 con 0x7f6f200a5080 2023-11-06T22:18:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.925+0000 7f6f1e7fc700 1 -- 172.21.15.33:0/422541893 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1800ba20 con 0x7f6f200a5080 2023-11-06T22:18:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.925+0000 7f6f1e7fc700 1 -- 172.21.15.33:0/422541893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1800bda0 con 0x7f6f200a5080 2023-11-06T22:18:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.926+0000 7f6f261ff700 1 -- 172.21.15.33:0/422541893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f200a5080 msgr2=0x7f6f200c1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.926+0000 7f6f261ff700 1 --2- 172.21.15.33:0/422541893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f200a5080 0x7f6f200c1840 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f6f20067510 tx=0x7f6f18009840).stop 2023-11-06T22:18:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.926+0000 7f6f261ff700 1 -- 172.21.15.33:0/422541893 shutdown_connections 2023-11-06T22:18:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.926+0000 7f6f261ff700 1 --2- 172.21.15.33:0/422541893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f200a5080 0x7f6f200c1840 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.926+0000 7f6f261ff700 1 --2- 172.21.15.33:0/422541893 >> v2:172.21.15.69:3300/0 conn(0x7f6f20157720 0x7f6f20158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.926+0000 7f6f261ff700 1 -- 172.21.15.33:0/422541893 >> 172.21.15.33:0/422541893 conn(0x7f6f200631e0 msgr2=0x7f6f200c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.926+0000 7f6f261ff700 1 -- 172.21.15.33:0/422541893 shutdown_connections 2023-11-06T22:18:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.926+0000 7f6f261ff700 1 -- 172.21.15.33:0/422541893 wait complete. 2023-11-06T22:18:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.927+0000 7f6f261ff700 1 Processor -- start 2023-11-06T22:18:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.927+0000 7f6f261ff700 1 -- start start 2023-11-06T22:18:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.928+0000 7f6f261ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f200a5080 0x7f6f2014c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.928+0000 7f6f261ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f200bccc0 0x7f6f20159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.928+0000 7f6f261ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f20157720 0x7f6f2015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.928+0000 7f6f261ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200aa8b0 con 0x7f6f20157720 2023-11-06T22:18:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.928+0000 7f6f261ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2015a8b0 con 0x7f6f200a5080 2023-11-06T22:18:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f261ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f201505c0 con 0x7f6f200bccc0 2023-11-06T22:18:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f20157720 0x7f6f2015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f20157720 0x7f6f2015a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33924/0 (socket says 172.21.15.33:33924) 2023-11-06T22:18:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1ffff700 1 -- 172.21.15.33:0/2922042813 learned_addr learned my addr 172.21.15.33:0/2922042813 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1f7fe700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f200a5080 0x7f6f2014c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1effd700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f200bccc0 0x7f6f20159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1ffff700 1 -- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f200bccc0 msgr2=0x7f6f20159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1ffff700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f200bccc0 0x7f6f20159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1ffff700 1 -- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f200a5080 msgr2=0x7f6f2014c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1ffff700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f200a5080 0x7f6f2014c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1ffff700 1 -- 172.21.15.33:0/2922042813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f100092a0 con 0x7f6f20157720 2023-11-06T22:18:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.929+0000 7f6f1f7fe700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f200a5080 0x7f6f2014c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.930+0000 7f6f1ffff700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f20157720 0x7f6f2015a3c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7f6f20063e90 tx=0x7f6f18009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.930+0000 7f6f077fe700 1 -- 172.21.15.33:0/2922042813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18009500 con 0x7f6f20157720 2023-11-06T22:18:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.930+0000 7f6f077fe700 1 -- 172.21.15.33:0/2922042813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f180092e0 con 0x7f6f20157720 2023-11-06T22:18:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.930+0000 7f6f077fe700 1 -- 172.21.15.33:0/2922042813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1801e6c0 con 0x7f6f20157720 2023-11-06T22:18:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.930+0000 7f6f261ff700 1 -- 172.21.15.33:0/2922042813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f18009ae0 con 0x7f6f20157720 2023-11-06T22:18:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.930+0000 7f6f261ff700 1 -- 172.21.15.33:0/2922042813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f20150000 con 0x7f6f20157720 2023-11-06T22:18:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.931+0000 7f6f261ff700 1 -- 172.21.15.33:0/2922042813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f200aa9f0 con 0x7f6f20157720 2023-11-06T22:18:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.932+0000 7f6f077fe700 1 -- 172.21.15.33:0/2922042813 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f6f18003040 con 0x7f6f20157720 2023-11-06T22:18:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.933+0000 7f6f077fe700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6f0c05f330 0x7f6f0c0617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.933+0000 7f6f077fe700 1 -- 172.21.15.33:0/2922042813 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f6f1807ba90 con 0x7f6f20157720 2023-11-06T22:18:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.933+0000 7f6f1f7fe700 1 -- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6f0c05f330 msgr2=0x7f6f0c0617f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.933+0000 7f6f1f7fe700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6f0c05f330 0x7f6f0c0617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:43.937+0000 7f6f077fe700 1 -- 172.21.15.33:0/2922042813 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6f200aa9f0 con 0x7f6f20157720 2023-11-06T22:18:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.120+0000 7f6f261ff700 1 -- 172.21.15.33:0/2922042813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f200aa9f0 con 0x7f6f20157720 2023-11-06T22:18:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.121+0000 7f6f077fe700 1 -- 172.21.15.33:0/2922042813 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v224) v1 ==== 74+0+17429 (secure 0 0 0) 0x7f6f200aa9f0 con 0x7f6f20157720 2023-11-06T22:18:44.124 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:44.125 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":224,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:37.037489+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:18:21.317255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T22:18:21.822232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/672707063":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/2065981382":"2023-11-07T22:18:37.037458+0000","172.21.15.69:6833/129946":"2023-11-07T22:18:37.037458+0000","172.21.15.69:6832/129946":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/3481533320":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/2510047576":"2023-11-07T22:18:37.037458+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.123+0000 7f6f261ff700 1 -- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6f0c05f330 msgr2=0x7f6f0c0617f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.123+0000 7f6f261ff700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6f0c05f330 0x7f6f0c0617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.123+0000 7f6f261ff700 1 -- 172.21.15.33:0/2922042813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f20157720 msgr2=0x7f6f2015a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.123+0000 7f6f261ff700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f20157720 0x7f6f2015a3c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7f6f20063e90 tx=0x7f6f18009840).stop 2023-11-06T22:18:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.124+0000 7f6f261ff700 1 -- 172.21.15.33:0/2922042813 shutdown_connections 2023-11-06T22:18:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.124+0000 7f6f261ff700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f6f0c05f330 0x7f6f0c0617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.124+0000 7f6f261ff700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f20157720 0x7f6f2015a3c0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.124+0000 7f6f261ff700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f200a5080 0x7f6f2014c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.124+0000 7f6f261ff700 1 --2- 172.21.15.33:0/2922042813 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6f200bccc0 0x7f6f20159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.124+0000 7f6f261ff700 1 -- 172.21.15.33:0/2922042813 >> 172.21.15.33:0/2922042813 conn(0x7f6f200631e0 msgr2=0x7f6f200c5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.124+0000 7f6f261ff700 1 -- 172.21.15.33:0/2922042813 shutdown_connections 2023-11-06T22:18:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.124+0000 7f6f261ff700 1 -- 172.21.15.33:0/2922042813 wait complete. 2023-11-06T22:18:44.144 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T22:18:44.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:44.358 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:44.356+0000 7f2e7097e680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:18:44.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.382+0000 7fe8f9fff700 1 Processor -- start 2023-11-06T22:18:44.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.383+0000 7fe8f9fff700 1 -- start start 2023-11-06T22:18:44.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.384+0000 7fe8f9fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 0x7fe8f40c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.384+0000 7fe8f9fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe8f41521a0 0x7fe8f4063110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.384+0000 7fe8f9fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f4154e90 con 0x7fe8f4155320 2023-11-06T22:18:44.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.384+0000 7fe8f9fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f40639d0 con 0x7fe8f41521a0 2023-11-06T22:18:44.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.384+0000 7fe8f9fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4063cb0 con 0x7fe8f4151dc0 2023-11-06T22:18:44.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.384+0000 7fe8f3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe8f4155320 0x7fe8f40c7850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47378/0 (socket says 172.21.15.33:47378) 2023-11-06T22:18:44.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.384+0000 7fe8f3fff700 1 -- 172.21.15.33:0/2270222603 learned_addr learned my addr 172.21.15.33:0/2270222603 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:44.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.384+0000 7fe8f2ffd700 1 --2- 172.21.15.33:0/2270222603 >> v2:172.21.15.69:3300/0 conn(0x7fe8f41521a0 0x7fe8f4063110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:44.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.384+0000 7fe8f37fe700 1 --2- 172.21.15.33:0/2270222603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 0x7fe8f40c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:44.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f27fc700 1 -- 172.21.15.33:0/2270222603 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144934209 0 0) 0x7fe8f4154e90 con 0x7fe8f4155320 2023-11-06T22:18:44.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f27fc700 1 -- 172.21.15.33:0/2270222603 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8dc0033f0 con 0x7fe8f4155320 2023-11-06T22:18:44.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f2ffd700 1 -- 172.21.15.33:0/2270222603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 msgr2=0x7fe8f40c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:44.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f2ffd700 1 --2- 172.21.15.33:0/2270222603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 0x7fe8f40c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f2ffd700 1 -- 172.21.15.33:0/2270222603 >> v1:172.21.15.33:6789/0 conn(0x7fe8f4155320 legacy=0x7fe8f40c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f2ffd700 1 -- 172.21.15.33:0/2270222603 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f4063f90 con 0x7fe8f41521a0 2023-11-06T22:18:44.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f37fe700 1 --2- 172.21.15.33:0/2270222603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 0x7fe8f40c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:44.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f2ffd700 1 --2- 172.21.15.33:0/2270222603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8f41521a0 0x7fe8f4063110 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7fe8e8009760 tx=0x7fe8e8009060).ready entity=mon.1 client_cookie=58386b075f1527c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:44.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f27fc700 1 -- 172.21.15.33:0/2270222603 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800d4a0 con 0x7fe8f41521a0 2023-11-06T22:18:44.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f27fc700 1 -- 172.21.15.33:0/2270222603 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e800d600 con 0x7fe8f41521a0 2023-11-06T22:18:44.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.385+0000 7fe8f27fc700 1 -- 172.21.15.33:0/2270222603 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800d980 con 0x7fe8f41521a0 2023-11-06T22:18:44.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.386+0000 7fe8f9fff700 1 -- 172.21.15.33:0/2270222603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8f41521a0 msgr2=0x7fe8f4063110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.386+0000 7fe8f9fff700 1 --2- 172.21.15.33:0/2270222603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8f41521a0 0x7fe8f4063110 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7fe8e8009760 tx=0x7fe8e8009060).stop 2023-11-06T22:18:44.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.386+0000 7fe8f9fff700 1 -- 172.21.15.33:0/2270222603 shutdown_connections 2023-11-06T22:18:44.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.386+0000 7fe8f9fff700 1 --2- 172.21.15.33:0/2270222603 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 0x7fe8f40c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.386+0000 7fe8f9fff700 1 --2- 172.21.15.33:0/2270222603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8f41521a0 0x7fe8f4063110 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.386+0000 7fe8f9fff700 1 -- 172.21.15.33:0/2270222603 >> 172.21.15.33:0/2270222603 conn(0x7fe8f405e1b0 msgr2=0x7fe8f405e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:44.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.386+0000 7fe8f9fff700 1 -- 172.21.15.33:0/2270222603 shutdown_connections 2023-11-06T22:18:44.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.386+0000 7fe8f9fff700 1 -- 172.21.15.33:0/2270222603 wait complete. 2023-11-06T22:18:44.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.388+0000 7fe8f9fff700 1 Processor -- start 2023-11-06T22:18:44.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.388+0000 7fe8f9fff700 1 -- start start 2023-11-06T22:18:44.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f9fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 0x7fe8f40c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f9fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8f41521a0 0x7fe8f4153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f9fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4155320 0x7fe8f40ac070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f9fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4151180 con 0x7fe8f4155320 2023-11-06T22:18:44.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f9fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f40c2550 con 0x7fe8f41521a0 2023-11-06T22:18:44.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f9fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f40c2830 con 0x7fe8f4151dc0 2023-11-06T22:18:44.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4155320 0x7fe8f40ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:44.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f37fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 0x7fe8f40c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4155320 0x7fe8f40ac070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33928/0 (socket says 172.21.15.33:33928) 2023-11-06T22:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f3fff700 1 -- 172.21.15.33:0/4043822018 learned_addr learned my addr 172.21.15.33:0/4043822018 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.389+0000 7fe8f2ffd700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8f41521a0 0x7fe8f4153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8f2ffd700 1 -- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 msgr2=0x7fe8f40c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8f2ffd700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 0x7fe8f40c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8f2ffd700 1 -- 172.21.15.33:0/4043822018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4155320 msgr2=0x7fe8f40ac070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8f2ffd700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4155320 0x7fe8f40ac070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8f2ffd700 1 -- 172.21.15.33:0/4043822018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e40092a0 con 0x7fe8f41521a0 2023-11-06T22:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8f37fe700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 0x7fe8f40c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8f3fff700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4155320 0x7fe8f40ac070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:44.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8f2ffd700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8f41521a0 0x7fe8f4153d40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7fe8e8013000 tx=0x7fe8e8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:44.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8db7fe700 1 -- 172.21.15.33:0/4043822018 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800dd30 con 0x7fe8f41521a0 2023-11-06T22:18:44.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8db7fe700 1 -- 172.21.15.33:0/4043822018 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e800de90 con 0x7fe8f41521a0 2023-11-06T22:18:44.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8f9fff700 1 -- 172.21.15.33:0/4043822018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e80092a0 con 0x7fe8f41521a0 2023-11-06T22:18:44.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8db7fe700 1 -- 172.21.15.33:0/4043822018 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800d980 con 0x7fe8f41521a0 2023-11-06T22:18:44.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.390+0000 7fe8f9fff700 1 -- 172.21.15.33:0/4043822018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f40c2df0 con 0x7fe8f41521a0 2023-11-06T22:18:44.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.392+0000 7fe8f9fff700 1 -- 172.21.15.33:0/4043822018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f4154b20 con 0x7fe8f41521a0 2023-11-06T22:18:44.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.392+0000 7fe8db7fe700 1 -- 172.21.15.33:0/4043822018 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fe8e801d640 con 0x7fe8f41521a0 2023-11-06T22:18:44.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.393+0000 7fe8db7fe700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe8e005f300 0x7fe8e00617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.393+0000 7fe8db7fe700 1 -- 172.21.15.33:0/4043822018 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7fe8e807bbb0 con 0x7fe8f41521a0 2023-11-06T22:18:44.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.393+0000 7fe8f37fe700 1 -- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe8e005f300 msgr2=0x7fe8e00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:44.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.393+0000 7fe8f37fe700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe8e005f300 0x7fe8e00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:44.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.397+0000 7fe8db7fe700 1 -- 172.21.15.33:0/4043822018 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe8e8082080 con 0x7fe8f41521a0 2023-11-06T22:18:44.447 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:44.445+0000 7f2e7097e680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:18:44.542 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:44.540+0000 7f2e7097e680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:18:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.594+0000 7fe8f37fe700 1 -- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe8e005f300 msgr2=0x7fe8e00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:44.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.594+0000 7fe8f37fe700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe8e005f300 0x7fe8e00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:18:44.617 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:44.614+0000 7f42cd07f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-06T22:18:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.620+0000 7fe8f9fff700 1 -- 172.21.15.33:0/4043822018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8f4062ac0 con 0x7fe8f41521a0 2023-11-06T22:18:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.621+0000 7fe8db7fe700 1 -- 172.21.15.33:0/4043822018 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4493 (secure 0 0 0) 0x7fe8e804cb50 con 0x7fe8f41521a0 2023-11-06T22:18:44.624 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:44.624 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":91,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3097639551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3097639551},{"type":"v1","addr":"172.21.15.69:6837","nonce":3097639551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9828,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1256245016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1256245016},{"type":"v1","addr":"172.21.15.33:6834","nonce":1256245016}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1680210655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1680210655},{"type":"v1","addr":"172.21.15.33:6835","nonce":1680210655}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:18:22.834249+0000","modified":"2023-11-06T22:18:25.840897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9837":{"gid":9837,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/3921249578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3921249578},{"type":"v1","addr":"172.21.15.69:6836","nonce":3921249578}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-06T22:18:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 -- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe8e005f300 msgr2=0x7fe8e00617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe8e005f300 0x7fe8e00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 -- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8f41521a0 msgr2=0x7fe8f4153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8f41521a0 0x7fe8f4153d40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7fe8e8013000 tx=0x7fe8e8016000).stop 2023-11-06T22:18:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 -- 172.21.15.33:0/4043822018 shutdown_connections 2023-11-06T22:18:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe8e005f300 0x7fe8e00617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe8f4151dc0 0x7fe8f40c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4155320 0x7fe8f40ac070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 --2- 172.21.15.33:0/4043822018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8f41521a0 0x7fe8f4153d40 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 -- 172.21.15.33:0/4043822018 >> 172.21.15.33:0/4043822018 conn(0x7fe8f405e1b0 msgr2=0x7fe8f415a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 -- 172.21.15.33:0/4043822018 shutdown_connections 2023-11-06T22:18:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.623+0000 7fe8f9fff700 1 -- 172.21.15.33:0/4043822018 wait complete. 2023-11-06T22:18:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 91 2023-11-06T22:18:44.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:18:44.723 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:44.721+0000 7f42cd07f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T22:18:44.728 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:44.726+0000 7f2e7097e680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:18:44.817 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:44.815+0000 7f2e7097e680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:18:44.824 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:44.822+0000 7f42cd07f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T22:18:44.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.848+0000 7f458dfff700 1 Processor -- start 2023-11-06T22:18:44.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.848+0000 7f458dfff700 1 -- start start 2023-11-06T22:18:44.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.849+0000 7f458dfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f45881521a0 0x7f45880c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.849+0000 7f458dfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4588154e90 con 0x7f4588155320 2023-11-06T22:18:44.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.849+0000 7f458dfff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45880639d0 con 0x7f4588151dc0 2023-11-06T22:18:44.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.849+0000 7f458dfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4588063cb0 con 0x7f45881521a0 2023-11-06T22:18:44.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.850+0000 7f4587fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4588155320 0x7f45880c7850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47384/0 (socket says 172.21.15.33:47384) 2023-11-06T22:18:44.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.850+0000 7f4587fff700 1 -- 172.21.15.33:0/876427509 learned_addr learned my addr 172.21.15.33:0/876427509 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:44.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.850+0000 7f4586ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f45881521a0 0x7f45880c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.850+0000 7f4586ffd700 1 -- 172.21.15.33:0/876427509 >> v1:172.21.15.69:6789/0 conn(0x7f4588151dc0 legacy=0x7f4588063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.850+0000 7f4586ffd700 1 -- 172.21.15.33:0/876427509 >> v1:172.21.15.33:6789/0 conn(0x7f4588155320 legacy=0x7f45880c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.850+0000 7f45867fc700 1 -- 172.21.15.33:0/876427509 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275014020 0 0) 0x7f4588154e90 con 0x7f4588155320 2023-11-06T22:18:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.850+0000 7f4586ffd700 1 -- 172.21.15.33:0/876427509 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4588063f90 con 0x7f45881521a0 2023-11-06T22:18:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.851+0000 7f4586ffd700 1 --2- 172.21.15.33:0/876427509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45881521a0 0x7f45880c0c50 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f457c009760 tx=0x7f457c009060).ready entity=mon.0 client_cookie=4af479adc4f04822 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.851+0000 7f45867fc700 1 -- 172.21.15.33:0/876427509 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c00b580 con 0x7f45881521a0 2023-11-06T22:18:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.851+0000 7f45867fc700 1 -- 172.21.15.33:0/876427509 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c00b6e0 con 0x7f45881521a0 2023-11-06T22:18:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.851+0000 7f45867fc700 1 -- 172.21.15.33:0/876427509 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c00ba60 con 0x7f45881521a0 2023-11-06T22:18:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.851+0000 7f458dfff700 1 -- 172.21.15.33:0/876427509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45881521a0 msgr2=0x7f45880c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.851+0000 7f458dfff700 1 --2- 172.21.15.33:0/876427509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45881521a0 0x7f45880c0c50 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f457c009760 tx=0x7f457c009060).stop 2023-11-06T22:18:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.852+0000 7f458dfff700 1 -- 172.21.15.33:0/876427509 shutdown_connections 2023-11-06T22:18:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.852+0000 7f458dfff700 1 --2- 172.21.15.33:0/876427509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45881521a0 0x7f45880c0c50 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.852+0000 7f458dfff700 1 -- 172.21.15.33:0/876427509 >> 172.21.15.33:0/876427509 conn(0x7f458805e1b0 msgr2=0x7f458805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.852+0000 7f458dfff700 1 -- 172.21.15.33:0/876427509 shutdown_connections 2023-11-06T22:18:44.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.852+0000 7f458dfff700 1 -- 172.21.15.33:0/876427509 wait complete. 2023-11-06T22:18:44.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.853+0000 7f458dfff700 1 Processor -- start 2023-11-06T22:18:44.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.853+0000 7f458dfff700 1 -- start start 2023-11-06T22:18:44.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.854+0000 7f458dfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4588151dc0 0x7f45880c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.854+0000 7f458dfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45881521a0 0x7f458814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.854+0000 7f458dfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588155320 0x7f4588156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.854+0000 7f458dfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45880ae880 con 0x7f4588155320 2023-11-06T22:18:44.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.854+0000 7f458dfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45880adf50 con 0x7f4588151dc0 2023-11-06T22:18:44.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.854+0000 7f458dfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45880ae230 con 0x7f45881521a0 2023-11-06T22:18:44.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f45877fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4588151dc0 0x7f45880c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:44.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f4586ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45881521a0 0x7f458814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:44.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f4587fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588155320 0x7f4588156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:44.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f45877fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4588151dc0 0x7f45880c1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:37684/0 (socket says 172.21.15.33:37684) 2023-11-06T22:18:44.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f4586ffd700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45881521a0 0x7f458814db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:44404/0 (socket says 172.21.15.33:44404) 2023-11-06T22:18:44.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f45877fe700 1 -- 172.21.15.33:0/3260068053 learned_addr learned my addr 172.21.15.33:0/3260068053 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:44.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f4587fff700 1 -- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45881521a0 msgr2=0x7f458814db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f4587fff700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45881521a0 0x7f458814db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f4587fff700 1 -- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4588151dc0 msgr2=0x7f45880c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:44.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f4587fff700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4588151dc0 0x7f45880c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f4587fff700 1 -- 172.21.15.33:0/3260068053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c0092a0 con 0x7f4588155320 2023-11-06T22:18:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.855+0000 7f4587fff700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588155320 0x7f4588156cb0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 rx=0x7f458000c7e0 tx=0x7f4580001420).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.856+0000 7f456f7fe700 1 -- 172.21.15.33:0/3260068053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580005d30 con 0x7f4588155320 2023-11-06T22:18:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.856+0000 7f456f7fe700 1 -- 172.21.15.33:0/3260068053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4580005e90 con 0x7f4588155320 2023-11-06T22:18:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.856+0000 7f456f7fe700 1 -- 172.21.15.33:0/3260068053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580007300 con 0x7f4588155320 2023-11-06T22:18:44.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.856+0000 7f458dfff700 1 -- 172.21.15.33:0/3260068053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45880ae510 con 0x7f4588155320 2023-11-06T22:18:44.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.856+0000 7f458dfff700 1 -- 172.21.15.33:0/3260068053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458814e040 con 0x7f4588155320 2023-11-06T22:18:44.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.857+0000 7f458dfff700 1 -- 172.21.15.33:0/3260068053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45880c0a30 con 0x7f4588155320 2023-11-06T22:18:44.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.858+0000 7f456f7fe700 1 -- 172.21.15.33:0/3260068053 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f45800076b0 con 0x7f4588155320 2023-11-06T22:18:44.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.858+0000 7f456f7fe700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f457405f2d0 0x7f4574061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:44.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.859+0000 7f456f7fe700 1 -- 172.21.15.33:0/3260068053 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f4580079ef0 con 0x7f4588155320 2023-11-06T22:18:44.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.859+0000 7f45877fe700 1 -- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f457405f2d0 msgr2=0x7f4574061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.859+0000 7f45877fe700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f457405f2d0 0x7f4574061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:44.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:44.866+0000 7f456f7fe700 1 -- 172.21.15.33:0/3260068053 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f458004ad90 con 0x7f4588155320 2023-11-06T22:18:44.960 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:44.958+0000 7f2e7097e680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:18:45.026 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:45.023+0000 7f42cd07f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T22:18:45.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.060+0000 7f45877fe700 1 -- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f457405f2d0 msgr2=0x7f4574061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.060+0000 7f45877fe700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f457405f2d0 0x7f4574061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-06T22:18:45.090 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.088+0000 7f2e7097e680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:45.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.093+0000 7f458dfff700 1 -- 172.21.15.33:0/3260068053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4588062ac0 con 0x7f4588155320 2023-11-06T22:18:45.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.093+0000 7f456f7fe700 1 -- 172.21.15.33:0/3260068053 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f458004e3b0 con 0x7f4588155320 2023-11-06T22:18:45.096 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:45.096 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":91,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3097639551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3097639551},{"type":"v1","addr":"172.21.15.69:6837","nonce":3097639551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9828,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1256245016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1256245016},{"type":"v1","addr":"172.21.15.33:6834","nonce":1256245016}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1680210655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1680210655},{"type":"v1","addr":"172.21.15.33:6835","nonce":1680210655}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T22:18:22.834249+0000","modified":"2023-11-06T22:18:25.840897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9837":{"gid":9837,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6836/3921249578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3921249578},{"type":"v1","addr":"172.21.15.69:6836","nonce":3921249578}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-06T22:18:45.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.095+0000 7f458dfff700 1 -- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f457405f2d0 msgr2=0x7f4574061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:45.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.095+0000 7f458dfff700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f457405f2d0 0x7f4574061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:45.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.095+0000 7f458dfff700 1 -- 172.21.15.33:0/3260068053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588155320 msgr2=0x7f4588156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.095+0000 7f458dfff700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588155320 0x7f4588156cb0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 rx=0x7f458000c7e0 tx=0x7f4580001420).stop 2023-11-06T22:18:45.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.096+0000 7f458dfff700 1 -- 172.21.15.33:0/3260068053 shutdown_connections 2023-11-06T22:18:45.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.096+0000 7f458dfff700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f457405f2d0 0x7f4574061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:45.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.096+0000 7f458dfff700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588155320 0x7f4588156cb0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.096+0000 7f458dfff700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4588151dc0 0x7f45880c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.096+0000 7f458dfff700 1 --2- 172.21.15.33:0/3260068053 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f45881521a0 0x7f458814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.096+0000 7f458dfff700 1 -- 172.21.15.33:0/3260068053 >> 172.21.15.33:0/3260068053 conn(0x7f458805e1b0 msgr2=0x7f45880ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:45.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.096+0000 7f458dfff700 1 -- 172.21.15.33:0/3260068053 shutdown_connections 2023-11-06T22:18:45.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.096+0000 7f458dfff700 1 -- 172.21.15.33:0/3260068053 wait complete. 2023-11-06T22:18:45.101 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 91 2023-11-06T22:18:45.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:45.122 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:45.119+0000 7f42cd07f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T22:18:45.277 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:45.275+0000 7f42cd07f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T22:18:45.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.314+0000 7f85c09ff700 1 Processor -- start 2023-11-06T22:18:45.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.314+0000 7f85c09ff700 1 -- start start 2023-11-06T22:18:45.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.315+0000 7f85c09ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f85bc0c1840 0x7f85bc0a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:45.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.315+0000 7f85c09ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85bc0aa2d0 0x7f85bc0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:45.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.315+0000 7f85c09ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc1554a0 con 0x7f85bc0c6240 2023-11-06T22:18:45.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.315+0000 7f85c09ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc154960 con 0x7f85bc0aa2d0 2023-11-06T22:18:45.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.315+0000 7f85c09ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc155930 con 0x7f85bc0c1840 2023-11-06T22:18:45.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.315+0000 7f85b9d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85bc0aa2d0 0x7f85bc0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:45.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.316+0000 7f85ba59c700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f85bc0c1840 0x7f85bc0a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:45.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.316+0000 7f85b9d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85bc0aa2d0 0x7f85bc0c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33954/0 (socket says 172.21.15.33:33954) 2023-11-06T22:18:45.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.316+0000 7f85b9d9b700 1 -- 172.21.15.33:0/388036111 learned_addr learned my addr 172.21.15.33:0/388036111 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:45.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.316+0000 7f85b9d9b700 1 -- 172.21.15.33:0/388036111 >> v2:172.21.15.69:3300/0 conn(0x7f85bc0c1840 msgr2=0x7f85bc0a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:45.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.316+0000 7f85b9d9b700 1 --2- 172.21.15.33:0/388036111 >> v2:172.21.15.69:3300/0 conn(0x7f85bc0c1840 0x7f85bc0a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.316+0000 7f85b9d9b700 1 -- 172.21.15.33:0/388036111 >> v1:172.21.15.69:6789/0 conn(0x7f85bc0c6240 legacy=0x7f85bc157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.316+0000 7f85b9d9b700 1 -- 172.21.15.33:0/388036111 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc154b70 con 0x7f85bc0aa2d0 2023-11-06T22:18:45.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.316+0000 7f85b9d9b700 1 --2- 172.21.15.33:0/388036111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0aa2d0 0x7f85bc0c5d50 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7f85bc0840b0 tx=0x7f85b00090f0).ready entity=mon.0 client_cookie=642271b81227b625 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:45.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.316+0000 7f85b959a700 1 -- 172.21.15.33:0/388036111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b000b520 con 0x7f85bc0aa2d0 2023-11-06T22:18:45.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.316+0000 7f85b959a700 1 -- 172.21.15.33:0/388036111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b000b680 con 0x7f85bc0aa2d0 2023-11-06T22:18:45.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.317+0000 7f85b959a700 1 -- 172.21.15.33:0/388036111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b000ba00 con 0x7f85bc0aa2d0 2023-11-06T22:18:45.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.317+0000 7f85c09ff700 1 -- 172.21.15.33:0/388036111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0aa2d0 msgr2=0x7f85bc0c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.317+0000 7f85c09ff700 1 --2- 172.21.15.33:0/388036111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0aa2d0 0x7f85bc0c5d50 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7f85bc0840b0 tx=0x7f85b00090f0).stop 2023-11-06T22:18:45.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.317+0000 7f85c09ff700 1 -- 172.21.15.33:0/388036111 shutdown_connections 2023-11-06T22:18:45.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.317+0000 7f85c09ff700 1 --2- 172.21.15.33:0/388036111 >> v2:172.21.15.69:3300/0 conn(0x7f85bc0c1840 0x7f85bc0a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.317+0000 7f85c09ff700 1 --2- 172.21.15.33:0/388036111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0aa2d0 0x7f85bc0c5d50 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.317+0000 7f85c09ff700 1 -- 172.21.15.33:0/388036111 >> 172.21.15.33:0/388036111 conn(0x7f85bc05ec80 msgr2=0x7f85bc0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:45.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.317+0000 7f85c09ff700 1 -- 172.21.15.33:0/388036111 shutdown_connections 2023-11-06T22:18:45.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.317+0000 7f85c09ff700 1 -- 172.21.15.33:0/388036111 wait complete. 2023-11-06T22:18:45.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.319+0000 7f85c09ff700 1 Processor -- start 2023-11-06T22:18:45.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.319+0000 7f85c09ff700 1 -- start start 2023-11-06T22:18:45.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85c09ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85bc0aa2d0 0x7f85bc158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:45.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85c09ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85bc0c1840 0x7f85bc144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:45.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85c09ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0c6240 0x7f85bc0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:45.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85c09ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc157020 con 0x7f85bc0c6240 2023-11-06T22:18:45.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85c09ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc157300 con 0x7f85bc0aa2d0 2023-11-06T22:18:45.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85c09ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc1575e0 con 0x7f85bc0c1840 2023-11-06T22:18:45.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85bad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0c6240 0x7f85bc0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85bad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0c6240 0x7f85bc0c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33968/0 (socket says 172.21.15.33:33968) 2023-11-06T22:18:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85ba59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85bc0aa2d0 0x7f85bc158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85bad9d700 1 -- 172.21.15.33:0/3038454274 learned_addr learned my addr 172.21.15.33:0/3038454274 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.320+0000 7f85b9d9b700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85bc0c1840 0x7f85bc144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:45.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85bad9d700 1 -- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85bc0c1840 msgr2=0x7f85bc144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85bad9d700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85bc0c1840 0x7f85bc144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85bad9d700 1 -- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85bc0aa2d0 msgr2=0x7f85bc158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85bad9d700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85bc0aa2d0 0x7f85bc158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85bad9d700 1 -- 172.21.15.33:0/3038454274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0009370 con 0x7f85bc0c6240 2023-11-06T22:18:45.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85bad9d700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0c6240 0x7f85bc0c9da0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7f85b4002230 tx=0x7f85b40015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:45.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85aa7fc700 1 -- 172.21.15.33:0/3038454274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400e4a0 con 0x7f85bc0c6240 2023-11-06T22:18:45.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85aa7fc700 1 -- 172.21.15.33:0/3038454274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b400e600 con 0x7f85bc0c6240 2023-11-06T22:18:45.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85aa7fc700 1 -- 172.21.15.33:0/3038454274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400e980 con 0x7f85bc0c6240 2023-11-06T22:18:45.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85c09ff700 1 -- 172.21.15.33:0/3038454274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc1578c0 con 0x7f85bc0c6240 2023-11-06T22:18:45.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.321+0000 7f85c09ff700 1 -- 172.21.15.33:0/3038454274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc144fa0 con 0x7f85bc0c6240 2023-11-06T22:18:45.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.322+0000 7f85c09ff700 1 -- 172.21.15.33:0/3038454274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc1451f0 con 0x7f85bc0c6240 2023-11-06T22:18:45.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.323+0000 7f85aa7fc700 1 -- 172.21.15.33:0/3038454274 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f85b401b3b0 con 0x7f85bc0c6240 2023-11-06T22:18:45.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.324+0000 7f85aa7fc700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f859c05f300 0x7f859c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:45.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.324+0000 7f85aa7fc700 1 -- 172.21.15.33:0/3038454274 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f85b4079cc0 con 0x7f85bc0c6240 2023-11-06T22:18:45.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.324+0000 7f85ba59c700 1 -- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f859c05f300 msgr2=0x7f859c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6832/129946 2023-11-06T22:18:45.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.324+0000 7f85ba59c700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f859c05f300 0x7f859c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-06T22:18:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.332+0000 7f85aa7fc700 1 -- 172.21.15.33:0/3038454274 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f85b4048af0 con 0x7f85bc0c6240 2023-11-06T22:18:45.426 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:45.424+0000 7f42cd07f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T22:18:45.437 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.435+0000 7f2e7097e680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:18:45.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.514+0000 7f85c09ff700 1 -- 172.21.15.33:0/3038454274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85bc06c6c0 con 0x7f85bc0c6240 2023-11-06T22:18:45.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.516+0000 7f85aa7fc700 1 -- 172.21.15.33:0/3038454274 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v224) v1 ==== 74+0+17429 (secure 0 0 0) 0x7f85b4080080 con 0x7f85bc0c6240 2023-11-06T22:18:45.519 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:45.519 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":224,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:37.037489+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T22:18:21.317255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T22:18:21.822232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/672707063":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/2065981382":"2023-11-07T22:18:37.037458+0000","172.21.15.69:6833/129946":"2023-11-07T22:18:37.037458+0000","172.21.15.69:6832/129946":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/3481533320":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/2510047576":"2023-11-07T22:18:37.037458+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:45.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 -- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f859c05f300 msgr2=0x7f859c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T22:18:45.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f859c05f300 0x7f859c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:45.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 -- 172.21.15.33:0/3038454274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0c6240 msgr2=0x7f85bc0c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0c6240 0x7f85bc0c9da0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7f85b4002230 tx=0x7f85b40015e0).stop 2023-11-06T22:18:45.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 -- 172.21.15.33:0/3038454274 shutdown_connections 2023-11-06T22:18:45.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f859c05f300 0x7f859c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:45.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0c6240 0x7f85bc0c9da0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85bc0aa2d0 0x7f85bc158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 --2- 172.21.15.33:0/3038454274 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f85bc0c1840 0x7f85bc144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 -- 172.21.15.33:0/3038454274 >> 172.21.15.33:0/3038454274 conn(0x7f85bc05ec80 msgr2=0x7f85bc150dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:45.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 -- 172.21.15.33:0/3038454274 shutdown_connections 2023-11-06T22:18:45.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.518+0000 7f85c09ff700 1 -- 172.21.15.33:0/3038454274 wait complete. 2023-11-06T22:18:45.526 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.524+0000 7f2e7097e680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:18:45.540 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T22:18:45.605 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.602+0000 7f2de8c8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:45.605 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.602+0000 7f2de8c8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:45.605 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.602+0000 7f2de8c8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:45.605 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.602+0000 7f2de8c8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:45.606 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.602+0000 7f2de8c8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:45.607 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.603+0000 7f2debc92700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:45.607 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.603+0000 7f2debc92700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:45.607 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.603+0000 7f2debc92700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:45.608 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.603+0000 7f2debc92700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:45.608 INFO:tasks.ceph.mgr.x.smithi069.stderr:2023-11-06T22:18:45.603+0000 7f2debc92700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-06T22:18:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.751+0000 7fd16e9ff700 1 Processor -- start 2023-11-06T22:18:45.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.751+0000 7fd16e9ff700 1 -- start start 2023-11-06T22:18:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.753+0000 7fd16e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd16814ff50 0x7fd168156770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.753+0000 7fd16e9ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd168155d20 con 0x7fd16814fb70 2023-11-06T22:18:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.753+0000 7fd16e9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1681562e0 con 0x7fd1680c3410 2023-11-06T22:18:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.753+0000 7fd16e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd168156c60 con 0x7fd16814ff50 2023-11-06T22:18:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.753+0000 7fd167fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd16814fb70 0x7fd168152750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47386/0 (socket says 172.21.15.33:47386) 2023-11-06T22:18:45.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.753+0000 7fd167fff700 1 -- 172.21.15.33:0/1180920479 learned_addr learned my addr 172.21.15.33:0/1180920479 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.753+0000 7fd1677fe700 1 --2- 172.21.15.33:0/1180920479 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd16814ff50 0x7fd168156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.753+0000 7fd166ffd700 1 -- 172.21.15.33:0/1180920479 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870445687 0 0) 0x7fd168155d20 con 0x7fd16814fb70 2023-11-06T22:18:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.753+0000 7fd166ffd700 1 -- 172.21.15.33:0/1180920479 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1500033f0 con 0x7fd16814fb70 2023-11-06T22:18:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.754+0000 7fd166ffd700 1 -- 172.21.15.33:0/1180920479 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939252400 0 0) 0x7fd1681562e0 con 0x7fd1680c3410 2023-11-06T22:18:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.754+0000 7fd166ffd700 1 -- 172.21.15.33:0/1180920479 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd168155d20 con 0x7fd1680c3410 2023-11-06T22:18:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.754+0000 7fd1677fe700 1 -- 172.21.15.33:0/1180920479 >> v1:172.21.15.69:6789/0 conn(0x7fd1680c3410 legacy=0x7fd1680bbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.754+0000 7fd1677fe700 1 -- 172.21.15.33:0/1180920479 >> v1:172.21.15.33:6789/0 conn(0x7fd16814fb70 legacy=0x7fd168152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.754+0000 7fd1677fe700 1 -- 172.21.15.33:0/1180920479 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168153280 con 0x7fd16814ff50 2023-11-06T22:18:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.754+0000 7fd1677fe700 1 --2- 172.21.15.33:0/1180920479 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd16814ff50 0x7fd168156770 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fd15c009790 tx=0x7fd15c009060).ready entity=mon.2 client_cookie=ba7f22bd7789cfc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.754+0000 7fd166ffd700 1 -- 172.21.15.33:0/1180920479 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00f510 con 0x7fd16814ff50 2023-11-06T22:18:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.754+0000 7fd166ffd700 1 -- 172.21.15.33:0/1180920479 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c00f670 con 0x7fd16814ff50 2023-11-06T22:18:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.754+0000 7fd166ffd700 1 -- 172.21.15.33:0/1180920479 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00f9f0 con 0x7fd16814ff50 2023-11-06T22:18:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.755+0000 7fd16e9ff700 1 -- 172.21.15.33:0/1180920479 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd16814ff50 msgr2=0x7fd168156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.755+0000 7fd16e9ff700 1 --2- 172.21.15.33:0/1180920479 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd16814ff50 0x7fd168156770 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fd15c009790 tx=0x7fd15c009060).stop 2023-11-06T22:18:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.755+0000 7fd16e9ff700 1 -- 172.21.15.33:0/1180920479 shutdown_connections 2023-11-06T22:18:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.755+0000 7fd16e9ff700 1 --2- 172.21.15.33:0/1180920479 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd16814ff50 0x7fd168156770 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.755+0000 7fd16e9ff700 1 -- 172.21.15.33:0/1180920479 >> 172.21.15.33:0/1180920479 conn(0x7fd168062710 msgr2=0x7fd168062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.755+0000 7fd16e9ff700 1 -- 172.21.15.33:0/1180920479 shutdown_connections 2023-11-06T22:18:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.755+0000 7fd16e9ff700 1 -- 172.21.15.33:0/1180920479 wait complete. 2023-11-06T22:18:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.757+0000 7fd16e9ff700 1 Processor -- start 2023-11-06T22:18:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.757+0000 7fd16e9ff700 1 -- start start 2023-11-06T22:18:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.758+0000 7fd16e9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1680c3410 0x7fd168157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.758+0000 7fd16e9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16814fb70 0x7fd16814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.758+0000 7fd16e9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd16814ff50 0x7fd1680c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.758+0000 7fd16e9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680a9a10 con 0x7fd16814fb70 2023-11-06T22:18:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.758+0000 7fd16e9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd168157b00 con 0x7fd16814ff50 2023-11-06T22:18:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.758+0000 7fd16e9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680bfec0 con 0x7fd1680c3410 2023-11-06T22:18:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd1677fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16814fb70 0x7fd16814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd1677fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16814fb70 0x7fd16814ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33976/0 (socket says 172.21.15.33:33976) 2023-11-06T22:18:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd1677fe700 1 -- 172.21.15.33:0/3018031558 learned_addr learned my addr 172.21.15.33:0/3018031558 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd167fff700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1680c3410 0x7fd168157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd16cf9c700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd16814ff50 0x7fd1680c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd1677fe700 1 -- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1680c3410 msgr2=0x7fd168157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd1677fe700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1680c3410 0x7fd168157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd1677fe700 1 -- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd16814ff50 msgr2=0x7fd1680c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd1677fe700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd16814ff50 0x7fd1680c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd1677fe700 1 -- 172.21.15.33:0/3018031558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c0092a0 con 0x7fd16814fb70 2023-11-06T22:18:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd16cf9c700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd16814ff50 0x7fd1680c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.759+0000 7fd1677fe700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16814fb70 0x7fd16814ca20 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 rx=0x7fd15c011000 tx=0x7fd15c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.760+0000 7fd14ffff700 1 -- 172.21.15.33:0/3018031558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00f7e0 con 0x7fd16814fb70 2023-11-06T22:18:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.760+0000 7fd14ffff700 1 -- 172.21.15.33:0/3018031558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c00f940 con 0x7fd16814fb70 2023-11-06T22:18:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.760+0000 7fd14ffff700 1 -- 172.21.15.33:0/3018031558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c01b660 con 0x7fd16814fb70 2023-11-06T22:18:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.760+0000 7fd16e9ff700 1 -- 172.21.15.33:0/3018031558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1680c01a0 con 0x7fd16814fb70 2023-11-06T22:18:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.760+0000 7fd16e9ff700 1 -- 172.21.15.33:0/3018031558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1680c0760 con 0x7fd16814fb70 2023-11-06T22:18:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.761+0000 7fd16e9ff700 1 -- 172.21.15.33:0/3018031558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd168153560 con 0x7fd16814fb70 2023-11-06T22:18:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.762+0000 7fd14ffff700 1 -- 172.21.15.33:0/3018031558 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 103) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fd15c01b9d0 con 0x7fd16814fb70 2023-11-06T22:18:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.763+0000 7fd14ffff700 1 -- 172.21.15.33:0/3018031558 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6638+0+0 (secure 0 0 0) 0x7fd15c07b5c0 con 0x7fd16814fb70 2023-11-06T22:18:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.767+0000 7fd14ffff700 1 -- 172.21.15.33:0/3018031558 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd15c04c750 con 0x7fd16814fb70 2023-11-06T22:18:45.808 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:45.806+0000 7f42cd07f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T22:18:45.904 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-11-06T22:18:45.902+0000 7f42cd07f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T22:18:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:45.997+0000 7fd16e9ff700 1 -- 172.21.15.33:0/3018031558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd168155f60 con 0x7fd16814fb70 2023-11-06T22:18:46.534 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-11-06T22:18:46.531+0000 7f73c95f7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T22:18:46.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.537+0000 7fd14ffff700 1 -- 172.21.15.33:0/3018031558 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 104) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fd15c079250 con 0x7fd16814fb70 2023-11-06T22:18:46.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.537+0000 7fd14ffff700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1540302f0 0x7fd154067b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:46.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.538+0000 7fd167fff700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1540302f0 0x7fd154067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:46.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.542+0000 7fd167fff700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1540302f0 0x7fd154067b30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd158005750 tx=0x7fd158009040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:46.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.548+0000 7fd14ffff700 1 -- 172.21.15.33:0/3018031558 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd15c03ea00 con 0x7fd16814fb70 2023-11-06T22:18:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 -- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1540302f0 msgr2=0x7fd154067b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1540302f0 0x7fd154067b30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd158005750 tx=0x7fd158009040).stop 2023-11-06T22:18:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 -- 172.21.15.33:0/3018031558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16814fb70 msgr2=0x7fd16814ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:46.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16814fb70 0x7fd16814ca20 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 rx=0x7fd15c011000 tx=0x7fd15c017000).stop 2023-11-06T22:18:46.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 -- 172.21.15.33:0/3018031558 shutdown_connections 2023-11-06T22:18:46.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fd1540302f0 0x7fd154067b30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fd1680c3410 0x7fd168157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd16814ff50 0x7fd1680c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 --2- 172.21.15.33:0/3018031558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16814fb70 0x7fd16814ca20 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 -- 172.21.15.33:0/3018031558 >> 172.21.15.33:0/3018031558 conn(0x7fd168062710 msgr2=0x7fd16814eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:46.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 -- 172.21.15.33:0/3018031558 shutdown_connections 2023-11-06T22:18:46.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.550+0000 7fd16e9ff700 1 -- 172.21.15.33:0/3018031558 wait complete. 2023-11-06T22:18:46.556 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T22:18:46.558 INFO:tasks.ceph.mds.d.smithi069.stderr: -2947> 2023-11-06T22:18:23.331+0000 7f5b4e7f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T22:18:46.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T22:18:46.601 INFO:tasks.ceph.mds.d.smithi069.stdout:starting mds.d at 2023-11-06T22:18:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.769+0000 7ff8395ff700 1 Processor -- start 2023-11-06T22:18:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.769+0000 7ff8395ff700 1 -- start start 2023-11-06T22:18:46.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff8395ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff834157700 0x7ff8340c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff8395ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8340aaf50 0x7ff834063130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff8395ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff8340aa4f0 0x7ff834063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff8395ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8340c18f0 con 0x7ff8340aaf50 2023-11-06T22:18:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff8395ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff834063b10 con 0x7ff8340aa4f0 2023-11-06T22:18:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff8395ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff834158c00 con 0x7ff834157700 2023-11-06T22:18:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff8327fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8340aaf50 0x7ff834063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff8327fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8340aaf50 0x7ff834063130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33986/0 (socket says 172.21.15.33:33986) 2023-11-06T22:18:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff8327fc700 1 -- 172.21.15.33:0/1921782632 learned_addr learned my addr 172.21.15.33:0/1921782632 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff832ffd700 1 --2- 172.21.15.33:0/1921782632 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff834157700 0x7ff8340c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.771+0000 7ff8337fe700 1 --2- 172.21.15.33:0/1921782632 >> v2:172.21.15.69:3300/0 conn(0x7ff8340aa4f0 0x7ff834063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.772+0000 7ff8327fc700 1 -- 172.21.15.33:0/1921782632 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff834157700 msgr2=0x7ff8340c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.772+0000 7ff8327fc700 1 --2- 172.21.15.33:0/1921782632 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff834157700 0x7ff8340c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.772+0000 7ff8327fc700 1 -- 172.21.15.33:0/1921782632 >> v2:172.21.15.69:3300/0 conn(0x7ff8340aa4f0 msgr2=0x7ff834063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.772+0000 7ff8327fc700 1 --2- 172.21.15.33:0/1921782632 >> v2:172.21.15.69:3300/0 conn(0x7ff8340aa4f0 0x7ff834063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.772+0000 7ff8327fc700 1 -- 172.21.15.33:0/1921782632 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff834157ff0 con 0x7ff8340aaf50 2023-11-06T22:18:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.772+0000 7ff8337fe700 1 --2- 172.21.15.33:0/1921782632 >> v2:172.21.15.69:3300/0 conn(0x7ff8340aa4f0 0x7ff834063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.772+0000 7ff8327fc700 1 --2- 172.21.15.33:0/1921782632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aaf50 0x7ff834063130 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 rx=0x7ff8340c53d0 tx=0x7ff8280090f0).ready entity=mon.0 client_cookie=d92f2782e8dce422 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.772+0000 7ff831ffb700 1 -- 172.21.15.33:0/1921782632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800b520 con 0x7ff8340aaf50 2023-11-06T22:18:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.772+0000 7ff831ffb700 1 -- 172.21.15.33:0/1921782632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82800b680 con 0x7ff8340aaf50 2023-11-06T22:18:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.772+0000 7ff831ffb700 1 -- 172.21.15.33:0/1921782632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800ba00 con 0x7ff8340aaf50 2023-11-06T22:18:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.773+0000 7ff8395ff700 1 -- 172.21.15.33:0/1921782632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aaf50 msgr2=0x7ff834063130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.773+0000 7ff8395ff700 1 --2- 172.21.15.33:0/1921782632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aaf50 0x7ff834063130 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 rx=0x7ff8340c53d0 tx=0x7ff8280090f0).stop 2023-11-06T22:18:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.773+0000 7ff8395ff700 1 -- 172.21.15.33:0/1921782632 shutdown_connections 2023-11-06T22:18:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.773+0000 7ff8395ff700 1 --2- 172.21.15.33:0/1921782632 >> v2:172.21.15.69:3300/0 conn(0x7ff8340aa4f0 0x7ff834063620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.773+0000 7ff8395ff700 1 --2- 172.21.15.33:0/1921782632 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff834157700 0x7ff8340c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.773+0000 7ff8395ff700 1 --2- 172.21.15.33:0/1921782632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aaf50 0x7ff834063130 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.773+0000 7ff8395ff700 1 -- 172.21.15.33:0/1921782632 >> 172.21.15.33:0/1921782632 conn(0x7ff83405ec80 msgr2=0x7ff8340cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.773+0000 7ff8395ff700 1 -- 172.21.15.33:0/1921782632 shutdown_connections 2023-11-06T22:18:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.773+0000 7ff8395ff700 1 -- 172.21.15.33:0/1921782632 wait complete. 2023-11-06T22:18:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.774+0000 7ff8395ff700 1 Processor -- start 2023-11-06T22:18:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.774+0000 7ff8395ff700 1 -- start start 2023-11-06T22:18:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.775+0000 7ff8395ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aa4f0 0x7ff8341563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.775+0000 7ff8395ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8340aaf50 0x7ff834150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.775+0000 7ff8395ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff834157700 0x7ff8340ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.775+0000 7ff8395ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340ade00 con 0x7ff8340aa4f0 2023-11-06T22:18:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.775+0000 7ff8395ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340ae3c0 con 0x7ff8340aaf50 2023-11-06T22:18:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.775+0000 7ff8395ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8341568d0 con 0x7ff834157700 2023-11-06T22:18:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff832ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aa4f0 0x7ff8341563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff832ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aa4f0 0x7ff8341563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33988/0 (socket says 172.21.15.33:33988) 2023-11-06T22:18:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff832ffd700 1 -- 172.21.15.33:0/1088987113 learned_addr learned my addr 172.21.15.33:0/1088987113 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:46.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff8327fc700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8340aaf50 0x7ff834150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:46.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff8337fe700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff834157700 0x7ff8340ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:46.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff832ffd700 1 -- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff834157700 msgr2=0x7ff8340ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:46.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff832ffd700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff834157700 0x7ff8340ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff832ffd700 1 -- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8340aaf50 msgr2=0x7ff834150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:46.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff832ffd700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8340aaf50 0x7ff834150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:46.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff832ffd700 1 -- 172.21.15.33:0/1088987113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff828009370 con 0x7ff8340aa4f0 2023-11-06T22:18:46.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff832ffd700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aa4f0 0x7ff8341563e0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7ff82400e930 tx=0x7ff824009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:46.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff81affd700 1 -- 172.21.15.33:0/1088987113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82400c670 con 0x7ff8340aa4f0 2023-11-06T22:18:46.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff81affd700 1 -- 172.21.15.33:0/1088987113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82400c7d0 con 0x7ff8340aa4f0 2023-11-06T22:18:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff8395ff700 1 -- 172.21.15.33:0/1088987113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff834153f30 con 0x7ff8340aa4f0 2023-11-06T22:18:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.776+0000 7ff81affd700 1 -- 172.21.15.33:0/1088987113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82400cb50 con 0x7ff8340aa4f0 2023-11-06T22:18:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.777+0000 7ff8395ff700 1 -- 172.21.15.33:0/1088987113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8340ae050 con 0x7ff8340aa4f0 2023-11-06T22:18:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.778+0000 7ff8395ff700 1 -- 172.21.15.33:0/1088987113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8340ae6a0 con 0x7ff8340aa4f0 2023-11-06T22:18:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.778+0000 7ff81affd700 1 -- 172.21.15.33:0/1088987113 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 104) v1 ==== 75340+0+0 (secure 0 0 0) 0x7ff82401b400 con 0x7ff8340aa4f0 2023-11-06T22:18:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.779+0000 7ff81affd700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff82005f030 0x7ff8200614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.779+0000 7ff81affd700 1 -- 172.21.15.33:0/1088987113 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6724+0+0 (secure 0 0 0) 0x7ff82407a6d0 con 0x7ff8340aa4f0 2023-11-06T22:18:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.780+0000 7ff8327fc700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff82005f030 0x7ff8200614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.780+0000 7ff8327fc700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff82005f030 0x7ff8200614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff834084560 tx=0x7ff828015000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:46.783+0000 7ff81affd700 1 -- 172.21.15.33:0/1088987113 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff82404b960 con 0x7ff8340aa4f0 2023-11-06T22:18:47.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.014+0000 7ff8395ff700 1 -- 172.21.15.33:0/1088987113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff83406bce0 con 0x7ff8340aa4f0 2023-11-06T22:18:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.554+0000 7ff81affd700 1 -- 172.21.15.33:0/1088987113 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff83406bce0 con 0x7ff8340aa4f0 2023-11-06T22:18:47.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 -- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff82005f030 msgr2=0x7ff8200614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:47.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff82005f030 0x7ff8200614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff834084560 tx=0x7ff828015000).stop 2023-11-06T22:18:47.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 -- 172.21.15.33:0/1088987113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aa4f0 msgr2=0x7ff8341563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:47.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aa4f0 0x7ff8341563e0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7ff82400e930 tx=0x7ff824009060).stop 2023-11-06T22:18:47.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 -- 172.21.15.33:0/1088987113 shutdown_connections 2023-11-06T22:18:47.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff82005f030 0x7ff8200614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:47.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff834157700 0x7ff8340ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:47.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8340aa4f0 0x7ff8341563e0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:47.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 --2- 172.21.15.33:0/1088987113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8340aaf50 0x7ff834150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:47.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 -- 172.21.15.33:0/1088987113 >> 172.21.15.33:0/1088987113 conn(0x7ff83405ec80 msgr2=0x7ff8340c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:47.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 -- 172.21.15.33:0/1088987113 shutdown_connections 2023-11-06T22:18:47.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.556+0000 7ff8395ff700 1 -- 172.21.15.33:0/1088987113 wait complete. 2023-11-06T22:18:47.579 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T22:18:47.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.780+0000 7f7c097ff700 1 Processor -- start 2023-11-06T22:18:47.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.780+0000 7f7c097ff700 1 -- start start 2023-11-06T22:18:47.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.782+0000 7f7c097ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7c0414e630 0x7f7c0414ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:47.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.782+0000 7f7c097ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c040666b0 0x7f7c04066ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:47.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.782+0000 7f7c097ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c04155a20 con 0x7f7c04067c20 2023-11-06T22:18:47.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.782+0000 7f7c097ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c04066fd0 con 0x7f7c0414e630 2023-11-06T22:18:47.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.782+0000 7f7c097ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04068170 con 0x7f7c040666b0 2023-11-06T22:18:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.783+0000 7f7c02ffd700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7c0414e630 0x7f7c0414ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.783+0000 7f7c027fc700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f7c04067c20 0x7f7c04155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:60438/0 (socket says 172.21.15.33:60438) 2023-11-06T22:18:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.783+0000 7f7c037fe700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c040666b0 0x7f7c04066ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.783+0000 7f7c027fc700 1 -- 172.21.15.33:0/3712388289 learned_addr learned my addr 172.21.15.33:0/3712388289 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:47.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.783+0000 7f7c02ffd700 1 -- 172.21.15.33:0/3712388289 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c040666b0 msgr2=0x7f7c04066ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:47.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.783+0000 7f7c02ffd700 1 --2- 172.21.15.33:0/3712388289 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c040666b0 0x7f7c04066ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:47.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.783+0000 7f7c02ffd700 1 -- 172.21.15.33:0/3712388289 >> v1:172.21.15.69:6789/0 conn(0x7f7c04067c20 legacy=0x7f7c04155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:47.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.783+0000 7f7c02ffd700 1 -- 172.21.15.33:0/3712388289 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c04068450 con 0x7f7c0414e630 2023-11-06T22:18:47.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.783+0000 7f7c01ffb700 1 -- 172.21.15.33:0/3712388289 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612763281 0 0) 0x7f7c04155a20 con 0x7f7c04067c20 2023-11-06T22:18:47.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.783+0000 7f7c02ffd700 1 --2- 172.21.15.33:0/3712388289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0414e630 0x7f7c0414ea10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f7bf4009760 tx=0x7f7bf4009060).ready entity=mon.1 client_cookie=238b2c1a7dccf6eb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:47.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c01ffb700 1 -- 172.21.15.33:0/3712388289 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400f510 con 0x7f7c0414e630 2023-11-06T22:18:47.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c01ffb700 1 -- 172.21.15.33:0/3712388289 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf400f670 con 0x7f7c0414e630 2023-11-06T22:18:47.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c01ffb700 1 -- 172.21.15.33:0/3712388289 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400f9f0 con 0x7f7c0414e630 2023-11-06T22:18:47.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c097ff700 1 -- 172.21.15.33:0/3712388289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0414e630 msgr2=0x7f7c0414ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:47.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c097ff700 1 --2- 172.21.15.33:0/3712388289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0414e630 0x7f7c0414ea10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f7bf4009760 tx=0x7f7bf4009060).stop 2023-11-06T22:18:47.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c097ff700 1 -- 172.21.15.33:0/3712388289 shutdown_connections 2023-11-06T22:18:47.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c097ff700 1 --2- 172.21.15.33:0/3712388289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0414e630 0x7f7c0414ea10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:47.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c097ff700 1 --2- 172.21.15.33:0/3712388289 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c040666b0 0x7f7c04066ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c097ff700 1 -- 172.21.15.33:0/3712388289 >> 172.21.15.33:0/3712388289 conn(0x7f7c040c8e60 msgr2=0x7f7c040c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c097ff700 1 -- 172.21.15.33:0/3712388289 shutdown_connections 2023-11-06T22:18:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.784+0000 7f7c097ff700 1 -- 172.21.15.33:0/3712388289 wait complete. 2023-11-06T22:18:47.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.786+0000 7f7c097ff700 1 Processor -- start 2023-11-06T22:18:47.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.787+0000 7f7c097ff700 1 -- start start 2023-11-06T22:18:47.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c097ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c040666b0 0x7f7c041374b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:47.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c097ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c04067c20 0x7f7c04168280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:47.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c097ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c0414e630 0x7f7c04168770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:47.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c097ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04168c60 con 0x7f7c0414e630 2023-11-06T22:18:47.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c097ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04138ec0 con 0x7f7c04067c20 2023-11-06T22:18:47.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c097ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040c1540 con 0x7f7c040666b0 2023-11-06T22:18:47.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c0414e630 0x7f7c04168770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:47.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c0414e630 0x7f7c04168770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33990/0 (socket says 172.21.15.33:33990) 2023-11-06T22:18:47.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c037fe700 1 -- 172.21.15.33:0/3579554399 learned_addr learned my addr 172.21.15.33:0/3579554399 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:47.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c02ffd700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c040666b0 0x7f7c041374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:47.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.788+0000 7f7c027fc700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c04067c20 0x7f7c04168280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:47.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7c037fe700 1 -- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c040666b0 msgr2=0x7f7c041374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:47.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7c037fe700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c040666b0 0x7f7c041374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:47.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7c037fe700 1 -- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c04067c20 msgr2=0x7f7c04168280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:47.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7c037fe700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c04067c20 0x7f7c04168280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:47.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7c037fe700 1 -- 172.21.15.33:0/3579554399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf40092a0 con 0x7f7c0414e630 2023-11-06T22:18:47.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7c037fe700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c0414e630 0x7f7c04168770 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 rx=0x7f7bfc009e90 tx=0x7f7bfc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:47.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7beaffd700 1 -- 172.21.15.33:0/3579554399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00caa0 con 0x7f7c0414e630 2023-11-06T22:18:47.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7beaffd700 1 -- 172.21.15.33:0/3579554399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc00f040 con 0x7f7c0414e630 2023-11-06T22:18:47.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7beaffd700 1 -- 172.21.15.33:0/3579554399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00ce10 con 0x7f7c0414e630 2023-11-06T22:18:47.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7c097ff700 1 -- 172.21.15.33:0/3579554399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c040c1820 con 0x7f7c0414e630 2023-11-06T22:18:47.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.789+0000 7f7c097ff700 1 -- 172.21.15.33:0/3579554399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c040c1de0 con 0x7f7c0414e630 2023-11-06T22:18:47.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.790+0000 7f7c097ff700 1 -- 172.21.15.33:0/3579554399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c041390c0 con 0x7f7c0414e630 2023-11-06T22:18:47.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.792+0000 7f7beaffd700 1 -- 172.21.15.33:0/3579554399 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 104) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f7bfc013610 con 0x7f7c0414e630 2023-11-06T22:18:47.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.792+0000 7f7beaffd700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f030 0x7f7bf00614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:47.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.792+0000 7f7beaffd700 1 -- 172.21.15.33:0/3579554399 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f7bfc07ab50 con 0x7f7c0414e630 2023-11-06T22:18:47.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.793+0000 7f7c02ffd700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f030 0x7f7bf00614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:47.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.793+0000 7f7c02ffd700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f030 0x7f7bf00614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7bf4009fd0 tx=0x7f7bf4017000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:47.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.797+0000 7f7beaffd700 1 -- 172.21.15.33:0/3579554399 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7c041390c0 con 0x7f7c0414e630 2023-11-06T22:18:47.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:47.978+0000 7f7c097ff700 1 -- 172.21.15.33:0/3579554399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7c040657e0 con 0x7f7c0414e630 2023-11-06T22:18:48.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.323+0000 7f7beaffd700 1 -- 172.21.15.33:0/3579554399 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f7bfc078780 con 0x7f7c0414e630 2023-11-06T22:18:48.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.561+0000 7f7beaffd700 1 -- 172.21.15.33:0/3579554399 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v228) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7bfc072a70 con 0x7f7c0414e630 2023-11-06T22:18:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.565+0000 7f7c097ff700 1 -- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f030 msgr2=0x7f7bf00614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.565+0000 7f7c097ff700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f030 0x7f7bf00614f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7bf4009fd0 tx=0x7f7bf4017000).stop 2023-11-06T22:18:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.565+0000 7f7c097ff700 1 -- 172.21.15.33:0/3579554399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c0414e630 msgr2=0x7f7c04168770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.565+0000 7f7c097ff700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c0414e630 0x7f7c04168770 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 rx=0x7f7bfc009e90 tx=0x7f7bfc00b040).stop 2023-11-06T22:18:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.565+0000 7f7c097ff700 1 -- 172.21.15.33:0/3579554399 shutdown_connections 2023-11-06T22:18:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.565+0000 7f7c097ff700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f7bf005f030 0x7f7bf00614f0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0x7f7bf4009fd0 tx=0x7f7bf4017000).stop 2023-11-06T22:18:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.565+0000 7f7c097ff700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f7c040666b0 0x7f7c041374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.565+0000 7f7c097ff700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c0414e630 0x7f7c04168770 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.565+0000 7f7c097ff700 1 --2- 172.21.15.33:0/3579554399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c04067c20 0x7f7c04168280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.565+0000 7f7c097ff700 1 -- 172.21.15.33:0/3579554399 >> 172.21.15.33:0/3579554399 conn(0x7f7c040c8e60 msgr2=0x7f7c040c54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:48.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.567+0000 7f7c097ff700 1 -- 172.21.15.33:0/3579554399 shutdown_connections 2023-11-06T22:18:48.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.567+0000 7f7c097ff700 1 -- 172.21.15.33:0/3579554399 wait complete. 2023-11-06T22:18:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-11-06T22:18:48.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T22:18:48.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.788+0000 7f3414dff700 1 Processor -- start 2023-11-06T22:18:48.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.788+0000 7f3414dff700 1 -- start start 2023-11-06T22:18:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.790+0000 7f3414dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34100bc090 0x7f341015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.790+0000 7f3414dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3410154c60 con 0x7f34101514c0 2023-11-06T22:18:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.790+0000 7f3414dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3410068f50 con 0x7f34101510e0 2023-11-06T22:18:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.790+0000 7f3414dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3410151dc0 con 0x7f34100bc090 2023-11-06T22:18:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.790+0000 7f340dd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f34101514c0 0x7f34100bb980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33522/0 (socket says 172.21.15.33:33522) 2023-11-06T22:18:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.790+0000 7f340dd9b700 1 -- 172.21.15.33:0/1015107685 learned_addr learned my addr 172.21.15.33:0/1015107685 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.790+0000 7f340ed9d700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34100bc090 0x7f341015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.790+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484368642 0 0) 0x7f3410154c60 con 0x7f34101514c0 2023-11-06T22:18:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33f80033f0 con 0x7f34101514c0 2023-11-06T22:18:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195452690 0 0) 0x7f3410068f50 con 0x7f34101510e0 2023-11-06T22:18:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3410154c60 con 0x7f34101510e0 2023-11-06T22:18:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140847400 0 0) 0x7f33f80033f0 con 0x7f34101514c0 2023-11-06T22:18:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34100bc090 msgr2=0x7f341015bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:48.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34100bc090 0x7f341015bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:48.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 >> v1:172.21.15.69:6789/0 conn(0x7f34101510e0 legacy=0x7f34100a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:48.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34101520a0 con 0x7f34101514c0 2023-11-06T22:18:48.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f3404001470 con 0x7f34101514c0 2023-11-06T22:18:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 >> v1:172.21.15.33:6789/0 conn(0x7f34101514c0 legacy=0x7f34100bb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33f8003aa0 0x7f33f8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33f8004690 0x7f33f8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34100bc090 0x7f33f8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3410151dc0 con 0x7f33f8004690 2023-11-06T22:18:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f800db20 con 0x7f33f8003aa0 2023-11-06T22:18:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f800dce0 con 0x7f34100bc090 2023-11-06T22:18:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340dd9b700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33f8004690 0x7f33f8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340ed9d700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34100bc090 0x7f33f8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.791+0000 7f340e59c700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33f8003aa0 0x7f33f8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.792+0000 7f340dd9b700 1 -- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34100bc090 msgr2=0x7f33f8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.792+0000 7f340dd9b700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f34100bc090 0x7f33f8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.792+0000 7f340dd9b700 1 -- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33f8003aa0 msgr2=0x7f33f8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.792+0000 7f340dd9b700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33f8003aa0 0x7f33f8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.792+0000 7f340dd9b700 1 -- 172.21.15.33:0/1015107685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80033f0 con 0x7f33f8004690 2023-11-06T22:18:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.792+0000 7f340dd9b700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33f8004690 0x7f33f8004b10 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7f340400c110 tx=0x7f3404001e10).ready entity=mon.0 client_cookie=ac0b1b1eea1971a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.792+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400ca40 con 0x7f33f8004690 2023-11-06T22:18:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.792+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340400cba0 con 0x7f33f8004690 2023-11-06T22:18:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.792+0000 7f340d59a700 1 -- 172.21.15.33:0/1015107685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34040070d0 con 0x7f33f8004690 2023-11-06T22:18:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.793+0000 7f3414dff700 1 -- 172.21.15.33:0/1015107685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33f8004690 msgr2=0x7f33f8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:48.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.793+0000 7f3414dff700 1 --2- 172.21.15.33:0/1015107685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33f8004690 0x7f33f8004b10 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7f340400c110 tx=0x7f3404001e10).stop 2023-11-06T22:18:48.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.793+0000 7f340ed9d700 1 -- 172.21.15.33:0/1015107685 reap_dead start 2023-11-06T22:18:48.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.793+0000 7f3414dff700 1 -- 172.21.15.33:0/1015107685 shutdown_connections 2023-11-06T22:18:48.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.793+0000 7f3414dff700 1 -- 172.21.15.33:0/1015107685 >> 172.21.15.33:0/1015107685 conn(0x7f3410062710 msgr2=0x7f3410062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:48.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.793+0000 7f3414dff700 1 -- 172.21.15.33:0/1015107685 shutdown_connections 2023-11-06T22:18:48.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.793+0000 7f3414dff700 1 -- 172.21.15.33:0/1015107685 wait complete. 2023-11-06T22:18:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.794+0000 7f3414dff700 1 Processor -- start 2023-11-06T22:18:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.794+0000 7f3414dff700 1 -- start start 2023-11-06T22:18:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.795+0000 7f3414dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f33f8004690 0x7f34100aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.795+0000 7f3414dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34100a4d90 0x7f341014ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.795+0000 7f3414dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3410062b20 0x7f3410062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.795+0000 7f3414dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3410063490 con 0x7f3410062b20 2023-11-06T22:18:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.795+0000 7f3414dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34100aa9a0 con 0x7f34100a4d90 2023-11-06T22:18:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.795+0000 7f3414dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34100aac80 con 0x7f33f8004690 2023-11-06T22:18:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3410062b20 0x7f3410062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3410062b20 0x7f3410062fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46292/0 (socket says 172.21.15.33:46292) 2023-11-06T22:18:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340ed9d700 1 -- 172.21.15.33:0/3619884583 learned_addr learned my addr 172.21.15.33:0/3619884583 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340e59c700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f33f8004690 0x7f34100aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340dd9b700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34100a4d90 0x7f341014ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:48.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340ed9d700 1 -- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f33f8004690 msgr2=0x7f34100aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:48.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340ed9d700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f33f8004690 0x7f34100aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:48.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340ed9d700 1 -- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34100a4d90 msgr2=0x7f341014ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:48.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340ed9d700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34100a4d90 0x7f341014ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:48.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340ed9d700 1 -- 172.21.15.33:0/3619884583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34100aaf60 con 0x7f3410062b20 2023-11-06T22:18:48.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340dd9b700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34100a4d90 0x7f341014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:48.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f340ed9d700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3410062b20 0x7f3410062fa0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 rx=0x7f34080094d0 tx=0x7f340800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:48.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f33fe7fc700 1 -- 172.21.15.33:0/3619884583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340800e040 con 0x7f3410062b20 2023-11-06T22:18:48.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.796+0000 7f33fe7fc700 1 -- 172.21.15.33:0/3619884583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340800f040 con 0x7f3410062b20 2023-11-06T22:18:48.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.797+0000 7f33fe7fc700 1 -- 172.21.15.33:0/3619884583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340800fe20 con 0x7f3410062b20 2023-11-06T22:18:48.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.797+0000 7f3414dff700 1 -- 172.21.15.33:0/3619884583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34100ab240 con 0x7f3410062b20 2023-11-06T22:18:48.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.797+0000 7f3414dff700 1 -- 172.21.15.33:0/3619884583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341014f1c0 con 0x7f3410062b20 2023-11-06T22:18:48.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.798+0000 7f3414dff700 1 -- 172.21.15.33:0/3619884583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3410152380 con 0x7f3410062b20 2023-11-06T22:18:48.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.798+0000 7f33fe7fc700 1 -- 172.21.15.33:0/3619884583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3408015660 con 0x7f3410062b20 2023-11-06T22:18:48.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.799+0000 7f33fe7fc700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f33f005f270 0x7f33f0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:48.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.799+0000 7f33fe7fc700 1 -- 172.21.15.33:0/3619884583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f340807afe0 con 0x7f3410062b20 2023-11-06T22:18:48.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.800+0000 7f340e59c700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f33f005f270 0x7f33f0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:48.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.800+0000 7f340e59c700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f33f005f270 0x7f33f0061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f3400007fd0 tx=0x7f3400007bb0).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.804+0000 7f33fe7fc700 1 -- 172.21.15.33:0/3619884583 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f340804bff0 con 0x7f3410062b20 2023-11-06T22:18:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:48.983+0000 7f3414dff700 1 -- 172.21.15.33:0/3619884583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3410065570 con 0x7f3410062b20 2023-11-06T22:18:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.563+0000 7f33fe7fc700 1 -- 172.21.15.33:0/3619884583 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v229) v1 ==== 169+0+0 (secure 0 0 0) 0x7f340804f610 con 0x7f3410062b20 2023-11-06T22:18:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.566+0000 7f3414dff700 1 -- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f33f005f270 msgr2=0x7f33f0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.566+0000 7f3414dff700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f33f005f270 0x7f33f0061730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f3400007fd0 tx=0x7f3400007bb0).stop 2023-11-06T22:18:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.566+0000 7f3414dff700 1 -- 172.21.15.33:0/3619884583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3410062b20 msgr2=0x7f3410062fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.566+0000 7f3414dff700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3410062b20 0x7f3410062fa0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 rx=0x7f34080094d0 tx=0x7f340800b040).stop 2023-11-06T22:18:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.568+0000 7f3414dff700 1 -- 172.21.15.33:0/3619884583 shutdown_connections 2023-11-06T22:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.568+0000 7f3414dff700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f33f005f270 0x7f33f0061730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.568+0000 7f3414dff700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f33f8004690 0x7f34100aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.568+0000 7f3414dff700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3410062b20 0x7f3410062fa0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.568+0000 7f3414dff700 1 --2- 172.21.15.33:0/3619884583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34100a4d90 0x7f341014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.568+0000 7f3414dff700 1 -- 172.21.15.33:0/3619884583 >> 172.21.15.33:0/3619884583 conn(0x7f3410062710 msgr2=0x7f34100c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.568+0000 7f3414dff700 1 -- 172.21.15.33:0/3619884583 shutdown_connections 2023-11-06T22:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.568+0000 7f3414dff700 1 -- 172.21.15.33:0/3619884583 wait complete. 2023-11-06T22:18:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-11-06T22:18:49.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-06T22:18:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.793+0000 7ff4415ff700 1 Processor -- start 2023-11-06T22:18:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.793+0000 7ff4415ff700 1 -- start start 2023-11-06T22:18:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.794+0000 7ff4415ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 0x7ff43c062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.794+0000 7ff4415ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff43c1547d0 0x7ff43c15e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff4415ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c063c00 con 0x7ff43c155410 2023-11-06T22:18:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff4415ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c064090 con 0x7ff43c1547d0 2023-11-06T22:18:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff4415ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c062ff0 con 0x7ff43c155030 2023-11-06T22:18:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff43b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff43c1547d0 0x7ff43c15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff43b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff43c1547d0 0x7ff43c15e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46296/0 (socket says 172.21.15.33:46296) 2023-11-06T22:18:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff43b7fe700 1 -- 172.21.15.33:0/1870367183 learned_addr learned my addr 172.21.15.33:0/1870367183 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff43affd700 1 --2- 172.21.15.33:0/1870367183 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 0x7ff43c062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff43b7fe700 1 -- 172.21.15.33:0/1870367183 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 msgr2=0x7ff43c062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff43b7fe700 1 --2- 172.21.15.33:0/1870367183 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 0x7ff43c062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff43b7fe700 1 -- 172.21.15.33:0/1870367183 >> v1:172.21.15.69:6789/0 conn(0x7ff43c155410 legacy=0x7ff43c0644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff43b7fe700 1 -- 172.21.15.33:0/1870367183 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0632d0 con 0x7ff43c1547d0 2023-11-06T22:18:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.795+0000 7ff43affd700 1 --2- 172.21.15.33:0/1870367183 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 0x7ff43c062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.796+0000 7ff43b7fe700 1 --2- 172.21.15.33:0/1870367183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c1547d0 0x7ff43c15e060 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 rx=0x7ff434009f20 tx=0x7ff434009840).ready entity=mon.0 client_cookie=c0db0662539754a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.796+0000 7ff439ffb700 1 -- 172.21.15.33:0/1870367183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43400e070 con 0x7ff43c1547d0 2023-11-06T22:18:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.796+0000 7ff439ffb700 1 -- 172.21.15.33:0/1870367183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43400ba80 con 0x7ff43c1547d0 2023-11-06T22:18:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.796+0000 7ff439ffb700 1 -- 172.21.15.33:0/1870367183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43400be00 con 0x7ff43c1547d0 2023-11-06T22:18:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.796+0000 7ff4415ff700 1 -- 172.21.15.33:0/1870367183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c1547d0 msgr2=0x7ff43c15e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.796+0000 7ff4415ff700 1 --2- 172.21.15.33:0/1870367183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c1547d0 0x7ff43c15e060 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 rx=0x7ff434009f20 tx=0x7ff434009840).stop 2023-11-06T22:18:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.796+0000 7ff4415ff700 1 -- 172.21.15.33:0/1870367183 shutdown_connections 2023-11-06T22:18:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.796+0000 7ff4415ff700 1 --2- 172.21.15.33:0/1870367183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c1547d0 0x7ff43c15e060 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.796+0000 7ff4415ff700 1 --2- 172.21.15.33:0/1870367183 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 0x7ff43c062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.796+0000 7ff4415ff700 1 -- 172.21.15.33:0/1870367183 >> 172.21.15.33:0/1870367183 conn(0x7ff43c0c3d30 msgr2=0x7ff43c0c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.797+0000 7ff4415ff700 1 -- 172.21.15.33:0/1870367183 shutdown_connections 2023-11-06T22:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.797+0000 7ff4415ff700 1 -- 172.21.15.33:0/1870367183 wait complete. 2023-11-06T22:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.798+0000 7ff4415ff700 1 Processor -- start 2023-11-06T22:18:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.798+0000 7ff4415ff700 1 -- start start 2023-11-06T22:18:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff4415ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff43c1547d0 0x7ff43c0accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff4415ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 0x7ff43c13d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff4415ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c155410 0x7ff43c0c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff4415ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0c0760 con 0x7ff43c155410 2023-11-06T22:18:49.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff4415ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0c0bf0 con 0x7ff43c1547d0 2023-11-06T22:18:49.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff4415ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0bfb50 con 0x7ff43c155030 2023-11-06T22:18:49.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff43b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c155410 0x7ff43c0c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:49.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff43b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c155410 0x7ff43c0c23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46310/0 (socket says 172.21.15.33:46310) 2023-11-06T22:18:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff43b7fe700 1 -- 172.21.15.33:0/1287742524 learned_addr learned my addr 172.21.15.33:0/1287742524 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff43affd700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff43c1547d0 0x7ff43c0accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.799+0000 7ff43a7fc700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 0x7ff43c13d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff43b7fe700 1 -- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 msgr2=0x7ff43c13d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff43b7fe700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 0x7ff43c13d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff43b7fe700 1 -- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff43c1547d0 msgr2=0x7ff43c0accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff43b7fe700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff43c1547d0 0x7ff43c0accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff43b7fe700 1 -- 172.21.15.33:0/1287742524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff434009ae0 con 0x7ff43c155410 2023-11-06T22:18:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff43b7fe700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c155410 0x7ff43c0c23c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 rx=0x7ff43400b6c0 tx=0x7ff434009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff422ffd700 1 -- 172.21.15.33:0/1287742524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434013d60 con 0x7ff43c155410 2023-11-06T22:18:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff422ffd700 1 -- 172.21.15.33:0/1287742524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43401d460 con 0x7ff43c155410 2023-11-06T22:18:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff422ffd700 1 -- 172.21.15.33:0/1287742524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43401d820 con 0x7ff43c155410 2023-11-06T22:18:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff4415ff700 1 -- 172.21.15.33:0/1287742524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c0bfe30 con 0x7ff43c155410 2023-11-06T22:18:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.800+0000 7ff4415ff700 1 -- 172.21.15.33:0/1287742524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c0c03f0 con 0x7ff43c155410 2023-11-06T22:18:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.802+0000 7ff4415ff700 1 -- 172.21.15.33:0/1287742524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c0c0910 con 0x7ff43c155410 2023-11-06T22:18:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.803+0000 7ff422ffd700 1 -- 172.21.15.33:0/1287742524 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff434003040 con 0x7ff43c155410 2023-11-06T22:18:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.804+0000 7ff422ffd700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff42805f270 0x7ff428061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.804+0000 7ff422ffd700 1 -- 172.21.15.33:0/1287742524 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7ff43407bba0 con 0x7ff43c155410 2023-11-06T22:18:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.805+0000 7ff43affd700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff42805f270 0x7ff428061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.805+0000 7ff43affd700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff42805f270 0x7ff428061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7ff42c00abd0 tx=0x7ff42c008040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.808+0000 7ff422ffd700 1 -- 172.21.15.33:0/1287742524 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff43c0c0910 con 0x7ff43c155410 2023-11-06T22:18:49.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.990+0000 7ff4415ff700 1 -- 172.21.15.33:0/1287742524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7ff43c0c08a0 con 0x7ff43c155410 2023-11-06T22:18:49.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.994+0000 7ff422ffd700 1 -- 172.21.15.33:0/1287742524 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v44) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff434022070 con 0x7ff43c155410 2023-11-06T22:18:49.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.996+0000 7ff4415ff700 1 -- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff42805f270 msgr2=0x7ff428061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:49.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.996+0000 7ff4415ff700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff42805f270 0x7ff428061730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7ff42c00abd0 tx=0x7ff42c008040).stop 2023-11-06T22:18:50.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.996+0000 7ff4415ff700 1 -- 172.21.15.33:0/1287742524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c155410 msgr2=0x7ff43c0c23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.996+0000 7ff4415ff700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c155410 0x7ff43c0c23c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 rx=0x7ff43400b6c0 tx=0x7ff434009840).stop 2023-11-06T22:18:50.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.996+0000 7ff4415ff700 1 -- 172.21.15.33:0/1287742524 shutdown_connections 2023-11-06T22:18:50.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.996+0000 7ff4415ff700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff42805f270 0x7ff428061730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.996+0000 7ff4415ff700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c155410 0x7ff43c0c23c0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.996+0000 7ff4415ff700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff43c1547d0 0x7ff43c0accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.996+0000 7ff4415ff700 1 --2- 172.21.15.33:0/1287742524 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff43c155030 0x7ff43c13d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.996+0000 7ff4415ff700 1 -- 172.21.15.33:0/1287742524 >> 172.21.15.33:0/1287742524 conn(0x7ff43c0c3d30 msgr2=0x7ff43c152270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:50.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.997+0000 7ff4415ff700 1 -- 172.21.15.33:0/1287742524 shutdown_connections 2023-11-06T22:18:50.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:49.997+0000 7ff4415ff700 1 -- 172.21.15.33:0/1287742524 wait complete. 2023-11-06T22:18:50.017 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T22:18:50.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.213+0000 7ff7375ff700 1 Processor -- start 2023-11-06T22:18:50.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.213+0000 7ff7375ff700 1 -- start start 2023-11-06T22:18:50.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.214+0000 7ff7375ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7300c0d40 0x7ff7301544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.214+0000 7ff7375ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff730063120 0x7ff730063560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.214+0000 7ff7375ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7300c5cb0 con 0x7ff7301540c0 2023-11-06T22:18:50.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.214+0000 7ff7375ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff730158e50 con 0x7ff7300c0d40 2023-11-06T22:18:50.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.214+0000 7ff7375ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff730158660 con 0x7ff730063120 2023-11-06T22:18:50.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.215+0000 7ff734b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7300c0d40 0x7ff7301544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.215+0000 7ff734b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7300c0d40 0x7ff7301544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46314/0 (socket says 172.21.15.33:46314) 2023-11-06T22:18:50.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.215+0000 7ff734b9a700 1 -- 172.21.15.33:0/2101421022 learned_addr learned my addr 172.21.15.33:0/2101421022 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:50.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.215+0000 7ff735b9c700 1 --2- 172.21.15.33:0/2101421022 >> v2:172.21.15.69:3300/0 conn(0x7ff730063120 0x7ff730063560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.215+0000 7ff734b9a700 1 -- 172.21.15.33:0/2101421022 >> v2:172.21.15.69:3300/0 conn(0x7ff730063120 msgr2=0x7ff730063560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.215+0000 7ff734b9a700 1 --2- 172.21.15.33:0/2101421022 >> v2:172.21.15.69:3300/0 conn(0x7ff730063120 0x7ff730063560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.215+0000 7ff734b9a700 1 -- 172.21.15.33:0/2101421022 >> v1:172.21.15.69:6789/0 conn(0x7ff7301540c0 legacy=0x7ff730157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.215+0000 7ff734b9a700 1 -- 172.21.15.33:0/2101421022 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7301588d0 con 0x7ff7300c0d40 2023-11-06T22:18:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.215+0000 7ff734b9a700 1 --2- 172.21.15.33:0/2101421022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7300c0d40 0x7ff7301544a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7ff730154990 tx=0x7ff7200090f0).ready entity=mon.0 client_cookie=f8347736182ee857 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff727fff700 1 -- 172.21.15.33:0/2101421022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000b520 con 0x7ff7300c0d40 2023-11-06T22:18:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff727fff700 1 -- 172.21.15.33:0/2101421022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72000b680 con 0x7ff7300c0d40 2023-11-06T22:18:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff727fff700 1 -- 172.21.15.33:0/2101421022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000ba20 con 0x7ff7300c0d40 2023-11-06T22:18:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff7375ff700 1 -- 172.21.15.33:0/2101421022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7300c0d40 msgr2=0x7ff7301544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff7375ff700 1 --2- 172.21.15.33:0/2101421022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7300c0d40 0x7ff7301544a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7ff730154990 tx=0x7ff7200090f0).stop 2023-11-06T22:18:50.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff7375ff700 1 -- 172.21.15.33:0/2101421022 shutdown_connections 2023-11-06T22:18:50.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff7375ff700 1 --2- 172.21.15.33:0/2101421022 >> v2:172.21.15.69:3300/0 conn(0x7ff730063120 0x7ff730063560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff7375ff700 1 --2- 172.21.15.33:0/2101421022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7300c0d40 0x7ff7301544a0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff7375ff700 1 -- 172.21.15.33:0/2101421022 >> 172.21.15.33:0/2101421022 conn(0x7ff73005ec80 msgr2=0x7ff7300cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff7375ff700 1 -- 172.21.15.33:0/2101421022 shutdown_connections 2023-11-06T22:18:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.216+0000 7ff7375ff700 1 -- 172.21.15.33:0/2101421022 wait complete. 2023-11-06T22:18:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.218+0000 7ff7375ff700 1 Processor -- start 2023-11-06T22:18:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.218+0000 7ff7375ff700 1 -- start start 2023-11-06T22:18:50.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff7375ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730063120 0x7ff7301418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff7375ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7300c0d40 0x7ff730152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff7375ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7301540c0 0x7ff730152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff7375ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730152f70 con 0x7ff730063120 2023-11-06T22:18:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff7375ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300aed10 con 0x7ff7300c0d40 2023-11-06T22:18:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff7375ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300aeff0 con 0x7ff7301540c0 2023-11-06T22:18:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff73539b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730063120 0x7ff7301418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff73539b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730063120 0x7ff7301418d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46328/0 (socket says 172.21.15.33:46328) 2023-11-06T22:18:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff73539b700 1 -- 172.21.15.33:0/2328017161 learned_addr learned my addr 172.21.15.33:0/2328017161 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff734b9a700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7300c0d40 0x7ff730152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.219+0000 7ff735b9c700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7301540c0 0x7ff730152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff73539b700 1 -- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7301540c0 msgr2=0x7ff730152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff73539b700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7301540c0 0x7ff730152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff73539b700 1 -- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7300c0d40 msgr2=0x7ff730152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff73539b700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7300c0d40 0x7ff730152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff73539b700 1 -- 172.21.15.33:0/2328017161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720009370 con 0x7ff730063120 2023-11-06T22:18:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff734b9a700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7300c0d40 0x7ff730152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff73539b700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730063120 0x7ff7301418d0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 rx=0x7ff72c00c440 tx=0x7ff72c007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff7257fa700 1 -- 172.21.15.33:0/2328017161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00cea0 con 0x7ff730063120 2023-11-06T22:18:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff7257fa700 1 -- 172.21.15.33:0/2328017161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c00f470 con 0x7ff730063120 2023-11-06T22:18:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff7257fa700 1 -- 172.21.15.33:0/2328017161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00f810 con 0x7ff730063120 2023-11-06T22:18:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff7375ff700 1 -- 172.21.15.33:0/2328017161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7300af2d0 con 0x7ff730063120 2023-11-06T22:18:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.220+0000 7ff7375ff700 1 -- 172.21.15.33:0/2328017161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7300af890 con 0x7ff730063120 2023-11-06T22:18:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.221+0000 7ff7375ff700 1 -- 172.21.15.33:0/2328017161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff730155720 con 0x7ff730063120 2023-11-06T22:18:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.222+0000 7ff7257fa700 1 -- 172.21.15.33:0/2328017161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7ff72c00fbc0 con 0x7ff730063120 2023-11-06T22:18:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.223+0000 7ff7257fa700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff71405f270 0x7ff714061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.223+0000 7ff7257fa700 1 -- 172.21.15.33:0/2328017161 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7ff72c079880 con 0x7ff730063120 2023-11-06T22:18:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.224+0000 7ff734b9a700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff71405f270 0x7ff714061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.224+0000 7ff734b9a700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff71405f270 0x7ff714061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff7300ad300 tx=0x7ff720017000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.227+0000 7ff7257fa700 1 -- 172.21.15.33:0/2328017161 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff72c04a9d0 con 0x7ff730063120 2023-11-06T22:18:50.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.410+0000 7ff7375ff700 1 -- 172.21.15.33:0/2328017161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff73006c6c0 con 0x7ff730063120 2023-11-06T22:18:50.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.416+0000 7ff7257fa700 1 -- 172.21.15.33:0/2328017161 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff72c04dff0 con 0x7ff730063120 2023-11-06T22:18:50.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 -- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff71405f270 msgr2=0x7ff714061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff71405f270 0x7ff714061730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff7300ad300 tx=0x7ff720017000).stop 2023-11-06T22:18:50.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 -- 172.21.15.33:0/2328017161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730063120 msgr2=0x7ff7301418d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730063120 0x7ff7301418d0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 rx=0x7ff72c00c440 tx=0x7ff72c007db0).stop 2023-11-06T22:18:50.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 -- 172.21.15.33:0/2328017161 shutdown_connections 2023-11-06T22:18:50.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7ff71405f270 0x7ff714061730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7ff7301540c0 0x7ff730152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730063120 0x7ff7301418d0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 --2- 172.21.15.33:0/2328017161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7300c0d40 0x7ff730152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 -- 172.21.15.33:0/2328017161 >> 172.21.15.33:0/2328017161 conn(0x7ff73005ec80 msgr2=0x7ff7300aa060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:50.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 -- 172.21.15.33:0/2328017161 shutdown_connections 2023-11-06T22:18:50.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.418+0000 7ff7375ff700 1 -- 172.21.15.33:0/2328017161 wait complete. 2023-11-06T22:18:50.438 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-11-06T22:18:50.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.633+0000 7f5fee3ff700 1 Processor -- start 2023-11-06T22:18:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.633+0000 7f5fee3ff700 1 -- start start 2023-11-06T22:18:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fee3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5fe81521a0 0x7f5fe80c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fee3ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5fe8155320 0x7f5fe80c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fee3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe8154e90 con 0x7f5fe8151dc0 2023-11-06T22:18:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fee3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe80639d0 con 0x7f5fe81521a0 2023-11-06T22:18:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fee3ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe8063cb0 con 0x7f5fe8155320 2023-11-06T22:18:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fe77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5fe81521a0 0x7f5fe80c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fe7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5fe8151dc0 0x7f5fe8063110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33536/0 (socket says 172.21.15.33:33536) 2023-11-06T22:18:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fe7fff700 1 -- 172.21.15.33:0/770106543 learned_addr learned my addr 172.21.15.33:0/770106543 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fec99c700 1 --2- 172.21.15.33:0/770106543 >> v2:172.21.15.69:3300/0 conn(0x7f5fe8155320 0x7f5fe80c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fe77fe700 1 -- 172.21.15.33:0/770106543 >> v2:172.21.15.69:3300/0 conn(0x7f5fe8155320 msgr2=0x7f5fe80c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fe77fe700 1 --2- 172.21.15.33:0/770106543 >> v2:172.21.15.69:3300/0 conn(0x7f5fe8155320 0x7f5fe80c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fe77fe700 1 -- 172.21.15.33:0/770106543 >> v1:172.21.15.33:6789/0 conn(0x7f5fe8151dc0 legacy=0x7f5fe8063110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fe77fe700 1 -- 172.21.15.33:0/770106543 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe8063f90 con 0x7f5fe81521a0 2023-11-06T22:18:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.635+0000 7f5fe6ffd700 1 -- 172.21.15.33:0/770106543 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441739605 0 0) 0x7f5fe8154e90 con 0x7f5fe8151dc0 2023-11-06T22:18:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.636+0000 7f5fe77fe700 1 --2- 172.21.15.33:0/770106543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 0x7f5fe80c0c50 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f5fe8154a10 tx=0x7f5fdc009060).ready entity=mon.0 client_cookie=7265c449a4de1755 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.636+0000 7f5fe6ffd700 1 -- 172.21.15.33:0/770106543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00b580 con 0x7f5fe81521a0 2023-11-06T22:18:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.636+0000 7f5fe6ffd700 1 -- 172.21.15.33:0/770106543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc00b6e0 con 0x7f5fe81521a0 2023-11-06T22:18:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.636+0000 7f5fe6ffd700 1 -- 172.21.15.33:0/770106543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00ba60 con 0x7f5fe81521a0 2023-11-06T22:18:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.636+0000 7f5fee3ff700 1 -- 172.21.15.33:0/770106543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 msgr2=0x7f5fe80c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.636+0000 7f5fee3ff700 1 --2- 172.21.15.33:0/770106543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 0x7f5fe80c0c50 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f5fe8154a10 tx=0x7f5fdc009060).stop 2023-11-06T22:18:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.636+0000 7f5fee3ff700 1 -- 172.21.15.33:0/770106543 shutdown_connections 2023-11-06T22:18:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.636+0000 7f5fee3ff700 1 --2- 172.21.15.33:0/770106543 >> v2:172.21.15.69:3300/0 conn(0x7f5fe8155320 0x7f5fe80c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.636+0000 7f5fee3ff700 1 --2- 172.21.15.33:0/770106543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 0x7f5fe80c0c50 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.636+0000 7f5fee3ff700 1 -- 172.21.15.33:0/770106543 >> 172.21.15.33:0/770106543 conn(0x7f5fe805e1b0 msgr2=0x7f5fe805e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.637+0000 7f5fee3ff700 1 -- 172.21.15.33:0/770106543 shutdown_connections 2023-11-06T22:18:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.637+0000 7f5fee3ff700 1 -- 172.21.15.33:0/770106543 wait complete. 2023-11-06T22:18:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.639+0000 7f5fee3ff700 1 Processor -- start 2023-11-06T22:18:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.639+0000 7f5fee3ff700 1 -- start start 2023-11-06T22:18:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.640+0000 7f5fee3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5fe8151dc0 0x7f5fe80c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.640+0000 7f5fee3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 0x7f5fe80c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.640+0000 7f5fee3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe8155320 0x7f5fe80c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.640+0000 7f5fee3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80ae780 con 0x7f5fe81521a0 2023-11-06T22:18:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.640+0000 7f5fee3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80c29c0 con 0x7f5fe8155320 2023-11-06T22:18:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.640+0000 7f5fee3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80c2ca0 con 0x7f5fe8151dc0 2023-11-06T22:18:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.640+0000 7f5fe77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 0x7f5fe80c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.640+0000 7f5fe77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 0x7f5fe80c1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46332/0 (socket says 172.21.15.33:46332) 2023-11-06T22:18:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.640+0000 7f5fe77fe700 1 -- 172.21.15.33:0/2703211531 learned_addr learned my addr 172.21.15.33:0/2703211531 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fe7fff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5fe8151dc0 0x7f5fe80c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fec99c700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe8155320 0x7f5fe80c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fe77fe700 1 -- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5fe8151dc0 msgr2=0x7f5fe80c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fe77fe700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5fe8151dc0 0x7f5fe80c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fe77fe700 1 -- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe8155320 msgr2=0x7f5fe80c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fe77fe700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe8155320 0x7f5fe80c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fe77fe700 1 -- 172.21.15.33:0/2703211531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0092a0 con 0x7f5fe81521a0 2023-11-06T22:18:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fec99c700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe8155320 0x7f5fe80c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fe7fff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5fe8151dc0 0x7f5fe80c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fe77fe700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 0x7f5fe80c1ec0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 rx=0x7f5fdc009fd0 tx=0x7f5fdc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fcffff700 1 -- 172.21.15.33:0/2703211531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00b850 con 0x7f5fe81521a0 2023-11-06T22:18:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fcffff700 1 -- 172.21.15.33:0/2703211531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc00b9b0 con 0x7f5fe81521a0 2023-11-06T22:18:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fcffff700 1 -- 172.21.15.33:0/2703211531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc01b660 con 0x7f5fe81521a0 2023-11-06T22:18:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fee3ff700 1 -- 172.21.15.33:0/2703211531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe80ae9f0 con 0x7f5fe81521a0 2023-11-06T22:18:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.641+0000 7f5fee3ff700 1 -- 172.21.15.33:0/2703211531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe80c2f80 con 0x7f5fe81521a0 2023-11-06T22:18:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.642+0000 7f5fee3ff700 1 -- 172.21.15.33:0/2703211531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe80ae5e0 con 0x7f5fe81521a0 2023-11-06T22:18:50.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.643+0000 7f5fcffff700 1 -- 172.21.15.33:0/2703211531 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f5fdc01ba90 con 0x7f5fe81521a0 2023-11-06T22:18:50.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.644+0000 7f5fcffff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5fd405f270 0x7f5fd4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:50.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.644+0000 7f5fcffff700 1 -- 172.21.15.33:0/2703211531 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f5fdc07be30 con 0x7f5fe81521a0 2023-11-06T22:18:50.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.645+0000 7f5fe7fff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5fd405f270 0x7f5fd4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:50.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.645+0000 7f5fe7fff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5fd405f270 0x7f5fd4061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f5fd8007fd0 tx=0x7f5fd8009040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:50.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.648+0000 7f5fcffff700 1 -- 172.21.15.33:0/2703211531 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5fe80ae5e0 con 0x7f5fe81521a0 2023-11-06T22:18:50.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:50.895+0000 7f5fee3ff700 1 -- 172.21.15.33:0/2703211531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f5fe80ae5e0 con 0x7f5fe81521a0 2023-11-06T22:18:51.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.570+0000 7f5fcffff700 1 -- 172.21.15.33:0/2703211531 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3063) v1 ==== 155+0+0 (secure 0 0 0) 0x7f5fe80ae5e0 con 0x7f5fe81521a0 2023-11-06T22:18:51.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 -- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5fd405f270 msgr2=0x7f5fd4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:51.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5fd405f270 0x7f5fd4061730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f5fd8007fd0 tx=0x7f5fd8009040).stop 2023-11-06T22:18:51.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 -- 172.21.15.33:0/2703211531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 msgr2=0x7f5fe80c1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:51.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 0x7f5fe80c1ec0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 rx=0x7f5fdc009fd0 tx=0x7f5fdc017000).stop 2023-11-06T22:18:51.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 -- 172.21.15.33:0/2703211531 shutdown_connections 2023-11-06T22:18:51.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f5fd405f270 0x7f5fd4061730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:51.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f5fe8151dc0 0x7f5fe80c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:51.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe8155320 0x7f5fe80c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:51.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 --2- 172.21.15.33:0/2703211531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81521a0 0x7f5fe80c1ec0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:51.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 -- 172.21.15.33:0/2703211531 >> 172.21.15.33:0/2703211531 conn(0x7f5fe805e1b0 msgr2=0x7f5fe8154060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:51.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 -- 172.21.15.33:0/2703211531 shutdown_connections 2023-11-06T22:18:51.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:51.573+0000 7f5fee3ff700 1 -- 172.21.15.33:0/2703211531 wait complete. 2023-11-06T22:18:51.596 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-11-06T22:18:51.597 INFO:tasks.cephfs_test_runner: 2023-11-06T22:18:51.597 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-06T22:18:51.597 INFO:tasks.cephfs_test_runner:Ran 11 tests in 5646.826s 2023-11-06T22:18:51.597 INFO:tasks.cephfs_test_runner: 2023-11-06T22:18:51.598 INFO:tasks.cephfs_test_runner:OK 2023-11-06T22:18:51.598 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-06T22:18:51.614 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-06T22:18:51.629 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-06T22:18:51.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:51.630 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T22:18:51.680 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T22:18:51.681 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T22:18:51.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:51.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T22:18:51.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:51.758 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T22:18:51.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:51.834 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2023-11-06T22:18:51.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:51.958 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-11-06T22:18:52.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:52.009 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2023-11-06T22:18:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2023-11-06T22:18:52.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T22:18:52.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:52.091 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2023-11-06T22:18:52.144 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-06T22:18:52.144 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-06T22:18:52.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:52.146 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-06T22:18:52.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:52.221 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-06T22:18:52.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:52.297 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2023-11-06T22:18:52.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:52.436 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2023-11-06T22:18:52.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T22:18:52.487 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2023-11-06T22:18:52.563 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2023-11-06T22:18:52.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T22:18:52.567 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-06T22:18:52.579 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-11-06T22:18:52.579 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.782+0000 7fe59c9ff700 1 Processor -- start 2023-11-06T22:18:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.782+0000 7fe59c9ff700 1 -- start start 2023-11-06T22:18:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.783+0000 7fe59c9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe5981584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.783+0000 7fe59c9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe5980a7910 0x7fe5980a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.783+0000 7fe59c9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe598064010 con 0x7fe5981580e0 2023-11-06T22:18:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.783+0000 7fe59c9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5980be430 con 0x7fe5980a7910 2023-11-06T22:18:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.783+0000 7fe59c9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980be150 con 0x7fe5980bdbe0 2023-11-06T22:18:52.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.784+0000 7fe595d9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe5981584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:52.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.784+0000 7fe595d9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe5981584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3301/0 says I am v2:172.21.15.33:36220/0 (socket says 172.21.15.33:36220) 2023-11-06T22:18:52.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.784+0000 7fe595d9b700 1 -- 172.21.15.33:0/1687513512 learned_addr learned my addr 172.21.15.33:0/1687513512 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.784+0000 7fe596d9d700 1 --2- 172.21.15.33:0/1687513512 >> v2:172.21.15.69:3300/0 conn(0x7fe5980a7910 0x7fe5980a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:52.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.784+0000 7fe595d9b700 1 -- 172.21.15.33:0/1687513512 >> v2:172.21.15.69:3300/0 conn(0x7fe5980a7910 msgr2=0x7fe5980a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:52.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.784+0000 7fe595d9b700 1 --2- 172.21.15.33:0/1687513512 >> v2:172.21.15.69:3300/0 conn(0x7fe5980a7910 0x7fe5980a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:52.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.784+0000 7fe59559a700 1 -- 172.21.15.33:0/1687513512 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937278874 0 0) 0x7fe598064010 con 0x7fe5981580e0 2023-11-06T22:18:52.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.784+0000 7fe595d9b700 1 -- 172.21.15.33:0/1687513512 >> v1:172.21.15.69:6789/0 conn(0x7fe5981580e0 legacy=0x7fe5980684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:52.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.784+0000 7fe595d9b700 1 -- 172.21.15.33:0/1687513512 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5980639c0 con 0x7fe5980bdbe0 2023-11-06T22:18:52.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.784+0000 7fe595d9b700 1 --2- 172.21.15.33:0/1687513512 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe5981584c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fe58c006fd0 tx=0x7fe58c0090f0).ready entity=mon.2 client_cookie=20fbd4c966af023b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:52.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59559a700 1 -- 172.21.15.33:0/1687513512 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00b580 con 0x7fe5980bdbe0 2023-11-06T22:18:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59559a700 1 -- 172.21.15.33:0/1687513512 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c00b6e0 con 0x7fe5980bdbe0 2023-11-06T22:18:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59559a700 1 -- 172.21.15.33:0/1687513512 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00ba60 con 0x7fe5980bdbe0 2023-11-06T22:18:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59c9ff700 1 -- 172.21.15.33:0/1687513512 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 msgr2=0x7fe5981584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:52.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59c9ff700 1 --2- 172.21.15.33:0/1687513512 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe5981584c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fe58c006fd0 tx=0x7fe58c0090f0).stop 2023-11-06T22:18:52.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59c9ff700 1 -- 172.21.15.33:0/1687513512 shutdown_connections 2023-11-06T22:18:52.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59c9ff700 1 --2- 172.21.15.33:0/1687513512 >> v2:172.21.15.69:3300/0 conn(0x7fe5980a7910 0x7fe5980a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:52.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59c9ff700 1 --2- 172.21.15.33:0/1687513512 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe5981584c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59c9ff700 1 -- 172.21.15.33:0/1687513512 >> 172.21.15.33:0/1687513512 conn(0x7fe59805ec80 msgr2=0x7fe5980664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59c9ff700 1 -- 172.21.15.33:0/1687513512 shutdown_connections 2023-11-06T22:18:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.785+0000 7fe59c9ff700 1 -- 172.21.15.33:0/1687513512 wait complete. 2023-11-06T22:18:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.787+0000 7fe59c9ff700 1 Processor -- start 2023-11-06T22:18:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.787+0000 7fe59c9ff700 1 -- start start 2023-11-06T22:18:52.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe59c9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5980a7910 0x7fe598168600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:52.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe59c9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe598152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:52.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe59c9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5981580e0 0x7fe598138f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:52.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe59c9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598165190 con 0x7fe5980a7910 2023-11-06T22:18:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe59c9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598164580 con 0x7fe5981580e0 2023-11-06T22:18:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe59c9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe598164860 con 0x7fe5980bdbe0 2023-11-06T22:18:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe59659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5980a7910 0x7fe598168600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe59659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5980a7910 0x7fe598168600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46338/0 (socket says 172.21.15.33:46338) 2023-11-06T22:18:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe59659c700 1 -- 172.21.15.33:0/331951177 learned_addr learned my addr 172.21.15.33:0/331951177 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe595d9b700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe598152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.788+0000 7fe596d9d700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5981580e0 0x7fe598138f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe59659c700 1 -- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 msgr2=0x7fe598152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe59659c700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe598152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe59659c700 1 -- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5981580e0 msgr2=0x7fe598138f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe59659c700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5981580e0 0x7fe598138f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe59659c700 1 -- 172.21.15.33:0/331951177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c009370 con 0x7fe5980a7910 2023-11-06T22:18:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe595d9b700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe598152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:52.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe59659c700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5980a7910 0x7fe598168600 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 rx=0x7fe588001eb0 tx=0x7fe588007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe5867fc700 1 -- 172.21.15.33:0/331951177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58800ce60 con 0x7fe5980a7910 2023-11-06T22:18:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe5867fc700 1 -- 172.21.15.33:0/331951177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58800f470 con 0x7fe5980a7910 2023-11-06T22:18:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe59c9ff700 1 -- 172.21.15.33:0/331951177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe598164b40 con 0x7fe5980a7910 2023-11-06T22:18:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe5867fc700 1 -- 172.21.15.33:0/331951177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58800f810 con 0x7fe5980a7910 2023-11-06T22:18:52.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.789+0000 7fe59c9ff700 1 -- 172.21.15.33:0/331951177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe598152a30 con 0x7fe5980a7910 2023-11-06T22:18:52.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.790+0000 7fe59c9ff700 1 -- 172.21.15.33:0/331951177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe598165470 con 0x7fe5980a7910 2023-11-06T22:18:52.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.791+0000 7fe5867fc700 1 -- 172.21.15.33:0/331951177 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe58800fb80 con 0x7fe5980a7910 2023-11-06T22:18:52.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.792+0000 7fe5867fc700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe57805f270 0x7fe578061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:52.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.792+0000 7fe5867fc700 1 -- 172.21.15.33:0/331951177 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fe588079880 con 0x7fe5980a7910 2023-11-06T22:18:52.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.792+0000 7fe595d9b700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe57805f270 0x7fe578061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:52.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.793+0000 7fe595d9b700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe57805f270 0x7fe578061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe58c009130 tx=0x7fe58c017000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:52.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.796+0000 7fe5867fc700 1 -- 172.21.15.33:0/331951177 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe58804aa30 con 0x7fe5980a7910 2023-11-06T22:18:52.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.975+0000 7fe59c9ff700 1 -- 172.21.15.33:0/331951177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe598066bf0 con 0x7fe5980a7910 2023-11-06T22:18:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.976+0000 7fe5867fc700 1 -- 172.21.15.33:0/331951177 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v229) v1 ==== 74+0+14648 (secure 0 0 0) 0x7fe58804e050 con 0x7fe5980a7910 2023-11-06T22:18:52.978 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:52.979 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":229,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:49.560229+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"227","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6836/3921249578":"2023-11-07T22:18:45.998989+0000","172.21.15.69:6834/3921249578":"2023-11-07T22:18:45.998989+0000","172.21.15.69:0/1054757794":"2023-11-07T22:18:45.528113+0000","172.21.15.69:0/3848122667":"2023-11-07T22:18:45.528113+0000","172.21.15.69:0/3627333266":"2023-11-07T22:18:45.528113+0000","172.21.15.69:0/2379584039":"2023-11-07T22:18:45.528113+0000","172.21.15.69:0/856300318":"2023-11-07T22:18:45.528113+0000","172.21.15.69:0/672707063":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/2065981382":"2023-11-07T22:18:37.037458+0000","172.21.15.69:6833/129946":"2023-11-07T22:18:37.037458+0000","172.21.15.69:6832/129946":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/3481533320":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/2510047576":"2023-11-07T22:18:37.037458+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:52.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 -- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe57805f270 msgr2=0x7fe578061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:52.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe57805f270 0x7fe578061730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe58c009130 tx=0x7fe58c017000).stop 2023-11-06T22:18:52.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 -- 172.21.15.33:0/331951177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5980a7910 msgr2=0x7fe598168600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:52.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5980a7910 0x7fe598168600 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 rx=0x7fe588001eb0 tx=0x7fe588007c70).stop 2023-11-06T22:18:52.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 -- 172.21.15.33:0/331951177 shutdown_connections 2023-11-06T22:18:52.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe57805f270 0x7fe578061730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5981580e0 0x7fe598138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5980a7910 0x7fe598168600 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 --2- 172.21.15.33:0/331951177 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe5980bdbe0 0x7fe598152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 -- 172.21.15.33:0/331951177 >> 172.21.15.33:0/331951177 conn(0x7fe59805ec80 msgr2=0x7fe59814e660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 -- 172.21.15.33:0/331951177 shutdown_connections 2023-11-06T22:18:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:52.978+0000 7fe59c9ff700 1 -- 172.21.15.33:0/331951177 wait complete. 2023-11-06T22:18:52.998 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-06T22:18:52.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:18:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.192+0000 7fdceb7ff700 1 Processor -- start 2023-11-06T22:18:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.192+0000 7fdceb7ff700 1 -- start start 2023-11-06T22:18:53.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.193+0000 7fdceb7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdce40c3600 0x7fdce40c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdceb7ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdce4136f50 0x7fdce4137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdceb7ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce40c17e0 con 0x7fdce4069410 2023-11-06T22:18:53.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdceb7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce4069f90 con 0x7fdce40c3600 2023-11-06T22:18:53.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdceb7ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce406a740 con 0x7fdce4136f50 2023-11-06T22:18:53.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdce959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdce40c3600 0x7fdce40c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:53.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdce959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdce40c3600 0x7fdce40c39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46344/0 (socket says 172.21.15.33:46344) 2023-11-06T22:18:53.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdce959b700 1 -- 172.21.15.33:0/1906272545 learned_addr learned my addr 172.21.15.33:0/1906272545 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:53.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdce959b700 1 -- 172.21.15.33:0/1906272545 >> v2:172.21.15.69:3300/0 conn(0x7fdce4136f50 msgr2=0x7fdce4137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:53.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdce959b700 1 --2- 172.21.15.33:0/1906272545 >> v2:172.21.15.69:3300/0 conn(0x7fdce4136f50 0x7fdce4137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:53.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdce959b700 1 -- 172.21.15.33:0/1906272545 >> v1:172.21.15.69:6789/0 conn(0x7fdce4069410 legacy=0x7fdce4069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:53.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.194+0000 7fdce959b700 1 -- 172.21.15.33:0/1906272545 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce406a180 con 0x7fdce40c3600 2023-11-06T22:18:53.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.195+0000 7fdce959b700 1 --2- 172.21.15.33:0/1906272545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 0x7fdce40c39e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 rx=0x7fdce40c1750 tx=0x7fdce00090f0).ready entity=mon.0 client_cookie=bb97acebd7578dd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:53.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.195+0000 7fdcdbfff700 1 -- 172.21.15.33:0/1906272545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce000b5a0 con 0x7fdce40c3600 2023-11-06T22:18:53.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.195+0000 7fdcdbfff700 1 -- 172.21.15.33:0/1906272545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce000b700 con 0x7fdce40c3600 2023-11-06T22:18:53.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.195+0000 7fdcdbfff700 1 -- 172.21.15.33:0/1906272545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce000ba80 con 0x7fdce40c3600 2023-11-06T22:18:53.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.195+0000 7fdceb7ff700 1 -- 172.21.15.33:0/1906272545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 msgr2=0x7fdce40c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:53.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.195+0000 7fdceb7ff700 1 --2- 172.21.15.33:0/1906272545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 0x7fdce40c39e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 rx=0x7fdce40c1750 tx=0x7fdce00090f0).stop 2023-11-06T22:18:53.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.195+0000 7fdceb7ff700 1 -- 172.21.15.33:0/1906272545 shutdown_connections 2023-11-06T22:18:53.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.195+0000 7fdceb7ff700 1 --2- 172.21.15.33:0/1906272545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 0x7fdce40c39e0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.195+0000 7fdceb7ff700 1 --2- 172.21.15.33:0/1906272545 >> v2:172.21.15.69:3300/0 conn(0x7fdce4136f50 0x7fdce4137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:53.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.195+0000 7fdceb7ff700 1 -- 172.21.15.33:0/1906272545 >> 172.21.15.33:0/1906272545 conn(0x7fdce4062710 msgr2=0x7fdce4062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:53.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.196+0000 7fdceb7ff700 1 -- 172.21.15.33:0/1906272545 shutdown_connections 2023-11-06T22:18:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.196+0000 7fdceb7ff700 1 -- 172.21.15.33:0/1906272545 wait complete. 2023-11-06T22:18:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.197+0000 7fdceb7ff700 1 Processor -- start 2023-11-06T22:18:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.197+0000 7fdceb7ff700 1 -- start start 2023-11-06T22:18:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.198+0000 7fdceb7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdce4069410 0x7fdce41480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.198+0000 7fdceb7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 0x7fdce4163ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.198+0000 7fdceb7ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdce4136f50 0x7fdce414b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.198+0000 7fdceb7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce414bae0 con 0x7fdce40c3600 2023-11-06T22:18:53.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.198+0000 7fdceb7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce4147c20 con 0x7fdce4069410 2023-11-06T22:18:53.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.198+0000 7fdceb7ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce41472f0 con 0x7fdce4136f50 2023-11-06T22:18:53.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.198+0000 7fdce8d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 0x7fdce4163ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:53.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.198+0000 7fdce8d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 0x7fdce4163ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46346/0 (socket says 172.21.15.33:46346) 2023-11-06T22:18:53.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.198+0000 7fdce8d9a700 1 -- 172.21.15.33:0/48616977 learned_addr learned my addr 172.21.15.33:0/48616977 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:53.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdce959b700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdce4069410 0x7fdce41480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:53.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdce9d9c700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdce4136f50 0x7fdce414b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:53.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdce8d9a700 1 -- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdce4136f50 msgr2=0x7fdce414b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdce8d9a700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdce4136f50 0x7fdce414b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdce8d9a700 1 -- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdce4069410 msgr2=0x7fdce41480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdce8d9a700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdce4069410 0x7fdce41480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdce8d9a700 1 -- 172.21.15.33:0/48616977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce0009370 con 0x7fdce40c3600 2023-11-06T22:18:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdce9d9c700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdce4136f50 0x7fdce414b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdce959b700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdce4069410 0x7fdce41480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdce8d9a700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 0x7fdce4163ac0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 rx=0x7fdcd400b4c0 tx=0x7fdcd4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdcd97fa700 1 -- 172.21.15.33:0/48616977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4002320 con 0x7fdce40c3600 2023-11-06T22:18:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdcd97fa700 1 -- 172.21.15.33:0/48616977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd40024a0 con 0x7fdce40c3600 2023-11-06T22:18:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdceb7ff700 1 -- 172.21.15.33:0/48616977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce41475d0 con 0x7fdce40c3600 2023-11-06T22:18:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.199+0000 7fdcd97fa700 1 -- 172.21.15.33:0/48616977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4002860 con 0x7fdce40c3600 2023-11-06T22:18:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.200+0000 7fdceb7ff700 1 -- 172.21.15.33:0/48616977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce4148780 con 0x7fdce40c3600 2023-11-06T22:18:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.201+0000 7fdceb7ff700 1 -- 172.21.15.33:0/48616977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce4147db0 con 0x7fdce40c3600 2023-11-06T22:18:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.201+0000 7fdcd97fa700 1 -- 172.21.15.33:0/48616977 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fdcd4002c10 con 0x7fdce40c3600 2023-11-06T22:18:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.202+0000 7fdcd97fa700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdcc805f270 0x7fdcc8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.203+0000 7fdcd97fa700 1 -- 172.21.15.33:0/48616977 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fdcd407a110 con 0x7fdce40c3600 2023-11-06T22:18:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.203+0000 7fdce959b700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdcc805f270 0x7fdcc8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.203+0000 7fdce959b700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdcc805f270 0x7fdcc8061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fdce0009130 tx=0x7fdce0015000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.206+0000 7fdcd97fa700 1 -- 172.21.15.33:0/48616977 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdcd404b2c0 con 0x7fdce40c3600 2023-11-06T22:18:53.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.382+0000 7fdceb7ff700 1 -- 172.21.15.33:0/48616977 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdce406a460 con 0x7fdcc805f270 2023-11-06T22:18:53.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.386+0000 7fdcd97fa700 1 -- 172.21.15.33:0/48616977 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70598 (secure 0 0 0) 0x7fdce406a460 con 0x7fdcc805f270 2023-11-06T22:18:53.389 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:53.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 -- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdcc805f270 msgr2=0x7fdcc8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:53.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdcc805f270 0x7fdcc8061730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fdce0009130 tx=0x7fdce0015000).stop 2023-11-06T22:18:53.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 -- 172.21.15.33:0/48616977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 msgr2=0x7fdce4163ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:53.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 0x7fdce4163ac0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 rx=0x7fdcd400b4c0 tx=0x7fdcd4009060).stop 2023-11-06T22:18:53.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 -- 172.21.15.33:0/48616977 shutdown_connections 2023-11-06T22:18:53.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fdcc805f270 0x7fdcc8061730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fdce4136f50 0x7fdce414b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdce4069410 0x7fdce41480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 --2- 172.21.15.33:0/48616977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce40c3600 0x7fdce4163ac0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 -- 172.21.15.33:0/48616977 >> 172.21.15.33:0/48616977 conn(0x7fdce4062710 msgr2=0x7fdce40bbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:53.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 -- 172.21.15.33:0/48616977 shutdown_connections 2023-11-06T22:18:53.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.388+0000 7fdceb7ff700 1 -- 172.21.15.33:0/48616977 wait complete. 2023-11-06T22:18:53.394 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:18:53.408 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-11-06T22:18:51.550432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2464684,"kb_used_data":22380,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752510036,"statfs":{"total":773094113280,"available":770570276864,"internally_reserved":0,"allocated":22917120,"data_stored":11797579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.009297"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":195,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:49.073878+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T22:18:49.073878+0000","last_peered":"2023-11-06T22:18:49.073878+0000","last_clean":"2023-11-06T22:18:49.073878+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:18:49.073878+0000","last_undegraded":"2023-11-06T22:18:49.073878+0000","last_fullsized":"2023-11-06T22:18:49.073878+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.567718+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T22:18:48.567718+0000","last_peered":"2023-11-06T22:18:48.567718+0000","last_clean":"2023-11-06T22:18:48.567718+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:18:48.567718+0000","last_undegraded":"2023-11-06T22:18:48.567718+0000","last_fullsized":"2023-11-06T22:18:48.567718+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":217,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.565920+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T22:18:48.565920+0000","last_peered":"2023-11-06T22:18:48.565920+0000","last_clean":"2023-11-06T22:18:48.565920+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:18:48.565920+0000","last_undegraded":"2023-11-06T22:18:48.565920+0000","last_fullsized":"2023-11-06T22:18:48.565920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.568883+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T22:18:48.568883+0000","last_peered":"2023-11-06T22:18:48.568883+0000","last_clean":"2023-11-06T22:18:48.568883+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:18:48.568883+0000","last_undegraded":"2023-11-06T22:18:48.568883+0000","last_fullsized":"2023-11-06T22:18:48.568883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":194,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.565826+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T22:18:48.565826+0000","last_peered":"2023-11-06T22:18:48.565826+0000","last_clean":"2023-11-06T22:18:48.565826+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:18:48.565826+0000","last_undegraded":"2023-11-06T22:18:48.565826+0000","last_fullsized":"2023-11-06T22:18:48.565826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.566133+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T22:18:48.566133+0000","last_peered":"2023-11-06T22:18:48.566133+0000","last_clean":"2023-11-06T22:18:48.566133+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:18:48.566133+0000","last_undegraded":"2023-11-06T22:18:48.566133+0000","last_fullsized":"2023-11-06T22:18:48.566133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.567159+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:18:48.567159+0000","last_peered":"2023-11-06T22:18:48.567159+0000","last_clean":"2023-11-06T22:18:48.567159+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:18:48.567159+0000","last_undegraded":"2023-11-06T22:18:48.567159+0000","last_fullsized":"2023-11-06T22:18:48.567159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":191,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-11-06T22:18:45.605973+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T22:18:45.605973+0000","last_peered":"2023-11-06T22:18:45.605973+0000","last_clean":"2023-11-06T22:18:45.605973+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:18:45.605973+0000","last_undegraded":"2023-11-06T22:18:45.605973+0000","last_fullsized":"2023-11-06T22:18:45.605973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.568057+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:18:48.568057+0000","last_peered":"2023-11-06T22:18:48.568057+0000","last_clean":"2023-11-06T22:18:48.568057+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:18:48.568057+0000","last_undegraded":"2023-11-06T22:18:48.568057+0000","last_fullsized":"2023-11-06T22:18:48.568057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575973,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.97199999999999998,"15min":1.0980000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.51200000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.94599999999999995}]},{"osd":1,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.871,"5min":0.98799999999999999,"15min":1.038},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":0.96599999999999997,"15min":1.0680000000000001},"last":0.80400000000000005}]},{"osd":2,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94799999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.032},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88600000000000001,"5min":0.996,"15min":1.0209999999999999},"last":0.81100000000000005}]},{"osd":3,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.499},"max":{"1min":0.88200000000000001,"5min":0.90700000000000003,"15min":0.95799999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.59199999999999997,"5min":0.54200000000000004,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.91500000000000004}]},{"osd":4,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.44900000000000001},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60599999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.436,"5min":0.432,"15min":0.40400000000000003},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.432},"max":{"1min":1.0329999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":1.004}]},{"osd":6,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.48399999999999999},"max":{"1min":1.0669999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.052},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.52800000000000002,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.069},"last":0.77700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68100000000000005,"15min":0.70999999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.75},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":1.0469999999999999,"15min":1.0660000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":1.016},"last":0.81000000000000005}]},{"osd":2,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0780000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.046},"last":0.88}]},{"osd":3,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.873,"5min":0.94799999999999995,"15min":1.002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.93799999999999994}]},{"osd":4,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.53600000000000003,"5min":0.42299999999999999,"15min":0.41699999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.98299999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76900000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.67100000000000004},"min":{"1min":0.44,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83499999999999996,"5min":0.98499999999999999,"15min":0.98999999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.442},"max":{"1min":0.80000000000000004,"5min":0.996,"15min":1.056},"last":0.85199999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79800000000000004,"5min":1.012,"15min":1.0429999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0569999999999999,"15min":1.1279999999999999},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575973,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.97999999999999998},"last":0.311},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.33800000000000002},"max":{"1min":0.66700000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.40400000000000003}]},{"osd":2,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.307,"15min":0.307},"max":{"1min":0.67400000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.95699999999999996,"15min":1.0189999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74099999999999999,"5min":0.76200000000000001,"15min":1.0509999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":0.92700000000000005},"last":0.437}]},{"osd":4,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80900000000000005,"15min":0.80800000000000005},"min":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.45200000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.753},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84899999999999998,"15min":0.85199999999999998},"min":{"1min":0.68300000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.73299999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.81599999999999995},"min":{"1min":0.58899999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.052,"5min":1.2210000000000001,"15min":1.2709999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81499999999999995,"15min":0.80200000000000005},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.55400000000000005},"max":{"1min":1.224,"5min":1.224,"15min":1.3109999999999999},"last":0.57999999999999996}]},{"osd":6,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.3149999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.84399999999999997,"15min":0.84899999999999998},"min":{"1min":0.64900000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1459999999999999,"5min":1.21,"15min":1.21},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1970000000000001,"15min":1.2509999999999999},"last":0.72099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47499999999999998,"15min":0.46899999999999997},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.499}]},{"osd":2,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.48499999999999999},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.435}]},{"osd":3,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.375},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.85199999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.54800000000000004},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.84199999999999997,"15min":0.85399999999999998},"min":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81699999999999995,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.52600000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83599999999999997,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.71499999999999997}]},{"osd":7,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82999999999999996,"15min":0.85699999999999998},"min":{"1min":0.55200000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.268},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.85499999999999998,"15min":0.876},"min":{"1min":0.58299999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.67600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575973,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.629,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70499999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.34100000000000003}]},{"osd":1,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.77400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.48799999999999999,"5min":0.46000000000000002,"15min":0.45100000000000001},"max":{"1min":0.79300000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.439}]},{"osd":3,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.40999999999999998,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.74299999999999999,"5min":0.74399999999999999,"15min":0.78300000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.42499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72299999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.82199999999999995,"15min":0.82699999999999996},"min":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.53900000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.349},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2609999999999999,"5min":1.2669999999999999,"15min":1.339},"last":0.56200000000000006}]},{"osd":5,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81200000000000006,"15min":0.81699999999999995},"min":{"1min":0.54600000000000004,"5min":0.54300000000000004,"15min":0.48999999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.337},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84199999999999997,"15min":0.85499999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1899999999999999,"5min":1.212,"15min":1.468},"last":0.56999999999999995}]},{"osd":6,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.83099999999999996,"15min":0.84899999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.54600000000000004},"max":{"1min":1.147,"5min":1.155,"15min":1.387},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.84499999999999997,"15min":0.85099999999999998},"min":{"1min":0.627,"5min":0.56499999999999995,"15min":0.56100000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.4299999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.86599999999999999,"15min":0.876},"min":{"1min":0.61199999999999999,"5min":0.58599999999999997,"15min":0.55700000000000005},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.496},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85999999999999999,"15min":0.876},"min":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1459999999999999,"5min":1.2470000000000001,"15min":1.508},"last":0.65300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575974,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45700000000000002,"15min":0.47499999999999998},"min":{"1min":0.28399999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.31},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.495}]},{"osd":1,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68700000000000006,"5min":1.05,"15min":1.05},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.57199999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70499999999999996,"5min":0.873,"15min":0.873},"last":0.58599999999999997}]},{"osd":2,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54400000000000004,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.87},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.69199999999999995}]},{"osd":4,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0669999999999999,"5min":1.161,"15min":1.2370000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.82899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1200000000000001,"5min":1.147,"15min":1.226},"last":1.0389999999999999}]},{"osd":5,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77700000000000002,"15min":0.80700000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0449999999999999,"5min":1.071,"15min":1.2270000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.3140000000000001},"last":0.95199999999999996}]},{"osd":6,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.82399999999999995},"min":{"1min":0.52700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0429999999999999,"5min":1.149,"15min":1.2290000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81200000000000006,"15min":0.85199999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.099,"5min":1.099,"15min":1.1879999999999999},"last":0.97399999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.196,"5min":1.196,"15min":1.2829999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.81799999999999995,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.113,"5min":1.169,"15min":1.248},"last":1.022}]}]},{"osd":4,"up_from":13,"seq":55834575972,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308040,"kb_used_data":2752,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063800,"statfs":{"total":96636764160,"available":96321331200,"internally_reserved":0,"allocated":2818048,"data_stored":1450480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70199999999999996,"15min":0.71699999999999997},"min":{"1min":0.58399999999999996,"5min":0.378,"15min":0.378},"max":{"1min":0.94899999999999995,"5min":0.96599999999999997,"15min":0.98999999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69099999999999995,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.006},"last":0.501}]},{"osd":1,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.0289999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.018,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72199999999999998}]},{"osd":2,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73999999999999999,"15min":0.75700000000000001},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.42599999999999999},"max":{"1min":0.90600000000000003,"5min":0.98599999999999999,"15min":1.087},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.63800000000000001,"5min":0.53500000000000003,"15min":0.48299999999999998},"max":{"1min":0.98499999999999999,"5min":1.04,"15min":1.071},"last":0.77400000000000002}]},{"osd":3,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.57599999999999996,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.0329999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90900000000000003,"5min":1.236,"15min":1.236},"last":0.74299999999999999}]},{"osd":5,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.42399999999999999,"15min":0.40100000000000002},"max":{"1min":0.81499999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63700000000000001,"15min":0.63800000000000001},"min":{"1min":0.46400000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.85599999999999998,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.63600000000000001}]},{"osd":6,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.871,"5min":1.018,"15min":1.018},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.46300000000000002},"max":{"1min":0.91700000000000004,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.80900000000000005}]},{"osd":7,"last update":"Mon Nov 6 22:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.441,"15min":0.441},"max":{"1min":0.96499999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.50800000000000001},"max":{"1min":0.94199999999999995,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575974,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.309},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1639999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.70099999999999996}]},{"osd":1,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77800000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47999999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.95999999999999996}]},{"osd":2,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78700000000000003,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.48299999999999998},"max":{"1min":0.96499999999999997,"5min":4.9779999999999998,"15min":4.9779999999999998},"last":0.63100000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.36499999999999999},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.752,"15min":0.747},"min":{"1min":0.499,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.2390000000000001,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":0.92300000000000004}]},{"osd":4,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.38800000000000001},"max":{"1min":1.4019999999999999,"5min":1.585,"15min":1.585},"last":0.86299999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":1.4510000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77400000000000002,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.441,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.41399999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78200000000000003,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":1.4810000000000001,"5min":6.891,"15min":6.891},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:18:53.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:18:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.605+0000 7f4a891ff700 1 Processor -- start 2023-11-06T22:18:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.605+0000 7f4a891ff700 1 -- start start 2023-11-06T22:18:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.606+0000 7f4a891ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a840c1840 0x7f4a840a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.606+0000 7f4a891ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a840aa2d0 0x7f4a840c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.606+0000 7f4a891ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a841554a0 con 0x7f4a840c6240 2023-11-06T22:18:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.606+0000 7f4a891ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a84154960 con 0x7f4a840c1840 2023-11-06T22:18:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.606+0000 7f4a891ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84155930 con 0x7f4a840aa2d0 2023-11-06T22:18:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.607+0000 7f4a82d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a840c1840 0x7f4a840a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.607+0000 7f4a82d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a840c1840 0x7f4a840a9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46360/0 (socket says 172.21.15.33:46360) 2023-11-06T22:18:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.607+0000 7f4a82d9d700 1 -- 172.21.15.33:0/3684658102 learned_addr learned my addr 172.21.15.33:0/3684658102 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:53.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.607+0000 7f4a8259c700 1 --2- 172.21.15.33:0/3684658102 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a840aa2d0 0x7f4a840c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:53.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.607+0000 7f4a82d9d700 1 -- 172.21.15.33:0/3684658102 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a840aa2d0 msgr2=0x7f4a840c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:53.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.607+0000 7f4a82d9d700 1 --2- 172.21.15.33:0/3684658102 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a840aa2d0 0x7f4a840c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.607+0000 7f4a82d9d700 1 -- 172.21.15.33:0/3684658102 >> v1:172.21.15.69:6789/0 conn(0x7f4a840c6240 legacy=0x7f4a84157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:53.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.607+0000 7f4a82d9d700 1 -- 172.21.15.33:0/3684658102 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840c66b0 con 0x7f4a840c1840 2023-11-06T22:18:53.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.607+0000 7f4a8259c700 1 --2- 172.21.15.33:0/3684658102 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a840aa2d0 0x7f4a840c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:53.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.607+0000 7f4a82d9d700 1 --2- 172.21.15.33:0/3684658102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840c1840 0x7f4a840a9de0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 rx=0x7f4a74006fd0 tx=0x7f4a740090f0).ready entity=mon.0 client_cookie=a75b1ba77f7f6dc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a81d9b700 1 -- 172.21.15.33:0/3684658102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400b600 con 0x7f4a840c1840 2023-11-06T22:18:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a81d9b700 1 -- 172.21.15.33:0/3684658102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7400b760 con 0x7f4a840c1840 2023-11-06T22:18:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a81d9b700 1 -- 172.21.15.33:0/3684658102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400bae0 con 0x7f4a840c1840 2023-11-06T22:18:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a891ff700 1 -- 172.21.15.33:0/3684658102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840c1840 msgr2=0x7f4a840a9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a891ff700 1 --2- 172.21.15.33:0/3684658102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840c1840 0x7f4a840a9de0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 rx=0x7f4a74006fd0 tx=0x7f4a740090f0).stop 2023-11-06T22:18:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a891ff700 1 -- 172.21.15.33:0/3684658102 shutdown_connections 2023-11-06T22:18:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a891ff700 1 --2- 172.21.15.33:0/3684658102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840c1840 0x7f4a840a9de0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a891ff700 1 --2- 172.21.15.33:0/3684658102 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a840aa2d0 0x7f4a840c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a891ff700 1 -- 172.21.15.33:0/3684658102 >> 172.21.15.33:0/3684658102 conn(0x7f4a8405ec80 msgr2=0x7f4a840cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a891ff700 1 -- 172.21.15.33:0/3684658102 shutdown_connections 2023-11-06T22:18:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.608+0000 7f4a891ff700 1 -- 172.21.15.33:0/3684658102 wait complete. 2023-11-06T22:18:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.610+0000 7f4a891ff700 1 Processor -- start 2023-11-06T22:18:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.610+0000 7f4a891ff700 1 -- start start 2023-11-06T22:18:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.611+0000 7f4a891ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840aa2d0 0x7f4a840c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.611+0000 7f4a891ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a840c1840 0x7f4a840b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.611+0000 7f4a891ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a840c6240 0x7f4a840b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.611+0000 7f4a891ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840b0da0 con 0x7f4a840aa2d0 2023-11-06T22:18:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.611+0000 7f4a891ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840c4280 con 0x7f4a840c1840 2023-11-06T22:18:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.611+0000 7f4a891ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840c4c00 con 0x7f4a840c6240 2023-11-06T22:18:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a82d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840aa2d0 0x7f4a840c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a82d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840aa2d0 0x7f4a840c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46368/0 (socket says 172.21.15.33:46368) 2023-11-06T22:18:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a82d9d700 1 -- 172.21.15.33:0/175097593 learned_addr learned my addr 172.21.15.33:0/175097593 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a82d9d700 1 -- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a840c6240 msgr2=0x7f4a840b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a82d9d700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a840c6240 0x7f4a840b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a8259c700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a840c1840 0x7f4a840b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a82d9d700 1 -- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a840c1840 msgr2=0x7f4a840b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a82d9d700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a840c1840 0x7f4a840b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a82d9d700 1 -- 172.21.15.33:0/175097593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a74009370 con 0x7f4a840aa2d0 2023-11-06T22:18:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a8259c700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a840c1840 0x7f4a840b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a82d9d700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840aa2d0 0x7f4a840c4710 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 rx=0x7f4a74009f50 tx=0x7f4a74017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a6affd700 1 -- 172.21.15.33:0/175097593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400bae0 con 0x7f4a840aa2d0 2023-11-06T22:18:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.612+0000 7f4a6affd700 1 -- 172.21.15.33:0/175097593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a74000c90 con 0x7f4a840aa2d0 2023-11-06T22:18:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.613+0000 7f4a891ff700 1 -- 172.21.15.33:0/175097593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a840c33d0 con 0x7f4a840aa2d0 2023-11-06T22:18:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.613+0000 7f4a6affd700 1 -- 172.21.15.33:0/175097593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7401c650 con 0x7f4a840aa2d0 2023-11-06T22:18:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.613+0000 7f4a891ff700 1 -- 172.21.15.33:0/175097593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a840c3950 con 0x7f4a840aa2d0 2023-11-06T22:18:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.614+0000 7f4a891ff700 1 -- 172.21.15.33:0/175097593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8406c6c0 con 0x7f4a840aa2d0 2023-11-06T22:18:53.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.615+0000 7f4a6affd700 1 -- 172.21.15.33:0/175097593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4a7401c9c0 con 0x7f4a840aa2d0 2023-11-06T22:18:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.615+0000 7f4a6affd700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4a7005f2c0 0x7f4a70061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.615+0000 7f4a6affd700 1 -- 172.21.15.33:0/175097593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f4a7407c1b0 con 0x7f4a840aa2d0 2023-11-06T22:18:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.616+0000 7f4a8259c700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4a7005f2c0 0x7f4a70061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.616+0000 7f4a8259c700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4a7005f2c0 0x7f4a70061780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f4a78006fd0 tx=0x7f4a78008040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.620+0000 7f4a6affd700 1 -- 172.21.15.33:0/175097593 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4a7404d4e0 con 0x7f4a840aa2d0 2023-11-06T22:18:53.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.794+0000 7f4a891ff700 1 -- 172.21.15.33:0/175097593 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a84063f20 con 0x7f4a7005f2c0 2023-11-06T22:18:53.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.798+0000 7f4a6affd700 1 -- 172.21.15.33:0/175097593 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70582 (secure 0 0 0) 0x7f4a84063f20 con 0x7f4a7005f2c0 2023-11-06T22:18:53.802 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.800+0000 7f4a891ff700 1 -- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4a7005f2c0 msgr2=0x7f4a70061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.800+0000 7f4a891ff700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4a7005f2c0 0x7f4a70061780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f4a78006fd0 tx=0x7f4a78008040).stop 2023-11-06T22:18:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.800+0000 7f4a891ff700 1 -- 172.21.15.33:0/175097593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840aa2d0 msgr2=0x7f4a840c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.800+0000 7f4a891ff700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840aa2d0 0x7f4a840c4710 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 rx=0x7f4a74009f50 tx=0x7f4a74017000).stop 2023-11-06T22:18:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.801+0000 7f4a891ff700 1 -- 172.21.15.33:0/175097593 shutdown_connections 2023-11-06T22:18:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.801+0000 7f4a891ff700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4a7005f2c0 0x7f4a70061780 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.801+0000 7f4a891ff700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4a840c6240 0x7f4a840b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-06T22:18:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.801+0000 7f4a891ff700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a840aa2d0 0x7f4a840c4710 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.801+0000 7f4a891ff700 1 --2- 172.21.15.33:0/175097593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a840c1840 0x7f4a840b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.801+0000 7f4a891ff700 1 -- 172.21.15.33:0/175097593 >> 172.21.15.33:0/175097593 conn(0x7f4a8405ec80 msgr2=0x7f4a840ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.801+0000 7f4a891ff700 1 -- 172.21.15.33:0/175097593 shutdown_connections 2023-11-06T22:18:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:53.801+0000 7f4a891ff700 1 -- 172.21.15.33:0/175097593 wait complete. 2023-11-06T22:18:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:18:53.821 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-11-06T22:18:53.550799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2464720,"kb_used_data":22416,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752510000,"statfs":{"total":773094113280,"available":770570240000,"internally_reserved":0,"allocated":22953984,"data_stored":11818770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.009297"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":195,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:49.073878+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T22:18:49.073878+0000","last_peered":"2023-11-06T22:18:49.073878+0000","last_clean":"2023-11-06T22:18:49.073878+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:18:49.073878+0000","last_undegraded":"2023-11-06T22:18:49.073878+0000","last_fullsized":"2023-11-06T22:18:49.073878+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.567718+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T22:18:48.567718+0000","last_peered":"2023-11-06T22:18:48.567718+0000","last_clean":"2023-11-06T22:18:48.567718+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:18:48.567718+0000","last_undegraded":"2023-11-06T22:18:48.567718+0000","last_fullsized":"2023-11-06T22:18:48.567718+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":217,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.565920+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T22:18:48.565920+0000","last_peered":"2023-11-06T22:18:48.565920+0000","last_clean":"2023-11-06T22:18:48.565920+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:18:48.565920+0000","last_undegraded":"2023-11-06T22:18:48.565920+0000","last_fullsized":"2023-11-06T22:18:48.565920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.568883+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T22:18:48.568883+0000","last_peered":"2023-11-06T22:18:48.568883+0000","last_clean":"2023-11-06T22:18:48.568883+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:18:48.568883+0000","last_undegraded":"2023-11-06T22:18:48.568883+0000","last_fullsized":"2023-11-06T22:18:48.568883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":194,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.565826+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T22:18:48.565826+0000","last_peered":"2023-11-06T22:18:48.565826+0000","last_clean":"2023-11-06T22:18:48.565826+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:18:48.565826+0000","last_undegraded":"2023-11-06T22:18:48.565826+0000","last_fullsized":"2023-11-06T22:18:48.565826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.566133+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T22:18:48.566133+0000","last_peered":"2023-11-06T22:18:48.566133+0000","last_clean":"2023-11-06T22:18:48.566133+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:18:48.566133+0000","last_undegraded":"2023-11-06T22:18:48.566133+0000","last_fullsized":"2023-11-06T22:18:48.566133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.567159+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:18:48.567159+0000","last_peered":"2023-11-06T22:18:48.567159+0000","last_clean":"2023-11-06T22:18:48.567159+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:18:48.567159+0000","last_undegraded":"2023-11-06T22:18:48.567159+0000","last_fullsized":"2023-11-06T22:18:48.567159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":194,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:49.069344+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T22:18:49.069344+0000","last_peered":"2023-11-06T22:18:49.069344+0000","last_clean":"2023-11-06T22:18:49.069344+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:18:49.069344+0000","last_undegraded":"2023-11-06T22:18:49.069344+0000","last_fullsized":"2023-11-06T22:18:49.069344+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.568057+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:18:48.568057+0000","last_peered":"2023-11-06T22:18:48.568057+0000","last_clean":"2023-11-06T22:18:48.568057+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:18:48.568057+0000","last_undegraded":"2023-11-06T22:18:48.568057+0000","last_fullsized":"2023-11-06T22:18:48.568057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575973,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.97199999999999998,"15min":1.0980000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.51200000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.94599999999999995}]},{"osd":1,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.871,"5min":0.98799999999999999,"15min":1.038},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":0.96599999999999997,"15min":1.0680000000000001},"last":0.80400000000000005}]},{"osd":2,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94799999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.032},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88600000000000001,"5min":0.996,"15min":1.0209999999999999},"last":0.81100000000000005}]},{"osd":3,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.499},"max":{"1min":0.88200000000000001,"5min":0.90700000000000003,"15min":0.95799999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.59199999999999997,"5min":0.54200000000000004,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.91500000000000004}]},{"osd":4,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.44900000000000001},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60599999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.436,"5min":0.432,"15min":0.40400000000000003},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.432},"max":{"1min":1.0329999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":1.004}]},{"osd":6,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.48399999999999999},"max":{"1min":1.0669999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.052},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.52800000000000002,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.069},"last":0.77700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68100000000000005,"15min":0.70999999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.75},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":1.0469999999999999,"15min":1.0660000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":1.016},"last":0.81000000000000005}]},{"osd":2,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0780000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.046},"last":0.88}]},{"osd":3,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.873,"5min":0.94799999999999995,"15min":1.002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.93799999999999994}]},{"osd":4,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.53600000000000003,"5min":0.42299999999999999,"15min":0.41699999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.98299999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76900000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.67100000000000004},"min":{"1min":0.44,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83499999999999996,"5min":0.98499999999999999,"15min":0.98999999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.442},"max":{"1min":0.80000000000000004,"5min":0.996,"15min":1.056},"last":0.85199999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79800000000000004,"5min":1.012,"15min":1.0429999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0569999999999999,"15min":1.1279999999999999},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575973,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.97999999999999998},"last":0.311},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.33800000000000002},"max":{"1min":0.66700000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.40400000000000003}]},{"osd":2,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.307,"15min":0.307},"max":{"1min":0.67400000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.95699999999999996,"15min":1.0189999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74099999999999999,"5min":0.76200000000000001,"15min":1.0509999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":0.92700000000000005},"last":0.437}]},{"osd":4,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80900000000000005,"15min":0.80800000000000005},"min":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.45200000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.753},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84899999999999998,"15min":0.85199999999999998},"min":{"1min":0.68300000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.73299999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.81599999999999995},"min":{"1min":0.58899999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.052,"5min":1.2210000000000001,"15min":1.2709999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81499999999999995,"15min":0.80200000000000005},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.55400000000000005},"max":{"1min":1.224,"5min":1.224,"15min":1.3109999999999999},"last":0.57999999999999996}]},{"osd":6,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.3149999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.84399999999999997,"15min":0.84899999999999998},"min":{"1min":0.64900000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1459999999999999,"5min":1.21,"15min":1.21},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1970000000000001,"15min":1.2509999999999999},"last":0.72099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47499999999999998,"15min":0.46899999999999997},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.499}]},{"osd":2,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.48499999999999999},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.435}]},{"osd":3,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.375},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.85199999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.54800000000000004},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.84199999999999997,"15min":0.85399999999999998},"min":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81699999999999995,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.52600000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83599999999999997,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.71499999999999997}]},{"osd":7,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82999999999999996,"15min":0.85699999999999998},"min":{"1min":0.55200000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.268},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.85499999999999998,"15min":0.876},"min":{"1min":0.58299999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.67600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575973,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.629,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70499999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.34100000000000003}]},{"osd":1,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.77400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.48799999999999999,"5min":0.46000000000000002,"15min":0.45100000000000001},"max":{"1min":0.79300000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.439}]},{"osd":3,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.40999999999999998,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.74299999999999999,"5min":0.74399999999999999,"15min":0.78300000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.42499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72299999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.82199999999999995,"15min":0.82699999999999996},"min":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.53900000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.349},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2609999999999999,"5min":1.2669999999999999,"15min":1.339},"last":0.56200000000000006}]},{"osd":5,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81200000000000006,"15min":0.81699999999999995},"min":{"1min":0.54600000000000004,"5min":0.54300000000000004,"15min":0.48999999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.337},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84199999999999997,"15min":0.85499999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1899999999999999,"5min":1.212,"15min":1.468},"last":0.56999999999999995}]},{"osd":6,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.83099999999999996,"15min":0.84899999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.54600000000000004},"max":{"1min":1.147,"5min":1.155,"15min":1.387},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.84499999999999997,"15min":0.85099999999999998},"min":{"1min":0.627,"5min":0.56499999999999995,"15min":0.56100000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.4299999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.86599999999999999,"15min":0.876},"min":{"1min":0.61199999999999999,"5min":0.58599999999999997,"15min":0.55700000000000005},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.496},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85999999999999999,"15min":0.876},"min":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1459999999999999,"5min":1.2470000000000001,"15min":1.508},"last":0.65300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575974,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45700000000000002,"15min":0.47499999999999998},"min":{"1min":0.28399999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.31},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.495}]},{"osd":1,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68700000000000006,"5min":1.05,"15min":1.05},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.57199999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70499999999999996,"5min":0.873,"15min":0.873},"last":0.58599999999999997}]},{"osd":2,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54400000000000004,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.87},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.69199999999999995}]},{"osd":4,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0669999999999999,"5min":1.161,"15min":1.2370000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.82899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1200000000000001,"5min":1.147,"15min":1.226},"last":1.0389999999999999}]},{"osd":5,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77700000000000002,"15min":0.80700000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0449999999999999,"5min":1.071,"15min":1.2270000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.3140000000000001},"last":0.95199999999999996}]},{"osd":6,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.82399999999999995},"min":{"1min":0.52700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0429999999999999,"5min":1.149,"15min":1.2290000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81200000000000006,"15min":0.85199999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.099,"5min":1.099,"15min":1.1879999999999999},"last":0.97399999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.196,"5min":1.196,"15min":1.2829999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.81799999999999995,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.113,"5min":1.169,"15min":1.248},"last":1.022}]}]},{"osd":4,"up_from":13,"seq":55834575973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308076,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063764,"statfs":{"total":96636764160,"available":96321294336,"internally_reserved":0,"allocated":2854912,"data_stored":1471671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70499999999999996,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98999999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68100000000000005,"15min":0.71099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82699999999999996,"5min":0.95799999999999996,"15min":1.006},"last":0.79300000000000004}]},{"osd":1,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70299999999999996,"15min":0.72699999999999998},"min":{"1min":0.53700000000000003,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92500000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.748}]},{"osd":2,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.98599999999999999,"15min":1.087},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.48299999999999998},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.071},"last":0.90300000000000002}]},{"osd":3,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":1.0329999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90400000000000003,"5min":1.236,"15min":1.236},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.48399999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88700000000000001,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90600000000000003,"5min":1.018,"15min":1.018},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.505,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0369999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.53200000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.54400000000000004,"5min":0.51500000000000001,"15min":0.50800000000000001},"max":{"1min":1.0469999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.77100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575974,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.309},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1639999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.70099999999999996}]},{"osd":1,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77800000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47999999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.95999999999999996}]},{"osd":2,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78700000000000003,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.48299999999999998},"max":{"1min":0.96499999999999997,"5min":4.9779999999999998,"15min":4.9779999999999998},"last":0.63100000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.36499999999999999},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.752,"15min":0.747},"min":{"1min":0.499,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.2390000000000001,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":0.92300000000000004}]},{"osd":4,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.38800000000000001},"max":{"1min":1.4019999999999999,"5min":1.585,"15min":1.585},"last":0.86299999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":1.4510000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77400000000000002,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.441,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.41399999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78200000000000003,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":1.4810000000000001,"5min":6.891,"15min":6.891},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:18:53.823 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-06T22:18:53.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:18:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.017+0000 7f4730dff700 1 Processor -- start 2023-11-06T22:18:54.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.017+0000 7f4730dff700 1 -- start start 2023-11-06T22:18:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.018+0000 7f4730dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f472c0c3600 0x7f472c0c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.018+0000 7f4730dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f472c069410 0x7f472c069860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.018+0000 7f4730dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472c0c17e0 con 0x7f472c136f50 2023-11-06T22:18:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.018+0000 7f4730dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472c06a740 con 0x7f472c0c3600 2023-11-06T22:18:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.018+0000 7f4730dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472c06a180 con 0x7f472c069410 2023-11-06T22:18:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.019+0000 7f472a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f472c0c3600 0x7f472c0c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.019+0000 7f472a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f472c0c3600 0x7f472c0c39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46370/0 (socket says 172.21.15.33:46370) 2023-11-06T22:18:54.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.019+0000 7f472a59c700 1 -- 172.21.15.33:0/562150574 learned_addr learned my addr 172.21.15.33:0/562150574 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.019+0000 7f472ad9d700 1 --2- 172.21.15.33:0/562150574 >> v2:172.21.15.69:3300/0 conn(0x7f472c069410 0x7f472c069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.019+0000 7f472a59c700 1 -- 172.21.15.33:0/562150574 >> v2:172.21.15.69:3300/0 conn(0x7f472c069410 msgr2=0x7f472c069860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.019+0000 7f472a59c700 1 --2- 172.21.15.33:0/562150574 >> v2:172.21.15.69:3300/0 conn(0x7f472c069410 0x7f472c069860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.019+0000 7f472a59c700 1 -- 172.21.15.33:0/562150574 >> v1:172.21.15.69:6789/0 conn(0x7f472c136f50 legacy=0x7f472c137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:54.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.019+0000 7f472a59c700 1 -- 172.21.15.33:0/562150574 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c06a460 con 0x7f472c0c3600 2023-11-06T22:18:54.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.019+0000 7f472ad9d700 1 --2- 172.21.15.33:0/562150574 >> v2:172.21.15.69:3300/0 conn(0x7f472c069410 0x7f472c069860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:54.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.019+0000 7f472a59c700 1 --2- 172.21.15.33:0/562150574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c0c3600 0x7f472c0c39e0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 rx=0x7f472c137b40 tx=0x7f471c0090f0).ready entity=mon.0 client_cookie=67dffb72a46bf964 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:54.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f472959a700 1 -- 172.21.15.33:0/562150574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c00b5a0 con 0x7f472c0c3600 2023-11-06T22:18:54.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f472959a700 1 -- 172.21.15.33:0/562150574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c00b700 con 0x7f472c0c3600 2023-11-06T22:18:54.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f472959a700 1 -- 172.21.15.33:0/562150574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c00ba80 con 0x7f472c0c3600 2023-11-06T22:18:54.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f4730dff700 1 -- 172.21.15.33:0/562150574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c0c3600 msgr2=0x7f472c0c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f4730dff700 1 --2- 172.21.15.33:0/562150574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c0c3600 0x7f472c0c39e0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 rx=0x7f472c137b40 tx=0x7f471c0090f0).stop 2023-11-06T22:18:54.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f4730dff700 1 -- 172.21.15.33:0/562150574 shutdown_connections 2023-11-06T22:18:54.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f4730dff700 1 --2- 172.21.15.33:0/562150574 >> v2:172.21.15.69:3300/0 conn(0x7f472c069410 0x7f472c069860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f4730dff700 1 --2- 172.21.15.33:0/562150574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c0c3600 0x7f472c0c39e0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f4730dff700 1 -- 172.21.15.33:0/562150574 >> 172.21.15.33:0/562150574 conn(0x7f472c062710 msgr2=0x7f472c062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:54.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f4730dff700 1 -- 172.21.15.33:0/562150574 shutdown_connections 2023-11-06T22:18:54.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.020+0000 7f4730dff700 1 -- 172.21.15.33:0/562150574 wait complete. 2023-11-06T22:18:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.022+0000 7f4730dff700 1 Processor -- start 2023-11-06T22:18:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.022+0000 7f4730dff700 1 -- start start 2023-11-06T22:18:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f4730dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f472c069410 0x7f472c1310b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f4730dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c0c3600 0x7f472c164620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f4730dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c136f50 0x7f472c0c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f4730dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c165550 con 0x7f472c136f50 2023-11-06T22:18:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f4730dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c164f90 con 0x7f472c0c3600 2023-11-06T22:18:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f4730dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472c165270 con 0x7f472c069410 2023-11-06T22:18:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f472ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c136f50 0x7f472c0c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f472ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c136f50 0x7f472c0c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46386/0 (socket says 172.21.15.33:46386) 2023-11-06T22:18:54.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f472ad9d700 1 -- 172.21.15.33:0/119745702 learned_addr learned my addr 172.21.15.33:0/119745702 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:54.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f4729d9b700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c0c3600 0x7f472c164620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.023+0000 7f472a59c700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f472c069410 0x7f472c1310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f472ad9d700 1 -- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f472c069410 msgr2=0x7f472c1310b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f472ad9d700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f472c069410 0x7f472c1310b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f472ad9d700 1 -- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c0c3600 msgr2=0x7f472c164620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f472ad9d700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c0c3600 0x7f472c164620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f472ad9d700 1 -- 172.21.15.33:0/119745702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c009370 con 0x7f472c136f50 2023-11-06T22:18:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f472ad9d700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c136f50 0x7f472c0c5390 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 rx=0x7f472400a380 tx=0x7f4724007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f471a7fc700 1 -- 172.21.15.33:0/119745702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724016470 con 0x7f472c136f50 2023-11-06T22:18:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f471a7fc700 1 -- 172.21.15.33:0/119745702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47240165d0 con 0x7f472c136f50 2023-11-06T22:18:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f4730dff700 1 -- 172.21.15.33:0/119745702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c0bde10 con 0x7f472c136f50 2023-11-06T22:18:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f471a7fc700 1 -- 172.21.15.33:0/119745702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724016970 con 0x7f472c136f50 2023-11-06T22:18:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.024+0000 7f4730dff700 1 -- 172.21.15.33:0/119745702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472c1315a0 con 0x7f472c136f50 2023-11-06T22:18:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.025+0000 7f4730dff700 1 -- 172.21.15.33:0/119745702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472c0c1470 con 0x7f472c136f50 2023-11-06T22:18:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.026+0000 7f471a7fc700 1 -- 172.21.15.33:0/119745702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4724013070 con 0x7f472c136f50 2023-11-06T22:18:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.027+0000 7f471a7fc700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f470c05f2a0 0x7f470c061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.027+0000 7f471a7fc700 1 -- 172.21.15.33:0/119745702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f472407ab40 con 0x7f472c136f50 2023-11-06T22:18:54.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.027+0000 7f472a59c700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f470c05f2a0 0x7f470c061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.028+0000 7f472a59c700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f470c05f2a0 0x7f470c061760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f471c009130 tx=0x7f471c015000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:54.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.031+0000 7f471a7fc700 1 -- 172.21.15.33:0/119745702 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f472404bd30 con 0x7f472c136f50 2023-11-06T22:18:54.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.211+0000 7f4730dff700 1 -- 172.21.15.33:0/119745702 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f472c05f0b0 con 0x7f470c05f2a0 2023-11-06T22:18:54.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.214+0000 7f471a7fc700 1 -- 172.21.15.33:0/119745702 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70582 (secure 0 0 0) 0x7f472c05f0b0 con 0x7f470c05f2a0 2023-11-06T22:18:54.218 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:54.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 -- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f470c05f2a0 msgr2=0x7f470c061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f470c05f2a0 0x7f470c061760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f471c009130 tx=0x7f471c015000).stop 2023-11-06T22:18:54.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 -- 172.21.15.33:0/119745702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c136f50 msgr2=0x7f472c0c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c136f50 0x7f472c0c5390 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 rx=0x7f472400a380 tx=0x7f4724007900).stop 2023-11-06T22:18:54.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 -- 172.21.15.33:0/119745702 shutdown_connections 2023-11-06T22:18:54.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f470c05f2a0 0x7f470c061760 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f472c069410 0x7f472c1310b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f472c136f50 0x7f472c0c5390 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 --2- 172.21.15.33:0/119745702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c0c3600 0x7f472c164620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 -- 172.21.15.33:0/119745702 >> 172.21.15.33:0/119745702 conn(0x7f472c062710 msgr2=0x7f472c150560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:54.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 -- 172.21.15.33:0/119745702 shutdown_connections 2023-11-06T22:18:54.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.217+0000 7f4730dff700 1 -- 172.21.15.33:0/119745702 wait complete. 2023-11-06T22:18:54.223 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:18:54.237 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-11-06T22:18:53.550799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2464720,"kb_used_data":22416,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752510000,"statfs":{"total":773094113280,"available":770570240000,"internally_reserved":0,"allocated":22953984,"data_stored":11818770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.009297"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":195,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:49.073878+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T22:18:49.073878+0000","last_peered":"2023-11-06T22:18:49.073878+0000","last_clean":"2023-11-06T22:18:49.073878+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:18:49.073878+0000","last_undegraded":"2023-11-06T22:18:49.073878+0000","last_fullsized":"2023-11-06T22:18:49.073878+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.567718+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T22:18:48.567718+0000","last_peered":"2023-11-06T22:18:48.567718+0000","last_clean":"2023-11-06T22:18:48.567718+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:18:48.567718+0000","last_undegraded":"2023-11-06T22:18:48.567718+0000","last_fullsized":"2023-11-06T22:18:48.567718+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":217,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.565920+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T22:18:48.565920+0000","last_peered":"2023-11-06T22:18:48.565920+0000","last_clean":"2023-11-06T22:18:48.565920+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:18:48.565920+0000","last_undegraded":"2023-11-06T22:18:48.565920+0000","last_fullsized":"2023-11-06T22:18:48.565920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.568883+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T22:18:48.568883+0000","last_peered":"2023-11-06T22:18:48.568883+0000","last_clean":"2023-11-06T22:18:48.568883+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:18:48.568883+0000","last_undegraded":"2023-11-06T22:18:48.568883+0000","last_fullsized":"2023-11-06T22:18:48.568883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":194,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.565826+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T22:18:48.565826+0000","last_peered":"2023-11-06T22:18:48.565826+0000","last_clean":"2023-11-06T22:18:48.565826+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:18:48.565826+0000","last_undegraded":"2023-11-06T22:18:48.565826+0000","last_fullsized":"2023-11-06T22:18:48.565826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.566133+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T22:18:48.566133+0000","last_peered":"2023-11-06T22:18:48.566133+0000","last_clean":"2023-11-06T22:18:48.566133+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:18:48.566133+0000","last_undegraded":"2023-11-06T22:18:48.566133+0000","last_fullsized":"2023-11-06T22:18:48.566133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.567159+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:18:48.567159+0000","last_peered":"2023-11-06T22:18:48.567159+0000","last_clean":"2023-11-06T22:18:48.567159+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:18:48.567159+0000","last_undegraded":"2023-11-06T22:18:48.567159+0000","last_fullsized":"2023-11-06T22:18:48.567159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":194,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:49.069344+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T22:18:49.069344+0000","last_peered":"2023-11-06T22:18:49.069344+0000","last_clean":"2023-11-06T22:18:49.069344+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:18:49.069344+0000","last_undegraded":"2023-11-06T22:18:49.069344+0000","last_fullsized":"2023-11-06T22:18:49.069344+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.568057+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:18:48.568057+0000","last_peered":"2023-11-06T22:18:48.568057+0000","last_clean":"2023-11-06T22:18:48.568057+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:18:48.568057+0000","last_undegraded":"2023-11-06T22:18:48.568057+0000","last_fullsized":"2023-11-06T22:18:48.568057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575973,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.97199999999999998,"15min":1.0980000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.51200000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.94599999999999995}]},{"osd":1,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.871,"5min":0.98799999999999999,"15min":1.038},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":0.96599999999999997,"15min":1.0680000000000001},"last":0.80400000000000005}]},{"osd":2,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94799999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.032},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88600000000000001,"5min":0.996,"15min":1.0209999999999999},"last":0.81100000000000005}]},{"osd":3,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.499},"max":{"1min":0.88200000000000001,"5min":0.90700000000000003,"15min":0.95799999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.59199999999999997,"5min":0.54200000000000004,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.91500000000000004}]},{"osd":4,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.44900000000000001},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60599999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.436,"5min":0.432,"15min":0.40400000000000003},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.432},"max":{"1min":1.0329999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":1.004}]},{"osd":6,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.48399999999999999},"max":{"1min":1.0669999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.052},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.52800000000000002,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.069},"last":0.77700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68100000000000005,"15min":0.70999999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.75},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":1.0469999999999999,"15min":1.0660000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":1.016},"last":0.81000000000000005}]},{"osd":2,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0780000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.046},"last":0.88}]},{"osd":3,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.873,"5min":0.94799999999999995,"15min":1.002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.93799999999999994}]},{"osd":4,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.53600000000000003,"5min":0.42299999999999999,"15min":0.41699999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.98299999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76900000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.67100000000000004},"min":{"1min":0.44,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83499999999999996,"5min":0.98499999999999999,"15min":0.98999999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.442},"max":{"1min":0.80000000000000004,"5min":0.996,"15min":1.056},"last":0.85199999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79800000000000004,"5min":1.012,"15min":1.0429999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0569999999999999,"15min":1.1279999999999999},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575973,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.97999999999999998},"last":0.311},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.33800000000000002},"max":{"1min":0.66700000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.40400000000000003}]},{"osd":2,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.307,"15min":0.307},"max":{"1min":0.67400000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.95699999999999996,"15min":1.0189999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74099999999999999,"5min":0.76200000000000001,"15min":1.0509999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":0.92700000000000005},"last":0.437}]},{"osd":4,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80900000000000005,"15min":0.80800000000000005},"min":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.45200000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.753},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84899999999999998,"15min":0.85199999999999998},"min":{"1min":0.68300000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.73299999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.81599999999999995},"min":{"1min":0.58899999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.052,"5min":1.2210000000000001,"15min":1.2709999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81499999999999995,"15min":0.80200000000000005},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.55400000000000005},"max":{"1min":1.224,"5min":1.224,"15min":1.3109999999999999},"last":0.57999999999999996}]},{"osd":6,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.3149999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.84399999999999997,"15min":0.84899999999999998},"min":{"1min":0.64900000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1459999999999999,"5min":1.21,"15min":1.21},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1970000000000001,"15min":1.2509999999999999},"last":0.72099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47499999999999998,"15min":0.46899999999999997},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.499}]},{"osd":2,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.48499999999999999},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.435}]},{"osd":3,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.375},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.85199999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.54800000000000004},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.84199999999999997,"15min":0.85399999999999998},"min":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81699999999999995,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.52600000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83599999999999997,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.71499999999999997}]},{"osd":7,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82999999999999996,"15min":0.85699999999999998},"min":{"1min":0.55200000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.268},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.85499999999999998,"15min":0.876},"min":{"1min":0.58299999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.67600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575973,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.629,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70499999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.34100000000000003}]},{"osd":1,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.77400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.48799999999999999,"5min":0.46000000000000002,"15min":0.45100000000000001},"max":{"1min":0.79300000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.439}]},{"osd":3,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.40999999999999998,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.74299999999999999,"5min":0.74399999999999999,"15min":0.78300000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.42499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72299999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.82199999999999995,"15min":0.82699999999999996},"min":{"1min":0.64800000000000002,"5min":0.58599999999999997,"15min":0.53900000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.349},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2609999999999999,"5min":1.2669999999999999,"15min":1.339},"last":0.56200000000000006}]},{"osd":5,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.81200000000000006,"15min":0.81699999999999995},"min":{"1min":0.54600000000000004,"5min":0.54300000000000004,"15min":0.48999999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.337},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84199999999999997,"15min":0.85499999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1899999999999999,"5min":1.212,"15min":1.468},"last":0.56999999999999995}]},{"osd":6,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.83099999999999996,"15min":0.84899999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.54600000000000004},"max":{"1min":1.147,"5min":1.155,"15min":1.387},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.84499999999999997,"15min":0.85099999999999998},"min":{"1min":0.627,"5min":0.56499999999999995,"15min":0.56100000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.4299999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.86599999999999999,"15min":0.876},"min":{"1min":0.61199999999999999,"5min":0.58599999999999997,"15min":0.55700000000000005},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.496},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85999999999999999,"15min":0.876},"min":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1459999999999999,"5min":1.2470000000000001,"15min":1.508},"last":0.65300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575974,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45700000000000002,"15min":0.47499999999999998},"min":{"1min":0.28399999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.31},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.495}]},{"osd":1,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68700000000000006,"5min":1.05,"15min":1.05},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.57199999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70499999999999996,"5min":0.873,"15min":0.873},"last":0.58599999999999997}]},{"osd":2,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54400000000000004,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.87},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.69199999999999995}]},{"osd":4,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0669999999999999,"5min":1.161,"15min":1.2370000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.82899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1200000000000001,"5min":1.147,"15min":1.226},"last":1.0389999999999999}]},{"osd":5,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77700000000000002,"15min":0.80700000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0449999999999999,"5min":1.071,"15min":1.2270000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.3140000000000001},"last":0.95199999999999996}]},{"osd":6,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.82399999999999995},"min":{"1min":0.52700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0429999999999999,"5min":1.149,"15min":1.2290000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81200000000000006,"15min":0.85199999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.099,"5min":1.099,"15min":1.1879999999999999},"last":0.97399999999999998}]},{"osd":7,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.196,"5min":1.196,"15min":1.2829999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.81799999999999995,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.113,"5min":1.169,"15min":1.248},"last":1.022}]}]},{"osd":4,"up_from":13,"seq":55834575973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308076,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063764,"statfs":{"total":96636764160,"available":96321294336,"internally_reserved":0,"allocated":2854912,"data_stored":1471671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70499999999999996,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98999999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68100000000000005,"15min":0.71099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82699999999999996,"5min":0.95799999999999996,"15min":1.006},"last":0.79300000000000004}]},{"osd":1,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70299999999999996,"15min":0.72699999999999998},"min":{"1min":0.53700000000000003,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92500000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.748}]},{"osd":2,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.98599999999999999,"15min":1.087},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.48299999999999998},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.071},"last":0.90300000000000002}]},{"osd":3,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":1.0329999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90400000000000003,"5min":1.236,"15min":1.236},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.48399999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88700000000000001,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90600000000000003,"5min":1.018,"15min":1.018},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.505,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0369999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.53200000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.54400000000000004,"5min":0.51500000000000001,"15min":0.50800000000000001},"max":{"1min":1.0469999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.77100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575974,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.309},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1639999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.70099999999999996}]},{"osd":1,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77800000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47999999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.95999999999999996}]},{"osd":2,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78700000000000003,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.48299999999999998},"max":{"1min":0.96499999999999997,"5min":4.9779999999999998,"15min":4.9779999999999998},"last":0.63100000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.36499999999999999},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.752,"15min":0.747},"min":{"1min":0.499,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.2390000000000001,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":0.92300000000000004}]},{"osd":4,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.38800000000000001},"max":{"1min":1.4019999999999999,"5min":1.585,"15min":1.585},"last":0.86299999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":1.4510000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77400000000000002,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.441,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.41399999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78200000000000003,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":1.4810000000000001,"5min":6.891,"15min":6.891},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:18:54.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T22:18:54.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.435+0000 7f688cfff700 1 Processor -- start 2023-11-06T22:18:54.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.435+0000 7f688cfff700 1 -- start start 2023-11-06T22:18:54.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f688cfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6888063140 0x7f6888063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f688cfff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f68880c3680 0x7f6888154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f688cfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68880c31f0 con 0x7f688805e1b0 2023-11-06T22:18:54.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f688cfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688806a0f0 con 0x7f6888063140 2023-11-06T22:18:54.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f688cfff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68880c2920 con 0x7f68880c3680 2023-11-06T22:18:54.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f688659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6888063140 0x7f6888063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f6885d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f688805e1b0 0x7f68880c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33538/0 (socket says 172.21.15.33:33538) 2023-11-06T22:18:54.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f6885d9b700 1 -- 172.21.15.33:0/4272124883 learned_addr learned my addr 172.21.15.33:0/4272124883 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:54.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f6886d9d700 1 --2- 172.21.15.33:0/4272124883 >> v2:172.21.15.69:3300/0 conn(0x7f68880c3680 0x7f6888154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f688659c700 1 -- 172.21.15.33:0/4272124883 >> v2:172.21.15.69:3300/0 conn(0x7f68880c3680 msgr2=0x7f6888154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:54.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f688659c700 1 --2- 172.21.15.33:0/4272124883 >> v2:172.21.15.69:3300/0 conn(0x7f68880c3680 0x7f6888154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f688659c700 1 -- 172.21.15.33:0/4272124883 >> v1:172.21.15.33:6789/0 conn(0x7f688805e1b0 legacy=0x7f68880c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f688659c700 1 -- 172.21.15.33:0/4272124883 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688805e640 con 0x7f6888063140 2023-11-06T22:18:54.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.437+0000 7f6886d9d700 1 --2- 172.21.15.33:0/4272124883 >> v2:172.21.15.69:3300/0 conn(0x7f68880c3680 0x7f6888154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:54.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.438+0000 7f688659c700 1 --2- 172.21.15.33:0/4272124883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 0x7f6888063520 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7f68880c3160 tx=0x7f6878009060).ready entity=mon.0 client_cookie=3d7bfad8373763f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:54.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.438+0000 7f688559a700 1 -- 172.21.15.33:0/4272124883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687800b580 con 0x7f6888063140 2023-11-06T22:18:54.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.438+0000 7f688559a700 1 -- 172.21.15.33:0/4272124883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687800b6e0 con 0x7f6888063140 2023-11-06T22:18:54.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.438+0000 7f688559a700 1 -- 172.21.15.33:0/4272124883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687800ba60 con 0x7f6888063140 2023-11-06T22:18:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.438+0000 7f688cfff700 1 -- 172.21.15.33:0/4272124883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 msgr2=0x7f6888063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.438+0000 7f688cfff700 1 --2- 172.21.15.33:0/4272124883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 0x7f6888063520 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7f68880c3160 tx=0x7f6878009060).stop 2023-11-06T22:18:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.438+0000 7f688cfff700 1 -- 172.21.15.33:0/4272124883 shutdown_connections 2023-11-06T22:18:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.438+0000 7f688cfff700 1 --2- 172.21.15.33:0/4272124883 >> v2:172.21.15.69:3300/0 conn(0x7f68880c3680 0x7f6888154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.438+0000 7f688cfff700 1 --2- 172.21.15.33:0/4272124883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 0x7f6888063520 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.438+0000 7f688cfff700 1 -- 172.21.15.33:0/4272124883 >> 172.21.15.33:0/4272124883 conn(0x7f68880c7d60 msgr2=0x7f68880c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:54.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.439+0000 7f688cfff700 1 -- 172.21.15.33:0/4272124883 shutdown_connections 2023-11-06T22:18:54.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.439+0000 7f688cfff700 1 -- 172.21.15.33:0/4272124883 wait complete. 2023-11-06T22:18:54.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.440+0000 7f688cfff700 1 Processor -- start 2023-11-06T22:18:54.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.440+0000 7f688cfff700 1 -- start start 2023-11-06T22:18:54.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.441+0000 7f688cfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 0x7f6888156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.441+0000 7f688cfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68880c3680 0x7f6888151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.441+0000 7f688cfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6888151c20 0x7f6888164f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.441+0000 7f688cfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6888165c70 con 0x7f6888063140 2023-11-06T22:18:54.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.441+0000 7f688cfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881668f0 con 0x7f68880c3680 2023-11-06T22:18:54.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.441+0000 7f688cfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6888152080 con 0x7f6888151c20 2023-11-06T22:18:54.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.441+0000 7f688659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 0x7f6888156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.441+0000 7f688659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 0x7f6888156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46406/0 (socket says 172.21.15.33:46406) 2023-11-06T22:18:54.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.441+0000 7f688659c700 1 -- 172.21.15.33:0/735628688 learned_addr learned my addr 172.21.15.33:0/735628688 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:54.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.441+0000 7f6885d9b700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68880c3680 0x7f6888151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f6886d9d700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6888151c20 0x7f6888164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f688659c700 1 -- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6888151c20 msgr2=0x7f6888164f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f688659c700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6888151c20 0x7f6888164f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f688659c700 1 -- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68880c3680 msgr2=0x7f6888151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f688659c700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68880c3680 0x7f6888151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f688659c700 1 -- 172.21.15.33:0/735628688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68780092a0 con 0x7f6888063140 2023-11-06T22:18:54.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f6886d9d700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6888151c20 0x7f6888164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:54.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f688659c700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 0x7f6888156620 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 rx=0x7f688805ebe0 tx=0x7f6878017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:54.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f68767fc700 1 -- 172.21.15.33:0/735628688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687800ba60 con 0x7f6888063140 2023-11-06T22:18:54.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f68767fc700 1 -- 172.21.15.33:0/735628688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878000c90 con 0x7f6888063140 2023-11-06T22:18:54.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f68767fc700 1 -- 172.21.15.33:0/735628688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687801b660 con 0x7f6888063140 2023-11-06T22:18:54.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f688cfff700 1 -- 172.21.15.33:0/735628688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6888156b10 con 0x7f6888063140 2023-11-06T22:18:54.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.442+0000 7f688cfff700 1 -- 172.21.15.33:0/735628688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68881662a0 con 0x7f6888063140 2023-11-06T22:18:54.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.443+0000 7f688cfff700 1 -- 172.21.15.33:0/735628688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6888064420 con 0x7f6888063140 2023-11-06T22:18:54.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.444+0000 7f68767fc700 1 -- 172.21.15.33:0/735628688 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f687801b9d0 con 0x7f6888063140 2023-11-06T22:18:54.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.446+0000 7f68767fc700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f686805f270 0x7f6868061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:54.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.446+0000 7f68767fc700 1 -- 172.21.15.33:0/735628688 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f687807bfa0 con 0x7f6888063140 2023-11-06T22:18:54.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.446+0000 7f6885d9b700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f686805f270 0x7f6868061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:54.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.447+0000 7f6885d9b700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f686805f270 0x7f6868061730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f687c007fd0 tx=0x7f687c009040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:54.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.450+0000 7f68767fc700 1 -- 172.21.15.33:0/735628688 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f687804d150 con 0x7f6888063140 2023-11-06T22:18:54.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.633+0000 7f688cfff700 1 -- 172.21.15.33:0/735628688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6888069b80 con 0x7f6888063140 2023-11-06T22:18:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.634+0000 7f68767fc700 1 -- 172.21.15.33:0/735628688 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v229) v1 ==== 74+0+14648 (secure 0 0 0) 0x7f6878029020 con 0x7f6888063140 2023-11-06T22:18:54.637 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:18:54.637 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":229,"fsid":"097ce162-8fe3-4461-8445-f7551959ee06","created":"2023-11-06T20:43:46.974668+0000","modified":"2023-11-06T22:18:49.560229+0000","last_up_change":"2023-11-06T20:43:57.827451+0000","last_in_change":"2023-11-06T20:43:50.451599+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-06T20:43:57.893748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"227","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T20:43:59.610088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"04ae67d0-5222-4147-8dbe-c91a41c71ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":115395},{"type":"v1","addr":"172.21.15.33:6809","nonce":115395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":115395},{"type":"v1","addr":"172.21.15.33:6811","nonce":115395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":115395},{"type":"v1","addr":"172.21.15.33:6815","nonce":115395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":115395},{"type":"v1","addr":"172.21.15.33:6813","nonce":115395}]},"public_addr":"172.21.15.33:6809/115395","cluster_addr":"172.21.15.33:6811/115395","heartbeat_back_addr":"172.21.15.33:6815/115395","heartbeat_front_addr":"172.21.15.33:6813/115395","state":["exists","up"]},{"osd":1,"uuid":"6b36f155-ed6d-4dbd-86a9-82c284781ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":115394},{"type":"v1","addr":"172.21.15.33:6801","nonce":115394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":115394},{"type":"v1","addr":"172.21.15.33:6803","nonce":115394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":115394},{"type":"v1","addr":"172.21.15.33:6807","nonce":115394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":115394},{"type":"v1","addr":"172.21.15.33:6805","nonce":115394}]},"public_addr":"172.21.15.33:6801/115394","cluster_addr":"172.21.15.33:6803/115394","heartbeat_back_addr":"172.21.15.33:6807/115394","heartbeat_front_addr":"172.21.15.33:6805/115394","state":["exists","up"]},{"osd":2,"uuid":"e9f10394-ee41-4786-95c9-76d96c52df77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":115398},{"type":"v1","addr":"172.21.15.33:6820","nonce":115398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":115398},{"type":"v1","addr":"172.21.15.33:6824","nonce":115398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":115398},{"type":"v1","addr":"172.21.15.33:6831","nonce":115398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":115398},{"type":"v1","addr":"172.21.15.33:6827","nonce":115398}]},"public_addr":"172.21.15.33:6820/115398","cluster_addr":"172.21.15.33:6824/115398","heartbeat_back_addr":"172.21.15.33:6831/115398","heartbeat_front_addr":"172.21.15.33:6827/115398","state":["exists","up"]},{"osd":3,"uuid":"64d3860c-4915-4ff6-b485-ad289ed7126c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":115401},{"type":"v1","addr":"172.21.15.33:6817","nonce":115401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6819","nonce":115401},{"type":"v1","addr":"172.21.15.33:6821","nonce":115401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":115401},{"type":"v1","addr":"172.21.15.33:6830","nonce":115401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6823","nonce":115401},{"type":"v1","addr":"172.21.15.33:6825","nonce":115401}]},"public_addr":"172.21.15.33:6817/115401","cluster_addr":"172.21.15.33:6821/115401","heartbeat_back_addr":"172.21.15.33:6830/115401","heartbeat_front_addr":"172.21.15.33:6825/115401","state":["exists","up"]},{"osd":4,"uuid":"6abb58b5-ff91-4d81-90c2-6d27605dcdd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6801","nonce":115620},{"type":"v1","addr":"172.21.15.69:6803","nonce":115620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6805","nonce":115620},{"type":"v1","addr":"172.21.15.69:6807","nonce":115620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6813","nonce":115620},{"type":"v1","addr":"172.21.15.69:6815","nonce":115620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6809","nonce":115620},{"type":"v1","addr":"172.21.15.69:6811","nonce":115620}]},"public_addr":"172.21.15.69:6803/115620","cluster_addr":"172.21.15.69:6807/115620","heartbeat_back_addr":"172.21.15.69:6815/115620","heartbeat_front_addr":"172.21.15.69:6811/115620","state":["exists","up"]},{"osd":5,"uuid":"9291f261-440c-40b8-8ae2-45fbec712e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":115642},{"type":"v1","addr":"172.21.15.69:6817","nonce":115642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":115642},{"type":"v1","addr":"172.21.15.69:6819","nonce":115642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":115642},{"type":"v1","addr":"172.21.15.69:6826","nonce":115642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":115642},{"type":"v1","addr":"172.21.15.69:6822","nonce":115642}]},"public_addr":"172.21.15.69:6817/115642","cluster_addr":"172.21.15.69:6819/115642","heartbeat_back_addr":"172.21.15.69:6826/115642","heartbeat_front_addr":"172.21.15.69:6822/115642","state":["exists","up"]},{"osd":6,"uuid":"f37d1d81-3a4e-41bc-9b26-99744a422362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":115621},{"type":"v1","addr":"172.21.15.69:6802","nonce":115621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":115621},{"type":"v1","addr":"172.21.15.69:6806","nonce":115621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":115621},{"type":"v1","addr":"172.21.15.69:6814","nonce":115621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":115621},{"type":"v1","addr":"172.21.15.69:6810","nonce":115621}]},"public_addr":"172.21.15.69:6802/115621","cluster_addr":"172.21.15.69:6806/115621","heartbeat_back_addr":"172.21.15.69:6814/115621","heartbeat_front_addr":"172.21.15.69:6810/115621","state":["exists","up"]},{"osd":7,"uuid":"6f9b4dbb-2660-4430-8ad2-003531697dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6821","nonce":115651},{"type":"v1","addr":"172.21.15.69:6823","nonce":115651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6825","nonce":115651},{"type":"v1","addr":"172.21.15.69:6827","nonce":115651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":115651},{"type":"v1","addr":"172.21.15.69:6831","nonce":115651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":115651},{"type":"v1","addr":"172.21.15.69:6829","nonce":115651}]},"public_addr":"172.21.15.69:6823/115651","cluster_addr":"172.21.15.69:6827/115651","heartbeat_back_addr":"172.21.15.69:6831/115651","heartbeat_front_addr":"172.21.15.69:6829/115651","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.014325+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.016456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.089277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.126151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.028444+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.058074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:55.975422+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T20:43:56.082059+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6836/3921249578":"2023-11-07T22:18:45.998989+0000","172.21.15.69:6834/3921249578":"2023-11-07T22:18:45.998989+0000","172.21.15.69:0/1054757794":"2023-11-07T22:18:45.528113+0000","172.21.15.69:0/3848122667":"2023-11-07T22:18:45.528113+0000","172.21.15.69:0/3627333266":"2023-11-07T22:18:45.528113+0000","172.21.15.69:0/2379584039":"2023-11-07T22:18:45.528113+0000","172.21.15.69:0/856300318":"2023-11-07T22:18:45.528113+0000","172.21.15.69:0/672707063":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/2065981382":"2023-11-07T22:18:37.037458+0000","172.21.15.69:6833/129946":"2023-11-07T22:18:37.037458+0000","172.21.15.69:6832/129946":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/3481533320":"2023-11-07T22:18:37.037458+0000","172.21.15.69:0/2510047576":"2023-11-07T22:18:37.037458+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T22:18:54.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 -- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f686805f270 msgr2=0x7f6868061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f686805f270 0x7f6868061730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f687c007fd0 tx=0x7f687c009040).stop 2023-11-06T22:18:54.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 -- 172.21.15.33:0/735628688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 msgr2=0x7f6888156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:54.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 0x7f6888156620 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 rx=0x7f688805ebe0 tx=0x7f6878017000).stop 2023-11-06T22:18:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 -- 172.21.15.33:0/735628688 shutdown_connections 2023-11-06T22:18:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f686805f270 0x7f6868061730 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f6888151c20 0x7f6888164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6888063140 0x7f6888156620 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 --2- 172.21.15.33:0/735628688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68880c3680 0x7f6888151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 -- 172.21.15.33:0/735628688 >> 172.21.15.33:0/735628688 conn(0x7f68880c7d60 msgr2=0x7f688814f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 -- 172.21.15.33:0/735628688 shutdown_connections 2023-11-06T22:18:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:54.636+0000 7f688cfff700 1 -- 172.21.15.33:0/735628688 wait complete. 2023-11-06T22:18:55.657 INFO:tasks.ceph:Scrubbing osd.0 2023-11-06T22:18:55.658 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-06T22:18:55.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.866+0000 7fa456dff700 1 Processor -- start 2023-11-06T22:18:55.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.866+0000 7fa456dff700 1 -- start start 2023-11-06T22:18:55.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.868+0000 7fa456dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa450153ed0 0x7fa450158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:55.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.868+0000 7fa456dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa4500c8ee0 0x7fa4500ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:55.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.868+0000 7fa456dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4500c9f80 con 0x7fa4501542b0 2023-11-06T22:18:55.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.868+0000 7fa456dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4500c6000 con 0x7fa4500c8ee0 2023-11-06T22:18:55.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.868+0000 7fa456dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4500c9370 con 0x7fa450153ed0 2023-11-06T22:18:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.868+0000 7fa44ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa4500c8ee0 0x7fa4500ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.868+0000 7fa45539c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa4501542b0 0x7fa4500c76f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33544/0 (socket says 172.21.15.33:33544) 2023-11-06T22:18:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.868+0000 7fa45539c700 1 -- 172.21.15.33:0/4084850896 learned_addr learned my addr 172.21.15.33:0/4084850896 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.868+0000 7fa454b9b700 1 --2- 172.21.15.33:0/4084850896 >> v2:172.21.15.69:3300/0 conn(0x7fa450153ed0 0x7fa450158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:55.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.869+0000 7fa44f7fe700 1 -- 172.21.15.33:0/4084850896 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806905665 0 0) 0x7fa4500c9f80 con 0x7fa4501542b0 2023-11-06T22:18:55.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.869+0000 7fa44f7fe700 1 -- 172.21.15.33:0/4084850896 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4400033f0 con 0x7fa4501542b0 2023-11-06T22:18:55.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.869+0000 7fa44ffff700 1 -- 172.21.15.33:0/4084850896 >> v2:172.21.15.69:3300/0 conn(0x7fa450153ed0 msgr2=0x7fa450158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:55.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.869+0000 7fa44ffff700 1 --2- 172.21.15.33:0/4084850896 >> v2:172.21.15.69:3300/0 conn(0x7fa450153ed0 0x7fa450158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:55.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.869+0000 7fa44ffff700 1 -- 172.21.15.33:0/4084850896 >> v1:172.21.15.33:6789/0 conn(0x7fa4501542b0 legacy=0x7fa4500c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:55.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.869+0000 7fa44ffff700 1 -- 172.21.15.33:0/4084850896 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500c9650 con 0x7fa4500c8ee0 2023-11-06T22:18:55.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.869+0000 7fa44ffff700 1 --2- 172.21.15.33:0/4084850896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4500c8ee0 0x7fa4500ca410 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 rx=0x7fa45006bdb0 tx=0x7fa43c009060).ready entity=mon.0 client_cookie=344b4af3a51efdd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:55.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.869+0000 7fa44f7fe700 1 -- 172.21.15.33:0/4084850896 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c00d4a0 con 0x7fa4500c8ee0 2023-11-06T22:18:55.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.869+0000 7fa44f7fe700 1 -- 172.21.15.33:0/4084850896 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c00d600 con 0x7fa4500c8ee0 2023-11-06T22:18:55.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.869+0000 7fa44f7fe700 1 -- 172.21.15.33:0/4084850896 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c00d980 con 0x7fa4500c8ee0 2023-11-06T22:18:55.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.870+0000 7fa456dff700 1 -- 172.21.15.33:0/4084850896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4500c8ee0 msgr2=0x7fa4500ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:55.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.870+0000 7fa456dff700 1 --2- 172.21.15.33:0/4084850896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4500c8ee0 0x7fa4500ca410 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 rx=0x7fa45006bdb0 tx=0x7fa43c009060).stop 2023-11-06T22:18:55.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.870+0000 7fa456dff700 1 -- 172.21.15.33:0/4084850896 shutdown_connections 2023-11-06T22:18:55.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.870+0000 7fa456dff700 1 --2- 172.21.15.33:0/4084850896 >> v2:172.21.15.69:3300/0 conn(0x7fa450153ed0 0x7fa450158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:55.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.870+0000 7fa456dff700 1 --2- 172.21.15.33:0/4084850896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4500c8ee0 0x7fa4500ca410 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:55.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.870+0000 7fa456dff700 1 -- 172.21.15.33:0/4084850896 >> 172.21.15.33:0/4084850896 conn(0x7fa45005e1b0 msgr2=0x7fa45005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:55.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.870+0000 7fa456dff700 1 -- 172.21.15.33:0/4084850896 shutdown_connections 2023-11-06T22:18:55.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.870+0000 7fa456dff700 1 -- 172.21.15.33:0/4084850896 wait complete. 2023-11-06T22:18:55.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.871+0000 7fa456dff700 1 Processor -- start 2023-11-06T22:18:55.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.871+0000 7fa456dff700 1 -- start start 2023-11-06T22:18:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.872+0000 7fa456dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa450153ed0 0x7fa4500af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.872+0000 7fa456dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4501542b0 0x7fa4500afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.872+0000 7fa456dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4500addd0 0x7fa45014fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:55.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.872+0000 7fa456dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500aee70 con 0x7fa4501542b0 2023-11-06T22:18:55.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.872+0000 7fa456dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45014e080 con 0x7fa4500addd0 2023-11-06T22:18:55.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.872+0000 7fa456dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500ae260 con 0x7fa450153ed0 2023-11-06T22:18:55.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4501542b0 0x7fa4500afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:55.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4501542b0 0x7fa4500afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46432/0 (socket says 172.21.15.33:46432) 2023-11-06T22:18:55.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44ffff700 1 -- 172.21.15.33:0/1888796747 learned_addr learned my addr 172.21.15.33:0/1888796747 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:55.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa45539c700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4500addd0 0x7fa45014fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:55.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa454b9b700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa450153ed0 0x7fa4500af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:55.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44ffff700 1 -- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa450153ed0 msgr2=0x7fa4500af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:55.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44ffff700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa450153ed0 0x7fa4500af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:55.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44ffff700 1 -- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4500addd0 msgr2=0x7fa45014fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:55.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44ffff700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4500addd0 0x7fa45014fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:55.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44ffff700 1 -- 172.21.15.33:0/1888796747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c0092a0 con 0x7fa4501542b0 2023-11-06T22:18:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa45539c700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4500addd0 0x7fa45014fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa454b9b700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fa450153ed0 0x7fa4500af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44ffff700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4501542b0 0x7fa4500afb10 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 rx=0x7fa43c009d20 tx=0x7fa43c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44cff9700 1 -- 172.21.15.33:0/1888796747 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c021070 con 0x7fa4501542b0 2023-11-06T22:18:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44cff9700 1 -- 172.21.15.33:0/1888796747 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c000c90 con 0x7fa4501542b0 2023-11-06T22:18:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.873+0000 7fa44cff9700 1 -- 172.21.15.33:0/1888796747 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c00d9a0 con 0x7fa4501542b0 2023-11-06T22:18:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.874+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4500ae540 con 0x7fa4501542b0 2023-11-06T22:18:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.874+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4500aeb00 con 0x7fa4501542b0 2023-11-06T22:18:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.874+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa45006c6c0 con 0x7fa4501542b0 2023-11-06T22:18:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.875+0000 7fa44cff9700 1 -- 172.21.15.33:0/1888796747 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fa43c01d640 con 0x7fa4501542b0 2023-11-06T22:18:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.877+0000 7fa44cff9700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa43405f270 0x7fa434061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.877+0000 7fa44cff9700 1 -- 172.21.15.33:0/1888796747 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fa43c07c080 con 0x7fa4501542b0 2023-11-06T22:18:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.877+0000 7fa44cff9700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] conn(0x7fa434065c20 0x7fa434068080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.877+0000 7fa44cff9700 1 -- 172.21.15.33:0/1888796747 --> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa434068700 con 0x7fa434065c20 2023-11-06T22:18:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.877+0000 7fa44cff9700 1 -- 172.21.15.33:0/1888796747 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa43c07c4a0 con 0x7fa4501542b0 2023-11-06T22:18:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.877+0000 7fa454b9b700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa43405f270 0x7fa434061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.877+0000 7fa45539c700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] conn(0x7fa434065c20 0x7fa434068080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.878+0000 7fa45539c700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] conn(0x7fa434065c20 0x7fa434068080 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.878+0000 7fa454b9b700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa43405f270 0x7fa434061730 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa444006d80 tx=0x7fa444008040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.878+0000 7fa44cff9700 1 -- 172.21.15.33:0/1888796747 <== osd.0 v2:172.21.15.33:6808/115395 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fa434068700 con 0x7fa434065c20 2023-11-06T22:18:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.892+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 --> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa4500aefb0 con 0x7fa434065c20 2023-11-06T22:18:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa44cff9700 1 -- 172.21.15.33:0/1888796747 <== osd.0 v2:172.21.15.33:6808/115395 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa4500aefb0 con 0x7fa434065c20 2023-11-06T22:18:55.896 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:18:55.896 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T22:18:55.896 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:18:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 >> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] conn(0x7fa434065c20 msgr2=0x7fa434068080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa456dff700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.33:6808/115395,v1:172.21.15.33:6809/115395] conn(0x7fa434065c20 0x7fa434068080 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa43405f270 msgr2=0x7fa434061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa456dff700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fa43405f270 0x7fa434061730 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa444006d80 tx=0x7fa444008040).stop 2023-11-06T22:18:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4501542b0 msgr2=0x7fa4500afb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa456dff700 1 --2- 172.21.15.33:0/1888796747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4501542b0 0x7fa4500afb10 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 rx=0x7fa43c009d20 tx=0x7fa43c016000).stop 2023-11-06T22:18:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa45539c700 1 -- 172.21.15.33:0/1888796747 reap_dead start 2023-11-06T22:18:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 shutdown_connections 2023-11-06T22:18:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 >> 172.21.15.33:0/1888796747 conn(0x7fa45005e1b0 msgr2=0x7fa45014f400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 shutdown_connections 2023-11-06T22:18:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:55.893+0000 7fa456dff700 1 -- 172.21.15.33:0/1888796747 wait complete. 2023-11-06T22:18:55.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-06T22:18:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.106+0000 7f454cdff700 1 Processor -- start 2023-11-06T22:18:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.106+0000 7f454cdff700 1 -- start start 2023-11-06T22:18:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.108+0000 7f454cdff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f45480636d0 0x7f4548158960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.108+0000 7f454cdff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f454814e500 0x7f454814e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.108+0000 7f454cdff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45481584d0 con 0x7f45480632f0 2023-11-06T22:18:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.108+0000 7f454cdff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4548063b10 con 0x7f454814e500 2023-11-06T22:18:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.108+0000 7f454cdff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45481559c0 con 0x7f45480636d0 2023-11-06T22:18:56.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.108+0000 7f4546d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f454814e500 0x7f454814e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.108+0000 7f4546d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f454814e500 0x7f454814e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46446/0 (socket says 172.21.15.33:46446) 2023-11-06T22:18:56.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.108+0000 7f4546d9d700 1 -- 172.21.15.33:0/2387331238 learned_addr learned my addr 172.21.15.33:0/2387331238 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.109+0000 7f4545d9b700 1 --2- 172.21.15.33:0/2387331238 >> v2:172.21.15.69:3300/0 conn(0x7f45480636d0 0x7f4548158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.109+0000 7f4546d9d700 1 -- 172.21.15.33:0/2387331238 >> v2:172.21.15.69:3300/0 conn(0x7f45480636d0 msgr2=0x7f4548158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.109+0000 7f4546d9d700 1 --2- 172.21.15.33:0/2387331238 >> v2:172.21.15.69:3300/0 conn(0x7f45480636d0 0x7f4548158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.109+0000 7f4546d9d700 1 -- 172.21.15.33:0/2387331238 >> v1:172.21.15.69:6789/0 conn(0x7f45480632f0 legacy=0x7f45481549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:56.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.109+0000 7f4546d9d700 1 -- 172.21.15.33:0/2387331238 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4548158e50 con 0x7f454814e500 2023-11-06T22:18:56.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.109+0000 7f4545d9b700 1 --2- 172.21.15.33:0/2387331238 >> v2:172.21.15.69:3300/0 conn(0x7f45480636d0 0x7f4548158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:56.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.109+0000 7f4546d9d700 1 --2- 172.21.15.33:0/2387331238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f454814e500 0x7f454814e970 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7f45480c56e0 tx=0x7f4540009840).ready entity=mon.0 client_cookie=8819799ae099cef4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.109+0000 7f454559a700 1 -- 172.21.15.33:0/2387331238 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000e070 con 0x7f454814e500 2023-11-06T22:18:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.109+0000 7f454559a700 1 -- 172.21.15.33:0/2387331238 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454000ba40 con 0x7f454814e500 2023-11-06T22:18:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.109+0000 7f454559a700 1 -- 172.21.15.33:0/2387331238 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000bdc0 con 0x7f454814e500 2023-11-06T22:18:56.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.110+0000 7f454cdff700 1 -- 172.21.15.33:0/2387331238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f454814e500 msgr2=0x7f454814e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.110+0000 7f454cdff700 1 --2- 172.21.15.33:0/2387331238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f454814e500 0x7f454814e970 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7f45480c56e0 tx=0x7f4540009840).stop 2023-11-06T22:18:56.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.110+0000 7f454cdff700 1 -- 172.21.15.33:0/2387331238 shutdown_connections 2023-11-06T22:18:56.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.110+0000 7f454cdff700 1 --2- 172.21.15.33:0/2387331238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f454814e500 0x7f454814e970 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.110+0000 7f454cdff700 1 --2- 172.21.15.33:0/2387331238 >> v2:172.21.15.69:3300/0 conn(0x7f45480636d0 0x7f4548158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.110+0000 7f454cdff700 1 -- 172.21.15.33:0/2387331238 >> 172.21.15.33:0/2387331238 conn(0x7f45480cb910 msgr2=0x7f45480c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.110+0000 7f454cdff700 1 -- 172.21.15.33:0/2387331238 shutdown_connections 2023-11-06T22:18:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.110+0000 7f454cdff700 1 -- 172.21.15.33:0/2387331238 wait complete. 2023-11-06T22:18:56.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.111+0000 7f454cdff700 1 Processor -- start 2023-11-06T22:18:56.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.112+0000 7f454cdff700 1 -- start start 2023-11-06T22:18:56.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f454cdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45480632f0 0x7f45480ac340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f454cdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45480636d0 0x7f454814d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f454cdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f454814e500 0x7f45480af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f454cdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45480c2030 con 0x7f45480636d0 2023-11-06T22:18:56.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f454cdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45480ac830 con 0x7f45480632f0 2023-11-06T22:18:56.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f454cdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45480b03a0 con 0x7f454814e500 2023-11-06T22:18:56.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f4545d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45480636d0 0x7f454814d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f4545d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45480636d0 0x7f454814d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46456/0 (socket says 172.21.15.33:46456) 2023-11-06T22:18:56.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f4545d9b700 1 -- 172.21.15.33:0/2117462476 learned_addr learned my addr 172.21.15.33:0/2117462476 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:56.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f454659c700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45480632f0 0x7f45480ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f4546d9d700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f454814e500 0x7f45480af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f4545d9b700 1 -- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f454814e500 msgr2=0x7f45480af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f4545d9b700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f454814e500 0x7f45480af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f4545d9b700 1 -- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45480632f0 msgr2=0x7f45480ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f4545d9b700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45480632f0 0x7f45480ac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.113+0000 7f4545d9b700 1 -- 172.21.15.33:0/2117462476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4540009ae0 con 0x7f45480636d0 2023-11-06T22:18:56.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.114+0000 7f4545d9b700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45480636d0 0x7f454814d9f0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 rx=0x7f453c00c880 tx=0x7f453c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:56.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.114+0000 7f45367fc700 1 -- 172.21.15.33:0/2117462476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c010580 con 0x7f45480636d0 2023-11-06T22:18:56.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.114+0000 7f45367fc700 1 -- 172.21.15.33:0/2117462476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c0106e0 con 0x7f45480636d0 2023-11-06T22:18:56.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.114+0000 7f45367fc700 1 -- 172.21.15.33:0/2117462476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c010a60 con 0x7f45480636d0 2023-11-06T22:18:56.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.114+0000 7f454cdff700 1 -- 172.21.15.33:0/2117462476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45480b0680 con 0x7f45480636d0 2023-11-06T22:18:56.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.114+0000 7f454cdff700 1 -- 172.21.15.33:0/2117462476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45480c2280 con 0x7f45480636d0 2023-11-06T22:18:56.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.115+0000 7f454cdff700 1 -- 172.21.15.33:0/2117462476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454814ee60 con 0x7f45480636d0 2023-11-06T22:18:56.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.116+0000 7f45367fc700 1 -- 172.21.15.33:0/2117462476 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f453c01b460 con 0x7f45480636d0 2023-11-06T22:18:56.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.117+0000 7f45367fc700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f452805f270 0x7f4528061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.117+0000 7f45367fc700 1 -- 172.21.15.33:0/2117462476 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f453c07a280 con 0x7f45480636d0 2023-11-06T22:18:56.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.117+0000 7f454659c700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f452805f270 0x7f4528061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.118+0000 7f454659c700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f452805f270 0x7f4528061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f4538003900 tx=0x7f4538007d40).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:56.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.121+0000 7f45367fc700 1 -- 172.21.15.33:0/2117462476 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f453c04b3d0 con 0x7f45480636d0 2023-11-06T22:18:56.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.300+0000 7f454cdff700 1 -- 172.21.15.33:0/2117462476 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f454806ab40 con 0x7f452805f270 2023-11-06T22:18:56.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.301+0000 7f45367fc700 1 -- 172.21.15.33:0/2117462476 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f454806ab40 con 0x7f452805f270 2023-11-06T22:18:56.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.303+0000 7f454cdff700 1 -- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f452805f270 msgr2=0x7f4528061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.303+0000 7f454cdff700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f452805f270 0x7f4528061730 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f4538003900 tx=0x7f4538007d40).stop 2023-11-06T22:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.303+0000 7f454cdff700 1 -- 172.21.15.33:0/2117462476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45480636d0 msgr2=0x7f454814d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.303+0000 7f454cdff700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45480636d0 0x7f454814d9f0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 rx=0x7f453c00c880 tx=0x7f453c008040).stop 2023-11-06T22:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.303+0000 7f454cdff700 1 -- 172.21.15.33:0/2117462476 shutdown_connections 2023-11-06T22:18:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.303+0000 7f454cdff700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f452805f270 0x7f4528061730 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.303+0000 7f454cdff700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f454814e500 0x7f45480af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.304+0000 7f454cdff700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45480632f0 0x7f45480ac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.304+0000 7f454cdff700 1 --2- 172.21.15.33:0/2117462476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45480636d0 0x7f454814d9f0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.304+0000 7f454cdff700 1 -- 172.21.15.33:0/2117462476 >> 172.21.15.33:0/2117462476 conn(0x7f45480cb910 msgr2=0x7f4548157060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:56.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.304+0000 7f454cdff700 1 -- 172.21.15.33:0/2117462476 shutdown_connections 2023-11-06T22:18:56.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.304+0000 7f454cdff700 1 -- 172.21.15.33:0/2117462476 wait complete. 2023-11-06T22:18:56.310 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 0 to deep-scrub 2023-11-06T22:18:56.324 INFO:tasks.ceph:Scrubbing osd.1 2023-11-06T22:18:56.325 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-06T22:18:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.523+0000 7fb4567ff700 1 Processor -- start 2023-11-06T22:18:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.523+0000 7fb4567ff700 1 -- start start 2023-11-06T22:18:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.525+0000 7fb4567ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb450153ed0 0x7fb450158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.525+0000 7fb4567ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4500c8ee0 0x7fb4500ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.525+0000 7fb4567ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4500c9f80 con 0x7fb4501542b0 2023-11-06T22:18:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.525+0000 7fb4567ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4500c6000 con 0x7fb4500c8ee0 2023-11-06T22:18:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.525+0000 7fb4567ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4500c9370 con 0x7fb450153ed0 2023-11-06T22:18:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.525+0000 7fb44f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4500c8ee0 0x7fb4500ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.525+0000 7fb44f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4500c8ee0 0x7fb4500ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46466/0 (socket says 172.21.15.33:46466) 2023-11-06T22:18:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.525+0000 7fb44f7fe700 1 -- 172.21.15.33:0/3287785211 learned_addr learned my addr 172.21.15.33:0/3287785211 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.525+0000 7fb44ffff700 1 --2- 172.21.15.33:0/3287785211 >> v2:172.21.15.69:3300/0 conn(0x7fb450153ed0 0x7fb450158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.526+0000 7fb44f7fe700 1 -- 172.21.15.33:0/3287785211 >> v2:172.21.15.69:3300/0 conn(0x7fb450153ed0 msgr2=0x7fb450158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.526+0000 7fb44f7fe700 1 --2- 172.21.15.33:0/3287785211 >> v2:172.21.15.69:3300/0 conn(0x7fb450153ed0 0x7fb450158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.526+0000 7fb44f7fe700 1 -- 172.21.15.33:0/3287785211 >> v1:172.21.15.69:6789/0 conn(0x7fb4501542b0 legacy=0x7fb4500c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.526+0000 7fb44f7fe700 1 -- 172.21.15.33:0/3287785211 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4500c9650 con 0x7fb4500c8ee0 2023-11-06T22:18:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.526+0000 7fb44f7fe700 1 --2- 172.21.15.33:0/3287785211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4500c8ee0 0x7fb4500ca410 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7fb45006bdb0 tx=0x7fb4440090f0).ready entity=mon.0 client_cookie=4c46b429baa2a555 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.526+0000 7fb44effd700 1 -- 172.21.15.33:0/3287785211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44400b520 con 0x7fb4500c8ee0 2023-11-06T22:18:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.526+0000 7fb44effd700 1 -- 172.21.15.33:0/3287785211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44400b680 con 0x7fb4500c8ee0 2023-11-06T22:18:56.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.526+0000 7fb44effd700 1 -- 172.21.15.33:0/3287785211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44400ba00 con 0x7fb4500c8ee0 2023-11-06T22:18:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.527+0000 7fb4567ff700 1 -- 172.21.15.33:0/3287785211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4500c8ee0 msgr2=0x7fb4500ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.527+0000 7fb4567ff700 1 --2- 172.21.15.33:0/3287785211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4500c8ee0 0x7fb4500ca410 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7fb45006bdb0 tx=0x7fb4440090f0).stop 2023-11-06T22:18:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.527+0000 7fb4567ff700 1 -- 172.21.15.33:0/3287785211 shutdown_connections 2023-11-06T22:18:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.527+0000 7fb4567ff700 1 --2- 172.21.15.33:0/3287785211 >> v2:172.21.15.69:3300/0 conn(0x7fb450153ed0 0x7fb450158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.527+0000 7fb4567ff700 1 --2- 172.21.15.33:0/3287785211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4500c8ee0 0x7fb4500ca410 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.527+0000 7fb4567ff700 1 -- 172.21.15.33:0/3287785211 >> 172.21.15.33:0/3287785211 conn(0x7fb45005e1b0 msgr2=0x7fb45005e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.527+0000 7fb4567ff700 1 -- 172.21.15.33:0/3287785211 shutdown_connections 2023-11-06T22:18:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.527+0000 7fb4567ff700 1 -- 172.21.15.33:0/3287785211 wait complete. 2023-11-06T22:18:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.528+0000 7fb4567ff700 1 Processor -- start 2023-11-06T22:18:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.528+0000 7fb4567ff700 1 -- start start 2023-11-06T22:18:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.529+0000 7fb4567ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb450153ed0 0x7fb4500adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.529+0000 7fb4567ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4501542b0 0x7fb450152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.529+0000 7fb4567ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4500b0320 0x7fb45014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.529+0000 7fb4567ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4501530c0 con 0x7fb4500b0320 2023-11-06T22:18:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.529+0000 7fb4567ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4500b0b40 con 0x7fb4501542b0 2023-11-06T22:18:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.529+0000 7fb4567ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45014c420 con 0x7fb450153ed0 2023-11-06T22:18:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb454d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4500b0320 0x7fb45014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb44f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4501542b0 0x7fb450152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb44f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4501542b0 0x7fb450152bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.33:33756/0 (socket says 172.21.15.33:33756) 2023-11-06T22:18:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb44f7fe700 1 -- 172.21.15.33:0/3528957765 learned_addr learned my addr 172.21.15.33:0/3528957765 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb44ffff700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb450153ed0 0x7fb4500adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb454d9c700 1 -- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb450153ed0 msgr2=0x7fb4500adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb454d9c700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb450153ed0 0x7fb4500adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb454d9c700 1 -- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4501542b0 msgr2=0x7fb450152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb454d9c700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4501542b0 0x7fb450152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb454d9c700 1 -- 172.21.15.33:0/3528957765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb444009370 con 0x7fb4500b0320 2023-11-06T22:18:56.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb44f7fe700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4501542b0 0x7fb450152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:56.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.530+0000 7fb454d9c700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4500b0320 0x7fb45014fe10 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7fb4500c9f30 tx=0x7fb448001460).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:56.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.531+0000 7fb437fff700 1 -- 172.21.15.33:0/3528957765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800f470 con 0x7fb4500b0320 2023-11-06T22:18:56.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.531+0000 7fb437fff700 1 -- 172.21.15.33:0/3528957765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44800f5d0 con 0x7fb4500b0320 2023-11-06T22:18:56.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.531+0000 7fb437fff700 1 -- 172.21.15.33:0/3528957765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800f950 con 0x7fb4500b0320 2023-11-06T22:18:56.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.531+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45014c700 con 0x7fb4500b0320 2023-11-06T22:18:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.531+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45014ccc0 con 0x7fb4500b0320 2023-11-06T22:18:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.531+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb45006c6c0 con 0x7fb4500b0320 2023-11-06T22:18:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.533+0000 7fb437fff700 1 -- 172.21.15.33:0/3528957765 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb44800fd40 con 0x7fb4500b0320 2023-11-06T22:18:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.533+0000 7fb437fff700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb43c05f270 0x7fb43c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.533+0000 7fb437fff700 1 -- 172.21.15.33:0/3528957765 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fb44807a530 con 0x7fb4500b0320 2023-11-06T22:18:56.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.534+0000 7fb437fff700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] conn(0x7fb43c065c20 0x7fb43c068080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.534+0000 7fb437fff700 1 -- 172.21.15.33:0/3528957765 --> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb43c068700 con 0x7fb43c065c20 2023-11-06T22:18:56.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.534+0000 7fb437fff700 1 -- 172.21.15.33:0/3528957765 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb44807a900 con 0x7fb4500b0320 2023-11-06T22:18:56.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.534+0000 7fb44ffff700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb43c05f270 0x7fb43c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.534+0000 7fb44f7fe700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] conn(0x7fb43c065c20 0x7fb43c068080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.534+0000 7fb44ffff700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb43c05f270 0x7fb43c061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb440009c30 tx=0x7fb440009060).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:56.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.534+0000 7fb44f7fe700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] conn(0x7fb43c065c20 0x7fb43c068080 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:56.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.535+0000 7fb437fff700 1 -- 172.21.15.33:0/3528957765 <== osd.1 v2:172.21.15.33:6800/115394 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fb43c068700 con 0x7fb43c065c20 2023-11-06T22:18:56.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.550+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 --> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb4500c9930 con 0x7fb43c065c20 2023-11-06T22:18:56.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.550+0000 7fb437fff700 1 -- 172.21.15.33:0/3528957765 <== osd.1 v2:172.21.15.33:6800/115394 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb4500c9930 con 0x7fb43c065c20 2023-11-06T22:18:56.553 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:18:56.553 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T22:18:56.554 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:18:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 >> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] conn(0x7fb43c065c20 msgr2=0x7fb43c068080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb4567ff700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.33:6800/115394,v1:172.21.15.33:6801/115394] conn(0x7fb43c065c20 0x7fb43c068080 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb43c05f270 msgr2=0x7fb43c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb4567ff700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb43c05f270 0x7fb43c061730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb440009c30 tx=0x7fb440009060).stop 2023-11-06T22:18:56.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4500b0320 msgr2=0x7fb45014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb4567ff700 1 --2- 172.21.15.33:0/3528957765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4500b0320 0x7fb45014fe10 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7fb4500c9f30 tx=0x7fb448001460).stop 2023-11-06T22:18:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb454d9c700 1 -- 172.21.15.33:0/3528957765 reap_dead start 2023-11-06T22:18:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 shutdown_connections 2023-11-06T22:18:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 >> 172.21.15.33:0/3528957765 conn(0x7fb45005e1b0 msgr2=0x7fb4500ab600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 shutdown_connections 2023-11-06T22:18:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.551+0000 7fb4567ff700 1 -- 172.21.15.33:0/3528957765 wait complete. 2023-11-06T22:18:56.570 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-06T22:18:56.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.766+0000 7f2bc27ff700 1 Processor -- start 2023-11-06T22:18:56.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.766+0000 7f2bc27ff700 1 -- start start 2023-11-06T22:18:56.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.768+0000 7f2bc27ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2bbc0c6d90 0x7f2bbc0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.768+0000 7f2bc27ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc0c7700 con 0x7f2bbc154210 2023-11-06T22:18:56.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.768+0000 7f2bc27ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc05efc0 con 0x7f2bbc0bfd60 2023-11-06T22:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.768+0000 7f2bc27ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bbc0c8ea0 con 0x7f2bbc0c6d90 2023-11-06T22:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.768+0000 7f2bbbfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2bbc154210 0x7f2bbc0a7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33560/0 (socket says 172.21.15.33:33560) 2023-11-06T22:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.768+0000 7f2bbbfff700 1 -- 172.21.15.33:0/3048610291 learned_addr learned my addr 172.21.15.33:0/3048610291 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbb7fe700 1 --2- 172.21.15.33:0/3048610291 >> v2:172.21.15.69:3300/0 conn(0x7f2bbc0c6d90 0x7f2bbc0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535049169 0 0) 0x7f2bac001110 con 0x7f2bbc154210 2023-11-06T22:18:56.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba40033f0 con 0x7f2bbc154210 2023-11-06T22:18:56.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755555796 0 0) 0x7f2bbc05efc0 con 0x7f2bbc0bfd60 2023-11-06T22:18:56.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bac001110 con 0x7f2bbc0bfd60 2023-11-06T22:18:56.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2167296909 0 0) 0x7f2ba40033f0 con 0x7f2bbc154210 2023-11-06T22:18:56.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 >> v2:172.21.15.69:3300/0 conn(0x7f2bbc0c6d90 msgr2=0x7f2bbc0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:56.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 --2- 172.21.15.33:0/3048610291 >> v2:172.21.15.69:3300/0 conn(0x7f2bbc0c6d90 0x7f2bbc0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 >> v1:172.21.15.69:6789/0 conn(0x7f2bbc0bfd60 legacy=0x7f2bbc0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc0c9140 con 0x7f2bbc154210 2023-11-06T22:18:56.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7f2bac001600 con 0x7f2bbc154210 2023-11-06T22:18:56.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbb7fe700 1 --2- 172.21.15.33:0/3048610291 >> v2:172.21.15.69:3300/0 conn(0x7f2bbc0c6d90 0x7f2bbc0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:56.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 >> v1:172.21.15.33:6789/0 conn(0x7f2bbc154210 legacy=0x7f2bbc0a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2ba4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2ba4004690 0x7f2ba4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ba4005000 0x7f2ba400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0c8ea0 con 0x7f2ba4003a70 2023-11-06T22:18:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba400be80 con 0x7f2ba4005000 2023-11-06T22:18:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.769+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba400c040 con 0x7f2ba4004690 2023-11-06T22:18:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bbbfff700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2ba4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bc0d9c700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ba4005000 0x7f2ba400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bbb7fe700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2ba4004690 0x7f2ba4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bbbfff700 1 -- 172.21.15.33:0/3048610291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2ba4004690 msgr2=0x7f2ba4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bbbfff700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2ba4004690 0x7f2ba4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bbbfff700 1 -- 172.21.15.33:0/3048610291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ba4005000 msgr2=0x7f2ba400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bbbfff700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ba4005000 0x7f2ba400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bbbfff700 1 -- 172.21.15.33:0/3048610291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba40033f0 con 0x7f2ba4003a70 2023-11-06T22:18:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bc0d9c700 1 -- 172.21.15.33:0/3048610291 reap_dead start 2023-11-06T22:18:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bbb7fe700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2ba4004690 0x7f2ba4004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bbbfff700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2ba4003e70 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 rx=0x7f2bac00c2a0 tx=0x7f2bac007880).ready entity=mon.0 client_cookie=18033007127fd14e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.770+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac007b70 con 0x7f2ba4003a70 2023-11-06T22:18:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.771+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac00cbd0 con 0x7f2ba4003a70 2023-11-06T22:18:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.771+0000 7f2bbaffd700 1 -- 172.21.15.33:0/3048610291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0070d0 con 0x7f2ba4003a70 2023-11-06T22:18:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.771+0000 7f2bc27ff700 1 -- 172.21.15.33:0/3048610291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 msgr2=0x7f2ba4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.771+0000 7f2bc27ff700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2ba4003e70 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 rx=0x7f2bac00c2a0 tx=0x7f2bac007880).stop 2023-11-06T22:18:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.771+0000 7f2bc27ff700 1 -- 172.21.15.33:0/3048610291 shutdown_connections 2023-11-06T22:18:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.771+0000 7f2bc27ff700 1 --2- 172.21.15.33:0/3048610291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2ba4003e70 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.771+0000 7f2bc27ff700 1 -- 172.21.15.33:0/3048610291 >> 172.21.15.33:0/3048610291 conn(0x7f2bbc05e990 msgr2=0x7f2bbc0cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.771+0000 7f2bc27ff700 1 -- 172.21.15.33:0/3048610291 shutdown_connections 2023-11-06T22:18:56.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.771+0000 7f2bc27ff700 1 -- 172.21.15.33:0/3048610291 wait complete. 2023-11-06T22:18:56.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.773+0000 7f2bc27ff700 1 Processor -- start 2023-11-06T22:18:56.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.773+0000 7f2bc27ff700 1 -- start start 2023-11-06T22:18:56.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bc27ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2bbc14b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bc27ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bbc0a9450 0x7f2bbc14d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bc27ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bbc14b650 0x7f2bbc14d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bc27ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc154690 con 0x7f2ba4003a70 2023-11-06T22:18:56.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bc27ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc14bb90 con 0x7f2bbc14b650 2023-11-06T22:18:56.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bc27ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc154900 con 0x7f2bbc0a9450 2023-11-06T22:18:56.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bbbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2bbc14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bbbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2bbc14b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46498/0 (socket says 172.21.15.33:46498) 2023-11-06T22:18:56.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bbbfff700 1 -- 172.21.15.33:0/1384089884 learned_addr learned my addr 172.21.15.33:0/1384089884 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:56.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bbb7fe700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bbc0a9450 0x7f2bbc14d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.774+0000 7f2bc0d9c700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bbc14b650 0x7f2bbc14d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2bbbfff700 1 -- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bbc0a9450 msgr2=0x7f2bbc14d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2bbbfff700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bbc0a9450 0x7f2bbc14d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2bbbfff700 1 -- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bbc14b650 msgr2=0x7f2bbc14d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2bbbfff700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bbc14b650 0x7f2bbc14d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2bbbfff700 1 -- 172.21.15.33:0/1384089884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc154be0 con 0x7f2ba4003a70 2023-11-06T22:18:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2bc0d9c700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bbc14b650 0x7f2bbc14d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2bbb7fe700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bbc0a9450 0x7f2bbc14d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2bbbfff700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2bbc14b160 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 rx=0x7f2bac00c940 tx=0x7f2bac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2ba3fff700 1 -- 172.21.15.33:0/1384089884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac000d00 con 0x7f2ba4003a70 2023-11-06T22:18:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2ba3fff700 1 -- 172.21.15.33:0/1384089884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac00cd40 con 0x7f2ba4003a70 2023-11-06T22:18:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2ba3fff700 1 -- 172.21.15.33:0/1384089884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac007650 con 0x7f2ba4003a70 2023-11-06T22:18:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2bc27ff700 1 -- 172.21.15.33:0/1384089884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc14de20 con 0x7f2ba4003a70 2023-11-06T22:18:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.775+0000 7f2bc27ff700 1 -- 172.21.15.33:0/1384089884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc14be70 con 0x7f2ba4003a70 2023-11-06T22:18:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.776+0000 7f2bc27ff700 1 -- 172.21.15.33:0/1384089884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc154f50 con 0x7f2ba4003a70 2023-11-06T22:18:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.777+0000 7f2ba3fff700 1 -- 172.21.15.33:0/1384089884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2bac013450 con 0x7f2ba4003a70 2023-11-06T22:18:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.778+0000 7f2ba3fff700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2ba805f270 0x7f2ba8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.778+0000 7f2ba3fff700 1 -- 172.21.15.33:0/1384089884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f2bac07b330 con 0x7f2ba4003a70 2023-11-06T22:18:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.778+0000 7f2bbb7fe700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2ba805f270 0x7f2ba8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.779+0000 7f2bbb7fe700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2ba805f270 0x7f2ba8061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f2bb0009270 tx=0x7f2bb0008040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.782+0000 7f2ba3fff700 1 -- 172.21.15.33:0/1384089884 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2bac04c480 con 0x7f2ba4003a70 2023-11-06T22:18:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.963+0000 7f2bc27ff700 1 -- 172.21.15.33:0/1384089884 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bbc0c7700 con 0x7f2ba805f270 2023-11-06T22:18:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.963+0000 7f2ba3fff700 1 -- 172.21.15.33:0/1384089884 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2bbc0c7700 con 0x7f2ba805f270 2023-11-06T22:18:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 -- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2ba805f270 msgr2=0x7f2ba8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2ba805f270 0x7f2ba8061730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f2bb0009270 tx=0x7f2bb0008040).stop 2023-11-06T22:18:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 -- 172.21.15.33:0/1384089884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 msgr2=0x7f2bbc14b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2bbc14b160 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 rx=0x7f2bac00c940 tx=0x7f2bac015000).stop 2023-11-06T22:18:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 -- 172.21.15.33:0/1384089884 shutdown_connections 2023-11-06T22:18:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2ba805f270 0x7f2ba8061730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bbc14b650 0x7f2bbc14d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ba4003a70 0x7f2bbc14b160 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 --2- 172.21.15.33:0/1384089884 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bbc0a9450 0x7f2bbc14d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 -- 172.21.15.33:0/1384089884 >> 172.21.15.33:0/1384089884 conn(0x7f2bbc05e990 msgr2=0x7f2bbc0c0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 -- 172.21.15.33:0/1384089884 shutdown_connections 2023-11-06T22:18:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:56.965+0000 7f2bc27ff700 1 -- 172.21.15.33:0/1384089884 wait complete. 2023-11-06T22:18:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 1 to deep-scrub 2023-11-06T22:18:56.985 INFO:tasks.ceph:Scrubbing osd.2 2023-11-06T22:18:56.986 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-06T22:18:57.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.183+0000 7fadfb3ff700 1 Processor -- start 2023-11-06T22:18:57.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.183+0000 7fadfb3ff700 1 -- start start 2023-11-06T22:18:57.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadfb3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf41542b0 0x7fadf40c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadfb3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf40c9f80 con 0x7fadf4153ed0 2023-11-06T22:18:57.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadfb3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf40cab20 con 0x7fadf40c8ee0 2023-11-06T22:18:57.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadfb3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40c6000 con 0x7fadf41542b0 2023-11-06T22:18:57.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadf919b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fadf4153ed0 0x7fadf4158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33576/0 (socket says 172.21.15.33:33576) 2023-11-06T22:18:57.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadf919b700 1 -- 172.21.15.33:0/4246193449 learned_addr learned my addr 172.21.15.33:0/4246193449 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:57.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadf999c700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf41542b0 0x7fadf40c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297455342 0 0) 0x7fadf40c9f80 con 0x7fadf4153ed0 2023-11-06T22:18:57.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade00033f0 con 0x7fadf4153ed0 2023-11-06T22:18:57.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054107211 0 0) 0x7fadf40cab20 con 0x7fadf40c8ee0 2023-11-06T22:18:57.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.185+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadf40c9f80 con 0x7fadf40c8ee0 2023-11-06T22:18:57.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3782648936 0 0) 0x7fade00033f0 con 0x7fadf4153ed0 2023-11-06T22:18:57.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf41542b0 msgr2=0x7fadf40c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:57.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf41542b0 0x7fadf40c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 >> v1:172.21.15.69:6789/0 conn(0x7fadf40c8ee0 legacy=0x7fadf40ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf40c9370 con 0x7fadf4153ed0 2023-11-06T22:18:57.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3281574812 0 0) 0x7fadf0001470 con 0x7fadf4153ed0 2023-11-06T22:18:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 >> v1:172.21.15.33:6789/0 conn(0x7fadf4153ed0 legacy=0x7fadf4158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade0004b00 0x7fade0004f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fade0003630 0x7fade0003ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf41542b0 0x7fade0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40c6000 con 0x7fade0004b00 2023-11-06T22:18:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade000b910 con 0x7fade0003630 2023-11-06T22:18:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade000bad0 con 0x7fadf41542b0 2023-11-06T22:18:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadf919b700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade0004b00 0x7fade0004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadf899a700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fade0003630 0x7fade0003ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadf999c700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf41542b0 0x7fade0003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadf919b700 1 -- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf41542b0 msgr2=0x7fade0003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadf919b700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf41542b0 0x7fade0003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadf919b700 1 -- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fade0003630 msgr2=0x7fade0003ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.186+0000 7fadf919b700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fade0003630 0x7fade0003ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.187+0000 7fadf919b700 1 -- 172.21.15.33:0/4246193449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade00033f0 con 0x7fade0004b00 2023-11-06T22:18:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.187+0000 7fadf919b700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade0004b00 0x7fade0004f00 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 rx=0x7fadf000c110 tx=0x7fadf0001e10).ready entity=mon.0 client_cookie=337f523389ded582 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.187+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000ca40 con 0x7fade0004b00 2023-11-06T22:18:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.187+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf000cba0 con 0x7fade0004b00 2023-11-06T22:18:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.187+0000 7fadebfff700 1 -- 172.21.15.33:0/4246193449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00070d0 con 0x7fade0004b00 2023-11-06T22:18:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.187+0000 7fadfb3ff700 1 -- 172.21.15.33:0/4246193449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade0004b00 msgr2=0x7fade0004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.187+0000 7fadfb3ff700 1 --2- 172.21.15.33:0/4246193449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade0004b00 0x7fade0004f00 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 rx=0x7fadf000c110 tx=0x7fadf0001e10).stop 2023-11-06T22:18:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.187+0000 7fadf999c700 1 -- 172.21.15.33:0/4246193449 reap_dead start 2023-11-06T22:18:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.187+0000 7fadfb3ff700 1 -- 172.21.15.33:0/4246193449 shutdown_connections 2023-11-06T22:18:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.187+0000 7fadfb3ff700 1 -- 172.21.15.33:0/4246193449 >> 172.21.15.33:0/4246193449 conn(0x7fadf405e1b0 msgr2=0x7fadf405e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.188+0000 7fadfb3ff700 1 -- 172.21.15.33:0/4246193449 shutdown_connections 2023-11-06T22:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.188+0000 7fadfb3ff700 1 -- 172.21.15.33:0/4246193449 wait complete. 2023-11-06T22:18:57.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.189+0000 7fadfb3ff700 1 Processor -- start 2023-11-06T22:18:57.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.189+0000 7fadfb3ff700 1 -- start start 2023-11-06T22:18:57.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.190+0000 7fadfb3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fade0004b00 0x7fadf414d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.190+0000 7fadfb3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf414dbf0 0x7fadf4142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.190+0000 7fadfb3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf4143250 0x7fadf4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.190+0000 7fadfb3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40afa20 con 0x7fadf414dbf0 2023-11-06T22:18:57.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.190+0000 7fadfb3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40c1ec0 con 0x7fade0004b00 2023-11-06T22:18:57.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.190+0000 7fadfb3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40c21a0 con 0x7fadf4143250 2023-11-06T22:18:57.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.190+0000 7fadf899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf414dbf0 0x7fadf4142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.190+0000 7fadf899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf414dbf0 0x7fadf4142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46518/0 (socket says 172.21.15.33:46518) 2023-11-06T22:18:57.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.190+0000 7fadf899a700 1 -- 172.21.15.33:0/2387278344 learned_addr learned my addr 172.21.15.33:0/2387278344 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:57.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fadf999c700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf4143250 0x7fadf4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fadf919b700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fade0004b00 0x7fadf414d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fadf899a700 1 -- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf4143250 msgr2=0x7fadf4159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fadf899a700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fadf4143250 0x7fadf4159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fadf899a700 1 -- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fade0004b00 msgr2=0x7fadf414d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fadf899a700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fade0004b00 0x7fadf414d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fadf899a700 1 -- 172.21.15.33:0/2387278344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4143950 con 0x7fadf414dbf0 2023-11-06T22:18:57.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fadf899a700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf414dbf0 0x7fadf4142d60 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 rx=0x7fade4002a50 tx=0x7fade4007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fade97fa700 1 -- 172.21.15.33:0/2387278344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400f640 con 0x7fadf414dbf0 2023-11-06T22:18:57.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fade97fa700 1 -- 172.21.15.33:0/2387278344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade400f7a0 con 0x7fadf414dbf0 2023-11-06T22:18:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fade97fa700 1 -- 172.21.15.33:0/2387278344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400fb40 con 0x7fadf414dbf0 2023-11-06T22:18:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf40afc00 con 0x7fadf414dbf0 2023-11-06T22:18:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.191+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf40b0180 con 0x7fadf414dbf0 2023-11-06T22:18:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.192+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fadf406c6c0 con 0x7fadf414dbf0 2023-11-06T22:18:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.194+0000 7fade97fa700 1 -- 172.21.15.33:0/2387278344 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fade401d3b0 con 0x7fadf414dbf0 2023-11-06T22:18:57.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.195+0000 7fade97fa700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fadd805f2a0 0x7fadd8061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.195+0000 7fade97fa700 1 -- 172.21.15.33:0/2387278344 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fade407a830 con 0x7fadf414dbf0 2023-11-06T22:18:57.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.195+0000 7fade97fa700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] conn(0x7fadd8065c50 0x7fadd80680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.195+0000 7fade97fa700 1 -- 172.21.15.33:0/2387278344 --> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fadd8068730 con 0x7fadd8065c50 2023-11-06T22:18:57.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.195+0000 7fade97fa700 1 -- 172.21.15.33:0/2387278344 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fade407ac00 con 0x7fadf414dbf0 2023-11-06T22:18:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.195+0000 7fadf919b700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fadd805f2a0 0x7fadd8061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.195+0000 7fadf999c700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] conn(0x7fadd8065c50 0x7fadd80680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.196+0000 7fadf919b700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fadd805f2a0 0x7fadd8061760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fadf0001ed0 tx=0x7fadf0015000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.196+0000 7fadf999c700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] conn(0x7fadd8065c50 0x7fadd80680b0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.197+0000 7fade97fa700 1 -- 172.21.15.33:0/2387278344 <== osd.2 v2:172.21.15.33:6818/115398 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fadd8068730 con 0x7fadd8065c50 2023-11-06T22:18:57.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.211+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 --> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fadf40c22e0 con 0x7fadd8065c50 2023-11-06T22:18:57.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.211+0000 7fade97fa700 1 -- 172.21.15.33:0/2387278344 <== osd.2 v2:172.21.15.33:6818/115398 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fadf40c22e0 con 0x7fadd8065c50 2023-11-06T22:18:57.214 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:18:57.214 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T22:18:57.214 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:18:57.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.211+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 >> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] conn(0x7fadd8065c50 msgr2=0x7fadd80680b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.211+0000 7fadfb3ff700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.33:6818/115398,v1:172.21.15.33:6820/115398] conn(0x7fadd8065c50 0x7fadd80680b0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.211+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fadd805f2a0 msgr2=0x7fadd8061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.211+0000 7fadfb3ff700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fadd805f2a0 0x7fadd8061760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fadf0001ed0 tx=0x7fadf0015000).stop 2023-11-06T22:18:57.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.211+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf414dbf0 msgr2=0x7fadf4142d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.211+0000 7fadfb3ff700 1 --2- 172.21.15.33:0/2387278344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf414dbf0 0x7fadf4142d60 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 rx=0x7fade4002a50 tx=0x7fade4007bb0).stop 2023-11-06T22:18:57.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.212+0000 7fadf999c700 1 -- 172.21.15.33:0/2387278344 reap_dead start 2023-11-06T22:18:57.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.212+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 shutdown_connections 2023-11-06T22:18:57.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.212+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 >> 172.21.15.33:0/2387278344 conn(0x7fadf405e1b0 msgr2=0x7fadf414e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:57.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.212+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 shutdown_connections 2023-11-06T22:18:57.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.212+0000 7fadfb3ff700 1 -- 172.21.15.33:0/2387278344 wait complete. 2023-11-06T22:18:57.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-06T22:18:57.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.426+0000 7fcb7abff700 1 Processor -- start 2023-11-06T22:18:57.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.426+0000 7fcb7abff700 1 -- start start 2023-11-06T22:18:57.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.427+0000 7fcb7abff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcb741544a0 0x7fcb740c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.427+0000 7fcb7abff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb740637f0 0x7fcb740bfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.427+0000 7fcb7abff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb741553d0 con 0x7fcb740c6180 2023-11-06T22:18:57.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.427+0000 7fcb7abff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb74154880 con 0x7fcb740637f0 2023-11-06T22:18:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.427+0000 7fcb7abff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb740c6710 con 0x7fcb741544a0 2023-11-06T22:18:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.428+0000 7fcb7919c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb740637f0 0x7fcb740bfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.428+0000 7fcb7919c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb740637f0 0x7fcb740bfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46528/0 (socket says 172.21.15.33:46528) 2023-11-06T22:18:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.428+0000 7fcb7919c700 1 -- 172.21.15.33:0/597418998 learned_addr learned my addr 172.21.15.33:0/597418998 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.428+0000 7fcb7899b700 1 --2- 172.21.15.33:0/597418998 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcb741544a0 0x7fcb740c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.428+0000 7fcb7919c700 1 -- 172.21.15.33:0/597418998 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcb741544a0 msgr2=0x7fcb740c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:57.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.428+0000 7fcb7919c700 1 --2- 172.21.15.33:0/597418998 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcb741544a0 0x7fcb740c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.428+0000 7fcb7919c700 1 -- 172.21.15.33:0/597418998 >> v1:172.21.15.69:6789/0 conn(0x7fcb740c6180 legacy=0x7fcb740630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.428+0000 7fcb7919c700 1 -- 172.21.15.33:0/597418998 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb74154aa0 con 0x7fcb740637f0 2023-11-06T22:18:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.428+0000 7fcb7919c700 1 --2- 172.21.15.33:0/597418998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb740637f0 0x7fcb740bfd50 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 rx=0x7fcb6c009f20 tx=0x7fcb6c009840).ready entity=mon.0 client_cookie=6f483939a02ab183 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb737fe700 1 -- 172.21.15.33:0/597418998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00e070 con 0x7fcb740637f0 2023-11-06T22:18:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb737fe700 1 -- 172.21.15.33:0/597418998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c00b9d0 con 0x7fcb740637f0 2023-11-06T22:18:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb737fe700 1 -- 172.21.15.33:0/597418998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00bd50 con 0x7fcb740637f0 2023-11-06T22:18:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb7abff700 1 -- 172.21.15.33:0/597418998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb740637f0 msgr2=0x7fcb740bfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb7abff700 1 --2- 172.21.15.33:0/597418998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb740637f0 0x7fcb740bfd50 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 rx=0x7fcb6c009f20 tx=0x7fcb6c009840).stop 2023-11-06T22:18:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb7abff700 1 -- 172.21.15.33:0/597418998 shutdown_connections 2023-11-06T22:18:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb7abff700 1 --2- 172.21.15.33:0/597418998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb740637f0 0x7fcb740bfd50 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb7abff700 1 --2- 172.21.15.33:0/597418998 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcb741544a0 0x7fcb740c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb7abff700 1 -- 172.21.15.33:0/597418998 >> 172.21.15.33:0/597418998 conn(0x7fcb7405ec80 msgr2=0x7fcb740cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb7abff700 1 -- 172.21.15.33:0/597418998 shutdown_connections 2023-11-06T22:18:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.429+0000 7fcb7abff700 1 -- 172.21.15.33:0/597418998 wait complete. 2023-11-06T22:18:57.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.431+0000 7fcb7abff700 1 Processor -- start 2023-11-06T22:18:57.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.431+0000 7fcb7abff700 1 -- start start 2023-11-06T22:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.432+0000 7fcb7abff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcb740637f0 0x7fcb7413f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.432+0000 7fcb7abff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb740c6180 0x7fcb741684e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.432+0000 7fcb7abff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb741544a0 0x7fcb740b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.432+0000 7fcb7abff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7413f840 con 0x7fcb741544a0 2023-11-06T22:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.432+0000 7fcb7abff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb74157dd0 con 0x7fcb740c6180 2023-11-06T22:18:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.432+0000 7fcb7abff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb741689d0 con 0x7fcb740637f0 2023-11-06T22:18:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.432+0000 7fcb7919c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb741544a0 0x7fcb740b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.432+0000 7fcb7919c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb741544a0 0x7fcb740b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46544/0 (socket says 172.21.15.33:46544) 2023-11-06T22:18:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.432+0000 7fcb7919c700 1 -- 172.21.15.33:0/1484334905 learned_addr learned my addr 172.21.15.33:0/1484334905 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb7899b700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcb740637f0 0x7fcb7413f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb73fff700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb740c6180 0x7fcb741684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb7919c700 1 -- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcb740637f0 msgr2=0x7fcb7413f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb7919c700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcb740637f0 0x7fcb7413f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb7919c700 1 -- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb740c6180 msgr2=0x7fcb741684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb7919c700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb740c6180 0x7fcb741684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb7919c700 1 -- 172.21.15.33:0/1484334905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c009ae0 con 0x7fcb741544a0 2023-11-06T22:18:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb7919c700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb741544a0 0x7fcb740b06b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 rx=0x7fcb6c00b610 tx=0x7fcb6c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb70ff9700 1 -- 172.21.15.33:0/1484334905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c0092e0 con 0x7fcb741544a0 2023-11-06T22:18:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb70ff9700 1 -- 172.21.15.33:0/1484334905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c01e470 con 0x7fcb741544a0 2023-11-06T22:18:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb70ff9700 1 -- 172.21.15.33:0/1484334905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c01e830 con 0x7fcb741544a0 2023-11-06T22:18:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.433+0000 7fcb7abff700 1 -- 172.21.15.33:0/1484334905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb741574a0 con 0x7fcb741544a0 2023-11-06T22:18:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.434+0000 7fcb7abff700 1 -- 172.21.15.33:0/1484334905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb74157a60 con 0x7fcb741544a0 2023-11-06T22:18:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.435+0000 7fcb7abff700 1 -- 172.21.15.33:0/1484334905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb74157f80 con 0x7fcb741544a0 2023-11-06T22:18:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.435+0000 7fcb70ff9700 1 -- 172.21.15.33:0/1484334905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fcb6c003040 con 0x7fcb741544a0 2023-11-06T22:18:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.437+0000 7fcb70ff9700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcb5805f270 0x7fcb58061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.437+0000 7fcb70ff9700 1 -- 172.21.15.33:0/1484334905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fcb6c07b610 con 0x7fcb741544a0 2023-11-06T22:18:57.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.438+0000 7fcb7899b700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcb5805f270 0x7fcb58061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.439+0000 7fcb7899b700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcb5805f270 0x7fcb58061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fcb68009b90 tx=0x7fcb68009060).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.441+0000 7fcb70ff9700 1 -- 172.21.15.33:0/1484334905 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcb6c04c910 con 0x7fcb741544a0 2023-11-06T22:18:57.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.622+0000 7fcb7abff700 1 -- 172.21.15.33:0/1484334905 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb7406c6c0 con 0x7fcb5805f270 2023-11-06T22:18:57.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.623+0000 7fcb70ff9700 1 -- 172.21.15.33:0/1484334905 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcb7406c6c0 con 0x7fcb5805f270 2023-11-06T22:18:57.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 -- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcb5805f270 msgr2=0x7fcb58061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcb5805f270 0x7fcb58061730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fcb68009b90 tx=0x7fcb68009060).stop 2023-11-06T22:18:57.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 -- 172.21.15.33:0/1484334905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb741544a0 msgr2=0x7fcb740b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb741544a0 0x7fcb740b06b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 rx=0x7fcb6c00b610 tx=0x7fcb6c009840).stop 2023-11-06T22:18:57.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 -- 172.21.15.33:0/1484334905 shutdown_connections 2023-11-06T22:18:57.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fcb5805f270 0x7fcb58061730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fcb740637f0 0x7fcb7413f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb741544a0 0x7fcb740b06b0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 --2- 172.21.15.33:0/1484334905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb740c6180 0x7fcb741684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 -- 172.21.15.33:0/1484334905 >> 172.21.15.33:0/1484334905 conn(0x7fcb7405ec80 msgr2=0x7fcb740aba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 -- 172.21.15.33:0/1484334905 shutdown_connections 2023-11-06T22:18:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.625+0000 7fcb7abff700 1 -- 172.21.15.33:0/1484334905 wait complete. 2023-11-06T22:18:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 2 to deep-scrub 2023-11-06T22:18:57.645 INFO:tasks.ceph:Scrubbing osd.3 2023-11-06T22:18:57.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-06T22:18:57.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.843+0000 7f3c7bfff700 1 Processor -- start 2023-11-06T22:18:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.843+0000 7f3c7bfff700 1 -- start start 2023-11-06T22:18:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.844+0000 7f3c7bfff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3c741545e0 0x7f3c74062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.844+0000 7f3c7bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c74069c90 con 0x7f3c7406a120 2023-11-06T22:18:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.845+0000 7f3c7bfff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c741549c0 con 0x7f3c74155420 2023-11-06T22:18:57.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.845+0000 7f3c7bfff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c74155aa0 con 0x7f3c741545e0 2023-11-06T22:18:57.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.845+0000 7f3c7959a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3c7406a120 0x7f3c74154d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33586/0 (socket says 172.21.15.33:33586) 2023-11-06T22:18:57.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.845+0000 7f3c7959a700 1 -- 172.21.15.33:0/3719622371 learned_addr learned my addr 172.21.15.33:0/3719622371 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:57.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.845+0000 7f3c79d9b700 1 --2- 172.21.15.33:0/3719622371 >> v2:172.21.15.69:3300/0 conn(0x7f3c741545e0 0x7f3c74062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.845+0000 7f3c78d99700 1 -- 172.21.15.33:0/3719622371 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965452061 0 0) 0x7f3c74069c90 con 0x7f3c7406a120 2023-11-06T22:18:57.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.845+0000 7f3c78d99700 1 -- 172.21.15.33:0/3719622371 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c600033f0 con 0x7f3c7406a120 2023-11-06T22:18:57.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.845+0000 7f3c78d99700 1 -- 172.21.15.33:0/3719622371 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025905810 0 0) 0x7f3c741549c0 con 0x7f3c74155420 2023-11-06T22:18:57.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.845+0000 7f3c78d99700 1 -- 172.21.15.33:0/3719622371 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c74069c90 con 0x7f3c74155420 2023-11-06T22:18:57.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.845+0000 7f3c79d9b700 1 -- 172.21.15.33:0/3719622371 >> v1:172.21.15.69:6789/0 conn(0x7f3c74155420 legacy=0x7f3c740c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.846+0000 7f3c79d9b700 1 -- 172.21.15.33:0/3719622371 >> v1:172.21.15.33:6789/0 conn(0x7f3c7406a120 legacy=0x7f3c74154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.846+0000 7f3c79d9b700 1 -- 172.21.15.33:0/3719622371 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c74069360 con 0x7f3c741545e0 2023-11-06T22:18:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.846+0000 7f3c79d9b700 1 --2- 172.21.15.33:0/3719622371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c741545e0 0x7f3c74062710 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f3c70009760 tx=0x7f3c70009060).ready entity=mon.1 client_cookie=d7b392da3e11b068 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.846+0000 7f3c78d99700 1 -- 172.21.15.33:0/3719622371 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70009120 con 0x7f3c741545e0 2023-11-06T22:18:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.846+0000 7f3c78d99700 1 -- 172.21.15.33:0/3719622371 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7000d4a0 con 0x7f3c741545e0 2023-11-06T22:18:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.846+0000 7f3c78d99700 1 -- 172.21.15.33:0/3719622371 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000d820 con 0x7f3c741545e0 2023-11-06T22:18:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.846+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3719622371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c741545e0 msgr2=0x7f3c74062710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.846+0000 7f3c7bfff700 1 --2- 172.21.15.33:0/3719622371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c741545e0 0x7f3c74062710 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f3c70009760 tx=0x7f3c70009060).stop 2023-11-06T22:18:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.847+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3719622371 shutdown_connections 2023-11-06T22:18:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.847+0000 7f3c7bfff700 1 --2- 172.21.15.33:0/3719622371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c741545e0 0x7f3c74062710 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.847+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3719622371 >> 172.21.15.33:0/3719622371 conn(0x7f3c740c7b30 msgr2=0x7f3c740c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.847+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3719622371 shutdown_connections 2023-11-06T22:18:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.847+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3719622371 wait complete. 2023-11-06T22:18:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.848+0000 7f3c7bfff700 1 Processor -- start 2023-11-06T22:18:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.848+0000 7f3c7bfff700 1 -- start start 2023-11-06T22:18:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.849+0000 7f3c7bfff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3c7406a120 0x7f3c740ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.849+0000 7f3c7bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c741545e0 0x7f3c740ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.849+0000 7f3c7bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c74155420 0x7f3c74066660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.849+0000 7f3c7bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74149400 con 0x7f3c74155420 2023-11-06T22:18:57.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.849+0000 7f3c7bfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740adbe0 con 0x7f3c741545e0 2023-11-06T22:18:57.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.849+0000 7f3c7bfff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74152b60 con 0x7f3c7406a120 2023-11-06T22:18:57.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.849+0000 7f3c7a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c74155420 0x7f3c74066660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c79d9b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3c7406a120 0x7f3c740ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c7a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c74155420 0x7f3c74066660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46556/0 (socket says 172.21.15.33:46556) 2023-11-06T22:18:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c7a59c700 1 -- 172.21.15.33:0/3900887835 learned_addr learned my addr 172.21.15.33:0/3900887835 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c7959a700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c741545e0 0x7f3c740ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c7a59c700 1 -- 172.21.15.33:0/3900887835 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3c7406a120 msgr2=0x7f3c740ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c7a59c700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3c7406a120 0x7f3c740ad6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c7a59c700 1 -- 172.21.15.33:0/3900887835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c741545e0 msgr2=0x7f3c740ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c7a59c700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c741545e0 0x7f3c740ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c7a59c700 1 -- 172.21.15.33:0/3900887835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c700092a0 con 0x7f3c74155420 2023-11-06T22:18:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c7a59c700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c74155420 0x7f3c74066660 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 rx=0x7f3c6c00c740 tx=0x7f3c6c00ca50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c69ffb700 1 -- 172.21.15.33:0/3900887835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c00e4a0 con 0x7f3c74155420 2023-11-06T22:18:57.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c69ffb700 1 -- 172.21.15.33:0/3900887835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c00e600 con 0x7f3c74155420 2023-11-06T22:18:57.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c69ffb700 1 -- 172.21.15.33:0/3900887835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c00e980 con 0x7f3c74155420 2023-11-06T22:18:57.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.850+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c740aad90 con 0x7f3c74155420 2023-11-06T22:18:57.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.851+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c74152000 con 0x7f3c74155420 2023-11-06T22:18:57.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.851+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3c740635d0 con 0x7f3c74155420 2023-11-06T22:18:57.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.852+0000 7f3c69ffb700 1 -- 172.21.15.33:0/3900887835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3c6c012400 con 0x7f3c74155420 2023-11-06T22:18:57.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.853+0000 7f3c69ffb700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c5805f270 0x7f3c58061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.853+0000 7f3c69ffb700 1 -- 172.21.15.33:0/3900887835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f3c6c07a5d0 con 0x7f3c74155420 2023-11-06T22:18:57.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.853+0000 7f3c69ffb700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] conn(0x7f3c58065c20 0x7f3c58068080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:57.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.853+0000 7f3c69ffb700 1 -- 172.21.15.33:0/3900887835 --> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c58068700 con 0x7f3c58065c20 2023-11-06T22:18:57.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.853+0000 7f3c79d9b700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c5805f270 0x7f3c58061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.853+0000 7f3c69ffb700 1 -- 172.21.15.33:0/3900887835 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c6c07a8c0 con 0x7f3c74155420 2023-11-06T22:18:57.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.854+0000 7f3c7959a700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] conn(0x7f3c58065c20 0x7f3c58068080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:57.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.854+0000 7f3c79d9b700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c5805f270 0x7f3c58061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f3c70006d60 tx=0x7f3c70016000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.854+0000 7f3c7959a700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] conn(0x7f3c58065c20 0x7f3c58068080 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.855+0000 7f3c69ffb700 1 -- 172.21.15.33:0/3900887835 <== osd.3 v2:172.21.15.33:6816/115401 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f3c58068700 con 0x7f3c58065c20 2023-11-06T22:18:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.869+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 --> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3c74152ca0 con 0x7f3c58065c20 2023-11-06T22:18:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.869+0000 7f3c69ffb700 1 -- 172.21.15.33:0/3900887835 <== osd.3 v2:172.21.15.33:6816/115401 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3c74152ca0 con 0x7f3c58065c20 2023-11-06T22:18:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:18:57.872 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T22:18:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:18:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 >> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] conn(0x7f3c58065c20 msgr2=0x7f3c58068080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7bfff700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.33:6816/115401,v1:172.21.15.33:6817/115401] conn(0x7f3c58065c20 0x7f3c58068080 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c5805f270 msgr2=0x7f3c58061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7bfff700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c5805f270 0x7f3c58061730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f3c70006d60 tx=0x7f3c70016000).stop 2023-11-06T22:18:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c74155420 msgr2=0x7f3c74066660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7bfff700 1 --2- 172.21.15.33:0/3900887835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c74155420 0x7f3c74066660 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 rx=0x7f3c6c00c740 tx=0x7f3c6c00ca50).stop 2023-11-06T22:18:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7a59c700 1 -- 172.21.15.33:0/3900887835 reap_dead start 2023-11-06T22:18:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 shutdown_connections 2023-11-06T22:18:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 >> 172.21.15.33:0/3900887835 conn(0x7f3c740c7b30 msgr2=0x7f3c740a6210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 shutdown_connections 2023-11-06T22:18:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:57.870+0000 7f3c7bfff700 1 -- 172.21.15.33:0/3900887835 wait complete. 2023-11-06T22:18:57.889 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-06T22:18:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.082+0000 7fb716fff700 1 Processor -- start 2023-11-06T22:18:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.083+0000 7fb716fff700 1 -- start start 2023-11-06T22:18:58.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.084+0000 7fb716fff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb71013ce50 0x7fb71013d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.084+0000 7fb716fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb7100c4560 0x7fb7100c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.084+0000 7fb716fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb710157fe0 0x7fb710158430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.084+0000 7fb716fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb710158d50 con 0x7fb7100c4560 2023-11-06T22:18:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.084+0000 7fb716fff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7100c4f00 con 0x7fb71013ce50 2023-11-06T22:18:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.084+0000 7fb716fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb710062840 con 0x7fb710157fe0 2023-11-06T22:18:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.084+0000 7fb70ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb7100c4560 0x7fb7100c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.084+0000 7fb70ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb7100c4560 0x7fb7100c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52548/0 (socket says 172.21.15.33:52548) 2023-11-06T22:18:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.084+0000 7fb70ffff700 1 -- 172.21.15.33:0/2925665571 learned_addr learned my addr 172.21.15.33:0/2925665571 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb714d9b700 1 --2- 172.21.15.33:0/2925665571 >> v2:172.21.15.69:3300/0 conn(0x7fb71013ce50 0x7fb71013d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb71559c700 1 --2- 172.21.15.33:0/2925665571 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb710157fe0 0x7fb710158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb70ffff700 1 -- 172.21.15.33:0/2925665571 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb710157fe0 msgr2=0x7fb710158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb70ffff700 1 --2- 172.21.15.33:0/2925665571 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb710157fe0 0x7fb710158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb70ffff700 1 -- 172.21.15.33:0/2925665571 >> v2:172.21.15.69:3300/0 conn(0x7fb71013ce50 msgr2=0x7fb71013d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb70ffff700 1 --2- 172.21.15.33:0/2925665571 >> v2:172.21.15.69:3300/0 conn(0x7fb71013ce50 0x7fb71013d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb70ffff700 1 -- 172.21.15.33:0/2925665571 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb710062b20 con 0x7fb7100c4560 2023-11-06T22:18:58.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb714d9b700 1 --2- 172.21.15.33:0/2925665571 >> v2:172.21.15.69:3300/0 conn(0x7fb71013ce50 0x7fb71013d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:58.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb70ffff700 1 --2- 172.21.15.33:0/2925665571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7100c4560 0x7fb7100c4a10 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 rx=0x7fb71006c2d0 tx=0x7fb6fc0090f0).ready entity=mon.0 client_cookie=82dabeaa42ce6ce8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:58.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb70f7fe700 1 -- 172.21.15.33:0/2925665571 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00b600 con 0x7fb7100c4560 2023-11-06T22:18:58.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb70f7fe700 1 -- 172.21.15.33:0/2925665571 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00b760 con 0x7fb7100c4560 2023-11-06T22:18:58.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.085+0000 7fb70f7fe700 1 -- 172.21.15.33:0/2925665571 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00bae0 con 0x7fb7100c4560 2023-11-06T22:18:58.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.086+0000 7fb716fff700 1 -- 172.21.15.33:0/2925665571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7100c4560 msgr2=0x7fb7100c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.086+0000 7fb716fff700 1 --2- 172.21.15.33:0/2925665571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7100c4560 0x7fb7100c4a10 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 rx=0x7fb71006c2d0 tx=0x7fb6fc0090f0).stop 2023-11-06T22:18:58.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.086+0000 7fb716fff700 1 -- 172.21.15.33:0/2925665571 shutdown_connections 2023-11-06T22:18:58.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.086+0000 7fb716fff700 1 --2- 172.21.15.33:0/2925665571 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb710157fe0 0x7fb710158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.086+0000 7fb716fff700 1 --2- 172.21.15.33:0/2925665571 >> v2:172.21.15.69:3300/0 conn(0x7fb71013ce50 0x7fb71013d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.086+0000 7fb716fff700 1 --2- 172.21.15.33:0/2925665571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7100c4560 0x7fb7100c4a10 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.086+0000 7fb716fff700 1 -- 172.21.15.33:0/2925665571 >> 172.21.15.33:0/2925665571 conn(0x7fb7100c7dc0 msgr2=0x7fb7100c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:58.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.086+0000 7fb716fff700 1 -- 172.21.15.33:0/2925665571 shutdown_connections 2023-11-06T22:18:58.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.086+0000 7fb716fff700 1 -- 172.21.15.33:0/2925665571 wait complete. 2023-11-06T22:18:58.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.087+0000 7fb716fff700 1 Processor -- start 2023-11-06T22:18:58.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.088+0000 7fb716fff700 1 -- start start 2023-11-06T22:18:58.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.088+0000 7fb716fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7100c4560 0x7fb710138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.088+0000 7fb716fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb71013ce50 0x7fb7100a8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.088+0000 7fb716fff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb710157fe0 0x7fb7100a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.088+0000 7fb716fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100a9250 con 0x7fb71013ce50 2023-11-06T22:18:58.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.088+0000 7fb716fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71013a9e0 con 0x7fb7100c4560 2023-11-06T22:18:58.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.088+0000 7fb716fff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100ad340 con 0x7fb710157fe0 2023-11-06T22:18:58.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb70ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb71013ce50 0x7fb7100a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb70ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb71013ce50 0x7fb7100a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52556/0 (socket says 172.21.15.33:52556) 2023-11-06T22:18:58.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb70ffff700 1 -- 172.21.15.33:0/1786483813 learned_addr learned my addr 172.21.15.33:0/1786483813 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:58.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb714d9b700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7100c4560 0x7fb710138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb71559c700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb710157fe0 0x7fb7100a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb70ffff700 1 -- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb710157fe0 msgr2=0x7fb7100a8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb70ffff700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb710157fe0 0x7fb7100a8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb70ffff700 1 -- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7100c4560 msgr2=0x7fb710138ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb70ffff700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7100c4560 0x7fb710138ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb70ffff700 1 -- 172.21.15.33:0/1786483813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc009370 con 0x7fb71013ce50 2023-11-06T22:18:58.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb714d9b700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7100c4560 0x7fb710138ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:58.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb70ffff700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb71013ce50 0x7fb7100a8870 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 rx=0x7fb6fc006fd0 tx=0x7fb6fc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:58.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.089+0000 7fb70cff9700 1 -- 172.21.15.33:0/1786483813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00b8d0 con 0x7fb71013ce50 2023-11-06T22:18:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.090+0000 7fb70cff9700 1 -- 172.21.15.33:0/1786483813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00ba30 con 0x7fb71013ce50 2023-11-06T22:18:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.090+0000 7fb70cff9700 1 -- 172.21.15.33:0/1786483813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc01b690 con 0x7fb71013ce50 2023-11-06T22:18:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.090+0000 7fb716fff700 1 -- 172.21.15.33:0/1786483813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71013a690 con 0x7fb71013ce50 2023-11-06T22:18:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.090+0000 7fb716fff700 1 -- 172.21.15.33:0/1786483813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7101561c0 con 0x7fb71013ce50 2023-11-06T22:18:58.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.091+0000 7fb716fff700 1 -- 172.21.15.33:0/1786483813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71013ab50 con 0x7fb71013ce50 2023-11-06T22:18:58.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.091+0000 7fb70cff9700 1 -- 172.21.15.33:0/1786483813 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fb6fc01ba40 con 0x7fb71013ce50 2023-11-06T22:18:58.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.092+0000 7fb70cff9700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb6f405f270 0x7fb6f4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.093+0000 7fb70cff9700 1 -- 172.21.15.33:0/1786483813 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fb6fc07b8f0 con 0x7fb71013ce50 2023-11-06T22:18:58.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.093+0000 7fb714d9b700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb6f405f270 0x7fb6f4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.093+0000 7fb714d9b700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb6f405f270 0x7fb6f4061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb704006fd0 tx=0x7fb704008040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:58.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.096+0000 7fb70cff9700 1 -- 172.21.15.33:0/1786483813 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb6fc04cc20 con 0x7fb71013ce50 2023-11-06T22:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.275+0000 7fb716fff700 1 -- 172.21.15.33:0/1786483813 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb71013a510 con 0x7fb6f405f270 2023-11-06T22:18:58.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.276+0000 7fb70cff9700 1 -- 172.21.15.33:0/1786483813 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb70400b7c0 con 0x7fb6f405f270 2023-11-06T22:18:58.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 -- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb6f405f270 msgr2=0x7fb6f4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb6f405f270 0x7fb6f4061730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb704006fd0 tx=0x7fb704008040).stop 2023-11-06T22:18:58.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 -- 172.21.15.33:0/1786483813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb71013ce50 msgr2=0x7fb7100a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb71013ce50 0x7fb7100a8870 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 rx=0x7fb6fc006fd0 tx=0x7fb6fc017000).stop 2023-11-06T22:18:58.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 -- 172.21.15.33:0/1786483813 shutdown_connections 2023-11-06T22:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fb6f405f270 0x7fb6f4061730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fb710157fe0 0x7fb7100a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7100c4560 0x7fb710138ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 --2- 172.21.15.33:0/1786483813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb71013ce50 0x7fb7100a8870 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 -- 172.21.15.33:0/1786483813 >> 172.21.15.33:0/1786483813 conn(0x7fb7100c7dc0 msgr2=0x7fb71013d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 -- 172.21.15.33:0/1786483813 shutdown_connections 2023-11-06T22:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.278+0000 7fb716fff700 1 -- 172.21.15.33:0/1786483813 wait complete. 2023-11-06T22:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 3 to deep-scrub 2023-11-06T22:18:58.299 INFO:tasks.ceph:Scrubbing osd.4 2023-11-06T22:18:58.299 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-06T22:18:58.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.496+0000 7f20cc1ff700 1 Processor -- start 2023-11-06T22:18:58.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.496+0000 7f20cc1ff700 1 -- start start 2023-11-06T22:18:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.498+0000 7f20cc1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f20c413dea0 0x7f20c4158060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.498+0000 7f20cc1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20c4158550 0x7f20c4065520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.498+0000 7f20cc1ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f20c4065a10 0x7f20c4065ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.498+0000 7f20cc1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c40a3da0 con 0x7f20c413dea0 2023-11-06T22:18:58.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.498+0000 7f20cc1ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c4065090 con 0x7f20c4065a10 2023-11-06T22:18:58.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.498+0000 7f20cc1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c41589d0 con 0x7f20c4158550 2023-11-06T22:18:58.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.498+0000 7f20c9f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f20c413dea0 0x7f20c4158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.498+0000 7f20c9f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f20c413dea0 0x7f20c4158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52562/0 (socket says 172.21.15.33:52562) 2023-11-06T22:18:58.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.498+0000 7f20c9f9b700 1 -- 172.21.15.33:0/1754213161 learned_addr learned my addr 172.21.15.33:0/1754213161 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20c979a700 1 --2- 172.21.15.33:0/1754213161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20c4158550 0x7f20c4065520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20ca79c700 1 --2- 172.21.15.33:0/1754213161 >> v2:172.21.15.69:3300/0 conn(0x7f20c4065a10 0x7f20c4065ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20c9f9b700 1 -- 172.21.15.33:0/1754213161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20c4158550 msgr2=0x7f20c4065520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20c9f9b700 1 --2- 172.21.15.33:0/1754213161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20c4158550 0x7f20c4065520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20c9f9b700 1 -- 172.21.15.33:0/1754213161 >> v2:172.21.15.69:3300/0 conn(0x7f20c4065a10 msgr2=0x7f20c4065ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20c9f9b700 1 --2- 172.21.15.33:0/1754213161 >> v2:172.21.15.69:3300/0 conn(0x7f20c4065a10 0x7f20c4065ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20c9f9b700 1 -- 172.21.15.33:0/1754213161 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40c3200 con 0x7f20c413dea0 2023-11-06T22:18:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20c9f9b700 1 --2- 172.21.15.33:0/1754213161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c413dea0 0x7f20c4158060 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 rx=0x7f20c405f2b0 tx=0x7f20c00090f0).ready entity=mon.0 client_cookie=21ebcf42793996d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20c8f99700 1 -- 172.21.15.33:0/1754213161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c000b600 con 0x7f20c413dea0 2023-11-06T22:18:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20c8f99700 1 -- 172.21.15.33:0/1754213161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c000b760 con 0x7f20c413dea0 2023-11-06T22:18:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.499+0000 7f20c8f99700 1 -- 172.21.15.33:0/1754213161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c000bb00 con 0x7f20c413dea0 2023-11-06T22:18:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.500+0000 7f20cc1ff700 1 -- 172.21.15.33:0/1754213161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c413dea0 msgr2=0x7f20c4158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.500+0000 7f20cc1ff700 1 --2- 172.21.15.33:0/1754213161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c413dea0 0x7f20c4158060 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 rx=0x7f20c405f2b0 tx=0x7f20c00090f0).stop 2023-11-06T22:18:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.500+0000 7f20cc1ff700 1 -- 172.21.15.33:0/1754213161 shutdown_connections 2023-11-06T22:18:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.500+0000 7f20cc1ff700 1 --2- 172.21.15.33:0/1754213161 >> v2:172.21.15.69:3300/0 conn(0x7f20c4065a10 0x7f20c4065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.500+0000 7f20cc1ff700 1 --2- 172.21.15.33:0/1754213161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c413dea0 0x7f20c4158060 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.500+0000 7f20cc1ff700 1 --2- 172.21.15.33:0/1754213161 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20c4158550 0x7f20c4065520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.500+0000 7f20cc1ff700 1 -- 172.21.15.33:0/1754213161 >> 172.21.15.33:0/1754213161 conn(0x7f20c40c6e70 msgr2=0x7f20c40c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.500+0000 7f20cc1ff700 1 -- 172.21.15.33:0/1754213161 shutdown_connections 2023-11-06T22:18:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.500+0000 7f20cc1ff700 1 -- 172.21.15.33:0/1754213161 wait complete. 2023-11-06T22:18:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.501+0000 7f20cc1ff700 1 Processor -- start 2023-11-06T22:18:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.502+0000 7f20cc1ff700 1 -- start start 2023-11-06T22:18:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.502+0000 7f20cc1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20c4065a10 0x7f20c40a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20cc1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20c413dea0 0x7f20c40a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20cc1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c4158550 0x7f20c413ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20cc1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40a8f70 con 0x7f20c4158550 2023-11-06T22:18:58.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20cc1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40a8360 con 0x7f20c4065a10 2023-11-06T22:18:58.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20cc1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40a8640 con 0x7f20c413dea0 2023-11-06T22:18:58.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20ca79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c4158550 0x7f20c413ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20ca79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c4158550 0x7f20c413ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52576/0 (socket says 172.21.15.33:52576) 2023-11-06T22:18:58.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20ca79c700 1 -- 172.21.15.33:0/2539486960 learned_addr learned my addr 172.21.15.33:0/2539486960 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:58.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20c9f9b700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20c4065a10 0x7f20c40a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20c979a700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20c413dea0 0x7f20c40a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20ca79c700 1 -- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20c413dea0 msgr2=0x7f20c40a75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20ca79c700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f20c413dea0 0x7f20c40a75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20ca79c700 1 -- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20c4065a10 msgr2=0x7f20c40a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20ca79c700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20c4065a10 0x7f20c40a9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20ca79c700 1 -- 172.21.15.33:0/2539486960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c0009370 con 0x7f20c4158550 2023-11-06T22:18:58.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.503+0000 7f20ca79c700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c4158550 0x7f20c413ce30 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 rx=0x7f20bc007f20 tx=0x7f20bc009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:58.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.504+0000 7f20b9ffb700 1 -- 172.21.15.33:0/2539486960 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00eca0 con 0x7f20c4158550 2023-11-06T22:18:58.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.504+0000 7f20b9ffb700 1 -- 172.21.15.33:0/2539486960 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc00ee00 con 0x7f20c4158550 2023-11-06T22:18:58.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.504+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c40a8920 con 0x7f20c4158550 2023-11-06T22:18:58.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.504+0000 7f20b9ffb700 1 -- 172.21.15.33:0/2539486960 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc014610 con 0x7f20c4158550 2023-11-06T22:18:58.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.504+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c40a98f0 con 0x7f20c4158550 2023-11-06T22:18:58.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.505+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f20c4064760 con 0x7f20c4158550 2023-11-06T22:18:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.506+0000 7f20b9ffb700 1 -- 172.21.15.33:0/2539486960 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f20bc01d3b0 con 0x7f20c4158550 2023-11-06T22:18:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.506+0000 7f20b9ffb700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f20a805f270 0x7f20a8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.507+0000 7f20b9ffb700 1 -- 172.21.15.33:0/2539486960 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f20bc07afa0 con 0x7f20c4158550 2023-11-06T22:18:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.507+0000 7f20b9ffb700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] conn(0x7f20a8065c20 0x7f20a8068080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.507+0000 7f20b9ffb700 1 -- 172.21.15.33:0/2539486960 --> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20a8068700 con 0x7f20a8065c20 2023-11-06T22:18:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.507+0000 7f20b9ffb700 1 -- 172.21.15.33:0/2539486960 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20bc07b290 con 0x7f20c4158550 2023-11-06T22:18:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.507+0000 7f20c9f9b700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f20a805f270 0x7f20a8061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.507+0000 7f20c979a700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] conn(0x7f20a8065c20 0x7f20a8068080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.508+0000 7f20c9f9b700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f20a805f270 0x7f20a8061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f20c0009fd0 tx=0x7f20c0017000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.508+0000 7f20c979a700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] conn(0x7f20a8065c20 0x7f20a8068080 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:58.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.509+0000 7f20b9ffb700 1 -- 172.21.15.33:0/2539486960 <== osd.4 v2:172.21.15.69:6801/115620 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f20a8068700 con 0x7f20a8065c20 2023-11-06T22:18:58.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.523+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 --> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f20c40a90b0 con 0x7f20a8065c20 2023-11-06T22:18:58.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.523+0000 7f20b9ffb700 1 -- 172.21.15.33:0/2539486960 <== osd.4 v2:172.21.15.69:6801/115620 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f20c40a90b0 con 0x7f20a8065c20 2023-11-06T22:18:58.526 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:18:58.526 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T22:18:58.527 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:18:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] conn(0x7f20a8065c20 msgr2=0x7f20a8068080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20cc1ff700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:6801/115620,v1:172.21.15.69:6803/115620] conn(0x7f20a8065c20 0x7f20a8068080 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f20a805f270 msgr2=0x7f20a8061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20cc1ff700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f20a805f270 0x7f20a8061730 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f20c0009fd0 tx=0x7f20c0017000).stop 2023-11-06T22:18:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c4158550 msgr2=0x7f20c413ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20cc1ff700 1 --2- 172.21.15.33:0/2539486960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c4158550 0x7f20c413ce30 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 rx=0x7f20bc007f20 tx=0x7f20bc009860).stop 2023-11-06T22:18:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20ca79c700 1 -- 172.21.15.33:0/2539486960 reap_dead start 2023-11-06T22:18:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 shutdown_connections 2023-11-06T22:18:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 >> 172.21.15.33:0/2539486960 conn(0x7f20c40c6e70 msgr2=0x7f20c41579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 shutdown_connections 2023-11-06T22:18:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.524+0000 7f20cc1ff700 1 -- 172.21.15.33:0/2539486960 wait complete. 2023-11-06T22:18:58.543 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-06T22:18:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.737+0000 7f4f0e3ff700 1 Processor -- start 2023-11-06T22:18:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.737+0000 7f4f0e3ff700 1 -- start start 2023-11-06T22:18:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.739+0000 7f4f0e3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4f08154210 0x7f4f080a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.739+0000 7f4f0e3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0805efc0 con 0x7f4f080ca610 2023-11-06T22:18:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.739+0000 7f4f0e3ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f080c8ea0 con 0x7f4f080bfd60 2023-11-06T22:18:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.739+0000 7f4f0e3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f080caa80 con 0x7f4f08154210 2023-11-06T22:18:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.739+0000 7f4f0c99c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4f080ca610 0x7f4f080c6d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46638/0 (socket says 172.21.15.33:46638) 2023-11-06T22:18:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.739+0000 7f4f07fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4f08154210 0x7f4f080a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.739+0000 7f4f0c99c700 1 -- 172.21.15.33:0/3047558677 learned_addr learned my addr 172.21.15.33:0/3047558677 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.740+0000 7f4f07fff700 1 -- 172.21.15.33:0/3047558677 >> v1:172.21.15.69:6789/0 conn(0x7f4f080bfd60 legacy=0x7f4f080c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.740+0000 7f4f07fff700 1 -- 172.21.15.33:0/3047558677 >> v1:172.21.15.33:6789/0 conn(0x7f4f080ca610 legacy=0x7f4f080c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.740+0000 7f4f07fff700 1 -- 172.21.15.33:0/3047558677 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f080c9140 con 0x7f4f08154210 2023-11-06T22:18:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.740+0000 7f4f06ffd700 1 -- 172.21.15.33:0/3047558677 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265586889 0 0) 0x7f4f0805efc0 con 0x7f4f080ca610 2023-11-06T22:18:58.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.740+0000 7f4f07fff700 1 --2- 172.21.15.33:0/3047558677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f08154210 0x7f4f080a7bd0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7f4ef8009760 tx=0x7f4ef8009060).ready entity=mon.0 client_cookie=97dbcd1cdf8188b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:58.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.740+0000 7f4f06ffd700 1 -- 172.21.15.33:0/3047558677 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800b580 con 0x7f4f08154210 2023-11-06T22:18:58.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.740+0000 7f4f06ffd700 1 -- 172.21.15.33:0/3047558677 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef800b6e0 con 0x7f4f08154210 2023-11-06T22:18:58.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.740+0000 7f4f06ffd700 1 -- 172.21.15.33:0/3047558677 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800ba60 con 0x7f4f08154210 2023-11-06T22:18:58.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.741+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/3047558677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f08154210 msgr2=0x7f4f080a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.741+0000 7f4f0e3ff700 1 --2- 172.21.15.33:0/3047558677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f08154210 0x7f4f080a7bd0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7f4ef8009760 tx=0x7f4ef8009060).stop 2023-11-06T22:18:58.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.741+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/3047558677 shutdown_connections 2023-11-06T22:18:58.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.741+0000 7f4f0e3ff700 1 --2- 172.21.15.33:0/3047558677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f08154210 0x7f4f080a7bd0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.741+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/3047558677 >> 172.21.15.33:0/3047558677 conn(0x7f4f0805e990 msgr2=0x7f4f080cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:58.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.741+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/3047558677 shutdown_connections 2023-11-06T22:18:58.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.741+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/3047558677 wait complete. 2023-11-06T22:18:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.742+0000 7f4f0e3ff700 1 Processor -- start 2023-11-06T22:18:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.743+0000 7f4f0e3ff700 1 -- start start 2023-11-06T22:18:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.743+0000 7f4f0e3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f080bfd60 0x7f4f080ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.743+0000 7f4f0e3ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4f080ca610 0x7f4f08156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f0e3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f08154210 0x7f4f080c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f0e3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08165940 con 0x7f4f080bfd60 2023-11-06T22:18:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f0e3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f080abc50 con 0x7f4f08154210 2023-11-06T22:18:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f0e3ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08164d30 con 0x7f4f080ca610 2023-11-06T22:18:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f07fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f080bfd60 0x7f4f080ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f07fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f080bfd60 0x7f4f080ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52590/0 (socket says 172.21.15.33:52590) 2023-11-06T22:18:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f07fff700 1 -- 172.21.15.33:0/1522081371 learned_addr learned my addr 172.21.15.33:0/1522081371 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f0c99c700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f08154210 0x7f4f080c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f077fe700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4f080ca610 0x7f4f08156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f07fff700 1 -- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4f080ca610 msgr2=0x7f4f08156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f07fff700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4f080ca610 0x7f4f08156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f07fff700 1 -- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f08154210 msgr2=0x7f4f080c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f07fff700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f08154210 0x7f4f080c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f07fff700 1 -- 172.21.15.33:0/1522081371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef80092a0 con 0x7f4f080bfd60 2023-11-06T22:18:58.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.744+0000 7f4f07fff700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f080bfd60 0x7f4f080ab760 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 rx=0x7f4ef800b340 tx=0x7f4ef8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:58.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.745+0000 7f4eeffff700 1 -- 172.21.15.33:0/1522081371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800ba60 con 0x7f4f080bfd60 2023-11-06T22:18:58.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.745+0000 7f4eeffff700 1 -- 172.21.15.33:0/1522081371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef8000c90 con 0x7f4f080bfd60 2023-11-06T22:18:58.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.745+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/1522081371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f08165010 con 0x7f4f080bfd60 2023-11-06T22:18:58.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.745+0000 7f4eeffff700 1 -- 172.21.15.33:0/1522081371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef801b670 con 0x7f4f080bfd60 2023-11-06T22:18:58.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.745+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/1522081371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f081655d0 con 0x7f4f080bfd60 2023-11-06T22:18:58.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.746+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/1522081371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f081556f0 con 0x7f4f080bfd60 2023-11-06T22:18:58.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.747+0000 7f4eeffff700 1 -- 172.21.15.33:0/1522081371 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4ef801ba60 con 0x7f4f080bfd60 2023-11-06T22:18:58.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.748+0000 7f4eeffff700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4ef405f270 0x7f4ef4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:58.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.748+0000 7f4eeffff700 1 -- 172.21.15.33:0/1522081371 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f4ef807bc90 con 0x7f4f080bfd60 2023-11-06T22:18:58.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.748+0000 7f4f077fe700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4ef405f270 0x7f4ef4061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:58.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.749+0000 7f4f077fe700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4ef405f270 0x7f4ef4061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f4f08151c60 tx=0x7f4efc009040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:58.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.752+0000 7f4eeffff700 1 -- 172.21.15.33:0/1522081371 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4ef804ce40 con 0x7f4f080bfd60 2023-11-06T22:18:58.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.935+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/1522081371 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f080c9420 con 0x7f4ef405f270 2023-11-06T22:18:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.936+0000 7f4eeffff700 1 -- 172.21.15.33:0/1522081371 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4f080c9420 con 0x7f4ef405f270 2023-11-06T22:18:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4ef405f270 msgr2=0x7f4ef4061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4ef405f270 0x7f4ef4061730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f4f08151c60 tx=0x7f4efc009040).stop 2023-11-06T22:18:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/1522081371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f080bfd60 msgr2=0x7f4f080ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f080bfd60 0x7f4f080ab760 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 rx=0x7f4ef800b340 tx=0x7f4ef8017000).stop 2023-11-06T22:18:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/1522081371 shutdown_connections 2023-11-06T22:18:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f4ef405f270 0x7f4ef4061730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f08154210 0x7f4f080c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f080bfd60 0x7f4f080ab760 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 --2- 172.21.15.33:0/1522081371 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f4f080ca610 0x7f4f08156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/1522081371 >> 172.21.15.33:0/1522081371 conn(0x7f4f0805e990 msgr2=0x7f4f081511e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/1522081371 shutdown_connections 2023-11-06T22:18:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:58.938+0000 7f4f0e3ff700 1 -- 172.21.15.33:0/1522081371 wait complete. 2023-11-06T22:18:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 4 to deep-scrub 2023-11-06T22:18:58.958 INFO:tasks.ceph:Scrubbing osd.5 2023-11-06T22:18:58.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-06T22:18:59.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.156+0000 7fbb8fdff700 1 Processor -- start 2023-11-06T22:18:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.156+0000 7fbb8fdff700 1 -- start start 2023-11-06T22:18:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.157+0000 7fbb8fdff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbb88153ed0 0x7fbb880aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.157+0000 7fbb8fdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbb881542b0 0x7fbb88157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.157+0000 7fbb8fdff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb880c02c0 con 0x7fbb880631c0 2023-11-06T22:18:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.157+0000 7fbb8fdff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb881581e0 con 0x7fbb88153ed0 2023-11-06T22:18:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.157+0000 7fbb8fdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb88158460 con 0x7fbb881542b0 2023-11-06T22:18:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.157+0000 7fbb8e39c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbb880631c0 0x7fbb880635e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46648/0 (socket says 172.21.15.33:46648) 2023-11-06T22:18:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.157+0000 7fbb8e39c700 1 -- 172.21.15.33:0/1343495846 learned_addr learned my addr 172.21.15.33:0/1343495846 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:18:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.158+0000 7fbb8d39a700 1 --2- 172.21.15.33:0/1343495846 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbb881542b0 0x7fbb88157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.158+0000 7fbb8db9b700 1 --2- 172.21.15.33:0/1343495846 >> v2:172.21.15.69:3300/0 conn(0x7fbb88153ed0 0x7fbb880aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.158+0000 7fbb8cb99700 1 -- 172.21.15.33:0/1343495846 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945215098 0 0) 0x7fbb880c02c0 con 0x7fbb880631c0 2023-11-06T22:18:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.158+0000 7fbb8cb99700 1 -- 172.21.15.33:0/1343495846 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb740033f0 con 0x7fbb880631c0 2023-11-06T22:18:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.158+0000 7fbb8d39a700 1 -- 172.21.15.33:0/1343495846 >> v2:172.21.15.69:3300/0 conn(0x7fbb88153ed0 msgr2=0x7fbb880aa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.158+0000 7fbb8d39a700 1 --2- 172.21.15.33:0/1343495846 >> v2:172.21.15.69:3300/0 conn(0x7fbb88153ed0 0x7fbb880aa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.158+0000 7fbb8d39a700 1 -- 172.21.15.33:0/1343495846 >> v1:172.21.15.33:6789/0 conn(0x7fbb880631c0 legacy=0x7fbb880635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.158+0000 7fbb8d39a700 1 -- 172.21.15.33:0/1343495846 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88158740 con 0x7fbb881542b0 2023-11-06T22:18:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.158+0000 7fbb8db9b700 1 --2- 172.21.15.33:0/1343495846 >> v2:172.21.15.69:3300/0 conn(0x7fbb88153ed0 0x7fbb880aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:18:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.158+0000 7fbb8d39a700 1 --2- 172.21.15.33:0/1343495846 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbb881542b0 0x7fbb88157cf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fbb78009760 tx=0x7fbb78009060).ready entity=mon.2 client_cookie=d2630f058158629 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8cb99700 1 -- 172.21.15.33:0/1343495846 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7800d4a0 con 0x7fbb881542b0 2023-11-06T22:18:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8cb99700 1 -- 172.21.15.33:0/1343495846 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7800d600 con 0x7fbb881542b0 2023-11-06T22:18:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8cb99700 1 -- 172.21.15.33:0/1343495846 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7800d980 con 0x7fbb881542b0 2023-11-06T22:18:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8fdff700 1 -- 172.21.15.33:0/1343495846 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbb881542b0 msgr2=0x7fbb88157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8fdff700 1 --2- 172.21.15.33:0/1343495846 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbb881542b0 0x7fbb88157cf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fbb78009760 tx=0x7fbb78009060).stop 2023-11-06T22:18:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8fdff700 1 -- 172.21.15.33:0/1343495846 shutdown_connections 2023-11-06T22:18:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8fdff700 1 --2- 172.21.15.33:0/1343495846 >> v2:172.21.15.69:3300/0 conn(0x7fbb88153ed0 0x7fbb880aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8fdff700 1 --2- 172.21.15.33:0/1343495846 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbb881542b0 0x7fbb88157cf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8fdff700 1 -- 172.21.15.33:0/1343495846 >> 172.21.15.33:0/1343495846 conn(0x7fbb8805ec80 msgr2=0x7fbb880cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8fdff700 1 -- 172.21.15.33:0/1343495846 shutdown_connections 2023-11-06T22:18:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.159+0000 7fbb8fdff700 1 -- 172.21.15.33:0/1343495846 wait complete. 2023-11-06T22:18:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.161+0000 7fbb8fdff700 1 Processor -- start 2023-11-06T22:18:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.161+0000 7fbb8fdff700 1 -- start start 2023-11-06T22:18:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8fdff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbb880631c0 0x7fbb881566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8fdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb88153ed0 0x7fbb88156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8fdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb881542b0 0x7fbb881570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8fdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb88156230 con 0x7fbb881542b0 2023-11-06T22:18:59.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8fdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb88157590 con 0x7fbb88153ed0 2023-11-06T22:18:59.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8fdff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb880ad0b0 con 0x7fbb880631c0 2023-11-06T22:18:59.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8e39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb881542b0 0x7fbb881570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8e39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb881542b0 0x7fbb881570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52602/0 (socket says 172.21.15.33:52602) 2023-11-06T22:18:59.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8e39c700 1 -- 172.21.15.33:0/2087264231 learned_addr learned my addr 172.21.15.33:0/2087264231 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:59.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8d39a700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb88153ed0 0x7fbb88156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.162+0000 7fbb8db9b700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbb880631c0 0x7fbb881566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb8e39c700 1 -- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbb880631c0 msgr2=0x7fbb881566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb8e39c700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbb880631c0 0x7fbb881566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb8e39c700 1 -- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb88153ed0 msgr2=0x7fbb88156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb8e39c700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb88153ed0 0x7fbb88156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb8e39c700 1 -- 172.21.15.33:0/2087264231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840092a0 con 0x7fbb881542b0 2023-11-06T22:18:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb8e39c700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb881542b0 0x7fbb881570a0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7fbb80005ad0 tx=0x7fbb80001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb7dffb700 1 -- 172.21.15.33:0/2087264231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80005d30 con 0x7fbb881542b0 2023-11-06T22:18:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb7dffb700 1 -- 172.21.15.33:0/2087264231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb80005e90 con 0x7fbb881542b0 2023-11-06T22:18:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb780092a0 con 0x7fbb881542b0 2023-11-06T22:18:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb7dffb700 1 -- 172.21.15.33:0/2087264231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80007320 con 0x7fbb881542b0 2023-11-06T22:18:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.163+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb88155be0 con 0x7fbb881542b0 2023-11-06T22:18:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.164+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb88158a20 con 0x7fbb881542b0 2023-11-06T22:18:59.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.165+0000 7fbb7dffb700 1 -- 172.21.15.33:0/2087264231 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbb80007710 con 0x7fbb881542b0 2023-11-06T22:18:59.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.166+0000 7fbb7dffb700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbb6c05f270 0x7fbb6c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.166+0000 7fbb7dffb700 1 -- 172.21.15.33:0/2087264231 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fbb8007ab40 con 0x7fbb881542b0 2023-11-06T22:18:59.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.166+0000 7fbb7dffb700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] conn(0x7fbb6c065c20 0x7fbb6c068080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.166+0000 7fbb7dffb700 1 -- 172.21.15.33:0/2087264231 --> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb6c068700 con 0x7fbb6c065c20 2023-11-06T22:18:59.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.166+0000 7fbb7dffb700 1 -- 172.21.15.33:0/2087264231 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb8007af10 con 0x7fbb881542b0 2023-11-06T22:18:59.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.166+0000 7fbb8db9b700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbb6c05f270 0x7fbb6c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.166+0000 7fbb8d39a700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] conn(0x7fbb6c065c20 0x7fbb6c068080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.167+0000 7fbb8db9b700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbb6c05f270 0x7fbb6c061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fbb84009dc0 tx=0x7fbb84009060).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.167+0000 7fbb8d39a700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] conn(0x7fbb6c065c20 0x7fbb6c068080 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.168+0000 7fbb7dffb700 1 -- 172.21.15.33:0/2087264231 <== osd.5 v2:172.21.15.69:6816/115642 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fbb6c068700 con 0x7fbb6c065c20 2023-11-06T22:18:59.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.182+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 --> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbb88156370 con 0x7fbb6c065c20 2023-11-06T22:18:59.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.183+0000 7fbb7dffb700 1 -- 172.21.15.33:0/2087264231 <== osd.5 v2:172.21.15.69:6816/115642 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbb88156370 con 0x7fbb6c065c20 2023-11-06T22:18:59.185 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:18:59.186 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T22:18:59.186 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:18:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.183+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] conn(0x7fbb6c065c20 msgr2=0x7fbb6c068080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.183+0000 7fbb8fdff700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:6816/115642,v1:172.21.15.69:6817/115642] conn(0x7fbb6c065c20 0x7fbb6c068080 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.183+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbb6c05f270 msgr2=0x7fbb6c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.183+0000 7fbb8fdff700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbb6c05f270 0x7fbb6c061730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fbb84009dc0 tx=0x7fbb84009060).stop 2023-11-06T22:18:59.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.183+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb881542b0 msgr2=0x7fbb881570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.183+0000 7fbb8fdff700 1 --2- 172.21.15.33:0/2087264231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb881542b0 0x7fbb881570a0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7fbb80005ad0 tx=0x7fbb80001650).stop 2023-11-06T22:18:59.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.183+0000 7fbb8e39c700 1 -- 172.21.15.33:0/2087264231 reap_dead start 2023-11-06T22:18:59.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.183+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 shutdown_connections 2023-11-06T22:18:59.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.184+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 >> 172.21.15.33:0/2087264231 conn(0x7fbb8805ec80 msgr2=0x7fbb88064680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:59.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.184+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 shutdown_connections 2023-11-06T22:18:59.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.184+0000 7fbb8fdff700 1 -- 172.21.15.33:0/2087264231 wait complete. 2023-11-06T22:18:59.202 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-06T22:18:59.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.397+0000 7f3caadff700 1 Processor -- start 2023-11-06T22:18:59.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.397+0000 7f3caadff700 1 -- start start 2023-11-06T22:18:59.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.398+0000 7f3caadff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3ca4151f30 0x7f3ca4068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.398+0000 7f3caadff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca4152310 0x7f3ca40bdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.398+0000 7f3caadff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3ca40be0a0 0x7f3ca4154b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3caadff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca40be550 con 0x7f3ca4151f30 2023-11-06T22:18:59.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3caadff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca4068100 con 0x7f3ca40be0a0 2023-11-06T22:18:59.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3caadff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca40658c0 con 0x7f3ca4152310 2023-11-06T22:18:59.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca8b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3ca4151f30 0x7f3ca4068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca8b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3ca4151f30 0x7f3ca4068590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52604/0 (socket says 172.21.15.33:52604) 2023-11-06T22:18:59.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca8b9b700 1 -- 172.21.15.33:0/261860987 learned_addr learned my addr 172.21.15.33:0/261860987 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:59.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca939c700 1 --2- 172.21.15.33:0/261860987 >> v2:172.21.15.69:3300/0 conn(0x7f3ca40be0a0 0x7f3ca4154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca3fff700 1 --2- 172.21.15.33:0/261860987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca4152310 0x7f3ca40bdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca8b9b700 1 -- 172.21.15.33:0/261860987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca4152310 msgr2=0x7f3ca40bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:59.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca8b9b700 1 --2- 172.21.15.33:0/261860987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca4152310 0x7f3ca40bdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca8b9b700 1 -- 172.21.15.33:0/261860987 >> v2:172.21.15.69:3300/0 conn(0x7f3ca40be0a0 msgr2=0x7f3ca4154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:59.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca8b9b700 1 --2- 172.21.15.33:0/261860987 >> v2:172.21.15.69:3300/0 conn(0x7f3ca40be0a0 0x7f3ca4154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca8b9b700 1 -- 172.21.15.33:0/261860987 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca4068a80 con 0x7f3ca4151f30 2023-11-06T22:18:59.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca939c700 1 --2- 172.21.15.33:0/261860987 >> v2:172.21.15.69:3300/0 conn(0x7f3ca40be0a0 0x7f3ca4154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:59.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.399+0000 7f3ca3fff700 1 --2- 172.21.15.33:0/261860987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca4152310 0x7f3ca40bdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:59.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3ca8b9b700 1 --2- 172.21.15.33:0/261860987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 0x7f3ca4068590 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 rx=0x7f3ca40be7a0 tx=0x7f3c980090f0).ready entity=mon.0 client_cookie=bad8062d8d9dde7f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3ca37fe700 1 -- 172.21.15.33:0/261860987 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800b600 con 0x7f3ca4151f30 2023-11-06T22:18:59.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3ca37fe700 1 -- 172.21.15.33:0/261860987 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9800b760 con 0x7f3ca4151f30 2023-11-06T22:18:59.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3ca37fe700 1 -- 172.21.15.33:0/261860987 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800bae0 con 0x7f3ca4151f30 2023-11-06T22:18:59.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3caadff700 1 -- 172.21.15.33:0/261860987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 msgr2=0x7f3ca4068590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3caadff700 1 --2- 172.21.15.33:0/261860987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 0x7f3ca4068590 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 rx=0x7f3ca40be7a0 tx=0x7f3c980090f0).stop 2023-11-06T22:18:59.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3caadff700 1 -- 172.21.15.33:0/261860987 shutdown_connections 2023-11-06T22:18:59.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3caadff700 1 --2- 172.21.15.33:0/261860987 >> v2:172.21.15.69:3300/0 conn(0x7f3ca40be0a0 0x7f3ca4154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3caadff700 1 --2- 172.21.15.33:0/261860987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 0x7f3ca4068590 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3caadff700 1 --2- 172.21.15.33:0/261860987 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca4152310 0x7f3ca40bdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.400+0000 7f3caadff700 1 -- 172.21.15.33:0/261860987 >> 172.21.15.33:0/261860987 conn(0x7f3ca40c56f0 msgr2=0x7f3ca40c5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:59.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.401+0000 7f3caadff700 1 -- 172.21.15.33:0/261860987 shutdown_connections 2023-11-06T22:18:59.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.401+0000 7f3caadff700 1 -- 172.21.15.33:0/261860987 wait complete. 2023-11-06T22:18:59.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.402+0000 7f3caadff700 1 Processor -- start 2023-11-06T22:18:59.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.402+0000 7f3caadff700 1 -- start start 2023-11-06T22:18:59.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.403+0000 7f3caadff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca40be0a0 0x7f3ca414ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.403+0000 7f3caadff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 0x7f3ca40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.403+0000 7f3caadff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4152310 0x7f3ca4164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.403+0000 7f3caadff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca4165260 con 0x7f3ca4151f30 2023-11-06T22:18:59.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.403+0000 7f3caadff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca4165540 con 0x7f3ca4152310 2023-11-06T22:18:59.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.403+0000 7f3caadff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca40a4980 con 0x7f3ca40be0a0 2023-11-06T22:18:59.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.403+0000 7f3ca3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 0x7f3ca40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.403+0000 7f3ca3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 0x7f3ca40a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52610/0 (socket says 172.21.15.33:52610) 2023-11-06T22:18:59.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca3fff700 1 -- 172.21.15.33:0/3499486137 learned_addr learned my addr 172.21.15.33:0/3499486137 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:59.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca939c700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4152310 0x7f3ca4164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca8b9b700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca40be0a0 0x7f3ca414ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca3fff700 1 -- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca40be0a0 msgr2=0x7f3ca414ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca3fff700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca40be0a0 0x7f3ca414ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca3fff700 1 -- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4152310 msgr2=0x7f3ca4164680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca3fff700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4152310 0x7f3ca4164680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca3fff700 1 -- 172.21.15.33:0/3499486137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c98009370 con 0x7f3ca4151f30 2023-11-06T22:18:59.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca939c700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4152310 0x7f3ca4164680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:59.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca8b9b700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca40be0a0 0x7f3ca414ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:18:59.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca3fff700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 0x7f3ca40a5460 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 rx=0x7f3c9000c880 tx=0x7f3c90008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca0ff9700 1 -- 172.21.15.33:0/3499486137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90010580 con 0x7f3ca4151f30 2023-11-06T22:18:59.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca0ff9700 1 -- 172.21.15.33:0/3499486137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c900106e0 con 0x7f3ca4151f30 2023-11-06T22:18:59.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3ca0ff9700 1 -- 172.21.15.33:0/3499486137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90010a60 con 0x7f3ca4151f30 2023-11-06T22:18:59.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.404+0000 7f3caadff700 1 -- 172.21.15.33:0/3499486137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca40a4c60 con 0x7f3ca4151f30 2023-11-06T22:18:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.405+0000 7f3caadff700 1 -- 172.21.15.33:0/3499486137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca40ae690 con 0x7f3ca4151f30 2023-11-06T22:18:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.406+0000 7f3caadff700 1 -- 172.21.15.33:0/3499486137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca40a52b0 con 0x7f3ca4151f30 2023-11-06T22:18:59.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.406+0000 7f3ca0ff9700 1 -- 172.21.15.33:0/3499486137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f3c9001b460 con 0x7f3ca4151f30 2023-11-06T22:18:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.408+0000 7f3ca0ff9700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c8805f270 0x7f3c88061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.408+0000 7f3ca0ff9700 1 -- 172.21.15.33:0/3499486137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f3c9007a2d0 con 0x7f3ca4151f30 2023-11-06T22:18:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.409+0000 7f3ca8b9b700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c8805f270 0x7f3c88061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.409+0000 7f3ca8b9b700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c8805f270 0x7f3c88061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f3c9800b3c0 tx=0x7f3c98017000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.412+0000 7f3ca0ff9700 1 -- 172.21.15.33:0/3499486137 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3c9004b420 con 0x7f3ca4151f30 2023-11-06T22:18:59.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.592+0000 7f3caadff700 1 -- 172.21.15.33:0/3499486137 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ca4155750 con 0x7f3c8805f270 2023-11-06T22:18:59.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.593+0000 7f3ca0ff9700 1 -- 172.21.15.33:0/3499486137 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3ca4155750 con 0x7f3c8805f270 2023-11-06T22:18:59.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 -- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c8805f270 msgr2=0x7f3c88061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c8805f270 0x7f3c88061730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f3c9800b3c0 tx=0x7f3c98017000).stop 2023-11-06T22:18:59.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 -- 172.21.15.33:0/3499486137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 msgr2=0x7f3ca40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 0x7f3ca40a5460 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 rx=0x7f3c9000c880 tx=0x7f3c90008040).stop 2023-11-06T22:18:59.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 -- 172.21.15.33:0/3499486137 shutdown_connections 2023-11-06T22:18:59.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f3c8805f270 0x7f3c88061730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f3ca40be0a0 0x7f3ca414ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4152310 0x7f3ca4164680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 --2- 172.21.15.33:0/3499486137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca4151f30 0x7f3ca40a5460 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 -- 172.21.15.33:0/3499486137 >> 172.21.15.33:0/3499486137 conn(0x7f3ca40c56f0 msgr2=0x7f3ca40637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 -- 172.21.15.33:0/3499486137 shutdown_connections 2023-11-06T22:18:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.595+0000 7f3caadff700 1 -- 172.21.15.33:0/3499486137 wait complete. 2023-11-06T22:18:59.601 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 5 to deep-scrub 2023-11-06T22:18:59.615 INFO:tasks.ceph:Scrubbing osd.6 2023-11-06T22:18:59.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-06T22:18:59.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.813+0000 7fe79edff700 1 Processor -- start 2023-11-06T22:18:59.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.813+0000 7fe79edff700 1 -- start start 2023-11-06T22:18:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.814+0000 7fe79edff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe79813a960 0x7fe798155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.814+0000 7fe79edff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7980c9340 0x7fe798066490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.814+0000 7fe79edff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe798066e10 con 0x7fe798066980 2023-11-06T22:18:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.814+0000 7fe79edff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe798064cc0 con 0x7fe7980c9340 2023-11-06T22:18:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.814+0000 7fe79edff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe798155930 con 0x7fe79813a960 2023-11-06T22:18:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.815+0000 7fe797fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7980c9340 0x7fe798066490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.815+0000 7fe797fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7980c9340 0x7fe798066490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52616/0 (socket says 172.21.15.33:52616) 2023-11-06T22:18:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.815+0000 7fe797fff700 1 -- 172.21.15.33:0/2317646352 learned_addr learned my addr 172.21.15.33:0/2317646352 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.815+0000 7fe79cb9b700 1 --2- 172.21.15.33:0/2317646352 >> v2:172.21.15.69:3300/0 conn(0x7fe79813a960 0x7fe798155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.815+0000 7fe797fff700 1 -- 172.21.15.33:0/2317646352 >> v2:172.21.15.69:3300/0 conn(0x7fe79813a960 msgr2=0x7fe798155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:18:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.815+0000 7fe797fff700 1 --2- 172.21.15.33:0/2317646352 >> v2:172.21.15.69:3300/0 conn(0x7fe79813a960 0x7fe798155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.815+0000 7fe797fff700 1 -- 172.21.15.33:0/2317646352 >> v1:172.21.15.69:6789/0 conn(0x7fe798066980 legacy=0x7fe7980679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.815+0000 7fe797fff700 1 -- 172.21.15.33:0/2317646352 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798063950 con 0x7fe7980c9340 2023-11-06T22:18:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.815+0000 7fe797fff700 1 --2- 172.21.15.33:0/2317646352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7980c9340 0x7fe798066490 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 rx=0x7fe79805f080 tx=0x7fe7840090f0).ready entity=mon.0 client_cookie=b2494e678c52647c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe7977fe700 1 -- 172.21.15.33:0/2317646352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78400b520 con 0x7fe7980c9340 2023-11-06T22:18:59.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe7977fe700 1 -- 172.21.15.33:0/2317646352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78400b680 con 0x7fe7980c9340 2023-11-06T22:18:59.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe7977fe700 1 -- 172.21.15.33:0/2317646352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78400ba00 con 0x7fe7980c9340 2023-11-06T22:18:59.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe79edff700 1 -- 172.21.15.33:0/2317646352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7980c9340 msgr2=0x7fe798066490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe79edff700 1 --2- 172.21.15.33:0/2317646352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7980c9340 0x7fe798066490 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 rx=0x7fe79805f080 tx=0x7fe7840090f0).stop 2023-11-06T22:18:59.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe79edff700 1 -- 172.21.15.33:0/2317646352 shutdown_connections 2023-11-06T22:18:59.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe79edff700 1 --2- 172.21.15.33:0/2317646352 >> v2:172.21.15.69:3300/0 conn(0x7fe79813a960 0x7fe798155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe79edff700 1 --2- 172.21.15.33:0/2317646352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7980c9340 0x7fe798066490 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe79edff700 1 -- 172.21.15.33:0/2317646352 >> 172.21.15.33:0/2317646352 conn(0x7fe79805ec70 msgr2=0x7fe7980c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:59.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe79edff700 1 -- 172.21.15.33:0/2317646352 shutdown_connections 2023-11-06T22:18:59.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.816+0000 7fe79edff700 1 -- 172.21.15.33:0/2317646352 wait complete. 2023-11-06T22:18:59.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.818+0000 7fe79edff700 1 Processor -- start 2023-11-06T22:18:59.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.818+0000 7fe79edff700 1 -- start start 2023-11-06T22:18:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe79edff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe798066980 0x7fe798165750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe79edff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7980c9340 0x7fe7980ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe79edff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79813a960 0x7fe7980c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe79edff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980c7850 con 0x7fe79813a960 2023-11-06T22:18:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe79edff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980ac1e0 con 0x7fe798066980 2023-11-06T22:18:59.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe79edff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980ac4c0 con 0x7fe7980c9340 2023-11-06T22:18:59.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe79d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79813a960 0x7fe7980c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe79d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79813a960 0x7fe7980c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52632/0 (socket says 172.21.15.33:52632) 2023-11-06T22:18:59.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe79d39c700 1 -- 172.21.15.33:0/2203087906 learned_addr learned my addr 172.21.15.33:0/2203087906 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:18:59.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe797fff700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7980c9340 0x7fe7980ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.819+0000 7fe79cb9b700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe798066980 0x7fe798165750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe79d39c700 1 -- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7980c9340 msgr2=0x7fe7980ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe79d39c700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7980c9340 0x7fe7980ae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe79d39c700 1 -- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe798066980 msgr2=0x7fe798165750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe79d39c700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe798066980 0x7fe798165750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe79d39c700 1 -- 172.21.15.33:0/2203087906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe784009370 con 0x7fe79813a960 2023-11-06T22:18:59.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe797fff700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fe7980c9340 0x7fe7980ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:18:59.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe79d39c700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79813a960 0x7fe7980c7070 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7fe79000c7c0 tx=0x7fe790001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe794ff9700 1 -- 172.21.15.33:0/2203087906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79000e4a0 con 0x7fe79813a960 2023-11-06T22:18:59.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe794ff9700 1 -- 172.21.15.33:0/2203087906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79000e600 con 0x7fe79813a960 2023-11-06T22:18:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7980ac7a0 con 0x7fe79813a960 2023-11-06T22:18:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe794ff9700 1 -- 172.21.15.33:0/2203087906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79000e980 con 0x7fe79813a960 2023-11-06T22:18:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.820+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7980c7560 con 0x7fe79813a960 2023-11-06T22:18:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.821+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe798065f30 con 0x7fe79813a960 2023-11-06T22:18:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.822+0000 7fe794ff9700 1 -- 172.21.15.33:0/2203087906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fe790012400 con 0x7fe79813a960 2023-11-06T22:18:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.823+0000 7fe794ff9700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe77c05f270 0x7fe77c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.823+0000 7fe794ff9700 1 -- 172.21.15.33:0/2203087906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fe79007a570 con 0x7fe79813a960 2023-11-06T22:18:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.824+0000 7fe794ff9700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] conn(0x7fe77c065c20 0x7fe77c068080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:18:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.824+0000 7fe794ff9700 1 -- 172.21.15.33:0/2203087906 --> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe77c068700 con 0x7fe77c065c20 2023-11-06T22:18:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.824+0000 7fe794ff9700 1 -- 172.21.15.33:0/2203087906 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe79007a860 con 0x7fe79813a960 2023-11-06T22:18:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.824+0000 7fe79cb9b700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe77c05f270 0x7fe77c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.824+0000 7fe797fff700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] conn(0x7fe77c065c20 0x7fe77c068080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:18:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.824+0000 7fe79cb9b700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe77c05f270 0x7fe77c061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe78c009c30 tx=0x7fe78c009060).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.825+0000 7fe797fff700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] conn(0x7fe77c065c20 0x7fe77c068080 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:18:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.826+0000 7fe794ff9700 1 -- 172.21.15.33:0/2203087906 <== osd.6 v2:172.21.15.69:6800/115621 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fe77c068700 con 0x7fe77c065c20 2023-11-06T22:18:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.840+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 --> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe7980accc0 con 0x7fe77c065c20 2023-11-06T22:18:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.840+0000 7fe794ff9700 1 -- 172.21.15.33:0/2203087906 <== osd.6 v2:172.21.15.69:6800/115621 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe7980accc0 con 0x7fe77c065c20 2023-11-06T22:18:59.843 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:18:59.843 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T22:18:59.843 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:18:59.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.840+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] conn(0x7fe77c065c20 msgr2=0x7fe77c068080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.840+0000 7fe79edff700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:6800/115621,v1:172.21.15.69:6802/115621] conn(0x7fe77c065c20 0x7fe77c068080 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:18:59.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.841+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe77c05f270 msgr2=0x7fe77c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.841+0000 7fe79edff700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fe77c05f270 0x7fe77c061730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe78c009c30 tx=0x7fe78c009060).stop 2023-11-06T22:18:59.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.841+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79813a960 msgr2=0x7fe7980c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:18:59.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.841+0000 7fe79edff700 1 --2- 172.21.15.33:0/2203087906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79813a960 0x7fe7980c7070 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7fe79000c7c0 tx=0x7fe790001510).stop 2023-11-06T22:18:59.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.841+0000 7fe79d39c700 1 -- 172.21.15.33:0/2203087906 reap_dead start 2023-11-06T22:18:59.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.841+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 shutdown_connections 2023-11-06T22:18:59.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.841+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 >> 172.21.15.33:0/2203087906 conn(0x7fe79805ec70 msgr2=0x7fe7980aa700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:18:59.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.841+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 shutdown_connections 2023-11-06T22:18:59.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:18:59.841+0000 7fe79edff700 1 -- 172.21.15.33:0/2203087906 wait complete. 2023-11-06T22:18:59.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-06T22:19:00.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.054+0000 7f808c1ff700 1 Processor -- start 2023-11-06T22:19:00.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.055+0000 7f808c1ff700 1 -- start start 2023-11-06T22:19:00.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.056+0000 7f808c1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8084154210 0x7f80840a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.056+0000 7f808c1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80840bfd60 0x7f80840c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.056+0000 7f808c1ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840c06d0 con 0x7f80840c9f00 2023-11-06T22:19:00.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.056+0000 7f808c1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80840caaa0 con 0x7f8084154210 2023-11-06T22:19:00.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.056+0000 7f808c1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808405efc0 con 0x7f80840bfd60 2023-11-06T22:19:00.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.056+0000 7f8089f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8084154210 0x7f80840a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.056+0000 7f8089f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8084154210 0x7f80840a7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52638/0 (socket says 172.21.15.33:52638) 2023-11-06T22:19:00.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.056+0000 7f8089f9b700 1 -- 172.21.15.33:0/2804758436 learned_addr learned my addr 172.21.15.33:0/2804758436 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:00.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.056+0000 7f808979a700 1 --2- 172.21.15.33:0/2804758436 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80840bfd60 0x7f80840c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f8089f9b700 1 -- 172.21.15.33:0/2804758436 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80840bfd60 msgr2=0x7f80840c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:00.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f8089f9b700 1 --2- 172.21.15.33:0/2804758436 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80840bfd60 0x7f80840c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f8089f9b700 1 -- 172.21.15.33:0/2804758436 >> v1:172.21.15.69:6789/0 conn(0x7f80840c9f00 legacy=0x7f80840ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:00.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f8089f9b700 1 -- 172.21.15.33:0/2804758436 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840c8ea0 con 0x7f8084154210 2023-11-06T22:19:00.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f808979a700 1 --2- 172.21.15.33:0/2804758436 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80840bfd60 0x7f80840c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-06T22:19:00.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f8089f9b700 1 --2- 172.21.15.33:0/2804758436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8084154210 0x7f80840a7bd0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 rx=0x7f8080006fd0 tx=0x7f80800090f0).ready entity=mon.0 client_cookie=6abcc1a0f1a71145 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:00.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f8088f99700 1 -- 172.21.15.33:0/2804758436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808000b600 con 0x7f8084154210 2023-11-06T22:19:00.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f8088f99700 1 -- 172.21.15.33:0/2804758436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808000b760 con 0x7f8084154210 2023-11-06T22:19:00.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f8088f99700 1 -- 172.21.15.33:0/2804758436 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808000bae0 con 0x7f8084154210 2023-11-06T22:19:00.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f808c1ff700 1 -- 172.21.15.33:0/2804758436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8084154210 msgr2=0x7f80840a7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.057+0000 7f808c1ff700 1 --2- 172.21.15.33:0/2804758436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8084154210 0x7f80840a7bd0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 rx=0x7f8080006fd0 tx=0x7f80800090f0).stop 2023-11-06T22:19:00.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.058+0000 7f808c1ff700 1 -- 172.21.15.33:0/2804758436 shutdown_connections 2023-11-06T22:19:00.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.058+0000 7f808c1ff700 1 --2- 172.21.15.33:0/2804758436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8084154210 0x7f80840a7bd0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.058+0000 7f808c1ff700 1 --2- 172.21.15.33:0/2804758436 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f80840bfd60 0x7f80840c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.058+0000 7f808c1ff700 1 -- 172.21.15.33:0/2804758436 >> 172.21.15.33:0/2804758436 conn(0x7f808405e990 msgr2=0x7f80840cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.058+0000 7f808c1ff700 1 -- 172.21.15.33:0/2804758436 shutdown_connections 2023-11-06T22:19:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.058+0000 7f808c1ff700 1 -- 172.21.15.33:0/2804758436 wait complete. 2023-11-06T22:19:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.059+0000 7f808c1ff700 1 Processor -- start 2023-11-06T22:19:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.059+0000 7f808c1ff700 1 -- start start 2023-11-06T22:19:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.060+0000 7f808c1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80840bfd60 0x7f808414f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.060+0000 7f808c1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80840c9f00 0x7f80840b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.060+0000 7f808c1ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8084154210 0x7f8084064050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.060+0000 7f808c1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808414fda0 con 0x7f80840bfd60 2023-11-06T22:19:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.060+0000 7f808c1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80840b0ca0 con 0x7f80840c9f00 2023-11-06T22:19:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.060+0000 7f808c1ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80840af730 con 0x7f8084154210 2023-11-06T22:19:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8089f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80840bfd60 0x7f808414f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8089f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80840bfd60 0x7f808414f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52642/0 (socket says 172.21.15.33:52642) 2023-11-06T22:19:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8089f9b700 1 -- 172.21.15.33:0/2484561830 learned_addr learned my addr 172.21.15.33:0/2484561830 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f808979a700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80840c9f00 0x7f80840b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f808a79c700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8084154210 0x7f8084064050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8089f9b700 1 -- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8084154210 msgr2=0x7f8084064050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8089f9b700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8084154210 0x7f8084064050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8089f9b700 1 -- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80840c9f00 msgr2=0x7f80840b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8089f9b700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80840c9f00 0x7f80840b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8089f9b700 1 -- 172.21.15.33:0/2484561830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8080009370 con 0x7f80840bfd60 2023-11-06T22:19:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8089f9b700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80840bfd60 0x7f808414f8b0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7f8084083660 tx=0x7f8080017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8079ffb700 1 -- 172.21.15.33:0/2484561830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808000b8d0 con 0x7f80840bfd60 2023-11-06T22:19:00.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f8079ffb700 1 -- 172.21.15.33:0/2484561830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808000ba30 con 0x7f80840bfd60 2023-11-06T22:19:00.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.061+0000 7f808c1ff700 1 -- 172.21.15.33:0/2484561830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80840afa10 con 0x7f80840bfd60 2023-11-06T22:19:00.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.062+0000 7f8079ffb700 1 -- 172.21.15.33:0/2484561830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808001b620 con 0x7f80840bfd60 2023-11-06T22:19:00.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.062+0000 7f808c1ff700 1 -- 172.21.15.33:0/2484561830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8084167ad0 con 0x7f80840bfd60 2023-11-06T22:19:00.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.063+0000 7f808c1ff700 1 -- 172.21.15.33:0/2484561830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808406c6c0 con 0x7f80840bfd60 2023-11-06T22:19:00.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.063+0000 7f8079ffb700 1 -- 172.21.15.33:0/2484561830 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f808001b990 con 0x7f80840bfd60 2023-11-06T22:19:00.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.064+0000 7f8079ffb700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f806805f270 0x7f8068061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.064+0000 7f8079ffb700 1 -- 172.21.15.33:0/2484561830 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f808007bdf0 con 0x7f80840bfd60 2023-11-06T22:19:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.065+0000 7f808979a700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f806805f270 0x7f8068061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.065+0000 7f808979a700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f806805f270 0x7f8068061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f80841532e0 tx=0x7f8074008040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.068+0000 7f8079ffb700 1 -- 172.21.15.33:0/2484561830 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f808004cfa0 con 0x7f80840bfd60 2023-11-06T22:19:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.252+0000 7f808c1ff700 1 -- 172.21.15.33:0/2484561830 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f808405eda0 con 0x7f806805f270 2023-11-06T22:19:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.253+0000 7f8079ffb700 1 -- 172.21.15.33:0/2484561830 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f808405eda0 con 0x7f806805f270 2023-11-06T22:19:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 -- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f806805f270 msgr2=0x7f8068061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f806805f270 0x7f8068061730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f80841532e0 tx=0x7f8074008040).stop 2023-11-06T22:19:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 -- 172.21.15.33:0/2484561830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80840bfd60 msgr2=0x7f808414f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80840bfd60 0x7f808414f8b0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7f8084083660 tx=0x7f8080017000).stop 2023-11-06T22:19:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 -- 172.21.15.33:0/2484561830 shutdown_connections 2023-11-06T22:19:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f806805f270 0x7f8068061730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8084154210 0x7f8084064050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80840bfd60 0x7f808414f8b0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 --2- 172.21.15.33:0/2484561830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80840c9f00 0x7f80840b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 -- 172.21.15.33:0/2484561830 >> 172.21.15.33:0/2484561830 conn(0x7f808405e990 msgr2=0x7f80840c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 -- 172.21.15.33:0/2484561830 shutdown_connections 2023-11-06T22:19:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.255+0000 7f808c1ff700 1 -- 172.21.15.33:0/2484561830 wait complete. 2023-11-06T22:19:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 6 to deep-scrub 2023-11-06T22:19:00.275 INFO:tasks.ceph:Scrubbing osd.7 2023-11-06T22:19:00.275 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-06T22:19:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.473+0000 7f41a3dff700 1 Processor -- start 2023-11-06T22:19:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.473+0000 7f41a3dff700 1 -- start start 2023-11-06T22:19:00.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.474+0000 7f41a3dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f419c153ed0 0x7f419c158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.474+0000 7f41a3dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419c0c8ee0 0x7f419c0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.474+0000 7f41a3dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f419c1542b0 0x7f419c0c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.474+0000 7f41a3dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419c0c9f80 con 0x7f419c153ed0 2023-11-06T22:19:00.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.474+0000 7f41a3dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419c0c6000 con 0x7f419c1542b0 2023-11-06T22:19:00.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.474+0000 7f41a3dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c0c9370 con 0x7f419c0c8ee0 2023-11-06T22:19:00.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a1b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f419c153ed0 0x7f419c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a1b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f419c153ed0 0x7f419c158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52644/0 (socket says 172.21.15.33:52644) 2023-11-06T22:19:00.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a1b9b700 1 -- 172.21.15.33:0/3186128103 learned_addr learned my addr 172.21.15.33:0/3186128103 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:00.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a239c700 1 --2- 172.21.15.33:0/3186128103 >> v2:172.21.15.69:3300/0 conn(0x7f419c1542b0 0x7f419c0c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a139a700 1 --2- 172.21.15.33:0/3186128103 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419c0c8ee0 0x7f419c0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a1b9b700 1 -- 172.21.15.33:0/3186128103 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419c0c8ee0 msgr2=0x7f419c0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:00.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a1b9b700 1 --2- 172.21.15.33:0/3186128103 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419c0c8ee0 0x7f419c0ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a1b9b700 1 -- 172.21.15.33:0/3186128103 >> v2:172.21.15.69:3300/0 conn(0x7f419c1542b0 msgr2=0x7f419c0c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:00.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a1b9b700 1 --2- 172.21.15.33:0/3186128103 >> v2:172.21.15.69:3300/0 conn(0x7f419c1542b0 0x7f419c0c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a1b9b700 1 -- 172.21.15.33:0/3186128103 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c0c9650 con 0x7f419c153ed0 2023-11-06T22:19:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.475+0000 7f41a1b9b700 1 --2- 172.21.15.33:0/3186128103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419c153ed0 0x7f419c158600 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 rx=0x7f419c0c53d0 tx=0x7f41980090f0).ready entity=mon.0 client_cookie=6131648d5ebce3b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a0b99700 1 -- 172.21.15.33:0/3186128103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419800b520 con 0x7f419c153ed0 2023-11-06T22:19:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a0b99700 1 -- 172.21.15.33:0/3186128103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419800b680 con 0x7f419c153ed0 2023-11-06T22:19:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a0b99700 1 -- 172.21.15.33:0/3186128103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419800ba00 con 0x7f419c153ed0 2023-11-06T22:19:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a3dff700 1 -- 172.21.15.33:0/3186128103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419c153ed0 msgr2=0x7f419c158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a3dff700 1 --2- 172.21.15.33:0/3186128103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419c153ed0 0x7f419c158600 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 rx=0x7f419c0c53d0 tx=0x7f41980090f0).stop 2023-11-06T22:19:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a3dff700 1 -- 172.21.15.33:0/3186128103 shutdown_connections 2023-11-06T22:19:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a3dff700 1 --2- 172.21.15.33:0/3186128103 >> v2:172.21.15.69:3300/0 conn(0x7f419c1542b0 0x7f419c0c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a3dff700 1 --2- 172.21.15.33:0/3186128103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419c153ed0 0x7f419c158600 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a3dff700 1 --2- 172.21.15.33:0/3186128103 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419c0c8ee0 0x7f419c0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a3dff700 1 -- 172.21.15.33:0/3186128103 >> 172.21.15.33:0/3186128103 conn(0x7f419c05e1b0 msgr2=0x7f419c05e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a3dff700 1 -- 172.21.15.33:0/3186128103 shutdown_connections 2023-11-06T22:19:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.476+0000 7f41a3dff700 1 -- 172.21.15.33:0/3186128103 wait complete. 2023-11-06T22:19:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.478+0000 7f41a3dff700 1 Processor -- start 2023-11-06T22:19:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.478+0000 7f41a3dff700 1 -- start start 2023-11-06T22:19:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a3dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419c153ed0 0x7f419c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a3dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f419c1542b0 0x7f419c0a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a3dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419c164310 0x7f419c164790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a3dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c0a4f10 con 0x7f419c164310 2023-11-06T22:19:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a3dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c0a41b0 con 0x7f419c1542b0 2023-11-06T22:19:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a3dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c0a35a0 con 0x7f419c153ed0 2023-11-06T22:19:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a239c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419c164310 0x7f419c164790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a239c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419c164310 0x7f419c164790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52650/0 (socket says 172.21.15.33:52650) 2023-11-06T22:19:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a239c700 1 -- 172.21.15.33:0/3291507804 learned_addr learned my addr 172.21.15.33:0/3291507804 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a1b9b700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419c153ed0 0x7f419c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.479+0000 7f41a139a700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f419c1542b0 0x7f419c0a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f41a239c700 1 -- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419c153ed0 msgr2=0x7f419c0a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f41a239c700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f419c153ed0 0x7f419c0a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f41a239c700 1 -- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f419c1542b0 msgr2=0x7f419c0a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f41a239c700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f419c1542b0 0x7f419c0a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f41a239c700 1 -- 172.21.15.33:0/3291507804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4198009370 con 0x7f419c164310 2023-11-06T22:19:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f41a239c700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419c164310 0x7f419c164790 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 rx=0x7f4194009df0 tx=0x7f419400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f4191ffb700 1 -- 172.21.15.33:0/3291507804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419400caa0 con 0x7f419c164310 2023-11-06T22:19:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f4191ffb700 1 -- 172.21.15.33:0/3291507804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419400f040 con 0x7f419c164310 2023-11-06T22:19:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419c0a3880 con 0x7f419c164310 2023-11-06T22:19:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f4191ffb700 1 -- 172.21.15.33:0/3291507804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419400ce10 con 0x7f419c164310 2023-11-06T22:19:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.480+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419c0a3e40 con 0x7f419c164310 2023-11-06T22:19:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.481+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f419c06c6c0 con 0x7f419c164310 2023-11-06T22:19:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.482+0000 7f4191ffb700 1 -- 172.21.15.33:0/3291507804 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4194013610 con 0x7f419c164310 2023-11-06T22:19:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.483+0000 7f4191ffb700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f418005f270 0x7f4180061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.483+0000 7f4191ffb700 1 -- 172.21.15.33:0/3291507804 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f419407a980 con 0x7f419c164310 2023-11-06T22:19:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.483+0000 7f4191ffb700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] conn(0x7f4180065c20 0x7f4180068080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.483+0000 7f4191ffb700 1 -- 172.21.15.33:0/3291507804 --> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4180068700 con 0x7f4180065c20 2023-11-06T22:19:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.483+0000 7f4191ffb700 1 -- 172.21.15.33:0/3291507804 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7f419407f240 con 0x7f419c164310 2023-11-06T22:19:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.484+0000 7f41a1b9b700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f418005f270 0x7f4180061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.484+0000 7f41a139a700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] conn(0x7f4180065c20 0x7f4180068080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.484+0000 7f41a1b9b700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f418005f270 0x7f4180061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f4198009130 tx=0x7f4198017000).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:00.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.484+0000 7f41a139a700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] conn(0x7f4180065c20 0x7f4180068080 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:00.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.485+0000 7f4191ffb700 1 -- 172.21.15.33:0/3291507804 <== osd.7 v2:172.21.15.69:6821/115651 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f4180068700 con 0x7f4180065c20 2023-11-06T22:19:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.499+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 --> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f419c0c9930 con 0x7f4180065c20 2023-11-06T22:19:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.500+0000 7f4191ffb700 1 -- 172.21.15.33:0/3291507804 <== osd.7 v2:172.21.15.69:6821/115651 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f419c0c9930 con 0x7f4180065c20 2023-11-06T22:19:00.503 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-11-06T22:19:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T22:19:00.503 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-11-06T22:19:00.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.500+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] conn(0x7f4180065c20 msgr2=0x7f4180068080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.500+0000 7f41a3dff700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:6821/115651,v1:172.21.15.69:6823/115651] conn(0x7f4180065c20 0x7f4180068080 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.501+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f418005f270 msgr2=0x7f4180061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.501+0000 7f41a3dff700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f418005f270 0x7f4180061730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f4198009130 tx=0x7f4198017000).stop 2023-11-06T22:19:00.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.501+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419c164310 msgr2=0x7f419c164790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.501+0000 7f41a3dff700 1 --2- 172.21.15.33:0/3291507804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419c164310 0x7f419c164790 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 rx=0x7f4194009df0 tx=0x7f419400b040).stop 2023-11-06T22:19:00.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.501+0000 7f41a239c700 1 -- 172.21.15.33:0/3291507804 reap_dead start 2023-11-06T22:19:00.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.501+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 shutdown_connections 2023-11-06T22:19:00.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.501+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 >> 172.21.15.33:0/3291507804 conn(0x7f419c05e1b0 msgr2=0x7f419c0ab070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:00.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.501+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 shutdown_connections 2023-11-06T22:19:00.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.501+0000 7f41a3dff700 1 -- 172.21.15.33:0/3291507804 wait complete. 2023-11-06T22:19:00.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-06T22:19:00.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.714+0000 7f2bfa9ff700 1 Processor -- start 2023-11-06T22:19:00.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.714+0000 7f2bfa9ff700 1 -- start start 2023-11-06T22:19:00.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.715+0000 7f2bfa9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2bf40c6d90 0x7f2bf40c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.715+0000 7f2bfa9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f2bf40bfd60 0x7f2bf40c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.715+0000 7f2bfa9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf40c7700 con 0x7f2bf4154210 2023-11-06T22:19:00.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.715+0000 7f2bfa9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf40c06c0 con 0x7f2bf40c6d90 2023-11-06T22:19:00.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.716+0000 7f2bfa9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf405efc0 con 0x7f2bf40bfd60 2023-11-06T22:19:00.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.716+0000 7f2bf37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2bf40c6d90 0x7f2bf40c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.716+0000 7f2bf37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2bf40c6d90 0x7f2bf40c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52658/0 (socket says 172.21.15.33:52658) 2023-11-06T22:19:00.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.716+0000 7f2bf37fe700 1 -- 172.21.15.33:0/1125973345 learned_addr learned my addr 172.21.15.33:0/1125973345 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.716+0000 7f2bf8f9c700 1 --2- 172.21.15.33:0/1125973345 >> v2:172.21.15.69:3300/0 conn(0x7f2bf40bfd60 0x7f2bf40c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.716+0000 7f2bf37fe700 1 -- 172.21.15.33:0/1125973345 >> v2:172.21.15.69:3300/0 conn(0x7f2bf40bfd60 msgr2=0x7f2bf40c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.716+0000 7f2bf37fe700 1 --2- 172.21.15.33:0/1125973345 >> v2:172.21.15.69:3300/0 conn(0x7f2bf40bfd60 0x7f2bf40c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.716+0000 7f2bf37fe700 1 -- 172.21.15.33:0/1125973345 >> v1:172.21.15.69:6789/0 conn(0x7f2bf4154210 legacy=0x7f2bf40a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.716+0000 7f2bf37fe700 1 -- 172.21.15.33:0/1125973345 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf40c8ea0 con 0x7f2bf40c6d90 2023-11-06T22:19:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.716+0000 7f2bf37fe700 1 --2- 172.21.15.33:0/1125973345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 0x7f2bf40c7210 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 rx=0x7f2bf4083a30 tx=0x7f2be80090f0).ready entity=mon.0 client_cookie=e933a596d6ada258 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bf2ffd700 1 -- 172.21.15.33:0/1125973345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be800b5a0 con 0x7f2bf40c6d90 2023-11-06T22:19:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bf2ffd700 1 -- 172.21.15.33:0/1125973345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be800b700 con 0x7f2bf40c6d90 2023-11-06T22:19:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bf2ffd700 1 -- 172.21.15.33:0/1125973345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be800ba80 con 0x7f2bf40c6d90 2023-11-06T22:19:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/1125973345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 msgr2=0x7f2bf40c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bfa9ff700 1 --2- 172.21.15.33:0/1125973345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 0x7f2bf40c7210 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 rx=0x7f2bf4083a30 tx=0x7f2be80090f0).stop 2023-11-06T22:19:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/1125973345 shutdown_connections 2023-11-06T22:19:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bfa9ff700 1 --2- 172.21.15.33:0/1125973345 >> v2:172.21.15.69:3300/0 conn(0x7f2bf40bfd60 0x7f2bf40c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bfa9ff700 1 --2- 172.21.15.33:0/1125973345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 0x7f2bf40c7210 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/1125973345 >> 172.21.15.33:0/1125973345 conn(0x7f2bf405e990 msgr2=0x7f2bf40cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/1125973345 shutdown_connections 2023-11-06T22:19:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.717+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/1125973345 wait complete. 2023-11-06T22:19:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.719+0000 7f2bfa9ff700 1 Processor -- start 2023-11-06T22:19:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.719+0000 7f2bfa9ff700 1 -- start start 2023-11-06T22:19:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bfa9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bf40bfd60 0x7f2bf414c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bfa9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 0x7f2bf40add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bfa9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bf4154210 0x7f2bf414da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bfa9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf40ab270 con 0x7f2bf40c6d90 2023-11-06T22:19:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bfa9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf40b0900 con 0x7f2bf4154210 2023-11-06T22:19:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bfa9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf40b0be0 con 0x7f2bf40bfd60 2023-11-06T22:19:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bf37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 0x7f2bf40add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bf37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 0x7f2bf40add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52664/0 (socket says 172.21.15.33:52664) 2023-11-06T22:19:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bf37fe700 1 -- 172.21.15.33:0/3860455063 learned_addr learned my addr 172.21.15.33:0/3860455063 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bf8f9c700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bf4154210 0x7f2bf414da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.720+0000 7f2bf3fff700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bf40bfd60 0x7f2bf414c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bf37fe700 1 -- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bf40bfd60 msgr2=0x7f2bf414c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bf37fe700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bf40bfd60 0x7f2bf414c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bf37fe700 1 -- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bf4154210 msgr2=0x7f2bf414da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bf37fe700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bf4154210 0x7f2bf414da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bf37fe700 1 -- 172.21.15.33:0/3860455063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be8009370 con 0x7f2bf40c6d90 2023-11-06T22:19:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bf37fe700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 0x7f2bf40add90 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 rx=0x7f2be8009130 tx=0x7f2be8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bdbfff700 1 -- 172.21.15.33:0/3860455063 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be800bec0 con 0x7f2bf40c6d90 2023-11-06T22:19:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bdbfff700 1 -- 172.21.15.33:0/3860455063 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be800b870 con 0x7f2bf40c6d90 2023-11-06T22:19:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bdbfff700 1 -- 172.21.15.33:0/3860455063 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be801c650 con 0x7f2bf40c6d90 2023-11-06T22:19:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/3860455063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf40aac20 con 0x7f2bf40c6d90 2023-11-06T22:19:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.721+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/3860455063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf414ce80 con 0x7f2bf40c6d90 2023-11-06T22:19:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.722+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/3860455063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf40ab3e0 con 0x7f2bf40c6d90 2023-11-06T22:19:00.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.723+0000 7f2bdbfff700 1 -- 172.21.15.33:0/3860455063 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f2be801ca80 con 0x7f2bf40c6d90 2023-11-06T22:19:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.724+0000 7f2bdbfff700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2be005f270 0x7f2be0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.724+0000 7f2bdbfff700 1 -- 172.21.15.33:0/3860455063 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f2be807be50 con 0x7f2bf40c6d90 2023-11-06T22:19:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.724+0000 7f2bf3fff700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2be005f270 0x7f2be0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.725+0000 7f2bf3fff700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2be005f270 0x7f2be0061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2be400b9b0 tx=0x7f2be4008040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.728+0000 7f2bdbfff700 1 -- 172.21.15.33:0/3860455063 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2be804d000 con 0x7f2bf40c6d90 2023-11-06T22:19:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.905+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/3860455063 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bf40c7700 con 0x7f2be005f270 2023-11-06T22:19:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.906+0000 7f2bdbfff700 1 -- 172.21.15.33:0/3860455063 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2bf40c7700 con 0x7f2be005f270 2023-11-06T22:19:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2be005f270 msgr2=0x7f2be0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2be005f270 0x7f2be0061730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2be400b9b0 tx=0x7f2be4008040).stop 2023-11-06T22:19:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/3860455063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 msgr2=0x7f2bf40add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 0x7f2bf40add90 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 rx=0x7f2be8009130 tx=0x7f2be8015000).stop 2023-11-06T22:19:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/3860455063 shutdown_connections 2023-11-06T22:19:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f2be005f270 0x7f2be0061730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f2bf40bfd60 0x7f2bf414c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bf4154210 0x7f2bf414da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 --2- 172.21.15.33:0/3860455063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf40c6d90 0x7f2bf40add90 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/3860455063 >> 172.21.15.33:0/3860455063 conn(0x7f2bf405e990 msgr2=0x7f2bf4152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/3860455063 shutdown_connections 2023-11-06T22:19:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:00.908+0000 7f2bfa9ff700 1 -- 172.21.15.33:0/3860455063 wait complete. 2023-11-06T22:19:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 7 to deep-scrub 2023-11-06T22:19:00.928 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:19:01.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.121+0000 7fbfb8dff700 1 Processor -- start 2023-11-06T22:19:01.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.122+0000 7fbfb8dff700 1 -- start start 2023-11-06T22:19:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.123+0000 7fbfb8dff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbfb40c5d50 0x7fbfb40c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.123+0000 7fbfb8dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbfb40bfd70 0x7fbfb40c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.123+0000 7fbfb8dff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb41554a0 con 0x7fbfb40c1840 2023-11-06T22:19:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.123+0000 7fbfb8dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb4154960 con 0x7fbfb40bfd70 2023-11-06T22:19:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.123+0000 7fbfb8dff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb40c66c0 con 0x7fbfb40c5d50 2023-11-06T22:19:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.123+0000 7fbfb259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbfb40c1840 0x7fbfb40a9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46660/0 (socket says 172.21.15.33:46660) 2023-11-06T22:19:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.123+0000 7fbfb259c700 1 -- 172.21.15.33:0/3735072736 learned_addr learned my addr 172.21.15.33:0/3735072736 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:19:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.123+0000 7fbfb2d9d700 1 --2- 172.21.15.33:0/3735072736 >> v2:172.21.15.33:3300/0 conn(0x7fbfb40bfd70 0x7fbfb40c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.123+0000 7fbfb1d9b700 1 --2- 172.21.15.33:0/3735072736 >> v2:172.21.15.69:3300/0 conn(0x7fbfb40c5d50 0x7fbfb40c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.124+0000 7fbfb159a700 1 -- 172.21.15.33:0/3735072736 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246136306 0 0) 0x7fbfb41554a0 con 0x7fbfb40c1840 2023-11-06T22:19:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.124+0000 7fbfb159a700 1 -- 172.21.15.33:0/3735072736 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf9c0033f0 con 0x7fbfb40c1840 2023-11-06T22:19:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.124+0000 7fbfb2d9d700 1 -- 172.21.15.33:0/3735072736 >> v2:172.21.15.69:3300/0 conn(0x7fbfb40c5d50 msgr2=0x7fbfb40c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.124+0000 7fbfb2d9d700 1 --2- 172.21.15.33:0/3735072736 >> v2:172.21.15.69:3300/0 conn(0x7fbfb40c5d50 0x7fbfb40c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.124+0000 7fbfb2d9d700 1 -- 172.21.15.33:0/3735072736 >> v1:172.21.15.33:6789/0 conn(0x7fbfb40c1840 legacy=0x7fbfb40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.124+0000 7fbfb2d9d700 1 -- 172.21.15.33:0/3735072736 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4155930 con 0x7fbfb40bfd70 2023-11-06T22:19:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.124+0000 7fbfb2d9d700 1 --2- 172.21.15.33:0/3735072736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40bfd70 0x7fbfb40c0180 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7fbfb40adb40 tx=0x7fbfac009780).ready entity=mon.0 client_cookie=db37f862024b63ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.124+0000 7fbfb159a700 1 -- 172.21.15.33:0/3735072736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00e070 con 0x7fbfb40bfd70 2023-11-06T22:19:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.124+0000 7fbfb159a700 1 -- 172.21.15.33:0/3735072736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac00bab0 con 0x7fbfb40bfd70 2023-11-06T22:19:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.124+0000 7fbfb159a700 1 -- 172.21.15.33:0/3735072736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00be30 con 0x7fbfb40bfd70 2023-11-06T22:19:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.125+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3735072736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40bfd70 msgr2=0x7fbfb40c0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.125+0000 7fbfb8dff700 1 --2- 172.21.15.33:0/3735072736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40bfd70 0x7fbfb40c0180 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7fbfb40adb40 tx=0x7fbfac009780).stop 2023-11-06T22:19:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.125+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3735072736 shutdown_connections 2023-11-06T22:19:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.125+0000 7fbfb8dff700 1 --2- 172.21.15.33:0/3735072736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40bfd70 0x7fbfb40c0180 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.125+0000 7fbfb8dff700 1 --2- 172.21.15.33:0/3735072736 >> v2:172.21.15.69:3300/0 conn(0x7fbfb40c5d50 0x7fbfb40c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.125+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3735072736 >> 172.21.15.33:0/3735072736 conn(0x7fbfb405ec80 msgr2=0x7fbfb40cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.125+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3735072736 shutdown_connections 2023-11-06T22:19:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.125+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3735072736 wait complete. 2023-11-06T22:19:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.127+0000 7fbfb8dff700 1 Processor -- start 2023-11-06T22:19:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.127+0000 7fbfb8dff700 1 -- start start 2023-11-06T22:19:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb8dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb40bfd70 0x7fbfb414b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb8dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40c1840 0x7fbfb4148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb8dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbfb40c5d50 0x7fbfb4159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb8dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb41483e0 con 0x7fbfb40c1840 2023-11-06T22:19:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb8dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb41498c0 con 0x7fbfb40bfd70 2023-11-06T22:19:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb8dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40aeb30 con 0x7fbfb40c5d50 2023-11-06T22:19:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40c1840 0x7fbfb4148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40c1840 0x7fbfb4148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52682/0 (socket says 172.21.15.33:52682) 2023-11-06T22:19:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb1d9b700 1 -- 172.21.15.33:0/3367161271 learned_addr learned my addr 172.21.15.33:0/3367161271 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb259c700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb40bfd70 0x7fbfb414b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:01.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.128+0000 7fbfb2d9d700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbfb40c5d50 0x7fbfb4159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:01.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfb1d9b700 1 -- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbfb40c5d50 msgr2=0x7fbfb4159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:01.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfb1d9b700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbfb40c5d50 0x7fbfb4159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:01.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfb1d9b700 1 -- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb40bfd70 msgr2=0x7fbfb414b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:01.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfb1d9b700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb40bfd70 0x7fbfb414b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:01.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfb1d9b700 1 -- 172.21.15.33:0/3367161271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac009a20 con 0x7fbfb40c1840 2023-11-06T22:19:01.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfb259c700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb40bfd70 0x7fbfb414b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:19:01.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfb1d9b700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40c1840 0x7fbfb4148e80 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7fbfa800e900 tx=0x7fbfa8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:01.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3367161271 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800c640 con 0x7fbfb40c1840 2023-11-06T22:19:01.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3367161271 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa800c7a0 con 0x7fbfb40c1840 2023-11-06T22:19:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3367161271 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800cb20 con 0x7fbfb40c1840 2023-11-06T22:19:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3367161271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb40ae200 con 0x7fbfb40c1840 2023-11-06T22:19:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.129+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3367161271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb40ae7c0 con 0x7fbfb40c1840 2023-11-06T22:19:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.130+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3367161271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb406c6c0 con 0x7fbfb40c1840 2023-11-06T22:19:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.131+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3367161271 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7fbfa801b400 con 0x7fbfb40c1840 2023-11-06T22:19:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.132+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbf9405f270 0x7fbf94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.132+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3367161271 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7fbfa807a720 con 0x7fbfb40c1840 2023-11-06T22:19:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.132+0000 7fbfb259c700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbf9405f270 0x7fbf94061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.133+0000 7fbfb259c700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbf9405f270 0x7fbf94061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fbfb414d780 tx=0x7fbfa4009040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.136+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3367161271 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbfa804b910 con 0x7fbfb40c1840 2023-11-06T22:19:01.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:19:01.245 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:19:01.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.313+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3367161271 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfb414d7b0 con 0x7fbf9405f270 2023-11-06T22:19:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.317+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3367161271 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70492 (secure 0 0 0) 0x7fbfb414d7b0 con 0x7fbf9405f270 2023-11-06T22:19:01.321 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:19:01.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbf9405f270 msgr2=0x7fbf94061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:01.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbf9405f270 0x7fbf94061730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fbfb414d780 tx=0x7fbfa4009040).stop 2023-11-06T22:19:01.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3367161271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40c1840 msgr2=0x7fbfb4148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:01.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40c1840 0x7fbfb4148e80 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7fbfa800e900 tx=0x7fbfa8009060).stop 2023-11-06T22:19:01.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3367161271 shutdown_connections 2023-11-06T22:19:01.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7fbf9405f270 0x7fbf94061730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:01.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7fbfb40c5d50 0x7fbfb4159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:01.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb40bfd70 0x7fbfb414b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 --2- 172.21.15.33:0/3367161271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40c1840 0x7fbfb4148e80 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3367161271 >> 172.21.15.33:0/3367161271 conn(0x7fbfb405ec80 msgr2=0x7fbfb4152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3367161271 shutdown_connections 2023-11-06T22:19:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:01.319+0000 7fbfb8dff700 1 -- 172.21.15.33:0/3367161271 wait complete. 2023-11-06T22:19:01.325 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:19:01.339 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-11-06T22:18:59.552103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2464736,"kb_used_data":22432,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752509984,"statfs":{"total":773094113280,"available":770570223616,"internally_reserved":0,"allocated":22970368,"data_stored":11825256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.009297"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":195,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:49.073878+0000","last_change":"2023-11-06T20:44:00.954878+0000","last_active":"2023-11-06T22:18:49.073878+0000","last_peered":"2023-11-06T22:18:49.073878+0000","last_clean":"2023-11-06T22:18:49.073878+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:18:49.073878+0000","last_undegraded":"2023-11-06T22:18:49.073878+0000","last_fullsized":"2023-11-06T22:18:49.073878+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.567718+0000","last_change":"2023-11-06T20:44:00.955086+0000","last_active":"2023-11-06T22:18:48.567718+0000","last_peered":"2023-11-06T22:18:48.567718+0000","last_clean":"2023-11-06T22:18:48.567718+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:18:48.567718+0000","last_undegraded":"2023-11-06T22:18:48.567718+0000","last_fullsized":"2023-11-06T22:18:48.567718+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":217,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.565920+0000","last_change":"2023-11-06T20:44:00.857036+0000","last_active":"2023-11-06T22:18:48.565920+0000","last_peered":"2023-11-06T22:18:48.565920+0000","last_clean":"2023-11-06T22:18:48.565920+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:18:48.565920+0000","last_undegraded":"2023-11-06T22:18:48.565920+0000","last_fullsized":"2023-11-06T22:18:48.565920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.568883+0000","last_change":"2023-11-06T20:44:01.003297+0000","last_active":"2023-11-06T22:18:48.568883+0000","last_peered":"2023-11-06T22:18:48.568883+0000","last_clean":"2023-11-06T22:18:48.568883+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:18:48.568883+0000","last_undegraded":"2023-11-06T22:18:48.568883+0000","last_fullsized":"2023-11-06T22:18:48.568883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":194,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.565826+0000","last_change":"2023-11-06T20:43:59.859759+0000","last_active":"2023-11-06T22:18:48.565826+0000","last_peered":"2023-11-06T22:18:48.565826+0000","last_clean":"2023-11-06T22:18:48.565826+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:18:48.565826+0000","last_undegraded":"2023-11-06T22:18:48.565826+0000","last_fullsized":"2023-11-06T22:18:48.565826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:58.839022+0000","last_clean_scrub_stamp":"2023-11-06T20:43:58.839022+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.566133+0000","last_change":"2023-11-06T20:44:00.859268+0000","last_active":"2023-11-06T22:18:48.566133+0000","last_peered":"2023-11-06T22:18:48.566133+0000","last_clean":"2023-11-06T22:18:48.566133+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:18:48.566133+0000","last_undegraded":"2023-11-06T22:18:48.566133+0000","last_fullsized":"2023-11-06T22:18:48.566133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.567159+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:18:48.567159+0000","last_peered":"2023-11-06T22:18:48.567159+0000","last_clean":"2023-11-06T22:18:48.567159+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:18:48.567159+0000","last_undegraded":"2023-11-06T22:18:48.567159+0000","last_fullsized":"2023-11-06T22:18:48.567159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":194,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:49.069344+0000","last_change":"2023-11-06T20:44:00.968556+0000","last_active":"2023-11-06T22:18:49.069344+0000","last_peered":"2023-11-06T22:18:49.069344+0000","last_clean":"2023-11-06T22:18:49.069344+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:18:49.069344+0000","last_undegraded":"2023-11-06T22:18:49.069344+0000","last_fullsized":"2023-11-06T22:18:49.069344+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.568057+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:18:48.568057+0000","last_peered":"2023-11-06T22:18:48.568057+0000","last_clean":"2023-11-06T22:18:48.568057+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:18:48.568057+0000","last_undegraded":"2023-11-06T22:18:48.568057+0000","last_fullsized":"2023-11-06T22:18:48.568057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.97199999999999998,"15min":1.0980000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.51200000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.871,"5min":0.98799999999999999,"15min":1.038},"last":0.5},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":0.96599999999999997,"15min":1.0680000000000001},"last":0.64900000000000002}]},{"osd":2,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94799999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88600000000000001,"5min":0.996,"15min":1.0209999999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.499},"max":{"1min":0.88200000000000001,"5min":0.90700000000000003,"15min":0.95799999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.59199999999999997,"5min":0.54200000000000004,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60499999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.44900000000000001},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58899999999999997}]},{"osd":5,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.436,"5min":0.432,"15min":0.40400000000000003},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.432},"max":{"1min":1.0329999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.48399999999999999},"max":{"1min":1.0669999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.52800000000000002,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.069},"last":0.625}]}]},{"osd":6,"up_from":13,"seq":55834575974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68100000000000005,"15min":0.70999999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.48699999999999999}]},{"osd":1,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":1.0469999999999999,"15min":1.0660000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":1.016},"last":0.52200000000000002}]},{"osd":2,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0780000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.046},"last":0.58399999999999996}]},{"osd":3,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.873,"5min":0.94799999999999995,"15min":1.002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.75600000000000001}]},{"osd":4,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.53600000000000003,"5min":0.42299999999999999,"15min":0.41699999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.98299999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76900000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.79300000000000004}]},{"osd":5,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.67100000000000004},"min":{"1min":0.44,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83499999999999996,"5min":0.98499999999999999,"15min":0.98999999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.442},"max":{"1min":0.80000000000000004,"5min":0.996,"15min":1.056},"last":0.77600000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79800000000000004,"5min":1.012,"15min":1.0429999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0569999999999999,"15min":1.1279999999999999},"last":0.88300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.97999999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.33800000000000002},"max":{"1min":0.66700000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.435}]},{"osd":2,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.307,"15min":0.307},"max":{"1min":0.67400000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.95699999999999996,"15min":1.0189999999999999},"last":0.5}]},{"osd":3,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74099999999999999,"5min":0.76200000000000001,"15min":1.0509999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":0.92700000000000005},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.80900000000000005,"15min":0.80800000000000005},"min":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.45200000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84899999999999998,"15min":0.85199999999999998},"min":{"1min":0.68300000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.86299999999999999}]},{"osd":5,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.81599999999999995},"min":{"1min":0.58899999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.052,"5min":1.2210000000000001,"15min":1.2709999999999999},"last":0.998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81499999999999995,"15min":0.80200000000000005},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.55400000000000005},"max":{"1min":1.224,"5min":1.224,"15min":1.3109999999999999},"last":0.91300000000000003}]},{"osd":6,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.3149999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.84399999999999997,"15min":0.84899999999999998},"min":{"1min":0.64900000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.024}]},{"osd":7,"last update":"Mon Nov 6 22:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1459999999999999,"5min":1.21,"15min":1.21},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1970000000000001,"15min":1.2509999999999999},"last":0.875}]}]},{"osd":0,"up_from":13,"seq":55834575974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47499999999999998,"15min":0.46899999999999997},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.499}]},{"osd":2,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.48499999999999999},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.435}]},{"osd":3,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.375},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.85199999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.54800000000000004},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.84199999999999997,"15min":0.85399999999999998},"min":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81699999999999995,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.52600000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83599999999999997,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.71499999999999997}]},{"osd":7,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82999999999999996,"15min":0.85699999999999998},"min":{"1min":0.55200000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.268},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.85499999999999998,"15min":0.876},"min":{"1min":0.58299999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.67600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70799999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":0.83799999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55900000000000005,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83999999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.74299999999999999,"5min":0.83699999999999997,"15min":0.89900000000000002},"last":0.64800000000000002}]},{"osd":3,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68899999999999995,"5min":0.74399999999999999,"15min":0.78300000000000003},"last":0.497},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.39100000000000001},"max":{"1min":0.753,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59199999999999997}]},{"osd":4,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.81200000000000006,"15min":0.82199999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.53900000000000003},"max":{"1min":1.0569999999999999,"5min":1.246,"15min":1.349},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.80600000000000005,"15min":0.81699999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0700000000000001,"5min":1.2669999999999999,"15min":1.339},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.48999999999999999},"max":{"1min":1.0860000000000001,"5min":1.276,"15min":1.329},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.82299999999999995,"15min":0.84999999999999998},"min":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1419999999999999,"5min":1.212,"15min":1.468},"last":0.63900000000000001}]},{"osd":6,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.81999999999999995,"15min":0.84199999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.123,"5min":1.155,"15min":1.387},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.82199999999999995,"15min":0.84499999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97299999999999998,"5min":1.175,"15min":1.4299999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84499999999999997,"15min":0.86899999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98799999999999999,"5min":1.2150000000000001,"15min":1.496},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.64200000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.042,"5min":1.2470000000000001,"15min":1.508},"last":0.66600000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575975,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45700000000000002,"15min":0.47499999999999998},"min":{"1min":0.28399999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.31},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68700000000000006,"5min":1.05,"15min":1.05},"last":0.436},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.57199999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70499999999999996,"5min":0.873,"15min":0.873},"last":0.76400000000000001}]},{"osd":2,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54400000000000004,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.87},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.81499999999999995}]},{"osd":4,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0669999999999999,"5min":1.161,"15min":1.2370000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.82899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1200000000000001,"5min":1.147,"15min":1.226},"last":0.80200000000000005}]},{"osd":5,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77700000000000002,"15min":0.80700000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0449999999999999,"5min":1.071,"15min":1.2270000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.3140000000000001},"last":0.877}]},{"osd":6,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.82399999999999995},"min":{"1min":0.52700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0429999999999999,"5min":1.149,"15min":1.2290000000000001},"last":0.75},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81200000000000006,"15min":0.85199999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.099,"5min":1.099,"15min":1.1879999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.196,"5min":1.196,"15min":1.2829999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.81799999999999995,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.113,"5min":1.169,"15min":1.248},"last":0.84699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70499999999999996,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98999999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68100000000000005,"15min":0.71099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82699999999999996,"5min":0.95799999999999996,"15min":1.006},"last":0.54200000000000004}]},{"osd":1,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70299999999999996,"15min":0.72699999999999998},"min":{"1min":0.53700000000000003,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92500000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.52200000000000002}]},{"osd":2,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.98599999999999999,"15min":1.087},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.48299999999999998},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.071},"last":0.76100000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":1.0329999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90400000000000003,"5min":1.236,"15min":1.236},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.48399999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88700000000000001,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.64600000000000002}]},{"osd":6,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90600000000000003,"5min":1.018,"15min":1.018},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.505,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0369999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.79600000000000004}]},{"osd":7,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.53200000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.54400000000000004,"5min":0.51500000000000001,"15min":0.50800000000000001},"max":{"1min":1.0469999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.309},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1639999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.64700000000000002}]},{"osd":1,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77800000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47999999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.61699999999999999}]},{"osd":2,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.748},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78700000000000003,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.48299999999999998},"max":{"1min":0.96499999999999997,"5min":4.9779999999999998,"15min":4.9779999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.36499999999999999},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.752,"15min":0.747},"min":{"1min":0.499,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.2390000000000001,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":0.71499999999999997}]},{"osd":4,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.38800000000000001},"max":{"1min":1.4019999999999999,"5min":1.585,"15min":1.585},"last":0.72299999999999998}]},{"osd":6,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":1.4510000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77400000000000002,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.441,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":0.86799999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.41399999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78200000000000003,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":1.4810000000000001,"5min":6.891,"15min":6.891},"last":0.77600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:19:01.344 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-06T20:43:59.844722+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:01.344 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-06T20:43:59.844722+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:01.345 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-06T20:43:59.844722+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:01.345 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-06T20:43:59.844722+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:01.345 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-06T20:43:58.839022+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=58, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:01.345 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-06T20:43:59.844722+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:01.346 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-06T20:43:59.844722+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:01.346 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-06T20:43:59.844722+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:01.346 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-06T20:43:59.844722+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:01.347 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-06T22:19:21.348 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:19:21.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.555+0000 7f81bf9ff700 1 Processor -- start 2023-11-06T22:19:21.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.555+0000 7f81bf9ff700 1 -- start start 2023-11-06T22:19:21.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bf9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81b8063140 0x7f81b8063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bf9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81b80c3680 0x7f81b8154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bf9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b80c31f0 con 0x7f81b805e1b0 2023-11-06T22:19:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bf9ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b806a0f0 con 0x7f81b80c3680 2023-11-06T22:19:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bf9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80c2920 con 0x7f81b8063140 2023-11-06T22:19:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bdf9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81b80c3680 0x7f81b8154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bd79b700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81b8063140 0x7f81b8063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bdf9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81b80c3680 0x7f81b8154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34152/0 (socket says 172.21.15.33:34152) 2023-11-06T22:19:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bdf9c700 1 -- 172.21.15.33:0/3958580361 learned_addr learned my addr 172.21.15.33:0/3958580361 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bdf9c700 1 -- 172.21.15.33:0/3958580361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81b8063140 msgr2=0x7f81b8063520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:21.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bdf9c700 1 --2- 172.21.15.33:0/3958580361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81b8063140 0x7f81b8063520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:21.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bdf9c700 1 -- 172.21.15.33:0/3958580361 >> v1:172.21.15.69:6789/0 conn(0x7f81b805e1b0 legacy=0x7f81b80c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:21.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81bdf9c700 1 -- 172.21.15.33:0/3958580361 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b805e640 con 0x7f81b80c3680 2023-11-06T22:19:21.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.557+0000 7f81affff700 1 -- 172.21.15.33:0/3958580361 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505068926 0 0) 0x7f81b80c31f0 con 0x7f81b805e1b0 2023-11-06T22:19:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.558+0000 7f81bdf9c700 1 --2- 172.21.15.33:0/3958580361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 0x7f81b8154050 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 rx=0x7f81b0009f20 tx=0x7f81b0009840).ready entity=mon.0 client_cookie=976691ef40562493 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.558+0000 7f81affff700 1 -- 172.21.15.33:0/3958580361 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b000e070 con 0x7f81b80c3680 2023-11-06T22:19:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.558+0000 7f81affff700 1 -- 172.21.15.33:0/3958580361 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b000b9d0 con 0x7f81b80c3680 2023-11-06T22:19:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.558+0000 7f81affff700 1 -- 172.21.15.33:0/3958580361 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b000bd50 con 0x7f81b80c3680 2023-11-06T22:19:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.558+0000 7f81bf9ff700 1 -- 172.21.15.33:0/3958580361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 msgr2=0x7f81b8154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.558+0000 7f81bf9ff700 1 --2- 172.21.15.33:0/3958580361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 0x7f81b8154050 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 rx=0x7f81b0009f20 tx=0x7f81b0009840).stop 2023-11-06T22:19:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.559+0000 7f81bf9ff700 1 -- 172.21.15.33:0/3958580361 shutdown_connections 2023-11-06T22:19:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.559+0000 7f81bf9ff700 1 --2- 172.21.15.33:0/3958580361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 0x7f81b8154050 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.559+0000 7f81bf9ff700 1 --2- 172.21.15.33:0/3958580361 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81b8063140 0x7f81b8063520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.559+0000 7f81bf9ff700 1 -- 172.21.15.33:0/3958580361 >> 172.21.15.33:0/3958580361 conn(0x7f81b80c7d60 msgr2=0x7f81b80c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.559+0000 7f81bf9ff700 1 -- 172.21.15.33:0/3958580361 shutdown_connections 2023-11-06T22:19:21.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.559+0000 7f81bf9ff700 1 -- 172.21.15.33:0/3958580361 wait complete. 2023-11-06T22:19:21.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.560+0000 7f81bf9ff700 1 Processor -- start 2023-11-06T22:19:21.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.560+0000 7f81bf9ff700 1 -- start start 2023-11-06T22:19:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.561+0000 7f81bf9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81b8063140 0x7f81b8158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.561+0000 7f81bf9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 0x7f81b814c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.561+0000 7f81bf9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81b80abe40 0x7f81b80ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.561+0000 7f81bf9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b8159a80 con 0x7f81b80c3680 2023-11-06T22:19:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.561+0000 7f81bf9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b81585d0 con 0x7f81b80abe40 2023-11-06T22:19:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.561+0000 7f81bf9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80ac790 con 0x7f81b8063140 2023-11-06T22:19:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.561+0000 7f81bcf9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 0x7f81b814c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bcf9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 0x7f81b814c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34154/0 (socket says 172.21.15.33:34154) 2023-11-06T22:19:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bcf9a700 1 -- 172.21.15.33:0/184500333 learned_addr learned my addr 172.21.15.33:0/184500333 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bdf9c700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81b80abe40 0x7f81b80ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bd79b700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81b8063140 0x7f81b8158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bcf9a700 1 -- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81b8063140 msgr2=0x7f81b8158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bcf9a700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81b8063140 0x7f81b8158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bcf9a700 1 -- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81b80abe40 msgr2=0x7f81b80ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bcf9a700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81b80abe40 0x7f81b80ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bcf9a700 1 -- 172.21.15.33:0/184500333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0009ae0 con 0x7f81b80c3680 2023-11-06T22:19:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bdf9c700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81b80abe40 0x7f81b80ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T22:19:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bcf9a700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 0x7f81b814c120 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 rx=0x7f81a800c450 tx=0x7f81a8007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81ad7fa700 1 -- 172.21.15.33:0/184500333 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800f470 con 0x7f81b80c3680 2023-11-06T22:19:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81ad7fa700 1 -- 172.21.15.33:0/184500333 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a800f5d0 con 0x7f81b80c3680 2023-11-06T22:19:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81bf9ff700 1 -- 172.21.15.33:0/184500333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b8157a20 con 0x7f81b80c3680 2023-11-06T22:19:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.562+0000 7f81ad7fa700 1 -- 172.21.15.33:0/184500333 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800f950 con 0x7f81b80c3680 2023-11-06T22:19:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.563+0000 7f81bf9ff700 1 -- 172.21.15.33:0/184500333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b8157f80 con 0x7f81b80c3680 2023-11-06T22:19:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.564+0000 7f81bf9ff700 1 -- 172.21.15.33:0/184500333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b8158780 con 0x7f81b80c3680 2023-11-06T22:19:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.564+0000 7f81ad7fa700 1 -- 172.21.15.33:0/184500333 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f81a800fd40 con 0x7f81b80c3680 2023-11-06T22:19:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.565+0000 7f81ad7fa700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f819c05f270 0x7f819c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.566+0000 7f81ad7fa700 1 -- 172.21.15.33:0/184500333 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f81a8079700 con 0x7f81b80c3680 2023-11-06T22:19:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.566+0000 7f81bd79b700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f819c05f270 0x7f819c061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.566+0000 7f81bd79b700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f819c05f270 0x7f819c061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f81b4009bc0 tx=0x7f81b4008040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.570+0000 7f81ad7fa700 1 -- 172.21.15.33:0/184500333 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f81a8048af0 con 0x7f81b80c3680 2023-11-06T22:19:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.748+0000 7f81bf9ff700 1 -- 172.21.15.33:0/184500333 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81b806bfd0 con 0x7f819c05f270 2023-11-06T22:19:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.751+0000 7f81ad7fa700 1 -- 172.21.15.33:0/184500333 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70519 (secure 0 0 0) 0x7f81b806bfd0 con 0x7f819c05f270 2023-11-06T22:19:21.754 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:19:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 -- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f819c05f270 msgr2=0x7f819c061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f819c05f270 0x7f819c061730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f81b4009bc0 tx=0x7f81b4008040).stop 2023-11-06T22:19:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 -- 172.21.15.33:0/184500333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 msgr2=0x7f81b814c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 0x7f81b814c120 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 rx=0x7f81a800c450 tx=0x7f81a8007bd0).stop 2023-11-06T22:19:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 -- 172.21.15.33:0/184500333 shutdown_connections 2023-11-06T22:19:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f819c05f270 0x7f819c061730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f81b8063140 0x7f81b8158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81b80abe40 0x7f81b80ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 --2- 172.21.15.33:0/184500333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81b80c3680 0x7f81b814c120 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 -- 172.21.15.33:0/184500333 >> 172.21.15.33:0/184500333 conn(0x7f81b80c7d60 msgr2=0x7f81b80c61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 -- 172.21.15.33:0/184500333 shutdown_connections 2023-11-06T22:19:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:21.754+0000 7f81bf9ff700 1 -- 172.21.15.33:0/184500333 wait complete. 2023-11-06T22:19:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:19:21.773 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T22:19:21.557106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2464736,"kb_used_data":22432,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752509984,"statfs":{"total":773094113280,"available":770570223616,"internally_reserved":0,"allocated":22970368,"data_stored":11825256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.009297"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":200,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:00.452024+0000","last_change":"2023-11-06T22:19:00.452024+0000","last_active":"2023-11-06T22:19:00.452024+0000","last_peered":"2023-11-06T22:19:00.452024+0000","last_clean":"2023-11-06T22:19:00.452024+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:19:00.452024+0000","last_undegraded":"2023-11-06T22:19:00.452024+0000","last_fullsized":"2023-11-06T22:19:00.452024+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:00.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:00.451887+0000","last_clean_scrub_stamp":"2023-11-06T22:19:00.451887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":213,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:16.835278+0000","last_change":"2023-11-06T22:19:16.835278+0000","last_active":"2023-11-06T22:19:16.835278+0000","last_peered":"2023-11-06T22:19:16.835278+0000","last_clean":"2023-11-06T22:19:16.835278+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:19:16.835278+0000","last_undegraded":"2023-11-06T22:19:16.835278+0000","last_fullsized":"2023-11-06T22:19:16.835278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:16.835174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:16.835174+0000","last_clean_scrub_stamp":"2023-11-06T22:19:16.835174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":222,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:02.758980+0000","last_change":"2023-11-06T22:19:02.758980+0000","last_active":"2023-11-06T22:19:02.758980+0000","last_peered":"2023-11-06T22:19:02.758980+0000","last_clean":"2023-11-06T22:19:02.758980+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:19:02.758980+0000","last_undegraded":"2023-11-06T22:19:02.758980+0000","last_fullsized":"2023-11-06T22:19:02.758980+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:02.758843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:02.758843+0000","last_clean_scrub_stamp":"2023-11-06T22:19:02.758843+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":213,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:20.823011+0000","last_change":"2023-11-06T22:19:20.823011+0000","last_active":"2023-11-06T22:19:20.823011+0000","last_peered":"2023-11-06T22:19:20.823011+0000","last_clean":"2023-11-06T22:19:20.823011+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:19:20.823011+0000","last_undegraded":"2023-11-06T22:19:20.823011+0000","last_fullsized":"2023-11-06T22:19:20.823011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:20.822901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:20.822901+0000","last_clean_scrub_stamp":"2023-11-06T22:19:20.822901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":199,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:18:58.186050+0000","last_change":"2023-11-06T22:18:58.186050+0000","last_active":"2023-11-06T22:18:58.186050+0000","last_peered":"2023-11-06T22:18:58.186050+0000","last_clean":"2023-11-06T22:18:58.186050+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:18:58.186050+0000","last_undegraded":"2023-11-06T22:18:58.186050+0000","last_fullsized":"2023-11-06T22:18:58.186050+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:58.185959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:58.185959+0000","last_clean_scrub_stamp":"2023-11-06T22:18:58.185959+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":213,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:09.962944+0000","last_change":"2023-11-06T22:19:09.962944+0000","last_active":"2023-11-06T22:19:09.962944+0000","last_peered":"2023-11-06T22:19:09.962944+0000","last_clean":"2023-11-06T22:19:09.962944+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:19:09.962944+0000","last_undegraded":"2023-11-06T22:19:09.962944+0000","last_fullsized":"2023-11-06T22:19:09.962944+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:09.962824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:09.962824+0000","last_clean_scrub_stamp":"2023-11-06T22:19:09.962824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.567159+0000","last_change":"2023-11-06T20:44:00.862233+0000","last_active":"2023-11-06T22:18:48.567159+0000","last_peered":"2023-11-06T22:18:48.567159+0000","last_clean":"2023-11-06T22:18:48.567159+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:18:48.567159+0000","last_undegraded":"2023-11-06T22:18:48.567159+0000","last_fullsized":"2023-11-06T22:18:48.567159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":199,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:18:59.722034+0000","last_change":"2023-11-06T22:18:59.722034+0000","last_active":"2023-11-06T22:18:59.722034+0000","last_peered":"2023-11-06T22:18:59.722034+0000","last_clean":"2023-11-06T22:18:59.722034+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:18:59.722034+0000","last_undegraded":"2023-11-06T22:18:59.722034+0000","last_fullsized":"2023-11-06T22:18:59.722034+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:59.721903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:59.721903+0000","last_clean_scrub_stamp":"2023-11-06T22:18:59.721903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":208,"reported_epoch":228,"state":"active+clean","last_fresh":"2023-11-06T22:18:48.568057+0000","last_change":"2023-11-06T20:44:00.968432+0000","last_active":"2023-11-06T22:18:48.568057+0000","last_peered":"2023-11-06T22:18:48.568057+0000","last_clean":"2023-11-06T22:18:48.568057+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:18:48.568057+0000","last_undegraded":"2023-11-06T22:18:48.568057+0000","last_fullsized":"2023-11-06T22:18:48.568057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T20:43:59.844722+0000","last_clean_scrub_stamp":"2023-11-06T20:43:59.844722+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575979,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.97199999999999998,"15min":1.0980000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.51200000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61299999999999999}]},{"osd":1,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.871,"5min":0.98799999999999999,"15min":1.038},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":0.96599999999999997,"15min":1.0680000000000001},"last":0.78700000000000003}]},{"osd":2,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94799999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88600000000000001,"5min":0.996,"15min":1.0209999999999999},"last":0.72399999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.499},"max":{"1min":0.88200000000000001,"5min":0.90700000000000003,"15min":0.95799999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.59199999999999997,"5min":0.54200000000000004,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75}]},{"osd":4,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.44900000000000001},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76900000000000002}]},{"osd":5,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.436,"5min":0.432,"15min":0.40400000000000003},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.432},"max":{"1min":1.0329999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Mon Nov 6 22:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.48399999999999999},"max":{"1min":1.0669999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.52800000000000002,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.069},"last":0.73099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575979,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68100000000000005,"15min":0.70999999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.72799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":1.0469999999999999,"15min":1.0660000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":1.016},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0780000000000001},"last":0.621},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.046},"last":0.73999999999999999}]},{"osd":3,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.873,"5min":0.94799999999999995,"15min":1.002},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.80400000000000005}]},{"osd":4,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.53600000000000003,"5min":0.42299999999999999,"15min":0.41699999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.98299999999999998},"last":0.88},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76900000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.91900000000000004}]},{"osd":5,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.67100000000000004},"min":{"1min":0.44,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83499999999999996,"5min":0.98499999999999999,"15min":0.98999999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.442},"max":{"1min":0.80000000000000004,"5min":0.996,"15min":1.056},"last":0.98599999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79800000000000004,"5min":1.012,"15min":1.0429999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85299999999999998,"5min":1.0569999999999999,"15min":1.1279999999999999},"last":0.95999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575979,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46000000000000002,"15min":0.46500000000000002},"min":{"1min":0.311,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66700000000000004,"5min":0.69899999999999995,"15min":0.97999999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.37,"5min":0.34599999999999997,"15min":0.33800000000000002},"max":{"1min":0.95999999999999996,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53300000000000003}]},{"osd":2,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.5,"15min":0.50600000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72399999999999998,"5min":0.77100000000000002,"15min":0.94199999999999995},"last":0.504},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69899999999999995,"5min":0.80700000000000005,"15min":1.0189999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.442,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.0509999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79800000000000004,"5min":0.79900000000000004,"15min":0.87},"last":0.45800000000000002}]},{"osd":4,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79600000000000004,"15min":0.80500000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.51200000000000001},"max":{"1min":0.91900000000000004,"5min":1.329,"15min":1.329},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1599999999999999,"5min":1.337,"15min":1.337},"last":0.75600000000000001}]},{"osd":5,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79000000000000004,"15min":0.81100000000000005},"min":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0329999999999999,"5min":1.2210000000000001,"15min":1.2709999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.80300000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.55400000000000005},"max":{"1min":1.149,"5min":1.224,"15min":1.3109999999999999},"last":0.84599999999999997}]},{"osd":6,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81100000000000005,"15min":0.82699999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.119,"5min":1.2709999999999999,"15min":1.3149999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.83699999999999997,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.81399999999999995}]},{"osd":7,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.86099999999999999,"15min":0.86599999999999999},"min":{"1min":0.65900000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.014,"5min":1.1970000000000001,"15min":1.2509999999999999},"last":0.65200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575979,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47499999999999998,"15min":0.46899999999999997},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.48499999999999999},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.504}]},{"osd":3,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.375},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.55300000000000005}]},{"osd":4,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.85199999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.54800000000000004},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.89000000000000001}]},{"osd":5,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.84199999999999997,"15min":0.85399999999999998},"min":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81699999999999995,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.52600000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83599999999999997,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.73699999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82999999999999996,"15min":0.85699999999999998},"min":{"1min":0.55200000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.268},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.85499999999999998,"15min":0.876},"min":{"1min":0.58299999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.874}]}]},{"osd":2,"up_from":13,"seq":55834575979,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70799999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":0.83799999999999997},"last":0.47899999999999998}]},{"osd":1,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55900000000000005,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83999999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.74299999999999999,"5min":0.83699999999999997,"15min":0.89900000000000002},"last":0.54200000000000004}]},{"osd":3,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68899999999999995,"5min":0.74399999999999999,"15min":0.78300000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.39100000000000001},"max":{"1min":0.753,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59699999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.81200000000000006,"15min":0.82199999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.53900000000000003},"max":{"1min":1.0569999999999999,"5min":1.246,"15min":1.349},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.80600000000000005,"15min":0.81699999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0700000000000001,"5min":1.2669999999999999,"15min":1.339},"last":0.77100000000000002}]},{"osd":5,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.48999999999999999},"max":{"1min":1.0860000000000001,"5min":1.276,"15min":1.329},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.82299999999999995,"15min":0.84999999999999998},"min":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1419999999999999,"5min":1.212,"15min":1.468},"last":0.751}]},{"osd":6,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.81999999999999995,"15min":0.84199999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.123,"5min":1.155,"15min":1.387},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.82199999999999995,"15min":0.84499999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97299999999999998,"5min":1.175,"15min":1.4299999999999999},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84499999999999997,"15min":0.86899999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98799999999999999,"5min":1.2150000000000001,"15min":1.496},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.64200000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.042,"5min":1.2470000000000001,"15min":1.508},"last":0.78800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575980,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45700000000000002,"15min":0.47499999999999998},"min":{"1min":0.28399999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.31},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68700000000000006,"5min":1.05,"15min":1.05},"last":0.439},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.57199999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70499999999999996,"5min":0.873,"15min":0.873},"last":0.54800000000000004}]},{"osd":2,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54400000000000004,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.87},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.505}]},{"osd":4,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0669999999999999,"5min":1.161,"15min":1.2370000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.82899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1200000000000001,"5min":1.147,"15min":1.226},"last":0.82199999999999995}]},{"osd":5,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77700000000000002,"15min":0.80700000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0449999999999999,"5min":1.071,"15min":1.2270000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.3140000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.82399999999999995},"min":{"1min":0.52700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0429999999999999,"5min":1.149,"15min":1.2290000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81200000000000006,"15min":0.85199999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.099,"5min":1.099,"15min":1.1879999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.196,"5min":1.196,"15min":1.2829999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.81799999999999995,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.113,"5min":1.169,"15min":1.248},"last":0.86399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575979,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70499999999999996,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98999999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68100000000000005,"15min":0.71099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82699999999999996,"5min":0.95799999999999996,"15min":1.006},"last":0.69199999999999995}]},{"osd":1,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70299999999999996,"15min":0.72699999999999998},"min":{"1min":0.53700000000000003,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92500000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.80100000000000005}]},{"osd":2,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.98599999999999999,"15min":1.087},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.48299999999999998},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.071},"last":0.86099999999999999}]},{"osd":3,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":1.0329999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90400000000000003,"5min":1.236,"15min":1.236},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.48399999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88700000000000001,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.53300000000000003}]},{"osd":6,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90600000000000003,"5min":1.018,"15min":1.018},"last":0.498},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.505,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0369999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.53200000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.54400000000000004,"5min":0.51500000000000001,"15min":0.50800000000000001},"max":{"1min":1.0469999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.59099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575980,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.309},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1639999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77800000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47999999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78700000000000003,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.48299999999999998},"max":{"1min":0.96499999999999997,"5min":4.9779999999999998,"15min":4.9779999999999998},"last":0.85399999999999998}]},{"osd":3,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.36499999999999999},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.752,"15min":0.747},"min":{"1min":0.499,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.2390000000000001,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":0.83699999999999997}]},{"osd":4,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.38800000000000001},"max":{"1min":1.4019999999999999,"5min":1.585,"15min":1.585},"last":0.753}]},{"osd":6,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":1.4510000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77400000000000002,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.441,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":0.82399999999999995}]},{"osd":7,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.41399999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.62},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78200000000000003,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":1.4810000000000001,"5min":6.891,"15min":6.891},"last":0.73199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:19:21.776 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-06T20:43:59.844722+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:21.776 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-06T20:43:59.844722+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=20, tm_min=43, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=22, tm_min=18, tm_sec=54, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T22:19:21.776 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-06T22:19:31.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:19:31.393 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:19:41.777 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T22:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.982+0000 7f435c9ff700 1 Processor -- start 2023-11-06T22:19:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.982+0000 7f435c9ff700 1 -- start start 2023-11-06T22:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.984+0000 7f435c9ff700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f43580bdbe0 0x7f43581584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.984+0000 7f435c9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580a7910 0x7f43580a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.984+0000 7f435c9ff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4358064010 con 0x7f43581580e0 2023-11-06T22:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.984+0000 7f435c9ff700 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43580be430 con 0x7f43580bdbe0 2023-11-06T22:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.984+0000 7f435c9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43580a8290 con 0x7f43580a7910 2023-11-06T22:19:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.985+0000 7f4356d9d700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580a7910 0x7f43580a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.985+0000 7f4355d9b700 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f43580bdbe0 0x7f43581584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.985+0000 7f435659c700 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f43581580e0 0x7f43580684c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.33:48500/0 (socket says 172.21.15.33:48500) 2023-11-06T22:19:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.985+0000 7f435659c700 1 -- 172.21.15.33:0/2414373879 learned_addr learned my addr 172.21.15.33:0/2414373879 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-11-06T22:19:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.985+0000 7f4356d9d700 1 -- 172.21.15.33:0/2414373879 >> v2:172.21.15.69:3300/0 conn(0x7f43580bdbe0 msgr2=0x7f43581584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.986+0000 7f4356d9d700 1 --2- 172.21.15.33:0/2414373879 >> v2:172.21.15.69:3300/0 conn(0x7f43580bdbe0 0x7f43581584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.986+0000 7f4356d9d700 1 -- 172.21.15.33:0/2414373879 >> v1:172.21.15.69:6789/0 conn(0x7f43581580e0 legacy=0x7f43580684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.986+0000 7f4356d9d700 1 -- 172.21.15.33:0/2414373879 --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43580be150 con 0x7f43580a7910 2023-11-06T22:19:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.986+0000 7f4355d9b700 1 --2- 172.21.15.33:0/2414373879 >> v2:172.21.15.69:3300/0 conn(0x7f43580bdbe0 0x7f43581584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-06T22:19:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.986+0000 7f435559a700 1 -- 172.21.15.33:0/2414373879 <== mon.1 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631857629 0 0) 0x7f4358064010 con 0x7f43581580e0 2023-11-06T22:19:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.986+0000 7f4356d9d700 1 --2- 172.21.15.33:0/2414373879 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580a7910 0x7f43580a7da0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f4350009e60 tx=0x7f4350009780).ready entity=mon.2 client_cookie=7f69299da4b3ac9c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.986+0000 7f435559a700 1 -- 172.21.15.33:0/2414373879 <== mon.2 v2:172.21.15.69:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435000e070 con 0x7f43580a7910 2023-11-06T22:19:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.986+0000 7f435559a700 1 -- 172.21.15.33:0/2414373879 <== mon.2 v2:172.21.15.69:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435000bb60 con 0x7f43580a7910 2023-11-06T22:19:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.986+0000 7f435559a700 1 -- 172.21.15.33:0/2414373879 <== mon.2 v2:172.21.15.69:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350012410 con 0x7f43580a7910 2023-11-06T22:19:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.987+0000 7f435c9ff700 1 -- 172.21.15.33:0/2414373879 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580a7910 msgr2=0x7f43580a7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.987+0000 7f435c9ff700 1 --2- 172.21.15.33:0/2414373879 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580a7910 0x7f43580a7da0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f4350009e60 tx=0x7f4350009780).stop 2023-11-06T22:19:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.987+0000 7f435c9ff700 1 -- 172.21.15.33:0/2414373879 shutdown_connections 2023-11-06T22:19:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.987+0000 7f435c9ff700 1 --2- 172.21.15.33:0/2414373879 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580a7910 0x7f43580a7da0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:41.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.987+0000 7f435c9ff700 1 --2- 172.21.15.33:0/2414373879 >> v2:172.21.15.69:3300/0 conn(0x7f43580bdbe0 0x7f43581584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:41.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.987+0000 7f435c9ff700 1 -- 172.21.15.33:0/2414373879 >> 172.21.15.33:0/2414373879 conn(0x7f435805ec80 msgr2=0x7f43580664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:41.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.987+0000 7f435c9ff700 1 -- 172.21.15.33:0/2414373879 shutdown_connections 2023-11-06T22:19:41.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.987+0000 7f435c9ff700 1 -- 172.21.15.33:0/2414373879 wait complete. 2023-11-06T22:19:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.989+0000 7f435c9ff700 1 Processor -- start 2023-11-06T22:19:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.989+0000 7f435c9ff700 1 -- start start 2023-11-06T22:19:42.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.990+0000 7f435c9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43580a7910 0x7f43580c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.990+0000 7f435c9ff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580bdbe0 0x7f4358139970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.990+0000 7f435c9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43581580e0 0x7f43580a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.990+0000 7f435c9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43580ae920 con 0x7f43580a7910 2023-11-06T22:19:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.990+0000 7f435c9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43580c4160 con 0x7f43581580e0 2023-11-06T22:19:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.990+0000 7f435c9ff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43580c4440 con 0x7f43580bdbe0 2023-11-06T22:19:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f435659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43580a7910 0x7f43580c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f435659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43580a7910 0x7f43580c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48652/0 (socket says 172.21.15.33:48652) 2023-11-06T22:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f435659c700 1 -- 172.21.15.33:0/3069631654 learned_addr learned my addr 172.21.15.33:0/3069631654 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f4356d9d700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43581580e0 0x7f43580a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f4355d9b700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580bdbe0 0x7f4358139970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f435659c700 1 -- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580bdbe0 msgr2=0x7f4358139970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f435659c700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580bdbe0 0x7f4358139970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f435659c700 1 -- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43581580e0 msgr2=0x7f43580a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f435659c700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43581580e0 0x7f43580a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f435659c700 1 -- 172.21.15.33:0/3069631654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c0092a0 con 0x7f43580a7910 2023-11-06T22:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.991+0000 7f435659c700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43580a7910 0x7f43580c4c80 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7f4348001eb0 tx=0x7f4348007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.992+0000 7f43467fc700 1 -- 172.21.15.33:0/3069631654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480072e0 con 0x7f43580a7910 2023-11-06T22:19:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.992+0000 7f43467fc700 1 -- 172.21.15.33:0/3069631654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348007440 con 0x7f43580a7910 2023-11-06T22:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.992+0000 7f43467fc700 1 -- 172.21.15.33:0/3069631654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480057a0 con 0x7f43580a7910 2023-11-06T22:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.992+0000 7f435c9ff700 1 -- 172.21.15.33:0/3069631654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4350009a20 con 0x7f43580a7910 2023-11-06T22:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.992+0000 7f435c9ff700 1 -- 172.21.15.33:0/3069631654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43580a9dd0 con 0x7f43580a7910 2023-11-06T22:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.993+0000 7f435c9ff700 1 -- 172.21.15.33:0/3069631654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4358067fa0 con 0x7f43580a7910 2023-11-06T22:19:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.994+0000 7f43467fc700 1 -- 172.21.15.33:0/3069631654 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f4348005b90 con 0x7f43580a7910 2023-11-06T22:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.995+0000 7f43467fc700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f433805f270 0x7f4338061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.995+0000 7f43467fc700 1 -- 172.21.15.33:0/3069631654 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f4348079a40 con 0x7f43580a7910 2023-11-06T22:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.995+0000 7f4355d9b700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f433805f270 0x7f4338061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.996+0000 7f4355d9b700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f433805f270 0x7f4338061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f434c009270 tx=0x7f434c008040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:41.999+0000 7f43467fc700 1 -- 172.21.15.33:0/3069631654 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f434807be60 con 0x7f43580a7910 2023-11-06T22:19:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.179+0000 7f435c9ff700 1 -- 172.21.15.33:0/3069631654 --> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4358066bf0 con 0x7f433805f270 2023-11-06T22:19:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.183+0000 7f43467fc700 1 -- 172.21.15.33:0/3069631654 <== mgr.9718 v2:172.21.15.69:6832/129946 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70427 (secure 0 0 0) 0x7f4358066bf0 con 0x7f433805f270 2023-11-06T22:19:42.188 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:19:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 -- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f433805f270 msgr2=0x7f4338061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f433805f270 0x7f4338061730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f434c009270 tx=0x7f434c008040).stop 2023-11-06T22:19:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 -- 172.21.15.33:0/3069631654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43580a7910 msgr2=0x7f43580c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43580a7910 0x7f43580c4c80 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7f4348001eb0 tx=0x7f4348007e90).stop 2023-11-06T22:19:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 -- 172.21.15.33:0/3069631654 shutdown_connections 2023-11-06T22:19:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f433805f270 0x7f4338061730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43581580e0 0x7f43580a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43580a7910 0x7f43580c4c80 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 --2- 172.21.15.33:0/3069631654 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f43580bdbe0 0x7f4358139970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 -- 172.21.15.33:0/3069631654 >> 172.21.15.33:0/3069631654 conn(0x7f435805ec80 msgr2=0x7f4358155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 -- 172.21.15.33:0/3069631654 shutdown_connections 2023-11-06T22:19:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.185+0000 7f435c9ff700 1 -- 172.21.15.33:0/3069631654 wait complete. 2023-11-06T22:19:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-11-06T22:19:42.206 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-11-06T22:19:41.561639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2464736,"kb_used_data":22432,"kb_used_omap":0,"kb_used_meta":2442240,"kb_avail":752509984,"statfs":{"total":773094113280,"available":770570223616,"internally_reserved":0,"allocated":22970368,"data_stored":11825256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2500853760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13041,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.009297"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":200,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:00.452024+0000","last_change":"2023-11-06T22:19:00.452024+0000","last_active":"2023-11-06T22:19:00.452024+0000","last_peered":"2023-11-06T22:19:00.452024+0000","last_clean":"2023-11-06T22:19:00.452024+0000","last_became_active":"2023-11-06T20:44:00.954496+0000","last_became_peered":"2023-11-06T20:44:00.954496+0000","last_unstale":"2023-11-06T22:19:00.452024+0000","last_undegraded":"2023-11-06T22:19:00.452024+0000","last_fullsized":"2023-11-06T22:19:00.452024+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:00.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:00.451887+0000","last_clean_scrub_stamp":"2023-11-06T22:19:00.451887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":213,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:16.835278+0000","last_change":"2023-11-06T22:19:16.835278+0000","last_active":"2023-11-06T22:19:16.835278+0000","last_peered":"2023-11-06T22:19:16.835278+0000","last_clean":"2023-11-06T22:19:16.835278+0000","last_became_active":"2023-11-06T20:44:00.954609+0000","last_became_peered":"2023-11-06T20:44:00.954609+0000","last_unstale":"2023-11-06T22:19:16.835278+0000","last_undegraded":"2023-11-06T22:19:16.835278+0000","last_fullsized":"2023-11-06T22:19:16.835278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:16.835174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:16.835174+0000","last_clean_scrub_stamp":"2023-11-06T22:19:16.835174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":222,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:02.758980+0000","last_change":"2023-11-06T22:19:02.758980+0000","last_active":"2023-11-06T22:19:02.758980+0000","last_peered":"2023-11-06T22:19:02.758980+0000","last_clean":"2023-11-06T22:19:02.758980+0000","last_became_active":"2023-11-06T20:44:00.856663+0000","last_became_peered":"2023-11-06T20:44:00.856663+0000","last_unstale":"2023-11-06T22:19:02.758980+0000","last_undegraded":"2023-11-06T22:19:02.758980+0000","last_fullsized":"2023-11-06T22:19:02.758980+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:02.758843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:02.758843+0000","last_clean_scrub_stamp":"2023-11-06T22:19:02.758843+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":213,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:20.823011+0000","last_change":"2023-11-06T22:19:20.823011+0000","last_active":"2023-11-06T22:19:20.823011+0000","last_peered":"2023-11-06T22:19:20.823011+0000","last_clean":"2023-11-06T22:19:20.823011+0000","last_became_active":"2023-11-06T20:44:01.002701+0000","last_became_peered":"2023-11-06T20:44:01.002701+0000","last_unstale":"2023-11-06T22:19:20.823011+0000","last_undegraded":"2023-11-06T22:19:20.823011+0000","last_fullsized":"2023-11-06T22:19:20.823011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:20.822901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:20.822901+0000","last_clean_scrub_stamp":"2023-11-06T22:19:20.822901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":199,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:18:58.186050+0000","last_change":"2023-11-06T22:18:58.186050+0000","last_active":"2023-11-06T22:18:58.186050+0000","last_peered":"2023-11-06T22:18:58.186050+0000","last_clean":"2023-11-06T22:18:58.186050+0000","last_became_active":"2023-11-06T20:43:59.859386+0000","last_became_peered":"2023-11-06T20:43:59.859386+0000","last_unstale":"2023-11-06T22:18:58.186050+0000","last_undegraded":"2023-11-06T22:18:58.186050+0000","last_fullsized":"2023-11-06T22:18:58.186050+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:58.185959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:58.185959+0000","last_clean_scrub_stamp":"2023-11-06T22:18:58.185959+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":213,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:09.962944+0000","last_change":"2023-11-06T22:19:09.962944+0000","last_active":"2023-11-06T22:19:09.962944+0000","last_peered":"2023-11-06T22:19:09.962944+0000","last_clean":"2023-11-06T22:19:09.962944+0000","last_became_active":"2023-11-06T20:44:00.858949+0000","last_became_peered":"2023-11-06T20:44:00.858949+0000","last_unstale":"2023-11-06T22:19:09.962944+0000","last_undegraded":"2023-11-06T22:19:09.962944+0000","last_fullsized":"2023-11-06T22:19:09.962944+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:09.962824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:09.962824+0000","last_clean_scrub_stamp":"2023-11-06T22:19:09.962824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":213,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:22.873981+0000","last_change":"2023-11-06T22:19:22.873981+0000","last_active":"2023-11-06T22:19:22.873981+0000","last_peered":"2023-11-06T22:19:22.873981+0000","last_clean":"2023-11-06T22:19:22.873981+0000","last_became_active":"2023-11-06T20:44:00.861864+0000","last_became_peered":"2023-11-06T20:44:00.861864+0000","last_unstale":"2023-11-06T22:19:22.873981+0000","last_undegraded":"2023-11-06T22:19:22.873981+0000","last_fullsized":"2023-11-06T22:19:22.873981+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:22.873874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:22.873874+0000","last_clean_scrub_stamp":"2023-11-06T22:19:22.873874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":199,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:18:59.722034+0000","last_change":"2023-11-06T22:18:59.722034+0000","last_active":"2023-11-06T22:18:59.722034+0000","last_peered":"2023-11-06T22:18:59.722034+0000","last_clean":"2023-11-06T22:18:59.722034+0000","last_became_active":"2023-11-06T20:44:00.968133+0000","last_became_peered":"2023-11-06T20:44:00.968133+0000","last_unstale":"2023-11-06T22:18:59.722034+0000","last_undegraded":"2023-11-06T22:18:59.722034+0000","last_fullsized":"2023-11-06T22:18:59.722034+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:18:59.721903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:18:59.721903+0000","last_clean_scrub_stamp":"2023-11-06T22:18:59.721903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":213,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-11-06T22:19:27.011456+0000","last_change":"2023-11-06T22:19:27.011456+0000","last_active":"2023-11-06T22:19:27.011456+0000","last_peered":"2023-11-06T22:19:27.011456+0000","last_clean":"2023-11-06T22:19:27.011456+0000","last_became_active":"2023-11-06T20:44:00.968075+0000","last_became_peered":"2023-11-06T20:44:00.968075+0000","last_unstale":"2023-11-06T22:19:27.011456+0000","last_undegraded":"2023-11-06T22:19:27.011456+0000","last_fullsized":"2023-11-06T22:19:27.011456+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T22:19:27.011329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T22:19:27.011329+0000","last_clean_scrub_stamp":"2023-11-06T22:19:27.011329+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.45300000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90100000000000002,"5min":0.95099999999999996,"15min":1.0980000000000001},"last":0.753},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.72599999999999998},"min":{"1min":0.48199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96899999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.73199999999999998}]},{"osd":1,"last update":"Mon Nov 6 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71299999999999997},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88900000000000001,"5min":0.94899999999999995,"15min":1.038},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70499999999999996,"15min":0.73099999999999998},"min":{"1min":0.59499999999999997,"5min":0.443,"15min":0.443},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":1.0680000000000001},"last":0.71699999999999997}]},{"osd":2,"last update":"Mon Nov 6 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.54600000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.032,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.57499999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90900000000000003,"5min":0.996,"15min":1.0209999999999999},"last":0.68200000000000005}]},{"osd":3,"last update":"Mon Nov 6 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.499},"max":{"1min":0.879,"5min":0.90700000000000003,"15min":0.95799999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.504},"max":{"1min":1.044,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Mon Nov 6 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.61899999999999999,"15min":0.60699999999999998},"min":{"1min":0.40899999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73099999999999998,"5min":1.052,"15min":1.052},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.45200000000000001},"max":{"1min":0.76900000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60399999999999998}]},{"osd":5,"last update":"Mon Nov 6 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.432,"15min":0.40400000000000003},"max":{"1min":0.83399999999999996,"5min":1.137,"15min":1.137},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.50800000000000001,"5min":0.46600000000000003,"15min":0.432},"max":{"1min":1.004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.625}]},{"osd":6,"last update":"Mon Nov 6 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.74199999999999999,"15min":0.73799999999999999},"min":{"1min":0.56699999999999995,"5min":0.504,"15min":0.48399999999999999},"max":{"1min":1.052,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95099999999999996,"5min":1.0409999999999999,"15min":1.069},"last":0.74299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575983,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66500000000000004,"15min":0.70499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.877,"5min":1.01,"15min":1.01},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68500000000000005,"15min":0.70899999999999996},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.434},"max":{"1min":0.89800000000000002,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Mon Nov 6 22:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69699999999999995,"15min":0.72399999999999998},"min":{"1min":0.442,"5min":0.372,"15min":0.372},"max":{"1min":0.92800000000000005,"5min":1.022,"15min":1.0660000000000001},"last":0.622},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88,"5min":0.92500000000000004,"15min":1.016},"last":0.71699999999999997}]},{"osd":2,"last update":"Mon Nov 6 22:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.78700000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.498},"max":{"1min":0.878,"5min":1.0209999999999999,"15min":1.0780000000000001},"last":0.745},{"interface":"front","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.78400000000000003},"min":{"1min":0.57099999999999995,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94599999999999995,"5min":0.97699999999999998,"15min":1.046},"last":0.77200000000000002}]},{"osd":3,"last update":"Mon Nov 6 22:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.72699999999999998},"min":{"1min":0.53400000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91600000000000004,"5min":0.94799999999999995,"15min":1.002},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.751,"15min":0.75700000000000001},"min":{"1min":0.55000000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"max":{"1min":0.93799999999999994,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.64900000000000002}]},{"osd":4,"last update":"Mon Nov 6 22:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88,"5min":0.89200000000000002,"15min":0.98299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91900000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.439}]},{"osd":5,"last update":"Mon Nov 6 22:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.47199999999999998,"5min":0.435,"15min":0.42799999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.49399999999999999,"5min":0.46100000000000002,"15min":0.442},"max":{"1min":0.98599999999999999,"5min":0.996,"15min":1.056},"last":0.52100000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68100000000000005,"15min":0.70799999999999996},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.497},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.72899999999999998,"15min":0.752},"min":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95999999999999996,"5min":1.0569999999999999,"15min":1.1279999999999999},"last":0.55800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46000000000000002,"15min":0.46500000000000002},"min":{"1min":0.311,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66700000000000004,"5min":0.69899999999999995,"15min":0.97999999999999998},"last":0.442},{"interface":"front","average":{"1min":0.496,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.37,"5min":0.34599999999999997,"15min":0.33800000000000002},"max":{"1min":0.95999999999999996,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.505}]},{"osd":2,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.5,"15min":0.50600000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72399999999999998,"5min":0.77100000000000002,"15min":0.94199999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69899999999999995,"5min":0.80700000000000005,"15min":1.0189999999999999},"last":0.57299999999999995}]},{"osd":3,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.442,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.0509999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79800000000000004,"5min":0.79900000000000004,"15min":0.87},"last":0.59999999999999998}]},{"osd":4,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79600000000000004,"15min":0.80500000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.51200000000000001},"max":{"1min":0.91900000000000004,"5min":1.329,"15min":1.329},"last":0.872},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1599999999999999,"5min":1.337,"15min":1.337},"last":0.88300000000000001}]},{"osd":5,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79000000000000004,"15min":0.81100000000000005},"min":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0329999999999999,"5min":1.2210000000000001,"15min":1.2709999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.80300000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.55400000000000005},"max":{"1min":1.149,"5min":1.224,"15min":1.3109999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81100000000000005,"15min":0.82699999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.119,"5min":1.2709999999999999,"15min":1.3149999999999999},"last":0.745},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.83699999999999997,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Mon Nov 6 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.86099999999999999,"15min":0.86599999999999999},"min":{"1min":0.65900000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.014,"5min":1.1970000000000001,"15min":1.2509999999999999},"last":0.91200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575983,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47499999999999998,"15min":0.46899999999999997},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"last":0.308},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.34000000000000002}]},{"osd":2,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.48499999999999999},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.58099999999999996}]},{"osd":3,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.375},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.40000000000000002}]},{"osd":4,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.85199999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.59699999999999998,"5min":0.59299999999999997,"15min":0.54800000000000004},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.84199999999999997,"15min":0.85399999999999998},"min":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.64500000000000002}]},{"osd":6,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81699999999999995,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.52600000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83599999999999997,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.63100000000000001}]},{"osd":7,"last update":"Mon Nov 6 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82999999999999996,"15min":0.85699999999999998},"min":{"1min":0.55200000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.268},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.85499999999999998,"15min":0.876},"min":{"1min":0.58299999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.72999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70799999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":0.83799999999999997},"last":0.502}]},{"osd":1,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55900000000000005,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83999999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.74299999999999999,"5min":0.83699999999999997,"15min":0.89900000000000002},"last":0.61099999999999999}]},{"osd":3,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68899999999999995,"5min":0.74399999999999999,"15min":0.78300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.39100000000000001},"max":{"1min":0.753,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63700000000000001}]},{"osd":4,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.81200000000000006,"15min":0.82199999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.53900000000000003},"max":{"1min":1.0569999999999999,"5min":1.246,"15min":1.349},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.80600000000000005,"15min":0.81699999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0700000000000001,"5min":1.2669999999999999,"15min":1.339},"last":0.79200000000000004}]},{"osd":5,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.48999999999999999},"max":{"1min":1.0860000000000001,"5min":1.276,"15min":1.329},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.82299999999999995,"15min":0.84999999999999998},"min":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1419999999999999,"5min":1.212,"15min":1.468},"last":0.86199999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.81999999999999995,"15min":0.84199999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.123,"5min":1.155,"15min":1.387},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.82199999999999995,"15min":0.84499999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97299999999999998,"5min":1.175,"15min":1.4299999999999999},"last":0.76900000000000002}]},{"osd":7,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84499999999999997,"15min":0.86899999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98799999999999999,"5min":1.2150000000000001,"15min":1.496},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.64200000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.042,"5min":1.2470000000000001,"15min":1.508},"last":0.878}]}]},{"osd":3,"up_from":13,"seq":55834575984,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45700000000000002,"15min":0.47499999999999998},"min":{"1min":0.28399999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.31},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58199999999999996}]},{"osd":1,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.68700000000000006,"5min":1.05,"15min":1.05},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.57199999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70499999999999996,"5min":0.873,"15min":0.873},"last":0.66200000000000003}]},{"osd":2,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54400000000000004,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.80300000000000005,"15min":0.87},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.70299999999999996}]},{"osd":4,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0669999999999999,"5min":1.161,"15min":1.2370000000000001},"last":1.012},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.79900000000000004,"15min":0.82899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1200000000000001,"5min":1.147,"15min":1.226},"last":1.145}]},{"osd":5,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77700000000000002,"15min":0.80700000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0449999999999999,"5min":1.071,"15min":1.2270000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.3140000000000001},"last":1.1299999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.82399999999999995},"min":{"1min":0.52700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0429999999999999,"5min":1.149,"15min":1.2290000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81200000000000006,"15min":0.85199999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.099,"5min":1.099,"15min":1.1879999999999999},"last":1.069}]},{"osd":7,"last update":"Mon Nov 6 22:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.196,"5min":1.196,"15min":1.2829999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.81799999999999995,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.113,"5min":1.169,"15min":1.248},"last":1.101}]}]},{"osd":4,"up_from":13,"seq":55834575983,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70499999999999996,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98999999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68100000000000005,"15min":0.71099999999999997},"min":{"1min":0.45700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82699999999999996,"5min":0.95799999999999996,"15min":1.006},"last":0.71499999999999997}]},{"osd":1,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70299999999999996,"15min":0.72699999999999998},"min":{"1min":0.53700000000000003,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92500000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.98599999999999999,"15min":1.087},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.48299999999999998},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.071},"last":0.84599999999999997}]},{"osd":3,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.875,"5min":0.93600000000000005,"15min":1.0329999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90400000000000003,"5min":1.236,"15min":1.236},"last":0.67000000000000004}]},{"osd":5,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93600000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.48399999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88700000000000001,"5min":0.95999999999999996,"15min":0.96799999999999997},"last":0.48799999999999999}]},{"osd":6,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90600000000000003,"5min":1.018,"15min":1.018},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.505,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0369999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Mon Nov 6 22:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.53200000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.54400000000000004,"5min":0.51500000000000001,"15min":0.50800000000000001},"max":{"1min":1.0469999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575984,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308092,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063748,"statfs":{"total":96636764160,"available":96321277952,"internally_reserved":0,"allocated":2871296,"data_stored":1478157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.309},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1639999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.65900000000000003}]},{"osd":1,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77800000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47999999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.76700000000000002}]},{"osd":2,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78700000000000003,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.48299999999999998},"max":{"1min":0.96499999999999997,"5min":4.9779999999999998,"15min":4.9779999999999998},"last":0.61299999999999999}]},{"osd":3,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.36499999999999999},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.752,"15min":0.747},"min":{"1min":0.499,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.2390000000000001,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":0.81000000000000005}]},{"osd":4,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":1.502,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.38800000000000001},"max":{"1min":1.4019999999999999,"5min":1.585,"15min":1.585},"last":0.51200000000000001}]},{"osd":6,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":1.4510000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77400000000000002,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.441,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":5.1029999999999998,"15min":5.1029999999999998},"last":0.752}]},{"osd":7,"last update":"Mon Nov 6 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.41399999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.62},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78200000000000003,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":1.4810000000000001,"5min":6.891,"15min":6.891},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T22:19:42.210 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-06T22:19:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.398+0000 7feee3dff700 1 Processor -- start 2023-11-06T22:19:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.398+0000 7feee3dff700 1 -- start start 2023-11-06T22:19:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.399+0000 7feee3dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feedc154010 0x7feedc1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.399+0000 7feee3dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feedc064670 0x7feedc064b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.399+0000 7feee3dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc065010 con 0x7feedc0aafc0 2023-11-06T22:19:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.399+0000 7feee3dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc0c6280 con 0x7feedc064670 2023-11-06T22:19:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.399+0000 7feee3dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0c0d90 con 0x7feedc154010 2023-11-06T22:19:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.400+0000 7feee239c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feedc064670 0x7feedc064b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.400+0000 7feee239c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feedc064670 0x7feedc064b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48666/0 (socket says 172.21.15.33:48666) 2023-11-06T22:19:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.400+0000 7feee239c700 1 -- 172.21.15.33:0/2253586773 learned_addr learned my addr 172.21.15.33:0/2253586773 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.400+0000 7feee1b9b700 1 --2- 172.21.15.33:0/2253586773 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feedc154010 0x7feedc1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.400+0000 7feee239c700 1 -- 172.21.15.33:0/2253586773 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feedc154010 msgr2=0x7feedc1543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.400+0000 7feee239c700 1 --2- 172.21.15.33:0/2253586773 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feedc154010 0x7feedc1543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.400+0000 7feee239c700 1 -- 172.21.15.33:0/2253586773 >> v1:172.21.15.69:6789/0 conn(0x7feedc0aafc0 legacy=0x7feedc0c0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.400+0000 7feee239c700 1 -- 172.21.15.33:0/2253586773 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc0c7720 con 0x7feedc064670 2023-11-06T22:19:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.400+0000 7feee239c700 1 --2- 172.21.15.33:0/2253586773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc064670 0x7feedc064b20 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 rx=0x7feed4009f20 tx=0x7feed4009840).ready entity=mon.0 client_cookie=911d6e5901fbd11f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee0b99700 1 -- 172.21.15.33:0/2253586773 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400e070 con 0x7feedc064670 2023-11-06T22:19:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee0b99700 1 -- 172.21.15.33:0/2253586773 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed400ba40 con 0x7feedc064670 2023-11-06T22:19:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee0b99700 1 -- 172.21.15.33:0/2253586773 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400bdc0 con 0x7feedc064670 2023-11-06T22:19:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee3dff700 1 -- 172.21.15.33:0/2253586773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc064670 msgr2=0x7feedc064b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee3dff700 1 --2- 172.21.15.33:0/2253586773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc064670 0x7feedc064b20 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 rx=0x7feed4009f20 tx=0x7feed4009840).stop 2023-11-06T22:19:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee3dff700 1 -- 172.21.15.33:0/2253586773 shutdown_connections 2023-11-06T22:19:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee3dff700 1 --2- 172.21.15.33:0/2253586773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc064670 0x7feedc064b20 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee3dff700 1 --2- 172.21.15.33:0/2253586773 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feedc154010 0x7feedc1543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee3dff700 1 -- 172.21.15.33:0/2253586773 >> 172.21.15.33:0/2253586773 conn(0x7feedc0cb8e0 msgr2=0x7feedc0c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:42.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee3dff700 1 -- 172.21.15.33:0/2253586773 shutdown_connections 2023-11-06T22:19:42.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.401+0000 7feee3dff700 1 -- 172.21.15.33:0/2253586773 wait complete. 2023-11-06T22:19:42.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.403+0000 7feee3dff700 1 Processor -- start 2023-11-06T22:19:42.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.403+0000 7feee3dff700 1 -- start start 2023-11-06T22:19:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee3dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feedc064670 0x7feedc0aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee3dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feedc154010 0x7feedc0af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee3dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc0afb70 0x7feedc1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee3dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0affd0 con 0x7feedc0afb70 2023-11-06T22:19:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee3dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc150290 con 0x7feedc154010 2023-11-06T22:19:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee3dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0aa6e0 con 0x7feedc064670 2023-11-06T22:19:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee239c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc0afb70 0x7feedc1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee239c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc0afb70 0x7feedc1525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48682/0 (socket says 172.21.15.33:48682) 2023-11-06T22:19:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee239c700 1 -- 172.21.15.33:0/849268935 learned_addr learned my addr 172.21.15.33:0/849268935 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee1b9b700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feedc064670 0x7feedc0aa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.404+0000 7feee139a700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feedc154010 0x7feedc0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feee239c700 1 -- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feedc064670 msgr2=0x7feedc0aa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feee239c700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feedc064670 0x7feedc0aa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feee239c700 1 -- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feedc154010 msgr2=0x7feedc0af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feee239c700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feedc154010 0x7feedc0af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feee239c700 1 -- 172.21.15.33:0/849268935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed4009ae0 con 0x7feedc0afb70 2023-11-06T22:19:42.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feee239c700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc0afb70 0x7feedc1525a0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7feedc062d70 tx=0x7feedc0c61f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:42.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feed1ffb700 1 -- 172.21.15.33:0/849268935 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40092e0 con 0x7feedc0afb70 2023-11-06T22:19:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feed1ffb700 1 -- 172.21.15.33:0/849268935 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed4013e30 con 0x7feedc0afb70 2023-11-06T22:19:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feee3dff700 1 -- 172.21.15.33:0/849268935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc1644a0 con 0x7feedc0afb70 2023-11-06T22:19:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feed1ffb700 1 -- 172.21.15.33:0/849268935 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed401e6c0 con 0x7feedc0afb70 2023-11-06T22:19:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.405+0000 7feee3dff700 1 -- 172.21.15.33:0/849268935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc0b04d0 con 0x7feedc0afb70 2023-11-06T22:19:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.406+0000 7feee3dff700 1 -- 172.21.15.33:0/849268935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc0652f0 con 0x7feedc0afb70 2023-11-06T22:19:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.407+0000 7feed1ffb700 1 -- 172.21.15.33:0/849268935 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7feed4003040 con 0x7feedc0afb70 2023-11-06T22:19:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.408+0000 7feed1ffb700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7feec005f270 0x7feec0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.408+0000 7feed1ffb700 1 -- 172.21.15.33:0/849268935 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7feed407b6f0 con 0x7feedc0afb70 2023-11-06T22:19:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.408+0000 7feee1b9b700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7feec005f270 0x7feec0061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.409+0000 7feee1b9b700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7feec005f270 0x7feec0061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7feed800bb10 tx=0x7feed8008040).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.412+0000 7feed1ffb700 1 -- 172.21.15.33:0/849268935 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feed404c8a0 con 0x7feedc0afb70 2023-11-06T22:19:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.590+0000 7feee3dff700 1 -- 172.21.15.33:0/849268935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7feedc065010 con 0x7feedc0afb70 2023-11-06T22:19:42.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.596+0000 7feed1ffb700 1 -- 172.21.15.33:0/849268935 <== mon.0 v2:172.21.15.33:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7feed404c8a0 con 0x7feedc0afb70 2023-11-06T22:19:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.596+0000 7feed1ffb700 1 -- 172.21.15.33:0/849268935 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v46)=0 v46) v1 ==== 123+0+0 (secure 0 0 0) 0x7feedc065010 con 0x7feedc0afb70 2023-11-06T22:19:42.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 -- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7feec005f270 msgr2=0x7feec0061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7feec005f270 0x7feec0061730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7feed800bb10 tx=0x7feed8008040).stop 2023-11-06T22:19:42.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 -- 172.21.15.33:0/849268935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc0afb70 msgr2=0x7feedc1525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc0afb70 0x7feedc1525a0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7feedc062d70 tx=0x7feedc0c61f0).stop 2023-11-06T22:19:42.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 -- 172.21.15.33:0/849268935 shutdown_connections 2023-11-06T22:19:42.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7feec005f270 0x7feec0061730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7feedc064670 0x7feedc0aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feedc0afb70 0x7feedc1525a0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 --2- 172.21.15.33:0/849268935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feedc154010 0x7feedc0af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 -- 172.21.15.33:0/849268935 >> 172.21.15.33:0/849268935 conn(0x7feedc0cb8e0 msgr2=0x7feedc0abe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:42.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 -- 172.21.15.33:0/849268935 shutdown_connections 2023-11-06T22:19:42.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.599+0000 7feee3dff700 1 -- 172.21.15.33:0/849268935 wait complete. 2023-11-06T22:19:42.619 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T22:19:42.620 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T22:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.812+0000 7f0bf5dff700 1 Processor -- start 2023-11-06T22:19:42.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.812+0000 7f0bf5dff700 1 -- start start 2023-11-06T22:19:42.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.814+0000 7f0bf5dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0bf0063280 0x7f0bf0063660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.814+0000 7f0bf5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bf00c15f0 0x7f0bf014e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.814+0000 7f0bf5dff700 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf014ee10 con 0x7f0bf014e4a0 2023-11-06T22:19:42.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.814+0000 7f0bf5dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf0158e20 con 0x7f0bf0063280 2023-11-06T22:19:42.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.814+0000 7f0bf5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf01582a0 con 0x7f0bf00c15f0 2023-11-06T22:19:42.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.814+0000 7f0bef7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0bf0063280 0x7f0bf0063660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.814+0000 7f0bef7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0bf0063280 0x7f0bf0063660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48694/0 (socket says 172.21.15.33:48694) 2023-11-06T22:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.814+0000 7f0bef7fe700 1 -- 172.21.15.33:0/2982343379 learned_addr learned my addr 172.21.15.33:0/2982343379 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.814+0000 7f0beffff700 1 --2- 172.21.15.33:0/2982343379 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bf00c15f0 0x7f0bf014e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.815+0000 7f0bef7fe700 1 -- 172.21.15.33:0/2982343379 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bf00c15f0 msgr2=0x7f0bf014e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T22:19:42.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.815+0000 7f0bef7fe700 1 --2- 172.21.15.33:0/2982343379 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bf00c15f0 0x7f0bf014e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.815+0000 7f0bef7fe700 1 -- 172.21.15.33:0/2982343379 >> v1:172.21.15.69:6789/0 conn(0x7f0bf014e4a0 legacy=0x7f0bf00645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.815+0000 7f0bef7fe700 1 -- 172.21.15.33:0/2982343379 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0158580 con 0x7f0bf0063280 2023-11-06T22:19:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.815+0000 7f0bef7fe700 1 --2- 172.21.15.33:0/2982343379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0063280 0x7f0bf0063660 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7f0be0006fd0 tx=0x7f0be00090f0).ready entity=mon.0 client_cookie=b467544d0604084 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.815+0000 7f0bee7fc700 1 -- 172.21.15.33:0/2982343379 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be000b580 con 0x7f0bf0063280 2023-11-06T22:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.815+0000 7f0bee7fc700 1 -- 172.21.15.33:0/2982343379 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0be000b6e0 con 0x7f0bf0063280 2023-11-06T22:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.815+0000 7f0bee7fc700 1 -- 172.21.15.33:0/2982343379 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be000ba80 con 0x7f0bf0063280 2023-11-06T22:19:42.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.816+0000 7f0bf5dff700 1 -- 172.21.15.33:0/2982343379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0063280 msgr2=0x7f0bf0063660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.816+0000 7f0bf5dff700 1 --2- 172.21.15.33:0/2982343379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0063280 0x7f0bf0063660 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7f0be0006fd0 tx=0x7f0be00090f0).stop 2023-11-06T22:19:42.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.816+0000 7f0bf5dff700 1 -- 172.21.15.33:0/2982343379 shutdown_connections 2023-11-06T22:19:42.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.816+0000 7f0bf5dff700 1 --2- 172.21.15.33:0/2982343379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0063280 0x7f0bf0063660 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.816+0000 7f0bf5dff700 1 --2- 172.21.15.33:0/2982343379 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bf00c15f0 0x7f0bf014e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.816+0000 7f0bf5dff700 1 -- 172.21.15.33:0/2982343379 >> 172.21.15.33:0/2982343379 conn(0x7f0bf00cade0 msgr2=0x7f0bf00cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:42.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.816+0000 7f0bf5dff700 1 -- 172.21.15.33:0/2982343379 shutdown_connections 2023-11-06T22:19:42.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.816+0000 7f0bf5dff700 1 -- 172.21.15.33:0/2982343379 wait complete. 2023-11-06T22:19:42.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.817+0000 7f0bf5dff700 1 Processor -- start 2023-11-06T22:19:42.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.817+0000 7f0bf5dff700 1 -- start start 2023-11-06T22:19:42.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.818+0000 7f0bf5dff700 1 --2- >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bf0063280 0x7f0bf00ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.818+0000 7f0bf5dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf00c15f0 0x7f0bf00c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.818+0000 7f0bf5dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf014e4a0 0x7f0bf00a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.818+0000 7f0bf5dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00a20c0 con 0x7f0bf00c15f0 2023-11-06T22:19:42.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.818+0000 7f0bf5dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00a23a0 con 0x7f0bf014e4a0 2023-11-06T22:19:42.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.818+0000 7f0bf5dff700 1 -- --> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00a2680 con 0x7f0bf0063280 2023-11-06T22:19:42.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0beeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf00c15f0 0x7f0bf00c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0beeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf00c15f0 0x7f0bf00c4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48698/0 (socket says 172.21.15.33:48698) 2023-11-06T22:19:42.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0beeffd700 1 -- 172.21.15.33:0/394845702 learned_addr learned my addr 172.21.15.33:0/394845702 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-11-06T22:19:42.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0bef7fe700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bf0063280 0x7f0bf00ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0beffff700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf014e4a0 0x7f0bf00a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0beeffd700 1 -- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bf0063280 msgr2=0x7f0bf00ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0beeffd700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bf0063280 0x7f0bf00ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0beeffd700 1 -- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf014e4a0 msgr2=0x7f0bf00a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:42.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0beeffd700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf014e4a0 0x7f0bf00a3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:42.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0beeffd700 1 -- 172.21.15.33:0/394845702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0009370 con 0x7f0bf00c15f0 2023-11-06T22:19:42.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.819+0000 7f0beeffd700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf00c15f0 0x7f0bf00c4050 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 rx=0x7f0be400c4c0 tx=0x7f0be4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:42.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.820+0000 7f0bd77fe700 1 -- 172.21.15.33:0/394845702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be400ce70 con 0x7f0bf00c15f0 2023-11-06T22:19:42.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.820+0000 7f0bd77fe700 1 -- 172.21.15.33:0/394845702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0be4001e90 con 0x7f0bf00c15f0 2023-11-06T22:19:42.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.820+0000 7f0bf5dff700 1 -- 172.21.15.33:0/394845702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf00a2960 con 0x7f0bf00c15f0 2023-11-06T22:19:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.820+0000 7f0bd77fe700 1 -- 172.21.15.33:0/394845702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4007270 con 0x7f0bf00c15f0 2023-11-06T22:19:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.820+0000 7f0bf5dff700 1 -- 172.21.15.33:0/394845702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf013f480 con 0x7f0bf00c15f0 2023-11-06T22:19:42.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.821+0000 7f0bf5dff700 1 -- 172.21.15.33:0/394845702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf006c6c0 con 0x7f0bf00c15f0 2023-11-06T22:19:42.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.821+0000 7f0bd77fe700 1 -- 172.21.15.33:0/394845702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 105) v1 ==== 75548+0+0 (secure 0 0 0) 0x7f0be40066a0 con 0x7f0bf00c15f0 2023-11-06T22:19:42.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.822+0000 7f0bd77fe700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0bdc05f270 0x7f0bdc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-06T22:19:42.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.822+0000 7f0bd77fe700 1 -- 172.21.15.33:0/394845702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5872+0+0 (secure 0 0 0) 0x7f0be4079c10 con 0x7f0bf00c15f0 2023-11-06T22:19:42.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.823+0000 7f0bef7fe700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0bdc05f270 0x7f0bdc061730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-06T22:19:42.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.823+0000 7f0bef7fe700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0bdc05f270 0x7f0bdc061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0be0009fd0 tx=0x7f0be00090f0).ready entity=mgr.9718 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T22:19:42.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:42.826+0000 7f0bd77fe700 1 -- 172.21.15.33:0/394845702 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0be404ae40 con 0x7f0bf00c15f0 2023-11-06T22:19:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.050+0000 7f0bf5dff700 1 -- 172.21.15.33:0/394845702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bf00bfeb0 con 0x7f0bf00c15f0 2023-11-06T22:19:43.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.051+0000 7f0bd77fe700 1 -- 172.21.15.33:0/394845702 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f0be404e460 con 0x7f0bf00c15f0 2023-11-06T22:19:43.054 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-11-06T22:19:43.054 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8195,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3097639551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3097639551},{"type":"v1","addr":"172.21.15.69:6837","nonce":3097639551}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9828,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1256245016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1256245016},{"type":"v1","addr":"172.21.15.33:6834","nonce":1256245016}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1680210655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1680210655},{"type":"v1","addr":"172.21.15.33:6835","nonce":1680210655}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1949922141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1949922141},{"type":"v1","addr":"172.21.15.69:6836","nonce":1949922141}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[]} 2023-11-06T22:19:43.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 -- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0bdc05f270 msgr2=0x7f0bdc061730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:43.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0bdc05f270 0x7f0bdc061730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0be0009fd0 tx=0x7f0be00090f0).stop 2023-11-06T22:19:43.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 -- 172.21.15.33:0/394845702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf00c15f0 msgr2=0x7f0bf00c4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T22:19:43.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf00c15f0 0x7f0bf00c4050 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 rx=0x7f0be400c4c0 tx=0x7f0be4007e00).stop 2023-11-06T22:19:43.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 -- 172.21.15.33:0/394845702 shutdown_connections 2023-11-06T22:19:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:6832/129946,v1:172.21.15.69:6833/129946] conn(0x7f0bdc05f270 0x7f0bdc061730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f0bf0063280 0x7f0bf00ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf014e4a0 0x7f0bf00a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 --2- 172.21.15.33:0/394845702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf00c15f0 0x7f0bf00c4050 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-06T22:19:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 -- 172.21.15.33:0/394845702 >> 172.21.15.33:0/394845702 conn(0x7f0bf00cade0 msgr2=0x7f0bf014dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T22:19:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 -- 172.21.15.33:0/394845702 shutdown_connections 2023-11-06T22:19:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-11-06T22:19:43.053+0000 7f0bf5dff700 1 -- 172.21.15.33:0/394845702 wait complete. 2023-11-06T22:19:43.058 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 93 2023-11-06T22:19:43.073 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-06T22:19:43.073 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-06T22:19:43.074 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T22:19:43.074 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.086 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T22:19:43.087 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T22:19:43.087 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.140 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T22:19:43.140 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T22:19:43.141 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.156 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T22:19:43.156 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T22:19:43.157 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.212 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T22:19:43.212 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-06T22:19:43.213 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-06T22:19:43.213 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.260 INFO:tasks.ceph.osd.0:Stopped 2023-11-06T22:19:43.260 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-06T22:19:43.261 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.311 INFO:tasks.ceph.osd.1:Stopped 2023-11-06T22:19:43.312 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-06T22:19:43.312 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.355 INFO:tasks.ceph.osd.2:Stopped 2023-11-06T22:19:43.355 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-06T22:19:43.355 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.407 INFO:tasks.ceph.osd.3:Stopped 2023-11-06T22:19:43.407 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-06T22:19:43.408 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.455 INFO:tasks.ceph.osd.4:Stopped 2023-11-06T22:19:43.456 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-06T22:19:43.456 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.513 INFO:tasks.ceph.osd.5:Stopped 2023-11-06T22:19:43.514 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-06T22:19:43.514 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.564 INFO:tasks.ceph.osd.6:Stopped 2023-11-06T22:19:43.565 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-06T22:19:43.565 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.607 INFO:tasks.ceph.osd.7:Stopped 2023-11-06T22:19:43.608 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-06T22:19:43.608 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-06T22:19:43.609 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.664 INFO:tasks.ceph.mgr.y:Stopped 2023-11-06T22:19:43.665 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-06T22:19:43.665 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.727 INFO:tasks.ceph.mgr.x:Stopped 2023-11-06T22:19:43.727 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-06T22:19:43.728 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-06T22:19:43.728 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.772 INFO:tasks.ceph.mon.a:Stopped 2023-11-06T22:19:43.773 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-06T22:19:43.773 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.804 INFO:tasks.ceph.mon.b:Stopped 2023-11-06T22:19:43.805 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-06T22:19:43.805 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T22:19:43.877 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-11-06T22:19:43.877 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-11-06T22:19:43.877 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-11-06T22:19:43.878 INFO:tasks.ceph.mon.c:Stopped 2023-11-06T22:19:43.878 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-06T22:19:43.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-06T22:19:43.964 INFO:teuthology.orchestra.run.smithi033.stdout:2023-11-06T20:46:26.917225+0000 mon.a (mon.0) 322 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2023-11-06T22:19:43.965 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-11-06T22:19:43.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-06T22:19:44.049 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-06T22:19:44.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-06T22:19:44.217 INFO:teuthology.orchestra.run.smithi033.stdout:2023-11-06T20:46:26.917225+0000 mon.a (mon.0) 322 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2023-11-06T22:19:44.218 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T22:19:44.219 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-06T22:19:44.433 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T22:19:44.434 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-06T22:19:44.554 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T22:19:44.555 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-06T22:19:44.696 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T22:19:44.697 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-06T22:19:44.814 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T22:19:44.814 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-06T22:19:45.035 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T22:19:45.036 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-06T22:19:45.153 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T22:19:45.153 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-06T22:19:45.285 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T22:19:45.286 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-06T22:19:45.445 INFO:tasks.ceph:Archiving mon data... 2023-11-06T22:19:45.447 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362/data/mon.a.tgz 2023-11-06T22:19:45.447 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2023-11-06T22:19:45.497 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.1Ufeb31UE6 2023-11-06T22:19:45.498 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.1Ufeb31UE6 2023-11-06T22:19:46.041 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.1Ufeb31UE6 2023-11-06T22:19:46.162 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.1Ufeb31UE6 is 2MB 2023-11-06T22:19:46.301 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.1Ufeb31UE6 2023-11-06T22:19:46.348 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362/data/mon.b.tgz 2023-11-06T22:19:46.349 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2023-11-06T22:19:46.399 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.TXZRRoyr7Y 2023-11-06T22:19:46.400 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.TXZRRoyr7Y 2023-11-06T22:19:46.967 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.TXZRRoyr7Y 2023-11-06T22:19:47.086 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.TXZRRoyr7Y is 2MB 2023-11-06T22:19:47.216 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.TXZRRoyr7Y 2023-11-06T22:19:47.263 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362/data/mon.c.tgz 2023-11-06T22:19:47.264 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2023-11-06T22:19:47.350 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.ProENHQBYA 2023-11-06T22:19:47.350 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ProENHQBYA 2023-11-06T22:19:47.902 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.ProENHQBYA 2023-11-06T22:19:48.021 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.ProENHQBYA is 2MB 2023-11-06T22:19:48.149 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.ProENHQBYA 2023-11-06T22:19:48.196 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-06T22:19:48.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-06T22:19:48.202 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-06T22:19:48.307 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-06T22:19:48.307 DEBUG:teuthology.orchestra.run.smithi033:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-06T22:19:48.313 DEBUG:teuthology.orchestra.run.smithi069:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-06T22:19:48.391 INFO:teuthology.orchestra.run.smithi069.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-06T22:19:48.403 INFO:teuthology.orchestra.run.smithi033.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-06T22:19:48.420 INFO:tasks.ceph:Archiving crash dumps... 2023-11-06T22:19:48.422 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362/remote/smithi033/crash 2023-11-06T22:19:48.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-06T22:19:48.494 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362/remote/smithi069/crash 2023-11-06T22:19:48.495 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-06T22:19:48.564 INFO:tasks.ceph:Shutting down logrotate 2023-11-06T22:19:48.565 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:19:48.570 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T22:19:48.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-06T22:19:48.737 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-06T22:19:48.807 INFO:tasks.ceph:Compressing logs... 2023-11-06T22:19:48.808 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-06T22:19:48.813 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-06T22:19:48.878 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-11-06T22:19:48.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-11-06T22:19:48.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-11-06T22:19:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133996.log 2023-11-06T22:19:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133937.log 2023-11-06T22:19:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.133996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128197.log 2023-11-06T22:19:48.881 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133996.log.gz 2023-11-06T22:19:48.882 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115141.log 2023-11-06T22:19:48.882 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-06T22:19:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.133937.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.133937.log.gz 2023-11-06T22:19:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161842.log 2023-11-06T22:19:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.128197.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128197.log.gz 2023-11-06T22:19:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153871.log 2023-11-06T22:19:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.115141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115493.log 2023-11-06T22:19:48.892 INFO:teuthology.orchestra.run.smithi069.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115141.log.gz 2023-11-06T22:19:48.892 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115443.log 2023-11-06T22:19:48.892 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.115493.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.115493.log.gz 2023-11-06T22:19:48.892 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119243.log 2023-11-06T22:19:48.893 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.115443.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.115443.log.gz 2023-11-06T22:19:48.893 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-06T22:19:48.893 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.119243.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119243.log.gz 2023-11-06T22:19:48.893 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115345.log 2023-11-06T22:19:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.120298.log 2023-11-06T22:19:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.115345.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115345.log.gz 2023-11-06T22:19:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.119758.log 2023-11-06T22:19:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.3.120298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.120298.log.gz 2023-11-06T22:19:48.897 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-06T22:19:48.898 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.1.119758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.119758.log.gz 2023-11-06T22:19:48.898 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-06T22:19:48.898 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119825.log 2023-11-06T22:19:48.899 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-mgr.x.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-06T22:19:48.899 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-06T22:19:48.899 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.119825.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-06T22:19:48.899 INFO:teuthology.orchestra.run.smithi069.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119825.log.gz 2023-11-06T22:19:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161842.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161842.log.gz 2023-11-06T22:19:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154522.log 2023-11-06T22:19:48.900 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153871.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153871.log.gz 2023-11-06T22:19:48.901 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161438.log 2023-11-06T22:19:48.901 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154522.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154522.log.gz 2023-11-06T22:19:48.901 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142447.log 2023-11-06T22:19:48.902 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161438.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161438.log.gz 2023-11-06T22:19:48.902 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175220.log 2023-11-06T22:19:48.902 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142447.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.142447.log.gz 2023-11-06T22:19:48.903 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127117.log 2023-11-06T22:19:48.903 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175220.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.175220.log.gz 2023-11-06T22:19:48.904 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149877.log 2023-11-06T22:19:48.904 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127117.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.127117.log.gz 2023-11-06T22:19:48.904 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148293.log 2023-11-06T22:19:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149877.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149877.log.gz 2023-11-06T22:19:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127733.log 2023-11-06T22:19:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148293.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148293.log.gz 2023-11-06T22:19:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153773.log 2023-11-06T22:19:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127733.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.127733.log.gz 2023-11-06T22:19:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176808.log 2023-11-06T22:19:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153773.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.153773.log.gz 2023-11-06T22:19:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173034.log 2023-11-06T22:19:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176808.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.176808.log.gz 2023-11-06T22:19:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176506.log 2023-11-06T22:19:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150706.log 2023-11-06T22:19:48.908 INFO:teuthology.orchestra.run.smithi033.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.173034.log.gz 2023-11-06T22:19:48.914 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-06T22:19:48.914 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115190.log 2023-11-06T22:19:48.914 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-client.admin.115190.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115190.log.gz 2023-11-06T22:19:48.914 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-06T22:19:48.914 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-06T22:19:48.915 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115295.log 2023-11-06T22:19:48.915 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-06T22:19:48.918 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.115295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115394.log 2023-11-06T22:19:48.919 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-osd.4.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115295.log.gz 2023-11-06T22:19:48.923 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.115394.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.115394.log.gz 2023-11-06T22:19:48.923 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-06T22:19:48.927 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176506.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.176506.log.gz 2023-11-06T22:19:48.927 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140750.log 2023-11-06T22:19:48.928 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150706.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150706.log.gz 2023-11-06T22:19:48.928 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152707.log 2023-11-06T22:19:48.928 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140750.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140750.log.gz 2023-11-06T22:19:48.928 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152654.log 2023-11-06T22:19:48.929 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152707.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.152707.log.gz 2023-11-06T22:19:48.929 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172826.log 2023-11-06T22:19:48.929 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152654.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.152654.log.gz 2023-11-06T22:19:48.929 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174660.log 2023-11-06T22:19:48.929 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172826.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.172826.log.gz 2023-11-06T22:19:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174709.log 2023-11-06T22:19:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174660.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.174660.log.gz 2023-11-06T22:19:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146127.log 2023-11-06T22:19:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174709.log: gzip 90.7% -- replaced with /var/log/ceph/ceph-client.admin.174709.log.gz-5 2023-11-06T22:19:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:--verbose -- /var/log/ceph/ceph-client.admin.124681.log 2023-11-06T22:19:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146127.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146127.log.gz 2023-11-06T22:19:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128295.log 2023-11-06T22:19:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124681.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124681.log.gz 2023-11-06T22:19:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160483.log 2023-11-06T22:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.128295.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128295.log.gz 2023-11-06T22:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127903.log 2023-11-06T22:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160483.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.160483.log.gz 2023-11-06T22:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152810.log 2023-11-06T22:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127903.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.127903.log.gz 2023-11-06T22:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125310.log 2023-11-06T22:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152810.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152810.log.gz 2023-11-06T22:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170037.log 2023-11-06T22:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125310.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125310.log.gz 2023-11-06T22:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139092.log 2023-11-06T22:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.170037.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.170037.log.gz 2023-11-06T22:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173872.log 2023-11-06T22:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139092.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.139092.log.gz 2023-11-06T22:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148597.log 2023-11-06T22:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173872.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.173872.log.gz 2023-11-06T22:19:48.935 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169793.log 2023-11-06T22:19:48.935 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148597.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.148597.log.gz 2023-11-06T22:19:48.935 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149221.log 2023-11-06T22:19:48.935 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169793.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169793.log.gz 2023-11-06T22:19:48.935 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149073.log 2023-11-06T22:19:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149221.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149221.log.gz 2023-11-06T22:19:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177004.log 2023-11-06T22:19:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149073.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.149073.log.gz 2023-11-06T22:19:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150511.log 2023-11-06T22:19:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177004.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177004.log.gz 2023-11-06T22:19:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169989.log 2023-11-06T22:19:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150511.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150511.log.gz 2023-11-06T22:19:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160782.log 2023-11-06T22:19:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165139.log 2023-11-06T22:19:48.937 INFO:teuthology.orchestra.run.smithi033.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.169989.log.gz 2023-11-06T22:19:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160782.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160782.log.gz 2023-11-06T22:19:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149828.log 2023-11-06T22:19:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165139.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165139.log.gz 2023-11-06T22:19:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124068.log 2023-11-06T22:19:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149828.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.149828.log.gz 2023-11-06T22:19:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160112.log 2023-11-06T22:19:48.945 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115246.log 2023-11-06T22:19:48.945 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.115246.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115246.log.gz 2023-11-06T22:19:48.945 INFO:teuthology.orchestra.run.smithi069.stderr: 94.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-06T22:19:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124068.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124068.log.gz 2023-11-06T22:19:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141499.log 2023-11-06T22:19:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160112.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160112.log.gz 2023-11-06T22:19:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152439.log 2023-11-06T22:19:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141499.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.141499.log.gz 2023-11-06T22:19:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135188.log 2023-11-06T22:19:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152439.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152439.log.gz 2023-11-06T22:19:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123382.log 2023-11-06T22:19:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170690.log 2023-11-06T22:19:48.957 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135188.log.gz 2023-11-06T22:19:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123382.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123382.log.gz 2023-11-06T22:19:48.958 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174562.log 2023-11-06T22:19:48.958 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.170690.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.170690.log.gz 2023-11-06T22:19:48.958 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119146.log 2023-11-06T22:19:48.958 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174562.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.174562.log.gz 2023-11-06T22:19:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175074.log 2023-11-06T22:19:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119146.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119146.log.gz 2023-11-06T22:19:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162797.log 2023-11-06T22:19:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175074.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.175074.log.gz 2023-11-06T22:19:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123186.log 2023-11-06T22:19:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162797.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162797.log.gz 2023-11-06T22:19:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150315.log 2023-11-06T22:19:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123186.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123186.log.gz 2023-11-06T22:19:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173088.log 2023-11-06T22:19:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150315.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150315.log.gz 2023-11-06T22:19:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119343.log 2023-11-06T22:19:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173088.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.173088.log.gz 2023-11-06T22:19:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173734.log 2023-11-06T22:19:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119343.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119343.log.gz 2023-11-06T22:19:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127229.log 2023-11-06T22:19:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173734.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.173734.log.gz 2023-11-06T22:19:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150609.log 2023-11-06T22:19:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127229.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127229.log.gz 2023-11-06T22:19:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129695.log 2023-11-06T22:19:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150609.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150609.log.gz 2023-11-06T22:19:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173970.log 2023-11-06T22:19:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129695.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129695.log.gz 2023-11-06T22:19:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139650.log 2023-11-06T22:19:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173970.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173970.log.gz 2023-11-06T22:19:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161793.log 2023-11-06T22:19:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139650.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139650.log.gz 2023-11-06T22:19:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146029.log 2023-11-06T22:19:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161793.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161793.log.gz 2023-11-06T22:19:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165286.log 2023-11-06T22:19:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146029.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146029.log.gz 2023-11-06T22:19:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140799.log 2023-11-06T22:19:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165286.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165286.log.gz 2023-11-06T22:19:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163452.log 2023-11-06T22:19:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140799.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140799.log.gz 2023-11-06T22:19:48.966 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145027.log 2023-11-06T22:19:48.966 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163452.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.163452.log.gz 2023-11-06T22:19:48.966 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154473.log 2023-11-06T22:19:48.971 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145027.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.145027.log.gz 2023-11-06T22:19:48.971 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147910.log 2023-11-06T22:19:48.971 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154473.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.154473.log.gz 2023-11-06T22:19:48.972 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177993.log 2023-11-06T22:19:48.972 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147910.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.147910.log.gz 2023-11-06T22:19:48.972 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177788.log 2023-11-06T22:19:48.972 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177993.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.177993.log.gz 2023-11-06T22:19:48.973 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123578.log 2023-11-06T22:19:48.973 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177788.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177788.log.gz 2023-11-06T22:19:48.973 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176359.log 2023-11-06T22:19:48.973 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123578.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.123578.log.gz 2023-11-06T22:19:48.973 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150364.log 2023-11-06T22:19:48.974 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176359.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.176359.log.gz 2023-11-06T22:19:48.974 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144905.log 2023-11-06T22:19:48.974 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150364.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150364.log.gz 2023-11-06T22:19:48.974 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125114.log 2023-11-06T22:19:48.974 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144905.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.144905.log.gz 2023-11-06T22:19:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174415.log 2023-11-06T22:19:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125114.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.125114.log.gz 2023-11-06T22:19:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.121143.log 2023-11-06T22:19:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174415.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.174415.log.gz 2023-11-06T22:19:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124289.log 2023-11-06T22:19:48.976 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121143.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.121143.log.gz 2023-11-06T22:19:48.976 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177396.log 2023-11-06T22:19:48.976 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124289.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124289.log.gz 2023-11-06T22:19:48.976 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126034.log 2023-11-06T22:19:48.976 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177396.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.177396.log.gz 2023-11-06T22:19:48.977 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147032.log 2023-11-06T22:19:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126034.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126034.log.gz 2023-11-06T22:19:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139435.log 2023-11-06T22:19:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147032.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147032.log.gz 2023-11-06T22:19:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167995.log 2023-11-06T22:19:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139435.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.139435.log.gz 2023-11-06T22:19:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165335.log 2023-11-06T22:19:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.167995.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.167995.log.gz 2023-11-06T22:19:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125212.log 2023-11-06T22:19:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135829.log 2023-11-06T22:19:48.988 INFO:teuthology.orchestra.run.smithi033.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.165335.log.gz 2023-11-06T22:19:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125212.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.125212.log.gz 2023-11-06T22:19:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126945.log 2023-11-06T22:19:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135829.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135829.log.gz 2023-11-06T22:19:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165188.log 2023-11-06T22:19:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126945.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126945.log.gz 2023-11-06T22:19:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127517.log 2023-11-06T22:19:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165188.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.165188.log.gz 2023-11-06T22:19:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139599.log 2023-11-06T22:19:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127517.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127517.log.gz 2023-11-06T22:19:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160929.log 2023-11-06T22:19:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139599.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139599.log.gz 2023-11-06T22:19:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161251.log 2023-11-06T22:19:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160929.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160929.log.gz 2023-11-06T22:19:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121437.log 2023-11-06T22:19:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161251.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.161251.log.gz 2023-11-06T22:19:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119714.log 2023-11-06T22:19:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121437.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.121437.log.gz 2023-11-06T22:19:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.162040.log 2023-11-06T22:19:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119714.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119714.log.gz 2023-11-06T22:19:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149466.log 2023-11-06T22:19:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162040.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.162040.log.gz 2023-11-06T22:19:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148865.log 2023-11-06T22:19:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149466.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149466.log.gz 2023-11-06T22:19:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153724.log 2023-11-06T22:19:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148865.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.148865.log.gz 2023-11-06T22:19:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160061.log 2023-11-06T22:19:48.998 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153724.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153724.log.gz 2023-11-06T22:19:48.998 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148813.log 2023-11-06T22:19:48.998 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160061.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160061.log.gz 2023-11-06T22:19:48.998 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145773.log 2023-11-06T22:19:48.998 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129407.log 2023-11-06T22:19:48.999 INFO:teuthology.orchestra.run.smithi033.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.148813.log.gz 2023-11-06T22:19:48.999 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145773.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145773.log.gz 2023-11-06T22:19:48.999 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135682.log 2023-11-06T22:19:48.999 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129407.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.129407.log.gz 2023-11-06T22:19:48.999 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124019.log 2023-11-06T22:19:49.001 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135682.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135682.log.gz 2023-11-06T22:19:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120604.log 2023-11-06T22:19:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124019.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124019.log.gz 2023-11-06T22:19:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127854.log 2023-11-06T22:19:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120604.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120604.log.gz 2023-11-06T22:19:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148545.log 2023-11-06T22:19:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133885.log 2023-11-06T22:19:49.003 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127854.log.gz 2023-11-06T22:19:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148545.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148545.log.gz 2023-11-06T22:19:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177690.log 2023-11-06T22:19:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.133885.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.133885.log.gz 2023-11-06T22:19:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121045.log 2023-11-06T22:19:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177690.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177690.log.gz 2023-11-06T22:19:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173306.log 2023-11-06T22:19:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121045.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121045.log.gz 2023-11-06T22:19:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-06T22:19:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173306.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.173306.log.gz 2023-11-06T22:19:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139756.log 2023-11-06T22:19:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169646.log 2023-11-06T22:19:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139756.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139756.log.gz 2023-11-06T22:19:49.012 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135090.log 2023-11-06T22:19:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169646.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.169646.log.gz 2023-11-06T22:19:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168559.log 2023-11-06T22:19:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135090.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.135090.log.gz 2023-11-06T22:19:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131221.log 2023-11-06T22:19:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168559.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.168559.log.gz 2023-11-06T22:19:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123235.log 2023-11-06T22:19:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.131221.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131221.log.gz 2023-11-06T22:19:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169744.log 2023-11-06T22:19:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123235.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123235.log.gz 2023-11-06T22:19:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140897.log 2023-11-06T22:19:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169744.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.169744.log.gz 2023-11-06T22:19:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121486.log 2023-11-06T22:19:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140897.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140897.log.gz 2023-11-06T22:19:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123529.log 2023-11-06T22:19:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121486.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.121486.log.gz 2023-11-06T22:19:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126279.log 2023-11-06T22:19:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123529.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123529.log.gz 2023-11-06T22:19:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135731.log 2023-11-06T22:19:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126279.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126279.log.gz 2023-11-06T22:19:49.022 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143998.log 2023-11-06T22:19:49.022 INFO:teuthology.orchestra.run.smithi069.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-06T22:19:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169191.log 2023-11-06T22:19:49.024 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135731.log.gz 2023-11-06T22:19:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143998.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-06T22:19:49.024 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143998.log.gz 2023-11-06T22:19:49.025 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169191.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.169191.log.gz 2023-11-06T22:19:49.025 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168306.log 2023-11-06T22:19:49.026 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174464.log 2023-11-06T22:19:49.026 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168306.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.168306.log.gz 2023-11-06T22:19:49.026 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173138.log 2023-11-06T22:19:49.027 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174464.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174464.log.gz 2023-11-06T22:19:49.027 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143338.log 2023-11-06T22:19:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173138.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.173138.log.gz 2023-11-06T22:19:49.028 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144394.log 2023-11-06T22:19:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143338.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143338.log.gz 2023-11-06T22:19:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161636.log 2023-11-06T22:19:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144394.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.144394.log.gz 2023-11-06T22:19:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167883.log 2023-11-06T22:19:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161636.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161636.log.gz 2023-11-06T22:19:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135286.log 2023-11-06T22:19:49.033 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.167883.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.167883.log.gz 2023-11-06T22:19:49.033 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123823.log 2023-11-06T22:19:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135286.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135286.log.gz 2023-11-06T22:19:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124632.log 2023-11-06T22:19:49.035 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123823.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123823.log.gz 2023-11-06T22:19:49.035 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169004.log 2023-11-06T22:19:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124632.log: 87.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.124632.log.gz--verbose 2023-11-06T22:19:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:-- /var/log/ceph/ceph-client.admin.142818.log 2023-11-06T22:19:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169004.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169004.log.gz 2023-11-06T22:19:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140037.log 2023-11-06T22:19:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142818.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.142818.log.gz 2023-11-06T22:19:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150158.log 2023-11-06T22:19:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140037.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140037.log.gz 2023-11-06T22:19:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129767.log 2023-11-06T22:19:49.041 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150158.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150158.log.gz 2023-11-06T22:19:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158500.log 2023-11-06T22:19:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129767.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.129767.log.gz 2023-11-06T22:19:49.043 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129015.log 2023-11-06T22:19:49.043 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.158500.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.158500.log.gz 2023-11-06T22:19:49.044 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126385.log 2023-11-06T22:19:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129015.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129015.log.gz 2023-11-06T22:19:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163401.log 2023-11-06T22:19:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126385.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.126385.log.gz 2023-11-06T22:19:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129479.log 2023-11-06T22:19:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163401.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163401.log.gz 2023-11-06T22:19:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168657.log 2023-11-06T22:19:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129983.log 2023-11-06T22:19:49.049 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129479.log.gz 2023-11-06T22:19:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168657.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.168657.log.gz 2023-11-06T22:19:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124240.log 2023-11-06T22:19:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129983.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.129983.log.gz 2023-11-06T22:19:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133334.log 2023-11-06T22:19:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124240.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124240.log.gz 2023-11-06T22:19:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176604.log 2023-11-06T22:19:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.133334.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133334.log.gz 2023-11-06T22:19:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119861.log 2023-11-06T22:19:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176604.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.176604.log.gz 2023-11-06T22:19:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175781.log 2023-11-06T22:19:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119861.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119861.log.gz 2023-11-06T22:19:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128246.log 2023-11-06T22:19:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175781.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175781.log.gz 2023-11-06T22:19:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.147804.log 2023-11-06T22:19:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.128246.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.128246.log.gz 2023-11-06T22:19:49.062 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134153.log 2023-11-06T22:19:49.062 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.147804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.147804.log.gz 2023-11-06T22:19:49.062 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146983.log 2023-11-06T22:19:49.063 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134153.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134153.log.gz 2023-11-06T22:19:49.063 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149779.log 2023-11-06T22:19:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146983.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.146983.log.gz 2023-11-06T22:19:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163558.log 2023-11-06T22:19:49.068 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149779.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.149779.log.gz 2023-11-06T22:19:49.068 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119616.log 2023-11-06T22:19:49.068 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163558.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163558.log.gz 2023-11-06T22:19:49.068 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153256.log 2023-11-06T22:19:49.069 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119616.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119616.log.gz 2023-11-06T22:19:49.069 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141156.log 2023-11-06T22:19:49.069 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153256.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.153256.log.gz 2023-11-06T22:19:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120408.log 2023-11-06T22:19:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141156.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141156.log.gz 2023-11-06T22:19:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151072.log 2023-11-06T22:19:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120408.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120408.log.gz 2023-11-06T22:19:49.072 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177102.log 2023-11-06T22:19:49.073 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.151072.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151072.log.gz 2023-11-06T22:19:49.073 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142974.log 2023-11-06T22:19:49.073 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177102.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177102.log.gz 2023-11-06T22:19:49.073 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146636.log 2023-11-06T22:19:49.075 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142974.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.142974.log.gz 2023-11-06T22:19:49.075 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175295.log 2023-11-06T22:19:49.075 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146636.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.146636.log.gz 2023-11-06T22:19:49.076 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154277.log 2023-11-06T22:19:49.076 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175295.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.175295.log.gz 2023-11-06T22:19:49.077 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160732.log 2023-11-06T22:19:49.077 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154277.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.154277.log.gz 2023-11-06T22:19:49.078 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174019.log 2023-11-06T22:19:49.078 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160732.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.160732.log.gz 2023-11-06T22:19:49.079 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147497.log 2023-11-06T22:19:49.079 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174019.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.174019.log.gz 2023-11-06T22:19:49.080 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161487.log 2023-11-06T22:19:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147497.log: gzip -5 90.9%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.147497.log.gz/var/log/ceph/ceph-client.admin.147545.log 2023-11-06T22:19:49.081 INFO:teuthology.orchestra.run.smithi033.stderr: 2023-11-06T22:19:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161487.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.161487.log.gz 2023-11-06T22:19:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125506.log 2023-11-06T22:19:49.083 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147545.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147545.log.gz 2023-11-06T22:19:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165723.log 2023-11-06T22:19:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125506.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125506.log.gz 2023-11-06T22:19:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173823.log 2023-11-06T22:19:49.086 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165723.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165723.log.gz 2023-11-06T22:19:49.086 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123872.log 2023-11-06T22:19:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173823.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.173823.log.gz 2023-11-06T22:19:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149022.log 2023-11-06T22:19:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123872.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123872.log.gz 2023-11-06T22:19:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134481.log 2023-11-06T22:19:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149022.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149022.log.gz 2023-11-06T22:19:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161536.log 2023-11-06T22:19:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134481.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134481.log.gz 2023-11-06T22:19:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135335.log 2023-11-06T22:19:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161536.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161536.log.gz 2023-11-06T22:19:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143847.log 2023-11-06T22:19:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135335.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135335.log.gz 2023-11-06T22:19:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176308.log 2023-11-06T22:19:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143847.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143847.log.gz 2023-11-06T22:19:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.126839.log 2023-11-06T22:19:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176308.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.176308.log.gz 2023-11-06T22:19:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142715.log 2023-11-06T22:19:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.126839.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.126839.log.gz 2023-11-06T22:19:49.098 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-06T22:19:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142715.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142715.log.gz 2023-11-06T22:19:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149270.log 2023-11-06T22:19:49.100 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164745.log 2023-11-06T22:19:49.102 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149270.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.149270.log.gz 2023-11-06T22:19:49.102 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143387.log 2023-11-06T22:19:49.103 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164745.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164745.log.gz 2023-11-06T22:19:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141254.log 2023-11-06T22:19:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143387.log: 90.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-06T22:19:49.107 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143387.log.gz 2023-11-06T22:19:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175833.log 2023-11-06T22:19:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141254.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141254.log.gz 2023-11-06T22:19:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125025.log 2023-11-06T22:19:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175833.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.175833.log.gz 2023-11-06T22:19:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125408.log 2023-11-06T22:19:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125025.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.125025.log.gz 2023-11-06T22:19:49.110 INFO:teuthology.orchestra.run.smithi033.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-06T22:19:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125985.log 2023-11-06T22:19:49.111 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125408.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125408.log.gz 2023-11-06T22:19:49.111 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164352.log 2023-11-06T22:19:49.112 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125985.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125985.log.gz 2023-11-06T22:19:49.113 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139912.log 2023-11-06T22:19:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164352.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.164352.log.gz 2023-11-06T22:19:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149730.log 2023-11-06T22:19:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139912.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139912.log.gz 2023-11-06T22:19:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154935.log 2023-11-06T22:19:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149730.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149730.log.gz 2023-11-06T22:19:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177347.log 2023-11-06T22:19:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154935.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154935.log.gz 2023-11-06T22:19:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144589.log 2023-11-06T22:19:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177347.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.177347.log.gz 2023-11-06T22:19:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146832.log 2023-11-06T22:19:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144589.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.144589.log.gz 2023-11-06T22:19:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146734.log 2023-11-06T22:19:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146832.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146832.log.gz 2023-11-06T22:19:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160223.log 2023-11-06T22:19:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146734.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.146734.log.gz 2023-11-06T22:19:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175483.log 2023-11-06T22:19:49.124 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160223.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160223.log.gz 2023-11-06T22:19:49.124 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163981.log 2023-11-06T22:19:49.125 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175483.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.175483.log.gz 2023-11-06T22:19:49.126 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173559.log 2023-11-06T22:19:49.126 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163981.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163981.log.gz 2023-11-06T22:19:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123431.log 2023-11-06T22:19:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173559.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.173559.log.gz 2023-11-06T22:19:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168151.log 2023-11-06T22:19:49.129 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123431.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123431.log.gz 2023-11-06T22:19:49.130 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140086.log 2023-11-06T22:19:49.130 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168151.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.168151.log.gz 2023-11-06T22:19:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170402.log 2023-11-06T22:19:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140086.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.140086.log.gz 2023-11-06T22:19:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175385.log 2023-11-06T22:19:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.170402.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170402.log.gz 2023-11-06T22:19:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164253.log 2023-11-06T22:19:49.134 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175385.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.175385.log.gz 2023-11-06T22:19:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144296.log 2023-11-06T22:19:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164253.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164253.log.gz 2023-11-06T22:19:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144492.log 2023-11-06T22:19:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144296.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144296.log.gz 2023-11-06T22:19:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134531.log 2023-11-06T22:19:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144492.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.144492.log.gz 2023-11-06T22:19:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152119.log 2023-11-06T22:19:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134531.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.134531.log.gz 2023-11-06T22:19:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140848.log 2023-11-06T22:19:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152119.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152119.log.gz 2023-11-06T22:19:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167097.log 2023-11-06T22:19:49.141 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136659.log 2023-11-06T22:19:49.142 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140848.log.gz 2023-11-06T22:19:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.167097.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.167097.log.gz 2023-11-06T22:19:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160549.log 2023-11-06T22:19:49.145 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.136659.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136659.log.gz 2023-11-06T22:19:49.145 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151022.log 2023-11-06T22:19:49.147 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160549.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.160549.log.gz 2023-11-06T22:19:49.147 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119294.log 2023-11-06T22:19:49.147 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.151022.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.151022.log.gz 2023-11-06T22:19:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145929.log 2023-11-06T22:19:49.149 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119294.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119294.log.gz 2023-11-06T22:19:49.149 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154571.log 2023-11-06T22:19:49.149 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145929.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145929.log.gz 2023-11-06T22:19:49.150 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134205.log 2023-11-06T22:19:49.151 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161942.log 2023-11-06T22:19:49.151 INFO:teuthology.orchestra.run.smithi033.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.154571.log.gz 2023-11-06T22:19:49.151 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134205.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134205.log.gz 2023-11-06T22:19:49.152 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120947.log 2023-11-06T22:19:49.152 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161942.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161942.log.gz 2023-11-06T22:19:49.153 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129551.log 2023-11-06T22:19:49.153 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146225.log 2023-11-06T22:19:49.154 INFO:teuthology.orchestra.run.smithi033.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120947.log.gz 2023-11-06T22:19:49.154 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129551.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.129551.log.gz 2023-11-06T22:19:49.155 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150101.log 2023-11-06T22:19:49.155 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146225.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146225.log.gz 2023-11-06T22:19:49.155 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141205.log 2023-11-06T22:19:49.156 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150101.log: gzip 87.8%-5 -- replaced with /var/log/ceph/ceph-client.admin.150101.log.gz--verbose 2023-11-06T22:19:49.156 INFO:teuthology.orchestra.run.smithi033.stderr:-- /var/log/ceph/ceph-client.admin.136708.log 2023-11-06T22:19:49.157 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141205.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.141205.log.gz 2023-11-06T22:19:49.157 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123676.log 2023-11-06T22:19:49.158 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.136708.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.136708.log.gz 2023-11-06T22:19:49.158 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173453.log 2023-11-06T22:19:49.159 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123676.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123676.log.gz 2023-11-06T22:19:49.160 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119910.log 2023-11-06T22:19:49.160 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173453.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.173453.log.gz 2023-11-06T22:19:49.161 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155057.log 2023-11-06T22:19:49.161 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119910.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119910.log.gz 2023-11-06T22:19:49.162 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142924.log 2023-11-06T22:19:49.162 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.155057.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155057.log.gz 2023-11-06T22:19:49.163 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120506.log 2023-11-06T22:19:49.163 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142924.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.142924.log.gz 2023-11-06T22:19:49.164 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165237.log 2023-11-06T22:19:49.164 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120506.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120506.log.gz 2023-11-06T22:19:49.165 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174366.log 2023-11-06T22:19:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153822.log 2023-11-06T22:19:49.166 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.165237.log.gz 2023-11-06T22:19:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174366.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174366.log.gz 2023-11-06T22:19:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168706.log 2023-11-06T22:19:49.168 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153822.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153822.log.gz 2023-11-06T22:19:49.168 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143289.log 2023-11-06T22:19:49.169 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168706.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.168706.log.gz 2023-11-06T22:19:49.169 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149417.log 2023-11-06T22:19:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143289.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143289.log.gz 2023-11-06T22:19:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177837.log 2023-11-06T22:19:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149417.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149417.log.gz 2023-11-06T22:19:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135584.log 2023-11-06T22:19:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177837.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.177837.log.gz 2023-11-06T22:19:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149172.log 2023-11-06T22:19:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135584.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.135584.log.gz 2023-11-06T22:19:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160380.log 2023-11-06T22:19:49.175 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149172.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.149172.log.gz 2023-11-06T22:19:49.175 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163660.log 2023-11-06T22:19:49.176 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160380.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160380.log.gz 2023-11-06T22:19:49.176 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147448.log 2023-11-06T22:19:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163660.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.163660.log.gz 2023-11-06T22:19:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119959.log 2023-11-06T22:19:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147448.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.147448.log.gz 2023-11-06T22:19:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145878.log 2023-11-06T22:19:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119959.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119959.log.gz 2023-11-06T22:19:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168461.log 2023-11-06T22:19:49.180 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145878.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.145878.log.gz 2023-11-06T22:19:49.180 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-06T22:19:49.181 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168461.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168461.log.gz 2023-11-06T22:19:49.181 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172409.log 2023-11-06T22:19:49.182 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134776.log 2023-11-06T22:19:49.183 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172409.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172409.log.gz 2023-11-06T22:19:49.183 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135633.log 2023-11-06T22:19:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134776.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.134776.log.gz 2023-11-06T22:19:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127952.log 2023-11-06T22:19:49.185 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135633.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135633.log.gz 2023-11-06T22:19:49.185 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175730.log 2023-11-06T22:19:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127952.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127952.log.gz 2023-11-06T22:19:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174611.log 2023-11-06T22:19:49.188 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175730.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.175730.log.gz 2023-11-06T22:19:49.188 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153035.log 2023-11-06T22:19:49.189 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174611.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174611.log.gz 2023-11-06T22:19:49.189 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155222.log 2023-11-06T22:19:49.190 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153035.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153035.log.gz 2023-11-06T22:19:49.190 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140290.log 2023-11-06T22:19:49.191 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.155222.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.155222.log.gz 2023-11-06T22:19:49.191 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146498.log 2023-11-06T22:19:49.192 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120457.log 2023-11-06T22:19:49.193 INFO:teuthology.orchestra.run.smithi033.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.140290.log.gz 2023-11-06T22:19:49.194 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146498.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146498.log.gz 2023-11-06T22:19:49.194 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160275.log 2023-11-06T22:19:49.195 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120457.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120457.log.gz 2023-11-06T22:19:49.195 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143240.log 2023-11-06T22:19:49.196 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160275.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160275.log.gz 2023-11-06T22:19:49.196 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140184.log 2023-11-06T22:19:49.197 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143240.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143240.log.gz 2023-11-06T22:19:49.197 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145200.log 2023-11-06T22:19:49.198 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140184.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.140184.log.gz 2023-11-06T22:19:49.198 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120751.log 2023-11-06T22:19:49.199 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145200.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145200.log.gz 2023-11-06T22:19:49.200 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124877.log 2023-11-06T22:19:49.200 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120751.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120751.log.gz 2023-11-06T22:19:49.201 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.136307.log 2023-11-06T22:19:49.201 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124877.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124877.log.gz 2023-11-06T22:19:49.202 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141961.log 2023-11-06T22:19:49.203 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.136307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.136307.log.gz 2023-11-06T22:19:49.203 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119097.log 2023-11-06T22:19:49.204 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141961.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141961.log.gz 2023-11-06T22:19:49.205 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168828.log 2023-11-06T22:19:49.206 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119097.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119097.log.gz 2023-11-06T22:19:49.206 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118910.log 2023-11-06T22:19:49.207 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168828.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168828.log.gz 2023-11-06T22:19:49.208 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121241.log 2023-11-06T22:19:49.209 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.118910.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118910.log.gz 2023-11-06T22:19:49.209 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162960.log 2023-11-06T22:19:49.210 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121241.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121241.log.gz 2023-11-06T22:19:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-06T22:19:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162960.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.162960.log.gz 2023-11-06T22:19:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153971.log 2023-11-06T22:19:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150658.log 2023-11-06T22:19:49.213 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153971.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153971.log.gz 2023-11-06T22:19:49.214 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136364.log 2023-11-06T22:19:49.215 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177053.log 2023-11-06T22:19:49.216 INFO:teuthology.orchestra.run.smithi033.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.150658.log.gz 2023-11-06T22:19:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.136364.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136364.log.gz 2023-11-06T22:19:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173355.log 2023-11-06T22:19:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177053.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.177053.log.gz 2023-11-06T22:19:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147179.log 2023-11-06T22:19:49.220 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125885.log 2023-11-06T22:19:49.220 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.173355.log.gz 2023-11-06T22:19:49.221 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147179.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147179.log.gz 2023-11-06T22:19:49.222 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177494.log 2023-11-06T22:19:49.222 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125885.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125885.log.gz 2023-11-06T22:19:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145670.log 2023-11-06T22:19:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163347.log 2023-11-06T22:19:49.225 INFO:teuthology.orchestra.run.smithi033.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.177494.log.gz 2023-11-06T22:19:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145670.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145670.log.gz 2023-11-06T22:19:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149641.log 2023-11-06T22:19:49.227 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133683.log 2023-11-06T22:19:49.228 INFO:teuthology.orchestra.run.smithi033.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.163347.log.gz 2023-11-06T22:19:49.229 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149641.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.149641.log.gz 2023-11-06T22:19:49.230 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145980.log 2023-11-06T22:19:49.230 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.133683.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.133683.log.gz 2023-11-06T22:19:49.231 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177298.log 2023-11-06T22:19:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145980.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145980.log.gz 2023-11-06T22:19:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136000.log 2023-11-06T22:19:49.233 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177298.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177298.log.gz 2023-11-06T22:19:49.233 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161152.log 2023-11-06T22:19:49.235 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.136000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142221.log 2023-11-06T22:19:49.235 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161152.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.136000.log.gz 2023-11-06T22:19:49.236 INFO:teuthology.orchestra.run.smithi033.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.161152.log.gz 2023-11-06T22:19:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169595.log 2023-11-06T22:19:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142221.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142221.log.gz 2023-11-06T22:19:49.238 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148035.log 2023-11-06T22:19:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118861.log 2023-11-06T22:19:49.239 INFO:teuthology.orchestra.run.smithi033.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.169595.log.gz 2023-11-06T22:19:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148035.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148035.log.gz 2023-11-06T22:19:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121388.log 2023-11-06T22:19:49.242 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.118861.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118861.log.gz 2023-11-06T22:19:49.242 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163764.log 2023-11-06T22:19:49.244 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169546.log 2023-11-06T22:19:49.244 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.121388.log.gz 2023-11-06T22:19:49.245 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163764.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163764.log.gz 2023-11-06T22:19:49.245 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142558.log 2023-11-06T22:19:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169546.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.169546.log.gz 2023-11-06T22:19:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134100.log 2023-11-06T22:19:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142558.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.142558.log.gz 2023-11-06T22:19:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147326.log 2023-11-06T22:19:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134100.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.134100.log.gz 2023-11-06T22:19:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140233.log 2023-11-06T22:19:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147326.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147326.log.gz 2023-11-06T22:19:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142109.log 2023-11-06T22:19:49.253 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140233.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.140233.log.gz 2023-11-06T22:19:49.253 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122442.log 2023-11-06T22:19:49.255 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152759.log 2023-11-06T22:19:49.255 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142109.log.gz 2023-11-06T22:19:49.256 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.122442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.122442.log.gz 2023-11-06T22:19:49.256 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145566.log 2023-11-06T22:19:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152759.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152759.log.gz 2023-11-06T22:19:49.258 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121339.log 2023-11-06T22:19:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120653.log 2023-11-06T22:19:49.260 INFO:teuthology.orchestra.run.smithi033.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.145566.log.gz 2023-11-06T22:19:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121339.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.121339.log.gz 2023-11-06T22:19:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143896.log 2023-11-06T22:19:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120653.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120653.log.gz 2023-11-06T22:19:49.262 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134629.log 2023-11-06T22:19:49.263 INFO:teuthology.orchestra.run.smithi069.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-06T22:19:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.144848.log 2023-11-06T22:19:49.264 INFO:teuthology.orchestra.run.smithi033.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.143896.log.gz 2023-11-06T22:19:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134629.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134629.log.gz 2023-11-06T22:19:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153207.log 2023-11-06T22:19:49.266 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.144848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.144848.log.gz 2023-11-06T22:19:49.267 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121682.log 2023-11-06T22:19:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153207.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.153207.log.gz 2023-11-06T22:19:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168047.log 2023-11-06T22:19:49.269 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121682.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121682.log.gz 2023-11-06T22:19:49.270 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134412.log 2023-11-06T22:19:49.271 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167832.log 2023-11-06T22:19:49.272 INFO:teuthology.orchestra.run.smithi033.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.168047.log.gz 2023-11-06T22:19:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134412.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.134412.log.gz 2023-11-06T22:19:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140563.log 2023-11-06T22:19:49.274 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.167832.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.167832.log.gz 2023-11-06T22:19:49.274 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140388.log 2023-11-06T22:19:49.275 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134580.log 2023-11-06T22:19:49.276 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.140563.log.gz 2023-11-06T22:19:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140388.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140388.log.gz 2023-11-06T22:19:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176204.log 2023-11-06T22:19:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134580.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134580.log.gz 2023-11-06T22:19:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124730.log 2023-11-06T22:19:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176204.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.176204.log.gz 2023-11-06T22:19:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160831.log 2023-11-06T22:19:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124730.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124730.log.gz 2023-11-06T22:19:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154022.log 2023-11-06T22:19:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166766.log 2023-11-06T22:19:49.284 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160831.log.gz 2023-11-06T22:19:49.284 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154022.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154022.log.gz 2023-11-06T22:19:49.285 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178040.log 2023-11-06T22:19:49.286 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.166766.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.166766.log.gz 2023-11-06T22:19:49.286 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143142.log 2023-11-06T22:19:49.287 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.178040.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.178040.log.gz 2023-11-06T22:19:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141107.log 2023-11-06T22:19:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143142.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.143142.log.gz 2023-11-06T22:19:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139703.log 2023-11-06T22:19:49.291 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141107.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141107.log.gz 2023-11-06T22:19:49.291 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168360.log 2023-11-06T22:19:49.292 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139703.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139703.log.gz 2023-11-06T22:19:49.293 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135001.log 2023-11-06T22:19:49.294 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168360.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.168360.log.gz 2023-11-06T22:19:49.295 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121535.log 2023-11-06T22:19:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135001.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135001.log.gz 2023-11-06T22:19:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164079.log 2023-11-06T22:19:49.297 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121535.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121535.log.gz 2023-11-06T22:19:49.298 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176152.log 2023-11-06T22:19:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164079.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164079.log.gz 2023-11-06T22:19:49.299 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165944.log 2023-11-06T22:19:49.300 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176152.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176152.log.gz 2023-11-06T22:19:49.301 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152966.log 2023-11-06T22:19:49.302 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143651.log 2023-11-06T22:19:49.303 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.165944.log.gz 2023-11-06T22:19:49.303 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152966.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.152966.log.gz 2023-11-06T22:19:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147228.log 2023-11-06T22:19:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143651.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143651.log.gz 2023-11-06T22:19:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158549.log 2023-11-06T22:19:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147228.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.147228.log.gz 2023-11-06T22:19:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149368.log 2023-11-06T22:19:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.158549.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.158549.log.gz 2023-11-06T22:19:49.309 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135486.log 2023-11-06T22:19:49.310 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149368.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149368.log.gz 2023-11-06T22:19:49.310 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126083.log 2023-11-06T22:19:49.311 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135486.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.135486.log.gz 2023-11-06T22:19:49.312 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129113.log 2023-11-06T22:19:49.313 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126083.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.126083.log.gz 2023-11-06T22:19:49.314 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148244.log 2023-11-06T22:19:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129113.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129113.log.gz 2023-11-06T22:19:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120849.log 2023-11-06T22:19:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148244.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.148244.log.gz 2023-11-06T22:19:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146078.log 2023-11-06T22:19:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120849.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120849.log.gz 2023-11-06T22:19:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126328.log 2023-11-06T22:19:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146078.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146078.log.gz 2023-11-06T22:19:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130127.log 2023-11-06T22:19:49.321 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126328.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.126328.log.gz 2023-11-06T22:19:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154619.log 2023-11-06T22:19:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.130127.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130127.log.gz 2023-11-06T22:19:49.324 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115519.log 2023-11-06T22:19:49.324 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154619.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154619.log.gz 2023-11-06T22:19:49.325 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150050.log 2023-11-06T22:19:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.115519.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115519.log.gz 2023-11-06T22:19:49.327 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120057.log 2023-11-06T22:19:49.328 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150050.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150050.log.gz 2023-11-06T22:19:49.328 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174268.log 2023-11-06T22:19:49.329 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120057.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120057.log.gz 2023-11-06T22:19:49.330 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161744.log 2023-11-06T22:19:49.331 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174268.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.174268.log.gz 2023-11-06T22:19:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144954.log 2023-11-06T22:19:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161744.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161744.log.gz 2023-11-06T22:19:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165845.log 2023-11-06T22:19:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144954.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144954.log.gz 2023-11-06T22:19:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152861.log 2023-11-06T22:19:49.336 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165845.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165845.log.gz 2023-11-06T22:19:49.336 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139987.log 2023-11-06T22:19:49.337 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152861.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152861.log.gz 2023-11-06T22:19:49.338 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120800.log 2023-11-06T22:19:49.339 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139987.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.139987.log.gz 2023-11-06T22:19:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169289.log 2023-11-06T22:19:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120800.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120800.log.gz 2023-11-06T22:19:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177151.log 2023-11-06T22:19:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169289.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169289.log.gz 2023-11-06T22:19:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153586.log 2023-11-06T22:19:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177151.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177151.log.gz 2023-11-06T22:19:49.345 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134727.log 2023-11-06T22:19:49.346 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153586.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.153586.log.gz 2023-11-06T22:19:49.346 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162236.log 2023-11-06T22:19:49.347 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134727.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134727.log.gz 2023-11-06T22:19:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168098.log 2023-11-06T22:19:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162236.log: 90.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.162236.log.gz-5 2023-11-06T22:19:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:--verbose -- /var/log/ceph/ceph-client.admin.145454.log 2023-11-06T22:19:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168098.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.168098.log.gz 2023-11-06T22:19:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176408.log 2023-11-06T22:19:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145454.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.145454.log.gz 2023-11-06T22:19:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168254.log 2023-11-06T22:19:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176408.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.176408.log.gz 2023-11-06T22:19:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121192.log 2023-11-06T22:19:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168254.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.168254.log.gz 2023-11-06T22:19:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132986.log 2023-11-06T22:19:49.357 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121192.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121192.log.gz 2023-11-06T22:19:49.358 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127018.log 2023-11-06T22:19:49.359 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.132986.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.132986.log.gz 2023-11-06T22:19:49.359 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163833.log 2023-11-06T22:19:49.360 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127018.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.127018.log.gz 2023-11-06T22:19:49.361 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141596.log 2023-11-06T22:19:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163833.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163833.log.gz 2023-11-06T22:19:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119245.log 2023-11-06T22:19:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141596.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141596.log.gz 2023-11-06T22:19:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177249.log 2023-11-06T22:19:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153920.log 2023-11-06T22:19:49.366 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119245.log.gz 2023-11-06T22:19:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177249.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177249.log.gz 2023-11-06T22:19:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143947.log 2023-11-06T22:19:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153920.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153920.log.gz 2023-11-06T22:19:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.122956.log 2023-11-06T22:19:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143947.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143947.log.gz 2023-11-06T22:19:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148494.log 2023-11-06T22:19:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.2.122956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.122956.log.gz 2023-11-06T22:19:49.372 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161389.log 2023-11-06T22:19:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148494.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.148494.log.gz 2023-11-06T22:19:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123086.log 2023-11-06T22:19:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161389.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.161389.log.gz 2023-11-06T22:19:49.376 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125653.log 2023-11-06T22:19:49.376 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123086.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123086.log.gz 2023-11-06T22:19:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177200.log 2023-11-06T22:19:49.378 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125653.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125653.log.gz 2023-11-06T22:19:49.379 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164441.log 2023-11-06T22:19:49.380 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177200.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.177200.log.gz 2023-11-06T22:19:49.380 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154326.log 2023-11-06T22:19:49.381 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164441.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.164441.log.gz 2023-11-06T22:19:49.382 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166033.log 2023-11-06T22:19:49.383 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154326.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154326.log.gz 2023-11-06T22:19:49.383 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163031.log 2023-11-06T22:19:49.384 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.166033.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.166033.log.gz 2023-11-06T22:19:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147130.log 2023-11-06T22:19:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163031.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163031.log.gz 2023-11-06T22:19:49.387 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120555.log 2023-11-06T22:19:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147130.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.147130.log.gz 2023-11-06T22:19:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161687.log 2023-11-06T22:19:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120555.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120555.log.gz 2023-11-06T22:19:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135927.log 2023-11-06T22:19:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161687.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.161687.log.gz 2023-11-06T22:19:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142870.log 2023-11-06T22:19:49.392 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135927.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135927.log.gz 2023-11-06T22:19:49.393 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134049.log 2023-11-06T22:19:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142870.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142870.log.gz 2023-11-06T22:19:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.150965.log 2023-11-06T22:19:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134049.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.134049.log.gz 2023-11-06T22:19:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126896.log 2023-11-06T22:19:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.150965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.150965.log.gz 2023-11-06T22:19:49.398 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160612.log 2023-11-06T22:19:49.399 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126896.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.126896.log.gz 2023-11-06T22:19:49.400 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163505.log 2023-11-06T22:19:49.401 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160612.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.160612.log.gz 2023-11-06T22:19:49.401 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167935.log 2023-11-06T22:19:49.402 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163505.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163505.log.gz 2023-11-06T22:19:49.403 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.162543.log 2023-11-06T22:19:49.404 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.167935.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.167935.log.gz 2023-11-06T22:19:49.405 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120261.log 2023-11-06T22:19:49.405 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.162543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.162543.log.gz 2023-11-06T22:19:49.406 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172291.log 2023-11-06T22:19:49.407 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120261.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120261.log.gz 2023-11-06T22:19:49.407 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135386.log 2023-11-06T22:19:49.409 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172291.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.172291.log.gz 2023-11-06T22:19:49.409 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176906.log 2023-11-06T22:19:49.410 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135386.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135386.log.gz 2023-11-06T22:19:49.411 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161893.log 2023-11-06T22:19:49.412 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176906.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176906.log.gz 2023-11-06T22:19:49.412 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144096.log 2023-11-06T22:19:49.413 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161893.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161893.log.gz 2023-11-06T22:19:49.414 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151145.log 2023-11-06T22:19:49.415 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144096.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144096.log.gz 2023-11-06T22:19:49.416 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172931.log 2023-11-06T22:19:49.417 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.151145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128148.log 2023-11-06T22:19:49.417 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151145.log.gz 2023-11-06T22:19:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172931.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.172931.log.gz 2023-11-06T22:19:49.419 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135437.log 2023-11-06T22:19:49.420 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.128148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165383.log 2023-11-06T22:19:49.420 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128148.log.gz 2023-11-06T22:19:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135437.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.135437.log.gz 2023-11-06T22:19:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150207.log 2023-11-06T22:19:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165383.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.165383.log.gz 2023-11-06T22:19:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176457.log 2023-11-06T22:19:49.424 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150207.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150207.log.gz 2023-11-06T22:19:49.425 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151220.log 2023-11-06T22:19:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176457.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.176457.log.gz 2023-11-06T22:19:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130176.log 2023-11-06T22:19:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.151220.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151220.log.gz 2023-11-06T22:19:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162089.log 2023-11-06T22:19:49.429 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.130176.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.130176.log.gz 2023-11-06T22:19:49.430 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176857.log 2023-11-06T22:19:49.431 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162089.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162089.log.gz 2023-11-06T22:19:49.431 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149926.log 2023-11-06T22:19:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176857.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176857.log.gz 2023-11-06T22:19:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123921.log 2023-11-06T22:19:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149926.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.149926.log.gz 2023-11-06T22:19:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140339.log 2023-11-06T22:19:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123921.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123921.log.gz 2023-11-06T22:19:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163182.log 2023-11-06T22:19:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140339.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.140339.log.gz 2023-11-06T22:19:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124828.log 2023-11-06T22:19:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163182.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.163182.log.gz 2023-11-06T22:19:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119665.log 2023-11-06T22:19:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124828.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124828.log.gz 2023-11-06T22:19:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168203.log 2023-11-06T22:19:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119665.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119665.log.gz 2023-11-06T22:19:49.443 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176555.log 2023-11-06T22:19:49.444 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168203.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.168203.log.gz 2023-11-06T22:19:49.444 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162649.log 2023-11-06T22:19:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176555.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176555.log.gz 2023-11-06T22:19:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173510.log 2023-11-06T22:19:49.447 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162649.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162649.log.gz 2023-11-06T22:19:49.447 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148657.log 2023-11-06T22:19:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173510.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.173510.log.gz 2023-11-06T22:19:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170353.log 2023-11-06T22:19:49.450 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148657.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.148657.log.gz 2023-11-06T22:19:49.450 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120008.log 2023-11-06T22:19:49.452 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.170353.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.170353.log.gz 2023-11-06T22:19:49.452 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146685.log 2023-11-06T22:19:49.453 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120008.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120008.log.gz 2023-11-06T22:19:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123970.log 2023-11-06T22:19:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146685.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.146685.log.gz 2023-11-06T22:19:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141352.log 2023-11-06T22:19:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123970.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123970.log.gz 2023-11-06T22:19:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148709.log 2023-11-06T22:19:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121290.log 2023-11-06T22:19:49.459 INFO:teuthology.orchestra.run.smithi033.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.141352.log.gz 2023-11-06T22:19:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148709.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.148709.log.gz 2023-11-06T22:19:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174757.log 2023-11-06T22:19:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121290.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.121290.log.gz 2023-11-06T22:19:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128966.log 2023-11-06T22:19:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174757.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.174757.log.gz 2023-11-06T22:19:49.463 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154984.log 2023-11-06T22:19:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.128966.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.128966.log.gz 2023-11-06T22:19:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149123.log 2023-11-06T22:19:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154984.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154984.log.gz 2023-11-06T22:19:49.467 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144443.log 2023-11-06T22:19:49.468 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149123.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.149123.log.gz 2023-11-06T22:19:49.469 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141450.log 2023-11-06T22:19:49.469 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144443.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.144443.log.gz 2023-11-06T22:19:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120114.log 2023-11-06T22:19:49.471 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141450.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.141450.log.gz 2023-11-06T22:19:49.472 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127805.log 2023-11-06T22:19:49.473 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120114.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120114.log.gz 2023-11-06T22:19:49.473 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125555.log 2023-11-06T22:19:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127805.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.127805.log.gz 2023-11-06T22:19:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168510.log 2023-11-06T22:19:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125555.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.125555.log.gz 2023-11-06T22:19:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145617.log 2023-11-06T22:19:49.478 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168510.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.168510.log.gz 2023-11-06T22:19:49.478 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.165642.log 2023-11-06T22:19:49.479 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145617.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.145617.log.gz 2023-11-06T22:19:49.480 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173404.log 2023-11-06T22:19:49.480 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.165642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.165642.log.gz 2023-11-06T22:19:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142034.log 2023-11-06T22:19:49.482 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173404.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.173404.log.gz 2023-11-06T22:19:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121731.log 2023-11-06T22:19:49.484 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127373.log 2023-11-06T22:19:49.484 INFO:teuthology.orchestra.run.smithi033.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.142034.log.gz 2023-11-06T22:19:49.485 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121731.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121731.log.gz 2023-11-06T22:19:49.486 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120310.log 2023-11-06T22:19:49.487 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127373.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127373.log.gz 2023-11-06T22:19:49.488 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147081.log 2023-11-06T22:19:49.489 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120310.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120310.log.gz 2023-11-06T22:19:49.489 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124436.log 2023-11-06T22:19:49.490 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147081.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147081.log.gz 2023-11-06T22:19:49.491 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131569.log 2023-11-06T22:19:49.492 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124436.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124436.log.gz 2023-11-06T22:19:49.492 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163883.log 2023-11-06T22:19:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.131569.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131569.log.gz 2023-11-06T22:19:49.494 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124779.log 2023-11-06T22:19:49.495 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172722.log 2023-11-06T22:19:49.496 INFO:teuthology.orchestra.run.smithi033.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.163883.log.gz 2023-11-06T22:19:49.496 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124779.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.124779.log.gz 2023-11-06T22:19:49.497 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.141855.log 2023-11-06T22:19:49.498 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172722.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.172722.log.gz 2023-11-06T22:19:49.498 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169497.log 2023-11-06T22:19:49.499 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.141855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.141855.log.gz 2023-11-06T22:19:49.500 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142499.log 2023-11-06T22:19:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169497.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169497.log.gz 2023-11-06T22:19:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169842.log 2023-11-06T22:19:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142499.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142499.log.gz 2023-11-06T22:19:49.503 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131917.log 2023-11-06T22:19:49.504 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169842.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.169842.log.gz 2023-11-06T22:19:49.505 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127301.log 2023-11-06T22:19:49.506 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.131917.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131917.log.gz 2023-11-06T22:19:49.506 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169389.log 2023-11-06T22:19:49.507 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127301.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.127301.log.gz 2023-11-06T22:19:49.508 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174317.log 2023-11-06T22:19:49.509 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169389.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.169389.log.gz 2023-11-06T22:19:49.509 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124338.log 2023-11-06T22:19:49.511 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154179.log 2023-11-06T22:19:49.511 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.174317.log.gz 2023-11-06T22:19:49.512 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124338.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124338.log.gz 2023-11-06T22:19:49.513 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149975.log 2023-11-06T22:19:49.513 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154179.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154179.log.gz 2023-11-06T22:19:49.514 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129260.log 2023-11-06T22:19:49.515 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149975.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.149975.log.gz 2023-11-06T22:19:49.516 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174513.log 2023-11-06T22:19:49.517 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129260.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129260.log.gz 2023-11-06T22:19:49.517 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140948.log 2023-11-06T22:19:49.518 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174513.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174513.log.gz 2023-11-06T22:19:49.519 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148968.log 2023-11-06T22:19:49.520 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169093.log 2023-11-06T22:19:49.521 INFO:teuthology.orchestra.run.smithi033.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.140948.log.gz 2023-11-06T22:19:49.521 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148968.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148968.log.gz 2023-11-06T22:19:49.522 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162138.log 2023-11-06T22:19:49.523 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169093.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.169093.log.gz 2023-11-06T22:19:49.524 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177445.log 2023-11-06T22:19:49.524 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162138.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162138.log.gz 2023-11-06T22:19:49.525 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161340.log 2023-11-06T22:19:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177445.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.177445.log.gz 2023-11-06T22:19:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145351.log 2023-11-06T22:19:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161340.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.161340.log.gz 2023-11-06T22:19:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146783.log 2023-11-06T22:19:49.529 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145351.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145351.log.gz 2023-11-06T22:19:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177920.log 2023-11-06T22:19:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146783.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146783.log.gz 2023-11-06T22:19:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164177.log 2023-11-06T22:19:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177920.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177920.log.gz 2023-11-06T22:19:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160164.log 2023-11-06T22:19:49.534 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164177.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.164177.log.gz 2023-11-06T22:19:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153313.log 2023-11-06T22:19:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152915.log 2023-11-06T22:19:49.536 INFO:teuthology.orchestra.run.smithi033.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.160164.log.gz 2023-11-06T22:19:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153313.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.153313.log.gz 2023-11-06T22:19:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143798.log 2023-11-06T22:19:49.539 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152915.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152915.log.gz 2023-11-06T22:19:49.539 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164490.log 2023-11-06T22:19:49.540 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143798.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143798.log.gz 2023-11-06T22:19:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146399.log 2023-11-06T22:19:49.542 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164490.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164490.log.gz 2023-11-06T22:19:49.543 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134362.log 2023-11-06T22:19:49.544 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146399.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.146399.log.gz 2023-11-06T22:19:49.544 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143700.log 2023-11-06T22:19:49.545 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134362.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134362.log.gz 2023-11-06T22:19:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129064.log 2023-11-06T22:19:49.547 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143700.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143700.log.gz 2023-11-06T22:19:49.547 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120359.log 2023-11-06T22:19:49.549 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129064.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.129064.log.gz 2023-11-06T22:19:49.549 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145825.log 2023-11-06T22:19:49.550 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120359.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120359.log.gz 2023-11-06T22:19:49.551 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168905.log 2023-11-06T22:19:49.553 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145825.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145825.log.gz 2023-11-06T22:19:49.554 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125457.log 2023-11-06T22:19:49.554 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129839.log 2023-11-06T22:19:49.555 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.168905.log.gz 2023-11-06T22:19:49.555 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125457.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125457.log.gz 2023-11-06T22:19:49.556 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160880.log 2023-11-06T22:19:49.557 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154424.log 2023-11-06T22:19:49.557 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.129839.log.gz 2023-11-06T22:19:49.557 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160880.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.160880.log.gz 2023-11-06T22:19:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129211.log 2023-11-06T22:19:49.560 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154424.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-06T22:19:49.560 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154424.log.gz 2023-11-06T22:19:49.560 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129211.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129211.log.gz 2023-11-06T22:19:49.561 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-06T22:19:49.562 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141303.log 2023-11-06T22:19:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129358.log 2023-11-06T22:19:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141303.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141303.log.gz 2023-11-06T22:19:49.566 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129623.log 2023-11-06T22:19:49.567 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129358.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.129358.log.gz 2023-11-06T22:19:49.567 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153411.log 2023-11-06T22:19:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129623.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.129623.log.gz 2023-11-06T22:19:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144541.log 2023-11-06T22:19:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153411.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.153411.log.gz 2023-11-06T22:19:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144345.log 2023-11-06T22:19:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144541.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.144541.log.gz 2023-11-06T22:19:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.144345.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.164845.log 2023-11-06T22:19:49.574 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144345.log.gz 2023-11-06T22:19:49.575 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126230.log 2023-11-06T22:19:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164845.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164845.log.gz 2023-11-06T22:19:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123333.log 2023-11-06T22:19:49.578 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126230.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.126230.log.gz 2023-11-06T22:19:49.578 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163713.log 2023-11-06T22:19:49.580 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123333.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123333.log.gz 2023-11-06T22:19:49.580 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125704.log 2023-11-06T22:19:49.581 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163713.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.163713.log.gz 2023-11-06T22:19:49.582 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176049.log 2023-11-06T22:19:49.583 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125704.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125704.log.gz 2023-11-06T22:19:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152388.log 2023-11-06T22:19:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176049.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176049.log.gz 2023-11-06T22:19:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177641.log 2023-11-06T22:19:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152388.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152388.log.gz 2023-11-06T22:19:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144145.log 2023-11-06T22:19:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177641.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177641.log.gz 2023-11-06T22:19:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139233.log 2023-11-06T22:19:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144145.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.144145.log.gz 2023-11-06T22:19:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150462.log 2023-11-06T22:19:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139233.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139233.log.gz 2023-11-06T22:19:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150462.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.150462.log.gz 2023-11-06T22:19:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135878.log 2023-11-06T22:19:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119763.log 2023-11-06T22:19:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135878.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135878.log.gz 2023-11-06T22:19:49.599 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119763.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119763.log.gz 2023-11-06T22:19:49.599 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126483.log 2023-11-06T22:19:49.600 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168608.log 2023-11-06T22:19:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126483.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126483.log.gz 2023-11-06T22:19:49.603 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168608.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.168608.log.gz 2023-11-06T22:19:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164645.log 2023-11-06T22:19:49.605 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.175016.log 2023-11-06T22:19:49.607 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164645.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164645.log.gz 2023-11-06T22:19:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.175016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.175016.log.gz 2023-11-06T22:19:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163932.log 2023-11-06T22:19:49.610 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169240.log 2023-11-06T22:19:49.612 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163932.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163932.log.gz 2023-11-06T22:19:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145722.log 2023-11-06T22:19:49.614 INFO:teuthology.orchestra.run.smithi033.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.169240.log.gz 2023-11-06T22:19:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147861.log 2023-11-06T22:19:49.616 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145722.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145722.log.gz 2023-11-06T22:19:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147861.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147861.log.gz 2023-11-06T22:19:49.618 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-06T22:19:49.619 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164539.log 2023-11-06T22:19:49.622 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.164539.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.164539.log.gz 2023-11-06T22:19:49.623 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126532.log 2023-11-06T22:19:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126532.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.126532.log.gz 2023-11-06T22:19:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150413.log 2023-11-06T22:19:49.630 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150413.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150413.log.gz 2023-11-06T22:19:49.630 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145513.log 2023-11-06T22:19:49.632 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123725.log 2023-11-06T22:19:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145513.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145513.log.gz 2023-11-06T22:19:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153675.log 2023-11-06T22:19:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123725.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123725.log.gz 2023-11-06T22:19:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153675.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153675.log.gz 2023-11-06T22:19:49.639 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134902.log 2023-11-06T22:19:49.641 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134902.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.134902.log.gz 2023-11-06T22:19:49.642 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135139.log 2023-11-06T22:19:49.645 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135139.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.135139.log.gz 2023-11-06T22:19:49.646 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121633.log 2023-11-06T22:19:49.648 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175945.log 2023-11-06T22:19:49.649 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.121633.log.gz 2023-11-06T22:19:49.650 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172663.log 2023-11-06T22:19:49.653 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163285.log 2023-11-06T22:19:49.654 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172663.log: /var/log/ceph/ceph-client.admin.175945.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.172663.log.gz 2023-11-06T22:19:49.654 INFO:teuthology.orchestra.run.smithi033.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.175945.log.gz 2023-11-06T22:19:49.656 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119812.log 2023-11-06T22:19:49.658 INFO:teuthology.orchestra.run.smithi033.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163285.log.gz 2023-11-06T22:19:49.658 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172879.log 2023-11-06T22:19:49.661 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152550.log 2023-11-06T22:19:49.661 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172879.log: /var/log/ceph/ceph-client.admin.119812.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.172879.log.gz 2023-11-06T22:19:49.662 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119812.log.gz 2023-11-06T22:19:49.664 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152550.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152550.log.gz 2023-11-06T22:19:49.664 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132638.log 2023-11-06T22:19:49.666 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140135.log 2023-11-06T22:19:49.669 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.132638.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.132638.log.gz 2023-11-06T22:19:49.669 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143043.log 2023-11-06T22:19:49.670 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140135.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140135.log.gz 2023-11-06T22:19:49.671 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143043.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.143043.log.gz 2023-11-06T22:19:49.673 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168410.log 2023-11-06T22:19:49.675 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168410.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.168410.log.gz 2023-11-06T22:19:49.675 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143463.log 2023-11-06T22:19:49.678 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143463.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143463.log.gz 2023-11-06T22:19:49.679 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.170296.log 2023-11-06T22:19:49.682 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.170296.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.170296.log.gz 2023-11-06T22:19:49.682 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164588.log 2023-11-06T22:19:49.684 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164796.log 2023-11-06T22:19:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164588.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164588.log.gz 2023-11-06T22:19:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173207.log 2023-11-06T22:19:49.688 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164796.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164796.log.gz 2023-11-06T22:19:49.689 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173207.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.173207.log.gz 2023-11-06T22:19:49.690 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119567.log 2023-11-06T22:19:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119567.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119567.log.gz 2023-11-06T22:19:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172982.log 2023-11-06T22:19:49.695 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174170.log 2023-11-06T22:19:49.698 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172982.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.172982.log.gz 2023-11-06T22:19:49.698 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177592.log 2023-11-06T22:19:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174170.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174170.log.gz 2023-11-06T22:19:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175434.log 2023-11-06T22:19:49.701 INFO:teuthology.orchestra.run.smithi033.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.177592.log.gz 2023-11-06T22:19:49.702 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169695.log 2023-11-06T22:19:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175434.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.175434.log.gz 2023-11-06T22:19:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169695.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169695.log.gz 2023-11-06T22:19:49.707 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151310.log 2023-11-06T22:19:49.710 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.151310.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151310.log.gz 2023-11-06T22:19:49.711 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146587.log 2023-11-06T22:19:49.713 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146587.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.146587.log.gz 2023-11-06T22:19:49.714 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153487.log 2023-11-06T22:19:49.716 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159909.log 2023-11-06T22:19:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153487.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153487.log.gz 2023-11-06T22:19:49.720 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-06T22:19:49.720 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.159909.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159909.log.gz 2023-11-06T22:19:49.720 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164992.log 2023-11-06T22:19:49.723 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.164992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147983.log 2023-11-06T22:19:49.723 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.164992.log.gz 2023-11-06T22:19:49.728 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153362.log 2023-11-06T22:19:49.728 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147983.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.147983.log.gz 2023-11-06T22:19:49.729 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143191.log 2023-11-06T22:19:49.732 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143191.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143191.log.gz 2023-11-06T22:19:49.733 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153362.log.gz 2023-11-06T22:19:49.733 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146176.log 2023-11-06T22:19:49.734 INFO:teuthology.orchestra.run.smithi033.stderr: 89.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-06T22:19:49.737 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146176.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.146176.log.gz 2023-11-06T22:19:49.737 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134825.log 2023-11-06T22:19:49.739 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146934.log 2023-11-06T22:19:49.741 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134825.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.134825.log.gz 2023-11-06T22:19:49.741 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163233.log 2023-11-06T22:19:49.743 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164030.log 2023-11-06T22:19:49.744 INFO:teuthology.orchestra.run.smithi033.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.146934.log.gz 2023-11-06T22:19:49.745 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163233.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.163233.log.gz 2023-11-06T22:19:49.746 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149542.log 2023-11-06T22:19:49.747 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164030.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164030.log.gz 2023-11-06T22:19:49.748 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149542.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.149542.log.gz 2023-11-06T22:19:49.749 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148125.log 2023-11-06T22:19:49.752 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148125.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.148125.log.gz 2023-11-06T22:19:49.753 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126434.log 2023-11-06T22:19:49.755 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126434.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126434.log.gz 2023-11-06T22:19:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139806.log 2023-11-06T22:19:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139806.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.139806.log.gz 2023-11-06T22:19:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136486.log 2023-11-06T22:19:49.762 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.136486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129911.log 2023-11-06T22:19:49.763 INFO:teuthology.orchestra.run.smithi033.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.136486.log.gz 2023-11-06T22:19:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128050.log 2023-11-06T22:19:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129911.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.129911.log.gz 2023-11-06T22:19:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148916.log 2023-11-06T22:19:49.768 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.128050.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128050.log.gz 2023-11-06T22:19:49.769 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163608.log 2023-11-06T22:19:49.770 INFO:teuthology.orchestra.run.smithi033.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.148916.log.gz 2023-11-06T22:19:49.771 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170550.log 2023-11-06T22:19:49.774 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.163608.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163608.log.gz 2023-11-06T22:19:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.170550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139487.log 2023-11-06T22:19:49.775 INFO:teuthology.orchestra.run.smithi033.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.170550.log.gz 2023-11-06T22:19:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139487.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139487.log.gz 2023-11-06T22:19:49.777 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121780.log 2023-11-06T22:19:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121780.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.121780.log.gz 2023-11-06T22:19:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119392.log 2023-11-06T22:19:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119392.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.119392.log.gz 2023-11-06T22:19:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129162.log 2023-11-06T22:19:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130055.log 2023-11-06T22:19:49.788 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129162.log.gz 2023-11-06T22:19:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119196.log 2023-11-06T22:19:49.792 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.130055.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130055.log.gz 2023-11-06T22:19:49.792 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174068.log 2023-11-06T22:19:49.793 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119196.log.gz 2023-11-06T22:19:49.794 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174068.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.174068.log.gz 2023-11-06T22:19:49.795 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172611.log 2023-11-06T22:19:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172611.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.172611.log.gz 2023-11-06T22:19:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139858.log 2023-11-06T22:19:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139858.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.139858.log.gz 2023-11-06T22:19:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136413.log 2023-11-06T22:19:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.136413.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136413.log.gz 2023-11-06T22:19:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121094.log 2023-11-06T22:19:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136048.log 2023-11-06T22:19:49.809 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139547.log 2023-11-06T22:19:49.809 INFO:teuthology.orchestra.run.smithi033.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121094.log.gz 2023-11-06T22:19:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115397.log 2023-11-06T22:19:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.136048.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.136048.log.gz 2023-11-06T22:19:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139547.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.139547.log.gz 2023-11-06T22:19:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.115397.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.115397.log.gz 2023-11-06T22:19:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177739.log 2023-11-06T22:19:49.818 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177739.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.177739.log.gz 2023-11-06T22:19:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128001.log 2023-11-06T22:19:49.822 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.128001.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.128001.log.gz 2023-11-06T22:19:49.822 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143093.log 2023-11-06T22:19:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143093.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143093.log.gz 2023-11-06T22:19:49.826 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127445.log 2023-11-06T22:19:49.829 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127445.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127445.log.gz 2023-11-06T22:19:49.830 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125755.log 2023-11-06T22:19:49.833 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125755.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.125755.log.gz 2023-11-06T22:19:49.833 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167218.log 2023-11-06T22:19:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.167218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125934.log 2023-11-06T22:19:49.837 INFO:teuthology.orchestra.run.smithi033.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.167218.log.gz 2023-11-06T22:19:49.838 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169440.log 2023-11-06T22:19:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.154878.log 2023-11-06T22:19:49.840 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125934.log.gz 2023-11-06T22:19:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169440.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.169440.log.gz 2023-11-06T22:19:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.154878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173921.log 2023-11-06T22:19:49.843 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.154878.log.gz 2023-11-06T22:19:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173921.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.173921.log.gz 2023-11-06T22:19:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154130.log 2023-11-06T22:19:49.849 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154130.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154130.log.gz 2023-11-06T22:19:49.849 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127589.log 2023-11-06T22:19:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127589.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.127589.log.gz 2023-11-06T22:19:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146883.log 2023-11-06T22:19:49.856 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146883.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146883.log.gz 2023-11-06T22:19:49.856 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175996.log 2023-11-06T22:19:49.858 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162284.log 2023-11-06T22:19:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125604.log 2023-11-06T22:19:49.860 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.175996.log.gz 2023-11-06T22:19:49.862 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162284.log: /var/log/ceph/ceph-client.admin.125604.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162284.log.gz 2023-11-06T22:19:49.863 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125604.log.gz 2023-11-06T22:19:49.863 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124387.log 2023-11-06T22:19:49.866 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124387.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124387.log.gz 2023-11-06T22:19:49.866 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130525.log 2023-11-06T22:19:49.869 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.130525.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.130525.log.gz 2023-11-06T22:19:49.870 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121584.log 2023-11-06T22:19:49.873 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160327.log 2023-11-06T22:19:49.874 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121584.log.gz 2023-11-06T22:19:49.874 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169338.log 2023-11-06T22:19:49.876 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150264.log 2023-11-06T22:19:49.877 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.160327.log.gz 2023-11-06T22:19:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169338.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.169338.log.gz 2023-11-06T22:19:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120212.log 2023-11-06T22:19:49.881 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150264.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150264.log.gz 2023-11-06T22:19:49.882 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120212.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.120212.log.gz 2023-11-06T22:19:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140652.log 2023-11-06T22:19:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140652.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140652.log.gz 2023-11-06T22:19:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115054.log 2023-11-06T22:19:49.889 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.115054.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115054.log.gz 2023-11-06T22:19:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168779.log 2023-11-06T22:19:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142662.log 2023-11-06T22:19:49.895 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.168779.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.168779.log.gz 2023-11-06T22:19:49.896 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161585.log 2023-11-06T22:19:49.896 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142662.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.142662.log.gz 2023-11-06T22:19:49.897 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124926.log 2023-11-06T22:19:49.899 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161585.log.gz 2023-11-06T22:19:49.899 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148342.log 2023-11-06T22:19:49.900 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155132.log 2023-11-06T22:19:49.902 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124926.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.124926.log.gz 2023-11-06T22:19:49.905 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148342.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148342.log.gz 2023-11-06T22:19:49.905 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140999.log 2023-11-06T22:19:49.905 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.155132.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.155132.log.gz 2023-11-06T22:19:49.906 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140999.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.140999.log.gz 2023-11-06T22:19:49.907 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130873.log 2023-11-06T22:19:49.910 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.130873.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.130873.log.gz 2023-11-06T22:19:49.911 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134678.log 2023-11-06T22:19:49.913 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161027.log 2023-11-06T22:19:49.915 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134678.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134678.log.gz 2023-11-06T22:19:49.915 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143562.log 2023-11-06T22:19:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161027.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.161027.log.gz 2023-11-06T22:19:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139384.log 2023-11-06T22:19:49.918 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143562.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143562.log.gz 2023-11-06T22:19:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.139384.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139384.log.gz 2023-11-06T22:19:49.921 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161991.log 2023-11-06T22:19:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161991.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161991.log.gz 2023-11-06T22:19:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169891.log 2023-11-06T22:19:49.927 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169891.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169891.log.gz 2023-11-06T22:19:49.928 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125836.log 2023-11-06T22:19:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142767.log 2023-11-06T22:19:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125836.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125836.log.gz 2023-11-06T22:19:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172560.log 2023-11-06T22:19:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142767.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142767.log.gz 2023-11-06T22:19:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136561.log 2023-11-06T22:19:49.936 INFO:teuthology.orchestra.run.smithi033.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.172560.log.gz 2023-11-06T22:19:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164694.log 2023-11-06T22:19:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.136561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173257.log 2023-11-06T22:19:49.939 INFO:teuthology.orchestra.run.smithi033.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136561.log.gz 2023-11-06T22:19:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164694.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164694.log.gz 2023-11-06T22:19:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173257.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.173257.log.gz 2023-11-06T22:19:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145079.log 2023-11-06T22:19:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121924.log 2023-11-06T22:19:49.946 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.145079.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145079.log.gz 2023-11-06T22:19:49.948 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.121924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120898.log 2023-11-06T22:19:49.949 INFO:teuthology.orchestra.run.smithi033.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121924.log.gz 2023-11-06T22:19:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147399.log 2023-11-06T22:19:49.951 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120898.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120898.log.gz 2023-11-06T22:19:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147399.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.147399.log.gz 2023-11-06T22:19:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141056.log 2023-11-06T22:19:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141056.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141056.log.gz 2023-11-06T22:19:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165772.log 2023-11-06T22:19:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165772.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165772.log.gz 2023-11-06T22:19:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176955.log 2023-11-06T22:19:49.965 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176955.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.176955.log.gz 2023-11-06T22:19:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132266.log 2023-11-06T22:19:49.968 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.132266.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.132266.log.gz 2023-11-06T22:19:49.969 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164128.log 2023-11-06T22:19:49.971 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164128.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164128.log.gz 2023-11-06T22:19:49.972 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144245.log 2023-11-06T22:19:49.975 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144245.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.144245.log.gz 2023-11-06T22:19:49.975 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175123.log 2023-11-06T22:19:49.978 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175123.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175123.log.gz 2023-11-06T22:19:49.979 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174219.log 2023-11-06T22:19:49.982 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174219.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.174219.log.gz 2023-11-06T22:19:49.983 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140701.log 2023-11-06T22:19:49.985 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.140701.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140701.log.gz 2023-11-06T22:19:49.986 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133834.log 2023-11-06T22:19:49.989 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.133834.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133834.log.gz 2023-11-06T22:19:49.989 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153109.log 2023-11-06T22:19:49.992 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153109.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153109.log.gz 2023-11-06T22:19:49.993 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141401.log 2023-11-06T22:19:49.995 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141401.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.141401.log.gz 2023-11-06T22:19:49.996 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142372.log 2023-11-06T22:19:49.999 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142372.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142372.log.gz 2023-11-06T22:19:50.000 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173635.log 2023-11-06T22:19:50.003 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.173635.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.173635.log.gz 2023-11-06T22:19:50.003 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126181.log 2023-11-06T22:19:50.005 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128099.log 2023-11-06T22:19:50.008 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144194.log 2023-11-06T22:19:50.008 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.128099.log: /var/log/ceph/ceph-client.admin.126181.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128099.log.gz 2023-11-06T22:19:50.009 INFO:teuthology.orchestra.run.smithi033.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.126181.log.gz 2023-11-06T22:19:50.009 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135780.log 2023-11-06T22:19:50.013 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144194.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144194.log.gz 2023-11-06T22:19:50.013 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155271.log 2023-11-06T22:19:50.013 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135780.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135780.log.gz 2023-11-06T22:19:50.015 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.155271.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.155271.log.gz 2023-11-06T22:19:50.015 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164943.log 2023-11-06T22:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164943.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164943.log.gz 2023-11-06T22:19:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118792.log 2023-11-06T22:19:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.118792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175578.log 2023-11-06T22:19:50.023 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118792.log.gz 2023-11-06T22:19:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120702.log 2023-11-06T22:19:50.027 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164894.log 2023-11-06T22:19:50.027 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120702.log: /var/log/ceph/ceph-client.admin.175578.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120702.log.gz 2023-11-06T22:19:50.028 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175578.log.gz 2023-11-06T22:19:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161076.log 2023-11-06T22:19:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.164894.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.164894.log.gz 2023-11-06T22:19:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.161076.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161076.log.gz 2023-11-06T22:19:50.032 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170641.log 2023-11-06T22:19:50.035 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.170641.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.170641.log.gz 2023-11-06T22:19:50.035 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124534.log 2023-11-06T22:19:50.038 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124534.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124534.log.gz 2023-11-06T22:19:50.039 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124583.log 2023-11-06T22:19:50.042 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124583.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124583.log.gz 2023-11-06T22:19:50.042 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127661.log 2023-11-06T22:19:50.044 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123480.log 2023-11-06T22:19:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.127661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125261.log 2023-11-06T22:19:50.046 INFO:teuthology.orchestra.run.smithi033.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.127661.log.gz 2023-11-06T22:19:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165090.log 2023-11-06T22:19:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123480.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123480.log.gz 2023-11-06T22:19:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125261.log: /var/log/ceph/ceph-client.admin.165090.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125261.log.gz 2023-11-06T22:19:50.051 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.165090.log.gz 2023-11-06T22:19:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141548.log 2023-11-06T22:19:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141548.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.141548.log.gz 2023-11-06T22:19:50.055 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152237.log 2023-11-06T22:19:50.056 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148760.log 2023-11-06T22:19:50.059 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178089.log 2023-11-06T22:19:50.059 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152237.log.gz 2023-11-06T22:19:50.060 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160432.log 2023-11-06T22:19:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.148760.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.148760.log.gz 2023-11-06T22:19:50.064 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.178089.log: /var/log/ceph/ceph-client.admin.160432.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.178089.log.gz 87.3% -- replaced with /var/log/ceph/ceph-client.admin.160432.log.gz 2023-11-06T22:19:50.064 INFO:teuthology.orchestra.run.smithi033.stderr: 2023-11-06T22:19:50.064 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162722.log 2023-11-06T22:19:50.067 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162722.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162722.log.gz 2023-11-06T22:19:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125163.log 2023-11-06T22:19:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125163.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125163.log.gz 2023-11-06T22:19:50.071 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147277.log 2023-11-06T22:19:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143749.log 2023-11-06T22:19:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.147277.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147277.log.gz 2023-11-06T22:19:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-06T22:19:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.143749.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.143749.log.gz 2023-11-06T22:19:50.079 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123627.log 2023-11-06T22:19:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169142.log 2023-11-06T22:19:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169142.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169142.log.gz 2023-11-06T22:19:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125359.log 2023-11-06T22:19:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123627.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123627.log.gz 2023-11-06T22:19:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.125359.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125359.log.gz 2023-11-06T22:19:50.090 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124485.log 2023-11-06T22:19:50.092 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162911.log 2023-11-06T22:19:50.095 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.124485.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123137.log 2023-11-06T22:19:50.096 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.124485.log.gz 2023-11-06T22:19:50.098 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167681.log 2023-11-06T22:19:50.100 INFO:teuthology.orchestra.run.smithi033.stderr: 87.3%/var/log/ceph/ceph-client.admin.123137.log: -- replaced with /var/log/ceph/ceph-client.admin.162911.log.gz 2023-11-06T22:19:50.101 INFO:teuthology.orchestra.run.smithi033.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123137.log.gz 2023-11-06T22:19:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119516.log 2023-11-06T22:19:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.167681.log: 88.5%/var/log/ceph/ceph-client.admin.119516.log: -- replaced with /var/log/ceph/ceph-client.admin.167681.log.gz 2023-11-06T22:19:50.108 INFO:teuthology.orchestra.run.smithi033.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119516.log.gz 2023-11-06T22:19:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129309.log 2023-11-06T22:19:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.108500.log 2023-11-06T22:19:50.113 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.129309.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140464.log 2023-11-06T22:19:50.114 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.129309.log.gz 2023-11-06T22:19:50.117 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.tmp-client.admin.108500.log: /var/log/ceph/ceph-client.admin.140464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152602.log 2023-11-06T22:19:50.118 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.108500.log.gz 2023-11-06T22:19:50.118 INFO:teuthology.orchestra.run.smithi033.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.140464.log.gz 2023-11-06T22:19:50.120 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152491.log 2023-11-06T22:19:50.124 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.152602.log: /var/log/ceph/ceph-client.admin.152491.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152602.log.gz 2023-11-06T22:19:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123284.log 2023-11-06T22:19:50.127 INFO:teuthology.orchestra.run.smithi033.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.152491.log.gz 2023-11-06T22:19:50.127 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176258.log 2023-11-06T22:19:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123284.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.123284.log.gz 2023-11-06T22:19:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176258.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176258.log.gz 2023-11-06T22:19:50.134 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122484.log 2023-11-06T22:19:50.136 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150560.log 2023-11-06T22:19:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.122484.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122484.log.gz 2023-11-06T22:19:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.150560.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150560.log.gz 2023-11-06T22:19:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162600.log 2023-11-06T22:19:50.142 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162600.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.162600.log.gz 2023-11-06T22:19:50.143 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154079.log 2023-11-06T22:19:50.146 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172775.log 2023-11-06T22:19:50.149 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120163.log 2023-11-06T22:19:50.149 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154079.log.gz 2023-11-06T22:19:50.150 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.172775.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.172775.log.gz 2023-11-06T22:19:50.151 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176101.log 2023-11-06T22:19:50.153 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120163.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120163.log.gz 2023-11-06T22:19:50.154 INFO:teuthology.orchestra.run.smithi069.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-06T22:19:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.176101.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176101.log.gz 2023-11-06T22:19:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144047.log 2023-11-06T22:19:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.144047.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144047.log.gz 2023-11-06T22:19:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126580.log 2023-11-06T22:19:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126580.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126580.log.gz 2023-11-06T22:19:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153158.log 2023-11-06T22:19:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145402.log 2023-11-06T22:19:50.168 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.153158.log: /var/log/ceph/ceph-client.admin.145402.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.145402.log.gz 2023-11-06T22:19:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134256.log 2023-11-06T22:19:50.173 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134256.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.134256.log.gz 2023-11-06T22:19:50.174 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153158.log.gz 2023-11-06T22:19:50.174 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160663.log 2023-11-06T22:19:50.176 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160663.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.160663.log.gz 2023-11-06T22:19:50.177 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142611.log 2023-11-06T22:19:50.179 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166717.log 2023-11-06T22:19:50.182 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.166717.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.166717.log.gz 2023-11-06T22:19:50.184 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.142611.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142611.log.gz 2023-11-06T22:19:50.184 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135237.log 2023-11-06T22:19:50.187 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135237.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135237.log.gz 2023-11-06T22:19:50.187 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134308.log 2023-11-06T22:19:50.190 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.134308.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134308.log.gz 2023-11-06T22:19:50.191 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119465.log 2023-11-06T22:19:50.194 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119465.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119465.log.gz 2023-11-06T22:19:50.195 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175892.log 2023-11-06T22:19:50.197 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.175892.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.175892.log.gz 2023-11-06T22:19:50.198 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165041.log 2023-11-06T22:19:50.201 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.165041.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165041.log.gz 2023-11-06T22:19:50.202 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162187.log 2023-11-06T22:19:50.204 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.162187.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162187.log.gz 2023-11-06T22:19:50.205 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120996.log 2023-11-06T22:19:50.207 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160978.log 2023-11-06T22:19:50.210 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.160978.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160978.log.gz 2023-11-06T22:19:50.211 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.120996.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120996.log.gz 2023-11-06T22:19:50.212 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146274.log 2023-11-06T22:19:50.215 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146274.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146274.log.gz 2023-11-06T22:19:50.215 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123774.log 2023-11-06T22:19:50.218 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123774.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123774.log.gz 2023-11-06T22:19:50.219 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146323.log 2023-11-06T22:19:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.146323.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.146323.log.gz 2023-11-06T22:19:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154228.log 2023-11-06T22:19:50.225 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154228.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.154228.log.gz 2023-11-06T22:19:50.226 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167169.log 2023-11-06T22:19:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.167169.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.167169.log.gz 2023-11-06T22:19:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169940.log 2023-11-06T22:19:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.169940.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169940.log.gz 2023-11-06T22:19:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177543.log 2023-11-06T22:19:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.177543.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.177543.log.gz 2023-11-06T22:19:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174119.log 2023-11-06T22:19:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.174119.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.174119.log.gz 2023-11-06T22:19:50.240 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126132.log 2023-11-06T22:19:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.126132.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.126132.log.gz 2023-11-06T22:19:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119009.log 2023-11-06T22:19:50.246 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.119009.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.119009.log.gz 2023-11-06T22:19:50.247 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154375.log 2023-11-06T22:19:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.154375.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.154375.log.gz 2023-11-06T22:19:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135535.log 2023-11-06T22:19:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.135535.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135535.log.gz 2023-11-06T22:19:50.254 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141912.log 2023-11-06T22:19:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.141912.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141912.log.gz 2023-11-06T22:19:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170475.log 2023-11-06T22:19:50.259 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123037.log 2023-11-06T22:19:50.261 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.170475.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.170475.log.gz 2023-11-06T22:19:50.263 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.123037.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123037.log.gz 2023-11-06T22:19:50.264 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149319.log 2023-11-06T22:19:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.149319.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149319.log.gz 2023-11-06T22:19:50.417 INFO:teuthology.orchestra.run.smithi033.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-06T22:19:50.557 INFO:teuthology.orchestra.run.smithi033.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-06T22:19:50.748 INFO:teuthology.orchestra.run.smithi069.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-06T22:19:51.081 INFO:teuthology.orchestra.run.smithi069.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-06T22:19:51.416 INFO:teuthology.orchestra.run.smithi069.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-06T22:19:51.466 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-06T22:19:51.617 INFO:teuthology.orchestra.run.smithi069.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-06T22:19:51.911 INFO:teuthology.orchestra.run.smithi069.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-06T22:19:51.961 INFO:teuthology.orchestra.run.smithi069.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-06T22:19:51.964 INFO:teuthology.orchestra.run.smithi069.stderr: 2023-11-06T22:19:51.965 INFO:teuthology.orchestra.run.smithi069.stderr:real 0m3.108s 2023-11-06T22:19:51.965 INFO:teuthology.orchestra.run.smithi069.stderr:user 0m15.716s 2023-11-06T22:19:51.965 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m1.340s 2023-11-06T22:19:51.966 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-06T22:19:52.037 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-06T22:19:52.797 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-06T22:19:53.248 INFO:teuthology.orchestra.run.smithi033.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-06T22:19:53.252 INFO:teuthology.orchestra.run.smithi033.stderr: 2023-11-06T22:19:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m4.399s 2023-11-06T22:19:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:user 0m15.412s 2023-11-06T22:19:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m3.172s 2023-11-06T22:19:53.254 INFO:tasks.ceph:Archiving logs... 2023-11-06T22:19:53.255 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362/remote/smithi033/log 2023-11-06T22:19:53.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-06T22:19:56.021 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362/remote/smithi069/log 2023-11-06T22:19:56.022 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-06T22:19:57.991 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-06T22:19:58.001 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-06T22:19:58.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-06T22:19:58.008 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-06T22:19:58.137 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-06T22:19:58.147 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-06T22:19:58.147 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T22:19:58.152 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T22:19:58.195 INFO:teuthology.orchestra.run.smithi033.stderr:bash: ntpq: command not found 2023-11-06T22:19:58.196 INFO:teuthology.orchestra.run.smithi069.stderr:bash: ntpq: command not found 2023-11-06T22:19:58.204 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-06T22:19:58.204 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2023-11-06T22:19:58.204 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 43 +151us[ +166us] +/- 79ms 2023-11-06T22:19:58.204 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -1896us[-1881us] +/- 66ms 2023-11-06T22:19:58.205 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +2120us[+2120us] +/- 73ms 2023-11-06T22:19:58.205 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T22:19:58.206 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-06T22:19:58.207 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2023-11-06T22:19:58.207 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 108 +168us[ +185us] +/- 78ms 2023-11-06T22:19:58.207 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -1843us[-1829us] +/- 66ms 2023-11-06T22:19:58.207 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +2178us[+2192us] +/- 72ms 2023-11-06T22:19:58.208 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-11-06T22:19:58.208 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-06T22:19:58.217 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-06T22:19:58.218 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-06T22:19:58.228 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-06T22:19:58.339 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-06T22:19:58.507 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-06T22:19:58.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T22:19:58.581 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-06T22:19:58.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T22:19:58.659 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-06T22:19:58.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-06T22:19:58.681 INFO:teuthology.task.internal:Duration was 6651.429072 seconds 2023-11-06T22:19:58.682 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-06T22:19:58.693 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-06T22:19:58.693 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-06T22:19:58.698 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-06T22:19:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-06T22:19:58.815 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-06T22:19:59.069 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-06T22:19:59.069 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2023-11-06T22:19:59.070 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-06T22:19:59.134 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2023-11-06T22:19:59.135 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-06T22:19:59.197 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-06T22:19:59.198 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T22:19:59.205 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T22:19:59.293 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-06T22:19:59.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-06T22:19:59.298 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-06T22:19:59.435 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-06T22:19:59.446 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-06T22:19:59.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-06T22:19:59.519 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-06T22:19:59.589 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-06T22:19:59.599 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-06T22:19:59.605 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-06T22:19:59.671 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2023-11-06T22:19:59.674 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2023-11-06T22:19:59.708 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-06T22:19:59.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T22:19:59.754 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-06T22:19:59.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T22:19:59.800 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-06T22:19:59.810 INFO:teuthology.task.internal:Transferring archived files... 2023-11-06T22:19:59.811 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362/remote/smithi033 2023-11-06T22:19:59.812 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-06T22:19:59.890 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362/remote/smithi069 2023-11-06T22:19:59.891 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-06T22:19:59.972 INFO:teuthology.task.internal:Removing archive directory... 2023-11-06T22:19:59.972 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-06T22:19:59.977 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-06T22:20:00.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-06T22:20:00.039 INFO:teuthology.task.internal:Not uploading archives. 2023-11-06T22:20:00.039 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-06T22:20:00.061 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-06T22:20:00.062 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-06T22:20:00.067 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-06T22:20:00.109 INFO:teuthology.orchestra.run.smithi033.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 6 22:20 /home/ubuntu/cephtest 2023-11-06T22:20:00.114 INFO:teuthology.orchestra.run.smithi069.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 6 22:20 /home/ubuntu/cephtest 2023-11-06T22:20:00.118 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-06T22:20:00.132 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-06T22:20:00.173 INFO:teuthology.nuke:Checking targets against current locks 2023-11-06T22:20:00.200 DEBUG:teuthology.nuke:shortname: smithi033 2023-11-06T22:20:00.201 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T22:20:00.220 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 20:19:39.592479', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBhYb7C6mh7TStE9qnElUtlktEmsqwCqHWi6AuvtIjAc5I+LEPtMn5m6Tc5YouBmPFlV8RWjsDjd/cj8RwnXvFw='} 2023-11-06T22:20:00.226 DEBUG:teuthology.nuke:shortname: smithi069 2023-11-06T22:20:00.226 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T22:20:00.253 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-06_07:50:57-fs-wip-vshankar-testing1-2023-11-02-1726-pacific-testing-default-smithi/7449362', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 20:19:39.590569', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUKXzZAO/PeanAIVEqBgMAF/dV5YdwYatA66k8kMbvkXopSNomPc8YE8Wu/L9oujrpYY+2GBQZ6hTHfS4+cXPQ='} 2023-11-06T22:20:00.263 INFO:teuthology.orchestra.console.smithi033:Power off 2023-11-06T22:20:00.264 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-06T22:20:00.300 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2023-11-06T22:20:00.300 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T22:20:00.317 INFO:teuthology.orchestra.console.smithi069:Power off 2023-11-06T22:20:00.317 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-06T22:20:00.328 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2023-11-06T22:20:00.349 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2023-11-06T22:20:00.349 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T22:20:00.375 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-11-06T22:20:04.330 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T22:20:04.377 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T22:20:04.441 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2023-11-06T22:20:04.489 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-11-06T22:20:08.446 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T22:20:08.489 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T22:20:08.556 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2023-11-06T22:20:08.601 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2023-11-06T22:20:08.701 INFO:teuthology.orchestra.console.smithi069:Power off completed 2023-11-06T22:20:12.558 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T22:20:12.669 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2023-11-06T22:20:12.770 INFO:teuthology.orchestra.console.smithi033:Power off completed 2023-11-06T22:20:12.874 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 client.1-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 description: fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6651.429072141647 failure_reason: '"2023-11-06T20:46:26.917225+0000 mon.a (mon.0) 322 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2023-11-06T22:20:12.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-06T22:20:12.958 INFO:teuthology.run:FAIL